Blogged: Interoperability between lakehouse table formats with Apache XTable (Incubating). Scenario: Dremio -> Apache Hudi ❌ Dremio -> Apache Iceberg ✅ There are 2 analytics teams in an organization. Team A uses Hudi to manage some of their most critical, low-latency data pipelines. Team B focuses on ad hoc analysis, BI, and reporting, using Dremio's compute engine with Iceberg. ❌ Challenge: Team B wants to do analytics using data from the 2 different datasets within the organization. To do so, Team B has to use the dataset generated by Team A (stored as a Hudi table) and combine it with their dataset (the Iceberg table) ✅ Solution: Interoperability between the formats with Apache XTable. Detailed reading link in comments. #dataengineering #softwareengineering
Join me on May 2nd where I will be talking about all of this work. And always a pleasure to work together with Alex Merced 🙌🏻 https://www.dremio.com/subsurface/sessions/
Staff Data Engineer Advocate @Onehouse.ai | Apache Hudi, Iceberg Contributor | Author of "Engineering Lakehouses"
7moBlog; https://www.onehouse.ai/blog/dremio-lakehouse-analytics-with-hudi-and-iceberg-using-xtable