Get Kafka-Nated Episode 4
Join Hugh Evans and Alex Merced from Dremio as they explore the future of streaming data with Apache Iceberg. In this episode, we dive into how Iceberg is transforming the way data lakes work, making it easier for engineers to manage, query, and stream data efficiently.
Alex shares his journey at Dremio and explains:
🔷 What Apache Iceberg is and why it matters for Kafka users and data engineers.
🔷 The benefits and patterns of streaming data directly into Iceberg tables.
🔷 How Iceberg handles schema evolution in real-time streaming scenarios.
🔷 The evolving ecosystem of lakehouse architectures and how Iceberg fits alongside concepts like diskless Kafka.
Whether you’re an engineer, architect, or data enthusiast, this episode will give you a deep understanding of modern streaming architectures and practical insights into leveraging Iceberg for your projects.
Timestamp:
0:00 – Intro
0:08 – What is Apache Iceberg?
1:08 – Solving the Data Lake Problem
3:38 – Streaming Kafka to Iceberg
7:02 – Compaction Strategies Explained
11:15 – Handling Schema Evolution
13:13 – Time Travel in Iceberg
17:09 – Iceberg vs Delta Lake & Hoodie
23:11 – Catalogs & Ecosystem Adoption
31:26 – Resources & Closing Thoughts
Learn more about Aiven for Apache Kafka: https://aiven.io/kafka
Learn more about Aiven Inkless: https://aiven.io/inkless
Read the blog: Getting Started with Icebergs Topics for Apache Kafka: https://aiven.io/blog/getting-started-with-iceberg-topics-for-apache-kafkar-a-beginners-guide
Watch on-demand Building Your Streaming LakeHouse with Kafka and Iceberg: https://aiven.io/workshop/unlocking-real-time-insights
Watch more episodes of Get Kafka-Nated on YouTube: https://www.youtube.com/playlist
#apachekafka #IcebergTopics #datastreaming #aiven #Dremio #GetKafkaNated
Connect With Us
Website: http://aiven.io
LinkedIn: https://www.linkedin.com/company/aiven/
GitHub: https://github.com/aiven
X: https://twitter.com/aiven_io