The Book on Using Apache Iceberg with Python
Published by Self-published · Tech
A hands-on guide to working with Apache Iceberg using Python tools like PyIceberg, PySpark, and other Python-based engines.
Who this book is for
Data engineers, data architects, and Python developers who want to master the Apache Iceberg ecosystem using Python.
Table of contents
- The Basics of Apache Iceberg
- PyIceberg
- PySpark
- PyFlink
- DataFusion
- Iceframe
- Iceberg-cli
- Dremio - The Agentic Lakehouse Platform
- Bauplan - The Programmable Lakehouse
- DuckDB - The Engine in Your Pocket
- Daft - The Multimodal DataFrame
- SpiceAI - The Edge-Accelerated Lakehouse
Excerpt
The shift from managing files to managing tables is the single most important evolution in modern data architecture. For years, data engineers built "data lakes" by dumping thousands of Parquet, CSV, or JSON files into object storage buckets. They relied on directory structures and Hive metastores to make sense of this data. This approach worked until it didn't. As data volumes grew, listing files became slow, schema changes became risky, and data consistency became a nightmare. If a job failed halfway through writing a thousand files, you were left with a corrupted dataset and no easy way to roll back.
