Vendor-neutral reference
The open
lakehouse
A data architecture that stores data in open file and table formats on commodity object storage, so that any compliant engine can read and write it without vendor lock-in.
- Reference entries
- 201
- Architecture layers
- 9
- Comparisons
- 3
The stack, one layer at a time
An open lakehouse is assembled from layers, each with one job and several implementations. Any one of them can be replaced without disturbing the others, which is the entire point of the architecture.
- Agents & AI 18 entries How do agents use any of this safely? Agentic analyticsRAGText-to-SQLVector search
- Semantics 12 entries What does this data mean? Semantic layerMetricsOntologyModeling
- Movement 15 entries How does data get in, and stay current? AirflowdbtDagsterCDCStreaming
- Interchange 3 entries How does data move between systems? Apache ArrowArrow FlightFlight SQL
- Compute 43 entries What actually runs the query? SparkTrinoFlinkDuckDBDremio
- Catalog 17 entries How do engines find and govern those tables? Apache PolarisIceberg RESTNessieGlueUnity
- Table format 50 entries What makes those files behave like a table? Apache IcebergDelta LakeApache HudiApache Paimon
- Storage format 26 entries How is the data physically written down? Apache ParquetORCAvroObject storage
- Foundations 17 entries What kind of system are we building? LakehouseData lakeData warehouseMedallion
Start here
Four pages that cover the whole subject, in the order they make sense.
The choices, side by side
Each layer has several viable options. These comparisons lead with the table.
Look something up
201 in-depth entries, each mapped to the layer it belongs to, plus a one-line glossary for when that is all you need.
Implementation reading
Written by vendors, listed here because the technical content is useful. Nothing on this site is a recommendation to buy anything.
- Semantic layer: a definitive guide Dremio
- Apache Polaris as a catalog standard Dremio
- What are table formats, and why were they needed? Dremio
- Open source and the data lakehouse Dremio
More reading, including independent sources, is collected in the blog roll, video roll, and book roll.