Vendor-neutral reference

The open
lakehouse

A data architecture that stores data in open file and table formats on commodity object storage, so that any compliant engine can read and write it without vendor lock-in.

Reference entries
201
Architecture layers
9
Comparisons
3
Spark Trino Flink DuckDB SAP/Dremio Catalog Table format Data file Data file Data file Data file Object storage you control

The stack, one layer at a time

An open lakehouse is assembled from layers, each with one job and several implementations. Any one of them can be replaced without disturbing the others, which is the entire point of the architecture.

  1. Agents & AI 18 entries How do agents use any of this safely? Agentic analyticsRAGText-to-SQLVector search
  2. Semantics 12 entries What does this data mean? Semantic layerMetricsOntologyModeling
  3. Movement 15 entries How does data get in, and stay current? AirflowdbtDagsterCDCStreaming
  4. Interchange 3 entries How does data move between systems? Apache ArrowArrow FlightFlight SQL
  5. Compute 43 entries What actually runs the query? SparkTrinoFlinkDuckDBDremio
  6. Catalog 17 entries How do engines find and govern those tables? Apache PolarisIceberg RESTNessieGlueUnity
  7. Table format 50 entries What makes those files behave like a table? Apache IcebergDelta LakeApache HudiApache Paimon
  8. Storage format 26 entries How is the data physically written down? Apache ParquetORCAvroObject storage
  9. Foundations 17 entries What kind of system are we building? LakehouseData lakeData warehouseMedallion

Start here

Four pages that cover the whole subject, in the order they make sense.

The choices, side by side

Each layer has several viable options. These comparisons lead with the table.

All comparisons

Look something up

201 in-depth entries, each mapped to the layer it belongs to, plus a one-line glossary for when that is all you need.

Browse the library

Implementation reading

Written by vendors, listed here because the technical content is useful. Nothing on this site is a recommendation to buy anything.

More reading, including independent sources, is collected in the blog roll, video roll, and book roll.