Questions people actually ask

A data architecture that stores data in open file and table formats on commodity object storage, so any compliant engine can read and write it without vendor lock-in.

Both, and the distinction is worth holding onto. The underlying technical change is real and specific: adding a metadata layer over files on object storage gives them ACID transactions. The word itself gets applied to almost anything, so it is more useful to ask which table format, which catalog, and which engines are involved.

No, and most organisations do not. The common path is to move the largest and least structured workloads onto lakehouse tables first, keep the warehouse for what it already serves well, and let the boundary move over time. Warehouses increasingly read open table formats directly, which makes coexistence easier than it was.

Iceberg if several engines from different vendors need the same tables, Delta if your platform centres on Databricks, Hudi if your workload is dominated by upserts, Paimon if you are streaming-first on Flink. The comparison page works through the trade-offs in detail.

More easily than you might expect. Apache XTable and Delta UniForm let one physical dataset present as more than one format, which turns what used to be a migration into a compatibility layer. It is still work, but it is no longer a one-way door.

A file format such as Parquet describes how one file encodes its rows and columns. A table format such as Iceberg describes which files make up a table right now, what the schema is, and how concurrent writers agree on changes. Files are the bytes; the table format is the bookkeeping over them.

Because query cost on object storage is dominated by request count and metadata overhead rather than raw bytes. Thousands of tiny files force an engine into thousands of round trips. Compaction rewrites them into fewer, larger files, and neglecting it is the most common reason a lakehouse feels slow.

Once a second engine appears, yes. The table format defines what a table is; the catalog tracks which tables exist, where the current metadata pointer is, and who may access them. Without one, engines cannot reliably agree on a table's current state.

It lets the catalog hand an engine short-lived, narrowly scoped storage credentials for exactly the data it is allowed to touch. The alternative is giving every engine long-lived storage keys, which means your real access control is in the storage layer rather than the catalog, and is nearly impossible to audit.

You can, and many organisations end up there by accident. It is worth avoiding deliberately, because it splits the authority on who can access what. Catalog federation is the current answer to estates that already have several.

Four recurring jobs: compacting small files, expiring old snapshots, removing orphaned files, and rewriting manifests when metadata grows. None is difficult, all are easy to postpone, and postponing them is what makes tables slow and storage bills grow.

Storage usually is, by a wide margin, because it is ordinary object storage. Compute depends entirely on how you run it. What reliably changes is the shape of the bill: you pay for storage continuously and for compute only when engines run, rather than paying one vendor for both together.

The constraint is rarely technical. Each additional engine adds a dialect, a set of connector quirks, and another place where permissions must be right. Two or three with clearly separate jobs is common and manageable.

Agents need the same things analysts need, more strictly: data they can discover, meaning they can trust, and permissions that hold. A governed catalog and a semantic layer give an agent a way to find the right table and understand what its columns mean, rather than guessing from column names.

It can, and it will often produce plausible, wrong answers. Raw tables carry no information about which of four revenue columns is the governed one. That is what the semantic layer is for, and it is why agentic analytics tends to expose curated models rather than raw schemas.

No. They solve different problems. Retrieval augmented generation is good at finding relevant text; a lakehouse is where governed structured data lives. Questions about numbers should be answered by querying the numbers, not by retrieving documents that mention them.