Comparing lakehouse catalogs
Which catalog should govern the tables?
The catalog choice is mostly a governance decision: all of them can point engines at tables, but they differ sharply in how they handle permissions, credential vending, and whether their API is open enough to swap later.
| Open API | Access control | Credential vending | Best suited to | |
|---|---|---|---|---|
| Apache Polaris | Iceberg REST, open source | Role based, fine grained | Yes | Multi-engine estates wanting an open governed catalog |
| Project Nessie | Iceberg REST plus its own API | Basic | Limited | Teams that want Git-style branching over data |
| AWS Glue Data Catalog | Proprietary, with an Iceberg REST endpoint | IAM and Lake Formation | Via Lake Formation | Estates already committed to AWS |
| Unity Catalog | Open sourced, Databricks led | Rich, unified across assets | Yes | Databricks-centred platforms |
| Hive Metastore | Thrift, ubiquitous but dated | Minimal | No | Legacy compatibility rather than new builds |
| Lakekeeper | Iceberg REST, open source | Role based | Yes | Lightweight self-hosted deployments |
Choosing
- Several engines from different vendors need the same tablesAn Iceberg REST catalog such as Polaris or Lakekeeper
- Everything runs inside one cloud providerThat provider's managed catalog, accepting the coupling
- You need experimentation branches over production dataNessie, for its branching model
- You are migrating off HiveAn Iceberg REST catalog, and plan the migration as its own project
Caveats
- Credential vending is the feature that most often decides this in practice. Without it, every engine needs its own long-lived storage credentials, which undoes much of the governance benefit.
- The Iceberg REST specification is what makes catalogs swappable. A catalog that implements it can usually be replaced; one that does not, cannot.