Foundations
What kind of system are we building?
The architectural ideas the rest of the stack assumes: what a lakehouse is, and how it differs from a lake or a warehouse.
17 reference entries
Bronze LayerIn the Medallion Architecture that structures modern data lakehouses, data does not simply arrive and immediately become analytically ready.Data FabricModern enterprises rarely store their data in a single place.Data GravityData Gravity is an analogy coined by Dave McCrory in 2010 to describe the phenomenon by which large concentrations of data attract applications, services.Data LakeThe explosion of digital information over the last decade created a massive storage problem.Data LakehouseA comprehensive definition of Data Lakehouse architecture, combining data warehouse reliability with data lake scalability via open table formats.Data MeshFor most of the 2010s, the standard blueprint for a modern data platform involved building a centralized data lake, staffing a central data engineering team.Data SwampA comprehensive guide to what causes a Data Lake to become a Data Swamp, how to recognize the warning signs, and the governance practices that prevent it.Data WarehouseA data warehouse is a centralized repository engineered specifically to store highly structured, historical data.Gold LayerAt the top of the Medallion Architecture sits the Gold Layer: the final destination for data that has been ingested, cleaned, validated, enriched, and now.Kappa ArchitectureUnderstanding Kappa Architecture, the simplified alternative to Lambda that treats everything as a stream.Lambda ArchitectureLambda Architecture is a data deployment model introduced by Nathan Marz designed to handle massive quantities of data by taking advantage of both batch and.Medallion ArchitectureWhen organizations first started building data lakehouses, they faced a structuring problem.Open LakehouseAn open lakehouse is a data architecture that stores data in open file and table formats on commodity object storage, so that any compliant engine can read and write it without vendor lock-in.Polyglot PersistenceFor a long time, the default answer to any data storage question in enterprise software development was a relational database.Separation of Compute and StorageSeparation of Compute and Storage is an architectural principle in which the processing layer (compute engines that execute queries and transformations) and.Silver LayerIn the Medallion Architecture, data quality is not enforced at the point of ingestion.Zero-ETLData pipelines are expensive to build and expensive to maintain.