Foundations
What kind of system are we building?
The architectural ideas the rest of the stack assumes: what a lakehouse is, and how it differs from a lake or a warehouse.
17 reference entries
Bronze Layer In the Medallion Architecture that structures modern data lakehouses, data does not simply arrive and immediately become analytically ready. Data Fabric Modern enterprises rarely store their data in a single place. Data Gravity Data Gravity is an analogy coined by Dave McCrory in 2010 to describe the phenomenon by which large concentrations of data attract applications, services. Data Lake The explosion of digital information over the last decade created a massive storage problem. Data Lakehouse A comprehensive definition of Data Lakehouse architecture, combining data warehouse reliability with data lake scalability via open table formats. Data Mesh For most of the 2010s, the standard blueprint for a modern data platform involved building a centralized data lake, staffing a central data engineering team. Data Swamp A comprehensive guide to what causes a Data Lake to become a Data Swamp, how to recognize the warning signs, and the governance practices that prevent it. Data Warehouse A data warehouse is a centralized repository engineered specifically to store highly structured, historical data. Gold Layer At the top of the Medallion Architecture sits the Gold Layer: the final destination for data that has been ingested, cleaned, validated, enriched, and now. Kappa Architecture Understanding Kappa Architecture, the simplified alternative to Lambda that treats everything as a stream. Lambda Architecture Lambda Architecture is a data deployment model introduced by Nathan Marz designed to handle massive quantities of data by taking advantage of both batch and. Medallion Architecture When organizations first started building data lakehouses, they faced a structuring problem. Open Lakehouse An open lakehouse is a data architecture that stores data in open file and table formats on commodity object storage, so that any compliant engine can read and write it without vendor lock-in. Polyglot Persistence For a long time, the default answer to any data storage question in enterprise software development was a relational database. Separation of Compute and Storage Separation of Compute and Storage is an architectural principle in which the processing layer (compute engines that execute queries and transformations) and. Silver Layer In the Medallion Architecture, data quality is not enforced at the point of ingestion. Zero-ETL Data pipelines are expensive to build and expensive to maintain.