Glossary

Every term, one line each

201 definitions on a single page, for when you need to know what a word means and nothing more. Each one links to the full reference entry.

A

ACID Transactions Table format: ACID is the foundational set of properties that define the correctness guarantees a data storage system must provide for transactions to be considered.
Agentic Analytics Agents & AI: Agentic Analytics represents the next frontier in business intelligence and data engineering.
Agentic Workflows Agents & AI: An Agentic Workflow is a structured sequence of AI-driven steps in which one or more autonomous agents coordinate to accomplish a complex, multi-stage.
AI Agents Agents & AI: An AI Agent is an autonomous software system that uses a Large Language Model (LLM) as its core reasoning engine to perceive its environment, form plans.
Amazon Athena Compute: Amazon Athena is a serverless, interactive query service provided by Amazon Web Services (AWS) that allows users to analyze data directly in Amazon Simple.
Amazon S3 Storage format: Amazon Simple Storage Service (Amazon S3) is an object storage service provided by Amazon Web Services (AWS) that offers industry-leading scalability, data.
Apache Airflow Movement: Apache Airflow is an open-source platform created by Airbnb in 2014 and later donated to the Apache Software Foundation.
Apache Arrow Interchange: Apache Arrow is a cross-language development platform for in-memory data.
Apache Doris Compute: Apache Doris is a modern, open-source Massively Parallel Processing (MPP) analytical database designed for blazing-fast, real-time data warehousing.
Apache Flink Compute: Apache Flink is an open-source, unified stream-processing and batch-processing framework.
Apache Hudi Table format: Apache Hudi (Hadoop Upserts Deletes and Incrementals) is the third major pillar of the Open Table Format ecosystem alongside Apache Iceberg and Delta Lake.
Apache Iceberg Table format: As data lakes grew in popularity, organizations quickly realized that simply dumping raw Parquet or ORC files into Amazon S3 was not a viable long-term.
Apache Paimon Table format: Apache Paimon is the youngest and most architecturally distinctive of the major Open Table Formats.
Apache Spark Compute: Apache Spark is a unified analytics engine designed for large-scale data processing.
Apache XTable (OneTable) Table format: Apache XTable, originally released by Onehouse as "OneTable" and donated to the Apache Software Foundation, represents a fundamentally different category of.
Arrow Flight Interchange: Arrow Flight is an open-source, high-performance Remote Procedure Call (RPC) framework developed as part of the Apache Arrow project.
Arrow Flight SQL Interchange: Arrow Flight SQL is a protocol extension built on top of the Apache Arrow Flight framework.
Attribute-Based Access Control (ABAC) Catalog: Attribute-Based Access Control is an access management model that makes authorization decisions based on the attributes of the subject (who is requesting.
Autonomous Analytics Agents & AI: Autonomous Analytics is the discipline and set of technologies that enable AI systems to independently discover, retrieve, analyze, and interpret enterprise.
Avro Format Storage format: While Apache Parquet dominates the field of analytical data storage, it is not the only file format found in a modern data lakehouse.
AWS Glue Data Catalog Catalog: The AWS Glue Data Catalog is Amazon Web Services' fully managed, serverless metadata catalog service: the central metadata registry for the entire AWS.
Azure Blob Storage Storage format: Azure Blob Storage is Microsoft's massively scalable object storage solution for the cloud.

B

Batch Processing Movement: Batch Processing is the execution of a series of jobs in a computer program without manual intervention (non-interactive).
Bloom Filters Storage format: A Bloom Filter is a space-efficient probabilistic data structure invented by Burton Howard Bloom in 1970.
Branching (WAP) Table format: Software engineering relies heavily on version control systems like Git.
Broadcast Join Compute: A Broadcast Join is a distributed join algorithm where the smaller of the two join inputs is completely replicated ("broadcast") to every worker node in the.
Bronze Layer Foundations: In the Medallion Architecture that structures modern data lakehouses, data does not simply arrive and immediately become analytically ready.

C

Caching Compute: Caching in data systems is the practice of storing copies of frequently accessed data in a faster, closer storage layer so that subsequent requests for the.
Catalog Migration Catalog: Catalog Migration is the process of transitioning Apache Iceberg tables from one catalog implementation to another, from Hive Metastore to a REST Catalog.
Change Data Capture (CDC) Movement: Change Data Capture (CDC) is a set of software design patterns and technologies used to determine and track the data that has changed within a source.
ClickHouse Compute: ClickHouse is an open-source, column-oriented database management system (DBMS) built expressly for online analytical processing (OLAP).
Column-Level Statistics Storage format: Column-Level Statistics are metadata measurements computed over the values in individual columns of data files, stored alongside the data to enable two.
Columnar Formats Storage format: Columnar formats are data storage layouts where data is physically organized and stored by column, rather than by row.
Commit (Iceberg) Table format: In a traditional file-system-based data lake, "committing" data usually just meant finishing the upload of a Parquet file to a directory.
Compaction Table format: A modern data lakehouse is often fed by continuous, real-time data streams (like Apache Kafka or Flink).
Compute Engine Compute: In traditional data warehousing, storage and compute were inextricably linked.
Context Window Agents & AI: The context window of a Large Language Model is the total amount of text, measured in tokens, that the model can process and reason about simultaneously.
Copy-on-Write (CoW) Table format: In a traditional relational database, if you execute an UPDATE statement to change a user's address, the database engine physically overwrites the specific.
Cost-Based Optimizer (CBO) Compute: A Cost-Based Optimizer (CBO) is the intelligent "brain" within a modern relational database or distributed compute engine.
Credential Vending Catalog: Credential Vending is the security mechanism in the Apache Iceberg REST Catalog specification that enables catalog services to dynamically generate.

D

Dagster Movement: Dagster is an open-source data orchestration platform designed to address some of the architectural limitations of Apache Airflow.
Data Fabric Foundations: Modern enterprises rarely store their data in a single place.
Data File Storage format: If you peel back the complex layers of an open data lakehouse (past the query engines, past the Catalog, past the metadata tree of Manifest Lists and.
Data Gravity Foundations: Data Gravity is an analogy coined by Dave McCrory in 2010 to describe the phenomenon by which large concentrations of data attract applications, services.
Data Lake Foundations: The explosion of digital information over the last decade created a massive storage problem.
Data Lakehouse Foundations: A comprehensive definition of Data Lakehouse architecture, combining data warehouse reliability with data lake scalability via open table formats.
Data Lineage Semantics: Data Lineage is the documented record of how data moves, transforms, and evolves through an analytical system, from its origination at source systems through.
Data Mesh Foundations: For most of the 2010s, the standard blueprint for a modern data platform involved building a centralized data lake, staffing a central data engineering team.
Data Modeling Semantics: Data Modeling is the process of creating a visual and logical representation of either a whole information system or parts of it to communicate connections.
Data Pipeline Movement: A Data Pipeline is an automated set of processes and infrastructure that extracts data from various source systems, transforms it into a clean and usable.
Data Quality Semantics: Data Quality is the set of properties, practices, and enforcement mechanisms that ensure data assets in a lakehouse are fit for the purposes for which they.
Data Skew Compute: Data Skew in distributed query execution is the unequal distribution of data or computational work across the worker nodes of a cluster, where some workers.
Data Skipping Compute: Data Skipping is a query optimization technique in which a query engine uses pre-computed metadata statistics about data files to determine, at planning.
Data Swamp Foundations: A comprehensive guide to what causes a Data Lake to become a Data Swamp, how to recognize the warning signs, and the governance practices that prevent it.
Data Warehouse Foundations: A data warehouse is a centralized repository engineered specifically to store highly structured, historical data.
Databricks Compute: An extensive overview of Databricks, the unified data analytics platform that pioneered the data lakehouse paradigm and developed Delta Lake and Apache Spark.
dbt (data build tool) Movement: Understanding dbt, the transformative framework that brought software engineering best practices to SQL-based data transformations.
Delete Files Table format: In an immutable storage layer like Amazon S3, you cannot open a Parquet file, locate a specific row, and hit the "delete" key.
Delta Lake Table format: Delta Lake is one of the three dominant Open Table Formats that define the modern data lakehouse.
Delta UniForm Table format: Delta UniForm, short for Universal Format, is a feature introduced in Delta Lake 3.0 that allows a Delta Lake table to be simultaneously readable as an.
Deserialization Compute: An in-depth look at deserialization and its performance impacts on analytical query engines.
Dictionary Encoding Storage format: Dictionary Encoding is a highly effective data compression technique predominantly used in columnar storage formats like Apache Parquet and Apache ORC.
Dimension Table Semantics: Understanding Dimension Tables, the descriptive context that gives meaning to analytical data.
Dimensional Modeling Semantics: Dimensional Modeling is a specialized data design methodology primarily utilized for data warehouses, data marts, and modern data lakehouses.
Directed Acyclic Graph (DAG) Movement: A Directed Acyclic Graph (DAG) is a conceptual mathematical model heavily utilized in computer science, specifically within the area of data engineering and.
Distributed Compute Compute: A foundational overview of distributed compute architectures in data processing, explaining master-worker topologies, data shuffling, and fault tolerance.
Dremio Compute: Dremio is an open lakehouse platform purpose-built for Apache Iceberg, providing a high-performance query engine, a semantic layer, an integrated Apache.
Dremio Arctic Catalog: A historical overview of Dremio Arctic, the Git-for-data catalog that evolved into Apache Polaris and the Nessie open-source project.
DuckDB Compute: DuckDB is an open-source, in-process SQL OLAP database management system.
Dynamic Catalogs Catalog: Dynamic Catalogs refers to the architectural pattern of configuring query engines and data platforms to connect with multiple Iceberg catalog instances.

E

ELT (Extract, Load, Transform) Movement: For decades, the dominant pattern for moving data from source systems into analytical databases was ETL: Extract, Transform, Load.
Equality Deletes Table format: When dealing with high-velocity streaming data, such as a Change Data Capture (CDC) pipeline ingesting thousands of database updates per second, data.
ETL (Extract, Transform, Load) Movement: Before a business analyst can run a query against a data warehouse, someone has to get the data there.
Eventual Consistency Movement: In any discussion of data reliability in distributed systems, the conversation inevitably polarizes around two competing consistency models: ACID (Atomicity.
Expire Snapshots Table format: A defining feature of Apache Iceberg is its ability to create a new Snapshot for every single transaction.

F

Fact Table Semantics: In dimensional modeling and data warehousing (specifically within a Star Schema or Snowflake Schema), a Fact Table is the central table that stores the.
File Block Size Storage format: File Block Size (also referred to as row group size or split size) is a critical physical configuration parameter in big data storage systems.
File Format Storage format: In the architecture of a modern data lakehouse, there is a strict separation between the Table Format (like Apache Iceberg or Delta Lake) and the File Format.
File Skipping Compute: File Skipping is the collective term for the multi-level system of techniques that allow a data lakehouse query engine to identify and eliminate data files.
Fine-Grained Access Control (FGAC) Catalog: Fine-Grained Access Control refers to the collection of data access enforcement mechanisms that operate below the table level: controlling which rows.
Format Conversion Table format: Format Conversion is the process of reading data stored in one physical file format and rewriting it in a different physical file format.
Format Interoperability Table format: The promise of the Open Data Lakehouse is simple to state and brutally difficult to execute: store data once, on cheap cloud object storage, and make it.

G

Gold Layer Foundations: At the top of the Medallion Architecture sits the Gold Layer: the final destination for data that has been ingested, cleaned, validated, enriched, and now.
Google BigQuery Compute: Google BigQuery is a fully managed, serverless, enterprise data warehouse that enables scalable analysis over petabytes of data.
Google Cloud Storage (GCS) Storage format: Google Cloud Storage (GCS) is a managed, highly scalable object storage service provided by Google Cloud Platform (GCP).
GZIP Compression Storage format: GZIP (GNU zip) is one of the most widely used lossless data compression utilities in the history of computing.

H

Hadoop Catalog Catalog: The Hadoop Catalog (also referred to as the Filesystem Catalog or HadoopCatalog in the Iceberg codebase) is the simplest possible catalog implementation for.
Hallucination Mitigation Agents & AI: Hallucination in Large Language Models refers to the generation of content that is factually incorrect, unverifiable, or entirely fabricated, but presented.
Hash Join Compute: A Hash Join is a join algorithm that uses a hash table to match rows from two input relations based on their join key values.
Hidden Partitioning Table format: Partitioning is a core optimization strategy in massive data lakehouses.
Hilbert Curves Compute: The Hilbert Curve is a mathematically elegant space-filling curve first described by mathematician David Hilbert in 1891.
Hive Metastore (HMS) Catalog: The Hive Metastore is the metadata management service at the heart of the Apache Hive data warehouse ecosystem, and by extension the foundational catalog.

I

Iceberg Catalog Catalog: The Iceberg Catalog is the central nervous system of any Apache Iceberg deployment.
Indexing (Data Lakes) Compute: Indexing in the context of data lakes and lakehouses refers to the set of auxiliary data structures that enable query engines to quickly locate relevant data.

J

JDBC Catalog Catalog: The JDBC Catalog is a catalog implementation for Apache Iceberg that uses any JDBC-compatible relational database as its persistent metadata backend.
Join Strategies Compute: A Join Strategy is the specific algorithm a query engine uses to physically implement a SQL JOIN operation: matching rows from two (or more) tables based on.

K

Kappa Architecture Foundations: Understanding Kappa Architecture, the simplified alternative to Lambda that treats everything as a stream.
Knowledge Graphs Semantics: A Knowledge Graph is a structured representation of knowledge as a network of entities and the relationships between them.

L

Lambda Architecture Foundations: Lambda Architecture is a data deployment model introduced by Nathan Marz designed to handle massive quantities of data by taking advantage of both batch and.
Large Language Models (LLMs) Agents & AI: A Large Language Model (LLM) is an artificial intelligence model trained on massive quantities of text data to understand, generate, and reason with human.
LZ4 Compression Storage format: LZ4 is a lossless data compression algorithm focused on incredibly fast compression and decompression speeds.

M

Manifest File Table format: In the Apache Iceberg metadata hierarchy, the Manifest File is the critical layer sitting directly above the raw data.
Manifest List Table format: In the hierarchical metadata tree of Apache Iceberg, the Snapshot defines the state of the table, but it does not directly list the millions of data files.
Materialized Views Compute: A Materialized View is a pre-computed, physically stored result of a SQL query whose output is saved to storage and can be queried directly, rather than.
Medallion Architecture Foundations: When organizations first started building data lakehouses, they faced a structuring problem.
Merge-on-Read (MoR) Table format: While Copy-on-Write (CoW) provides blistering read performance, its massive Write Amplification makes it unsuitable for high-frequency updates, such as.
Metadata Layer Table format: If data files (like Parquet or ORC) are the muscle of a data lakehouse, and compute engines (like Spark or Dremio) are the brain, the Metadata Layer is the.
Metadata Log Table format: Apache Iceberg achieves ACID transactions on object storage without requiring a continuous compute engine.
Metadata Pointer Table format: In an open data lakehouse architecture based on Apache Iceberg, a single table might consist of millions of Parquet data files, thousands of Avro manifest.
Metadata Translation Table format: Metadata Translation is the technical process of reading the metadata representation of a data table in one Open Table Format and generating an equivalent.
Micro-batching Movement: Exploring Micro-batching, the architectural compromise that simulates streaming using rapid, tiny batch jobs.
Min-Max Statistics Storage format: Min-Max Statistics are per-column metadata stored alongside data in Parquet files and in Iceberg Manifest Files, recording the minimum value, maximum value.
MinIO Storage format: MinIO is an open-source, high-performance, distributed object storage server.
Model Fine-Tuning Agents & AI: Model fine-tuning is the process of taking a pre-trained Large Language Model and continuing its training on a curated, domain-specific dataset to adapt its.
MPP (Massively Parallel Processing) Compute: Massively Parallel Processing (MPP) is an architectural design paradigm for distributed computing and databases.
Multi-Agent Systems Agents & AI: A Multi-Agent System (MAS) is an architecture in which multiple independent AI agents collaborate to accomplish complex tasks that exceed what any single.

O

Object Storage Storage format: Object Storage is a data storage architecture designed to manage massive amounts of unstructured and structured data.
Observability (AI Systems) Agents & AI: AI System Observability is the practice of instrumenting, collecting, and analyzing telemetry data from Large Language Model applications and agentic systems.
Ontology Semantics: An ontology is a formal, explicit specification of a shared conceptualization: a structured vocabulary that defines the types of entities that exist in a.
Open Lakehouse Foundations: An open lakehouse is a data architecture that stores data in open file and table formats on commodity object storage, so that any compliant engine can read and write it without vendor lock-in.
Open Table Formats Table format: A definitive, detailed look guide into Open Table Formats, exploring the architectural shift in approach that bridges the gap between data lakes and data warehouses, featuring an exhaustive analysis of Apache Iceberg, Delta Lake, and Apache Hudi.
Optimistic Concurrency Control (OCC) Table format: When multiple systems attempt to write data to the same table at the exact same time, a database must have a mechanism to resolve the conflict.
ORC Format Storage format: Apache ORC (Optimized Row Columnar) is the second major columnar file format found in modern data lakehouses.
Orchestration Movement: In data engineering, Orchestration is the automated configuration, coordination, and management of complex computer systems, software, and services.
Out-of-Memory (OOM) Errors Compute: An Out-of-Memory (OOM) error occurs when a query execution process attempts to allocate more memory than is available to it, either on a single node.

P

Parquet Format Storage format: Apache Parquet is an open-source, columnar file format designed specifically for fast data processing and massive storage efficiency in the Hadoop and data.
Partition Evolution Table format: In the lifecycle of a data lakehouse, data volume rarely remains static.
Partition Pruning Compute: Partition Pruning is the most powerful form of data skipping available to query engines in the data lakehouse.
Partition Spec Table format: Partitioning is a fundamental technique in data engineering.
Polaris Catalog Catalog: Apache Polaris is the premier open-source implementation of the Apache Iceberg REST Catalog specification: a production-grade, vendor-neutral catalog service.
Polyglot Persistence Foundations: For a long time, the default answer to any data storage question in enterprise software development was a relational database.
Position Deletes Table format: Within the Merge-on-Read (MoR) architecture of Apache Iceberg, there are two methods for creating a logical tombstone to hide a record.
Predicate Pushdown Compute: Predicate Pushdown is a query optimization technique in which filter conditions (predicates) from the WHERE clause of a SQL query are applied as early as.
Prefect Movement: Exploring Prefect, the dynamic, Python-native workflow orchestration framework.
Presto Compute: Presto (often referred to as PrestoDB to distinguish it from its fork, Trino) is an open-source, distributed SQL query engine designed for running.
Project Nessie Catalog: Project Nessie is an open-source transactional catalog for Apache Iceberg that introduces Git-like version control semantics to data lakehouse metadata.
Projection Pushdown Compute: Projection Pushdown (also called column pruning) is a query optimization technique in which only the specific columns required by a SQL query are read from.
Prompt Engineering Agents & AI: Prompt Engineering is the discipline of designing, structuring, and optimizing the text inputs (prompts) provided to a Large Language Model to elicit.
Pushdown Optimization Compute: Pushdown Optimization (specifically Predicate Pushdown and Projection Pushdown) is one of the most critical performance techniques in distributed data.

Q

Query Execution Compute: Query Execution is the runtime phase in which a database or query engine carries out the physical execution plan produced by the query planner, reading data.
Query Planning Compute: Query Planning is the process by which a database or query engine transforms a declarative SQL query into a detailed, optimized execution plan: a concrete.

R

Read Amplification Table format: Read Amplification is the phenomenon where a query must read more data from storage than is logically required to satisfy the query's result, due to the.
Remove Orphan Files Table format: Data lakehouse operations are inherently distributed and prone to environmental failures.
REST Catalog Catalog: The Iceberg REST Catalog specification, commonly called the REST Catalog or IRC, is the single most architecturally significant standard to emerge from the.
Retrieval-Augmented Generation (RAG) Agents & AI: Retrieval-Augmented Generation (RAG) is an AI architecture pattern that enhances a Large Language Model's responses by dynamically retrieving relevant.
Rewrite Data Files Table format: In Apache Iceberg, the abstract concept of "Compaction" is practically executed using a specific maintenance API called rewriteDataFiles.
Rewrite Manifests Table format: While rewriteDataFiles focuses on optimizing the physical Parquet data, it does not optimize the metadata layer.
Role-Based Access Control (RBAC) Catalog: Role-Based Access Control is the dominant access management model in enterprise data infrastructure: a system that grants permissions to named roles, and.
Rollback Table format: Despite the best data quality checks and Write-Audit-Publish patterns, human error inevitably occurs in data engineering.
Row-Oriented Formats Storage format: Row-oriented formats are data storage layouts where the data associated with a single record (a row) is stored contiguously on physical storage media.
Rule-Based Optimizer (RBO) Compute: A Rule-Based Optimizer (RBO) is a critical component of a database's query planning phase.
Run-Length Encoding (RLE) Storage format: Understanding Run-Length Encoding (RLE), a foundational compression algorithm for sorted columnar data.

S

S3 API Compatibility Storage format: S3 API Compatibility refers to the industry-wide phenomenon where competing cloud providers, independent software vendors, and hardware manufacturers have.
Schema Evolution Table format: A comprehensive guide to Schema Evolution in Apache Iceberg, detailing how metadata-only operations provide safe, instantaneous updates to data structures.
Schema Spec Table format: A comprehensive guide to the Schema Spec in Apache Iceberg, detailing how strict column ID tracking enables safe, instantaneous schema evolution without rewriting data.
Semantic Layer Semantics: One of the most persistent and costly problems in enterprise analytics is metric inconsistency.
Semantic Search Agents & AI: Semantic Search is a search methodology that understands the intent and contextual meaning behind a query rather than performing a literal word-for-word.
Separation of Compute and Storage Foundations: Separation of Compute and Storage is an architectural principle in which the processing layer (compute engines that execute queries and transformations) and.
Sequence Number Table format: In a highly concurrent data lakehouse, determining the exact order in which events occurred is critical.
Serialization Compute: Serialization is the process of translating data structures or object state into a format that can be stored (for example, in a file or memory buffer) or.
Shuffle Compute: In distributed query execution, a Shuffle (also called an Exchange or Repartition) is the operation of redistributing data rows across the worker nodes of a.
Silver Layer Foundations: In the Medallion Architecture, data quality is not enforced at the point of ingestion.
Slowly Changing Dimensions (SCD) Semantics: Slowly Changing Dimensions (SCD) is a fundamental concept in data warehousing that deals with a critical problem: How do you handle dimensional data that.
Small File Problem Storage format: The Small File Problem refers to the performance and operational degradation that occurs when a data lake or lakehouse accumulates a very large number of.
Snappy Compression Storage format: Snappy is a fast, lossless data compression and decompression library written in C++ and originally developed by Google.
Snapshot Table format: In a traditional relational database, if you execute an UPDATE statement to change a user's address, the database physically overwrites the old address on.
Snapshot Isolation Table format: When multiple systems interact with the same data simultaneously, chaos can easily ensue.
Snowflake Compute: Snowflake is a fully managed cloud data platform that fundamentally reshaped the data warehousing industry.
Snowflake Schema Semantics: The Snowflake Schema is a logical arrangement of tables in a multidimensional database that is an extension and variation of the Star Schema.
Sort Order Spec Table format: While the Partition Spec determines how data is logically divided into coarse-grained directories (like by year or month), the Sort Order Spec determines how.
Sort-Merge Join Compute: A Sort-Merge Join (SMJ) is a join algorithm that sorts both input relations by their join key and then performs a single linear merge pass through both.
Spilling to Disk Compute: Spilling to Disk is a query engine mechanism that writes intermediate query results (hash tables, sort buffers, shuffle data) to local disk storage when the.
SQL Dialects Compute: SQL (Structured Query Language) is the lingua franca of data analytics.
Staged Commits Table format: The Write-Audit-Publish (WAP) pattern is the gold standard for maintaining data quality in a lakehouse.
Star Schema Semantics: Understanding the Star Schema, the fundamental dimensional modeling technique optimized for analytical query performance.
StarRocks Compute: StarRocks is an open-source, next-generation Massive Parallel Processing (MPP) database designed for blazing-fast, real-time analytics.
Storage Layer Storage format: In the architecture of a modern data lakehouse, the Storage Layer is the foundational bedrock upon which everything else is built.
Streaming Data Movement: Streaming Data refers to data that is continuously generated by thousands of data sources, which typically send in the data records simultaneously, and in.
Strict Metrics Table format: When a query engine like Apache Spark or Trino runs a query against a data lakehouse, its primary goal during the planning phase is to read as little data as.
Strong Consistency Movement: Strong Consistency is the most demanding correctness guarantee a distributed storage system can provide.

T

Table Format Table format: If you look inside a traditional data lake, you will find a collection of directories and files.
Table Maintenance Table format: Table Maintenance is the collection of recurring operational procedures that keep Apache Iceberg tables performant, storage-efficient, and operationally.
Table UUID Table format: When a user interacts with a database, they use human-readable names.
Tabular Catalog: Tabular was a managed data lakehouse platform built entirely around Apache Iceberg, founded in 2021 by the original creators of the Apache Iceberg project.
Tagging (Iceberg) Table format: While Time Travel allows you to query historical data by providing a specific Snapshot ID or Timestamp, remembering a random 19-digit number like.
Target File Size Storage format: Target File Size is the configured desired size (in bytes) for data files written to or produced by a data lakehouse table format like Apache Iceberg.
Text Embeddings Agents & AI: A text embedding is a numerical representation of a piece of text (a word, sentence, paragraph, or entire document) expressed as a dense vector of.
Text-to-SQL Agents & AI: Text-to-SQL is the task of automatically translating a natural language question or instruction into a valid SQL query that, when executed against the target.
Time Travel Table format: In traditional relational databases, querying the past is extremely difficult.
Tool Use (Function Calling) Agents & AI: Tool Use, also called Function Calling, is the capability of modern Large Language Models to generate structured requests to execute predefined external.
Transaction Log Table format: The Transaction Log is one of the most fundamental data structures in computer science.
Trino Compute: Trino (formerly known as PrestoSQL) is a highly parallel and distributed open-source SQL query engine.

U

Unity Catalog Catalog: Unity Catalog is Databricks' enterprise governance layer for the lakehouse: a centralized metadata, access control, and data discovery service that manages.

V

Vector Databases Agents & AI: A Vector Database is a specialized database management system engineered to store, index, and efficiently query high-dimensional vector embeddings at scale.
Vector Search Agents & AI: Vector Search (also called semantic search or similarity search) is a retrieval technique that finds results based on the conceptual meaning and semantic.
Vectorized Execution Compute: A detailed explanation of vectorized execution, the hardware-optimized processing model that allows modern compute engines to achieve blistering speeds.

W

Write Amplification Table format: Write Amplification is a phenomenon in data storage and processing systems where the actual amount of data written to storage is significantly larger than.
Write-Audit-Publish (WAP) Table format: The "silent failure" is the most dangerous event in data engineering.

Z

Z-Ordering Compute: Z-Ordering is a multi-dimensional data clustering technique used in data lakehouse environments to physically co-locate records with similar values across.
Zero-ETL Foundations: Data pipelines are expensive to build and expensive to maintain.
Zstandard (Zstd) Storage format: Zstandard, commonly abbreviated as Zstd, is a fast, lossless data compression algorithm developed by Yann Collet at Facebook (Meta).