Open Source Tools

Welcome to the Open Source Tools section of the Data & AI Wiki. Below is an index of prominent open-source engines, table formats, catalog systems, and orchestration tools.

Compute & Processing Engines

• Apache Spark
• Trino
• Apache Flink
• DuckDB
• Apache Spark Structured Streaming

Table Formats & Storage

• Apache Iceberg
• Delta Lake
• Apache Hudi

Catalogs & Governance

• Apache Polaris
• Project Nessie
• Hive Metastore

Orchestration & Pipelines

• Apache Airflow
• Dagster

Part of the Data & AI Terms glossary.