Collecting my attempts to improve at tech, art, and life

Apache Airflow

Apache Airflow

Platform to author, schedule and monitor workflows in Python


Interacts with all components (RDBMS, Spark, Hadoop, etc.) using operators and connections


DAG (workflow)
sequence of Tasks to process a set of data
created by instantiating and configuring an Operator class
Task Instance
created by executing a Task in the context of a particular DAG Run

Added to vault 2024-03-08. Updated on 2024-05-07