Using Airflow’s TaskFlow API to Replace Manual XCom Boilerplate
Learn how the @task decorator turns Python functions into tasks with automatic XCom handling, making DAGs cleaner and easier to maintain.
Loading…
ReadMeFeed / Community knowledge
#apache-airflow
Real questions. Useful conversations. Find the people who know your stack.
Architecting Data Flow in Airflow: Transitioning to the TaskFlow API
Learn how to use the Apache Airflow TaskFlow API to eliminate XCom boilerplate, manage data boundaries, and avoid metadata database bloat in your pipelines.
apacheairflowChoosing the Right Airflow Executor for Your Workload
A decision guide for selecting an Apache Airflow executor (Local, Celery, Kubernetes, CeleryKubernetes) based on workload scale, infrastructure constraints, and operational trade-offs, with configuration examples and validation steps.
Using Apache Airflow TaskFlow API to Simplify DAG Definition
Learn how the @task decorator turns Python functions into Airflow tasks with automatic XCom handling, see a concrete ETL example, and understand size limits and common pitfalls.
Using Apache Airflow TaskFlow API to Eliminate XCom Boilerplate
Learn how the TaskFlow API lets you write tasks as plain Python functions, automatically handling dependencies and data passing, and understand its pickle‑based limitations.
pythonDiagnosing Apache Airflow Scheduler Deadlocks that Keep Tasks in Queued State
Identify why Airflow tasks stay queued due to scheduler deadlocks, check logs, DB transactions, and process counts, then apply targeted fixes and know when to escalate.
Scaling Pipeline Deployment with Dynamic DAG Generation in Apache Airflow
Stop copy-pasting DAG files. Learn how to use dynamic DAG generation in Apache Airflow to scale your pipelines using configuration files while avoiding Scheduler performance pitfalls.
Selecting the Right Apache Airflow Executor for Your Workload
A technical guide for choosing between Local, Celery, and Kubernetes executors in Apache Airflow based on scaling needs, infrastructure constraints, and fault tolerance.
kubernetesAirflow 2.6 db upgrade failure leaves no supported rollback path — how to recover the metadata database safely?
Failed airflow db upgrade on 2.6.x When an Airflow upgrade applies Alembic migrations via airflow db upgrade and one script fails midway, the metadata database can be left in a partially migrated state. As of Airflow 2.6.x there is no built-in rollback command — no airflow db rollback exists, and Alembic's downgrade is not exposed through the Airflow CLI. Co