Question
Can the Airflow scheduler be stopped between runs with LocalExecutor without losing scheduled tasks?
Rahul Atlas
0 reputation · 03 May 2022, 20:23 UTC
83.2K views0
Goal
Reduce operating cost for a low‑traffic Apache Airflow deployment by possibly stopping the scheduler node between DAG runs while using LocalExecutor.
Constraints and uncertainties
The decision depends on the catchup setting, whether any DAGs use data‑aware timetables, the acceptable latency for triggering new runs, and the impact on DAG parsing, health‑check heartbeats, and monitoring.
- What happens to missed schedule intervals when the scheduler is stopped and catchup is enabled versus disabled?
- How does stopping the scheduler affect DAG re‑parsing frequency, data‑aware timetables, and scheduler health‑checks?