Airflow on CPython: how do I know which Airflow release will be the last to support my Python version?
0 reputation · 26 Jun 2022, 11:33 UTC
0 reputation · 26 Jun 2022, 11:33 UTC
We run Apache Airflow 2.x on CPython and are planning upgrades across the Airflow↔Python boundary. Each Airflow minor release declares its supported Python range via the Requires-Python package metadata, and Airflow publishes per-Python constraints files for installation, but there seems to be no published schedule for when an end-of-life Python version is dropped.
Our constraint: we want to stay on a supported pairing and need to predict the last Airflow minor release supporting our Python version before committing to an upgrade path. The uncertainty is whether we can rely on the Requires-Python range alone, or whether provider packages may declare different Python requirements than core Airflow.
Is the Requires-Python metadata of the apache-airflow package the authoritative declaration of the supported Python range for a given Airflow minor release, and can pip be trusted to refuse an incompatible combination? Is there any documented policy or announcement mechanism for when Airflow drops a Python version, and do provider packages follow the same range as core Airflow?
34275 reputation · 26 Jun 2022, 13:28 UTC
Yes — the Requires-Python field in the apache-airflow package metadata is the authoritative declaration of the supported Python range for a given release, and pip enforces it. There is no fixed calendar for Python drops, but Airflow follows a predictable convention: a Python version is dropped in the first Airflow minor release cut after that Python version reaches its official end-of-life. Providers normally inherit the core range, but they are allowed to declare a narrower one.
Confirmed by tooling behavior: each Airflow release on PyPI carries a Requires-Python specifier (e.g., >=3.8,<3.12). When you run pip on an interpreter outside that range, resolution fails with a message like Requires-Python >=3.9; python_version is "3.8". This is a hard gate, not a warning — pip will not install the combination, so you cannot accidentally end up on an unsupported pairing through normal installs.
Likely, based on project convention: Airflow aligns Python support with the CPython lifecycle rather than publishing its own EOL table. In practice, a Python version disappears from the supported range shortly after it goes end-of-life upstream, and the change is called out in release notes and on the Airflow blog. Treat this as a convention to monitor, not a guarantee — there is no automated schedule you can subscribe to beyond watching release notes and the dev mailing list.
Check the metadata of successive releases directly. The fastest way:
python -V # your interpreter, e.g. Python 3.9.18
pip index versions apache-airflow # list available releases
pip install "apache-airflow==2.9.3" --dry-run --no-depsThe dry run on your target interpreter either resolves (supported) or fails on the Requires-Python check (not supported). Walk backward through minor releases until one resolves; that is your last supported pairing. You can also inspect metadata without installing:
pip download --no-deps apache-airflow==2.9.3 -d /tmp/af
tar -xzf /tmp/af/apache_airflow-2.9.3.tar.gz
grep -i "requires-python" /tmp/af/apache_airflow-2.9.3/PKG-INFOOne caveat on the dry-run method: it tells you what installs, not what is tested. The constraints files Airflow publishes per Python version are the better signal for a tested dependency set — if Airflow ships constraints for constraints-3.9.txt for a release, that Python is a first-class citizen of that release.
Usually yes, but not contractually. Provider packages (apache-airflow-providers-*) are released independently of core and declare their own Requires-Python. A provider can tighten its range early — typically because a third-party library it wraps (a cloud SDK, a database driver) dropped your Python version before Airflow core did. So even if core Airflow 2.x still supports your interpreter, a specific provider version you need might not.
Verify per provider the same way:
pip install "apache-airflow-providers-amazon==8.20.0" --dry-run --no-depsAnd pin providers with the matching constraints file rather than letting pip float them, so a provider release with a narrower range can't surprise you mid-upgrade.
Requires-Python includes your interpreter.If you share your current Python version and the providers you depend on, the exact last-supported Airflow minor can be narrowed down precisely — the provider set is the one variable that changes the answer.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.