Airflow Workers and Secrets Backends: Runtime Connection Authorization and Expired Credential Boundaries

Direct answer

No. Apache Airflow 2.x does not provide a native per-connection runtime authorization layer that limits which connection IDs a task can reference independent of web UI RBAC. Admin, User, and Viewer roles govern access to the Airflow UI and REST API; they do not filter get_connection lookups inside a worker. If the worker process can reach the configured secrets backend and the backend returns a value for a requested connection ID, the task can use it.

What controls credential freshness after rotation or expiry

Airflow resolves connections lazily, typically when a hook or operator runs inside the task worker, not at scheduler dispatch. Whether a running worker sees a rotated credential before task execution depends on:

• when get_connection is called relative to the rotation,
• in-process caching of connection objects within the worker,
• secrets-backend or SDK credential caching and TTL, and
• executor process lifetime.

This behavior is provider-specific. Airflow core exposes no single TTL or cache-refresh setting for AWS Secrets Manager, Google Cloud Secret Manager, HashiCorp Vault, or environment variables. Provider-specific knobs, if any, live in backend_kwargs and the installed provider package documentation.

Executor differences

Long-lived workers, such as Celery workers, may retain a fetched connection or credential for the duration of a task or across tasks until the connection is refetched or the process restarts. Ephemeral task runners, such as KubernetesExecutor, typically fetch the connection in a new task pod, so rotation is more likely to be visible on the next task execution. This distinction changes the operational response to expired credentials.

Preflight credential validation

Airflow core has no documented scheduler- or worker-level preflight gate that validates credential freshness before a task is dispatched. Common compensatory patterns are:

• a custom secrets-backend wrapper that inspects secret version or metadata before returning a connection,
• an upstream pre-check task or sensor that attempts a lightweight authorization call and fails fast, and
• short-lived tokens or file-mounted secrets with mtime checks, combined with worker restarts after rotation.

Verification steps

1. Inspect the configured backend: airflow config get-value secrets backend and airflow config get-value secrets backend_kwargs.
2. Confirm the executor: airflow config get-value core executor.
3. Rotate a test secret, wait past any suspected client cache TTL, then run a task referencing the same connection ID without restarting the worker and observe whether the old or new value is used.
4. Review the hook or operator code path to confirm connection retrieval happens in the worker at runtime, not during DAG parsing.

Missing diagnostic detail

Which executor are you using? The recommendation for forcing credential rotation differs between long-lived Celery workers, which may require process restarts, and ephemeral executors, which usually pick up changes on the next task pod.

Assumptions: Airflow 2.x core behavior; provider package versions may change caching behavior. Verify against your installed provider package rather than Airflow core documentation. Claims about specific provider TTL defaults are omitted because they are version-dependent and require current verification.