Task Retry Idempotency and Side Effect Duplication

Apache Airflow 2.x provides a robust retry mechanism through the retries and retry_delay parameters, allowing failed task instances to be rescheduled automatically. However, the platform does not natively enforce idempotency for side effects occurring within operators, such as database insertions or API requests.

While the TaskInstance object provides try_number and run_id via the task context, these values can be reset if a task is manually cleared from the UI. This creates a gap between the Airflow scheduler's state and the state of external systems that may have partially processed a previous attempt.

Given that Airflow lacks a built-in transactional outbox or checkpointing mechanism to synchronize internal retries with external writes, how can a deterministic idempotency key be constructed to survive manual task clears? Is there a recommended way to distinguish between an automated retry and a manual re-run to prevent duplicate data entry?