SQLAlchemy Session and Alembic Migrations: Can rollback() Safely Recover After a Failed Schema Change?
0 reputation · 06 Dec 2022, 23:40 UTC
0 reputation · 06 Dec 2022, 23:40 UTC
An application runs Alembic migrations and ordinary ORM work through SQLAlchemy 2.x Sessions against both PostgreSQL and MySQL. When a migration fails partway through, the recovery code calls session.rollback() and continues using the same Session and its mapped objects.
PostgreSQL supports transactional DDL, so a failed ALTER TABLE can be undone by rollback. MySQL implicitly commits around most DDL, so the same rollback leaves the schema change in place. Alembic exposes a transactional_ddl flag per dialect, which suggests the tool itself treats safe rollback as backend-dependent rather than guaranteed.
Separately, SQLAlchemy requires rollback() after any DBAPI error before the connection is usable again (otherwise a PendingRollbackError is raised), and ORM objects cached in the identity map are not revalidated against the altered schema. It is unclear whether expiring or expunging mapped instances is sufficient, or whether the whole Session must be discarded.
transactional_ddl value for the active dialect a reliable way to decide whether in-place rollback is even meaningful?32775 reputation · 07 Dec 2022, 02:20 UTC
Call session.rollback() to clear the failed transaction state — that part is mandatory — but then discard the Session and start a new one rather than continuing with the same mapped objects. Whether rollback() actually reverts the schema depends entirely on the backend, and the ORM's identity map has no way to know which case it is in. Discarding the Session is the only recovery path that behaves identically on PostgreSQL and MySQL.
The failure modes differ per dialect:
ALTER TABLE inside the migration transaction is reverted by rollback(), so the schema returns to its prior state.rollback() clears SQLAlchemy's transaction bookkeeping but leaves any already-executed schema changes in place — a silent partial migration.Separately, after any DBAPI error SQLAlchemy 2.x requires a rollback() before the connection can be used again; otherwise the next statement raises PendingRollbackError. That rollback fixes the connection state, but it does nothing for the Session state: instances in the identity map still carry attribute snapshots taken against whatever schema existed when they were loaded, and SQLAlchemy does not revalidate them against the altered schema.
transactional_ddl a reliable decision input?It is a reliable indicator of capability, not of outcome. Alembic's per-dialect transactional_ddl flag tells you whether the backend can wrap DDL in a transaction (true for PostgreSQL, false for MySQL). You can inspect it via the migration context or dialect, e.g. context.get_bind().dialect.transactional_ddl. But it cannot make MySQL transactional, and even on PostgreSQL it does not tell you whether your specific failed statement was inside the transaction that got rolled back. Treat it as documentation of why behavior differs, not as a runtime switch that makes in-place rollback safe.
Not reliably. session.expire_all() (or expire() on affected classes) forces attributes to reload on next access, which refreshes column values — but it assumes the objects' identity keys and the table structure they map to are still valid. A failed migration may have dropped or renamed columns mid-flight before the rollback, and any flush attempted during the failure may have left the Session in a questionable state. Expire handles stale data; it does not prove structural consistency. Since a fresh Session costs almost nothing, expiring is an optimization with no real payoff.
try:
run_migration() # alembic upgrade, or ORM DDL work
except Exception:
session.rollback() # required: clears PendingRollbackError state
session.close() # release connection, drop identity map
session = Session() # fresh session for any recovery/logging work
raiseKeep migration connections and application Sessions separate where possible, so a failed migration never shares an identity map with live ORM objects.
Because DDL transactional behavior is backend- and version-specific, confirm on your actual stack:
rollback(); confirm via information_schema.columns that the column is absent.SELECT without rolling back and confirm PendingRollbackError is raised.One caveat: exact implicit-commit behavior on MySQL varies by statement and server version, so verify against your target version rather than assuming.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.