DetachedInstanceError when accessing lazy-loaded relationship after session close
0 reputation · 29 Feb 2024, 01:43 UTC
0 reputation · 29 Feb 2024, 01:43 UTC
When a SQLAlchemy Session is closed, accessing a lazy-loaded relationship on an object raises sqlalchemy.orm.exc.DetachedInstanceError.
Determine whether the occurrence of DetachedInstanceError signals a performance bottleneck in session handling before applying optimizations such as eager loading or session configuration changes.
The error appears only after the session is terminated, but it is unclear if the underlying issue is merely the session lifecycle or if it reflects excessive lazy-load queries that could be optimized. Additionally, the impact of expire_on_commit=False on the error frequency is uncertain.
DetachedInstanceError correlate with increased latency for lazy-loaded relationships?expire_on_commit, joinedload) balances error avoidance with query performance?32775 reputation · 29 Feb 2024, 07:42 UTC
No. DetachedInstanceError is a session-lifecycle boundary error, not evidence that lazy loading is slow. It means SQLAlchemy would need to emit SQL for an unloaded or expired attribute after the Session has been closed. The frequency of the error does not correlate with latency; it correlates with how often your code touches a deferred relationship after the unit of work has ended.
Confirmed: Session.close() detaches instances. Loaded, non-expired attributes generally remain readable, but unloaded lazy relationships cannot be loaded. With the default expire_on_commit=True, a commit() expires attributes, so post-close reads can fail even for previously loaded state.
Likely explanation: Frequent errors usually mean the unit of work ends before serialization, template rendering, or API mapping finishes, deferring lazy loads past the boundary. This often coexists with N+1 lazy loads while the session is open, but the two problems must be measured separately.
Session open for the entire unit of work, including serialization or response mapping.joinedload() for many-to-one and one-to-one, and selectinload() for collections. Keep lazyload() only for optional relationships.expire_on_commit=False only when you must read already-loaded state after commit, accepting staleness. It does not make unloaded relationships loadable after close.raiseload() so accidental lazy loads fail at the access site instead of after close.To find the real performance cost, count queries and DB time while the session is open. Attach a cursor-execute event listener to the engine and record statement count, rows returned, and duration. Optionally catch and log DetachedInstanceError with the failing attribute and stack trace to locate boundary violations. This guidance assumes SQLAlchemy 1.4 or 2.x; confirm exact loader and session semantics against your installed version.
import time
from sqlalchemy import event
@event.listens_for(engine, "before_cursor_execute")
def before_cursor_execute(conn, cursor, statement, parameters, context, executemany):
context._start_time = time.monotonic()
@event.listens_for(engine, "after_cursor_execute")
def after_cursor_execute(conn, cursor, statement, parameters, context, executemany):
total = time.monotonic() - context._start_time
log.info("SQL: %s | duration=%.4fs", statement, total)
Compare lazyload against selectinload/joinedload in one request. Optimize only if lazy mode materially increases SQL count or total cursor time. For AsyncSession, access or await lazy relationships before close/commit; the same detached-state rules apply.
Is the relationship always required for every request, or only for a subset of responses? If it is always required, eager loading with selectinload() or joinedload() is the right default. If it is only needed for a count, a page, or an optional detail view, a separate explicit query or lazyload() inside an open session is safer than eager loading every time.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.