Request‑Scoped Write Consistency in Django Using ATOMIC_REQUESTS

Problem

In a typical Django view you may perform several database writes (e.g., create an order, decrement inventory, log an audit entry). If an exception occurs after the first write, the partial changes remain committed unless you wrap the whole request in a transaction. Without a clear, low‑overhead strategy you either get inconsistent data or you add boilerplate everywhere.

Takeaway

Enable ATOMIC_REQUESTS on the primary write database so every HTTP request runs inside a single transaction by default. Use explicit transaction.atomic blocks only for multi‑statement invariants that must fail together, and defer any external side‑effects (email, payment calls, cache invalidation) to transaction.on_commit or an outbox table. This gives request‑scoped write consistency with minimal code changes and clear operational checks.

Requirements

• All database writes triggered by a single request must commit together or roll back together.
• Read‑after‑write within the same request must see the writes made earlier in that request.
• The solution should add little latency to the normal synchronous request/response path.
• External side‑effects must not be rolled back with the database; they must run only on successful commit.

Smallest Suitable Design

Enable ATOMIC_REQUESTS per database

In settings.py set the flag for the alias that receives writes (commonly default).

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'mydb',
        'ATOMIC_REQUESTS': True,   # <‑‑ enables per‑request transaction
    },
    # optional read replica
    'replica': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'mydb_replica',
        'ATOMIC_REQUESTS': False,
    }
}

With this setting, Django wraps each view (or middleware that calls get_response) in a transaction.atomic block using the specified alias. All ORM writes executed on that alias share the same transaction and are committed when the response is sent, or rolled back if an exception propagates.

Read‑after‑write on the same connection

To guarantee that a read sees the writes of the current request, keep the read on the same alias. If you route reads to a replica, do it only after the transaction has committed (see the “Conditions that change the design” section).

# Example view
from django.shortcuts import render
from .models import Order, InventoryLog

def place_order(request):
    # write 1
    Order.objects.create(customer=request.user, total=100)
    # write 2 on same alias
    InventoryLog.objects.create(product_id=42, delta=-1)
    # read‑after‑write (still on default)
    latest = InventoryLog.objects.filter(product_id=42).latest('timestamp')
    return render(request, 'thanks.html', {'log': latest})

Explicit transaction.atomic for multi‑statement invariants

When a group of statements must succeed or fail together independently of the view’s outcome (e.g., a transfer that touches two unrelated tables), wrap them in an explicit atomic block. This creates a savepoint inside the request transaction; if the block fails, only its changes are rolled back while the outer request transaction may still commit.

from django.db import transaction

def transfer_funds(request):
    with transaction.atomic():
        # both statements must succeed together
        Account.objects.filter(id=src_id).update(balance=F('balance') - amount)
        Account.objects.filter(id=dst_id).update(balance=F('balance') + amount)
    # any later writes in the view are still part of the request transaction
    AuditLog.objects.create(action='transfer', amount=amount)
    return HttpResponse('OK')

Trust/Data Boundary

The database transaction is the only atomic boundary Django provides out‑of‑the‑box. Anything that cannot be undone by a database rollback—external APIs, email sends, cache invalidation, search indexing, or message‑queue publishes—must be deferred until after the transaction commits.

Using transaction.on_commit

Register a callback that runs only when the outer transaction commits successfully.

from django.db import transaction

def place_order(request):
    Order.objects.create(customer=request.user, total=100)
    def send_confirmation():
        # external side‑effect, safe to run only on success
        send_mail('Order confirmed', ..., [request.user.email])
    transaction.on_commit(send_confirmation)
    return render(request, 'thanks.html')

Outbox pattern for guaranteed delivery

If you need stricter guarantees (e.g., exactly‑once message publishing), write a record to an outbox table inside the same transaction, then have a separate worker process that reads committed outbox rows and publishes them.

class Outbox(models.Model):
    payload = JSONField()
    published = BooleanField(default=False)
    created_at = DateTimeField(auto_now_add=True)

# inside the view
with transaction.atomic():
    Order.objects.create(...)
    Outbox.objects.create(payload={'order_id': order.id, 'type': 'order.created'})

# worker (runs outside the request)
for obj in Outbox.objects.filter(published=False):
    publish_to_queue(obj.payload)
    obj.published = True
    obj.save()

Operational Checks

• Test rollback behavior: In a test case, raise an exception after the first write and assert that no rows persist for any model touched in the view.
• Verify post‑commit callbacks: Use transaction.on_commit with a test utility like captureOnCommitCallbacks (available in Django 4.2+) to ensure the callback runs only when the view returns a successful response.
• Inspect SQL and locks: Capture the queries for one request with django.db.connection.queries or a database‑specific log. Confirm that all writes appear within a single BEGIN … COMMIT block and that locks are released promptly after the response.
• Multi‑database verification: In a request, call transaction.get_connection('default') before and after a write; ensure the same connection object is used for subsequent reads on that alias.
• Load‑test for contention: Run a realistic concurrency scenario (e.g., with Locust or k6) targeting the write path. Monitor lock wait time, deadlock rate, and replication lag before enabling ATOMIC_REQUESTS globally.

Failure Modes

• Long‑running work inside a request: Because every request holds a transaction, any CPU‑bound or I/O‑bound work (e.g., calling a slow external API) keeps locks longer, increasing contention.
• Nested atomic blocks: They create savepoints; an exception inside a nested block rolls back only to that savepoint, not the whole request, which can be surprising if you expect a full rollback.
• Background jobs and separate requests: They do not inherit the request transaction; each has its own scope.
• Swallowed exceptions: Catching an exception inside an atomic block without calling transaction.set_rollback(True) or re‑raising will cause the transaction to commit, potentially persisting partial work.

Conditions That Change the Design

• Cross‑service invariants: If consistency must span multiple services or message queues, replace the database‑only transaction with an outbox/saga pattern where each service publishes events that are processed idempotently.
• Read scaling after commit: When read traffic is high and stale reads are acceptable, configure a database router to direct reads to a replica after the response is sent (e.g., in a middleware that runs on response).
• High lock contention or replication lag: Shorten transactions by moving non‑essential work out of the view (e.g., defer heavy calculations to a task queue) or reduce the scope of ATOMIC_REQUESTS to only those aliases that truly need it.

Practical Verification Steps

1. Enable ATOMIC_REQUESTS on the write alias in settings.py.
2. Write a test that posts to the view, raises Exception after the first model save, and asserts Model.objects.count() == 0 for all touched models.
3. Add a transaction.on_commit callback that increments a test counter; run the view successfully and confirm the counter increased, then run it with an exception and confirm the counter stayed unchanged.
4. In a staging environment, enable the log_duration option on your database or use django.db.connection.queries to log the duration of the transaction for a typical request; ensure it stays within your latency budget.
5. Run a load test with 50 concurrent users performing the write path; watch the database’s lock wait metrics (e.g., pg_locks for PostgreSQL) and verify they do not show a rising trend.

Limitations

• The approach assumes a synchronous request/response flow. Async views (async def) do not automatically share the same transaction across await points; you must manage transactions explicitly.
• If you use multiple write databases, you must set ATOMIC_REQUESTS on each alias you want to behave this way and remember to pass using= to any explicit atomic block.
• Some database backends (e.g., SQLite) have limited support for concurrent write transactions; test thoroughly before enabling globally in production.