How can I detect and verify a memory leak in Flask's request context caused by uncleaned g data?
0 reputation · 06 Jan 2022, 17:59 UTC
0 reputation · 06 Jan 2022, 17:59 UTC
I want to determine whether storing large temporary objects in the Flask `g` proxy for each request leads to a memory leak, and how to confirm that a fix using `teardown_appcontext` actually releases those objects. The goal is to observe memory growth over many requests and ensure it stabilizes after the cleanup.
I am working with a standard Flask application that uses the request context and the `g` object, and I can reproduce the scenario locally. I am unsure which profiling or monitoring tools reliably show per‑request memory usage in Flask, and how to assert that the context is properly torn down so that no residual references remain.
What tools or techniques can be used to monitor memory usage per request in Flask? How can I verify that `g` is cleared after each request? Is there a reliable way to assert that no objects remain in the context after the teardown callback?
To confirm whether storing large temporary objects in Flask’s g proxy creates a memory leak, you can measure memory usage over many requests and verify that the teardown_appcontext callback releases those objects. The following approach uses only standard Python tools and Flask’s built‑in signals.
g.
from flask import Flask, g
import json
app = Flask(__name__)
@app.before_request
def load_big():
# Simulate a large temporary payload (e.g., 1 MB list)
g.big_data = list(range(250_000)) # ~2 MB of ints
@app.route('/')
def index():
return 'OK'
# Optional: explicit cleanup via teardown_appcontext
@app.teardown_appcontext
def cleanup_g(exc=None):
g.big_data = None # break reference
tracemalloc (or psutil) to take a snapshot before the loop, after each N requests, and after the loop.
import tracemalloc
from flask.testing import FlaskClient
tracemalloc.start()
client = FlaskClient(app)
snapshot0 = tracemalloc.take_snapshot()
for i in range(0, 5000, 100):
for _ in range(100):
client.get('/')
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.compare_to(snapshot0, 'lineno')
print(f'After {i+100} requests:')
for stat in top_stats[:5]:
print(stat)
teardown_appcontext cleanup), the leak is controlled. Without the cleanup, you will see a steady upward trend.
g is empty after teardown.
Connect to the appcontext_tearing_down signal and inspect the proxy’s internal object.
from flask import appcontext_tearing_down
@appcontext_tearing_down
def assert_g_empty(sender, **extra):
# Get the real object behind the proxy
real_g = g._get_current_object()
assert not vars(real_g), f'g still holds {vars(real_g)} after teardown'
If the assertion never fires, the context is properly cleared.
g is either:
teardown_appcontext handler (as shown), org at all for long‑lived data.Use comments to ask for clarification. Post a solution as an answer.
26,525 reputation · 06 Jan 2022, 21:48 UTC
While tracemalloc is excellent for identifying where memory is allocated, it doesn't always reveal why an object isn't being garbage collected. If you suspect that teardown_appcontext is running but objects still persist, the objgraph library provides a more targeted way to verify the leak.
You can use objgraph.show_growth() between request batches to see exactly which object types are increasing in count. To find the specific reference keeping a g-assigned object alive, you can generate a reference graph:
import objgraph
# After a series of requests
leaking_objects = objgraph.by_type('YourLargeObjectClass')
if leaking_objects:
objgraph.show_backrefs(leaking_objects[0], filename='refs.png')
This is particularly useful for detecting "hidden" leaks, such as when a large object in g is accidentally captured by a closure or a global registry, preventing the standard Flask teardown from actually freeing the memory.