Server-Side Matplotlib with Agg: An Architecture Note for Headless Rendering

The problem: charts without a display

When a Python service or batch job needs to produce a PNG chart on a server that has no GUI, no X11, and often no DISPLAY environment variable, the default Matplotlib import can fail or hang. A second, quieter problem appears in long-running workers: figures created through the global pyplot state machine accumulate in memory until the process is restarted. The useful takeaway is that a headless rendering path should explicitly select the Agg backend, build figures through the object-oriented Figure/Axes API, and close every figure after saving.

Requirements for a headless rendering path

• No display server dependency. The code must run when DISPLAY is unset or points to a nonexistent X11 socket.
• Deterministic output. The same input data and library version should produce byte-identical or visually identical PNGs across runs.
• Bounded memory. A worker that renders thousands of charts must not leak figure objects or file descriptors.
• Task isolation. Concurrent requests must not contaminate each other’s axes, titles, or color cycles.
• Operational visibility. Failures—missing glyphs, unwritable paths, backend mismatches—should surface as explicit errors, not silent blank images.

Smallest suitable design

Force the Agg backend before any other import

The Agg backend is a raster renderer bundled with Matplotlib. It writes PNG and other raster formats directly to a buffer or file without opening a GUI toolkit window. On a headless host, calling matplotlib.use("Agg") before importing pyplot or a canvas class guarantees that no interactive backend is loaded. If DISPLAY is unset, Matplotlib may already fall back to Agg, but relying on that fallback makes the behavior dependent on the host environment rather than the application.

Use the object-oriented API, not pyplot’s global state

The pyplot module maintains a global registry of figures and a “current” axes. In a web handler or a multi-tenant batch loop, that global state is a liability: one request can inherit another request’s axes limits or legend. The object-oriented API creates a Figure instance, attaches a FigureCanvasAgg, and adds Axes explicitly. There is no implicit current figure, so each rendering task owns its objects.

Concern	pyplot global API	Figure/Axes object API
State ownership	Global figure registry	Explicit local references
Cross-request risk	High if plt.close() is missed	Low; garbage-collected when references drop
Thread safety	Unsafe to share pyplot state	Unsafe to share one Figure, but easy to isolate per task
Headless fit	Works with Agg, but global state persists	Natural fit for batch and server loops

Close figures to bound memory

Matplotlib does not automatically release a Figure when savefig returns. In a loop, unclosed figures leak memory and eventually file descriptors. Calling fig.clf() clears the axes and artists; deleting the local references allows the canvas and figure to be garbage-collected. If the application also imports pyplot, plt.close(fig) removes the figure from the global registry as well.

Trust and data boundaries

Server-side rendering often handles untrusted or semi-trusted input: user-supplied labels, titles, or numeric ranges. Three boundaries matter. First, font fallback: the default DejaVu Sans covers Latin scripts, but missing glyphs render as hollow boxes. If the service accepts non-Latin labels, register an appropriate font file through matplotlib.font_manager and set it as the default family, or reject input that cannot be rendered. Second, resource exhaustion: a request that asks for a 50,000×50,000 pixel figure can exhaust RAM. Enforce maximum figsize and dpi values in the application layer. Third, interactivity: Agg produces static raster images only. It cannot service GUI event loops, zoom callbacks, or widget picking. Any requirement for interactive exploration belongs on the client side, not in the Agg rendering path.

A minimal rendering function

The following pattern is the smallest design that satisfies the requirements above. Run it inside your application process (for example, a Celery worker, FastAPI background task, or CLI script), not in a shell one-liner. The process needs write permission to output_path; no elevated privileges are required.

import matplotlib
matplotlib.use("Agg")  # set before importing pyplot or canvas classes

from matplotlib.figure import Figure
from matplotlib.backends.backend_agg import FigureCanvasAgg

def render_chart(data, output_path, width_in=6.0, height_in=4.0, dpi=100):
    # data: dict with keys "x", "y", optional "title"
    fig = Figure(figsize=(width_in, height_in), dpi=dpi)
    canvas = FigureCanvasAgg(fig)
    ax = fig.add_subplot(1, 1, 1)
    ax.plot(data["x"], data["y"])
    if data.get("title"):
        ax.set_title(data["title"])
    # bbox_inches="tight" trims outer whitespace
    fig.savefig(output_path, format="png", bbox_inches="tight")
    fig.clf()
    del fig, canvas
    return output_path

Expected checks: after the call returns, confirm that output_path exists and is larger than zero bytes. Open it with Pillow or ImageMagick to verify the pixel dimensions match width_in * dpi and height_in * dpi (within the tight-bbox crop). If the file is missing or empty, the most likely causes are an unwritable directory or an exception swallowed by the caller.

Rollback: because this operation writes a file, wrap the validation in a conditional that removes output_path if the checks fail, so downstream consumers do not read a partial artifact.

Operational checks

1. Backend confirmation. On the headless host, run python -c "import matplotlib; print(matplotlib.get_backend())". It should print Agg or another non-GUI backend. If it prints a GUI backend, the environment is injecting a configuration that overrides the application setting.
2. Raster output sanity. Generate a known chart and inspect the file header. A PNG written by Agg starts with the standard PNG magic bytes; tools like file or Pillow will report the image mode and size.
3. Memory stability. In a staging loop that creates and closes thousands of figures, monitor process RSS. A correctly closed figure loop should plateau; a steadily rising RSS indicates missing clf() or lingering references. If the application imports pyplot, plt.close('all') after each task achieves the same effect.
4. Version pinning. Record the exact Matplotlib version in requirements.txt (for example, matplotlib==3.8.2). After any upgrade, compare a baseline image hash to detect font or layout drift before deploying.

Failure modes

• Thread crashes. Matplotlib is not thread-safe. Sharing a single Figure or the pyplot state across threads can corrupt internal caches and segfault. Give each thread or task its own figure, or move rendering to separate processes.
• Layout engine conflict. tight_layout() and constrained_layout are mutually exclusive. Enabling both on the same axes raises an error or produces unpredictable spacing. Pick one per figure.
• Missing glyphs. Non-Latin text without a registered font renders as boxes. Detect this by inspecting the output image or by pre-validating label character sets against the font’s cmap.
• Silent backend fallback. If matplotlib.use("Agg") is called after pyplot has already selected a backend, the call may be ignored or warn. Place the use() call at the very top of the entry-point module.
• Version drift. Default fonts, tick locators, and savefig defaults can change between major Matplotlib releases. A pinned version plus image-hash comparison catches regressions.

Conditions that would change the design

• Interactive charts required. If users need zooming, panning, or tooltips in the browser, render on the client with a JavaScript library, or use a Matplotlib backend that supports web assembly/WebGL and accept the added complexity.
• Vector output required. Agg is raster-only. For PDF or SVG deliverables, switch the canvas to FigureCanvasPDF or FigureCanvasSVG; the object-oriented structure remains the same.
• Extreme throughput. If the service renders millions of small charts, the per-figure Python overhead may dominate. Profile first; alternatives include caching identical outputs, batching into a single multi-page figure, or moving to a lower-level drawing library.
• Multi-process isolation. When thread-safety concerns outweigh process-spawn cost, run each rendering job in a short-lived subprocess and communicate via temp files or pipes.

Limitations and a practical check

This architecture covers static raster generation in headless Python processes. It does not provide interactive exploration, vector graphics, or protection against maliciously large figure parameters. To check the result in practice, deploy the rendering function to a staging host with DISPLAY unset, invoke it with a representative data payload, and verify three things: the backend reports Agg, the PNG opens with expected dimensions, and a repeated loop of 1,000 renders leaves RSS roughly flat. If any check fails, the failure is almost always in backend selection, figure cleanup, or font availability—not in the data itself.