One Dashboard, Many Environments: Grafana Templating Variables Done Right

You've built a solid dashboard for your production API. Now staging wants the same view. Then the EU region. Then a second service. The copy-paste approach gives you four dashboards that drift apart the first time someone fixes a panel in one of them.

Grafana's templating variables solve this: one dashboard, a dropdown at the top, and panels that rewrite their own queries when the selection changes. This post covers how they work, a concrete chained-variable example, and the performance trap that catches people who go too far.

What a variable actually is

A templating variable is a named placeholder — $region, $service — that you reference inside panel queries, titles, and even other variables. Grafana renders a dropdown at the top of the dashboard; when a user picks a value, every panel referencing that variable re-runs its query automatically. No manual refresh, no duplicated dashboards.

Variables come in a few flavors:

• Query — options are fetched from a data source, e.g. label_values(up, job) against Prometheus. This is the workhorse: the dropdown always reflects what actually exists in your metrics.
• Custom — a static comma-separated list you type in. Good for fixed sets like prod,staging,dev.
• Interval, Data source, Constant, Text box — for time ranges, switching data sources, or free-form input.

Variables can be scoped per-dashboard in the UI, or provisioned globally via JSON/YAML files if you manage dashboards as code.

A worked example: region, then service

The most useful pattern is chaining: one variable's options depend on another's selection. Say your Prometheus metrics carry region and service labels, and you only want to see services that exist in the chosen region.

In Dashboard settings → Variables, create two query variables against your Prometheus data source:

# Variable 1: name = region, type = Query
label_values(up, region)

# Variable 2: name = service, type = Query
label_values(up{region="$region"}, service)

The second query references $region, so when the user switches from us-east to eu-west, Grafana re-runs the service query and the dropdown repopulates with only EU services. Panels then use both:

sum(rate(http_requests_total{region="$region", service="$service"}[5m]))

Enable Multi-value and Include All on the service variable if you want users to compare several services at once. With Prometheus, multi-value variables expand into regex matchers, so {service=~"$service"} is the safer form than = when multi-select is on.

To verify it works: change the region dropdown and watch the service dropdown's options change, then confirm the panel refreshes without touching anything else. Export the dashboard JSON (Share → Export) and check the templating.list section — the variable definitions there should match what you configured in the UI. Re-import the JSON into a scratch folder to confirm it survives round-tripping.

The trade-off: every dropdown change is a query storm

Chained variables are convenient, but each selection change triggers two kinds of work: the dependent variable re-queries the data source to rebuild its options, and every panel referencing the chain re-executes. On a dashboard with twenty panels and a busy Prometheus, a user idly clicking through regions can generate real load.

Practical mitigations:

• Keep variable queries cheap. label_values on a high-cardinality metric is slower than on a small one; point the query at a low-cardinality series if you can.
• Set the variable's Refresh option to "On dashboard load" rather than "On time range change" unless the option set genuinely changes that fast.
• Don't chain three or four levels deep unless you need to — each level multiplies the re-query fan-out.

You can observe the cost directly: toggle a variable while watching your Grafana server logs (or your Prometheus query count metrics) and note how many queries fire per change. If the number surprises you, trim panel count or simplify the chain.

Hygiene that pays off later

Two small habits keep templated dashboards maintainable. First, name variables clearly and avoid reserved characters — $, [, ], and spaces cause pain in queries and in the dashboard JSON. service_name beats var1 when someone edits the raw model six months from now. Second, if you provision dashboards as code, keep the variable definitions in version control alongside the panels so a broken label_values query shows up in review, not in production.

The actionable version: pick your most-duplicated dashboard, replace the hardcoded environment label with a query variable, and verify the dropdown drives the panels. Once that works, add one chained level. You'll retire the copies within the week.