Grafana Transformations: Join by Field – Correlate Metrics and Logs in One Panel

Why you should care about joining data in Grafana

When you’re troubleshooting a service, you often need to see both the metric trend and the log message that triggered it. Pulling those from two different data sources and stitching them together in a single panel can be a game‑changer. Grafana’s Join by field transformation lets you do that without writing a custom query or moving data into an intermediate database.

What the Join by Field transformation actually does

After a panel’s queries finish, Grafana receives the raw data frames from each source and runs the transformation chain in the browser. Join by field takes two or more data frames and merges rows that share a common value in a specified field (e.g., job, instance, or service). The result is a single table that contains every column from the inputs, with rows aligned on the join key.

Because the operation is client‑side, the full unfiltered result set is still transmitted over the network. That means you should keep the raw queries lean and use the transformation only for the final shaping.

Step‑by‑step example: Prometheus metrics + Loki logs

1. Create two queries in the same panel
   • Query 1: Prometheus – rate(http_requests_total{job="web"}[5m]) by (instance)
   • Query 2: Loki – {job="web"} | json | line_format "{{.instance}}: {{.message}}"
2. Add a Join by field transformation
   Open the panel editor, go to Transformations, click Add transformation and choose Join by field. Set Field name to instance and Join mode to Inner join (or Left outer join if you want all metric rows even when no log exists).
3. Verify the result
   In the preview pane you should see a table with columns: instance, rate, message. The number of rows will equal the number of unique instance values that appear in both frames.
4. Check for cardinality issues
   Use the Filter data by query transformation before the join to restrict each frame to a reasonable time window or a subset of instances. Large result sets can freeze the UI.

What to watch out for – trade‑offs and limitations

• Client‑side processing – The browser must hold the entire result set. If your Prometheus query returns 50k rows and your Loki query returns 30k rows, the join could create millions of rows and cause the UI to hang.
• High‑cardinality join keys – Joining on a field that has many distinct values (e.g., container_name) can lead to a Cartesian product if the values aren’t unique in both frames. Always inspect the row count after the join.
• Data frame compatibility – Some data sources (e.g., CloudWatch) return frames with inconsistent field types. If the join field has different types across frames, the transformation will error out.
• Not usable in alerts – Alert rules evaluate queries directly; transformations aren’t applied to alert data. If you need alerting on joined data, you must perform the join in the query language itself.

How to keep the panel responsive

1. Push filtering to the source. For Prometheus, use label selectors or range functions; for Loki, use | json | line_format with a limit if possible.

2. Apply transformations in the right order. Place Filter data by query before Join by field so the join works on a smaller set.

3. Monitor performance. Open Chrome dev‑tools, go to the Performance tab, and record while interacting with the panel. Look for main‑thread blocking times over a second.

Actionable next steps

1. Recreate the Prometheus + Loki example in your own dashboard.
2. Use Filter data by query to limit each frame to the last 15 minutes.
3. Measure the row count after the join using the Table panel’s row counter in the preview pane.
4. If the UI lags, consider moving the join to the query language (e.g., use Loki’s join operator if available) or keep the join in a separate panel with a smaller data set.
5. Export the dashboard JSON and verify that the transformations array contains the join step. This makes the configuration portable for GitOps.

By mastering the Join by field transformation, you can keep your dashboards lean, avoid extra data pipelines, and give operators a single view that ties metrics to logs. Just remember: the browser is doing the heavy lifting, so keep the data set small and the join key low‑cardinality.