Choosing Kibana Lens vs. Custom Vega for Fast Data Exploration

The problem: building a quick dashboard without writing DSL

When you need to explore logs, metrics, or traces, writing Elasticsearch DSL by hand can slow you down, especially if you are not comfortable with the query language. You want a way to drag fields onto a canvas and see a chart appear instantly, while still knowing that the visualization is backed by a real Elasticsearch aggregation.

Thesis: Lens gives you instant, portable visualizations for most exploratory needs, while Vega remains the tool for custom calculations.

Kibana Lens provides a drag‑and‑drop interface that automatically translates your selections into an Elasticsearch aggregation request. The result is a visualization that can be saved, shared, and moved between clusters just like any other Kibana saved object. For calculations that Lens cannot express—such as scripted fields, pipeline aggregations, or layered mark types—you fall back to authoring a Vega (or Vega‑Lite) specification directly.

How Lens translates UI choices into Elasticsearch aggregations

Each drop target in Lens corresponds to a specific aggregation type: a date histogram becomes a date_histogram bucket, a terms split becomes a terms bucket, and a metric such as Average or Sum becomes an avg or sum metric on the numeric field. Lens builds a concise JSON representation of these choices; the Kibana server then sends that JSON to Elasticsearch as the query body. Because only the aggregated result set is returned, network traffic stays low even for large indices.

When Lens is the right choice

• You need a quick bar, line, pie, heatmap, or table view of one or two fields.
• The visualization does not require custom calculations beyond built‑in metrics (count, average, sum, min, max, percentiles).
• You want the visualization to be version‑controlled as a saved object and easily moved between spaces or clusters.
• You prefer to inspect the generated DSL occasionally via the “View source” button to learn the underlying query.

When to fall back to custom Vega

• You need a Painless script or runtime field to derive a value on the fly.
• Your chart requires a pipeline aggregation such as a moving average, cumulative sum, or derivative that Lens does not expose.
• You want to layer different mark types (e.g., a line over a bar) or combine multiple independent aggregations in a single view.
• You must fine‑tune the Vega‑Lite configuration (axes, legends, tooltips) beyond what Lens offers.

Worked example: creating a time‑series histogram with Lens

1. Open Kibana → Stack Management → Index Patterns. Confirm you have an index pattern (for example logs-*) that contains a date field @timestamp and a numeric field response_time. You need read permission on the matching indices.
2. Navigate to Analytics → Lens.
3. Drag @timestamp to the X‑axis, choose “Date histogram” as the bucket type, and leave the interval on “auto” (or set a fixed interval like “1h”).
4. Drag response_time to the Y‑axis, select “Average” as the metric.
5. Click the “View source” button. A modal shows the generated Elasticsearch DSL; verify it contains a date_histogram aggregation on @timestamp and an avg metric on response_time.
6. Press “Save”, give the visualization a name (e.g., Response time over time), and add it to a dashboard.

To confirm the efficiency, open your browser’s developer tools, go to the Network tab, and locate the request to /_es_admin/_query (or similar) triggered by the Lens panel. The request body will match the DSL you saw in “View source”, and the response will contain only the bucketed averages—typically a few kilobytes even when the source index holds millions of documents.

Trade‑off: flexibility vs. convenience

Because Lens hides the DSL, troubleshooting an unexpected chart means you must first click “View source” to see the query that was sent. If the result still looks wrong, you cannot inject a Painless script or a custom pipeline aggregation directly from the Lens UI; you must either pre‑compute the field in Elasticsearch or recreate the visualization in Vega. This trade‑off is worthwhile for rapid exploration, but becomes a limitation when you need advanced analytics.

Actionable closing

Start every new exploratory visualization in Lens. Use the “View source” button as a learning tool to verify that the generated DSL matches your intent. When you encounter a Lens limitation, export the Lens spec as a JSON saved object, open it in the Vega editor, and extend it with the required calculations. This approach keeps your dashboards portable while giving you the power of Vega only when necessary.