Downsampling InfluxDB 2.x with Flux Tasks: Keep the Signal, Drop the Storage Bill

Your IoT fleet writes a point per second per sensor. After a month, the raw bucket is tens of gigabytes, queries over long ranges crawl, and someone asks why you need second-level resolution from February anyway. The honest answer: you don't. What you need is a way to keep recent data at full resolution and older data as compact aggregates — and InfluxDB 2.x gives you the pieces to do exactly that: Flux tasks plus bucket retention policies.

The thesis is simple: let the database downsample itself on a schedule, and let retention policies enforce the lifecycle. No external cron jobs, no batch pipelines, no separate analytics store.

The two-bucket pattern

The standard layout uses two buckets with different retention periods:

• sensor_raw — full-resolution writes, retention of 30 days.
• sensor_5m — 5-minute averages, retention of 2 years.

A scheduled Flux task reads the last few minutes from sensor_raw, aggregates each field into 5-minute means, and writes the result into sensor_5m. When raw points age past 30 days, InfluxDB deletes them automatically. Queries that need recent detail hit the raw bucket; dashboards and long-range analytics hit the small downsampled bucket and run dramatically faster because they scan far fewer points.

A worked example: the downsampling task

Here is a complete task you can paste into the InfluxDB UI (Tasks → New Task) or create via the CLI. It runs every 5 minutes and aggregates the previous 6 minutes — the one-minute overlap guards against points that arrive slightly late.

option task = {name: "downsample-sensors-5m", every: 5m}

from(bucket: "sensor_raw")
  |> range(start: -task.every - 1m)
  |> filter(fn: (r) => r._measurement == "sensor")
  |> aggregateWindow(every: 5m, fn: mean, createEmpty: false)
  |> to(bucket: "sensor_5m")

A few things worth understanding before you adapt this:

• aggregateWindow groups points into 5-minute windows and applies mean to each field. createEmpty: false prevents writing empty windows, which would otherwise bloat the downsampled bucket with null rows.
• task.every inside the range keeps the query window tied to the schedule, so if you later change the interval to 15 minutes, the query follows.
• The task runs with the permissions of the token that created it — it needs read on sensor_raw and write on sensor_5m.

Set the retention on each bucket when you create it (UI: Load Data → Buckets, or influx bucket create --name sensor_raw --retention 30d). Retention in 2.x is a bucket property, not the per-measurement retention policies of 1.x.

The trade-offs you are accepting

Downsampling is lossy by design. Once raw data expires, sub-5-minute detail is gone forever — a 30-second spike at 03:00 becomes invisible in the mean. If you need to catch spikes, aggregate max into a separate field or a third bucket instead of relying on mean alone.

Second, there is lag: downsampled points appear up to one task interval after the raw data. For dashboards this is usually fine; for alerting, always query the raw bucket.

Third, task failures create gaps. If the task is down for an hour, that hour is never downsampled unless the raw data is still within retention and you backfill manually. Check Tasks → Runs in the UI for failed runs, and alert on task failure status if this pipeline matters.

Verify it actually works

Don't trust the green "task created" message. Write test points spaced one second apart into sensor_raw (the CLI works: influx write --bucket sensor_raw "sensor,host=a temp=21.5 $(date +%s)" in a loop), wait two task intervals, then query:

from(bucket: "sensor_5m")
  |> range(start: -1h)
  |> filter(fn: (r) => r._measurement == "sensor")

You should see one point per 5-minute window per field, with timestamps aligned to window boundaries. If you see nothing, check the task's run history for errors — a wrong bucket name or missing measurement filter fails silently in the data plane but shows up in the run log. Finally, confirm retention behaves: points older than 30 days should be absent from sensor_raw while the same time range still returns aggregates from sensor_5m.

Closing

The actionable takeaway: pick your raw retention from how far back you ever need full detail, pick the downsample interval from the coarsest query you actually run, and make the task window slightly wider than its schedule. Start with one measurement, verify the output bucket by hand, then roll the pattern out to the rest of your high-frequency streams.