Downsampling High-Frequency InfluxDB Data Without Losing Long-Term Trends

The storage problem nobody talks about until it hurts

You're collecting 10-second metrics from a fleet of sensors. At 8,640 points per day per series, a modest 100-series deployment generates 864,000 points daily. After a month you're at 26 million points. After a year? Nearly a billion. The default retention policy solves the disk problem by deleting old data — but it also deletes your ability to answer "how did this trend look six months ago?"

Two buckets, one task: the pattern that works

The standard InfluxDB 2.x approach uses two buckets with different retention periods:

• Raw bucket: 7–30 day retention, stores every 10-second point
• Downsampled bucket: 1–2 year retention, stores hourly aggregates (mean, max, min)

A scheduled Flux task reads from the raw bucket, aggregates using aggregateWindow, and writes to the downsampled bucket. The raw data expires naturally; the aggregated data persists for trend analysis.

Writing the Flux task

Create this task in the InfluxDB UI (Data → Tasks → Create Task) or via the influx CLI. You need Admin or Write permissions on both buckets.

option task = {name: "downsample-1h", every: 1h, offset: 5m}

from(bucket: "raw-metrics")
  |> range(start: -task.every)
  |> filter(fn: (r) => r._measurement == "sensor_readings")
  |> aggregateWindow(every: 1h, fn: mean, createEmpty: false)
  |> to(bucket: "downsampled-metrics", org: "my-org")

Key parameters:

• every: 1h — task runs hourly; adjust to match your aggregation window
• offset: 5m — waits 5 minutes after the hour boundary so late-arriving points are included
• range(start: -task.every) — processes exactly one hour of data per run
• createEmpty: false — skips windows with no data (avoids writing null points)

Run the task manually once (influx task run retry --task-id <id> --run-id <run-id>) and verify points appear in downsampled-metrics with timestamps aligned to hour boundaries.

The backfill gap nobody warns you about

Tasks only process data arriving after creation. Your existing three months of raw data won't be downsampled automatically. You need a one-off backfill query:

from(bucket: "raw-metrics")
  |> range(start: 2024-01-01T00:00:00Z, stop: 2024-04-01T00:00:00Z)
  |> filter(fn: (r) => r._measurement == "sensor_readings")
  |> aggregateWindow(every: 1h, fn: mean, createEmpty: false)
  |> to(bucket: "downsampled-metrics", org: "my-org")

Run this in the Data Explorer or via influx query. For large date ranges, chunk it into weekly segments to avoid query timeouts. After backfill, compare point counts:

// Raw count (should be ~8,640 points/day/series)
from(bucket: "raw-metrics")
  |> range(start: -30d)
  |> filter(fn: (r) => r._measurement == "sensor_readings")
  |> count()

// Downsampled count (should be ~24 points/day/series)
from(bucket: "downsampled-metrics")
  |> range(start: -30d)
  |> filter(fn: (r) => r._measurement == "sensor_readings")
  |> count()

Expect roughly 360× reduction (8,640 → 24 points/day). Actual ratio depends on series cardinality and gaps.

What you lose — and how to choose what to keep

Downsampling is lossy. mean erases spikes; max preserves peaks but loses typical behavior; last captures state but misses variation. The choice is irreversible once raw data expires.

Match the aggregate function to your query patterns:

Query need	Function	Trade-off
Trend lines, capacity planning	mean	Smooths spikes
Alert thresholds, SLA breach detection	max	Overstates typical load
State tracking (on/off, open/closed)	last	Misses duration
Distribution shape	percentile (Flux 0.180+)	Higher storage, more compute

Consider writing multiple aggregates to the same downsampled bucket with different _field names (e.g., temperature_mean, temperature_max). Storage cost is still ~1/180th of raw.

Operational guardrails

Task failures create silent gaps. Monitor task runs:

influx task list --user  # shows last run status
influx task log --task-id <id>  # shows recent run output

Set up a dead-man's switch: a simple Flux query that alerts if the downsampled bucket hasn't received points in the last 2 hours. If a task misses runs, re-run it over the missed window using an explicit range(start: ..., stop: ...) — the same pattern as backfill.

Before you shorten that retention window

Test your dashboards and alerts against the downsampled bucket before reducing the raw retention from 30 days to 7. Verify that mean-based trend lines still answer your questions. Once raw data expires, there's no undo.

Start with a 30-day raw / 1-year downsampled split. Run the backfill. Validate point counts. Then — and only then — drop the raw retention to 7 days. Your disk usage will thank you, and your six-month trend queries will still work.