Managing Storage Costs with InfluxDB Retention Policies

The Problem: Unbounded Time-Series Growth

Time-series databases are designed for high-velocity writes. Without a strategy to expire old data, disk utilization will grow linearly until the system crashes or storage costs become unsustainable. The solution is a Retention Policy (RP)—a rule that defines how long data remains in the database before it is automatically purged.

Prerequisites and Versioning

This guide focuses on InfluxDB 1.x (InfluxQL) and the compatibility layers of 2.x. In InfluxDB 2.x, the concept of a "Retention Policy" is integrated directly into the Bucket configuration. Ensure you have administrative access to the InfluxDB shell or API to modify these settings.

Defining a Custom Retention Policy

By default, InfluxDB uses a policy called autogen with an infinite duration. To prevent disk exhaustion, you must create a policy with a specific time window (e.g., 30 days).

Run the following command in the InfluxDB CLI (influx):

CREATE RETENTION POLICY "30_day_metrics" ON "your_database" DURATION 30d REPLICATION 1

Parameter Breakdown:

• "30_day_metrics": The unique name for the policy.
• DURATION 30d: The window of time data is kept. Valid units include h (hours), d (days), and w (weeks).
• REPLICATION 1: The number of copies of the data across a cluster. Set to 1 for single-node installations.

Directing Data to Specific Policies

Creating a policy does not automatically move existing data. You must specify the RP during the write process to ensure new data follows the expiration rule.

When writing via the HTTP API or CLI, include the RP in the request. For example, using the CLI:

INSERT "30_day_metrics" cpu_load,host=server01 value=0.64

If you omit the RP, InfluxDB uses the Default Retention Policy. You can change which policy is the default for a database using:

ALTER RETENTION POLICY "30_day_metrics" ON "your_database" DEFAULT

Comparison: Infinite vs. Fixed Retention

Feature	Infinite Policy	Fixed Policy (e.g., 30d)
Disk Impact	Continuous growth	Stabilizes after duration period
Use Case	Audit logs, critical KPIs	High-resolution telemetry, debug logs
Recovery	Manual deletion required	Automatic shard deletion

Verifying Policy Application

To confirm the policy is active and correctly configured, run the following command in the InfluxQL shell:

SHOW RETENTION POLICIES ON "your_database"

Functional Test: To verify the purge mechanism is working, insert a point with a timestamp that is older than your retention window:

INSERT "30_day_metrics" test_metric,tag=test value=1.0 1600000000000000000

Query the data immediately. If the timestamp is outside the RP window, the point will either be rejected or deleted during the next shard group cleanup cycle, and will not appear in SELECT * FROM test_metric.

Risks and Rollback

Irreversibility: Shortening a retention policy (e.g., changing 90d to 30d) triggers an immediate and permanent deletion of data that now falls outside the new window. This cannot be undone via InfluxDB commands.

Fragmentation: Frequently creating and deleting policies can lead to fragmented shard groups, potentially impacting query performance.

Rollback Procedure: If you accidentally set a retention period too short, you can extend it to prevent further data loss:

ALTER RETENTION POLICY "30_day_metrics" ON "your_database" DURATION 90d

Note: This will not recover data already purged; it only preserves remaining data and future writes.