Choosing Between Bucket‑Level and Tag‑Based Retention Policies in InfluxDB 2.x

Problem: How long should each series live?

When you ingest high‑volume time‑series data, you need a clear data‑life‑cycle policy. In InfluxDB 2.x you can either apply a single bucket‑level retention policy or create per‑series rules that depend on tags. The choice determines how much storage you use, how fast queries run, and how complex your administration becomes.

Decision: Define the retention policy scope

Choose bucket‑level if:

• Your datasets are homogeneous or you’re okay with a single lifetime for all series.
• Simple administration and predictable backup/restore are priorities.
• You don’t need fine‑grained cost controls.

Choose tag‑based per‑series if:

• Some series are high‑value and must be kept longer.
• Others can be pruned quickly to save space.
• You’re comfortable managing multiple rules and monitoring their interaction.

Compact Comparison

Feature	Bucket‑Level	Tag‑Based (Flux)
Scope	All series in the bucket	Series matching specific tag values
Administration	Single policy per bucket	Multiple rules, one per tag set
Flexibility	Low – cannot vary by series	High – target only critical data
Impact on Query Planning	Minimal – planner sees one policy	Higher – planner must evaluate tag filters per query
Backup/Restore Simplicity	Easy – one policy per bucket	Complex – need to preserve rule set
Risk of Unintended Pruning	Low – uniform rule	Higher if tag cardinality is high
Compaction Alignment	Direct – compaction aligns with policy	Requires careful mapping of compaction windows

Trade‑Offs Explained

Bucket‑level policies are immutable once created; changing the duration means dropping and recreating the policy, which can temporarily delete data if not handled correctly. Tag‑based policies let you keep, for example, error logs for 90 days while dropping metrics after 30 days, but they add metadata overhead and can cause rule conflicts if you have many tag combinations.

Mixing Flux and InfluxQL retention definitions in the same bucket is not supported. An attempt to do so will produce a configuration error and can leave the bucket in an inconsistent state.

Retention policies also control when compaction runs. If you set a 30‑day bucket policy but your compaction window is 7 days, you risk removing data before the policy’s intended lifetime. Align compaction windows to the longest retention period you use.

Concrete Implementation: Flux Example

Below is a Flux script that creates a 30‑day bucket‑level retention policy named rp_30d and sets it as the default for bucket mydb. Replace mydb with your bucket name.

import "influxdb" as db

// Set the target bucket
_ = db.setDatabase("mydb")

// Create a 30‑day absolute retention policy
_ = db.createRetentionPolicy(
    name: "rp_30d",
    policyType: "absolute",
    duration: 30d,
    replicaCount: 1,
    default: true
)

Run this script in the InfluxDB CLI or the UI Flux editor. You need write permission on the bucket and admin rights to create policies.

Verify the Policy

1. Execute SHOW RETENTION POLICIES ON mydb in the InfluxDB CLI.
   • Check that the rp_30d policy is listed, has a duration of 30d, and is marked default = true.
2. Run a query that requests data older than the retention window:

   from(bucket: "mydb")
     |> range(start: -35d)
     |> filter(fn: (r) => r._measurement == "measurement")
     |> limit(n: 1)
   

   • If the result set is empty, data older than 30 days has been pruned successfully.

Tag‑Based Retention Example

Suppose you want to keep series where critical=true for 90 days and all others for 30 days. In Flux you can create two policies and apply a tag filter when writing data:

import "influxdb" as db

// Create two policies
_ = db.createRetentionPolicy(name: "rp_90d", policyType: "absolute", duration: 90d, replicaCount: 1, default: false)
_ = db.createRetentionPolicy(name: "rp_30d", policyType: "absolute", duration: 30d, replicaCount: 1, default: true)

// When writing, choose policy based on tag
write = (data) => {
  policy = if data["critical"] == "true" then "rp_90d" else "rp_30d"
  db.write(bucket: "mydb", policy: policy, data: data)
}

Note: The policy selection logic lives in the ingestion pipeline (e.g., Telegraf, custom app). InfluxDB itself does not automatically apply tag‑based retention; you must route data accordingly.

Limitations & Practical Checks

• Retention policies are immutable. To change a duration, drop the policy and recreate it. Always back up your data before doing so.
• High tag cardinality can create many policies, increasing metadata load and query planning time.
• Flux retention definitions cannot coexist with InfluxQL definitions in the same bucket.
• Compaction settings must match the longest retention period you use to avoid premature deletion.

To monitor policy health, schedule periodic runs of SHOW RETENTION POLICIES and compare the duration and default flags against your design document.

Practical Decision Checklist

1. Do you have heterogeneous data lifetimes? Yes → consider tag‑based.
2. Is administrative simplicity more important than storage savings? Yes → bucket‑level.
3. Do you have a high cardinality of tags? No → tag‑based is feasible.
4. Can you tolerate the risk of accidental data loss during policy changes? No → avoid frequent changes; prefer bucket‑level.

Conclusion

Choosing the right retention strategy in InfluxDB 2.x boils down to balancing flexibility against operational overhead. Bucket‑level policies offer a single, easy‑to‑understand rule that covers all series, while tag‑based policies give you fine‑grained control at the cost of added complexity. Use the decision checklist to align the choice with your organization’s data‑lifecycle goals.