Question
Null-only measurements are omitted during InfluxDB 1.x-to-2.x bucket migration
Cig Orbit
0 reputation · 09 May 2020, 11:38 UTC
139.7K views1
Loading…
0 reputation · 09 May 2020, 11:38 UTC
The influxd-upgrade utility converts InfluxDB 1.x line protocol data into the 2.x bucket model. During this conversion, any data point where every field value is null is omitted from the target bucket. The behavior is documented in the 1.x-to-2.x migration guidance, and no configuration flag is available in current 2.x releases to alter it; operators should confirm the exact behavior against their target version.
This creates a compatibility boundary for teams that rely on null-only points to represent intentional gaps or to drive downstream sample-count calculations. Because the tool logs dropped null points only when logging is set to info or higher, a default or lower verbosity run may not surface the loss. Whether null-only points should be preserved remains an open design question in the project, so the long-term direction is unresolved.
Before committing to an upgrade path, operators must decide whether to pre-clean the 1.x data, reinsert null points manually after migration, or accept the resulting cardinality reduction. The choice depends on whether the affected series carry semantic meaning beyond their null values.
What is the recommended approach for preserving intentionally null measurements when running influxd-upgrade in current 2.x releases? Does the tool expose any logging or dry-run mechanism to reliably identify all null-only series before the actual migration? Are there documented plans to add a configuration option that retains null-only points in future 2.x versions?
In current InfluxDB 2.x releases, influxd-upgrade omits any 1.x point where every field value is null, and the utility does not expose a configuration flag to retain them. To preserve intentionally null measurements, you must either transform the source data before migration or re-write the null-only points after the upgrade using the InfluxDB 2.x write API.
The tool supports a --dry-run mode that simulates the conversion without writing to the target bucket. When combined with --log-level info, the dry run logs each dropped null-only point. A default verbosity level (typically WARN) will not surface these messages, so raising the log level is necessary for reliable detection.
influxdbinfluxd-upgrade --dry-run --log-level infoRun this against a small copy of the 1.x dataset first. Review the info-level output for dropped-point messages, then compare the resulting bucket contents to confirm that null-only series are absent. Note that info logging can significantly increase log volume; ensure retention policies can handle the output for the duration of the run.
Two approaches are available, depending on whether the null values carry semantic weight:
influxd-upgrade completes, use the InfluxDB 2.x write API to re-insert the null-only points. This preserves the original schema but increases cardinality and storage; verify that the target bucket has sufficient capacity and that downstream queries tolerate the extra points.There is no confirmed, documented plan to add a flag that retains null-only points in future 2.x releases. The long-term direction remains unresolved, so do not design a migration strategy that depends on a future configuration option.
These steps assume a 2.x target where influxd-upgrade still uses the documented null-omission behavior. Before production use, run influxd-upgrade --help to confirm that --dry-run and --log-level are available in your specific version, and consult the current 1.x-to-2.x migration documentation for your release.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.