Automate GCS Object Retention with Versioning and Lifecycle Rules

Problem: Uncontrolled version growth drives unexpected storage spend

Many teams enable Google Cloud Storage (GCS) Object Versioning to protect against accidental overwrites, but then leave the bucket to retain every version forever. Over time, especially with frequently updated logs or media assets, the number of non‑current versions can explode, inflating storage bills and complicating compliance audits.

Thesis: Pair versioning with a lifecycle rule that deletes non‑current versions after a set period

By turning on versioning for a bucket and attaching a lifecycle configuration that removes non‑current versions after, for example, 30 days, you keep the ability to restore recent mistakes while ensuring older copies are automatically reclaimed. The rule runs as a background process, so no manual cleanup is required.

Enable Object Versioning on a bucket

Versioning can be enabled via the gsutil command line or the Cloud Console. Once enabled, it applies to all existing and future objects. Note that versioning cannot be turned off without deleting the bucket.

1. Ensure you have the storage.buckets.update permission on the bucket (typically granted by the roles/storage.admin role).
2. Run the following command, replacing MY_BUCKET with your bucket name:

gsutil versioning set on gs://MY_BUCKET

To verify the state, run:

gsutil versioning get gs://MY_BUCKET

The expected output should show gs://MY_BUCKET: Enabled.

Define a lifecycle rule for non‑current versions

Lifecycle rules are expressed in JSON. A rule that deletes non‑current versions after 30 days looks like this:

{
  "rule": [
    {
      "action": {
        "type": "Delete"
      },
      "condition": {
        "age": 30,
        "isLive": false
      }
    }
  ]
}

The isLive: false condition targets only non‑current versions; age counts days since the object became non‑current.

1. Save the JSON to a file, e.g., lifecycle.json.
2. Apply it to the bucket:

gsutil lifecycle set lifecycle.json gs://MY_BUCKET

Confirm the rule is active:

gsutil lifecycle get gs://MY_BUCKET

Worked Example: Daily log bucket

Suppose you have a bucket logs-bucket that receives a new log file named app.log every hour. Overwrite‑style uploads create a new version each hour.

• Day 0: Enable versioning.
• Day 1–30: Each hour creates a new version; after 30 days there are 720 versions of app.log.
• Day 31: The lifecycle rule evaluates the condition. Any version that became non‑current more than 30 days ago is marked for deletion.
• Execution: Within 24 hours, the background process removes those old versions, freeing the space they occupied.

After the first month, you can expect a storage reduction proportional to the fraction of versions older than the retention period. If you keep only the latest 30 days of hourly versions, you retain 720 versions and delete the rest, cutting stored bytes significantly depending on the object size.

Trade‑off / Limitation: Permanent loss of older versions

Once a non‑current version is deleted by the lifecycle rule, it cannot be recovered. If you need longer‑term archival for compliance or audit purposes, you must either:

• Create a separate bucket with a longer retention period or use the Archive storage class.
• Adjust the age value in the lifecycle rule to match your required retention window.

Additionally, versioning adds metadata overhead (approximately a few bytes per version). For buckets with millions of versions, this overhead can become noticeable, though it is usually small compared to the data stored.

Actionable Closing: Test, monitor, and iterate

1. Pick a non‑production bucket to validate the workflow.
2. Enable versioning, upload a few objects, then overwrite them to generate multiple versions.
3. Apply a lifecycle rule with a short test period (e.g., "age": 1) and wait up to 24 hours for the rule to run.
4. Check that non‑current versions are gone:

gsutil ls -a gs://TEST_BUCKET/**

Monitor storage usage via Cloud Monitoring:

• Navigate to Monitoring → Metrics Explorer.
• Select resource type GCS Bucket and metric Total bytes stored.
• Compare the value before and after the rule execution.

Review audit logs in Cloud Logging for storage.objects.delete entries to confirm the lifecycle deletions occurred as expected. Once satisfied, promote the configuration to production and adjust the age to your business‑required retention window.