Automate GCS Cleanup: Delete Objects Older Than 90 Days with Lifecycle Management

Why You Need Automated GCS Deletion

In a busy environment, data that is no longer needed can pile up quickly. Every byte in Google Cloud Storage (GCS) incurs a small cost, and manual cleanup is error‑prone. Object Lifecycle Management lets you define rules that automatically transition or delete objects based on age, storage class, or custom labels. This eliminates the need for cron jobs or ad‑hoc scripts, and gives you predictable cost control.

Key Take‑away

By adding a simple lifecycle rule that deletes objects older than 90 days, you can keep your bucket lean, reduce storage spend, and free up time for other tasks.

Understanding Lifecycle Rules

What a Rule Looks Like

A lifecycle rule is a JSON object that lists one or more actions to perform when an object meets a set of conditions. The most common action is Delete, but you can also SetStorageClass to move data to cheaper tiers.

Evaluation Order

• Rules are applied at the bucket level.
• They are evaluated in the order they appear in the JSON file.
• The first rule that matches an object stops further evaluation for that object.

When the Rule Fires

• GCS evaluates rules once per day.
• Deletion is eventually consistent; an object may be removed up to a few minutes after the age threshold is met.
• Objects already in Coldline or Archive storage classes are not affected unless a transition rule precedes the delete action.

Step‑by‑Step: Delete Objects Older Than 90 Days

1. Create a Bucket (if you don’t already have one)

gsutil mb gs://my-test-bucket-llm/

Run this from a shell with the gcloud SDK installed and authenticated with a user or service account that has storage.buckets.create permission.

2. Write the Lifecycle Configuration

Create a file named lifecycle.json with the following content. Replace my-test-bucket-llm with your bucket name if different.

{
  "rule": [
    {
      "action": { "type": "Delete" },
      "condition": { "age": 90 }
    }
  ]
}

Explanation:

• age: 90 means the rule applies to objects that have existed for 90 or more days.
• Only one action is defined, so the bucket will delete matching objects.

3. Apply the Lifecycle Rule

gsutil lifecycle set lifecycle.json gs://my-test-bucket-llm/

Verify the rule was applied:

gsutil lifecycle get gs://my-test-bucket-llm/

The output should echo the JSON you just uploaded.

4. Test the Rule with a Dummy Object

Upload a test file:

echo "test data" > test.txt
gsutil cp test.txt gs://my-test-bucket-llm/

Simulate the age condition by setting a negative age in the rule (e.g., "age": 0) and re‑applying:

{
  "rule": [
    {
      "action": { "type": "Delete" },
      "condition": { "age": 0 }
    }
  ]
}

gsutil lifecycle set lifecycle.json gs://my-test-bucket-llm/

Run gsutil ls gs://my-test-bucket-llm/ to confirm the object disappears. Remember that real 90‑day deletions will happen the next day after the object reaches the threshold.

Trade‑offs and Limitations

• Daily Evaluation: If you need near‑real‑time deletion, consider a Cloud Function triggered by OBJECT_FINALIZE events instead.
• Eventual Consistency: Deletion may take a few minutes; do not rely on immediate removal for compliance.
• Storage Class Interaction: Objects already in Coldline or Archive are not automatically deleted. Add a SetStorageClass transition rule before the delete action if you want them removed after a certain age.
• Permissions: The account running gsutil lifecycle set must have storage.buckets.update permission.

Actionable Checklist

1. Ensure you have the gcloud SDK and gsutil installed.
2. Verify you have the storage.buckets.update permission for the target bucket.
3. Create or identify the bucket where the rule should apply.
4. Draft the lifecycle.json file with the desired age threshold.
5. Apply the rule with gsutil lifecycle set.
6. Use gsutil lifecycle get to confirm the rule is in place.
7. Optionally run a quick test with a negative age to see the rule in action.
8. Monitor the bucket over the next 24‑48 hours to ensure objects older than 90 days are deleted.
9. Adjust the rule if you need to transition storage classes or add more granular conditions.

Wrap‑Up

Object Lifecycle Management is a low‑maintenance, cost‑effective way to keep GCS buckets tidy. By defining a straightforward delete rule for objects older than 90 days, you let Google Cloud do the housekeeping for you. Just remember the daily evaluation cadence and eventual consistency behavior, and you’ll have a dependable cleanup strategy in place.