Managing Terraform State with Remote Backends and State Locking

The problem with local state

When Terraform runs locally, it writes the state file to disk. In a team setting, two engineers can run terraform apply at the same time, each reading the same stale state and trying to write conflicting updates. The result is a corrupted state file or lost resources.

Thesis: a remote backend with locking gives a single source of truth

Storing the state file in a shared, durable location and using a lock mechanism ensures only one Terraform process can modify the state at any moment.

Configuring an S3 backend with DynamoDB locking

Add a backend block to your root module:

terraform {
  backend "s3" {
    bucket         = "my-company-tfstate"
    key            = "prod/network.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "tf-locks"
  }
}

Before running terraform init, make sure the IAM role or user has:

• s3:GetObject, s3:PutObject, s3:DeleteObject on the bucket
• dynamodb:GetItem, dynamodb:PutItem, dynamodb:DeleteItem on the lock table

Run the initialization from your workstation:

# In the directory containing main.tf
terraform init -migrate-state

Terraform will detect the existing local state and prompt to copy it to the remote bucket. Confirm the migration; you should see a message like "Successfully migrated state to s3://my-company-tfstate/prod/network.tfstate".

Observing the lock in action

Open two terminal windows. In the first, start a long‑running plan (e.g., terraform apply -auto-approve on a large module). While it holds the lock, switch to the second window and run terraform apply. The second command will block with an error similar to:

Error acquiring the state lock: LockInfo.ID: ... Is it held by another process?

Once the first apply finishes, the lock is released automatically and the second command can proceed.

Trade‑offs and limitations

• Operational overhead: you must provision and secure an S3 bucket and a DynamoDB table.
• Cost: storage is cheap, but DynamoDB read/write capacity (or on‑demand) adds a recurring fee.
• Lock hanging: if a Terraform process is killed (Ctrl‑C) while holding the lock, the lock entry can remain. You can clear it with terraform force-unlock LOCK_ID after confirming no other process is running.
• Secrets in state: even with server‑side encryption, the state file is decrypted in memory. Restrict bucket access and consider encrypting sensitive values with terraform sensitive marking or external secret management.

Actionable closing

1. Create an S3 bucket with versioning enabled and server‑side encryption (SSE‑S3 or SSE‑KMS).
2. Create a DynamoDB table with a single primary key LockID (string).
3. Add the backend block shown above, adjust bucket, key, region, and table name.
4. Run terraform init -migrate-state and verify the migration prompt appears.
5. After a successful apply, check the bucket for the .tfstate file and the DynamoDB table for a lock item (should be absent when idle).
6. If you ever see a stale lock, run terraform force-unlock <LOCK_ID> only after confirming no Terraform process is active.

Following these steps gives your team a reliable, centrally stored state with automatic locking, eliminating the most common source of Terraform‑related conflicts.