Terraform Remote State with S3 and DynamoDB: Team Collaboration Made Reliable

The Problem: Concurrent Terraform Changes Break Infrastructure

When multiple engineers run terraform apply simultaneously, Terraform can't track who's modifying what. This leads to state corruption, resource conflicts, and infrastructure drift. Local state files don't scale beyond a single developer's laptop.

The Solution: S3 Backend with DynamoDB Locking

AWS S3 stores the Terraform state file for durability and shared access. DynamoDB provides advisory locking to prevent concurrent modifications. This combination is Terraform's recommended approach for team collaboration.

Required Infrastructure Setup

Before configuring Terraform, create two AWS resources:

1. S3 bucket with versioning and server-side encryption enabled
2. DynamoDB table with primary key LockID using on-demand capacity

Run these commands in your terminal with AWS CLI configured:

# Create encrypted S3 bucket with versioning
aws s3api create-bucket --bucket my-terraform-state-bucket --region us-west-2
aws s3api put-bucket-versioning --bucket my-terraform-state-bucket --versioning-configuration Status=Enabled
aws s3api put-bucket-encryption --bucket my-terraform-state-bucket --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}'

# Create DynamoDB table for state locking
aws dynamodb create-table --table-name terraform-state-lock --attribute-definitions AttributeName=LockID,AttributeType=S --key-schema AttributeName=LockID,KeyType=HASH --billing-mode PAY_PER_REQUEST

Configuring the Backend

Add this configuration block to your Terraform code:

terraform {
  backend "s3" {
    bucket         = "my-terraform-state-bucket"
    key            = "production/terraform.tfstate"
    region         = "us-west-2"
    dynamodb_table = "terraform-state-lock"
    encrypt        = true
  }
}

The key parameter organizes state files by environment. Use different keys like development/terraform.tfstate or staging/terraform.tfstate for multiple workspaces.

Migrating Existing State

If you have local state, run terraform init and answer y to migrate. Terraform will upload your current state to S3:

terraform init

Enter a value: y  # When prompted to migrate local state

After initialization, verify the state file exists in S3:

aws s3 ls s3://my-terraform-state-bucket/production/terraform.tfstate

Verifying State Locking Works

Test locking by opening two terminal sessions. In the first, run:

terraform plan

While that plan is running, in a second terminal run:

terraform apply

The second command should fail with a message like Error acquiring the state lock, confirming DynamoDB locking prevents concurrent modifications.

Trade-offs and Limitations

Cost considerations: DynamoDB on-demand billing charges per read/write operation. For teams running plans multiple times daily, expect $1-3 monthly. S3 storage costs are negligible for small state files.

Sensitivity to network issues: If a Terraform process crashes while holding a lock, DynamoDB's default 30-minute expiration ensures eventual release. However, long-running operations may require manual intervention.

State file size: Large state files slow down operations. Consider using -state-out flag for partial states or state splitting strategies for very large infrastructures.

Actionable Next Steps

1. Create the S3 bucket and DynamoDB table using the AWS CLI commands above
2. Add the backend configuration block to your Terraform code
3. Run terraform init and migrate existing state if prompted
4. Test concurrent access to verify locking behavior
5. Set up IAM policies restricting access to the state bucket and DynamoDB table

This setup provides reliable, shared Terraform state for teams of any size. The combination of S3 durability and DynamoDB locking eliminates the most common causes of Terraform state corruption in collaborative environments.