Configure Terraform Remote State with AWS S3 and DynamoDB Locking

Desired Outcome

Store Terraform state in a centralized, durable location so that multiple team members can run terraform commands concurrently without overwriting each other’s state files. The state file resides in an Amazon S3 bucket with versioning and server‑side encryption, while an Amazon DynamoDB table provides a lock to prevent simultaneous writes.

Prerequisites

• An AWS account with permissions to create S3 buckets, enable versioning, set bucket policies, create DynamoDB tables, and put/get items.
• Terraform CLI version ≥ 1.0 installed locally.
• Existing Terraform configuration that currently uses the local backend (state stored in .tfstate files).
• AWS CLI configured with a profile or environment variables that have the necessary IAM permissions.

Procedure

1. Create the S3 bucket

   Run the following commands, replacing placeholders with your own values. Ensure you are in a shell where the AWS CLI is authenticated.

   # Replace  with a globally unique name
   # Replace  with your preferred AWS region, e.g., us-east-1
   aws s3api create-bucket \
       --bucket  \
       --region  \
       --create-bucket-configuration LocationConstraint=
   
   # Enable versioning (required for state recovery)
   aws s3api put-bucket-versioning \
       --bucket  \
       --versioning-configuration Status=Enabled
   
   # Enable server‑side encryption with AES256
   aws s3api put-bucket-encryption \
       --bucket  \
       --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}'
   
   # (Optional) Restrict bucket access to specific IAM roles/users
   # Example policy JSON saved to bucket-policy.json
   cat > bucket-policy.json <<'EOF'
   {
     "Version":"2012-10-17",
     "Statement":[
       {
         "Effect":"Allow",
         "Principal":{"AWS":"arn:aws:iam:::role/"},
         "Action":["s3:GetObject","s3:PutObject","s3:DeleteObject"],
         "Resource":"arn:aws:s3:::/*"
       }
     ]
   }
   EOF
   aws s3api put-bucket-policy --bucket  --policy file://bucket-policy.json
   

2. Create the DynamoDB lock table

   The table must have a partition key named LockID (type String). No other attributes are required for Terraform locking.

   # Replace  with a name of your choice
   aws dynamodb create-table \
       --table-name  \
       --attribute-definitions AttributeName=LockID,AttributeType=S \
       --key-schema AttributeName=LockID,KeyType=HASH \
       --billing-mode PAY_PER_REQUEST
   

3. Add the backend block to your Terraform configuration

   In the root module of your Terraform code, create or edit a file named backend.tf (or any .tf file) and include the following block. Replace the placeholders with the names you used above.

   terraform {
     backend "s3" {
       bucket         = ""
       key            = ""   # e.g., prod/network/terraform.tfstate
       region         = ""
       encrypt        = true
       dynamodb_table = ""
     }
   }
   

4. Initialize and migrate existing state

   Run terraform init with the -migrate-state flag. This copies the current local .tfstate file to the newly configured S3 backend and removes the local copy.

   terraform init -migrate-state
   

   You should see output indicating that the backend was initialized and that the state was migrated. No manual file copying is required.

5. Verify the lock mechanism

   After a successful terraform init, run a command that acquires a lock, such as terraform plan. In a separate terminal session, attempt to run another Terraform command (e.g., terraform apply) targeting the same workspace. The second command should pause and report that it is waiting for the state lock to be released.

   # Terminal 1
   terraform plan
   
   # Terminal 2 (while the first command is still running)
   terraform apply
   # Expected output: "Acquiring state lock. This may take a few moments..."
   

Expected Checks

• Backend initialization: The output of terraform init contains a line similar to Backend configuration changed! and Successfully configured the backend "s3".
• State file location: After initialization, running terraform state list should work without a local .tfstate file present.
• Lock entry: You can inspect the DynamoDB table to see a lock item while a Terraform operation is running:

  aws dynamodb scan \
      --table-name  \
      --filter-expression "#k = :v" \
      --expression-attribute-names '{"#k":"LockID"}' \
      --expression-attribute-values '{":v":{"S":"/"}}'
  

  The item should contain attributes LockID, Info (JSON with Terraform metadata), and TTL (epoch time when the lock expires).
• Concurrency protection: A second Terraform command initiated while the first holds the lock should not proceed until the lock is released or times out.

Recovery / Rollback Options

If the migration to the remote backend fails or you need to revert to local state for troubleshooting:

1. Abort the migration: If terraform init -migrate-state returns an error, the local .tfstate file remains unchanged. Simply remove the backend block from your configuration and run terraform init again to return to the local backend.
2. Manual rollback: Should the state have been copied to S3 but you wish to go back locally, you can pull the state from S3 and then remove the backend block:

   # Pull the remote state to a local file (optional backup)
   terraform state pull > local.tfstate.backup
   
   # Remove the backend block from your .tf files
   # Then re‑initialize without migrating
   terraform init
   
   # Push the backed‑up state back as the new local state
   mv local.tfstate.backup .tfstate
   

   After this, Terraform will operate against the local file again. Remember to delete the remote state object from S3 if you no longer need it to avoid confusion.

3. Accidental lock leakage: If a Terraform process crashes and does not release the lock, the lock item in DynamoDB will expire after the TTL (default 300 seconds). You can manually delete the lock item to unblock immediately:

   aws dynamodb delete-item \
       --table-name  \
       --key '{"LockID":{"S":"/"}}'
   

   Use this only when you are certain no Terraform process is still running.