Datadog Agent and Terraform: Can Agent settings be fully controlled via Terraform variables?

Answer to the Core Question

Short answer: No. The Datadog Terraform provider can set most scalar Agent settings (API key, site, tags, resource limits) via DD_* environment variables, but it cannot fully replace the datadog.yaml file or the conf.d directory that are required for autodiscovery rules, custom checks, and advanced eBPF/system‑probe configurations. Those features still need file‑based configuration or templated mounts.

Which Agent settings are controllable via Terraform variables

• DD_API_KEY – API key for the Agent
• DD_SITE – Datadog site (e.g., datadoghq.com)
• DD_TAGS – Global tags applied to every metric
• DD_DOGSTATSD_PORT – Port for DogStatsD
• DD_PROCESS_CONFIG_ENABLED, DD_PROCESS_AGENT_ENABLED, DD_LOGS_ENABLED – Boolean flags for resource limits and features
• Other scalar flags documented as DD_* env vars in the Agent release notes.

What still requires file‑based configuration

• Autodiscovery rules (the conf.d/*-autodiscovery.yaml files)
• Custom check definitions (YAML files in conf.d)
• Advanced eBPF / system‑probe settings (the system-probe.yaml file)
• Any setting that the Agent documentation lists as “only available in datadog.yaml” or “requires a config file”

Secret Management in a Terraform‑driven workflow

Never hard‑code the API key in a datadog.yaml or commit it to state. The recommended pattern is:

1. Store the key in a secrets manager (e.g., AWS Secrets Manager, Azure Key Vault, HashiCorp Vault).
2. Mark the Terraform variable as sensitive = true.

   variable "dd_api_key" {
     type      = string
     sensitive = true
   }
   

3. Inject the key into the Agent via an environment variable in the instance creation step (user_data, cloud‑init, or container env).
4. Verify that the key is not present in the Terraform state by running terraform state show and checking the variable value is omitted.

Version Alignment

The provider’s datadog-agent module only exposes a handful of DD_* inputs. If you upgrade the Agent binary, new settings may appear that are not yet exposed by the provider. Always compare the DD_* list in the Agent release notes with the provider’s schema (terraform providers schema -json) to avoid silent mismatches.

Practical Steps for a Repeatable Dev Environment

# main.tf
module "dd_agent" {
  source  = "github.com/DataDog/terraform-datadog-aws//modules/datadog-agent"
  dd_api_key      = var.dd_api_key
  dd_site         = var.dd_site
  dd_tags         = var.dd_tags
  dd_process_enabled = true
  # No support for autodiscovery via this module
}

# user_data for EC2 (or container env for ECS)
data "template_file" "user_data" {
  template = file("user_data.sh.tpl")
  vars = {
    DD_API_KEY = var.dd_api_key
    DD_SITE    = var.dd_site
    DD_TAGS    = var.dd_tags
  }
}

resource "aws_instance" "agent" {
  ami           = var.ami
  instance_type = var.instance_type
  user_data     = data.template_file.user_data.rendered
  tags = {
    Name = "dd-agent"
  }
}

After terraform apply, run datadog-agent status on the host to confirm the effective configuration. Verify that the Agent is listening on the expected DogStatsD port and that the global tags appear in the Agent UI.

When You Need Full File‑Based Control

• Autodiscovery: Mount a conf.d directory from a ConfigMap (K8s) or from a local file provisioner.
• Custom checks: Provide a YAML file via a local_file resource and mount it into the Agent container.
• eBPF/system‑probe: Use a template_file to generate system-probe.yaml and mount it.

In short, Terraform can handle the “simple” env‑var knobs, but for the richer, structured configuration that Datadog’s Agent supports, you must still supply YAML files. The combination of Terraform variables for env vars and templated file mounts gives you a repeatable, declarative setup without manual edits.