Configure Dynatrace OneAgent on Kubernetes with the Dynatrace Operator for Full-Stack Monitoring

Problem and Takeaway

Instrumenting every container in a Kubernetes cluster manually is impractical. The Dynatrace Operator automates OneAgent injection so that pods in selected namespaces start sending metrics, traces, and logs to your Dynatrace tenant without code changes. This guide walks through the complete setup: creating the required secret, installing the Operator via Helm, defining a DynaKube custom resource, enabling namespace injection, and verifying the result.

Prerequisites

• A Dynatrace SaaS or Managed tenant with an active license.
• Kubernetes cluster v1.21 or later.
• kubectl access with permissions to create namespaces, secrets, CustomResourceDefinitions, and DaemonSets.
• A Dynatrace API token that has InstallerDownload and ReadConfig scopes (create it in Settings → Integration → Dynatrace API).
• Helm v3 installed locally.

Add the official Helm repository once:

helm repo add dynatrace https://raw.githubusercontent.com/Dynatrace/dynatrace-oneagent-operator/master/deploy/helm

helm repo update

Step 1: Create the Authentication Secret

Store the tenant URL and API token in a secret that the Operator will read. Replace the placeholders with your actual values.

kubectl create namespace dynatrace

kubectl -n dynatrace create secret generic dynakube-secret \
  --from-literal=apiToken="YOUR_API_TOKEN" \
  --from-literal=apiUrl="https://YOUR_ENVIRONMENT_ID.live.dynatrace.com/api"

Permissions: The secret must exist in the same namespace where you install the Operator (here dynatrace). The token should be rotated periodically; avoid embedding it in Git.

Step 2: Install the Dynatrace Operator

Choose a chart version that matches the OneAgent version you want. The Operator and OneAgent versions are coupled; consult the compatibility matrix before upgrading.

helm install dynatrace-operator dynatrace/dynatrace-operator \
  --namespace dynatrace \
  --version 0.14.0 \
  --set "operator.image=registry.dynatrace.com/dynatrace-operator"

Wait for the Operator deployment to become ready:

kubectl -n dynatrace rollout status deployment/dynatrace-operator

Step 3: Define the DynaKube Custom Resource

The DynaKube CR tells the Operator which features to enable and how to connect to the tenant. Apply the following manifest (adjust apiUrl if you use a Managed cluster with a different endpoint).

cat <<'EOF' | kubectl apply -f -
apiVersion: dynatrace.com/v1beta4
kind: DynaKube
metadata:
  name: dynakube
  namespace: dynatrace
spec:
  apiUrl: https://YOUR_ENVIRONMENT_ID.live.dynatrace.com/api
  tokens: "dynakube-secret"
  oneagent:
    cloudNativeFullstack: {}
    # applicationMonitoring: {}  # alternative: code-level only
  # Enable log monitoring (requires OneAgent 1.267+)
  featureLogMonitoring: true
EOF

Mode choice: cloudNativeFullstack runs a privileged DaemonSet that injects OneAgent into every pod via an init container. applicationMonitoring uses a mutating webhook for code-level injection only (no infrastructure metrics). Pick one; they are mutually exclusive.

Step 4: Enable Auto-Injection for Target Namespaces

Annotation drives the mutating webhook. Only pods created after the annotation appears will be instrumented.

kubectl annotate namespace production \
  oneagent.dynatrace.com/inject="true" --overwrite

kubectl annotate namespace staging \
  oneagent.dynatrace.com/inject="true" --overwrite

Step 5: Restart Workloads to Trigger Injection

Existing pods do not receive the init container automatically. Roll them out:

kubectl -n production rollout restart deployment

kubectl -n staging rollout restart deployment

For StatefulSets or DaemonSets, use the same rollout restart command targeting the resource type.

Expected Checks

1. Operator and OneAgent Pods Running

kubectl -n dynatrace get pods -l app=dynatrace-operator
kubectl -n dynatrace get pods -l app=oneagent

You should see the Operator deployment and a OneAgent DaemonSet (one pod per node) in Running state.

2. Application Pods Show OneAgent Environment

Pick any pod in an annotated namespace and inspect its environment:

kubectl -n production exec  -- \
  sh -c 'cat /proc/1/environ | tr "\0" "\n" | grep ONEAGENT'

Expected output includes variables such as ONEAGENT_INSTALLER_URL, ONEAGENT_INSTALLED_VERSION, and DT_TENANT.

3. DynaKube Status Shows Healthy Phase

kubectl -n dynatrace get dynakube dynakube -o yaml | grep -A5 status:

Look for phase: Running and oneagent: running under status.

4. Dynatrace UI Verification

1. Navigate to Kubernetes → Clusters. Your cluster should appear with a green OneAgent status.
2. Open Hosts and filter by the cluster name; each node should show a monitored host.
3. Under Services, confirm that your instrumented services appear with request throughput and response time.
4. Open Distributed traces and search for a trace ID that spans multiple services (e.g., frontend → API → database) to validate end-to-end tracing.
5. If featureLogMonitoring is enabled, check Logs for log lines from the instrumented containers.

Limitations and Security Considerations

• Privileged DaemonSet: cloudNativeFullstack requires privileged: true and host filesystem access. Clusters with PodSecurityPolicies, Kyverno, or OPA Gatekeeper may block it. Use applicationMonitoring if privileged workloads are prohibited.
• Version coupling: Operator, OneAgent, and Kubernetes versions must align. Upgrading one without checking the matrix can break injection.
• Token scope: The API token used here has installer download and config read permissions. Do not reuse a token with broader scopes (e.g., WriteConfig).
• Existing pods: Annotation only affects new pods. A rollout restart is mandatory for workloads deployed before the annotation.
• Log monitoring minimum: Requires OneAgent 1.267+ and the featureLogMonitoring: true flag in the DynaKube CR.

Recovery and Rollback

If injection fails or you need to remove the integration:

1. Check Operator logs for errors:

   kubectl -n dynatrace logs -l app=dynatrace-operator --tail=100

2. Inspect DynaKube conditions:

   kubectl -n dynatrace describe dynakube dynakube

3. Common fixes:
   • Verify the API token has the correct scopes and has not expired.
   • Confirm egress from cluster nodes to *.live.dynatrace.com (or your Managed endpoint) on port 443.
   • Ensure the OneAgent version pulled by the Operator is compatible with your Kubernetes version.
4. To uninstall completely (this changes cluster state):

   kubectl -n dynatrace delete dynakube dynakube
   helm -n dynatrace uninstall dynatrace-operator
   kubectl delete namespace dynatrace

Quick Verification Checklist