Deploying Dynatrace OneAgent Auto‑Injection in Kubernetes via a DaemonSet

Requirements

Deploy the Dynatrace OneAgent as a DaemonSet so every node runs the agent, enabling automatic instrumentation of Java processes without code changes.

Minimal Suitable Design

One DaemonSet using the official OneAgent image, a Kubernetes secret that holds the API token, and an environment variable DT_TENANT pointing to your Dynatrace tenant. No side‑car containers are required.

Trust and Data Boundaries

The OneAgent container runs in privileged mode, giving it access to the node’s filesystem and PID namespace. It gathers metrics and traces locally and sends the telemetry over TLS to the Dynatrace SaaS endpoint. No application data leaves the cluster except the encrypted telemetry payload.

Operational Checks

1. Verify the DaemonSet pods are Ready: kubectl get daemonset -n dynatrace oneagent and ensure READY shows 1/1 for each node.
2. Inspect the agent logs for the startup message: kubectl logs -n dynatrace <oneagent-pod> | grep 'Agent started successfully'.
3. In the Dynatrace UI, go to Infrastructure → Hosts and confirm each node appears with the OneAgent version; then open a Java process to see process‑level metrics and traces.

Failure Modes and Design‑Change Conditions

• Network loss: If a node cannot reach the Dynatrace endpoint, the agent buffers telemetry locally. When the buffer reaches its limit (default ~50 MB) newer data is dropped. Persistent loss requires reviewing network policies or deploying an ActiveGate to act as a local collector.
• Buffer overflow: Prolonged outages can fill the buffer, causing loss of recent telemetry. Monitor the built‑in metric dt.oneagent.buffer.usage and set an alert on a threshold (e.g., 80 %).
• OneAgent version upgrade: Moving to a new major version may change the API or required privileges. Update the DaemonSet image tag, test in a staging namespace, and verify that Java processes still appear with process‑level monitoring before rolling out to production.

Example Configuration

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: oneagent
  namespace: dynatrace
  labels:
    app: oneagent
spec:
  selector:
    matchLabels:
      app: oneagent
  template:
    metadata:
      labels:
        app: oneagent
    spec:
      serviceAccountName: dynatrace-oneagent
      containers:
      - name: oneagent
        image: dynatrace/oneagent:latest
        securityContext:
          privileged: true
        env:
        - name: DT_TENANT
          value: "your-tenant-id.live.dynatrace.com"
        - name: DT_API_TOKEN
          valueFrom:
            secretKeyRef:
              name: dynatrace-secret
              key: api-token
        volumeMounts:
        - name: host-root
          mountPath: /host/root
          readOnly: true
      volumes:
      - name: host-root
        hostPath:
          path: /

Apply the manifest with kubectl apply -f oneagent-daemonset.yaml (requires cluster‑admin rights to create the privileged pod and the service account).

Verification Steps

• Check pod readiness: kubectl get ds -n dynatrace oneagent -o wide
• View logs for startup confirmation: kubectl logs -n dynatrace $(kubectl get pod -n dynatrace -l app=oneagent -o jsonpath='{.items[0].metadata.name}') | grep 'Agent started successfully'
• In Dynatrace UI, verify each node under Infrastructure → Hosts and inspect a Java process for process‑level metrics.

Limitations and Practical Checks

Running the OneAgent in privileged mode grants broad node access; ensure your cluster’s security policies (PSP/OPA) allow this. If privileged containers are forbidden, consider using a DaemonSet that runs the agent in unprivileged mode with the necessary hostPath mounts and capabilities, but note that some low‑level instrumentation may be unavailable.

Monitor buffer usage via the metric dt.oneagent.buffer.usage (available in the Dynatrace UI under Custom Devices → OneAgent) and set an alert to avoid silent data loss.