Choosing a Dynatrace OneAgent Deployment Mode on Kubernetes

Decision and Constraints

When adding Dynatrace observability to a Kubernetes cluster you must pick a OneAgent deployment mode that matches your visibility needs, host access privileges, and overhead budget. The three supported modes are:

1. Full‑stack DaemonSet – OneAgent runs on every node and captures host, Kubernetes, container, process and network data.
2. Application‑only injection – OneAgent is injected into selected workloads only; no host‑level agent is required.
3. Infrastructure‑only monitoring – OneAgent monitors nodes and cluster resources without code‑level tracing.

Constraints to consider:

• Do you have privileged access to the nodes?
• Is full process‑group detection and code‑level tracing required?
• What is the acceptable CPU/memory overhead per node?
• Are there language runtimes or network policies that could block the agent?

Comparison of Supported Options

Feature	Full‑stack DaemonSet	Application‑only Injection	Infrastructure‑only
Host‑level metrics	Yes	No (unless paired with infra)	Yes
Container & Kubernetes metadata	Yes	Partial (via injected sidecar)	Yes
Process‑group detection	Yes	Limited to injected processes	No
Code‑level traces & Davis AI	Yes	Yes (for injected apps)	No
Real User Monitoring (RUM)	Yes	Yes (if web app injected)	No
Typical overhead per node	Higher (scales with process count)	Lower (only on injected pods)	Lowest
Requires privileged node access	Yes	No (needs Operator/admission controller)	Yes (for node daemonset)

Trade‑offs

Full‑stack DaemonSet gives the richest context – host, network, and process‑group data – which enables Davis AI root‑cause analysis and service‑level dependency maps. The penalty is higher CPU and memory consumption, especially on nodes running many short‑lived containers. If host access is restricted (e.g., managed node services where you cannot run privileged daemonsets) or you only need application metrics, application‑only injection avoids privileged agents but loses host‑level visibility unless you also run an infrastructure‑only DaemonSet on the same cluster. Infrastructure‑only monitoring is the lightest option, suitable for capacity planning and alerting on node health, but it provides no code‑level traces, RUM, or service maps.

Running both full‑stack and application‑only agents on the same workload can duplicate metrics and increase overhead; avoid this unless you have a specific reason to separate host and app data.

Concrete Implementation: Full‑stack DaemonSet via Dynatrace Operator

The following steps show how to deploy the full‑stack mode using the official Dynatrace Operator. Adjust placeholders for your environment.

Prerequisites

• kubectl configured with cluster‑admin permissions.
• A Dynatrace environment token with ingest and metrics ingest scopes.
• Kubernetes version 1.21‑1.27 (check the Dynatrace compatibility matrix for your OneAgent release).
• Nodes running a supported Linux distribution (e.g., Ubuntu 20.04, RHEL 8, Amazon Linux 2).

Step 1: Install the Operator

# Run on your admin workstation
kubectl create namespace dynatrace
kubectl apply -f https://github.com/Dynatrace/dynatrace-operator/releases/latest/download/kubernetes.yaml -n dynatrace

Verify the Operator pods are running:

kubectl get pods -n dynatrace -l app.kubernetes.io/name=dynatrace-operator

Step 2: Create a Secret with your Environment ID and Token

# Replace  and  with your values
kubectl -n dynatrace create secret generic dynatrace \
  --from-literal=environmentId= \
  --from-literal=apiToken=

Step 3: Deploy the DaemonSet Custom Resource

cat <<EOF | kubectl apply -f -
apiVersion: dynatrace.com/v1beta1
kind: DynaKube
metadata:
  name: fullstack
  namespace: dynatrace
spec:
  apiUrl: https://.live.dynatrace.com/api
  skipCertCheck: false
  nodeAgent:
    resources:
      limits:
        cpu: 500m
        memory: 512Mi
      requests:
        cpu: 200m
        memory: 256Mi
  # Enable full‑stack monitoring (default)
EOF

This manifest requests roughly 200 mCPU and 256 MiB RAM per node; adjust based on your observed load.

Step 4: Verify Rollout

Check that a OneAgent pod exists on each node:

kubectl get pods -n dynatrace -l app.kubernetes.io/name=oneagent

You should see one pod per node (e.g., oneagent-abcde).

Next, inspect the OneAgent status via the Operator:

kubectl -n dynatrace describe dakube fullstack

Look for NodeAgentStatus: Ready on each node.

Step 5: Validate Visibility in Dynatrace UI

1. Log in to your Dynatrace environment. 2. Navigate to Kubernetes → Clusters and confirm the cluster name appears. 3. Open a node’s details page; you should see host CPU, memory, disk, and network metrics. 4. To test code‑level tracing, deploy a simple instrumented application (e.g., a Java Spring Boot pod) and send a request:

kubectl run test-client --rm -i --tty --image=curlimages/curl -- sh
# inside the pod
curl http://..svc.cluster.local

After the request, return to Dynatrace and verify:

• A Process group for the application appears.
• A Service is detected.
• A Distributed trace

Limitations and Practical Checks

Even with a successful rollout, consider these limits:

• Kernel versions older than 3.10 may block eBPF‑based networking features; the Operator will fall back to traditional mode with reduced network visibility.
• If a node runs a container‑runtime sandbox (e.g., Kata Containers) the OneAgent may not see processes inside the sandbox.
• Network policies that deny outbound traffic to *.live.dynatrace.com on port 443 will prevent data ingestion; test connectivity from a node with:

curl -v https://.live.dynatrace.com/api/v2/ingest

To check ongoing overhead, collect node metrics before and after enabling the DaemonSet:

# Before (baseline)
kubelet --v=2 --system-reserved=cpu=200m,memory=256Mi ...
# After deployment, observe the same node via kubectl top node
kubectl top node 

If CPU or memory consistently exceeds your budget, reduce the resources.requests and limits in the DynaKube spec and redeploy.

Conclusion

Choose full‑stack DaemonSet when you need host, network, and process‑group context and can tolerate moderate overhead. Opt for application‑only injection when host access is unavailable and you only require application metrics, remembering to pair it with infrastructure‑only monitoring if you still need node‑level data. Infrastructure‑only alone is best for lightweight capacity monitoring. Validate each choice by confirming the expected metrics appear in Dynatrace and measuring resource usage on a representative node.