ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
DevOps & observability · #datadog
Data Retention Trade-offs When managing high-volume log streams in Datadog, organizations must balance the need for immediate observability with storage cost constraints. The platform offers two primary paths for long-term data: maintaining a continuous index or offloading raw logs to external cloud storage (S3, Azure Blob Storage, or GCS). Operational Const
When building shared monitoring dashboards in Datadog, teams must decide between enforcing the platform's built-in high-contrast mode or preserving customizable light and dark themes with manually adjusted widget colors. High-contrast mode applies WCAG AA contrast ratios across all UI elements, but it overrides any user-selected dashboard color palettes. Cus
Goal: Revert a Datadog monitor, logs pipeline, or APM service definition to a known‑good schema version after a change that was applied successfully but later deemed undesirable. Constraint: When a schema change succeeds, Datadog stores the new version as the active configuration and retains the previous version only as an immutable snapshot; there is no doc
Symptom Datadog API keys are created without an expiration date and remain valid until explicitly revoked or deleted. This behavior persists across all tiers and custom RBAC roles, leaving long‑lived credentials that can be used beyond their intended lifecycle. Goal Determine whether Datadog plans to introduce an automated expiration or rotation workflow tha
Integration boundary Django REST Framework emits structured exceptions and logs that monitoring backends—Sentry, Prometheus, Datadog—ingest for alerting. The integration point is DRF's DEFAULT_EXCEPTION_HANDLER and custom throttle classes, which enrich or convert validation failures ( serializer.is_valid() returning False ) into events or metrics before they
Goal: determine whether Datadog APM retains a tracer-marked priority-1 trace when the Agent APM intake rate limit has already been reached. Tracer sampling priority and Agent intake rate limiting are documented as separate controls, and public documentation does not clearly state which takes precedence when both apply. Uncertainty: the outcome may depend on
The goal is to maintain continuous collection of container metrics and logs from a Datadog Agent when moving from a local development cluster to a production Kubernetes environment that enforces strict security policies. In local setups the Agent runs with privileged=true and mounts the node’s /var/run/docker.sock via a hostPath, allowing it to observe Docke
Datadog APM uses ingest-side sampling to manage the volume of trace data. While this ensures platform stability, it introduces uncertainty when attempting to isolate rare latency spikes that occur exclusively during periods of high concurrent request volume. The primary goal is to ensure that tail latency outliers—specifically those caused by race conditions
Datadog monitors configured for Multi Alerts evaluate thresholds against each unique combination of tags independently. While this allows for granular visibility, high-cardinality tags can result in a significant volume of individual notifications when multiple groups cross the threshold simultaneously. When utilizing complex boolean logic in monitor queries
prometheus
kubernetes