Dynatrace Log Ingestion Filters: Balancing Coverage and Cost for Low-Traffic Workloads

To balance cost and visibility in low-traffic Dynatrace environments, you must filter data at the earliest ingestion point—specifically OneAgent log source rules or the log ingestion pipeline—to prevent low-value data from ever incurring DDU (Data Dm) or storage costs.

High-Value Ingestion Filtering Strategies

For low-traffic workloads, the best cost-benefit ratio is achieved by excluding high-volume, low-signal events. Instead of broad ingestion, focus on these specific filters:

• Severity-Based Filtering: Configure ingestion filters to exclude DEBUG and INFO levels by default. In low-traffic environments, these often represent heartbeat noise without providing diagnostic value until an active incident occurs.
• Pattern-Based Exclusion: Use regex to filter out health checks, heartbeat signals, and routine connection polling logs that flood storage but contain no unique state changes.
• Log-Source Pruning: If a specific service produces verbose logs (e.g., framework-level traces) that are rarely queried, disable that log source within the OneAgent configuration rather than filtering it globally.

Retention and Storage Cost Management

Retention periods are a primary driver of cost. For low-traffic production, a tiered retention strategy is most effective:

1. Operational Logs: Keep ERROR and WARN logs for a standard period (e.g., 30 days) to facilitate post-mortem analysis.
2. Audit/Security: Move security-related logs or access audits to a lower-cost storage tier or a shorter retention period if possible, to meet compliance without paying for premium storage on transient application data.

Host-Group Tagging for Auto-Scaled Environments

To prevent idle or auto-scaled hosts from inflating licensing usage, implement a dynamic tagging strategy based on host state:

• Lifecycle Tagging: Apply tags like env: ephemeral or status:idle to auto-scaled instances. Use these tags in your ingestion filters to drop logs from hosts that are not currently processing production-level traffic.
• Resource Groups: Ensure that host groups used for temporary testing or development are excluded from the global log ingestion policy to avoid "quiet creep" in DDU consumption.

Assumptions and Uncertainty: This guide assumes the use of Dynatrace SaaS with DDU-based log storage pricing. The effectiveness of regex filters depends on the specific log format of your application; always verify that critical error patterns are not caught by aggressive exclusion.

To provide a more specific recommendation, what is the current log volume (in GB) generated by these low-traffic hosts?