Dropwizard 4.x health check flapping and metric cardinality overwhelm Alertmanager rule evaluation

Symptom

Production services running Dropwizard 4.x generate excessive alert volume from health check state changes and high-cardinality metrics exported via Micrometer's Prometheus registry. The admin endpoint returns degraded status intermittently under load, while custom checks that call downstream dependencies cause cascading timeouts on the admin thread pool. Meanwhile, the default Micrometer registry exports every meter with request-scoped tags, inflating label cardinality and slowing Alertmanager rule evaluation.

Constraints and uncertainty

Dropwizard does not prescribe alerting thresholds or a mapping for the degraded state, leaving runbook authors to decide whether it maps to warning or critical. No built-in deduplication exists, so each poll interval with a failing check emits a new alert unless the external layer (Prometheus Alertmanager, Datadog, PagerDuty) implements suppression. The migration from Dropwizard Metrics to Micrometer requires rewriting custom MeterFilter implementations to drop noisy tags such as request.id or user.id. Synchronous health check execution blocks the admin thread pool, turning a slow downstream call into a service-wide availability signal.

Goal

Define a practical approach to tune health check thresholds, filter high-cardinality metrics at the registry level, and configure external alerting rules so that actionable incidents surface without notification noise.

What registry-level MeterFilter rules effectively drop request-scoped tags without losing service-level SLI metrics? How should the degraded health state be mapped to alert severity in runbooks to avoid ambiguous paging? What Alertmanager inhibition or grouping rules deduplicate repeated health check failures caused by transient downstream latency?