ekg and prometheus-haskell: ensuring metric compatibility for Prometheus alerting

ekg and prometheus-haskell do not share a registry. Metrics created with System.Metrics live in ekg's internal store and will never appear on a prometheus-haskell /metrics endpoint by themselves. The working pattern is a bridge that samples ekg and mirrors values into a prometheus-haskell registry — either on a timer or lazily at scrape time.

What is confirmed vs. assumed

Confirmed behavior: ekg keeps its own store; System.Metrics.sampleAll returns every metric name and current value; prometheus-haskell exposes only what is explicitly registered in its Registry. Assumed (verify against your pinned versions): exact module paths and helper functions differ between releases of both libraries, so treat the sketch below as architecture, not copy-paste code.

Steps to align names without duplication

1. Serve exactly one endpoint. Do not run ekg's built-in HTTP server (or firewall it off). Prometheus scrapes only the prometheus-haskell endpoint, so there is no double exposition.
2. Sanitize names once, at startup. ekg accepts arbitrary text names; Prometheus requires [a-zA-Z_:][a-zA-Z0-9_:]*. Map dots and dashes to underscores (e.g. app.requests.total → app_requests_total), then enumerate all ekg metrics via sampleAll and assert the mapping is injective — two distinct ekg names collapsing to one Prometheus name is a real collision risk.
3. Pre-register mirrored metrics. For each sanitized ekg name, register a corresponding counter or gauge in the prometheus-haskell registry at startup. At each interval (or in a scrape callback), call sampleAll and push values into the registered metrics. This avoids any per-update registration cost.

-- sketch, not compilable as-is
bridge store registry = forever $ do
  sample <- sampleAll store
  forM_ (HashMap.toList sample) $ \(name, val) ->
    updateMirrored registry (sanitize name) val
  threadDelay 5000000  -- 5s, well under a typical 15s scrape interval

Labels: preserving ekg dimensions

ekg has no first-class label concept — dimensions are usually baked into the name (db.pool.read.queries). To make Prometheus alert rules reference those dimensions, your bridge must parse the ekg name into (base name, label set) and register labeled metrics explicitly:

-- "db.pool.read.queries" ->
--   db_pool_queries_total{pool="read"}

Keep the parsing deterministic and total: any ekg name that fails to parse should fall back to a sanitized unlabeled metric rather than being dropped, or alerts will silently see nothing. Write the alert rule against the labeled form and trigger it in a test environment before relying on it.

Histograms: a caution

ekg histograms and Prometheus histograms are not the same abstraction — bucket layout and quantile semantics differ. Copying raw values naively produces misleading histogram_quantile results. Prefer re-recording observations directly into a prometheus-haskell histogram at the instrumentation site, or expose ekg distribution summaries as gauges (min/max/mean) and alert on those instead.

Overhead

ekg counter/gauge updates are cheap atomic operations; the bridge cost scales with total metric count × sampling frequency, not with the hot code path. So automatic mirroring does not add measurable latency to high-frequency instrumentation — but verify rather than assume: benchmark the hot path with and without ekg instrumentation, and time one sampleAll pass at your production metric count. If you have tens of thousands of metrics, lower the sampling frequency rather than the scrape fidelity.

Verification checklist

• Create one ekg counter, run the bridge, curl /metrics, confirm the sanitized name appears exactly once.
• Enumerate all ekg names and check the sanitizer output for duplicates.
• Fire the alert condition in staging and confirm the rule evaluates against the labeled metric.

One detail that would change this advice: if your ekg metric set is dynamic (names appear at runtime), the pre-registration step needs a reconciliation loop — say so and the design shifts accordingly.