Prometheus metric spikes during KrakenJS plugin hot-reloading

Short answer

KrakenJS core does not de-register Prometheus metrics for you. When a plugin hot-reloads, Kraken invokes the old plugin instance's destroy hook (if the plugin exports one) and then initializes the new instance — but nothing in the framework touches the prom-client registry. If @krakenjs/plugin-prometheus (or your own instrumentation) registered metrics on the default global registry and its destroy implementation does not call registry.clear() or unregisterMetric(), those series persist in process memory. The result is exactly what you're seeing: duplicate or stale series exposed on /metrics until the process restarts, and false spikes in rate()/increase() alerts because Prometheus interprets the disappearing-then-reappearing counter as a reset.

Likely explanation vs. confirmed facts

Confirmed by prom-client semantics: a Registry holds metrics until explicitly cleared; registering the same metric name twice on one registry throws a duplicate-metric error; scrapes simply read whatever the registry currently holds.

Likely, but verify against your versions: the exact reload sequencing (whether Kraken awaits an async destroy before initializing the replacement) varies across KrakenJS v2.x/v3.x and dev-tooling such as file watchers. If destroy is not awaited, there is a race window where old and new metric instances coexist — which matches transient spikes that self-resolve after one or two scrape intervals. Treat this as a hypothesis until you log the hook order in your environment.

How to confirm in your setup

1. Check which registry the plugin uses:

   grep -rn "promClient.register\|new Registry\|register.clear" node_modules/@krakenjs/plugin-prometheus/

   Registration on the shared default registry without cleanup in destroy confirms the leak.
2. Add temporary logging to the plugin's initialize and destroy hooks and trigger a reload; note whether destroy completes before the new initialize runs.
3. Diff the scrape output around a reload:

   curl -s localhost:8000/metrics | sort > before.txt
   # trigger hot-reload
   curl -s localhost:8000/metrics | sort > after.txt
   diff before.txt after.txt

   Look for duplicated HELP/TYPE lines or series that should have vanished.

Preventing the false alerts

There is no documented KrakenJS configuration flag for metric de-registration on reload — cleanup is the plugin's responsibility. Practical options, in order of preference:

• Per-instance registry: have the plugin create its own new promClient.Registry() in initialize and call registry.clear() in destroy. This isolates the metric lifecycle to the plugin lifecycle.
• Explicit unregistration: if the default registry must be shared, track every metric the plugin creates and call register.unregisterMetric(metric) (or register.removeSingleMetric(name)) in destroy.
• Scraper-side tolerance: as a stopgap, add a for: duration on alert rules so a one-scrape transient doesn't page anyone, and avoid alerting on raw counter resets (resets()) for reload-prone targets.

Note that hot-reload is normally a development-only mechanism; if these reloads happen in production, the cleaner fix is deploying new processes and letting Prometheus target discovery swap endpoints, which sidesteps in-process registry lifecycle entirely.

One detail that changes the recommendation

Which KrakenJS and @krakenjs/plugin-prometheus versions are you running, and does the plugin export a destroy hook at all? If it doesn't, no registry strategy will help — you'd need to wrap or fork the plugin to add cleanup.