Kibana rules vs connectors: where should alert throttling live to cut noise without delaying critical notifications

Answer

Place throttling primarily in the Kibana rule (action frequency and flapping detection). Use connectors only for delivery guarantees and retries; avoid adding extra rate limits on the connector or provider that would compound with rule‑side settings.

Confirmed facts

• Elastic documentation states rule‑level action frequency (especially “On status change”) and flapping detection are the primary noise‑control mechanisms.
• Connectors provide retry logic and delivery assurances but do not apply business‑logic throttling.
• Alert delay postpones the first notification; if flapping detection is active, a flapping condition during the delay window can suppress the notification entirely.
• Connector‑level rate limiting (e.g., Slack webhook batching, email digest) is opaque to Kibana; dropped deliveries appear as successful rule executions with failed connector deliveries in the rule execution history.

Likely explanation / recommendation

When both rule‑side and connector‑side throttling are active, latencies multiply (rule interval × connector batch × provider limit). To keep critical alerts immediate, set noisy rules to longer custom intervals or enable flapping detection, while critical rules stay on “On status change” with no custom interval. Keep connector throttling at its default retry behavior only.

Steps to implement

1. Edit each rule → Actions → Action frequency: choose “On status change” for critical rules; for noisy rules pick a custom interval (e.g., 5 min) or rely on flapping detection.
2. Enable Flapping detection in the rule editor (default 10 m window, 3 changes). Adjust per rule only if you have API access; otherwise accept defaults.
3. Set Alert delay only if you need a brief wait before the first notification; remember it applies before flapping evaluation.
4. Leave connector settings at default (retry on failure, no extra rate limit). If you must enforce a provider limit, add it on the connector side and test end‑to‑end latency.
5. Monitor: open Rule details → History to see execution timestamps and action status (sent/failed). Correlate with Connector → Logs (enable debug logging) to see delivery attempts and provider responses.
6. If you suspect provider‑side drops, check Slack API activity logs or email provider bounce/drop reports for timestamps matching rule executions.

Missing diagnostic detail

Do you have access to provider‑side logs (Slack webhook activity or email provider bounce reports) to verify whether a notification was dropped after Kibana reported a successful connector delivery? If not, enable connector debug logging and request those logs from your provider to close the visibility gap.