Turning Elasticsearch Data into Slack Alerts with Kibana Index Threshold Rules

Your application logs are already flowing into Elasticsearch. Someone asks the obvious question: "Can we get pinged in Slack when errors spike?" The good news is you don't need a new monitoring system for this. Kibana's built-in alerting can watch an index on a schedule and fire a notification when a threshold is crossed — no dashboards need to be open, and no extra infrastructure is required beyond what you already run.

This post walks through the practical shape of that setup: an index threshold rule over an application-logs index, a reusable Slack connector, and a message template that tells people what actually happened instead of "something fired."

Why server-side rules beat dashboard-watching

The key property of Kibana alerting is that rules run on the Kibana server on a check interval you choose — every minute, every five minutes, whatever fits. A dashboard only tells you something when a human happens to look at it. A rule evaluates continuously, which is the difference between monitoring and browsing.

An index threshold rule is the simplest useful rule type: it counts documents (or aggregates a numeric field) in an index over a time window, compares the result to a threshold, and triggers when the condition is met. "More than 50 documents with level: error in the last 5 minutes" is a one-line expression of that idea.

Connectors are configured once, reused everywhere

Actions — the "what happens when a rule fires" part — are delivered through connectors. You configure a connector once under Stack Management (Slack, email, or a generic webhook), and any number of rules can use it. This matters operationally: when a webhook URL rotates or a Slack app token changes, you update one connector instead of editing every rule.

The webhook connector is the escape hatch. If your target system has no native connector, point a webhook at its API and shape the payload yourself. A request-capture service is handy during development so you can inspect exactly what Kibana sends before pointing it at a real system.

Worked example: error spike to Slack

Assume an index pattern like app-logs-* where each document has a level field. The rule definition, conceptually:

• Index: app-logs-*
• Condition: count of documents where level: "error" is above 50
• Time window: last 5 minutes
• Check every: 1 minute
• Action: Slack connector, posting to #on-call

Action messages support mustache-style templating, so the notification can carry context instead of a bare alarm:

Error spike in {{rule.name}}: {{context.value}} error documents
in the last {{context.window}} (threshold: {{rule.params.threshold}}).

Exact variable names vary between Kibana versions and rule types, so treat this as a shape, not gospel. The reliable way to confirm which variables your version exposes is to create the rule with a webhook connector pointed at a capture endpoint, trigger it once with test data, and inspect the JSON payload. That five-minute experiment beats guessing from documentation written for a different release.

To validate the whole pipeline in a dev environment, index a batch of synthetic error documents — more than your threshold, with fresh timestamps — and watch for the rule to fire on its next check. Then index fewer than the threshold and confirm it recovers. Both directions matter; a rule that fires but never resolves will train your team to ignore the channel.

The honest trade-offs

Licensing is tiered. Basic rule types are available in the free distribution, but some connectors and advanced rule types require a paid Elastic license, and availability differs between self-managed deployments and Elastic Cloud. Before promising a Slack integration to your team, open Stack Management > Rules and Connectors in your own deployment and confirm the connector you want actually exists in your tier.

Alerting shares Kibana's fate. Rules are evaluated by Kibana's task manager. If the Kibana instance is down, overloaded, or mid-upgrade, your alerts are too. For "the site is on fire" signals, you want a monitoring path with an independent failure domain. Kibana alerting is excellent for "errors are trending up" and "this batch job stopped producing documents" — less ideal as your only pager.

Threshold tuning is your problem. A static count threshold that is right at 2 p.m. may page people every night at 3 a.m. when traffic patterns differ. Start with a threshold that clearly indicates abnormality, and tighten it based on a week or two of observed behavior rather than guessing on day one.

Where to start

Pick one noisy-but-actionable signal — error rate is the classic — and build the smallest version: one index threshold rule, one connector, one templated message. Verify it fires with synthetic data, verify it recovers, and let it run for a week before adding more rules. Alerting systems earn trust slowly and lose it fast; a single rule that is always worth reading beats ten that aren't.