Diagnosing Elasticsearch Yellow and Red Health from Unassigned Shards

The problem and what to do first

When Elasticsearch reports yellow or red cluster health, the cluster is not necessarily down, but data protection is degraded. Yellow means every primary shard is assigned to a node, but at least one replica shard has nowhere to run. Red means at least one primary shard is unassigned, so some data is unavailable for reads and writes.

Useful takeaway: do not restart nodes or force allocation until you know why the shard is unassigned. The allocation explain API tells you the exact decision Elasticsearch made, so start there instead of guessing.

Recognize the condition

Common symptoms:

• GET /_cluster/health returns status yellow or red
• GET /_cat/shards shows shard state UNASSIGNED
• Searches return partial results or fail with shard failures when the cluster is red
• Index writes are blocked when the flood-stage disk watermark is tripped

Common causes and where to look first

Observed condition	Likely cause	First command
Yellow only	Replica cannot find an eligible node	GET /_cluster/health?level=shards
Red	Primary shard unassigned after node loss or corruption	GET /_cat/shards?v&h=index,shard,prirep,state,node,unassigned.reason
Disk pressure	Node crossed low, high, or flood-stage watermark	GET /_cat/allocation?v
Awareness mismatch	Zone or rack rule has no matching node	GET /_cat/nodeattrs?v
Master quorum lost	Too few master-eligible nodes reachable	Check discovery settings and logs

Ordered diagnostic checks

1. Identify the unassigned shards. Run in Kibana Dev Tools or via curl with cluster:monitor/health permission:

   GET /_cluster/health?level=shards

   Look for indices where unassigned_shards is greater than zero. Then list the shards:

   GET /_cat/shards?v&h=index,shard,prirep,state,node,unassigned.reason

   Note whether the unassigned shard is a primary (p) or replica (r). Red requires a primary; yellow is usually a replica.
2. Ask Elasticsearch why it will not allocate the shard. Run with cluster:admin/allocation/explain permission or equivalent:

   GET /_cluster/allocation/explain?index=&shard=&primary=

   Replace <index>, <shard-id>, and <true|false>. The response includes a decision such as "no" and an explanation like the node does not match the required awareness attributes or disk usage exceeded the flood-stage watermark. This names the blocker directly.
3. Check disk watermarks. Run:

   GET /_cat/allocation?v

   and:

   GET /_cluster/settings?include_defaults=true

   Look for cluster.routing.allocation.disk.watermark.low (default 85%), high (default 90%), and flood_stage (default 95%). Above 85%, Elasticsearch stops allocating new replicas to that node. Above 90%, it moves existing shards away. Above 95%, indices become read-only and block writes.
4. Verify allocation awareness attributes. If the explanation mentions awareness, run:

   GET /_cat/nodeattrs?v

   Confirm that node.attr.zone or other attributes match cluster.routing.allocation.awareness.attributes in elasticsearch.yml. If only one node has the required attribute, a replica cannot be placed on a different node in the same zone.
5. Check master eligibility and discovery. For 7.x, confirm enough master-eligible nodes are reachable relative to discovery.zen.minimum_master_nodes. For 8.x, confirm voting-only nodes can reach a quorum. cluster.initial_master_nodes is used only at first bootstrap in 8.x; changing it later does not repair quorum.
6. Inspect logs for corruption or breaker trips. If a shard failed after a node restart, search Elasticsearch logs for failed to recover shard or corrupt index. Also run:

   GET /_nodes/stats/breaker

   A tripped circuit breaker can crash a node and leave its shards unassigned.

Fixes tied to the finding

Disk watermark exceeded

Free disk space or add data nodes. Do not simply raise the watermark to hide pressure; that risks node instability. After space is available, shards should reallocate automatically.

Allocation awareness mismatch

Add a node in the missing zone, or relax the awareness requirement if the topology no longer supports it. Avoid setting index.routing.allocation.require._name to a single node as a workaround; that creates hotspots.

Unassigned replica after node loss

If the node returns, shards usually reassign. If the node is gone permanently and replicas exist elsewhere, the cluster may recover once enough nodes are available. If no replica exists and the primary is unassigned, you must restore from a snapshot or accept data loss for that shard.

Shard corruption

Verify a snapshot exists before destructive action:

GET /_snapshot///_status

Restore to the same cluster or a separate test cluster before deleting anything. Deleting an index to clear unassigned shards destroys data unless a recent snapshot is verified.

Master quorum lost

Restore network connectivity or bring back master-eligible nodes. In 8.x, change the voting configuration through the proper cluster API rather than editing cluster.initial_master_nodes after bootstrap.

Forced allocation: use only after the root cause is fixed

If a replica remains stuck and you have confirmed the underlying issue is resolved, you can request allocation:

POST /_cluster/reroute
{
  "commands": [
    {
      "allocate_replica": {
        "index": "",
        "shard": ,
        "node": "",
        "accept_data_loss": true
      }
    }
  ]
}

This changes cluster state. accept_data_loss can discard data on the target shard. Only use it after verifying the shard is not corrupt and a snapshot exists if the data matters. There is no clean rollback for a forced allocation; if the shard fails again, return to allocation explain rather than forcing another node.

Escalation criteria

Escalate to the data platform team or Elastic support when:

• Any primary shard remains unassigned for more than a few minutes during an incident
• The allocation explain output references corruption or a missing shard generation
• You must use accept_data_loss to force allocation
• Disk is at flood-stage across multiple nodes and writes are blocked
• Master quorum cannot be restored from reachable nodes

Verify the result

Run:

GET /_cluster/health?wait_for_status=green&timeout=30s

If the request returns before the timeout with status green, the cluster has recovered. Then confirm:

GET /_cat/shards?v&h=index,shard,prirep,state,node,unassigned.reason

No rows should show UNASSIGNED. Monitor nodes for at least ten minutes:

GET /_cat/nodes?v&h=name,disk.used_percent,heap.percent

Disk and heap should remain stable below watermark and breaker limits.

Limitations

This guide covers unassigned shards caused by allocation decisions, disk pressure, awareness rules, and master quorum. It does not cover network partitions that prevent client access, security authentication failures, or cluster-level bugs requiring a version upgrade. Defaults cited here apply to standard Elasticsearch distributions; managed offerings may override them.