Designing a Robust Index Rollover Strategy with Elasticsearch Aliases

Problem Statement

In high‑write environments, a single index can grow too large for efficient search and indexing. When the index size or document count reaches a threshold, queries slow down and cluster health can degrade. Manual re‑indexing is costly and error‑prone. The challenge is to create a rollover mechanism that keeps the active index small, preserves query performance, and integrates cleanly with existing applications.

Requirements

• Write operations always target the newest, “active” index.
• Read operations should be agnostic to the underlying index, using a logical name.
• Automatic rollover should be triggered by size, document count, or age, without manual intervention.
• Old indexes must be retained for a defined retention period and then removed automatically.
• Mappings and settings should be consistent across all indexes to avoid schema drift.
• Rollover should not introduce data loss or query inconsistency.

Smallest Suitable Design

The core of the strategy is an alias (e.g., logs) that points to the current active index (e.g., logs‑2026.10.13). When the active index reaches the configured threshold, a rollover operation creates a new index (e.g., logs‑2026.10.14) and updates the alias to point to it. The old index remains searchable by its name but no longer receives writes.

Key Components

• Alias – logical name used by applications for both reads and writes.
• Rollover API – POST //_rollover triggers index creation based on conditions.
• Index Template – defines mappings and settings that are copied to every new index.
• Lifecycle Policy – optional ILM policy or Curator job to delete old indexes after the retention period.

Trust & Data Boundaries

All writes are directed to the active index only. The alias is configured with is_write_index:true for the active index and false for others. This prevents accidental writes to archived data. Read operations use the alias, so the application code does not need to know which physical index is current. By isolating writes to the newest shard set, we keep shard allocation predictable and avoid fragmentation.

Implementation Example

1. Define an Index Template

PUT /_template/logs_template
{
  "index_patterns": ["logs-*"],
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1,
    "refresh_interval": "1s"
  },
  "mappings": {
    "properties": {
      "@timestamp": {"type": "date"},
      "message": {"type": "text"}
    }
  }
}

2. Create the Initial Index and Alias

PUT /logs-2026.10.13
{
  "aliases": {
    "logs": {"is_write_index": true}
  }
}

Applications now write to logs and read from it. The index name follows the pattern logs-YYYY.MM.DD, but the alias hides this detail.

3. Configure Rollover Conditions

PUT /_ilm/policy/logs_policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_size": "50gb",
            "max_age": "7d"
          }
        }
      },
      "delete": {
        "min_age": "30d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

Attach the policy to the template:

PUT /_template/logs_template
{
  "index_patterns": ["logs-*"],
  "settings": {
    "index.lifecycle.name": "logs_policy",
    "index.lifecycle.rollover_alias": "logs"
  },
  "mappings": { ... }
}

4. Trigger a Manual Rollover (Optional)

POST /logs/_rollover
{
  "conditions": {
    "max_size": "10gb",
    "max_docs": 1000000
  }
}

Running the command on a dev cluster verifies that a new index is created and the alias is updated.

5. Verify the Result

• Check alias mapping:

GET /_alias/logs

• Search through the alias before and after rollover:

GET /logs/_search
{
  "query": {"match_all": {}}
}

• Confirm the old index is still searchable by name:

GET /logs-2026.10.13/_search

Operational Checks

• Monitor cluster.indices.indexing_pressure to ensure shards are not overloaded.
• Use GET /_cat/indices?format=json to verify that the active index is always the one with is_write_index:true.
• Set up alerting on indices.rollover metrics to detect failures.
• Validate that deleted indexes are removed after the retention period using GET /_cat/indices?format=json and checking index.lifecycle.phase.

Failure Modes & Mitigation

• Race Condition: If an application writes to the alias while a rollover is in progress, the document may be indexed into the old index. Mitigation: Use is_write_index:true exclusively for the active index and avoid bulk operations that span multiple indices.
• Storage Exhaustion: Without a delete phase, old indexes accumulate and consume disk space. Mitigation: Always configure a delete phase or use Curator to purge old indices.
• Schema Drift: Manual updates to mappings on existing indexes can lead to inconsistencies. Mitigation: Rely on the template for all new indexes and avoid manual mapping changes.
• Rollover Failure: If the cluster is under heavy load, the rollover API may time out. Mitigation: Increase request_timeout and monitor cluster health before triggering rollover.

When the Design Needs to Change

• High Cardinality Fields: If new fields with high cardinality are added, consider adjusting the index template and re‑indexing older data to avoid memory pressure.
• Search‑Only Workload: If the cluster becomes read‑heavy, adding more replicas or moving to a dedicated search cluster may be necessary.
• Multi‑Tenant Data: For isolation, create separate aliases per tenant and enforce role‑based access controls.
• Compliance Requirements: If data must be immutable for a longer period, adjust the delete phase or use snapshot archives instead of deletion.

Conclusion

By combining an alias with the Rollover API and a lifecycle policy, you can keep the active index small, maintain stable query performance, and automate data retention. The architecture is minimal, requires no code changes, and scales with your ingestion rate.