Automating Index Configuration with Elasticsearch Index Templates

Solving the Manual Indexing Bottleneck

Manually configuring shards, replicas, and field mappings every time a new index is created—especially in time-series data like logs or metrics—is prone to error and leads to inconsistent data types across your cluster. The solution is Index Templates, which allow you to define a blueprint that Elasticsearch automatically applies to any new index matching a specific naming pattern.

The primary takeaway is that templates act as a trigger: when a request to create an index arrives, Elasticsearch checks all available templates, identifies those whose index_patterns match the new index name, and merges the settings and mappings into the final index configuration.

The Mechanism: Patterns, Priority, and Mappings

An index template consists of three main components: the index_patterns (which indices this applies to), the priority (which template wins if multiple match), and the template body (the actual settings and mappings).

Implementation Example

Suppose you are managing application logs that rotate daily (e.g., logs-app-2026.10.18). You want to ensure all these indices use a specific shard count and that the timestamp field is always treated as a date, not a string.

Run the following command via the Kibana Dev Tools console or curl. This requires manage_index_templates permissions.

PUT /_index_template/app_logs_template
{
  "index_patterns": ["logs-app-*"],
  "priority": 100,
  "template": {
    "settings": {
      "number_of_shards": 3,
      "number_of_replicas": 1,
      "index.mapping.total_fields.limit": 1000
    },
    "mappings": {
      "properties": {
        "timestamp": { "type": "date" },
        "log_level": { "type": "keyword" },
        "message": { "type": "text" }
      }
    }
  }
}

Verification Process

To verify the template is functioning, create a new index that matches the pattern and inspect its mapping:

1. Create the index: PUT /logs-app-2026.10.19
2. Check mappings: GET /logs-app-2026.10.19/_mapping

If successful, the response will show the timestamp field as a date type, even though you did not specify it during the PUT request.

Handling Conflicts with Priority

In complex environments, you may have a general template for all logs and a specific template for application logs. Elasticsearch resolves this using the priority field. A higher integer value takes precedence.

Template Name	Pattern	Priority	Setting: Shards
global_logs	logs-*	10	1
app_logs	logs-app-*	100	3

If you create logs-app-2026.10.19, both templates match. Because app_logs has a priority of 100, the index will be created with 3 shards.

Critical Limitations and Common Mistakes

No Retroactive Application

Index templates are not migration tools. They only apply at the moment of index creation. If you update a template to change a field from text to keyword, existing indices will remain unchanged. To apply the new mapping to old data, you must use the _reindex API to move data into a new index.

The "Over-Sharding" Trap

A common mistake is defining a high number of shards (e.g., 5 or 10) in a template for small datasets. Because this template applies to every new daily index, you can quickly accumulate thousands of shards, leading to "shard overhead" which consumes significant heap memory and can crash the cluster master node.

Dynamic Mapping Bloat

While templates allow you to define specific fields, leaving the rest of the index to dynamic: true (the default) can lead to a "mapping explosion." This occurs when incoming documents contain hundreds of unique, randomly named fields, causing the cluster state to grow too large to manage. Always consider setting dynamic: strict or false in your template for production environments where the schema is known.

Rollback Procedure

If a template is causing indices to be created with incorrect settings, delete the template to prevent further misconfigurations:

DELETE /_index_template/app_logs_template

Note: This will not fix indices already created by the template; those must be reindexed or deleted manually.