Preventing Mapping Drift with Elasticsearch Index Templates

The Problem: The 'Daily Index' Mapping Trap

When managing time-series data—such as logs, metrics, or audit trails—the common practice is to rotate indices daily or hourly (e.g., logs-2026-10-17). The danger here is mapping drift. If a new field is introduced or a data type changes in your application, Elasticsearch's dynamic mapping might guess the type differently for Tuesday's index than it did for Monday's.

When you run a cross-index search, this inconsistency causes failures or missing results because the same field is a keyword in one index and text in another. Manually creating every daily index is impossible at scale. The solution is to move the index definition from the ingestion script to the cluster using Index Templates.

Defining the Blueprint with Index Templates

An Index Template is a predefined set of settings and mappings that Elasticsearch automatically applies to any new index whose name matches a specific pattern. Instead of defining the schema at the moment of data insertion, you define it once at the cluster level.

Modern Elasticsearch (7.8+) uses Composable Index Templates. These separate the template into two parts: Component Templates (reusable building blocks) and the Index Template (the final assembly). This modularity allows you to define a standard "logging settings" component and reuse it across different types of log indices.

Worked Example: Standardizing Log Ingestion

In this scenario, we want all indices starting with logs-app- to have a specific shard count and ensure the user_id field is always a keyword for exact matching, rather than being analyzed as text.

Step 1: Create a Component Template

Run this via the Dev Tools console or curl. This defines the reusable settings for all log indices.

PUT _component_template/logs-settings
{
  "template": {
    "settings": {
      "index.number_of_shards": 3,
      "index.number_of_replicas": 1
    }
  }
}

Step 2: Create the Index Template

Now, we link that component to a pattern. We also define the explicit mapping for our critical fields.

PUT _index_template/app-logs-template
{
  "index_patterns": ["logs-app-*"],
  "priority": 10,
  "composed_of": ["logs-settings"],
  "template": {
    "mappings": {
      "properties": {
        "user_id": {
          "type": "keyword"
        },
        "timestamp": {
          "type": "date"
        },
        "message": {
          "type": "text"
        }
      }
    }
  }
}

Step 3: Verification

To verify the template is working, create an index that matches the pattern and check its mapping. You do not need to provide any mapping during creation; the template handles it.

# Create the index
PUT logs-app-2026-10-17

# Verify the mapping
GET logs-app-2026-10-17/_mapping

Expected Result: The response should show user_id as keyword and the index settings should show 3 shards, confirming the template was applied.

Handling Template Conflicts and Priority

In complex environments, a single index name might match multiple templates. For example, an index named logs-app-web-01 might match both logs-* and logs-app-*.

Elasticsearch resolves this using the priority field. The template with the highest priority number wins. If two templates have the same priority, the behavior is undefined, and you risk inconsistent mappings. Always assign explicit, tiered priority levels (e.g., general logs = 10, specific application logs = 20) to ensure the most specific rules take precedence.

Critical Limitations

• No Retroactivity: Templates only apply to indices created after the template is saved. If you update a template to change a field from text to keyword, existing indices remain unchanged. You must use the _reindex API to move old data into a new index with the updated mapping.
• Dynamic Mapping Risks: While templates define known fields, any new fields sent by the application will still be dynamically mapped. To prevent this entirely, set "dynamic": "strict" in the mapping section of your template. This will reject any document containing a field not explicitly defined.

Actionable Summary

To eliminate mapping drift in your time-series data:

1. Identify common settings (shards, replicas) and move them into Component Templates.
2. Define Index Templates with specific index_patterns and a clear priority hierarchy.
3. Explicitly map high-cardinality fields (IDs, status codes) as keyword to avoid the overhead and inaccuracy of full-text analysis.
4. Verify the application of templates using the GET /index_name/_mapping API immediately after the first index of a new cycle is created.