Indexed fields with tstats versus search-time extraction for high-volume Splunk searches

Answer

For a scan‑bound search, pick the approach that delivers the best trade‑off between search speed and maintenance cost. Indexed fields + tstats win when the query is run frequently and the indexable fields are stable; they reduce scanCount but lock the schema and enlarge tsidx storage. Search‑time extraction keeps the schema flexible and avoids extra storage, but every run must parse raw events, which is CPU‑heavy. Data‑model acceleration sits in between: it builds tsidx summaries only for the fields that fit the model, adding background CPU and disk usage but keeping a flexible schema for events that fall outside the model.

Factors to Consider

• Query Frequency vs Data Volume – If the same query runs daily over millions of events, tstats can be worth the index cost. For ad‑hoc or infrequent searches, search‑time extraction may be cheaper.
• Event Size & Complexity – Large, complex events amplify the cost of parsing each event. Indexed fields cut that cost.
• Index Size Impact – Adding indexed fields can increase tsidx by 30–50 % per field. Verify the storage impact with indexer summary before committing.
• Schema Rigidity – Once a field is indexed, changing the extraction logic requires re‑ingestion of all affected data. Use search‑time extraction if extraction rules are expected to evolve.
• Background Load – Data‑model acceleration runs incremental jobs. In a busy indexer, this can compete with real‑time searches.
• Coverage – Accelerated data models only include events that match the model’s event type. If a query touches many non‑model events, tstats on indexed fields may still be faster.

Side‑by‑Side Comparison

Run the same query three times and capture the same metrics in the Search Job Inspector:

1. Indexed fields + tstats
   

   tstats count where index=web sourcetype=access_combined by status
   

2. Search‑time extraction
   

   search index=web sourcetype=access_combined | stats count by status
   

3. Data‑model acceleration
   

   | datamodel WebAccess | search status=200 | stats count by status
   

For each run, note:

• scanCount
• CPU usage (user+sys)
• Memory peak
• Elapsed time
• tsidx size (for indexed fields)
• Acceleration status (for DM)

Compare the speed gain against the increase in tsidx storage. A common rule of thumb is to adopt indexed fields if the query is >50 % faster and the storage increase is ≤ 30 % of the current index size.

Missing Diagnostic Detail

To refine the recommendation, we need the average number of events per search and the average event size. These figures influence whether the cost of re‑ingestion for indexed fields outweighs the scan savings.