Configure Splunk Index-time Field Extractions with props.conf and transforms.conf

Searching high-volume Splunk data with rex repeatedly consumes search-head CPU. If you always filter on the same field, extracting it at index time moves that cost to ingestion—but makes the extraction permanent for newly indexed data.

Desired outcome

Extract a specific field, such as a transaction ID, during ingestion so searches can filter on it without using rex at query time. This is useful for fields that appear in most events and are used in most searches.

Prerequisites

• Splunk Enterprise 8.x or 9.x indexer or heavy forwarder where data is parsed. A search head alone does not apply index-time extractions. Universal forwarders do not parse these files locally.
• File-system access to $SPLUNK_HOME/etc/system/local/ or an app directory, typically as the Splunk service user or with sudo.
• Splunk admin credentials to restart the instance.
• A sample of the raw log data and a regex that reliably captures the target field.

Configure transforms.conf

Define the extraction in transforms.conf on the indexer or heavy forwarder. Named capture groups in the regex become field names.

[extract_transaction_id]
REGEX = (?<transaction_id>TXN-[0-9A-F]{8})
SOURCE_KEY = _raw

Where to run: indexer or heavy forwarder CLI. Permissions: write access to the configuration directory. Risk: a malformed regex can misparse data; test the pattern with rex in search before deploying it here.

Configure props.conf

Map the transform to a sourcetype or source in props.conf on the same parsing instance.

[mysourcetype:apache_access]
TRANSFORMS-txn = extract_transaction_id

The TRANSFORMS-txn setting references the stanza name in transforms.conf. Multiple transforms can be listed, separated by commas.

Apply the configuration

1. Back up both files before editing.
2. Place transforms.conf and props.conf on the indexer or heavy forwarder that parses the data.
3. Restart Splunk on that instance, for example: sudo /opt/splunk/bin/splunk restart.
4. Ingest new events matching the sourcetype.

Index-time changes apply only to newly indexed data. Existing events in indexes are not rewritten.

Verify the result

1. Run a search over newly ingested data:

index=main sourcetype=mysourcetype:apache_access earliest=-15m
| stats count by transaction_id

Expected check: results return events grouped by transaction_id without using rex or extract in the search string.

1. Open the Search app, run the same search, and confirm transaction_id appears in the Interesting Fields sidebar.
2. Compare search job execution time before and after for a repeated query over a large time range.

Limitations

• Permanent for indexed data: Changing the regex does not update events already on disk. You must re-index the data to apply a new pattern to old events.
• Disk growth: Each extracted field increases index size. Over-extraction causes measurable bloat.
• Parse location: If data passes through a heavy forwarder, configure the heavy forwarder. If data goes directly to indexers, configure the indexers. Configuring only the search head has no effect on index-time parsing.

Recovery and rollback

Because this operation changes index-time parsing state, rollback is possible for future data but not retroactive:

1. Remove or comment out the TRANSFORMS line in props.conf.
2. Optionally remove the stanza from transforms.conf.
3. Restart Splunk.
4. New events stop extracting the field. Previously indexed events keep the field as stored.

Practical decision checklist

Factor	Index-time extraction	Search-time extraction (rex)
CPU cost	Indexer at ingestion	Search head at query time
Existing data	Not affected	Works on all indexed data
Index size	Increases	No change
Change flexibility	Requires restart; old data unchanged	Change search string anytime

If you are unsure whether the regex is safe, test it first with the rex command in search against sample events. Only move a validated pattern to transforms.conf.