Forced Alignment with Kaldi’s HMM‑GMM Pipeline: A Practical Walkthrough

Problem: You need phone‑level timestamps for a known transcript

When building a speech corpus or preparing data for downstream models, you often have a clean audio recording and its exact transcript. What you lack is the precise start and end time for each phoneme. Manually labeling hundreds of utterances is tedious and error‑prone. Kaldi’s forced‑alignment tool solves this by aligning the audio to the transcript using a pre‑trained HMM‑GMM acoustic model and outputting time‑stamped phone boundaries.

Thesis: With a working acoustic model, a correctly built lexicon/grammar, and the align utility, you can generate reliable phone alignments in a few command‑line steps.

How Kaldi’s forced alignment works

Kaldi treats forced alignment as a decoding problem where the search graph is constrained to the exact phone sequence dictated by the transcript. The steps are:

1. Convert the transcript to a phone sequence using a lexicon (word → phones).
2. Build a simple grammar (a linear chain) that forces the decoder to follow that phone sequence.
3. Run the Viterbi algorithm on the HMM‑GMM acoustic model to find the most likely state sequence given the acoustic features.
4. Map the resulting state sequence back to phone boundaries and write them to a text file.

The accuracy of this process hinges on the acoustic model’s ability to represent the phones in your speech data. If the model was trained on a very different dialect or recording condition, alignments will drift.

Preparing the lexicon and grammar

Assume you have a directory data/my_corpus containing:

• wav.scp – maps utterance IDs to audio file paths.
• text – maps utterance IDs to the transcript (words separated by spaces).

You need a lexicon file lexicon.txt with lines like:

HELLO HH AH L OW
WORLD W ER L D

And a optional nonsilence_phones.txt, silence_phones.txt, etc., as expected by Kaldi’s utils/prepare_lang.sh. Running:

utils/prepare_lang.sh data/local/dict "" data/local/lang data/lang

creates the necessary language model files (L.fst, etc.) that the align step will consume.

Worked example: running the align utility

With the language directory ready and an acoustic model in exp/tri3 (a typical triphone GMM model), the forced alignment command looks like:

steps/align_si.sh --nj 4 data/my_corpus data/lang exp/tri3 exp/tri3_ali_my_corpus

Where:

• --nj 4 runs four parallel jobs (adjust to your CPU cores).
• data/my_corpus is the data directory described above.
• data/lang is the output of prepare_lang.sh.
• exp/tri3 points to the pre‑trained acoustic model.
• exp/tri3_ali_my_corpus will contain the alignment outputs.

You must have read permission on the audio files, the model directory, and write permission on the output directory. If the lexicon contains phones absent from the model’s phone set, the step will abort with an error about missing phones.

Verifying the alignment and spotting limitations

After the job finishes, check the file exp/tri3_ali_my_corpus/ali.1.gz (or the merged ali.txt if you ran scripts/ali_to_phones.pl). A quick sanity check is to ensure timestamps are monotonic:

ali-to-phones --per-frame=true exp/tri3/final.mdl "ark:gunzip -c exp/tri3_ali_my_corpus/ali.1.gz|" ark,t:- | head -20

Each line should show a phone ID followed by a start and end frame number that never decreases. If you see sudden jumps or overlapping intervals, the alignment likely failed due to:

• Mismatch between lexicon phones and the acoustic model’s phone set.
• Very long audio files causing excessive memory use (consider segmenting with utils/data/segment.py).
• Channel noise or reverberation not present in the model’s training data.

To gauge accuracy, manually annotate a short segment (e.g., 5 seconds) using a tool like Praat and compare the phone boundaries. Discrepancies larger than ~20 ms often indicate model mismatch.

Trade‑off: model specificity vs. coverage

Using a model trained on broad, multi‑speaker data yields decent alignments for many dialects but may struggle with speaker‑specific traits. Conversely, adapting the model (e.g., with fMLLR) improves accuracy at the cost of extra adaptation steps and longer runtimes. For most corpus‑building pipelines, a speaker‑adapted triphone model provides a good balance.

Actionable closing

Start by verifying that your lexicon phones are a subset of the model’s phone list (grep -o "[A-Z]\+" lexicon.txt | sort -u | comm -23 - <(grep -o "[A-Z]\+" exp/tri3/phones.txt | sort -u)). If the output is empty, you’re ready to run align_si.sh. After alignment, run the monotonic‑time check on a subset of jobs; if it passes, you can trust the generated timestamps for downstream tasks such as creating phonetic datasets or training neural acoustic models.