Kaldi Chain Models: Why LF-MMI and a 3x Slower Frame Rate Changed ASR Training

The alignment problem chain models solve

If you train a conventional neural-network acoustic model in Kaldi, you first need frame-level alignments: a GMM-HMM system tells you which HMM state each 10 ms frame belongs to, and the network learns to classify frames with cross-entropy. Then, if you want better accuracy, you bolt on sequence-discriminative training (sMBR or MMI) afterward. That pipeline works, but it is long, brittle, and the sequence training stage is finicky because it depends on generating lattices with the very model you are trying to improve.

Chain models collapse this into one stage. They train the neural network directly with a lattice-free MMI (LF-MMI) objective, so no frame-level alignment is ever needed — only the utterance-level transcript. In practice this both simplifies the recipe and usually beats the cross-entropy-then-sequence-training pipeline on word error rate. If you are starting a Kaldi project today, a chain model is the default choice, and the rest of this post explains the two design decisions that make it work.

Decision one: one output label every three frames

Standard Kaldi acoustic models produce a posterior for every 10 ms frame. Chain models instead emit one label roughly every 30 ms — a frame-subsampling factor of 3, set in the nnet3 config. This is possible because the HMM topology used for chain training gives each phone a single state with a self-loop (plus a blank-like path), so the model does not need three frames to traverse a three-state phone.

The payoff is large: training and decoding both get roughly a 3x speedup because there are three times fewer output frames to score and three times fewer graph states to traverse during search. The cost is that the HCLG decoding graph must be built with the matching 3-fold self-loop topology. If you build the graph with a standard topology, decoding will silently produce garbage or fail — this mismatch is one of the most common chain-model setup bugs. You can verify the factor by inspecting the nnet3 config for the frame-subsampling-factor and confirming the graph-building step used --self-loop-scale 0.1 with the chain-style topology, as the WSJ and Switchboard recipes do.

Decision two: a phone-level denominator FST instead of lattices

MMI training needs two quantities: the score of the correct transcript (numerator) and the total score of all possible transcripts (denominator). Classic MMI computes the denominator from recognition lattices, which is expensive and couples training to decoding. LF-MMI replaces the lattices with a single fixed FST built from a phone-level language model — essentially a small n-gram over phonemes. The forward-backward computation over this FST is done on the GPU with Kaldi's CUDA chain-training code, which is what makes the objective tractable at scale.

This design has a practical consequence: the denominator FST is baked into training, and if it is inconsistent with the acoustic scale or the alignments used to build the numerator, the objective diverges. When you see NaN or wildly oscillating LF-MMI objective values in the training logs, the usual culprits are bad input alignments (fix the GMM baseline first) or an initial learning rate that is too aggressive — halve it before changing anything else.

A practical workflow

Kaldi is recipe-driven, and flags and directory layouts vary between egs/ directories and toolkit versions, so the reliable path is to copy a working recipe rather than assemble commands from memory. The typical chain flow, as in egs/wsj/s5 or egs/swbd/s5c, looks like this:

1. Prepare data and train a GMM-HMM baseline (tri3b-style) to produce lattice-alignments for the numerator.
2. Build a new tree with fewer leaves and a 2-state reduced-context topology (steps/nnet3/chain/build_tree.sh).
3. Generate the denominator FST and training examples (steps/nnet3/chain/get_egs.sh).
4. Train a TDNN or TDNN+LSTM network (steps/nnet3/chain/train.py).
5. Build HCLG with the chain topology and decode with latgen-faster-mapped; optionally rescore lattices with a larger language model, which is a cheap and reliable WER improvement.

For streaming use, the online2 binaries combine the nnet3 chain model with i-vector speaker adaptation. I-vectors are estimated incrementally as audio arrives, so latency stays low — but note the extractor is part of the trained system, not an optional add-on.

Worked check: confirming your frame rate matches your graph

Before decoding, sanity-check the model. From your experiment directory (as a normal user with read access to the model files), run:

nnet3-info exp/chain/tdnn/final.mdl | grep -i subsampling

This prints the frame-subsampling factor embedded in the model; for a standard chain model it should be 3. Then confirm the decoding graph was built by the recipe's mkgraph.sh step with the chain self-loop scale. A quick end-to-end check is to decode a small held-out set and compare WER against the recipe's published RESULTS file — a number within a few tenths of the reference means the pipeline is wired correctly; a catastrophically high WER almost always means a topology or graph mismatch, not a bad model.

Limitations worth knowing

Chain training is sensitive to its denominator FST and learning-rate schedule, and debugging divergence requires reading training logs, not just rerunning. I-vector extractors trained on clean read speech degrade noticeably on mismatched domains like telephone or far-field audio — domain-matched training data matters more than model size here. Finally, Kaldi development has slowed as much of the community has moved to successors like K2 and icefall, so check repository activity before committing to a long-term project to it. The concepts in this post carry over directly, though: LF-MMI and low frame rates are the foundation those newer toolkits build on.

Where to start

Run a small published recipe end to end — egs/yesno for a smoke test, then egs/wsj/s5 if you have the data — and confirm the chain stage completes with a plausible WER. Once you have seen the full flow work once, adapting it to your own audio is mostly a data-preparation exercise, not a modeling one.