Deploying Kaldi Chain Models with Online Decoding: A Practical Guide

Why Online Chain Decoding Matters

For real‑time voice interfaces, you need a recognizer that can start outputting hypotheses while the user is still speaking. Kaldi’s chain training framework (lattice‑free MMI) delivers high‑accuracy models, and its online2-wav-nnet3-latgen-faster binary turns that accuracy into low‑latency streaming decoding. The key takeaway: combine a 3× subsampled chain model with the online feature pipeline (MFCC, CMN, i‑vector) and you get a production‑ready recognizer that runs on commodity hardware.

How the Online Pipeline Works

• Feature extraction: Incremental MFCCs or filter‑bank energies are computed on the fly. The default mfcc.conf uses a 25 ms frame, 10 ms shift, and 16 kHz sampling rate.
• Cepstral mean normalization (CMN): A sliding window (default 400 frames) removes channel bias. The online binary applies CMN as soon as the window is filled.
• i‑vector adaptation: Using ivector_extractor.conf, the system estimates a 100‑dimensional i‑vector from the first few seconds of the utterance. Subsequent frames are conditioned on this vector, improving speaker adaptation.
• Chain decoding: The neural network outputs a 3‑frame‑subsampled posterior. The online2-wav-nnet3-latgen-faster binary expands this to a lattice over the HCLG graph, then applies a cost‑based endpointing rule.

Worked Configuration

Below is a minimal, reproducible setup that mirrors the egs/wsj/s5 recipe but strips away grid‑engine dependencies. Assume you already have a trained chain model (final.mdl) and a decoding graph (HCLG.fst).

# 1. Create a config directory
mkdir -p online_conf
# 2. Copy the default configs (adjust paths if needed)
cp egs/wsj/s5/steps/online/nnet3/prepare_online_decoding.sh online_conf/
# 3. Run the script to generate the configs
bash online_conf/prepare_online_decoding.sh \
  --data-dir=data/train \
  --model-dir=exp/nnet3/tdnn_1a_chain \
  --ivector-dir=exp/nnet3/ivector \
  --config-dir=online_conf
# 4. Decode a single WAV file
online2-wav-nnet3-latgen-faster \
  --config=online_conf/conf/online.conf \
  --word-symbol-table=data/lang/words.txt \
  exp/nnet3/tdnn_1a_chain/final.mdl \
  exp/chain/graph/HCLG.fst \
  ark:spk2utt scp:wav.scp ark,t:lat.1

Key points:

• --config points to a directory containing mfcc.conf, ivector_extractor.conf, and online.conf.
• The spk2utt file maps speaker IDs to utterances; for pure streaming you can supply a dummy speaker ID.
• Output lattices are written to lat.1 in Kaldi’s ark format, ready for lattice-best-path or rescoring.

Endpointing and Latency Control

Endpointing is handled automatically, but you can fine‑tune it via endpoint.conf (generated by the prepare script). The most common parameters are:

Parameter	Description
endpoint.silence_phones	Phone IDs considered silence (e.g., 0 1 2).
endpoint.min_silence_len	Minimum silence duration (ms) before ending.
endpoint.max_silence_len	Maximum silence duration to allow before forced cut‑off.

Typical values for a voice assistant are 200–400 ms for min_silence_len and 800–1200 ms for max_silence_len. Adjusting these trades off latency against premature termination.

Limits and Common Pitfalls

• Frame‑rate mismatch: Chain models were trained with a 3× subsampling (i.e., 30 ms effective frame). Feeding 10 ms frames without adjusting the network’s subsampling-factor silently degrades accuracy.
• Cold i‑vector starts: For utterances shorter than ~2 s, the online extractor has insufficient data to estimate a reliable i‑vector, leading to higher WER compared to offline decoding. A workaround is to provide a pre‑computed i‑vector via ivector.scp or to use a smaller ivector_extractor.conf that accepts fewer frames.
• Sampling‑rate assumptions: The default MFCC pipeline expects 16 kHz. If your audio is 8 kHz or 24 kHz, convert it first or modify mfcc.conf accordingly.
• Graph compatibility: The HCLG graph must be built with the same language model used during training. Mixing a 3‑gram LM with a 4‑gram LM will produce mismatched costs.
• Hardware constraints: The online binary is CPU‑bound; for high‑throughput services, run it on a machine with at least 4 cores and enable the --num-threads flag.

Verification Checklist

1. Run online2-wav-nnet3-latgen-faster on a known test set and compare the first‑best WER with the recipe’s reported value.
2. Decode the same audio offline (nnet3-latgen-faster) and verify that the online WER is within 1–2 % of the offline result.
3. Inspect the lattice with lattice-best-path and confirm that the output text matches the reference transcription.
4. Check the online.conf for the frame_shift_ms value; it should be 10 ms for 16 kHz audio.

Conclusion

Kaldi’s chain models, when paired with the online feature pipeline, provide a solid foundation for low‑latency, high‑accuracy speech recognition. By following the configuration steps above, tuning endpointing, and watching for the common pitfalls, you can deploy a recognizer that performs close to offline accuracy while delivering real‑time responses. For teams seeking a newer framework, consider K2 or Icefall, but the principles—subsampled neural nets, online i‑vector adaptation, and lattice‑free training—remain the same.