Kaldi beam & max‑active tuning for low‑traffic cost

Question

For low‑traffic speech‑recognition services using Kaldi’s online decoder (online2-wav-nnet3-latgen-faster), the objective is to limit the real‑time factor (RTF) per utterance to a predefined cost threshold—say ≤0.04—while ensuring that the word error rate (WER) does not rise more than 2 % relative to the baseline configuration (beam = 13.0, max‑active = 5000).

Although the decoder exposes independent --beam and --max-active knobs, there is no published guideline for choosing them together to hit both the RTF and WER targets. Lowering the beam reduces computation but may prune correct paths; decreasing max‑active caps worst‑case latency but can cause search failures if set too low. The trade‑off surface between the two parameters remains unmapped for specific cost‑accuracy pairs.

What beam and max‑active values achieve an RTF of 0.04 with ≤2 % relative WER increase on a representative test set? How does the interaction change when the traffic pattern shifts from bursty to uniformly sparse? Which combination yields the lowest latency variance without triggering empty‑lattice errors?

Tasadduq Burney · Answer

Short answer
No published beam/max-active pair universally delivers RTF ≤0.04 with ≤2% relative WER increase. The correct values depend on the acoustic model, language model, audio conditions, and Kaldi/nnet3 version. Measure them on a held-out representative test set; the procedure below finds the pair for your deployment.

Confirmed behavior vs likely explanation
Confirmed

--beam controls per-frame graph pruning; lowering it reduces average computation and RTF but can prune the correct hypothesis and raise WER.
--max-active bounds the maximum active states per frame; it caps worst-case latency and memory but does not directly control average RTF.
Empty lattices indicate the search space is too constrained; raise max-active before relaxing beam.
Bursty traffic adds queueing and CPU contention; uniformly sparse traffic isolates per-utterance RTF and usually lowers observed latency variance.

Likely explanation (unverified for your model)
Reducing beam first, then lowering max-active only until RTF plateaus, typically yields the lowest variance while preserving WER, because max-active acts as a ceiling rather than an average-cost lever.

Tuning steps

Measure baseline RTF and WER with beam=13.0, max-active=5000 on the held-out test set.
Decrease beam in small steps, for example 13.0 → 11.0 → 9.0 → 7.0, holding max-active=5000, and record RTF and WER at each step.
Stop at the largest beam whose WER increase is within the 2% relative budget and whose RTF is at or below 0.04.
For that beam, decrease max-active in steps, for example 5000 → 3000 → 1500 → 750, until RTF stops falling or empty lattices appear.
Select the highest max-active that still meets the RTF target; this minimizes empty-lattice risk and latency variance.
Repeat under bursty and uniformly sparse arrival patterns; compare p50/p99 latency, not just mean RTF.

Verification commands
Run timing on CPU with wall-clock measurement, not real-time audio playback:
time online2-wav-nnet3-latgen-faster --beam=9.0 --max-active=3000 --online=false ... test.wav lat.ark
Compute RTF as decode seconds divided by audio seconds. Compare WER against the baseline lattice using Kaldi scoring scripts.

Missing diagnostic detail
Provide the baseline RTF and WER for beam=13.0, max-active=5000 on your representative test set. Without those numbers, it is impossible to say whether 0.04 is reachable or how much WER headroom exists.

Assumptions and uncertainty

Assumes Kaldi nnet3 chain online decoder; frame rate and model topology change absolute cost.
RTF target is per-utterance decode time, not system throughput under concurrency.
≤2% relative WER requires significance testing on a held-out set; single-run differences are not reliable.
Specific numeric beam/max-active values are omitted because they are not transferable.

Kaldi beam & max‑active tuning for low‑traffic cost

1 answer

Short answer

Confirmed behavior vs likely explanation

Confirmed

Likely explanation (unverified for your model)

Tuning steps

Verification commands

Missing diagnostic detail

Assumptions and uncertainty

Your perspective belongs here.

1 question comment