HCLG Graph Memory Limits and Beam Width Trade-offs

When deploying Kaldi for low-traffic workloads on resource-constrained hardware, memory consumption is primarily dictated by the size of the HCLG.fst graph loaded into RAM. To minimize infrastructure costs, developers often utilize on-the-fly rescoring to reduce the initial graph footprint.

However, reducing hardware resources necessitates a careful balance between the Viterbi beam search parameters and the resulting Word Error Rate (WER). There is uncertainty regarding the stability of the decoding process when scaling down CPU resources while maintaining a specific beam width.

• What is the expected impact on WER when reducing beam width to accommodate lower memory overhead on small instances?
• At what threshold does the reduction in beam width lead to a non-linear increase in decoding errors for standard HCLG graphs?