How does the leaky HMM coefficient interact with L2 regularization in Kaldi chain model training?

Goal

Determine whether the leaky HMM coefficient and L2 regularization term in Kaldi's LF-MMI chain training should be jointly tuned or can be set independently based on a principled criterion.

Constraints

Kaldi's chain models use lattice-free MMI with a denominator graph constructed from training data. The leaky HMM coefficient adds smoothing probability to prevent zero transitions, while L2 regularization penalizes large weights. Standard recipes use 0.1 and 0.00005 respectively, but no formal analysis justifies this ratio.

Uncertainty

Modifying the leaky HMM coefficient without adjusting L2 regularization can destabilize early training through gradient explosion or vanishing gradients. The optimal relationship between these hyperparameters remains undocumented, with practitioners defaulting to recipe values rather than deriving them from theory or systematic experimentation.

Questions

<

1. Does increasing the leaky HMM coefficient proportionally require adjusting the L2 regularization strength to maintain stable convergence?
2. Is there a theoretical or empirical basis for the commonly used ratio of 2000:1 between these hyperparameters?