Kaldi online2 decoding pipeline & nnet3 model pruning – optimal configuration for low‑traffic workloads?

Goal

Determine a cost‑effective deployment strategy for Kaldi on low‑traffic speech workloads by balancing the real‑time online2 decoding pipeline with aggressive nnet3 model pruning.

Constraints & Uncertainty

The online2‑wav‑nnet3‑latency script defaults to a 30 ms stride, yet the trade‑off between this stride setting, decoding latency, and word error rate (WER) is not fully documented. Additionally, pruning can reduce model size but may impact accuracy, and GPU acceleration requires matching CUDA versions.

Unresolved Questions

• What is the measurable impact of reducing the stride from 30 ms to 10 ms on latency and WER for a typical low‑traffic scenario?
• How aggressively can nnet3‑copy with --transform prune the model before accuracy drops below an acceptable threshold for low‑traffic use?
• Is there a recommended configuration that balances CPU usage, memory footprint, and decoding latency on a t3.medium instance while remaining within cost constraints?