[hts-users:02454] Implementing coupled HMM by HMM stream tying

[Po-Sen Huang <huang.person@xxxxxxxxx>]

Hi everyone

I am implementing coupled HMM by HMM stream tying by Stephen Chu, Thomas Huang paper "Multi-modal sensory fusion with application to audio-visual speech recognition"

In my setting, there are two streams in each state. I use HTS for stream tying. However, in reestimation in mixture>=3 case, there will be some errors:

1.Standard outputs: Merging results:   ERROR [+7031]  CheckStream: ph: Mix weights sum 0.000000e+00 for s=1, j=2

2.ERROR [+7191]  Infinite WtAcc!
3. WARNING [-2330]  UpdateWeights: Model 5[ap]: no use of mixtures
in 6.1 in /HTKTools/HERest

These errors will be obtained in different cases such longer scp file, shorter scp file.

I think I correctly followed the paper. I am not sure whether it is HTS's problem or not. Have anyone implemented coupled using HMM stream tying before? Thank you very much

Sincerely,

Po-Sen