[hts-users:02211] Bug Report: Multi-process supporting patch for HTS-demo_CMU-ARCTIC-SLT

[康世胤 <kangshiyin@xxxxxxxxx>]

hi, all

I think there is a potential bug in Multi-process supporting patch for HTS-demo_CMU-ARCTIC-SLT,whoese patch package may called "HHEd-JM-Patch_for_HTS-2.1-demo.tar.bz2".


And this bug may also exist in [hts-users:02210] Multi-process supporting patch for HTS-demo_CMU-ARCTIC-ADAPT


This bug will not impact you if you do everything right while training,
but if you do something wrong in data preparation or use the option "-T 3" in HHEd,
the whole training process may stop and never terminate unless been killed manually, and you can not see any output from HHEd.


In these patches, the following code is used to creat sub processes.
Every child processes write their output to a separate pipe.

 if ($pids[$i] = open($subPipe[$i], "-|")) { # Parent
    $nSubProc++;

 } else { # Child
    die "Cannot fork: $!" unless defined $pids[$i];

    shell("$HHEd{'trn'} -H $clusmmf{$set} $mdl{$type} -w $clusmmf{$i.$set} $cxc{$i.$type} $lst{'ful'}");

    exit;
 }
 
The following code shows that the parent process will not collect the output of child processes untill all child processes terminate.

  # Wait all subprocesses to terminate
  for ($i=2;$i<=$n+1;$i++) {
     waitpid($pids[$i],0);
     # Catch output of subprocess
     while(readline($subPipe[$i])) {
        $subOutput[$i] .= $_;
     }
     close($subPipe[$i]);
     print STDOUT $subOutput[$i];
  }

Question is, the size of the pipe buffer is limited. If the child process's output full fill the pipe buffer before it terminates, the pipe will stop the child process and wait the parent process to clear the buffer. The whole training process will be locked here due to the code above. 

-- 
康世胤
Shiyin Kang