[hts-users:03454] Re: AW: Additional stream question

[Tomasz Kuczmarski <faqster@xxxxxxxxx>]

Thanks a lot,

I suspected that it should be done in the manner you described.

I was just affraid I am missing on something important.

Thanks again,
Tomasz

2012/11/5 Schabus, Dietmar <Schabus@xxxxxx>:
> Tomasz,
>
> I am working on joint audio-visual speech synthesis. It depends on from what exactly you are starting, but basically, the procedure is something like:
>
> 1) Start from a working audio-only training procedure
> 2) Make sure that the feature extraction step creates .cmp files that contain values for your new stream (with deltas and delta-deltas) in addition to the audio features
> 3) Add your new stream to the protos (functions make_proto etc)
> 4) Go through all steps of the training, wherever there is something stream-specific going on, add code for your new stream
>
> I think for me, that was about it.
>
> Good luck ;)
> Dietmar
>
> --
> ----------------------------------------------------------------------
> Dipl.-Ing. Dietmar Schabus | Researcher
> phone +43 1 5052830-48 | fax -99 | schabus@xxxxxx | http://userver.ftw.at/~schabus/
>
> FTW Telecommunications Research Center Vienna
> Donau-City-Straße 1/3 | 1220 Vienna | Austria | www.ftw.at
>
> ________________________________________
> Von: Tomasz Kuczmarski [faqster@xxxxxxxxx]
> Gesendet: Montag, 05. November 2012 16:29
> An: hts-users@xxxxxxxxxxxxxxx
> Betreff: [hts-users:03452] Additional stream question
>
> Dear all,
>
> I am trying to perform integrated training and synthesis of speech
> acoustics and articulatory data. I have a speech corpus with
> accompanying 2D EMA data.
> I am guessing that I should introduce an additional data stream to my
> models. This idea is pretty clear to me. However I do not really know
> how to start.
>
> What are the parts of code I should modify? Has anyone here done
> anything similar (multimodal synthesis for example) and is willing to
> share the basic technical details with me?
>
> If I knew what is the general scheme of data/code preparation I could
> work the rest out myself.
>
> Thank you in advance,
>
> Tomasz Kuczmarski
>
>