[hts-users:01513] Re: Possible memory leakage in HMGenS
Hi,
Xuchen Yao wrote (2008/07/09 12:44):
At the end of the function DoGenenration() in HMGenS.c:
/* free memory */
Dispose(&gstack, ++utt->o);
ResetGenInfo(genInfo);
I think since utt->o is made by MakeObservation(), it should be first
freed like this:
for (t=utt->T; t>0; t--) {
ResetObservation(&gstack, &utt->o[t], hset->swidth, 0);
}
valgrind doesn't report memory loss whether adding or removing the
above lines. And I'm not sure about this. Could you please check?
About 2 years ago I had the same question and discussed it with Gunnar Evermann
and Mark Gales, maintainers of HTK. The discussions are as follows. Please
read it.
Best regards,
Heiga ZEN (Byung Ha CHUN)
--
------------------------------------------------
Heiga ZEN (in Japanese pronunciation)
Byung Ha CHUN (in Korean pronunciation)
Department of Computer Science and Engineering
Nagoya Institute of Technology
Gokiso-cho, Showa-ku, Nagoya 466-8555 Japan
http://www.sp.nitech.ac.jp/~zen
------------------------------------------------
-------- Original Message --------
Subject: [HTK-Developers] Objects created but not freed in HHEd.c
Date: Mon, 26 Jun 2006 15:21:25 +0900
From: Heiga ZEN (Byung Ha CHUN) <zen@xxxxxxxxxxxxxxxx>
Organization: Nagoya Institute of Technology
To: htk-developers@xxxxxxxxxxxxx <htk-developers@xxxxxxxxxxxxx>
Dear HTK-Developers,
I found that some objects were created but not freed explicitly in HHEd.c.
void Clustering(ILink ilist, int *numReq, float threshold,
char type, char *macName)
{
...
cvec = BuildCVec(numClust,ilist);
idist = CreateMatrix(&tmpHeap,numClust,numClust);
gdist = CreateMatrix(&tmpHeap,numClust,numClust);
SetIDist(cvec,idist,numClust,type); /* compute inter-item distances */
...
Dispose(&tmpHeap,cvec);
}
In Clustering, Matrix idist and gdist are created but not freed.
void FloorAverageCommand(void)
{
...
if (hset->ckind != DIAGC)
HError(2640,"FloorAverageCommand: Only implemented for DIAGC models");
S= hset->swidth[0];
/* allocate accumulators */
varAcc = (DVector*)New(&gstack,S*sizeof(DVector));
for (s=1;s<=S;s++) {
varAcc[s] = CreateDVector(&gstack,hset->swidth[s]);
ZeroDVector(varAcc[s]);
}
...
/* and apply floors */
if (applyVFloor)
ApplyVFloor(hset);
else
HError(-7023,"FloorAverageCommand: variance floors have not been applied
to mixes");
Dispose(&gstack,varAcc);
}
In FloorAverageCommand, DVector *varAcc and DVector varAcc[1..S] are created but
only varAcc is freed.
void CalcClusterDistribution(RNode *n, int vSize)
{
CoList *c;
AccSum *acc;
int k;
float occ;
Vector sum, sqr;
sum = CreateVector(&gstack, vSize);
sqr = CreateVector(&gstack, vSize);
occ = 0.0;
ZeroVector(sum);
ZeroVector(sqr);
.....
FreeVector(&gstack, sum);
}
In CalcClusterDistribution, Vector sum and sqr are created but only sum is freed.
void CalcDistance(CoList *list, RNode *ch1, RNode *ch2, int vSize)
{
int k;
CoList *c;
AccSum *acc;
float score1, score2;
Vector mean, sum1, sum2;
sum1 = CreateVector(&gstack, vSize);
ZeroVector(sum1);
sum2 = CreateVector(&gstack, vSize);
ZeroVector(sum2);
...
FreeVector(&gstack, sum1);
}
In CalcDistance, Vector sum1 and sum2 are created but only sum1 is freed.
void PrintRegTree(FILE *f, RegTree *t, int nNodes, char* rname, char* bname)
{
RegNode **tVec;
RNode *n;
int i,j;
tVec = (RegNode **) New(&gstack, nNodes*sizeof(RegNode *));
--tVec;
...
}
...
void PrintBaseClass(FILE *f, RegTree *t, int nNodes, char *bname)
{
CoList *c;
RNode *n;
int i, class=0;
RegNode **tVec;
tVec = (RegNode **) New(&gstack, nNodes*sizeof(RegNode *));
--tVec;
GetTreeVector(tVec, t->root);
...
}
In PrintRegTree and PrintBaseClass, RegNode ** tVec is created but not freed.
void ReOrderFeaturesCommand()
{
...
vSize = hset->vecSize;
frv = CreateIntVec(&gstack,vSize);
for (i=1; i<=vSize; i++)
frv[i] = ChkedInt("Feature index",1,vSize);
tmpm = CreateVector(&gstack,vSize);
tmpv = CreateVector(&gstack,vSize);
...
if (hset->xf != NULL) {
xform = hset->xf->xform->xform[1];
nc = NumCols(xform);
nr = hset->xf->xform->vecSize;
if (nr != vSize)
HError(2623,"ReOrderFeaturesCommand: Num rows %d in InputXForm does
not match the vecsize %d",nr,vSize);
tmpxf = CreateMatrix(&gstack,nr,nc);
for (i=1; i<=nr; i++)
for (j=1; j<=nc; j++)
tmpxf[i][j] = xform[frv[i]][j];
CopyMatrix(tmpxf,xform);
if (trace & T_BID) {
printf(" RF: Transform re-ordered according to %d, %d, %d,
...\n",frv[1],frv[2],frv[3]);
fflush(stdout);
}
}
if (trace & T_BID) {
printf(" RF: Parameters re-ordered according to %d, %d, %d,
...\n",frv[1],frv[2],frv[3]);
fflush(stdout);
}
FreeIntVec(&gstack,frv);
}
In ReOrderFeaturesCommand, IntVec frv, Vector tmpm and tmpv, and Matrix tmpxf
are created but only frv is freed.
I think these objects should be freed explicitly within the functions.
Best regards,
Heiga Zen (Byung Ha Chun)
-------- Original Message --------
Subject: Re: [HTK-Developers] Objects created but not freed in HHEd.c
Date: Mon, 26 Jun 2006 06:10:46 -0400
From: Gunnar Evermann <gunnar.evermann@xxxxxxxxx>
To: Heiga ZEN (Byung Ha CHUN) <zen@xxxxxxxxxxxxxxxx>
References: <449F7CE5.50808@xxxxxxxxxxxxxxxx>
On 6/26/06, Heiga ZEN (Byung Ha CHUN) <zen@xxxxxxxxxxxxxxxx> wrote:
> Dear HTK-Developers,
>
> I found that some objects were created but not freed explicitly
Heiga,
I didn't look at these in details, but in some of the cases the
allocation is done on a stack, so that freeing an object implicitly
frees everything allocated afterwards.
Gunnar
-------- Original Message --------
Subject: Re: [HTK-Developers] Objects created but not freed in HHEd.c
Date: Mon, 26 Jun 2006 21:38:22 +0900
From: Heiga ZEN (Byung Ha CHUN) <zen@xxxxxxxxxxxxxxxx>
Organization: Nagoya Institute of Technology
To: Gunnar Evermann <gunnar.evermann@xxxxxxxxx>
References: <449F7CE5.50808@xxxxxxxxxxxxxxxx>
<36909e520606260310i361159achef3ecad91f27592a@xxxxxxxxxxxxxx>
Hi Gunnar,
Thank you very much for replying :-)
Gunnar Evermann wrote:
> On 6/26/06, Heiga ZEN (Byung Ha CHUN) <zen@xxxxxxxxxxxxxxxx> wrote:
>> Dear HTK-Developers,
>>
>> I found that some objects were created but not freed explicitly
>
> Heiga,
>
> I didn't look at these in details, but in some of the cases the
> allocation is done on a stack, so that freeing an object implicitly
> frees everything allocated afterwards.
OK I see, definitely you're right, freeing an object in a stack will free
everything allocated afterwards.
But in my opinion ( philosophy? :-) ), objects which are created should be freed.
Heiga Zen (Byung Ha Chun)
-------- Original Message --------
Subject: Re: [HTK-Developers] Objects created but not freed in HHEd.c
Date: Mon, 26 Jun 2006 09:56:43 -0400
From: Gunnar Evermann <gunnar.evermann@xxxxxxxxx>
To: Heiga ZEN (Byung Ha CHUN) <zen@xxxxxxxxxxxxxxxx>
References: <449F7CE5.50808@xxxxxxxxxxxxxxxx>
<36909e520606260310i361159achef3ecad91f27592a@xxxxxxxxxxxxxx>
<449FD53E.9040308@xxxxxxxxxxxxxxxx>
On 6/26/06, Heiga ZEN (Byung Ha CHUN) <zen@xxxxxxxxxxxxxxxx> wrote:
> Hi Gunnar,
>
> Thank you very much for replying :-)
thank you for sending all these reports. As you know I am no longer
the maintainer of HTK, so I can't fix them myself in the official
version... :-(
> Gunnar Evermann wrote:
> > I didn't look at these in details, but in some of the cases the
> > allocation is done on a stack, so that freeing an object implicitly
> > frees everything allocated afterwards.
>
> OK I see, definitely you're right, freeing an object in a stack will free
everything allocated afterwards.
> But in my opinion ( philosophy? :-) ), objects which are created should be freed.
Yes, I guess I mostly agree, unless this is in some time-critical
function where freeing large chunks of structures in one go is useful.
However, you will find many places in HTK where the code makes use of
freeing lots of data at once.
Based on my experience, misunderstandings about the operation of the
HTK memory management was the biggest source of bugs when people (at
CUED and elsewhere) modified or extended HTK :-(
Gunnar
-------- Original Message --------
Subject: Re: [HTK-Developers] Objects created but not freed in HHEd.c
Date: Tue, 27 Jun 2006 09:44:41 +0100
From: M. J. F. Gales <mjfg100@xxxxxxxxxxxxx>
To: Heiga ZEN (Byung Ha CHUN) <zen@xxxxxxxxxxxxxxxx>
References: <449F7CE5.50808@xxxxxxxxxxxxxxxx>
Heiga,
Thanks for the comments. I am currently in the process of applying these
are other fixes to HTK V3.4 alpha to come up with an updated version.
I don't think that the memory issues you mention below are bugs. The
objects are created on stacks, so disposing of the first created object
ensures that all objects created after it are removed.
Are you obseverving severe memory leaks using HHEd?
Thanks
Mark
-------- Original Message --------
Subject: Re: [HTK-Developers] Objects created but not freed in HHEd.c
Date: Tue, 27 Jun 2006 18:03:05 +0900
From: Heiga ZEN (Byung Ha CHUN) <zen@xxxxxxxxxxxxxxxx>
Organization: Nagoya Institute of Technology
To: M. J. F. Gales <mjfg100@xxxxxxxxxxxxx>
References: <449F7CE5.50808@xxxxxxxxxxxxxxxx> <44A0EFF9.1050204@xxxxxxxxxxxxx>
Dear Mark,
M. J. F. Gales wrote:
> I don't think that the memory issues you mention below are bugs. The
> objects are created on stacks, so disposing of the first created object
> ensures that all objects created after it are removed.
You're right, Gunnar Evermann also pointed out this yesterday as follows:
On 6/26/06, Heiga ZEN (Byung Ha CHUN) <zen@xxxxxxxxxxxxxxxx> wrote:
> Dear HTK-Developers,
>
> I found that some objects were created but not freed explicitly
Heiga,
I didn't look at these in details, but in some of the cases the
allocation is done on a stack, so that freeing an object implicitly
frees everything allocated afterwards.
Gunnar
> Are you obseverving severe memory leaks using HHEd?
I just looked at the HHEd source code and thought that these objects were
created but not freed.
I should have reported this to htk-developers after checking memory leaks, sorry
:-(
Heiga Zen (Byung Ha Chun)
- Follow-Ups
-
- [hts-users:01514] Re: Possible memory leakage in HMGenS, Xuchen Yao
- References
-
- [hts-users:01512] Possible memory leakage in HMGenS, Xuchen Yao