[LTP] mtest01: free memory allocated

[LTP] mtest01: free memory allocated

[Francesco RUNDO]

[-- Attachment #1: Type: text/plain, Size: 632 bytes --]

Hi All,

I'm running LTP (I'm using ltp-full-20090731....but asap I will upgrade 
to latest)  on SH based platforms.
Now, during a test-session. I've noted that the test "mtest01" reduced 
drastically the system memory and after its execution this memory wasn't 
de-allocated.

I've analysed the mtest01.c code and I've noted that no "free()" 
istruction was associated to the related malloc:

......
if((mem = (char*)malloc(chunksize)) == NULL) {
......

I've simply added a "free(mem)" of the allocated memory and the issue 
was addressed successfully.

I've attached the trivial patch I've developed.

Best Regards,
--
Francesco

[-- Attachment #2: ltp-full-20090731-fix-mtest01_memory_issue.patch --]
[-- Type: text/plain, Size: 477 bytes --]

Added missed "free" istruction to release memory previosuly allocated.
Signed-off-by: Francesco Rundo <francesco.rundo@st.com>
--- ltp-full-20090731/testcases/kernel/mem/mtest01/mtest01.c.origin	2009-02-26 13:02:27.000000000 +0100
+++ ltp-full-20090731/testcases/kernel/mem/mtest01/mtest01.c	2009-12-15 14:03:28.779240000 +0100
@@ -258,5 +258,8 @@
     else
       tst_resm(TPASS, "%llu kbytes allocated only.", original_maxbytes/1024);
   }
+
+  free(mem);
+  
   exit(0);
 }

[-- Attachment #3: Type: text/plain, Size: 390 bytes --]

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 

[-- Attachment #4: Type: text/plain, Size: 155 bytes --]

_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] mtest01: free memory allocated

[Garrett Cooper]

On Jan 4, 2010, at 4:55 AM, Francesco RUNDO <francesco.rundo@st.com>  
wrote:

> Hi All,
>
> I'm running LTP (I'm using ltp-full-20090731....but asap I will  
> upgrade to latest)  on SH based platforms.
> Now, during a test-session. I've noted that the test "mtest01"  
> reduced drastically the system memory and after its execution this  
> memory wasn't de-allocated.
>
> I've analysed the mtest01.c code and I've noted that no "free()"  
> istruction was associated to the related malloc:
>
> ......
> if((mem = (char*)malloc(chunksize)) == NULL) {
> ......
>
> I've simply added a "free(mem)" of the allocated memory and the  
> issue was addressed successfully.
>
> I've attached the trivial patch I've developed.
>
> Best Regards,
> --
> Francesco
> Added missed "free" istruction to release memory previosuly allocated.
> Signed-off-by: Francesco Rundo <francesco.rundo@st.com>
> --- ltp-full-20090731/testcases/kernel/mem/mtest01/ 
> mtest01.c.origin    2009-02-26 13:02:27.000000000 +0100
> +++ ltp-full-20090731/testcases/kernel/mem/mtest01/mtest01.c     
> 2009-12-15 14:03:28.779240000 +0100
> @@ -258,5 +258,8 @@
>     else
>       tst_resm(TPASS, "%llu kbytes allocated only.",  
> original_maxbytes/1024);
>   }
> +
> +  free(mem);
> +
>   exit(0);
> }

1. Does the version off cvs still have this issue?
2. Unless the test exits the code block immediately, I'd definitely do:

if (mem)
     free (mem);

to avoid making a bad free call.

Thanks,
-Garrett

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] mtest01: free memory allocated

[Li Zefan]

> 2. Unless the test exits the code block immediately, I'd definitely do:
> 
> if (mem)
>      free (mem);
> 
> to avoid making a bad free call.
> 

free(NULL) is valid.


------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] mtest01: free memory allocated

[Subrata Modak]

Garrett,

On Mon, 2010-01-04 at 18:10 -0800, Garrett Cooper wrote: 
> On Jan 4, 2010, at 4:55 AM, Francesco RUNDO <francesco.rundo@st.com>  
> wrote:
> 
> > Hi All,
> >
> > I'm running LTP (I'm using ltp-full-20090731....but asap I will  
> > upgrade to latest)  on SH based platforms.
> > Now, during a test-session. I've noted that the test "mtest01"  
> > reduced drastically the system memory and after its execution this  
> > memory wasn't de-allocated.
> >
> > I've analysed the mtest01.c code and I've noted that no "free()"  
> > istruction was associated to the related malloc:
> >
> > ......
> > if((mem = (char*)malloc(chunksize)) == NULL) {
> > ......
> >
> > I've simply added a "free(mem)" of the allocated memory and the  
> > issue was addressed successfully.
> >
> > I've attached the trivial patch I've developed.
> >
> > Best Regards,
> > --
> > Francesco
> > Added missed "free" istruction to release memory previosuly allocated.
> > Signed-off-by: Francesco Rundo <francesco.rundo@st.com>
> > --- ltp-full-20090731/testcases/kernel/mem/mtest01/ 
> > mtest01.c.origin    2009-02-26 13:02:27.000000000 +0100
> > +++ ltp-full-20090731/testcases/kernel/mem/mtest01/mtest01.c     
> > 2009-12-15 14:03:28.779240000 +0100
> > @@ -258,5 +258,8 @@
> >     else
> >       tst_resm(TPASS, "%llu kbytes allocated only.",  
> > original_maxbytes/1024);
> >   }
> > +
> > +  free(mem);
> > +
> >   exit(0);
> > }
> 
> 1. Does the version off cvs still have this issue?
> 2. Unless the test exits the code block immediately, I'd definitely do:
> 
> if (mem)
>      free (mem);
> 
> to avoid making a bad free call.

Have you made any changes to this ? I do not see any commits.

Regards--
Subrata

> 
> Thanks,
> -Garrett
> 
> ------------------------------------------------------------------------------
> This SF.Net email is sponsored by the Verizon Developer Community
> Take advantage of Verizon's best-in-class app development support
> A streamlined, 14 day to market process makes app distribution fast and easy
> Join now and get one step closer to millions of Verizon customers
> http://p.sf.net/sfu/verizon-dev2dev 
> _______________________________________________
> Ltp-list mailing list
> Ltp-list@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/ltp-list


------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] mtest01: free memory allocated

[Garrett Cooper]

On Jan 7, 2010, at 3:36 AM, Subrata Modak wrote:

> Garrett,
> 
> On Mon, 2010-01-04 at 18:10 -0800, Garrett Cooper wrote: 
>> On Jan 4, 2010, at 4:55 AM, Francesco RUNDO <francesco.rundo@st.com>  
>> wrote:
>> 
>>> Hi All,
>>> 
>>> I'm running LTP (I'm using ltp-full-20090731....but asap I will  
>>> upgrade to latest)  on SH based platforms.
>>> Now, during a test-session. I've noted that the test "mtest01"  
>>> reduced drastically the system memory and after its execution this  
>>> memory wasn't de-allocated.
>>> 
>>> I've analysed the mtest01.c code and I've noted that no "free()"  
>>> istruction was associated to the related malloc:
>>> 
>>> ......
>>> if((mem = (char*)malloc(chunksize)) == NULL) {
>>> ......
>>> 
>>> I've simply added a "free(mem)" of the allocated memory and the  
>>> issue was addressed successfully.
>>> 
>>> I've attached the trivial patch I've developed.
>>> 
>>> Best Regards,
>>> --
>>> Francesco
>>> Added missed "free" istruction to release memory previosuly allocated.
>>> Signed-off-by: Francesco Rundo <francesco.rundo@st.com>
>>> --- ltp-full-20090731/testcases/kernel/mem/mtest01/ 
>>> mtest01.c.origin    2009-02-26 13:02:27.000000000 +0100
>>> +++ ltp-full-20090731/testcases/kernel/mem/mtest01/mtest01.c     
>>> 2009-12-15 14:03:28.779240000 +0100
>>> @@ -258,5 +258,8 @@
>>>    else
>>>      tst_resm(TPASS, "%llu kbytes allocated only.",  
>>> original_maxbytes/1024);
>>>  }
>>> +
>>> +  free(mem);
>>> +
>>>  exit(0);
>>> }
>> 
>> 1. Does the version off cvs still have this issue?
>> 2. Unless the test exits the code block immediately, I'd definitely do:
>> 
>> if (mem)
>>     free (mem);
>> 
>> to avoid making a bad free call.
> 
> Have you made any changes to this ? I do not see any commits.
> 
> Regards--
> Subrata

Nope. I've left it untouched because of the discussion that was being made in the thread.
Thanks,
-Garrett
> 


------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] mtest01: free memory allocated

[Francesco RUNDO]

[-- Attachment #1.1: Type: text/plain, Size: 3075 bytes --]

Dear All,

Ok, likely a sort of flag has to be added in the "mtest01.c" 
code.....something like that:

mem_flag=0;
....................

if((mem = (char*)malloc(chunksize)) == NULL) {

.......

exit(1)
}

mem_flag=1:

.......


in order to understand if the malloc() has been executed or not.....and 
then the patch will be:

if ((mem_flag==1)&&(mem != NULL))
{
         free(mem);
         mem_flag=0;
}

Without a patch as above (or like that), in the code of  "mtest01" we 
got scenarios (mainly in the cross architectures like embedded systems) 
in which the LTP run a "malloc()" without "free()".
This is always wrong even though the system resource is enough to cover 
this memory consumption.

Regards
--
FR


Subrata Modak wrote:

>Garrett,
>
>On Mon, 2010-01-04 at 18:10 -0800, Garrett Cooper wrote: 
>  
>
>>On Jan 4, 2010, at 4:55 AM, Francesco RUNDO <francesco.rundo@st.com>  
>>wrote:
>>
>>    
>>
>>>Hi All,
>>>
>>>I'm running LTP (I'm using ltp-full-20090731....but asap I will  
>>>upgrade to latest)  on SH based platforms.
>>>Now, during a test-session. I've noted that the test "mtest01"  
>>>reduced drastically the system memory and after its execution this  
>>>memory wasn't de-allocated.
>>>
>>>I've analysed the mtest01.c code and I've noted that no "free()"  
>>>istruction was associated to the related malloc:
>>>
>>>......
>>>if((mem = (char*)malloc(chunksize)) == NULL) {
>>>......
>>>
>>>I've simply added a "free(mem)" of the allocated memory and the  
>>>issue was addressed successfully.
>>>
>>>I've attached the trivial patch I've developed.
>>>
>>>Best Regards,
>>>--
>>>Francesco
>>>Added missed "free" istruction to release memory previosuly allocated.
>>>Signed-off-by: Francesco Rundo <francesco.rundo@st.com>
>>>--- ltp-full-20090731/testcases/kernel/mem/mtest01/ 
>>>mtest01.c.origin    2009-02-26 13:02:27.000000000 +0100
>>>+++ ltp-full-20090731/testcases/kernel/mem/mtest01/mtest01.c     
>>>2009-12-15 14:03:28.779240000 +0100
>>>@@ -258,5 +258,8 @@
>>>    else
>>>      tst_resm(TPASS, "%llu kbytes allocated only.",  
>>>original_maxbytes/1024);
>>>  }
>>>+
>>>+  free(mem);
>>>+
>>>  exit(0);
>>>}
>>>      
>>>
>>1. Does the version off cvs still have this issue?
>>2. Unless the test exits the code block immediately, I'd definitely do:
>>
>>if (mem)
>>     free (mem);
>>
>>to avoid making a bad free call.
>>    
>>
>
>Have you made any changes to this ? I do not see any commits.
>
>Regards--
>Subrata
>
>  
>
>>Thanks,
>>-Garrett
>>
>>------------------------------------------------------------------------------
>>This SF.Net email is sponsored by the Verizon Developer Community
>>Take advantage of Verizon's best-in-class app development support
>>A streamlined, 14 day to market process makes app distribution fast and easy
>>Join now and get one step closer to millions of Verizon customers
>>http://p.sf.net/sfu/verizon-dev2dev 
>>_______________________________________________
>>Ltp-list mailing list
>>Ltp-list@lists.sourceforge.net
>>https://lists.sourceforge.net/lists/listinfo/ltp-list
>>    
>>
>
>
>  
>

[-- Attachment #1.2: Type: text/html, Size: 4169 bytes --]

[-- Attachment #2: Type: text/plain, Size: 390 bytes --]

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 

[-- Attachment #3: Type: text/plain, Size: 155 bytes --]

_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] mtest01: free memory allocated

[Jiri Palecek]

Francesco RUNDO napsal(a):
> Hi All,
>
> I'm running LTP (I'm using ltp-full-20090731....but asap I will upgrade
> to latest) on SH based platforms.
> Now, during a test-session. I've noted that the test "mtest01" reduced
> drastically the system memory and after its execution this memory wasn't
> de-allocated.
>
> I've analysed the mtest01.c code and I've noted that no "free()"
> istruction was associated to the related malloc:

Does this mean the kernel doesn't free processes' allocated memory on exit? Is 
this intentional (and documented somewhere)?

> ......
> if((mem = (char*)malloc(chunksize)) == NULL) {
> ......
>
> I've simply added a "free(mem)" of the allocated memory and the issue
> was addressed successfully.

This isn't complete by far. You don't free all the allocations, and there are 
code paths which don't pass your line before exit.

Regards
     Jiri Palecek

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] mtest01: free memory allocated

[Henry Yei]

> -----Original Message-----
> From: Jiri Palecek [mailto:jpalecek@web.de]
> Sent: Tuesday, January 05, 2010 5:20 PM
> To: Francesco RUNDO
> Cc: LTP
> Subject: Re: [LTP] mtest01: free memory allocated
> 
> Francesco RUNDO napsal(a):
> > Hi All,
> >
> > I'm running LTP (I'm using ltp-full-20090731....but asap I will
> upgrade
> > to latest) on SH based platforms.
> > Now, during a test-session. I've noted that the test "mtest01"
> reduced
> > drastically the system memory and after its execution this memory
> wasn't
> > de-allocated.
> >
> > I've analysed the mtest01.c code and I've noted that no "free()"
> > istruction was associated to the related malloc:
> 
> Does this mean the kernel doesn't free processes' allocated memory on
> exit? Is
> this intentional (and documented somewhere)?
> 
> > ......
> > if((mem = (char*)malloc(chunksize)) == NULL) {
> > ......
> >
> > I've simply added a "free(mem)" of the allocated memory and the issue
> > was addressed successfully.
> 
> This isn't complete by far. You don't free all the allocations, and
> there are
> code paths which don't pass your line before exit.
> 
> Regards
>      Jiri Palecek
> 

We've documented this behavior internally as a low priority issue for a while now. 
This is the root cause one of our developers came back with, which might clarify what Francesco is seeing:

Issue:

Across architectures, mtest tests sometimes report a fail because they won't
run unless less than 80% of memory is used. This is because the test attempts
to fill up the memory to the 80% limit specified by the -p option.

Some previous test in the LTP suite is taking up memory it should not have, as
this happens even on machines with 1GB of memory.

mtest01 runs "mtest01 -p80" which mallocs chunks of memory until 80% of memory
is used.
mtest01w runs "mtest01 -p80 -w" which mallocs and writes chunks of memory until
80% of memory is used.


Response:

mtest01 -p80 -w; mtest01 -p80 -w; free; free; free... ; mtest01 -p80 -w

on the same command line. the second mtest01 always returns with a failure.
while the mtest01 after enough free(usually 4 or 5) always passes.

Looks like it is because of the aging of the caches, which are not freed yet
when the second mtest01 is executed and are dropped due to aging by the time
last mtest01 is executed.
The information (sysinfo()) mtest is using to determine the amount of memory
allocated does not include the memory in caches(free). mtest01 sees that 80% of
the memory is already in use and thus do not go for malloc-ing and returns with
a failure, however if it had tried, the malloc would have succeeded because of
the reuse of the free slab caches.

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] mtest01: free memory allocated

[Jiri Palecek]

Henry Yei napsal(a):
>> -----Original Message-----
>> From: Jiri Palecek [mailto:jpalecek@web.de]
>> Sent: Tuesday, January 05, 2010 5:20 PM
>> To: Francesco RUNDO
>> Cc: LTP
>> Subject: Re: [LTP] mtest01: free memory allocated
>>
>> Francesco RUNDO napsal(a):
>>> Hi All,
>>>
>>> I'm running LTP (I'm using ltp-full-20090731....but asap I will
>> upgrade
>>> to latest) on SH based platforms.
>>> Now, during a test-session. I've noted that the test "mtest01"
>> reduced
>>> drastically the system memory and after its execution this memory
>> wasn't
>>> de-allocated.
>>>
>>> I've analysed the mtest01.c code and I've noted that no "free()"
>>> istruction was associated to the related malloc:
>>
>> Does this mean the kernel doesn't free processes' allocated memory on
>> exit? Is
>> this intentional (and documented somewhere)?
>>
>>> ......
>>> if((mem = (char*)malloc(chunksize)) == NULL) {
>>> ......
>>>
>>> I've simply added a "free(mem)" of the allocated memory and the issue
>>> was addressed successfully.
>>
>> This isn't complete by far. You don't free all the allocations, and
>> there are
>> code paths which don't pass your line before exit.
>>
>> Regards
>>       Jiri Palecek
>>
>
> We've documented this behavior internally as a low priority issue for a while now.
> This is the root cause one of our developers came back with, which might clarify what Francesco is seeing:
>
> Issue:
>
> Across architectures, mtest tests sometimes report a fail because they won't
> run unless less than 80% of memory is used. This is because the test attempts
> to fill up the memory to the 80% limit specified by the -p option.
>
> Some previous test in the LTP suite is taking up memory it should not have, as
> this happens even on machines with 1GB of memory.
>
> mtest01 runs "mtest01 -p80" which mallocs chunks of memory until 80% of memory
> is used.
> mtest01w runs "mtest01 -p80 -w" which mallocs and writes chunks of memory until
> 80% of memory is used.
>
>
> Response:
>
> mtest01 -p80 -w; mtest01 -p80 -w; free; free; free... ; mtest01 -p80 -w
>
> on the same command line. the second mtest01 always returns with a failure.
> while the mtest01 after enough free(usually 4 or 5) always passes.

Indeed, I reproduced it on a machine without swap (is this reproducible with 
swap?), although the second mtest didn't fail, but just allocated ridiculously 
small amount of memory. However, I'm curious if this is what Francesco is seeing 
(eg. if the memory is eventually freed on his machine).

> Looks like it is because of the aging of the caches, which are not freed yet
> when the second mtest01 is executed and are dropped due to aging by the time
> last mtest01 is executed.
> The information (sysinfo()) mtest is using to determine the amount of memory
> allocated does not include the memory in caches(free). mtest01 sees that 80% of
> the memory is already in use and thus do not go for malloc-ing and returns with
> a failure, however if it had tried, the malloc would have succeeded because of
> the reuse of the free slab caches.

I'm afraid this isn't related to slab caches, as slabinfo between the two tests 
doesn't show anything suspicious (it's logical, too - if the test just before 
consumed 80% of memory in user pages, it's unlikely the memory would be 
allocated to slab after that).

Regards
     Jiri Palecek

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list