Problem with OOM killer killing process even when there is plenty of RAM.

Problem with OOM killer killing process even when there is plenty of RAM.

[mindentropy]

Hi,

  I am trying to allocate 512MB of RAM in my driver loaded as a module, but 
the OOM killer starts killing all my processes. This machine has around 24GB 
RAM and is a 8 core Xeon. The RAM is allocated in page size chunks (i.e. 
131072 chunks each of size PAGE_SIZE). I am using a 32 bit kernel with PAE 
enabled. The allocation works fine on a machine with 8GB of RAM.

Thanks.

Problem with OOM killer killing process even when there is plenty of RAM.

[Rajat Sharma]

driver allocating 512M !!?? what kind of driver it is?

-Rajat

On Wed, Dec 7, 2011 at 12:17 AM, mindentropy <mindentropy@gmail.com> wrote:
> Hi,
>
> ?I am trying to allocate 512MB of RAM in my driver loaded as a module, but
> the OOM killer starts killing all my processes. This machine has around 24GB
> RAM and is a 8 core Xeon. The RAM is allocated in page size chunks (i.e.
> 131072 chunks each of size PAGE_SIZE). I am using a 32 bit kernel with PAE
> enabled. The allocation works fine on a machine with 8GB of RAM.
>
> Thanks.
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Problem with OOM killer killing process even when there is plenty of RAM.

[mindentropy]

On Wednesday 07 Dec 2011 11:07:00 AM Rajat Sharma wrote:
> driver allocating 512M !!?? what kind of driver it is?
> 
> -Rajat
> 
For a high speed frame grabber. The data would be in bursts the queue is meant 
to smoothen the pressure. I am not sure why its surprising since I am using 
only 2% of the memory (Only 63MB is being used overall by the system).

Problem with OOM killer killing process even when there is plenty of RAM.

[Mulyadi Santosa]

On Wed, Dec 7, 2011 at 01:47, mindentropy <mindentropy@gmail.com> wrote:
> Hi,
>
> ?I am trying to allocate 512MB of RAM in my driver loaded as a module, but
> the OOM killer starts killing all my processes. This machine has around 24GB
> RAM and is a 8 core Xeon. The RAM is allocated in page size chunks (i.e.
> 131072 chunks each of size PAGE_SIZE).

mind to tell us, how do you allocate memory? kmalloc?

furthermore, could you kindly paste the content of /proc/meminfo right
before OOM kicks in?

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Problem with OOM killer killing process even when there is plenty of RAM.

[mindentropy]

On Thursday 08 Dec 2011 1:38:19 AM Mulyadi Santosa wrote:
> On Wed, Dec 7, 2011 at 01:47, mindentropy <mindentropy@gmail.com> wrote:
> > Hi,
> > 
> >  I am trying to allocate 512MB of RAM in my driver loaded as a module,
> > but the OOM killer starts killing all my processes. This machine has
> > around 24GB RAM and is a 8 core Xeon. The RAM is allocated in page size
> > chunks (i.e. 131072 chunks each of size PAGE_SIZE).
> 
> mind to tell us, how do you allocate memory? kmalloc?
Via kmalloc. 

> 
> furthermore, could you kindly paste the content of /proc/meminfo right
> before OOM kicks in?
Its kind of tough to do it as it happens so fast. I will try to write a script 
which polls.

Problem with OOM killer killing process even when there is plenty of RAM.

[Greg KH]

On Thu, Dec 08, 2011 at 12:36:43AM +0530, mindentropy wrote:
> On Thursday 08 Dec 2011 1:38:19 AM Mulyadi Santosa wrote:
> > On Wed, Dec 7, 2011 at 01:47, mindentropy <mindentropy@gmail.com> wrote:
> > > Hi,
> > > 
> > >  I am trying to allocate 512MB of RAM in my driver loaded as a module,
> > > but the OOM killer starts killing all my processes. This machine has
> > > around 24GB RAM and is a 8 core Xeon. The RAM is allocated in page size
> > > chunks (i.e. 131072 chunks each of size PAGE_SIZE).
> > 
> > mind to tell us, how do you allocate memory? kmalloc?
> Via kmalloc. 

Please use the functions written to give you large memory chunks.  As
you are using a video-for-linux device, why not use the apis that this
framework provides you for this type of thing?

thanks,

greg k-h

Problem with OOM killer killing process even when there is plenty of RAM.

[mindentropy]

On Wednesday 07 Dec 2011 11:35:36 AM Greg KH wrote:
> On Thu, Dec 08, 2011 at 12:36:43AM +0530, mindentropy wrote:
> > On Thursday 08 Dec 2011 1:38:19 AM Mulyadi Santosa wrote:
> > > On Wed, Dec 7, 2011 at 01:47, mindentropy <mindentropy@gmail.com> wrote:
> > > > Hi,
> > > > 
> > > >  I am trying to allocate 512MB of RAM in my driver loaded as a
> > > >  module,> > > 
> > > > but the OOM killer starts killing all my processes. This machine
> > > > has
> > > > around 24GB RAM and is a 8 core Xeon. The RAM is allocated in
> > > > page size chunks (i.e. 131072 chunks each of size PAGE_SIZE).
> > > 
> > > mind to tell us, how do you allocate memory? kmalloc?
> > 
> > Via kmalloc.
> 
> Please use the functions written to give you large memory chunks.  As
> you are using a video-for-linux device, why not use the apis that this
> framework provides you for this type of thing?
> 

Yes V4L would be done next. There are some things being tried out, this being 
a custom FPGA hence some controls being experimented.

As for the allocating in PAGE_SIZE chunks why am I running out of memory? 
Can't it steal smaller chunks from the larger chunks? Just curious about the 
reason.

Problem with OOM killer killing process even when there is plenty of RAM.

[Greg KH]

On Thu, Dec 08, 2011 at 01:26:42AM +0530, mindentropy wrote:
> On Wednesday 07 Dec 2011 11:35:36 AM Greg KH wrote:
> > On Thu, Dec 08, 2011 at 12:36:43AM +0530, mindentropy wrote:
> > > On Thursday 08 Dec 2011 1:38:19 AM Mulyadi Santosa wrote:
> > > > On Wed, Dec 7, 2011 at 01:47, mindentropy <mindentropy@gmail.com> wrote:
> > > > > Hi,
> > > > > 
> > > > >  I am trying to allocate 512MB of RAM in my driver loaded as a
> > > > >  module,> > > 
> > > > > but the OOM killer starts killing all my processes. This machine
> > > > > has
> > > > > around 24GB RAM and is a 8 core Xeon. The RAM is allocated in
> > > > > page size chunks (i.e. 131072 chunks each of size PAGE_SIZE).
> > > > 
> > > > mind to tell us, how do you allocate memory? kmalloc?
> > > 
> > > Via kmalloc.
> > 
> > Please use the functions written to give you large memory chunks.  As
> > you are using a video-for-linux device, why not use the apis that this
> > framework provides you for this type of thing?
> > 
> 
> Yes V4L would be done next. There are some things being tried out, this being 
> a custom FPGA hence some controls being experimented.
> 
> As for the allocating in PAGE_SIZE chunks why am I running out of memory? 

The kernel doesn't like allocating such large sizes all at once.

> Can't it steal smaller chunks from the larger chunks? Just curious about the 
> reason.

Use vmalloc if you need/want larger chunks, kmalloc is not for that at
all.

greg k-h

Problem with OOM killer killing process even when there is plenty of RAM.

[Rajat Sharma]

On Thu, Dec 8, 2011 at 1:41 AM, Greg KH <greg@kroah.com> wrote:
> On Thu, Dec 08, 2011 at 01:26:42AM +0530, mindentropy wrote:
>> On Wednesday 07 Dec 2011 11:35:36 AM Greg KH wrote:
>> > On Thu, Dec 08, 2011 at 12:36:43AM +0530, mindentropy wrote:
>> > > On Thursday 08 Dec 2011 1:38:19 AM Mulyadi Santosa wrote:
>> > > > On Wed, Dec 7, 2011 at 01:47, mindentropy <mindentropy@gmail.com> wrote:
>> > > > > Hi,
>> > > > >
>> > > > > ?I am trying to allocate 512MB of RAM in my driver loaded as a
>> > > > > ?module,> > >
>> > > > > but the OOM killer starts killing all my processes. This machine
>> > > > > has
>> > > > > around 24GB RAM and is a 8 core Xeon. The RAM is allocated in
>> > > > > page size chunks (i.e. 131072 chunks each of size PAGE_SIZE).
>> > > >
>> > > > mind to tell us, how do you allocate memory? kmalloc?
>> > >
>> > > Via kmalloc.
>> >
>> > Please use the functions written to give you large memory chunks. ?As
>> > you are using a video-for-linux device, why not use the apis that this
>> > framework provides you for this type of thing?
>> >
>>
>> Yes V4L would be done next. There are some things being tried out, this being
>> a custom FPGA hence some controls being experimented.
>>
>> As for the allocating in PAGE_SIZE chunks why am I running out of memory?
>
> The kernel doesn't like allocating such large sizes all at once.
>
>> Can't it steal smaller chunks from the larger chunks? Just curious about the
>> reason.
>
> Use vmalloc if you need/want larger chunks, kmalloc is not for that at
> all.
>
> greg k-h
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Probably if it is some custom hardware, and you can afford to allocate
memory at boot time, then reserve it exclusively for your use only.

-Rajat

Problem with OOM killer killing process even when there is plenty of RAM.

[Mulyadi Santosa]

Hi...

On Thu, Dec 8, 2011 at 02:06, mindentropy <mindentropy@gmail.com> wrote:
>> mind to tell us, how do you allocate memory? kmalloc?
> Via kmalloc.

As Greg said, you probably better use vmalloc for such allocation.
Vmalloc can deal with non contigous physical pages...

>>
>> furthermore, could you kindly paste the content of /proc/meminfo right
>> before OOM kicks in?
> Its kind of tough to do it as it happens so fast. I will try to write a script
> which polls.

Alright, I just thought it will be better if we understand the overall
virtual memory situation better.


-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Problem with OOM killer killing process even when there is plenty of RAM.

[Pritam Bankar]

Hi,
use of
get_zeroed_page(unsigned int flags);  or
get_free_page(unsigned int flags);
might help you.


Thanks,
Pritam

On Thu, Dec 8, 2011 at 9:47 PM, Mulyadi Santosa
<mulyadi.santosa@gmail.com>wrote:

> Hi...
>
> On Thu, Dec 8, 2011 at 02:06, mindentropy <mindentropy@gmail.com> wrote:
> >> mind to tell us, how do you allocate memory? kmalloc?
> > Via kmalloc.
>
> As Greg said, you probably better use vmalloc for such allocation.
> Vmalloc can deal with non contigous physical pages...
>
> >>
> >> furthermore, could you kindly paste the content of /proc/meminfo right
> >> before OOM kicks in?
> > Its kind of tough to do it as it happens so fast. I will try to write a
> script
> > which polls.
>
> Alright, I just thought it will be better if we understand the overall
> virtual memory situation better.
>
>
> --
> regards,
>
> Mulyadi Santosa
> Freelance Linux trainer and consultant
>
> blog: the-hydra.blogspot.com
> training: mulyaditraining.blogspot.com
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>



-- 

Pritam Bankar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20111209/1ddbc396/attachment.html