Getting Out of Memory errors at random intervals.

Getting Out of Memory errors at random intervals.

[Andrew Brink]

Hi. I've been getting random Out of Memory: Killing Processs pid (name)
On _two_ different boxes. One is running 2.4.16 while the other 
is 2.4.17.  Most of the time the process that gets killed is apache
(these boxen are webservers) but sometimes its mysql, exim, htdig and
others.

These boxes used to be rock stable, and now they get the OOM error at
random periodic times.

Is there anything I can check for, or do to clear this up?

Any help would be greatly appreciated.
Thanks,
Andrew Brink

Re: Getting Out of Memory errors at random intervals.

[Andrew Brink]

On Sun, Jan 13, 2002 at 11:45:42PM +0000, Alan Cox wrote:
> Run something that has a sane VM in all the known awkward cases (eg the Red Hat
> 2.4.9 tree) and you should be just fine. If not I'd be interested to know

I take it the vanilla 2.4.9 would also do?

> 
> For the newer trees I can only get stability under high loads with either the
> Andrea -aa vm patches or Rik's rmap-11b patch. Both of which seem to help
> no end.

As for High loads....these boxes don't even get a load.

Andrew

Re: Getting Out of Memory errors at random intervals.

[Alan Cox]

> Hi. I've been getting random Out of Memory: Killing Processs pid (name)
> On _two_ different boxes. One is running 2.4.16 while the other 
> is 2.4.17.  Most of the time the process that gets killed is apache
> (these boxen are webservers) but sometimes its mysql, exim, htdig and
> others.
> 
> These boxes used to be rock stable, and now they get the OOM error at
> random periodic times.
> 
> Is there anything I can check for, or do to clear this up?

Run something that has a sane VM in all the known awkward cases (eg the Red Hat
2.4.9 tree) and you should be just fine. If not I'd be interested to know

For the newer trees I can only get stability under high loads with either the
Andrea -aa vm patches or Rik's rmap-11b patch. Both of which seem to help
no end.

Alan

Re: Getting Out of Memory errors at random intervals.

[Alan Cox]

> On Sun, Jan 13, 2002 at 11:45:42PM +0000, Alan Cox wrote:
> > Run something that has a sane VM in all the known awkward cases (eg the Red Hat
> > 2.4.9 tree) and you should be just fine. If not I'd be interested to know
> 
> I take it the vanilla 2.4.9 would also do?

Nothing like it. The 2.4.9-RH tree is very different VM wise from the 2.4.9
base tree. Linus never took the VM updates from it.

> > Andrea -aa vm patches or Rik's rmap-11b patch. Both of which seem to help
> > no end.
> 
> As for High loads....these boxes don't even get a load.

I suspect they are - possibly only a sudden sharp burst of web traffic
causing a lot of cgi/mysql/apache process activity.

Alan

Re: Getting Out of Memory errors at random intervals.

[Andrew Brink]

On Mon, Jan 14, 2002 at 12:09:14AM +0000, Alan Cox wrote:
> > On Sun, Jan 13, 2002 at 11:45:42PM +0000, Alan Cox wrote:
> > > Run something that has a sane VM in all the known awkward cases (eg the Red Hat
> > > 2.4.9 tree) and you should be just fine. If not I'd be interested to know
> > 
> > I take it the vanilla 2.4.9 would also do?
> 
> Nothing like it. The 2.4.9-RH tree is very different VM wise from the 2.4.9
> base tree. Linus never took the VM updates from it.

Okay. Will do.

> 
> > > Andrea -aa vm patches or Rik's rmap-11b patch. Both of which seem to help
> > > no end.
> > 
> > As for High loads....these boxes don't even get a load.
> 
> I suspect they are - possibly only a sudden sharp burst of web traffic
> causing a lot of cgi/mysql/apache process activity.

*Shrug* I've done some experimenting with this, having a lab (30 people)
all hit the site at the same time. Holds it fine most the time.  Usually
the OOM's come during the middle of the night.

> 
> Alan

Re: Getting Out of Memory errors at random intervals.

[Andrew Brink]

On second thought, I went and reviewed some logs.

Happened a lot on one box around 8ish.

On Mon, Jan 14, 2002 at 12:44:51AM +0000, Alan Cox wrote:
> > *Shrug* I've done some experimenting with this, having a lab (30 people)
> > all hit the site at the same time. Holds it fine most the time.  Usually
> > the OOM's come during the middle of the night.
> 
> About 4am by any chance ?

Re: Getting Out of Memory errors at random intervals.

[Alan Cox]

> *Shrug* I've done some experimenting with this, having a lab (30 people)
> all hit the site at the same time. Holds it fine most the time.  Usually
> the OOM's come during the middle of the night.

About 4am by any chance ?

Re: Getting Out of Memory errors at random intervals.

[Andrew Brink]

Will do. However, might be a while, since this isn't reproducable on
demand.

On Mon, Jan 14, 2002 at 12:52:28AM +0000, Alan Cox wrote:
> > On Mon, Jan 14, 2002 at 12:44:51AM +0000, Alan Cox wrote:
> > > > *Shrug* I've done some experimenting with this, having a lab (30 people)
> > > > all hit the site at the same time. Holds it fine most the time.  Usually
> > > > the OOM's come during the middle of the night.
> > > 
> > > About 4am by any chance ?
> > 
> > On second thought, I went and reviewed some logs.
> > Happened a lot on one box around 8ish.
> > 
> 
> Ok. Let me know how trying the other things work (also the list). I'm sure
> Andrea, Marcelo and Rik all want to look at these cases

Re: Getting Out of Memory errors at random intervals.

[Alan Cox]

> On Mon, Jan 14, 2002 at 12:44:51AM +0000, Alan Cox wrote:
> > > *Shrug* I've done some experimenting with this, having a lab (30 people)
> > > all hit the site at the same time. Holds it fine most the time.  Usually
> > > the OOM's come during the middle of the night.
> > 
> > About 4am by any chance ?
> 
> On second thought, I went and reviewed some logs.
> Happened a lot on one box around 8ish.
> 

Ok. Let me know how trying the other things work (also the list). I'm sure
Andrea, Marcelo and Rik all want to look at these cases