Re: OOM killer

Re: OOM killer

[Jesper Juhl]

Thomas Hood wrote:

> How about assigning each process a property similar to its niceness
> which would be used to decide which process to kill in the event of
> OOM?

Or maybe make it a configure option if Linux should over commit memory or
not.
In some cases it would be nice if you could be sure that the memory you got
was actually there, and for those cases you could build the kernel with
CONFIG_NO_MEM_OVERCOMMIT (or something like that) so that linux would simply
report ENOMEM when there's no more memory.


Best regards,
Jesper Juhl
juhl@eisenstein.dk

Re: OOM killer

[Jesper Juhl]

Alex Bligh - linux-kernel wrote:

> shed:~# cat /proc/sys/vm/overcommit_memory
> 0
> shed:~# echo 1 >/proc/sys/vm/overcommit_memory
> shed:~# cat /proc/sys/vm/overcommit_memory
> 1
> shed:~# echo 0 >/proc/sys/vm/overcommit_memory
> shed:~# cat /proc/sys/vm/overcommit_memory
> 0

ahh, I see. Well, you live and learn ;)

I think I've got to do my research better before writing mails to lkml.



Best regards,
Jesper Juhl
juhl@eisenstein.dk

OOM killer

[Thomas Hood]

After meeting the OOM killer for the first time (not a pleasant meeting)
I went looking for info about it and read through some of the threads
that took place on l-k a few months ago.  I'm sure I didn't read
everything, and I am far from being an expert in this area, but are my
two cents anyway.

It is all very well to have a "smart" function that tries to
minimize the damage done when the OS has to start killing
processes in order to recover memory.  However:

1) It's better if this situation doesn't arise in the first place, and,
2) once it does arise, it's better to let the administrator decide 
   what to kill.  Sometimes the administrator will get fired for killing
   the database and sometimes he'll get fired for killing Netscape---
   there is no way for the authors of Linux to know in advance.

Some OSes don't allow memory overcommit at all.  On these OSes, a process
will simply fail if it tries to allocate memory that's not available.
That Linux allows processes to go ahead under such circumstances can,
I suppose, be called a 'feature'.  But the result of that feature is
that OOM can occur and then a kill decision has to be made.  How this
decision is made ought to be under the administrator's control.

How about assigning each process a property similar to its niceness
which would be used to decide which process to kill in the event of
OOM?  Let's call this property 'humility'.  By default, processes 
would run with humility zero.  A process run with negative humility
would never be allowed to proceed unless there was enough VM to back
up its memory request.  A process with non-negative humility would
be allowed to proceed under such circumstances, but it would be
taking the chance that it would be killed later.

So the system starts to run out of memory.  If all processes have the
same humility then the OOM-killer adjudicator is left to decide among
them just as it does now.  Those with negative humility will never be
killed, and they will never have to be killed because all the memory
they allocated really exists.  Among remaining process, the humblest
get killed first; among those equally humble, the baddest get killed.

So someday in the future I fire up my webserver and start Apache with
negative humility since it's a mission-critical app.  My boss logs in
and starts writing e-mail (humility 0 process).  There's memory 
pressure, so I take the precaution of starting Netscape with
    $ humble communicator
to make sure that communicator gets the axe if we run oom.

Your humble servant,
Thomas Hood

Re: OOM killer

[Alex Bligh - linux-kernel]

--On Wednesday, 26 September, 2001 7:11 PM +0200 Jesper Juhl 
<juhl@eisenstein.dk> wrote:

> Or maybe make it a configure option if Linux should over commit memory or
> not.

deja vu

shed:~# cat /proc/sys/vm/overcommit_memory
0
shed:~# echo 1 >/proc/sys/vm/overcommit_memory
shed:~# cat /proc/sys/vm/overcommit_memory
1
shed:~# echo 0 >/proc/sys/vm/overcommit_memory
shed:~# cat /proc/sys/vm/overcommit_memory
0

--
Alex Bligh

Re: OOM killer

[Alan Cox]

> > shed:~# echo 0 >/proc/sys/vm/overcommit_memory
> > shed:~# cat /proc/sys/vm/overcommit_memory
> > 0
> 
> ahh, I see. Well, you live and learn ;)
> 
> I think I've got to do my research better before writing mails to lkml.

In part.

The option you want is '2' which isnt implemented 8)

0	-	I don't care
1	-	Use heuristics to guesstimate avoiding overcommit

Re: OOM killer

[Jesper Juhl]

Alan Cox wrote:

>>> shed:~# echo 0 >/proc/sys/vm/overcommit_memory
>>> shed:~# cat /proc/sys/vm/overcommit_memory
>>> 0
>> 
>> ahh, I see. Well, you live and learn ;)
>> 
>> I think I've got to do my research better before writing mails to lkml.
> 
> 
> In part.
> 
> The option you want is '2' which isnt implemented 8)
> 
> 0	-	I don't care
> 1	-	Use heuristics to guesstimate avoiding overcommit


Thank you for that info :)

I wrote a small test program that allocated memory in increasingly 
larger chunks, and I saw no major difference with a setting of "0" or 
"1", it seemed both settings allowed my program to allocate exactely the 
same amount of mem before ENOMEM was returned (I can send the test 
program on request).

I'll be looking forward to a setting of "2" becomming available :)


Best regards
Jesper Juhl

Re: OOM killer

[Thomas Glanzmann]

have look at the following posting some time ago:
http://uwsg.iu.edu/hypermail/linux/kernel/0003.2/0303.html

greetings Thomas,

--


On Fri, 28 Sep 2001, Jesper Juhl wrote:

> Alan Cox wrote:
>
> >>> shed:~# echo 0 >/proc/sys/vm/overcommit_memory
> >>> shed:~# cat /proc/sys/vm/overcommit_memory
> >>> 0
> >>
> >> ahh, I see. Well, you live and learn ;)
> >>
> >> I think I've got to do my research better before writing mails to lkml.
> >
> >
> > In part.
> >
> > The option you want is '2' which isnt implemented 8)
> >
> > 0	-	I don't care
> > 1	-	Use heuristics to guesstimate avoiding overcommit
>
>
> Thank you for that info :)
>
> I wrote a small test program that allocated memory in increasingly
> larger chunks, and I saw no major difference with a setting of "0" or
> "1", it seemed both settings allowed my program to allocate exactely the
> same amount of mem before ENOMEM was returned (I can send the test
> program on request).
>
> I'll be looking forward to a setting of "2" becomming available :)
>
>
> Best regards
> Jesper Juhl
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

OOM killer.

[Andrey Nekrasov]

Help.

Is it possible to disable VM-killer completely?

Or make it not kill some critical tools. I want, for example, that
in any situation sshd/nfsd/cron remained alive.

-- 
bye.
Andrey Nekrasov, SpyLOG.

Re: OOM killer.

[Helge Hafting]

Andrey Nekrasov wrote:
> 
> Help.
> 
> Is it possible to disable VM-killer completely?
Obviously not - it triggers only when there isn't
enough memory to continue.  The alternative to a
OOM killer routine is either a kernel that locks up
effectively killing everything, or a kernel that
allocates all memory early requiring a _lot_ more
memory to run at all.  (If you had that memory
you wouldn't run out in the first place.)
> 
> Or make it not kill some critical tools. I want, for example, that
> in any situation sshd/nfsd/cron remained alive.

The trivial fix is to start sshd/nfsd/cron from /etc/inittab,
similiar to how getty usually starts.  They may still get
killed, but init will restart them asap.

Adding a lot of swap is also a good idea - the machine will
merely slow down instead of killing something.  

Helge Hafting

OOM killer

[Paco Martinez]

Do you know any newer kernel that solves problem about "OOM Killer" ??

Thank you !!!!

Re: OOM killer

[Alan Cox]

> Do you know any newer kernel that solves problem about "OOM Killer" ??
> Thank you !!!!

I've had no problem with bogus out of memory cases in either 2.4.18-rc, or
the 2.4.18-ac tree (which adds the rmap vm improvements). I'm also working
at the moment on adding support for strict memory overcommit handling so
that you can opt to be sure OOM will not happen, and that a program will
always get out of memory returns from a syscall (or if you are really
really unlucky a kill from a stackfault on an app that doesnt take the
right care)

Re: OOM killer

[Denis Vlasenko]

On 19 February 2002 08:02, Paco Martinez wrote:
> Do you know any newer kernel that solves problem about "OOM Killer" ??
>
> Thank you !!!!

Dont know whether it is in 2.4 mainline, but I use this small patch
--
vda



Author: M.H.VanLeeuwen <vanl@megsinet.net>
--- linux.virgin/mm/vmscan.c	Mon Dec 31 12:46:25 2001
+++ linux/mm/vmscan.c	Fri Jan 11 18:03:05 2002
@@ -394,9 +394,9 @@
 		if (PageDirty(page) && is_page_cache_freeable(page) && page->mapping) {
 			/*
 			 * It is not critical here to write it only if
-			 * the page is unmapped beause any direct writer
+			 * the page is unmapped because any direct writer
 			 * like O_DIRECT would set the PG_dirty bitflag
-			 * on the phisical page after having successfully
+			 * on the physical page after having successfully
 			 * pinned it and after the I/O to the page is finished,
 			 * so the direct writes to the page cannot get lost.
 			 */
@@ -480,11 +480,14 @@

 			/*
 			 * Alert! We've found too many mapped pages on the
-			 * inactive list, so we start swapping out now!
+			 * inactive list.
+			 * Move referenced pages to the active list.
 			 */
-			spin_unlock(&pagemap_lru_lock);
-			swap_out(priority, gfp_mask, classzone);
-			return nr_pages;
+			if (PageReferenced(page) && !PageLocked(page)) {
+				del_page_from_inactive_list(page);
+				add_page_to_active_list(page);
+			}
+			continue;
 		}

 		/*
@@ -521,6 +524,9 @@
 	}
 	spin_unlock(&pagemap_lru_lock);

+	if (max_mapped <= 0 && (nr_pages > 0 || priority < DEF_PRIORITY))
+		swap_out(priority, gfp_mask, classzone);
+
 	return nr_pages;
 }

Re: OOM killer

[Jeffrey Nowland]

I have seen some wierdness in this area.  I think I might see how it comes
about.

The machine and configuration:
IBM x330 SMP 1.266 GHz Pentium III
2GB RAM
Rawhide 2.4.17-0.16 kernel
**NO SWAP**

Anyway, it appears that at some point the system is running low on
absolutely free RAM and the vmscan kicks in to try to free some pages.  At
this point my top output says I have around 200MB of free memory and about
1.5GB of file cache so one could say this counts as shouldn't OOM anything.

If, at this point, I try to build both the i386 and i686 kernels from a
single rpm command (not sure if just running two sequential rpm -ba's does
it or not) I will start losing kdeinit's with the following showing up in
/var/log/messages:

kernel: Out of memory: Killed process 1630 (kdeinit)
(many more follow as the rpm processes continue valiantly trying to compile
kernels).

Somewhere along the lines here, try_to_free_pages in mm/vmscan.c is called
(not sure why, but doesn't matter, it just does).  In do_try_to_free_pages
it calls page_launder, shrink_dcache_memory, shrink_icache_memory and
shirnk_dqcache_memory in succession and (eventually) if none of them frees
any pages and free_low(ANY_ZONE) > 0 then out of memory is declared.

Now here's the kick, if the gfp_mask indicates that this is a file system
(!(gfp_mask & __GFP_FS)) request then in shrink_dcache_memory there is a
blurb about a deadlock and the dcache refuses to release anything (no matter
how much it has: see shrink_dcache_memory in fs/dcache.c).

So it would appear to me that if the system is feeling some memory pressure
and the only thing that might be freeable is dcache and the file system asks
for some memory causing try_to_free_pages to be called, that you are just
SOL.  dcache isn't giving you anything and there's nothing can be done about
it (well except implement the micro-suggestion in the DEADLOCK blurb in
dcache.c).

Anyway, I don't know if this helps or is even relevent, but I thought I'd
give it a whirl.

Jeff---

----- Original Message -----
From: "Alan Cox" <alan@lxorguk.ukuu.org.uk>
To: "Paco Martinez" <pmartinez@heraldo.es>
Cc: "kernel list" <linux-kernel@vger.kernel.org>
Sent: Tuesday, February 19, 2002 4:34 AM
Subject: Re: OOM killer


> > Do you know any newer kernel that solves problem about "OOM Killer" ??
> > Thank you !!!!
>
> I've had no problem with bogus out of memory cases in either 2.4.18-rc, or
> the 2.4.18-ac tree (which adds the rmap vm improvements). I'm also working
> at the moment on adding support for strict memory overcommit handling so
> that you can opt to be sure OOM will not happen, and that a program will
> always get out of memory returns from a syscall (or if you are really
> really unlucky a kill from a stackfault on an app that doesnt take the
> right care)
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

oom killer

[mbneto]

Hi,

I am trying to figure out how to enable/use a oom killer in order to
prevent a single process to trash the machine.

Any tips will be great.

thanks.

Ps. Is this the right forum ?

Re: oom killer

[John Livingston]

>Ps. Is this the right forum ?
>  
>
I would recommend the openmosix-devel mailing list 
(http://lists.sourceforge.net/lists/listinfo/openmosix-devel).

~John Livingston