overcommit verses MAP_NORESERVE

overcommit verses MAP_NORESERVE

[Nicholas Miell]

Why does overcommit in mode 2 (OVERCOMMIT_NEVER) explicitly force
MAP_NORESERVE mappings to reserve memory?

My understanding is that MAP_NORESERVE is a way for apps to state that
they are aware that the memory allocated may not exist and that they
might get a SIGSEGV and that's OK with them.

Failing to do this makes certain well-know apps (*cough* Sun Java
*cough*) fail to run, which seems to be rather unhelpful.

-- 
Nicholas Miell <nmiell@comcast.net>

Re: overcommit verses MAP_NORESERVE

[Alan Cox]

On Sad, 2005-08-06 at 20:52 -0700, Nicholas Miell wrote:
> Why does overcommit in mode 2 (OVERCOMMIT_NEVER) explicitly force
> MAP_NORESERVE mappings to reserve memory?
> 
> My understanding is that MAP_NORESERVE is a way for apps to state that
> they are aware that the memory allocated may not exist and that they
> might get a SIGSEGV and that's OK with them.

Because a MAP_NORESERVE space that is filled with pages might cause
insufficient memory to be left available for another object that is not
MAP_NORESERVE.

You are right it could be improved but that would require someone
writing code that forcibly reclaimed MAP_NORESERVE objects when we were
close to out of memory.  At the moment nobody has done this, but nothing
is stopping someone having a go.

Re: overcommit verses MAP_NORESERVE

[Nicholas Miell]

On Sun, 2005-08-07 at 12:49 +0100, Alan Cox wrote:
> On Sad, 2005-08-06 at 20:52 -0700, Nicholas Miell wrote:
> > Why does overcommit in mode 2 (OVERCOMMIT_NEVER) explicitly force
> > MAP_NORESERVE mappings to reserve memory?
> > 
> > My understanding is that MAP_NORESERVE is a way for apps to state that
> > they are aware that the memory allocated may not exist and that they
> > might get a SIGSEGV and that's OK with them.
> 
> Because a MAP_NORESERVE space that is filled with pages might cause
> insufficient memory to be left available for another object that is not
> MAP_NORESERVE.
> 
> You are right it could be improved but that would require someone
> writing code that forcibly reclaimed MAP_NORESERVE objects when we were
> close to out of memory.  At the moment nobody has done this, but nothing
> is stopping someone having a go.

I don't think you can forcibly reclaim MAP_NORESERVE objects (I'm
assuming you mean completely throwing away dirty pages).

MAP_NORESERVE isn't standardized, so all we can go by is what everybody
else does (and what makes the most sense).

Based on the Linux and Solaris man pages (none of FreeBSD, Irix, HP-UX,
or AIX implement anything similar), I think calls to mmap() with
MAP_NORESERVE should always succeed (regardless of memory conditions,
not other errors) and individual writes to unallocated pages in a
MAP_NORESERVE region should either allocate a new page if possible or
send a SIGSEGV without triggering the OOM killer.

-- 
Nicholas Miell <nmiell@comcast.net>

Re: overcommit verses MAP_NORESERVE

[Alan Cox]

On Llu, 2005-08-08 at 00:22 -0700, Nicholas Miell wrote:
> I don't think you can forcibly reclaim MAP_NORESERVE objects (I'm
> assuming you mean completely throwing away dirty pages).

In which case there is no real difference between MAP_NORESERVE and not
setting it when doing zero overcommit as we do the accounting currently.


> not other errors) and individual writes to unallocated pages in a
> MAP_NORESERVE region should either allocate a new page if possible or
> send a SIGSEGV without triggering the OOM killer.

You'd need to track vma cost seperately which we don't do now and adjust
it for such mappings in the page faulting cases. Certainly practical -
send a patch