A question about using a private anonymous mmap

A question about using a private anonymous mmap

[Julian Phillips]

I have a program which creates a reasonably large private anonymous map. 
The program then writes into a few places in the map, but ends up reading 
from all of them.

When I run this program on a system running 2.6.20.7 the process only ever 
seems to use enough memory to hold the data that has actually been written 
(well - in units of PAGE_SIZE).  When I run the program on a system 
running 2.6.24.5 then as it reads the map the amount of memory used 
continues to increase until the complete map has actually been allocated 
(and since the total size is greater than the physically available RAM 
causes swapping).  Basically I seem to be seeing copy-on-read instead of 
copy-on-write type behaviour.

Is this an expected change, and is there any option I can tweak to get the 
old behaviour back?

TIA,

-- 
Julian

  ---
I can't die until the government finds a safe place to bury my liver.
 		-- Phil Harris

Re: A question about using a private anonymous mmap

[Robert Hancock]

Julian Phillips wrote:
> I have a program which creates a reasonably large private anonymous map. 
> The program then writes into a few places in the map, but ends up 
> reading from all of them.
> 
> When I run this program on a system running 2.6.20.7 the process only 
> ever seems to use enough memory to hold the data that has actually been 
> written (well - in units of PAGE_SIZE).  When I run the program on a 
> system running 2.6.24.5 then as it reads the map the amount of memory 
> used continues to increase until the complete map has actually been 
> allocated (and since the total size is greater than the physically 
> available RAM causes swapping).  Basically I seem to be seeing 
> copy-on-read instead of copy-on-write type behaviour.
> 
> Is this an expected change, and is there any option I can tweak to get 
> the old behaviour back?

Looks like this was as a result of the ZERO_PAGE removal:

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=557ed1fa2620dc119adb86b34c614e152a629a80

The commit comment notes: "Inserting a ZERO_PAGE for anonymous read 
faults appears to be a false optimisation: if an application is 
performance critical, it would not be doing many read faults of new 
memory, or at least it could be expected to write to that memory soon 
afterwards. If cache or memory use is critical, it should not be working 
with a significant number of ZERO_PAGEs anyway (a more compact 
representation of zeroes should be used)."

Re: A question about using a private anonymous mmap

[Julian Phillips]

On Thu, 4 Jun 2009, Robert Hancock wrote:

> Julian Phillips wrote:
>>  I have a program which creates a reasonably large private anonymous map.
>>  The program then writes into a few places in the map, but ends up reading
>>  from all of them.
>>
>>  When I run this program on a system running 2.6.20.7 the process only ever
>>  seems to use enough memory to hold the data that has actually been written
>>  (well - in units of PAGE_SIZE).  When I run the program on a system
>>  running 2.6.24.5 then as it reads the map the amount of memory used
>>  continues to increase until the complete map has actually been allocated
>>  (and since the total size is greater than the physically available RAM
>>  causes swapping).  Basically I seem to be seeing copy-on-read instead of
>>  copy-on-write type behaviour.
>>
>>  Is this an expected change, and is there any option I can tweak to get the
>>  old behaviour back?
>
> Looks like this was as a result of the ZERO_PAGE removal:
>
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=557ed1fa2620dc119adb86b34c614e152a629a80
>
> The commit comment notes: "Inserting a ZERO_PAGE for anonymous read faults 
> appears to be a false optimisation: if an application is performance 
> critical, it would not be doing many read faults of new memory, or at least 
> it could be expected to write to that memory soon afterwards. If cache or 
> memory use is critical, it should not be working with a significant number of 
> ZERO_PAGEs anyway (a more compact representation of zeroes should be used)."

That does seem to be the cause, or at least when I revert that commit I 
get the same behaviour from 2.6.24.5 as I do from 2.6.20.7.

Thanks.

-- 
Julian

  ---
If there is no wind, row.
 		-- Polish proverb

Re: A question about using a private anonymous mmap

[KAMEZAWA Hiroyuki]

On Thu, 4 Jun 2009 17:48:29 +0100 (BST)
Julian Phillips <julian@quantumfyre.co.uk> wrote:

> I have a program which creates a reasonably large private anonymous map. 
> The program then writes into a few places in the map, but ends up reading 
> from all of them.
> 
> When I run this program on a system running 2.6.20.7 the process only ever 
> seems to use enough memory to hold the data that has actually been written 
> (well - in units of PAGE_SIZE).  When I run the program on a system 
> running 2.6.24.5 then as it reads the map the amount of memory used 
> continues to increase until the complete map has actually been allocated 
> (and since the total size is greater than the physically available RAM 
> causes swapping).  Basically I seem to be seeing copy-on-read instead of 
> copy-on-write type behaviour.
> 
> Is this an expected change, and is there any option I can tweak to get the 
> old behaviour back?
> 

It comes from removal of ZERO_PAGE.

IMHO, What you can do is mmap(/dev/zero, MAP_PRIVATE) instead of ANON.

Thanks,
-Kame

> TIA,
> 
> -- 
> Julian
> 
>   ---
> I can't die until the government finds a safe place to bury my liver.
>  		-- Phil Harris
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: A question about using a private anonymous mmap

[KAMEZAWA Hiroyuki]

On Fri, 5 Jun 2009 12:06:18 +0900
KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> wrote:

> On Thu, 4 Jun 2009 17:48:29 +0100 (BST)
> Julian Phillips <julian@quantumfyre.co.uk> wrote:
> 
> > I have a program which creates a reasonably large private anonymous map. 
> > The program then writes into a few places in the map, but ends up reading 
> > from all of them.
> > 
> > When I run this program on a system running 2.6.20.7 the process only ever 
> > seems to use enough memory to hold the data that has actually been written 
> > (well - in units of PAGE_SIZE).  When I run the program on a system 
> > running 2.6.24.5 then as it reads the map the amount of memory used 
> > continues to increase until the complete map has actually been allocated 
> > (and since the total size is greater than the physically available RAM 
> > causes swapping).  Basically I seem to be seeing copy-on-read instead of 
> > copy-on-write type behaviour.
> > 
> > Is this an expected change, and is there any option I can tweak to get the 
> > old behaviour back?
> > 
> 
> It comes from removal of ZERO_PAGE.
> 
> IMHO, What you can do is mmap(/dev/zero, MAP_PRIVATE) instead of ANON.
> 
Ah, sorry, mmap(/dev/zero) trick is no help in current kernel, I misunderstood.

Thanks,
-Kame