vma_merge issue

Bill Speirs
2009-08-10T20:47:04Z

Hello-

I came across an issue where adjacent pages are not properly coalesced
together when changing protections on them. This can be shown by doing
the following:

1) Map 3 pages with PROT_NONE and MAP_PRIVATE | MAP_ANONYMOUS
2) Set the middle page's protection to PROT_READ | PROT_WRITE
3) Set the middle page's protection back to PROT_NONE

You are left with 3 entries in /proc/self/map where you should only
have 1. If you only change the protection to PROT_READ in step 2, then
it is properly merged together. I noticed in mprotect.c the following
comment in the function mprotect_fixup; I'm not sure if it applies or
not:
/*
* If we make a private mapping writable we increase our commit;
* but (without finer accounting) cannot reduce our commit if we
* make it unwritable again.
*
* FIXME? We haven't defined a VM_NORESERVE flag, so mprotecting
* a MAP_NORESERVE private mapping to writable will now reserve.
*/

I think this only applies to setting charged = nrpages; however,
VM_ACCOUNT is also added to newflags. Could it be that the adjacent
blocks don't have VM_ACCOUNT and so the call to vma_merge cannot merge
because the flags for the adjacent vma are not the same?

Can anyone shed some light on this? While it isn't an issue for 3
pages, I'm mmaping 200K+ pages and changing the perms on random pages
throughout and then back but I quickly run into the max_map_count when
I don't actually need that many mappings.

Thanks...

Bill-