Re: [Linux-ia64] Fix for for memory leak in IA32 mmap

[n0ano@indstorage.com]

Tony-

A couple of points:

1)  Multiple `mmap's all freed with 1 `munmap'.  If those were all
non-fixed `mmap's then that is a pretty broken program that deserves
to fail (there is no guarantee that the `mmap's returned contiguous
memory).  If they were all fixed `mmap's then there shouldn't be a
problem even with the current scheme.

2)  An mmap(MAP_FIXED) doesn't create a problem even now, it won't
in the future.

3)  I wasn't intending to keep a bitmap for all of IA32 addressable
memory.  I intend to keep a list of all the partial pages, basically
the first and last page of an `mmap' request, and then keep a bitmap
for each of the 4K chunks inside of that page.  Maintaining the bitmap
will be a little tricky to handle all of the `mmap/munmap' possibilities
but it shouldn't be all that hard to get it right.

I agree, there could be a pathological IA32 program that just won't
work without 4K pages, we've said all along that that is a possibility.
Fortunately, we haven't found any real world programs that we can't
make work yet and I'm going to try real hard to make sure that all
real world programs work.

On Tue, Mar 05, 2002 at 11:44:20AM -0800, Luck, Tony wrote:
> I'm not sure that you really need a list ... in fact if you have
> a list, I think that I can still come up with pathalogical programs
> that will break:  E.g. I might use several mmap() calls to set up
> some blocks of memory, but clear them all with one call to munmap()
> that spans them all, or I might not do any munmap() at all and
> just mmap(MAP_FIXED) things onto the same addresses (since mmap will
> throw away existing mappings before creating new ones). Just using
> the bitmap to determine whether to round up the end (and round down
> the start) address of munmap() requests based on whether the partial
> pages have been used should solve most of the problems.
> 
> IA-32 programs are limited to the bottom 4G of address space, and
> they believe that it is divided into 2^20 * 4KB pages.  A bitmap
> for the whole of that would be 128KB, which might be somewhat high
> of an overhead for every IA-32 program ... but a two-level table
> would most likely be very sparsely filled, limiting the memory
> overhead to something acceptable.
> 
> Even with this change, there will still be programs that can only
> work correctly with a 4k kernel pagesize (e.g. a program that maps
> a 4K page from two different files, read+write into the same 16K page)
> 
> -Tony
> 
> P.S.  Here is a C program that performs the same mmap/munmap operations
> in the same order as our nasty Fortran program:
> 
> #include <sys/mman.h>
> 
> main()
> {
>         void *a, *b, *c;
>         int i;
> 
> 
>         for (i = 0; i < 1000; i++) {
>                 a = mmap(0, 0x201000, PROT_READ|PROT_WRITE,
>                         MAP_SHARED|MAP_ANONYMOUS, -1, 0);
>                 b = mmap(0, 0x101000, PROT_READ|PROT_WRITE, 
>                         MAP_SHARED|MAP_ANONYMOUS, -1, 0);
>                 c = mmap(0, 0x101000, PROT_READ|PROT_WRITE, 
>                         MAP_SHARED|MAP_ANONYMOUS, -1, 0);
>                 if ((long)a = -1 || (long)b = -1 || (long)c = -1)
>                         abort();
> 
>                 munmap(a, 0x201000);
>                 munmap(b, 0x101000);
>                 munmap(c, 0x101000);
>         }
> 
>         return 0;
> }
> 
> 
> 
> -----Original Message-----
> From: David Mosberger [mailto:davidm@napali.hpl.hp.com]
> Sent: Tuesday, March 05, 2002 10:59 AM
> To: Don Dugger
> Cc: davidm@hpl.hp.com; linux-ia64@linuxia64.org
> Subject: Re: [Linux-ia64] Fix for for memory leak in IA32 mmap
> 
> 
> >>>>> On Tue, 5 Mar 2002 10:46:29 -0700, Don Dugger <n0ano@n0ano.com> said:
> 
>   Don> David- Yep, it was a virtual memory leak.  Intel came up with a
>   Don> Fortran program that was allocating and freeing lots of
>   Don> anonymous `mmap's.  It was really nasty because it wasn't even
>   Don> the same request all the time, it had something like 3
>   Don> different odd size requests that it was `mmap'ing and
>   Don> `munmap'ing, all in a loop and eventually it ran out of VM.
> 
> OK, thanks for the background.
> 
>   Don> I like the idea of keeping a bitmap.  I still have to keep a
>   Don> list, it'll actually be a bigger list since I'll have to keep
>   Don> track of fixed requests also, but that should handle ALL cases
>   Don> (even the case where a program makes an odd sized non-fixed
>   Don> `mmap' followed by a fixed `mmap' into the middle of the last
>   Don> page).  Give me a few days and I'll see if I can't come up with
>   Don> something.
> 
> Yes, I agree: the list is still needed and an entry needs to be
> created whenever an ia64 page is partially mapped.
> 
> Thanks,
> 
> 	--david
> 
> _______________________________________________
> Linux-IA64 mailing list
> Linux-IA64@linuxia64.org
> http://lists.linuxia64.org/lists/listinfo/linux-ia64
> 
> _______________________________________________
> Linux-IA64 mailing list
> Linux-IA64@linuxia64.org
> http://lists.linuxia64.org/lists/listinfo/linux-ia64

-- 
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
n0ano@indstorage.com
Ph: 303/652-0870x117