Forbid deletion of memory mappings

Forbid deletion of memory mappings

[Clemens Kolbitsch]

Hi everyone!
I have a strange problem where I don't even know if there is a solution to it 
at the moment:

I'm working on a new way of doing memory-management and currently I allocate 
memory mappings (at non-fixed locations) in user-memory (i.e. < TASK_SIZE) in 
addition to the regular pages mapped for the apps (e.g. heap memory, etc.)

It all works perfectly well (creating & deleting the additional mappings), 
however, when the kernel feels like it needs to allocate a mapping in 
user-space it sometimes deletes my mapping and overwrites it with the new 
one, although there is plenty of free memory at some other location.

typically my mappings are (automatically chosen) located somewhere around 
0xb7xxxxxx although there's plenty free space around 0xayyyyyyy. (i know that 
this is a bad location because of stack, etc.) but i need the kernel to 
choose a good location for me...

is there a way to make sure my mappings are not removed from memory?? or is 
there a location where i can put my mappings that they will not be removed 
(i.e. by using MAP_FIXED)?
Note that i need a big amount of memory so the range will have to be fairly 
big (to be precise: i need a lot of 0x1000byte mappings - they are always 
page-sized, not bigger, not smaller, but A LOT of them :-/ )

hope someone can help me with this!
thanks & greets,
  Clemens

Re: Forbid deletion of memory mappings

[Jiri Kosina]

On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:

> It all works perfectly well (creating & deleting the additional 
> mappings), however, when the kernel feels like it needs to allocate a 
> mapping in user-space it sometimes deletes my mapping and overwrites it 
> with the new one, although there is plenty of free memory at some other 
> location.

Hi Clemens,

what do you mean by "overwrites it"? It just probably merges your vma with 
the newly created one, right?

-- 
Jiri Kosina

Re: Forbid deletion of memory mappings

[Clemens Kolbitsch]

On Thursday 30 August 2007 19:07:05 Jiri Kosina wrote:
> On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> > It all works perfectly well (creating & deleting the additional
> > mappings), however, when the kernel feels like it needs to allocate a
> > mapping in user-space it sometimes deletes my mapping and overwrites it
> > with the new one, although there is plenty of free memory at some other
> > location.
>
> Hi Clemens,
>
> what do you mean by "overwrites it"? It just probably merges your vma with
> the newly created one, right?

that's exactly the problem... it replaces my mapping with the new one... i.e. 
it first deletes my mapping and then creates the new one at the same 
location.

i *guess* this all happens in exec.c, lines 1033ff:

munmap_back:
	vma = find_vma_prepare(mm, addr, &prev, &rb_link, &rb_parent);
	if (vma && vma->vm_start < addr + len) {
		if (do_munmap(mm, addr, len))
			return -ENOMEM;
		goto munmap_back;
	}

Re: Forbid deletion of memory mappings

[Clemens Kolbitsch]

On Thursday 30 August 2007 19:07:05 you wrote:
> On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> > It all works perfectly well (creating & deleting the additional
> > mappings), however, when the kernel feels like it needs to allocate a
> > mapping in user-space it sometimes deletes my mapping and overwrites it
> > with the new one, although there is plenty of free memory at some other
> > location.
>
> Hi Clemens,
>
> what do you mean by "overwrites it"? It just probably merges your vma with
> the newly created one, right?

does really noone have an answer for me?? i'm having the hardest time to find 
a work-around for it.

is there no way to tell the kernel, that a certain mapping must not be 
removed, no matter what (except of course an explicit call to sys_unmap, of 
course)?

Re: Forbid deletion of memory mappings

[Jiri Kosina]

On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:

> is there no way to tell the kernel, that a certain mapping must not be 
> removed, no matter what (except of course an explicit call to sys_unmap, 
> of course)?

I don't seem to get what is the issue here. Your mapping is not removed, 
only the VMAs are merged together into one larger VMA if they have 
neighbouring address ranges and compatible protection bits. See 
vma_merge().

-- 
Jiri Kosina

Re: Forbid deletion of memory mappings

[Clemens Kolbitsch]

On Thursday 30 August 2007 23:34:52 you wrote:
> On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> > is there no way to tell the kernel, that a certain mapping must not be
> > removed, no matter what (except of course an explicit call to sys_unmap,
> > of course)?
>
> I don't seem to get what is the issue here. Your mapping is not removed,
> only the VMAs are merged together into one larger VMA if they have
> neighbouring address ranges and compatible protection bits. See
> vma_merge().

the thing is that they are not. the kernel chooses to REPLACE my mapping.

consider the user-space code:

mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);

here, the second call to mmap will shorten the first mapping to 0x1000 bytes 
and create one big vma with size 0x5000 bytes.

is there a way to tell it that the second mmap MUST fail?

Re: Forbid deletion of memory mappings

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]

On Thu, 30 Aug 2007 23:41:09 +0200, Clemens Kolbitsch said:
> On Thursday 30 August 2007 23:34:52 you wrote:
> > On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> > > is there no way to tell the kernel, that a certain mapping must not be
> > > removed, no matter what (except of course an explicit call to sys_unmap,
> > > of course)?
> >
> > I don't seem to get what is the issue here. Your mapping is not removed,
> > only the VMAs are merged together into one larger VMA if they have
> > neighbouring address ranges and compatible protection bits. See
> > vma_merge().
> 
> the thing is that they are not. the kernel chooses to REPLACE my mapping.
> 
> consider the user-space code:
> 
> mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
> mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
> 
> here, the second call to mmap will shorten the first mapping to 0x1000 bytes 
> and create one big vma with size 0x5000 bytes.
> 
> is there a way to tell it that the second mmap MUST fail?

There's an LSM exit point for mmap, you could perhaps do something there.

What are you trying to achieve by forcing the second one to fail?

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Forbid deletion of memory mappings

[Jiri Kosina]

On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:

> the thing is that they are not. the kernel chooses to REPLACE my 
> mapping.
> consider the user-space code:
> mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
> mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
> here, the second call to mmap will shorten the first mapping to 0x1000 bytes 
> and create one big vma with size 0x5000 bytes.

Which is exactly in compliance with what POSIX says about MAP_FIXED mmaps 
- see http://opengroup.org/onlinepubs/007908799/xsh/mmap.html

-- 
Jiri Kosina

Re: Forbid deletion of memory mappings

[Clemens Kolbitsch]

On Thursday 30 August 2007 23:50:21 Valdis.Kletnieks@vt.edu wrote:
> On Thu, 30 Aug 2007 23:41:09 +0200, Clemens Kolbitsch said:
> > On Thursday 30 August 2007 23:34:52 you wrote:
> > > On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> > > > is there no way to tell the kernel, that a certain mapping must not
> > > > be removed, no matter what (except of course an explicit call to
> > > > sys_unmap, of course)?
> > >
> > > I don't seem to get what is the issue here. Your mapping is not
> > > removed, only the VMAs are merged together into one larger VMA if they
> > > have neighbouring address ranges and compatible protection bits. See
> > > vma_merge().
> >
> > the thing is that they are not. the kernel chooses to REPLACE my mapping.
> >
> > consider the user-space code:
> >
> > mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
> > mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
> >
> > here, the second call to mmap will shorten the first mapping to 0x1000
> > bytes and create one big vma with size 0x5000 bytes.
> >
> > is there a way to tell it that the second mmap MUST fail?
>
> There's an LSM exit point for mmap, you could perhaps do something there.
>
> What are you trying to achieve by forcing the second one to fail?

puh... that is a good question :-)

I'm writing my master's thesis on a new model of memory protection and need to 
have every memory mapping in userspace duplicated. I also have kind of a 
second PGD/PTD that allows finding this mirrored mapping.

However, as the number of original mappings grows, I suddenly have the problem 
that the kernel tries to allocate a new mapping and picks the address of a 
mirrored memory page, which it shouldn't.

Honestly, I don't understand why it does so, but it simply does... so 
basically I want these second mappings to stay in memory as long as the 
original page.

What do you mean exactly with

> There's an LSM exit point for mmap, you could perhaps do something there.

??

By the way / @Jiri Kosina:

> Which is exactly in compliance with what POSIX says about MAP_FIXED mmaps 
> - see http://opengroup.org/onlinepubs/007908799/xsh/mmap.html

I know... I'm not even saying that this is a bug... I just wonder if there is 
some way to avoid this problem :-)

Thanks for your help - i really appreciate that!!

Re: Forbid deletion of memory mappings

[Helge Hafting]

Clemens Kolbitsch wrote:
> On Thursday 30 August 2007 23:50:21 Valdis.Kletnieks@vt.edu wrote:
>   
>> On Thu, 30 Aug 2007 23:41:09 +0200, Clemens Kolbitsch said:
>>     
>>> On Thursday 30 August 2007 23:34:52 you wrote:
>>>       
>>>> On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
>>>>         
>>>>> is there no way to tell the kernel, that a certain mapping must not
>>>>> be removed, no matter what (except of course an explicit call to
>>>>> sys_unmap, of course)?
>>>>>           
>>>> I don't seem to get what is the issue here. Your mapping is not
>>>> removed, only the VMAs are merged together into one larger VMA if they
>>>> have neighbouring address ranges and compatible protection bits. See
>>>> vma_merge().
>>>>         
>>> the thing is that they are not. the kernel chooses to REPLACE my mapping.
>>>
>>> consider the user-space code:
>>>
>>> mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
>>> mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
>>>
>>> here, the second call to mmap will shorten the first mapping to 0x1000
>>> bytes and create one big vma with size 0x5000 bytes.
>>>
>>> is there a way to tell it that the second mmap MUST fail?
>>>       
>> There's an LSM exit point for mmap, you could perhaps do something there.
>>
>> What are you trying to achieve by forcing the second one to fail?
>>     
>
> puh... that is a good question :-)
>
> I'm writing my master's thesis on a new model of memory protection and need to 
> have every memory mapping in userspace duplicated. I also have kind of a 
> second PGD/PTD that allows finding this mirrored mapping.
>
> However, as the number of original mappings grows, I suddenly have the problem 
> that the kernel tries to allocate a new mapping and picks the address of a 
> mirrored memory page, which it shouldn't.
>
> Honestly, I don't understand why it does so, 
The "why" is easy: Having many mappings is expensive,
so merging them (when this cause no problems) is
a smart thing to do. So that is what linux does.
It means fewer mappings to keep track of. Less resources in
use means that linux moves faster.

If you are doing research, consider these methods:
1. Change vma_merge() so it always fail to merge mappings

or

2. Set up your "mappings duplicated in userspace" so
    they too merge in the same way.

Helge Hafting

Re: Forbid deletion of memory mappings

[Clemens Kolbitsch]

> If you are doing research, consider these methods:
> 1. Change vma_merge() so it always fail to merge mappings
>
> or
>
> 2. Set up your "mappings duplicated in userspace" so
>     they too merge in the same way.
>
> Helge Hafting

Hi!
Thanks for your answer, however you (too) misunderstood: Merging of the vma's 
is not the problem and I understand why the kernel does so.

The real problem is that it deletes mappings it should not be deleting... in 
my case, I've found out that a possible reason for the deletions is that the 
stack tries to grow or other large areas of memory are allocated and my 
mapping is taking up the memory that the kernel wants to reserve.

However, this has been dealt with in some other reply - I decided to simply 
relocate my mappings to another memory area as soon as the kernel tries to 
delete them.

This is not really all that efficient, but it does not matter right now in my 
case.

Thanks again for your answer - I appreciate it!

Re: Forbid deletion of memory mappings

[linux-os (Dick Johnson)]

On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:

> On Thursday 30 August 2007 23:50:21 Valdis.Kletnieks@vt.edu wrote:
>> On Thu, 30 Aug 2007 23:41:09 +0200, Clemens Kolbitsch said:
>>> On Thursday 30 August 2007 23:34:52 you wrote:
>>>> On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
>>>>> is there no way to tell the kernel, that a certain mapping must not
>>>>> be removed, no matter what (except of course an explicit call to
>>>>> sys_unmap, of course)?
>>>>
>>>> I don't seem to get what is the issue here. Your mapping is not
>>>> removed, only the VMAs are merged together into one larger VMA if they
>>>> have neighbouring address ranges and compatible protection bits. See
>>>> vma_merge().
>>>
>>> the thing is that they are not. the kernel chooses to REPLACE my mapping.
>>>
>>> consider the user-space code:
>>>
>>> mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
>>> mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
>>>
>>> here, the second call to mmap will shorten the first mapping to 0x1000
>>> bytes and create one big vma with size 0x5000 bytes.
>>>
>>> is there a way to tell it that the second mmap MUST fail?
>>
>> There's an LSM exit point for mmap, you could perhaps do something there.
>>
>> What are you trying to achieve by forcing the second one to fail?
>
> puh... that is a good question :-)
>
> I'm writing my master's thesis on a new model of memory protection and need to
> have every memory mapping in userspace duplicated. I also have kind of a
> second PGD/PTD that allows finding this mirrored mapping.
>
> However, as the number of original mappings grows, I suddenly have the problem
> that the kernel tries to allocate a new mapping and picks the address of a
> mirrored memory page, which it shouldn't.
>
> Honestly, I don't understand why it does so, but it simply does... so
> basically I want these second mappings to stay in memory as long as the
> original page.
>
> What do you mean exactly with
>
>> There's an LSM exit point for mmap, you could perhaps do something there.
>
> ??
>
> By the way / @Jiri Kosina:
>
>> Which is exactly in compliance with what POSIX says about MAP_FIXED mmaps
>> - see http://opengroup.org/onlinepubs/007908799/xsh/mmap.html
>
> I know... I'm not even saying that this is a bug... I just wonder if there is
> some way to avoid this problem :-)
>
> Thanks for your help - i really appreciate that!!

You might need to keep track of your memory mappings and
not remap something that will overlap. You can mmap
a single page in which you write the addresses. If you
map it shared, all your processes can access it and
call a common memory-mapping procedure (that you write)
which keeps track of its kernel allocations for you.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.22.1 on an i686 machine (5588.30 BogoMips).
My book : http://www.AbominableFirebug.com/
_


****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.