[PATCH v2][RFC] add MAP_UNLOCKED mmap flag

[PATCH v2][RFC] add MAP_UNLOCKED mmap flag

[Gleb Natapov]

If application does mlockall(MCL_FUTURE) it is no longer possible to
mmap file bigger than main memory or allocate big area of anonymous
memory. Sometimes it is desirable to lock everything related to program
execution into memory, but still be able to mmap big file or allocate
huge amount of memory and allow OS to swap them on demand. MAP_UNLOCKED
allows to do that.

Signed-off-by: Gleb Natapov <gleb@redhat.com>
---

v1->v2
 - adding new flag to all archs
 - fixing typo

diff --git a/arch/alpha/include/asm/mman.h b/arch/alpha/include/asm/mman.h
index 99c56d4..cfc51ac 100644
--- a/arch/alpha/include/asm/mman.h
+++ b/arch/alpha/include/asm/mman.h
@@ -30,6 +30,7 @@
 #define MAP_NONBLOCK	0x40000		/* do not block on IO */
 #define MAP_STACK	0x80000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x100000	/* create a huge page mapping */
+#define MAP_UNLOCKED	0x200000	/* force page unlocking */
 
 #define MS_ASYNC	1		/* sync memory asynchronously */
 #define MS_SYNC		2		/* synchronous memory sync */
diff --git a/arch/mips/include/asm/mman.h b/arch/mips/include/asm/mman.h
index a2250f3..c161e27 100644
--- a/arch/mips/include/asm/mman.h
+++ b/arch/mips/include/asm/mman.h
@@ -48,6 +48,7 @@
 #define MAP_NONBLOCK	0x20000		/* do not block on IO */
 #define MAP_STACK	0x40000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x80000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x100000	/* force page unlocking */
 
 /*
  * Flags for msync
diff --git a/arch/parisc/include/asm/mman.h b/arch/parisc/include/asm/mman.h
index 9749c8a..4e8b9bf 100644
--- a/arch/parisc/include/asm/mman.h
+++ b/arch/parisc/include/asm/mman.h
@@ -24,6 +24,7 @@
 #define MAP_NONBLOCK	0x20000		/* do not block on IO */
 #define MAP_STACK	0x40000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x80000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x100000	/* force page unlocking */
 
 #define MS_SYNC		1		/* synchronous memory sync */
 #define MS_ASYNC	2		/* sync memory asynchronously */
diff --git a/arch/powerpc/include/asm/mman.h b/arch/powerpc/include/asm/mman.h
index d4a7f64..7d33f01 100644
--- a/arch/powerpc/include/asm/mman.h
+++ b/arch/powerpc/include/asm/mman.h
@@ -27,6 +27,7 @@
 #define MAP_NONBLOCK	0x10000		/* do not block on IO */
 #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x80000		/* force page unlocking */
 
 #ifdef __KERNEL__
 #ifdef CONFIG_PPC64
diff --git a/arch/sparc/include/asm/mman.h b/arch/sparc/include/asm/mman.h
index c3029ad..f80d203 100644
--- a/arch/sparc/include/asm/mman.h
+++ b/arch/sparc/include/asm/mman.h
@@ -22,6 +22,7 @@
 #define MAP_NONBLOCK	0x10000		/* do not block on IO */
 #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x80000		/* force page unlocking */
 
 #ifdef __KERNEL__
 #ifndef __ASSEMBLY__
diff --git a/arch/xtensa/include/asm/mman.h b/arch/xtensa/include/asm/mman.h
index fca4db4..c62bcd8 100644
--- a/arch/xtensa/include/asm/mman.h
+++ b/arch/xtensa/include/asm/mman.h
@@ -55,6 +55,7 @@
 #define MAP_NONBLOCK	0x20000		/* do not block on IO */
 #define MAP_STACK	0x40000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x80000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x100000	/* force page unlocking */
 
 /*
  * Flags for msync
diff --git a/include/asm-generic/mman.h b/include/asm-generic/mman.h
index 32c8bd6..59e0f29 100644
--- a/include/asm-generic/mman.h
+++ b/include/asm-generic/mman.h
@@ -12,6 +12,7 @@
 #define MAP_NONBLOCK	0x10000		/* do not block on IO */
 #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x80000         /* force page unlocking */
 
 #define MCL_CURRENT	1		/* lock all current mappings */
 #define MCL_FUTURE	2		/* lock all future mappings */
diff --git a/mm/mmap.c b/mm/mmap.c
index 73f5e4b..7c2abdb 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -985,6 +985,9 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 		if (!can_do_mlock())
 			return -EPERM;
 
+        if (flags & MAP_UNLOKED)
+                vm_flags &= ~VM_LOCKED;
+
 	/* mlock MCL_FUTURE? */
 	if (vm_flags & VM_LOCKED) {
 		unsigned long locked, lock_limit;
--
			Gleb.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2][RFC] add MAP_UNLOCKED mmap flag

[Christoph Lameter]

Looks good.

Reviewed-by: Christoph Lameter <cl@linux-foundation.org>


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2][RFC] add MAP_UNLOCKED mmap flag

[Johannes Weiner]

On Tue, Oct 06, 2009 at 07:02:18PM +0200, Gleb Natapov wrote:
> diff --git a/include/asm-generic/mman.h b/include/asm-generic/mman.h
> index 32c8bd6..59e0f29 100644
> --- a/include/asm-generic/mman.h
> +++ b/include/asm-generic/mman.h
> @@ -12,6 +12,7 @@
>  #define MAP_NONBLOCK	0x10000		/* do not block on IO */
>  #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
>  #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
> +#define MAP_UNLOCKED	0x80000         /* force page unlocking */
>  
>  #define MCL_CURRENT	1		/* lock all current mappings */
>  #define MCL_FUTURE	2		/* lock all future mappings */
> diff --git a/mm/mmap.c b/mm/mmap.c
> index 73f5e4b..7c2abdb 100644
> --- a/mm/mmap.c
> +++ b/mm/mmap.c
> @@ -985,6 +985,9 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
>  		if (!can_do_mlock())
>  			return -EPERM;
>  
> +        if (flags & MAP_UNLOKED)
> +                vm_flags &= ~VM_LOCKED;

That needs changing into MAP_UNLOCKED as well.

Should we do something special about (MAP_UNLOCKED | MAP_LOCKED)?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2][RFC] add MAP_UNLOCKED mmap flag

[Gleb Natapov]

On Tue, Oct 06, 2009 at 08:34:36PM +0200, Johannes Weiner wrote:
> On Tue, Oct 06, 2009 at 07:02:18PM +0200, Gleb Natapov wrote:
> > diff --git a/include/asm-generic/mman.h b/include/asm-generic/mman.h
> > index 32c8bd6..59e0f29 100644
> > --- a/include/asm-generic/mman.h
> > +++ b/include/asm-generic/mman.h
> > @@ -12,6 +12,7 @@
> >  #define MAP_NONBLOCK	0x10000		/* do not block on IO */
> >  #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
> >  #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
> > +#define MAP_UNLOCKED	0x80000         /* force page unlocking */
> >  
> >  #define MCL_CURRENT	1		/* lock all current mappings */
> >  #define MCL_FUTURE	2		/* lock all future mappings */
> > diff --git a/mm/mmap.c b/mm/mmap.c
> > index 73f5e4b..7c2abdb 100644
> > --- a/mm/mmap.c
> > +++ b/mm/mmap.c
> > @@ -985,6 +985,9 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
> >  		if (!can_do_mlock())
> >  			return -EPERM;
> >  
> > +        if (flags & MAP_UNLOKED)
> > +                vm_flags &= ~VM_LOCKED;
> 
> That needs changing into MAP_UNLOCKED as well.
> 
That's for editing patch manually :(

> Should we do something special about (MAP_UNLOCKED | MAP_LOCKED)?
It is simpler to just ignore it. What do you think?

--
			Gleb.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2][RFC] add MAP_UNLOCKED mmap flag

[Johannes Weiner]

On Tue, Oct 06, 2009 at 08:53:44PM +0200, Gleb Natapov wrote:
> On Tue, Oct 06, 2009 at 08:34:36PM +0200, Johannes Weiner wrote:
> > On Tue, Oct 06, 2009 at 07:02:18PM +0200, Gleb Natapov wrote:
> > > diff --git a/include/asm-generic/mman.h b/include/asm-generic/mman.h
> > > index 32c8bd6..59e0f29 100644
> > > --- a/include/asm-generic/mman.h
> > > +++ b/include/asm-generic/mman.h
> > > @@ -12,6 +12,7 @@
> > >  #define MAP_NONBLOCK	0x10000		/* do not block on IO */
> > >  #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
> > >  #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
> > > +#define MAP_UNLOCKED	0x80000         /* force page unlocking */
> > >  
> > >  #define MCL_CURRENT	1		/* lock all current mappings */
> > >  #define MCL_FUTURE	2		/* lock all future mappings */
> > > diff --git a/mm/mmap.c b/mm/mmap.c
> > > index 73f5e4b..7c2abdb 100644
> > > --- a/mm/mmap.c
> > > +++ b/mm/mmap.c
> > > @@ -985,6 +985,9 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
> > >  		if (!can_do_mlock())
> > >  			return -EPERM;
> > >  
> > > +        if (flags & MAP_UNLOKED)
> > > +                vm_flags &= ~VM_LOCKED;
> 
> > Should we do something special about (MAP_UNLOCKED | MAP_LOCKED)?
>
> It is simpler to just ignore it. What do you think?

I think we should filter it out.  It's unlocking even when MAP_LOCKED
is set and doing mlock-specific checks also no mlock will happen.

	Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2][RFC] add MAP_UNLOCKED mmap flag

[Gleb Natapov]

On Thu, Oct 08, 2009 at 11:37:52AM +0200, Johannes Weiner wrote:
> On Tue, Oct 06, 2009 at 08:53:44PM +0200, Gleb Natapov wrote:
> > On Tue, Oct 06, 2009 at 08:34:36PM +0200, Johannes Weiner wrote:
> > > On Tue, Oct 06, 2009 at 07:02:18PM +0200, Gleb Natapov wrote:
> > > > diff --git a/include/asm-generic/mman.h b/include/asm-generic/mman.h
> > > > index 32c8bd6..59e0f29 100644
> > > > --- a/include/asm-generic/mman.h
> > > > +++ b/include/asm-generic/mman.h
> > > > @@ -12,6 +12,7 @@
> > > >  #define MAP_NONBLOCK	0x10000		/* do not block on IO */
> > > >  #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
> > > >  #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
> > > > +#define MAP_UNLOCKED	0x80000         /* force page unlocking */
> > > >  
> > > >  #define MCL_CURRENT	1		/* lock all current mappings */
> > > >  #define MCL_FUTURE	2		/* lock all future mappings */
> > > > diff --git a/mm/mmap.c b/mm/mmap.c
> > > > index 73f5e4b..7c2abdb 100644
> > > > --- a/mm/mmap.c
> > > > +++ b/mm/mmap.c
> > > > @@ -985,6 +985,9 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
> > > >  		if (!can_do_mlock())
> > > >  			return -EPERM;
> > > >  
> > > > +        if (flags & MAP_UNLOKED)
> > > > +                vm_flags &= ~VM_LOCKED;
> > 
> > > Should we do something special about (MAP_UNLOCKED | MAP_LOCKED)?
> >
> > It is simpler to just ignore it. What do you think?
> 
> I think we should filter it out.  It's unlocking even when MAP_LOCKED
> is set and doing mlock-specific checks also no mlock will happen.
> 
What do you mean by "filter it out"? Return error. mlock-specific checks
don't make much sense indeed since locking will not happen.

--
			Gleb.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2][RFC] add MAP_UNLOCKED mmap flag

[Johannes Weiner]

On Thu, Oct 08, 2009 at 11:40:55AM +0200, Gleb Natapov wrote:
> On Thu, Oct 08, 2009 at 11:37:52AM +0200, Johannes Weiner wrote:
> > On Tue, Oct 06, 2009 at 08:53:44PM +0200, Gleb Natapov wrote:
> > > On Tue, Oct 06, 2009 at 08:34:36PM +0200, Johannes Weiner wrote:
> > > > On Tue, Oct 06, 2009 at 07:02:18PM +0200, Gleb Natapov wrote:
> > > > > diff --git a/include/asm-generic/mman.h b/include/asm-generic/mman.h
> > > > > index 32c8bd6..59e0f29 100644
> > > > > --- a/include/asm-generic/mman.h
> > > > > +++ b/include/asm-generic/mman.h
> > > > > @@ -12,6 +12,7 @@
> > > > >  #define MAP_NONBLOCK	0x10000		/* do not block on IO */
> > > > >  #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
> > > > >  #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
> > > > > +#define MAP_UNLOCKED	0x80000         /* force page unlocking */
> > > > >  
> > > > >  #define MCL_CURRENT	1		/* lock all current mappings */
> > > > >  #define MCL_FUTURE	2		/* lock all future mappings */
> > > > > diff --git a/mm/mmap.c b/mm/mmap.c
> > > > > index 73f5e4b..7c2abdb 100644
> > > > > --- a/mm/mmap.c
> > > > > +++ b/mm/mmap.c
> > > > > @@ -985,6 +985,9 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
> > > > >  		if (!can_do_mlock())
> > > > >  			return -EPERM;
> > > > >  
> > > > > +        if (flags & MAP_UNLOKED)
> > > > > +                vm_flags &= ~VM_LOCKED;
> > > 
> > > > Should we do something special about (MAP_UNLOCKED | MAP_LOCKED)?
> > >
> > > It is simpler to just ignore it. What do you think?
> > 
> > I think we should filter it out.  It's unlocking even when MAP_LOCKED
> > is set and doing mlock-specific checks also no mlock will happen.
>
> What do you mean by "filter it out"? Return error.

Yes, sorry for being vague.  I mean returning -EINVAL for both flags
being set.

	Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>