* [PATCH 1/3] hugetlbfs: Allow the creation of files suitable for MAP_PRIVATE on the vfs internal mount V2
2009-08-13 9:57 [PATCH 0/3] Add pseudo-anonymous huge page mappings V2 Eric B Munson
@ 2009-08-13 9:57 ` Eric B Munson
2009-08-13 9:57 ` [PATCH 2/3] Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions V2 Eric B Munson
2009-08-13 9:57 ` [PATCH 3/3] Add MAP_HUGETLB example to vm/hugetlbpage.txt V2 Eric B Munson
2 siblings, 0 replies; 9+ messages in thread
From: Eric B Munson @ 2009-08-13 9:57 UTC (permalink / raw)
To: linux-kernel, linux-mm; +Cc: linux-man, akpm, mtk.manpages, Eric B Munson
There are two means of creating mappings backed by huge pages:
1. mmap() a file created on hugetlbfs
2. Use shm which creates a file on an internal mount which essentially
maps it MAP_SHARED
The internal mount is only used for shared mappings but there is very
little that stops it being used for private mappings. This patch extends
hugetlbfs_file_setup() to deal with the creation of files that will be
mapped MAP_PRIVATE on the internal hugetlbfs mount. This extended API is
used in a subsequent patch to implement the MAP_HUGETLB mmap() flag.
Signed-off-by: Eric Munson <ebmunson@us.ibm.com>
---
Changes from V1:
Rebase to newest linux-2.6 tree
fs/hugetlbfs/inode.c | 22 ++++++++++++++++++----
include/linux/hugetlb.h | 10 +++++++++-
ipc/shm.c | 3 ++-
3 files changed, 29 insertions(+), 6 deletions(-)
diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index 941c842..361f536 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -506,6 +506,13 @@ static struct inode *hugetlbfs_get_inode(struct super_block *sb, uid_t uid,
inode->i_atime = inode->i_mtime = inode->i_ctime = CURRENT_TIME;
INIT_LIST_HEAD(&inode->i_mapping->private_list);
info = HUGETLBFS_I(inode);
+ /*
+ * The policy is initialized here even if we are creating a
+ * private inode because initialization simply creates an
+ * an empty rb tree and calls spin_lock_init(), later when we
+ * call mpol_free_shared_policy() it will just return because
+ * the rb tree will still be empty.
+ */
mpol_shared_policy_init(&info->policy, NULL);
switch (mode & S_IFMT) {
default:
@@ -930,12 +937,19 @@ static struct file_system_type hugetlbfs_fs_type = {
static struct vfsmount *hugetlbfs_vfsmount;
-static int can_do_hugetlb_shm(void)
+static int can_do_hugetlb_shm(int creat_flags)
{
- return capable(CAP_IPC_LOCK) || in_group_p(sysctl_hugetlb_shm_group);
+ if (!(creat_flags & HUGETLB_SHMFS_INODE))
+ return 0;
+ if (capable(CAP_IPC_LOCK))
+ return 1;
+ if (in_group_p(sysctl_hugetlb_shm_group))
+ return 1;
+ return 0;
}
-struct file *hugetlb_file_setup(const char *name, size_t size, int acctflag)
+struct file *hugetlb_file_setup(const char *name, size_t size, int acctflag,
+ int creat_flags)
{
int error = -ENOMEM;
int unlock_shm = 0;
@@ -948,7 +962,7 @@ struct file *hugetlb_file_setup(const char *name, size_t size, int acctflag)
if (!hugetlbfs_vfsmount)
return ERR_PTR(-ENOENT);
- if (!can_do_hugetlb_shm()) {
+ if (!can_do_hugetlb_shm(creat_flags)) {
if (user_shm_lock(size, user)) {
unlock_shm = 1;
WARN_ONCE(1,
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 2723513..78b6ddf 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -109,6 +109,14 @@ static inline void hugetlb_report_meminfo(struct seq_file *m)
#endif /* !CONFIG_HUGETLB_PAGE */
+enum {
+ /*
+ * The file will be used as an shm file so shmfs accounting rules
+ * apply
+ */
+ HUGETLB_SHMFS_INODE = 0x01,
+};
+
#ifdef CONFIG_HUGETLBFS
struct hugetlbfs_config {
uid_t uid;
@@ -146,7 +154,7 @@ static inline struct hugetlbfs_sb_info *HUGETLBFS_SB(struct super_block *sb)
extern const struct file_operations hugetlbfs_file_operations;
extern struct vm_operations_struct hugetlb_vm_ops;
-struct file *hugetlb_file_setup(const char *name, size_t, int);
+struct file *hugetlb_file_setup(const char *name, size_t, int, int);
int hugetlb_get_quota(struct address_space *mapping, long delta);
void hugetlb_put_quota(struct address_space *mapping, long delta);
diff --git a/ipc/shm.c b/ipc/shm.c
index 15dd238..801c68a 100644
--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -369,7 +369,8 @@ static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
/* hugetlb_file_setup applies strict accounting */
if (shmflg & SHM_NORESERVE)
acctflag = VM_NORESERVE;
- file = hugetlb_file_setup(name, size, acctflag);
+ file = hugetlb_file_setup(name, size, acctflag,
+ HUGETLB_SHMFS_INODE);
shp->mlock_user = current_user();
} else {
/*
--
1.6.3.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH 2/3] Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions V2
2009-08-13 9:57 [PATCH 0/3] Add pseudo-anonymous huge page mappings V2 Eric B Munson
2009-08-13 9:57 ` [PATCH 1/3] hugetlbfs: Allow the creation of files suitable for MAP_PRIVATE on the vfs internal mount V2 Eric B Munson
@ 2009-08-13 9:57 ` Eric B Munson
[not found] ` <83949d066e2a7221a25dd74d12d6dcf7e8b4e9ba.1250156841.git.ebmunson-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org>
2009-08-13 9:57 ` [PATCH 3/3] Add MAP_HUGETLB example to vm/hugetlbpage.txt V2 Eric B Munson
2 siblings, 1 reply; 9+ messages in thread
From: Eric B Munson @ 2009-08-13 9:57 UTC (permalink / raw)
To: linux-kernel, linux-mm; +Cc: linux-man, akpm, mtk.manpages, Eric B Munson
This patch adds a flag for mmap that will be used to request a huge
page region that will look like anonymous memory to user space. This
is accomplished by using a file on the internal vfsmount. MAP_HUGETLB
is a modifier of MAP_ANONYMOUS and so must be specified with it. The
region will behave the same as a MAP_ANONYMOUS region using small pages.
Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
---
Changes from V1
Rebase to newest linux-2.6 tree
Rename MAP_LARGEPAGE to MAP_HUGETLB to match flag name for huge page shm
include/asm-generic/mman-common.h | 1 +
include/linux/hugetlb.h | 7 +++++++
mm/mmap.c | 16 ++++++++++++++++
3 files changed, 24 insertions(+), 0 deletions(-)
diff --git a/include/asm-generic/mman-common.h b/include/asm-generic/mman-common.h
index 3b69ad3..12f5982 100644
--- a/include/asm-generic/mman-common.h
+++ b/include/asm-generic/mman-common.h
@@ -19,6 +19,7 @@
#define MAP_TYPE 0x0f /* Mask for type of mapping */
#define MAP_FIXED 0x10 /* Interpret addr exactly */
#define MAP_ANONYMOUS 0x20 /* don't use a file */
+#define MAP_HUGETLB 0x40 /* create a huge page mapping */
#define MS_ASYNC 1 /* sync memory asynchronously */
#define MS_INVALIDATE 2 /* invalidate the caches */
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 78b6ddf..b84361c 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -109,12 +109,19 @@ static inline void hugetlb_report_meminfo(struct seq_file *m)
#endif /* !CONFIG_HUGETLB_PAGE */
+#define HUGETLB_ANON_FILE "anon_hugepage"
+
enum {
/*
* The file will be used as an shm file so shmfs accounting rules
* apply
*/
HUGETLB_SHMFS_INODE = 0x01,
+ /*
+ * The file is being created on the internal vfs mount and shmfs
+ * accounting rules do not apply
+ */
+ HUGETLB_ANONHUGE_INODE = 0x02,
};
#ifdef CONFIG_HUGETLBFS
diff --git a/mm/mmap.c b/mm/mmap.c
index 34579b2..3612b20 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -29,6 +29,7 @@
#include <linux/rmap.h>
#include <linux/mmu_notifier.h>
#include <linux/perf_counter.h>
+#include <linux/hugetlb.h>
#include <asm/uaccess.h>
#include <asm/cacheflush.h>
@@ -954,6 +955,21 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
if (mm->map_count > sysctl_max_map_count)
return -ENOMEM;
+ if (flags & MAP_HUGETLB) {
+ if (file)
+ return -EINVAL;
+
+ /*
+ * VM_NORESERVE is used because the reservations will be
+ * taken when vm_ops->mmap() is called
+ */
+ len = ALIGN(len, huge_page_size(&default_hstate));
+ file = hugetlb_file_setup(HUGETLB_ANON_FILE, len, VM_NORESERVE,
+ HUGETLB_ANONHUGE_INODE);
+ if (IS_ERR(file))
+ return -ENOMEM;
+ }
+
/* Obtain the address to map to. we verify (or select) it and ensure
* that it represents a valid section of the address space.
*/
--
1.6.3.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH 3/3] Add MAP_HUGETLB example to vm/hugetlbpage.txt V2
2009-08-13 9:57 [PATCH 0/3] Add pseudo-anonymous huge page mappings V2 Eric B Munson
2009-08-13 9:57 ` [PATCH 1/3] hugetlbfs: Allow the creation of files suitable for MAP_PRIVATE on the vfs internal mount V2 Eric B Munson
2009-08-13 9:57 ` [PATCH 2/3] Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions V2 Eric B Munson
@ 2009-08-13 9:57 ` Eric B Munson
2009-08-13 21:49 ` David Rientjes
2 siblings, 1 reply; 9+ messages in thread
From: Eric B Munson @ 2009-08-13 9:57 UTC (permalink / raw)
To: linux-kernel, linux-mm; +Cc: linux-man, akpm, mtk.manpages, Eric B Munson
This patch adds an example of how to use the MAP_HUGETLB flag to
the vm documentation.
Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
---
Changes from V1:
Rebase to newest linux-2.6 tree
Change MAP_LARGEPAGE to MAP_HUGETLB to match flag name in huge page shm
Documentation/vm/hugetlbpage.txt | 80 ++++++++++++++++++++++++++++++++++++++
1 files changed, 80 insertions(+), 0 deletions(-)
diff --git a/Documentation/vm/hugetlbpage.txt b/Documentation/vm/hugetlbpage.txt
index ea8714f..d30fa1a 100644
--- a/Documentation/vm/hugetlbpage.txt
+++ b/Documentation/vm/hugetlbpage.txt
@@ -337,3 +337,83 @@ int main(void)
return 0;
}
+
+*******************************************************************
+
+/*
+ * Example of using hugepage memory in a user application using the mmap
+ * system call with MAP_LARGEPAGE flag. Before running this program make
+ * sure the administrator has allocated enough default sized huge pages
+ * to cover the 256 MB allocation.
+ *
+ * For ia64 architecture, Linux kernel reserves Region number 4 for hugepages.
+ * That means the addresses starting with 0x800000... will need to be
+ * specified. Specifying a fixed address is not required on ppc64, i386
+ * or x86_64.
+ */
+#include <stdlib.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <sys/mman.h>
+#include <fcntl.h>
+
+#define LENGTH (256UL*1024*1024)
+#define PROTECTION (PROT_READ | PROT_WRITE)
+
+#ifndef MAP_HUGETLB
+#define MAP_HUGETLB 0x40
+#endif
+
+/* Only ia64 requires this */
+#ifdef __ia64__
+#define ADDR (void *)(0x8000000000000000UL)
+#define FLAGS (MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB | MAP_FIXED)
+#else
+#define ADDR (void *)(0x0UL)
+#define FLAGS (MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB)
+#endif
+
+void check_bytes(char *addr)
+{
+ printf("First hex is %x\n", *((unsigned int *)addr));
+}
+
+void write_bytes(char *addr)
+{
+ unsigned long i;
+
+ for (i = 0; i < LENGTH; i++)
+ *(addr + i) = (char)i;
+}
+
+void read_bytes(char *addr)
+{
+ unsigned long i;
+
+ check_bytes(addr);
+ for (i = 0; i < LENGTH; i++)
+ if (*(addr + i) != (char)i) {
+ printf("Mismatch at %lu\n", i);
+ break;
+ }
+}
+
+int main(void)
+{
+ void *addr;
+
+ addr = mmap(ADDR, LENGTH, PROTECTION, FLAGS, 0, 0);
+ if (addr == MAP_FAILED) {
+ perror("mmap");
+ exit(1);
+ }
+
+ printf("Returned address is %p\n", addr);
+ check_bytes(addr);
+ write_bytes(addr);
+ read_bytes(addr);
+
+ munmap(addr, LENGTH);
+
+ return 0;
+}
--
1.6.3.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH 3/3] Add MAP_HUGETLB example to vm/hugetlbpage.txt V2
2009-08-13 9:57 ` [PATCH 3/3] Add MAP_HUGETLB example to vm/hugetlbpage.txt V2 Eric B Munson
@ 2009-08-13 21:49 ` David Rientjes
2009-08-14 0:46 ` Randy Dunlap
0 siblings, 1 reply; 9+ messages in thread
From: David Rientjes @ 2009-08-13 21:49 UTC (permalink / raw)
To: Eric B Munson; +Cc: linux-kernel, linux-mm, linux-man, akpm, mtk.manpages
On Thu, 13 Aug 2009, Eric B Munson wrote:
> This patch adds an example of how to use the MAP_HUGETLB flag to
> the vm documentation.
>
> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
> ---
> Changes from V1:
> Rebase to newest linux-2.6 tree
> Change MAP_LARGEPAGE to MAP_HUGETLB to match flag name in huge page shm
>
> Documentation/vm/hugetlbpage.txt | 80 ++++++++++++++++++++++++++++++++++++++
> 1 files changed, 80 insertions(+), 0 deletions(-)
>
> diff --git a/Documentation/vm/hugetlbpage.txt b/Documentation/vm/hugetlbpage.txt
> index ea8714f..d30fa1a 100644
> --- a/Documentation/vm/hugetlbpage.txt
> +++ b/Documentation/vm/hugetlbpage.txt
> @@ -337,3 +337,83 @@ int main(void)
>
> return 0;
> }
> +
> +*******************************************************************
> +
> +/*
> + * Example of using hugepage memory in a user application using the mmap
> + * system call with MAP_LARGEPAGE flag. Before running this program make
s/MAP_LARGEPAGE/MAP_HUGETLB/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 3/3] Add MAP_HUGETLB example to vm/hugetlbpage.txt V2
2009-08-13 21:49 ` David Rientjes
@ 2009-08-14 0:46 ` Randy Dunlap
2009-08-14 12:39 ` Eric B Munson
0 siblings, 1 reply; 9+ messages in thread
From: Randy Dunlap @ 2009-08-14 0:46 UTC (permalink / raw)
To: David Rientjes
Cc: Eric B Munson, linux-kernel, linux-mm, linux-man, akpm,
mtk.manpages
David Rientjes wrote:
> On Thu, 13 Aug 2009, Eric B Munson wrote:
>
>> This patch adds an example of how to use the MAP_HUGETLB flag to
>> the vm documentation.
>>
>> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
>> ---
>> Changes from V1:
>> Rebase to newest linux-2.6 tree
>> Change MAP_LARGEPAGE to MAP_HUGETLB to match flag name in huge page shm
>>
>> Documentation/vm/hugetlbpage.txt | 80 ++++++++++++++++++++++++++++++++++++++
>> 1 files changed, 80 insertions(+), 0 deletions(-)
>>
>> diff --git a/Documentation/vm/hugetlbpage.txt b/Documentation/vm/hugetlbpage.txt
>> index ea8714f..d30fa1a 100644
>> --- a/Documentation/vm/hugetlbpage.txt
>> +++ b/Documentation/vm/hugetlbpage.txt
>> @@ -337,3 +337,83 @@ int main(void)
>>
>> return 0;
>> }
>> +
>> +*******************************************************************
>> +
>> +/*
>> + * Example of using hugepage memory in a user application using the mmap
>> + * system call with MAP_LARGEPAGE flag. Before running this program make
>
> s/MAP_LARGEPAGE/MAP_HUGETLB/
I'm (slowly) making source code examples in Documentation/ buildable,
as this one should be, please.
I.e., put it in a separate source file (hugetlbpage.txt can refer to the
source file if you want it to) and add a Makefile similar to other
Makefiles in the Documentation/ tree.
~Randy
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 3/3] Add MAP_HUGETLB example to vm/hugetlbpage.txt V2
2009-08-14 0:46 ` Randy Dunlap
@ 2009-08-14 12:39 ` Eric B Munson
0 siblings, 0 replies; 9+ messages in thread
From: Eric B Munson @ 2009-08-14 12:39 UTC (permalink / raw)
To: Randy Dunlap
Cc: David Rientjes, linux-kernel, linux-mm, linux-man, akpm,
mtk.manpages
[-- Attachment #1: Type: text/plain, Size: 1637 bytes --]
On Thu, 13 Aug 2009, Randy Dunlap wrote:
> David Rientjes wrote:
> > On Thu, 13 Aug 2009, Eric B Munson wrote:
> >
> >> This patch adds an example of how to use the MAP_HUGETLB flag to
> >> the vm documentation.
> >>
> >> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
> >> ---
> >> Changes from V1:
> >> Rebase to newest linux-2.6 tree
> >> Change MAP_LARGEPAGE to MAP_HUGETLB to match flag name in huge page shm
> >>
> >> Documentation/vm/hugetlbpage.txt | 80 ++++++++++++++++++++++++++++++++++++++
> >> 1 files changed, 80 insertions(+), 0 deletions(-)
> >>
> >> diff --git a/Documentation/vm/hugetlbpage.txt b/Documentation/vm/hugetlbpage.txt
> >> index ea8714f..d30fa1a 100644
> >> --- a/Documentation/vm/hugetlbpage.txt
> >> +++ b/Documentation/vm/hugetlbpage.txt
> >> @@ -337,3 +337,83 @@ int main(void)
> >>
> >> return 0;
> >> }
> >> +
> >> +*******************************************************************
> >> +
> >> +/*
> >> + * Example of using hugepage memory in a user application using the mmap
> >> + * system call with MAP_LARGEPAGE flag. Before running this program make
> >
> > s/MAP_LARGEPAGE/MAP_HUGETLB/
>
> I'm (slowly) making source code examples in Documentation/ buildable,
> as this one should be, please.
>
> I.e., put it in a separate source file (hugetlbpage.txt can refer to the
> source file if you want it to) and add a Makefile similar to other
> Makefiles in the Documentation/ tree.
>
> ~Randy
>
I will make these changes for V3, thanks for the reviews.
--
Eric B Munson
IBM Linux Technology Center
ebmunson@us.ibm.com
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread