* Re: THP: AnonHugePages in /proc/[pid]/smaps is correct or not?
2013-04-01 22:23 ` David Rientjes
@ 2013-04-02 3:09 ` Zhouping Liu
2013-04-02 3:40 ` Lin Feng
2013-04-02 12:23 ` Simon Jeons
2013-04-02 12:26 ` Simon Jeons
2 siblings, 1 reply; 10+ messages in thread
From: Zhouping Liu @ 2013-04-02 3:09 UTC (permalink / raw)
To: David Rientjes
Cc: Andrea Arcangeli, Hugh Dickins, Mel Gorman, linux-mm,
linux-kernel, Amos Kong
On 04/02/2013 06:23 AM, David Rientjes wrote:
> On Mon, 1 Apr 2013, Zhouping Liu wrote:
>
>> Hi all,
>>
>> I found THP can't correctly distinguish one anonymous hugepage map.
>>
>> 1. when /sys/kernel/mm/transparent_hugepage/enabled is 'always', the
>> amount of THP always is one less.
>>
> It's not a problem with identifying an anonymous mapping as a hugepage,
> setting thp enabled to "always" does not guarantee that they will always
> be allocatable or that your mmap() will be 2MB aligned. Your sample code
> is using mmap() instead of posix_memalign() so you'll probably only get
> 100% hugepages only 1/512th of the time.
I don't understand clearly the last sentence 'you'll probably only get
100% hugepages only 1/512th of the time.'
could you please explain more details about 'only 1/512th of the time'?
>
>> 2. when /sys/kernel/mm/transparent_hugepage/enabled is 'madvise', THP can't
>> distinguish any one anonymous hugepage size:
>>
>> Testing code:
>> -------- snip --------
>> unsigned long hugepagesize = (1UL << 21);
>>
>> int main()
>> {
>> void *addr;
>> int i;
>>
>> printf("pid is %d\n", getpid());
>>
>> for (i = 0; i < 5; i++) {
>> addr = mmap(NULL, hugepagesize, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON, -1, 0);
>>
>> if (addr == MAP_FAILED) {
>> perror("mmap");
>> return -1;
>> }
>>
>> if (madvise(addr, hugepagesize, MADV_HUGEPAGE) == -1) {
>> perror("madvise");
>> return -1;
>> }
>>
>> memset(addr, i, hugepagesize);
>> }
>>
>> sleep(50);
>>
>> return 0;
>> }
>> --------- snip ----------
>>
>> The result is that it can't find any AnonHugePages from /proc/[pid]/smaps :
>> -------------- snip -------
>> 7f0b38cd0000-7f0b396d0000 rw-p 00000000 00:00 0
>> Size: 10240 kB
>> Rss: 10240 kB
>> Pss: 10240 kB
>> Shared_Clean: 0 kB
>> Shared_Dirty: 0 kB
>> Private_Clean: 0 kB
>> Private_Dirty: 10240 kB
>> Referenced: 10240 kB
>> Anonymous: 10240 kB
>> AnonHugePages: 0 kB
>> Swap: 0 kB
>> KernelPageSize: 4 kB
>> MMUPageSize: 4 kB
>> Locked: 0 kB
>> VmFlags: rd wr mr mw me ac
> "hg" would be shown in VmFlags if your MADV_HUGEPAGE was successful, are
> you sure this is the right vma?
I think it's the same issue as the above, according to the sample code,
it does mmap() five times in total,
and each time it map 2MB anonymous maps, as all the maps maybe aren't
2MB aligned, so "AnonHugePages"
show 0 kB, and no "hg" VmFalgs.
so, again, if I understand correctly, thp should tune the naturally
aligned maps, such as generated by mmap()/malloc(),
make such maps 'hugepagesize' aligned if the maps or vma is equal and
greater than 'hugepagesize', doesn't it?
Thanks,
Zhouping
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: THP: AnonHugePages in /proc/[pid]/smaps is correct or not?
2013-04-02 3:09 ` Zhouping Liu
@ 2013-04-02 3:40 ` Lin Feng
2013-04-02 3:44 ` David Rientjes
2013-04-02 3:48 ` Zhouping Liu
0 siblings, 2 replies; 10+ messages in thread
From: Lin Feng @ 2013-04-02 3:40 UTC (permalink / raw)
To: Zhouping Liu
Cc: David Rientjes, Andrea Arcangeli, Hugh Dickins, Mel Gorman,
linux-mm, linux-kernel, Amos Kong
Hi Zhouping,
On 04/02/2013 11:09 AM, Zhouping Liu wrote:
> I don't understand clearly the last sentence 'you'll probably only get 100% hugepages only 1/512th of the time.'
> could you please explain more details about 'only 1/512th of the time'?
IIUC, thp size is 2M so it may be comprised of 512 normal page(size 4k).
Since your test code is not 2M aligned(not using posix_memalign()) so
the start address of the mapped vma will be random, such as
2M*i+4k*1, 2M*i+4k*2...2M*i+k4*511, there is 512 possibilities.
The only chance you get thp happens when the first map just starts at 2M*i,
and the consequent maps also benefit from this.
-------- snip --------
>
> so, again, if I understand correctly, thp should tune the naturally aligned maps, such as generated by mmap()/malloc(),
> make such maps 'hugepagesize' aligned if the maps or vma is equal and greater than 'hugepagesize', doesn't it?
We may gain performance improving from this.
thanks,
linfeng
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: THP: AnonHugePages in /proc/[pid]/smaps is correct or not?
2013-04-02 3:40 ` Lin Feng
@ 2013-04-02 3:44 ` David Rientjes
2013-04-02 3:48 ` Zhouping Liu
1 sibling, 0 replies; 10+ messages in thread
From: David Rientjes @ 2013-04-02 3:44 UTC (permalink / raw)
To: Lin Feng
Cc: Zhouping Liu, Andrea Arcangeli, Hugh Dickins, Mel Gorman,
linux-mm, linux-kernel, Amos Kong
On Tue, 2 Apr 2013, Lin Feng wrote:
> > so, again, if I understand correctly, thp should tune the naturally aligned maps, such as generated by mmap()/malloc(),
> > make such maps 'hugepagesize' aligned if the maps or vma is equal and greater than 'hugepagesize', doesn't it?
>
> We may gain performance improving from this.
>
To attain the maximum number of hugepages, you would naturally want to
ensure that the mappings are done aligned to 2MB; for very large
allocations, missing one or two hugepages typically won't hurt performance
much. posix_memalign() is the best way of doing this which just wraps
mmap() for the needed alignment. More interesting is creating your own
custom malloc() that allocates in 2MB aligned chunks, if possible, and
uses 2MB aligned arenas for its own metadata. If you do that for
malloc(), then you'll only need to make code that does its own mmap()s to
use posix_memalign().
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: THP: AnonHugePages in /proc/[pid]/smaps is correct or not?
2013-04-02 3:40 ` Lin Feng
2013-04-02 3:44 ` David Rientjes
@ 2013-04-02 3:48 ` Zhouping Liu
1 sibling, 0 replies; 10+ messages in thread
From: Zhouping Liu @ 2013-04-02 3:48 UTC (permalink / raw)
To: Lin Feng
Cc: David Rientjes, Andrea Arcangeli, Hugh Dickins, Mel Gorman,
linux-mm, linux-kernel, Amos Kong
On 04/02/2013 11:40 AM, Lin Feng wrote:
> Hi Zhouping,
>
> On 04/02/2013 11:09 AM, Zhouping Liu wrote:
>> I don't understand clearly the last sentence 'you'll probably only get 100% hugepages only 1/512th of the time.'
>> could you please explain more details about 'only 1/512th of the time'?
> IIUC, thp size is 2M so it may be comprised of 512 normal page(size 4k).
> Since your test code is not 2M aligned(not using posix_memalign()) so
> the start address of the mapped vma will be random, such as
> 2M*i+4k*1, 2M*i+4k*2...2M*i+k4*511, there is 512 possibilities.
>
> The only chance you get thp happens when the first map just starts at 2M*i,
> and the consequent maps also benefit from this.
Feng, it's easy to understand now, thanks for your detailed explanation :)
Zhouping
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: THP: AnonHugePages in /proc/[pid]/smaps is correct or not?
2013-04-01 22:23 ` David Rientjes
2013-04-02 3:09 ` Zhouping Liu
@ 2013-04-02 12:23 ` Simon Jeons
2013-04-02 18:09 ` David Rientjes
2013-04-02 12:26 ` Simon Jeons
2 siblings, 1 reply; 10+ messages in thread
From: Simon Jeons @ 2013-04-02 12:23 UTC (permalink / raw)
To: David Rientjes
Cc: Zhouping Liu, Andrea Arcangeli, Hugh Dickins, Mel Gorman,
linux-mm, linux-kernel, Amos Kong
Hi David,
On 04/02/2013 06:23 AM, David Rientjes wrote:
> On Mon, 1 Apr 2013, Zhouping Liu wrote:
>
>> Hi all,
>>
>> I found THP can't correctly distinguish one anonymous hugepage map.
>>
>> 1. when /sys/kernel/mm/transparent_hugepage/enabled is 'always', the
>> amount of THP always is one less.
>>
> It's not a problem with identifying an anonymous mapping as a hugepage,
> setting thp enabled to "always" does not guarantee that they will always
> be allocatable or that your mmap() will be 2MB aligned. Your sample code
Both thp and hugetlb pages should be 2MB aligned, correct?
> is using mmap() instead of posix_memalign() so you'll probably only get
> 100% hugepages only 1/512th of the time.
>
>> 2. when /sys/kernel/mm/transparent_hugepage/enabled is 'madvise', THP can't
>> distinguish any one anonymous hugepage size:
>>
>> Testing code:
>> -------- snip --------
>> unsigned long hugepagesize = (1UL << 21);
>>
>> int main()
>> {
>> void *addr;
>> int i;
>>
>> printf("pid is %d\n", getpid());
>>
>> for (i = 0; i < 5; i++) {
>> addr = mmap(NULL, hugepagesize, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON, -1, 0);
>>
>> if (addr == MAP_FAILED) {
>> perror("mmap");
>> return -1;
>> }
>>
>> if (madvise(addr, hugepagesize, MADV_HUGEPAGE) == -1) {
>> perror("madvise");
>> return -1;
>> }
>>
>> memset(addr, i, hugepagesize);
>> }
>>
>> sleep(50);
>>
>> return 0;
>> }
>> --------- snip ----------
>>
>> The result is that it can't find any AnonHugePages from /proc/[pid]/smaps :
>> -------------- snip -------
>> 7f0b38cd0000-7f0b396d0000 rw-p 00000000 00:00 0
>> Size: 10240 kB
>> Rss: 10240 kB
>> Pss: 10240 kB
>> Shared_Clean: 0 kB
>> Shared_Dirty: 0 kB
>> Private_Clean: 0 kB
>> Private_Dirty: 10240 kB
>> Referenced: 10240 kB
>> Anonymous: 10240 kB
>> AnonHugePages: 0 kB
>> Swap: 0 kB
>> KernelPageSize: 4 kB
>> MMUPageSize: 4 kB
>> Locked: 0 kB
>> VmFlags: rd wr mr mw me ac
> "hg" would be shown in VmFlags if your MADV_HUGEPAGE was successful, are
> you sure this is the right vma?
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: THP: AnonHugePages in /proc/[pid]/smaps is correct or not?
2013-04-02 12:23 ` Simon Jeons
@ 2013-04-02 18:09 ` David Rientjes
2013-04-02 23:58 ` Simon Jeons
0 siblings, 1 reply; 10+ messages in thread
From: David Rientjes @ 2013-04-02 18:09 UTC (permalink / raw)
To: Simon Jeons
Cc: Zhouping Liu, Andrea Arcangeli, Hugh Dickins, Mel Gorman,
linux-mm, linux-kernel, Amos Kong
On Tue, 2 Apr 2013, Simon Jeons wrote:
> Both thp and hugetlb pages should be 2MB aligned, correct?
>
To answer this question and your followup reply at the same time: they
come from one level higher in the page table so they will naturally need
to be 2MB aligned.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: THP: AnonHugePages in /proc/[pid]/smaps is correct or not?
2013-04-02 18:09 ` David Rientjes
@ 2013-04-02 23:58 ` Simon Jeons
0 siblings, 0 replies; 10+ messages in thread
From: Simon Jeons @ 2013-04-02 23:58 UTC (permalink / raw)
To: David Rientjes
Cc: Zhouping Liu, Andrea Arcangeli, Hugh Dickins, Mel Gorman,
linux-mm, linux-kernel, Amos Kong
Hi David,
On 04/03/2013 02:09 AM, David Rientjes wrote:
> On Tue, 2 Apr 2013, Simon Jeons wrote:
>
>> Both thp and hugetlb pages should be 2MB aligned, correct?
>>
> To answer this question and your followup reply at the same time: they
> come from one level higher in the page table so they will naturally need
> to be 2MB aligned.
When I hacking arch/x86/mm/hugetlbpage.c like this,
diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c
index ae1aa71..87f34ee 100644
--- a/arch/x86/mm/hugetlbpage.c
+++ b/arch/x86/mm/hugetlbpage.c
@@ -354,14 +354,13 @@ hugetlb_get_unmapped_area(struct file *file,
unsigned long addr,
#endif /*HAVE_ARCH_HUGETLB_UNMAPPED_AREA*/
-#ifdef CONFIG_X86_64
static __init int setup_hugepagesz(char *opt)
{
unsigned long ps = memparse(opt, &opt);
if (ps == PMD_SIZE) {
hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
- } else if (ps == PUD_SIZE && cpu_has_gbpages) {
- hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+ } else if (ps == PUD_SIZE) {
+ hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT+4);
} else {
printk(KERN_ERR "hugepagesz: Unsupported page size %lu M\n",
ps >> 20);
I set boot=hugepagesz=1G hugepages=10, then I got 10 32MB huge pages.
What's the difference between these pages which I hacking and normal
huge pages?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: THP: AnonHugePages in /proc/[pid]/smaps is correct or not?
2013-04-01 22:23 ` David Rientjes
2013-04-02 3:09 ` Zhouping Liu
2013-04-02 12:23 ` Simon Jeons
@ 2013-04-02 12:26 ` Simon Jeons
2 siblings, 0 replies; 10+ messages in thread
From: Simon Jeons @ 2013-04-02 12:26 UTC (permalink / raw)
To: David Rientjes
Cc: Zhouping Liu, Andrea Arcangeli, Hugh Dickins, Mel Gorman,
linux-mm, linux-kernel, Amos Kong
On 04/02/2013 06:23 AM, David Rientjes wrote:
> On Mon, 1 Apr 2013, Zhouping Liu wrote:
>
>> Hi all,
>>
>> I found THP can't correctly distinguish one anonymous hugepage map.
>>
>> 1. when /sys/kernel/mm/transparent_hugepage/enabled is 'always', the
>> amount of THP always is one less.
>>
> It's not a problem with identifying an anonymous mapping as a hugepage,
> setting thp enabled to "always" does not guarantee that they will always
> be allocatable or that your mmap() will be 2MB aligned. Your sample code
Btw, why need 2MB aligned? Does it has relationship with tlb?
>
> is using mmap() instead of posix_memalign() so you'll probably only get
> 100% hugepages only 1/512th of the time.
>
>> 2. when /sys/kernel/mm/transparent_hugepage/enabled is 'madvise', THP can't
>> distinguish any one anonymous hugepage size:
>>
>> Testing code:
>> -------- snip --------
>> unsigned long hugepagesize = (1UL << 21);
>>
>> int main()
>> {
>> void *addr;
>> int i;
>>
>> printf("pid is %d\n", getpid());
>>
>> for (i = 0; i < 5; i++) {
>> addr = mmap(NULL, hugepagesize, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON, -1, 0);
>>
>> if (addr == MAP_FAILED) {
>> perror("mmap");
>> return -1;
>> }
>>
>> if (madvise(addr, hugepagesize, MADV_HUGEPAGE) == -1) {
>> perror("madvise");
>> return -1;
>> }
>>
>> memset(addr, i, hugepagesize);
>> }
>>
>> sleep(50);
>>
>> return 0;
>> }
>> --------- snip ----------
>>
>> The result is that it can't find any AnonHugePages from /proc/[pid]/smaps :
>> -------------- snip -------
>> 7f0b38cd0000-7f0b396d0000 rw-p 00000000 00:00 0
>> Size: 10240 kB
>> Rss: 10240 kB
>> Pss: 10240 kB
>> Shared_Clean: 0 kB
>> Shared_Dirty: 0 kB
>> Private_Clean: 0 kB
>> Private_Dirty: 10240 kB
>> Referenced: 10240 kB
>> Anonymous: 10240 kB
>> AnonHugePages: 0 kB
>> Swap: 0 kB
>> KernelPageSize: 4 kB
>> MMUPageSize: 4 kB
>> Locked: 0 kB
>> VmFlags: rd wr mr mw me ac
> "hg" would be shown in VmFlags if your MADV_HUGEPAGE was successful, are
> you sure this is the right vma?
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 10+ messages in thread