* Transparent Hugepage Support #29
@ 2010-08-03 13:56 Andrea Arcangeli
2010-08-03 16:01 ` Kernel build bench with " Andrea Arcangeli
` (2 more replies)
0 siblings, 3 replies; 11+ messages in thread
From: Andrea Arcangeli @ 2010-08-03 13:56 UTC (permalink / raw)
To: linux-mm, Andrew Morton, linux-kernel
Cc: Marcelo Tosatti, Adam Litke, Avi Kivity, Izik Eidus, Hugh Dickins,
Nick Piggin, Rik van Riel, Mel Gorman, Dave Hansen,
Benjamin Herrenschmidt, Ingo Molnar, Mike Travis,
KAMEZAWA Hiroyuki, Christoph Lameter, Chris Wright, bpicco,
KOSAKI Motohiro, Balbir Singh, Michael S. Tsirkin, Peter Zijlstra,
Johannes Weiner, Daisuke Nishimura, Chris Mason, Borislav Petkov
http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=shortlog
first: git clone git://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git
or first: git clone --reference linux-2.6 git://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git
later: git fetch; git checkout -f origin/master
The tree is rebased and git pull won't work.
http://www.kernel.org/pub/linux/kernel/people/andrea/patches/v2.6/2.6.35/transparent_hugepage-29/
http://www.kernel.org/pub/linux/kernel/people/andrea/patches/v2.6/2.6.35/transparent_hugepage-29.gz
Diff #28 -> #29:
THP-disable-on-small-systems | 41 ++++++++
Disable THP on <500M systems, from Rik.
compaction-kswapd | 194 ++++++++++++++++++++++++++++++++++++++++
Add a compaction mode for kswapd to fully replace obsolete blind lumpy
reclaim.
free_pages-count | 60 ++++++++++++
free_pages-drain_all_pages | 62 ++++++++++++
free_pages-vmstat | 156 ++++++++++++++++++++++++++++++++
Make free page statistics more accurate from Mel.
ksmd-khugepaged-freeze | 97 ++++++++++++++++++++
kswapd uses set_freezing, so shall ksmd and khugepaged.
transparent-hugepage-nr_rotated | 36 +++++++
transparent-hugepage-stat | 166 ++++++++++++++++++++++++++++++++++
Fix inactive/active stats and nr_rotated, from Rik.
Documentation/vm/transhuge.txt | 283 ++++
arch/alpha/include/asm/mman.h | 2
arch/mips/include/asm/mman.h | 2
arch/parisc/include/asm/mman.h | 2
arch/powerpc/mm/gup.c | 12
arch/x86/include/asm/kvm_host.h | 1
arch/x86/include/asm/paravirt.h | 23
arch/x86/include/asm/paravirt_types.h | 6
arch/x86/include/asm/pgtable-2level.h | 9
arch/x86/include/asm/pgtable-3level.h | 23
arch/x86/include/asm/pgtable.h | 149 ++
arch/x86/include/asm/pgtable_64.h | 28
arch/x86/include/asm/pgtable_types.h | 3
arch/x86/kernel/paravirt.c | 3
arch/x86/kernel/vm86_32.c | 1
arch/x86/kvm/mmu.c | 68 -
arch/x86/kvm/paging_tmpl.h | 6
arch/x86/mm/gup.c | 28
arch/x86/mm/pgtable.c | 66 +
arch/xtensa/include/asm/mman.h | 2
fs/Kconfig | 2
fs/exec.c | 44
fs/proc/meminfo.c | 14
fs/proc/page.c | 14
include/asm-generic/mman-common.h | 2
include/asm-generic/pgtable.h | 130 +
include/linux/compaction.h | 13
include/linux/gfp.h | 14
include/linux/huge_mm.h | 151 ++
include/linux/khugepaged.h | 66 +
include/linux/ksm.h | 20
include/linux/kvm_host.h | 4
include/linux/memory_hotplug.h | 14
include/linux/mm.h | 114 +
include/linux/mm_inline.h | 19
include/linux/mm_types.h | 3
include/linux/mmu_notifier.h | 66 +
include/linux/mmzone.h | 1
include/linux/page-flags.h | 36
include/linux/res_counter.h | 12
include/linux/rmap.h | 33
include/linux/sched.h | 1
include/linux/swap.h | 2
include/linux/vmstat.h | 4
kernel/fork.c | 12
kernel/futex.c | 55
mm/Kconfig | 38
mm/Makefile | 1
mm/compaction.c | 48
mm/huge_memory.c | 2212 ++++++++++++++++++++++++++++++++++
mm/hugetlb.c | 69 -
mm/ksm.c | 86 -
mm/madvise.c | 8
mm/memcontrol.c | 169 +-
mm/memory-failure.c | 2
mm/memory.c | 241 +++
mm/memory_hotplug.c | 14
mm/mempolicy.c | 14
mm/migrate.c | 22
mm/mincore.c | 7
mm/mmap.c | 57
mm/mmu_notifier.c | 20
mm/mprotect.c | 20
mm/mremap.c | 8
mm/oom_kill.c | 1
mm/page_alloc.c | 58
mm/pagewalk.c | 1
mm/rmap.c | 228 ++-
mm/sparse.c | 4
mm/swap.c | 117 +
mm/swap_state.c | 6
mm/swapfile.c | 2
mm/vmscan.c | 102 -
mm/vmstat.c | 31
virt/kvm/iommu.c | 2
virt/kvm/kvm_main.c | 56
76 files changed, 4680 insertions(+), 527 deletions(-)
^ permalink raw reply [flat|nested] 11+ messages in thread* Kernel build bench with Transparent Hugepage Support #29 2010-08-03 13:56 Transparent Hugepage Support #29 Andrea Arcangeli @ 2010-08-03 16:01 ` Andrea Arcangeli 2010-08-04 12:15 ` Ed Tomlinson 2010-08-17 20:56 ` [patch] transparent hugepage sysfs meminfo David Rientjes 2 siblings, 0 replies; 11+ messages in thread From: Andrea Arcangeli @ 2010-08-03 16:01 UTC (permalink / raw) To: linux-mm, Andrew Morton, linux-kernel Cc: Marcelo Tosatti, Adam Litke, Avi Kivity, Izik Eidus, Hugh Dickins, Nick Piggin, Rik van Riel, Mel Gorman, Dave Hansen, Benjamin Herrenschmidt, Ingo Molnar, Mike Travis, KAMEZAWA Hiroyuki, Christoph Lameter, Chris Wright, bpicco, KOSAKI Motohiro, Balbir Singh, Michael S. Tsirkin, Peter Zijlstra, Johannes Weiner, Daisuke Nishimura, Chris Mason, Borislav Petkov, linux-kernel Transparent Hugepage Support worst case macro benchmark: kernel build. host: 24-way SMP (12cores, 2 sockets) 16G RAM guest: 24-way SMP (24 vcpus) 15G RAM - Same kernel in guest and host: aa.git tag THP-29 (2.6.35 based) http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=summary (see linux-mm for more info) - Same gcc (and userland) patched with the patch at the end. - Same kernel source, same .config, on tmpfs (tmpfs only eliminates the measurement error across different runs but ext4 leads to the same average results). - khugepaged default settings (khugepaged taking 0% CPU) - no glibc align tweak (that would improve performance a little further with THP always) The measurement also includes the "make clean", and the full "make -j32" that includes lots of other time consuming operations not getting any benefit from transparent hugepages. If this was pure "gcc" the percentage speedup would be much higher than this. This is a very real life workload that we run on a daily basis, not a microbenchmark at all. Kernel build on bare metal (note the dTLB-load-misses): ====== build ====== #!/bin/bash make clean >/dev/null; make -j32 >/dev/null =================== perf stat -e cycles -e instructions -e dtlb-loads -e dtlb-load-misses --repeat 3 ./build =================== THP always host (fastest base result) Performance counter stats for './build' (3 runs): 4420734012848 cycles ( +- 0.007% ) 2692414418384 instructions # 0.609 IPC ( +- 0.000% ) 696638665612 dTLB-loads ( +- 0.001% ) 2982343758 dTLB-load-misses ( +- 0.051% ) 83.855147696 seconds time elapsed ( +- 0.058% ) THP never host (slowdown 4.06%) Performance counter stats for './build' (3 runs): 4599325985460 cycles ( +- 0.013% ) 2747874065083 instructions # 0.597 IPC ( +- 0.000% ) 710631792376 dTLB-loads ( +- 0.000% ) 4425816093 dTLB-load-misses ( +- 0.039% ) 87.260443531 seconds time elapsed ( +- 0.075% ) Kernel build on KVM powered guest: ======= time (make clean; make -j32) >/dev/null ======= THP always guest, EPT on + THP always host (slowdown 5.67%) NOTE: the total KVM virtualization slowdown for the kernel build with THP always in guest and host compared with bare metal with THP never (like current upstream) is only 1.54%. real 1m28.612s -> 88.612 seconds user 26m13.862s sys 2m11.376s THP never guest, EPT on + THP always host (slowdown 12.71%) real 1m34.516s -> 94.516 seconds user 26m52.929s sys 3m35.509s THP never guest, EPT on + THP never host (slowdown 24.81%) real 1m44.663s -> 104.663 seconds user 28m13.382s sys 5m39.373s THP always guest, EPT off + THP always host (slowdown 198.33%) real 4m10.166s -> 250.166 seconds user 41m5.674s sys 47m37.671s THP never guest, EPT off + THP always host (slowdown 254.43%) real 4m57.211s -> 297.211 seconds user 53m44.302s sys 53m21.600s THP never guest, EPT off + THP never host (slowdown 260.15%) real 5m2.006s -> 302.006 seconds user 53m25.876s sys 53m32.649s This is trivial to reproduce, you can try yourself with aa.git and the below gcc patch. The results are similar to what I got with NPT some time ago with a less complete benchmark (no gcc patched in guest) and only 4 cores. After this worst case macro benchmark, I'll go ahead with more optimal benchmarks (with bigger memory footprint and longer-living tasks not quitting so fast and not including make -j32/make clean etc...) and I expect more pronounced speedups, like the qemu-kvm translate.o gcc build (the file with the automatic generated .c source for JIT emulation), Java etc... And I'll include these results and all other results I'll be getting, in my KVM Forum 2010 talk on Transparent Hugepage Support next week in Boston. Thanks! Andrea --- /var/tmp/portage/sys-devel/gcc-4.4.2/work/gcc-4.4.2/gcc/ggc-page.c 2008-07-28 16:33:56.000000000 +0200 +++ /tmp/gcc-4.4.2/gcc/ggc-page.c 2010-04-25 06:01:32.829753566 +0200 @@ -450,6 +450,11 @@ #define BITMAP_SIZE(Num_objects) \ (CEIL ((Num_objects), HOST_BITS_PER_LONG) * sizeof(long)) +#ifdef __x86_64__ +#define HPAGE_SIZE (2*1024*1024) +#define GGC_QUIRE_SIZE 512 +#endif + /* Allocate pages in chunks of this size, to throttle calls to memory allocation routines. The first page is used, the rest go onto the free list. This cannot be larger than HOST_BITS_PER_INT for the @@ -457,7 +462,7 @@ can override this by defining GGC_QUIRE_SIZE explicitly. */ #ifndef GGC_QUIRE_SIZE # ifdef USING_MMAP -# define GGC_QUIRE_SIZE 256 +# define GGC_QUIRE_SIZE 512 # else # define GGC_QUIRE_SIZE 16 # endif @@ -654,6 +659,23 @@ #ifdef HAVE_MMAP_ANON char *page = (char *) mmap (pref, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); +#ifdef HPAGE_SIZE + if (!(size & (HPAGE_SIZE-1)) && + page != (char *) MAP_FAILED && (size_t) page & (HPAGE_SIZE-1)) { + char *old_page; + munmap(page, size); + page = (char *) mmap (pref, size + HPAGE_SIZE-1, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + old_page = page; + page = (char *) (((size_t)page + HPAGE_SIZE-1) + & ~(HPAGE_SIZE-1)); + if (old_page != page) + munmap(old_page, page-old_page); + if (page != old_page + HPAGE_SIZE-1) + munmap(page+size, old_page+HPAGE_SIZE-1-page); + } +#endif #endif #ifdef HAVE_MMAP_DEV_ZERO char *page = (char *) mmap (pref, size, PROT_READ | PROT_WRITE, ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Transparent Hugepage Support #29 2010-08-03 13:56 Transparent Hugepage Support #29 Andrea Arcangeli 2010-08-03 16:01 ` Kernel build bench with " Andrea Arcangeli @ 2010-08-04 12:15 ` Ed Tomlinson 2010-08-04 12:44 ` Andrea Arcangeli 2010-08-17 20:56 ` [patch] transparent hugepage sysfs meminfo David Rientjes 2 siblings, 1 reply; 11+ messages in thread From: Ed Tomlinson @ 2010-08-04 12:15 UTC (permalink / raw) To: Andrea Arcangeli; +Cc: linux-kernel Andrea, I have been tracking & testing this for a while. It looks very much like Linus wants to merge (or a big chunk of it): git://git.kernel.org/pub/scm/linux/kernel/git/npiggin/linux-npiggin.git vfs-scale-working When I pull an aa-29 on top of this there is one conflict in vmscan.c that git cannot resolve. It would be really nice to have a version of the THS patchs that can be tested with the vfs scaling tree. TIA Ed Tomlinson On Tuesday 03 August 2010 09:56:15 you wrote: > http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=shortlog > > first: git clone git://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git > or first: git clone --reference linux-2.6 git://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git > later: git fetch; git checkout -f origin/master > > The tree is rebased and git pull won't work. > > http://www.kernel.org/pub/linux/kernel/people/andrea/patches/v2.6/2.6.35/transparent_hugepage-29/ > http://www.kernel.org/pub/linux/kernel/people/andrea/patches/v2.6/2.6.35/transparent_hugepage-29.gz > > Diff #28 -> #29: > > THP-disable-on-small-systems | 41 ++++++++ > > Disable THP on <500M systems, from Rik. > > compaction-kswapd | 194 ++++++++++++++++++++++++++++++++++++++++ > > Add a compaction mode for kswapd to fully replace obsolete blind lumpy > reclaim. > > free_pages-count | 60 ++++++++++++ > free_pages-drain_all_pages | 62 ++++++++++++ > free_pages-vmstat | 156 ++++++++++++++++++++++++++++++++ > > Make free page statistics more accurate from Mel. > > ksmd-khugepaged-freeze | 97 ++++++++++++++++++++ > > kswapd uses set_freezing, so shall ksmd and khugepaged. > > transparent-hugepage-nr_rotated | 36 +++++++ > transparent-hugepage-stat | 166 ++++++++++++++++++++++++++++++++++ > > Fix inactive/active stats and nr_rotated, from Rik. > > Documentation/vm/transhuge.txt | 283 ++++ > arch/alpha/include/asm/mman.h | 2 > arch/mips/include/asm/mman.h | 2 > arch/parisc/include/asm/mman.h | 2 > arch/powerpc/mm/gup.c | 12 > arch/x86/include/asm/kvm_host.h | 1 > arch/x86/include/asm/paravirt.h | 23 > arch/x86/include/asm/paravirt_types.h | 6 > arch/x86/include/asm/pgtable-2level.h | 9 > arch/x86/include/asm/pgtable-3level.h | 23 > arch/x86/include/asm/pgtable.h | 149 ++ > arch/x86/include/asm/pgtable_64.h | 28 > arch/x86/include/asm/pgtable_types.h | 3 > arch/x86/kernel/paravirt.c | 3 > arch/x86/kernel/vm86_32.c | 1 > arch/x86/kvm/mmu.c | 68 - > arch/x86/kvm/paging_tmpl.h | 6 > arch/x86/mm/gup.c | 28 > arch/x86/mm/pgtable.c | 66 + > arch/xtensa/include/asm/mman.h | 2 > fs/Kconfig | 2 > fs/exec.c | 44 > fs/proc/meminfo.c | 14 > fs/proc/page.c | 14 > include/asm-generic/mman-common.h | 2 > include/asm-generic/pgtable.h | 130 + > include/linux/compaction.h | 13 > include/linux/gfp.h | 14 > include/linux/huge_mm.h | 151 ++ > include/linux/khugepaged.h | 66 + > include/linux/ksm.h | 20 > include/linux/kvm_host.h | 4 > include/linux/memory_hotplug.h | 14 > include/linux/mm.h | 114 + > include/linux/mm_inline.h | 19 > include/linux/mm_types.h | 3 > include/linux/mmu_notifier.h | 66 + > include/linux/mmzone.h | 1 > include/linux/page-flags.h | 36 > include/linux/res_counter.h | 12 > include/linux/rmap.h | 33 > include/linux/sched.h | 1 > include/linux/swap.h | 2 > include/linux/vmstat.h | 4 > kernel/fork.c | 12 > kernel/futex.c | 55 > mm/Kconfig | 38 > mm/Makefile | 1 > mm/compaction.c | 48 > mm/huge_memory.c | 2212 ++++++++++++++++++++++++++++++++++ > mm/hugetlb.c | 69 - > mm/ksm.c | 86 - > mm/madvise.c | 8 > mm/memcontrol.c | 169 +- > mm/memory-failure.c | 2 > mm/memory.c | 241 +++ > mm/memory_hotplug.c | 14 > mm/mempolicy.c | 14 > mm/migrate.c | 22 > mm/mincore.c | 7 > mm/mmap.c | 57 > mm/mmu_notifier.c | 20 > mm/mprotect.c | 20 > mm/mremap.c | 8 > mm/oom_kill.c | 1 > mm/page_alloc.c | 58 > mm/pagewalk.c | 1 > mm/rmap.c | 228 ++- > mm/sparse.c | 4 > mm/swap.c | 117 + > mm/swap_state.c | 6 > mm/swapfile.c | 2 > mm/vmscan.c | 102 - > mm/vmstat.c | 31 > virt/kvm/iommu.c | 2 > virt/kvm/kvm_main.c | 56 > 76 files changed, 4680 insertions(+), 527 deletions(-) > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Transparent Hugepage Support #29 2010-08-04 12:15 ` Ed Tomlinson @ 2010-08-04 12:44 ` Andrea Arcangeli 2010-08-05 11:27 ` Ed Tomlinson 0 siblings, 1 reply; 11+ messages in thread From: Andrea Arcangeli @ 2010-08-04 12:44 UTC (permalink / raw) To: Ed Tomlinson; +Cc: linux-kernel Hi Ed, On Wed, Aug 04, 2010 at 08:15:02AM -0400, Ed Tomlinson wrote: > Andrea, > > I have been tracking & testing this for a while. It looks very much like Linus wants to merge (or a big chunk of it): > > git://git.kernel.org/pub/scm/linux/kernel/git/npiggin/linux-npiggin.git vfs-scale-working > > When I pull an aa-29 on top of this there is one conflict in vmscan.c that git cannot resolve. > It would be really nice to have a version of the THS patchs that can be tested with the vfs scaling > tree. please try: zcat transparent-hugepage-29.gz |patch -p1 It didn't spawn rejects here, maybe something's wrong in some intra-patch or git is too picky. If it doesn't build let me know and I will look into it, thanks! Andrea ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Transparent Hugepage Support #29 2010-08-04 12:44 ` Andrea Arcangeli @ 2010-08-05 11:27 ` Ed Tomlinson 2010-08-05 11:49 ` Andrea Arcangeli 0 siblings, 1 reply; 11+ messages in thread From: Ed Tomlinson @ 2010-08-05 11:27 UTC (permalink / raw) To: Andrea Arcangeli; +Cc: linux-kernel On Wednesday 04 August 2010 08:44:42 Andrea Arcangeli wrote: > Hi Ed, > > On Wed, Aug 04, 2010 at 08:15:02AM -0400, Ed Tomlinson wrote: > > Andrea, > > > > I have been tracking & testing this for a while. It looks very much like Linus wants to merge (or a big chunk of it): > > > > git://git.kernel.org/pub/scm/linux/kernel/git/npiggin/linux-npiggin.git vfs-scale-working > > > > When I pull an aa-29 on top of this there is one conflict in vmscan.c that git cannot resolve. > > It would be really nice to have a version of the THS patchs that can be tested with the vfs scaling > > tree. > > please try: > > zcat transparent-hugepage-29.gz |patch -p1 > > It didn't spawn rejects here, maybe something's wrong in some > intra-patch or git is too picky. If it doesn't build let me know and I > will look into it, thanks! On a tree with: commit 9d9cec888c3c4a2d5b5f01c3f3984fc4bc602681 Merge: dd6c927 9edd35f Author: Ed <edt@aei.ca> Date: Wed Aug 4 15:18:53 2010 -0400 Merge branch 'vfs-scale-working' of git://git.kernel.org/pub/scm/linux/kernel/git/npiggin/linux-npiggin into vfs-scaling merged on top of commit 9fe6206f400646a2322096b56c59891d530e8d51 Author: Linus Torvalds <torvalds@linux-foundation.org> Date: Sun Aug 1 15:11:14 2010 -0700 Linux 2.6.35 doing: bzcat ../transparent-hugepage-29.bz2 | patch -p1 I get one reject: patching file mm/vmscan.c Hunk #3 succeeded at 612 (offset 43 lines). Hunk #4 succeeded at 968 (offset 43 lines). Hunk #5 succeeded at 1026 (offset 43 lines). Hunk #6 succeeded at 1063 (offset 43 lines). Hunk #7 succeeded at 1168 (offset 43 lines). Hunk #8 succeeded at 1179 (offset 43 lines). Hunk #9 succeeded at 1198 (offset 43 lines). Hunk #10 succeeded at 1236 (offset 43 lines). Hunk #11 succeeded at 1263 (offset 43 lines). Hunk #12 succeeded at 1333 (offset 43 lines). Hunk #13 succeeded at 1401 (offset 43 lines). Hunk #14 succeeded at 1658 (offset 43 lines). Hunk #15 FAILED at 1629. Hunk #16 succeeded at 2112 with fuzz 2 (offset 51 lines). Hunk #17 FAILED at 2093. Hunk #18 succeeded at 2169 (offset 46 lines). 2 out of 18 hunks FAILED -- saving rejects to file mm/vmscan.c.rej patching file mm/vmstat.c --- cat mm/vmscan.c.rej --- mm/vmscan.c +++ mm/vmscan.c @@ -1629,8 +1586,6 @@ get_scan_count(zone, sc, nr, priority); - set_lumpy_reclaim_mode(priority, sc); - while (nr[LRU_INACTIVE_ANON] || nr[LRU_ACTIVE_FILE] || nr[LRU_INACTIVE_FILE]) { for_each_evictable_lru(l) { @@ -2093,9 +2049,25 @@ lru_pages); sc.nr_reclaimed += reclaim_state->reclaimed_slab; total_scanned += sc.nr_scanned; + compaction = 0; + + if (order && + zone_watermark_ok(zone, 0, + high_wmark_pages(zone), + end_zone, 0) && + !zone_watermark_ok(zone, order, + high_wmark_pages(zone), + end_zone, 0)) { + compact_zone_order(zone, + order, + sc.gfp_mask, + COMPACT_MODE_KSWAPD); + compaction = 1; + } + if (zone->all_unreclaimable) continue; - if (nr_slab == 0 && + if (!compaction && nr_slab == 0 && zone->pages_scanned >= (zone_reclaimable_pages(zone) * 6)) zone->all_unreclaimable = 1; /* -- Ideas? Ed Tomlinson ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Transparent Hugepage Support #29 2010-08-05 11:27 ` Ed Tomlinson @ 2010-08-05 11:49 ` Andrea Arcangeli 2010-08-05 11:57 ` Ed Tomlinson 0 siblings, 1 reply; 11+ messages in thread From: Andrea Arcangeli @ 2010-08-05 11:49 UTC (permalink / raw) To: Ed Tomlinson; +Cc: linux-kernel On Thu, Aug 05, 2010 at 07:27:24AM -0400, Ed Tomlinson wrote: > On a tree with: > > commit 9d9cec888c3c4a2d5b5f01c3f3984fc4bc602681 > Merge: dd6c927 9edd35f > Author: Ed <edt@aei.ca> > Date: Wed Aug 4 15:18:53 2010 -0400 > > Merge branch 'vfs-scale-working' of git://git.kernel.org/pub/scm/linux/kernel/git/npiggin/linux-npiggin into vfs-scaling > > merged on top of > > commit 9fe6206f400646a2322096b56c59891d530e8d51 > Author: Linus Torvalds <torvalds@linux-foundation.org> > Date: Sun Aug 1 15:11:14 2010 -0700 > > Linux 2.6.35 Well I didn't realize you also merged 2.6.35 on top of it. This is no standard tree, so I suggest you to checkout -f origin/master of Nick's tree until he updates to 2.6.35 himself (then I can port to it). ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Transparent Hugepage Support #29 2010-08-05 11:49 ` Andrea Arcangeli @ 2010-08-05 11:57 ` Ed Tomlinson 2010-08-05 14:22 ` Nick Piggin 2010-08-05 19:45 ` Nick Piggin 0 siblings, 2 replies; 11+ messages in thread From: Ed Tomlinson @ 2010-08-05 11:57 UTC (permalink / raw) To: Andrea Arcangeli; +Cc: linux-kernel, Nick Piggin On Thursday 05 August 2010 07:49:15 Andrea Arcangeli wrote: > On Thu, Aug 05, 2010 at 07:27:24AM -0400, Ed Tomlinson wrote: > > On a tree with: > > > > commit 9d9cec888c3c4a2d5b5f01c3f3984fc4bc602681 > > Merge: dd6c927 9edd35f > > Author: Ed <edt@aei.ca> > > Date: Wed Aug 4 15:18:53 2010 -0400 > > > > Merge branch 'vfs-scale-working' of git://git.kernel.org/pub/scm/linux/kernel/git/npiggin/linux-npiggin into vfs-scaling > > > > merged on top of > > > > commit 9fe6206f400646a2322096b56c59891d530e8d51 > > Author: Linus Torvalds <torvalds@linux-foundation.org> > > Date: Sun Aug 1 15:11:14 2010 -0700 > > > > Linux 2.6.35 > > Well I didn't realize you also merged 2.6.35 on top of it. This is no > standard tree, so I suggest you to checkout -f origin/master of Nick's > tree until he updates to 2.6.35 himself (then I can port to it). Nick, Any chance of the above happening? I'd love to test both the vfs scaling changes and Andrea's much improved hugepage support together on top of .35 TIA Ed Tomlinson ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Transparent Hugepage Support #29 2010-08-05 11:57 ` Ed Tomlinson @ 2010-08-05 14:22 ` Nick Piggin 2010-08-05 19:45 ` Nick Piggin 1 sibling, 0 replies; 11+ messages in thread From: Nick Piggin @ 2010-08-05 14:22 UTC (permalink / raw) To: Ed Tomlinson; +Cc: Andrea Arcangeli, linux-kernel, Nick Piggin On Thu, Aug 05, 2010 at 07:57:52AM -0400, Ed Tomlinson wrote: > On Thursday 05 August 2010 07:49:15 Andrea Arcangeli wrote: > > On Thu, Aug 05, 2010 at 07:27:24AM -0400, Ed Tomlinson wrote: > > > On a tree with: > > > > > > commit 9d9cec888c3c4a2d5b5f01c3f3984fc4bc602681 > > > Merge: dd6c927 9edd35f > > > Author: Ed <edt@aei.ca> > > > Date: Wed Aug 4 15:18:53 2010 -0400 > > > > > > Merge branch 'vfs-scale-working' of git://git.kernel.org/pub/scm/linux/kernel/git/npiggin/linux-npiggin into vfs-scaling > > > > > > merged on top of > > > > > > commit 9fe6206f400646a2322096b56c59891d530e8d51 > > > Author: Linus Torvalds <torvalds@linux-foundation.org> > > > Date: Sun Aug 1 15:11:14 2010 -0700 > > > > > > Linux 2.6.35 > > > > Well I didn't realize you also merged 2.6.35 on top of it. This is no > > standard tree, so I suggest you to checkout -f origin/master of Nick's > > tree until he updates to 2.6.35 himself (then I can port to it). > > Nick, > > Any chance of the above happening? I'd love to test both the vfs > scaling changes and Andrea's much improved hugepage support together > on top of .35 I was going to rebase to 2.6.35 sometime, but it may have to wait until I get back from LSF/MM if I don't get time this week. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Transparent Hugepage Support #29 2010-08-05 11:57 ` Ed Tomlinson 2010-08-05 14:22 ` Nick Piggin @ 2010-08-05 19:45 ` Nick Piggin 1 sibling, 0 replies; 11+ messages in thread From: Nick Piggin @ 2010-08-05 19:45 UTC (permalink / raw) To: Ed Tomlinson; +Cc: Andrea Arcangeli, linux-kernel, Nick Piggin On Thu, Aug 05, 2010 at 07:57:52AM -0400, Ed Tomlinson wrote: > On Thursday 05 August 2010 07:49:15 Andrea Arcangeli wrote: > > On Thu, Aug 05, 2010 at 07:27:24AM -0400, Ed Tomlinson wrote: > > > On a tree with: > > > > > > commit 9d9cec888c3c4a2d5b5f01c3f3984fc4bc602681 > > > Merge: dd6c927 9edd35f > > > Author: Ed <edt@aei.ca> > > > Date: Wed Aug 4 15:18:53 2010 -0400 > > > > > > Merge branch 'vfs-scale-working' of git://git.kernel.org/pub/scm/linux/kernel/git/npiggin/linux-npiggin into vfs-scaling > > > > > > merged on top of > > > > > > commit 9fe6206f400646a2322096b56c59891d530e8d51 > > > Author: Linus Torvalds <torvalds@linux-foundation.org> > > > Date: Sun Aug 1 15:11:14 2010 -0700 > > > > > > Linux 2.6.35 > > > > Well I didn't realize you also merged 2.6.35 on top of it. This is no > > standard tree, so I suggest you to checkout -f origin/master of Nick's > > tree until he updates to 2.6.35 himself (then I can port to it). > > Nick, > > Any chance of the above happening? I'd love to test both the vfs scaling changes > and Andrea's much improved hugepage support together on top of .35 Try vfs-scale branch now. Cleaned up and rebased on 2.6.35 although it is not as well tested as I would like but I won't have any more time on it until next week. vfs-scale-working retained history of fixes. I'll start another vfs-scale-working-2.6.36 branch for new fixes. Thanks, Nick ^ permalink raw reply [flat|nested] 11+ messages in thread
* [patch] transparent hugepage sysfs meminfo 2010-08-03 13:56 Transparent Hugepage Support #29 Andrea Arcangeli 2010-08-03 16:01 ` Kernel build bench with " Andrea Arcangeli 2010-08-04 12:15 ` Ed Tomlinson @ 2010-08-17 20:56 ` David Rientjes 2010-08-17 21:01 ` Rik van Riel 2 siblings, 1 reply; 11+ messages in thread From: David Rientjes @ 2010-08-17 20:56 UTC (permalink / raw) To: Andrea Arcangeli; +Cc: Rik van Riel, linux-mm, linux-kernel Add hugepage statistics to per-node sysfs meminfo Cc: Rik van Riel <riel@redhat.com> Signed-off-by: David Rientjes <rientjes@google.com> --- drivers/base/node.c | 21 ++++++++++++++++++--- 1 files changed, 18 insertions(+), 3 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -98,7 +98,11 @@ static ssize_t node_read_meminfo(struct sys_device * dev, "Node %d WritebackTmp: %8lu kB\n" "Node %d Slab: %8lu kB\n" "Node %d SReclaimable: %8lu kB\n" - "Node %d SUnreclaim: %8lu kB\n", + "Node %d SUnreclaim: %8lu kB\n" +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + "Node %d AnonHugePages: %8lu kB\n" +#endif + , nid, K(i.totalram), nid, K(i.freeram), nid, K(i.totalram - i.freeram), @@ -122,7 +126,12 @@ static ssize_t node_read_meminfo(struct sys_device * dev, nid, K(node_page_state(nid, NR_WRITEBACK)), nid, K(node_page_state(nid, NR_FILE_PAGES)), nid, K(node_page_state(nid, NR_FILE_MAPPED)), - nid, K(node_page_state(nid, NR_ANON_PAGES)), + nid, K(node_page_state(nid, NR_ANON_PAGES) +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + + node_page_state(nid, NR_ANON_TRANSPARENT_HUGEPAGES) * + HPAGE_PMD_NR +#endif + ), nid, K(node_page_state(nid, NR_SHMEM)), nid, node_page_state(nid, NR_KERNEL_STACK) * THREAD_SIZE / 1024, @@ -133,7 +142,13 @@ static ssize_t node_read_meminfo(struct sys_device * dev, nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE) + node_page_state(nid, NR_SLAB_UNRECLAIMABLE)), nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE)), - nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE))); + nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE)) +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + , nid, + K(node_page_state(nid, NR_ANON_TRANSPARENT_HUGEPAGES) * + HPAGE_PMD_NR) +#endif + ); n += hugetlb_report_node_meminfo(nid, buf + n); return n; } ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [patch] transparent hugepage sysfs meminfo 2010-08-17 20:56 ` [patch] transparent hugepage sysfs meminfo David Rientjes @ 2010-08-17 21:01 ` Rik van Riel 0 siblings, 0 replies; 11+ messages in thread From: Rik van Riel @ 2010-08-17 21:01 UTC (permalink / raw) To: David Rientjes; +Cc: Andrea Arcangeli, linux-mm, linux-kernel On 08/17/2010 04:56 PM, David Rientjes wrote: > Add hugepage statistics to per-node sysfs meminfo > > Cc: Rik van Riel<riel@redhat.com> > Signed-off-by: David Rientjes<rientjes@google.com> Reviewed-by: Rik van Riel <riel@redhat.com> -- All rights reversed ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2010-08-17 21:01 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-08-03 13:56 Transparent Hugepage Support #29 Andrea Arcangeli 2010-08-03 16:01 ` Kernel build bench with " Andrea Arcangeli 2010-08-04 12:15 ` Ed Tomlinson 2010-08-04 12:44 ` Andrea Arcangeli 2010-08-05 11:27 ` Ed Tomlinson 2010-08-05 11:49 ` Andrea Arcangeli 2010-08-05 11:57 ` Ed Tomlinson 2010-08-05 14:22 ` Nick Piggin 2010-08-05 19:45 ` Nick Piggin 2010-08-17 20:56 ` [patch] transparent hugepage sysfs meminfo David Rientjes 2010-08-17 21:01 ` Rik van Riel
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).