* [PATCH] mm: numa: Return the number of base pages altered by protection changes
@ 2013-11-09 14:37 Mel Gorman
2013-11-11 0:22 ` Rik van Riel
0 siblings, 1 reply; 4+ messages in thread
From: Mel Gorman @ 2013-11-09 14:37 UTC (permalink / raw)
To: Andrew Morton; +Cc: Rik van Riel, Alex Thorlton, linux-kernel, linux-mm
Commit 0255d491 (mm: Account for a THP NUMA hinting update as one PTE
update) was added to account for the number of PTE updates when marking
pages prot_numa. task_numa_work was using the old return value to track
how much address space had been updated. Altering the return value causes
the scanner to do more work than it is configured or documented to in a
single unit of work.
This patch reverts 0255d491 and accounts for the number of THP updates
separately in vmstat. It is up to the administrator to interpret the pair
of values correctly. This is a straight-forward operation and likely to
only be of interest when actively debugging NUMA balancing problems.
The impact of this patch is that the NUMA PTE scanner will scan slower when
THP is enabled and workloads may converge slower as a result. On the flip
size system CPU usage should be lower than recent tests reported. This is
an illustrative example of a short single JVM specjbb test
specjbb
3.12.0 3.12.0
vanilla acctupdates
TPut 1 26143.00 ( 0.00%) 25747.00 ( -1.51%)
TPut 7 185257.00 ( 0.00%) 183202.00 ( -1.11%)
TPut 13 329760.00 ( 0.00%) 346577.00 ( 5.10%)
TPut 19 442502.00 ( 0.00%) 460146.00 ( 3.99%)
TPut 25 540634.00 ( 0.00%) 549053.00 ( 1.56%)
TPut 31 512098.00 ( 0.00%) 519611.00 ( 1.47%)
TPut 37 461276.00 ( 0.00%) 474973.00 ( 2.97%)
TPut 43 403089.00 ( 0.00%) 414172.00 ( 2.75%)
3.12.0 3.12.0
vanillaacctupdates
User 5169.64 5184.14
System 100.45 80.02
Elapsed 252.75 251.85
Performance is similar but note the reduction in system CPU time. While
this showed a performance gain, it will not be universal but at least
it'll be behaving as documented. The vmstats are obviously different but
here is an obvious interpretation of them from mmtests.
3.12.0 3.12.0
vanillaacctupdates
NUMA page range updates 1408326 11043064
NUMA huge PMD updates 0 21040
NUMA PTE updates 1408326 291624
"NUMA page range updates" == nr_pte_updates and is the value returned to
the NUMA pte scanner. NUMA huge PMD updates were the number of THP updates
which in combination can be used to calculate how many ptes were updated
from userspace.
Reported-by: Alex Thorlton <athorlton@sgi.com>
Cc: stable@vger.kernel.org
Signed-off-by: Mel Gorman <mgorman@suse.de>
---
include/linux/vm_event_item.h | 1 +
mm/mprotect.c | 6 +++++-
mm/vmstat.c | 1 +
3 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
index 1855f0a..c557c6d 100644
--- a/include/linux/vm_event_item.h
+++ b/include/linux/vm_event_item.h
@@ -39,6 +39,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
PAGEOUTRUN, ALLOCSTALL, PGROTATED,
#ifdef CONFIG_NUMA_BALANCING
NUMA_PTE_UPDATES,
+ NUMA_HUGE_PTE_UPDATES,
NUMA_HINT_FAULTS,
NUMA_HINT_FAULTS_LOCAL,
NUMA_PAGE_MIGRATE,
diff --git a/mm/mprotect.c b/mm/mprotect.c
index 412ba2b..f94d2bd 100644
--- a/mm/mprotect.c
+++ b/mm/mprotect.c
@@ -138,6 +138,7 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma,
pmd_t *pmd;
unsigned long next;
unsigned long pages = 0;
+ unsigned long nr_huge_updates = 0;
bool all_same_node;
pmd = pmd_offset(pud, addr);
@@ -148,7 +149,8 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma,
split_huge_page_pmd(vma, addr, pmd);
else if (change_huge_pmd(vma, pmd, addr, newprot,
prot_numa)) {
- pages++;
+ pages += HPAGE_PMD_NR;
+ nr_huge_updates++;
continue;
}
/* fall through */
@@ -168,6 +170,8 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma,
change_pmd_protnuma(vma->vm_mm, addr, pmd);
} while (pmd++, addr = next, addr != end);
+ if (nr_huge_updates)
+ count_vm_numa_events(NUMA_HUGE_PTE_UPDATES, nr_huge_updates);
return pages;
}
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 9bb3145..5a442a7 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -812,6 +812,7 @@ const char * const vmstat_text[] = {
#ifdef CONFIG_NUMA_BALANCING
"numa_pte_updates",
+ "numa_huge_pte_updates",
"numa_hint_faults",
"numa_hint_faults_local",
"numa_pages_migrated",
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] mm: numa: Return the number of base pages altered by protection changes
2013-11-09 14:37 [PATCH] mm: numa: Return the number of base pages altered by protection changes Mel Gorman
@ 2013-11-11 0:22 ` Rik van Riel
0 siblings, 0 replies; 4+ messages in thread
From: Rik van Riel @ 2013-11-11 0:22 UTC (permalink / raw)
To: Mel Gorman; +Cc: Andrew Morton, Alex Thorlton, linux-kernel, linux-mm
On 11/09/2013 09:37 AM, Mel Gorman wrote:
> Commit 0255d491 (mm: Account for a THP NUMA hinting update as one PTE
> update) was added to account for the number of PTE updates when marking
> pages prot_numa. task_numa_work was using the old return value to track
> how much address space had been updated. Altering the return value causes
> the scanner to do more work than it is configured or documented to in a
> single unit of work.
>
> This patch reverts 0255d491 and accounts for the number of THP updates
> separately in vmstat. It is up to the administrator to interpret the pair
> of values correctly. This is a straight-forward operation and likely to
> only be of interest when actively debugging NUMA balancing problems.
>
> The impact of this patch is that the NUMA PTE scanner will scan slower when
> THP is enabled and workloads may converge slower as a result. On the flip
> size system CPU usage should be lower than recent tests reported. This is
> an illustrative example of a short single JVM specjbb test
Reviewed-by: Rik van Riel <riel@redhat.com>
--
All rights reversed
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH] mm: numa: return the number of base pages altered by protection changes
[not found] <1385922957129@kroah.com>
@ 2013-12-03 15:04 ` Mel Gorman
2013-12-04 16:51 ` Luis Henriques
0 siblings, 1 reply; 4+ messages in thread
From: Mel Gorman @ 2013-12-03 15:04 UTC (permalink / raw)
To: stable; +Cc: akpm, athorlton, riel, gregkh, torvalds, linux-kernel
commit 72403b4a0fbdf433c1fe0127e49864658f6f6468 upstream.
Commit 0255d4918480 ("mm: Account for a THP NUMA hinting update as one
PTE update") was added to account for the number of PTE updates when
marking pages prot_numa. task_numa_work was using the old return value
to track how much address space had been updated. Altering the return
value causes the scanner to do more work than it is configured or
documented to in a single unit of work.
This patch reverts that commit and accounts for the number of THP
updates separately in vmstat. It is up to the administrator to
interpret the pair of values correctly. This is a straight-forward
operation and likely to only be of interest when actively debugging NUMA
balancing problems.
The impact of this patch is that the NUMA PTE scanner will scan slower
when THP is enabled and workloads may converge slower as a result. On
the flip size system CPU usage should be lower than recent tests
reported. This is an illustrative example of a short single JVM specjbb
test
specjbb
3.12.0 3.12.0
vanilla acctupdates
TPut 1 26143.00 ( 0.00%) 25747.00 ( -1.51%)
TPut 7 185257.00 ( 0.00%) 183202.00 ( -1.11%)
TPut 13 329760.00 ( 0.00%) 346577.00 ( 5.10%)
TPut 19 442502.00 ( 0.00%) 460146.00 ( 3.99%)
TPut 25 540634.00 ( 0.00%) 549053.00 ( 1.56%)
TPut 31 512098.00 ( 0.00%) 519611.00 ( 1.47%)
TPut 37 461276.00 ( 0.00%) 474973.00 ( 2.97%)
TPut 43 403089.00 ( 0.00%) 414172.00 ( 2.75%)
3.12.0 3.12.0
vanillaacctupdates
User 5169.64 5184.14
System 100.45 80.02
Elapsed 252.75 251.85
Performance is similar but note the reduction in system CPU time. While
this showed a performance gain, it will not be universal but at least
it'll be behaving as documented. The vmstats are obviously different but
here is an obvious interpretation of them from mmtests.
3.12.0 3.12.0
vanillaacctupdates
NUMA page range updates 1408326 11043064
NUMA huge PMD updates 0 21040
NUMA PTE updates 1408326 291624
"NUMA page range updates" == nr_pte_updates and is the value returned to
the NUMA pte scanner. NUMA huge PMD updates were the number of THP
updates which in combination can be used to calculate how many ptes were
updated from userspace.
Signed-off-by: Mel Gorman <mgorman@suse.de>
Reported-by: Alex Thorlton <athorlton@sgi.com>
Reviewed-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Mel Gorman <mgorman@suse.de>
---
include/linux/vm_event_item.h | 1 +
mm/mprotect.c | 7 ++++++-
mm/vmstat.c | 1 +
3 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
index 1855f0a..c557c6d 100644
--- a/include/linux/vm_event_item.h
+++ b/include/linux/vm_event_item.h
@@ -39,6 +39,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
PAGEOUTRUN, ALLOCSTALL, PGROTATED,
#ifdef CONFIG_NUMA_BALANCING
NUMA_PTE_UPDATES,
+ NUMA_HUGE_PTE_UPDATES,
NUMA_HINT_FAULTS,
NUMA_HINT_FAULTS_LOCAL,
NUMA_PAGE_MIGRATE,
diff --git a/mm/mprotect.c b/mm/mprotect.c
index 412ba2b..6c3f56f 100644
--- a/mm/mprotect.c
+++ b/mm/mprotect.c
@@ -138,6 +138,7 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma,
pmd_t *pmd;
unsigned long next;
unsigned long pages = 0;
+ unsigned long nr_huge_updates = 0;
bool all_same_node;
pmd = pmd_offset(pud, addr);
@@ -148,7 +149,8 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma,
split_huge_page_pmd(vma, addr, pmd);
else if (change_huge_pmd(vma, pmd, addr, newprot,
prot_numa)) {
- pages++;
+ pages += HPAGE_PMD_NR;
+ nr_huge_updates++;
continue;
}
/* fall through */
@@ -168,6 +170,9 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma,
change_pmd_protnuma(vma->vm_mm, addr, pmd);
} while (pmd++, addr = next, addr != end);
+ if (nr_huge_updates)
+ count_vm_numa_events(NUMA_HUGE_PTE_UPDATES, nr_huge_updates);
+
return pages;
}
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 9bb3145..5a442a7 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -812,6 +812,7 @@ const char * const vmstat_text[] = {
#ifdef CONFIG_NUMA_BALANCING
"numa_pte_updates",
+ "numa_huge_pte_updates",
"numa_hint_faults",
"numa_hint_faults_local",
"numa_pages_migrated",
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] mm: numa: return the number of base pages altered by protection changes
2013-12-03 15:04 ` [PATCH] mm: numa: return " Mel Gorman
@ 2013-12-04 16:51 ` Luis Henriques
0 siblings, 0 replies; 4+ messages in thread
From: Luis Henriques @ 2013-12-04 16:51 UTC (permalink / raw)
To: Mel Gorman; +Cc: stable, akpm, athorlton, riel, gregkh, torvalds, linux-kernel
On Tue, Dec 03, 2013 at 03:04:00PM +0000, Mel Gorman wrote:
> commit 72403b4a0fbdf433c1fe0127e49864658f6f6468 upstream.
Thank you Mel, I'll queue this backport for the 3.11 kernel.
Cheers,
--
Luis
>
> Commit 0255d4918480 ("mm: Account for a THP NUMA hinting update as
> one PTE update") was added to account for the number of PTE updates
> when marking pages prot_numa. task_numa_work was using the old
> return value to track how much address space had been updated.
> Altering the return value causes the scanner to do more work than it
> is configured or documented to in a single unit of work.
>
> This patch reverts that commit and accounts for the number of THP
> updates separately in vmstat. It is up to the administrator to
> interpret the pair of values correctly. This is a straight-forward
> operation and likely to only be of interest when actively debugging NUMA
> balancing problems.
>
> The impact of this patch is that the NUMA PTE scanner will scan slower
> when THP is enabled and workloads may converge slower as a result. On
> the flip size system CPU usage should be lower than recent tests
> reported. This is an illustrative example of a short single JVM specjbb
> test
>
> specjbb
> 3.12.0 3.12.0
> vanilla acctupdates
> TPut 1 26143.00 ( 0.00%) 25747.00 ( -1.51%)
> TPut 7 185257.00 ( 0.00%) 183202.00 ( -1.11%)
> TPut 13 329760.00 ( 0.00%) 346577.00 ( 5.10%)
> TPut 19 442502.00 ( 0.00%) 460146.00 ( 3.99%)
> TPut 25 540634.00 ( 0.00%) 549053.00 ( 1.56%)
> TPut 31 512098.00 ( 0.00%) 519611.00 ( 1.47%)
> TPut 37 461276.00 ( 0.00%) 474973.00 ( 2.97%)
> TPut 43 403089.00 ( 0.00%) 414172.00 ( 2.75%)
>
> 3.12.0 3.12.0
> vanillaacctupdates
> User 5169.64 5184.14
> System 100.45 80.02
> Elapsed 252.75 251.85
>
> Performance is similar but note the reduction in system CPU time. While
> this showed a performance gain, it will not be universal but at least
> it'll be behaving as documented. The vmstats are obviously different but
> here is an obvious interpretation of them from mmtests.
>
> 3.12.0 3.12.0
> vanillaacctupdates
> NUMA page range updates 1408326 11043064
> NUMA huge PMD updates 0 21040
> NUMA PTE updates 1408326 291624
>
> "NUMA page range updates" == nr_pte_updates and is the value returned to
> the NUMA pte scanner. NUMA huge PMD updates were the number of THP
> updates which in combination can be used to calculate how many ptes were
> updated from userspace.
>
> Signed-off-by: Mel Gorman <mgorman@suse.de>
> Reported-by: Alex Thorlton <athorlton@sgi.com>
> Reviewed-by: Rik van Riel <riel@redhat.com>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
> Signed-off-by: Mel Gorman <mgorman@suse.de>
> ---
> include/linux/vm_event_item.h | 1 +
> mm/mprotect.c | 7 ++++++-
> mm/vmstat.c | 1 +
> 3 files changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
> index 1855f0a..c557c6d 100644
> --- a/include/linux/vm_event_item.h
> +++ b/include/linux/vm_event_item.h
> @@ -39,6 +39,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
> PAGEOUTRUN, ALLOCSTALL, PGROTATED,
> #ifdef CONFIG_NUMA_BALANCING
> NUMA_PTE_UPDATES,
> + NUMA_HUGE_PTE_UPDATES,
> NUMA_HINT_FAULTS,
> NUMA_HINT_FAULTS_LOCAL,
> NUMA_PAGE_MIGRATE,
> diff --git a/mm/mprotect.c b/mm/mprotect.c
> index 412ba2b..6c3f56f 100644
> --- a/mm/mprotect.c
> +++ b/mm/mprotect.c
> @@ -138,6 +138,7 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma,
> pmd_t *pmd;
> unsigned long next;
> unsigned long pages = 0;
> + unsigned long nr_huge_updates = 0;
> bool all_same_node;
>
> pmd = pmd_offset(pud, addr);
> @@ -148,7 +149,8 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma,
> split_huge_page_pmd(vma, addr, pmd);
> else if (change_huge_pmd(vma, pmd, addr, newprot,
> prot_numa)) {
> - pages++;
> + pages += HPAGE_PMD_NR;
> + nr_huge_updates++;
> continue;
> }
> /* fall through */
> @@ -168,6 +170,9 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma,
> change_pmd_protnuma(vma->vm_mm, addr, pmd);
> } while (pmd++, addr = next, addr != end);
>
> + if (nr_huge_updates)
> + count_vm_numa_events(NUMA_HUGE_PTE_UPDATES, nr_huge_updates);
> +
> return pages;
> }
>
> diff --git a/mm/vmstat.c b/mm/vmstat.c
> index 9bb3145..5a442a7 100644
> --- a/mm/vmstat.c
> +++ b/mm/vmstat.c
> @@ -812,6 +812,7 @@ const char * const vmstat_text[] = {
>
> #ifdef CONFIG_NUMA_BALANCING
> "numa_pte_updates",
> + "numa_huge_pte_updates",
> "numa_hint_faults",
> "numa_hint_faults_local",
> "numa_pages_migrated",
> --
> To unsubscribe from this list: send the line "unsubscribe stable" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2013-12-04 16:52 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-11-09 14:37 [PATCH] mm: numa: Return the number of base pages altered by protection changes Mel Gorman
2013-11-11 0:22 ` Rik van Riel
[not found] <1385922957129@kroah.com>
2013-12-03 15:04 ` [PATCH] mm: numa: return " Mel Gorman
2013-12-04 16:51 ` Luis Henriques
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).