[PATCH 0/3] Suppress page allocator warnings about order >= MAX_ORDER (resend)

[PATCH 0/3] Suppress page allocator warnings about order >= MAX_ORDER (resend)

[Mel Gorman]

This is a resend of the series to suppress warnings when the order is too
high but the caller knows how to handle it. The possible problematic patch
in this series is patch 3 altering dccp. A better fix for that problem would
be to use alloc_large_system_hash() and replace the code in question. A
patch to do that could be buried in a git tree somewhere and the people
cc'd should know if it exists and where.

The page allocator warns once when callers specify an order that is too
high. This is because the path is slow and it's important to verify that
callers are really doing the right thing and recovering by specifying
smaller orders rather than simply falling back to vmalloc().

The problem is that there is no way of suppressing the warning when the
callers are doing the right thing. Patch 1 of this series allows the warning
to be suppressed with __GFP_NOWARN. The second two patches suppress warnings
generated by the profile= and the DCCP network protocol as those callers
are recovering in a sensible fashion.

 kernel/profile.c |    5 +++--
 mm/page_alloc.c  |    4 +++-
 net/dccp/proto.c |    4 ++--
 3 files changed, 8 insertions(+), 5 deletions(-)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 1/3] page-allocator: Allow too high-order warning messages to be suppressed with __GFP_NOWARN

[Mel Gorman]

The page allocator warns once when an order >= MAX_ORDER is specified.
This is to catch callers of the allocator that are always falling back
to their worst-case when it was not expected. However, there are cases
where the caller is behaving correctly but cannot suppress the warning.
This patch allows the warning to be suppressed by the callers by
specifying __GFP_NOWARN.

Signed-off-by: Mel Gorman <mel@csn.ul.ie>
---
 mm/page_alloc.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index caa9268..b469a05 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1740,8 +1740,10 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	 * be using allocators in order of preference for an area that is
 	 * too large.
 	 */
-	if (WARN_ON_ONCE(order >= MAX_ORDER))
+	if (order >= MAX_ORDER) {
+		WARN_ON_ONCE(!(gfp_mask & __GFP_NOWARN));
 		return NULL;
+	}
 
 	/*
 	 * GFP_THISNODE (meaning __GFP_THISNODE, __GFP_NORETRY and
-- 
1.5.6.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/3] page-allocator: Allow too high-order warning messages to be suppressed with __GFP_NOWARN

[David Rientjes]

On Wed, 15 Jul 2009, Mel Gorman wrote:

> The page allocator warns once when an order >= MAX_ORDER is specified.
> This is to catch callers of the allocator that are always falling back
> to their worst-case when it was not expected. However, there are cases
> where the caller is behaving correctly but cannot suppress the warning.
> This patch allows the warning to be suppressed by the callers by
> specifying __GFP_NOWARN.
> 
> Signed-off-by: Mel Gorman <mel@csn.ul.ie>

Acked-by: David Rientjes <rientjes@google.com>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/3] page-allocator: Allow too high-order warning messages to be suppressed with __GFP_NOWARN

[KOSAKI Motohiro]

> The page allocator warns once when an order >= MAX_ORDER is specified.
> This is to catch callers of the allocator that are always falling back
> to their worst-case when it was not expected. However, there are cases
> where the caller is behaving correctly but cannot suppress the warning.
> This patch allows the warning to be suppressed by the callers by
> specifying __GFP_NOWARN.
> 
> Signed-off-by: Mel Gorman <mel@csn.ul.ie>
> ---
>  mm/page_alloc.c |    4 +++-
>  1 files changed, 3 insertions(+), 1 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index caa9268..b469a05 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -1740,8 +1740,10 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	 * be using allocators in order of preference for an area that is
>  	 * too large.
>  	 */
> -	if (WARN_ON_ONCE(order >= MAX_ORDER))
> +	if (order >= MAX_ORDER) {
> +		WARN_ON_ONCE(!(gfp_mask & __GFP_NOWARN));
>  		return NULL;
> +	}

Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 2/3] profile: Suppress warning about large allocations when profile=1 is specified

[Mel Gorman]

When profile= is used, a large buffer is allocated early at boot. This
can be larger than what the page allocator can provide so it prints a
warning. However, the caller is able to handle the situation so this patch
suppresses the warning.

Signed-off-by: Mel Gorman <mel@csn.ul.ie>
---
 kernel/profile.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/kernel/profile.c b/kernel/profile.c
index 69911b5..419250e 100644
--- a/kernel/profile.c
+++ b/kernel/profile.c
@@ -117,11 +117,12 @@ int __ref profile_init(void)
 
 	cpumask_copy(prof_cpu_mask, cpu_possible_mask);
 
-	prof_buffer = kzalloc(buffer_bytes, GFP_KERNEL);
+	prof_buffer = kzalloc(buffer_bytes, GFP_KERNEL|__GFP_NOWARN);
 	if (prof_buffer)
 		return 0;
 
-	prof_buffer = alloc_pages_exact(buffer_bytes, GFP_KERNEL|__GFP_ZERO);
+	prof_buffer = alloc_pages_exact(buffer_bytes,
+					GFP_KERNEL|__GFP_ZERO|__GFP_NOWARN);
 	if (prof_buffer)
 		return 0;
 
-- 
1.5.6.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 2/3] profile: Suppress warning about large allocations when profile=1 is specified

[KOSAKI Motohiro]

> When profile= is used, a large buffer is allocated early at boot. This
> can be larger than what the page allocator can provide so it prints a
> warning. However, the caller is able to handle the situation so this patch
> suppresses the warning.

I'm confused.

Currently caller doesn't handle error return.

----------------------------------------------------------
asmlinkage void __init start_kernel(void)
{
(snip)
        init_timers();
        hrtimers_init();
        softirq_init();
        timekeeping_init();
        time_init();
        sched_clock_init();
        profile_init();           <-- ignore return value
------------------------------------------------------------

and, if user want to use linus profiler, the user should choice select
proper bucket size by boot parameter.
Currently, allocation failure message tell user about specified bucket size
is wrong.
I think this patch hide it.


> 
> Signed-off-by: Mel Gorman <mel@csn.ul.ie>
> ---
>  kernel/profile.c |    5 +++--
>  1 files changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/profile.c b/kernel/profile.c
> index 69911b5..419250e 100644
> --- a/kernel/profile.c
> +++ b/kernel/profile.c
> @@ -117,11 +117,12 @@ int __ref profile_init(void)
>  
>  	cpumask_copy(prof_cpu_mask, cpu_possible_mask);
>  
> -	prof_buffer = kzalloc(buffer_bytes, GFP_KERNEL);
> +	prof_buffer = kzalloc(buffer_bytes, GFP_KERNEL|__GFP_NOWARN);
>  	if (prof_buffer)
>  		return 0;
>  
> -	prof_buffer = alloc_pages_exact(buffer_bytes, GFP_KERNEL|__GFP_ZERO);
> +	prof_buffer = alloc_pages_exact(buffer_bytes,
> +					GFP_KERNEL|__GFP_ZERO|__GFP_NOWARN);
>  	if (prof_buffer)
>  		return 0;



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 2/3] profile: Suppress warning about large allocations when profile=1 is specified

[Mel Gorman]

On Thu, Jul 16, 2009 at 10:12:20AM +0900, KOSAKI Motohiro wrote:
> > When profile= is used, a large buffer is allocated early at boot. This
> > can be larger than what the page allocator can provide so it prints a
> > warning. However, the caller is able to handle the situation so this patch
> > suppresses the warning.
> 
> I'm confused.
> 
> Currently caller doesn't handle error return.
> 
> ----------------------------------------------------------
> asmlinkage void __init start_kernel(void)
> {
> (snip)
>         init_timers();
>         hrtimers_init();
>         softirq_init();
>         timekeeping_init();
>         time_init();
>         sched_clock_init();
>         profile_init();           <-- ignore return value
> ------------------------------------------------------------
> 
> and, if user want to use linus profiler, the user should choice select
> proper bucket size by boot parameter.
> Currently, allocation failure message tell user about specified bucket size
> is wrong.
> I think this patch hide it.
> 

Look at what profile_init() itself is doing. You can't see it from the
patch context but when alloc_pages_exact() fails, it calls vmalloc(). If
that fails, profiling is just disabled. There isn't really anything the
caller of profile_init() can do about it and the page allocator doesn't
need to scream about it.

> 
> > 
> > Signed-off-by: Mel Gorman <mel@csn.ul.ie>
> > ---
> >  kernel/profile.c |    5 +++--
> >  1 files changed, 3 insertions(+), 2 deletions(-)
> > 
> > diff --git a/kernel/profile.c b/kernel/profile.c
> > index 69911b5..419250e 100644
> > --- a/kernel/profile.c
> > +++ b/kernel/profile.c
> > @@ -117,11 +117,12 @@ int __ref profile_init(void)
> >  
> >  	cpumask_copy(prof_cpu_mask, cpu_possible_mask);
> >  
> > -	prof_buffer = kzalloc(buffer_bytes, GFP_KERNEL);
> > +	prof_buffer = kzalloc(buffer_bytes, GFP_KERNEL|__GFP_NOWARN);
> >  	if (prof_buffer)
> >  		return 0;
> >  
> > -	prof_buffer = alloc_pages_exact(buffer_bytes, GFP_KERNEL|__GFP_ZERO);
> > +	prof_buffer = alloc_pages_exact(buffer_bytes,
> > +					GFP_KERNEL|__GFP_ZERO|__GFP_NOWARN);
> >  	if (prof_buffer)
> >  		return 0;
> 
> 
> 

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 2/3] profile: Suppress warning about large allocations when profile=1 is specified

[KOSAKI Motohiro]

> On Thu, Jul 16, 2009 at 10:12:20AM +0900, KOSAKI Motohiro wrote:
> > > When profile= is used, a large buffer is allocated early at boot. This
> > > can be larger than what the page allocator can provide so it prints a
> > > warning. However, the caller is able to handle the situation so this patch
> > > suppresses the warning.
> > 
> > I'm confused.
> > 
> > Currently caller doesn't handle error return.
> > 
> > ----------------------------------------------------------
> > asmlinkage void __init start_kernel(void)
> > {
> > (snip)
> >         init_timers();
> >         hrtimers_init();
> >         softirq_init();
> >         timekeeping_init();
> >         time_init();
> >         sched_clock_init();
> >         profile_init();           <-- ignore return value
> > ------------------------------------------------------------
> > 
> > and, if user want to use linus profiler, the user should choice select
> > proper bucket size by boot parameter.
> > Currently, allocation failure message tell user about specified bucket size
> > is wrong.
> > I think this patch hide it.
> > 
> 
> Look at what profile_init() itself is doing. You can't see it from the
> patch context but when alloc_pages_exact() fails, it calls vmalloc(). If
> that fails, profiling is just disabled. There isn't really anything the
> caller of profile_init() can do about it and the page allocator doesn't
> need to scream about it.

Indeed. Thanks correct me.
	Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>

> 
> > 
> > > 
> > > Signed-off-by: Mel Gorman <mel@csn.ul.ie>
> > > ---
> > >  kernel/profile.c |    5 +++--
> > >  1 files changed, 3 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/kernel/profile.c b/kernel/profile.c
> > > index 69911b5..419250e 100644
> > > --- a/kernel/profile.c
> > > +++ b/kernel/profile.c
> > > @@ -117,11 +117,12 @@ int __ref profile_init(void)
> > >  
> > >  	cpumask_copy(prof_cpu_mask, cpu_possible_mask);
> > >  
> > > -	prof_buffer = kzalloc(buffer_bytes, GFP_KERNEL);
> > > +	prof_buffer = kzalloc(buffer_bytes, GFP_KERNEL|__GFP_NOWARN);
> > >  	if (prof_buffer)
> > >  		return 0;
> > >  
> > > -	prof_buffer = alloc_pages_exact(buffer_bytes, GFP_KERNEL|__GFP_ZERO);
> > > +	prof_buffer = alloc_pages_exact(buffer_bytes,
> > > +					GFP_KERNEL|__GFP_ZERO|__GFP_NOWARN);
> > >  	if (prof_buffer)
> > >  		return 0;
> > 
> > 
> > 
> 
> -- 
> Mel Gorman
> Part-time Phd Student                          Linux Technology Center
> University of Limerick                         IBM Dublin Software Lab



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[tip:tracing/urgent] profile: Suppress warning about large allocations when profile=1 is specified

[tip-bot for Mel Gorman]

Commit-ID:  e5d490b252423605a77c54b2e35b10ea663763df
Gitweb:     http://git.kernel.org/tip/e5d490b252423605a77c54b2e35b10ea663763df
Author:     Mel Gorman <mel@csn.ul.ie>
AuthorDate: Wed, 15 Jul 2009 12:23:11 +0100
Committer:  Ingo Molnar <mingo@elte.hu>
CommitDate: Sat, 18 Jul 2009 12:55:28 +0200

profile: Suppress warning about large allocations when profile=1 is specified

When profile= is used, a large buffer is allocated early at
boot. This can be larger than what the page allocator can
provide so it prints a warning. However, the caller is able to
handle the situation so this patch suppresses the warning.

Signed-off-by: Mel Gorman <mel@csn.ul.ie>
Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Linux Memory Management List <linux-mm@kvack.org>
Cc: Heinz Diehl <htd@fancy-poultry.org>
Cc: David Miller <davem@davemloft.net>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: Andrew Morton <akpm@linux-foundation.org>
LKML-Reference: <1247656992-19846-3-git-send-email-mel@csn.ul.ie>
Signed-off-by: Ingo Molnar <mingo@elte.hu>


---
 kernel/profile.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/kernel/profile.c b/kernel/profile.c
index 69911b5..419250e 100644
--- a/kernel/profile.c
+++ b/kernel/profile.c
@@ -117,11 +117,12 @@ int __ref profile_init(void)
 
 	cpumask_copy(prof_cpu_mask, cpu_possible_mask);
 
-	prof_buffer = kzalloc(buffer_bytes, GFP_KERNEL);
+	prof_buffer = kzalloc(buffer_bytes, GFP_KERNEL|__GFP_NOWARN);
 	if (prof_buffer)
 		return 0;
 
-	prof_buffer = alloc_pages_exact(buffer_bytes, GFP_KERNEL|__GFP_ZERO);
+	prof_buffer = alloc_pages_exact(buffer_bytes,
+					GFP_KERNEL|__GFP_ZERO|__GFP_NOWARN);
 	if (prof_buffer)
 		return 0;
 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 3/3] net-dccp: Suppress warning about large allocations from DCCP

[Mel Gorman]

The DCCP protocol tries to allocate some large hash tables during
initialisation using the largest size possible.  This can be larger than
what the page allocator can provide so it prints a warning. However, the
caller is able to handle the situation so this patch suppresses the warning.

Signed-off-by: Mel Gorman <mel@csn.ul.ie>
---
 net/dccp/proto.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/net/dccp/proto.c b/net/dccp/proto.c
index 94ca8ea..3281013 100644
--- a/net/dccp/proto.c
+++ b/net/dccp/proto.c
@@ -1066,7 +1066,7 @@ static int __init dccp_init(void)
 		       (dccp_hashinfo.ehash_size - 1))
 			dccp_hashinfo.ehash_size--;
 		dccp_hashinfo.ehash = (struct inet_ehash_bucket *)
-			__get_free_pages(GFP_ATOMIC, ehash_order);
+			__get_free_pages(GFP_ATOMIC|__GFP_NOWARN, ehash_order);
 	} while (!dccp_hashinfo.ehash && --ehash_order > 0);
 
 	if (!dccp_hashinfo.ehash) {
@@ -1091,7 +1091,7 @@ static int __init dccp_init(void)
 		    bhash_order > 0)
 			continue;
 		dccp_hashinfo.bhash = (struct inet_bind_hashbucket *)
-			__get_free_pages(GFP_ATOMIC, bhash_order);
+			__get_free_pages(GFP_ATOMIC|__GFP_NOWARN, bhash_order);
 	} while (!dccp_hashinfo.bhash && --bhash_order >= 0);
 
 	if (!dccp_hashinfo.bhash) {
-- 
1.5.6.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 3/3] net-dccp: Suppress warning about large allocations from DCCP

[Arnaldo Carvalho de Melo]

Em Wed, Jul 15, 2009 at 12:23:12PM +0100, Mel Gorman escreveu:
> The DCCP protocol tries to allocate some large hash tables during
> initialisation using the largest size possible.  This can be larger than
> what the page allocator can provide so it prints a warning. However, the
> caller is able to handle the situation so this patch suppresses the warning.
> 
> Signed-off-by: Mel Gorman <mel@csn.ul.ie>

Thanks again,

Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>