[PATCH 0/3] Suppress page allocator warnings about order >= MAX_ORDER

[PATCH 0/3] Suppress page allocator warnings about order >= MAX_ORDER

[Mel Gorman]

The page allocator warns once when callers specify an order that is too
high. This is because the path is slow and it's important to verify that
callers are really doing the right thing and recovering by specifying
smaller orders rather than simply falling back to vmalloc().

The problem is that there is no way of suppressing the warning when the
callers are doing the right thing. Patch 1 of this series allows the warning
to be suppressed with __GFP_NOWARN. The second two patches suppress warnings
generated by the profile= and the DCCP network protocol as those callers
are recovering in a sensible fashion.

 kernel/profile.c |    5 +++--
 mm/page_alloc.c  |    4 +++-
 net/dccp/proto.c |    4 ++--
 3 files changed, 8 insertions(+), 5 deletions(-)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 1/3] page-allocator: Allow too high-order warning messages to be suppressed with __GFP_NOWARN

[Mel Gorman]

The page allocator warns once when an order >= MAX_ORDER is specified.
This is to catch callers of the allocator that are always falling back
to their worst-case when it was not expected. However, there are cases
where the caller is behaving correctly but cannot suppress the warning.
This patch allows the warning to be suppressed by the callers by
specifying __GFP_NOWARN.

Signed-off-by: Mel Gorman <mel@csn.ul.ie>
---
 mm/page_alloc.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a5f3c27..005b32d 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1740,8 +1740,10 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	 * be using allocators in order of preference for an area that is
 	 * too large.
 	 */
-	if (WARN_ON_ONCE(order >= MAX_ORDER))
+	if (order >= MAX_ORDER) {
+		WARN_ON_ONCE(!(gfp_mask & __GFP_NOWARN));
 		return NULL;
+	}
 
 	/*
 	 * GFP_THISNODE (meaning __GFP_THISNODE, __GFP_NORETRY and
-- 
1.5.6.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 2/3] profile: Suppress warning about large allocations when profile=1 is specified

[Mel Gorman]

When profile= is used, a large buffer is allocated early at boot. This
can be larger than what the page allocator can provide so it prints a
warning. However, the caller is able to handle the situation so this patch
suppresses the warning.

Signed-off-by: Mel Gorman <mel@csn.ul.ie>
---
 kernel/profile.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/kernel/profile.c b/kernel/profile.c
index 69911b5..419250e 100644
--- a/kernel/profile.c
+++ b/kernel/profile.c
@@ -117,11 +117,12 @@ int __ref profile_init(void)
 
 	cpumask_copy(prof_cpu_mask, cpu_possible_mask);
 
-	prof_buffer = kzalloc(buffer_bytes, GFP_KERNEL);
+	prof_buffer = kzalloc(buffer_bytes, GFP_KERNEL|__GFP_NOWARN);
 	if (prof_buffer)
 		return 0;
 
-	prof_buffer = alloc_pages_exact(buffer_bytes, GFP_KERNEL|__GFP_ZERO);
+	prof_buffer = alloc_pages_exact(buffer_bytes,
+					GFP_KERNEL|__GFP_ZERO|__GFP_NOWARN);
 	if (prof_buffer)
 		return 0;
 
-- 
1.5.6.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 3/3] net-dccp: Suppress warning about large allocations from DCCP

[Mel Gorman]

The DCCP protocol tries to allocate some large hash tables during
initialisation using the largest size possible.  This can be larger than
what the page allocator can provide so it prints a warning. However, the
caller is able to handle the situation so this patch suppresses the warning.

Signed-off-by: Mel Gorman <mel@csn.ul.ie>
---
 net/dccp/proto.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/net/dccp/proto.c b/net/dccp/proto.c
index 314a1b5..fd21676 100644
--- a/net/dccp/proto.c
+++ b/net/dccp/proto.c
@@ -1066,7 +1066,7 @@ static int __init dccp_init(void)
 		       (dccp_hashinfo.ehash_size - 1))
 			dccp_hashinfo.ehash_size--;
 		dccp_hashinfo.ehash = (struct inet_ehash_bucket *)
-			__get_free_pages(GFP_ATOMIC, ehash_order);
+			__get_free_pages(GFP_ATOMIC|__GFP_NOWARN, ehash_order);
 	} while (!dccp_hashinfo.ehash && --ehash_order > 0);
 
 	if (!dccp_hashinfo.ehash) {
@@ -1091,7 +1091,7 @@ static int __init dccp_init(void)
 		    bhash_order > 0)
 			continue;
 		dccp_hashinfo.bhash = (struct inet_bind_hashbucket *)
-			__get_free_pages(GFP_ATOMIC, bhash_order);
+			__get_free_pages(GFP_ATOMIC|__GFP_NOWARN, bhash_order);
 	} while (!dccp_hashinfo.bhash && --bhash_order >= 0);
 
 	if (!dccp_hashinfo.bhash) {
-- 
1.5.6.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 3/3] net-dccp: Suppress warning about large allocations from DCCP

[David Miller]

From: Mel Gorman <mel@csn.ul.ie>
Date: Mon, 22 Jun 2009 16:43:34 +0100

> The DCCP protocol tries to allocate some large hash tables during
> initialisation using the largest size possible.  This can be larger than
> what the page allocator can provide so it prints a warning. However, the
> caller is able to handle the situation so this patch suppresses the warning.
> 
> Signed-off-by: Mel Gorman <mel@csn.ul.ie>

It's probably much more appropriate to make this stuff use
alloc_large_system_hash(), like TCP does (see net/ipv4/tcp.c
tcp_init()).

All of this complicated DCCP hash table size computation code will
simply disappear.  And it'll fix the warning too :-)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 3/3] net-dccp: Suppress warning about large allocations from DCCP

[Arnaldo Carvalho de Melo]

Em Mon, Jun 22, 2009 at 04:15:02PM -0700, David Miller escreveu:
> From: Mel Gorman <mel@csn.ul.ie>
> Date: Mon, 22 Jun 2009 16:43:34 +0100
> 
> > The DCCP protocol tries to allocate some large hash tables during
> > initialisation using the largest size possible.  This can be larger than
> > what the page allocator can provide so it prints a warning. However, the
> > caller is able to handle the situation so this patch suppresses the warning.
> > 
> > Signed-off-by: Mel Gorman <mel@csn.ul.ie>
> 
> It's probably much more appropriate to make this stuff use
> alloc_large_system_hash(), like TCP does (see net/ipv4/tcp.c
> tcp_init()).
> 
> All of this complicated DCCP hash table size computation code will
> simply disappear.  And it'll fix the warning too :-)

He mentioned that in the conversation that lead to this new patch
series, problem is that alloc_large_system_hash is __init, so when you
try to load dccp.ko it will not be available.

- Arnaldo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 3/3] net-dccp: Suppress warning about large allocations from DCCP

[David Miller]

From: Arnaldo Carvalho de Melo <acme@redhat.com>
Date: Mon, 22 Jun 2009 23:39:36 -0300

> Em Mon, Jun 22, 2009 at 04:15:02PM -0700, David Miller escreveu:
>> It's probably much more appropriate to make this stuff use
>> alloc_large_system_hash(), like TCP does (see net/ipv4/tcp.c
>> tcp_init()).
>> 
>> All of this complicated DCCP hash table size computation code will
>> simply disappear.  And it'll fix the warning too :-)
> 
> He mentioned that in the conversation that lead to this new patch
> series, problem is that alloc_large_system_hash is __init, so when you
> try to load dccp.ko it will not be available.

Fair enough.

It's such an unfortunate duplication of code, it's likely therefore
better to remove the __init tag and export that symbol.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 3/3] net-dccp: Suppress warning about large allocations from DCCP

[Eric Dumazet]

David Miller a ecrit :
> From: Arnaldo Carvalho de Melo <acme@redhat.com>
> Date: Mon, 22 Jun 2009 23:39:36 -0300
> 
>> Em Mon, Jun 22, 2009 at 04:15:02PM -0700, David Miller escreveu:
>>> It's probably much more appropriate to make this stuff use
>>> alloc_large_system_hash(), like TCP does (see net/ipv4/tcp.c
>>> tcp_init()).
>>>
>>> All of this complicated DCCP hash table size computation code will
>>> simply disappear.  And it'll fix the warning too :-)
>> He mentioned that in the conversation that lead to this new patch
>> series, problem is that alloc_large_system_hash is __init, so when you
>> try to load dccp.ko it will not be available.
> 
> Fair enough.
> 
> It's such an unfortunate duplication of code, it's likely therefore
> better to remove the __init tag and export that symbol.

Agreed, I once considered using this function for futex hash table allocation
and just forgot about it...

But it has some bootmem references, it might need more work than just exporting it.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 3/3] net-dccp: Suppress warning about large allocations from DCCP

[David Miller]

From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 23 Jun 2009 13:03:54 +0200

> But it has some bootmem references, it might need more work than
> just exporting it.

In that case we should probably just apply the original patch
for now, and leave this cleanup as a future change.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 3/3] net-dccp: Suppress warning about large allocations from DCCP

[Arnaldo Carvalho de Melo]

Em Tue, Jun 23, 2009 at 04:05:51AM -0700, David Miller escreveu:
> From: Eric Dumazet <eric.dumazet@gmail.com>
> Date: Tue, 23 Jun 2009 13:03:54 +0200
> 
> > But it has some bootmem references, it might need more work than
> > just exporting it.
> 
> In that case we should probably just apply the original patch
> for now, and leave this cleanup as a future change.

Full circle! That was my suggestion as well :-)

- Arnaldo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 3/3] net-dccp: Suppress warning about large allocations from DCCP

[Mel Gorman]

On Mon, Jun 22, 2009 at 04:15:02PM -0700, David Miller wrote:
> From: Mel Gorman <mel@csn.ul.ie>
> Date: Mon, 22 Jun 2009 16:43:34 +0100
> 
> > The DCCP protocol tries to allocate some large hash tables during
> > initialisation using the largest size possible.  This can be larger than
> > what the page allocator can provide so it prints a warning. However, the
> > caller is able to handle the situation so this patch suppresses the warning.
> > 
> > Signed-off-by: Mel Gorman <mel@csn.ul.ie>
> 
> It's probably much more appropriate to make this stuff use
> alloc_large_system_hash(), like TCP does (see net/ipv4/tcp.c
> tcp_init()).
> 

I agree. In another mail I asked why it wasn't used. I guessed it might be
because of the __init tag but nothing stops that being deleted. It should
not take significant effort to make it usable by DCCP.

> All of this complicated DCCP hash table size computation code will
> simply disappear.  And it'll fix the warning too :-)
>  

It would be my preferred option :)

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 0/3] Suppress page allocator warnings about order >= MAX_ORDER (resend)

[Mel Gorman]

This is a resend of the series to suppress warnings when the order is too
high but the caller knows how to handle it. The possible problematic patch
in this series is patch 3 altering dccp. A better fix for that problem would
be to use alloc_large_system_hash() and replace the code in question. A
patch to do that could be buried in a git tree somewhere and the people
cc'd should know if it exists and where.

The page allocator warns once when callers specify an order that is too
high. This is because the path is slow and it's important to verify that
callers are really doing the right thing and recovering by specifying
smaller orders rather than simply falling back to vmalloc().

The problem is that there is no way of suppressing the warning when the
callers are doing the right thing. Patch 1 of this series allows the warning
to be suppressed with __GFP_NOWARN. The second two patches suppress warnings
generated by the profile= and the DCCP network protocol as those callers
are recovering in a sensible fashion.

 kernel/profile.c |    5 +++--
 mm/page_alloc.c  |    4 +++-
 net/dccp/proto.c |    4 ++--
 3 files changed, 8 insertions(+), 5 deletions(-)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 3/3] net-dccp: Suppress warning about large allocations from DCCP

[Mel Gorman]

The DCCP protocol tries to allocate some large hash tables during
initialisation using the largest size possible.  This can be larger than
what the page allocator can provide so it prints a warning. However, the
caller is able to handle the situation so this patch suppresses the warning.

Signed-off-by: Mel Gorman <mel@csn.ul.ie>
---
 net/dccp/proto.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/net/dccp/proto.c b/net/dccp/proto.c
index 94ca8ea..3281013 100644
--- a/net/dccp/proto.c
+++ b/net/dccp/proto.c
@@ -1066,7 +1066,7 @@ static int __init dccp_init(void)
 		       (dccp_hashinfo.ehash_size - 1))
 			dccp_hashinfo.ehash_size--;
 		dccp_hashinfo.ehash = (struct inet_ehash_bucket *)
-			__get_free_pages(GFP_ATOMIC, ehash_order);
+			__get_free_pages(GFP_ATOMIC|__GFP_NOWARN, ehash_order);
 	} while (!dccp_hashinfo.ehash && --ehash_order > 0);
 
 	if (!dccp_hashinfo.ehash) {
@@ -1091,7 +1091,7 @@ static int __init dccp_init(void)
 		    bhash_order > 0)
 			continue;
 		dccp_hashinfo.bhash = (struct inet_bind_hashbucket *)
-			__get_free_pages(GFP_ATOMIC, bhash_order);
+			__get_free_pages(GFP_ATOMIC|__GFP_NOWARN, bhash_order);
 	} while (!dccp_hashinfo.bhash && --bhash_order >= 0);
 
 	if (!dccp_hashinfo.bhash) {
-- 
1.5.6.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 3/3] net-dccp: Suppress warning about large allocations from DCCP

[Arnaldo Carvalho de Melo]

Em Wed, Jul 15, 2009 at 12:23:12PM +0100, Mel Gorman escreveu:
> The DCCP protocol tries to allocate some large hash tables during
> initialisation using the largest size possible.  This can be larger than
> what the page allocator can provide so it prints a warning. However, the
> caller is able to handle the situation so this patch suppresses the warning.
> 
> Signed-off-by: Mel Gorman <mel@csn.ul.ie>

Thanks again,

Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>