[RFC] net: use atomic allocation for order-3 page allocation

[RFC] net: use atomic allocation for order-3 page allocation

[Shaohua Li]

We saw excessive memory compaction triggered by skb_page_frag_refill.
This causes performance issues. Commit 5640f7685831e0 introduces the
order-3 allocation to improve performance. But memory compaction has
high overhead. The benefit of order-3 allocation can't compensate the
overhead of memory compaction.

This patch makes the order-3 page allocation atomic. If there is no
memory pressure and memory isn't fragmented, the alloction will still
success, so we don't sacrifice the order-3 benefit here. If the atomic
allocation fails, compaction will not be triggered and we will fallback
to order-0 immediately.

The mellanox driver does similar thing, if this is accepted, we must fix
the driver too.

Cc: Eric Dumazet <edumazet@google.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
 net/core/sock.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/core/sock.c b/net/core/sock.c
index 292f422..e9855a4 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -1883,7 +1883,7 @@ bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t gfp)
 
 	pfrag->offset = 0;
 	if (SKB_FRAG_PAGE_ORDER) {
-		pfrag->page = alloc_pages(gfp | __GFP_COMP |
+		pfrag->page = alloc_pages((gfp & ~__GFP_WAIT) | __GFP_COMP |
 					  __GFP_NOWARN | __GFP_NORETRY,
 					  SKB_FRAG_PAGE_ORDER);
 		if (likely(pfrag->page)) {
-- 
1.8.1

Re: [RFC] net: use atomic allocation for order-3 page allocation

[Eric Dumazet]

On Thu, 2015-06-11 at 13:24 -0700, Shaohua Li wrote:
> We saw excessive memory compaction triggered by skb_page_frag_refill.
> This causes performance issues. Commit 5640f7685831e0 introduces the
> order-3 allocation to improve performance. But memory compaction has
> high overhead. The benefit of order-3 allocation can't compensate the
> overhead of memory compaction.
> 
> This patch makes the order-3 page allocation atomic. If there is no
> memory pressure and memory isn't fragmented, the alloction will still
> success, so we don't sacrifice the order-3 benefit here. If the atomic
> allocation fails, compaction will not be triggered and we will fallback
> to order-0 immediately.
> 
> The mellanox driver does similar thing, if this is accepted, we must fix
> the driver too.
> 
> Cc: Eric Dumazet <edumazet@google.com>
> Signed-off-by: Shaohua Li <shli@fb.com>
> ---
>  net/core/sock.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/net/core/sock.c b/net/core/sock.c
> index 292f422..e9855a4 100644
> --- a/net/core/sock.c
> +++ b/net/core/sock.c
> @@ -1883,7 +1883,7 @@ bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t gfp)
>  
>  	pfrag->offset = 0;
>  	if (SKB_FRAG_PAGE_ORDER) {
> -		pfrag->page = alloc_pages(gfp | __GFP_COMP |
> +		pfrag->page = alloc_pages((gfp & ~__GFP_WAIT) | __GFP_COMP |
>  					  __GFP_NOWARN | __GFP_NORETRY,
>  					  SKB_FRAG_PAGE_ORDER);
>  		if (likely(pfrag->page)) {

This is not a specific networking issue, but mm one.

You really need to start a discussion with mm experts.

Your changelog does not exactly explains what _is_ the problem.

If the problem lies in mm layer, it might be time to fix it, instead of
work around the bug by never triggering it from this particular point,
which is a safe point where a process is willing to wait a bit.

Memory compaction is either working as intending, or not.

If we enabled it but never run it because it hurts, what is the point
enabling it ?

Re: [RFC] net: use atomic allocation for order-3 page allocation

[Chris Mason]

On 06/11/2015 04:48 PM, Eric Dumazet wrote:
> On Thu, 2015-06-11 at 13:24 -0700, Shaohua Li wrote:
>> We saw excessive memory compaction triggered by skb_page_frag_refill.
>> This causes performance issues. Commit 5640f7685831e0 introduces the
>> order-3 allocation to improve performance. But memory compaction has
>> high overhead. The benefit of order-3 allocation can't compensate the
>> overhead of memory compaction.
>>
>> This patch makes the order-3 page allocation atomic. If there is no
>> memory pressure and memory isn't fragmented, the alloction will still
>> success, so we don't sacrifice the order-3 benefit here. If the atomic
>> allocation fails, compaction will not be triggered and we will fallback
>> to order-0 immediately.
>>
>> The mellanox driver does similar thing, if this is accepted, we must fix
>> the driver too.
>>
>> Cc: Eric Dumazet <edumazet@google.com>
>> Signed-off-by: Shaohua Li <shli@fb.com>
>> ---
>>  net/core/sock.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/net/core/sock.c b/net/core/sock.c
>> index 292f422..e9855a4 100644
>> --- a/net/core/sock.c
>> +++ b/net/core/sock.c
>> @@ -1883,7 +1883,7 @@ bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t gfp)
>>  
>>  	pfrag->offset = 0;
>>  	if (SKB_FRAG_PAGE_ORDER) {
>> -		pfrag->page = alloc_pages(gfp | __GFP_COMP |
>> +		pfrag->page = alloc_pages((gfp & ~__GFP_WAIT) | __GFP_COMP |
>>  					  __GFP_NOWARN | __GFP_NORETRY,
>>  					  SKB_FRAG_PAGE_ORDER);
>>  		if (likely(pfrag->page)) {
> 
> This is not a specific networking issue, but mm one.
> 
> You really need to start a discussion with mm experts.
> 
> Your changelog does not exactly explains what _is_ the problem.
> 
> If the problem lies in mm layer, it might be time to fix it, instead of
> work around the bug by never triggering it from this particular point,
> which is a safe point where a process is willing to wait a bit.
> 
> Memory compaction is either working as intending, or not.
> 
> If we enabled it but never run it because it hurts, what is the point
> enabling it ?

networking is asking for 32KB, and the MM layer is doing what it can to
provide it.  Are the gains from getting 32KB contig bigger than the cost
of moving pages around if the MM has to actually go into compaction?
Should we start disk IO to give back 32KB contig?

I think we want to tell the MM to compact in the background and give
networking 32KB if it happens to have it available.  If not, fall back
to smaller allocations without doing anything expensive.

-chris

Re: [RFC] net: use atomic allocation for order-3 page allocation

[Eric Dumazet]

On Thu, 2015-06-11 at 17:16 -0400, Chris Mason wrote:
> On 06/11/2015 04:48 PM, Eric Dumazet wrote:
> > On Thu, 2015-06-11 at 13:24 -0700, Shaohua Li wrote:
> >> We saw excessive memory compaction triggered by skb_page_frag_refill.
> >> This causes performance issues. Commit 5640f7685831e0 introduces the
> >> order-3 allocation to improve performance. But memory compaction has
> >> high overhead. The benefit of order-3 allocation can't compensate the
> >> overhead of memory compaction.
> >>
> >> This patch makes the order-3 page allocation atomic. If there is no
> >> memory pressure and memory isn't fragmented, the alloction will still
> >> success, so we don't sacrifice the order-3 benefit here. If the atomic
> >> allocation fails, compaction will not be triggered and we will fallback
> >> to order-0 immediately.
> >>
> >> The mellanox driver does similar thing, if this is accepted, we must fix
> >> the driver too.
> >>
> >> Cc: Eric Dumazet <edumazet@google.com>
> >> Signed-off-by: Shaohua Li <shli@fb.com>
> >> ---
> >>  net/core/sock.c | 2 +-
> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/net/core/sock.c b/net/core/sock.c
> >> index 292f422..e9855a4 100644
> >> --- a/net/core/sock.c
> >> +++ b/net/core/sock.c
> >> @@ -1883,7 +1883,7 @@ bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t gfp)
> >>  
> >>  	pfrag->offset = 0;
> >>  	if (SKB_FRAG_PAGE_ORDER) {
> >> -		pfrag->page = alloc_pages(gfp | __GFP_COMP |
> >> +		pfrag->page = alloc_pages((gfp & ~__GFP_WAIT) | __GFP_COMP |
> >>  					  __GFP_NOWARN | __GFP_NORETRY,
> >>  					  SKB_FRAG_PAGE_ORDER);
> >>  		if (likely(pfrag->page)) {
> > 
> > This is not a specific networking issue, but mm one.
> > 
> > You really need to start a discussion with mm experts.
> > 
> > Your changelog does not exactly explains what _is_ the problem.
> > 
> > If the problem lies in mm layer, it might be time to fix it, instead of
> > work around the bug by never triggering it from this particular point,
> > which is a safe point where a process is willing to wait a bit.
> > 
> > Memory compaction is either working as intending, or not.
> > 
> > If we enabled it but never run it because it hurts, what is the point
> > enabling it ?
> 
> networking is asking for 32KB, and the MM layer is doing what it can to
> provide it.  Are the gains from getting 32KB contig bigger than the cost
> of moving pages around if the MM has to actually go into compaction?
> Should we start disk IO to give back 32KB contig?
> 
> I think we want to tell the MM to compact in the background and give
> networking 32KB if it happens to have it available.  If not, fall back
> to smaller allocations without doing anything expensive.

Exactly my point. (And I mentioned this about 4 months ago)



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] net: use atomic allocation for order-3 page allocation

[Shaohua Li]

On Thu, Jun 11, 2015 at 02:22:13PM -0700, Eric Dumazet wrote:
> On Thu, 2015-06-11 at 17:16 -0400, Chris Mason wrote:
> > On 06/11/2015 04:48 PM, Eric Dumazet wrote:
> > > On Thu, 2015-06-11 at 13:24 -0700, Shaohua Li wrote:
> > >> We saw excessive memory compaction triggered by skb_page_frag_refill.
> > >> This causes performance issues. Commit 5640f7685831e0 introduces the
> > >> order-3 allocation to improve performance. But memory compaction has
> > >> high overhead. The benefit of order-3 allocation can't compensate the
> > >> overhead of memory compaction.
> > >>
> > >> This patch makes the order-3 page allocation atomic. If there is no
> > >> memory pressure and memory isn't fragmented, the alloction will still
> > >> success, so we don't sacrifice the order-3 benefit here. If the atomic
> > >> allocation fails, compaction will not be triggered and we will fallback
> > >> to order-0 immediately.
> > >>
> > >> The mellanox driver does similar thing, if this is accepted, we must fix
> > >> the driver too.
> > >>
> > >> Cc: Eric Dumazet <edumazet@google.com>
> > >> Signed-off-by: Shaohua Li <shli@fb.com>
> > >> ---
> > >>  net/core/sock.c | 2 +-
> > >>  1 file changed, 1 insertion(+), 1 deletion(-)
> > >>
> > >> diff --git a/net/core/sock.c b/net/core/sock.c
> > >> index 292f422..e9855a4 100644
> > >> --- a/net/core/sock.c
> > >> +++ b/net/core/sock.c
> > >> @@ -1883,7 +1883,7 @@ bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t gfp)
> > >>  
> > >>  	pfrag->offset = 0;
> > >>  	if (SKB_FRAG_PAGE_ORDER) {
> > >> -		pfrag->page = alloc_pages(gfp | __GFP_COMP |
> > >> +		pfrag->page = alloc_pages((gfp & ~__GFP_WAIT) | __GFP_COMP |
> > >>  					  __GFP_NOWARN | __GFP_NORETRY,
> > >>  					  SKB_FRAG_PAGE_ORDER);
> > >>  		if (likely(pfrag->page)) {
> > > 
> > > This is not a specific networking issue, but mm one.
> > > 
> > > You really need to start a discussion with mm experts.
> > > 
> > > Your changelog does not exactly explains what _is_ the problem.
> > > 
> > > If the problem lies in mm layer, it might be time to fix it, instead of
> > > work around the bug by never triggering it from this particular point,
> > > which is a safe point where a process is willing to wait a bit.
> > > 
> > > Memory compaction is either working as intending, or not.
> > > 
> > > If we enabled it but never run it because it hurts, what is the point
> > > enabling it ?
> > 
> > networking is asking for 32KB, and the MM layer is doing what it can to
> > provide it.  Are the gains from getting 32KB contig bigger than the cost
> > of moving pages around if the MM has to actually go into compaction?
> > Should we start disk IO to give back 32KB contig?
> > 
> > I think we want to tell the MM to compact in the background and give
> > networking 32KB if it happens to have it available.  If not, fall back
> > to smaller allocations without doing anything expensive.
> 
> Exactly my point. (And I mentioned this about 4 months ago)

This is exactly what the patch try to do. Atomic 32k allocation will
fail with memory pressure, kswapd is waken up to do compaction and we
fallback to 4k.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] net: use atomic allocation for order-3 page allocation

[Eric Dumazet]

On Thu, 2015-06-11 at 14:45 -0700, Shaohua Li wrote:

> This is exactly what the patch try to do. Atomic 32k allocation will
> fail with memory pressure, kswapd is waken up to do compaction and we
> fallback to 4k.

Read your changelog, then read what you just wrote.

Your changelog  said :

'compaction will not be triggered and we will fallback to order-0
immediately.'

Now you tell me that compaction is started.

What is the truth ?

Please make sure changelog is precise, this would avoid many mails.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] net: use atomic allocation for order-3 page allocation

[Shaohua Li]

On Thu, Jun 11, 2015 at 02:56:26PM -0700, Eric Dumazet wrote:
> On Thu, 2015-06-11 at 14:45 -0700, Shaohua Li wrote:
> 
> > This is exactly what the patch try to do. Atomic 32k allocation will
> > fail with memory pressure, kswapd is waken up to do compaction and we
> > fallback to 4k.
> 
> Read your changelog, then read what you just wrote.
> 
> Your changelog  said :
> 
> 'compaction will not be triggered and we will fallback to order-0
> immediately.'
> 
> Now you tell me that compaction is started.
> 
> What is the truth ?
> 
> Please make sure changelog is precise, this would avoid many mails.

Ah, ok. I mean direct compaction isn't triggered, kswapd is still waken
up to do compaction. I'll update the changelog.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] net: use atomic allocation for order-3 page allocation

[Chris Mason]

On 06/11/2015 05:22 PM, Eric Dumazet wrote:
> On Thu, 2015-06-11 at 17:16 -0400, Chris Mason wrote:
>> On 06/11/2015 04:48 PM, Eric Dumazet wrote:

>>
>> networking is asking for 32KB, and the MM layer is doing what it can to
>> provide it.  Are the gains from getting 32KB contig bigger than the cost
>> of moving pages around if the MM has to actually go into compaction?
>> Should we start disk IO to give back 32KB contig?
>>
>> I think we want to tell the MM to compact in the background and give
>> networking 32KB if it happens to have it available.  If not, fall back
>> to smaller allocations without doing anything expensive.
> 
> Exactly my point. (And I mentioned this about 4 months ago)

Sorry, reading this again I wasn't very clear.  I agree with Shaohua's
patch because it is telling the allocator that we don't want to wait for
reclaim or compaction to find contiguous pages.

But, is there any fallback to a single page allocation somewhere else?
If this is the only way to get memory, we might want to add a single
alloc_page path that won't trigger compaction but is at least able to
wait for kswapd to make progress.

-chris




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] net: use atomic allocation for order-3 page allocation

[Eric Dumazet]

On Thu, 2015-06-11 at 18:18 -0400, Chris Mason wrote:

> But, is there any fallback to a single page allocation somewhere else?
> If this is the only way to get memory, we might want to add a single
> alloc_page path that won't trigger compaction but is at least able to
> wait for kswapd to make progress.

Sure, there is a fallback to order-0 in both skb_page_frag_refill() and
alloc_skb_with_frags() 

They also use __GFP_NOWARN | __GFP_NORETRY


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] net: use atomic allocation for order-3 page allocation

[Debabrata Banerjee]

There is no "background" it doesn't matter if this activity happens
synchronously or asynchronously, unless you're sensitive to the
latency on that single operation. If you're driving all your cpu's and
memory hard then this is work that still takes resources. If there's a
kernel thread with compaction running, then obviously your process is
not.

Your patch should help in that not every atomic allocation failure
should mean yet another run at compaction/reclaim.

-Deb

On Thu, Jun 11, 2015 at 5:16 PM, Chris Mason <clm@fb.com> wrote:

> networking is asking for 32KB, and the MM layer is doing what it can to
> provide it.  Are the gains from getting 32KB contig bigger than the cost
> of moving pages around if the MM has to actually go into compaction?
> Should we start disk IO to give back 32KB contig?
>
> I think we want to tell the MM to compact in the background and give
> networking 32KB if it happens to have it available.  If not, fall back
> to smaller allocations without doing anything expensive.
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] net: use atomic allocation for order-3 page allocation

[David Miller]

Please stop top-posting.

Quote the relevant material you are replying to first, the add your
response commentary afterwards rather than beforehand.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] net: use atomic allocation for order-3 page allocation

[Vlastimil Babka]

On 06/11/2015 11:35 PM, Debabrata Banerjee wrote:
> There is no "background" it doesn't matter if this activity happens
> synchronously or asynchronously, unless you're sensitive to the
> latency on that single operation. If you're driving all your cpu's and
> memory hard then this is work that still takes resources. If there's a
> kernel thread with compaction running, then obviously your process is
> not.

Well that of course depends on the CPU utilization of "your process".

> Your patch should help in that not every atomic allocation failure
> should mean yet another run at compaction/reclaim.

If you don't want to wake up kswapd, add also __GFP_NO_KSWAPD flag. 
Additionally, gfp_to_alloc_flags() will stop treating such allocation as 
atomic - it allows atomic allocations to bypass cpusets and lowers the 
watermark by 1/4 (unless there's also __GFP_NOMEMALLOC). It might 
actually make sense to add __GFP_NO_KSWAPD for an allocation like this 
one that has a simple order-0 fallback.

Vlastimil


> -Deb
>
> On Thu, Jun 11, 2015 at 5:16 PM, Chris Mason <clm@fb.com> wrote:
>
>> networking is asking for 32KB, and the MM layer is doing what it can to
>> provide it.  Are the gains from getting 32KB contig bigger than the cost
>> of moving pages around if the MM has to actually go into compaction?
>> Should we start disk IO to give back 32KB contig?
>>
>> I think we want to tell the MM to compact in the background and give
>> networking 32KB if it happens to have it available.  If not, fall back
>> to smaller allocations without doing anything expensive.
>>
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] net: use atomic allocation for order-3 page allocation

[Debabrata Banerjee]

[-- Attachment #1: Type: text/plain, Size: 3466 bytes --]

It's somewhat an intractable problem to know if compaction will succeed
without trying it, and you can certainly end up in a state where memory is
heavily fragmented, even with compaction running. You can't compact kernel
pages for example, so you can end up in a state where compaction does
nothing through no fault of it's own.

In this case you waste time in compaction routines, then end up reclaiming
precious page cache pages or swapping out for whatever it is your machine
was doing trying to do to satisfy these order-3 allocations, after which
all those pages need to be restored from disk almost immediately. This is
not a happy server. Any mm fix may be years away. The only simple solution
I can think of is specifically caching these allocations, in any other case
under memory pressure they will be split by other smaller allocations.

We've been forcing these allocations to order-0 internally until we can
think of something else.

-Deb

On Thu, Jun 11, 2015 at 4:48 PM, Eric Dumazet <eric.dumazet@gmail.com>
wrote:

> On Thu, 2015-06-11 at 13:24 -0700, Shaohua Li wrote:
> > We saw excessive memory compaction triggered by skb_page_frag_refill.
> > This causes performance issues. Commit 5640f7685831e0 introduces the
> > order-3 allocation to improve performance. But memory compaction has
> > high overhead. The benefit of order-3 allocation can't compensate the
> > overhead of memory compaction.
> >
> > This patch makes the order-3 page allocation atomic. If there is no
> > memory pressure and memory isn't fragmented, the alloction will still
> > success, so we don't sacrifice the order-3 benefit here. If the atomic
> > allocation fails, compaction will not be triggered and we will fallback
> > to order-0 immediately.
> >
> > The mellanox driver does similar thing, if this is accepted, we must fix
> > the driver too.
> >
> > Cc: Eric Dumazet <edumazet@google.com>
> > Signed-off-by: Shaohua Li <shli@fb.com>
> > ---
> >  net/core/sock.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/net/core/sock.c b/net/core/sock.c
> > index 292f422..e9855a4 100644
> > --- a/net/core/sock.c
> > +++ b/net/core/sock.c
> > @@ -1883,7 +1883,7 @@ bool skb_page_frag_refill(unsigned int sz, struct
> page_frag *pfrag, gfp_t gfp)
> >
> >       pfrag->offset = 0;
> >       if (SKB_FRAG_PAGE_ORDER) {
> > -             pfrag->page = alloc_pages(gfp | __GFP_COMP |
> > +             pfrag->page = alloc_pages((gfp & ~__GFP_WAIT) | __GFP_COMP
> |
> >                                         __GFP_NOWARN | __GFP_NORETRY,
> >                                         SKB_FRAG_PAGE_ORDER);
> >               if (likely(pfrag->page)) {
>
> This is not a specific networking issue, but mm one.
>
> You really need to start a discussion with mm experts.
>
> Your changelog does not exactly explains what _is_ the problem.
>
> If the problem lies in mm layer, it might be time to fix it, instead of
> work around the bug by never triggering it from this particular point,
> which is a safe point where a process is willing to wait a bit.
>
> Memory compaction is either working as intending, or not.
>
> If we enabled it but never run it because it hurts, what is the point
> enabling it ?
>
>
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>

[-- Attachment #2: Type: text/html, Size: 4591 bytes --]

Re: [RFC] net: use atomic allocation for order-3 page allocation

[Debabrata Banerjee]

Resend in plaintext, thanks gmail:

It's somewhat an intractable problem to know if compaction will succeed
without trying it, and you can certainly end up in a state where memory is
heavily fragmented, even with compaction running. You can't compact kernel
pages for example, so you can end up in a state where compaction does
nothing through no fault of it's own.

In this case you waste time in compaction routines, then end up reclaiming
precious page cache pages or swapping out for whatever it is your machine
was doing trying to do to satisfy these order-3 allocations, after which all
those pages need to be restored from disk almost immediately. This is not a
happy server. Any mm fix may be years away. The only simple solution I can
think of is specifically caching these allocations, in any other case under
memory pressure they will be split by other smaller allocations.

We've been forcing these allocations to order-0 internally until we can
think of something else.

-Deb


> On Thu, Jun 11, 2015 at 4:48 PM, Eric Dumazet <eric.dumazet@gmail.com>
> wrote:
>>
>> On Thu, 2015-06-11 at 13:24 -0700, Shaohua Li wrote:
>> > We saw excessive memory compaction triggered by skb_page_frag_refill.
>> > This causes performance issues. Commit 5640f7685831e0 introduces the
>> > order-3 allocation to improve performance. But memory compaction has
>> > high overhead. The benefit of order-3 allocation can't compensate the
>> > overhead of memory compaction.
>> >
>> > This patch makes the order-3 page allocation atomic. If there is no
>> > memory pressure and memory isn't fragmented, the alloction will still
>> > success, so we don't sacrifice the order-3 benefit here. If the atomic
>> > allocation fails, compaction will not be triggered and we will fallback
>> > to order-0 immediately.
>> >
>> > The mellanox driver does similar thing, if this is accepted, we must fix
>> > the driver too.
>> >
>> > Cc: Eric Dumazet <edumazet@google.com>
>> > Signed-off-by: Shaohua Li <shli@fb.com>
>> > ---
>> >  net/core/sock.c | 2 +-
>> >  1 file changed, 1 insertion(+), 1 deletion(-)
>> >
>> > diff --git a/net/core/sock.c b/net/core/sock.c
>> > index 292f422..e9855a4 100644
>> > --- a/net/core/sock.c
>> > +++ b/net/core/sock.c
>> > @@ -1883,7 +1883,7 @@ bool skb_page_frag_refill(unsigned int sz, struct
>> > page_frag *pfrag, gfp_t gfp)
>> >
>> >       pfrag->offset = 0;
>> >       if (SKB_FRAG_PAGE_ORDER) {
>> > -             pfrag->page = alloc_pages(gfp | __GFP_COMP |
>> > +             pfrag->page = alloc_pages((gfp & ~__GFP_WAIT) | __GFP_COMP
>> > |
>> >                                         __GFP_NOWARN | __GFP_NORETRY,
>> >                                         SKB_FRAG_PAGE_ORDER);
>> >               if (likely(pfrag->page)) {
>>
>> This is not a specific networking issue, but mm one.
>>
>> You really need to start a discussion with mm experts.
>>
>> Your changelog does not exactly explains what _is_ the problem.
>>
>> If the problem lies in mm layer, it might be time to fix it, instead of
>> work around the bug by never triggering it from this particular point,
>> which is a safe point where a process is willing to wait a bit.
>>
>> Memory compaction is either working as intending, or not.
>>
>> If we enabled it but never run it because it hurts, what is the point
>> enabling it ?
>>
>>
>>
>> --
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to majordomo@kvack.org.  For more info on Linux MM,
>> see: http://www.linux-mm.org/ .
>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] net: use atomic allocation for order-3 page allocation

[Vlastimil Babka]

On 06/11/2015 11:28 PM, Debabrata Banerjee wrote:
> Resend in plaintext, thanks gmail:
>
> It's somewhat an intractable problem to know if compaction will succeed
> without trying it,

There are heuristics, but those cannot be perfect by definition. I think 
the worse problem here is the extra latency, even if it does succeed, 
though.

> and you can certainly end up in a state where memory is
> heavily fragmented, even with compaction running. You can't compact kernel
> pages for example, so you can end up in a state where compaction does
> nothing through no fault of it's own.

Correct.

> In this case you waste time in compaction routines, then end up reclaiming
> precious page cache pages or swapping out for whatever it is your machine
> was doing trying to do to satisfy these order-3 allocations, after which all
> those pages need to be restored from disk almost immediately. This is not a
> happy server.

That sounds like an overloaded server to me.

> Any mm fix may be years away.

Well, what kind of "fix"? There's no way to always avoid fragmentation 
without some kind of an oracle that will tell you which unmovable 
allocations (e.g. kernel pages) to put side by side because they will be 
freed at the same time.

> The only simple solution I can
> think of is specifically caching these allocations, in any other case under
> memory pressure they will be split by other smaller allocations.

In this case the allocations have simple fallback to order-0, so caching 
them would make sense only if someone shows that the benefits of having 
order-3 instead of order-0 them are worth it.

> We've been forcing these allocations to order-0 internally until we can
> think of something else.

I think the proposed patch is better than forcing everything to order-0. 
It makes the attempt to allocate order-3 cheap.

The VM should generally serve you better if it's told your requirements. 
Communicating that the order-3 allocation is just an opportunistic 
attempt with simple fallback is the right way.

> -Deb
>
>
>> On Thu, Jun 11, 2015 at 4:48 PM, Eric Dumazet <eric.dumazet@gmail.com>
>> wrote:
>>>
>>> On Thu, 2015-06-11 at 13:24 -0700, Shaohua Li wrote:
>>>> We saw excessive memory compaction triggered by skb_page_frag_refill.
>>>> This causes performance issues. Commit 5640f7685831e0 introduces the
>>>> order-3 allocation to improve performance. But memory compaction has
>>>> high overhead. The benefit of order-3 allocation can't compensate the
>>>> overhead of memory compaction.
>>>>
>>>> This patch makes the order-3 page allocation atomic. If there is no
>>>> memory pressure and memory isn't fragmented, the alloction will still
>>>> success, so we don't sacrifice the order-3 benefit here. If the atomic
>>>> allocation fails, compaction will not be triggered and we will fallback
>>>> to order-0 immediately.
>>>>
>>>> The mellanox driver does similar thing, if this is accepted, we must fix
>>>> the driver too.
>>>>
>>>> Cc: Eric Dumazet <edumazet@google.com>
>>>> Signed-off-by: Shaohua Li <shli@fb.com>
>>>> ---
>>>>   net/core/sock.c | 2 +-
>>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/net/core/sock.c b/net/core/sock.c
>>>> index 292f422..e9855a4 100644
>>>> --- a/net/core/sock.c
>>>> +++ b/net/core/sock.c
>>>> @@ -1883,7 +1883,7 @@ bool skb_page_frag_refill(unsigned int sz, struct
>>>> page_frag *pfrag, gfp_t gfp)
>>>>
>>>>        pfrag->offset = 0;
>>>>        if (SKB_FRAG_PAGE_ORDER) {
>>>> -             pfrag->page = alloc_pages(gfp | __GFP_COMP |
>>>> +             pfrag->page = alloc_pages((gfp & ~__GFP_WAIT) | __GFP_COMP
>>>> |
>>>>                                          __GFP_NOWARN | __GFP_NORETRY,
>>>>                                          SKB_FRAG_PAGE_ORDER);
>>>>                if (likely(pfrag->page)) {
>>>
>>> This is not a specific networking issue, but mm one.
>>>
>>> You really need to start a discussion with mm experts.
>>>
>>> Your changelog does not exactly explains what _is_ the problem.
>>>
>>> If the problem lies in mm layer, it might be time to fix it, instead of
>>> work around the bug by never triggering it from this particular point,
>>> which is a safe point where a process is willing to wait a bit.
>>>
>>> Memory compaction is either working as intending, or not.
>>>
>>> If we enabled it but never run it because it hurts, what is the point
>>> enabling it ?
>>>
>>>
>>>
>>> --
>>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>>> the body to majordomo@kvack.org.  For more info on Linux MM,
>>> see: http://www.linux-mm.org/ .
>>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>>
>>
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC v2] net: use atomic allocation for order-3 page allocation

[Eric Dumazet]

On Thu, 2015-06-11 at 15:27 -0700, Shaohua Li wrote:
> We saw excessive direct memory compaction triggered by skb_page_frag_refill.
> This causes performance issues and add latency. Commit 5640f7685831e0
> introduces the order-3 allocation. According to the changelog, the order-3
> allocation isn't a must-have but to improve performance. But direct memory
> compaction has high overhead. The benefit of order-3 allocation can't
> compensate the overhead of direct memory compaction.
> 
> This patch makes the order-3 page allocation atomic. If there is no memory
> pressure and memory isn't fragmented, the alloction will still success, so we
> don't sacrifice the order-3 benefit here. If the atomic allocation fails,
> direct memory compaction will not be triggered, skb_page_frag_refill will
> fallback to order-0 immediately, hence the direct memory compaction overhead is
> avoided. In the allocation failure case, kswapd is waken up and doing
> compaction, so chances are allocation could success next time.
> 
> The mellanox driver does similar thing, if this is accepted, we must fix
> the driver too.
> 
> V2: make the changelog clearer
> 
> Cc: Eric Dumazet <edumazet@google.com>
> Signed-off-by: Shaohua Li <shli@fb.com>
> ---
>  net/core/sock.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/net/core/sock.c b/net/core/sock.c
> index 292f422..e9855a4 100644
> --- a/net/core/sock.c
> +++ b/net/core/sock.c
> @@ -1883,7 +1883,7 @@ bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t gfp)
>  
>  	pfrag->offset = 0;
>  	if (SKB_FRAG_PAGE_ORDER) {
> -		pfrag->page = alloc_pages(gfp | __GFP_COMP |
> +		pfrag->page = alloc_pages((gfp & ~__GFP_WAIT) | __GFP_COMP |
>  					  __GFP_NOWARN | __GFP_NORETRY,
>  					  SKB_FRAG_PAGE_ORDER);
>  		if (likely(pfrag->page)) {


OK, now what about alloc_skb_with_frags() ?

This should have same problem right ?

Thanks.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC v2] net: use atomic allocation for order-3 page allocation

[Shaohua Li]

On Thu, Jun 11, 2015 at 03:53:04PM -0700, Eric Dumazet wrote:
> On Thu, 2015-06-11 at 15:27 -0700, Shaohua Li wrote:
> > We saw excessive direct memory compaction triggered by skb_page_frag_refill.
> > This causes performance issues and add latency. Commit 5640f7685831e0
> > introduces the order-3 allocation. According to the changelog, the order-3
> > allocation isn't a must-have but to improve performance. But direct memory
> > compaction has high overhead. The benefit of order-3 allocation can't
> > compensate the overhead of direct memory compaction.
> > 
> > This patch makes the order-3 page allocation atomic. If there is no memory
> > pressure and memory isn't fragmented, the alloction will still success, so we
> > don't sacrifice the order-3 benefit here. If the atomic allocation fails,
> > direct memory compaction will not be triggered, skb_page_frag_refill will
> > fallback to order-0 immediately, hence the direct memory compaction overhead is
> > avoided. In the allocation failure case, kswapd is waken up and doing
> > compaction, so chances are allocation could success next time.
> > 
> > The mellanox driver does similar thing, if this is accepted, we must fix
> > the driver too.
> > 
> > V2: make the changelog clearer
> > 
> > Cc: Eric Dumazet <edumazet@google.com>
> > Signed-off-by: Shaohua Li <shli@fb.com>
> > ---
> >  net/core/sock.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/net/core/sock.c b/net/core/sock.c
> > index 292f422..e9855a4 100644
> > --- a/net/core/sock.c
> > +++ b/net/core/sock.c
> > @@ -1883,7 +1883,7 @@ bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t gfp)
> >  
> >  	pfrag->offset = 0;
> >  	if (SKB_FRAG_PAGE_ORDER) {
> > -		pfrag->page = alloc_pages(gfp | __GFP_COMP |
> > +		pfrag->page = alloc_pages((gfp & ~__GFP_WAIT) | __GFP_COMP |
> >  					  __GFP_NOWARN | __GFP_NORETRY,
> >  					  SKB_FRAG_PAGE_ORDER);
> >  		if (likely(pfrag->page)) {
> 
> 
> OK, now what about alloc_skb_with_frags() ?
> 
> This should have same problem right ?

Ok, looks similar, added. Didn't trigger this one though.


>From 940dde18f7f655377a4c30d5de54c9eff15ab5a5 Mon Sep 17 00:00:00 2001
Message-Id: <940dde18f7f655377a4c30d5de54c9eff15ab5a5.1434065353.git.shli@fb.com>
From: Shaohua Li <shli@fb.com>
Date: Thu, 11 Jun 2015 16:16:21 -0700
Subject: [RFC] net: use atomic allocation for order-3 page allocation

We saw excessive direct memory compaction triggered by skb_page_frag_refill.
This causes performance issues and add latency. Commit 5640f7685831e0
introduces the order-3 allocation. According to the changelog, the order-3
allocation isn't a must-have but to improve performance. But direct memory
compaction has high overhead. The benefit of order-3 allocation can't
compensate the overhead of direct memory compaction.

This patch makes the order-3 page allocation atomic. If there is no memory
pressure and memory isn't fragmented, the alloction will still success, so we
don't sacrifice the order-3 benefit here. If the atomic allocation fails,
direct memory compaction will not be triggered, skb_page_frag_refill will
fallback to order-0 immediately, hence the direct memory compaction overhead is
avoided. In the allocation failure case, kswapd is waken up and doing
compaction, so chances are allocation could success next time.

alloc_skb_with_frags is the same.

The mellanox driver does similar thing, if this is accepted, we must fix
the driver too.

V3: fix the same issue in alloc_skb_with_frags as pointed out by Eric
V2: make the changelog clearer

Cc: Eric Dumazet <edumazet@google.com>
Cc: Chris Mason <clm@fb.com>
Cc: Debabrata Banerjee <dbavatar@gmail.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
 net/core/skbuff.c | 4 +++-
 net/core/sock.c   | 2 +-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 3cfff2a..9856c7a 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -4398,7 +4398,9 @@ struct sk_buff *alloc_skb_with_frags(unsigned long header_len,
 
 		while (order) {
 			if (npages >= 1 << order) {
-				page = alloc_pages(gfp_mask |
+				gfp_t gfp = order > 0 ?
+					gfp_mask & ~__GFP_WAIT : gfp_mask;
+				page = alloc_pages(gfp |
 						   __GFP_COMP |
 						   __GFP_NOWARN |
 						   __GFP_NORETRY,
diff --git a/net/core/sock.c b/net/core/sock.c
index 292f422..e9855a4 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -1883,7 +1883,7 @@ bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t gfp)
 
 	pfrag->offset = 0;
 	if (SKB_FRAG_PAGE_ORDER) {
-		pfrag->page = alloc_pages(gfp | __GFP_COMP |
+		pfrag->page = alloc_pages((gfp & ~__GFP_WAIT) | __GFP_COMP |
 					  __GFP_NOWARN | __GFP_NORETRY,
 					  SKB_FRAG_PAGE_ORDER);
 		if (likely(pfrag->page)) {
-- 
1.8.1

Re: [RFC v2] net: use atomic allocation for order-3 page allocation

[Eric Dumazet]

On Thu, 2015-06-11 at 16:32 -0700, Shaohua Li wrote:

> 
> Ok, looks similar, added. Didn't trigger this one though.

Probably because you do not use af_unix with big enough messages.

> diff --git a/net/core/skbuff.c b/net/core/skbuff.c
> index 3cfff2a..9856c7a 100644
> --- a/net/core/skbuff.c
> +++ b/net/core/skbuff.c
> @@ -4398,7 +4398,9 @@ struct sk_buff *alloc_skb_with_frags(unsigned long header_len,
>  
>  		while (order) {
>  			if (npages >= 1 << order) {
> -				page = alloc_pages(gfp_mask |

Here, order is > 0 (Look at while (order) right above) 

> +				gfp_t gfp = order > 0 ?
> +					gfp_mask & ~__GFP_WAIT : gfp_mask;
> +				page = alloc_pages(gfp |
>  						   __GFP_COMP |
>  						   __GFP_NOWARN |



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>