[PATCH] netfilter: avoid stalls in nf_ct_alloc_hashtable

[PATCH] netfilter: avoid stalls in nf_ct_alloc_hashtable

[Li RongQing]

when system forks a process with CLONE_NEWNET flag under the
high memory pressure, it will trigger memory reclaim and stall
for a long time because nf_ct_alloc_hashtable need to allocate
high-order memory at that time. The calltrace as below:

	delay_tsc
	__delay
	_raw_spin_lock
	_spin_lock
	mmu_shrink
	shrink_slab
	zone_reclaim
	get_page_from_freelist
	__alloc_pages_nodemask
	alloc_pages_current
	__get_free_pages
	nf_ct_alloc_hashtable
	nf_conntrack_init_net
	setup_net
	copy_net_ns
	create_new_namespaces
	copy_namespaces
	copy_process
	do_fork
	sys_clone
	stub_clone
	__clone

not use the directly memory reclaim flag to avoid stall

Signed-off-by: Ni Xun <nixun@baidu.com>
Signed-off-by: Zhang Yu <zhangyu31@baidu.com>
Signed-off-by: Wang Li <wangli39@baidu.com>
Signed-off-by: Li RongQing <lirongqing@baidu.com>
---
 net/netfilter/nf_conntrack_core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c
index 8a113ca1eea2..672c5960530d 100644
--- a/net/netfilter/nf_conntrack_core.c
+++ b/net/netfilter/nf_conntrack_core.c
@@ -2120,8 +2120,8 @@ void *nf_ct_alloc_hashtable(unsigned int *sizep, int nulls)
 		return NULL;
 
 	sz = nr_slots * sizeof(struct hlist_nulls_head);
-	hash = (void *)__get_free_pages(GFP_KERNEL | __GFP_NOWARN | __GFP_ZERO,
-					get_order(sz));
+	hash = (void *)__get_free_pages((GFP_KERNEL & ~__GFP_DIRECT_RECLAIM) |
+				    __GFP_NOWARN | __GFP_ZERO, get_order(sz));
 	if (!hash)
 		hash = vzalloc(sz);
 
-- 
2.16.2

Re: [PATCH] netfilter: avoid stalls in nf_ct_alloc_hashtable

[Florian Westphal]

Li RongQing <lirongqing@baidu.com> wrote:
> when system forks a process with CLONE_NEWNET flag under the
> high memory pressure, it will trigger memory reclaim and stall
> for a long time because nf_ct_alloc_hashtable need to allocate
> high-order memory at that time. The calltrace as below:

> 	nf_ct_alloc_hashtable
> 	nf_conntrack_init_net

This call trace is from a kernel < 4.7.

commit 56d52d4892d0e478a005b99ed10d0a7f488ea8c1
netfilter: conntrack: use a single hashtable for all namespaces

removed per-netns hash table.

答复: [PATCH] netfilter: avoid stalls in nf_ct_alloc_hashtable

[Li,Rongqing]

> -----邮件原件-----
> 发件人: Florian Westphal [mailto:fw@strlen.de]
> 发送时间: 2018年7月24日 17:20
> 收件人: Li,Rongqing <lirongqing@baidu.com>
> 抄送: netdev@vger.kernel.org; pablo@netfilter.org;
> kadlec@blackhole.kfki.hu; fw@strlen.de
> 主题: Re: [PATCH] netfilter: avoid stalls in nf_ct_alloc_hashtable
> 
> Li RongQing <lirongqing@baidu.com> wrote:
> > when system forks a process with CLONE_NEWNET flag under the high
> > memory pressure, it will trigger memory reclaim and stall for a long
> > time because nf_ct_alloc_hashtable need to allocate high-order memory
> > at that time. The calltrace as below:
> 
> > 	nf_ct_alloc_hashtable
> > 	nf_conntrack_init_net
> 
> This call trace is from a kernel < 4.7.
> 

Sorry;  it is

> commit 56d52d4892d0e478a005b99ed10d0a7f488ea8c1
> netfilter: conntrack: use a single hashtable for all namespaces
> 
> removed per-netns hash table.

Thanks, Your patch fixes my issue;  

My patch may be able to reduce stall when modprobe nf module in memory stress, 
Do you think this patch has any value?

-RongQing

Re: [PATCH] netfilter: avoid stalls in nf_ct_alloc_hashtable

[Sergei Shtylyov]

Hello!

On 7/24/2018 10:19 AM, Li RongQing wrote:

> when system forks a process with CLONE_NEWNET flag under the
> high memory pressure, it will trigger memory reclaim and stall
> for a long time because nf_ct_alloc_hashtable need to allocate
> high-order memory at that time. The calltrace as below:
> 
> 	delay_tsc
> 	__delay
> 	_raw_spin_lock
> 	_spin_lock
> 	mmu_shrink
> 	shrink_slab
> 	zone_reclaim
> 	get_page_from_freelist
> 	__alloc_pages_nodemask
> 	alloc_pages_current
> 	__get_free_pages
> 	nf_ct_alloc_hashtable
> 	nf_conntrack_init_net
> 	setup_net
> 	copy_net_ns
> 	create_new_namespaces
> 	copy_namespaces
> 	copy_process
> 	do_fork
> 	sys_clone
> 	stub_clone
> 	__clone
> 
> not use the directly memory reclaim flag to avoid stall

    You mean "do not use"?

> Signed-off-by: Ni Xun <nixun@baidu.com>
> Signed-off-by: Zhang Yu <zhangyu31@baidu.com>
> Signed-off-by: Wang Li <wangli39@baidu.com>
> Signed-off-by: Li RongQing <lirongqing@baidu.com>
[...]

MBR, Sergei

Re: 答复: [PATCH] netfilter: avoid stalls in nf_ct_alloc_hashtable

[Eric Dumazet]

On 07/24/2018 02:50 AM, Li,Rongqing wrote:

> Thanks, Your patch fixes my issue;  
> 
> My patch may be able to reduce stall when modprobe nf module in memory stress, 
> Do you think this patch has any value?

Only if you make it use kvzalloc()/kvfree() 

Thanks.

答复: 答复: [PATCH] netfilter: avoid stalls in nf_ct_alloc_hashtable

[Li,Rongqing]

> 
> On 07/24/2018 02:50 AM, Li,Rongqing wrote:
> 
> > Thanks, Your patch fixes my issue;
> >
> > My patch may be able to reduce stall when modprobe nf module in
> memory
> > stress, Do you think this patch has any value?
> 
> Only if you make it use kvzalloc()/kvfree()
> 
> Thanks.


I will send v2, free to give your signature.

Thanks,

-RongQing