From: Jakub Sitnicki <jakub@cloudflare.com>
To: John Fastabend <john.fastabend@gmail.com>
Cc: daniel@iogearbox.net, xiyou.wangcong@gmail.com,
alexei.starovoitov@gmail.com, bpf@vger.kernel.org,
netdev@vger.kernel.org
Subject: Re: [PATCH bpf 0/3] sockmap fixes picked up by stress tests
Date: Wed, 21 Jul 2021 18:38:51 +0200 [thread overview]
Message-ID: <87r1frr59g.fsf@cloudflare.com> (raw)
In-Reply-To: <20210719214834.125484-1-john.fastabend@gmail.com>
On Mon, Jul 19, 2021 at 11:48 PM CEST, John Fastabend wrote:
> Running stress tests with recent patch to remove an extra lock in sockmap
> resulted in a couple new issues popping up. It seems only one of them
> is actually related to the patch:
>
> 799aa7f98d53 ("skmsg: Avoid lock_sock() in sk_psock_backlog()")
>
> The other two issues had existed long before, but I guess the timing
> with the serialization we had before was too tight to get any of
> our tests or deployments to hit it.
>
> With attached series stress testing sockmap+TCP with workloads that
> create lots of short-lived connections no more splats like below were
> seen on upstream bpf branch.
>
> [224913.935822] WARNING: CPU: 3 PID: 32100 at net/core/stream.c:208 sk_stream_kill_queues+0x212/0x220
> [224913.935841] Modules linked in: fuse overlay bpf_preload x86_pkg_temp_thermal intel_uncore wmi_bmof squashfs sch_fq_codel efivarfs ip_tables x_tables uas xhci_pci ixgbe mdio xfrm_algo xhci_hcd wmi
> [224913.935897] CPU: 3 PID: 32100 Comm: fgs-bench Tainted: G I 5.14.0-rc1alu+ #181
> [224913.935908] Hardware name: Dell Inc. Precision 5820 Tower/002KVM, BIOS 1.9.2 01/24/2019
> [224913.935914] RIP: 0010:sk_stream_kill_queues+0x212/0x220
> [224913.935923] Code: 8b 83 20 02 00 00 85 c0 75 20 5b 5d 41 5c 41 5d 41 5e 41 5f c3 48 89 df e8 2b 11 fe ff eb c3 0f 0b e9 7c ff ff ff 0f 0b eb ce <0f> 0b 5b 5d 41 5c 41 5d 41 5e 41 5f c3 90 0f 1f 44 00 00 41 57 41
> [224913.935932] RSP: 0018:ffff88816271fd38 EFLAGS: 00010206
> [224913.935941] RAX: 0000000000000ae8 RBX: ffff88815acd5240 RCX: dffffc0000000000
> [224913.935948] RDX: 0000000000000003 RSI: 0000000000000ae8 RDI: ffff88815acd5460
> [224913.935954] RBP: ffff88815acd5460 R08: ffffffff955c0ae8 R09: fffffbfff2e6f543
> [224913.935961] R10: ffffffff9737aa17 R11: fffffbfff2e6f542 R12: ffff88815acd5390
> [224913.935967] R13: ffff88815acd5480 R14: ffffffff98d0c080 R15: ffffffff96267500
> [224913.935974] FS: 00007f86e6bd1700(0000) GS:ffff888451cc0000(0000) knlGS:0000000000000000
> [224913.935981] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [224913.935988] CR2: 000000c0008eb000 CR3: 00000001020e0005 CR4: 00000000003706e0
> [224913.935994] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [224913.936000] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [224913.936007] Call Trace:
> [224913.936016] inet_csk_destroy_sock+0xba/0x1f0
> [224913.936033] __tcp_close+0x620/0x790
> [224913.936047] tcp_close+0x20/0x80
> [224913.936056] inet_release+0x8f/0xf0
> [224913.936070] __sock_release+0x72/0x120
>
> John Fastabend (3):
> bpf, sockmap: zap ingress queues after stopping strparser
> bpf, sockmap: on cleanup we additionally need to remove cached skb
> bpf, sockmap: fix memleak on ingress msg enqueue
>
> include/linux/skmsg.h | 54 ++++++++++++++++++++++++++++---------------
> net/core/skmsg.c | 37 +++++++++++++++++++++--------
> 2 files changed, 62 insertions(+), 29 deletions(-)
Except for the uninitialized memory read reported by 0-day CI, this
series LGTM. Feel free to add my stamp to v2:
Acked-by: Jakub Sitnicki <jakub@cloudflare.com>
next prev parent reply other threads:[~2021-07-21 16:38 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-19 21:48 [PATCH bpf 0/3] sockmap fixes picked up by stress tests John Fastabend
2021-07-19 21:48 ` [PATCH bpf 1/3] bpf, sockmap: zap ingress queues after stopping strparser John Fastabend
2021-07-20 10:27 ` Jakub Sitnicki
2021-07-20 17:41 ` John Fastabend
2021-07-19 21:48 ` [PATCH bpf 2/3] bpf, sockmap: on cleanup we additionally need to remove cached skb John Fastabend
2021-07-21 10:13 ` Jakub Sitnicki
2021-07-19 21:48 ` [PATCH bpf 3/3] bpf, sockmap: fix memleak on ingress msg enqueue John Fastabend
2021-07-21 16:36 ` Jakub Sitnicki
2021-07-21 18:02 ` Martin KaFai Lau
2021-07-21 16:38 ` Jakub Sitnicki [this message]
-- strict thread matches above, loose matches on Subject: below --
2021-07-20 10:53 [PATCH bpf 2/3] bpf, sockmap: on cleanup we additionally need to remove cached skb kernel test robot
2021-07-21 9:38 ` Dan Carpenter
2021-07-21 9:38 ` Dan Carpenter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87r1frr59g.fsf@cloudflare.com \
--to=jakub@cloudflare.com \
--cc=alexei.starovoitov@gmail.com \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=john.fastabend@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=xiyou.wangcong@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.