* [PATCH bpf 1/2] Revert "xsk: support redirect to any socket bound to the same umem"
2024-06-04 12:29 [PATCH bpf 0/2] Revert "xsk: support redirect to any socket bound to the same umem" Magnus Karlsson
@ 2024-06-04 12:29 ` Magnus Karlsson
2024-06-04 12:29 ` [PATCH bpf 2/2] Revert "xsk: document ability to " Magnus Karlsson
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Magnus Karlsson @ 2024-06-04 12:29 UTC (permalink / raw)
To: magnus.karlsson, bjorn, ast, daniel, netdev, maciej.fijalkowski,
bpf; +Cc: YuvalE
From: Magnus Karlsson <magnus.karlsson@intel.com>
This reverts commit 2863d665ea41282379f108e4da6c8a2366ba66db.
This patch introduced a potential kernel crash when multiple napi
instances redirect to the same AF_XDP socket. By removing the
queue_index check, it is possible for multiple napi instances to
access the Rx ring at the same time, which will result in a corrupted
ring state which can lead to a crash when flushing the rings in
__xsk_flush(). This can happen when the linked list of sockets to
flush gets corrupted by concurrent accesses. A quick and small fix is
not possible, so let us revert this for now.
Reported-by: Yuval El-Hanany <YuvalE@radware.com>
Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Link: https://lore.kernel.org/xdp-newbies/8100DBDC-0B7C-49DB-9995-6027F6E63147@radware.com/
---
net/xdp/xsk.c | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c
index 727aa20be4bd..7d1c0986f9bb 100644
--- a/net/xdp/xsk.c
+++ b/net/xdp/xsk.c
@@ -313,13 +313,10 @@ static bool xsk_is_bound(struct xdp_sock *xs)
static int xsk_rcv_check(struct xdp_sock *xs, struct xdp_buff *xdp, u32 len)
{
- struct net_device *dev = xdp->rxq->dev;
- u32 qid = xdp->rxq->queue_index;
-
if (!xsk_is_bound(xs))
return -ENXIO;
- if (!dev->_rx[qid].pool || xs->umem != dev->_rx[qid].pool->umem)
+ if (xs->dev != xdp->rxq->dev || xs->queue_id != xdp->rxq->queue_index)
return -EINVAL;
if (len > xsk_pool_get_rx_frame_size(xs->pool) && !xs->sg) {
--
2.45.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH bpf 2/2] Revert "xsk: document ability to redirect to any socket bound to the same umem"
2024-06-04 12:29 [PATCH bpf 0/2] Revert "xsk: support redirect to any socket bound to the same umem" Magnus Karlsson
2024-06-04 12:29 ` [PATCH bpf 1/2] " Magnus Karlsson
@ 2024-06-04 12:29 ` Magnus Karlsson
2024-06-04 23:03 ` [PATCH bpf 0/2] Revert "xsk: support " Willem de Bruijn
2024-06-05 7:50 ` patchwork-bot+netdevbpf
3 siblings, 0 replies; 7+ messages in thread
From: Magnus Karlsson @ 2024-06-04 12:29 UTC (permalink / raw)
To: magnus.karlsson, bjorn, ast, daniel, netdev, maciej.fijalkowski,
bpf; +Cc: YuvalE
From: Magnus Karlsson <magnus.karlsson@intel.com>
This reverts commit 968595a93669b6b4f6d1fcf80cf2d97956b6868f.
Reported-by: Yuval El-Hanany <YuvalE@radware.com>
Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Link: https://lore.kernel.org/xdp-newbies/8100DBDC-0B7C-49DB-9995-6027F6E63147@radware.com/
---
Documentation/networking/af_xdp.rst | 33 ++++++++++++-----------------
1 file changed, 14 insertions(+), 19 deletions(-)
diff --git a/Documentation/networking/af_xdp.rst b/Documentation/networking/af_xdp.rst
index 72da7057e4cf..dceeb0d763aa 100644
--- a/Documentation/networking/af_xdp.rst
+++ b/Documentation/networking/af_xdp.rst
@@ -329,24 +329,23 @@ XDP_SHARED_UMEM option and provide the initial socket's fd in the
sxdp_shared_umem_fd field as you registered the UMEM on that
socket. These two sockets will now share one and the same UMEM.
-In this case, it is possible to use the NIC's packet steering
-capabilities to steer the packets to the right queue. This is not
-possible in the previous example as there is only one queue shared
-among sockets, so the NIC cannot do this steering as it can only steer
-between queues.
-
-In libxdp (or libbpf prior to version 1.0), you need to use the
-xsk_socket__create_shared() API as it takes a reference to a FILL ring
-and a COMPLETION ring that will be created for you and bound to the
-shared UMEM. You can use this function for all the sockets you create,
-or you can use it for the second and following ones and use
-xsk_socket__create() for the first one. Both methods yield the same
-result.
+There is no need to supply an XDP program like the one in the previous
+case where sockets were bound to the same queue id and
+device. Instead, use the NIC's packet steering capabilities to steer
+the packets to the right queue. In the previous example, there is only
+one queue shared among sockets, so the NIC cannot do this steering. It
+can only steer between queues.
+
+In libbpf, you need to use the xsk_socket__create_shared() API as it
+takes a reference to a FILL ring and a COMPLETION ring that will be
+created for you and bound to the shared UMEM. You can use this
+function for all the sockets you create, or you can use it for the
+second and following ones and use xsk_socket__create() for the first
+one. Both methods yield the same result.
Note that a UMEM can be shared between sockets on the same queue id
and device, as well as between queues on the same device and between
-devices at the same time. It is also possible to redirect to any
-socket as long as it is bound to the same umem with XDP_SHARED_UMEM.
+devices at the same time.
XDP_USE_NEED_WAKEUP bind flag
-----------------------------
@@ -823,10 +822,6 @@ A: The short answer is no, that is not supported at the moment. The
switch, or other distribution mechanism, in your NIC to direct
traffic to the correct queue id and socket.
- Note that if you are using the XDP_SHARED_UMEM option, it is
- possible to switch traffic between any socket bound to the same
- umem.
-
Q: My packets are sometimes corrupted. What is wrong?
A: Care has to be taken not to feed the same buffer in the UMEM into
--
2.45.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [PATCH bpf 0/2] Revert "xsk: support redirect to any socket bound to the same umem"
2024-06-04 12:29 [PATCH bpf 0/2] Revert "xsk: support redirect to any socket bound to the same umem" Magnus Karlsson
2024-06-04 12:29 ` [PATCH bpf 1/2] " Magnus Karlsson
2024-06-04 12:29 ` [PATCH bpf 2/2] Revert "xsk: document ability to " Magnus Karlsson
@ 2024-06-04 23:03 ` Willem de Bruijn
2024-06-05 7:45 ` Magnus Karlsson
2024-06-05 7:50 ` patchwork-bot+netdevbpf
3 siblings, 1 reply; 7+ messages in thread
From: Willem de Bruijn @ 2024-06-04 23:03 UTC (permalink / raw)
To: Magnus Karlsson, magnus.karlsson, bjorn, ast, daniel, netdev,
maciej.fijalkowski, bpf
Cc: Magnus Karlsson, YuvalE
Magnus Karlsson wrote:
> Revert "xsk: support redirect to any socket bound to the same umem"
>
> This patch introduced a potential kernel crash when multiple napi
> instances redirect to the same AF_XDP socket. By removing the
> queue_index check, it is possible for multiple napi instances to
> access the Rx ring at the same time, which will result in a corrupted
> ring state which can lead to a crash when flushing the rings in
> __xsk_flush(). This can happen when the linked list of sockets to
> flush gets corrupted by concurrent accesses. A quick and small fix is
> unfortunately not possible, so let us revert this for now.
This is a very useful feature, to be able to use AF_XDP sockets with
a standard RSS nic configuration.
Not all AF_XDP use cases require the absolute highest packet rate.
Can this be addressed with an optional spinlock on the RxQ, only for
this case?
If there is no simple enough fix in the short term, do you plan to
reintroduce this in another form later?
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH bpf 0/2] Revert "xsk: support redirect to any socket bound to the same umem"
2024-06-04 23:03 ` [PATCH bpf 0/2] Revert "xsk: support " Willem de Bruijn
@ 2024-06-05 7:45 ` Magnus Karlsson
2024-06-05 19:28 ` Willem de Bruijn
0 siblings, 1 reply; 7+ messages in thread
From: Magnus Karlsson @ 2024-06-05 7:45 UTC (permalink / raw)
To: Willem de Bruijn
Cc: magnus.karlsson, bjorn, ast, daniel, netdev, maciej.fijalkowski,
bpf, YuvalE
On Wed, 5 Jun 2024 at 01:03, Willem de Bruijn
<willemdebruijn.kernel@gmail.com> wrote:
>
> Magnus Karlsson wrote:
> > Revert "xsk: support redirect to any socket bound to the same umem"
> >
> > This patch introduced a potential kernel crash when multiple napi
> > instances redirect to the same AF_XDP socket. By removing the
> > queue_index check, it is possible for multiple napi instances to
> > access the Rx ring at the same time, which will result in a corrupted
> > ring state which can lead to a crash when flushing the rings in
> > __xsk_flush(). This can happen when the linked list of sockets to
> > flush gets corrupted by concurrent accesses. A quick and small fix is
> > unfortunately not possible, so let us revert this for now.
>
> This is a very useful feature, to be able to use AF_XDP sockets with
> a standard RSS nic configuration.
I completely agree.
> Not all AF_XDP use cases require the absolute highest packet rate.
>
> Can this be addressed with an optional spinlock on the RxQ, only for
> this case?
Yes, or with a MPSC ring implementation.
> If there is no simple enough fix in the short term, do you plan to
> reintroduce this in another form later?
Yuval and I are looking into a solution based around an optional
spinlock since it is easier to pull off than an MPSC ring. The
discussion is on-going on the xdp-newbies list [0], but as soon as we
have a first patch, we will post it here for review and debate.
[0] https://lore.kernel.org/xdp-newbies/8100DBDC-0B7C-49DB-9995-6027F6E63147@radware.com/
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH bpf 0/2] Revert "xsk: support redirect to any socket bound to the same umem"
2024-06-05 7:45 ` Magnus Karlsson
@ 2024-06-05 19:28 ` Willem de Bruijn
0 siblings, 0 replies; 7+ messages in thread
From: Willem de Bruijn @ 2024-06-05 19:28 UTC (permalink / raw)
To: Magnus Karlsson, Willem de Bruijn
Cc: magnus.karlsson, bjorn, ast, daniel, netdev, maciej.fijalkowski,
bpf, YuvalE
Magnus Karlsson wrote:
> On Wed, 5 Jun 2024 at 01:03, Willem de Bruijn
> <willemdebruijn.kernel@gmail.com> wrote:
> >
> > Magnus Karlsson wrote:
> > > Revert "xsk: support redirect to any socket bound to the same umem"
> > >
> > > This patch introduced a potential kernel crash when multiple napi
> > > instances redirect to the same AF_XDP socket. By removing the
> > > queue_index check, it is possible for multiple napi instances to
> > > access the Rx ring at the same time, which will result in a corrupted
> > > ring state which can lead to a crash when flushing the rings in
> > > __xsk_flush(). This can happen when the linked list of sockets to
> > > flush gets corrupted by concurrent accesses. A quick and small fix is
> > > unfortunately not possible, so let us revert this for now.
> >
> > This is a very useful feature, to be able to use AF_XDP sockets with
> > a standard RSS nic configuration.
>
> I completely agree.
>
> > Not all AF_XDP use cases require the absolute highest packet rate.
> >
> > Can this be addressed with an optional spinlock on the RxQ, only for
> > this case?
>
> Yes, or with a MPSC ring implementation.
>
> > If there is no simple enough fix in the short term, do you plan to
> > reintroduce this in another form later?
>
> Yuval and I are looking into a solution based around an optional
> spinlock since it is easier to pull off than an MPSC ring. The
> discussion is on-going on the xdp-newbies list [0], but as soon as we
> have a first patch, we will post it here for review and debate.
>
> [0] https://lore.kernel.org/xdp-newbies/8100DBDC-0B7C-49DB-9995-6027F6E63147@radware.com/
Glad to hear that it's intended to be supported, and even being worked
on, thanks! I'll follow the conversation there.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH bpf 0/2] Revert "xsk: support redirect to any socket bound to the same umem"
2024-06-04 12:29 [PATCH bpf 0/2] Revert "xsk: support redirect to any socket bound to the same umem" Magnus Karlsson
` (2 preceding siblings ...)
2024-06-04 23:03 ` [PATCH bpf 0/2] Revert "xsk: support " Willem de Bruijn
@ 2024-06-05 7:50 ` patchwork-bot+netdevbpf
3 siblings, 0 replies; 7+ messages in thread
From: patchwork-bot+netdevbpf @ 2024-06-05 7:50 UTC (permalink / raw)
To: Magnus Karlsson
Cc: magnus.karlsson, bjorn, ast, daniel, netdev, maciej.fijalkowski,
bpf, YuvalE
Hello:
This series was applied to bpf/bpf.git (master)
by Daniel Borkmann <daniel@iogearbox.net>:
On Tue, 4 Jun 2024 14:29:24 +0200 you wrote:
> Revert "xsk: support redirect to any socket bound to the same umem"
>
> This patch introduced a potential kernel crash when multiple napi
> instances redirect to the same AF_XDP socket. By removing the
> queue_index check, it is possible for multiple napi instances to
> access the Rx ring at the same time, which will result in a corrupted
> ring state which can lead to a crash when flushing the rings in
> __xsk_flush(). This can happen when the linked list of sockets to
> flush gets corrupted by concurrent accesses. A quick and small fix is
> unfortunately not possible, so let us revert this for now.
>
> [...]
Here is the summary with links:
- [bpf,1/2] Revert "xsk: support redirect to any socket bound to the same umem"
https://git.kernel.org/bpf/bpf/c/7fcf26b315bb
- [bpf,2/2] Revert "xsk: document ability to redirect to any socket bound to the same umem"
https://git.kernel.org/bpf/bpf/c/03e38d315f3c
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 7+ messages in thread