Under which circumstances does `xsk_ring_prod__reserve` return 0?

Under which circumstances does `xsk_ring_prod__reserve` return 0?

[Gaul, Maximilian]

Hello,

I have this system which handles multiple multicast-streams at once where the user can choose the amount of RX-Queues of the NIC he wants to utilize.

I am testing the scenario where only one RX-Queue is used. Thus, shared umem is used.

At the time a user decides to receive the first multicast-stream, a program ("AF-XDP") is launched by another program ("handler") which handles the user inputs.
This AF-XDP program receives information about a Linux System V message queue through command line arguments.
The first thing the AF-XDP program does is to map a shared memory segment and configuring a umem with it.
After the program launched successfully, the handler sends a message to said queue telling the AF-XDP program to create a new XSK and place its file descriptor into a certain index in the XDP map.
The XSK runs in a separate thread checking every millisecond for packets and sending information about those packet addresses in umem to the handler-program.

To this point, everything works fine. But as soon as the user decides to receive another multicast-stream on the same RX-Queue, `xsk_ring_prod__reserve` suddenly starts to return 0 (e.g. it wasn't able to reserve any packets) even if I call this function a hundred times.
So of course to create another XSK, I have to call `xsk_socket__create` which also takes the umem as an argument.
As this system is running in parallel I was wondering if this problem can occur because suddenly another XSK is registered at the same time as the first XSK tries to register umem frames for its packets?

Best regards

Max

Re: Under which circumstances does `xsk_ring_prod__reserve` return 0?

[Magnus Karlsson]

On Thu, May 14, 2020 at 2:05 PM Gaul, Maximilian <maximilian.gaul@hm.edu> wrote:
>
> Hello,
>
> I have this system which handles multiple multicast-streams at once where the user can choose the amount of RX-Queues of the NIC he wants to utilize.
>
> I am testing the scenario where only one RX-Queue is used. Thus, shared umem is used.
>
> At the time a user decides to receive the first multicast-stream, a program ("AF-XDP") is launched by another program ("handler") which handles the user inputs.
> This AF-XDP program receives information about a Linux System V message queue through command line arguments.
> The first thing the AF-XDP program does is to map a shared memory segment and configuring a umem with it.
> After the program launched successfully, the handler sends a message to said queue telling the AF-XDP program to create a new XSK and place its file descriptor into a certain index in the XDP map.
> The XSK runs in a separate thread checking every millisecond for packets and sending information about those packet addresses in umem to the handler-program.
>
> To this point, everything works fine. But as soon as the user decides to receive another multicast-stream on the same RX-Queue, `xsk_ring_prod__reserve` suddenly starts to return 0 (e.g. it wasn't able to reserve any packets) even if I call this function a hundred times.
> So of course to create another XSK, I have to call `xsk_socket__create` which also takes the umem as an argument.
> As this system is running in parallel I was wondering if this problem can occur because suddenly another XSK is registered at the same time as the first XSK tries to register umem frames for its packets?

Do not fully understand your set up, but here are some initial
observations and questions. When you use the XDP_SHARED_UMEM mode you
will end up with one unique set of fill and completion rings tied to
the umem you are sharing, but one set of Rx and Tx (if you use the
latter) rings per socket. So you need to serialize accesses to the
fill and completion rings. Are you doing that? You also need an XDP
program to route your traffic between the two sockets. Do you have
that? So everything works up until you create the second socket?

/Magnus

> Best regards
>
> Max