[PATCH net-next 0/3] fbnic: Support larger io_uring zcrx buffers

["Björn Töpel" <bjorn@kernel.org>]

Hi!

Fbnic programs receive buffers through BDQs. The hardware consumes
BDQs as 4 KiB fragments, and receive completions report the consumed
buffer by returning the BDQ buffer ID in the RCD.

The driver currently derives the BDQ fragment layout from PAGE_SIZE.
That works while HPQ and PPQ use the same allocation size, but
io_uring zcrx can provide larger receive buffers through rx_buf_len.
For zcrx, the PPQ page pool allocation size and the PPQ BDQ fragment
geometry need to match the requested buffer size, without changing
HPQ.
                                                                                                                                                                                                                                                                                                                                                  
Make the BDQ fragment geometry per ring, then use the rendered RX
queue rx_page_size when creating the PPQ page pool. The NIC still
consumes the PPQ as 4 KiB fragments; a larger zcrx buffer is
represented as multiple BDQ fragments belonging to one net_iov.

Fbnic also validates rx_page_size against its own queue geometry. The
core validates the zcrx request and checks that the imported memory
can be represented as rx_buf_len-sized DMA chunks, but fbnic still
needs to make sure the PPQ retains usable depth after expanding one
software buffer into multiple 4 KiB hardware fragments.

The normal open path uses the rendered per-queue rx_page_size as well.
This preserves a memory-provider binding made while the netdev is
down, instead of falling back to the default PPQ geometry on open.

The selftest change adds an optional iou-zcrx helper check for manual
driver testing. It is not wired into the generic large-chunk test
because different drivers may legitimately return different CQE
boundaries.

Manual testing
==============

The fbnic QEMU model and firmware setup are described here:

  https://lore.kernel.org/netdev/20260309113852.2c654de5@kernel.org/

I use something like:

  KERNEL=/path/to/linux                                                                                                                                                                                                                                                                                                                           
  DISK=/path/to/fedora-qemu.raw                                                                                                                                                                                                                                                                                                                   
  OVMF_CODE=/path/to/OVMF_CODE.fd                                                                                                                                                                                                                                                                                                                 
  OVMF_VARS=/path/to/OVMF_VARS.fd                                                                                                                                                                                                                                                                                                                 
  MODS=/tmp/fbnic-modules                                                                                                                                                                                                                                                                                                                         
  QEMU=/path/to/fbnic-qemu/build/qemu-system-x86_64                                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                                                                                  
  $QEMU \                                                                                                                                                                                                                                                                                                                                         
    -machine type=q35,accel=kvm \                                                                                                                                                                                                                                                                                                                 
    -drive if=pflash,format=raw,unit=0,file=$OVMF_CODE,readonly=on \                                                                                                                                                                                                                                                                              
    -drive if=pflash,format=raw,unit=1,file=$OVMF_VARS \                                                                                                                                                                                                                                                                                          
    -smp 16 -m 16G \                                                                                                                                                                                                                                                                                                                              
    -object memory-backend-memfd,id=mem,size=16G,share=on \                                                                                                                                                                                                                                                                                       
    -numa node,memdev=mem \                                                                                                                                                                                                                                                                                                                       
    -kernel $KERNEL/arch/x86/boot/bzImage \                                                                                                                                                                                                                                                                                                       
    -append "root=/dev/vda2 rw console=ttyS0 earlycon" \                                                                                                                                                                                                                                                                                          
    -drive file=$DISK,format=raw,if=none,id=drive0 \                                                                                                                                                                                                                                                                                              
    -device virtio-blk-pci,drive=drive0 \                                                                                                                                                                                                                                                                                                         
    -no-user-config -nodefaults -nographic \                                                                                                                                                                                                                                                                                                      
    -virtfs local,path=$MODS/lib/modules,mount_tag=modules,security_model=none,readonly=on \                                                                                                                                                                                                                                                      
    -virtfs local,path=$KERNEL,mount_tag=hostshare,security_model=none,readonly=on \                                                                                                                                                                                                                                                              
    -netdev user,id=hostnet0,hostfwd=tcp::9999-:9999 \                                                                                                                                                                                                                                                                                            
    -netdev hubport,id=hub_uplink,hubid=0,netdev=hostnet0 \                                                                                                                                                                                                                                                                                       
    -device virtio-net-pci,netdev=n1 \                                                                                                                                                                                                                                                                                                            
    -netdev hubport,id=n1,hubid=0 \                                                                                                                                                                                                                                                                                                               
    -device pcie-root-port,id=pcie.1,bus=pcie.0,chassis=1 \                                                                                                                                                                                                                                                                                       
    -device fbnic,bus=pcie.1,id=fbnic.1,mac=00:de:ad:be:ef:01,netdev=n2,rbt=skt.0,bar4=ctrl.1 \                                                                                                                                                                                                                                                   
    -netdev hubport,id=n2,hubid=0 \                                                                                                                                                                                                                                                                                                               
    -chardev socket,id=ctrl.1,path=/tmp/fbnic-ctrl-skt \                                                                                                                                                                                                                                                                                          
    -netdev socket,id=skt.0,connect=localhost:9000 \                                                                                                                                                                                                                                                                                              
    -serial mon:stdio                                                                                                                                                                                                                                                                                                                             

Here you'll get a fbnic device, host port forwarding for TCP port
9999, and a 9p mount for the kernel tree and modules.

Inside the guest:

  mount -t 9p -o trans=virtio,version=9p2000.L hostshare /host

  cd /host/tools/testing/selftests/drivers/net/hw

  ethtool -L enp1s0 combined 2
  ethtool -G enp1s0 tcp-data-split on hds-thresh 0 rx 64
  ethtool -X enp1s0 equal 1
  ethtool -N enp1s0 flow-type tcp4 dst-ip 10.0.2.15 dst-port 9999 action 1

  echo 64 > /proc/sys/vm/nr_hugepages
  ./iou-zcrx -s -i enp1s0 -p 9999 -q 1 -x 2

On the host:

  cd /path/to/linux/tools/testing/selftests/drivers/net/hw
  ./iou-zcrx -c -h 127.0.0.1 -p 9999 -l 12840

For fbnic-specific manual checking that traffic reaches the second 4
KiB fragment of an 8 KiB zcrx buffer, run the receiver with:

  ./iou-zcrx -s -i enp1s0 -p 9999 -q 1 -x 2 -F 4096


Björn Töpel (3):
  fbnic: Track BDQ fragment geometry per ring
  fbnic: Support larger zcrx receive buffers
  selftests: drv-net: Add zcrx payload offset check

 drivers/net/ethernet/meta/fbnic/fbnic_csr.h   |  29 +--
 .../net/ethernet/meta/fbnic/fbnic_debugfs.c   |   5 +-
 drivers/net/ethernet/meta/fbnic/fbnic_txrx.c  | 168 ++++++++++++++----
 drivers/net/ethernet/meta/fbnic/fbnic_txrx.h  |   6 +
 .../selftests/drivers/net/hw/iou-zcrx.c       |  28 ++-
 5 files changed, 176 insertions(+), 60 deletions(-)


base-commit: 1a1f055318d82e64485a6ff8420e5f70b4267998
--
2.53.0