Re: [xdp-cloud] Questions about Offloads and XDP-Hints regarding a Cloud-Provider Use-Case

[Lorenzo Bianconi <lorenzo.bianconi@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 2752 bytes --]

> On 9/28/22 11:07 AM, Jesper Dangaard Brouer wrote:
> > 
> > On 28/09/2022 15.54, Marcus Wichelmann wrote:
> >>
> >> I'm working for a cloud hosting provider and we're working on a new
> >> XDP-based networking stack for our VM-Hosts that uses XDP to
> >> accelerate the connectivity of our qemu/KVM VMs to the outside.
> >>
> > 
> > Welcome to the community! Sounds like an excellent use-case and
> > opportunity for speeding up the RX packets from physical NIC into the
> > VM.  Good to hear someone (again) having this use-case. I've personally
> 
> +1
> 
> > not been focused on this use-case lately, mostly because community
> > members that I was interacting with changed jobs, away from cloud
> > hosting companies. Good to have a user back in this area!
> > 
> > 
> >> For this, we use XDP_REDIRECT to forward packets between the physical
> >> host NIC and the VM tap devices. The main issue we have now is, that
> >> our VM guests have some virtio NIC offloads enabled: rx/tx
> >> checksumming, TSO/GSO, GRO and Scatter-Gather.
> > 
> > Supporting RX-checksumming is part of the plans for XDP-hints, although
> > virtio_net is not part of my initial patchset.
> 
> Lorenzo and I both had versions of a patch to propagate rx csum
> validation to the VM on xdp redirect. I do not recall a version after
> this one:
> 
> https://lore.kernel.org/netdev/cover.1622222367.git.lorenzo@kernel.org/
> 
> and I lost of track of what change is needed for it to go in.

iirc the blocking point was the missing CHECKSUM_COMPLETE support (we support
just CHECKSUM_UNNECESSARY). We can conver it with XDP hw-hints.

Regards,
Lorenzo

> 
> > 
> > XDP-redirect with GRO and Scatter-Gather frames are part of the
> > multi-buff effort (Cc Lorenzo), but currently XDP_REDIRECT with
> > multi-buff is disabled (except for cpumap), because the lack of
> > XDP-feature bits, meaning we cannot determine (in kernel) if receiving
> > net_device supports multi-buff (Cc Kumar).
> > 
> >> Currently, these offloads (especially TSO/GSO) are incompatible with
> >> XDP_REDIRECT and result in packets being dropped. Because disabling
> >> these offloads in all our customer VMs is not a good option, we're
> >> searching for ways to support these offloads with XDP.
> >>
> > 
> > To David Ahern, didn't the kernel recently loosen up on having to
> > disable these offloads for KVM virtio_net?
> 
> not that I am aware. Still need tx offloads disabled.
> 
> 
> This summarizes what I was looking into back in 2020, along with the
> current state of XDP for VM use case:
> 
> https://legacy.netdevconf.info/0x14/pub/slides/24/netdev-0x14-XDP-and-the-cloud.pdf
> 
> source code is still on github too.
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]