Re: [RFC PATCH fwctl 0/5] pds_fwctl: fwctl for AMD/Pensando core devices

[Jason Gunthorpe <jgg@nvidia.com>]

On Wed, Feb 12, 2025 at 02:40:45PM +0100, Andrew Lunn wrote:

> Isn't this even generic for any sort of SR-IOV? Wouldn't you need the
> same sort of operation for a GPU, or anything with a pool of resources
> which can be mapped to VFs?

We've been calling this device profiling in the vfio discussions,
generally yes the general idea of profiling is common, but the actual
detail of the profile is very device specific.

In vfio land we think fwctl is a good choice here. We already have
things like libvirt and Kubernetes that have generic userspace plugin
mechanims and an existing mature ecosystem for device profiling built
up around that. All sophisticated devices have their own plugins
because they have unique capabilities. It seems to be working well in
that world.

From a kernel perspective fwctl is alot better than some of what has
been tried so far, ie various vfio drivers having questionable
device-specific sysfs, and then a libvirt/k8s plugin anyhow.

Even the better stuff like mlx5's devlink is only partially capable
and the existing mlx plugins still has to do stuff beyond that.

The kernel isn't the only point, or necessarily the most appropriate
point, to insert a consolidation layer in the stack. We don't want to
move chunks of existing k8s operator code into the kernel, for
instance.

Again, this isn't an exclusive thing, that fwctl can profile a PCI
function doesn't in any way exclude other kernel options, like
devlink, from doing that too.

Jason