Re: [PATCH V2 mlx5-next 12/14] vfio/mlx5: Implement vfio_pci driver for mlx5 devices

[Jason Gunthorpe <jgg@nvidia.com>]

On Wed, Nov 03, 2021 at 09:44:09AM -0600, Alex Williamson wrote:

> In one email I read that QEMU clearly should not be performing SET_IRQS
> while the device is _RESUMING (which it does) and we need to require an
> interim state before the device becomes _RUNNING to poke at the device
> (which QEMU doesn't do and the uAPI doesn't require), and the next I
> read that we should proceed with some useful quanta of work despite
> that we clearly don't intend to retain much of the protocol of the
> current uAPI long term...

mlx5 implements the protocol as is today, in a way that is compatible
with today's qemu. Qemu has various problems like the P2P issue we
talked about, but it is something working.

If you want to do a full re-review of the protocol and make changes,
then fine, let's do that, but everything should be on the table, and
changing qemu shouldn't be a blocker.

In one email you are are saying we need to document and decide things
as a pre-condition to move the driver forward, and then in the next
email you say whatever qemu does is the specification, and can't
change it.

Part of this messy discussion is my fault as I've been a little
unclear in mixing my "community view" of how the protocol should be
designed to maximize future HW support and then switching to topics
that have direct relevance to mlx5 itself.

I want to see devices like hns be supportable and, from experience,
I'm very skeptical about placing HW design restrictions into a
uAPI. So I don't like those things.

However, mlx5's HW is robust and more functional than hns, and doesn't
care which way things are decided.

> Too much is in flux and we're only getting breadcrumbs of the
> changes to come.

We have no intention to go in and change the uapi after merging beyond
solving the P2P issue.

Since we now have confirmation that hns cannot do P2P I see no issue
to keep the current design as the non-p2p baseline that hns will
implement and the P2P upgrade should be designed separately.

> It's becoming more evident that we're likely to sufficiently modify
> the uAPI to the point where I'd probably suggest a new "v2" subtype
> for the region.

I don't think this is evident. It is really your/community choice what
to do in VFIO.

If vfio sticks with the uAPI "as is" then it places additional
requirements on future HW designs.

If you want to relax these requirements before stabilizing the uAPI,
then we need to make those changes now.

It is your decision. I don't know of any upcoming HW designs that have
a problem with any of the choices.

Thanks,
Jason