Re: [MAINTAINERS SUMMIT] Device Passthrough Considered Harmful?

Jonathan Cameron
2024-07-11T11:00:36Z

On Wed, 10 Jul 2024 14:22:38 +0100
Jonathan Cameron <Jonathan.Cameron@Huawei.com> wrote:

> On Tue, 9 Jul 2024 15:15:13 -0700
> Dan Williams <dan.j.williams@intel.com> wrote:
>
> > James Bottomley wrote:
> > > > The upstream discussion has yielded the full spectrum of positions on
> > > > device specific functionality, and it is a topic that needs cross-
> > > > kernel consensus as hardware increasingly spans cross-subsystem
> > > > concerns. Please consider it for a Maintainers Summit discussion.
> > >
> > > I'm with Greg on this ... can you point to some of the contrary
> > > positions?
> >
> > This thread has that discussion:
> >
> > http://lore.kernel.org/0-v1-9912f1a11620+2a-fwctl_jgg@nvidia.com
> >
> > I do not want to speak for others on the saliency of their points, all I
> > can say is that the contrary positions have so far not moved me to drop
> > consideration of fwctl for CXL.
>
> I was resisting rat holing. Oh well...

To throw another 'fun' one in there. For anything integrated with the host
there is a proposal to provide a MCTP via PCC (ACPI described mailbox). [1]
I don't think it makes sense to rule that out as it's logically no
different from MCTP in general (e.g. a host controller for PCI VDM, or
I2C etc)

Anyone who has a suitable firmware can do whatever they like with that
and the interfaces is exposed directly to userspace. Adam, perhaps you can
describe your use case a little? Is it applicable to general server distros?

We might suggest distributions don't enable MCTP but does that
actually get us anywhere? Anyhow, I suspect there are other similar routes, but
this one happens to be under review at the moment.

[1] https://lore.kernel.org/all/20240702225845.322234-1-admiyo@os.amperecomputing.com/

>
> For a 'subset' of CXL. There are a wide range of controls that are highly
> destructive, potentially to other hosts (simplest one is a command that
> will surprise remove someone else's memory). For those I'm not sure
> fwctl gets us anywhere - but we still need a solution (Subject to
> config gates etc as typically this is BMCs not hosts).
> Maybe fwctl eventually ends up with levels of 'safety' (beyond the
> current read vs write vs write_full, or maybe those are enough).
>
> Complexities such as message tunneling to multiple components are also
> going to be fun, but we want the non destructive bits of those to work
> as part of the safe set, so we can get telemetry from downstream devices.
>
> Good to cover the debug and telemetry usecase, but it still leaves us with
> gaping holes were we need to solve the permissions problem, perhaps that
> is layered on top of fwctl, perhaps something else is needed.
>
> So if fwctl is adopted, I do want the means to use it for the highly
> destructive stuff as well! Maybe that's a future discussion.
>
>
> >
> > Where CXL has a Command Effects Log that is a reasonable protocol for
> > making decisions about opaque command codes, and that CXL already has a
> > few years of experience with the commands that *do* need a Linux-command
> > wrapper.
>
> Worth asking if this will incorporate unknown but not vendor defined
> commands. There is a long tail of stuff in the spec we haven't caught up
> with yet. Or you thinking keep this for the strictly vendor defined stuff?
>
> >
> > Some open questions from that thread are: what does it mean for the fate
> > of a proposal if one subsystem Acks the ABI and another Naks it for a
> > device that crosses subsystem functionality? Would a cynical hardware
> > response just lead to plumbing an NVME admin queue, or CXL mailbox to
> > get device-specific commands past another subsystem's objection?
> >
> > My reconsideration of the "debug-build only" policy for CXL
> > device-specific commands was influenced by a conversation with a distro
> > developer where they asserted, paraphrasing: "at what point is a device
> > vendor incentivized to ship an out-of-tree module just to restore their
> > passthrough functionality?. At that point upstream has lost out on
> > collaboration and distro kernel ABI has gained another out-of-tree
> > consumer."
> >
> > So the tension is healthy, but it has diminishing returns past a certain
> > point.
> >
>
>
>