Re: [MAINTAINERS SUMMIT] Device Passthrough Considered Harmful?

[Jonathan Cameron <Jonathan.Cameron@Huawei.com>]

On Wed, 10 Jul 2024 14:22:38 +0100
Jonathan Cameron <Jonathan.Cameron@Huawei.com> wrote:

> On Tue, 9 Jul 2024 15:15:13 -0700
> Dan Williams <dan.j.williams@intel.com> wrote:
> 
> > James Bottomley wrote:  
> > > > The upstream discussion has yielded the full spectrum of positions on
> > > > device specific functionality, and it is a topic that needs cross-
> > > > kernel consensus as hardware increasingly spans cross-subsystem
> > > > concerns. Please consider it for a Maintainers Summit discussion.    
> > > 
> > > I'm with Greg on this ... can you point to some of the contrary
> > > positions?    
> > 
> > This thread has that discussion:
> > 
> > http://lore.kernel.org/0-v1-9912f1a11620+2a-fwctl_jgg@nvidia.com
> > 
> > I do not want to speak for others on the saliency of their points, all I
> > can say is that the contrary positions have so far not moved me to drop
> > consideration of fwctl for CXL.  
> 
> I was resisting rat holing. Oh well...

To throw another 'fun' one in there.  For anything integrated with the host
there is a proposal to provide a MCTP via PCC (ACPI described mailbox). [1]
I don't think it makes sense to rule that out as it's logically no
different from MCTP in general (e.g. a host controller for PCI VDM, or
I2C etc)

Anyone who has a suitable firmware can do whatever they like with that
and the interfaces is exposed directly to userspace. Adam, perhaps you can
describe your use case a little?  Is it applicable to general server distros?

We might suggest distributions don't enable MCTP but does that
actually get us anywhere?  Anyhow, I suspect there are other similar routes, but
this one happens to be under review at the moment.

[1] https://lore.kernel.org/all/20240702225845.322234-1-admiyo@os.amperecomputing.com/

> 
> For a 'subset' of CXL.  There are a wide range of controls that are highly
> destructive, potentially to other hosts (simplest one is a command that
> will surprise remove someone else's memory). For those I'm not sure
> fwctl gets us anywhere - but we still need a solution (Subject to
> config gates etc as typically this is BMCs not hosts).
> Maybe fwctl eventually ends up with levels of 'safety' (beyond the
> current read vs write vs write_full, or maybe those are enough).
> 
> Complexities such as message tunneling to multiple components are also
> going to be fun, but we want the non destructive bits of those to work
> as part of the safe set, so we can get telemetry from downstream devices.
> 
> Good to cover the debug and telemetry usecase, but it still leaves us with
> gaping holes were we need to solve the permissions problem, perhaps that
> is layered on top of fwctl, perhaps something else is needed.
> 
> So if fwctl is adopted, I do want the means to use it for the highly
> destructive stuff as well!  Maybe that's a future discussion.
> 
> 
> > 
> > Where CXL has a Command Effects Log that is a reasonable protocol for
> > making decisions about opaque command codes, and that CXL already has a
> > few years of experience with the commands that *do* need a Linux-command
> > wrapper.  
> 
> Worth asking if this will incorporate unknown but not vendor defined
> commands.  There is a long tail of stuff in the spec we haven't caught up
> with yet.  Or you thinking keep this for the strictly vendor defined stuff?
> 
> > 
> > Some open questions from that thread are: what does it mean for the fate
> > of a proposal if one subsystem Acks the ABI and another Naks it for a
> > device that crosses subsystem functionality? Would a cynical hardware
> > response just lead to plumbing an NVME admin queue, or CXL mailbox to
> > get device-specific commands past another subsystem's objection?
> > 
> > My reconsideration of the "debug-build only" policy for CXL
> > device-specific commands was influenced by a conversation with a distro
> > developer where they asserted, paraphrasing: "at what point is a device
> > vendor incentivized to ship an out-of-tree module just to restore their
> > passthrough functionality?. At that point upstream has lost out on
> > collaboration and distro kernel ABI has gained another out-of-tree
> > consumer."
> > 
> > So the tension is healthy, but it has diminishing returns past a certain
> > point.
> >   
> 
> 
>