Re: [RFC PATCH v3 2/4] PCI/doe: Add Data Object Exchange support

[Greg KH <gregkh@linuxfoundation.org>]

On Mon, May 17, 2021 at 09:40:45AM +0100, Jonathan Cameron wrote:
> On Fri, 14 May 2021 11:37:12 -0700
> Dan Williams <dan.j.williams@intel.com> wrote:
> 
> > On Fri, May 14, 2021 at 1:50 AM Jonathan Cameron
> > <Jonathan.Cameron@huawei.com> wrote:
> > [..]
> > > > If it simplifies the kernel implementation to assume single
> > > > kernel-initiator then I think that's more than enough reason to block
> > > > out userspace, and/or provide userspace a method to get into the
> > > > kernel's queue for service.  
> > >
> > > This last suggestion makes sense to me. Let's provide a 'right' way
> > > to access the DOE from user space. I like the idea if it being possible
> > > to run CXL compliance tests from userspace whilst the driver is loaded.  
> > 
> > Ah, and I like your observation that once the kernel provides a
> > "right" way to access DOE then userspace direct-access of DOE is
> > indeed a "you get to keep the pieces" event like any other unwanted
> > userspace config-write.
> > 
> > > Bjorn, given this would be a generic PCI thing, any preference for what
> > > this interface might look like?   /dev/pcidoe[xxxxxx].i with ioctls similar
> > > to those for the BAR based CXL mailboxes?  
> > 
> > (warning, anti-ioctl bias incoming...)
> 
> I feel very similar about ioctls - my immediate thought was to shove this in
> debugfs, but that feels the wrong choice if we are trying to persuade people
> to use it instead of writing code that directly accesses the config space.
> 
> > 
> > Hmm, DOE has an enumeration capability, could the DOE driver use a
> > scheme to have a sysfs bin_attr per discovered object type? This would
> > make it simliar to the pci-vpd sysfs interface.
> 
> We can discover the protocols, but anything beyond that is protocol
> specific.  I don't think there is a enough info available by any standards
> defined method. Also part of the reason to allow a safe userspace interface
> would be to provide a generic interface for vendor protocols and things like
> CXL compliance tests where we will almost certainly never provide a more
> specific kernel interface.
> 
> Whilst sysfs would work for CDAT, some protocols are challenge response rather
> than simple read back and that really doesn't fit well for sysfs model.
> If we get other protocols that are simple data read back, then I would
> advocate giving them a simple sysfs interface much like proposed for CDAT
> as it will always be simpler to use + self describing.
> 
> On a lesser note it might be helpful to provide sysfs attrs for
> what protocols are supported.  The alternative is to let userspace run
> the discovery protocol. Perhaps we can do this as a later phase.
> 
> > 
> > Then the kernel could cache objects like CDAT that don't change
> > outside of some invalidation event.
> 
> It's been a while since I last saw any conversation on sysfs bin_attrs
> but mostly I thought the feeling was pretty strongly against them for anything
> but a few niche usecases.
> 
> Feels to me like it would break most of the usual rules in a way vpd does
> not (IIRC VPD is supposed to be a simple in the sense that if you write a value
> to a writable part, you will read back the same value).
> 
> +CC Greg who is a fount of knowledge in this area (and regularly + correctly
> screams at the ways I try to abuse sysfs :)  Note I don't think Dan was
> suggesting implementing response / request directly, but I think that is
> all we could do given DOE protocols can be vendor specific and the standard
> discovery protocol doesn't let us know the fine grained support (what commands
> within a given protocol).

sysfs binary files are ONLY for pass-through things that go to/from
userspace/hardware without the kernel touching them at all.  Like raw
PCI config descriptors.

challenge/response type stuff, really does still fit the ioctl model, so
that is a viable solution if needed.

thanks,

greg k-h