Re: [PATCH 6/7] samples/devsec: Introduce a "Device Security TSM" sample driver

[Jason Gunthorpe <jgg@nvidia.com>]

On Thu, Aug 28, 2025 at 02:38:14PM -0700, dan.j.williams@intel.com wrote:
> > device_cc_probe() doesn't save anything, doesn't this just get into an
> > endless loop of EPROBE_DEFER? Usually the kernel will retry these
> > things during booting?
> 
> Hmm, no, deferred probing retriggers after a one-time boot timeout
> (extended by driver registration events) and after any device
> successfully completes probe.

So it is not "endless" but it is also not "single probe then wait till
accept". I'm not keen on using this mechanism, I think the things
people want to do in the T=0 mode are going to be time consuming and
repeatedly doing that time consuming step is not a good idea.

How about this instead:

1) Drivers compiled into the kernel are "safe" and can freely bind
   at will during early boot

2) The first thing the initrd does is set
   /sys/bus/*/drivers_autoprobe to false.
   This stops all kernel driver autobinding.

   Maybe we also need a small kernel change to allow userspace to make
   drivers_autoprobe false for all future busses too.

3) Userspace then evaluates what devices are present, checks its
   policy, loads modules and issues /sys/.../bind operations.

   We need to close the general security gap I gave earlier, userspace
   policy should be able to implement the statement:

     mlx5 is allowed to bind to a RUN device after measuring and
     verifying it, and never otherwise.

   a) For non-TDISP devices userspace checks if a driver is "trusted"
      before binding it, a fancier CC only deny list stored in the
      initrd.
   b) For TDISP devices userspace runs through the
      prepare/measure/lock/run sequence then binds the final
      driver.
   c) Something something RAS driver restart

   Basically userspace policy is entirely in control if a device is
   "accepted" by the ccVM or not. The kernel won't auto bind
   a driver to a physical device. It would be driven off of
   uevents, I guess through new CC focused features in udev.

   I think the needed kernel support is already here, the main gap I
   see is that the modules.alias does not include the driver names, it
   just has the module names. We ran into this with vfio (see below)
   so it would be nice to fix, though it can be worked around like
   VFIO did by making the driver name == module name.

4) Userspace sequences the special "prepare" pre-T=0 drivers, perhaps
   discovered through modinfo matching similar to VFIO:

   $ grep vfio /lib/modules/`uname -r`/modules.alias
   alias vfio_pci:v000015B3d0000101Esv*sd*bc*sc*i* mlx5_vfio_pci
         ^^^^^^
   PCI driver but special for VFIO usage. So I imagine a
   ccprepare_pci:... driver variation.

   Userspace can inspect the modules.alias, find if the device's
   modalias has a ccprepare_pci: match and if so it will bind/unbind
   that driver before going to locked/run. When it reaches run it will
   find the pci: match and bind that driver which is the operating
   driver.

   Policy if the ccprepare device should even be permitted is also
   controlled by userspace.

   Userspace sequences all of this based on its policy to accept a
   device and push it to RUN, not the kernel, again probably
   through some new CC features in udev.

   The kernel side of this is a commit like cc6711b0bf36 ("PCI / VFIO:
   Add 'override_only' support for VFIO PCI sub system")

This is much less kernel change and gives the big thing CC needs -
driver binding policy decisions in userspace.

> > As we discussed in the prior chain we need to have a policy decision
> > before auto-binding drivers at all in a CC environment, I don't see
> > that in the code though the cover letter talked about it??
> 
> The aim was for the "'struct device' has an acceptance flag" discussion
> to settle before starting a "device-core policy for unaccepted devices"
> discussion. I am ok to put more logs on the fire if there is an appetite
> for that.

Sure, I think you shold drop this patch from this series and have this
series focus only on creating an accepted struct device environment
that a driver can bind to and operate. This is a long journey already,
once this basic support is landed we still need to do all the arch
support to enable DMA/IOMMU/etc as many followup series.

The questions about when and what drivers are probed can be left to a
different series, at this point it will be usable for development but
not secure like it should be.

The device_cc_probe() type issue should be solved in yet another
series, IMHO, and that should come with a really strong justification
why the kernel needs to do anything at all, vs just rely on userspace
as I outline above.

> I was hoping to put the onus of that on the vendors that think they need
> this Enlightened driver path. The path of least resistance for device
> vendors is design the hardware so that it can be locked without needing
> a driver to take any configuration action ahead of time. Otherwise,
> explain to users that they need to adjust/replace the eventual udev
> sysfs script that does:

So if we already imagine changing udev, lets imagine the above
instead?

> > This also disables BME, we talked about that a lot, the commit
> > messages didn't seem to describe what solution was settled on here?
> 
> Your proposal to put 100% of the onus of not clobbering the RUN state of
> the device via configuration writes to standard registers on the VMM has
> grown on me. Make VMM responsible for trapping and declining requests to
> clear BME and MSE while the device is in LOCKED or RUN state.

OK, worth explaining :)

Jason