Re: [PATCH 6/7] samples/devsec: Introduce a "Device Security TSM" sample driver

[<dan.j.williams@intel.com>]

Jason Gunthorpe wrote:
> On Tue, Aug 26, 2025 at 08:52:58PM -0700, Dan Williams wrote:
> > +static int devsec_pci_probe(struct pci_dev *pdev,
> > +			    const struct pci_device_id *id)
> > +{
> > +	void __iomem *base;
> > +	int rc;
> > +
> > +	rc = pcim_enable_device(pdev);
> > +	if (rc)
> > +		return dev_err_probe(&pdev->dev, rc, "enable failed\n");
> > +
> > +	base = pcim_iomap_region(pdev, 0, KBUILD_MODNAME);
> > +	if (IS_ERR(base))
> > +		return dev_err_probe(&pdev->dev, PTR_ERR(base),
> > +				     "iomap failed\n");
> > +
> > +	rc = device_cc_probe(&pdev->dev);
> > +	if (rc)
> > +		return rc;
> 
> I really don't understand what the proposal is here?

tl;dr I will include an end-to-end flow document in Documentation in the
next posting.

> 
> device_cc_probe() doesn't save anything, doesn't this just get into an
> endless loop of EPROBE_DEFER? Usually the kernel will retry these
> things during booting?

Hmm, no, deferred probing retriggers after a one-time boot timeout
(extended by driver registration events) and after any device
successfully completes probe.

1/ TVM policy decides it is committed to operate $device in confidential
   mode.

2/ Enlightened driver learns about that policy "somehow" (I am open to
   this policy being conveyed via driver-core standard mechanism, but for
   now assume the driver learns the policy by driver specific means).

3/ Enlightened driver uses device_cc_probe() to exit its probe routine
   at a known point where the device's configuration is ready to be
   locked, or otherwise falls back to shared operation if confidential
   operation is not requested. This is different than standard device
   teardown path which may "unconfigure" the device and scuttle
   acceptance.

4/ Whenever TVM is ready to measure and accept the device it triggers
   manual bind. Meanwhile deferred probing has probably gone into "wait
   for userspace to manually kick this device" mode by this point.

Unenlightened drivers skip all of this and just assume that the device
arrives in a "ready-to-lock" configuration.

> How does userspace accept through the sysfs retrigger probing?

Yes.

> As we discussed in the prior chain we need to have a policy decision
> before auto-binding drivers at all in a CC environment, I don't see
> that in the code though the cover letter talked about it??

The aim was for the "'struct device' has an acceptance flag" discussion
to settle before starting a "device-core policy for unaccepted devices"
discussion. I am ok to put more logs on the fire if there is an appetite
for that.

> How does the kernel/userspace tell the difference between drivers that
> want this early binding and those that don't?

I was hoping to put the onus of that on the vendors that think they need
this Enlightened driver path. The path of least resistance for device
vendors is design the hardware so that it can be locked without needing
a driver to take any configuration action ahead of time. Otherwise,
explain to users that they need to adjust/replace the eventual udev
sysfs script that does:

   lock
   accept
   bind

...instead needs to do:

   bind (defer)
   lock
   accept
   bind

Now, there is a debugfs method to learn the probe deferral reason, but
there is no requirement for debugfs to be mounted, and it turns out that
probe deferral reason is only updated if the device is autoprobed. If
the first time a driver sees a device is via explicit bind debugfs does
not convey the deferral reason.

> Can you write out the whole flow from a userspace perspective in one
> of the commit messages?

I will do that in v2.

> This also disables BME, we talked about that a lot, the commit
> messages didn't seem to describe what solution was settled on here?

Your proposal to put 100% of the onus of not clobbering the RUN state of
the device via configuration writes to standard registers on the VMM has
grown on me. Make VMM responsible for trapping and declining requests to
clear BME and MSE while the device is in LOCKED or RUN state.

Enlightened drivers could skip clearing BME + MSE when locked, but
unenlightened drivers should assume either the VMM traps the
configuration request or the TVM must re-lock re-accept when the VMM
fails to meet that requirement.