Re: [PATCH 00/15] Device Evidence and Trust for PCI Security Protocol (TDISP)

["Dan Williams (nvidia)" <djbw@kernel.org>]

Jason Gunthorpe wrote:
> On Tue, Jul 07, 2026 at 05:12:44PM -0700, Dan Williams (nvidia) wrote:
>  
> > 1/ We previously discussed a use case to operate a device's private MMIO
> > while not allowing access to private memory (software encrypted NVME
> > with private MMIO [1]). Many of the following comments are based on
> > preserving this assumption so you can save some reading if we agree that
> > use case can be abandoned.
> 
> force_dma_unencrypted() does not *prevent* device access to private
> memory and provides no security properties on its own. It's only
> purpose is to inform the DMA API what the HW restrictions are for
> doing DMA.

Right, to be clear, this mode's security properties come from never
asking the TSM to enable private DMA while the device is in RUN.

force_dma_unencrypted() has no security properties.

> The use case I had in mind relied on the vIOMMU to police access to
> the private memory. I think if you do T=1 and ADVERSARY you must have
> a vIOMMU or it is an illogical configuration.

Ok, the software encryption aspect of the suggestion was what threw me.
So we are saying, "no need, outside of defense in depth or data at rest
concerns, to include software encryption on top of an NVME device being
operated adversarially with T=1."

> So, IMHO there is no use case for a T=1 device to only use shared
> memory, I would abandon that combination.
> 
> > 2/ The confirmation of the trust level and the enabling of DMA are
> > separated in time from setting the trust level and entering the RUN
> > state.
> > 
> > All the archs separate the RUN step from the ENABLE DMA step, and the
> > implementation separates those steps in time.
> 
> Yes, until the device is fully accepted the DMA has to be kept off. If
> you have to measure in the RUN state then we must have a second step
> after measurement completes, however we are measuring in LOCKED right?

Yes, measure while in LOCKED, but nothing strictly prevents taking
measurements after RUN.

> > echo tsm0 > $pdev/tsm/lock
> > cat $nonce | device-evidence dump $pdev
> > device-evidence validate $pdev $generation
> 
> So the device is in LOCKED here, DMA remains off both because of
> LOCKED and becuase ENABLE DMA has not been done.
> 
> > echo 1 > $pdev/tsm/accept
> 
> And now it is RUN. So I don't see the issue with enabling DMA at the
> same time as gonig to RUN? (though defering it to driver probe would
> be a very nice touch as well)

It requires "accept" to consider the trust level. E.g. what does it mean
to do something like change requirements after accept?

echo full $pdev/trust
echo 1 > $pdev/tsm/accept
echo adversary > $pdev/trust

In that scenario this now adversary device may have been allowed to
operate without an enforcing vIOMMU, and needs to unlock the device to
correct that.

Delaying enable DMA means the implementation can say, "all trust and
security privileges are reconciled at device driver bind, and revoked at
unbind".

> With this API you can't set the devices to RUN and *then* collect the
> evidence, is that restriction OK?

The Enable DMA step requires evidence in the end. So it is more
implementation ergonomics driving whether "accept" includes Enable DMA
or defers to driver bind.

> > echo full > $pdev/trust
> > echo $pdev > $driver/bind
> 
> Then we can set the trust and bind the driver, knowing that the T=1
> security property is in place.

Or, "accept" success tells you the full T=1 property is required for
"bind" to succeed.

> > The configuration window where $pdev/trust and $pdev/accept can be
> > dynamically changing should not be changing the force_dma_unencrypyted()
> > result if only because that value will mismatch the hardware state until
> > the next ->dma_configure() event.
> 
> T=x cannot be changed while a driver is bound, we need to block
> that. Changing it will mess up all DMA and the DMA API won't work
> anymore.
> 
> Thus force_dma_unencrypted() can only change when a driver is
> unbound.
> 
> If a driver is unbound nothing ever reads force_dma_unencrypted() so
> it is safe to change it, thus we can just synchronize T=1 and
> force_dma_unencrypted() whenever T changes.
> 
> The error case has to wait for the driver to unbind if userspace wants
> to go from ERROR->UNLOCKED. IMHO for RAS we are going to need a flow
> to logically go from ERROR->RUN, probably with the 'same device
> acceptance' I suggested earlier. In this case the DMAs will be halted,
> the devices gets back to T=1 and RUN, then the DMAs are turned on
> again. The driver remains in T=1 and force_dma_unencrypted() is always
> false.

Ideally this recovery does not need userspace round trips, but for that
the kernel needs SPDM transcript parsing. The measurement digest that
CCA provides is over the full previous transcript. A fresh nonce to
validate same device leads to the kernel need to validate the signature
with a cached key and compare the extracted measurement payload from the
previous transcript.

> > There are also buses and paravisors that may know that private-DMA is
> > enabled for a device by construction. In that case it is also a "trusted
> > to access" signal, and not a "required to access" signal.
> 
> In this case they wire force_dma_unencrypted()=false.

Yeah, I just need a scheme where modular bus providers do not end up
compromising the private / unexported method of changing the flag that
force_dma_unencrypted() consumes.

> > My answer, given the concerns of drivers dangerously flipping the
> > force_dma_unencrypted() result at runtime was to place it in 'struct
> > device_private' 
> 
> Yes, this seems good idea in general
> 
> > Otherwise it is confusing when 2 devices are at FULL trust, but
> > one has private memory access and the other does not. One is FULL the
> > other is FULL+.
> 
> It is confusing when phrased like this, but this is also why I was
> suggesting below that a pure trust level may not be sufficient as we
> have quite a cross product of scenarios.
> 
> If the user wants to operate a device without T=1 and without
> ADVERSARY then that's goofy, but technically the kernel doesn't really
> need to care? It can do it.
> 
> IMHO a big question is how much do we want the kernel to nanny
> userspace by having options to double check its work?
> 
> Either we trust the user to have checked all the policy conditions
> before writing /sys/trust, or we encode some policy conditions and
> tell the kernel so it can double-check prior to probe.
> 
> At least if we omit the double check it can be fairly easy to add in
> later if it really was needed for some time of use reason.

As long as IOMMU presence can be enumerated prior to acceptance, T=1
always includes private DMA, and the T=1 status is tracked independent
of the trust level then yes, the cross product can be avoided.

> > The question is whether these requirements belong on a central device
> > trust mechanism bitmap or should be pushed out to other ABI.
> 
> Yes, for sure, I don't have a good sense what is better.
> 
> > For
> > example, if the presence of the secure IOMMU is enumerated
> > after the device is in the LOCKED state then userspace policy can know
> > what it is getting into without needing to tell the kernel. 
> 
> Yes, I think for almost all of this we could push it out to userspace
> and have knobs all over. Like IOMMU already has a knob to switch to
> DMA mode, we could add an IOMMU specific trust level to set ADVERSARY
> also if we go in this direction.
> 
> But, I also do really like the idea of a central knob that was fairly
> general and simple DISABLE/ADVERSARY/FULL which all sorts of things
> can hang off. Including IOMMU and the drivers themselves that may
> change operation in ADVERSARY mode.

Yes, even if some of the ancillary requirements move to other ABI the
central question of DISABLE/ADVERSARY/FULL stays common.

> That means they need simple general definitions for what they are, and
> I really like this ADVERSARY naming vs trust as it makes it very clear
> what the device disposition intention is.

I will do the "device_adversarial()" conversion throughout.

'struct device' grows some "request" policy for trust and T=1 that can
be changed by modules etc. 'struct device_private' grows the operational
trust level stable under device_lock() and a new flag to reflect T=1.