Re: [PATCH 00/15] Device Evidence and Trust for PCI Security Protocol (TDISP)

[Jason Gunthorpe <jgg@ziepe.ca>]

On Tue, Jul 07, 2026 at 05:12:44PM -0700, Dan Williams (nvidia) wrote:
 
> 1/ We previously discussed a use case to operate a device's private MMIO
> while not allowing access to private memory (software encrypted NVME
> with private MMIO [1]). Many of the following comments are based on
> preserving this assumption so you can save some reading if we agree that
> use case can be abandoned.

force_dma_unencrypted() does not *prevent* device access to private
memory and provides no security properties on its own. It's only
purpose is to inform the DMA API what the HW restrictions are for
doing DMA.

The use case I had in mind relied on the vIOMMU to police access to
the private memory. I think if you do T=1 and ADVERSARY you must have
a vIOMMU or it is an illogical configuration.

So, IMHO there is no use case for a T=1 device to only use shared
memory, I would abandon that combination.

> 2/ The confirmation of the trust level and the enabling of DMA are
> separated in time from setting the trust level and entering the RUN
> state.
> 
> All the archs separate the RUN step from the ENABLE DMA step, and the
> implementation separates those steps in time.

Yes, until the device is fully accepted the DMA has to be kept off. If
you have to measure in the RUN state then we must have a second step
after measurement completes, however we are measuring in LOCKED right?

> echo tsm0 > $pdev/tsm/lock
> cat $nonce | device-evidence dump $pdev
> device-evidence validate $pdev $generation

So the device is in LOCKED here, DMA remains off both because of
LOCKED and becuase ENABLE DMA has not been done.

> echo 1 > $pdev/tsm/accept

And now it is RUN. So I don't see the issue with enabling DMA at the
same time as gonig to RUN? (though defering it to driver probe would
be a very nice touch as well)

With this API you can't set the devices to RUN and *then* collect the
evidence, is that restriction OK?

> echo full > $pdev/trust
> echo $pdev > $driver/bind

Then we can set the trust and bind the driver, knowing that the T=1
security property is in place.

> The configuration window where $pdev/trust and $pdev/accept can be
> dynamically changing should not be changing the force_dma_unencrypyted()
> result if only because that value will mismatch the hardware state until
> the next ->dma_configure() event.

T=x cannot be changed while a driver is bound, we need to block
that. Changing it will mess up all DMA and the DMA API won't work
anymore.

Thus force_dma_unencrypted() can only change when a driver is
unbound.

If a driver is unbound nothing ever reads force_dma_unencrypted() so
it is safe to change it, thus we can just synchronize T=1 and
force_dma_unencrypted() whenever T changes.

The error case has to wait for the driver to unbind if userspace wants
to go from ERROR->UNLOCKED. IMHO for RAS we are going to need a flow
to logically go from ERROR->RUN, probably with the 'same device
acceptance' I suggested earlier. In this case the DMAs will be halted,
the devices gets back to T=1 and RUN, then the DMAs are turned on
again. The driver remains in T=1 and force_dma_unencrypted() is always
false.

> There are also buses and paravisors that may know that private-DMA is
> enabled for a device by construction. In that case it is also a "trusted
> to access" signal, and not a "required to access" signal.

In this case they wire force_dma_unencrypted()=false.
 
> My answer, given the concerns of drivers dangerously flipping the
> force_dma_unencrypted() result at runtime was to place it in 'struct
> device_private' 

Yes, this seems good idea in general

> Otherwise it is confusing when 2 devices are at FULL trust, but
> one has private memory access and the other does not. One is FULL the
> other is FULL+.

It is confusing when phrased like this, but this is also why I was
suggesting below that a pure trust level may not be sufficient as we
have quite a cross product of scenarios.

If the user wants to operate a device without T=1 and without
ADVERSARY then that's goofy, but technically the kernel doesn't really
need to care? It can do it.

IMHO a big question is how much do we want the kernel to nanny
userspace by having options to double check its work?

Either we trust the user to have checked all the policy conditions
before writing /sys/trust, or we encode some policy conditions and
tell the kernel so it can double-check prior to probe.

At least if we omit the double check it can be fairly easy to add in
later if it really was needed for some time of use reason.

> The question is whether these requirements belong on a central device
> trust mechanism bitmap or should be pushed out to other ABI.

Yes, for sure, I don't have a good sense what is better.

> For
> example, if the presence of the secure IOMMU is enumerated
> after the device is in the LOCKED state then userspace policy can know
> what it is getting into without needing to tell the kernel. 

Yes, I think for almost all of this we could push it out to userspace
and have knobs all over. Like IOMMU already has a knob to switch to
DMA mode, we could add an IOMMU specific trust level to set ADVERSARY
also if we go in this direction.

But, I also do really like the idea of a central knob that was fairly
general and simple DISABLE/ADVERSARY/FULL which all sorts of things
can hang off. Including IOMMU and the drivers themselves that may
change operation in ADVERSARY mode.

That means they need simple general definitions for what they are, and
I really like this ADVERSARY naming vs trust as it makes it very clear
what the device disposition intention is.

> Not essential, but it is useful to have the "operate device" intent,
> IOMMU default domain configuration, trust level, and private DMA enable
> all in the same bus operation (->dma_configure). Which is why "accept"
> was demoted to just set RUN state and leave the rest to be finalized
> later.

I agree it is nice, and I think this can work fine, the bus code when
it does the dma configure step can reach out to the TSM and have it
enable DMA.

We also want the IOMMU to change to a default-disabled state, so we
also want its hook in dma configure to switch to non-default too.

It makes alot of sense to me that *all* DMA enable knobs will be
disabled until right before probe starts and then disabled again once
remove completes. TSM, IOMMU, PCI, etc.

Jason