Re: [RFC PATCH 08/12] vfio/pci: Create host unaccessible dma-buf for private device

[Xu Yilun <yilun.xu@linux.intel.com>]

On Mon, Jan 20, 2025 at 09:25:25AM -0400, Jason Gunthorpe wrote:
> On Mon, Jun 24, 2024 at 03:59:53AM +0800, Xu Yilun wrote:
> > > But it also seems to me that VFIO should be able to support putting
> > > the device into the RUN state
> > 
> > Firstly I think VFIO should support putting device into *LOCKED* state.
> > From LOCKED to RUN, there are many evidence fetching and attestation
> > things that only guest cares. I don't think VFIO needs to opt-in.
> 
> VFIO is not just about running VMs. If someone wants to run DPDK on
> VFIO they should be able to get the device into a RUN state and work
> with secure memory without requiring a KVM. Yes there are many steps
> to this, but we should imagine how it can work.

Interesting Question. I've never thought about native TIO before.

And you are also thinking about VFIO usage in CoCo-VM. So I believe
VFIO could be able to support putting the device into the RUN state,
but no need a uAPI for that, this happens when VFIO works as a TEE
attester.

In different cases, VFIO plays different roles:

1. TEE helper, but itself is out of TEE.
2. TEE attester, it is within the TEE.
3. TEE user, it is within the TEE.

As a TEE helper, it works on a untrusted device and help put the device
in LOCKED state, waiting for attestation. For VM use case, VM acts as the
attester to do attestation and move device into trusted/RUN state (lets
say 'accept'). The attestation and accept could be direct talks between
attester and device (maybe via TSM sysfs node), because from
LOCKED -> RUN VFIO doesn't change its way of handling device so seems
no need to introduce extra uAPIs and complexity just for passing the
talks. That's my expectation of VFIO's responsibility as a TEE
helper - serve until LOCKED, no care about the rest, UNLOCK rollbacks
everything.

I imagine in bare metal, if DPDK works as an attester (within TEE) and
VFIO still as a TEE helper (out of TEE), this model seems still work.



When VFIO works as a TEE user in VM, it means an attester (e.g. PCI
subsystem) has already moved the device to RUN state. So VFIO & DPDK
are all TEE users, no need to manipulate TDISP state between them.
AFAICS, this is the most preferred TIO usage in CoCo-VM.

When VFIO works as a TEE attester in VM, it means the VM's PCI
subsystem leaves the attestation work to device drivers. VFIO should do
the attestation and accept before pass through to DPDK, again no need to
manipulate TDISP state between them.

I image the possibility TIO happens on bare metal, that a device is
configured as waiting for attestation by whatever kernel module, then
PCI subsystem or VFIO try to attest, accept and use it, just the same as
in CoCo VM.

> 
> > > without involving KVM or cVMs.
> > 
> > It may not be feasible for all vendors. 
> 
> It must be. A CC guest with an in kernel driver can definately get the
> PCI device into RUN, so VFIO running in the guest should be able as
> well.

You are talking about VFIO in CoCo-VM as an attester, then definiately
yes.

> 
> > I believe AMD would have one firmware call that requires cVM handle
> > *AND* move device into LOCKED state. It really depends on firmware
> > implementation.
> 
> IMHO, you would not use the secure firmware if you are not using VMs.
> 
> > Yes, the secure EPT is in the secure world and managed by TDX firmware.
> > Now a SW Mirror Secure EPT is introduced in KVM and managed by KVM
> > directly, and KVM will finally use firmware calls to propagate Mirror
> > Secure EPT changes to secure EPT.
> 
> If the secure world managed it then the secure world can have rules
> that work with the IOMMU as well..

Yes.

Thanks,
Yilun

> 
> Jason