Re: [RFC PATCH 00/12] Private MMIO support for private assigned dev

[Alexey Kardashevskiy <aik@amd.com>]

On 29/4/25 16:48, Alexey Kardashevskiy wrote:
> On 8/1/25 01:27, Xu Yilun wrote:
>> This series is based on an earlier kvm-coco-queue version (v6.12-rc2)
> 
> Has this been pushed somewhere public? The patchset does not apply on top of v6.12-rc2, for example (I fixed locally).
> Also, is there somewhere a QEMU tree using this? I am trying to use this new DMA_BUF feature and this require quite some not so obvious plumbing. Thanks,


More to the point, to make it work, QEMU needs to register VFIO MMIO BAR with KVM_SET_USER_MEMORY_REGION2 which passes slot->guest_memfd to KVM which essentially comes from VFIORegion->mmaps[0].mem->ram_block->guest_memfd. But since you disabled mmap for private MMIO, there is no MR which QEMU would even try registering as KVM memslot and there are many ways to fix it. I took a shortcut and reenabled mmap() but wonder what exactly you did. Makes sense? Thanks,


> 
>> which includes all basic TDX patches.
>>
>> The series is to start the early stage discussion of the private MMIO
>> handling for Coco-VM, which is part of the Private Device
>> Assignment (aka TEE-IO, TIO) enabling. There are already some
>> disscusion about the context of TIO:
>>
>> https://lore.kernel.org/linux-coco/173343739517.1074769.13134786548545925484.stgit@dwillia2-xfh.jf.intel.com/
>> https://lore.kernel.org/all/20240823132137.336874-1-aik@amd.com/
>>
>> Private MMIOs are resources owned by Private assigned devices. Like
>> private memory, they are also not intended to be accessed by host, only
>> accessible by Coco-VM via some secondary MMUs (e.g. Secure EPT). This
>> series is for KVM to map these MMIO resources without firstly mapping
>> into the host. For this purpose, This series uses the FD based MMIO
>> resources for secure mapping, and the dma-buf is chosen as the FD based
>> backend, just like guest_memfd for private memory. Patch 6 in this
>> series has more detailed description.
>>
>>
>> Patch 1 changes dma-buf core, expose a new kAPI for importers to get
>> dma-buf's PFN without DMA mapping. KVM could use this kAPI to build
>> GPA -> HPA mapping in KVM MMU.
>>
>> Patch 2-4 are from Jason & Vivek, allow vfio-pci to export MMIO
>> resources as dma-buf. The original series are for native P2P DMA and
>> focus on p2p DMA mapping opens. I removed these p2p DMA mapping code
>> just to focus the early stage discussion of private MMIO. The original
>> series:
>>
>> https://lore.kernel.org/all/0-v2-472615b3877e+28f7-vfio_dma_buf_jgg@nvidia.com/
>> https://lore.kernel.org/kvm/20240624065552.1572580-1-vivek.kasireddy@intel.com/
>>
>> Patch 5 is the implementation of get_pfn() callback for vfio dma-buf
>> exporter.
>>
>> Patch 6-7 is about KVM supports the private MMIO memory slot backed by
>> vfio dma-buf.
>>
>> Patch 8-10 is about how KVM verifies the user provided dma-buf fd
>> eligible for private MMIO slot.
>>
>> Patch 11-12 is the example of how KVM TDX setup the Secure EPT for
>> private MMIO.
>>
>>
>> TODOs:
>>
>> - Follow up the evolving of original VFIO dma-buf series.
>> - Follow up the evolving of basic TDX patches.
>>
>>
>> Vivek Kasireddy (3):
>>    vfio: Export vfio device get and put registration helpers
>>    vfio/pci: Share the core device pointer while invoking feature
>>      functions
>>    vfio/pci: Allow MMIO regions to be exported through dma-buf
>>
>> Xu Yilun (9):
>>    dma-buf: Introduce dma_buf_get_pfn_unlocked() kAPI
>>    vfio/pci: Support get_pfn() callback for dma-buf
>>    KVM: Support vfio_dmabuf backed MMIO region
>>    KVM: x86/mmu: Handle page fault for vfio_dmabuf backed MMIO
>>    vfio/pci: Create host unaccessible dma-buf for private device
>>    vfio/pci: Export vfio dma-buf specific info for importers
>>    KVM: vfio_dmabuf: Fetch VFIO specific dma-buf data for sanity check
>>    KVM: x86/mmu: Export kvm_is_mmio_pfn()
>>    KVM: TDX: Implement TDX specific private MMIO map/unmap for SEPT
>>
>>   Documentation/virt/kvm/api.rst     |   7 +
>>   arch/x86/include/asm/tdx.h         |   3 +
>>   arch/x86/kvm/mmu.h                 |   1 +
>>   arch/x86/kvm/mmu/mmu.c             |  25 ++-
>>   arch/x86/kvm/mmu/spte.c            |   3 +-
>>   arch/x86/kvm/vmx/tdx.c             |  57 +++++-
>>   arch/x86/virt/vmx/tdx/tdx.c        |  52 ++++++
>>   arch/x86/virt/vmx/tdx/tdx.h        |   3 +
>>   drivers/dma-buf/dma-buf.c          |  90 ++++++++--
>>   drivers/vfio/device_cdev.c         |   9 +-
>>   drivers/vfio/pci/Makefile          |   1 +
>>   drivers/vfio/pci/dma_buf.c         | 273 +++++++++++++++++++++++++++++
>>   drivers/vfio/pci/vfio_pci_config.c |  22 ++-
>>   drivers/vfio/pci/vfio_pci_core.c   |  64 +++++--
>>   drivers/vfio/pci/vfio_pci_priv.h   |  27 +++
>>   drivers/vfio/pci/vfio_pci_rdwr.c   |   3 +
>>   drivers/vfio/vfio_main.c           |   2 +
>>   include/linux/dma-buf.h            |  13 ++
>>   include/linux/kvm_host.h           |  25 ++-
>>   include/linux/vfio.h               |  22 +++
>>   include/linux/vfio_pci_core.h      |   1 +
>>   include/uapi/linux/kvm.h           |   1 +
>>   include/uapi/linux/vfio.h          |  34 +++-
>>   virt/kvm/Kconfig                   |   6 +
>>   virt/kvm/Makefile.kvm              |   1 +
>>   virt/kvm/kvm_main.c                |  32 +++-
>>   virt/kvm/kvm_mm.h                  |  19 ++
>>   virt/kvm/vfio_dmabuf.c             | 151 ++++++++++++++++
>>   28 files changed, 896 insertions(+), 51 deletions(-)
>>   create mode 100644 drivers/vfio/pci/dma_buf.c
>>   create mode 100644 virt/kvm/vfio_dmabuf.c
>>
> 

-- 
Alexey