All of lore.kernel.org
 help / color / mirror / Atom feed
From: <mhonap@nvidia.com>
To: <alex@shazbot.org>, <ankita@nvidia.com>, <jic23@kernel.org>,
	<dave.jiang@intel.com>, <alejandro.lucero-palau@amd.com>,
	<smadhavan@nvidia.com>, <pierrick.bouvier@oss.qualcomm.com>,
	<mst@redhat.com>, <imammedo@redhat.com>, <anisinha@redhat.com>,
	<pbonzini@redhat.com>, <eric.auger@redhat.com>,
	<peter.maydell@linaro.org>, <richard.henderson@linaro.org>,
	<clg@redhat.com>, <cohuck@redhat.com>
Cc: <kjaju@nvidia.com>, <vsethi@nvidia.com>, <zhiw@nvidia.com>,
	<mhonap@nvidia.com>, <qemu-devel@nongnu.org>,
	<qemu-arm@nongnu.org>
Subject: [PATCH 08/10] docs/cxl: Document CXL Type-2 device passthrough
Date: Thu, 13 Aug 2026 18:36:21 +0530	[thread overview]
Message-ID: <20260813130623.2499506-9-mhonap@nvidia.com> (raw)
In-Reply-To: <20260813130623.2499506-1-mhonap@nvidia.com>

From: Manish Honap <mhonap@nvidia.com>

Describe the supported single-endpoint passthrough topology, the command
line, and the kernel dependency, and note that a guest CXL reset is
handled by the host kernel rather than QEMU.

Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 docs/system/devices/cxl.rst | 46 +++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/docs/system/devices/cxl.rst b/docs/system/devices/cxl.rst
index 9d0771cdfd..33a9fb169b 100644
--- a/docs/system/devices/cxl.rst
+++ b/docs/system/devices/cxl.rst
@@ -418,6 +418,52 @@ Volatile Memory device::
   -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0 \
   -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
 
+Type 2 device passthrough
+-------------------------
+
+A CXL Type 2 device (an accelerator with host-managed device memory) can be
+assigned to a guest with vfio-pci, so the guest reaches the device memory
+through its own CXL stack. This pairs with the kernel vfio-cxl series: the
+host kernel fixes the device memory at a host physical range before the guest
+sees the device, and QEMU maps that range at the guest physical address the
+guest programs into its endpoint HDM decoder. The guest chooses only the GPA;
+the host physical placement is never reprogrammed by the guest.
+
+The device memory reaches the guest as a CXL fixed memory window (``cxl-fmw``),
+advertised through CEDT, exactly like a Type 3 window; there is no separate
+device memory-map slot. Only the endpoint decoder is programmed, by the guest,
+so the host bridge must stay in HDM passthrough mode: a single ``cxl-rp`` under
+the ``pxb-cxl`` and no ``hdm_for_passthrough``. Switch-attached and interleaved
+topologies are rejected.
+
+Example command line::
+
+  -machine q35,cxl=on
+  -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1
+  -device cxl-rp,port=0,bus=cxl.1,id=rp0,chassis=0,slot=2
+  -device vfio-pci,host=<BDF>,bus=rp0,id=cxl-ep0
+  -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=<device-mem-size>
+
+The window must be a single-target ``cxl-fmw`` that targets the device's
+``pxb-cxl`` and is at least the size of the device memory. The guest triggers a
+CXL reset by writing the CXL Device DVSEC; the host kernel runs that sequence,
+so QEMU has no reset handling of its own.
+
+On arm64, pass ``accel=on`` to the ``arm-smmuv3`` when passing a Type 2 device
+through. The accelerated SMMUv3 describes the device MSI doorbell to the guest
+through an IORT Reserved Memory Range (RMR) node, which reserves a fixed guest
+IOVA, so OSPM must preserve the firmware PCI resource assignments rather than
+re-enumerate them. QEMU requests that through PCI Firmware ``_DSM`` function 5
+(preserve firmware PCI configuration), which on arm64 is emitted for the CXL
+host bridge only when the machine requests preserved configuration, that is,
+the accelerated SMMUv3 path. This ``_DSM`` is not about the CXL decoder:
+vfio-pci keeps the host BAR fixed, and QEMU's trapped component-register block
+is a subregion of the guest BAR MemoryRegion, so it follows any guest-visible
+BAR relocation on its own. x86 does not use the accelerated SMMU, so it does
+not advertise or implement preserve-configuration function 5; the CXL host
+bridge still emits the ``_DSM`` method, but its function 0 returns an empty
+support mask.
+
 Deprecations
 ------------
 
-- 
2.25.1



  parent reply	other threads:[~2026-08-13 13:08 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-13 13:06 [PATCH 00/10] QEMU: CXL Type-2 device passthrough via vfio-pci mhonap
2026-08-13 13:06 ` [PATCH 01/10] linux-headers: Update vfio.h for CXL Type-2 passthrough mhonap
2026-08-13 13:06 ` [PATCH 02/10] hw/vfio/region: Add vfio_region_setup_with_ops() mhonap
2026-08-13 13:06 ` [PATCH 03/10] hw/vfio/pci: Detect a CXL Type-2 device and read its geometry mhonap
2026-08-13 13:06 ` [PATCH 04/10] hw/vfio/pci: Enforce the passthrough topology for a CXL device mhonap
2026-08-13 13:06 ` [PATCH 05/10] hw/vfio/pci: Back the CXL memory with a RAM-device region mhonap
2026-08-13 13:06 ` [PATCH 06/10] hw/vfio/pci: Bind a CXL device to its fixed memory window mhonap
2026-08-13 13:06 ` [PATCH 07/10] hw/vfio/pci: Map the CXL memory on the guest decoder commit mhonap
2026-08-13 13:06 ` mhonap [this message]
2026-08-13 13:06 ` [PATCH 09/10] hw/arm/smmu-common: Allow pxb-cxl as an SMMUv3 primary bus mhonap
2026-08-13 13:06 ` [PATCH 10/10] hw/pci-host: Emit a _DSM on pxb-cxl to preserve firmware PCI config mhonap
2026-08-13 17:42 ` [PATCH 00/10] QEMU: CXL Type-2 device passthrough via vfio-pci Cédric Le Goater

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260813130623.2499506-9-mhonap@nvidia.com \
    --to=mhonap@nvidia.com \
    --cc=alejandro.lucero-palau@amd.com \
    --cc=alex@shazbot.org \
    --cc=anisinha@redhat.com \
    --cc=ankita@nvidia.com \
    --cc=clg@redhat.com \
    --cc=cohuck@redhat.com \
    --cc=dave.jiang@intel.com \
    --cc=eric.auger@redhat.com \
    --cc=imammedo@redhat.com \
    --cc=jic23@kernel.org \
    --cc=kjaju@nvidia.com \
    --cc=mst@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=peter.maydell@linaro.org \
    --cc=pierrick.bouvier@oss.qualcomm.com \
    --cc=qemu-arm@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=richard.henderson@linaro.org \
    --cc=smadhavan@nvidia.com \
    --cc=vsethi@nvidia.com \
    --cc=zhiw@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.