Re: [PATCH 18/20] docs: vfio-pci: Document CXL Type-2 device passthrough

[Alex Williamson <alex@shazbot.org>]

On Fri, 13 Mar 2026 12:13:41 +0000
Jonathan Cameron <jonathan.cameron@huawei.com> wrote:

> On Thu, 12 Mar 2026 02:04:38 +0530
> mhonap@nvidia.com wrote:
> 
> > From: Manish Honap <mhonap@nvidia.com>
> > ---
> >  Documentation/driver-api/index.rst        |   1 +
> >  Documentation/driver-api/vfio-pci-cxl.rst | 216 ++++++++++++++++++++++
> >  2 files changed, 217 insertions(+)
> >  create mode 100644 Documentation/driver-api/vfio-pci-cxl.rst
> > 
> > diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst
> > index 1833e6a0687e..7ec661846f6b 100644
> > --- a/Documentation/driver-api/index.rst
> > +++ b/Documentation/driver-api/index.rst  
> 
> >  
> >  Bus-level documentation
> >  =======================
> > diff --git a/Documentation/driver-api/vfio-pci-cxl.rst b/Documentation/driver-api/vfio-pci-cxl.rst
> > new file mode 100644
> > index 000000000000..f2cbe2fdb036
> > --- /dev/null
> > +++ b/Documentation/driver-api/vfio-pci-cxl.rst  
> 
> > +Device Detection
> > +----------------
> > +
> > +CXL Type-2 detection happens automatically when ``vfio-pci`` registers a
> > +device that has:
> > +
> > +1. A CXL Device DVSEC capability (PCIe DVSEC Vendor ID 0x1E98, ID 0x0000).
> > +2. Bit 2 (Mem_Capable) set in the CXL Capability register within that DVSEC.  
> 
> FWIW to be type 2 as opposed to a type 3 non class code device (e.g. the
> compressed memory devices Gregory Price and others are using) you need
> Cache_capable as well.  Might be worth making this all about
> CXL Type-2 and non class code Type-3.
> 
> > +3. A PCI class code that is **not** ``0x050210`` (CXL Type-3 memory device).
> > +4. An HDM Decoder block discoverable via the Register Locator DVSEC.
> > +5. A pre-committed HDM decoder (BIOS/firmware programmed) with non-zero size.  
> 
> This is the bit that we need to make more general. Otherwise you'll have
> to have a bios upgrade for every type 2 device (and no native hotplug).
> Note native hotplug is quite likely if anyone is switch based device
> pooling.
> 
> I assume that you are doing this today to get something upstream
> and presume it works for the type 2 device you have on the host you
> care about.  I'm not sure there are 'general' solutions but maybe
> there are some heuristics or sufficient conditions for establishing the
> size.
> 
> Type 2 might have any of:
> - Conveniently preprogrammed HDM decoders (the case you use)
> - Maximum of 2 HDM decoders + the same number of Range registers.
>   In general the problem with range registers is they are a legacy feature
>   and there are only 2 of them whereas a real device may have many more
>   DPA ranges. In this corner case though, is it enough to give us the
>   necessary sizes?  I think it might be but would like others familiar
>   with the spec to confirm. (If needed I'll take this to the consortium
>   for an 'official' view).
> - A DOE and table access protocol.  CDAT should give us enough info to
>   be fairly sure what is needed.
> - A CXL mailbox (maybe the version in the PCI spec now) and the spec defined
>   commands to query what is there.  Reading the intro to 8.2.10.9 Memory
>   Device Command Sets, it's a little unclear on whether these are valid on
>   non class code devices but I believe having the appropriate Mailbox
>   type identifier is enough to say we expect to get them.
> 
> None of this is required though and the mailboxes are non trivial.
> So personally I think we should propose a new DVSEC that provides any
> info we need for generic passthrough.  Starting with what we need
> to get the regions right.  Until something like that is in place we
> will have to store this info somewhere.
> 
> There is (maybe) an alternative of doing the region allocation on demand.
> That is emulate the HDM decoders in QEMU (on top of the emulation
> here) and when settings corresponding to a region setup occur,
> go request one from the CXL core. The problem is we can't guarantee
> it will be available at that time. So we can 'guess' what to provide
> to the VM in terms of CXL fixed memory windows, but short of heuristics
> (either whole of the host offer, or divide it up based on devices present
>  vs what is in the VM) that is going to be prone to it not being available
> later.
> 
> Where do people think this should be?  We are going to end up with
> a device list somewhere. Could be in kernel, or in QEMU or make it an
> orchestrator problem (applying the 'someone else's problem' solution).

That's the typical approach.  That's what we did with resizable BARs.
If we cannot guarantee allocation on demand, we need to push the policy
to the device, via something that indicates the size to use, or to the
orchestration, via something that allows the size to be committed
out-of-band.  As with REBAR, we then need to be able to restrict the
guest behavior to select only the configured option.

I imagine this means for the non-pre-allocated case, we need to develop
some sysfs attributes that allows that out-of-band sizing, which would
then appear as a fixed, pre-allocated configuration to the guest.
Thanks,

Alex