* [RFC QEMU PATCH 00/10] Initial Support for VCS Switching
@ 2026-04-29 13:48 Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 01/10] docs: Add documentation for cxl-vcs-switch Joshua Lant
` (9 more replies)
0 siblings, 10 replies; 19+ messages in thread
From: Joshua Lant @ 2026-04-29 13:48 UTC (permalink / raw)
To: linux-cxl; +Cc: qemu-devel, Jonathan.Cameron, arpit1.kumar, Joshua Lant
Hi,
(All references to the CXL specification here are made to v3.2.)
VCS= Virtual CXL Switch
PPB= PCI-PCI bridge
vPPB= virtual PCI-PCI bridge
SLD= Single Logical Device
FM= Fabric Manager
USP/DSP=Upstream/Downstream Port
This patchset provides basic functionality for emulation of an FM-owned,
multi-VCS CXL switch (7.1.3). Primarily it adds a new object defined
in hw/cxl/cxl-vcs-switch.c, and supports the FMAPI commands vppb get
info/bind/unbind 0x5200/0x5201/0x5202 (7.6.7.2.1-7.6.7.2.3) to be used on SLD’s.
It is posted as an RFC since:
1). There are several limitations with this work and I would like to discuss
how to proceed.
2). There has been prior discussion on potential restructuring of the CXL
cci/switch code[1], which I suppose will dictate how to move forward with parts
of this series…
-------------------
VCS Emulation
The VCS command set should ultimately allow for multi-host sharing of a MLD/DCD
device through the same switched fabric, with a single Fabric Manager/CCI-mailbox
endpoint for configuration/management. The work here is a stepping stone toward
that, allowing for SLD’s to be bound/unbound to a local QEMU instance by the FM.
The cxl-vcs-switch is comprised of one or more VCSs, and one or more hidden
endpoint devices. A single VCS is defined as a single cxl-upstream-port
(meaning a 1:1 mapping between physical upstream PPBs and number of VCS’s),
and one or more cxl-downstream-ports (these form the vPPBs of each VCS). The
number of vcs-attached endpoints defined on the CLI forms the number of
downstream PPBs the switch would have. These endpoints are hidden on boot, and
connect to one of the vPPBs upon bind. This means that there is in effect
no real downstream PPB in QEMU. The vPPB device effectively becomes the
downstream PPB following bind.
When the topology is initialised:
- Upstream/downstream ports instantiated as part of a VCS switch are realized
normally, but additionally register with the cxl-vcs-switch object, which are
then referenced by the bind/unbind FMAPI commands etc.
- The endpoint devices which connect to the downstream PPBs use the
DeviceListener functionality for hiding devices (see the reference in the
commit message of patch 2). The QDict from the CLI is stored in the VCS structs,
which are then realized/unrealized on bind/unbind commands from the FM.
All this means that on boot the guest sees its upstream port and all downstream
ports (vppbs) enumerated (as is described in section 7.1.4 and 7.2.1.3), but
none of the endpoints are seen.
The cxl-vcs-switch itself is implemented as a user creatable class, since it
does not fit the single inheritance device model of QEMU, which would force
association with a single PCIe bus, which will not work for multiple USPs.
The cxl-vcs-switch uses local-fm=true/false CLI option to dictate whether the
object will have a CCI mailbox attached. VCS state information will be held in
the local-fm=true instance, and the FM will communicate directly with this
instance only. IPC will be used (in multi-USP, multi-QEMU process environments)
in order to maintain correct state information in the local-fm, and to
bind/unbind devices in remote QEMU processes.
Currently only the “managed hot-remove” flow is complete (Table 7-34) for the
unbind operation, notifying the guest of removal and awaiting signal from
the OS for unbind completion from the unrealize DeviceListener function.
This is tested with the topology below and some additional libcxlmi test
programs[2]. I am able to bind and unbind correctly to multiple VCSs, see the
updated switch state, and see the delay in the unrealize listener callback from
the delays in OS notification.
-------------------
Limitations and open questions
1. Unbound, but fully realized devices (allowing proper MLDs/DCDs)
The current method of hiding the endpoint device and storing the QDicts works
for simple devices only. But ultimately the FM should be able to tunnel commands
through the switch to communicate with the device, whether a guest is bound to
it or not... We need a method of fully realizing the device, but on some sort of
dummy bus that is not seen by the guest. I am unsure how to do this currently,
since AFAICT it goes against the qdev model of device realization, being
inherently associated with attaching to the guest’s bus (please correct me if
I’m wrong on this). It will require a way to properly realize the devices, but
bypass automatic association with the guest’s QOM tree? I don’t know if there
is any precedent for behaviour like this in QEMU?
2. Integration with Physical Switch Command set.
Currently the physical switch command set in cci-mailbox-utils.c has not been
modified to account for a VCS target (i.e. using those commands with a VCS
target currently breaks things). This is where the discussion in [1] comes in.
> - Move the call that caches state to the cxl_upstream_port reset
> to ensure downstream ports are in place before it is called.
> Also will make it available from whatever CCI. If we ever support
> multiple VCS switches this will need to move an appropriate structure
> representing whole switch information. With only one USP that is
> a reasonable place to put full switch info.
Since in my implementation the CXLPhyPortInfo will end up being associated with
the VCS object and not with the USP, further refactoring will be required for
this patch series to generalise the physical switch command set functions.
3. Distributed VCS control
Since the ultimate aim of this is to give the FM control of multiple VCSs in
multiple QEMU instances, some method of sending FM commands between QEMU
processes is needed, since the switch state will be distributed over these
processes (with status kept in the local-fm=true instance). Does QEMU have a
standard way for implementing such IPC or shall I just add some simple sockets
communication into the cxl-vcs-switch.c?
4. Unimplemented commands from virtual switch command set.
The actual bind process described in 7.6.6.7 shows how event records must be
used, and the FMAPI command should return success without waiting on binding
completion. Further work is needed to completely emulate the flow as described
in the specification, and implement the remaining FMAPI commands.
5. Tiered switching.
Currently the downstream PPBs are nothing more than a struct, and only endpoint
devices can be hidden. There should be some way to implement another complete
switch below the downstream PPB of the first switch, as described in
section 9.12.2.
-------------------
Testing
topology:
-device usb-ehci,id=ehci \
-object memory-backend-file,id=cxl-mem1,share=on,mem-path=/$LOG_DIR/t3_cxl1.raw,size=8G \
-object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/$LOG_DIR/t3_lsa1.raw,size=1M \
-object memory-backend-file,id=cxl-mem2,share=on,mem-path=/$LOG_DIR/t3_cxl2.raw,size=8G \
-object memory-backend-file,id=cxl-lsa2,share=on,mem-path=/$LOG_DIR/t3_lsa2.raw,size=1M \
-object memory-backend-file,id=cxl-mem3,share=on,mem-path=/$LOG_DIR/t3_cxl3.raw,size=8G \
-object memory-backend-file,id=cxl-lsa3,share=on,mem-path=/$LOG_DIR/t3_lsa3.raw,size=1M \
-object memory-backend-file,id=cxl-mem4,share=on,mem-path=/$LOG_DIR/t3_cxl4.raw,size=8G \
-object memory-backend-file,id=cxl-lsa4,share=on,mem-path=/$LOG_DIR/t3_lsa4.raw,size=1M \
-object cxl-vcs-switch,id=vcs0,usp-ppbs=2,dsp-ppbs=4,local-fm=true \
-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.0,hdm_for_passthrough=true \
-device cxl-rp,port=0,bus=cxl.0,id=root_port1,chassis=0,slot=1 \
-device pxb-cxl,bus_nr=22,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
-device cxl-rp,port=0,bus=cxl.1,id=root_port2,chassis=1,slot=1 \
-device cxl-upstream,port=0,sn=1234,bus=root_port1,id=us0,addr=0.0,multifunction=on,vcs=vcs0,usppb=0 \
-device cxl-upstream,port=0,sn=5678,bus=root_port2,id=us1,addr=0.0,multifunction=on,vcs=vcs0,usppb=1 \
-device cxl-switch-mailbox-cci,bus=root_port1,addr=0.3,target=vcs0 \
-device usb-cxl-mctp,bus=ehci.0,id=usb0,target=vcs0 \
-device cxl-downstream,port=0,bus=us0,id=swport0,slot=3 \
-device cxl-downstream,port=1,bus=us0,id=swport1,slot=4 \
-device cxl-downstream,port=2,bus=us0,id=swport2,slot=5 \
-device cxl-downstream,port=3,bus=us0,id=swport3,slot=6 \
-device cxl-downstream,port=0,bus=us1,id=swport4,slot=7 \
-device cxl-downstream,port=1,bus=us1,id=swport5,slot=8 \
-device cxl-downstream,port=2,bus=us1,id=swport6,slot=9 \
-device cxl-downstream,port=3,bus=us1,id=swport7,slot=10 \
-device cxl-type3,persistent-memdev=cxl-mem1,id=cxl-ep1,lsa=cxl-lsa1,sn=99,vcs=vcs0,dsppb=0 \
-device cxl-type3,persistent-memdev=cxl-mem2,id=cxl-ep2,lsa=cxl-lsa2,sn=100,vcs=vcs0,dsppb=1 \
-device cxl-type3,volatile-dc-memdev=cxl-mem3,id=cxl-dcd1,lsa=cxl-lsa3,num-dc-regions=8,sn=101,vcs=vcs0,dsppb=2 \
-device cxl-type3,volatile-dc-memdev=cxl-mem4,id=cxl-dcd2,lsa=cxl-lsa4,num-dc-regions=8,sn=102,vcs=vcs0,dsppb=3 \
-machine cxl-fmw.0.targets.0=cxl.0,cxl-fmw.0.size=8G,cxl-fmw.1.targets.0=cxl.1,cxl-fmw.1.size=8G
libcxlmi:
See [2]…
1. Setup MCTP communication: e.g.
mctp link set mctpusb0 up;
mctp addr add 8 dev mctpusb0;
mctp link set mctpusb0 net 1;
systemctl restart mctpd.service
busctl call au.com.codeconstruct.MCTP1 /au/com/codeconstruct/mctp1/interfaces/mctpusb0 au.com.codeconstruct.MCTP.BusOwner1 SetupEndpoint ay 0
2. Build, then run libcxlmi commands to bind vcs0:
./build/examples/vcs-bind-mctp 1 9 0 0 0
./build/examples/vcs-get-virtual-switch-info-mctp 1 9
3. Demonstrate the CXL device is usable:
cxl create-region -m -t pmem -d decoder0.0 -w 1 -g 1024 -s 256M mem0
ndctl create-namespace --region region0 --mode fsdax --size 256M
echo "HELLO WORLD..." > /dev/pmem0
cat /dev/pmem0
4. Teardown the device, and rebind to vcs1, and check that it maps
correctly:
ndctl disable-namespace namespace0.0
ndctl destroy-namespace namespace0.0
cxl disable-region region0
cxl destroy-region region0
cxl disable-memdev mem0
./build/examples/vcs-unbind-mctp 1 9 0 0 1
# wait 5s for the hp notification... see lspci/dmesg change
# bind the same device to the other VCS.
./build/examples/vcs-bind-mctp 1 9 1 0 0
cxl create-region -m -t pmem -d decoder0.1 -w 1 -g 1024 -s 256M mem0
ndctl create-namespace --region region1 --mode fsdax --size 256M
cat /dev/pmem1
# See the hello world originally written by vcs0!
-------------------
Build
The patches are applied on the upstream qemu 10.2 release, on top of the
following patchsets from various branches of Jonathan’s fork:
1: [PATCH qemu v5 0/5] cxl: r3.2 specification event updates.
https://lore.kernel.org/linux-cxl/20260205112350.60681-1-Jonathan.Cameron@huawei.com/
2: [PATCH qemu for 10.2 0/3] cxl: Additional RAS features support.
https://lore.kernel.org/linux-cxl/20250917143330.294698-1-Jonathan.Cameron@huawei.com/
3: [PATCH qemu 0/2] hw/cxl: Two media operations related fixes.
https://lore.kernel.org/linux-cxl/20260102154731.474859-1-Jonathan.Cameron@huawei.com/
4: [PATCH qemu v7 0/7] hw/cxl: Support Back-Invalidate (+ PCIe Flit mode)
https://lore.kernel.org/linux-cxl/20260204170936.43959-1-Jonathan.Cameron@huawei.com/
5: [PATCH qemu v5 0/3] hw/cxl: FM-API Physical Switch Command Set Support.
https://lore.kernel.org/linux-cxl/20260204173223.44122-1-Jonathan.Cameron@huawei.com/
6: [RFC PATCH qemu 0/5] hw/cxl/mctp/i2c/usb: MCTP for OoB control of CXL devices.
https://lore.kernel.org/linux-cxl/20250609163334.922346-1-Jonathan.Cameron@huawei.com/
-------------------
References
[1] https://lore.kernel.org/linux-cxl/20260127152350.00006447@huawei.com/
[2] https://github.com/joshualant/libcxlmi/tree/vcs-testing
Many thanks,
Josh
Joshua Lant (10):
docs: Add documentation for cxl-vcs-switch
qdev/qbus: Allow hidden devices to be busless on QEMU startup
cxl-type3: Properly unmap the memory-backend on device exit
cxl_downstream: enable power controller present capability.
cxl-vcs-switch: Initial support for CXL VCS.
cxl-upstream-port: Add support for targeting a VCS switch
cxl-downstream-port: Add support for VCS switching
cxl-cci-mailbox: Add support for targeting a VCS switch
cxl-mailbox-utils: Add support for VCS bind/unbind commands.
cxl-mailbox-utils: Add support for VCS Get Virtual CXL Switch Info
command.
docs/system/devices/cxl.rst | 90 +++-
hw/cxl/cxl-mailbox-utils.c | 207 ++++++++-
hw/cxl/cxl-vcs-switch.c | 524 ++++++++++++++++++++++
hw/cxl/meson.build | 1 +
hw/cxl/switch-mailbox-cci.c | 33 +-
hw/mem/cxl_type3.c | 3 +
hw/pci-bridge/cxl_downstream.c | 13 +
hw/pci-bridge/cxl_upstream.c | 20 +
hw/usb/dev-mctp.c | 23 +-
include/hw/cxl/cxl_device.h | 10 +-
include/hw/cxl/cxl_vcs_switch.h | 134 ++++++
include/hw/pci-bridge/cxl_upstream_port.h | 2 +
qapi/qom.json | 19 +
system/qdev-monitor.c | 10 +-
14 files changed, 1055 insertions(+), 34 deletions(-)
create mode 100644 hw/cxl/cxl-vcs-switch.c
create mode 100644 include/hw/cxl/cxl_vcs_switch.h
--
2.47.3
^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC QEMU PATCH 01/10] docs: Add documentation for cxl-vcs-switch
2026-04-29 13:48 [RFC QEMU PATCH 00/10] Initial Support for VCS Switching Joshua Lant
@ 2026-04-29 13:48 ` Joshua Lant
2026-05-19 14:33 ` Jonathan Cameron
2026-04-29 13:48 ` [RFC QEMU PATCH 02/10] qdev/qbus: Allow hidden devices to be busless on QEMU startup Joshua Lant
` (8 subsequent siblings)
9 siblings, 1 reply; 19+ messages in thread
From: Joshua Lant @ 2026-04-29 13:48 UTC (permalink / raw)
To: linux-cxl; +Cc: qemu-devel, Jonathan.Cameron, arpit1.kumar, Joshua Lant
Signed-off-by: Joshua Lant <joshualant@gmail.com>
---
docs/system/devices/cxl.rst | 90 ++++++++++++++++++++++++++++++++++---
1 file changed, 85 insertions(+), 5 deletions(-)
diff --git a/docs/system/devices/cxl.rst b/docs/system/devices/cxl.rst
index 32b1b5d773..9e8452e576 100644
--- a/docs/system/devices/cxl.rst
+++ b/docs/system/devices/cxl.rst
@@ -119,11 +119,11 @@ and associated component register access via PCI bars.
CXL Switch
~~~~~~~~~~
Here we consider a simple CXL switch with only a single
-virtual hierarchy. Whilst more complex devices exist, their
-visibility to a particular host is generally the same as for
-a simple switch design. Hosts often have no awareness
-of complex rerouting and device pooling, they simply see
-devices being hot added or hot removed.
+virtual hierarchy. Whilst more complex devices exist (see VCS
+Switching below), their visibility to a particular host is
+generally the same as for a simple switch design. Hosts often
+have no awareness of complex rerouting and device pooling,
+they simply see devices being hot added or hot removed.
A CXL switch has a similar architecture to those in PCIe,
with a single upstream port, internal PCI bus and multiple
@@ -467,6 +467,86 @@ Example configuration:
Guest OS communication with the MCTP CCI can then be established using standard
MCTP configuration tools.
+CXL Multi-VCS Switching
+-----------------------
+
+The cxl-vcs-switch object allows for a Fabric Manager to dynamically reconfigure
+the switching within a multi-upstream port CXL/PCIe topology, This moves beyond
+the static switching configuration described above. The use of vcs=X on an
+endpoint device indicates that it should be hidden from guests at boot. Each
+upstream port with vcs=X set will conceptually become an upstream PPB. Any
+downstream port that is connected to an upstream port with vcs=X set will
+automatically become a vPPB for that VCS. The overall cxl-virtual-switch has a
+single CCI mailbox used for config/status of all ports within the switch.
+Setting local-fm=true indicates that this QEMU instance has the CCI mailbox
+attached. Setting it false will create listeners for commands from a remote
+QEMU process (yet to be implemented).
+
+An example of how the topology is described on the CLI is shown below:
+
+ -object cxl-vcs-switch,id=vcs0,usp-ppbs=2,dsp-ppbs=4,local-fm=true \
+ -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.0,hdm_for_passthrough=true \
+ -device cxl-rp,port=0,bus=cxl.0,id=root_port1,chassis=0,slot=1 \
+ -device pxb-cxl,bus_nr=22,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
+ -device cxl-rp,port=0,bus=cxl.1,id=root_port2,chassis=1,slot=1 \
+ -device cxl-upstream,port=0,sn=1234,bus=root_port1,id=us0,addr=0.0,multifunction=on,vcs=vcs0,usppb=0 \
+ -device cxl-upstream,port=0,sn=5678,bus=root_port2,id=us1,addr=0.0,multifunction=on,vcs=vcs0,usppb=1 \
+ -device cxl-switch-mailbox-cci,bus=root_port1,addr=0.3,target=vcs0 \
+ -device usb-cxl-mctp,bus=ehci.0,id=usb0,target=vcs0 \
+ -device cxl-downstream,port=0,bus=us0,id=dsp0,slot=3 \
+ -device cxl-downstream,port=1,bus=us0,id=dsp1,slot=4 \
+ -device cxl-downstream,port=0,bus=us1,id=dsp2,slot=7 \
+ -device cxl-downstream,port=1,bus=us1,id=dsp3,slot=8 \
+ -device cxl-type3,persistent-memdev=cxl-mem1,id=cxl-ep1,lsa=cxl-lsa1,sn=99,vcs=vcs0,dsppb=0 \
+ -device cxl-type3,persistent-memdev=cxl-mem2,id=cxl-ep2,lsa=cxl-lsa2,sn=100,vcs=vcs0,dsppb=1 \
+ -device cxl-type3,persistent-memdev=cxl-mem3,id=cxl-ep3,lsa=cxl-lsa3,sn=101,vcs=vcs0,dsppb=2 \
+ -device cxl-type3,persistent-memdev=cxl-mem4,id=cxl-ep4,lsa=cxl-lsa4,sn=102,vcs=vcs0,dsppb=3 \
+ -machine cxl-fmw.0.targets.0=cxl.0,cxl-fmw.0.size=8G,cxl-fmw.1.targets.0=cxl.1,cxl-fmw.1.size=8G
+
+Example topology involving VCS switching::
+
+ +--------------------+ +--------------------+
+ | Host Bridge 0 | | Host Bridge 1 |
+ +----------+---------+ +----------+---------+
+ +-------+ | |
+ | MCTP | | |
+ | USB/ | +----------+---------+ +----------+---------+
+ | I2C | | Root Port 0 | | Root Port 1 |
+ +-----+-+ +----------+---------+ +----------+---------+
+ | | |
+ | | |
+ +------|---------------+-----------------------+-----------------------+
+ | +-+--------+ | cxl-vcs-switch (vcs0)| |
+ | +--| CCI MBOX |---* | | |
+ | | +----------+ | | |
+ | | +-----------------+--------+ +-------+------------------+ |
+ | +--+ | VCS0 | *---+ | VCS1 | |
+ | | +---------------+------+ | | +-----+----------------+ | |
+ | | | | | | | | | |
+ | | | USP 0 | | | | USP 1 | | |
+ | | | | | | | | | |
+ | | +----+------------+----+ | | +----+------------+----+ | |
+ | | | | | | | | | |
+ | | +----+----+ +----+----+ | | +----+----+ +----+----+ | |
+ | | | DSP 0 | | DSP 1 | | | | DSP 2 | | DSP 3 | | |
+ | | |(vPPB 0) | |(vPPB 1) | | | |(vPPB 0) | |(vPPB 1) | | |
+ | | | | | | | | | | | | | |
+ | | +---------+ +---------+ | | +---------+ +----+----+ | |
+ | +--------------------------+ +-------------------+------+ |
+ | | |
+ | +----------------------------------------------+ |
+ | | |
+ | | - - - |
+ +-----------|------------|--------------------|------------|-----------+
+ | | | |
+ +---------+ +---------+ +---------+ +---------+
+ |CXL/PCIe | |CXL/PCIe | |CXL/PCIe | |CXL/PCIe |
+ | EP 0 | | EP 1 | | EP 2 | | EP 3 |
+ | (PPB0) | | (PPB1) | | (PPB2) | | (PPB3) |
+ +---------+ +---------+ +---------+ +---------+
+ PPB0 Bound to VCS1, vPPB1. Others unbound...
+
+
References
----------
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [RFC QEMU PATCH 02/10] qdev/qbus: Allow hidden devices to be busless on QEMU startup
2026-04-29 13:48 [RFC QEMU PATCH 00/10] Initial Support for VCS Switching Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 01/10] docs: Add documentation for cxl-vcs-switch Joshua Lant
@ 2026-04-29 13:48 ` Joshua Lant
2026-05-20 10:02 ` Jonathan Cameron
2026-04-29 13:48 ` [RFC QEMU PATCH 03/10] cxl-type3: Properly unmap the memory-backend on device exit Joshua Lant
` (7 subsequent siblings)
9 siblings, 1 reply; 19+ messages in thread
From: Joshua Lant @ 2026-04-29 13:48 UTC (permalink / raw)
To: linux-cxl; +Cc: qemu-devel, Jonathan.Cameron, arpit1.kumar, Joshua Lant
Extend the capability for hiding devices, introduced for virtio-net
device in:
commit f3a8505656935cde32e28c1c6317f725084da1e0
Author: Jens Freimann <jfreimann@redhat.com>
Date: Tue Oct 29 12:48:55 2019 +0100
qdev/qbus: add hidden device support
Currently only endpoint devices can be hidden with a primary device
and failover (known static configuration). However, looking at future
composable systems, we see a need for hidden devices which have no associated
bus upon boot. Move the check for hidden devices to before the bus
search, and if it is hidden ignore the case where the device was
described on the CLI without the "bus=" field.
This is motivated by a specific use-case: implementing the VCS
command set, part of the CXL specification (CXL r3.2 Section 7.1.3). In
this scenario a switch controlled by a Fabric Manager is able to change the
virtual hierarchy of devices seen by a guest within a fixed physical system
topology. The connecting bus is not known until runtime when
a bind command is issued by the Fabric Manager.
Signed-off-by: Joshua Lant <joshualant@gmail.com>
---
system/qdev-monitor.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/system/qdev-monitor.c b/system/qdev-monitor.c
index f2aa400a77..b51dfe0645 100644
--- a/system/qdev-monitor.c
+++ b/system/qdev-monitor.c
@@ -650,6 +650,7 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
DeviceState *dev;
BusState *bus = NULL;
QDict *properties;
+ bool hide_device;
driver = qdict_get_try_str(opts, "driver");
if (!driver) {
@@ -663,6 +664,11 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
return NULL;
}
+ /* Is the device hidden from the guest?
+ * If yes, no need to find a default bus if none given...
+ * Bus could be provided at runtime (i.e. in a switch)*/
+ hide_device = qdev_should_hide_device(opts, from_json, errp);
+
/* find bus */
path = qdict_get_try_str(opts, "bus");
if (path != NULL) {
@@ -675,14 +681,14 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
driver, object_get_typename(OBJECT(bus)));
return NULL;
}
- } else if (dc->bus_type != NULL) {
+ } else if (dc->bus_type != NULL && !hide_device) {
bus = qdev_find_default_bus(dc, errp);
if (!bus) {
return NULL;
}
}
- if (qdev_should_hide_device(opts, from_json, errp)) {
+ if (hide_device) {
if (bus && !qbus_is_hotpluggable(bus)) {
error_setg(errp, "Bus '%s' does not support hotplugging",
bus->name);
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [RFC QEMU PATCH 03/10] cxl-type3: Properly unmap the memory-backend on device exit
2026-04-29 13:48 [RFC QEMU PATCH 00/10] Initial Support for VCS Switching Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 01/10] docs: Add documentation for cxl-vcs-switch Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 02/10] qdev/qbus: Allow hidden devices to be busless on QEMU startup Joshua Lant
@ 2026-04-29 13:48 ` Joshua Lant
2026-05-19 15:01 ` Jonathan Cameron
2026-04-29 13:48 ` [RFC QEMU PATCH 04/10] cxl_downstream: enable power controller present capability Joshua Lant
` (6 subsequent siblings)
9 siblings, 1 reply; 19+ messages in thread
From: Joshua Lant @ 2026-04-29 13:48 UTC (permalink / raw)
To: linux-cxl; +Cc: qemu-devel, Jonathan.Cameron, arpit1.kumar, Joshua Lant
Currently the backend remains mapped, meaning that if the device owning
the backend is hot-removed, it cannot be readded in the same QEMU
instance.
Signed-off-by: Joshua Lant <joshualant@gmail.com>
---
hw/mem/cxl_type3.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 68cd04b7d9..414c776028 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1072,12 +1072,15 @@ static void ct3_exit(PCIDevice *pci_dev)
cxl_destroy_cci(&ct3d->cci);
if (ct3d->dc.host_dc) {
cxl_destroy_dc_regions(ct3d);
+ host_memory_backend_set_mapped(ct3d->dc.host_dc, false);
address_space_destroy(&ct3d->dc.host_dc_as);
}
if (ct3d->hostpmem) {
+ host_memory_backend_set_mapped(ct3d->hostpmem, false);
address_space_destroy(&ct3d->hostpmem_as);
}
if (ct3d->hostvmem) {
+ host_memory_backend_set_mapped(ct3d->hostvmem, false);
address_space_destroy(&ct3d->hostvmem_as);
}
}
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [RFC QEMU PATCH 04/10] cxl_downstream: enable power controller present capability.
2026-04-29 13:48 [RFC QEMU PATCH 00/10] Initial Support for VCS Switching Joshua Lant
` (2 preceding siblings ...)
2026-04-29 13:48 ` [RFC QEMU PATCH 03/10] cxl-type3: Properly unmap the memory-backend on device exit Joshua Lant
@ 2026-04-29 13:48 ` Joshua Lant
2026-05-19 15:03 ` Jonathan Cameron
2026-04-29 13:48 ` [RFC QEMU PATCH 05/10] cxl-vcs-switch: Initial support for CXL VCS Joshua Lant
` (5 subsequent siblings)
9 siblings, 1 reply; 19+ messages in thread
From: Joshua Lant @ 2026-04-29 13:48 UTC (permalink / raw)
To: linux-cxl; +Cc: qemu-devel, Jonathan.Cameron, arpit1.kumar, Joshua Lant
PCI_EXP_SLTCAP_PCP (Power Controller Present) must be set in the DSP's
slot capabilities for PCIe managed hot-remove to complete. Without this
notification from the guest of removal cannot be sent back to the
device, so device listeners for unrealizing will not fire.
Signed-off-by: Joshua Lant <joshualant@gmail.com>
---
hw/pci-bridge/cxl_downstream.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/hw/pci-bridge/cxl_downstream.c b/hw/pci-bridge/cxl_downstream.c
index 91c5e6a605..5322c46900 100644
--- a/hw/pci-bridge/cxl_downstream.c
+++ b/hw/pci-bridge/cxl_downstream.c
@@ -228,6 +228,8 @@ static const Property cxl_dsp_props[] = {
DEFINE_PROP_PCIE_LINK_WIDTH("x-width", PCIESlot,
width, PCIE_LINK_WIDTH_16),
DEFINE_PROP_BOOL("x-256b-flit", PCIESlot, flitmode, true),
+ DEFINE_PROP_BIT(COMPAT_PROP_PCP, PCIDevice, cap_present,
+ QEMU_PCIE_SLTCAP_PCP_BITNR, true),
};
static void cxl_dsp_class_init(ObjectClass *oc, const void *data)
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [RFC QEMU PATCH 05/10] cxl-vcs-switch: Initial support for CXL VCS.
2026-04-29 13:48 [RFC QEMU PATCH 00/10] Initial Support for VCS Switching Joshua Lant
` (3 preceding siblings ...)
2026-04-29 13:48 ` [RFC QEMU PATCH 04/10] cxl_downstream: enable power controller present capability Joshua Lant
@ 2026-04-29 13:48 ` Joshua Lant
2026-05-19 15:19 ` Jonathan Cameron
2026-04-29 13:48 ` [RFC QEMU PATCH 06/10] cxl-upstream-port: Add support for targeting a VCS switch Joshua Lant
` (4 subsequent siblings)
9 siblings, 1 reply; 19+ messages in thread
From: Joshua Lant @ 2026-04-29 13:48 UTC (permalink / raw)
To: linux-cxl; +Cc: qemu-devel, Jonathan.Cameron, arpit1.kumar, Joshua Lant
Add initial support for VCS (multi-USP/multi-logic-device) switch
emulation in CXL. Currently the object only supports the identify/bind/unbind
commands (0x5200/0x5201/0x5202), as well as support for device hiding
listeners. This enables preliminary testing of the VCS capability,
but will not allow for complete emulated flow as described in
spec (CXL v3.2, 7.2.3). Nor will it allow for multiple USP's to be
disributed over multiple QEMU instances.
Signed-off-by: Joshua Lant <joshualant@gmail.com>
---
hw/cxl/cxl-vcs-switch.c | 524 ++++++++++++++++++++++++++++++++
hw/cxl/meson.build | 1 +
include/hw/cxl/cxl_vcs_switch.h | 134 ++++++++
qapi/qom.json | 19 ++
4 files changed, 678 insertions(+)
create mode 100644 hw/cxl/cxl-vcs-switch.c
create mode 100644 include/hw/cxl/cxl_vcs_switch.h
diff --git a/hw/cxl/cxl-vcs-switch.c b/hw/cxl/cxl-vcs-switch.c
new file mode 100644
index 0000000000..9a492330cc
--- /dev/null
+++ b/hw/cxl/cxl-vcs-switch.c
@@ -0,0 +1,524 @@
+/*
+ * CXL VCS Capable Switch Object
+ *
+ * Copyright(C) 2026 University of Manchester.
+ * Author: Joshua Lant <joshualant@gmail.com>.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See the
+ * COPYING file in the top-level directory.
+ *
+ * SPDX-License-Identifier: GPL-v2-only
+ */
+#include "hw/cxl/cxl_vcs_switch.h"
+#include "qobject/qdict.h"
+#include "monitor/qdev.h"
+
+/* Primary FM here is used when the FM is local to the QEMU instance. Setting
+ * local-fm=false in cli results in instantiation of remote listener for FM
+ * commands (TODO: to be implemented) */
+static bool cxl_vcs_get_local_fm(Object *obj, Error **errp)
+{
+ CXLVCSSwitch *vcs = CXL_VCS_SWITCH(obj);
+ return vcs->local_fm;
+}
+static void cxl_vcs_set_local_fm(Object *obj, bool value, Error **errp)
+{
+ CXLVCSSwitch *vcs = CXL_VCS_SWITCH(obj);
+ vcs->local_fm = value;
+}
+
+/* Binds an already realized MLD to a VPPB in the switch */
+static CXLRetCode cxl_vcs_bind_realized_vppb(CXLVCSSwitch *sw, uint8_t vcs_id,
+ uint8_t vppb_id, uint8_t dsp_ppb_id, uint8_t ld_id)
+{
+ /* TODO: Implement... Required for handling MLD devices... */
+ return CXL_MBOX_UNSUPPORTED;
+}
+
+/* Binds a hidden device (simple SLD) to a VPPB in the switch */
+static CXLRetCode cxl_vcs_bind_qdict_vppb(CXLVCSSwitch *sw, uint8_t vcs_id,
+ uint8_t vppb_id, uint8_t dsp_ppb_id, uint8_t ld_id)
+{
+ Error *local_err = NULL;
+ CXLVPPBInfo *vppb;
+ CXLDownstreamPPB *dspppb;
+ QDict *bind_opts;
+ DeviceState *dev;
+
+ // Upstream to bind
+ if(!sw->usp_ppbs[vcs_id]) {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+ vppb = sw->usp_ppbs[vcs_id]->info->vppbs[vppb_id];
+ if(!vppb) {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+ PCIDevice *vppb_dev = PCI_DEVICE(vppb->dsp);
+
+ //Downstream to bind
+ dspppb = sw->dsp_ppbs[dsp_ppb_id];
+ if(!dspppb) {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+
+ // Get DSP bus id from qdev
+ bind_opts = qdict_clone_shallow(dspppb->opts);
+ qdict_put_str(bind_opts, "bus", vppb_dev->qdev.id);
+ qdict_del(bind_opts, "vcs");
+ qdict_del(bind_opts, "dsppb");
+
+ uint8_t prev_binding_status = vppb->binding_status;
+ vppb->binding_status = CXL_VPPB_BINDING_STATUS_IN_PROGRESS;
+ dev = qdev_device_add_from_qdict(bind_opts, dspppb->from_json, &local_err);
+ if(!dev || local_err) {
+ vppb->binding_status = prev_binding_status;
+ return CXL_MBOX_INTERNAL_ERROR;
+ }
+
+ vppb->bound_port_id = dsp_ppb_id;
+ vppb->bound_ld_id = CXL_INVALID_BOUND_LD_ID; // TODO: support MLDs.
+ vppb->binding_status = CXL_VPPB_BINDING_STATUS_BOUND_PORT;
+ dspppb->dev = dev;
+ dspppb->is_bound = true;
+ dspppb->bound_vcs_id = vcs_id;
+ dspppb->bound_vppb_id = vppb_id;
+ qobject_unref(bind_opts);
+
+ return CXL_MBOX_SUCCESS;
+}
+
+CXLRetCode cxl_vcs_bind_vppb(CXLVCSSwitch *sw, uint8_t vcs_id, uint8_t vppb_id,
+ uint8_t dsp_ppb_id, uint16_t ld_id)
+{
+ if(cxl_vcs_get_local_fm(OBJECT(sw), NULL)) {
+ if(ld_id == CXL_UNSUPPORTED_LD_ID) {
+ return cxl_vcs_bind_qdict_vppb(sw, vcs_id, vppb_id,
+ dsp_ppb_id, ld_id);
+ } else {
+ // TODO: Connect up already realized device
+ // (needed for MLD support).
+ return cxl_vcs_bind_realized_vppb(sw, vcs_id, vppb_id,
+ dsp_ppb_id, ld_id);
+ }
+ }
+ else {
+ // TODO: Send bind command to remote QEMU process...
+ return CXL_MBOX_UNSUPPORTED;
+ }
+}
+
+static CXLRetCode cxl_vcs_unbind_qdict_vppb(CXLVCSSwitch *sw, uint8_t vcs_id,
+ uint8_t vppb_id, uint8_t options)
+{
+ Error *local_err = NULL;
+ CXLVPPBInfo *vppb_info;
+ CXLDownstreamPPB *dspppb = NULL;
+
+ if(!sw->usp_ppbs[vcs_id]) {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+
+ vppb_info = sw->usp_ppbs[vcs_id]->info->vppbs[vppb_id];
+ if (!vppb_info ||
+ (vppb_info->binding_status == CXL_VPPB_BINDING_STATUS_UNBOUND) ||
+ (vppb_info->binding_status == CXL_VPPB_BINDING_STATUS_IN_PROGRESS)) {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+
+ for (int i = 0; i < sw->num_dsp_ppbs; i++) {
+ if (sw->dsp_ppbs[i] &&
+ sw->dsp_ppbs[i]->is_bound &&
+ sw->dsp_ppbs[i]->bound_vcs_id == vcs_id &&
+ sw->dsp_ppbs[i]->bound_vppb_id == vppb_id) {
+ dspppb = sw->dsp_ppbs[i];
+ break;
+ }
+ }
+
+ if (!dspppb) {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+
+ if (!dspppb->dev) {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+
+ /* Options from cxl v3.2 (Table 7-34), Bits[3:0].
+ — 0h = Wait for port Link Down before unbinding
+ — 1h = Simulate Managed Hot-Remove
+ — 2h = Simulate Surprise Hot-Remove */
+ options = options & 0x0F;
+ if (options == CXL_VPPB_UNBIND_WAIT_FOR_LINK_DOWN) {
+ // TODO: implement
+ return CXL_MBOX_INVALID_INPUT;
+ }
+ else if(options == CXL_VPPB_UNBIND_MANAGED_HOT_REMOVE) {
+ // unrealize listener will fire once guest notifies the port...
+ qdev_unplug(dspppb->dev, &local_err);
+ vppb_info->binding_status = CXL_VPPB_BINDING_STATUS_IN_PROGRESS;
+ if (local_err) {
+ return CXL_MBOX_INTERNAL_ERROR;
+ }
+ return CXL_MBOX_SUCCESS;
+ }
+ else if (options == CXL_VPPB_UNBIND_SURPRISE_HOT_REMOVE) {
+ // TODO: implement, This isn't a true surprise removal...
+ qdev_unplug(dspppb->dev, &local_err);
+ if (local_err) {
+ return CXL_MBOX_INTERNAL_ERROR;
+ }
+ object_unparent(OBJECT(dspppb->dev));
+ dspppb->is_bound = false;
+ dspppb->bound_vcs_id = 0;
+ dspppb->bound_vppb_id = 0;
+
+ /* Clear VPPB binding state so Get Virtual Switch Info reflects unbound */
+ vppb_info->binding_status = CXL_VPPB_BINDING_STATUS_UNBOUND;
+ vppb_info->bound_port_id = 0;
+ vppb_info->bound_ld_id = CXL_INVALID_BOUND_LD_ID;
+ dspppb->dev = NULL;
+ /* TODO: Generate Virtual CXL Switch Event Record per CXL spec
+ * section 7.6.6.6 */
+ return CXL_MBOX_SUCCESS;
+ }
+ else {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+
+}
+
+CXLRetCode cxl_vcs_unbind_vppb(CXLVCSSwitch *sw, uint8_t vcs_id,
+ uint8_t vppb_id, uint16_t option)
+{
+ /*TODO: Only currently unbinding an unrealizing whole device.
+ * Implement for MLD partial unbinding of single extents etc.
+ * */
+ return cxl_vcs_unbind_qdict_vppb(sw, vcs_id, vppb_id, option);
+}
+
+void cxl_vcs_identify_switch_device(CXLVCSSwitch *sw)
+{
+ //TODO: implement
+}
+void cxl_vcs_get_physical_port_state(CXLVCSSwitch *sw)
+{
+ //TODO: implement
+}
+void cxl_vcs_physical_port_control(CXLVCSSwitch *sw)
+{
+ //TODO: implement
+}
+void cxl_vcs_get_virtual_switch_info(CXLVCSSwitch *sw)
+{
+ //TODO: implement
+}
+
+void cxl_vcs_register_usp(CXLVCSSwitch *sw, CXLUpstreamPort *usp,
+ Error **errp)
+{
+ uint8_t ppb = usp->ppb;
+
+ if(strcmp(object_get_canonical_path_component(OBJECT(sw)), usp->vcs_name)) {
+ error_setg(errp, "VCS id for USP and switch do not match...\n");
+ }
+
+ if (ppb >= sw->num_usp_ppbs) {
+ error_setg(errp, "vcs '%s': ppb %u >= usp-ppbs %u",
+ object_get_canonical_path_component(OBJECT(sw)),
+ ppb, sw->num_usp_ppbs);
+ return;
+ }
+ if (sw->usp_ppbs[ppb]) {
+ error_setg(errp, "vcs '%s': USP slot %u already registered",
+ object_get_canonical_path_component(OBJECT(sw)), ppb);
+ return;
+ }
+
+ sw->usp_ppbs[ppb] = g_new0(CXLUpstreamPPB, 1);
+ sw->usp_ppbs[ppb]->usp = usp;
+ sw->usp_ppbs[ppb]->info = g_new0(CXLVCSInfoBlock, 1);
+ sw->usp_ppbs[ppb]->info->vcs_id = ppb;
+ sw->usp_ppbs[ppb]->info->usp_id = ppb;
+ sw->usp_ppbs[ppb]->info->num_vppbs = 0;
+ sw->usp_ppbs[ppb]->info->vcs_state = CXL_VCS_STATE_ENABLED;
+ usp->swcci.vcs = sw;
+}
+
+void cxl_vcs_register_vppb(CXLVCSSwitch *sw, CXLUpstreamPort *usp,
+ CXLDownstreamPort *dsp, Error **errp)
+{
+ CXLUpstreamPPB *vcs;
+
+ for(int i = 0; i < CXL_MAX_VCS_PORTS; i++) {
+ if(sw->usp_ppbs[i]) {
+ if(usp == sw->usp_ppbs[i]->usp) {
+ vcs = sw->usp_ppbs[i];
+ break;
+ }
+ } else {
+ error_setg(errp, "The USP was not found in the VCS list...");
+ return;
+ }
+ }
+
+ for(int i = 0; i < CXL_MAX_VPPB_PER_VCS; i++) {
+ if(!vcs->info->vppbs[i]) {
+ // Free vppb slot.. lets allocate and populate it...
+ CXLVPPBInfo *vppb = g_new0(CXLVPPBInfo, 1);
+ vppb->dsp = PCI_DEVICE(dsp);
+ vppb->binding_status = CXL_VPPB_BINDING_STATUS_UNBOUND;
+ vcs->info->vppbs[i] = vppb;
+ vcs->info->num_vppbs = vcs->info->num_vppbs + 1;
+ return;
+ }
+ }
+
+ error_setg(errp, "No free VPPB slots in the VCS...");
+ return;
+}
+
+void cxl_vcs_register_qdict_dsppb(CXLVCSSwitch *sw, const QDict *opts,
+ bool from_json, Error **errp)
+{
+ QDict *dev_opts;
+ int ppb;
+ const char *ppb_str = qdict_get_try_str(opts, "dsppb");
+ ppb = atoi(ppb_str);
+ if(ppb == -1) {
+ error_setg(errp, "No ppb id given in cli.");
+ return;
+ }
+
+ if (ppb >= sw->num_dsp_ppbs) {
+ error_setg(errp, "vcs '%s': dsppb %u >= dsp-ppbs %u",
+ object_get_canonical_path_component(OBJECT(sw)),
+ ppb, sw->num_dsp_ppbs);
+ return;
+ }
+ if (sw->dsp_ppbs[ppb]) {
+ error_setg(errp, "vcs '%s': DSP PPB slot %u already occupied",
+ object_get_canonical_path_component(OBJECT(sw)), ppb);
+ return;
+ }
+
+ dev_opts = qdict_clone_shallow(opts);
+ qdict_del(dev_opts, "vcs");
+ qdict_del(dev_opts, "dsppb");
+ qdict_del(dev_opts, "bus");
+
+ sw->dsp_ppbs[ppb] = g_new0(CXLDownstreamPPB, 1);
+ sw->dsp_ppbs[ppb]->opts = dev_opts;
+ sw->dsp_ppbs[ppb]->from_json = from_json;
+ sw->dsp_ppbs[ppb]->is_bound = false;
+}
+
+void cxl_vcs_register_dsppb(CXLVCSSwitch *sw, const QDict *opts,
+ bool from_json, Error **errp)
+{
+
+ DeviceState *dev = qdev_new(qdict_get_str(opts, "driver"));
+ qdev_set_id(dev, g_strdup(qdict_get_try_str(opts, "id")), errp);
+ int ppb;
+ const char *ppb_str = qdict_get_try_str(opts, "dsppb");
+ ppb = atoi(ppb_str);
+ if(ppb == -1) {
+ error_setg(errp, "No ppb id given in cli.");
+ return;
+ }
+
+ QDict *dev_opts = qdict_clone_shallow(opts);
+ qdict_del(dev_opts, "driver");
+ qdict_del(dev_opts, "bus");
+ qdict_del(dev_opts, "id");
+ qdict_del(dev_opts, "vcs");
+ qdict_del(dev_opts, "dsppb");
+ object_set_properties_from_keyval(OBJECT(dev), dev_opts, from_json, errp);
+ qobject_unref(dev_opts);
+ // store, don't realize
+ sw->dsp_ppbs[ppb]->dev = dev;
+}
+
+static bool cxl_vcs_switch_can_be_deleted(UserCreatable *uc)
+{
+ return false;
+}
+
+/* When a device is instantiated downstream of a VCS's PPB, we
+ * store the qdicts from the CLI, to realize the device at a
+ * future time.
+ */
+bool cxl_vcs_hide_device_listener(DeviceListener *listener, const QDict *opts,
+ bool from_json, Error **errp)
+{
+ CXLVCSSwitch *vcs = container_of(listener, CXLVCSSwitch, listener);
+ const char *vcs_id_str = qdict_get_try_str(opts, "vcs");
+ const char *ppb_str = qdict_get_try_str(opts, "dsppb");
+ int ppb;
+
+ /* Not our device — don't claim it */
+ if (!vcs_id_str || !ppb_str) {
+ return false;
+ }
+ if(vcs_id_str) {
+ if(strcmp(vcs_id_str,
+ object_get_canonical_path_component(OBJECT(vcs)))) {
+ return false;
+ }
+ }
+ if(ppb_str) {
+ ppb = atoi(ppb_str);
+ if(vcs->dsp_ppbs[ppb]) {
+ error_setg(errp,
+ "The ppb %s is already populated by another device.", ppb_str);
+ return false;
+ }
+ }
+
+ cxl_vcs_register_qdict_dsppb(vcs, opts, from_json, errp);
+
+ return true;
+}
+
+/*
+ * Listener is added for unbinding. When a device is unbound using the
+ * Fabric Manager with 'managed' hot-remove option, a notificaiton is
+ * sent to the guest, which should perform a graceful teardown and notify
+ * the port of completion. At this point this listener unrealizes the device.
+ */
+static void cxl_vcs_ppb_unrealize_listener(DeviceListener *listener,
+ DeviceState *dev)
+{
+ CXLVCSSwitch *sw = container_of(listener, CXLVCSSwitch, listener);
+ CXLDownstreamPPB *dspppb;
+ CXLVPPBInfo *vppb_info;
+
+ for (int i = 0; i < sw->num_dsp_ppbs; i++) {
+ if (sw->dsp_ppbs[i] && (sw->dsp_ppbs[i]->dev == dev)) {
+ dspppb = sw->dsp_ppbs[i];
+ break;
+ }
+ }
+ if(!dspppb) {
+ return;
+ }
+
+ vppb_info =
+ sw->usp_ppbs[dspppb->bound_vcs_id]->info->vppbs[dspppb->bound_vppb_id];
+ object_unparent(OBJECT(dspppb->dev));
+ dspppb->dev = NULL;
+ dspppb->is_bound = false;
+ dspppb->bound_vcs_id = 0;
+ dspppb->bound_vppb_id = 0;
+ vppb_info->binding_status = CXL_VPPB_BINDING_STATUS_UNBOUND;
+ vppb_info->bound_port_id = 0;
+ vppb_info->bound_ld_id = CXL_INVALID_BOUND_LD_ID;
+ /* TODO: generate Virtual CXL Switch Event Record */
+}
+
+static void cxl_vcs_switch_complete(UserCreatable *uc, Error **errp)
+{
+ CXLVCSSwitch *sw = CXL_VCS_SWITCH(uc);
+ sw->listener.hide_device = cxl_vcs_hide_device_listener;
+ sw->listener.unrealize = cxl_vcs_ppb_unrealize_listener;
+ device_listener_register(&sw->listener);
+}
+
+static void vcs_get_usp_ppbs(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ CXLVCSSwitch *vcs = CXL_VCS_SWITCH(obj);
+ uint8_t val = vcs->num_usp_ppbs;
+ visit_type_uint8(v, name, &val, errp);
+}
+
+static void vcs_set_usp_ppbs(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ CXLVCSSwitch *vcs = CXL_VCS_SWITCH(obj);
+ uint8_t val;
+ if (!visit_type_uint8(v, name, &val, errp)) {
+ return;
+ }
+ vcs->num_usp_ppbs = val;
+}
+static void vcs_get_dsp_ppbs(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ CXLVCSSwitch *vcs = CXL_VCS_SWITCH(obj);
+ uint8_t val = vcs->num_dsp_ppbs;
+ visit_type_uint8(v, name, &val, errp);
+}
+
+static void vcs_set_dsp_ppbs(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ CXLVCSSwitch *vcs = CXL_VCS_SWITCH(obj);
+ uint8_t val;
+ if (!visit_type_uint8(v, name, &val, errp)) {
+ return;
+ }
+ vcs->num_dsp_ppbs = val;
+}
+
+static void cxl_vcs_class_init(ObjectClass *oc, const void *data)
+{
+ CXLVCSSwitchClass *cc = CXL_VCS_SWITCH_CLASS(oc);
+ UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
+ ucc->complete = cxl_vcs_switch_complete;
+ ucc->can_be_deleted = cxl_vcs_switch_can_be_deleted;
+ cc->identify_switch_device = cxl_vcs_identify_switch_device;
+ cc->get_physical_port_state = cxl_vcs_get_physical_port_state;
+ cc->physical_port_control = cxl_vcs_physical_port_control;
+ cc->get_virtual_switch_info = cxl_vcs_get_virtual_switch_info;
+ cc->bind_vppb = cxl_vcs_bind_vppb;
+ cc->unbind_vppb = cxl_vcs_unbind_vppb;
+
+ object_class_property_add_bool(oc, "local-fm",
+ cxl_vcs_get_local_fm, cxl_vcs_set_local_fm);
+ object_class_property_set_description(oc, "local-fm",
+ "true = FM authority (mctp connected guest), \
+ false = slave listener (IPC)");
+ object_class_property_add(oc, "usp-ppbs", "uint8",
+ vcs_get_usp_ppbs, vcs_set_usp_ppbs,
+ NULL, NULL);
+ object_class_property_set_description(oc, "usp-ppbs",
+ "Number of upstream ports in the switch");
+ object_class_property_add(oc, "dsp-ppbs", "uint8",
+ vcs_get_dsp_ppbs, vcs_set_dsp_ppbs,
+ NULL, NULL);
+ object_class_property_set_description(oc, "dsp-ppbs",
+ "Number of downstream ports in the switch");
+}
+
+static void cxl_vcs_instance_init(Object *obj)
+{
+ // TODO: Nothing here yet...
+}
+
+static void cxl_vcs_instance_finalize(Object *obj)
+{
+ // TODO: Nothing here yet...
+}
+
+static const InterfaceInfo cxl_vcs_interfaces[] = {
+ { TYPE_USER_CREATABLE },
+ { }
+};
+
+static const TypeInfo cxl_vcs_info = {
+ .name = TYPE_CXL_VCS_SWITCH,
+ .parent = TYPE_OBJECT,
+ .instance_size = sizeof(CXLVCSSwitch),
+ .class_size = sizeof(CXLVCSSwitchClass),
+ .class_init = cxl_vcs_class_init,
+ .instance_init = cxl_vcs_instance_init,
+ .instance_finalize = cxl_vcs_instance_finalize,
+ .interfaces = cxl_vcs_interfaces,
+};
+
+static void cxl_vcs_register(void)
+{
+ type_register_static(&cxl_vcs_info);
+}
+
+type_init(cxl_vcs_register)
diff --git a/hw/cxl/meson.build b/hw/cxl/meson.build
index ccad565c5c..c37563cd05 100644
--- a/hw/cxl/meson.build
+++ b/hw/cxl/meson.build
@@ -7,6 +7,7 @@ system_ss.add(when: 'CONFIG_CXL',
'cxl-cdat.c',
'cxl-events.c',
'switch-mailbox-cci.c',
+ 'cxl-vcs-switch.c',
),
if_false: files(
'cxl-host-stubs.c',
diff --git a/include/hw/cxl/cxl_vcs_switch.h b/include/hw/cxl/cxl_vcs_switch.h
new file mode 100644
index 0000000000..6576870bf3
--- /dev/null
+++ b/include/hw/cxl/cxl_vcs_switch.h
@@ -0,0 +1,134 @@
+/*
+ * CXL VCS Capable Switch Object Header
+ *
+ * Copyright(C) 2026 University of Manchester
+ * Author: Joshua Lant <joshualant@gmail.com>.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See the
+ * COPYING file in the top-level directory.
+ *
+ * SPDX-License-Identifier: GPL-v2-only
+ */
+
+#ifndef CXL_VCS_SWITCH_H
+#define CXL_VCS_SWITCH_H
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qom/object.h"
+#include "qom/object_interfaces.h"
+#include "hw/pci-bridge/cxl_upstream_port.h"
+#include "hw/pci-bridge/cxl_downstream_port.h"
+#include "include/hw/cxl/cxl_device.h"
+
+#define TYPE_CXL_VCS_SWITCH "cxl-vcs-switch"
+OBJECT_DECLARE_TYPE(CXLVCSSwitch, CXLVCSSwitchClass, CXL_VCS_SWITCH);
+
+#define CXL_VPPB_BINDING_STATUS_UNBOUND 0x00
+#define CXL_VPPB_BINDING_STATUS_IN_PROGRESS 0x01
+#define CXL_VPPB_BINDING_STATUS_BOUND_PORT 0x02
+#define CXL_VPPB_BINDING_STATUS_BOUND_LD 0x03
+#define CXL_VPPB_BINDING_STATUS_BOUND_PID 0x04
+
+#define CXL_VPPB_UNBIND_WAIT_FOR_LINK_DOWN 0x0
+#define CXL_VPPB_UNBIND_MANAGED_HOT_REMOVE 0x1
+#define CXL_VPPB_UNBIND_SURPRISE_HOT_REMOVE 0x2
+
+#define CXL_UNSUPPORTED_LD_ID 0xFFFF
+#define CXL_INVALID_BOUND_LD_ID 0xFF
+
+#define CXL_VCS_STATE_INVALID_ID 0xFF
+#define CXL_VCS_STATE_DISABLED 0x0
+#define CXL_VCS_STATE_ENABLED 0x1
+
+#define CXL_MAX_VCS_PORTS 0x8
+#define CXL_MAX_VPPB_PER_VCS 0x8
+#define CXL_MAX_USP_PPBS CXL_MAX_VCS_PORTS
+#define CXL_MAX_DSP_PPBS CXL_MAX_VPPB_PER_VCS
+
+#define CXL_MAX_PPBS (CXL_MAX_USP_PPBS + CXL_MAX_DSP_PPBS)
+#define VPPB_LIST_LIMIT 8
+
+struct CXLVCSSwitchClass {
+ ObjectClass parent_class;
+ void (*identify_switch_device)(CXLVCSSwitch *sw);
+ void (*get_physical_port_state)(CXLVCSSwitch *sw);
+ void (*physical_port_control)(CXLVCSSwitch *sw);
+ void (*get_virtual_switch_info)(CXLVCSSwitch *sw);
+ CXLRetCode (*bind_vppb)(CXLVCSSwitch *sw, uint8_t vcs_id, uint8_t vppb_id,
+ uint8_t dsppb_id, uint16_t ld_id);
+ CXLRetCode (*unbind_vppb)(CXLVCSSwitch *sw, uint8_t vcs_id, uint8_t vppb_id,
+ uint16_t option);
+};
+
+/* CXL r3.2 Table 7-32: Get Virtual CXL Switch Info VCS Info Block Format */
+typedef struct CXLVPPBInfo {
+ PCIDevice *dsp;
+ uint8_t binding_status;
+ uint8_t bound_port_id;
+ uint8_t bound_ld_id;
+ uint8_t rsv1;
+} CXLVPPBInfo;
+
+typedef struct CXLVCSInfoBlock {
+ uint8_t vcs_id;
+ uint8_t vcs_state;
+ uint8_t usp_id;
+ uint8_t num_vppbs;
+ struct CXLVPPBInfo *vppbs[CXL_MAX_VPPB_PER_VCS];
+} CXLVCSInfoBlock;
+
+/* Physical Upstream and Downstream PCI-PCI Bridge (PPBs) structs */
+typedef struct CXLUpstreamPPB {
+ CXLUpstreamPort *usp;
+ struct CXLVCSInfoBlock *info;
+} CXLUpstreamPPB;
+
+typedef struct CXLDownstreamPPB {
+ DeviceState *dev;
+ PCIDevice *pdev;
+ // store opts and json in case of simple device hiding.
+ QDict *opts;
+ bool from_json;
+ bool is_bound;
+ uint8_t bound_vcs_id;
+ uint8_t bound_vppb_id;
+} CXLDownstreamPPB;
+
+typedef struct CXLVCSSwitch {
+ Object parent;
+ uint8_t num_usp_ppbs;
+ uint8_t num_dsp_ppbs;
+ uint8_t num_vppbs_per_vcs;
+ CXLCCI swcci;
+ CXLCCI mctpcci;
+ DeviceListener listener;
+ CXLUpstreamPPB *usp_ppbs[CXL_MAX_USP_PPBS];
+ CXLDownstreamPPB *dsp_ppbs[CXL_MAX_DSP_PPBS];
+ // true if FM is in this guest, false if remote FM.
+ bool local_fm;
+} CXLVCSSwitch;
+
+// Called by USP/DSP/EP CXL devices to build the VCS structures.
+void cxl_vcs_register_usp(CXLVCSSwitch *sw, CXLUpstreamPort *usp, Error **errp);
+void cxl_vcs_register_vppb(CXLVCSSwitch *sw, CXLUpstreamPort *usp,
+ CXLDownstreamPort *dsp, Error **errp);
+void cxl_vcs_register_dsppb(CXLVCSSwitch *sw, const QDict *opts,
+ bool from_json, Error **errp);
+void cxl_vcs_register_qdict_dsppb(CXLVCSSwitch *sw, const QDict *opts,
+ bool from_json, Error **errp);
+bool cxl_vcs_hide_device_listener(DeviceListener *listener,
+ const QDict *device_opts, bool from_json, Error **errp);
+
+
+// Called by the CCI mailbox utils for FMAPI control of VCS.
+CXLRetCode cxl_vcs_bind_vppb(CXLVCSSwitch *sw, uint8_t vcs_id, uint8_t vppb_id,
+ uint8_t dsppb_id, uint16_t ld_id);
+CXLRetCode cxl_vcs_unbind_vppb(CXLVCSSwitch *sw, uint8_t vcs_id,
+ uint8_t vppb_id, uint16_t option);
+void cxl_vcs_identify_switch_device(CXLVCSSwitch *sw);
+void cxl_vcs_get_physical_port_state(CXLVCSSwitch *sw);
+void cxl_vcs_physical_port_control(CXLVCSSwitch *sw);
+void cxl_vcs_get_virtual_switch_info(CXLVCSSwitch *sw);
+
+#endif /* CXL_VCS_SWITCH_H */
diff --git a/qapi/qom.json b/qapi/qom.json
index 6f5c9de0f0..f66ef6b68b 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -357,6 +357,23 @@
'data': { 'chardev': 'str' },
'if': 'CONFIG_VHOST_CRYPTO' }
+##
+# @CXLVCSSwitchProperties:
+#
+# Properties for cxl-vcs-switch objects.
+#
+# @usp-ppbs: number of physical upstream PPBs in the switch
+# @dsp-ppbs: number of physical downstream PPBs in the switch
+# @local-fm: true if this instance of qemu contains the MCTP device
+# (FM capabilities), false if vcs will listen for incoming traffic
+# from the remote FM.
+##
+{ 'struct': 'CXLVCSSwitchProperties',
+ 'data': {
+ '*usp-ppbs': 'uint8',
+ '*dsp-ppbs': 'uint8',
+ '*local-fm': 'bool'} }
+
##
# @DBusVMStateProperties:
#
@@ -1194,6 +1211,7 @@
'cryptodev-backend-lkcf',
{ 'name': 'cryptodev-vhost-user',
'if': 'CONFIG_VHOST_CRYPTO' },
+ 'cxl-vcs-switch',
'dbus-vmstate',
'filter-buffer',
'filter-dump',
@@ -1272,6 +1290,7 @@
'cryptodev-backend-lkcf': 'CryptodevBackendProperties',
'cryptodev-vhost-user': { 'type': 'CryptodevVhostUserProperties',
'if': 'CONFIG_VHOST_CRYPTO' },
+ 'cxl-vcs-switch': 'CXLVCSSwitchProperties',
'dbus-vmstate': 'DBusVMStateProperties',
'filter-buffer': 'FilterBufferProperties',
'filter-dump': 'FilterDumpProperties',
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [RFC QEMU PATCH 06/10] cxl-upstream-port: Add support for targeting a VCS switch
2026-04-29 13:48 [RFC QEMU PATCH 00/10] Initial Support for VCS Switching Joshua Lant
` (4 preceding siblings ...)
2026-04-29 13:48 ` [RFC QEMU PATCH 05/10] cxl-vcs-switch: Initial support for CXL VCS Joshua Lant
@ 2026-04-29 13:48 ` Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 07/10] cxl-downstream-port: Add support for VCS switching Joshua Lant
` (3 subsequent siblings)
9 siblings, 0 replies; 19+ messages in thread
From: Joshua Lant @ 2026-04-29 13:48 UTC (permalink / raw)
To: linux-cxl; +Cc: qemu-devel, Jonathan.Cameron, arpit1.kumar, Joshua Lant
Allows an upstream port to be registered on the CLI as a VCS within a
cxl-vcs-switch. A VCS is defined as an upstream pci-pci bridge, and a set of
downstream virtual pci-pci bridges VPPBs, which form a single Virtual
Hierarchy within the switch.
Signed-off-by: Joshua Lant <joshualant@gmail.com>
---
hw/pci-bridge/cxl_upstream.c | 20 ++++++++++++++++++++
include/hw/pci-bridge/cxl_upstream_port.h | 2 ++
2 files changed, 22 insertions(+)
diff --git a/hw/pci-bridge/cxl_upstream.c b/hw/pci-bridge/cxl_upstream.c
index 3fcfc2cde6..a91f721c9e 100644
--- a/hw/pci-bridge/cxl_upstream.c
+++ b/hw/pci-bridge/cxl_upstream.c
@@ -16,6 +16,8 @@
#include "hw/pci/pcie.h"
#include "hw/pci/pcie_port.h"
#include "hw/pci-bridge/cxl_upstream_port.h"
+#include "hw/cxl/cxl_vcs_switch.h"
+
/*
* Null value of all Fs suggested by IEEE RA guidelines for use of
* EU, OUI and CID
@@ -346,6 +348,22 @@ static void cxl_usp_realize(PCIDevice *d, Error **errp)
goto err_cap;
}
+ if (usp->vcs_name && usp->ppb != UINT8_MAX) {
+ Object *obj = object_resolve_path_component(
+ object_get_objects_root(), usp->vcs_name);
+ if (!obj) {
+ error_setg(errp, "vcs '%s' not found", usp->vcs_name);
+ goto err_cap;
+ }
+ if (!object_dynamic_cast(obj, TYPE_CXL_VCS_SWITCH)) {
+ error_setg(errp, "'%s' is not a cxl-vcs-switch", usp->vcs_name);
+ goto err_cap;
+ }
+ cxl_vcs_register_usp(CXL_VCS_SWITCH(obj), usp, errp);
+ if (*errp)
+ goto err_cap;
+ }
+
return;
err_cap:
@@ -372,6 +390,8 @@ static const Property cxl_upstream_props[] = {
DEFINE_PROP_PCIE_LINK_WIDTH("x-width", CXLUpstreamPort,
width, PCIE_LINK_WIDTH_16),
DEFINE_PROP_BOOL("x-256b-flit", CXLUpstreamPort, flitmode, false),
+ DEFINE_PROP_STRING("vcs", CXLUpstreamPort, vcs_name),
+ DEFINE_PROP_UINT8("usppb", CXLUpstreamPort, ppb, UINT8_MAX),
};
static void cxl_upstream_class_init(ObjectClass *oc, const void *data)
diff --git a/include/hw/pci-bridge/cxl_upstream_port.h b/include/hw/pci-bridge/cxl_upstream_port.h
index ce248f3dca..05632286db 100644
--- a/include/hw/pci-bridge/cxl_upstream_port.h
+++ b/include/hw/pci-bridge/cxl_upstream_port.h
@@ -22,6 +22,8 @@ typedef struct CXLUpstreamPort {
DOECap doe_cdat;
uint64_t sn;
+ char *vcs_name;
+ uint8_t ppb;
} CXLUpstreamPort;
#endif /* CXL_SUP_H */
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [RFC QEMU PATCH 07/10] cxl-downstream-port: Add support for VCS switching
2026-04-29 13:48 [RFC QEMU PATCH 00/10] Initial Support for VCS Switching Joshua Lant
` (5 preceding siblings ...)
2026-04-29 13:48 ` [RFC QEMU PATCH 06/10] cxl-upstream-port: Add support for targeting a VCS switch Joshua Lant
@ 2026-04-29 13:48 ` Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 08/10] cxl-cci-mailbox: Add support for targeting a VCS switch Joshua Lant
` (2 subsequent siblings)
9 siblings, 0 replies; 19+ messages in thread
From: Joshua Lant @ 2026-04-29 13:48 UTC (permalink / raw)
To: linux-cxl; +Cc: qemu-devel, Jonathan.Cameron, arpit1.kumar, Joshua Lant
When a downstream port is connnected to an upstream port which is
defined as a VCS within a cxl-vcs-switch, the port will automatically be
instantiated as a VPPB within that VCS. This downstream port will then
appear normally to the guest, but can be connected to a real downstream
PPB at runtime when the switch is issued a bind command by the Fabric
Manager.
Signed-off-by: Joshua Lant <joshualant@gmail.com>
---
hw/pci-bridge/cxl_downstream.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/hw/pci-bridge/cxl_downstream.c b/hw/pci-bridge/cxl_downstream.c
index 5322c46900..5e7ce6410d 100644
--- a/hw/pci-bridge/cxl_downstream.c
+++ b/hw/pci-bridge/cxl_downstream.c
@@ -14,11 +14,14 @@
#include "hw/pci/pcie.h"
#include "hw/pci/pcie_port.h"
#include "hw/pci-bridge/cxl_downstream_port.h"
+#include "hw/pci-bridge/cxl_upstream_port.h"
#include "hw/qdev-properties.h"
#include "hw/qdev-properties-system.h"
#include "hw/cxl/cxl.h"
#include "hw/cxl/cxl_port.h"
#include "qapi/error.h"
+#include "hw/cxl/cxl_vcs_switch.h"
+
typedef struct CXLDownstreamPort {
/*< private >*/
@@ -199,6 +202,14 @@ static void cxl_dsp_realize(PCIDevice *d, Error **errp)
PCI_BASE_ADDRESS_MEM_TYPE_64,
component_bar);
+ PCIDevice *parent = pci_bridge_get_device(pci_get_bus(PCI_DEVICE(d)));
+ if (parent && object_dynamic_cast(OBJECT(parent), TYPE_CXL_USP)) {
+ CXLUpstreamPort *usp = CXL_USP(parent);
+ if (usp->vcs_name) {
+ cxl_vcs_register_vppb(usp->swcci.vcs, usp, dsp, errp);
+ }
+ }
+
return;
err_chassis:
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [RFC QEMU PATCH 08/10] cxl-cci-mailbox: Add support for targeting a VCS switch
2026-04-29 13:48 [RFC QEMU PATCH 00/10] Initial Support for VCS Switching Joshua Lant
` (6 preceding siblings ...)
2026-04-29 13:48 ` [RFC QEMU PATCH 07/10] cxl-downstream-port: Add support for VCS switching Joshua Lant
@ 2026-04-29 13:48 ` Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 09/10] cxl-mailbox-utils: Add support for VCS bind/unbind commands Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 10/10] cxl-mailbox-utils: Add support for VCS Get Virtual CXL Switch Info command Joshua Lant
9 siblings, 0 replies; 19+ messages in thread
From: Joshua Lant @ 2026-04-29 13:48 UTC (permalink / raw)
To: linux-cxl; +Cc: qemu-devel, Jonathan.Cameron, arpit1.kumar, Joshua Lant
Allows a cci mailbox and mctp device to target a cxl-vcs-switch. The
target needs to be changed from a PCIDevice to a more generic Object,
since the target can now also be a cxl-vcs-switch, as well as USP or
type3 device. The VCS is an object which is composed of many
PCI devices, but cannot be classed as one itself (since it is busless).
Signed-off-by: Joshua Lant <joshualant@gmail.com>
---
hw/cxl/cxl-mailbox-utils.c | 44 +++++++++++++++++++++++--------------
hw/cxl/switch-mailbox-cci.c | 33 ++++++++++++++++++++++------
hw/usb/dev-mctp.c | 23 ++++++++++++++++---
include/hw/cxl/cxl_device.h | 10 ++++++++-
4 files changed, 83 insertions(+), 27 deletions(-)
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index f12e42a648..779138cdc8 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -4600,22 +4600,24 @@ int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
}
/* forbid any selected commands while the media is disabled */
- if (object_dynamic_cast(OBJECT(cci->d), TYPE_CXL_TYPE3)) {
- cxl_dstate = &CXL_TYPE3(cci->d)->cxl_dstate;
-
- if (cxl_dev_media_disabled(cxl_dstate)) {
- if (h == cmd_events_get_records ||
- h == cmd_ccls_get_partition_info ||
- h == cmd_ccls_set_lsa ||
- h == cmd_ccls_get_lsa ||
- h == cmd_logs_get_log ||
- h == cmd_media_get_poison_list ||
- h == cmd_media_inject_poison ||
- h == cmd_media_clear_poison ||
- h == cmd_sanitize_overwrite ||
- h == cmd_firmware_update_transfer ||
- h == cmd_firmware_update_activate) {
- return CXL_MBOX_MEDIA_DISABLED;
+ if(cci->d) {
+ if (object_dynamic_cast(OBJECT(cci->d), TYPE_CXL_TYPE3)) {
+ cxl_dstate = &CXL_TYPE3(cci->d)->cxl_dstate;
+
+ if (cxl_dev_media_disabled(cxl_dstate)) {
+ if (h == cmd_events_get_records ||
+ h == cmd_ccls_get_partition_info ||
+ h == cmd_ccls_set_lsa ||
+ h == cmd_ccls_get_lsa ||
+ h == cmd_logs_get_log ||
+ h == cmd_media_get_poison_list ||
+ h == cmd_media_inject_poison ||
+ h == cmd_media_clear_poison ||
+ h == cmd_sanitize_overwrite ||
+ h == cmd_firmware_update_transfer ||
+ h == cmd_firmware_update_activate) {
+ return CXL_MBOX_MEDIA_DISABLED;
+ }
}
}
}
@@ -4909,3 +4911,13 @@ void cxl_initialize_usp_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
cci->intf = intf;
cxl_init_cci(cci, payload_max);
}
+
+void cxl_initialize_vcs_mctpcci(CXLCCI *cci, CXLVCSSwitch *vcs, DeviceState *d,
+ DeviceState *intf, size_t payload_max)
+{
+ cxl_copy_cci_commands(cci, cxl_cmd_set_usp_mctp);
+ cci->vcs = vcs;
+ cci->d = d;
+ cci->intf = intf;
+ cxl_init_cci(cci, payload_max);
+}
diff --git a/hw/cxl/switch-mailbox-cci.c b/hw/cxl/switch-mailbox-cci.c
index 223f220433..73f76484cc 100644
--- a/hw/cxl/switch-mailbox-cci.c
+++ b/hw/cxl/switch-mailbox-cci.c
@@ -16,6 +16,7 @@
#include "qemu/module.h"
#include "hw/qdev-properties.h"
#include "hw/cxl/cxl.h"
+#include "include/hw/cxl/cxl_vcs_switch.h"
#define CXL_SWCCI_MSIX_MBOX 3
@@ -32,17 +33,26 @@ static void cswbcci_realize(PCIDevice *pci_dev, Error **errp)
CXLDeviceState *cxl_dstate = &cswmb->cxl_dstate;
CXLDVSECRegisterLocator *regloc_dvsec;
CXLUpstreamPort *usp;
+ CXLVCSSwitch *vcs = 0;
if (!cswmb->target) {
- error_setg(errp, "Target not set");
+ error_setg(errp, "No target or vcs has been set...");
return;
}
- usp = CXL_USP(cswmb->target);
pcie_endpoint_cap_init(pci_dev, 0x80);
cxl_cstate->dvsec_offset = 0x100;
cxl_cstate->pdev = pci_dev;
- cswmb->cci = &usp->swcci;
+ if (object_dynamic_cast(cswmb->target, TYPE_CXL_VCS_SWITCH)) {
+ vcs = CXL_VCS_SWITCH(cswmb->target);
+ cswmb->cci = &vcs->swcci;
+ } else if (object_dynamic_cast(cswmb->target, TYPE_CXL_USP)) {
+ usp = CXL_USP(cswmb->target);
+ cswmb->cci = &usp->swcci;
+ } else {
+ error_setg(errp, "Target must be a VCS switch or USP");
+ return;
+ }
cxl_device_register_block_init(OBJECT(pci_dev), cxl_dstate, cswmb->cci);
pci_register_bar(pci_dev, 0,
PCI_BASE_ADDRESS_SPACE_MEMORY |
@@ -57,9 +67,18 @@ static void cswbcci_realize(PCIDevice *pci_dev, Error **errp)
REG_LOC_DVSEC_LENGTH, REG_LOC_DVSEC,
REG_LOC_DVSEC_REVID, (uint8_t *)regloc_dvsec);
- cxl_initialize_mailbox_swcci(cswmb->cci, DEVICE(pci_dev),
- DEVICE(cswmb->target),
- CXL_MAILBOX_MAX_PAYLOAD_SIZE);
+ if(vcs) {
+ if(vcs->usp_ppbs[0]) {
+ cxl_initialize_vcs_mctpcci(cswmb->cci,
+ vcs, DEVICE(vcs->usp_ppbs[0]->usp), DEVICE(pci_dev), CXL_MAILBOX_MAX_PAYLOAD_SIZE);
+ } else {
+ error_setg(errp, "The VCS requires a master USP...");
+ }
+ } else {
+ cxl_initialize_mailbox_swcci(cswmb->cci, DEVICE(pci_dev),
+ DEVICE(CXL_USP(cswmb->target)),
+ CXL_MAILBOX_MAX_PAYLOAD_SIZE);
+ }
}
static void cswmbcci_exit(PCIDevice *pci_dev)
@@ -69,7 +88,7 @@ static void cswmbcci_exit(PCIDevice *pci_dev)
static const Property cxl_switch_cci_props[] = {
DEFINE_PROP_LINK("target", CSWMBCCIDev,
- target, TYPE_CXL_USP, PCIDevice *),
+ target, TYPE_OBJECT, Object *),
};
static void cswmbcci_class_init(ObjectClass *oc, const void *data)
diff --git a/hw/usb/dev-mctp.c b/hw/usb/dev-mctp.c
index aafb9e7e96..792a375245 100644
--- a/hw/usb/dev-mctp.c
+++ b/hw/usb/dev-mctp.c
@@ -25,6 +25,7 @@
#include "hw/usb.h"
#include "hw/usb/desc.h"
#include "net/mctp.h"
+#include "include/hw/cxl/cxl_vcs_switch.h"
/* TODO: Move to header */
@@ -77,7 +78,7 @@ enum cxl_dev_type {
typedef struct USBCXLMCTPState {
USBDevice dev;
- PCIDevice *target;
+ Object *target;
CXLCCI *cci;
enum cxl_dev_type type;
USBPacket *cached_tohost;
@@ -600,12 +601,28 @@ static void usb_cxl_mctp_realize(USBDevice *dev, Error **errp)
return;
}
+ if (object_dynamic_cast(OBJECT(s->target), TYPE_CXL_VCS_SWITCH)) {
+ CXLVCSSwitch *sw = CXL_VCS_SWITCH(s->target);
+
+ s->type = cxl_switch;
+ s->cci = &sw->mctpcci;
+ s->cci->vcs = sw;
+
+ if(sw->usp_ppbs[0]) {
+ cxl_initialize_vcs_mctpcci(s->cci, CXL_VCS_SWITCH(s->target),
+ DEVICE(sw->usp_ppbs[0]->usp), DEVICE(dev), MCTP_CXL_MAILBOX_BYTES);
+ } else {
+ error_setg(errp, "No master USP associated with the VCS");
+ }
+ return;
+ }
+
error_setg(errp, "Unhandled target type for CXL MCTP EP");
}
static const Property usb_cxl_mctp_properties[] = {
- DEFINE_PROP_LINK("target", USBCXLMCTPState, target, TYPE_PCI_DEVICE,
- PCIDevice *),
+ DEFINE_PROP_LINK("target", USBCXLMCTPState, target, TYPE_OBJECT,
+ Object *),
};
static void usb_cxl_mctp_class_initfn(ObjectClass *klass, const void *data)
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 6158560e6c..5d9f537a03 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -143,6 +143,7 @@ typedef enum {
} CXLDSMASFlags;
typedef struct CXLCCI CXLCCI;
+typedef struct CXLVCSSwitch CXLVCSSwitch;
typedef struct cxl_device_state CXLDeviceState;
struct cxl_cmd;
typedef CXLRetCode (*opcode_handler)(const struct cxl_cmd *cmd,
@@ -212,6 +213,8 @@ typedef struct CXLCCI {
DeviceState *d;
/* Pointer to the device hosting the protocol conversion */
DeviceState *intf;
+ /* Pointer to the vcs hosting the CCI (if used) */
+ CXLVCSSwitch *vcs;
bool initialized;
} CXLCCI;
@@ -349,6 +352,10 @@ void cxl_initialize_t3_ld_cci(CXLCCI *cci, DeviceState *d,
void cxl_initialize_usp_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
size_t payload_max);
+void cxl_initialize_vcs_mctpcci(CXLCCI *cci, CXLVCSSwitch *vcs,
+ DeviceState *d, DeviceState *intf, size_t payload_max);
+
+
#define cxl_device_cap_init(dstate, reg, cap_id, ver) \
do { \
uint32_t *cap_hdrs = dstate->caps_reg_state32; \
@@ -820,9 +827,10 @@ struct CXLType3Class {
uint8_t *data);
};
+
struct CSWMBCCIDev {
PCIDevice parent_obj;
- PCIDevice *target;
+ Object *target;
CXLComponentState cxl_cstate;
CXLDeviceState cxl_dstate;
CXLCCI *cci;
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [RFC QEMU PATCH 09/10] cxl-mailbox-utils: Add support for VCS bind/unbind commands.
2026-04-29 13:48 [RFC QEMU PATCH 00/10] Initial Support for VCS Switching Joshua Lant
` (7 preceding siblings ...)
2026-04-29 13:48 ` [RFC QEMU PATCH 08/10] cxl-cci-mailbox: Add support for targeting a VCS switch Joshua Lant
@ 2026-04-29 13:48 ` Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 10/10] cxl-mailbox-utils: Add support for VCS Get Virtual CXL Switch Info command Joshua Lant
9 siblings, 0 replies; 19+ messages in thread
From: Joshua Lant @ 2026-04-29 13:48 UTC (permalink / raw)
To: linux-cxl; +Cc: qemu-devel, Jonathan.Cameron, arpit1.kumar, Joshua Lant
Adds support for FMAPI commands 0x5201 and 0x5202, for more
information see CXL Specification v3.2 (7.6.7.2.2 and 7.6.7.2.3).
Signed-off-by: Joshua Lant <joshualant@gmail.com>
---
hw/cxl/cxl-mailbox-utils.c | 81 +++++++++++++++++++++++++++++++++++++-
1 file changed, 80 insertions(+), 1 deletion(-)
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 779138cdc8..cf6cb65eb6 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -27,6 +27,7 @@
#include "system/hostmem.h"
#include "qemu/range.h"
#include "qapi/qapi-types-cxl.h"
+#include "hw/cxl/cxl_vcs_switch.h"
#define CXL_CAPACITY_MULTIPLIER (256 * MiB)
#define CXL_DC_EVENT_LOG_SIZE 8
@@ -121,6 +122,9 @@ enum {
#define IDENTIFY_SWITCH_DEVICE 0x0
#define GET_PHYSICAL_PORT_STATE 0x1
#define PHYSICAL_PORT_CONTROL 0x2
+ VIRTUAL_SWITCH = 0x52,
+ #define BIND_VPPB 0x1
+ #define UNBIND_VPPB 0x2
TUNNEL = 0x53,
#define MANAGEMENT_COMMAND 0x0
FMAPI_DCD_MGMT = 0x56,
@@ -821,6 +825,51 @@ static CXLRetCode cmd_physical_port_control(const struct cxl_cmd *cmd,
}
}
+/* CXL r3.2 Section 7.6.7.2.2: Bind vPPB (Opcode 5201h) */
+static CXLRetCode cmd_bind_vppb(const struct cxl_cmd *cmd,
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLCCI *cci)
+{
+ struct cxl_fmapi_bind_vppb_req_pl {
+ uint8_t vcs_id;
+ uint8_t vppb_id;
+ uint8_t physp_id;
+ uint8_t rsvd;
+ uint16_t ld_id;
+ } QEMU_PACKED *in = (void *)payload_in;
+ if(!cci->vcs) {
+ return CXL_MBOX_UNSUPPORTED;
+ }
+ // Multi-logical device currently not supported (force 0xFFFF).
+ if(in->ld_id != CXL_UNSUPPORTED_LD_ID) {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+ return cxl_vcs_bind_vppb(cci->vcs, in->vcs_id, in->vppb_id,
+ in->physp_id, CXL_UNSUPPORTED_LD_ID);
+}
+
+/* CXL r3.2 Section 7.6.7.2.3: Unbind vPPB (Opcode 5202h) */
+static CXLRetCode cmd_unbind_vppb(const struct cxl_cmd *cmd,
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLCCI *cci)
+{
+ struct cxl_fmapi_bind_vppb_req_pl {
+ uint8_t vcs_id;
+ uint8_t vppb_id;
+ uint16_t option;
+ } QEMU_PACKED *in = (void *)payload_in;
+ if(!cci->vcs) {
+ return CXL_MBOX_UNSUPPORTED;
+ }
+ return cxl_vcs_unbind_vppb(cci->vcs, in->vcs_id, in->vppb_id, in->option);
+}
+
/* CXL r3.1 Section 8.2.9.1.2: Background Operation Status (Opcode 0002h) */
static CXLRetCode cmd_infostat_bg_op_sts(const struct cxl_cmd *cmd,
uint8_t *payload_in,
@@ -4903,6 +4952,36 @@ static const struct cxl_cmd cxl_cmd_set_usp_mctp[256][256] = {
cmd_tunnel_management_cmd, ~0, 0 },
};
+static const struct cxl_cmd cxl_cmd_set_vcs_mctp[256][256] = {
+ [INFOSTAT][IS_IDENTIFY] = { "IDENTIFY", cmd_infostat_identify, 0, 0 },
+ [INFOSTAT][GET_RESPONSE_MSG_LIMIT] = { "GET_RESPONSE_MSG_LIMIT",
+ cmd_get_response_msg_limit, 0, 0 },
+ [INFOSTAT][SET_RESPONSE_MSG_LIMIT] = { "SET_RESPONSE_MSG_LIMIT",
+ cmd_set_response_msg_limit, 1, 0 },
+ [INFOSTAT][BACKGROUND_OPERATION_STATUS] = { "BACKGROUND_OPERATION_STATUS",
+ cmd_infostat_bg_op_sts, 0, 0 },
+ [INFOSTAT][BACKGROUND_OPERATION_ABORT] = { "BACKGROUND_OPERATION_ABORT",
+ cmd_infostat_bg_op_abort, 0, 0 },
+ [TIMESTAMP][GET] = { "TIMESTAMP_GET", cmd_timestamp_get, 0, 0 },
+ [TIMESTAMP][SET] = { "TIMESTAMP_SET", cmd_timestamp_set, 8,
+ CXL_MBOX_IMMEDIATE_POLICY_CHANGE },
+ [LOGS][GET_SUPPORTED] = { "LOGS_GET_SUPPORTED", cmd_logs_get_supported,
+ 0, 0 },
+ [LOGS][GET_LOG] = { "LOGS_GET_LOG", cmd_logs_get_log, 0x18, 0 },
+ [PHYSICAL_SWITCH][IDENTIFY_SWITCH_DEVICE] = { "IDENTIFY_SWITCH_DEVICE",
+ cmd_identify_switch_device, 0, 0 },
+ [PHYSICAL_SWITCH][GET_PHYSICAL_PORT_STATE] = { "SWITCH_PHYSICAL_PORT_STATS",
+ cmd_get_physical_port_state, ~0, 0 },
+ [PHYSICAL_SWITCH][PHYSICAL_PORT_CONTROL] = { "SWITCH_PHYSICAL_PORT_CONTROL",
+ cmd_physical_port_control, 2, 0 },
+ [VIRTUAL_SWITCH][BIND_VPPB] = { "BIND_VPPB",
+ cmd_bind_vppb, ~0, 0 },
+ [VIRTUAL_SWITCH][UNBIND_VPPB] = { "UNBIND_VPPB",
+ cmd_unbind_vppb, ~0, 0 },
+ [TUNNEL][MANAGEMENT_COMMAND] = { "TUNNEL_MANAGEMENT_COMMAND",
+ cmd_tunnel_management_cmd, ~0, 0 },
+};
+
void cxl_initialize_usp_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
size_t payload_max)
{
@@ -4915,7 +4994,7 @@ void cxl_initialize_usp_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
void cxl_initialize_vcs_mctpcci(CXLCCI *cci, CXLVCSSwitch *vcs, DeviceState *d,
DeviceState *intf, size_t payload_max)
{
- cxl_copy_cci_commands(cci, cxl_cmd_set_usp_mctp);
+ cxl_copy_cci_commands(cci, cxl_cmd_set_vcs_mctp);
cci->vcs = vcs;
cci->d = d;
cci->intf = intf;
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [RFC QEMU PATCH 10/10] cxl-mailbox-utils: Add support for VCS Get Virtual CXL Switch Info command.
2026-04-29 13:48 [RFC QEMU PATCH 00/10] Initial Support for VCS Switching Joshua Lant
` (8 preceding siblings ...)
2026-04-29 13:48 ` [RFC QEMU PATCH 09/10] cxl-mailbox-utils: Add support for VCS bind/unbind commands Joshua Lant
@ 2026-04-29 13:48 ` Joshua Lant
9 siblings, 0 replies; 19+ messages in thread
From: Joshua Lant @ 2026-04-29 13:48 UTC (permalink / raw)
To: linux-cxl; +Cc: qemu-devel, Jonathan.Cameron, arpit1.kumar, Joshua Lant
Adds support for FMAPI command 0x5200, for more
information see CXL Specification v3.2 (7.6.7.2.1).
Signed-off-by: Joshua Lant <joshualant@gmail.com>
---
hw/cxl/cxl-mailbox-utils.c | 84 ++++++++++++++++++++++++++++++++++++++
1 file changed, 84 insertions(+)
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index cf6cb65eb6..825ef35179 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -123,6 +123,7 @@ enum {
#define GET_PHYSICAL_PORT_STATE 0x1
#define PHYSICAL_PORT_CONTROL 0x2
VIRTUAL_SWITCH = 0x52,
+ #define GET_VIRTUAL_SWITCH_INFO 0x0
#define BIND_VPPB 0x1
#define UNBIND_VPPB 0x2
TUNNEL = 0x53,
@@ -825,6 +826,87 @@ static CXLRetCode cmd_physical_port_control(const struct cxl_cmd *cmd,
}
}
+/* CXL r3.2 Section 7.6.7.2.1: Get Virtual CXL Switch Info (Opcode 5200h) */
+static CXLRetCode cmd_get_virtual_switch_info(const struct cxl_cmd *cmd,
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLCCI *cci)
+{
+ struct cxl_fmapi_get_virtual_switch_info_req_pl {
+ uint8_t start_vppb;
+ uint8_t vppb_list_limit;
+ uint8_t number_of_vcs;
+ uint8_t vcs_id_list[];
+ } QEMU_PACKED *in = (void *)payload_in;
+ struct vppb_info {
+ uint8_t binding_status;
+ uint8_t bound_port_id;
+ uint8_t bound_ld_id;
+ uint8_t rsv1;
+ } QEMU_PACKED;
+ struct vcs_info_block {
+ uint8_t vcs_id;
+ uint8_t vcs_state;
+ uint8_t usp_id;
+ uint8_t num_vppbs;
+ struct vppb_info vppbs[];
+ } QEMU_PACKED;
+ struct cxl_fmapi_get_virtual_switch_info_rsp_pl {
+ uint8_t num_vcs;
+ uint8_t rsv1[3];
+ struct vcs_info_block vcs_info_list[];
+ } QEMU_PACKED *out = (void *)payload_out;
+
+ uint8_t num_vcs_ret = cci->vcs->num_usp_ppbs;
+ uint8_t num_vppbs_ret;
+ ssize_t len_out_tmp = *len_out;
+ len_out_tmp = sizeof(struct cxl_fmapi_get_virtual_switch_info_rsp_pl) +
+ (num_vcs_ret * sizeof(*out->vcs_info_list));
+ for(int i = 0; i < num_vcs_ret; i++) {
+ //num_vppbs_ret = get_num_vppbs_in_vcs(cci->d, i);
+ num_vppbs_ret = cci->vcs->usp_ppbs[i]->info->num_vppbs;
+ //vPPB List Entry Count=min(vPPB List Limit, Number of vPPBs).
+ len_out_tmp = (num_vppbs_ret <= VPPB_LIST_LIMIT) ?
+ len_out_tmp + (num_vppbs_ret * sizeof(struct vppb_info)) :
+ len_out_tmp + (VPPB_LIST_LIMIT * sizeof(struct vppb_info));
+ }
+ *len_out = len_out_tmp;
+
+ out->num_vcs = num_vcs_ret;
+ out->rsv1[0] = 0;
+ out->rsv1[1] = 0;
+ out->rsv1[2] = 0;
+
+ uint8_t *ptr = (uint8_t *)out->vcs_info_list;
+
+ for (int i = 0; i < num_vcs_ret; i++) {
+ CXLVCSInfoBlock *info = cci->vcs->usp_ppbs[i]->info;
+ uint8_t num_vppbs = info->num_vppbs;
+ uint8_t vppb_count = MIN(num_vppbs - in->start_vppb,
+ in->vppb_list_limit);
+
+ /* write fixed vcs_info_block header (4 bytes) */
+ *ptr++ = info->vcs_id;
+ *ptr++ = info->vcs_state;
+ *ptr++ = info->usp_id;
+ *ptr++ = num_vppbs;
+
+ /* write vppb entries for current vcs */
+ for (int j = in->start_vppb; j < in->start_vppb + vppb_count; j++) {
+ struct vppb_info *v = (struct vppb_info *)ptr;
+ v->binding_status = info->vppbs[j]->binding_status;
+ v->bound_port_id = info->vppbs[j]->bound_port_id;
+ v->bound_ld_id = info->vppbs[j]->bound_ld_id;
+ v->rsv1 = 0;
+ ptr += sizeof(struct vppb_info);
+ }
+ }
+
+ return CXL_MBOX_SUCCESS;
+}
+
/* CXL r3.2 Section 7.6.7.2.2: Bind vPPB (Opcode 5201h) */
static CXLRetCode cmd_bind_vppb(const struct cxl_cmd *cmd,
uint8_t *payload_in,
@@ -4974,6 +5056,8 @@ static const struct cxl_cmd cxl_cmd_set_vcs_mctp[256][256] = {
cmd_get_physical_port_state, ~0, 0 },
[PHYSICAL_SWITCH][PHYSICAL_PORT_CONTROL] = { "SWITCH_PHYSICAL_PORT_CONTROL",
cmd_physical_port_control, 2, 0 },
+ [VIRTUAL_SWITCH][GET_VIRTUAL_SWITCH_INFO] = { "GET_VIRTUAL_SWITCH_INFO",
+ cmd_get_virtual_switch_info, ~0, 0 },
[VIRTUAL_SWITCH][BIND_VPPB] = { "BIND_VPPB",
cmd_bind_vppb, ~0, 0 },
[VIRTUAL_SWITCH][UNBIND_VPPB] = { "UNBIND_VPPB",
--
2.47.3
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [RFC QEMU PATCH 01/10] docs: Add documentation for cxl-vcs-switch
2026-04-29 13:48 ` [RFC QEMU PATCH 01/10] docs: Add documentation for cxl-vcs-switch Joshua Lant
@ 2026-05-19 14:33 ` Jonathan Cameron
2026-05-21 9:42 ` Joshua Lant
0 siblings, 1 reply; 19+ messages in thread
From: Jonathan Cameron @ 2026-05-19 14:33 UTC (permalink / raw)
To: Joshua Lant; +Cc: linux-cxl, qemu-devel, Jonathan.Cameron, arpit1.kumar
On Wed, 29 Apr 2026 14:48:35 +0100
Joshua Lant <joshualant@gmail.com> wrote:
> Signed-off-by: Joshua Lant <joshualant@gmail.com>
Hi Joshua,
Sorry it's taken me a while to get to this! I blame to much activity
on other open source projects! :)
I've mused in the past on how to do the command lines for these.
So some thoughts are based on that - feel free to argue why we
the structure you have here works better.
When I get through the series I may well change my mind on some
of what follows ;)
> ---
> docs/system/devices/cxl.rst | 90 ++++++++++++++++++++++++++++++++++---
> 1 file changed, 85 insertions(+), 5 deletions(-)
>
> diff --git a/docs/system/devices/cxl.rst b/docs/system/devices/cxl.rst
> index 32b1b5d773..9e8452e576 100644
> --- a/docs/system/devices/cxl.rst
> +++ b/docs/system/devices/cxl.rst
> @@ -119,11 +119,11 @@ and associated component register access via PCI bars.
> CXL Switch
> ~~~~~~~~~~
> Here we consider a simple CXL switch with only a single
> -virtual hierarchy. Whilst more complex devices exist, their
> -visibility to a particular host is generally the same as for
> -a simple switch design. Hosts often have no awareness
> -of complex rerouting and device pooling, they simply see
> -devices being hot added or hot removed.
> +virtual hierarchy. Whilst more complex devices exist (see VCS
> +Switching below), their visibility to a particular host is
> +generally the same as for a simple switch design. Hosts often
> +have no awareness of complex rerouting and device pooling,
> +they simply see devices being hot added or hot removed.
>
> A CXL switch has a similar architecture to those in PCIe,
> with a single upstream port, internal PCI bus and multiple
> @@ -467,6 +467,86 @@ Example configuration:
> Guest OS communication with the MCTP CCI can then be established using standard
> MCTP configuration tools.
>
> +CXL Multi-VCS Switching
> +-----------------------
> +
> +The cxl-vcs-switch object allows for a Fabric Manager to dynamically reconfigure
> +the switching within a multi-upstream port CXL/PCIe topology, This moves beyond
> +the static switching configuration described above. The use of vcs=X on an
> +endpoint device indicates that it should be hidden from guests at boot.
That bit seems rather unintuitive. EPs shouldn't really be involved in this
at all. I guess you are using them as a proxy for a physical downstream port?
Interesting idea if a bit non intuitive. I wonder if we can put in an explicit
physical DSP device in. When linked it just proxies the vPPD.
Maybe we can get away without that but it leaves us with no physical port hotplug
as we can't connect an empty physical downstream port to a VCS.
> Each
> +upstream port with vcs=X set will conceptually become an upstream PPB. Any
> +downstream port that is connected to an upstream port with vcs=X set will
> +automatically become a vPPB for that VCS. The overall cxl-virtual-switch has a
Neat not to have to set it for the DSPs, but I think we will need them to
grow new functionality so maybe a different device type is good.
> +single CCI mailbox used for config/status of all ports within the switch.
Need to support both MCTP and switch-cci but that should be fine.
> +Setting local-fm=true indicates that this QEMU instance has the CCI mailbox
> +attached. Setting it false will create listeners for commands from a remote
> +QEMU process (yet to be implemented).
Nice but make that the default for now (And drop the parameter).
Absence of a connected CCI might be sufficient though that's a bit ugly
to check.
> +
> +An example of how the topology is described on the CLI is shown below:
> +
> + -object cxl-vcs-switch,id=vcs0,usp-ppbs=2,dsp-ppbs=4,local-fm=true \
Interesting. I'd kind of like it to be a device, but it has no presence
on any bus in of itself (arguably it is on a whole load of them). So maybe not.
> + -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.0,hdm_for_passthrough=true \
Small side note - avoid the passthrough trick. It means a bunch of code
paths aren't exercised and has hidden various OS bugs.
> + -device cxl-rp,port=0,bus=cxl.0,id=root_port1,chassis=0,slot=1 \
> + -device pxb-cxl,bus_nr=22,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
> + -device cxl-rp,port=0,bus=cxl.1,id=root_port2,chassis=1,slot=1 \
> + -device cxl-upstream,port=0,sn=1234,bus=root_port1,id=us0,addr=0.0,multifunction=on,vcs=vcs0,usppb=0 \
> + -device cxl-upstream,port=0,sn=5678,bus=root_port2,id=us1,addr=0.0,multifunction=on,vcs=vcs0,usppb=1 \
How can we have two upstream ports in a single vcs? To me those are separate VCSs
where a VCS is normally a tree topology below a given USP.
I think we have a terminology problem. If I read this right you are using VCS
to mean the whole physical switch? Been a little while but I don't think
that corresponds at all to it's meaning in the CXL Spec. Your VCS0/1 below
are right.
> + -device cxl-switch-mailbox-cci,bus=root_port1,addr=0.3,target=vcs0 \
> + -device usb-cxl-mctp,bus=ehci.0,id=usb0,target=vcs0 \
> + -device cxl-downstream,port=0,bus=us0,id=dsp0,slot=3 \
> + -device cxl-downstream,port=1,bus=us0,id=dsp1,slot=4 \
> + -device cxl-downstream,port=0,bus=us1,id=dsp2,slot=7 \
> + -device cxl-downstream,port=1,bus=us1,id=dsp3,slot=8 \
Ok. So these only know they are virtual because they are connected to a virtual USP.
Might be enough - or we might want to make that more explicit via
a new device type.
> + -device cxl-type3,persistent-memdev=cxl-mem1,id=cxl-ep1,lsa=cxl-lsa1,sn=99,vcs=vcs0,dsppb=0 \
> + -device cxl-type3,persistent-memdev=cxl-mem2,id=cxl-ep2,lsa=cxl-lsa2,sn=100,vcs=vcs0,dsppb=1 \
> + -device cxl-type3,persistent-memdev=cxl-mem3,id=cxl-ep3,lsa=cxl-lsa3,sn=101,vcs=vcs0,dsppb=2 \
> + -device cxl-type3,persistent-memdev=cxl-mem4,id=cxl-ep4,lsa=cxl-lsa4,sn=102,vcs=vcs0,dsppb=3 \
This I mention above. I 'think' you are using the dsppb to instantiate something that is pretending
to be a the physical DSP.
I haven't yet read thee series, but gut feeling is that will make the querying of link
properties etc rather different from the normal case.
> + -machine cxl-fmw.0.targets.0=cxl.0,cxl-fmw.0.size=8G,cxl-fmw.1.targets.0=cxl.1,cxl-fmw.1.size=8G
> +
> +Example topology involving VCS switching::
> +
> + +--------------------+ +--------------------+
> + | Host Bridge 0 | | Host Bridge 1 |
> + +----------+---------+ +----------+---------+
> + +-------+ | |
> + | MCTP | | |
> + | USB/ | +----------+---------+ +----------+---------+
> + | I2C | | Root Port 0 | | Root Port 1 |
> + +-----+-+ +----------+---------+ +----------+---------+
> + | | |
> + | | |
> + +------|---------------+-----------------------+-----------------------+
> + | +-+--------+ | cxl-vcs-switch (vcs0)| |
> + | +--| CCI MBOX |---* | | |
> + | | +----------+ | | |
> + | | +-----------------+--------+ +-------+------------------+ |
> + | +--+ | VCS0 | *---+ | VCS1 | |
> + | | +---------------+------+ | | +-----+----------------+ | |
> + | | | | | | | | | |
> + | | | USP 0 | | | | USP 1 | | |
> + | | | | | | | | | |
> + | | +----+------------+----+ | | +----+------------+----+ | |
> + | | | | | | | | | |
> + | | +----+----+ +----+----+ | | +----+----+ +----+----+ | |
> + | | | DSP 0 | | DSP 1 | | | | DSP 2 | | DSP 3 | | |
> + | | |(vPPB 0) | |(vPPB 1) | | | |(vPPB 0) | |(vPPB 1) | | |
> + | | | | | | | | | | | | | |
> + | | +---------+ +---------+ | | +---------+ +----+----+ | |
> + | +--------------------------+ +-------------------+------+ |
> + | | |
> + | +----------------------------------------------+ |
> + | | |
> + | | - - - |
> + +-----------|------------|--------------------|------------|-----------+
> + | | | |
> + +---------+ +---------+ +---------+ +---------+
> + |CXL/PCIe | |CXL/PCIe | |CXL/PCIe | |CXL/PCIe |
> + | EP 0 | | EP 1 | | EP 2 | | EP 3 |
> + | (PPB0) | | (PPB1) | | (PPB2) | | (PPB3) |
> + +---------+ +---------+ +---------+ +---------+
> + PPB0 Bound to VCS1, vPPB1. Others unbound...
> +
Good to have the diagram as makes it easier to discuss.
What you have here is a bit of a hack because only some entities created
exist in the command line - the others are spun up implicitly. I suspect
we really want to make them explicit. The one thing I never looked into in
the following is how hard it would be to poke a vDSP in front of a physical
DSP and basically proxy stuff through or not. Some stuff will be programmed
at boot (windows etc for hotplug later) but other stuff will fire in the hotplug
flow on an attach of a physical port. Will need some care and stitching up
memory regions across the boundary.
The command line I'd be looking at for this as a target (feel free to shoot
at it) would be something like (I went with one PXB - but need to test both options).
Note some of this is probably garbage as I haven't checked parameters are right.
-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.0 \
-device cxl-rp,bus,cxl.0,id=root_port1...
-device cxl-rp,bus,cxl.0,id=root_port2..
-device cxl-upstream,port=0,sn=1234,bus=root_port1,id=us0,addr=0.0,multifunction=on,virtual=on \
-device cxl-upstream,port=0,sn=5678,bus=root_port2,id=us1,addr=0.0,virtual=on \
#note I extended current target to a list
-device cxl-virtual-downstream,vport=0,bus=us0,id=vppb0 \
-device cxl-virtual-downstream,vport=1,bus=us0,id=vppb0 \
-device cxl-virtual-downstream,vport=2,bus=us0,id=vppb0 \
-device cxl-virtual-downstream,vport=0,bus=us1,id=vppb0 \
-device cxl-virtual-downstream,vport=1,bus=us1,id=vppb0 \
-device cxl-virtual-downstream,vport=2,bus=us1,id=vppb0 \
# Note more virtual ports than physical - likely common situation.
-object cxl-switch,usps.0=usp0,usps.1=usp1,id=vsw0 \
#list of usps so we can navigate downwards from this.
-device cxl-switch-mailbox-cci,id=swcci0,bus=root_por1,multifunction=on,target=vsw0\
# Maybe hang the unconnected physical dsps on a bus created by the cxl-switch?
-device cxl-downstream,port=0,bus=vsw0,id=dsp0,slot=3 \
-device cxl-downstream,port=1,bus=vsw0,id=dsp1,slot=4 \
-device cxl-downstream,port=2,bus=vsw0,id=dsp2,slot=7 \
-device cxl-downstream,port=3,bus=vsw0,id=dsp3,slot=8 \
#ideally a device but need to think where to hang it.
-device cxl-type3,persistent-memdev=cxl-mem1,id=cxl-ep1,lsa=cxl-lsa1,sn=99,bus=dsp0 \
-device cxl-type3,persistent-memdev=cxl-mem2,id=cxl-ep2,lsa=cxl-lsa2,sn=100,bus=dsp1 \
-device cxl-type3,persistent-memdev=cxl-mem3,id=cxl-ep3,lsa=cxl-lsa3,sn=101,bus=dsp2 \
#note not all DSPs have anything on them.
Few reasons for this structure.
1) The unconnected physical port - we want to make sure physical hotplug works both
when not associated with a VCS and when it is.
2) We need to be able to talk to EPs via FM interfaces when they aren't connected
Given we have to make that look like PCI, let's make it PCI. I'm not sure how
much hackery that will take as we'll need to do some level of enumeration from
the the switch controller. Only need that once we want to do more than check
training etc though - so maybe job for another day. In theory we can do everything
with devices in that state (be it slowly) so would need all the addresses programmed
etc. Not as general as current discussions on enumerating full PCI bus in QEMU as
all direct connect.
Anyhow it's fiddly with this scheme but I think a little more general
than your current one and closer representation of the hardware which will
matter as we add all the introspection stuff etc in the FMAPI.
Jonathan
> +
> References
> ----------
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [RFC QEMU PATCH 03/10] cxl-type3: Properly unmap the memory-backend on device exit
2026-04-29 13:48 ` [RFC QEMU PATCH 03/10] cxl-type3: Properly unmap the memory-backend on device exit Joshua Lant
@ 2026-05-19 15:01 ` Jonathan Cameron
0 siblings, 0 replies; 19+ messages in thread
From: Jonathan Cameron @ 2026-05-19 15:01 UTC (permalink / raw)
To: Joshua Lant; +Cc: linux-cxl, qemu-devel, Jonathan.Cameron, arpit1.kumar
On Wed, 29 Apr 2026 14:48:37 +0100
Joshua Lant <joshualant@gmail.com> wrote:
> Currently the backend remains mapped, meaning that if the device owning
> the backend is hot-removed, it cannot be readded in the same QEMU
> instance.
>
> Signed-off-by: Joshua Lant <joshualant@gmail.com>
Gah. That sounds like a fix we should have in place today.
Can you send it as a separate patch with an appropriate
Fixes tag.
Jonathan
> ---
> hw/mem/cxl_type3.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index 68cd04b7d9..414c776028 100644
> --- a/hw/mem/cxl_type3.c
> +++ b/hw/mem/cxl_type3.c
> @@ -1072,12 +1072,15 @@ static void ct3_exit(PCIDevice *pci_dev)
> cxl_destroy_cci(&ct3d->cci);
> if (ct3d->dc.host_dc) {
> cxl_destroy_dc_regions(ct3d);
> + host_memory_backend_set_mapped(ct3d->dc.host_dc, false);
> address_space_destroy(&ct3d->dc.host_dc_as);
> }
> if (ct3d->hostpmem) {
> + host_memory_backend_set_mapped(ct3d->hostpmem, false);
> address_space_destroy(&ct3d->hostpmem_as);
> }
> if (ct3d->hostvmem) {
> + host_memory_backend_set_mapped(ct3d->hostvmem, false);
> address_space_destroy(&ct3d->hostvmem_as);
> }
> }
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [RFC QEMU PATCH 04/10] cxl_downstream: enable power controller present capability.
2026-04-29 13:48 ` [RFC QEMU PATCH 04/10] cxl_downstream: enable power controller present capability Joshua Lant
@ 2026-05-19 15:03 ` Jonathan Cameron
0 siblings, 0 replies; 19+ messages in thread
From: Jonathan Cameron @ 2026-05-19 15:03 UTC (permalink / raw)
To: Joshua Lant; +Cc: linux-cxl, qemu-devel, Jonathan.Cameron, arpit1.kumar
On Wed, 29 Apr 2026 14:48:38 +0100
Joshua Lant <joshualant@gmail.com> wrote:
> PCI_EXP_SLTCAP_PCP (Power Controller Present) must be set in the DSP's
> slot capabilities for PCIe managed hot-remove to complete. Without this
> notification from the guest of removal cannot be sent back to the
> device, so device listeners for unrealizing will not fire.
>
> Signed-off-by: Joshua Lant <joshualant@gmail.com>
Another fix by the sound of it? Please send separately (Well with the other
one perhaps) with a suitable Fixes tag.
> ---
> hw/pci-bridge/cxl_downstream.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/hw/pci-bridge/cxl_downstream.c b/hw/pci-bridge/cxl_downstream.c
> index 91c5e6a605..5322c46900 100644
> --- a/hw/pci-bridge/cxl_downstream.c
> +++ b/hw/pci-bridge/cxl_downstream.c
> @@ -228,6 +228,8 @@ static const Property cxl_dsp_props[] = {
> DEFINE_PROP_PCIE_LINK_WIDTH("x-width", PCIESlot,
> width, PCIE_LINK_WIDTH_16),
> DEFINE_PROP_BOOL("x-256b-flit", PCIESlot, flitmode, true),
> + DEFINE_PROP_BIT(COMPAT_PROP_PCP, PCIDevice, cap_present,
> + QEMU_PCIE_SLTCAP_PCP_BITNR, true),
> };
>
> static void cxl_dsp_class_init(ObjectClass *oc, const void *data)
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [RFC QEMU PATCH 05/10] cxl-vcs-switch: Initial support for CXL VCS.
2026-04-29 13:48 ` [RFC QEMU PATCH 05/10] cxl-vcs-switch: Initial support for CXL VCS Joshua Lant
@ 2026-05-19 15:19 ` Jonathan Cameron
0 siblings, 0 replies; 19+ messages in thread
From: Jonathan Cameron @ 2026-05-19 15:19 UTC (permalink / raw)
To: Joshua Lant; +Cc: linux-cxl, qemu-devel, Jonathan.Cameron, arpit1.kumar
On Wed, 29 Apr 2026 14:48:39 +0100
Joshua Lant <joshualant@gmail.com> wrote:
> Add initial support for VCS (multi-USP/multi-logic-device) switch
> emulation in CXL. Currently the object only supports the identify/bind/unbind
> commands (0x5200/0x5201/0x5202), as well as support for device hiding
> listeners. This enables preliminary testing of the VCS capability,
> but will not allow for complete emulated flow as described in
> spec (CXL v3.2, 7.2.3). Nor will it allow for multiple USP's to be
> disributed over multiple QEMU instances.
>
> Signed-off-by: Joshua Lant <joshualant@gmail.com>
> ---
> hw/cxl/cxl-vcs-switch.c | 524 ++++++++++++++++++++++++++++++++
We need to make it explicit this is a multiple-vcs switch
Or not bother given other switches don't have an explicit representation
so we could just call it cxl_switch :)
A few trivial things follow.
I'm trying to get my head around how we handle state given we ultimately
can't wait to realise until binding because we need to be able to
do pre binding init of devices etc.
MLDs will be more fun at somepoint as well if we care to go that way.
Anyhow, this is very much first look rather than a detailed review.
> +
> +/* When a device is instantiated downstream of a VCS's PPB, we
> + * store the qdicts from the CLI, to realize the device at a
> + * future time.
> + */
> +bool cxl_vcs_hide_device_listener(DeviceListener *listener, const QDict *opts,
> + bool from_json, Error **errp)
> +{
> + CXLVCSSwitch *vcs = container_of(listener, CXLVCSSwitch, listener);
> + const char *vcs_id_str = qdict_get_try_str(opts, "vcs");
stray space.
> + const char *ppb_str = qdict_get_try_str(opts, "dsppb");
> + int ppb;
> +
> +
> +static void vcs_set_usp_ppbs(Object *obj, Visitor *v, const char *name,
> + void *opaque, Error **errp)
> +{
> + CXLVCSSwitch *vcs = CXL_VCS_SWITCH(obj);
> + uint8_t val;
> + if (!visit_type_uint8(v, name, &val, errp)) {
> + return;
> + }
> + vcs->num_usp_ppbs = val;
> +}
Blank line between functions.
> +static void vcs_get_dsp_ppbs(Object *obj, Visitor *v, const char *name,
> + void *opaque, Error **errp)
> +{
> + CXLVCSSwitch *vcs = CXL_VCS_SWITCH(obj);
> + uint8_t val = vcs->num_dsp_ppbs;
> + visit_type_uint8(v, name, &val, errp);
> +}
> +
> +static void vcs_set_dsp_ppbs(Object *obj, Visitor *v, const char *name,
> + void *opaque, Error **errp)
> +{
> + CXLVCSSwitch *vcs = CXL_VCS_SWITCH(obj);
> + uint8_t val;
Blank line - same in similar places. Normally separate declarations and
body of code.
> + if (!visit_type_uint8(v, name, &val, errp)) {
> + return;
> + }
> + vcs->num_dsp_ppbs = val;
> +}
> +
> +static void cxl_vcs_class_init(ObjectClass *oc, const void *data)
> +{
> + CXLVCSSwitchClass *cc = CXL_VCS_SWITCH_CLASS(oc);
> + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
Bank line here.
> + ucc->complete = cxl_vcs_switch_complete;
> + ucc->can_be_deleted = cxl_vcs_switch_can_be_deleted;
> + cc->identify_switch_device = cxl_vcs_identify_switch_device;
> + cc->get_physical_port_state = cxl_vcs_get_physical_port_state;
> + cc->physical_port_control = cxl_vcs_physical_port_control;
> + cc->get_virtual_switch_info = cxl_vcs_get_virtual_switch_info;
> + cc->bind_vppb = cxl_vcs_bind_vppb;
> + cc->unbind_vppb = cxl_vcs_unbind_vppb;
> +
> + object_class_property_add_bool(oc, "local-fm",
> + cxl_vcs_get_local_fm, cxl_vcs_set_local_fm);
As below - drop for now.
> + object_class_property_set_description(oc, "local-fm",
> + "true = FM authority (mctp connected guest), \
> + false = slave listener (IPC)");
Likewise.
> + object_class_property_add(oc, "usp-ppbs", "uint8",
> + vcs_get_usp_ppbs, vcs_set_usp_ppbs,
> + NULL, NULL);
> + object_class_property_set_description(oc, "usp-ppbs",
> + "Number of upstream ports in the switch");
> + object_class_property_add(oc, "dsp-ppbs", "uint8",
> + vcs_get_dsp_ppbs, vcs_set_dsp_ppbs,
> + NULL, NULL);
> + object_class_property_set_description(oc, "dsp-ppbs",
> + "Number of downstream ports in the switch");
> +}
>
> diff --git a/include/hw/cxl/cxl_vcs_switch.h b/include/hw/cxl/cxl_vcs_switch.h
> new file mode 100644
> index 0000000000..6576870bf3
> --- /dev/null
> +++ b/include/hw/cxl/cxl_vcs_switch.h
> @@ -0,0 +1,134 @@
> +/*
> + * CXL VCS Capable Switch Object Header
No point in stating it's a header.
> + *
> + * Copyright(C) 2026 University of Manchester
> + * Author: Joshua Lant <joshualant@gmail.com>.
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2. See the
> + * COPYING file in the top-level directory.
If you are fine with many folk just use SPDX on it's own now.
> + *
> + * SPDX-License-Identifier: GPL-v2-only
> + */
> +
> +#ifndef CXL_VCS_SWITCH_H
> +#define CXL_VCS_SWITCH_H
> +
> +#include "qemu/osdep.h"
> +#include "qapi/error.h"
> +#include "qom/object.h"
> +#include "qom/object_interfaces.h"
> +#include "hw/pci-bridge/cxl_upstream_port.h"
> +#include "hw/pci-bridge/cxl_downstream_port.h"
> +#include "include/hw/cxl/cxl_device.h"
> +
> +#define TYPE_CXL_VCS_SWITCH "cxl-vcs-switch"
> +OBJECT_DECLARE_TYPE(CXLVCSSwitch, CXLVCSSwitchClass, CXL_VCS_SWITCH);
> +
> +#define CXL_VPPB_BINDING_STATUS_UNBOUND 0x00
> +#define CXL_VPPB_BINDING_STATUS_IN_PROGRESS 0x01
> +#define CXL_VPPB_BINDING_STATUS_BOUND_PORT 0x02
> +#define CXL_VPPB_BINDING_STATUS_BOUND_LD 0x03
> +#define CXL_VPPB_BINDING_STATUS_BOUND_PID 0x04
> +
> +#define CXL_VPPB_UNBIND_WAIT_FOR_LINK_DOWN 0x0
> +#define CXL_VPPB_UNBIND_MANAGED_HOT_REMOVE 0x1
> +#define CXL_VPPB_UNBIND_SURPRISE_HOT_REMOVE 0x2
> +
> +#define CXL_UNSUPPORTED_LD_ID 0xFFFF
> +#define CXL_INVALID_BOUND_LD_ID 0xFF
> +
> +#define CXL_VCS_STATE_INVALID_ID 0xFF
> +#define CXL_VCS_STATE_DISABLED 0x0
> +#define CXL_VCS_STATE_ENABLED 0x1
> +
> +#define CXL_MAX_VCS_PORTS 0x8
> +#define CXL_MAX_VPPB_PER_VCS 0x8
> +#define CXL_MAX_USP_PPBS CXL_MAX_VCS_PORTS
> +#define CXL_MAX_DSP_PPBS CXL_MAX_VPPB_PER_VCS
> +
> +#define CXL_MAX_PPBS (CXL_MAX_USP_PPBS + CXL_MAX_DSP_PPBS)
> +#define VPPB_LIST_LIMIT 8
> +
> +struct CXLVCSSwitchClass {
> + ObjectClass parent_class;
> + void (*identify_switch_device)(CXLVCSSwitch *sw);
> + void (*get_physical_port_state)(CXLVCSSwitch *sw);
> + void (*physical_port_control)(CXLVCSSwitch *sw);
> + void (*get_virtual_switch_info)(CXLVCSSwitch *sw);
> + CXLRetCode (*bind_vppb)(CXLVCSSwitch *sw, uint8_t vcs_id, uint8_t vppb_id,
> + uint8_t dsppb_id, uint16_t ld_id);
> + CXLRetCode (*unbind_vppb)(CXLVCSSwitch *sw, uint8_t vcs_id, uint8_t vppb_id,
> + uint16_t option);
> +};
> +
> +/* CXL r3.2 Table 7-32: Get Virtual CXL Switch Info VCS Info Block Format */
We tend to use latest available spec for new stuff. So if you can
reference CXL 4.0 that would preferred.
This is driven by the silly policy of now making old specs easily
available.
> +typedef struct CXLVPPBInfo {
> + PCIDevice *dsp;
> + uint8_t binding_status;
> + uint8_t bound_port_id;
> + uint8_t bound_ld_id;
> + uint8_t rsv1;
> +} CXLVPPBInfo;
> +
> +typedef struct CXLVCSInfoBlock {
> + uint8_t vcs_id;
> + uint8_t vcs_state;
> + uint8_t usp_id;
> + uint8_t num_vppbs;
> + struct CXLVPPBInfo *vppbs[CXL_MAX_VPPB_PER_VCS];
> +} CXLVCSInfoBlock;
> +
> +/* Physical Upstream and Downstream PCI-PCI Bridge (PPBs) structs */
I'd split the comment in two. Kind of obvious but I misread
it as referring to this struct for both of them.
> +typedef struct CXLUpstreamPPB {
> + CXLUpstreamPort *usp;
> + struct CXLVCSInfoBlock *info;
> +} CXLUpstreamPPB;
> +
> +typedef struct CXLDownstreamPPB {
> + DeviceState *dev;
> + PCIDevice *pdev;
> + // store opts and json in case of simple device hiding.
> + QDict *opts;
> + bool from_json;
> + bool is_bound;
> + uint8_t bound_vcs_id;
> + uint8_t bound_vppb_id;
> +} CXLDownstreamPPB;
> +
> +typedef struct CXLVCSSwitch {
> + Object parent;
> + uint8_t num_usp_ppbs;
> + uint8_t num_dsp_ppbs;
> + uint8_t num_vppbs_per_vcs;
> + CXLCCI swcci;
> + CXLCCI mctpcci;
> + DeviceListener listener;
> + CXLUpstreamPPB *usp_ppbs[CXL_MAX_USP_PPBS];
> + CXLDownstreamPPB *dsp_ppbs[CXL_MAX_DSP_PPBS];
> + // true if FM is in this guest, false if remote FM.
> + bool local_fm;
As mentioned in comments on cover letter. Drop this for now.
> +} CXLVCSSwitch;
> +
> +// Called by USP/DSP/EP CXL devices to build the VCS structures.
> +void cxl_vcs_register_usp(CXLVCSSwitch *sw, CXLUpstreamPort *usp, Error **errp);
> +void cxl_vcs_register_vppb(CXLVCSSwitch *sw, CXLUpstreamPort *usp,
> + CXLDownstreamPort *dsp, Error **errp);
> +void cxl_vcs_register_dsppb(CXLVCSSwitch *sw, const QDict *opts,
> + bool from_json, Error **errp);
> +void cxl_vcs_register_qdict_dsppb(CXLVCSSwitch *sw, const QDict *opts,
> + bool from_json, Error **errp);
> +bool cxl_vcs_hide_device_listener(DeviceListener *listener,
> + const QDict *device_opts, bool from_json, Error **errp);
> +
> +
> +// Called by the CCI mailbox utils for FMAPI control of VCS.
> +CXLRetCode cxl_vcs_bind_vppb(CXLVCSSwitch *sw, uint8_t vcs_id, uint8_t vppb_id,
> + uint8_t dsppb_id, uint16_t ld_id);
> +CXLRetCode cxl_vcs_unbind_vppb(CXLVCSSwitch *sw, uint8_t vcs_id,
> + uint8_t vppb_id, uint16_t option);
> +void cxl_vcs_identify_switch_device(CXLVCSSwitch *sw);
> +void cxl_vcs_get_physical_port_state(CXLVCSSwitch *sw);
> +void cxl_vcs_physical_port_control(CXLVCSSwitch *sw);
> +void cxl_vcs_get_virtual_switch_info(CXLVCSSwitch *sw);
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [RFC QEMU PATCH 02/10] qdev/qbus: Allow hidden devices to be busless on QEMU startup
2026-04-29 13:48 ` [RFC QEMU PATCH 02/10] qdev/qbus: Allow hidden devices to be busless on QEMU startup Joshua Lant
@ 2026-05-20 10:02 ` Jonathan Cameron
2026-05-21 9:59 ` Joshua Lant
0 siblings, 1 reply; 19+ messages in thread
From: Jonathan Cameron @ 2026-05-20 10:02 UTC (permalink / raw)
To: Joshua Lant; +Cc: linux-cxl, qemu-devel, Jonathan.Cameron, arpit1.kumar
On Wed, 29 Apr 2026 14:48:36 +0100
Joshua Lant <joshualant@gmail.com> wrote:
> Extend the capability for hiding devices, introduced for virtio-net
> device in:
>
> commit f3a8505656935cde32e28c1c6317f725084da1e0
> Author: Jens Freimann <jfreimann@redhat.com>
> Date: Tue Oct 29 12:48:55 2019 +0100
> qdev/qbus: add hidden device support
>
> Currently only endpoint devices can be hidden with a primary device
> and failover (known static configuration). However, looking at future
> composable systems, we see a need for hidden devices which have no associated
> bus upon boot. Move the check for hidden devices to before the bus
> search, and if it is hidden ignore the case where the device was
> described on the CLI without the "bus=" field.
>
> This is motivated by a specific use-case: implementing the VCS
> command set, part of the CXL specification (CXL r3.2 Section 7.1.3). In
> this scenario a switch controlled by a Fabric Manager is able to change the
> virtual hierarchy of devices seen by a guest within a fixed physical system
> topology. The connecting bus is not known until runtime when
> a bind command is issued by the Fabric Manager.
So this comes back to the earlier comment on whether they are completely
hidden from a guest. They aren't quite - we need to be able to
get to them via the MCTP or cxl-switch-cci (in band mailbox) paths.
Maybe we can solve that later without any compatibility problems
or by cheating and just searching hidden devices. I'm not sure.
>
> Signed-off-by: Joshua Lant <joshualant@gmail.com>
> ---
> system/qdev-monitor.c | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/system/qdev-monitor.c b/system/qdev-monitor.c
> index f2aa400a77..b51dfe0645 100644
> --- a/system/qdev-monitor.c
> +++ b/system/qdev-monitor.c
> @@ -650,6 +650,7 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
> DeviceState *dev;
> BusState *bus = NULL;
> QDict *properties;
> + bool hide_device;
>
> driver = qdict_get_try_str(opts, "driver");
> if (!driver) {
> @@ -663,6 +664,11 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
> return NULL;
> }
>
> + /* Is the device hidden from the guest?
> + * If yes, no need to find a default bus if none given...
> + * Bus could be provided at runtime (i.e. in a switch)*/
> + hide_device = qdev_should_hide_device(opts, from_json, errp);
> +
> /* find bus */
> path = qdict_get_try_str(opts, "bus");
> if (path != NULL) {
> @@ -675,14 +681,14 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
> driver, object_get_typename(OBJECT(bus)));
> return NULL;
> }
> - } else if (dc->bus_type != NULL) {
> + } else if (dc->bus_type != NULL && !hide_device) {
> bus = qdev_find_default_bus(dc, errp);
> if (!bus) {
> return NULL;
> }
> }
>
> - if (qdev_should_hide_device(opts, from_json, errp)) {
> + if (hide_device) {
> if (bus && !qbus_is_hotpluggable(bus)) {
> error_setg(errp, "Bus '%s' does not support hotplugging",
> bus->name);
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [RFC QEMU PATCH 01/10] docs: Add documentation for cxl-vcs-switch
2026-05-19 14:33 ` Jonathan Cameron
@ 2026-05-21 9:42 ` Joshua Lant
0 siblings, 0 replies; 19+ messages in thread
From: Joshua Lant @ 2026-05-21 9:42 UTC (permalink / raw)
To: Jonathan Cameron; +Cc: linux-cxl, qemu-devel, Jonathan.Cameron, arpit1.kumar
> On Wed, 29 Apr 2026 14:48:35 +0100
> Joshua Lant <joshualant@gmail.com> wrote:
>
> > Signed-off-by: Joshua Lant <joshualant@gmail.com>
> Hi Joshua,
>
> Sorry it's taken me a while to get to this! I blame to much activity
> on other open source projects! :)
No problem at all! I appreciate you taking time to review this.
>
> I've mused in the past on how to do the command lines for these.
> So some thoughts are based on that - feel free to argue why we
> the structure you have here works better.
>
> When I get through the series I may well change my mind on some
> of what follows ;)
>
>
> > ---
> > docs/system/devices/cxl.rst | 90 ++++++++++++++++++++++++++++++++++---
> > 1 file changed, 85 insertions(+), 5 deletions(-)
> >
> > diff --git a/docs/system/devices/cxl.rst b/docs/system/devices/cxl.rst
> > index 32b1b5d773..9e8452e576 100644
> > --- a/docs/system/devices/cxl.rst
> > +++ b/docs/system/devices/cxl.rst
> > @@ -119,11 +119,11 @@ and associated component register access via PCI bars.
> > CXL Switch
> > ~~~~~~~~~~
> > Here we consider a simple CXL switch with only a single
> > -virtual hierarchy. Whilst more complex devices exist, their
> > -visibility to a particular host is generally the same as for
> > -a simple switch design. Hosts often have no awareness
> > -of complex rerouting and device pooling, they simply see
> > -devices being hot added or hot removed.
> > +virtual hierarchy. Whilst more complex devices exist (see VCS
> > +Switching below), their visibility to a particular host is
> > +generally the same as for a simple switch design. Hosts often
> > +have no awareness of complex rerouting and device pooling,
> > +they simply see devices being hot added or hot removed.
> >
> > A CXL switch has a similar architecture to those in PCIe,
> > with a single upstream port, internal PCI bus and multiple
> > @@ -467,6 +467,86 @@ Example configuration:
> > Guest OS communication with the MCTP CCI can then be established using standard
> > MCTP configuration tools.
> >
> > +CXL Multi-VCS Switching
> > +-----------------------
> > +
> > +The cxl-vcs-switch object allows for a Fabric Manager to dynamically reconfigure
> > +the switching within a multi-upstream port CXL/PCIe topology, This moves beyond
> > +the static switching configuration described above. The use of vcs=X on an
> > +endpoint device indicates that it should be hidden from guests at boot.
>
> That bit seems rather unintuitive. EPs shouldn't really be involved in this
> at all. I guess you are using them as a proxy for a physical downstream port?
correct...
> Interesting idea if a bit non intuitive. I wonder if we can put in an explicit
> physical DSP device in. When linked it just proxies the vPPD.
>
> Maybe we can get away without that but it leaves us with no physical port hotplug
> as we can't connect an empty physical downstream port to a VCS.
>
The main reason it has been designed this way is becasue I could not see
a way of replacing one DSP (the vPPB seen by the guest) with another DSP
(the actual physical port to which the device would be attached). Doing this
could be done in 1 of 3 ways. Two of which I dont think will work. The
third I have not figured out how to do it yet.
1. Reconnect USP to PPB and endpoint rather than the vPPB on bind.
The connection(s) to the USP needs to remain static since the USP itself is not
hotpluggable. This is a mandate from the PCIe spec regarding slot/device
characteristics, rather than a QEMU implementation detail AFAICT.
2.
The connection between DSP (PPB) and endpoint is severed on bind, and
then the endpoint is then attached to the vPPB.
This will mean in QEMU that the device has to go through the teardown process
and reenumeration, so will potentially lose device state information. The
alternative being (i'd imagine) a messy piece of code to rewrite the exit
functions depending what is happening (real exit or "rewiring")...
3.
Create some sort of new device which looks like a DSP but actually is
more of a shim of some sort and is able to connect pre-realized devices to
pre-existing DSPs (the vPPBs) but somehow trigger a hotplug event in the guest
as if it was only just being realized.
> > Each
> > +upstream port with vcs=X set will conceptually become an upstream PPB. Any
> > +downstream port that is connected to an upstream port with vcs=X set will
> > +automatically become a vPPB for that VCS. The overall cxl-virtual-switch has a
> Neat not to have to set it for the DSPs, but I think we will need them to
> grow new functionality so maybe a different device type is good.
>
> > +single CCI mailbox used for config/status of all ports within the switch.
>
> Need to support both MCTP and switch-cci but that should be fine.
>
> > +Setting local-fm=true indicates that this QEMU instance has the CCI mailbox
> > +attached. Setting it false will create listeners for commands from a remote
> > +QEMU process (yet to be implemented).
>
> Nice but make that the default for now (And drop the parameter).
> Absence of a connected CCI might be sufficient though that's a bit ugly
> to check.
>
> > +
> > +An example of how the topology is described on the CLI is shown below:
> > +
> > + -object cxl-vcs-switch,id=vcs0,usp-ppbs=2,dsp-ppbs=4,local-fm=true \
> Interesting. I'd kind of like it to be a device, but it has no presence
> on any bus in of itself (arguably it is on a whole load of them). So maybe not.
>
I mentioned this briefly in the cover letter (sorry if unclear). I would
have liked it to be a device as well, but I couldn't figure out how
to do this given the way QEMU associates the buses and
devices to one another in the QOM tree. You can't not give it a bus
option, becasue it will then choose a single default bus to associate
with. Im unsure about adding multiple bus association. Doing this will require
deeper changes to the qdev/qbus which might be messy or unpalatable to upstream
folk. I made my changes to qbus as minimal as possible.
> > + -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.0,hdm_for_passthrough=true \
>
> Small side note - avoid the passthrough trick. It means a bunch of code
> paths aren't exercised and has hidden various OS bugs.
>
> > + -device cxl-rp,port=0,bus=cxl.0,id=root_port1,chassis=0,slot=1 \
> > + -device pxb-cxl,bus_nr=22,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
> > + -device cxl-rp,port=0,bus=cxl.1,id=root_port2,chassis=1,slot=1 \
> > + -device cxl-upstream,port=0,sn=1234,bus=root_port1,id=us0,addr=0.0,multifunction=on,vcs=vcs0,usppb=0 \
> > + -device cxl-upstream,port=0,sn=5678,bus=root_port2,id=us1,addr=0.0,multifunction=on,vcs=vcs0,usppb=1 \
>
> How can we have two upstream ports in a single vcs? To me those are separate VCSs
> where a VCS is normally a tree topology below a given USP.
>
> I think we have a terminology problem. If I read this right you are using VCS
> to mean the whole physical switch? Been a little while but I don't think
> that corresponds at all to it's meaning in the CXL Spec. Your VCS0/1 below
> are right.
>
Yeah I was struggling with the naming somewhat trying not to conflate a "VCS
capable switch" with the VCS within the switch itself. Seems I didn't do a
great job. Just to be clear 1 VCS = 1 USP. The whole switch is the collection
of all the VCS's. The id's here of the CLI devices is misleading I see
that.
>
>
> > + -device cxl-switch-mailbox-cci,bus=root_port1,addr=0.3,target=vcs0 \
> > + -device usb-cxl-mctp,bus=ehci.0,id=usb0,target=vcs0 \
> > + -device cxl-downstream,port=0,bus=us0,id=dsp0,slot=3 \
> > + -device cxl-downstream,port=1,bus=us0,id=dsp1,slot=4 \
> > + -device cxl-downstream,port=0,bus=us1,id=dsp2,slot=7 \
> > + -device cxl-downstream,port=1,bus=us1,id=dsp3,slot=8 \
> Ok. So these only know they are virtual because they are connected to a virtual USP.
> Might be enough - or we might want to make that more explicit via
> a new device type.
>
> > + -device cxl-type3,persistent-memdev=cxl-mem1,id=cxl-ep1,lsa=cxl-lsa1,sn=99,vcs=vcs0,dsppb=0 \
> > + -device cxl-type3,persistent-memdev=cxl-mem2,id=cxl-ep2,lsa=cxl-lsa2,sn=100,vcs=vcs0,dsppb=1 \
> > + -device cxl-type3,persistent-memdev=cxl-mem3,id=cxl-ep3,lsa=cxl-lsa3,sn=101,vcs=vcs0,dsppb=2 \
> > + -device cxl-type3,persistent-memdev=cxl-mem4,id=cxl-ep4,lsa=cxl-lsa4,sn=102,vcs=vcs0,dsppb=3 \
> This I mention above. I 'think' you are using the dsppb to instantiate something that is pretending
> to be a the physical DSP.
> I haven't yet read thee series, but gut feeling is that will make the querying of link
> properties etc rather different from the normal case.
>
> > + -machine cxl-fmw.0.targets.0=cxl.0,cxl-fmw.0.size=8G,cxl-fmw.1.targets.0=cxl.1,cxl-fmw.1.size=8G
> > +
> > +Example topology involving VCS switching::
> > +
> > + +--------------------+ +--------------------+
> > + | Host Bridge 0 | | Host Bridge 1 |
> > + +----------+---------+ +----------+---------+
> > + +-------+ | |
> > + | MCTP | | |
> > + | USB/ | +----------+---------+ +----------+---------+
> > + | I2C | | Root Port 0 | | Root Port 1 |
> > + +-----+-+ +----------+---------+ +----------+---------+
> > + | | |
> > + | | |
> > + +------|---------------+-----------------------+-----------------------+
> > + | +-+--------+ | cxl-vcs-switch (vcs0)| |
> > + | +--| CCI MBOX |---* | | |
> > + | | +----------+ | | |
> > + | | +-----------------+--------+ +-------+------------------+ |
> > + | +--+ | VCS0 | *---+ | VCS1 | |
> > + | | +---------------+------+ | | +-----+----------------+ | |
> > + | | | | | | | | | |
> > + | | | USP 0 | | | | USP 1 | | |
> > + | | | | | | | | | |
> > + | | +----+------------+----+ | | +----+------------+----+ | |
> > + | | | | | | | | | |
> > + | | +----+----+ +----+----+ | | +----+----+ +----+----+ | |
> > + | | | DSP 0 | | DSP 1 | | | | DSP 2 | | DSP 3 | | |
> > + | | |(vPPB 0) | |(vPPB 1) | | | |(vPPB 0) | |(vPPB 1) | | |
> > + | | | | | | | | | | | | | |
> > + | | +---------+ +---------+ | | +---------+ +----+----+ | |
> > + | +--------------------------+ +-------------------+------+ |
> > + | | |
> > + | +----------------------------------------------+ |
> > + | | |
> > + | | - - - |
> > + +-----------|------------|--------------------|------------|-----------+
> > + | | | |
> > + +---------+ +---------+ +---------+ +---------+
> > + |CXL/PCIe | |CXL/PCIe | |CXL/PCIe | |CXL/PCIe |
> > + | EP 0 | | EP 1 | | EP 2 | | EP 3 |
> > + | (PPB0) | | (PPB1) | | (PPB2) | | (PPB3) |
> > + +---------+ +---------+ +---------+ +---------+
> > + PPB0 Bound to VCS1, vPPB1. Others unbound...
> > +
> Good to have the diagram as makes it easier to discuss.
>
> What you have here is a bit of a hack because only some entities created
> exist in the command line - the others are spun up implicitly. I suspect
> we really want to make them explicit. The one thing I never looked into in
> the following is how hard it would be to poke a vDSP in front of a physical
> DSP and basically proxy stuff through or not. Some stuff will be programmed
> at boot (windows etc for hotplug later) but other stuff will fire in the hotplug
> flow on an attach of a physical port. Will need some care and stitching up
> memory regions across the boundary.
>
I think that this is really the crux of the issue with this series, and
all relates to the first question I posed in the cover letter and some of the
other comments you make on other patches, and with my comments below.
The way these patches work currently make the software side of the
hotplug on binding trivial, but they disallow FM comms to the device when it is
not bound. If we solve this and have realized devices dangling off the
DSPs, then it makes the hotplug to the guest more complex.
The issue is that the hotplug events and the realize/exit functions seem so
tied up with one another. I'm unsure at the moment how to decouple them cleanly
in the VCS case.
> The command line I'd be looking at for this as a target (feel free to shoot
> at it) would be something like (I went with one PXB - but need to test both options).
> Note some of this is probably garbage as I haven't checked parameters are right.
> -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.0 \
> -device cxl-rp,bus,cxl.0,id=root_port1...
> -device cxl-rp,bus,cxl.0,id=root_port2..
> -device cxl-upstream,port=0,sn=1234,bus=root_port1,id=us0,addr=0.0,multifunction=on,virtual=on \
> -device cxl-upstream,port=0,sn=5678,bus=root_port2,id=us1,addr=0.0,virtual=on \
>
> #note I extended current target to a list
> -device cxl-virtual-downstream,vport=0,bus=us0,id=vppb0 \
> -device cxl-virtual-downstream,vport=1,bus=us0,id=vppb0 \
> -device cxl-virtual-downstream,vport=2,bus=us0,id=vppb0 \
> -device cxl-virtual-downstream,vport=0,bus=us1,id=vppb0 \
> -device cxl-virtual-downstream,vport=1,bus=us1,id=vppb0 \
> -device cxl-virtual-downstream,vport=2,bus=us1,id=vppb0 \
> # Note more virtual ports than physical - likely common situation.
> -object cxl-switch,usps.0=usp0,usps.1=usp1,id=vsw0 \
> #list of usps so we can navigate downwards from this.
> -device cxl-switch-mailbox-cci,id=swcci0,bus=root_por1,multifunction=on,target=vsw0\
> # Maybe hang the unconnected physical dsps on a bus created by the cxl-switch?
I had tried writing it like this originally. The issue comes from the
fact that having a "bus" implies something has been added to the QOM Tree, which
forces an association to the guest. When actually we need this whole bus
to be invisible to the guest (but visible to an FM) on boot.
I'm sure this can be solved, but I can't see a way of doing it wihtout
more invasive changes to the qbus/qdev code (patch 2), and conceptually
to how the QOMTree represents the whole machine in general. I'm not sure yet
if this could open up a can of worms with other stuff going on deeper in the
guts of QEMU...
Ultimately this is at the boundary of my knowledge of QEMU internals
currently, but I think something like this might be necessary to
solve the issues regarding FM communication with the
unbound devices etc. I would really appreciate input about this
specific issue.
> -device cxl-downstream,port=0,bus=vsw0,id=dsp0,slot=3 \
> -device cxl-downstream,port=1,bus=vsw0,id=dsp1,slot=4 \
> -device cxl-downstream,port=2,bus=vsw0,id=dsp2,slot=7 \
> -device cxl-downstream,port=3,bus=vsw0,id=dsp3,slot=8 \
> #ideally a device but need to think where to hang it.
> -device cxl-type3,persistent-memdev=cxl-mem1,id=cxl-ep1,lsa=cxl-lsa1,sn=99,bus=dsp0 \
> -device cxl-type3,persistent-memdev=cxl-mem2,id=cxl-ep2,lsa=cxl-lsa2,sn=100,bus=dsp1 \
> -device cxl-type3,persistent-memdev=cxl-mem3,id=cxl-ep3,lsa=cxl-lsa3,sn=101,bus=dsp2 \
> #note not all DSPs have anything on them.
>
> Few reasons for this structure.
> 1) The unconnected physical port - we want to make sure physical hotplug works both
> when not associated with a VCS and when it is.
> 2) We need to be able to talk to EPs via FM interfaces when they aren't connected
> Given we have to make that look like PCI, let's make it PCI. I'm not sure how
> much hackery that will take as we'll need to do some level of enumeration from
> the the switch controller. Only need that once we want to do more than check
> training etc though - so maybe job for another day. In theory we can do everything
> with devices in that state (be it slowly) so would need all the addresses programmed
> etc. Not as general as current discussions on enumerating full PCI bus in QEMU as
> all direct connect.
What discussions are you refering to?
>
> Anyhow it's fiddly with this scheme but I think a little more general
> than your current one and closer representation of the hardware which will
> matter as we add all the introspection stuff etc in the FMAPI.
>
I agree fully. It would be much better if I could get this general
solution. I just need to figure out how to do it...
Cheers,
Josh
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [RFC QEMU PATCH 02/10] qdev/qbus: Allow hidden devices to be busless on QEMU startup
2026-05-20 10:02 ` Jonathan Cameron
@ 2026-05-21 9:59 ` Joshua Lant
2026-05-21 10:23 ` Joshua Lant
0 siblings, 1 reply; 19+ messages in thread
From: Joshua Lant @ 2026-05-21 9:59 UTC (permalink / raw)
To: Jonathan Cameron; +Cc: linux-cxl, qemu-devel, Jonathan.Cameron, arpit1.kumar
> On Wed, 29 Apr 2026 14:48:36 +0100
> Joshua Lant <joshualant@gmail.com> wrote:
>
> > Extend the capability for hiding devices, introduced for virtio-net
> > device in:
> >
> > commit f3a8505656935cde32e28c1c6317f725084da1e0
> > Author: Jens Freimann <jfreimann@redhat.com>
> > Date: Tue Oct 29 12:48:55 2019 +0100
> > qdev/qbus: add hidden device support
> >
> > Currently only endpoint devices can be hidden with a primary device
> > and failover (known static configuration). However, looking at future
> > composable systems, we see a need for hidden devices which have no associated
> > bus upon boot. Move the check for hidden devices to before the bus
> > search, and if it is hidden ignore the case where the device was
> > described on the CLI without the "bus=" field.
> >
> > This is motivated by a specific use-case: implementing the VCS
> > command set, part of the CXL specification (CXL r3.2 Section 7.1.3). In
> > this scenario a switch controlled by a Fabric Manager is able to change the
> > virtual hierarchy of devices seen by a guest within a fixed physical system
> > topology. The connecting bus is not known until runtime when
> > a bind command is issued by the Fabric Manager.
>
> So this comes back to the earlier comment on whether they are completely
> hidden from a guest. They aren't quite - we need to be able to
> get to them via the MCTP or cxl-switch-cci (in band mailbox) paths.
>
> Maybe we can solve that later without any compatibility problems
> or by cheating and just searching hidden devices. I'm not sure.
Funnily enough I had something like this when I was originally writing
this series. I scrapped it becasue I ran into issues. I was storing
as a tree the QDicts of hidden busses and devices below them, but then realised
I couldnt hotplug into the USP and abandoned it. I was doing this with the
aim of eventual support for cascaded switches.
There was another issue I cant fully remember... That the searching was easy
when it was referring to unrealized devices with QDicts from the CLI, but it
became much more difficult to get proper reference to some structures I needed
from the devices inside the qdev/qbus code when they were realized...
> >
> > Signed-off-by: Joshua Lant <joshualant@gmail.com>
> > ---
> > system/qdev-monitor.c | 10 ++++++++--
> > 1 file changed, 8 insertions(+), 2 deletions(-)
> >
> > diff --git a/system/qdev-monitor.c b/system/qdev-monitor.c
> > index f2aa400a77..b51dfe0645 100644
> > --- a/system/qdev-monitor.c
> > +++ b/system/qdev-monitor.c
> > @@ -650,6 +650,7 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
> > DeviceState *dev;
> > BusState *bus = NULL;
> > QDict *properties;
> > + bool hide_device;
> >
> > driver = qdict_get_try_str(opts, "driver");
> > if (!driver) {
> > @@ -663,6 +664,11 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
> > return NULL;
> > }
> >
> > + /* Is the device hidden from the guest?
> > + * If yes, no need to find a default bus if none given...
> > + * Bus could be provided at runtime (i.e. in a switch)*/
> > + hide_device = qdev_should_hide_device(opts, from_json, errp);
> > +
> > /* find bus */
> > path = qdict_get_try_str(opts, "bus");
> > if (path != NULL) {
> > @@ -675,14 +681,14 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
> > driver, object_get_typename(OBJECT(bus)));
> > return NULL;
> > }
> > - } else if (dc->bus_type != NULL) {
> > + } else if (dc->bus_type != NULL && !hide_device) {
> > bus = qdev_find_default_bus(dc, errp);
> > if (!bus) {
> > return NULL;
> > }
> > }
> >
> > - if (qdev_should_hide_device(opts, from_json, errp)) {
> > + if (hide_device) {
> > if (bus && !qbus_is_hotpluggable(bus)) {
> > error_setg(errp, "Bus '%s' does not support hotplugging",
> > bus->name);
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [RFC QEMU PATCH 02/10] qdev/qbus: Allow hidden devices to be busless on QEMU startup
2026-05-21 9:59 ` Joshua Lant
@ 2026-05-21 10:23 ` Joshua Lant
0 siblings, 0 replies; 19+ messages in thread
From: Joshua Lant @ 2026-05-21 10:23 UTC (permalink / raw)
To: Jonathan Cameron; +Cc: linux-cxl, qemu-devel, Jonathan.Cameron, arpit1.kumar
> > On Wed, 29 Apr 2026 14:48:36 +0100
> > Joshua Lant <joshualant@gmail.com> wrote:
> >
> > > Extend the capability for hiding devices, introduced for virtio-net
> > > device in:
> > >
> > > commit f3a8505656935cde32e28c1c6317f725084da1e0
> > > Author: Jens Freimann <jfreimann@redhat.com>
> > > Date: Tue Oct 29 12:48:55 2019 +0100
> > > qdev/qbus: add hidden device support
> > >
> > > Currently only endpoint devices can be hidden with a primary device
> > > and failover (known static configuration). However, looking at future
> > > composable systems, we see a need for hidden devices which have no associated
> > > bus upon boot. Move the check for hidden devices to before the bus
> > > search, and if it is hidden ignore the case where the device was
> > > described on the CLI without the "bus=" field.
> > >
> > > This is motivated by a specific use-case: implementing the VCS
> > > command set, part of the CXL specification (CXL r3.2 Section 7.1.3). In
> > > this scenario a switch controlled by a Fabric Manager is able to change the
> > > virtual hierarchy of devices seen by a guest within a fixed physical system
> > > topology. The connecting bus is not known until runtime when
> > > a bind command is issued by the Fabric Manager.
> >
> > So this comes back to the earlier comment on whether they are completely
> > hidden from a guest. They aren't quite - we need to be able to
> > get to them via the MCTP or cxl-switch-cci (in band mailbox) paths.
> >
> > Maybe we can solve that later without any compatibility problems
> > or by cheating and just searching hidden devices. I'm not sure.
>
> Funnily enough I had something like this when I was originally writing
> this series. I scrapped it becasue I ran into issues. I was storing
> as a tree the QDicts of hidden busses and devices below them, but then realised
> I couldnt hotplug into the USP and abandoned it. I was doing this with the
> aim of eventual support for cascaded switches.
>
> There was another issue I cant fully remember... That the searching was easy
> when it was referring to unrealized devices with QDicts from the CLI, but it
> became much more difficult to get proper reference to some structures I needed
> from the devices inside the qdev/qbus code when they were realized...
>
I just remembered... I was having to introduce cxl specific code into the qdev
code... I'm sure it could be separated but would require a fair bit more work.
> > >
> > > Signed-off-by: Joshua Lant <joshualant@gmail.com>
> > > ---
> > > system/qdev-monitor.c | 10 ++++++++--
> > > 1 file changed, 8 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/system/qdev-monitor.c b/system/qdev-monitor.c
> > > index f2aa400a77..b51dfe0645 100644
> > > --- a/system/qdev-monitor.c
> > > +++ b/system/qdev-monitor.c
> > > @@ -650,6 +650,7 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
> > > DeviceState *dev;
> > > BusState *bus = NULL;
> > > QDict *properties;
> > > + bool hide_device;
> > >
> > > driver = qdict_get_try_str(opts, "driver");
> > > if (!driver) {
> > > @@ -663,6 +664,11 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
> > > return NULL;
> > > }
> > >
> > > + /* Is the device hidden from the guest?
> > > + * If yes, no need to find a default bus if none given...
> > > + * Bus could be provided at runtime (i.e. in a switch)*/
> > > + hide_device = qdev_should_hide_device(opts, from_json, errp);
> > > +
> > > /* find bus */
> > > path = qdict_get_try_str(opts, "bus");
> > > if (path != NULL) {
> > > @@ -675,14 +681,14 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
> > > driver, object_get_typename(OBJECT(bus)));
> > > return NULL;
> > > }
> > > - } else if (dc->bus_type != NULL) {
> > > + } else if (dc->bus_type != NULL && !hide_device) {
> > > bus = qdev_find_default_bus(dc, errp);
> > > if (!bus) {
> > > return NULL;
> > > }
> > > }
> > >
> > > - if (qdev_should_hide_device(opts, from_json, errp)) {
> > > + if (hide_device) {
> > > if (bus && !qbus_is_hotpluggable(bus)) {
> > > error_setg(errp, "Bus '%s' does not support hotplugging",
> > > bus->name);
> >
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2026-05-21 10:23 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-04-29 13:48 [RFC QEMU PATCH 00/10] Initial Support for VCS Switching Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 01/10] docs: Add documentation for cxl-vcs-switch Joshua Lant
2026-05-19 14:33 ` Jonathan Cameron
2026-05-21 9:42 ` Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 02/10] qdev/qbus: Allow hidden devices to be busless on QEMU startup Joshua Lant
2026-05-20 10:02 ` Jonathan Cameron
2026-05-21 9:59 ` Joshua Lant
2026-05-21 10:23 ` Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 03/10] cxl-type3: Properly unmap the memory-backend on device exit Joshua Lant
2026-05-19 15:01 ` Jonathan Cameron
2026-04-29 13:48 ` [RFC QEMU PATCH 04/10] cxl_downstream: enable power controller present capability Joshua Lant
2026-05-19 15:03 ` Jonathan Cameron
2026-04-29 13:48 ` [RFC QEMU PATCH 05/10] cxl-vcs-switch: Initial support for CXL VCS Joshua Lant
2026-05-19 15:19 ` Jonathan Cameron
2026-04-29 13:48 ` [RFC QEMU PATCH 06/10] cxl-upstream-port: Add support for targeting a VCS switch Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 07/10] cxl-downstream-port: Add support for VCS switching Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 08/10] cxl-cci-mailbox: Add support for targeting a VCS switch Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 09/10] cxl-mailbox-utils: Add support for VCS bind/unbind commands Joshua Lant
2026-04-29 13:48 ` [RFC QEMU PATCH 10/10] cxl-mailbox-utils: Add support for VCS Get Virtual CXL Switch Info command Joshua Lant
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.