* [RFC PATCH qemu 0/5] hw/cxl/mctp/i2c/usb: MCTP for OoB control of CXL devices.
@ 2025-06-09 16:33 Jonathan Cameron
2025-06-09 16:33 ` [RFC PATCH qemu 1/5] hw/i2c: add smbus pec utility function Jonathan Cameron
` (5 more replies)
0 siblings, 6 replies; 8+ messages in thread
From: Jonathan Cameron @ 2025-06-09 16:33 UTC (permalink / raw)
To: Klaus Jensen, cminyard, Fan Ni, Anisa Su, qemu-devel, linux-cxl,
mst
Cc: linuxarm, Philippe Mathieu-Daudé
This posting is primarily about sharing the USB device emulation to get some
early feedback.
RFC reasons:
- Known 'inaccuracies' in emulation (not obeying MTU in the to host direction for
example)./
- Not sure what to do wrt to Klaus' I2C MCTP support given that has been stalled
for some time. For now only the headers are really shared between the
two implementations.
- This is more of an FYI / request for testing than a formal suggestion that this
might be ready for upstream.
Why add a CXL FM-API over MCTP over USB device?
- Can be emulated on pretty much any host system as USB is discoverable and
expandable. If you want a giggle, see the hacks on i386/pc and arm/virt on
we've been using until now given only I2C controller that works is the aspeed
one. e.g. https://gitlab.com/jic23/qemu/-/commit/134c2e3952b
- Being able to talk to both the fabric management out of band interfaces
and the in band devices on the same host makes testing much simpler.
Background:
Back in 2022 I posted some support for controlling the CXL fabric via the
spec defined out of band interfaces (CXL Fabric Management API - FM-API)
over MCTP on I2C
https://lore.kernel.org/qemu-devel/20220520170128.4436-1-Jonathan.Cameron@huawei.com/
I reworked that on top of the NVME-MI work from Klauss.
To that end I hacked the aspeed-i2c controller onto both i386/pc and arm/virt
and posted kernel patches to enabled ACPI support for that device (more or less).
It worked and has been useful in the meantime, but adding that i2c controller
to those boards was obviously not going to be upstreamable - and to build
reliable tests against it I don't want to carry this out of tree for ever.
I messed around with a PCI hosted aspeed controller and might come back to
that at some point.
In the meantime, DMTF published a transport binding for MCTP over USB
https://www.dmtf.org/sites/default/files/standards/documents/DSP0283_1.0.1.pdf
Kernel support duly followed early this year: drivers/net/mctp/mctp-usb.c
https://codeconstruct.com.au/docs/mctp-over-usb/
Given the ease of adding a suitable USB controller on a PCI bus, emulating a
suitable endpoint provides what I think is an upstreamable solution.
To use this:
-device usb-ehci,id=ehci
-device usb-cxl-mctp,bus=ehci.0,id=fred,target=us0
where target is either a CXL switch upstream port, or a type 3 device.
Then install the mctp userspace tools in your guest and configure it with
mctp addr add 8 dev mctpusb0
mctp link set mctpusb0 net 11
mctp link set mctpusb0 up
systemctl start mctpd.service
busctl call au.com.codeconstruct.MCTP1 /au/com/codeconstruct/mctp1/interfaces/mctpusb0 au.com.codeconstruct.MCTP.BusOwner1 SetupEndpoint ay 0
I've been testing the CXL commands with
https://gitlab.com/jic23/cxl-fmapi-tests
(mostly because I still had them in my image from the i2c work)
but libcxlmi is probably a better bet
https://github.com/computexpresslink/libcxlmi
I'll post a tree with this on at gitlab.com/jic23/qemu shortly
(cxl-<latest date>).
Jonathan Cameron (3):
hw/cxl/i2c_mctp_cxl: Initial device emulation
docs: cxl: Add example commandline for MCTP CXL CCIs
usb/mctp/cxl: CXL FMAPI interface via MCTP over usb.
Klaus Jensen (2):
hw/i2c: add smbus pec utility function
hw/i2c: add mctp core
MAINTAINERS | 7 +
docs/system/devices/cxl.rst | 27 +
include/hw/cxl/cxl_device.h | 8 +
include/hw/i2c/mctp.h | 125 +++++
include/hw/i2c/smbus_master.h | 2 +
include/hw/pci-bridge/cxl_upstream_port.h | 1 +
include/hw/usb.h | 1 +
include/net/mctp.h | 100 ++++
hw/cxl/cxl-mailbox-utils.c | 49 ++
hw/cxl/i2c_mctp_cxl.c | 289 ++++++++++
hw/i2c/mctp.c | 414 ++++++++++++++
hw/i2c/smbus_master.c | 26 +
hw/usb/dev-mctp.c | 639 ++++++++++++++++++++++
hw/arm/Kconfig | 1 +
hw/cxl/Kconfig | 4 +
hw/cxl/meson.build | 4 +
hw/i2c/Kconfig | 4 +
hw/i2c/meson.build | 1 +
hw/i2c/trace-events | 14 +
hw/usb/Kconfig | 5 +
hw/usb/meson.build | 1 +
21 files changed, 1722 insertions(+)
create mode 100644 include/hw/i2c/mctp.h
create mode 100644 include/net/mctp.h
create mode 100644 hw/cxl/i2c_mctp_cxl.c
create mode 100644 hw/i2c/mctp.c
create mode 100644 hw/usb/dev-mctp.c
--
2.48.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [RFC PATCH qemu 1/5] hw/i2c: add smbus pec utility function
2025-06-09 16:33 [RFC PATCH qemu 0/5] hw/cxl/mctp/i2c/usb: MCTP for OoB control of CXL devices Jonathan Cameron
@ 2025-06-09 16:33 ` Jonathan Cameron
2025-06-09 16:33 ` [RFC PATCH qemu 2/5] hw/i2c: add mctp core Jonathan Cameron
` (4 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Jonathan Cameron @ 2025-06-09 16:33 UTC (permalink / raw)
To: Klaus Jensen, cminyard, Fan Ni, Anisa Su, qemu-devel, linux-cxl,
mst
Cc: linuxarm, Philippe Mathieu-Daudé
From: Klaus Jensen <k.jensen@samsung.com>
Add i2c_smbus_pec() to calculate the SMBus Packet Error Code for a
message.
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
Acked-by: Corey Minyard <cminyard@mvista.com>
Link: https://lore.kernel.org/r/20230914-nmi-i2c-v6-1-11bbb4f74d18@samsung.com
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/i2c/smbus_master.h | 2 ++
hw/i2c/smbus_master.c | 26 ++++++++++++++++++++++++++
2 files changed, 28 insertions(+)
diff --git a/include/hw/i2c/smbus_master.h b/include/hw/i2c/smbus_master.h
index bb13bc423c..d90f81767d 100644
--- a/include/hw/i2c/smbus_master.h
+++ b/include/hw/i2c/smbus_master.h
@@ -27,6 +27,8 @@
#include "hw/i2c/i2c.h"
+uint8_t i2c_smbus_pec(uint8_t crc, uint8_t *buf, size_t len);
+
/* Master device commands. */
int smbus_quick_command(I2CBus *bus, uint8_t addr, int read);
int smbus_receive_byte(I2CBus *bus, uint8_t addr);
diff --git a/hw/i2c/smbus_master.c b/hw/i2c/smbus_master.c
index 6a53c34e70..01a8e47002 100644
--- a/hw/i2c/smbus_master.c
+++ b/hw/i2c/smbus_master.c
@@ -15,6 +15,32 @@
#include "hw/i2c/i2c.h"
#include "hw/i2c/smbus_master.h"
+static uint8_t crc8(uint16_t data)
+{
+ int i;
+
+ for (i = 0; i < 8; i++) {
+ if (data & 0x8000) {
+ data ^= 0x1070U << 3;
+ }
+
+ data <<= 1;
+ }
+
+ return (uint8_t)(data >> 8);
+}
+
+uint8_t i2c_smbus_pec(uint8_t crc, uint8_t *buf, size_t len)
+{
+ int i;
+
+ for (i = 0; i < len; i++) {
+ crc = crc8((crc ^ buf[i]) << 8);
+ }
+
+ return crc;
+}
+
/* Master device commands. */
int smbus_quick_command(I2CBus *bus, uint8_t addr, int read)
{
--
2.48.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [RFC PATCH qemu 2/5] hw/i2c: add mctp core
2025-06-09 16:33 [RFC PATCH qemu 0/5] hw/cxl/mctp/i2c/usb: MCTP for OoB control of CXL devices Jonathan Cameron
2025-06-09 16:33 ` [RFC PATCH qemu 1/5] hw/i2c: add smbus pec utility function Jonathan Cameron
@ 2025-06-09 16:33 ` Jonathan Cameron
2025-06-09 16:33 ` [RFC PATCH qemu 3/5] hw/cxl/i2c_mctp_cxl: Initial device emulation Jonathan Cameron
` (3 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Jonathan Cameron @ 2025-06-09 16:33 UTC (permalink / raw)
To: Klaus Jensen, cminyard, Fan Ni, Anisa Su, qemu-devel, linux-cxl,
mst
Cc: linuxarm, Philippe Mathieu-Daudé
From: Klaus Jensen <k.jensen@samsung.com>
Add an abstract MCTP over I2C endpoint model. This implements MCTP
control message handling as well as handling the actual I2C transport
(packetization).
Devices are intended to derive from this and implement the class
methods.
Parts of this implementation is inspired by code[1] previously posted by
Jonathan Cameron.
Squashed a fix[2] from Matt Johnston.
[1]: https://lore.kernel.org/qemu-devel/20220520170128.4436-1-Jonathan.Cameron@huawei.com/
[2]: https://lore.kernel.org/qemu-devel/20221121080445.GA29062@codeconstruct.com.au/
Tested-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
Acked-by: Corey Minyard <cminyard@mvista.com>
Link: https://lore.kernel.org/r/20230914-nmi-i2c-v6-2-11bbb4f74d18@samsung.com
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
MAINTAINERS | 7 +
include/hw/i2c/mctp.h | 125 ++++++++++++
include/net/mctp.h | 35 ++++
hw/i2c/mctp.c | 431 ++++++++++++++++++++++++++++++++++++++++++
hw/arm/Kconfig | 1 +
hw/i2c/Kconfig | 4 +
hw/i2c/meson.build | 1 +
hw/i2c/trace-events | 14 ++
8 files changed, 618 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index aa6763077e..4ad35a1aa3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3782,6 +3782,13 @@ F: include/hw/fsi/*
F: docs/specs/fsi.rst
F: tests/qtest/aspeed_fsi-test.c
+MCTP I2C Transport
+M: Klaus Jensen <k.jensen@samsung.com>
+S: Maintained
+F: hw/i2c/mctp.c
+F: include/hw/i2c/mctp.h
+F: include/net/mctp.h
+
Firmware schema specifications
M: Philippe Mathieu-Daudé <philmd@linaro.org>
R: Daniel P. Berrange <berrange@redhat.com>
diff --git a/include/hw/i2c/mctp.h b/include/hw/i2c/mctp.h
new file mode 100644
index 0000000000..10c3fb9048
--- /dev/null
+++ b/include/hw/i2c/mctp.h
@@ -0,0 +1,125 @@
+#ifndef QEMU_I2C_MCTP_H
+#define QEMU_I2C_MCTP_H
+
+#include "qom/object.h"
+#include "hw/qdev-core.h"
+
+#define TYPE_MCTP_I2C_ENDPOINT "mctp-i2c-endpoint"
+OBJECT_DECLARE_TYPE(MCTPI2CEndpoint, MCTPI2CEndpointClass, MCTP_I2C_ENDPOINT)
+
+struct MCTPI2CEndpointClass {
+ I2CSlaveClass parent_class;
+
+ /**
+ * put_buf() - receive incoming message fragment
+ *
+ * Return 0 for success or negative for error.
+ */
+ int (*put_buf)(MCTPI2CEndpoint *mctp, uint8_t *buf, size_t len);
+
+ /**
+ * get_buf() - provide pointer to message fragment
+ *
+ * Called by the mctp subsystem to request a pointer to the next message
+ * fragment. Subsequent calls MUST return next fragment (if any).
+ *
+ * Must return the number of bytes in message fragment.
+ */
+ size_t (*get_buf)(MCTPI2CEndpoint *mctp, const uint8_t **buf,
+ size_t maxlen, uint8_t *mctp_flags);
+
+ /**
+ * handle() - handle an MCTP message
+ *
+ * Called by the mctp subsystem when a full message has been delivered and
+ * may be parsed and processed.
+ */
+ void (*handle)(MCTPI2CEndpoint *mctp);
+
+ /**
+ * reset() - reset internal state
+ *
+ * Called by the mctp subsystem in the event of some transport error.
+ * Implementation must reset its internal state and drop any fragments
+ * previously receieved.
+ */
+ void (*reset)(MCTPI2CEndpoint *mctp);
+
+ /**
+ * get_types() - provide supported mctp message types
+ *
+ * Must provide a buffer with a full MCTP supported message types payload
+ * (i.e. `0x0(SUCCESS),0x1(COUNT),0x4(NMI)`).
+ *
+ * Returns the size of the response.
+ */
+ size_t (*get_types)(MCTPI2CEndpoint *mctp, const uint8_t **data);
+};
+
+/*
+ * Maximum value of the SMBus Block Write "Byte Count" field (8 bits).
+ *
+ * This is the count of bytes that follow the Byte Count field and up to, but
+ * not including, the PEC byte.
+ */
+#define I2C_MCTP_MAXBLOCK 255
+
+/*
+ * Maximum Transmission Unit under I2C.
+ *
+ * This is for the MCTP Packet Payload (255, subtracting the 4 byte MCTP Packet
+ * Header and the 1 byte MCTP/I2C piggy-backed source address).
+ */
+#define I2C_MCTP_MAXMTU (I2C_MCTP_MAXBLOCK - (sizeof(MCTPPacketHeader) + 1))
+
+/*
+ * Maximum length of an MCTP/I2C packet.
+ *
+ * This is the sum of the three I2C header bytes (Destination target address,
+ * Command Code and Byte Count), the maximum number of bytes in a message (255)
+ * and the 1 byte Packet Error Code.
+ */
+#define I2C_MCTP_MAX_LENGTH (3 + I2C_MCTP_MAXBLOCK + 1)
+
+typedef enum {
+ I2C_MCTP_STATE_IDLE,
+ I2C_MCTP_STATE_RX_STARTED,
+ I2C_MCTP_STATE_RX,
+ I2C_MCTP_STATE_WAIT_TX,
+ I2C_MCTP_STATE_TX,
+} MCTPState;
+
+typedef enum {
+ I2C_MCTP_STATE_TX_START_SEND,
+ I2C_MCTP_STATE_TX_SEND_BYTE,
+} MCTPTxState;
+
+typedef struct MCTPI2CEndpoint {
+ I2CSlave parent_obj;
+ I2CBus *i2c;
+
+ MCTPState state;
+
+ /* mctp endpoint identifier */
+ uint8_t my_eid;
+
+ uint8_t buffer[I2C_MCTP_MAX_LENGTH];
+ uint64_t pos;
+ size_t len;
+
+ struct {
+ MCTPTxState state;
+ bool is_control;
+
+ uint8_t eid;
+ uint8_t addr;
+ uint8_t pktseq;
+ uint8_t tag;
+
+ QEMUBH *bh;
+ } tx;
+} MCTPI2CEndpoint;
+
+void i2c_mctp_schedule_send(MCTPI2CEndpoint *mctp);
+
+#endif /* QEMU_I2C_MCTP_H */
diff --git a/include/net/mctp.h b/include/net/mctp.h
new file mode 100644
index 0000000000..5d26d855db
--- /dev/null
+++ b/include/net/mctp.h
@@ -0,0 +1,35 @@
+#ifndef QEMU_MCTP_H
+#define QEMU_MCTP_H
+
+#include "hw/registerfields.h"
+
+/* DSP0236 1.3.0, Section 8.3.1 */
+#define MCTP_BASELINE_MTU 64
+
+/* DSP0236 1.3.0, Table 1, Message body */
+FIELD(MCTP_MESSAGE_H, TYPE, 0, 7)
+FIELD(MCTP_MESSAGE_H, IC, 7, 1)
+
+/* DSP0236 1.3.0, Table 1, MCTP transport header */
+FIELD(MCTP_H_FLAGS, TAG, 0, 3);
+FIELD(MCTP_H_FLAGS, TO, 3, 1);
+FIELD(MCTP_H_FLAGS, PKTSEQ, 4, 2);
+FIELD(MCTP_H_FLAGS, EOM, 6, 1);
+FIELD(MCTP_H_FLAGS, SOM, 7, 1);
+
+/* DSP0236 1.3.0, Figure 4 */
+typedef struct MCTPPacketHeader {
+ uint8_t version;
+ struct {
+ uint8_t dest;
+ uint8_t source;
+ } eid;
+ uint8_t flags;
+} MCTPPacketHeader;
+
+typedef struct MCTPPacket {
+ MCTPPacketHeader hdr;
+ uint8_t payload[];
+} MCTPPacket;
+
+#endif /* QEMU_MCTP_H */
diff --git a/hw/i2c/mctp.c b/hw/i2c/mctp.c
new file mode 100644
index 0000000000..cf45a46706
--- /dev/null
+++ b/hw/i2c/mctp.c
@@ -0,0 +1,431 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * SPDX-FileCopyrightText: Copyright (c) 2023 Samsung Electronics Co., Ltd.
+ * SPDX-FileContributor: Klaus Jensen <k.jensen@samsung.com>
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/main-loop.h"
+
+#include "hw/qdev-properties.h"
+#include "hw/i2c/i2c.h"
+#include "hw/i2c/smbus_master.h"
+#include "hw/i2c/mctp.h"
+#include "net/mctp.h"
+
+#include "trace.h"
+
+/* DSP0237 1.2.0, Figure 1 */
+typedef struct MCTPI2CPacketHeader {
+ uint8_t dest;
+#define MCTP_I2C_COMMAND_CODE 0xf
+ uint8_t command_code;
+ uint8_t byte_count;
+ uint8_t source;
+} MCTPI2CPacketHeader;
+
+typedef struct MCTPI2CPacket {
+ MCTPI2CPacketHeader i2c;
+ MCTPPacket mctp;
+} MCTPI2CPacket;
+
+#define i2c_mctp_payload_offset offsetof(MCTPI2CPacket, mctp.payload)
+#define i2c_mctp_payload(buf) (buf + i2c_mctp_payload_offset)
+
+/* DSP0236 1.3.0, Figure 20 */
+typedef struct MCTPControlMessage {
+#define MCTP_MESSAGE_TYPE_CONTROL 0x0
+ uint8_t type;
+#define MCTP_CONTROL_FLAGS_RQ (1 << 7)
+#define MCTP_CONTROL_FLAGS_D (1 << 6)
+ uint8_t flags;
+ uint8_t command_code;
+ uint8_t data[];
+} MCTPControlMessage;
+
+enum MCTPControlCommandCodes {
+ MCTP_CONTROL_SET_EID = 0x01,
+ MCTP_CONTROL_GET_EID = 0x02,
+ MCTP_CONTROL_GET_VERSION = 0x04,
+ MCTP_CONTROL_GET_MESSAGE_TYPE_SUPPORT = 0x05,
+};
+
+#define MCTP_CONTROL_ERROR_UNSUPPORTED_CMD 0x5
+
+#define i2c_mctp_control_data_offset \
+ (i2c_mctp_payload_offset + offsetof(MCTPControlMessage, data))
+#define i2c_mctp_control_data(buf) (buf + i2c_mctp_control_data_offset)
+
+/**
+ * The byte count field in the SMBUS Block Write containers the number of bytes
+ * *following* the field itself.
+ *
+ * This is at least 5.
+ *
+ * 1 byte for the MCTP/I2C piggy-backed I2C source address in addition to the
+ * size of the MCTP transport/packet header.
+ */
+#define MCTP_I2C_BYTE_COUNT_OFFSET (sizeof(MCTPPacketHeader) + 1)
+
+void i2c_mctp_schedule_send(MCTPI2CEndpoint *mctp)
+{
+ I2CBus *i2c = I2C_BUS(qdev_get_parent_bus(DEVICE(mctp)));
+
+ mctp->tx.state = I2C_MCTP_STATE_TX_START_SEND;
+
+ i2c_bus_master(i2c, mctp->tx.bh);
+}
+
+static void i2c_mctp_tx(void *opaque)
+{
+ DeviceState *dev = DEVICE(opaque);
+ I2CBus *i2c = I2C_BUS(qdev_get_parent_bus(dev));
+ I2CSlave *slave = I2C_SLAVE(dev);
+ MCTPI2CEndpoint *mctp = MCTP_I2C_ENDPOINT(dev);
+ MCTPI2CEndpointClass *mc = MCTP_I2C_ENDPOINT_GET_CLASS(mctp);
+ MCTPI2CPacket *pkt = (MCTPI2CPacket *)mctp->buffer;
+ uint8_t flags = 0;
+
+ switch (mctp->tx.state) {
+ case I2C_MCTP_STATE_TX_SEND_BYTE:
+ if (mctp->pos < mctp->len) {
+ uint8_t byte = mctp->buffer[mctp->pos];
+
+ trace_i2c_mctp_tx_send_byte(mctp->pos, byte);
+
+ /* send next byte */
+ i2c_send_async(i2c, byte);
+
+ mctp->pos++;
+
+ break;
+ }
+
+ /* packet sent */
+ i2c_end_transfer(i2c);
+
+ /* end of any control data */
+ mctp->len = 0;
+
+ /* fall through */
+
+ case I2C_MCTP_STATE_TX_START_SEND:
+ if (mctp->tx.is_control) {
+ /* packet payload is already in buffer; max 1 packet */
+ flags = FIELD_DP8(flags, MCTP_H_FLAGS, SOM, 1);
+ flags = FIELD_DP8(flags, MCTP_H_FLAGS, EOM, 1);
+ } else {
+ const uint8_t *payload;
+
+ /* get message bytes from derived device */
+ mctp->len = mc->get_buf(mctp, &payload, I2C_MCTP_MAXMTU, &flags);
+ assert(mctp->len <= I2C_MCTP_MAXMTU);
+
+ memcpy(pkt->mctp.payload, payload, mctp->len);
+ }
+
+ if (!mctp->len) {
+ trace_i2c_mctp_tx_done();
+
+ /* no more packets needed; release the bus */
+ i2c_bus_release(i2c);
+
+ mctp->state = I2C_MCTP_STATE_IDLE;
+ mctp->tx.is_control = false;
+
+ break;
+ }
+
+ mctp->state = I2C_MCTP_STATE_TX;
+
+ pkt->i2c = (MCTPI2CPacketHeader) {
+ .dest = mctp->tx.addr << 1,
+ .command_code = MCTP_I2C_COMMAND_CODE,
+ .byte_count = MCTP_I2C_BYTE_COUNT_OFFSET + mctp->len,
+
+ /* DSP0237 1.2.0, Figure 1 */
+ .source = slave->address << 1 | 0x1,
+ };
+
+ pkt->mctp.hdr = (MCTPPacketHeader) {
+ .version = 0x1,
+ .eid.dest = mctp->tx.eid,
+ .eid.source = mctp->my_eid,
+ .flags = flags,
+ };
+
+ pkt->mctp.hdr.flags = FIELD_DP8(pkt->mctp.hdr.flags, MCTP_H_FLAGS,
+ PKTSEQ, mctp->tx.pktseq++);
+ pkt->mctp.hdr.flags = FIELD_DP8(pkt->mctp.hdr.flags, MCTP_H_FLAGS, TAG,
+ mctp->tx.tag);
+
+ mctp->len += sizeof(MCTPI2CPacket);
+ assert(mctp->len < I2C_MCTP_MAX_LENGTH);
+
+ mctp->buffer[mctp->len] = i2c_smbus_pec(0, mctp->buffer, mctp->len);
+ mctp->len++;
+
+ trace_i2c_mctp_tx_start_send(mctp->len);
+
+ i2c_start_send_async(i2c, pkt->i2c.dest >> 1);
+
+ /* already "sent" the destination slave address */
+ mctp->pos = 1;
+
+ mctp->tx.state = I2C_MCTP_STATE_TX_SEND_BYTE;
+
+ break;
+ }
+}
+
+static void i2c_mctp_set_control_data(MCTPI2CEndpoint *mctp, const void * buf,
+ size_t len)
+{
+ assert(i2c_mctp_control_data_offset < I2C_MCTP_MAX_LENGTH - len);
+ memcpy(i2c_mctp_control_data(mctp->buffer), buf, len);
+
+ assert(mctp->len < I2C_MCTP_MAX_LENGTH - len);
+ mctp->len += len;
+}
+
+static void i2c_mctp_handle_control_set_eid(MCTPI2CEndpoint *mctp, uint8_t eid)
+{
+ mctp->my_eid = eid;
+
+ uint8_t buf[] = {
+ 0x0, 0x0, eid, 0x0,
+ };
+
+ i2c_mctp_set_control_data(mctp, buf, sizeof(buf));
+}
+
+static void i2c_mctp_handle_control_get_eid(MCTPI2CEndpoint *mctp)
+{
+ uint8_t buf[] = {
+ 0x0, mctp->my_eid, 0x0, 0x0,
+ };
+
+ i2c_mctp_set_control_data(mctp, buf, sizeof(buf));
+}
+
+static void i2c_mctp_handle_control_get_version(MCTPI2CEndpoint *mctp)
+{
+ uint8_t buf[] = {
+ 0x0, 0x1, 0x0, 0x1, 0x3, 0x1,
+ };
+
+ i2c_mctp_set_control_data(mctp, buf, sizeof(buf));
+}
+
+static void i2c_mctp_handle_get_message_type_support(MCTPI2CEndpoint *mctp)
+{
+ MCTPI2CEndpointClass *mc = MCTP_I2C_ENDPOINT_GET_CLASS(mctp);
+ const uint8_t *types;
+ size_t len;
+
+ len = mc->get_types(mctp, &types);
+ assert(mctp->len <= MCTP_BASELINE_MTU - len);
+
+ i2c_mctp_set_control_data(mctp, types, len);
+}
+
+static void i2c_mctp_handle_control(MCTPI2CEndpoint *mctp)
+{
+ MCTPControlMessage *msg = (MCTPControlMessage *)i2c_mctp_payload(mctp->buffer);
+
+ /* clear Rq/D */
+ msg->flags &= ~(MCTP_CONTROL_FLAGS_RQ | MCTP_CONTROL_FLAGS_D);
+
+ mctp->len = sizeof(MCTPControlMessage);
+
+ trace_i2c_mctp_handle_control(msg->command_code);
+
+ switch (msg->command_code) {
+ case MCTP_CONTROL_SET_EID:
+ i2c_mctp_handle_control_set_eid(mctp, msg->data[1]);
+ break;
+
+ case MCTP_CONTROL_GET_EID:
+ i2c_mctp_handle_control_get_eid(mctp);
+ break;
+
+ case MCTP_CONTROL_GET_VERSION:
+ i2c_mctp_handle_control_get_version(mctp);
+ break;
+
+ case MCTP_CONTROL_GET_MESSAGE_TYPE_SUPPORT:
+ i2c_mctp_handle_get_message_type_support(mctp);
+ break;
+
+ default:
+ trace_i2c_mctp_unhandled_control(msg->command_code);
+
+ msg->data[0] = MCTP_CONTROL_ERROR_UNSUPPORTED_CMD;
+ mctp->len++;
+
+ break;
+ }
+
+ assert(mctp->len <= MCTP_BASELINE_MTU);
+
+ i2c_mctp_schedule_send(mctp);
+}
+
+static int i2c_mctp_event_cb(I2CSlave *i2c, enum i2c_event event)
+{
+ MCTPI2CEndpoint *mctp = MCTP_I2C_ENDPOINT(i2c);
+ MCTPI2CEndpointClass *mc = MCTP_I2C_ENDPOINT_GET_CLASS(mctp);
+ MCTPI2CPacket *pkt = (MCTPI2CPacket *)mctp->buffer;
+ size_t payload_len;
+ uint8_t pec, pktseq, msgtype;
+ int ret;
+
+ switch (event) {
+ case I2C_START_SEND:
+ if (mctp->state == I2C_MCTP_STATE_IDLE) {
+ mctp->state = I2C_MCTP_STATE_RX_STARTED;
+ } else if (mctp->state != I2C_MCTP_STATE_RX) {
+ return -1;
+ }
+
+ /* the i2c core eats the slave address, so put it back in */
+ pkt->i2c.dest = i2c->address << 1;
+ mctp->len = 1;
+
+ return 0;
+
+ case I2C_FINISH:
+ if (mctp->len < sizeof(MCTPI2CPacket) + 1) {
+ trace_i2c_mctp_drop_short_packet(mctp->len);
+ goto drop;
+ }
+
+ payload_len = mctp->len - (1 + offsetof(MCTPI2CPacket, mctp.payload));
+
+ if (pkt->i2c.byte_count + 3 != mctp->len - 1) {
+ trace_i2c_mctp_drop_invalid_length(pkt->i2c.byte_count + 3,
+ mctp->len - 1);
+ goto drop;
+ }
+
+ pec = i2c_smbus_pec(0, mctp->buffer, mctp->len - 1);
+ if (mctp->buffer[mctp->len - 1] != pec) {
+ trace_i2c_mctp_drop_invalid_pec(mctp->buffer[mctp->len - 1], pec);
+ goto drop;
+ }
+
+ if (!(pkt->mctp.hdr.eid.dest == mctp->my_eid ||
+ pkt->mctp.hdr.eid.dest == 0)) {
+ trace_i2c_mctp_drop_invalid_eid(pkt->mctp.hdr.eid.dest,
+ mctp->my_eid);
+ goto drop;
+ }
+
+ pktseq = FIELD_EX8(pkt->mctp.hdr.flags, MCTP_H_FLAGS, PKTSEQ);
+
+ if (FIELD_EX8(pkt->mctp.hdr.flags, MCTP_H_FLAGS, SOM)) {
+ mctp->tx.is_control = false;
+
+ if (mctp->state == I2C_MCTP_STATE_RX) {
+ mc->reset(mctp);
+ }
+
+ mctp->state = I2C_MCTP_STATE_RX;
+
+ mctp->tx.addr = pkt->i2c.source >> 1;
+ mctp->tx.eid = pkt->mctp.hdr.eid.source;
+ mctp->tx.tag = FIELD_EX8(pkt->mctp.hdr.flags, MCTP_H_FLAGS, TAG);
+ mctp->tx.pktseq = pktseq;
+
+ msgtype = FIELD_EX8(pkt->mctp.payload[0], MCTP_MESSAGE_H, TYPE);
+
+ if (msgtype == MCTP_MESSAGE_TYPE_CONTROL) {
+ mctp->tx.is_control = true;
+
+ i2c_mctp_handle_control(mctp);
+
+ return 0;
+ }
+ } else if (mctp->state == I2C_MCTP_STATE_RX_STARTED) {
+ trace_i2c_mctp_drop_expected_som();
+ goto drop;
+ } else if (pktseq != (++mctp->tx.pktseq & 0x3)) {
+ trace_i2c_mctp_drop_invalid_pktseq(pktseq, mctp->tx.pktseq & 0x3);
+ goto drop;
+ }
+
+ ret = mc->put_buf(mctp, i2c_mctp_payload(mctp->buffer), payload_len);
+ if (ret < 0) {
+ goto drop;
+ }
+
+ if (FIELD_EX8(pkt->mctp.hdr.flags, MCTP_H_FLAGS, EOM)) {
+ mc->handle(mctp);
+ mctp->state = I2C_MCTP_STATE_WAIT_TX;
+ }
+
+ return 0;
+
+ default:
+ return -1;
+ }
+
+drop:
+ mc->reset(mctp);
+
+ mctp->state = I2C_MCTP_STATE_IDLE;
+
+ return 0;
+}
+
+static int i2c_mctp_send_cb(I2CSlave *i2c, uint8_t data)
+{
+ MCTPI2CEndpoint *mctp = MCTP_I2C_ENDPOINT(i2c);
+
+ if (mctp->len < I2C_MCTP_MAX_LENGTH) {
+ mctp->buffer[mctp->len++] = data;
+ return 0;
+ }
+
+ return -1;
+}
+
+static void i2c_mctp_instance_init(Object *obj)
+{
+ MCTPI2CEndpoint *mctp = MCTP_I2C_ENDPOINT(obj);
+
+ mctp->tx.bh = qemu_bh_new(i2c_mctp_tx, mctp);
+}
+
+static const Property mctp_i2c_props[] = {
+ DEFINE_PROP_UINT8("eid", MCTPI2CEndpoint, my_eid, 0x9),
+};
+
+static void i2c_mctp_class_init(ObjectClass *oc, const void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+ I2CSlaveClass *k = I2C_SLAVE_CLASS(oc);
+
+ k->event = i2c_mctp_event_cb;
+ k->send = i2c_mctp_send_cb;
+
+ device_class_set_props(dc, mctp_i2c_props);
+}
+
+static const TypeInfo i2c_mctp_info = {
+ .name = TYPE_MCTP_I2C_ENDPOINT,
+ .parent = TYPE_I2C_SLAVE,
+ .abstract = true,
+ .instance_init = i2c_mctp_instance_init,
+ .instance_size = sizeof(MCTPI2CEndpoint),
+ .class_init = i2c_mctp_class_init,
+ .class_size = sizeof(MCTPI2CEndpointClass),
+};
+
+static void register_types(void)
+{
+ type_register_static(&i2c_mctp_info);
+}
+
+type_init(register_types)
diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
index f543d944c3..d12f575c8d 100644
--- a/hw/arm/Kconfig
+++ b/hw/arm/Kconfig
@@ -530,6 +530,7 @@ config ASPEED_SOC
select DS1338
select FTGMAC100
select I2C
+ select I2C_MCTP
select DPS310
select PCA9552
select SERIAL_MM
diff --git a/hw/i2c/Kconfig b/hw/i2c/Kconfig
index 596a7a3165..794cccc053 100644
--- a/hw/i2c/Kconfig
+++ b/hw/i2c/Kconfig
@@ -6,6 +6,10 @@ config I2C_DEVICES
# to any board's i2c bus
bool
+config I2C_MCTP
+ bool
+ select I2C
+
config SMBUS
bool
select I2C
diff --git a/hw/i2c/meson.build b/hw/i2c/meson.build
index c459adcb59..a86457e610 100644
--- a/hw/i2c/meson.build
+++ b/hw/i2c/meson.build
@@ -1,5 +1,6 @@
i2c_ss = ss.source_set()
i2c_ss.add(when: 'CONFIG_I2C', if_true: files('core.c'))
+i2c_ss.add(when: 'CONFIG_I2C_MCTP', if_true: files('mctp.c'))
i2c_ss.add(when: 'CONFIG_SMBUS', if_true: files('smbus_slave.c', 'smbus_master.c'))
i2c_ss.add(when: 'CONFIG_ACPI_SMBUS', if_true: files('pm_smbus.c'))
i2c_ss.add(when: 'CONFIG_ACPI_ICH9', if_true: files('smbus_ich9.c'))
diff --git a/hw/i2c/trace-events b/hw/i2c/trace-events
index 1ad0e95c0e..da78fa327f 100644
--- a/hw/i2c/trace-events
+++ b/hw/i2c/trace-events
@@ -61,3 +61,17 @@ pca954x_read_data(uint8_t value) "PCA954X read data: 0x%02x"
imx_i2c_read(const char *id, const char *reg, uint64_t ofs, uint64_t value) "%s:[%s (0x%" PRIx64 ")] -> 0x%02" PRIx64
imx_i2c_write(const char *id, const char *reg, uint64_t ofs, uint64_t value) "%s:[%s (0x%" PRIx64 ")] <- 0x%02" PRIx64
+
+# mctp.c
+
+i2c_mctp_tx_start_send(size_t len) "len %zu"
+i2c_mctp_tx_send_byte(size_t pos, uint8_t byte) "pos %zu byte 0x%"PRIx8""
+i2c_mctp_tx_done(void) "packet sent"
+i2c_mctp_handle_control(uint8_t command) "command 0x%"PRIx8""
+i2c_mctp_unhandled_control(uint8_t command) "command 0x%"PRIx8""
+i2c_mctp_drop_invalid_length(unsigned byte_count, size_t expected) "byte_count %u expected %zu"
+i2c_mctp_drop_invalid_pec(uint8_t pec, uint8_t expected) "pec 0x%"PRIx8" expected 0x%"PRIx8""
+i2c_mctp_drop_invalid_eid(uint8_t eid, uint8_t expected) "eid 0x%"PRIx8" expected 0x%"PRIx8""
+i2c_mctp_drop_invalid_pktseq(uint8_t pktseq, uint8_t expected) "pktseq 0x%"PRIx8" expected 0x%"PRIx8""
+i2c_mctp_drop_short_packet(size_t len) "len %zu"
+i2c_mctp_drop_expected_som(void) ""
--
2.48.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [RFC PATCH qemu 3/5] hw/cxl/i2c_mctp_cxl: Initial device emulation
2025-06-09 16:33 [RFC PATCH qemu 0/5] hw/cxl/mctp/i2c/usb: MCTP for OoB control of CXL devices Jonathan Cameron
2025-06-09 16:33 ` [RFC PATCH qemu 1/5] hw/i2c: add smbus pec utility function Jonathan Cameron
2025-06-09 16:33 ` [RFC PATCH qemu 2/5] hw/i2c: add mctp core Jonathan Cameron
@ 2025-06-09 16:33 ` Jonathan Cameron
2025-06-10 16:39 ` Jonathan Cameron
2025-06-09 16:33 ` [RFC PATCH qemu 4/5] docs: cxl: Add example commandline for MCTP CXL CCIs Jonathan Cameron
` (2 subsequent siblings)
5 siblings, 1 reply; 8+ messages in thread
From: Jonathan Cameron @ 2025-06-09 16:33 UTC (permalink / raw)
To: Klaus Jensen, cminyard, Fan Ni, Anisa Su, qemu-devel, linux-cxl,
mst
Cc: linuxarm, Philippe Mathieu-Daudé
The CCI and Fabric Manager APIs are used to configure CXL switches and
devices. DMTF has defined an MCTP binding specification to carry these
messages. The end goal of this work is to hook this up to emulated CXL
switches and devices to allow control of the configuration.
Since this relies on i2c target mode, this can currently only be used with
an SoC that includes the Aspeed I2C controller.
Note, only get timestamp added for now.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/cxl/cxl_device.h | 8 +
include/hw/pci-bridge/cxl_upstream_port.h | 1 +
hw/cxl/cxl-mailbox-utils.c | 49 ++++
hw/cxl/i2c_mctp_cxl.c | 289 ++++++++++++++++++++++
hw/cxl/Kconfig | 4 +
hw/cxl/meson.build | 4 +
6 files changed, 355 insertions(+)
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 6086d4c85b..8d87c7151e 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -360,6 +360,10 @@ int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
size_t len_in, uint8_t *pl_in,
size_t *len_out, uint8_t *pl_out,
bool *bg_started);
+
+void cxl_initialize_t3_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
+ size_t payload_max);
+
void cxl_initialize_t3_fm_owned_ld_mctpcci(CXLCCI *cci, DeviceState *d,
DeviceState *intf,
size_t payload_max);
@@ -367,6 +371,9 @@ void cxl_initialize_t3_fm_owned_ld_mctpcci(CXLCCI *cci, DeviceState *d,
void cxl_initialize_t3_ld_cci(CXLCCI *cci, DeviceState *d,
DeviceState *intf, size_t payload_max);
+void cxl_initialize_usp_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
+ size_t payload_max);
+
#define cxl_device_cap_init(dstate, reg, cap_id, ver) \
do { \
uint32_t *cap_hdrs = dstate->caps_reg_state32; \
@@ -606,6 +613,7 @@ struct CXLType3Dev {
CXLComponentState cxl_cstate;
CXLDeviceState cxl_dstate;
CXLCCI cci; /* Primary PCI mailbox CCI */
+ CXLCCI oob_mctp_cci; /* Initialized only if targetted */
/* Always initialized as no way to know if a VDM might show up */
CXLCCI vdm_fm_owned_ld_mctp_cci;
CXLCCI ld0_cci;
diff --git a/include/hw/pci-bridge/cxl_upstream_port.h b/include/hw/pci-bridge/cxl_upstream_port.h
index f208397ffe..7e529e0b5a 100644
--- a/include/hw/pci-bridge/cxl_upstream_port.h
+++ b/include/hw/pci-bridge/cxl_upstream_port.h
@@ -12,6 +12,7 @@ typedef struct CXLUpstreamPort {
/*< public >*/
CXLComponentState cxl_cstate;
CXLCCI swcci;
+ CXLCCI mctpcci;
PCIExpLinkSpeed speed;
PCIExpLinkWidth width;
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 475547f212..4c9852642e 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -3690,6 +3690,29 @@ void cxl_initialize_mailbox_t3(CXLCCI *cci, DeviceState *d, size_t payload_max)
cxl_init_cci(cci, payload_max);
}
+static const struct cxl_cmd cxl_cmd_set_t3_mctp[256][256] = {
+ [INFOSTAT][IS_IDENTIFY] = { "IDENTIFY", cmd_infostat_identify, 0, 0 },
+ [INFOSTAT][GET_RESPONSE_MSG_LIMIT] = { "GET_RESPONSE_MSG_LIMIT",
+ cmd_get_response_msg_limit, 0, 0 },
+ [INFOSTAT][SET_RESPONSE_MSG_LIMIT] = { "SET_RESPONSE_MSG_LIMIT",
+ cmd_set_response_msg_limit, 1, 0 },
+ [TIMESTAMP][GET] = { "TIMESTAMP_GET", cmd_timestamp_get, 0, 0 },
+ [LOGS][GET_SUPPORTED] = { "LOGS_GET_SUPPORTED", cmd_logs_get_supported, 0,
+ 0 },
+ [LOGS][GET_LOG] = { "LOGS_GET_LOG", cmd_logs_get_log, 0x18, 0 },
+ [TUNNEL][MANAGEMENT_COMMAND] = { "TUNNEL_MANAGEMENT_COMMAND",
+ cmd_tunnel_management_cmd, ~0, 0 },
+};
+
+void cxl_initialize_t3_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
+ size_t payload_max)
+{
+ cxl_copy_cci_commands(cci, cxl_cmd_set_t3_mctp);
+ cci->d = d;
+ cci->intf = intf;
+ cxl_init_cci(cci, payload_max);
+}
+
static const struct cxl_cmd cxl_cmd_set_t3_ld[256][256] = {
[INFOSTAT][IS_IDENTIFY] = { "IDENTIFY", cmd_infostat_identify, 0, 0 },
[LOGS][GET_SUPPORTED] = { "LOGS_GET_SUPPORTED", cmd_logs_get_supported, 0,
@@ -3729,3 +3752,29 @@ void cxl_initialize_t3_fm_owned_ld_mctpcci(CXLCCI *cci, DeviceState *d,
cci->intf = intf;
cxl_init_cci(cci, payload_max);
}
+
+static const struct cxl_cmd cxl_cmd_set_usp_mctp[256][256] = {
+ [INFOSTAT][IS_IDENTIFY] = { "IDENTIFY", cmd_infostat_identify, 0, 0 },
+ [INFOSTAT][GET_RESPONSE_MSG_LIMIT] = { "GET_RESPONSE_MSG_LIMIT",
+ cmd_get_response_msg_limit, 0, 0 },
+ [INFOSTAT][SET_RESPONSE_MSG_LIMIT] = { "SET_RESPONSE_MSG_LIMIT",
+ cmd_set_response_msg_limit, 1, 0 },
+ [LOGS][GET_SUPPORTED] = { "LOGS_GET_SUPPORTED", cmd_logs_get_supported,
+ 0, 0 },
+ [LOGS][GET_LOG] = { "LOGS_GET_LOG", cmd_logs_get_log, 0x18, 0 },
+ [PHYSICAL_SWITCH][IDENTIFY_SWITCH_DEVICE] = { "IDENTIFY_SWITCH_DEVICE",
+ cmd_identify_switch_device, 0, 0 },
+ [PHYSICAL_SWITCH][GET_PHYSICAL_PORT_STATE] = { "SWITCH_PHYSICAL_PORT_STATS",
+ cmd_get_physical_port_state, ~0, 0 },
+ [TUNNEL][MANAGEMENT_COMMAND] = { "TUNNEL_MANAGEMENT_COMMAND",
+ cmd_tunnel_management_cmd, ~0, 0 },
+};
+
+void cxl_initialize_usp_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
+ size_t payload_max)
+{
+ cxl_copy_cci_commands(cci, cxl_cmd_set_usp_mctp);
+ cci->d = d;
+ cci->intf = intf;
+ cxl_init_cci(cci, payload_max);
+}
diff --git a/hw/cxl/i2c_mctp_cxl.c b/hw/cxl/i2c_mctp_cxl.c
new file mode 100644
index 0000000000..1714f36e8e
--- /dev/null
+++ b/hw/cxl/i2c_mctp_cxl.c
@@ -0,0 +1,289 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Emulation of a CXL Switch Fabric Management interface over MCTP over I2C.
+ *
+ * Copyright (c) 2023 Huawei Technologies.
+ *
+ * Reference list:
+ * From www.dmtf.org
+ * DSP0236 Management Component Transport Protocol (MCTP) Base Specification
+ * 1.3.0
+ * DPS0234 CXL Fabric Manager API over MCTP Binding Specification 1.0.0
+ * DSP0281 CXL Type 3 Device Component Command Interface over MCTP Binding
+ * Specification (note some commands apply to switches as well)
+ * From www.computeexpresslink.org
+ * Compute Express Link (CXL) Specification revision 3.0 Version 1.0
+ */
+
+#include "qemu/osdep.h"
+#include "hw/i2c/i2c.h"
+#include "hw/i2c/mctp.h"
+#include "net/mctp.h"
+#include "hw/irq.h"
+#include "migration/vmstate.h"
+#include "qapi/error.h"
+#include "hw/cxl/cxl.h"
+#include "hw/pci-bridge/cxl_upstream_port.h"
+#include "hw/pci/pcie.h"
+#include "hw/pci/pcie_port.h"
+#include "hw/qdev-properties.h"
+#include "hw/registerfields.h"
+
+#define TYPE_I2C_MCTP_CXL "i2c_mctp_cxl"
+
+/* DMTF DSP0234 CXL Fabric Manager API over MCTP Binding Specification */
+#define MCTP_MT_CXL_FMAPI 0x7
+/*
+ * DMTF DSP0281 CXL Type 3 Deivce Component Command Interface over MCTP
+ * Binding Specification
+ */
+#define MCTP_MT_CXL_TYPE3 0x8
+
+/* FMAPI binding specification defined */
+#define MCTP_CXL_MAX_MSG_LEN 1088
+
+/* Implementation choice - may make this configurable */
+#define MCTP_CXL_MAILBOX_BYTES 512
+
+typedef struct CXLMCTPMessage {
+ /*
+ * DSP0236 (MCTP Base) Integrity Check + Message Type
+ * DSP0234/DSP0281 (CXL bindings) state no Integrity Check
+ * so just the message type.
+ */
+ uint8_t message_type;
+ /* Remaing fields from CXL r3.0 Table 7-14 CCI Message Format */
+ uint8_t category;
+ uint8_t tag;
+ uint8_t rsvd;
+ /*
+ * CXL r3.0 - Table 8-36 Generic Component Command Opcodes:
+ * Command opcode is split into two sub fields
+ */
+ uint8_t command;
+ uint8_t command_set;
+ uint8_t pl_length[3];
+ uint16_t rc;
+ uint16_t vendor_status;
+ uint8_t payload[];
+} QEMU_PACKED CXLMCTPMessage;
+
+enum cxl_dev_type {
+ cxl_type3,
+ cxl_switch,
+};
+
+struct I2C_MCTP_CXL_State {
+ MCTPI2CEndpoint mctp;
+ PCIDevice *target;
+ CXLCCI *cci;
+ enum cxl_dev_type type;
+ size_t len;
+ int64_t pos;
+ uint8_t buffer[MCTP_CXL_MAX_MSG_LEN];
+ uint8_t scratch[MCTP_CXL_MAX_MSG_LEN];
+};
+
+OBJECT_DECLARE_SIMPLE_TYPE(I2C_MCTP_CXL_State, I2C_MCTP_CXL)
+
+static const Property i2c_mctp_cxl_props[] = {
+ DEFINE_PROP_LINK("target", I2C_MCTP_CXL_State,
+ target, TYPE_PCI_DEVICE, PCIDevice *),
+};
+
+static size_t i2c_mctp_cxl_get_buf(MCTPI2CEndpoint *mctp,
+ const uint8_t **buf,
+ size_t maxlen,
+ uint8_t *mctp_flags)
+{
+ I2C_MCTP_CXL_State *s = I2C_MCTP_CXL(mctp);
+ size_t len;
+
+ len = MIN(maxlen, s->len - s->pos);
+
+ if (len == 0) {
+ return 0;
+ }
+
+ if (s->pos == 0) {
+ *mctp_flags = FIELD_DP8(*mctp_flags, MCTP_H_FLAGS, SOM, 1);
+ }
+
+ *buf = s->scratch + s->pos;
+ s->pos += len;
+
+ if (s->pos == s->len) {
+ *mctp_flags = FIELD_DP8(*mctp_flags, MCTP_H_FLAGS, EOM, 1);
+
+ s->pos = s->len = 0;
+ }
+
+ return len;
+}
+
+static int i2c_mctp_cxl_put_buf(MCTPI2CEndpoint *mctp,
+ uint8_t *buf, size_t len)
+{
+ I2C_MCTP_CXL_State *s = I2C_MCTP_CXL(mctp);
+
+ if (s->len + len > MCTP_CXL_MAX_MSG_LEN) {
+ return -1;
+ }
+
+ memcpy(s->buffer + s->len, buf, len);
+ s->len += len;
+
+ return 0;
+}
+
+static size_t i2c_mctp_cxl_get_types(MCTPI2CEndpoint *mctp,
+ const uint8_t **data)
+{
+ static const uint8_t buf[] = {
+ 0x0, /* Success */
+ 2, /* Message types in list - supported in addition to control */
+ MCTP_MT_CXL_FMAPI,
+ MCTP_MT_CXL_TYPE3,
+ };
+ *data = buf;
+
+ return sizeof(buf);
+}
+
+static void i2c_mctp_cxl_reset_message(MCTPI2CEndpoint *mctp)
+{
+ I2C_MCTP_CXL_State *s = I2C_MCTP_CXL(mctp);
+
+ s->len = 0;
+}
+
+static void i2c_mctp_cxl_handle_message(MCTPI2CEndpoint *mctp)
+{
+ I2C_MCTP_CXL_State *s = I2C_MCTP_CXL(mctp);
+ CXLMCTPMessage *msg = (CXLMCTPMessage *)s->buffer;
+ CXLMCTPMessage *buf = (CXLMCTPMessage *)s->scratch;
+
+ *buf = (CXLMCTPMessage) {
+ .message_type = msg->message_type,
+ .category = 1,
+ .tag = msg->tag,
+ .command = msg->command,
+ .command_set = msg->command_set,
+ };
+ s->pos = sizeof(*buf);
+ if (s->cci) {
+ bool bg_started;
+ size_t len_out = 0;
+ size_t len_in;
+ int rc;
+
+ /*
+ * As it was not immediately obvious from the various specifications,
+ * clarification was sort for which binding applies for which command
+ * set. The outcome was:
+ *
+ * Any command forming part of the CXL FM-API command set
+ * e.g. Present in CXL r3.0 Table 8-132: CXL FM API Command Opcodes
+ * (and equivalent in later CXL specifications) is valid only with
+ * the CXL Fabric Manager API over MCTP binding (DSP0234).
+ *
+ * Any other CXL command currently should be sent using the
+ * CXL Type 3 Device Component Command interface over MCTP binding,
+ * even if it is being sent to a switch.
+ *
+ * If tunneling is used, the component creating the PCIe VDMs must
+ * use the appropriate binding for sending the tunnel contents
+ * onwards.
+ */
+
+ if (!(msg->message_type == MCTP_MT_CXL_TYPE3 &&
+ msg->command_set < 0x51) &&
+ !(msg->message_type == MCTP_MT_CXL_FMAPI &&
+ msg->command_set >= 0x51 && msg->command_set < 0x56)) {
+ buf->rc = CXL_MBOX_UNSUPPORTED;
+ st24_le_p(buf->pl_length, len_out);
+ s->len = s->pos;
+ s->pos = 0;
+ i2c_mctp_schedule_send(mctp);
+ return;
+ }
+
+ len_in = msg->pl_length[2] << 16 | msg->pl_length[1] << 8 |
+ msg->pl_length[0];
+
+ rc = cxl_process_cci_message(s->cci, msg->command_set, msg->command,
+ len_in, msg->payload,
+ &len_out,
+ s->scratch + sizeof(CXLMCTPMessage),
+ &bg_started);
+ buf->rc = rc;
+ s->pos += len_out;
+ s->len = s->pos;
+ st24_le_p(buf->pl_length, len_out);
+ s->pos = 0;
+ i2c_mctp_schedule_send(mctp);
+ } else {
+ g_assert_not_reached(); /* The cci must be hooked up */
+ }
+}
+
+static void i2c_mctp_cxl_realize(DeviceState *d, Error **errp)
+{
+ I2C_MCTP_CXL_State *s = I2C_MCTP_CXL(d);
+
+ /* Check this is a type we support */
+ if (object_dynamic_cast(OBJECT(s->target), TYPE_CXL_USP)) {
+ CXLUpstreamPort *usp = CXL_USP(s->target);
+
+ s->type = cxl_switch;
+ s->cci = &usp->mctpcci;
+
+ cxl_initialize_usp_mctpcci(s->cci, DEVICE(s->target), d,
+ MCTP_CXL_MAILBOX_BYTES);
+
+ return;
+ }
+
+ if (object_dynamic_cast(OBJECT(s->target), TYPE_CXL_TYPE3)) {
+ CXLType3Dev *ct3d = CXL_TYPE3(s->target);
+
+ s->type = cxl_type3;
+ s->cci = &ct3d->oob_mctp_cci;
+
+ cxl_initialize_t3_fm_owned_ld_mctpcci(s->cci, DEVICE(s->target), d,
+ MCTP_CXL_MAILBOX_BYTES);
+ return;
+ }
+
+ error_setg(errp, "Unhandled target type for CXL MCTP EP");
+}
+
+static void i2c_mctp_cxl_class_init(ObjectClass *klass, const void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(klass);
+ MCTPI2CEndpointClass *mc = MCTP_I2C_ENDPOINT_CLASS(klass);
+
+ dc->realize = i2c_mctp_cxl_realize;
+ mc->get_types = i2c_mctp_cxl_get_types;
+ mc->get_buf = i2c_mctp_cxl_get_buf;
+ mc->put_buf = i2c_mctp_cxl_put_buf;
+
+ mc->handle = i2c_mctp_cxl_handle_message;
+ mc->reset = i2c_mctp_cxl_reset_message;
+ device_class_set_props(dc, i2c_mctp_cxl_props);
+}
+
+static const TypeInfo i2c_mctp_cxl_info = {
+ .name = TYPE_I2C_MCTP_CXL,
+ .parent = TYPE_MCTP_I2C_ENDPOINT,
+ .instance_size = sizeof(I2C_MCTP_CXL_State),
+ .class_init = i2c_mctp_cxl_class_init,
+};
+
+static void i2c_mctp_cxl_register_types(void)
+{
+ type_register_static(&i2c_mctp_cxl_info);
+}
+
+type_init(i2c_mctp_cxl_register_types)
diff --git a/hw/cxl/Kconfig b/hw/cxl/Kconfig
index 8e67519b16..bc259fdf67 100644
--- a/hw/cxl/Kconfig
+++ b/hw/cxl/Kconfig
@@ -1,3 +1,7 @@
config CXL
bool
default y if PCI_EXPRESS
+
+config I2C_MCTP_CXL
+ bool
+ default y if CXL && I2C_MCTP
diff --git a/hw/cxl/meson.build b/hw/cxl/meson.build
index e3abb49d27..90fd83f680 100644
--- a/hw/cxl/meson.build
+++ b/hw/cxl/meson.build
@@ -12,3 +12,7 @@ system_ss.add(when: 'CONFIG_CXL',
if_false: files(
'cxl-host-stubs.c',
))
+system_ss.add(when: 'CONFIG_I2C_MCTP_CXL', if_true: files('i2c_mctp_cxl.c'))
+
+system_ss.add(when: 'CONFIG_ALL', if_true: files('cxl-host-stubs.c'))
+
--
2.48.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [RFC PATCH qemu 4/5] docs: cxl: Add example commandline for MCTP CXL CCIs
2025-06-09 16:33 [RFC PATCH qemu 0/5] hw/cxl/mctp/i2c/usb: MCTP for OoB control of CXL devices Jonathan Cameron
` (2 preceding siblings ...)
2025-06-09 16:33 ` [RFC PATCH qemu 3/5] hw/cxl/i2c_mctp_cxl: Initial device emulation Jonathan Cameron
@ 2025-06-09 16:33 ` Jonathan Cameron
2025-06-09 16:33 ` [RFC PATCH qemu 5/5] usb/mctp/cxl: CXL FMAPI interface via MCTP over usb Jonathan Cameron
2025-06-24 19:47 ` [RFC PATCH qemu 0/5] hw/cxl/mctp/i2c/usb: MCTP for OoB control of CXL devices Anisa Su
5 siblings, 0 replies; 8+ messages in thread
From: Jonathan Cameron @ 2025-06-09 16:33 UTC (permalink / raw)
To: Klaus Jensen, cminyard, Fan Ni, Anisa Su, qemu-devel, linux-cxl,
mst
Cc: linuxarm, Philippe Mathieu-Daudé
Add initial documentation for the MCTP over I2C management device. At
current time this can only be used with the Aspeed I2C controller which
is only available in aspeed SoCs, though can be added to other
emulated boards.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
docs/system/devices/cxl.rst | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/docs/system/devices/cxl.rst b/docs/system/devices/cxl.rst
index e307caf3f8..523f99f205 100644
--- a/docs/system/devices/cxl.rst
+++ b/docs/system/devices/cxl.rst
@@ -406,6 +406,33 @@ OS management of CXL memory devices as described here.
* CONFIG_CXL_PORT
* CONFIG_CXL_REGION
+
+CCI access via MCTP over I2C
+----------------------------
+
+In order to make use of this device, an I2C controller that supports MCTP
+is required. The aspeed-i2c controller is an example of such a controller.
+
+Both CXL switches and CXL Type 3 devices support configuration via
+MCTP access to Component Command Interfaces (CCIs) on the devices.
+
+Example configuration:
+
+ -device cxl-upstream,port=33,bus=root_port0,id=us0,multifunction=on,addr=0.0,sn=12345678 \
+ -device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \
+ -device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \
+ -device cxl-downstream,port=2,bus=us0,id=swport2,chassis=0,slot=6 \
+ -device cxl-type3,bus=swport0,persistent-memdev=cxl-mem1,id=cxl-pmem0,lsa=cxl-lsa1,sn=3 \
+ -device cxl-type3,bus=swport1,persistent-memdev=cxl-mem2,id=cxl-pmem1,lsa=cxl-lsa2,sn=4 \
+ -device cxl-type3,bus=swport2,persistent-memdev=cxl-mem3,id=cxl-pmem2,lsa=cxl-lsa3,sn=5 \
+ -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=4,target=us0 \
+ -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=5,target=cxl-pmem0 \
+ -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=6,target=cxl-pmem1 \
+ -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=7,target=cxl-pmem2
+
+Guest OS communication with the MCTP CCI can then be established using standard
+MCTP configuration tools.
+
References
----------
--
2.48.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [RFC PATCH qemu 5/5] usb/mctp/cxl: CXL FMAPI interface via MCTP over usb.
2025-06-09 16:33 [RFC PATCH qemu 0/5] hw/cxl/mctp/i2c/usb: MCTP for OoB control of CXL devices Jonathan Cameron
` (3 preceding siblings ...)
2025-06-09 16:33 ` [RFC PATCH qemu 4/5] docs: cxl: Add example commandline for MCTP CXL CCIs Jonathan Cameron
@ 2025-06-09 16:33 ` Jonathan Cameron
2025-06-24 19:47 ` [RFC PATCH qemu 0/5] hw/cxl/mctp/i2c/usb: MCTP for OoB control of CXL devices Anisa Su
5 siblings, 0 replies; 8+ messages in thread
From: Jonathan Cameron @ 2025-06-09 16:33 UTC (permalink / raw)
To: Klaus Jensen, cminyard, Fan Ni, Anisa Su, qemu-devel, linux-cxl,
mst
Cc: linuxarm, Philippe Mathieu-Daudé
Given the challenges in testing systems with the previously
posted i2c based mctp support for x86 / arm64 machines (as only
emulated controller that is suitably capable is the aspeed one)
providing a USB interface greatly simplifies things.
This is a PoC of such an interface.
RFC reasons:
- Not all error conditions are fully checked / handled.
- MTU in the 'to host' direction is ignored. Seems the Linux
stack doesn't care about this today but that should be brought
into compliance with the specification anyway.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/usb.h | 1 +
include/net/mctp.h | 77 +++++-
hw/i2c/mctp.c | 17 --
hw/usb/dev-mctp.c | 639 +++++++++++++++++++++++++++++++++++++++++++++
hw/usb/Kconfig | 5 +
hw/usb/meson.build | 1 +
6 files changed, 717 insertions(+), 23 deletions(-)
diff --git a/include/hw/usb.h b/include/hw/usb.h
index 26a9f3ecde..ae5f3f77cb 100644
--- a/include/hw/usb.h
+++ b/include/hw/usb.h
@@ -81,6 +81,7 @@
#define USB_CLASS_CDC_DATA 0x0a
#define USB_CLASS_CSCID 0x0b
#define USB_CLASS_CONTENT_SEC 0x0d
+#define USB_CLASS_MCTP 0x14
#define USB_CLASS_APP_SPEC 0xfe
#define USB_CLASS_VENDOR_SPEC 0xff
diff --git a/include/net/mctp.h b/include/net/mctp.h
index 5d26d855db..f278b17ce2 100644
--- a/include/net/mctp.h
+++ b/include/net/mctp.h
@@ -3,21 +3,21 @@
#include "hw/registerfields.h"
-/* DSP0236 1.3.0, Section 8.3.1 */
+/* DSP0236 1.3.3, Section 8.4.2 Baseline transmission unit */
#define MCTP_BASELINE_MTU 64
-/* DSP0236 1.3.0, Table 1, Message body */
+/* DSP0236 1.3.3, Table 1, Message body */
FIELD(MCTP_MESSAGE_H, TYPE, 0, 7)
FIELD(MCTP_MESSAGE_H, IC, 7, 1)
-/* DSP0236 1.3.0, Table 1, MCTP transport header */
+/* DSP0236 1.3.3, Table 1, MCTP transport header */
FIELD(MCTP_H_FLAGS, TAG, 0, 3);
FIELD(MCTP_H_FLAGS, TO, 3, 1);
FIELD(MCTP_H_FLAGS, PKTSEQ, 4, 2);
FIELD(MCTP_H_FLAGS, EOM, 6, 1);
FIELD(MCTP_H_FLAGS, SOM, 7, 1);
-/* DSP0236 1.3.0, Figure 4 */
+/* DSP0236 1.3.3, Figure 4 Generic message fields */
typedef struct MCTPPacketHeader {
uint8_t version;
struct {
@@ -25,11 +25,76 @@ typedef struct MCTPPacketHeader {
uint8_t source;
} eid;
uint8_t flags;
-} MCTPPacketHeader;
+} QEMU_PACKED MCTPPacketHeader;
typedef struct MCTPPacket {
MCTPPacketHeader hdr;
uint8_t payload[];
-} MCTPPacket;
+} QEMU_PACKED MCTPPacket;
+
+/* DSP0236 1.3.3, Figure 20 MCTP control message format */
+typedef struct MCTPControlMessage {
+#define MCTP_MESSAGE_TYPE_CONTROL 0x0
+ uint8_t type;
+#define MCTP_CONTROL_FLAGS_RQ (1 << 7)
+#define MCTP_CONTROL_FLAGS_D (1 << 6)
+ uint8_t flags;
+ uint8_t command_code;
+ uint8_t data[];
+} QEMU_PACKED MCTPControlMessage;
+
+enum MCTPControlCommandCodes {
+ MCTP_CONTROL_SET_EID = 0x01,
+ MCTP_CONTROL_GET_EID = 0x02,
+ MCTP_CONTROL_GET_UUID = 0x03,
+ MCTP_CONTROL_GET_VERSION = 0x04,
+ MCTP_CONTROL_GET_MESSAGE_TYPE_SUPPORT = 0x05,
+};
+
+/* DSP0236 1.3.3, Table 13 MCTP control message completion codes */
+#define MCTP_CONTROL_CC_SUCCESS 0x0
+#define MCTP_CONTROL_CC_ERROR 0x1
+#define MCTP_CONTROL_CC_ERROR_INVALID_DATA 0x2
+#define MCTP_CONTROL_CC_ERROR_INVALID_LENGTH 0x3
+#define MCTP_CONTROL_CC_ERROR_NOT_READY 0x4
+#define MCTP_CONTROL_CC_ERROR_UNSUP_COMMAND 0x5
+
+typedef struct MCTPControlErrRsp {
+ uint8_t completion_code;
+} MCTPControlErrRsp;
+
+/* DSP0236 1.3.3 Table 14 - Set Endpoint ID message */
+typedef struct MCTPControlSetEIDReq {
+ uint8_t operation;
+ uint8_t eid;
+} MCTPControlSetEIDReq;
+
+typedef struct MCTPControlSetEIDRsp {
+ uint8_t completion_code;
+ uint8_t operation_result; /* Not named in spec */
+ uint8_t eid_setting;
+ uint8_t eid_pool_size;
+} MCTPControlSetEIDRsp;
+
+/* DSP0236 1.3.3 Table 15 - Get Endpoint ID message */
+typedef struct MCTPControlGetEIDRsp {
+ uint8_t completion_code;
+ uint8_t endpoint_id;
+ uint8_t endpoint_type;
+ uint8_t medium_specific_info;
+} MCTPControlGetEIDRsp;
+
+/* DSP0236 1.3.3 Table 16 - Get Endpoint UUID message format */
+typedef struct MCTPControlGetUUIDRsp {
+ uint8_t completion_code;
+ uint8_t uuid[0x10];
+} MCTPControlGetUUIDRsp;
+
+/* DSP0236 1.3.3 Table 19 - Get Message Type Support message */
+typedef struct MCTPControlGetMessageTypeRsp {
+ uint8_t completion_code;
+ uint8_t message_type_count;
+ uint8_t types[];
+} MCTPControlGetMessageTypeRsp;
#endif /* QEMU_MCTP_H */
diff --git a/hw/i2c/mctp.c b/hw/i2c/mctp.c
index cf45a46706..ac9bebb3c4 100644
--- a/hw/i2c/mctp.c
+++ b/hw/i2c/mctp.c
@@ -33,23 +33,6 @@ typedef struct MCTPI2CPacket {
#define i2c_mctp_payload_offset offsetof(MCTPI2CPacket, mctp.payload)
#define i2c_mctp_payload(buf) (buf + i2c_mctp_payload_offset)
-/* DSP0236 1.3.0, Figure 20 */
-typedef struct MCTPControlMessage {
-#define MCTP_MESSAGE_TYPE_CONTROL 0x0
- uint8_t type;
-#define MCTP_CONTROL_FLAGS_RQ (1 << 7)
-#define MCTP_CONTROL_FLAGS_D (1 << 6)
- uint8_t flags;
- uint8_t command_code;
- uint8_t data[];
-} MCTPControlMessage;
-
-enum MCTPControlCommandCodes {
- MCTP_CONTROL_SET_EID = 0x01,
- MCTP_CONTROL_GET_EID = 0x02,
- MCTP_CONTROL_GET_VERSION = 0x04,
- MCTP_CONTROL_GET_MESSAGE_TYPE_SUPPORT = 0x05,
-};
#define MCTP_CONTROL_ERROR_UNSUPPORTED_CMD 0x5
diff --git a/hw/usb/dev-mctp.c b/hw/usb/dev-mctp.c
new file mode 100644
index 0000000000..aafb9e7e96
--- /dev/null
+++ b/hw/usb/dev-mctp.c
@@ -0,0 +1,639 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright Huawei 2025
+ *
+ * Author: Jonathan Cameron <jonathan.cameron@huawei.com>
+ *
+ * CXL MCTP device
+ *
+ * TODO:
+ * - Respect MTU on packets being sent to host. For now it work in Linux but
+ * who knows longer term.
+ * - More complete sanity checking of command flags etc.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+
+#include "hw/pci/pci_device.h"
+#include "hw/qdev-properties.h"
+#include "hw/cxl/cxl.h"
+#include "hw/pci-bridge/cxl_upstream_port.h"
+#include "hw/pci/pcie.h"
+#include "hw/pci/pcie_port.h"
+#include "hw/usb.h"
+#include "hw/usb/desc.h"
+#include "net/mctp.h"
+
+/* TODO: Move to header */
+
+/* DMTF DSP0234 CXL Fabric Manager API over MCTP Binding Specification */
+#define MCTP_MT_CXL_FMAPI 0x7
+/*
+ * DMTF DSP0281 CXL Type 3 Device Component Command Interface over MCTP
+ * Binding Specification
+ */
+#define MCTP_MT_CXL_TYPE3 0x8
+typedef struct CXLMCTPMessage {
+ /*
+ * DSP0236 (MCTP Base) Integrity Check + Message Type
+ * DSP0234/DSP0281 (CXL bindings) state no Integrity Check
+ * so just the message type.
+ */
+ uint8_t message_type;
+ /* Remaining fields from CXL r3.0 Table 7-14 CCI Message Format */
+ uint8_t category;
+ uint8_t tag;
+ uint8_t rsvd;
+ /*
+ * CXL r3.0 - Table 8-36 Generic Component Command Opcodes:
+ * Command opcode is split into two sub fields
+ */
+ uint8_t command;
+ uint8_t command_set;
+ /* Only bits 4:0 of pl_length[2] are part of the length */
+ uint8_t pl_length[3];
+ uint16_t rc;
+ uint16_t vendor_status;
+ uint8_t payload[];
+} QEMU_PACKED CXLMCTPMessage;
+
+/* DSP0283 1.0.0 Figure 5 */
+typedef struct MCTPUSBPacket {
+ uint16_t dmtf_id;
+ uint8_t resv;
+ uint8_t length;
+ /* Do we see the header? */
+ MCTPPacket mctp;
+} MCTPUSBPacket;
+
+#define MCTP_CXL_MAILBOX_BYTES 128
+
+enum cxl_dev_type {
+ cxl_type3,
+ cxl_switch,
+};
+
+typedef struct USBCXLMCTPState {
+ USBDevice dev;
+ PCIDevice *target;
+ CXLCCI *cci;
+ enum cxl_dev_type type;
+ USBPacket *cached_tohost;
+ USBPacket *cached_fromhost;
+ uint8_t my_eid;
+ bool building_input;
+/* TODO, match size to what we report as acceptable */
+#define MCTPUSBCXL_MAX_SIZE 1024 * 1024
+ MCTPUSBPacket *pack0; /* Current incoming packet */
+ MCTPUSBPacket *pack; /* Aggregate packet - what we'd get with large MTU */
+} USBCLXMCTState;
+
+#define TYPE_USB_CXL_MCTP "usb-cxl-mctp"
+OBJECT_DECLARE_SIMPLE_TYPE(USBCXLMCTPState, USB_CXL_MCTP)
+
+enum {
+ STR_MANUFACTURER = 1,
+ STR_PRODUCT,
+ STR_SERIALNUMBER,
+ STR_MCTP,
+ STR_CONFIG_FULL,
+ STR_CONFIG_HIGH,
+ STR_CONFIG_SUPER,
+};
+
+static const USBDescStrings desc_strings = {
+ [STR_MANUFACTURER] = "QEMU",
+ [STR_PRODUCT] = "QEMU CXL MCTP",
+ [STR_SERIALNUMBER] = "34618",
+ [STR_MCTP] = "MCTP",
+ [STR_CONFIG_FULL] = "Full speed config (usb 1.1)",
+ [STR_CONFIG_HIGH] = "High speed config (usb 2.0)",
+ [STR_CONFIG_SUPER] = "Super speed config (usb 3.0)",
+};
+
+static const USBDescIface desc_iface_full = {
+ .bInterfaceNumber = 0,
+ .bNumEndpoints = 2,
+ .bInterfaceClass = USB_CLASS_MCTP,
+ .bInterfaceSubClass = 0x0,
+ .bInterfaceProtocol = 0x1,
+ .iInterface = STR_MCTP,
+ .eps = (USBDescEndpoint[]) {
+ {
+ /* DSP0283 6.1.4.2.1 Out Bulk endpoint descriptor */
+ .bEndpointAddress = USB_DIR_OUT | 0x1,
+ .bmAttributes = USB_ENDPOINT_XFER_BULK,
+ .wMaxPacketSize = 512,
+ .bInterval = 0x1,
+ }, {
+ /* DSP0283 6.1.4.2.2 In Bulk endpoint descriptor */
+ .bEndpointAddress = USB_DIR_IN | 0x1,
+ .bmAttributes = USB_ENDPOINT_XFER_BULK,
+ .wMaxPacketSize = 512,
+ .bInterval = 0x1,
+ },
+ },
+};
+
+static const USBDescDevice desc_device_full = {
+ .bcdUSB = 0x200,
+ .bMaxPacketSize0 = 8,
+ .bNumConfigurations = 1,
+ .confs = (USBDescConfig[]) {
+ {
+ .bNumInterfaces = 1,
+ .bConfigurationValue = 1,
+ .iConfiguration = STR_CONFIG_FULL,
+ .bmAttributes = USB_CFG_ATT_ONE | USB_CFG_ATT_WAKEUP,
+ .bMaxPower = 2,
+ .nif = 1,
+ .ifs = &desc_iface_full,
+ },
+ },
+};
+
+static const USBDescMSOS desc_msos = {
+ .CompatibleID = "MCTP",
+ .SelectiveSuspendEnabled = true,
+};
+
+static const USBDesc desc = {
+ .id = {
+ .idVendor = 0x46f4, /* CRC16() of "QEMU" */
+ .idProduct = 0x0006,
+ .bcdDevice = 0,
+ .iManufacturer = STR_MANUFACTURER,
+ .iSerialNumber = STR_SERIALNUMBER,
+ },
+ .full = &desc_device_full,
+ .high = &desc_device_full,
+ .str = desc_strings,
+ .msos = &desc_msos,
+};
+
+static void usb_cxl_mctp_handle_reset(USBDevice *dev)
+{
+ USBCXLMCTPState *s = USB_CXL_MCTP(dev);
+
+ s->cached_tohost = NULL;
+ s->cached_fromhost = NULL;
+ s->building_input = 0;
+ s->my_eid = 0;
+}
+
+static void usb_cxl_mctp_handle_control(USBDevice *dev, USBPacket *p,
+ int request, int value, int index,
+ int length, uint8_t *data)
+{
+ usb_desc_handle_control(dev, p, request, value, index, length, data);
+}
+
+/* A lot of fields are the same for all control responses */
+static void usb_mctp_fill_common(MCTPUSBPacket *o_usb_pkt,
+ MCTPUSBPacket *i_usb_pkt,
+ uint8_t usb_pkt_len)
+{
+ MCTPPacket *o_mctp_pkt = &o_usb_pkt->mctp;
+ uint8_t tag, flags;
+
+ o_usb_pkt->dmtf_id = cpu_to_be16(0x1AB4);
+ o_usb_pkt->length = usb_pkt_len;
+
+ o_mctp_pkt->hdr.version = 1;
+ o_mctp_pkt->hdr.eid.dest = i_usb_pkt->mctp.hdr.eid.source;
+ o_mctp_pkt->hdr.eid.source = i_usb_pkt->mctp.hdr.eid.dest;
+
+ tag = FIELD_EX8(i_usb_pkt->mctp.hdr.flags, MCTP_H_FLAGS, TAG);
+
+ flags = FIELD_DP8(0, MCTP_H_FLAGS, PKTSEQ, 0);
+ flags = FIELD_DP8(flags, MCTP_H_FLAGS, TAG, tag);
+ flags = FIELD_DP8(flags, MCTP_H_FLAGS, SOM, 1);
+ flags = FIELD_DP8(flags, MCTP_H_FLAGS, EOM, 1);
+ o_mctp_pkt->hdr.flags = flags;
+}
+
+static MCTPUSBPacket *usb_mctp_handle_control(USBCXLMCTPState *s,
+ MCTPControlMessage *ctrlmsg,
+ USBPacket *p)
+{
+ switch (ctrlmsg->command_code) {
+ case MCTP_CONTROL_SET_EID: {
+ MCTPControlSetEIDReq *req = (MCTPControlSetEIDReq *)(ctrlmsg->data);
+ uint8_t usb_pkt_len = sizeof(MCTPUSBPacket) +
+ sizeof(MCTPControlMessage) +
+ sizeof(MCTPControlSetEIDRsp);
+ MCTPUSBPacket *o_usb_pkt = g_malloc0(usb_pkt_len);
+ MCTPPacket *o_mctp_pkt = &o_usb_pkt->mctp;
+ MCTPControlMessage *o_mctp_ctrl =
+ (MCTPControlMessage *)(o_mctp_pkt->payload);
+ MCTPControlSetEIDRsp *rsp = (MCTPControlSetEIDRsp *)o_mctp_ctrl->data;
+
+ /* Todo - check flags in request */
+ s->my_eid = req->eid;
+
+ o_mctp_ctrl->type = 0;
+ o_mctp_ctrl->command_code = ctrlmsg->command_code;
+ o_mctp_ctrl->flags = ctrlmsg->flags &
+ ~(MCTP_CONTROL_FLAGS_RQ | MCTP_CONTROL_FLAGS_D);
+ *rsp = (MCTPControlSetEIDRsp) {
+ .completion_code = MCTP_CONTROL_CC_SUCCESS,
+ .operation_result = 0,
+ .eid_setting = s->my_eid,
+ .eid_pool_size = 0,
+ };
+
+ usb_mctp_fill_common(o_usb_pkt, s->pack, usb_pkt_len);
+
+ return o_usb_pkt;
+ }
+ case MCTP_CONTROL_GET_EID: {
+ uint8_t usb_pkt_len = sizeof(MCTPUSBPacket) +
+ sizeof(MCTPControlMessage) +
+ sizeof(MCTPControlGetEIDRsp);
+ MCTPUSBPacket *o_usb_pkt = g_malloc0(usb_pkt_len);
+ MCTPPacket *o_mctp_pkt = &o_usb_pkt->mctp;
+ MCTPControlMessage *o_mctp_ctrl =
+ (MCTPControlMessage *)(o_mctp_pkt->payload);
+ MCTPControlGetEIDRsp *rsp = (MCTPControlGetEIDRsp *)o_mctp_ctrl->data;
+
+ o_mctp_ctrl->type = 0;
+ o_mctp_ctrl->command_code = ctrlmsg->command_code;
+ o_mctp_ctrl->flags = ctrlmsg->flags &
+ ~(MCTP_CONTROL_FLAGS_RQ | MCTP_CONTROL_FLAGS_D);
+ *rsp = (MCTPControlGetEIDRsp) {
+ .completion_code = MCTP_CONTROL_CC_SUCCESS,
+ .endpoint_id = s->my_eid,
+ .endpoint_type = 0,
+ .medium_specific_info = 0,
+ };
+ usb_mctp_fill_common(o_usb_pkt, s->pack, usb_pkt_len);
+
+ return o_usb_pkt;
+ }
+ case MCTP_CONTROL_GET_UUID: {
+ uint8_t usb_pkt_len = sizeof(MCTPUSBPacket) +
+ sizeof(MCTPControlMessage) +
+ sizeof(MCTPControlGetUUIDRsp);
+ MCTPUSBPacket *o_usb_pkt = g_malloc0(usb_pkt_len);
+ MCTPPacket *o_mctp_pkt = &o_usb_pkt->mctp;
+ MCTPControlMessage *o_mctp_ctrl =
+ (MCTPControlMessage *)(o_mctp_pkt->payload);
+ MCTPControlGetUUIDRsp *rsp = (MCTPControlGetUUIDRsp *)o_mctp_ctrl->data;
+
+ o_mctp_ctrl->type = 0;
+ o_mctp_ctrl->command_code = ctrlmsg->command_code;
+ o_mctp_ctrl->flags = ctrlmsg->flags &
+ ~(MCTP_CONTROL_FLAGS_RQ | MCTP_CONTROL_FLAGS_D);
+ *rsp = (MCTPControlGetUUIDRsp) {
+ .completion_code = MCTP_CONTROL_CC_SUCCESS,
+ /* TODO: Fill in uuid */
+ };
+ usb_mctp_fill_common(o_usb_pkt, s->pack, usb_pkt_len);
+
+ return o_usb_pkt;
+ }
+ case MCTP_CONTROL_GET_MESSAGE_TYPE_SUPPORT: {
+ const uint8_t types[] = { MCTP_MT_CXL_FMAPI, MCTP_MT_CXL_TYPE3, };
+ uint8_t usb_pkt_len = sizeof(MCTPUSBPacket) +
+ sizeof(MCTPControlMessage) +
+ sizeof(MCTPControlGetMessageTypeRsp) +
+ sizeof(types);
+ MCTPUSBPacket *o_usb_pkt = g_malloc0(usb_pkt_len);
+ MCTPPacket *o_mctp_pkt = &o_usb_pkt->mctp;
+ MCTPControlMessage *o_mctp_ctrl =
+ (MCTPControlMessage *)(o_mctp_pkt->payload);
+ MCTPControlGetMessageTypeRsp *rsp =
+ (MCTPControlGetMessageTypeRsp *)o_mctp_ctrl->data;
+
+ o_mctp_ctrl->type = 0;
+ o_mctp_ctrl->command_code = ctrlmsg->command_code;
+ o_mctp_ctrl->flags = ctrlmsg->flags &
+ ~(MCTP_CONTROL_FLAGS_RQ | MCTP_CONTROL_FLAGS_D);
+ *rsp = (MCTPControlGetMessageTypeRsp) {
+ .completion_code = MCTP_CONTROL_CC_SUCCESS,
+ .message_type_count = sizeof(types),
+ };
+ memcpy(rsp->types, types, sizeof(types));
+
+ usb_mctp_fill_common(o_usb_pkt, s->pack, usb_pkt_len);
+
+ return o_usb_pkt;
+ }
+ default: {
+ uint8_t usb_pkt_len = sizeof(MCTPUSBPacket) +
+ sizeof(MCTPControlMessage) +
+ sizeof(MCTPControlErrRsp);
+ MCTPUSBPacket *o_usb_pkt = g_malloc0(usb_pkt_len);
+ MCTPPacket *o_mctp_pkt = &o_usb_pkt->mctp;
+ MCTPControlMessage *o_mctp_ctrl =
+ (MCTPControlMessage *)(o_mctp_pkt->payload);
+ MCTPControlErrRsp *rsp = (MCTPControlErrRsp *)o_mctp_ctrl->data;
+
+ o_mctp_ctrl->type = 0;
+ o_mctp_ctrl->command_code = ctrlmsg->command_code;
+ o_mctp_ctrl->flags = ctrlmsg->flags &
+ ~(MCTP_CONTROL_FLAGS_RQ | MCTP_CONTROL_FLAGS_D);
+ *rsp = (MCTPControlErrRsp) {
+ .completion_code = MCTP_CONTROL_CC_ERROR_UNSUP_COMMAND,
+ };
+ usb_mctp_fill_common(o_usb_pkt, s->pack, usb_pkt_len);
+ return o_usb_pkt;
+ }
+ }
+}
+
+static void usb_cxl_mctp_handle_data(USBDevice *dev, USBPacket *p)
+{
+ USBCXLMCTPState *s = USB_CXL_MCTP(dev);
+ USBPacket *tohost, *fromhost;
+ uint8_t message_type;
+ bool som, eom;
+
+ /*
+ * In and out on EP 0x1: anything else is bug.
+ * Not sure what right answer is.
+ */
+ if (p->ep->nr != 1) {
+ p->status = USB_RET_STALL;
+ return;
+ }
+
+ /*
+ * Conservative approach - don't proceed until we have at least one packet
+ * in each direction. For fragmented messages cases we only need this to be
+ * true for the EOM packet (potential optimization).
+ */
+ if (p->pid == USB_TOKEN_IN) { /* Direction from point of view of host */
+ tohost = p;
+ if (s->cached_fromhost == NULL) {
+ s->cached_tohost = tohost;
+ tohost->status = USB_RET_ASYNC;
+ return;
+ }
+ fromhost = s->cached_fromhost;
+ } else {
+ fromhost = p;
+ if (s->cached_tohost == NULL) {
+ s->cached_fromhost = fromhost;
+ fromhost->status = USB_RET_ASYNC;
+ return;
+ }
+ tohost = s->cached_tohost;
+ }
+
+ /* DSP0236 1.3.3 section 8.7 Dropped packets, physical layer errors */
+ if (fromhost->iov.size < sizeof(s->pack->mctp)) {
+ goto err_drop;
+ }
+
+ usb_packet_copy(fromhost, s->pack0, fromhost->iov.size);
+ /* DSP0236 1.3.3 section 8.7 Dropped packets, physical layer errors */
+ if ((be16_to_cpu(s->pack0->dmtf_id) != 0x1AB4) ||
+ (fromhost->iov.size != s->pack0->length)) {
+ goto err_drop;
+ }
+
+ eom = FIELD_EX8(s->pack0->mctp.hdr.flags, MCTP_H_FLAGS, EOM);
+ som = FIELD_EX8(s->pack0->mctp.hdr.flags, MCTP_H_FLAGS, SOM);
+
+ /* DSP0236 1.3.3 section 8.7 Dropped packets, unexpected middle or end */
+ if (!som && !s->building_input) {
+ goto err_drop;
+ }
+
+ if (som) {
+ /* Note repeated SOM without EOM is not an error */
+ s->building_input = true;
+ /* Put first part of full message in place */
+ memcpy(s->pack, s->pack0, s->pack0->length);
+ } else {
+ size_t additional_len;
+
+ additional_len = s->pack0->length - 8;
+ memcpy((void *)(s->pack) + s->pack->length, s->pack0->mctp.payload,
+ additional_len);
+ s->pack->length += additional_len;
+ }
+
+ if (eom) {
+ s->building_input = false;
+ } else { /* More to come so complete message from host to let it flow */
+ if (p->pid == USB_TOKEN_IN) {
+ /* Hold the tohost packet */
+ p->status = USB_RET_ASYNC;
+ s->cached_tohost = p;
+ /* Release the fromhost packet */
+ fromhost->status = USB_RET_SUCCESS;
+ s->cached_fromhost = NULL;
+ usb_packet_complete(dev, fromhost);
+ } else {
+ /* From host - handled synchronously */
+ p->status = USB_RET_SUCCESS;
+ }
+ return;
+ }
+
+ /* DSP0236 1.3.3 section 8.7 Dropped packets, bad header version */
+ if (s->pack->mctp.hdr.version != 1) {
+ goto err_drop;
+ }
+
+ /* DSP0236 1.3.3 section 8.7 Dropped packets, unknown eid */
+ if ((s->pack->mctp.hdr.eid.dest != s->my_eid) &&
+ (s->pack->mctp.hdr.eid.dest != 0)) {
+ goto err_drop;
+ }
+
+ message_type = s->pack->mctp.payload[0];
+ switch (message_type) {
+ case MCTP_MESSAGE_TYPE_CONTROL: {
+ MCTPUSBPacket *o_usb_pkt;
+ MCTPControlMessage *ctrlmsg =
+ (MCTPControlMessage *)(s->pack->mctp.payload);
+
+ /* DSP0236 1.3.3 section 8.7 Dropped packets, physical layer errors */
+ if (fromhost->iov.size < sizeof(s->pack->mctp) + sizeof(*ctrlmsg)) {
+ goto err_drop;
+ }
+
+ o_usb_pkt = usb_mctp_handle_control(s, ctrlmsg, tohost);
+
+ usb_packet_copy(tohost, o_usb_pkt, o_usb_pkt->length);
+ g_free(o_usb_pkt);
+
+ break;
+ }
+ case MCTP_MT_CXL_TYPE3:
+ case MCTP_MT_CXL_FMAPI: {
+ size_t usb_pkt_len = MCTPUSBCXL_MAX_SIZE;
+ g_autofree MCTPUSBPacket *o_usb_pkt = g_malloc0(usb_pkt_len);
+ MCTPPacket *o_mctp_pkt = &o_usb_pkt->mctp;
+ CXLMCTPMessage *rsp = (CXLMCTPMessage *)(o_mctp_pkt->payload);
+ CXLMCTPMessage *req = (CXLMCTPMessage *)(s->pack->mctp.payload);
+ bool bg_started;
+ size_t len_out = 0;
+ size_t len_in;
+ int rc;
+
+ *rsp = (CXLMCTPMessage) {
+ .message_type = req->message_type,
+ .category = 1,
+ .tag = req->tag,
+ .command = req->command,
+ .command_set = req->command_set,
+ };
+
+ /*
+ * As it was not immediately obvious from the various specifications,
+ * clarification was sort for which binding applies for which command
+ * set. The outcome was:
+ *
+ * Any command forming part of the CXL FM-API command set
+ * e.g. Present in CXL r3.0 Table 8-132: CXL FM API Command Opcodes
+ * (and equivalent in later CXL specifications) is valid only with
+ * the CXL Fabric Manager API over MCTP binding (DSP0234).
+ *
+ * Any other CXL command currently should be sent using the
+ * CXL Type 3 Device Component Command interface over MCTP binding,
+ * even if it is being sent to a switch.
+ *
+ * If tunneling is used, the component creating the PCIe VDMs must
+ * use the appropriate binding for sending the tunnel contents
+ * onwards.
+ */
+ if (!(req->message_type == MCTP_MT_CXL_TYPE3 &&
+ req->command_set < 0x51) &&
+ !(req->message_type == MCTP_MT_CXL_FMAPI &&
+ req->command_set >= 0x51 && req->command_set < 0x56)) {
+ len_out = 0;
+ usb_pkt_len = sizeof(MCTPUSBPacket) + sizeof(CXLMCTPMessage) +
+ len_out;
+ rsp->rc = CXL_MBOX_UNSUPPORTED;
+ st24_le_p(rsp->pl_length, len_out);
+
+ usb_mctp_fill_common(o_usb_pkt, s->pack, usb_pkt_len);
+ usb_packet_copy(tohost, o_usb_pkt, usb_pkt_len);
+ break;
+ }
+ /* TODO: add ld24_le_p */
+ len_in = req->pl_length[2] << 16 | req->pl_length[1] << 8 |
+ req->pl_length[0];
+ rc = cxl_process_cci_message(s->cci, req->command_set,
+ req->command,
+ len_in, req->payload,
+ &len_out,
+ (uint8_t *)(rsp + 1),
+ &bg_started);
+ rsp->rc = rc;
+ st24_le_p(rsp->pl_length, len_out);
+ usb_pkt_len = sizeof(MCTPUSBPacket) + sizeof(CXLMCTPMessage) +
+ len_out;
+
+ usb_mctp_fill_common(o_usb_pkt, s->pack, usb_pkt_len);
+ usb_packet_copy(tohost, o_usb_pkt, usb_pkt_len);
+ break;
+ }
+ default:
+ /*
+ * 8.9 Dropped messages - message type unsuported.
+ * Dropping after assembly
+ */
+ goto err_drop;
+ }
+ /* Something to send */
+ tohost->status = USB_RET_SUCCESS;
+ fromhost->status = USB_RET_SUCCESS;
+ if (p->pid == USB_TOKEN_IN) {
+ s->cached_fromhost = NULL;
+ usb_packet_complete(dev, fromhost);
+ } else {
+ s->cached_tohost = NULL;
+ usb_packet_complete(dev, tohost);
+ }
+ return;
+
+err_drop:
+ /* Reply with 'nothing' as dropping packet */
+ fromhost->status = USB_RET_SUCCESS;
+ if (p->pid == USB_TOKEN_IN) {
+ /* Hold the tohost packet */
+ tohost->status = USB_RET_ASYNC;
+ s->cached_tohost = p;
+ s->cached_fromhost = NULL;
+ usb_packet_complete(dev, fromhost);
+ }
+}
+
+static void usb_cxl_mctp_realize(USBDevice *dev, Error **errp)
+{
+ USBCXLMCTPState *s = USB_CXL_MCTP(dev);
+
+ s->pack = g_malloc0(MCTPUSBCXL_MAX_SIZE);
+ s->pack0 = g_malloc0(MCTPUSBCXL_MAX_SIZE);
+ usb_desc_create_serial(dev);
+ usb_desc_init(dev);
+
+ /* Check this is a type we support */
+ if (object_dynamic_cast(OBJECT(s->target), TYPE_CXL_USP)) {
+ CXLUpstreamPort *usp = CXL_USP(s->target);
+
+ s->type = cxl_switch;
+ s->cci = &usp->mctpcci;
+
+ cxl_initialize_usp_mctpcci(s->cci, DEVICE(s->target), DEVICE(dev),
+ MCTP_CXL_MAILBOX_BYTES);
+
+ return;
+ }
+
+ if (object_dynamic_cast(OBJECT(s->target), TYPE_CXL_TYPE3)) {
+ CXLType3Dev *ct3d = CXL_TYPE3(s->target);
+
+ s->type = cxl_type3;
+ s->cci = &ct3d->oob_mctp_cci;
+
+ cxl_initialize_t3_fm_owned_ld_mctpcci(s->cci, DEVICE(s->target),
+ DEVICE(dev),
+ MCTP_CXL_MAILBOX_BYTES);
+ return;
+ }
+
+ error_setg(errp, "Unhandled target type for CXL MCTP EP");
+}
+
+static const Property usb_cxl_mctp_properties[] = {
+ DEFINE_PROP_LINK("target", USBCXLMCTPState, target, TYPE_PCI_DEVICE,
+ PCIDevice *),
+};
+
+static void usb_cxl_mctp_class_initfn(ObjectClass *klass, const void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(klass);
+ USBDeviceClass *uc = USB_DEVICE_CLASS(klass);
+
+ uc->realize = usb_cxl_mctp_realize;
+ uc->product_desc = "QEMU USB CXL MCTP";
+ uc->usb_desc = &desc;
+ uc->handle_attach = usb_desc_attach;
+ uc->handle_reset = usb_cxl_mctp_handle_reset;
+ uc->handle_control = usb_cxl_mctp_handle_control;
+ uc->handle_data = usb_cxl_mctp_handle_data;
+ dc->desc = "USB CXL MCTP device";
+ dc->fw_name = "mctp";
+ device_class_set_props(dc, usb_cxl_mctp_properties);
+}
+
+static const TypeInfo usb_cxl_mctp_info = {
+ .name = TYPE_USB_CXL_MCTP,
+ .parent = TYPE_USB_DEVICE,
+ .instance_size = sizeof(USBCXLMCTPState),
+ .class_init = usb_cxl_mctp_class_initfn,
+};
+
+static void usb_cxl_mctp_register_types(void)
+{
+ type_register_static(&usb_cxl_mctp_info);
+}
+type_init(usb_cxl_mctp_register_types)
diff --git a/hw/usb/Kconfig b/hw/usb/Kconfig
index 69c663be52..4340475708 100644
--- a/hw/usb/Kconfig
+++ b/hw/usb/Kconfig
@@ -105,6 +105,11 @@ config USB_SERIAL
default y
depends on USB
+config USB_MCTP
+ bool
+ default y if CXL
+ depends on USB
+
config USB_NETWORK
bool
default y
diff --git a/hw/usb/meson.build b/hw/usb/meson.build
index 17360a5b5a..c2910f9818 100644
--- a/hw/usb/meson.build
+++ b/hw/usb/meson.build
@@ -43,6 +43,7 @@ system_ss.add(when: 'CONFIG_USB_STORAGE_UAS', if_true: files('dev-uas.c'))
system_ss.add(when: 'CONFIG_USB_AUDIO', if_true: files('dev-audio.c'))
system_ss.add(when: 'CONFIG_USB_SERIAL', if_true: files('dev-serial.c'))
system_ss.add(when: 'CONFIG_USB_NETWORK', if_true: files('dev-network.c'))
+system_ss.add(when: 'CONFIG_USB_MCTP', if_true: files('dev-mctp.c'))
if host_os != 'windows'
system_ss.add(when: 'CONFIG_USB_STORAGE_MTP', if_true: files('dev-mtp.c'))
endif
--
2.48.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [RFC PATCH qemu 3/5] hw/cxl/i2c_mctp_cxl: Initial device emulation
2025-06-09 16:33 ` [RFC PATCH qemu 3/5] hw/cxl/i2c_mctp_cxl: Initial device emulation Jonathan Cameron
@ 2025-06-10 16:39 ` Jonathan Cameron
0 siblings, 0 replies; 8+ messages in thread
From: Jonathan Cameron @ 2025-06-10 16:39 UTC (permalink / raw)
To: Jonathan Cameron via, linuxarm
Cc: Jonathan Cameron, Klaus Jensen, cminyard, Fan Ni, Anisa Su,
linux-cxl, mst, Philippe Mathieu-Daudé
On Mon, 9 Jun 2025 17:33:31 +0100
Jonathan Cameron via <qemu-devel@nongnu.org> wrote:
> The CCI and Fabric Manager APIs are used to configure CXL switches and
> devices. DMTF has defined an MCTP binding specification to carry these
> messages. The end goal of this work is to hook this up to emulated CXL
> switches and devices to allow control of the configuration.
>
> Since this relies on i2c target mode, this can currently only be used with
> an SoC that includes the Aspeed I2C controller.
>
> Note, only get timestamp added for now.
>
> Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Not very important given this is an RFC, but a chunk of code in here isn't
used so I'll drop it from future version...
(I noticed whilst wondering why Anisa didn't update it in they
FMAPI DCD series).
> ---
> include/hw/cxl/cxl_device.h | 8 +
> include/hw/pci-bridge/cxl_upstream_port.h | 1 +
> hw/cxl/cxl-mailbox-utils.c | 49 ++++
> hw/cxl/i2c_mctp_cxl.c | 289 ++++++++++++++++++++++
> hw/cxl/Kconfig | 4 +
> hw/cxl/meson.build | 4 +
> 6 files changed, 355 insertions(+)
>
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index 6086d4c85b..8d87c7151e 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -360,6 +360,10 @@ int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
> size_t len_in, uint8_t *pl_in,
> size_t *len_out, uint8_t *pl_out,
> bool *bg_started);
> +
> +void cxl_initialize_t3_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
> + size_t payload_max);
> +
> diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
> index 475547f212..4c9852642e 100644
> --- a/hw/cxl/cxl-mailbox-utils.c
> +++ b/hw/cxl/cxl-mailbox-utils.c
> @@ -3690,6 +3690,29 @@ void cxl_initialize_mailbox_t3(CXLCCI *cci, DeviceState *d, size_t payload_max)
> cxl_init_cci(cci, payload_max);
> }
>
> +static const struct cxl_cmd cxl_cmd_set_t3_mctp[256][256] = {
> + [INFOSTAT][IS_IDENTIFY] = { "IDENTIFY", cmd_infostat_identify, 0, 0 },
> + [INFOSTAT][GET_RESPONSE_MSG_LIMIT] = { "GET_RESPONSE_MSG_LIMIT",
> + cmd_get_response_msg_limit, 0, 0 },
> + [INFOSTAT][SET_RESPONSE_MSG_LIMIT] = { "SET_RESPONSE_MSG_LIMIT",
> + cmd_set_response_msg_limit, 1, 0 },
> + [TIMESTAMP][GET] = { "TIMESTAMP_GET", cmd_timestamp_get, 0, 0 },
> + [LOGS][GET_SUPPORTED] = { "LOGS_GET_SUPPORTED", cmd_logs_get_supported, 0,
> + 0 },
> + [LOGS][GET_LOG] = { "LOGS_GET_LOG", cmd_logs_get_log, 0x18, 0 },
> + [TUNNEL][MANAGEMENT_COMMAND] = { "TUNNEL_MANAGEMENT_COMMAND",
> + cmd_tunnel_management_cmd, ~0, 0 },
> +};
> +
> +void cxl_initialize_t3_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
> + size_t payload_max)
> +{
> + cxl_copy_cci_commands(cci, cxl_cmd_set_t3_mctp);
> + cci->d = d;
> + cci->intf = intf;
> + cxl_init_cci(cci, payload_max);
> +}
> +
At some point a while back, when adding tunneling we decided to pretend
we have an MLD (because it's more general) and as such introduced the fm owned
mctpcci. That's the one one that is used for the landing point of mctp
links to a type 3 device. So this code is unused.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [RFC PATCH qemu 0/5] hw/cxl/mctp/i2c/usb: MCTP for OoB control of CXL devices.
2025-06-09 16:33 [RFC PATCH qemu 0/5] hw/cxl/mctp/i2c/usb: MCTP for OoB control of CXL devices Jonathan Cameron
` (4 preceding siblings ...)
2025-06-09 16:33 ` [RFC PATCH qemu 5/5] usb/mctp/cxl: CXL FMAPI interface via MCTP over usb Jonathan Cameron
@ 2025-06-24 19:47 ` Anisa Su
5 siblings, 0 replies; 8+ messages in thread
From: Anisa Su @ 2025-06-24 19:47 UTC (permalink / raw)
To: Jonathan Cameron
Cc: Klaus Jensen, cminyard, Fan Ni, qemu-devel, linux-cxl, mst,
linuxarm, Philippe Mathieu-Daudé
On Mon, Jun 09, 2025 at 05:33:28PM +0100, Jonathan Cameron wrote:
> This posting is primarily about sharing the USB device emulation to get some
> early feedback.
>
> RFC reasons:
> - Known 'inaccuracies' in emulation (not obeying MTU in the to host direction for
> example)./
> - Not sure what to do wrt to Klaus' I2C MCTP support given that has been stalled
> for some time. For now only the headers are really shared between the
> two implementations.
> - This is more of an FYI / request for testing than a formal suggestion that this
> might be ready for upstream.
>
> Why add a CXL FM-API over MCTP over USB device?
> - Can be emulated on pretty much any host system as USB is discoverable and
> expandable. If you want a giggle, see the hacks on i386/pc and arm/virt on
> we've been using until now given only I2C controller that works is the aspeed
> one. e.g. https://gitlab.com/jic23/qemu/-/commit/134c2e3952b
> - Being able to talk to both the fabric management out of band interfaces
> and the in band devices on the same host makes testing much simpler.
>
> Background:
>
> Back in 2022 I posted some support for controlling the CXL fabric via the
> spec defined out of band interfaces (CXL Fabric Management API - FM-API)
> over MCTP on I2C
>
> https://lore.kernel.org/qemu-devel/20220520170128.4436-1-Jonathan.Cameron@huawei.com/
> I reworked that on top of the NVME-MI work from Klauss.
>
> To that end I hacked the aspeed-i2c controller onto both i386/pc and arm/virt
> and posted kernel patches to enabled ACPI support for that device (more or less).
> It worked and has been useful in the meantime, but adding that i2c controller
> to those boards was obviously not going to be upstreamable - and to build
> reliable tests against it I don't want to carry this out of tree for ever.
> I messed around with a PCI hosted aspeed controller and might come back to
> that at some point.
>
> In the meantime, DMTF published a transport binding for MCTP over USB
> https://www.dmtf.org/sites/default/files/standards/documents/DSP0283_1.0.1.pdf
>
> Kernel support duly followed early this year: drivers/net/mctp/mctp-usb.c
> https://codeconstruct.com.au/docs/mctp-over-usb/
>
> Given the ease of adding a suitable USB controller on a PCI bus, emulating a
> suitable endpoint provides what I think is an upstreamable solution.
>
> To use this:
> -device usb-ehci,id=ehci
> -device usb-cxl-mctp,bus=ehci.0,id=fred,target=us0
>
> where target is either a CXL switch upstream port, or a type 3 device.
>
> Then install the mctp userspace tools in your guest and configure it with
>
> mctp addr add 8 dev mctpusb0
> mctp link set mctpusb0 net 11
> mctp link set mctpusb0 up
>
> systemctl start mctpd.service
> busctl call au.com.codeconstruct.MCTP1 /au/com/codeconstruct/mctp1/interfaces/mctpusb0 au.com.codeconstruct.MCTP.BusOwner1 SetupEndpoint ay 0
>
> I've been testing the CXL commands with
>
> https://gitlab.com/jic23/cxl-fmapi-tests
> (mostly because I still had them in my image from the i2c work)
> but libcxlmi is probably a better bet
>
I tested this patchset using libcxlmi with the following configuration:
- kernel version: https://github.com/weiny2/linux-kernel/tree/dcd-v6-2025-04-13
- QEMU: upstream ToT with MCTP USB CXL and FMAPI DCD support:
- MCTP USB CXL (https://lore.kernel.org/linux-cxl/20250609163334.922346-1-Jonathan.Cameron@huawei.com/T/#m21b9e0dfc689cb1890bb4d961710c23379e04902)
- note: I modified the condition on line hw/usb/dev-mctp.c:509 to allow
the FMAPI DCD Management command set (0x56) to be processed
- FMAPI DCD Management patches (https://lore.kernel.org/linux-cxl/20250605234227.970187-1-anisa.su887@gmail.com/)
- Topology (1 DCD attached to downstream port of CXL Switch)
'-device usb-ehci,id=ehci \
-object memory-backend-file,id=cxl-mem1,mem-path=/tmp/t3_cxl1.raw,size=4G \
-object memory-backend-file,id=cxl-lsa1,mem-path=/tmp/t3_lsa1.raw,size=1M \
-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
-device cxl-rp,port=0,bus=cxl.1,id=cxl_rp_port0,chassis=0,slot=2 \
-device cxl-upstream,port=2,sn=1234,bus=cxl_rp_port0,id=us0,addr=0.0,multifunction=on, \
-device cxl-switch-mailbox-cci,bus=cxl_rp_port0,addr=0.1,target=us0 \
-device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \
-device cxl-type3,bus=swport0,volatile-dc-memdev=cxl-mem1,id=cxl-dcd0,lsa=cxl-lsa1,num-dc-regions=2,sn=99 \
-device usb-cxl-mctp,bus=ehci.0,id=usb0,target=us0 \
-device usb-cxl-mctp,bus=ehci.0,id=usb1,target=cxl-dcd0\
-machine cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=1k'
After creating the VM with the above topology, to configure the 2 MCTP EPs
(mctpusb0 and mctpusb1):
mctp link set mctpusb0 up
mctp addr add 50 dev mctpusb0
mctp link set mctpusb0 net 11
mctp link set mctpusb1 up
mctp addr add 51 dev mctpusb1
mctp link set mctpusb0 net 12
systemctl stop mctpd.service
systemctl start mctpd.service
busctl call au.com.codeconstruct.MCTP1 /au/com/codeconstruct/mctp1/interfaces/mctpusb0 au.com.codeconstruct.MCTP.BusOwner1 SetupEndpoint ay 0
busctl call au.com.codeconstruct.MCTP1 /au/com/codeconstruct/mctp1/interfaces/mctpusb1 au.com.codeconstruct.MCTP.BusOwner1 SetupEndpoint ay 0
To test this patchset, I used this program from libcxlmi:
https://github.com/computexpresslink/libcxlmi/blob/main/examples/cxl-mctp.c
To summarize, this program scans the D-bus for MCTP endpoints or takes the NID:EID
of the endpoint you want to open as arguments, then sends a variety of commands
(identify, supported logs, etc.) through the endpoint(s) and prints the results,
which I verified. Both methods of opening endpoints are successful and shows
the expected output.
To ensure this works end-to-end with an FM workflow I also tested with
this program, which allows users to interactively send the FMAPI Add/Release
commands from the command line:
https://github.com/computexpresslink/libcxlmi/blob/main/examples/fmapi-mctp.c
First I created a DC Region using this ndctl command:
cxl create-region -m mem0 -d decoder0.0 -s 1G -t dynamic_ram_a
Then I ran the program to add an extent of 512MB and was able to successfully
create a DAX Device using the following commands:
daxctl create-device -r region0
daxctl reconfigure-device dax0.1 -m system-ram
The output of 'lsmem' shows 512M added:
RANGE SIZE STATE REMOVABLE BLOCK
0x0000000000000000-0x000000007fffffff 2G online yes 0-15
0x0000000100000000-0x000000027fffffff 6G online yes 32-79
0x0000001290000000-0x00000012afffffff 512M online yes 594-597
> https://github.com/computexpresslink/libcxlmi
>
> I'll post a tree with this on at gitlab.com/jic23/qemu shortly
> (cxl-<latest date>).
>
> Jonathan Cameron (3):
> hw/cxl/i2c_mctp_cxl: Initial device emulation
> docs: cxl: Add example commandline for MCTP CXL CCIs
> usb/mctp/cxl: CXL FMAPI interface via MCTP over usb.
>
> Klaus Jensen (2):
> hw/i2c: add smbus pec utility function
> hw/i2c: add mctp core
>
> MAINTAINERS | 7 +
> docs/system/devices/cxl.rst | 27 +
> include/hw/cxl/cxl_device.h | 8 +
> include/hw/i2c/mctp.h | 125 +++++
> include/hw/i2c/smbus_master.h | 2 +
> include/hw/pci-bridge/cxl_upstream_port.h | 1 +
> include/hw/usb.h | 1 +
> include/net/mctp.h | 100 ++++
> hw/cxl/cxl-mailbox-utils.c | 49 ++
> hw/cxl/i2c_mctp_cxl.c | 289 ++++++++++
> hw/i2c/mctp.c | 414 ++++++++++++++
> hw/i2c/smbus_master.c | 26 +
> hw/usb/dev-mctp.c | 639 ++++++++++++++++++++++
> hw/arm/Kconfig | 1 +
> hw/cxl/Kconfig | 4 +
> hw/cxl/meson.build | 4 +
> hw/i2c/Kconfig | 4 +
> hw/i2c/meson.build | 1 +
> hw/i2c/trace-events | 14 +
> hw/usb/Kconfig | 5 +
> hw/usb/meson.build | 1 +
> 21 files changed, 1722 insertions(+)
> create mode 100644 include/hw/i2c/mctp.h
> create mode 100644 include/net/mctp.h
> create mode 100644 hw/cxl/i2c_mctp_cxl.c
> create mode 100644 hw/i2c/mctp.c
> create mode 100644 hw/usb/dev-mctp.c
>
> --
> 2.48.1
>
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2025-06-24 19:47 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-06-09 16:33 [RFC PATCH qemu 0/5] hw/cxl/mctp/i2c/usb: MCTP for OoB control of CXL devices Jonathan Cameron
2025-06-09 16:33 ` [RFC PATCH qemu 1/5] hw/i2c: add smbus pec utility function Jonathan Cameron
2025-06-09 16:33 ` [RFC PATCH qemu 2/5] hw/i2c: add mctp core Jonathan Cameron
2025-06-09 16:33 ` [RFC PATCH qemu 3/5] hw/cxl/i2c_mctp_cxl: Initial device emulation Jonathan Cameron
2025-06-10 16:39 ` Jonathan Cameron
2025-06-09 16:33 ` [RFC PATCH qemu 4/5] docs: cxl: Add example commandline for MCTP CXL CCIs Jonathan Cameron
2025-06-09 16:33 ` [RFC PATCH qemu 5/5] usb/mctp/cxl: CXL FMAPI interface via MCTP over usb Jonathan Cameron
2025-06-24 19:47 ` [RFC PATCH qemu 0/5] hw/cxl/mctp/i2c/usb: MCTP for OoB control of CXL devices Anisa Su
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox