[RFC 0/3] Qemu FM emulation

[RFC 0/3] Qemu FM emulation

[nifan.cxl]

From: Fan Ni <fan.ni@samsung.com>

The RFC provides a way for FM emulation in Qemu. The goal is to provide
a context where we can have more FM emulation discussions and share solutions
for a reasonable FM implementation in Qemu.

The basic idea is,

We have two VMs, one is the VM we want to test (named Target VM) and one is the
FM VM. The target VM has the kernel which we are interested (for example, DCD
or RAS feature enabled). The FM VM can be VM with any kernel version as long as
OOB communication support is enabled.

An application running in the FM VM issues FM commands to the underlying device
with OOB channel (e.g., MCTP over I2C), when the device receives the message,
it will not response to the request locally, instead the request will be stored
in a share buffer (implemented with /dev/shm), and a QMP request will be sent
to the target VM to notify there is a MCTP message in the shared buffer,
which needs to be processed. The FM will wait the completion of the request.
The target VM will read the buffer and process the message.
When the process completes, the output payload and any information needs to
return is stored in buffer, and a state field will be reset to notify the FM of
the completion of the processing.

The nice points of the method:
1. It is simple model (consumer-produce model with shm as shared buffer).
2. The communication between the two VMs through the qmp interface is simple.
One qmp interface works for all MCTP messages. Moreover, the qmp interface may
be able to use as a way for the communication between two VMs in different
context.

How we run the test?
Step 1: Start the VM we want to Target VM.
The device interested having "allow-fm-attach=on,mctp-buf-init=on"
For example, for my test, it is the DCD device.

In our test, the kernel run on the target VM is Ira's DCD branch:
https://github.com/weiny2/linux-kernel/tree/dcd-v4-2024-12-11.

qemu-system-x86_64 -gdb tcp::1235  -kernel bzImage -append "root=/dev/sda rw console=ttyS0,115200 ignore_loglevel nokaslr" \
-smp 8 -accel kvm -serial mon:stdio  -nographic  -qmp tcp:localhost:4445,server,wait=off \
-netdev user,id=network0,hostfwd=tcp::2024-:22    \
-device e1000,netdev=network0  -monitor telnet:127.0.0.1:12346,server,nowait \
-drive file=/home/fan/cxl/images/qemu-image.img,index=0,media=disk,format=raw \
-machine q35,cxl=on -cpu qemu64,mce=on -m 8G,maxmem=64G,slots=8 \
-virtfs local,path=/opt/lib/modules,mount_tag=modshare,security_model=mapped  \
-virtfs local,path=/home/fan,mount_tag=homeshare,security_model=mapped \
-object memory-backend-file,id=cxl-mem2,mem-path=/tmp/host0/t3_cxl2.raw,size=4G \
-object memory-backend-file,id=cxl-lsa2,mem-path=/tmp/host0/t3_lsa2.raw,size=1M \
-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
-device cxl-rp,port=0,bus=cxl.1,id=cxl_rp_port0,chassis=0,slot=2 \
-device cxl-upstream,port=2,sn=1234,bus=cxl_rp_port0,id=us0,addr=0.0,multifunction=on, \
-device cxl-switch-mailbox-cci,bus=cxl_rp_port0,addr=0.1,target=us0 \
-device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \
-device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \
-device cxl-downstream,port=3,bus=us0,id=swport2,chassis=0,slot=6 \
-device cxl-type3,bus=swport2,volatile-dc-memdev=cxl-mem2,id=cxl-dcd0,lsa=cxl-lsa2,num-dc-regions=2,sn=99,allow-fm-attach=on,mctp-buf-init=on \
-machine cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=1k \
-device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=4,target=us0 \
-device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=6,target=cxl-dcd0 \
-device virtio-rng-pci,bus=swport1

Step 2: Start the FM VM and run the test program to send MCTP requests and
forward to the target VM for processing.

Note: the kernel for FM VM should have MCTP support.

In the test, we use linux-v6.6-rc6 with Jonathan's MCTP hack patches:
https://github.com/moking/cxl-test-tool/blob/main/test-workflows/mctp/mctp-patches-kernel.patch

qemu-system-x86_64 -gdb tcp::1236 -kernel fm-bzImage -append "root=/dev/sda rw console=ttyS0,115200 ignore_loglevel nokaslr " \
-smp 8 -accel kvm -serial mon:stdio  -nographic  -qmp tcp:localhost:4446,server,wait=off \
-netdev user,id=network0,hostfwd=tcp::2025-:22    \
-device e1000,netdev=network0  -monitor telnet:127.0.0.1:12347,server,nowait \
-drive file=/home/fan/cxl/images/qemu-image-fm.img,index=0,media=disk,format=raw \
-machine q35,cxl=on -cpu qemu64,mce=on -m 8G,maxmem=64G,slots=8  \
-virtfs local,path=/opt/lib/modules,mount_tag=modshare,security_model=mapped  \
-virtfs local,path=/home/fan,mount_tag=homeshare,security_model=mapped \
-object memory-backend-file,id=cxl-mem2,mem-path=/tmp/host1/t3_cxl2.raw,size=4G \
-object memory-backend-file,id=cxl-lsa2,mem-path=/tmp/host1/t3_lsa2.raw,size=1M \
-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
-device cxl-rp,port=0,bus=cxl.1,id=cxl_rp_port0,chassis=0,slot=2 \
-device cxl-upstream,port=2,sn=1234,bus=cxl_rp_port0,id=us0,addr=0.0,multifunction=on, \
-device cxl-switch-mailbox-cci,bus=cxl_rp_port0,addr=0.1,target=us0 \
-device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \
-device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \
-device cxl-downstream,port=3,bus=us0,id=swport2,chassis=0,slot=6 \
-device cxl-type3,bus=swport2,volatile-dc-memdev=cxl-mem2,id=cxl-dcd0,lsa=cxl-lsa2,num-dc-regions=2,sn=99,allow-fm-attach=on \
-machine cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=1k \
-device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=4,target=us0 \
-device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=6,target=cxl-dcd0,qmp=127.0.0.1:4445,mctp-msg-forward=on \
-device virtio-rng-pci,bus=swport1

Currently, the code is not clean at all, it is a POC to prove the idea. Only
type3 (including DCD) devices can accept requests from the FM, which should be
easy to extend to support switch-targeted FM command processing.

The code is based on Jonathan's cxl-2025-03-20 branch.
A qemu branch with the code: https://github.com/moking/qemu-jic-clone/tree/fm-qmp

FYI.
I have a tool to make the test easier.
https://github.com/moking/cxl-test-tool/tree/main

Part of .var.config, see run_vars.example

QEMU_ROOT=~/cxl/jic/qemu
# for FM VM
FM_KERNEL_ROOT=~/cxl/linux-v6.6-rc6/
FM_QEMU_IMG=~/cxl/images/qemu-image-fm.img

# for Target VM
KERNEL_ROOT=~/cxl/linux-dcd/
QEMU_IMG=~/cxl/images/qemu-image.img

command:
1. cxl-tool.py --run -T FM_TARGET
2. cxl-tool.py --attach-VM -T FM_CLIENT
3. cxl-tool.py --install-libcxlmi-fm
4. cxl-tool.py --setup-mctp-fm
5. cxl-tool.py --login-fm (run the test program with libcxlmi)

Fan Ni (3):
  cxl_type3: Preparing information sharing between VMs
  cxl_type3: Add qmp_cxl_process_mctp_message qmp interface
  cxl/i2c_mctp_cxl: Add support to process MCTP command remotely

 hw/cxl/cxl-mctp-qmp.c             |  85 +++++++++++++++
 hw/cxl/i2c_mctp_cxl.c             |  68 ++++++++++--
 hw/cxl/meson.build                |   2 +-
 hw/mem/cxl_type3.c                | 166 +++++++++++++++++++++++++++++-
 hw/mem/cxl_type3_stubs.c          |   5 +
 include/hw/cxl/cxl_device.h       |   8 ++
 include/hw/cxl/cxl_mctp_message.h |  43 ++++++++
 qapi/cxl.json                     |  18 ++++
 8 files changed, 387 insertions(+), 8 deletions(-)
 create mode 100644 hw/cxl/cxl-mctp-qmp.c
 create mode 100644 include/hw/cxl/cxl_mctp_message.h

-- 
2.47.2

[RFC 1/3] cxl_type3: Preparing information sharing between VMs

[nifan.cxl]

From: Fan Ni <fan.ni@samsung.com>

Add two data structures for sharing information between multiple VMs.
The global cci_map_buf is used to provide mappings between cci name and cci
pointer.
Each VM has its own cci_map_buf. However, since we expect the two VMs share
the same configuration, the same CCI name should points to its own CCI.
On the FM, we need to use the cci pointer to find the cci name.
While on the test VM, we use the name to find cci pointer for MCTP
message process.
The mctp_shared_buffer is used to pass MCTP command information between
FM and test VM for QMP command process.

Signed-off-by: Fan Ni <fan.ni@samsung.com>
---
 hw/mem/cxl_type3.c                | 125 +++++++++++++++++++++++++++++-
 include/hw/cxl/cxl_device.h       |   6 ++
 include/hw/cxl/cxl_mctp_message.h |  40 ++++++++++
 3 files changed, 170 insertions(+), 1 deletion(-)
 create mode 100644 include/hw/cxl/cxl_mctp_message.h

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 11c38a9292..7f85616ca1 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -29,6 +29,9 @@
 #include "system/numa.h"
 #include "hw/cxl/cxl.h"
 #include "hw/pci/msix.h"
+#include "hw/cxl/cxl_mctp_message.h"
+
+struct CXLCCINamePtrMaps *cci_map_buf;
 
 /* type3 device private */
 enum CXL_T3_MSIX_VECTOR {
@@ -998,6 +1001,97 @@ static void init_alert_config(CXLType3Dev *ct3d)
     };
 }
 
+static int ct3_mctp_buf_open(const char *filename, int flags)
+{
+    char name[128];
+    snprintf(name, sizeof(name), "/%s", filename);
+    return shm_open(name, flags, 0666);
+}
+
+static int ct3_mctp_buf_unlink(const char *filename)
+{
+    char name[128];
+    snprintf(name, sizeof(name), "/%s", filename);
+    return shm_unlink(name);
+}
+
+static struct CXLMCTPSharedBuf *ct3_mctp_buf_map(int fd, int size)
+{
+    void *map;
+
+    if (fd < 0) {
+        return NULL;
+    }
+
+    map = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
+    if (map == MAP_FAILED) {
+        return NULL;
+    }
+
+    return (CXLMCTPSharedBuf *)map;
+}
+
+
+static int ct3_mctp_buf_create(const char *filename, size_t size)
+{
+    int fd, rc;
+
+    fd = ct3_mctp_buf_open(filename, O_RDWR | O_CREAT);
+    if (fd == -1) {
+        return -1;
+    }
+
+    rc = ftruncate(fd, size);
+
+    if (rc) {
+        close(fd);
+        return -1;
+    }
+
+    return fd;
+}
+
+static int ct3_setup_mctp_command_share_buffer(CXLType3Dev *ct3d, bool create)
+{
+    int fd;
+    int size = sizeof(*ct3d->mctp_shared_buffer);
+    sprintf(ct3d->mctp_buf_name, MCTP_MESSAGE_BUF_NAME);
+
+    if (create) {
+        fd = ct3_mctp_buf_create(ct3d->mctp_buf_name, size);
+    } else {
+        fd = ct3_mctp_buf_open(ct3d->mctp_buf_name, O_RDWR | O_CREAT);
+    }
+
+    if (fd == -1) {
+        return fd;
+    }
+    ct3d->mctp_buf_fd = fd;
+    ct3d->mctp_shared_buffer = ct3_mctp_buf_map(ct3d->mctp_buf_fd, size);
+    if (ct3d->mctp_shared_buffer) {
+        return 0;
+    }
+    return -1;
+}
+
+static int init_cci_name_ptr_mapping(void)
+{
+    if (!cci_map_buf) {
+        cci_map_buf = g_malloc(sizeof(*cci_map_buf));
+    }
+    return 0;
+}
+
+static void add_cci_name_ptr_mapping(const char *name, void *p)
+{
+    int n = cci_map_buf->num_mappings;
+    struct CXLCCINamePtrMap *map = &cci_map_buf->maps[n];
+
+    strcpy(map->cci_name, name);
+    map->cci_pointer = p;
+    cci_map_buf->num_mappings++;
+}
+
 void ct3_realize(PCIDevice *pci_dev, Error **errp)
 {
     ERRP_GUARD();
@@ -1108,6 +1202,14 @@ void ct3_realize(PCIDevice *pci_dev, Error **errp)
         ct3d->ecs_attrs.fru_attrs[count].ecs_flags = 0;
     }
 
+    if (ct3d->allow_fm_attach) {
+        init_cci_name_ptr_mapping();
+        if (ct3d->mctp_buf_init) {
+            ct3_setup_mctp_command_share_buffer(ct3d, true);
+        } else {
+            ct3_setup_mctp_command_share_buffer(ct3d, false);
+        }
+    }
     return;
 
 err_release_cdat:
@@ -1150,6 +1252,15 @@ void ct3_exit(PCIDevice *pci_dev)
     if (ct3d->hostvmem) {
         address_space_destroy(&ct3d->hostvmem_as);
     }
+
+    if (ct3d->mctp_shared_buffer) {
+        munmap(ct3d->mctp_shared_buffer, sizeof(*ct3d->mctp_shared_buffer));
+        close(ct3d->mctp_buf_fd);
+        ct3_mctp_buf_unlink(ct3d->mctp_buf_name);
+        ct3d->mctp_shared_buffer = NULL;
+    }
+    g_free(cci_map_buf);
+    cci_map_buf = NULL;
 }
 
 /*
@@ -1352,6 +1463,16 @@ void ct3d_reset(DeviceState *dev)
     }
     cxl_initialize_t3_ld_cci(&ct3d->ld0_cci, DEVICE(ct3d), DEVICE(ct3d),
                              512); /* Max payload made up */
+    if (ct3d->allow_fm_attach) {
+        char name[64];
+
+        memset(name, 0, 64);
+        sprintf(name, "%lu:%s", ct3d->sn, "oob_mctp_cci");
+        add_cci_name_ptr_mapping(name, &ct3d->oob_mctp_cci);
+        cxl_initialize_t3_fm_owned_ld_mctpcci(&ct3d->oob_mctp_cci,
+                                              DEVICE(ct3d), DEVICE(ct3d),
+                                              MCTP_CXL_MAILBOX_BYTES);
+    }
 }
 
 static const Property ct3_props[] = {
@@ -1372,7 +1493,9 @@ static const Property ct3_props[] = {
                                 speed, PCIE_LINK_SPEED_32),
     DEFINE_PROP_PCIE_LINK_WIDTH("x-width", CXLType3Dev,
                                 width, PCIE_LINK_WIDTH_16),
-    DEFINE_PROP_UINT16("chmu-port", CXLType3Dev, cxl_dstate.chmu[0].port, 0), 
+    DEFINE_PROP_UINT16("chmu-port", CXLType3Dev, cxl_dstate.chmu[0].port, 0),
+    DEFINE_PROP_BOOL("allow-fm-attach", CXLType3Dev, allow_fm_attach, false),
+    DEFINE_PROP_BOOL("mctp-buf-init", CXLType3Dev, mctp_buf_init, false),
 };
 
 static uint64_t get_lsa_size(CXLType3Dev *ct3d)
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index ca515cab13..9a00ef7a1e 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -707,6 +707,12 @@ struct CXLType3Dev {
     } dc;
 
     struct CXLSanitizeInfo *media_op_sanitize;
+
+    bool allow_fm_attach;
+    bool mctp_buf_init;
+    struct CXLMCTPSharedBuf *mctp_shared_buffer;
+    char mctp_buf_name[64];
+    int mctp_buf_fd;
 };
 
 #define TYPE_CXL_TYPE3 "cxl-type3"
diff --git a/include/hw/cxl/cxl_mctp_message.h b/include/hw/cxl/cxl_mctp_message.h
new file mode 100644
index 0000000000..85b3664cf7
--- /dev/null
+++ b/include/hw/cxl/cxl_mctp_message.h
@@ -0,0 +1,40 @@
+#ifndef CXL_MCTP_MESSAGE_H_H
+#define CXL_MCTP_MESSAGE_H_H
+#include<stdint.h>
+
+#define MCTP_CXL_MAILBOX_BYTES 512
+#define MCTP_MESSAGE_BUF_NAME "mctp-message-buf"
+
+struct CXLMCTPCommandBuf {
+    /* uint8_t cci_name[64]; */
+    uint8_t command_set;
+    uint8_t command;
+    size_t len_in;
+    size_t len_out;
+    uint8_t payload[MCTP_CXL_MAILBOX_BYTES];
+    uint8_t payload_out[MCTP_CXL_MAILBOX_BYTES];
+    bool bg_started;
+    int ret_val;
+};
+
+typedef struct CXLMCTPCommandBuf CXLMCTPCommandBuf;
+
+struct CXLCCINamePtrMap {
+    char cci_name[64];
+    void *cci_pointer; /* This should be filled by the target VM */
+};
+
+struct CXLCCINamePtrMaps {
+    int num_mappings;
+    struct CXLCCINamePtrMap maps[32];
+};
+
+struct CXLMCTPSharedBuf {
+    /* set to 1 when sent to target VM and wait for 0 as it completes */
+    int status;
+    CXLMCTPCommandBuf command_buf;
+};
+
+typedef struct CXLMCTPSharedBuf CXLMCTPSharedBuf;
+extern struct CXLCCINamePtrMaps *cci_map_buf;
+#endif
-- 
2.47.2

[RFC 2/3] cxl_type3: Add qmp_cxl_process_mctp_message qmp interface

[nifan.cxl]

From: Fan Ni <fan.ni@samsung.com>

Add a new QAPI "qmp-cxl-process-mctp-message" to process the MCTP
request cached in the shared buffer from FM VM.

Signed-off-by: Fan Ni <fan.ni@samsung.com>
---
 hw/mem/cxl_type3.c       | 41 ++++++++++++++++++++++++++++++++++++++++
 hw/mem/cxl_type3_stubs.c |  5 +++++
 qapi/cxl.json            | 18 ++++++++++++++++++
 3 files changed, 64 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 7f85616ca1..f7ac8e8da7 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1614,6 +1614,47 @@ void cxl_clear_poison_list_overflowed(CXLType3Dev *ct3d)
     ct3d->poison_list_overflow_ts = 0;
 }
 
+void qmp_cxl_process_mctp_message(const char *cci_name, Error **errp)
+{
+    int i;
+    struct CXLCCINamePtrMap *map;
+    struct CXLMCTPCommandBuf *buf;
+    CXLCCI *cci = NULL;
+    CXLType3Dev *ct3d;
+
+    if (!cci_map_buf) {
+        error_setg(errp, "CCI name mapping buffer not setup");
+        return;
+    }
+
+    for (i = 0; i < cci_map_buf->num_mappings; i++) {
+        map = &cci_map_buf->maps[i];
+        if (!strcmp(map->cci_name, cci_name)) {
+            cci = (CXLCCI *) map->cci_pointer;
+            break;
+        }
+    }
+    if (!cci) {
+        error_setg(errp, "CCI instance is not found with name: %s", cci_name);
+        return;
+    }
+    ct3d = CXL_TYPE3(OBJECT(cci->d));
+    if (!ct3d) {
+        error_setg(errp, "No Type3 device associated with the cci");
+        return;
+    }
+    if (ct3d->mctp_shared_buffer->status != 1) {
+        error_setg(errp, "MCTP buffer status is not set to 1, skip");
+        return;
+    }
+    buf = &ct3d->mctp_shared_buffer->command_buf;
+    buf->ret_val = cxl_process_cci_message(cci, buf->command_set, buf->command,
+                                           buf->len_in, buf->payload,
+                                           &buf->len_out, buf->payload_out,
+                                           &buf->bg_started);
+    ct3d->mctp_shared_buffer->status = 0;
+}
+
 void qmp_cxl_inject_poison(const char *path, uint64_t start, uint64_t length,
                            Error **errp)
 {
diff --git a/hw/mem/cxl_type3_stubs.c b/hw/mem/cxl_type3_stubs.c
index c1a5e4a7c1..dbf18f2af2 100644
--- a/hw/mem/cxl_type3_stubs.c
+++ b/hw/mem/cxl_type3_stubs.c
@@ -92,3 +92,8 @@ void qmp_cxl_release_dynamic_capacity(const char *path, uint16_t host_id,
 {
     error_setg(errp, "CXL Type 3 support is not compiled in");
 }
+
+void qmp_cxl_process_mctp_message(const char *cci_name, Error **errp)
+{
+    error_setg(errp, "CXL Type 3 support is not compiled in");
+}
diff --git a/qapi/cxl.json b/qapi/cxl.json
index dd947d3bbc..fcb1e0b3fd 100644
--- a/qapi/cxl.json
+++ b/qapi/cxl.json
@@ -553,3 +553,21 @@
            },
   'features': [ 'unstable' ]
 }
+
+##
+# @cxl-process-mctp-message:
+#
+# Command to process a MCTP command sent from another VM.
+#
+# @cci-name: name of the CCI to process the command;
+#
+# Features:
+#
+# @unstable: For now this command is subject to change.
+#
+# Since: 9.1
+##
+{'command': 'cxl-process-mctp-message',
+ 'data': {'cci-name': 'str'},
+  'features': [ 'unstable' ]
+ }
-- 
2.47.2

[RFC 3/3] cxl/i2c_mctp_cxl: Add support to process MCTP command remotely

[nifan.cxl]

From: Fan Ni <fan.ni@samsung.com>

With the change, if "mctp-msg-forward" property is turned on for
i2c_mctp_cxl device, we setup a QMP connection to the other VM identified
by the "qmp" property, and forward the MCTP requests to the VM for processing.

Note:
1. The VM with "mctp-msg-forward=on" acts as a FM VM.
2. Before the FM VM is started, the target VM to connect to must be on
   and ready to connect.
3. For the FM VM instance, it needs to share the same CXL topology with
   the VM we want to connect to, and the cxl device to connect needs to
   have "allow-fm-attach=on".

Signed-off-by: Fan Ni <fan.ni@samsung.com>
---
 hw/cxl/cxl-mctp-qmp.c             | 85 +++++++++++++++++++++++++++++++
 hw/cxl/i2c_mctp_cxl.c             | 68 ++++++++++++++++++++++---
 hw/cxl/meson.build                |  2 +-
 hw/mem/cxl_type3.c                |  4 +-
 include/hw/cxl/cxl_device.h       |  2 +
 include/hw/cxl/cxl_mctp_message.h |  3 ++
 6 files changed, 155 insertions(+), 9 deletions(-)
 create mode 100644 hw/cxl/cxl-mctp-qmp.c

diff --git a/hw/cxl/cxl-mctp-qmp.c b/hw/cxl/cxl-mctp-qmp.c
new file mode 100644
index 0000000000..f0ea5e2cc0
--- /dev/null
+++ b/hw/cxl/cxl-mctp-qmp.c
@@ -0,0 +1,85 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <arpa/inet.h>
+#include <sys/socket.h>
+#include <stdbool.h>
+
+#include "hw/cxl/cxl_mctp_message.h"
+
+#define BUFFER_SIZE 4096
+
+void read_qmp_response(int sockfd)
+{
+    char buffer[BUFFER_SIZE];
+    memset(buffer, 0, BUFFER_SIZE);
+    int len = read(sockfd, buffer, BUFFER_SIZE - 1);
+    if (len > 0) {
+        printf("QMP Response:\n%s\n", buffer);
+    }
+}
+
+static void send_qmp_command(int sockfd, const char *cmd)
+{
+    send(sockfd, cmd, strlen(cmd), 0);
+}
+
+static void send_qmp_cap_command(int sockfd)
+{
+    const char *cap_cmd = "{ \"execute\": \"qmp_capabilities\" }\n";
+    send_qmp_command(sockfd, cap_cmd);
+    read_qmp_response(sockfd);
+}
+
+static int connect_to_qmp(const char *server, uint16_t port)
+{
+    int sockfd;
+    struct sockaddr_in serv_addr;
+
+    sockfd = socket(AF_INET, SOCK_STREAM, 0);
+    if (sockfd < 0) {
+        return -1;
+    }
+
+    serv_addr.sin_family = AF_INET;
+    serv_addr.sin_port = htons(port);
+
+    if (inet_pton(AF_INET, server, &serv_addr.sin_addr) <= 0) {
+        return -1;
+    }
+
+    if (connect(sockfd, (struct sockaddr *)&serv_addr, sizeof(serv_addr)) < 0) {
+        return -1;
+    }
+
+    send_qmp_cap_command(sockfd);
+
+    return sockfd;
+}
+
+int setup_mctp_qmp_connection(const char *qmp_str)
+{
+    char host[256];
+    uint16_t port;
+
+    memset(host, 0, 256);
+    if (sscanf(qmp_str, "%255[^:]:%hu", host, &port) != 2) {
+        return -1;
+    }
+
+    return connect_to_qmp(host, port);
+}
+
+void qmp_cxl_mctp_process_cci_message(const int sockfd, const char *cci_name)
+{
+    char command[256];
+
+    memset(command, 0, 256);
+    sprintf(command, "{ \"execute\": \"cxl-process-mctp-message\", \
+                             \"arguments\": { \
+                             \"cci-name\": \"%s\" \
+}}\n", cci_name);
+    send_qmp_command(sockfd, command);
+    read_qmp_response(sockfd);
+}
diff --git a/hw/cxl/i2c_mctp_cxl.c b/hw/cxl/i2c_mctp_cxl.c
index 7d2cbc3b75..e146fa2039 100644
--- a/hw/cxl/i2c_mctp_cxl.c
+++ b/hw/cxl/i2c_mctp_cxl.c
@@ -29,6 +29,7 @@
 #include "hw/pci/pcie_port.h"
 #include "hw/qdev-properties.h"
 #include "hw/registerfields.h"
+#include "hw/cxl/cxl_mctp_message.h"
 
 #define TYPE_I2C_MCTP_CXL "i2c_mctp_cxl"
 
@@ -83,6 +84,10 @@ struct I2C_MCTP_CXL_State {
     int64_t pos;
     uint8_t buffer[MCTP_CXL_MAX_MSG_LEN];
     uint8_t scratch[MCTP_CXL_MAX_MSG_LEN];
+    char *qmp_str;
+    int qmp_fd;
+
+    bool mctp_msg_forward;
 };
 
 OBJECT_DECLARE_SIMPLE_TYPE(I2C_MCTP_CXL_State, I2C_MCTP_CXL)
@@ -90,6 +95,9 @@ OBJECT_DECLARE_SIMPLE_TYPE(I2C_MCTP_CXL_State, I2C_MCTP_CXL)
 static const Property i2c_mctp_cxl_props[] = {
     DEFINE_PROP_LINK("target", I2C_MCTP_CXL_State,
                      target, TYPE_PCI_DEVICE, PCIDevice *),
+    DEFINE_PROP_STRING("qmp", I2C_MCTP_CXL_State, qmp_str),
+    DEFINE_PROP_BOOL("mctp-msg-forward", I2C_MCTP_CXL_State,
+                     mctp_msg_forward, false),
 };
 
 static size_t i2c_mctp_cxl_get_buf(MCTPI2CEndpoint *mctp,
@@ -211,12 +219,46 @@ static void i2c_mctp_cxl_handle_message(MCTPI2CEndpoint *mctp)
 
         len_in = msg->pl_length[2] << 16 | msg->pl_length[1] << 8 |
             msg->pl_length[0];
-
-        rc = cxl_process_cci_message(s->cci, msg->command_set, msg->command,
-                                     len_in, msg->payload,
-                                     &len_out,
-                                     s->scratch + sizeof(CXLMCTPMessage),
-                                     &bg_started);
+        if (s->mctp_msg_forward) {
+            CXLType3Dev *ct3d = CXL_TYPE3(s->target);
+            CXLMCTPCommandBuf *mctp_buf;
+            int i;
+            char *cci_name = NULL;
+
+            g_assert(cci_map_buf);
+            g_assert(ct3d->mctp_shared_buffer);
+
+            for (i = 0; i < cci_map_buf->num_mappings; i++) {
+                if (cci_map_buf->maps[i].cci_pointer == s->cci) {
+                    break;
+                }
+            }
+
+            g_assert(i < cci_map_buf->num_mappings);
+            cci_name = cci_map_buf->maps[i].cci_name;
+
+            mctp_buf = &ct3d->mctp_shared_buffer->command_buf;
+            g_assert(mctp_buf);
+
+            mctp_buf->command_set = msg->command_set;
+            mctp_buf->command = msg->command;
+            mctp_buf->len_in = len_in;
+            memcpy(mctp_buf->payload, msg->payload, len_in);
+            ct3d->mctp_shared_buffer->status = 1;
+            qmp_cxl_mctp_process_cci_message(s->qmp_fd, cci_name);
+            if (mctp_buf->len_out) {
+                memcpy(s->scratch + sizeof(CXLMCTPMessage),
+                       mctp_buf->payload_out, mctp_buf->len_out);
+            }
+            rc = mctp_buf->ret_val;
+            len_out = mctp_buf->len_out;
+        } else {
+            rc = cxl_process_cci_message(s->cci, msg->command_set, msg->command,
+                                         len_in, msg->payload,
+                                         &len_out,
+                                         s->scratch + sizeof(CXLMCTPMessage),
+                                         &bg_started);
+        }
         buf->rc = rc;
         s->pos += len_out;
         s->len = s->pos;
@@ -253,6 +295,20 @@ static void i2c_mctp_cxl_realize(DeviceState *d, Error **errp)
 
         cxl_initialize_t3_fm_owned_ld_mctpcci(s->cci, DEVICE(s->target), d,
                                               MCTP_CXL_MAILBOX_BYTES);
+        if (s->mctp_msg_forward) {
+            g_assert(s->qmp_str);
+            s->qmp_fd = setup_mctp_qmp_connection(s->qmp_str);
+            if (s->qmp_fd < 0) {
+                error_setg(errp, "setup connection to qmp server failed");
+            } else {
+                read_qmp_response(s->qmp_fd);
+            }
+
+            init_cci_name_ptr_mapping();
+            ct3_setup_mctp_command_share_buffer(ct3d, false);
+        } else {
+            s->qmp_fd = -1;
+        }
         return;
     }
 
diff --git a/hw/cxl/meson.build b/hw/cxl/meson.build
index e780344d85..d0a8f78462 100644
--- a/hw/cxl/meson.build
+++ b/hw/cxl/meson.build
@@ -13,7 +13,7 @@ system_ss.add(when: 'CONFIG_CXL',
                if_false: files(
                    'cxl-host-stubs.c',
                ))
-system_ss.add(when: 'CONFIG_I2C_MCTP_CXL', if_true: files('i2c_mctp_cxl.c'))
+system_ss.add(when: 'CONFIG_I2C_MCTP_CXL', if_true: files('i2c_mctp_cxl.c', 'cxl-mctp-qmp.c'))
 
 system_ss.add(when: 'CONFIG_ALL', if_true: files('cxl-host-stubs.c'))
 
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index f7ac8e8da7..d44b1b6dc6 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1051,7 +1051,7 @@ static int ct3_mctp_buf_create(const char *filename, size_t size)
     return fd;
 }
 
-static int ct3_setup_mctp_command_share_buffer(CXLType3Dev *ct3d, bool create)
+int ct3_setup_mctp_command_share_buffer(CXLType3Dev *ct3d, bool create)
 {
     int fd;
     int size = sizeof(*ct3d->mctp_shared_buffer);
@@ -1074,7 +1074,7 @@ static int ct3_setup_mctp_command_share_buffer(CXLType3Dev *ct3d, bool create)
     return -1;
 }
 
-static int init_cci_name_ptr_mapping(void)
+int init_cci_name_ptr_mapping(void)
 {
     if (!cci_map_buf) {
         cci_map_buf = g_malloc(sizeof(*cci_map_buf));
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 9a00ef7a1e..5c8813af20 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -810,4 +810,6 @@ void ct3_clear_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
                                    uint64_t len);
 bool ct3_test_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
                                   uint64_t len);
+int ct3_setup_mctp_command_share_buffer(CXLType3Dev *ct3d, bool create);
+int init_cci_name_ptr_mapping(void);
 #endif
diff --git a/include/hw/cxl/cxl_mctp_message.h b/include/hw/cxl/cxl_mctp_message.h
index 85b3664cf7..718dc8c0a2 100644
--- a/include/hw/cxl/cxl_mctp_message.h
+++ b/include/hw/cxl/cxl_mctp_message.h
@@ -37,4 +37,7 @@ struct CXLMCTPSharedBuf {
 
 typedef struct CXLMCTPSharedBuf CXLMCTPSharedBuf;
 extern struct CXLCCINamePtrMaps *cci_map_buf;
+int setup_mctp_qmp_connection(const char *qmp_str);
+void read_qmp_response(int sockfd);
+void qmp_cxl_mctp_process_cci_message(const int sockfd, const char *cci_name);
 #endif
-- 
2.47.2

Re: [RFC 0/3] Qemu FM emulation

[Gregory Price]

On Mon, Apr 07, 2025 at 09:20:27PM -0700, nifan.cxl@gmail.com wrote:
> From: Fan Ni <fan.ni@samsung.com>
> 
> The RFC provides a way for FM emulation in Qemu. The goal is to provide
> a context where we can have more FM emulation discussions and share solutions
> for a reasonable FM implementation in Qemu.
>
... snip ...

Took a browse of the series, and I like this method.  It seems simple
and straight-forward, avoids any complex networking between the vms and
gives us what we want.

I'll wait for Jonathan's commentary, but solid prototype (bn_n)b

~Gregory

Re: [RFC 0/3] Qemu FM emulation

[Fan Ni]

On Tue, Apr 08, 2025 at 11:04:20AM -0400, Gregory Price wrote:
> On Mon, Apr 07, 2025 at 09:20:27PM -0700, nifan.cxl@gmail.com wrote:
> > From: Fan Ni <fan.ni@samsung.com>
> > 
> > The RFC provides a way for FM emulation in Qemu. The goal is to provide
> > a context where we can have more FM emulation discussions and share solutions
> > for a reasonable FM implementation in Qemu.
> >
> ... snip ...
> 
> Took a browse of the series, and I like this method.  It seems simple
> and straight-forward, avoids any complex networking between the vms and
> gives us what we want.
> 
> I'll wait for Jonathan's commentary, but solid prototype (bn_n)b
> 
> ~Gregory

Hi Jonathan,

Any feedback for this RFC?

Fan

Re: [RFC 0/3] Qemu FM emulation

[Jonathan Cameron via]

On Mon, 14 Apr 2025 08:44:07 -0700
Fan Ni <nifan.cxl@gmail.com> wrote:

> On Tue, Apr 08, 2025 at 11:04:20AM -0400, Gregory Price wrote:
> > On Mon, Apr 07, 2025 at 09:20:27PM -0700, nifan.cxl@gmail.com wrote:  
> > > From: Fan Ni <fan.ni@samsung.com>
> > > 
> > > The RFC provides a way for FM emulation in Qemu. The goal is to provide
> > > a context where we can have more FM emulation discussions and share solutions
> > > for a reasonable FM implementation in Qemu.
> > >  
> > ... snip ...
> > 
> > Took a browse of the series, and I like this method.  It seems simple
> > and straight-forward, avoids any complex networking between the vms and
> > gives us what we want.
> > 
> > I'll wait for Jonathan's commentary, but solid prototype (bn_n)b
> > 
> > ~Gregory  
> 
> Hi Jonathan,
> 
> Any feedback for this RFC?

Immediate question is whether anything similar is done in other use cases
in QEMU?   There are vaguely similar things that work via a socket but
I'm not sure the mix of a shared buffer and a qmp based doorbell is done
elsewhere.  There is use of shared memory for inter VM comms but that uses
a socket for it's doorbell / interrupt path, not qmp.
https://www.qemu.org/docs/master/specs/ivshmem-spec.html

So without looking in that much detail yet, I'm not yet convinced this is
preferable to a socket over which we can send the mctp packets.

In general we need to also solve how to upstream the mctp support in
qemu or this is adding yet more stuff to my cxl staging tree.

+CC Markus for QMP part.

https://lore.kernel.org/all/20250408043051.430340-1-nifan.cxl@gmail.com/
is start of thread.
https://lore.kernel.org/all/20250408043051.430340-3-nifan.cxl@gmail.com/
the qmp patch adding what is more or less a doorbell pinged by a device
on a different  QEMU instance.

Jonathan

> 
> Fan