* [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support
@ 2023-07-17 17:16 Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 01/17] hw/pci-bridge/cxl_upstream: Move defintion of device to header Jonathan Cameron via
` (16 more replies)
0 siblings, 17 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
Includes addition of I2C-MCTP CCIs and tunnelling via switch-cci.
Base for this series messy, so I'd suggest the tree at
https://gitlab.com/jic23/qemu cxl-2023-07-17
which includes this series in the middle.
RFC for various reasons:
1 - I have some specification issues to figure out / query.
2 - The I2C MCTP stuff isn't upstream yet, and it may well change.
3 - The MCTP I2C controllers might be a tad controversial as currently
hacked in :)
We have had separate PoCs for the switch CCI and MCTP-I2C, but they
shared very little of the infrastructure with the main CXL mailbox.
When looking at adding tunneling support it became clear this was
unsustainable.
Tunneling provides access the to MCTP CCIs below a given device -
initially providing emulation of a message being sent over the
Switch CCI mailbox (in band PCI mailbox in a BAR) which then routes
it as MCTP over PCIe VDM to the downstream device. In QEMU at least
the downstream device is a Type3 device below a switch downstream port.
Tunnelling is also used to access the various CCIs inside more complex
devices such as Multi Head Devices (not implemented yet!)
The particular test setup I have been using is:
qemu-system-aarch64 -M virt,nvdimm=on,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8 -cpu max -smp 4 \
-kernel Image \
-drive if=none,file=full.qcow2,format=qcow2,id=hd \
-device pcie-root-port,id=root_port1 -device virtio-blk-pci,drive=hd \
-netdev type=user,id=mynet,hostfwd=tcp::5555-:22 \
-qmp tcp:localhost:4445,server=on,wait=off \
-device virtio-net-pci,netdev=mynet,id=bob \
-nographic -no-reboot -append 'earlycon root=/dev/vda2 fsck.mode=skip maxcpus=4 tp_printk' \
-monitor telnet:127.0.0.1:1234,server,nowait -bios QEMU_EFI.fd \
-object memory-backend-ram,size=4G,id=mem0 \
-numa node,nodeid=0,cpus=0-3,memdev=mem0 \
-object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/t3_cxl1.raw,size=256M,align=256M \
-object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/t3_lsa1.raw,size=1M,align=1M \
-object memory-backend-file,id=cxl-mem2,share=on,mem-path=/tmp/t3_cxl2.raw,size=256M,align=256M \
-object memory-backend-file,id=cxl-lsa2,share=on,mem-path=/tmp/t3_lsa2.raw,size=1M,align=1M \
-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
-device cxl-rp,port=0,bus=cxl.1,id=cxl_rp_port0,chassis=0,slot=2 \
-device cxl-upstream,bus=cxl_rp_port0,id=us0,addr=0.0,multifunction=on, \
-device cxl-switch-mailbox-cci,bus=cxl_rp_port0,addr=0.1,target=us0 \
-device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \
-device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \
-device cxl-type3,bus=swport0,memdev=cxl-mem1,id=cxl-pmem1,lsa=cxl-lsa1,sn=3 \
-device cxl-type3,bus=swport1,memdev=cxl-mem2,id=cxl-pmem2,lsa=cxl-lsa2,sn=4 \
-machine cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=1k \
-device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=4,target=us0 \
-device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=5,target=cxl-pmem1 \
-device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=6,target=cxl-pmem2
Kernel side needs switch-cci driver support
https://lore.kernel.org/linux-cxl/20230717162557.8625-1-Jonathan.Cameron@huawei.com/T/#t
A test program to poke the switch-cci is in the cover letter for that series.
For the MCTP devices:
For ACPI you need the kernel patches from:
https://lore.kernel.org/linux-cxl/20230525152203.32190-1-Jonathan.Cameron@huawei.com/T/#u
To make the aspeed i2c controller play nicely with PRP0001 based ACPI description.
Install the MCTP daemon from:
https://github.com/CodeConstruct/mctp
Bring up and enumerate the MCTP EPs:
# Bring up the link
mctp link set mctpi2c0 up
# Assign an address to the aspeed-i2c controller
mctp addr add 50 dev mctpi2c0
# Assign a neetwork ID to the link (11)
mctp link set mctpi2c0 net 11
# Start the daemon that uses dbus for configuration.
systemctl start mctpd.service
# Assign EIDs to the EPs
busctl call xyz.openbmc_project.MCTP /xyz/openbmc_project/mctp au.com.CodeConstruct.MCTP AssignEndpoint say mctpi2c0 1 0x5
busctl introspect xyz.openbmc_project.MCTP /xyz/openbmc_project/mctp/11/8 xyz.openbmc_project.MCTP.Endpoint
busctl call xyz.openbmc_project.MCTP /xyz/openbmc_project/mctp au.com.CodeConstruct.MCTP AssignEndpoint say mctpi2c0 1 0x6
busctl introspect xyz.openbmc_project.MCTP /xyz/openbmc_project/mctp/11/9 xyz.openbmc_project.MCTP.Endpoint
busctl call xyz.openbmc_project.MCTP /xyz/openbmc_project/mctp au.com.CodeConstruct.MCTP AssignEndpoint say mctpi2c0 1 0x4
busctl introspect xyz.openbmc_project.MCTP /xyz/openbmc_project/mctp/11/10 xyz.openbmc_project.MCTP.Endpoint
Suitable test program:
/*
* Trivial example program to exercise QEMU FMAPI Emulation over MCTP over I2C
*/
#include <sys/socket.h>
#include <stdio.h>
#include <string.h>
#include <stdint.h>
#include <stdlib.h>
#include "mctp.h"
#include "cxl_fmapi.h"
enum cxl_type {
cxl_switch,
cxl_type3,
};
static int query_cci_identify(int sd, struct sockaddr_mctp *addr, int *tag, enum cxl_type *type)
{
struct cxl_cci_infostat_identify_resp_pl *pl;
uint8_t buf[1024];
struct cxl_fmapi_header *rsp_head = buf;
int rc;
ssize_t len;
struct sockaddr_mctp addrrx;
socklen_t addrlen = sizeof(addrrx);
struct cxl_fmapi_header req = {
.category = CXL_MCTP_CATEGORY_REQ,
.tag = *tag++,
.command = 1,
.command_set = 0,
.vendor_ext_status = 0xabcd,
};
printf("trying to identify\n");
len = sendto(sd, &req, sizeof(req), 0, (struct sockaddr *)addr, sizeof(*addr));
if (len != sizeof(req)) {
printf("Failed to send whole request for phys switch info %d %d\n", len, sizeof(req));
return -1;
}
len = recvfrom(sd, buf, sizeof(buf), 0, (struct sockaddr *)&addrrx, &addrlen);
if (len < 0) {
printf("Failed to receive response\n");
return -1;
}
printf("length %d\n", len);
pl = (void *)(rsp_head + 1);
printf("Vendor id : %04x\n", pl->vendor_id);
printf("Device id : %04x\n", pl->device_id);
printf("Subsys vid : %04x\n", pl->subsys_vendor_id);
printf("Subsys id : %04x\n", pl->subsys_id);
switch (pl->component_type) {
case 0x00:
printf("Switch!\n");
*type = cxl_switch;
break;
case 0x03:
printf("Type3!\n");
*type = cxl_type3;
break;
}
return 0;
}
static int parse_physical_switch_identify_switch_device(void *buf, size_t buf_len)
{
struct cxl_fmapi_header *rsp_head = buf;
struct cxl_fmapi_ident_switch_dev_resp_pl *pl = (void *)(rsp_head + 1);
uint8_t *b;
if (rsp_head->return_code != 0) {
printf("Error code in response %d\n", rsp_head->return_code);
printf("%x %x %x %x %x %x %x %x %x %x %x %x\n", ((uint8_t *)rsp_head)[0],((uint8_t *)rsp_head)[1],
((uint8_t *)rsp_head)[2],((uint8_t *)rsp_head)[3],
((uint8_t *)rsp_head)[4],((uint8_t *)rsp_head)[5],((uint8_t *)rsp_head)[6],
((uint8_t *)rsp_head)[7], ((uint8_t *)rsp_head)[8], ((uint8_t *)rsp_head)[9],
((uint8_t *)rsp_head)[10], ((uint8_t *)rsp_head)[11]);
return -1;
}
printf("Num total vppb %d\n", pl->num_total_vppb);
printf("Ports %d\n", pl->num_physical_ports);
b = pl->active_port_bitmask;
printf("ActivePortMask");
for (int i = 0; i < 32; i++)
printf("%02x", b[i]);
printf("\n");
return 0;
}
int query_physical_switch_info(int sd, struct sockaddr_mctp *addr, int *tag)
{
uint8_t buf[1024];
int rc;
ssize_t len;
struct sockaddr_mctp addrrx;
socklen_t addrlen = sizeof(addrrx);
struct cxl_fmapi_header req = {
.category = CXL_MCTP_CATEGORY_REQ,
.tag = *tag++,
.command = CXL_IDENTIFY_SWITCH_DEVICE,
.command_set = CXL_FM_API_CMD_SET_PHYSICAL_SWITCH,
.vendor_ext_status = 0xabcd,
};
len = sendto(sd, &req, sizeof(req), 0, (struct sockaddr *)addr, sizeof(*addr));
if (len != sizeof(req)) {
printf("Failed to send whole request for phys switch info %d %d\n", len, sizeof(req));
return -1;
}
len = recvfrom(sd, buf, sizeof(buf), 0, (struct sockaddr *)&addrrx, &addrlen);
if (len < 0) {
printf("Failed to receive response\n");
return -1;
}
printf("length %d\n", len);
rc = parse_physical_switch_identify_switch_device(buf, len);
if (rc)
return -1;
return 0;
}
static int parse_phys_port_state_rsp(void * buf, size_t buf_len, struct cxl_fmapi_header *head)
{
struct cxl_fmapi_header *rsp_head = buf;
struct cxl_fmapi_get_phys_port_state_resp_pl *pl = (void *)(rsp_head + 1);
uint32_t pl_length = rsp_head->pl_length[0] |
(rsp_head->pl_length[1] << 8) |
((rsp_head->pl_length[2] & 0xf) << 16);
if (rsp_head->category != CXL_MCTP_CATEGORY_RESP) {
printf("Message not a response\n");
return -1;
}
if (rsp_head->tag != head->tag) {
printf("Reply has wrong tag\n");
return -1;
}
if ((rsp_head->command != head->command) ||
(rsp_head->command_set != head->command_set)) {
printf("Response to wrong command\n");
return -1;
}
if (rsp_head->return_code != 0) {
printf("Error code in response %d\n", rsp_head->return_code);
printf("%x %x %x %x %x %x %x %x %x %x %x %x\n", ((uint8_t *)rsp_head)[0],((uint8_t *)rsp_head)[1],
((uint8_t *)rsp_head)[2],((uint8_t *)rsp_head)[3],
((uint8_t *)rsp_head)[4],((uint8_t *)rsp_head)[5],((uint8_t *)rsp_head)[6],
((uint8_t *)rsp_head)[7], ((uint8_t *)rsp_head)[8], ((uint8_t *)rsp_head)[9],
((uint8_t *)rsp_head)[10], ((uint8_t *)rsp_head)[11]);
return -1;
}
if (pl_length < 4 || pl_length < (pl->num_ports * sizeof(pl->ports[0]))) {
printf("too short %d %d %d\n", pl->num_ports, pl_length, pl->num_ports * sizeof(pl->ports[0]));
return -1;
}
for (int i = 0; i < pl->num_ports; i++) {
struct cxl_fmapi_port_state_info_block *port = &pl->ports[i];
const char *port_states[] = {
[0x0] = "Disabled",
[0x1] = "Bind in progress",
[0x2] = "Unbind in progress",
[0x3] = "DSP",
[0x4] = "USP",
[0x5] = "Reserved",
//other values not present.
[0xf] = "Invalid Port ID"
};
const char *connected_device_modes[] = {
[0] = "Not CXL / connected",
[1] = "CXL 1.1",
[2] = "CXL 2.0",
};
const char *connected_device_type[] = {
[0] = "No device detected",
[1] = "PCIe device",
[2] = "CXL type 1 device",
[3] = "CXL type 2 device",
[4] = "CXL type 3 device",
[5] = "CXL type 3 pooled device",
[6] = "Reserved",
};
const char *ltssm_states[] = {
[0] = "Detect",
[1] = "Polling",
[2] = "Configuration",
[3] = "Recovery",
[4] = "L0",
[5] = "L0s",
[6] = "L1",
[7] = "L2",
[8] = "Disabled",
[9] = "Loop Back",
[10] = "Hot Reset",
};
printf("Port%02d:\n ", port->port_id);
printf("\tPort state: ");
if (port_states[port->config_state & 0xf])
printf("%s\n", port_states[port->config_state]);
else
printf("Unknown state\n");
if (port->config_state == 3) { /* DSP so device could be there */
printf("\tConnected Device CXL Version: ");
if (port->connected_device_cxl_version > 2)
printf("Unknown CXL Version\n");
else
printf("%s\n", connected_device_modes[port->connected_device_cxl_version]);
printf("\tConnected Device Type: ");
if (port->connected_device_type > 7)
printf("Unknown\n");
else
printf("%s\n", connected_device_type[port->connected_device_type]);
}
printf("\tSupported CXL Modes:");
if (port->port_cxl_version_bitmask & 0x1)
printf(" 1.1");
if (port->port_cxl_version_bitmask & 0x2)
printf(" 2.0");
printf("\n");
printf("\tMaximum Link Width: %d Negotiated Width %d\n",
port->max_link_width,
port->negotiated_link_width);
printf("\tSupported Speeds: ");
if (port->supported_link_speeds_vector & 0x1)
printf(" 2.5 GT/s");
if (port->supported_link_speeds_vector & 0x2)
printf(" 5.0 GT/s");
if (port->supported_link_speeds_vector & 0x4)
printf(" 8.0 GT/s");
if (port->supported_link_speeds_vector & 0x8)
printf(" 16.0 GT/s");
if (port->supported_link_speeds_vector & 0x10)
printf(" 32.0 GT/s");
if (port->supported_link_speeds_vector & 0x20)
printf(" 64.0 GT/s");
printf("\n");
printf("\tLTSSM: ");
if (port->ltssm_state >= sizeof(ltssm_states))
printf("Unkown\n");
else
printf("%s\n", ltssm_states[port->ltssm_state]);
}
}
int query_ports(int sd, struct sockaddr_mctp *addr, int *tag)
{
uint8_t buf[1024];
ssize_t len;
int num_ports = 4;
int rc;
uint8_t port_list[4] = { 0, 3, 7, 4 };
struct sockaddr_mctp addrrx;
socklen_t addrlen = sizeof(addrrx);
struct cxl_fmapi_header *head;
struct cxl_fmapi_get_phys_port_state_req_pl *reqpl;
size_t req_sz = sizeof(*reqpl) + num_ports + sizeof(*head) ;
head = malloc(req_sz);
*head = (struct cxl_fmapi_header) {
.category = CXL_MCTP_CATEGORY_REQ,
.tag = *tag++,
.command = CXL_GET_PHYSICAL_PORT_STATE,
.command_set = CXL_FM_API_CMD_SET_PHYSICAL_SWITCH,
.pl_length = {
req_sz & 0xff,
(req_sz >> 8) & 0xff,
(req_sz >> 16) & 0xf },
.vendor_ext_status = 0x1234,
};
reqpl = (void *)(head + 1);
*reqpl = (struct cxl_fmapi_get_phys_port_state_req_pl) {
.num_ports = num_ports,
};
for (int j = 0; j < num_ports; j++)
reqpl->ports[j] = port_list[j];
len = sendto(sd, head, req_sz, 0,
(struct sockaddr *)addr, sizeof(*addr));
len = recvfrom(sd, buf, sizeof(buf), 0,
(struct sockaddr *)&addrrx, &addrlen);
printf("got back %d\n", len);
if (len < sizeof(struct cxl_fmapi_header)) {
printf("Too short for header\n");
}
//TODO generic check of reply.
if (addrrx.smctp_type != 0x7) {
printf("Reply does not match expected message type %x\n", addrrx.smctp_type);
}
rc = parse_phys_port_state_rsp(buf, len, head);
if (rc)
return rc;
return 0;
}
int main(int argv, char **argc)
{
int rc, cci_sd, fmapi_sd;
int tag = 0; /* will increment on each use */
ssize_t len;
if (argv < 2) {
printf("Give an address\n");
return -1;
}
int dev_addr = atoi(argc[1]);
struct sockaddr_mctp cci_addr = {
.smctp_family = AF_MCTP,
.smctp_network = 11,
.smctp_addr.s_addr = dev_addr,
.smctp_type = 0x8, /* CXL CCI */
.smctp_tag = MCTP_TAG_OWNER,
};
struct sockaddr_mctp fmapi_addr = {
.smctp_family = AF_MCTP,
.smctp_network = 11,
.smctp_addr.s_addr = dev_addr,
.smctp_type = 0x7, /* CXL FMAPI */
.smctp_tag = MCTP_TAG_OWNER,
};
struct sockaddr_mctp addrrx;
socklen_t addrlen = sizeof(addrrx);
enum cxl_type type;
cci_sd = socket(AF_MCTP, SOCK_DGRAM, 0);
rc = bind(cci_sd, (struct sockaddr *)&cci_addr, sizeof(cci_addr));
if (rc) {
printf("Bind failed\n");
return -1;
}
printf("first query\n");
rc = query_cci_identify(cci_sd, &cci_addr, &tag, &type);
if (rc)
return rc;
if (type == cxl_switch) {
fmapi_sd = socket(AF_MCTP, SOCK_DGRAM, 0);
rc = bind(fmapi_sd, (struct sockaddr *)&fmapi_addr, sizeof(fmapi_addr));
if (rc) {
printf("Bind failed\n");
return -1;
}
rc = query_physical_switch_info(fmapi_sd, &fmapi_addr, &tag);
if (rc)
return rc;
/* Next query some of the ports */
rc = query_ports(fmapi_sd, &fmapi_addr, &tag);
if (rc)
return rc;
}
return 0;
}
Jonathan Cameron (15):
hw/pci-bridge/cxl_upstream: Move defintion of device to header.
hw/cxl/mailbox: Enable mulitple mailbox command sets
cxl/mbox: Pull the payload out of struct cxl_cmd and make instances
constant
hw/mbox: Split mailbox command payload into separate input and output
cxl/mbox: Pull the CCI definition out of the CXLDeviceState
cxl/mbox: Generalize the CCI command processing
hw/acpi/aml-build: add function for i2c slave device serial bus
description
misc/i2c_mctp_cxl: Initial device emulation
HACK: arm/virt: Add aspeed-i2c controller and MCTP EP to enable MCTP
testing
HACK: hw/arm/virt: Add ACPI support for aspeed-i2c / mctp
HACK: hw/i386/pc: Add Aspeed i2c controller + MCTP with ACPI tables
docs: cxl: Add example commandline for MCTP CXL CCIs
hw/cxl: Add a switch mailbox CCI function.
hw/cxl: Implement Physical Ports status retrieval
hw/cxl: Add tunneled command support to mailbox for switch cci.
Klaus Jensen (1):
hw/i2c: add mctp core
Matt Johnston (1):
i2c/mctp: Allow receiving messages to dest eid 0
MAINTAINERS | 7 +
docs/system/devices/cxl.rst | 27 +
include/hw/acpi/aml-build.h | 1 +
include/hw/arm/virt.h | 2 +
include/hw/cxl/cxl.h | 6 +
include/hw/cxl/cxl_component.h | 3 +-
include/hw/cxl/cxl_device.h | 81 ++-
include/hw/i2c/mctp.h | 114 ++++
include/hw/i2c/smbus_master.h | 3 +
include/hw/i386/pc.h | 1 +
include/hw/pci-bridge/cxl_upstream_port.h | 20 +
include/net/mctp.h | 43 ++
hw/acpi/aml-build.c | 17 +
hw/arm/virt-acpi-build.c | 60 ++
hw/arm/virt.c | 86 +++
hw/cxl/cxl-device-utils.c | 97 ++-
hw/cxl/cxl-events.c | 2 +-
hw/cxl/cxl-mailbox-utils.c | 715 +++++++++++++++++-----
hw/cxl/i2c_mctp_cxl.c | 237 +++++++
hw/cxl/switch-mailbox-cci.c | 98 +++
hw/i2c/mctp.c | 353 +++++++++++
hw/i2c/smbus_master.c | 28 +
hw/i386/acpi-build.c | 65 ++
hw/i386/pc.c | 20 +-
hw/mem/cxl_type3.c | 9 +-
hw/pci-bridge/cxl_downstream.c | 5 +-
hw/pci-bridge/cxl_upstream.c | 11 +-
hw/arm/Kconfig | 3 +
hw/cxl/Kconfig | 3 +
hw/cxl/meson.build | 2 +
hw/i2c/Kconfig | 4 +
hw/i2c/meson.build | 3 +-
hw/i2c/trace-events | 12 +
hw/i386/Kconfig | 2 +
34 files changed, 1956 insertions(+), 184 deletions(-)
create mode 100644 include/hw/i2c/mctp.h
create mode 100644 include/hw/pci-bridge/cxl_upstream_port.h
create mode 100644 include/net/mctp.h
create mode 100644 hw/cxl/i2c_mctp_cxl.c
create mode 100644 hw/cxl/switch-mailbox-cci.c
create mode 100644 hw/i2c/mctp.c
--
2.39.2
^ permalink raw reply [flat|nested] 23+ messages in thread
* [RFC PATCH 01/17] hw/pci-bridge/cxl_upstream: Move defintion of device to header.
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 02/17] hw/cxl/mailbox: Enable mulitple mailbox command sets Jonathan Cameron via
` (15 subsequent siblings)
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
To avoid repitition of switch upstream port specific data in
the CXLDeviceState structure it will be necessary to call
access the switch USP specific from mailbox callbacks.
Hence move it to a header so it is no longer an opaque
structure.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/pci-bridge/cxl_upstream_port.h | 18 ++++++++++++++++++
hw/cxl/cxl-mailbox-utils.c | 1 +
hw/pci-bridge/cxl_upstream.c | 11 +----------
3 files changed, 20 insertions(+), 10 deletions(-)
diff --git a/include/hw/pci-bridge/cxl_upstream_port.h b/include/hw/pci-bridge/cxl_upstream_port.h
new file mode 100644
index 0000000000..b02aa8f659
--- /dev/null
+++ b/include/hw/pci-bridge/cxl_upstream_port.h
@@ -0,0 +1,18 @@
+
+#ifndef CXL_USP_H
+#define CXL_USP_H
+#include "hw/pci/pcie.h"
+#include "hw/pci/pcie_port.h"
+#include "hw/cxl/cxl.h"
+
+typedef struct CXLUpstreamPort {
+ /*< private >*/
+ PCIEPort parent_obj;
+
+ /*< public >*/
+ CXLComponentState cxl_cstate;
+ DOECap doe_cdat;
+ uint64_t sn;
+} CXLUpstreamPort;
+
+#endif /* CXL_SUP_H */
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 02f9b5a870..1e0d96f64f 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -11,6 +11,7 @@
#include "hw/cxl/cxl.h"
#include "hw/cxl/cxl_events.h"
#include "hw/pci/pci.h"
+#include "hw/pci-bridge/cxl_upstream_port.h"
#include "qemu/cutils.h"
#include "qemu/log.h"
#include "qemu/units.h"
diff --git a/hw/pci-bridge/cxl_upstream.c b/hw/pci-bridge/cxl_upstream.c
index 96d093add1..f9627f820e 100644
--- a/hw/pci-bridge/cxl_upstream.c
+++ b/hw/pci-bridge/cxl_upstream.c
@@ -14,6 +14,7 @@
#include "hw/pci/msi.h"
#include "hw/pci/pcie.h"
#include "hw/pci/pcie_port.h"
+#include "hw/pci-bridge/cxl_upstream_port.h"
/*
* Null value of all Fs suggested by IEEE RA guidelines for use of
* EU, OUI and CID
@@ -28,16 +29,6 @@
#define CXL_UPSTREAM_PORT_DVSEC_OFFSET \
(CXL_UPSTREAM_PORT_AER_OFFSET + PCI_ERR_SIZEOF)
-typedef struct CXLUpstreamPort {
- /*< private >*/
- PCIEPort parent_obj;
-
- /*< public >*/
- CXLComponentState cxl_cstate;
- DOECap doe_cdat;
- uint64_t sn;
-} CXLUpstreamPort;
-
CXLComponentState *cxl_usp_to_cstate(CXLUpstreamPort *usp)
{
return &usp->cxl_cstate;
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 02/17] hw/cxl/mailbox: Enable mulitple mailbox command sets
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 01/17] hw/pci-bridge/cxl_upstream: Move defintion of device to header Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 03/17] cxl/mbox: Pull the payload out of struct cxl_cmd and make instances constant Jonathan Cameron via
` (14 subsequent siblings)
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
Until now, we have supported only a single set of comamnds.
To allow introduction of switch CCI functions, we need to
be able to pick between different sets for a given mailbox
instance. This patch should make not functional changes, but
enable them in the following patches.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/cxl/cxl_device.h | 12 ++++++++++++
hw/cxl/cxl-mailbox-utils.c | 17 ++++-------------
2 files changed, 16 insertions(+), 13 deletions(-)
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 887a38bdbc..2c239fca47 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -134,6 +134,17 @@ typedef enum {
CXL_MBOX_MAX = 0x17
} CXLRetCode;
+struct cxl_cmd;
+typedef CXLRetCode (*opcode_handler)(struct cxl_cmd *cmd,
+ CXLDeviceState *cxl_dstate, uint16_t *len);
+struct cxl_cmd {
+ const char *name;
+ opcode_handler handler;
+ ssize_t in;
+ uint16_t effect; /* Reported in CEL */
+ uint8_t *payload;
+};
+
typedef struct CXLEvent {
CXLEventRecordRaw data;
QSIMPLEQ_ENTRY(CXLEvent) node;
@@ -202,6 +213,7 @@ typedef struct cxl_device_state {
uint64_t pmem_size;
uint64_t vmem_size;
+ struct cxl_cmd (*cxl_cmd_set)[256];
CPMUState cpmu[CXL_NUM_CPMU_INSTANCES];
CXLEventLog event_logs[CXL_EVENT_TYPE_MAX];
} CXLDeviceState;
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 1e0d96f64f..c8feeffbeb 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -70,16 +70,6 @@ enum {
#define CLEAR_POISON 0x2
};
-struct cxl_cmd;
-typedef CXLRetCode (*opcode_handler)(struct cxl_cmd *cmd,
- CXLDeviceState *cxl_dstate, uint16_t *len);
-struct cxl_cmd {
- const char *name;
- opcode_handler handler;
- ssize_t in;
- uint16_t effect; /* Reported in CEL */
- uint8_t *payload;
-};
static CXLRetCode cmd_events_get_records(struct cxl_cmd *cmd,
CXLDeviceState *cxlds,
@@ -711,7 +701,7 @@ void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
uint8_t set = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND_SET);
uint8_t cmd = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND);
uint16_t len = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, LENGTH);
- cxl_cmd = &cxl_cmd_set[set][cmd];
+ cxl_cmd = &cxl_dstate->cxl_cmd_set[set][cmd];
h = cxl_cmd->handler;
if (h) {
if (len == cxl_cmd->in || cxl_cmd->in == ~0) {
@@ -746,10 +736,11 @@ void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate)
{
+ cxl_dstate->cxl_cmd_set = cxl_cmd_set;
for (int set = 0; set < 256; set++) {
for (int cmd = 0; cmd < 256; cmd++) {
- if (cxl_cmd_set[set][cmd].handler) {
- struct cxl_cmd *c = &cxl_cmd_set[set][cmd];
+ if (cxl_dstate->cxl_cmd_set[set][cmd].handler) {
+ struct cxl_cmd *c = &cxl_dstate->cxl_cmd_set[set][cmd];
struct cel_log *log =
&cxl_dstate->cel_log[cxl_dstate->cel_size];
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 03/17] cxl/mbox: Pull the payload out of struct cxl_cmd and make instances constant
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 01/17] hw/pci-bridge/cxl_upstream: Move defintion of device to header Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 02/17] hw/cxl/mailbox: Enable mulitple mailbox command sets Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 04/17] hw/mbox: Split mailbox command payload into separate input and output Jonathan Cameron via
` (13 subsequent siblings)
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
Putting the pointer in the structure for command handling puts
a single variable element inside an otherwise constant structure.
Move it out as a directly passed variable and take the cxl_cmd
structures constant.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/cxl/cxl_device.h | 6 +-
hw/cxl/cxl-mailbox-utils.c | 106 +++++++++++++++++++++---------------
2 files changed, 65 insertions(+), 47 deletions(-)
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 2c239fca47..f84f6813aa 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -135,14 +135,14 @@ typedef enum {
} CXLRetCode;
struct cxl_cmd;
-typedef CXLRetCode (*opcode_handler)(struct cxl_cmd *cmd,
+typedef CXLRetCode (*opcode_handler)(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxl_dstate, uint16_t *len);
struct cxl_cmd {
const char *name;
opcode_handler handler;
ssize_t in;
uint16_t effect; /* Reported in CEL */
- uint8_t *payload;
};
typedef struct CXLEvent {
@@ -213,7 +213,7 @@ typedef struct cxl_device_state {
uint64_t pmem_size;
uint64_t vmem_size;
- struct cxl_cmd (*cxl_cmd_set)[256];
+ const struct cxl_cmd (*cxl_cmd_set)[256];
CPMUState cpmu[CXL_NUM_CPMU_INSTANCES];
CXLEventLog event_logs[CXL_EVENT_TYPE_MAX];
} CXLDeviceState;
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index c8feeffbeb..f0d94b9ae4 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -71,7 +71,8 @@ enum {
};
-static CXLRetCode cmd_events_get_records(struct cxl_cmd *cmd,
+static CXLRetCode cmd_events_get_records(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxlds,
uint16_t *len)
{
@@ -83,9 +84,9 @@ static CXLRetCode cmd_events_get_records(struct cxl_cmd *cmd,
return CXL_MBOX_INVALID_INPUT;
}
- log_type = *((uint8_t *)cmd->payload);
+ log_type = payload[0];
- pl = (CXLGetEventPayload *)cmd->payload;
+ pl = (CXLGetEventPayload *)payload;
memset(pl, 0, sizeof(*pl));
max_recs = (cxlds->payload_size - CXL_EVENT_PAYLOAD_HDR_SIZE) /
@@ -97,25 +98,27 @@ static CXLRetCode cmd_events_get_records(struct cxl_cmd *cmd,
return cxl_event_get_records(cxlds, pl, log_type, max_recs, len);
}
-static CXLRetCode cmd_events_clear_records(struct cxl_cmd *cmd,
+static CXLRetCode cmd_events_clear_records(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxlds,
uint16_t *len)
{
CXLClearEventPayload *pl;
- pl = (CXLClearEventPayload *)cmd->payload;
+ pl = (CXLClearEventPayload *)payload;
*len = 0;
return cxl_event_clear_records(cxlds, pl);
}
-static CXLRetCode cmd_events_get_interrupt_policy(struct cxl_cmd *cmd,
+static CXLRetCode cmd_events_get_interrupt_policy(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxlds,
uint16_t *len)
{
CXLEventInterruptPolicy *policy;
CXLEventLog *log;
- policy = (CXLEventInterruptPolicy *)cmd->payload;
+ policy = (CXLEventInterruptPolicy *)payload;
memset(policy, 0, sizeof(*policy));
log = &cxlds->event_logs[CXL_EVENT_TYPE_INFO];
@@ -148,7 +151,8 @@ static CXLRetCode cmd_events_get_interrupt_policy(struct cxl_cmd *cmd,
return CXL_MBOX_SUCCESS;
}
-static CXLRetCode cmd_events_set_interrupt_policy(struct cxl_cmd *cmd,
+static CXLRetCode cmd_events_set_interrupt_policy(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxlds,
uint16_t *len)
{
@@ -159,7 +163,7 @@ static CXLRetCode cmd_events_set_interrupt_policy(struct cxl_cmd *cmd,
return CXL_MBOX_INVALID_PAYLOAD_LENGTH;
}
- policy = (CXLEventInterruptPolicy *)cmd->payload;
+ policy = (CXLEventInterruptPolicy *)payload;
log = &cxlds->event_logs[CXL_EVENT_TYPE_INFO];
log->irq_enabled = (policy->info_settings & CXL_EVENT_INT_MODE_MASK) ==
@@ -191,7 +195,8 @@ static CXLRetCode cmd_events_set_interrupt_policy(struct cxl_cmd *cmd,
}
/* 8.2.9.2.1 */
-static CXLRetCode cmd_firmware_update_get_info(struct cxl_cmd *cmd,
+static CXLRetCode cmd_firmware_update_get_info(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxl_dstate,
uint16_t *len)
{
@@ -212,7 +217,7 @@ static CXLRetCode cmd_firmware_update_get_info(struct cxl_cmd *cmd,
return CXL_MBOX_INTERNAL_ERROR;
}
- fw_info = (void *)cmd->payload;
+ fw_info = (void *)payload;
memset(fw_info, 0, sizeof(*fw_info));
fw_info->slots_supported = 2;
@@ -225,27 +230,29 @@ static CXLRetCode cmd_firmware_update_get_info(struct cxl_cmd *cmd,
}
/* 8.2.9.3.1 */
-static CXLRetCode cmd_timestamp_get(struct cxl_cmd *cmd,
+static CXLRetCode cmd_timestamp_get(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxl_dstate,
uint16_t *len)
{
uint64_t final_time = cxl_device_get_timestamp(cxl_dstate);
- stq_le_p(cmd->payload, final_time);
+ stq_le_p(payload, final_time);
*len = 8;
return CXL_MBOX_SUCCESS;
}
/* 8.2.9.3.2 */
-static CXLRetCode cmd_timestamp_set(struct cxl_cmd *cmd,
- CXLDeviceState *cxl_dstate,
- uint16_t *len)
+static CXLRetCode cmd_timestamp_set(const struct cxl_cmd *cmd,
+ uint8_t *payload,
+ CXLDeviceState *cxl_dstate,
+ uint16_t *len)
{
cxl_dstate->timestamp.set = true;
cxl_dstate->timestamp.last_set = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
- cxl_dstate->timestamp.host_set = le64_to_cpu(*(uint64_t *)cmd->payload);
+ cxl_dstate->timestamp.host_set = le64_to_cpu(*(uint64_t *)payload);
*len = 0;
return CXL_MBOX_SUCCESS;
@@ -258,7 +265,8 @@ static const QemuUUID cel_uuid = {
};
/* 8.2.9.4.1 */
-static CXLRetCode cmd_logs_get_supported(struct cxl_cmd *cmd,
+static CXLRetCode cmd_logs_get_supported(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxl_dstate,
uint16_t *len)
{
@@ -269,7 +277,7 @@ static CXLRetCode cmd_logs_get_supported(struct cxl_cmd *cmd,
QemuUUID uuid;
uint32_t size;
} log_entries[1];
- } QEMU_PACKED *supported_logs = (void *)cmd->payload;
+ } QEMU_PACKED *supported_logs = (void *)payload;
QEMU_BUILD_BUG_ON(sizeof(*supported_logs) != 0x1c);
supported_logs->entries = 1;
@@ -281,7 +289,8 @@ static CXLRetCode cmd_logs_get_supported(struct cxl_cmd *cmd,
}
/* 8.2.9.4.2 */
-static CXLRetCode cmd_logs_get_log(struct cxl_cmd *cmd,
+static CXLRetCode cmd_logs_get_log(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxl_dstate,
uint16_t *len)
{
@@ -289,7 +298,9 @@ static CXLRetCode cmd_logs_get_log(struct cxl_cmd *cmd,
QemuUUID uuid;
uint32_t offset;
uint32_t length;
- } QEMU_PACKED QEMU_ALIGNED(16) *get_log = (void *)cmd->payload;
+ } QEMU_PACKED QEMU_ALIGNED(16) *get_log;
+
+ get_log = (void *)payload;
/*
* 8.2.9.4.2
@@ -315,14 +326,15 @@ static CXLRetCode cmd_logs_get_log(struct cxl_cmd *cmd,
/* Store off everything to local variables so we can wipe out the payload */
*len = get_log->length;
- memmove(cmd->payload, cxl_dstate->cel_log + get_log->offset,
+ memmove(payload, cxl_dstate->cel_log + get_log->offset,
get_log->length);
return CXL_MBOX_SUCCESS;
}
/* 8.2.9.5.1.1 */
-static CXLRetCode cmd_identify_memory_device(struct cxl_cmd *cmd,
+static CXLRetCode cmd_identify_memory_device(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxl_dstate,
uint16_t *len)
{
@@ -352,7 +364,7 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd *cmd,
return CXL_MBOX_INTERNAL_ERROR;
}
- id = (void *)cmd->payload;
+ id = (void *)payload;
memset(id, 0, sizeof(*id));
snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0);
@@ -370,7 +382,8 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd *cmd,
return CXL_MBOX_SUCCESS;
}
-static CXLRetCode cmd_ccls_get_partition_info(struct cxl_cmd *cmd,
+static CXLRetCode cmd_ccls_get_partition_info(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxl_dstate,
uint16_t *len)
{
@@ -379,7 +392,7 @@ static CXLRetCode cmd_ccls_get_partition_info(struct cxl_cmd *cmd,
uint64_t active_pmem;
uint64_t next_vmem;
uint64_t next_pmem;
- } QEMU_PACKED *part_info = (void *)cmd->payload;
+ } QEMU_PACKED *part_info = (void *)payload;
QEMU_BUILD_BUG_ON(sizeof(*part_info) != 0x20);
if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER)) ||
@@ -400,7 +413,8 @@ static CXLRetCode cmd_ccls_get_partition_info(struct cxl_cmd *cmd,
return CXL_MBOX_SUCCESS;
}
-static CXLRetCode cmd_ccls_get_lsa(struct cxl_cmd *cmd,
+static CXLRetCode cmd_ccls_get_lsa(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxl_dstate,
uint16_t *len)
{
@@ -412,7 +426,7 @@ static CXLRetCode cmd_ccls_get_lsa(struct cxl_cmd *cmd,
CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
uint32_t offset, length;
- get_lsa = (void *)cmd->payload;
+ get_lsa = (void *)payload;
offset = get_lsa->offset;
length = get_lsa->length;
@@ -425,7 +439,8 @@ static CXLRetCode cmd_ccls_get_lsa(struct cxl_cmd *cmd,
return CXL_MBOX_SUCCESS;
}
-static CXLRetCode cmd_ccls_set_lsa(struct cxl_cmd *cmd,
+static CXLRetCode cmd_ccls_set_lsa(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxl_dstate,
uint16_t *len)
{
@@ -434,7 +449,7 @@ static CXLRetCode cmd_ccls_set_lsa(struct cxl_cmd *cmd,
uint32_t rsvd;
uint8_t data[];
} QEMU_PACKED;
- struct set_lsa_pl *set_lsa_payload = (void *)cmd->payload;
+ struct set_lsa_pl *set_lsa_payload = (void *)payload;
CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
const size_t hdr_len = offsetof(struct set_lsa_pl, data);
@@ -460,7 +475,8 @@ static CXLRetCode cmd_ccls_set_lsa(struct cxl_cmd *cmd,
* make this stateful. We may want to allow longer poison lists to aid
* testing that kernel functionality.
*/
-static CXLRetCode cmd_media_get_poison_list(struct cxl_cmd *cmd,
+static CXLRetCode cmd_media_get_poison_list(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxl_dstate,
uint16_t *len)
{
@@ -482,8 +498,8 @@ static CXLRetCode cmd_media_get_poison_list(struct cxl_cmd *cmd,
} QEMU_PACKED records[];
} QEMU_PACKED;
- struct get_poison_list_pl *in = (void *)cmd->payload;
- struct get_poison_list_out_pl *out = (void *)cmd->payload;
+ struct get_poison_list_pl *in = (void *)payload;
+ struct get_poison_list_out_pl *out = (void *)payload;
CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
uint16_t record_count = 0, i = 0;
uint64_t query_start, query_length;
@@ -536,7 +552,8 @@ static CXLRetCode cmd_media_get_poison_list(struct cxl_cmd *cmd,
return CXL_MBOX_SUCCESS;
}
-static CXLRetCode cmd_media_inject_poison(struct cxl_cmd *cmd,
+static CXLRetCode cmd_media_inject_poison(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxl_dstate,
uint16_t *len_unused)
{
@@ -546,7 +563,7 @@ static CXLRetCode cmd_media_inject_poison(struct cxl_cmd *cmd,
struct inject_poison_pl {
uint64_t dpa;
};
- struct inject_poison_pl *in = (void *)cmd->payload;
+ struct inject_poison_pl *in = (void *)payload;
uint64_t dpa = ldq_le_p(&in->dpa);
CXLPoison *p;
@@ -575,7 +592,8 @@ static CXLRetCode cmd_media_inject_poison(struct cxl_cmd *cmd,
return CXL_MBOX_SUCCESS;
}
-static CXLRetCode cmd_media_clear_poison(struct cxl_cmd *cmd,
+static CXLRetCode cmd_media_clear_poison(const struct cxl_cmd *cmd,
+ uint8_t *payload,
CXLDeviceState *cxl_dstate,
uint16_t *len_unused)
{
@@ -589,7 +607,7 @@ static CXLRetCode cmd_media_clear_poison(struct cxl_cmd *cmd,
CXLPoison *ent;
uint64_t dpa;
- struct clear_poison_pl *in = (void *)cmd->payload;
+ struct clear_poison_pl *in = (void *)payload;
dpa = ldq_le_p(&in->dpa);
if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->mem_size) {
@@ -659,7 +677,7 @@ static CXLRetCode cmd_media_clear_poison(struct cxl_cmd *cmd,
#define IMMEDIATE_POLICY_CHANGE (1 << 3)
#define IMMEDIATE_LOG_CHANGE (1 << 4)
-static struct cxl_cmd cxl_cmd_set[256][256] = {
+static const struct cxl_cmd cxl_cmd_set[256][256] = {
[EVENTS][GET_RECORDS] = { "EVENTS_GET_RECORDS",
cmd_events_get_records, 1, 0 },
[EVENTS][CLEAR_RECORDS] = { "EVENTS_CLEAR_RECORDS",
@@ -693,21 +711,21 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
{
uint16_t ret = CXL_MBOX_SUCCESS;
- struct cxl_cmd *cxl_cmd;
- uint64_t status_reg;
+ const struct cxl_cmd *cxl_cmd;
+ uint64_t status_reg = 0;
opcode_handler h;
uint64_t command_reg = cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_CMD];
uint8_t set = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND_SET);
uint8_t cmd = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND);
uint16_t len = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, LENGTH);
+ uint8_t *pl = cxl_dstate->mbox_reg_state + A_CXL_DEV_CMD_PAYLOAD;
+
cxl_cmd = &cxl_dstate->cxl_cmd_set[set][cmd];
h = cxl_cmd->handler;
if (h) {
if (len == cxl_cmd->in || cxl_cmd->in == ~0) {
- cxl_cmd->payload = cxl_dstate->mbox_reg_state +
- A_CXL_DEV_CMD_PAYLOAD;
- ret = (*h)(cxl_cmd, cxl_dstate, &len);
+ ret = (*h)(cxl_cmd, pl, cxl_dstate, &len);
assert(len <= cxl_dstate->payload_size);
} else {
ret = CXL_MBOX_INVALID_PAYLOAD_LENGTH;
@@ -740,7 +758,7 @@ void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate)
for (int set = 0; set < 256; set++) {
for (int cmd = 0; cmd < 256; cmd++) {
if (cxl_dstate->cxl_cmd_set[set][cmd].handler) {
- struct cxl_cmd *c = &cxl_dstate->cxl_cmd_set[set][cmd];
+ const struct cxl_cmd *c = &cxl_dstate->cxl_cmd_set[set][cmd];
struct cel_log *log =
&cxl_dstate->cel_log[cxl_dstate->cel_size];
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 04/17] hw/mbox: Split mailbox command payload into separate input and output
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (2 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 03/17] cxl/mbox: Pull the payload out of struct cxl_cmd and make instances constant Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 05/17] cxl/mbox: Pull the CCI definition out of the CXLDeviceState Jonathan Cameron via
` (12 subsequent siblings)
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
New CCI types that will be supported shortly do not have a single
buffer used in both directions. As such, split it up.
For CXL mailboxes the two pointers will be aliases of the same memory
so all callbacks must allow for that.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/cxl/cxl_device.h | 7 +-
hw/cxl/cxl-events.c | 2 +-
hw/cxl/cxl-mailbox-utils.c | 222 +++++++++++++++++++++---------------
3 files changed, 132 insertions(+), 99 deletions(-)
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index f84f6813aa..fc15c2729d 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -136,8 +136,9 @@ typedef enum {
struct cxl_cmd;
typedef CXLRetCode (*opcode_handler)(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxl_dstate, uint16_t *len);
+ uint8_t *payload_in, size_t len_in,
+ uint8_t *payload_out, size_t *len_out,
+ CXLDeviceState *cxl_dstate);
struct cxl_cmd {
const char *name;
opcode_handler handler;
@@ -416,7 +417,7 @@ bool cxl_event_insert(CXLDeviceState *cxlds, CXLEventLogType log_type,
CXLEventRecordRaw *event);
CXLRetCode cxl_event_get_records(CXLDeviceState *cxlds, CXLGetEventPayload *pl,
uint8_t log_type, int max_recs,
- uint16_t *len);
+ size_t *len);
CXLRetCode cxl_event_clear_records(CXLDeviceState *cxlds,
CXLClearEventPayload *pl);
diff --git a/hw/cxl/cxl-events.c b/hw/cxl/cxl-events.c
index d161d57456..a985bb3e09 100644
--- a/hw/cxl/cxl-events.c
+++ b/hw/cxl/cxl-events.c
@@ -143,7 +143,7 @@ bool cxl_event_insert(CXLDeviceState *cxlds, CXLEventLogType log_type,
CXLRetCode cxl_event_get_records(CXLDeviceState *cxlds, CXLGetEventPayload *pl,
uint8_t log_type, int max_recs,
- uint16_t *len)
+ size_t *len)
{
CXLEventLog *log;
CXLEvent *entry;
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index f0d94b9ae4..785c3fa7d0 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -72,9 +72,9 @@ enum {
static CXLRetCode cmd_events_get_records(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxlds,
- uint16_t *len)
+ uint8_t *payload_in, size_t len_in,
+ uint8_t *payload_out, size_t *len_out,
+ CXLDeviceState *cxlds)
{
CXLGetEventPayload *pl;
uint8_t log_type;
@@ -84,9 +84,9 @@ static CXLRetCode cmd_events_get_records(const struct cxl_cmd *cmd,
return CXL_MBOX_INVALID_INPUT;
}
- log_type = payload[0];
+ log_type = payload_in[0];
- pl = (CXLGetEventPayload *)payload;
+ pl = (CXLGetEventPayload *)payload_out;
memset(pl, 0, sizeof(*pl));
max_recs = (cxlds->payload_size - CXL_EVENT_PAYLOAD_HDR_SIZE) /
@@ -95,30 +95,34 @@ static CXLRetCode cmd_events_get_records(const struct cxl_cmd *cmd,
max_recs = 0xFFFF;
}
- return cxl_event_get_records(cxlds, pl, log_type, max_recs, len);
+ return cxl_event_get_records(cxlds, pl, log_type, max_recs, len_out);
}
static CXLRetCode cmd_events_clear_records(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxlds,
- uint16_t *len)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxlds)
{
CXLClearEventPayload *pl;
- pl = (CXLClearEventPayload *)payload;
- *len = 0;
+ pl = (CXLClearEventPayload *)payload_in;
+ *len_out = 0;
return cxl_event_clear_records(cxlds, pl);
}
static CXLRetCode cmd_events_get_interrupt_policy(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxlds,
- uint16_t *len)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxlds)
{
CXLEventInterruptPolicy *policy;
CXLEventLog *log;
- policy = (CXLEventInterruptPolicy *)payload;
+ policy = (CXLEventInterruptPolicy *)payload_out;
memset(policy, 0, sizeof(*policy));
log = &cxlds->event_logs[CXL_EVENT_TYPE_INFO];
@@ -147,23 +151,25 @@ static CXLRetCode cmd_events_get_interrupt_policy(const struct cxl_cmd *cmd,
policy->dyn_cap_settings = CXL_INT_MSI_MSIX;
}
- *len = sizeof(*policy);
+ *len_out = sizeof(*policy);
return CXL_MBOX_SUCCESS;
}
static CXLRetCode cmd_events_set_interrupt_policy(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxlds,
- uint16_t *len)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxlds)
{
CXLEventInterruptPolicy *policy;
CXLEventLog *log;
- if (*len < CXL_EVENT_INT_SETTING_MIN_LEN) {
+ if (len_in < CXL_EVENT_INT_SETTING_MIN_LEN) {
return CXL_MBOX_INVALID_PAYLOAD_LENGTH;
}
- policy = (CXLEventInterruptPolicy *)payload;
+ policy = (CXLEventInterruptPolicy *)payload_in;
log = &cxlds->event_logs[CXL_EVENT_TYPE_INFO];
log->irq_enabled = (policy->info_settings & CXL_EVENT_INT_MODE_MASK) ==
@@ -182,7 +188,7 @@ static CXLRetCode cmd_events_set_interrupt_policy(const struct cxl_cmd *cmd,
CXL_INT_MSI_MSIX;
/* DCD is optional */
- if (*len < sizeof(*policy)) {
+ if (len_in < sizeof(*policy)) {
return CXL_MBOX_SUCCESS;
}
@@ -190,15 +196,17 @@ static CXLRetCode cmd_events_set_interrupt_policy(const struct cxl_cmd *cmd,
log->irq_enabled = (policy->dyn_cap_settings & CXL_EVENT_INT_MODE_MASK) ==
CXL_INT_MSI_MSIX;
- *len = sizeof(*policy);
+ *len_out = 0;
return CXL_MBOX_SUCCESS;
}
/* 8.2.9.2.1 */
static CXLRetCode cmd_firmware_update_get_info(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxl_dstate,
- uint16_t *len)
+ uint8_t *payload_in,
+ size_t len,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxl_dstate)
{
struct {
uint8_t slots_supported;
@@ -217,7 +225,7 @@ static CXLRetCode cmd_firmware_update_get_info(const struct cxl_cmd *cmd,
return CXL_MBOX_INTERNAL_ERROR;
}
- fw_info = (void *)payload;
+ fw_info = (void *)payload_out;
memset(fw_info, 0, sizeof(*fw_info));
fw_info->slots_supported = 2;
@@ -225,36 +233,40 @@ static CXLRetCode cmd_firmware_update_get_info(const struct cxl_cmd *cmd,
fw_info->caps = 0;
pstrcpy(fw_info->fw_rev1, sizeof(fw_info->fw_rev1), "BWFW VERSION 0");
- *len = sizeof(*fw_info);
+ *len_out = sizeof(*fw_info);
return CXL_MBOX_SUCCESS;
}
/* 8.2.9.3.1 */
static CXLRetCode cmd_timestamp_get(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxl_dstate,
- uint16_t *len)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxl_dstate)
{
uint64_t final_time = cxl_device_get_timestamp(cxl_dstate);
- stq_le_p(payload, final_time);
- *len = 8;
+ stq_le_p(payload_out, final_time);
+ *len_out = 8;
return CXL_MBOX_SUCCESS;
}
/* 8.2.9.3.2 */
static CXLRetCode cmd_timestamp_set(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxl_dstate,
- uint16_t *len)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxl_dstate)
{
cxl_dstate->timestamp.set = true;
cxl_dstate->timestamp.last_set = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
- cxl_dstate->timestamp.host_set = le64_to_cpu(*(uint64_t *)payload);
+ cxl_dstate->timestamp.host_set = le64_to_cpu(*(uint64_t *)payload_in);
- *len = 0;
+ *len_out = 0;
return CXL_MBOX_SUCCESS;
}
@@ -266,9 +278,11 @@ static const QemuUUID cel_uuid = {
/* 8.2.9.4.1 */
static CXLRetCode cmd_logs_get_supported(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxl_dstate,
- uint16_t *len)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxl_dstate)
{
struct {
uint16_t entries;
@@ -277,22 +291,24 @@ static CXLRetCode cmd_logs_get_supported(const struct cxl_cmd *cmd,
QemuUUID uuid;
uint32_t size;
} log_entries[1];
- } QEMU_PACKED *supported_logs = (void *)payload;
+ } QEMU_PACKED *supported_logs = (void *)payload_out;
QEMU_BUILD_BUG_ON(sizeof(*supported_logs) != 0x1c);
supported_logs->entries = 1;
supported_logs->log_entries[0].uuid = cel_uuid;
supported_logs->log_entries[0].size = 4 * cxl_dstate->cel_size;
- *len = sizeof(*supported_logs);
+ *len_out = sizeof(*supported_logs);
return CXL_MBOX_SUCCESS;
}
/* 8.2.9.4.2 */
static CXLRetCode cmd_logs_get_log(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxl_dstate,
- uint16_t *len)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxl_dstate)
{
struct {
QemuUUID uuid;
@@ -300,7 +316,7 @@ static CXLRetCode cmd_logs_get_log(const struct cxl_cmd *cmd,
uint32_t length;
} QEMU_PACKED QEMU_ALIGNED(16) *get_log;
- get_log = (void *)payload;
+ get_log = (void *)payload_in;
/*
* 8.2.9.4.2
@@ -324,19 +340,21 @@ static CXLRetCode cmd_logs_get_log(const struct cxl_cmd *cmd,
}
/* Store off everything to local variables so we can wipe out the payload */
- *len = get_log->length;
+ *len_out = get_log->length;
- memmove(payload, cxl_dstate->cel_log + get_log->offset,
- get_log->length);
+ memmove(payload_out, cxl_dstate->cel_log + get_log->offset,
+ get_log->length);
return CXL_MBOX_SUCCESS;
}
/* 8.2.9.5.1.1 */
static CXLRetCode cmd_identify_memory_device(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxl_dstate,
- uint16_t *len)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxl_dstate)
{
struct {
char fw_revision[0x10];
@@ -364,7 +382,7 @@ static CXLRetCode cmd_identify_memory_device(const struct cxl_cmd *cmd,
return CXL_MBOX_INTERNAL_ERROR;
}
- id = (void *)payload;
+ id = (void *)payload_out;
memset(id, 0, sizeof(*id));
snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0);
@@ -378,21 +396,23 @@ static CXLRetCode cmd_identify_memory_device(const struct cxl_cmd *cmd,
/* No limit - so limited by main poison record limit */
stw_le_p(&id->inject_poison_limit, 0);
- *len = sizeof(*id);
+ *len_out = sizeof(*id);
return CXL_MBOX_SUCCESS;
}
static CXLRetCode cmd_ccls_get_partition_info(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxl_dstate,
- uint16_t *len)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxl_dstate)
{
struct {
uint64_t active_vmem;
uint64_t active_pmem;
uint64_t next_vmem;
uint64_t next_pmem;
- } QEMU_PACKED *part_info = (void *)payload;
+ } QEMU_PACKED *part_info = (void *)payload_out;
QEMU_BUILD_BUG_ON(sizeof(*part_info) != 0x20);
if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER)) ||
@@ -409,14 +429,16 @@ static CXLRetCode cmd_ccls_get_partition_info(const struct cxl_cmd *cmd,
stq_le_p(&part_info->active_pmem, cxl_dstate->pmem_size / CXL_CAPACITY_MULTIPLIER);
stq_le_p(&part_info->next_pmem, 0);
- *len = sizeof(*part_info);
+ *len_out = sizeof(*part_info);
return CXL_MBOX_SUCCESS;
}
static CXLRetCode cmd_ccls_get_lsa(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxl_dstate,
- uint16_t *len)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxl_dstate)
{
struct {
uint32_t offset;
@@ -426,46 +448,47 @@ static CXLRetCode cmd_ccls_get_lsa(const struct cxl_cmd *cmd,
CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
uint32_t offset, length;
- get_lsa = (void *)payload;
+ get_lsa = (void *)payload_in;
offset = get_lsa->offset;
length = get_lsa->length;
if (offset + length > cvc->get_lsa_size(ct3d)) {
- *len = 0;
+ *len_out = 0;
return CXL_MBOX_INVALID_INPUT;
}
- *len = cvc->get_lsa(ct3d, get_lsa, length, offset);
+ *len_out = cvc->get_lsa(ct3d, payload_out, length, offset);
return CXL_MBOX_SUCCESS;
}
static CXLRetCode cmd_ccls_set_lsa(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxl_dstate,
- uint16_t *len)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxl_dstate)
{
struct set_lsa_pl {
uint32_t offset;
uint32_t rsvd;
uint8_t data[];
} QEMU_PACKED;
- struct set_lsa_pl *set_lsa_payload = (void *)payload;
+ struct set_lsa_pl *set_lsa_payload = (void *)payload_in;
CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
const size_t hdr_len = offsetof(struct set_lsa_pl, data);
- uint16_t plen = *len;
- *len = 0;
- if (!plen) {
+ *len_out = 0;
+ if (!len_in) {
return CXL_MBOX_SUCCESS;
}
- if (set_lsa_payload->offset + plen > cvc->get_lsa_size(ct3d) + hdr_len) {
+ if (set_lsa_payload->offset + len_in > cvc->get_lsa_size(ct3d) + hdr_len) {
return CXL_MBOX_INVALID_INPUT;
}
- plen -= hdr_len;
+ len_in -= hdr_len;
- cvc->set_lsa(ct3d, set_lsa_payload->data, plen, set_lsa_payload->offset);
+ cvc->set_lsa(ct3d, set_lsa_payload->data, len_in, set_lsa_payload->offset);
return CXL_MBOX_SUCCESS;
}
@@ -476,9 +499,11 @@ static CXLRetCode cmd_ccls_set_lsa(const struct cxl_cmd *cmd,
* testing that kernel functionality.
*/
static CXLRetCode cmd_media_get_poison_list(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxl_dstate,
- uint16_t *len)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxl_dstate)
{
struct get_poison_list_pl {
uint64_t pa;
@@ -498,8 +523,8 @@ static CXLRetCode cmd_media_get_poison_list(const struct cxl_cmd *cmd,
} QEMU_PACKED records[];
} QEMU_PACKED;
- struct get_poison_list_pl *in = (void *)payload;
- struct get_poison_list_out_pl *out = (void *)payload;
+ struct get_poison_list_pl *in = (void *)payload_in;
+ struct get_poison_list_out_pl *out = (void *)payload_out;
CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
uint16_t record_count = 0, i = 0;
uint64_t query_start, query_length;
@@ -548,14 +573,16 @@ static CXLRetCode cmd_media_get_poison_list(const struct cxl_cmd *cmd,
stq_le_p(&out->overflow_timestamp, ct3d->poison_list_overflow_ts);
}
stw_le_p(&out->count, record_count);
- *len = out_pl_len;
+ *len_out = out_pl_len;
return CXL_MBOX_SUCCESS;
}
static CXLRetCode cmd_media_inject_poison(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxl_dstate,
- uint16_t *len_unused)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxl_dstate)
{
CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
CXLPoisonList *poison_list = &ct3d->poison_list;
@@ -563,7 +590,7 @@ static CXLRetCode cmd_media_inject_poison(const struct cxl_cmd *cmd,
struct inject_poison_pl {
uint64_t dpa;
};
- struct inject_poison_pl *in = (void *)payload;
+ struct inject_poison_pl *in = (void *)payload_in;
uint64_t dpa = ldq_le_p(&in->dpa);
CXLPoison *p;
@@ -588,14 +615,17 @@ static CXLRetCode cmd_media_inject_poison(const struct cxl_cmd *cmd,
*/
QLIST_INSERT_HEAD(poison_list, p, node);
ct3d->poison_list_cnt++;
+ *len_out = 0;
return CXL_MBOX_SUCCESS;
}
static CXLRetCode cmd_media_clear_poison(const struct cxl_cmd *cmd,
- uint8_t *payload,
- CXLDeviceState *cxl_dstate,
- uint16_t *len_unused)
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLDeviceState *cxl_dstate)
{
CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
CXLPoisonList *poison_list = &ct3d->poison_list;
@@ -607,7 +637,7 @@ static CXLRetCode cmd_media_clear_poison(const struct cxl_cmd *cmd,
CXLPoison *ent;
uint64_t dpa;
- struct clear_poison_pl *in = (void *)payload;
+ struct clear_poison_pl *in = (void *)payload_in;
dpa = ldq_le_p(&in->dpa);
if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->mem_size) {
@@ -668,6 +698,7 @@ static CXLRetCode cmd_media_clear_poison(const struct cxl_cmd *cmd,
}
/* Any fragments have been added, free original entry */
g_free(ent);
+ *len_out = 0;
return CXL_MBOX_SUCCESS;
}
@@ -718,15 +749,16 @@ void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
uint8_t set = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND_SET);
uint8_t cmd = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND);
- uint16_t len = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, LENGTH);
+ uint16_t len_in = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, LENGTH);
uint8_t *pl = cxl_dstate->mbox_reg_state + A_CXL_DEV_CMD_PAYLOAD;
+ size_t len_out = 0;
cxl_cmd = &cxl_dstate->cxl_cmd_set[set][cmd];
h = cxl_cmd->handler;
if (h) {
- if (len == cxl_cmd->in || cxl_cmd->in == ~0) {
- ret = (*h)(cxl_cmd, pl, cxl_dstate, &len);
- assert(len <= cxl_dstate->payload_size);
+ if (len_in == cxl_cmd->in || cxl_cmd->in == ~0) {
+ ret = (*h)(cxl_cmd, pl, len_in, pl, &len_out, cxl_dstate);
+ assert(len_out <= cxl_dstate->payload_size);
} else {
ret = CXL_MBOX_INVALID_PAYLOAD_LENGTH;
}
@@ -742,7 +774,7 @@ void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
/* Set the return length */
command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND_SET, 0);
command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND, 0);
- command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD, LENGTH, len);
+ command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD, LENGTH, len_out);
cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_CMD] = command_reg;
cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_STS] = status_reg;
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 05/17] cxl/mbox: Pull the CCI definition out of the CXLDeviceState
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (3 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 04/17] hw/mbox: Split mailbox command payload into separate input and output Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 06/17] cxl/mbox: Generalize the CCI command processing Jonathan Cameron via
` (11 subsequent siblings)
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
Enables having multiple CCIs per devices. Each CCI (mailbox)
has it's own state and command list, so they can't share
a single structure.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/cxl/cxl_device.h | 48 ++++++++++++++-----
hw/cxl/cxl-device-utils.c | 31 +++++++++---
hw/cxl/cxl-mailbox-utils.c | 94 ++++++++++++++++++++++---------------
hw/mem/cxl_type3.c | 5 +-
4 files changed, 120 insertions(+), 58 deletions(-)
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index fc15c2729d..813d4b926c 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -134,11 +134,12 @@ typedef enum {
CXL_MBOX_MAX = 0x17
} CXLRetCode;
+typedef struct CXLCCI CXLCCI;
struct cxl_cmd;
typedef CXLRetCode (*opcode_handler)(const struct cxl_cmd *cmd,
uint8_t *payload_in, size_t len_in,
uint8_t *payload_out, size_t *len_out,
- CXLDeviceState *cxl_dstate);
+ CXLCCI *cci);
struct cxl_cmd {
const char *name;
opcode_handler handler;
@@ -162,6 +163,31 @@ typedef struct CXLEventLog {
QSIMPLEQ_HEAD(, CXLEvent) events;
} CXLEventLog;
+typedef struct CXLCCI {
+ const struct cxl_cmd (*cxl_cmd_set)[256];
+ struct cel_log {
+ uint16_t opcode;
+ uint16_t effect;
+ } cel_log[1 << 16];
+ size_t cel_size;
+
+ /* background command handling (times in ms) */
+ struct {
+ uint16_t opcode;
+ uint16_t complete_pct;
+ uint16_t ret_code; /* Current value of retcode */
+ uint64_t starttime;
+ /* set by each bg cmd, cleared by the bg_timer when complete */
+ uint64_t runtime;
+ QEMUTimer *timer;
+ } bg;
+ size_t payload_max;
+ /* Pointer to device hosting the CCI */
+ DeviceState *d;
+ /* Pointer to the device hosting the protocol conversion */
+ DeviceState *intf;
+} CXLCCI;
+
typedef struct cxl_device_state {
MemoryRegion device_registers;
@@ -196,11 +222,6 @@ typedef struct cxl_device_state {
uint32_t mbox_reg_state32[CXL_MAILBOX_REGISTERS_LENGTH / 4];
uint64_t mbox_reg_state64[CXL_MAILBOX_REGISTERS_LENGTH / 8];
};
- struct cel_log {
- uint16_t opcode;
- uint16_t effect;
- } cel_log[1 << 16];
- size_t cel_size;
};
struct {
@@ -214,16 +235,17 @@ typedef struct cxl_device_state {
uint64_t pmem_size;
uint64_t vmem_size;
- const struct cxl_cmd (*cxl_cmd_set)[256];
CPMUState cpmu[CXL_NUM_CPMU_INSTANCES];
CXLEventLog event_logs[CXL_EVENT_TYPE_MAX];
} CXLDeviceState;
/* Initialize the register block for a device */
-void cxl_device_register_block_init(Object *obj, CXLDeviceState *dev);
+void cxl_device_register_block_init(Object *obj, CXLDeviceState *dev,
+ CXLCCI *cci);
+typedef struct CXLType3Dev CXLType3Dev;
/* Set up default values for the register block */
-void cxl_device_register_init_common(CXLDeviceState *dev);
+void cxl_device_register_init_t3(CXLType3Dev *ct3d);
/*
* CXL 2.0 - 8.2.8.1 including errata F4
@@ -269,8 +291,9 @@ CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MEMORY_DEVICE,
CXL_DEVICE_CAP_HDR1_OFFSET +
CXL_DEVICE_CAP_REG_SIZE * 2)
-void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate);
-void cxl_process_mailbox(CXLDeviceState *cxl_dstate);
+void cxl_initialize_mailbox_t3(CXLCCI *cci, DeviceState *d, size_t payload_max);
+void cxl_init_cci(CXLCCI *cci, size_t payload_max);
+void cxl_process_mailbox(CXLCCI *cci);
#define cxl_device_cap_init(dstate, reg, cap_id, ver) \
do { \
@@ -371,7 +394,8 @@ struct CXLType3Dev {
AddressSpace hostpmem_as;
CXLComponentState cxl_cstate;
CXLDeviceState cxl_dstate;
-
+ CXLCCI cci;
+
/* DOE */
DOECap doe_cdat;
DOECap doe_comp;
diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
index 517f06d869..caf61ab1f5 100644
--- a/hw/cxl/cxl-device-utils.c
+++ b/hw/cxl/cxl-device-utils.c
@@ -59,7 +59,14 @@ static uint64_t dev_reg_read(void *opaque, hwaddr offset, unsigned size)
static uint64_t mailbox_reg_read(void *opaque, hwaddr offset, unsigned size)
{
- CXLDeviceState *cxl_dstate = opaque;
+ CXLDeviceState *cxl_dstate;
+ CXLCCI *cci = opaque;
+
+ if (object_dynamic_cast(OBJECT(cci->intf), TYPE_CXL_TYPE3)) {
+ cxl_dstate = &CXL_TYPE3(cci->intf)->cxl_dstate;
+ } else {
+ return 0;
+ }
switch (size) {
case 1:
@@ -120,7 +127,14 @@ static void mailbox_mem_writeq(uint64_t *reg_state, hwaddr offset,
static void mailbox_reg_write(void *opaque, hwaddr offset, uint64_t value,
unsigned size)
{
- CXLDeviceState *cxl_dstate = opaque;
+ CXLDeviceState *cxl_dstate;
+ CXLCCI *cci = opaque;
+
+ if (object_dynamic_cast(OBJECT(cci->intf), TYPE_CXL_TYPE3)) {
+ cxl_dstate = &CXL_TYPE3(cci->intf)->cxl_dstate;
+ } else {
+ return;
+ }
if (offset >= A_CXL_DEV_CMD_PAYLOAD) {
memcpy(cxl_dstate->mbox_reg_state + offset, &value, size);
@@ -140,7 +154,7 @@ static void mailbox_reg_write(void *opaque, hwaddr offset, uint64_t value,
if (ARRAY_FIELD_EX32(cxl_dstate->mbox_reg_state32, CXL_DEV_MAILBOX_CTRL,
DOORBELL)) {
- cxl_process_mailbox(cxl_dstate);
+ cxl_process_mailbox(cci);
}
}
@@ -220,7 +234,8 @@ static const MemoryRegionOps caps_ops = {
},
};
-void cxl_device_register_block_init(Object *obj, CXLDeviceState *cxl_dstate)
+void cxl_device_register_block_init(Object *obj, CXLDeviceState *cxl_dstate,
+ CXLCCI *cci)
{
/* This will be a BAR, so needs to be rounded up to pow2 for PCI spec */
memory_region_init(&cxl_dstate->device_registers, obj, "device-registers",
@@ -230,7 +245,7 @@ void cxl_device_register_block_init(Object *obj, CXLDeviceState *cxl_dstate)
"cap-array", CXL_CAPS_SIZE);
memory_region_init_io(&cxl_dstate->device, obj, &dev_ops, cxl_dstate,
"device-status", CXL_DEVICE_STATUS_REGISTERS_LENGTH);
- memory_region_init_io(&cxl_dstate->mailbox, obj, &mailbox_ops, cxl_dstate,
+ memory_region_init_io(&cxl_dstate->mailbox, obj, &mailbox_ops, cci,
"mailbox", CXL_MAILBOX_REGISTERS_LENGTH);
memory_region_init_io(&cxl_dstate->memory_device, obj, &mdev_ops,
cxl_dstate, "memory device caps",
@@ -281,8 +296,9 @@ static void mailbox_reg_init_common(CXLDeviceState *cxl_dstate)
static void memdev_reg_init_common(CXLDeviceState *cxl_dstate) { }
-void cxl_device_register_init_common(CXLDeviceState *cxl_dstate)
+void cxl_device_register_init_t3(CXLType3Dev *ct3d)
{
+ CXLDeviceState *cxl_dstate = &ct3d->cxl_dstate;
uint64_t *cap_hdrs = cxl_dstate->caps_reg_state64;
const int cap_count = 3;
@@ -300,7 +316,8 @@ void cxl_device_register_init_common(CXLDeviceState *cxl_dstate)
cxl_device_cap_init(cxl_dstate, MEMORY_DEVICE, 0x4000, 1);
memdev_reg_init_common(cxl_dstate);
- cxl_initialize_mailbox(cxl_dstate);
+ cxl_initialize_mailbox_t3(&ct3d->cci, DEVICE(ct3d),
+ CXL_MAILBOX_MAX_PAYLOAD_SIZE);
}
uint64_t cxl_device_get_timestamp(CXLDeviceState *cxl_dstate)
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 785c3fa7d0..399df6e3af 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -74,8 +74,9 @@ enum {
static CXLRetCode cmd_events_get_records(const struct cxl_cmd *cmd,
uint8_t *payload_in, size_t len_in,
uint8_t *payload_out, size_t *len_out,
- CXLDeviceState *cxlds)
+ CXLCCI *cci)
{
+ CXLDeviceState *cxlds = &CXL_TYPE3(cci->d)->cxl_dstate;
CXLGetEventPayload *pl;
uint8_t log_type;
int max_recs;
@@ -103,8 +104,9 @@ static CXLRetCode cmd_events_clear_records(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxlds)
+ CXLCCI *cci)
{
+ CXLDeviceState *cxlds = &CXL_TYPE3(cci->d)->cxl_dstate;
CXLClearEventPayload *pl;
pl = (CXLClearEventPayload *)payload_in;
@@ -117,8 +119,9 @@ static CXLRetCode cmd_events_get_interrupt_policy(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxlds)
+ CXLCCI *cci)
{
+ CXLDeviceState *cxlds = &CXL_TYPE3(cci->d)->cxl_dstate;
CXLEventInterruptPolicy *policy;
CXLEventLog *log;
@@ -160,8 +163,9 @@ static CXLRetCode cmd_events_set_interrupt_policy(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxlds)
+ CXLCCI *cci)
{
+ CXLDeviceState *cxlds = &CXL_TYPE3(cci->d)->cxl_dstate;
CXLEventInterruptPolicy *policy;
CXLEventLog *log;
@@ -206,8 +210,9 @@ static CXLRetCode cmd_firmware_update_get_info(const struct cxl_cmd *cmd,
size_t len,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxl_dstate)
+ CXLCCI *cci)
{
+ CXLDeviceState *cxl_dstate = &CXL_TYPE3(cci->d)->cxl_dstate;
struct {
uint8_t slots_supported;
uint8_t slot_info;
@@ -243,8 +248,9 @@ static CXLRetCode cmd_timestamp_get(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxl_dstate)
+ CXLCCI *cci)
{
+ CXLDeviceState *cxl_dstate = &CXL_TYPE3(cci->d)->cxl_dstate;
uint64_t final_time = cxl_device_get_timestamp(cxl_dstate);
stq_le_p(payload_out, final_time);
@@ -259,8 +265,10 @@ static CXLRetCode cmd_timestamp_set(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxl_dstate)
+ CXLCCI *cci)
{
+ CXLDeviceState *cxl_dstate = &CXL_TYPE3(cci->d)->cxl_dstate;
+
cxl_dstate->timestamp.set = true;
cxl_dstate->timestamp.last_set = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
@@ -282,7 +290,7 @@ static CXLRetCode cmd_logs_get_supported(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxl_dstate)
+ CXLCCI *cci)
{
struct {
uint16_t entries;
@@ -296,7 +304,7 @@ static CXLRetCode cmd_logs_get_supported(const struct cxl_cmd *cmd,
supported_logs->entries = 1;
supported_logs->log_entries[0].uuid = cel_uuid;
- supported_logs->log_entries[0].size = 4 * cxl_dstate->cel_size;
+ supported_logs->log_entries[0].size = 4 * cci->cel_size;
*len_out = sizeof(*supported_logs);
return CXL_MBOX_SUCCESS;
@@ -308,7 +316,7 @@ static CXLRetCode cmd_logs_get_log(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxl_dstate)
+ CXLCCI *cci)
{
struct {
QemuUUID uuid;
@@ -331,7 +339,7 @@ static CXLRetCode cmd_logs_get_log(const struct cxl_cmd *cmd,
* the only possible failure would be if the mailbox itself isn't big
* enough.
*/
- if (get_log->offset + get_log->length > cxl_dstate->payload_size) {
+ if (get_log->offset + get_log->length > cci->payload_max) {
return CXL_MBOX_INVALID_INPUT;
}
@@ -342,8 +350,7 @@ static CXLRetCode cmd_logs_get_log(const struct cxl_cmd *cmd,
/* Store off everything to local variables so we can wipe out the payload */
*len_out = get_log->length;
- memmove(payload_out, cxl_dstate->cel_log + get_log->offset,
- get_log->length);
+ memmove(payload_out, cci->cel_log + get_log->offset, get_log->length);
return CXL_MBOX_SUCCESS;
}
@@ -354,7 +361,7 @@ static CXLRetCode cmd_identify_memory_device(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxl_dstate)
+ CXLCCI *cci)
{
struct {
char fw_revision[0x10];
@@ -373,9 +380,9 @@ static CXLRetCode cmd_identify_memory_device(const struct cxl_cmd *cmd,
uint8_t qos_telemetry_caps;
} QEMU_PACKED *id;
QEMU_BUILD_BUG_ON(sizeof(*id) != 0x43);
-
- CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
+ CXLType3Dev *ct3d = CXL_TYPE3(cci->d);
CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
+ CXLDeviceState *cxl_dstate = &ct3d->cxl_dstate;
if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER)) ||
(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))) {
@@ -405,8 +412,9 @@ static CXLRetCode cmd_ccls_get_partition_info(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxl_dstate)
+ CXLCCI *cci)
{
+ CXLDeviceState *cxl_dstate = &CXL_TYPE3(cci->d)->cxl_dstate;
struct {
uint64_t active_vmem;
uint64_t active_pmem;
@@ -438,13 +446,13 @@ static CXLRetCode cmd_ccls_get_lsa(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxl_dstate)
+ CXLCCI *cci)
{
struct {
uint32_t offset;
uint32_t length;
} QEMU_PACKED *get_lsa;
- CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
+ CXLType3Dev *ct3d = CXL_TYPE3(cci->d);
CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
uint32_t offset, length;
@@ -466,7 +474,7 @@ static CXLRetCode cmd_ccls_set_lsa(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxl_dstate)
+ CXLCCI *cci)
{
struct set_lsa_pl {
uint32_t offset;
@@ -474,7 +482,7 @@ static CXLRetCode cmd_ccls_set_lsa(const struct cxl_cmd *cmd,
uint8_t data[];
} QEMU_PACKED;
struct set_lsa_pl *set_lsa_payload = (void *)payload_in;
- CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
+ CXLType3Dev *ct3d = CXL_TYPE3(cci->d);
CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
const size_t hdr_len = offsetof(struct set_lsa_pl, data);
@@ -503,7 +511,7 @@ static CXLRetCode cmd_media_get_poison_list(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxl_dstate)
+ CXLCCI *cci)
{
struct get_poison_list_pl {
uint64_t pa;
@@ -525,7 +533,7 @@ static CXLRetCode cmd_media_get_poison_list(const struct cxl_cmd *cmd,
struct get_poison_list_pl *in = (void *)payload_in;
struct get_poison_list_out_pl *out = (void *)payload_out;
- CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
+ CXLType3Dev *ct3d = CXL_TYPE3(cci->d);
uint16_t record_count = 0, i = 0;
uint64_t query_start, query_length;
CXLPoisonList *poison_list = &ct3d->poison_list;
@@ -582,9 +590,9 @@ static CXLRetCode cmd_media_inject_poison(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxl_dstate)
+ CXLCCI *cci)
{
- CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
+ CXLType3Dev *ct3d = CXL_TYPE3(cci->d);
CXLPoisonList *poison_list = &ct3d->poison_list;
CXLPoison *ent;
struct inject_poison_pl {
@@ -625,9 +633,10 @@ static CXLRetCode cmd_media_clear_poison(const struct cxl_cmd *cmd,
size_t len_in,
uint8_t *payload_out,
size_t *len_out,
- CXLDeviceState *cxl_dstate)
+ CXLCCI *cci)
{
- CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
+ CXLType3Dev *ct3d = CXL_TYPE3(cci->d);
+ CXLDeviceState *cxl_dstate = &ct3d->cxl_dstate;
CXLPoisonList *poison_list = &ct3d->poison_list;
CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
struct clear_poison_pl {
@@ -739,12 +748,13 @@ static const struct cxl_cmd cxl_cmd_set[256][256] = {
cmd_media_clear_poison, 72, 0 },
};
-void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
+void cxl_process_mailbox(CXLCCI *cci)
{
uint16_t ret = CXL_MBOX_SUCCESS;
const struct cxl_cmd *cxl_cmd;
uint64_t status_reg = 0;
opcode_handler h;
+ CXLDeviceState *cxl_dstate = &CXL_TYPE3(cci->d)->cxl_dstate;
uint64_t command_reg = cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_CMD];
uint8_t set = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND_SET);
@@ -753,12 +763,12 @@ void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
uint8_t *pl = cxl_dstate->mbox_reg_state + A_CXL_DEV_CMD_PAYLOAD;
size_t len_out = 0;
- cxl_cmd = &cxl_dstate->cxl_cmd_set[set][cmd];
+ cxl_cmd = &cci->cxl_cmd_set[set][cmd];
h = cxl_cmd->handler;
if (h) {
if (len_in == cxl_cmd->in || cxl_cmd->in == ~0) {
- ret = (*h)(cxl_cmd, pl, len_in, pl, &len_out, cxl_dstate);
- assert(len_out <= cxl_dstate->payload_size);
+ ret = (*h)(cxl_cmd, pl, len_in, pl, &len_out, cci);
+ assert(len_out <= cci->payload_max);
} else {
ret = CXL_MBOX_INVALID_PAYLOAD_LENGTH;
}
@@ -784,20 +794,30 @@ void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
DOORBELL, 0);
}
-void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate)
+void cxl_init_cci(CXLCCI *cci, size_t payload_max)
{
- cxl_dstate->cxl_cmd_set = cxl_cmd_set;
+ cci->payload_max = payload_max;
for (int set = 0; set < 256; set++) {
for (int cmd = 0; cmd < 256; cmd++) {
- if (cxl_dstate->cxl_cmd_set[set][cmd].handler) {
- const struct cxl_cmd *c = &cxl_dstate->cxl_cmd_set[set][cmd];
+ if (cci->cxl_cmd_set[set][cmd].handler) {
+ const struct cxl_cmd *c = &cci->cxl_cmd_set[set][cmd];
struct cel_log *log =
- &cxl_dstate->cel_log[cxl_dstate->cel_size];
+ &cci->cel_log[cci->cel_size];
log->opcode = (set << 8) | cmd;
log->effect = c->effect;
- cxl_dstate->cel_size++;
+ cci->cel_size++;
}
}
}
}
+
+void cxl_initialize_mailbox_t3(CXLCCI *cci, DeviceState *d, size_t payload_max)
+{
+ cci->cxl_cmd_set = cxl_cmd_set;
+ cci->d = d;
+
+ /* No separation for PCI MB as protocol handled in PCI device */
+ cci->intf = d;
+ cxl_init_cci(cci, payload_max);
+}
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index ff52106f8d..f479dc67e8 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -832,7 +832,8 @@ static void ct3_realize(PCIDevice *pci_dev, Error **errp)
pci_dev, CXL_COMPONENT_REG_BAR_IDX,
PCI_BASE_ADDRESS_SPACE_MEMORY | PCI_BASE_ADDRESS_MEM_TYPE_64, mr);
- cxl_device_register_block_init(OBJECT(pci_dev), &ct3d->cxl_dstate);
+ cxl_device_register_block_init(OBJECT(pci_dev), &ct3d->cxl_dstate,
+ &ct3d->cci);
cxl_cpmu_register_block_init(OBJECT(pci_dev), &ct3d->cxl_dstate, 0, 6);
cxl_cpmu_register_block_init(OBJECT(pci_dev), &ct3d->cxl_dstate, 1, 7);
pci_register_bar(pci_dev, CXL_DEVICE_REG_BAR_IDX,
@@ -1040,7 +1041,7 @@ static void ct3d_reset(DeviceState *dev)
uint32_t *write_msk = ct3d->cxl_cstate.crb.cache_mem_regs_write_mask;
cxl_component_register_init_common(reg_state, write_msk, CXL2_TYPE3_DEVICE);
- cxl_device_register_init_common(&ct3d->cxl_dstate);
+ cxl_device_register_init_t3(ct3d);
}
static Property ct3_props[] = {
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 06/17] cxl/mbox: Generalize the CCI command processing
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (4 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 05/17] cxl/mbox: Pull the CCI definition out of the CXLDeviceState Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 07/17] hw/acpi/aml-build: add function for i2c slave device serial bus description Jonathan Cameron via
` (10 subsequent siblings)
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
By moving the parts of the mailbox command handling that are
CCI type specific out to the caller, make the main handling
code generic. Rename it to cxl_process_cci_message() to
reflect this new generality.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/cxl/cxl_device.h | 5 ++++-
hw/cxl/cxl-device-utils.c | 43 ++++++++++++++++++++++++++++++++++++-
hw/cxl/cxl-mailbox-utils.c | 42 ++++++++----------------------------
3 files changed, 55 insertions(+), 35 deletions(-)
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 813d4b926c..962b2bee7c 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -293,7 +293,10 @@ CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MEMORY_DEVICE,
void cxl_initialize_mailbox_t3(CXLCCI *cci, DeviceState *d, size_t payload_max);
void cxl_init_cci(CXLCCI *cci, size_t payload_max);
-void cxl_process_mailbox(CXLCCI *cci);
+int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
+ size_t len_in, uint8_t *pl_in,
+ size_t *len_out, uint8_t *pl_out,
+ bool *bg_started);
#define cxl_device_cap_init(dstate, reg, cap_id, ver) \
do { \
diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
index caf61ab1f5..7dbc8b72d7 100644
--- a/hw/cxl/cxl-device-utils.c
+++ b/hw/cxl/cxl-device-utils.c
@@ -76,6 +76,22 @@ static uint64_t mailbox_reg_read(void *opaque, hwaddr offset, unsigned size)
case 4:
return cxl_dstate->mbox_reg_state32[offset / size];
case 8:
+ if (offset == A_CXL_DEV_BG_CMD_STS) {
+ uint64_t bg_status_reg;
+ bg_status_reg = FIELD_DP64(0, CXL_DEV_BG_CMD_STS, OP, cci->bg.opcode);
+ bg_status_reg = FIELD_DP64(bg_status_reg, CXL_DEV_BG_CMD_STS,
+ PERCENTAGE_COMP, cci->bg.complete_pct);
+ bg_status_reg = FIELD_DP64(bg_status_reg, CXL_DEV_BG_CMD_STS,
+ RET_CODE, cci->bg.ret_code);
+ cxl_dstate->mbox_reg_state64[offset / size] = bg_status_reg; /* endian? */
+ }
+ if (offset == A_CXL_DEV_MAILBOX_STS) {
+ uint64_t status_reg = cxl_dstate->mbox_reg_state64[offset / size];
+ if (cci->bg.complete_pct) {
+ status_reg = FIELD_DP64(status_reg, CXL_DEV_MAILBOX_STS, BG_OP, 0);
+ cxl_dstate->mbox_reg_state64[offset / size] = status_reg;
+ }
+ }
return cxl_dstate->mbox_reg_state64[offset / size];
default:
g_assert_not_reached();
@@ -154,7 +170,32 @@ static void mailbox_reg_write(void *opaque, hwaddr offset, uint64_t value,
if (ARRAY_FIELD_EX32(cxl_dstate->mbox_reg_state32, CXL_DEV_MAILBOX_CTRL,
DOORBELL)) {
- cxl_process_mailbox(cci);
+ uint64_t command_reg = cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_CMD];
+ uint8_t cmd_set = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND_SET);
+ uint8_t cmd = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND);
+ size_t len_in = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, LENGTH);
+ uint8_t *pl = cxl_dstate->mbox_reg_state + A_CXL_DEV_CMD_PAYLOAD;
+ size_t len_out;
+ uint64_t status_reg;
+ bool bg_started;
+ int rc;
+
+ rc = cxl_process_cci_message(cci, cmd_set, cmd, len_in, pl,
+ &len_out, pl, &bg_started);
+
+ /* Set bg and the return code */
+ status_reg = FIELD_DP64(0, CXL_DEV_MAILBOX_STS, BG_OP, bg_started ? 1 : 0);
+ status_reg = FIELD_DP64(status_reg, CXL_DEV_MAILBOX_STS, ERRNO, rc);
+ /* Set the return length */
+ command_reg = FIELD_DP64(0, CXL_DEV_MAILBOX_CMD, COMMAND_SET, cmd_set);
+ command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND, cmd);
+ command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD, LENGTH, len_out);
+
+ cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_CMD] = command_reg;
+ cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_STS] = status_reg;
+ /* Tell the host we're done */
+ ARRAY_FIELD_DP32(cxl_dstate->mbox_reg_state32, CXL_DEV_MAILBOX_CTRL,
+ DOORBELL, 0);
}
}
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 399df6e3af..2db1258946 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -748,50 +748,26 @@ static const struct cxl_cmd cxl_cmd_set[256][256] = {
cmd_media_clear_poison, 72, 0 },
};
-void cxl_process_mailbox(CXLCCI *cci)
+int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
+ size_t len_in, uint8_t *pl_in, size_t *len_out,
+ uint8_t *pl_out, bool *bg_started)
{
- uint16_t ret = CXL_MBOX_SUCCESS;
const struct cxl_cmd *cxl_cmd;
- uint64_t status_reg = 0;
opcode_handler h;
- CXLDeviceState *cxl_dstate = &CXL_TYPE3(cci->d)->cxl_dstate;
- uint64_t command_reg = cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_CMD];
-
- uint8_t set = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND_SET);
- uint8_t cmd = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND);
- uint16_t len_in = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, LENGTH);
- uint8_t *pl = cxl_dstate->mbox_reg_state + A_CXL_DEV_CMD_PAYLOAD;
- size_t len_out = 0;
cxl_cmd = &cci->cxl_cmd_set[set][cmd];
h = cxl_cmd->handler;
- if (h) {
- if (len_in == cxl_cmd->in || cxl_cmd->in == ~0) {
- ret = (*h)(cxl_cmd, pl, len_in, pl, &len_out, cci);
- assert(len_out <= cci->payload_max);
- } else {
- ret = CXL_MBOX_INVALID_PAYLOAD_LENGTH;
- }
- } else {
+ if (!h) {
qemu_log_mask(LOG_UNIMP, "Command %04xh not implemented\n",
set << 8 | cmd);
- ret = CXL_MBOX_UNSUPPORTED;
+ return CXL_MBOX_UNSUPPORTED;
}
- /* Set the return code */
- status_reg = FIELD_DP64(0, CXL_DEV_MAILBOX_STS, ERRNO, ret);
-
- /* Set the return length */
- command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND_SET, 0);
- command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND, 0);
- command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD, LENGTH, len_out);
-
- cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_CMD] = command_reg;
- cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_STS] = status_reg;
+ if (len_in != cxl_cmd->in && cxl_cmd->in != ~0) {
+ return CXL_MBOX_INVALID_PAYLOAD_LENGTH;
+ }
- /* Tell the host we're done */
- ARRAY_FIELD_DP32(cxl_dstate->mbox_reg_state32, CXL_DEV_MAILBOX_CTRL,
- DOORBELL, 0);
+ return (*h)(cxl_cmd, pl_in, len_in, pl_out, len_out, cci);
}
void cxl_init_cci(CXLCCI *cci, size_t payload_max)
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 07/17] hw/acpi/aml-build: add function for i2c slave device serial bus description
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (5 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 06/17] cxl/mbox: Generalize the CCI command processing Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 08/17] hw/i2c: add mctp core Jonathan Cameron via
` (9 subsequent siblings)
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
Needed for later patches that add MCTP over I2C support to both
x86 and ARM boards.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/acpi/aml-build.h | 1 +
hw/acpi/aml-build.c | 17 +++++++++++++++++
2 files changed, 18 insertions(+)
diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
index fc2b949fb5..28db028b17 100644
--- a/include/hw/acpi/aml-build.h
+++ b/include/hw/acpi/aml-build.h
@@ -382,6 +382,7 @@ Aml *aml_dma(AmlDmaType typ, AmlDmaBusMaster bm, AmlTransferSize sz,
uint8_t channel);
Aml *aml_sleep(uint64_t msec);
Aml *aml_i2c_serial_bus_device(uint16_t address, const char *resource_source);
+Aml *aml_i2c_slv_serial_bus_device(uint16_t address, const char *resource_source);
/* Block AML object primitives */
Aml *aml_scope(const char *name_format, ...) G_GNUC_PRINTF(1, 2);
diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index 918cbb5b9d..22d7ecd753 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -2478,3 +2478,20 @@ Aml *aml_i2c_serial_bus_device(uint16_t address, const char *resource_source)
return var;
}
+
+/* ACPI 5.0: 6.4.3.8.2.1 I2C Serial Bus Connection Resource Descriptor */
+Aml *aml_i2c_slv_serial_bus_device(uint16_t address, const char *resource_source)
+{
+ uint16_t resource_source_len = strlen(resource_source) + 1;
+ Aml *var = aml_serial_bus_device(AML_SERIAL_BUS_TYPE_I2C, 1, 0, 1,
+ 6, resource_source_len);
+
+ /* Connection Speed. Just set to 100K for now, it doesn't really matter. */
+ build_append_int_noprefix(var->buf, 100000, 4);
+ build_append_int_noprefix(var->buf, address, sizeof(address));
+
+ /* This is a string, not a name, so just copy it directly in. */
+ g_array_append_vals(var->buf, resource_source, resource_source_len);
+
+ return var;
+}
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 08/17] hw/i2c: add mctp core
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (6 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 07/17] hw/acpi/aml-build: add function for i2c slave device serial bus description Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 09/17] i2c/mctp: Allow receiving messages to dest eid 0 Jonathan Cameron via
` (8 subsequent siblings)
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
From: Klaus Jensen <k.jensen@samsung.com>
Add an abstract MCTP over I2C endpoint model. This implements MCTP
control message handling as well as handling the actual I2C transport
(packetization).
Devices are intended to derive from this and implement the class
methods.
Parts of this implementation is inspired by code[1] previously posted by
Jonathan Cameron.
[1]: https://lore.kernel.org/qemu-devel/20220520170128.4436-1-Jonathan.Cameron@huawei.com/
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
MAINTAINERS | 7 +
include/hw/i2c/mctp.h | 114 +++++++++++
include/hw/i2c/smbus_master.h | 3 +
include/net/mctp.h | 43 +++++
hw/i2c/mctp.c | 352 ++++++++++++++++++++++++++++++++++
hw/i2c/smbus_master.c | 28 +++
hw/arm/Kconfig | 1 +
hw/i2c/Kconfig | 4 +
hw/i2c/meson.build | 1 +
hw/i2c/trace-events | 12 ++
10 files changed, 565 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 12e59b6b27..02a5bce814 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3386,6 +3386,13 @@ F: tests/qtest/adm1272-test.c
F: tests/qtest/max34451-test.c
F: tests/qtest/isl_pmbus_vr-test.c
+MCTP I2C Transport
+M: Klaus Jensen <k.jensen@samsung.com>
+S: Maintained
+F: hw/i2c/mctp.c
+F: include/hw/i2c/mctp.h
+F: include/net/mctp.h
+
Firmware schema specifications
M: Philippe Mathieu-Daudé <philmd@linaro.org>
R: Daniel P. Berrange <berrange@redhat.com>
diff --git a/include/hw/i2c/mctp.h b/include/hw/i2c/mctp.h
new file mode 100644
index 0000000000..c53ee6a3b6
--- /dev/null
+++ b/include/hw/i2c/mctp.h
@@ -0,0 +1,114 @@
+#ifndef QEMU_I2C_MCTP_H
+#define QEMU_I2C_MCTP_H
+
+#include "qom/object.h"
+#include "hw/qdev-core.h"
+#include "net/mctp.h"
+
+typedef struct MCTPI2CPacketHeader {
+ uint8_t dest;
+ uint8_t prot;
+ uint8_t byte_count;
+ uint8_t source;
+} MCTPI2CPacketHeader;
+
+typedef struct MCTPI2CPacket {
+ MCTPI2CPacketHeader i2c;
+ MCTPPacket mctp;
+} MCTPI2CPacket;
+
+#define i2c_mctp_payload(buf) (buf + offsetof(MCTPI2CPacket, mctp.payload))
+
+#define TYPE_MCTP_I2C_ENDPOINT "mctp-i2c-endpoint"
+OBJECT_DECLARE_TYPE(MCTPI2CEndpoint, MCTPI2CEndpointClass, MCTP_I2C_ENDPOINT)
+
+struct MCTPI2CEndpointClass {
+ I2CSlaveClass parent_class;
+
+ int (*put_message_bytes)(MCTPI2CEndpoint *mctp, uint8_t *buf, size_t len);
+ size_t (*get_message_bytes)(MCTPI2CEndpoint *mctp, uint8_t *buf,
+ size_t maxlen, uint8_t *mctp_flags);
+
+ void (*handle_message)(MCTPI2CEndpoint *mctp);
+ void (*reset_message)(MCTPI2CEndpoint *mctp);
+
+ size_t (*get_message_types)(MCTPI2CEndpoint *mctp, uint8_t *data,
+ size_t maxlen);
+};
+
+/*
+ * Maximum value of the SMBus Block Write "Byte Count" field (8 bits).
+ *
+ * This is the count of bytes that follow the Byte Count field and up to, but
+ * not including, the PEC byte.
+ */
+#define I2C_MCTP_MAXBLOCK 255
+
+/*
+ * Maximum Transmission Unit under I2C.
+ *
+ * This is for the MCTP Packet Payload (255, subtracting the 4 byte MCTP Packet
+ * Header or the 1 byte MCTP/I2C piggy-backed source address).
+ */
+#define I2C_MCTP_MAXMTU (I2C_MCTP_MAXBLOCK - (sizeof(MCTPPacketHeader) + 1))
+
+/*
+ * Maximum length of an MCTP/I2C packet.
+ *
+ * This is the sum of the three I2C header bytes (Destination target address,
+ * Command Code and Byte Count), the maximum number of bytes in a message (255)
+ * and the 1 byte Packet Error Code.
+ */
+#define I2C_MCTP_MAX_LENGTH (3 + I2C_MCTP_MAXBLOCK + 1)
+
+/*
+ * Maximum length of an MCTP/I2C Control Message.
+ *
+ * This is the 64 byte MCTP Baseline Maximum Transmission Unit, adding the
+ * combined MCTP/I2C headers and the trailing 1 byte PEC.
+ */
+#define I2C_MCTP_CONTROL_MAX_LENGTH \
+ (sizeof(MCTPI2CPacket) + MCTP_BASELINE_MTU + 1)
+
+typedef enum {
+ I2C_MCTP_STATE_IDLE,
+ I2C_MCTP_STATE_RX_STARTED,
+ I2C_MCTP_STATE_RX,
+ I2C_MCTP_STATE_WAIT_TX,
+ I2C_MCTP_STATE_TX,
+} MCTPState;
+
+typedef enum {
+ I2C_MCTP_STATE_TX_START_SEND,
+ I2C_MCTP_STATE_TX_SEND_BYTE,
+} MCTPTxState;
+
+typedef struct MCTPI2CEndpoint {
+ I2CSlave parent_obj;
+ I2CBus *i2c;
+
+ MCTPState state;
+
+ /* mctp endpoint identifier */
+ uint8_t my_eid;
+
+ uint8_t buffer[I2C_MCTP_MAX_LENGTH];
+ uint64_t pos;
+ size_t len;
+
+ struct {
+ MCTPTxState state;
+ bool is_control;
+
+ uint8_t eid;
+ uint8_t addr;
+ uint8_t pktseq;
+ uint8_t flags;
+
+ QEMUBH *bh;
+ } tx;
+} MCTPI2CEndpoint;
+
+void i2c_mctp_schedule_send(MCTPI2CEndpoint *mctp);
+
+#endif /* QEMU_I2C_MCTP_H */
diff --git a/include/hw/i2c/smbus_master.h b/include/hw/i2c/smbus_master.h
index bb13bc423c..ea5eff3a2c 100644
--- a/include/hw/i2c/smbus_master.h
+++ b/include/hw/i2c/smbus_master.h
@@ -27,6 +27,9 @@
#include "hw/i2c/i2c.h"
+/* SMBus PEC */
+uint8_t i2c_smbus_pec(uint8_t crc, uint8_t *buf, size_t len);
+
/* Master device commands. */
int smbus_quick_command(I2CBus *bus, uint8_t addr, int read);
int smbus_receive_byte(I2CBus *bus, uint8_t addr);
diff --git a/include/net/mctp.h b/include/net/mctp.h
new file mode 100644
index 0000000000..c936224ecf
--- /dev/null
+++ b/include/net/mctp.h
@@ -0,0 +1,43 @@
+#ifndef QEMU_MCTP_H
+#define QEMU_MCTP_H
+
+#define MCTP_BASELINE_MTU 64
+
+enum {
+ MCTP_H_FLAGS_EOM = 1 << 6,
+ MCTP_H_FLAGS_SOM = 1 << 7,
+};
+
+enum {
+ MCTP_MESSAGE_TYPE_CONTROL = 0x0,
+ MCTP_MESSAGE_TYPE_NMI = 0x4,
+
+ MCTP_MESSAGE_IC = 1 << 7,
+};
+
+typedef struct MCTPPacketHeader {
+ uint8_t version;
+ struct {
+ uint8_t dest;
+ uint8_t source;
+ } eid;
+ uint8_t flags;
+} MCTPPacketHeader;
+
+typedef struct MCTPPacket {
+ MCTPPacketHeader hdr;
+ uint8_t payload[];
+} MCTPPacket;
+
+typedef struct MCTPControlMessage {
+ uint8_t type;
+ uint8_t flags;
+ uint8_t command;
+ uint8_t data[];
+} MCTPControlMessage;
+
+enum {
+ MCTP_CONTROL_ERROR_UNSUPPORTED_CMD = 0x5,
+};
+
+#endif /* QEMU_MCTP_H */
diff --git a/hw/i2c/mctp.c b/hw/i2c/mctp.c
new file mode 100644
index 0000000000..0f4045d0d6
--- /dev/null
+++ b/hw/i2c/mctp.c
@@ -0,0 +1,352 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * SPDX-FileCopyrightText: Copyright (c) 2022 Samsung Electronics Co., Ltd.
+ * SPDX-FileContributor: Klaus Jensen <k.jensen@samsung.com>
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/main-loop.h"
+
+#include "hw/qdev-properties.h"
+#include "hw/i2c/i2c.h"
+#include "hw/i2c/smbus_master.h"
+#include "hw/i2c/mctp.h"
+
+#include "trace.h"
+
+void i2c_mctp_schedule_send(MCTPI2CEndpoint *mctp)
+{
+ I2CBus *i2c = I2C_BUS(qdev_get_parent_bus(DEVICE(mctp)));
+
+ mctp->tx.state = I2C_MCTP_STATE_TX_START_SEND;
+
+ i2c_bus_master(i2c, mctp->tx.bh);
+}
+
+static void i2c_mctp_tx(void *opaque)
+{
+ DeviceState *dev = DEVICE(opaque);
+ I2CBus *i2c = I2C_BUS(qdev_get_parent_bus(dev));
+ I2CSlave *slave = I2C_SLAVE(dev);
+ MCTPI2CEndpoint *mctp = MCTP_I2C_ENDPOINT(dev);
+ MCTPI2CEndpointClass *mc = MCTP_I2C_ENDPOINT_GET_CLASS(mctp);
+ MCTPI2CPacket *pkt = (MCTPI2CPacket *)mctp->buffer;
+ uint8_t flags = 0;
+
+ switch (mctp->tx.state) {
+ case I2C_MCTP_STATE_TX_SEND_BYTE:
+ if (mctp->pos < mctp->len) {
+ uint8_t byte = mctp->buffer[mctp->pos];
+
+ trace_i2c_mctp_tx_send_byte(mctp->pos, byte);
+
+ /* send next byte */
+ i2c_send_async(i2c, byte);
+
+ mctp->pos++;
+
+ break;
+ }
+
+ /* packet sent */
+ i2c_end_transfer(i2c);
+
+ /* end of any control data */
+ mctp->len = 0;
+
+ /* fall through */
+
+ case I2C_MCTP_STATE_TX_START_SEND:
+ if (mctp->tx.is_control) {
+ /* packet payload is already in buffer */
+ flags |= MCTP_H_FLAGS_SOM | MCTP_H_FLAGS_EOM;
+ } else {
+ /* get message bytes from derived device */
+ mctp->len = mc->get_message_bytes(mctp, pkt->mctp.payload,
+ I2C_MCTP_MAXMTU, &flags);
+ }
+
+ if (!mctp->len) {
+ trace_i2c_mctp_tx_done();
+
+ /* no more packets needed; release the bus */
+ i2c_bus_release(i2c);
+
+ mctp->state = I2C_MCTP_STATE_IDLE;
+ mctp->tx.is_control = false;
+
+ break;
+ }
+
+ mctp->state = I2C_MCTP_STATE_TX;
+
+ pkt->i2c = (MCTPI2CPacketHeader) {
+ .dest = mctp->tx.addr & ~0x1,
+ .prot = 0xf,
+ .byte_count = 5 + mctp->len,
+ .source = slave->address << 1 | 0x1,
+ };
+
+ pkt->mctp.hdr = (MCTPPacketHeader) {
+ .version = 0x1,
+ .eid.dest = mctp->tx.eid,
+ .eid.source = mctp->my_eid,
+ .flags = flags | (mctp->tx.pktseq++ & 0x3) << 4 | mctp->tx.flags,
+ };
+
+ mctp->len += sizeof(MCTPI2CPacket);
+ assert(mctp->len < I2C_MCTP_MAX_LENGTH);
+
+ mctp->buffer[mctp->len] = i2c_smbus_pec(0, mctp->buffer, mctp->len);
+ mctp->len++;
+
+ trace_i2c_mctp_tx_start_send(mctp->len);
+
+ i2c_start_send_async(i2c, pkt->i2c.dest >> 1);
+
+ /* already "sent" the destination slave address */
+ mctp->pos = 1;
+
+ mctp->tx.state = I2C_MCTP_STATE_TX_SEND_BYTE;
+
+ break;
+ }
+}
+
+#define i2c_mctp_control_data(buf) \
+ (i2c_mctp_payload(buf) + offsetof(MCTPControlMessage, data))
+
+static void i2c_mctp_handle_control_set_eid(MCTPI2CEndpoint *mctp, uint8_t eid)
+{
+ mctp->my_eid = eid;
+
+ uint8_t buf[] = {
+ 0x0, 0x0, eid, 0x0,
+ };
+
+ memcpy(i2c_mctp_control_data(mctp->buffer), buf, sizeof(buf));
+ mctp->len += sizeof(buf);
+}
+
+static void i2c_mctp_handle_control_get_eid(MCTPI2CEndpoint *mctp)
+{
+ uint8_t buf[] = {
+ 0x0, mctp->my_eid, 0x0, 0x0,
+ };
+
+ memcpy(i2c_mctp_control_data(mctp->buffer), buf, sizeof(buf));
+ mctp->len += sizeof(buf);
+}
+
+static void i2c_mctp_handle_control_get_version(MCTPI2CEndpoint *mctp)
+{
+ uint8_t buf[] = {
+ 0x0, 0x1, 0x0, 0x1, 0x3, 0x1,
+ };
+
+ memcpy(i2c_mctp_control_data(mctp->buffer), buf, sizeof(buf));
+ mctp->len += sizeof(buf);
+}
+
+enum {
+ MCTP_CONTROL_SET_EID = 0x01,
+ MCTP_CONTROL_GET_EID = 0x02,
+ MCTP_CONTROL_GET_VERSION = 0x04,
+ MCTP_CONTROL_GET_MESSAGE_TYPE_SUPPORT = 0x05,
+};
+
+static void i2c_mctp_handle_control(MCTPI2CEndpoint *mctp)
+{
+ MCTPI2CEndpointClass *mc = MCTP_I2C_ENDPOINT_GET_CLASS(mctp);
+ MCTPControlMessage *msg = (MCTPControlMessage *)i2c_mctp_payload(mctp->buffer);
+
+ /* clear Rq/D */
+ msg->flags &= 0x1f;
+
+ mctp->len = sizeof(MCTPControlMessage);
+
+ trace_i2c_mctp_handle_control(msg->command);
+
+ switch (msg->command) {
+ case MCTP_CONTROL_SET_EID:
+ i2c_mctp_handle_control_set_eid(mctp, msg->data[1]);
+ break;
+
+ case MCTP_CONTROL_GET_EID:
+ i2c_mctp_handle_control_get_eid(mctp);
+ break;
+
+ case MCTP_CONTROL_GET_VERSION:
+ i2c_mctp_handle_control_get_version(mctp);
+ break;
+
+ case MCTP_CONTROL_GET_MESSAGE_TYPE_SUPPORT:
+ mctp->len += mc->get_message_types(mctp, i2c_mctp_control_data(mctp->buffer),
+ MCTP_BASELINE_MTU - mctp->len);
+ break;
+
+ default:
+ trace_i2c_mctp_unhandled_control(msg->command);
+
+ msg->data[0] = MCTP_CONTROL_ERROR_UNSUPPORTED_CMD;
+ mctp->len++;
+
+ break;
+ }
+
+ assert(mctp->len <= MCTP_BASELINE_MTU);
+
+ i2c_mctp_schedule_send(mctp);
+}
+
+static int i2c_mctp_event_cb(I2CSlave *i2c, enum i2c_event event)
+{
+ MCTPI2CEndpoint *mctp = MCTP_I2C_ENDPOINT(i2c);
+ MCTPI2CEndpointClass *mc = MCTP_I2C_ENDPOINT_GET_CLASS(mctp);
+ MCTPI2CPacket *pkt = (MCTPI2CPacket *)mctp->buffer;
+ size_t payload_len;
+ uint8_t pec;
+
+ switch (event) {
+ case I2C_START_SEND:
+ if (mctp->state == I2C_MCTP_STATE_IDLE) {
+ mctp->state = I2C_MCTP_STATE_RX_STARTED;
+ } else if (mctp->state != I2C_MCTP_STATE_RX) {
+ return -1;
+ }
+
+ /* the i2c core eats the slave address, so put it back in */
+ pkt->i2c.dest = i2c->address << 1;
+ mctp->len = 1;
+
+ return 0;
+
+ case I2C_FINISH:
+ if (mctp->len < sizeof(MCTPI2CPacket) + 1) {
+ trace_i2c_mctp_drop("short packet");
+ goto drop;
+ }
+
+ payload_len = mctp->len - (1 + offsetof(MCTPI2CPacket, mctp.payload));
+
+ if (pkt->i2c.byte_count + 3 != mctp->len - 1) {
+ trace_i2c_mctp_drop_invalid_length(pkt->i2c.byte_count + 3,
+ mctp->len - 1);
+ goto drop;
+ }
+
+ pec = i2c_smbus_pec(0, mctp->buffer, mctp->len - 1);
+ if (mctp->buffer[mctp->len - 1] != pec) {
+ trace_i2c_mctp_drop_invalid_pec(mctp->buffer[mctp->len - 1], pec);
+ goto drop;
+ }
+
+ if (pkt->mctp.hdr.eid.dest != mctp->my_eid) {
+ trace_i2c_mctp_drop_invalid_eid(pkt->mctp.hdr.eid.dest,
+ mctp->my_eid);
+ goto drop;
+ }
+
+ if (pkt->mctp.hdr.flags & MCTP_H_FLAGS_SOM) {
+ mctp->tx.is_control = false;
+
+ if (mctp->state == I2C_MCTP_STATE_RX) {
+ mc->reset_message(mctp);
+ }
+
+ mctp->state = I2C_MCTP_STATE_RX;
+
+ mctp->tx.addr = pkt->i2c.source;
+ mctp->tx.eid = pkt->mctp.hdr.eid.source;
+ mctp->tx.flags = pkt->mctp.hdr.flags & 0x7;
+ mctp->tx.pktseq = (pkt->mctp.hdr.flags >> 4) & 0x3;
+
+ if ((pkt->mctp.payload[0] & 0x7f) == MCTP_MESSAGE_TYPE_CONTROL) {
+ mctp->tx.is_control = true;
+
+ i2c_mctp_handle_control(mctp);
+
+ return 0;
+ }
+ } else if (mctp->state == I2C_MCTP_STATE_RX_STARTED) {
+ trace_i2c_mctp_drop("expected SOM");
+ goto drop;
+ } else if (((pkt->mctp.hdr.flags >> 4) & 0x3) != (++mctp->tx.pktseq & 0x3)) {
+ trace_i2c_mctp_drop_invalid_pktseq((pkt->mctp.hdr.flags >> 4) & 0x3,
+ mctp->tx.pktseq & 0x3);
+ goto drop;
+ }
+
+ mc->put_message_bytes(mctp, i2c_mctp_payload(mctp->buffer), payload_len);
+
+ if (pkt->mctp.hdr.flags & MCTP_H_FLAGS_EOM) {
+ mc->handle_message(mctp);
+ mctp->state = I2C_MCTP_STATE_WAIT_TX;
+ }
+
+ return 0;
+
+ default:
+ return -1;
+ }
+
+drop:
+ mc->reset_message(mctp);
+
+ mctp->state = I2C_MCTP_STATE_IDLE;
+
+ return 0;
+}
+
+static int i2c_mctp_send_cb(I2CSlave *i2c, uint8_t data)
+{
+ MCTPI2CEndpoint *mctp = MCTP_I2C_ENDPOINT(i2c);
+
+ if (mctp->len < I2C_MCTP_MAX_LENGTH) {
+ mctp->buffer[mctp->len++] = data;
+ return 0;
+ }
+
+ return -1;
+}
+
+static void i2c_mctp_instance_init(Object *obj)
+{
+ MCTPI2CEndpoint *mctp = MCTP_I2C_ENDPOINT(obj);
+
+ mctp->tx.bh = qemu_bh_new(i2c_mctp_tx, mctp);
+}
+
+static Property mctp_i2c_props[] = {
+ DEFINE_PROP_UINT8("eid", MCTPI2CEndpoint, my_eid, 0x9),
+ DEFINE_PROP_END_OF_LIST(),
+};
+
+static void i2c_mctp_class_init(ObjectClass *oc, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+ I2CSlaveClass *k = I2C_SLAVE_CLASS(oc);
+
+ k->event = i2c_mctp_event_cb;
+ k->send = i2c_mctp_send_cb;
+
+ device_class_set_props(dc, mctp_i2c_props);
+}
+
+static const TypeInfo i2c_mctp_info = {
+ .name = TYPE_MCTP_I2C_ENDPOINT,
+ .parent = TYPE_I2C_SLAVE,
+ .abstract = true,
+ .instance_init = i2c_mctp_instance_init,
+ .instance_size = sizeof(MCTPI2CEndpoint),
+ .class_init = i2c_mctp_class_init,
+ .class_size = sizeof(MCTPI2CEndpointClass),
+};
+
+static void register_types(void)
+{
+ type_register_static(&i2c_mctp_info);
+}
+
+type_init(register_types)
diff --git a/hw/i2c/smbus_master.c b/hw/i2c/smbus_master.c
index 6a53c34e70..47f9eb24e0 100644
--- a/hw/i2c/smbus_master.c
+++ b/hw/i2c/smbus_master.c
@@ -15,6 +15,34 @@
#include "hw/i2c/i2c.h"
#include "hw/i2c/smbus_master.h"
+static uint8_t crc8(uint16_t data)
+{
+#define POLY (0x1070U << 3)
+ int i;
+
+ for (i = 0; i < 8; i++) {
+ if (data & 0x8000) {
+ data = data ^ POLY;
+ }
+
+ data = data << 1;
+ }
+
+ return (uint8_t)(data >> 8);
+#undef POLY
+}
+
+uint8_t i2c_smbus_pec(uint8_t crc, uint8_t *buf, size_t len)
+{
+ int i;
+
+ for (i = 0; i < len; i++) {
+ crc = crc8((crc ^ buf[i]) << 8);
+ }
+
+ return crc;
+}
+
/* Master device commands. */
int smbus_quick_command(I2CBus *bus, uint8_t addr, int read)
{
diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
index 7e68348440..3308d9e0bd 100644
--- a/hw/arm/Kconfig
+++ b/hw/arm/Kconfig
@@ -541,6 +541,7 @@ config ASPEED_SOC
select DS1338
select FTGMAC100
select I2C
+ select MCTP_I2C
select DPS310
select PCA9552
select SERIAL
diff --git a/hw/i2c/Kconfig b/hw/i2c/Kconfig
index 14886b35da..3415e8421a 100644
--- a/hw/i2c/Kconfig
+++ b/hw/i2c/Kconfig
@@ -45,3 +45,7 @@ config PCA954X
config PMBUS
bool
select SMBUS
+
+config MCTP_I2C
+ bool
+ select I2C
diff --git a/hw/i2c/meson.build b/hw/i2c/meson.build
index b58bc167db..d707ec58f7 100644
--- a/hw/i2c/meson.build
+++ b/hw/i2c/meson.build
@@ -1,5 +1,6 @@
i2c_ss = ss.source_set()
i2c_ss.add(when: 'CONFIG_I2C', if_true: files('core.c'))
+i2c_ss.add(when: 'CONFIG_MCTP_I2C', if_true: files('mctp.c'))
i2c_ss.add(when: 'CONFIG_SMBUS', if_true: files('smbus_slave.c', 'smbus_master.c'))
i2c_ss.add(when: 'CONFIG_ACPI_SMBUS', if_true: files('pm_smbus.c'))
i2c_ss.add(when: 'CONFIG_ACPI_ICH9', if_true: files('smbus_ich9.c'))
diff --git a/hw/i2c/trace-events b/hw/i2c/trace-events
index 8e88aa24c1..2e3065a998 100644
--- a/hw/i2c/trace-events
+++ b/hw/i2c/trace-events
@@ -45,3 +45,15 @@ npcm7xx_smbus_recv_fifo(const char *id, uint8_t received, uint8_t expected) "%s
pca954x_write_bytes(uint8_t value) "PCA954X write data: 0x%02x"
pca954x_read_data(uint8_t value) "PCA954X read data: 0x%02x"
+
+# mctp.c
+i2c_mctp_tx_start_send(size_t len) "len %zu"
+i2c_mctp_tx_send_byte(size_t pos, uint8_t byte) "pos %zu byte 0x%"PRIx8""
+i2c_mctp_tx_done(void) "packet sent"
+i2c_mctp_handle_control(uint8_t command) "command 0x%"PRIx8""
+i2c_mctp_unhandled_control(uint8_t command) "command 0x%"PRIx8""
+i2c_mctp_drop(const char *reason) "%s"
+i2c_mctp_drop_invalid_length(unsigned byte_count, size_t expected) "byte_count %u expected %zu"
+i2c_mctp_drop_invalid_pec(uint8_t pec, uint8_t expected) "pec 0x%"PRIx8" expected 0x%"PRIx8""
+i2c_mctp_drop_invalid_eid(uint8_t eid, uint8_t expected) "eid 0x%"PRIx8" expected 0x%"PRIx8""
+i2c_mctp_drop_invalid_pktseq(uint8_t pktseq, uint8_t expected) "pktseq 0x%"PRIx8" expected 0x%"PRIx8""
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 09/17] i2c/mctp: Allow receiving messages to dest eid 0
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (7 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 08/17] hw/i2c: add mctp core Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 10/17] misc/i2c_mctp_cxl: Initial device emulation Jonathan Cameron via
` (7 subsequent siblings)
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
From: Matt Johnston <matt@codeconstruct.com.au>
The Null Destination ID, 0, is used for MCTP control messages when
addressing by physical ID. That is used for Get Endpoint ID and
Set Endpoint ID when querying/assigning an EID to an endpoint.
Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
hw/i2c/mctp.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/hw/i2c/mctp.c b/hw/i2c/mctp.c
index 0f4045d0d6..db42dc7226 100644
--- a/hw/i2c/mctp.c
+++ b/hw/i2c/mctp.c
@@ -242,7 +242,8 @@ static int i2c_mctp_event_cb(I2CSlave *i2c, enum i2c_event event)
goto drop;
}
- if (pkt->mctp.hdr.eid.dest != mctp->my_eid) {
+ if (!(pkt->mctp.hdr.eid.dest == mctp->my_eid ||
+ pkt->mctp.hdr.eid.dest == 0)) {
trace_i2c_mctp_drop_invalid_eid(pkt->mctp.hdr.eid.dest,
mctp->my_eid);
goto drop;
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 10/17] misc/i2c_mctp_cxl: Initial device emulation
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (8 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 09/17] i2c/mctp: Allow receiving messages to dest eid 0 Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-18 21:30 ` Gregory Price
2023-07-17 17:16 ` [RFC PATCH 11/17] HACK: arm/virt: Add aspeed-i2c controller and MCTP EP to enable MCTP testing Jonathan Cameron via
` (6 subsequent siblings)
16 siblings, 1 reply; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
The CCI and Fabric Manager APIs are used to configure CXL switches and
devices. DMTF has defined an MCTP binding specification to carry
these messages. The end goal of this work is to hook this
up to emulated CXL switches and devices to allow control of the
configuration.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/cxl/cxl_device.h | 9 +-
include/hw/pci-bridge/cxl_upstream_port.h | 1 +
hw/cxl/cxl-mailbox-utils.c | 73 +++++++
hw/cxl/i2c_mctp_cxl.c | 237 ++++++++++++++++++++++
hw/arm/Kconfig | 2 +
hw/cxl/Kconfig | 3 +
hw/cxl/meson.build | 1 +
hw/i386/Kconfig | 2 +
8 files changed, 326 insertions(+), 2 deletions(-)
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 962b2bee7c..c49ff7e9b3 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -297,6 +297,10 @@ int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
size_t len_in, uint8_t *pl_in,
size_t *len_out, uint8_t *pl_out,
bool *bg_started);
+void cxl_initialize_t3_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
+ size_t payload_max);
+void cxl_initialize_usp_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
+ size_t payload_max);
#define cxl_device_cap_init(dstate, reg, cap_id, ver) \
do { \
@@ -397,8 +401,9 @@ struct CXLType3Dev {
AddressSpace hostpmem_as;
CXLComponentState cxl_cstate;
CXLDeviceState cxl_dstate;
- CXLCCI cci;
-
+ CXLCCI cci; /* Primary PCI mailbox CCI */
+ CXLCCI oob_mctp_cci; /* Initialized only if targetted */
+
/* DOE */
DOECap doe_cdat;
DOECap doe_comp;
diff --git a/include/hw/pci-bridge/cxl_upstream_port.h b/include/hw/pci-bridge/cxl_upstream_port.h
index b02aa8f659..15db8c8582 100644
--- a/include/hw/pci-bridge/cxl_upstream_port.h
+++ b/include/hw/pci-bridge/cxl_upstream_port.h
@@ -11,6 +11,7 @@ typedef struct CXLUpstreamPort {
/*< public >*/
CXLComponentState cxl_cstate;
+ CXLCCI mctpcci;
DOECap doe_cdat;
uint64_t sn;
} CXLUpstreamPort;
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 2db1258946..2d3d19dce8 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -45,6 +45,8 @@
*/
enum {
+ INFOSTAT = 0x00,
+ #define IS_IDENTIFY 0x1
EVENTS = 0x01,
#define GET_RECORDS 0x0
#define CLEAR_RECORDS 0x1
@@ -203,6 +205,52 @@ static CXLRetCode cmd_events_set_interrupt_policy(const struct cxl_cmd *cmd,
*len_out = 0;
return CXL_MBOX_SUCCESS;
}
+/* CXL r3 8.2.9.1.1 */
+static CXLRetCode cmd_infostat_identify(const struct cxl_cmd *cmd,
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLCCI *cci)
+{
+ PCIDeviceClass *class = PCI_DEVICE_GET_CLASS(cci->d);
+ struct {
+ uint16_t pcie_vid;
+ uint16_t pcie_did;
+ uint16_t pcie_subsys_vid;
+ uint16_t pcie_subsys_id;
+ uint64_t sn;
+ uint8_t max_message_size;
+ uint8_t component_type;
+ } QEMU_PACKED *is_identify;
+ QEMU_BUILD_BUG_ON(sizeof(*is_identify) != 18);
+
+ is_identify = (void *)payload_out;
+ memset(is_identify, 0, sizeof(*is_identify));
+ /*
+ * Messy question - which IDs? Those of the CCI Function, or those of
+ * the USP?
+ */
+ is_identify->pcie_vid = class->vendor_id;
+ is_identify->pcie_did = class->device_id;
+ if (object_dynamic_cast(OBJECT(cci->d), TYPE_CXL_USP)) {
+ is_identify->sn = CXL_USP(cci->d)->sn;
+ /* Subsystem info not defined for a USP */
+ is_identify->pcie_subsys_vid = 0;
+ is_identify->pcie_subsys_id = 0;
+ is_identify->component_type = 0x0; /* Switch */
+ } else if (object_dynamic_cast(OBJECT(cci->d), TYPE_CXL_TYPE3)) {
+ is_identify->sn = CXL_TYPE3(cci->d)->sn;
+ is_identify->pcie_subsys_vid = class->subsystem_vendor_id;
+ is_identify->pcie_subsys_id = class->subsystem_id;
+ is_identify->component_type = 0x3; /* Type 3 */
+ }
+
+ /* FIXME: This depends on interface */
+ is_identify->max_message_size = CXL_MAILBOX_PAYLOAD_SHIFT;
+ *len_out = sizeof(*is_identify);
+ return CXL_MBOX_SUCCESS;
+}
/* 8.2.9.2.1 */
static CXLRetCode cmd_firmware_update_get_info(const struct cxl_cmd *cmd,
@@ -797,3 +845,28 @@ void cxl_initialize_mailbox_t3(CXLCCI *cci, DeviceState *d, size_t payload_max)
cci->intf = d;
cxl_init_cci(cci, payload_max);
}
+
+static const struct cxl_cmd cxl_cmd_set_t3_mctp[256][256] = {
+ [INFOSTAT][IS_IDENTIFY] = { "IDENTIFY", cmd_infostat_identify, 0, 18 },
+};
+
+void cxl_initialize_t3_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
+ size_t payload_max)
+{
+ cci->cxl_cmd_set = cxl_cmd_set_t3_mctp;
+ cci->d = d;
+ cci->intf = intf;
+ cxl_init_cci(cci, payload_max);
+}
+
+static const struct cxl_cmd cxl_cmd_set_usp_mctp[256][256] = {
+ [INFOSTAT][IS_IDENTIFY] = { "IDENTIFY", cmd_infostat_identify, 0, 18 },
+};
+
+void cxl_initialize_usp_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf, size_t payload_max)
+{
+ cci->cxl_cmd_set = cxl_cmd_set_usp_mctp;
+ cci->d = d;
+ cci->intf = intf;
+ cxl_init_cci(cci, payload_max);
+}
diff --git a/hw/cxl/i2c_mctp_cxl.c b/hw/cxl/i2c_mctp_cxl.c
new file mode 100644
index 0000000000..58c3042aab
--- /dev/null
+++ b/hw/cxl/i2c_mctp_cxl.c
@@ -0,0 +1,237 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Emulation of a CXL Switch Fabric Management interface over MCTP over I2C.
+ *
+ * Copyright (c) 2023 Huawei Technologies.
+ *
+ * Reference list:
+ * From www.dmtf.org
+ * DSP0236 Management Component Transport Protocol (MCTP) Base Specification 1.3.0
+ * DPS0234 CXL Fabric Manager API over MCTP Binding Specification 1.0.0
+ * DSP0281 CXL Type 3 Deivce Component Command Interface over MCTP Binding
+ * Specification (note some commands apply to switches as well)
+ * From www.computeexpresslink.org
+ * Compute Express Link (CXL) Specification revision 3.0 Version 1.0
+ */
+
+#include "qemu/osdep.h"
+#include "hw/i2c/i2c.h"
+#include "hw/i2c/mctp.h"
+#include "hw/irq.h"
+#include "migration/vmstate.h"
+#include "qapi/error.h"
+#include "hw/cxl/cxl.h"
+#include "hw/pci-bridge/cxl_upstream_port.h"
+#include "hw/pci/pcie.h"
+#include "hw/pci/pcie_port.h"
+#include "hw/qdev-properties.h"
+
+#define TYPE_I2C_MCTP_CXL "i2c_mctp_cxl"
+
+#define MCTP_CXL_MAX_MSG_LEN 1088 /* CXL FMAPI binding spec */
+
+typedef struct CXLMCTPMessage {
+ /*
+ * DSP0236 (MCTP Base) Integrity Check + Message Type
+ * DSP0234/DSP0281 (CXL bindings) state no Integrity Check
+ * so just the message type.
+ */
+ uint8_t message_type;
+ /* Remaing fields from CXL r3.0 Table 7-14 CCI Message Format */
+ uint8_t category;
+ uint8_t tag;
+ uint8_t rsvd;
+ /*
+ * CXL r3.0 - Table 8-36 Generic Component Command Opcodes:
+ * Command opcode is split into two sub fields
+ */
+ uint8_t command;
+ uint8_t command_set;
+ uint8_t pl_length[3];
+ uint16_t vendor_status;
+ uint16_t rc;
+ uint8_t payload[];
+} QEMU_PACKED CXLMCTPMessage;
+
+enum cxl_dev_type {
+ cxl_type3,
+ cxl_switch,
+};
+
+struct I2C_MCTP_CXL_State {
+ MCTPI2CEndpoint mctp;
+ PCIDevice *target;
+ CXLCCI *cci;
+ enum cxl_dev_type type;
+ size_t len;
+ int64_t pos;
+ uint8_t buffer[MCTP_CXL_MAX_MSG_LEN];
+ uint8_t scratch[MCTP_CXL_MAX_MSG_LEN];
+};
+
+OBJECT_DECLARE_SIMPLE_TYPE(I2C_MCTP_CXL_State, I2C_MCTP_CXL)
+
+static Property i2c_mctp_cxl_props[] = {
+ DEFINE_PROP_LINK("target", I2C_MCTP_CXL_State,
+ target, TYPE_PCI_DEVICE, PCIDevice *),
+ DEFINE_PROP_END_OF_LIST(),
+};
+
+static size_t i2c_mctp_cxl_get_message_bytes(MCTPI2CEndpoint *mctp,
+ uint8_t *buf,
+ size_t maxlen,
+ uint8_t *mctp_flags)
+{
+ I2C_MCTP_CXL_State *s = I2C_MCTP_CXL(mctp);
+ size_t len;
+
+ len = MIN(maxlen, s->len - s->pos);
+
+ if (len == 0) {
+ return 0;
+ }
+
+ if (s->pos == 0) {
+ *mctp_flags |= MCTP_H_FLAGS_SOM;
+ }
+
+ memcpy(buf, s->scratch + s->pos, len);
+ s->pos += len;
+
+ if (s->pos == s->len) {
+ *mctp_flags |= MCTP_H_FLAGS_EOM;
+
+ s->pos = s->len = 0;
+ }
+
+ return len;
+}
+
+static int i2c_mctp_cxl_put_message_bytes(MCTPI2CEndpoint *mctp,
+ uint8_t *buf, size_t len)
+{
+ I2C_MCTP_CXL_State *s = I2C_MCTP_CXL(mctp);
+
+ if (s->len + len > MCTP_CXL_MAX_MSG_LEN) {
+ return -1;
+ }
+
+ memcpy(s->buffer + s->len, buf, len);
+ s->len += len;
+
+ return 0;
+}
+
+static size_t i2c_mctp_cxl_get_message_types(MCTPI2CEndpoint *mctp,
+ uint8_t *data,
+ size_t maxlen)
+{
+ uint8_t buf[] = {
+ 0x0, 0x7, 0x8, /* Control, CXL FM-API and CXL CCI */
+ };
+
+ memcpy(data, buf, sizeof(buf));
+
+ return sizeof(buf);
+}
+
+static void i2c_mctp_cxl_reset_message(MCTPI2CEndpoint *mctp)
+{
+ I2C_MCTP_CXL_State *s = I2C_MCTP_CXL(mctp);
+
+ s->len = 0;
+}
+
+static void i2c_mctp_cxl_handle_message(MCTPI2CEndpoint *mctp)
+{
+ I2C_MCTP_CXL_State *s = I2C_MCTP_CXL(mctp);
+ CXLMCTPMessage *msg = (CXLMCTPMessage *)s->buffer;
+ CXLMCTPMessage *buf = (CXLMCTPMessage *)s->scratch;
+
+ *buf = (CXLMCTPMessage) {
+ .message_type = msg->message_type,
+ .category = 1,
+ .tag = msg->tag,
+ };
+ s->pos = sizeof(buf);
+
+ /* To fix, msg->message_type currently ignored */
+ if (s->cci) {
+ bool bg_started;
+ int rc;
+ size_t len_out;
+ size_t len_in = msg->pl_length[2] << 16 | msg->pl_length[1] << 8 |
+ msg->pl_length[0]; //hack
+
+ rc = cxl_process_cci_message(s->cci, msg->command_set, msg->command,
+ len_in, msg->payload,
+ &len_out, s->scratch + sizeof(CXLMCTPMessage),
+ &bg_started);
+ buf->command = msg->command;
+ buf->command_set = msg->command_set;
+ buf->rc = rc;
+ s->len += len_out;
+ st24_le_p(buf->pl_length, len_out);
+ s->pos = 0;
+ i2c_mctp_schedule_send(mctp);
+ }
+}
+
+static void i2c_mctp_cxl_realize(DeviceState *d, Error **errp)
+{
+ I2C_MCTP_CXL_State *s = I2C_MCTP_CXL(d);
+
+ /* Check this is a type we support */
+ if (object_dynamic_cast(OBJECT(s->target), TYPE_CXL_USP)) {
+ CXLUpstreamPort *usp = CXL_USP(s->target);
+
+ s->type = cxl_switch;
+ s->cci = &usp->mctpcci;
+ /* Figure out right size */
+ cxl_initialize_usp_mctpcci(s->cci, DEVICE(s->target), d, 512);
+
+ return;
+ }
+
+ if (object_dynamic_cast(OBJECT(s->target), TYPE_CXL_TYPE3)) {
+ CXLType3Dev *ct3d = CXL_TYPE3(s->target);
+
+ s->type = cxl_type3;
+ s->cci = &ct3d->oob_mctp_cci;
+
+ cxl_initialize_t3_mctpcci(s->cci, DEVICE(s->target), d, 512);
+ return;
+ }
+
+ error_setg(errp, "Unhandled target type for CXL MCTP EP");
+}
+
+static void i2c_mctp_cxl_class_init(ObjectClass *klass, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(klass);
+ MCTPI2CEndpointClass *mc = MCTP_I2C_ENDPOINT_CLASS(klass);
+
+ dc->realize = i2c_mctp_cxl_realize;
+ mc->get_message_types = i2c_mctp_cxl_get_message_types;
+ mc->get_message_bytes = i2c_mctp_cxl_get_message_bytes;
+ mc->put_message_bytes = i2c_mctp_cxl_put_message_bytes;
+
+ mc->handle_message = i2c_mctp_cxl_handle_message;
+ mc->reset_message = i2c_mctp_cxl_reset_message;
+ device_class_set_props(dc, i2c_mctp_cxl_props);
+}
+
+static const TypeInfo i2c_mctp_cxl_info = {
+ .name = TYPE_I2C_MCTP_CXL,
+ .parent = TYPE_MCTP_I2C_ENDPOINT,
+ .instance_size = sizeof(I2C_MCTP_CXL_State),
+ .class_init = i2c_mctp_cxl_class_init,
+};
+
+static void i2c_mctp_cxl_register_types(void)
+{
+ type_register_static(&i2c_mctp_cxl_info);
+}
+
+type_init(i2c_mctp_cxl_register_types)
diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
index 3308d9e0bd..f2415e0d44 100644
--- a/hw/arm/Kconfig
+++ b/hw/arm/Kconfig
@@ -32,6 +32,8 @@ config ARM_VIRT
select VIRTIO_MEM_SUPPORTED
select ACPI_CXL
select ACPI_HMAT
+ select MCTP_I2C
+ select I2C_MCTP_CXL
config CHEETAH
bool
diff --git a/hw/cxl/Kconfig b/hw/cxl/Kconfig
index 8e67519b16..c9b2e46bac 100644
--- a/hw/cxl/Kconfig
+++ b/hw/cxl/Kconfig
@@ -1,3 +1,6 @@
config CXL
bool
default y if PCI_EXPRESS
+
+config I2C_MCTP_CXL
+ bool
diff --git a/hw/cxl/meson.build b/hw/cxl/meson.build
index 9024e9a9a6..7a885e7b84 100644
--- a/hw/cxl/meson.build
+++ b/hw/cxl/meson.build
@@ -11,5 +11,6 @@ system_ss.add(when: 'CONFIG_CXL',
if_false: files(
'cxl-host-stubs.c',
))
+system_ss.add(when: 'CONFIG_I2C_MCTP_CXL', if_true: files('i2c_mctp_cxl.c'))
system_ss.add(when: 'CONFIG_ALL', if_true: files('cxl-host-stubs.c'))
diff --git a/hw/i386/Kconfig b/hw/i386/Kconfig
index 9051083c1e..9644e81254 100644
--- a/hw/i386/Kconfig
+++ b/hw/i386/Kconfig
@@ -45,6 +45,8 @@ config PC
select ACPI_VMGENID
select VIRTIO_PMEM_SUPPORTED
select VIRTIO_MEM_SUPPORTED
+ select MCTP_I2C
+ select I2C_MCTP_CXL
config PC_PCI
bool
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 11/17] HACK: arm/virt: Add aspeed-i2c controller and MCTP EP to enable MCTP testing
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (9 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 10/17] misc/i2c_mctp_cxl: Initial device emulation Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 12/17] HACK: hw/arm/virt: Add ACPI support for aspeed-i2c / mctp Jonathan Cameron via
` (5 subsequent siblings)
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
As the only I2C emulation in QEMU that supports being both
a master and a slave, suitable for MCTP over i2c is aspeed-i2c
add this controller to the arm virt model and hook up our new
i2c_mctp_cxl_fmapi device.
The current Linux driver for aspeed-i2c has a hard requirement on
a reset controller. Throw down the simplest reset controller
I could find so as to avoid need to make any changes to the kernel
code.
Patch also builds appropriate device tree.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/arm/virt.h | 2 +
hw/arm/virt.c | 86 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 88 insertions(+)
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index 9fc582fc5f..ea3a64f4a8 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -72,6 +72,8 @@ enum {
VIRT_SMMU,
VIRT_UART,
VIRT_MMIO,
+ VIRT_I2C,
+ VIRT_RESET_FAKE,
VIRT_RTC,
VIRT_FW_CFG,
VIRT_PCIE,
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 5512ad4ba4..67e694550e 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -85,6 +85,8 @@
#include "hw/cxl/cxl.h"
#include "hw/cxl/cxl_host.h"
#include "qemu/guest-random.h"
+#include "hw/i2c/i2c.h"
+#include "hw/i2c/aspeed_i2c.h"
#define DEFINE_VIRT_MACHINE_LATEST(major, minor, latest) \
static void virt_##major##_##minor##_class_init(ObjectClass *oc, \
@@ -161,6 +163,8 @@ static const MemMapEntry base_memmap[] = {
[VIRT_PVTIME] = { 0x090a0000, 0x00010000 },
[VIRT_SECURE_GPIO] = { 0x090b0000, 0x00001000 },
[VIRT_MMIO] = { 0x0a000000, 0x00000200 },
+ [VIRT_I2C] = { 0x0b000000, 0x00004000 },
+ [VIRT_RESET_FAKE] = { 0x0b004000, 0x00000010 },
/* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */
[VIRT_PLATFORM_BUS] = { 0x0c000000, 0x02000000 },
[VIRT_SECURE_MEM] = { 0x0e000000, 0x01000000 },
@@ -203,6 +207,7 @@ static const int a15irqmap[] = {
[VIRT_GPIO] = 7,
[VIRT_SECURE_UART] = 8,
[VIRT_ACPI_GED] = 9,
+ [VIRT_I2C] = 10,
[VIRT_MMIO] = 16, /* ...to 16 + NUM_VIRTIO_TRANSPORTS - 1 */
[VIRT_GIC_V2M] = 48, /* ...to 48 + NUM_GICV2M_SPIS - 1 */
[VIRT_SMMU] = 74, /* ...to 74 + NUM_SMMU_IRQS - 1 */
@@ -2268,6 +2273,85 @@ static void virt_cpu_post_init(VirtMachineState *vms, MemoryRegion *sysmem)
}
}
+static void create_mctp(MachineState *ms)
+{
+ VirtMachineState *vms = VIRT_MACHINE(ms);
+ MemoryRegion *sysmem = get_system_memory();
+ AspeedI2CState *aspeedi2c;
+ struct DeviceState *dev;
+ char *nodename_i2c_master;
+ char *nodename_i2c_sub;
+ char *nodename_reset;
+ uint32_t clk_phandle, reset_phandle;
+ MemoryRegion *sysmem2;
+
+ dev = qdev_new("aspeed.i2c-ast2600");
+ aspeedi2c = ASPEED_I2C(dev);
+ object_property_set_link(OBJECT(dev), "dram", OBJECT(ms->ram),
+ &error_fatal);
+ sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
+ sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, vms->memmap[VIRT_I2C].base);
+ sysbus_connect_irq(SYS_BUS_DEVICE(&aspeedi2c->busses[0]), 0,
+ qdev_get_gpio_in(vms->gic, vms->irqmap[VIRT_I2C]));
+
+ /* I2C bus DT */
+ reset_phandle = qemu_fdt_alloc_phandle(ms->fdt);
+ nodename_reset = g_strdup_printf("/reset@%" PRIx64,
+ vms->memmap[VIRT_RESET_FAKE].base);
+ qemu_fdt_add_subnode(ms->fdt, nodename_reset);
+ qemu_fdt_setprop_string(ms->fdt, nodename_reset,
+ "compatible", "snps,dw-low-reset");
+ qemu_fdt_setprop_sized_cells(ms->fdt, nodename_reset, "reg",
+ 2, vms->memmap[VIRT_RESET_FAKE].base,
+ 2, vms->memmap[VIRT_RESET_FAKE].size);
+ qemu_fdt_setprop_cell(ms->fdt, nodename_reset, "#reset-cells", 0x1);
+ qemu_fdt_setprop_cell(ms->fdt, nodename_reset, "phandle", reset_phandle);
+ sysmem2 = g_new(MemoryRegion, 1);
+ memory_region_init_ram(sysmem2, NULL, "reset",
+ vms->memmap[VIRT_RESET_FAKE].size, NULL);
+ memory_region_add_subregion(sysmem,
+ vms->memmap[VIRT_RESET_FAKE].base, sysmem2);
+
+ clk_phandle = qemu_fdt_alloc_phandle(ms->fdt);
+
+ qemu_fdt_add_subnode(ms->fdt, "/mclk");
+ qemu_fdt_setprop_string(ms->fdt, "/mclk", "compatible", "fixed-clock");
+ qemu_fdt_setprop_cell(ms->fdt, "/mclk", "#clock-cells", 0x0);
+ qemu_fdt_setprop_cell(ms->fdt, "/mclk", "clock-frequency", 24000);
+ qemu_fdt_setprop_string(ms->fdt, "/mclk", "clock-output-names", "bobsclk");
+ qemu_fdt_setprop_cell(ms->fdt, "/mclk", "phandle", clk_phandle);
+
+ nodename_i2c_master = g_strdup_printf("/i2c@%" PRIx64,
+ vms->memmap[VIRT_I2C].base);
+ qemu_fdt_add_subnode(ms->fdt, nodename_i2c_master);
+ qemu_fdt_setprop_string(ms->fdt, nodename_i2c_master,
+ "compatible", "aspeed,ast2600-i2c-bus");
+ qemu_fdt_setprop_cells(ms->fdt, nodename_i2c_master, "multi-master");
+ qemu_fdt_setprop_cell(ms->fdt, nodename_i2c_master, "#size-cells", 0);
+ qemu_fdt_setprop_cell(ms->fdt, nodename_i2c_master, "#address-cells", 1);
+ qemu_fdt_setprop_cell(ms->fdt, nodename_i2c_master, "clocks", clk_phandle);
+ qemu_fdt_setprop_string(ms->fdt, nodename_i2c_master,
+ "clock-names", "bobsclk");
+ qemu_fdt_setprop(ms->fdt, nodename_i2c_master, "mctp-controller", NULL, 0);
+ qemu_fdt_setprop_cells(ms->fdt, nodename_i2c_master,
+ "interrupts", GIC_FDT_IRQ_TYPE_SPI,
+ vms->irqmap[VIRT_I2C], GIC_FDT_IRQ_FLAGS_LEVEL_HI);
+ /* Offset to the first bus is 0x80, next one at 0x100 etc */
+ qemu_fdt_setprop_sized_cells(ms->fdt, nodename_i2c_master, "reg",
+ 2, vms->memmap[VIRT_I2C].base + 0x80,
+ 2, 0x80);
+ qemu_fdt_setprop_cells(ms->fdt, nodename_i2c_master,
+ "resets", reset_phandle, 0);
+
+ nodename_i2c_sub = g_strdup_printf("/i2c@%" PRIx64 "/mctp@%" PRIx64,
+ vms->memmap[VIRT_I2C].base, 0x50l);
+ qemu_fdt_add_subnode(ms->fdt, nodename_i2c_sub);
+ qemu_fdt_setprop_string(ms->fdt, nodename_i2c_sub,
+ "compatible", "mctp-i2c-controller");
+ qemu_fdt_setprop_sized_cells(ms->fdt, nodename_i2c_sub,
+ "reg", 1, 0x50 | 0x40000000);
+}
+
static void machvirt_init(MachineState *machine)
{
VirtMachineState *vms = VIRT_MACHINE(machine);
@@ -2555,6 +2639,8 @@ static void machvirt_init(MachineState *machine)
create_gpio_devices(vms, VIRT_SECURE_GPIO, secure_sysmem);
}
+ create_mctp(machine);
+
/* connect powerdown request */
vms->powerdown_notifier.notify = virt_powerdown_req;
qemu_register_powerdown_notifier(&vms->powerdown_notifier);
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 12/17] HACK: hw/arm/virt: Add ACPI support for aspeed-i2c / mctp
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (10 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 11/17] HACK: arm/virt: Add aspeed-i2c controller and MCTP EP to enable MCTP testing Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 13/17] HACK: hw/i386/pc: Add Aspeed i2c controller + MCTP with ACPI tables Jonathan Cameron via
` (4 subsequent siblings)
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
Enable this for FM-API testing for CXL devices via MCTP over I2C
Example DSDT block:
Device (MCTP)
{
Name (_HID, "PRP0001") // _HID: Hardware ID
Name (_DSD, Package (0x02) // _DSD: Device-Specific Data
{
ToUUID ("daffd814-6eba-4d8c-8a91-bc9bbf4aa301") /* Device Properties for _DSD */,
Package (0x03)
{
Package (0x02)
{
"compatible",
"aspeed,ast2600-i2c-bus"
},
Package (0x02)
{
"bus-frequency",
0x00061A80
},
Package (0x02)
{
"mctp-controller",
One
}
}
})
Name (_CRS, ResourceTemplate () // _CRS: Current Resource Settings
{
Memory32Fixed (ReadWrite,
0x0B000080, // Address Base
0x00000080, // Address Length
)
Interrupt (ResourceConsumer, Level, ActiveHigh, Exclusive, ,, )
{
0x0000002A,
}
})
}
Device (MCTS)
{
Name (_HID, "PRP0001") // _HID: Hardware ID
Name (_CRS, ResourceTemplate () // _CRS: Current Resource Settings
{
I2cSerialBusV2 (0x0050, DeviceInitiated, 0x000186A0,
AddressingMode7Bit, "\\_SB.MCTP",
0x00, ResourceProducer, , Exclusive,
)
})
Name (_DSD, Package (0x02) // _DSD: Device-Specific Data
{
ToUUID ("daffd814-6eba-4d8c-8a91-bc9bbf4aa301") /* Device Properties for _DSD */,
Package (0x01)
{
Package (0x02)
{
"compatible",
"mctp-i2c-controller"
}
}
})
}
Tests not updated given I'm not currently proposing this for upstream.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
hw/arm/virt-acpi-build.c | 60 ++++++++++++++++++++++++++++++++++++++++
hw/i2c/meson.build | 2 +-
2 files changed, 61 insertions(+), 1 deletion(-)
diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index e4a74324b8..ad31e2055e 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -78,6 +78,65 @@ static void acpi_dsdt_add_cpus(Aml *scope, VirtMachineState *vms)
}
}
+static void acpi_dsdt_add_mctp(Aml *scope, VirtMachineState *vms)
+{
+ uint32_t interrupt = vms->irqmap[VIRT_I2C] + ARM_SPI_BASE;
+ Aml *main_dev = aml_device("MCTP");
+ Aml *sub_dev = aml_device("MCTS");
+ Aml *dsd_pkg = aml_package(2);
+ Aml *props_pkg = aml_package(3);
+ Aml *pkg = aml_package(2);
+ Aml *crs = aml_resource_template();
+
+ aml_append(main_dev, aml_name_decl("_HID", aml_string("PRP0001")));
+
+ aml_append(pkg, aml_string("compatible"));
+ aml_append(pkg, aml_string("aspeed,ast2600-i2c-bus"));
+ aml_append(props_pkg, pkg);
+
+ pkg = aml_package(2);
+ aml_append(pkg, aml_string("bus-frequency"));
+ aml_append(pkg, aml_int(400000));
+ aml_append(props_pkg, pkg);
+
+ pkg = aml_package(2);
+ aml_append(pkg, aml_string("mctp-controller"));
+ aml_append(pkg, aml_int(1));
+ aml_append(props_pkg, pkg);
+
+ aml_append(dsd_pkg, aml_touuid("DAFFD814-6EBA-4D8C-8A91-BC9BBF4AA301"));
+ aml_append(dsd_pkg, props_pkg);
+ aml_append(main_dev, aml_name_decl("_DSD", dsd_pkg));
+
+ aml_append(crs, aml_memory32_fixed(vms->memmap[VIRT_I2C].base + 0x80,
+ 0x80, AML_READ_WRITE));
+ aml_append(crs,
+ aml_interrupt(AML_CONSUMER, AML_LEVEL, AML_ACTIVE_HIGH,
+ AML_EXCLUSIVE, &interrupt, 1));
+ aml_append(main_dev, aml_name_decl("_CRS", crs));
+
+ aml_append(sub_dev, aml_name_decl("_HID", aml_string("PRP0001")));
+
+ dsd_pkg = aml_package(2);
+ aml_append(dsd_pkg, aml_touuid("DAFFD814-6EBA-4D8C-8A91-BC9BBF4AA301"));
+
+ props_pkg = aml_package(1);
+
+ pkg = aml_package(2);
+ aml_append(pkg, aml_string("compatible"));
+ aml_append(pkg, aml_string("mctp-i2c-controller"));
+ aml_append(props_pkg, pkg);
+ aml_append(dsd_pkg, props_pkg);
+
+ crs = aml_resource_template();
+ aml_append(crs, aml_i2c_slv_serial_bus_device(0x50, "\\_SB.MCTP"));
+ aml_append(sub_dev, aml_name_decl("_CRS", crs));
+ aml_append(sub_dev, aml_name_decl("_DSD", dsd_pkg));
+
+ aml_append(scope, main_dev);
+ aml_append(scope, sub_dev);
+}
+
static void acpi_dsdt_add_uart(Aml *scope, const MemMapEntry *uart_memmap,
uint32_t uart_irq)
{
@@ -888,6 +947,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
*/
scope = aml_scope("\\_SB");
acpi_dsdt_add_cpus(scope, vms);
+ acpi_dsdt_add_mctp(scope, vms);
acpi_dsdt_add_uart(scope, &memmap[VIRT_UART],
(irqmap[VIRT_UART] + ARM_SPI_BASE));
if (vmc->acpi_expose_flash) {
diff --git a/hw/i2c/meson.build b/hw/i2c/meson.build
index d707ec58f7..75db13c10e 100644
--- a/hw/i2c/meson.build
+++ b/hw/i2c/meson.build
@@ -4,7 +4,7 @@ i2c_ss.add(when: 'CONFIG_MCTP_I2C', if_true: files('mctp.c'))
i2c_ss.add(when: 'CONFIG_SMBUS', if_true: files('smbus_slave.c', 'smbus_master.c'))
i2c_ss.add(when: 'CONFIG_ACPI_SMBUS', if_true: files('pm_smbus.c'))
i2c_ss.add(when: 'CONFIG_ACPI_ICH9', if_true: files('smbus_ich9.c'))
-i2c_ss.add(when: 'CONFIG_ASPEED_SOC', if_true: files('aspeed_i2c.c'))
+i2c_ss.add(when: 'CONFIG_I2C', if_true: files('aspeed_i2c.c'))
i2c_ss.add(when: 'CONFIG_BITBANG_I2C', if_true: files('bitbang_i2c.c'))
i2c_ss.add(when: 'CONFIG_EXYNOS4', if_true: files('exynos4210_i2c.c'))
i2c_ss.add(when: 'CONFIG_IMX_I2C', if_true: files('imx_i2c.c'))
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 13/17] HACK: hw/i386/pc: Add Aspeed i2c controller + MCTP with ACPI tables
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (11 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 12/17] HACK: hw/arm/virt: Add ACPI support for aspeed-i2c / mctp Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 14/17] docs: cxl: Add example commandline for MCTP CXL CCIs Jonathan Cameron via
` (3 subsequent siblings)
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
CXL devices provide a standard Fabric Management API - FM-API.
See CXL specification r3.0 from https://www.computeexpresslink.org
In many real setups that will be used by a separate host from the
one actually using the CXL devices (BMC or similar) but it is
helpful to be able to use the main CXL emulation and the
Fabric Management emulation on a single host. This 'hack' enables
that (with minor kernel driver changes).
There are many many things wrong with how this is done but for
now it enables use of this aspeed controller with ACPI FW
on an x86 host. That is useful for testing MCTP over I2C.
If anyone has either:
1) Docs for an I2C controller with MCTP support that might actually
appear on an x86 host.
2) A nice solution for how wrap this up in a device whilst minimising
kernel changes.
3) A guide / reference example to how to do the interrupt 'right'
(I'm an ARM focused developer so got lost in the x86 interrupt
stuff).
then let me know.
For now this works and I will carry it out of tree on
gitlab.com/jic23/qemu.
DSDT blob - as this is a hack I haven't included test updates
Scope (_SB)
{
Device (MCTP)
{
Name (_HID, "PRP0001") // _HID: Hardware ID
Name (_DSD, Package (0x02) // _DSD: Device-Specific Data
{
ToUUID ("daffd814-6eba-4d8c-8a91-bc9bbf4aa301") /* Device Properties for _DSD */,
Package (0x03)
{
Package (0x02)
{
"compatible",
"aspeed,ast2600-i2c-bus"
},
Package (0x02)
{
"bus-frequency",
0x00061A80
},
Package (0x02)
{
"mctp-controller",
One
}
}
})
Name (_CRS, ResourceTemplate () // _CRS: Current Resource Settings
{
QWordMemory (ResourceProducer, PosDecode, MinFixed, MaxFixed, NonCacheable, ReadWrite,
0x0000000000000000, // Granularity
0x00000004800FC080, // Range Minimum
0x00000004800FC0FF, // Range Maximum
0x0000000000000000, // Translation Offset
0x0000000000000080, // Length
,, , AddressRangeMemory, TypeStatic)
Interrupt (ResourceConsumer, Level, ActiveHigh, Shared, ,, )
{
0x00000007,
}
})
}
Device (MCTS)
{
Name (_HID, "PRP0001") // _HID: Hardware ID
Name (_CRS, ResourceTemplate () // _CRS: Current Resource Settings
{
I2cSerialBusV2 (0x0050, DeviceInitiated, 0x000186A0,
AddressingMode7Bit, "\\_SB.MCTP",
0x00, ResourceProducer, , Exclusive,
)
})
Name (_DSD, Package (0x02) // _DSD: Device-Specific Data
{
ToUUID ("daffd814-6eba-4d8c-8a91-bc9bbf4aa301") /* Device Properties for _DSD */,
Package (0x01)
{
Package (0x02)
{
"compatible",
"mctp-i2c-controller"
}
}
})
}
}
To add devices to the bus use something like:
-device i2c_mctp_cxl_switch,bus=aspeed.i2c.bus.0,address=4,target=us0
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/i386/pc.h | 1 +
hw/i386/acpi-build.c | 65 ++++++++++++++++++++++++++++++++++++++++++++
hw/i386/pc.c | 20 +++++++++++++-
3 files changed, 85 insertions(+), 1 deletion(-)
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index d54e8b1101..d60c3d44a7 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -56,6 +56,7 @@ typedef struct PCMachineState {
SGXEPCState sgx_epc;
CXLState cxl_devices_state;
+ hwaddr i2c_base;
} PCMachineState;
#define PC_MACHINE_ACPI_DEVICE_PROP "acpi-device"
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index c791fb2760..481ad87438 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1424,6 +1424,68 @@ static void build_acpi0017(Aml *table)
aml_append(table, scope);
}
+static void acpi_dsdt_add_mctp(Aml *scope, PCMachineState *pcms)
+{
+ uint32_t interrupt = 7;
+ Aml *main_dev = aml_device("MCTP");
+ Aml *sub_dev = aml_device("MCTS");
+ Aml *dsd_pkg = aml_package(2);
+ Aml *props_pkg = aml_package(3);
+ Aml *pkg = aml_package(2);
+ Aml *crs = aml_resource_template();
+
+ aml_append(main_dev, aml_name_decl("_HID", aml_string("PRP0001")));
+
+ aml_append(pkg, aml_string("compatible"));
+ aml_append(pkg, aml_string("aspeed,ast2600-i2c-bus"));
+ aml_append(props_pkg, pkg);
+
+ pkg = aml_package(2);
+ aml_append(pkg, aml_string("bus-frequency"));
+ aml_append(pkg, aml_int(400000));
+ aml_append(props_pkg, pkg);
+
+ pkg = aml_package(2);
+ aml_append(pkg, aml_string("mctp-controller"));
+ aml_append(pkg, aml_int(1));
+ aml_append(props_pkg, pkg);
+
+ aml_append(dsd_pkg, aml_touuid("DAFFD814-6EBA-4D8C-8A91-BC9BBF4AA301"));
+ aml_append(dsd_pkg, props_pkg);
+ aml_append(main_dev, aml_name_decl("_DSD", dsd_pkg));
+
+ aml_append(crs, aml_qword_memory(AML_POS_DECODE, AML_MIN_FIXED,
+ AML_MAX_FIXED, AML_NON_CACHEABLE,
+ AML_READ_WRITE, 0, pcms->i2c_base + 0x80,
+ pcms->i2c_base + 0x80 + 0x80 - 1,
+ 0, 0x80));
+ aml_append(crs,
+ aml_interrupt(AML_CONSUMER, AML_LEVEL, AML_ACTIVE_HIGH,
+ AML_SHARED, &interrupt, 1));
+ aml_append(main_dev, aml_name_decl("_CRS", crs));
+
+ aml_append(sub_dev, aml_name_decl("_HID", aml_string("PRP0001")));
+
+ dsd_pkg = aml_package(2);
+ aml_append(dsd_pkg, aml_touuid("DAFFD814-6EBA-4D8C-8A91-BC9BBF4AA301"));
+
+ props_pkg = aml_package(1);
+
+ pkg = aml_package(2);
+ aml_append(pkg, aml_string("compatible"));
+ aml_append(pkg, aml_string("mctp-i2c-controller"));
+ aml_append(props_pkg, pkg);
+ aml_append(dsd_pkg, props_pkg);
+
+ crs = aml_resource_template();
+ aml_append(crs, aml_i2c_slv_serial_bus_device(0x50, "\\_SB.MCTP"));
+ aml_append(sub_dev, aml_name_decl("_CRS", crs));
+ aml_append(sub_dev, aml_name_decl("_DSD", dsd_pkg));
+
+ aml_append(scope, main_dev);
+ aml_append(scope, sub_dev);
+}
+
/*
* Precompute the crs ranges and bus numbers that will be used in PXB entries
* in PXB SSDT.
@@ -1652,6 +1714,9 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
build_hpet_aml(dsdt);
}
+ sb_scope = aml_scope("_SB");
+ acpi_dsdt_add_mctp(sb_scope, pcms);
+ aml_append(dsdt, sb_scope);
if (vmbus_bridge) {
sb_scope = aml_scope("_SB");
aml_append(sb_scope, build_vmbus_device_aml(vmbus_bridge));
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 3109d5e0e0..335b6f1265 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -93,6 +93,7 @@
#include "hw/i386/kvm/xen_evtchn.h"
#include "hw/i386/kvm/xen_gnttab.h"
#include "hw/i386/kvm/xen_xenstore.h"
+#include "hw/i2c/aspeed_i2c.h"
#include "sysemu/replay.h"
#include "target/i386/cpu.h"
#include "e820_memory_layout.h"
@@ -1074,6 +1075,8 @@ void pc_memory_init(PCMachineState *pcms,
memory_region_init(mr, OBJECT(machine), "cxl_host_reg", cxl_size);
memory_region_add_subregion(system_memory, cxl_base, mr);
cxl_resv_end = cxl_base + cxl_size;
+ pcms->i2c_base = cxl_resv_end - 0x4000;
+
if (pcms->cxl_devices_state.fixed_windows) {
hwaddr cxl_fmw_base;
GList *it;
@@ -1157,7 +1160,7 @@ uint64_t pc_pci_hole64_start(void)
ram_addr_t size = 0;
if (pcms->cxl_devices_state.is_enabled) {
- hole64_start = pc_get_cxl_range_end(pcms);
+ hole64_start = pc_get_cxl_range_end(pcms) + 0x4000;
} else if (pcmc->has_reserved_memory && (ms->ram_size < ms->maxram_size)) {
pc_get_device_memory_range(pcms, &hole64_start, &size);
if (!pcmc->broken_reserved_end) {
@@ -1354,6 +1357,21 @@ void pc_basic_device_init(struct PCMachineState *pcms,
/* Super I/O */
pc_superio_init(isa_bus, create_fdctrl, pcms->i8042_enabled,
pcms->vmport != ON_OFF_AUTO_ON);
+
+ {
+ AspeedI2CState *aspeed_i2c;
+ struct DeviceState *dev;
+
+ dev = qdev_new("aspeed.i2c-ast2600");
+ aspeed_i2c = ASPEED_I2C(dev);
+ object_property_set_link(OBJECT(dev), "dram",
+ OBJECT(MACHINE(pcms)->ram), &error_fatal);
+ sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
+ sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, pcms->i2c_base);
+ /* Hack ;) - Steal unused interrupt 7 */
+ sysbus_connect_irq(SYS_BUS_DEVICE(&aspeed_i2c->busses[0]), 0,
+ x86ms->gsi[7]);
+ }
}
void pc_nic_init(PCMachineClass *pcmc, ISABus *isa_bus, PCIBus *pci_bus)
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 14/17] docs: cxl: Add example commandline for MCTP CXL CCIs
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (12 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 13/17] HACK: hw/i386/pc: Add Aspeed i2c controller + MCTP with ACPI tables Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 15/17] hw/cxl: Add a switch mailbox CCI function Jonathan Cameron via
` (2 subsequent siblings)
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
A lot more needed here on what these are for and what can be done
with them.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
docs/system/devices/cxl.rst | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/docs/system/devices/cxl.rst b/docs/system/devices/cxl.rst
index b742120657..62b6161474 100644
--- a/docs/system/devices/cxl.rst
+++ b/docs/system/devices/cxl.rst
@@ -406,6 +406,33 @@ OS management of CXL memory devices as described here.
* CONFIG_CXL_PORT
* CONFIG_CXL_REGION
+
+CCI access via MCTP over I2C
+----------------------------
+
+TODO: Add some more info here on what this actually is.
+
+Both CXL switches and CXL Type 3 devices support configuration via
+MCTP access to Component Command Interfaces (CCIs) on the devices.
+
+Example configuration:
+
+ -device cxl-upstream,port=33,bus=root_port0,id=us0,multifunction=on,addr=0.0,sn=12345678 \
+ -device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \
+ -device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \
+ -device cxl-downstream,port=2,bus=us0,id=swport2,chassis=0,slot=6 \
+ -device cxl-type3,bus=swport0,persistent-memdev=cxl-mem1,id=cxl-pmem0,lsa=cxl-lsa1,sn=3 \
+ -device cxl-type3,bus=swport1,persistent-memdev=cxl-mem2,id=cxl-pmem1,lsa=cxl-lsa2,sn=4 \
+ -device cxl-type3,bus=swport2,persistent-memdev=cxl-mem3,id=cxl-pmem2,lsa=cxl-lsa3,sn=5 \
+ -machine cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=1k \
+ -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=4,target=us0 \
+ -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=5,target=cxl-pmem0 \
+ -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=6,target=cxl-pmem1 \
+ -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=7,target=cxl-pmem2
+
+Communication with the MCTP CCI can then be established using standard MCTP configuration
+tools.
+
References
----------
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 15/17] hw/cxl: Add a switch mailbox CCI function.
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (13 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 14/17] docs: cxl: Add example commandline for MCTP CXL CCIs Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 16/17] hw/cxl: Implement Physical Ports status retrieval Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 17/17] hw/cxl: Add tunneled command support to mailbox for switch cci Jonathan Cameron via
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
CXL switch CCIs were added in CXL r3.0. They are a PCI function,
identified by class code that provides a CXL mailbox (identical
to that previously defined for CXL type 3 memory devices) over which
various FM-API commands may be used. Whilst the intent of this
feature is enable switch control from a BMC attached to a switch
upstream port, it is also useful to allow emulation of this feature
on the upstream port connected to a host using the CXL devices as
this greatly simplifies testing.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/cxl/cxl.h | 6 +
include/hw/cxl/cxl_component.h | 3 +-
include/hw/cxl/cxl_device.h | 15 +++
include/hw/pci-bridge/cxl_upstream_port.h | 1 +
hw/cxl/cxl-device-utils.c | 25 +++++
hw/cxl/cxl-mailbox-utils.c | 129 ++++++++++++++++++++++
hw/cxl/switch-mailbox-cci.c | 98 ++++++++++++++++
hw/pci-bridge/cxl_downstream.c | 5 +-
hw/cxl/meson.build | 1 +
9 files changed, 279 insertions(+), 4 deletions(-)
diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h
index cd7567d6a8..69484f2553 100644
--- a/include/hw/cxl/cxl.h
+++ b/include/hw/cxl/cxl.h
@@ -63,4 +63,10 @@ OBJECT_DECLARE_SIMPLE_TYPE(CXLHost, PXB_CXL_HOST)
typedef struct CXLUpstreamPort CXLUpstreamPort;
DECLARE_INSTANCE_CHECKER(CXLUpstreamPort, CXL_USP, TYPE_CXL_USP)
CXLComponentState *cxl_usp_to_cstate(CXLUpstreamPort *usp);
+
+#define TYPE_CXL_DSP "cxl-downstream"
+
+typedef struct CXLDownstreamPort CXLDownstreamPort;
+DECLARE_INSTANCE_CHECKER(CXLDownstreamPort, CXL_DSP, TYPE_CXL_DSP)
+
#endif
diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h
index c355192b7c..52b6a2d67f 100644
--- a/include/hw/cxl/cxl_component.h
+++ b/include/hw/cxl/cxl_component.h
@@ -26,7 +26,8 @@ enum reg_type {
CXL2_LOGICAL_DEVICE,
CXL2_ROOT_PORT,
CXL2_UPSTREAM_PORT,
- CXL2_DOWNSTREAM_PORT
+ CXL2_DOWNSTREAM_PORT,
+ CXL3_SWITCH_MAILBOX_CCI,
};
/*
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index c49ff7e9b3..215383ba37 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -244,8 +244,10 @@ void cxl_device_register_block_init(Object *obj, CXLDeviceState *dev,
CXLCCI *cci);
typedef struct CXLType3Dev CXLType3Dev;
+typedef struct CSWMBCCIDev CSWMBCCIDev;
/* Set up default values for the register block */
void cxl_device_register_init_t3(CXLType3Dev *ct3d);
+void cxl_device_register_init_swcci(CSWMBCCIDev *sw);
/*
* CXL 2.0 - 8.2.8.1 including errata F4
@@ -292,6 +294,8 @@ CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MEMORY_DEVICE,
CXL_DEVICE_CAP_REG_SIZE * 2)
void cxl_initialize_mailbox_t3(CXLCCI *cci, DeviceState *d, size_t payload_max);
+void cxl_initialize_mailbox_swcci(CXLCCI *cci, DeviceState *intf,
+ DeviceState *d, size_t payload_max);
void cxl_init_cci(CXLCCI *cci, size_t payload_max);
int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
size_t len_in, uint8_t *pl_in,
@@ -437,6 +441,17 @@ struct CXLType3Class {
bool (*set_cacheline)(CXLType3Dev *ct3d, uint64_t dpa_offset, uint8_t *data);
};
+struct CSWMBCCIDev {
+ PCIDevice parent_obj;
+ PCIDevice *target;
+ CXLComponentState cxl_cstate;
+ CXLDeviceState cxl_dstate;
+ CXLCCI *cci;
+};
+
+#define TYPE_CXL_SWITCH_MAILBOX_CCI "cxl-switch-mailbox-cci"
+OBJECT_DECLARE_TYPE(CSWMBCCIDev, CSWMBCCIClass, CXL_SWITCH_MAILBOX_CCI)
+
MemTxResult cxl_type3_read(PCIDevice *d, hwaddr host_addr, uint64_t *data,
unsigned size, MemTxAttrs attrs);
MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data,
diff --git a/include/hw/pci-bridge/cxl_upstream_port.h b/include/hw/pci-bridge/cxl_upstream_port.h
index 15db8c8582..7275102cfa 100644
--- a/include/hw/pci-bridge/cxl_upstream_port.h
+++ b/include/hw/pci-bridge/cxl_upstream_port.h
@@ -12,6 +12,7 @@ typedef struct CXLUpstreamPort {
/*< public >*/
CXLComponentState cxl_cstate;
CXLCCI mctpcci;
+ CXLCCI swcci;
DOECap doe_cdat;
uint64_t sn;
} CXLUpstreamPort;
diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
index 7dbc8b72d7..658321fa4a 100644
--- a/hw/cxl/cxl-device-utils.c
+++ b/hw/cxl/cxl-device-utils.c
@@ -64,6 +64,8 @@ static uint64_t mailbox_reg_read(void *opaque, hwaddr offset, unsigned size)
if (object_dynamic_cast(OBJECT(cci->intf), TYPE_CXL_TYPE3)) {
cxl_dstate = &CXL_TYPE3(cci->intf)->cxl_dstate;
+ } else if (object_dynamic_cast(OBJECT(cci->intf), TYPE_CXL_SWITCH_MAILBOX_CCI)) {
+ cxl_dstate = &CXL_SWITCH_MAILBOX_CCI(cci->intf)->cxl_dstate;
} else {
return 0;
}
@@ -148,6 +150,8 @@ static void mailbox_reg_write(void *opaque, hwaddr offset, uint64_t value,
if (object_dynamic_cast(OBJECT(cci->intf), TYPE_CXL_TYPE3)) {
cxl_dstate = &CXL_TYPE3(cci->intf)->cxl_dstate;
+ } else if (object_dynamic_cast(OBJECT(cci->intf), TYPE_CXL_SWITCH_MAILBOX_CCI)) {
+ cxl_dstate = &CXL_SWITCH_MAILBOX_CCI(cci->intf)->cxl_dstate;
} else {
return;
}
@@ -361,6 +365,27 @@ void cxl_device_register_init_t3(CXLType3Dev *ct3d)
CXL_MAILBOX_MAX_PAYLOAD_SIZE);
}
+void cxl_device_register_init_swcci(CSWMBCCIDev *sw)
+{
+ CXLDeviceState *cxl_dstate = &sw->cxl_dstate;
+ uint64_t *cap_hdrs = cxl_dstate->caps_reg_state64;
+ const int cap_count = 3;
+
+ /* CXL Device Capabilities Array Register */
+ ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_ID, 0);
+ ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_VERSION, 1);
+ ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_COUNT, cap_count);
+
+ cxl_device_cap_init(cxl_dstate, DEVICE_STATUS, 1, 2);
+ device_reg_init_common(cxl_dstate);
+
+ cxl_device_cap_init(cxl_dstate, MAILBOX, 2, 1);
+ mailbox_reg_init_common(cxl_dstate);
+
+ cxl_device_cap_init(cxl_dstate, MEMORY_DEVICE, 0x4000, 1);
+ memdev_reg_init_common(cxl_dstate);
+}
+
uint64_t cxl_device_get_timestamp(CXLDeviceState *cxl_dstate)
{
uint64_t time, delta;
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 2d3d19dce8..9c1020d9ab 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -47,6 +47,7 @@
enum {
INFOSTAT = 0x00,
#define IS_IDENTIFY 0x1
+ #define BACKGROUND_OPERATION_STATUS 0x2
EVENTS = 0x01,
#define GET_RECORDS 0x0
#define CLEAR_RECORDS 0x1
@@ -70,6 +71,8 @@ enum {
#define GET_POISON_LIST 0x0
#define INJECT_POISON 0x1
#define CLEAR_POISON 0x2
+ PHYSICAL_SWITCH = 0x51,
+ #define IDENTIFY_SWITCH_DEVICE 0x0
};
@@ -205,6 +208,7 @@ static CXLRetCode cmd_events_set_interrupt_policy(const struct cxl_cmd *cmd,
*len_out = 0;
return CXL_MBOX_SUCCESS;
}
+
/* CXL r3 8.2.9.1.1 */
static CXLRetCode cmd_infostat_identify(const struct cxl_cmd *cmd,
uint8_t *payload_in,
@@ -252,6 +256,97 @@ static CXLRetCode cmd_infostat_identify(const struct cxl_cmd *cmd,
return CXL_MBOX_SUCCESS;
}
+static void cxl_set_dsp_active_bm(PCIBus *b, PCIDevice *d,
+ void *private)
+{
+ uint8_t *bm = private;
+ if (object_dynamic_cast(OBJECT(d), TYPE_CXL_DSP)) {
+ uint8_t port = PCIE_PORT(d)->port;
+ bm[port / 8] |= 1 << (port % 8);
+ }
+}
+
+/* CXL r3 8.2.9.1.1 */
+static CXLRetCode cmd_identify_switch_device(const struct cxl_cmd *cmd,
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLCCI *cci)
+{
+ PCIEPort *usp = PCIE_PORT(cci->d);
+ PCIBus *bus = &PCI_BRIDGE(cci->d)->sec_bus;
+ int num_phys_ports = pcie_count_ds_ports(bus);
+
+ struct cxl_fmapi_ident_switch_dev_resp_pl {
+ uint8_t ingress_port_id;
+ uint8_t rsvd;
+ uint8_t num_physical_ports;
+ uint8_t num_vcss;
+ uint8_t active_port_bitmask[0x20];
+ uint8_t active_vcs_bitmask[0x20];
+ uint16_t total_vppbs;
+ uint16_t bound_vppbs;
+ uint8_t num_hdm_decoders_per_usp;
+ } QEMU_PACKED *out;
+ QEMU_BUILD_BUG_ON(sizeof(*out) != 0x49);
+
+ out = (struct cxl_fmapi_ident_switch_dev_resp_pl *)payload_out;
+ *out = (struct cxl_fmapi_ident_switch_dev_resp_pl) {
+ .num_physical_ports = num_phys_ports + 1, /* 1 USP */
+ .num_vcss = 1, /* Not yet support multiple VCS - potentialy tricky */
+ .active_vcs_bitmask[0] = 0x1,
+ .total_vppbs = num_phys_ports + 1,
+ .bound_vppbs = num_phys_ports + 1,
+ .num_hdm_decoders_per_usp = 4,
+ };
+
+ /* Depends on the CCI type */
+ if (object_dynamic_cast(OBJECT(cci->intf), TYPE_PCIE_PORT)) {
+ out->ingress_port_id = PCIE_PORT(cci->intf)->port;
+ } else {
+ /* MCTP? */
+ out->ingress_port_id = 0;
+ }
+
+ pci_for_each_device_under_bus(bus, cxl_set_dsp_active_bm,
+ out->active_port_bitmask);
+ out->active_port_bitmask[usp->port / 8] |= (1 << usp->port % 8);
+
+ *len_out = sizeof(*out);
+
+ return CXL_MBOX_SUCCESS;
+}
+
+/* CXL r3.0 8.2.9.1.2 */
+static CXLRetCode cmd_infostat_bg_op_sts(const struct cxl_cmd *cmd,
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLCCI *cci)
+{
+ struct {
+ uint8_t status;
+ uint8_t rsvd;
+ uint16_t opcode;
+ uint16_t returncode;
+ uint16_t vendor_ext_status;
+ } QEMU_PACKED *bg_op_status;
+ QEMU_BUILD_BUG_ON(sizeof(*bg_op_status) != 8);
+
+ bg_op_status = (void *)payload_out;
+ memset(bg_op_status, 0, sizeof(*bg_op_status));
+ bg_op_status->status = cci->bg.complete_pct << 1;
+ if (cci->bg.runtime > 0) {
+ bg_op_status->status |= 1U << 0;
+ }
+ bg_op_status->opcode = cci->bg.opcode;
+ bg_op_status->returncode = cci->bg.ret_code;
+ *len_out = sizeof(*bg_op_status);
+ return CXL_MBOX_SUCCESS;
+}
+
/* 8.2.9.2.1 */
static CXLRetCode cmd_firmware_update_get_info(const struct cxl_cmd *cmd,
uint8_t *payload_in,
@@ -796,6 +891,30 @@ static const struct cxl_cmd cxl_cmd_set[256][256] = {
cmd_media_clear_poison, 72, 0 },
};
+static const struct cxl_cmd cxl_cmd_set_sw[256][256] = {
+ [INFOSTAT][IS_IDENTIFY] = { "IDENTIFY", cmd_infostat_identify, 0, 18 },
+ [INFOSTAT][BACKGROUND_OPERATION_STATUS] = { "BACKGROUND_OPERATION_STATUS",
+ cmd_infostat_bg_op_sts, 0, 8 },
+ /*
+ * TODO get / set response message limit - requires all messages over
+ * 256 bytes to support chunking.
+ */
+ [TIMESTAMP][GET] = { "TIMESTAMP_GET", cmd_timestamp_get, 0, 0 },
+ [TIMESTAMP][SET] = { "TIMESTAMP_SET", cmd_timestamp_set, 8, IMMEDIATE_POLICY_CHANGE },
+ [LOGS][GET_SUPPORTED] = { "LOGS_GET_SUPPORTED", cmd_logs_get_supported, 0, 0 },
+ [LOGS][GET_LOG] = { "LOGS_GET_LOG", cmd_logs_get_log, 0x18, 0 },
+ [PHYSICAL_SWITCH][IDENTIFY_SWITCH_DEVICE] = {"IDENTIFY_SWITCH_DEVICE",
+ cmd_identify_switch_device, 0, 0x49 },
+};
+
+/*
+ * While the command is executing in the background, the device should
+ * update the percentage complete in the Background Command Status Register
+ * at least once per second.
+ */
+
+#define CXL_MBOX_BG_UPDATE_FREQ 1000UL
+
int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
size_t len_in, uint8_t *pl_in, size_t *len_out,
uint8_t *pl_out, bool *bg_started)
@@ -836,6 +955,14 @@ void cxl_init_cci(CXLCCI *cci, size_t payload_max)
}
}
+void cxl_initialize_mailbox_swcci(CXLCCI *cci, DeviceState *intf, DeviceState *d, size_t payload_max)
+{
+ cci->cxl_cmd_set = cxl_cmd_set_sw;
+ cci->d = d;
+ cci->intf = intf;
+ cxl_init_cci(cci, payload_max);
+}
+
void cxl_initialize_mailbox_t3(CXLCCI *cci, DeviceState *d, size_t payload_max)
{
cci->cxl_cmd_set = cxl_cmd_set;
@@ -861,6 +988,8 @@ void cxl_initialize_t3_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf,
static const struct cxl_cmd cxl_cmd_set_usp_mctp[256][256] = {
[INFOSTAT][IS_IDENTIFY] = { "IDENTIFY", cmd_infostat_identify, 0, 18 },
+ [PHYSICAL_SWITCH][IDENTIFY_SWITCH_DEVICE] = {"IDENTIFY_SWITCH_DEVICE",
+ cmd_identify_switch_device, 0, 0x49 },
};
void cxl_initialize_usp_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf, size_t payload_max)
diff --git a/hw/cxl/switch-mailbox-cci.c b/hw/cxl/switch-mailbox-cci.c
new file mode 100644
index 0000000000..278d979d80
--- /dev/null
+++ b/hw/cxl/switch-mailbox-cci.c
@@ -0,0 +1,98 @@
+#include "qemu/osdep.h"
+#include "hw/pci/pci.h"
+#include "hw/pci-bridge/cxl_upstream_port.h"
+#include "qapi/error.h"
+#include "qemu/log.h"
+#include "qemu/module.h"
+#include "hw/qdev-properties.h"
+#include "hw/cxl/cxl.h"
+
+struct CSWMBCCIClass {
+ PCIDeviceClass parent_class;
+};
+
+static void cswmbcci_reset(DeviceState *dev)
+{
+ CSWMBCCIDev *cswmb = CXL_SWITCH_MAILBOX_CCI(dev);
+ cxl_device_register_init_swcci(cswmb);
+}
+
+static void cswbcci_realize(PCIDevice *pci_dev, Error **errp)
+{
+ CSWMBCCIDev *cswmb = CXL_SWITCH_MAILBOX_CCI(pci_dev);
+ CXLComponentState *cxl_cstate = &cswmb->cxl_cstate;
+ CXLDeviceState *cxl_dstate = &cswmb->cxl_dstate;
+ CXLDVSECRegisterLocator *regloc_dvsec;
+ CXLUpstreamPort *usp;
+
+ if (!cswmb->target) {
+ error_setg(errp, "Target not set");
+ return;
+ }
+ usp = CXL_USP(cswmb->target);
+
+ pcie_endpoint_cap_init(pci_dev, 0x80);
+ cxl_cstate->dvsec_offset = 0x100;
+ cxl_cstate->pdev = pci_dev;
+ cswmb->cci = &usp->swcci;
+ cxl_device_register_block_init(OBJECT(pci_dev), cxl_dstate, cswmb->cci);
+ pci_register_bar(pci_dev, 0,
+ PCI_BASE_ADDRESS_SPACE_MEMORY |
+ PCI_BASE_ADDRESS_MEM_TYPE_64,
+ &cxl_dstate->device_registers);
+ regloc_dvsec = &(CXLDVSECRegisterLocator) {
+ .rsvd = 0,
+ .reg_base[0].lo = RBI_CXL_DEVICE_REG | 0,
+ .reg_base[0].hi = 0,
+ };
+ cxl_component_create_dvsec(cxl_cstate, CXL3_SWITCH_MAILBOX_CCI,
+ REG_LOC_DVSEC_LENGTH, REG_LOC_DVSEC,
+ REG_LOC_DVSEC_REVID, (uint8_t *)regloc_dvsec);
+
+ cxl_initialize_mailbox_swcci(cswmb->cci, DEVICE(pci_dev), DEVICE(cswmb->target),
+ CXL_MAILBOX_MAX_PAYLOAD_SIZE);
+}
+
+static void cswmbcci_exit(PCIDevice *pci_dev)
+{
+}
+
+static Property cxl_switch_cci_props[] = {
+ DEFINE_PROP_LINK("target", CSWMBCCIDev,
+ target, TYPE_CXL_USP, PCIDevice *),
+ DEFINE_PROP_END_OF_LIST(),
+};
+
+static void cswmbcci_class_init(ObjectClass *oc, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+ PCIDeviceClass *pc = PCI_DEVICE_CLASS(oc);
+
+ pc->realize = cswbcci_realize;
+ pc->exit = cswmbcci_exit;
+ pc->class_id = 0x0c0b; /* Serial bus, CXL Switch CCI */
+ pc->vendor_id = 0x19e5;
+ pc->device_id = 0xbeef; /* FIXME - assign a valid ID for this function */
+ pc->revision = 0;
+ dc->desc = "CXL Switch Mailbox CCI";
+ dc->reset = cswmbcci_reset;
+ device_class_set_props(dc, cxl_switch_cci_props);
+}
+
+static const TypeInfo cswmbcci_info = {
+ .name = TYPE_CXL_SWITCH_MAILBOX_CCI,
+ .parent = TYPE_PCI_DEVICE,
+ .class_size = sizeof(struct CSWMBCCIClass),
+ .class_init = cswmbcci_class_init,
+ .instance_size = sizeof(CSWMBCCIDev),
+ .interfaces = (InterfaceInfo[]) {
+ { INTERFACE_PCIE_DEVICE },
+ { }
+ },
+};
+
+static void cxl_switch_mailbox_cci_register(void)
+{
+ type_register_static(&cswmbcci_info);
+}
+type_init(cxl_switch_mailbox_cci_register);
diff --git a/hw/pci-bridge/cxl_downstream.c b/hw/pci-bridge/cxl_downstream.c
index 7eb1981be3..573c583be4 100644
--- a/hw/pci-bridge/cxl_downstream.c
+++ b/hw/pci-bridge/cxl_downstream.c
@@ -13,7 +13,9 @@
#include "hw/pci/msi.h"
#include "hw/pci/pcie.h"
#include "hw/pci/pcie_port.h"
+#include "hw/cxl/cxl.h"
#include "qapi/error.h"
+#include "hw/cxl/cxl.h"
typedef struct CXLDownstreamPort {
/*< private >*/
@@ -23,9 +25,6 @@ typedef struct CXLDownstreamPort {
CXLComponentState cxl_cstate;
} CXLDownstreamPort;
-#define TYPE_CXL_DSP "cxl-downstream"
-DECLARE_INSTANCE_CHECKER(CXLDownstreamPort, CXL_DSP, TYPE_CXL_DSP)
-
#define CXL_DOWNSTREAM_PORT_MSI_OFFSET 0x70
#define CXL_DOWNSTREAM_PORT_MSI_NR_VECTOR 1
#define CXL_DOWNSTREAM_PORT_EXP_OFFSET 0x90
diff --git a/hw/cxl/meson.build b/hw/cxl/meson.build
index 7a885e7b84..1393821fc4 100644
--- a/hw/cxl/meson.build
+++ b/hw/cxl/meson.build
@@ -7,6 +7,7 @@ system_ss.add(when: 'CONFIG_CXL',
'cxl-cdat.c',
'cxl-events.c',
'cxl-cpmu.c',
+ 'switch-mailbox-cci.c',
),
if_false: files(
'cxl-host-stubs.c',
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 16/17] hw/cxl: Implement Physical Ports status retrieval
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (14 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 15/17] hw/cxl: Add a switch mailbox CCI function Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 17/17] hw/cxl: Add tunneled command support to mailbox for switch cci Jonathan Cameron via
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
Signed-of-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
hw/cxl/cxl-mailbox-utils.c | 87 ++++++++++++++++++++++++++++++++++++++
1 file changed, 87 insertions(+)
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 9c1020d9ab..4cddd6eae1 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -73,6 +73,7 @@ enum {
#define CLEAR_POISON 0x2
PHYSICAL_SWITCH = 0x51,
#define IDENTIFY_SWITCH_DEVICE 0x0
+ #define GET_PHYSICAL_PORT_STATE 0x1
};
@@ -318,6 +319,88 @@ static CXLRetCode cmd_identify_switch_device(const struct cxl_cmd *cmd,
return CXL_MBOX_SUCCESS;
}
+static CXLRetCode cmd_get_physical_port_state(const struct cxl_cmd *cmd,
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLCCI *cci)
+{
+ /*
+ * CXL r3.0 7.6.7.1.2 Get Physical Port State (Opcode 5101h)
+ */
+ /* CXL r3.0 Table 7-18 Get Physical Port State Request Payload */
+ struct cxl_fmapi_get_phys_port_state_req_pl {
+ uint8_t num_ports; /* CHECK. may get too large for MCTP message size */
+ uint8_t ports[];
+ } QEMU_PACKED *in;
+
+ /* CXL r3.0 Table 7-20 Get Physical Port State Port Information Block Format */
+ struct cxl_fmapi_port_state_info_block {
+ uint8_t port_id;
+ uint8_t config_state;
+ uint8_t connected_device_cxl_version;
+ uint8_t rsv1;
+ uint8_t connected_device_type;
+ uint8_t port_cxl_version_bitmask;
+ uint8_t max_link_width;
+ uint8_t negotiated_link_width;
+ uint8_t supported_link_speeds_vector;
+ uint8_t max_link_speed;
+ uint8_t current_link_speed;
+ uint8_t ltssm_state;
+ uint8_t first_lane_num;
+ uint16_t link_state;
+ uint8_t supported_ld_count;
+ } QEMU_PACKED;
+
+ /* CXL r3.0 Table 7-19 Get Physical Port State Response Payload */
+ struct cxl_fmapi_get_phys_port_state_resp_pl {
+ uint8_t num_ports;
+ uint8_t rsv1[3];
+ struct cxl_fmapi_port_state_info_block ports[];
+ } QEMU_PACKED *out;
+ PCIBus *bus = &PCI_BRIDGE(cci->d)->sec_bus;
+ int num_phys_ports = pcie_count_ds_ports(bus);
+ int i;
+ size_t pl_size;
+
+ in = (struct cxl_fmapi_get_phys_port_state_req_pl *)payload_in;
+ out = (struct cxl_fmapi_get_phys_port_state_resp_pl *)payload_out;
+ /* Not currently matching against requested */
+ out->num_ports = num_phys_ports;
+
+ for (i = 0; i < out->num_ports; i++) {
+ struct cxl_fmapi_port_state_info_block *port;
+ port = &out->ports[i];
+ port->port_id = i; /* TODO: Right port number */
+ if (port->port_id < 1) { /* 1 upstream ports */
+ port->config_state = 4;
+ port->connected_device_type = 0;
+ } else { /* remainder downstream ports */
+ port->config_state = 3;
+ port->connected_device_type = 4; /* TODO: Check. CXL type 3 */
+ port->supported_ld_count = 3;
+ }
+ port->connected_device_cxl_version = 2;
+ port->port_cxl_version_bitmask = 0x2;
+ port->max_link_width = 0x10; /* x16 */
+ port->negotiated_link_width = 0x10;
+ port->supported_link_speeds_vector = 0x1c; /* 8, 16, 32 GT/s */
+ port->max_link_speed = 5;
+ port->current_link_speed = 5; /* 32 */
+ port->ltssm_state = 0x7; /* L2 */
+ port->first_lane_num = 0;
+ port->link_state = 0;
+ }
+
+ pl_size = sizeof(out) + sizeof(*out->ports) * in->num_ports;
+
+ *len_out = pl_size;
+
+ return CXL_MBOX_SUCCESS;
+}
+
/* CXL r3.0 8.2.9.1.2 */
static CXLRetCode cmd_infostat_bg_op_sts(const struct cxl_cmd *cmd,
uint8_t *payload_in,
@@ -905,6 +988,8 @@ static const struct cxl_cmd cxl_cmd_set_sw[256][256] = {
[LOGS][GET_LOG] = { "LOGS_GET_LOG", cmd_logs_get_log, 0x18, 0 },
[PHYSICAL_SWITCH][IDENTIFY_SWITCH_DEVICE] = {"IDENTIFY_SWITCH_DEVICE",
cmd_identify_switch_device, 0, 0x49 },
+ [PHYSICAL_SWITCH][GET_PHYSICAL_PORT_STATE] = { "SWITCH_PHYSICAL_PORT_STATS",
+ cmd_get_physical_port_state, ~0, ~0 },
};
/*
@@ -990,6 +1075,8 @@ static const struct cxl_cmd cxl_cmd_set_usp_mctp[256][256] = {
[INFOSTAT][IS_IDENTIFY] = { "IDENTIFY", cmd_infostat_identify, 0, 18 },
[PHYSICAL_SWITCH][IDENTIFY_SWITCH_DEVICE] = {"IDENTIFY_SWITCH_DEVICE",
cmd_identify_switch_device, 0, 0x49 },
+ [PHYSICAL_SWITCH][GET_PHYSICAL_PORT_STATE] = { "SWITCH_PHYSICAL_PORT_STATS",
+ cmd_get_physical_port_state, ~0, ~0 },
};
void cxl_initialize_usp_mctpcci(CXLCCI *cci, DeviceState *d, DeviceState *intf, size_t payload_max)
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH 17/17] hw/cxl: Add tunneled command support to mailbox for switch cci.
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
` (15 preceding siblings ...)
2023-07-17 17:16 ` [RFC PATCH 16/17] hw/cxl: Implement Physical Ports status retrieval Jonathan Cameron via
@ 2023-07-17 17:16 ` Jonathan Cameron via
16 siblings, 0 replies; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-17 17:16 UTC (permalink / raw)
To: linux-cxl, Dan Williams, qemu-devel
Cc: linuxarm, Alison Schofield, Ira Weiny, Dave Jiang,
Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
Allow a switch CCI to perform basic tunneling (which is transported
in real hardware via PCIe VDM) to downstream devices.
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
include/hw/cxl/cxl_device.h | 1 +
hw/cxl/cxl-mailbox-utils.c | 92 +++++++++++++++++++++++++++++++++++++
hw/mem/cxl_type3.c | 4 ++
3 files changed, 97 insertions(+)
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 215383ba37..4f8095847e 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -407,6 +407,7 @@ struct CXLType3Dev {
CXLDeviceState cxl_dstate;
CXLCCI cci; /* Primary PCI mailbox CCI */
CXLCCI oob_mctp_cci; /* Initialized only if targetted */
+ CXLCCI vdm_mctp_cci; /* Always intialized as no way to know if a VDM might show up */
/* DOE */
DOECap doe_cdat;
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 4cddd6eae1..33f7b9a9a5 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -74,8 +74,98 @@ enum {
PHYSICAL_SWITCH = 0x51,
#define IDENTIFY_SWITCH_DEVICE 0x0
#define GET_PHYSICAL_PORT_STATE 0x1
+ TUNNEL = 0x53,
+ #define MANAGEMENT_COMMAND 0x0
};
+/* CCI Message Format CXL r3.0 Figure 7-19 */
+typedef struct CXLCCIMessage {
+ uint8_t category;
+ uint8_t tag;
+ uint8_t resv1;
+ uint8_t command;
+ uint8_t command_set;
+ uint8_t pl_length[3];
+ uint16_t vendor_specific;
+ uint16_t rc;
+ uint8_t payload[];
+} QEMU_PACKED CXLCCIMessage;
+
+static CXLRetCode cmd_tunnel_management_cmd(const struct cxl_cmd *cmd,
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLCCI *cci)
+{
+ CXLUpstreamPort *usp = CXL_USP(cci->d);
+ PCIDevice *tunnel_target;
+ struct {
+ uint8_t port_or_ld_id;
+ uint8_t target_type;
+ uint16_t size;
+ CXLCCIMessage ccimessage;
+ } *in;
+ struct {
+ uint16_t resp_len;
+ uint8_t resv[2];
+ CXLCCIMessage ccimessage;
+ } *out;
+
+ if (cmd->in < sizeof(*in)) {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+ in = (void *)payload_in;
+ out = (void*)payload_out;
+
+ if (cmd->in < sizeof(*in) + in->size) {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+ if (in->size < 3 * sizeof(uint32_t)) {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+ /* Need to find target CCI */
+ //Lets assume simple tunnel to port - find that device.
+ if (in->target_type != 0) {
+ printf("QEMU: sent to FM-LD which makes no sense yet\n");
+ }
+
+ tunnel_target = pcie_find_port_by_pn(&PCI_BRIDGE(usp)->sec_bus, in->port_or_ld_id);
+ if (!tunnel_target) {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+
+ tunnel_target = pci_bridge_get_sec_bus(PCI_BRIDGE(tunnel_target))->devices[0];
+ if (!tunnel_target) {
+ return CXL_MBOX_INVALID_INPUT;
+ }
+
+ if (object_dynamic_cast(OBJECT(tunnel_target), TYPE_CXL_TYPE3)) {
+ CXLType3Dev *ct3d = CXL_TYPE3(tunnel_target);
+ size_t pl_length = in->ccimessage.pl_length[2] << 16 |
+ in->ccimessage.pl_length[1] << 8 | in->ccimessage.pl_length[0];
+ size_t length_out;
+ bool bg_started;
+ int rc;
+
+ rc = cxl_process_cci_message(&ct3d->vdm_mctp_cci,
+ in->ccimessage.command_set,
+ in->ccimessage.command,
+ pl_length, in->ccimessage.payload,
+ &length_out, out->ccimessage.payload,
+ &bg_started);
+ /* Payload should be in place.. But rest of CCI header and needs filling */
+ out->resp_len = length_out + sizeof(CXLCCIMessage); /* CHECK */
+ st24_le_p(out->ccimessage.pl_length, length_out);
+ out->ccimessage.rc = rc;
+ printf("len_out is %lu\n", length_out);
+ *len_out = length_out + sizeof(*out);
+
+ return CXL_MBOX_SUCCESS;
+ }
+
+ return CXL_MBOX_INVALID_INPUT;
+}
static CXLRetCode cmd_events_get_records(const struct cxl_cmd *cmd,
uint8_t *payload_in, size_t len_in,
@@ -990,6 +1080,8 @@ static const struct cxl_cmd cxl_cmd_set_sw[256][256] = {
cmd_identify_switch_device, 0, 0x49 },
[PHYSICAL_SWITCH][GET_PHYSICAL_PORT_STATE] = { "SWITCH_PHYSICAL_PORT_STATS",
cmd_get_physical_port_state, ~0, ~0 },
+ [TUNNEL][MANAGEMENT_COMMAND] = { "TUNNEL_MANAGEMENT_COMMAND",
+ cmd_tunnel_management_cmd, ~0, ~0 },
};
/*
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index f479dc67e8..5714ea0c77 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1042,6 +1042,10 @@ static void ct3d_reset(DeviceState *dev)
cxl_component_register_init_common(reg_state, write_msk, CXL2_TYPE3_DEVICE);
cxl_device_register_init_t3(ct3d);
+
+ /* Bring up an endpoint to target with MCTP over VDM */
+ cxl_initialize_usp_mctpcci(&ct3d->vdm_mctp_cci, DEVICE(ct3d), DEVICE(ct3d),
+ 512); /* Max payload made up */
}
static Property ct3_props[] = {
--
2.39.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [RFC PATCH 10/17] misc/i2c_mctp_cxl: Initial device emulation
2023-07-17 17:16 ` [RFC PATCH 10/17] misc/i2c_mctp_cxl: Initial device emulation Jonathan Cameron via
@ 2023-07-18 21:30 ` Gregory Price
2023-07-19 8:19 ` Jonathan Cameron via
0 siblings, 1 reply; 23+ messages in thread
From: Gregory Price @ 2023-07-18 21:30 UTC (permalink / raw)
To: Jonathan Cameron
Cc: linux-cxl, Dan Williams, qemu-devel, linuxarm, Alison Schofield,
Ira Weiny, Dave Jiang, Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
On Mon, Jul 17, 2023 at 06:16:39PM +0100, Jonathan Cameron wrote:
> @@ -397,8 +401,9 @@ struct CXLType3Dev {
> AddressSpace hostpmem_as;
> CXLComponentState cxl_cstate;
> CXLDeviceState cxl_dstate;
> - CXLCCI cci;
> -
> + CXLCCI cci; /* Primary PCI mailbox CCI */
> + CXLCCI oob_mctp_cci; /* Initialized only if targetted */
> +
I've been humming and hawing over this on the MHD stuff because I wanted
to figure out how to "add a CCI command" to a type-3 device without
either having a billion definitions for CCI command sets - or doing
something like this.
I don't hate this design pattern, I just want to ask whether your
intent is to end up with CXLType3Dev hosting many CXLCCI's based on what
wrapper types you have.
Example: a type-3 device with mctp pass through and the MHD command set
CXLType3Dev {
...
CXLCCI cci;
CXLCCI oob_mctp_cci;
CXLCCI mhd_cci;
...
}
Instantiate:
-device cxl-type3,bus=swport0,memdev=cxl-mem1,id=cxl-pmem1,lsa=cxl-lsa1,sn=3
-device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=5,target=cxl-pmem1
-device cxl-mhd,target=cxl-pmem1,...whatever else...
where the MHD code is contained within its own type/file, and the type3
device hosts the CCI for it. Similar to how you've implemented the MTCP
stuff here.
The reason I ask is because certain CCI's don't necessarily get
associated with "a bus" so much as "a device". the MHD example - it's
still part of "the device", but it's optional. So does it make sense
to create this wrapper without a bus association, or to just pile it on
top CXLType3Dev and have to duplicate the code across any other
multi-headed devices that folks may conjur up?
~Gregory
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [RFC PATCH 10/17] misc/i2c_mctp_cxl: Initial device emulation
2023-07-18 21:30 ` Gregory Price
@ 2023-07-19 8:19 ` Jonathan Cameron via
2023-07-19 18:49 ` Gregory Price
0 siblings, 1 reply; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-19 8:19 UTC (permalink / raw)
To: Gregory Price
Cc: linux-cxl, Dan Williams, qemu-devel, linuxarm, Alison Schofield,
Ira Weiny, Dave Jiang, Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
On Tue, 18 Jul 2023 17:30:57 -0400
Gregory Price <gregory.price@memverge.com> wrote:
> On Mon, Jul 17, 2023 at 06:16:39PM +0100, Jonathan Cameron wrote:
> > @@ -397,8 +401,9 @@ struct CXLType3Dev {
> > AddressSpace hostpmem_as;
> > CXLComponentState cxl_cstate;
> > CXLDeviceState cxl_dstate;
> > - CXLCCI cci;
> > -
> > + CXLCCI cci; /* Primary PCI mailbox CCI */
> > + CXLCCI oob_mctp_cci; /* Initialized only if targetted */
> > +
>
> I've been humming and hawing over this on the MHD stuff because I wanted
> to figure out how to "add a CCI command" to a type-3 device without
> either having a billion definitions for CCI command sets - or doing
> something like this.
>
> I don't hate this design pattern, I just want to ask whether your
> intent is to end up with CXLType3Dev hosting many CXLCCI's based on what
> wrapper types you have.
>
> Example: a type-3 device with mctp pass through and the MHD command set
>
> CXLType3Dev {
> ...
> CXLCCI cci;
> CXLCCI oob_mctp_cci;
> CXLCCI mhd_cci;
> ...
> }
Yes - that's what I was thinking. In some cases a CCI may be accessed by
tunneling on a different CCI on the same device as well as the option
of tunneling to different devices.
So far the set that we'll end up with isn't too large. And if some aren't
used for a given instantiation that's fine if it keeps the code simple.
We may end up with other MCTP buses and to keep things consistent each one
will need it's own target CXLCCI. If we need to rethink and make it dynamic
to some degree we can look at it later.
>
> Instantiate:
> -device cxl-type3,bus=swport0,memdev=cxl-mem1,id=cxl-pmem1,lsa=cxl-lsa1,sn=3
> -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=5,target=cxl-pmem1
> -device cxl-mhd,target=cxl-pmem1,...whatever else...
Not sure on this - it may be implicit in creating an MHD rather than requiring
a command line to target through. Depends on what the MHD creation code
looks like - but this is definitely a possibility.
>
> where the MHD code is contained within its own type/file, and the type3
> device hosts the CCI for it. Similar to how you've implemented the MTCP
> stuff here.
>
> The reason I ask is because certain CCI's don't necessarily get
> associated with "a bus" so much as "a device". the MHD example - it's
> still part of "the device", but it's optional.
For emulation I don't think we care if it's optional. I think we implement
it whatever and if it is not accessed that is fine.
> So does it make sense
> to create this wrapper without a bus association, or to just pile it on
> top CXLType3Dev and have to duplicate the code across any other
> multi-headed devices that folks may conjur up?
Piling it on top of CXLType3Dev was what I was thinking. We can rethink if
there other multi-headed devices using similar interfaces.
Jonathan
>
> ~Gregory
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [RFC PATCH 10/17] misc/i2c_mctp_cxl: Initial device emulation
2023-07-19 8:19 ` Jonathan Cameron via
@ 2023-07-19 18:49 ` Gregory Price
2023-07-20 12:18 ` Jonathan Cameron via
0 siblings, 1 reply; 23+ messages in thread
From: Gregory Price @ 2023-07-19 18:49 UTC (permalink / raw)
To: Jonathan Cameron
Cc: linux-cxl, Dan Williams, qemu-devel, linuxarm, Alison Schofield,
Ira Weiny, Dave Jiang, Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
On Wed, Jul 19, 2023 at 09:19:47AM +0100, Jonathan Cameron wrote:
> On Tue, 18 Jul 2023 17:30:57 -0400
> Gregory Price <gregory.price@memverge.com> wrote:
>
> > On Mon, Jul 17, 2023 at 06:16:39PM +0100, Jonathan Cameron wrote:
> > > @@ -397,8 +401,9 @@ struct CXLType3Dev {
> > > AddressSpace hostpmem_as;
> > > CXLComponentState cxl_cstate;
> > > CXLDeviceState cxl_dstate;
> > > - CXLCCI cci;
> > > -
> > > + CXLCCI cci; /* Primary PCI mailbox CCI */
> > > + CXLCCI oob_mctp_cci; /* Initialized only if targetted */
> > > +
> >
> > I've been humming and hawing over this on the MHD stuff because I wanted
> > to figure out how to "add a CCI command" to a type-3 device without
> > either having a billion definitions for CCI command sets - or doing
> > something like this.
> >
> > I don't hate this design pattern, I just want to ask whether your
> > intent is to end up with CXLType3Dev hosting many CXLCCI's based on what
> > wrapper types you have.
> >
> > Example: a type-3 device with mctp pass through and the MHD command set
> >
> > CXLType3Dev {
> > ...
> > CXLCCI cci;
> > CXLCCI oob_mctp_cci;
> > CXLCCI mhd_cci;
> > ...
> > }
>
> Yes - that's what I was thinking. In some cases a CCI may be accessed by
> tunneling on a different CCI on the same device as well as the option
> of tunneling to different devices.
>
> So far the set that we'll end up with isn't too large. And if some aren't
> used for a given instantiation that's fine if it keeps the code simple.
> We may end up with other MCTP buses and to keep things consistent each one
> will need it's own target CXLCCI. If we need to rethink and make it dynamic
> to some degree we can look at it later.
>
Maybe a dangerous suggestion. Right now the CCI's are static:
static const struct cxl_cmd cxl_cmd_set[256][256]
how difficult might it be to allow these tables to be dynamic instead?
Then we could add an interface like this:
void cxl_add_cmd_set(CXLCCI *cci, CXLCCI *cmd_set, payload_max) {
copy(cci, cmd_set);
}
This would enable not just adding sub-components piece-meal, but also if
someone wants to model a real device with custom CCI commands, they can
simply define a CCI set and pass it in via
cxl_add_cmd_set(&ct3d->cci, my_cmd_set, payload_max);
Which lets the existing /dev/cxl/memN device dispatch those commands,
and makes modeling real devices an easier endeavor.
Only downside is that this may require changing the command structure to
include a callback type and pointer per cci function. The upside is this
would also allow commands to be written somewhat agnostic to the device
they're being inherited by and allow for device nesting like...
-device cxl-type3, id=ct3d
-device cxl-mhd, target=ct3d
-device my_vendor_cxl_type3, target=ct3d
etc etc
otherwise we're probably going to end up with a cxl-type3 -device line
300 characters long.
Maybe that's over-generalizing a bit much n.n;
~Gregory
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [RFC PATCH 10/17] misc/i2c_mctp_cxl: Initial device emulation
2023-07-19 18:49 ` Gregory Price
@ 2023-07-20 12:18 ` Jonathan Cameron via
2023-07-20 19:33 ` Gregory Price
0 siblings, 1 reply; 23+ messages in thread
From: Jonathan Cameron via @ 2023-07-20 12:18 UTC (permalink / raw)
To: Gregory Price
Cc: linux-cxl, Dan Williams, qemu-devel, linuxarm, Alison Schofield,
Ira Weiny, Dave Jiang, Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
On Wed, 19 Jul 2023 14:49:07 -0400
Gregory Price <gregory.price@memverge.com> wrote:
> On Wed, Jul 19, 2023 at 09:19:47AM +0100, Jonathan Cameron wrote:
> > On Tue, 18 Jul 2023 17:30:57 -0400
> > Gregory Price <gregory.price@memverge.com> wrote:
> >
> > > On Mon, Jul 17, 2023 at 06:16:39PM +0100, Jonathan Cameron wrote:
> > > > @@ -397,8 +401,9 @@ struct CXLType3Dev {
> > > > AddressSpace hostpmem_as;
> > > > CXLComponentState cxl_cstate;
> > > > CXLDeviceState cxl_dstate;
> > > > - CXLCCI cci;
> > > > -
> > > > + CXLCCI cci; /* Primary PCI mailbox CCI */
> > > > + CXLCCI oob_mctp_cci; /* Initialized only if targetted */
> > > > +
> > >
> > > I've been humming and hawing over this on the MHD stuff because I wanted
> > > to figure out how to "add a CCI command" to a type-3 device without
> > > either having a billion definitions for CCI command sets - or doing
> > > something like this.
> > >
> > > I don't hate this design pattern, I just want to ask whether your
> > > intent is to end up with CXLType3Dev hosting many CXLCCI's based on what
> > > wrapper types you have.
> > >
> > > Example: a type-3 device with mctp pass through and the MHD command set
> > >
> > > CXLType3Dev {
> > > ...
> > > CXLCCI cci;
> > > CXLCCI oob_mctp_cci;
> > > CXLCCI mhd_cci;
> > > ...
> > > }
> >
> > Yes - that's what I was thinking. In some cases a CCI may be accessed by
> > tunneling on a different CCI on the same device as well as the option
> > of tunneling to different devices.
> >
> > So far the set that we'll end up with isn't too large. And if some aren't
> > used for a given instantiation that's fine if it keeps the code simple.
> > We may end up with other MCTP buses and to keep things consistent each one
> > will need it's own target CXLCCI. If we need to rethink and make it dynamic
> > to some degree we can look at it later.
> >
>
> Maybe a dangerous suggestion. Right now the CCI's are static:
>
> static const struct cxl_cmd cxl_cmd_set[256][256]
That's defined by the ID space for the commands. There can't be more than
that many currently..
>
> how difficult might it be to allow these tables to be dynamic instead?
> Then we could add an interface like this:
>
> void cxl_add_cmd_set(CXLCCI *cci, CXLCCI *cmd_set, payload_max) {
> copy(cci, cmd_set);
> }
>
> This would enable not just adding sub-components piece-meal, but also if
> someone wants to model a real device with custom CCI commands, they can
> simply define a CCI set and pass it in via
>
> cxl_add_cmd_set(&ct3d->cci, my_cmd_set, payload_max);
Ok. I'm potentially fine with people adding an interface for this, but
only if they plan to also upstream the QEMU emulation of their actual
device.
>
> Which lets the existing /dev/cxl/memN device dispatch those commands,
> and makes modeling real devices an easier endeavor.
>
> Only downside is that this may require changing the command structure to
> include a callback type and pointer per cci function. The upside is this
> would also allow commands to be written somewhat agnostic to the device
> they're being inherited by and allow for device nesting like...
>
> -device cxl-type3, id=ct3d
> -device cxl-mhd, target=ct3d
> -device my_vendor_cxl_type3, target=ct3d
> etc etc
>
> otherwise we're probably going to end up with a cxl-type3 -device line
> 300 characters long.
>
> Maybe that's over-generalizing a bit much n.n;
I'd look to just inherit from a cxl type 3, like Ira did in the PoC for
type 2 support. We can then easily add a path to replace the commands
set with whatever anyone wants. I'm not sure we want the command line
to be used to configure such a device as it'll both get very complex and
prove increasingly hard to test more than a small subset of options.
https://lore.kernel.org/all/20230517-rfc-type2-dev-v1-0-6eb2e470981b@intel.com/
Jonathan
>
> ~Gregory
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [RFC PATCH 10/17] misc/i2c_mctp_cxl: Initial device emulation
2023-07-20 12:18 ` Jonathan Cameron via
@ 2023-07-20 19:33 ` Gregory Price
0 siblings, 0 replies; 23+ messages in thread
From: Gregory Price @ 2023-07-20 19:33 UTC (permalink / raw)
To: Jonathan Cameron
Cc: linux-cxl, Dan Williams, qemu-devel, linuxarm, Alison Schofield,
Ira Weiny, Dave Jiang, Davidlohr Bueso, Viacheslav Dubeyko,
Shesha Bhushan Sreenivasamurthy, Fan Ni, Michael Tsirkin,
Jonathan Zhang, Klaus Jensen
On Thu, Jul 20, 2023 at 01:18:33PM +0100, Jonathan Cameron wrote:
> On Wed, 19 Jul 2023 14:49:07 -0400
> Gregory Price <gregory.price@memverge.com> wrote:
>
> >
> > Maybe a dangerous suggestion. Right now the CCI's are static:
> >
> > static const struct cxl_cmd cxl_cmd_set[256][256]
>
> That's defined by the ID space for the commands. There can't be more than
> that many currently..
>
Meant commands beyond what is defined in the spec, not beyond the 256
limits.
> >
> > how difficult might it be to allow these tables to be dynamic instead?
> > Then we could add an interface like this:
> >
> > void cxl_add_cmd_set(CXLCCI *cci, CXLCCI *cmd_set, payload_max) {
> > copy(cci, cmd_set);
> > }
> >
> > This would enable not just adding sub-components piece-meal, but also if
> > someone wants to model a real device with custom CCI commands, they can
> > simply define a CCI set and pass it in via
> >
> > cxl_add_cmd_set(&ct3d->cci, my_cmd_set, payload_max);
>
> Ok. I'm potentially fine with people adding an interface for this, but
> only if they plan to also upstream the QEMU emulation of their actual
> device.
>
Working on it :]
Hoping to show off a fully functional MHSLD with some custom commands
soon, and I think I'm happy with the abstraction that fell out on top of
this CCI work. Previously it required duplicating the type3 device or
hacking directly on it, which is a maintenance nightmare / not
upstreamable.
>
> I'd look to just inherit from a cxl type 3, like Ira did in the PoC for
> type 2 support. We can then easily add a path to replace the commands
> set with whatever anyone wants. I'm not sure we want the command line
> to be used to configure such a device as it'll both get very complex and
> prove increasingly hard to test more than a small subset of options.
>
> https://lore.kernel.org/all/20230517-rfc-type2-dev-v1-0-6eb2e470981b@intel.com/
>
> Jonathan
>
I made an attempt at this at first, but due to the custom commands, i
think everyone would (rightly) scoff at the idea of adding
non-specification defined stuff into the core type 3 device. Once I
shifted to modifying the CCI and overriding entries, the entire vendor
device ended up as mostly the CCI command definitions, which is exactly
how i envisioned doing it in the first place.
I'll post some additional patches to my MHD RFC, the changes were pretty
minor.
Hopefully will be able to tack on a concrete MHSLD following that..
~Gregory
^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2023-07-20 19:39 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-07-17 17:16 [RFC PATCH 00/17] hw/cxl: hw/cxl: Generic CCI emulation support Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 01/17] hw/pci-bridge/cxl_upstream: Move defintion of device to header Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 02/17] hw/cxl/mailbox: Enable mulitple mailbox command sets Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 03/17] cxl/mbox: Pull the payload out of struct cxl_cmd and make instances constant Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 04/17] hw/mbox: Split mailbox command payload into separate input and output Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 05/17] cxl/mbox: Pull the CCI definition out of the CXLDeviceState Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 06/17] cxl/mbox: Generalize the CCI command processing Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 07/17] hw/acpi/aml-build: add function for i2c slave device serial bus description Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 08/17] hw/i2c: add mctp core Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 09/17] i2c/mctp: Allow receiving messages to dest eid 0 Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 10/17] misc/i2c_mctp_cxl: Initial device emulation Jonathan Cameron via
2023-07-18 21:30 ` Gregory Price
2023-07-19 8:19 ` Jonathan Cameron via
2023-07-19 18:49 ` Gregory Price
2023-07-20 12:18 ` Jonathan Cameron via
2023-07-20 19:33 ` Gregory Price
2023-07-17 17:16 ` [RFC PATCH 11/17] HACK: arm/virt: Add aspeed-i2c controller and MCTP EP to enable MCTP testing Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 12/17] HACK: hw/arm/virt: Add ACPI support for aspeed-i2c / mctp Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 13/17] HACK: hw/i386/pc: Add Aspeed i2c controller + MCTP with ACPI tables Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 14/17] docs: cxl: Add example commandline for MCTP CXL CCIs Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 15/17] hw/cxl: Add a switch mailbox CCI function Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 16/17] hw/cxl: Implement Physical Ports status retrieval Jonathan Cameron via
2023-07-17 17:16 ` [RFC PATCH 17/17] hw/cxl: Add tunneled command support to mailbox for switch cci Jonathan Cameron via
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).