From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01EAEC433E0 for ; Wed, 27 Jan 2021 21:22:31 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 26E5560C3E for ; Wed, 27 Jan 2021 21:22:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 26E5560C3E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:50994 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l4sGi-0007U9-Tc for qemu-devel@archiver.kernel.org; Wed, 27 Jan 2021 16:22:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42576) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l4sFf-0006QT-2F for qemu-devel@nongnu.org; Wed, 27 Jan 2021 16:21:23 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:42422) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1l4sFa-0000PZ-L1 for qemu-devel@nongnu.org; Wed, 27 Jan 2021 16:21:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1611782477; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RRjFj4jzBHGAJKpqju103NfAVvZRpzd/CdXZnwQoMC0=; b=KW4l+BDCPoDM+FpKIGEOUOSGqhkWMgNVxvjampMwFMShzeqDBYp9ChxkSj9ekeeopvNWhI Upp1BFnzUx9sYYhCEtWxQ4iqjorjCo0b3PXFVXv8jvDGF3nEehJwzgdctvYk6YJolBEasE jO0E+gb9FN+3obWsC7jkKL8pYggIzgI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-595-JgDBx_gmMmO77USp4uF0tA-1; Wed, 27 Jan 2021 16:21:14 -0500 X-MC-Unique: JgDBx_gmMmO77USp4uF0tA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 799C01936B61; Wed, 27 Jan 2021 21:21:12 +0000 (UTC) Received: from localhost (unknown [10.40.208.33]) by smtp.corp.redhat.com (Postfix) with ESMTP id 439165D9CA; Wed, 27 Jan 2021 21:21:05 +0000 (UTC) Date: Wed, 27 Jan 2021 22:21:04 +0100 From: Igor Mammedov To: Ben Widawsky Subject: Re: [RFC PATCH v2 24/32] hw/cxl/device: Add a memory device (8.2.8.5) Message-ID: <20210127222104.7b7ad988@redhat.com> In-Reply-To: <20210127211116.3425vvaylg6rzbuv@intel.com> References: <20210105165323.783725-1-ben.widawsky@intel.com> <20210105165323.783725-25-ben.widawsky@intel.com> <20210127220312.6850abe2@redhat.com> <20210127211116.3425vvaylg6rzbuv@intel.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=imammedo@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Received-SPF: pass client-ip=216.205.24.124; envelope-from=imammedo@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.308, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , "Michael S. Tsirkin" , Vishal Verma , Chris Browy , Philippe =?UTF-8?B?TWF0aGlldS1EYXVkw6k=?= , qemu-devel@nongnu.org, Jonathan Cameron , Prashant V Agarwal , Dan Williams Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Wed, 27 Jan 2021 13:11:16 -0800 Ben Widawsky wrote: > On 21-01-27 22:03:12, Igor Mammedov wrote: > > On Tue, 5 Jan 2021 08:53:15 -0800 > > Ben Widawsky wrote: > > > > > A CXL memory device (AKA Type 3) is a CXL component that contains some > > > combination of volatile and persistent memory. It also implements the > > > previously defined mailbox interface as well as the memory device > > > firmware interface. > > > > > > The following example will create a 256M device in a 512M window: > > > > > > -object "memory-backend-file,id=cxl-mem1,share,mem-path=cxl-type3,size=512M" > > > -device "cxl-type3,bus=rp0,memdev=cxl-mem1,id=cxl-pmem0,size=256M" > > > > I'd expect whole backend used by frontend, so one would not need "size" property > > on frontend (like we do with memory devices). > > So question is why it partially uses memdev? > > A CXL memory device may participate in an interleave set. In such a case, it > would be < the total size of the memory window. > > This isn't implemented in the code yet, but it is planned. could you add here how it supposed to look like CLI interface wise? also see other questions below. > > > > > > > Signed-off-by: Ben Widawsky > > > --- > > > hw/core/numa.c | 3 + > > > hw/cxl/cxl-mailbox-utils.c | 41 ++++++ > > > hw/i386/pc.c | 1 + > > > hw/mem/Kconfig | 5 + > > > hw/mem/cxl_type3.c | 262 +++++++++++++++++++++++++++++++++++++ > > > hw/mem/meson.build | 1 + > > > hw/pci/pcie.c | 30 +++++ > > > include/hw/cxl/cxl.h | 2 + > > > include/hw/cxl/cxl_pci.h | 22 ++++ > > > include/hw/pci/pci_ids.h | 1 + > > > monitor/hmp-cmds.c | 15 +++ > > > qapi/machine.json | 1 + > > > 12 files changed, 384 insertions(+) > > > create mode 100644 hw/mem/cxl_type3.c > > > > > > diff --git a/hw/core/numa.c b/hw/core/numa.c > > > index 68cee65f61..cd7df371e6 100644 > > > --- a/hw/core/numa.c > > > +++ b/hw/core/numa.c > > > @@ -770,6 +770,9 @@ static void numa_stat_memory_devices(NumaNodeMem node_mem[]) > > > node_mem[pcdimm_info->node].node_plugged_mem += > > > pcdimm_info->size; > > > break; > > > + case MEMORY_DEVICE_INFO_KIND_CXL: > > > + /* FINISHME */ > > > + break; > > > case MEMORY_DEVICE_INFO_KIND_VIRTIO_PMEM: > > > vpi = value->u.virtio_pmem.data; > > > /* TODO: once we support numa, assign to right node */ > > > diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c > > > index f68ec5b5b9..eeb10b8943 100644 > > > --- a/hw/cxl/cxl-mailbox-utils.c > > > +++ b/hw/cxl/cxl-mailbox-utils.c > > > @@ -49,6 +49,8 @@ enum { > > > LOGS = 0x04, > > > #define GET_SUPPORTED 0x0 > > > #define GET_LOG 0x1 > > > + IDENTIFY = 0x40, > > > + #define MEMORY_DEVICE 0x0 > > > }; > > > > > > /* 8.2.8.4.5.1 Command Return Codes */ > > > @@ -127,6 +129,7 @@ declare_mailbox_handler(TIMESTAMP_GET); > > > declare_mailbox_handler(TIMESTAMP_SET); > > > declare_mailbox_handler(LOGS_GET_SUPPORTED); > > > declare_mailbox_handler(LOGS_GET_LOG); > > > +declare_mailbox_handler(IDENTIFY_MEMORY_DEVICE); > > > > > > #define IMMEDIATE_CONFIG_CHANGE (1 << 1) > > > #define IMMEDIATE_POLICY_CHANGE (1 << 3) > > > @@ -144,6 +147,7 @@ static struct cxl_cmd cxl_cmd_set[256][256] = { > > > CXL_CMD(TIMESTAMP, SET, 8, IMMEDIATE_POLICY_CHANGE), > > > CXL_CMD(LOGS, GET_SUPPORTED, 0, 0), > > > CXL_CMD(LOGS, GET_LOG, 0x18, 0), > > > + CXL_CMD(IDENTIFY, MEMORY_DEVICE, 0, 0), > > > }; > > > > > > #undef CXL_CMD > > > @@ -262,6 +266,43 @@ define_mailbox_handler(LOGS_GET_LOG) > > > return CXL_MBOX_SUCCESS; > > > } > > > > > > +/* 8.2.9.5.1.1 */ > > > +define_mailbox_handler(IDENTIFY_MEMORY_DEVICE) > > > +{ > > > + struct { > > > + char fw_revision[0x10]; > > > + uint64_t total_capacity; > > > + uint64_t volatile_capacity; > > > + uint64_t persistent_capacity; > > > + uint64_t partition_align; > > > + uint16_t info_event_log_size; > > > + uint16_t warning_event_log_size; > > > + uint16_t failure_event_log_size; > > > + uint16_t fatal_event_log_size; > > > + uint32_t lsa_size; > > > + uint8_t poison_list_max_mer[3]; > > > + uint16_t inject_poison_limit; > > > + uint8_t poison_caps; > > > + uint8_t qos_telemetry_caps; > > > + } __attribute__((packed)) *id; > > > + _Static_assert(sizeof(*id) == 0x43, "Bad identify size"); > > > + > > > + if (memory_region_size(cxl_dstate->pmem) < (256 << 20)) { > > > + return CXL_MBOX_INTERNAL_ERROR; > > > + } > > > + > > > + id = (void *)cmd->payload; > > > + memset(id, 0, sizeof(*id)); > > > + > > > + /* PMEM only */ > > > + snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0); > > > + id->total_capacity = memory_region_size(cxl_dstate->pmem); > > > + id->persistent_capacity = memory_region_size(cxl_dstate->pmem); > > > + > > > + *len = sizeof(*id); > > > + return CXL_MBOX_SUCCESS; > > > +} > > > + > > > void cxl_process_mailbox(CXLDeviceState *cxl_dstate) > > > { > > > uint16_t ret = CXL_MBOX_SUCCESS; > > > diff --git a/hw/i386/pc.c b/hw/i386/pc.c > > > index 5458f61d10..5d41809b37 100644 > > > --- a/hw/i386/pc.c > > > +++ b/hw/i386/pc.c > > > @@ -79,6 +79,7 @@ > > > #include "acpi-build.h" > > > #include "hw/mem/pc-dimm.h" > > > #include "hw/mem/nvdimm.h" > > > +#include "hw/cxl/cxl.h" > > > #include "qapi/error.h" > > > #include "qapi/qapi-visit-common.h" > > > #include "qapi/visitor.h" > > > diff --git a/hw/mem/Kconfig b/hw/mem/Kconfig > > > index a0ef2cf648..7d9d1ced3e 100644 > > > --- a/hw/mem/Kconfig > > > +++ b/hw/mem/Kconfig > > > @@ -10,3 +10,8 @@ config NVDIMM > > > default y > > > depends on (PC || PSERIES || ARM_VIRT) > > > select MEM_DEVICE > > > + > > > +config CXL_MEM_DEVICE > > > + bool > > > + default y if CXL > > > + select MEM_DEVICE > > > diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c > > > new file mode 100644 > > > index 0000000000..3985bb8d0b > > > --- /dev/null > > > +++ b/hw/mem/cxl_type3.c > > > @@ -0,0 +1,262 @@ > > > +#include "qemu/osdep.h" > > > +#include "qemu/units.h" > > > +#include "qemu/error-report.h" > > > +#include "hw/mem/memory-device.h" > > > +#include "hw/mem/pc-dimm.h" > > > +#include "hw/pci/pci.h" > > > +#include "hw/qdev-properties.h" > > > +#include "qapi/error.h" > > > +#include "qemu/log.h" > > > +#include "qemu/module.h" > > > +#include "qemu/range.h" > > > +#include "qemu/rcu.h" > > > +#include "sysemu/hostmem.h" > > > +#include "hw/cxl/cxl.h" > > > + > > > +typedef struct cxl_type3_dev { > > > + /* Private */ > > > + PCIDevice parent_obj; > > > + > > > + /* Properties */ > > > + uint64_t size; > > > + HostMemoryBackend *hostmem; > > > + > > > + /* State */ > > > + CXLComponentState cxl_cstate; > > > + CXLDeviceState cxl_dstate; > > > +} CXLType3Dev; > > > + > > > +#define CT3(obj) OBJECT_CHECK(CXLType3Dev, (obj), TYPE_CXL_TYPE3_DEV) > > > + > > > +static void build_dvsecs(CXLType3Dev *ct3d) > > > +{ > > > + CXLComponentState *cxl_cstate = &ct3d->cxl_cstate; > > > + uint8_t *dvsec; > > > + > > > + dvsec = (uint8_t *)&(struct dvsec_device){ > > > + .cap = 0x1e, > > > + .ctrl = 0x6, > > > + .status2 = 0x2, > > > + .range1_size_hi = 0, > > > + .range1_size_lo = (2 << 5) | (2 << 2) | 0x3 | ct3d->size, > > > + .range1_base_hi = 0, > > > + .range1_base_lo = 0, > > > + }; > > > + cxl_component_create_dvsec(cxl_cstate, PCIE_CXL_DEVICE_DVSEC_LENGTH, > > > + PCIE_CXL_DEVICE_DVSEC, > > > + PCIE_CXL2_DEVICE_DVSEC_REVID, dvsec); > > > + > > > + dvsec = (uint8_t *)&(struct dvsec_register_locator){ > > > + .rsvd = 0, > > > + .reg0_base_lo = RBI_COMPONENT_REG | COMPONENT_REG_BAR_IDX, > > > + .reg0_base_hi = 0, > > > + .reg1_base_lo = RBI_CXL_DEVICE_REG | DEVICE_REG_BAR_IDX, > > > + .reg1_base_hi = 0, > > > + }; > > > + cxl_component_create_dvsec(cxl_cstate, REG_LOC_DVSEC_LENGTH, REG_LOC_DVSEC, > > > + REG_LOC_DVSEC_REVID, dvsec); > > > +} > > > + > > > +static void ct3_instance_init(Object *obj) > > > +{ > > > + /* MemoryDeviceClass *mdc = MEMORY_DEVICE_GET_CLASS(obj); */ > > > +} > > > + > > > +static void ct3_finalize(Object *obj) > > > +{ > > > + CXLType3Dev *ct3d = CT3(obj); > > > + > > > + g_free(ct3d->cxl_dstate.pmem); > > > +} > > > + > > > +static void cxl_setup_memory(CXLType3Dev *ct3d, Error **errp) > > > +{ > > > + MemoryRegionSection mrs; > > > + MemoryRegion *mr; > > > + uint64_t offset = 0; > > > + size_t remaining_size; > > > + > > > + if (!ct3d->hostmem) { > > > + error_setg(errp, "memdev property must be set"); > > > + return; > > > + } > > > + > > > + /* FIXME: need to check mr is the host bridge's MR */ > > > + mr = host_memory_backend_get_memory(ct3d->hostmem); > > > + > > > + /* Create our new subregion */ > > > + ct3d->cxl_dstate.pmem = g_new(MemoryRegion, 1); > > > + > > > + /* Find the first free space in the window */ > > > + WITH_RCU_READ_LOCK_GUARD() > > > + { > > > + mrs = memory_region_find(mr, offset, 1); > > > + while (mrs.mr && mrs.mr != mr) { > > > + offset += memory_region_size(mrs.mr); > > > + mrs = memory_region_find(mr, offset, 1); > > > + } > > > + } > > > + > > > + remaining_size = memory_region_size(mr) - offset; > > > + if (remaining_size < ct3d->size) { > > > + g_free(ct3d->cxl_dstate.pmem); > > > + error_setg(errp, > > > + "Not enough free space (%zd) required for device (%" PRId64 ")", > > > + remaining_size, ct3d->size); > > > + } > > > + > > > + /* Register our subregion as non-volatile */ > > > + memory_region_init_ram(ct3d->cxl_dstate.pmem, OBJECT(ct3d), > > > + "cxl_type3-memory", ct3d->size, errp); > > this allocates ct3d->size of anon RAM, was this an intention? > > If yes, can you clarify why extra RAM is used instead of using what > > backend provides? > > > > > + memory_region_set_nonvolatile(ct3d->cxl_dstate.pmem, true); > > > > > +#ifdef SET_PMEM_PADDR > > > + memory_region_add_subregion(mr, offset, ct3d->cxl_dstate.pmem); > > > +#endif > > What this hunk is supposed to do, why it's ifdef-ed? > > > > > > > +} > > > + > > > +static MemoryRegion *cxl_md_get_memory_region(MemoryDeviceState *md, > > > + Error **errp) > > > +{ > > > + CXLType3Dev *ct3d = CT3(md); > > > + > > > + if (!ct3d->cxl_dstate.pmem) { > > > + cxl_setup_memory(ct3d, errp); > > > + } > > > + > > > + return ct3d->cxl_dstate.pmem; > > > +} > > > + > > > +static void ct3_realize(PCIDevice *pci_dev, Error **errp) > > > +{ > > > + CXLType3Dev *ct3d = CT3(pci_dev); > > > + CXLComponentState *cxl_cstate = &ct3d->cxl_cstate; > > > + ComponentRegisters *regs = &cxl_cstate->crb; > > > + MemoryRegion *mr = ®s->component_registers; > > > + uint8_t *pci_conf = pci_dev->config; > > > + > > > + if (!ct3d->cxl_dstate.pmem) { > > > + cxl_setup_memory(ct3d, errp); > > > + } > > > + > > > + pci_config_set_prog_interface(pci_conf, 0x10); > > > + pci_config_set_class(pci_conf, PCI_CLASS_MEMORY_CXL); > > > + > > > + pcie_endpoint_cap_init(pci_dev, 0x80); > > > + cxl_cstate->dvsec_offset = 0x100; > > > + > > > + ct3d->cxl_cstate.pdev = pci_dev; > > > + build_dvsecs(ct3d); > > > + > > > + cxl_component_register_block_init(OBJECT(pci_dev), cxl_cstate, > > > + TYPE_CXL_TYPE3_DEV); > > > + > > > + pci_register_bar( > > > + pci_dev, COMPONENT_REG_BAR_IDX, > > > + PCI_BASE_ADDRESS_SPACE_MEMORY | PCI_BASE_ADDRESS_MEM_TYPE_64, mr); > > > + > > > + cxl_device_register_block_init(OBJECT(pci_dev), &ct3d->cxl_dstate); > > > + pci_register_bar(pci_dev, DEVICE_REG_BAR_IDX, > > > + PCI_BASE_ADDRESS_SPACE_MEMORY | > > > + PCI_BASE_ADDRESS_MEM_TYPE_64, > > > + &ct3d->cxl_dstate.device_registers); > > > +} > > > + > > > +static uint64_t cxl_md_get_addr(const MemoryDeviceState *md) > > > +{ > > > + CXLType3Dev *ct3d = CT3(md); > > > + > > > + return memory_region_get_ram_addr(ct3d->cxl_dstate.pmem); > > > +} > > > + > > > +static void cxl_md_set_addr(MemoryDeviceState *md, uint64_t addr, Error **errp) > > > +{ > > > + object_property_set_uint(OBJECT(md), "paddr", addr, errp); > > > +} > > > + > > > +static void ct3d_reset(DeviceState *dev) > > > +{ > > > + CXLType3Dev *ct3d = CT3(dev); > > > + uint32_t *reg_state = ct3d->cxl_cstate.crb.cache_mem_registers; > > > + > > > + cxl_component_register_init_common(reg_state, CXL2_TYPE3_DEVICE); > > > + cxl_device_register_init_common(&ct3d->cxl_dstate); > > > +} > > > + > > > +static Property ct3_props[] = { > > > + DEFINE_PROP_SIZE("size", CXLType3Dev, size, -1), > > > + DEFINE_PROP_LINK("memdev", CXLType3Dev, hostmem, TYPE_MEMORY_BACKEND, > > > + HostMemoryBackend *), > > > + DEFINE_PROP_END_OF_LIST(), > > > +}; > > > + > > > +static void pc_dimm_md_fill_device_info(const MemoryDeviceState *md, > > > + MemoryDeviceInfo *info) > > > +{ > > > + PCDIMMDeviceInfo *di = g_new0(PCDIMMDeviceInfo, 1); > > > + const DeviceClass *dc = DEVICE_GET_CLASS(md); > > > + const DeviceState *dev = DEVICE(md); > > > + CXLType3Dev *ct3d = CT3(md); > > > + > > > + if (dev->id) { > > > + di->has_id = true; > > > + di->id = g_strdup(dev->id); > > > + } > > > + di->hotplugged = dev->hotplugged; > > > + di->hotpluggable = dc->hotpluggable; > > > + di->addr = cxl_md_get_addr(md); > > > + di->slot = 0; > > > + di->node = 0; > > > + di->size = memory_device_get_region_size(md, NULL); > > > + di->memdev = object_get_canonical_path(OBJECT(ct3d->hostmem)); > > > + > > > + > > > + info->u.cxl.data = di; > > > + info->type = MEMORY_DEVICE_INFO_KIND_CXL; > > > +} > > > + > > > +static void ct3_class_init(ObjectClass *oc, void *data) > > > +{ > > > + DeviceClass *dc = DEVICE_CLASS(oc); > > > + PCIDeviceClass *pc = PCI_DEVICE_CLASS(oc); > > > + MemoryDeviceClass *mdc = MEMORY_DEVICE_CLASS(oc); > > > + > > > + pc->realize = ct3_realize; > > > + pc->class_id = PCI_CLASS_STORAGE_EXPRESS; > > > + pc->vendor_id = PCI_VENDOR_ID_INTEL; > > > + pc->device_id = 0xd93; /* LVF for now */ > > > + pc->revision = 1; > > > + > > > + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); > > > + dc->desc = "CXL PMEM Device (Type 3)"; > > > + dc->reset = ct3d_reset; > > > + device_class_set_props(dc, ct3_props); > > > + > > > + mdc->get_memory_region = cxl_md_get_memory_region; > > > + mdc->get_addr = cxl_md_get_addr; > > > + mdc->fill_device_info = pc_dimm_md_fill_device_info; > > > + mdc->get_plugged_size = memory_device_get_region_size; > > > + mdc->set_addr = cxl_md_set_addr; > > > +} > > > + > > > +static const TypeInfo ct3d_info = { > > > + .name = TYPE_CXL_TYPE3_DEV, > > > + .parent = TYPE_PCI_DEVICE, > > > + .class_init = ct3_class_init, > > > + .instance_size = sizeof(CXLType3Dev), > > > + .instance_init = ct3_instance_init, > > > + .instance_finalize = ct3_finalize, > > > + .interfaces = (InterfaceInfo[]) { > > > + { TYPE_MEMORY_DEVICE }, > > > + { INTERFACE_CXL_DEVICE }, > > > + { INTERFACE_PCIE_DEVICE }, > > > + {} > > > + }, > > > +}; > > > + > > > +static void ct3d_registers(void) > > > +{ > > > + type_register_static(&ct3d_info); > > > +} > > > + > > > +type_init(ct3d_registers); > > > diff --git a/hw/mem/meson.build b/hw/mem/meson.build > > > index 0d22f2b572..d13c3ed117 100644 > > > --- a/hw/mem/meson.build > > > +++ b/hw/mem/meson.build > > > @@ -3,5 +3,6 @@ mem_ss.add(files('memory-device.c')) > > > mem_ss.add(when: 'CONFIG_DIMM', if_true: files('pc-dimm.c')) > > > mem_ss.add(when: 'CONFIG_NPCM7XX', if_true: files('npcm7xx_mc.c')) > > > mem_ss.add(when: 'CONFIG_NVDIMM', if_true: files('nvdimm.c')) > > > +mem_ss.add(when: 'CONFIG_CXL_MEM_DEVICE', if_true: files('cxl_type3.c')) > > > > > > softmmu_ss.add_all(when: 'CONFIG_MEM_DEVICE', if_true: mem_ss) > > > diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c > > > index d4010cf8f3..1ecf6f6a55 100644 > > > --- a/hw/pci/pcie.c > > > +++ b/hw/pci/pcie.c > > > @@ -20,6 +20,7 @@ > > > > > > #include "qemu/osdep.h" > > > #include "qapi/error.h" > > > +#include "hw/mem/memory-device.h" > > > #include "hw/pci/pci_bridge.h" > > > #include "hw/pci/pcie.h" > > > #include "hw/pci/msix.h" > > > @@ -27,6 +28,8 @@ > > > #include "hw/pci/pci_bus.h" > > > #include "hw/pci/pcie_regs.h" > > > #include "hw/pci/pcie_port.h" > > > +#include "hw/cxl/cxl.h" > > > +#include "hw/boards.h" > > > #include "qemu/range.h" > > > > > > //#define DEBUG_PCIE > > > @@ -419,6 +422,28 @@ void pcie_cap_slot_pre_plug_cb(HotplugHandler *hotplug_dev, DeviceState *dev, > > > } > > > > > > pcie_cap_slot_plug_common(PCI_DEVICE(hotplug_dev), dev, errp); > > > + > > > +#ifdef CXL_MEM_DEVICE > > > + /* > > > + * FIXME: > > > + * if (object_dynamic_cast(OBJECT(dev), TYPE_CXL_TYPE3_DEV)) { > > > + * HotplugHandler *hotplug_ctrl; > > > + * Error *local_err = NULL; > > > + * hotplug_ctrl = qdev_get_hotplug_handler(dev); > > > + * if (hotplug_ctrl) { > > > + * hotplug_handler_pre_plug(hotplug_ctrl, dev, &local_err); > > > + * if (local_err) { > > > + * error_propagate(errp, local_err); > > > + * return; > > > + * } > > > + * } > > > + */ > > > + if (object_dynamic_cast(OBJECT(dev), TYPE_CXL_TYPE3_DEV)) { > > > + memory_device_pre_plug(MEMORY_DEVICE(dev), MACHINE(qdev_get_machine()), > > > + NULL, errp); > > > + } > > > > why use MEMORY_DEVICE interface instead of exposing memory as PCI BAR? > > > > > +#endif > > > } > > > > > > void pcie_cap_slot_plug_cb(HotplugHandler *hotplug_dev, DeviceState *dev, > > > @@ -455,6 +480,11 @@ void pcie_cap_slot_plug_cb(HotplugHandler *hotplug_dev, DeviceState *dev, > > > pcie_cap_slot_event(hotplug_pdev, > > > PCI_EXP_HP_EV_PDC | PCI_EXP_HP_EV_ABP); > > > } > > > + > > > +#ifdef CXL_MEM_DEVICE > > > + if (object_dynamic_cast(OBJECT(dev), TYPE_CXL_TYPE3_DEV)) > > > + memory_device_plug(MEMORY_DEVICE(dev), MACHINE(qdev_get_machine())); > > > +#endif > > > } > > > > > > void pcie_cap_slot_unplug_cb(HotplugHandler *hotplug_dev, DeviceState *dev, > > > diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h > > > index b1e5f4a8fa..809ed7de60 100644 > > > --- a/include/hw/cxl/cxl.h > > > +++ b/include/hw/cxl/cxl.h > > > @@ -17,6 +17,8 @@ > > > #define COMPONENT_REG_BAR_IDX 0 > > > #define DEVICE_REG_BAR_IDX 2 > > > > > > +#define TYPE_CXL_TYPE3_DEV "cxl-type3" > > > + > > > #define CXL_HOST_BASE 0xD0000000 > > > #define CXL_WINDOW_MAX 10 > > > > > > diff --git a/include/hw/cxl/cxl_pci.h b/include/hw/cxl/cxl_pci.h > > > index a53c2e5ae7..9ec28c9feb 100644 > > > --- a/include/hw/cxl/cxl_pci.h > > > +++ b/include/hw/cxl/cxl_pci.h > > > @@ -64,6 +64,28 @@ _Static_assert(sizeof(struct dvsec_header) == 10, > > > * CXL 2.0 Downstream Port: 3, 4, 7, 8 > > > */ > > > > > > +/* CXL 2.0 - 8.1.3 (ID 0001) */ > > > +struct dvsec_device { > > > + struct dvsec_header hdr; > > > + uint16_t cap; > > > + uint16_t ctrl; > > > + uint16_t status; > > > + uint16_t ctrl2; > > > + uint16_t status2; > > > + uint16_t lock; > > > + uint16_t cap2; > > > + uint32_t range1_size_hi; > > > + uint32_t range1_size_lo; > > > + uint32_t range1_base_hi; > > > + uint32_t range1_base_lo; > > > + uint32_t range2_size_hi; > > > + uint32_t range2_size_lo; > > > + uint32_t range2_base_hi; > > > + uint32_t range2_base_lo; > > > +}; > > > +_Static_assert(sizeof(struct dvsec_device) == 0x38, > > > + "dvsec device size incorrect"); > > > + > > > /* CXL 2.0 - 8.1.5 (ID 0003) */ > > > struct extensions_dvsec_port { > > > struct dvsec_header hdr; > > > diff --git a/include/hw/pci/pci_ids.h b/include/hw/pci/pci_ids.h > > > index 11f8ab7149..76bf3ed590 100644 > > > --- a/include/hw/pci/pci_ids.h > > > +++ b/include/hw/pci/pci_ids.h > > > @@ -53,6 +53,7 @@ > > > #define PCI_BASE_CLASS_MEMORY 0x05 > > > #define PCI_CLASS_MEMORY_RAM 0x0500 > > > #define PCI_CLASS_MEMORY_FLASH 0x0501 > > > +#define PCI_CLASS_MEMORY_CXL 0x0502 > > > #define PCI_CLASS_MEMORY_OTHER 0x0580 > > > > > > #define PCI_BASE_CLASS_BRIDGE 0x06 > > > diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c > > > index 0dd594f92b..0f67bc61ce 100644 > > > --- a/monitor/hmp-cmds.c > > > +++ b/monitor/hmp-cmds.c > > > @@ -1887,6 +1887,21 @@ void hmp_info_memory_devices(Monitor *mon, const QDict *qdict) > > > monitor_printf(mon, " hotpluggable: %s\n", > > > di->hotpluggable ? "true" : "false"); > > > break; > > > + case MEMORY_DEVICE_INFO_KIND_CXL: > > > + di = value->u.cxl.data; > > > + monitor_printf(mon, "Memory device [%s]: \"%s\"\n", > > > + MemoryDeviceInfoKind_str(value->type), > > > + di->id ? di->id : ""); > > > + monitor_printf(mon, " addr: 0x%" PRIx64 "\n", di->addr); > > > + monitor_printf(mon, " slot: %" PRId64 "\n", di->slot); > > > + monitor_printf(mon, " node: %" PRId64 "\n", di->node); > > > + monitor_printf(mon, " size: %" PRIu64 "\n", di->size); > > > + monitor_printf(mon, " memdev: %s\n", di->memdev); > > > + monitor_printf(mon, " hotplugged: %s\n", > > > + di->hotplugged ? "true" : "false"); > > > + monitor_printf(mon, " hotpluggable: %s\n", > > > + di->hotpluggable ? "true" : "false"); > > > + break; > > > case MEMORY_DEVICE_INFO_KIND_VIRTIO_PMEM: > > > vpi = value->u.virtio_pmem.data; > > > monitor_printf(mon, "Memory device [%s]: \"%s\"\n", > > > diff --git a/qapi/machine.json b/qapi/machine.json > > > index 330189efe3..aa96d662bd 100644 > > > --- a/qapi/machine.json > > > +++ b/qapi/machine.json > > > @@ -1394,6 +1394,7 @@ > > > { 'union': 'MemoryDeviceInfo', > > > 'data': { 'dimm': 'PCDIMMDeviceInfo', > > > 'nvdimm': 'PCDIMMDeviceInfo', > > > + 'cxl': 'PCDIMMDeviceInfo', > > > 'virtio-pmem': 'VirtioPMEMDeviceInfo', > > > 'virtio-mem': 'VirtioMEMDeviceInfo' > > > } > > >