From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E39950AC14 for ; Tue, 8 Sep 2026 10:22:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.43 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788862927; cv=none; b=uQXQewnmSCRlb6GpSkU7Xsd22CPVRD1WV9s+4lFgisatx9U6pczujgxY0AbqzVG3jQh136C2Ld1OYzGwocp1uvceN7zoyHZlllKT6HbAseKvbbWf+DFJkagp5rtPrpJ6hK/R1kqTAVQM7IVaz1SqNv2zD31d9x9v7xp3nyiIQU0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788862927; c=relaxed/simple; bh=QGabIKARS0Ot2NyBx1l+pS2bwXrbt5gxUGD8uJdonuA=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=fcNBLTk0RMakXpil0Dq6e5T7hYpdONcWk3phugzPTBu6/hQU5X84HTUylHMmRkcSHTmBuBW2XV0Gkmq7m/kdzAvOMkSCxSdWT8kfoz9T/Q1GRa+aOxjFdT/JVaECdwA2pT1Btj88yzRB3/Y7JS7/SaODVNyp6KvYv4iY7h6RcYs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=tB5Vc1SA; arc=none smtp.client-ip=209.85.216.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="tB5Vc1SA" Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-3964dfb5b9aso5441544a91.1 for ; Tue, 08 Sep 2026 03:22:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788862925; x=1789467725; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=ieAyrn9dOUbi/uQHpx4XPNGdMoW3ipgyyKgN9345CbI=; b=tB5Vc1SAQb+gJ8eVUGmV6dqmD1+4DY3Yq5Z2eGwMc/Hssxne7dKwDH4at4mIbwjk6Y Vlo0n9eN/28hl8Dqm0KNiNRDWJ/Ler8LrjXMtmoJWwOcsw39Xu9n4e/SnrK1as+eyUhc m/7pK4F0+Ng/g6Sv8621NgzHajIAT/wrafBnl7f5Pg4p+0Gt+E07Q0BB4WpN3DvCXxQV mpkXhdHRXG5Ex58Ct7VXhQtl0HOotoKTd1Z6XsCmitW8Dr+173CHcnx9zdUCSm+wd0wi 1LL5MSe23lhDqED56L0DT3JOH8uoFjQVPtgb+2NuuksudAze95v2Bj2m1DyFueaNilka n6SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788862925; x=1789467725; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ieAyrn9dOUbi/uQHpx4XPNGdMoW3ipgyyKgN9345CbI=; b=I81Z7cma6TEQ0vsk/b2Ew2IJb6iKeUBghxGsVXEjdGzfnU9OzMmwH02yihcPcgPd7/ GYlPoEn3J1IRytguclDkvU1jo4UjWI9S712QK2o9Srp8ABuW7Z4k65UPCko/7L7Pu4Lb +8YL7KZssBVBAIFMpiTcYJ71nmtU0O3RSIXPFm0rO8Wt6YVgRnUf0D7xh4JSQwSHNbfP ZMi8JSBxRrDBv/0vcNCVUUAoCVqSHS6mKRxAbr7fFONzAyZxPjsbS/pz7jQ3hGakgvNW 4Xkfbv/GZ9lgoxrN+zkK2qSmMdO3uodUozEM3I+BVNRvLaNSKsEjjvQxuEW+N0g9CN4W lrWg== X-Gm-Message-State: AFuF++kRpVJfGEarSniv6VbDlLGIU9ZTydStqwNM0s0rswGuuP1nUpKo 9pFpQLBojePUjnnxa8uEIaAkP42Kh8i2nORDDTCmyWlDtxm3puzCJfA/sKNmPA== X-Gm-Gg: AYBFou0pRfVQ2HT2QxKvfk9GN2kFBB9O+tFr1+FRyETKGn56U/eZpp+nrCSHbP8RVwB n7TG8snuLYjOBwV1uw6eAOANIwwBSs9iqLGVfAudbAeVJbC+yr73JsJtqNpRQzcdxaD1z+NRv7f VXZt5WNvxMn00/CV3awlwPB5gdrdLI/2vri4ipBtnrN7GFkAX4U0C4NNYG+dGYip7SDs2lJO801 2ByZYo3Vq67Gd4lgcYv2OK6DfkEaWZT8P0HnNKLHpZwsGgQmTbsDhHGnlZD2CR1n6wCtnEgxmvb ZASIL4RASn38c1YnvVfQ6R8pF7MqvEt4+psSmVf294C/f4D4Sl/FYzaut7Yfh+08tP5i5s2LFiv GpDNAZiW/4rEegZGJHkyguv8IgvITHZNy1QmzK23cLag3vdnmjaBxI/8lWfn1K/tgGjs47npRby cV3FkW3bvktKwSDMVKE75U3OixPbuULeTR204c/ruzHOaLrL5wEFbANab/baG4TEJCt2H+pdMWk 66OrBNpQ1jl9qqnwJJ8exuakAmKFbqmmGt1yGjZuMKq4lF5rBEBYA2Uk+ai6h6Bu6lmohqna45N X-Received: by 2002:a17:90b:5343:b0:398:e46e:ade3 with SMTP id 98e67ed59e1d1-39b2628fc1amr44985992a91.23.1788862924327; Tue, 08 Sep 2026 03:22:04 -0700 (PDT) Received: from AnisaLaptop.localdomain (c-73-170-217-179.hsd1.ca.comcast.net. [73.170.217.179]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-3396e7e477csm3435658eec.29.2026.09.08.03.22.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Sep 2026 03:22:02 -0700 (PDT) From: Anisa Su X-Google-Original-From: Anisa Su To: linux-cxl@vger.kernel.org Cc: alison.schofield@intel.com, jic23@kernel.org, dave.jiang@intel.com, gourry@gourry.net, icheng@nvidia.com, ming.li@zohomail.com, vishal.l.verma@intel.com, dave@stgolabs.net, benjamin.cheatham@amd.com, Anisa Su Subject: [RESEND PATCH v13 0/8] DCD Prep Series Date: Tue, 8 Sep 2026 03:15:04 -0700 Message-ID: <20260908102124.2231730-2-anisa.su@samsung.com> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit First send only sent cover letter :( Resend with patches. These are the first eight patches of the Dynamic Capacity Device (DCD) series. They set up: DCD enumeration, CDAT handling for DC partitions, the event interrupt plumbing, a preparatory refactor, and documentation of the DPA partition layout. They stand alone and keep DCD disabled. They are not dependent on the rest of the series. Patch 1 detects the 48h command set and then leaves mds->dcd_supported false. The flag is only turned on by the series that adds DCD event handling, so every DCD specific path added here is shut off at runtime: cxl_configure_dcd() never runs, no DC partition is created, no DCD interrupt is armed, and the CXL_PARTMODE_DYNAMIC_RAM_1 paths are unreachable. The intent is to land the enumeration and plumbing while the extent to DAX pipeline is still under review. Based on v7.3-rc2 GitHub Branch: https://github.com/anisa-su993/anisa-linux-kernel/tree/dcd-prep-v13 v12: https://lore.kernel.org/linux-cxl/20260731084901.1512819-1-anisa.su@samsung.com/ v12 -> v13 Changes: =================== - Dropped: cxl/core: Return endpoint decoder information from region search (v12 patch 7). The new interface is not paired with its first caller in this series -- all three call sites pass NULL -- so its contract cannot be reviewed here. It will be posted with its first user instead (Alison). v12 patch 8 is now patch 7. - Added: Documentation/cxl: Document DPA partition layout and ordering rules (patch 8) in-tree in addition to the commit messages in order to be explicit about places Linux is stricter than the specification. Add a section to cxl-driver.rst separating what the spec mandates for ram and pmem from what Linux chooses for dynamic capacity. It is documentation only. The series is eight patches, one dropped and one added. - Patch 1: cxl/mbox: Flag support for Dynamic Capacity Devices (DCD) - Commit message explains why mds->dcd_supported is left false and what turns it back on (Dave). - Drop the single use cxl_verify_dcd_cmds() helper and test bitmap_full(dcd_cmds, CXL_DCD_ENABLED_MAX) directly (Alison). - Patch 2: cxl/mem: Read dynamic capacity configuration from the device - Validate only DC partition 0. Every partition the device reports is still read, so the framework stays the same if we decide to add support for additional partitions. But no validation is done for the other partitions, so a defect in other partitions that we don't use no longer prevents the driver from configuring dynamic capacity (Richard / Alison). - Rework the cxl_dc_check() validation: checks now run in dependency order, block size first, and each failure names the field that is wrong instead of four conditions sharing one "invalid length" message (Alison). - Drop the block size alignment requirement on the region base. Table 8-347 requires 256M for the base and states the block size alignment for extent start DPAs, not for the base, so Linux was enforcing a constraint the spec does not state (Alison). - Range check the device's available partition count against the 1 to 8 of Table 8-346, before it is used as the read loop bound (Alison). - Split cxl_configure_dcd()'s base check: an error for a DC partition overlapping static capacity, a warning for a gap above it, both printing the addresses. The old message said "pmem" when the comparison is against total static capacity (Alison). - cxl_configure_dcd() returns an error instead of void; cxl_pci_probe() disables DCD and carries on rather than failing the memdev (Jonathan). - Check the block size is a power of 2 in 64 bits. is_power_of_2() takes an unsigned long, so a 32 bit build truncated blk_size and accepted values such as 0x100000040 (Sashiko). - Patch 3: cxl/cdat: Gather DSMAS data for DCD partitions - Only assign shareable for the dynamic ram 1 partition. It was set for every partition the DSMAS range fell into, while the handle check beside it is DC only (Alison). - Reduce nesting in cxl_memdev_set_qos_class(): invert range_contains() to continue early, and use a 'part' local instead of repeating cxlds->part[i] (Alison). Doing this first is what made the reworded warning fit in 100 columns. - Patch 4: cxl/events: Split event msgnum configuration from irq setup - No change from v12. - Patch 5: cxl/pci: Factor out interrupt policy check - Rename cxl_event_validate_mem_policy() to cxl_event_fw_owns_mem_logs() and invert it. It never validated the policy, it reported whether firmware still owns the standard memory event logs despite _OSC granting the OS control, so the name now states that condition and true means firmware owns them (Alison). - Commit message: "validation" was the wrong word for what the helper does; it checks for firmware ownership of the memory event logs (Alison). - Patch 6: cxl/mem: Configure dynamic capacity interrupts - Disable DCD when the device supports the DCD commands but its interrupt policy is too short to carry dcd_settings. The Set is sized by the device's own policy length, so on a short policy dcd_settings is never sent, and the re-read afterwards leaves the driver's own value in the struct because the device does not return that byte. cxl_irqsetup() would then arm the DCD interrupt from a setting the device never accepted. A conforming device cannot be in that state -- DCD is CXL 3.x and a 3.x policy carries the field -- but nothing tied the two facts together (Alison). - Move the DCD bit out of cxl_event_drain_mask(), dropping the mds argument it no longer needs, so the bit lands in the patch that adds the code draining that log. This patch still defines CXLDEV_EVENT_STATUS_DCD, adds it to CXLDEV_EVENT_STATUS_ALL and arms the DCD interrupt; only the drain bit moved (Alison). - Patch 7: cxl/core: Enforce partition order/simplify partition calls - Document the DPA partition layout above enum cxl_partition_mode in cxl.h with ascii art, and cite where the order comes from: ram before pmem is mandated by CXL r4.0 Table 8-310, while Table 8-347 only requires the DC Region Base to be 256MB aligned, so a dynamic partition following pmem is a Linux choice that cxl_configure_dcd() enforces (Alison). - Commit message: say what the partition order is, where it comes from, and that this patch starts enforcing it. The previous text named an "implied partition order" without ever saying what it implied (Alison). - Name the partitions in the failure: "Partition order mismatch: %d (%s) follows %d (%s)" via cxl_mode_name(), rather than the bare "Partition order mismatch" (Alison). Review status ============= Patch 1 + Reviewed-by from Dave Jiang and Gregory Price from v12. Patch 1 did change in v13: dropped the single use cxl_verify_dcd_cmds() helper so cxl_walk_cel() now tests bitmap_full() directly. That is a no-op cleanup -- the helper was a one-line wrapper around the same bitmap_full() call and had no other caller -- so carried both tags over rather than dropping them. Dave, Gregory, please lmk if you would rather re-review. Patch 2 had the most changes from v12, is the patch that most needs eyes. Patch 3: Minor changes. Dropped Dave's tag. Patch 4 is unchanged from v12 and carries its existing tag. Patches 5 and 7 did change, but quite trivially, so kept the tags. Patch 5 renames cxl_event_validate_mem_policy() to cxl_event_fw_owns_mem_logs(). Patch 7 picks up a loop-local index, a reworded error message and a comment on enum cxl_partition_mode. Patch 8 (partition layout Documentation) is new in v13: no tags. Patch organization ================== 1-3: enumerate DCD command support, read the DC configuration, and pick up DSMAS data for DC partitions. 4-6: split event msgnum configuration from irq setup, factor out the interrupt policy check, then configure the DCD event interrupt. 7: preparatory refactor -- enforce partition order while collapsing the ram/pmem specific partition helpers into one. 8: document the DPA partition layout the previous patches enforce, and where Linux is stricter than the specification. Testing ======= This series leaves mds->dcd_supported false on purpose, so cxl_configure_dcd(), cxl_dev_dc_identify() and cxl_dc_check() are unreachable at runtime as posted. The tests below are done with the rest of the series on top, otherwise testing would not yield any meaningful results. HW: Tested add/release of a single extent to ensure interrupts are configured and working and DSMAS attributes are read correctly. Additional testing is done on QEMU to catch cases that would not occur on the hardware because of configuration differences. Four topologies are used, one guest boot each, with a DCD partition after static capacity. 1. Eight DC partitions, only the first validated ------------------------------------------------ 2G ram + 512M pmem static, so static capacity ends at 0xa0000000, and eight 512M DC partitions above it. Five partition records fit in one reply on the 256 byte mailbox, so the read continues from partition 5 in a second command. cxl_pci 0000:0d:00.0: Read 5/8 DC partitions cxl_pci 0000:0d:00.0: DC partition 0 start 0xa0000000 size 0x20000000 blk_size: 0x200000 cxl_pci 0000:0d:00.0: Read 3/8 DC partitions cxl_pci 0000:0d:00.0: Adding dynamic ram partition 1; 0xa0000000 size 0x20000000 cxl_pci 0000:0d:00.0: DPA(ram): [mem 0x00000000-0x7fffffff flags 0x200] cxl_pci 0000:0d:00.0: DPA(pmem): [mem 0x80000000-0x9fffffff flags 0x200] cxl_pci 0000:0d:00.0: DPA(dynamic_ram_1): [mem 0xa0000000-0xbfffffff flags 0x200] All eight partitions are read, but only one "DC partition" line appears: partition 0 is the only one validated. 2. Volatile-only static capacity --------------------------------- 2G ram, no pmem, one 512M DC partition. cxl_configure_dcd() compares the DC base against the end of total static capacity, not the end of pmem; a device with no persistent capacity is what tells those two apart. cxl_pci 0000:0d:00.0: Read 1/1 DC partitions cxl_pci 0000:0d:00.0: DC partition 0 start 0x80000000 size 0x20000000 blk_size: 0x200000 cxl_pci 0000:0d:00.0: Adding dynamic ram partition 1; 0x80000000 size 0x20000000 cxl_pci 0000:0d:00.0: DPA(ram): [mem 0x00000000-0x7fffffff flags 0x200] cxl_pci 0000:0d:00.0: DPA(dynamic_ram_1): [mem 0x80000000-0x9fffffff flags 0x200] The DC partition lands at 0x80000000, directly after ram, and no static-capacity complaint is logged. 3. No static capacity at all ----------------------------- No ram, no pmem, one 512M DC partition based at DPA 0. This is the boundary of the contiguity rule: with no static capacity the end of static capacity is DPA 0, so a DC base of 0 meets it exactly and neither the overlap error nor the gap warning may fire. cxl_pci 0000:0d:00.0: Read 1/1 DC partitions cxl_pci 0000:0d:00.0: DC partition 0 start 0x0 size 0x20000000 blk_size: 0x200000 cxl_pci 0000:0d:00.0: Adding dynamic ram partition 1; 0x0 size 0x20000000 cxl_pci 0000:0d:00.0: DPA(dynamic_ram_1): [mem 0x00000000-0x1fffffff flags 0x200] 4. Per-field validation ----------------------- Each row is one malformed field on partition 0, everything else left valid. The message names the field that is wrong. block size 0x100000040 DC partition 0 invalid block size 0x100000040 block size 0x30 DC partition 0 invalid block size 0x30 decode length 0 DC partition 0 zero decode length base 0xa0001000 DC partition 0 base 0xa0001000 not aligned to 256M DSMAD handle 0x100 DSMAD handle 0x100 exceeds the 8 bit CDAT DSMAD handle 0 partitions available Device reported 0 available DC partitions, expected 1 to 8 9 partitions available Device reported 9 available DC partitions, expected 1 to 8 The first row is the 32 bit truncation Sashiko reported: 0x100000040 is not a power of 2, but its low 32 bits are, so is_power_of_2() accepted it on a 32 bit build. All seven cases disable DCD and leave the memdev attached with its static capacity. DC partition 0 placed against static capacity, which cxl_configure_dcd() checks separately, gives one message for the overlap and one for the gap: cxl_pci 0000:0d:00.0: DC partition 0 base 0x90000000 overlaps static capacity ending at 0xa0000000 cxl_pci 0000:0d:00.0: DC partition 0 base 0xb0000000 leaves a gap from static capacity ending at 0xa0000000 Also used ndctl-test-runner to run cxl_test for regressions. Anisa Su (1): Documentation/cxl: Document DPA partition layout and ordering rules Ira Weiny (7): cxl/mbox: Flag support for Dynamic Capacity Devices (DCD) cxl/mem: Read dynamic capacity configuration from the device cxl/cdat: Gather DSMAS data for DCD partitions cxl/events: Split event msgnum configuration from irq setup cxl/pci: Factor out interrupt policy check cxl/mem: Configure dynamic capacity interrupts cxl/core: Enforce partition order/simplify partition calls .../driver-api/cxl/linux/cxl-driver.rst | 38 +++ drivers/cxl/core/cdat.c | 32 +- drivers/cxl/core/hdm.c | 13 + drivers/cxl/core/mbox.c | 283 ++++++++++++++++++ drivers/cxl/core/memdev.c | 84 ++++-- drivers/cxl/cxl.h | 4 +- drivers/cxl/cxlmem.h | 88 +++++- drivers/cxl/mem.c | 2 +- drivers/cxl/pci.c | 159 ++++++++-- include/cxl/cxl.h | 27 +- 10 files changed, 653 insertions(+), 77 deletions(-) -- 2.43.0