[PATCH v4 0/9] dax/hmem, cxl: Coordinate Soft Reserved handling with CXL and HMEM

Smita Koralahalli
2025-11-20T03:19:41Z

This series aims to address long-standing conflicts between HMEM and
CXL when handling Soft Reserved memory ranges.

Reworked from Dan's patch:
https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/patch/?id=ab70c6227ee6165a562c215d9dcb4a1c55620d5d

Previous work:
https://lore.kernel.org/all/20250715180407.47426-1-Smita.KoralahalliChannabasappa@amd.com/

Link to v3:
https://lore.kernel.org/all/20250930044757.214798-1-Smita.KoralahalliChannabasappa@amd.com

This series should be applied on top of:
"214291cbaace: acpi/hmat: Fix lockdep warning for hmem_register_resource()"
and is based on:
base-commit: 211ddde0823f1442e4ad052a2f30f050145ccada

I initially tried picking up the three probe ordering patches from v20/v21
of Type 2 support, but I hit a NULL pointer dereference in
devm_cxl_add_memdev() and cycle dependency with all patches so I left
them out for now. With my current series rebased on 6.18-rc2 plus
214291cbaace, probe ordering behaves correctly on AMD systems and I have
verified the scenarios mentioned below. I can pull those three patches
back in for a future revision once the failures are sorted out.

Probe order patches of interest:
cxl/mem: refactor memdev allocation
cxl/mem: Arrange for always-synchronous memdev attach
cxl/port: Arrange for always synchronous endpoint attach

[1] Hotplug looks okay. After offlining the memory I can tear down the
regions and recreate it back if CXL owns entire SR range as Soft Reserved
is gone. dax_cxl creates dax devices and onlines memory.
850000000-284fffffff : CXL Window 0
850000000-284fffffff : region0
850000000-284fffffff : dax0.0
850000000-284fffffff : System RAM (kmem)

[2] With CONFIG_CXL_REGION disabled, all the resources are handled by
HMEM. Soft Reserved range shows up in /proc/iomem, no regions come up
and dax devices are created from HMEM.
850000000-284fffffff : CXL Window 0
850000000-284fffffff : Soft Reserved
850000000-284fffffff : dax0.0
850000000-284fffffff : System RAM (kmem)

[3] Region assembly failures also behave okay and work same as [2].

Before:
2850000000-484fffffff : Soft Reserved
2850000000-484fffffff : CXL Window 1
2850000000-484fffffff : dax4.0
2850000000-484fffffff : System RAM (kmem)

After tearing down dax4.0 and creating it back:

Logs:
[ 547.847764] unregister_dax_mapping: mapping0: unregister_dax_mapping
[ 547.855000] trim_dev_dax_range: dax dax4.0: delete range[0]: 0x2850000000:0x484fffffff
[ 622.474580] alloc_dev_dax_range: dax dax4.1: alloc range[0]: 0x0000002850000000:0x000000484fffffff
[ 752.766194] Fallback order for Node 0: 0 1
[ 752.766199] Fallback order for Node 1: 1 0
[ 752.766200] Built 2 zonelists, mobility grouping on. Total pages: 8096220
[ 752.783234] Policy zone: Normal
[ 752.808604] Demotion targets for Node 0: preferred: 1, fallback: 1
[ 752.815509] Demotion targets for Node 1: null

After:
2850000000-484fffffff : Soft Reserved
2850000000-484fffffff : CXL Window 1
2850000000-484fffffff : dax4.1
2850000000-484fffffff : System RAM (kmem)

[4] A small hack to tear down the fully assembled and probed region
(i.e region in committed state) for range 850000000-284fffffff.
This is to test the region teardown path for regions which don't
fully cover the Soft Reserved range.

850000000-284fffffff : Soft Reserved
850000000-284fffffff : CXL Window 0
850000000-284fffffff : dax5.0
850000000-284fffffff : System RAM (kmem)
2850000000-484fffffff : CXL Window 1
2850000000-484fffffff : region1
2850000000-484fffffff : dax1.0
2850000000-484fffffff : System RAM (kmem)
.4850000000-684fffffff : CXL Window 2
4850000000-684fffffff : region2
4850000000-684fffffff : dax2.0
4850000000-684fffffff : System RAM (kmem)

daxctl list -R -u
[
{
"path":"\/platform\/ACPI0017:00\/root0\/decoder0.1\/region1\/dax_region1",
"id":1,
"size":"128.00 GiB (137.44 GB)",
"align":2097152
},
{
"path":"\/platform\/hmem.5",
"id":5,
"size":"128.00 GiB (137.44 GB)",
"align":2097152
},
{
"path":"\/platform\/ACPI0017:00\/root0\/decoder0.2\/region2\/dax_region2",
"id":2,
"size":"128.00 GiB (137.44 GB)",
"align":2097152
}
]

I couldn't test multiple regions under same Soft Reserved range
with/without contiguous mapping due to limiting BIOS support. Hopefully
that works.

v4 updates:
- No changes patches 1-3.
- New patches 4-7.
- handle_deferred_cxl() has been enhanced to handle case where CXL
regions do not contiguously and fully cover Soft Reserved ranges.
- Support added to defer cxl_dax registration.
- Support added to teardown cxl regions.

v3 updates:
- Fixed two "From".

v2 updates:
- Removed conditional check on CONFIG_EFI_SOFT_RESERVE as dax_hmem
depends on CONFIG_EFI_SOFT_RESERVE. (Zhijian)
- Added TODO note. (Zhijian)
- Included region_intersects_soft_reserve() inside CONFIG_EFI_SOFT_RESERVE
conditional check. (Zhijian)
- insert_resource_late() -> insert_resource_expand_to_fit() and
__insert_resource_expand_to_fit() replacement. (Boris)
- Fixed Co-developed and Signed-off by. (Dan)
- Combined 2/6 and 3/6 into a single patch. (Zhijian).
- Skip local variable in remove_soft_reserved. (Jonathan)
- Drop kfree with __free(). (Jonathan)
- return 0 -> return dev_add_action_or_reset(host...) (Jonathan)
- Dropped 6/6.
- Reviewed-by tags (Dave, Jonathan)

Dan Williams (4):
dax/hmem, e820, resource: Defer Soft Reserved insertion until hmem is
ready
dax/hmem: Request cxl_acpi and cxl_pci before walking Soft Reserved
ranges
dax/hmem: Gate Soft Reserved deferral on DEV_DAX_CXL
dax/hmem: Defer handling of Soft Reserved ranges that overlap CXL
windows

Smita Koralahalli (5):
cxl/region, dax/hmem: Arbitrate Soft Reserved ownership with
cxl_regions_fully_map()
cxl/region: Add register_dax flag to control probe-time devdax setup
cxl/region, dax/hmem: Register devdax only when CXL owns Soft Reserved
span
cxl/region, dax/hmem: Tear down CXL regions when HMEM reclaims Soft
Reserved
dax/hmem: Reintroduce Soft Reserved ranges back into the iomem tree

arch/x86/kernel/e820.c | 2 +-
drivers/cxl/acpi.c | 2 +-
drivers/cxl/core/region.c | 181 ++++++++++++++++++++++++++++++++++++--
drivers/cxl/cxl.h | 17 ++++
drivers/dax/Kconfig | 2 +
drivers/dax/hmem/device.c | 4 +-
drivers/dax/hmem/hmem.c | 137 ++++++++++++++++++++++++++---
include/linux/ioport.h | 13 ++-
kernel/resource.c | 92 ++++++++++++++++---
9 files changed, 415 insertions(+), 35 deletions(-)

base-commit: 211ddde0823f1442e4ad052a2f30f050145ccada
--
2.17.1