RE: Should bios always mark CXL DRAM as EFI_MEMORY_SP?

["Li Qiang \(Johnny Li\)" <johnny.li@montage-tech.com>]

I think BIOS should follow CDAT spec v1.01 Device Scoped EFI Memory Type Structure (DSEMTS) structure

In Table 8 Device Scoped EFI Memory Type Structure, field EFI Memory Type and Attribute has below definition
  0 – EfiConventionalMemory
  1 - EfiConventionalMemory Type with EFI_MEMORY_SP Attribute
  2 – EfiReservedMemoryType
  3-255 – Reserved encoding
  The memory attribute EFI_MEMORY_NV may be inferred from NonVolatile flag in DSMAS.
  Memory types other than EfiConventionalMemory and EfiReservedMemoryType are not permitted.

Thanks
Johnny
-----Original Message-----
From: John Groves (john@jagalactic.com) [mailto:john@jagalactic.com] 
Sent: Friday, January 28, 2022 12:19 AM
To: Dan Williams; linux-cxl@vger.kernel.org
Cc: Jonathan Cameron; Ben Widawsky; John Groves
Subject: Should bios always mark CXL DRAM as EFI_MEMORY_SP?

I’d like to seek some feedback and see whether a consensus exists or can be developed regarding how system firmware (bios/efi/etc) should present CXL DRAM to a system in a pre-fabric world (CXL 1.1/2.0).

 

The CXL spec, along with the Intel documentation are pretty specific and useful, but one open issue seems not to be outright specified: should the system mark CXL-attached DRAM as “specific purpose” (EFI_MEMORY_SP)? Consistency across platforms is certainly desirable. If this behavior is not prescribed, we could end up with inconsistent behavior across server and bios vendors.

 

If this is already specified, no need to read on (but please point me to where it’s specified).

 

Objective: I think everyone will likely agree that it should be possible to use CXL DRAM as either general-purpose memory, or via DAX, or a mix.

 

What’s the difference?

 

Memory marked as EFI_MEMORY_SP:

 

·      Mappable via DAX

·      Can be online-converted to general purpose memory via daxctl

·      Can be boot-converted to general-purpose with efi=nosoftreserve on Linux command line

 

Memory NOT marked as EFI_MEMORY_SP:

 

·      CXL is general-purpose memory (NUMA node with no local CPU cores)

·      Some of the contents appear to be used for in-memory metadata (presumably buddy lists, etc.)

·      Can be boot-converted to DAX with an efi_fake_mem= argument on the Linux command line

·      Currently cannot be online-converted to DAX-managed (can this work? Is it intended to be working?)

 

If online conversion from general-purpose to DAX is not going to work, it seems that the default should preserve the ability to use it either way: mark the memory as EFI_MEMORY_SP.

 

Is there a right and wrong answer re:EFI_MEMORY_SP? How important is it to have consistency across platforms?

 

If there is a consensus, the next question is who should express it. Perhaps the CXL consortium. I’m a part of that, but it seemed like the Linux dev community was the right place to start.

 

Thanks for any thoughts.


John Groves

Micron