RE: CXL volatile memory: How to restore the previous region/Interleave set

[Dan Williams <dan.j.williams@intel.com>]

Yasunori Gotou (Fujitsu) wrote:
[..]
> > What is currently missing on the Linux OS side is a default policy for unmapped
> > volatile capacity after all initial device probing has completed.
> > 
> > > One scenario I understand is that in clusters using an Orchestrator,
> > > such as K8S, when a node (worker) restarts, K8S is able to read the
> > > Interleave Set from the database and sets it for the corresponding node.
> > 
> > Yes, for sophisticated environments a configuration database could store and
> > replay region configurations each boot.
> 
> Just an idea, I suppose that cxl command should have two features.
>  - save current regions configuration to a file which is specified by its operand.
>  - reconfigure regions depends on the specified file.
>   (If hardware condition is changed, the command return error and display what is changed.)
> 
> Probably, it may be enough for the most of users.... But what do you think it?

It is not clear that the tool needs a save / restore option vs just
replaying the same create-region command from one boot to the next.

You can imagine having a startup script that issues one of the
following:

	1/ cxl create-region
	2/ cxl create-region -d decoder0.4
	3/ cxl create-region -m mem0 mem1
	4/ cxl create-region -S 0x12345 0xabcde
	4/ cxl create-region -S 0x12345 0xabcde -s 1T

1/ Find the CXL window with the largest available capacity and create a
maximally sized region.

2/ Limit the search to a specific CXL window, but create the largest
available region from any spare capacity found there.

3/ Try to create the largest possible region with mem0 and mem1. NOTE
that this could produce wildly different results from boot to boot
because CXL device scanning is asynchronous and mem0 from one boot may
not match mem0 in the current boot even if the hardware configuration
has not changed.

4/ Same as 3/ but guaranteed to get the exact same devices because they
are addressed by serial number.

5/ Same as 4/ but limit the size.

So, a lot can be done without needing to save restore the exact
configuration, which for volatile the exact configuration does not
matter as much as the the capacity and performance characteristics.

NOTE, some of the above examples are not implemented yet, patches
welcome! For example the -S option to treat the arguments as
memory-device serial numbers rather memory-device id numbers is not
there today, "cxl create-region" by itself does not know how to search
for capacity, and "cxl create-region -m" without specifying the
decoder/window throws an error even though it is relatively
straightforward to figure out the set of CXL windows that map a given
memdev.