Re: [PATCH 0/4] Support dynamic (de)configuration of memory

Sumanth Korikkar
2025-10-07T17:56:22Z

> > With the new interface, s390 will not add all possible hotplug memory in
> > advance, like before, to make it visible in sysfs for online/offline
> > actions. Instead, before memory block can be set online, it has to be
> > configured via a new interface in /sys/firmware/memory/memoryX/config,
> > which makes s390 similar to others. i.e. Adding of hotpluggable memory is
> > controlled by the user instead of adding it at boottime.
>
> Before I dig into the details, will onlining/offling still trigger
> hypervisor action, or does that now really happen when memory is
> added/removed?
>
> That would be really nice, because it would remove the whole need for
> "standby" memory, and having to treat hotplugged memory differently under
> LPAR/z/VM than anywhere else (-> keep it offline).

With this approach, hypervisor actions are triggered only when memory is
actually added or removed.

Online and offline operations are common code memory hotplug actions and
the s390 memory notifier actions are none/minimal.

> > s390 kernel sysfs interface to configure/deconfigure memory with
> > memmap_on_memory (with upcoming lsmem changes):
> > * Initial memory layout:
> > lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP_ON_MEMORY
> > RANGE SIZE STATE BLOCK CONFIGURED MEMMAP_ON_MEMORY
> > 0x00000000-0x7fffffff 2G online 0-15 yes no
> > 0x80000000-0xffffffff 2G offline 16-31 no yes
>
> Could we instead modify "STATE" to reflect that it is "not added" / "not
> configured" / "disabled" etc?
>
> Like
>
> lsmem -o RANGE,SIZE,STATE,BLOCK,MEMMAP_ON_MEMORY
> RANGE SIZE STATE BLOCK
> 0x00000000-0x7fffffff 2G online 0-15
> 0x80000000-0xffffffff 2G disabled 16-31
>
> Or is that an attempt to maintain backwards compatibility?

Mostly. Also, similar to lscpu output, where CPU status shows
CONFIGURED/STATE column.

Also, older scripts to get list of offline memory typically use:
lsmem | grep offline

and

chmem -e <SIZE> would work as usual, where <SIZE> specifies amount of
memory to set online.

chmem changes would look like:
chmem -c 128M -m 1 : configure memory with memmap-on-memory enabled
chmem -g 128M : deconfigure memory
chmem -e 128M : optionally configure (if supported by architecture) and
always online memory
chmem -d 128M : offline and optionally deconfigure memory (if supported
by architecture)

> > * Configure memory
> > echo 1 > /sys/firmware/memory/memory16/config
>
> The granularity here is also memory_block_size_bytes(), correct?

Yes, correct.

> > lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP_ON_MEMORY
> > RANGE SIZE STATE BLOCK CONFIGURED MEMMAP_ON_MEMORY
> > 0x00000000-0x7fffffff 2G online 0-15 yes no
> > 0x80000000-0x87ffffff 128M offline 16 yes yes
> > 0x88000000-0xffffffff 1.9G offline 17-31 no yes
> >
> > * Deconfigure memory
> > echo 0 > /sys/firmware/memory/memory16/config
> > lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP_ON_MEMORY
> > RANGE SIZE STATE BLOCK CONFIGURED MEMMAP_ON_MEMORY
> > 0x00000000-0x7fffffff 2G online 0-15 yes no
> > 0x80000000-0xffffffff 2G offline 16-31 no yes
> >
> > * Enable memmap_on_memory and online it.
> > (Deconfigure first)
> > echo 0 > /sys/devices/system/memory/memory5/online
> > echo 0 > /sys/firmware/memory/memory5/config
> >
> > lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP_ON_MEMORY
> > RANGE SIZE STATE BLOCK CONFIGURED MEMMAP_ON_MEMORY
> > 0x00000000-0x27ffffff 640M online 0-4 yes no
> > 0x28000000-0x2fffffff 128M offline 5 no no
> > 0x30000000-0x7fffffff 1.3G online 6-15 yes no
> > 0x80000000-0xffffffff 2G offline 16-31 no yes
> >
> > (Enable memmap_on_memory and online it)
> > echo 1 > /sys/firmware/memory/memory5/memmap_on_memory
> > echo 1 > /sys/firmware/memory/memory5/config
> > echo 1 > /sys/devices/system/memory/memory5/online
>
> I guess the use for memmap_on_memory would now be limited to making hotplug
> more likely to succeed in OOM scenarios.

Yes. with memmap-on-memory enabled, mainly in OOM situations.

However, it also provides flexibility to the user to configure few
memory blocks with memmap-on-memory enabled and few with
memmap-on-memory disabled (When the user needs continuous physical
memory across memory blocks).

> > Patch 4 removes the MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE notifiers. It
> > is no longer needed. Memory can be brought to accessible state before
> > adding memory now, with runtime (de)configuration of memory.
>
> Nice.

Thank you David