[PATCH v2 0/6] chmem/lsmem: dynamic (de)configuration of memory

[PATCH v2 0/6] chmem/lsmem: dynamic (de)configuration of memory

[Sumanth Korikkar]

Hi all,

Patchset extends chmem and lsmem with support for dynamically configuring and
deconfiguring hotpluggable memory blocks on s390, including per-block
memmap-on-memory handling.

On s390, the memmap-on-memory feature was
introduced to ensure that the struct page array (metadata) for hotpluggable
standby memory can be allocated from the memory block itself. This allowed
hot-add operations even under memory pressure, especially in systems with an
imbalance between boot-time online memory and standby memory.

The original implementation had few limitations:
* All hotpluggable standby memory was added at boot, making blocks
  visible for online/offline operations earlier.
* The use of memmap-on-memory was global and static, decided at boot
  time. Either all standby blocks used it, or none of them did.
* memmap-on-memory choice could not be changed at runtime, limiting
  flexibility.

The s390 kernel (linux-next) ff18dcb19aab ("s390/sclp: Add support for
dynamic (de)configuration of memory") no longer pre-adds all standby
memory at boot. Instead, users must explicitly configure a block before
it can be used for online/offline actions.  At configuration time, users
can dynamically decide whether to use optional memmap-on-memory for each
memory block, where value of 1 allocates metadata (such as struct pages
array) from the hotplug memory itself, enabling hot-add operations even
under memory pressure. A value of 0 stores metadata in regular system
memory and enables continuous physical memory across memory blocks.

Kernel changes for dynamic (de)configuration of memory (available on
linux-next):
https://lore.kernel.org/all/20251010085147.2175918-1-sumanthk@linux.ibm.com/
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/log/drivers/s390/char/sclp_mem.c

The kernel sysfs interface provides configuration and memmap-on-memory toggling:
echo 1 > /sys/firmware/memory/memoryX/config – configure block
echo 0 > /sys/firmware/memory/memoryX/config – deconfigure block
echo 1 > /sys/firmware/memory/memoryX/memmap_on_memory – enable memmap-on-memory
echo 0 > /sys/firmware/memory/memoryX/memmap_on_memory – disable memmap-on-memory

Patchset teaches chmem and lsmem to make use of these interfaces, mirroring
existing online/offline semantics:

chmem -c 128M -m 1 : configure memory with memmap-on-memory
chmem -g 128M : deconfigure memory
chmem -e 128M : configure (if supported by architecture) and online memory
chmem -d 128M : offline and deconfigure memory (if supported by
architecture)
lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP-ON-MEMORY

memmap-on-memory can only be toggled when a block is in a deconfigured state,
and is supported via the --configure option.

v2 (Thanks Karel):
* lsmem:
  * Use _PATH_SYS_MEMMAP_PARM instead of const char *path.
  * Use N_()/_() in get_memmap_mode()/printf().
  * Improve skip_memconfig_column().
  * Check /sys/firmware/memory/memory0 existance before read_basic_info(). 
    The latter can run after column setup. Doing so reduces
    ul_path_access(lsmem->sysmemconfig, F_OK, 'memory0') calls to just one.
* chmem:
  * Rename chmem_memmap_on_memory_option_enabled() to chmem_memmap_enabled().
  * Improve configure check in warn().
  * Optimization in chmem_set_memmap_on_memory().
  * Remove "\n" in warn().
  * Add _() when printing info.

Sumanth Korikkar (6):
  lsmem: display global memmap on memory parameter
  lsmem: add support to display dynamic (de)configuration of memory
  chmem: add support for dynamic (de)configuration of hotplug memory
  chmem: add chmem documentation for dynamic (de)configuration of memory
  lsmem: add doc for dynamic (de)configuration and memmap-on-memory
    support
  lsmem,chmem: add configure/deconfigure bash completion options

 bash-completion/chmem  |   3 +
 bash-completion/lsmem  |   2 +-
 sys-utils/chmem.8.adoc |  47 +++++-
 sys-utils/chmem.c      | 371 +++++++++++++++++++++++++++++++++++++----
 sys-utils/lsmem.1.adoc |  46 ++++-
 sys-utils/lsmem.c      | 136 +++++++++++++--
 6 files changed, 555 insertions(+), 50 deletions(-)

-- 
2.41.0

[PATCH v2 1/6] lsmem: display global memmap on memory parameter

[Sumanth Korikkar]

Display the output of global memmap-on-memory parameter for memory
hotplug. Retrieve the details via
/sys/module/memory_hotplug/parameters/memmap_on_memory.

lsmem
RANGE                                 SIZE  STATE REMOVABLE BLOCK
0x0000000000000000-0x00000001ffffffff   8G online       yes  0-63

Memory block size:                128M
Total online memory:                8G
Total offline memory:               0B
Memmap on memory parameter:        yes

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
---
 sys-utils/lsmem.1.adoc |  2 ++
 sys-utils/lsmem.c      | 29 +++++++++++++++++++++++++++++
 2 files changed, 31 insertions(+)

diff --git a/sys-utils/lsmem.1.adoc b/sys-utils/lsmem.1.adoc
index d588051a8..9c9397631 100644
--- a/sys-utils/lsmem.1.adoc
+++ b/sys-utils/lsmem.1.adoc
@@ -28,6 +28,8 @@ Not all columns are supported on all systems. If an unsupported column is specif
 
 Use the *--help* option to see the columns description.
 
+Memmap on memory parameter output displays the globally enabled memmap-on-memory setting for memory_hotplug. This is typically set on the kernel command line via memory_hotplug.memmap_on_memory.
+
 == OPTIONS
 
 *-a*, *--all*::
diff --git a/sys-utils/lsmem.c b/sys-utils/lsmem.c
index 39967bfc9..6151e269f 100644
--- a/sys-utils/lsmem.c
+++ b/sys-utils/lsmem.c
@@ -32,6 +32,7 @@
 #include "optutils.h"
 
 #define _PATH_SYS_MEMORY		"/sys/devices/system/memory"
+#define _PATH_SYS_MEMMAP_PARM		"/sys/module/memory_hotplug/parameters/memmap_on_memory"
 
 #define MEMORY_STATE_ONLINE		0
 #define MEMORY_STATE_OFFLINE		1
@@ -306,8 +307,24 @@ static void fill_scols_table(struct lsmem *lsmem)
 		add_scols_line(lsmem, &lsmem->blocks[i]);
 }
 
+static int get_memmap_mode(char *res, char *src, int len)
+{
+	if (!strncmp(src, "Y", 1))
+		strncpy(res, N_("yes"), len);
+	else if (!strncmp(src, "N", 1))
+		strncpy(res, N_("no"), len);
+	else if (!strncmp(src, "force", 5))
+		strncpy(res, N_("force"), len);
+	else
+		return -1;
+	return 0;
+}
+
 static void print_summary(struct lsmem *lsmem)
 {
+	char buf[8], res[8];
+	FILE *memmap;
+
 	if (lsmem->bytes) {
 		printf("%-32s %15"PRId64"\n",_("Memory block size:"), lsmem->block_size);
 		printf("%-32s %15"PRId64"\n",_("Total online memory:"), lsmem->mem_online);
@@ -327,6 +344,18 @@ static void print_summary(struct lsmem *lsmem)
 			printf("%-32s %5s\n",_("Total offline memory:"), p);
 		free(p);
 	}
+	memmap = fopen(_PATH_SYS_MEMMAP_PARM, "r");
+	if (!memmap)
+		return;
+	if (fgets(buf, sizeof(buf), memmap)) {
+		if (!get_memmap_mode(res, buf, sizeof(res))) {
+			if (lsmem->bytes)
+				printf("%-32s %15s\n", _("Memmap on memory parameter:"), _(res));
+			else
+				printf("%-32s %5s\n", _("Memmap on memory parameter:"), _(res));
+		}
+	}
+	fclose(memmap);
 }
 
 static int memory_block_get_node(struct lsmem *lsmem, char *name)
-- 
2.41.0

[PATCH v2 2/6] lsmem: add support to display dynamic (de)configuration of memory

[Sumanth Korikkar]

Extend lsmem to display (de)configured blocks and memmap_on_memory
state. With the new s390 kernel interface (linux-next) ff18dcb19aab
("s390/sclp: Add support for dynamic (de)configuration of memory"),
standby memory blocks are no longer pre-added at boot, but must be
explicitly configured before being eligible for online/offline
operations. At configuration time, users can also decide whether to use
memmap-on-memory per block.

Add CONFIGURED column : indicate if a memory block has been explicitly
configured.

Add MEMMAP-ON-MEMORY column : indicate if a memory block uses
memmap-on-memory.

memmap-on-memory reference:
https://docs.kernel.org/admin-guide/mm/memory-hotplug.html

Users can now inspect memory configuration state and retrieve
memmap-on-memory state per block.

lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP-ON-MEMORY
RANGE                   SIZE   STATE   BLOCK CONFIGURED MEMMAP-ON-MEMORY
0x00000000-0x7fffffff   2G     online  0-15  yes        no
0x80000000-0xffffffff   2G     offline 16-31 no         yes

Memory block size:                128M
Total online memory:                2G
Total offline memory:               2G
Memmap on memory parameter:        yes

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
---
 sys-utils/lsmem.c | 107 ++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 95 insertions(+), 12 deletions(-)

diff --git a/sys-utils/lsmem.c b/sys-utils/lsmem.c
index 6151e269f..648c0bc27 100644
--- a/sys-utils/lsmem.c
+++ b/sys-utils/lsmem.c
@@ -33,6 +33,7 @@
 
 #define _PATH_SYS_MEMORY		"/sys/devices/system/memory"
 #define _PATH_SYS_MEMMAP_PARM		"/sys/module/memory_hotplug/parameters/memmap_on_memory"
+#define _PATH_SYS_MEMCONFIG		"/sys/firmware/memory"
 
 #define MEMORY_STATE_ONLINE		0
 #define MEMORY_STATE_OFFLINE		1
@@ -59,12 +60,17 @@ struct memory_block {
 	int		nr_zones;
 	int		zones[MAX_NR_ZONES];
 	bool		removable;
+	bool		config;
+	bool		memmap_on_memory;
 };
 
 struct lsmem {
 	struct path_cxt		*sysmem;		/* _PATH_SYS_MEMORY directory handler */
+	struct path_cxt		*sysmemconfig;		/* _PATH_SYS_MEMCONFIG directory handler */
 	struct dirent		**dirs;
+	struct dirent		**memconfig_dirs;
 	int			ndirs;
+	int			memconfig_ndirs;
 	struct memory_block	*blocks;
 	int			nblocks;
 	uint64_t		block_size;
@@ -86,7 +92,10 @@ struct lsmem {
 				split_by_state : 1,
 				split_by_removable : 1,
 				split_by_zones : 1,
-				have_zones : 1;
+				split_by_memmap_on_memory : 1,
+				split_by_config : 1,
+				have_zones : 1,
+				have_memconfig : 1;
 };
 
 
@@ -98,6 +107,8 @@ enum {
 	COL_BLOCK,
 	COL_NODE,
 	COL_ZONES,
+	COL_CONFIG,
+	COL_MEMMAP,
 };
 
 static const char *const zone_names[] = {
@@ -128,6 +139,10 @@ static struct coldesc coldescs[] = {
 	[COL_BLOCK]	= { "BLOCK", 0, SCOLS_FL_RIGHT, N_("memory block number or blocks range")},
 	[COL_NODE]	= { "NODE", 0, SCOLS_FL_RIGHT, N_("numa node of memory")},
 	[COL_ZONES]	= { "ZONES", 0, SCOLS_FL_RIGHT, N_("valid zones for the memory range")},
+	[COL_CONFIG]	= { "CONFIGURED", 0, SCOLS_FL_RIGHT,
+			    N_("configuration status of the memory range")},
+	[COL_MEMMAP]	= { "MEMMAP-ON-MEMORY", 0, SCOLS_FL_RIGHT,
+			    N_("memmap-on-memory status of the memory range")},
 };
 
 /* columns[] array specifies all currently wanted output column. The columns
@@ -197,6 +212,8 @@ static inline void reset_split_policy(struct lsmem *l, int enable)
 	l->split_by_node = enable;
 	l->split_by_removable = enable;
 	l->split_by_zones = enable;
+	l->split_by_memmap_on_memory = enable;
+	l->split_by_config = enable;
 }
 
 static void set_split_policy(struct lsmem *l, int cols[], size_t ncols)
@@ -219,12 +236,23 @@ static void set_split_policy(struct lsmem *l, int cols[], size_t ncols)
 		case COL_ZONES:
 			l->split_by_zones = 1;
 			break;
+		case COL_MEMMAP:
+			l->split_by_memmap_on_memory = 1;
+			break;
+		case COL_CONFIG:
+			l->split_by_config = 1;
+			break;
 		default:
 			break;
 		}
 	}
 }
 
+static bool skip_memconfig_column(struct lsmem *lsmem, int id)
+{
+	return (id == COL_MEMMAP || id == COL_CONFIG) && !lsmem->have_memconfig;
+}
+
 static void add_scols_line(struct lsmem *lsmem, struct memory_block *blk)
 {
 	size_t i;
@@ -237,6 +265,8 @@ static void add_scols_line(struct lsmem *lsmem, struct memory_block *blk)
 	for (i = 0; i < ncolumns; i++) {
 		char *str = NULL;
 
+		if (skip_memconfig_column(lsmem, get_column_id(i)))
+			continue;
 		switch (get_column_id(i)) {
 		case COL_RANGE:
 		{
@@ -292,6 +322,14 @@ static void add_scols_line(struct lsmem *lsmem, struct memory_block *blk)
 				str = xstrdup(valid_zones);
 			}
 			break;
+		case COL_CONFIG:
+			if (lsmem->have_memconfig)
+				str = xstrdup(blk->config ? _("yes") : _("no"));
+			break;
+		case COL_MEMMAP:
+			if (lsmem->have_memconfig)
+				str = xstrdup(blk->memmap_on_memory ? _("yes") : _("no"));
+			break;
 		}
 
 		if (str && scols_line_refer_data(line, i, str) != 0)
@@ -400,6 +438,15 @@ static int memory_block_read_attrs(struct lsmem *lsmem, char *name,
 	if (errno)
 		rc = -errno;
 
+	if (lsmem->have_memconfig) {
+		if (ul_path_readf_s32(lsmem->sysmemconfig, &x, "%s/config", name) == 0)
+			blk->config = x == 1;
+		if (ul_path_readf_s32(lsmem->sysmemconfig, &x, "%s/memmap_on_memory", name) == 0)
+			blk->memmap_on_memory = x == 1;
+		blk->state = MEMORY_STATE_OFFLINE;
+		if (!blk->config)
+			return rc;
+	}
 	if (ul_path_readf_s32(lsmem->sysmem, &x, "%s/removable", name) == 0)
 		blk->removable = x == 1;
 
@@ -463,6 +510,10 @@ static int is_mergeable(struct lsmem *lsmem, struct memory_block *blk)
 				return 0;
 		}
 	}
+	if (lsmem->split_by_config && curr->config != blk->config)
+		return 0;
+	if (lsmem->split_by_memmap_on_memory && curr->memmap_on_memory != blk->memmap_on_memory)
+		return 0;
 	return 1;
 }
 
@@ -478,11 +529,33 @@ static void free_info(struct lsmem *lsmem)
 	free(lsmem->dirs);
 }
 
+static int memory_block_filter(const struct dirent *de)
+{
+	if (strncmp("memory", de->d_name, 6) != 0)
+		return 0;
+	return isdigit_string(de->d_name + 6);
+}
+
+static void read_memconfig(struct lsmem *lsmem)
+{
+	char dir[PATH_MAX];
+
+	if (!lsmem->have_memconfig) {
+		lsmem->memconfig_ndirs = 0;
+		return;
+	}
+	ul_path_get_abspath(lsmem->sysmemconfig, dir, sizeof(dir), NULL);
+	lsmem->memconfig_ndirs = scandir(dir, &lsmem->memconfig_dirs,
+					 memory_block_filter, versionsort);
+	if (lsmem->memconfig_ndirs <= 0)
+		err(EXIT_FAILURE, _("Failed to read %s"), dir);
+}
+
 static void read_info(struct lsmem *lsmem)
 {
 	struct memory_block blk;
-	char buf[128];
-	int i;
+	char buf[128], *name;
+	int i, num_dirs;
 
 	if (ul_path_read_buffer(lsmem->sysmem, buf, sizeof(buf), "block_size_bytes") <= 0)
 		err(EXIT_FAILURE, _("failed to read memory block size"));
@@ -492,8 +565,17 @@ static void read_info(struct lsmem *lsmem)
 	if (errno)
 		err(EXIT_FAILURE, _("failed to read memory block size"));
 
-	for (i = 0; i < lsmem->ndirs; i++) {
-		memory_block_read_attrs(lsmem, lsmem->dirs[i]->d_name, &blk);
+	read_memconfig(lsmem);
+	if (lsmem->have_memconfig)
+		num_dirs = lsmem->memconfig_ndirs;
+	else
+		num_dirs = lsmem->ndirs;
+	for (i = 0; i < num_dirs; i++) {
+		if (lsmem->have_memconfig)
+			name = lsmem->memconfig_dirs[i]->d_name;
+		else
+			name = lsmem->dirs[i]->d_name;
+		memory_block_read_attrs(lsmem, name, &blk);
 		if (blk.state == MEMORY_STATE_ONLINE)
 			lsmem->mem_online += lsmem->block_size;
 		else
@@ -508,13 +590,6 @@ static void read_info(struct lsmem *lsmem)
 	}
 }
 
-static int memory_block_filter(const struct dirent *de)
-{
-	if (strncmp("memory", de->d_name, 6) != 0)
-		return 0;
-	return isdigit_string(de->d_name + 6);
-}
-
 static void read_basic_info(struct lsmem *lsmem)
 {
 	char dir[PATH_MAX];
@@ -696,6 +771,12 @@ int main(int argc, char **argv)
 	lsmem->sysmem = ul_new_path(_PATH_SYS_MEMORY);
 	if (!lsmem->sysmem)
 		err(EXIT_FAILURE, _("failed to initialize %s handler"), _PATH_SYS_MEMORY);
+	lsmem->sysmemconfig = ul_new_path(_PATH_SYS_MEMCONFIG);
+	/* Always check for the existence of /sys/firmware/memory/memory0 first */
+	if (ul_path_access(lsmem->sysmemconfig, F_OK, "memory0") == 0)
+		lsmem->have_memconfig = 1;
+	if (!lsmem->sysmemconfig)
+		err(EXIT_FAILURE, _("failed to initialized %s handler"), _PATH_SYS_MEMCONFIG);
 	if (prefix && ul_path_set_prefix(lsmem->sysmem, prefix) != 0)
 		err(EXIT_FAILURE, _("invalid argument to --sysroot"));
 	if (!ul_path_is_accessible(lsmem->sysmem))
@@ -745,6 +826,8 @@ int main(int argc, char **argv)
 		struct coldesc *ci = get_column_desc(i);
 		struct libscols_column *cl;
 
+		if (skip_memconfig_column(lsmem, get_column_id(i)))
+			continue;
 		cl = scols_table_new_column(lsmem->table, ci->name, ci->whint, ci->flags);
 		if (!cl)
 			err(EXIT_FAILURE, _("failed to initialize output column"));
-- 
2.41.0

[PATCH v2 3/6] chmem: add support for dynamic (de)configuration of hotplug memory

[Sumanth Korikkar]

Extend chmem to use the new s390 kernel interface for configuring and
deconfiguring hotpluggable memory blocks, with memmap-on-memory support.

Background:
On s390, memmap-on-memory was introduced to ensure that the struct page
array (metadata) for hotpluggable standby memory is allocated from the
memory block itself. This allowed hot-add operations even under memory
pressure, particularly in cases with a strong imbalance between
boot-time online memory and standby memory.

The original design, however, had few limitations:
* All hotpluggable standby memory was added at boot.
* The use of memmap-on-memory was global and static, decided at boot
  time. Either all standby blocks used it, or none of them did.
* memmap-on-memory choice could not be changed at runtime, limiting
  flexibility. For example, when continuous physical memory was required
  later across memory blocks.

The s390 kernel ff18dcb19aab ("s390/sclp: Add support for dynamic
(de)configuration of memory") no longer pre-adds all standby memory at
boot. Instead, users must explicitly configure a block before it can be
used for online/offline actions.  At configuration time, users can
dynamically decide whether to use optional memmap-on-memory for each
memory block, where value of 1 allocates metadata (such as struct pages
array) from the hotplug memory itself, enabling hot-add operations even
under memory pressure. A value of 0 stores metadata in regular system
memory and enables continuous physical memory across memory blocks.

s390 kernel sysfs interface to configure/deconfigure memory with
memmap-on-memory support looks as shown below:

1. Configure memory
echo 1 > /sys/firmware/memory/memoryX/config  

2. Deconfigure memory
echo 0 > /sys/firmware/memory/memoryX/config

3. Enable memmap-on-memory
echo 1 > /sys/firmware/memory/memoryX/memmap_on_memory

4. Disable memmap-on-memory
echo 0 > /sys/firmware/memory/memoryX/memmap_on_memory

* Initial memory layout:
lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP-ON-MEMORY
RANGE                   SIZE   STATE   BLOCK CONFIGURED MEMMAP-ON-MEMORY
0x00000000-0x7fffffff   2G     online  0-15  yes        no
0x80000000-0xffffffff   2G     offline 16-31 no         yes

Memory block size:                128M
Total online memory:                2G
Total offline memory:               2G
Memmap on memory parameter:        yes

* Configure memory with memmap-on-memory.
chmem -c 128M -m 1
lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP-ON-MEMORY
RANGE                   SIZE   STATE    BLOCK CONFIGURED MEMMAP-ON-MEMORY
0x00000000-0x7fffffff   2G     online   0-15  yes        no
0x80000000-0x87ffffff   128M   offline  16    yes        yes
0x88000000-0xffffffff   1.9G   offline  17-31 no         yes

Memory block size:                128M
Total online memory:                2G
Total offline memory:               2G
Memmap on memory parameter:        yes

* Deconfigure memory
chmem -g 128M
lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP-ON-MEMORY
RANGE                   SIZE   STATE    BLOCK CONFIGURED MEMMAP-ON-MEMORY
0x00000000-0x7fffffff   2G     online   0-15  yes        no
0x80000000-0xffffffff   2G     offline  16-31 no         yes

Memory block size:                128M
Total online memory:                2G
Total offline memory:               2G
Memmap on memory parameter:        yes

* Online memory.
If the memory is in deconfigured state, configure and online it.

chmem -e 128M -v
Memory Block 16 (0x0000000080000000-0x0000000087ffffff) configured
Memory Block 16 (0x0000000080000000-0x0000000087ffffff) enabled

lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP-ON-MEMORY
RANGE                   SIZE   STATE    BLOCK CONFIGURED MEMMAP-ON-MEMORY
0x00000000-0x7fffffff    2G    online   0-15  yes        no
0x80000000-0x87ffffff    128M  online   16    yes        yes
0x88000000-0xffffffff    1.9G  offline  17-31 no         yes

Memory block size:                128M
Total online memory:              2.1G
Total offline memory:             1.9G
Memmap on memory parameter:        yes

* Offline memory
If the memory is in online state, then offline it and deconfigure it.

chmem -d 128M -v
Memory Block 16 (0x0000000080000000-0x0000000087ffffff) disabled
Memory Block 16 (0x0000000080000000-0x0000000087ffffff) deconfigured

lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP-ON-MEMORY
RANGE                   SIZE   STATE    BLOCK CONFIGURED MEMMAP-ON-MEMORY
0x00000000-0x7fffffff   2G     online   0-15  yes        no
0x80000000-0xffffffff   2G     offline  16-31 no         yes

Memory block size:                128M
Total online memory:                2G
Total offline memory:               2G
Memmap on memory parameter:        yes

Just like online and offline actions, memory configuration and
deconfiguration can be controlled through similar options. Also,
memmap-on-memory setting can be changed, only when the memory block is
in deconfigured state. This means, it is usable only via --configure
option.

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
---
 sys-utils/chmem.c | 371 ++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 342 insertions(+), 29 deletions(-)

diff --git a/sys-utils/chmem.c b/sys-utils/chmem.c
index bee2a90f8..51240bf5a 100644
--- a/sys-utils/chmem.c
+++ b/sys-utils/chmem.c
@@ -37,11 +37,16 @@
 #define CHMEM_EXIT_SOMEOK		64
 
 #define _PATH_SYS_MEMORY		"/sys/devices/system/memory"
+#define _PATH_SYS_MEMCONFIG		"/sys/firmware/memory"
 
 struct chmem_desc {
 	struct path_cxt	*sysmem;	/* _PATH_SYS_MEMORY handler */
+	struct path_cxt *sysmemconfig;	/* _PATH_SYS_MEMCONFIG directory handler */
 	struct dirent	**dirs;
+	struct dirent   **memconfig_dirs;
 	int		ndirs;
+	int		memconfig_ndirs;
+	int		memmap_on_memory;
 	uint64_t	block_size;
 	uint64_t	start;
 	uint64_t	end;
@@ -50,11 +55,14 @@ struct chmem_desc {
 	unsigned int	is_size	   : 1;
 	unsigned int	verbose	   : 1;
 	unsigned int	have_zones : 1;
+	unsigned int	have_memconfig : 1;
 };
 
 enum {
 	CMD_MEMORY_ENABLE = 0,
 	CMD_MEMORY_DISABLE,
+	CMD_MEMORY_CONFIGURE,
+	CMD_MEMORY_DECONFIGURE,
 	CMD_NONE
 };
 
@@ -101,16 +109,129 @@ static void idxtostr(struct chmem_desc *desc, uint64_t idx, char *buf, size_t bu
 		 idx, start, end);
 }
 
-static int chmem_size(struct chmem_desc *desc, int enable, int zone_id)
+static bool chmem_memmap_enabled(struct chmem_desc *desc)
+{
+	if (desc->memmap_on_memory == 0 || desc->memmap_on_memory == 1)
+		return true;
+	else
+		return false;
+}
+
+static int chmem_set_memmap_on_memory(struct chmem_desc *desc, char *name)
+{
+	int rc, index;
+
+	index = strtou64_or_err(name + 6, _("Failed to parse index"));
+	rc = ul_path_writef_u64(desc->sysmemconfig, desc->memmap_on_memory,
+				"%s/memmap_on_memory", name);
+	if (rc) {
+		char str[64];
+		idxtostr(desc, index, str, sizeof(str));
+		warn(_("%s memmap-on-memory failed"), str);
+	}
+	return rc;
+}
+
+static int chmem_config(struct chmem_desc *desc, char *name, int configure)
+{
+	int mblock_configured, memmap, rc, index;
+	char str[BUFSIZ], state[BUFSIZ];
+
+	index = strtou64_or_err(name + 6, _("Failed to parse index"));
+	idxtostr(desc, index, str, sizeof(str));
+	rc = ul_path_readf_s32(desc->sysmemconfig, &mblock_configured, "%s/config", name);
+	if (rc)
+		goto out;
+	rc = ul_path_readf_s32(desc->sysmemconfig, &memmap, "%s/memmap_on_memory", name);
+	if (rc)
+		goto out;
+	if (mblock_configured) {
+		if (configure) {
+			if (chmem_memmap_enabled(desc) &&
+					memmap != desc->memmap_on_memory) {
+				if (!desc->is_size || desc->verbose)
+					fprintf(stdout,
+						_("%s must be deconfigured before using -m option\n"), str);
+				rc = -1;
+			} else if (desc->is_size) {
+				/*
+				 * Allow chmem_onoff_size() to proceed with
+				 * configuring different memory blocks when the
+				 * current block is already configured.
+				 */
+				rc = -1;
+			} else if (desc->verbose) {
+				fprintf(stdout, _("%s already configured\n"), str);
+			}
+			goto out;
+		} else if (ul_path_readf_buffer(desc->sysmem, state,
+						sizeof(state), "%s/state", name) > 0 &&
+			   strncmp("online", state, 6) == 0) {
+			if (!desc->is_size || desc->verbose)
+				fprintf(stdout, _("%s must be offline before deconfiguration\n"), str);
+			rc = -1;
+			goto out;
+		}
+	} else {
+		/*
+		 * If memory block is currently in deconfigured state, set
+		 * memmap-on-memory if -m option is enabled.
+		 */
+		if (chmem_memmap_enabled(desc)) {
+			rc = chmem_set_memmap_on_memory(desc, name);
+			if (rc)
+				goto out;
+		} else if (!configure) {
+			/*
+			 * Allow chmem_onoff_size() to proceed with
+			 * deconfiguring different memory blocks when the
+			 * current block is already deconfigured.
+			 */
+			if (desc->is_size)
+				rc = -1;
+			else if (desc->verbose)
+				fprintf(stdout, _("%s already deconfigured\n"), str);
+			goto out;
+		}
+	}
+	rc = ul_path_writef_u64(desc->sysmemconfig, configure ? 1 : 0, "%s/config", name);
+	if (rc) {
+		if (!desc->is_size) {
+			warn(configure ? _("%s configure failed") :
+					 _("%s deconfigure failed"), str);
+		} else if (desc->verbose) {
+			if (configure)
+				fprintf(stdout, _("%s configure failed\n"), str);
+			else
+				fprintf(stdout, _("%s deconfigure failed\n"), str);
+		}
+	} else if (desc->verbose) {
+		if (configure)
+			fprintf(stdout, _("%s configured\n"), str);
+		else
+			fprintf(stdout, _("%s deconfigured\n"), str);
+	}
+out:
+	return rc;
+}
+
+static int chmem_configured(struct chmem_desc *desc, char *name)
+{
+	int mblock_configured = 0;
+
+	ul_path_readf_s32(desc->sysmemconfig, &mblock_configured, "%s/config", name);
+	return mblock_configured;
+}
+
+static int chmem_onoff_size(struct chmem_desc *desc, int enable, int zone_id)
 {
 	char *name, *onoff, line[BUFSIZ], str[BUFSIZ];
 	uint64_t size, index;
+	int i, rc = 0, ndirs;
 	const char *zn;
-	int i, rc;
 
 	size = desc->size;
 	onoff = enable ? "online" : "offline";
-	i = enable ? 0 : desc->ndirs - 1;
 
 	if (enable && zone_id >= 0) {
 		if (zone_id == ZONE_MOVABLE)
@@ -118,15 +239,30 @@ static int chmem_size(struct chmem_desc *desc, int enable, int zone_id)
 		else
 			onoff = "online_kernel";
 	}
-
-	for (; i >= 0 && i < desc->ndirs && size; i += enable ? 1 : -1) {
-		name = desc->dirs[i]->d_name;
+	ndirs = desc->have_memconfig ? desc->memconfig_ndirs : desc->ndirs;
+	i = enable ? 0 : ndirs - 1;
+	for (; i >= 0 && i < ndirs && size; i += enable ? 1 : -1) {
+		if (desc->have_memconfig)
+			name = desc->memconfig_dirs[i]->d_name;
+		else
+			name = desc->dirs[i]->d_name;
 		index = strtou64_or_err(name + 6, _("Failed to parse index"));
-
-		if (ul_path_readf_buffer(desc->sysmem, line, sizeof(line), "%s/state", name) > 0
-		    && strncmp(onoff, line, 6) == 0)
+		if (enable && desc->have_memconfig && !chmem_configured(desc, name)) {
+			/* Configure memory block */
+			rc = chmem_config(desc, name, enable);
+			if (rc)
+				continue;
+		} else if (ul_path_readf_buffer(desc->sysmem, line, sizeof(line), "%s/state", name) > 0) {
+			if (strncmp(onoff, line, 6) == 0)
+				continue;
+		} else if (!enable) {
+			/*
+			 * If /sys/devices/system/memory/memoryX is
+			 * unavailable, memory block is offline and
+			 * deconfigured.
+			 */
 			continue;
-
+		}
 		if (desc->have_zones) {
 			ul_path_readf_buffer(desc->sysmem, line, sizeof(line), "%s/valid_zones", name);
 			if (zone_id >= 0) {
@@ -143,7 +279,6 @@ static int chmem_size(struct chmem_desc *desc, int enable, int zone_id)
 					onoff = "online";
 			}
 		}
-
 		idxtostr(desc, index, str, sizeof(str));
 		rc = ul_path_writef_string(desc->sysmem, onoff, "%s/state", name);
 		if (rc != 0 && desc->verbose) {
@@ -157,6 +292,12 @@ static int chmem_size(struct chmem_desc *desc, int enable, int zone_id)
 			else
 				fprintf(stdout, _("%s disabled\n"), str);
 		}
+		if (!rc && !enable && desc->have_memconfig) {
+			/* Deconfigure memory block */
+			rc = chmem_config(desc, name, enable);
+			if (rc)
+				continue;
+		}
 		if (rc == 0)
 			size--;
 	}
@@ -175,12 +316,80 @@ static int chmem_size(struct chmem_desc *desc, int enable, int zone_id)
 	return size == 0 ? 0 : size == desc->size ? -1 : 1;
 }
 
-static int chmem_range(struct chmem_desc *desc, int enable, int zone_id)
+static int chmem_config_size(struct chmem_desc *desc, int configure)
+{
+	uint64_t size;
+	char *name;
+	int i, rc;
+
+	if (!desc->have_memconfig) {
+		if (configure)
+			fprintf(stdout,
+				_("Skip configuration — use chmem -e instead\n"));
+		else
+			fprintf(stdout,
+				_("Skip deconfiguration - use chmem -d instead\n"));
+		return -1;
+	}
+	size = desc->size;
+	i = configure ? 0 : desc->memconfig_ndirs - 1;
+	for (; i >= 0 && i < desc->memconfig_ndirs && size; i += configure ? 1 : -1) {
+		name = desc->memconfig_dirs[i]->d_name;
+		rc = chmem_config(desc, name, configure);
+		if (rc == 0)
+			size--;
+	}
+	if (size) {
+		uint64_t bytes;
+		char *sizestr;
+
+		bytes = (desc->size - size) * desc->block_size;
+		sizestr = size_to_human_string(SIZE_SUFFIX_1LETTER, bytes);
+		if (configure)
+			fprintf(stdout, _("Could only configure %s of memory\n"), sizestr);
+		else
+			fprintf(stdout, _("Could only deconfigure %s of memory\n"), sizestr);
+		free(sizestr);
+	}
+	return size == 0 ? 0 : size == desc->size ? -1 : 1;
+}
+
+static int chmem_config_range(struct chmem_desc *desc, int configure)
+{
+	uint64_t index, todo;
+	char *name;
+	int rc, i;
+
+	if (!desc->have_memconfig) {
+		if (configure)
+			fprintf(stdout,
+				_("Skip configuration — use chmem -e instead\n"));
+		else
+			fprintf(stdout,
+				_("Skip deconfiguration - use chmem -d instead\n"));
+		return -1;
+	}
+	todo = desc->end - desc->start + 1;
+	for (i = 0; i < desc->memconfig_ndirs; i++) {
+		name = desc->memconfig_dirs[i]->d_name;
+		index = strtou64_or_err(name + 6, _("Failed to parse index"));
+		if (index < desc->start)
+			continue;
+		if (index > desc->end)
+			break;
+		rc = chmem_config(desc, name, configure);
+		if (rc == 0)
+			todo--;
+	}
+	return todo == 0 ? 0 : todo == desc->end - desc->start + 1 ? -1 : 1;
+}
+
+static int chmem_onoff_range(struct chmem_desc *desc, int enable, int zone_id)
 {
 	char *name, *onoff, line[BUFSIZ], str[BUFSIZ];
 	uint64_t index, todo;
+	int i, rc, ndirs;
 	const char *zn;
-	int i, rc;
 
 	todo = desc->end - desc->start + 1;
 	onoff = enable ? "online" : "offline";
@@ -192,22 +401,43 @@ static int chmem_range(struct chmem_desc *desc, int enable, int zone_id)
 			onoff = "online_kernel";
 	}
 
-	for (i = 0; i < desc->ndirs; i++) {
-		name = desc->dirs[i]->d_name;
+	ndirs = desc->have_memconfig ? desc->memconfig_ndirs : desc->ndirs;
+	for (i = 0; i < ndirs; i++) {
+		name = desc->have_memconfig ? desc->memconfig_dirs[i]->d_name :
+					      desc->dirs[i]->d_name;
 		index = strtou64_or_err(name + 6, _("Failed to parse index"));
 		if (index < desc->start)
 			continue;
 		if (index > desc->end)
 			break;
+		if (enable && desc->have_memconfig && !chmem_configured(desc, name)) {
+			/* Configure memory block */
+			rc = chmem_config(desc, name, enable);
+			if (rc)
+				continue;
+		}
 		idxtostr(desc, index, str, sizeof(str));
-		if (ul_path_readf_buffer(desc->sysmem, line, sizeof(line), "%s/state", name) > 0
-		    && strncmp(onoff, line, 6) == 0) {
-			if (desc->verbose && enable)
-				fprintf(stdout, _("%s already enabled\n"), str);
-			else if (desc->verbose && !enable)
-				fprintf(stdout, _("%s already disabled\n"), str);
-			todo--;
-			continue;
+		if (ul_path_readf_buffer(desc->sysmem, line, sizeof(line), "%s/state", name) > 0) {
+			if (strncmp(onoff, line, 6) == 0) {
+				if (desc->verbose && enable)
+					fprintf(stdout, _("%s already enabled\n"), str);
+				else if (desc->verbose && !enable)
+					fprintf(stdout, _("%s already disabled\n"), str);
+				todo--;
+				continue;
+			}
+		} else {
+			/*
+			 * If /sys/devices/system/memory/memoryX is
+			 * unavailable, memory block is offline and
+			 * deconfigured.
+			 */
+			if (!enable) {
+				if (desc->verbose)
+					fprintf(stdout, _("%s already disabled\n"), str);
+				todo--;
+				continue;
+			}
 		}
 
 		if (desc->have_zones) {
@@ -243,6 +473,12 @@ static int chmem_range(struct chmem_desc *desc, int enable, int zone_id)
 			else
 				fprintf(stdout, _("%s disabled\n"), str);
 		}
+		if (!rc && !enable && desc->have_memconfig) {
+			/* Deconfigure memory block */
+			rc = chmem_config(desc, name, enable);
+			if (rc)
+				continue;
+		}
 		if (rc == 0)
 			todo--;
 	}
@@ -256,6 +492,16 @@ static int filter(const struct dirent *de)
 	return isdigit_string(de->d_name + 6);
 }
 
+static void read_conf(struct chmem_desc *desc)
+{
+	if (!desc->have_memconfig)
+		return;
+	desc->memconfig_ndirs = scandir(_PATH_SYS_MEMCONFIG, &desc->memconfig_dirs,
+					filter, versionsort);
+	if (desc->memconfig_ndirs <= 0)
+		err(EXIT_FAILURE, _("Failed to read %s"), _PATH_SYS_MEMCONFIG);
+}
+
 static void read_info(struct chmem_desc *desc)
 {
 	char line[128];
@@ -269,6 +515,7 @@ static void read_info(struct chmem_desc *desc)
 	desc->block_size = strtoumax(line, NULL, 16);
 	if (errno)
 		goto fail;
+	read_conf(desc);
 	return;
 fail:
 	err(EXIT_FAILURE, _("Failed to read %s"), _PATH_SYS_MEMORY);
@@ -347,6 +594,9 @@ static void __attribute__((__noreturn__)) usage(void)
 	fputs(_(" -d, --disable      disable memory\n"), out);
 	fputs(_(" -b, --blocks       use memory blocks\n"), out);
 	fputs(_(" -z, --zone <name>  select memory zone (see below)\n"), out);
+	fputs(_(" -c, --configure    configure range\n"), out);
+	fputs(_(" -g, --deconfigure  deconfigure range\n"), out);
+	fputs(_(" -m, --memmap-on-memory select memmap-on-memory\n"), out);
 	fputs(_(" -v, --verbose      verbose output\n"), out);
 	fprintf(out, USAGE_HELP_OPTIONS(20));
 
@@ -359,6 +609,52 @@ static void __attribute__((__noreturn__)) usage(void)
 	exit(EXIT_SUCCESS);
 }
 
+static int chmem_range(struct chmem_desc *desc, int cmd, int zone_id)
+{
+	int rc = -1;
+
+	switch (cmd) {
+	case CMD_MEMORY_ENABLE:
+		rc = chmem_onoff_range(desc, 1, zone_id);
+		break;
+	case CMD_MEMORY_DISABLE:
+		rc = chmem_onoff_range(desc, 0, zone_id);
+		break;
+	case CMD_MEMORY_CONFIGURE:
+		rc = chmem_config_range(desc, 1);
+		break;
+	case CMD_MEMORY_DECONFIGURE:
+		rc = chmem_config_range(desc, 0);
+		break;
+	default:
+		break;
+	}
+	return rc;
+}
+
+static int chmem_size(struct chmem_desc *desc, int cmd, int zone_id)
+{
+	int rc = -1;
+
+	switch (cmd) {
+	case CMD_MEMORY_ENABLE:
+		rc = chmem_onoff_size(desc, 1, zone_id);
+		break;
+	case CMD_MEMORY_DISABLE:
+		rc = chmem_onoff_size(desc, 0, zone_id);
+		break;
+	case CMD_MEMORY_CONFIGURE:
+		rc = chmem_config_size(desc, 1);
+		break;
+	case CMD_MEMORY_DECONFIGURE:
+		rc = chmem_config_size(desc, 0);
+		break;
+	default:
+		break;
+	}
+	return rc;
+}
+
 int main(int argc, char **argv)
 {
 	struct chmem_desc _desc = { 0 }, *desc = &_desc;
@@ -374,11 +670,15 @@ int main(int argc, char **argv)
 		{"verbose",	no_argument,		NULL, 'v'},
 		{"version",	no_argument,		NULL, 'V'},
 		{"zone",	required_argument,	NULL, 'z'},
+		{"configure",	no_argument,		NULL, 'c'},
+		{"deconfigure", no_argument,		NULL, 'g'},
+		{"memmap-on-memory", required_argument,	NULL, 'm'},
 		{NULL,		0,			NULL, 0}
 	};
 
 	static const ul_excl_t excl[] = {	/* rows and cols in ASCII order */
-		{ 'd','e' },
+		{ 'd', 'e', 'g', 'm' },
+		{ 'c', 'd', 'e', 'g' },
 		{ 0 }
 	};
 	int excl_st[ARRAY_SIZE(excl)] = UL_EXCL_STATUS_INIT;
@@ -389,13 +689,18 @@ int main(int argc, char **argv)
 	close_stdout_atexit();
 
 	ul_path_init_debug();
+	desc->memmap_on_memory = -1;
 	desc->sysmem = ul_new_path(_PATH_SYS_MEMORY);
 	if (!desc->sysmem)
 		err(EXIT_FAILURE, _("failed to initialize %s handler"), _PATH_SYS_MEMORY);
-
+	desc->sysmemconfig = ul_new_path(_PATH_SYS_MEMCONFIG);
+	if (!desc->sysmemconfig)
+		err(EXIT_FAILURE, _("failed to initialize %s handler"), _PATH_SYS_MEMCONFIG);
+	if (ul_path_access(desc->sysmemconfig, F_OK, "memory0") == 0)
+		desc->have_memconfig = 1;
 	read_info(desc);
 
-	while ((c = getopt_long(argc, argv, "bdehvVz:", longopts, NULL)) != -1) {
+	while ((c = getopt_long(argc, argv, "bcdeghm:vVz:", longopts, NULL)) != -1) {
 
 		err_exclusive_options(c, longopts, excl, excl_st);
 
@@ -415,7 +720,15 @@ int main(int argc, char **argv)
 		case 'z':
 			zone = xstrdup(optarg);
 			break;
-
+		case 'c':
+			cmd = CMD_MEMORY_CONFIGURE;
+			break;
+		case 'g':
+			cmd = CMD_MEMORY_DECONFIGURE;
+			break;
+		case 'm':
+			desc->memmap_on_memory = atoi(optarg);
+			break;
 		case 'h':
 			usage();
 		case 'V':
@@ -448,9 +761,9 @@ int main(int argc, char **argv)
 	}
 
 	if (desc->is_size)
-		rc = chmem_size(desc, cmd == CMD_MEMORY_ENABLE ? 1 : 0, zone_id);
+		rc = chmem_size(desc, cmd, zone_id);
 	else
-		rc = chmem_range(desc, cmd == CMD_MEMORY_ENABLE ? 1 : 0, zone_id);
+		rc = chmem_range(desc, cmd, zone_id);
 
 	ul_unref_path(desc->sysmem);
 
-- 
2.41.0

[PATCH v2 4/6] chmem: add chmem documentation for dynamic (de)configuration of memory

[Sumanth Korikkar]

Describe chmem configure, deconfigure and memmap-on-memory options:

ff18dcb19aab ("s390/sclp: Add support for dynamic (de)configuration of
memory") s390 kernel no longer pre-adds all standby memory at boot.
Instead, users must explicitly configure a block before it can be used
for online/offline actions. At configuration time, users can dynamically
decide whether to use optional memmap-on-memory for each memory block,
where value of 1 allocates metadata (such as struct pages array) from
the hotplug memory itself, enabling hot-add operations even under memory
pressure. A value of 0 stores metadata in regular system memory, which
may require additional free memory, but enables continuous physical
memory across memory blocks.

Add documentation to reflect the following options:
* chmem --configure 128M --memmap-on-memory 1
* chmem --deconfigure 128M
* chmem --enable 128M # implicitly configure memory if supported by
  architecture and online it
* chmem --disable 128M  # offline memory and implicitly deconfigure if
  supported by the architecture.

Just like online and offline actions, memory configuration and
deconfiguration can be controlled through similar options. Also,
memmap-on-memory setting can be changed, only when the memory block is
in deconfigured state. This means, it is usable only via --configure
option.

Reviewed-by: Maria Eisenhaendler <maria1@de.ibm.com>
Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
---
 sys-utils/chmem.8.adoc | 47 +++++++++++++++++++++++++++++++++++++-----
 1 file changed, 42 insertions(+), 5 deletions(-)

diff --git a/sys-utils/chmem.8.adoc b/sys-utils/chmem.8.adoc
index 5067a98b8..537beec63 100644
--- a/sys-utils/chmem.8.adoc
+++ b/sys-utils/chmem.8.adoc
@@ -12,7 +12,7 @@ chmem - configure memory
 
 == SYNOPSIS
 
-*chmem* [*-h*] [*-V*] [*-v*] [*-e*|*-d*] [_SIZE_|_RANGE_|*-b* _BLOCKRANGE_] [*-z* _ZONE_]
+*chmem* [*-h*] [*-V*] [*-v*] [*-c*|*-e*|*-d*|*-g*] [_SIZE_|_RANGE_|*-b* _BLOCKRANGE_] [*-z* _ZONE_] [*-m* _MEMMAP-ON-MEMORY_]
 
 == DESCRIPTION
 
@@ -26,23 +26,48 @@ The *chmem* command sets a particular size or range of memory online or offline.
 
 * Specify _ZONE_ as the name of a memory zone, as shown in the output of the *lsmem -o +ZONES* command. The output shows one or more valid memory zones for each memory range. If multiple zones are shown, then the memory range currently belongs to the first zone. By default, *chmem* will set memory online to the zone Movable, if this is among the valid zones. This default can be changed by specifying the *--zone* option with another valid zone. For memory ballooning, it is recommended to select the zone Movable for memory online and offline, if possible. Memory in this zone is much more likely to be able to be offlined again, but it cannot be used for arbitrary kernel allocations, only for migratable pages (e.g., anonymous and page cache pages). Use the *--help* option to see all available zones.
 
+* Specify _MEMMAP-ON-MEMORY_ as 1 or 0. A value of 1 allocates hotplug metadata (such as the struct pages array) from the hotplug memory itself, enabling hot-add operations even under memory pressure and without requiring additional system memory to do so. A value of 0 stores hotplugged memory metadata in regular system memory, which helps avoid issues related to fragmentation of continuous physical memory across memory blocks. The value can only be set when the memory block is in a deconfigured state, and *--memmap-on-memory* is valid only with *--configure*. If not specified, and if supported, *chmem* uses the default value shown in *lsmem* output.
+
+The *--enable* option configures the memory, if this is supported by the architecture. If configuring memory is not supported by the architecture, *--enable* still brings the memory online.
+
+The *--disable* option brings the memory offline and performs an optional deconfigure step if this is supported by the architecture.
+
+The *--configure* option requests memory from the hypervisor without bringing it online, when supported by the architecture, allowing explicit control and use of *--memmap-on-memory*.
+
+The *--deconfigure* option returns memory resources to the hypervisor if supported by the architecture.
+
 _SIZE_ and _RANGE_ must be aligned to the Linux memory block size, as shown in the output of the *lsmem*(1) command.
 
 Setting memory online can fail for various reasons. On virtualized systems it can fail if the hypervisor does not have enough memory left, for example because memory was overcommitted. Setting memory offline can fail if Linux cannot free the memory. If only part of the requested memory can be set online or offline, a message tells you how much memory was set online or offline instead of the requested amount.
 
-When setting memory online *chmem* starts with the lowest memory block numbers. When setting memory offline *chmem* starts with the highest memory block numbers.
+When setting memory online or when configuring memory, *chmem* starts with the lowest memory block numbers. When setting memory offline or deconfiguring memory, *chmem* starts with the highest memory block numbers.
+
+== ARCHITECTURE
+
+* s390 architecture:
+
+_MEMMAP-ON-MEMORY_: For memory blocks configured online at boot, the default value is 0 because they are added without memmap-on-memory support. Memory added dynamically at runtime uses the default value displayed in *lsmem* output.
 
 == OPTIONS
 
 *-b*, *--blocks*::
 Use a _BLOCKRANGE_ parameter instead of _RANGE_ or _SIZE_ for the *--enable* and *--disable* options.
 
+*-c*, *--configure*::
+Set the specified _RANGE_, _SIZE_, or _BLOCKRANGE_ of memory to be configured.
+
 *-d*, *--disable*::
 Set the specified _RANGE_, _SIZE_, or _BLOCKRANGE_ of memory offline.
 
 *-e*, *--enable*::
 Set the specified _RANGE_, _SIZE_, or _BLOCKRANGE_ of memory online.
 
+*-g*, *--deconfigure*::
+Set the specified _RANGE_, _SIZE_, or _BLOCKRANGE_ of memory to be deconfigured.
+
+*-m*, *--memmap-on-memory*::
+Select memmap-on-memory for the specified _RANGE_, _SIZE_, or _BLOCKRANGE_ of memory. This option is valid only with *--configure*.
+
 *-z*, *--zone*::
 Select the memory _ZONE_ where to set the specified _RANGE_, _SIZE_, or _BLOCKRANGE_ of memory online or offline. By default, memory will be set online to the zone Movable, if possible.
 
@@ -70,13 +95,25 @@ partial success
 This command requests 1024 MiB of memory to be set online.
 
 *chmem -e 2g*::
-This command requests 2 GiB of memory to be set online.
+This command requests 2 GB of memory to be brought online and, if supported by the architecture, configures the memory beforehand.
 
 *chmem --disable 0x00000000e4000000-0x00000000f3ffffff*::
-This command requests the memory range starting with 0x00000000e4000000 and ending with 0x00000000f3ffffff to be set offline.
+This command takes the memory range from 0x00000000e4000000 to 0x00000000f3ffffff offline and deconfigures it if supported by the architecture.
 
 *chmem -b -d 10*::
-This command requests the memory block number 10 to be set offline.
+This command takes memory block number 10 offline.
+
+*chmem -b -c 10 -m 1*::
+This command configures memory block 10 with  _MEMMAP-ON-MEMORY_ set. The block must be in a deconfigured state.
+
+*chmem -b -c 10*::
+This command configures memory block 10 with the default _MEMMAP-ON-MEMORY_ setting. The default value is displayed in *lsmem --output-all*. The block must be in a deconfigured state.
+
+*chmem -b -g 10*::
+This command deconfigures memory block 10. The block must be offline.
+
+*chmem -d 5g*::
+This command takes 5 GB of memory offline and deconfigures it if supported by the architecture. Blocks that are already offline but still configured are skipped and must be explicitly deconfigured with *--deconfigure*.
 
 == SEE ALSO
 
-- 
2.41.0

[PATCH v2 5/6] lsmem: add doc for dynamic (de)configuration and memmap-on-memory support

[Sumanth Korikkar]

lsmem --output-all now displays two new columns:

CONFIGURED : yes/no indicating if a memory block has been explicitly
configured.

MEMMAP-ON-MEMORY : yes/no indicating whether the block uses
memmap-on-memory.

lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP-ON-MEMORY
RANGE                   SIZE   STATE   BLOCK CONFIGURED MEMMAP-ON-MEMORY
0x00000000-0x7fffffff   2G     online  0-15  yes        no
0x80000000-0xffffffff   2G     offline 16-31 no         yes

Memory block size:                128M
Total online memory:                2G
Total offline memory:               2G
Memmap on memory parameter:        yes

Add documentation for new fields.

Reviewed-by: Maria Eisenhaendler <maria1@de.ibm.com>
Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
---
 sys-utils/lsmem.1.adoc | 44 +++++++++++++++++++++++++++++++++++++++---
 1 file changed, 41 insertions(+), 3 deletions(-)

diff --git a/sys-utils/lsmem.1.adoc b/sys-utils/lsmem.1.adoc
index 9c9397631..e7226725a 100644
--- a/sys-utils/lsmem.1.adoc
+++ b/sys-utils/lsmem.1.adoc
@@ -24,7 +24,45 @@ The *lsmem* command lists a new memory range always when the current memory bloc
 
 Note that some output columns may provide inaccurate information if a split policy forces *lsmem* to ignore differences in some attributes. For example if you merge removable and non-removable memory blocks to the one range than all the range will be marked as non-removable on *lsmem* output.
 
-Not all columns are supported on all systems. If an unsupported column is specified, *lsmem* prints the column but does not provide any data for it.
+The supported columns are RANGE, SIZE, STATE, REMOVABLE, BLOCK, NODE, ZONES, CONFIGURED, MEMMAP-ON-MEMORY.
+RANGE
+The start and end physical address of the memory range.
+
+SIZE
+The size of the memory range, representing the total amount of memory in that range.
+
+STATE
+The current online status of the memory range. Common states include online, offline or transitional states.
+
+BLOCK
+The specific memory block number.
+
+NODE
+The NUMA (Non-Uniform Memory Access) node to which the memory block belongs.
+
+ZONES
+The memory zones to which the blocks belongs, such as DMA, Normal, Movable.
+
+CONFIGURED
+The configuration state of a memory block. Refer to *chmem* for details on configuring or deconfiguring memory blocks.
+
+MEMMAP-ON-MEMORY
+The memmap-on-memory state of the memory block at configuration time. This setting indicates where memory hotplug stores its internal metadata (the struct pages array or memmap). If MEMMAP-ON-MEMORY is set to 1, the metadata is allocated directly from the newly added hotplugged memory, enabling hot-add operations even when the system is under high memory pressure. If set to 0, the memmap metadata is allocated from existing system memory.
+
+Possible BLOCK, CONFIGURED, STATE, MEMMAP-ON-MEMORY states::
+
+[cols="10,10,10,15,60", options="header"]
+|===
+| BLOCK | STATE   | CONFIGURED | MEMMAP-ON-MEMORY | Description
+
+| 0     | online  | yes        | yes/no           | The memory is configured with memmap-on-memory set to (1 or 0) and memory is currently online.
+
+| 1     | offline | yes        | yes/no           | The memory is configured, but memory is offline.
+
+| 2     | offline | no         | yes/no           | The memory is offline and deconfigured.
+|===
+
+Not all columns are supported on all systems. If an unsupported column is specified, *lsmem* prints the column but does not provide any data for it. Additionally, *lsmem* may skip columns like CONFIGURED or MEMMAP-ON-MEMORY if these states are not relevant to the system's architecture.
 
 Use the *--help* option to see the columns description.
 
@@ -45,7 +83,7 @@ Use JSON output format.
 Do not print a header line.
 
 *-o*, *--output* _list_::
-Specify which output columns to print. Use *--help* to get a list of all supported columns. The default list of columns may be extended if _list_ is specified in the format **+**__list__ (e.g., *lsmem -o +NODE*).
+Specify which output columns to print. Use *--help* to obtain a list of all supported columns. To extend the default list of columns specify _list_ in the format **+**__list__. For example, *lsmem -o +NODE*.
 
 *--output-all*::
 Output all available columns.
@@ -57,7 +95,7 @@ Produce output in the form of key="value" pairs. All potentially unsafe value ch
 Produce output in raw format. All potentially unsafe characters are hex-escaped (\x<code>).
 
 *-S*, *--split* _list_::
-Specify which columns (attributes) use to split memory blocks to ranges. The supported columns are STATE, REMOVABLE, NODE and ZONES, or "none". The other columns are silently ignored. For more details see *DESCRIPTION* above.
+Specify which columns are used to split memory blocks to ranges. The supported columns are STATE, REMOVABLE, NODE, ZONES, CONFIGURED, MEMMAP-ON-MEMORY or "none". The other columns are silently ignored. For more details see *DESCRIPTION* above.
 
 *-s*, *--sysroot* _directory_::
 Gather memory data for a Linux instance other than the instance from which the *lsmem* command is issued. The specified _directory_ is the system root of the Linux instance to be inspected.
-- 
2.41.0

[PATCH v2 6/6] lsmem,chmem: add configure/deconfigure bash completion options

[Sumanth Korikkar]

Add bash completion for configure/deconfigure options in chmem and lsmem.

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
---
 bash-completion/chmem | 3 +++
 bash-completion/lsmem | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/bash-completion/chmem b/bash-completion/chmem
index 3e3af87ac..f10646677 100644
--- a/bash-completion/chmem
+++ b/bash-completion/chmem
@@ -14,6 +14,9 @@ _chmem_module()
 			OPTS="
 				--enable
 				--disable
+				--configure
+				--deconfigure
+				--memmap-on-memory
 				--blocks
 				--verbose
 				--zone
diff --git a/bash-completion/lsmem b/bash-completion/lsmem
index 7d6e84247..185a15fd2 100644
--- a/bash-completion/lsmem
+++ b/bash-completion/lsmem
@@ -9,7 +9,7 @@ _lsmem_module()
 			local prefix realcur OUTPUT_ALL OUTPUT
 			realcur="${cur##*,}"
 			prefix="${cur%$realcur}"
-			OUTPUT_ALL='RANGE SIZE STATE REMOVABLE BLOCK NODE ZONES'
+			OUTPUT_ALL='RANGE SIZE STATE REMOVABLE BLOCK NODE ZONES CONFIGURED MEMMAP-ON-MEMORY'
 			for WORD in $OUTPUT_ALL; do
 				if ! [[ $prefix == *"$WORD"* ]]; then
 					OUTPUT="$WORD ${OUTPUT:-""}"
-- 
2.41.0