[PATCH RFC] memory-hotplug: add automatic onlining policy for the newly added memory

[PATCH RFC] memory-hotplug: add automatic onlining policy for the newly added memory

[Vitaly Kuznetsov]

Currently, all newly added memory blocks remain in 'offline' state unless
someone onlines them, some linux distributions carry special udev rules
like:

SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online"

to make this happen automatically. This is not a great solution for virtual
machines where memory hotplug is being used to address high memory pressure
situations as such onlining is slow and a userspace process doing this
(udev) has a chance of being killed by the OOM killer as it will probably
require to allocate some memory.

Introduce default policy for the newly added memory blocks in
/sys/devices/system/memory/hotplug_autoonline file with two possible
values: "offline" (the default) which preserves the current behavior and
"online" which causes all newly added memory blocks to go online as
soon as they're added.

Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Daniel Kiper <daniel.kiper@oracle.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Tang Chen <tangchen@cn.fujitsu.com>
Cc: David Vrabel <david.vrabel@citrix.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
Cc: Xishi Qiu <qiuxishi@huawei.com>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: "K. Y. Srinivasan" <kys@microsoft.com>
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
- I was able to find previous attempts to fix the issue, e.g.:
  http://marc.info/?l=linux-kernel&m=137425951924598&w=2
  http://marc.info/?l=linux-acpi&m=127186488905382
  but I'm not completely sure why it didn't work out and the solution
  I suggest is not 'smart enough', thus 'RFC'.
---
 Documentation/memory-hotplug.txt | 21 +++++++++++++++++----
 drivers/base/memory.c            | 35 +++++++++++++++++++++++++++++++++++
 include/linux/memory_hotplug.h   |  2 ++
 mm/memory_hotplug.c              |  8 ++++++++
 4 files changed, 62 insertions(+), 4 deletions(-)

diff --git a/Documentation/memory-hotplug.txt b/Documentation/memory-hotplug.txt
index ce2cfcf..fe576d9 100644
--- a/Documentation/memory-hotplug.txt
+++ b/Documentation/memory-hotplug.txt
@@ -254,12 +254,25 @@ If the memory block is online, you'll read "online".
 If the memory block is offline, you'll read "offline".
 
 
-5.2. How to online memory
+5.2. Memory onlining
 ------------
-Even if the memory is hot-added, it is not at ready-to-use state.
-For using newly added memory, you have to "online" the memory block.
+When the memory is hot-added, the kernel decides whether or not to "online"
+it according to the policy which can be read from "hotplug_autoonline" file:
 
-For onlining, you have to write "online" to the memory block's state file as:
+% cat /sys/devices/system/memory/hotplug_autoonline
+
+The default is "offline" which means the newly added memory will not be at
+ready-to-use state and you have to "online" the newly added memory blocks
+manually.
+
+Automatic onlining can be requested by writing "online" to "hotplug_autoonline"
+file:
+
+% echo online > /sys/devices/system/memory/hotplug_autoonline
+
+If the automatic onlining wasn't requested or some memory block was offlined
+it is possible to change the individual block's state by writing to the "state"
+file:
 
 % echo online > /sys/devices/system/memory/memoryXXX/state
 
diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 25425d3..001fefe 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -438,6 +438,40 @@ print_block_size(struct device *dev, struct device_attribute *attr,
 
 static DEVICE_ATTR(block_size_bytes, 0444, print_block_size, NULL);
 
+
+/*
+ * Memory auto online policy.
+ */
+
+static ssize_t
+show_memhp_autoonline(struct device *dev, struct device_attribute *attr,
+		      char *buf)
+{
+	if (memhp_autoonline == MMOP_ONLINE_KEEP)
+		return sprintf(buf, "online\n");
+	else if (memhp_autoonline == MMOP_OFFLINE)
+		return sprintf(buf, "offline\n");
+	else
+		return sprintf(buf, "unknown\n");
+}
+
+static ssize_t
+store_memhp_autoonline(struct device *dev, struct device_attribute *attr,
+		       const char *buf, size_t count)
+{
+	if (sysfs_streq(buf, "online"))
+		memhp_autoonline = MMOP_ONLINE_KEEP;
+	else if (sysfs_streq(buf, "offline"))
+		memhp_autoonline = MMOP_OFFLINE;
+	else
+		return -EINVAL;
+
+	return count;
+}
+
+static DEVICE_ATTR(hotplug_autoonline, 0644, show_memhp_autoonline,
+		   store_memhp_autoonline);
+
 /*
  * Some architectures will have custom drivers to do this, and
  * will not need to do it from userspace.  The fake hot-add code
@@ -737,6 +771,7 @@ static struct attribute *memory_root_attrs[] = {
 #endif
 
 	&dev_attr_block_size_bytes.attr,
+	&dev_attr_hotplug_autoonline.attr,
 	NULL
 };
 
diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
index 2ea574f..fb64eea 100644
--- a/include/linux/memory_hotplug.h
+++ b/include/linux/memory_hotplug.h
@@ -99,6 +99,8 @@ extern void __online_page_free(struct page *page);
 
 extern int try_online_node(int nid);
 
+extern int memhp_autoonline;
+
 #ifdef CONFIG_MEMORY_HOTREMOVE
 extern bool is_pageblock_removable_nolock(struct page *page);
 extern int arch_remove_memory(u64 start, u64 size);
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 67d488a..37a190e 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -76,6 +76,9 @@ static struct {
 #define memhp_lock_acquire()      lock_map_acquire(&mem_hotplug.dep_map)
 #define memhp_lock_release()      lock_map_release(&mem_hotplug.dep_map)
 
+int memhp_autoonline = MMOP_OFFLINE;
+EXPORT_SYMBOL_GPL(memhp_autoonline);
+
 void get_online_mems(void)
 {
 	might_sleep();
@@ -1292,6 +1295,11 @@ int __ref add_memory_resource(int nid, struct resource *res)
 	/* create new memmap entry */
 	firmware_map_add_hotplug(start, start + size, "System RAM");
 
+	/* online the region if it is requested by current policy */
+	if (memhp_autoonline != MMOP_OFFLINE)
+		online_pages(start >> PAGE_SHIFT, size >> PAGE_SHIFT,
+			     memhp_autoonline);
+
 	goto out;
 
 error:
-- 
2.4.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH RFC] memory-hotplug: add automatic onlining policy for the newly added memory

[Daniel Kiper]

Hey Vitaly,

On Tue, Dec 15, 2015 at 07:05:53PM +0100, Vitaly Kuznetsov wrote:
> Currently, all newly added memory blocks remain in 'offline' state unless
> someone onlines them, some linux distributions carry special udev rules
> like:
>
> SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online"
>
> to make this happen automatically. This is not a great solution for virtual
> machines where memory hotplug is being used to address high memory pressure
> situations as such onlining is slow and a userspace process doing this
> (udev) has a chance of being killed by the OOM killer as it will probably
> require to allocate some memory.
>
> Introduce default policy for the newly added memory blocks in
> /sys/devices/system/memory/hotplug_autoonline file with two possible
> values: "offline" (the default) which preserves the current behavior and
> "online" which causes all newly added memory blocks to go online as
> soon as they're added.

In general idea make sense for me but...

> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Daniel Kiper <daniel.kiper@oracle.com>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: Tang Chen <tangchen@cn.fujitsu.com>
> Cc: David Vrabel <david.vrabel@citrix.com>
> Cc: David Rientjes <rientjes@google.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
> Cc: Xishi Qiu <qiuxishi@huawei.com>
> Cc: Mel Gorman <mgorman@techsingularity.net>
> Cc: "K. Y. Srinivasan" <kys@microsoft.com>
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
> - I was able to find previous attempts to fix the issue, e.g.:
>   http://marc.info/?l=linux-kernel&m=137425951924598&w=2
>   http://marc.info/?l=linux-acpi&m=127186488905382
>   but I'm not completely sure why it didn't work out and the solution
>   I suggest is not 'smart enough', thus 'RFC'.
> ---
>  Documentation/memory-hotplug.txt | 21 +++++++++++++++++----
>  drivers/base/memory.c            | 35 +++++++++++++++++++++++++++++++++++
>  include/linux/memory_hotplug.h   |  2 ++
>  mm/memory_hotplug.c              |  8 ++++++++
>  4 files changed, 62 insertions(+), 4 deletions(-)
>
> diff --git a/Documentation/memory-hotplug.txt b/Documentation/memory-hotplug.txt
> index ce2cfcf..fe576d9 100644
> --- a/Documentation/memory-hotplug.txt
> +++ b/Documentation/memory-hotplug.txt
> @@ -254,12 +254,25 @@ If the memory block is online, you'll read "online".
>  If the memory block is offline, you'll read "offline".
>
>
> -5.2. How to online memory
> +5.2. Memory onlining
>  ------------
> -Even if the memory is hot-added, it is not at ready-to-use state.
> -For using newly added memory, you have to "online" the memory block.
> +When the memory is hot-added, the kernel decides whether or not to "online"
> +it according to the policy which can be read from "hotplug_autoonline" file:
>
> -For onlining, you have to write "online" to the memory block's state file as:
> +% cat /sys/devices/system/memory/hotplug_autoonline
> +
> +The default is "offline" which means the newly added memory will not be at
> +ready-to-use state and you have to "online" the newly added memory blocks
> +manually.
> +
> +Automatic onlining can be requested by writing "online" to "hotplug_autoonline"
> +file:
> +
> +% echo online > /sys/devices/system/memory/hotplug_autoonline
> +
> +If the automatic onlining wasn't requested or some memory block was offlined
> +it is possible to change the individual block's state by writing to the "state"
> +file:
>
>  % echo online > /sys/devices/system/memory/memoryXXX/state
>
> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> index 25425d3..001fefe 100644
> --- a/drivers/base/memory.c
> +++ b/drivers/base/memory.c
> @@ -438,6 +438,40 @@ print_block_size(struct device *dev, struct device_attribute *attr,
>
>  static DEVICE_ATTR(block_size_bytes, 0444, print_block_size, NULL);
>
> +
> +/*
> + * Memory auto online policy.
> + */
> +
> +static ssize_t
> +show_memhp_autoonline(struct device *dev, struct device_attribute *attr,
> +		      char *buf)
> +{
> +	if (memhp_autoonline == MMOP_ONLINE_KEEP)
> +		return sprintf(buf, "online\n");
> +	else if (memhp_autoonline == MMOP_OFFLINE)
> +		return sprintf(buf, "offline\n");
> +	else
> +		return sprintf(buf, "unknown\n");

You do not allow unknown state below, so, I do not know how it can appear
here. If it appears out of the blue then I think that we should be alert
because something magic happens around us. Hence, if you wish to leave
this unknown stuff then I suppose we should at least call WARN_ON() if
not BUG_ON() there too (well, I am not convinced about latter).

> +}
> +
> +static ssize_t
> +store_memhp_autoonline(struct device *dev, struct device_attribute *attr,
> +		       const char *buf, size_t count)
> +{
> +	if (sysfs_streq(buf, "online"))
> +		memhp_autoonline = MMOP_ONLINE_KEEP;
> +	else if (sysfs_streq(buf, "offline"))
> +		memhp_autoonline = MMOP_OFFLINE;
> +	else
> +		return -EINVAL;

Here you are not able to set anything which is not allowed.
So, please look above.

Daniel

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH RFC] memory-hotplug: add automatic onlining policy for the newly added memory

[Xishi Qiu]

On 2015/12/16 2:05, Vitaly Kuznetsov wrote:

> Currently, all newly added memory blocks remain in 'offline' state unless
> someone onlines them, some linux distributions carry special udev rules
> like:
> 
> SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online"
> 
> to make this happen automatically. This is not a great solution for virtual
> machines where memory hotplug is being used to address high memory pressure
> situations as such onlining is slow and a userspace process doing this
> (udev) has a chance of being killed by the OOM killer as it will probably
> require to allocate some memory.
> 
> Introduce default policy for the newly added memory blocks in
> /sys/devices/system/memory/hotplug_autoonline file with two possible
> values: "offline" (the default) which preserves the current behavior and
> "online" which causes all newly added memory blocks to go online as
> soon as they're added.
> 
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Daniel Kiper <daniel.kiper@oracle.com>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: Tang Chen <tangchen@cn.fujitsu.com>
> Cc: David Vrabel <david.vrabel@citrix.com>
> Cc: David Rientjes <rientjes@google.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
> Cc: Xishi Qiu <qiuxishi@huawei.com>
> Cc: Mel Gorman <mgorman@techsingularity.net>
> Cc: "K. Y. Srinivasan" <kys@microsoft.com>
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
> - I was able to find previous attempts to fix the issue, e.g.:
>   http://marc.info/?l=linux-kernel&m=137425951924598&w=2
>   http://marc.info/?l=linux-acpi&m=127186488905382
>   but I'm not completely sure why it didn't work out and the solution
>   I suggest is not 'smart enough', thus 'RFC'.

+ CC: 
yanxiaofeng@inspur.com
liuchangsheng@inspur.com

Hi Vitaly,

Why not use udev rule? I think it can online pages automatically.

Thanks,
Xishi Qiu

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH RFC] memory-hotplug: add automatic onlining policy for the newly added memory

[Vitaly Kuznetsov]

Xishi Qiu <qiuxishi@huawei.com> writes:

> On 2015/12/16 2:05, Vitaly Kuznetsov wrote:
>
>> Currently, all newly added memory blocks remain in 'offline' state unless
>> someone onlines them, some linux distributions carry special udev rules
>> like:
>> 
>> SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online"
>> 
>> to make this happen automatically. This is not a great solution for virtual
>> machines where memory hotplug is being used to address high memory pressure
>> situations as such onlining is slow and a userspace process doing this
>> (udev) has a chance of being killed by the OOM killer as it will probably
>> require to allocate some memory.
>> 
>> Introduce default policy for the newly added memory blocks in
>> /sys/devices/system/memory/hotplug_autoonline file with two possible
>> values: "offline" (the default) which preserves the current behavior and
>> "online" which causes all newly added memory blocks to go online as
>> soon as they're added.
>> 
>> Cc: Jonathan Corbet <corbet@lwn.net>
>> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>> Cc: Daniel Kiper <daniel.kiper@oracle.com>
>> Cc: Dan Williams <dan.j.williams@intel.com>
>> Cc: Tang Chen <tangchen@cn.fujitsu.com>
>> Cc: David Vrabel <david.vrabel@citrix.com>
>> Cc: David Rientjes <rientjes@google.com>
>> Cc: Andrew Morton <akpm@linux-foundation.org>
>> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
>> Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
>> Cc: Xishi Qiu <qiuxishi@huawei.com>
>> Cc: Mel Gorman <mgorman@techsingularity.net>
>> Cc: "K. Y. Srinivasan" <kys@microsoft.com>
>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> ---
>> - I was able to find previous attempts to fix the issue, e.g.:
>>   http://marc.info/?l=linux-kernel&m=137425951924598&w=2
>>   http://marc.info/?l=linux-acpi&m=127186488905382
>>   but I'm not completely sure why it didn't work out and the solution
>>   I suggest is not 'smart enough', thus 'RFC'.
>
> + CC: 
> yanxiaofeng@inspur.com
> liuchangsheng@inspur.com
>
> Hi Vitaly,
>
> Why not use udev rule? I think it can online pages automatically.
>

Two main reasons:
1) I remember someone saying "You never need a mouse in order to add
another mouse to the kernel" -- but we  we need memory to add more
memory. Udev has a chance of being killed by the OOM killer as
performing an action will probably require to allocate some
memory. Other than that udev actions are generally slow compared to what
we can do in kernel.

2) I agree with Kay that '... unconditional hotplug loop through
userspace is absolutely pointless' (https://lkml.org/lkml/2013/7/25/354). 
(... and I should had add him to CC, adding now). Udev maintainers
refused to add a rule for unconditional memory onlining to udev and now
linux distros have to carry such custom rules.

-- 
  Vitaly

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH RFC] memory-hotplug: add automatic onlining policy for the newly added memory

[Vitaly Kuznetsov]

Daniel Kiper <daniel.kiper@oracle.com> writes:

> Hey Vitaly,
>
> On Tue, Dec 15, 2015 at 07:05:53PM +0100, Vitaly Kuznetsov wrote:
>> Currently, all newly added memory blocks remain in 'offline' state unless
>> someone onlines them, some linux distributions carry special udev rules
>> like:
>>
>> SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online"
>>
>> to make this happen automatically. This is not a great solution for virtual
>> machines where memory hotplug is being used to address high memory pressure
>> situations as such onlining is slow and a userspace process doing this
>> (udev) has a chance of being killed by the OOM killer as it will probably
>> require to allocate some memory.
>>
>> Introduce default policy for the newly added memory blocks in
>> /sys/devices/system/memory/hotplug_autoonline file with two possible
>> values: "offline" (the default) which preserves the current behavior and
>> "online" which causes all newly added memory blocks to go online as
>> soon as they're added.
>
> In general idea make sense for me but...
>
>> Cc: Jonathan Corbet <corbet@lwn.net>
>> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>> Cc: Daniel Kiper <daniel.kiper@oracle.com>
>> Cc: Dan Williams <dan.j.williams@intel.com>
>> Cc: Tang Chen <tangchen@cn.fujitsu.com>
>> Cc: David Vrabel <david.vrabel@citrix.com>
>> Cc: David Rientjes <rientjes@google.com>
>> Cc: Andrew Morton <akpm@linux-foundation.org>
>> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
>> Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
>> Cc: Xishi Qiu <qiuxishi@huawei.com>
>> Cc: Mel Gorman <mgorman@techsingularity.net>
>> Cc: "K. Y. Srinivasan" <kys@microsoft.com>
>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> ---
>> - I was able to find previous attempts to fix the issue, e.g.:
>>   http://marc.info/?l=linux-kernel&m=137425951924598&w=2
>>   http://marc.info/?l=linux-acpi&m=127186488905382
>>   but I'm not completely sure why it didn't work out and the solution
>>   I suggest is not 'smart enough', thus 'RFC'.
>> ---
>>  Documentation/memory-hotplug.txt | 21 +++++++++++++++++----
>>  drivers/base/memory.c            | 35 +++++++++++++++++++++++++++++++++++
>>  include/linux/memory_hotplug.h   |  2 ++
>>  mm/memory_hotplug.c              |  8 ++++++++
>>  4 files changed, 62 insertions(+), 4 deletions(-)
>>
>> diff --git a/Documentation/memory-hotplug.txt b/Documentation/memory-hotplug.txt
>> index ce2cfcf..fe576d9 100644
>> --- a/Documentation/memory-hotplug.txt
>> +++ b/Documentation/memory-hotplug.txt
>> @@ -254,12 +254,25 @@ If the memory block is online, you'll read "online".
>>  If the memory block is offline, you'll read "offline".
>>
>>
>> -5.2. How to online memory
>> +5.2. Memory onlining
>>  ------------
>> -Even if the memory is hot-added, it is not at ready-to-use state.
>> -For using newly added memory, you have to "online" the memory block.
>> +When the memory is hot-added, the kernel decides whether or not to "online"
>> +it according to the policy which can be read from "hotplug_autoonline" file:
>>
>> -For onlining, you have to write "online" to the memory block's state file as:
>> +% cat /sys/devices/system/memory/hotplug_autoonline
>> +
>> +The default is "offline" which means the newly added memory will not be at
>> +ready-to-use state and you have to "online" the newly added memory blocks
>> +manually.
>> +
>> +Automatic onlining can be requested by writing "online" to "hotplug_autoonline"
>> +file:
>> +
>> +% echo online > /sys/devices/system/memory/hotplug_autoonline
>> +
>> +If the automatic onlining wasn't requested or some memory block was offlined
>> +it is possible to change the individual block's state by writing to the "state"
>> +file:
>>
>>  % echo online > /sys/devices/system/memory/memoryXXX/state
>>
>> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
>> index 25425d3..001fefe 100644
>> --- a/drivers/base/memory.c
>> +++ b/drivers/base/memory.c
>> @@ -438,6 +438,40 @@ print_block_size(struct device *dev, struct device_attribute *attr,
>>
>>  static DEVICE_ATTR(block_size_bytes, 0444, print_block_size, NULL);
>>
>> +
>> +/*
>> + * Memory auto online policy.
>> + */
>> +
>> +static ssize_t
>> +show_memhp_autoonline(struct device *dev, struct device_attribute *attr,
>> +		      char *buf)
>> +{
>> +	if (memhp_autoonline == MMOP_ONLINE_KEEP)
>> +		return sprintf(buf, "online\n");
>> +	else if (memhp_autoonline == MMOP_OFFLINE)
>> +		return sprintf(buf, "offline\n");
>> +	else
>> +		return sprintf(buf, "unknown\n");
>
> You do not allow unknown state below, so, I do not know how it can appear
> here. If it appears out of the blue then I think that we should be alert
> because something magic happens around us. Hence, if you wish to leave
> this unknown stuff then I suppose we should at least call WARN_ON() if
> not BUG_ON() there too (well, I am not convinced about latter).
>

Sure,

it is definitelly impossible to see 'unknown' with the current code. The
one (small) reason I added it is that there are two more possible values
for /sys/devices/system/memory/memoryXXX/state: online_movable and
online_kernel. While I don't see how 'online_kernel' can be used for
automatic hotplug (all new memory blocks are being added to ZONE_NORMAL)
'online_movable' could (in theory) be used here so all new memroy blocks
will go to ZONE_MOVABLE. To achieve that someone could set
memhp_autoonline to MMOP_ONLINE_MOVABLE and it will (in theory) work but
I didn't want to add such option to the
/sys/devices/system/memory/hotplug_autoonline interface without a
reason.

I can of course remove this (or replace with WARN_ON) for now.

Thanks,

>> +}
>> +
>> +static ssize_t
>> +store_memhp_autoonline(struct device *dev, struct device_attribute *attr,
>> +		       const char *buf, size_t count)
>> +{
>> +	if (sysfs_streq(buf, "online"))
>> +		memhp_autoonline = MMOP_ONLINE_KEEP;
>> +	else if (sysfs_streq(buf, "offline"))
>> +		memhp_autoonline = MMOP_OFFLINE;
>> +	else
>> +		return -EINVAL;
>
> Here you are not able to set anything which is not allowed.
> So, please look above.
>
> Daniel

-- 
  Vitaly

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH RFC] memory-hotplug: add automatic onlining policy for the newly added memory

[Xishi Qiu]

On 2015/12/16 17:17, Vitaly Kuznetsov wrote:

> Xishi Qiu <qiuxishi@huawei.com> writes:
> 
>> On 2015/12/16 2:05, Vitaly Kuznetsov wrote:
>>
>>> Currently, all newly added memory blocks remain in 'offline' state unless
>>> someone onlines them, some linux distributions carry special udev rules
>>> like:
>>>
>>> SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online"
>>>
>>> to make this happen automatically. This is not a great solution for virtual
>>> machines where memory hotplug is being used to address high memory pressure
>>> situations as such onlining is slow and a userspace process doing this
>>> (udev) has a chance of being killed by the OOM killer as it will probably
>>> require to allocate some memory.
>>>
>>> Introduce default policy for the newly added memory blocks in
>>> /sys/devices/system/memory/hotplug_autoonline file with two possible
>>> values: "offline" (the default) which preserves the current behavior and
>>> "online" which causes all newly added memory blocks to go online as
>>> soon as they're added.
>>>
>>> Cc: Jonathan Corbet <corbet@lwn.net>
>>> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>>> Cc: Daniel Kiper <daniel.kiper@oracle.com>
>>> Cc: Dan Williams <dan.j.williams@intel.com>
>>> Cc: Tang Chen <tangchen@cn.fujitsu.com>
>>> Cc: David Vrabel <david.vrabel@citrix.com>
>>> Cc: David Rientjes <rientjes@google.com>
>>> Cc: Andrew Morton <akpm@linux-foundation.org>
>>> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
>>> Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
>>> Cc: Xishi Qiu <qiuxishi@huawei.com>
>>> Cc: Mel Gorman <mgorman@techsingularity.net>
>>> Cc: "K. Y. Srinivasan" <kys@microsoft.com>
>>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>>> ---
>>> - I was able to find previous attempts to fix the issue, e.g.:
>>>   http://marc.info/?l=linux-kernel&m=137425951924598&w=2
>>>   http://marc.info/?l=linux-acpi&m=127186488905382
>>>   but I'm not completely sure why it didn't work out and the solution
>>>   I suggest is not 'smart enough', thus 'RFC'.
>>
>> + CC: 
>> yanxiaofeng@inspur.com
>> liuchangsheng@inspur.com
>>
>> Hi Vitaly,
>>
>> Why not use udev rule? I think it can online pages automatically.
>>
> 
> Two main reasons:
> 1) I remember someone saying "You never need a mouse in order to add
> another mouse to the kernel" -- but we  we need memory to add more
> memory. Udev has a chance of being killed by the OOM killer as
> performing an action will probably require to allocate some
> memory. Other than that udev actions are generally slow compared to what
> we can do in kernel.

Hi Vitaly,

So why we add memory when there is almost no free memory left?
I think the administrator should add memory when the free memory is low
or he should do something to stop free memory become worse.

> 
> 2) I agree with Kay that '... unconditional hotplug loop through
> userspace is absolutely pointless' (https://lkml.org/lkml/2013/7/25/354). 
> (... and I should had add him to CC, adding now). Udev maintainers
> refused to add a rule for unconditional memory onlining to udev and now
> linux distros have to carry such custom rules.
> 

If the administrator don't know how to config the udev, he could use sysfs
(echo 1 > /sys/devices/system/node/nodeXX/memoryXX/online) to online it,
or write a script to do this.

Thanks,
Xishi Qiu


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH RFC] memory-hotplug: add automatic onlining policy for the newly added memory

[Vitaly Kuznetsov]

Xishi Qiu <qiuxishi@huawei.com> writes:

> On 2015/12/16 17:17, Vitaly Kuznetsov wrote:
>
>> Xishi Qiu <qiuxishi@huawei.com> writes:
>> 
>>> On 2015/12/16 2:05, Vitaly Kuznetsov wrote:
>>>
>>>> Currently, all newly added memory blocks remain in 'offline' state unless
>>>> someone onlines them, some linux distributions carry special udev rules
>>>> like:
>>>>
>>>> SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online"
>>>>
>>>> to make this happen automatically. This is not a great solution for virtual
>>>> machines where memory hotplug is being used to address high memory pressure
>>>> situations as such onlining is slow and a userspace process doing this
>>>> (udev) has a chance of being killed by the OOM killer as it will probably
>>>> require to allocate some memory.
>>>>
>>>> Introduce default policy for the newly added memory blocks in
>>>> /sys/devices/system/memory/hotplug_autoonline file with two possible
>>>> values: "offline" (the default) which preserves the current behavior and
>>>> "online" which causes all newly added memory blocks to go online as
>>>> soon as they're added.
>>>>
>>>> Cc: Jonathan Corbet <corbet@lwn.net>
>>>> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>>>> Cc: Daniel Kiper <daniel.kiper@oracle.com>
>>>> Cc: Dan Williams <dan.j.williams@intel.com>
>>>> Cc: Tang Chen <tangchen@cn.fujitsu.com>
>>>> Cc: David Vrabel <david.vrabel@citrix.com>
>>>> Cc: David Rientjes <rientjes@google.com>
>>>> Cc: Andrew Morton <akpm@linux-foundation.org>
>>>> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
>>>> Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
>>>> Cc: Xishi Qiu <qiuxishi@huawei.com>
>>>> Cc: Mel Gorman <mgorman@techsingularity.net>
>>>> Cc: "K. Y. Srinivasan" <kys@microsoft.com>
>>>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>>>> ---
>>>> - I was able to find previous attempts to fix the issue, e.g.:
>>>>   http://marc.info/?l=linux-kernel&m=137425951924598&w=2
>>>>   http://marc.info/?l=linux-acpi&m=127186488905382
>>>>   but I'm not completely sure why it didn't work out and the solution
>>>>   I suggest is not 'smart enough', thus 'RFC'.
>>>
>>> + CC: 
>>> yanxiaofeng@inspur.com
>>> liuchangsheng@inspur.com
>>>
>>> Hi Vitaly,
>>>
>>> Why not use udev rule? I think it can online pages automatically.
>>>
>> 
>> Two main reasons:
>> 1) I remember someone saying "You never need a mouse in order to add
>> another mouse to the kernel" -- but we  we need memory to add more
>> memory. Udev has a chance of being killed by the OOM killer as
>> performing an action will probably require to allocate some
>> memory. Other than that udev actions are generally slow compared to what
>> we can do in kernel.
>
> Hi Vitaly,
>
> So why we add memory when there is almost no free memory left?
> I think the administrator should add memory when the free memory is low
> or he should do something to stop free memory become worse.

I have virtual machines use-case in my mind where hypervisor adds new
memory on high memory pressure reports from the guest (e.g. Hyper-V
behaves like that). This is an automatic action.

>
>> 
>> 2) I agree with Kay that '... unconditional hotplug loop through
>> userspace is absolutely pointless' (https://lkml.org/lkml/2013/7/25/354). 
>> (... and I should had add him to CC, adding now). Udev maintainers
>> refused to add a rule for unconditional memory onlining to udev and now
>> linux distros have to carry such custom rules.
>> 
>
> If the administrator don't know how to config the udev, he could use sysfs
> (echo 1 > /sys/devices/system/node/nodeXX/memoryXX/online) to online it,
> or write a script to do this.

Oh, no, I'm not taking about manual actions here. My suggestion doesn't
eliminate this possibility and it doesn't even change the default --
memory blocks stay in 'offline' state unless someone requests the
auto-online policy.

-- 
  Vitaly

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>