All of lore.kernel.org
 help / color / mirror / Atom feed
From: Tang Chen <tangchen@cn.fujitsu.com>
To: Vasilis Liaskovitis <vasilis.liaskovitis@profitbricks.com>
Cc: mingo@redhat.com, hpa@zytor.com, akpm@linux-foundation.org,
	yinghai@kernel.org, jiang.liu@huawei.com, wency@cn.fujitsu.com,
	isimatu.yasuaki@jp.fujitsu.com, tj@kernel.org,
	laijs@cn.fujitsu.com, davem@davemloft.net, mgorman@suse.de,
	minchan@kernel.org, mina86@mina86.com, x86@kernel.org,
	linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-mm@kvack.org
Subject: Re: [PATCH v2 10/13] x86, acpi, numa, mem-hotplug: Introduce MEMBLK_HOTPLUGGABLE to mark and reserve hotpluggable memory.
Date: Mon, 06 May 2013 10:27:44 +0800	[thread overview]
Message-ID: <51871520.6020703@cn.fujitsu.com> (raw)
In-Reply-To: <20130503105037.GA4533@dhcp-192-168-178-175.profitbricks.localdomain>

Hi Vasilis,

Sorry for the delay and thank you for reviewing and testing. :)

On 05/03/2013 06:50 PM, Vasilis Liaskovitis wrote:
>
> Should we skip ranges on nodes that the kernel uses? e.g. with
>
>          if (memblock_is_kernel_node(nid))
>              continue;

Yes. I think I forgot to call it in this patch.
Will update in the next version.

>
>
> - I am getting a "PANIC: early exception" when rebooting with movablecore=acpi
> after hotplugging memory on node0 or node1 of a 2-node VM. The guest kernel is
> based on
> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git
> for-x86-mm (e9058baf) + these v2 patches.
>
> This happens with or without the above memblock_is_kernel_node(nid) check.
> Perhaps I am missing something or I need a newer "ACPI, numa: Parse numa info
> early" patch-set?

I didn't test it on a VM. But on my real box, I haven't got a panic
when rebooting. I think I can help to test it in a VM, but would you 
please to
tell me how to setup a environment as yours ?

>
> A general question: Disabling hot-pluggability/zone-movable eligibility for a
> whole node sounds a bit inflexible, if the machine only has one node to begin
> with.  Would it be possible to keep movable information per SRAT entry? I.e
> if the BIOS presents multiple SRAT entries for one node/PXM (say node 0), and
> there is no memblock/kernel allocation on one of these SRAT entries, could
> we still mark this SRAT entry's range as hot-pluggable/movable?  Not sure if
> many real machine BIOSes would do this, but seabios could.  This implies that
> SRAT entries are processed for movable-zone eligilibity before they are merged
> on node/PXM basis entry-granularity (I think numa_cleanup_meminfo currently does
> this merge).

Yes, this can be done. But in real usage, part of the memory in a node
is hot-removable makes no sense, I think. We cannot remove the whole node,
so we cannot remove a real hardware device.

But in virtualization, would you please give a reason why we need this
entry-granularity ?


Another thinking. Assume I didn't understand your question correctly. :)

Now in kernel, we can recognize a node (by PXM in SRAT), but we cannot
recognize a memory device. Are you saying if we have this 
entry-granularity,
we can hotplug a single memory device in a node ? (Perhaps there are more
than on memory device in a node.)

If so, it makes sense. But I don't the kernel is able to recognize which
device a memory range belongs to now. And I'm not sure if we can do this.

>
> Of course the kernel should still have enough memory(i.e. non movable zone) to
> boot. Can we ensure that at least certain amount of memory is non-movable, and
> then, given more separate SRAT entries for node0 not used by kernel, treat
> these rest entries as movable?

I tried this idea before. But as HPA said, it seems no way to calculate 
how much
memory the kernel needs.
https://lkml.org/lkml/2012/11/27/29


Thanks. :)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Tang Chen <tangchen@cn.fujitsu.com>
To: Vasilis Liaskovitis <vasilis.liaskovitis@profitbricks.com>
Cc: mingo@redhat.com, hpa@zytor.com, akpm@linux-foundation.org,
	yinghai@kernel.org, jiang.liu@huawei.com, wency@cn.fujitsu.com,
	isimatu.yasuaki@jp.fujitsu.com, tj@kernel.org,
	laijs@cn.fujitsu.com, davem@davemloft.net, mgorman@suse.de,
	minchan@kernel.org, mina86@mina86.com, x86@kernel.org,
	linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-mm@kvack.org
Subject: Re: [PATCH v2 10/13] x86, acpi, numa, mem-hotplug: Introduce MEMBLK_HOTPLUGGABLE to mark and reserve hotpluggable memory.
Date: Mon, 06 May 2013 10:27:44 +0800	[thread overview]
Message-ID: <51871520.6020703@cn.fujitsu.com> (raw)
In-Reply-To: <20130503105037.GA4533@dhcp-192-168-178-175.profitbricks.localdomain>

Hi Vasilis,

Sorry for the delay and thank you for reviewing and testing. :)

On 05/03/2013 06:50 PM, Vasilis Liaskovitis wrote:
>
> Should we skip ranges on nodes that the kernel uses? e.g. with
>
>          if (memblock_is_kernel_node(nid))
>              continue;

Yes. I think I forgot to call it in this patch.
Will update in the next version.

>
>
> - I am getting a "PANIC: early exception" when rebooting with movablecore=acpi
> after hotplugging memory on node0 or node1 of a 2-node VM. The guest kernel is
> based on
> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git
> for-x86-mm (e9058baf) + these v2 patches.
>
> This happens with or without the above memblock_is_kernel_node(nid) check.
> Perhaps I am missing something or I need a newer "ACPI, numa: Parse numa info
> early" patch-set?

I didn't test it on a VM. But on my real box, I haven't got a panic
when rebooting. I think I can help to test it in a VM, but would you 
please to
tell me how to setup a environment as yours ?

>
> A general question: Disabling hot-pluggability/zone-movable eligibility for a
> whole node sounds a bit inflexible, if the machine only has one node to begin
> with.  Would it be possible to keep movable information per SRAT entry? I.e
> if the BIOS presents multiple SRAT entries for one node/PXM (say node 0), and
> there is no memblock/kernel allocation on one of these SRAT entries, could
> we still mark this SRAT entry's range as hot-pluggable/movable?  Not sure if
> many real machine BIOSes would do this, but seabios could.  This implies that
> SRAT entries are processed for movable-zone eligilibity before they are merged
> on node/PXM basis entry-granularity (I think numa_cleanup_meminfo currently does
> this merge).

Yes, this can be done. But in real usage, part of the memory in a node
is hot-removable makes no sense, I think. We cannot remove the whole node,
so we cannot remove a real hardware device.

But in virtualization, would you please give a reason why we need this
entry-granularity ?


Another thinking. Assume I didn't understand your question correctly. :)

Now in kernel, we can recognize a node (by PXM in SRAT), but we cannot
recognize a memory device. Are you saying if we have this 
entry-granularity,
we can hotplug a single memory device in a node ? (Perhaps there are more
than on memory device in a node.)

If so, it makes sense. But I don't the kernel is able to recognize which
device a memory range belongs to now. And I'm not sure if we can do this.

>
> Of course the kernel should still have enough memory(i.e. non movable zone) to
> boot. Can we ensure that at least certain amount of memory is non-movable, and
> then, given more separate SRAT entries for node0 not used by kernel, treat
> these rest entries as movable?

I tried this idea before. But as HPA said, it seems no way to calculate 
how much
memory the kernel needs.
https://lkml.org/lkml/2012/11/27/29


Thanks. :)


  reply	other threads:[~2013-05-06  2:24 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-04-30  9:21 [PATCH v2 00/13] Arrange hotpluggable memory in SRAT as ZONE_MOVABLE Tang Chen
2013-04-30  9:21 ` Tang Chen
2013-04-30  9:21 ` [PATCH v2 01/13] x86: get pg_data_t's memory from other node Tang Chen
2013-04-30  9:21   ` Tang Chen
2013-05-22  8:55   ` Chen Gong
2013-05-22  9:24     ` Tang Chen
2013-05-22  9:24       ` Tang Chen
2013-04-30  9:21 ` [PATCH v2 02/13] acpi: Print Hot-Pluggable Field in SRAT Tang Chen
2013-04-30  9:21   ` Tang Chen
2013-04-30  9:21 ` [PATCH v2 03/13] page_alloc, mem-hotplug: Improve movablecore to {en|dis}able using SRAT Tang Chen
2013-04-30  9:21   ` Tang Chen
2013-04-30  9:21 ` [PATCH v2 04/13] x86, numa, acpi, memory-hotplug: Introduce hotplug info into struct numa_meminfo Tang Chen
2013-04-30  9:21   ` Tang Chen
2013-04-30  9:21 ` [PATCH v2 05/13] x86, numa, acpi, memory-hotplug: Consider hotplug info when cleanup numa_meminfo Tang Chen
2013-04-30  9:21   ` Tang Chen
2013-04-30  9:21 ` [PATCH v2 06/13] memblock, numa: Introduce flag into memblock Tang Chen
2013-04-30  9:21   ` Tang Chen
2013-04-30  9:21 ` [PATCH v2 07/13] x86, numa, mem-hotplug: Mark nodes which the kernel resides in Tang Chen
2013-04-30  9:21   ` Tang Chen
2013-05-31 16:15   ` Vasilis Liaskovitis
2013-05-31 16:15     ` Vasilis Liaskovitis
2013-05-31 16:25     ` Vasilis Liaskovitis
2013-05-31 16:25       ` Vasilis Liaskovitis
2013-04-30  9:21 ` [PATCH v2 08/13] x86, numa: Move memory_add_physaddr_to_nid() to CONFIG_NUMA Tang Chen
2013-04-30  9:21   ` Tang Chen
2013-04-30  9:21 ` [PATCH v2 09/13] x86, numa, memblock: Introduce MEMBLK_LOCAL_NODE to mark and reserve node-life-cycle data Tang Chen
2013-04-30  9:21   ` Tang Chen
2013-04-30  9:21 ` [PATCH v2 10/13] x86, acpi, numa, mem-hotplug: Introduce MEMBLK_HOTPLUGGABLE to mark and reserve hotpluggable memory Tang Chen
2013-04-30  9:21   ` Tang Chen
2013-05-03 10:50   ` Vasilis Liaskovitis
2013-05-03 10:50     ` Vasilis Liaskovitis
2013-05-06  2:27     ` Tang Chen [this message]
2013-05-06  2:27       ` Tang Chen
2013-05-06 10:37       ` Vasilis Liaskovitis
2013-05-06 10:37         ` Vasilis Liaskovitis
2013-05-07  2:16         ` Tang Chen
2013-05-07  2:16           ` Tang Chen
2013-04-30  9:21 ` [PATCH v2 11/13] x86, memblock, mem-hotplug: Free hotpluggable memory reserved by memblock Tang Chen
2013-04-30  9:21   ` Tang Chen
2013-04-30  9:21 ` [PATCH v2 12/13] x86, numa, acpi, memory-hotplug: Make movablecore=acpi have higher priority Tang Chen
2013-04-30  9:21   ` Tang Chen
2013-05-22  4:43   ` Tang Chen
2013-05-22  4:43     ` Tang Chen
2013-04-30  9:21 ` [PATCH v2 13/13] doc, page_alloc, acpi, mem-hotplug: Add doc for movablecore=acpi boot option Tang Chen
2013-04-30  9:21   ` Tang Chen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=51871520.6020703@cn.fujitsu.com \
    --to=tangchen@cn.fujitsu.com \
    --cc=akpm@linux-foundation.org \
    --cc=davem@davemloft.net \
    --cc=hpa@zytor.com \
    --cc=isimatu.yasuaki@jp.fujitsu.com \
    --cc=jiang.liu@huawei.com \
    --cc=laijs@cn.fujitsu.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@suse.de \
    --cc=mina86@mina86.com \
    --cc=minchan@kernel.org \
    --cc=mingo@redhat.com \
    --cc=tj@kernel.org \
    --cc=vasilis.liaskovitis@profitbricks.com \
    --cc=wency@cn.fujitsu.com \
    --cc=x86@kernel.org \
    --cc=yinghai@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.