All of lore.kernel.org
 help / color / mirror / Atom feed
From: Yasunori Goto <y-goto@jp.fujitsu.com>
To: Linux Kernel ML <linux-kernel@vger.kernel.org>,
	linux-mm <linux-mm@kvack.org>
Cc: Andrew Morton <akpm@osdl.org>,
	Christoph Lameter <clameter@sgi.com>, Mel Gorman <mel@csn.ul.ie>
Subject: [RFC] memory hotremove patch take 2 [00/10]
Date: Wed, 09 May 2007 11:59:38 +0900	[thread overview]
Message-ID: <20070509115506.B904.Y-GOTO@jp.fujitsu.com> (raw)


Hello.

I rebased and debugged Kame-san's memory hot-remove patches.
This work is not finished yet. (Some pages keep un-removable status)
But, I would like to show current progress of it, because it has
been a long time since previous post, and some bugs are fixed.

If you have concern, please check this. Any comments are welcome.

Thanks.

---

These patches are for memory hot-remove.

How to use
  - kernelcore=xx[GMK] must be specified at boottime option to create
    ZONE_MOVABLE area.
  - After bootup, execute following.
     # echo "offline" > /sys/devices/system/memory/memoryX/status
    


Change log from previous version.
  - Rebase to 2.6.21-mm1.
  - Old original ZONE_MOVABLE code is removed. Mel-san's ZONE_REMOVABLE
    for anti-fragmentation is used.
  - Fix wrong return code check of isolate_lru_page()
  - Page is isolated ASAP, which was source of page migration when
    memory-hotremove. In old code, it uses just put_page(),
    and we expected that migrated source page is catched in
    __free_one_page() as isolated page. But, it is spooled in
    per_cpu_page and used soon for next destination page of migration.
    This was cause of eternal loop in offline_pages().
  - There is a page which is not mapped but added to swapcache in
    swap-in code. It was cause of panic in try_to_unmap(). fixed it.
  - end_pfn is rounded up at memmap_init. If there is a small hole on
    end of section. These page is not initialized.

TODO:
  - There are some pages which are un-removable page on memory stress
    condition. (These pages are set PG_swapcache or PG_mappedtodisk
    without connecting to lru.)
  - Should make i386/x86-64/powerpc interface code. But not yet 
    (really sorry :-( ).
  - If bootmem parameter or efi's memory map is stored by efi, memory
    can't be removed even if it is in removable zone.
  - node hotplug support. (this may needs some amount of patches.)
  - test under heavy work load and more careful race check.
  - Fix where we should allocate migration target page from.
  - Hmmmm.... And so on.

[1] counters patch -- per-zone counter for ZONE_MOVABLE

==page isolation==
[2] page isolation patch ..... basic defintions of page isolation.
[3] drain_all_zone_pages patch ..... drain all cpus' pcp pages.
[4] isolate freed page patch ..... isolate pages in free_area[]

==memory unplug==
offline a section of pages. isolate specified section and migrate
content of used pages to out of section. (Because free pages in a
section is isolated, it never be returned by alloc_pages())
This patch doesn't care where we should allocate migration new pages from.
[5] memory unplug core patch --- maybe need more work.
[6] interface patch          --- "offline" interface support 

==migration nocontext==
Fix race condition of page migration without process context
(not taking mm->sem). This patch delayes kmem_cache_free() of
anon_vma until migration ends.
[7] migration nocontext patch --- support page migration without
    acquiring mm->sem. need careful debug...

==other fixes==
[8] round up end_pfn at memmap_init
[9] page isolation ASAP when memory-hotremove case.
[10] fix swapping-in page panic.

-- 
Yasunori Goto 



WARNING: multiple messages have this Message-ID (diff)
From: Yasunori Goto <y-goto@jp.fujitsu.com>
To: Linux Kernel ML <linux-kernel@vger.kernel.org>,
	linux-mm <linux-mm@kvack.org>
Cc: Andrew Morton <akpm@osdl.org>,
	Christoph Lameter <clameter@sgi.com>, Mel Gorman <mel@csn.ul.ie>
Subject: [RFC] memory hotremove patch take 2 [00/10]
Date: Wed, 09 May 2007 11:59:38 +0900	[thread overview]
Message-ID: <20070509115506.B904.Y-GOTO@jp.fujitsu.com> (raw)

Hello.

I rebased and debugged Kame-san's memory hot-remove patches.
This work is not finished yet. (Some pages keep un-removable status)
But, I would like to show current progress of it, because it has
been a long time since previous post, and some bugs are fixed.

If you have concern, please check this. Any comments are welcome.

Thanks.

---

These patches are for memory hot-remove.

How to use
  - kernelcore=xx[GMK] must be specified at boottime option to create
    ZONE_MOVABLE area.
  - After bootup, execute following.
     # echo "offline" > /sys/devices/system/memory/memoryX/status
    


Change log from previous version.
  - Rebase to 2.6.21-mm1.
  - Old original ZONE_MOVABLE code is removed. Mel-san's ZONE_REMOVABLE
    for anti-fragmentation is used.
  - Fix wrong return code check of isolate_lru_page()
  - Page is isolated ASAP, which was source of page migration when
    memory-hotremove. In old code, it uses just put_page(),
    and we expected that migrated source page is catched in
    __free_one_page() as isolated page. But, it is spooled in
    per_cpu_page and used soon for next destination page of migration.
    This was cause of eternal loop in offline_pages().
  - There is a page which is not mapped but added to swapcache in
    swap-in code. It was cause of panic in try_to_unmap(). fixed it.
  - end_pfn is rounded up at memmap_init. If there is a small hole on
    end of section. These page is not initialized.

TODO:
  - There are some pages which are un-removable page on memory stress
    condition. (These pages are set PG_swapcache or PG_mappedtodisk
    without connecting to lru.)
  - Should make i386/x86-64/powerpc interface code. But not yet 
    (really sorry :-( ).
  - If bootmem parameter or efi's memory map is stored by efi, memory
    can't be removed even if it is in removable zone.
  - node hotplug support. (this may needs some amount of patches.)
  - test under heavy work load and more careful race check.
  - Fix where we should allocate migration target page from.
  - Hmmmm.... And so on.

[1] counters patch -- per-zone counter for ZONE_MOVABLE

==page isolation==
[2] page isolation patch ..... basic defintions of page isolation.
[3] drain_all_zone_pages patch ..... drain all cpus' pcp pages.
[4] isolate freed page patch ..... isolate pages in free_area[]

==memory unplug==
offline a section of pages. isolate specified section and migrate
content of used pages to out of section. (Because free pages in a
section is isolated, it never be returned by alloc_pages())
This patch doesn't care where we should allocate migration new pages from.
[5] memory unplug core patch --- maybe need more work.
[6] interface patch          --- "offline" interface support 

==migration nocontext==
Fix race condition of page migration without process context
(not taking mm->sem). This patch delayes kmem_cache_free() of
anon_vma until migration ends.
[7] migration nocontext patch --- support page migration without
    acquiring mm->sem. need careful debug...

==other fixes==
[8] round up end_pfn at memmap_init
[9] page isolation ASAP when memory-hotremove case.
[10] fix swapping-in page panic.

-- 
Yasunori Goto 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

             reply	other threads:[~2007-05-09  3:01 UTC|newest]

Thread overview: 60+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-05-09  2:59 Yasunori Goto [this message]
2007-05-09  2:59 ` [RFC] memory hotremove patch take 2 [00/10] Yasunori Goto
2007-05-09  3:10 ` [RFC] memory hotremove patch take 2 [01/10] (counter of removable page) Yasunori Goto
2007-05-09  3:10   ` Yasunori Goto
2007-05-10 12:46   ` Mel Gorman
2007-05-10 12:46     ` Mel Gorman
2007-05-10 13:44   ` Andi Kleen
2007-05-10 13:44     ` Andi Kleen
2007-05-11  1:08     ` KAMEZAWA Hiroyuki
2007-05-11  1:08       ` KAMEZAWA Hiroyuki
2007-05-10 18:00   ` Christoph Lameter
2007-05-10 18:00     ` Christoph Lameter
2007-05-11  1:09     ` KAMEZAWA Hiroyuki
2007-05-11  1:09       ` KAMEZAWA Hiroyuki
2007-05-09  3:10 ` [RFC] memory hotremove patch take 2 [02/10] (make page unused) Yasunori Goto
2007-05-09  3:10   ` Yasunori Goto
2007-05-10 15:34   ` Mel Gorman
2007-05-10 15:34     ` Mel Gorman
2007-05-11  0:47     ` KAMEZAWA Hiroyuki
2007-05-11  0:47       ` KAMEZAWA Hiroyuki
2007-05-10 18:04   ` Christoph Lameter
2007-05-10 18:04     ` Christoph Lameter
2007-05-11  0:49     ` KAMEZAWA Hiroyuki
2007-05-11  0:49       ` KAMEZAWA Hiroyuki
2007-05-09  3:10 ` [RFC] memory hotremove patch take 2 [03/10] (drain all pages) Yasunori Goto
2007-05-09  3:10   ` Yasunori Goto
2007-05-10 15:35   ` Mel Gorman
2007-05-10 15:35     ` Mel Gorman
2007-05-11  0:53     ` KAMEZAWA Hiroyuki
2007-05-11  0:53       ` KAMEZAWA Hiroyuki
2007-05-10 18:07   ` Christoph Lameter
2007-05-10 18:07     ` Christoph Lameter
2007-05-11  0:54     ` KAMEZAWA Hiroyuki
2007-05-11  0:54       ` KAMEZAWA Hiroyuki
2007-05-09  3:10 ` [RFC] memory hotremove patch take 2 [04/10] (isolate all free pages) Yasunori Goto
2007-05-09  3:10   ` Yasunori Goto
2007-05-10 16:42   ` Mel Gorman
2007-05-10 16:42     ` Mel Gorman
2007-05-11  0:58     ` KAMEZAWA Hiroyuki
2007-05-11  0:58       ` KAMEZAWA Hiroyuki
2007-05-10 18:07   ` Christoph Lameter
2007-05-10 18:07     ` Christoph Lameter
2007-05-09  3:11 ` [RFC] memory hotremove patch take 2 [05/10] (make basic remove code) Yasunori Goto
2007-05-09  3:11   ` Yasunori Goto
2007-05-10 18:09   ` Christoph Lameter
2007-05-10 18:09     ` Christoph Lameter
2007-05-11  1:05     ` KAMEZAWA Hiroyuki
2007-05-11  1:05       ` KAMEZAWA Hiroyuki
2007-05-09  3:11 ` [RFC] memory hotremove patch take 2 [06/10] (ia64's remove_memory code) Yasunori Goto
2007-05-09  3:11   ` Yasunori Goto
2007-05-09  3:11 ` [RFC] memory hotremove patch take 2 [07/10] (delay freeing anon_vma) Yasunori Goto
2007-05-09  3:11   ` Yasunori Goto
2007-05-09  3:12 ` [RFC] memory hotremove patch take 2 [08/10] (memap init alignment) Yasunori Goto
2007-05-09  3:12   ` Yasunori Goto
2007-05-09  3:12 ` [RFC] memory hotremove patch take 2 [09/10] (direct isolation for remove) Yasunori Goto
2007-05-09  3:12   ` Yasunori Goto
2007-05-09  3:12 ` [RFC] memory hotremove patch take 2 [10/10] (retry swap-in page) Yasunori Goto
2007-05-09  3:12   ` Yasunori Goto
2007-05-09  3:26   ` KAMEZAWA Hiroyuki
2007-05-09  3:26     ` KAMEZAWA Hiroyuki

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20070509115506.B904.Y-GOTO@jp.fujitsu.com \
    --to=y-goto@jp.fujitsu.com \
    --cc=akpm@osdl.org \
    --cc=clameter@sgi.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mel@csn.ul.ie \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.