[RFC][PATCH] migrate_pages:skip migration between intersect nodes

[RFC][PATCH] migrate_pages:skip migration between intersect nodes

[Bob Liu]

In current do_migrate_pages(),if from_nodes and to_nodes have some
intersect nodes,pages in these intersect nodes will also be
migrated.
eg. Assume that, from_nodes: 1,2,3,4 to_nodes: 2,3,4,5. Then these
migrates will happen:
migrate_pages(4,5);
migrate_pages(3,4);
migrate_pages(2,3);
migrate_pages(1,2);

But the user just want all pages in from_nodes move to to_nodes,
only migrate(1,2)(ignore the intersect nodes.) can satisfied 
the user's request.

I amn't sure what's migrate_page's semantic.
Hoping for your suggestions.

Signed-off-by: Bob Liu <lliubbo@gmail.com>
---
 mm/mempolicy.c |    7 ++-----
 1 files changed, 2 insertions(+), 5 deletions(-)

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 08f40a2..c6dd931 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -922,7 +922,7 @@ int do_migrate_pages(struct mm_struct *mm,
 	 * moved to an empty node, then there is nothing left worth migrating.
 	 */
 
-	tmp = *from_nodes;
+	nodes_andnot(tmp, *from_nodes, *to_nodes);
 	while (!nodes_empty(tmp)) {
 		int s,d;
 		int source = -1;
@@ -935,10 +935,7 @@ int do_migrate_pages(struct mm_struct *mm,
 
 			source = s;	/* Node moved. Memorize */
 			dest = d;
-
-			/* dest not in remaining from nodes? */
-			if (!node_isset(dest, tmp))
-				break;
+			break;
 		}
 		if (source == -1)
 			break;
-- 
1.5.6.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] migrate_pages:skip migration between intersect nodes

[Lee Schermerhorn]

On Mon, 2010-03-29 at 22:57 +0800, Bob Liu wrote:
> In current do_migrate_pages(),if from_nodes and to_nodes have some
> intersect nodes,pages in these intersect nodes will also be
> migrated.
> eg. Assume that, from_nodes: 1,2,3,4 to_nodes: 2,3,4,5. Then these
> migrates will happen:
> migrate_pages(4,5);
> migrate_pages(3,4);
> migrate_pages(2,3);
> migrate_pages(1,2);
> 
> But the user just want all pages in from_nodes move to to_nodes,
> only migrate(1,2)(ignore the intersect nodes.) can satisfied 
> the user's request.
> 
> I amn't sure what's migrate_page's semantic.
> Hoping for your suggestions.

I believe that the current code matches the intended semantics.  I can't
find a man pages for the migrate_pages() system call, but the
migratepages(8) man page says:

"If  multiple  nodes  are specified for from-nodes or to-nodes then an
attempt is made to preserve the relative location of each page in each
nodeset."

If one just wanted to move all task pages from node 1 to 2, one would
specify:

to_nodes: 1
from_nodes: 2

Christoph L may have more to say.  I believe he wrote migrate_pages()
and that very interesting remapping function.

Lee


> 
> Signed-off-by: Bob Liu <lliubbo@gmail.com>
> ---
>  mm/mempolicy.c |    7 ++-----
>  1 files changed, 2 insertions(+), 5 deletions(-)
> 
> diff --git a/mm/mempolicy.c b/mm/mempolicy.c
> index 08f40a2..c6dd931 100644
> --- a/mm/mempolicy.c
> +++ b/mm/mempolicy.c
> @@ -922,7 +922,7 @@ int do_migrate_pages(struct mm_struct *mm,
>  	 * moved to an empty node, then there is nothing left worth migrating.
>  	 */
>  
> -	tmp = *from_nodes;
> +	nodes_andnot(tmp, *from_nodes, *to_nodes);
>  	while (!nodes_empty(tmp)) {
>  		int s,d;
>  		int source = -1;
> @@ -935,10 +935,7 @@ int do_migrate_pages(struct mm_struct *mm,
>  
>  			source = s;	/* Node moved. Memorize */
>  			dest = d;
> -
> -			/* dest not in remaining from nodes? */
> -			if (!node_isset(dest, tmp))
> -				break;
> +			break;
>  		}
>  		if (source == -1)
>  			break;

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] migrate_pages:skip migration between intersect nodes

[KOSAKI Motohiro]

> I believe that the current code matches the intended semantics.  I can't
> find a man pages for the migrate_pages() system call, but the
> migratepages(8) man page says:
> 
> "If  multiple  nodes  are specified for from-nodes or to-nodes then an
> attempt is made to preserve the relative location of each page in each
> nodeset."

Offtopic>
Christoph, Why migrate_pages(2) doesn't have man pages? Is it unrecommended
syscall?



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] migrate_pages:skip migration between intersect nodes

[Christoph Lameter]

On Tue, 30 Mar 2010, KOSAKI Motohiro wrote:

> > I believe that the current code matches the intended semantics.  I can't
> > find a man pages for the migrate_pages() system call, but the
> > migratepages(8) man page says:
> >
> > "If  multiple  nodes  are specified for from-nodes or to-nodes then an
> > attempt is made to preserve the relative location of each page in each
> > nodeset."
>
> Offtopic>
> Christoph, Why migrate_pages(2) doesn't have man pages? Is it unrecommended
> syscall?

The manpage is in the numatools package3.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] migrate_pages:skip migration between intersect nodes

[Andi Kleen]

On Tue, Mar 30, 2010 at 11:28:21AM -0500, Christoph Lameter wrote:
> On Tue, 30 Mar 2010, KOSAKI Motohiro wrote:
> 
> > > I believe that the current code matches the intended semantics.  I can't
> > > find a man pages for the migrate_pages() system call, but the
> > > migratepages(8) man page says:
> > >
> > > "If  multiple  nodes  are specified for from-nodes or to-nodes then an
> > > attempt is made to preserve the relative location of each page in each
> > > nodeset."
> >
> > Offtopic>
> > Christoph, Why migrate_pages(2) doesn't have man pages? Is it unrecommended
> > syscall?
> 
> The manpage is in the numatools package3.

I sent the manpage to the manpage maintainer some time ago, but he has a 
turnaround time of several months and dropped that update with the last
release.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] migrate_pages:skip migration between intersect nodes

[Minchan Kim]

Hi, Bob

On Mon, Mar 29, 2010 at 11:57 PM, Bob Liu <lliubbo@gmail.com> wrote:
> In current do_migrate_pages(),if from_nodes and to_nodes have some
> intersect nodes,pages in these intersect nodes will also be
> migrated.
> eg. Assume that, from_nodes: 1,2,3,4 to_nodes: 2,3,4,5. Then these
> migrates will happen:
> migrate_pages(4,5);
> migrate_pages(3,4);
> migrate_pages(2,3);
> migrate_pages(1,2);
>
> But the user just want all pages in from_nodes move to to_nodes,
> only migrate(1,2)(ignore the intersect nodes.) can satisfied
> the user's request.
>
> I amn't sure what's migrate_page's semantic.
> Hoping for your suggestions.

I didn't see 8:migratepages Lee pointed at that time.
The description matches current migrate_pages's behavior exactly.

I agree Lee's opinion.
Let's wait Christoph's reply what is semantic
and why it doesn't have man page.


-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] migrate_pages:skip migration between intersect nodes

[Christoph Lameter]

On Tue, 30 Mar 2010, Minchan Kim wrote:

> Hi, Bob
>
> On Mon, Mar 29, 2010 at 11:57 PM, Bob Liu <lliubbo@gmail.com> wrote:
> > In current do_migrate_pages(),if from_nodes and to_nodes have some
> > intersect nodes,pages in these intersect nodes will also be
> > migrated.
> > eg. Assume that, from_nodes: 1,2,3,4 to_nodes: 2,3,4,5. Then these
> > migrates will happen:
> > migrate_pages(4,5);
> > migrate_pages(3,4);
> > migrate_pages(2,3);
> > migrate_pages(1,2);
> >
> > But the user just want all pages in from_nodes move to to_nodes,
> > only migrate(1,2)(ignore the intersect nodes.) can satisfied
> > the user's request.
> >
> > I amn't sure what's migrate_page's semantic.
> > Hoping for your suggestions.
>
> I didn't see 8:migratepages Lee pointed at that time.
> The description matches current migrate_pages's behavior exactly.
>
> I agree Lee's opinion.
> Let's wait Christoph's reply what is semantic
> and why it doesn't have man page.

Manpage is part of numatools.

The intended semantic is the preservation of the relative position of the
page to the beginning of the node set. If you do not want to preserve the
relative position then just move portions of the nodes around.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] migrate_pages:skip migration between intersect nodes

[Bob Liu]

On 3/31/10, Christoph Lameter <cl@linux-foundation.org> wrote:
> On Tue, 30 Mar 2010, Minchan Kim wrote:
>
>> Hi, Bob
>>
>> On Mon, Mar 29, 2010 at 11:57 PM, Bob Liu <lliubbo@gmail.com> wrote:
>> > In current do_migrate_pages(),if from_nodes and to_nodes have some
>> > intersect nodes,pages in these intersect nodes will also be
>> > migrated.
>> > eg. Assume that, from_nodes: 1,2,3,4 to_nodes: 2,3,4,5. Then these
>> > migrates will happen:
>> > migrate_pages(4,5);
>> > migrate_pages(3,4);
>> > migrate_pages(2,3);
>> > migrate_pages(1,2);
>> >
>> > But the user just want all pages in from_nodes move to to_nodes,
>> > only migrate(1,2)(ignore the intersect nodes.) can satisfied
>> > the user's request.
>> >
>> > I amn't sure what's migrate_page's semantic.
>> > Hoping for your suggestions.
>>
>> I didn't see 8:migratepages Lee pointed at that time.
>> The description matches current migrate_pages's behavior exactly.
>>
>> I agree Lee's opinion.
>> Let's wait Christoph's reply what is semantic
>> and why it doesn't have man page.
>
> Manpage is part of numatools.
>
> The intended semantic is the preservation of the relative position of the
> page to the beginning of the node set. If you do not want to preserve the
> relative position then just move portions of the nodes around.
>

Hmm.,
Sorry I still haven't understand your mention :-)

My concern was why move the pages in the intersect nodes.I think skipping
this migration we can also satisfy the user's request.
In the above semantic, I  haven't got the result.

Thanks!
-- 
Regards,
-Bob

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] migrate_pages:skip migration between intersect nodes

[Minchan Kim]

On Wed, Mar 31, 2010 at 10:36 AM, Bob Liu <lliubbo@gmail.com> wrote:
> On 3/31/10, Christoph Lameter <cl@linux-foundation.org> wrote:
>> On Tue, 30 Mar 2010, Minchan Kim wrote:
>>
>>> Hi, Bob
>>>
>>> On Mon, Mar 29, 2010 at 11:57 PM, Bob Liu <lliubbo@gmail.com> wrote:
>>> > In current do_migrate_pages(),if from_nodes and to_nodes have some
>>> > intersect nodes,pages in these intersect nodes will also be
>>> > migrated.
>>> > eg. Assume that, from_nodes: 1,2,3,4 to_nodes: 2,3,4,5. Then these
>>> > migrates will happen:
>>> > migrate_pages(4,5);
>>> > migrate_pages(3,4);
>>> > migrate_pages(2,3);
>>> > migrate_pages(1,2);
>>> >
>>> > But the user just want all pages in from_nodes move to to_nodes,
>>> > only migrate(1,2)(ignore the intersect nodes.) can satisfied
>>> > the user's request.
>>> >
>>> > I amn't sure what's migrate_page's semantic.
>>> > Hoping for your suggestions.
>>>
>>> I didn't see 8:migratepages Lee pointed at that time.
>>> The description matches current migrate_pages's behavior exactly.
>>>
>>> I agree Lee's opinion.
>>> Let's wait Christoph's reply what is semantic
>>> and why it doesn't have man page.
>>
>> Manpage is part of numatools.
>>
>> The intended semantic is the preservation of the relative position of the
>> page to the beginning of the node set. If you do not want to preserve the
>> relative position then just move portions of the nodes around.
>>
>
> Hmm.,
> Sorry I still haven't understand your mention :-)
>
> My concern was why move the pages in the intersect nodes.I think skipping
> this migration we can also satisfy the user's request.
> In the above semantic, I  haven't got the result.
>

man page said.

"For example if we move from nodes 2-5 to 7,9,12-13 then the preferred mode of
operation is to move pages from 2->7, 3->9, 4->12 and 5->13. However, this
is only posssible if enough memory is available."

If user uses migratepages((1,2,3,4), (2,3,4,5)), He want to move pages
(1->2), (2-3), (3->4), (4-5). It matches with magpage.

But with your suggestion, only (1-2).
I think It doesn't match with man page.

Do you want to add some words to make manpage more clear?

> Thanks!
> --
> Regards,
> -Bob
>



-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] migrate_pages:skip migration between intersect nodes

[Bob Liu]

On 3/31/10, Minchan Kim <minchan.kim@gmail.com> wrote:
> On Wed, Mar 31, 2010 at 10:36 AM, Bob Liu <lliubbo@gmail.com> wrote:
>> On 3/31/10, Christoph Lameter <cl@linux-foundation.org> wrote:
>>> On Tue, 30 Mar 2010, Minchan Kim wrote:
>>>
>>>> Hi, Bob
>>>>
>>>> On Mon, Mar 29, 2010 at 11:57 PM, Bob Liu <lliubbo@gmail.com> wrote:
>>>> > In current do_migrate_pages(),if from_nodes and to_nodes have some
>>>> > intersect nodes,pages in these intersect nodes will also be
>>>> > migrated.
>>>> > eg. Assume that, from_nodes: 1,2,3,4 to_nodes: 2,3,4,5. Then these
>>>> > migrates will happen:
>>>> > migrate_pages(4,5);
>>>> > migrate_pages(3,4);
>>>> > migrate_pages(2,3);
>>>> > migrate_pages(1,2);
>>>> >
>>>> > But the user just want all pages in from_nodes move to to_nodes,
>>>> > only migrate(1,2)(ignore the intersect nodes.) can satisfied
>>>> > the user's request.
>>>> >
>>>> > I amn't sure what's migrate_page's semantic.
>>>> > Hoping for your suggestions.
>>>>
>>>> I didn't see 8:migratepages Lee pointed at that time.
>>>> The description matches current migrate_pages's behavior exactly.
>>>>
>>>> I agree Lee's opinion.
>>>> Let's wait Christoph's reply what is semantic
>>>> and why it doesn't have man page.
>>>
>>> Manpage is part of numatools.
>>>
>>> The intended semantic is the preservation of the relative position of the
>>> page to the beginning of the node set. If you do not want to preserve the
>>> relative position then just move portions of the nodes around.
>>>
>>
>> Hmm.,
>> Sorry I still haven't understand your mention :-)
>>
>> My concern was why move the pages in the intersect nodes.I think skipping
>> this migration we can also satisfy the user's request.
>> In the above semantic, I  haven't got the result.
>>
>
> man page said.
>
> "For example if we move from nodes 2-5 to 7,9,12-13 then the preferred mode
> of
> operation is to move pages from 2->7, 3->9, 4->12 and 5->13. However, this
> is only posssible if enough memory is available."
>
> If user uses migratepages((1,2,3,4), (2,3,4,5)), He want to move pages
> (1->2), (2-3), (3->4), (4-5). It matches with magpage.
>
> But with your suggestion, only (1-2).
> I think It doesn't match with man page.
>

Ok. I got it.. I used to didn't find numatools.

Thanks all!

> Do you want to add some words to make manpage more clear?
>
>> Thanks!
>> --
>> Regards,
>> -Bob
>>
>
>
>
> --
> Kind regards,
> Minchan Kim
>


-- 
Regards,
--Bob

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH] migrate_pages:skip migration between intersect nodes

[Christoph Lameter]

On Wed, 31 Mar 2010, Bob Liu wrote:

> > The intended semantic is the preservation of the relative position of the
> > page to the beginning of the node set. If you do not want to preserve the
> > relative position then just move portions of the nodes around.
> >
>
> Hmm.,
> Sorry I still haven't understand your mention :-)
>
> My concern was why move the pages in the intersect nodes.I think skipping
> this migration we can also satisfy the user's request.
> In the above semantic, I  haven't got the result.

No skipping does *not* satisfy the users request since the relative
position of the page from the beginning of the nodesset is not
preserved.

You end up with a mess without this requirement. F.e. if you use page
migration (or cpuset automigration) to shift an application running on 10
nodes up by two nodes to make a hole that would allow you to run another
application on the lower nodes. Applications place pages intentionally on
certain nodes to be able to manage memory distances.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>