[PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Larry Woodman]

[-- Attachment #1: Type: text/plain, Size: 1233 bytes --]

While moving tasks between cpusets I noticed some strange behavior.  Specifically if the nodes of the destination
cpuset are a subset of the nodes of the source cpuset do_migrate_pages() will move pages that are already on a node
in the destination cpuset.  The reason for this is do_migrate_pages() does not check whether each node in the source
nodemask is in the destination nodemask before calling migrate_to_node().  If we simply do this check and skip them
when the source is in the destination moving we wont move nodes that dont need to be moved.
  
Adding a little debug printk to migrate_to_node():

Without this change migrating tasks from a cpuset containing nodes 0-7 to a cpuset containing nodes 3-4, we migrate
from ALL the nodes even if they are in the both the source and destination nodesets:

   Migrating 7 to 4
   Migrating 6 to 3
   Migrating 5 to 4
   Migrating 4 to 3
   Migrating 1 to 4
   Migrating 3 to 4
   Migrating 0 to 3
   Migrating 2 to 3


With this change we only migrate from nodes that are not in the destination nodesets:

   Migrating 7 to 4
   Migrating 6 to 3
   Migrating 5 to 4
   Migrating 2 to 3
   Migrating 1 to 4
   Migrating 0 to 3

Signed-off-by: Larry Woodman<lwoodman@redhat.com>


[-- Attachment #2: upstream-do_migrate_pages.patch --]
[-- Type: text/plain, Size: 407 bytes --]

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 47296fe..2bd13e9 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -1012,6 +1012,9 @@ int do_migrate_pages(struct mm_struct *mm,
 		int dest = 0;
 
 		for_each_node_mask(s, tmp) {
+			/* no need to move if its already there */
+			if (node_isset(s, *to_nodes))
+				continue;
 			d = node_remap(s, *from_nodes, *to_nodes);
 			if (s == d)
 				continue;

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Rik van Riel]

On 03/22/2012 02:14 PM, Larry Woodman wrote:

> With this change we only migrate from nodes that are not in the
> destination nodesets:

That's a pretty obvious improvement :)

> Migrating 7 to 4
> Migrating 6 to 3
> Migrating 5 to 4
> Migrating 2 to 3
> Migrating 1 to 4
> Migrating 0 to 3
>
> Signed-off-by: Larry Woodman<lwoodman@redhat.com>

Acked-by: Rik van Riel <riel@redhat.com>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[KOSAKI Motohiro]

CC to Christoph.

> While moving tasks between cpusets I noticed some strange behavior. Specifically if the nodes of the destination
> cpuset are a subset of the nodes of the source cpuset do_migrate_pages() will move pages that are already on a node
> in the destination cpuset. The reason for this is do_migrate_pages() does not check whether each node in the source
> nodemask is in the destination nodemask before calling migrate_to_node(). If we simply do this check and skip them
> when the source is in the destination moving we wont move nodes that dont need to be moved.
>
> Adding a little debug printk to migrate_to_node():
>
> Without this change migrating tasks from a cpuset containing nodes 0-7 to a cpuset containing nodes 3-4, we migrate
> from ALL the nodes even if they are in the both the source and destination nodesets:
>
> Migrating 7 to 4
> Migrating 6 to 3
> Migrating 5 to 4
> Migrating 4 to 3
> Migrating 1 to 4
> Migrating 3 to 4
> Migrating 0 to 3
> Migrating 2 to 3

Wait.

This may be non-optimal for cpusets, but maybe optimal migrate_pages, especially
the usecase is HPC. I guess this is intended behavior. I think we need to hear
Christoph's intention.

But, I'm not against this if he has no objection.




>
>
> With this change we only migrate from nodes that are not in the destination nodesets:
>
> Migrating 7 to 4
> Migrating 6 to 3
> Migrating 5 to 4
> Migrating 2 to 3
> Migrating 1 to 4
> Migrating 0 to 3
>
> Signed-off-by: Larry Woodman<lwoodman@redhat.com>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Rik van Riel]

On 03/22/2012 02:45 PM, KOSAKI Motohiro wrote:
> CC to Christoph.
>
>> While moving tasks between cpusets I noticed some strange behavior.
>> Specifically if the nodes of the destination
>> cpuset are a subset of the nodes of the source cpuset
>> do_migrate_pages() will move pages that are already on a node
>> in the destination cpuset. The reason for this is do_migrate_pages()
>> does not check whether each node in the source
>> nodemask is in the destination nodemask before calling
>> migrate_to_node(). If we simply do this check and skip them
>> when the source is in the destination moving we wont move nodes that
>> dont need to be moved.
>>
>> Adding a little debug printk to migrate_to_node():
>>
>> Without this change migrating tasks from a cpuset containing nodes 0-7
>> to a cpuset containing nodes 3-4, we migrate
>> from ALL the nodes even if they are in the both the source and
>> destination nodesets:
>>
>> Migrating 7 to 4
>> Migrating 6 to 3
>> Migrating 5 to 4
>> Migrating 4 to 3
>> Migrating 1 to 4
>> Migrating 3 to 4
>> Migrating 0 to 3
>> Migrating 2 to 3
>
> Wait.
>
> This may be non-optimal for cpusets, but maybe optimal migrate_pages,
> especially
> the usecase is HPC. I guess this is intended behavior. I think we need
> to hear
> Christoph's intention.

How is the moving of pages that are already in the
intended cpuset better than just leaving them alone?

> But, I'm not against this if he has no objection.

Sure. Christoph?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Larry Woodman]

On 03/22/2012 02:45 PM, KOSAKI Motohiro wrote:
> CC to Christoph.
>
>
Sorry Christoph I meant to cc you on the original message!

Larry

> Wait.
>
> This may be non-optimal for cpusets, but maybe optimal migrate_pages, 
> especially
> the usecase is HPC. I guess this is intended behavior. I think we need 
> to hear
> Christoph's intention.
>
> But, I'm not against this if he has no objection.
>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Christoph Lameter]

On Thu, 22 Mar 2012, KOSAKI Motohiro wrote:

> CC to Christoph.
>
> > While moving tasks between cpusets I noticed some strange behavior.
> > Specifically if the nodes of the destination
> > cpuset are a subset of the nodes of the source cpuset do_migrate_pages()
> > will move pages that are already on a node
> > in the destination cpuset. The reason for this is do_migrate_pages() does
> > not check whether each node in the source
> > nodemask is in the destination nodemask before calling migrate_to_node(). If
> > we simply do this check and skip them
> > when the source is in the destination moving we wont move nodes that dont
> > need to be moved.
> >
> > Adding a little debug printk to migrate_to_node():
> >
> > Without this change migrating tasks from a cpuset containing nodes 0-7 to a
> > cpuset containing nodes 3-4, we migrate
> > from ALL the nodes even if they are in the both the source and destination
> > nodesets:
> >
> > Migrating 7 to 4
> > Migrating 6 to 3
> > Migrating 5 to 4
> > Migrating 4 to 3
> > Migrating 1 to 4
> > Migrating 3 to 4
> > Migrating 0 to 3
> > Migrating 2 to 3
>
> Wait.
>
> This may be non-optimal for cpusets, but maybe optimal migrate_pages,
> especially
> the usecase is HPC. I guess this is intended behavior. I think we need to hear
> Christoph's intention.
>
> But, I'm not against this if he has no objection.

The use case for this is if you have an app running on nodes 3,4,5 on your
machine and now you want to shift it to 4,5,6. The expectation is that the
location of the pages relative to the first node stay the same.
Application may manage their locality given a range of nodes and each of
the x .. x+n nodes has their particular purpose.

If you justd copy 3 to 6 then the app may get confused when doing
additional allocations since different types of information is now stored
on the "first" node (which is now 4).



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Larry Woodman]

On 03/22/2012 02:51 PM, Christoph Lameter wrote:
> On Thu, 22 Mar 2012, KOSAKI Motohiro wrote:
>
>> CC to Christoph.
>>
>>> While moving tasks between cpusets I noticed some strange behavior.
>>> Specifically if the nodes of the destination
>>> cpuset are a subset of the nodes of the source cpuset do_migrate_pages()
>>> will move pages that are already on a node
>>> in the destination cpuset. The reason for this is do_migrate_pages() does
>>> not check whether each node in the source
>>> nodemask is in the destination nodemask before calling migrate_to_node(). If
>>> we simply do this check and skip them
>>> when the source is in the destination moving we wont move nodes that dont
>>> need to be moved.
>>>
>>> Adding a little debug printk to migrate_to_node():
>>>
>>> Without this change migrating tasks from a cpuset containing nodes 0-7 to a
>>> cpuset containing nodes 3-4, we migrate
>>> from ALL the nodes even if they are in the both the source and destination
>>> nodesets:
>>>
>>> Migrating 7 to 4
>>> Migrating 6 to 3
>>> Migrating 5 to 4
>>> Migrating 4 to 3
>>> Migrating 1 to 4
>>> Migrating 3 to 4
>>> Migrating 0 to 3
>>> Migrating 2 to 3
>> Wait.
>>
>> This may be non-optimal for cpusets, but maybe optimal migrate_pages,
>> especially
>> the usecase is HPC. I guess this is intended behavior. I think we need to hear
>> Christoph's intention.
>>
>> But, I'm not against this if he has no objection.
> The use case for this is if you have an app running on nodes 3,4,5 on your
> machine and now you want to shift it to 4,5,6. The expectation is that the
> location of the pages relative to the first node stay the same.
> Application may manage their locality given a range of nodes and each of
> the x .. x+n nodes has their particular purpose.
So to be clear on this, in that case the intention would be move 3 to 4, 
4 to 5 and 5 to 6
to keep the node ordering the same?

Larry
> If you justd copy 3 to 6 then the app may get confused when doing
> additional allocations since different types of information is now stored
> on the "first" node (which is now 4).
>
>
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
> Don't email:<a href=mailto:"dont@kvack.org">  email@kvack.org</a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Christoph Lameter]

On Thu, 22 Mar 2012, Larry Woodman wrote:

> > Application may manage their locality given a range of nodes and each of
> > the x .. x+n nodes has their particular purpose.
> So to be clear on this, in that case the intention would be move 3 to 4, 4 to
> 5 and 5 to 6
> to keep the node ordering the same?

Yup. Have a look at do_migrate_pages and the descrition in the comment by
there by Paul Jackson.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Larry Woodman]

[-- Attachment #1: Type: text/plain, Size: 521 bytes --]

On 03/22/2012 03:30 PM, Christoph Lameter wrote:
> On Thu, 22 Mar 2012, Larry Woodman wrote:
>
>>> Application may manage their locality given a range of nodes and each of
>>> the x .. x+n nodes has their particular purpose.
>> So to be clear on this, in that case the intention would be move 3 to 4, 4 to
>> 5 and 5 to 6
>> to keep the node ordering the same?
> Yup. Have a look at do_migrate_pages and the descrition in the comment by
> there by Paul Jackson.
>
>
Christoph and others what do you think about this???



[-- Attachment #2: upstream-do_migrate_pages.patch --]
[-- Type: text/plain, Size: 702 bytes --]

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 47296fe..6c189fa 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -1012,6 +1012,16 @@ int do_migrate_pages(struct mm_struct *mm,
 		int dest = 0;
 
 		for_each_node_mask(s, tmp) {
+
+			/* IFF there is an equal number of source and
+			 * destination nodes, maintain relative node distance
+			 * even when source and destination nodes overlap.
+			 * However, when the node weight is unequal, never move
+			 * memory out of any destination nodes */
+			if ((nodes_weight(*from_nodes) != nodes_weight(*to_nodes)) && 
+						(node_isset(s, *to_nodes)))
+				continue;
+
 			d = node_remap(s, *from_nodes, *to_nodes);
 			if (s == d)
 				continue;

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[KOSAKI Motohiro]

(3/29/12 2:00 PM), Larry Woodman wrote:
> On 03/22/2012 03:30 PM, Christoph Lameter wrote:
>> On Thu, 22 Mar 2012, Larry Woodman wrote:
>>
>>>> Application may manage their locality given a range of nodes and each of
>>>> the x .. x+n nodes has their particular purpose.
>>> So to be clear on this, in that case the intention would be move 3 to 4, 4 to
>>> 5 and 5 to 6
>>> to keep the node ordering the same?
>> Yup. Have a look at do_migrate_pages and the descrition in the comment by
>> there by Paul Jackson.
>>
>>
> Christoph and others what do you think about this???
>
>
> 		for_each_node_mask(s, tmp) {
>+
>+			/* IFF there is an equal number of source and
>+			 * destination nodes, maintain relative node distance
>+			 * even when source and destination nodes overlap.
>+			 * However, when the node weight is unequal, never move
>+			 * memory out of any destination nodes */
>+			if ((nodes_weight(*from_nodes) != nodes_weight(*to_nodes)) &&
>+						(node_isset(s, *to_nodes)))
>+				continue;
>+
> 			d = node_remap(s, *from_nodes, *to_nodes);
> 			if (s == d)
> 				continue;

I'm confused. Could you please explain why you choose nodes_weight()? On my first impression,
it seems almostly unrelated factor.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Larry Woodman]

On 03/29/2012 03:43 PM, KOSAKI Motohiro wrote:
> (3/29/12 2:00 PM), Larry Woodman wrote:
>> On 03/22/2012 03:30 PM, Christoph Lameter wrote:
>>> On Thu, 22 Mar 2012, Larry Woodman wrote:
>>>
>>>>> Application may manage their locality given a range of nodes and 
>>>>> each of
>>>>> the x .. x+n nodes has their particular purpose.
>>>> So to be clear on this, in that case the intention would be move 3 
>>>> to 4, 4 to
>>>> 5 and 5 to 6
>>>> to keep the node ordering the same?
>>> Yup. Have a look at do_migrate_pages and the descrition in the 
>>> comment by
>>> there by Paul Jackson.
>>>
>>>
>> Christoph and others what do you think about this???
>>
>>
>>         for_each_node_mask(s, tmp) {
>> +
>> +            /* IFF there is an equal number of source and
>> +             * destination nodes, maintain relative node distance
>> +             * even when source and destination nodes overlap.
>> +             * However, when the node weight is unequal, never move
>> +             * memory out of any destination nodes */
>> +            if ((nodes_weight(*from_nodes) != 
>> nodes_weight(*to_nodes)) &&
>> +                        (node_isset(s, *to_nodes)))
>> +                continue;
>> +
>>             d = node_remap(s, *from_nodes, *to_nodes);
>>             if (s == d)
>>                 continue;
>
> I'm confused. Could you please explain why you choose nodes_weight()? 
> On my first impression,
> it seems almostly unrelated factor.

nodes_weight() tells us the number of nodes in the cpuset so if you are 
migrating
from say 2, 3 &4 to 3, 4 &5 we wont go from 2 to 5 and call it done like 
the original
patch did.  With this patch we will preserve the migrating of 2, 3 &4 to 
3, 4 &5  yet
if we are migrating from 0-7 to 3-4 we wont do this:

Migrating 7 to 4
Migrating 6 to 3
Migrating 5 to 4
Migrating 4 to 3
Migrating 1 to 4
Migrating 3 to 4
Migrating 0 to 3
Migrating 2 to 3

Instead, will do this:

Migrating 7 to 4
Migrating 6 to 3
Migrating 5 to 4
Migrating 1 to 4
Migrating 0 to 3
Migrating 2 to 3

Larry

>
>
> -- 
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign 
> http://stopthemeter.ca/
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Christoph Lameter]

On Thu, 29 Mar 2012, KOSAKI Motohiro wrote:

> >
> > 		for_each_node_mask(s, tmp) {
> > +
> > +			/* IFF there is an equal number of source and
> > +			 * destination nodes, maintain relative node distance
> > +			 * even when source and destination nodes overlap.
> > +			 * However, when the node weight is unequal, never
> > move
> > +			 * memory out of any destination nodes */
> > +			if ((nodes_weight(*from_nodes) !=
> > nodes_weight(*to_nodes)) &&
> > +						(node_isset(s, *to_nodes)))
> > +				continue;
> > +
> > 			d = node_remap(s, *from_nodes, *to_nodes);
> > 			if (s == d)
> > 				continue;
>
> I'm confused. Could you please explain why you choose nodes_weight()? On my
> first impression,
> it seems almostly unrelated factor.

Isnt this the original code by Paul? I would think that the 1-1 movement
is only useful to do if the number of nodes in both the destination and
the source is the same.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Larry Woodman]

On 03/30/2012 12:15 PM, Christoph Lameter wrote:
> On Thu, 29 Mar 2012, KOSAKI Motohiro wrote:
>
>>> 		for_each_node_mask(s, tmp) {
>>> +
>>> +			/* IFF there is an equal number of source and
>>> +			 * destination nodes, maintain relative node distance
>>> +			 * even when source and destination nodes overlap.
>>> +			 * However, when the node weight is unequal, never
>>> move
>>> +			 * memory out of any destination nodes */
>>> +			if ((nodes_weight(*from_nodes) !=
>>> nodes_weight(*to_nodes))&&
>>> +						(node_isset(s, *to_nodes)))
>>> +				continue;
>>> +
>>> 			d = node_remap(s, *from_nodes, *to_nodes);
>>> 			if (s == d)
>>> 				continue;
>> I'm confused. Could you please explain why you choose nodes_weight()? On my
>> first impression,
>> it seems almostly unrelated factor.
> Isnt this the original code by Paul?
No, I added the test to see if the source and destination has the same 
number of nodes.
>   I would think that the 1-1 movement
> is only useful to do if the number of nodes in both the destination and
> the source is the same.
Agreed, thats exactly what this patch does.  are you OK with this change 
then???

Larry


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Christoph Lameter]

On Fri, 30 Mar 2012, Larry Woodman wrote:

> of nodes.
> >   I would think that the 1-1 movement
> > is only useful to do if the number of nodes in both the destination and
> > the source is the same.
> Agreed, thats exactly what this patch does.  are you OK with this change
> then???

Please add to the patch description some explanation how this patch
changes the way page migration does things.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 330 bytes --]

On Thu, 22 Mar 2012 15:07:00 -0400, Larry Woodman said:

> So to be clear on this, in that case the intention would be move 3 to 4,
> 4 to 5 and 5 to 6
> to keep the node ordering the same?

Would it make more sense to do 5->6, 4->5, 3->4?  If we move stuff
from 3 to 4 before clearing the old 4 stuff out, it might get crowded?


[-- Attachment #2: Type: application/pgp-signature, Size: 865 bytes --]

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Christoph Lameter]

On Thu, 22 Mar 2012, Valdis.Kletnieks@vt.edu wrote:

> > So to be clear on this, in that case the intention would be move 3 to 4,
> > 4 to 5 and 5 to 6
> > to keep the node ordering the same?
>
> Would it make more sense to do 5->6, 4->5, 3->4?  If we move stuff
> from 3 to 4 before clearing the old 4 stuff out, it might get crowded?

Right. I thought Paul did take care of that way back when it was written?


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Larry Woodman]

On 03/22/2012 03:36 PM, Valdis.Kletnieks@vt.edu wrote:
> On Thu, 22 Mar 2012 15:07:00 -0400, Larry Woodman said:
>
>> So to be clear on this, in that case the intention would be move 3 to 4,
>> 4 to 5 and 5 to 6
>> to keep the node ordering the same?
> Would it make more sense to do 5->6, 4->5, 3->4?  If we move stuff
> from 3 to 4 before clearing the old 4 stuff out, it might get crowded?
>
Yes, I didnt try to imply the order in which pages were moved just
the additional moving necessary.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

On Thu, 22 Mar 2012 16:14:04 -0400, Larry Woodman said:
> On 03/22/2012 03:36 PM, Valdis.Kletnieks@vt.edu wrote:
> > On Thu, 22 Mar 2012 15:07:00 -0400, Larry Woodman said:
> >
> >> So to be clear on this, in that case the intention would be move 3 to 4,
> >> 4 to 5 and 5 to 6
> >> to keep the node ordering the same?
> > Would it make more sense to do 5->6, 4->5, 3->4?  If we move stuff
> > from 3 to 4 before clearing the old 4 stuff out, it might get crowded?
> >
> Yes, I didnt try to imply the order in which pages were moved just
> the additional moving necessary.

Oh, OK.. :)


[-- Attachment #2: Type: application/pgp-signature, Size: 865 bytes --]

Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node

[KOSAKI Motohiro]

(3/22/12 2:51 PM), Christoph Lameter wrote:
> On Thu, 22 Mar 2012, KOSAKI Motohiro wrote:
>
>> CC to Christoph.
>>
>>> While moving tasks between cpusets I noticed some strange behavior.
>>> Specifically if the nodes of the destination
>>> cpuset are a subset of the nodes of the source cpuset do_migrate_pages()
>>> will move pages that are already on a node
>>> in the destination cpuset. The reason for this is do_migrate_pages() does
>>> not check whether each node in the source
>>> nodemask is in the destination nodemask before calling migrate_to_node(). If
>>> we simply do this check and skip them
>>> when the source is in the destination moving we wont move nodes that dont
>>> need to be moved.
>>>
>>> Adding a little debug printk to migrate_to_node():
>>>
>>> Without this change migrating tasks from a cpuset containing nodes 0-7 to a
>>> cpuset containing nodes 3-4, we migrate
>>> from ALL the nodes even if they are in the both the source and destination
>>> nodesets:
>>>
>>> Migrating 7 to 4
>>> Migrating 6 to 3
>>> Migrating 5 to 4
>>> Migrating 4 to 3
>>> Migrating 1 to 4
>>> Migrating 3 to 4
>>> Migrating 0 to 3
>>> Migrating 2 to 3
>>
>> Wait.
>>
>> This may be non-optimal for cpusets, but maybe optimal migrate_pages,
>> especially
>> the usecase is HPC. I guess this is intended behavior. I think we need to hear
>> Christoph's intention.
>>
>> But, I'm not against this if he has no objection.
>
> The use case for this is if you have an app running on nodes 3,4,5 on your
> machine and now you want to shift it to 4,5,6. The expectation is that the
> location of the pages relative to the first node stay the same.
> Application may manage their locality given a range of nodes and each of
> the x .. x+n nodes has their particular purpose.
>
> If you justd copy 3 to 6 then the app may get confused when doing
> additional allocations since different types of information is now stored
> on the "first" node (which is now 4).

MPOL_INTERLEAVE is more simple situaltion. applications naturally assume the
memory is mapped intealeaving and application threads optimize for it. if we
broke intereaving, the applications may slow down.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>