From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx173.postini.com [74.125.245.173]) by kanga.kvack.org (Postfix) with SMTP id CF9146B005A for ; Fri, 19 Oct 2012 11:53:47 -0400 (EDT) Message-ID: <5081777A.8050104@redhat.com> Date: Fri, 19 Oct 2012 11:53:30 -0400 From: Rik van Riel MIME-Version: 1.0 Subject: question on NUMA page migration Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Peter Zijlstra Cc: Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List Hi Andrea, Peter, I have a question on page refcounting in your NUMA page migration code. In Peter's case, I wonder why you introduce a new MIGRATE_FAULT migration mode. If the normal page migration / compaction logic can do without taking an extra reference count, why does your code need it? In Andrea's case, we have a comment suggesting an extra refcount is needed, immediately followed by a put_page: /* * Pin the head subpage at least until the first * __isolate_lru_page succeeds (__isolate_lru_page pins it * again when it succeeds). If we unpin before * __isolate_lru_page successd, the page could be freed and * reallocated out from under us. Thus our previous checks on * the page, and the split_huge_page, would be worthless. * * We really only need to do this if "ret > 0" but it doesn't * hurt to do it unconditionally as nobody can reference * "page" anymore after this and so we can avoid an "if (ret > * 0)" branch here. */ put_page(page); This also confuses me. If we do not need the extra refcount (and I do not understand why NUMA migrate-on-fault needs one more refcount than normal page migration), we can get rid of the MIGRATE_FAULT mode. If we do need the extra refcount, why is normal page migration safe? :) -- All rights reversed -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx167.postini.com [74.125.245.167]) by kanga.kvack.org (Postfix) with SMTP id E9E5F6B005A for ; Fri, 19 Oct 2012 12:39:17 -0400 (EDT) Message-ID: <1350664742.2768.40.camel@twins> Subject: Re: question on NUMA page migration From: Peter Zijlstra Date: Fri, 19 Oct 2012 18:39:02 +0200 In-Reply-To: <5081777A.8050104@redhat.com> References: <5081777A.8050104@redhat.com> Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 Sender: owner-linux-mm@kvack.org List-ID: To: Rik van Riel Cc: Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List On Fri, 2012-10-19 at 11:53 -0400, Rik van Riel wrote: >=20 > If we do need the extra refcount, why is normal > page migration safe? :)=20 Its mostly a matter of how convoluted you make the code, regular page migration is about as bad as you can get Normal does: follow_page(FOLL_GET) +1 isolate_lru_page() +1 put_page() -1 ending up with a page with a single reference (for anon, or one extra each for the mapping and buffer). And while I suppose I could do a put_page() in migrate_misplaced_page() that makes the function frob the page-count depending on the return value. I always try and avoid conditional locks/refs, therefore the code ends up doing: page =3D vm_normal_page() if (page) { get_page() migrate_misplaced_page() put_page() } where migrate_misplaced_page() does isolate_lru_page()/putback_lru_page, and this leaves the page-count invariant. We got a ref, therefore we must put a ref, is easier than we got a ref and must put except when... -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx193.postini.com [74.125.245.193]) by kanga.kvack.org (Postfix) with SMTP id 81B0C6B005A for ; Fri, 19 Oct 2012 13:13:50 -0400 (EDT) Message-ID: <50818A41.7030909@redhat.com> Date: Fri, 19 Oct 2012 13:13:37 -0400 From: Rik van Riel MIME-Version: 1.0 Subject: Re: question on NUMA page migration References: <5081777A.8050104@redhat.com> <1350664742.2768.40.camel@twins> In-Reply-To: <1350664742.2768.40.camel@twins> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Peter Zijlstra Cc: Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List On 10/19/2012 12:39 PM, Peter Zijlstra wrote: > On Fri, 2012-10-19 at 11:53 -0400, Rik van Riel wrote: >> >> If we do need the extra refcount, why is normal >> page migration safe? :) > > Its mostly a matter of how convoluted you make the code, regular page > migration is about as bad as you can get > > Normal does: > > follow_page(FOLL_GET) +1 > > isolate_lru_page() +1 > > put_page() -1 > > ending up with a page with a single reference (for anon, or one extra > each for the mapping and buffer). Would it make sense to have the normal page migration code always work with the extra refcount, so we do not have to introduce a new MIGRATE_FAULT migration mode? On the other hand, compaction does not take the extra reference... Another alternative might be to do the put_page inside do_prot_none_numa(). That would be analogous to do_wp_page disposing of the old page for the caller. I am not real happy about NUMA migration introducing its own migration mode... -- All rights reversed -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx193.postini.com [74.125.245.193]) by kanga.kvack.org (Postfix) with SMTP id F00966B005A for ; Fri, 19 Oct 2012 13:54:40 -0400 (EDT) Message-ID: <1350669236.2768.66.camel@twins> Subject: Re: question on NUMA page migration From: Peter Zijlstra Date: Fri, 19 Oct 2012 19:53:56 +0200 In-Reply-To: <50818A41.7030909@redhat.com> References: <5081777A.8050104@redhat.com> <1350664742.2768.40.camel@twins> <50818A41.7030909@redhat.com> Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 Sender: owner-linux-mm@kvack.org List-ID: To: Rik van Riel Cc: Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List On Fri, 2012-10-19 at 13:13 -0400, Rik van Riel wrote: > Would it make sense to have the normal page migration code always > work with the extra refcount, so we do not have to introduce a new > MIGRATE_FAULT migration mode? >=20 > On the other hand, compaction does not take the extra reference... Right, it appears to not do this, it gets pages from the pfn and zone->lock and the isolate_lru_page() call is the first reference. > Another alternative might be to do the put_page inside > do_prot_none_numa(). That would be analogous to do_wp_page > disposing of the old page for the caller. It'd have to be inside migrate_misplaced_page(), can't do before isolate_lru_page() or the page might disappear. Doing it after is (obviously) too late. > I am not real happy about NUMA migration introducing its own > migration mode... You didn't seem to mind too much earlier, but I can remove it if you want. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx125.postini.com [74.125.245.125]) by kanga.kvack.org (Postfix) with SMTP id AEC266B005A for ; Fri, 19 Oct 2012 14:33:31 -0400 (EDT) Message-ID: <50819CED.30803@redhat.com> Date: Fri, 19 Oct 2012 14:33:17 -0400 From: Rik van Riel MIME-Version: 1.0 Subject: Re: question on NUMA page migration References: <5081777A.8050104@redhat.com> <1350664742.2768.40.camel@twins> <50818A41.7030909@redhat.com> <1350669236.2768.66.camel@twins> In-Reply-To: <1350669236.2768.66.camel@twins> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Peter Zijlstra Cc: Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List On 10/19/2012 01:53 PM, Peter Zijlstra wrote: > On Fri, 2012-10-19 at 13:13 -0400, Rik van Riel wrote: >> Another alternative might be to do the put_page inside >> do_prot_none_numa(). That would be analogous to do_wp_page >> disposing of the old page for the caller. > > It'd have to be inside migrate_misplaced_page(), can't do before > isolate_lru_page() or the page might disappear. Doing it after is > (obviously) too late. Keeping an extra refcount on the page might _still_ result in it disappearing from the process by some other means, in-between you grabbing the refcount and invoking migration of the page. >> I am not real happy about NUMA migration introducing its own >> migration mode... > > You didn't seem to mind too much earlier, but I can remove it if you > want. Could have been reviewing fatigue :) And yes, it would have been nice to not have a special migration mode for sched/numa. Speaking of, when do you guys plan to submit a (cleaned up) version of the sched/numa patch series for review on lkml? -- All rights reversed -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx188.postini.com [74.125.245.188]) by kanga.kvack.org (Postfix) with SMTP id BAC866B005A for ; Fri, 19 Oct 2012 21:23:51 -0400 (EDT) Received: by mail-wi0-f179.google.com with SMTP id hq7so674287wib.8 for ; Fri, 19 Oct 2012 18:23:50 -0700 (PDT) Date: Sat, 20 Oct 2012 03:23:46 +0200 From: Ingo Molnar Subject: Re: question on NUMA page migration Message-ID: <20121020012345.GA24667@gmail.com> References: <5081777A.8050104@redhat.com> <1350664742.2768.40.camel@twins> <50818A41.7030909@redhat.com> <1350669236.2768.66.camel@twins> <50819CED.30803@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50819CED.30803@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Rik van Riel Cc: Peter Zijlstra , Andrea Arcangeli , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List * Rik van Riel wrote: > On 10/19/2012 01:53 PM, Peter Zijlstra wrote: > >On Fri, 2012-10-19 at 13:13 -0400, Rik van Riel wrote: > > >>Another alternative might be to do the put_page inside > >>do_prot_none_numa(). That would be analogous to do_wp_page > >>disposing of the old page for the caller. > > > >It'd have to be inside migrate_misplaced_page(), can't do before > >isolate_lru_page() or the page might disappear. Doing it after is > >(obviously) too late. > > Keeping an extra refcount on the page might _still_ > result in it disappearing from the process by some > other means, in-between you grabbing the refcount > and invoking migration of the page. > > >>I am not real happy about NUMA migration introducing its own > >>migration mode... > > > >You didn't seem to mind too much earlier, but I can remove it if you > >want. > > Could have been reviewing fatigue :) :-) > And yes, it would have been nice to not have a special > migration mode for sched/numa. > > Speaking of, when do you guys plan to submit a (cleaned up) > version of the sched/numa patch series for review on lkml? Which commit(s) worry you specifically? Thanks, Ingo -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx161.postini.com [74.125.245.161]) by kanga.kvack.org (Postfix) with SMTP id 7F34D6B0062 for ; Sat, 20 Oct 2012 12:02:46 -0400 (EDT) Message-ID: <5082CB18.6060300@redhat.com> Date: Sat, 20 Oct 2012 12:02:32 -0400 From: Rik van Riel MIME-Version: 1.0 Subject: Re: question on NUMA page migration References: <5081777A.8050104@redhat.com> <1350664742.2768.40.camel@twins> <50818A41.7030909@redhat.com> <1350669236.2768.66.camel@twins> <50819CED.30803@redhat.com> <20121020012345.GA24667@gmail.com> In-Reply-To: <20121020012345.GA24667@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Ingo Molnar Cc: Peter Zijlstra , Andrea Arcangeli , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List On 10/19/2012 09:23 PM, Ingo Molnar wrote: > > * Rik van Riel wrote: > >> On 10/19/2012 01:53 PM, Peter Zijlstra wrote: >>> On Fri, 2012-10-19 at 13:13 -0400, Rik van Riel wrote: >> >>>> Another alternative might be to do the put_page inside >>>> do_prot_none_numa(). That would be analogous to do_wp_page >>>> disposing of the old page for the caller. >>> >>> It'd have to be inside migrate_misplaced_page(), can't do before >>> isolate_lru_page() or the page might disappear. Doing it after is >>> (obviously) too late. >> >> Keeping an extra refcount on the page might _still_ >> result in it disappearing from the process by some >> other means, in-between you grabbing the refcount >> and invoking migration of the page. >> >>>> I am not real happy about NUMA migration introducing its own >>>> migration mode... >>> >>> You didn't seem to mind too much earlier, but I can remove it if you >>> want. >> >> Could have been reviewing fatigue :) > > :-) > >> And yes, it would have been nice to not have a special >> migration mode for sched/numa. >> >> Speaking of, when do you guys plan to submit a (cleaned up) >> version of the sched/numa patch series for review on lkml? > > Which commit(s) worry you specifically? One of them would be the commit that introduces MIGRATE_FAULT. Adding it in one patch, and removing it into a next just makes things harder on the reviewers. 03a040f6c17ab81659579ba6abe267c0562097e4 If the changesets with NUMA syscalls are still in your tree's history, they should not be submitted as part of the patch series, either. -- All rights reversed -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx138.postini.com [74.125.245.138]) by kanga.kvack.org (Postfix) with SMTP id 0E2C76B0062 for ; Sat, 20 Oct 2012 22:39:35 -0400 (EDT) Received: by mail-ie0-f169.google.com with SMTP id 10so2869217ied.14 for ; Sat, 20 Oct 2012 19:39:34 -0700 (PDT) Message-ID: <50836060.4050408@gmail.com> Date: Sun, 21 Oct 2012 10:39:28 +0800 From: Ni zhan Chen MIME-Version: 1.0 Subject: Re: question on NUMA page migration References: <5081777A.8050104@redhat.com> In-Reply-To: <5081777A.8050104@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Rik van Riel Cc: Peter Zijlstra , Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List On 10/19/2012 11:53 PM, Rik van Riel wrote: > Hi Andrea, Peter, > > I have a question on page refcounting in your NUMA > page migration code. > > In Peter's case, I wonder why you introduce a new > MIGRATE_FAULT migration mode. If the normal page > migration / compaction logic can do without taking > an extra reference count, why does your code need it? Hi Rik van Riel, This is which part of codes? Why I can't find MIGRATE_FAULT in latest v3.7-rc2? Regards, Chen > > In Andrea's case, we have a comment suggesting an > extra refcount is needed, immediately followed by > a put_page: > > /* > * Pin the head subpage at least until the first > * __isolate_lru_page succeeds (__isolate_lru_page pins it > * again when it succeeds). If we unpin before > * __isolate_lru_page successd, the page could be freed and > * reallocated out from under us. Thus our previous checks on > * the page, and the split_huge_page, would be worthless. > * > * We really only need to do this if "ret > 0" but it doesn't > * hurt to do it unconditionally as nobody can reference > * "page" anymore after this and so we can avoid an "if (ret > > * 0)" branch here. > */ > put_page(page); > > This also confuses me. > > If we do not need the extra refcount (and I do not > understand why NUMA migrate-on-fault needs one more > refcount than normal page migration), we can get > rid of the MIGRATE_FAULT mode. > > If we do need the extra refcount, why is normal > page migration safe? :) > -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx116.postini.com [74.125.245.116]) by kanga.kvack.org (Postfix) with SMTP id ED9406B0062 for ; Sat, 20 Oct 2012 22:40:26 -0400 (EDT) Message-ID: <5083608E.6040209@redhat.com> Date: Sat, 20 Oct 2012 22:40:14 -0400 From: Rik van Riel MIME-Version: 1.0 Subject: Re: question on NUMA page migration References: <5081777A.8050104@redhat.com> <50836060.4050408@gmail.com> In-Reply-To: <50836060.4050408@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Ni zhan Chen Cc: Peter Zijlstra , Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List On 10/20/2012 10:39 PM, Ni zhan Chen wrote: > On 10/19/2012 11:53 PM, Rik van Riel wrote: >> Hi Andrea, Peter, >> >> I have a question on page refcounting in your NUMA >> page migration code. >> >> In Peter's case, I wonder why you introduce a new >> MIGRATE_FAULT migration mode. If the normal page >> migration / compaction logic can do without taking >> an extra reference count, why does your code need it? > > Hi Rik van Riel, > > This is which part of codes? Why I can't find MIGRATE_FAULT in latest > v3.7-rc2? It is in tip.git in the numa/core branch. -- All rights reversed -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx127.postini.com [74.125.245.127]) by kanga.kvack.org (Postfix) with SMTP id 6EE416B0062 for ; Sun, 21 Oct 2012 08:30:14 -0400 (EDT) Received: by mail-wi0-f169.google.com with SMTP id hq4so1506575wib.2 for ; Sun, 21 Oct 2012 05:30:12 -0700 (PDT) Date: Sun, 21 Oct 2012 14:30:08 +0200 From: Ingo Molnar Subject: Re: question on NUMA page migration Message-ID: <20121021123008.GA19229@gmail.com> References: <5081777A.8050104@redhat.com> <1350664742.2768.40.camel@twins> <50818A41.7030909@redhat.com> <1350669236.2768.66.camel@twins> <50819CED.30803@redhat.com> <20121020012345.GA24667@gmail.com> <5082CB18.6060300@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5082CB18.6060300@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Rik van Riel Cc: Peter Zijlstra , Andrea Arcangeli , Linux Memory Management List , Mel Gorman , Thomas Gleixner , Linux kernel Mailing List * Rik van Riel wrote: > On 10/19/2012 09:23 PM, Ingo Molnar wrote: > > > >* Rik van Riel wrote: > > > >>On 10/19/2012 01:53 PM, Peter Zijlstra wrote: > >>>On Fri, 2012-10-19 at 13:13 -0400, Rik van Riel wrote: > >> > >>>>Another alternative might be to do the put_page inside > >>>>do_prot_none_numa(). That would be analogous to do_wp_page > >>>>disposing of the old page for the caller. > >>> > >>>It'd have to be inside migrate_misplaced_page(), can't do before > >>>isolate_lru_page() or the page might disappear. Doing it after is > >>>(obviously) too late. > >> > >>Keeping an extra refcount on the page might _still_ > >>result in it disappearing from the process by some > >>other means, in-between you grabbing the refcount > >>and invoking migration of the page. > >> > >>>>I am not real happy about NUMA migration introducing its own > >>>>migration mode... > >>> > >>>You didn't seem to mind too much earlier, but I can remove it if you > >>>want. > >> > >>Could have been reviewing fatigue :) > > > >:-) > > > >>And yes, it would have been nice to not have a special > >>migration mode for sched/numa. > >> > >>Speaking of, when do you guys plan to submit a (cleaned up) > >>version of the sched/numa patch series for review on lkml? > > > >Which commit(s) worry you specifically? > > One of them would be the commit that introduces MIGRATE_FAULT. MIGRATE_FAULT is still used. > Adding it in one patch, and removing it into a next just makes > things harder on the reviewers. Yes. > 03a040f6c17ab81659579ba6abe267c0562097e4 It's still used: comet:~/tip> git grep MIGRATE_FAULT include/linux/migrate_mode.h: * MIGRATE_FAULT called from the fault path to migrate-on-fault for mempolicy include/linux/migrate_mode.h: MIGRATE_FAULT, mm/migrate.c: if (mode != MIGRATE_ASYNC && mode != MIGRATE_FAULT) { mm/migrate.c: if (mode == MIGRATE_FAULT) { mm/migrate.c: * MIGRATE_FAULT has an extra reference on the page and mm/migrate.c: if ((mode == MIGRATE_ASYNC || mode == MIGRATE_FAULT) && head && mm/migrate.c: if (mode != MIGRATE_ASYNC && mode != MIGRATE_FAULT) mm/migrate.c: if (!force || mode == MIGRATE_ASYNC || mode == MIGRATE_FAULT) mm/migrate.c: ret = __unmap_and_move(page, newpage, 0, 0, MIGRATE_FAULT); > If the changesets with NUMA syscalls are still in your tree's > history, they should not be submitted as part of the patch > series, either. No, the syscalls have not been there for quite some time. If you have trouble with any specific commit, please quote the specific SHA1 so that I can have a look and resolve any specific concerns. Otherwise, lets continue with the integration? Thanks, Ingo -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx198.postini.com [74.125.245.198]) by kanga.kvack.org (Postfix) with SMTP id 6F0486B0062 for ; Sun, 21 Oct 2012 08:31:53 -0400 (EDT) Received: by mail-wg0-f45.google.com with SMTP id dq12so1319477wgb.26 for ; Sun, 21 Oct 2012 05:31:51 -0700 (PDT) Date: Sun, 21 Oct 2012 14:31:47 +0200 From: Ingo Molnar Subject: Re: question on NUMA page migration Message-ID: <20121021123147.GB19229@gmail.com> References: <5081777A.8050104@redhat.com> <50836060.4050408@gmail.com> <5083608E.6040209@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5083608E.6040209@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Rik van Riel Cc: Ni zhan Chen , Peter Zijlstra , Andrea Arcangeli , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List * Rik van Riel wrote: > On 10/20/2012 10:39 PM, Ni zhan Chen wrote: > >On 10/19/2012 11:53 PM, Rik van Riel wrote: > >>Hi Andrea, Peter, > >> > >>I have a question on page refcounting in your NUMA > >>page migration code. > >> > >>In Peter's case, I wonder why you introduce a new > >>MIGRATE_FAULT migration mode. If the normal page > >>migration / compaction logic can do without taking > >>an extra reference count, why does your code need it? > > > >Hi Rik van Riel, > > > >This is which part of codes? Why I can't find MIGRATE_FAULT in latest > >v3.7-rc2? > > It is in tip.git in the numa/core branch. The Git access URI is: git pull git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git numa/core git clone git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git numa/core Thanks, Ingo -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759282Ab2JSPxv (ORCPT ); Fri, 19 Oct 2012 11:53:51 -0400 Received: from mx1.redhat.com ([209.132.183.28]:12768 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932856Ab2JSPxs (ORCPT ); Fri, 19 Oct 2012 11:53:48 -0400 Message-ID: <5081777A.8050104@redhat.com> Date: Fri, 19 Oct 2012 11:53:30 -0400 From: Rik van Riel User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: Peter Zijlstra CC: Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List Subject: question on NUMA page migration Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Andrea, Peter, I have a question on page refcounting in your NUMA page migration code. In Peter's case, I wonder why you introduce a new MIGRATE_FAULT migration mode. If the normal page migration / compaction logic can do without taking an extra reference count, why does your code need it? In Andrea's case, we have a comment suggesting an extra refcount is needed, immediately followed by a put_page: /* * Pin the head subpage at least until the first * __isolate_lru_page succeeds (__isolate_lru_page pins it * again when it succeeds). If we unpin before * __isolate_lru_page successd, the page could be freed and * reallocated out from under us. Thus our previous checks on * the page, and the split_huge_page, would be worthless. * * We really only need to do this if "ret > 0" but it doesn't * hurt to do it unconditionally as nobody can reference * "page" anymore after this and so we can avoid an "if (ret > * 0)" branch here. */ put_page(page); This also confuses me. If we do not need the extra refcount (and I do not understand why NUMA migrate-on-fault needs one more refcount than normal page migration), we can get rid of the MIGRATE_FAULT mode. If we do need the extra refcount, why is normal page migration safe? :) -- All rights reversed From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1422669Ab2JSQjV (ORCPT ); Fri, 19 Oct 2012 12:39:21 -0400 Received: from casper.infradead.org ([85.118.1.10]:51825 "EHLO casper.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758406Ab2JSQjS convert rfc822-to-8bit (ORCPT ); Fri, 19 Oct 2012 12:39:18 -0400 Message-ID: <1350664742.2768.40.camel@twins> Subject: Re: question on NUMA page migration From: Peter Zijlstra To: Rik van Riel Cc: Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List Date: Fri, 19 Oct 2012 18:39:02 +0200 In-Reply-To: <5081777A.8050104@redhat.com> References: <5081777A.8050104@redhat.com> Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7BIT X-Mailer: Evolution 3.2.2- Mime-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 2012-10-19 at 11:53 -0400, Rik van Riel wrote: > > If we do need the extra refcount, why is normal > page migration safe? :) Its mostly a matter of how convoluted you make the code, regular page migration is about as bad as you can get Normal does: follow_page(FOLL_GET) +1 isolate_lru_page() +1 put_page() -1 ending up with a page with a single reference (for anon, or one extra each for the mapping and buffer). And while I suppose I could do a put_page() in migrate_misplaced_page() that makes the function frob the page-count depending on the return value. I always try and avoid conditional locks/refs, therefore the code ends up doing: page = vm_normal_page() if (page) { get_page() migrate_misplaced_page() put_page() } where migrate_misplaced_page() does isolate_lru_page()/putback_lru_page, and this leaves the page-count invariant. We got a ref, therefore we must put a ref, is easier than we got a ref and must put except when... From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757781Ab2JSRNw (ORCPT ); Fri, 19 Oct 2012 13:13:52 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51273 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751359Ab2JSRNv (ORCPT ); Fri, 19 Oct 2012 13:13:51 -0400 Message-ID: <50818A41.7030909@redhat.com> Date: Fri, 19 Oct 2012 13:13:37 -0400 From: Rik van Riel User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: Peter Zijlstra CC: Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List Subject: Re: question on NUMA page migration References: <5081777A.8050104@redhat.com> <1350664742.2768.40.camel@twins> In-Reply-To: <1350664742.2768.40.camel@twins> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/19/2012 12:39 PM, Peter Zijlstra wrote: > On Fri, 2012-10-19 at 11:53 -0400, Rik van Riel wrote: >> >> If we do need the extra refcount, why is normal >> page migration safe? :) > > Its mostly a matter of how convoluted you make the code, regular page > migration is about as bad as you can get > > Normal does: > > follow_page(FOLL_GET) +1 > > isolate_lru_page() +1 > > put_page() -1 > > ending up with a page with a single reference (for anon, or one extra > each for the mapping and buffer). Would it make sense to have the normal page migration code always work with the extra refcount, so we do not have to introduce a new MIGRATE_FAULT migration mode? On the other hand, compaction does not take the extra reference... Another alternative might be to do the put_page inside do_prot_none_numa(). That would be analogous to do_wp_page disposing of the old page for the caller. I am not real happy about NUMA migration introducing its own migration mode... -- All rights reversed From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932260Ab2JSRyo (ORCPT ); Fri, 19 Oct 2012 13:54:44 -0400 Received: from casper.infradead.org ([85.118.1.10]:53064 "EHLO casper.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752292Ab2JSRym convert rfc822-to-8bit (ORCPT ); Fri, 19 Oct 2012 13:54:42 -0400 Message-ID: <1350669236.2768.66.camel@twins> Subject: Re: question on NUMA page migration From: Peter Zijlstra To: Rik van Riel Cc: Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List Date: Fri, 19 Oct 2012 19:53:56 +0200 In-Reply-To: <50818A41.7030909@redhat.com> References: <5081777A.8050104@redhat.com> <1350664742.2768.40.camel@twins> <50818A41.7030909@redhat.com> Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7BIT X-Mailer: Evolution 3.2.2- Mime-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 2012-10-19 at 13:13 -0400, Rik van Riel wrote: > Would it make sense to have the normal page migration code always > work with the extra refcount, so we do not have to introduce a new > MIGRATE_FAULT migration mode? > > On the other hand, compaction does not take the extra reference... Right, it appears to not do this, it gets pages from the pfn and zone->lock and the isolate_lru_page() call is the first reference. > Another alternative might be to do the put_page inside > do_prot_none_numa(). That would be analogous to do_wp_page > disposing of the old page for the caller. It'd have to be inside migrate_misplaced_page(), can't do before isolate_lru_page() or the page might disappear. Doing it after is (obviously) too late. > I am not real happy about NUMA migration introducing its own > migration mode... You didn't seem to mind too much earlier, but I can remove it if you want. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757917Ab2JSSdd (ORCPT ); Fri, 19 Oct 2012 14:33:33 -0400 Received: from mx1.redhat.com ([209.132.183.28]:13956 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753694Ab2JSSdc (ORCPT ); Fri, 19 Oct 2012 14:33:32 -0400 Message-ID: <50819CED.30803@redhat.com> Date: Fri, 19 Oct 2012 14:33:17 -0400 From: Rik van Riel User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: Peter Zijlstra CC: Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List Subject: Re: question on NUMA page migration References: <5081777A.8050104@redhat.com> <1350664742.2768.40.camel@twins> <50818A41.7030909@redhat.com> <1350669236.2768.66.camel@twins> In-Reply-To: <1350669236.2768.66.camel@twins> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/19/2012 01:53 PM, Peter Zijlstra wrote: > On Fri, 2012-10-19 at 13:13 -0400, Rik van Riel wrote: >> Another alternative might be to do the put_page inside >> do_prot_none_numa(). That would be analogous to do_wp_page >> disposing of the old page for the caller. > > It'd have to be inside migrate_misplaced_page(), can't do before > isolate_lru_page() or the page might disappear. Doing it after is > (obviously) too late. Keeping an extra refcount on the page might _still_ result in it disappearing from the process by some other means, in-between you grabbing the refcount and invoking migration of the page. >> I am not real happy about NUMA migration introducing its own >> migration mode... > > You didn't seem to mind too much earlier, but I can remove it if you > want. Could have been reviewing fatigue :) And yes, it would have been nice to not have a special migration mode for sched/numa. Speaking of, when do you guys plan to submit a (cleaned up) version of the sched/numa patch series for review on lkml? -- All rights reversed From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759523Ab2JTBXw (ORCPT ); Fri, 19 Oct 2012 21:23:52 -0400 Received: from mail-wi0-f178.google.com ([209.85.212.178]:49571 "EHLO mail-wi0-f178.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755490Ab2JTBXv (ORCPT ); Fri, 19 Oct 2012 21:23:51 -0400 Date: Sat, 20 Oct 2012 03:23:46 +0200 From: Ingo Molnar To: Rik van Riel Cc: Peter Zijlstra , Andrea Arcangeli , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List Subject: Re: question on NUMA page migration Message-ID: <20121020012345.GA24667@gmail.com> References: <5081777A.8050104@redhat.com> <1350664742.2768.40.camel@twins> <50818A41.7030909@redhat.com> <1350669236.2768.66.camel@twins> <50819CED.30803@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50819CED.30803@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org * Rik van Riel wrote: > On 10/19/2012 01:53 PM, Peter Zijlstra wrote: > >On Fri, 2012-10-19 at 13:13 -0400, Rik van Riel wrote: > > >>Another alternative might be to do the put_page inside > >>do_prot_none_numa(). That would be analogous to do_wp_page > >>disposing of the old page for the caller. > > > >It'd have to be inside migrate_misplaced_page(), can't do before > >isolate_lru_page() or the page might disappear. Doing it after is > >(obviously) too late. > > Keeping an extra refcount on the page might _still_ > result in it disappearing from the process by some > other means, in-between you grabbing the refcount > and invoking migration of the page. > > >>I am not real happy about NUMA migration introducing its own > >>migration mode... > > > >You didn't seem to mind too much earlier, but I can remove it if you > >want. > > Could have been reviewing fatigue :) :-) > And yes, it would have been nice to not have a special > migration mode for sched/numa. > > Speaking of, when do you guys plan to submit a (cleaned up) > version of the sched/numa patch series for review on lkml? Which commit(s) worry you specifically? Thanks, Ingo From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752485Ab2JTQCs (ORCPT ); Sat, 20 Oct 2012 12:02:48 -0400 Received: from mx1.redhat.com ([209.132.183.28]:11467 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752321Ab2JTQCr (ORCPT ); Sat, 20 Oct 2012 12:02:47 -0400 Message-ID: <5082CB18.6060300@redhat.com> Date: Sat, 20 Oct 2012 12:02:32 -0400 From: Rik van Riel User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: Ingo Molnar CC: Peter Zijlstra , Andrea Arcangeli , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List Subject: Re: question on NUMA page migration References: <5081777A.8050104@redhat.com> <1350664742.2768.40.camel@twins> <50818A41.7030909@redhat.com> <1350669236.2768.66.camel@twins> <50819CED.30803@redhat.com> <20121020012345.GA24667@gmail.com> In-Reply-To: <20121020012345.GA24667@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/19/2012 09:23 PM, Ingo Molnar wrote: > > * Rik van Riel wrote: > >> On 10/19/2012 01:53 PM, Peter Zijlstra wrote: >>> On Fri, 2012-10-19 at 13:13 -0400, Rik van Riel wrote: >> >>>> Another alternative might be to do the put_page inside >>>> do_prot_none_numa(). That would be analogous to do_wp_page >>>> disposing of the old page for the caller. >>> >>> It'd have to be inside migrate_misplaced_page(), can't do before >>> isolate_lru_page() or the page might disappear. Doing it after is >>> (obviously) too late. >> >> Keeping an extra refcount on the page might _still_ >> result in it disappearing from the process by some >> other means, in-between you grabbing the refcount >> and invoking migration of the page. >> >>>> I am not real happy about NUMA migration introducing its own >>>> migration mode... >>> >>> You didn't seem to mind too much earlier, but I can remove it if you >>> want. >> >> Could have been reviewing fatigue :) > > :-) > >> And yes, it would have been nice to not have a special >> migration mode for sched/numa. >> >> Speaking of, when do you guys plan to submit a (cleaned up) >> version of the sched/numa patch series for review on lkml? > > Which commit(s) worry you specifically? One of them would be the commit that introduces MIGRATE_FAULT. Adding it in one patch, and removing it into a next just makes things harder on the reviewers. 03a040f6c17ab81659579ba6abe267c0562097e4 If the changesets with NUMA syscalls are still in your tree's history, they should not be submitted as part of the patch series, either. -- All rights reversed From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755396Ab2JUCjf (ORCPT ); Sat, 20 Oct 2012 22:39:35 -0400 Received: from mail-ie0-f174.google.com ([209.85.223.174]:50718 "EHLO mail-ie0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754633Ab2JUCje (ORCPT ); Sat, 20 Oct 2012 22:39:34 -0400 Message-ID: <50836060.4050408@gmail.com> Date: Sun, 21 Oct 2012 10:39:28 +0800 From: Ni zhan Chen User-Agent: Mozilla/5.0 (X11; Linux i686; rv:16.0) Gecko/20121011 Thunderbird/16.0.1 MIME-Version: 1.0 To: Rik van Riel CC: Peter Zijlstra , Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List Subject: Re: question on NUMA page migration References: <5081777A.8050104@redhat.com> In-Reply-To: <5081777A.8050104@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/19/2012 11:53 PM, Rik van Riel wrote: > Hi Andrea, Peter, > > I have a question on page refcounting in your NUMA > page migration code. > > In Peter's case, I wonder why you introduce a new > MIGRATE_FAULT migration mode. If the normal page > migration / compaction logic can do without taking > an extra reference count, why does your code need it? Hi Rik van Riel, This is which part of codes? Why I can't find MIGRATE_FAULT in latest v3.7-rc2? Regards, Chen > > In Andrea's case, we have a comment suggesting an > extra refcount is needed, immediately followed by > a put_page: > > /* > * Pin the head subpage at least until the first > * __isolate_lru_page succeeds (__isolate_lru_page pins it > * again when it succeeds). If we unpin before > * __isolate_lru_page successd, the page could be freed and > * reallocated out from under us. Thus our previous checks on > * the page, and the split_huge_page, would be worthless. > * > * We really only need to do this if "ret > 0" but it doesn't > * hurt to do it unconditionally as nobody can reference > * "page" anymore after this and so we can avoid an "if (ret > > * 0)" branch here. > */ > put_page(page); > > This also confuses me. > > If we do not need the extra refcount (and I do not > understand why NUMA migrate-on-fault needs one more > refcount than normal page migration), we can get > rid of the MIGRATE_FAULT mode. > > If we do need the extra refcount, why is normal > page migration safe? :) > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756371Ab2JUCk2 (ORCPT ); Sat, 20 Oct 2012 22:40:28 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40237 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755088Ab2JUCk1 (ORCPT ); Sat, 20 Oct 2012 22:40:27 -0400 Message-ID: <5083608E.6040209@redhat.com> Date: Sat, 20 Oct 2012 22:40:14 -0400 From: Rik van Riel User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: Ni zhan Chen CC: Peter Zijlstra , Andrea Arcangeli , Ingo Molnar , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List Subject: Re: question on NUMA page migration References: <5081777A.8050104@redhat.com> <50836060.4050408@gmail.com> In-Reply-To: <50836060.4050408@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/20/2012 10:39 PM, Ni zhan Chen wrote: > On 10/19/2012 11:53 PM, Rik van Riel wrote: >> Hi Andrea, Peter, >> >> I have a question on page refcounting in your NUMA >> page migration code. >> >> In Peter's case, I wonder why you introduce a new >> MIGRATE_FAULT migration mode. If the normal page >> migration / compaction logic can do without taking >> an extra reference count, why does your code need it? > > Hi Rik van Riel, > > This is which part of codes? Why I can't find MIGRATE_FAULT in latest > v3.7-rc2? It is in tip.git in the numa/core branch. -- All rights reversed From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753491Ab2JUMaQ (ORCPT ); Sun, 21 Oct 2012 08:30:16 -0400 Received: from mail-wi0-f170.google.com ([209.85.212.170]:65376 "EHLO mail-wi0-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753228Ab2JUMaO (ORCPT ); Sun, 21 Oct 2012 08:30:14 -0400 Date: Sun, 21 Oct 2012 14:30:08 +0200 From: Ingo Molnar To: Rik van Riel Cc: Peter Zijlstra , Andrea Arcangeli , Linux Memory Management List , Mel Gorman , Thomas Gleixner , Linux kernel Mailing List Subject: Re: question on NUMA page migration Message-ID: <20121021123008.GA19229@gmail.com> References: <5081777A.8050104@redhat.com> <1350664742.2768.40.camel@twins> <50818A41.7030909@redhat.com> <1350669236.2768.66.camel@twins> <50819CED.30803@redhat.com> <20121020012345.GA24667@gmail.com> <5082CB18.6060300@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5082CB18.6060300@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org * Rik van Riel wrote: > On 10/19/2012 09:23 PM, Ingo Molnar wrote: > > > >* Rik van Riel wrote: > > > >>On 10/19/2012 01:53 PM, Peter Zijlstra wrote: > >>>On Fri, 2012-10-19 at 13:13 -0400, Rik van Riel wrote: > >> > >>>>Another alternative might be to do the put_page inside > >>>>do_prot_none_numa(). That would be analogous to do_wp_page > >>>>disposing of the old page for the caller. > >>> > >>>It'd have to be inside migrate_misplaced_page(), can't do before > >>>isolate_lru_page() or the page might disappear. Doing it after is > >>>(obviously) too late. > >> > >>Keeping an extra refcount on the page might _still_ > >>result in it disappearing from the process by some > >>other means, in-between you grabbing the refcount > >>and invoking migration of the page. > >> > >>>>I am not real happy about NUMA migration introducing its own > >>>>migration mode... > >>> > >>>You didn't seem to mind too much earlier, but I can remove it if you > >>>want. > >> > >>Could have been reviewing fatigue :) > > > >:-) > > > >>And yes, it would have been nice to not have a special > >>migration mode for sched/numa. > >> > >>Speaking of, when do you guys plan to submit a (cleaned up) > >>version of the sched/numa patch series for review on lkml? > > > >Which commit(s) worry you specifically? > > One of them would be the commit that introduces MIGRATE_FAULT. MIGRATE_FAULT is still used. > Adding it in one patch, and removing it into a next just makes > things harder on the reviewers. Yes. > 03a040f6c17ab81659579ba6abe267c0562097e4 It's still used: comet:~/tip> git grep MIGRATE_FAULT include/linux/migrate_mode.h: * MIGRATE_FAULT called from the fault path to migrate-on-fault for mempolicy include/linux/migrate_mode.h: MIGRATE_FAULT, mm/migrate.c: if (mode != MIGRATE_ASYNC && mode != MIGRATE_FAULT) { mm/migrate.c: if (mode == MIGRATE_FAULT) { mm/migrate.c: * MIGRATE_FAULT has an extra reference on the page and mm/migrate.c: if ((mode == MIGRATE_ASYNC || mode == MIGRATE_FAULT) && head && mm/migrate.c: if (mode != MIGRATE_ASYNC && mode != MIGRATE_FAULT) mm/migrate.c: if (!force || mode == MIGRATE_ASYNC || mode == MIGRATE_FAULT) mm/migrate.c: ret = __unmap_and_move(page, newpage, 0, 0, MIGRATE_FAULT); > If the changesets with NUMA syscalls are still in your tree's > history, they should not be submitted as part of the patch > series, either. No, the syscalls have not been there for quite some time. If you have trouble with any specific commit, please quote the specific SHA1 so that I can have a look and resolve any specific concerns. Otherwise, lets continue with the integration? Thanks, Ingo From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753515Ab2JUMby (ORCPT ); Sun, 21 Oct 2012 08:31:54 -0400 Received: from mail-wi0-f172.google.com ([209.85.212.172]:49357 "EHLO mail-wi0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752929Ab2JUMbx (ORCPT ); Sun, 21 Oct 2012 08:31:53 -0400 Date: Sun, 21 Oct 2012 14:31:47 +0200 From: Ingo Molnar To: Rik van Riel Cc: Ni zhan Chen , Peter Zijlstra , Andrea Arcangeli , Linux Memory Management List , Mel Gorman , Linux kernel Mailing List Subject: Re: question on NUMA page migration Message-ID: <20121021123147.GB19229@gmail.com> References: <5081777A.8050104@redhat.com> <50836060.4050408@gmail.com> <5083608E.6040209@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5083608E.6040209@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org * Rik van Riel wrote: > On 10/20/2012 10:39 PM, Ni zhan Chen wrote: > >On 10/19/2012 11:53 PM, Rik van Riel wrote: > >>Hi Andrea, Peter, > >> > >>I have a question on page refcounting in your NUMA > >>page migration code. > >> > >>In Peter's case, I wonder why you introduce a new > >>MIGRATE_FAULT migration mode. If the normal page > >>migration / compaction logic can do without taking > >>an extra reference count, why does your code need it? > > > >Hi Rik van Riel, > > > >This is which part of codes? Why I can't find MIGRATE_FAULT in latest > >v3.7-rc2? > > It is in tip.git in the numa/core branch. The Git access URI is: git pull git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git numa/core git clone git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git numa/core Thanks, Ingo