From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrea Arcangeli Subject: Re: [PATCH 04/17] mm: gup: make get_user_pages_fast and __get_user_pages_fast latency conscious Date: Mon, 6 Oct 2014 16:14:28 +0200 Message-ID: <20141006141428.GU2342@redhat.com> References: <1412356087-16115-1-git-send-email-aarcange@redhat.com> <1412356087-16115-5-git-send-email-aarcange@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org To: Linus Torvalds Cc: qemu-devel@nongnu.org, KVM list , Linux Kernel Mailing List , linux-mm , Linux API , Andres Lagar-Cavilla , Dave Hansen , Paolo Bonzini , Rik van Riel , Mel Gorman , Andy Lutomirski , Andrew Morton , Sasha Levin , Hugh Dickins , Peter Feiner , "\\Dr. David Alan Gilbert\\" , Christopher Covington , Johannes Weiner , Android Kernel Team , Robert Love , Dmitry Adamushko List-Id: linux-api@vger.kernel.org Hello, On Fri, Oct 03, 2014 at 11:23:53AM -0700, Linus Torvalds wrote: > On Fri, Oct 3, 2014 at 10:07 AM, Andrea Arcangeli wrote: > > This teaches gup_fast and __gup_fast to re-enable irqs and > > cond_resched() if possible every BATCH_PAGES. > > This is disgusting. > > Many (most?) __gup_fast() users just want a single page, and the > stupid overhead of the multi-page version is already unnecessary. > This just makes things much worse. > > Quite frankly, we should make a single-page version of __gup_fast(), > and convert existign users to use that. After that, the few multi-page > users could have this extra latency control stuff. Ok. I didn't think at a better way to add the latency control other than to reduce nr_pages in a outer loop instead of altering the inner calls, but this is what I got after implementing it... If somebody has a cleaner way to implement the latency control stuff that's welcome and I'd be glad to replace it. > And yes, the single-page version of get_user_pages_fast() is actually > latency-critical. shared futexes hit it hard, and yes, I've seen this > in profiles. KVM would save a few cycles from a single-page version too. I just thought further optimizations could be added later and this was better than nothing. Considering I've no better idea how to implement the latency control stuff, for now I'll just drop this controversial patch, and I'll convert those get_user_pages to gup_unlocked instead of converting them to gup_fast, which is more than enough to obtain the mmap_sem holding scalability improvement (that also solves the mmap_sem trouble for the userfaultfd). gup_unlocked isn't as good as gup_fast but it's at least better than the current get_user_pages(). I got into this gup_fast latency control stuff purely because there were a few get_user_pages that could have been converted to get_user_pages_fast as they were using "current" and "current->mm" as the first two parameters, except for the risk of disabling irq for long. So I tried to do the right thing and fix gup_fast but I'll leave this further optimization queued for later. About the missing commit header for the other patch Paolo already replied to it, to clarify this a bit further in short I expect that FOLL_TRIED flag to be merged through the KVM git tree which already contains it. I'll add a comment to the commit header to specify it. Sorry for the confusion about that patch. Thanks, Andrea -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrea Arcangeli Subject: Re: [PATCH 04/17] mm: gup: make get_user_pages_fast and __get_user_pages_fast latency conscious Date: Mon, 6 Oct 2014 16:14:28 +0200 Message-ID: <20141006141428.GU2342@redhat.com> References: <1412356087-16115-1-git-send-email-aarcange@redhat.com> <1412356087-16115-5-git-send-email-aarcange@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: qemu-devel@nongnu.org, KVM list , Linux Kernel Mailing List , linux-mm , Linux API , Andres Lagar-Cavilla , Dave Hansen , Paolo Bonzini , Rik van Riel , Mel Gorman , Andy Lutomirski , Andrew Morton , Sasha Levin , Hugh Dickins , Peter Feiner , "\\Dr. David Alan Gilbert\\" , Christopher Covington , Johannes Weiner , Android Kernel Team , Robert Love , Dmitry Adamushko Return-path: Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-Id: kvm.vger.kernel.org Hello, On Fri, Oct 03, 2014 at 11:23:53AM -0700, Linus Torvalds wrote: > On Fri, Oct 3, 2014 at 10:07 AM, Andrea Arcangeli wrote: > > This teaches gup_fast and __gup_fast to re-enable irqs and > > cond_resched() if possible every BATCH_PAGES. > > This is disgusting. > > Many (most?) __gup_fast() users just want a single page, and the > stupid overhead of the multi-page version is already unnecessary. > This just makes things much worse. > > Quite frankly, we should make a single-page version of __gup_fast(), > and convert existign users to use that. After that, the few multi-page > users could have this extra latency control stuff. Ok. I didn't think at a better way to add the latency control other than to reduce nr_pages in a outer loop instead of altering the inner calls, but this is what I got after implementing it... If somebody has a cleaner way to implement the latency control stuff that's welcome and I'd be glad to replace it. > And yes, the single-page version of get_user_pages_fast() is actually > latency-critical. shared futexes hit it hard, and yes, I've seen this > in profiles. KVM would save a few cycles from a single-page version too. I just thought further optimizations could be added later and this was better than nothing. Considering I've no better idea how to implement the latency control stuff, for now I'll just drop this controversial patch, and I'll convert those get_user_pages to gup_unlocked instead of converting them to gup_fast, which is more than enough to obtain the mmap_sem holding scalability improvement (that also solves the mmap_sem trouble for the userfaultfd). gup_unlocked isn't as good as gup_fast but it's at least better than the current get_user_pages(). I got into this gup_fast latency control stuff purely because there were a few get_user_pages that could have been converted to get_user_pages_fast as they were using "current" and "current->mm" as the first two parameters, except for the risk of disabling irq for long. So I tried to do the right thing and fix gup_fast but I'll leave this further optimization queued for later. About the missing commit header for the other patch Paolo already replied to it, to clarify this a bit further in short I expect that FOLL_TRIED flag to be merged through the KVM git tree which already contains it. I'll add a comment to the commit header to specify it. Sorry for the confusion about that patch. Thanks, Andrea -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f182.google.com (mail-wi0-f182.google.com [209.85.212.182]) by kanga.kvack.org (Postfix) with ESMTP id 9F6AE6B0069 for ; Mon, 6 Oct 2014 10:15:11 -0400 (EDT) Received: by mail-wi0-f182.google.com with SMTP id n3so4744883wiv.9 for ; Mon, 06 Oct 2014 07:15:09 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id m4si5981273wjb.25.2014.10.06.07.15.08 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Oct 2014 07:15:09 -0700 (PDT) Date: Mon, 6 Oct 2014 16:14:28 +0200 From: Andrea Arcangeli Subject: Re: [PATCH 04/17] mm: gup: make get_user_pages_fast and __get_user_pages_fast latency conscious Message-ID: <20141006141428.GU2342@redhat.com> References: <1412356087-16115-1-git-send-email-aarcange@redhat.com> <1412356087-16115-5-git-send-email-aarcange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: To: Linus Torvalds Cc: qemu-devel@nongnu.org, KVM list , Linux Kernel Mailing List , linux-mm , Linux API , Andres Lagar-Cavilla , Dave Hansen , Paolo Bonzini , Rik van Riel , Mel Gorman , Andy Lutomirski , Andrew Morton , Sasha Levin , Hugh Dickins , Peter Feiner , "\\Dr. David Alan Gilbert\\" , Christopher Covington , Johannes Weiner , Android Kernel Team , Robert Love , Dmitry Adamushko , Neil Brown , Mike Hommey , Taras Glek , Jan Kara , KOSAKI Motohiro , Michel Lespinasse , Minchan Kim , Keith Packard , "Huangpeng (Peter)" , Isaku Yamahata , Anthony Liguori , Stefan Hajnoczi , Wenchao Xia , Andrew Jones , Juan Quintela Hello, On Fri, Oct 03, 2014 at 11:23:53AM -0700, Linus Torvalds wrote: > On Fri, Oct 3, 2014 at 10:07 AM, Andrea Arcangeli wrote: > > This teaches gup_fast and __gup_fast to re-enable irqs and > > cond_resched() if possible every BATCH_PAGES. > > This is disgusting. > > Many (most?) __gup_fast() users just want a single page, and the > stupid overhead of the multi-page version is already unnecessary. > This just makes things much worse. > > Quite frankly, we should make a single-page version of __gup_fast(), > and convert existign users to use that. After that, the few multi-page > users could have this extra latency control stuff. Ok. I didn't think at a better way to add the latency control other than to reduce nr_pages in a outer loop instead of altering the inner calls, but this is what I got after implementing it... If somebody has a cleaner way to implement the latency control stuff that's welcome and I'd be glad to replace it. > And yes, the single-page version of get_user_pages_fast() is actually > latency-critical. shared futexes hit it hard, and yes, I've seen this > in profiles. KVM would save a few cycles from a single-page version too. I just thought further optimizations could be added later and this was better than nothing. Considering I've no better idea how to implement the latency control stuff, for now I'll just drop this controversial patch, and I'll convert those get_user_pages to gup_unlocked instead of converting them to gup_fast, which is more than enough to obtain the mmap_sem holding scalability improvement (that also solves the mmap_sem trouble for the userfaultfd). gup_unlocked isn't as good as gup_fast but it's at least better than the current get_user_pages(). I got into this gup_fast latency control stuff purely because there were a few get_user_pages that could have been converted to get_user_pages_fast as they were using "current" and "current->mm" as the first two parameters, except for the risk of disabling irq for long. So I tried to do the right thing and fix gup_fast but I'll leave this further optimization queued for later. About the missing commit header for the other patch Paolo already replied to it, to clarify this a bit further in short I expect that FOLL_TRIED flag to be merged through the KVM git tree which already contains it. I'll add a comment to the commit header to specify it. Sorry for the confusion about that patch. Thanks, Andrea -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752861AbaJFOPX (ORCPT ); Mon, 6 Oct 2014 10:15:23 -0400 Received: from mx1.redhat.com ([209.132.183.28]:13077 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752441AbaJFOPV (ORCPT ); Mon, 6 Oct 2014 10:15:21 -0400 Date: Mon, 6 Oct 2014 16:14:28 +0200 From: Andrea Arcangeli To: Linus Torvalds Cc: qemu-devel@nongnu.org, KVM list , Linux Kernel Mailing List , linux-mm , Linux API , Andres Lagar-Cavilla , Dave Hansen , Paolo Bonzini , Rik van Riel , Mel Gorman , Andy Lutomirski , Andrew Morton , Sasha Levin , Hugh Dickins , Peter Feiner , "\\Dr. David Alan Gilbert\\" , Christopher Covington , Johannes Weiner , Android Kernel Team , Robert Love , Dmitry Adamushko , Neil Brown , Mike Hommey , Taras Glek , Jan Kara , KOSAKI Motohiro , Michel Lespinasse , Minchan Kim , Keith Packard , "Huangpeng (Peter)" , Isaku Yamahata , Anthony Liguori , Stefan Hajnoczi , Wenchao Xia , Andrew Jones , Juan Quintela Subject: Re: [PATCH 04/17] mm: gup: make get_user_pages_fast and __get_user_pages_fast latency conscious Message-ID: <20141006141428.GU2342@redhat.com> References: <1412356087-16115-1-git-send-email-aarcange@redhat.com> <1412356087-16115-5-git-send-email-aarcange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello, On Fri, Oct 03, 2014 at 11:23:53AM -0700, Linus Torvalds wrote: > On Fri, Oct 3, 2014 at 10:07 AM, Andrea Arcangeli wrote: > > This teaches gup_fast and __gup_fast to re-enable irqs and > > cond_resched() if possible every BATCH_PAGES. > > This is disgusting. > > Many (most?) __gup_fast() users just want a single page, and the > stupid overhead of the multi-page version is already unnecessary. > This just makes things much worse. > > Quite frankly, we should make a single-page version of __gup_fast(), > and convert existign users to use that. After that, the few multi-page > users could have this extra latency control stuff. Ok. I didn't think at a better way to add the latency control other than to reduce nr_pages in a outer loop instead of altering the inner calls, but this is what I got after implementing it... If somebody has a cleaner way to implement the latency control stuff that's welcome and I'd be glad to replace it. > And yes, the single-page version of get_user_pages_fast() is actually > latency-critical. shared futexes hit it hard, and yes, I've seen this > in profiles. KVM would save a few cycles from a single-page version too. I just thought further optimizations could be added later and this was better than nothing. Considering I've no better idea how to implement the latency control stuff, for now I'll just drop this controversial patch, and I'll convert those get_user_pages to gup_unlocked instead of converting them to gup_fast, which is more than enough to obtain the mmap_sem holding scalability improvement (that also solves the mmap_sem trouble for the userfaultfd). gup_unlocked isn't as good as gup_fast but it's at least better than the current get_user_pages(). I got into this gup_fast latency control stuff purely because there were a few get_user_pages that could have been converted to get_user_pages_fast as they were using "current" and "current->mm" as the first two parameters, except for the risk of disabling irq for long. So I tried to do the right thing and fix gup_fast but I'll leave this further optimization queued for later. About the missing commit header for the other patch Paolo already replied to it, to clarify this a bit further in short I expect that FOLL_TRIED flag to be merged through the KVM git tree which already contains it. I'll add a comment to the commit header to specify it. Sorry for the confusion about that patch. Thanks, Andrea From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46148) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xb94I-00083Y-VR for qemu-devel@nongnu.org; Mon, 06 Oct 2014 10:15:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Xb94B-0005ZQ-IA for qemu-devel@nongnu.org; Mon, 06 Oct 2014 10:15:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52551) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xb94B-0005Yt-As for qemu-devel@nongnu.org; Mon, 06 Oct 2014 10:15:11 -0400 Date: Mon, 6 Oct 2014 16:14:28 +0200 From: Andrea Arcangeli Message-ID: <20141006141428.GU2342@redhat.com> References: <1412356087-16115-1-git-send-email-aarcange@redhat.com> <1412356087-16115-5-git-send-email-aarcange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [Qemu-devel] [PATCH 04/17] mm: gup: make get_user_pages_fast and __get_user_pages_fast latency conscious List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Linus Torvalds Cc: Robert Love , Dave Hansen , Jan Kara , KVM list , Neil Brown , Stefan Hajnoczi , qemu-devel@nongnu.org, linux-mm , KOSAKI Motohiro , Michel Lespinasse , Taras Glek , Andrew Jones , Juan Quintela , Hugh Dickins , Isaku Yamahata , Mel Gorman , Sasha Levin , Android Kernel Team , "\\Dr. David Alan Gilbert\\" , "Huangpeng (Peter)" , Andres Lagar-Cavilla , Christopher Covington , Anthony Liguori , Mike Hommey , Keith Packard , Wenchao Xia , Linux API , Linux Kernel Mailing List , Andy Lutomirski , Minchan Kim , Dmitry Adamushko , Johannes Weiner , Paolo Bonzini , Andrew Morton , Peter Feiner Hello, On Fri, Oct 03, 2014 at 11:23:53AM -0700, Linus Torvalds wrote: > On Fri, Oct 3, 2014 at 10:07 AM, Andrea Arcangeli wrote: > > This teaches gup_fast and __gup_fast to re-enable irqs and > > cond_resched() if possible every BATCH_PAGES. > > This is disgusting. > > Many (most?) __gup_fast() users just want a single page, and the > stupid overhead of the multi-page version is already unnecessary. > This just makes things much worse. > > Quite frankly, we should make a single-page version of __gup_fast(), > and convert existign users to use that. After that, the few multi-page > users could have this extra latency control stuff. Ok. I didn't think at a better way to add the latency control other than to reduce nr_pages in a outer loop instead of altering the inner calls, but this is what I got after implementing it... If somebody has a cleaner way to implement the latency control stuff that's welcome and I'd be glad to replace it. > And yes, the single-page version of get_user_pages_fast() is actually > latency-critical. shared futexes hit it hard, and yes, I've seen this > in profiles. KVM would save a few cycles from a single-page version too. I just thought further optimizations could be added later and this was better than nothing. Considering I've no better idea how to implement the latency control stuff, for now I'll just drop this controversial patch, and I'll convert those get_user_pages to gup_unlocked instead of converting them to gup_fast, which is more than enough to obtain the mmap_sem holding scalability improvement (that also solves the mmap_sem trouble for the userfaultfd). gup_unlocked isn't as good as gup_fast but it's at least better than the current get_user_pages(). I got into this gup_fast latency control stuff purely because there were a few get_user_pages that could have been converted to get_user_pages_fast as they were using "current" and "current->mm" as the first two parameters, except for the risk of disabling irq for long. So I tried to do the right thing and fix gup_fast but I'll leave this further optimization queued for later. About the missing commit header for the other patch Paolo already replied to it, to clarify this a bit further in short I expect that FOLL_TRIED flag to be merged through the KVM git tree which already contains it. I'll add a comment to the commit header to specify it. Sorry for the confusion about that patch. Thanks, Andrea