From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Message-ID: <1449092226.31589.50.camel@hpe.com> Subject: Re: [PATCH] mm: Fix mmap MAP_POPULATE for DAX pmd mapping From: Toshi Kani Date: Wed, 02 Dec 2015 14:37:06 -0700 In-Reply-To: References: <1448309082-20851-1-git-send-email-toshi.kani@hpe.com> <1449022764.31589.24.camel@hpe.com> <1449078237.31589.30.camel@hpe.com> <1449084362.31589.37.camel@hpe.com> <1449086521.31589.39.camel@hpe.com> <1449087125.31589.45.camel@hpe.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org To: Dan Williams Cc: Andrew Morton , "Kirill A. Shutemov" , Matthew Wilcox , Ross Zwisler , mauricio.porto@hpe.com, Linux MM , linux-fsdevel , "linux-nvdimm@lists.01.org" , "linux-kernel@vger.kernel.org" List-ID: On Wed, 2015-12-02 at 11:57 -0800, Dan Williams wrote: > On Wed, Dec 2, 2015 at 12:12 PM, Toshi Kani wrote: > > On Wed, 2015-12-02 at 13:02 -0700, Toshi Kani wrote: > > > On Wed, 2015-12-02 at 11:00 -0800, Dan Williams wrote: > > > > On Wed, Dec 2, 2015 at 11:26 AM, Toshi Kani wrote: > > > > > On Wed, 2015-12-02 at 10:06 -0800, Dan Williams wrote: > > > > > > On Wed, Dec 2, 2015 at 9:01 AM, Dan Williams < > > > > > > dan.j.williams@intel.com> > > > > > > wrote: > > > > > > > On Wed, Dec 2, 2015 at 9:43 AM, Toshi Kani > > > > > > > wrote: > > > > > > > > Oh, I see. I will setup the memmap array and run the tests > > > > > > > > again. > > > > > > > > > > > > > > > > But, why does the PMD mapping depend on the memmap array? We > > > > > > > > have observed major performance improvement with PMD. This > > > > > > > > feature should always be enabled with DAX regardless of the > > > > > > > > option to allocate the memmap array. > > > > > > > > > > > > > > > > > > > > > > Several factors drove this decision, I'm open to considering > > > > > > > alternatives but here's the reasoning: > > > > > > > > > > > > > > 1/ DAX pmd mappings caused crashes in the get_user_pages path > > > > > > > leading to commit e82c9ed41e8 "dax: disable pmd mappings". The > > > > > > > reason pte mappings don't crash and instead trigger -EFAULT is due > > > > > > > to the _PAGE_SPECIAL pte bit. > > > > > > > > > > > > > > 2/ To enable get_user_pages for DAX, in both the page and huge > > > > > > > -page case, we need a new pte bit _PAGE_DEVMAP. > > > > > > > > > > > > > > 3/ Given the pte bits are hard to come I'm assuming we won't get > > > > > > > two, i.e. both _PAGE_DEVMAP and a new _PAGE_SPECIAL for pmds. > > > > > > > Even if we could get a _PAGE_SPECIAL for pmds I'm not in favor of > > > > > > > pursuing it. > > > > > > > > > > > > Actually, Dave says they aren't that hard to come by for pmds, so we > > > > > > could go add _PMD_SPECIAL if we really wanted to support the limited > > > > > > page-less DAX-pmd case. > > > > > > > > > > > > But I'm still of the opinion that we run away from the page-less > > > > > > case until it can be made a full class citizen with O_DIRECT for pfn > > > > > > support. > > > > > > > > > > I may be missing something, but per vm_normal_page(), I think > > > > > _PAGE_SPECIAL can be substituted by the following check when we do not > > > > > have the memmap. > > > > > > > > > > if ((vma->vm_flags & VM_PFNMAP) || > > > > > ((vma->vm_flags & VM_MIXEDMAP) && (!pfn_valid(pfn)))) { > > > > > > > > > > This is what I did in this patch for follow_trans_huge_pmd(), although > > > > > I missed the pfn_valid() check. > > > > > > > > That works for __get_user_pages but not __get_user_pages_fast where we > > > > don't have access to the vma. > > > > > > __get_user_page_fast already refers current->mm, so we should be able to > > > get the vma, and pass it down to gup_pud_range(). > > > > Alternatively, we can obtain the vma from current->mm in gup_huge_pmd() when > > the !pfn_valid() condition is met, so that we do not add the code to the > > main path of __get_user_pages_fast. > > The whole point of __get_user_page_fast() is to avoid the overhead of > taking the mm semaphore to access the vma. _PAGE_SPECIAL simply tells > __get_user_pages_fast that it needs to fallback to the > __get_user_pages slow path. I see. Then, I think gup_huge_pmd() can simply return 0 when !pfn_valid(), instead of VM_BUG_ON. Thanks, -Toshi -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755963AbbLBUmI (ORCPT ); Wed, 2 Dec 2015 15:42:08 -0500 Received: from g4t3426.houston.hp.com ([15.201.208.54]:3620 "EHLO g4t3426.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751775AbbLBUmG (ORCPT ); Wed, 2 Dec 2015 15:42:06 -0500 Message-ID: <1449092226.31589.50.camel@hpe.com> Subject: Re: [PATCH] mm: Fix mmap MAP_POPULATE for DAX pmd mapping From: Toshi Kani To: Dan Williams Cc: Andrew Morton , "Kirill A. Shutemov" , Matthew Wilcox , Ross Zwisler , mauricio.porto@hpe.com, Linux MM , linux-fsdevel , "linux-nvdimm@lists.01.org" , "linux-kernel@vger.kernel.org" Date: Wed, 02 Dec 2015 14:37:06 -0700 In-Reply-To: References: <1448309082-20851-1-git-send-email-toshi.kani@hpe.com> <1449022764.31589.24.camel@hpe.com> <1449078237.31589.30.camel@hpe.com> <1449084362.31589.37.camel@hpe.com> <1449086521.31589.39.camel@hpe.com> <1449087125.31589.45.camel@hpe.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.16.5 (3.16.5-3.fc22) Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 2015-12-02 at 11:57 -0800, Dan Williams wrote: > On Wed, Dec 2, 2015 at 12:12 PM, Toshi Kani wrote: > > On Wed, 2015-12-02 at 13:02 -0700, Toshi Kani wrote: > > > On Wed, 2015-12-02 at 11:00 -0800, Dan Williams wrote: > > > > On Wed, Dec 2, 2015 at 11:26 AM, Toshi Kani wrote: > > > > > On Wed, 2015-12-02 at 10:06 -0800, Dan Williams wrote: > > > > > > On Wed, Dec 2, 2015 at 9:01 AM, Dan Williams < > > > > > > dan.j.williams@intel.com> > > > > > > wrote: > > > > > > > On Wed, Dec 2, 2015 at 9:43 AM, Toshi Kani > > > > > > > wrote: > > > > > > > > Oh, I see. I will setup the memmap array and run the tests > > > > > > > > again. > > > > > > > > > > > > > > > > But, why does the PMD mapping depend on the memmap array? We > > > > > > > > have observed major performance improvement with PMD. This > > > > > > > > feature should always be enabled with DAX regardless of the > > > > > > > > option to allocate the memmap array. > > > > > > > > > > > > > > > > > > > > > > Several factors drove this decision, I'm open to considering > > > > > > > alternatives but here's the reasoning: > > > > > > > > > > > > > > 1/ DAX pmd mappings caused crashes in the get_user_pages path > > > > > > > leading to commit e82c9ed41e8 "dax: disable pmd mappings". The > > > > > > > reason pte mappings don't crash and instead trigger -EFAULT is due > > > > > > > to the _PAGE_SPECIAL pte bit. > > > > > > > > > > > > > > 2/ To enable get_user_pages for DAX, in both the page and huge > > > > > > > -page case, we need a new pte bit _PAGE_DEVMAP. > > > > > > > > > > > > > > 3/ Given the pte bits are hard to come I'm assuming we won't get > > > > > > > two, i.e. both _PAGE_DEVMAP and a new _PAGE_SPECIAL for pmds. > > > > > > > Even if we could get a _PAGE_SPECIAL for pmds I'm not in favor of > > > > > > > pursuing it. > > > > > > > > > > > > Actually, Dave says they aren't that hard to come by for pmds, so we > > > > > > could go add _PMD_SPECIAL if we really wanted to support the limited > > > > > > page-less DAX-pmd case. > > > > > > > > > > > > But I'm still of the opinion that we run away from the page-less > > > > > > case until it can be made a full class citizen with O_DIRECT for pfn > > > > > > support. > > > > > > > > > > I may be missing something, but per vm_normal_page(), I think > > > > > _PAGE_SPECIAL can be substituted by the following check when we do not > > > > > have the memmap. > > > > > > > > > > if ((vma->vm_flags & VM_PFNMAP) || > > > > > ((vma->vm_flags & VM_MIXEDMAP) && (!pfn_valid(pfn)))) { > > > > > > > > > > This is what I did in this patch for follow_trans_huge_pmd(), although > > > > > I missed the pfn_valid() check. > > > > > > > > That works for __get_user_pages but not __get_user_pages_fast where we > > > > don't have access to the vma. > > > > > > __get_user_page_fast already refers current->mm, so we should be able to > > > get the vma, and pass it down to gup_pud_range(). > > > > Alternatively, we can obtain the vma from current->mm in gup_huge_pmd() when > > the !pfn_valid() condition is met, so that we do not add the code to the > > main path of __get_user_pages_fast. > > The whole point of __get_user_page_fast() is to avoid the overhead of > taking the mm semaphore to access the vma. _PAGE_SPECIAL simply tells > __get_user_pages_fast that it needs to fallback to the > __get_user_pages slow path. I see. Then, I think gup_huge_pmd() can simply return 0 when !pfn_valid(), instead of VM_BUG_ON. Thanks, -Toshi