From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752457AbbE0Fwg (ORCPT ); Wed, 27 May 2015 01:52:36 -0400 Received: from e23smtp07.au.ibm.com ([202.81.31.140]:50467 "EHLO e23smtp07.au.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751422AbbE0Fwc convert rfc822-to-8bit (ORCPT ); Wed, 27 May 2015 01:52:32 -0400 From: "Aneesh Kumar K.V" To: j.glisse@gmail.com, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , joro@8bytes.org, Mel Gorman , "H. Peter Anvin" , Peter Zijlstra , Andrea Arcangeli , Johannes Weiner , Larry Woodman , Rik van Riel , Dave Airlie , Brendan Conoboy , Joe Donohue , Duncan Poole , Sherry Cheung , Subhash Gutti , John Hubbard , Mark Hairgrove , Lucien Dunning , Cameron Buschardt , Arvind Gopalakrishnan , Haggai Eran , Shachar Raindel , Liran Liss , Roland Dreier , Ben Sander , Greg Stoner , John Bridgman , Michael Mantor , Paul Blinzer , Laurent Morichetti , Alexander Deucher , Oded Gabbay , =?utf-8?B?SsOpcsO0bWU=?= Glisse , Jatin Kumar , linux-rdma@vger.kernel.org Subject: Re: [PATCH 05/36] HMM: introduce heterogeneous memory management v3. In-Reply-To: <1432236705-4209-6-git-send-email-j.glisse@gmail.com> References: <1432236705-4209-1-git-send-email-j.glisse@gmail.com> <1432236705-4209-6-git-send-email-j.glisse@gmail.com> User-Agent: Notmuch/0.19+103~g294bb6d (http://notmuchmail.org) Emacs/24.4.1 (x86_64-pc-linux-gnu) Date: Wed, 27 May 2015 11:20:05 +0530 Message-ID: <87twuylgc2.fsf@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8BIT X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15052705-0025-0000-0000-00000192BACA Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org j.glisse@gmail.com writes: > From: Jérôme Glisse > > This patch only introduce core HMM functions for registering a new > mirror and stopping a mirror as well as HMM device registering and > unregistering. > > The lifecycle of HMM object is handled differently then the one of > mmu_notifier because unlike mmu_notifier there can be concurrent > call from both mm code to HMM code and/or from device driver code > to HMM code. Moreover lifetime of HMM can be uncorrelated from the > lifetime of the process that is being mirror (GPU might take longer > time to cleanup). > ...... > +struct hmm_device_ops { > + /* release() - mirror must stop using the address space. > + * > + * @mirror: The mirror that link process address space with the device. > + * > + * When this is call, device driver must kill all device thread using s/call/called, ? > + * this mirror. Also, this callback is the last thing call by HMM and > + * HMM will not access the mirror struct after this call (ie no more > + * dereference of it so it is safe for the device driver to free it). > + * It is call either from : > + * - mm dying (all process using this mm exiting). > + * - hmm_mirror_unregister() (if no other thread holds a reference) > + * - outcome of some device error reported by any of the device > + * callback against that mirror. > + */ > + void (*release)(struct hmm_mirror *mirror); > +}; > + > + > +/* struct hmm - per mm_struct HMM states. > + * > + * @mm: The mm struct this hmm is associated with. > + * @mirrors: List of all mirror for this mm (one per device). > + * @vm_end: Last valid address for this mm (exclusive). > + * @kref: Reference counter. > + * @rwsem: Serialize the mirror list modifications. > + * @mmu_notifier: The mmu_notifier of this mm. > + * @rcu: For delayed cleanup call from mmu_notifier.release() callback. > + * > + * For each process address space (mm_struct) there is one and only one hmm > + * struct. hmm functions will redispatch to each devices the change made to > + * the process address space. > + * > + * Device driver must not access this structure other than for getting the > + * mm pointer. > + */ ..... > #ifndef AT_VECTOR_SIZE_ARCH > #define AT_VECTOR_SIZE_ARCH 0 > #endif > @@ -451,6 +455,16 @@ struct mm_struct { > #ifdef CONFIG_MMU_NOTIFIER > struct mmu_notifier_mm *mmu_notifier_mm; > #endif > +#ifdef CONFIG_HMM > + /* > + * hmm always register an mmu_notifier we rely on mmu notifier to keep > + * refcount on mm struct as well as forbiding registering hmm on a > + * dying mm > + * > + * This field is set with mmap_sem old in write mode. s/old/held/ ? > + */ > + struct hmm *hmm; > +#endif > #if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS > pgtable_t pmd_huge_pte; /* protected by page_table_lock */ > #endif > diff --git a/kernel/fork.c b/kernel/fork.c > index 0e0ae9a..4083be7 100644 > --- a/kernel/fork.c > +++ b/kernel/fork.c > @@ -27,6 +27,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -597,6 +598,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p) > mm_init_aio(mm); > mm_init_owner(mm, p); > mmu_notifier_mm_init(mm); > + hmm_mm_init(mm); > clear_tlb_flush_pending(mm); > #if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS > mm->pmd_huge_pte = NULL; > diff --git a/mm/Kconfig b/mm/Kconfig > index 52ffb86..189e48f 100644 > --- a/mm/Kconfig > +++ b/mm/Kconfig > @@ -653,3 +653,18 @@ config DEFERRED_STRUCT_PAGE_INIT > when kswapd starts. This has a potential performance impact on > processes running early in the lifetime of the systemm until kswapd > finishes the initialisation. > + > +if STAGING > +config HMM > + bool "Enable heterogeneous memory management (HMM)" > + depends on MMU > + select MMU_NOTIFIER > + select GENERIC_PAGE_TABLE What is GENERIC_PAGE_TABLE ? > + default n > + help > + Heterogeneous memory management provide infrastructure for a device > + to mirror a process address space into an hardware mmu or into any > + things supporting pagefault like event. > + > + If unsure, say N to disable hmm. -aneesh