From mboxrd@z Thu Jan 1 00:00:00 1970 From: Nitesh Narayan Lal Subject: Re: [RFC PATCH 4/4] mm: Add merge page notifier Date: Mon, 11 Feb 2019 11:24:02 -0500 Message-ID: References: <20190204181118.12095.38300.stgit@localhost.localdomain> <20190204181558.12095.83484.stgit@localhost.localdomain> <20190209195325-mutt-send-email-mst@kernel.org> <7fcb61d6-64f0-f3ae-5e32-0e9f587fdd49@redhat.com> <20190211091623-mutt-send-email-mst@kernel.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="qXx3fZQQSn93zl79DSJ7RyaJquTUjxsqL" Cc: Alexander Duyck , linux-mm@kvack.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, rkrcmar@redhat.com, alexander.h.duyck@linux.intel.com, x86@kernel.org, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, pbonzini@redhat.com, tglx@linutronix.de, akpm@linux-foundation.org To: "Michael S. Tsirkin" Return-path: In-Reply-To: <20190211091623-mutt-send-email-mst@kernel.org> Sender: linux-kernel-owner@vger.kernel.org List-Id: kvm.vger.kernel.org This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --qXx3fZQQSn93zl79DSJ7RyaJquTUjxsqL Content-Type: multipart/mixed; boundary="IyEb9NGqXcudpVZ93K6BModc8xL8gsLEU"; protected-headers="v1" From: Nitesh Narayan Lal To: "Michael S. Tsirkin" Cc: Alexander Duyck , linux-mm@kvack.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, rkrcmar@redhat.com, alexander.h.duyck@linux.intel.com, x86@kernel.org, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, pbonzini@redhat.com, tglx@linutronix.de, akpm@linux-foundation.org Message-ID: Subject: Re: [RFC PATCH 4/4] mm: Add merge page notifier References: <20190204181118.12095.38300.stgit@localhost.localdomain> <20190204181558.12095.83484.stgit@localhost.localdomain> <20190209195325-mutt-send-email-mst@kernel.org> <7fcb61d6-64f0-f3ae-5e32-0e9f587fdd49@redhat.com> <20190211091623-mutt-send-email-mst@kernel.org> In-Reply-To: <20190211091623-mutt-send-email-mst@kernel.org> --IyEb9NGqXcudpVZ93K6BModc8xL8gsLEU Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Content-Language: en-US On 2/11/19 9:17 AM, Michael S. Tsirkin wrote: > On Mon, Feb 11, 2019 at 08:30:03AM -0500, Nitesh Narayan Lal wrote: >> On 2/9/19 7:57 PM, Michael S. Tsirkin wrote: >>> On Mon, Feb 04, 2019 at 10:15:58AM -0800, Alexander Duyck wrote: >>>> From: Alexander Duyck >>>> >>>> Because the implementation was limiting itself to only providing hin= ts on >>>> pages huge TLB order sized or larger we introduced the possibility f= or free >>>> pages to slip past us because they are freed as something less then >>>> huge TLB in size and aggregated with buddies later. >>>> >>>> To address that I am adding a new call arch_merge_page which is call= ed >>>> after __free_one_page has merged a pair of pages to create a higher = order >>>> page. By doing this I am able to fill the gap and provide full cover= age for >>>> all of the pages huge TLB order or larger. >>>> >>>> Signed-off-by: Alexander Duyck >>> Looks like this will be helpful whenever active free page >>> hints are added. So I think it's a good idea to >>> add a hook. >>> >>> However, could you split adding the hook to a separate >>> patch from the KVM hypercall based implementation? >>> >>> Then e.g. Nilal's patches could reuse it too. >> With the current design of my patch-set, if I use this hook to report >> free pages. I will end up making redundant hints for the same pfns. >> >> This is because the pages once freed by the host, are returned back to= >> the buddy. > Suggestions on how you'd like to fix this? You do need this if > you introduce a size cut-off right? I do, there are two ways to go about it. One is to=C2=A0 use this and have a flag in the page structure indicating= whether that page has been freed/used or not. Though I am not sure if this will be acceptable upstream. Second is to find another place to invoke guest_free_page() post buddy merging. > >>> >>>> --- >>>> arch/x86/include/asm/page.h | 12 ++++++++++++ >>>> arch/x86/kernel/kvm.c | 28 ++++++++++++++++++++++++++++ >>>> include/linux/gfp.h | 4 ++++ >>>> mm/page_alloc.c | 2 ++ >>>> 4 files changed, 46 insertions(+) >>>> >>>> diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page= =2Eh >>>> index 4487ad7a3385..9540a97c9997 100644 >>>> --- a/arch/x86/include/asm/page.h >>>> +++ b/arch/x86/include/asm/page.h >>>> @@ -29,6 +29,18 @@ static inline void arch_free_page(struct page *pa= ge, unsigned int order) >>>> if (static_branch_unlikely(&pv_free_page_hint_enabled)) >>>> __arch_free_page(page, order); >>>> } >>>> + >>>> +struct zone; >>>> + >>>> +#define HAVE_ARCH_MERGE_PAGE >>>> +void __arch_merge_page(struct zone *zone, struct page *page, >>>> + unsigned int order); >>>> +static inline void arch_merge_page(struct zone *zone, struct page *= page, >>>> + unsigned int order) >>>> +{ >>>> + if (static_branch_unlikely(&pv_free_page_hint_enabled)) >>>> + __arch_merge_page(zone, page, order); >>>> +} >>>> #endif >>>> =20 >>>> #include >>>> diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c >>>> index 09c91641c36c..957bb4f427bb 100644 >>>> --- a/arch/x86/kernel/kvm.c >>>> +++ b/arch/x86/kernel/kvm.c >>>> @@ -785,6 +785,34 @@ void __arch_free_page(struct page *page, unsign= ed int order) >>>> PAGE_SIZE << order); >>>> } >>>> =20 >>>> +void __arch_merge_page(struct zone *zone, struct page *page, >>>> + unsigned int order) >>>> +{ >>>> + /* >>>> + * The merging logic has merged a set of buddies up to the >>>> + * KVM_PV_UNUSED_PAGE_HINT_MIN_ORDER. Since that is the case, take= >>>> + * advantage of this moment to notify the hypervisor of the free >>>> + * memory. >>>> + */ >>>> + if (order !=3D KVM_PV_UNUSED_PAGE_HINT_MIN_ORDER) >>>> + return; >>>> + >>>> + /* >>>> + * Drop zone lock while processing the hypercall. This >>>> + * should be safe as the page has not yet been added >>>> + * to the buddy list as of yet and all the pages that >>>> + * were merged have had their buddy/guard flags cleared >>>> + * and their order reset to 0. >>>> + */ >>>> + spin_unlock(&zone->lock); >>>> + >>>> + kvm_hypercall2(KVM_HC_UNUSED_PAGE_HINT, page_to_phys(page), >>>> + PAGE_SIZE << order); >>>> + >>>> + /* reacquire lock and resume freeing memory */ >>>> + spin_lock(&zone->lock); >>>> +} >>>> + >>>> #ifdef CONFIG_PARAVIRT_SPINLOCKS >>>> =20 >>>> /* Kick a cpu by its apicid. Used to wake up a halted vcpu */ >>>> diff --git a/include/linux/gfp.h b/include/linux/gfp.h >>>> index fdab7de7490d..4746d5560193 100644 >>>> --- a/include/linux/gfp.h >>>> +++ b/include/linux/gfp.h >>>> @@ -459,6 +459,10 @@ static inline struct zonelist *node_zonelist(in= t nid, gfp_t flags) >>>> #ifndef HAVE_ARCH_FREE_PAGE >>>> static inline void arch_free_page(struct page *page, int order) { }= >>>> #endif >>>> +#ifndef HAVE_ARCH_MERGE_PAGE >>>> +static inline void >>>> +arch_merge_page(struct zone *zone, struct page *page, int order) { = } >>>> +#endif >>>> #ifndef HAVE_ARCH_ALLOC_PAGE >>>> static inline void arch_alloc_page(struct page *page, int order) { = } >>>> #endif >>>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c >>>> index c954f8c1fbc4..7a1309b0b7c5 100644 >>>> --- a/mm/page_alloc.c >>>> +++ b/mm/page_alloc.c >>>> @@ -913,6 +913,8 @@ static inline void __free_one_page(struct page *= page, >>>> page =3D page + (combined_pfn - pfn); >>>> pfn =3D combined_pfn; >>>> order++; >>>> + >>>> + arch_merge_page(zone, page, order); >>>> } >>>> if (max_order < MAX_ORDER) { >>>> /* If we are here, it means order is >=3D pageblock_order. >> --=20 >> Regards >> Nitesh >> > > --=20 Regards Nitesh --IyEb9NGqXcudpVZ93K6BModc8xL8gsLEU-- --qXx3fZQQSn93zl79DSJ7RyaJquTUjxsqL Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEkXcoRVGaqvbHPuAGo4ZA3AYyozkFAlxhoaIACgkQo4ZA3AYy ozmlFg/+JqGEyRG/6eSJo76R07PR4myO0ukkvUcxhPRYRQPFKG+64BaaZam/k58p jVYWns005s/F4ZvmavRpwY5VgmEOiBMpjijEn886elQfGVG95hAyxiFAgbsNYxmB lSX3KYc5VrICx71byZenddUZPSz1t8vXKdOP4x2r95aPFOmEd0kV7GdL/zaq2EQ2 YGlVnKd3dpwkSb4h0HZFOKcZ2g7XsZfCkHCxKBMWuFSu9C0XwRTD76MgyKtOc926 wxrmel0oJnRxbNF9jCj70qaaTuWYvwrIMPySrHlid0VM3t0mUoj2cgfatnAdj+3e rI4NGZdj8niD4rvbGMkCs0Rxs8TIXyiywxjFjVrp/rEnOJI/Vmi0T7MrlzYWab4Z lCQRI/4vNI4CaaBannrN8ctD3Fl4dOH+DM+B9ifyk9mns0SkQqy96Kjroh3o7CKG x9kGKzdSUqyJXb+E61QtaCyWpFMHFdOlmvv5AtYWQyNvoJEcQ5anytzX2BM7WZbe oo/WXq84O0V87UVId0JkCAJoB1jGrc1WsxpqM3+vzZvdaV+carbQrWx4ZgVItXov FsoWQMoPLexeJdZk+Af+HgJiJimOfZEyLTbyS2Vh6wAcewLGbhbImKYt6xjNTfd8 Le9j2oqkdEtHF+Efcate4JFjvgnhV+ygWkuixjISTRY9Vf1AvVs= =R7Q8 -----END PGP SIGNATURE----- --qXx3fZQQSn93zl79DSJ7RyaJquTUjxsqL--