From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 809E6C433F5 for ; Sun, 6 Mar 2022 23:35:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230331AbiCFXgA (ORCPT ); Sun, 6 Mar 2022 18:36:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60586 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234440AbiCFXgA (ORCPT ); Sun, 6 Mar 2022 18:36:00 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A9BF21801 for ; Sun, 6 Mar 2022 15:35:06 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 7BAF6B80EC4 for ; Sun, 6 Mar 2022 23:35:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1DA9CC340EC; Sun, 6 Mar 2022 23:35:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1646609704; bh=Dw3bibUW84NPWXY8zrfv5iTGbpv0fOsoHSi4fdFAEz4=; h=Date:To:From:Subject:From; b=MXVlwfDwmtA8s3kJBHDauU5+U/z149DCJpR5MMcffmQm3OR2KggPfbK71qaPmQbrx xNDEjuXCnIc+CahnJvb+XIwJ36fd2mrnCIyeEwmyImeBalqKafWq31f24RMmCGY8J8 0/4OWFQNIJ2TcRfpyyXb5uetlXpXytkkY8liqiu8= Date: Sun, 06 Mar 2022 15:35:03 -0800 To: mm-commits@vger.kernel.org, weixugc@google.com, vbabka@suse.cz, shakeelb@google.com, rientjes@google.com, mhocko@kernel.org, mgorman@techsingularity.net, hughd@google.com, gthelen@google.com, edumazet@google.com, akpm@linux-foundation.org From: Andrew Morton Subject: + mm-page_alloc-call-check_new_pages-while-zone-spinlock-is-not-held.patch added to -mm tree Message-Id: <20220306233504.1DA9CC340EC@smtp.kernel.org> Precedence: bulk Reply-To: linux-kernel@vger.kernel.org List-ID: X-Mailing-List: mm-commits@vger.kernel.org The patch titled Subject: mm/page_alloc: call check_new_pages() while zone spinlock is not held has been added to the -mm tree. Its filename is mm-page_alloc-call-check_new_pages-while-zone-spinlock-is-not-held.patch This patch should soon appear at https://ozlabs.org/~akpm/mmots/broken-out/mm-page_alloc-call-check_new_pages-while-zone-spinlock-is-not-held.patch and later at https://ozlabs.org/~akpm/mmotm/broken-out/mm-page_alloc-call-check_new_pages-while-zone-spinlock-is-not-held.patch Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next and is updated there every 3-4 working days ------------------------------------------------------ From: Eric Dumazet Subject: mm/page_alloc: call check_new_pages() while zone spinlock is not held For high order pages not using pcp, rmqueue() is currently calling the costly check_new_pages() while zone spinlock is held, and hard irqs masked. This is not needed, we can release the spinlock sooner to reduce zone spinlock contention. Note that after this patch, we call __mod_zone_freepage_state() before deciding to leak the page because it is in bad state. Link: https://lkml.kernel.org/r/20220304170215.1868106-1-eric.dumazet@gmail.com Signed-off-by: Eric Dumazet Reviewed-by: Shakeel Butt Acked-by: David Rientjes Cc: Mel Gorman Cc: Vlastimil Babka Cc: Michal Hocko Cc: Wei Xu Cc: Greg Thelen Cc: Hugh Dickins Signed-off-by: Andrew Morton --- mm/page_alloc.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) --- a/mm/page_alloc.c~mm-page_alloc-call-check_new_pages-while-zone-spinlock-is-not-held +++ a/mm/page_alloc.c @@ -3668,10 +3668,10 @@ struct page *rmqueue(struct zone *prefer * allocate greater than order-1 page units with __GFP_NOFAIL. */ WARN_ON_ONCE((gfp_flags & __GFP_NOFAIL) && (order > 1)); - spin_lock_irqsave(&zone->lock, flags); do { page = NULL; + spin_lock_irqsave(&zone->lock, flags); /* * order-0 request can reach here when the pcplist is skipped * due to non-CMA allocation context. HIGHATOMIC area is @@ -3683,15 +3683,15 @@ struct page *rmqueue(struct zone *prefer if (page) trace_mm_page_alloc_zone_locked(page, order, migratetype); } - if (!page) + if (!page) { page = __rmqueue(zone, order, migratetype, alloc_flags); - } while (page && check_new_pages(page, order)); - if (!page) - goto failed; - - __mod_zone_freepage_state(zone, -(1 << order), - get_pcppage_migratetype(page)); - spin_unlock_irqrestore(&zone->lock, flags); + if (!page) + goto failed; + } + __mod_zone_freepage_state(zone, -(1 << order), + get_pcppage_migratetype(page)); + spin_unlock_irqrestore(&zone->lock, flags); + } while (check_new_pages(page, order)); __count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order); zone_statistics(preferred_zone, zone, 1); _ Patches currently in -mm which might be from edumazet@google.com are mm-page_alloc-call-check_new_pages-while-zone-spinlock-is-not-held.patch