From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2016C43332 for ; Thu, 11 Mar 2021 20:38:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C9BA764F87 for ; Thu, 11 Mar 2021 20:38:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230259AbhCKUhr (ORCPT ); Thu, 11 Mar 2021 15:37:47 -0500 Received: from mail.kernel.org ([198.145.29.99]:53520 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230404AbhCKUhm (ORCPT ); Thu, 11 Mar 2021 15:37:42 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 071CB64F85; Thu, 11 Mar 2021 20:37:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1615495062; bh=2rFMDdHayu1FbIutZg8OgFLguOhskFyLscZbDXZxhwQ=; h=Date:From:To:Subject:From; b=tkxSZrAZoYgUyWKBNDu4EIxc6fI/i+HbMkNluQ6A+mSnP4D2rICg9l74VR/OLZfQN 3VawYNasIEpafmmqXE9KEpLzlPRUT15GqBvvRn8h95jCFBLPO/Y5j/fvfOQY5zUBWt aWK+ku5GqrBerVGnCdDygANJk9FM8xR80uu8XUhM= Date: Thu, 11 Mar 2021 12:37:40 -0800 From: akpm@linux-foundation.org To: david@fromorbit.com, guro@fb.com, hannes@cmpxchg.org, ktkhai@virtuozzo.com, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, shy828301@gmail.com, vbabka@suse.cz Subject: + mm-vmscan-use-per-memcg-nr_deferred-of-shrinker.patch added to -mm tree Message-ID: <20210311203740.3ext3bOVX%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 Precedence: bulk Reply-To: linux-kernel@vger.kernel.org List-ID: X-Mailing-List: mm-commits@vger.kernel.org The patch titled Subject: mm: vmscan: use per memcg nr_deferred of shrinker has been added to the -mm tree. Its filename is mm-vmscan-use-per-memcg-nr_deferred-of-shrinker.patch This patch should soon appear at https://ozlabs.org/~akpm/mmots/broken-out/mm-vmscan-use-per-memcg-nr_deferred-of-shrinker.patch and later at https://ozlabs.org/~akpm/mmotm/broken-out/mm-vmscan-use-per-memcg-nr_deferred-of-shrinker.patch Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next and is updated there every 3-4 working days ------------------------------------------------------ From: Yang Shi Subject: mm: vmscan: use per memcg nr_deferred of shrinker Use per memcg's nr_deferred for memcg aware shrinkers. The shrinker's nr_deferred will be used in the following cases: 1. Non memcg aware shrinkers 2. !CONFIG_MEMCG 3. memcg is disabled by boot parameter Link: https://lkml.kernel.org/r/20210311190845.9708-11-shy828301@gmail.com Signed-off-by: Yang Shi Acked-by: Roman Gushchin Acked-by: Kirill Tkhai Reviewed-by: Shakeel Butt Cc: Dave Chinner Cc: Johannes Weiner Cc: Michal Hocko Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- mm/vmscan.c | 78 ++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 66 insertions(+), 12 deletions(-) --- a/mm/vmscan.c~mm-vmscan-use-per-memcg-nr_deferred-of-shrinker +++ a/mm/vmscan.c @@ -376,6 +376,24 @@ static void unregister_memcg_shrinker(st idr_remove(&shrinker_idr, id); } +static long xchg_nr_deferred_memcg(int nid, struct shrinker *shrinker, + struct mem_cgroup *memcg) +{ + struct shrinker_info *info; + + info = shrinker_info_protected(memcg, nid); + return atomic_long_xchg(&info->nr_deferred[shrinker->id], 0); +} + +static long add_nr_deferred_memcg(long nr, int nid, struct shrinker *shrinker, + struct mem_cgroup *memcg) +{ + struct shrinker_info *info; + + info = shrinker_info_protected(memcg, nid); + return atomic_long_add_return(nr, &info->nr_deferred[shrinker->id]); +} + static bool cgroup_reclaim(struct scan_control *sc) { return sc->target_mem_cgroup; @@ -414,6 +432,18 @@ static void unregister_memcg_shrinker(st { } +static long xchg_nr_deferred_memcg(int nid, struct shrinker *shrinker, + struct mem_cgroup *memcg) +{ + return 0; +} + +static long add_nr_deferred_memcg(long nr, int nid, struct shrinker *shrinker, + struct mem_cgroup *memcg) +{ + return 0; +} + static bool cgroup_reclaim(struct scan_control *sc) { return false; @@ -425,6 +455,39 @@ static bool writeback_throttling_sane(st } #endif +static long xchg_nr_deferred(struct shrinker *shrinker, + struct shrink_control *sc) +{ + int nid = sc->nid; + + if (!(shrinker->flags & SHRINKER_NUMA_AWARE)) + nid = 0; + + if (sc->memcg && + (shrinker->flags & SHRINKER_MEMCG_AWARE)) + return xchg_nr_deferred_memcg(nid, shrinker, + sc->memcg); + + return atomic_long_xchg(&shrinker->nr_deferred[nid], 0); +} + + +static long add_nr_deferred(long nr, struct shrinker *shrinker, + struct shrink_control *sc) +{ + int nid = sc->nid; + + if (!(shrinker->flags & SHRINKER_NUMA_AWARE)) + nid = 0; + + if (sc->memcg && + (shrinker->flags & SHRINKER_MEMCG_AWARE)) + return add_nr_deferred_memcg(nr, nid, shrinker, + sc->memcg); + + return atomic_long_add_return(nr, &shrinker->nr_deferred[nid]); +} + /* * This misses isolated pages which are not accounted for to save counters. * As the data only determines if reclaim or compaction continues, it is @@ -561,14 +624,10 @@ static unsigned long do_shrink_slab(stru long freeable; long nr; long new_nr; - int nid = shrinkctl->nid; long batch_size = shrinker->batch ? shrinker->batch : SHRINK_BATCH; long scanned = 0, next_deferred; - if (!(shrinker->flags & SHRINKER_NUMA_AWARE)) - nid = 0; - freeable = shrinker->count_objects(shrinker, shrinkctl); if (freeable == 0 || freeable == SHRINK_EMPTY) return freeable; @@ -578,7 +637,7 @@ static unsigned long do_shrink_slab(stru * and zero it so that other concurrent shrinker invocations * don't also do this scanning work. */ - nr = atomic_long_xchg(&shrinker->nr_deferred[nid], 0); + nr = xchg_nr_deferred(shrinker, shrinkctl); total_scan = nr; if (shrinker->seeks) { @@ -669,14 +728,9 @@ static unsigned long do_shrink_slab(stru next_deferred = 0; /* * move the unused scan count back into the shrinker in a - * manner that handles concurrent updates. If we exhausted the - * scan, there is no need to do an update. + * manner that handles concurrent updates. */ - if (next_deferred > 0) - new_nr = atomic_long_add_return(next_deferred, - &shrinker->nr_deferred[nid]); - else - new_nr = atomic_long_read(&shrinker->nr_deferred[nid]); + new_nr = add_nr_deferred(next_deferred, shrinker, shrinkctl); trace_mm_shrink_slab_end(shrinker, shrinkctl->nid, freed, nr, new_nr, total_scan); return freed; _ Patches currently in -mm which might be from shy828301@gmail.com are mm-vmscan-use-nid-from-shrink_control-for-tracepoint.patch mm-vmscan-consolidate-shrinker_maps-handling-code.patch mm-vmscan-use-shrinker_rwsem-to-protect-shrinker_maps-allocation.patch mm-vmscan-remove-memcg_shrinker_map_size.patch mm-vmscan-use-kvfree_rcu-instead-of-call_rcu.patch mm-memcontrol-rename-shrinker_map-to-shrinker_info.patch mm-vmscan-add-shrinker_info_protected-helper.patch mm-vmscan-use-a-new-flag-to-indicate-shrinker-is-registered.patch mm-vmscan-add-per-memcg-shrinker-nr_deferred.patch mm-vmscan-use-per-memcg-nr_deferred-of-shrinker.patch mm-vmscan-dont-need-allocate-shrinker-nr_deferred-for-memcg-aware-shrinkers.patch mm-memcontrol-reparent-nr_deferred-when-memcg-offline.patch mm-vmscan-shrink-deferred-objects-proportional-to-priority.patch