From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9EDD7C00528 for ; Tue, 8 Aug 2023 02:44:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YOZE/pe2p+wOcDQ0uX1yYRkOYQAY4LkGlSHlsgJjXm0=; b=yy4H6pCCIKJ9St 3177gpqDdrfZTnKj+fZCaVk6H26zmH2MQPE6CDjFbotwEhxbx5wgeqZ7Pm4LMppI+GCEpiX8WtsF/ bXGlkY13D1SdJa7oyggvo0SZlS3L9vWvir7AigiGQ+qVU/1QATOgCVkC/mCp9iWqhiKmWOeX8ZnE8 XVyj3wnghpc4RBDhdVdeelAGAd2peimgVLlJUm4k81QhV681D/jwlXQhnF97EyWMTmKixAwYXUzCs hCHUNNxS7j/vX3FlTntpuLFI27LSpvm7wFy119QJPRjYizgVdPuCSZn24Nr+jlz48HsC+aBRgRhJR Oti1O6uUhJUXFPsrD3Ug==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qTChm-001Zcc-35; Tue, 08 Aug 2023 02:44:18 +0000 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qTChj-001Zbt-37 for linux-mtd@lists.infradead.org; Tue, 08 Aug 2023 02:44:17 +0000 Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1bbf3da0ea9so33830245ad.2 for ; Mon, 07 Aug 2023 19:44:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20221208.gappssmtp.com; s=20221208; t=1691462655; x=1692067455; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ZZ3n364wJBBn5i6olC7X8xOphz5aoEwi3PPn9Humxy4=; b=c9iOPpy41VC6QSzv5vLbmCPsxZB2K7hAItf8MEtryT/BAlfvdA0UPM4PAg3DtN7rr+ GFW2nm1AoLFEe5zmSSiqyIhn92P6iecybeuExkqa7jbnj+LEVM1LioV7SZGiR/8Kzy2+ mJYrJW8XMi2+dbRvP7Xg6QuSzUnp/yHkq29M7vbAYNrF1NokfShgkpn7COiSBb38dkF0 MWE0ZNG9ZS4LzaWgVeAy0Pmh7etHcKYaOYLW+oCVP3Q2zlcEbOnfLG0lI+GjqQwBJGqv OKDqkP3u3mDYRaZI2hDArPl64kIP09oRYh791qhymi4aMBqJthtqlGcAIAw7e+Im4Q2d AiTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691462655; x=1692067455; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=ZZ3n364wJBBn5i6olC7X8xOphz5aoEwi3PPn9Humxy4=; b=Nk5G4TTUWXkuIQGQFcmoQCP0y/pybP/DFR6hgJQrl6KQ3RnXv7ZCUqrbjiT3DK5PMC MpHa5SAHRo7C+NA/n+p+4f9Fsv7m9CYM3V+WFZcjfl2G5N01u15NlT83FG9xnbRflQnU nOCnGyrvgFdTswS9kvpt0jy2ALri+nwKVFJ3z5MncJGKM0D5BQF/UFtP4OSt3f3a+uci Re4mEEn3UnN48HxLQzdAEAfZdwYcxeQkW/anfeUigNUco9jDG9CgYDMDWaPj/XkdqsFU AbPWrzzkPG0dOgGjVoj2C5uUyW/fGkGUZFMF4J8sREMMVIe24g205WAdb1imQXEaGIGe yKQg== X-Gm-Message-State: AOJu0YwhPvjHfXrBuYOFvhGNxx5Rujp9rpcjlDKENcFlptSXfuZ31AfU L7F/zvrL9Ioy4HLgOGBDRZI8HA== X-Google-Smtp-Source: AGHT+IEZT2gUmOSYrkIbMjP5ySF6z1UEt5Q+o+GMN8zU98oO9pgzqqi+i8QK+swPthTlPTk4WQ36Qw== X-Received: by 2002:a17:902:7287:b0:1b3:f5c7:4e75 with SMTP id d7-20020a170902728700b001b3f5c74e75mr9699662pll.58.1691462654669; Mon, 07 Aug 2023 19:44:14 -0700 (PDT) Received: from dread.disaster.area (pa49-180-166-213.pa.nsw.optusnet.com.au. [49.180.166.213]) by smtp.gmail.com with ESMTPSA id u2-20020a170902e80200b001b893b689a0sm7632067plg.84.2023.08.07.19.44.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Aug 2023 19:44:14 -0700 (PDT) Received: from dave by dread.disaster.area with local (Exim 4.96) (envelope-from ) id 1qTChf-002XBg-27; Tue, 08 Aug 2023 12:44:11 +1000 Date: Tue, 8 Aug 2023 12:44:11 +1000 From: Dave Chinner To: Qi Zheng Cc: akpm@linux-foundation.org, tkhai@ya.ru, vbabka@suse.cz, roman.gushchin@linux.dev, djwong@kernel.org, brauner@kernel.org, paulmck@kernel.org, tytso@mit.edu, steven.price@arm.com, cel@kernel.org, senozhatsky@chromium.org, yujie.liu@intel.com, gregkh@linuxfoundation.org, muchun.song@linux.dev, simon.horman@corigine.com, dlemoal@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-erofs@lists.ozlabs.org, linux-f2fs-devel@lists.sourceforge.net, cluster-devel@redhat.com, linux-nfs@vger.kernel.org, linux-mtd@lists.infradead.org, rcu@vger.kernel.org, netdev@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, dm-devel@redhat.com, linux-raid@vger.kernel.org, linux-bcache@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, linux-btrfs@vger.kernel.org Subject: Re: [PATCH v4 46/48] mm: shrinker: make memcg slab shrink lockless Message-ID: References: <20230807110936.21819-1-zhengqi.arch@bytedance.com> <20230807110936.21819-47-zhengqi.arch@bytedance.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230807110936.21819-47-zhengqi.arch@bytedance.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230807_194416_030961_76EC6DD6 X-CRM114-Status: GOOD ( 28.31 ) X-BeenThere: linux-mtd@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Linux MTD discussion mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-mtd" Errors-To: linux-mtd-bounces+linux-mtd=archiver.kernel.org@lists.infradead.org On Mon, Aug 07, 2023 at 07:09:34PM +0800, Qi Zheng wrote: > Like global slab shrink, this commit also uses refcount+RCU method to make > memcg slab shrink lockless. This patch does random code cleanups amongst the actual RCU changes. Can you please move the cleanups to a spearate patch to reduce the noise in this one? > diff --git a/mm/shrinker.c b/mm/shrinker.c > index d318f5621862..fee6f62904fb 100644 > --- a/mm/shrinker.c > +++ b/mm/shrinker.c > @@ -107,6 +107,12 @@ static struct shrinker_info *shrinker_info_protected(struct mem_cgroup *memcg, > lockdep_is_held(&shrinker_rwsem)); > } > > +static struct shrinker_info *shrinker_info_rcu(struct mem_cgroup *memcg, > + int nid) > +{ > + return rcu_dereference(memcg->nodeinfo[nid]->shrinker_info); > +} This helper doesn't add value. It doesn't tell me that rcu_read_lock() needs to be held when it is called, for one.... > static int expand_one_shrinker_info(struct mem_cgroup *memcg, int new_size, > int old_size, int new_nr_max) > { > @@ -198,7 +204,7 @@ void set_shrinker_bit(struct mem_cgroup *memcg, int nid, int shrinker_id) > struct shrinker_info_unit *unit; > > rcu_read_lock(); > - info = rcu_dereference(memcg->nodeinfo[nid]->shrinker_info); > + info = shrinker_info_rcu(memcg, nid); ... whilst the original code here was obviously correct. > unit = info->unit[shriner_id_to_index(shrinker_id)]; > if (!WARN_ON_ONCE(shrinker_id >= info->map_nr_max)) { > /* Pairs with smp mb in shrink_slab() */ > @@ -211,7 +217,7 @@ void set_shrinker_bit(struct mem_cgroup *memcg, int nid, int shrinker_id) > > static DEFINE_IDR(shrinker_idr); > > -static int prealloc_memcg_shrinker(struct shrinker *shrinker) > +static int shrinker_memcg_alloc(struct shrinker *shrinker) Cleanups in a separate patch. > @@ -253,10 +258,15 @@ static long xchg_nr_deferred_memcg(int nid, struct shrinker *shrinker, > { > struct shrinker_info *info; > struct shrinker_info_unit *unit; > + long nr_deferred; > > - info = shrinker_info_protected(memcg, nid); > + rcu_read_lock(); > + info = shrinker_info_rcu(memcg, nid); > unit = info->unit[shriner_id_to_index(shrinker->id)]; > - return atomic_long_xchg(&unit->nr_deferred[shriner_id_to_offset(shrinker->id)], 0); > + nr_deferred = atomic_long_xchg(&unit->nr_deferred[shriner_id_to_offset(shrinker->id)], 0); > + rcu_read_unlock(); > + > + return nr_deferred; > } This adds two rcu_read_lock() sections to every call to do_shrink_slab(). It's not at all clear ifrom any of the other code that do_shrink_slab() now has internal rcu_read_lock() sections.... > @@ -464,18 +480,23 @@ static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, > if (!mem_cgroup_online(memcg)) > return 0; > > - if (!down_read_trylock(&shrinker_rwsem)) > - return 0; > - > - info = shrinker_info_protected(memcg, nid); > +again: > + rcu_read_lock(); > + info = shrinker_info_rcu(memcg, nid); > if (unlikely(!info)) > goto unlock; > > - for (; index < shriner_id_to_index(info->map_nr_max); index++) { > + if (index < shriner_id_to_index(info->map_nr_max)) { > struct shrinker_info_unit *unit; > > unit = info->unit[index]; > > + /* > + * The shrinker_info_unit will not be freed, so we can > + * safely release the RCU lock here. > + */ > + rcu_read_unlock(); Why - what guarantees that the shrinker_info_unit exists at this point? We hold no reference to it, we hold no reference to any shrinker, etc. What provides this existence guarantee? > + > for_each_set_bit(offset, unit->map, SHRINKER_UNIT_BITS) { > struct shrink_control sc = { > .gfp_mask = gfp_mask, > @@ -485,12 +506,14 @@ static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, > struct shrinker *shrinker; > int shrinker_id = calc_shrinker_id(index, offset); > > + rcu_read_lock(); > shrinker = idr_find(&shrinker_idr, shrinker_id); > - if (unlikely(!shrinker || !(shrinker->flags & SHRINKER_REGISTERED))) { > - if (!shrinker) > - clear_bit(offset, unit->map); > + if (unlikely(!shrinker || !shrinker_try_get(shrinker))) { > + clear_bit(offset, unit->map); > + rcu_read_unlock(); > continue; > } > + rcu_read_unlock(); > > /* Call non-slab shrinkers even though kmem is disabled */ > if (!memcg_kmem_online() && > @@ -523,15 +546,20 @@ static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, > set_shrinker_bit(memcg, nid, shrinker_id); > } > freed += ret; > - > - if (rwsem_is_contended(&shrinker_rwsem)) { > - freed = freed ? : 1; > - goto unlock; > - } > + shrinker_put(shrinker); Ok, so why is this safe to call without holding the rcu read lock? The global shrinker has to hold the rcu_read_lock() whilst calling shrinker_put() to guarantee the validity of the list next pointer, but we don't hold off RCU here so what guarantees a racing global shrinker walk doesn't trip over this shrinker_put() call dropping the refcount to zero and freeing occuring in a different context... > + /* > + * We have already exited the read-side of rcu critical section > + * before calling do_shrink_slab(), the shrinker_info may be > + * released in expand_one_shrinker_info(), so reacquire the > + * shrinker_info. > + */ > + index++; > + goto again; With that, what makes the use of shrinker_info in xchg_nr_deferred_memcg() in do_shrink_slab() coherent and valid? -Dave. -- Dave Chinner david@fromorbit.com ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/