From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C222C54E71 for ; Fri, 22 Mar 2024 07:51:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9B326B0092; Fri, 22 Mar 2024 03:51:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B4B3D6B0093; Fri, 22 Mar 2024 03:51:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A13246B0095; Fri, 22 Mar 2024 03:51:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 91A2A6B0092 for ; Fri, 22 Mar 2024 03:51:38 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 47CBC80B7E for ; Fri, 22 Mar 2024 07:51:38 +0000 (UTC) X-FDA: 81923905476.06.37452BC Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by imf14.hostedemail.com (Postfix) with ESMTP id CAC46100005 for ; Fri, 22 Mar 2024 07:51:34 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of shikemeng@huaweicloud.com designates 45.249.212.51 as permitted sender) smtp.mailfrom=shikemeng@huaweicloud.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711093896; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dXxB0zO+eMaHE05B1ESajB+H0xjC7hJNxqnncNYO2nM=; b=wpQtqFQ2aZfxUg1tEuTkRkQstwPmnuDlZyrLntRPJGldLJ35WTq5BL7DQBQ/azyGG87ifx 7XAYUsKzSJ1evofH9KlxLwfbqmy68lNVtPyqMrzqFkkYuMhZ2L8k20tB18bLJMpiCN+9iu 1BHYX/WMEd4yXtm2/W80Ou+LYUP9sQ4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711093896; a=rsa-sha256; cv=none; b=UKIX5dj+gViFEgK4RfzQ/r46Od8Kwh4qeIaQI/TWfKFdxQHx+3mwdpDbjPBNxEamhQ2X5e bZujvuidZs7iJO6VhAvbW9TNvTxqRUzoQegv7UAfDIRnN9/uzr22fBfe6xRhguO0SjhRJQ QSZ0kiQnBqIHzTQmgDvmcHSXVgY77uQ= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of shikemeng@huaweicloud.com designates 45.249.212.51 as permitted sender) smtp.mailfrom=shikemeng@huaweicloud.com; dmarc=none Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4V1DzY3ryqz4f3jLp for ; Fri, 22 Mar 2024 15:51:21 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 816B31A06E0 for ; Fri, 22 Mar 2024 15:51:29 +0800 (CST) Received: from [10.174.178.129] (unknown [10.174.178.129]) by APP4 (Coremail) with SMTP id gCh0CgCn+mx_OP1l4WJkHw--.62143S2; Fri, 22 Mar 2024 15:51:29 +0800 (CST) Subject: Re: [PATCH 1/6] writeback: collect stats of all wb of bdi in bdi_debug_stats_show To: Jan Kara Cc: akpm@linux-foundation.org, tj@kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, willy@infradead.org, bfoster@redhat.com, dsterba@suse.com, mjguzik@gmail.com, dhowells@redhat.com, peterz@infradead.org References: <20240320110222.6564-1-shikemeng@huaweicloud.com> <20240320110222.6564-2-shikemeng@huaweicloud.com> <20240321180620.mbint45pbyc74vpg@quack3> From: Kemeng Shi Message-ID: Date: Fri, 22 Mar 2024 15:51:27 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <20240321180620.mbint45pbyc74vpg@quack3> Content-Type: text/plain; charset=gbk Content-Transfer-Encoding: 7bit X-CM-TRANSID:gCh0CgCn+mx_OP1l4WJkHw--.62143S2 X-Coremail-Antispam: 1UD129KBjvJXoWxZFy5JryrKrW5Zw43uw1rWFg_yoWrZw45pF s8Ga1rGr48Xry7W3ZxZFWDXrWftw4Fqw12qFyIyay5Cw1DZF1akFy3uFyjyFykArZ7GFyj yan0y3s3urWUtFJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUvab4IE77IF4wAFF20E14v26r4j6ryUM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28lY4IEw2IIxxk0rwA2F7IY1VAKz4 vEj48ve4kI8wA2z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7Cj xVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x 0267AKxVW0oVCq3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG 6I80ewAv7VC0I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFV Cjc4AY6r1j6r4UM4x0Y48IcVAKI48JM4IIrI8v6xkF7I0E8cxan2IY04v7Mxk0xIA0c2IE e2xFo4CEbIxvr21l42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxV Aqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1q 6r43MIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr0_JF4lIxAIcVC0I7IYx2IY6x kF7I0E14v26r4j6F4UMIIF0xvE42xK8VAvwI8IcIk0rVWrJr0_WFyUJwCI42IY6I8E87Iv 67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyT uYvjxUrR6zUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-Rspamd-Queue-Id: CAC46100005 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: peugjdfrntq884x9pt9cbkm556n58y58 X-HE-Tag: 1711093894-234031 X-HE-Meta: U2FsdGVkX1/66SJKVfRo5EEgbpMLQm0/v621Kk7bvggEny5mjzoP9V+DyPnFbG7v/wPxplxgwgC96Iu4SPm7d0TZAHVZr3LoLE7+z/haztqansFDPhcbGSeZueSMIxbEpqHfKBN5VL/hIxratlIg+pYJncB+oV0TCHQfAchojILJsa0AiNX5vwG6pAA9WESHiaHjJ3Kk6Xt20gfIk9CknOB3hMoKpzvz1Q1yBmUpk9QAmf4KMk51M9DWYyicIrpqIuj8C7XtmEL49NF8a/F9mp7HlQTrQ51FRKDslEHWof8Wzno0Ep4l5SFY0Z39s/aVWmOtlvoFn+VzFVLP4gRQ5xIy8nM1frcwd46iaq4z02yRi879UanJ7+HyqxUYvFoMSA8O5UKS6B6WBYGP1Fn1626uBVAgMvOF6S6zm+a2aBCWD+8Q2xTa8woeggrsAnzuApf4tReeQeIDxLVEzessEU41J3P7upwsix0J9C1zPO0eHZii/RzaLRFZWKjsi0hREHcrOOTwLA7/H3LoTv6+1v/l/wy1306Q+3jv6hUsSsp3iROo7YgYGGFMHiUMFWtdb7VfDF2fVkmN8/itrZ+K6aU+MMpEfbmXDMgbT8ThzecwMvAdNgYXawOOL08w8/nkny2kJZbi98VdFLJvQ9fV9IF6xbOa5ya3YkhpEV0tvgTpmkoKxlEU2cCNXN4NCftPQyopKjjVQagqG68PxHdCLg4whB4skL+GLPB4mttgGysbRn5coNY1NPIEnUwLLE+PTRB+9o0CF0zDpF9kBs5taK7VyqnyoCswzjf+e+AOdMEpsq/1w03i0/ZZf2+suzYESrjd4YbOdl/wxkXuWtyRpvMqogZzXbcyYBGW0VCnf83vVFMy7SsZfrQ8OJMbilmmVouB/DJ439MDEmgOKtAeyUO9T1z5vYOofdwAibpSmPFmHNSlFmq2Ji+ZC2EAxmDSlNziBC9JBTssq1lPWpR EPT9+QfW ni4e6ZVX+vx8bPXeOnDAuW3PabHnjrz1ro15LA2ryuE7hmT1dY/bs4SsyNAPhArzAFDzMO8uKa3ZJFBwXXEroLJNo2IP7GOH0nkZYRcivGqDM3srql82sD57ZEbo1Ez1pK6WM0uvWlXqveVH6ObUReU32ur3wR4FOgeiA8bcJcTUhybCaqfz+doHr9xKqvf/w+C1oJ8oXAPDTPVkESY1P2ZR0mjCexcAYuQmR3VDoL1cbN+DsmVAi6kM8NhI9fzIRmql9j/9GamQIVfQ1t9Z34cILGE2sTw0c7AftDeMYB17VVlv6jloKW/851amv4aZkP2sa X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: on 3/22/2024 2:06 AM, Jan Kara wrote: > On Wed 20-03-24 19:02:17, Kemeng Shi wrote: >> /sys/kernel/debug/bdi/xxx/stats is supposed to show writeback information >> of whole bdi, but only writeback information of bdi in root cgroup is >> collected. So writeback information in non-root cgroup are missing now. >> To be more specific, considering following case: >> >> /* create writeback cgroup */ >> cd /sys/fs/cgroup >> echo "+memory +io" > cgroup.subtree_control >> mkdir group1 >> cd group1 >> echo $$ > cgroup.procs >> /* do writeback in cgroup */ >> fio -name test -filename=/dev/vdb ... >> /* get writeback info of bdi */ >> cat /sys/kernel/debug/bdi/xxx/stats >> The cat result unexpectedly implies that there is no writeback on target >> bdi. >> >> Fix this by collecting stats of all wb in bdi instead of only wb in >> root cgroup. >> >> Signed-off-by: Kemeng Shi > > Looks mostly good, one comment below: > >> --- >> mm/backing-dev.c | 93 ++++++++++++++++++++++++++++++++++++------------ >> 1 file changed, 70 insertions(+), 23 deletions(-) >> >> diff --git a/mm/backing-dev.c b/mm/backing-dev.c >> index 5f2be8c8df11..788702b6c5dd 100644 >> --- a/mm/backing-dev.c >> +++ b/mm/backing-dev.c >> @@ -39,6 +39,19 @@ struct workqueue_struct *bdi_wq; >> #include >> #include >> >> +struct wb_stats { >> + unsigned long nr_dirty; >> + unsigned long nr_io; >> + unsigned long nr_more_io; >> + unsigned long nr_dirty_time; >> + unsigned long nr_writeback; >> + unsigned long nr_reclaimable; >> + unsigned long nr_dirtied; >> + unsigned long nr_written; >> + unsigned long dirty_thresh; >> + unsigned long wb_thresh; >> +}; >> + >> static struct dentry *bdi_debug_root; >> >> static void bdi_debug_init(void) >> @@ -46,31 +59,65 @@ static void bdi_debug_init(void) >> bdi_debug_root = debugfs_create_dir("bdi", NULL); >> } >> >> -static int bdi_debug_stats_show(struct seq_file *m, void *v) >> +static void collect_wb_stats(struct wb_stats *stats, >> + struct bdi_writeback *wb) >> { >> - struct backing_dev_info *bdi = m->private; >> - struct bdi_writeback *wb = &bdi->wb; >> - unsigned long background_thresh; >> - unsigned long dirty_thresh; >> - unsigned long wb_thresh; >> - unsigned long nr_dirty, nr_io, nr_more_io, nr_dirty_time; >> struct inode *inode; >> >> - nr_dirty = nr_io = nr_more_io = nr_dirty_time = 0; >> spin_lock(&wb->list_lock); >> list_for_each_entry(inode, &wb->b_dirty, i_io_list) >> - nr_dirty++; >> + stats->nr_dirty++; >> list_for_each_entry(inode, &wb->b_io, i_io_list) >> - nr_io++; >> + stats->nr_io++; >> list_for_each_entry(inode, &wb->b_more_io, i_io_list) >> - nr_more_io++; >> + stats->nr_more_io++; >> list_for_each_entry(inode, &wb->b_dirty_time, i_io_list) >> if (inode->i_state & I_DIRTY_TIME) >> - nr_dirty_time++; >> + stats->nr_dirty_time++; >> spin_unlock(&wb->list_lock); >> >> + stats->nr_writeback += wb_stat(wb, WB_WRITEBACK); >> + stats->nr_reclaimable += wb_stat(wb, WB_RECLAIMABLE); >> + stats->nr_dirtied += wb_stat(wb, WB_DIRTIED); >> + stats->nr_written += wb_stat(wb, WB_WRITTEN); >> + stats->wb_thresh += wb_calc_thresh(wb, stats->dirty_thresh); >> +} >> + >> +#ifdef CONFIG_CGROUP_WRITEBACK >> +static void bdi_collect_stats(struct backing_dev_info *bdi, >> + struct wb_stats *stats) >> +{ >> + struct bdi_writeback *wb; >> + >> + /* protect wb from release */ >> + mutex_lock(&bdi->cgwb_release_mutex); >> + list_for_each_entry(wb, &bdi->wb_list, bdi_node) >> + collect_wb_stats(stats, wb); >> + mutex_unlock(&bdi->cgwb_release_mutex); >> +} > > So AFAICT this function can race against > bdi_unregister() -> wb_shutdown(&bdi->wb) > > because that doesn't take the cgwb_release_mutex. So we either need the RCU > protection as Brian suggested or cgwb_lock or something. But given > collect_wb_stats() can take a significant amount of time (traversing all > the lists etc.) I think we'll need something more clever. Sorry for missing this. I only pay attention to wb in cgroup as there is no much change to how we use bdi->wb. It seems that there was always a race between orginal bdi_debug_stats_show and release of bdi as following cat /.../stats bdi_debug_stats_show bdi_unregister bdi_put release_bdi kfree(bdi) use after free I will fix this in next version. Thanks! > > Honza >