From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61CEBC48BF6 for ; Thu, 7 Mar 2024 09:27:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E5F646B013A; Thu, 7 Mar 2024 04:26:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E0F496B013B; Thu, 7 Mar 2024 04:26:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD6B96B013C; Thu, 7 Mar 2024 04:26:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B7EE26B013A for ; Thu, 7 Mar 2024 04:26:59 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7EA0641126 for ; Thu, 7 Mar 2024 09:26:59 +0000 (UTC) X-FDA: 81869713758.23.F5EF390 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf05.hostedemail.com (Postfix) with ESMTP id 64D14100017 for ; Thu, 7 Mar 2024 09:26:57 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=Zryh4i7G; dkim=pass header.d=suse.com header.s=susede1 header.b=bjSGKP6w; spf=pass (imf05.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.131 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709803617; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=49eI9FsfEc0os9WuNfnd5K+4CQXJgYzosIDjRsUtlwQ=; b=Bgrklfc0fjSgWsnoi4yc+K45DPrUPFYRAI6RSqRkn6AtTS5TqMRC4zjxdNCVxwreRNyyut +HzyYigDJwhSCVledaYwCSeh0cmTyrv8tbEA/ceo6t36R+AJN81rIvJYh+RSMxdeBKMgyb l1yiTMMv15N+Di/ut0HMElDMmaEkBME= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=Zryh4i7G; dkim=pass header.d=suse.com header.s=susede1 header.b=bjSGKP6w; spf=pass (imf05.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.131 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709803617; a=rsa-sha256; cv=none; b=oZLb0lreHl2xlMqQQOQmMuGvVzLPNtTGU8JU7fsTpGkgMl6B/czdhbf6G7egkfWvYfkThL drUbpgsb9/00cpWUTk2i/ETBFF+nu39u7DauA3vqdS5hV4xb/+9ZBm6AHB01OP3EI/6lXZ gOaOifDJvJleGipseS8U4yg4Y3xwTp0= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id A92A13EB89; Thu, 7 Mar 2024 09:26:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1709803588; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=49eI9FsfEc0os9WuNfnd5K+4CQXJgYzosIDjRsUtlwQ=; b=Zryh4i7GQWJaae+0LvHwuuEXcojnHFGrRwQ6bIjfVfe/yjuT7YrMDQ5Jd1mlZaXID9jD79 b6KRE0KDDhqJyUrrmKvGqwgDJfPNTmTQ3EIOYZEE/ZolmQ4Kxo8qqDOVNfxR8mf0reFFcy etSQf4Ci4ZkKrvcx6ReBFrgt08EDF6M= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1709803587; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=49eI9FsfEc0os9WuNfnd5K+4CQXJgYzosIDjRsUtlwQ=; b=bjSGKP6wJyNQzALLKtdz6JeZvDQ4XpOaszh2ul11+k2G2wP9pWsgTI4dKY7izI9m06Ge0t 9MGtZkCCYsPDw0Kk9/A+8kU6jUUrkYgMNoK4ldUAbUcQkcR+qNMKxs1+qSENqy1QYs+rAg o7yxPV6C8R/smpbn56Vx6J78+jCcEtU= Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 81CFC12FC5; Thu, 7 Mar 2024 09:26:27 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id S+uwHEOI6WXgGQAAD6G6ig (envelope-from ); Thu, 07 Mar 2024 09:26:27 +0000 Date: Thu, 7 Mar 2024 10:26:18 +0100 From: Michal Hocko To: Bixuan Cui Cc: Steven Rostedt , akpm@linux-foundation.org, mhiramat@kernel.org, mathieu.desnoyers@efficios.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org, opensource.kernel@vivo.com Subject: Re: [PATCH -next v6 0/2] Make memory reclamation measurable Message-ID: References: <20240105013607.2868-1-cuibixuan@vivo.com> <20240220212202.59ddc123@gandalf.local.home> <3d4f44ee-f533-446f-a9e6-7f58afc78d65@vivo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 64D14100017 X-Rspam-User: X-Stat-Signature: hy458fq4z8m4xz7qu3d8uyhrttg3cs4k X-Rspamd-Server: rspam01 X-HE-Tag: 1709803617-201381 X-HE-Meta: U2FsdGVkX1+96jZN4TTHMUrGMT/jPpHSDfXy9eJFcKZ1Hb26DQWw7KJX+uuHJ2wQpc0/0xyIiLcgdcUQYe3FTpo12Cz20qXh0v7Lb3d1HzFekbRjECbHYYKhOmkLHd2yR0hv0MuMT0ITcf+rtHj4GSr/61BjYoOP6AhiRkrfwa9vVqN6LJ+Ma4UZqWc7YbsSvCz8rIP9QDnGiK6RVZOMBpmYX3Qad2rP+CHFCF+PIupvreNXgnk9mh6MDSq7b73zAZ/4jg1m7I1glFmMkcApJIwLaeyB8A5cKDw6JYstEjA2dNcmiBOzs2zXkO3N78WT2T/3iHxa+jdt/MMKMErFYPrlshIsiLk7L+Bf8udLJJdUGyD1HfAa1QG1/wED2pon9wToKQ/bRiRV5bKud/KjdffowXsXRjicAldag//dENLJ39NUprVY0jN8NZR7Ta/pP0A5NDA9Rif1USYMQFBgynowWhWP5OUVUy8eggn0JaDXEDVHnA8z2jmo1moj6uullJOS3z/Sux1SZsos0OiMOeP+FshMkqatV6FfpVKROTjZN4547z4nPNgkbg7RabNOn1kZjK1eRGBTn/qLCQcxtog6AhDIErHw7JZ1cWydPj319H00fKu3NiyNe4XcXg3iADSHVkamC5+uRumu5fh6QgckIj6kjI+o4etn1TQWHzjfpKRg//lZgL4QX8lJU8lwPak9Y+W+XqNAlaQuWCOiuIRojYLW+UTzsQJDHL6SsaKOo/qJq3R4SKvdGYta4wqyyB+wmKAVN9JI+76GyE0uUc89mJCdHPrAhlDRfSYro/vZyOcgOfwv6zQqabmhXBnSZjnd+DjBcmhTFPMfY9khUMWBJBtUqZ64FhYVj7BQ/sEkFIeiCGeYJ9EVdHTU+aPQoJPMQo0nGjY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu 07-03-24 15:40:29, Bixuan Cui wrote: [...] > Currently, with the help of kernel trace events or tools like Perfetto, we > can only see that kswapd is competing for CPU and the frequency of memory > reclamation triggers, but we do not have detailed information or metrics > about memory reclamation, such as the duration and amount of each > reclamation, or who is releasing memory (super_cache, f2fs, ext4), etc. This > makes it impossible to locate the above problems. I am not sure I agree with you here. We do provide insight into LRU and shrinkers reclaim. Why isn't that enough. In general I would advise you to focus more on describing why the existing infrastructure is insuficient (having examples would be really appreciated). > Currently this patch helps us solve 2 actual performance problems (kswapd > preempts the CPU causing game delay) > 1. The increased memory allocation in the game (across different versions) > has led to the degradation of kswapd. > This is found by calculating the total amount of Reclaim(page) during > the game startup phase. > > 2. The adoption of a different file system in the new system version has > resulted in a slower reclamation rate. > This is discovered through the OBJ_NAME change. For example, OBJ_NAME > changes from super_cache_scan to ext4_es_scan. > > Subsequently, it is also possible to calculate the memory reclamation rate > to evaluate the memory performance of different versions. Why cannot you achive this with existing tracing or /proc/vmstat infrastructure? > The main reasons for adding static tracepoints are: > 1. To subdivide the time spent in the shrinker->count_objects() and > shrinker->scan_objects() functions within the do_shrink_slab function. Using > BPF kprobe, we can only track the time spent in the do_shrink_slab function. > 2. When tracing frequently called functions, static tracepoints (BPF > tp/tracepoint) have lower performance impact compared to dynamic tracepoints > (BPF kprobe). You can track the time process has been preempted by other means, no? We have context switching tracepoints in place. Have you considered that option? -- Michal Hocko SUSE Labs