From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB3D9C4345F for ; Thu, 18 Apr 2024 08:00:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 66A8F6B0083; Thu, 18 Apr 2024 04:00:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F2356B0085; Thu, 18 Apr 2024 04:00:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 492FF6B0087; Thu, 18 Apr 2024 04:00:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2A3BB6B0083 for ; Thu, 18 Apr 2024 04:00:31 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 935A81212CC for ; Thu, 18 Apr 2024 08:00:28 +0000 (UTC) X-FDA: 82021905336.01.603BACF Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf22.hostedemail.com (Postfix) with ESMTP id AA325C0006 for ; Thu, 18 Apr 2024 08:00:25 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=gbNnVMCs; spf=pass (imf22.hostedemail.com: domain of hawk@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=hawk@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713427226; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zu7l8wbiLGqVB4JdaBgTKGQLvd53sCWlM9pIPM62Tbw=; b=gVprzYe2Av709dlFYk4uhgDOBM+W7oG4xT2v5KWj4XJ67Vbx5/c/zuhjJ6WIXCK6cGFrPN W1lHCmjhyT4D51q+pT9lCFIu8aA9i9Cf8R6ptRZYhl7fHtFqe7+qARjki9/Yhtulg0sa/4 Q986V6P5BJ2VIL+SG4DQwzLiL/3F3XU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713427226; a=rsa-sha256; cv=none; b=RKQ0d7N22ud87TiPohMrUarQNtBZ8dvxrtqJkiOLIEZOT0I0BTm1L/gLcK94s+qjqXIxqF jXSSEcS0GbG7XTc+7jFbAQhXSzSMZ28LqPiCLgC9CQBOkwFWKLKwgmWaGk3yiH3vbIDhJk eenPpsggqMk50KBGoIvJhfyMyMwHDM0= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=gbNnVMCs; spf=pass (imf22.hostedemail.com: domain of hawk@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=hawk@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 2805FCE1751; Thu, 18 Apr 2024 08:00:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EF511C113CE; Thu, 18 Apr 2024 08:00:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1713427221; bh=8xkMYyuEcKoA6xltCA5FHMbWSOH1kFJ6bzMXvN0SpMY=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=gbNnVMCs3OpZ7tw3N/dPN0SJ6ZdFxArs40SVqZA5Ck30OuzATWr6xJPV4JA3o4Yr1 b1Wcy7Nd/n4ZsLB/XDkxkqHZE2hU6cfwI2oquZcJEnYExFg4JHxetExSmOPZS0gc3S hOxVkGla6AERdW7iJxLvvHcNwIUKVXX3kZJ//Bvugn/wv6J4By7WhNBrZLzp8O1jrZ rW6HXz2lIB9WdNjkGOhNS2Kcb1MGluHl5pThVJ8QIiXJexT6mkhFytklyPkdimV+iG R07jRsGzXA3zgtZLW6yz+cB+5YfJPI1/mAitJZTjpGjQSAdBouZwbYfkrDmrHpzeR2 YzC017NummtaA== Message-ID: <2f08b2c8-a931-4fc9-85c1-e63b49dbfebc@kernel.org> Date: Thu, 18 Apr 2024 10:00:17 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v1 1/3] cgroup/rstat: add cgroup_rstat_lock helpers and tracepoints To: Tejun Heo Cc: hannes@cmpxchg.org, lizefan.x@bytedance.com, cgroups@vger.kernel.org, yosryahmed@google.com, longman@redhat.com, netdev@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, shakeel.butt@linux.dev, kernel-team@cloudflare.com, Arnaldo Carvalho de Melo , Sebastian Andrzej Siewior , mhocko@kernel.org References: <171328983017.3930751.9484082608778623495.stgit@firesoul> <171328988660.3930751.17537768209042139758.stgit@firesoul> Content-Language: en-US From: Jesper Dangaard Brouer In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: AA325C0006 X-Rspam-User: X-Stat-Signature: ko6abeqrn1y3qxz6mdd1jng4ct1xdj6u X-HE-Tag: 1713427225-837314 X-HE-Meta: U2FsdGVkX19u6oJK1pFdLW7ieuF/8dEeaTc3TzPwqWNnFFHqdeC67pHHVcQSMzwWu7hduGmGzks75ea3YD0UYuNFMIwBtb4hAk91hnZDhFzJ9PKEYZ5gZNaODEsRlrVRLc923fF2Vk2I/mgPSYh1JVEkWecORSLGeRZE3EEPVmwRM5yrzfmQFO9adXGbiLI/fsGJCUNMOa0JiXGxCac+E+ucwf+gEQYYYxfkgx7694LwFNSAbgb/mGhbfmQ26rMn276L0u0qCO8zI7C8xm12KDaYJblFYB4JXzrLIrjCPaBQK/wfyMAse/pjl7AxquChBn9un2AVnISVWnDq+CasREWxEy/n0Ii30PMvZYuJDUGRYBMpitKUc6MrM//iXrXPfhe4dOIF0DP1JOeBqqPjnk0CAY14S85wwdgHN1BzcR4OlukoscNRp+kuzaMiOaVWQmbgrfmIr7d2L4DmNmtvErSDQDNKekaLfL2+vEaAF/OYf4IXcLYrZK7/Kf4UKsiFzrFuqB1DEa8YBUXBDUHT15KPzxUACHVZc2PJ9yHzB71m6F7tkKl53CQ2bBwHDU1gQz8/L5BoHqTWYuu3ZmxCLt4whhSp7zaaavBe04A2tJ64X3dvicEuccB2denUvOXJq1GvNzxEssDA6UkXqviaWA1dnKF5nk+wihz53SkCUiEUVyUXra2S7klztl/FFNDAXfVrmkKK658tpu0s5thhUvdVowJioifdjNab/wSS0k7CTUYjk8yL/RrhK22CJU0KwMT2qWdPHOqlmT6ySXnFIoX5/oBNLbAAvIezceIbPzN6S4zKI8oVTb8hN+CvLd5Vtav/MBjBaZEM89b3VoxzuKoUib26JFSgV0fQ09lj30sU/Bi31R0PC1JZhMOcXh+WMQUoCHCJX0v8/cm4UwOPIyotdWMt//EKFTzn0vwctorxiGXZxSBsuAbJmmyDeLlWtFcBjgDlGCsfpTPBvoV dZm5qnpu hv55X X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 16/04/2024 23.36, Tejun Heo wrote: > On Tue, Apr 16, 2024 at 07:51:26PM +0200, Jesper Dangaard Brouer wrote: >> This commit enhances the ability to troubleshoot the global >> cgroup_rstat_lock by introducing wrapper helper functions for the lock >> along with associated tracepoints. > > Applied to cgroup/for-6.10. > Thanks for applying the tracepoint patch. I've backported this to our main production kernels v6.6 LTS (with before mentioned upstream cgroup work from Yosry and Longman). I have it running in production on two machines this morning. Doing manual bpftrace script inspection now, but plan is monitor this continuously (ebpf_exporter[1]) and even have alerts on excessive wait time on contention. It makes sense to delay applying the next two patches, until we have some production experiments with those two patches, and I have fleet monitoring in place. I'm be offline next week (on dive trip), so I'll resume work on this 29 April, before I start doing prod experiments. --Jesper [1] https://github.com/cloudflare/ebpf_exporter