From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0279C3DA5D for ; Fri, 19 Jul 2024 07:54:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2532A6B0082; Fri, 19 Jul 2024 03:54:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 204F36B0089; Fri, 19 Jul 2024 03:54:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0F2DC6B0092; Fri, 19 Jul 2024 03:54:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E8E096B0082 for ; Fri, 19 Jul 2024 03:54:53 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A15A4A16B3 for ; Fri, 19 Jul 2024 07:54:53 +0000 (UTC) X-FDA: 82355740866.05.06D0203 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf12.hostedemail.com (Postfix) with ESMTP id 3600840018 for ; Fri, 19 Jul 2024 07:54:49 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mInGDfpV; spf=pass (imf12.hostedemail.com: domain of hawk@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=hawk@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721375650; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6zu89xNRpEBKqovEDqmmvqKMRIBr425oJ4Ux4q1ELSY=; b=4taRGEzzYhykS+5uOWlISW3bqkycu2hK6Lsp4TTDbrDUet4tnamrI1VPLoyl7RlcsWvxUs yOnbFi2IO/UmT7QNRs1vfaCTMK8slKbDzgm5XmOmksapwfltyL+/GLqNbUjFYWT4Nwit0v gMClc14LzSYsYC4mMMzHer8fQRJbMxg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721375650; a=rsa-sha256; cv=none; b=tijFELUuVEAhmtDyx+27H9YwmMjzWIvqrCZuupccljIH4MrDGRoV48joG4ZrMjpkB9eaQr MyKUs5LnKrF/oJCrDHFsm6nhsO5wGRP3ZhRTteat65rsffY0b3PswhUFRYy816ySJ+WAAR zGs8mXNTbHpDiiRYRel9PxENpQ3Z05g= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mInGDfpV; spf=pass (imf12.hostedemail.com: domain of hawk@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=hawk@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 696BECE1AF2; Fri, 19 Jul 2024 07:54:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DBF6BC32782; Fri, 19 Jul 2024 07:54:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1721375684; bh=bWcDV70r6eQGJyJJqDs1xMTbnoM4r5EvMsf4/4R/5M0=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=mInGDfpVQ/i2+n7SY2uYp0BP2jZYocblZYrIAA7jjLUApJ1OpF1tWxmFi8MS3fIop OvLc6YOJicmzLA15i+cqWyY09SxyNA7MOVm/Xcx/HWNKYvOtL8gNfAnRP4XIrs910S fG+HiTMibYRCU6pGLTZhFzzP79LgGNu3RT3SfZZvAvHs/J+SE7BFLzUEnkRr/7O06P 5WnoFXxYLE5bifVCkrKm4wOfk2fYpc9/zQDQmc6yhIR7j8+dMZUTDfSL7xdog/Phgx gPWT2yGprtXjIEiE1Lc5Q5BmOe+k5bBhaKYcBsAPeeI2Sxs+vV+t+fGExdhVXzyIgx Dgpj2iF63PoQQ== Message-ID: <5ccc693a-2142-489d-b3f1-426758883c1e@kernel.org> Date: Fri, 19 Jul 2024 09:54:41 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH V7 1/2] cgroup/rstat: Avoid thundering herd problem by kswapd across NUMA nodes To: Shakeel Butt Cc: Yosry Ahmed , tj@kernel.org, cgroups@vger.kernel.org, hannes@cmpxchg.org, lizefan.x@bytedance.com, longman@redhat.com, kernel-team@cloudflare.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <172070450139.2992819.13210624094367257881.stgit@firesoul> <100caebf-c11c-45c9-b864-d8562e2a5ac5@kernel.org> Content-Language: en-US From: Jesper Dangaard Brouer In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 3600840018 X-Stat-Signature: c99pq4ygesz8icbmjw5rebfx6uxfi3yb X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1721375689-404110 X-HE-Meta: U2FsdGVkX18HI5PJbMnsO8gfB4LD0fXmlse1DUboq6u5Os5MHUowRA9qCu9O6uzZsvuOAgJEbI2QGmvsKlHZwbLY2w/5KI1AHyquTqhCJPBTNSiEYj7n6dKCHTVra4UpTlUqEOaKUzl4Qcq0oKepRzG6Ph31XGK++pu2g3CPwZj6Fhtblx705E1qJUw5eQJgcdNxHiMWEv5TSVChF+acW5M0TjXT20L4x+4ipzWRC5vjJhnrMBCWq4Zwwnqyhf2c8ToPXPB1NSiPXk+yxB3w8kGKbj81NTkQkcypqI9UM35ZuT9i/I1l4MGNSrqyzH/v3MG1blPVWDGLUy3UAdSxFFAC34GJVJ4WBStzuDf84Ylwem+s7ZxFqvkrYfi97aN8TOendM3s8PmdTcqY3EH45RxbsSt8/9n4FZjh4xrhFst1I7cvNBIv7BeQ1uyiNQ6f6e8ZGxE647wRe3KoR+utwiH6RuHjroD2SVUp1nqq4Oz37uRWSWMaM7FurXh33CFtQBz5lT6NNDOMsw/y+XKSXTupp4onu13sJyu/cEqsrn8W6vY3imq/n2C1m9O295x9SKi9I5p02TEMlOTxtuaUcK3Pp1Qn28N7wPZhkMYQXmT3TGy2v2eFmFm7nXzJ9yC93cXGAm/pMPMYwCKAokjO4goNI7y7SgtZZtkCSaYrxBEYoJTkJzu2ByAxDe4oWQizrLcM6qsDRGlOJvHWkX4q0H6GioM9nQoNpnuJ40DMiqrT8UvQOcMbIYIvzIzovHWuhBIhgKAtS15jNU1t03itW9oLUTbOUzo6ShVhSaqPCamPoOmqd7DnzaVhYSNfEjOBh1qm3IEpW96ePVnrehp+8YvZRU7NONsM5dxy2lC4+00GtqHtCxCbWwocUep8aP+dqpF6fGL2SAFR6N7luSrRzU4oPgmuhQtYT148a+1H582uT3uRmvQitT2QuywTMAWyYY2FsUS8zQnQxu+GZe5 8uIQPte0 rv2tyNP/WYLkbedWFzM4Bs/SgLDYnUQcS34VulOmL8CQUhUuxdrV/h92JYZ9XxRrE5jYnr/39xbZGqI2T7xOIy5ZgfTevt7f5Id9cv3EZPe/mBSeLGeZHNAVUYoMqQm8rU7R/triWAGXStnXo5o9YJkJaHjQI00VIchiiiJ2Vi4Q2uz/iVMRFhioAwI3dudfQulDMSBcLBuxjgFd6uZbDCJm56b1ya6cmRyh1MlG8YWIxiRORRl3cGZ5GALkY62sxr48mpoNNML8GJzQ9s1V6jtsT9g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.003123, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 19/07/2024 02.40, Shakeel Butt wrote: > Hi Jesper, > > On Wed, Jul 17, 2024 at 06:36:28PM GMT, Jesper Dangaard Brouer wrote: >> > [...] >> >> >> Looking at the production numbers for the time the lock is held for level 0: >> >> @locked_time_level[0]: >> [4M, 8M) 623 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | >> [8M, 16M) 860 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| >> [16M, 32M) 295 |@@@@@@@@@@@@@@@@@ | >> [32M, 64M) 275 |@@@@@@@@@@@@@@@@ | >> > > Is it possible to get the above histogram for other levels as well? Data from other levels available in [1]: [1] https://lore.kernel.org/all/8c123882-a5c5-409a-938b-cb5aec9b9ab5@kernel.org/ IMHO the data shows we will get most out of skipping level-0 root-cgroup flushes. Copy-pasted here (adj whitespaces hoping it better fit email): Time below is in nanosec: - M -> ms - K -> usec @locked_time_level[0]: [4M, 8M) 623 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [8M, 16M) 860 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [16M, 32M) 295 |@@@@@@@@@@@@@@@@@ | [32M, 64M) 275 |@@@@@@@@@@@@@@@@ | @locked_time_level[1]: [4K, 8K) 6 |@@@@ | [8K, 16K) 65 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [16K, 32K) 52 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [32K, 64K) 23 |@@@@@@@@@@@@@@@@@@ | [64K, 128K) 15 |@@@@@@@@@@@@ | [128K, 256K) 10 |@@@@@@@@ | [256K, 512K) 6 |@@@@ | [512K, 1M) 15 |@@@@@@@@@@@@ | [1M, 2M) 2 |@ | [2M, 4M) 14 |@@@@@@@@@@@ | [4M, 8M) 6 |@@@@ | [8M, 16M) 7 |@@@@@ | [16M, 32M) 1 | | @locked_time_level[2]: [2K, 4K) 1 | | [4K, 8K) 160 |@@@@@@@@@ | [8K, 16K) 733 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [16K, 32K) 901 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [32K, 64K) 191 |@@@@@@@@@@@ | [64K, 128K) 115 |@@@@@@ | [128K, 256K) 61 |@@@ | [256K, 512K) 70 |@@@@ | [512K, 1M) 59 |@@@ | [1M, 2M) 27 |@ | [2M, 4M) 9 | | @locked_time_level[3]: [1K, 2K) 3 | | [2K, 4K) 2 | | [4K, 8K) 5 | | [8K, 16K) 147 |@@@@@@ | [16K, 32K) 1222 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [32K, 64K) 266 |@@@@@@@@@@@ | [64K, 128K) 199 |@@@@@@@@ | [128K, 256K) 146 |@@@@@@ | [256K, 512K) 124 |@@@@@ | [512K, 1M) 17 | | [1M, 2M) 0 | | [2M, 4M) 0 | | [4M, 8M) 1 | | @locked_time_level[4]: [4K, 8K) 2 |@@ | [8K, 16K) 17 |@@@@@@@@@@@@@@@@@@@@@@ | [16K, 32K) 40 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [32K, 64K) 4 |@@@@@ | > I know this is 12 numa node machine, how many total CPUs are there? 192 CPU cores (incl HyperThreaded). --Jesper