From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEB68CDB474 for ; Fri, 13 Oct 2023 02:38:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 02B018D00D7; Thu, 12 Oct 2023 22:38:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EF6CA8D0015; Thu, 12 Oct 2023 22:38:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D96D48D00D7; Thu, 12 Oct 2023 22:38:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C7A338D0015 for ; Thu, 12 Oct 2023 22:38:54 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 957FF8048D for ; Fri, 13 Oct 2023 02:38:54 +0000 (UTC) X-FDA: 81338880588.11.406E587 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) by imf09.hostedemail.com (Postfix) with ESMTP id C264C14000D for ; Fri, 13 Oct 2023 02:38:52 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=la2WS9Zk; spf=pass (imf09.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.51 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697164732; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=phWhzj3KBeGp1S5QdijAT03d7GpdmyxZnvI+iabRGkI=; b=GwMR87DqgSvKIc+6lSrayW7YXsvsSK3wNs/C79FLT1gMEZ+1sG7GJFgDY+ge7IIkwsYWwL KYBYkPGEXvBC+19RbqBYEH4qP8Xjui8h+TweGCDmGh9Rg1gfF+g5lM9hAs5hkNntJQ/hN3 7SoNKWuxr6VPDKDRxvhkInENe7oiPxc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697164732; a=rsa-sha256; cv=none; b=R1ai8Z39L20wFB08aUdu+vdHXVs94cPjMvGfUq6cg1JRd7BafY0iBng3/OZOhIyfM2FTuN PZcnPwrT0bFSqoDv2zHOyQJqqeaOpH02TtV+ecC50qp6e0F6HbHdrmk7Gf/aYCyjFW1F9/ e+C336reFbc70Ey8g4TiJB6LwXHf9FI= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=la2WS9Zk; spf=pass (imf09.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.51 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-9ae75ece209so262803366b.3 for ; Thu, 12 Oct 2023 19:38:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1697164731; x=1697769531; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=phWhzj3KBeGp1S5QdijAT03d7GpdmyxZnvI+iabRGkI=; b=la2WS9ZkCo5MMeErX6/q9vtXWIIGZ9NgvxR12kTNV2paqhWy0/S/3I6Lmos8jbWrJJ 9AKjbsRIHvRl4w1nuz00udpTCfBUeZ4+4r2W6fQl6tB6EnRBGH1AgtvbW0CH35MUPYwV kmiy7gF3FFHIcylCRLD+oH3l83WrTh5vjE6JXem+MTCrPaVKyR+MXni2It+d6M3LXdsl WHo5VKVj2Qk5ItioMlZ0hVLIf2Ir6P+iRGvqINxF9ZXpXAFgl6ezWwIhXw1UVAprOFUP 8N5pGJWw1SpJd7SfMwkFf02YjPxPa5GovQWJ1csAr32LcbEmkghX7zLRwPm3FnkaSakt VnGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697164731; x=1697769531; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=phWhzj3KBeGp1S5QdijAT03d7GpdmyxZnvI+iabRGkI=; b=ohAfHBZCyKsghWB5wHOa3oB+YCiW9VJQlU5aGocHXODEIxvUjGkA42WKP+y/jP93Fe sY/9vN+mKjVFGMNq6iv/QKNW75g+WShQRsRxo4fhcGD8FPp5x9NV95AI8Et2DUIh3sZM E8wH8nZmbSD7wyWbJV0c/e7HccqMSuEi/1YPOatjU2y764PfAJs6safJOpX8Tso9sZjO BmybfIzdJHU95HxwFLFUhkstgX8jrPIIwEz+O+ak38JdEYavtp6T4R25/Cbcbnk2zV0q qyKwj0MDD+QP5tkiQk8hdfQiemRw3dpzpEh8RiHL4sNUvweYGlVB7yDYEYomlO3MA9ur /Y0w== X-Gm-Message-State: AOJu0Yy+xrGTbkzJeOFw62JvjFOl42DwKnYhKUWpvNVva1K8q3xR8GPj rTr2h2R77lnaaJPQgx+s/fX3U6zv3GneLkFS1qY+ZA== X-Google-Smtp-Source: AGHT+IGRXCZ+TRpCk0/Kqrt/KCkKlF9/hfbFXcuhNnmWZ2bqjM1/An4E7omja7b+DsDYbZ66D5RfxOj4ExmFVPGXpOE= X-Received: by 2002:a17:906:8467:b0:9ae:62ec:f4a1 with SMTP id hx7-20020a170906846700b009ae62ecf4a1mr22417903ejc.33.1697164731128; Thu, 12 Oct 2023 19:38:51 -0700 (PDT) MIME-Version: 1.0 References: <20231011003646.dt5rlqmnq6ybrlnd@google.com> <20231012132946.GA470544@cmpxchg.org> <20231013023329.GG470544@cmpxchg.org> In-Reply-To: <20231013023329.GG470544@cmpxchg.org> From: Yosry Ahmed Date: Thu, 12 Oct 2023 19:38:12 -0700 Message-ID: Subject: Re: [PATCH v2 3/5] mm: memcg: make stats flushing threshold per-memcg To: Johannes Weiner Cc: Shakeel Butt , Andrew Morton , Michal Hocko , Roman Gushchin , Muchun Song , Ivan Babrou , Tejun Heo , =?UTF-8?Q?Michal_Koutn=C3=BD?= , Waiman Long , kernel-team@cloudflare.com, Wei Xu , Greg Thelen , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: C264C14000D X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: z6yqkm1w7ju6f9mh1ntd68nxoyjc3j3y X-HE-Tag: 1697164732-694372 X-HE-Meta: U2FsdGVkX18tA6by4w1Z/DvK42X31sb19cTEMuRGk9IODZ2o8RRcUfBecmm+PMs55F4sdTMIZ8W2HBbu4MLUPe1MJU+AcVU9QCzFb+FC7mKC01E7lG/x0S/JA/+SfDteXmBTqNOwaanb2arpIzUKcTB9TqIeVb6QAzhQRHZknlYE/zTNTfK67YOs44PSuZxMsh3EqIBAH91bYgKGhfsb1wtGEZtXhl51dhD+nFtT5hyY8Gok1W4gIW1RRUSEHnHm9UaANw8XHbQJTjhGQLHHBq279oQxmWFI+PTgf9YMmr/pqTUsQF3ntMteKR/JCPE3j/nUXsPwiGcLAQHOy4YBtnYXWipuSafJ++NMfDIgnyGRAk12ClYeE28cffMLOH+5oK0FGfNz8tChGMKrHyfsGWSyU4uSlBR8tDVlWgz6yfuOnD1Ea134+z4t++LXHilwS+AhpgcsU0GojiBvhi+0J7IeDpab6oZb4ysU+P5PK+QEvtVlEwESp73PHEILsA5BvxhDYUeSB/IslxwEGKwUIYkbKlIPwfp2j8SgMN3uHRiO+NyEi+qX59wRcw7oUiFai3DjM49CTNksPT5AAJFnHgJqHzB92DQLz1cQQODysS1S23eFn2b9trMtFZnz/nYFZb6ssCqVXqMph3GA9vBXEpBlclRhAotdJSFynZ5aCmTsXBB2OYvOUil+VMkIvzSBztmgeJOGl3KuzhS8+b5uCJQMI5mMECBg4P+3hXLS9YliYWBSSsAL7pukbqRpuTWNfqRh+BnjcqaPEuTXSzcWuSeWwBfSAQLhqS4NHGkFfFggqA7c2uJ+V5r7/8nti/Nc2yuLCHEElmM1/JKPK6W7LgjJgx26FTg6gFmPF91ewjC5kvZAkphnLIJF4Cif/b9j8cGw1wjA5dEb0Ch+KP6z89tN2RM5Jnh96ix3adF4T/YShwLAU4HLBs244hYWQPUnjvY/TlZO64e6ELK80Ko H57KJMW4 zBfTLZ/xAmToM/WkZLPZ8/e7Rzv688uuiYl7stoOfNE2fSbJ0wp03b25fd2VmYtr+4yJ5Aw49attVpeaEHY9EGPEvPCWd/8UJeISl4f+hBHBJ/UGDSN/Q/b709MI2WPA0yFCqA2KtedTggjLO5XSJbUPO51JyOJwGARdVYWjLuoYrt2S2Xk6SNWDO+TUJuYg9oFgK+STMhLdr1DyFuV6fSkji7Rby5bIdzUje5lV5IxA1oqIbxGEC5Wka5PvR0ueBBQC5c+OZUnffawvLLVwhpMrp/0J5AbXMDReEKLgozBG6GEX9OXpOU9QWiV0uTo3P3oNA5ZJBAlZD+UEe5UdRnKyeKg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.002908, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Oct 12, 2023 at 7:33=E2=80=AFPM Johannes Weiner wrote: > > On Thu, Oct 12, 2023 at 04:28:49PM -0700, Yosry Ahmed wrote: > > [..] > > > > > > > > > > Using next-20231009 and a similar 44 core machine with hyperthrea= ding > > > > > disabled, I ran 22 instances of netperf in parallel and got the > > > > > following numbers from averaging 20 runs: > > > > > > > > > > Base: 33076.5 mbps > > > > > Patched: 31410.1 mbps > > > > > > > > > > That's about 5% diff. I guess the number of iterations helps redu= ce > > > > > the noise? I am not sure. > > > > > > > > > > Please also keep in mind that in this case all netperf instances = are > > > > > in the same cgroup and at a 4-level depth. I imagine in a practic= al > > > > > setup processes would be a little more spread out, which means le= ss > > > > > common ancestors, so less contended atomic operations. > > > > > > > > > > > > (Resending the reply as I messed up the last one, was not in plain = text) > > > > > > > > I was curious, so I ran the same testing in a cgroup 2 levels deep > > > > (i.e /sys/fs/cgroup/a/b), which is a much more common setup in my > > > > experience. Here are the numbers: > > > > > > > > Base: 40198.0 mbps > > > > Patched: 38629.7 mbps > > > > > > > > The regression is reduced to ~3.9%. > > > > > > > > What's more interesting is that going from a level 2 cgroup to a le= vel > > > > 4 cgroup is already a big hit with or without this patch: > > > > > > > > Base: 40198.0 -> 33076.5 mbps (~17.7% regression) > > > > Patched: 38629.7 -> 31410.1 (~18.7% regression) > > > > > > > > So going from level 2 to 4 is already a significant regression for > > > > other reasons (e.g. hierarchical charging). This patch only makes i= t > > > > marginally worse. This puts the numbers more into perspective imo t= han > > > > comparing values at level 4. What do you think? > > > > > > I think it's reasonable. > > > > > > Especially comparing to how many cachelines we used to touch on the > > > write side when all flushing happened there. This looks like a good > > > trade-off to me. > > > > Thanks. > > > > Still wanting to figure out if this patch is what you suggested in our > > previous discussion [1], to add a > > Suggested-by if appropriate :) > > > > [1]https://lore.kernel.org/lkml/20230913153758.GB45543@cmpxchg.org/ > > Haha, sort of. I suggested the cgroup-level flush-batching, but my > proposal was missing the clever upward propagation of the pending stat > updates that you added. > > You can add the tag if you're feeling generous, but I wouldn't be mad > if you don't! I like to think that I am a generous person :) Will add it in the next respin.