From: "tj@kernel.org" <tj@kernel.org>
To: "Wlodarczyk, Bertrand" <bertrand.wlodarczyk@intel.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>,
"hannes@cmpxchg.org" <hannes@cmpxchg.org>,
"mkoutny@suse.com" <mkoutny@suse.com>,
"cgroups@vger.kernel.org" <cgroups@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"inwardvessel@gmail.com" <inwardvessel@gmail.com>
Subject: Re: [PATCH v2] cgroup/rstat: change cgroup_base_stat to atomic
Date: Fri, 4 Jul 2025 07:57:47 -1000 [thread overview]
Message-ID: <aGgWG_eSHVxntfEh@slm.duckdns.org> (raw)
In-Reply-To: <CH3PR11MB7894766BBF3A24A4174D41A5F142A@CH3PR11MB7894.namprd11.prod.outlook.com>
Hello,
On Fri, Jul 04, 2025 at 01:13:56PM +0000, Wlodarczyk, Bertrand wrote:
...
> After 54 units both solutions have the same result.
> What's the issue here? Why user seeing A = 1, B = 0, C = 0 in 22 unit (instead of spin) is a bad thing in rstat scenario?
Because some stats are related to each other - e.g. in blkcg, BPS and IOPS.
Here, overlapping cputime stats if we ever add [soft]irq time breakdowns,
and that can lead to non-sensical calculations (divide by zero, underflow,
and so on) in its users, just rare enough to not debugged easily but
frequent enough to be a headache in larger / longer deployments. And,
because we can usually do better.
> > Can you please try a different approach?
>
> In last few days I've investigated this, have some success but nowhere
> near to the improvements yield by atomics use. For the reasons I mentioned
> above, locks approach is much more complex to optimize.
So, I'm not converting these stats to atomics. It's just not a good long
term direction. Please find a better solution. I'm pretty sure there are
multiple.
>> Yeah, I saw the benchmark but I was more curious what actual use case
>> would lead to behaviors like that because you'd have to hammer on those
>> stats really hard for this to be a problem. In most use cases that I'm
>> aware of, the polling frequencies of these stats are >= 1sec. I guess the
>> users in your use case were banging on them way harder, at least
>> previously.
>
> From what I know, the https://github.com/google/cadvisor instances
> deployed on the client machine hammered these stats. Sharing servers
> between independent teams or orgs in big corps is frequent. Every
> interested party deployed its own, or similar, instance. We can say just
> don't do that and be fine, but it will be happening anyway. It's better to
> just make rstats more robust.
I do think this is a valid use case. I just want to get some sense on the
numbers involved. Do you happen to know what frequency cAdvisor was polling
the stats at and how many instances were running? The numbers don't have to
be accurate. I just want to know the ballpark numbers.
Thanks.
--
tejun
next prev parent reply other threads:[~2025-07-04 17:57 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-06-24 14:45 [PATCH v2] cgroup/rstat: change cgroup_base_stat to atomic Bertrand Wlodarczyk
2025-06-26 19:15 ` Shakeel Butt
2025-06-27 13:15 ` Wlodarczyk, Bertrand
2025-06-27 16:50 ` tj
2025-06-30 14:25 ` Wlodarczyk, Bertrand
2025-06-30 15:48 ` tj
2025-07-04 13:13 ` Wlodarczyk, Bertrand
2025-07-04 17:57 ` tj [this message]
2025-07-21 11:48 ` Wlodarczyk, Bertrand
2025-06-27 16:55 ` JP Kobryn
2025-06-27 17:17 ` Shakeel Butt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aGgWG_eSHVxntfEh@slm.duckdns.org \
--to=tj@kernel.org \
--cc=bertrand.wlodarczyk@intel.com \
--cc=cgroups@vger.kernel.org \
--cc=hannes@cmpxchg.org \
--cc=inwardvessel@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mkoutny@suse.com \
--cc=shakeel.butt@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).