Re: [RFC PATCH v2] mm: use per-numa-node atomics instead of percpu_counters

[Mateusz Guzik <mjguzik@gmail.com>]

On Tue, Apr 1, 2025 at 12:36 AM Sweet Tea Dorminy
<sweettea-kernel@dorminy.me> wrote:
>
> [Resend as requested as RFC and minus prereq-patch-id junk]
>
> Recently, several internal services had an RSS usage regression as part of a
> kernel upgrade. Previously, they were on a pre-6.2 kernel and were able to
> read RSS statistics in a backup watchdog process to monitor and decide if
> they'd overrun their memory budget. Now, however, a representative service
> with five threads, expected to use about a hundred MB of memory, on a 250-cpu
> machine had memory usage tens of megabytes different from the expected amount
> -- this constituted a significant percentage of inaccuracy, causing the
> watchdog to act.
>
[snip]
> I think the important part is that this improves accuracy; the current
> scheme is difficult to use on many-cored machines. It improves
> performance, but there are tradeoffs; but it tightly bounds the
> inaccuracy so that decisions can actually be reasonably made with the
> resulting numbers.
>

Even disregarding this specific report, a prior patch submission
points a result which is so off that it already constitutes a bug:
https://lwn.net/ml/linux-kernel/20220728204511.56348-1-ryncsn@gmail.com/

So something definitely needs to be done to improve the accuracy. But
that always will be a tradeoff vs update performance.

This brings me to a question: how often does the watchdog thing look
at the stats?

I wonder if it would make sense add another file to proc, similar to
"status", but returning *exact* values. So in particular with percpu
counters it would walk all CPUs on to generate the answer.

Then interested parties would still get an accurate count and not get
in the way, provided they don't relentlessly do it.
-- 
Mateusz Guzik <mjguzik gmail.com>