* Re: [PATCH v2 00/10] sched: Flatten the pick
From: Peter Zijlstra @ 2026-05-18 7:14 UTC (permalink / raw)
To: Tejun Heo
Cc: mingo, longman, chenridong, juri.lelli, vincent.guittot,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid, hannes,
mkoutny, cgroups, linux-kernel, jstultz, kprateek.nayak, qyousef
In-Reply-To: <agN1QbsjFv2aXFhK@slm.duckdns.org>
On Tue, May 12, 2026 at 08:45:21AM -1000, Tejun Heo wrote:
> Hello, Peter.
>
> On Tue, May 12, 2026 at 10:10:00AM +0200, Peter Zijlstra wrote:
> ...
> > Anyway, this is why I've been looking at these alternative weight
> > schemes, to get the nominal fraction near 1 and make these problems go
> > away. It is both the numerical issues and the disparity between levels
> > (with root being at level 0 being the most obvious).
>
> I see. I think what bothers me is that I'm unsure what the weight config
> would mean when the shares are scaled by the number of active cpus in that
> cgroup.
Relative weight per active cpu :-), but yes, that is a somewhat more
difficult concept I suppose.
> Here's a simple example:
>
> - There are 256 cpus.
> - /cgroup-A has weight 100 and 128 active threads. No pinning.
> - /cgroup-B has weight 100 and 256 active thredas. No pinning.
>
> In the current code, assuming math holds up, cgroup-A and B would get about
> the same shares - ~128 CPUs each. However, if we scale the share by active
> CPUs in each cgroup, B's tasks would end up with the same weight as A's on
> CPUs that they end up competing on, which would lead to ~ 1:3 distribution.
> Is that the right reading of the code?
Indeed. So both A and B will get ~1024 weight per (active) CPU, such
that on the CPUs they contend they will get 1:1 and then B will get the
full CPU on the uncontested CPUs, resulting in a total of 1:3
distribution.
This can of course be compensated by increasing the relative
weight of A, if that is so desired. But the alternative view is that for
those 128 CPUs they overlap, A and B will get equal parts, it is just
that B consumes another 128 CPUs and will not have contention there.
So the current scheme will inflate the part of A to be double the weight
(of B), giving them 2 out of 3 parts on the contended CPUs, but then B
will still get complete / uncontested access to those extra 128 CPUs,
resulting in a 2:4 weight distribution.
Which also isn't as straight forward as one might think.
So perhaps 'weight on the CPUs you contest on' isn't as unintuitive as
it seems on first glance, its just different.
And it has tremendous advantages as outlined before; it is naturally
normalized -- the disparity between nesting levels goes away, and the
edge case of a single CPU active will be sane.
Eg. consider your example except now A will have 1 active thread. Then A
will get the full group weight (1024) on its one CPU, while B will get
(1024/256=8) on each CPU.
So for the one contended CPU A gets 256 out of 257 parts, while B gets
the full CPU for the remaining 255 CPUs, for a:
256 1 257
--- : --- + 255*--- = 256:65535 ~ 1:256
257 257 257
distribution. While with the new scheme it would be:
1 1 2
- : - + 255*- = 1:511
2 2 2
Which, realistically isn't all that different, except the old scheme has
this really large weight to deal with.
So from where I'm sitting, yes different, but it behaves better.
^ permalink raw reply
* Re: [PATCH RFC 2/5] dma-heap: charge dma-buf memory via explicit memcg
From: Christian König @ 2026-05-18 7:19 UTC (permalink / raw)
To: T.J. Mercier, Christian Brauner
Cc: Albert Esteve, Tejun Heo, Johannes Weiner, Michal Koutný,
Jonathan Corbet, Shuah Khan, Sumit Semwal, Michal Hocko,
Roman Gushchin, Shakeel Butt, Muchun Song, Andrew Morton,
Benjamin Gaignard, Brian Starkey, John Stultz, Paul Moore,
James Morris, Serge E. Hallyn, Stephen Smalley, Ondrej Mosnacek,
Shuah Khan, cgroups, linux-doc, linux-kernel, linux-media,
dri-devel, linaro-mm-sig, linux-mm, linux-security-module,
selinux, linux-kselftest, mripard, echanude
In-Reply-To: <CABdmKX0d6Zsg+_TxXjB80UZR23ZvXzxYoWzORgwmx=ZiuE+Nzw@mail.gmail.com>
On 5/15/26 19:06, T.J. Mercier wrote:
> On Fri, May 15, 2026 at 6:53 AM Christian Brauner <brauner@kernel.org> wrote:
>>
>> On Tue, May 12, 2026 at 11:10:44AM +0200, Albert Esteve wrote:
>>> On embedded platforms a central process often allocates dma-buf
>>> memory on behalf of client applications. Without a way to
>>> attribute the charge to the requesting client's cgroup, the
>>> cost lands on the allocator, making per-cgroup memory limits
>>> ineffective for the actual consumers.
>>>
>>> Add charge_pid_fd to struct dma_heap_allocation_data. When set to
>>
>> Please be aware that pidfds come in two flavors:
>>
>> thread-group pidfds and thread-specific pidfds. Make sure that your API
>> doesn't implicitly depend on this distinction not existing.
>
> Hi Christian,
>
> Memcg is not a controller that supports "thread mode" so all threads
> in a group should belong to the same memcg.
BTW: Exactly that is the requirement automotive has with their native context use case.
The use case is that you have a deamon which has multiple threads were each one is acting on behalve of some other process.
At the moment we basically say they are simply not using cgroups for that use case, but it would be really nice if we could handle that as well.
Summarizing the requirement of that use case: You need a different cgroup for each thread of a process.
Regards,
Christian.
>
> Checking the flags from pidfd_get_pid would be the best way for an
> explicit check of the pidfd type?
>
>>> a valid pidfd, DMA_HEAP_IOCTL_ALLOC resolves the target task's
>>> memcg and charges the buffer there via mem_cgroup_charge_dmabuf()
>>> inside dma_heap_buffer_alloc(). Without charge_pid_fd, and with
>>> the mem_accounting module parameter enabled, the buffer is charged
>>> to the allocator's own cgroup.
>>>
>>> Additionally, commit 3c227be90659 ("dma-buf: system_heap: account for
>>> system heap allocation in memcg") adds __GFP_ACCOUNT to system-heap
>>> page allocations. Keeping __GFP_ACCOUNT would charge the same pages
>>> twice (once to kmem, once to MEMCG_DMABUF), thus remove it and route
>>> all accounting through a single MEMCG_DMABUF path.
>>>
>>> Usage examples:
>>>
>>> 1. Central allocator charging to a client at allocation time.
>>> The allocator knows the client's PID (e.g., from binder's
>>> sender_pid) and uses pidfd to attribute the charge:
>>>
>>> pid_t client_pid = txn->sender_pid;
>>> int pidfd = pidfd_open(client_pid, 0);
>>>
>>> struct dma_heap_allocation_data alloc = {
>>> .len = buffer_size,
>>> .fd_flags = O_RDWR | O_CLOEXEC,
>>> .charge_pid_fd = pidfd,
>>> };
>>> ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc);
>>> close(pidfd);
>>> /* alloc.fd is now charged to client's cgroup */
>>>
>>> 2. Default allocation (no pidfd, mem_accounting=1).
>>> When charge_pid_fd is not set and the mem_accounting module
>>> parameter is enabled, the buffer is charged to the allocator's
>>> own cgroup:
>>>
>>> struct dma_heap_allocation_data alloc = {
>>> .len = buffer_size,
>>> .fd_flags = O_RDWR | O_CLOEXEC,
>>> };
>>> ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc);
>>> /* charged to current process's cgroup */
>>>
>>> Current limitations:
>>>
>>> - Single-owner model: a dma-buf carries one memcg charge regardless of
>>> how many processes share it. Means only the first owner (and exporter)
>>> of the shared buffer bears the charge.
>>> - Only memcg accounting supported. While this makes sense for system
>>> heap buffers, other heaps (e.g., CMA heaps) will require selectively
>>> charging also for the dmem controller.
>>>
>>> Signed-off-by: Albert Esteve <aesteve@redhat.com>
>>> ---
>>> Documentation/admin-guide/cgroup-v2.rst | 5 ++--
>>> drivers/dma-buf/dma-buf.c | 16 ++++---------
>>> drivers/dma-buf/dma-heap.c | 42 ++++++++++++++++++++++++++++++---
>>> drivers/dma-buf/heaps/system_heap.c | 2 --
>>> include/uapi/linux/dma-heap.h | 6 +++++
>>> 5 files changed, 53 insertions(+), 18 deletions(-)
>>>
>>> diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
>>> index 8bdbc2e866430..824d269531eb1 100644
>>> --- a/Documentation/admin-guide/cgroup-v2.rst
>>> +++ b/Documentation/admin-guide/cgroup-v2.rst
>>> @@ -1636,8 +1636,9 @@ The following nested keys are defined.
>>> structures.
>>>
>>> dmabuf (npn)
>>> - Amount of memory used for exported DMA buffers allocated by the cgroup.
>>> - Stays with the allocating cgroup regardless of how the buffer is shared.
>>> + Amount of memory used for exported DMA buffers allocated by or on
>>> + behalf of the cgroup. Stays with the allocating cgroup regardless
>>> + of how the buffer is shared.
>>>
>>> workingset_refault_anon
>>> Number of refaults of previously evicted anonymous pages.
>>> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
>>> index ce02377f48908..23fb758b78297 100644
>>> --- a/drivers/dma-buf/dma-buf.c
>>> +++ b/drivers/dma-buf/dma-buf.c
>>> @@ -181,8 +181,11 @@ static void dma_buf_release(struct dentry *dentry)
>>> */
>>> BUG_ON(dmabuf->cb_in.active || dmabuf->cb_out.active);
>>>
>>> - mem_cgroup_uncharge_dmabuf(dmabuf->memcg, PAGE_ALIGN(dmabuf->size) / PAGE_SIZE);
>>> - mem_cgroup_put(dmabuf->memcg);
>>> + if (dmabuf->memcg) {
>>> + mem_cgroup_uncharge_dmabuf(dmabuf->memcg,
>>> + PAGE_ALIGN(dmabuf->size) / PAGE_SIZE);
>>> + mem_cgroup_put(dmabuf->memcg);
>>> + }
>>>
>>> dmabuf->ops->release(dmabuf);
>>>
>>> @@ -764,13 +767,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
>>> dmabuf->resv = resv;
>>> }
>>>
>>> - dmabuf->memcg = get_mem_cgroup_from_mm(current->mm);
>>> - if (!mem_cgroup_charge_dmabuf(dmabuf->memcg, PAGE_ALIGN(dmabuf->size) / PAGE_SIZE,
>>> - GFP_KERNEL)) {
>>> - ret = -ENOMEM;
>>> - goto err_memcg;
>>> - }
>>> -
>>> file->private_data = dmabuf;
>>> file->f_path.dentry->d_fsdata = dmabuf;
>>> dmabuf->file = file;
>>> @@ -781,8 +777,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
>>>
>>> return dmabuf;
>>>
>>> -err_memcg:
>>> - mem_cgroup_put(dmabuf->memcg);
>>> err_file:
>>> fput(file);
>>> err_module:
>>> diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c
>>> index ac5f8685a6494..ff6e259afcdc0 100644
>>> --- a/drivers/dma-buf/dma-heap.c
>>> +++ b/drivers/dma-buf/dma-heap.c
>>> @@ -7,13 +7,17 @@
>>> */
>>>
>>> #include <linux/cdev.h>
>>> +#include <linux/cgroup.h>
>>> #include <linux/device.h>
>>> #include <linux/dma-buf.h>
>>> #include <linux/dma-heap.h>
>>> +#include <linux/memcontrol.h>
>>> +#include <linux/sched/mm.h>
>>> #include <linux/err.h>
>>> #include <linux/export.h>
>>> #include <linux/list.h>
>>> #include <linux/nospec.h>
>>> +#include <linux/pidfd.h>
>>> #include <linux/syscalls.h>
>>> #include <linux/uaccess.h>
>>> #include <linux/xarray.h>
>>> @@ -55,10 +59,12 @@ MODULE_PARM_DESC(mem_accounting,
>>> "Enable cgroup-based memory accounting for dma-buf heap allocations (default=false).");
>>>
>>> static int dma_heap_buffer_alloc(struct dma_heap *heap, size_t len,
>>> - u32 fd_flags,
>>> - u64 heap_flags)
>>> + u32 fd_flags, u64 heap_flags,
>>> + struct mem_cgroup *charge_to)
>>> {
>>> struct dma_buf *dmabuf;
>>> + unsigned int nr_pages;
>>> + struct mem_cgroup *memcg = charge_to;
>>> int fd;
>>>
>>> /*
>>> @@ -73,6 +79,22 @@ static int dma_heap_buffer_alloc(struct dma_heap *heap, size_t len,
>>> if (IS_ERR(dmabuf))
>>> return PTR_ERR(dmabuf);
>>>
>>> + nr_pages = len / PAGE_SIZE;
>>> +
>>> + if (memcg)
>>> + css_get(&memcg->css);
>>> + else if (mem_accounting)
>>> + memcg = get_mem_cgroup_from_mm(current->mm);
>>> +
>>> + if (memcg) {
>>> + if (!mem_cgroup_charge_dmabuf(memcg, nr_pages, GFP_KERNEL)) {
>>> + mem_cgroup_put(memcg);
>>> + dma_buf_put(dmabuf);
>>> + return -ENOMEM;
>>> + }
>>> + dmabuf->memcg = memcg;
>>> + }
>>> +
>>> fd = dma_buf_fd(dmabuf, fd_flags);
>>> if (fd < 0) {
>>> dma_buf_put(dmabuf);
>>> @@ -102,6 +124,9 @@ static long dma_heap_ioctl_allocate(struct file *file, void *data)
>>> {
>>> struct dma_heap_allocation_data *heap_allocation = data;
>>> struct dma_heap *heap = file->private_data;
>>> + struct mem_cgroup *memcg = NULL;
>>> + struct task_struct *task;
>>> + unsigned int pidfd_flags;
>>> int fd;
>>>
>>> if (heap_allocation->fd)
>>> @@ -113,9 +138,20 @@ static long dma_heap_ioctl_allocate(struct file *file, void *data)
>>> if (heap_allocation->heap_flags & ~DMA_HEAP_VALID_HEAP_FLAGS)
>>> return -EINVAL;
>>>
>>> + if (heap_allocation->charge_pid_fd) {
>>> + task = pidfd_get_task(heap_allocation->charge_pid_fd, &pidfd_flags);
>>
>> Will always get a thread-group leader pidfd and will fail if this is a
>> thread-specific pidfd. pidfd_open(1234, PIDFD_THREAD) can be used to
>> open a thread-specific pidfd.
>>
>>> + if (IS_ERR(task))
>>> + return PTR_ERR(task);
>>> +
>>> + memcg = get_mem_cgroup_from_mm(task->mm);
>>> + put_task_struct(task);
>>> + }
>>> +
>>> fd = dma_heap_buffer_alloc(heap, heap_allocation->len,
>>> heap_allocation->fd_flags,
>>> - heap_allocation->heap_flags);
>>> + heap_allocation->heap_flags,
>>> + memcg);
>>> + mem_cgroup_put(memcg);
>>> if (fd < 0)
>>> return fd;
>>>
>>> diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c
>>> index 03c2b87cb1112..95d7688167b93 100644
>>> --- a/drivers/dma-buf/heaps/system_heap.c
>>> +++ b/drivers/dma-buf/heaps/system_heap.c
>>> @@ -385,8 +385,6 @@ static struct page *alloc_largest_available(unsigned long size,
>>> if (max_order < orders[i])
>>> continue;
>>> flags = order_flags[i];
>>> - if (mem_accounting)
>>> - flags |= __GFP_ACCOUNT;
>>> page = alloc_pages(flags, orders[i]);
>>> if (!page)
>>> continue;
>>> diff --git a/include/uapi/linux/dma-heap.h b/include/uapi/linux/dma-heap.h
>>> index a4cf716a49fa6..e02b0f8cbc6a1 100644
>>> --- a/include/uapi/linux/dma-heap.h
>>> +++ b/include/uapi/linux/dma-heap.h
>>> @@ -29,6 +29,10 @@
>>> * handle to the allocated dma-buf
>>> * @fd_flags: file descriptor flags used when allocating
>>> * @heap_flags: flags passed to heap
>>> + * @charge_pid_fd: optional pidfd of the process whose cgroup should be
>>> + * charged for this allocation; 0 means charge the calling
>>> + * process's cgroup
>>> + * @__padding: reserved, must be zero
>>> *
>>> * Provided by userspace as an argument to the ioctl
>>> */
>>> @@ -37,6 +41,8 @@ struct dma_heap_allocation_data {
>>> __u32 fd;
>>> __u32 fd_flags;
>>> __u64 heap_flags;
>>> + __u32 charge_pid_fd;
>>> + __u32 __padding;
>>> };
>>>
>>> #define DMA_HEAP_IOC_MAGIC 'H'
>>>
>>> --
>>> 2.53.0
>>>
^ permalink raw reply
* Re: [Linaro-mm-sig] Re: [PATCH RFC 2/5] dma-heap: charge dma-buf memory via explicit memcg
From: Christian König @ 2026-05-18 7:34 UTC (permalink / raw)
To: Barry Song, T.J. Mercier
Cc: Albert Esteve, Tejun Heo, Johannes Weiner, Michal Koutný,
Jonathan Corbet, Shuah Khan, Sumit Semwal, Michal Hocko,
Roman Gushchin, Shakeel Butt, Muchun Song, Andrew Morton,
Benjamin Gaignard, Brian Starkey, John Stultz, Christian Brauner,
Paul Moore, James Morris, Serge E. Hallyn, Stephen Smalley,
Ondrej Mosnacek, Shuah Khan, cgroups, linux-doc, linux-kernel,
linux-media, dri-, linaro-mm-sig, linux-mm, linux-security-module,
selinux, linux-kselftest, mripard, echanude
In-Reply-To: <CAGsJ_4zyecY6E-=Tm4_couT7uoM9LMcFdTMUPkZAjj4zUKE-dQ@mail.gmail.com>
On 5/16/26 11:19, Barry Song wrote:
> On Thu, May 14, 2026 at 12:35 AM T.J. Mercier <tjmercier@google.com> wrote:
> [...]
>>>> I have a question about this part. Albert I guess you are interested
>>>> only in accounting dmabuf-heap allocations, or do you expect to add
>>>> __GFP_ACCOUNT or mem_cgroup_charge_dmabuf calls to other
>>>> non-dmabuf-heap exporters?
>>>
>>> We're scoping this to dma-buf heaps for now. CMA heaps and the dmem
>>> controller are on the radar for follow-up/parallel work (there will be
>>> dragons and will surely need discussion). For DRM and V4L2 the
>>> long-term intent is migration to heaps, which would make direct
>>> accounting on those paths unnecessary.
>>
>> Ah I see. GEM buffers exported to dmabufs are what I had in mind. I
>> guess this would only leave the odd non-DRM driver with the need to
>> add their own accounting calls, which I don't expect would be a big
>> problem.
>>
>
> sounds like we still have a long way to go to correctly account for
> various v4l2, drm, GEM, CMA, etc. In patch 1, the charging is done in
> dma_buf_export(), so I guess it covers all dma-buf types except
> dma_heap, but the problem is that it has no remote charging support at
> all?
No, just the other way around
DMA-buf heaps can be handled here because we know that it is pure system memory and nothing special so memcg always applies.
dma_buf_export() on the other hand handles tons of different use cases, ranging from buffer accounted to dmem, over special resources which aren't even memory all the way to buffers which can migrate from dmem to memcg and back during their lifetime.
>>> udmabufs are already
>>> memcg-charged, so adding a separate MEMCG_DMABUF would double count.
>>> Are there any other exporters you had in mind that would benefit from
>>> this approach?
Well apart from DMA-buf memfd_create() is one of the things which as broken our neck in the past a couple of times.
But thinking more about it what if instead of making this DMA-buf heaps specific what if we have a general cgroups function which allows to change accounting of a buffer referenced by a file descriptor to a different process?
That would cover not only the DMA-buf heaps use case, but also all other DMA-buf with dmem and whatever we come up in the future as well.
The only drawback I can see is that DMA-buf heap allocations would be temporarily accounted to the memory allocation daemon, but I don't think that this would be a problem.
Regards,
Christian.
>
> Thanks
> Barry
^ permalink raw reply
* [PATCH] mm: memcontrol: propagate NMI slab stats to memcg vmstats
From: Alexandre Ghiti @ 2026-05-18 8:28 UTC (permalink / raw)
To: Andrew Morton
Cc: Johannes Weiner, Michal Hocko, Roman Gushchin, Shakeel Butt,
Muchun Song, Vlastimil Babka, stable, linux-mm, linux-kernel,
cgroups, Alexandre Ghiti
flush_nmi_stats() drains per-node NMI slab atomics into the per-node
lruvec_stats, but does not propagate them to the memcg-level vmstats.
For non NMI case, account_slab_nmi_safe() calls mod_memcg_lruvec_state()
which updates both per-node lruvec_stats and memcg-level vmstats, so
flush_nmi_stats() needs to flush to per-node lruvec_stats as well as
memcg-level vmstats.
So fix this by flushing to the memcg-level vmstats for NMI too.
Fixes: 940b01fc8dc1 ("memcg: nmi safe memcg stats for specific archs")
Cc: stable@vger.kernel.org
Acked-by: Shakeel Butt <shakeel.butt@linux.dev>
Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
---
mm/memcontrol.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index c3d98ab41f1f..953a3c0556a5 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -4343,6 +4343,9 @@ static void flush_nmi_stats(struct mem_cgroup *memcg, struct mem_cgroup *parent,
lstats->state[index] += slab;
if (plstats)
plstats->state_pending[index] += slab;
+ memcg->vmstats->state[index] += slab;
+ if (parent)
+ parent->vmstats->state_pending[index] += slab;
}
if (atomic_read(&pn->slab_unreclaimable)) {
int slab = atomic_xchg(&pn->slab_unreclaimable, 0);
@@ -4351,6 +4354,9 @@ static void flush_nmi_stats(struct mem_cgroup *memcg, struct mem_cgroup *parent,
lstats->state[index] += slab;
if (plstats)
plstats->state_pending[index] += slab;
+ memcg->vmstats->state[index] += slab;
+ if (parent)
+ parent->vmstats->state_pending[index] += slab;
}
}
}
--
2.54.0
^ permalink raw reply related
* Re: [Linaro-mm-sig] Re: [PATCH RFC 2/5] dma-heap: charge dma-buf memory via explicit memcg
From: Albert Esteve @ 2026-05-18 12:06 UTC (permalink / raw)
To: Christian König
Cc: Barry Song, T.J. Mercier, Tejun Heo, Johannes Weiner,
Michal Koutný, Jonathan Corbet, Shuah Khan, Sumit Semwal,
Michal Hocko, Roman Gushchin, Shakeel Butt, Muchun Song,
Andrew Morton, Benjamin Gaignard, Brian Starkey, John Stultz,
Christian Brauner, Paul Moore, James Morris, Serge E. Hallyn,
Stephen Smalley, Ondrej Mosnacek, Shuah Khan, cgroups, linux-doc,
linux-kernel, linux-media, dri-, linaro-mm-sig, linux-mm,
linux-security-module, selinux, linux-kselftest, mripard,
echanude
In-Reply-To: <cb84c2ee-9de1-4565-b2e0-60984721228f@amd.com>
On Mon, May 18, 2026 at 9:34 AM Christian König
<christian.koenig@amd.com> wrote:
>
> On 5/16/26 11:19, Barry Song wrote:
> > On Thu, May 14, 2026 at 12:35 AM T.J. Mercier <tjmercier@google.com> wrote:
> > [...]
> >>>> I have a question about this part. Albert I guess you are interested
> >>>> only in accounting dmabuf-heap allocations, or do you expect to add
> >>>> __GFP_ACCOUNT or mem_cgroup_charge_dmabuf calls to other
> >>>> non-dmabuf-heap exporters?
> >>>
> >>> We're scoping this to dma-buf heaps for now. CMA heaps and the dmem
> >>> controller are on the radar for follow-up/parallel work (there will be
> >>> dragons and will surely need discussion). For DRM and V4L2 the
> >>> long-term intent is migration to heaps, which would make direct
> >>> accounting on those paths unnecessary.
> >>
> >> Ah I see. GEM buffers exported to dmabufs are what I had in mind. I
> >> guess this would only leave the odd non-DRM driver with the need to
> >> add their own accounting calls, which I don't expect would be a big
> >> problem.
> >>
> >
> > sounds like we still have a long way to go to correctly account for
> > various v4l2, drm, GEM, CMA, etc. In patch 1, the charging is done in
> > dma_buf_export(), so I guess it covers all dma-buf types except
> > dma_heap, but the problem is that it has no remote charging support at
> > all?
>
> No, just the other way around
>
> DMA-buf heaps can be handled here because we know that it is pure system memory and nothing special so memcg always applies.
>
> dma_buf_export() on the other hand handles tons of different use cases, ranging from buffer accounted to dmem, over special resources which aren't even memory all the way to buffers which can migrate from dmem to memcg and back during their lifetime.
>
> >>> udmabufs are already
> >>> memcg-charged, so adding a separate MEMCG_DMABUF would double count.
> >>> Are there any other exporters you had in mind that would benefit from
> >>> this approach?
>
> Well apart from DMA-buf memfd_create() is one of the things which as broken our neck in the past a couple of times.
>
> But thinking more about it what if instead of making this DMA-buf heaps specific what if we have a general cgroups function which allows to change accounting of a buffer referenced by a file descriptor to a different process?
>
> That would cover not only the DMA-buf heaps use case, but also all other DMA-buf with dmem and whatever we come up in the future as well.
I removed a draft adding an ioctl for charge transfer from the series
before sending because I wanted to focus on the charge_pid_fd approach
and keep things simple, deferring the recharge path to a follow-up
depending on feedback.
The main difference between my removed draft and what you're
describing, iiuc, is scope and layer: my draft was an explicit ioctl
on the dma-buf fd that the consumer calls to claim the charge (see
below), while you seem to be suggesting a more general kernel-internal
function that could work across buffer types and cgroup controllers,
so not necessarily userspace-initiated? A kernel-internal function
will need a way to identify the target process, which sounds similar
to the binder-backed approach from TJ [1]. For everything else, the
receiver still needs to declare itself, which the ioctl accomplishes.
```
# When an app imports a daemon-allocated buffer, it can transfer the
charge to itself:
int buf_fd = receive_dmabuf_from_daemon();
ioctl(buf_fd, DMA_BUF_IOCTL_XFER_CHARGE); /* charge now attributed to
apps's cgroup */
```
[1] https://lore.kernel.org/cgroups/20230109213809.418135-1-tjmercier@google.com/
>
> The only drawback I can see is that DMA-buf heap allocations would be temporarily accounted to the memory allocation daemon, but I don't think that this would be a problem.
The main reasons we moved away from TJ's transfer-based approach
toward `charge_pid_fd` are: avoid the transient charge window on the
daemon's cgroup; and to decouple from Binder, allowing any allocator
to use it.
Technically, both approaches could coexist, though. Of the three
scenarios TJ described:
- Scenario 2 is directly addressed by charge_pid_fd approach without
any transient charge on the daemon at the cost of one extra field in
the heap ioctl uAPI struct.
- Scenario 3 can be handled by the charge transfer function without
changes to SurfaceFlinger. The app or dequeueBuffer claims the charge
for itself or the app, respectively (depending on whether we include a
pid_fd field in the transfer ioctl). It also covers non-heap
exporters. The con in both variants is the transient charge window on
the daemon.
Both approaches shift the responsibility for correct charging
attribution to userspace: first, 'charge_pid_fd` on the allocator's
side, and the transfer charge on the consumer's side.
Deciding on one, the other or both depends on how much we value
avoiding transient attribution, and how much we need a non-heap
generic solution. With the XFER_CHARGE we can cover both. Thus, the
`charge_pid_fd` approach in this RFC can be seen as a
performance/strictness optimisation, eliminating transient charges to
the daemon at the cost of a permanent uAPI addition to the heap ioctl
struct, but not strictly required for correctness. On the other hand,
if we agree on the end goal of migrating other exporters to use
dma-buf heaps, and scenario 3 is addressed by adding the app's pid_fd
to SurfaceFlinger, then `charge_pid_fd` alone is a coherent/sufficient
approach despite the uAPI change.
>
> Regards,
> Christian.
>
> >
> > Thanks
> > Barry
>
^ permalink raw reply
* Re: [PATCH RFC 2/5] dma-heap: charge dma-buf memory via explicit memcg
From: Albert Esteve @ 2026-05-18 12:16 UTC (permalink / raw)
To: Barry Song
Cc: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Sumit Semwal, Christian König, Michal Hocko,
Roman Gushchin, Shakeel Butt, Muchun Song, Andrew Morton,
Benjamin Gaignard, Brian Starkey, John Stultz, T.J. Mercier,
Christian Brauner, Paul Moore, James Morris, Serge E. Hallyn,
Stephen Smalley, Ondrej Mosnacek, Shuah Khan, cgroups, linux-doc,
linux-kernel, linux-media, dri-devel, linaro-mm-sig, linux-mm,
linux-security-module, selinux, linux-kselftest, mripard,
echanude
In-Reply-To: <CAGsJ_4xfznffbjOaNKwnN6oZk_H6pqOzYqd1zx4Q9XrocdzV8A@mail.gmail.com>
On Sat, May 16, 2026 at 9:37 AM Barry Song <baohua@kernel.org> wrote:
>
> On Tue, May 12, 2026 at 5:18 PM Albert Esteve <aesteve@redhat.com> wrote:
> >
> > On embedded platforms a central process often allocates dma-buf
> > memory on behalf of client applications. Without a way to
> > attribute the charge to the requesting client's cgroup, the
> > cost lands on the allocator, making per-cgroup memory limits
> > ineffective for the actual consumers.
> >
> > Add charge_pid_fd to struct dma_heap_allocation_data. When set to
> > a valid pidfd, DMA_HEAP_IOCTL_ALLOC resolves the target task's
> > memcg and charges the buffer there via mem_cgroup_charge_dmabuf()
> > inside dma_heap_buffer_alloc(). Without charge_pid_fd, and with
> > the mem_accounting module parameter enabled, the buffer is charged
> > to the allocator's own cgroup.
> >
> > Additionally, commit 3c227be90659 ("dma-buf: system_heap: account for
> > system heap allocation in memcg") adds __GFP_ACCOUNT to system-heap
> > page allocations. Keeping __GFP_ACCOUNT would charge the same pages
> > twice (once to kmem, once to MEMCG_DMABUF), thus remove it and route
> > all accounting through a single MEMCG_DMABUF path.
> >
> [...]
>
> > - if (mem_accounting)
> > - flags |= __GFP_ACCOUNT;
>
> Hi Albert,
>
> would it be better to move this and its description to patch 1? It
> looks like patch 1 already introduces the double accounting changes,
> and patch 2 is mainly just supporting remote charging.
Hi Barry,
Thanks for looking into this series! Yes, in my head I was trying to
keep patch 1, which was taken from a previous, different series, and
then diverge from it starting with patch 2. This would clarify the
difference between the two. But I can see it just added some confusion
(for example, patch 1 charges on dma_buf_export() and then it is moved
to dma_heap_buffer_alloc() in patch 2). I will reorganize it better
for the next version, including your suggestion.
>
> Also, mem_accounting is only used by system_heap.c; has this patchset
> also eliminated its need?
No, mem_accounting is still handled in this patch for the general case
where no `charge_pid_fd` is used. See dma_heap_buffer_alloc() code:
+ if (memcg)
+ css_get(&memcg->css);
+ else if (mem_accounting)
+ memcg = get_mem_cgroup_from_mm(current->mm);
>
> Thanks
> Barry
>
^ permalink raw reply
* Re: [PATCH RFC 2/5] dma-heap: charge dma-buf memory via explicit memcg
From: Albert Esteve @ 2026-05-18 12:50 UTC (permalink / raw)
To: Christian König
Cc: T.J. Mercier, Christian Brauner, Tejun Heo, Johannes Weiner,
Michal Koutný, Jonathan Corbet, Shuah Khan, Sumit Semwal,
Michal Hocko, Roman Gushchin, Shakeel Butt, Muchun Song,
Andrew Morton, Benjamin Gaignard, Brian Starkey, John Stultz,
Paul Moore, James Morris, Serge E. Hallyn, Stephen Smalley,
Ondrej Mosnacek, Shuah Khan, cgroups, linux-doc, linux-kernel,
linux-media, dri-devel, linaro-mm-sig, linux-mm,
linux-security-module, selinux, linux-kselftest, mripard,
echanude
In-Reply-To: <208fb820-d8eb-4832-a343-ef8b360e8120@amd.com>
On Mon, May 18, 2026 at 9:20 AM Christian König
<christian.koenig@amd.com> wrote:
>
> On 5/15/26 19:06, T.J. Mercier wrote:
> > On Fri, May 15, 2026 at 6:53 AM Christian Brauner <brauner@kernel.org> wrote:
> >>
> >> On Tue, May 12, 2026 at 11:10:44AM +0200, Albert Esteve wrote:
> >>> On embedded platforms a central process often allocates dma-buf
> >>> memory on behalf of client applications. Without a way to
> >>> attribute the charge to the requesting client's cgroup, the
> >>> cost lands on the allocator, making per-cgroup memory limits
> >>> ineffective for the actual consumers.
> >>>
> >>> Add charge_pid_fd to struct dma_heap_allocation_data. When set to
> >>
> >> Please be aware that pidfds come in two flavors:
> >>
> >> thread-group pidfds and thread-specific pidfds. Make sure that your API
> >> doesn't implicitly depend on this distinction not existing.
> >
> > Hi Christian,
> >
> > Memcg is not a controller that supports "thread mode" so all threads
> > in a group should belong to the same memcg.
>
> BTW: Exactly that is the requirement automotive has with their native context use case.
>
> The use case is that you have a deamon which has multiple threads were each one is acting on behalve of some other process.
>
> At the moment we basically say they are simply not using cgroups for that use case, but it would be really nice if we could handle that as well.
>
> Summarizing the requirement of that use case: You need a different cgroup for each thread of a process.
Hi Christian,
Thanks for sharing this atuomotive usecase. If I understand correctly,
the actual requirement is attributing dma-buf charges to the right
client, not putting each daemon thread in a different cgroup? If so,
the `charge_pid_fd` approach achieves this directly by passing the
client's `pid_fd`, without needing to add per-thread cgroup
infrastructure.
>
> Regards,
> Christian.
>
> >
> > Checking the flags from pidfd_get_pid would be the best way for an
> > explicit check of the pidfd type?
> >
> >>> a valid pidfd, DMA_HEAP_IOCTL_ALLOC resolves the target task's
> >>> memcg and charges the buffer there via mem_cgroup_charge_dmabuf()
> >>> inside dma_heap_buffer_alloc(). Without charge_pid_fd, and with
> >>> the mem_accounting module parameter enabled, the buffer is charged
> >>> to the allocator's own cgroup.
> >>>
> >>> Additionally, commit 3c227be90659 ("dma-buf: system_heap: account for
> >>> system heap allocation in memcg") adds __GFP_ACCOUNT to system-heap
> >>> page allocations. Keeping __GFP_ACCOUNT would charge the same pages
> >>> twice (once to kmem, once to MEMCG_DMABUF), thus remove it and route
> >>> all accounting through a single MEMCG_DMABUF path.
> >>>
> >>> Usage examples:
> >>>
> >>> 1. Central allocator charging to a client at allocation time.
> >>> The allocator knows the client's PID (e.g., from binder's
> >>> sender_pid) and uses pidfd to attribute the charge:
> >>>
> >>> pid_t client_pid = txn->sender_pid;
> >>> int pidfd = pidfd_open(client_pid, 0);
> >>>
> >>> struct dma_heap_allocation_data alloc = {
> >>> .len = buffer_size,
> >>> .fd_flags = O_RDWR | O_CLOEXEC,
> >>> .charge_pid_fd = pidfd,
> >>> };
> >>> ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc);
> >>> close(pidfd);
> >>> /* alloc.fd is now charged to client's cgroup */
> >>>
> >>> 2. Default allocation (no pidfd, mem_accounting=1).
> >>> When charge_pid_fd is not set and the mem_accounting module
> >>> parameter is enabled, the buffer is charged to the allocator's
> >>> own cgroup:
> >>>
> >>> struct dma_heap_allocation_data alloc = {
> >>> .len = buffer_size,
> >>> .fd_flags = O_RDWR | O_CLOEXEC,
> >>> };
> >>> ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc);
> >>> /* charged to current process's cgroup */
> >>>
> >>> Current limitations:
> >>>
> >>> - Single-owner model: a dma-buf carries one memcg charge regardless of
> >>> how many processes share it. Means only the first owner (and exporter)
> >>> of the shared buffer bears the charge.
> >>> - Only memcg accounting supported. While this makes sense for system
> >>> heap buffers, other heaps (e.g., CMA heaps) will require selectively
> >>> charging also for the dmem controller.
> >>>
> >>> Signed-off-by: Albert Esteve <aesteve@redhat.com>
> >>> ---
> >>> Documentation/admin-guide/cgroup-v2.rst | 5 ++--
> >>> drivers/dma-buf/dma-buf.c | 16 ++++---------
> >>> drivers/dma-buf/dma-heap.c | 42 ++++++++++++++++++++++++++++++---
> >>> drivers/dma-buf/heaps/system_heap.c | 2 --
> >>> include/uapi/linux/dma-heap.h | 6 +++++
> >>> 5 files changed, 53 insertions(+), 18 deletions(-)
> >>>
> >>> diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
> >>> index 8bdbc2e866430..824d269531eb1 100644
> >>> --- a/Documentation/admin-guide/cgroup-v2.rst
> >>> +++ b/Documentation/admin-guide/cgroup-v2.rst
> >>> @@ -1636,8 +1636,9 @@ The following nested keys are defined.
> >>> structures.
> >>>
> >>> dmabuf (npn)
> >>> - Amount of memory used for exported DMA buffers allocated by the cgroup.
> >>> - Stays with the allocating cgroup regardless of how the buffer is shared.
> >>> + Amount of memory used for exported DMA buffers allocated by or on
> >>> + behalf of the cgroup. Stays with the allocating cgroup regardless
> >>> + of how the buffer is shared.
> >>>
> >>> workingset_refault_anon
> >>> Number of refaults of previously evicted anonymous pages.
> >>> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
> >>> index ce02377f48908..23fb758b78297 100644
> >>> --- a/drivers/dma-buf/dma-buf.c
> >>> +++ b/drivers/dma-buf/dma-buf.c
> >>> @@ -181,8 +181,11 @@ static void dma_buf_release(struct dentry *dentry)
> >>> */
> >>> BUG_ON(dmabuf->cb_in.active || dmabuf->cb_out.active);
> >>>
> >>> - mem_cgroup_uncharge_dmabuf(dmabuf->memcg, PAGE_ALIGN(dmabuf->size) / PAGE_SIZE);
> >>> - mem_cgroup_put(dmabuf->memcg);
> >>> + if (dmabuf->memcg) {
> >>> + mem_cgroup_uncharge_dmabuf(dmabuf->memcg,
> >>> + PAGE_ALIGN(dmabuf->size) / PAGE_SIZE);
> >>> + mem_cgroup_put(dmabuf->memcg);
> >>> + }
> >>>
> >>> dmabuf->ops->release(dmabuf);
> >>>
> >>> @@ -764,13 +767,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
> >>> dmabuf->resv = resv;
> >>> }
> >>>
> >>> - dmabuf->memcg = get_mem_cgroup_from_mm(current->mm);
> >>> - if (!mem_cgroup_charge_dmabuf(dmabuf->memcg, PAGE_ALIGN(dmabuf->size) / PAGE_SIZE,
> >>> - GFP_KERNEL)) {
> >>> - ret = -ENOMEM;
> >>> - goto err_memcg;
> >>> - }
> >>> -
> >>> file->private_data = dmabuf;
> >>> file->f_path.dentry->d_fsdata = dmabuf;
> >>> dmabuf->file = file;
> >>> @@ -781,8 +777,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
> >>>
> >>> return dmabuf;
> >>>
> >>> -err_memcg:
> >>> - mem_cgroup_put(dmabuf->memcg);
> >>> err_file:
> >>> fput(file);
> >>> err_module:
> >>> diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c
> >>> index ac5f8685a6494..ff6e259afcdc0 100644
> >>> --- a/drivers/dma-buf/dma-heap.c
> >>> +++ b/drivers/dma-buf/dma-heap.c
> >>> @@ -7,13 +7,17 @@
> >>> */
> >>>
> >>> #include <linux/cdev.h>
> >>> +#include <linux/cgroup.h>
> >>> #include <linux/device.h>
> >>> #include <linux/dma-buf.h>
> >>> #include <linux/dma-heap.h>
> >>> +#include <linux/memcontrol.h>
> >>> +#include <linux/sched/mm.h>
> >>> #include <linux/err.h>
> >>> #include <linux/export.h>
> >>> #include <linux/list.h>
> >>> #include <linux/nospec.h>
> >>> +#include <linux/pidfd.h>
> >>> #include <linux/syscalls.h>
> >>> #include <linux/uaccess.h>
> >>> #include <linux/xarray.h>
> >>> @@ -55,10 +59,12 @@ MODULE_PARM_DESC(mem_accounting,
> >>> "Enable cgroup-based memory accounting for dma-buf heap allocations (default=false).");
> >>>
> >>> static int dma_heap_buffer_alloc(struct dma_heap *heap, size_t len,
> >>> - u32 fd_flags,
> >>> - u64 heap_flags)
> >>> + u32 fd_flags, u64 heap_flags,
> >>> + struct mem_cgroup *charge_to)
> >>> {
> >>> struct dma_buf *dmabuf;
> >>> + unsigned int nr_pages;
> >>> + struct mem_cgroup *memcg = charge_to;
> >>> int fd;
> >>>
> >>> /*
> >>> @@ -73,6 +79,22 @@ static int dma_heap_buffer_alloc(struct dma_heap *heap, size_t len,
> >>> if (IS_ERR(dmabuf))
> >>> return PTR_ERR(dmabuf);
> >>>
> >>> + nr_pages = len / PAGE_SIZE;
> >>> +
> >>> + if (memcg)
> >>> + css_get(&memcg->css);
> >>> + else if (mem_accounting)
> >>> + memcg = get_mem_cgroup_from_mm(current->mm);
> >>> +
> >>> + if (memcg) {
> >>> + if (!mem_cgroup_charge_dmabuf(memcg, nr_pages, GFP_KERNEL)) {
> >>> + mem_cgroup_put(memcg);
> >>> + dma_buf_put(dmabuf);
> >>> + return -ENOMEM;
> >>> + }
> >>> + dmabuf->memcg = memcg;
> >>> + }
> >>> +
> >>> fd = dma_buf_fd(dmabuf, fd_flags);
> >>> if (fd < 0) {
> >>> dma_buf_put(dmabuf);
> >>> @@ -102,6 +124,9 @@ static long dma_heap_ioctl_allocate(struct file *file, void *data)
> >>> {
> >>> struct dma_heap_allocation_data *heap_allocation = data;
> >>> struct dma_heap *heap = file->private_data;
> >>> + struct mem_cgroup *memcg = NULL;
> >>> + struct task_struct *task;
> >>> + unsigned int pidfd_flags;
> >>> int fd;
> >>>
> >>> if (heap_allocation->fd)
> >>> @@ -113,9 +138,20 @@ static long dma_heap_ioctl_allocate(struct file *file, void *data)
> >>> if (heap_allocation->heap_flags & ~DMA_HEAP_VALID_HEAP_FLAGS)
> >>> return -EINVAL;
> >>>
> >>> + if (heap_allocation->charge_pid_fd) {
> >>> + task = pidfd_get_task(heap_allocation->charge_pid_fd, &pidfd_flags);
> >>
> >> Will always get a thread-group leader pidfd and will fail if this is a
> >> thread-specific pidfd. pidfd_open(1234, PIDFD_THREAD) can be used to
> >> open a thread-specific pidfd.
> >>
> >>> + if (IS_ERR(task))
> >>> + return PTR_ERR(task);
> >>> +
> >>> + memcg = get_mem_cgroup_from_mm(task->mm);
> >>> + put_task_struct(task);
> >>> + }
> >>> +
> >>> fd = dma_heap_buffer_alloc(heap, heap_allocation->len,
> >>> heap_allocation->fd_flags,
> >>> - heap_allocation->heap_flags);
> >>> + heap_allocation->heap_flags,
> >>> + memcg);
> >>> + mem_cgroup_put(memcg);
> >>> if (fd < 0)
> >>> return fd;
> >>>
> >>> diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c
> >>> index 03c2b87cb1112..95d7688167b93 100644
> >>> --- a/drivers/dma-buf/heaps/system_heap.c
> >>> +++ b/drivers/dma-buf/heaps/system_heap.c
> >>> @@ -385,8 +385,6 @@ static struct page *alloc_largest_available(unsigned long size,
> >>> if (max_order < orders[i])
> >>> continue;
> >>> flags = order_flags[i];
> >>> - if (mem_accounting)
> >>> - flags |= __GFP_ACCOUNT;
> >>> page = alloc_pages(flags, orders[i]);
> >>> if (!page)
> >>> continue;
> >>> diff --git a/include/uapi/linux/dma-heap.h b/include/uapi/linux/dma-heap.h
> >>> index a4cf716a49fa6..e02b0f8cbc6a1 100644
> >>> --- a/include/uapi/linux/dma-heap.h
> >>> +++ b/include/uapi/linux/dma-heap.h
> >>> @@ -29,6 +29,10 @@
> >>> * handle to the allocated dma-buf
> >>> * @fd_flags: file descriptor flags used when allocating
> >>> * @heap_flags: flags passed to heap
> >>> + * @charge_pid_fd: optional pidfd of the process whose cgroup should be
> >>> + * charged for this allocation; 0 means charge the calling
> >>> + * process's cgroup
> >>> + * @__padding: reserved, must be zero
> >>> *
> >>> * Provided by userspace as an argument to the ioctl
> >>> */
> >>> @@ -37,6 +41,8 @@ struct dma_heap_allocation_data {
> >>> __u32 fd;
> >>> __u32 fd_flags;
> >>> __u64 heap_flags;
> >>> + __u32 charge_pid_fd;
> >>> + __u32 __padding;
> >>> };
> >>>
> >>> #define DMA_HEAP_IOC_MAGIC 'H'
> >>>
> >>> --
> >>> 2.53.0
> >>>
>
^ permalink raw reply
* Re: [PATCH] mm: memcontrol: propagate NMI slab stats to memcg vmstats
From: Johannes Weiner @ 2026-05-18 13:21 UTC (permalink / raw)
To: Alexandre Ghiti
Cc: Andrew Morton, Michal Hocko, Roman Gushchin, Shakeel Butt,
Muchun Song, Vlastimil Babka, stable, linux-mm, linux-kernel,
cgroups
In-Reply-To: <20260518082830.599102-1-alex@ghiti.fr>
On Mon, May 18, 2026 at 10:28:19AM +0200, Alexandre Ghiti wrote:
> flush_nmi_stats() drains per-node NMI slab atomics into the per-node
> lruvec_stats, but does not propagate them to the memcg-level vmstats.
>
> For non NMI case, account_slab_nmi_safe() calls mod_memcg_lruvec_state()
> which updates both per-node lruvec_stats and memcg-level vmstats, so
> flush_nmi_stats() needs to flush to per-node lruvec_stats as well as
> memcg-level vmstats.
>
> So fix this by flushing to the memcg-level vmstats for NMI too.
>
> Fixes: 940b01fc8dc1 ("memcg: nmi safe memcg stats for specific archs")
> Cc: stable@vger.kernel.org
> Acked-by: Shakeel Butt <shakeel.butt@linux.dev>
> Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
^ permalink raw reply
* Re: [PATCH v2 00/10] sched: Flatten the pick
From: Vincent Guittot @ 2026-05-18 13:34 UTC (permalink / raw)
To: Peter Zijlstra
Cc: mingo, longman, chenridong, juri.lelli, dietmar.eggemann, rostedt,
bsegall, mgorman, vschneid, tj, hannes, mkoutny, cgroups,
linux-kernel, jstultz, kprateek.nayak, qyousef
In-Reply-To: <20260513113510.GK1889694@noisy.programming.kicks-ass.net>
On Wed, 13 May 2026 at 13:35, Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Tue, May 12, 2026 at 10:42:33AM +0200, Vincent Guittot wrote:
>
> > I haven't reviewed the patches yet but I ran some tests with it while
> > testing sched latency related changes for short slice wakeup
> > preemption. I have some large hackbench regressions with this series
> > on HMP system with and without EAS. those figures are unexpected
> > because the benchs run on root cfs
> >
> > One example with hackbench 8 groups thread pipe
> > tip/sched/core tip/sched/core +this patchset +this patchset
> > slice 2.8ms 16ms 2.8ms 16ms
> > dragonboard rb5 with EAS
> > 0,748(+/-4,6%) 0,621(+/-3.6%) +17% 1,915(+/-7.9%) -156%
> > 0,689(+/- 9.1%) +8%
> >
> > radxa orion6 HMP without EAS
> > 0,588(+/-5.8%) 0,677(+/-5.9%) -15% 1,505(+/-10%) -156%
> > 1,071(+/-5.9%) -82%
> >
> > Increasing the slice partly removes regressions but tis is surprising
> > because the bench runs at root cfs and I thought that results will not
> > change in such a case
>
> D'oh :/
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index e54da4c6c945..77d0e1937f2c 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -9071,7 +9071,7 @@ static void wakeup_preempt_fair(struct rq *rq, struct task_struct *p, int wake_f
> enum preempt_wakeup_action preempt_action = PREEMPT_WAKEUP_PICK;
> struct task_struct *donor = rq->donor;
> struct sched_entity *nse, *se = &donor->se, *pse = &p->se;
> - struct cfs_rq *cfs_rq = task_cfs_rq(donor);
> + struct cfs_rq *cfs_rq = &rq->cfs;
I tested this patch on top of the series but it doesn't fix the perf
regression on rb5
hackbench 8 groups thread pipe is still at 1.907(+/-7.6%) with default
slice duration
> int cse_is_idle, pse_is_idle;
>
> /*
^ permalink raw reply
* Re: [PATCH RFC 2/5] dma-heap: charge dma-buf memory via explicit memcg
From: Christian König @ 2026-05-18 14:06 UTC (permalink / raw)
To: Albert Esteve
Cc: T.J. Mercier, Christian Brauner, Tejun Heo, Johannes Weiner,
Michal Koutný, Jonathan Corbet, Shuah Khan, Sumit Semwal,
Michal Hocko, Roman Gushchin, Shakeel Butt, Muchun Song,
Andrew Morton, Benjamin Gaignard, Brian Starkey, John Stultz,
Paul Moore, James Morris, Serge E. Hallyn, Stephen Smalley,
Ondrej Mosnacek, Shuah Khan, cgroups, linux-doc, linux-kernel,
linux-media, dri-devel, linaro-mm-sig, linux-mm,
linux-security-module, selinux, linux-kselftest, mripard,
echanude
In-Reply-To: <CADSE00Lh95ygoXGKJGsYvQGEsFV8sVmwEC3uvh8M6r3ERzaJwg@mail.gmail.com>
On 5/18/26 14:50, Albert Esteve wrote:
> On Mon, May 18, 2026 at 9:20 AM Christian König
> <christian.koenig@amd.com> wrote:
>>
>> On 5/15/26 19:06, T.J. Mercier wrote:
>>> On Fri, May 15, 2026 at 6:53 AM Christian Brauner <brauner@kernel.org> wrote:
>>>>
>>>> On Tue, May 12, 2026 at 11:10:44AM +0200, Albert Esteve wrote:
>>>>> On embedded platforms a central process often allocates dma-buf
>>>>> memory on behalf of client applications. Without a way to
>>>>> attribute the charge to the requesting client's cgroup, the
>>>>> cost lands on the allocator, making per-cgroup memory limits
>>>>> ineffective for the actual consumers.
>>>>>
>>>>> Add charge_pid_fd to struct dma_heap_allocation_data. When set to
>>>>
>>>> Please be aware that pidfds come in two flavors:
>>>>
>>>> thread-group pidfds and thread-specific pidfds. Make sure that your API
>>>> doesn't implicitly depend on this distinction not existing.
>>>
>>> Hi Christian,
>>>
>>> Memcg is not a controller that supports "thread mode" so all threads
>>> in a group should belong to the same memcg.
>>
>> BTW: Exactly that is the requirement automotive has with their native context use case.
>>
>> The use case is that you have a deamon which has multiple threads were each one is acting on behalve of some other process.
>>
>> At the moment we basically say they are simply not using cgroups for that use case, but it would be really nice if we could handle that as well.
>>
>> Summarizing the requirement of that use case: You need a different cgroup for each thread of a process.
>
> Hi Christian,
>
> Thanks for sharing this atuomotive usecase. If I understand correctly,
> the actual requirement is attributing dma-buf charges to the right
> client, not putting each daemon thread in a different cgroup?
Nope, exactly that's the difference.
The thread acts as a filtering agent for both memory allocation and command submission for somebody else, the process on which behalve the daemon does things can even be in a client VM, completely remote over some network or even something like a microcontroller.
Everything the thread does regarding CPU time, GPU driver memory allocation as well as resources like GPU processing and I/O time etc.. needs to be accounted to one client which can be different for each thread of the process.
The only thing which is shared with the main process thread is CPU memory resources, e.g. malloc() because that is basically just needed for housekeeping and pretty much irrelevant for this kind of use case.
The problem is now you can't do that with cgroups at the moment but unfortunately only the kernel has the information you need to know to do this.
So what you end up with is to define tons of interfaces just to get the necessary information from the kernel into userspace and then essentially duplicate the same infrastructure cgroup provides in the kernel in userspace again.
> If so,
> the `charge_pid_fd` approach achieves this directly by passing the
> client's `pid_fd`, without needing to add per-thread cgroup
> infrastructure.
Well it's already a massive improvemt, we could basically stop doing the whole duplication part for the GPU driver stack and just use cgroups for this part.
Doing that automatically for CPU and I/O time would just be nice to have additionally.
Regards,
Christian.
>
>>
>> Regards,
>> Christian.
>>
>>>
>>> Checking the flags from pidfd_get_pid would be the best way for an
>>> explicit check of the pidfd type?
>>>
>>>>> a valid pidfd, DMA_HEAP_IOCTL_ALLOC resolves the target task's
>>>>> memcg and charges the buffer there via mem_cgroup_charge_dmabuf()
>>>>> inside dma_heap_buffer_alloc(). Without charge_pid_fd, and with
>>>>> the mem_accounting module parameter enabled, the buffer is charged
>>>>> to the allocator's own cgroup.
>>>>>
>>>>> Additionally, commit 3c227be90659 ("dma-buf: system_heap: account for
>>>>> system heap allocation in memcg") adds __GFP_ACCOUNT to system-heap
>>>>> page allocations. Keeping __GFP_ACCOUNT would charge the same pages
>>>>> twice (once to kmem, once to MEMCG_DMABUF), thus remove it and route
>>>>> all accounting through a single MEMCG_DMABUF path.
>>>>>
>>>>> Usage examples:
>>>>>
>>>>> 1. Central allocator charging to a client at allocation time.
>>>>> The allocator knows the client's PID (e.g., from binder's
>>>>> sender_pid) and uses pidfd to attribute the charge:
>>>>>
>>>>> pid_t client_pid = txn->sender_pid;
>>>>> int pidfd = pidfd_open(client_pid, 0);
>>>>>
>>>>> struct dma_heap_allocation_data alloc = {
>>>>> .len = buffer_size,
>>>>> .fd_flags = O_RDWR | O_CLOEXEC,
>>>>> .charge_pid_fd = pidfd,
>>>>> };
>>>>> ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc);
>>>>> close(pidfd);
>>>>> /* alloc.fd is now charged to client's cgroup */
>>>>>
>>>>> 2. Default allocation (no pidfd, mem_accounting=1).
>>>>> When charge_pid_fd is not set and the mem_accounting module
>>>>> parameter is enabled, the buffer is charged to the allocator's
>>>>> own cgroup:
>>>>>
>>>>> struct dma_heap_allocation_data alloc = {
>>>>> .len = buffer_size,
>>>>> .fd_flags = O_RDWR | O_CLOEXEC,
>>>>> };
>>>>> ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc);
>>>>> /* charged to current process's cgroup */
>>>>>
>>>>> Current limitations:
>>>>>
>>>>> - Single-owner model: a dma-buf carries one memcg charge regardless of
>>>>> how many processes share it. Means only the first owner (and exporter)
>>>>> of the shared buffer bears the charge.
>>>>> - Only memcg accounting supported. While this makes sense for system
>>>>> heap buffers, other heaps (e.g., CMA heaps) will require selectively
>>>>> charging also for the dmem controller.
>>>>>
>>>>> Signed-off-by: Albert Esteve <aesteve@redhat.com>
>>>>> ---
>>>>> Documentation/admin-guide/cgroup-v2.rst | 5 ++--
>>>>> drivers/dma-buf/dma-buf.c | 16 ++++---------
>>>>> drivers/dma-buf/dma-heap.c | 42 ++++++++++++++++++++++++++++++---
>>>>> drivers/dma-buf/heaps/system_heap.c | 2 --
>>>>> include/uapi/linux/dma-heap.h | 6 +++++
>>>>> 5 files changed, 53 insertions(+), 18 deletions(-)
>>>>>
>>>>> diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
>>>>> index 8bdbc2e866430..824d269531eb1 100644
>>>>> --- a/Documentation/admin-guide/cgroup-v2.rst
>>>>> +++ b/Documentation/admin-guide/cgroup-v2.rst
>>>>> @@ -1636,8 +1636,9 @@ The following nested keys are defined.
>>>>> structures.
>>>>>
>>>>> dmabuf (npn)
>>>>> - Amount of memory used for exported DMA buffers allocated by the cgroup.
>>>>> - Stays with the allocating cgroup regardless of how the buffer is shared.
>>>>> + Amount of memory used for exported DMA buffers allocated by or on
>>>>> + behalf of the cgroup. Stays with the allocating cgroup regardless
>>>>> + of how the buffer is shared.
>>>>>
>>>>> workingset_refault_anon
>>>>> Number of refaults of previously evicted anonymous pages.
>>>>> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
>>>>> index ce02377f48908..23fb758b78297 100644
>>>>> --- a/drivers/dma-buf/dma-buf.c
>>>>> +++ b/drivers/dma-buf/dma-buf.c
>>>>> @@ -181,8 +181,11 @@ static void dma_buf_release(struct dentry *dentry)
>>>>> */
>>>>> BUG_ON(dmabuf->cb_in.active || dmabuf->cb_out.active);
>>>>>
>>>>> - mem_cgroup_uncharge_dmabuf(dmabuf->memcg, PAGE_ALIGN(dmabuf->size) / PAGE_SIZE);
>>>>> - mem_cgroup_put(dmabuf->memcg);
>>>>> + if (dmabuf->memcg) {
>>>>> + mem_cgroup_uncharge_dmabuf(dmabuf->memcg,
>>>>> + PAGE_ALIGN(dmabuf->size) / PAGE_SIZE);
>>>>> + mem_cgroup_put(dmabuf->memcg);
>>>>> + }
>>>>>
>>>>> dmabuf->ops->release(dmabuf);
>>>>>
>>>>> @@ -764,13 +767,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
>>>>> dmabuf->resv = resv;
>>>>> }
>>>>>
>>>>> - dmabuf->memcg = get_mem_cgroup_from_mm(current->mm);
>>>>> - if (!mem_cgroup_charge_dmabuf(dmabuf->memcg, PAGE_ALIGN(dmabuf->size) / PAGE_SIZE,
>>>>> - GFP_KERNEL)) {
>>>>> - ret = -ENOMEM;
>>>>> - goto err_memcg;
>>>>> - }
>>>>> -
>>>>> file->private_data = dmabuf;
>>>>> file->f_path.dentry->d_fsdata = dmabuf;
>>>>> dmabuf->file = file;
>>>>> @@ -781,8 +777,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
>>>>>
>>>>> return dmabuf;
>>>>>
>>>>> -err_memcg:
>>>>> - mem_cgroup_put(dmabuf->memcg);
>>>>> err_file:
>>>>> fput(file);
>>>>> err_module:
>>>>> diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c
>>>>> index ac5f8685a6494..ff6e259afcdc0 100644
>>>>> --- a/drivers/dma-buf/dma-heap.c
>>>>> +++ b/drivers/dma-buf/dma-heap.c
>>>>> @@ -7,13 +7,17 @@
>>>>> */
>>>>>
>>>>> #include <linux/cdev.h>
>>>>> +#include <linux/cgroup.h>
>>>>> #include <linux/device.h>
>>>>> #include <linux/dma-buf.h>
>>>>> #include <linux/dma-heap.h>
>>>>> +#include <linux/memcontrol.h>
>>>>> +#include <linux/sched/mm.h>
>>>>> #include <linux/err.h>
>>>>> #include <linux/export.h>
>>>>> #include <linux/list.h>
>>>>> #include <linux/nospec.h>
>>>>> +#include <linux/pidfd.h>
>>>>> #include <linux/syscalls.h>
>>>>> #include <linux/uaccess.h>
>>>>> #include <linux/xarray.h>
>>>>> @@ -55,10 +59,12 @@ MODULE_PARM_DESC(mem_accounting,
>>>>> "Enable cgroup-based memory accounting for dma-buf heap allocations (default=false).");
>>>>>
>>>>> static int dma_heap_buffer_alloc(struct dma_heap *heap, size_t len,
>>>>> - u32 fd_flags,
>>>>> - u64 heap_flags)
>>>>> + u32 fd_flags, u64 heap_flags,
>>>>> + struct mem_cgroup *charge_to)
>>>>> {
>>>>> struct dma_buf *dmabuf;
>>>>> + unsigned int nr_pages;
>>>>> + struct mem_cgroup *memcg = charge_to;
>>>>> int fd;
>>>>>
>>>>> /*
>>>>> @@ -73,6 +79,22 @@ static int dma_heap_buffer_alloc(struct dma_heap *heap, size_t len,
>>>>> if (IS_ERR(dmabuf))
>>>>> return PTR_ERR(dmabuf);
>>>>>
>>>>> + nr_pages = len / PAGE_SIZE;
>>>>> +
>>>>> + if (memcg)
>>>>> + css_get(&memcg->css);
>>>>> + else if (mem_accounting)
>>>>> + memcg = get_mem_cgroup_from_mm(current->mm);
>>>>> +
>>>>> + if (memcg) {
>>>>> + if (!mem_cgroup_charge_dmabuf(memcg, nr_pages, GFP_KERNEL)) {
>>>>> + mem_cgroup_put(memcg);
>>>>> + dma_buf_put(dmabuf);
>>>>> + return -ENOMEM;
>>>>> + }
>>>>> + dmabuf->memcg = memcg;
>>>>> + }
>>>>> +
>>>>> fd = dma_buf_fd(dmabuf, fd_flags);
>>>>> if (fd < 0) {
>>>>> dma_buf_put(dmabuf);
>>>>> @@ -102,6 +124,9 @@ static long dma_heap_ioctl_allocate(struct file *file, void *data)
>>>>> {
>>>>> struct dma_heap_allocation_data *heap_allocation = data;
>>>>> struct dma_heap *heap = file->private_data;
>>>>> + struct mem_cgroup *memcg = NULL;
>>>>> + struct task_struct *task;
>>>>> + unsigned int pidfd_flags;
>>>>> int fd;
>>>>>
>>>>> if (heap_allocation->fd)
>>>>> @@ -113,9 +138,20 @@ static long dma_heap_ioctl_allocate(struct file *file, void *data)
>>>>> if (heap_allocation->heap_flags & ~DMA_HEAP_VALID_HEAP_FLAGS)
>>>>> return -EINVAL;
>>>>>
>>>>> + if (heap_allocation->charge_pid_fd) {
>>>>> + task = pidfd_get_task(heap_allocation->charge_pid_fd, &pidfd_flags);
>>>>
>>>> Will always get a thread-group leader pidfd and will fail if this is a
>>>> thread-specific pidfd. pidfd_open(1234, PIDFD_THREAD) can be used to
>>>> open a thread-specific pidfd.
>>>>
>>>>> + if (IS_ERR(task))
>>>>> + return PTR_ERR(task);
>>>>> +
>>>>> + memcg = get_mem_cgroup_from_mm(task->mm);
>>>>> + put_task_struct(task);
>>>>> + }
>>>>> +
>>>>> fd = dma_heap_buffer_alloc(heap, heap_allocation->len,
>>>>> heap_allocation->fd_flags,
>>>>> - heap_allocation->heap_flags);
>>>>> + heap_allocation->heap_flags,
>>>>> + memcg);
>>>>> + mem_cgroup_put(memcg);
>>>>> if (fd < 0)
>>>>> return fd;
>>>>>
>>>>> diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c
>>>>> index 03c2b87cb1112..95d7688167b93 100644
>>>>> --- a/drivers/dma-buf/heaps/system_heap.c
>>>>> +++ b/drivers/dma-buf/heaps/system_heap.c
>>>>> @@ -385,8 +385,6 @@ static struct page *alloc_largest_available(unsigned long size,
>>>>> if (max_order < orders[i])
>>>>> continue;
>>>>> flags = order_flags[i];
>>>>> - if (mem_accounting)
>>>>> - flags |= __GFP_ACCOUNT;
>>>>> page = alloc_pages(flags, orders[i]);
>>>>> if (!page)
>>>>> continue;
>>>>> diff --git a/include/uapi/linux/dma-heap.h b/include/uapi/linux/dma-heap.h
>>>>> index a4cf716a49fa6..e02b0f8cbc6a1 100644
>>>>> --- a/include/uapi/linux/dma-heap.h
>>>>> +++ b/include/uapi/linux/dma-heap.h
>>>>> @@ -29,6 +29,10 @@
>>>>> * handle to the allocated dma-buf
>>>>> * @fd_flags: file descriptor flags used when allocating
>>>>> * @heap_flags: flags passed to heap
>>>>> + * @charge_pid_fd: optional pidfd of the process whose cgroup should be
>>>>> + * charged for this allocation; 0 means charge the calling
>>>>> + * process's cgroup
>>>>> + * @__padding: reserved, must be zero
>>>>> *
>>>>> * Provided by userspace as an argument to the ioctl
>>>>> */
>>>>> @@ -37,6 +41,8 @@ struct dma_heap_allocation_data {
>>>>> __u32 fd;
>>>>> __u32 fd_flags;
>>>>> __u64 heap_flags;
>>>>> + __u32 charge_pid_fd;
>>>>> + __u32 __padding;
>>>>> };
>>>>>
>>>>> #define DMA_HEAP_IOC_MAGIC 'H'
>>>>>
>>>>> --
>>>>> 2.53.0
>>>>>
>>
>
^ permalink raw reply
* Re: [PATCH 0/8] per-memcg-per-node kmem accounting
From: Joshua Hahn @ 2026-05-18 14:57 UTC (permalink / raw)
To: Alexandre Ghiti
Cc: Andrew Morton, Michal Hocko, Roman Gushchin, Shakeel Butt,
Muchun Song, Dennis Zhou, Tejun Heo, Christoph Lameter,
Vlastimil Babka, Yosry Ahmed, Nhat Pham, Sergey Senozhatsky,
Chengming Zhou, Suren Baghdasaryan, Qi Zheng, David Hildenbrand,
Lorenzo Stoakes, Minchan Kim, Mike Rapoport, Axel Rasmussen,
Barry Song, Kairui Song, Wei Xu, Yuanchu Xie, Liam R . Howlett,
linux-mm, linux-kernel, cgroups
In-Reply-To: <20260511202136.330358-1-alex@ghiti.fr>
On Mon, 11 May 2026 22:20:35 +0200 Alexandre Ghiti <alex@ghiti.fr> wrote:
> This series pursues the work initiated by Joshua [1]. We need kernel
> memory to be accounted on a per-node basis in order to be able to
> know the memcg and physical memory association.
>
> This series takes advantage of the recent introduction of per-node
> obj_cgroup [2] and makes those obj_cgroup tied to their numa node.
>
> The bulk of the series is percpu per-node accounting: percpu
> "precharges" the memcg before we know the actual location of the pages
> it uses, so charging and accounting had to be split. All other kmem
> users (slab, zswap, __memcg_kmem_charge_page) are straightforward
> conversions (zswap support is limited in this series because Joshua
> is working on it in parallel [3]).
>
> Thanks Joshua for your early feedbacks!
Hello Alex,
Thank you for your work!
Overall I think the direction makes sense to me. Pre-overcharging makes sense to
me as an approach, we would much rather overaccount than underaccount and
later have to breach limits.
I do have some concerns on performance, though. Namely, I think there are
some expensive operations that I think would benefit from some performane
benchmarking with this patch added (maybe some simple microbenchmarks that
demonstrates kernel allocation overhead could be useful).
From what I can tell, there is some additional performance overhead that has
to do with iterating over num_possible_cpus() x pages_per_alloc, which
doesn't seem trivial to me.
Another concern that I see is the stock credit system. Maybe we could be
bypassing the stock check leading to more time spent doing the atomic
operations.
obj_stock caches a single obj_cgroup, which means that if we split the objcg
to be per-node (in patch 6), then the obj_stock basically gets invalidated
every operation since we iterate over more objcgs (even though we are in
the same logical objcg). Maybe I'm missing something?
I haven't taken a deep look at the implementation details but just wanted to
raise some high level items that I noticed. Of course, all of these concerns
are just theoretical, if you can show that the performance delta is not
noticable then all of my concerns don't matter.
I also want to talk more about the local credit system but let's first see
what the numbers are first.
Thanks again, Alex. And I really like patch 2 because it is a solution to
a problem that I ran into in my percpu tracking series that I couldn't think
of before! Thank you for solving my problem too : -)
Have a great day!
Joshua
> [1] https://lore.kernel.org/linux-mm/20260404033844.1892595-1-joshua.hahnjy@gmail.com/
> [2] https://lore.kernel.org/linux-mm/56c04b1c5d54f75ccdc12896df6c1ca35403ecc3.1772711148.git.zhengqi.arch@bytedance.com/
> [3] https://lore.kernel.org/linux-mm/20260311195153.4013476-1-joshua.hahnjy@gmail.com/
>
> Alexandre Ghiti (8):
> mm: memcontrol: propagate NMI slab stats to memcg vmstats
> mm: percpu: charge obj_exts allocation with __GFP_ACCOUNT
> mm: percpu: Split memcg charging and kmem accounting
> mm: memcontrol: track MEMCG_KMEM per NUMA node
> mm: memcontrol: per-node kmem accounting for page charges
> mm: slab: per-node kmem accounting for slab
> mm: percpu: per-node kmem accounting using local credit
> mm: zswap: per-node kmem accounting for zswap/zsmalloc
>
> include/linux/memcontrol.h | 27 +++++--
> include/linux/mmzone.h | 1 +
> include/linux/zsmalloc.h | 2 +
> mm/memcontrol.c | 150 ++++++++++++++++++++++++++++---------
> mm/percpu-internal.h | 16 +---
> mm/percpu.c | 90 ++++++++++++++++++++--
> mm/vmstat.c | 1 +
> mm/zsmalloc.c | 11 +++
> mm/zswap.c | 9 ++-
> 9 files changed, 242 insertions(+), 65 deletions(-)
>
> --
> 2.54.0
>
>
^ permalink raw reply
* Re: [RFC PATCH v5 20/29] sched/deadline: Allow deeper hierarchies of RT cgroups
From: Yuri Andriaccio @ 2026-05-18 15:27 UTC (permalink / raw)
To: Tejun Heo
Cc: luca abeni, Peter Zijlstra, Yuri Andriaccio, Ingo Molnar,
Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt,
Ben Segall, Mel Gorman, Valentin Schneider, linux-kernel, hannes,
mkoutny, cgroups
In-Reply-To: <8672eb9e7bbd6abde7762feb267799c5@kernel.org>
Hi Tejun,
me and Luca were discussing further the interface for HCBS and agreed
that from the whole discussion we seem to be converging to something
stable. To clarify things, I'm writing a recap here and I'll also add
some visual examples to make sure every "corner case" we might encounter
can be managed.
Interface:
- File: cpu.rt.max
- Format: <runtime>|"max" <period>
- Default value:
"max" <parent period> - if the parent schedules on the root runqueue.
0 <parent period> - if the parent is instead using HCBS.
- Meaning (incomplete/dubious):
The bandwidth allocated to the specific cgroup and all of its children.
Since sum(children bw) <= own bw, a cgroup's servers will be configured
with (own bw - sum(children bw)) bandwidth.
A cgroup set to "max" whose parents are all set to "max" (root
cgroup excluded)
will run their tasks in the root runqueue.
A cgroup set to "max" whose parent has a non-zero reservation will
inherit the parent's configuration.
The root cgroup's cpu.rt.max file reserves the maximum HCBS
bandwidth for
the whole hierarchy. Root set to "max" disable HCBS (as if set with
a zero runtime).
Corner Cases with Examples:
Root (70 100)
|
G2(50 100)
/ | \
G3("max" 100) G4(20 100) ---G6---
|
---G5---
In this example, only the root cgroup may run tasks in the root
runqueue, groups G2, G3 and G4 use HCBS instead. G5 and G6 are freshly
allocated.
- Is this a valid configuration?
- What bandwidth does G3 take?
- Do tasks in G3 share the same runqueue of G2?
- Are the defaults mentioned above suitable?
Otherwise, what default settings do G5 and G6 take?
Root (70 100)
/ \
G0 ("max" 0) G2(50 100)
/ / \
G1 ("max" 0) G3(10 100) G4(20 100)
Groups Root, G0 and G1 may run tasks in the root runqueue, groups G2, G3
and G4 use HCBS instead.
- Is this a valid configuration?
- Is it possible to set G0 to (20 100)?
- Is it possible to set G2 to ("max" 0)?
Root (70 100)
|
G0 ("max" 0)
/ \
G1 ("max" 0) G2(50 100)
/ \
G3(10 100) G4(20 100)
Groups Root, G0 and G1 may run tasks in the root runqueue, groups G2, G3
and G4 use HCBS instead.
- Is this a valid configuration?
- Is it possible to set G1 to (20 100)?
- What happens when we set G2 to "max"?
- Shall this be disallowed, as G2's children have a reservation?
- Are they silently reverted to "max"?
- Or G3 and G4 are kept with their reservations?
- What happens when we set G0 to (70 100)?
- If allowed, what if we set G0 to (40 100)?
I think here we have most of the corner cases that are still of doubt
for us for the final implementation.
Thanks in advance,
Yuri
^ permalink raw reply
* Re: [linus:master] [mm] 01b9da291c: stress-ng.switch.ops_per_sec 67.7% regression
From: Shakeel Butt @ 2026-05-18 16:39 UTC (permalink / raw)
To: Oliver Sang
Cc: Qi Zheng, oe-lkp, lkp, linux-kernel, Andrew Morton, David Carlier,
Allen Pais, Axel Rasmussen, Baoquan He, Chengming Zhou,
Chen Ridong, David Hildenbrand, Hamza Mahfooz, Harry Yoo,
Hugh Dickins, Imran Khan, Johannes Weiner, Kamalesh Babulal,
Lance Yang, Liam Howlett, Lorenzo Stoakes, Michal Hocko,
Michal Koutný, Mike Rapoport, Muchun Song, Muchun Song,
Nhat Pham, Roman Gushchin, Suren Baghdasaryan, Usama Arif,
Vlastimil Babka, Wei Xu, Yosry Ahmed, Yuanchu Xie, Zi Yan,
Usama Arif, cgroups, linux-mm
In-Reply-To: <agoYp1zW9afZ6uQz@linux.dev>
On Sun, May 17, 2026 at 12:38:48PM -0700, Shakeel Butt wrote:
> On Sun, May 17, 2026 at 08:55:50PM +0800, Oliver Sang wrote:
> > hi, Shakeel, hi, Qi,
> >
> > #2: when we test above patch, we found the server easy to crash while running
> > tests. we try to run up to 20 times, only 2 of them run successfully (above
> > 37739220 is just the average data from these 2 runs, since the data is stable,
> > we think maybe it's ok to report to you with this data).
> > we also noticed for [1] there is a [syzbot ci] report in [2]. since we don't
> > have serial output for our test server in this report which is for performance
> > tests, we cannot say if other 18 runs failed due to similar reason. just FYI.
> >
>
> The syzbot report is simply a rcu warning which will be fixed in v2. Do you
> have more details on the crash you are seeing? Is it page counter underflow
> warning?
>
> Thanks again for the help.
Hi Oliver, it seems like sashiko found another issue with v2, so, if you have
not yet started the test, you can skip it.
Also I am rethinking the approach, so I will send a prototype in response on
this email for which I will need your help in testing.
^ permalink raw reply
* Re: [PATCH v2] memcg: cache obj_stock by memcg, not by objcg pointer
From: Shakeel Butt @ 2026-05-18 16:46 UTC (permalink / raw)
To: Andrew Morton
Cc: Johannes Weiner, Michal Hocko, Roman Gushchin, Muchun Song,
Qi Zheng, Meta kernel team, linux-mm, cgroups, linux-kernel,
kernel test robot, alex, joshua.hahnjy
In-Reply-To: <20260517194308.952655-1-shakeel.butt@linux.dev>
Cc Alex, Joshua (since they are working on making per-num kmem accounting work)
On Sun, May 17, 2026 at 12:43:08PM -0700, Shakeel Butt wrote:
> Commit 01b9da291c49 ("mm: memcontrol: convert objcg to be per-memcg
> per-node type") split a memcg's single obj_cgroup into one per NUMA
> node, but the per-CPU obj_stock_pcp still keys cached_objcg by
> pointer. Cross-NUMA workloads now see a drain on every refill and a
> miss on every consume that targets a sibling per-node objcg of the
> same memcg, producing the 67.7% stress-ng switch-mq regression
> reported by LKP.
>
> stock->nr_bytes are fungible across per-node objcgs of one memcg.
> Treat the cache as keyed by memcg in __consume_obj_stock() and
> __refill_obj_stock() so siblings share the reserve. Compare via
> READ_ONCE(objcg->memcg) directly: pointer-compare only, no deref, so
> the rcu_read_lock contract on obj_cgroup_memcg() does not apply.
>
> In the same-memcg refill path also fold the incoming objcg's
> nr_charged_bytes into the stock; otherwise sub-page residue
> accumulates on whichever sibling was cached at drain time and
> obj_cgroup_release() silently drops it, leaking up to nr_node_ids *
> (PAGE_SIZE - 1) bytes per memcg lifecycle from the page_counter.
> This issue was reported by Sashiko.
>
> Update the now-stale invariant comment on __account_obj_stock().
>
> Qi Zheng built a specialized reproducer [1] for the corner case and
> confirmed the fix.
>
> Reported-by: kernel test robot <oliver.sang@intel.com>
> Closes: https://lore.kernel.org/oe-lkp/202605121641.b6a60cb0-lkp@intel.com
> Fixes: 01b9da291c49 ("mm: memcontrol: convert objcg to be per-memcg per-node type")
> Link: https://lore.kernel.org/19693be6-7132-446e-b3fc-b7e9f56e5949@linux.dev/ [1]
> Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
> Debugged-by: Qi Zheng <qi.zheng@linux.dev>
> Tested-by: Qi Zheng <qi.zheng@linux.dev>
Sashiko [1] reported two issues. First one seems benign but the second one is
real. However I think we need to take a step back and rethink on how to solve
this issue in more future proof way.
It seems like Alex and Joshua are working on enabling per-node kmem accounting
and that would need accurate per-numa association for each per-node objcg.
So, checking objcg->memcg in consume and refill, would go against the per-node
kmem accounting.
One way to fix the regression and be future proof is to follow the approach we
have for memcg_stock_pcp which is multiple per-cpu objcg stocks. We will need to
test it more and depending on the additional code complexity, we will need to
decide to backport it to 7.2 or not.
[1] https://sashiko.dev/#/patchset/20260517194308.952655-1-shakeel.butt@linux.dev?part=1
^ permalink raw reply
* Re: [linus:master] [mm] 01b9da291c: stress-ng.switch.ops_per_sec 67.7% regression
From: Shakeel Butt @ 2026-05-18 17:54 UTC (permalink / raw)
To: Oliver Sang
Cc: Qi Zheng, oe-lkp, lkp, linux-kernel, Andrew Morton, David Carlier,
Allen Pais, Axel Rasmussen, Baoquan He, Chengming Zhou,
Chen Ridong, David Hildenbrand, Hamza Mahfooz, Harry Yoo,
Hugh Dickins, Imran Khan, Johannes Weiner, Kamalesh Babulal,
Lance Yang, Liam Howlett, Lorenzo Stoakes, Michal Hocko,
Michal Koutný, Mike Rapoport, Muchun Song, Muchun Song,
Nhat Pham, Roman Gushchin, Suren Baghdasaryan, Usama Arif,
Vlastimil Babka, Wei Xu, Yosry Ahmed, Yuanchu Xie, Zi Yan,
Usama Arif, cgroups, linux-mm
In-Reply-To: <agtATZG9mIlYzMUl@linux.dev>
On Mon, May 18, 2026 at 09:39:00AM -0700, Shakeel Butt wrote:
> On Sun, May 17, 2026 at 12:38:48PM -0700, Shakeel Butt wrote:
> > On Sun, May 17, 2026 at 08:55:50PM +0800, Oliver Sang wrote:
> > > hi, Shakeel, hi, Qi,
> > >
> > > #2: when we test above patch, we found the server easy to crash while running
> > > tests. we try to run up to 20 times, only 2 of them run successfully (above
> > > 37739220 is just the average data from these 2 runs, since the data is stable,
> > > we think maybe it's ok to report to you with this data).
> > > we also noticed for [1] there is a [syzbot ci] report in [2]. since we don't
> > > have serial output for our test server in this report which is for performance
> > > tests, we cannot say if other 18 runs failed due to similar reason. just FYI.
> > >
> >
> > The syzbot report is simply a rcu warning which will be fixed in v2. Do you
> > have more details on the crash you are seeing? Is it page counter underflow
> > warning?
> >
> > Thanks again for the help.
>
> Hi Oliver, it seems like sashiko found another issue with v2, so, if you have
> not yet started the test, you can skip it.
>
> Also I am rethinking the approach, so I will send a prototype in response on
> this email for which I will need your help in testing.
Hi Oliver, can you please test the following patch?
From: Shakeel Butt <shakeel.butt@linux.dev>
Subject: [PATCH] memcg: shrink obj_stock_pcp and cache multiple objcgs
Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
---
mm/memcontrol.c | 213 +++++++++++++++++++++++++++++++++++-------------
1 file changed, 156 insertions(+), 57 deletions(-)
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index d978e18b9b2d..2a9e5136a956 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -150,14 +150,14 @@ static void obj_cgroup_release(struct percpu_ref *ref)
* However, it can be PAGE_SIZE or (x * PAGE_SIZE).
*
* The following sequence can lead to it:
- * 1) CPU0: objcg == stock->cached_objcg
+ * 1) CPU0: objcg cached in one of stock->cached[i]
* 2) CPU1: we do a small allocation (e.g. 92 bytes),
* PAGE_SIZE bytes are charged
* 3) CPU1: a process from another memcg is allocating something,
* the stock if flushed,
* objcg->nr_charged_bytes = PAGE_SIZE - 92
* 5) CPU0: we do release this object,
- * 92 bytes are added to stock->nr_bytes
+ * 92 bytes are added to stock->nr_bytes[i]
* 6) CPU0: stock is flushed,
* 92 bytes are added to objcg->nr_charged_bytes
*
@@ -2017,13 +2017,25 @@ static DEFINE_PER_CPU_ALIGNED(struct memcg_stock_pcp, memcg_stock) = {
.lock = INIT_LOCAL_TRYLOCK(lock),
};
+/*
+ * NR_OBJ_STOCK is sized so the entire hot path of obj_stock_pcp
+ * (lock, accounting metadata, nr_bytes[] and cached[]) fits within a
+ * single 64-byte cache line on non-debug 64-bit builds. With 5 slots:
+ * lock(1) + index(1) + node_id(2) + slab stats(4) + nr_bytes(10)
+ * + pad(6) + cached(40) == 64 bytes.
+ * A CPU can thus consume/refill/account against five different objcgs
+ * (typically per-node variants of the same memcg) while incurring at
+ * most one cache miss on the stock.
+ */
+#define NR_OBJ_STOCK 5
struct obj_stock_pcp {
local_trylock_t lock;
- unsigned int nr_bytes;
- struct obj_cgroup *cached_objcg;
- struct pglist_data *cached_pgdat;
- int nr_slab_reclaimable_b;
- int nr_slab_unreclaimable_b;
+ int8_t index;
+ int16_t node_id;
+ int16_t nr_slab_reclaimable_b;
+ int16_t nr_slab_unreclaimable_b;
+ uint16_t nr_bytes[NR_OBJ_STOCK];
+ struct obj_cgroup *cached[NR_OBJ_STOCK];
struct work_struct work;
unsigned long flags;
@@ -2031,10 +2043,13 @@ struct obj_stock_pcp {
static DEFINE_PER_CPU_ALIGNED(struct obj_stock_pcp, obj_stock) = {
.lock = INIT_LOCAL_TRYLOCK(lock),
+ .index = -1,
+ .node_id = NUMA_NO_NODE,
};
static DEFINE_MUTEX(percpu_charge_mutex);
+static void drain_obj_stock_slot(struct obj_stock_pcp *stock, int i);
static void drain_obj_stock(struct obj_stock_pcp *stock);
static bool obj_stock_flush_required(struct obj_stock_pcp *stock,
struct mem_cgroup *root_memcg);
@@ -3152,39 +3167,68 @@ static void unlock_stock(struct obj_stock_pcp *stock)
local_unlock(&obj_stock.lock);
}
-/* Call after __refill_obj_stock() to ensure stock->cached_objg == objcg */
+/* Call after __refill_obj_stock() so a slot for objcg exists in the stock */
static void __account_obj_stock(struct obj_cgroup *objcg,
struct obj_stock_pcp *stock, int nr,
struct pglist_data *pgdat, enum node_stat_item idx)
{
- int *bytes;
+ int16_t *bytes;
+ int i;
- if (!stock || READ_ONCE(stock->cached_objcg) != objcg)
+ /*
+ * node_id is stored as int16_t and -1 is used as the "no pgdat
+ * cached" sentinel, so MAX_NUMNODES must fit in a positive int16_t.
+ */
+ BUILD_BUG_ON(MAX_NUMNODES >= S16_MAX);
+
+ if (!stock)
+ goto direct;
+
+ for (i = 0; i < NR_OBJ_STOCK; ++i) {
+ if (READ_ONCE(stock->cached[i]) == objcg)
+ break;
+ }
+ if (i == NR_OBJ_STOCK)
goto direct;
/*
* Save vmstat data in stock and skip vmstat array update unless
- * accumulating over a page of vmstat data or when pgdat changes.
+ * accumulating over a page of vmstat data or when the objcg slot or
+ * pgdat the stats belong to changes.
*/
- if (stock->cached_pgdat != pgdat) {
- /* Flush the existing cached vmstat data */
- struct pglist_data *oldpg = stock->cached_pgdat;
+ if (stock->index < 0) {
+ stock->index = i;
+ stock->node_id = pgdat->node_id;
+ } else if (stock->index != i || stock->node_id != pgdat->node_id) {
+ struct obj_cgroup *old = READ_ONCE(stock->cached[stock->index]);
+ struct pglist_data *oldpg = NODE_DATA(stock->node_id);
if (stock->nr_slab_reclaimable_b) {
- mod_objcg_mlstate(objcg, oldpg, NR_SLAB_RECLAIMABLE_B,
+ mod_objcg_mlstate(old, oldpg, NR_SLAB_RECLAIMABLE_B,
stock->nr_slab_reclaimable_b);
stock->nr_slab_reclaimable_b = 0;
}
if (stock->nr_slab_unreclaimable_b) {
- mod_objcg_mlstate(objcg, oldpg, NR_SLAB_UNRECLAIMABLE_B,
+ mod_objcg_mlstate(old, oldpg, NR_SLAB_UNRECLAIMABLE_B,
stock->nr_slab_unreclaimable_b);
stock->nr_slab_unreclaimable_b = 0;
}
- stock->cached_pgdat = pgdat;
+ stock->index = i;
+ stock->node_id = pgdat->node_id;
}
bytes = (idx == NR_SLAB_RECLAIMABLE_B) ? &stock->nr_slab_reclaimable_b
: &stock->nr_slab_unreclaimable_b;
+ /*
+ * Cached stats are int16_t; flush directly if accumulating @nr would
+ * overflow or underflow the cache.
+ */
+ if (abs(nr + *bytes) >= S16_MAX) {
+ nr += *bytes;
+ *bytes = 0;
+ goto direct;
+ }
+
/*
* Even for large object >= PAGE_SIZE, the vmstat data will still be
* cached locally at least once before pushing it out.
@@ -3210,10 +3254,16 @@ static bool __consume_obj_stock(struct obj_cgroup *objcg,
struct obj_stock_pcp *stock,
unsigned int nr_bytes)
{
- if (objcg == READ_ONCE(stock->cached_objcg) &&
- stock->nr_bytes >= nr_bytes) {
- stock->nr_bytes -= nr_bytes;
- return true;
+ int i;
+
+ for (i = 0; i < NR_OBJ_STOCK; ++i) {
+ if (READ_ONCE(stock->cached[i]) != objcg)
+ continue;
+ if (stock->nr_bytes[i] >= nr_bytes) {
+ stock->nr_bytes[i] -= nr_bytes;
+ return true;
+ }
+ return false;
}
return false;
@@ -3234,16 +3284,42 @@ static bool consume_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes)
return ret;
}
-static void drain_obj_stock(struct obj_stock_pcp *stock)
+/* Flush the cached slab stats (if any) back to their owning objcg/pgdat. */
+static void drain_obj_stock_stats(struct obj_stock_pcp *stock)
+{
+ struct obj_cgroup *old;
+ struct pglist_data *oldpg;
+
+ if (stock->index < 0)
+ return;
+
+ old = READ_ONCE(stock->cached[stock->index]);
+ oldpg = NODE_DATA(stock->node_id);
+
+ if (stock->nr_slab_reclaimable_b) {
+ mod_objcg_mlstate(old, oldpg, NR_SLAB_RECLAIMABLE_B,
+ stock->nr_slab_reclaimable_b);
+ stock->nr_slab_reclaimable_b = 0;
+ }
+ if (stock->nr_slab_unreclaimable_b) {
+ mod_objcg_mlstate(old, oldpg, NR_SLAB_UNRECLAIMABLE_B,
+ stock->nr_slab_unreclaimable_b);
+ stock->nr_slab_unreclaimable_b = 0;
+ }
+ stock->index = -1;
+ stock->node_id = NUMA_NO_NODE;
+}
+
+static void drain_obj_stock_slot(struct obj_stock_pcp *stock, int i)
{
- struct obj_cgroup *old = READ_ONCE(stock->cached_objcg);
+ struct obj_cgroup *old = READ_ONCE(stock->cached[i]);
if (!old)
return;
- if (stock->nr_bytes) {
- unsigned int nr_pages = stock->nr_bytes >> PAGE_SHIFT;
- unsigned int nr_bytes = stock->nr_bytes & (PAGE_SIZE - 1);
+ if (stock->nr_bytes[i]) {
+ unsigned int nr_pages = stock->nr_bytes[i] >> PAGE_SHIFT;
+ unsigned int nr_bytes = stock->nr_bytes[i] & (PAGE_SIZE - 1);
if (nr_pages) {
struct mem_cgroup *memcg;
@@ -3269,44 +3345,43 @@ static void drain_obj_stock(struct obj_stock_pcp *stock)
* so it might be changed in the future.
*/
atomic_add(nr_bytes, &old->nr_charged_bytes);
- stock->nr_bytes = 0;
+ stock->nr_bytes[i] = 0;
}
- /*
- * Flush the vmstat data in current stock
- */
- if (stock->nr_slab_reclaimable_b || stock->nr_slab_unreclaimable_b) {
- if (stock->nr_slab_reclaimable_b) {
- mod_objcg_mlstate(old, stock->cached_pgdat,
- NR_SLAB_RECLAIMABLE_B,
- stock->nr_slab_reclaimable_b);
- stock->nr_slab_reclaimable_b = 0;
- }
- if (stock->nr_slab_unreclaimable_b) {
- mod_objcg_mlstate(old, stock->cached_pgdat,
- NR_SLAB_UNRECLAIMABLE_B,
- stock->nr_slab_unreclaimable_b);
- stock->nr_slab_unreclaimable_b = 0;
- }
- stock->cached_pgdat = NULL;
- }
+ /* Flush vmstat data when its owning slot is being drained. */
+ if (stock->index == i)
+ drain_obj_stock_stats(stock);
- WRITE_ONCE(stock->cached_objcg, NULL);
+ WRITE_ONCE(stock->cached[i], NULL);
obj_cgroup_put(old);
}
+static void drain_obj_stock(struct obj_stock_pcp *stock)
+{
+ int i;
+
+ for (i = 0; i < NR_OBJ_STOCK; ++i)
+ drain_obj_stock_slot(stock, i);
+}
+
static bool obj_stock_flush_required(struct obj_stock_pcp *stock,
struct mem_cgroup *root_memcg)
{
- struct obj_cgroup *objcg = READ_ONCE(stock->cached_objcg);
+ struct obj_cgroup *objcg;
struct mem_cgroup *memcg;
bool flush = false;
+ int i;
rcu_read_lock();
- if (objcg) {
+ for (i = 0; i < NR_OBJ_STOCK; ++i) {
+ objcg = READ_ONCE(stock->cached[i]);
+ if (!objcg)
+ continue;
memcg = obj_cgroup_memcg(objcg);
- if (memcg && mem_cgroup_is_descendant(memcg, root_memcg))
+ if (memcg && mem_cgroup_is_descendant(memcg, root_memcg)) {
flush = true;
+ break;
+ }
}
rcu_read_unlock();
@@ -3319,6 +3394,8 @@ static void __refill_obj_stock(struct obj_cgroup *objcg,
bool allow_uncharge)
{
unsigned int nr_pages = 0;
+ unsigned int stock_nr_bytes;
+ int i, slot = -1, empty_slot = -1;
if (!stock) {
nr_pages = nr_bytes >> PAGE_SHIFT;
@@ -3327,21 +3404,43 @@ static void __refill_obj_stock(struct obj_cgroup *objcg,
goto out;
}
- if (READ_ONCE(stock->cached_objcg) != objcg) { /* reset if necessary */
- drain_obj_stock(stock);
+ for (i = 0; i < NR_OBJ_STOCK; ++i) {
+ struct obj_cgroup *cached = READ_ONCE(stock->cached[i]);
+
+ if (!cached) {
+ if (empty_slot == -1)
+ empty_slot = i;
+ continue;
+ }
+ if (cached == objcg) {
+ slot = i;
+ break;
+ }
+ }
+
+ if (slot == -1) {
+ slot = empty_slot;
+ if (slot == -1) {
+ slot = get_random_u32_below(NR_OBJ_STOCK);
+ drain_obj_stock_slot(stock, slot);
+ }
obj_cgroup_get(objcg);
- stock->nr_bytes = atomic_read(&objcg->nr_charged_bytes)
+ stock->nr_bytes[slot] = atomic_read(&objcg->nr_charged_bytes)
? atomic_xchg(&objcg->nr_charged_bytes, 0) : 0;
- WRITE_ONCE(stock->cached_objcg, objcg);
+ WRITE_ONCE(stock->cached[slot], objcg);
allow_uncharge = true; /* Allow uncharge when objcg changes */
}
- stock->nr_bytes += nr_bytes;
- if (allow_uncharge && (stock->nr_bytes > PAGE_SIZE)) {
- nr_pages = stock->nr_bytes >> PAGE_SHIFT;
- stock->nr_bytes &= (PAGE_SIZE - 1);
+ stock_nr_bytes = (unsigned int)stock->nr_bytes[slot] + nr_bytes;
+
+ /* nr_bytes[] is uint16_t; flush if we would refill >= U16_MAX. */
+ if ((allow_uncharge && (stock_nr_bytes > PAGE_SIZE)) ||
+ stock_nr_bytes >= U16_MAX) {
+ nr_pages = stock_nr_bytes >> PAGE_SHIFT;
+ stock_nr_bytes &= (PAGE_SIZE - 1);
}
+ stock->nr_bytes[slot] = stock_nr_bytes;
out:
if (nr_pages)
--
2.53.0-Meta
^ permalink raw reply related
* Re: [PATCH v5 00/12] mm, swap: swap table phase IV: unify allocation and reduce static metadata
From: Kairui Song @ 2026-05-18 18:11 UTC (permalink / raw)
To: kasong
Cc: linux-mm, Andrew Morton, David Hildenbrand, Zi Yan, Baolin Wang,
Barry Song, Hugh Dickins, Chris Li, Kemeng Shi, Nhat Pham,
Baoquan He, Johannes Weiner, Youngjun Park, Chengming Zhou,
Roman Gushchin, Shakeel Butt, Muchun Song, Usama Arif,
linux-kernel, cgroups, Lorenzo Stoakes, Yosry Ahmed, Qi Zheng
In-Reply-To: <20260517-swap-table-p4-v5-0-88ae43e064c7@tencent.com>
On Sun, May 17, 2026 at 11:40 PM Kairui Song via B4 Relay
<devnull+kasong.tencent.com@kernel.org> wrote:
>
> From: Kairui Song <kasong@tencent.com>
>
> This series unifies the allocation and charging of anon and shmem swap
> in folios, provides better synchronization, consolidates the metadata
> management, hence dropping the static array and map, and improves the
> performance. The static metadata overhead is now close to zero, and
> workload performance is slightly improved.
>
Sashiko only gave a warning this time (and it's false positive):
> For devices using the swap cache, __swap_cache_add_check() enforces
> uniform zero flags. If the flags are mixed, it rejects the insertion with
> -EBUSY. Could the readahead and swapin fault paths then treat this as a
> transient race and unconditionally retry in an infinite loop, causing a
> kernel livelock?
> For devices that support synchronous IO and bypass the swap cache,
> swap_read_folio_zeromap() detects the mixed status, triggers a warning,
> and returns true without marking the folio uptodate. Would this cause
> do_swap_page() to abort with a SIGBUS?
> Should can_swapin_thp() retain a check to verify the uniformity of the
> zeromap status across the batch before allowing the swapin?
And no we don't need that, __swap_cache_add_check already unifed the
check. There is no device bypassing swap cache now.
swap_cache_alloc_folio now handles the fallback or returns the proper
error code.
^ permalink raw reply
* Re: [PATCH v2] memcg: cache obj_stock by memcg, not by objcg pointer
From: Shakeel Butt @ 2026-05-18 18:32 UTC (permalink / raw)
To: Andrew Morton
Cc: Johannes Weiner, Michal Hocko, Roman Gushchin, Muchun Song,
Qi Zheng, Meta kernel team, linux-mm, cgroups, linux-kernel,
kernel test robot, alex, joshua.hahnjy
In-Reply-To: <ags818dAvMjylVmP@linux.dev>
On Mon, May 18, 2026 at 09:46:04AM -0700, Shakeel Butt wrote:
> Cc Alex, Joshua (since they are working on making per-num kmem accounting work)
>
> On Sun, May 17, 2026 at 12:43:08PM -0700, Shakeel Butt wrote:
> > Commit 01b9da291c49 ("mm: memcontrol: convert objcg to be per-memcg
> > per-node type") split a memcg's single obj_cgroup into one per NUMA
> > node, but the per-CPU obj_stock_pcp still keys cached_objcg by
> > pointer. Cross-NUMA workloads now see a drain on every refill and a
> > miss on every consume that targets a sibling per-node objcg of the
> > same memcg, producing the 67.7% stress-ng switch-mq regression
> > reported by LKP.
> >
> > stock->nr_bytes are fungible across per-node objcgs of one memcg.
> > Treat the cache as keyed by memcg in __consume_obj_stock() and
> > __refill_obj_stock() so siblings share the reserve. Compare via
> > READ_ONCE(objcg->memcg) directly: pointer-compare only, no deref, so
> > the rcu_read_lock contract on obj_cgroup_memcg() does not apply.
> >
> > In the same-memcg refill path also fold the incoming objcg's
> > nr_charged_bytes into the stock; otherwise sub-page residue
> > accumulates on whichever sibling was cached at drain time and
> > obj_cgroup_release() silently drops it, leaking up to nr_node_ids *
> > (PAGE_SIZE - 1) bytes per memcg lifecycle from the page_counter.
> > This issue was reported by Sashiko.
> >
> > Update the now-stale invariant comment on __account_obj_stock().
> >
> > Qi Zheng built a specialized reproducer [1] for the corner case and
> > confirmed the fix.
> >
> > Reported-by: kernel test robot <oliver.sang@intel.com>
> > Closes: https://lore.kernel.org/oe-lkp/202605121641.b6a60cb0-lkp@intel.com
> > Fixes: 01b9da291c49 ("mm: memcontrol: convert objcg to be per-memcg per-node type")
> > Link: https://lore.kernel.org/19693be6-7132-446e-b3fc-b7e9f56e5949@linux.dev/ [1]
> > Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
> > Debugged-by: Qi Zheng <qi.zheng@linux.dev>
> > Tested-by: Qi Zheng <qi.zheng@linux.dev>
>
> Sashiko [1] reported two issues. First one seems benign but the second one is
> real. However I think we need to take a step back and rethink on how to solve
> this issue in more future proof way.
>
> It seems like Alex and Joshua are working on enabling per-node kmem accounting
> and that would need accurate per-numa association for each per-node objcg.
> So, checking objcg->memcg in consume and refill, would go against the per-node
> kmem accounting.
>
> One way to fix the regression and be future proof is to follow the approach we
> have for memcg_stock_pcp which is multiple per-cpu objcg stocks. We will need to
> test it more and depending on the additional code complexity, we will need to
> decide to backport it to 7.2 or not.
>
>
> [1] https://sashiko.dev/#/patchset/20260517194308.952655-1-shakeel.butt@linux.dev?part=1
>
Previously I had prototyped the multiple per-cpu objcg stocks (when I worked on
multi-memcg percpu stock) which I just rebased on latest linux-next and sent [1].
That patch is additional 100 LOC. For upstreaming, I will break it up into at
least 4 patches. However I am questioning about backporting them to 7.1. One
thing I can do is fix whatever sashiko is asking for and send v3 which can be
ported to 7.1 and then later for 7.2+, revert this short term fix and send out
the multiple objcg patch series.
Any concerns?
[1] http://lore.kernel.org/agtPMpQK2jXdQAY4@linux.dev
^ permalink raw reply
* Re: [RFC PATCH v5 20/29] sched/deadline: Allow deeper hierarchies of RT cgroups
From: Tejun Heo @ 2026-05-18 18:47 UTC (permalink / raw)
To: Yuri Andriaccio
Cc: luca abeni, Peter Zijlstra, Yuri Andriaccio, Ingo Molnar,
Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt,
Ben Segall, Mel Gorman, Valentin Schneider, linux-kernel, hannes,
mkoutny, cgroups
In-Reply-To: <c51eb4b9-143d-495f-a35b-090fb688cca4@santannapisa.it>
Hello, Yuri.
On Mon, May 18, 2026 at 05:27:17PM +0200, Yuri Andriaccio wrote:
> Interface:
> - File: cpu.rt.max
> - Format: <runtime>|"max" <period>
> - Default value:
> "max" <parent period> - if the parent schedules on the root runqueue.
> 0 <parent period> - if the parent is instead using HCBS.
> - Meaning (incomplete/dubious):
> The bandwidth allocated to the specific cgroup and all of its children.
> Since sum(children bw) <= own bw, a cgroup's servers will be configured
> with (own bw - sum(children bw)) bandwidth.
> A cgroup set to "max" whose parents are all set to "max" (root cgroup
> excluded)
> will run their tasks in the root runqueue.
> A cgroup set to "max" whose parent has a non-zero reservation will
> inherit the parent's configuration.
> The root cgroup's cpu.rt.max file reserves the maximum HCBS bandwidth
> for
> the whole hierarchy. Root set to "max" disable HCBS (as if set with a
> zero runtime).
I wonder whether it can be generalized more. Would something like the
following work? I'm going to ignore period for the sake of simplicity as it
doesn't seem to affect admission decisions.
- There is no root cgroup.rt.max in line with other control knobs.
- max means running in the nearest ancestor that has budget configuration.
Obviously, if no one has budget configured, run in root.
- Setting a budget is subject to admission control in both directions - the
budget source (the nearest budgeted ancestor, or the root pool if none)
should have enough to give out and the target budget should be big enough
to contain the actual usages and !max descendants in the subtree. Going
to max is always fine - the source previously gave the budget out, so it
has room to take everything back.
It seems like the above would give fairly generic behavior without abrupt
system-wide switches while staying relatively close to the behaviors of
other resource knobs. I could be missing something tho. Would something like
this work?
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH v2 00/10] sched: Flatten the pick
From: Tejun Heo @ 2026-05-18 19:11 UTC (permalink / raw)
To: Peter Zijlstra
Cc: mingo, longman, chenridong, juri.lelli, vincent.guittot,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid, hannes,
mkoutny, cgroups, linux-kernel, jstultz, kprateek.nayak, qyousef
In-Reply-To: <20260518071456.GO3102624@noisy.programming.kicks-ass.net>
Hello, Peter.
On Mon, May 18, 2026 at 09:14:56AM +0200, Peter Zijlstra wrote:
...
> So the current scheme will inflate the part of A to be double the weight
> (of B), giving them 2 out of 3 parts on the contended CPUs, but then B
> will still get complete / uncontested access to those extra 128 CPUs,
> resulting in a 2:4 weight distribution.
>
> Which also isn't as straight forward as one might think.
Right, the current behavior isn't quite what people would expect intuitively
either.
...
> So for the one contended CPU A gets 256 out of 257 parts, while B gets
> the full CPU for the remaining 255 CPUs, for a:
>
> 256 1 257
> --- : --- + 255*--- = 256:65535 ~ 1:256
> 257 257 257
>
> distribution. While with the new scheme it would be:
>
> 1 1 2
> - : - + 255*- = 1:511
> 2 2 2
>
> Which, realistically isn't all that different, except the old scheme has
> this really large weight to deal with.
>
> So from where I'm sitting, yes different, but it behaves better.
I see. Thread cardinality and affinity problems make weight based
distribution such a pain. I wonder whether this can be better solved by
turning it into a two-layer allocation problem - groups to CPUs and then
timeshare on CPUs as necessary. That comes with a lot of its own problems
but it can, aspirationally at least, approximate global weight distribution
and would have better locality properties.
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH] cgroup/rdma: drop unnecessary READ_ONCE() on event counters
From: Tejun Heo @ 2026-05-18 19:31 UTC (permalink / raw)
To: Tao Cui; +Cc: hannes, mkoutny, cgroups
In-Reply-To: <20260516052537.450732-1-cuitao@kylinos.cn>
Hello,
On Sat, May 16, 2026 at 01:25:37PM +0800, Tao Cui wrote:
> All accesses to the event counters are serialized by rdmacg_mutex,
> making the READ_ONCE() annotations unnecessary. Remove them.
Applied to cgroup/for-7.2.
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH v2] cgroup/rstat: validate cpu before css_rstat_cpu() access
From: Tejun Heo @ 2026-05-18 19:36 UTC (permalink / raw)
To: Qing Ming
Cc: Josef Bacik, Jens Axboe, Johannes Weiner, Michal Koutný,
Michal Hocko, Roman Gushchin, Shakeel Butt, Muchun Song,
Andrew Morton, Alexei Starovoitov, Hao Luo, Yosry Ahmed, cgroups,
linux-block, linux-kernel, linux-mm, bpf
In-Reply-To: <20260516070849.106141-1-a0yami@mailbox.org>
Hello,
> Qing Ming (1):
> cgroup/rstat: validate cpu before css_rstat_cpu() access
Applied to cgroup/for-7.1-fixes.
In hindsight, we should have added a separate kfunc wrapper from the
start instead of tagging css_rstat_updated() with __bpf_kfunc directly,
which would have avoided the rename. Oh well, it is what it is.
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH RFC 2/5] dma-heap: charge dma-buf memory via explicit memcg
From: T.J. Mercier @ 2026-05-18 21:12 UTC (permalink / raw)
To: Barry Song
Cc: Christian König, Albert Esteve, Tejun Heo, Johannes Weiner,
Michal Koutný, Jonathan Corbet, Shuah Khan, Sumit Semwal,
Michal Hocko, Roman Gushchin, Shakeel Butt, Muchun Song,
Andrew Morton, Benjamin Gaignard, Brian Starkey, John Stultz,
Christian Brauner, Paul Moore, James Morris, Serge E. Hallyn,
Stephen Smalley, Ondrej Mosnacek, Shuah Khan, cgroups, linux-doc,
linux-kernel, linux-media, dri-devel, linaro-mm-sig, linux-mm,
linux-security-module, selinux, linux-kselftest, mripard,
echanude
In-Reply-To: <CAGsJ_4zjrFJYQQsLThTGXR6g+2PXzeAhjyDpLHfDFqVViWvyBQ@mail.gmail.com>
On Sat, May 16, 2026 at 1:40 AM Barry Song <baohua@kernel.org> wrote:
>
> On Wed, May 13, 2026 at 2:54 AM T.J. Mercier <tjmercier@google.com> wrote:
> >
> > On Tue, May 12, 2026 at 3:14 AM Christian König
> > <christian.koenig@amd.com> wrote:
> > >
> > > On 5/12/26 11:10, Albert Esteve wrote:
> > > > On embedded platforms a central process often allocates dma-buf
> > > > memory on behalf of client applications. Without a way to
> > > > attribute the charge to the requesting client's cgroup, the
> > > > cost lands on the allocator, making per-cgroup memory limits
> > > > ineffective for the actual consumers.
> > > >
> > > > Add charge_pid_fd to struct dma_heap_allocation_data. When set to
> > > > a valid pidfd, DMA_HEAP_IOCTL_ALLOC resolves the target task's
> > > > memcg and charges the buffer there via mem_cgroup_charge_dmabuf()
> > > > inside dma_heap_buffer_alloc(). Without charge_pid_fd, and with
> > > > the mem_accounting module parameter enabled, the buffer is charged
> > > > to the allocator's own cgroup.
> > > >
> > > > Additionally, commit 3c227be90659 ("dma-buf: system_heap: account for
> > > > system heap allocation in memcg") adds __GFP_ACCOUNT to system-heap
> > > > page allocations. Keeping __GFP_ACCOUNT would charge the same pages
> > > > twice (once to kmem, once to MEMCG_DMABUF), thus remove it and route
> > > > all accounting through a single MEMCG_DMABUF path.
> > > >
> > > > Usage examples:
> > > >
> > > > 1. Central allocator charging to a client at allocation time.
> > > > The allocator knows the client's PID (e.g., from binder's
> > > > sender_pid) and uses pidfd to attribute the charge:
> > > >
> > > > pid_t client_pid = txn->sender_pid;
> > > > int pidfd = pidfd_open(client_pid, 0);
> > > >
> > > > struct dma_heap_allocation_data alloc = {
> > > > .len = buffer_size,
> > > > .fd_flags = O_RDWR | O_CLOEXEC,
> > > > .charge_pid_fd = pidfd,
> > > > };
> > > > ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc);
> > > > close(pidfd);
> > > > /* alloc.fd is now charged to client's cgroup */
> > > >
> > > > 2. Default allocation (no pidfd, mem_accounting=1).
> > > > When charge_pid_fd is not set and the mem_accounting module
> > > > parameter is enabled, the buffer is charged to the allocator's
> > > > own cgroup:
> > > >
> > > > struct dma_heap_allocation_data alloc = {
> > > > .len = buffer_size,
> > > > .fd_flags = O_RDWR | O_CLOEXEC,
> > > > };
> > > > ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc);
> > > > /* charged to current process's cgroup */
> > > >
> > > > Current limitations:
> > > >
> > > > - Single-owner model: a dma-buf carries one memcg charge regardless of
> > > > how many processes share it. Means only the first owner (and exporter)
> > > > of the shared buffer bears the charge.
> > > > - Only memcg accounting supported. While this makes sense for system
> > > > heap buffers, other heaps (e.g., CMA heaps) will require selectively
> > > > charging also for the dmem controller.
> > >
> > > Well that doesn't looks soo bad, it at least seems to tackle the problem at hand for Android and some of other embedded use cases.
> >
> > Yeah I think this might work. I know of 3 cases, and it trivially
> > solves the first two. The third requires some work on our end to
> > extend our userspace interfaces to include the pidfd but it seems
> > doable. I'm checking with our graphics folks.
> >
> > 1) Direct allocation from user (e.g. app -> allocation ioctl on
> > /dev/dma_heap/foo)
> > No changes required to userspace. mem_accounting=1 charges the app.
> >
> > 2) Single hop remote allocation (e.g. app -> AHardwareBuffer_allocate
> > -> gralloc)
> > gralloc has the caller's pid as described in the commit message. Open
> > a pidfd and pass it in the dma_heap_allocation_data.
> >
> > 3) Double hop remote allocation (e.g. app -> dequeueBuffer ->
> > SurfaceFlinger -> gralloc)
> > In this case gralloc knows SurfaceFlinger's pid, but not the app's. So
> > we need to add the app's pidfd to the SurfaceFlinger -> gralloc
> > interface, or transfer the memcg charge from SurfaceFlinger to the app
> > after the allocation.
> > It'd be nice to avoid the charge transfer option entirely, but if we
> > need it that doesn't seem so bad in this case because it's a bulk
> > charge for the entire dmabuf rather than per-page. So the exporter
> > doesn't need to get involved (we wouldn't need a new dma_buf_op) and
> > we wouldn't have to worry about looping and locking for each page.
> >
>
> Hi T.J.,
>
> Your description of the three different cases sounds very interesting.
> It helps me understand how difficult it can be to correctly charge
> dma-buf in the current user scenarios.
>
> I’m wondering where I can find Android userspace code that transfers
> the PID of RPC callers. Do we have any existing sample code in Android
> for this?
Hi Barry,
In Java android.os.Binder.getCallingPid() will provide it. Here
> > > I'm just not sure if this is future prove and will work for all use cases, e.g. cloud gaming, native context for automotive etc...
>
> Thanks
> Barry
^ permalink raw reply
* Re: [PATCH v2 00/10] sched: Flatten the pick
From: Peter Zijlstra @ 2026-05-18 21:12 UTC (permalink / raw)
To: Vincent Guittot
Cc: mingo, longman, chenridong, juri.lelli, dietmar.eggemann, rostedt,
bsegall, mgorman, vschneid, tj, hannes, mkoutny, cgroups,
linux-kernel, jstultz, kprateek.nayak, qyousef
In-Reply-To: <CAKfTPtCXOnjtVV1gKLnbS8Lo6W4r8hbdUDYVYLMd2Qc1ZqBq4w@mail.gmail.com>
On Mon, May 18, 2026 at 03:34:51PM +0200, Vincent Guittot wrote:
> On Wed, 13 May 2026 at 13:35, Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > On Tue, May 12, 2026 at 10:42:33AM +0200, Vincent Guittot wrote:
> >
> > > I haven't reviewed the patches yet but I ran some tests with it while
> > > testing sched latency related changes for short slice wakeup
> > > preemption. I have some large hackbench regressions with this series
> > > on HMP system with and without EAS. those figures are unexpected
> > > because the benchs run on root cfs
> > >
> > > One example with hackbench 8 groups thread pipe
> > > tip/sched/core tip/sched/core +this patchset +this patchset
> > > slice 2.8ms 16ms 2.8ms 16ms
> > > dragonboard rb5 with EAS
> > > 0,748(+/-4,6%) 0,621(+/-3.6%) +17% 1,915(+/-7.9%) -156%
> > > 0,689(+/- 9.1%) +8%
> > >
> > > radxa orion6 HMP without EAS
> > > 0,588(+/-5.8%) 0,677(+/-5.9%) -15% 1,505(+/-10%) -156%
> > > 1,071(+/-5.9%) -82%
> > >
> > > Increasing the slice partly removes regressions but tis is surprising
> > > because the bench runs at root cfs and I thought that results will not
> > > change in such a case
> >
> > D'oh :/
> >
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index e54da4c6c945..77d0e1937f2c 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -9071,7 +9071,7 @@ static void wakeup_preempt_fair(struct rq *rq, struct task_struct *p, int wake_f
> > enum preempt_wakeup_action preempt_action = PREEMPT_WAKEUP_PICK;
> > struct task_struct *donor = rq->donor;
> > struct sched_entity *nse, *se = &donor->se, *pse = &p->se;
> > - struct cfs_rq *cfs_rq = task_cfs_rq(donor);
> > + struct cfs_rq *cfs_rq = &rq->cfs;
>
> I tested this patch on top of the series but it doesn't fix the perf
> regression on rb5
>
> hackbench 8 groups thread pipe is still at 1.907(+/-7.6%) with default
> slice duration
Weird, I can't reproduce anymore with this fixed :/
I'll try more hackbench variants tomorrow I suppose.
^ permalink raw reply
* Re: [PATCH RFC 2/5] dma-heap: charge dma-buf memory via explicit memcg
From: T.J. Mercier @ 2026-05-18 21:17 UTC (permalink / raw)
To: Barry Song
Cc: Christian König, Albert Esteve, Tejun Heo, Johannes Weiner,
Michal Koutný, Jonathan Corbet, Shuah Khan, Sumit Semwal,
Michal Hocko, Roman Gushchin, Shakeel Butt, Muchun Song,
Andrew Morton, Benjamin Gaignard, Brian Starkey, John Stultz,
Christian Brauner, Paul Moore, James Morris, Serge E. Hallyn,
Stephen Smalley, Ondrej Mosnacek, Shuah Khan, cgroups, linux-doc,
linux-kernel, linux-media, dri-devel, linaro-mm-sig, linux-mm,
linux-security-module, selinux, linux-kselftest, mripard,
echanude
In-Reply-To: <CABdmKX0gqg309hcXcOHSj_yTg0h1zwDL34GDk8mX3wp4YoyfDg@mail.gmail.com>
On Mon, May 18, 2026 at 2:12 PM T.J. Mercier <tjmercier@google.com> wrote:
>
> On Sat, May 16, 2026 at 1:40 AM Barry Song <baohua@kernel.org> wrote:
> >
> > On Wed, May 13, 2026 at 2:54 AM T.J. Mercier <tjmercier@google.com> wrote:
> > >
> > > On Tue, May 12, 2026 at 3:14 AM Christian König
> > > <christian.koenig@amd.com> wrote:
> > > >
> > > > On 5/12/26 11:10, Albert Esteve wrote:
> > > > > On embedded platforms a central process often allocates dma-buf
> > > > > memory on behalf of client applications. Without a way to
> > > > > attribute the charge to the requesting client's cgroup, the
> > > > > cost lands on the allocator, making per-cgroup memory limits
> > > > > ineffective for the actual consumers.
> > > > >
> > > > > Add charge_pid_fd to struct dma_heap_allocation_data. When set to
> > > > > a valid pidfd, DMA_HEAP_IOCTL_ALLOC resolves the target task's
> > > > > memcg and charges the buffer there via mem_cgroup_charge_dmabuf()
> > > > > inside dma_heap_buffer_alloc(). Without charge_pid_fd, and with
> > > > > the mem_accounting module parameter enabled, the buffer is charged
> > > > > to the allocator's own cgroup.
> > > > >
> > > > > Additionally, commit 3c227be90659 ("dma-buf: system_heap: account for
> > > > > system heap allocation in memcg") adds __GFP_ACCOUNT to system-heap
> > > > > page allocations. Keeping __GFP_ACCOUNT would charge the same pages
> > > > > twice (once to kmem, once to MEMCG_DMABUF), thus remove it and route
> > > > > all accounting through a single MEMCG_DMABUF path.
> > > > >
> > > > > Usage examples:
> > > > >
> > > > > 1. Central allocator charging to a client at allocation time.
> > > > > The allocator knows the client's PID (e.g., from binder's
> > > > > sender_pid) and uses pidfd to attribute the charge:
> > > > >
> > > > > pid_t client_pid = txn->sender_pid;
> > > > > int pidfd = pidfd_open(client_pid, 0);
> > > > >
> > > > > struct dma_heap_allocation_data alloc = {
> > > > > .len = buffer_size,
> > > > > .fd_flags = O_RDWR | O_CLOEXEC,
> > > > > .charge_pid_fd = pidfd,
> > > > > };
> > > > > ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc);
> > > > > close(pidfd);
> > > > > /* alloc.fd is now charged to client's cgroup */
> > > > >
> > > > > 2. Default allocation (no pidfd, mem_accounting=1).
> > > > > When charge_pid_fd is not set and the mem_accounting module
> > > > > parameter is enabled, the buffer is charged to the allocator's
> > > > > own cgroup:
> > > > >
> > > > > struct dma_heap_allocation_data alloc = {
> > > > > .len = buffer_size,
> > > > > .fd_flags = O_RDWR | O_CLOEXEC,
> > > > > };
> > > > > ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc);
> > > > > /* charged to current process's cgroup */
> > > > >
> > > > > Current limitations:
> > > > >
> > > > > - Single-owner model: a dma-buf carries one memcg charge regardless of
> > > > > how many processes share it. Means only the first owner (and exporter)
> > > > > of the shared buffer bears the charge.
> > > > > - Only memcg accounting supported. While this makes sense for system
> > > > > heap buffers, other heaps (e.g., CMA heaps) will require selectively
> > > > > charging also for the dmem controller.
> > > >
> > > > Well that doesn't looks soo bad, it at least seems to tackle the problem at hand for Android and some of other embedded use cases.
> > >
> > > Yeah I think this might work. I know of 3 cases, and it trivially
> > > solves the first two. The third requires some work on our end to
> > > extend our userspace interfaces to include the pidfd but it seems
> > > doable. I'm checking with our graphics folks.
> > >
> > > 1) Direct allocation from user (e.g. app -> allocation ioctl on
> > > /dev/dma_heap/foo)
> > > No changes required to userspace. mem_accounting=1 charges the app.
> > >
> > > 2) Single hop remote allocation (e.g. app -> AHardwareBuffer_allocate
> > > -> gralloc)
> > > gralloc has the caller's pid as described in the commit message. Open
> > > a pidfd and pass it in the dma_heap_allocation_data.
> > >
> > > 3) Double hop remote allocation (e.g. app -> dequeueBuffer ->
> > > SurfaceFlinger -> gralloc)
> > > In this case gralloc knows SurfaceFlinger's pid, but not the app's. So
> > > we need to add the app's pidfd to the SurfaceFlinger -> gralloc
> > > interface, or transfer the memcg charge from SurfaceFlinger to the app
> > > after the allocation.
> > > It'd be nice to avoid the charge transfer option entirely, but if we
> > > need it that doesn't seem so bad in this case because it's a bulk
> > > charge for the entire dmabuf rather than per-page. So the exporter
> > > doesn't need to get involved (we wouldn't need a new dma_buf_op) and
> > > we wouldn't have to worry about looping and locking for each page.
> > >
> >
> > Hi T.J.,
> >
> > Your description of the three different cases sounds very interesting.
> > It helps me understand how difficult it can be to correctly charge
> > dma-buf in the current user scenarios.
> >
> > I’m wondering where I can find Android userspace code that transfers
> > the PID of RPC callers. Do we have any existing sample code in Android
> > for this?
>
> Hi Barry,
>
> In Java android.os.Binder.getCallingPid() will provide it. Here
... let me try again
Here are some examples from the framework code:
https://cs.android.com/search?q=getCallingPid%20f:ActivityManager&sq=&ss=android%2Fplatform%2Fsuperproject
In native code we have AIBinder_getCallingPid and
android::IPCThreadState::self()->getCallingPid() (or
android::hardware::IPCThreadState::self()->getCallingPid() for HIDL)
https://cs.android.com/search?q=getCallingPid%20l:cpp%20-f:prebuilt&ss=android%2Fplatform%2Fsuperproject
^ permalink raw reply
* Re: [PATCH v5 00/12] mm, swap: swap table phase IV: unify allocation and reduce static metadata
From: Andrew Morton @ 2026-05-18 21:51 UTC (permalink / raw)
To: Kairui Song
Cc: kasong, linux-mm, David Hildenbrand, Zi Yan, Baolin Wang,
Barry Song, Hugh Dickins, Chris Li, Kemeng Shi, Nhat Pham,
Baoquan He, Johannes Weiner, Youngjun Park, Chengming Zhou,
Roman Gushchin, Shakeel Butt, Muchun Song, Usama Arif,
linux-kernel, cgroups, Lorenzo Stoakes, Yosry Ahmed, Qi Zheng,
Roman Gushchin
In-Reply-To: <CAMgjq7DryNOmJbJ38tiwFadVT3oaMTTtQ3=BxD70s5AVjG8pbw@mail.gmail.com>
On Tue, 19 May 2026 02:11:35 +0800 Kairui Song <ryncsn@gmail.com> wrote:
> On Sun, May 17, 2026 at 11:40 PM Kairui Song via B4 Relay
> <devnull+kasong.tencent.com@kernel.org> wrote:
> >
> > From: Kairui Song <kasong@tencent.com>
> >
> > This series unifies the allocation and charging of anon and shmem swap
> > in folios, provides better synchronization, consolidates the metadata
> > management, hence dropping the static array and map, and improves the
> > performance. The static metadata overhead is now close to zero, and
> > workload performance is slightly improved.
> >
>
> Sashiko only gave a warning this time (and it's false positive):
Sashiko behaved unusually. "Note: The format of this report is altered
due to recitation restrictions. Direct quotes from the original patch
are omitted, and a free-form summary is provided instead.".
https://sashiko.dev/#/patchset/20260517-swap-table-p4-v5-0-88ae43e064c7@tencent.com
Roman, what's that all about?
Thanks, I'll add this to mm-new for some testing. Review is thin at
this time, but we have a large and dedicated band of swap maintainers,
so I'm sure that will change ;)
I understand that there are some architectural/directional differences
amongst the team (or there used to be), so please don't be shy about
weighing in if you think we should be taking things in a different
direction.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox