Re: [PATCH v8 0/6] cgroup/dmem, drm/ttm: Improve protection in contended cases

["Timur Kristóf" <timur.kristof@gmail.com>]

On Tuesday, August 4, 2026 10:25:15 AM Central European Summer Time Natalie 
Vock wrote:
> Hi all,
> 
> I've been looking into some cases where dmem protection fails to prevent
> allocations from ending up in GTT when VRAM gets scarce and apps start
> competing hard.
> 
> In short, this is because other (unprotected) applications end up
> filling VRAM before protected applications do. This causes TTM to back
> off and try allocating in GTT before anything else, and that is where
> the allocation is placed in the end. The existing eviction protection
> cannot prevent this, because no attempt at evicting is ever made
> (although you could consider the backing-off as an immediate eviction to
> GTT).
> 
> This series tries to alleviate this by adding a special case when the
> allocation is protected by cgroups: Instead of backing off immediately,
> TTM will try evicting unprotected buffers from the domain to make space
> for the protected one. This ensures that applications can actually use
> all the memory protection awarded to them by the system, without being
> prone to ping-ponging (only protected allocations can evict unprotected
> ones, never the other way around).
> 
> The first two patches just add a few small utilities needed to implement
> this to the dmem controller. The other patches are the TTM implementation:
> 
> "drm/ttm: Be more aggressive..." decouples cgroup charging from resource
> allocation to allow us to hold on to the charge even if allocation fails
> on first try, and adds a path to call ttm_bo_evict_alloc when the
> charged allocation falls within min/low protection limits.
> 
> "drm/ttm: Use common ancestor..." is a more general improvement in
> correctly implementing cgroup protection semantics. With recursive
> protection rules, unused memory protection afforded to a parent node is
> transferred to children recursively, which helps protect entire
> subtrees from stealing each others' memory without needing to protect
> each cgroup individually. This doesn't apply when considering direct
> siblings inside the same subtree, so in order to not break
> prioritization between these siblings, we need to consider the
> relationship of evictor and evictee when calculating protection.
> In practice, this fixes cases where a protected cgroup cannot steal
> memory from unprotected siblings (which, in turn, leads to eviction
> failures and new allocations being placed in GTT).
> 
> Thanks,
> Natalie
> 
> Signed-off-by: Natalie Vock <natalie.vock@gmx.de>

Very nice work!

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>

for the whole series.

> ---
> Changes in v8:
> - Fix a few Sashiko complaints
> - Link to v7:
> https://patch.msgid.link/20260721-dmemcg-aggressive-protect-v7-0-97c4b10143
> 15@gmx.de
> 
> Changes in v7:
> - Fix incorrect refcounting/locking around dmem_cgroup_common_ancestor
>   (Cascardo)
> - Rebase on latest drm-misc-next
> - Link to v6:
> https://lore.kernel.org/r/20260313-dmemcg-aggressive-protect-v6-0-7c71cc149
> 2db@gmx.de
> 
> Changes in v6:
> - Rename dmem_cgroup_common_ancestor to dmem_cgroup_get_common_ancestor
>   (Maarten)
> - Use ttm_bo_alloc_at_place in eviction cb as well (Tvrtko)
> - Clean up the more aggressive eviction logic (Tvrtko)
> - Link to v5:
> https://lore.kernel.org/r/20260302-dmemcg-aggressive-protect-v5-0-ffd3a2602
> 309@gmx.de
> 
> Changes in v5:
> - Added cgroup_common_ancestor helper to use with
>   dmem_cgroup_common_ancestor (Tejun)
> - Note: "drm/ttm: Use common ancestor..." needed minor changes since
>   dmem_cgroup_common_ancestor now grabs a reference to the ancestor
>   pool which needs to be dropped after use
> - Removed extraneous whitespaces in "drm/ttm: Split cgroup charge..."
>   and unnecessary changes done in "drm/ttm: Extract code..." (Tvrtko)
> - Applied a comment from v3 about below_low not needing to be
>   initialized in "drm/ttm: Be more aggressive..." (Tvrtko)
> - Fixed uncharging the cgroup on allocation failure (Tvrtko)
> - Fixed a typo in the message of "drm/ttm: Split cgroup charge..."
>   (Tvrtko)
> - Added case in ttm_bo_evict_cb for when charging fails, since we need
>   to retry the charge (found myself)
> - Link to v4:
> https://lore.kernel.org/r/20260225-dmemcg-aggressive-protect-v4-0-de847ab35
> 184@gmx.de
> 
> Changes in v4:
> - Split cgroup charge decoupling and eviction logic changes into
>   separate commits (Tvrtko)
> - Fix two cases of errno handling in ttm_bo_alloc_place and its caller
>   (Tvrtko)
> - Improve commit message/description of "drm/ttm: Make a helper..." (now
>   "drm/ttm: Extract code...") (Tvrtko)
> - Documentation improvements for new TTM eviction logic (Tvrtko)
> - Formatting fixes (Tvrtko)
> - Link to v3:
> https://lore.kernel.org/r/20251110-dmemcg-aggressive-protect-v3-0-219ffcfc5
> 4e9@gmx.de
> 
> Changes in v3:
> - Improved documentation around cgroup queries and TTM eviction helpers
>   (Maarten)
> - Fixed up ttm_alloc_at_place charge failure logic to return either
>   -EBUSY or -ENOSPC, not -EAGAIN (found this myself)
> - Link to v2:
> https://lore.kernel.org/r/20251015-dmemcg-aggressive-protect-v2-0-36644fb4e
> 37f@gmx.de
> 
> Changes in v2:
> - Factored out the ttm logic for charging/allocating/evicting into a
>   separate helper to keep things simpler
> - Link to v1:
> https://lore.kernel.org/r/20250915-dmemcg-aggressive-protect-v1-0-2f3353bfc
> dac@gmx.de
> 
> ---
> Natalie Vock (6):
>       cgroup/dmem: Add queries for protection values
>       cgroup,cgroup/dmem: Add (dmem_)cgroup_common_ancestor helper
>       drm/ttm: Extract code for attempting allocation in a place
>       drm/ttm: Split cgroup charge and resource allocation
>       drm/ttm: Be more aggressive when allocating below protection limit
>       drm/ttm: Use common ancestor of evictor and evictee as limit pool
> 
>  drivers/gpu/drm/ttm/ttm_bo.c       | 223
> ++++++++++++++++++++++++++++++++----- drivers/gpu/drm/ttm/ttm_resource.c | 
> 51 ++++++---
>  include/drm/ttm/ttm_resource.h     |   6 +-
>  include/linux/cgroup.h             |  21 ++++
>  include/linux/cgroup_dmem.h        |  25 +++++
>  kernel/cgroup/dmem.c               | 103 +++++++++++++++++
>  6 files changed, 383 insertions(+), 46 deletions(-)
> ---
> base-commit: 44e9eb5a762142a4aa46c0b5da7c39bfeb78910e
> change-id: 20250915-dmemcg-aggressive-protect-5cf37f717cdb
> 
> Best regards,
> --
> Natalie Vock <natalie.vock@gmx.de>