* xfs: properly account xfs_buf items as reclaimable
@ 2026-08-13 20:40 Eric Sandeen
2026-08-13 20:40 ` [PATCH 1/2] xfs: mark kmalloc'd xfs_buf items as __GFP_RECLAIMABLE Eric Sandeen
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Eric Sandeen @ 2026-08-13 20:40 UTC (permalink / raw)
To: linux-xfs; +Cc: linux-mm
It was reported that xfs_buf items are not accounted for as
reclaimable, and therefore various statistics in /proc/meminfo
and /proc/vmstat do not reflect their reclaimable nature as
other slab-allocated items with shrinkers would do, which in
turn affects the accuracy of stats such as MemAvailable.
2 patches here to add accounting for xfs_bufs allocated via
xfs_buf_alloc_kmem() and via xfs_buf_alloc_folio().
I was not sure how to handle xfs_buf_alloc_vmalloc but I think
that's a relatively rare path, and the above two will capture
the majority of xfs_buf allocations.
I've cc:d linux-mm just because i think this is the first
user of NR_KERNEL_MISC_RECLAIMABLE in the tree?
Thanks,
-Eric
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 1/2] xfs: mark kmalloc'd xfs_buf items as __GFP_RECLAIMABLE 2026-08-13 20:40 xfs: properly account xfs_buf items as reclaimable Eric Sandeen @ 2026-08-13 20:40 ` Eric Sandeen 2026-08-14 7:19 ` Christoph Hellwig 2026-08-13 20:40 ` [PATCH 2/2] xfs: count folio alloc'd xfs_buf items in NR_KERNEL_MISC_RECLAIMABLE Eric Sandeen 2026-08-14 7:30 ` xfs: properly account xfs_buf items as reclaimable Dave Chinner 2 siblings, 1 reply; 6+ messages in thread From: Eric Sandeen @ 2026-08-13 20:40 UTC (permalink / raw) To: linux-xfs; +Cc: linux-mm, Eric Sandeen xfs_bufs have a shrinker and are therefore reclaimable; mark them as such in the kmalloc path, so that they are accounted for in this way. Signed-off-by: Eric Sandeen <sandeen@redhat.com> --- fs/xfs/xfs_buf.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c index 48d7dfd3e15f..2e00a33ebb62 100644 --- a/fs/xfs/xfs_buf.c +++ b/fs/xfs/xfs_buf.c @@ -145,7 +145,7 @@ xfs_buf_alloc_kmem( ASSERT(is_power_of_2(size)); ASSERT(size < PAGE_SIZE); - bp->b_addr = kmalloc(size, gfp_mask); + bp->b_addr = kmalloc(size, gfp_mask | __GFP_RECLAIMABLE); if (!bp->b_addr) return -ENOMEM; -- 2.55.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] xfs: mark kmalloc'd xfs_buf items as __GFP_RECLAIMABLE 2026-08-13 20:40 ` [PATCH 1/2] xfs: mark kmalloc'd xfs_buf items as __GFP_RECLAIMABLE Eric Sandeen @ 2026-08-14 7:19 ` Christoph Hellwig 0 siblings, 0 replies; 6+ messages in thread From: Christoph Hellwig @ 2026-08-14 7:19 UTC (permalink / raw) To: Eric Sandeen; +Cc: linux-xfs, linux-mm On Thu, Aug 13, 2026 at 03:40:52PM -0500, Eric Sandeen wrote: > xfs_bufs have a shrinker and are therefore reclaimable; mark > them as such in the kmalloc path, so that they are accounted > for in this way. You can use up all 73 characters for your commit log :) Otherwise looks fine: Reviewed-by: Christoph Hellwig <hch@lst.de> ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 2/2] xfs: count folio alloc'd xfs_buf items in NR_KERNEL_MISC_RECLAIMABLE 2026-08-13 20:40 xfs: properly account xfs_buf items as reclaimable Eric Sandeen 2026-08-13 20:40 ` [PATCH 1/2] xfs: mark kmalloc'd xfs_buf items as __GFP_RECLAIMABLE Eric Sandeen @ 2026-08-13 20:40 ` Eric Sandeen 2026-08-14 7:25 ` Christoph Hellwig 2026-08-14 7:30 ` xfs: properly account xfs_buf items as reclaimable Dave Chinner 2 siblings, 1 reply; 6+ messages in thread From: Eric Sandeen @ 2026-08-13 20:40 UTC (permalink / raw) To: linux-xfs; +Cc: linux-mm, Eric Sandeen xfs_buf allocations via xfs_buf_alloc_folio are actually reclaimable, but are not accounted for as such. NR_KERNEL_MISC_RECLAIMABLE seems made for this purpose, although AFAICT there are no users today. To achieve this, add a new flag _XBF_PAGES to track what we have allocated this way, increment the NR_KERNEL_MISC_RECLAIMABLE count when we do, and then do the reverse when they are freed. These allocations now show up under nr_kernel_misc_reclaimable in /proc/vmstat when allocations are active, which in turn makes MemAvailable more accurate in /proc/meminfo. Signed-off-by: Eric Sandeen <sandeen@redhat.com> --- fs/xfs/xfs_buf.c | 16 +++++++++++++--- fs/xfs/xfs_buf.h | 2 ++ 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c index 2e00a33ebb62..0a853ec361d0 100644 --- a/fs/xfs/xfs_buf.c +++ b/fs/xfs/xfs_buf.c @@ -114,8 +114,15 @@ xfs_buf_free( vfree(bp->b_addr); else if (bp->b_flags & _XBF_KMEM) kfree(bp->b_addr); - else if (bp->b_addr) - folio_put(virt_to_folio(bp->b_addr)); + else if (bp->b_addr) { + struct folio *folio = virt_to_folio(bp->b_addr); + + if (bp->b_flags & _XBF_PAGES) + mod_node_page_state(folio_pgdat(folio), + NR_KERNEL_MISC_RECLAIMABLE, + -folio_nr_pages(folio)); + folio_put(folio); + } call_rcu(&bp->b_rcu, xfs_buf_free_callback); } @@ -132,6 +139,9 @@ xfs_buf_alloc_folio( if (!folio) return -ENOMEM; bp->b_addr = folio_address(folio); + bp->b_flags |= _XBF_PAGES; + mod_node_page_state(folio_pgdat(folio), NR_KERNEL_MISC_RECLAIMABLE, + folio_nr_pages(folio)); trace_xfs_buf_backing_folio(bp, _RET_IP_); return 0; } @@ -427,7 +437,7 @@ xfs_buf_find_lock( return -ENOENT; } ASSERT((bp->b_flags & _XBF_DELWRI_Q) == 0); - bp->b_flags &= _XBF_KMEM; + bp->b_flags &= (_XBF_PAGES | _XBF_KMEM); bp->b_ops = NULL; } return 0; diff --git a/fs/xfs/xfs_buf.h b/fs/xfs/xfs_buf.h index 79cc9c3f0254..ba7bfafe67b6 100644 --- a/fs/xfs/xfs_buf.h +++ b/fs/xfs/xfs_buf.h @@ -38,6 +38,7 @@ struct xfs_buf; #define _XBF_LOGRECOVERY (1u << 18)/* log recovery buffer */ /* flags used only internally */ +#define _XBF_PAGES (1u << 20)/* backed by folio/page allocator */ #define _XBF_KMEM (1u << 21)/* backed by heap memory */ #define _XBF_DELWRI_Q (1u << 22)/* buffer on a delwri queue */ @@ -62,6 +63,7 @@ typedef unsigned int xfs_buf_flags_t; { XBF_STALE, "STALE" }, \ { XBF_WRITE_FAIL, "WRITE_FAIL" }, \ { _XBF_LOGRECOVERY, "LOG_RECOVERY" }, \ + { _XBF_PAGES, "PAGES" }, \ { _XBF_KMEM, "KMEM" }, \ { _XBF_DELWRI_Q, "DELWRI_Q" }, \ /* The following interface flags should never be set */ \ -- 2.55.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] xfs: count folio alloc'd xfs_buf items in NR_KERNEL_MISC_RECLAIMABLE 2026-08-13 20:40 ` [PATCH 2/2] xfs: count folio alloc'd xfs_buf items in NR_KERNEL_MISC_RECLAIMABLE Eric Sandeen @ 2026-08-14 7:25 ` Christoph Hellwig 0 siblings, 0 replies; 6+ messages in thread From: Christoph Hellwig @ 2026-08-14 7:25 UTC (permalink / raw) To: Eric Sandeen; +Cc: linux-xfs, linux-mm, Vlastimil Babka, Roman Gushchin On Thu, Aug 13, 2026 at 03:40:53PM -0500, Eric Sandeen wrote: > xfs_buf allocations via xfs_buf_alloc_folio are actually reclaimable, > but are not accounted for as such. NR_KERNEL_MISC_RECLAIMABLE seems > made for this purpose, although AFAICT there are no users today. > > To achieve this, add a new flag _XBF_PAGES to track what we have > allocated this way, increment the NR_KERNEL_MISC_RECLAIMABLE > count when we do, and then do the reverse when they are freed. I don't see why we'd need the flag. All buffers are either allocated using vmalloc, kmalloc or as folios. So it should always be set for folio allocations (and is misnamed since it now covers folios as well). > > These allocations now show up under nr_kernel_misc_reclaimable in > /proc/vmstat when allocations are active, which in turn makes > MemAvailable more accurate in /proc/meminfo. Adding linux-mm and a few people that touched this counter for opinions. I guess the counter is only for full folios, and we should not set it for the small than page kmalloc allocations? But even with that, why not set it for the vmalloc allocations? > > Signed-off-by: Eric Sandeen <sandeen@redhat.com> > --- > fs/xfs/xfs_buf.c | 16 +++++++++++++--- > fs/xfs/xfs_buf.h | 2 ++ > 2 files changed, 15 insertions(+), 3 deletions(-) > > diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c > index 2e00a33ebb62..0a853ec361d0 100644 > --- a/fs/xfs/xfs_buf.c > +++ b/fs/xfs/xfs_buf.c > @@ -114,8 +114,15 @@ xfs_buf_free( > vfree(bp->b_addr); > else if (bp->b_flags & _XBF_KMEM) > kfree(bp->b_addr); > - else if (bp->b_addr) > - folio_put(virt_to_folio(bp->b_addr)); > + else if (bp->b_addr) { > + struct folio *folio = virt_to_folio(bp->b_addr); > + > + if (bp->b_flags & _XBF_PAGES) > + mod_node_page_state(folio_pgdat(folio), > + NR_KERNEL_MISC_RECLAIMABLE, > + -folio_nr_pages(folio)); > + folio_put(folio); > + } > > call_rcu(&bp->b_rcu, xfs_buf_free_callback); > } > @@ -132,6 +139,9 @@ xfs_buf_alloc_folio( > if (!folio) > return -ENOMEM; > bp->b_addr = folio_address(folio); > + bp->b_flags |= _XBF_PAGES; > + mod_node_page_state(folio_pgdat(folio), NR_KERNEL_MISC_RECLAIMABLE, > + folio_nr_pages(folio)); > trace_xfs_buf_backing_folio(bp, _RET_IP_); > return 0; > } > @@ -427,7 +437,7 @@ xfs_buf_find_lock( > return -ENOENT; > } > ASSERT((bp->b_flags & _XBF_DELWRI_Q) == 0); > - bp->b_flags &= _XBF_KMEM; > + bp->b_flags &= (_XBF_PAGES | _XBF_KMEM); > bp->b_ops = NULL; > } > return 0; > diff --git a/fs/xfs/xfs_buf.h b/fs/xfs/xfs_buf.h > index 79cc9c3f0254..ba7bfafe67b6 100644 > --- a/fs/xfs/xfs_buf.h > +++ b/fs/xfs/xfs_buf.h > @@ -38,6 +38,7 @@ struct xfs_buf; > #define _XBF_LOGRECOVERY (1u << 18)/* log recovery buffer */ > > /* flags used only internally */ > +#define _XBF_PAGES (1u << 20)/* backed by folio/page allocator */ > #define _XBF_KMEM (1u << 21)/* backed by heap memory */ > #define _XBF_DELWRI_Q (1u << 22)/* buffer on a delwri queue */ > > @@ -62,6 +63,7 @@ typedef unsigned int xfs_buf_flags_t; > { XBF_STALE, "STALE" }, \ > { XBF_WRITE_FAIL, "WRITE_FAIL" }, \ > { _XBF_LOGRECOVERY, "LOG_RECOVERY" }, \ > + { _XBF_PAGES, "PAGES" }, \ > { _XBF_KMEM, "KMEM" }, \ > { _XBF_DELWRI_Q, "DELWRI_Q" }, \ > /* The following interface flags should never be set */ \ > -- > 2.55.0 > > ---end quoted text--- ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: xfs: properly account xfs_buf items as reclaimable 2026-08-13 20:40 xfs: properly account xfs_buf items as reclaimable Eric Sandeen 2026-08-13 20:40 ` [PATCH 1/2] xfs: mark kmalloc'd xfs_buf items as __GFP_RECLAIMABLE Eric Sandeen 2026-08-13 20:40 ` [PATCH 2/2] xfs: count folio alloc'd xfs_buf items in NR_KERNEL_MISC_RECLAIMABLE Eric Sandeen @ 2026-08-14 7:30 ` Dave Chinner 2 siblings, 0 replies; 6+ messages in thread From: Dave Chinner @ 2026-08-14 7:30 UTC (permalink / raw) To: Eric Sandeen; +Cc: linux-xfs, linux-mm On Thu, Aug 13, 2026 at 03:40:51PM -0500, Eric Sandeen wrote: > It was reported that xfs_buf items are not accounted for as > reclaimable, Actually, they are: xfs_buf_cache = kmem_cache_create("xfs_buf", sizeof(struct xfs_buf), 0, SLAB_HWCACHE_ALIGN | >>>>>> SLAB_RECLAIM_ACCOUNT, NULL); So the xfs_buf items themselves are accounted as reclaimable slab objects, and these are what the xfs_buf shrinker itself acts on. > and therefore various statistics in /proc/meminfo > and /proc/vmstat do not reflect their reclaimable nature as > other slab-allocated items with shrinkers would do, which in > turn affects the accuracy of stats such as MemAvailable. > > 2 patches here to add accounting for xfs_bufs allocated via > xfs_buf_alloc_kmem() and via xfs_buf_alloc_folio(). That's accounting for the memory attached to the xfs_buf, not the xfs_buf itself. We don't use the size of that memory for reclaim purposes, hence we haven't ever tracked it. It is, however, fed back into shrinker based memory reclaim via mm_account_reclaimed_pages() in xfs_buf_free() and hence memory reclaim correctly tracks how much memory was released by the xfs_buf shrinker scan, even if the user visible stats don't show it. > I was not sure how to handle xfs_buf_alloc_vmalloc but I think > that's a relatively rare path, and the above two will capture > the majority of xfs_buf allocations. I think both folio and vmalloc should be accounted in the same manner - as a number of pages based on the size of the buffer (i.e. same as mm_account_reclaimed_pages() does already). It doesn't matter if it is vmalloc or a high order folio, the amount of memory is the same. If it gets accounted to the node of the first folio, then it will at least always be consistently accounted, if not always 100% accurate for the vmalloc case. > I've cc:d linux-mm just because i think this is the first > user of NR_KERNEL_MISC_RECLAIMABLE in the tree? These are fs buffers - shouldn't they be accounted as something that reports as "cached" or "buffers" in /proc/meminfo? I mean, if you're going to do this to make meminfo/vmstat report the memory usage, shouldn't we make the effort to classify the memory usage correctly for the user? Cheers, Dave. -- Dave Chinner dgc@kernel.org ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-08-14 7:30 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-08-13 20:40 xfs: properly account xfs_buf items as reclaimable Eric Sandeen 2026-08-13 20:40 ` [PATCH 1/2] xfs: mark kmalloc'd xfs_buf items as __GFP_RECLAIMABLE Eric Sandeen 2026-08-14 7:19 ` Christoph Hellwig 2026-08-13 20:40 ` [PATCH 2/2] xfs: count folio alloc'd xfs_buf items in NR_KERNEL_MISC_RECLAIMABLE Eric Sandeen 2026-08-14 7:25 ` Christoph Hellwig 2026-08-14 7:30 ` xfs: properly account xfs_buf items as reclaimable Dave Chinner
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox