From: Dave Chinner <david@fromorbit.com>
To: Vlastimil Babka <vbabka@suse.cz>
Cc: Michal Hocko <mhocko@kernel.org>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Andrew Morton <akpm@linux-foundation.org>,
Mel Gorman <mgorman@techsingularity.net>,
Johannes Weiner <hannes@cmpxchg.org>,
"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
Hugh Dickins <hughd@google.com>,
David Rientjes <rientjes@google.com>
Subject: Re: [RFC] fs/proc/meminfo: introduce Unaccounted statistic
Date: Fri, 21 Oct 2016 22:17:32 +1100 [thread overview]
Message-ID: <20161021111732.GR14023@dastard> (raw)
In-Reply-To: <70fe5da3-c739-58ce-0531-299b48e0ca9e@suse.cz>
On Fri, Oct 21, 2016 at 09:25:10AM +0200, Vlastimil Babka wrote:
> On 10/21/2016 12:59 AM, Dave Chinner wrote:
> >On Thu, Oct 20, 2016 at 03:33:58PM +0200, Michal Hocko wrote:
> >>On Thu 20-10-16 14:11:49, Vlastimil Babka wrote:
> >>[...]
> >>> Hi, I'm wondering if people would find this useful. If you think it is, and
> >>> to not make performance worse, I could also make sure in proper submission
> >>> that values are not read via global_page_state() multiple times etc...
> >>
> >>I definitely find this information useful and hate to do the math all
> >>the time but on the other hand this is quite fragile and I can imagine
> >>we can easily forget to add something there and provide a misleading
> >>information to the userspace. So I would be worried with a long term
> >>maintainability of this.
> >
> >This will result in valid memory usage by subsystems like the XFS
> >buffer cache being reported as "unaccounted". Given this cache
> >(whose size is shrinker controlled) can grow to gigabytes in size
> >under various metadata intensive workloads, there's every chance
> >that such reporting will make users incorrectly think they have a
> >massive memory leak....
>
> Is the XFS buffer cache accounted (and visible) somewhere then? I'd
> say getting such large consumers to become visible on the same level
> as others would be another advantage...
It's handles are visible via the xfs_buf slab cache. By the time
you've got enough memory in the buffer cache for it to be noticed,
the xfs_buf slab is near the top of the list in slabtop.
Of course, because of the crazy way slub names caches, this can be
impossible to find because there isn't a "xfs_buf" slab cache that
shows up in slabtop. It'll end being called something like
"mnt_cache"....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Dave Chinner <david@fromorbit.com>
To: Vlastimil Babka <vbabka@suse.cz>
Cc: Michal Hocko <mhocko@kernel.org>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Andrew Morton <akpm@linux-foundation.org>,
Mel Gorman <mgorman@techsingularity.net>,
Johannes Weiner <hannes@cmpxchg.org>,
"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
Hugh Dickins <hughd@google.com>,
David Rientjes <rientjes@google.com>
Subject: Re: [RFC] fs/proc/meminfo: introduce Unaccounted statistic
Date: Fri, 21 Oct 2016 22:17:32 +1100 [thread overview]
Message-ID: <20161021111732.GR14023@dastard> (raw)
In-Reply-To: <70fe5da3-c739-58ce-0531-299b48e0ca9e@suse.cz>
On Fri, Oct 21, 2016 at 09:25:10AM +0200, Vlastimil Babka wrote:
> On 10/21/2016 12:59 AM, Dave Chinner wrote:
> >On Thu, Oct 20, 2016 at 03:33:58PM +0200, Michal Hocko wrote:
> >>On Thu 20-10-16 14:11:49, Vlastimil Babka wrote:
> >>[...]
> >>> Hi, I'm wondering if people would find this useful. If you think it is, and
> >>> to not make performance worse, I could also make sure in proper submission
> >>> that values are not read via global_page_state() multiple times etc...
> >>
> >>I definitely find this information useful and hate to do the math all
> >>the time but on the other hand this is quite fragile and I can imagine
> >>we can easily forget to add something there and provide a misleading
> >>information to the userspace. So I would be worried with a long term
> >>maintainability of this.
> >
> >This will result in valid memory usage by subsystems like the XFS
> >buffer cache being reported as "unaccounted". Given this cache
> >(whose size is shrinker controlled) can grow to gigabytes in size
> >under various metadata intensive workloads, there's every chance
> >that such reporting will make users incorrectly think they have a
> >massive memory leak....
>
> Is the XFS buffer cache accounted (and visible) somewhere then? I'd
> say getting such large consumers to become visible on the same level
> as others would be another advantage...
It's handles are visible via the xfs_buf slab cache. By the time
you've got enough memory in the buffer cache for it to be noticed,
the xfs_buf slab is near the top of the list in slabtop.
Of course, because of the crazy way slub names caches, this can be
impossible to find because there isn't a "xfs_buf" slab cache that
shows up in slabtop. It'll end being called something like
"mnt_cache"....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2016-10-21 11:17 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-10-20 12:11 [RFC] fs/proc/meminfo: introduce Unaccounted statistic Vlastimil Babka
2016-10-20 12:11 ` Vlastimil Babka
2016-10-20 13:33 ` Michal Hocko
2016-10-20 13:33 ` Michal Hocko
2016-10-20 22:59 ` Dave Chinner
2016-10-20 22:59 ` Dave Chinner
2016-10-21 7:25 ` Vlastimil Babka
2016-10-21 7:25 ` Vlastimil Babka
2016-10-21 11:17 ` Dave Chinner [this message]
2016-10-21 11:17 ` Dave Chinner
2016-10-21 7:36 ` Michal Hocko
2016-10-21 7:36 ` Michal Hocko
2016-10-21 7:31 ` Vlastimil Babka
2016-10-21 7:31 ` Vlastimil Babka
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20161021111732.GR14023@dastard \
--to=david@fromorbit.com \
--cc=akpm@linux-foundation.org \
--cc=hannes@cmpxchg.org \
--cc=hughd@google.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@techsingularity.net \
--cc=mhocko@kernel.org \
--cc=rientjes@google.com \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.