computing drop-able caches

computing drop-able caches

[Daniel Walker]

Hi,

My colleague Khalid and I are working on a patch which will provide a 
/proc file to output the size of the drop-able page cache.
One way to implement this is to use the current drop_caches /proc 
routine, but instead of actually droping the caches just add
up the amount.

Here's a quote Khalid,

"Currently there is no way to figure out the droppable pagecache size
from the meminfo output. The MemFree size can shrink during normal
system operation, when some of the memory pages get cached and is
reflected in "Cached" field. Similarly for file operations some of
the buffer memory gets cached and it is reflected in "Buffers" field.
The kernel automatically reclaims all this cached & buffered memory,
when it is needed elsewhere on the system. The only way to manually
reclaim this memory is by writing 1 to /proc/sys/vm/drop_caches. "

So my impression is that the drop-able cache is spread over two fields 
in meminfo.

Alright, the question is does this info live someplace else that we 
don't know about? Or someplace in the kernel where it could be
added to meminfo trivially ?

The point of the whole exercise is to get a better idea of free memory 
for our employer. Does it make sense to do this for computing free memory?

Any comments welcome..

Daniel

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Johannes Weiner]

On Thu, Jan 28, 2016 at 03:42:53PM -0800, Daniel Walker wrote:
> "Currently there is no way to figure out the droppable pagecache size
> from the meminfo output. The MemFree size can shrink during normal
> system operation, when some of the memory pages get cached and is
> reflected in "Cached" field. Similarly for file operations some of
> the buffer memory gets cached and it is reflected in "Buffers" field.
> The kernel automatically reclaims all this cached & buffered memory,
> when it is needed elsewhere on the system. The only way to manually
> reclaim this memory is by writing 1 to /proc/sys/vm/drop_caches. "

[...]

> The point of the whole exercise is to get a better idea of free memory for
> our employer. Does it make sense to do this for computing free memory?

/proc/meminfo::MemAvailable was added for this purpose. See the doc
text in Documentation/filesystem/proc.txt.

It's an approximation, however, because this question is not easy to
answer. Pages might be in various states and uses that can make them
unreclaimable.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Daniel Walker]

On 01/28/2016 03:58 PM, Johannes Weiner wrote:
> On Thu, Jan 28, 2016 at 03:42:53PM -0800, Daniel Walker wrote:
>> "Currently there is no way to figure out the droppable pagecache size
>> from the meminfo output. The MemFree size can shrink during normal
>> system operation, when some of the memory pages get cached and is
>> reflected in "Cached" field. Similarly for file operations some of
>> the buffer memory gets cached and it is reflected in "Buffers" field.
>> The kernel automatically reclaims all this cached & buffered memory,
>> when it is needed elsewhere on the system. The only way to manually
>> reclaim this memory is by writing 1 to /proc/sys/vm/drop_caches. "
> [...]
>
>> The point of the whole exercise is to get a better idea of free memory for
>> our employer. Does it make sense to do this for computing free memory?
> /proc/meminfo::MemAvailable was added for this purpose. See the doc
> text in Documentation/filesystem/proc.txt.
>
> It's an approximation, however, because this question is not easy to
> answer. Pages might be in various states and uses that can make them
> unreclaimable.


Khalid was telling me that our internal sources rejected MemAvailable 
because it was not accurate enough. It says in the description,
"The estimate takes into account that the system needs some page cache 
to function well". I suspect that's part of the inaccuracy. I asked 
Khalid to respond with more details on this.

Do you know of any work to make it more accurate?

Daniel

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Daniel Walker]

[-- Attachment #1: Type: text/plain, Size: 2162 bytes --]

On 01/28/2016 05:03 PM, Daniel Walker wrote:
> On 01/28/2016 03:58 PM, Johannes Weiner wrote:
>> On Thu, Jan 28, 2016 at 03:42:53PM -0800, Daniel Walker wrote:
>>> "Currently there is no way to figure out the droppable pagecache size
>>> from the meminfo output. The MemFree size can shrink during normal
>>> system operation, when some of the memory pages get cached and is
>>> reflected in "Cached" field. Similarly for file operations some of
>>> the buffer memory gets cached and it is reflected in "Buffers" field.
>>> The kernel automatically reclaims all this cached & buffered memory,
>>> when it is needed elsewhere on the system. The only way to manually
>>> reclaim this memory is by writing 1 to /proc/sys/vm/drop_caches. "
>> [...]
>>
>>> The point of the whole exercise is to get a better idea of free 
>>> memory for
>>> our employer. Does it make sense to do this for computing free memory?
>> /proc/meminfo::MemAvailable was added for this purpose. See the doc
>> text in Documentation/filesystem/proc.txt.
>>
>> It's an approximation, however, because this question is not easy to
>> answer. Pages might be in various states and uses that can make them
>> unreclaimable.
>
>
> Khalid was telling me that our internal sources rejected MemAvailable 
> because it was not accurate enough. It says in the description,
> "The estimate takes into account that the system needs some page cache 
> to function well". I suspect that's part of the inaccuracy. I asked 
> Khalid to respond with more details on this.
>

Some quotes,

"
[regarding MemAvaiable]

This new metric purportedly helps usrespace assess available memory. But,
its again based on heuristic, it takes 1/2 of page cache as reclaimable..

Somewhat arbitrary choice. Maybe appropriate for desktops, where page
cache is mainly used as page cache, not as a first class store which is
the case on embedded systems. Our systems are swap less, they have little
secondary storage, they use in-memory databases/filesystems/shared memories/
etc. which are all setup on page caches).. This metric as it is implemented
in 3.14 leads to a totally mis-leading picture of available memory"

Daniel

[-- Attachment #2: Type: text/html, Size: 4737 bytes --]

Re: computing drop-able caches

[Johannes Weiner]

On Thu, Jan 28, 2016 at 05:29:41PM -0800, Daniel Walker wrote:
> On 01/28/2016 05:03 PM, Daniel Walker wrote:
> [regarding MemAvaiable]
> 
> This new metric purportedly helps usrespace assess available memory. But,
> its again based on heuristic, it takes 1/2 of page cache as reclaimable..

No, it takes the smaller value of cache/2 and the low watermark, which
is a fraction of memory. Actually, that does look a little weird. Rik?

We don't age cache without memory pressure, you don't know how much is
used until you start taking some away. Heuristics is all we can offer.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Daniel Walker]

On 01/28/2016 05:55 PM, Johannes Weiner wrote:
> On Thu, Jan 28, 2016 at 05:29:41PM -0800, Daniel Walker wrote:
>> On 01/28/2016 05:03 PM, Daniel Walker wrote:
>> [regarding MemAvaiable]
>>
>> This new metric purportedly helps usrespace assess available memory. But,
>> its again based on heuristic, it takes 1/2 of page cache as reclaimable..
> No, it takes the smaller value of cache/2 and the low watermark, which
> is a fraction of memory. Actually, that does look a little weird. Rik?
>
> We don't age cache without memory pressure, you don't know how much is
> used until you start taking some away. Heuristics is all we can offer.

With a simple busybox root system I get this,

MemTotal:          16273996 kB
MemFree:          16137920 kB
MemAvailable:   16046132 kB

shouldn't MemAvailable be at least the same as MemFree ? I changed the 
code somewhat so it subtracted the wmark_low only, or the pagecache/2 
only, both are still under MemFree. This system has very little 
drop-able caches.

Daniel

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Johannes Weiner]

On Fri, Jan 29, 2016 at 01:21:47PM -0800, Daniel Walker wrote:
> On 01/28/2016 05:55 PM, Johannes Weiner wrote:
> >On Thu, Jan 28, 2016 at 05:29:41PM -0800, Daniel Walker wrote:
> >>On 01/28/2016 05:03 PM, Daniel Walker wrote:
> >>[regarding MemAvaiable]
> >>
> >>This new metric purportedly helps usrespace assess available memory. But,
> >>its again based on heuristic, it takes 1/2 of page cache as reclaimable..
> >No, it takes the smaller value of cache/2 and the low watermark, which
> >is a fraction of memory. Actually, that does look a little weird. Rik?
> >
> >We don't age cache without memory pressure, you don't know how much is
> >used until you start taking some away. Heuristics is all we can offer.
> 
> With a simple busybox root system I get this,
> 
> MemTotal:          16273996 kB
> MemFree:          16137920 kB
> MemAvailable:   16046132 kB
> 
> shouldn't MemAvailable be at least the same as MemFree ? I changed the code
> somewhat so it subtracted the wmark_low only, or the pagecache/2 only, both
> are still under MemFree. This system has very little drop-able caches.

No, a portion of memory is reserved for the kernel and not available
to userland. If the kernel doesn't use it it will remain free. Hence
the lower MemAvailable.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Rik van Riel]

On 01/28/2016 08:55 PM, Johannes Weiner wrote:
> On Thu, Jan 28, 2016 at 05:29:41PM -0800, Daniel Walker wrote:
>> On 01/28/2016 05:03 PM, Daniel Walker wrote:
>> [regarding MemAvaiable]
>>
>> This new metric purportedly helps usrespace assess available memory. But,
>> its again based on heuristic, it takes 1/2 of page cache as reclaimable..
> 
> No, it takes the smaller value of cache/2 and the low watermark, which
> is a fraction of memory. Actually, that does look a little weird. Rik?

No, not quite.  The page cache calculation spans two lines:

        pagecache = pages[LRU_ACTIVE_FILE] + pages[LRU_INACTIVE_FILE];
        pagecache -= min(pagecache / 2, wmark_low);

The assumption is that ALL of active & inactive file LRUs are
freeable, except for the minimum of the low watermark, or
half the page cache.

-- 
All rights reversed

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Khalid Mughal (khalidm)]

How do we explain the discrepancy between MemAvaiable and MemFree count
after we drop cache? In following output, which one represents correct
data?

[Linux_0:/]$ cat /proc/meminfo
MemTotal:        3977836 kB
MemFree:          747832 kB
MemAvailable:    1441736 kB
Buffers:          123976 kB
Cached:          1210272 kB
Active:          2496932 kB
Inactive:         585364 kB
Active(anon):    2243932 kB
Inactive(anon):   142676 kB
Active(file):     253000 kB
Inactive(file):   442688 kB
Dirty:                44 kB
AnonPages:       1748088 kB
Mapped:           406512 kB
Shmem:            638564 kB
Slab:              65656 kB
SReclaimable:      30120 kB
SUnreclaim:        35536 kB
KernelStack:        5920 kB
PageTables:        19040 kB
CommitLimit:     1988916 kB
Committed_AS:    3765252 kB

[Linux_0:/]$ echo 3 > /proc/sys/vm/drop_caches
[Linux_0:/]$ cat /proc/meminfo
MemTotal:        3977836 kB
MemFree:         1095012 kB
MemAvailable:    1434148 kB
Buffers:            4348 kB
Cached:           988228 kB
Active:          1990868 kB
Inactive:         757532 kB
Active(anon):    1759968 kB
Inactive(anon):   634568 kB
Active(file):     230900 kB
Inactive(file):   122964 kB
Dirty:               196 kB
AnonPages:       1755916 kB
Mapped:           409856 kB
Shmem:            638712 kB
Slab:              51432 kB
SReclaimable:      15688 kB
SUnreclaim:        35744 kB
KernelStack:        5760 kB
PageTables:        19060 kB
CommitLimit:     1988916 kB
Committed_AS:    3765644 kB


Thanks,
-KM


How do we explain the discrepancy between MemAvaiable and MemFree count

On 1/29/16, 2:41 PM, "Rik van Riel" <riel@redhat.com> wrote:

>On 01/28/2016 08:55 PM, Johannes Weiner wrote:
>>On Thu, Jan 28, 2016 at 05:29:41PM -0800, Daniel Walker wrote:
>>>On 01/28/2016 05:03 PM, Daniel Walker wrote:
>>>[regarding MemAvaiable]
>>>
>>>This new metric purportedly helps usrespace assess available memory.
>>>But,
>>>its again based on heuristic, it takes 1/2 of page cache as
>>>reclaimable..
>>No, it takes the smaller value of cache/2 and the low watermark, which
>>is a fraction of memory. Actually, that does look a little weird. Rik?
>
>No, not quite.  The page cache calculation spans two lines:
>
>        pagecache = pages[LRU_ACTIVE_FILE] + pages[LRU_INACTIVE_FILE];
>        pagecache -= min(pagecache / 2, wmark_low);
>
>The assumption is that ALL of active & inactive file LRUs are
>freeable, except for the minimum of the low watermark, or
>half the page cache.
>
>--
>All rights reversed
>


On 1/29/16, 2:33 PM, "Johannes Weiner" <hannes@cmpxchg.org> wrote:

>On Fri, Jan 29, 2016 at 01:21:47PM -0800, Daniel Walker wrote:
>>On 01/28/2016 05:55 PM, Johannes Weiner wrote:
>>>On Thu, Jan 28, 2016 at 05:29:41PM -0800, Daniel Walker wrote:
>>>>On 01/28/2016 05:03 PM, Daniel Walker wrote:
>>>>[regarding MemAvaiable]
>>>>
>>>>This new metric purportedly helps usrespace assess available memory.
>>>>But,
>>>>its again based on heuristic, it takes 1/2 of page cache as
>>>>reclaimable..
>>>No, it takes the smaller value of cache/2 and the low watermark, which
>>>is a fraction of memory. Actually, that does look a little weird. Rik?
>>>
>>>We don't age cache without memory pressure, you don't know how much is
>>>used until you start taking some away. Heuristics is all we can offer.
>>With a simple busybox root system I get this,
>>MemTotal:          16273996 kB
>>MemFree:          16137920 kB
>>MemAvailable:   16046132 kB
>>shouldn't MemAvailable be at least the same as MemFree ? I changed the
>>code
>>somewhat so it subtracted the wmark_low only, or the pagecache/2 only,
>>both
>>are still under MemFree. This system has very little drop-able caches.
>
>No, a portion of memory is reserved for the kernel and not available
>to userland. If the kernel doesn't use it it will remain free. Hence
>the lower MemAvailable.
>



On 1/29/16, 1:21 PM, "Daniel Walker (danielwa)" <danielwa@cisco.com> wrote:

>On 01/28/2016 05:55 PM, Johannes Weiner wrote:
>>On Thu, Jan 28, 2016 at 05:29:41PM -0800, Daniel Walker wrote:
>>>On 01/28/2016 05:03 PM, Daniel Walker wrote:
>>>[regarding MemAvaiable]
>>>
>>>This new metric purportedly helps usrespace assess available memory.
>>>But,
>>>its again based on heuristic, it takes 1/2 of page cache as
>>>reclaimable..
>>No, it takes the smaller value of cache/2 and the low watermark, which
>>is a fraction of memory. Actually, that does look a little weird. Rik?
>>
>>We don't age cache without memory pressure, you don't know how much is
>>used until you start taking some away. Heuristics is all we can offer.
>
>With a simple busybox root system I get this,
>
>MemTotal:          16273996 kB
>MemFree:          16137920 kB
>MemAvailable:   16046132 kB
>
>shouldn't MemAvailable be at least the same as MemFree ? I changed the
>code somewhat so it subtracted the wmark_low only, or the pagecache/2
>only, both are still under MemFree. This system has very little
>drop-able caches.
>
>Daniel
>


On 1/28/16, 5:55 PM, "Johannes Weiner" <hannes@cmpxchg.org> wrote:

>On Thu, Jan 28, 2016 at 05:29:41PM -0800, Daniel Walker wrote:
>>On 01/28/2016 05:03 PM, Daniel Walker wrote:
>>[regarding MemAvaiable]
>>This new metric purportedly helps usrespace assess available memory. But,
>>its again based on heuristic, it takes 1/2 of page cache as reclaimable..
>
>No, it takes the smaller value of cache/2 and the low watermark, which
>is a fraction of memory. Actually, that does look a little weird. Rik?
>
>We don't age cache without memory pressure, you don't know how much is
>used until you start taking some away. Heuristics is all we can offer.
>



On 01/28/2016 05:03 PM, Daniel Walker wrote:

On 01/28/2016 03:58 PM, Johannes Weiner wrote:
On Thu, Jan 28, 2016 at 03:42:53PM -0800, Daniel Walker wrote:
"Currently there is no way to figure out the droppable pagecache size
from the meminfo output. The MemFree size can shrink during normal
system operation, when some of the memory pages get cached and is
reflected in "Cached" field. Similarly for file operations some of
the buffer memory gets cached and it is reflected in "Buffers" field.
The kernel automatically reclaims all this cached & buffered memory,
when it is needed elsewhere on the system. The only way to manually
reclaim this memory is by writing 1 to /proc/sys/vm/drop_caches. "

[...] 

The point of the whole exercise is to get a better idea of free memory for
our employer. Does it make sense to do this for computing free memory?


/proc/meminfo::MemAvailable was added for this purpose. See the doc
text in Documentation/filesystem/proc.txt.

It's an approximation, however, because this question is not easy to
answer. Pages might be in various states and uses that can make them
unreclaimable. 

Khalid was telling me that our internal sources rejected MemAvailable
because it was not accurate enough. It says in the description,
"The estimate takes into account that the system needs some page cache to
function well". I suspect that's part of the inaccuracy. I asked Khalid to
respond with more details on this.

Some quotes,

"[regarding MemAvaiable]

This new metric purportedly helps usrespace assess available memory. But,
its again based on heuristic, it takes 1/2 of page cache as reclaimable..

Somewhat arbitrary choice. Maybe appropriate for desktops, where page
cache is mainly used as page cache, not as a first class store which is
the case on embedded systems. Our systems are swap less, they have little
secondary storage, they use in-memory databases/filesystems/shared
memories/
etc. which are all setup on page caches).. This metric as it is implemented
in 3.14 leads to a totally mis-leading picture of available memory"

Daniel



On 01/28/2016 03:58 PM, Johannes Weiner wrote:
>On Thu, Jan 28, 2016 at 03:42:53PM -0800, Daniel Walker wrote:
>>"Currently there is no way to figure out the droppable pagecache size
>>from the meminfo output. The MemFree size can shrink during normal
>>system operation, when some of the memory pages get cached and is
>>reflected in "Cached" field. Similarly for file operations some of
>>the buffer memory gets cached and it is reflected in "Buffers" field.
>>The kernel automatically reclaims all this cached & buffered memory,
>>when it is needed elsewhere on the system. The only way to manually
>>reclaim this memory is by writing 1 to /proc/sys/vm/drop_caches. "
>[...]
>
>>The point of the whole exercise is to get a better idea of free memory
>>for
>>our employer. Does it make sense to do this for computing free memory?
>/proc/meminfo::MemAvailable was added for this purpose. See the doc
>text in Documentation/filesystem/proc.txt.
>
>It's an approximation, however, because this question is not easy to
>answer. Pages might be in various states and uses that can make them
>unreclaimable.


Khalid was telling me that our internal sources rejected MemAvailable
because it was not accurate enough. It says in the description,
"The estimate takes into account that the system needs some page cache
to function well". I suspect that's part of the inaccuracy. I asked
Khalid to respond with more details on this.

Do you know of any work to make it more accurate?

Daniel


On 1/28/16, 3:58 PM, "Johannes Weiner" <hannes@cmpxchg.org> wrote:
>On Thu, Jan 28, 2016 at 03:42:53PM -0800, Daniel Walker wrote:
>>"Currently there is no way to figure out the droppable pagecache size
>>from the meminfo output. The MemFree size can shrink during normal
>>system operation, when some of the memory pages get cached and is
>>reflected in "Cached" field. Similarly for file operations some of
>>the buffer memory gets cached and it is reflected in "Buffers" field.
>>The kernel automatically reclaims all this cached & buffered memory,
>>when it is needed elsewhere on the system. The only way to manually
>>reclaim this memory is by writing 1 to /proc/sys/vm/drop_caches. "
>
>[...]
>
>>The point of the whole exercise is to get a better idea of free memory
>>for
>>our employer. Does it make sense to do this for computing free memory?
>
>/proc/meminfo::MemAvailable was added for this purpose. See the doc
>text in Documentation/filesystem/proc.txt.
>
>It's an approximation, however, because this question is not easy to
>answer. Pages might be in various states and uses that can make them
>unreclaimable.
>


On 1/28/16, 3:42 PM, "Daniel Walker (danielwa)" <danielwa@cisco.com> wrote:
>Hi,
>
>My colleague Khalid and I are working on a patch which will provide a
>/proc file to output the size of the drop-able page cache.
>One way to implement this is to use the current drop_caches /proc
>routine, but instead of actually droping the caches just add
>up the amount.
>
>Here's a quote Khalid,
>
>"Currently there is no way to figure out the droppable pagecache size
>from the meminfo output. The MemFree size can shrink during normal
>system operation, when some of the memory pages get cached and is
>reflected in "Cached" field. Similarly for file operations some of
>the buffer memory gets cached and it is reflected in "Buffers" field.
>The kernel automatically reclaims all this cached & buffered memory,
>when it is needed elsewhere on the system. The only way to manually
>reclaim this memory is by writing 1 to /proc/sys/vm/drop_caches. "
>
>So my impression is that the drop-able cache is spread over two fields
>in meminfo.
>
>Alright, the question is does this info live someplace else that we
>don't know about? Or someplace in the kernel where it could be
>added to meminfo trivially ?
>
>The point of the whole exercise is to get a better idea of free memory
>for our employer. Does it make sense to do this for computing free memory?
>
>Any comments welcome..
>
>Daniel
>


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Daniel Walker]

On 02/08/2016 12:57 PM, Khalid Mughal (khalidm) wrote:
> How do we explain the discrepancy between MemAvaiable and MemFree count
> after we drop cache? In following output, which one represents correct
> data?
>
> [Linux_0:/]$ cat /proc/meminfo
> MemTotal:        3977836 kB
> MemFree:          747832 kB
> MemAvailable:    1441736 kB
> Buffers:          123976 kB
> Cached:          1210272 kB
> Active:          2496932 kB
> Inactive:         585364 kB
> Active(anon):    2243932 kB
> Inactive(anon):   142676 kB
> Active(file):     253000 kB
> Inactive(file):   442688 kB
> Dirty:                44 kB
> AnonPages:       1748088 kB
> Mapped:           406512 kB
> Shmem:            638564 kB
> Slab:              65656 kB
> SReclaimable:      30120 kB
> SUnreclaim:        35536 kB
> KernelStack:        5920 kB
> PageTables:        19040 kB
> CommitLimit:     1988916 kB
> Committed_AS:    3765252 kB
>
> [Linux_0:/]$ echo 3 > /proc/sys/vm/drop_caches
> [Linux_0:/]$ cat /proc/meminfo
> MemTotal:        3977836 kB
> MemFree:         1095012 kB
> MemAvailable:    1434148 kB

I suspect MemAvailable takes into account more than just the droppable 
caches. For instance, reclaimable slab is included, but I don't think 
drop_caches drops that part.

Daniel

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Dave Hansen]

On 02/10/2016 10:04 AM, Daniel Walker wrote:
>> [Linux_0:/]$ echo 3 > /proc/sys/vm/drop_caches
>> [Linux_0:/]$ cat /proc/meminfo
>> MemTotal:        3977836 kB
>> MemFree:         1095012 kB
>> MemAvailable:    1434148 kB
> 
> I suspect MemAvailable takes into account more than just the droppable
> caches. For instance, reclaimable slab is included, but I don't think
> drop_caches drops that part.

There's a bit for page cache and a bit for slab, see:

	https://kernel.org/doc/Documentation/sysctl/vm.txt


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Daniel Walker]

On 02/10/2016 10:13 AM, Dave Hansen wrote:
> On 02/10/2016 10:04 AM, Daniel Walker wrote:
>>> [Linux_0:/]$ echo 3 > /proc/sys/vm/drop_caches
>>> [Linux_0:/]$ cat /proc/meminfo
>>> MemTotal:        3977836 kB
>>> MemFree:         1095012 kB
>>> MemAvailable:    1434148 kB
>> I suspect MemAvailable takes into account more than just the droppable
>> caches. For instance, reclaimable slab is included, but I don't think
>> drop_caches drops that part.
> There's a bit for page cache and a bit for slab, see:
>
> 	https://kernel.org/doc/Documentation/sysctl/vm.txt
>
>

Ok, then this looks like a defect then. I would think MemAvailable would 
always be smaller then MemFree (after echo 3 > 
/proc/sys/vm/drop_caches).. Unless there is something else be accounted 
for that we aren't aware of.

Daniel

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Rik van Riel]

[-- Attachment #1: Type: text/plain, Size: 1244 bytes --]

On Wed, 2016-02-10 at 11:11 -0800, Daniel Walker wrote:
> On 02/10/2016 10:13 AM, Dave Hansen wrote:
> > On 02/10/2016 10:04 AM, Daniel Walker wrote:
> > > > [Linux_0:/]$ echo 3 > /proc/sys/vm/drop_caches
> > > > [Linux_0:/]$ cat /proc/meminfo
> > > > MemTotal:        3977836 kB
> > > > MemFree:         1095012 kB
> > > > MemAvailable:    1434148 kB
> > > I suspect MemAvailable takes into account more than just the
> > > droppable
> > > caches. For instance, reclaimable slab is included, but I don't
> > > think
> > > drop_caches drops that part.
> > There's a bit for page cache and a bit for slab, see:
> > 
> > 	https://kernel.org/doc/Documentation/sysctl/vm.txt
> > 
> > 
> 
> Ok, then this looks like a defect then. I would think MemAvailable
> would 
> always be smaller then MemFree (after echo 3 > 
> /proc/sys/vm/drop_caches).. Unless there is something else be
> accounted 
> for that we aren't aware of.

echo 3 > /proc/sys/vm/drop_caches will only
drop unmapped page cache, IIRC

The system may still have a number of page
cache pages left that are mapped in processes,
but will be reclaimable if the VM needs the
memory for something else.

-- 
All rights reversed

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: computing drop-able caches

[Khalid Mughal (khalidm)]

[-- Attachment #1: Type: text/plain, Size: 2450 bytes --]

I did an experiment on our system. I added a small kernel patch, as
mentioned by Daniel in first email of this thread, to compute the
droppable pagecache without actually dropping it. Using this value I
computed the Available memory by adding droppable-page count to memFree
count. Then used a test application to analyze the difference between
memAvailable, and dropcacheinfo output.
The application creates 32 threads. Each allocates 64MB block using
malloc(). There is 5 second interval between each allocation, which allows
me to gather data. After all allocation is done, the threads then write
data to these blocks, using memset. This is also done incrementally,
allowing me track meminfo and dropcacheinfo.
If you look at the attached pdf, you will notice that OOM messages start
to appear when memAvailable is showing 253MB (259228 KB) Free, memFree is
13.5MB (14008 KB) Free, and dropcache based calculation ³Available memory²
is showing 21MB (21720 KB) Free.

So, it appears that memAvailable is not as accurate, especially if data is
used to warn user about system running low on memory.

-KM



On 2/11/16, 2:11 PM, "Rik van Riel" <riel@redhat.com> wrote:

>On Wed, 2016-02-10 at 11:11 -0800, Daniel Walker wrote:
>> On 02/10/2016 10:13 AM, Dave Hansen wrote:
>> > On 02/10/2016 10:04 AM, Daniel Walker wrote:
>> > > > [Linux_0:/]$ echo 3 > /proc/sys/vm/drop_caches
>> > > > [Linux_0:/]$ cat /proc/meminfo
>> > > > MemTotal:        3977836 kB
>> > > > MemFree:         1095012 kB
>> > > > MemAvailable:    1434148 kB
>> > > I suspect MemAvailable takes into account more than just the
>> > > droppable
>> > > caches. For instance, reclaimable slab is included, but I don't
>> > > think
>> > > drop_caches drops that part.
>> > There's a bit for page cache and a bit for slab, see:
>> > 
>> > 	https://kernel.org/doc/Documentation/sysctl/vm.txt
>> > 
>> > 
>> 
>> Ok, then this looks like a defect then. I would think MemAvailable
>> would 
>> always be smaller then MemFree (after echo 3 >
>> /proc/sys/vm/drop_caches).. Unless there is something else be
>> accounted 
>> for that we aren't aware of.
>
>echo 3 > /proc/sys/vm/drop_caches will only
>drop unmapped page cache, IIRC
>
>The system may still have a number of page
>cache pages left that are mapped in processes,
>but will be reclaimable if the VM needs the
>memory for something else.
>
>-- 
>All rights reversed


[-- Attachment #2: dropcacheinfo_data.pdf --]
[-- Type: application/pdf, Size: 48742 bytes --]

Re: computing drop-able caches

[Dave Hansen]

On 01/28/2016 03:42 PM, Daniel Walker wrote:
> My colleague Khalid and I are working on a patch which will provide a
> /proc file to output the size of the drop-able page cache.
> One way to implement this is to use the current drop_caches /proc
> routine, but instead of actually droping the caches just add
> up the amount.

Code, please.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Daniel Walker]

On 02/12/2016 10:06 AM, Dave Hansen wrote:
> On 01/28/2016 03:42 PM, Daniel Walker wrote:
>> My colleague Khalid and I are working on a patch which will provide a
>> /proc file to output the size of the drop-able page cache.
>> One way to implement this is to use the current drop_caches /proc
>> routine, but instead of actually droping the caches just add
>> up the amount.
> Code, please.

We have a process for release code which doesn't allow us to send it 
immediately. B

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Dave Hansen]

On 02/12/2016 10:15 AM, Daniel Walker wrote:
> On 02/12/2016 10:06 AM, Dave Hansen wrote:
>> On 01/28/2016 03:42 PM, Daniel Walker wrote:
>>> My colleague Khalid and I are working on a patch which will provide a
>>> /proc file to output the size of the drop-able page cache.
>>> One way to implement this is to use the current drop_caches /proc
>>> routine, but instead of actually droping the caches just add
>>> up the amount.
>> Code, please.
> 
> We have a process for release code which doesn't allow us to send it
> immediately. B

OK, how about we continue this discussion once you can release it?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Daniel Walker]

On 02/12/2016 10:18 AM, Dave Hansen wrote:
> On 02/12/2016 10:15 AM, Daniel Walker wrote:
>> On 02/12/2016 10:06 AM, Dave Hansen wrote:
>>> On 01/28/2016 03:42 PM, Daniel Walker wrote:
>>>> My colleague Khalid and I are working on a patch which will provide a
>>>> /proc file to output the size of the drop-able page cache.
>>>> One way to implement this is to use the current drop_caches /proc
>>>> routine, but instead of actually droping the caches just add
>>>> up the amount.
>>> Code, please.
>> We have a process for release code which doesn't allow us to send it
>> immediately. B
> OK, how about we continue this discussion once you can release it?

I understand you want to see it, and we will release it (sometime today) 
.. But the code is not sophisticated, it just counts the caches which 
would be dropped reusing much of fs/drop_caches.c .

Daniel

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Daniel Walker]

On 02/12/2016 10:18 AM, Dave Hansen wrote:
> On 02/12/2016 10:15 AM, Daniel Walker wrote:
>> On 02/12/2016 10:06 AM, Dave Hansen wrote:
>>> On 01/28/2016 03:42 PM, Daniel Walker wrote:
>>>> My colleague Khalid and I are working on a patch which will provide a
>>>> /proc file to output the size of the drop-able page cache.
>>>> One way to implement this is to use the current drop_caches /proc
>>>> routine, but instead of actually droping the caches just add
>>>> up the amount.
>>> Code, please.
>> We have a process for release code which doesn't allow us to send it
>> immediately. B
> OK, how about we continue this discussion once you can release it?

Ok, you should have the code now."kernel: fs: drop_caches: add dds 
drop_caches_count"

Daniel

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Dave Hansen]

On 02/12/2016 10:01 AM, Khalid Mughal (khalidm) wrote:
> If you look at the attached pdf, you will notice that OOM messages start
> to appear when memAvailable is showing 253MB (259228 KB) Free, memFree is
> 13.5MB (14008 KB) Free, and dropcache based calculation �Available memory�
> is showing 21MB (21720 KB) Free.
> 
> So, it appears that memAvailable is not as accurate, especially if data is
> used to warn user about system running low on memory.

Yep, that's true.

But, MemAvailable is calculated from some very cheap counters.  The
"dropcache-based-calculation" requires iterating over every 4k page
cache page in the system.

Do you have some ideas for doing cheap(er) MemAvailable calculations?

We track dirty and writebackw with counters, so we should theoretically
be able to pull those out of MemAvailable fairly cheaply.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: computing drop-able caches

[Johannes Weiner]

On Fri, Feb 12, 2016 at 01:46:54PM -0800, Dave Hansen wrote:
> On 02/12/2016 10:01 AM, Khalid Mughal (khalidm) wrote:
> > If you look at the attached pdf, you will notice that OOM messages start
> > to appear when memAvailable is showing 253MB (259228 KB) Free, memFree is
> > 13.5MB (14008 KB) Free, and dropcache based calculation �Available memory�
> > is showing 21MB (21720 KB) Free.
> > 
> > So, it appears that memAvailable is not as accurate, especially if data is
> > used to warn user about system running low on memory.
> 
> Yep, that's true.
> 
> But, MemAvailable is calculated from some very cheap counters.  The
> "dropcache-based-calculation" requires iterating over every 4k page
> cache page in the system.

It's also completely off when the dominating cache consumer uses
mmap() instead of buffered IO.

> We track dirty and writebackw with counters, so we should theoretically
> be able to pull those out of MemAvailable fairly cheaply.

Yeah, that sounds like a good idea.

But the fact remains that this will never be fully accurate, and there
will always be scenarios where the system will thrash and trigger OOM
before this counter depletes, simply because maintaining uptodate heat
information of the page cache would be crazy expensive.

IMO, the best way to use the MemAvailable counter is to calculate a
utilization percentage against MemTotal, and then maintaining a
healthy number like 80-90% - depending on expected runtime variance
and an educated guess of how hot the page cache is.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>