Buffer cache not working.

Buffer cache not working.

[Rogier Wolff]

Hi Guys, 

I have a file-server that has plenty of memory to cache most of the
things that I use. But after I've upgraded the machine it seems to
expire things from the cache even when there is zero memory pressure.

The case that irritates me most is my mailbox. Ok, I should clean it
out, but also my linux-kernel mailbox has 20k messages (= 1 month).

When I open such a mailbox in mutt, it takes 2 minutes for it to read
all the messages. If I quit and restart, it takes less than a second. 
I've resorted to a workaround: every 30 seconds: head Mailbox/cur/* . 
That worked. 

Oh, before a recent upgrade all this worked as I expected: Once a day,
after the backup had run, scanning the mailbox takes a while and then
the rest of the day things are quick. 

Then I gathered: Other people must have the same problem. It is
probably a "tuning" setting which works for desktop machines, but not
for my server. So I've googled a few "tuning your Linux system" pages
and all I could think of that might relate to my problem is
"swappiness".

I've decreased the value from 100 (in a few steps) to 5 now, and this
seems to help somewhat. But still after a few minutes, the buffer
cache drops my maildir, and starting mutt takes a long time again.

All this with the machine reporting 11Gb or more of RAM "Free"
(empty). 

With the default swappiness, I've tested that in a minute already some
of the maildir gets ejected from the cache, while when accessing all
files every 30 seconds seems to keep them cached. 

Kernel: from Ubuntu 14.04, 3.13.0-19-generic
32-bit userspace
16Gb RAM installed. 
Disks: 3x 3T in RAID5

ganglia shows the server memory graph as follows: 
http://prive.bitwizard.nl/server_mem.png

All other computers have a "nice" filled graph there.

So, is there a bug in my kernel or am I missing an important tuning
parameter?


	Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
**    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    **
*-- BitWizard writes Linux device drivers for any device you may have! --*
The plan was simple, like my brother-in-law Phil. But unlike
Phil, this plan just might work.

Re: Buffer cache not working.

[Michal Hocko]

On Wed 14-05-14 16:46:32, Rogier Wolff wrote:
[...]
> Kernel: from Ubuntu 14.04, 3.13.0-19-generic

The kernel is 64b right?

> 32-bit userspace
> 16Gb RAM installed. 
> Disks: 3x 3T in RAID5
> 
> ganglia shows the server memory graph as follows: 
> http://prive.bitwizard.nl/server_mem.png

This shows that the memory usage didn't go over 5G. That is suspicious.
Is this a NUMA machine? Is it possible that zone_reclaim_mode is enabled
(cat /proc/sys/vm/zone_reclaim_mode)?
 
> All other computers have a "nice" filled graph there.
> 
> So, is there a bug in my kernel or am I missing an important tuning
> parameter?
> 
> 
> 	Roger. 
> 
> -- 
> ** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
> **    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    **
> *-- BitWizard writes Linux device drivers for any device you may have! --*
> The plan was simple, like my brother-in-law Phil. But unlike
> Phil, this plan just might work.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Michal Hocko
SUSE Labs