Re: Large buffer cache in EXT4

[Eric Sandeen <sandeen@redhat.com>]

On 2/17/13 12:28 AM, Andreas Dilger wrote:
> On 2013-02-16, at 21:04, Subranshu Patel <spatel.ml@gmail.com> wrote:
> 
>> I created 2 filesystem on my system (RHEL 6.3, kernel version 2.6.32)
>> - XFS and EXT4 and mounted them.
>>
>> On both the filesystem I executed the mdtest tool(opensource tool) for
>> 64 concurrent process. Each process performed the following:
>> - Create large number of directories
>> - Remove all the directories
>>
>> During this time I monitored the memory usage of the system using sar
>> command. I checked the 3 components - kbmemused, kbbuffers and
>> kbcached
>>
>> kbmemused - Amount of used memory in kilobytes. This does not take
>> into account memory used by the kernel itself.
>> kbbuffers - buffer cache
>> kbcached - page cache
>>
>> While the kbmemused and kbcached component was almost similar in EXT4
>> and XFS (XFS being a little higher), the kbbuffer showed a totally
>> different trend.
>>
>> For EXT4, kbbuffers was:
>> 390999KB for dir creation
>> 364803KB for dir removal
>> For XFS, kbbuffers was:
>>
>> 1701KB for dir creation
>> 2738KB for dir removal
>>
>> In kernel 2.6, both buffer cache and page cache are merged. The page
>> cache caches pages of files. The buffer cache caches disk blocks which
>> consists of mainly metadata (not file data).
>>
>> Why is the buffer cache large in case of EXT4 and what is stored in
>> the buffer cache?
> 
> XFS does not use buffer cache, while ext[234] does use buffer cache. 
> 
> This is just a different code design. Ext4 uses the buffer cache to track metadata for journaling. 

Use slabtop or similar to see what xfs slab caches grow during the test; look at xfs_buf for example.

-Eric