[RFC] Peeling off dcache_lock

[Maneesh Soni <maneesh@in.ibm.com>]

Hi All,

We have been doing experiments with dcache_lock to provide some relief from it.
Though dcache_lock is not a very hot lock in comparision to BKL but on higher
end machines it becomes quite contentious. We would like to have feedbacks,
comments about the approach taken and guidance on how to improve this further.

As dcache_lock is acquired maximum number of times while doing lookup, the main
idea was to remove dcache_lock from d_lookup. With the help of Read Copy Update
it is possible to lookup the hash list of dentries without taking dcache_lock
but as d_lookup also updates the lru list, removing dcache_lock from d_lookup
was not straight forward. Various approaches were tried for that and the most
successful is doing lazy updation of lru list. 

Using this the dcache_lock contention was down from 16.5% to 0.95% on an 
8-way SMP box and lock utilization was down from 5.3% to 0.89%

The patch for lazy lru updation using RCU can be found here:

http://lse.sourceforge.net/locking/dcache/patches/2.4.17/dcache_rcu-lazy_lru-2.4.17-06.patch

The basic conecpt for this approach is to not update the lru list in d_lookup
facilitating lockless d_lookup. The lru list can now have dentries with non zero
reference count also. The lru list is updated while pruning dentry cache. The
dcache_lock is kept around so that there are no changes in the update side 
locking.

The implementation details for lazy lru list updation and other approaches
can be found here:

http://lse.sourceforge.net/locking/dcache/dcache_lock.html 

The above patch works fine with various file systems like ext2, JFS, ext3, /procand has been tested ok with ltp-20020108, dbench, httperf. And doesnot not have
any adverse effects on uni processor performance.


Regards,
Maneesh


-- 
Maneesh Soni
IBM Linux Technology Center, 
IBM India Software Lab, Bangalore.
Phone: +91-80-5044999 email: maneesh@in.ibm.com
http://lse.sourceforge.net/locking/rcupdate.html