Comparing buffer cache algorithms on 2.2.17. Suggestions?

Comparing buffer cache algorithms on 2.2.17. Suggestions?

[Fireball Freddy]

Howdy,

  Trying to implement some different buffer caching
algorithms in Linux.  This is just for comparison
purposes for a thesis, not suggesting any problem with
the current scheme.  Here is what I'm attempting:

  o Eliminate BUF_CLEAN, BUF_DIRTY, and BUF_LOCKED
lists in favor of a single BUF_LRU list.  This because
I don't see the point of maintaining three lists...
the only time I need to find all the dirty blocks is
on a sync of some sort.  I don't mind if the sync
takes a while longer if the normal operating condition
is faster.

  Once I have the cache working on a single list
(assuming I can!) I plan on using an I/O generator to
get some repeatable traces so I can compare the
following:

  o LRU vs SLRU with static region percentages vs SLRU
with dynamic region percentages
  o Periodic flushing of dirty blocks enabled vs
disabled
  o Write-through caching vs write-back caching vs
adaptive (switching between wt and wb based on recent
activity) vs my own method (write to disk on first
write, only to buffer after that, write to disk again
when block is flushed, if dirty)
  o Of course, I'll also do a run with the default
Linux 2.2.17 to see how it compares with the others

  It looks like the ext2 fs is going to complicate
this somewhat, as it sets blocks dirty and/or writes
them itself sometimes.  Not sure how I'm going to get
around this... will probably ignore it for now.

  I'd appreciate any advice on this undertaking. 
Specifically, how many things (tools, etc) is this
going to break, warnings about common pitfalls, and
other suggestions.  I would like to have started this
on 2.4, but I wanted to work on a release that had
been out for a bit, so I'd know that any problems were
caused by *me* and not just kernel bugs.  :)

  Also, are there any specific people to whom I should
direct memory management questions?  I've had some in
the past few months but haven't been able to get
answers.  Or should I post them to this list (I assume
not).  Finally, I have noticed references to a
kernelnewbies chat room.  I prefer newsgroups to
chat... is there a good mailing list devoted to kernel
newbies?

  Thanks to everyone for your hard work, and apologies
for adding one more e-mail to the pile.

    Andy Cottrell

__________________________________________________
Do You Yahoo!?
Get personalized email addresses from Yahoo! Mail - only $35 
a year!  http://personal.mail.yahoo.com/

Re: Comparing buffer cache algorithms on 2.2.17. Suggestions?

[Stephen Tweedie]

Hi,

On Sat, Feb 17, 2001 at 02:15:40PM -0800, Fireball Freddy wrote:
> 
>   o Eliminate BUF_CLEAN, BUF_DIRTY, and BUF_LOCKED
> lists in favor of a single BUF_LRU list.  This because
> I don't see the point of maintaining three lists...
> the only time I need to find all the dirty blocks is
> on a sync of some sort.  I don't mind if the sync
> takes a while longer if the normal operating condition
> is faster.

What if you have a gig of buffer cache in the system?  You still need
to maintain the single LRU, so you're not going to be saving that much
on the main codepaths, and walking a gig of unnecessary buffer_heads
every kupdate() flush is not going to be cheap.

>   It looks like the ext2 fs is going to complicate
> this somewhat, as it sets blocks dirty and/or writes
> them itself sometimes.  Not sure how I'm going to get
> around this... will probably ignore it for now.

Yep, this behaviour on the superblock and group descriptors has hurt
people in the past.

Cheers,
 Stephen