Re: How does NILFS2 handle directory management

[Reinoud Zandijk <reinoud-S783fYmB3Ccdnm+yROfE0A@public.gmane.org>]

Hi Ryusuke,

On Sat, Sep 26, 2009 at 12:47:30AM +0900, Ryusuke Konishi wrote:
> > So its basicly the same aproach as mine only with the disadvantage
> > that the hash is stored on-disc and hacked into the directory
> > entries.  That explains the deletion penelty to be paid since the
> > has to be updated. One could cheat by not updating the hash on disc
> > on deletion since its result will backfire on lookup but thats
> > silly.
> 
> You have a point.  Ext3 inserts intermediate structures on directory
> files in somewhat tricky way.  So, it would suffer penalty for
> deletion.  It's actually one of my concern for this approach.

The Ext3 approach also is hacky and shoe-horned. How can it even see if the
hash table is still OK? an Ext2 system could have radically reformed and
reformatted the dirents.

> Well, I think your on-memory hash scheme is a hopeful candidate.
> What kind of on-memory structures are used in your implementation?
> 
> I'd like to know how to rebuild your approach on Linux basis.  If
> possible, I'd like to compare patches including the ext3 approach with
> benchmarks.
> 
...
> So, it uses 24/28 bytes on-memory entry per valid on-disk dirent.  How
> about lifetime of the cache entries?  Are the entries designed to be
> left on memory until they get shrunk by memory pressure?
> Or flushed out in LRU ?

The maximum memory used for entries is run-time configurable but normally say
1Mb, but 2Mb could easily be overdone. All cache entries are associated with a
directory and are created/destroyed on a directory basis(!) to keep the
lookup/creation/deletion authorative. The directories are LRU replaced and
least used directories caches are freed to satisfy the maximum memory usage.
One could also go for the largest one from the least used but thats a choice.

I'll mail the source-snippets to you privately; but thats'll have to wait till
tomorrow. Please do remind me if you haven't recieved them!

With regards,
Reinoud Zandijk