Re: ext5

[tytso@mit.edu]

On Thu, Feb 11, 2010 at 09:08:36AM +0530, Manish Katiyar wrote:
> 
> Is this design somewhere on net so that we can read/lookup it up ?

Not yet, but the basic idea is to do the compression in userspace,
using libz with regular resync points every 64k or 128k of
uncompressed data.  An array, indexed by each fixed-block of
uncompressed data, is located at the beginning of the file indicating
where each compressed block begins.  The file is stored on-disk
written by the installer in a compressed format, and then the
installer flips an attribute bit which marks the file as containing
compressed data, and which also makes the file immutable.  (Any
attempt to open the file read/write will result in an error.)

The advantage of this scheme is that it minimizes in-kernel
complexity, since the kernel doesn't have to deal with compressing
data, and it means we can use a simpler format which is more efficient
from a compression standpoint since we don't have to deal with random
access writes.

Another advantage of this scheme is that it's relatively easy to do
the bulk of the work in a file system-independent layer.  Some minor
work would be needed to interface the attribute bit and the mmap
reading function into each client filesystem, but it should be
possible to isolate 95% of the work into a filesystem independent
layer, and then make this something which multiple file systems could
take advantage of --- not just ext3 and ext4, but also xfs, btrfs,
etc.

						- Ted