Re: Re: Reviewing ext3 improvement patches (delalloc, mballoc, extents)

[Suparna Bhattacharya <suparna@in.ibm.com>]

On Thu, Mar 03, 2005 at 02:40:21AM -0700, Andreas Dilger wrote:
> On Mar 03, 2005  14:03 +0530, Suparna Bhattacharya wrote:
> > diffstat of the 3 patches : 22 files changed, 5920 insertions(+), 
> > 47 deletions. The largest is in the extents patch (2743), mballoc 
> > is 1968, and delalloc is 1209. To use delalloc, which gives us
> > all the performance benefits, right now we need all the 3 patches
> > to be used in conjunction. Supporting extent map btrees as well 
> > as traditional indexing and associated options for compatibility etc
> > is perhaps the more invasive of changes. Given that keeping ext3 
> > stable and maintainable is a key concern (that is after all a major 
> > reason why a lot of users rely on ext3), a somewhat incremental 
> > approach is desirable. 
> > 
> > So, I'll start from the direction that has been suggested by
> > some -- (1) delayed allocation without changing the
> > on-disk format. And then later (2) go on to breaking format with 
> > all changes for scalability to larger files with full extents 
> > support (haven't thought enough about this yet - maybe in a
> > separate mail)
> 
> Well, for a starter, the extents format changes are not forced on
> users, only if they mount with "-o extents" and write files will
> it mark the superblock incompatible and start allocating files
> this way.  I believe (though I have never tested) that even if
> extents are enabled, writes to a block-mapped file will continue
> to work and that file will not be converted to an extent file.

Files that are created with extents will not be viewable by an older
kernel, though (I think) - which is where the format breakage comes
in (is that correct ?). But I don't see this as a major issue, since 
it can perhaps be taken care of through a little bit of migration 
tooling as Ted indicated. 

So, compatibility in itself wasn't the main concern bothering me 
but how we could make it easier to assure stability & maintainability
even with all the cool stuff. For example, if we have both mballoc 
and regular balloc and similarly extents and regular indexing based 
on growth patterns (a nice idea, btw), does it multiply the 
scenarios to verify on the testing front ? Or in dealing with changes
in the future ? I'm guessing that this might be one of the things (besides
agreement on the disk layout) holding up inclusion of extents, despite
the patches being around for a while now .. but then I could be wrong.
B-tree based extent maps were mentioned by sct way back in his 2000 
paper ! And of course every filesystem out there implements B-trees in
its own way.

I can see arguments flying both ways ... at what point do we decide
to break towards an ext4 ? 

BTW, has anyone tried playing with the idea of ext4 as not a 
cp -r fs/ext3 fs/ext4 and edit, but if possible using some layered
filesystem techniques to reuse much of ext3 directly, and just override
a few operations (like get_blocks for extents etc) where there 
is a layout impact ? 

Alex, have you had a chance to prototype your idea of rooting extents
in ea ?

> 
> > A few random things that come to mind for (1), going through the code:
> > 
> > - There might be possibilities for code reduction, by extending
> >   generic routines as far as possible, e.g. ext3_wb_writepages
> >   has a lot in common with generic writepages. That would
> >   also make it easier to maintain.
> 
> I'm sure some support for this could be gotten from e.g. XFS as well,
> since their filesystem (on Irix at least) was all about delayed alloc
> (not sure what it does under Linux), and I believe ReiserFS/Reiser4
> also desire the ability to have delayed allocation from the VFS (i.e.
> some sort of light-weight "reserve space" call for each page dirtied
> and then getting the actual file + offsets en masse later (if the
> VFS/VM doesn't discard the whole thing).

*nod*

Regards
Suparna

> 
> Cheers, Andreas
> --
> Andreas Dilger
> http://sourceforge.net/projects/ext2resize/
> http://members.shaw.ca/adilger/             http://members.shaw.ca/golinux/
> 



-- 
Suparna Bhattacharya (suparna@in.ibm.com)
Linux Technology Center
IBM Software Lab, India



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click