Re: [PATCH RFC 00/30] Ext4 snapshots - core patches

[Ted Ts'o <tytso@mit.edu>]

On Mon, Jun 06, 2011 at 10:31:33AM -0500, Eric Sandeen wrote:
> > For one reason, a snapshot file format is currently an indirect file
> > and big_alloc
> > doesn't support indirect mapped files.
> > I am not saying it cannot be done, but if it does, there would be
> > several obstacles
> > to cross.
> 
> I know I'm kind of just throwing a bomb out here, but I am very concerned
> about the ever-growing feature (in)compatibility matrix in ext4.

bigalloc doesn't support indirect blocks mainly because it was faster
to get things working if I didn't have to worry about indirect blocks.
It wouldn't be _that_ hard to make bigalloc work on indirect blocks.
I'll get around to it at some point.

dioread_nolock is something that I had hoped to clean up by now, by
making this the default way we do all buffered writebacks, for all
block sizes.

> Take for example dioread_nolock caveats:
> 
>           "However this does not work with nobh
>            option and the mount will fail. Nor does it work with
>            data journaling and dioread_nolock option will be
>            ignored with kernel warning. Note that dioread_nolock
>            code path is only used for extent-based files."

Hey, at least we got rid of nobh!  :-)

> If ext4 matches the lifespan of ext3, in 10 years I fear that it will look
> more like a collection of various individuals' pet projects, rather than
> any kind of well-designed, cohesive project.
> 
> How long can we really keep adding features which are semi- or wholly-
> incompatible with other features?
> 
> Consider this a cry in the wilderness for less rushed feature introduction,
> and a more holistic approach to ext4 design...

It's something I do worry about; and I do share your concern.  At the
same time, the reality is that we are a little like the Old Dutch
Masters, who had take into account the preference of their patrons
(i.e., in our case, those who pay our paychecks :-).  

In the case of dioread_nolock, I allowed dioread_nolock in even though
it was a not a complete solution since internally, we had critical
business for it, and in my judgement, (a) it wasn't that horrible
(most of the horrible code paths was already being used for AIO/DIO),
and (b) I had a plan for how to clean it up eventually.  The
fs/ext4/page_io.c implementation was in fact the first part of my
cleanup plan, so we've made some progress; it's just not gone as fast
as I would like.

Snapshots are an example of a feature where I am very much worried
about taking on technical debt.  On the other hand, there are a lot of
people who are quite excited of it as a feature, so I'm hoping we can
clean it up enough we don't put a huge maintenance burden on
ourselves.

It should be possible to make snapshots work on bigalloc file systems,
once support is added for indirect blocks.  The COW granulaity will
have to be done at the cluster level, of course, though.  So from a
design perspective it should be possible to make things knit together.

       		      	     		    - Ted