Re: Mis-Design of Btrfs?

[Chris Mason <chris.mason@oracle.com>]

Excerpts from Ric Wheeler's message of 2011-07-15 09:31:37 -0400:
> On 07/15/2011 02:20 PM, Chris Mason wrote:
> > Excerpts from Ric Wheeler's message of 2011-07-15 08:58:04 -0400:
> >> On 07/15/2011 12:34 PM, Chris Mason wrote:
> > [ triggering IO retries on failed crc or other checks ]
> >
> >>> But, maybe the whole btrfs model is backwards for a generic layer.
> >>> Instead of sending down ios and testing when they come back, we could
> >>> just set a verification function (or stack of them?).
> >>>
> >>> For metadata, btrfs compares the crc and a few other fields of the
> >>> metadata block, so we can easily add a compare function pointer and a
> >>> void * to pass in.
> >>>
> >>> The problem is the crc can take a lot of CPU, so btrfs kicks it off to
> >>> threading pools so saturate all the cpus on the box.  But there's no
> >>> reason we can't make that available lower down.
> >>>
> >>> If we pushed the verification down, the retries could bubble up the
> >>> stack instead of the other way around.
> >>>
> >>> -chris
> >> I do like the idea of having the ability to do the verification and retries down
> >> the stack where you actually have the most context to figure out what is possible...
> >>
> >> Why would you need to bubble back up anything other than an error when all
> >> retries have failed?
> > By bubble up I mean that if you have multiple layers capable of doing
> > retries, the lowest levels would retry first.  Basically by the time we
> > get an -EIO_ALREADY_RETRIED we know there's nothing that lower level can
> > do to help.
> >
> > -chris
> 
> Absolutely sounds like the most sane way to go to me, thanks!
> 

It really seemed like a good idea, but I just realized it doesn't work
well when parts of the stack transform the data.

Picture dm-crypt on top of raid1.  If raid1 is responsible for the
crc retries, there's no way to crc the data because it needs to be
decrypted first.

I think the raided dm-crypt config is much more common (and interesting)
than multiple layers that can retry for other reasons (raid1 on top of
raid10?)

In other words, do we really want to do a lot of design work for
multiple layers where each one maintains multiple copies of the data
blocks?  Are there configs where this really makes sense?

-chris