Re: [BUG,PATCH] raid1 behind write ordering (barrier) protection

[Brett Russ <brett@linux.vnet.ibm.com>]

On 12/02/2013 06:08 PM, NeilBrown wrote:
> How about just keeping a record of whether there is a BIO_FLUSH request
> outstanding on each "behind" leg.  While there is we don't submit new
> requests.
> So we have a queue of bios for each leg which are waiting for a BIO_FLUSH to
> complete, and we send them on down as soon as it does.

In these circumstances, it's MD who's created the situation, not an upper 
layer's BIO_FLUSH.  So, we can't key off of that.  Additionally, the patch below 
also fixes another issue related to BIO_FLUSH:

 >>> +    /* If this is a flush/fua request don't
 >>> +     * ever let it go "behind".  Keep all the
 >>> +     * mirrors in sync.
 >>> +     */
 >>> +    if (bio_rw_flagged(bio, BIO_FLUSH | BIO_FUA)) {
 >>> +        set_bit(R1BIO_BehindIO, &r1_bio->state);
 >>> +        do_flush_fua =  bio->bi_rw & (BIO_FLUSH | BIO_FUA);
 >>> +    }

so we avoid the BIO_FLUSH "behind" issue that way.  This probably should be a 
separate patch...

We could divide the behind write ordering problem into two:
1) detecting the condition to protect
2) protecting against that condition

Solutions for (1) include:
a) keeping a list of behind writes
b) keeping a count of behind writes
c) ?

Solutions for (2) include:
i) blocking the I/O
j) ?

The advantages to solution (a) are:
-nothing gets blocked unless it overlaps (previously all reads would)
-list depth limited to max behind writes allowed (typically small)

I wish there were alternatives to solution (i) but recognize that since barriers 
were removed in favor of the filesystem owning the ordering problem, MD is 
effectively assuming the role of the filesystem in this case.

Thanks,
BR