Re: [PATCH] ide write barrier support

[Daniel Phillips <phillips@arcor.de>]

On Friday 24 October 2003 11:36, Helge Hafting wrote:
> On Thu, Oct 23, 2003 at 07:20:39PM +0200, Daniel Phillips wrote:
> > These are essentially the same, they both rely on draining the downstream
> > queues.  But if we could keep the downstream queues full, bus transfers
> > for post-barrier writes will overlap the media transfers for pre-barrier
> > writes, which would seem to be worth some extra effort.
> >
> > To keep the downstream queues full, we must submit write barriers to all
> > the downstream devices and not wait for completion.  That is, as soon as
> > a barrier is issued to a given downstream device we can start passing
> > through post-barrier writes to it.
>
> This approach may fail:
>
> a. Some pre-barrier writes go to all devices
> b. barrier is sent to all devices
> c. Post-barrier writes go to all devices
> d. drive 1 commits all its pre-barrier writes, then
>    commits its post-barrier writes.
> e. drive 2 is slow and havent done the pre-barrier writes yet
> f. power is lost - leaving inconsistent devices.
>
> The problem is that drive 1 don't know wether drive 2
> did the barrier yet.

I was originally talking about SCSI multipath where more than one host adapter 
issues commands to the same drive, however this idea works for M adapters 
connected to N disks as well.  Several barriers sent down different paths 
share a count of the number of paths; on receiving a barrier, a driver 
decrements the count and if it is nonzero it blocks; if zero it unblocks the 
other drivers and each driver submits a barrier to its respective device 
before resuming normal processing.

Under balanced load this will keep all the device queues full, and it should 
be clear that there is no hole in this multi-device barrier for a write to 
tunnel through.  This strategy does however require some mechanism that isn't 
currently present in the barrier API.  I agree with Jens that for now the 
easiest thing to do is to block at the point of the multipath virtual device 
and allow the queues to drain.

The real purpose of this line of thinking was to see whether barriers want to 
be a third type of request, distinct from read or write.  This gets away from 
arbitrary tying of the barrier to a specific IO request, which might work out 
ok in some usages but leads to awkward posturing in others.  As a bonus, it 
gets rid of the question of how many bits to reserve for barrier type and 
makes it easy to set aside fields for such a purpose as a multi-queue 
barrier, as described above.  The cost is that the barrier needs to be 
submitted as a seperate request, which is not a big deal.

Overall, it's conceptually correct to treat a barrier as a separator rather 
than as a property of some other request.

Regards,

Daniel