Re: T10 WCE interpretation in Linux & device level access

[Mike Snitzer <snitzer@redhat.com>]

On Wed, Apr 24 2013 at  8:12am -0400,
Hannes Reinecke <hare@suse.de> wrote:

> On 04/24/2013 02:08 PM, Paolo Bonzini wrote:
> > Il 24/04/2013 14:07, Hannes Reinecke ha scritto:
> >> On 04/24/2013 01:17 PM, Paolo Bonzini wrote:
> >>> Il 23/04/2013 22:07, James Bottomley ha scritto:
> >>>> On Tue, 2013-04-23 at 15:41 -0400, Ric Wheeler wrote:
> >>>>> For many years, we have used WCE as an indication that a device has a volatile 
> >>>>> write cache (not just a write cache) and used this as a trigger to send down 
> >>>>> SYNCHRONIZE_CACHE commands as needed.
> >>>>>
> >>>>> Some arrays with non-volatile cache seem to have WCE set and simply ignore the 
> >>>>> command.
> >>>>
> >>>> I bet they don't; they probably obey the spec.  There's a SYNC_NV bit
> >>>> which if unset (which it is in our implementation) means only sync your
> >>>> non-NV cache.  For a device with all NV, that equates to nop.
> >>>
> >>> Isn't it the other way round?
> >>>
> >>> SYNC_NV = 0 means "sync all your caches to the medium", and it's what we do.
> >>>
> >>> SYNC_NV = 1 means "sync volatile to non-volatile", and it's what Ric wants.
> >>>
> >>> So we should set SYNC_NV=1 if NV_SUP is set, perhaps only if the medium
> >>> is non-removable just to err on the safe side.
> >>
> >> Or use 'WRITE_AND_VERIFY' here; that's guaranteed to hit the disk.
> >> Plus it even has a guarantee about data consistency on the disk,
> >> which the normal WRITE command has not.
> > 
> > The point is to _avoid_ hitting the disk. :)
> > 
> Ah. Really?
> 
> Why do we discuss SYNCHRONIZE CACHE then?
> I was under the impression that we're talking various 'barriers'
> (or rather 'flush' nowadays) implementations.
> Which require that some data needs to be written to disk before
> continuing.
> 
> Or did I somehow misread the thread?

This thread was motivated by the fact that the storage is reporting
WCE=1 and OracleDB (with ASM) is issuing SYNCHRONIZE CACHE (via
REQ_FLUSH) which the array in question handles _very_ slowly (even
though it is battery backed).

So the question Ric had is: should we expose a new knob that allows
admins to impose WCE=0 behavior (avoiding the SYNCHRONIZE CACHE).

I'm concerned such a knob will be abused for the benefit of speed and
all data integrity caution will get thrown to the wind (much like the
nobarrier FS mount option).

Mike