Re: T10 WCE interpretation in Linux & device level access

[James Bottomley <James.Bottomley@HansenPartnership.com>]

On Tue, 2013-04-23 at 15:41 -0400, Ric Wheeler wrote:
> For many years, we have used WCE as an indication that a device has a volatile 
> write cache (not just a write cache) and used this as a trigger to send down 
> SYNCHRONIZE_CACHE commands as needed.
> 
> Some arrays with non-volatile cache seem to have WCE set and simply ignore the 
> command.

I bet they don't; they probably obey the spec.  There's a SYNC_NV bit
which if unset (which it is in our implementation) means only sync your
non-NV cache.  For a device with all NV, that equates to nop.

> Some arrays with non-volatile cache seem to not set WCE.
> 
> Others arrays with non-volatile cache - our problem arrays - set WCE and do 
> something horrible and slow when sent the SYNCHRONIZE_CACHE commands.

These arrays sound to be out of spec, so we should probably just
blacklist them.

> Note that for file systems, you can override this behavior by mounting with our 
> barriers disabled (mount -o nobarrier .....). There is currently no way do 
> disable this for anything using the device directly, not through the file system.
> 
> Some applications run against block devices - not through a file system - and 
> want not to slow to a crawl when they have an array in my problem set.
> 
> Giving them a hook to ignore WCE seems to be a hack, but one that would resolve 
> issues with users who won't want to wait months (years?) for us to convince the 
> array vendors.
> 
> Is this a hook worth doing?

We already have the ability to set the cache type in sysfs.  It tries to
do a mode select back to the array, but the USB guys want it for the
reverse problem (write back cache behind bridge declared as write
through).

> Have we hashed this out in the T10 committee?

SBC-3 contains everything one could wish for about handling devices with
volatile and NV cache, I thought.

James