Re: T10 WCE interpretation in Linux & device level access

[Hannes Reinecke <hare@suse.de>]

On 04/23/2013 10:07 PM, James Bottomley wrote:
> On Tue, 2013-04-23 at 15:41 -0400, Ric Wheeler wrote:
>> For many years, we have used WCE as an indication that a device has a volatile 
>> write cache (not just a write cache) and used this as a trigger to send down 
>> SYNCHRONIZE_CACHE commands as needed.
>>
>> Some arrays with non-volatile cache seem to have WCE set and simply ignore the 
>> command.
> 
> I bet they don't; they probably obey the spec.  There's a SYNC_NV bit
> which if unset (which it is in our implementation) means only sync your
> non-NV cache.  For a device with all NV, that equates to nop.
> 
>> Some arrays with non-volatile cache seem to not set WCE.
>>
>> Others arrays with non-volatile cache - our problem arrays - set WCE and do 
>> something horrible and slow when sent the SYNCHRONIZE_CACHE commands.
> 
> These arrays sound to be out of spec, so we should probably just
> blacklist them.
> 
Don't think so.
There is no time limit for the SYNCHRONIZE_CACHE command, so the
array might take all day to write out the cache.

In fact, I've recently had a rather heated discussion with a certain
storage vendor which reserved his right to take up to several
seconds when flushing the cache.
Might be that we're in fact talking about the same one ... are we on
a naming-and-shaming policy here ?
If so I could tell you some really _interesting_ details about their
behaviour ...

Also note that SYNCHRONIZE_CACHE was always problematic; and as
we're not even setting the LBA range we're even have cross-speak
when issuing it to partitioned devices.

>> Note that for file systems, you can override this behavior by mounting with our 
>> barriers disabled (mount -o nobarrier .....). There is currently no way do 
>> disable this for anything using the device directly, not through the file system.
>>
>> Some applications run against block devices - not through a file system - and 
>> want not to slow to a crawl when they have an array in my problem set.
>>
>> Giving them a hook to ignore WCE seems to be a hack, but one that would resolve 
>> issues with users who won't want to wait months (years?) for us to convince the 
>> array vendors.
>>
>> Is this a hook worth doing?
> 
> We already have the ability to set the cache type in sysfs.  It tries to
> do a mode select back to the array, but the USB guys want it for the
> reverse problem (write back cache behind bridge declared as write
> through).
> 
You can always set the 'IMMED' bit for these arrays :-)

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html