[PATCH] dm-raid.txt: document discard support

[PATCH] dm-raid.txt: document discard support

[heinzm]

From: Heinz Mauelshagen <heinzm@redhat.com>

For raid4/5/6 data integrity reasons, 'discard_zeroes_data' needs to work properly.

This patch ducuments the background of this critical requirement.


Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>

---
 Documentation/device-mapper/dm-raid.txt | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/Documentation/device-mapper/dm-raid.txt b/Documentation/device-mapper/dm-raid.txt
index cb12af3..597ba0a 100644
--- a/Documentation/device-mapper/dm-raid.txt
+++ b/Documentation/device-mapper/dm-raid.txt
@@ -209,6 +209,32 @@ include:
 	"repair" - Initiate a repair of the array.
 	"reshape"- Currently unsupported (-EINVAL).
 
+Discard Support
+---------------
+The implementation of discard support amoung hardware vendors varies. When
+a block is discarded, some storage devices will return zeros when it is
+read.  These devices set the 'discard_zeros_data' attribute.  Other devices
+will return random data.  Confusingly, some devices will set
+'discard_zeros_data', but will still not always return zeros when
+discarded blocks are read!  Since RAID 4/5/6 uses blocks from a number of
+devices to calculate parity blocks and relies on 'discard_zeros_data' returning
+zeroes doing so, it is important that the devices be consistent.  Blocks may be
+discarded in the middle of a RAID 4/5/6 stripe and if the read results are
+not consistent, the parity blocks may be calculated differently at any time;
+making the parity blocks useless for redundancy.  It is important to
+understand how your hardware behaves with discards if you are going
+to use RAID 4/5/6.
+
+Since the behavior of storage devices is unreliable in this respect - even
+when reporting 'discard_zeros_data' - RAID 4/5/6 does not enable discard
+support by default.  If the storage has been verified to return zeros when
+reading a discarded block, the following module parameter can be set:
+
+       'devices_handle_discards_safely'
+
+Once set, RAID 4/5/6 will enable discards and can safely use them.
+
+
 Version History
 ---------------
 1.0.0	Initial version.  Support for RAID 4/5/6
-- 
2.1.0

Re: [PATCH] dm-raid.txt: document discard support

[Martin K. Petersen]

>>>>> "Heinz" == heinzm  <heinzm@redhat.com> writes:

Heinz,

> +Discard Support
> +---------------
> +The implementation of discard support amoung hardware vendors varies. When
> +a block is discarded, some storage devices will return zeros when it is
> +read.  These devices set the 'discard_zeros_data' attribute.  Other devices
> +will return random data.  Confusingly, some devices will set
> +'discard_zeros_data', but will still not always return zeros when
> +discarded blocks are read!

This used to be correct but that's no longer the case. We only set
discard_zeroes_data when we have hard guarantees. We no longer
exclusively rely on what the device reports.

discard_zeroes_data is only set if the device reports LBPME/LBPRZ and
uses WRITE SAME 10 or 16 (SCSI) or if it reports DRAT/RZAT and is
explicitly whitelisted (ATA).

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [PATCH] dm-raid.txt: document discard support

[Heinz Mauelshagen]

On 08/08/2015 06:27 PM, Martin K. Petersen wrote:
>>>>>> "Heinz" == heinzm  <heinzm@redhat.com> writes:
> Heinz,
>
>> +Discard Support
>> +---------------
>> +The implementation of discard support amoung hardware vendors varies. When
>> +a block is discarded, some storage devices will return zeros when it is
>> +read.  These devices set the 'discard_zeros_data' attribute.  Other devices
>> +will return random data.  Confusingly, some devices will set
>> +'discard_zeros_data', but will still not always return zeros when
>> +discarded blocks are read!
> This used to be correct but that's no longer the case. We only set
> discard_zeroes_data when we have hard guarantees. We no longer
> exclusively rely on what the device reports.

Martin,

so can we rely on discard_zeroes_data if reported 100% now?

Heinz

>
> discard_zeroes_data is only set if the device reports LBPME/LBPRZ and
> uses WRITE SAME 10 or 16 (SCSI) or if it reports DRAT/RZAT and is
> explicitly whitelisted (ATA).
>

Re: [PATCH] dm-raid.txt: document discard support

[Martin K. Petersen]

>>>>> "Heinz" == Heinz Mauelshagen <heinzm@redhat.com> writes:

Heinz,

Heinz> so can we rely on discard_zeroes_data if reported 100% now?

The intent is for the reporting to be accurate, yes.

This means that for SCSI we only turn it on when using the WRITE
SAME(10/16) commands. These require the storage to physically zero out
any portions of the request that can not be successfully deallocated.

For SATA we can't trust the protocol but vendors are sometimes willing
to provide guarantees above and beyond what is in the spec. I explicitly
whitelist devices from vendors that have done so.

It appears Microsoft is as disenchanted with the standards bodies as we
are and are now requiring vendors to provide hard guarantees. As a
result we should be able to leverage Windows qualifications going
forward.

That said, all storage is buggy so there is always some risk involved.
But as of 3.19, discard_zeroes_data=1 has much stricter requirements
than it did in prior kernels.

-- 
Martin K. Petersen	Oracle Linux Engineering