Re: [PATCH 1/6] block: add support for REQ_OP_VERIFY

[Chaitanya Kulkarni <chaitanyak@nvidia.com>]

On 7/5/22 01:34, Christoph Hellwig wrote:
> On Thu, Jun 30, 2022 at 02:14:01AM -0700, Chaitanya Kulkarni wrote:
>> This adds a new block layer operation to offload verifying a range of
>> LBAs. This support is needed in order to provide file systems and
>> fabrics, kernel components to offload LBA verification when it is
>> supported by the hardware controller. In case hardware offloading is
>> not supported then we provide API to emulate the same. The prominent
>> example of that is SCSI and NVMe Verify command. We also provide
>> an emulation of the same operation that can be used in case H/W does
>> not support verify. This is still useful when block device is remotely
>> attached e.g. using NVMeOF.
> 
> What is the point of providing the offload?

Data block verification is done at the time of file-scrubbing
e.g. see [1], having support to offload the verify command will :-

1. Reduce the DMA transfer at the time of scrubbing :-

In the absense of verify command user has to send the read
command that will trigger same behaviour as verify command, but
with the DMA traffic with operating system storage stack overhead
of REQ_OP_READ. This overhead gets duplicated for the fabrics
controller where host and target now has to issue REQ_OP_READ
leading to significant DMA transfer as compare to REQ_OP_VERIFY
for each protocol SCSI/NVMe etc. This makes it possible to do
a low-level scrub of the stored data without being bottlenecked
by the host interface bandwidth.

2. Allow us to use to unify interface for applications :-

Currently in linux there is no unified interface to issue
verify command so each application will have to duplicate the
code for the discovering controllers protocol type, opencoding
device passthru ioctl for protocol spefcific verify command and
issuing verify read emulation if it is not supported, see [1].

3. Allow us to use controller's internal bandwidth :-

For some controllers offloading the verify command can reault in
decrease in data block verification time, since their internal
bandwidth can be higher than host DMA transfer + OS storage stack
overhead of the read command.

4. Pro-actively avoiding unrecoverable read errors:-

Verify command does everything a normal read command does, except
for returning the data to the host system. This makes it possible
to do a low-level scrub of the stored data without being
bottlenecked by the host interface bandwidth.

Please note that analyzing controller verify command performance
for common protocols (SCSI/NVMe) is out of scope for REQ_OP_VERIFY.

-ck

[1] xfs_scrub issueing the verify command :-
xfs-progs/scrub/disk.c 340: disk_read_verify()->disk_scsi_verify()