Re: [PATCH 1/1] verify: move meta header to generic verify_header

[Jens Axboe <axboe@kernel.dk>]

On 07/24/2015 01:49 PM, Roman Pen wrote:
> fio does not provide any possibility to verify checksum of a block with meta
> information inside. You can create configuration for verifincation checksum
> of random data either you can verify meta information with some pattern or
> random data, but not both.
>
> Why checksumming and meta together can be useful? Meta helps to figure out internally
> on filesystem or storage what block was written in case of corruption, i.e. offset
> of the block and block number explicitly tell us the virtual address of the block.
> On the other hand checksum of random data helps to detect corruption. Using meta
> and pattern together do not help a lot, since 'verify_interval' can be big enough
> and same sequence of pattern bytes will be undistinguishable internally on filesystem
> or storage.
>
> Also, it seems to me that keeping meta header separately from generic verify header
> does not make a lot of sense, since generic verify header can include all members
> of meta header without any performance or other impact.
>
> In this patch I move all members from vhdr_meta structure to generic verify_header,
> always verifying meta with the possiblity to checksum the following data: random
> or pattern.
>
> You are allowed to specify verify_pattern=str with any of the possible verification
> methods and have also meta verification, i.e.
>
>     verify=md5
>     verify_pattern=0xfe
>
>   or
>
>     verify=sha1
>     verify_pattern=0xff
>
>   etc.
>
> To keep everything compatible with old configurations it is still possible to specify
>
>     verify=meta
>
> but this option marked and depricated and kept only for compatibility reasons.
>
> Before that patch the verification layout according to the specified options looks
> as the following, e.g.:
>
>   #1
>      --
>      verify=meta
>      verify_pattern=0xff
>      --
>
>      result layout of each block: [hdr|meta|pattern]
>
>   #2
>      --
>      verify_pattern=0xff
>      --
>
>      result layout of each block: [hdr|pattern]
>
>   #3
>      --
>      verify=pattern
>      verify_pattern=0xff
>      --
>
>      result layout of each block: [pattern]
>
> After applying of the patch 'vhdr_meta' is always embedded into 'verify_header' and layout
> looks as the following, e.g.:
>
>   #1
>      --
>      verify=meta
>      verify_pattern=0xff
>      --
>
>      result layout of each block: [hdr+meta|pattern]
>   #2
>      --
>      verify=md5|sha1|etc
>      verify_pattern=0xff
>      --
>
>      result layout of each block: [hdr+meta|cksum|pattern]
>
>   #3
>      --
>      verify_pattern=0xff
>      --
>
>      result layout of each block: [hdr+meta|pattern]
>
>   #4
>      --
>      verify=pattern
>      verify_pattern=0xff
>      --
>
>      result layout of each block: [pattern]

Good work! I'll make some time this week to fully review this, so we can 
get it committed.

If you feel like you are up for more in this area, I'd love to have a 
mode that writes verify info based the location on disk/file, and hence 
doesn't need a header. This could/should work with any of the hashes, 
and it should have a way to specify the seed for the data. The generated 
data for the block would be a function of the offset, hash, seed.

-- 
Jens Axboe