Re: [LFS/MM TOPIC] fs reflink issues, fs online scrub/check, etc

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

[-- Attachment #1.1: Type: text/plain, Size: 3528 bytes --]

Didn't see the original mail, so reply here.

On 2020/1/21 上午8:58, Steve French wrote:
> Since SMB3 protocol has at least three ways to do copy offload (server
> side copy),
> the reflink topic is of interest to me and likely useful to discuss
> for Samba server as
> well as client (cifs.ko)
> 
> On Wed, Feb 10, 2016 at 1:19 PM Darrick J. Wong <darrick.wong@oracle.com> wrote:
>>
>> [resend, email exploded, sorry...]
>>
>> Hi,
>>
>> I want to discuss a few FS related topics that I haven't already seen on
>> the mailing lists:
>>
>>  * Shared pagecache pages for reflinked files (and by extension making dax
>>    work with reflink on xfs)

IIRC Goldwyn Rodrigues <rgoldwyn@suse.com> is working on this, mostly
for btrfs, but should also apply to other fses.

>>
>>  * Providing a simple interface for scrubbing filesystem metadata in the
>>    background (the online check thing).  Ideally we'd make it easy to discover
>>    what kind of metadata there is to check and provide a simple interface to
>>    check the metadata, once discovered.  This is a tricky interface topic
>>    since FS design differs pretty widely.

Although btrfs has already implemented scrub for a long time, and btrfs
is coming over the point where kernel can detects more problems than
btrfs-check, I still hesitate to call it "check the metadata".

It looks more like "extended metadata sanity verification", other than
full cross-ref check implemented in user-space tools.

Currently, btrfs (and I guess xfs too) can detect corrupted metadata by:
- checksum
  All modern fses have similar mechanism.

- internal fields checking at read time
  At least btrfs is trying to do a byte-per-byte check for each fields.
  And latest such check has killed tons of fuzzed images (I guess that's
  why a lot of such CVE/fuzzed image reporters only want to report on
  older kernels).

  But this is mostly based on the fact that btrfs on-disk fields has a
  lot of redundancy. E.g. btrfs uses bytenr, not block count, allowing
  us to detect bit flips in lower bits easily.
  So not all fs could follow this step.

  But this provides us a good centralized place to validate most tree
  blocks.

  Maybe other fs devs could share such details too?

- runtime sanity check across metadata boundaries
  This is the traditional methods.

So is that xfs online scrub just a similar thing, or a full cross-ref check?

And if so, can we find a less confusing naming for the interface first?

>>
>>  * Rudimentary online repair and rebuilding (xfs) from secondary metadata

My guess is, it's checksum and copy based.
Or just like btrfs, if checksum passes but internal checks still failed,
just try next copy?

>>
>>  * Working out the variances in the btrfs/xfs/ocfs2/nfs reflink implementations
>>    and making sure they all get test coverage

That would be great!

Thanks,
Qu

>>
>> I would also like participate in some of the proposed discussions:
>>
>>  * The ext4 summit (and whatever meeting of XFS devs may happen)
>>
>>  * Integrating existing filesystems into pmem, or hallway bofs about designing
>>    new filesystems for pmem
>>
>>  * Actually seeing the fs developers (well, everyone!) in person again :)
>>
>> --Darrick
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]