Re: [PATCH v3] btrfs: add BTRFS_IOC_GET_CSUMS ioctl

[Mark Harmstone <mark@harmstone.com>]

On 15/04/2026 5.43 pm, David Sterba wrote:
> On Wed, Apr 15, 2026 at 02:56:49PM +0100, Mark Harmstone wrote:
>>>> When using the --reflink option added in btrfs-progs v6.16.1, we can forgo
>>>> reading the data entirely, resulting a ~2200% speed-up on the same test
>>>> (128s to 6s).
>>>
>>> Repeated mkfs is a specific use case, normaly it happens just once but I
>>> understand that for preparing various images the time savings are
>>> significiant.
>>>
>>> Once this ioctl is available I think the deduplication tools will make
>>> use of it as an initial filter before actually doing the real
>>> deduplication of extents.
>>
>> IIRC the mathematics of the birthday paradox is that for an n-bit hash,
>> you need 2^(n/2) sectors to have a 50% chance of a collision.
>>
>> For the 32-bit hashes, that means a 50% chance of a collision every 256
>> MB. For the 256-bit hashes, that's 2^140 bytes... whatever that works
>> out as. So the dedupe logic would no doubt be that for larger hash sizes
>> GET_CSUMS itself will be sufficient.
> 
> It's used as a hint in the initial filter (e.g. in
> https://github.com/lakshmipathi/dduper), the final deduplication does
> full byte compariison (memcmp() inside vfs_dedupe_file_range_compare()).

I've only had a brief look at the project, but what Lakshmipathi calls 
the "insane mode" ought to be safe for SHA256.