Re: [PATCH v2 0/4] Support xxhash64 checksums

[Adam Borowski <kilobyte@angband.pl>]

On Fri, Aug 23, 2019 at 09:43:22AM +0000, Paul Jones wrote:
> > > Am Do., 22. Aug. 2019 um 16:41 Uhr schrieb Holger Hoffstätte
> > > <holger@applied-asynchrony.com>:
> > > > but how does btrfs benefit from this compared to using crc32-intel?
> > >
> > > As i know, crc32c  is as far as ~3x faster than xxhash. But xxHash was
> > > created with a differend design goal.
> > > If you using a cpu without hardware crc32 support, xxHash provides you
> > > a maximum portability and speed. Look at arm, mips, power, etc. or old
> > > intel cpus like Core 2 Duo.
> > 
> > I've got a modified version of smhasher
> > (https://github.com/PeeJay/smhasher) that tests speed and cryptographics
> > of various hashing functions.
> 
> I forgot to add xxhash32
>  
> Crc32 Software -  379.91 MiB/sec
> Crc32 Hardware - 7338.60 MiB/sec
> XXhash64 Software - 12094.40 MiB/sec
> XXhash32 Software - 6060.11 MiB/sec
> 
> Testing done on a 1st Gen Ryzen. Impressive numbers from XXhash64.

Newest biggest Threadripper (2990WX, no 3* version released yet):
crc32      -   492.75 MiB/sec
crc32hw    -  9447.37 MiB/sec
crc64      -  1959.51 MiB/sec
xxhash32   -  7479.29 MiB/sec
xxhash64   - 14911.58 MiB/sec

An old Skylake (i7-6700):
crc32      -   359.32 MiB/sec
crc32hw    - 21119.68 MiB/sec
crc64      -  1656.34 MiB/sec
xxhash32   -  5989.87 MiB/sec
xxhash64   - 11949.41 MiB/sec

Cascade Lake (0000%@):
crc32hw 1.92× as fast as xxhash64.

So you want crc32hw on Intel, xxhash64 on AMD.

crc32 also allows going back to old kernels; the improved collision
resistance of xxhash64 is not a reason as if you intend to dedupe you want
a crypto hash so you don't need to verify.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋  The root of a real enemy is an imaginary friend.
⠈⠳⣄⠀⠀⠀⠀