From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,NORMAL_HTTP_TO_IP,NUMERIC_HTTP_ADDR, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95F4FC3A5A2 for ; Fri, 23 Aug 2019 17:09:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7904922CE3 for ; Fri, 23 Aug 2019 17:09:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732819AbfHWRJU (ORCPT ); Fri, 23 Aug 2019 13:09:20 -0400 Received: from tartarus.angband.pl ([54.37.238.230]:52454 "EHLO tartarus.angband.pl" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732783AbfHWRIy (ORCPT ); Fri, 23 Aug 2019 13:08:54 -0400 Received: from kilobyte by tartarus.angband.pl with local (Exim 4.92) (envelope-from ) id 1i1D3N-0005nZ-SX; Fri, 23 Aug 2019 19:08:45 +0200 Date: Fri, 23 Aug 2019 19:08:45 +0200 From: Adam Borowski To: Paul Jones Cc: Peter Becker , Holger =?iso-8859-1?Q?Hoffst=E4tte?= , Linux BTRFS Mailinglist Subject: Re: [PATCH v2 0/4] Support xxhash64 checksums Message-ID: <20190823170845.GD17075@angband.pl> References: <20190822114029.11225-1-jthumshirn@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Junkbait: aaron@angband.pl, zzyx@angband.pl User-Agent: Mutt/1.10.1 (2018-07-13) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: kilobyte@angband.pl X-SA-Exim-Scanned: No (on tartarus.angband.pl); SAEximRunCond expanded to false Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org On Fri, Aug 23, 2019 at 09:43:22AM +0000, Paul Jones wrote: > > > Am Do., 22. Aug. 2019 um 16:41 Uhr schrieb Holger Hoffstätte > > > : > > > > but how does btrfs benefit from this compared to using crc32-intel? > > > > > > As i know, crc32c is as far as ~3x faster than xxhash. But xxHash was > > > created with a differend design goal. > > > If you using a cpu without hardware crc32 support, xxHash provides you > > > a maximum portability and speed. Look at arm, mips, power, etc. or old > > > intel cpus like Core 2 Duo. > > > > I've got a modified version of smhasher > > (https://github.com/PeeJay/smhasher) that tests speed and cryptographics > > of various hashing functions. > > I forgot to add xxhash32 > > Crc32 Software - 379.91 MiB/sec > Crc32 Hardware - 7338.60 MiB/sec > XXhash64 Software - 12094.40 MiB/sec > XXhash32 Software - 6060.11 MiB/sec > > Testing done on a 1st Gen Ryzen. Impressive numbers from XXhash64. Newest biggest Threadripper (2990WX, no 3* version released yet): crc32 - 492.75 MiB/sec crc32hw - 9447.37 MiB/sec crc64 - 1959.51 MiB/sec xxhash32 - 7479.29 MiB/sec xxhash64 - 14911.58 MiB/sec An old Skylake (i7-6700): crc32 - 359.32 MiB/sec crc32hw - 21119.68 MiB/sec crc64 - 1656.34 MiB/sec xxhash32 - 5989.87 MiB/sec xxhash64 - 11949.41 MiB/sec Cascade Lake (0000%@): crc32hw 1.92× as fast as xxhash64. So you want crc32hw on Intel, xxhash64 on AMD. crc32 also allows going back to old kernels; the improved collision resistance of xxhash64 is not a reason as if you intend to dedupe you want a crypto hash so you don't need to verify. Meow! -- ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ ⢿⡄⠘⠷⠚⠋ The root of a real enemy is an imaginary friend. ⠈⠳⣄⠀⠀⠀⠀