From: Nick Terrell <nickrterrell@gmail.com>
To: linux-kernel@vger.kernel.org
Cc: Yann Collet <cyan@meta.com>, Nick Terrell <terrelln@meta.com>,
Nick Terrell <nickrterrell@gmail.com>,
Kernel Team <Kernel-team@fb.com>,
Giovanni Cabiddu <giovanni.cabiddu@intel.com>,
Linus Torvalds <torvalds@linux-foundation.org>
Subject: [PATCH 0/2] zstd: import upstream v1.5.5
Date: Mon, 20 Nov 2023 17:03:16 -0800 [thread overview]
Message-ID: <20231121010318.524570-1-nickrterrell@gmail.com> (raw)
From: Nick Terrell <terrelln@meta.com>
Import upstream zstd v1.5.5 to expose upstream's QAT integration.
The following changes since commit 77618db346455129424fadbbaec596a09feaf3bb:
zstd: Fix array-index-out-of-bounds UBSAN warning (2023-11-14 17:12:52 -0800)
are also available in the Git repository at:
https://github.com/terrelln/linux.git tags/2023-11-20-zstd-v1.5.5-v1
In addition to keeping the kernel's copy of zstd up to date, this update
was requested by Intel to expose upstream zstd's external match provider
API to the kernel, which allows QAT to accelerate the LZ match finding
stage.
The result of the series is that zstd is cleanly imported from the
tag v1.5.5-kernel [0], which is signed with upstream's signing key
EF8FE99528B52FFD [1]. Meaning that running this script creates no diff:
export ZSTD=/path/to/repo/zstd/
export LINUX=/path/to/repo/linux/
cd "$ZSTD/contrib/linux-kernel"
git checkout v1.5.5-kernel
make import LINUX="$LINUX"
The first patch updates to v1.5.5 with three cherry picked backports
from the Linux kernel and the upstream dev branch. The second patch
negates a regression in decompression speed caused by the update.
The series has been tested on x86-64, and has been boot tested with a
zstd compressed kernel on i386 and aarch64. I benchmarked the series
on x86-64 with gcc-13.2.1 on an Intel i9-9900K by measuring the
performance of compressed filesystems read & writes.
Component, Level, C. time delta, size delta, D. time delta
Btrfs , 1, -1.5%, +0.0%, +1.8%
Btrfs , 3, -5.8%, +0.0%, +3.2%
Btrfs , 5, -5.7%, +0.0%, +0.0%
Btrfs , 7, -6.5%, +0.0%, +0.4%
Btrfs , 9, -6.9%, +0.0%, +1.5%
Squashfs , 1, N/A, 0.0%, +1.0%
For levels 3 and above, there is a ~6% reduction in write+compression time.
There is also a small increase in read+decompression time.
If everything looks good, I will submit this patch set with the rest of my
tree in the v6.8 merge window.
[0] https://github.com/facebook/zstd/tree/v1.5.5-kernel
[1] https://keyserver.ubuntu.com/pks/lookup?search=EF8FE99528B52FFD&fingerprint=on&op=index
Nick Terrell (2):
zstd: import upstream v1.5.5
zstd: Backport Huffman speed improvement from upstream
include/linux/zstd.h | 2 +-
include/linux/zstd_errors.h | 23 +-
include/linux/zstd_lib.h | 697 +++++--
lib/zstd/Makefile | 2 +-
lib/zstd/common/allocations.h | 56 +
lib/zstd/common/bits.h | 149 ++
lib/zstd/common/bitstream.h | 53 +-
lib/zstd/common/compiler.h | 14 +-
lib/zstd/common/cpu.h | 3 +-
lib/zstd/common/debug.c | 3 +-
lib/zstd/common/debug.h | 3 +-
lib/zstd/common/entropy_common.c | 42 +-
lib/zstd/common/error_private.c | 12 +-
lib/zstd/common/error_private.h | 3 +-
lib/zstd/common/fse.h | 89 +-
lib/zstd/common/fse_decompress.c | 94 +-
lib/zstd/common/huf.h | 222 +--
lib/zstd/common/mem.h | 2 +-
lib/zstd/common/portability_macros.h | 26 +-
lib/zstd/common/zstd_common.c | 38 +-
lib/zstd/common/zstd_deps.h | 16 +-
lib/zstd/common/zstd_internal.h | 99 +-
lib/zstd/compress/clevels.h | 3 +-
lib/zstd/compress/fse_compress.c | 59 +-
lib/zstd/compress/hist.c | 3 +-
lib/zstd/compress/hist.h | 3 +-
lib/zstd/compress/huf_compress.c | 372 ++--
lib/zstd/compress/zstd_compress.c | 1762 ++++++++++++-----
lib/zstd/compress/zstd_compress_internal.h | 333 +++-
lib/zstd/compress/zstd_compress_literals.c | 155 +-
lib/zstd/compress/zstd_compress_literals.h | 25 +-
lib/zstd/compress/zstd_compress_sequences.c | 7 +-
lib/zstd/compress/zstd_compress_sequences.h | 3 +-
lib/zstd/compress/zstd_compress_superblock.c | 47 +-
lib/zstd/compress/zstd_compress_superblock.h | 3 +-
lib/zstd/compress/zstd_cwksp.h | 149 +-
lib/zstd/compress/zstd_double_fast.c | 129 +-
lib/zstd/compress/zstd_double_fast.h | 6 +-
lib/zstd/compress/zstd_fast.c | 582 ++++--
lib/zstd/compress/zstd_fast.h | 6 +-
lib/zstd/compress/zstd_lazy.c | 518 ++---
lib/zstd/compress/zstd_lazy.h | 7 +-
lib/zstd/compress/zstd_ldm.c | 11 +-
lib/zstd/compress/zstd_ldm.h | 3 +-
lib/zstd/compress/zstd_ldm_geartab.h | 3 +-
lib/zstd/compress/zstd_opt.c | 187 +-
lib/zstd/compress/zstd_opt.h | 3 +-
lib/zstd/decompress/huf_decompress.c | 770 ++++---
lib/zstd/decompress/zstd_ddict.c | 9 +-
lib/zstd/decompress/zstd_ddict.h | 3 +-
lib/zstd/decompress/zstd_decompress.c | 261 ++-
lib/zstd/decompress/zstd_decompress_block.c | 283 ++-
lib/zstd/decompress/zstd_decompress_block.h | 8 +-
.../decompress/zstd_decompress_internal.h | 7 +-
lib/zstd/decompress_sources.h | 2 +-
lib/zstd/zstd_common_module.c | 5 +-
lib/zstd/zstd_compress_module.c | 2 +-
lib/zstd/zstd_decompress_module.c | 4 +-
58 files changed, 4787 insertions(+), 2594 deletions(-)
create mode 100644 lib/zstd/common/allocations.h
create mode 100644 lib/zstd/common/bits.h
--
2.42.1
next reply other threads:[~2023-11-21 0:51 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-11-21 1:03 Nick Terrell [this message]
2023-11-21 1:03 ` [PATCH 1/2] zstd: import upstream v1.5.5 Nick Terrell
2023-11-21 1:03 ` [PATCH 2/2] zstd: Backport Huffman speed improvement from upstream Nick Terrell
2023-11-21 17:23 ` Linus Torvalds
2023-11-21 19:59 ` Nick Terrell
2023-11-21 20:12 ` Linus Torvalds
2023-11-21 20:35 ` Nick Terrell
2023-11-21 20:54 ` Linus Torvalds
2023-11-22 1:09 ` Nick Terrell
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231121010318.524570-1-nickrterrell@gmail.com \
--to=nickrterrell@gmail.com \
--cc=Kernel-team@fb.com \
--cc=cyan@meta.com \
--cc=giovanni.cabiddu@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=terrelln@meta.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox