All of lore.kernel.org
 help / color / mirror / Atom feed
* [failures] lib-lz4-add-software-pipelined-prefetch-to-decode-fast-loop.patch removed from -mm tree
@ 2026-10-14 15:55 Andrew Morton
  0 siblings, 0 replies; only message in thread
From: Andrew Morton @ 2026-10-14 15:55 UTC (permalink / raw)
  To: mm-commits, yann.collet.73, xiang, terrelln, guoxuenan, arnd,
	Biplab.Raut, akpm

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 3336 bytes --]


The quilt patch titled
     Subject: lib/lz4: add software-pipelined prefetch to decode fast loop
has been removed from the -mm tree.  Its filename was
     lib-lz4-add-software-pipelined-prefetch-to-decode-fast-loop.patch

This patch was dropped because it had testing failures

------------------------------------------------------
From: "S. Biplab Raut" <Biplab.Raut@amd.com>
Subject: lib/lz4: add software-pipelined prefetch to decode fast loop
Date: Tue, 13 Oct 2026 03:20:10 +0000

LZ4 decompression of cold 4 KiB pages (the zswap/zram hot path) stalls on
cache misses because the input stream (compressed) and output buffer
(uncompressed page) are not resident in L1/L2.  Insert read-ahead and
write-intent prefetch hints at the top of the decode loop in
LZ4_decompress_generic() to overlap memory latency with decode work.

The prefetch distances (96 B read, 128 B write) are tuned for the
page-at-a-time workload where both buffers fit in L1d.  Prefetch is
architecturally a no-op hint on all Linux-supported architectures; it
never faults, never changes correctness, and compiles to nothing on
targets without hardware prefetch.

Measured on AMD EPYC 9755 (Turin, 2S×128C), Ubuntu 24.04, kernel
6.8.0-90-generic, real zswap path (cgroup-constrained MADV_PAGEOUT + full
scan-back), Silesia corpus, 15 paired rounds:

  Kernel LZ4 median scan:  1.0994 s
  Patched  median scan:    1.0472 s
  Median paired speedup:   5.09%  (range 4.19%–6.99%)
  Patched was faster in all 15 rounds.

Link: https://lore.kernel.org/20261013032010.2453893-1-Biplab.Raut@amd.com
Signed-off-by: S. Biplab Raut <Biplab.Raut@amd.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Gao Xiang <xiang@kernel.org>
Cc: Nick Terrell <terrelln@fb.com>
Cc: Yann Collet <yann.collet.73@gmail.com>
Cc: Guo Xuenan <guoxuenan@huawei.com>
Cc: Arnd Bergmann <arnd@arndb.de>
---

 lib/lz4/lz4_decompress.c |   21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

--- a/lib/lz4/lz4_decompress.c~lib-lz4-add-software-pipelined-prefetch-to-decode-fast-loop
+++ a/lib/lz4/lz4_decompress.c
@@ -37,6 +37,7 @@
 #include <linux/init.h>
 #include <linux/module.h>
 #include <linux/kernel.h>
+#include <linux/prefetch.h>
 #include <linux/unaligned.h>
 
 /*-*****************************
@@ -122,6 +123,26 @@ static FORCE_INLINE int LZ4_decompress_g
 		const BYTE *match;
 		size_t offset;
 
+		/*
+		 * Software-pipelined prefetch hints.  On the zswap/zram
+		 * hot path, both the compressed input and the output page
+		 * are cold (just faulted in or freshly allocated).  Prime
+		 * the L1d for the next loop iteration so that the actual
+		 * loads/stores hit warm cache lines.
+		 *
+		 * Distances are tuned for 4 KiB page-at-a-time workloads:
+		 *   96 B read-ahead on ip  ~  6 tokens ahead
+		 *  128 B write-intent on op ~ 2 long-literal copies ahead
+		 *  192 B deeper read-ahead  ~  covers the match-copy tail
+		 *
+		 * prefetch() / prefetchw() are architecture-portable hints
+		 * that compile to NOP on targets without HW prefetch.  They
+		 * never fault, even past buffer boundaries.
+		 */
+		prefetch(ip + 96);
+		prefetchw(op + 128);
+		prefetch(ip + 192);
+
 		/* get literal length */
 		unsigned int const token = *ip++;
 		length = token>>ML_BITS;
_

Patches currently in -mm which might be from Biplab.Raut@amd.com are



^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2026-10-14 15:55 UTC | newest]

Thread overview: (only message) (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-14 15:55 [failures] lib-lz4-add-software-pipelined-prefetch-to-decode-fast-loop.patch removed from -mm tree Andrew Morton

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.