* [PATCH RFC 1/3] zram: allow backends to decompress from two input buffers
2026-11-21 17:09 [PATCH RFC 0/3] zram: avoid copying spanning LZ4 objects before decompression Xueyuan Chen
@ 2026-11-21 17:09 ` Xueyuan Chen
2026-11-21 17:09 ` [PATCH RFC 2/3] lib/lz4: add safe decompression " Xueyuan Chen
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Xueyuan Chen @ 2026-11-21 17:09 UTC (permalink / raw)
To: Minchan Kim, Sergey Senozhatsky, Jens Axboe
Cc: linux-kernel, linux-block, Xueyuan Chen
Let backends decompress a spanning zsmalloc object directly from its
two scatterlist entries. Keep both fragments mapped for the call.
Use the linear path if the callback is absent or returns -EOPNOTSUPP
before writing output. Propagate other errors.
Signed-off-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
---
drivers/block/zram/zcomp.c | 28 ++++++++++++++++++++++++++++
drivers/block/zram/zcomp.h | 8 ++++++++
2 files changed, 36 insertions(+)
diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
index 028e2f0f587e..265108b9f332 100644
--- a/drivers/block/zram/zcomp.c
+++ b/drivers/block/zram/zcomp.c
@@ -159,6 +159,28 @@ int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm,
return ret;
}
+static int zcomp_decompress_split(struct zcomp *comp, struct zcomp_strm *zstrm,
+ struct scatterlist *sg, void *dst)
+{
+ struct scatterlist *next = sg_next(sg);
+ /* Each entry from the zsmalloc read path fits within one page. */
+ void *first = kmap_local_page(sg_page(sg));
+ void *second = kmap_local_page(sg_page(next));
+ struct zcomp_req head = {
+ .src = first + sg->offset,
+ .src_len = sg->length,
+ .dst = dst,
+ .dst_len = PAGE_SIZE,
+ };
+ int ret;
+
+ ret = comp->ops->decompress_split(comp->params, &zstrm->ctx, &head,
+ second + next->offset, next->length);
+ kunmap_local(second);
+ kunmap_local(first);
+ return ret;
+}
+
int zcomp_decompress(struct zcomp *comp, struct zcomp_strm *zstrm,
struct scatterlist *sg, unsigned int src_len, void *dst)
{
@@ -177,6 +199,12 @@ int zcomp_decompress(struct zcomp *comp, struct zcomp_strm *zstrm,
src = kmap_local_page(sg_page(sg));
req.src = src + sg->offset;
} else {
+ if (comp->ops->decompress_split) {
+ ret = zcomp_decompress_split(comp, zstrm, sg, dst);
+ if (ret != -EOPNOTSUPP)
+ return ret;
+ }
+
/* the object spans two pages, linearize it into local copy */
sg_copy_to_buffer(sg, 2, zstrm->local_copy, src_len);
req.src = zstrm->local_copy;
diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h
index f38fd31f9e4e..92a93630aead 100644
--- a/drivers/block/zram/zcomp.h
+++ b/drivers/block/zram/zcomp.h
@@ -61,6 +61,14 @@ struct zcomp_ops {
struct zcomp_req *req);
int (*decompress)(struct zcomp_params *params, struct zcomp_ctx *ctx,
struct zcomp_req *req);
+ /*
+ * Decode one block from head->src and tail into head->dst. The input
+ * mappings are valid during the call. Return 0 for a complete output
+ * or -EOPNOTSUPP before writing output to request a linear retry.
+ */
+ int (*decompress_split)(struct zcomp_params *params, struct zcomp_ctx *ctx,
+ const struct zcomp_req *head,
+ const void *tail, size_t tail_len);
int (*create_ctx)(struct zcomp_params *params, struct zcomp_ctx *ctx);
void (*destroy_ctx)(struct zcomp_ctx *ctx);
--
2.47.3
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH RFC 2/3] lib/lz4: add safe decompression from two input buffers
2026-11-21 17:09 [PATCH RFC 0/3] zram: avoid copying spanning LZ4 objects before decompression Xueyuan Chen
2026-11-21 17:09 ` [PATCH RFC 1/3] zram: allow backends to decompress from two input buffers Xueyuan Chen
@ 2026-11-21 17:09 ` Xueyuan Chen
2026-11-21 17:09 ` [PATCH RFC 3/3] zram: use split-input decompression for LZ4 and LZ4HC Xueyuan Chen
2026-11-21 20:15 ` [PATCH RFC 0/3] zram: avoid copying spanning LZ4 objects before decompression Eric Biggers
3 siblings, 0 replies; 5+ messages in thread
From: Xueyuan Chen @ 2026-11-21 17:09 UTC (permalink / raw)
To: Minchan Kim, Sergey Senozhatsky, Jens Axboe
Cc: linux-kernel, linux-block, Xueyuan Chen
LZ4_decompress_generic() assumes contiguous input. Its wide literal
loads can cross the boundary between independently mapped fragments.
Add a dictionary-free block decoder that reads from two buffers and
handles literals, lengths and offsets that cross the split.
Signed-off-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
---
include/linux/lz4.h | 18 ++++++
lib/lz4/lz4_decompress.c | 163 +++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 181 insertions(+)
diff --git a/include/linux/lz4.h b/include/linux/lz4.h
index ad6042a718b5..3b6217bd9a9b 100644
--- a/include/linux/lz4.h
+++ b/include/linux/lz4.h
@@ -292,6 +292,24 @@ int LZ4_decompress_fast(const char *source, char *dest, int originalSize);
int LZ4_decompress_safe(const char *source, char *dest, int compressedSize,
int maxDecompressedSize);
+/**
+ * LZ4_decompress_safe_split() - Decode a single block from two buffers
+ * @first: address of the first input fragment
+ * @first_len: bytes in the first input fragment
+ * @second: address of the second input fragment
+ * @second_len: bytes in the second input fragment
+ * @dst: output buffer, disjoint from the input buffers
+ * @capacity: bytes available in the output buffer
+ *
+ * The fragments, in order, contain one dictionary-free LZ4 block.
+ * Their addresses need not be adjacent. An empty fragment may be NULL.
+ *
+ * Return: the decoded size, or -EINVAL on error.
+ */
+int LZ4_decompress_safe_split(const u8 *first, size_t first_len,
+ const u8 *second, size_t second_len,
+ u8 *dst, int capacity);
+
/**
* LZ4_decompress_safe_partial() - Decompress a block of size 'compressedSize'
* at position 'source' into buffer 'dest'
diff --git a/lib/lz4/lz4_decompress.c b/lib/lz4/lz4_decompress.c
index 3a2cd9acada4..548197faa3e3 100644
--- a/lib/lz4/lz4_decompress.c
+++ b/lib/lz4/lz4_decompress.c
@@ -39,6 +39,11 @@
#include <linux/kernel.h>
#include <linux/unaligned.h>
+#ifndef STATIC
+#include <linux/errno.h>
+#include <linux/limits.h>
+#endif
+
/*-*****************************
* Decompression functions
*******************************/
@@ -705,6 +710,164 @@ int LZ4_decompress_fast_usingDict(const char *source, char *dest,
}
#ifndef STATIC
+struct lz4_split_input {
+ const u8 *first;
+ const u8 *second;
+ size_t split;
+ size_t size;
+ size_t pos;
+};
+
+static __always_inline size_t
+lz4_split_input_left(const struct lz4_split_input *in)
+{
+ return in->size - in->pos;
+}
+
+static __always_inline int lz4_split_get_byte(struct lz4_split_input *in)
+{
+ size_t pos = in->pos;
+
+ if (unlikely(pos == in->size))
+ return -1;
+ in->pos++;
+ if (pos < in->split)
+ return in->first[pos];
+ return in->second[pos - in->split];
+}
+
+/* The caller has checked that two bytes remain in the block. */
+static __always_inline int lz4_split_get_offset(struct lz4_split_input *in)
+{
+ size_t pos = in->pos;
+
+ in->pos += 2;
+ if (pos >= in->split)
+ return get_unaligned_le16(in->second + (pos - in->split));
+ if (in->split - pos >= 2)
+ return get_unaligned_le16(in->first + pos);
+ return in->first[pos] | (in->second[0] << 8);
+}
+
+static __always_inline bool
+lz4_split_copy_literals(struct lz4_split_input *in, u8 *dst, size_t len)
+{
+ size_t pos = in->pos;
+
+ if (unlikely(len > lz4_split_input_left(in)))
+ return false;
+ if (!len)
+ return true;
+ if (pos < in->split) {
+ size_t n = min(len, in->split - pos);
+
+ memcpy(dst, in->first + pos, n);
+ pos += n;
+ dst += n;
+ len -= n;
+ }
+ if (len)
+ memcpy(dst, in->second + (pos - in->split), len);
+ in->pos = pos + len;
+ return true;
+}
+
+/* The decoder has checked the lookbehind distance and output capacity. */
+static __always_inline void
+lz4_split_copy_match(u8 *dst, size_t distance, size_t len)
+{
+ size_t copied = min(distance, len);
+
+ if (distance == 1) {
+ memset(dst, dst[-1], len);
+ return;
+ }
+
+ memcpy(dst, dst - distance, copied);
+ /* Reuse the period already written, keeping each copy disjoint. */
+ while (len - copied > copied) {
+ memcpy(dst + copied, dst, copied);
+ copied *= 2;
+ }
+ if (copied < len)
+ memcpy(dst + copied, dst, len - copied);
+}
+
+static __always_inline int
+lz4_split_read_length(struct lz4_split_input *in, int length, int limit)
+{
+ int byte;
+
+ if (length > limit)
+ return -EINVAL;
+ if (length < RUN_MASK)
+ return length;
+
+ do {
+ byte = lz4_split_get_byte(in);
+ if (byte < 0 || byte > limit - length)
+ return -EINVAL;
+ length += byte;
+ } while (byte == 255);
+ return length;
+}
+
+int LZ4_decompress_safe_split(const u8 *first, size_t first_len,
+ const u8 *second, size_t second_len,
+ u8 *dst, int capacity)
+{
+ struct lz4_split_input in = {
+ .first = first,
+ .second = second,
+ .split = first_len,
+ };
+ int written = 0, min_last_literals = 0;
+
+ if (capacity < 0 || !dst || (!first && first_len) ||
+ (!second && second_len) || first_len > SIZE_MAX - second_len)
+ return -EINVAL;
+ in.size = first_len + second_len;
+
+ for (;;) {
+ int token, literal, distance, match;
+
+ token = lz4_split_get_byte(&in);
+ if (token < 0)
+ return -EINVAL;
+ literal = lz4_split_read_length(&in, token >> ML_BITS,
+ capacity - written);
+ if (literal < 0 ||
+ !lz4_split_copy_literals(&in, dst + written, literal))
+ return -EINVAL;
+ written += literal;
+ if (!lz4_split_input_left(&in))
+ return literal >= min_last_literals ? written : -EINVAL;
+
+ /* A match must leave room for the final literal sequence. */
+ if (lz4_split_input_left(&in) < 2 + 1 + LASTLITERALS ||
+ capacity - written < MFLIMIT)
+ return -EINVAL;
+
+ distance = lz4_split_get_offset(&in);
+ if (!distance || distance > written)
+ return -EINVAL;
+ match = lz4_split_read_length(&in, token & ML_MASK,
+ capacity - written - MINMATCH -
+ LASTLITERALS);
+ if (match < 0)
+ return -EINVAL;
+ match += MINMATCH;
+ lz4_split_copy_match(dst + written, distance, match);
+ written += match;
+ /*
+ * Leave at least LASTLITERALS bytes and end at least MFLIMIT
+ * bytes past the start of the last match.
+ */
+ min_last_literals = max(LASTLITERALS, MFLIMIT - match);
+ }
+}
+EXPORT_SYMBOL_GPL(LZ4_decompress_safe_split);
+
EXPORT_SYMBOL(LZ4_decompress_safe);
EXPORT_SYMBOL(LZ4_decompress_safe_partial);
EXPORT_SYMBOL(LZ4_decompress_fast);
--
2.47.3
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH RFC 3/3] zram: use split-input decompression for LZ4 and LZ4HC
2026-11-21 17:09 [PATCH RFC 0/3] zram: avoid copying spanning LZ4 objects before decompression Xueyuan Chen
2026-11-21 17:09 ` [PATCH RFC 1/3] zram: allow backends to decompress from two input buffers Xueyuan Chen
2026-11-21 17:09 ` [PATCH RFC 2/3] lib/lz4: add safe decompression " Xueyuan Chen
@ 2026-11-21 17:09 ` Xueyuan Chen
2026-11-21 20:15 ` [PATCH RFC 0/3] zram: avoid copying spanning LZ4 objects before decompression Eric Biggers
3 siblings, 0 replies; 5+ messages in thread
From: Xueyuan Chen @ 2026-11-21 17:09 UTC (permalink / raw)
To: Minchan Kim, Sergey Senozhatsky, Jens Axboe
Cc: linux-kernel, linux-block, Xueyuan Chen
Use the split decoder for dictionary-free LZ4 and LZ4HC blocks.
Keep dictionary-backed reads on the linear path and require the
split decoder to produce a full page.
Signed-off-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
---
drivers/block/zram/backend_lz4.c | 1 +
drivers/block/zram/backend_lz4.h | 19 +++++++++++++++++++
drivers/block/zram/backend_lz4hc.c | 2 ++
3 files changed, 22 insertions(+)
diff --git a/drivers/block/zram/backend_lz4.c b/drivers/block/zram/backend_lz4.c
index 1e4ad31d39a6..2cf8ff656f13 100644
--- a/drivers/block/zram/backend_lz4.c
+++ b/drivers/block/zram/backend_lz4.c
@@ -150,6 +150,7 @@ static int lz4_decompress(struct zcomp_params *params, struct zcomp_ctx *ctx,
const struct zcomp_ops backend_lz4 = {
.compress = lz4_compress,
.decompress = lz4_decompress,
+ .decompress_split = lz4_decompress_split,
.create_ctx = lz4_create,
.destroy_ctx = lz4_destroy,
.setup_params = lz4_setup_params,
diff --git a/drivers/block/zram/backend_lz4.h b/drivers/block/zram/backend_lz4.h
index c11fa602a703..6c1736ea1a87 100644
--- a/drivers/block/zram/backend_lz4.h
+++ b/drivers/block/zram/backend_lz4.h
@@ -3,8 +3,27 @@
#ifndef __BACKEND_LZ4_H__
#define __BACKEND_LZ4_H__
+#include <linux/errno.h>
+#include <linux/lz4.h>
+
#include "zcomp.h"
+static inline int lz4_decompress_split(struct zcomp_params *params,
+ struct zcomp_ctx *ctx,
+ const struct zcomp_req *head,
+ const void *tail, size_t tail_len)
+{
+ int decoded;
+
+ if (params->dict_sz)
+ return -EOPNOTSUPP;
+ decoded = LZ4_decompress_safe_split(head->src, head->src_len, tail,
+ tail_len, head->dst, head->dst_len);
+ if (decoded < 0)
+ return decoded;
+ return decoded == head->dst_len ? 0 : -EINVAL;
+}
+
extern const struct zcomp_ops backend_lz4;
#endif /* __BACKEND_LZ4_H__ */
diff --git a/drivers/block/zram/backend_lz4hc.c b/drivers/block/zram/backend_lz4hc.c
index d8aa01bb258f..56a6eecf8d67 100644
--- a/drivers/block/zram/backend_lz4hc.c
+++ b/drivers/block/zram/backend_lz4hc.c
@@ -7,6 +7,7 @@
#include <linux/slab.h>
#include <linux/vmalloc.h>
+#include "backend_lz4.h"
#include "backend_lz4hc.h"
struct lz4hc_ctx {
@@ -134,6 +135,7 @@ static int lz4hc_decompress(struct zcomp_params *params, struct zcomp_ctx *ctx,
const struct zcomp_ops backend_lz4hc = {
.compress = lz4hc_compress,
.decompress = lz4hc_decompress,
+ .decompress_split = lz4_decompress_split,
.create_ctx = lz4hc_create,
.destroy_ctx = lz4hc_destroy,
.setup_params = lz4hc_setup_params,
--
2.47.3
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH RFC 0/3] zram: avoid copying spanning LZ4 objects before decompression
2026-11-21 17:09 [PATCH RFC 0/3] zram: avoid copying spanning LZ4 objects before decompression Xueyuan Chen
` (2 preceding siblings ...)
2026-11-21 17:09 ` [PATCH RFC 3/3] zram: use split-input decompression for LZ4 and LZ4HC Xueyuan Chen
@ 2026-11-21 20:15 ` Eric Biggers
3 siblings, 0 replies; 5+ messages in thread
From: Eric Biggers @ 2026-11-21 20:15 UTC (permalink / raw)
To: Xueyuan Chen
Cc: Minchan Kim, Sergey Senozhatsky, Jens Axboe, linux-kernel,
linux-block
Hi Xueyuan,
On Sun, Nov 22, 2026 at 01:09:23AM +0800, Xueyuan Chen wrote:
> I measured the time to swap in 1 GiB of synthetic data on arm64 with
> 4 KiB pages. Each page started with pseudorandom data of one of 16
> lengths, followed by zeros. On this workload, 32.6% of compressed
> reads spanned two pages with LZ4 and 34.4% with LZ4HC.
>
> The table shows median swap-in times over seven pairs. The last two
> columns summarize the percentage change within each pair.
>
> Codec Workers Baseline (ms) Patched (ms) Median Range
> LZ4 1 1403.81 1344.45 -4.21% -4.64% to -1.61%
> LZ4 8 222.57 216.36 -2.73% -4.19% to -1.97%
> LZ4HC 1 1403.35 1336.99 -4.74% -5.07% to -2.90%
> LZ4HC 8 222.10 215.77 -3.16% -4.39% to -1.27%
>
> These timings include both skipping the copy and switching decoders.
>
> I used Codex to help implement this series.
> Do you think the swap-in improvement justifies maintaining a second
> LZ4 decoder?
>
> Assisted-by: LLM
> Signed-off-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
LZ4_decompress_safe_split() isn't actually very well optimized. While
it avoids the copy of the source data into a contiguous buffer, it
copies every match and literal run with a variable-length memcpy(),
which on most data is a very slow way to implement LZ decompression.
I suspect it looked good on your synthetic benchmark (pseudorandom data
followed by 0's) only because that tended to generate 1 very long
literal run and 1 very long match. Whereas normal data would have lots
of interleaved literal runs and matches of much shorter lengths where an
external memcpy() call does very poorly; that is why the existing LZ4
decompression code does not do it that way.
To some extent the literal run and match copy performance is fixable
with a better implementation. However, having to handle two different
source buffers still introduces overhead that cannot go away.
So I feel this approach is probably not worth it. If you'd like to
continue to explore it, it would need to be tested on different data and
would need to be properly optimized.
- Eric
^ permalink raw reply [flat|nested] 5+ messages in thread