* [PATCH v2 01/15] lib/decompress_unlz4: use LZ4_decompress_safe_partial() before boot
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 02/15] lib/lz4: remove LZ4_decompress_fast() and its variants Michal Wilczynski
` (12 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
The pre-boot decompressor is the only in-tree user of
LZ4_decompress_fast(), which upstream has deprecated since v1.9. It
never checks the input against the chunk length in the header.
Decode with LZ4_decompress_safe_partial() instead, bounded by the chunk
length and by the expected output length. Count every chunk against the
size the image ends with, and fail a chunk that decodes short or comes
after that size is reached.
x86 decompresses in place; the margin arch/x86/boot/header.S reserves,
(output >> 8) + 128K, covers the safe decoder's 32-byte overshoot.
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
lib/decompress_unlz4.c | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/lib/decompress_unlz4.c b/lib/decompress_unlz4.c
index c0dbb3cea915eb91d8f4ac26c38c4785bbf19770..dc20c22e087900e21da49f9fcb267e6b511e1072 100644
--- a/lib/decompress_unlz4.c
+++ b/lib/decompress_unlz4.c
@@ -151,14 +151,17 @@ STATIC inline int INIT unlz4(u8 *input, long in_len,
}
}
#ifdef PREBOOT
- if (out_len >= uncomp_chunksize) {
+ if (out_len >= uncomp_chunksize)
dest_len = uncomp_chunksize;
- out_len -= dest_len;
- } else
+ else
dest_len = out_len;
+ out_len -= dest_len;
- ret = LZ4_decompress_fast(inp, outp, dest_len);
- chunksize = ret;
+ ret = LZ4_decompress_safe_partial(inp, outp, chunksize,
+ dest_len, dest_len);
+ /* A chunk once the image's stated size is reached is corrupt. */
+ if (!dest_len || ret != (int)dest_len)
+ ret = -1;
#else
dest_len = uncomp_chunksize;
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 02/15] lib/lz4: remove LZ4_decompress_fast() and its variants
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 01/15] lib/decompress_unlz4: use LZ4_decompress_safe_partial() before boot Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 03/15] zram: lz4hc: reset the dictionary stream once, not per page Michal Wilczynski
` (11 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
LZ4_decompress_fast(), LZ4_decompress_fast_continue() and
LZ4_decompress_fast_usingDict() have no users left. Upstream has
deprecated all three since v1.9: they take only the decompressed size,
so a corrupt or malicious stream has nothing to stop the decoder from
reading past the end of its input.
Remove them, their exports and their declarations in <linux/lz4.h>,
along with the two helpers only they used. Doing it before lib/lz4 moves
to upstream LZ4 keeps them out of the glue that move adds.
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
include/linux/lz4.h | 81 ----------------------------------------------
lib/lz4/lz4_decompress.c | 84 ------------------------------------------------
2 files changed, 165 deletions(-)
diff --git a/include/linux/lz4.h b/include/linux/lz4.h
index ad6042a718b5428792b795db7a8d4ad44c24a985..0df69e2b1ad523c291e3a96ab5595df9239f7252 100644
--- a/include/linux/lz4.h
+++ b/include/linux/lz4.h
@@ -249,27 +249,6 @@ int LZ4_compress_destSize(const char *source, char *dest, int *sourceSizePtr,
* Decompression Functions
**************************************************************************/
-/**
- * LZ4_decompress_fast() - Decompresses data from 'source' into 'dest'
- * @source: source address of the compressed data
- * @dest: output buffer address of the uncompressed data
- * which must be already allocated with 'originalSize' bytes
- * @originalSize: is the original and therefore uncompressed size
- *
- * Decompresses data from 'source' into 'dest'.
- * This function fully respect memory boundaries for properly formed
- * compressed data.
- * It is a bit faster than LZ4_decompress_safe().
- * However, it does not provide any protection against intentionally
- * modified data stream (malicious input).
- * Use this function in trusted environment only
- * (data to decode comes from a trusted source).
- *
- * Return: number of bytes read from the source buffer
- * or a negative result if decompression fails.
- */
-int LZ4_decompress_fast(const char *source, char *dest, int originalSize);
-
/**
* LZ4_decompress_safe() - Decompression protected against buffer overflow
* @source: source address of the compressed data
@@ -561,44 +540,6 @@ int LZ4_decompress_safe_continue(LZ4_streamDecode_t *LZ4_streamDecode,
const char *source, char *dest, int compressedSize,
int maxDecompressedSize);
-/**
- * LZ4_decompress_fast_continue() - Decompress blocks in streaming mode
- * @LZ4_streamDecode: the 'LZ4_streamDecode_t' structure
- * @source: source address of the compressed data
- * @dest: output buffer address of the uncompressed data
- * which must be already allocated with 'originalSize' bytes
- * @originalSize: is the original and therefore uncompressed size
- *
- * This decoding function allows decompression of multiple blocks
- * in "streaming" mode.
- * Previously decoded blocks *must* remain available at the memory position
- * where they were decoded (up to 64 KB)
- * In the case of a ring buffers, decoding buffer must be either :
- * - Exactly same size as encoding buffer, with same update rule
- * (block boundaries at same positions) In which case,
- * the decoding & encoding ring buffer can have any size,
- * including very small ones ( < 64 KB).
- * - Larger than encoding buffer, by a minimum of maxBlockSize more bytes.
- * maxBlockSize is implementation dependent.
- * It's the maximum size you intend to compress into a single block.
- * In which case, encoding and decoding buffers do not need
- * to be synchronized, and encoding ring buffer can have any size,
- * including small ones ( < 64 KB).
- * - _At least_ 64 KB + 8 bytes + maxBlockSize.
- * In which case, encoding and decoding buffers do not need to be
- * synchronized, and encoding ring buffer can have any size,
- * including larger than decoding buffer. W
- * Whenever these conditions are not possible, save the last 64KB of decoded
- * data into a safe buffer, and indicate where it is saved
- * using LZ4_setStreamDecode()
- *
- * Return: number of bytes decompressed into destination buffer
- * (necessarily <= maxDecompressedSize)
- * or a negative result in case of error
- */
-int LZ4_decompress_fast_continue(LZ4_streamDecode_t *LZ4_streamDecode,
- const char *source, char *dest, int originalSize);
-
/**
* LZ4_decompress_safe_usingDict() - Same as LZ4_setStreamDecode()
* followed by LZ4_decompress_safe_continue()
@@ -623,28 +564,6 @@ int LZ4_decompress_safe_usingDict(const char *source, char *dest,
int compressedSize, int maxDecompressedSize, const char *dictStart,
int dictSize);
-/**
- * LZ4_decompress_fast_usingDict() - Same as LZ4_setStreamDecode()
- * followed by LZ4_decompress_fast_continue()
- * @source: source address of the compressed data
- * @dest: output buffer address of the uncompressed data
- * which must be already allocated with 'originalSize' bytes
- * @originalSize: is the original and therefore uncompressed size
- * @dictStart: pointer to the start of the dictionary in memory
- * @dictSize: size of dictionary
- *
- * This decoding function works the same as
- * a combination of LZ4_setStreamDecode() followed by
- * LZ4_decompress_fast_continue()
- * It is stand-alone, and doesn't need an LZ4_streamDecode_t structure.
- *
- * Return: number of bytes decompressed into destination buffer
- * (necessarily <= maxDecompressedSize)
- * or a negative result in case of error
- */
-int LZ4_decompress_fast_usingDict(const char *source, char *dest,
- int originalSize, const char *dictStart, int dictSize);
-
#define LZ4_DECOMPRESS_INPLACE_MARGIN(compressedSize) (((compressedSize) >> 8) + 32)
#ifndef LZ4_DISTANCE_MAX /* history window size; can be user-defined at compile time */
diff --git a/lib/lz4/lz4_decompress.c b/lib/lz4/lz4_decompress.c
index 3a2cd9acada4a09ef70a344090d48167c46088a1..f89b0e547f2156990c16770f69f89e0d8b46156d 100644
--- a/lib/lz4/lz4_decompress.c
+++ b/lib/lz4/lz4_decompress.c
@@ -474,14 +474,6 @@ int LZ4_decompress_safe_partial(const char *src, char *dst,
noDict, (BYTE *)dst, NULL, 0);
}
-int LZ4_decompress_fast(const char *source, char *dest, int originalSize)
-{
- return LZ4_decompress_generic(source, dest, 0, originalSize,
- endOnOutputSize, decode_full_block,
- withPrefix64k,
- (BYTE *)dest - 64 * KB, NULL, 0);
-}
-
/* ===== Instantiate a few more decoding cases, used more than once. ===== */
static int LZ4_decompress_safe_withPrefix64k(const char *source, char *dest,
@@ -517,17 +509,6 @@ static int LZ4_decompress_safe_forceExtDict(const char *source, char *dest,
(const BYTE *)dictStart, dictSize);
}
-static int LZ4_decompress_fast_extDict(const char *source, char *dest,
- int originalSize,
- const void *dictStart, size_t dictSize)
-{
- return LZ4_decompress_generic(source, dest,
- 0, originalSize,
- endOnOutputSize, decode_full_block,
- usingExtDict, (BYTE *)dest,
- (const BYTE *)dictStart, dictSize);
-}
-
/*
* The "double dictionary" mode, for use with e.g. ring buffers: the first part
* of the dictionary is passed as prefix, and the second via dictStart + dictSize.
@@ -546,18 +527,6 @@ int LZ4_decompress_safe_doubleDict(const char *source, char *dest,
(const BYTE *)dictStart, dictSize);
}
-static FORCE_INLINE
-int LZ4_decompress_fast_doubleDict(const char *source, char *dest,
- int originalSize, size_t prefixSize,
- const void *dictStart, size_t dictSize)
-{
- return LZ4_decompress_generic(source, dest,
- 0, originalSize,
- endOnOutputSize, decode_full_block,
- usingExtDict, (BYTE *)dest - prefixSize,
- (const BYTE *)dictStart, dictSize);
-}
-
/* ===== streaming decompression functions ===== */
int LZ4_setStreamDecode(LZ4_streamDecode_t *LZ4_streamDecode,
@@ -636,45 +605,6 @@ int LZ4_decompress_safe_continue(LZ4_streamDecode_t *LZ4_streamDecode,
return result;
}
-int LZ4_decompress_fast_continue(LZ4_streamDecode_t *LZ4_streamDecode,
- const char *source, char *dest, int originalSize)
-{
- LZ4_streamDecode_t_internal *lz4sd = &LZ4_streamDecode->internal_donotuse;
- int result;
-
- if (lz4sd->prefixSize == 0) {
- assert(lz4sd->extDictSize == 0);
- result = LZ4_decompress_fast(source, dest, originalSize);
- if (result <= 0)
- return result;
- lz4sd->prefixSize = originalSize;
- lz4sd->prefixEnd = (BYTE *)dest + originalSize;
- } else if (lz4sd->prefixEnd == (BYTE *)dest) {
- if (lz4sd->prefixSize >= 64 * KB - 1 ||
- lz4sd->extDictSize == 0)
- result = LZ4_decompress_fast(source, dest,
- originalSize);
- else
- result = LZ4_decompress_fast_doubleDict(source, dest,
- originalSize, lz4sd->prefixSize,
- lz4sd->externalDict, lz4sd->extDictSize);
- if (result <= 0)
- return result;
- lz4sd->prefixSize += originalSize;
- lz4sd->prefixEnd += originalSize;
- } else {
- lz4sd->extDictSize = lz4sd->prefixSize;
- lz4sd->externalDict = lz4sd->prefixEnd - lz4sd->extDictSize;
- result = LZ4_decompress_fast_extDict(source, dest,
- originalSize, lz4sd->externalDict, lz4sd->extDictSize);
- if (result <= 0)
- return result;
- lz4sd->prefixSize = originalSize;
- lz4sd->prefixEnd = (BYTE *)dest + originalSize;
- }
- return result;
-}
-
int LZ4_decompress_safe_usingDict(const char *source, char *dest,
int compressedSize, int maxOutputSize,
const char *dictStart, int dictSize)
@@ -693,26 +623,12 @@ int LZ4_decompress_safe_usingDict(const char *source, char *dest,
compressedSize, maxOutputSize, dictStart, dictSize);
}
-int LZ4_decompress_fast_usingDict(const char *source, char *dest,
- int originalSize,
- const char *dictStart, int dictSize)
-{
- if (dictSize == 0 || dictStart + dictSize == dest)
- return LZ4_decompress_fast(source, dest, originalSize);
-
- return LZ4_decompress_fast_extDict(source, dest, originalSize,
- dictStart, dictSize);
-}
-
#ifndef STATIC
EXPORT_SYMBOL(LZ4_decompress_safe);
EXPORT_SYMBOL(LZ4_decompress_safe_partial);
-EXPORT_SYMBOL(LZ4_decompress_fast);
EXPORT_SYMBOL(LZ4_setStreamDecode);
EXPORT_SYMBOL(LZ4_decompress_safe_continue);
-EXPORT_SYMBOL(LZ4_decompress_fast_continue);
EXPORT_SYMBOL(LZ4_decompress_safe_usingDict);
-EXPORT_SYMBOL(LZ4_decompress_fast_usingDict);
MODULE_LICENSE("Dual BSD/GPL");
MODULE_DESCRIPTION("LZ4 decompressor");
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 03/15] zram: lz4hc: reset the dictionary stream once, not per page
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 01/15] lib/decompress_unlz4: use LZ4_decompress_safe_partial() before boot Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 02/15] lib/lz4: remove LZ4_decompress_fast() and its variants Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 05/15] lib/lz4: backport upstream's -Wmissing-prototypes fix Michal Wilczynski
` (10 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
With a dictionary, lz4hc_compress() calls LZ4_resetStreamHC() and then
LZ4_loadDictHC() for every page. LZ4_loadDictHC() re-initialises the
stream itself and keeps the level, so one reset at creation is enough.
The reset is cheap today, but upstream LZ4, which lib/lz4 moves to later
in this series, clears the whole 256K state in both calls. Reset once in
lz4hc_create() instead.
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
drivers/block/zram/backend_lz4hc.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/block/zram/backend_lz4hc.c b/drivers/block/zram/backend_lz4hc.c
index d8aa01bb258fec760b597a88296dfaa4e0049693..06286c805cf7c56bd760f460cc854dc4014f293b 100644
--- a/drivers/block/zram/backend_lz4hc.c
+++ b/drivers/block/zram/backend_lz4hc.c
@@ -72,6 +72,7 @@ static int lz4hc_create(struct zcomp_params *params, struct zcomp_ctx *ctx)
zctx->cstrm = kzalloc_obj(*zctx->cstrm);
if (!zctx->cstrm)
goto error;
+ LZ4_resetStreamHC(zctx->cstrm, params->level);
}
return 0;
@@ -92,8 +93,7 @@ static int lz4hc_compress(struct zcomp_params *params, struct zcomp_ctx *ctx,
req->dst_len, params->level,
zctx->mem);
} else {
- /* Cstrm needs to be reset */
- LZ4_resetStreamHC(zctx->cstrm, params->level);
+ /* Reloading the dict resets cstrm and keeps its level */
ret = LZ4_loadDictHC(zctx->cstrm, params->dict,
params->dict_sz);
if (ret != params->dict_sz)
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 05/15] lib/lz4: backport upstream's -Wmissing-prototypes fix
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
` (2 preceding siblings ...)
2026-11-06 14:42 ` [PATCH v2 03/15] zram: lz4hc: reset the dictionary stream once, not per page Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 06/15] lib/lz4: backport upstream's aarch64 fast decode loop enablement Michal Wilczynski
` (9 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
v1.10.0 defines LZ4_loadDict_internal() in lz4.c and LZ4HC_searchExtDict()
in lz4hc.c with external linkage and no prototype, which the kernel's
-Wmissing-prototypes rejects. The first would also prevent linking once
lz4_compress.c and lz4_decompress.c both include lz4.c, later in this
series, since each would define it.
Neither can be forward declared static in the lz4_deps.h a later patch
adds, as the other bare helpers are, because their signatures use types
declared inside the .c files.
Take the two hunks of upstream commit 5ef1f16929b5
("fix -Wmissing-prototypes warnings") that apply to the files we vendor.
This goes away at the next re-sync.
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
lib/lz4/upstream/lz4.c | 2 +-
lib/lz4/upstream/lz4hc.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/lz4/upstream/lz4.c b/lib/lz4/upstream/lz4.c
index a2f7abee19fb9a5c768f2a6c266acf5b571f0855..0d474be0236a698d297ab9a430f5082485eb3709 100644
--- a/lib/lz4/upstream/lz4.c
+++ b/lib/lz4/upstream/lz4.c
@@ -1584,7 +1584,7 @@ int LZ4_freeStream (LZ4_stream_t* LZ4_stream)
typedef enum { _ld_fast, _ld_slow } LoadDict_mode_e;
#define HASH_UNIT sizeof(reg_t)
-int LZ4_loadDict_internal(LZ4_stream_t* LZ4_dict,
+static int LZ4_loadDict_internal(LZ4_stream_t* LZ4_dict,
const char* dictionary, int dictSize,
LoadDict_mode_e _ld)
{
diff --git a/lib/lz4/upstream/lz4hc.c b/lib/lz4/upstream/lz4hc.c
index 4d8c36a6978fcae09e4c4a936572b4b271513466..32b29f382268d961d65383b24c7225494d9044ad 100644
--- a/lib/lz4/upstream/lz4hc.c
+++ b/lib/lz4/upstream/lz4hc.c
@@ -360,7 +360,7 @@ typedef struct {
int back; /* negative value */
} LZ4HC_match_t;
-LZ4HC_match_t LZ4HC_searchExtDict(const BYTE* ip, U32 ipIndex,
+static LZ4HC_match_t LZ4HC_searchExtDict(const BYTE* ip, U32 ipIndex,
const BYTE* const iLowLimit, const BYTE* const iHighLimit,
const LZ4HC_CCtx_internal* dictCtx, U32 gDictEndIndex,
int currentBestML, int nbAttempts)
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 06/15] lib/lz4: backport upstream's aarch64 fast decode loop enablement
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
` (3 preceding siblings ...)
2026-11-06 14:42 ` [PATCH v2 05/15] lib/lz4: backport upstream's -Wmissing-prototypes fix Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 07/15] lib/lz4: backport upstream's match length limit fix Michal Wilczynski
` (8 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
v1.10.0 leaves LZ4_FAST_DEC_LOOP off for aarch64 built with clang, after
a slowdown reported on some Android phones in 2019 (upstream PR #707).
An arm64 kernel built with clang therefore decodes with the slower loop.
Upstream has since enabled it for all aarch64, in three unreleased
commits:
commit 0aa33884faf5 ("[LZ4] Limit disabling LZ4_FAST_DEC_LOOP to aarch64+clang+android")
commit 51f8af506b4a ("Fix missing endif")
commit ed86bc1ed802 ("AArch64: Enable LZ4_FAST_DEC_LOOP for Android")
Arm measured +3.5% to +15.6% on Cortex-A510 to X4 with clang. Take the
lz4.c hunks of all three. This goes away at the next re-sync.
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
lib/lz4/upstream/lz4.c | 7 +------
1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/lib/lz4/upstream/lz4.c b/lib/lz4/upstream/lz4.c
index 0d474be0236a698d297ab9a430f5082485eb3709..ee69d57aae112f4057c050bbdab4cc0f2d30f9d9 100644
--- a/lib/lz4/upstream/lz4.c
+++ b/lib/lz4/upstream/lz4.c
@@ -478,12 +478,7 @@ static const int dec64table[8] = {0, 0, 0, -1, -4, 1, 2, 3};
#ifndef LZ4_FAST_DEC_LOOP
# if defined __i386__ || defined _M_IX86 || defined __x86_64__ || defined _M_X64
# define LZ4_FAST_DEC_LOOP 1
-# elif defined(__aarch64__) && defined(__APPLE__)
-# define LZ4_FAST_DEC_LOOP 1
-# elif defined(__aarch64__) && !defined(__clang__)
- /* On non-Apple aarch64, we disable this optimization for clang because
- * on certain mobile chipsets, performance is reduced with clang. For
- * more information refer to https://github.com/lz4/lz4/pull/707 */
+# elif defined(__aarch64__)
# define LZ4_FAST_DEC_LOOP 1
# else
# define LZ4_FAST_DEC_LOOP 0
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 07/15] lib/lz4: backport upstream's match length limit fix
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
` (4 preceding siblings ...)
2026-11-06 14:42 ` [PATCH v2 06/15] lib/lz4: backport upstream's aarch64 fast decode loop enablement Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 08/15] lib/lz4: add the build environment for the vendored sources Michal Wilczynski
` (7 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
v1.10.0 lets a match length's extension bytes run to four bytes before
the end of the block, which leaves room for only a token and three
literals where the format requires five. The fork rejected such blocks,
v1.10.0 decodes them: 1f 41 01 00 ff 00 30 7a 7a 7a decodes to 278
bytes instead of failing.
Upstream fixed this in 587ed9557708 ("add comments explaining
read_variable_length() ilimit bounds, fix match length limit"), on top
of a read_variable_length() rework v1.10.0 does not have. Both versions
accept the same final ip for a given limit, so change the limit at the
two match length call sites to iend - (1 + LASTLITERALS), as upstream
does. This is one byte stricter than the fork; no compressor produces
the blocks it newly rejects.
This goes away at the next re-sync.
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
lib/lz4/upstream/lz4.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/lz4/upstream/lz4.c b/lib/lz4/upstream/lz4.c
index ee69d57aae112f4057c050bbdab4cc0f2d30f9d9..a45f3828141eec9baea59a5e97a3ff37d6474d6e 100644
--- a/lib/lz4/upstream/lz4.c
+++ b/lib/lz4/upstream/lz4.c
@@ -2120,7 +2120,7 @@ LZ4_decompress_generic(
DEBUGLOG(7, " match length token = %u (len==%u)", (unsigned)length, (unsigned)length+MINMATCH);
if (length == ML_MASK) {
- size_t const addl = read_variable_length(&ip, iend - LASTLITERALS + 1, 0);
+ size_t const addl = read_variable_length(&ip, iend - (1 + LASTLITERALS), 0);
if (addl == rvl_error) {
DEBUGLOG(5, "error reading long match length");
goto _output_error;
@@ -2338,7 +2338,7 @@ LZ4_decompress_generic(
_copy_match:
if (length == ML_MASK) {
- size_t const addl = read_variable_length(&ip, iend - LASTLITERALS + 1, 0);
+ size_t const addl = read_variable_length(&ip, iend - (1 + LASTLITERALS), 0);
if (addl == rvl_error) { goto _output_error; }
length += addl;
if (unlikely((uptrval)(op)+length<(uptrval)op)) goto _output_error; /* overflow detection */
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 08/15] lib/lz4: add the build environment for the vendored sources
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
` (5 preceding siblings ...)
2026-11-06 14:42 ` [PATCH v2 07/15] lib/lz4: backport upstream's match length limit fix Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 09/15] arch: boot: put the LZ4 freestanding headers on the decompressor path Michal Wilczynski
` (6 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
lz4_deps.h makes the vendored upstream sources build in the kernel. It
gives every upstream entry point internal linkage, so an object carries
only the codec it forwards, selects heap mode with failing allocators so
LZ4HC's ~64K optimal-parser buffer stays off the stack, and renames the
entry points the kernel re-exports so the kernel's own definitions can
use the real names. LZ4LIB_VISIBILITY carries __maybe_unused for the
functions an object does not use.
freestanding/ forwards the four ISO C headers upstream includes to their
<linux/> equivalents. lz4_kernel_api.h undoes the renaming; it carries
its own copy of the exported prototypes until a later patch can include
<linux/lz4.h> instead.
Two settings upstream leaves to the build:
- Unaligned access goes through memcpy() where the architecture has
efficient unaligned access, and through packed structs, as
get_unaligned() does, elsewhere. Left alone, upstream uses plain
misaligned loads on ARMv6.
- On ppc64le, lz4.c marks functions __attribute__((optimize("O2"))) for
GCC, which drops command line options such as -fno-strict-aliasing
for them; see commit 080b6f407635 ("bpf: Don't rely on GCC
__attribute__((optimize)) to disable GCSE"). lz4_deps.h defines
optimize() to nothing for upstream's sources.
Nothing includes either header yet.
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
lib/lz4/freestanding/limits.h | 14 +++++
lib/lz4/freestanding/stddef.h | 15 ++++++
lib/lz4/freestanding/stdint.h | 14 +++++
lib/lz4/freestanding/string.h | 14 +++++
lib/lz4/lz4_deps.h | 118 ++++++++++++++++++++++++++++++++++++++++++
lib/lz4/lz4_kernel_api.h | 90 ++++++++++++++++++++++++++++++++
6 files changed, 265 insertions(+)
diff --git a/lib/lz4/freestanding/limits.h b/lib/lz4/freestanding/limits.h
new file mode 100644
index 0000000000000000000000000000000000000000..0d1cafbc77eb5d2f564b51feaa911748446abf60
--- /dev/null
+++ b/lib/lz4/freestanding/limits.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2026 Samsung Electronics Co., Ltd.
+ * Author: Michal Wilczynski <m.wilczynski@samsung.com>
+ *
+ * Freestanding <limits.h> for the vendored upstream LZ4 sources; lz4.c needs
+ * UINT_MAX and lz4hc.c needs INT_MAX.
+ */
+#ifndef __LZ4_FREESTANDING_LIMITS_H__
+#define __LZ4_FREESTANDING_LIMITS_H__
+
+#include <linux/limits.h>
+
+#endif
diff --git a/lib/lz4/freestanding/stddef.h b/lib/lz4/freestanding/stddef.h
new file mode 100644
index 0000000000000000000000000000000000000000..aaafd2413849210fa4ec4fa03ba0e9453a9fe026
--- /dev/null
+++ b/lib/lz4/freestanding/stddef.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2026 Samsung Electronics Co., Ltd.
+ * Author: Michal Wilczynski <m.wilczynski@samsung.com>
+ *
+ * Freestanding <stddef.h> for the vendored upstream LZ4 sources; -nostdinc
+ * drops the compiler's copy, and lz4.h needs size_t and NULL.
+ */
+#ifndef __LZ4_FREESTANDING_STDDEF_H__
+#define __LZ4_FREESTANDING_STDDEF_H__
+
+#include <linux/stddef.h> /* NULL, offsetof */
+#include <linux/types.h> /* size_t, ptrdiff_t */
+
+#endif
diff --git a/lib/lz4/freestanding/stdint.h b/lib/lz4/freestanding/stdint.h
new file mode 100644
index 0000000000000000000000000000000000000000..923db05b19f23fb4051d16e5df2bf134cd5ac11e
--- /dev/null
+++ b/lib/lz4/freestanding/stdint.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2026 Samsung Electronics Co., Ltd.
+ * Author: Michal Wilczynski <m.wilczynski@samsung.com>
+ *
+ * Freestanding <stdint.h> for the vendored upstream LZ4 sources; forwards to
+ * <linux/types.h>.
+ */
+#ifndef __LZ4_FREESTANDING_STDINT_H__
+#define __LZ4_FREESTANDING_STDINT_H__
+
+#include <linux/types.h>
+
+#endif
diff --git a/lib/lz4/freestanding/string.h b/lib/lz4/freestanding/string.h
new file mode 100644
index 0000000000000000000000000000000000000000..8f3ea3b63013608a780a52fe9c33bf5df557584e
--- /dev/null
+++ b/lib/lz4/freestanding/string.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2026 Samsung Electronics Co., Ltd.
+ * Author: Michal Wilczynski <m.wilczynski@samsung.com>
+ *
+ * Freestanding <string.h> for the vendored upstream LZ4 sources; lz4.c
+ * includes it for memcpy()/memset(), which must resolve under -nostdinc.
+ */
+#ifndef __LZ4_FREESTANDING_STRING_H__
+#define __LZ4_FREESTANDING_STRING_H__
+
+#include <linux/string.h>
+
+#endif
diff --git a/lib/lz4/lz4_deps.h b/lib/lz4/lz4_deps.h
new file mode 100644
index 0000000000000000000000000000000000000000..23d69c75867f71224cb533e9341bd38439b9dee8
--- /dev/null
+++ b/lib/lz4/lz4_deps.h
@@ -0,0 +1,118 @@
+/* SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause */
+/*
+ * Copyright (c) 2026 Samsung Electronics Co., Ltd.
+ * Author: Michal Wilczynski <m.wilczynski@samsung.com>
+ *
+ * Build environment for the vendored upstream LZ4 sources.
+ *
+ * upstream/ is LZ4 v1.10.0 plus three backports: 5ef1f16929b5, the aarch64
+ * fast loop (0aa33884faf5, 51f8af506b4a, ed86bc1ed802) and the match length
+ * limit of 587ed9557708, adapted. Replace it with a later release rather
+ * than patch it further. The three entry point files include this header
+ * first.
+ */
+
+#ifndef __LZ4_DEPS_H__
+#define __LZ4_DEPS_H__
+
+#include <linux/build_bug.h> /* static_assert */
+#include <linux/compiler.h> /* __maybe_unused */
+#include <linux/string.h>
+#include <linux/types.h>
+
+/* lz4.c uses "current" as a local; <asm/current.h> breaks the build. */
+#undef current
+
+/* Static so an object carries only the codec it forwards. Each object
+ * forwards only some of them, so mark them __maybe_unused rather than turn
+ * off -Wunused-function, which the pre-boot builds would not see anyway.
+ */
+#define LZ4LIB_VISIBILITY static __maybe_unused
+
+/* Route upstream's "static linking only" entry points through
+ * LZ4LIB_VISIBILITY too.
+ */
+#define LZ4_PUBLISH_STATIC_FUNCTIONS
+
+/* Heap mode only: the stack mode puts LZ4HC's ~64K opt[] in a frame.
+ * Allocation goes through the failing stubs below; upstream handles
+ * NULL by returning 0.
+ */
+#define LZ4_USER_MEMORY_FUNCTIONS
+#define LZ4_HEAPMODE 1
+#define LZ4HC_HEAPMODE 1
+
+/* Failing stubs; nothing in the kernel reaches these. */
+static void *LZ4_malloc(size_t s) { return NULL; }
+static void *LZ4_calloc(size_t n, size_t s) { return NULL; }
+static void LZ4_free(void *p) { }
+
+/* Left alone, upstream reads unaligned words through packed structs, and
+ * on ARMv6 through plain misaligned dereferences. Use memcpy(), upstream's
+ * documented default, where unaligned access is cheap, and packed structs,
+ * as get_unaligned() does, where it is not: there GCC builds a memcpy()'d
+ * word through a stack slot.
+ */
+#ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
+#define LZ4_FORCE_MEMORY_ACCESS 0
+#else
+#define LZ4_FORCE_MEMORY_ACCESS 1
+#endif
+
+/* On ppc64le GCC, lz4.c marks its decompression entry points and every
+ * LZ4_FORCE_INLINE helper optimize("O2"), which makes GCC forget command
+ * line options such as -fno-strict-aliasing for those functions (see commit
+ * 080b6f407635). Expand the attribute to nothing; lz4_kernel_api.h undoes
+ * this.
+ */
+#define optimize(level)
+
+/* lz4.c sets this before including lz4.h, since its own code calls
+ * deprecated entry points. lz4.h is included below, ahead of lz4.c.
+ */
+#define LZ4_DISABLE_DEPRECATE_WARNINGS 1
+
+/* Rename upstream's entry points out of the way; four of them take a
+ * trailing wrkmem argument in the kernel.
+ */
+#define LZ4_compress_fast __lz4_compress_fast
+#define LZ4_compress_default __lz4_compress_default
+#define LZ4_compress_destSize __lz4_compress_destSize
+#define LZ4_resetStream __lz4_resetStream
+#define LZ4_loadDict __lz4_loadDict
+#define LZ4_saveDict __lz4_saveDict
+#define LZ4_compress_fast_continue __lz4_compress_fast_continue
+
+#define LZ4_decompress_safe __lz4_decompress_safe
+#define LZ4_decompress_safe_partial __lz4_decompress_safe_partial
+#define LZ4_setStreamDecode __lz4_setStreamDecode
+#define LZ4_decompress_safe_continue __lz4_decompress_safe_continue
+#define LZ4_decompress_safe_usingDict __lz4_decompress_safe_usingDict
+
+#define LZ4_compress_HC __lz4_compress_HC
+#define LZ4_resetStreamHC __lz4_resetStreamHC
+#define LZ4_loadDictHC __lz4_loadDictHC
+#define LZ4_compress_HC_continue __lz4_compress_HC_continue
+#define LZ4_saveDictHC __lz4_saveDictHC
+
+/* Upstream declares these bare, so both entry point objects would define
+ * them. Declaring them static first gives them internal linkage (C11
+ * 6.2.2p4): this one in lz4.h, the three below in lz4.c.
+ */
+static __maybe_unused int LZ4_compress_destSize_extState(void *state, const char *src,
+ char *dst, int *srcSizePtr, int targetDstSize,
+ int acceleration);
+
+#include "upstream/lz4.h"
+
+static __maybe_unused int LZ4_compress_forceExtDict(LZ4_stream_t *LZ4_dict,
+ const char *source, char *dest, int srcSize);
+static __maybe_unused int LZ4_decompress_safe_forceExtDict(const char *source,
+ char *dest,
+ int compressedSize, int maxOutputSize,
+ const void *dictStart, size_t dictSize);
+static __maybe_unused int LZ4_decompress_safe_partial_forceExtDict(const char *source,
+ char *dest, int compressedSize, int targetOutputSize,
+ int dstCapacity, const void *dictStart, size_t dictSize);
+
+#endif /* __LZ4_DEPS_H__ */
diff --git a/lib/lz4/lz4_kernel_api.h b/lib/lz4/lz4_kernel_api.h
new file mode 100644
index 0000000000000000000000000000000000000000..de75e5d976eb9dd318da5318a26b91d7aed81042
--- /dev/null
+++ b/lib/lz4/lz4_kernel_api.h
@@ -0,0 +1,90 @@
+/* SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause */
+/*
+ * Copyright (c) 2026 Samsung Electronics Co., Ltd.
+ * Author: Michal Wilczynski <m.wilczynski@samsung.com>
+ *
+ * lz4_kernel_api.h -- the LZ4 API this directory exports
+ *
+ * lz4_deps.h renames upstream's entry points to __lz4_* so that the kernel's
+ * own definitions can use the real names; this header undoes that renaming
+ * and the optimize() override, and declares what lib/lz4 exports.
+ *
+ * The same functions are declared to callers in <linux/lz4.h>, in terms of
+ * opaque stream types. We cannot include that header here -- it and
+ * upstream's lz4.h describe the same library and collide -- so the
+ * declarations are repeated, expressed in upstream's own types. Keep the two
+ * in step; <linux/lz4.h> is the one callers compile against.
+ *
+ * Include after upstream/lz4.c or upstream/lz4hc.c.
+ */
+
+#undef LZ4_compress_fast
+#undef LZ4_compress_default
+#undef LZ4_compress_destSize
+#undef LZ4_resetStream
+#undef LZ4_loadDict
+#undef LZ4_saveDict
+#undef LZ4_compress_fast_continue
+
+#undef LZ4_decompress_safe
+#undef LZ4_decompress_safe_partial
+#undef LZ4_setStreamDecode
+#undef LZ4_decompress_safe_continue
+#undef LZ4_decompress_safe_usingDict
+
+#undef LZ4_compress_HC
+#undef LZ4_resetStreamHC
+#undef LZ4_loadDictHC
+#undef LZ4_compress_HC_continue
+#undef LZ4_saveDictHC
+
+#undef optimize
+
+/*
+ * The objects that do not build lz4hc.c never see this type, and it is only
+ * ever used through a pointer here. Upstream declares it the same way.
+ */
+typedef union LZ4_streamHC_u LZ4_streamHC_t;
+
+/* Compression. wrkmem is LZ4_MEM_COMPRESS bytes, supplied by the caller. */
+int LZ4_compress_default(const char *source, char *dest, int inputSize,
+ int maxOutputSize, void *wrkmem);
+int LZ4_compress_fast(const char *source, char *dest, int inputSize,
+ int maxOutputSize, int acceleration, void *wrkmem);
+int LZ4_compress_destSize(const char *source, char *dest, int *sourceSizePtr,
+ int targetDestSize, void *wrkmem);
+
+/* Streaming compression. */
+void LZ4_resetStream(LZ4_stream_t *LZ4_stream);
+int LZ4_loadDict(LZ4_stream_t *streamPtr, const char *dictionary,
+ int dictSize);
+int LZ4_saveDict(LZ4_stream_t *streamPtr, char *safeBuffer, int dictSize);
+int LZ4_compress_fast_continue(LZ4_stream_t *streamPtr, const char *src,
+ char *dst, int srcSize, int maxDstSize,
+ int acceleration);
+
+/* Decompression. */
+int LZ4_decompress_safe(const char *source, char *dest, int compressedSize,
+ int maxDecompressedSize);
+int LZ4_decompress_safe_partial(const char *source, char *dest,
+ int compressedSize, int targetOutputSize,
+ int maxDecompressedSize);
+int LZ4_setStreamDecode(LZ4_streamDecode_t *LZ4_streamDecode,
+ const char *dictionary, int dictSize);
+int LZ4_decompress_safe_continue(LZ4_streamDecode_t *LZ4_streamDecode,
+ const char *source, char *dest,
+ int compressedSize, int maxDecompressedSize);
+int LZ4_decompress_safe_usingDict(const char *source, char *dest,
+ int compressedSize, int maxDecompressedSize,
+ const char *dictStart, int dictSize);
+
+/* HC compression. wrkmem is LZ4HC_MEM_COMPRESS bytes. */
+int LZ4_compress_HC(const char *src, char *dst, int srcSize, int dstCapacity,
+ int compressionLevel, void *wrkmem);
+void LZ4_resetStreamHC(LZ4_streamHC_t *streamHCPtr, int compressionLevel);
+int LZ4_loadDictHC(LZ4_streamHC_t *streamHCPtr, const char *dictionary,
+ int dictSize);
+int LZ4_compress_HC_continue(LZ4_streamHC_t *streamHCPtr, const char *src,
+ char *dst, int srcSize, int maxDstSize);
+int LZ4_saveDictHC(LZ4_streamHC_t *streamHCPtr, char *safeBuffer,
+ int maxDictSize);
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 09/15] arch: boot: put the LZ4 freestanding headers on the decompressor path
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
` (6 preceding siblings ...)
2026-11-06 14:42 ` [PATCH v2 08/15] lib/lz4: add the build environment for the vendored sources Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 10/15] lib/lz4: switch the compressor to the vendored sources Michal Wilczynski
` (5 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
lib/decompress_unlz4.c pulls lib/lz4/lz4_decompress.c into the pre-boot
decompressor on these five arches. A later patch makes that file include
the vendored LZ4 sources, which include ISO C headers that -nostdinc does
not provide, so put lib/lz4/freestanding on the include path first.
Only the one object that includes decompress_unlz4.c gets it, so the
rest of each boot directory keeps -nostdinc's view.
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
arch/arm/boot/compressed/Makefile | 3 +++
arch/mips/boot/compressed/Makefile | 4 ++++
arch/parisc/boot/compressed/Makefile | 3 +++
arch/s390/boot/Makefile | 3 +++
arch/x86/boot/compressed/Makefile | 3 +++
5 files changed, 16 insertions(+)
diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
index e3f550d6285786c8405b16768255af87b4dc8222..713225008aa71e614af8c9c6472fda180354569b 100644
--- a/arch/arm/boot/compressed/Makefile
+++ b/arch/arm/boot/compressed/Makefile
@@ -92,6 +92,9 @@ targets := vmlinux vmlinux.lds piggy_data piggy.o \
head.o $(OBJS)
KBUILD_CFLAGS += -DDISABLE_BRANCH_PROFILING
+# decompress.c includes decompress_unlz4.c, which builds the vendored LZ4
+# sources; they include a few ISO C headers -nostdinc drops.
+CFLAGS_decompress.o += -I$(srctree)/lib/lz4/freestanding
ccflags-y := -fpic $(call cc-option,-mno-single-pic-base,) -fno-builtin \
-I$(srctree)/scripts/dtc/libfdt -fno-stack-protector \
diff --git a/arch/mips/boot/compressed/Makefile b/arch/mips/boot/compressed/Makefile
index e0b8ec9a9516281933b6cc5b175e82f4013c2f54..48f8a63c54a66826889634a54d3537d735f63512 100644
--- a/arch/mips/boot/compressed/Makefile
+++ b/arch/mips/boot/compressed/Makefile
@@ -30,6 +30,10 @@ endif
KBUILD_CFLAGS := $(KBUILD_CFLAGS) -D__KERNEL__ -D__DISABLE_EXPORTS \
-DBOOT_HEAP_SIZE=$(BOOT_HEAP_SIZE) -D"VMLINUX_LOAD_ADDRESS_ULL=$(VMLINUX_LOAD_ADDRESS)ull"
+# decompress.c includes decompress_unlz4.c, which builds the vendored LZ4
+# sources; they include a few ISO C headers -nostdinc drops.
+CFLAGS_decompress.o += -I$(srctree)/lib/lz4/freestanding
+
KBUILD_AFLAGS := $(KBUILD_AFLAGS) -D__ASSEMBLY__ \
-DBOOT_HEAP_SIZE=$(BOOT_HEAP_SIZE) \
-DKERNEL_ENTRY=$(VMLINUX_ENTRY_ADDRESS)
diff --git a/arch/parisc/boot/compressed/Makefile b/arch/parisc/boot/compressed/Makefile
index 14eefb5ed5d1d781a4ee0cb2ef61ab86c905cc56..9e82f9d1285f62b164073bdacc0d4a1828d4965c 100644
--- a/arch/parisc/boot/compressed/Makefile
+++ b/arch/parisc/boot/compressed/Makefile
@@ -12,6 +12,9 @@ targets += $(OBJECTS) sizes.h
KBUILD_CFLAGS := -D__KERNEL__ -O2 -DBOOTLOADER
KBUILD_CFLAGS += -DDISABLE_BRANCH_PROFILING
+# misc.c includes decompress_unlz4.c, which builds the vendored LZ4
+# sources; they include a few ISO C headers -nostdinc drops.
+CFLAGS_misc.o += -I$(srctree)/lib/lz4/freestanding
KBUILD_CFLAGS += -fno-strict-aliasing
KBUILD_CFLAGS += $(cflags-y) -fno-delete-null-pointer-checks -fno-builtin-printf
KBUILD_CFLAGS += -fno-PIE -mno-space-regs -mdisable-fpregs -Os
diff --git a/arch/s390/boot/Makefile b/arch/s390/boot/Makefile
index 10b75e053a6f6bb9083548e8f82796f0aa9a9958..44355051a8c7f96b5c4dab5d110bc8aaf79c8f55 100644
--- a/arch/s390/boot/Makefile
+++ b/arch/s390/boot/Makefile
@@ -21,6 +21,9 @@ KBUILD_AFLAGS := $(filter-out $(CC_FLAGS_MARCH),$(KBUILD_AFLAGS_DECOMPRESSOR))
KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_MARCH),$(KBUILD_CFLAGS_DECOMPRESSOR))
KBUILD_AFLAGS += $(CC_FLAGS_MARCH_MINIMUM) -D__DISABLE_EXPORTS
KBUILD_CFLAGS += $(CC_FLAGS_MARCH_MINIMUM) -D__DISABLE_EXPORTS
+# decompressor.c includes decompress_unlz4.c, which builds the vendored LZ4
+# sources; they include a few ISO C headers -nostdinc drops.
+CFLAGS_decompressor.o += -I$(srctree)/lib/lz4/freestanding
KBUILD_CFLAGS += $(call cc-option, -Wno-default-const-init-unsafe)
CFLAGS_sclp_early_core.o += -I$(srctree)/drivers/s390/char
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
index 06934f9691d6a9d6952532ae5f7d96f5858719dd..328eafcfc35bd14a16bc4ec5472f6c6dc72336a1 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -30,6 +30,9 @@ KBUILD_CFLAGS += -fno-strict-aliasing -fPIE
KBUILD_CFLAGS += -fno-jump-tables
KBUILD_CFLAGS += -Wundef
KBUILD_CFLAGS += -DDISABLE_BRANCH_PROFILING
+# misc.c includes decompress_unlz4.c, which builds the vendored LZ4
+# sources; they include a few ISO C headers -nostdinc drops.
+CFLAGS_misc.o += -I$(srctree)/lib/lz4/freestanding
cflags-$(CONFIG_X86_32) := -march=i386
cflags-$(CONFIG_X86_64) := -mcmodel=small -mno-red-zone
KBUILD_CFLAGS += $(cflags-y)
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 10/15] lib/lz4: switch the compressor to the vendored sources
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
` (7 preceding siblings ...)
2026-11-06 14:42 ` [PATCH v2 09/15] arch: boot: put the LZ4 freestanding headers on the decompressor path Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
2026-11-07 1:40 ` kernel test robot
2026-11-10 5:36 ` Sergey Senozhatsky
2026-11-06 14:42 ` [PATCH v2 11/15] lib/lz4: switch the HC " Michal Wilczynski
` (4 subsequent siblings)
13 siblings, 2 replies; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
Replace the forked compressor with thin entry points over the vendored
lz4.c. LZ4_compress_default(), LZ4_compress_fast() and
LZ4_compress_destSize() keep the trailing wrkmem argument and forward to
upstream's *_extState() functions; the rest are plain forwarders.
Upstream's LZ4_initStream() returns NULL for a misaligned workspace,
which LZ4_compress_fast_extState() dereferences and
LZ4_compress_destSize_extState() ignores. Check the alignment in the
entry points, warn once and fail the call. In-tree callers allocate with
kmalloc() or vmalloc() and are not affected.
Compressed output differs from the fork's on a small fraction of inputs
by a byte or two; both decode each other's output. Compressing 4K
blocks, upstream is 2-4% faster on kernel source and binaries and up to
9% slower on datagen.
LZ4_stream_t becomes an incomplete type, as upstream declares it, so
<linux/lz4.h> and upstream's lz4.h can share a translation unit. The
macros that laid out the forked structure go, and LZ4_MEM_COMPRESS keeps
its value, 16416. zram, the only caller using sizeof(LZ4_stream_t), now
allocates LZ4_MEM_COMPRESS bytes.
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
drivers/block/zram/backend_lz4.c | 6 +-
include/linux/lz4.h | 47 +-
lib/lz4/Makefile | 12 +-
lib/lz4/lz4_compress.c | 946 +++------------------------------------
4 files changed, 80 insertions(+), 931 deletions(-)
diff --git a/drivers/block/zram/backend_lz4.c b/drivers/block/zram/backend_lz4.c
index 1e4ad31d39a63f45b3bddd3d8a467d8d7d095beb..50265e3ce256490994cb6dae32a45da376e408e0 100644
--- a/drivers/block/zram/backend_lz4.c
+++ b/drivers/block/zram/backend_lz4.c
@@ -42,7 +42,7 @@ static int lz4_setup_params(struct zcomp_params *params)
if (!params->dict || !params->dict_sz)
return 0;
- dict_stream = kzalloc_obj(*dict_stream);
+ dict_stream = kzalloc(LZ4_MEM_COMPRESS, GFP_KERNEL);
if (!dict_stream)
return -ENOMEM;
@@ -88,7 +88,7 @@ static int lz4_create(struct zcomp_params *params, struct zcomp_ctx *ctx)
if (!zctx->dstrm)
goto error;
- zctx->cstrm = kzalloc_obj(*zctx->cstrm);
+ zctx->cstrm = kzalloc(LZ4_MEM_COMPRESS, GFP_KERNEL);
if (!zctx->cstrm)
goto error;
}
@@ -112,7 +112,7 @@ static int lz4_compress(struct zcomp_params *params, struct zcomp_ctx *ctx,
zctx->mem);
} else {
/* Cstrm needs to be reset */
- memcpy(zctx->cstrm, params->drv_data, sizeof(*zctx->cstrm));
+ memcpy(zctx->cstrm, params->drv_data, LZ4_MEM_COMPRESS);
ret = LZ4_compress_fast_continue(zctx->cstrm, req->src,
req->dst, req->src_len,
req->dst_len, params->level);
diff --git a/include/linux/lz4.h b/include/linux/lz4.h
index 0df69e2b1ad523c291e3a96ab5595df9239f7252..031a410b8e3878e9a24cc49af36cfcbba399d991 100644
--- a/include/linux/lz4.h
+++ b/include/linux/lz4.h
@@ -47,16 +47,6 @@
/*-************************************************************************
* CONSTANTS
**************************************************************************/
-/*
- * LZ4_MEMORY_USAGE :
- * Memory usage formula : N->2^N Bytes
- * (examples : 10 -> 1KB; 12 -> 4KB ; 16 -> 64KB; 20 -> 1MB; etc.)
- * Increasing memory usage improves compression ratio
- * Reduced memory usage can improve speed, due to cache effect
- * Default value is 14, for 16KB, which nicely fits into Intel x86 L1 cache
- */
-#define LZ4_MEMORY_USAGE 14
-
#define LZ4_MAX_INPUT_SIZE 0x7E000000 /* 2 113 929 216 bytes */
#define LZ4_COMPRESSBOUND(isize) (\
(unsigned int)(isize) > (unsigned int)LZ4_MAX_INPUT_SIZE \
@@ -64,9 +54,6 @@
: (isize) + ((isize)/255) + 16)
#define LZ4_ACCELERATION_DEFAULT 1
-#define LZ4_HASHLOG (LZ4_MEMORY_USAGE-2)
-#define LZ4_HASHTABLESIZE (1 << LZ4_MEMORY_USAGE)
-#define LZ4_HASH_SIZE_U32 (1 << LZ4_HASHLOG)
#define LZ4HC_MIN_CLEVEL 3
#define LZ4HC_DEFAULT_CLEVEL 9
@@ -82,9 +69,6 @@
/*-************************************************************************
* STREAMING CONSTANTS AND STRUCTURES
**************************************************************************/
-#define LZ4_STREAMSIZE_U64 ((1 << (LZ4_MEMORY_USAGE - 3)) + 4)
-#define LZ4_STREAMSIZE (LZ4_STREAMSIZE_U64 * sizeof(unsigned long long))
-
#define LZ4_STREAMHCSIZE 262192
#define LZ4_STREAMHCSIZE_SIZET (262192 / sizeof(size_t))
@@ -93,20 +77,10 @@
sizeof(unsigned long long))
/*
- * LZ4_stream_t - information structure to track an LZ4 stream.
+ * LZ4_stream_t - an LZ4 stream. Incomplete: lib/lz4 owns the layout.
+ * Allocate LZ4_MEM_COMPRESS bytes and cast, do not sizeof().
*/
-typedef struct {
- uint32_t hashTable[LZ4_HASH_SIZE_U32];
- uint32_t currentOffset;
- uint32_t initCheck;
- const uint8_t *dictionary;
- uint8_t *bufferStart;
- uint32_t dictSize;
-} LZ4_stream_t_internal;
-typedef union {
- unsigned long long table[LZ4_STREAMSIZE_U64];
- LZ4_stream_t_internal internal_donotuse;
-} LZ4_stream_t;
+typedef union LZ4_stream_u LZ4_stream_t;
/*
* LZ4_streamHC_t - information structure to track an LZ4HC stream.
@@ -153,7 +127,12 @@ typedef union {
/*-************************************************************************
* SIZE OF STATE
**************************************************************************/
-#define LZ4_MEM_COMPRESS LZ4_STREAMSIZE
+/*
+ * Working memory for the compressors. It must be aligned to at least 8
+ * bytes; anything from kmalloc() or vmalloc() already is. The entry
+ * points warn about a misaligned buffer and fail the call.
+ */
+#define LZ4_MEM_COMPRESS 16416
#define LZ4HC_MEM_COMPRESS LZ4_STREAMHCSIZE
/*-************************************************************************
@@ -180,7 +159,7 @@ static inline int LZ4_compressBound(size_t isize)
* @maxOutputSize: full or partial size of buffer 'dest'
* which must be already allocated
* @wrkmem: address of the working memory.
- * This requires 'workmem' of LZ4_MEM_COMPRESS.
+ * This requires 'workmem' of LZ4_MEM_COMPRESS, aligned to 8 bytes.
*
* Compresses 'sourceSize' bytes from buffer 'source'
* into already allocated 'dest' buffer of size 'maxOutputSize'.
@@ -206,7 +185,7 @@ int LZ4_compress_default(const char *source, char *dest, int inputSize,
* which must be already allocated
* @acceleration: acceleration factor
* @wrkmem: address of the working memory.
- * This requires 'workmem' of LZ4_MEM_COMPRESS.
+ * This requires 'workmem' of LZ4_MEM_COMPRESS, aligned to 8 bytes.
*
* Same as LZ4_compress_default(), but allows to select an "acceleration"
* factor. The larger the acceleration value, the faster the algorithm,
@@ -230,7 +209,7 @@ int LZ4_compress_fast(const char *source, char *dest, int inputSize,
* from 'source' to fill 'dest'. New value is necessarily <= old value.
* @targetDestSize: Size of buffer 'dest' which must be already allocated
* @wrkmem: address of the working memory.
- * This requires 'workmem' of LZ4_MEM_COMPRESS.
+ * This requires 'workmem' of LZ4_MEM_COMPRESS, aligned to 8 bytes.
*
* Reverse the logic, by compressing as much data as possible
* from 'source' buffer into already allocated buffer 'dest'
@@ -314,7 +293,7 @@ int LZ4_decompress_safe_partial(const char *source, char *dest,
* value between 1 and LZ4HC_MAX_CLEVEL will work.
* Values >LZ4HC_MAX_CLEVEL behave the same as 16.
* @wrkmem: address of the working memory.
- * This requires 'wrkmem' of size LZ4HC_MEM_COMPRESS.
+ * This requires 'wrkmem' of size LZ4HC_MEM_COMPRESS, aligned to 8 bytes.
*
* Compress data from 'src' into 'dst', using the more powerful
* but slower "HC" algorithm. Compression is guaranteed to succeed if
diff --git a/lib/lz4/Makefile b/lib/lz4/Makefile
index 5b42242afaa20deb165d8a2f0b1c0c842dfcdc39..8b4eb07694e67dd99ec7cc85212fb68138d3981e 100644
--- a/lib/lz4/Makefile
+++ b/lib/lz4/Makefile
@@ -1,6 +1,12 @@
# SPDX-License-Identifier: GPL-2.0-only
ccflags-y += -O3
-obj-$(CONFIG_LZ4_COMPRESS) += lz4_compress.o
-obj-$(CONFIG_LZ4HC_COMPRESS) += lz4hc_compress.o
-obj-$(CONFIG_LZ4_DECOMPRESS) += lz4_decompress.o
+# upstream/ holds the upstream LZ4 sources lz4_deps.h describes; replace them
+# with a later release rather than patch them further. The three entry point
+# files include them. freestanding/ supplies the ISO C headers upstream
+# includes that -nostdinc does not provide.
+ccflags-y += -I $(src)/freestanding
+
+obj-$(CONFIG_LZ4_COMPRESS) += lz4_compress.o
+obj-$(CONFIG_LZ4HC_COMPRESS) += lz4hc_compress.o
+obj-$(CONFIG_LZ4_DECOMPRESS) += lz4_decompress.o
diff --git a/lib/lz4/lz4_compress.c b/lib/lz4/lz4_compress.c
index 2a397bb2c661d9c2e4a7a67d5f50f06bfd42f1f2..941e32a19ac972aef543c9a03b447a8a98bf5215 100644
--- a/lib/lz4/lz4_compress.c
+++ b/lib/lz4/lz4_compress.c
@@ -1,937 +1,101 @@
+// SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
/*
- * LZ4 - Fast LZ compression algorithm
* Copyright (C) 2011 - 2016, Yann Collet.
- * BSD 2 - Clause License (http://www.opensource.org/licenses/bsd - license.php)
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are
- * met:
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above
- * copyright notice, this list of conditions and the following disclaimer
- * in the documentation and/or other materials provided with the
- * distribution.
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- * You can contact the author at :
- * - LZ4 homepage : http://www.lz4.org
- * - LZ4 source repository : https://github.com/lz4/lz4
+ * Copyright (C) 2016, Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
+ * Copyright (c) 2026 Samsung Electronics Co., Ltd.
+ * Author: Michal Wilczynski <m.wilczynski@samsung.com>
*
- * Changed for kernel usage by:
- * Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
+ * LZ4 compressor -- kernel entry points
+ *
+ * The compressor is upstream's, in the verbatim lz4.c included below. The
+ * kernel has no heap at these call sites, so LZ4_compress_default(),
+ * LZ4_compress_fast() and LZ4_compress_destSize() take a trailing wrkmem
+ * argument and forward to upstream's *_extState() entry points.
*/
-/*-************************************
- * Dependencies
- **************************************/
-#include "lz4defs.h"
-#include <linux/module.h>
-#include <linux/kernel.h>
-#include <linux/unaligned.h>
-
-static const int LZ4_minLength = (MFLIMIT + 1);
-static const int LZ4_64Klimit = ((64 * KB) + (MFLIMIT - 1));
-
-/*-******************************
- * Compression functions
- ********************************/
-static FORCE_INLINE U32 LZ4_hash4(
- U32 sequence,
- tableType_t const tableType)
-{
- if (tableType == byU16)
- return ((sequence * 2654435761U)
- >> ((MINMATCH * 8) - (LZ4_HASHLOG + 1)));
- else
- return ((sequence * 2654435761U)
- >> ((MINMATCH * 8) - LZ4_HASHLOG));
-}
-
-static FORCE_INLINE U32 LZ4_hash5(
- U64 sequence,
- tableType_t const tableType)
-{
- const U32 hashLog = (tableType == byU16)
- ? LZ4_HASHLOG + 1
- : LZ4_HASHLOG;
-
-#if LZ4_LITTLE_ENDIAN
- static const U64 prime5bytes = 889523592379ULL;
-
- return (U32)(((sequence << 24) * prime5bytes) >> (64 - hashLog));
-#else
- static const U64 prime8bytes = 11400714785074694791ULL;
-
- return (U32)(((sequence >> 24) * prime8bytes) >> (64 - hashLog));
-#endif
-}
-
-static FORCE_INLINE U32 LZ4_hashPosition(
- const void *p,
- tableType_t const tableType)
-{
-#if LZ4_ARCH64
- if (tableType == byU32)
- return LZ4_hash5(LZ4_read_ARCH(p), tableType);
-#endif
-
- return LZ4_hash4(LZ4_read32(p), tableType);
-}
-
-static void LZ4_putPositionOnHash(
- const BYTE *p,
- U32 h,
- void *tableBase,
- tableType_t const tableType,
- const BYTE *srcBase)
-{
- switch (tableType) {
- case byPtr:
- {
- const BYTE **hashTable = (const BYTE **)tableBase;
-
- hashTable[h] = p;
- return;
- }
- case byU32:
- {
- U32 *hashTable = (U32 *) tableBase;
-
- hashTable[h] = (U32)(p - srcBase);
- return;
- }
- case byU16:
- {
- U16 *hashTable = (U16 *) tableBase;
-
- hashTable[h] = (U16)(p - srcBase);
- return;
- }
- }
-}
-
-static FORCE_INLINE void LZ4_putPosition(
- const BYTE *p,
- void *tableBase,
- tableType_t tableType,
- const BYTE *srcBase)
-{
- U32 const h = LZ4_hashPosition(p, tableType);
+#include "lz4_deps.h"
+#include "upstream/lz4.c"
- LZ4_putPositionOnHash(p, h, tableBase, tableType, srcBase);
-}
-
-static const BYTE *LZ4_getPositionOnHash(
- U32 h,
- void *tableBase,
- tableType_t tableType,
- const BYTE *srcBase)
-{
- if (tableType == byPtr) {
- const BYTE **hashTable = (const BYTE **) tableBase;
+/* Upstream's short helper macros; MIN and MAX clash with <linux/minmax.h>. */
+#undef MIN
+#undef MAX
+#undef KB
+#undef MB
+#undef GB
- return hashTable[h];
- }
-
- if (tableType == byU32) {
- const U32 * const hashTable = (U32 *) tableBase;
-
- return hashTable[h] + srcBase;
- }
-
- {
- /* default, to ensure a return */
- const U16 * const hashTable = (U16 *) tableBase;
-
- return hashTable[h] + srcBase;
- }
-}
-
-static FORCE_INLINE const BYTE *LZ4_getPosition(
- const BYTE *p,
- void *tableBase,
- tableType_t tableType,
- const BYTE *srcBase)
-{
- U32 const h = LZ4_hashPosition(p, tableType);
-
- return LZ4_getPositionOnHash(h, tableBase, tableType, srcBase);
-}
+#include "lz4_kernel_api.h"
+#include <linux/bug.h>
+#include <linux/export.h>
+#include <linux/module.h>
+/* Catch any divergence from upstream's layout at build time. */
+static_assert(sizeof(LZ4_stream_t) == LZ4_STREAM_MINSIZE);
+static_assert(LZ4_STREAM_MINSIZE == 16416); /* LZ4_MEM_COMPRESS */
-/*
- * LZ4_compress_generic() :
- * inlined, to ensure branches are decided at compilation time
+/* LZ4_initStream() returns NULL for a misaligned state, which
+ * LZ4_compress_fast_extState() dereferences and
+ * LZ4_compress_destSize_extState() ignores, behind assert()s the kernel
+ * compiles out. Fail the call instead.
*/
-static FORCE_INLINE int LZ4_compress_generic(
- LZ4_stream_t_internal * const dictPtr,
- const char * const source,
- char * const dest,
- const int inputSize,
- const int maxOutputSize,
- const limitedOutput_directive outputLimited,
- const tableType_t tableType,
- const dict_directive dict,
- const dictIssue_directive dictIssue,
- const U32 acceleration)
+static bool lz4_wrkmem_misaligned(const void *wrkmem)
{
- const BYTE *ip = (const BYTE *) source;
- const BYTE *base;
- const BYTE *lowLimit;
- const BYTE * const lowRefLimit = ip - dictPtr->dictSize;
- const BYTE * const dictionary = dictPtr->dictionary;
- const BYTE * const dictEnd = dictionary + dictPtr->dictSize;
- const size_t dictDelta = dictEnd - (const BYTE *)source;
- const BYTE *anchor = (const BYTE *) source;
- const BYTE * const iend = ip + inputSize;
- const BYTE * const mflimit = iend - MFLIMIT;
- const BYTE * const matchlimit = iend - LASTLITERALS;
-
- BYTE *op = (BYTE *) dest;
- BYTE * const olimit = op + maxOutputSize;
-
- U32 forwardH;
- size_t refDelta = 0;
-
- /* Init conditions */
- if ((U32)inputSize > (U32)LZ4_MAX_INPUT_SIZE) {
- /* Unsupported inputSize, too large (or negative) */
- return 0;
- }
-
- switch (dict) {
- case noDict:
- default:
- base = (const BYTE *)source;
- lowLimit = (const BYTE *)source;
- break;
- case withPrefix64k:
- base = (const BYTE *)source - dictPtr->currentOffset;
- lowLimit = (const BYTE *)source - dictPtr->dictSize;
- break;
- case usingExtDict:
- base = (const BYTE *)source - dictPtr->currentOffset;
- lowLimit = (const BYTE *)source;
- break;
- }
-
- if ((tableType == byU16)
- && (inputSize >= LZ4_64Klimit)) {
- /* Size too large (not within 64K limit) */
- return 0;
- }
-
- if (inputSize < LZ4_minLength) {
- /* Input too small, no compression (all literals) */
- goto _last_literals;
- }
-
- /* First Byte */
- LZ4_putPosition(ip, dictPtr->hashTable, tableType, base);
- ip++;
- forwardH = LZ4_hashPosition(ip, tableType);
-
- /* Main Loop */
- for ( ; ; ) {
- const BYTE *match;
- BYTE *token;
-
- /* Find a match */
- {
- const BYTE *forwardIp = ip;
- unsigned int step = 1;
- unsigned int searchMatchNb = acceleration << LZ4_SKIPTRIGGER;
-
- do {
- U32 const h = forwardH;
-
- ip = forwardIp;
- forwardIp += step;
- step = (searchMatchNb++ >> LZ4_SKIPTRIGGER);
-
- if (unlikely(forwardIp > mflimit))
- goto _last_literals;
-
- match = LZ4_getPositionOnHash(h,
- dictPtr->hashTable,
- tableType, base);
-
- if (dict == usingExtDict) {
- if (match < (const BYTE *)source) {
- refDelta = dictDelta;
- lowLimit = dictionary;
- } else {
- refDelta = 0;
- lowLimit = (const BYTE *)source;
- } }
-
- forwardH = LZ4_hashPosition(forwardIp,
- tableType);
-
- LZ4_putPositionOnHash(ip, h, dictPtr->hashTable,
- tableType, base);
- } while (((dictIssue == dictSmall)
- ? (match < lowRefLimit)
- : 0)
- || ((tableType == byU16)
- ? 0
- : (match + MAX_DISTANCE < ip))
- || (LZ4_read32(match + refDelta)
- != LZ4_read32(ip)));
- }
-
- /* Catch up */
- while (((ip > anchor) & (match + refDelta > lowLimit))
- && (unlikely(ip[-1] == match[refDelta - 1]))) {
- ip--;
- match--;
- }
-
- /* Encode Literals */
- {
- unsigned const int litLength = (unsigned int)(ip - anchor);
-
- token = op++;
-
- if ((outputLimited) &&
- /* Check output buffer overflow */
- (unlikely(op + litLength +
- (2 + 1 + LASTLITERALS) +
- (litLength / 255) > olimit)))
- return 0;
-
- if (litLength >= RUN_MASK) {
- int len = (int)litLength - RUN_MASK;
-
- *token = (RUN_MASK << ML_BITS);
-
- for (; len >= 255; len -= 255)
- *op++ = 255;
- *op++ = (BYTE)len;
- } else
- *token = (BYTE)(litLength << ML_BITS);
-
- /* Copy Literals */
- LZ4_wildCopy(op, anchor, op + litLength);
- op += litLength;
- }
-
-_next_match:
- /* Encode Offset */
- LZ4_writeLE16(op, (U16)(ip - match));
- op += 2;
-
- /* Encode MatchLength */
- {
- unsigned int matchCode;
-
- if ((dict == usingExtDict)
- && (lowLimit == dictionary)) {
- const BYTE *limit;
-
- match += refDelta;
- limit = ip + (dictEnd - match);
-
- if (limit > matchlimit)
- limit = matchlimit;
-
- matchCode = LZ4_count(ip + MINMATCH,
- match + MINMATCH, limit);
-
- ip += MINMATCH + matchCode;
-
- if (ip == limit) {
- unsigned const int more = LZ4_count(ip,
- (const BYTE *)source,
- matchlimit);
-
- matchCode += more;
- ip += more;
- }
- } else {
- matchCode = LZ4_count(ip + MINMATCH,
- match + MINMATCH, matchlimit);
- ip += MINMATCH + matchCode;
- }
-
- if (outputLimited &&
- /* Check output buffer overflow */
- (unlikely(op +
- (1 + LASTLITERALS) +
- (matchCode >> 8) > olimit)))
- return 0;
-
- if (matchCode >= ML_MASK) {
- *token += ML_MASK;
- matchCode -= ML_MASK;
- LZ4_write32(op, 0xFFFFFFFF);
-
- while (matchCode >= 4 * 255) {
- op += 4;
- LZ4_write32(op, 0xFFFFFFFF);
- matchCode -= 4 * 255;
- }
-
- op += matchCode / 255;
- *op++ = (BYTE)(matchCode % 255);
- } else
- *token += (BYTE)(matchCode);
- }
-
- anchor = ip;
-
- /* Test end of chunk */
- if (ip > mflimit)
- break;
-
- /* Fill table */
- LZ4_putPosition(ip - 2, dictPtr->hashTable, tableType, base);
-
- /* Test next position */
- match = LZ4_getPosition(ip, dictPtr->hashTable,
- tableType, base);
-
- if (dict == usingExtDict) {
- if (match < (const BYTE *)source) {
- refDelta = dictDelta;
- lowLimit = dictionary;
- } else {
- refDelta = 0;
- lowLimit = (const BYTE *)source;
- }
- }
-
- LZ4_putPosition(ip, dictPtr->hashTable, tableType, base);
-
- if (((dictIssue == dictSmall) ? (match >= lowRefLimit) : 1)
- && (match + MAX_DISTANCE >= ip)
- && (LZ4_read32(match + refDelta) == LZ4_read32(ip))) {
- token = op++;
- *token = 0;
- goto _next_match;
- }
-
- /* Prepare next loop */
- forwardH = LZ4_hashPosition(++ip, tableType);
- }
-
-_last_literals:
- /* Encode Last Literals */
- {
- size_t const lastRun = (size_t)(iend - anchor);
-
- if ((outputLimited) &&
- /* Check output buffer overflow */
- ((op - (BYTE *)dest) + lastRun + 1 +
- ((lastRun + 255 - RUN_MASK) / 255) > (U32)maxOutputSize))
- return 0;
-
- if (lastRun >= RUN_MASK) {
- size_t accumulator = lastRun - RUN_MASK;
- *op++ = RUN_MASK << ML_BITS;
- for (; accumulator >= 255; accumulator -= 255)
- *op++ = 255;
- *op++ = (BYTE) accumulator;
- } else {
- *op++ = (BYTE)(lastRun << ML_BITS);
- }
-
- LZ4_memcpy(op, anchor, lastRun);
-
- op += lastRun;
- }
-
- /* End */
- return (int) (((char *)op) - dest);
-}
-
-static int LZ4_compress_fast_extState(
- void *state,
- const char *source,
- char *dest,
- int inputSize,
- int maxOutputSize,
- int acceleration)
-{
- LZ4_stream_t_internal *ctx = &((LZ4_stream_t *)state)->internal_donotuse;
-#if LZ4_ARCH64
- const tableType_t tableType = byU32;
-#else
- const tableType_t tableType = byPtr;
-#endif
-
- LZ4_resetStream((LZ4_stream_t *)state);
-
- if (acceleration < 1)
- acceleration = LZ4_ACCELERATION_DEFAULT;
-
- if (maxOutputSize >= LZ4_COMPRESSBOUND(inputSize)) {
- if (inputSize < LZ4_64Klimit)
- return LZ4_compress_generic(ctx, source,
- dest, inputSize, 0,
- noLimit, byU16, noDict,
- noDictIssue, acceleration);
- else
- return LZ4_compress_generic(ctx, source,
- dest, inputSize, 0,
- noLimit, tableType, noDict,
- noDictIssue, acceleration);
- } else {
- if (inputSize < LZ4_64Klimit)
- return LZ4_compress_generic(ctx, source,
- dest, inputSize,
- maxOutputSize, limitedOutput, byU16, noDict,
- noDictIssue, acceleration);
- else
- return LZ4_compress_generic(ctx, source,
- dest, inputSize,
- maxOutputSize, limitedOutput, tableType, noDict,
- noDictIssue, acceleration);
- }
+ return WARN_ON_ONCE(!LZ4_isAligned(wrkmem, LZ4_stream_t_alignment()));
}
int LZ4_compress_fast(const char *source, char *dest, int inputSize,
- int maxOutputSize, int acceleration, void *wrkmem)
+ int maxOutputSize, int acceleration, void *wrkmem)
{
+ if (lz4_wrkmem_misaligned(wrkmem))
+ return 0;
+
return LZ4_compress_fast_extState(wrkmem, source, dest, inputSize,
- maxOutputSize, acceleration);
+ maxOutputSize, acceleration);
}
EXPORT_SYMBOL(LZ4_compress_fast);
int LZ4_compress_default(const char *source, char *dest, int inputSize,
- int maxOutputSize, void *wrkmem)
+ int maxOutputSize, void *wrkmem)
{
- return LZ4_compress_fast(source, dest, inputSize,
- maxOutputSize, LZ4_ACCELERATION_DEFAULT, wrkmem);
+ return LZ4_compress_fast(source, dest, inputSize, maxOutputSize,
+ LZ4_ACCELERATION_DEFAULT, wrkmem);
}
EXPORT_SYMBOL(LZ4_compress_default);
-/*-******************************
- * *_destSize() variant
- ********************************/
-static int LZ4_compress_destSize_generic(
- LZ4_stream_t_internal * const ctx,
- const char * const src,
- char * const dst,
- int * const srcSizePtr,
- const int targetDstSize,
- const tableType_t tableType)
+int LZ4_compress_destSize(const char *source, char *dest, int *sourceSizePtr,
+ int targetDestSize, void *wrkmem)
{
- const BYTE *ip = (const BYTE *) src;
- const BYTE *base = (const BYTE *) src;
- const BYTE *lowLimit = (const BYTE *) src;
- const BYTE *anchor = ip;
- const BYTE * const iend = ip + *srcSizePtr;
- const BYTE * const mflimit = iend - MFLIMIT;
- const BYTE * const matchlimit = iend - LASTLITERALS;
-
- BYTE *op = (BYTE *) dst;
- BYTE * const oend = op + targetDstSize;
- BYTE * const oMaxLit = op + targetDstSize - 2 /* offset */
- - 8 /* because 8 + MINMATCH == MFLIMIT */ - 1 /* token */;
- BYTE * const oMaxMatch = op + targetDstSize
- - (LASTLITERALS + 1 /* token */);
- BYTE * const oMaxSeq = oMaxLit - 1 /* token */;
-
- U32 forwardH;
-
- /* Init conditions */
- /* Impossible to store anything */
- if (targetDstSize < 1)
- return 0;
- /* Unsupported input size, too large (or negative) */
- if ((U32)*srcSizePtr > (U32)LZ4_MAX_INPUT_SIZE)
- return 0;
- /* Size too large (not within 64K limit) */
- if ((tableType == byU16) && (*srcSizePtr >= LZ4_64Klimit))
+ if (lz4_wrkmem_misaligned(wrkmem))
return 0;
- /* Input too small, no compression (all literals) */
- if (*srcSizePtr < LZ4_minLength)
- goto _last_literals;
-
- /* First Byte */
- *srcSizePtr = 0;
- LZ4_putPosition(ip, ctx->hashTable, tableType, base);
- ip++; forwardH = LZ4_hashPosition(ip, tableType);
-
- /* Main Loop */
- for ( ; ; ) {
- const BYTE *match;
- BYTE *token;
-
- /* Find a match */
- {
- const BYTE *forwardIp = ip;
- unsigned int step = 1;
- unsigned int searchMatchNb = 1 << LZ4_SKIPTRIGGER;
-
- do {
- U32 h = forwardH;
-
- ip = forwardIp;
- forwardIp += step;
- step = (searchMatchNb++ >> LZ4_SKIPTRIGGER);
-
- if (unlikely(forwardIp > mflimit))
- goto _last_literals;
-
- match = LZ4_getPositionOnHash(h, ctx->hashTable,
- tableType, base);
- forwardH = LZ4_hashPosition(forwardIp,
- tableType);
- LZ4_putPositionOnHash(ip, h,
- ctx->hashTable, tableType,
- base);
-
- } while (((tableType == byU16)
- ? 0
- : (match + MAX_DISTANCE < ip))
- || (LZ4_read32(match) != LZ4_read32(ip)));
- }
-
- /* Catch up */
- while ((ip > anchor)
- && (match > lowLimit)
- && (unlikely(ip[-1] == match[-1]))) {
- ip--;
- match--;
- }
-
- /* Encode Literal length */
- {
- unsigned int litLength = (unsigned int)(ip - anchor);
-
- token = op++;
- if (op + ((litLength + 240) / 255)
- + litLength > oMaxLit) {
- /* Not enough space for a last match */
- op--;
- goto _last_literals;
- }
- if (litLength >= RUN_MASK) {
- unsigned int len = litLength - RUN_MASK;
- *token = (RUN_MASK<<ML_BITS);
- for (; len >= 255; len -= 255)
- *op++ = 255;
- *op++ = (BYTE)len;
- } else
- *token = (BYTE)(litLength << ML_BITS);
-
- /* Copy Literals */
- LZ4_wildCopy(op, anchor, op + litLength);
- op += litLength;
- }
-
-_next_match:
- /* Encode Offset */
- LZ4_writeLE16(op, (U16)(ip - match)); op += 2;
-
- /* Encode MatchLength */
- {
- size_t matchLength = LZ4_count(ip + MINMATCH,
- match + MINMATCH, matchlimit);
-
- if (op + ((matchLength + 240)/255) > oMaxMatch) {
- /* Match description too long : reduce it */
- matchLength = (15 - 1) + (oMaxMatch - op) * 255;
- }
- ip += MINMATCH + matchLength;
-
- if (matchLength >= ML_MASK) {
- *token += ML_MASK;
- matchLength -= ML_MASK;
- while (matchLength >= 255) {
- matchLength -= 255;
- *op++ = 255;
- }
- *op++ = (BYTE)matchLength;
- } else
- *token += (BYTE)(matchLength);
- }
-
- anchor = ip;
-
- /* Test end of block */
- if (ip > mflimit)
- break;
- if (op > oMaxSeq)
- break;
-
- /* Fill table */
- LZ4_putPosition(ip - 2, ctx->hashTable, tableType, base);
-
- /* Test next position */
- match = LZ4_getPosition(ip, ctx->hashTable, tableType, base);
- LZ4_putPosition(ip, ctx->hashTable, tableType, base);
- if ((match + MAX_DISTANCE >= ip)
- && (LZ4_read32(match) == LZ4_read32(ip))) {
- token = op++; *token = 0;
- goto _next_match;
- }
-
- /* Prepare next loop */
- forwardH = LZ4_hashPosition(++ip, tableType);
- }
-
-_last_literals:
- /* Encode Last Literals */
- {
- size_t lastRunSize = (size_t)(iend - anchor);
-
- if (op + 1 /* token */
- + ((lastRunSize + 240) / 255) /* litLength */
- + lastRunSize /* literals */ > oend) {
- /* adapt lastRunSize to fill 'dst' */
- lastRunSize = (oend - op) - 1;
- lastRunSize -= (lastRunSize + 240) / 255;
- }
- ip = anchor + lastRunSize;
-
- if (lastRunSize >= RUN_MASK) {
- size_t accumulator = lastRunSize - RUN_MASK;
-
- *op++ = RUN_MASK << ML_BITS;
- for (; accumulator >= 255; accumulator -= 255)
- *op++ = 255;
- *op++ = (BYTE) accumulator;
- } else {
- *op++ = (BYTE)(lastRunSize<<ML_BITS);
- }
- LZ4_memcpy(op, anchor, lastRunSize);
- op += lastRunSize;
- }
-
- /* End */
- *srcSizePtr = (int) (((const char *)ip) - src);
- return (int) (((char *)op) - dst);
-}
-
-static int LZ4_compress_destSize_extState(
- LZ4_stream_t *state,
- const char *src,
- char *dst,
- int *srcSizePtr,
- int targetDstSize)
-{
-#if LZ4_ARCH64
- const tableType_t tableType = byU32;
-#else
- const tableType_t tableType = byPtr;
-#endif
-
- LZ4_resetStream(state);
-
- if (targetDstSize >= LZ4_COMPRESSBOUND(*srcSizePtr)) {
- /* compression success is guaranteed */
- return LZ4_compress_fast_extState(
- state, src, dst, *srcSizePtr,
- targetDstSize, 1);
- } else {
- if (*srcSizePtr < LZ4_64Klimit)
- return LZ4_compress_destSize_generic(
- &state->internal_donotuse,
- src, dst, srcSizePtr,
- targetDstSize, byU16);
- else
- return LZ4_compress_destSize_generic(
- &state->internal_donotuse,
- src, dst, srcSizePtr,
- targetDstSize, tableType);
- }
-}
-
-
-int LZ4_compress_destSize(
- const char *src,
- char *dst,
- int *srcSizePtr,
- int targetDstSize,
- void *wrkmem)
-{
- return LZ4_compress_destSize_extState(wrkmem, src, dst, srcSizePtr,
- targetDstSize);
+ return LZ4_compress_destSize_extState(wrkmem, source, dest,
+ sourceSizePtr, targetDestSize,
+ LZ4_ACCELERATION_DEFAULT);
}
EXPORT_SYMBOL(LZ4_compress_destSize);
-/*-******************************
- * Streaming functions
- ********************************/
void LZ4_resetStream(LZ4_stream_t *LZ4_stream)
{
- memset(LZ4_stream, 0, sizeof(LZ4_stream_t));
+ __lz4_resetStream(LZ4_stream);
}
-int LZ4_loadDict(LZ4_stream_t *LZ4_dict,
- const char *dictionary, int dictSize)
+int LZ4_loadDict(LZ4_stream_t *streamPtr, const char *dictionary, int dictSize)
{
- LZ4_stream_t_internal *dict = &LZ4_dict->internal_donotuse;
- const BYTE *p = (const BYTE *)dictionary;
- const BYTE * const dictEnd = p + dictSize;
- const BYTE *base;
-
- if ((dict->initCheck)
- || (dict->currentOffset > 1 * GB)) {
- /* Uninitialized structure, or reuse overflow */
- LZ4_resetStream(LZ4_dict);
- }
-
- if (dictSize < (int)HASH_UNIT) {
- dict->dictionary = NULL;
- dict->dictSize = 0;
- return 0;
- }
-
- if ((dictEnd - p) > 64 * KB)
- p = dictEnd - 64 * KB;
- dict->currentOffset += 64 * KB;
- base = p - dict->currentOffset;
- dict->dictionary = p;
- dict->dictSize = (U32)(dictEnd - p);
- dict->currentOffset += dict->dictSize;
-
- while (p <= dictEnd - HASH_UNIT) {
- LZ4_putPosition(p, dict->hashTable, byU32, base);
- p += 3;
- }
-
- return dict->dictSize;
+ return __lz4_loadDict(streamPtr, dictionary, dictSize);
}
EXPORT_SYMBOL(LZ4_loadDict);
-static void LZ4_renormDictT(LZ4_stream_t_internal *LZ4_dict,
- const BYTE *src)
+int LZ4_saveDict(LZ4_stream_t *streamPtr, char *safeBuffer, int dictSize)
{
- if ((LZ4_dict->currentOffset > 0x80000000) ||
- ((uptrval)LZ4_dict->currentOffset > (uptrval)src)) {
- /* address space overflow */
- /* rescale hash table */
- U32 const delta = LZ4_dict->currentOffset - 64 * KB;
- const BYTE *dictEnd = LZ4_dict->dictionary + LZ4_dict->dictSize;
- int i;
-
- for (i = 0; i < LZ4_HASH_SIZE_U32; i++) {
- if (LZ4_dict->hashTable[i] < delta)
- LZ4_dict->hashTable[i] = 0;
- else
- LZ4_dict->hashTable[i] -= delta;
- }
- LZ4_dict->currentOffset = 64 * KB;
- if (LZ4_dict->dictSize > 64 * KB)
- LZ4_dict->dictSize = 64 * KB;
- LZ4_dict->dictionary = dictEnd - LZ4_dict->dictSize;
- }
-}
-
-int LZ4_saveDict(LZ4_stream_t *LZ4_dict, char *safeBuffer, int dictSize)
-{
- LZ4_stream_t_internal * const dict = &LZ4_dict->internal_donotuse;
- const BYTE * const previousDictEnd = dict->dictionary + dict->dictSize;
-
- if ((U32)dictSize > 64 * KB) {
- /* useless to define a dictionary > 64 * KB */
- dictSize = 64 * KB;
- }
- if ((U32)dictSize > dict->dictSize)
- dictSize = dict->dictSize;
-
- memmove(safeBuffer, previousDictEnd - dictSize, dictSize);
-
- dict->dictionary = (const BYTE *)safeBuffer;
- dict->dictSize = (U32)dictSize;
-
- return dictSize;
+ return __lz4_saveDict(streamPtr, safeBuffer, dictSize);
}
EXPORT_SYMBOL(LZ4_saveDict);
-int LZ4_compress_fast_continue(LZ4_stream_t *LZ4_stream, const char *source,
- char *dest, int inputSize, int maxOutputSize, int acceleration)
+int LZ4_compress_fast_continue(LZ4_stream_t *streamPtr, const char *src,
+ char *dst, int srcSize, int maxDstSize,
+ int acceleration)
{
- LZ4_stream_t_internal *streamPtr = &LZ4_stream->internal_donotuse;
- const BYTE * const dictEnd = streamPtr->dictionary
- + streamPtr->dictSize;
-
- const BYTE *smallest = (const BYTE *) source;
-
- if (streamPtr->initCheck) {
- /* Uninitialized structure detected */
- return 0;
- }
-
- if ((streamPtr->dictSize > 0) && (smallest > dictEnd))
- smallest = dictEnd;
-
- LZ4_renormDictT(streamPtr, smallest);
-
- if (acceleration < 1)
- acceleration = LZ4_ACCELERATION_DEFAULT;
-
- /* Check overlapping input/dictionary space */
- {
- const BYTE *sourceEnd = (const BYTE *) source + inputSize;
-
- if ((sourceEnd > streamPtr->dictionary)
- && (sourceEnd < dictEnd)) {
- streamPtr->dictSize = (U32)(dictEnd - sourceEnd);
- if (streamPtr->dictSize > 64 * KB)
- streamPtr->dictSize = 64 * KB;
- if (streamPtr->dictSize < 4)
- streamPtr->dictSize = 0;
- streamPtr->dictionary = dictEnd - streamPtr->dictSize;
- }
- }
-
- /* prefix mode : source data follows dictionary */
- if (dictEnd == (const BYTE *)source) {
- int result;
-
- if ((streamPtr->dictSize < 64 * KB) &&
- (streamPtr->dictSize < streamPtr->currentOffset)) {
- result = LZ4_compress_generic(
- streamPtr, source, dest, inputSize,
- maxOutputSize, limitedOutput, byU32,
- withPrefix64k, dictSmall, acceleration);
- } else {
- result = LZ4_compress_generic(
- streamPtr, source, dest, inputSize,
- maxOutputSize, limitedOutput, byU32,
- withPrefix64k, noDictIssue, acceleration);
- }
- streamPtr->dictSize += (U32)inputSize;
- streamPtr->currentOffset += (U32)inputSize;
- return result;
- }
-
- /* external dictionary mode */
- {
- int result;
-
- if ((streamPtr->dictSize < 64 * KB) &&
- (streamPtr->dictSize < streamPtr->currentOffset)) {
- result = LZ4_compress_generic(
- streamPtr, source, dest, inputSize,
- maxOutputSize, limitedOutput, byU32,
- usingExtDict, dictSmall, acceleration);
- } else {
- result = LZ4_compress_generic(
- streamPtr, source, dest, inputSize,
- maxOutputSize, limitedOutput, byU32,
- usingExtDict, noDictIssue, acceleration);
- }
- streamPtr->dictionary = (const BYTE *)source;
- streamPtr->dictSize = (U32)inputSize;
- streamPtr->currentOffset += (U32)inputSize;
- return result;
- }
+ return __lz4_compress_fast_continue(streamPtr, src, dst, srcSize,
+ maxDstSize, acceleration);
}
EXPORT_SYMBOL(LZ4_compress_fast_continue);
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH v2 10/15] lib/lz4: switch the compressor to the vendored sources
2026-11-06 14:42 ` [PATCH v2 10/15] lib/lz4: switch the compressor to the vendored sources Michal Wilczynski
@ 2026-11-07 1:40 ` kernel test robot
2026-11-10 5:36 ` Sergey Senozhatsky
1 sibling, 0 replies; 19+ messages in thread
From: kernel test robot @ 2026-11-07 1:40 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: oe-kbuild-all, Linux Memory Management List, linux-kernel, llvm,
linux-arm-kernel, linux-mips, linux-parisc
Hi Michal,
kernel test robot noticed the following build errors:
[auto build test ERROR on 93f51579e7df248780214094418f205253383cc5]
url: https://github.com/intel-lab-lkp/linux/commits/Michal-Wilczynski/lib-decompress_unlz4-use-LZ4_decompress_safe_partial-before-boot/20261106-154241
base: 93f51579e7df248780214094418f205253383cc5
patch link: https://lore.kernel.org/r/20261106-lz4-vendor-upstream-v2-10-ee04c3f0d7dc%40samsung.com
patch subject: [PATCH v2 10/15] lib/lz4: switch the compressor to the vendored sources
config: arc-allmodconfig (https://download.01.org/0day-ci/archive/20261107/202611070943.EDg3udaH-lkp@intel.com/config)
compiler: arc-linux-gcc (GCC) 16.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20261107/202611070943.EDg3udaH-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202611070943.EDg3udaH-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from include/linux/rcupdate.h:27,
from include/linux/rbtree.h:24,
from include/linux/mm_types.h:11,
from include/linux/mmzone.h:22,
from include/linux/gfp.h:7,
from include/linux/umh.h:4,
from include/linux/kmod.h:9,
from include/linux/module.h:18,
from lib/lz4/lz4_compress.c:29:
include/linux/sched.h: In function 'is_percpu_thread':
>> include/linux/sched.h:1860:17: error: 'current' undeclared (first use in this function)
1860 | return (current->flags & PF_NO_SETAFFINITY) &&
| ^~~~~~~
include/linux/sched.h:1860:17: note: each undeclared identifier is reported only once for each function it appears in
include/linux/sched.h: In function 'current_restore_flags':
include/linux/sched.h:1918:9: error: 'current' undeclared (first use in this function)
1918 | current->flags &= ~flags;
| ^~~~~~~
include/linux/sched.h: In function 'set_need_resched_current':
include/linux/sched.h:2127:30: error: 'current' undeclared (first use in this function)
2127 | set_tsk_need_resched(current);
| ^~~~~~~
In file included from include/asm-generic/bug.h:7,
from arch/arc/include/asm/bug.h:30,
from include/linux/bug.h:5,
from lib/lz4/lz4_compress.c:27:
include/linux/sched.h: In function '__set_task_blocked_on':
include/linux/sched.h:2231:27: error: 'current' undeclared (first use in this function)
2231 | WARN_ON_ONCE(p != current);
| ^~~~~~~
include/linux/once_lite.h:28:41: note: in definition of macro 'DO_ONCE_LITE_IF'
28 | bool __ret_do_once = !!(condition); \
| ^~~~~~~~~
include/linux/sched.h:2231:9: note: in expansion of macro 'WARN_ON_ONCE'
2231 | WARN_ON_ONCE(p != current);
| ^~~~~~~~~~~~
In file included from include/linux/nodemask.h:92,
from include/linux/numa.h:6,
from include/linux/cpumask.h:15,
from include/linux/smp.h:13,
from include/linux/interrupt_rc.h:17,
from include/linux/spinlock.h:60,
from include/linux/mmzone.h:8:
include/linux/sched.h: In function 'alloc_tag_save':
include/linux/sched.h:2361:14: error: 'current' undeclared (first use in this function)
2361 | swap(current->alloc_tag, tag);
| ^~~~~~~
include/linux/minmax.h:308:21: note: in definition of macro 'swap'
308 | do { typeof(a) __tmp = (a); (a) = (b); (b) = __tmp; } while (0)
| ^
>> include/linux/minmax.h:308:52: error: assignment to 'struct alloc_tag *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
308 | do { typeof(a) __tmp = (a); (a) = (b); (b) = __tmp; } while (0)
| ^
include/linux/sched.h:2361:9: note: in expansion of macro 'swap'
2361 | swap(current->alloc_tag, tag);
| ^~~~
include/linux/sched.h: In function 'alloc_tag_restore':
include/linux/sched.h:2368:14: error: 'current' undeclared (first use in this function)
2368 | WARN(current->alloc_tag != tag, "current->alloc_tag was changed:\n");
| ^~~~~~~
include/asm-generic/bug.h:164:32: note: in definition of macro 'WARN'
164 | int __ret_warn_on = !!(condition); \
| ^~~~~~~~~
In file included from <command-line>:
include/linux/rcupdate.h: In function 'cond_resched_tasks_rcu_qs':
>> include/linux/rcupdate.h:226:22: error: 'current' undeclared (first use in this function)
226 | rcu_tasks_qs(current, false);
| ^~~~~~~
include/linux/compiler_types.h:682:23: note: in definition of macro '__compiletime_assert'
682 | if (!(condition)) \
| ^~~~~~~~~
include/linux/compiler_types.h:702:9: note: in expansion of macro '_compiletime_assert'
702 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^~~~~~~~~~~~~~~~~~~
include/asm-generic/rwonce.h:36:9: note: in expansion of macro 'compiletime_assert'
36 | compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long), \
| ^~~~~~~~~~~~~~~~~~
include/asm-generic/rwonce.h:36:28: note: in expansion of macro '__native_word'
36 | compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long), \
| ^~~~~~~~~~~~~
include/asm-generic/rwonce.h:49:9: note: in expansion of macro 'compiletime_assert_rwonce_type'
49 | compiletime_assert_rwonce_type(x); \
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/rcupdate.h:185:35: note: in expansion of macro 'READ_ONCE'
185 | if (!(preempt) && READ_ONCE((t)->rcu_tasks_holdout)) \
| ^~~~~~~~~
include/linux/rcupdate.h:197:34: note: in expansion of macro 'rcu_tasks_classic_qs'
197 | #define rcu_tasks_qs(t, preempt) rcu_tasks_classic_qs((t), (preempt))
| ^~~~~~~~~~~~~~~~~~~~
include/linux/rcupdate.h:226:9: note: in expansion of macro 'rcu_tasks_qs'
226 | rcu_tasks_qs(current, false);
| ^~~~~~~~~~~~
In file included from include/linux/local_lock.h:5,
from include/linux/mmzone.h:24:
include/linux/local_lock_internal.h: In function 'local_lock_acquire':
>> include/linux/local_lock_internal.h:48:20: error: 'current' undeclared (first use in this function)
48 | l->owner = current;
| ^~~~~~~
include/linux/local_lock_internal.h: In function 'local_trylock_acquire':
include/linux/local_lock_internal.h:55:20: error: 'current' undeclared (first use in this function)
55 | l->owner = current;
| ^~~~~~~
In file included from include/linux/build_bug.h:5,
from lib/lz4/lz4_deps.h:18,
from lib/lz4/lz4_compress.c:16:
include/linux/local_lock_internal.h: In function 'local_lock_release':
include/linux/local_lock_internal.h:60:41: error: 'current' undeclared (first use in this function)
60 | DEBUG_LOCKS_WARN_ON(l->owner != current);
| ^~~~~~~
include/linux/compiler.h:77:45: note: in definition of macro 'unlikely'
77 | # define unlikely(x) __builtin_expect(!!(x), 0)
| ^
include/linux/local_lock_internal.h:60:9: note: in expansion of macro 'DEBUG_LOCKS_WARN_ON'
60 | DEBUG_LOCKS_WARN_ON(l->owner != current);
| ^~~~~~~~~~~~~~~~~~~
include/linux/sched/mm.h: In function 'current_gfp_context':
>> include/linux/sched/mm.h:250:41: error: 'current' undeclared (first use in this function)
250 | unsigned int pflags = READ_ONCE(current->flags);
| ^~~~~~~
include/linux/compiler_types.h:682:23: note: in definition of macro '__compiletime_assert'
682 | if (!(condition)) \
| ^~~~~~~~~
include/linux/compiler_types.h:702:9: note: in expansion of macro '_compiletime_assert'
702 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^~~~~~~~~~~~~~~~~~~
include/asm-generic/rwonce.h:36:9: note: in expansion of macro 'compiletime_assert'
36 | compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long), \
| ^~~~~~~~~~~~~~~~~~
include/asm-generic/rwonce.h:36:28: note: in expansion of macro '__native_word'
36 | compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long), \
| ^~~~~~~~~~~~~
include/asm-generic/rwonce.h:49:9: note: in expansion of macro 'compiletime_assert_rwonce_type'
49 | compiletime_assert_rwonce_type(x); \
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/sched/mm.h:250:31: note: in expansion of macro 'READ_ONCE'
250 | unsigned int pflags = READ_ONCE(current->flags);
| ^~~~~~~~~
include/linux/sched/mm.h: In function 'memalloc_retry_wait':
include/linux/sched.h:170:17: error: 'current' undeclared (first use in this function)
170 | current->task_state_change = _THIS_IP_; \
| ^~~~~~~
include/linux/sched.h:242:17: note: in expansion of macro 'debug_normal_state_change'
242 | debug_normal_state_change((state_value)); \
| ^~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/sched/mm.h:293:9: note: in expansion of macro '__set_current_state'
293 | __set_current_state(TASK_UNINTERRUPTIBLE);
| ^~~~~~~~~~~~~~~~~~~
In file included from include/linux/xarray.h:21,
from include/linux/radix-tree.h:21,
from include/linux/idr.h:15,
from include/linux/kernfs.h:12,
from include/linux/sysfs.h:16,
from include/linux/kobject.h:20,
from include/linux/module.h:22:
include/linux/sched/mm.h: In function 'might_alloc':
include/linux/sched/mm.h:319:13: error: 'current' undeclared (first use in this function)
319 | if (current->flags & PF_MEMALLOC)
| ^~~~~~~
include/linux/sched/mm.h: In function 'memalloc_flags_save':
include/linux/sched/mm.h:334:30: error: 'current' undeclared (first use in this function)
334 | unsigned oldflags = ~current->flags & flags;
| ^~~~~~~
include/linux/sched/mm.h: In function 'memalloc_flags_restore':
include/linux/sched/mm.h:341:9: error: 'current' undeclared (first use in this function)
341 | current->flags &= ~flags;
| ^~~~~~~
include/linux/sched/mm.h: In function 'set_active_memcg':
include/linux/sched/mm.h:499:23: error: 'current' undeclared (first use in this function)
499 | old = current->active_memcg;
| ^~~~~~~
include/linux/sched/mm.h: In function 'membarrier_mm_sync_core_before_usermode':
include/linux/sched/mm.h:543:13: error: 'current' undeclared (first use in this function)
543 | if (current->mm != mm)
| ^~~~~~~
vim +308 include/linux/minmax.h
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 285)
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 286) /**
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 287) * in_range - Determine if a value lies within a range.
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 288) * @val: Value to test.
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 289) * @start: First value in range.
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 290) * @len: Number of values in range.
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 291) *
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 292) * This is more efficient than "if (start <= val && val < (start + len))".
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 293) * It also gives a different answer if @start + @len overflows the size of
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 294) * the type by a sufficient amount to encompass @val. Decide for yourself
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 295) * which behaviour you want, or prove that start + len never overflow.
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 296) * Do not blindly replace one form with the other.
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 297) */
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 298) #define in_range(val, start, len) \
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 299) ((sizeof(start) | sizeof(len) | sizeof(val)) <= sizeof(u32) ? \
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 300) in_range32(val, start, len) : in_range64(val, start, len))
f9bff0e31881d03 Matthew Wilcox (Oracle 2023-08-02 301)
b296a6d53339a79 Andy Shevchenko 2020-10-15 302 /**
b296a6d53339a79 Andy Shevchenko 2020-10-15 303 * swap - swap values of @a and @b
b296a6d53339a79 Andy Shevchenko 2020-10-15 304 * @a: first value
b296a6d53339a79 Andy Shevchenko 2020-10-15 305 * @b: second value
b296a6d53339a79 Andy Shevchenko 2020-10-15 306 */
b296a6d53339a79 Andy Shevchenko 2020-10-15 307 #define swap(a, b) \
b296a6d53339a79 Andy Shevchenko 2020-10-15 @308 do { typeof(a) __tmp = (a); (a) = (b); (b) = __tmp; } while (0)
b296a6d53339a79 Andy Shevchenko 2020-10-15 309
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PATCH v2 10/15] lib/lz4: switch the compressor to the vendored sources
2026-11-06 14:42 ` [PATCH v2 10/15] lib/lz4: switch the compressor to the vendored sources Michal Wilczynski
2026-11-07 1:40 ` kernel test robot
@ 2026-11-10 5:36 ` Sergey Senozhatsky
1 sibling, 0 replies; 19+ messages in thread
From: Sergey Senozhatsky @ 2026-11-10 5:36 UTC (permalink / raw)
To: Michal Wilczynski
Cc: Nathan Chancellor, Nick Desaulniers, Bill Wendling, Justin Stitt,
Russell King, Thomas Bogendoerfer, James E.J. Bottomley,
Helge Deller, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger, Sven Schnelle, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Dave Hansen, x86, H. Peter Anvin,
Minchan Kim, Sergey Senozhatsky, Jens Axboe, Andrew Morton,
linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
On (26/11/06 15:42), Michal Wilczynski wrote:
[..]
> diff --git a/drivers/block/zram/backend_lz4.c b/drivers/block/zram/backend_lz4.c
> index 1e4ad31d39a63f45b3bddd3d8a467d8d7d095beb..50265e3ce256490994cb6dae32a45da376e408e0 100644
> --- a/drivers/block/zram/backend_lz4.c
> +++ b/drivers/block/zram/backend_lz4.c
> @@ -42,7 +42,7 @@ static int lz4_setup_params(struct zcomp_params *params)
> if (!params->dict || !params->dict_sz)
> return 0;
>
> - dict_stream = kzalloc_obj(*dict_stream);
> + dict_stream = kzalloc(LZ4_MEM_COMPRESS, GFP_KERNEL);
> if (!dict_stream)
> return -ENOMEM;
>
> @@ -88,7 +88,7 @@ static int lz4_create(struct zcomp_params *params, struct zcomp_ctx *ctx)
> if (!zctx->dstrm)
> goto error;
>
> - zctx->cstrm = kzalloc_obj(*zctx->cstrm);
> + zctx->cstrm = kzalloc(LZ4_MEM_COMPRESS, GFP_KERNEL);
> if (!zctx->cstrm)
> goto error;
> }
> @@ -112,7 +112,7 @@ static int lz4_compress(struct zcomp_params *params, struct zcomp_ctx *ctx,
> zctx->mem);
> } else {
> /* Cstrm needs to be reset */
> - memcpy(zctx->cstrm, params->drv_data, sizeof(*zctx->cstrm));
> + memcpy(zctx->cstrm, params->drv_data, LZ4_MEM_COMPRESS);
> ret = LZ4_compress_fast_continue(zctx->cstrm, req->src,
> req->dst, req->src_len,
> req->dst_len, params->level);
The zram code here is outdated, it has been reworked quite a bit
(patches are in linux-next and Andrew's trees).
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v2 11/15] lib/lz4: switch the HC compressor to the vendored sources
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
` (8 preceding siblings ...)
2026-11-06 14:42 ` [PATCH v2 10/15] lib/lz4: switch the compressor to the vendored sources Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 12/15] lib/lz4: widen LZ4_DECOMPRESS_INPLACE_MARGIN for the fast decode loop Michal Wilczynski
` (3 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
Replace the forked LZ4HC with thin entry points over the vendored
lz4hc.c.
Upstream routes levels 10 and up to its optimal parser, whose ~64K
workspace the kernel cannot provide. Clamp those levels to 9 rather than
reject them, so existing f2fs and zram settings keep working, and add
LZ4HC_CLAMP_CLEVEL to <linux/lz4.h>. Document the clamp in f2fs.rst.
Levels 1 and 2 now use upstream's lz4mid: on kernel source they are
50-84% faster for 4-9% larger output. Levels 3 to 9 compress the same,
but at 4K are up to 15% slower on poorly compressible data, including at
level 9, zram's lz4hc default.
LZ4_compress_HC() and LZ4_resetStreamHC() check the state's alignment
like the compressor's entry points.
LZ4_streamHC_t becomes incomplete and the macros that laid out the forked
structure go. LZ4HC_MEM_COMPRESS grows from 262192 to upstream's 262200,
static_asserted against LZ4_STREAMHC_MINSIZE. lz4hc.c pulls in lz4.c's
decoder tables, unused here when the fast decode loop is off, so silence
-Wunused-const-variable for this object.
This retires commit b08918fb3f27 ("lz4: do not export static symbol"):
the glue exports only the five LZ4HC entry points.
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
Documentation/filesystems/f2fs.rst | 2 +
drivers/block/zram/backend_lz4hc.c | 2 +-
include/linux/lz4.h | 54 +--
lib/lz4/Makefile | 4 +
lib/lz4/lz4hc_compress.c | 792 ++++---------------------------------
5 files changed, 91 insertions(+), 763 deletions(-)
diff --git a/Documentation/filesystems/f2fs.rst b/Documentation/filesystems/f2fs.rst
index 771216f45207a797356d5d2a10328b74d1ab302a..4a2d4545c2752cbcea4939839ba3d2457bf92299 100644
--- a/Documentation/filesystems/f2fs.rst
+++ b/Documentation/filesystems/f2fs.rst
@@ -318,6 +318,8 @@ compress_algorithm=%s:%d Control compress algorithm and its compress level, now,
zstd 1 - 22
========= ===========
+ lz4 levels above 9 compress as level 9.
+
compress_log_size=%u Support configuring compress cluster size. The size will
be 4KB * (1 << %u). The default and minimum sizes are 16KB.
compress_extension=%s Support adding specified extension, so that f2fs can enable
diff --git a/drivers/block/zram/backend_lz4hc.c b/drivers/block/zram/backend_lz4hc.c
index 06286c805cf7c56bd760f460cc854dc4014f293b..e7ee3740219852f079d1e0866da6b213a3f3a583 100644
--- a/drivers/block/zram/backend_lz4hc.c
+++ b/drivers/block/zram/backend_lz4hc.c
@@ -69,7 +69,7 @@ static int lz4hc_create(struct zcomp_params *params, struct zcomp_ctx *ctx)
if (!zctx->dstrm)
goto error;
- zctx->cstrm = kzalloc_obj(*zctx->cstrm);
+ zctx->cstrm = kzalloc(LZ4HC_MEM_COMPRESS, GFP_KERNEL);
if (!zctx->cstrm)
goto error;
LZ4_resetStreamHC(zctx->cstrm, params->level);
diff --git a/include/linux/lz4.h b/include/linux/lz4.h
index 031a410b8e3878e9a24cc49af36cfcbba399d991..05060b9790be4621ddbe744172fa0ae37f6191f4 100644
--- a/include/linux/lz4.h
+++ b/include/linux/lz4.h
@@ -59,19 +59,16 @@
#define LZ4HC_DEFAULT_CLEVEL 9
#define LZ4HC_MAX_CLEVEL 16
-#define LZ4HC_DICTIONARY_LOGSIZE 16
-#define LZ4HC_MAXD (1<<LZ4HC_DICTIONARY_LOGSIZE)
-#define LZ4HC_MAXD_MASK (LZ4HC_MAXD - 1)
-#define LZ4HC_HASH_LOG (LZ4HC_DICTIONARY_LOGSIZE - 1)
-#define LZ4HC_HASHTABLESIZE (1 << LZ4HC_HASH_LOG)
-#define LZ4HC_HASH_MASK (LZ4HC_HASHTABLESIZE - 1)
+/* Levels from here up select the optimal parser, which needs a ~64K
+ * workspace lib/lz4 cannot give it, so lib/lz4 compresses them as
+ * LZ4HC_CLAMP_CLEVEL - 1. f2fs and zram still accept levels up to
+ * LZ4HC_MAX_CLEVEL.
+ */
+#define LZ4HC_CLAMP_CLEVEL 10
/*-************************************************************************
* STREAMING CONSTANTS AND STRUCTURES
**************************************************************************/
-#define LZ4_STREAMHCSIZE 262192
-#define LZ4_STREAMHCSIZE_SIZET (262192 / sizeof(size_t))
-
#define LZ4_STREAMDECODESIZE_U64 4
#define LZ4_STREAMDECODESIZE (LZ4_STREAMDECODESIZE_U64 * \
sizeof(unsigned long long))
@@ -83,29 +80,10 @@
typedef union LZ4_stream_u LZ4_stream_t;
/*
- * LZ4_streamHC_t - information structure to track an LZ4HC stream.
+ * LZ4_streamHC_t - an LZ4HC stream. Incomplete: lib/lz4 owns the layout.
+ * Allocate LZ4HC_MEM_COMPRESS bytes and cast, do not sizeof().
*/
-typedef struct {
- unsigned int hashTable[LZ4HC_HASHTABLESIZE];
- unsigned short chainTable[LZ4HC_MAXD];
- /* next block to continue on current prefix */
- const unsigned char *end;
- /* All index relative to this position */
- const unsigned char *base;
- /* alternate base for extDict */
- const unsigned char *dictBase;
- /* below that point, need extDict */
- unsigned int dictLimit;
- /* below that point, no more dict */
- unsigned int lowLimit;
- /* index from which to continue dict update */
- unsigned int nextToUpdate;
- unsigned int compressionLevel;
-} LZ4HC_CCtx_internal;
-typedef union {
- size_t table[LZ4_STREAMHCSIZE_SIZET];
- LZ4HC_CCtx_internal internal_donotuse;
-} LZ4_streamHC_t;
+typedef union LZ4_streamHC_u LZ4_streamHC_t;
/*
* LZ4_streamDecode_t - information structure to track an
@@ -133,7 +111,7 @@ typedef union {
* points warn about a misaligned buffer and fail the call.
*/
#define LZ4_MEM_COMPRESS 16416
-#define LZ4HC_MEM_COMPRESS LZ4_STREAMHCSIZE
+#define LZ4HC_MEM_COMPRESS 262200
/*-************************************************************************
* Compression Functions
@@ -289,9 +267,9 @@ int LZ4_decompress_safe_partial(const char *source, char *dest,
* @srcSize: size of the input data. Max supported value is LZ4_MAX_INPUT_SIZE
* @dstCapacity: full or partial size of buffer 'dst',
* which must be already allocated
- * @compressionLevel: Recommended values are between 4 and 9, although any
- * value between 1 and LZ4HC_MAX_CLEVEL will work.
- * Values >LZ4HC_MAX_CLEVEL behave the same as 16.
+ * @compressionLevel: Recommended values are between 4 and 9. Levels of
+ * LZ4HC_CLAMP_CLEVEL and above are clamped to LZ4HC_CLAMP_CLEVEL - 1;
+ * see that macro.
* @wrkmem: address of the working memory.
* This requires 'wrkmem' of size LZ4HC_MEM_COMPRESS, aligned to 8 bytes.
*
@@ -307,9 +285,9 @@ int LZ4_compress_HC(const char *src, char *dst, int srcSize, int dstCapacity,
/**
* LZ4_resetStreamHC() - Init an allocated 'LZ4_streamHC_t' structure
* @streamHCPtr: pointer to the 'LZ4_streamHC_t' structure
- * @compressionLevel: Recommended values are between 4 and 9, although any
- * value between 1 and LZ4HC_MAX_CLEVEL will work.
- * Values >LZ4HC_MAX_CLEVEL behave the same as 16.
+ * @compressionLevel: Recommended values are between 4 and 9. Levels of
+ * LZ4HC_CLAMP_CLEVEL and above are clamped to LZ4HC_CLAMP_CLEVEL - 1;
+ * see that macro.
*
* An LZ4_streamHC_t structure can be allocated once
* and re-used multiple times.
diff --git a/lib/lz4/Makefile b/lib/lz4/Makefile
index 8b4eb07694e67dd99ec7cc85212fb68138d3981e..5cbbe10beedce1199dee68992d542a1e52f89f8e 100644
--- a/lib/lz4/Makefile
+++ b/lib/lz4/Makefile
@@ -10,3 +10,7 @@ ccflags-y += -I $(src)/freestanding
obj-$(CONFIG_LZ4_COMPRESS) += lz4_compress.o
obj-$(CONFIG_LZ4HC_COMPRESS) += lz4hc_compress.o
obj-$(CONFIG_LZ4_DECOMPRESS) += lz4_decompress.o
+
+# lz4hc.c pulls in lz4.c's common definitions, which include two decoder
+# tables nothing here uses when LZ4_FAST_DEC_LOOP is off.
+CFLAGS_lz4hc_compress.o += $(call cc-disable-warning, unused-const-variable)
diff --git a/lib/lz4/lz4hc_compress.c b/lib/lz4/lz4hc_compress.c
index 91936dc3d14bca31752a361693380aad8eb3b98a..4d56d45028110805628be766b2758a8dffd56f0c 100644
--- a/lib/lz4/lz4hc_compress.c
+++ b/lib/lz4/lz4hc_compress.c
@@ -1,766 +1,110 @@
+// SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
/*
- * LZ4 HC - High Compression Mode of LZ4
- * Copyright (C) 2011-2015, Yann Collet.
+ * Copyright (C) 2011 - 2016, Yann Collet.
+ * Copyright (C) 2016, Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
+ * Copyright (c) 2026 Samsung Electronics Co., Ltd.
+ * Author: Michal Wilczynski <m.wilczynski@samsung.com>
*
- * BSD 2 - Clause License (http://www.opensource.org/licenses/bsd - license.php)
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are
- * met:
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above
- * copyright notice, this list of conditions and the following disclaimer
- * in the documentation and/or other materials provided with the
- * distribution.
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- * You can contact the author at :
- * - LZ4 homepage : http://www.lz4.org
- * - LZ4 source repository : https://github.com/lz4/lz4
+ * LZ4 HC compressor -- kernel entry points
*
- * Changed for kernel usage by:
- * Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
+ * The compressor is upstream's, in the verbatim lz4hc.c included below.
+ * LZ4_compress_HC() takes the kernel's trailing wrkmem argument and forwards
+ * to LZ4_compress_HC_extStateHC(); the level-taking entry points clamp the
+ * level below LZ4HC_CLEVEL_OPT_MIN (see lz4hc_clamp_level()).
*/
-/*-************************************
- * Dependencies
- **************************************/
-#include "lz4defs.h"
-#include <linux/module.h>
-#include <linux/kernel.h>
-#include <linux/string.h> /* memset */
-
-/* *************************************
- * Local Constants and types
- ***************************************/
-
-#define OPTIMAL_ML (int)((ML_MASK - 1) + MINMATCH)
-
-#define HASH_FUNCTION(i) (((i) * 2654435761U) \
- >> ((MINMATCH*8) - LZ4HC_HASH_LOG))
-#define DELTANEXTU16(p) chainTable[(U16)(p)] /* faster */
-
-static U32 LZ4HC_hashPtr(const void *ptr)
-{
- return HASH_FUNCTION(LZ4_read32(ptr));
-}
+#include "lz4_deps.h"
-/**************************************
- * HC Compression
- **************************************/
-static void LZ4HC_init(LZ4HC_CCtx_internal *hc4, const BYTE *start)
-{
- memset((void *)hc4->hashTable, 0, sizeof(hc4->hashTable));
- memset(hc4->chainTable, 0xFF, sizeof(hc4->chainTable));
- hc4->nextToUpdate = 64 * KB;
- hc4->base = start - 64 * KB;
- hc4->end = start;
- hc4->dictBase = start - 64 * KB;
- hc4->dictLimit = 64 * KB;
- hc4->lowLimit = 64 * KB;
-}
-
-/* Update chains up to ip (excluded) */
-static FORCE_INLINE void LZ4HC_Insert(LZ4HC_CCtx_internal *hc4,
- const BYTE *ip)
+/* lz4hc.c includes lz4.c with LZ4_COMMONDEFS_ONLY, which stops short of
+ * LZ4_compressBound(). Supply it for lz4hc.c's internal use.
+ */
+static int LZ4_compressBound(int isize)
{
- U16 * const chainTable = hc4->chainTable;
- U32 * const hashTable = hc4->hashTable;
- const BYTE * const base = hc4->base;
- U32 const target = (U32)(ip - base);
- U32 idx = hc4->nextToUpdate;
-
- while (idx < target) {
- U32 const h = LZ4HC_hashPtr(base + idx);
- size_t delta = idx - hashTable[h];
-
- if (delta > MAX_DISTANCE)
- delta = MAX_DISTANCE;
-
- DELTANEXTU16(idx) = (U16)delta;
-
- hashTable[h] = idx;
- idx++;
- }
-
- hc4->nextToUpdate = target;
+ return LZ4_COMPRESSBOUND(isize);
}
-static FORCE_INLINE int LZ4HC_InsertAndFindBestMatch(
- LZ4HC_CCtx_internal *hc4, /* Index table will be updated */
- const BYTE *ip,
- const BYTE * const iLimit,
- const BYTE **matchpos,
- const int maxNbAttempts)
-{
- U16 * const chainTable = hc4->chainTable;
- U32 * const HashTable = hc4->hashTable;
- const BYTE * const base = hc4->base;
- const BYTE * const dictBase = hc4->dictBase;
- const U32 dictLimit = hc4->dictLimit;
- const U32 lowLimit = (hc4->lowLimit + 64 * KB > (U32)(ip - base))
- ? hc4->lowLimit
- : (U32)(ip - base) - (64 * KB - 1);
- U32 matchIndex;
- int nbAttempts = maxNbAttempts;
- size_t ml = 0;
-
- /* HC4 match finder */
- LZ4HC_Insert(hc4, ip);
- matchIndex = HashTable[LZ4HC_hashPtr(ip)];
-
- while ((matchIndex >= lowLimit)
- && (nbAttempts)) {
- nbAttempts--;
- if (matchIndex >= dictLimit) {
- const BYTE * const match = base + matchIndex;
-
- if (*(match + ml) == *(ip + ml)
- && (LZ4_read32(match) == LZ4_read32(ip))) {
- size_t const mlt = LZ4_count(ip + MINMATCH,
- match + MINMATCH, iLimit) + MINMATCH;
-
- if (mlt > ml) {
- ml = mlt;
- *matchpos = match;
- }
- }
- } else {
- const BYTE * const match = dictBase + matchIndex;
+#include "upstream/lz4hc.c"
- if (LZ4_read32(match) == LZ4_read32(ip)) {
- size_t mlt;
- const BYTE *vLimit = ip
- + (dictLimit - matchIndex);
+/* Upstream's short helper macros; MIN and MAX clash with <linux/minmax.h>. */
+#undef MIN
+#undef MAX
+#undef KB
+#undef MB
+#undef GB
- if (vLimit > iLimit)
- vLimit = iLimit;
- mlt = LZ4_count(ip + MINMATCH,
- match + MINMATCH, vLimit) + MINMATCH;
- if ((ip + mlt == vLimit)
- && (vLimit < iLimit))
- mlt += LZ4_count(ip + mlt,
- base + dictLimit,
- iLimit);
- if (mlt > ml) {
- /* virtual matchpos */
- ml = mlt;
- *matchpos = base + matchIndex;
- }
- }
- }
- matchIndex -= DELTANEXTU16(matchIndex);
- }
+#include "lz4_kernel_api.h"
+#include <linux/bug.h>
+#include <linux/export.h>
+#include <linux/module.h>
- return (int)ml;
-}
+/* Catch any divergence from upstream's layout at build time. */
+static_assert(sizeof(LZ4_streamHC_t) == LZ4_STREAMHC_MINSIZE);
+static_assert(LZ4_STREAMHC_MINSIZE == 262200); /* LZ4HC_MEM_COMPRESS */
-static FORCE_INLINE int LZ4HC_InsertAndGetWiderMatch(
- LZ4HC_CCtx_internal *hc4,
- const BYTE * const ip,
- const BYTE * const iLowLimit,
- const BYTE * const iHighLimit,
- int longest,
- const BYTE **matchpos,
- const BYTE **startpos,
- const int maxNbAttempts)
+/* Levels >= LZ4HC_CLEVEL_OPT_MIN (10) reach the optimal parser, which needs
+ * a ~64K opt[]: too big for a kernel stack, and in the heap mode lz4_deps.h
+ * selects its allocation always fails, and so would the call. Clamp them
+ * to 9 rather than reject them, so existing f2fs and zram settings keep
+ * working. Levels 3..9 keep their meaning; 1 and 2 are upstream's lz4mid,
+ * faster and weaker than the shallow hash chain the fork used for them.
+ */
+static int lz4hc_clamp_level(int compressionLevel)
{
- U16 * const chainTable = hc4->chainTable;
- U32 * const HashTable = hc4->hashTable;
- const BYTE * const base = hc4->base;
- const U32 dictLimit = hc4->dictLimit;
- const BYTE * const lowPrefixPtr = base + dictLimit;
- const U32 lowLimit = (hc4->lowLimit + 64 * KB > (U32)(ip - base))
- ? hc4->lowLimit
- : (U32)(ip - base) - (64 * KB - 1);
- const BYTE * const dictBase = hc4->dictBase;
- U32 matchIndex;
- int nbAttempts = maxNbAttempts;
- int delta = (int)(ip - iLowLimit);
-
- /* First Match */
- LZ4HC_Insert(hc4, ip);
- matchIndex = HashTable[LZ4HC_hashPtr(ip)];
-
- while ((matchIndex >= lowLimit)
- && (nbAttempts)) {
- nbAttempts--;
- if (matchIndex >= dictLimit) {
- const BYTE *matchPtr = base + matchIndex;
-
- if (*(iLowLimit + longest)
- == *(matchPtr - delta + longest)) {
- if (LZ4_read32(matchPtr) == LZ4_read32(ip)) {
- int mlt = MINMATCH + LZ4_count(
- ip + MINMATCH,
- matchPtr + MINMATCH,
- iHighLimit);
- int back = 0;
-
- while ((ip + back > iLowLimit)
- && (matchPtr + back > lowPrefixPtr)
- && (ip[back - 1] == matchPtr[back - 1]))
- back--;
+ if (compressionLevel >= LZ4HC_CLEVEL_OPT_MIN)
+ return LZ4HC_CLEVEL_OPT_MIN - 1;
- mlt -= back;
-
- if (mlt > longest) {
- longest = (int)mlt;
- *matchpos = matchPtr + back;
- *startpos = ip + back;
- }
- }
- }
- } else {
- const BYTE * const matchPtr = dictBase + matchIndex;
-
- if (LZ4_read32(matchPtr) == LZ4_read32(ip)) {
- size_t mlt;
- int back = 0;
- const BYTE *vLimit = ip + (dictLimit - matchIndex);
-
- if (vLimit > iHighLimit)
- vLimit = iHighLimit;
-
- mlt = LZ4_count(ip + MINMATCH,
- matchPtr + MINMATCH, vLimit) + MINMATCH;
-
- if ((ip + mlt == vLimit) && (vLimit < iHighLimit))
- mlt += LZ4_count(ip + mlt, base + dictLimit,
- iHighLimit);
- while ((ip + back > iLowLimit)
- && (matchIndex + back > lowLimit)
- && (ip[back - 1] == matchPtr[back - 1]))
- back--;
-
- mlt -= back;
-
- if ((int)mlt > longest) {
- longest = (int)mlt;
- *matchpos = base + matchIndex + back;
- *startpos = ip + back;
- }
- }
- }
-
- matchIndex -= DELTANEXTU16(matchIndex);
- }
-
- return longest;
+ return compressionLevel;
}
-static FORCE_INLINE int LZ4HC_encodeSequence(
- const BYTE **ip,
- BYTE **op,
- const BYTE **anchor,
- int matchLength,
- const BYTE * const match,
- limitedOutput_directive limitedOutputBuffer,
- BYTE *oend)
+/* LZ4_initStreamHC() returns NULL for a misaligned state.
+ * LZ4_compress_HC_extStateHC() then fails quietly, and LZ4_resetStreamHC()
+ * ignores it and leaves the state uninitialised. Warn, and skip the reset.
+ */
+static bool lz4hc_state_misaligned(const void *state)
{
- int length;
- BYTE *token;
-
- /* Encode Literal length */
- length = (int)(*ip - *anchor);
- token = (*op)++;
-
- if ((limitedOutputBuffer)
- && ((*op + (length>>8)
- + length + (2 + 1 + LASTLITERALS)) > oend)) {
- /* Check output limit */
- return 1;
- }
- if (length >= (int)RUN_MASK) {
- int len;
-
- *token = (RUN_MASK<<ML_BITS);
- len = length - RUN_MASK;
- for (; len > 254 ; len -= 255)
- *(*op)++ = 255;
- *(*op)++ = (BYTE)len;
- } else
- *token = (BYTE)(length<<ML_BITS);
-
- /* Copy Literals */
- LZ4_wildCopy(*op, *anchor, (*op) + length);
- *op += length;
-
- /* Encode Offset */
- LZ4_writeLE16(*op, (U16)(*ip - match));
- *op += 2;
-
- /* Encode MatchLength */
- length = (int)(matchLength - MINMATCH);
-
- if ((limitedOutputBuffer)
- && (*op + (length>>8)
- + (1 + LASTLITERALS) > oend)) {
- /* Check output limit */
- return 1;
- }
-
- if (length >= (int)ML_MASK) {
- *token += ML_MASK;
- length -= ML_MASK;
-
- for (; length > 509 ; length -= 510) {
- *(*op)++ = 255;
- *(*op)++ = 255;
- }
-
- if (length > 254) {
- length -= 255;
- *(*op)++ = 255;
- }
-
- *(*op)++ = (BYTE)length;
- } else
- *token += (BYTE)(length);
-
- /* Prepare next loop */
- *ip += matchLength;
- *anchor = *ip;
-
- return 0;
+ return WARN_ON_ONCE(!LZ4_isAligned(state, LZ4_streamHC_t_alignment()));
}
-static int LZ4HC_compress_generic(
- LZ4HC_CCtx_internal *const ctx,
- const char * const source,
- char * const dest,
- int const inputSize,
- int const maxOutputSize,
- int compressionLevel,
- limitedOutput_directive limit
- )
+int LZ4_compress_HC(const char *src, char *dst, int srcSize, int dstCapacity,
+ int compressionLevel, void *wrkmem)
{
- const BYTE *ip = (const BYTE *) source;
- const BYTE *anchor = ip;
- const BYTE * const iend = ip + inputSize;
- const BYTE * const mflimit = iend - MFLIMIT;
- const BYTE * const matchlimit = (iend - LASTLITERALS);
-
- BYTE *op = (BYTE *) dest;
- BYTE * const oend = op + maxOutputSize;
-
- unsigned int maxNbAttempts;
- int ml, ml2, ml3, ml0;
- const BYTE *ref = NULL;
- const BYTE *start2 = NULL;
- const BYTE *ref2 = NULL;
- const BYTE *start3 = NULL;
- const BYTE *ref3 = NULL;
- const BYTE *start0;
- const BYTE *ref0;
-
- /* init */
- if (compressionLevel > LZ4HC_MAX_CLEVEL)
- compressionLevel = LZ4HC_MAX_CLEVEL;
- if (compressionLevel < 1)
- compressionLevel = LZ4HC_DEFAULT_CLEVEL;
- maxNbAttempts = 1 << (compressionLevel - 1);
- ctx->end += inputSize;
-
- ip++;
-
- /* Main Loop */
- while (ip < mflimit) {
- ml = LZ4HC_InsertAndFindBestMatch(ctx, ip,
- matchlimit, (&ref), maxNbAttempts);
- if (!ml) {
- ip++;
- continue;
- }
-
- /* saved, in case we would skip too much */
- start0 = ip;
- ref0 = ref;
- ml0 = ml;
-
-_Search2:
- if (ip + ml < mflimit)
- ml2 = LZ4HC_InsertAndGetWiderMatch(ctx,
- ip + ml - 2, ip + 0,
- matchlimit, ml, &ref2,
- &start2, maxNbAttempts);
- else
- ml2 = ml;
-
- if (ml2 == ml) {
- /* No better match */
- if (LZ4HC_encodeSequence(&ip, &op,
- &anchor, ml, ref, limit, oend))
- return 0;
- continue;
- }
-
- if (start0 < ip) {
- if (start2 < ip + ml0) {
- /* empirical */
- ip = start0;
- ref = ref0;
- ml = ml0;
- }
- }
-
- /* Here, start0 == ip */
- if ((start2 - ip) < 3) {
- /* First Match too small : removed */
- ml = ml2;
- ip = start2;
- ref = ref2;
- goto _Search2;
- }
-
-_Search3:
- /*
- * Currently we have :
- * ml2 > ml1, and
- * ip1 + 3 <= ip2 (usually < ip1 + ml1)
- */
- if ((start2 - ip) < OPTIMAL_ML) {
- int correction;
- int new_ml = ml;
-
- if (new_ml > OPTIMAL_ML)
- new_ml = OPTIMAL_ML;
- if (ip + new_ml > start2 + ml2 - MINMATCH)
- new_ml = (int)(start2 - ip) + ml2 - MINMATCH;
-
- correction = new_ml - (int)(start2 - ip);
-
- if (correction > 0) {
- start2 += correction;
- ref2 += correction;
- ml2 -= correction;
- }
- }
- /*
- * Now, we have start2 = ip + new_ml,
- * with new_ml = min(ml, OPTIMAL_ML = 18)
- */
-
- if (start2 + ml2 < mflimit)
- ml3 = LZ4HC_InsertAndGetWiderMatch(ctx,
- start2 + ml2 - 3, start2,
- matchlimit, ml2, &ref3, &start3,
- maxNbAttempts);
- else
- ml3 = ml2;
-
- if (ml3 == ml2) {
- /* No better match : 2 sequences to encode */
- /* ip & ref are known; Now for ml */
- if (start2 < ip + ml)
- ml = (int)(start2 - ip);
- /* Now, encode 2 sequences */
- if (LZ4HC_encodeSequence(&ip, &op, &anchor,
- ml, ref, limit, oend))
- return 0;
- ip = start2;
- if (LZ4HC_encodeSequence(&ip, &op, &anchor,
- ml2, ref2, limit, oend))
- return 0;
- continue;
- }
-
- if (start3 < ip + ml + 3) {
- /* Not enough space for match 2 : remove it */
- if (start3 >= (ip + ml)) {
- /* can write Seq1 immediately
- * ==> Seq2 is removed,
- * so Seq3 becomes Seq1
- */
- if (start2 < ip + ml) {
- int correction = (int)(ip + ml - start2);
-
- start2 += correction;
- ref2 += correction;
- ml2 -= correction;
- if (ml2 < MINMATCH) {
- start2 = start3;
- ref2 = ref3;
- ml2 = ml3;
- }
- }
-
- if (LZ4HC_encodeSequence(&ip, &op, &anchor,
- ml, ref, limit, oend))
- return 0;
- ip = start3;
- ref = ref3;
- ml = ml3;
-
- start0 = start2;
- ref0 = ref2;
- ml0 = ml2;
- goto _Search2;
- }
-
- start2 = start3;
- ref2 = ref3;
- ml2 = ml3;
- goto _Search3;
- }
-
- /*
- * OK, now we have 3 ascending matches;
- * let's write at least the first one
- * ip & ref are known; Now for ml
- */
- if (start2 < ip + ml) {
- if ((start2 - ip) < (int)ML_MASK) {
- int correction;
-
- if (ml > OPTIMAL_ML)
- ml = OPTIMAL_ML;
- if (ip + ml > start2 + ml2 - MINMATCH)
- ml = (int)(start2 - ip) + ml2 - MINMATCH;
- correction = ml - (int)(start2 - ip);
- if (correction > 0) {
- start2 += correction;
- ref2 += correction;
- ml2 -= correction;
- }
- } else
- ml = (int)(start2 - ip);
- }
- if (LZ4HC_encodeSequence(&ip, &op, &anchor, ml,
- ref, limit, oend))
- return 0;
-
- ip = start2;
- ref = ref2;
- ml = ml2;
-
- start2 = start3;
- ref2 = ref3;
- ml2 = ml3;
-
- goto _Search3;
- }
-
- /* Encode Last Literals */
- {
- int lastRun = (int)(iend - anchor);
-
- if ((limit)
- && (((char *)op - dest) + lastRun + 1
- + ((lastRun + 255 - RUN_MASK)/255)
- > (U32)maxOutputSize)) {
- /* Check output limit */
- return 0;
- }
- if (lastRun >= (int)RUN_MASK) {
- *op++ = (RUN_MASK<<ML_BITS);
- lastRun -= RUN_MASK;
- for (; lastRun > 254 ; lastRun -= 255)
- *op++ = 255;
- *op++ = (BYTE) lastRun;
- } else
- *op++ = (BYTE)(lastRun<<ML_BITS);
- LZ4_memcpy(op, anchor, iend - anchor);
- op += iend - anchor;
- }
-
- /* End */
- return (int) (((char *)op) - dest);
-}
-
-static int LZ4_compress_HC_extStateHC(
- void *state,
- const char *src,
- char *dst,
- int srcSize,
- int maxDstSize,
- int compressionLevel)
-{
- LZ4HC_CCtx_internal *ctx = &((LZ4_streamHC_t *)state)->internal_donotuse;
-
- if (((size_t)(state)&(sizeof(void *) - 1)) != 0) {
- /* Error : state is not aligned
- * for pointers (32 or 64 bits)
- */
+ if (lz4hc_state_misaligned(wrkmem))
return 0;
- }
- LZ4HC_init(ctx, (const BYTE *)src);
-
- if (maxDstSize < LZ4_compressBound(srcSize))
- return LZ4HC_compress_generic(ctx, src, dst,
- srcSize, maxDstSize, compressionLevel, limitedOutput);
- else
- return LZ4HC_compress_generic(ctx, src, dst,
- srcSize, maxDstSize, compressionLevel, noLimit);
-}
-
-int LZ4_compress_HC(const char *src, char *dst, int srcSize,
- int maxDstSize, int compressionLevel, void *wrkmem)
-{
- return LZ4_compress_HC_extStateHC(wrkmem, src, dst,
- srcSize, maxDstSize, compressionLevel);
+ return LZ4_compress_HC_extStateHC(wrkmem, src, dst, srcSize,
+ dstCapacity,
+ lz4hc_clamp_level(compressionLevel));
}
EXPORT_SYMBOL(LZ4_compress_HC);
-/**************************************
- * Streaming Functions
- **************************************/
-void LZ4_resetStreamHC(LZ4_streamHC_t *LZ4_streamHCPtr, int compressionLevel)
+void LZ4_resetStreamHC(LZ4_streamHC_t *streamHCPtr, int compressionLevel)
{
- LZ4_streamHCPtr->internal_donotuse.base = NULL;
- LZ4_streamHCPtr->internal_donotuse.compressionLevel = (unsigned int)compressionLevel;
+ if (lz4hc_state_misaligned(streamHCPtr))
+ return;
+
+ __lz4_resetStreamHC(streamHCPtr, lz4hc_clamp_level(compressionLevel));
}
EXPORT_SYMBOL(LZ4_resetStreamHC);
-int LZ4_loadDictHC(LZ4_streamHC_t *LZ4_streamHCPtr,
- const char *dictionary,
- int dictSize)
+int LZ4_loadDictHC(LZ4_streamHC_t *streamHCPtr, const char *dictionary,
+ int dictSize)
{
- LZ4HC_CCtx_internal *ctxPtr = &LZ4_streamHCPtr->internal_donotuse;
-
- if (dictSize > 64 * KB) {
- dictionary += dictSize - 64 * KB;
- dictSize = 64 * KB;
- }
- LZ4HC_init(ctxPtr, (const BYTE *)dictionary);
- if (dictSize >= 4)
- LZ4HC_Insert(ctxPtr, (const BYTE *)dictionary + (dictSize - 3));
- ctxPtr->end = (const BYTE *)dictionary + dictSize;
- return dictSize;
+ return __lz4_loadDictHC(streamHCPtr, dictionary, dictSize);
}
EXPORT_SYMBOL(LZ4_loadDictHC);
-/* compression */
-
-static void LZ4HC_setExternalDict(
- LZ4HC_CCtx_internal *ctxPtr,
- const BYTE *newBlock)
+int LZ4_compress_HC_continue(LZ4_streamHC_t *streamHCPtr, const char *src,
+ char *dst, int srcSize, int maxDstSize)
{
- if (ctxPtr->end >= ctxPtr->base + 4) {
- /* Referencing remaining dictionary content */
- LZ4HC_Insert(ctxPtr, ctxPtr->end - 3);
- }
-
- /*
- * Only one memory segment for extDict,
- * so any previous extDict is lost at this stage
- */
- ctxPtr->lowLimit = ctxPtr->dictLimit;
- ctxPtr->dictLimit = (U32)(ctxPtr->end - ctxPtr->base);
- ctxPtr->dictBase = ctxPtr->base;
- ctxPtr->base = newBlock - ctxPtr->dictLimit;
- ctxPtr->end = newBlock;
- /* match referencing will resume from there */
- ctxPtr->nextToUpdate = ctxPtr->dictLimit;
-}
-
-static int LZ4_compressHC_continue_generic(
- LZ4_streamHC_t *LZ4_streamHCPtr,
- const char *source,
- char *dest,
- int inputSize,
- int maxOutputSize,
- limitedOutput_directive limit)
-{
- LZ4HC_CCtx_internal *ctxPtr = &LZ4_streamHCPtr->internal_donotuse;
-
- /* auto - init if forgotten */
- if (ctxPtr->base == NULL)
- LZ4HC_init(ctxPtr, (const BYTE *) source);
-
- /* Check overflow */
- if ((size_t)(ctxPtr->end - ctxPtr->base) > 2 * GB) {
- size_t dictSize = (size_t)(ctxPtr->end - ctxPtr->base)
- - ctxPtr->dictLimit;
- if (dictSize > 64 * KB)
- dictSize = 64 * KB;
- LZ4_loadDictHC(LZ4_streamHCPtr,
- (const char *)(ctxPtr->end) - dictSize, (int)dictSize);
- }
-
- /* Check if blocks follow each other */
- if ((const BYTE *)source != ctxPtr->end)
- LZ4HC_setExternalDict(ctxPtr, (const BYTE *)source);
-
- /* Check overlapping input/dictionary space */
- {
- const BYTE *sourceEnd = (const BYTE *) source + inputSize;
- const BYTE * const dictBegin = ctxPtr->dictBase + ctxPtr->lowLimit;
- const BYTE * const dictEnd = ctxPtr->dictBase + ctxPtr->dictLimit;
-
- if ((sourceEnd > dictBegin)
- && ((const BYTE *)source < dictEnd)) {
- if (sourceEnd > dictEnd)
- sourceEnd = dictEnd;
- ctxPtr->lowLimit = (U32)(sourceEnd - ctxPtr->dictBase);
-
- if (ctxPtr->dictLimit - ctxPtr->lowLimit < 4)
- ctxPtr->lowLimit = ctxPtr->dictLimit;
- }
- }
-
- return LZ4HC_compress_generic(ctxPtr, source, dest,
- inputSize, maxOutputSize, ctxPtr->compressionLevel, limit);
-}
-
-int LZ4_compress_HC_continue(
- LZ4_streamHC_t *LZ4_streamHCPtr,
- const char *source,
- char *dest,
- int inputSize,
- int maxOutputSize)
-{
- if (maxOutputSize < LZ4_compressBound(inputSize))
- return LZ4_compressHC_continue_generic(LZ4_streamHCPtr,
- source, dest, inputSize, maxOutputSize, limitedOutput);
- else
- return LZ4_compressHC_continue_generic(LZ4_streamHCPtr,
- source, dest, inputSize, maxOutputSize, noLimit);
+ return __lz4_compress_HC_continue(streamHCPtr, src, dst, srcSize,
+ maxDstSize);
}
EXPORT_SYMBOL(LZ4_compress_HC_continue);
-/* dictionary saving */
-
-int LZ4_saveDictHC(
- LZ4_streamHC_t *LZ4_streamHCPtr,
- char *safeBuffer,
- int dictSize)
+int LZ4_saveDictHC(LZ4_streamHC_t *streamHCPtr, char *safeBuffer,
+ int maxDictSize)
{
- LZ4HC_CCtx_internal *const streamPtr = &LZ4_streamHCPtr->internal_donotuse;
- int const prefixSize = (int)(streamPtr->end
- - (streamPtr->base + streamPtr->dictLimit));
-
- if (dictSize > 64 * KB)
- dictSize = 64 * KB;
- if (dictSize < 4)
- dictSize = 0;
- if (dictSize > prefixSize)
- dictSize = prefixSize;
-
- memmove(safeBuffer, streamPtr->end - dictSize, dictSize);
-
- {
- U32 const endIndex = (U32)(streamPtr->end - streamPtr->base);
-
- streamPtr->end = (const BYTE *)safeBuffer + dictSize;
- streamPtr->base = streamPtr->end - endIndex;
- streamPtr->dictLimit = endIndex - dictSize;
- streamPtr->lowLimit = endIndex - dictSize;
-
- if (streamPtr->nextToUpdate < streamPtr->dictLimit)
- streamPtr->nextToUpdate = streamPtr->dictLimit;
- }
- return dictSize;
+ return __lz4_saveDictHC(streamHCPtr, safeBuffer, maxDictSize);
}
EXPORT_SYMBOL(LZ4_saveDictHC);
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 12/15] lib/lz4: widen LZ4_DECOMPRESS_INPLACE_MARGIN for the fast decode loop
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
` (9 preceding siblings ...)
2026-11-06 14:42 ` [PATCH v2 11/15] lib/lz4: switch the HC " Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
2026-11-11 22:02 ` Gao Xiang
2026-11-06 14:42 ` [PATCH v2 13/15] lib/lz4: switch the decompressor to the vendored sources Michal Wilczynski
` (2 subsequent siblings)
13 siblings, 1 reply; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
erofs decompresses a pcluster in place when the slack after the output
is at least LZ4_DECOMPRESS_INPLACE_MARGIN(compressed size), upstream's
(compressedSize >> 8) + 32.
That holds for the fork, which overshoots its output by at most 8
bytes. Upstream's LZ4_FAST_DEC_LOOP, which the next patch brings in for
x86 and arm64, copies in 32-byte steps and can overwrite input it has
not read yet: of 172410 valid streams laid out as erofs does at exactly
that margin, 3660 fail to decode, none with the fork. The worst needs
(compressedSize >> 8) + 33. Images mkfs.erofs built did not hit this,
but nothing in the format rules it out.
Use (compressedSize >> 8) + (compressedSize >> 15) + 36, which takes a
4K pcluster's margin from 48 to 52 bytes. Upstream's lz4.h defines the
macro too, so undefine it first.
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
include/linux/lz4.h | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/include/linux/lz4.h b/include/linux/lz4.h
index 05060b9790be4621ddbe744172fa0ae37f6191f4..e809149747c3dc97d129e42e0ce4385b5a219930 100644
--- a/include/linux/lz4.h
+++ b/include/linux/lz4.h
@@ -521,7 +521,18 @@ int LZ4_decompress_safe_usingDict(const char *source, char *dest,
int compressedSize, int maxDecompressedSize, const char *dictStart,
int dictSize);
-#define LZ4_DECOMPRESS_INPLACE_MARGIN(compressedSize) (((compressedSize) >> 8) + 32)
+/*
+ * Slack needed after the decompressed data for LZ4_decompress_safe() to
+ * decode in place, with the compressed data at the end of the buffer.
+ * Upstream's (compressedSize >> 8) + 32 can be a few bytes short: its fast
+ * decode loop copies in 32-byte steps, which may overwrite input it has
+ * not read yet. The >> 15 term is extra headroom for large pclusters.
+ * Upstream's lz4.h, which lib/lz4's objects also see, defines the macro
+ * too.
+ */
+#undef LZ4_DECOMPRESS_INPLACE_MARGIN
+#define LZ4_DECOMPRESS_INPLACE_MARGIN(compressedSize) \
+ (((compressedSize) >> 8) + ((compressedSize) >> 15) + 36)
#ifndef LZ4_DISTANCE_MAX /* history window size; can be user-defined at compile time */
#define LZ4_DISTANCE_MAX 65535 /* set to maximum value by default */
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH v2 12/15] lib/lz4: widen LZ4_DECOMPRESS_INPLACE_MARGIN for the fast decode loop
2026-11-06 14:42 ` [PATCH v2 12/15] lib/lz4: widen LZ4_DECOMPRESS_INPLACE_MARGIN for the fast decode loop Michal Wilczynski
@ 2026-11-11 22:02 ` Gao Xiang
0 siblings, 0 replies; 19+ messages in thread
From: Gao Xiang @ 2026-11-11 22:02 UTC (permalink / raw)
To: Michal Wilczynski
Cc: Nathan Chancellor, Nick Desaulniers, Bill Wendling, Justin Stitt,
Russell King, Thomas Bogendoerfer, James E.J. Bottomley,
Helge Deller, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger, Sven Schnelle, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Dave Hansen, x86, H. Peter Anvin,
Minchan Kim, Sergey Senozhatsky, Jens Axboe, Andrew Morton,
linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
Hi Michal,
On Fri, Nov 06, 2026 at 03:42:52PM +0100, Michal Wilczynski wrote:
> erofs decompresses a pcluster in place when the slack after the output
> is at least LZ4_DECOMPRESS_INPLACE_MARGIN(compressed size), upstream's
> (compressedSize >> 8) + 32.
>
> That holds for the fork, which overshoots its output by at most 8
> bytes. Upstream's LZ4_FAST_DEC_LOOP, which the next patch brings in for
> x86 and arm64, copies in 32-byte steps and can overwrite input it has
> not read yet: of 172410 valid streams laid out as erofs does at exactly
> that margin, 3660 fail to decode, none with the fork. The worst needs
> (compressedSize >> 8) + 33. Images mkfs.erofs built did not hit this,
> but nothing in the format rules it out.
>
> Use (compressedSize >> 8) + (compressedSize >> 15) + 36, which takes a
> 4K pcluster's margin from 48 to 52 bytes. Upstream's lz4.h defines the
> macro too, so undefine it first.
>
> Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
First, I really think kernel LZ4 should be updated for many years, so
thanks for this!
But my personal thought is that could you just fold this patch into
the next one ("lib/lz4: switch the decompressor to the vendored sources")?
Because this one should be part of the fast loop update.
Although the current patch order is also fine, but my potential concern
is that people don't know the details maybe pick the random ones for
backporting, which is what I'd like to avoid so I hope this patch could
be just part of it, otherwise the inplace decompression could be broken.
Thanks,
Gao Xiang
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v2 13/15] lib/lz4: switch the decompressor to the vendored sources
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
` (10 preceding siblings ...)
2026-11-06 14:42 ` [PATCH v2 12/15] lib/lz4: widen LZ4_DECOMPRESS_INPLACE_MARGIN for the fast decode loop Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
2026-11-10 5:39 ` Sergey Senozhatsky
2026-11-06 14:42 ` [PATCH v2 14/15] lib/lz4: replace lz4_kernel_api.h's prototypes with <linux/lz4.h> Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 15/15] MAINTAINERS: add an entry for the LZ4 compression library Michal Wilczynski
13 siblings, 1 reply; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
Replace the forked decompressor with thin entry points over the vendored
lz4.c; the API is signature-compatible, so every export is a plain
forwarder. lz4defs.h has no users left and goes.
lib/decompress_unlz4.c includes this file before boot, together with
<linux/lz4.h>, so LZ4_streamDecode_t becomes incomplete as upstream
declares it, and LZ4_MEM_DECOMPRESS (32) replaces the
LZ4_STREAMDECODESIZE macros. LZ4_COMPRESSBOUND is defined by both
headers and gets a guard; the LZ4_compressBound() wrapper goes, as
lib/decompress_unlz4.c now uses the macro.
LZ4_decompress_safe_partial() on a truncated block can now return the
bytes decoded so far rather than an error, depending on where the cut
falls; the kernel-doc says so.
This retires six local patches:
commit 8cb5d7482810 ("lib/lz4: make arrays static const, reduces object code size")
upstream 75fb878a9019 ("decompress_generic: Add fastpath for small offsets"), v1.9.0
commit b1a3e75e466d ("lz4: fix kernel decompression speed")
upstream fe2a1b3707d5 ("Call LZ4_memcpy() instead of memcpy()"), v1.9.3
commit 89b158635ad7 ("lib/lz4: explicitly support in-place decompression")
upstream 33cb8518ac38 ("decompress: changed final memcpy() into memmove()"), v1.9.2
commit 7fde9d6e839d ("lz4_decompress: declare LZ4_decompress_safe_withPrefix64k static")
LZ4LIB_VISIBILITY in lz4_deps.h, which makes every upstream entry point static
commit eafc0a02391b ("lz4: fix LZ4_decompress_safe_partial read out of bound")
upstream c5d6f8a8be39 ("fix #783"), v1.9.3
commit 2d8867f3e083 ("lib: make LZ4_decompress_safe_forceExtDict() static")
the static forward declaration in lz4_deps.h
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
drivers/block/zram/backend_lz4.c | 2 +-
drivers/block/zram/backend_lz4hc.c | 2 +-
include/linux/lz4.h | 60 ++--
lib/decompress_unlz4.c | 4 +-
lib/lz4/lz4_decompress.c | 640 +++----------------------------------
lib/lz4/lz4defs.h | 247 --------------
6 files changed, 71 insertions(+), 884 deletions(-)
diff --git a/drivers/block/zram/backend_lz4.c b/drivers/block/zram/backend_lz4.c
index 50265e3ce256490994cb6dae32a45da376e408e0..b9b8a5f678c155442fa3fd2c1984e45182e6fc38 100644
--- a/drivers/block/zram/backend_lz4.c
+++ b/drivers/block/zram/backend_lz4.c
@@ -84,7 +84,7 @@ static int lz4_create(struct zcomp_params *params, struct zcomp_ctx *ctx)
if (!zctx->mem)
goto error;
} else {
- zctx->dstrm = kzalloc_obj(*zctx->dstrm);
+ zctx->dstrm = kzalloc(LZ4_MEM_DECOMPRESS, GFP_KERNEL);
if (!zctx->dstrm)
goto error;
diff --git a/drivers/block/zram/backend_lz4hc.c b/drivers/block/zram/backend_lz4hc.c
index e7ee3740219852f079d1e0866da6b213a3f3a583..1e09c93016264259c149e2913b2819571b56fab6 100644
--- a/drivers/block/zram/backend_lz4hc.c
+++ b/drivers/block/zram/backend_lz4hc.c
@@ -65,7 +65,7 @@ static int lz4hc_create(struct zcomp_params *params, struct zcomp_ctx *ctx)
if (!zctx->mem)
goto error;
} else {
- zctx->dstrm = kzalloc_obj(*zctx->dstrm);
+ zctx->dstrm = kzalloc(LZ4_MEM_DECOMPRESS, GFP_KERNEL);
if (!zctx->dstrm)
goto error;
diff --git a/include/linux/lz4.h b/include/linux/lz4.h
index e809149747c3dc97d129e42e0ce4385b5a219930..fd9d7a162e04655a9d19639515466a6ba8e1e892 100644
--- a/include/linux/lz4.h
+++ b/include/linux/lz4.h
@@ -48,10 +48,16 @@
* CONSTANTS
**************************************************************************/
#define LZ4_MAX_INPUT_SIZE 0x7E000000 /* 2 113 929 216 bytes */
+
+/* Upstream's lz4.h defines the same macro, and lib/lz4's entry points and
+ * the pre-boot lib/decompress_unlz4.c see both headers.
+ */
+#ifndef LZ4_COMPRESSBOUND
#define LZ4_COMPRESSBOUND(isize) (\
(unsigned int)(isize) > (unsigned int)LZ4_MAX_INPUT_SIZE \
? 0 \
: (isize) + ((isize)/255) + 16)
+#endif
#define LZ4_ACCELERATION_DEFAULT 1
@@ -67,12 +73,8 @@
#define LZ4HC_CLAMP_CLEVEL 10
/*-************************************************************************
- * STREAMING CONSTANTS AND STRUCTURES
+ * STREAMING STRUCTURES
**************************************************************************/
-#define LZ4_STREAMDECODESIZE_U64 4
-#define LZ4_STREAMDECODESIZE (LZ4_STREAMDECODESIZE_U64 * \
- sizeof(unsigned long long))
-
/*
* LZ4_stream_t - an LZ4 stream. Incomplete: lib/lz4 owns the layout.
* Allocate LZ4_MEM_COMPRESS bytes and cast, do not sizeof().
@@ -86,21 +88,11 @@ typedef union LZ4_stream_u LZ4_stream_t;
typedef union LZ4_streamHC_u LZ4_streamHC_t;
/*
- * LZ4_streamDecode_t - information structure to track an
- * LZ4 stream during decompression.
- *
- * init this structure using LZ4_setStreamDecode (or memset()) before first use
+ * LZ4_streamDecode_t - an LZ4 stream during decompression. Incomplete:
+ * lib/lz4 owns the layout. Allocate LZ4_MEM_DECOMPRESS bytes and cast, do
+ * not sizeof(). Init with LZ4_setStreamDecode() (or zero it) before use.
*/
-typedef struct {
- const uint8_t *externalDict;
- size_t extDictSize;
- const uint8_t *prefixEnd;
- size_t prefixSize;
-} LZ4_streamDecode_t_internal;
-typedef union {
- unsigned long long table[LZ4_STREAMDECODESIZE_U64];
- LZ4_streamDecode_t_internal internal_donotuse;
-} LZ4_streamDecode_t;
+typedef union LZ4_streamDecode_u LZ4_streamDecode_t;
/*-************************************************************************
* SIZE OF STATE
@@ -112,23 +104,12 @@ typedef union {
*/
#define LZ4_MEM_COMPRESS 16416
#define LZ4HC_MEM_COMPRESS 262200
+#define LZ4_MEM_DECOMPRESS 32
/*-************************************************************************
* Compression Functions
**************************************************************************/
-/**
- * LZ4_compressBound() - Max. output size in worst case szenarios
- * @isize: Size of the input data
- *
- * Return: Max. size LZ4 may output in a "worst case" szenario
- * (data not compressible)
- */
-static inline int LZ4_compressBound(size_t isize)
-{
- return LZ4_COMPRESSBOUND(isize);
-}
-
/**
* LZ4_compress_default() - Compress data from source to dest
* @source: source address of the original data
@@ -142,7 +123,7 @@ static inline int LZ4_compressBound(size_t isize)
* Compresses 'sourceSize' bytes from buffer 'source'
* into already allocated 'dest' buffer of size 'maxOutputSize'.
* Compression is guaranteed to succeed if
- * 'maxOutputSize' >= LZ4_compressBound(inputSize).
+ * 'maxOutputSize' >= LZ4_COMPRESSBOUND(inputSize).
* It also runs faster, so it's a recommended setting.
* If the function cannot compress 'source' into a more limited 'dest' budget,
* compression stops *immediately*, and the function result is zero.
@@ -234,7 +215,7 @@ int LZ4_decompress_safe(const char *source, char *dest, int compressedSize,
* @source: source address of the compressed data
* @dest: output buffer address of the decompressed data which must be
* already allocated
- * @compressedSize: is the precise full size of the compressed block.
+ * @compressedSize: size of the compressed data at 'source'
* @targetOutputSize: the decompression operation will try
* to stop as soon as 'targetOutputSize' has been reached
* @maxDecompressedSize: is the size of destination buffer
@@ -249,8 +230,11 @@ int LZ4_decompress_safe(const char *source, char *dest, int compressedSize,
* It is therefore protected against malicious data packets.
*
* Return: the number of bytes decoded in the destination buffer
- * (necessarily <= maxDecompressedSize)
- * or a negative result in case of error
+ * (necessarily <= targetOutputSize and maxDecompressedSize)
+ * or a negative result in case of error. For a block cut short the
+ * result is either the bytes decoded so far, below targetOutputSize,
+ * or an error, depending on where the cut falls; check it against the
+ * size you expect.
*
*/
int LZ4_decompress_safe_partial(const char *source, char *dest,
@@ -275,7 +259,7 @@ int LZ4_decompress_safe_partial(const char *source, char *dest,
*
* Compress data from 'src' into 'dst', using the more powerful
* but slower "HC" algorithm. Compression is guaranteed to succeed if
- * `dstCapacity >= LZ4_compressBound(srcSize)
+ * `dstCapacity >= LZ4_COMPRESSBOUND(srcSize)
*
* Return : the number of bytes written into 'dst' or 0 if compression fails.
*/
@@ -339,7 +323,7 @@ int LZ4_loadDictHC(LZ4_streamHC_t *streamHCPtr, const char *dictionary,
* (including initial dictionary when present) must remain accessible
* and unmodified during compression.
* 'dst' buffer should be sized to handle worst case scenarios, using
- * LZ4_compressBound(), to ensure operation success.
+ * LZ4_COMPRESSBOUND(), to ensure operation success.
* If, for any reason, previous data blocks can't be preserved unmodified
* in memory during next compression block,
* you must save it to a safer memory space, using LZ4_saveDictHC().
@@ -435,7 +419,7 @@ int LZ4_saveDict(LZ4_stream_t *streamPtr, char *safeBuffer, int dictSize);
* as dictionary to improve compression ratio.
* Important : Previous data blocks are assumed to still
* be present and unmodified !
- * If maxDstSize >= LZ4_compressBound(srcSize),
+ * If maxDstSize >= LZ4_COMPRESSBOUND(srcSize),
* compression is guaranteed to succeed, and runs faster.
*
* Return: Number of bytes written into buffer 'dst' or 0 if compression fails
diff --git a/lib/decompress_unlz4.c b/lib/decompress_unlz4.c
index dc20c22e087900e21da49f9fcb267e6b511e1072..87eea1fbdea063e241dd98f994d7fee078b7a910 100644
--- a/lib/decompress_unlz4.c
+++ b/lib/decompress_unlz4.c
@@ -69,7 +69,7 @@ STATIC inline int INIT unlz4(u8 *input, long in_len,
error("NULL input pointer and missing fill function");
goto exit_1;
} else {
- inp = large_malloc(LZ4_compressBound(uncomp_chunksize));
+ inp = large_malloc(LZ4_COMPRESSBOUND(uncomp_chunksize));
if (!inp) {
error("Could not allocate input buffer");
goto exit_1;
@@ -140,7 +140,7 @@ STATIC inline int INIT unlz4(u8 *input, long in_len,
inp += 4;
size -= 4;
} else {
- if (chunksize > LZ4_compressBound(uncomp_chunksize)) {
+ if (chunksize > LZ4_COMPRESSBOUND(uncomp_chunksize)) {
error("chunk length is longer than allocated");
goto exit_2;
}
diff --git a/lib/lz4/lz4_decompress.c b/lib/lz4/lz4_decompress.c
index f89b0e547f2156990c16770f69f89e0d8b46156d..60b57862a2e387aa89fa26f6f8e2b5485b5d9995 100644
--- a/lib/lz4/lz4_decompress.c
+++ b/lib/lz4/lz4_decompress.c
@@ -1,626 +1,76 @@
+// SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
/*
- * LZ4 - Fast LZ compression algorithm
* Copyright (C) 2011 - 2016, Yann Collet.
- * BSD 2 - Clause License (http://www.opensource.org/licenses/bsd - license.php)
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are
- * met:
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above
- * copyright notice, this list of conditions and the following disclaimer
- * in the documentation and/or other materials provided with the
- * distribution.
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- * You can contact the author at :
- * - LZ4 homepage : http://www.lz4.org
- * - LZ4 source repository : https://github.com/lz4/lz4
+ * Copyright (C) 2016, Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
+ * Copyright (c) 2026 Samsung Electronics Co., Ltd.
+ * Author: Michal Wilczynski <m.wilczynski@samsung.com>
*
- * Changed for kernel usage by:
- * Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
+ * LZ4 decompressor -- kernel entry points
+ *
+ * The decompressor is upstream's, in the verbatim lz4.c included below. The
+ * API is signature-compatible, so every export is a plain forwarder. This
+ * file is also included by lib/decompress_unlz4.c for the pre-boot
+ * decompressor, which defines STATIC.
*/
-/*-************************************
- * Dependencies
- **************************************/
-#include "lz4defs.h"
-#include <linux/init.h>
-#include <linux/module.h>
-#include <linux/kernel.h>
-#include <linux/unaligned.h>
+#include "lz4_deps.h"
+#include "upstream/lz4.c"
-/*-*****************************
- * Decompression functions
- *******************************/
+/* Upstream's short helper macros; MIN and MAX clash with <linux/minmax.h>. */
+#undef MIN
+#undef MAX
+#undef KB
+#undef MB
+#undef GB
-#define DEBUGLOG(l, ...) {} /* disabled */
+#include "lz4_kernel_api.h"
-#ifndef assert
-#define assert(condition) ((void)0)
+#ifndef STATIC
+#include <linux/export.h>
+#include <linux/module.h>
#endif
-/*
- * LZ4_decompress_generic() :
- * This generic decompression function covers all use cases.
- * It shall be instantiated several times, using different sets of directives.
- * Note that it is important for performance that this function really get inlined,
- * in order to remove useless branches during compilation optimization.
- */
-static FORCE_INLINE int LZ4_decompress_generic(
- const char * const src,
- char * const dst,
- int srcSize,
- /*
- * If endOnInput == endOnInputSize,
- * this value is `dstCapacity`
- */
- int outputSize,
- /* endOnOutputSize, endOnInputSize */
- endCondition_directive endOnInput,
- /* full, partial */
- earlyEnd_directive partialDecoding,
- /* noDict, withPrefix64k, usingExtDict */
- dict_directive dict,
- /* always <= dst, == dst when no prefix */
- const BYTE * const lowPrefix,
- /* only if dict == usingExtDict */
- const BYTE * const dictStart,
- /* note : = 0 if noDict */
- const size_t dictSize
- )
-{
- const BYTE *ip = (const BYTE *) src;
- const BYTE * const iend = ip + srcSize;
-
- BYTE *op = (BYTE *) dst;
- BYTE * const oend = op + outputSize;
- BYTE *cpy;
-
- const BYTE * const dictEnd = (const BYTE *)dictStart + dictSize;
- static const unsigned int inc32table[8] = {0, 1, 2, 1, 0, 4, 4, 4};
- static const int dec64table[8] = {0, 0, 0, -1, -4, 1, 2, 3};
-
- const int safeDecode = (endOnInput == endOnInputSize);
- const int checkOffset = ((safeDecode) && (dictSize < (int)(64 * KB)));
-
- /* Set up the "end" pointers for the shortcut. */
- const BYTE *const shortiend = iend -
- (endOnInput ? 14 : 8) /*maxLL*/ - 2 /*offset*/;
- const BYTE *const shortoend = oend -
- (endOnInput ? 14 : 8) /*maxLL*/ - 18 /*maxML*/;
-
- DEBUGLOG(5, "%s (srcSize:%i, dstSize:%i)", __func__,
- srcSize, outputSize);
-
- /* Special cases */
- assert(lowPrefix <= op);
- assert(src != NULL);
-
- /* Empty output buffer */
- if ((endOnInput) && (unlikely(outputSize == 0)))
- return ((srcSize == 1) && (*ip == 0)) ? 0 : -1;
-
- if ((!endOnInput) && (unlikely(outputSize == 0)))
- return (*ip == 0 ? 1 : -1);
-
- if ((endOnInput) && unlikely(srcSize == 0))
- return -1;
-
- /* Main Loop : decode sequences */
- while (1) {
- size_t length;
- const BYTE *match;
- size_t offset;
-
- /* get literal length */
- unsigned int const token = *ip++;
- length = token>>ML_BITS;
-
- /* ip < iend before the increment */
- assert(!endOnInput || ip <= iend);
-
- /*
- * A two-stage shortcut for the most common case:
- * 1) If the literal length is 0..14, and there is enough
- * space, enter the shortcut and copy 16 bytes on behalf
- * of the literals (in the fast mode, only 8 bytes can be
- * safely copied this way).
- * 2) Further if the match length is 4..18, copy 18 bytes
- * in a similar manner; but we ensure that there's enough
- * space in the output for those 18 bytes earlier, upon
- * entering the shortcut (in other words, there is a
- * combined check for both stages).
- *
- * The & in the likely() below is intentionally not && so that
- * some compilers can produce better parallelized runtime code
- */
- if ((endOnInput ? length != RUN_MASK : length <= 8)
- /*
- * strictly "less than" on input, to re-enter
- * the loop with at least one byte
- */
- && likely((endOnInput ? ip < shortiend : 1) &
- (op <= shortoend))) {
- /* Copy the literals */
- LZ4_memcpy(op, ip, endOnInput ? 16 : 8);
- op += length; ip += length;
-
- /*
- * The second stage:
- * prepare for match copying, decode full info.
- * If it doesn't work out, the info won't be wasted.
- */
- length = token & ML_MASK; /* match length */
- offset = LZ4_readLE16(ip);
- ip += 2;
- match = op - offset;
- assert(match <= op); /* check overflow */
-
- /* Do not deal with overlapping matches. */
- if ((length != ML_MASK) &&
- (offset >= 8) &&
- (dict == withPrefix64k || match >= lowPrefix)) {
- /* Copy the match. */
- LZ4_memcpy(op + 0, match + 0, 8);
- LZ4_memcpy(op + 8, match + 8, 8);
- LZ4_memcpy(op + 16, match + 16, 2);
- op += length + MINMATCH;
- /* Both stages worked, load the next token. */
- continue;
- }
-
- /*
- * The second stage didn't work out, but the info
- * is ready. Propel it right to the point of match
- * copying.
- */
- goto _copy_match;
- }
-
- /* decode literal length */
- if (length == RUN_MASK) {
- unsigned int s;
-
- if (unlikely(endOnInput ? ip >= iend - RUN_MASK : 0)) {
- /* overflow detection */
- goto _output_error;
- }
- do {
- s = *ip++;
- length += s;
- } while (likely(endOnInput
- ? ip < iend - RUN_MASK
- : 1) & (s == 255));
-
- if ((safeDecode)
- && unlikely((uptrval)(op) +
- length < (uptrval)(op))) {
- /* overflow detection */
- goto _output_error;
- }
- if ((safeDecode)
- && unlikely((uptrval)(ip) +
- length < (uptrval)(ip))) {
- /* overflow detection */
- goto _output_error;
- }
- }
-
- /* copy literals */
- cpy = op + length;
- LZ4_STATIC_ASSERT(MFLIMIT >= WILDCOPYLENGTH);
-
- if (((endOnInput) && ((cpy > oend - MFLIMIT)
- || (ip + length > iend - (2 + 1 + LASTLITERALS))))
- || ((!endOnInput) && (cpy > oend - WILDCOPYLENGTH))) {
- if (partialDecoding) {
- if (cpy > oend) {
- /*
- * Partial decoding :
- * stop in the middle of literal segment
- */
- cpy = oend;
- length = oend - op;
- }
- if ((endOnInput)
- && (ip + length > iend)) {
- /*
- * Error :
- * read attempt beyond
- * end of input buffer
- */
- goto _output_error;
- }
- } else {
- if ((!endOnInput)
- && (cpy != oend)) {
- /*
- * Error :
- * block decoding must
- * stop exactly there
- */
- goto _output_error;
- }
- if ((endOnInput)
- && ((ip + length != iend)
- || (cpy > oend))) {
- /*
- * Error :
- * input must be consumed
- */
- goto _output_error;
- }
- }
-
- /*
- * supports overlapping memory regions; only matters
- * for in-place decompression scenarios
- */
- LZ4_memmove(op, ip, length);
- ip += length;
- op += length;
-
- /* Necessarily EOF when !partialDecoding.
- * When partialDecoding, it is EOF if we've either
- * filled the output buffer or
- * can't proceed with reading an offset for following match.
- */
- if (!partialDecoding || (cpy == oend) || (ip >= (iend - 2)))
- break;
- } else {
- /* may overwrite up to WILDCOPYLENGTH beyond cpy */
- LZ4_wildCopy(op, ip, cpy);
- ip += length;
- op = cpy;
- }
-
- /* get offset */
- offset = LZ4_readLE16(ip);
- ip += 2;
- match = op - offset;
-
- /* get matchlength */
- length = token & ML_MASK;
-
-_copy_match:
- if ((checkOffset) && (unlikely(match + dictSize < lowPrefix))) {
- /* Error : offset outside buffers */
- goto _output_error;
- }
-
- /* costs ~1%; silence an msan warning when offset == 0 */
- /*
- * note : when partialDecoding, there is no guarantee that
- * at least 4 bytes remain available in output buffer
- */
- if (!partialDecoding) {
- assert(oend > op);
- assert(oend - op >= 4);
-
- LZ4_write32(op, (U32)offset);
- }
-
- if (length == ML_MASK) {
- unsigned int s;
-
- do {
- s = *ip++;
-
- if ((endOnInput) && (ip > iend - LASTLITERALS))
- goto _output_error;
-
- length += s;
- } while (s == 255);
+static_assert(sizeof(LZ4_streamDecode_t) == LZ4_STREAMDECODE_MINSIZE);
+static_assert(LZ4_STREAMDECODE_MINSIZE == 32); /* LZ4_MEM_DECOMPRESS */
- if ((safeDecode)
- && unlikely(
- (uptrval)(op) + length < (uptrval)op)) {
- /* overflow detection */
- goto _output_error;
- }
- }
-
- length += MINMATCH;
-
- /* match starting within external dictionary */
- if ((dict == usingExtDict) && (match < lowPrefix)) {
- if (unlikely(op + length > oend - LASTLITERALS)) {
- /* doesn't respect parsing restriction */
- if (!partialDecoding)
- goto _output_error;
- length = min(length, (size_t)(oend - op));
- }
-
- if (length <= (size_t)(lowPrefix - match)) {
- /*
- * match fits entirely within external
- * dictionary : just copy
- */
- memmove(op, dictEnd - (lowPrefix - match),
- length);
- op += length;
- } else {
- /*
- * match stretches into both external
- * dictionary and current block
- */
- size_t const copySize = (size_t)(lowPrefix - match);
- size_t const restSize = length - copySize;
-
- LZ4_memcpy(op, dictEnd - copySize, copySize);
- op += copySize;
- if (restSize > (size_t)(op - lowPrefix)) {
- /* overlap copy */
- BYTE * const endOfMatch = op + restSize;
- const BYTE *copyFrom = lowPrefix;
-
- while (op < endOfMatch)
- *op++ = *copyFrom++;
- } else {
- LZ4_memcpy(op, lowPrefix, restSize);
- op += restSize;
- }
- }
- continue;
- }
-
- /* copy match within block */
- cpy = op + length;
-
- /*
- * partialDecoding :
- * may not respect endBlock parsing restrictions
- */
- assert(op <= oend);
- if (partialDecoding &&
- (cpy > oend - MATCH_SAFEGUARD_DISTANCE)) {
- size_t const mlen = min(length, (size_t)(oend - op));
- const BYTE * const matchEnd = match + mlen;
- BYTE * const copyEnd = op + mlen;
-
- if (matchEnd > op) {
- /* overlap copy */
- while (op < copyEnd)
- *op++ = *match++;
- } else {
- LZ4_memcpy(op, match, mlen);
- }
- op = copyEnd;
- if (op == oend)
- break;
- continue;
- }
-
- if (unlikely(offset < 8)) {
- op[0] = match[0];
- op[1] = match[1];
- op[2] = match[2];
- op[3] = match[3];
- match += inc32table[offset];
- LZ4_memcpy(op + 4, match, 4);
- match -= dec64table[offset];
- } else {
- LZ4_copy8(op, match);
- match += 8;
- }
-
- op += 8;
-
- if (unlikely(cpy > oend - MATCH_SAFEGUARD_DISTANCE)) {
- BYTE * const oCopyLimit = oend - (WILDCOPYLENGTH - 1);
-
- if (cpy > oend - LASTLITERALS) {
- /*
- * Error : last LASTLITERALS bytes
- * must be literals (uncompressed)
- */
- goto _output_error;
- }
-
- if (op < oCopyLimit) {
- LZ4_wildCopy(op, match, oCopyLimit);
- match += oCopyLimit - op;
- op = oCopyLimit;
- }
- while (op < cpy)
- *op++ = *match++;
- } else {
- LZ4_copy8(op, match);
- if (length > 16)
- LZ4_wildCopy(op + 8, match + 8, cpy);
- }
- op = cpy; /* wildcopy correction */
- }
-
- /* end of decoding */
- if (endOnInput) {
- /* Nb of output bytes decoded */
- return (int) (((char *)op) - dst);
- } else {
- /* Nb of input bytes read */
- return (int) (((const char *)ip) - src);
- }
-
- /* Overflow error detected */
-_output_error:
- return (int) (-(((const char *)ip) - src)) - 1;
-}
-
-int LZ4_decompress_safe(const char *source, char *dest,
- int compressedSize, int maxDecompressedSize)
-{
- return LZ4_decompress_generic(source, dest,
- compressedSize, maxDecompressedSize,
- endOnInputSize, decode_full_block,
- noDict, (BYTE *)dest, NULL, 0);
-}
-
-int LZ4_decompress_safe_partial(const char *src, char *dst,
- int compressedSize, int targetOutputSize, int dstCapacity)
-{
- dstCapacity = min(targetOutputSize, dstCapacity);
- return LZ4_decompress_generic(src, dst, compressedSize, dstCapacity,
- endOnInputSize, partial_decode,
- noDict, (BYTE *)dst, NULL, 0);
-}
-
-/* ===== Instantiate a few more decoding cases, used more than once. ===== */
-
-static int LZ4_decompress_safe_withPrefix64k(const char *source, char *dest,
- int compressedSize, int maxOutputSize)
+int LZ4_decompress_safe(const char *source, char *dest, int compressedSize,
+ int maxDecompressedSize)
{
- return LZ4_decompress_generic(source, dest,
- compressedSize, maxOutputSize,
- endOnInputSize, decode_full_block,
- withPrefix64k,
- (BYTE *)dest - 64 * KB, NULL, 0);
+ return __lz4_decompress_safe(source, dest, compressedSize,
+ maxDecompressedSize);
}
-static int LZ4_decompress_safe_withSmallPrefix(const char *source, char *dest,
- int compressedSize,
- int maxOutputSize,
- size_t prefixSize)
+int LZ4_decompress_safe_partial(const char *source, char *dest,
+ int compressedSize, int targetOutputSize,
+ int maxDecompressedSize)
{
- return LZ4_decompress_generic(source, dest,
- compressedSize, maxOutputSize,
- endOnInputSize, decode_full_block,
- noDict,
- (BYTE *)dest - prefixSize, NULL, 0);
+ return __lz4_decompress_safe_partial(source, dest, compressedSize,
+ targetOutputSize,
+ maxDecompressedSize);
}
-static int LZ4_decompress_safe_forceExtDict(const char *source, char *dest,
- int compressedSize, int maxOutputSize,
- const void *dictStart, size_t dictSize)
-{
- return LZ4_decompress_generic(source, dest,
- compressedSize, maxOutputSize,
- endOnInputSize, decode_full_block,
- usingExtDict, (BYTE *)dest,
- (const BYTE *)dictStart, dictSize);
-}
-
-/*
- * The "double dictionary" mode, for use with e.g. ring buffers: the first part
- * of the dictionary is passed as prefix, and the second via dictStart + dictSize.
- * These routines are used only once, in LZ4_decompress_*_continue().
- */
-static FORCE_INLINE
-int LZ4_decompress_safe_doubleDict(const char *source, char *dest,
- int compressedSize, int maxOutputSize,
- size_t prefixSize,
- const void *dictStart, size_t dictSize)
-{
- return LZ4_decompress_generic(source, dest,
- compressedSize, maxOutputSize,
- endOnInputSize, decode_full_block,
- usingExtDict, (BYTE *)dest - prefixSize,
- (const BYTE *)dictStart, dictSize);
-}
-
-/* ===== streaming decompression functions ===== */
-
int LZ4_setStreamDecode(LZ4_streamDecode_t *LZ4_streamDecode,
- const char *dictionary, int dictSize)
+ const char *dictionary, int dictSize)
{
- LZ4_streamDecode_t_internal *lz4sd =
- &LZ4_streamDecode->internal_donotuse;
-
- lz4sd->prefixSize = (size_t) dictSize;
- lz4sd->prefixEnd = (const BYTE *) dictionary + dictSize;
- lz4sd->externalDict = NULL;
- lz4sd->extDictSize = 0;
- return 1;
+ return __lz4_setStreamDecode(LZ4_streamDecode, dictionary, dictSize);
}
-/*
- * *_continue() :
- * These decoding functions allow decompression of multiple blocks
- * in "streaming" mode.
- * Previously decoded blocks must still be available at the memory
- * position where they were decoded.
- * If it's not possible, save the relevant part of
- * decoded data into a safe buffer,
- * and indicate where it stands using LZ4_setStreamDecode()
- */
int LZ4_decompress_safe_continue(LZ4_streamDecode_t *LZ4_streamDecode,
- const char *source, char *dest, int compressedSize, int maxOutputSize)
+ const char *source, char *dest,
+ int compressedSize, int maxDecompressedSize)
{
- LZ4_streamDecode_t_internal *lz4sd =
- &LZ4_streamDecode->internal_donotuse;
- int result;
-
- if (lz4sd->prefixSize == 0) {
- /* The first call, no dictionary yet. */
- assert(lz4sd->extDictSize == 0);
- result = LZ4_decompress_safe(source, dest,
- compressedSize, maxOutputSize);
- if (result <= 0)
- return result;
- lz4sd->prefixSize = result;
- lz4sd->prefixEnd = (BYTE *)dest + result;
- } else if (lz4sd->prefixEnd == (BYTE *)dest) {
- /* They're rolling the current segment. */
- if (lz4sd->prefixSize >= 64 * KB - 1)
- result = LZ4_decompress_safe_withPrefix64k(source, dest,
- compressedSize, maxOutputSize);
- else if (lz4sd->extDictSize == 0)
- result = LZ4_decompress_safe_withSmallPrefix(source,
- dest, compressedSize, maxOutputSize,
- lz4sd->prefixSize);
- else
- result = LZ4_decompress_safe_doubleDict(source, dest,
- compressedSize, maxOutputSize,
- lz4sd->prefixSize,
- lz4sd->externalDict, lz4sd->extDictSize);
- if (result <= 0)
- return result;
- lz4sd->prefixSize += result;
- lz4sd->prefixEnd += result;
- } else {
- /*
- * The buffer wraps around, or they're
- * switching to another buffer.
- */
- lz4sd->extDictSize = lz4sd->prefixSize;
- lz4sd->externalDict = lz4sd->prefixEnd - lz4sd->extDictSize;
- result = LZ4_decompress_safe_forceExtDict(source, dest,
- compressedSize, maxOutputSize,
- lz4sd->externalDict, lz4sd->extDictSize);
- if (result <= 0)
- return result;
- lz4sd->prefixSize = result;
- lz4sd->prefixEnd = (BYTE *)dest + result;
- }
-
- return result;
+ return __lz4_decompress_safe_continue(LZ4_streamDecode, source, dest,
+ compressedSize,
+ maxDecompressedSize);
}
int LZ4_decompress_safe_usingDict(const char *source, char *dest,
- int compressedSize, int maxOutputSize,
+ int compressedSize, int maxDecompressedSize,
const char *dictStart, int dictSize)
{
- if (dictSize == 0)
- return LZ4_decompress_safe(source, dest,
- compressedSize, maxOutputSize);
- if (dictStart+dictSize == dest) {
- if (dictSize >= 64 * KB - 1)
- return LZ4_decompress_safe_withPrefix64k(source, dest,
- compressedSize, maxOutputSize);
- return LZ4_decompress_safe_withSmallPrefix(source, dest,
- compressedSize, maxOutputSize, dictSize);
- }
- return LZ4_decompress_safe_forceExtDict(source, dest,
- compressedSize, maxOutputSize, dictStart, dictSize);
+ return __lz4_decompress_safe_usingDict(source, dest, compressedSize,
+ maxDecompressedSize, dictStart,
+ dictSize);
}
#ifndef STATIC
diff --git a/lib/lz4/lz4defs.h b/lib/lz4/lz4defs.h
deleted file mode 100644
index 17277ec16919f3a554afc7d91ad515c493d745ca..0000000000000000000000000000000000000000
--- a/lib/lz4/lz4defs.h
+++ /dev/null
@@ -1,247 +0,0 @@
-#ifndef __LZ4DEFS_H__
-#define __LZ4DEFS_H__
-
-/*
- * lz4defs.h -- common and architecture specific defines for the kernel usage
-
- * LZ4 - Fast LZ compression algorithm
- * Copyright (C) 2011-2016, Yann Collet.
- * BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are
- * met:
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above
- * copyright notice, this list of conditions and the following disclaimer
- * in the documentation and/or other materials provided with the
- * distribution.
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- * You can contact the author at :
- * - LZ4 homepage : http://www.lz4.org
- * - LZ4 source repository : https://github.com/lz4/lz4
- *
- * Changed for kernel usage by:
- * Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
- */
-
-#include <linux/unaligned.h>
-
-#include <linux/bitops.h>
-#include <linux/string.h> /* memset, memcpy */
-#include <linux/lz4.h>
-
-#define FORCE_INLINE __always_inline
-
-/*-************************************
- * Basic Types
- **************************************/
-#include <linux/types.h>
-
-typedef uint8_t BYTE;
-typedef uint16_t U16;
-typedef uint32_t U32;
-typedef int32_t S32;
-typedef uint64_t U64;
-typedef uintptr_t uptrval;
-
-/*-************************************
- * Architecture specifics
- **************************************/
-#if defined(CONFIG_64BIT)
-#define LZ4_ARCH64 1
-#else
-#define LZ4_ARCH64 0
-#endif
-
-#if defined(__LITTLE_ENDIAN)
-#define LZ4_LITTLE_ENDIAN 1
-#else
-#define LZ4_LITTLE_ENDIAN 0
-#endif
-
-/*-************************************
- * Constants
- **************************************/
-#define MINMATCH 4
-
-#define WILDCOPYLENGTH 8
-#define LASTLITERALS 5
-#define MFLIMIT (WILDCOPYLENGTH + MINMATCH)
-/*
- * ensure it's possible to write 2 x wildcopyLength
- * without overflowing output buffer
- */
-#define MATCH_SAFEGUARD_DISTANCE ((2 * WILDCOPYLENGTH) - MINMATCH)
-
-/* Increase this value ==> compression run slower on incompressible data */
-#define LZ4_SKIPTRIGGER 6
-
-#define HASH_UNIT sizeof(size_t)
-
-#define KB (1 << 10)
-#define MB (1 << 20)
-#define GB (1U << 30)
-
-#define MAX_DISTANCE LZ4_DISTANCE_MAX
-#define STEPSIZE sizeof(size_t)
-
-#define ML_BITS 4
-#define ML_MASK ((1U << ML_BITS) - 1)
-#define RUN_BITS (8 - ML_BITS)
-#define RUN_MASK ((1U << RUN_BITS) - 1)
-
-/*-************************************
- * Reading and writing into memory
- **************************************/
-static FORCE_INLINE U16 LZ4_read16(const void *ptr)
-{
- return get_unaligned((const U16 *)ptr);
-}
-
-static FORCE_INLINE U32 LZ4_read32(const void *ptr)
-{
- return get_unaligned((const U32 *)ptr);
-}
-
-static FORCE_INLINE size_t LZ4_read_ARCH(const void *ptr)
-{
- return get_unaligned((const size_t *)ptr);
-}
-
-static FORCE_INLINE void LZ4_write16(void *memPtr, U16 value)
-{
- put_unaligned(value, (U16 *)memPtr);
-}
-
-static FORCE_INLINE void LZ4_write32(void *memPtr, U32 value)
-{
- put_unaligned(value, (U32 *)memPtr);
-}
-
-static FORCE_INLINE U16 LZ4_readLE16(const void *memPtr)
-{
- return get_unaligned_le16(memPtr);
-}
-
-static FORCE_INLINE void LZ4_writeLE16(void *memPtr, U16 value)
-{
- return put_unaligned_le16(value, memPtr);
-}
-
-/*
- * LZ4 relies on memcpy with a constant size being inlined. In freestanding
- * environments, the compiler can't assume the implementation of memcpy() is
- * standard compliant, so apply its specialized memcpy() inlining logic. When
- * possible, use __builtin_memcpy() to tell the compiler to analyze memcpy()
- * as-if it were standard compliant, so it can inline it in freestanding
- * environments. This is needed when decompressing the Linux Kernel, for example.
- */
-#define LZ4_memcpy(dst, src, size) __builtin_memcpy(dst, src, size)
-#define LZ4_memmove(dst, src, size) __builtin_memmove(dst, src, size)
-
-static FORCE_INLINE void LZ4_copy8(void *dst, const void *src)
-{
-#if LZ4_ARCH64
- U64 a = get_unaligned((const U64 *)src);
-
- put_unaligned(a, (U64 *)dst);
-#else
- U32 a = get_unaligned((const U32 *)src);
- U32 b = get_unaligned((const U32 *)src + 1);
-
- put_unaligned(a, (U32 *)dst);
- put_unaligned(b, (U32 *)dst + 1);
-#endif
-}
-
-/*
- * customized variant of memcpy,
- * which can overwrite up to 7 bytes beyond dstEnd
- */
-static FORCE_INLINE void LZ4_wildCopy(void *dstPtr,
- const void *srcPtr, void *dstEnd)
-{
- BYTE *d = (BYTE *)dstPtr;
- const BYTE *s = (const BYTE *)srcPtr;
- BYTE *const e = (BYTE *)dstEnd;
-
- do {
- LZ4_copy8(d, s);
- d += 8;
- s += 8;
- } while (d < e);
-}
-
-static FORCE_INLINE unsigned int LZ4_NbCommonBytes(register size_t val)
-{
-#if LZ4_LITTLE_ENDIAN
- return __ffs(val) >> 3;
-#else
- return (BITS_PER_LONG - 1 - __fls(val)) >> 3;
-#endif
-}
-
-static FORCE_INLINE unsigned int LZ4_count(
- const BYTE *pIn,
- const BYTE *pMatch,
- const BYTE *pInLimit)
-{
- const BYTE *const pStart = pIn;
-
- while (likely(pIn < pInLimit - (STEPSIZE - 1))) {
- size_t const diff = LZ4_read_ARCH(pMatch) ^ LZ4_read_ARCH(pIn);
-
- if (!diff) {
- pIn += STEPSIZE;
- pMatch += STEPSIZE;
- continue;
- }
-
- pIn += LZ4_NbCommonBytes(diff);
-
- return (unsigned int)(pIn - pStart);
- }
-
-#if LZ4_ARCH64
- if ((pIn < (pInLimit - 3))
- && (LZ4_read32(pMatch) == LZ4_read32(pIn))) {
- pIn += 4;
- pMatch += 4;
- }
-#endif
-
- if ((pIn < (pInLimit - 1))
- && (LZ4_read16(pMatch) == LZ4_read16(pIn))) {
- pIn += 2;
- pMatch += 2;
- }
-
- if ((pIn < pInLimit) && (*pMatch == *pIn))
- pIn++;
-
- return (unsigned int)(pIn - pStart);
-}
-
-typedef enum { noLimit = 0, limitedOutput = 1 } limitedOutput_directive;
-typedef enum { byPtr, byU32, byU16 } tableType_t;
-
-typedef enum { noDict = 0, withPrefix64k, usingExtDict } dict_directive;
-typedef enum { noDictIssue = 0, dictSmall } dictIssue_directive;
-
-typedef enum { endOnOutputSize = 0, endOnInputSize = 1 } endCondition_directive;
-typedef enum { decode_full_block = 0, partial_decode = 1 } earlyEnd_directive;
-
-#define LZ4_STATIC_ASSERT(c) BUILD_BUG_ON(!(c))
-
-#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH v2 13/15] lib/lz4: switch the decompressor to the vendored sources
2026-11-06 14:42 ` [PATCH v2 13/15] lib/lz4: switch the decompressor to the vendored sources Michal Wilczynski
@ 2026-11-10 5:39 ` Sergey Senozhatsky
0 siblings, 0 replies; 19+ messages in thread
From: Sergey Senozhatsky @ 2026-11-10 5:39 UTC (permalink / raw)
To: Michal Wilczynski
Cc: Nathan Chancellor, Nick Desaulniers, Bill Wendling, Justin Stitt,
Russell King, Thomas Bogendoerfer, James E.J. Bottomley,
Helge Deller, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger, Sven Schnelle, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Dave Hansen, x86, H. Peter Anvin,
Minchan Kim, Sergey Senozhatsky, Jens Axboe, Andrew Morton,
linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
On (26/11/06 15:42), Michal Wilczynski wrote:
[..]
> diff --git a/drivers/block/zram/backend_lz4.c b/drivers/block/zram/backend_lz4.c
> index 50265e3ce256490994cb6dae32a45da376e408e0..b9b8a5f678c155442fa3fd2c1984e45182e6fc38 100644
> --- a/drivers/block/zram/backend_lz4.c
> +++ b/drivers/block/zram/backend_lz4.c
> @@ -84,7 +84,7 @@ static int lz4_create(struct zcomp_params *params, struct zcomp_ctx *ctx)
> if (!zctx->mem)
> goto error;
> } else {
> - zctx->dstrm = kzalloc_obj(*zctx->dstrm);
> + zctx->dstrm = kzalloc(LZ4_MEM_DECOMPRESS, GFP_KERNEL);
> if (!zctx->dstrm)
> goto error;
>
> diff --git a/drivers/block/zram/backend_lz4hc.c b/drivers/block/zram/backend_lz4hc.c
> index e7ee3740219852f079d1e0866da6b213a3f3a583..1e09c93016264259c149e2913b2819571b56fab6 100644
> --- a/drivers/block/zram/backend_lz4hc.c
> +++ b/drivers/block/zram/backend_lz4hc.c
> @@ -65,7 +65,7 @@ static int lz4hc_create(struct zcomp_params *params, struct zcomp_ctx *ctx)
> if (!zctx->mem)
> goto error;
> } else {
> - zctx->dstrm = kzalloc_obj(*zctx->dstrm);
> + zctx->dstrm = kzalloc(LZ4_MEM_DECOMPRESS, GFP_KERNEL);
> if (!zctx->dstrm)
> goto error;
Ditto, outdated zram code.
My ask would be to not squash zram changes with lz4 changes, when (iff)
possible.
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v2 14/15] lib/lz4: replace lz4_kernel_api.h's prototypes with <linux/lz4.h>
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
` (11 preceding siblings ...)
2026-11-06 14:42 ` [PATCH v2 13/15] lib/lz4: switch the decompressor to the vendored sources Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
2026-11-06 14:42 ` [PATCH v2 15/15] MAINTAINERS: add an entry for the LZ4 compression library Michal Wilczynski
13 siblings, 0 replies; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
lz4_kernel_api.h carried its own copy of the twenty public prototypes
because it could not include <linux/lz4.h>, which defined the stream
types that upstream's headers also define.
The types are incomplete now, so it can. Two prototype lists for one
ABI is a drift hazard: the entry points were built against one and their
callers against the other, so no compiler ever saw both, and MODVERSIONS
would not catch a mismatch either, since it hashes the exporter's view
only. Now there is one list.
It also puts the public constants in scope of the entry points, so the
size assertions can name them, and the HC level clamp can use
LZ4HC_CLAMP_CLEVEL, static_asserted equal to upstream's
LZ4HC_CLEVEL_OPT_MIN.
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
lib/lz4/lz4_compress.c | 2 +-
lib/lz4/lz4_decompress.c | 2 +-
lib/lz4/lz4_kernel_api.h | 64 +++---------------------------------------------
lib/lz4/lz4hc_compress.c | 8 +++---
4 files changed, 11 insertions(+), 65 deletions(-)
diff --git a/lib/lz4/lz4_compress.c b/lib/lz4/lz4_compress.c
index 941e32a19ac972aef543c9a03b447a8a98bf5215..3e272584ea0c905b48b6bf0b7e77acfbad131bd1 100644
--- a/lib/lz4/lz4_compress.c
+++ b/lib/lz4/lz4_compress.c
@@ -30,7 +30,7 @@
/* Catch any divergence from upstream's layout at build time. */
static_assert(sizeof(LZ4_stream_t) == LZ4_STREAM_MINSIZE);
-static_assert(LZ4_STREAM_MINSIZE == 16416); /* LZ4_MEM_COMPRESS */
+static_assert(LZ4_STREAM_MINSIZE == LZ4_MEM_COMPRESS);
/* LZ4_initStream() returns NULL for a misaligned state, which
* LZ4_compress_fast_extState() dereferences and
diff --git a/lib/lz4/lz4_decompress.c b/lib/lz4/lz4_decompress.c
index 60b57862a2e387aa89fa26f6f8e2b5485b5d9995..5d68a6f59a62ddb3701dd3531cf606c05ef4e607 100644
--- a/lib/lz4/lz4_decompress.c
+++ b/lib/lz4/lz4_decompress.c
@@ -31,7 +31,7 @@
#endif
static_assert(sizeof(LZ4_streamDecode_t) == LZ4_STREAMDECODE_MINSIZE);
-static_assert(LZ4_STREAMDECODE_MINSIZE == 32); /* LZ4_MEM_DECOMPRESS */
+static_assert(LZ4_STREAMDECODE_MINSIZE == LZ4_MEM_DECOMPRESS);
int LZ4_decompress_safe(const char *source, char *dest, int compressedSize,
int maxDecompressedSize)
diff --git a/lib/lz4/lz4_kernel_api.h b/lib/lz4/lz4_kernel_api.h
index de75e5d976eb9dd318da5318a26b91d7aed81042..43605a0e4d7f543662af83251184968f0d1b70dd 100644
--- a/lib/lz4/lz4_kernel_api.h
+++ b/lib/lz4/lz4_kernel_api.h
@@ -3,19 +3,10 @@
* Copyright (c) 2026 Samsung Electronics Co., Ltd.
* Author: Michal Wilczynski <m.wilczynski@samsung.com>
*
- * lz4_kernel_api.h -- the LZ4 API this directory exports
+ * lz4_kernel_api.h -- hand the LZ4 names back to the kernel
*
- * lz4_deps.h renames upstream's entry points to __lz4_* so that the kernel's
- * own definitions can use the real names; this header undoes that renaming
- * and the optimize() override, and declares what lib/lz4 exports.
- *
- * The same functions are declared to callers in <linux/lz4.h>, in terms of
- * opaque stream types. We cannot include that header here -- it and
- * upstream's lz4.h describe the same library and collide -- so the
- * declarations are repeated, expressed in upstream's own types. Keep the two
- * in step; <linux/lz4.h> is the one callers compile against.
- *
- * Include after upstream/lz4.c or upstream/lz4hc.c.
+ * Undo lz4_deps.h's renaming and its optimize() override, and pull in
+ * <linux/lz4.h>. Include after upstream/lz4.c or upstream/lz4hc.c.
*/
#undef LZ4_compress_fast
@@ -40,51 +31,4 @@
#undef optimize
-/*
- * The objects that do not build lz4hc.c never see this type, and it is only
- * ever used through a pointer here. Upstream declares it the same way.
- */
-typedef union LZ4_streamHC_u LZ4_streamHC_t;
-
-/* Compression. wrkmem is LZ4_MEM_COMPRESS bytes, supplied by the caller. */
-int LZ4_compress_default(const char *source, char *dest, int inputSize,
- int maxOutputSize, void *wrkmem);
-int LZ4_compress_fast(const char *source, char *dest, int inputSize,
- int maxOutputSize, int acceleration, void *wrkmem);
-int LZ4_compress_destSize(const char *source, char *dest, int *sourceSizePtr,
- int targetDestSize, void *wrkmem);
-
-/* Streaming compression. */
-void LZ4_resetStream(LZ4_stream_t *LZ4_stream);
-int LZ4_loadDict(LZ4_stream_t *streamPtr, const char *dictionary,
- int dictSize);
-int LZ4_saveDict(LZ4_stream_t *streamPtr, char *safeBuffer, int dictSize);
-int LZ4_compress_fast_continue(LZ4_stream_t *streamPtr, const char *src,
- char *dst, int srcSize, int maxDstSize,
- int acceleration);
-
-/* Decompression. */
-int LZ4_decompress_safe(const char *source, char *dest, int compressedSize,
- int maxDecompressedSize);
-int LZ4_decompress_safe_partial(const char *source, char *dest,
- int compressedSize, int targetOutputSize,
- int maxDecompressedSize);
-int LZ4_setStreamDecode(LZ4_streamDecode_t *LZ4_streamDecode,
- const char *dictionary, int dictSize);
-int LZ4_decompress_safe_continue(LZ4_streamDecode_t *LZ4_streamDecode,
- const char *source, char *dest,
- int compressedSize, int maxDecompressedSize);
-int LZ4_decompress_safe_usingDict(const char *source, char *dest,
- int compressedSize, int maxDecompressedSize,
- const char *dictStart, int dictSize);
-
-/* HC compression. wrkmem is LZ4HC_MEM_COMPRESS bytes. */
-int LZ4_compress_HC(const char *src, char *dst, int srcSize, int dstCapacity,
- int compressionLevel, void *wrkmem);
-void LZ4_resetStreamHC(LZ4_streamHC_t *streamHCPtr, int compressionLevel);
-int LZ4_loadDictHC(LZ4_streamHC_t *streamHCPtr, const char *dictionary,
- int dictSize);
-int LZ4_compress_HC_continue(LZ4_streamHC_t *streamHCPtr, const char *src,
- char *dst, int srcSize, int maxDstSize);
-int LZ4_saveDictHC(LZ4_streamHC_t *streamHCPtr, char *safeBuffer,
- int maxDictSize);
+#include <linux/lz4.h>
diff --git a/lib/lz4/lz4hc_compress.c b/lib/lz4/lz4hc_compress.c
index 4d56d45028110805628be766b2758a8dffd56f0c..4f40a12df17cfed2bab7f5d87ac087b4d423cef7 100644
--- a/lib/lz4/lz4hc_compress.c
+++ b/lib/lz4/lz4hc_compress.c
@@ -39,7 +39,7 @@ static int LZ4_compressBound(int isize)
/* Catch any divergence from upstream's layout at build time. */
static_assert(sizeof(LZ4_streamHC_t) == LZ4_STREAMHC_MINSIZE);
-static_assert(LZ4_STREAMHC_MINSIZE == 262200); /* LZ4HC_MEM_COMPRESS */
+static_assert(LZ4_STREAMHC_MINSIZE == LZ4HC_MEM_COMPRESS);
/* Levels >= LZ4HC_CLEVEL_OPT_MIN (10) reach the optimal parser, which needs
* a ~64K opt[]: too big for a kernel stack, and in the heap mode lz4_deps.h
@@ -48,10 +48,12 @@ static_assert(LZ4_STREAMHC_MINSIZE == 262200); /* LZ4HC_MEM_COMPRESS */
* working. Levels 3..9 keep their meaning; 1 and 2 are upstream's lz4mid,
* faster and weaker than the shallow hash chain the fork used for them.
*/
+static_assert(LZ4HC_CLAMP_CLEVEL == LZ4HC_CLEVEL_OPT_MIN);
+
static int lz4hc_clamp_level(int compressionLevel)
{
- if (compressionLevel >= LZ4HC_CLEVEL_OPT_MIN)
- return LZ4HC_CLEVEL_OPT_MIN - 1;
+ if (compressionLevel >= LZ4HC_CLAMP_CLEVEL)
+ return LZ4HC_CLAMP_CLEVEL - 1;
return compressionLevel;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 15/15] MAINTAINERS: add an entry for the LZ4 compression library
2026-11-06 14:42 ` [PATCH v2 00/15] lib/lz4: stop forking upstream LZ4, vendor it instead Michal Wilczynski
` (12 preceding siblings ...)
2026-11-06 14:42 ` [PATCH v2 14/15] lib/lz4: replace lz4_kernel_api.h's prototypes with <linux/lz4.h> Michal Wilczynski
@ 2026-11-06 14:42 ` Michal Wilczynski
13 siblings, 0 replies; 19+ messages in thread
From: Michal Wilczynski @ 2026-11-06 14:42 UTC (permalink / raw)
To: Michal Wilczynski, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Russell King, Thomas Bogendoerfer,
James E.J. Bottomley, Helge Deller, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Minchan Kim, Sergey Senozhatsky, Jens Axboe,
Andrew Morton
Cc: linux-kernel, llvm, linux-arm-kernel, linux-mips, linux-parisc,
linux-s390, linux-block, Yann Collet, Nick Terrell, Gao Xiang,
Chao Yu, Jaegeuk Kim, Eric Biggers, Herbert Xu, David Laight,
Sungguk, Jaehoon Chung, Marek Szyprowski, linux-erofs,
linux-f2fs-devel, linux-crypto
lib/lz4/ has never had one: get_maintainer.pl returns only the LKML
fallback for every file in it, including include/linux/lz4.h, which eight
subsystems build against.
The entry also takes lib/decompress_unlz4.c and its header, which
decode LZ4 kernel images and initramfs through lib/lz4, the way XZ
EMBEDDED covers its own equivalents, and crypto/lz4.c and crypto/lz4hc.c,
the crypto API wrappers zswap uses, as ZSTD takes crypto/zstd.c.
This matters more now than it did. With the previous patches lib/lz4 is
vendored upstream sources, and someone has to bump them to new upstream
releases and decide what the kernel-side glue may do. I am willing to do
that, so add myself as its maintainer.
Signed-off-by: Michal Wilczynski <m.wilczynski@samsung.com>
---
MAINTAINERS | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index cc3cae2e378b34aeb705cef654c720ca9f2ba223..598d87ca772eecbb1ed169d51ff0eb8cdd2e50ad 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -15606,6 +15606,16 @@ S: Supported
F: drivers/net/pcs/pcs-lynx.c
F: include/linux/pcs-lynx.h
+LZ4 COMPRESSION LIBRARY
+M: Michal Wilczynski <m.wilczynski@samsung.com>
+S: Maintained
+F: crypto/lz4.c
+F: crypto/lz4hc.c
+F: include/linux/decompress/unlz4.h
+F: include/linux/lz4.h
+F: lib/decompress_unlz4.c
+F: lib/lz4/
+
M68K ARCHITECTURE
M: Geert Uytterhoeven <geert@linux-m68k.org>
L: linux-m68k@lists.linux-m68k.org
--
2.34.1
^ permalink raw reply related [flat|nested] 19+ messages in thread