* [PATCH v3 0/2] zstd: use x86 feature infrastructure for BMI2 dispatch
@ 2026-09-01 11:07 Usama Arif
2026-09-01 11:07 ` [PATCH v3 1/2] lib/zstd: add fallback aliases for disabled BMI2 variants Usama Arif
2026-09-01 11:07 ` [PATCH v3 2/2] zstd: use cpu_feature_enabled() for in-kernel BMI2 dispatch Usama Arif
0 siblings, 2 replies; 6+ messages in thread
From: Usama Arif @ 2026-09-01 11:07 UTC (permalink / raw)
To: dsterba, linux-kernel, terrelln, terrelln, linux-crypto, yosry,
ebiggers, torvalds
Cc: hannes, nphamcs, chengming.zhou, shakeel.butt, kernel-team,
Usama Arif
Zstd currently probes CPUID whenever a compression or decompression
context is initialized, stores the result in the context, and tests that
value at each BMI2 dispatch site. For normal x86 kernel builds this
duplicates the kernel's CPU feature infrastructure, bypasses its feature
policy, and leaves an ordinary runtime test in the dispatch path.
Use cpu_feature_enabled(X86_FEATURE_BMI2) directly at the dispatch sites
for normal x86 kernel objects. This uses the x86 alternatives-backed
static CPU feature mechanism, allowing the feature test to be resolved at
boot instead of loading and testing a value stored in each context.
ZSTD_USE_BMI2() keeps the other build modes working as before. It expands
to the caller-provided flag for standalone and preboot builds and to false
when DYNAMIC_BMI2 is disabled. ZSTD_SET_BMI2() similarly stores the
caller-provided state only when it will be used, avoiding preprocessor
conditionals at the context initialization sites.
Patch 1 adds aliases from BMI2 function names to their default
implementations when the BMI2 variants are not compiled. This is a
no-functional-change preparation: after patch 2 removes the affected
selector-level preprocessor guards, the compiler must still resolve the
function named in an if (0) branch before eliminating it.
Patch 2 adds ZSTD_USE_BMI2() and ZSTD_SET_BMI2(), converts the runtime
selectors, and avoids Zstd's private CPUID probes in normal x86 kernel
objects. The kernel-specific policy lives in zstd_deps.h. Preboot builds
are excluded because the normal alternatives infrastructure is not
available there, so they retain the existing raw-CPUID dispatch.
A 4 KiB zstd-generic crypto_acomp benchmark [1] in a one-vCPU KVM guest
gave these median results:
Before After Change
Compression 16,634 ns 13,394 ns -19.5%
Decompression 3,480 ns 963 ns -72.3%
The improvement is especially large in a guest because raw CPUID causes
a VM exit.
[1] https://gist.github.com/uarif1/5cf02f0e22c23f0d1b3d84348f12914c
v2 -> v3: https://lore.kernel.org/all/20260830222100.2706175-1-usama.arif@linux.dev/
- Add ZSTD_SET_BMI2() and use it at the context initialization sites.
(Linus Torvalds)
- Move ZSTD_USE_KERNEL_CPU_FEATURES from Makefile define into zstd_deps.h.
(Linus Torvalds)
v1 -> v2:
https://lore.kernel.org/all/20260826122558.2662013-1-usama.arif@linux.dev/
- Replace the proposed cached feature value with
cpu_feature_enabled(X86_FEATURE_BMI2) at each dispatch site.
(Eric Biggers and Linus Torvalds)
- Check only X86_FEATURE_BMI2 instead of BMI1, BMI2, and ABM.
(Linus Torvalds)
- Split the fallback aliases into a separate no-functional-change patch.
Usama Arif (2):
lib/zstd: add fallback aliases for disabled BMI2 variants
zstd: use cpu_feature_enabled() for in-kernel BMI2 dispatch
lib/zstd/common/compiler.h | 12 +++++++
lib/zstd/common/entropy_common.c | 16 ++++++----
lib/zstd/common/fse_decompress.c | 9 ++++--
lib/zstd/common/zstd_deps.h | 5 +++
lib/zstd/compress/huf_compress.c | 6 +++-
lib/zstd/compress/zstd_compress.c | 12 +++----
lib/zstd/compress/zstd_compress_internal.h | 9 ++++++
lib/zstd/compress/zstd_compress_sequences.c | 12 +++++--
lib/zstd/compress/zstd_compress_superblock.c | 2 +-
lib/zstd/decompress/huf_decompress.c | 22 +++++++------
lib/zstd/decompress/zstd_decompress.c | 4 +--
lib/zstd/decompress/zstd_decompress_block.c | 31 ++++++++++++-------
.../decompress/zstd_decompress_internal.h | 2 +-
13 files changed, 97 insertions(+), 45 deletions(-)
base-commit: 4b18edbd8e70f7e6860d56370f13244896d0f95c
--
2.53.0-Meta
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v3 1/2] lib/zstd: add fallback aliases for disabled BMI2 variants
2026-09-01 11:07 [PATCH v3 0/2] zstd: use x86 feature infrastructure for BMI2 dispatch Usama Arif
@ 2026-09-01 11:07 ` Usama Arif
2026-09-03 12:35 ` Dhruva G
2026-09-01 11:07 ` [PATCH v3 2/2] zstd: use cpu_feature_enabled() for in-kernel BMI2 dispatch Usama Arif
1 sibling, 1 reply; 6+ messages in thread
From: Usama Arif @ 2026-09-01 11:07 UTC (permalink / raw)
To: dsterba, linux-kernel, terrelln, terrelln, linux-crypto, yosry,
ebiggers, torvalds
Cc: hannes, nphamcs, chengming.zhou, shakeel.butt, kernel-team,
Usama Arif
When dynamic BMI2 dispatch is disabled, the BMI2-specific functions are
not compiled and each selector is conditionally compiled to avoid naming
them.
The selector-level preprocessor guards will be replaced with a predicate
that becomes constant false when dynamic BMI2 dispatch is disabled.
Although the compiler eliminates an `if (0)` branch, it must still parse
and resolve the BMI2 function referenced by it.
Add aliases from the unavailable BMI2 function names to their default
implementations. These aliases make the names valid without emitting
BMI2-specific code. The selectors remain unchanged in this patch, so the
aliases are not used yet and there is no code-generation change.
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
lib/zstd/common/entropy_common.c | 4 ++++
lib/zstd/common/fse_decompress.c | 2 ++
lib/zstd/compress/zstd_compress_sequences.c | 4 ++++
lib/zstd/decompress/huf_decompress.c | 4 ++++
lib/zstd/decompress/zstd_decompress_block.c | 12 ++++++++++++
5 files changed, 26 insertions(+)
diff --git a/lib/zstd/common/entropy_common.c b/lib/zstd/common/entropy_common.c
index 6cdd82233fb59..15a7c14ae8040 100644
--- a/lib/zstd/common/entropy_common.c
+++ b/lib/zstd/common/entropy_common.c
@@ -202,6 +202,8 @@ BMI2_TARGET_ATTRIBUTE static size_t FSE_readNCount_body_bmi2(
{
return FSE_readNCount_body(normalizedCounter, maxSVPtr, tableLogPtr, headerBuffer, hbSize);
}
+#else
+#define FSE_readNCount_body_bmi2 FSE_readNCount_body_default
#endif
size_t FSE_readNCount_bmi2(
@@ -323,6 +325,8 @@ static BMI2_TARGET_ATTRIBUTE size_t HUF_readStats_body_bmi2(BYTE* huffWeight, si
{
return HUF_readStats_body(huffWeight, hwSize, rankStats, nbSymbolsPtr, tableLogPtr, src, srcSize, workSpace, wkspSize, 1);
}
+#else
+#define HUF_readStats_body_bmi2 HUF_readStats_body_default
#endif
size_t HUF_readStats_wksp(BYTE* huffWeight, size_t hwSize, U32* rankStats,
diff --git a/lib/zstd/common/fse_decompress.c b/lib/zstd/common/fse_decompress.c
index 15081d8dc607c..8d9c43b928833 100644
--- a/lib/zstd/common/fse_decompress.c
+++ b/lib/zstd/common/fse_decompress.c
@@ -300,6 +300,8 @@ BMI2_TARGET_ATTRIBUTE static size_t FSE_decompress_wksp_body_bmi2(void* dst, siz
{
return FSE_decompress_wksp_body(dst, dstCapacity, cSrc, cSrcSize, maxLog, workSpace, wkspSize, 1);
}
+#else
+#define FSE_decompress_wksp_body_bmi2 FSE_decompress_wksp_body_default
#endif
size_t FSE_decompress_wksp_bmi2(void* dst, size_t dstCapacity, const void* cSrc, size_t cSrcSize, unsigned maxLog, void* workSpace, size_t wkspSize, int bmi2)
diff --git a/lib/zstd/compress/zstd_compress_sequences.c b/lib/zstd/compress/zstd_compress_sequences.c
index 256980c9d85ad..64abcdf2c5714 100644
--- a/lib/zstd/compress/zstd_compress_sequences.c
+++ b/lib/zstd/compress/zstd_compress_sequences.c
@@ -415,6 +415,10 @@ ZSTD_encodeSequences_bmi2(
sequences, nbSeq, longOffsets);
}
+#else
+
+#define ZSTD_encodeSequences_bmi2 ZSTD_encodeSequences_default
+
#endif
size_t ZSTD_encodeSequences(
diff --git a/lib/zstd/decompress/huf_decompress.c b/lib/zstd/decompress/huf_decompress.c
index ac8b87f48f847..5663ae524cd83 100644
--- a/lib/zstd/decompress/huf_decompress.c
+++ b/lib/zstd/decompress/huf_decompress.c
@@ -700,6 +700,8 @@ size_t HUF_decompress4X1_usingDTable_internal_bmi2(void* dst, size_t dstSize, vo
size_t cSrcSize, HUF_DTable const* DTable) {
return HUF_decompress4X1_usingDTable_internal_body(dst, dstSize, cSrc, cSrcSize, DTable);
}
+#else
+#define HUF_decompress4X1_usingDTable_internal_bmi2 HUF_decompress4X1_usingDTable_internal_default
#endif
static
@@ -1503,6 +1505,8 @@ size_t HUF_decompress4X2_usingDTable_internal_bmi2(void* dst, size_t dstSize, vo
size_t cSrcSize, HUF_DTable const* DTable) {
return HUF_decompress4X2_usingDTable_internal_body(dst, dstSize, cSrc, cSrcSize, DTable);
}
+#else
+#define HUF_decompress4X2_usingDTable_internal_bmi2 HUF_decompress4X2_usingDTable_internal_default
#endif
static
diff --git a/lib/zstd/decompress/zstd_decompress_block.c b/lib/zstd/decompress/zstd_decompress_block.c
index 710eb0ffd5a37..9c4215e435014 100644
--- a/lib/zstd/decompress/zstd_decompress_block.c
+++ b/lib/zstd/decompress/zstd_decompress_block.c
@@ -622,6 +622,8 @@ BMI2_TARGET_ATTRIBUTE static void ZSTD_buildFSETable_body_bmi2(ZSTD_seqSymbol* d
ZSTD_buildFSETable_body(dt, normalizedCounter, maxSymbolValue,
baseValue, nbAdditionalBits, tableLog, wksp, wkspSize);
}
+#else
+#define ZSTD_buildFSETable_body_bmi2 ZSTD_buildFSETable_body_default
#endif
void ZSTD_buildFSETable(ZSTD_seqSymbol* dt,
@@ -1934,6 +1936,16 @@ ZSTD_decompressSequencesLong_bmi2(ZSTD_DCtx* dctx,
}
#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT */
+#else
+
+#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG
+#define ZSTD_decompressSequences_bmi2 ZSTD_decompressSequences_default
+#define ZSTD_decompressSequencesSplitLitBuffer_bmi2 ZSTD_decompressSequencesSplitLitBuffer_default
+#endif
+#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT
+#define ZSTD_decompressSequencesLong_bmi2 ZSTD_decompressSequencesLong_default
+#endif
+
#endif /* DYNAMIC_BMI2 */
#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG
--
2.53.0-Meta
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v3 2/2] zstd: use cpu_feature_enabled() for in-kernel BMI2 dispatch
2026-09-01 11:07 [PATCH v3 0/2] zstd: use x86 feature infrastructure for BMI2 dispatch Usama Arif
2026-09-01 11:07 ` [PATCH v3 1/2] lib/zstd: add fallback aliases for disabled BMI2 variants Usama Arif
@ 2026-09-01 11:07 ` Usama Arif
2026-09-02 17:40 ` Dhruva G
1 sibling, 1 reply; 6+ messages in thread
From: Usama Arif @ 2026-09-01 11:07 UTC (permalink / raw)
To: dsterba, linux-kernel, terrelln, terrelln, linux-crypto, yosry,
ebiggers, torvalds
Cc: hannes, nphamcs, chengming.zhou, shakeel.butt, kernel-team,
Usama Arif
Zstd's dynamic BMI2 implementation probes CPUID when a compression or
decompression context is initialized, stores the result in the context,
and tests that value at every dispatch site. In normal kernel builds this
bypasses the x86 feature policy and uses ordinary runtime branches instead
of allowing x86 alternatives to resolve the feature check at boot.
Add ZSTD_USE_BMI2() and use it at every runtime BMI2/default selector. For
normal x86 kernel objects, the predicate expands directly to
cpu_feature_enabled(X86_FEATURE_BMI2). When dynamic BMI2 dispatch is not
available it is false; other builds retain the caller-provided flag. Keep
the existing HUF conditional layout because DYNAMIC_BMI2 also controls
whether target-attributed variants are emitted.
Add the matching ZSTD_SET_BMI2() abstraction for context initialization.
Normal x86 kernel objects and builds without dynamic BMI2 do not cache CPU
state. Other builds with dynamic dispatch, including preboot, retain the
existing behavior. Keep the BMI2 members in the context structures so
their layouts do not change, and make the accessors return zero when
cached state is unused.
Select the normal-kernel policy in zstd_deps.h. Builds which define
__DISABLE_EXPORTS, including the x86 preboot decompressor, retain the existing
CPUID-backed dispatch because the normal alternatives infrastructure is not
available there.
A 4 KiB zstd-generic crypto_acomp benchmark in a one-vCPU KVM guest gave
these median results:
Before After Change
Compression 16,634 ns/op 13,394 ns/op -19.5%
Decompression 3,480 ns/op 963 ns/op -72.3%
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
lib/zstd/common/compiler.h | 12 ++++++++++++
lib/zstd/common/entropy_common.c | 12 ++++++------
lib/zstd/common/fse_decompress.c | 7 ++++---
lib/zstd/common/zstd_deps.h | 5 +++++
lib/zstd/compress/huf_compress.c | 6 +++++-
lib/zstd/compress/zstd_compress.c | 12 ++++++------
lib/zstd/compress/zstd_compress_internal.h | 9 +++++++++
lib/zstd/compress/zstd_compress_sequences.c | 8 +++++---
lib/zstd/compress/zstd_compress_superblock.c | 2 +-
lib/zstd/decompress/huf_decompress.c | 18 +++++++++---------
lib/zstd/decompress/zstd_decompress.c | 4 +---
lib/zstd/decompress/zstd_decompress_block.c | 19 +++++++------------
.../decompress/zstd_decompress_internal.h | 2 +-
13 files changed, 71 insertions(+), 45 deletions(-)
diff --git a/lib/zstd/common/compiler.h b/lib/zstd/common/compiler.h
index dc9bd15e174e9..47f6c0372c58c 100644
--- a/lib/zstd/common/compiler.h
+++ b/lib/zstd/common/compiler.h
@@ -14,6 +14,7 @@
#include <linux/types.h>
+#include "zstd_deps.h"
#include "portability_macros.h"
/*-*******************************************************
@@ -96,6 +97,17 @@
*/
#define BMI2_TARGET_ATTRIBUTE TARGET_ATTRIBUTE("lzcnt,bmi,bmi2")
+#if !DYNAMIC_BMI2
+# define ZSTD_USE_BMI2(bmi2) 0
+# define ZSTD_SET_BMI2(state, value) do { } while (0)
+#elif defined(ZSTD_USE_KERNEL_CPU_FEATURES)
+# define ZSTD_USE_BMI2(bmi2) cpu_feature_enabled(X86_FEATURE_BMI2)
+# define ZSTD_SET_BMI2(state, value) do { } while (0)
+#else
+# define ZSTD_USE_BMI2(bmi2) (bmi2)
+# define ZSTD_SET_BMI2(state, value) do { (state) = (value); } while (0)
+#endif
+
/* prefetch
* can be disabled, by declaring NO_PREFETCH build macro */
#if ( (__GNUC__ >= 4) || ( (__GNUC__ == 3) && (__GNUC_MINOR__ >= 1) ) )
diff --git a/lib/zstd/common/entropy_common.c b/lib/zstd/common/entropy_common.c
index 15a7c14ae8040..701bd495b13c8 100644
--- a/lib/zstd/common/entropy_common.c
+++ b/lib/zstd/common/entropy_common.c
@@ -16,6 +16,10 @@
/* *************************************
* Dependencies
***************************************/
+#include "zstd_deps.h"
+#if defined(ZSTD_USE_KERNEL_CPU_FEATURES)
+#include <asm/cpufeature.h>
+#endif
#include "mem.h"
#include "error_private.h" /* ERR_*, ERROR */
#define FSE_STATIC_LINKING_ONLY /* FSE_MIN_TABLELOG */
@@ -210,11 +214,9 @@ size_t FSE_readNCount_bmi2(
short* normalizedCounter, unsigned* maxSVPtr, unsigned* tableLogPtr,
const void* headerBuffer, size_t hbSize, int bmi2)
{
-#if DYNAMIC_BMI2
- if (bmi2) {
+ if (ZSTD_USE_BMI2(bmi2)) {
return FSE_readNCount_body_bmi2(normalizedCounter, maxSVPtr, tableLogPtr, headerBuffer, hbSize);
}
-#endif
(void)bmi2;
return FSE_readNCount_body_default(normalizedCounter, maxSVPtr, tableLogPtr, headerBuffer, hbSize);
}
@@ -335,11 +337,9 @@ size_t HUF_readStats_wksp(BYTE* huffWeight, size_t hwSize, U32* rankStats,
void* workSpace, size_t wkspSize,
int flags)
{
-#if DYNAMIC_BMI2
- if (flags & HUF_flags_bmi2) {
+ if (ZSTD_USE_BMI2(flags & HUF_flags_bmi2)) {
return HUF_readStats_body_bmi2(huffWeight, hwSize, rankStats, nbSymbolsPtr, tableLogPtr, src, srcSize, workSpace, wkspSize);
}
-#endif
(void)flags;
return HUF_readStats_body_default(huffWeight, hwSize, rankStats, nbSymbolsPtr, tableLogPtr, src, srcSize, workSpace, wkspSize);
}
diff --git a/lib/zstd/common/fse_decompress.c b/lib/zstd/common/fse_decompress.c
index 8d9c43b928833..008ce3401726a 100644
--- a/lib/zstd/common/fse_decompress.c
+++ b/lib/zstd/common/fse_decompress.c
@@ -24,6 +24,9 @@
#include "fse.h"
#include "error_private.h"
#include "zstd_deps.h" /* ZSTD_memcpy */
+#if defined(ZSTD_USE_KERNEL_CPU_FEATURES)
+#include <asm/cpufeature.h>
+#endif
#include "bits.h" /* ZSTD_highbit32 */
@@ -306,11 +309,9 @@ BMI2_TARGET_ATTRIBUTE static size_t FSE_decompress_wksp_body_bmi2(void* dst, siz
size_t FSE_decompress_wksp_bmi2(void* dst, size_t dstCapacity, const void* cSrc, size_t cSrcSize, unsigned maxLog, void* workSpace, size_t wkspSize, int bmi2)
{
-#if DYNAMIC_BMI2
- if (bmi2) {
+ if (ZSTD_USE_BMI2(bmi2)) {
return FSE_decompress_wksp_body_bmi2(dst, dstCapacity, cSrc, cSrcSize, maxLog, workSpace, wkspSize);
}
-#endif
(void)bmi2;
return FSE_decompress_wksp_body_default(dst, dstCapacity, cSrc, cSrcSize, maxLog, workSpace, wkspSize);
}
diff --git a/lib/zstd/common/zstd_deps.h b/lib/zstd/common/zstd_deps.h
index f931f7d0e2947..a312029df5660 100644
--- a/lib/zstd/common/zstd_deps.h
+++ b/lib/zstd/common/zstd_deps.h
@@ -26,6 +26,11 @@
#ifndef ZSTD_DEPS_COMMON
#define ZSTD_DEPS_COMMON
+#if defined(__KERNEL__) && defined(CONFIG_X86) && \
+ !defined(__DISABLE_EXPORTS)
+#define ZSTD_USE_KERNEL_CPU_FEATURES
+#endif
+
#include <linux/limits.h>
#include <linux/stddef.h>
diff --git a/lib/zstd/compress/huf_compress.c b/lib/zstd/compress/huf_compress.c
index 0b229f5d2ae22..f3f53240c981d 100644
--- a/lib/zstd/compress/huf_compress.c
+++ b/lib/zstd/compress/huf_compress.c
@@ -22,6 +22,9 @@
* Includes
****************************************************************/
#include "../common/zstd_deps.h" /* ZSTD_memcpy, ZSTD_memset */
+#if defined(ZSTD_USE_KERNEL_CPU_FEATURES)
+#include <asm/cpufeature.h>
+#endif
#include "../common/compiler.h"
#include "../common/bitstream.h"
#include "hist.h"
@@ -1138,9 +1141,10 @@ HUF_compress1X_usingCTable_internal(void* dst, size_t dstSize,
const void* src, size_t srcSize,
const HUF_CElt* CTable, const int flags)
{
- if (flags & HUF_flags_bmi2) {
+ if (ZSTD_USE_BMI2(flags & HUF_flags_bmi2)) {
return HUF_compress1X_usingCTable_internal_bmi2(dst, dstSize, src, srcSize, CTable);
}
+ (void)flags;
return HUF_compress1X_usingCTable_internal_default(dst, dstSize, src, srcSize, CTable);
}
diff --git a/lib/zstd/compress/zstd_compress.c b/lib/zstd/compress/zstd_compress.c
index c41a747413e01..638bc0bf83127 100644
--- a/lib/zstd/compress/zstd_compress.c
+++ b/lib/zstd/compress/zstd_compress.c
@@ -102,7 +102,7 @@ static void ZSTD_initCCtx(ZSTD_CCtx* cctx, ZSTD_customMem memManager)
assert(cctx != NULL);
ZSTD_memset(cctx, 0, sizeof(*cctx));
cctx->customMem = memManager;
- cctx->bmi2 = ZSTD_cpuSupportsBmi2();
+ ZSTD_SET_BMI2(cctx->bmi2, ZSTD_cpuSupportsBmi2());
{ size_t const err = ZSTD_CCtx_reset(cctx, ZSTD_reset_parameters);
assert(!ZSTD_isError(err));
(void)err;
@@ -142,7 +142,7 @@ ZSTD_CCtx* ZSTD_initStaticCCtx(void* workspace, size_t workspaceSize)
cctx->blockState.nextCBlock = (ZSTD_compressedBlockState_t*)ZSTD_cwksp_reserve_object(&cctx->workspace, sizeof(ZSTD_compressedBlockState_t));
cctx->tmpWorkspace = ZSTD_cwksp_reserve_object(&cctx->workspace, TMP_WORKSPACE_SIZE);
cctx->tmpWkspSize = TMP_WORKSPACE_SIZE;
- cctx->bmi2 = ZSTD_cpuid_bmi2(ZSTD_cpuid());
+ ZSTD_SET_BMI2(cctx->bmi2, ZSTD_cpuid_bmi2(ZSTD_cpuid()));
return cctx;
}
@@ -4042,7 +4042,7 @@ ZSTD_compressSeqStore_singleBlock(ZSTD_CCtx* zc,
op + ZSTD_blockHeaderSize, dstCapacity - ZSTD_blockHeaderSize,
srcSize,
zc->tmpWorkspace, zc->tmpWkspSize /* statically allocated in resetCCtx */,
- zc->bmi2);
+ ZSTD_CCtx_get_bmi2(zc));
FORWARD_IF_ERROR(cSeqsSize, "ZSTD_entropyCompressSeqStore failed!");
if (!zc->isFirstBlock &&
@@ -4332,7 +4332,7 @@ ZSTD_compressBlock_internal(ZSTD_CCtx* zc,
dst, dstCapacity,
srcSize,
zc->tmpWorkspace, zc->tmpWkspSize /* statically allocated in resetCCtx */,
- zc->bmi2);
+ ZSTD_CCtx_get_bmi2(zc));
if (frame &&
/* We don't want to emit our first block as a RLE even if it qualifies because
@@ -6796,7 +6796,7 @@ ZSTD_compressSequences_internal(ZSTD_CCtx* cctx,
op + ZSTD_blockHeaderSize /* Leave space for block header */, dstCapacity - ZSTD_blockHeaderSize,
blockSize,
cctx->tmpWorkspace, cctx->tmpWkspSize /* statically allocated in resetCCtx */,
- cctx->bmi2);
+ ZSTD_CCtx_get_bmi2(cctx));
FORWARD_IF_ERROR(compressedSeqsSize, "Compressing sequences of block failed");
DEBUGLOG(5, "Compressed sequences size: %zu", compressedSeqsSize);
@@ -7321,7 +7321,7 @@ ZSTD_compressSequencesAndLiterals_internal(ZSTD_CCtx* cctx,
&cctx->blockState.prevCBlock->entropy, &cctx->blockState.nextCBlock->entropy,
&cctx->appliedParams,
cctx->tmpWorkspace, cctx->tmpWkspSize /* statically allocated in resetCCtx */,
- cctx->bmi2);
+ ZSTD_CCtx_get_bmi2(cctx));
FORWARD_IF_ERROR(compressedSeqsSize, "Compressing sequences of block failed");
/* note: the spec forbids for any compressed block to be larger than maximum block size */
if (compressedSeqsSize > cctx->blockSizeMax) compressedSeqsSize = 0;
diff --git a/lib/zstd/compress/zstd_compress_internal.h b/lib/zstd/compress/zstd_compress_internal.h
index b109783858763..99ebe1e0c923f 100644
--- a/lib/zstd/compress/zstd_compress_internal.h
+++ b/lib/zstd/compress/zstd_compress_internal.h
@@ -537,6 +537,15 @@ struct ZSTD_CCtx_s {
size_t extSeqBufCapacity;
};
+MEM_STATIC int ZSTD_CCtx_get_bmi2(const struct ZSTD_CCtx_s *cctx) {
+#if DYNAMIC_BMI2 && !defined(ZSTD_USE_KERNEL_CPU_FEATURES)
+ return cctx->bmi2;
+#else
+ (void)cctx;
+ return 0;
+#endif
+}
+
typedef enum { ZSTD_dtlm_fast, ZSTD_dtlm_full } ZSTD_dictTableLoadMethod_e;
typedef enum { ZSTD_tfp_forCCtx, ZSTD_tfp_forCDict } ZSTD_tableFillPurpose_e;
diff --git a/lib/zstd/compress/zstd_compress_sequences.c b/lib/zstd/compress/zstd_compress_sequences.c
index 64abcdf2c5714..7a03a8fceea7d 100644
--- a/lib/zstd/compress/zstd_compress_sequences.c
+++ b/lib/zstd/compress/zstd_compress_sequences.c
@@ -12,6 +12,10 @@
/*-*************************************
* Dependencies
***************************************/
+#include "../common/zstd_deps.h"
+#if defined(ZSTD_USE_KERNEL_CPU_FEATURES)
+#include <asm/cpufeature.h>
+#endif
#include "zstd_compress_sequences.h"
/*
@@ -429,15 +433,13 @@ size_t ZSTD_encodeSequences(
SeqDef const* sequences, size_t nbSeq, int longOffsets, int bmi2)
{
DEBUGLOG(5, "ZSTD_encodeSequences: dstCapacity = %u", (unsigned)dstCapacity);
-#if DYNAMIC_BMI2
- if (bmi2) {
+ if (ZSTD_USE_BMI2(bmi2)) {
return ZSTD_encodeSequences_bmi2(dst, dstCapacity,
CTable_MatchLength, mlCodeTable,
CTable_OffsetBits, ofCodeTable,
CTable_LitLength, llCodeTable,
sequences, nbSeq, longOffsets);
}
-#endif
(void)bmi2;
return ZSTD_encodeSequences_default(dst, dstCapacity,
CTable_MatchLength, mlCodeTable,
diff --git a/lib/zstd/compress/zstd_compress_superblock.c b/lib/zstd/compress/zstd_compress_superblock.c
index dc12d64e935c4..fada5979c3b84 100644
--- a/lib/zstd/compress/zstd_compress_superblock.c
+++ b/lib/zstd/compress/zstd_compress_superblock.c
@@ -684,6 +684,6 @@ size_t ZSTD_compressSuperBlock(ZSTD_CCtx* zc,
&zc->appliedParams,
dst, dstCapacity,
src, srcSize,
- zc->bmi2, lastBlock,
+ ZSTD_CCtx_get_bmi2(zc), lastBlock,
zc->tmpWorkspace, zc->tmpWkspSize /* statically allocated in resetCCtx */);
}
diff --git a/lib/zstd/decompress/huf_decompress.c b/lib/zstd/decompress/huf_decompress.c
index 5663ae524cd83..8bf2279887e8f 100644
--- a/lib/zstd/decompress/huf_decompress.c
+++ b/lib/zstd/decompress/huf_decompress.c
@@ -17,6 +17,9 @@
* Dependencies
****************************************************************/
#include "../common/zstd_deps.h" /* ZSTD_memcpy, ZSTD_memset */
+#if defined(ZSTD_USE_KERNEL_CPU_FEATURES)
+#include <asm/cpufeature.h>
+#endif
#include "../common/compiler.h"
#include "../common/bitstream.h" /* BIT_* */
#include "../common/fse.h" /* to compress headers */
@@ -113,9 +116,10 @@ typedef size_t (*HUF_DecompressUsingDTableFn)(void *dst, size_t dstSize,
static size_t fn(void* dst, size_t dstSize, void const* cSrc, \
size_t cSrcSize, HUF_DTable const* DTable, int flags) \
{ \
- if (flags & HUF_flags_bmi2) { \
+ if (ZSTD_USE_BMI2(flags & HUF_flags_bmi2)) { \
return fn##_bmi2(dst, dstSize, cSrc, cSrcSize, DTable); \
} \
+ (void)flags; \
return fn##_default(dst, dstSize, cSrc, cSrcSize, DTable); \
}
@@ -899,18 +903,16 @@ static size_t HUF_decompress4X1_usingDTable_internal(void* dst, size_t dstSize,
HUF_DecompressUsingDTableFn fallbackFn = HUF_decompress4X1_usingDTable_internal_default;
HUF_DecompressFastLoopFn loopFn = HUF_decompress4X1_usingDTable_internal_fast_c_loop;
-#if DYNAMIC_BMI2
- if (flags & HUF_flags_bmi2) {
+ if (ZSTD_USE_BMI2(flags & HUF_flags_bmi2)) {
fallbackFn = HUF_decompress4X1_usingDTable_internal_bmi2;
# if ZSTD_ENABLE_ASM_X86_64_BMI2
if (!(flags & HUF_flags_disableAsm)) {
loopFn = HUF_decompress4X1_usingDTable_internal_fast_asm_loop;
}
# endif
- } else {
+ } else if (DYNAMIC_BMI2) {
return fallbackFn(dst, dstSize, cSrc, cSrcSize, DTable);
}
-#endif
#if ZSTD_ENABLE_ASM_X86_64_BMI2 && defined(__BMI2__)
if (!(flags & HUF_flags_disableAsm)) {
@@ -1723,18 +1725,16 @@ static size_t HUF_decompress4X2_usingDTable_internal(void* dst, size_t dstSize,
HUF_DecompressUsingDTableFn fallbackFn = HUF_decompress4X2_usingDTable_internal_default;
HUF_DecompressFastLoopFn loopFn = HUF_decompress4X2_usingDTable_internal_fast_c_loop;
-#if DYNAMIC_BMI2
- if (flags & HUF_flags_bmi2) {
+ if (ZSTD_USE_BMI2(flags & HUF_flags_bmi2)) {
fallbackFn = HUF_decompress4X2_usingDTable_internal_bmi2;
# if ZSTD_ENABLE_ASM_X86_64_BMI2
if (!(flags & HUF_flags_disableAsm)) {
loopFn = HUF_decompress4X2_usingDTable_internal_fast_asm_loop;
}
# endif
- } else {
+ } else if (DYNAMIC_BMI2) {
return fallbackFn(dst, dstSize, cSrc, cSrcSize, DTable);
}
-#endif
#if ZSTD_ENABLE_ASM_X86_64_BMI2 && defined(__BMI2__)
if (!(flags & HUF_flags_disableAsm)) {
diff --git a/lib/zstd/decompress/zstd_decompress.c b/lib/zstd/decompress/zstd_decompress.c
index bb009554e3a61..cc6ecd364970a 100644
--- a/lib/zstd/decompress/zstd_decompress.c
+++ b/lib/zstd/decompress/zstd_decompress.c
@@ -259,9 +259,7 @@ static void ZSTD_initDCtx_internal(ZSTD_DCtx* dctx)
dctx->noForwardProgress = 0;
dctx->oversizedDuration = 0;
dctx->isFrameDecompression = 1;
-#if DYNAMIC_BMI2
- dctx->bmi2 = ZSTD_cpuSupportsBmi2();
-#endif
+ ZSTD_SET_BMI2(dctx->bmi2, ZSTD_cpuSupportsBmi2());
dctx->ddictSet = NULL;
ZSTD_DCtx_resetParameters(dctx);
#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
diff --git a/lib/zstd/decompress/zstd_decompress_block.c b/lib/zstd/decompress/zstd_decompress_block.c
index 9c4215e435014..31804cbf1f7ed 100644
--- a/lib/zstd/decompress/zstd_decompress_block.c
+++ b/lib/zstd/decompress/zstd_decompress_block.c
@@ -16,6 +16,9 @@
* Dependencies
*********************************************************/
#include "../common/zstd_deps.h" /* ZSTD_memcpy, ZSTD_memmove, ZSTD_memset */
+#if defined(ZSTD_USE_KERNEL_CPU_FEATURES)
+#include <asm/cpufeature.h>
+#endif
#include "../common/compiler.h" /* prefetch */
#include "../common/cpu.h" /* bmi2 */
#include "../common/mem.h" /* low level memory routines */
@@ -631,13 +634,11 @@ void ZSTD_buildFSETable(ZSTD_seqSymbol* dt,
const U32* baseValue, const U8* nbAdditionalBits,
unsigned tableLog, void* wksp, size_t wkspSize, int bmi2)
{
-#if DYNAMIC_BMI2
- if (bmi2) {
+ if (ZSTD_USE_BMI2(bmi2)) {
ZSTD_buildFSETable_body_bmi2(dt, normalizedCounter, maxSymbolValue,
baseValue, nbAdditionalBits, tableLog, wksp, wkspSize);
return;
}
-#endif
(void)bmi2;
ZSTD_buildFSETable_body_default(dt, normalizedCounter, maxSymbolValue,
baseValue, nbAdditionalBits, tableLog, wksp, wkspSize);
@@ -1955,11 +1956,9 @@ ZSTD_decompressSequences(ZSTD_DCtx* dctx, void* dst, size_t maxDstSize,
const ZSTD_longOffset_e isLongOffset)
{
DEBUGLOG(5, "ZSTD_decompressSequences");
-#if DYNAMIC_BMI2
- if (ZSTD_DCtx_get_bmi2(dctx)) {
+ if (ZSTD_USE_BMI2(ZSTD_DCtx_get_bmi2(dctx))) {
return ZSTD_decompressSequences_bmi2(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset);
}
-#endif
return ZSTD_decompressSequences_default(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset);
}
static size_t
@@ -1968,11 +1967,9 @@ ZSTD_decompressSequencesSplitLitBuffer(ZSTD_DCtx* dctx, void* dst, size_t maxDst
const ZSTD_longOffset_e isLongOffset)
{
DEBUGLOG(5, "ZSTD_decompressSequencesSplitLitBuffer");
-#if DYNAMIC_BMI2
- if (ZSTD_DCtx_get_bmi2(dctx)) {
+ if (ZSTD_USE_BMI2(ZSTD_DCtx_get_bmi2(dctx))) {
return ZSTD_decompressSequencesSplitLitBuffer_bmi2(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset);
}
-#endif
return ZSTD_decompressSequencesSplitLitBuffer_default(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset);
}
#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG */
@@ -1991,11 +1988,9 @@ ZSTD_decompressSequencesLong(ZSTD_DCtx* dctx,
const ZSTD_longOffset_e isLongOffset)
{
DEBUGLOG(5, "ZSTD_decompressSequencesLong");
-#if DYNAMIC_BMI2
- if (ZSTD_DCtx_get_bmi2(dctx)) {
+ if (ZSTD_USE_BMI2(ZSTD_DCtx_get_bmi2(dctx))) {
return ZSTD_decompressSequencesLong_bmi2(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset);
}
-#endif
return ZSTD_decompressSequencesLong_default(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset);
}
#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT */
diff --git a/lib/zstd/decompress/zstd_decompress_internal.h b/lib/zstd/decompress/zstd_decompress_internal.h
index 2a225d1811c4f..401f2b526bbce 100644
--- a/lib/zstd/decompress/zstd_decompress_internal.h
+++ b/lib/zstd/decompress/zstd_decompress_internal.h
@@ -204,7 +204,7 @@ struct ZSTD_DCtx_s
}; /* typedef'd to ZSTD_DCtx within "zstd.h" */
MEM_STATIC int ZSTD_DCtx_get_bmi2(const struct ZSTD_DCtx_s *dctx) {
-#if DYNAMIC_BMI2
+#if DYNAMIC_BMI2 && !defined(ZSTD_USE_KERNEL_CPU_FEATURES)
return dctx->bmi2;
#else
(void)dctx;
--
2.53.0-Meta
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v3 2/2] zstd: use cpu_feature_enabled() for in-kernel BMI2 dispatch
2026-09-01 11:07 ` [PATCH v3 2/2] zstd: use cpu_feature_enabled() for in-kernel BMI2 dispatch Usama Arif
@ 2026-09-02 17:40 ` Dhruva G
2026-09-03 10:53 ` Usama Arif
0 siblings, 1 reply; 6+ messages in thread
From: Dhruva G @ 2026-09-02 17:40 UTC (permalink / raw)
To: Usama Arif, dsterba, linux-kernel, terrelln, terrelln,
linux-crypto, yosry, ebiggers, torvalds
Cc: hannes, nphamcs, chengming.zhou, shakeel.butt, kernel-team
Hi Usama,
On 01-09-2026 16:37, Usama Arif wrote:
> Zstd's dynamic BMI2 implementation probes CPUID when a compression or
> decompression context is initialized, stores the result in the context,
> and tests that value at every dispatch site. In normal kernel builds this
> bypasses the x86 feature policy and uses ordinary runtime branches instead
> of allowing x86 alternatives to resolve the feature check at boot.
>
> Add ZSTD_USE_BMI2() and use it at every runtime BMI2/default selector. For
> normal x86 kernel objects, the predicate expands directly to
> cpu_feature_enabled(X86_FEATURE_BMI2). When dynamic BMI2 dispatch is not
> available it is false; other builds retain the caller-provided flag. Keep
> the existing HUF conditional layout because DYNAMIC_BMI2 also controls
> whether target-attributed variants are emitted.
>
> Add the matching ZSTD_SET_BMI2() abstraction for context initialization.
> Normal x86 kernel objects and builds without dynamic BMI2 do not cache CPU
> state. Other builds with dynamic dispatch, including preboot, retain the
> existing behavior. Keep the BMI2 members in the context structures so
> their layouts do not change, and make the accessors return zero when
> cached state is unused.
>
> Select the normal-kernel policy in zstd_deps.h. Builds which define
> __DISABLE_EXPORTS, including the x86 preboot decompressor, retain the existing
> CPUID-backed dispatch because the normal alternatives infrastructure is not
> available there.
>
> A 4 KiB zstd-generic crypto_acomp benchmark in a one-vCPU KVM guest gave
> these median results:
>
> Before After Change
> Compression 16,634 ns/op 13,394 ns/op -19.5%
> Decompression 3,480 ns/op 963 ns/op -72.3%
>
> Signed-off-by: Usama Arif <usama.arif@linux.dev>
> ---
These comments still say BMI2 support is determined once per context:
- lib/zstd/compress/zstd_compress_internal.h:473
- lib/zstd/decompress/zstd_decompress_internal.h:159
> lib/zstd/common/compiler.h | 12 ++++++++++++
> lib/zstd/common/entropy_common.c | 12 ++++++------
> lib/zstd/common/fse_decompress.c | 7 ++++---
> lib/zstd/common/zstd_deps.h | 5 +++++
> lib/zstd/compress/huf_compress.c | 6 +++++-
> lib/zstd/compress/zstd_compress.c | 12 ++++++------
> lib/zstd/compress/zstd_compress_internal.h | 9 +++++++++
> lib/zstd/compress/zstd_compress_sequences.c | 8 +++++---
> lib/zstd/compress/zstd_compress_superblock.c | 2 +-
> lib/zstd/decompress/huf_decompress.c | 18 +++++++++---------
> lib/zstd/decompress/zstd_decompress.c | 4 +---
> lib/zstd/decompress/zstd_decompress_block.c | 19 +++++++------------
> .../decompress/zstd_decompress_internal.h | 2 +-
> 13 files changed, 71 insertions(+), 45 deletions(-)
>
> diff --git a/lib/zstd/common/compiler.h b/lib/zstd/common/compiler.h
> index dc9bd15e174e9..47f6c0372c58c 100644
> --- a/lib/zstd/common/compiler.h
> +++ b/lib/zstd/common/compiler.h
> @@ -14,6 +14,7 @@
>
> #include <linux/types.h>
>
> +#include "zstd_deps.h"
> #include "portability_macros.h"
>
> /*-*******************************************************
> @@ -96,6 +97,17 @@
> */
> #define BMI2_TARGET_ATTRIBUTE TARGET_ATTRIBUTE("lzcnt,bmi,bmi2")
>
> +#if !DYNAMIC_BMI2
> +# define ZSTD_USE_BMI2(bmi2) 0
> +# define ZSTD_SET_BMI2(state, value) do { } while (0)
> +#elif defined(ZSTD_USE_KERNEL_CPU_FEATURES)
> +# define ZSTD_USE_BMI2(bmi2) cpu_feature_enabled(X86_FEATURE_BMI2)
Here, we do not include <asm/cpufeature.h>. Instead, every current .c user includes that header separately.
This works today, but it maybe fragile: the next user of ZSTD_USE_BMI2() can fail to compile unless they know
about this hidden requirement.
Do you think perhaps we should provide that here in this header itself?
With that, feel free to add
Reviewed-by: Dhruva Gole <goledhruva@gmail.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v3 2/2] zstd: use cpu_feature_enabled() for in-kernel BMI2 dispatch
2026-09-02 17:40 ` Dhruva G
@ 2026-09-03 10:53 ` Usama Arif
0 siblings, 0 replies; 6+ messages in thread
From: Usama Arif @ 2026-09-03 10:53 UTC (permalink / raw)
To: Dhruva G, dsterba, linux-kernel, terrelln, terrelln, linux-crypto,
yosry, ebiggers, torvalds
Cc: hannes, nphamcs, chengming.zhou, shakeel.butt, kernel-team
On 02/09/2026 18:40, Dhruva G wrote:
> Hi Usama,
>
> On 01-09-2026 16:37, Usama Arif wrote:
>> Zstd's dynamic BMI2 implementation probes CPUID when a compression or
>> decompression context is initialized, stores the result in the context,
>> and tests that value at every dispatch site. In normal kernel builds this
>> bypasses the x86 feature policy and uses ordinary runtime branches instead
>> of allowing x86 alternatives to resolve the feature check at boot.
>>
>> Add ZSTD_USE_BMI2() and use it at every runtime BMI2/default selector. For
>> normal x86 kernel objects, the predicate expands directly to
>> cpu_feature_enabled(X86_FEATURE_BMI2). When dynamic BMI2 dispatch is not
>> available it is false; other builds retain the caller-provided flag. Keep
>> the existing HUF conditional layout because DYNAMIC_BMI2 also controls
>> whether target-attributed variants are emitted.
>>
>> Add the matching ZSTD_SET_BMI2() abstraction for context initialization.
>> Normal x86 kernel objects and builds without dynamic BMI2 do not cache CPU
>> state. Other builds with dynamic dispatch, including preboot, retain the
>> existing behavior. Keep the BMI2 members in the context structures so
>> their layouts do not change, and make the accessors return zero when
>> cached state is unused.
>>
>> Select the normal-kernel policy in zstd_deps.h. Builds which define
>> __DISABLE_EXPORTS, including the x86 preboot decompressor, retain the existing
>> CPUID-backed dispatch because the normal alternatives infrastructure is not
>> available there.
>>
>> A 4 KiB zstd-generic crypto_acomp benchmark in a one-vCPU KVM guest gave
>> these median results:
>>
>> Before After Change
>> Compression 16,634 ns/op 13,394 ns/op -19.5%
>> Decompression 3,480 ns/op 963 ns/op -72.3%
>>
>> Signed-off-by: Usama Arif <usama.arif@linux.dev>
>> ---
>
> These comments still say BMI2 support is determined once per context:
> - lib/zstd/compress/zstd_compress_internal.h:473
> - lib/zstd/decompress/zstd_decompress_internal.h:159
Good catch! Thanks. I will change to:
Cached for per-context dispatch: 1 if the CPU supports BMI2, 0 otherwise.
>
>> lib/zstd/common/compiler.h | 12 ++++++++++++
>> lib/zstd/common/entropy_common.c | 12 ++++++------
>> lib/zstd/common/fse_decompress.c | 7 ++++---
>> lib/zstd/common/zstd_deps.h | 5 +++++
>> lib/zstd/compress/huf_compress.c | 6 +++++-
>> lib/zstd/compress/zstd_compress.c | 12 ++++++------
>> lib/zstd/compress/zstd_compress_internal.h | 9 +++++++++
>> lib/zstd/compress/zstd_compress_sequences.c | 8 +++++---
>> lib/zstd/compress/zstd_compress_superblock.c | 2 +-
>> lib/zstd/decompress/huf_decompress.c | 18 +++++++++---------
>> lib/zstd/decompress/zstd_decompress.c | 4 +---
>> lib/zstd/decompress/zstd_decompress_block.c | 19 +++++++------------
>> .../decompress/zstd_decompress_internal.h | 2 +-
>> 13 files changed, 71 insertions(+), 45 deletions(-)
>>
>> diff --git a/lib/zstd/common/compiler.h b/lib/zstd/common/compiler.h
>> index dc9bd15e174e9..47f6c0372c58c 100644
>> --- a/lib/zstd/common/compiler.h
>> +++ b/lib/zstd/common/compiler.h
>> @@ -14,6 +14,7 @@
>>
>> #include <linux/types.h>
>>
>> +#include "zstd_deps.h"
>> #include "portability_macros.h"
>>
>> /*-*******************************************************
>> @@ -96,6 +97,17 @@
>> */
>> #define BMI2_TARGET_ATTRIBUTE TARGET_ATTRIBUTE("lzcnt,bmi,bmi2")
>>
>> +#if !DYNAMIC_BMI2
>> +# define ZSTD_USE_BMI2(bmi2) 0
>> +# define ZSTD_SET_BMI2(state, value) do { } while (0)
>> +#elif defined(ZSTD_USE_KERNEL_CPU_FEATURES)
>> +# define ZSTD_USE_BMI2(bmi2) cpu_feature_enabled(X86_FEATURE_BMI2)
>
> Here, we do not include <asm/cpufeature.h>. Instead, every current .c user includes that header separately.
> This works today, but it maybe fragile: the next user of ZSTD_USE_BMI2() can fail to compile unless they know
> about this hidden requirement.
> Do you think perhaps we should provide that here in this header itself?
I tried that, but compiler.h is included by unrelated zstd translation units.
On x86, <asm/cpufeature.h> eventually includes <asm/current.h>, which defines
current as get_current(). This breaks the existing local variable named current
in zstd_double_fast.c.
I think the current apporach is ok?
>
> With that, feel free to add
>
> Reviewed-by: Dhruva Gole <goledhruva@gmail.com>
Thanks for the review!
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v3 1/2] lib/zstd: add fallback aliases for disabled BMI2 variants
2026-09-01 11:07 ` [PATCH v3 1/2] lib/zstd: add fallback aliases for disabled BMI2 variants Usama Arif
@ 2026-09-03 12:35 ` Dhruva G
0 siblings, 0 replies; 6+ messages in thread
From: Dhruva G @ 2026-09-03 12:35 UTC (permalink / raw)
To: Usama Arif, dsterba, linux-kernel, terrelln, terrelln,
linux-crypto, yosry, ebiggers, torvalds
Cc: hannes, nphamcs, chengming.zhou, shakeel.butt, kernel-team
On 01-09-2026 16:37, Usama Arif wrote:
> When dynamic BMI2 dispatch is disabled, the BMI2-specific functions are
> not compiled and each selector is conditionally compiled to avoid naming
> them.
>
> The selector-level preprocessor guards will be replaced with a predicate
> that becomes constant false when dynamic BMI2 dispatch is disabled.
> Although the compiler eliminates an `if (0)` branch, it must still parse
> and resolve the BMI2 function referenced by it.
>
> Add aliases from the unavailable BMI2 function names to their default
> implementations. These aliases make the names valid without emitting
> BMI2-specific code. The selectors remain unchanged in this patch, so the
> aliases are not used yet and there is no code-generation change.
>
> Signed-off-by: Usama Arif <usama.arif@linux.dev>
> ---
Reviewed-by: Dhruva Gole <goledhruva@gmail.com>
> lib/zstd/common/entropy_common.c | 4 ++++
> lib/zstd/common/fse_decompress.c | 2 ++
> lib/zstd/compress/zstd_compress_sequences.c | 4 ++++
> lib/zstd/decompress/huf_decompress.c | 4 ++++
> lib/zstd/decompress/zstd_decompress_block.c | 12 ++++++++++++
> 5 files changed, 26 insertions(+)
>
> diff --git a/lib/zstd/common/entropy_common.c b/lib/zstd/common/entropy_common.c
> index 6cdd82233fb59..15a7c14ae8040 100644
> --- a/lib/zstd/common/entropy_common.c
> +++ b/lib/zstd/common/entropy_common.c
> @@ -202,6 +202,8 @@ BMI2_TARGET_ATTRIBUTE static size_t FSE_readNCount_body_bmi2(
> {
> return FSE_readNCount_body(normalizedCounter, maxSVPtr, tableLogPtr, headerBuffer, hbSize);
> }
> +#else
> +#define FSE_readNCount_body_bmi2 FSE_readNCount_body_default
> #endif
>
> size_t FSE_readNCount_bmi2(
> @@ -323,6 +325,8 @@ static BMI2_TARGET_ATTRIBUTE size_t HUF_readStats_body_bmi2(BYTE* huffWeight, si
> {
> return HUF_readStats_body(huffWeight, hwSize, rankStats, nbSymbolsPtr, tableLogPtr, src, srcSize, workSpace, wkspSize, 1);
> }
> +#else
> +#define HUF_readStats_body_bmi2 HUF_readStats_body_default
> #endif
>
> size_t HUF_readStats_wksp(BYTE* huffWeight, size_t hwSize, U32* rankStats,
> diff --git a/lib/zstd/common/fse_decompress.c b/lib/zstd/common/fse_decompress.c
> index 15081d8dc607c..8d9c43b928833 100644
> --- a/lib/zstd/common/fse_decompress.c
> +++ b/lib/zstd/common/fse_decompress.c
> @@ -300,6 +300,8 @@ BMI2_TARGET_ATTRIBUTE static size_t FSE_decompress_wksp_body_bmi2(void* dst, siz
> {
> return FSE_decompress_wksp_body(dst, dstCapacity, cSrc, cSrcSize, maxLog, workSpace, wkspSize, 1);
> }
> +#else
> +#define FSE_decompress_wksp_body_bmi2 FSE_decompress_wksp_body_default
> #endif
>
> size_t FSE_decompress_wksp_bmi2(void* dst, size_t dstCapacity, const void* cSrc, size_t cSrcSize, unsigned maxLog, void* workSpace, size_t wkspSize, int bmi2)
> diff --git a/lib/zstd/compress/zstd_compress_sequences.c b/lib/zstd/compress/zstd_compress_sequences.c
> index 256980c9d85ad..64abcdf2c5714 100644
> --- a/lib/zstd/compress/zstd_compress_sequences.c
> +++ b/lib/zstd/compress/zstd_compress_sequences.c
> @@ -415,6 +415,10 @@ ZSTD_encodeSequences_bmi2(
> sequences, nbSeq, longOffsets);
> }
>
> +#else
> +
> +#define ZSTD_encodeSequences_bmi2 ZSTD_encodeSequences_default
> +
> #endif
>
> size_t ZSTD_encodeSequences(
> diff --git a/lib/zstd/decompress/huf_decompress.c b/lib/zstd/decompress/huf_decompress.c
> index ac8b87f48f847..5663ae524cd83 100644
> --- a/lib/zstd/decompress/huf_decompress.c
> +++ b/lib/zstd/decompress/huf_decompress.c
> @@ -700,6 +700,8 @@ size_t HUF_decompress4X1_usingDTable_internal_bmi2(void* dst, size_t dstSize, vo
> size_t cSrcSize, HUF_DTable const* DTable) {
> return HUF_decompress4X1_usingDTable_internal_body(dst, dstSize, cSrc, cSrcSize, DTable);
> }
> +#else
> +#define HUF_decompress4X1_usingDTable_internal_bmi2 HUF_decompress4X1_usingDTable_internal_default
> #endif
>
> static
> @@ -1503,6 +1505,8 @@ size_t HUF_decompress4X2_usingDTable_internal_bmi2(void* dst, size_t dstSize, vo
> size_t cSrcSize, HUF_DTable const* DTable) {
> return HUF_decompress4X2_usingDTable_internal_body(dst, dstSize, cSrc, cSrcSize, DTable);
> }
> +#else
> +#define HUF_decompress4X2_usingDTable_internal_bmi2 HUF_decompress4X2_usingDTable_internal_default
> #endif
>
> static
> diff --git a/lib/zstd/decompress/zstd_decompress_block.c b/lib/zstd/decompress/zstd_decompress_block.c
> index 710eb0ffd5a37..9c4215e435014 100644
> --- a/lib/zstd/decompress/zstd_decompress_block.c
> +++ b/lib/zstd/decompress/zstd_decompress_block.c
> @@ -622,6 +622,8 @@ BMI2_TARGET_ATTRIBUTE static void ZSTD_buildFSETable_body_bmi2(ZSTD_seqSymbol* d
> ZSTD_buildFSETable_body(dt, normalizedCounter, maxSymbolValue,
> baseValue, nbAdditionalBits, tableLog, wksp, wkspSize);
> }
> +#else
> +#define ZSTD_buildFSETable_body_bmi2 ZSTD_buildFSETable_body_default
> #endif
>
> void ZSTD_buildFSETable(ZSTD_seqSymbol* dt,
> @@ -1934,6 +1936,16 @@ ZSTD_decompressSequencesLong_bmi2(ZSTD_DCtx* dctx,
> }
> #endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT */
>
> +#else
> +
> +#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG
> +#define ZSTD_decompressSequences_bmi2 ZSTD_decompressSequences_default
> +#define ZSTD_decompressSequencesSplitLitBuffer_bmi2 ZSTD_decompressSequencesSplitLitBuffer_default
> +#endif
> +#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT
> +#define ZSTD_decompressSequencesLong_bmi2 ZSTD_decompressSequencesLong_default
> +#endif
> +
> #endif /* DYNAMIC_BMI2 */
>
> #ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-09-03 12:35 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-01 11:07 [PATCH v3 0/2] zstd: use x86 feature infrastructure for BMI2 dispatch Usama Arif
2026-09-01 11:07 ` [PATCH v3 1/2] lib/zstd: add fallback aliases for disabled BMI2 variants Usama Arif
2026-09-03 12:35 ` Dhruva G
2026-09-01 11:07 ` [PATCH v3 2/2] zstd: use cpu_feature_enabled() for in-kernel BMI2 dispatch Usama Arif
2026-09-02 17:40 ` Dhruva G
2026-09-03 10:53 ` Usama Arif
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).