* [PATCH bpf-next 1/8] bpf: Add UHMUL and SHMUL instructions
2026-09-17 4:17 [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions Yazhou Tang
@ 2026-09-17 4:17 ` Yazhou Tang
2026-09-17 5:22 ` bot+bpf-ci
2026-09-17 17:04 ` Alexei Starovoitov
2026-09-17 4:17 ` [PATCH bpf-next 2/8] bpf, x86: JIT UHMUL and SHMUL on x86-64 Yazhou Tang
` (7 subsequent siblings)
8 siblings, 2 replies; 15+ messages in thread
From: Yazhou Tang @ 2026-09-17 4:17 UTC (permalink / raw)
To: bpf, ast, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye, Alexei Starovoitov
From: Tianci Cao <ziye@zju.edu.cn>
Define UHMUL and SHMUL as BPF_ALU64 BPF_MUL variants selected by insn->off.
UHMUL returns the high 64 bits of an unsigned 64-by-64-bit product, while
SHMUL returns the high 64 bits of a signed product. Keep off == 0 as the
existing low-half multiplication, and assign off == 1 and off == 2 to
UHMUL and SHMUL, respectively.
These operations are inspired by the UHMUL and SHMUL instructions introduced
as part of the PQR instruction class in sBPF sBPF (Solana's BPF variant)[1].
The encoding and immediate semantics differ from current sBPF.
1. sBPF assigns these operations dedicated opcodes in the BPF_PQR class,
whereas this implementation selects them through the offset field of BPF_MUL.
2. sBPF initially sign-extended the immediate operand of UHMUL, but later
changed unsigned PQR immediates, including UHMUL, to be zero-extended[2].
This was an ISA design choice to treat immediates of explicitly unsigned
operations as unsigned 32-bit values[3].
Linux instead follows the existing BPF_ALU64 immediate convention.
Both immediate forms first sign-extend the 32-bit immediate to 64 bits.
UHMUL then interprets the resulting bit pattern as unsigned, while
SHMUL interprets it as signed.
Add portable helpers for both operations, using mul_u64_u64_shr() and a
signed correction instead of relying on a compiler-provided 128-bit
integer type. Wire the variants into the interpreter and disassembler,
and document the encoding in the UAPI and BPF instruction set. Keep
BPF_ALU multiplication restricted to the existing low-half variant.
[1] https://github.com/solana-labs/rbpf/pull/498
[2] https://github.com/solana-labs/rbpf/pull/642
[3] https://lore.kernel.org/bpf/59e754a2-6158-485c-93b9-6f60d9f57528@zju.edu.cn/
Link: https://lore.kernel.org/bpf/CAADnVQ+2O2zgdy11+vxE6dOp+3Br4zZBysHF-Hne7cP+eeucnQ@mail.gmail.com/
Suggested-by: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Signed-off-by: Tianci Cao <ziye@zju.edu.cn>
Co-developed-by: Yazhou Tang <tangyazhou518@outlook.com>
Signed-off-by: Yazhou Tang <tangyazhou518@outlook.com>
Co-developed-by: Shenghao Yuan <shenghaoyuan0928@163.com>
Signed-off-by: Shenghao Yuan <shenghaoyuan0928@163.com>
---
.../bpf/standardization/instruction-set.rst | 26 ++++++++++-
include/linux/filter.h | 39 ++++++++++++++++
include/uapi/linux/bpf.h | 10 +++++
kernel/bpf/core.c | 45 ++++++++++++++++++-
kernel/bpf/disasm.c | 18 ++++++++
tools/include/uapi/linux/bpf.h | 10 +++++
6 files changed, 145 insertions(+), 3 deletions(-)
diff --git a/Documentation/bpf/standardization/instruction-set.rst b/Documentation/bpf/standardization/instruction-set.rst
index 39c74611752b..3c2cde11fa23 100644
--- a/Documentation/bpf/standardization/instruction-set.rst
+++ b/Documentation/bpf/standardization/instruction-set.rst
@@ -330,6 +330,8 @@ register.
ADD 0x0 0 dst += src
SUB 0x1 0 dst -= src
MUL 0x2 0 dst \*= src
+ UHMUL 0x2 1 unsigned high-half multiplication (``ALU64`` only)
+ SHMUL 0x2 2 signed high-half multiplication (``ALU64`` only)
DIV 0x3 0 dst = (src != 0) ? (dst / src) : 0
SDIV 0x3 1 dst = (src == 0) ? 0 : ((src == -1 && dst == LLONG_MIN) ? LLONG_MIN : (dst s/ src))
OR 0x4 0 dst \|= src
@@ -379,13 +381,33 @@ where '(u32)' indicates that the upper 32 bits are zeroed.
dst = dst ^ imm
-Note that most arithmetic instructions have 'offset' set to 0. Only three instructions
-(``SDIV``, ``SMOD``, ``MOVSX``) have a non-zero 'offset'.
+Note that most arithmetic instructions have 'offset' set to 0. The following
+instructions may have a non-zero 'offset':
+
+* ``SDIV`` and ``SMOD``: ``offset == 1`` selects the signed variant.
+* ``MOVSX``: ``offset`` selects the source operand width.
+* ``MUL``: ``offset`` selects the multiplication variant (see below).
Division, multiplication, and modulo operations for ``ALU`` are part
of the "divmul32" conformance group, and division, multiplication, and
modulo operations for ``ALU64`` are part of the "divmul64" conformance
group.
+
+For ``ALU64`` multiplication, ``offset == 1`` selects unsigned
+high-half multiplication (``UHMUL``), and ``offset == 2`` selects
+signed high-half multiplication (``SHMUL``).
+
+For ``X`` mode, ``src64`` is the 64-bit value in ``src_reg``. For
+``K`` mode, ``src64`` is ``imm`` sign-extended from 32 to 64 bits.
+The operations are::
+
+ UHMUL: dst = (u64)(((u128)dst * (u128)(u64)src64) >> 64)
+ SHMUL: dst = (u64)(((s128)(s64)dst * (s128)(s64)src64) >> 64)
+
+For ``ALU`` multiplication, only ``offset == 0`` is valid. For
+``ALU64`` multiplication, offset values other than 0, 1, and 2 are
+reserved.
+
The division and modulo operations support both unsigned and signed flavors.
For unsigned operations (``DIV`` and ``MOD``), for ``ALU``,
diff --git a/include/linux/filter.h b/include/linux/filter.h
index b17222db2efc..b6dc4b968a9c 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -23,6 +23,7 @@
#include <linux/sockptr.h>
#include <linux/static_call.h>
#include <linux/u64_stats_sync.h>
+#include <linux/math64.h>
#include <net/sch_generic.h>
@@ -147,6 +148,44 @@ struct ctl_table_header;
#define BPF_ALU32_IMM(OP, DST, IMM) \
BPF_ALU32_IMM_OFF(OP, DST, IMM, 0)
+static inline u64 bpf_uhmul64(u64 a, u64 b)
+{
+ return mul_u64_u64_shr(a, b, 64);
+}
+
+static inline u64 bpf_shmul64(s64 a, s64 b)
+{
+ u64 ua = a, ub = b;
+ u64 hi = bpf_uhmul64(ua, ub);
+
+ /*
+ * Let M = 2^64, sa = (a < 0), and sb = (b < 0). Then:
+ *
+ * a * b = (ua - sa * M) * (ub - sb * M)
+ * = ua * ub - sa * M * ub - sb * M * ua + sa * sb * M^2
+ *
+ * Therefore, signed_hi = unsigned_hi - sa * ub - sb * ua (mod M).
+ */
+ if (a < 0)
+ hi -= ub;
+ if (b < 0)
+ hi -= ua;
+ return hi;
+}
+
+static inline bool bpf_is_hmul_variant(s16 off)
+{
+ return off == BPF_MUL_VARIANT_UHMUL ||
+ off == BPF_MUL_VARIANT_SHMUL;
+}
+
+static inline bool bpf_insn_is_hmul(const struct bpf_insn *insn)
+{
+ return BPF_CLASS(insn->code) == BPF_ALU64 &&
+ BPF_OP(insn->code) == BPF_MUL &&
+ bpf_is_hmul_variant(insn->off);
+}
+
/* Endianess conversion, cpu_to_{l,b}e(), {l,b}e_to_cpu() */
#define BPF_ENDIAN(TYPE, DST, LEN) \
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 732b35cc08d1..845eee536198 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -34,6 +34,16 @@
#define BPF_FROM_LE BPF_TO_LE
#define BPF_FROM_BE BPF_TO_BE
+/*
+ * BPF_MUL variants selected by the off field.
+ * UHMUL and SHMUL are only valid for BPF_ALU64.
+ */
+enum bpf_mul_variant {
+ BPF_MUL_VARIANT_LO = 0,
+ BPF_MUL_VARIANT_UHMUL = 1,
+ BPF_MUL_VARIANT_SHMUL = 2,
+};
+
/* jmp encodings */
#define BPF_JNE 0x50 /* jump != */
#define BPF_JLT 0xa0 /* LT is unsigned, '<' */
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 4e208cc94752..4f746973481b 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1931,7 +1931,50 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
ALU(AND, &)
ALU(OR, |)
ALU(XOR, ^)
- ALU(MUL, *)
+ /*
+ * MUL needs explicit handlers because ALU64 insn->off selects
+ * low-half, unsigned high-half, or signed high-half multiplication.
+ */
+ ALU64_MUL_X:
+ switch (OFF) {
+ case BPF_MUL_VARIANT_LO:
+ DST = DST * SRC;
+ break;
+ case BPF_MUL_VARIANT_UHMUL:
+ DST = bpf_uhmul64(DST, SRC);
+ break;
+ case BPF_MUL_VARIANT_SHMUL:
+ DST = bpf_shmul64((s64)DST, (s64)SRC);
+ break;
+ default:
+ goto default_label;
+ }
+ CONT;
+ ALU_MUL_X:
+ if (OFF != BPF_MUL_VARIANT_LO)
+ goto default_label;
+ DST = (u32) DST * (u32) SRC;
+ CONT;
+ ALU64_MUL_K:
+ switch (OFF) {
+ case BPF_MUL_VARIANT_LO:
+ DST = DST * IMM;
+ break;
+ case BPF_MUL_VARIANT_UHMUL:
+ DST = bpf_uhmul64(DST, (u64)IMM);
+ break;
+ case BPF_MUL_VARIANT_SHMUL:
+ DST = bpf_shmul64((s64)DST, (s32)IMM);
+ break;
+ default:
+ goto default_label;
+ }
+ CONT;
+ ALU_MUL_K:
+ if (OFF != BPF_MUL_VARIANT_LO)
+ goto default_label;
+ DST = (u32) DST * (u32) IMM;
+ CONT;
SHT(LSH, <<)
SHT(RSH, >>)
#undef SHT
diff --git a/kernel/bpf/disasm.c b/kernel/bpf/disasm.c
index 3ce8d74b0e40..03f22845e169 100644
--- a/kernel/bpf/disasm.c
+++ b/kernel/bpf/disasm.c
@@ -163,6 +163,22 @@ static bool is_sdiv_smod(const struct bpf_insn *insn)
insn->off == 1;
}
+/* Return the UHMUL or SHMUL operator string, or NULL for all other encodings. */
+static const char *hmul_op_string(const struct bpf_insn *insn)
+{
+ if (BPF_CLASS(insn->code) != BPF_ALU64 ||
+ BPF_OP(insn->code) != BPF_MUL)
+ return NULL;
+ switch (insn->off) {
+ case BPF_MUL_VARIANT_UHMUL:
+ return "uh*=";
+ case BPF_MUL_VARIANT_SHMUL:
+ return "sh*=";
+ default:
+ return NULL;
+ }
+}
+
static bool is_movsx(const struct bpf_insn *insn)
{
return BPF_OP(insn->code) == BPF_MOV &&
@@ -215,6 +231,7 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs,
verbose(cbs->private_data, "(%02x) %c%d %s %s%c%d",
insn->code, class == BPF_ALU ? 'w' : 'r',
insn->dst_reg,
+ hmul_op_string(insn) ?:
is_sdiv_smod(insn) ? bpf_alu_sign_string[BPF_OP(insn->code) >> 4]
: bpf_alu_string[BPF_OP(insn->code) >> 4],
is_movsx(insn) ? bpf_movsx_string[(insn->off >> 3) - 1] : "",
@@ -224,6 +241,7 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs,
verbose(cbs->private_data, "(%02x) %c%d %s %d",
insn->code, class == BPF_ALU ? 'w' : 'r',
insn->dst_reg,
+ hmul_op_string(insn) ?:
is_sdiv_smod(insn) ? bpf_alu_sign_string[BPF_OP(insn->code) >> 4]
: bpf_alu_string[BPF_OP(insn->code) >> 4],
insn->imm);
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 732b35cc08d1..845eee536198 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -34,6 +34,16 @@
#define BPF_FROM_LE BPF_TO_LE
#define BPF_FROM_BE BPF_TO_BE
+/*
+ * BPF_MUL variants selected by the off field.
+ * UHMUL and SHMUL are only valid for BPF_ALU64.
+ */
+enum bpf_mul_variant {
+ BPF_MUL_VARIANT_LO = 0,
+ BPF_MUL_VARIANT_UHMUL = 1,
+ BPF_MUL_VARIANT_SHMUL = 2,
+};
+
/* jmp encodings */
#define BPF_JNE 0x50 /* jump != */
#define BPF_JLT 0xa0 /* LT is unsigned, '<' */
--
2.55.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH bpf-next 1/8] bpf: Add UHMUL and SHMUL instructions
2026-09-17 4:17 ` [PATCH bpf-next 1/8] " Yazhou Tang
@ 2026-09-17 5:22 ` bot+bpf-ci
2026-09-17 6:43 ` Yazhou Tang
2026-09-17 17:04 ` Alexei Starovoitov
1 sibling, 1 reply; 15+ messages in thread
From: bot+bpf-ci @ 2026-09-17 5:22 UTC (permalink / raw)
To: tangyazhou, bpf, ast, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye, alexei.starovoitov, ast, andrii, daniel, martin.lau,
eddyz87, yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 4171 bytes --]
> These operations are inspired by the UHMUL and SHMUL instructions introduced
> as part of the PQR instruction class in sBPF sBPF (Solana's BPF variant)[1].
Could you drop the duplicated "sBPF" here? It reads as "... in sBPF sBPF
(Solana's BPF variant)" which appears to be a copy-paste error.
> diff --git a/Documentation/bpf/standardization/instruction-set.rst b/Documentation/bpf/standardization/instruction-set.rst
> index 39c74611752b6..3c2cde11fa234 100644
> --- a/Documentation/bpf/standardization/instruction-set.rst
> +++ b/Documentation/bpf/standardization/instruction-set.rst
> @@ -330,6 +330,8 @@ register.
> ADD 0x0 0 dst += src
> SUB 0x1 0 dst -= src
> MUL 0x2 0 dst \*= src
> + UHMUL 0x2 1 unsigned high-half multiplication (``ALU64`` only)
> + SHMUL 0x2 2 signed high-half multiplication (``ALU64`` only)
> DIV 0x3 0 dst = (src != 0) ? (dst / src) : 0
[ ... ]
> @@ -379,13 +381,33 @@ where '(u32)' indicates that the upper 32 bits are zeroed.
>
> dst = dst ^ imm
>
> -Note that most arithmetic instructions have 'offset' set to 0. Only three instructions
> -(``SDIV``, ``SMOD``, ``MOVSX``) have a non-zero 'offset'.
> +Note that most arithmetic instructions have 'offset' set to 0. The following
> +instructions may have a non-zero 'offset':
> +
> +* ``SDIV`` and ``SMOD``: ``offset == 1`` selects the signed variant.
> +* ``MOVSX``: ``offset`` selects the source operand width.
> +* ``MUL``: ``offset`` selects the multiplication variant (see below).
>
> Division, multiplication, and modulo operations for ``ALU`` are part
> of the "divmul32" conformance group, and division, multiplication, and
> modulo operations for ``ALU64`` are part of the "divmul64" conformance
> group.
> +
> +For ``ALU64`` multiplication, ``offset == 1`` selects unsigned
> +high-half multiplication (``UHMUL``), and ``offset == 2`` selects
> +signed high-half multiplication (``SHMUL``).
Does this addition silently make UHMUL and SHMUL mandatory members of the
existing divmul64 conformance group? The document's own rules at lines
123-125 state that "supporting a conformance group means it MUST support
all instructions in that conformance group", and the unchanged sentence
above says "division, multiplication, and modulo operations for ALU64 are
part of the divmul64 conformance group."
UHMUL and SHMUL are ALU64 multiplication operations, so by that classifying
sentence they become members of divmul64. Because group membership is
defined as MUST-support, every runtime that currently advertises divmul64
conformance (rbpf, uBPF, the Windows eBPF runtime, and any other
implementation tracking RFC 9669) would become non-conformant the moment
this text lands, even though their code has not changed.
The classification is also internally ambiguous. The document states at
line 320 that "ALU64 instructions belong to the base64 conformance group
unless noted otherwise", at line 136 that "base32 includes all instructions
defined in this specification unless otherwise noted", yet the two new rows
carry no group note at all - only a parenthetical "(ALU64 only)". A reader
can arrive at base32, base64, or divmul64 for the same two instructions.
The expected pattern for new optional instructions in this document is a
new group with its own label (compare "atomic32"/"atomic64" and
"divmul32"/"divmul64", each introduced as its own bullet at lines 141-147).
The patch adds neither a new bullet nor a note on the new rows, and the
changelog does not mention conformance groups at all.
Shouldn't UHMUL and SHMUL either be assigned to a new conformance group
(for example, "hmul64"), or carry an explicit note that they are not part
of divmul64? The existing conformance framework was designed for capability
discovery between runtimes and tool chains - growing an existing group
without notice breaks exactly that contract.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/35182532243
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH bpf-next 1/8] bpf: Add UHMUL and SHMUL instructions
2026-09-17 5:22 ` bot+bpf-ci
@ 2026-09-17 6:43 ` Yazhou Tang
0 siblings, 0 replies; 15+ messages in thread
From: Yazhou Tang @ 2026-09-17 6:43 UTC (permalink / raw)
To: bot+bpf-ci, bpf, ast, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye, alexei.starovoitov, martin.lau, mason, ihor.solodrai
On 9/17/26 1:22 PM, bot+bpf-ci@kernel.org wrote:
>> These operations are inspired by the UHMUL and SHMUL instructions introduced
>> as part of the PQR instruction class in sBPF sBPF (Solana's BPF variant)[1].
>
> Could you drop the duplicated "sBPF" here? It reads as "... in sBPF sBPF
> (Solana's BPF variant)" which appears to be a copy-paste error.
Sorry, this is a typo. I'll fix it in the next version.
>
>> diff --git a/Documentation/bpf/standardization/instruction-set.rst b/Documentation/bpf/standardization/instruction-set.rst
>> index 39c74611752b6..3c2cde11fa234 100644
>> --- a/Documentation/bpf/standardization/instruction-set.rst
>> +++ b/Documentation/bpf/standardization/instruction-set.rst
>> @@ -330,6 +330,8 @@ register.
>> ADD 0x0 0 dst += src
>> SUB 0x1 0 dst -= src
>> MUL 0x2 0 dst \*= src
>> + UHMUL 0x2 1 unsigned high-half multiplication (``ALU64`` only)
>> + SHMUL 0x2 2 signed high-half multiplication (``ALU64`` only)
>> DIV 0x3 0 dst = (src != 0) ? (dst / src) : 0
>
> [ ... ]
>
>> @@ -379,13 +381,33 @@ where '(u32)' indicates that the upper 32 bits are zeroed.
>>
>> dst = dst ^ imm
>>
>> -Note that most arithmetic instructions have 'offset' set to 0. Only three instructions
>> -(``SDIV``, ``SMOD``, ``MOVSX``) have a non-zero 'offset'.
>> +Note that most arithmetic instructions have 'offset' set to 0. The following
>> +instructions may have a non-zero 'offset':
>> +
>> +* ``SDIV`` and ``SMOD``: ``offset == 1`` selects the signed variant.
>> +* ``MOVSX``: ``offset`` selects the source operand width.
>> +* ``MUL``: ``offset`` selects the multiplication variant (see below).
>>
>> Division, multiplication, and modulo operations for ``ALU`` are part
>> of the "divmul32" conformance group, and division, multiplication, and
>> modulo operations for ``ALU64`` are part of the "divmul64" conformance
>> group.
>> +
>> +For ``ALU64`` multiplication, ``offset == 1`` selects unsigned
>> +high-half multiplication (``UHMUL``), and ``offset == 2`` selects
>> +signed high-half multiplication (``SHMUL``).
>
> Does this addition silently make UHMUL and SHMUL mandatory members of the
> existing divmul64 conformance group? The document's own rules at lines
> 123-125 state that "supporting a conformance group means it MUST support
> all instructions in that conformance group", and the unchanged sentence
> above says "division, multiplication, and modulo operations for ALU64 are
> part of the divmul64 conformance group."
>
> UHMUL and SHMUL are ALU64 multiplication operations, so by that classifying
> sentence they become members of divmul64. Because group membership is
> defined as MUST-support, every runtime that currently advertises divmul64
> conformance (rbpf, uBPF, the Windows eBPF runtime, and any other
> implementation tracking RFC 9669) would become non-conformant the moment
> this text lands, even though their code has not changed.
>
> The classification is also internally ambiguous. The document states at
> line 320 that "ALU64 instructions belong to the base64 conformance group
> unless noted otherwise", at line 136 that "base32 includes all instructions
> defined in this specification unless otherwise noted", yet the two new rows
> carry no group note at all - only a parenthetical "(ALU64 only)". A reader
> can arrive at base32, base64, or divmul64 for the same two instructions.
>
> The expected pattern for new optional instructions in this document is a
> new group with its own label (compare "atomic32"/"atomic64" and
> "divmul32"/"divmul64", each introduced as its own bullet at lines 141-147).
> The patch adds neither a new bullet nor a note on the new rows, and the
> changelog does not mention conformance groups at all.
>
> Shouldn't UHMUL and SHMUL either be assigned to a new conformance group
> (for example, "hmul64"), or carry an explicit note that they are not part
> of divmul64? The existing conformance framework was designed for capability
> discovery between runtimes and tool chains - growing an existing group
> without notice breaks exactly that contract.
I agree that silently extending an existing conformance group could be
problematic.
My initial thought was that UHMUL and SHMUL naturally belong to "divmul64",
since they are 64-bit multiplication operations. However, I just realized
that RFC 9669 also says that new instructions should not be added to an
already registered conformance group.
I'm not very familiar with the conformance-group conventions here, so I'd
appreciate suggestions from people who are more familiar with this part of
the specification. Would a separate group such as "hmul64", or a new group
that extends "divmul64", be more appropriate here?
>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/35182532243
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH bpf-next 1/8] bpf: Add UHMUL and SHMUL instructions
2026-09-17 4:17 ` [PATCH bpf-next 1/8] " Yazhou Tang
2026-09-17 5:22 ` bot+bpf-ci
@ 2026-09-17 17:04 ` Alexei Starovoitov
2026-09-18 9:26 ` Yazhou Tang
1 sibling, 1 reply; 15+ messages in thread
From: Alexei Starovoitov @ 2026-09-17 17:04 UTC (permalink / raw)
To: Yazhou Tang, bpf, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye
On Thu, Sep 17, 2026 at 12:17 PM Yazhou Tang <tangyazhou@zju.edu.cn> wrote:
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index 732b35cc08d1..845eee536198 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -34,6 +34,16 @@
> #define BPF_FROM_LE BPF_TO_LE
> #define BPF_FROM_BE BPF_TO_BE
>
> +/*
> + * BPF_MUL variants selected by the off field.
> + * UHMUL and SHMUL are only valid for BPF_ALU64.
> + */
> +enum bpf_mul_variant {
> + BPF_MUL_VARIANT_LO = 0,
> + BPF_MUL_VARIANT_UHMUL = 1,
> + BPF_MUL_VARIANT_SHMUL = 2,
> +};
Slept on it and imo it's too verbose for uapi.
Maybe just BPF_UHMUL = 1, BPF_SHMUL = 2 ?
No need to name zero, sdiv doesn't.
> diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
> index 4e208cc94752..4f746973481b 100644
> --- a/kernel/bpf/core.c
> +++ b/kernel/bpf/core.c
[...]
> + ALU_MUL_X:
> + if (OFF != BPF_MUL_VARIANT_LO)
> + goto default_label;
> + DST = (u32) DST * (u32)
SRC;
> + CONT;
The verifier already rejected these. Interpreter doesn't
need to re-validate.
pw-bot: cr
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH bpf-next 1/8] bpf: Add UHMUL and SHMUL instructions
2026-09-17 17:04 ` Alexei Starovoitov
@ 2026-09-18 9:26 ` Yazhou Tang
0 siblings, 0 replies; 15+ messages in thread
From: Yazhou Tang @ 2026-09-18 9:26 UTC (permalink / raw)
To: Alexei Starovoitov, bpf, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye
On 9/18/26 01:04, Alexei Starovoitov wrote:
> On Thu, Sep 17, 2026 at 12:17 PM Yazhou Tang <tangyazhou@zju.edu.cn> wrote:
>> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
>> index 732b35cc08d1..845eee536198 100644
>> --- a/include/uapi/linux/bpf.h
>> +++ b/include/uapi/linux/bpf.h
>> @@ -34,6 +34,16 @@
>> #define BPF_FROM_LE BPF_TO_LE
>> #define BPF_FROM_BE BPF_TO_BE
>>
>> +/*
>> + * BPF_MUL variants selected by the off field.
>> + * UHMUL and SHMUL are only valid for BPF_ALU64.
>> + */
>> +enum bpf_mul_variant {
>> + BPF_MUL_VARIANT_LO = 0,
>> + BPF_MUL_VARIANT_UHMUL = 1,
>> + BPF_MUL_VARIANT_SHMUL = 2,
>> +};
>
> Slept on it and imo it's too verbose for uapi.
> Maybe just BPF_UHMUL = 1, BPF_SHMUL = 2 ?
> No need to name zero, sdiv doesn't.
>
>> diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
>> index 4e208cc94752..4f746973481b 100644
>> --- a/kernel/bpf/core.c
>> +++ b/kernel/bpf/core.c
>
> [...]
>
>> + ALU_MUL_X:
>> + if (OFF != BPF_MUL_VARIANT_LO)
>> + goto default_label;
>> + DST = (u32) DST * (u32)
> SRC;
>> + CONT;
>
> The verifier already rejected these. Interpreter doesn't
> need to re-validate.
>
> pw-bot: cr
Thanks, agreed on all three points. I'll simplify the UAPI
definitions, fix the SRC typo, and drop the redundant
interpreter check in the next revision.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH bpf-next 2/8] bpf, x86: JIT UHMUL and SHMUL on x86-64
2026-09-17 4:17 [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions Yazhou Tang
2026-09-17 4:17 ` [PATCH bpf-next 1/8] " Yazhou Tang
@ 2026-09-17 4:17 ` Yazhou Tang
2026-09-17 4:17 ` [PATCH bpf-next 3/8] bpf, arm64: JIT UHMUL and SHMUL Yazhou Tang
` (6 subsequent siblings)
8 siblings, 0 replies; 15+ messages in thread
From: Yazhou Tang @ 2026-09-17 4:17 UTC (permalink / raw)
To: bpf, ast, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye, Alexei Starovoitov
From: Tianci Cao <ziye@zju.edu.cn>
JIT the UHMUL and SHMUL instructions with the one-operand x86-64 mul and
imul instructions. These instructions multiply RAX by an explicit
source and return the full product in RDX:RAX, so the desired high half
is available in RDX.
Copy register or immediate sources into AUX_REG, load the destination
into RAX, and move RDX back to the BPF destination after multiplication.
Preserve RAX and RDX when they hold other live BPF registers, while
handling BPF_REG_0 and BPF_REG_3 destinations without restoring over the
result.
Leave the existing JIT handling of ordinary BPF_MUL unchanged.
Link: https://lore.kernel.org/bpf/CAADnVQ+2O2zgdy11+vxE6dOp+3Br4zZBysHF-Hne7cP+eeucnQ@mail.gmail.com/
Suggested-by: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Signed-off-by: Tianci Cao <ziye@zju.edu.cn>
Co-developed-by: Yazhou Tang <tangyazhou518@outlook.com>
Signed-off-by: Yazhou Tang <tangyazhou518@outlook.com>
Co-developed-by: Shenghao Yuan <shenghaoyuan0928@163.com>
Signed-off-by: Shenghao Yuan <shenghaoyuan0928@163.com>
---
arch/x86/net/bpf_jit_comp.c | 48 +++++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index d4a980140b48..645d0ac4db04 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -1225,6 +1225,38 @@ static int emit_kasan_check(struct bpf_verifier_env *env, u8 **pprog,
return 0;
}
+/*
+ * The one-operand x86 mul/imul forms use RAX as the implicit first
+ * operand and store the 128-bit product in RDX:RAX.
+ * The JIT uses AUX_REG as the explicit second operand.
+ */
+static void emit_hmul_aux(u8 **pprog, u32 dst_reg, bool is_signed)
+{
+ u8 *prog = *pprog;
+ bool save_rax = dst_reg != BPF_REG_0;
+ bool save_rdx = dst_reg != BPF_REG_3;
+ u8 mul_modrm_base = is_signed ? 0xE8 : 0xE0; /* imul /5 or mul /4 */
+
+ if (save_rax)
+ EMIT1(0x50); /* push rax */
+ if (save_rdx)
+ EMIT1(0x52); /* push rdx */
+
+ if (save_rax)
+ emit_mov_reg(&prog, true, BPF_REG_0, dst_reg);
+ maybe_emit_1mod(&prog, AUX_REG, true);
+ EMIT2(0xF7, add_1reg(mul_modrm_base, AUX_REG));
+ if (save_rdx)
+ emit_mov_reg(&prog, true, dst_reg, BPF_REG_3);
+
+ if (save_rdx)
+ EMIT1(0x5A); /* pop rdx */
+ if (save_rax)
+ EMIT1(0x58); /* pop rax */
+
+ *pprog = prog;
+}
+
/* LDX: dst_reg = *(u8*)(src_reg + off) */
static void emit_ldx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
{
@@ -2332,6 +2364,14 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
case BPF_ALU | BPF_MUL | BPF_K:
case BPF_ALU64 | BPF_MUL | BPF_K:
+ if (bpf_insn_is_hmul(insn)) {
+ /* imul/mul use AUX_REG as src. */
+ emit_mov_imm32(&prog, true, AUX_REG, imm32);
+ emit_hmul_aux(&prog, dst_reg,
+ insn->off == BPF_MUL_VARIANT_SHMUL);
+ break;
+ }
+
maybe_emit_mod(&prog, dst_reg, dst_reg,
BPF_CLASS(insn->code) == BPF_ALU64);
@@ -2348,6 +2388,14 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
case BPF_ALU | BPF_MUL | BPF_X:
case BPF_ALU64 | BPF_MUL | BPF_X:
+ if (bpf_insn_is_hmul(insn)) {
+ /* imul/mul use AUX_REG as src. */
+ emit_mov_reg(&prog, true, AUX_REG, src_reg);
+ emit_hmul_aux(&prog, dst_reg,
+ insn->off == BPF_MUL_VARIANT_SHMUL);
+ break;
+ }
+
maybe_emit_mod(&prog, src_reg, dst_reg,
BPF_CLASS(insn->code) == BPF_ALU64);
--
2.55.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH bpf-next 3/8] bpf, arm64: JIT UHMUL and SHMUL
2026-09-17 4:17 [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions Yazhou Tang
2026-09-17 4:17 ` [PATCH bpf-next 1/8] " Yazhou Tang
2026-09-17 4:17 ` [PATCH bpf-next 2/8] bpf, x86: JIT UHMUL and SHMUL on x86-64 Yazhou Tang
@ 2026-09-17 4:17 ` Yazhou Tang
2026-09-17 4:17 ` [PATCH bpf-next 4/8] bpf, riscv: JIT UHMUL and SHMUL on RV64 Yazhou Tang
` (5 subsequent siblings)
8 siblings, 0 replies; 15+ messages in thread
From: Yazhou Tang @ 2026-09-17 4:17 UTC (permalink / raw)
To: bpf, ast, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye, Alexei Starovoitov
From: Yazhou Tang <tangyazhou518@outlook.com>
arm64 provides UMULH and SMULH instructions that return the high 64 bits
of an unsigned or signed 64-bit multiplication, respectively.
Add encodings for these instructions and use them to lower the UHMUL and
SHMUL BPF_MUL variants for both register and immediate sources. Keep the
existing low-half and ALU32 multiplication lowering unchanged.
Suggested-by: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Signed-off-by: Yazhou Tang <tangyazhou518@outlook.com>
Co-developed-by: Tianci Cao <ziye@zju.edu.cn>
Signed-off-by: Tianci Cao <ziye@zju.edu.cn>
Co-developed-by: Shenghao Yuan <shenghaoyuan0928@163.com>
Signed-off-by: Shenghao Yuan <shenghaoyuan0928@163.com>
---
arch/arm64/include/asm/insn.h | 4 ++++
arch/arm64/lib/insn.c | 6 ++++++
arch/arm64/net/bpf_jit.h | 6 ++++++
arch/arm64/net/bpf_jit_comp.c | 15 +++++++++++++--
4 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/insn.h b/arch/arm64/include/asm/insn.h
index 1ce75a442638..f38738fa54a2 100644
--- a/arch/arm64/include/asm/insn.h
+++ b/arch/arm64/include/asm/insn.h
@@ -247,6 +247,8 @@ enum aarch64_insn_data2_type {
enum aarch64_insn_data3_type {
AARCH64_INSN_DATA3_MADD,
AARCH64_INSN_DATA3_MSUB,
+ AARCH64_INSN_DATA3_SMULH,
+ AARCH64_INSN_DATA3_UMULH,
};
enum aarch64_insn_logic_type {
@@ -396,6 +398,8 @@ __AARCH64_INSN_FUNCS(sub_ext, 0x7FE00000, 0x4B200000)
__AARCH64_INSN_FUNCS(subs_ext, 0x7FE00000, 0x6B200000)
__AARCH64_INSN_FUNCS(madd, 0x7FE08000, 0x1B000000)
__AARCH64_INSN_FUNCS(msub, 0x7FE08000, 0x1B008000)
+__AARCH64_INSN_FUNCS(smulh, 0x7FE0FC00, 0x1B407C00)
+__AARCH64_INSN_FUNCS(umulh, 0x7FE0FC00, 0x1BC07C00)
__AARCH64_INSN_FUNCS(udiv, 0x7FE0FC00, 0x1AC00800)
__AARCH64_INSN_FUNCS(sdiv, 0x7FE0FC00, 0x1AC00C00)
__AARCH64_INSN_FUNCS(lslv, 0x7FE0FC00, 0x1AC02000)
diff --git a/arch/arm64/lib/insn.c b/arch/arm64/lib/insn.c
index e70ac0238515..dc483d1b8bd5 100644
--- a/arch/arm64/lib/insn.c
+++ b/arch/arm64/lib/insn.c
@@ -1155,6 +1155,12 @@ u32 aarch64_insn_gen_data3(enum aarch64_insn_register dst,
case AARCH64_INSN_DATA3_MSUB:
insn = aarch64_insn_get_msub_value();
break;
+ case AARCH64_INSN_DATA3_SMULH:
+ insn = aarch64_insn_get_smulh_value();
+ break;
+ case AARCH64_INSN_DATA3_UMULH:
+ insn = aarch64_insn_get_umulh_value();
+ break;
default:
pr_err("%s: unknown data3 encoding %d\n", __func__, type);
return AARCH64_BREAK_FAULT;
diff --git a/arch/arm64/net/bpf_jit.h b/arch/arm64/net/bpf_jit.h
index b2fe6e6dcf44..70fc329c661d 100644
--- a/arch/arm64/net/bpf_jit.h
+++ b/arch/arm64/net/bpf_jit.h
@@ -281,6 +281,12 @@
A64_VARIANT(sf), AARCH64_INSN_DATA3_MSUB)
/* Rd = Rn * Rm */
#define A64_MUL(sf, Rd, Rn, Rm) A64_MADD(sf, Rd, A64_ZR, Rn, Rm)
+/* Rd = high 64 bits of signed Rn * Rm */
+#define A64_SMULH(Rd, Rn, Rm) aarch64_insn_gen_data3(Rd, A64_ZR, Rn, Rm, \
+ AARCH64_INSN_VARIANT_64BIT, AARCH64_INSN_DATA3_SMULH)
+/* Rd = high 64 bits of unsigned Rn * Rm */
+#define A64_UMULH(Rd, Rn, Rm) aarch64_insn_gen_data3(Rd, A64_ZR, Rn, Rm, \
+ AARCH64_INSN_VARIANT_64BIT, AARCH64_INSN_DATA3_UMULH)
/* Logical (shifted register) */
#define A64_LOGIC_SREG(sf, Rd, Rn, Rm, type) \
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 6c04fee46876..054c38775dd2 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -144,6 +144,17 @@ static inline void emit_a64_mov_i(const int is64, const int reg,
}
}
+static inline void emit_a64_mul(const bool is64, const u8 dst, const u8 src,
+ const s16 off, struct jit_ctx *ctx)
+{
+ if (is64 && off == BPF_MUL_VARIANT_UHMUL)
+ emit(A64_UMULH(dst, dst, src), ctx);
+ else if (is64 && off == BPF_MUL_VARIANT_SHMUL)
+ emit(A64_SMULH(dst, dst, src), ctx);
+ else
+ emit(A64_MUL(is64, dst, dst, src), ctx);
+}
+
static int i64_i16_blocks(const u64 val, bool inverse)
{
return (((val >> 0) & 0xffff) != (inverse ? 0xffff : 0x0000)) +
@@ -1454,7 +1465,7 @@ static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn
break;
case BPF_ALU | BPF_MUL | BPF_X:
case BPF_ALU64 | BPF_MUL | BPF_X:
- emit(A64_MUL(is64, dst, dst, src), ctx);
+ emit_a64_mul(is64, dst, src, off, ctx);
break;
case BPF_ALU | BPF_DIV | BPF_X:
case BPF_ALU64 | BPF_DIV | BPF_X:
@@ -1583,7 +1594,7 @@ static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn
case BPF_ALU | BPF_MUL | BPF_K:
case BPF_ALU64 | BPF_MUL | BPF_K:
emit_a64_mov_i(is64, tmp, imm, ctx);
- emit(A64_MUL(is64, dst, dst, tmp), ctx);
+ emit_a64_mul(is64, dst, tmp, off, ctx);
break;
case BPF_ALU | BPF_DIV | BPF_K:
case BPF_ALU64 | BPF_DIV | BPF_K:
--
2.55.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH bpf-next 4/8] bpf, riscv: JIT UHMUL and SHMUL on RV64
2026-09-17 4:17 [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions Yazhou Tang
` (2 preceding siblings ...)
2026-09-17 4:17 ` [PATCH bpf-next 3/8] bpf, arm64: JIT UHMUL and SHMUL Yazhou Tang
@ 2026-09-17 4:17 ` Yazhou Tang
2026-09-17 4:17 ` [PATCH bpf-next 5/8] bpf: Reject UHMUL/SHMUL in unsupported backends Yazhou Tang
` (4 subsequent siblings)
8 siblings, 0 replies; 15+ messages in thread
From: Yazhou Tang @ 2026-09-17 4:17 UTC (permalink / raw)
To: bpf, ast, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye, Alexei Starovoitov
From: Yazhou Tang <tangyazhou518@outlook.com>
On RV64, the RISC-V M extension provides MULHU and MULH instructions that
return the high 64 bits of an unsigned or signed 64-bit multiplication,
respectively.
Add the missing MULH encoder and use MULHU and MULH to lower the UHMUL and
SHMUL BPF_MUL variants for both register and immediate sources. Keep the
existing low-half and ALU32 multiplication lowering unchanged.
Suggested-by: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Signed-off-by: Yazhou Tang <tangyazhou518@outlook.com>
Co-developed-by: Tianci Cao <ziye@zju.edu.cn>
Signed-off-by: Tianci Cao <ziye@zju.edu.cn>
Co-developed-by: Shenghao Yuan <shenghaoyuan0928@163.com>
Signed-off-by: Shenghao Yuan <shenghaoyuan0928@163.com>
---
arch/riscv/net/bpf_jit.h | 5 +++++
arch/riscv/net/bpf_jit_comp64.c | 18 +++++++++++++++---
2 files changed, 20 insertions(+), 3 deletions(-)
diff --git a/arch/riscv/net/bpf_jit.h b/arch/riscv/net/bpf_jit.h
index 5c4f53060c91..21a190a6a6d9 100644
--- a/arch/riscv/net/bpf_jit.h
+++ b/arch/riscv/net/bpf_jit.h
@@ -436,6 +436,11 @@ static inline u32 rv_mul(u8 rd, u8 rs1, u8 rs2)
return rv_r_insn(1, rs2, rs1, 0, rd, 0x33);
}
+static inline u32 rv_mulh(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(1, rs2, rs1, 1, rd, 0x33);
+}
+
static inline u32 rv_mulhu(u8 rd, u8 rs1, u8 rs2)
{
return rv_r_insn(1, rs2, rs1, 3, rd, 0x33);
diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c
index ed0a6f871dea..a773506a9209 100644
--- a/arch/riscv/net/bpf_jit_comp64.c
+++ b/arch/riscv/net/bpf_jit_comp64.c
@@ -197,6 +197,19 @@ static void emit_imm(u8 rd, s64 val, struct rv_jit_context *ctx)
emit_addi(rd, rd, lower, ctx);
}
+static void emit_mul(u8 rd, u8 rs, bool is64, s16 off,
+ struct rv_jit_context *ctx)
+{
+ if (!is64)
+ emit(rv_mulw(rd, rd, rs), ctx);
+ else if (off == BPF_MUL_VARIANT_UHMUL)
+ emit(rv_mulhu(rd, rd, rs), ctx);
+ else if (off == BPF_MUL_VARIANT_SHMUL)
+ emit(rv_mulh(rd, rd, rs), ctx);
+ else
+ emit(rv_mul(rd, rd, rs), ctx);
+}
+
static void __build_epilogue(bool is_tail_call, struct rv_jit_context *ctx)
{
int stack_adjust = ctx->stack_size, store_offset = stack_adjust - 8;
@@ -1498,7 +1511,7 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,
break;
case BPF_ALU | BPF_MUL | BPF_X:
case BPF_ALU64 | BPF_MUL | BPF_X:
- emit(is64 ? rv_mul(rd, rd, rs) : rv_mulw(rd, rd, rs), ctx);
+ emit_mul(rd, rs, is64, off, ctx);
if (!is64 && !aux->verifier_zext)
emit_zextw(rd, rd, ctx);
break;
@@ -1634,8 +1647,7 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,
case BPF_ALU | BPF_MUL | BPF_K:
case BPF_ALU64 | BPF_MUL | BPF_K:
emit_imm(RV_REG_T1, imm, ctx);
- emit(is64 ? rv_mul(rd, rd, RV_REG_T1) :
- rv_mulw(rd, rd, RV_REG_T1), ctx);
+ emit_mul(rd, RV_REG_T1, is64, off, ctx);
if (!is64 && !aux->verifier_zext)
emit_zextw(rd, rd, ctx);
break;
--
2.55.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH bpf-next 5/8] bpf: Reject UHMUL/SHMUL in unsupported backends
2026-09-17 4:17 [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions Yazhou Tang
` (3 preceding siblings ...)
2026-09-17 4:17 ` [PATCH bpf-next 4/8] bpf, riscv: JIT UHMUL and SHMUL on RV64 Yazhou Tang
@ 2026-09-17 4:17 ` Yazhou Tang
2026-09-17 4:17 ` [PATCH bpf-next 6/8] bpf: Refactor ALU instruction variant validation Yazhou Tang
` (3 subsequent siblings)
8 siblings, 0 replies; 15+ messages in thread
From: Yazhou Tang @ 2026-09-17 4:17 UTC (permalink / raw)
To: bpf, ast, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye
From: Yazhou Tang <tangyazhou518@outlook.com>
CPU JIT backends without explicit UHMUL/SHMUL support currently lower
BPF_MUL based on the opcode without examining insn->off. Once the verifier
accepts UHMUL and SHMUL, a backend without explicit support would therefore
lower either variant as the existing low-half multiplication and silently
produce the wrong result.
Reject UHMUL and SHMUL in each affected CPU JIT without native support,
using its existing unsupported-instruction path. This makes JIT compilation
fail cleanly instead of emitting an incorrect low-half multiply, and allows
the normal interpreter fallback when available.
The NFP hardware offload compiler has the same opcode-only dispatch:
BPF_ALU64 MUL instructions are routed to mul_reg64() or mul_imm64()
without considering insn->off. Reject UHMUL and SHMUL in the NFP offload
verifier as well, before they can reach the ordinary multiplication lowering.
Signed-off-by: Yazhou Tang <tangyazhou518@outlook.com>
Co-developed-by: Tianci Cao <ziye@zju.edu.cn>
Signed-off-by: Tianci Cao <ziye@zju.edu.cn>
Co-developed-by: Shenghao Yuan <shenghaoyuan0928@163.com>
Signed-off-by: Shenghao Yuan <shenghaoyuan0928@163.com>
---
arch/arc/net/bpf_jit_core.c | 3 +++
arch/arm/net/bpf_jit_32.c | 3 +++
arch/loongarch/net/bpf_jit.c | 3 +++
arch/mips/net/bpf_jit_comp32.c | 3 +++
arch/mips/net/bpf_jit_comp64.c | 3 +++
arch/parisc/net/bpf_jit_comp32.c | 3 +++
arch/parisc/net/bpf_jit_comp64.c | 3 +++
arch/powerpc/net/bpf_jit_comp32.c | 3 +++
arch/powerpc/net/bpf_jit_comp64.c | 3 +++
arch/riscv/net/bpf_jit_comp32.c | 3 +++
arch/s390/net/bpf_jit_comp.c | 3 +++
arch/sparc/net/bpf_jit_comp_64.c | 3 +++
arch/x86/net/bpf_jit_comp32.c | 3 +++
drivers/net/ethernet/netronome/nfp/bpf/verifier.c | 6 ++++++
14 files changed, 45 insertions(+)
diff --git a/arch/arc/net/bpf_jit_core.c b/arch/arc/net/bpf_jit_core.c
index 639a2736f029..78b1819cee5a 100644
--- a/arch/arc/net/bpf_jit_core.c
+++ b/arch/arc/net/bpf_jit_core.c
@@ -734,6 +734,9 @@ static int handle_insn(struct jit_context *ctx, u32 idx)
u8 len = 0;
int ret = 0;
+ if (bpf_insn_is_hmul(insn))
+ return -EOPNOTSUPP;
+
switch (code) {
/* dst += src (32-bit) */
case BPF_ALU | BPF_ADD | BPF_X:
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index 9ede81afbc50..aa466fc79d96 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -1611,6 +1611,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx)
} while (0)
#define check_imm24(imm) check_imm(24, imm)
+ if (bpf_insn_is_hmul(insn))
+ goto notyet;
+
switch (code) {
/* ALU operations */
diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
index 4da278900938..7bb3813fda8e 100644
--- a/arch/loongarch/net/bpf_jit.c
+++ b/arch/loongarch/net/bpf_jit.c
@@ -709,6 +709,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext
const s32 imm = insn->imm;
const bool is32 = BPF_CLASS(insn->code) == BPF_ALU || BPF_CLASS(insn->code) == BPF_JMP32;
+ if (bpf_insn_is_hmul(insn))
+ return -EOPNOTSUPP;
+
switch (code) {
/* dst = src */
case BPF_ALU | BPF_MOV | BPF_X:
diff --git a/arch/mips/net/bpf_jit_comp32.c b/arch/mips/net/bpf_jit_comp32.c
index 40a878b672f5..1afe15162af2 100644
--- a/arch/mips/net/bpf_jit_comp32.c
+++ b/arch/mips/net/bpf_jit_comp32.c
@@ -1472,6 +1472,9 @@ int build_insn(const struct bpf_insn *insn, struct jit_context *ctx)
s32 val, rel;
u8 alu, jmp;
+ if (bpf_insn_is_hmul(insn))
+ goto notyet;
+
switch (code) {
/* ALU operations */
/* dst = imm */
diff --git a/arch/mips/net/bpf_jit_comp64.c b/arch/mips/net/bpf_jit_comp64.c
index fa7e9aa37f49..a9765ea25282 100644
--- a/arch/mips/net/bpf_jit_comp64.c
+++ b/arch/mips/net/bpf_jit_comp64.c
@@ -643,6 +643,9 @@ int build_insn(const struct bpf_insn *insn, struct jit_context *ctx)
s32 val, rel;
u8 alu, jmp;
+ if (bpf_insn_is_hmul(insn))
+ goto notyet;
+
switch (code) {
/* ALU operations */
/* dst = imm */
diff --git a/arch/parisc/net/bpf_jit_comp32.c b/arch/parisc/net/bpf_jit_comp32.c
index 5ff0cf925fe9..6bf72d04855b 100644
--- a/arch/parisc/net/bpf_jit_comp32.c
+++ b/arch/parisc/net/bpf_jit_comp32.c
@@ -1133,6 +1133,9 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct hppa_jit_context *ctx,
BPF_CLASS(code), code, (code & BPF_ALU64) ? 1:0, BPF_SIZE(code),
BPF_OP(code), insn->src_reg, insn->dst_reg);
+ if (bpf_insn_is_hmul(insn))
+ return -EOPNOTSUPP;
+
switch (code) {
/* dst = src */
case BPF_ALU64 | BPF_MOV | BPF_X:
diff --git a/arch/parisc/net/bpf_jit_comp64.c b/arch/parisc/net/bpf_jit_comp64.c
index 54b0d5e25e02..36b6078a146f 100644
--- a/arch/parisc/net/bpf_jit_comp64.c
+++ b/arch/parisc/net/bpf_jit_comp64.c
@@ -603,6 +603,9 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct hppa_jit_context *ctx,
init_regs(&rd, &rs, insn, ctx);
+ if (bpf_insn_is_hmul(insn))
+ return -EOPNOTSUPP;
+
switch (code) {
/* dst = src */
case BPF_ALU | BPF_MOV | BPF_X:
diff --git a/arch/powerpc/net/bpf_jit_comp32.c b/arch/powerpc/net/bpf_jit_comp32.c
index bfdc50740da8..6bfc97a029d7 100644
--- a/arch/powerpc/net/bpf_jit_comp32.c
+++ b/arch/powerpc/net/bpf_jit_comp32.c
@@ -349,6 +349,9 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct code
u32 true_cond;
u32 tmp_idx;
+ if (bpf_insn_is_hmul(&insn[i]))
+ return -EOPNOTSUPP;
+
if (i && (BPF_CLASS(code) == BPF_ALU64 || BPF_CLASS(code) == BPF_ALU) &&
(BPF_CLASS(prevcode) == BPF_ALU64 || BPF_CLASS(prevcode) == BPF_ALU) &&
BPF_OP(prevcode) == BPF_MOV && BPF_SRC(prevcode) == BPF_X &&
diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
index fc9db691e820..192c96e0ed34 100644
--- a/arch/powerpc/net/bpf_jit_comp64.c
+++ b/arch/powerpc/net/bpf_jit_comp64.c
@@ -992,6 +992,9 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct code
u32 tmp_idx;
u32 jmp_off;
+ if (bpf_insn_is_hmul(&insn[i]))
+ return -EOPNOTSUPP;
+
/*
* addrs[] maps a BPF bytecode address into a real offset from
* the start of the body code.
diff --git a/arch/riscv/net/bpf_jit_comp32.c b/arch/riscv/net/bpf_jit_comp32.c
index a9e0bd5cc81d..64bc6779d25c 100644
--- a/arch/riscv/net/bpf_jit_comp32.c
+++ b/arch/riscv/net/bpf_jit_comp32.c
@@ -1014,6 +1014,9 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,
const s8 *tmp1 = bpf2rv32[TMP_REG_1];
const s8 *tmp2 = bpf2rv32[TMP_REG_2];
+ if (bpf_insn_is_hmul(insn))
+ goto notsupported;
+
switch (code) {
case BPF_ALU64 | BPF_MOV | BPF_X:
if (insn->off != 0) {
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index c4b47070bb59..9e5c865d5ea8 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -985,6 +985,9 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
bpf_jit_probe_init(&probe);
+ if (bpf_insn_is_hmul(insn))
+ return -EOPNOTSUPP;
+
switch (insn->code) {
/*
* BPF_MOV
diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c
index 2fa0e9375127..1154ddc515b0 100644
--- a/arch/sparc/net/bpf_jit_comp_64.c
+++ b/arch/sparc/net/bpf_jit_comp_64.c
@@ -901,6 +901,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx)
const s16 off = insn->off;
const s32 imm = insn->imm;
+ if (bpf_insn_is_hmul(insn))
+ return -EOPNOTSUPP;
+
if (insn->src_reg == BPF_REG_FP)
ctx->saw_frame_pointer = true;
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index 852baf2e4db4..11ed65130d1e 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -1683,6 +1683,9 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image,
int ilen;
u8 *func;
+ if (bpf_insn_is_hmul(insn))
+ goto notyet;
+
switch (code) {
/* ALU operations */
/* dst = src */
diff --git a/drivers/net/ethernet/netronome/nfp/bpf/verifier.c b/drivers/net/ethernet/netronome/nfp/bpf/verifier.c
index 1caa87da72b5..effce27f3b88 100644
--- a/drivers/net/ethernet/netronome/nfp/bpf/verifier.c
+++ b/drivers/net/ethernet/netronome/nfp/bpf/verifier.c
@@ -3,6 +3,7 @@
#include <linux/bpf.h>
#include <linux/bpf_verifier.h>
+#include <linux/filter.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
#include <linux/pkt_cls.h>
@@ -561,6 +562,11 @@ nfp_bpf_check_alu(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta,
const struct bpf_reg_state *dreg =
cur_regs(env) + meta->insn.dst_reg;
+ if (bpf_insn_is_hmul(&meta->insn)) {
+ pr_vlog(env, "UHMUL/SHMUL are not supported\n");
+ return -EOPNOTSUPP;
+ }
+
meta->umin_src = min(meta->umin_src, reg_umin(sreg));
meta->umax_src = max(meta->umax_src, reg_umax(sreg));
meta->umin_dst = min(meta->umin_dst, reg_umin(dreg));
--
2.55.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH bpf-next 6/8] bpf: Refactor ALU instruction variant validation
2026-09-17 4:17 [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions Yazhou Tang
` (4 preceding siblings ...)
2026-09-17 4:17 ` [PATCH bpf-next 5/8] bpf: Reject UHMUL/SHMUL in unsupported backends Yazhou Tang
@ 2026-09-17 4:17 ` Yazhou Tang
2026-09-17 4:17 ` [PATCH bpf-next 7/8] bpf: Add verifier support for UHMUL and SHMUL Yazhou Tang
` (2 subsequent siblings)
8 siblings, 0 replies; 15+ messages in thread
From: Yazhou Tang @ 2026-09-17 4:17 UTC (permalink / raw)
To: bpf, ast, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye
From: Yazhou Tang <tangyazhou518@outlook.com>
check_alu_fields() currently validates insn->off together with the
source-specific reserved fields. This duplicates the DIV and MOD
variant rules between register and immediate forms and makes the valid
instruction variants difficult to identify.
Move the opcode-specific insn->off validation into is_valid_alu_variant().
The helper permits off == 1 for DIV and MOD and requires off == 0 for all
other ALU operations. Validate the variant once, then separately check the
reserved immediate field for register sources and src_reg for immediate sources.
This is a pure refactoring and does not change which instruction encodings
the verifier accepts.
Signed-off-by: Yazhou Tang <tangyazhou518@outlook.com>
Co-developed-by: Tianci Cao <ziye@zju.edu.cn>
Signed-off-by: Tianci Cao <ziye@zju.edu.cn>
Co-developed-by: Shenghao Yuan <shenghaoyuan0928@163.com>
Signed-off-by: Shenghao Yuan <shenghaoyuan0928@163.com>
---
kernel/bpf/verifier.c | 24 +++++++++++++++---------
1 file changed, 15 insertions(+), 9 deletions(-)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 6c6b8d8520cd..ed8742bf504f 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -19338,6 +19338,18 @@ static int fd_array_get_map_idx(struct bpf_verifier_env *env, u32 idx)
return -EPROTO;
}
+static bool is_valid_alu_variant(const struct bpf_insn *insn)
+{
+ switch (BPF_OP(insn->code)) {
+ case BPF_DIV:
+ case BPF_MOD:
+ /* off == 1 selects SDIV/SMOD. */
+ return insn->off == 0 || insn->off == 1;
+ default:
+ return insn->off == 0;
+ }
+}
+
static int check_alu_fields(struct bpf_verifier_env *env, struct bpf_insn *insn)
{
u8 class = BPF_CLASS(insn->code);
@@ -19393,15 +19405,9 @@ static int check_alu_fields(struct bpf_verifier_env *env, struct bpf_insn *insn)
case BPF_MUL:
case BPF_DIV:
case BPF_MOD:
- if (BPF_SRC(insn->code) == BPF_X) {
- if (insn->imm != 0 || (insn->off != 0 && insn->off != 1) ||
- (insn->off == 1 && opcode != BPF_MOD && opcode != BPF_DIV)) {
- verbose(env, "BPF_ALU uses reserved fields\n");
- return -EINVAL;
- }
- } else if (insn->src_reg != BPF_REG_0 ||
- (insn->off != 0 && insn->off != 1) ||
- (insn->off == 1 && opcode != BPF_MOD && opcode != BPF_DIV)) {
+ if (!is_valid_alu_variant(insn) ||
+ (BPF_SRC(insn->code) == BPF_X && insn->imm != 0) ||
+ (BPF_SRC(insn->code) == BPF_K && insn->src_reg != BPF_REG_0)) {
verbose(env, "BPF_ALU uses reserved fields\n");
return -EINVAL;
}
--
2.55.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH bpf-next 7/8] bpf: Add verifier support for UHMUL and SHMUL
2026-09-17 4:17 [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions Yazhou Tang
` (5 preceding siblings ...)
2026-09-17 4:17 ` [PATCH bpf-next 6/8] bpf: Refactor ALU instruction variant validation Yazhou Tang
@ 2026-09-17 4:17 ` Yazhou Tang
2026-09-17 4:17 ` [PATCH bpf-next 8/8] selftests/bpf: Add bytecode tests " Yazhou Tang
2026-09-17 18:00 ` [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions Alexei Starovoitov
8 siblings, 0 replies; 15+ messages in thread
From: Yazhou Tang @ 2026-09-17 4:17 UTC (permalink / raw)
To: bpf, ast, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye, Alexei Starovoitov
From: Tianci Cao <ziye@zju.edu.cn>
Extend ALU variant validation to accept off == 1 and off == 2 as UHMUL and
SHMUL, respectively, for BPF_ALU64 MUL instructions. Both offsets remain
reserved for BPF_ALU MUL, since the high-half variants are only defined
for 64-bit operations. All other MUL offsets remain invalid.
For scalar tracking, compute the exact high-half result when both operands
are constants. Mark the destination unknown when either operand is variable
until range analysis for these variants is added. Keep the existing scalar
analysis for ordinary low-half MUL unchanged.
Reject UHMUL and SHMUL when either operand is PTR_TO_ARENA. Arena
arithmetic can be rewritten from BPF_ALU64 to BPF_ALU to enforce its 32-bit
pointer representation, but such a rewrite cannot preserve the semantics of
variants that are only defined for BPF_ALU64.
Link: https://lore.kernel.org/bpf/CAADnVQ+2O2zgdy11+vxE6dOp+3Br4zZBysHF-Hne7cP+eeucnQ@mail.gmail.com/
Suggested-by: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Signed-off-by: Tianci Cao <ziye@zju.edu.cn>
Co-developed-by: Yazhou Tang <tangyazhou518@outlook.com>
Signed-off-by: Yazhou Tang <tangyazhou518@outlook.com>
Co-developed-by: Shenghao Yuan <shenghaoyuan0928@163.com>
Signed-off-by: Shenghao Yuan <shenghaoyuan0928@163.com>
---
kernel/bpf/verifier.c | 45 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 45 insertions(+)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index ed8742bf504f..92603a6a030b 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -15238,6 +15238,24 @@ static void scalar_min_max_mul(struct bpf_reg_state *dst_reg,
cnum64_from_srange(smin, smax));
}
+static void scalar_min_max_uhmul(struct bpf_reg_state *dst_reg,
+ struct bpf_reg_state *src_reg)
+{
+ u64 dst = reg_const_value(dst_reg, false);
+ u64 src = reg_const_value(src_reg, false);
+
+ ___mark_reg_known(dst_reg, bpf_uhmul64(dst, src));
+}
+
+static void scalar_min_max_shmul(struct bpf_reg_state *dst_reg,
+ struct bpf_reg_state *src_reg)
+{
+ s64 dst = reg_const_value(dst_reg, false);
+ s64 src = reg_const_value(src_reg, false);
+
+ ___mark_reg_known(dst_reg, bpf_shmul64(dst, src));
+}
+
static void scalar32_min_max_udiv(struct bpf_reg_state *dst_reg,
struct bpf_reg_state *src_reg)
{
@@ -15934,6 +15952,25 @@ static int adjust_scalar_min_max_vals(struct bpf_verifier_env *env,
dst_reg->var_off = tnum_neg(env->fake_reg[0].var_off);
break;
case BPF_MUL:
+ /*
+ * Range analysis for UHMUL/SHMUL is not implemented yet.
+ * Compute an exact result only when both operands are constants.
+ * Ordinary low-half MUL continues to use the analysis below.
+ */
+ if (bpf_insn_is_hmul(insn) &&
+ (!is_reg_const(dst_reg, false) ||
+ !is_reg_const(&src_reg, false))) {
+ __mark_reg_unknown(env, dst_reg);
+ break;
+ }
+ if (!alu32 && insn->off == BPF_MUL_VARIANT_UHMUL) {
+ scalar_min_max_uhmul(dst_reg, &src_reg);
+ break;
+ }
+ if (!alu32 && insn->off == BPF_MUL_VARIANT_SHMUL) {
+ scalar_min_max_shmul(dst_reg, &src_reg);
+ break;
+ }
dst_reg->var_off = tnum_mul(dst_reg->var_off, src_reg.var_off);
scalar32_min_max_mul(dst_reg, &src_reg);
scalar_min_max_mul(dst_reg, &src_reg);
@@ -16059,6 +16096,11 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env,
if (dst_reg->type == PTR_TO_ARENA || (src_reg && src_reg->type == PTR_TO_ARENA)) {
struct bpf_insn_aux_data *aux = cur_aux(env);
+ if (bpf_insn_is_hmul(insn)) {
+ verbose(env, "UHMUL/SHMUL with arena pointers are not supported\n");
+ return -EOPNOTSUPP;
+ }
+
if (dst_reg->type != PTR_TO_ARENA)
*dst_reg = *src_reg;
@@ -19341,6 +19383,9 @@ static int fd_array_get_map_idx(struct bpf_verifier_env *env, u32 idx)
static bool is_valid_alu_variant(const struct bpf_insn *insn)
{
switch (BPF_OP(insn->code)) {
+ case BPF_MUL:
+ return insn->off == BPF_MUL_VARIANT_LO ||
+ bpf_insn_is_hmul(insn);
case BPF_DIV:
case BPF_MOD:
/* off == 1 selects SDIV/SMOD. */
--
2.55.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH bpf-next 8/8] selftests/bpf: Add bytecode tests for UHMUL and SHMUL
2026-09-17 4:17 [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions Yazhou Tang
` (6 preceding siblings ...)
2026-09-17 4:17 ` [PATCH bpf-next 7/8] bpf: Add verifier support for UHMUL and SHMUL Yazhou Tang
@ 2026-09-17 4:17 ` Yazhou Tang
2026-09-17 18:00 ` [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions Alexei Starovoitov
8 siblings, 0 replies; 15+ messages in thread
From: Yazhou Tang @ 2026-09-17 4:17 UTC (permalink / raw)
To: bpf, ast, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye
From: Yazhou Tang <tangyazhou518@outlook.com>
Add test_progs coverage for UHMUL and SHMUL with the following changes:
1. Add arithmetic execution tests in verifier_hmul.c and register the new
test suite with the verifier test runner:
- Gate the execution cases on x86, arm64, and RV64, matching the JIT support
added in this series, and provide a dummy program for other targets.
- Inject only UHMUL and SHMUL as raw .8byte values built with
BPF_RAW_INSN(), while keeping the surrounding instructions in the
existing assembly syntax. This avoids requiring assembler support
for the new mnemonics.
- Exercise register and immediate sources, signed and unsigned boundary
values, negative immediate sign extension, and R0, R1, and R3
destinations. The R3 case covers the x86 JIT path where BPF R3 aliases
the implicit RDX result register. Run these cases in both privileged
and unprivileged modes.
- BPF_PROG_TEST_RUN exposes only the low 32 bits of R0 as its return
value, so a direct __retval() check cannot validate the upper 32 bits.
Build a 64-bit mismatch mask as actual ^ expected, then fold it to
32 bits as (u32)mismatch | (u32)(mismatch >> 32) and return the folded
value through R0. The return value is zero if and only if the complete
64-bit result matches the expected value.
2. Add arena pointer rejection tests in verifier_hmul.c. Use an arena
pointer as the destination of UHMUL and as the source of SHMUL, and
verify that the verifier rejects both programs.
3. Extend the illegal instruction encoding tests in verifier_value_illegal_alu.c.
Check reserved MUL offsets and UHMUL and SHMUL encodings in the BPF_ALU
class, reusing the existing DEFINE_BAD_OFFSET_TEST() macro.
Signed-off-by: Yazhou Tang <tangyazhou518@outlook.com>
Co-developed-by: Tianci Cao <ziye@zju.edu.cn>
Signed-off-by: Tianci Cao <ziye@zju.edu.cn>
Co-developed-by: Shenghao Yuan <shenghaoyuan0928@163.com>
Signed-off-by: Shenghao Yuan <shenghaoyuan0928@163.com>
---
.../selftests/bpf/prog_tests/verifier.c | 2 +
.../selftests/bpf/progs/verifier_hmul.c | 284 ++++++++++++++++++
.../bpf/progs/verifier_value_illegal_alu.c | 9 +-
3 files changed, 294 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/bpf/progs/verifier_hmul.c
diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c
index 7732df9bc870..bf5d9b140c45 100644
--- a/tools/testing/selftests/bpf/prog_tests/verifier.c
+++ b/tools/testing/selftests/bpf/prog_tests/verifier.c
@@ -51,6 +51,7 @@
#include "verifier_helper_packet_access.skel.h"
#include "verifier_helper_restricted.skel.h"
#include "verifier_helper_value_access.skel.h"
+#include "verifier_hmul.skel.h"
#include "verifier_int_ptr.skel.h"
#include "verifier_iterating_callbacks.skel.h"
#include "verifier_jeq_infer_not_null.skel.h"
@@ -217,6 +218,7 @@ void test_verifier_helper_access_var_len(void) { RUN(verifier_helper_access_var_
void test_verifier_helper_packet_access(void) { RUN(verifier_helper_packet_access); }
void test_verifier_helper_restricted(void) { RUN(verifier_helper_restricted); }
void test_verifier_helper_value_access(void) { RUN(verifier_helper_value_access); }
+void test_verifier_hmul(void) { RUN(verifier_hmul); }
void test_verifier_int_ptr(void) { RUN(verifier_int_ptr); }
void test_verifier_iterating_callbacks(void) { RUN(verifier_iterating_callbacks); }
void test_verifier_jeq_infer_not_null(void) { RUN(verifier_jeq_infer_not_null); }
diff --git a/tools/testing/selftests/bpf/progs/verifier_hmul.c b/tools/testing/selftests/bpf/progs/verifier_hmul.c
new file mode 100644
index 000000000000..197ceac3b798
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/verifier_hmul.c
@@ -0,0 +1,284 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/bpf.h>
+#include <limits.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+#include "../../../include/linux/filter.h"
+
+#define BPF_UHMUL64_REG(DST, SRC) \
+ BPF_RAW_INSN(BPF_ALU64 | BPF_MUL | BPF_X, DST, SRC, \
+ BPF_MUL_VARIANT_UHMUL, 0)
+
+#define BPF_UHMUL64_IMM(DST, IMM) \
+ BPF_RAW_INSN(BPF_ALU64 | BPF_MUL | BPF_K, DST, 0, \
+ BPF_MUL_VARIANT_UHMUL, IMM)
+
+#define BPF_SHMUL64_REG(DST, SRC) \
+ BPF_RAW_INSN(BPF_ALU64 | BPF_MUL | BPF_X, DST, SRC, \
+ BPF_MUL_VARIANT_SHMUL, 0)
+
+#define BPF_SHMUL64_IMM(DST, IMM) \
+ BPF_RAW_INSN(BPF_ALU64 | BPF_MUL | BPF_K, DST, 0, \
+ BPF_MUL_VARIANT_SHMUL, IMM)
+
+/*
+ * Inject UHMUL/SHMUL as raw instructions so these tests do not require
+ * assembler support for the new mnemonics. All surrounding instructions use
+ * existing BPF assembly syntax supported by the baseline selftests toolchain.
+ */
+
+#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARENA);
+ __uint(map_flags, BPF_F_MMAPABLE);
+ __uint(max_entries, 1);
+} arena SEC(".maps");
+
+SEC("syscall")
+__description("UHMUL64 with arena pointer dst")
+__failure __msg("UHMUL/SHMUL with arena pointers are not supported")
+__naked void uhmul64_arena_ptr(void)
+{
+ asm volatile (
+ "r1 = %[arena] ll;"
+ "r0 = 1;"
+ "r0 = addr_space_cast(r0, 0x0, 0x1);"
+ ".8byte %[hmul];"
+ "exit;"
+ :
+ : __imm_addr(arena),
+ __imm_insn(hmul, BPF_UHMUL64_IMM(BPF_REG_0, 2))
+ : __clobber_all);
+}
+
+SEC("syscall")
+__description("SHMUL64 with arena pointer src")
+__failure __msg("UHMUL/SHMUL with arena pointers are not supported")
+__naked void shmul64_arena_ptr(void)
+{
+ asm volatile (
+ "r1 = %[arena] ll;"
+ "r1 = 1;"
+ "r1 = addr_space_cast(r1, 0x0, 0x1);"
+ "r0 = 2;"
+ ".8byte %[hmul];"
+ "exit;"
+ :
+ : __imm_addr(arena),
+ __imm_insn(hmul, BPF_SHMUL64_REG(BPF_REG_0, BPF_REG_1))
+ : __clobber_all);
+}
+
+#endif
+
+#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || \
+ (defined(__TARGET_ARCH_riscv) && __riscv_xlen == 64)
+
+/*
+ * BPF_PROG_TEST_RUN reports a 32-bit retval. Fold both halves of
+ * (actual ^ expected) into w0 so retval == 0 checks the full 64-bit result.
+ */
+
+SEC("socket")
+__description("UHMUL64, U64_MAX * U64_MAX, register source")
+__success __success_unpriv __retval(0)
+__naked void uhmul64_max_reg(void)
+{
+ asm volatile (
+ "r0 = -1;"
+ "r1 = -1;"
+ ".8byte %[hmul];"
+ "r2 = -2;"
+ "r0 ^= r2;"
+ "r2 = r0;"
+ "r2 >>= 32;"
+ "w0 |= w2;"
+ "exit;"
+ :
+ : __imm_insn(hmul, BPF_UHMUL64_REG(BPF_REG_0, BPF_REG_1))
+ : __clobber_all);
+}
+
+SEC("socket")
+__description("UHMUL64, (1ULL << 63) * 2, register source")
+__success __success_unpriv __retval(0)
+__naked void uhmul64_pow2_reg(void)
+{
+ asm volatile (
+ "r0 = 2;"
+ "r1 = %[llong_min] ll;"
+ ".8byte %[hmul];"
+ "r0 = r1;"
+ "r2 = 1;"
+ "r0 ^= r2;"
+ "r2 = r0;"
+ "r2 >>= 32;"
+ "w0 |= w2;"
+ "exit;"
+ :
+ : __imm_const(llong_min, LLONG_MIN),
+ __imm_insn(hmul, BPF_UHMUL64_REG(BPF_REG_1, BPF_REG_0))
+ : __clobber_all);
+}
+
+SEC("socket")
+__description("UHMUL64, (1ULL << 63) * 2, R3 destination")
+__success __success_unpriv __retval(0)
+__naked void uhmul64_r3_dst(void)
+{
+ asm volatile (
+ "r3 = %[llong_min] ll;"
+ "r0 = 2;"
+ ".8byte %[hmul];"
+ "r2 = 1;"
+ "r3 ^= r2;"
+ "r2 = r3;"
+ "r2 >>= 32;"
+ "w3 |= w2;"
+ "r0 = r3;"
+ "exit;"
+ :
+ : __imm_const(llong_min, LLONG_MIN),
+ __imm_insn(hmul, BPF_UHMUL64_REG(BPF_REG_3, BPF_REG_0))
+ : __clobber_all);
+}
+
+SEC("socket")
+__description("UHMUL64, (1ULL << 63) * 2, immediate source")
+__success __success_unpriv __retval(0)
+__naked void uhmul64_pow2_imm(void)
+{
+ asm volatile (
+ "r0 = %[llong_min] ll;"
+ ".8byte %[hmul];"
+ "r2 = 1;"
+ "r0 ^= r2;"
+ "r2 = r0;"
+ "r2 >>= 32;"
+ "w0 |= w2;"
+ "exit;"
+ :
+ : __imm_const(llong_min, LLONG_MIN),
+ __imm_insn(hmul, BPF_UHMUL64_IMM(BPF_REG_0, 2))
+ : __clobber_all);
+}
+
+SEC("socket")
+__description("UHMUL64, (1ULL << 63) * -2, immediate source")
+__success __success_unpriv __retval(0)
+__naked void uhmul64_neg_imm(void)
+{
+ asm volatile (
+ "r0 = %[llong_min] ll;"
+ ".8byte %[hmul];"
+ "r2 = %[llong_max] ll;"
+ "r0 ^= r2;"
+ "r2 = r0;"
+ "r2 >>= 32;"
+ "w0 |= w2;"
+ "exit;"
+ :
+ : __imm_const(llong_min, LLONG_MIN),
+ __imm_const(llong_max, LLONG_MAX),
+ __imm_insn(hmul, BPF_UHMUL64_IMM(BPF_REG_0, -2))
+ : __clobber_all);
+}
+
+SEC("socket")
+__description("SHMUL64, INT64_MAX * INT64_MAX, register source")
+__success __success_unpriv __retval(0)
+__naked void shmul64_max_reg(void)
+{
+ asm volatile (
+ "r0 = %[llong_max] ll;"
+ "r1 = %[llong_max] ll;"
+ ".8byte %[hmul];"
+ "r2 = %[expected] ll;"
+ "r0 ^= r2;"
+ "r2 = r0;"
+ "r2 >>= 32;"
+ "w0 |= w2;"
+ "exit;"
+ :
+ : __imm_const(expected, 0x3fffffffffffffffULL),
+ __imm_const(llong_max, LLONG_MAX),
+ __imm_insn(hmul, BPF_SHMUL64_REG(BPF_REG_0, BPF_REG_1))
+ : __clobber_all);
+}
+
+SEC("socket")
+__description("SHMUL64, INT64_MIN * -2, register source")
+__success __success_unpriv __retval(0)
+__naked void shmul64_min_neg2_reg(void)
+{
+ asm volatile (
+ "r0 = %[llong_min] ll;"
+ "r1 = -2;"
+ ".8byte %[hmul];"
+ "r2 = 1;"
+ "r0 ^= r2;"
+ "r2 = r0;"
+ "r2 >>= 32;"
+ "w0 |= w2;"
+ "exit;"
+ :
+ : __imm_const(llong_min, LLONG_MIN),
+ __imm_insn(hmul, BPF_SHMUL64_REG(BPF_REG_0, BPF_REG_1))
+ : __clobber_all);
+}
+
+SEC("socket")
+__description("SHMUL64, 1 * -2, immediate source")
+__success __success_unpriv __retval(0)
+__naked void shmul64_neg_imm(void)
+{
+ asm volatile (
+ "r0 = 1;"
+ ".8byte %[hmul];"
+ "r2 = -1;"
+ "r0 ^= r2;"
+ "r2 = r0;"
+ "r2 >>= 32;"
+ "w0 |= w2;"
+ "exit;"
+ :
+ : __imm_insn(hmul, BPF_SHMUL64_IMM(BPF_REG_0, -2))
+ : __clobber_all);
+}
+
+SEC("socket")
+__description("SHMUL64, INT64_MIN * 2, immediate source")
+__success __success_unpriv __retval(0)
+__naked void shmul64_min_imm(void)
+{
+ asm volatile (
+ "r0 = %[llong_min] ll;"
+ ".8byte %[hmul];"
+ "r2 = -1;"
+ "r0 ^= r2;"
+ "r2 = r0;"
+ "r2 >>= 32;"
+ "w0 |= w2;"
+ "exit;"
+ :
+ : __imm_const(llong_min, LLONG_MIN),
+ __imm_insn(hmul, BPF_SHMUL64_IMM(BPF_REG_0, 2))
+ : __clobber_all);
+}
+
+#else
+
+SEC("socket")
+__description("UHMUL/SHMUL are not supported by this JIT, use a dummy test")
+__skip("UHMUL/SHMUL are not supported by this jit")
+__success
+int dummy_test(void)
+{
+ return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/verifier_value_illegal_alu.c b/tools/testing/selftests/bpf/progs/verifier_value_illegal_alu.c
index 4d8273c258d5..d7fb6b71b206 100644
--- a/tools/testing/selftests/bpf/progs/verifier_value_illegal_alu.c
+++ b/tools/testing/selftests/bpf/progs/verifier_value_illegal_alu.c
@@ -202,7 +202,8 @@ __naked void flow_keys_illegal_variable_offset_alu(void)
/*
* Offset fields of 0 and 1 are legal for BPF_{DIV,MOD} instructions.
- * Offset fields of 0 are legal for the rest of ALU instructions.
+ * Offset fields of 0, 1 and 2 are legal for BPF_ALU64 | BPF_MUL.
+ * Only offset 0 is legal for BPF_ALU | BPF_MUL.
* Test that error is reported for illegal offsets, assuming that tests
* for legal offsets exist.
*/
@@ -212,5 +213,11 @@ DEFINE_BAD_OFFSET_TEST(bad_offset_addx, BPF_ALU64 | BPF_ADD | BPF_X, -1, 0)
DEFINE_BAD_OFFSET_TEST(bad_offset_divx2, BPF_ALU64 | BPF_DIV | BPF_X, 2, 0)
DEFINE_BAD_OFFSET_TEST(bad_offset_modk2, BPF_ALU64 | BPF_MOD | BPF_K, 2, 1)
DEFINE_BAD_OFFSET_TEST(bad_offset_addx2, BPF_ALU64 | BPF_ADD | BPF_X, 1, 0)
+DEFINE_BAD_OFFSET_TEST(bad_offset_mulx, BPF_ALU64 | BPF_MUL | BPF_X, -1, 0)
+DEFINE_BAD_OFFSET_TEST(bad_offset_mulk, BPF_ALU64 | BPF_MUL | BPF_K, 3, 0)
+DEFINE_BAD_OFFSET_TEST(bad_uhmul32_x, BPF_ALU | BPF_MUL | BPF_X,
+ BPF_MUL_VARIANT_UHMUL, 0)
+DEFINE_BAD_OFFSET_TEST(bad_shmul32_k, BPF_ALU | BPF_MUL | BPF_K,
+ BPF_MUL_VARIANT_SHMUL, 0)
char _license[] SEC("license") = "GPL";
--
2.55.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions
2026-09-17 4:17 [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions Yazhou Tang
` (7 preceding siblings ...)
2026-09-17 4:17 ` [PATCH bpf-next 8/8] selftests/bpf: Add bytecode tests " Yazhou Tang
@ 2026-09-17 18:00 ` Alexei Starovoitov
2026-09-18 9:37 ` Yazhou Tang
8 siblings, 1 reply; 15+ messages in thread
From: Alexei Starovoitov @ 2026-09-17 18:00 UTC (permalink / raw)
To: Yazhou Tang, bpf, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye
On Thu, Sep 17, 2026 at 12:17 PM Yazhou Tang <tangyazhou@zju.edu.cn> wrote:
> Patch 8/8 uses raw bytecode for UHMUL and SHMUL because LLVM support has
> not been merged yet. Once it is available, the raw encodings can be
> replaced with the uh*= and sh*= mnemonics. A draft LLVM implementation
> is available at:
>
> https://github.com/ADSWT518/llvm-project/tree/bpf-hmul-insn
>
> We plan to finalize and submit it as an LLVM pull request after the
> kernel-side ISA changes are merged.
Pls send the LLVM PR now. It needs to be reviewed together
with kernel side.
I think it enables ISD::MULHU/MULHS legal under -mcpu=v4.
Does it mean that existing programs might generate new instructions
and stop loading on current kernels?
If so, we'd have to introduce -mcpu=v5 for that
and extra flag like we did for other insns.
> arch/arm64/include/asm/insn.h | 4 +
> arch/arm64/lib/insn.c | 6 +
> arch/arm64/net/bpf_jit.h
| 6 +
> arch/arm64/net/bpf_jit_comp.c | 15 +-
> arch/loongarch/net/bpf_jit.c | 3 +
> arch/mips/net/bpf_jit_comp32.c | 3 +
> arch/mips/net/bpf_jit_comp64.c | 3 +
> arch/parisc/net/bpf_jit_comp32.c | 3 +
> arch/parisc/net/bpf_jit_comp64.c | 3 +
> arch/powerpc/net/bpf_jit_comp32.c | 3 +
> arch/powerpc/net/bpf_jit_comp64.c | 3 +
> arch/riscv/net/bpf_jit.h | 5 +
> arch/riscv/net/bpf_jit_comp32.c | 3 +
> arch/riscv/net/bpf_jit_comp64.c | 18 +-
Pls cc arm64, riscv or other arch JIT maintainers on respin.
We need acks from them.
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions
2026-09-17 18:00 ` [PATCH bpf-next 0/8] bpf: Add UHMUL and SHMUL instructions Alexei Starovoitov
@ 2026-09-18 9:37 ` Yazhou Tang
0 siblings, 0 replies; 15+ messages in thread
From: Yazhou Tang @ 2026-09-18 9:37 UTC (permalink / raw)
To: Alexei Starovoitov, bpf, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, tangyazhou518, shenghaoyuan0928,
ziye
On 9/18/26 02:00, Alexei Starovoitov wrote:
> On Thu, Sep 17, 2026 at 12:17 PM Yazhou Tang <tangyazhou@zju.edu.cn> wrote:
>> Patch 8/8 uses raw bytecode for UHMUL and SHMUL because LLVM support has
>> not been merged yet. Once it is available, the raw encodings can be
>> replaced with the uh*= and sh*= mnemonics. A draft LLVM implementation
>> is available at:
>>
>> https://github.com/ADSWT518/llvm-project/tree/bpf-hmul-insn
>>
>> We plan to finalize and submit it as an LLVM pull request after the
>> kernel-side ISA changes are merged.
>
> Pls send the LLVM PR now. It needs to be reviewed together
> with kernel side.
>
> I think it enables ISD::MULHU/MULHS legal under -mcpu=v4.
> Does it mean that existing programs might generate new instructions
> and stop loading on current kernels?
> If so, we'd have to introduce -mcpu=v5 for that
> and extra flag like we did for other insns.
Thanks for pointing this out. Enabling HMUL under -mcpu=v4
would change code generation for existing programs and
could make them fail to load on older kernels.
I'll move HMUL to -mcpu=v5, keep a separate disable flag,
and send the LLVM PR for joint review with the kernel series.
>
>> arch/arm64/include/asm/insn.h | 4 +
>> arch/arm64/lib/insn.c | 6 +
>> arch/arm64/net/bpf_jit.h
> | 6 +
>> arch/arm64/net/bpf_jit_comp.c | 15 +-
>> arch/loongarch/net/bpf_jit.c | 3 +
>> arch/mips/net/bpf_jit_comp32.c | 3 +
>> arch/mips/net/bpf_jit_comp64.c | 3 +
>> arch/parisc/net/bpf_jit_comp32.c | 3 +
>> arch/parisc/net/bpf_jit_comp64.c | 3 +
>> arch/powerpc/net/bpf_jit_comp32.c | 3 +
>> arch/powerpc/net/bpf_jit_comp64.c | 3 +
>> arch/riscv/net/bpf_jit.h | 5 +
>> arch/riscv/net/bpf_jit_comp32.c | 3 +
>> arch/riscv/net/bpf_jit_comp64.c | 18 +-
>
> Pls cc arm64, riscv or other arch JIT maintainers on respin.
> We need acks from them.
Ok, I'll cc them on respin.
^ permalink raw reply [flat|nested] 15+ messages in thread