From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22519C433F5 for ; Tue, 18 Jan 2022 02:24:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245159AbiARCY5 (ORCPT ); Mon, 17 Jan 2022 21:24:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233803AbiARCVW (ORCPT ); Mon, 17 Jan 2022 21:21:22 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1A24C061757; Mon, 17 Jan 2022 18:21:21 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 8A808B81232; Tue, 18 Jan 2022 02:21:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5801EC36AE3; Tue, 18 Jan 2022 02:21:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1642472479; bh=wc9zOoYdRUlpL/9CmY8TpP2Rxm80a4UiTI/3souVWKU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cyhp/T4AFFnlFRcOCTcM7GSCyzG6t/NkFnKVdzaew24NoFOqlNp22CNB0R7HzH16Q oTzBY27j0PEAy/S9IQfgEZdm5enDYf+EnrIzP53xtSPObMO+smDGvEAavtM3W1dUGg 4eQ5n3lt5EfnHv/ykc/xTUxLWAGEKDxtQl4f6BUFQUp0h1quMF58DZsfY0ok+Bj1k+ JmUgkKLUaTyDiggvsNvYcLJgIidG8KnL2lznL6lFHPKe+OTBYKZSIcwh/UNXVqTese yb6vI5BIHmJ1lH+bdFZ7o33hHGLTzK16UXr1WgDBWPuX8rln6tJphJSVA3n1kKl+zE q7XxWgE69qYow== From: Sasha Levin To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Cc: Tiezhu Yang , Daniel Borkmann , Johan Almbladh , Ilya Leoshkevich , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Sasha Levin , illusionist.neo@gmail.com, linux@armlinux.org.uk, ast@kernel.org, andrii@kernel.org, zlim.lnx@gmail.com, catalin.marinas@arm.com, will@kernel.org, paulburton@kernel.org, tsbogend@alpha.franken.de, naveen.n.rao@linux.ibm.com, mpe@ellerman.id.au, luke.r.nels@gmail.com, xi.wang@gmail.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, hca@linux.ibm.com, gor@linux.ibm.com, borntraeger@linux.ibm.com, davem@davemloft.net, yoshfuji@linux-ipv6.org, dsahern@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, udknight@gmail.com, yhs@fb.com, john.fastabend@gmail.com, davemarchevsky@fb.com, joannekoong@fb.com, liuhangbin@gmail.com, revest@chromium.org, joe@cilium.io, brouer@redhat.com, netdev@vger.kernel.org, bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH AUTOSEL 5.16 026/217] bpf: Change value of MAX_TAIL_CALL_CNT from 32 to 33 Date: Mon, 17 Jan 2022 21:16:29 -0500 Message-Id: <20220118021940.1942199-26-sashal@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220118021940.1942199-1-sashal@kernel.org> References: <20220118021940.1942199-1-sashal@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-stable: review X-Patchwork-Hint: Ignore Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org From: Tiezhu Yang [ Upstream commit ebf7f6f0a6cdcc17a3da52b81e4b3a98c4005028 ] In the current code, the actual max tail call count is 33 which is greater than MAX_TAIL_CALL_CNT (defined as 32). The actual limit is not consistent with the meaning of MAX_TAIL_CALL_CNT and thus confusing at first glance. We can see the historical evolution from commit 04fd61ab36ec ("bpf: allow bpf programs to tail-call other bpf programs") and commit f9dabe016b63 ("bpf: Undo off-by-one in interpreter tail call count limit"). In order to avoid changing existing behavior, the actual limit is 33 now, this is reasonable. After commit 874be05f525e ("bpf, tests: Add tail call test suite"), we can see there exists failed testcase. On all archs when CONFIG_BPF_JIT_ALWAYS_ON is not set: # echo 0 > /proc/sys/net/core/bpf_jit_enable # modprobe test_bpf # dmesg | grep -w FAIL Tail call error path, max count reached jited:0 ret 34 != 33 FAIL On some archs: # echo 1 > /proc/sys/net/core/bpf_jit_enable # modprobe test_bpf # dmesg | grep -w FAIL Tail call error path, max count reached jited:1 ret 34 != 33 FAIL Although the above failed testcase has been fixed in commit 18935a72eb25 ("bpf/tests: Fix error in tail call limit tests"), it would still be good to change the value of MAX_TAIL_CALL_CNT from 32 to 33 to make the code more readable. The 32-bit x86 JIT was using a limit of 32, just fix the wrong comments and limit to 33 tail calls as the constant MAX_TAIL_CALL_CNT updated. For the mips64 JIT, use "ori" instead of "addiu" as suggested by Johan Almbladh. For the riscv JIT, use RV_REG_TCC directly to save one register move as suggested by Björn Töpel. For the other implementations, no function changes, it does not change the current limit 33, the new value of MAX_TAIL_CALL_CNT can reflect the actual max tail call count, the related tail call testcases in test_bpf module and selftests can work well for the interpreter and the JIT. Here are the test results on x86_64: # uname -m x86_64 # echo 0 > /proc/sys/net/core/bpf_jit_enable # modprobe test_bpf test_suite=test_tail_calls # dmesg | tail -1 test_bpf: test_tail_calls: Summary: 8 PASSED, 0 FAILED, [0/8 JIT'ed] # rmmod test_bpf # echo 1 > /proc/sys/net/core/bpf_jit_enable # modprobe test_bpf test_suite=test_tail_calls # dmesg | tail -1 test_bpf: test_tail_calls: Summary: 8 PASSED, 0 FAILED, [8/8 JIT'ed] # rmmod test_bpf # ./test_progs -t tailcalls #142 tailcalls:OK Summary: 1/11 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Tiezhu Yang Signed-off-by: Daniel Borkmann Tested-by: Johan Almbladh Tested-by: Ilya Leoshkevich Acked-by: Björn Töpel Acked-by: Johan Almbladh Acked-by: Ilya Leoshkevich Link: https://lore.kernel.org/bpf/1636075800-3264-1-git-send-email-yangtiezhu@loongson.cn Signed-off-by: Sasha Levin --- arch/arm/net/bpf_jit_32.c | 5 +++-- arch/arm64/net/bpf_jit_comp.c | 5 +++-- arch/mips/net/bpf_jit_comp32.c | 3 +-- arch/mips/net/bpf_jit_comp64.c | 2 +- arch/powerpc/net/bpf_jit_comp32.c | 4 ++-- arch/powerpc/net/bpf_jit_comp64.c | 4 ++-- arch/riscv/net/bpf_jit_comp32.c | 6 ++---- arch/riscv/net/bpf_jit_comp64.c | 7 +++---- arch/s390/net/bpf_jit_comp.c | 6 +++--- arch/sparc/net/bpf_jit_comp_64.c | 2 +- arch/x86/net/bpf_jit_comp.c | 10 +++++----- arch/x86/net/bpf_jit_comp32.c | 4 ++-- include/linux/bpf.h | 2 +- include/uapi/linux/bpf.h | 2 +- kernel/bpf/core.c | 3 ++- lib/test_bpf.c | 4 ++-- tools/include/uapi/linux/bpf.h | 2 +- 17 files changed, 35 insertions(+), 36 deletions(-) diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c index eeb6dc0ecf463..e59b41e9ab0c1 100644 --- a/arch/arm/net/bpf_jit_32.c +++ b/arch/arm/net/bpf_jit_32.c @@ -1199,7 +1199,8 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx) /* tmp2[0] = array, tmp2[1] = index */ - /* if (tail_call_cnt > MAX_TAIL_CALL_CNT) + /* + * if (tail_call_cnt >= MAX_TAIL_CALL_CNT) * goto out; * tail_call_cnt++; */ @@ -1208,7 +1209,7 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx) tc = arm_bpf_get_reg64(tcc, tmp, ctx); emit(ARM_CMP_I(tc[0], hi), ctx); _emit(ARM_COND_EQ, ARM_CMP_I(tc[1], lo), ctx); - _emit(ARM_COND_HI, ARM_B(jmp_offset), ctx); + _emit(ARM_COND_CS, ARM_B(jmp_offset), ctx); emit(ARM_ADDS_I(tc[1], tc[1], 1), ctx); emit(ARM_ADC_I(tc[0], tc[0], 0), ctx); arm_bpf_put_reg64(tcc, tmp, ctx); diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index 3a8a7140a9bfb..356fb2116c6b2 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -287,13 +287,14 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx) emit(A64_CMP(0, r3, tmp), ctx); emit(A64_B_(A64_COND_CS, jmp_offset), ctx); - /* if (tail_call_cnt > MAX_TAIL_CALL_CNT) + /* + * if (tail_call_cnt >= MAX_TAIL_CALL_CNT) * goto out; * tail_call_cnt++; */ emit_a64_mov_i64(tmp, MAX_TAIL_CALL_CNT, ctx); emit(A64_CMP(1, tcc, tmp), ctx); - emit(A64_B_(A64_COND_HI, jmp_offset), ctx); + emit(A64_B_(A64_COND_CS, jmp_offset), ctx); emit(A64_ADD_I(1, tcc, tcc, 1), ctx); /* prog = array->ptrs[index]; diff --git a/arch/mips/net/bpf_jit_comp32.c b/arch/mips/net/bpf_jit_comp32.c index bd996ede12f8e..044b11b65bcac 100644 --- a/arch/mips/net/bpf_jit_comp32.c +++ b/arch/mips/net/bpf_jit_comp32.c @@ -1381,8 +1381,7 @@ void build_prologue(struct jit_context *ctx) * 16-byte area in the parent's stack frame. On a tail call, the * calling function jumps into the prologue after these instructions. */ - emit(ctx, ori, MIPS_R_T9, MIPS_R_ZERO, - min(MAX_TAIL_CALL_CNT + 1, 0xffff)); + emit(ctx, ori, MIPS_R_T9, MIPS_R_ZERO, min(MAX_TAIL_CALL_CNT, 0xffff)); emit(ctx, sw, MIPS_R_T9, 0, MIPS_R_SP); /* diff --git a/arch/mips/net/bpf_jit_comp64.c b/arch/mips/net/bpf_jit_comp64.c index 815ade7242278..6475828ffb36d 100644 --- a/arch/mips/net/bpf_jit_comp64.c +++ b/arch/mips/net/bpf_jit_comp64.c @@ -552,7 +552,7 @@ void build_prologue(struct jit_context *ctx) * On a tail call, the calling function jumps into the prologue * after this instruction. */ - emit(ctx, addiu, tc, MIPS_R_ZERO, min(MAX_TAIL_CALL_CNT + 1, 0xffff)); + emit(ctx, ori, tc, MIPS_R_ZERO, min(MAX_TAIL_CALL_CNT, 0xffff)); /* === Entry-point for tail calls === */ diff --git a/arch/powerpc/net/bpf_jit_comp32.c b/arch/powerpc/net/bpf_jit_comp32.c index 0da31d41d4131..8a4faa05f9e41 100644 --- a/arch/powerpc/net/bpf_jit_comp32.c +++ b/arch/powerpc/net/bpf_jit_comp32.c @@ -221,13 +221,13 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o PPC_BCC(COND_GE, out); /* - * if (tail_call_cnt > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt >= MAX_TAIL_CALL_CNT) * goto out; */ EMIT(PPC_RAW_CMPLWI(_R0, MAX_TAIL_CALL_CNT)); /* tail_call_cnt++; */ EMIT(PPC_RAW_ADDIC(_R0, _R0, 1)); - PPC_BCC(COND_GT, out); + PPC_BCC(COND_GE, out); /* prog = array->ptrs[index]; */ EMIT(PPC_RAW_RLWINM(_R3, b2p_index, 2, 0, 29)); diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c index 8b5157ccfebae..8571aafcc9e1e 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -228,12 +228,12 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o PPC_BCC(COND_GE, out); /* - * if (tail_call_cnt > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt >= MAX_TAIL_CALL_CNT) * goto out; */ PPC_BPF_LL(b2p[TMP_REG_1], 1, bpf_jit_stack_tailcallcnt(ctx)); EMIT(PPC_RAW_CMPLWI(b2p[TMP_REG_1], MAX_TAIL_CALL_CNT)); - PPC_BCC(COND_GT, out); + PPC_BCC(COND_GE, out); /* * tail_call_cnt++; diff --git a/arch/riscv/net/bpf_jit_comp32.c b/arch/riscv/net/bpf_jit_comp32.c index e6497424cbf60..529a83b85c1c9 100644 --- a/arch/riscv/net/bpf_jit_comp32.c +++ b/arch/riscv/net/bpf_jit_comp32.c @@ -799,11 +799,10 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx) emit_bcc(BPF_JGE, lo(idx_reg), RV_REG_T1, off, ctx); /* - * temp_tcc = tcc - 1; - * if (tcc < 0) + * if (--tcc < 0) * goto out; */ - emit(rv_addi(RV_REG_T1, RV_REG_TCC, -1), ctx); + emit(rv_addi(RV_REG_TCC, RV_REG_TCC, -1), ctx); off = ninsns_rvoff(tc_ninsn - (ctx->ninsns - start_insn)); emit_bcc(BPF_JSLT, RV_REG_TCC, RV_REG_ZERO, off, ctx); @@ -829,7 +828,6 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx) if (is_12b_check(off, insn)) return -1; emit(rv_lw(RV_REG_T0, off, RV_REG_T0), ctx); - emit(rv_addi(RV_REG_TCC, RV_REG_T1, 0), ctx); /* Epilogue jumps to *(t0 + 4). */ __build_epilogue(true, ctx); return 0; diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c index f2a779c7e225d..603630b6f3c5b 100644 --- a/arch/riscv/net/bpf_jit_comp64.c +++ b/arch/riscv/net/bpf_jit_comp64.c @@ -327,12 +327,12 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx) off = ninsns_rvoff(tc_ninsn - (ctx->ninsns - start_insn)); emit_branch(BPF_JGE, RV_REG_A2, RV_REG_T1, off, ctx); - /* if (TCC-- < 0) + /* if (--TCC < 0) * goto out; */ - emit_addi(RV_REG_T1, tcc, -1, ctx); + emit_addi(RV_REG_TCC, tcc, -1, ctx); off = ninsns_rvoff(tc_ninsn - (ctx->ninsns - start_insn)); - emit_branch(BPF_JSLT, tcc, RV_REG_ZERO, off, ctx); + emit_branch(BPF_JSLT, RV_REG_TCC, RV_REG_ZERO, off, ctx); /* prog = array->ptrs[index]; * if (!prog) @@ -352,7 +352,6 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx) if (is_12b_check(off, insn)) return -1; emit_ld(RV_REG_T3, off, RV_REG_T2, ctx); - emit_mv(RV_REG_TCC, RV_REG_T1, ctx); __build_epilogue(true, ctx); return 0; } diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c index 233cc9bcd6527..9ff2bd83aad70 100644 --- a/arch/s390/net/bpf_jit_comp.c +++ b/arch/s390/net/bpf_jit_comp.c @@ -1369,7 +1369,7 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, jit->prg); /* - * if (tail_call_cnt++ > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT) * goto out; */ @@ -1381,9 +1381,9 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, EMIT4_IMM(0xa7080000, REG_W0, 1); /* laal %w1,%w0,off(%r15) */ EMIT6_DISP_LH(0xeb000000, 0x00fa, REG_W1, REG_W0, REG_15, off); - /* clij %w1,MAX_TAIL_CALL_CNT,0x2,out */ + /* clij %w1,MAX_TAIL_CALL_CNT-1,0x2,out */ patch_2_clij = jit->prg; - EMIT6_PCREL_RIEC(0xec000000, 0x007f, REG_W1, MAX_TAIL_CALL_CNT, + EMIT6_PCREL_RIEC(0xec000000, 0x007f, REG_W1, MAX_TAIL_CALL_CNT - 1, 2, jit->prg); /* diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c index 9a2f20cbd48b7..0bfe1c72a0c9e 100644 --- a/arch/sparc/net/bpf_jit_comp_64.c +++ b/arch/sparc/net/bpf_jit_comp_64.c @@ -867,7 +867,7 @@ static void emit_tail_call(struct jit_ctx *ctx) emit(LD32 | IMMED | RS1(SP) | S13(off) | RD(tmp), ctx); emit_cmpi(tmp, MAX_TAIL_CALL_CNT, ctx); #define OFFSET2 13 - emit_branch(BGU, ctx->idx, ctx->idx + OFFSET2, ctx); + emit_branch(BGEU, ctx->idx, ctx->idx + OFFSET2, ctx); emit_nop(ctx); emit_alu_K(ADD, tmp, 1, ctx); diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index bafe36e69227d..b87d98efd2240 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -412,7 +412,7 @@ static void emit_indirect_jump(u8 **pprog, int reg, u8 *ip) * ... bpf_tail_call(void *ctx, struct bpf_array *array, u64 index) ... * if (index >= array->map.max_entries) * goto out; - * if (++tail_call_cnt > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT) * goto out; * prog = array->ptrs[index]; * if (prog == NULL) @@ -446,14 +446,14 @@ static void emit_bpf_tail_call_indirect(u8 **pprog, bool *callee_regs_used, EMIT2(X86_JBE, offset); /* jbe out */ /* - * if (tail_call_cnt > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT) * goto out; */ EMIT2_off32(0x8B, 0x85, tcc_off); /* mov eax, dword ptr [rbp - tcc_off] */ EMIT3(0x83, 0xF8, MAX_TAIL_CALL_CNT); /* cmp eax, MAX_TAIL_CALL_CNT */ offset = ctx->tail_call_indirect_label - (prog + 2 - start); - EMIT2(X86_JA, offset); /* ja out */ + EMIT2(X86_JAE, offset); /* jae out */ EMIT3(0x83, 0xC0, 0x01); /* add eax, 1 */ EMIT2_off32(0x89, 0x85, tcc_off); /* mov dword ptr [rbp - tcc_off], eax */ @@ -504,14 +504,14 @@ static void emit_bpf_tail_call_direct(struct bpf_jit_poke_descriptor *poke, int offset; /* - * if (tail_call_cnt > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT) * goto out; */ EMIT2_off32(0x8B, 0x85, tcc_off); /* mov eax, dword ptr [rbp - tcc_off] */ EMIT3(0x83, 0xF8, MAX_TAIL_CALL_CNT); /* cmp eax, MAX_TAIL_CALL_CNT */ offset = ctx->tail_call_direct_label - (prog + 2 - start); - EMIT2(X86_JA, offset); /* ja out */ + EMIT2(X86_JAE, offset); /* jae out */ EMIT3(0x83, 0xC0, 0x01); /* add eax, 1 */ EMIT2_off32(0x89, 0x85, tcc_off); /* mov dword ptr [rbp - tcc_off], eax */ diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c index da9b7cfa46329..429a89c5468b5 100644 --- a/arch/x86/net/bpf_jit_comp32.c +++ b/arch/x86/net/bpf_jit_comp32.c @@ -1323,7 +1323,7 @@ static void emit_bpf_tail_call(u8 **pprog, u8 *ip) EMIT2(IA32_JBE, jmp_label(jmp_label1, 2)); /* - * if (tail_call_cnt > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT) * goto out; */ lo = (u32)MAX_TAIL_CALL_CNT; @@ -1337,7 +1337,7 @@ static void emit_bpf_tail_call(u8 **pprog, u8 *ip) /* cmp ecx,lo */ EMIT3(0x83, add_1reg(0xF8, IA32_ECX), lo); - /* ja out */ + /* jae out */ EMIT2(IA32_JAE, jmp_label(jmp_label1, 2)); /* add eax,0x1 */ diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 755f38e893be1..9f20b0f539f78 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1082,7 +1082,7 @@ struct bpf_array { }; #define BPF_COMPLEXITY_LIMIT_INSNS 1000000 /* yes. 1M insns */ -#define MAX_TAIL_CALL_CNT 32 +#define MAX_TAIL_CALL_CNT 33 #define BPF_F_ACCESS_MASK (BPF_F_RDONLY | \ BPF_F_RDONLY_PROG | \ diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index ba5af15e25f5c..b12cfceddb6e9 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1744,7 +1744,7 @@ union bpf_attr { * if the maximum number of tail calls has been reached for this * chain of programs. This limit is defined in the kernel by the * macro **MAX_TAIL_CALL_CNT** (not accessible to user space), - * which is currently set to 32. + * which is currently set to 33. * Return * 0 on success, or a negative error in case of failure. * diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 2405e39d800fe..b52dc845ecea3 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -1574,7 +1574,8 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn) if (unlikely(index >= array->map.max_entries)) goto out; - if (unlikely(tail_call_cnt > MAX_TAIL_CALL_CNT)) + + if (unlikely(tail_call_cnt >= MAX_TAIL_CALL_CNT)) goto out; tail_call_cnt++; diff --git a/lib/test_bpf.c b/lib/test_bpf.c index adae39567264f..0c5cb2d6436a4 100644 --- a/lib/test_bpf.c +++ b/lib/test_bpf.c @@ -14683,7 +14683,7 @@ static struct tail_call_test tail_call_tests[] = { BPF_EXIT_INSN(), }, .flags = FLAG_NEED_STATE | FLAG_RESULT_IN_STATE, - .result = (MAX_TAIL_CALL_CNT + 1 + 1) * MAX_TESTRUNS, + .result = (MAX_TAIL_CALL_CNT + 1) * MAX_TESTRUNS, }, { "Tail call count preserved across function calls", @@ -14705,7 +14705,7 @@ static struct tail_call_test tail_call_tests[] = { }, .stack_depth = 8, .flags = FLAG_NEED_STATE | FLAG_RESULT_IN_STATE, - .result = (MAX_TAIL_CALL_CNT + 1 + 1) * MAX_TESTRUNS, + .result = (MAX_TAIL_CALL_CNT + 1) * MAX_TESTRUNS, }, { "Tail call error path, NULL target", diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index ba5af15e25f5c..b12cfceddb6e9 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1744,7 +1744,7 @@ union bpf_attr { * if the maximum number of tail calls has been reached for this * chain of programs. This limit is defined in the kernel by the * macro **MAX_TAIL_CALL_CNT** (not accessible to user space), - * which is currently set to 32. + * which is currently set to 33. * Return * 0 on success, or a negative error in case of failure. * -- 2.34.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7EEA6C433EF for ; Tue, 18 Jan 2022 02:21:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ryEJxcQonL3Ub3S0r9uYPUe12RW1ce2zACGJB4Tzptc=; b=mAXpfaaCvEeRg/ WKBbVsRW4mMLfuyIGBqWD7d/d9jaerqYDo1e3tcKvvpHgh8Z7pd4KRKEbOH1pccPSavupozvA+5WM G30XVcUf6SyX3M/vMHA+0x8+dx+MhPGVe/z2VvtPEWGY0ZGxzj66szofUq9B/HMNJ5CH8gHvzkyZQ 4zpbMRvlK9aB86mrwhH3eytcnEUiSlJ42CeFvyqfbIUSHjeDZygi3SdNtaDC4Gr3//TAksl6EP5Fy vORziZx2zrul/ohGn7taQJU/UB0l988ofvUhObM9HXYBAYXMRMIJzEQQtx6xkVxuIQThSmhyt3m9z 0giXFINiF0Z+1h6oGLJw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1n9e7g-00Gse3-SE; Tue, 18 Jan 2022 02:21:24 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1n9e7c-00Gsc4-4v; Tue, 18 Jan 2022 02:21:22 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 90D82612D1; Tue, 18 Jan 2022 02:21:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5801EC36AE3; Tue, 18 Jan 2022 02:21:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1642472479; bh=wc9zOoYdRUlpL/9CmY8TpP2Rxm80a4UiTI/3souVWKU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cyhp/T4AFFnlFRcOCTcM7GSCyzG6t/NkFnKVdzaew24NoFOqlNp22CNB0R7HzH16Q oTzBY27j0PEAy/S9IQfgEZdm5enDYf+EnrIzP53xtSPObMO+smDGvEAavtM3W1dUGg 4eQ5n3lt5EfnHv/ykc/xTUxLWAGEKDxtQl4f6BUFQUp0h1quMF58DZsfY0ok+Bj1k+ JmUgkKLUaTyDiggvsNvYcLJgIidG8KnL2lznL6lFHPKe+OTBYKZSIcwh/UNXVqTese yb6vI5BIHmJ1lH+bdFZ7o33hHGLTzK16UXr1WgDBWPuX8rln6tJphJSVA3n1kKl+zE q7XxWgE69qYow== From: Sasha Levin To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Cc: Tiezhu Yang , Daniel Borkmann , Johan Almbladh , Ilya Leoshkevich , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Sasha Levin , illusionist.neo@gmail.com, linux@armlinux.org.uk, ast@kernel.org, andrii@kernel.org, zlim.lnx@gmail.com, catalin.marinas@arm.com, will@kernel.org, paulburton@kernel.org, tsbogend@alpha.franken.de, naveen.n.rao@linux.ibm.com, mpe@ellerman.id.au, luke.r.nels@gmail.com, xi.wang@gmail.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, hca@linux.ibm.com, gor@linux.ibm.com, borntraeger@linux.ibm.com, davem@davemloft.net, yoshfuji@linux-ipv6.org, dsahern@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, udknight@gmail.com, yhs@fb.com, john.fastabend@gmail.com, davemarchevsky@fb.com, joannekoong@fb.com, liuhangbin@gmail.com, revest@chromium.org, joe@cilium.io, brouer@redhat.com, netdev@vger.kernel.org, bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH AUTOSEL 5.16 026/217] bpf: Change value of MAX_TAIL_CALL_CNT from 32 to 33 Date: Mon, 17 Jan 2022 21:16:29 -0500 Message-Id: <20220118021940.1942199-26-sashal@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220118021940.1942199-1-sashal@kernel.org> References: <20220118021940.1942199-1-sashal@kernel.org> MIME-Version: 1.0 X-stable: review X-Patchwork-Hint: Ignore X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220117_182120_317945_257427F6 X-CRM114-Status: GOOD ( 23.85 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org RnJvbTogVGllemh1IFlhbmcgPHlhbmd0aWV6aHVAbG9vbmdzb24uY24+CgpbIFVwc3RyZWFtIGNv bW1pdCBlYmY3ZjZmMGE2Y2RjYzE3YTNkYTUyYjgxZTRiM2E5OGM0MDA1MDI4IF0KCkluIHRoZSBj dXJyZW50IGNvZGUsIHRoZSBhY3R1YWwgbWF4IHRhaWwgY2FsbCBjb3VudCBpcyAzMyB3aGljaCBp cyBncmVhdGVyCnRoYW4gTUFYX1RBSUxfQ0FMTF9DTlQgKGRlZmluZWQgYXMgMzIpLiBUaGUgYWN0 dWFsIGxpbWl0IGlzIG5vdCBjb25zaXN0ZW50CndpdGggdGhlIG1lYW5pbmcgb2YgTUFYX1RBSUxf Q0FMTF9DTlQgYW5kIHRodXMgY29uZnVzaW5nIGF0IGZpcnN0IGdsYW5jZS4KV2UgY2FuIHNlZSB0 aGUgaGlzdG9yaWNhbCBldm9sdXRpb24gZnJvbSBjb21taXQgMDRmZDYxYWIzNmVjICgiYnBmOiBh bGxvdwpicGYgcHJvZ3JhbXMgdG8gdGFpbC1jYWxsIG90aGVyIGJwZiBwcm9ncmFtcyIpIGFuZCBj b21taXQgZjlkYWJlMDE2YjYzCigiYnBmOiBVbmRvIG9mZi1ieS1vbmUgaW4gaW50ZXJwcmV0ZXIg dGFpbCBjYWxsIGNvdW50IGxpbWl0IikuIEluIG9yZGVyCnRvIGF2b2lkIGNoYW5naW5nIGV4aXN0 aW5nIGJlaGF2aW9yLCB0aGUgYWN0dWFsIGxpbWl0IGlzIDMzIG5vdywgdGhpcyBpcwpyZWFzb25h YmxlLgoKQWZ0ZXIgY29tbWl0IDg3NGJlMDVmNTI1ZSAoImJwZiwgdGVzdHM6IEFkZCB0YWlsIGNh bGwgdGVzdCBzdWl0ZSIpLCB3ZSBjYW4Kc2VlIHRoZXJlIGV4aXN0cyBmYWlsZWQgdGVzdGNhc2Uu CgpPbiBhbGwgYXJjaHMgd2hlbiBDT05GSUdfQlBGX0pJVF9BTFdBWVNfT04gaXMgbm90IHNldDoK ICMgZWNobyAwID4gL3Byb2Mvc3lzL25ldC9jb3JlL2JwZl9qaXRfZW5hYmxlCiAjIG1vZHByb2Jl IHRlc3RfYnBmCiAjIGRtZXNnIHwgZ3JlcCAtdyBGQUlMCiBUYWlsIGNhbGwgZXJyb3IgcGF0aCwg bWF4IGNvdW50IHJlYWNoZWQgaml0ZWQ6MCByZXQgMzQgIT0gMzMgRkFJTAoKT24gc29tZSBhcmNo czoKICMgZWNobyAxID4gL3Byb2Mvc3lzL25ldC9jb3JlL2JwZl9qaXRfZW5hYmxlCiAjIG1vZHBy b2JlIHRlc3RfYnBmCiAjIGRtZXNnIHwgZ3JlcCAtdyBGQUlMCiBUYWlsIGNhbGwgZXJyb3IgcGF0 aCwgbWF4IGNvdW50IHJlYWNoZWQgaml0ZWQ6MSByZXQgMzQgIT0gMzMgRkFJTAoKQWx0aG91Z2gg dGhlIGFib3ZlIGZhaWxlZCB0ZXN0Y2FzZSBoYXMgYmVlbiBmaXhlZCBpbiBjb21taXQgMTg5MzVh NzJlYjI1CigiYnBmL3Rlc3RzOiBGaXggZXJyb3IgaW4gdGFpbCBjYWxsIGxpbWl0IHRlc3RzIiks IGl0IHdvdWxkIHN0aWxsIGJlIGdvb2QKdG8gY2hhbmdlIHRoZSB2YWx1ZSBvZiBNQVhfVEFJTF9D QUxMX0NOVCBmcm9tIDMyIHRvIDMzIHRvIG1ha2UgdGhlIGNvZGUKbW9yZSByZWFkYWJsZS4KClRo ZSAzMi1iaXQgeDg2IEpJVCB3YXMgdXNpbmcgYSBsaW1pdCBvZiAzMiwganVzdCBmaXggdGhlIHdy b25nIGNvbW1lbnRzIGFuZApsaW1pdCB0byAzMyB0YWlsIGNhbGxzIGFzIHRoZSBjb25zdGFudCBN QVhfVEFJTF9DQUxMX0NOVCB1cGRhdGVkLiBGb3IgdGhlCm1pcHM2NCBKSVQsIHVzZSAib3JpIiBp bnN0ZWFkIG9mICJhZGRpdSIgYXMgc3VnZ2VzdGVkIGJ5IEpvaGFuIEFsbWJsYWRoLgpGb3IgdGhl IHJpc2N2IEpJVCwgdXNlIFJWX1JFR19UQ0MgZGlyZWN0bHkgdG8gc2F2ZSBvbmUgcmVnaXN0ZXIg bW92ZSBhcwpzdWdnZXN0ZWQgYnkgQmrDtnJuIFTDtnBlbC4gRm9yIHRoZSBvdGhlciBpbXBsZW1l bnRhdGlvbnMsIG5vIGZ1bmN0aW9uIGNoYW5nZXMsCml0IGRvZXMgbm90IGNoYW5nZSB0aGUgY3Vy cmVudCBsaW1pdCAzMywgdGhlIG5ldyB2YWx1ZSBvZiBNQVhfVEFJTF9DQUxMX0NOVApjYW4gcmVm bGVjdCB0aGUgYWN0dWFsIG1heCB0YWlsIGNhbGwgY291bnQsIHRoZSByZWxhdGVkIHRhaWwgY2Fs bCB0ZXN0Y2FzZXMKaW4gdGVzdF9icGYgbW9kdWxlIGFuZCBzZWxmdGVzdHMgY2FuIHdvcmsgd2Vs bCBmb3IgdGhlIGludGVycHJldGVyIGFuZCB0aGUKSklULgoKSGVyZSBhcmUgdGhlIHRlc3QgcmVz dWx0cyBvbiB4ODZfNjQ6CgogIyB1bmFtZSAtbQogeDg2XzY0CiAjIGVjaG8gMCA+IC9wcm9jL3N5 cy9uZXQvY29yZS9icGZfaml0X2VuYWJsZQogIyBtb2Rwcm9iZSB0ZXN0X2JwZiB0ZXN0X3N1aXRl PXRlc3RfdGFpbF9jYWxscwogIyBkbWVzZyB8IHRhaWwgLTEKIHRlc3RfYnBmOiB0ZXN0X3RhaWxf Y2FsbHM6IFN1bW1hcnk6IDggUEFTU0VELCAwIEZBSUxFRCwgWzAvOCBKSVQnZWRdCiAjIHJtbW9k IHRlc3RfYnBmCiAjIGVjaG8gMSA+IC9wcm9jL3N5cy9uZXQvY29yZS9icGZfaml0X2VuYWJsZQog IyBtb2Rwcm9iZSB0ZXN0X2JwZiB0ZXN0X3N1aXRlPXRlc3RfdGFpbF9jYWxscwogIyBkbWVzZyB8 IHRhaWwgLTEKIHRlc3RfYnBmOiB0ZXN0X3RhaWxfY2FsbHM6IFN1bW1hcnk6IDggUEFTU0VELCAw IEZBSUxFRCwgWzgvOCBKSVQnZWRdCiAjIHJtbW9kIHRlc3RfYnBmCiAjIC4vdGVzdF9wcm9ncyAt dCB0YWlsY2FsbHMKICMxNDIgdGFpbGNhbGxzOk9LCiBTdW1tYXJ5OiAxLzExIFBBU1NFRCwgMCBT S0lQUEVELCAwIEZBSUxFRAoKU2lnbmVkLW9mZi1ieTogVGllemh1IFlhbmcgPHlhbmd0aWV6aHVA bG9vbmdzb24uY24+ClNpZ25lZC1vZmYtYnk6IERhbmllbCBCb3JrbWFubiA8ZGFuaWVsQGlvZ2Vh cmJveC5uZXQ+ClRlc3RlZC1ieTogSm9oYW4gQWxtYmxhZGggPGpvaGFuLmFsbWJsYWRoQGFueWZp bmV0d29ya3MuY29tPgpUZXN0ZWQtYnk6IElseWEgTGVvc2hrZXZpY2ggPGlpaUBsaW51eC5pYm0u Y29tPgpBY2tlZC1ieTogQmrDtnJuIFTDtnBlbCA8Ympvcm5Aa2VybmVsLm9yZz4KQWNrZWQtYnk6 IEpvaGFuIEFsbWJsYWRoIDxqb2hhbi5hbG1ibGFkaEBhbnlmaW5ldHdvcmtzLmNvbT4KQWNrZWQt Ynk6IElseWEgTGVvc2hrZXZpY2ggPGlpaUBsaW51eC5pYm0uY29tPgpMaW5rOiBodHRwczovL2xv cmUua2VybmVsLm9yZy9icGYvMTYzNjA3NTgwMC0zMjY0LTEtZ2l0LXNlbmQtZW1haWwteWFuZ3Rp ZXpodUBsb29uZ3Nvbi5jbgpTaWduZWQtb2ZmLWJ5OiBTYXNoYSBMZXZpbiA8c2FzaGFsQGtlcm5l bC5vcmc+Ci0tLQogYXJjaC9hcm0vbmV0L2JwZl9qaXRfMzIuYyAgICAgICAgIHwgIDUgKysrLS0K IGFyY2gvYXJtNjQvbmV0L2JwZl9qaXRfY29tcC5jICAgICB8ICA1ICsrKy0tCiBhcmNoL21pcHMv bmV0L2JwZl9qaXRfY29tcDMyLmMgICAgfCAgMyArLS0KIGFyY2gvbWlwcy9uZXQvYnBmX2ppdF9j b21wNjQuYyAgICB8ICAyICstCiBhcmNoL3Bvd2VycGMvbmV0L2JwZl9qaXRfY29tcDMyLmMgfCAg NCArKy0tCiBhcmNoL3Bvd2VycGMvbmV0L2JwZl9qaXRfY29tcDY0LmMgfCAgNCArKy0tCiBhcmNo L3Jpc2N2L25ldC9icGZfaml0X2NvbXAzMi5jICAgfCAgNiArKy0tLS0KIGFyY2gvcmlzY3YvbmV0 L2JwZl9qaXRfY29tcDY0LmMgICB8ICA3ICsrKy0tLS0KIGFyY2gvczM5MC9uZXQvYnBmX2ppdF9j b21wLmMgICAgICB8ICA2ICsrKy0tLQogYXJjaC9zcGFyYy9uZXQvYnBmX2ppdF9jb21wXzY0LmMg IHwgIDIgKy0KIGFyY2gveDg2L25ldC9icGZfaml0X2NvbXAuYyAgICAgICB8IDEwICsrKysrLS0t LS0KIGFyY2gveDg2L25ldC9icGZfaml0X2NvbXAzMi5jICAgICB8ICA0ICsrLS0KIGluY2x1ZGUv bGludXgvYnBmLmggICAgICAgICAgICAgICB8ICAyICstCiBpbmNsdWRlL3VhcGkvbGludXgvYnBm LmggICAgICAgICAgfCAgMiArLQoga2VybmVsL2JwZi9jb3JlLmMgICAgICAgICAgICAgICAgIHwg IDMgKystCiBsaWIvdGVzdF9icGYuYyAgICAgICAgICAgICAgICAgICAgfCAgNCArKy0tCiB0b29s cy9pbmNsdWRlL3VhcGkvbGludXgvYnBmLmggICAgfCAgMiArLQogMTcgZmlsZXMgY2hhbmdlZCwg MzUgaW5zZXJ0aW9ucygrKSwgMzYgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvYXJjaC9hcm0v bmV0L2JwZl9qaXRfMzIuYyBiL2FyY2gvYXJtL25ldC9icGZfaml0XzMyLmMKaW5kZXggZWViNmRj MGVjZjQ2My4uZTU5YjQxZTlhYjBjMSAxMDA2NDQKLS0tIGEvYXJjaC9hcm0vbmV0L2JwZl9qaXRf MzIuYworKysgYi9hcmNoL2FybS9uZXQvYnBmX2ppdF8zMi5jCkBAIC0xMTk5LDcgKzExOTksOCBA QCBzdGF0aWMgaW50IGVtaXRfYnBmX3RhaWxfY2FsbChzdHJ1Y3Qgaml0X2N0eCAqY3R4KQogCiAJ LyogdG1wMlswXSA9IGFycmF5LCB0bXAyWzFdID0gaW5kZXggKi8KIAotCS8qIGlmICh0YWlsX2Nh bGxfY250ID4gTUFYX1RBSUxfQ0FMTF9DTlQpCisJLyoKKwkgKiBpZiAodGFpbF9jYWxsX2NudCA+ PSBNQVhfVEFJTF9DQUxMX0NOVCkKIAkgKglnb3RvIG91dDsKIAkgKiB0YWlsX2NhbGxfY250Kys7 CiAJICovCkBAIC0xMjA4LDcgKzEyMDksNyBAQCBzdGF0aWMgaW50IGVtaXRfYnBmX3RhaWxfY2Fs bChzdHJ1Y3Qgaml0X2N0eCAqY3R4KQogCXRjID0gYXJtX2JwZl9nZXRfcmVnNjQodGNjLCB0bXAs IGN0eCk7CiAJZW1pdChBUk1fQ01QX0kodGNbMF0sIGhpKSwgY3R4KTsKIAlfZW1pdChBUk1fQ09O RF9FUSwgQVJNX0NNUF9JKHRjWzFdLCBsbyksIGN0eCk7Ci0JX2VtaXQoQVJNX0NPTkRfSEksIEFS TV9CKGptcF9vZmZzZXQpLCBjdHgpOworCV9lbWl0KEFSTV9DT05EX0NTLCBBUk1fQihqbXBfb2Zm c2V0KSwgY3R4KTsKIAllbWl0KEFSTV9BRERTX0kodGNbMV0sIHRjWzFdLCAxKSwgY3R4KTsKIAll bWl0KEFSTV9BRENfSSh0Y1swXSwgdGNbMF0sIDApLCBjdHgpOwogCWFybV9icGZfcHV0X3JlZzY0 KHRjYywgdG1wLCBjdHgpOwpkaWZmIC0tZ2l0IGEvYXJjaC9hcm02NC9uZXQvYnBmX2ppdF9jb21w LmMgYi9hcmNoL2FybTY0L25ldC9icGZfaml0X2NvbXAuYwppbmRleCAzYThhNzE0MGE5YmZiLi4z NTZmYjIxMTZjNmIyIDEwMDY0NAotLS0gYS9hcmNoL2FybTY0L25ldC9icGZfaml0X2NvbXAuYwor KysgYi9hcmNoL2FybTY0L25ldC9icGZfaml0X2NvbXAuYwpAQCAtMjg3LDEzICsyODcsMTQgQEAg c3RhdGljIGludCBlbWl0X2JwZl90YWlsX2NhbGwoc3RydWN0IGppdF9jdHggKmN0eCkKIAllbWl0 KEE2NF9DTVAoMCwgcjMsIHRtcCksIGN0eCk7CiAJZW1pdChBNjRfQl8oQTY0X0NPTkRfQ1MsIGpt cF9vZmZzZXQpLCBjdHgpOwogCi0JLyogaWYgKHRhaWxfY2FsbF9jbnQgPiBNQVhfVEFJTF9DQUxM X0NOVCkKKwkvKgorCSAqIGlmICh0YWlsX2NhbGxfY250ID49IE1BWF9UQUlMX0NBTExfQ05UKQog CSAqICAgICBnb3RvIG91dDsKIAkgKiB0YWlsX2NhbGxfY250Kys7CiAJICovCiAJZW1pdF9hNjRf bW92X2k2NCh0bXAsIE1BWF9UQUlMX0NBTExfQ05ULCBjdHgpOwogCWVtaXQoQTY0X0NNUCgxLCB0 Y2MsIHRtcCksIGN0eCk7Ci0JZW1pdChBNjRfQl8oQTY0X0NPTkRfSEksIGptcF9vZmZzZXQpLCBj dHgpOworCWVtaXQoQTY0X0JfKEE2NF9DT05EX0NTLCBqbXBfb2Zmc2V0KSwgY3R4KTsKIAllbWl0 KEE2NF9BRERfSSgxLCB0Y2MsIHRjYywgMSksIGN0eCk7CiAKIAkvKiBwcm9nID0gYXJyYXktPnB0 cnNbaW5kZXhdOwpkaWZmIC0tZ2l0IGEvYXJjaC9taXBzL25ldC9icGZfaml0X2NvbXAzMi5jIGIv YXJjaC9taXBzL25ldC9icGZfaml0X2NvbXAzMi5jCmluZGV4IGJkOTk2ZWRlMTJmOGUuLjA0NGIx MWI2NWJjYWMgMTAwNjQ0Ci0tLSBhL2FyY2gvbWlwcy9uZXQvYnBmX2ppdF9jb21wMzIuYworKysg Yi9hcmNoL21pcHMvbmV0L2JwZl9qaXRfY29tcDMyLmMKQEAgLTEzODEsOCArMTM4MSw3IEBAIHZv aWQgYnVpbGRfcHJvbG9ndWUoc3RydWN0IGppdF9jb250ZXh0ICpjdHgpCiAJICogMTYtYnl0ZSBh cmVhIGluIHRoZSBwYXJlbnQncyBzdGFjayBmcmFtZS4gT24gYSB0YWlsIGNhbGwsIHRoZQogCSAq IGNhbGxpbmcgZnVuY3Rpb24ganVtcHMgaW50byB0aGUgcHJvbG9ndWUgYWZ0ZXIgdGhlc2UgaW5z dHJ1Y3Rpb25zLgogCSAqLwotCWVtaXQoY3R4LCBvcmksIE1JUFNfUl9UOSwgTUlQU19SX1pFUk8s Ci0JICAgICBtaW4oTUFYX1RBSUxfQ0FMTF9DTlQgKyAxLCAweGZmZmYpKTsKKwllbWl0KGN0eCwg b3JpLCBNSVBTX1JfVDksIE1JUFNfUl9aRVJPLCBtaW4oTUFYX1RBSUxfQ0FMTF9DTlQsIDB4ZmZm ZikpOwogCWVtaXQoY3R4LCBzdywgTUlQU19SX1Q5LCAwLCBNSVBTX1JfU1ApOwogCiAJLyoKZGlm ZiAtLWdpdCBhL2FyY2gvbWlwcy9uZXQvYnBmX2ppdF9jb21wNjQuYyBiL2FyY2gvbWlwcy9uZXQv YnBmX2ppdF9jb21wNjQuYwppbmRleCA4MTVhZGU3MjQyMjc4Li42NDc1ODI4ZmZiMzZkIDEwMDY0 NAotLS0gYS9hcmNoL21pcHMvbmV0L2JwZl9qaXRfY29tcDY0LmMKKysrIGIvYXJjaC9taXBzL25l dC9icGZfaml0X2NvbXA2NC5jCkBAIC01NTIsNyArNTUyLDcgQEAgdm9pZCBidWlsZF9wcm9sb2d1 ZShzdHJ1Y3Qgaml0X2NvbnRleHQgKmN0eCkKIAkgKiBPbiBhIHRhaWwgY2FsbCwgdGhlIGNhbGxp bmcgZnVuY3Rpb24ganVtcHMgaW50byB0aGUgcHJvbG9ndWUKIAkgKiBhZnRlciB0aGlzIGluc3Ry dWN0aW9uLgogCSAqLwotCWVtaXQoY3R4LCBhZGRpdSwgdGMsIE1JUFNfUl9aRVJPLCBtaW4oTUFY X1RBSUxfQ0FMTF9DTlQgKyAxLCAweGZmZmYpKTsKKwllbWl0KGN0eCwgb3JpLCB0YywgTUlQU19S X1pFUk8sIG1pbihNQVhfVEFJTF9DQUxMX0NOVCwgMHhmZmZmKSk7CiAKIAkvKiA9PT0gRW50cnkt cG9pbnQgZm9yIHRhaWwgY2FsbHMgPT09ICovCiAKZGlmZiAtLWdpdCBhL2FyY2gvcG93ZXJwYy9u ZXQvYnBmX2ppdF9jb21wMzIuYyBiL2FyY2gvcG93ZXJwYy9uZXQvYnBmX2ppdF9jb21wMzIuYwpp bmRleCAwZGEzMWQ0MWQ0MTMxLi44YTRmYWEwNWY5ZTQxIDEwMDY0NAotLS0gYS9hcmNoL3Bvd2Vy cGMvbmV0L2JwZl9qaXRfY29tcDMyLmMKKysrIGIvYXJjaC9wb3dlcnBjL25ldC9icGZfaml0X2Nv bXAzMi5jCkBAIC0yMjEsMTMgKzIyMSwxMyBAQCBzdGF0aWMgaW50IGJwZl9qaXRfZW1pdF90YWls X2NhbGwodTMyICppbWFnZSwgc3RydWN0IGNvZGVnZW5fY29udGV4dCAqY3R4LCB1MzIgbwogCVBQ Q19CQ0MoQ09ORF9HRSwgb3V0KTsKIAogCS8qCi0JICogaWYgKHRhaWxfY2FsbF9jbnQgPiBNQVhf VEFJTF9DQUxMX0NOVCkKKwkgKiBpZiAodGFpbF9jYWxsX2NudCA+PSBNQVhfVEFJTF9DQUxMX0NO VCkKIAkgKiAgIGdvdG8gb3V0OwogCSAqLwogCUVNSVQoUFBDX1JBV19DTVBMV0koX1IwLCBNQVhf VEFJTF9DQUxMX0NOVCkpOwogCS8qIHRhaWxfY2FsbF9jbnQrKzsgKi8KIAlFTUlUKFBQQ19SQVdf QURESUMoX1IwLCBfUjAsIDEpKTsKLQlQUENfQkNDKENPTkRfR1QsIG91dCk7CisJUFBDX0JDQyhD T05EX0dFLCBvdXQpOwogCiAJLyogcHJvZyA9IGFycmF5LT5wdHJzW2luZGV4XTsgKi8KIAlFTUlU KFBQQ19SQVdfUkxXSU5NKF9SMywgYjJwX2luZGV4LCAyLCAwLCAyOSkpOwpkaWZmIC0tZ2l0IGEv YXJjaC9wb3dlcnBjL25ldC9icGZfaml0X2NvbXA2NC5jIGIvYXJjaC9wb3dlcnBjL25ldC9icGZf aml0X2NvbXA2NC5jCmluZGV4IDhiNTE1N2NjZmViYWUuLjg1NzFhYWZjYzllMWUgMTAwNjQ0Ci0t LSBhL2FyY2gvcG93ZXJwYy9uZXQvYnBmX2ppdF9jb21wNjQuYworKysgYi9hcmNoL3Bvd2VycGMv bmV0L2JwZl9qaXRfY29tcDY0LmMKQEAgLTIyOCwxMiArMjI4LDEyIEBAIHN0YXRpYyBpbnQgYnBm X2ppdF9lbWl0X3RhaWxfY2FsbCh1MzIgKmltYWdlLCBzdHJ1Y3QgY29kZWdlbl9jb250ZXh0ICpj dHgsIHUzMiBvCiAJUFBDX0JDQyhDT05EX0dFLCBvdXQpOwogCiAJLyoKLQkgKiBpZiAodGFpbF9j YWxsX2NudCA+IE1BWF9UQUlMX0NBTExfQ05UKQorCSAqIGlmICh0YWlsX2NhbGxfY250ID49IE1B WF9UQUlMX0NBTExfQ05UKQogCSAqICAgZ290byBvdXQ7CiAJICovCiAJUFBDX0JQRl9MTChiMnBb VE1QX1JFR18xXSwgMSwgYnBmX2ppdF9zdGFja190YWlsY2FsbGNudChjdHgpKTsKIAlFTUlUKFBQ Q19SQVdfQ01QTFdJKGIycFtUTVBfUkVHXzFdLCBNQVhfVEFJTF9DQUxMX0NOVCkpOwotCVBQQ19C Q0MoQ09ORF9HVCwgb3V0KTsKKwlQUENfQkNDKENPTkRfR0UsIG91dCk7CiAKIAkvKgogCSAqIHRh aWxfY2FsbF9jbnQrKzsKZGlmZiAtLWdpdCBhL2FyY2gvcmlzY3YvbmV0L2JwZl9qaXRfY29tcDMy LmMgYi9hcmNoL3Jpc2N2L25ldC9icGZfaml0X2NvbXAzMi5jCmluZGV4IGU2NDk3NDI0Y2JmNjAu LjUyOWE4M2I4NWMxYzkgMTAwNjQ0Ci0tLSBhL2FyY2gvcmlzY3YvbmV0L2JwZl9qaXRfY29tcDMy LmMKKysrIGIvYXJjaC9yaXNjdi9uZXQvYnBmX2ppdF9jb21wMzIuYwpAQCAtNzk5LDExICs3OTks MTAgQEAgc3RhdGljIGludCBlbWl0X2JwZl90YWlsX2NhbGwoaW50IGluc24sIHN0cnVjdCBydl9q aXRfY29udGV4dCAqY3R4KQogCWVtaXRfYmNjKEJQRl9KR0UsIGxvKGlkeF9yZWcpLCBSVl9SRUdf VDEsIG9mZiwgY3R4KTsKIAogCS8qCi0JICogdGVtcF90Y2MgPSB0Y2MgLSAxOwotCSAqIGlmICh0 Y2MgPCAwKQorCSAqIGlmICgtLXRjYyA8IDApCiAJICogICBnb3RvIG91dDsKIAkgKi8KLQllbWl0 KHJ2X2FkZGkoUlZfUkVHX1QxLCBSVl9SRUdfVENDLCAtMSksIGN0eCk7CisJZW1pdChydl9hZGRp KFJWX1JFR19UQ0MsIFJWX1JFR19UQ0MsIC0xKSwgY3R4KTsKIAlvZmYgPSBuaW5zbnNfcnZvZmYo dGNfbmluc24gLSAoY3R4LT5uaW5zbnMgLSBzdGFydF9pbnNuKSk7CiAJZW1pdF9iY2MoQlBGX0pT TFQsIFJWX1JFR19UQ0MsIFJWX1JFR19aRVJPLCBvZmYsIGN0eCk7CiAKQEAgLTgyOSw3ICs4Mjgs NiBAQCBzdGF0aWMgaW50IGVtaXRfYnBmX3RhaWxfY2FsbChpbnQgaW5zbiwgc3RydWN0IHJ2X2pp dF9jb250ZXh0ICpjdHgpCiAJaWYgKGlzXzEyYl9jaGVjayhvZmYsIGluc24pKQogCQlyZXR1cm4g LTE7CiAJZW1pdChydl9sdyhSVl9SRUdfVDAsIG9mZiwgUlZfUkVHX1QwKSwgY3R4KTsKLQllbWl0 KHJ2X2FkZGkoUlZfUkVHX1RDQywgUlZfUkVHX1QxLCAwKSwgY3R4KTsKIAkvKiBFcGlsb2d1ZSBq dW1wcyB0byAqKHQwICsgNCkuICovCiAJX19idWlsZF9lcGlsb2d1ZSh0cnVlLCBjdHgpOwogCXJl dHVybiAwOwpkaWZmIC0tZ2l0IGEvYXJjaC9yaXNjdi9uZXQvYnBmX2ppdF9jb21wNjQuYyBiL2Fy Y2gvcmlzY3YvbmV0L2JwZl9qaXRfY29tcDY0LmMKaW5kZXggZjJhNzc5YzdlMjI1ZC4uNjAzNjMw YjZmM2M1YiAxMDA2NDQKLS0tIGEvYXJjaC9yaXNjdi9uZXQvYnBmX2ppdF9jb21wNjQuYworKysg Yi9hcmNoL3Jpc2N2L25ldC9icGZfaml0X2NvbXA2NC5jCkBAIC0zMjcsMTIgKzMyNywxMiBAQCBz dGF0aWMgaW50IGVtaXRfYnBmX3RhaWxfY2FsbChpbnQgaW5zbiwgc3RydWN0IHJ2X2ppdF9jb250 ZXh0ICpjdHgpCiAJb2ZmID0gbmluc25zX3J2b2ZmKHRjX25pbnNuIC0gKGN0eC0+bmluc25zIC0g c3RhcnRfaW5zbikpOwogCWVtaXRfYnJhbmNoKEJQRl9KR0UsIFJWX1JFR19BMiwgUlZfUkVHX1Qx LCBvZmYsIGN0eCk7CiAKLQkvKiBpZiAoVENDLS0gPCAwKQorCS8qIGlmICgtLVRDQyA8IDApCiAJ ICogICAgIGdvdG8gb3V0OwogCSAqLwotCWVtaXRfYWRkaShSVl9SRUdfVDEsIHRjYywgLTEsIGN0 eCk7CisJZW1pdF9hZGRpKFJWX1JFR19UQ0MsIHRjYywgLTEsIGN0eCk7CiAJb2ZmID0gbmluc25z X3J2b2ZmKHRjX25pbnNuIC0gKGN0eC0+bmluc25zIC0gc3RhcnRfaW5zbikpOwotCWVtaXRfYnJh bmNoKEJQRl9KU0xULCB0Y2MsIFJWX1JFR19aRVJPLCBvZmYsIGN0eCk7CisJZW1pdF9icmFuY2go QlBGX0pTTFQsIFJWX1JFR19UQ0MsIFJWX1JFR19aRVJPLCBvZmYsIGN0eCk7CiAKIAkvKiBwcm9n ID0gYXJyYXktPnB0cnNbaW5kZXhdOwogCSAqIGlmICghcHJvZykKQEAgLTM1Miw3ICszNTIsNiBA QCBzdGF0aWMgaW50IGVtaXRfYnBmX3RhaWxfY2FsbChpbnQgaW5zbiwgc3RydWN0IHJ2X2ppdF9j b250ZXh0ICpjdHgpCiAJaWYgKGlzXzEyYl9jaGVjayhvZmYsIGluc24pKQogCQlyZXR1cm4gLTE7 CiAJZW1pdF9sZChSVl9SRUdfVDMsIG9mZiwgUlZfUkVHX1QyLCBjdHgpOwotCWVtaXRfbXYoUlZf UkVHX1RDQywgUlZfUkVHX1QxLCBjdHgpOwogCV9fYnVpbGRfZXBpbG9ndWUodHJ1ZSwgY3R4KTsK IAlyZXR1cm4gMDsKIH0KZGlmZiAtLWdpdCBhL2FyY2gvczM5MC9uZXQvYnBmX2ppdF9jb21wLmMg Yi9hcmNoL3MzOTAvbmV0L2JwZl9qaXRfY29tcC5jCmluZGV4IDIzM2NjOWJjZDY1MjcuLjlmZjJi ZDgzYWFkNzAgMTAwNjQ0Ci0tLSBhL2FyY2gvczM5MC9uZXQvYnBmX2ppdF9jb21wLmMKKysrIGIv YXJjaC9zMzkwL25ldC9icGZfaml0X2NvbXAuYwpAQCAtMTM2OSw3ICsxMzY5LDcgQEAgc3RhdGlj IG5vaW5saW5lIGludCBicGZfaml0X2luc24oc3RydWN0IGJwZl9qaXQgKmppdCwgc3RydWN0IGJw Zl9wcm9nICpmcCwKIAkJCQkgaml0LT5wcmcpOwogCiAJCS8qCi0JCSAqIGlmICh0YWlsX2NhbGxf Y250KysgPiBNQVhfVEFJTF9DQUxMX0NOVCkKKwkJICogaWYgKHRhaWxfY2FsbF9jbnQrKyA+PSBN QVhfVEFJTF9DQUxMX0NOVCkKIAkJICogICAgICAgICBnb3RvIG91dDsKIAkJICovCiAKQEAgLTEz ODEsOSArMTM4MSw5IEBAIHN0YXRpYyBub2lubGluZSBpbnQgYnBmX2ppdF9pbnNuKHN0cnVjdCBi cGZfaml0ICpqaXQsIHN0cnVjdCBicGZfcHJvZyAqZnAsCiAJCUVNSVQ0X0lNTSgweGE3MDgwMDAw LCBSRUdfVzAsIDEpOwogCQkvKiBsYWFsICV3MSwldzAsb2ZmKCVyMTUpICovCiAJCUVNSVQ2X0RJ U1BfTEgoMHhlYjAwMDAwMCwgMHgwMGZhLCBSRUdfVzEsIFJFR19XMCwgUkVHXzE1LCBvZmYpOwot CQkvKiBjbGlqICV3MSxNQVhfVEFJTF9DQUxMX0NOVCwweDIsb3V0ICovCisJCS8qIGNsaWogJXcx LE1BWF9UQUlMX0NBTExfQ05ULTEsMHgyLG91dCAqLwogCQlwYXRjaF8yX2NsaWogPSBqaXQtPnBy ZzsKLQkJRU1JVDZfUENSRUxfUklFQygweGVjMDAwMDAwLCAweDAwN2YsIFJFR19XMSwgTUFYX1RB SUxfQ0FMTF9DTlQsCisJCUVNSVQ2X1BDUkVMX1JJRUMoMHhlYzAwMDAwMCwgMHgwMDdmLCBSRUdf VzEsIE1BWF9UQUlMX0NBTExfQ05UIC0gMSwKIAkJCQkgMiwgaml0LT5wcmcpOwogCiAJCS8qCmRp ZmYgLS1naXQgYS9hcmNoL3NwYXJjL25ldC9icGZfaml0X2NvbXBfNjQuYyBiL2FyY2gvc3BhcmMv bmV0L2JwZl9qaXRfY29tcF82NC5jCmluZGV4IDlhMmYyMGNiZDQ4YjcuLjBiZmUxYzcyYTBjOWUg MTAwNjQ0Ci0tLSBhL2FyY2gvc3BhcmMvbmV0L2JwZl9qaXRfY29tcF82NC5jCisrKyBiL2FyY2gv c3BhcmMvbmV0L2JwZl9qaXRfY29tcF82NC5jCkBAIC04NjcsNyArODY3LDcgQEAgc3RhdGljIHZv aWQgZW1pdF90YWlsX2NhbGwoc3RydWN0IGppdF9jdHggKmN0eCkKIAllbWl0KExEMzIgfCBJTU1F RCB8IFJTMShTUCkgfCBTMTMob2ZmKSB8IFJEKHRtcCksIGN0eCk7CiAJZW1pdF9jbXBpKHRtcCwg TUFYX1RBSUxfQ0FMTF9DTlQsIGN0eCk7CiAjZGVmaW5lIE9GRlNFVDIgMTMKLQllbWl0X2JyYW5j aChCR1UsIGN0eC0+aWR4LCBjdHgtPmlkeCArIE9GRlNFVDIsIGN0eCk7CisJZW1pdF9icmFuY2go QkdFVSwgY3R4LT5pZHgsIGN0eC0+aWR4ICsgT0ZGU0VUMiwgY3R4KTsKIAllbWl0X25vcChjdHgp OwogCiAJZW1pdF9hbHVfSyhBREQsIHRtcCwgMSwgY3R4KTsKZGlmZiAtLWdpdCBhL2FyY2gveDg2 L25ldC9icGZfaml0X2NvbXAuYyBiL2FyY2gveDg2L25ldC9icGZfaml0X2NvbXAuYwppbmRleCBi YWZlMzZlNjkyMjdkLi5iODdkOThlZmQyMjQwIDEwMDY0NAotLS0gYS9hcmNoL3g4Ni9uZXQvYnBm X2ppdF9jb21wLmMKKysrIGIvYXJjaC94ODYvbmV0L2JwZl9qaXRfY29tcC5jCkBAIC00MTIsNyAr NDEyLDcgQEAgc3RhdGljIHZvaWQgZW1pdF9pbmRpcmVjdF9qdW1wKHU4ICoqcHByb2csIGludCBy ZWcsIHU4ICppcCkKICAqIC4uLiBicGZfdGFpbF9jYWxsKHZvaWQgKmN0eCwgc3RydWN0IGJwZl9h cnJheSAqYXJyYXksIHU2NCBpbmRleCkgLi4uCiAgKiAgIGlmIChpbmRleCA+PSBhcnJheS0+bWFw Lm1heF9lbnRyaWVzKQogICogICAgIGdvdG8gb3V0OwotICogICBpZiAoKyt0YWlsX2NhbGxfY250 ID4gTUFYX1RBSUxfQ0FMTF9DTlQpCisgKiAgIGlmICh0YWlsX2NhbGxfY250KysgPj0gTUFYX1RB SUxfQ0FMTF9DTlQpCiAgKiAgICAgZ290byBvdXQ7CiAgKiAgIHByb2cgPSBhcnJheS0+cHRyc1tp bmRleF07CiAgKiAgIGlmIChwcm9nID09IE5VTEwpCkBAIC00NDYsMTQgKzQ0NiwxNCBAQCBzdGF0 aWMgdm9pZCBlbWl0X2JwZl90YWlsX2NhbGxfaW5kaXJlY3QodTggKipwcHJvZywgYm9vbCAqY2Fs bGVlX3JlZ3NfdXNlZCwKIAlFTUlUMihYODZfSkJFLCBvZmZzZXQpOyAgICAgICAgICAgICAgICAg ICAvKiBqYmUgb3V0ICovCiAKIAkvKgotCSAqIGlmICh0YWlsX2NhbGxfY250ID4gTUFYX1RBSUxf Q0FMTF9DTlQpCisJICogaWYgKHRhaWxfY2FsbF9jbnQrKyA+PSBNQVhfVEFJTF9DQUxMX0NOVCkK IAkgKglnb3RvIG91dDsKIAkgKi8KIAlFTUlUMl9vZmYzMigweDhCLCAweDg1LCB0Y2Nfb2ZmKTsg ICAgICAgICAvKiBtb3YgZWF4LCBkd29yZCBwdHIgW3JicCAtIHRjY19vZmZdICovCiAJRU1JVDMo MHg4MywgMHhGOCwgTUFYX1RBSUxfQ0FMTF9DTlQpOyAgICAgLyogY21wIGVheCwgTUFYX1RBSUxf Q0FMTF9DTlQgKi8KIAogCW9mZnNldCA9IGN0eC0+dGFpbF9jYWxsX2luZGlyZWN0X2xhYmVsIC0g KHByb2cgKyAyIC0gc3RhcnQpOwotCUVNSVQyKFg4Nl9KQSwgb2Zmc2V0KTsgICAgICAgICAgICAg ICAgICAgIC8qIGphIG91dCAqLworCUVNSVQyKFg4Nl9KQUUsIG9mZnNldCk7ICAgICAgICAgICAg ICAgICAgIC8qIGphZSBvdXQgKi8KIAlFTUlUMygweDgzLCAweEMwLCAweDAxKTsgICAgICAgICAg ICAgICAgICAvKiBhZGQgZWF4LCAxICovCiAJRU1JVDJfb2ZmMzIoMHg4OSwgMHg4NSwgdGNjX29m Zik7ICAgICAgICAgLyogbW92IGR3b3JkIHB0ciBbcmJwIC0gdGNjX29mZl0sIGVheCAqLwogCkBA IC01MDQsMTQgKzUwNCwxNCBAQCBzdGF0aWMgdm9pZCBlbWl0X2JwZl90YWlsX2NhbGxfZGlyZWN0 KHN0cnVjdCBicGZfaml0X3Bva2VfZGVzY3JpcHRvciAqcG9rZSwKIAlpbnQgb2Zmc2V0OwogCiAJ LyoKLQkgKiBpZiAodGFpbF9jYWxsX2NudCA+IE1BWF9UQUlMX0NBTExfQ05UKQorCSAqIGlmICh0 YWlsX2NhbGxfY250KysgPj0gTUFYX1RBSUxfQ0FMTF9DTlQpCiAJICoJZ290byBvdXQ7CiAJICov CiAJRU1JVDJfb2ZmMzIoMHg4QiwgMHg4NSwgdGNjX29mZik7ICAgICAgICAgICAgIC8qIG1vdiBl YXgsIGR3b3JkIHB0ciBbcmJwIC0gdGNjX29mZl0gKi8KIAlFTUlUMygweDgzLCAweEY4LCBNQVhf VEFJTF9DQUxMX0NOVCk7ICAgICAgICAgLyogY21wIGVheCwgTUFYX1RBSUxfQ0FMTF9DTlQgKi8K IAogCW9mZnNldCA9IGN0eC0+dGFpbF9jYWxsX2RpcmVjdF9sYWJlbCAtIChwcm9nICsgMiAtIHN0 YXJ0KTsKLQlFTUlUMihYODZfSkEsIG9mZnNldCk7ICAgICAgICAgICAgICAgICAgICAgICAgLyog amEgb3V0ICovCisJRU1JVDIoWDg2X0pBRSwgb2Zmc2V0KTsgICAgICAgICAgICAgICAgICAgICAg IC8qIGphZSBvdXQgKi8KIAlFTUlUMygweDgzLCAweEMwLCAweDAxKTsgICAgICAgICAgICAgICAg ICAgICAgLyogYWRkIGVheCwgMSAqLwogCUVNSVQyX29mZjMyKDB4ODksIDB4ODUsIHRjY19vZmYp OyAgICAgICAgICAgICAvKiBtb3YgZHdvcmQgcHRyIFtyYnAgLSB0Y2Nfb2ZmXSwgZWF4ICovCiAK ZGlmZiAtLWdpdCBhL2FyY2gveDg2L25ldC9icGZfaml0X2NvbXAzMi5jIGIvYXJjaC94ODYvbmV0 L2JwZl9qaXRfY29tcDMyLmMKaW5kZXggZGE5YjdjZmE0NjMyOS4uNDI5YTg5YzU0NjhiNSAxMDA2 NDQKLS0tIGEvYXJjaC94ODYvbmV0L2JwZl9qaXRfY29tcDMyLmMKKysrIGIvYXJjaC94ODYvbmV0 L2JwZl9qaXRfY29tcDMyLmMKQEAgLTEzMjMsNyArMTMyMyw3IEBAIHN0YXRpYyB2b2lkIGVtaXRf YnBmX3RhaWxfY2FsbCh1OCAqKnBwcm9nLCB1OCAqaXApCiAJRU1JVDIoSUEzMl9KQkUsIGptcF9s YWJlbChqbXBfbGFiZWwxLCAyKSk7CiAKIAkvKgotCSAqIGlmICh0YWlsX2NhbGxfY250ID4gTUFY X1RBSUxfQ0FMTF9DTlQpCisJICogaWYgKHRhaWxfY2FsbF9jbnQrKyA+PSBNQVhfVEFJTF9DQUxM X0NOVCkKIAkgKiAgICAgZ290byBvdXQ7CiAJICovCiAJbG8gPSAodTMyKU1BWF9UQUlMX0NBTExf Q05UOwpAQCAtMTMzNyw3ICsxMzM3LDcgQEAgc3RhdGljIHZvaWQgZW1pdF9icGZfdGFpbF9jYWxs KHU4ICoqcHByb2csIHU4ICppcCkKIAkvKiBjbXAgZWN4LGxvICovCiAJRU1JVDMoMHg4MywgYWRk XzFyZWcoMHhGOCwgSUEzMl9FQ1gpLCBsbyk7CiAKLQkvKiBqYSBvdXQgKi8KKwkvKiBqYWUgb3V0 ICovCiAJRU1JVDIoSUEzMl9KQUUsIGptcF9sYWJlbChqbXBfbGFiZWwxLCAyKSk7CiAKIAkvKiBh ZGQgZWF4LDB4MSAqLwpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9icGYuaCBiL2luY2x1ZGUv bGludXgvYnBmLmgKaW5kZXggNzU1ZjM4ZTg5M2JlMS4uOWYyMGIwZjUzOWY3OCAxMDA2NDQKLS0t IGEvaW5jbHVkZS9saW51eC9icGYuaAorKysgYi9pbmNsdWRlL2xpbnV4L2JwZi5oCkBAIC0xMDgy LDcgKzEwODIsNyBAQCBzdHJ1Y3QgYnBmX2FycmF5IHsKIH07CiAKICNkZWZpbmUgQlBGX0NPTVBM RVhJVFlfTElNSVRfSU5TTlMgICAgICAxMDAwMDAwIC8qIHllcy4gMU0gaW5zbnMgKi8KLSNkZWZp bmUgTUFYX1RBSUxfQ0FMTF9DTlQgMzIKKyNkZWZpbmUgTUFYX1RBSUxfQ0FMTF9DTlQgMzMKIAog I2RlZmluZSBCUEZfRl9BQ0NFU1NfTUFTSwkoQlBGX0ZfUkRPTkxZIHwJCVwKIAkJCQkgQlBGX0Zf UkRPTkxZX1BST0cgfAlcCmRpZmYgLS1naXQgYS9pbmNsdWRlL3VhcGkvbGludXgvYnBmLmggYi9p bmNsdWRlL3VhcGkvbGludXgvYnBmLmgKaW5kZXggYmE1YWYxNWUyNWY1Yy4uYjEyY2ZjZWRkYjZl OSAxMDA2NDQKLS0tIGEvaW5jbHVkZS91YXBpL2xpbnV4L2JwZi5oCisrKyBiL2luY2x1ZGUvdWFw aS9saW51eC9icGYuaApAQCAtMTc0NCw3ICsxNzQ0LDcgQEAgdW5pb24gYnBmX2F0dHIgewogICog CQlpZiB0aGUgbWF4aW11bSBudW1iZXIgb2YgdGFpbCBjYWxscyBoYXMgYmVlbiByZWFjaGVkIGZv ciB0aGlzCiAgKiAJCWNoYWluIG9mIHByb2dyYW1zLiBUaGlzIGxpbWl0IGlzIGRlZmluZWQgaW4g dGhlIGtlcm5lbCBieSB0aGUKICAqIAkJbWFjcm8gKipNQVhfVEFJTF9DQUxMX0NOVCoqIChub3Qg YWNjZXNzaWJsZSB0byB1c2VyIHNwYWNlKSwKLSAqIAkJd2hpY2ggaXMgY3VycmVudGx5IHNldCB0 byAzMi4KKyAqCQl3aGljaCBpcyBjdXJyZW50bHkgc2V0IHRvIDMzLgogICogCVJldHVybgogICog CQkwIG9uIHN1Y2Nlc3MsIG9yIGEgbmVnYXRpdmUgZXJyb3IgaW4gY2FzZSBvZiBmYWlsdXJlLgog ICoKZGlmZiAtLWdpdCBhL2tlcm5lbC9icGYvY29yZS5jIGIva2VybmVsL2JwZi9jb3JlLmMKaW5k ZXggMjQwNWUzOWQ4MDBmZS4uYjUyZGM4NDVlY2VhMyAxMDA2NDQKLS0tIGEva2VybmVsL2JwZi9j b3JlLmMKKysrIGIva2VybmVsL2JwZi9jb3JlLmMKQEAgLTE1NzQsNyArMTU3NCw4IEBAIHN0YXRp YyB1NjQgX19fYnBmX3Byb2dfcnVuKHU2NCAqcmVncywgY29uc3Qgc3RydWN0IGJwZl9pbnNuICpp bnNuKQogCiAJCWlmICh1bmxpa2VseShpbmRleCA+PSBhcnJheS0+bWFwLm1heF9lbnRyaWVzKSkK IAkJCWdvdG8gb3V0OwotCQlpZiAodW5saWtlbHkodGFpbF9jYWxsX2NudCA+IE1BWF9UQUlMX0NB TExfQ05UKSkKKworCQlpZiAodW5saWtlbHkodGFpbF9jYWxsX2NudCA+PSBNQVhfVEFJTF9DQUxM X0NOVCkpCiAJCQlnb3RvIG91dDsKIAogCQl0YWlsX2NhbGxfY250Kys7CmRpZmYgLS1naXQgYS9s aWIvdGVzdF9icGYuYyBiL2xpYi90ZXN0X2JwZi5jCmluZGV4IGFkYWUzOTU2NzI2NGYuLjBjNWNi MmQ2NDM2YTQgMTAwNjQ0Ci0tLSBhL2xpYi90ZXN0X2JwZi5jCisrKyBiL2xpYi90ZXN0X2JwZi5j CkBAIC0xNDY4Myw3ICsxNDY4Myw3IEBAIHN0YXRpYyBzdHJ1Y3QgdGFpbF9jYWxsX3Rlc3QgdGFp bF9jYWxsX3Rlc3RzW10gPSB7CiAJCQlCUEZfRVhJVF9JTlNOKCksCiAJCX0sCiAJCS5mbGFncyA9 IEZMQUdfTkVFRF9TVEFURSB8IEZMQUdfUkVTVUxUX0lOX1NUQVRFLAotCQkucmVzdWx0ID0gKE1B WF9UQUlMX0NBTExfQ05UICsgMSArIDEpICogTUFYX1RFU1RSVU5TLAorCQkucmVzdWx0ID0gKE1B WF9UQUlMX0NBTExfQ05UICsgMSkgKiBNQVhfVEVTVFJVTlMsCiAJfSwKIAl7CiAJCSJUYWlsIGNh bGwgY291bnQgcHJlc2VydmVkIGFjcm9zcyBmdW5jdGlvbiBjYWxscyIsCkBAIC0xNDcwNSw3ICsx NDcwNSw3IEBAIHN0YXRpYyBzdHJ1Y3QgdGFpbF9jYWxsX3Rlc3QgdGFpbF9jYWxsX3Rlc3RzW10g PSB7CiAJCX0sCiAJCS5zdGFja19kZXB0aCA9IDgsCiAJCS5mbGFncyA9IEZMQUdfTkVFRF9TVEFU RSB8IEZMQUdfUkVTVUxUX0lOX1NUQVRFLAotCQkucmVzdWx0ID0gKE1BWF9UQUlMX0NBTExfQ05U ICsgMSArIDEpICogTUFYX1RFU1RSVU5TLAorCQkucmVzdWx0ID0gKE1BWF9UQUlMX0NBTExfQ05U ICsgMSkgKiBNQVhfVEVTVFJVTlMsCiAJfSwKIAl7CiAJCSJUYWlsIGNhbGwgZXJyb3IgcGF0aCwg TlVMTCB0YXJnZXQiLApkaWZmIC0tZ2l0IGEvdG9vbHMvaW5jbHVkZS91YXBpL2xpbnV4L2JwZi5o IGIvdG9vbHMvaW5jbHVkZS91YXBpL2xpbnV4L2JwZi5oCmluZGV4IGJhNWFmMTVlMjVmNWMuLmIx MmNmY2VkZGI2ZTkgMTAwNjQ0Ci0tLSBhL3Rvb2xzL2luY2x1ZGUvdWFwaS9saW51eC9icGYuaAor KysgYi90b29scy9pbmNsdWRlL3VhcGkvbGludXgvYnBmLmgKQEAgLTE3NDQsNyArMTc0NCw3IEBA IHVuaW9uIGJwZl9hdHRyIHsKICAqIAkJaWYgdGhlIG1heGltdW0gbnVtYmVyIG9mIHRhaWwgY2Fs bHMgaGFzIGJlZW4gcmVhY2hlZCBmb3IgdGhpcwogICogCQljaGFpbiBvZiBwcm9ncmFtcy4gVGhp cyBsaW1pdCBpcyBkZWZpbmVkIGluIHRoZSBrZXJuZWwgYnkgdGhlCiAgKiAJCW1hY3JvICoqTUFY X1RBSUxfQ0FMTF9DTlQqKiAobm90IGFjY2Vzc2libGUgdG8gdXNlciBzcGFjZSksCi0gKiAJCXdo aWNoIGlzIGN1cnJlbnRseSBzZXQgdG8gMzIuCisgKgkJd2hpY2ggaXMgY3VycmVudGx5IHNldCB0 byAzMy4KICAqIAlSZXR1cm4KICAqIAkJMCBvbiBzdWNjZXNzLCBvciBhIG5lZ2F0aXZlIGVycm9y IGluIGNhc2Ugb2YgZmFpbHVyZS4KICAqCi0tIAoyLjM0LjEKCgpfX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1yaXNjdiBtYWlsaW5nIGxpc3QKbGlu dXgtcmlzY3ZAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9t YWlsbWFuL2xpc3RpbmZvL2xpbnV4LXJpc2N2Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C427BC433F5 for ; Tue, 18 Jan 2022 02:22:05 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JdCG40X4Sz3bZg for ; Tue, 18 Jan 2022 13:22:04 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=cyhp/T4A; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=139.178.84.217; helo=dfw.source.kernel.org; envelope-from=sashal@kernel.org; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=cyhp/T4A; dkim-atps=neutral Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JdCFG4P91z2yP9 for ; Tue, 18 Jan 2022 13:21:22 +1100 (AEDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 90D82612D1; Tue, 18 Jan 2022 02:21:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5801EC36AE3; Tue, 18 Jan 2022 02:21:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1642472479; bh=wc9zOoYdRUlpL/9CmY8TpP2Rxm80a4UiTI/3souVWKU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cyhp/T4AFFnlFRcOCTcM7GSCyzG6t/NkFnKVdzaew24NoFOqlNp22CNB0R7HzH16Q oTzBY27j0PEAy/S9IQfgEZdm5enDYf+EnrIzP53xtSPObMO+smDGvEAavtM3W1dUGg 4eQ5n3lt5EfnHv/ykc/xTUxLWAGEKDxtQl4f6BUFQUp0h1quMF58DZsfY0ok+Bj1k+ JmUgkKLUaTyDiggvsNvYcLJgIidG8KnL2lznL6lFHPKe+OTBYKZSIcwh/UNXVqTese yb6vI5BIHmJ1lH+bdFZ7o33hHGLTzK16UXr1WgDBWPuX8rln6tJphJSVA3n1kKl+zE q7XxWgE69qYow== From: Sasha Levin To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Subject: [PATCH AUTOSEL 5.16 026/217] bpf: Change value of MAX_TAIL_CALL_CNT from 32 to 33 Date: Mon, 17 Jan 2022 21:16:29 -0500 Message-Id: <20220118021940.1942199-26-sashal@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220118021940.1942199-1-sashal@kernel.org> References: <20220118021940.1942199-1-sashal@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-stable: review X-Patchwork-Hint: Ignore Content-Transfer-Encoding: 8bit X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: joannekoong@fb.com, revest@chromium.org, davemarchevsky@fb.com, zlim.lnx@gmail.com, dave.hansen@linux.intel.com, ast@kernel.org, andrii@kernel.org, sparclinux@vger.kernel.org, illusionist.neo@gmail.com, Tiezhu Yang , will@kernel.org, linux-riscv@lists.infradead.org, Sasha Levin , linux-s390@vger.kernel.org, Ilya Leoshkevich , Daniel Borkmann , x86@kernel.org, john.fastabend@gmail.com, linux@armlinux.org.uk, mingo@redhat.com, catalin.marinas@arm.com, naveen.n.rao@linux.ibm.com, yhs@fb.com, borntraeger@linux.ibm.com, xi.wang@gmail.com, gor@linux.ibm.com, joe@cilium.io, aou@eecs.berkeley.edu, Johan Almbladh , luke.r.nels@gmail.com, hca@linux.ibm.com, brouer@redhat.com, linux-mips@vger.kernel.org, bp@alien8.de, liuhangbin@gmail.com, paul.walmsley@sifive.com, tglx@linutronix.de, linux-arm-kernel@lists.infradead.org, tsbogend@alpha.franken.de, paulburton@kernel.org, yoshfuji@linux-ipv6.org, netdev@vger.kernel.org, dsahern@kernel.org, udknight@gmail.com, =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , palmer@dabbelt.com, bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, davem@davemloft.net Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: Tiezhu Yang [ Upstream commit ebf7f6f0a6cdcc17a3da52b81e4b3a98c4005028 ] In the current code, the actual max tail call count is 33 which is greater than MAX_TAIL_CALL_CNT (defined as 32). The actual limit is not consistent with the meaning of MAX_TAIL_CALL_CNT and thus confusing at first glance. We can see the historical evolution from commit 04fd61ab36ec ("bpf: allow bpf programs to tail-call other bpf programs") and commit f9dabe016b63 ("bpf: Undo off-by-one in interpreter tail call count limit"). In order to avoid changing existing behavior, the actual limit is 33 now, this is reasonable. After commit 874be05f525e ("bpf, tests: Add tail call test suite"), we can see there exists failed testcase. On all archs when CONFIG_BPF_JIT_ALWAYS_ON is not set: # echo 0 > /proc/sys/net/core/bpf_jit_enable # modprobe test_bpf # dmesg | grep -w FAIL Tail call error path, max count reached jited:0 ret 34 != 33 FAIL On some archs: # echo 1 > /proc/sys/net/core/bpf_jit_enable # modprobe test_bpf # dmesg | grep -w FAIL Tail call error path, max count reached jited:1 ret 34 != 33 FAIL Although the above failed testcase has been fixed in commit 18935a72eb25 ("bpf/tests: Fix error in tail call limit tests"), it would still be good to change the value of MAX_TAIL_CALL_CNT from 32 to 33 to make the code more readable. The 32-bit x86 JIT was using a limit of 32, just fix the wrong comments and limit to 33 tail calls as the constant MAX_TAIL_CALL_CNT updated. For the mips64 JIT, use "ori" instead of "addiu" as suggested by Johan Almbladh. For the riscv JIT, use RV_REG_TCC directly to save one register move as suggested by Björn Töpel. For the other implementations, no function changes, it does not change the current limit 33, the new value of MAX_TAIL_CALL_CNT can reflect the actual max tail call count, the related tail call testcases in test_bpf module and selftests can work well for the interpreter and the JIT. Here are the test results on x86_64: # uname -m x86_64 # echo 0 > /proc/sys/net/core/bpf_jit_enable # modprobe test_bpf test_suite=test_tail_calls # dmesg | tail -1 test_bpf: test_tail_calls: Summary: 8 PASSED, 0 FAILED, [0/8 JIT'ed] # rmmod test_bpf # echo 1 > /proc/sys/net/core/bpf_jit_enable # modprobe test_bpf test_suite=test_tail_calls # dmesg | tail -1 test_bpf: test_tail_calls: Summary: 8 PASSED, 0 FAILED, [8/8 JIT'ed] # rmmod test_bpf # ./test_progs -t tailcalls #142 tailcalls:OK Summary: 1/11 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Tiezhu Yang Signed-off-by: Daniel Borkmann Tested-by: Johan Almbladh Tested-by: Ilya Leoshkevich Acked-by: Björn Töpel Acked-by: Johan Almbladh Acked-by: Ilya Leoshkevich Link: https://lore.kernel.org/bpf/1636075800-3264-1-git-send-email-yangtiezhu@loongson.cn Signed-off-by: Sasha Levin --- arch/arm/net/bpf_jit_32.c | 5 +++-- arch/arm64/net/bpf_jit_comp.c | 5 +++-- arch/mips/net/bpf_jit_comp32.c | 3 +-- arch/mips/net/bpf_jit_comp64.c | 2 +- arch/powerpc/net/bpf_jit_comp32.c | 4 ++-- arch/powerpc/net/bpf_jit_comp64.c | 4 ++-- arch/riscv/net/bpf_jit_comp32.c | 6 ++---- arch/riscv/net/bpf_jit_comp64.c | 7 +++---- arch/s390/net/bpf_jit_comp.c | 6 +++--- arch/sparc/net/bpf_jit_comp_64.c | 2 +- arch/x86/net/bpf_jit_comp.c | 10 +++++----- arch/x86/net/bpf_jit_comp32.c | 4 ++-- include/linux/bpf.h | 2 +- include/uapi/linux/bpf.h | 2 +- kernel/bpf/core.c | 3 ++- lib/test_bpf.c | 4 ++-- tools/include/uapi/linux/bpf.h | 2 +- 17 files changed, 35 insertions(+), 36 deletions(-) diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c index eeb6dc0ecf463..e59b41e9ab0c1 100644 --- a/arch/arm/net/bpf_jit_32.c +++ b/arch/arm/net/bpf_jit_32.c @@ -1199,7 +1199,8 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx) /* tmp2[0] = array, tmp2[1] = index */ - /* if (tail_call_cnt > MAX_TAIL_CALL_CNT) + /* + * if (tail_call_cnt >= MAX_TAIL_CALL_CNT) * goto out; * tail_call_cnt++; */ @@ -1208,7 +1209,7 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx) tc = arm_bpf_get_reg64(tcc, tmp, ctx); emit(ARM_CMP_I(tc[0], hi), ctx); _emit(ARM_COND_EQ, ARM_CMP_I(tc[1], lo), ctx); - _emit(ARM_COND_HI, ARM_B(jmp_offset), ctx); + _emit(ARM_COND_CS, ARM_B(jmp_offset), ctx); emit(ARM_ADDS_I(tc[1], tc[1], 1), ctx); emit(ARM_ADC_I(tc[0], tc[0], 0), ctx); arm_bpf_put_reg64(tcc, tmp, ctx); diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index 3a8a7140a9bfb..356fb2116c6b2 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -287,13 +287,14 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx) emit(A64_CMP(0, r3, tmp), ctx); emit(A64_B_(A64_COND_CS, jmp_offset), ctx); - /* if (tail_call_cnt > MAX_TAIL_CALL_CNT) + /* + * if (tail_call_cnt >= MAX_TAIL_CALL_CNT) * goto out; * tail_call_cnt++; */ emit_a64_mov_i64(tmp, MAX_TAIL_CALL_CNT, ctx); emit(A64_CMP(1, tcc, tmp), ctx); - emit(A64_B_(A64_COND_HI, jmp_offset), ctx); + emit(A64_B_(A64_COND_CS, jmp_offset), ctx); emit(A64_ADD_I(1, tcc, tcc, 1), ctx); /* prog = array->ptrs[index]; diff --git a/arch/mips/net/bpf_jit_comp32.c b/arch/mips/net/bpf_jit_comp32.c index bd996ede12f8e..044b11b65bcac 100644 --- a/arch/mips/net/bpf_jit_comp32.c +++ b/arch/mips/net/bpf_jit_comp32.c @@ -1381,8 +1381,7 @@ void build_prologue(struct jit_context *ctx) * 16-byte area in the parent's stack frame. On a tail call, the * calling function jumps into the prologue after these instructions. */ - emit(ctx, ori, MIPS_R_T9, MIPS_R_ZERO, - min(MAX_TAIL_CALL_CNT + 1, 0xffff)); + emit(ctx, ori, MIPS_R_T9, MIPS_R_ZERO, min(MAX_TAIL_CALL_CNT, 0xffff)); emit(ctx, sw, MIPS_R_T9, 0, MIPS_R_SP); /* diff --git a/arch/mips/net/bpf_jit_comp64.c b/arch/mips/net/bpf_jit_comp64.c index 815ade7242278..6475828ffb36d 100644 --- a/arch/mips/net/bpf_jit_comp64.c +++ b/arch/mips/net/bpf_jit_comp64.c @@ -552,7 +552,7 @@ void build_prologue(struct jit_context *ctx) * On a tail call, the calling function jumps into the prologue * after this instruction. */ - emit(ctx, addiu, tc, MIPS_R_ZERO, min(MAX_TAIL_CALL_CNT + 1, 0xffff)); + emit(ctx, ori, tc, MIPS_R_ZERO, min(MAX_TAIL_CALL_CNT, 0xffff)); /* === Entry-point for tail calls === */ diff --git a/arch/powerpc/net/bpf_jit_comp32.c b/arch/powerpc/net/bpf_jit_comp32.c index 0da31d41d4131..8a4faa05f9e41 100644 --- a/arch/powerpc/net/bpf_jit_comp32.c +++ b/arch/powerpc/net/bpf_jit_comp32.c @@ -221,13 +221,13 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o PPC_BCC(COND_GE, out); /* - * if (tail_call_cnt > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt >= MAX_TAIL_CALL_CNT) * goto out; */ EMIT(PPC_RAW_CMPLWI(_R0, MAX_TAIL_CALL_CNT)); /* tail_call_cnt++; */ EMIT(PPC_RAW_ADDIC(_R0, _R0, 1)); - PPC_BCC(COND_GT, out); + PPC_BCC(COND_GE, out); /* prog = array->ptrs[index]; */ EMIT(PPC_RAW_RLWINM(_R3, b2p_index, 2, 0, 29)); diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c index 8b5157ccfebae..8571aafcc9e1e 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -228,12 +228,12 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o PPC_BCC(COND_GE, out); /* - * if (tail_call_cnt > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt >= MAX_TAIL_CALL_CNT) * goto out; */ PPC_BPF_LL(b2p[TMP_REG_1], 1, bpf_jit_stack_tailcallcnt(ctx)); EMIT(PPC_RAW_CMPLWI(b2p[TMP_REG_1], MAX_TAIL_CALL_CNT)); - PPC_BCC(COND_GT, out); + PPC_BCC(COND_GE, out); /* * tail_call_cnt++; diff --git a/arch/riscv/net/bpf_jit_comp32.c b/arch/riscv/net/bpf_jit_comp32.c index e6497424cbf60..529a83b85c1c9 100644 --- a/arch/riscv/net/bpf_jit_comp32.c +++ b/arch/riscv/net/bpf_jit_comp32.c @@ -799,11 +799,10 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx) emit_bcc(BPF_JGE, lo(idx_reg), RV_REG_T1, off, ctx); /* - * temp_tcc = tcc - 1; - * if (tcc < 0) + * if (--tcc < 0) * goto out; */ - emit(rv_addi(RV_REG_T1, RV_REG_TCC, -1), ctx); + emit(rv_addi(RV_REG_TCC, RV_REG_TCC, -1), ctx); off = ninsns_rvoff(tc_ninsn - (ctx->ninsns - start_insn)); emit_bcc(BPF_JSLT, RV_REG_TCC, RV_REG_ZERO, off, ctx); @@ -829,7 +828,6 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx) if (is_12b_check(off, insn)) return -1; emit(rv_lw(RV_REG_T0, off, RV_REG_T0), ctx); - emit(rv_addi(RV_REG_TCC, RV_REG_T1, 0), ctx); /* Epilogue jumps to *(t0 + 4). */ __build_epilogue(true, ctx); return 0; diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c index f2a779c7e225d..603630b6f3c5b 100644 --- a/arch/riscv/net/bpf_jit_comp64.c +++ b/arch/riscv/net/bpf_jit_comp64.c @@ -327,12 +327,12 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx) off = ninsns_rvoff(tc_ninsn - (ctx->ninsns - start_insn)); emit_branch(BPF_JGE, RV_REG_A2, RV_REG_T1, off, ctx); - /* if (TCC-- < 0) + /* if (--TCC < 0) * goto out; */ - emit_addi(RV_REG_T1, tcc, -1, ctx); + emit_addi(RV_REG_TCC, tcc, -1, ctx); off = ninsns_rvoff(tc_ninsn - (ctx->ninsns - start_insn)); - emit_branch(BPF_JSLT, tcc, RV_REG_ZERO, off, ctx); + emit_branch(BPF_JSLT, RV_REG_TCC, RV_REG_ZERO, off, ctx); /* prog = array->ptrs[index]; * if (!prog) @@ -352,7 +352,6 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx) if (is_12b_check(off, insn)) return -1; emit_ld(RV_REG_T3, off, RV_REG_T2, ctx); - emit_mv(RV_REG_TCC, RV_REG_T1, ctx); __build_epilogue(true, ctx); return 0; } diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c index 233cc9bcd6527..9ff2bd83aad70 100644 --- a/arch/s390/net/bpf_jit_comp.c +++ b/arch/s390/net/bpf_jit_comp.c @@ -1369,7 +1369,7 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, jit->prg); /* - * if (tail_call_cnt++ > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT) * goto out; */ @@ -1381,9 +1381,9 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, EMIT4_IMM(0xa7080000, REG_W0, 1); /* laal %w1,%w0,off(%r15) */ EMIT6_DISP_LH(0xeb000000, 0x00fa, REG_W1, REG_W0, REG_15, off); - /* clij %w1,MAX_TAIL_CALL_CNT,0x2,out */ + /* clij %w1,MAX_TAIL_CALL_CNT-1,0x2,out */ patch_2_clij = jit->prg; - EMIT6_PCREL_RIEC(0xec000000, 0x007f, REG_W1, MAX_TAIL_CALL_CNT, + EMIT6_PCREL_RIEC(0xec000000, 0x007f, REG_W1, MAX_TAIL_CALL_CNT - 1, 2, jit->prg); /* diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c index 9a2f20cbd48b7..0bfe1c72a0c9e 100644 --- a/arch/sparc/net/bpf_jit_comp_64.c +++ b/arch/sparc/net/bpf_jit_comp_64.c @@ -867,7 +867,7 @@ static void emit_tail_call(struct jit_ctx *ctx) emit(LD32 | IMMED | RS1(SP) | S13(off) | RD(tmp), ctx); emit_cmpi(tmp, MAX_TAIL_CALL_CNT, ctx); #define OFFSET2 13 - emit_branch(BGU, ctx->idx, ctx->idx + OFFSET2, ctx); + emit_branch(BGEU, ctx->idx, ctx->idx + OFFSET2, ctx); emit_nop(ctx); emit_alu_K(ADD, tmp, 1, ctx); diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index bafe36e69227d..b87d98efd2240 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -412,7 +412,7 @@ static void emit_indirect_jump(u8 **pprog, int reg, u8 *ip) * ... bpf_tail_call(void *ctx, struct bpf_array *array, u64 index) ... * if (index >= array->map.max_entries) * goto out; - * if (++tail_call_cnt > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT) * goto out; * prog = array->ptrs[index]; * if (prog == NULL) @@ -446,14 +446,14 @@ static void emit_bpf_tail_call_indirect(u8 **pprog, bool *callee_regs_used, EMIT2(X86_JBE, offset); /* jbe out */ /* - * if (tail_call_cnt > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT) * goto out; */ EMIT2_off32(0x8B, 0x85, tcc_off); /* mov eax, dword ptr [rbp - tcc_off] */ EMIT3(0x83, 0xF8, MAX_TAIL_CALL_CNT); /* cmp eax, MAX_TAIL_CALL_CNT */ offset = ctx->tail_call_indirect_label - (prog + 2 - start); - EMIT2(X86_JA, offset); /* ja out */ + EMIT2(X86_JAE, offset); /* jae out */ EMIT3(0x83, 0xC0, 0x01); /* add eax, 1 */ EMIT2_off32(0x89, 0x85, tcc_off); /* mov dword ptr [rbp - tcc_off], eax */ @@ -504,14 +504,14 @@ static void emit_bpf_tail_call_direct(struct bpf_jit_poke_descriptor *poke, int offset; /* - * if (tail_call_cnt > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT) * goto out; */ EMIT2_off32(0x8B, 0x85, tcc_off); /* mov eax, dword ptr [rbp - tcc_off] */ EMIT3(0x83, 0xF8, MAX_TAIL_CALL_CNT); /* cmp eax, MAX_TAIL_CALL_CNT */ offset = ctx->tail_call_direct_label - (prog + 2 - start); - EMIT2(X86_JA, offset); /* ja out */ + EMIT2(X86_JAE, offset); /* jae out */ EMIT3(0x83, 0xC0, 0x01); /* add eax, 1 */ EMIT2_off32(0x89, 0x85, tcc_off); /* mov dword ptr [rbp - tcc_off], eax */ diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c index da9b7cfa46329..429a89c5468b5 100644 --- a/arch/x86/net/bpf_jit_comp32.c +++ b/arch/x86/net/bpf_jit_comp32.c @@ -1323,7 +1323,7 @@ static void emit_bpf_tail_call(u8 **pprog, u8 *ip) EMIT2(IA32_JBE, jmp_label(jmp_label1, 2)); /* - * if (tail_call_cnt > MAX_TAIL_CALL_CNT) + * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT) * goto out; */ lo = (u32)MAX_TAIL_CALL_CNT; @@ -1337,7 +1337,7 @@ static void emit_bpf_tail_call(u8 **pprog, u8 *ip) /* cmp ecx,lo */ EMIT3(0x83, add_1reg(0xF8, IA32_ECX), lo); - /* ja out */ + /* jae out */ EMIT2(IA32_JAE, jmp_label(jmp_label1, 2)); /* add eax,0x1 */ diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 755f38e893be1..9f20b0f539f78 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1082,7 +1082,7 @@ struct bpf_array { }; #define BPF_COMPLEXITY_LIMIT_INSNS 1000000 /* yes. 1M insns */ -#define MAX_TAIL_CALL_CNT 32 +#define MAX_TAIL_CALL_CNT 33 #define BPF_F_ACCESS_MASK (BPF_F_RDONLY | \ BPF_F_RDONLY_PROG | \ diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index ba5af15e25f5c..b12cfceddb6e9 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1744,7 +1744,7 @@ union bpf_attr { * if the maximum number of tail calls has been reached for this * chain of programs. This limit is defined in the kernel by the * macro **MAX_TAIL_CALL_CNT** (not accessible to user space), - * which is currently set to 32. + * which is currently set to 33. * Return * 0 on success, or a negative error in case of failure. * diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 2405e39d800fe..b52dc845ecea3 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -1574,7 +1574,8 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn) if (unlikely(index >= array->map.max_entries)) goto out; - if (unlikely(tail_call_cnt > MAX_TAIL_CALL_CNT)) + + if (unlikely(tail_call_cnt >= MAX_TAIL_CALL_CNT)) goto out; tail_call_cnt++; diff --git a/lib/test_bpf.c b/lib/test_bpf.c index adae39567264f..0c5cb2d6436a4 100644 --- a/lib/test_bpf.c +++ b/lib/test_bpf.c @@ -14683,7 +14683,7 @@ static struct tail_call_test tail_call_tests[] = { BPF_EXIT_INSN(), }, .flags = FLAG_NEED_STATE | FLAG_RESULT_IN_STATE, - .result = (MAX_TAIL_CALL_CNT + 1 + 1) * MAX_TESTRUNS, + .result = (MAX_TAIL_CALL_CNT + 1) * MAX_TESTRUNS, }, { "Tail call count preserved across function calls", @@ -14705,7 +14705,7 @@ static struct tail_call_test tail_call_tests[] = { }, .stack_depth = 8, .flags = FLAG_NEED_STATE | FLAG_RESULT_IN_STATE, - .result = (MAX_TAIL_CALL_CNT + 1 + 1) * MAX_TESTRUNS, + .result = (MAX_TAIL_CALL_CNT + 1) * MAX_TESTRUNS, }, { "Tail call error path, NULL target", diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index ba5af15e25f5c..b12cfceddb6e9 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1744,7 +1744,7 @@ union bpf_attr { * if the maximum number of tail calls has been reached for this * chain of programs. This limit is defined in the kernel by the * macro **MAX_TAIL_CALL_CNT** (not accessible to user space), - * which is currently set to 32. + * which is currently set to 33. * Return * 0 on success, or a negative error in case of failure. * -- 2.34.1