From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B2B72D130A for ; Thu, 11 Sep 2025 14:58:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757602701; cv=none; b=ZhDjZIPPKc7O7ZRmS3WQCK5qDsFWLleFNkBn9dqxgCFjyZuNA+oVUFWd89q1ptGPnh/1PJweZLh46HW7tM/Qn563imB7lSReehkfcPor2LkFT1N6fSTne49G2kGzCKUkMZ1a+zQmKHVwxkFAAkjGFlOhasqqwapyGpdg0Ir3GRs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757602701; c=relaxed/simple; bh=lrvQJ3KJ/nEa/FD+Xd4qXpSrRmf65ggsNw6Yj6Gh0gg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jx1swr/BBBMEKWyqfK9MhWoEyiVfVV0v3rcSq0KsiZHHqds5KdAmpL5/g6WPO8RD25T/z2qEBz2B7kxKaE2LKJUf6BxjvocaPn5+NveO6JodL9yArOWwvAmFl8l5ccv2bSFwG233mkhiIktI0anwKy+0AFO0iTVS3LpbifCN9Pg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PkHLFGiv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PkHLFGiv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2EFFCC4CEF0; Thu, 11 Sep 2025 14:58:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1757602699; bh=lrvQJ3KJ/nEa/FD+Xd4qXpSrRmf65ggsNw6Yj6Gh0gg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PkHLFGivHX6daScd93jGA6netDkw5O/EaJqS7D6dYAcnj5fGSnKTHecFwKQBhwLcE XEV3sOiSQvmvWJRM4ZBVLHaml4yrMFNKx27/oZgk4c5kel+lRvP19NvoClGHpxTTKG 1gUvNTth6dSGT1qF8W4vvrApJfMsnvT3+fo31bXA3L8SGa4+W93oSdMUS72eyHss0c kRbH8QSGSnqn3OJRM8BpChZMQE9GUBlZdSWpaMBneosCdAUANQ3ykqAGgDxf3fVPaq Bi22ZBdu2CUw18zoqPZivJTF0Tzwv9tkCgUu8HMeiF3M/cYMSRhCgIoB0fb+ASjDrL 25ZzXN9gHHw9Q== From: Puranjay Mohan To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Puranjay Mohan , Xu Kuohai , Catalin Marinas , Will Deacon , Kumar Kartikeya Dwivedi , bpf@vger.kernel.org Cc: Xu Kuohai Subject: [PATCH bpf-next v7 1/6] bpf: arm64: simplify exception table handling Date: Thu, 11 Sep 2025 14:58:00 +0000 Message-ID: <20250911145808.58042-2-puranjay@kernel.org> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20250911145808.58042-1-puranjay@kernel.org> References: <20250911145808.58042-1-puranjay@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit BPF loads with BPF_PROBE_MEM(SX) can load from unsafe pointers and the JIT adds an exception table entry for the JITed instruction which allows the exeption handler to set the destination register of the load to zero and continue execution from the next instruction. As all arm64 instructions are AARCH64_INSN_SIZE size, the exception handler can just increment the pc by AARCH64_INSN_SIZE without needing the exact address of the instruction following the the faulting instruction. Simplify the exception table usage in arm64 JIT by only saving the destination register in ex->fixup and drop everything related to the fixup_offset. The fault handler is modified to add AARCH64_INSN_SIZE to the pc. Signed-off-by: Puranjay Mohan Acked-by: Yonghong Song Acked-by: Kumar Kartikeya Dwivedi Acked-by: Xu Kuohai --- arch/arm64/net/bpf_jit_comp.c | 25 +++---------------------- 1 file changed, 3 insertions(+), 22 deletions(-) diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index f0b1cb2c3bc4..e6d1fdc1e6f5 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -1066,19 +1066,18 @@ static void build_epilogue(struct jit_ctx *ctx, bool was_classic) emit(A64_RET(A64_LR), ctx); } -#define BPF_FIXUP_OFFSET_MASK GENMASK(26, 0) #define BPF_FIXUP_REG_MASK GENMASK(31, 27) #define DONT_CLEAR 5 /* Unused ARM64 register from BPF's POV */ bool ex_handler_bpf(const struct exception_table_entry *ex, struct pt_regs *regs) { - off_t offset = FIELD_GET(BPF_FIXUP_OFFSET_MASK, ex->fixup); int dst_reg = FIELD_GET(BPF_FIXUP_REG_MASK, ex->fixup); if (dst_reg != DONT_CLEAR) regs->regs[dst_reg] = 0; - regs->pc = (unsigned long)&ex->fixup - offset; + /* Skip the faulting instruction */ + regs->pc += AARCH64_INSN_SIZE; return true; } @@ -1088,7 +1087,6 @@ static int add_exception_handler(const struct bpf_insn *insn, int dst_reg) { off_t ins_offset; - off_t fixup_offset; unsigned long pc; struct exception_table_entry *ex; @@ -1119,22 +1117,6 @@ static int add_exception_handler(const struct bpf_insn *insn, if (WARN_ON_ONCE(ins_offset >= 0 || ins_offset < INT_MIN)) return -ERANGE; - /* - * Since the extable follows the program, the fixup offset is always - * negative and limited to BPF_JIT_REGION_SIZE. Store a positive value - * to keep things simple, and put the destination register in the upper - * bits. We don't need to worry about buildtime or runtime sort - * modifying the upper bits because the table is already sorted, and - * isn't part of the main exception table. - * - * The fixup_offset is set to the next instruction from the instruction - * that may fault. The execution will jump to this after handling the - * fault. - */ - fixup_offset = (long)&ex->fixup - (pc + AARCH64_INSN_SIZE); - if (!FIELD_FIT(BPF_FIXUP_OFFSET_MASK, fixup_offset)) - return -ERANGE; - /* * The offsets above have been calculated using the RO buffer but we * need to use the R/W buffer for writes. @@ -1147,8 +1129,7 @@ static int add_exception_handler(const struct bpf_insn *insn, if (BPF_CLASS(insn->code) != BPF_LDX) dst_reg = DONT_CLEAR; - ex->fixup = FIELD_PREP(BPF_FIXUP_OFFSET_MASK, fixup_offset) | - FIELD_PREP(BPF_FIXUP_REG_MASK, dst_reg); + ex->fixup = FIELD_PREP(BPF_FIXUP_REG_MASK, dst_reg); ex->type = EX_TYPE_BPF; -- 2.47.3