From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org,
Naresh Kamboju <naresh.kamboju@linaro.org>,
Jiri Olsa <jolsa@kernel.org>,
Jean-Philippe Brucker <jean-philippe@linaro.org>,
Yauheni Kaliuta <yauheni.kaliuta@redhat.com>,
Ilias Apalodimas <ilias.apalodimas@linaro.org>,
Will Deacon <will@kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Sasha Levin <sashal@kernel.org>
Subject: [PATCH 5.4 47/72] arm64: bpf: Fix branch offset in JIT
Date: Mon, 21 Sep 2020 18:31:26 +0200 [thread overview]
Message-ID: <20200921163124.113419164@linuxfoundation.org> (raw)
In-Reply-To: <20200921163121.870386357@linuxfoundation.org>
From: Ilias Apalodimas <ilias.apalodimas@linaro.org>
[ Upstream commit 32f6865c7aa3c422f710903baa6eb81abc6f559b ]
Running the eBPF test_verifier leads to random errors looking like this:
[ 6525.735488] Unexpected kernel BRK exception at EL1
[ 6525.735502] Internal error: ptrace BRK handler: f2000100 [#1] SMP
[ 6525.741609] Modules linked in: nls_utf8 cifs libdes libarc4 dns_resolver fscache binfmt_misc nls_ascii nls_cp437 vfat fat aes_ce_blk crypto_simd cryptd aes_ce_cipher ghash_ce gf128mul efi_pstore sha2_ce sha256_arm64 sha1_ce evdev efivars efivarfs ip_tables x_tables autofs4 btrfs blake2b_generic xor xor_neon zstd_compress raid6_pq libcrc32c crc32c_generic ahci xhci_pci libahci xhci_hcd igb libata i2c_algo_bit nvme realtek usbcore nvme_core scsi_mod t10_pi netsec mdio_devres of_mdio gpio_keys fixed_phy libphy gpio_mb86s7x
[ 6525.787760] CPU: 3 PID: 7881 Comm: test_verifier Tainted: G W 5.9.0-rc1+ #47
[ 6525.796111] Hardware name: Socionext SynQuacer E-series DeveloperBox, BIOS build #1 Jun 6 2020
[ 6525.804812] pstate: 20000005 (nzCv daif -PAN -UAO BTYPE=--)
[ 6525.810390] pc : bpf_prog_c3d01833289b6311_F+0xc8/0x9f4
[ 6525.815613] lr : bpf_prog_d53bb52e3f4483f9_F+0x38/0xc8c
[ 6525.820832] sp : ffff8000130cbb80
[ 6525.824141] x29: ffff8000130cbbb0 x28: 0000000000000000
[ 6525.829451] x27: 000005ef6fcbf39b x26: 0000000000000000
[ 6525.834759] x25: ffff8000130cbb80 x24: ffff800011dc7038
[ 6525.840067] x23: ffff8000130cbd00 x22: ffff0008f624d080
[ 6525.845375] x21: 0000000000000001 x20: ffff800011dc7000
[ 6525.850682] x19: 0000000000000000 x18: 0000000000000000
[ 6525.855990] x17: 0000000000000000 x16: 0000000000000000
[ 6525.861298] x15: 0000000000000000 x14: 0000000000000000
[ 6525.866606] x13: 0000000000000000 x12: 0000000000000000
[ 6525.871913] x11: 0000000000000001 x10: ffff8000000a660c
[ 6525.877220] x9 : ffff800010951810 x8 : ffff8000130cbc38
[ 6525.882528] x7 : 0000000000000000 x6 : 0000009864cfa881
[ 6525.887836] x5 : 00ffffffffffffff x4 : 002880ba1a0b3e9f
[ 6525.893144] x3 : 0000000000000018 x2 : ffff8000000a4374
[ 6525.898452] x1 : 000000000000000a x0 : 0000000000000009
[ 6525.903760] Call trace:
[ 6525.906202] bpf_prog_c3d01833289b6311_F+0xc8/0x9f4
[ 6525.911076] bpf_prog_d53bb52e3f4483f9_F+0x38/0xc8c
[ 6525.915957] bpf_dispatcher_xdp_func+0x14/0x20
[ 6525.920398] bpf_test_run+0x70/0x1b0
[ 6525.923969] bpf_prog_test_run_xdp+0xec/0x190
[ 6525.928326] __do_sys_bpf+0xc88/0x1b28
[ 6525.932072] __arm64_sys_bpf+0x24/0x30
[ 6525.935820] el0_svc_common.constprop.0+0x70/0x168
[ 6525.940607] do_el0_svc+0x28/0x88
[ 6525.943920] el0_sync_handler+0x88/0x190
[ 6525.947838] el0_sync+0x140/0x180
[ 6525.951154] Code: d4202000 d4202000 d4202000 d4202000 (d4202000)
[ 6525.957249] ---[ end trace cecc3f93b14927e2 ]---
The reason is the offset[] creation and later usage, while building
the eBPF body. The code currently omits the first instruction, since
build_insn() will increase our ctx->idx before saving it.
That was fine up until bounded eBPF loops were introduced. After that
introduction, offset[0] must be the offset of the end of prologue which
is the start of the 1st insn while, offset[n] holds the
offset of the end of n-th insn.
When "taken loop with back jump to 1st insn" test runs, it will
eventually call bpf2a64_offset(-1, 2, ctx). Since negative indexing is
permitted, the current outcome depends on the value stored in
ctx->offset[-1], which has nothing to do with our array.
If the value happens to be 0 the tests will work. If not this error
triggers.
commit 7c2e988f400e ("bpf: fix x64 JIT code generation for jmp to 1st insn")
fixed an indentical bug on x86 when eBPF bounded loops were introduced.
So let's fix it by creating the ctx->offset[] differently. Track the
beginning of instruction and account for the extra instruction while
calculating the arm instruction offsets.
Fixes: 2589726d12a1 ("bpf: introduce bounded loops")
Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org>
Reported-by: Jiri Olsa <jolsa@kernel.org>
Co-developed-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Co-developed-by: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Signed-off-by: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>
Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20200917084925.177348-1-ilias.apalodimas@linaro.org
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
arch/arm64/net/bpf_jit_comp.c | 43 +++++++++++++++++++++++++----------
1 file changed, 31 insertions(+), 12 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index cdc79de0c794a..945e5f690edec 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -141,14 +141,17 @@ static inline void emit_addr_mov_i64(const int reg, const u64 val,
}
}
-static inline int bpf2a64_offset(int bpf_to, int bpf_from,
+static inline int bpf2a64_offset(int bpf_insn, int off,
const struct jit_ctx *ctx)
{
- int to = ctx->offset[bpf_to];
- /* -1 to account for the Branch instruction */
- int from = ctx->offset[bpf_from] - 1;
-
- return to - from;
+ /* BPF JMP offset is relative to the next instruction */
+ bpf_insn++;
+ /*
+ * Whereas arm64 branch instructions encode the offset
+ * from the branch itself, so we must subtract 1 from the
+ * instruction offset.
+ */
+ return ctx->offset[bpf_insn + off] - (ctx->offset[bpf_insn] - 1);
}
static void jit_fill_hole(void *area, unsigned int size)
@@ -532,7 +535,7 @@ emit_bswap_uxt:
/* JUMP off */
case BPF_JMP | BPF_JA:
- jmp_offset = bpf2a64_offset(i + off, i, ctx);
+ jmp_offset = bpf2a64_offset(i, off, ctx);
check_imm26(jmp_offset);
emit(A64_B(jmp_offset), ctx);
break;
@@ -559,7 +562,7 @@ emit_bswap_uxt:
case BPF_JMP32 | BPF_JSLE | BPF_X:
emit(A64_CMP(is64, dst, src), ctx);
emit_cond_jmp:
- jmp_offset = bpf2a64_offset(i + off, i, ctx);
+ jmp_offset = bpf2a64_offset(i, off, ctx);
check_imm19(jmp_offset);
switch (BPF_OP(code)) {
case BPF_JEQ:
@@ -780,10 +783,21 @@ static int build_body(struct jit_ctx *ctx, bool extra_pass)
const struct bpf_prog *prog = ctx->prog;
int i;
+ /*
+ * - offset[0] offset of the end of prologue,
+ * start of the 1st instruction.
+ * - offset[1] - offset of the end of 1st instruction,
+ * start of the 2nd instruction
+ * [....]
+ * - offset[3] - offset of the end of 3rd instruction,
+ * start of 4th instruction
+ */
for (i = 0; i < prog->len; i++) {
const struct bpf_insn *insn = &prog->insnsi[i];
int ret;
+ if (ctx->image == NULL)
+ ctx->offset[i] = ctx->idx;
ret = build_insn(insn, ctx, extra_pass);
if (ret > 0) {
i++;
@@ -791,11 +805,16 @@ static int build_body(struct jit_ctx *ctx, bool extra_pass)
ctx->offset[i] = ctx->idx;
continue;
}
- if (ctx->image == NULL)
- ctx->offset[i] = ctx->idx;
if (ret)
return ret;
}
+ /*
+ * offset is allocated with prog->len + 1 so fill in
+ * the last element with the offset after the last
+ * instruction (end of program)
+ */
+ if (ctx->image == NULL)
+ ctx->offset[i] = ctx->idx;
return 0;
}
@@ -871,7 +890,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
memset(&ctx, 0, sizeof(ctx));
ctx.prog = prog;
- ctx.offset = kcalloc(prog->len, sizeof(int), GFP_KERNEL);
+ ctx.offset = kcalloc(prog->len + 1, sizeof(int), GFP_KERNEL);
if (ctx.offset == NULL) {
prog = orig_prog;
goto out_off;
@@ -951,7 +970,7 @@ skip_init_ctx:
prog->jited_len = image_size;
if (!prog->is_func || extra_pass) {
- bpf_prog_fill_jited_linfo(prog, ctx.offset);
+ bpf_prog_fill_jited_linfo(prog, ctx.offset + 1);
out_off:
kfree(ctx.offset);
kfree(jit_data);
--
2.25.1
next prev parent reply other threads:[~2020-09-21 16:52 UTC|newest]
Thread overview: 76+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-21 16:30 [PATCH 5.4 00/72] 5.4.67-rc1 review Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 01/72] gfs2: initialize transaction tr_ailX_lists earlier Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 02/72] RDMA/bnxt_re: Restrict the max_gids to 256 Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 03/72] e1000e: Add support for Comet Lake Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 04/72] dsa: Allow forwarding of redirected IGMP traffic Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 05/72] net: handle the return value of pskb_carve_frag_list() correctly Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 06/72] hv_netvsc: Remove "unlikely" from netvsc_select_queue Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 07/72] firmware_loader: fix memory leak for paged buffer Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 08/72] NFSv4.1 handle ERR_DELAY error reclaiming locking state on delegation recall Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 09/72] scsi: pm8001: Fix memleak in pm8001_exec_internal_task_abort Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 10/72] scsi: libfc: Fix for double free() Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 11/72] scsi: lpfc: Fix FLOGI/PLOGI receive race condition in pt2pt discovery Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 12/72] regulator: pwm: Fix machine constraints application Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 13/72] spi: spi-loopback-test: Fix out-of-bounds read Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 14/72] NFS: Zero-stateid SETATTR should first return delegation Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 15/72] SUNRPC: stop printk reading past end of string Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 16/72] rapidio: Replace select DMAENGINES with depends on Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 17/72] cifs: fix DFS mount with cifsacl/modefromsid Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 18/72] openrisc: Fix cache API compile issue when not inlining Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 19/72] nvme-fc: cancel async events before freeing event struct Greg Kroah-Hartman
2020-09-21 16:30 ` [PATCH 5.4 20/72] nvme-rdma: " Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 21/72] nvme-tcp: " Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 22/72] block: only call sched requeue_request() for scheduled requests Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 23/72] f2fs: fix indefinite loop scanning for free nid Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 24/72] f2fs: Return EOF on unaligned end of file DIO read Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 25/72] i2c: algo: pca: Reapply i2c bus settings after reset Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 26/72] spi: Fix memory leak on splited transfers Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 27/72] KVM: MIPS: Change the definition of kvm type Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 28/72] clk: davinci: Use the correct size when allocating memory Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 29/72] clk: rockchip: Fix initialization of mux_pll_src_4plls_p Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 30/72] ASoC: qcom: Set card->owner to avoid warnings Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 31/72] ASoC: qcom: common: Fix refcount imbalance on error Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 32/72] powerpc/book3s64/radix: Fix boot failure with large amount of guest memory Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 33/72] ASoC: meson: axg-toddr: fix channel order on g12 platforms Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 34/72] Drivers: hv: vmbus: hibernation: do not hang forever in vmbus_bus_resume() Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 35/72] scsi: libsas: Fix error path in sas_notify_lldd_dev_found() Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 36/72] arm64: Allow CPUs unffected by ARM erratum 1418040 to come in late Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 37/72] Drivers: hv: vmbus: Add timeout to vmbus_wait_for_unload Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 38/72] perf test: Fix the "signal" test inline assembly Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 39/72] MIPS: SNI: Fix MIPS_L1_CACHE_SHIFT Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 40/72] perf evlist: Fix cpu/thread map leak Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 41/72] perf parse-event: Fix memory leak in evsel->unit Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 42/72] perf test: Free formats for perf pmu parse test Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 43/72] fbcon: Fix user font detection test at fbcon_resize() Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 44/72] MIPS: SNI: Fix spurious interrupts Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 45/72] drm/mediatek: Add exception handing in mtk_drm_probe() if component init fail Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 46/72] drm/mediatek: Add missing put_device() call in mtk_hdmi_dt_parse_pdata() Greg Kroah-Hartman
2020-09-21 16:31 ` Greg Kroah-Hartman [this message]
2020-09-21 16:31 ` [PATCH 5.4 48/72] iommu/amd: Fix potential @entry null deref Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 49/72] i2c: mxs: use MXS_DMA_CTRL_WAIT4END instead of DMA_CTRL_ACK Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 50/72] riscv: Add sfence.vma after early page table changes Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 51/72] drm/i915: Filter wake_flags passed to default_wake_function Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 52/72] USB: quirks: Add USB_QUIRK_IGNORE_REMOTE_WAKEUP quirk for BYD zhaoxin notebook Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 53/72] USB: UAS: fix disconnect by unplugging a hub Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 54/72] usblp: fix race between disconnect() and read() Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 55/72] usb: typec: ucsi: Prevent mode overrun Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 56/72] i2c: i801: Fix resume bug Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 57/72] Revert "ALSA: hda - Fix silent audio output and corrupted input on MSI X570-A PRO" Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 58/72] ALSA: hda: fixup headset for ASUS GX502 laptop Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 59/72] ALSA: hda/realtek - The Mic on a RedmiBook doesnt work Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 60/72] percpu: fix first chunk size calculation for populated bitmap Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 61/72] Input: trackpoint - add new trackpoint variant IDs Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 62/72] Input: i8042 - add Entroware Proteus EL07R4 to nomux and reset lists Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 63/72] serial: 8250_pci: Add Realtek 816a and 816b Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 64/72] x86/boot/compressed: Disable relocation relaxation Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 65/72] s390/zcrypt: fix kmalloc 256k failure Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 66/72] ehci-hcd: Move include to keep CRC stable Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 67/72] powerpc/dma: Fix dma_map_ops::get_required_mask Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 68/72] selftests/vm: fix display of page size in map_hugetlb Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 69/72] dm/dax: Fix table reference counts Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 70/72] mm/memory_hotplug: drain per-cpu pages again during memory offline Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 71/72] dm: Call proper helper to determine dax support Greg Kroah-Hartman
2020-09-21 16:31 ` [PATCH 5.4 72/72] dax: Fix compilation for CONFIG_DAX && !CONFIG_FS_DAX Greg Kroah-Hartman
2020-09-22 6:00 ` [PATCH 5.4 00/72] 5.4.67-rc1 review Naresh Kamboju
2020-09-22 6:46 ` Jon Hunter
2020-09-22 20:19 ` Guenter Roeck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200921163124.113419164@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=catalin.marinas@arm.com \
--cc=ilias.apalodimas@linaro.org \
--cc=jean-philippe@linaro.org \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=naresh.kamboju@linaro.org \
--cc=sashal@kernel.org \
--cc=stable@vger.kernel.org \
--cc=will@kernel.org \
--cc=yauheni.kaliuta@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).