From: Jie Meng <jmeng@fb.com>
To: <bpf@vger.kernel.org>, <ast@kernel.org>, <andrii@kernel.org>,
<daniel@iogearbox.net>
Cc: Jie Meng <jmeng@fb.com>
Subject: [PATCH bpf-next v4 1/3] bpf,x64: avoid unnecessary instructions when shift dest is ecx
Date: Sat, 1 Oct 2022 22:11:41 -0700 [thread overview]
Message-ID: <20221002051143.831029-2-jmeng@fb.com> (raw)
In-Reply-To: <20221002051143.831029-1-jmeng@fb.com>
x64 JIT produces redundant instructions when a shift operation's
destination register is BPF_REG_4/ecx and this patch removes them.
Specifically, when dest reg is BPF_REG_4 but the src isn't, we
needn't push and pop ecx around shift only to get it overwritten
by r11 immediately afterwards.
In the rare case when both dest and src registers are BPF_REG_4,
a single shift instruction is sufficient and we don't need the
two MOV instructions around the shift.
To summarize using shift left as an example, without patch:
-------------------------------------------------
| dst == ecx | dst != ecx
=================================================
src == ecx | mov r11, ecx | shl dst, cl
| shl r11, ecx |
| mov ecx, r11 |
-------------------------------------------------
src != ecx | mov r11, ecx | push ecx
| push ecx | mov ecx, src
| mov ecx, src | shl dst, cl
| shl r11, cl | pop ecx
| pop ecx |
| mov ecx, r11 |
-------------------------------------------------
With patch:
-------------------------------------------------
| dst == ecx | dst != ecx
=================================================
src == ecx | shl ecx, cl | shl dst, cl
-------------------------------------------------
src != ecx | mov r11, ecx | push ecx
| mov ecx, src | mov ecx, src
| shl r11, cl | shl dst, cl
| mov ecx, r11 | pop ecx
-------------------------------------------------
Signed-off-by: Jie Meng <jmeng@fb.com>
---
arch/x86/net/bpf_jit_comp.c | 29 +++++++++++++++--------------
1 file changed, 15 insertions(+), 14 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 5b6230779cf3..d9ba997c5891 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -1136,16 +1136,15 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
case BPF_ALU64 | BPF_RSH | BPF_X:
case BPF_ALU64 | BPF_ARSH | BPF_X:
- /* Check for bad case when dst_reg == rcx */
- if (dst_reg == BPF_REG_4) {
- /* mov r11, dst_reg */
- EMIT_mov(AUX_REG, dst_reg);
- dst_reg = AUX_REG;
- }
-
if (src_reg != BPF_REG_4) { /* common case */
- EMIT1(0x51); /* push rcx */
-
+ /* Check for bad case when dst_reg == rcx */
+ if (dst_reg == BPF_REG_4) {
+ /* mov r11, dst_reg */
+ EMIT_mov(AUX_REG, dst_reg);
+ dst_reg = AUX_REG;
+ } else {
+ EMIT1(0x51); /* push rcx */
+ }
/* mov rcx, src_reg */
EMIT_mov(BPF_REG_4, src_reg);
}
@@ -1157,12 +1156,14 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
b3 = simple_alu_opcodes[BPF_OP(insn->code)];
EMIT2(0xD3, add_1reg(b3, dst_reg));
- if (src_reg != BPF_REG_4)
- EMIT1(0x59); /* pop rcx */
+ if (src_reg != BPF_REG_4) {
+ if (insn->dst_reg == BPF_REG_4)
+ /* mov dst_reg, r11 */
+ EMIT_mov(insn->dst_reg, AUX_REG);
+ else
+ EMIT1(0x59); /* pop rcx */
+ }
- if (insn->dst_reg == BPF_REG_4)
- /* mov dst_reg, r11 */
- EMIT_mov(insn->dst_reg, AUX_REG);
break;
case BPF_ALU | BPF_END | BPF_FROM_BE:
--
2.30.2
next prev parent reply other threads:[~2022-10-02 5:12 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-21 2:21 [PATCH bpf-next] bpf,x64: use shrx/sarx/shlx when available Jie Meng
2022-09-22 15:07 ` Daniel Borkmann
2022-09-24 0:32 ` [PATCH bpf-next v2 0/2] bpf,x64: Use BMI2 for shifts Jie Meng
2022-09-24 0:32 ` [PATCH bpf-next v2 1/2] bpf,x64: use shrx/sarx/shlx when available Jie Meng
2022-09-26 19:16 ` Daniel Borkmann
2022-09-27 0:38 ` Jie Meng
2022-09-27 9:45 ` Daniel Borkmann
2022-09-27 18:57 ` [PATCH bpf-next v3 0/3] bpf,x64: Use BMI2 for shifts Jie Meng
2022-09-27 18:57 ` [PATCH bpf-next v3 1/3] bpf,x64: avoid unnecessary instructions when shift dest is ecx Jie Meng
2022-09-27 18:58 ` [PATCH bpf-next v3 2/3] bpf,x64: use shrx/sarx/shlx when available Jie Meng
2022-09-27 18:58 ` [PATCH bpf-next v3 3/3] bpf: add selftests for lsh, rsh, arsh with reg operand Jie Meng
2022-10-02 5:11 ` [PATCH bpf-next v4 0/3] bpf,x64: Use BMI2 for shifts Jie Meng
2022-10-02 5:11 ` Jie Meng [this message]
2022-10-02 5:11 ` [PATCH bpf-next v4 2/3] bpf,x64: use shrx/sarx/shlx when available Jie Meng
2022-10-06 4:11 ` KP Singh
2022-10-07 3:14 ` Jie Meng
2022-10-07 18:11 ` KP Singh
2022-10-07 20:23 ` [PATCH bpf-next v5 0/3] bpf,x64: Use BMI2 for shifts Jie Meng
2022-10-20 0:00 ` patchwork-bot+netdevbpf
2022-10-07 20:23 ` [PATCH bpf-next v5 1/3] bpf,x64: avoid unnecessary instructions when shift dest is ecx Jie Meng
2022-10-07 20:23 ` [PATCH bpf-next v5 2/3] bpf,x64: use shrx/sarx/shlx when available Jie Meng
2022-10-07 20:23 ` [PATCH bpf-next v5 3/3] bpf: add selftests for lsh, rsh, arsh with reg operand Jie Meng
2022-10-02 5:11 ` [PATCH bpf-next v4 " Jie Meng
2022-09-24 0:32 ` [PATCH bpf-next v2 2/2] bpf: Add " Jie Meng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221002051143.831029-2-jmeng@fb.com \
--to=jmeng@fb.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox