[PATCH v5 bpf-next 00/23] BPF register bounds logic and testing improvements

[PATCH v5 bpf-next 00/23] BPF register bounds logic and testing improvements

[Andrii Nakryiko]

This patch set adds a big set of manual and auto-generated test cases
validating BPF verifier's register bounds tracking and deduction logic. See
details in the last patch.

We start with building a tester that validates existing <range> vs <scalar>
verifier logic for range bounds. To make all this work, BPF verifier's logic
needed a bunch of improvements to handle some cases that previously were not
covered. This had no implications as to correctness of verifier logic, but it
was incomplete enough to cause significant disagreements with alternative
implementation of register bounds logic that tests in this patch set
implement. So we need BPF verifier logic improvements to make all the tests
pass. This is what we do in patches #3 through #9.

Patch #10 implements tester. We guard millions of generated tests behind
SLOW_TESTS=1 envvar requirement, but also have a relatively small number of
tricky cases that came up during development and debugging of this work. Those
will be executed as part of a normal test_progs run.

With range vs const cases taken care of and well tested, we move to
generalizing this to handle generic range vs range cases. Patches #11-#17
perform preliminary refactorings without functionally changing anything. But
they do clean up check_cond_jmp_op() logic and generalize a bunch of other
pieces in is_branch_taken() logic.

With refactorings out of the way, patch #18 teaches reg_set_min_max() to
handle <range> vs <range>, whenever possible, and patch #19 adjusts
is_branch_taken() accordingly. Those two have to match each other, as
is_branch_taken() prevents some situations that reg_set_min_max() assumes not
possible from getting through to reg_set_min_max().

One such class of situations is when we mix 64-bit operations with 32-bit
operations on the same register. Depending on specific sequence, it's possible
to get to the point where u64/s64 bounds will be very generic (e.g., after
signed 32-bit comparison), while we still keep pretty tight u32/s32 bounds. If
in such state we proceed with 32-bit equality or inequality comparison,
reg_set_min_max() might have to deal with adjusting s32 bounds for two
registers that don't overlap, which breaks reg_set_min_max(). This doesn't
manifest in <range> vs <const> cases, because if that happens
reg_set_min_max() in effect will force s32 bounds to be a new "impossible"
constant (from original smin32/smax32 bounds point of view). Things get tricky
when we have <range> vs <range> adjustments, so instead of trying to somehow
make sense out of such situations, it's best to detect such impossible
situations and prune the branch that can't be take in is_branch_taken() logic.
This is taken care of in patch #20.

Note, this is not unique to <range> vs <range> logic. Just recently ([0])
a related issue was reported for existing verifier logic. This patch set does
fix that issues as well, as pointed out on the mailing list.

Wrapping up, patches #21-22 adjust reg_bounds selftests to handle and test
range vs range cases.

Finally, a tiny test which was, amazingly, an initial motivation for this
work, is added in patch #23, demonstrating how verifier is now smart enough to
track actual number of elements in the array and won't require additional
checks on loop iteration variable inside the bpf_for() loop.

  [0] https://lore.kernel.org/bpf/CAEf4Bzbgf-WQSCz8D4Omh3zFdS4oWS6XELnE7VeoUWgKf3cpig@mail.gmail.com/

v4->v5:
  - added entirety of verifier reg bounds tracking changes, now handling
    <range> vs <range> cases (Alexei);
  - added way more comments trying to explain why deductions added are
    correct, hopefully they are useful and clarify things a bit (Daniel,
    Shung-Hsi);
  - added two preliminary selftests fixes necessary for RELEASE=1 build to
    work again, it keeps breaking.
v3->v4:
  - improvements to reg_bounds tester (progress report, split 32-bit and
    64-bit ranges, fix various verbosity output issues, etc);
v2->v3:
  - fix a subtle little-endianness assumption inside parge_reg_state() (CI);
v1->v2:
  - fix compilation when building selftests with llvm-16 toolchain (CI).

Andrii Nakryiko (23):
  selftests/bpf: fix RELEASE=1 build for tc_opts
  selftests/bpf: satisfy compiler by having explicit return in btf test
  bpf: derive smin/smax from umin/max bounds
  bpf: derive smin32/smax32 from umin32/umax32 bounds
  bpf: derive subreg bounds from full bounds when upper 32 bits are constant
  bpf: add special smin32/smax32 derivation from 64-bit bounds
  bpf: improve deduction of 64-bit bounds from 32-bit bounds
  bpf: try harder to deduce register bounds from different numeric domains
  bpf: drop knowledge-losing __reg_combine_{32,64}_into_{64,32} logic
  selftests/bpf: BPF register range bounds tester
  bpf: rename is_branch_taken reg arguments to prepare for the second one
  bpf: generalize is_branch_taken() to work with two registers
  bpf: move is_branch_taken() down
  bpf: generalize is_branch_taken to handle all conditional jumps in one place
  bpf: unify 32-bit and 64-bit is_branch_taken logic
  bpf: prepare reg_set_min_max for second set of registers
  bpf: generalize reg_set_min_max() to handle two sets of two registers
  bpf: generalize reg_set_min_max() to handle non-const register comparisons
  bpf: generalize is_scalar_branch_taken() logic
  bpf: enhance BPF_JEQ/BPF_JNE is_branch_taken logic
  selftests/bpf: adjust OP_EQ/OP_NE handling to use subranges for branch taken
  selftests/bpf: add range x range test to reg_bounds
  selftests/bpf: add iter test requiring range x range logic

 include/linux/tnum.h                          |    4 +
 kernel/bpf/tnum.c                             |    7 +-
 kernel/bpf/verifier.c                         |  920 ++++----
 tools/testing/selftests/bpf/prog_tests/btf.c  |    1 +
 .../selftests/bpf/prog_tests/reg_bounds.c     | 1938 +++++++++++++++++
 .../selftests/bpf/prog_tests/tc_opts.c        |    6 +-
 tools/testing/selftests/bpf/progs/iters.c     |   22 +
 7 files changed, 2473 insertions(+), 425 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/reg_bounds.c

-- 
2.34.1

[PATCH v5 bpf-next 01/23] selftests/bpf: fix RELEASE=1 build for tc_opts

[Andrii Nakryiko]

Compiler complains about malloc(). We also don't need to dynamically
allocate anything, so make the life easier by using statically sized
buffer.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 tools/testing/selftests/bpf/prog_tests/tc_opts.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/tc_opts.c b/tools/testing/selftests/bpf/prog_tests/tc_opts.c
index 51883ccb8020..196abf223465 100644
--- a/tools/testing/selftests/bpf/prog_tests/tc_opts.c
+++ b/tools/testing/selftests/bpf/prog_tests/tc_opts.c
@@ -2387,12 +2387,9 @@ static int generate_dummy_prog(void)
 	const size_t prog_insn_cnt = sizeof(prog_insns) / sizeof(struct bpf_insn);
 	LIBBPF_OPTS(bpf_prog_load_opts, opts);
 	const size_t log_buf_sz = 256;
-	char *log_buf;
+	char log_buf[log_buf_sz];
 	int fd = -1;
 
-	log_buf = malloc(log_buf_sz);
-	if (!ASSERT_OK_PTR(log_buf, "log_buf_alloc"))
-		return fd;
 	opts.log_buf = log_buf;
 	opts.log_size = log_buf_sz;
 
@@ -2402,7 +2399,6 @@ static int generate_dummy_prog(void)
 			   prog_insns, prog_insn_cnt, &opts);
 	ASSERT_STREQ(log_buf, "", "log_0");
 	ASSERT_GE(fd, 0, "prog_fd");
-	free(log_buf);
 	return fd;
 }
 
-- 
2.34.1

[PATCH v5 bpf-next 02/23] selftests/bpf: satisfy compiler by having explicit return in btf test

[Andrii Nakryiko]

Some compilers complain about get_pprint_mapv_size() not returning value
in some code paths. Fix with explicit return.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 tools/testing/selftests/bpf/prog_tests/btf.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/btf.c b/tools/testing/selftests/bpf/prog_tests/btf.c
index 92d51f377fe5..8fb4a04fbbc0 100644
--- a/tools/testing/selftests/bpf/prog_tests/btf.c
+++ b/tools/testing/selftests/bpf/prog_tests/btf.c
@@ -5265,6 +5265,7 @@ static size_t get_pprint_mapv_size(enum pprint_mapv_kind_t mapv_kind)
 #endif
 
 	assert(0);
+	return 0;
 }
 
 static void set_pprint_mapv(enum pprint_mapv_kind_t mapv_kind,
-- 
2.34.1

[PATCH v5 bpf-next 03/23] bpf: derive smin/smax from umin/max bounds

[Andrii Nakryiko]

Add smin/smax derivation from appropriate umin/umax values. Previously the
logic was surprisingly asymmetric, trying to derive umin/umax from smin/smax
(if possible), but not trying to do the same in the other direction. A simple
addition to __reg64_deduce_bounds() fixes this.

Added also generic comment about u64/s64 ranges and their relationship.
Hopefully that helps readers to understand all the bounds deductions
a bit better.

Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 70 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 857d76694517..bf4193706744 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2358,6 +2358,76 @@ static void __reg32_deduce_bounds(struct bpf_reg_state *reg)
 
 static void __reg64_deduce_bounds(struct bpf_reg_state *reg)
 {
+	/* If u64 range forms a valid s64 range (due to matching sign bit),
+	 * try to learn from that. Let's do a bit of ASCII art to see when
+	 * this is happening. Let's take u64 range first:
+	 *
+	 * 0             0x7fffffffffffffff 0x8000000000000000        U64_MAX
+	 * |-------------------------------|--------------------------------|
+	 *
+	 * Valid u64 range is formed when umin and umax are anywhere in this
+	 * range [0, U64_MAX] and umin <= umax. u64 is simple and
+	 * straightforward. Let's where s64 range maps to this simple [0,
+	 * U64_MAX] range, annotated below the line for comparison:
+	 *
+	 * 0             0x7fffffffffffffff 0x8000000000000000        U64_MAX
+	 * |-------------------------------|--------------------------------|
+	 * 0                        S64_MAX S64_MIN                        -1
+	 *
+	 * So s64 values basically start in the middle and then are contiguous
+	 * to the right of it, wrapping around from -1 to 0, and then
+	 * finishing as S64_MAX (0x7fffffffffffffff) right before S64_MIN.
+	 * We can try drawing more visually continuity of u64 vs s64 values as
+	 * mapped to just actual hex valued range of values.
+	 *
+	 *  u64 start                                               u64 end
+	 *  _______________________________________________________________
+	 * /                                                               \
+	 * 0             0x7fffffffffffffff 0x8000000000000000        U64_MAX
+	 * |-------------------------------|--------------------------------|
+	 * 0                        S64_MAX S64_MIN                        -1
+	 *                                / \
+	 * >------------------------------   ------------------------------->
+	 * s64 continues...        s64 end   s64 start          s64 "midpoint"
+	 *
+	 * What this means is that in general, we can't always derive
+	 * something new about u64 from any random s64 range, and vice versa.
+	 * But we can do that in two particular cases. One is when entire
+	 * u64/s64 range is *entirely* contained within left half of the above
+	 * diagram or when it is *entirely* contained in the right half. I.e.:
+	 *
+	 * |-------------------------------|--------------------------------|
+	 *     ^                   ^            ^                 ^
+	 *     A                   B            C                 D
+	 *
+	 * [A, B] and [C, D] are contained entirely in their respective halves
+	 * and form valid contiguous ranges as both u64 and s64 values. [A, B]
+	 * will be non-negative both as u64 and s64 (and in fact it will be
+	 * identical ranges no matter the signedness). [C, D] treated as s64
+	 * will be a range of negative values, while in u64 it will be
+	 * non-negative range of values larger than 0x8000000000000000.
+	 *
+	 * Now, any other range here can't be represented in both u64 and s64
+	 * simultaneously. E.g., [A, C], [A, D], [B, C], [B, D] are valid
+	 * contiguous u64 ranges, but they are discontinuous in s64. [B, C]
+	 * in s64 would be properly presented as [S64_MIN, C] and [B, S64_MAX],
+	 * for example. Similarly, valid s64 range [D, A] (going from negative
+	 * to positive values), would be two separate [D, U64_MAX] and [0, A]
+	 * ranges as u64. Currently reg_state can't represent two segments per
+	 * numeric domain, so in such situations we can only derive maximal
+	 * possible range ([0, U64_MAX] for u64, and [S64_MIN, S64_MAX) for s64).
+	 *
+	 * So we use these facts to derive umin/umax from smin/smax and vice
+	 * versa only if they stay within the same "half". This is equivalent
+	 * to checking sign bit: lower half will have sign bit as zero, upper
+	 * half have sign bit 1. Below in code we simplify this by just
+	 * casting umin/umax as smin/smax and checking if they form valid
+	 * range, and vice versa. Those are equivalent checks.
+	 */
+	if ((s64)reg->umin_value <= (s64)reg->umax_value) {
+		reg->smin_value = max_t(s64, reg->smin_value, reg->umin_value);
+		reg->smax_value = min_t(s64, reg->smax_value, reg->umax_value);
+	}
 	/* Learn sign from signed bounds.
 	 * If we cannot cross the sign boundary, then signed and unsigned bounds
 	 * are the same, so combine.  This works even in the negative case, e.g.
-- 
2.34.1

[PATCH v5 bpf-next 04/23] bpf: derive smin32/smax32 from umin32/umax32 bounds

[Andrii Nakryiko]

All the logic that applies to u64 vs s64, equally applies for u32 vs s32
relationships (just taken in a smaller 32-bit numeric space). So do the
same deduction of smin32/smax32 from umin32/umax32, if we can.

Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index bf4193706744..0f66e9092c38 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2324,6 +2324,13 @@ static void __update_reg_bounds(struct bpf_reg_state *reg)
 /* Uses signed min/max values to inform unsigned, and vice-versa */
 static void __reg32_deduce_bounds(struct bpf_reg_state *reg)
 {
+	/* if u32 range forms a valid s32 range (due to matching sign bit),
+	 * try to learn from that
+	 */
+	if ((s32)reg->u32_min_value <= (s32)reg->u32_max_value) {
+		reg->s32_min_value = max_t(s32, reg->s32_min_value, reg->u32_min_value);
+		reg->s32_max_value = min_t(s32, reg->s32_max_value, reg->u32_max_value);
+	}
 	/* Learn sign from signed bounds.
 	 * If we cannot cross the sign boundary, then signed and unsigned bounds
 	 * are the same, so combine.  This works even in the negative case, e.g.
-- 
2.34.1

[PATCH v5 bpf-next 05/23] bpf: derive subreg bounds from full bounds when upper 32 bits are constant

[Andrii Nakryiko]

Comments in code try to explain the idea behind why this is correct.
Please check the code and comments.

Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 45 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 0f66e9092c38..5082ca1ea5dc 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2324,6 +2324,51 @@ static void __update_reg_bounds(struct bpf_reg_state *reg)
 /* Uses signed min/max values to inform unsigned, and vice-versa */
 static void __reg32_deduce_bounds(struct bpf_reg_state *reg)
 {
+	/* If upper 32 bits of u64/s64 range don't change, we can use lower 32
+	 * bits to improve our u32/s32 boundaries.
+	 *
+	 * E.g., the case where we have upper 32 bits as zero ([10, 20] in
+	 * u64) is pretty trivial, it's obvious that in u32 we'll also have
+	 * [10, 20] range. But this property holds for any 64-bit range as
+	 * long as upper 32 bits in that entire range of values stay the same.
+	 *
+	 * E.g., u64 range [0x10000000A, 0x10000000F] ([4294967306, 4294967311]
+	 * in decimal) has the same upper 32 bits throughout all the values in
+	 * that range. As such, lower 32 bits form a valid [0xA, 0xF] ([10, 15])
+	 * range.
+	 *
+	 * Note also, that [0xA, 0xF] is a valid range both in u32 and in s32,
+	 * following the rules outlined below about u64/s64 correspondence
+	 * (which equally applies to u32 vs s32 correspondence). In general it
+	 * depends on actual hexadecimal values of 32-bit range. They can form
+	 * only valid u32, or only valid s32 ranges in some cases.
+	 *
+	 * So we use all these insights to derive bounds for subregisters here.
+	 */
+	if ((reg->umin_value >> 32) == (reg->umax_value >> 32)) {
+		/* u64 to u32 casting preserves validity of low 32 bits as
+		 * a range, if upper 32 bits are the same
+		 */
+		reg->u32_min_value = max_t(u32, reg->u32_min_value, (u32)reg->umin_value);
+		reg->u32_max_value = min_t(u32, reg->u32_max_value, (u32)reg->umax_value);
+
+		if ((s32)reg->umin_value <= (s32)reg->umax_value) {
+			reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->umin_value);
+			reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->umax_value);
+		}
+	}
+	if ((reg->smin_value >> 32) == (reg->smax_value >> 32)) {
+		/* low 32 bits should form a proper u32 range */
+		if ((u32)reg->smin_value <= (u32)reg->smax_value) {
+			reg->u32_min_value = max_t(u32, reg->u32_min_value, (u32)reg->smin_value);
+			reg->u32_max_value = min_t(u32, reg->u32_max_value, (u32)reg->smax_value);
+		}
+		/* low 32 bits should form a proper s32 range */
+		if ((s32)reg->smin_value <= (s32)reg->smax_value) {
+			reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->smin_value);
+			reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->smax_value);
+		}
+	}
 	/* if u32 range forms a valid s32 range (due to matching sign bit),
 	 * try to learn from that
 	 */
-- 
2.34.1

[PATCH v5 bpf-next 06/23] bpf: add special smin32/smax32 derivation from 64-bit bounds

[Andrii Nakryiko]

Add a special case where we can derive valid s32 bounds from umin/umax
or smin/smax by stitching together negative s32 subrange and
non-negative s32 subrange. That requires upper 32 bits to form a [N, N+1]
range in u32 domain (taking into account wrap around, so 0xffffffff
to 0x00000000 is a valid [N, N+1] range in this sense). See code comment
for concrete examples.

Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 5082ca1ea5dc..38d21d0e46bd 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2369,6 +2369,29 @@ static void __reg32_deduce_bounds(struct bpf_reg_state *reg)
 			reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->smax_value);
 		}
 	}
+	/* Special case where upper bits form a small sequence of two
+	 * sequential numbers (in 32-bit unsigned space, so 0xffffffff to
+	 * 0x00000000 is also valid), while lower bits form a proper s32 range
+	 * going from negative numbers to positive numbers. E.g., let's say we
+	 * have s64 range [-1, 1] ([0xffffffffffffffff, 0x0000000000000001]).
+	 * Possible s64 values are {-1, 0, 1} ({0xffffffffffffffff,
+	 * 0x0000000000000000, 0x00000000000001}). Ignoring upper 32 bits,
+	 * we still get a valid s32 range [-1, 1] ([0xffffffff, 0x00000001]).
+	 * Note that it doesn't have to be 0xffffffff going to 0x00000000 in
+	 * upper 32 bits. As a random example, s64 range
+	 * [0xfffffff0ffffff00; 0xfffffff100000010], forms a valid s32 range
+	 * [-16, 16] ([0xffffff00; 0x00000010]) in its 32 bit subregister.
+	 */
+	if ((u32)(reg->umin_value >> 32) + 1 == (u32)(reg->umax_value >> 32) &&
+	    (s32)reg->umin_value < 0 && (s32)reg->umax_value >= 0) {
+		reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->umin_value);
+		reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->umax_value);
+	}
+	if ((u32)(reg->smin_value >> 32) + 1 == (u32)(reg->smax_value >> 32) &&
+	    (s32)reg->smin_value < 0 && (s32)reg->smax_value >= 0) {
+		reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->smin_value);
+		reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->smax_value);
+	}
 	/* if u32 range forms a valid s32 range (due to matching sign bit),
 	 * try to learn from that
 	 */
-- 
2.34.1

[PATCH v5 bpf-next 07/23] bpf: improve deduction of 64-bit bounds from 32-bit bounds

[Andrii Nakryiko]

Add a few interesting cases in which we can tighten 64-bit bounds based
on newly learnt information about 32-bit bounds. E.g., when full u64/s64
registers are used in BPF program, and then eventually compared as
u32/s32. The latter comparison doesn't change the value of full
register, but it does impose new restrictions on possible lower 32 bits
of such full registers. And we can use that to derive additional full
register bounds information.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 47 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 38d21d0e46bd..768247e3d667 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2535,10 +2535,57 @@ static void __reg64_deduce_bounds(struct bpf_reg_state *reg)
 	}
 }
 
+static void __reg_deduce_mixed_bounds(struct bpf_reg_state *reg)
+{
+	/* Try to tighten 64-bit bounds from 32-bit knowledge, using 32-bit
+	 * values on both sides of 64-bit range in hope to have tigher range.
+	 * E.g., if r1 is [0x1'00000000, 0x3'80000000], and we learn from
+	 * 32-bit signed > 0 operation that s32 bounds are now [1; 0x7fffffff].
+	 * With this, we can substitute 1 as low 32-bits of _low_ 64-bit bound
+	 * (0x100000000 -> 0x100000001) and 0x7fffffff as low 32-bits of
+	 * _high_ 64-bit bound (0x380000000 -> 0x37fffffff) and arrive at a
+	 * better overall bounds for r1 as [0x1'000000001; 0x3'7fffffff].
+	 * We just need to make sure that derived bounds we are intersecting
+	 * with are well-formed ranges in respecitve s64 or u64 domain, just
+	 * like we do with similar kinds of 32-to-64 or 64-to-32 adjustments.
+	 */
+	__u64 new_umin, new_umax;
+	__s64 new_smin, new_smax;
+
+	/* u32 -> u64 tightening, it's always well-formed */
+	new_umin = (reg->umin_value & ~0xffffffffULL) | reg->u32_min_value;
+	new_umax = (reg->umax_value & ~0xffffffffULL) | reg->u32_max_value;
+	reg->umin_value = max_t(u64, reg->umin_value, new_umin);
+	reg->umax_value = min_t(u64, reg->umax_value, new_umax);
+
+	/* s32 -> u64 tightening, s32 should be a valid u32 range (same sign) */
+	if ((u32)reg->s32_min_value <= (u32)reg->s32_max_value) {
+		new_umin = (reg->umin_value & ~0xffffffffULL) | (u32)reg->s32_min_value;
+		new_umax = (reg->umax_value & ~0xffffffffULL) | (u32)reg->s32_max_value;
+		reg->umin_value = max_t(u64, reg->umin_value, new_umin);
+		reg->umax_value = min_t(u64, reg->umax_value, new_umax);
+	}
+
+	/* u32 -> s64 tightening, u32 range embedded into s64 preserves range validity */
+	new_smin = (reg->smin_value & ~0xffffffffULL) | reg->u32_min_value;
+	new_smax = (reg->smax_value & ~0xffffffffULL) | reg->u32_max_value;
+	reg->smin_value = max_t(s64, reg->smin_value, new_smin);
+	reg->smax_value = min_t(s64, reg->smax_value, new_smax);
+
+	/* s32 -> s64 tightening, check that s32 range behaves as u32 range */
+	if ((u32)reg->s32_min_value <= (u32)reg->s32_max_value) {
+		new_smin = (reg->smin_value & ~0xffffffffULL) | (u32)reg->s32_min_value;
+		new_smax = (reg->smax_value & ~0xffffffffULL) | (u32)reg->s32_max_value;
+		reg->smin_value = max_t(s64, reg->smin_value, new_smin);
+		reg->smax_value = min_t(s64, reg->smax_value, new_smax);
+	}
+}
+
 static void __reg_deduce_bounds(struct bpf_reg_state *reg)
 {
 	__reg32_deduce_bounds(reg);
 	__reg64_deduce_bounds(reg);
+	__reg_deduce_mixed_bounds(reg);
 }
 
 /* Attempts to improve var_off based on unsigned min/max information */
-- 
2.34.1

[PATCH v5 bpf-next 08/23] bpf: try harder to deduce register bounds from different numeric domains

[Andrii Nakryiko]

There are cases (caught by subsequent reg_bounds tests in selftests/bpf)
where performing one round of __reg_deduce_bounds() doesn't propagate
all the information from, say, s32 to u32 bounds and than from newly
learned u32 bounds back to u64 and s64. So perform __reg_deduce_bounds()
twice to make sure such derivations are propagated fully after
reg_bounds_sync().

One such example is test `(s64)[0xffffffff00000001; 0] (u64)<
0xffffffff00000000` from selftest patch from this patch set. It demonstrates an
intricate dance of u64 -> s64 -> u64 -> u32 bounds adjustments, which requires
two rounds of __reg_deduce_bounds(). Here are corresponding refinement log from
selftest, showing evolution of knowledge.

REFINING (FALSE R1) (u64)SRC=[0xffffffff00000000; U64_MAX] (u64)DST_OLD=[0; U64_MAX] (u64)DST_NEW=[0xffffffff00000000; U64_MAX]
REFINING (FALSE R1) (u64)SRC=[0xffffffff00000000; U64_MAX] (s64)DST_OLD=[0xffffffff00000001; 0] (s64)DST_NEW=[0xffffffff00000001; -1]
REFINING (FALSE R1) (s64)SRC=[0xffffffff00000001; -1] (u64)DST_OLD=[0xffffffff00000000; U64_MAX] (u64)DST_NEW=[0xffffffff00000001; U64_MAX]
REFINING (FALSE R1) (u64)SRC=[0xffffffff00000001; U64_MAX] (u32)DST_OLD=[0; U32_MAX] (u32)DST_NEW=[1; U32_MAX]

R1 initially has smin/smax set to [0xffffffff00000001; -1], while umin/umax is
unknown. After (u64)< comparison, in FALSE branch we gain knowledge that
umin/umax is [0xffffffff00000000; U64_MAX]. That causes smin/smax to learn that
zero can't happen and upper bound is -1. Then smin/smax is adjusted from
umin/umax improving lower bound from 0xffffffff00000000 to 0xffffffff00000001.
And then eventually umin32/umax32 bounds are drived from umin/umax and become
[1; U32_MAX].

Selftest in the last patch is actually implementing a multi-round fixed-point
convergence logic, but so far all the tests are handled by two rounds of
reg_bounds_sync() on the verifier state, so we keep it simple for now.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 768247e3d667..6b0736c04ebe 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2607,6 +2607,7 @@ static void reg_bounds_sync(struct bpf_reg_state *reg)
 	__update_reg_bounds(reg);
 	/* We might have learned something about the sign bit. */
 	__reg_deduce_bounds(reg);
+	__reg_deduce_bounds(reg);
 	/* We might have learned some bits from the bounds. */
 	__reg_bound_offset(reg);
 	/* Intersecting with the old var_off might have improved our bounds
-- 
2.34.1

[PATCH v5 bpf-next 09/23] bpf: drop knowledge-losing __reg_combine_{32,64}_into_{64,32} logic

[Andrii Nakryiko]

When performing 32-bit conditional operation operating on lower 32 bits
of a full 64-bit register, register full value isn't changed. We just
potentially gain new knowledge about that register's lower 32 bits.

Unfortunately, __reg_combine_{32,64}_into_{64,32} logic that
reg_set_min_max() performs as a last step, can lose information in some
cases due to __mark_reg64_unbounded() and __reg_assign_32_into_64().
That's bad and completely unnecessary. Especially __reg_assign_32_into_64()
looks completely out of place here, because we are not performing
zero-extending subregister assignment during conditional jump.

So this patch replaced __reg_combine_* with just a normal
reg_bounds_sync() which will do a proper job of deriving u64/s64 bounds
from u32/s32, and vice versa (among all other combinations).

__reg_combine_64_into_32() is also used in one more place,
coerce_reg_to_size(), while handling 1- and 2-byte register loads.
Looking into this, it seems like besides marking subregister as
unbounded before performing reg_bounds_sync(), we were also performing
deduction of smin32/smax32 and umin32/umax32 bounds from respective
smin/smax and umin/umax bounds. It's now redundant as reg_bounds_sync()
performs all the same logic more generically (e.g., without unnecessary
assumption that upper 32 bits of full register should be zero).

Long story short, we remove __reg_combine_64_into_32() completely, and
coerce_reg_to_size() now only does resetting subreg to unbounded and then
performing reg_bounds_sync() to recover as much information as possible
from 64-bit umin/umax and smin/smax bounds, set explicitly in
coerce_reg_to_size() earlier.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 60 ++++++-------------------------------------
 1 file changed, 8 insertions(+), 52 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 6b0736c04ebe..f5fcb7fb2c67 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2641,51 +2641,6 @@ static void __reg_assign_32_into_64(struct bpf_reg_state *reg)
 	}
 }
 
-static void __reg_combine_32_into_64(struct bpf_reg_state *reg)
-{
-	/* special case when 64-bit register has upper 32-bit register
-	 * zeroed. Typically happens after zext or <<32, >>32 sequence
-	 * allowing us to use 32-bit bounds directly,
-	 */
-	if (tnum_equals_const(tnum_clear_subreg(reg->var_off), 0)) {
-		__reg_assign_32_into_64(reg);
-	} else {
-		/* Otherwise the best we can do is push lower 32bit known and
-		 * unknown bits into register (var_off set from jmp logic)
-		 * then learn as much as possible from the 64-bit tnum
-		 * known and unknown bits. The previous smin/smax bounds are
-		 * invalid here because of jmp32 compare so mark them unknown
-		 * so they do not impact tnum bounds calculation.
-		 */
-		__mark_reg64_unbounded(reg);
-	}
-	reg_bounds_sync(reg);
-}
-
-static bool __reg64_bound_s32(s64 a)
-{
-	return a >= S32_MIN && a <= S32_MAX;
-}
-
-static bool __reg64_bound_u32(u64 a)
-{
-	return a >= U32_MIN && a <= U32_MAX;
-}
-
-static void __reg_combine_64_into_32(struct bpf_reg_state *reg)
-{
-	__mark_reg32_unbounded(reg);
-	if (__reg64_bound_s32(reg->smin_value) && __reg64_bound_s32(reg->smax_value)) {
-		reg->s32_min_value = (s32)reg->smin_value;
-		reg->s32_max_value = (s32)reg->smax_value;
-	}
-	if (__reg64_bound_u32(reg->umin_value) && __reg64_bound_u32(reg->umax_value)) {
-		reg->u32_min_value = (u32)reg->umin_value;
-		reg->u32_max_value = (u32)reg->umax_value;
-	}
-	reg_bounds_sync(reg);
-}
-
 /* Mark a register as having a completely unknown (scalar) value. */
 static void __mark_reg_unknown(const struct bpf_verifier_env *env,
 			       struct bpf_reg_state *reg)
@@ -6382,9 +6337,10 @@ static void coerce_reg_to_size(struct bpf_reg_state *reg, int size)
 	 * values are also truncated so we push 64-bit bounds into
 	 * 32-bit bounds. Above were truncated < 32-bits already.
 	 */
-	if (size >= 4)
-		return;
-	__reg_combine_64_into_32(reg);
+	if (size < 4) {
+		__mark_reg32_unbounded(reg);
+		reg_bounds_sync(reg);
+	}
 }
 
 static void set_sext64_default_val(struct bpf_reg_state *reg, int size)
@@ -14623,13 +14579,13 @@ static void reg_set_min_max(struct bpf_reg_state *true_reg,
 					     tnum_subreg(false_32off));
 		true_reg->var_off = tnum_or(tnum_clear_subreg(true_64off),
 					    tnum_subreg(true_32off));
-		__reg_combine_32_into_64(false_reg);
-		__reg_combine_32_into_64(true_reg);
+		reg_bounds_sync(false_reg);
+		reg_bounds_sync(true_reg);
 	} else {
 		false_reg->var_off = false_64off;
 		true_reg->var_off = true_64off;
-		__reg_combine_64_into_32(false_reg);
-		__reg_combine_64_into_32(true_reg);
+		reg_bounds_sync(false_reg);
+		reg_bounds_sync(true_reg);
 	}
 }
 
-- 
2.34.1

[PATCH v5 bpf-next 10/23] selftests/bpf: BPF register range bounds tester

[Andrii Nakryiko]

Add test to validate BPF verifier's register range bounds tracking logic.

The main bulk is a lot of auto-generated tests based on a small set of
seed values for lower and upper 32 bits of full 64-bit values.
Currently we validate only range vs const comparisons, but the idea is
to start validating range over range comparisons in subsequent patch set.

When setting up initial register ranges we treat registers as one of
u64/s64/u32/s32 numeric types, and then independently perform conditional
comparisons based on a potentially different u64/s64/u32/s32 types. This
tests lots of tricky cases of deriving bounds information across
different numeric domains.

Given there are lots of auto-generated cases, we guard them behind
SLOW_TESTS=1 envvar requirement, and skip them altogether otherwise.
With current full set of upper/lower seed value, all supported
comparison operators and all the combinations of u64/s64/u32/s32 number
domains, we get about 7.7 million tests, which run in about 35 minutes
on my local qemu instance without parallelization. But we also split
those tests by init/cond numeric types, which allows to rely on
test_progs's parallelization of tests with `-j` option, getting run time
down to about 5 minutes on 8 cores. It's still something that shouldn't
be run during normal test_progs run.  But we can run it a reasonable
time, and so perhaps a nightly CI test run (once we have it) would be
a good option for this.

We also add a small set of tricky conditions that came up during
development and triggered various bugs or corner cases in either
selftest's reimplementation of range bounds logic or in verifier's logic
itself. These are fast enough to be run as part of normal test_progs
test run and are great for a quick sanity checking.

Let's take a look at test output to understand what's going on:

  $ sudo ./test_progs -t reg_bounds_crafted
  #191/1   reg_bounds_crafted/(u64)[0; 0xffffffff] (u64)< 0:OK
  ...
  #191/115 reg_bounds_crafted/(u64)[0; 0x17fffffff] (s32)< 0:OK
  ...
  #191/137 reg_bounds_crafted/(u64)[0xffffffff; 0x100000000] (u64)== 0:OK

Each test case is uniquely and fully described by this generated string.
E.g.: "(u64)[0; 0x17fffffff] (s32)< 0". This means that we
initialize a register (R6) in such a way that verifier knows that it can
have a value in [(u64)0; (u64)0x17fffffff] range. Another
register (R7) is also set up as u64, but this time a constant (zero in
this case). They then are compared using 32-bit signed < operation.
Resulting TRUE/FALSE branches are evaluated (including cases where it's
known that one of the branches will never be taken, in which case we
validate that verifier also determines this as a dead code). Test
validates that verifier's final register state matches expected state
based on selftest's own reg_state logic, implemented from scratch for
cross-checking purposes.

These test names can be conveniently used for further debugging, and if -vv
verboseness is requested we can get a corresponding verifier log (with
mark_precise logs filtered out as irrelevant and distracting). Example below is
slightly redacted for brevity, omitting irrelevant register output in
some places, marked with [...].

  $ sudo ./test_progs -a 'reg_bounds_crafted/(u32)[0; U32_MAX] (s32)< -1' -vv
  ...
  VERIFIER LOG:
  ========================
  func#0 @0
  0: R1=ctx(off=0,imm=0) R10=fp0
  0: (05) goto pc+2
  3: (85) call bpf_get_current_pid_tgid#14      ; R0_w=scalar()
  4: (bc) w6 = w0                       ; R0_w=scalar() R6_w=scalar(smin=0,smax=umax=4294967295,var_off=(0x0; 0xffffffff))
  5: (85) call bpf_get_current_pid_tgid#14      ; R0_w=scalar()
  6: (bc) w7 = w0                       ; R0_w=scalar() R7_w=scalar(smin=0,smax=umax=4294967295,var_off=(0x0; 0xffffffff))
  7: (b4) w1 = 0                        ; R1_w=0
  8: (b4) w2 = -1                       ; R2=4294967295
  9: (ae) if w6 < w1 goto pc-9
  9: R1=0 R6=scalar(smin=0,smax=umax=4294967295,var_off=(0x0; 0xffffffff))
  10: (2e) if w6 > w2 goto pc-10
  10: R2=4294967295 R6=scalar(smin=0,smax=umax=4294967295,var_off=(0x0; 0xffffffff))
  11: (b4) w1 = -1                      ; R1_w=4294967295
  12: (b4) w2 = -1                      ; R2_w=4294967295
  13: (ae) if w7 < w1 goto pc-13        ; R1_w=4294967295 R7=4294967295
  14: (2e) if w7 > w2 goto pc-14
  14: R2_w=4294967295 R7=4294967295
  15: (bc) w0 = w6                      ; [...] R6=scalar(id=1,smin=0,smax=umax=4294967295,var_off=(0x0; 0xffffffff))
  16: (bc) w0 = w7                      ; [...] R7=4294967295
  17: (ce) if w6 s< w7 goto pc+3        ; R6=scalar(id=1,smin=0,smax=umax=4294967295,smin32=-1,var_off=(0x0; 0xffffffff)) R7=4294967295
  18: (bc) w0 = w6                      ; [...] R6=scalar(id=1,smin=0,smax=umax=4294967295,smin32=-1,var_off=(0x0; 0xffffffff))
  19: (bc) w0 = w7                      ; [...] R7=4294967295
  20: (95) exit

  from 17 to 21: [...]
  21: (bc) w0 = w6                      ; [...] R6=scalar(id=1,smin=umin=umin32=2147483648,smax=umax=umax32=4294967294,smax32=-2,var_off=(0x80000000; 0x7fffffff))
  22: (bc) w0 = w7                      ; [...] R7=4294967295
  23: (95) exit

  from 13 to 1: [...]
  1: [...]
  1: (b7) r0 = 0                        ; R0_w=0
  2: (95) exit
  processed 24 insns (limit 1000000) max_states_per_insn 0 total_states 2 peak_states 2 mark_read 1
  =====================

Verifier log above is for `(u32)[0; U32_MAX] (s32)< -1` use cases, where u32
range is used for initialization, followed by signed < operator. Note
how we use w6/w7 in this case for register initialization (it would be
R6/R7 for 64-bit types) and then `if w6 s< w7` for comparison at
instruction #17. It will be `if R6 < R7` for 64-bit unsigned comparison.
Above example gives a good impression of the overall structure of a BPF
programs generated for reg_bounds tests.

In the future, this "framework" can be extended to test not just
conditional jumps, but also arithmetic operations. Adding randomized
testing is another possibility.

Some implementation notes. We basically have our own generics-like
operations on numbers, where all the numbers are stored in u64, but how
they are interpreted is passed as runtime argument enum num_t. Further,
`struct range` represents a bounds range, and those are collected
together into a minimal `struct reg_state`, which collects range bounds
across all four numberical domains: u64, s64, u32, s64.

Based on these primitives and `enum op` representing possible
conditional operation (<, <=, >, >=, ==, !=), there is a set of generic
helpers to perform "range arithmetics", which is used to maintain struct
reg_state. We simulate what verifier will do for reg bounds of R6 and R7
registers using these range and reg_state primitives. Simulated
information is used to determine branch taken conclusion and expected
exact register state across all four number domains.

Implementation of "range arithmetics" is more generic than what verifier
is currently performing: it allows range over range comparisons and
adjustments. This is the intended end goal of this patch set overall and verifier
logic is enhanced in subsequent patches in this series to handle range
vs range operations, at which point selftests are extended to validate
these conditions as well. For now it's range vs const cases only.

Note that tests are split into multiple groups by their numeric types
for initialization of ranges and for comparison operation. This allows
to use test_progs's -j parallelization to speed up tests, as we now have
16 groups of parallel running tests. Overall reduction of running time
that allows is pretty good, we go down from more than 30 minutes to
slightly less than 5 minutes running time.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 .../selftests/bpf/prog_tests/reg_bounds.c     | 1841 +++++++++++++++++
 1 file changed, 1841 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/reg_bounds.c

diff --git a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
new file mode 100644
index 000000000000..ac7354cfe139
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
@@ -0,0 +1,1841 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */
+
+#define _GNU_SOURCE
+#include <limits.h>
+#include <test_progs.h>
+#include <linux/filter.h>
+#include <linux/bpf.h>
+
+/* =================================
+ * SHORT AND CONSISTENT NUMBER TYPES
+ * =================================
+ */
+#define U64_MAX ((u64)UINT64_MAX)
+#define U32_MAX ((u32)UINT_MAX)
+#define S64_MIN ((s64)INT64_MIN)
+#define S64_MAX ((s64)INT64_MAX)
+#define S32_MIN ((s32)INT_MIN)
+#define S32_MAX ((s32)INT_MAX)
+
+typedef unsigned long long ___u64;
+typedef unsigned int ___u32;
+typedef long long ___s64;
+typedef int ___s32;
+
+/* avoid conflicts with already defined types in kernel headers */
+#define u64 ___u64
+#define u32 ___u32
+#define s64 ___s64
+#define s32 ___s32
+
+/* ==================================
+ * STRING BUF ABSTRACTION AND HELPERS
+ * ==================================
+ */
+struct strbuf {
+	size_t buf_sz;
+	int pos;
+	char buf[0];
+};
+
+#define DEFINE_STRBUF(name, N)						\
+	struct { struct strbuf buf; char data[(N)]; } ___##name;	\
+	struct strbuf *name = (___##name.buf.buf_sz = (N), ___##name.buf.pos = 0, &___##name.buf)
+
+__printf(2, 3)
+static inline void snappendf(struct strbuf *s, const char *fmt, ...)
+{
+	va_list args;
+
+	va_start(args, fmt);
+	s->pos += vsnprintf(s->buf + s->pos, s->buf_sz - s->pos, fmt, args);
+	va_end(args);
+}
+
+/* ==================================
+ * GENERIC NUMBER TYPE AND OPERATIONS
+ * ==================================
+ */
+enum num_t { U64, U32, S64, S32 };
+#define MIN_T U64
+#define MAX_T S32
+
+static __always_inline u64 min_t(enum num_t t, u64 x, u64 y)
+{
+	switch (t) {
+	case U64: return (u64)x < (u64)y ? (u64)x : (u64)y;
+	case U32: return (u32)x < (u32)y ? (u32)x : (u32)y;
+	case S64: return (s64)x < (s64)y ? (s64)x : (s64)y;
+	case S32: return (s32)x < (s32)y ? (s32)x : (s32)y;
+	default: printf("min_t!\n"); exit(1);
+	}
+}
+
+static __always_inline u64 max_t(enum num_t t, u64 x, u64 y)
+{
+	switch (t) {
+	case U64: return (u64)x > (u64)y ? (u64)x : (u64)y;
+	case U32: return (u32)x > (u32)y ? (u32)x : (u32)y;
+	case S64: return (s64)x > (s64)y ? (s64)x : (s64)y;
+	case S32: return (s32)x > (s32)y ? (u32)(s32)x : (u32)(s32)y;
+	default: printf("max_t!\n"); exit(1);
+	}
+}
+
+static const char *t_str(enum num_t t)
+{
+	switch (t) {
+	case U64: return "u64";
+	case U32: return "u32";
+	case S64: return "s64";
+	case S32: return "s32";
+	default: printf("t_str!\n"); exit(1);
+	}
+}
+
+static enum num_t t_is_32(enum num_t t)
+{
+	switch (t) {
+	case U64: return false;
+	case U32: return true;
+	case S64: return false;
+	case S32: return true;
+	default: printf("t_is_32!\n"); exit(1);
+	}
+}
+
+static enum num_t t_signed(enum num_t t)
+{
+	switch (t) {
+	case U64: return S64;
+	case U32: return S32;
+	case S64: return S64;
+	case S32: return S32;
+	default: printf("t_signed!\n"); exit(1);
+	}
+}
+
+static enum num_t t_unsigned(enum num_t t)
+{
+	switch (t) {
+	case U64: return U64;
+	case U32: return U32;
+	case S64: return U64;
+	case S32: return U32;
+	default: printf("t_unsigned!\n"); exit(1);
+	}
+}
+
+static bool num_is_small(enum num_t t, u64 x)
+{
+	switch (t) {
+	case U64: return (u64)x <= 256;
+	case U32: return (u32)x <= 256;
+	case S64: return (s64)x >= -256 && (s64)x <= 256;
+	case S32: return (s32)x >= -256 && (s32)x <= 256;
+	default: printf("num_is_small!\n"); exit(1);
+	}
+}
+
+static void snprintf_num(enum num_t t, struct strbuf *sb, u64 x)
+{
+	bool is_small = num_is_small(t, x);
+
+	if (is_small) {
+		switch (t) {
+		case U64: return snappendf(sb, "%llu", (u64)x);
+		case U32: return snappendf(sb, "%u", (u32)x);
+		case S64: return snappendf(sb, "%lld", (s64)x);
+		case S32: return snappendf(sb, "%d", (s32)x);
+		default: printf("snprintf_num!\n"); exit(1);
+		}
+	} else {
+		switch (t) {
+		case U64:
+			if (x == U64_MAX)
+				return snappendf(sb, "U64_MAX");
+			else if (x >= U64_MAX - 256)
+				return snappendf(sb, "U64_MAX-%llu", U64_MAX - x);
+			else
+				return snappendf(sb, "%#llx", (u64)x);
+		case U32:
+			if ((u32)x == U32_MAX)
+				return snappendf(sb, "U32_MAX");
+			else if ((u32)x >= U32_MAX - 256)
+				return snappendf(sb, "U32_MAX-%u", U32_MAX - (u32)x);
+			else
+				return snappendf(sb, "%#x", (u32)x);
+		case S64:
+			if ((s64)x == S64_MAX)
+				return snappendf(sb, "S64_MAX");
+			else if ((s64)x >= S64_MAX - 256)
+				return snappendf(sb, "S64_MAX-%lld", S64_MAX - (s64)x);
+			else if ((s64)x == S64_MIN)
+				return snappendf(sb, "S64_MIN");
+			else if ((s64)x <= S64_MIN + 256)
+				return snappendf(sb, "S64_MIN+%lld", (s64)x - S64_MIN);
+			else
+				return snappendf(sb, "%#llx", (s64)x);
+		case S32:
+			if ((s32)x == S32_MAX)
+				return snappendf(sb, "S32_MAX");
+			else if ((s32)x >= S32_MAX - 256)
+				return snappendf(sb, "S32_MAX-%d", S32_MAX - (s32)x);
+			else if ((s32)x == S32_MIN)
+				return snappendf(sb, "S32_MIN");
+			else if ((s32)x <= S32_MIN + 256)
+				return snappendf(sb, "S32_MIN+%d", (s32)x - S32_MIN);
+			else
+				return snappendf(sb, "%#x", (s32)x);
+		default: printf("snprintf_num!\n"); exit(1);
+		}
+	}
+}
+
+/* ===================================
+ * GENERIC RANGE STRUCT AND OPERATIONS
+ * ===================================
+ */
+struct range {
+	u64 a, b;
+};
+
+static void snprintf_range(enum num_t t, struct strbuf *sb, struct range x)
+{
+	if (x.a == x.b)
+		return snprintf_num(t, sb, x.a);
+
+	snappendf(sb, "[");
+	snprintf_num(t, sb, x.a);
+	snappendf(sb, "; ");
+	snprintf_num(t, sb, x.b);
+	snappendf(sb, "]");
+}
+
+static void print_range(enum num_t t, struct range x, const char *sfx)
+{
+	DEFINE_STRBUF(sb, 128);
+
+	snprintf_range(t, sb, x);
+	printf("%s%s", sb->buf, sfx);
+}
+
+static const struct range unkn[] = {
+	[U64] = { 0, U64_MAX },
+	[U32] = { 0, U32_MAX },
+	[S64] = { (u64)S64_MIN, (u64)S64_MAX },
+	[S32] = { (u64)(u32)S32_MIN, (u64)(u32)S32_MAX },
+};
+
+static struct range unkn_subreg(enum num_t t)
+{
+	switch (t) {
+	case U64: return unkn[U32];
+	case U32: return unkn[U32];
+	case S64: return unkn[U32];
+	case S32: return unkn[S32];
+	default: printf("unkn_subreg!\n"); exit(1);
+	}
+}
+
+static struct range range(enum num_t t, u64 a, u64 b)
+{
+	switch (t) {
+	case U64: return (struct range){ (u64)a, (u64)b };
+	case U32: return (struct range){ (u32)a, (u32)b };
+	case S64: return (struct range){ (s64)a, (s64)b };
+	case S32: return (struct range){ (u32)(s32)a, (u32)(s32)b };
+	default: printf("range!\n"); exit(1);
+	}
+}
+
+static __always_inline u32 sign64(u64 x) { return (x >> 63) & 1; }
+static __always_inline u32 sign32(u64 x) { return ((u32)x >> 31) & 1; }
+static __always_inline u32 upper32(u64 x) { return (u32)(x >> 32); }
+static __always_inline u64 swap_low32(u64 x, u32 y) { return (x & 0xffffffff00000000ULL) | y; }
+
+static bool range_eq(struct range x, struct range y)
+{
+	return x.a == y.a && x.b == y.b;
+}
+
+static struct range range_cast_to_s32(struct range x)
+{
+	u64 a = x.a, b = x.b;
+
+	/* if upper 32 bits are constant, lower 32 bits should form a proper
+	 * s32 range to be correct
+	 */
+	if (upper32(a) == upper32(b) && (s32)a <= (s32)b)
+		return range(S32, a, b);
+
+	/* Special case where upper bits form a small sequence of two
+	 * sequential numbers (in 32-bit unsigned space, so 0xffffffff to
+	 * 0x00000000 is also valid), while lower bits form a proper s32 range
+	 * going from negative numbers to positive numbers.
+	 *
+	 * E.g.: [0xfffffff0ffffff00; 0xfffffff100000010]. Iterating
+	 * over full 64-bit numbers range will form a proper [-16, 16]
+	 * ([0xffffff00; 0x00000010]) range in its lower 32 bits.
+	 */
+	if (upper32(a) + 1 == upper32(b) && (s32)a < 0 && (s32)b >= 0)
+		return range(S32, a, b);
+
+	/* otherwise we can't derive much meaningful information */
+	return unkn[S32];
+}
+
+static struct range range_cast_u64(enum num_t to_t, struct range x)
+{
+	u64 a = (u64)x.a, b = (u64)x.b;
+
+	switch (to_t) {
+	case U64:
+		return x;
+	case U32:
+		if (upper32(a) != upper32(b))
+			return unkn[U32];
+		return range(U32, a, b);
+	case S64:
+		if (sign64(a) != sign64(b))
+			return unkn[S64];
+		return range(S64, a, b);
+	case S32:
+		return range_cast_to_s32(x);
+	default: printf("range_cast_u64!\n"); exit(1);
+	}
+}
+
+static struct range range_cast_s64(enum num_t to_t, struct range x)
+{
+	s64 a = (s64)x.a, b = (s64)x.b;
+
+	switch (to_t) {
+	case U64:
+		/* equivalent to (s64)a <= (s64)b check */
+		if (sign64(a) != sign64(b))
+			return unkn[U64];
+		return range(U64, a, b);
+	case U32:
+		if (upper32(a) != upper32(b) || sign32(a) != sign32(b))
+			return unkn[U32];
+		return range(U32, a, b);
+	case S64:
+		return x;
+	case S32:
+		return range_cast_to_s32(x);
+	default: printf("range_cast_s64!\n"); exit(1);
+	}
+}
+
+static struct range range_cast_u32(enum num_t to_t, struct range x)
+{
+	u32 a = (u32)x.a, b = (u32)x.b;
+
+	switch (to_t) {
+	case U64:
+	case S64:
+		/* u32 is always a valid zero-extended u64/s64 */
+		return range(to_t, a, b);
+	case U32:
+		return x;
+	case S32:
+		return range_cast_to_s32(range(U32, a, b));
+	default: printf("range_cast_u32!\n"); exit(1);
+	}
+}
+
+static struct range range_cast_s32(enum num_t to_t, struct range x)
+{
+	s32 a = (s32)x.a, b = (s32)x.b;
+
+	switch (to_t) {
+	case U64:
+	case U32:
+	case S64:
+		if (sign32(a) != sign32(b))
+			return unkn[to_t];
+		return range(to_t, a, b);
+	case S32:
+		return x;
+	default: printf("range_cast_s32!\n"); exit(1);
+	}
+}
+
+/* Reinterpret range in *from_t* domain as a range in *to_t* domain preserving
+ * all possible information. Worst case, it will be unknown range within
+ * *to_t* domain, if nothing more specific can be guaranteed during the
+ * conversion
+ */
+static struct range range_cast(enum num_t from_t, enum num_t to_t, struct range from)
+{
+	switch (from_t) {
+	case U64: return range_cast_u64(to_t, from);
+	case U32: return range_cast_u32(to_t, from);
+	case S64: return range_cast_s64(to_t, from);
+	case S32: return range_cast_s32(to_t, from);
+	default: printf("range_cast!\n"); exit(1);
+	}
+}
+
+static bool is_valid_num(enum num_t t, u64 x)
+{
+	switch (t) {
+	case U64: return true;
+	case U32: return upper32(x) == 0;
+	case S64: return true;
+	case S32: return upper32(x) == 0;
+	default: printf("is_valid_num!\n"); exit(1);
+	}
+}
+
+static bool is_valid_range(enum num_t t, struct range x)
+{
+	if (!is_valid_num(t, x.a) || !is_valid_num(t, x.b))
+		return false;
+
+	switch (t) {
+	case U64: return (u64)x.a <= (u64)x.b;
+	case U32: return (u32)x.a <= (u32)x.b;
+	case S64: return (s64)x.a <= (s64)x.b;
+	case S32: return (s32)x.a <= (s32)x.b;
+	default: printf("is_valid_range!\n"); exit(1);
+	}
+}
+
+static struct range range_improve(enum num_t t, struct range old, struct range new)
+{
+	return range(t, max_t(t, old.a, new.a), min_t(t, old.b, new.b));
+}
+
+static struct range range_refine(enum num_t x_t, struct range x, enum num_t y_t, struct range y)
+{
+	struct range y_cast;
+
+	y_cast = range_cast(y_t, x_t, y);
+
+	/* the case when new range knowledge, *y*, is a 32-bit subregister
+	 * range, while previous range knowledge, *x*, is a full register
+	 * 64-bit range, needs special treatment to take into account upper 32
+	 * bits of full register range
+	 */
+	if (t_is_32(y_t) && !t_is_32(x_t)) {
+		struct range x_swap;
+
+		/* some combinations of upper 32 bits and sign bit can lead to
+		 * invalid ranges, in such cases it's easier to detect them
+		 * after cast/swap than try to enumerate all the conditions
+		 * under which transformation and knowledge transfer is valid
+		 */
+		x_swap = range(x_t, swap_low32(x.a, y_cast.a), swap_low32(x.b, y_cast.b));
+		if (!is_valid_range(x_t, x_swap))
+			return x;
+		return range_improve(x_t, x, x_swap);
+	}
+
+	/* otherwise, plain range cast and intersection works */
+	return range_improve(x_t, x, y_cast);
+}
+
+/* =======================
+ * GENERIC CONDITIONAL OPS
+ * =======================
+ */
+enum op { OP_LT, OP_LE, OP_GT, OP_GE, OP_EQ, OP_NE };
+#define MIN_OP OP_LT
+#define MAX_OP OP_NE
+
+static enum op complement_op(enum op op)
+{
+	switch (op) {
+	case OP_LT: return OP_GE;
+	case OP_LE: return OP_GT;
+	case OP_GT: return OP_LE;
+	case OP_GE: return OP_LT;
+	case OP_EQ: return OP_NE;
+	case OP_NE: return OP_EQ;
+	default: printf("complement_op!\n"); exit(1);
+	}
+}
+
+static const char *op_str(enum op op)
+{
+	switch (op) {
+	case OP_LT: return "<";
+	case OP_LE: return "<=";
+	case OP_GT: return ">";
+	case OP_GE: return ">=";
+	case OP_EQ: return "==";
+	case OP_NE: return "!=";
+	default: printf("op_str!\n"); exit(1);
+	}
+}
+
+/* Can register with range [x.a, x.b] *EVER* satisfy
+ * OP (<, <=, >, >=, ==, !=) relation to
+ * a regsiter with range [y.a, y.b]
+ * _in *num_t* domain_
+ */
+static bool range_canbe_op(enum num_t t, struct range x, struct range y, enum op op)
+{
+#define range_canbe(T) do {									\
+	switch (op) {										\
+	case OP_LT: return (T)x.a < (T)y.b;							\
+	case OP_LE: return (T)x.a <= (T)y.b;							\
+	case OP_GT: return (T)x.b > (T)y.a;							\
+	case OP_GE: return (T)x.b >= (T)y.a;							\
+	case OP_EQ: return (T)max_t(t, x.a, y.a) <= (T)min_t(t, x.b, y.b);			\
+	case OP_NE: return !((T)x.a == (T)x.b && (T)y.a == (T)y.b && (T)x.a == (T)y.a);		\
+	default: printf("range_canbe op %d\n", op); exit(1);					\
+	}											\
+} while (0)
+
+	switch (t) {
+	case U64: { range_canbe(u64); }
+	case U32: { range_canbe(u32); }
+	case S64: { range_canbe(s64); }
+	case S32: { range_canbe(s32); }
+	default: printf("range_canbe!\n"); exit(1);
+	}
+#undef range_canbe
+}
+
+/* Does register with range [x.a, x.b] *ALWAYS* satisfy
+ * OP (<, <=, >, >=, ==, !=) relation to
+ * a regsiter with range [y.a, y.b]
+ * _in *num_t* domain_
+ */
+static bool range_always_op(enum num_t t, struct range x, struct range y, enum op op)
+{
+	/* always op <=> ! canbe complement(op) */
+	return !range_canbe_op(t, x, y, complement_op(op));
+}
+
+/* Does register with range [x.a, x.b] *NEVER* satisfy
+ * OP (<, <=, >, >=, ==, !=) relation to
+ * a regsiter with range [y.a, y.b]
+ * _in *num_t* domain_
+ */
+static bool range_never_op(enum num_t t, struct range x, struct range y, enum op op)
+{
+	return !range_canbe_op(t, x, y, op);
+}
+
+/* similar to verifier's is_branch_taken():
+ *    1 - always taken;
+ *    0 - never taken,
+ *   -1 - unsure.
+ */
+static int range_branch_taken_op(enum num_t t, struct range x, struct range y, enum op op)
+{
+	if (range_always_op(t, x, y, op))
+		return 1;
+	if (range_never_op(t, x, y, op))
+		return 0;
+	return -1;
+}
+
+/* What would be the new estimates for register x and y ranges assuming truthful
+ * OP comparison between them. I.e., (x OP y == true) => x <- newx, y <- newy.
+ *
+ * We assume "interesting" cases where ranges overlap. Cases where it's
+ * obvious that (x OP y) is either always true or false should be filtered with
+ * range_never and range_always checks.
+ */
+static void range_cond(enum num_t t, struct range x, struct range y,
+		       enum op op, struct range *newx, struct range *newy)
+{
+	if (!range_canbe_op(t, x, y, op)) {
+		/* nothing to adjust, can't happen, return original values */
+		*newx = x;
+		*newy = y;
+		return;
+	}
+	switch (op) {
+	case OP_LT:
+		*newx = range(t, x.a, min_t(t, x.b, y.b - 1));
+		*newy = range(t, max_t(t, x.a + 1, y.a), y.b);
+		break;
+	case OP_LE:
+		*newx = range(t, x.a, min_t(t, x.b, y.b));
+		*newy = range(t, max_t(t, x.a, y.a), y.b);
+		break;
+	case OP_GT:
+		*newx = range(t, max_t(t, x.a, y.a + 1), x.b);
+		*newy = range(t, y.a, min_t(t, x.b - 1, y.b));
+		break;
+	case OP_GE:
+		*newx = range(t, max_t(t, x.a, y.a), x.b);
+		*newy = range(t, y.a, min_t(t, x.b, y.b));
+		break;
+	case OP_EQ:
+		*newx = range(t, max_t(t, x.a, y.a), min_t(t, x.b, y.b));
+		*newy = range(t, max_t(t, x.a, y.a), min_t(t, x.b, y.b));
+		break;
+	case OP_NE:
+		/* generic case, can't derive more information */
+		*newx = range(t, x.a, x.b);
+		*newy = range(t, y.a, y.b);
+		break;
+
+		/* below extended logic is not supported by verifier just yet */
+		if (x.a == x.b && x.a == y.a) {
+			/* X is a constant matching left side of Y */
+			*newx = range(t, x.a, x.b);
+			*newy = range(t, y.a + 1, y.b);
+		} else if (x.a == x.b && x.b == y.b) {
+			/* X is a constant matching rigth side of Y */
+			*newx = range(t, x.a, x.b);
+			*newy = range(t, y.a, y.b - 1);
+		} else if (y.a == y.b && x.a == y.a) {
+			/* Y is a constant matching left side of X */
+			*newx = range(t, x.a + 1, x.b);
+			*newy = range(t, y.a, y.b);
+		} else if (y.a == y.b && x.b == y.b) {
+			/* Y is a constant matching rigth side of X */
+			*newx = range(t, x.a, x.b - 1);
+			*newy = range(t, y.a, y.b);
+		} else {
+			/* generic case, can't derive more information */
+			*newx = range(t, x.a, x.b);
+			*newy = range(t, y.a, y.b);
+		}
+
+		break;
+	default:
+		break;
+	}
+}
+
+/* =======================
+ * REGISTER STATE HANDLING
+ * =======================
+ */
+struct reg_state {
+	struct range r[4]; /* indexed by enum num_t: U64, U32, S64, S32 */
+	bool valid;
+};
+
+static void print_reg_state(struct reg_state *r, const char *sfx)
+{
+	DEFINE_STRBUF(sb, 512);
+	enum num_t t;
+	int cnt = 0;
+
+	if (!r->valid) {
+		printf("<not found>%s", sfx);
+		return;
+	}
+
+	snappendf(sb, "scalar(");
+	for (t = MIN_T; t <= MAX_T; t++) {
+		snappendf(sb, "%s%s=", cnt++ ? "," : "", t_str(t));
+		snprintf_range(t, sb, r->r[t]);
+	}
+	snappendf(sb, ")");
+
+	printf("%s%s", sb->buf, sfx);
+}
+
+static void print_refinement(enum num_t s_t, struct range src,
+			     enum num_t d_t, struct range old, struct range new,
+			     const char *ctx)
+{
+	printf("REFINING (%s) (%s)SRC=", ctx, t_str(s_t));
+	print_range(s_t, src, "");
+	printf(" (%s)DST_OLD=", t_str(d_t));
+	print_range(d_t, old, "");
+	printf(" (%s)DST_NEW=", t_str(d_t));
+	print_range(d_t, new, "\n");
+}
+
+static void reg_state_refine(struct reg_state *r, enum num_t t, struct range x, const char *ctx)
+{
+	enum num_t d_t, s_t;
+	struct range old;
+	bool keep_going = false;
+
+again:
+	/* try to derive new knowledge from just learned range x of type t */
+	for (d_t = MIN_T; d_t <= MAX_T; d_t++) {
+		old = r->r[d_t];
+		r->r[d_t] = range_refine(d_t, r->r[d_t], t, x);
+		if (!range_eq(r->r[d_t], old)) {
+			keep_going = true;
+			if (env.verbosity >= VERBOSE_VERY)
+				print_refinement(t, x, d_t, old, r->r[d_t], ctx);
+		}
+	}
+
+	/* now see if we can derive anything new from updated reg_state's ranges */
+	for (s_t = MIN_T; s_t <= MAX_T; s_t++) {
+		for (d_t = MIN_T; d_t <= MAX_T; d_t++) {
+			old = r->r[d_t];
+			r->r[d_t] = range_refine(d_t, r->r[d_t], s_t, r->r[s_t]);
+			if (!range_eq(r->r[d_t], old)) {
+				keep_going = true;
+				if (env.verbosity >= VERBOSE_VERY)
+					print_refinement(s_t, r->r[s_t], d_t, old, r->r[d_t], ctx);
+			}
+		}
+	}
+
+	/* keep refining until we converge */
+	if (keep_going) {
+		keep_going = false;
+		goto again;
+	}
+}
+
+static void reg_state_set_const(struct reg_state *rs, enum num_t t, u64 val)
+{
+	enum num_t tt;
+
+	rs->valid = true;
+	for (tt = MIN_T; tt <= MAX_T; tt++)
+		rs->r[tt] = tt == t ? range(t, val, val) : unkn[tt];
+
+	reg_state_refine(rs, t, rs->r[t], "CONST");
+}
+
+static void reg_state_cond(enum num_t t, struct reg_state *x, struct reg_state *y, enum op op,
+			   struct reg_state *newx, struct reg_state *newy, const char *ctx)
+{
+	char buf[32];
+	enum num_t ts[2];
+	struct reg_state xx = *x, yy = *y;
+	int i, t_cnt;
+	struct range z1, z2;
+
+	if (op == OP_EQ || op == OP_NE) {
+		/* OP_EQ and OP_NE are sign-agnostic, so we need to process
+		 * both signed and unsigned domains at the same time
+		 */
+		ts[0] = t_unsigned(t);
+		ts[1] = t_signed(t);
+		t_cnt = 2;
+	} else {
+		ts[0] = t;
+		t_cnt = 1;
+	}
+
+	for (i = 0; i < t_cnt; i++) {
+		t = ts[i];
+		z1 = x->r[t];
+		z2 = y->r[t];
+
+		range_cond(t, z1, z2, op, &z1, &z2);
+
+		if (newx) {
+			snprintf(buf, sizeof(buf), "%s R1", ctx);
+			reg_state_refine(&xx, t, z1, buf);
+		}
+		if (newy) {
+			snprintf(buf, sizeof(buf), "%s R2", ctx);
+			reg_state_refine(&yy, t, z2, buf);
+		}
+	}
+
+	if (newx)
+		*newx = xx;
+	if (newy)
+		*newy = yy;
+}
+
+static int reg_state_branch_taken_op(enum num_t t, struct reg_state *x, struct reg_state *y,
+				     enum op op)
+{
+	if (op == OP_EQ || op == OP_NE) {
+		/* OP_EQ and OP_NE are sign-agnostic */
+		enum num_t tu = t_unsigned(t);
+		enum num_t ts = t_signed(t);
+		int br_u, br_s;
+
+		br_u = range_branch_taken_op(tu, x->r[tu], y->r[tu], op);
+		br_s = range_branch_taken_op(ts, x->r[ts], y->r[ts], op);
+
+		if (br_u >= 0 && br_s >= 0 && br_u != br_s)
+			ASSERT_FALSE(true, "branch taken inconsistency!\n");
+		if (br_u >= 0)
+			return br_u;
+		return br_s;
+	}
+	return range_branch_taken_op(t, x->r[t], y->r[t], op);
+}
+
+/* =====================================
+ * BPF PROGS GENERATION AND VERIFICATION
+ * =====================================
+ */
+struct case_spec {
+	/* whether to init full register (r1) or sub-register (w1) */
+	bool init_subregs;
+	/* whether to establish initial value range on full register (r1) or
+	 * sub-register (w1)
+	 */
+	bool setup_subregs;
+	/* whether to establish initial value range using signed or unsigned
+	 * comparisons (i.e., initialize umin/umax or smin/smax directly)
+	 */
+	bool setup_signed;
+	/* whether to perform comparison on full registers or sub-registers */
+	bool compare_subregs;
+	/* whether to perform comparison using signed or unsigned operations */
+	bool compare_signed;
+};
+
+/* Generate test BPF program based on provided test ranges, operation, and
+ * specifications about register bitness and signedness.
+ */
+static int load_range_cmp_prog(struct range x, struct range y, enum op op,
+			       int branch_taken, struct case_spec spec,
+			       char *log_buf, size_t log_sz,
+			       int *false_pos, int *true_pos)
+{
+#define emit(insn) ({							\
+	struct bpf_insn __insns[] = { insn };				\
+	int __i;							\
+	for (__i = 0; __i < ARRAY_SIZE(__insns); __i++)			\
+		insns[cur_pos + __i] = __insns[__i];			\
+	cur_pos += __i;							\
+})
+#define JMP_TO(target) (target - cur_pos - 1)
+	int cur_pos = 0, exit_pos, fd, op_code;
+	struct bpf_insn insns[64];
+	LIBBPF_OPTS(bpf_prog_load_opts, opts,
+		.log_level = 2,
+		.log_buf = log_buf,
+		.log_size = log_sz,
+	);
+
+	/* ; skip exit block below
+	 * goto +2;
+	 */
+	emit(BPF_JMP_A(2));
+	exit_pos = cur_pos;
+	/* ; exit block for all the preparatory conditionals
+	 * out:
+	 * r0 = 0;
+	 * exit;
+	 */
+	emit(BPF_MOV64_IMM(BPF_REG_0, 0));
+	emit(BPF_EXIT_INSN());
+	/*
+	 * ; assign r6/w6 and r7/w7 unpredictable u64/u32 value
+	 * call bpf_get_current_pid_tgid;
+	 * r6 = r0;               | w6 = w0;
+	 * call bpf_get_current_pid_tgid;
+	 * r7 = r0;               | w7 = w0;
+	 */
+	emit(BPF_EMIT_CALL(BPF_FUNC_get_current_pid_tgid));
+	if (spec.init_subregs)
+		emit(BPF_MOV32_REG(BPF_REG_6, BPF_REG_0));
+	else
+		emit(BPF_MOV64_REG(BPF_REG_6, BPF_REG_0));
+	emit(BPF_EMIT_CALL(BPF_FUNC_get_current_pid_tgid));
+	if (spec.init_subregs)
+		emit(BPF_MOV32_REG(BPF_REG_7, BPF_REG_0));
+	else
+		emit(BPF_MOV64_REG(BPF_REG_7, BPF_REG_0));
+	/* ; setup initial r6/w6 possible value range ([x.a, x.b])
+	 * r1 = %[x.a] ll;        | w1 = %[x.a];
+	 * r2 = %[x.b] ll;        | w2 = %[x.b];
+	 * if r6 < r1 goto out;   | if w6 < w1 goto out;
+	 * if r6 > r2 goto out;   | if w6 > w2 goto out;
+	 */
+	if (spec.setup_subregs) {
+		emit(BPF_MOV32_IMM(BPF_REG_1, (s32)x.a));
+		emit(BPF_MOV32_IMM(BPF_REG_2, (s32)x.b));
+		emit(BPF_JMP32_REG(spec.setup_signed ? BPF_JSLT : BPF_JLT,
+				   BPF_REG_6, BPF_REG_1, JMP_TO(exit_pos)));
+		emit(BPF_JMP32_REG(spec.setup_signed ? BPF_JSGT : BPF_JGT,
+				   BPF_REG_6, BPF_REG_2, JMP_TO(exit_pos)));
+	} else {
+		emit(BPF_LD_IMM64(BPF_REG_1, x.a));
+		emit(BPF_LD_IMM64(BPF_REG_2, x.b));
+		emit(BPF_JMP_REG(spec.setup_signed ? BPF_JSLT : BPF_JLT,
+				 BPF_REG_6, BPF_REG_1, JMP_TO(exit_pos)));
+		emit(BPF_JMP_REG(spec.setup_signed ? BPF_JSGT : BPF_JGT,
+				 BPF_REG_6, BPF_REG_2, JMP_TO(exit_pos)));
+	}
+	/* ; setup initial r7/w7 possible value range ([y.a, y.b])
+	 * r1 = %[y.a] ll;        | w1 = %[y.a];
+	 * r2 = %[y.b] ll;        | w2 = %[y.b];
+	 * if r7 < r1 goto out;   | if w7 < w1 goto out;
+	 * if r7 > r2 goto out;   | if w7 > w2 goto out;
+	 */
+	if (spec.setup_subregs) {
+		emit(BPF_MOV32_IMM(BPF_REG_1, (s32)y.a));
+		emit(BPF_MOV32_IMM(BPF_REG_2, (s32)y.b));
+		emit(BPF_JMP32_REG(spec.setup_signed ? BPF_JSLT : BPF_JLT,
+				   BPF_REG_7, BPF_REG_1, JMP_TO(exit_pos)));
+		emit(BPF_JMP32_REG(spec.setup_signed ? BPF_JSGT : BPF_JGT,
+				   BPF_REG_7, BPF_REG_2, JMP_TO(exit_pos)));
+	} else {
+		emit(BPF_LD_IMM64(BPF_REG_1, y.a));
+		emit(BPF_LD_IMM64(BPF_REG_2, y.b));
+		emit(BPF_JMP_REG(spec.setup_signed ? BPF_JSLT : BPF_JLT,
+				 BPF_REG_7, BPF_REG_1, JMP_TO(exit_pos)));
+		emit(BPF_JMP_REG(spec.setup_signed ? BPF_JSGT : BPF_JGT,
+				 BPF_REG_7, BPF_REG_2, JMP_TO(exit_pos)));
+	}
+	/* ; range test instruction
+	 * if r6 <op> r7 goto +3; | if w6 <op> w7 goto +3;
+	 */
+	switch (op) {
+	case OP_LT: op_code = spec.compare_signed ? BPF_JSLT : BPF_JLT; break;
+	case OP_LE: op_code = spec.compare_signed ? BPF_JSLE : BPF_JLE; break;
+	case OP_GT: op_code = spec.compare_signed ? BPF_JSGT : BPF_JGT; break;
+	case OP_GE: op_code = spec.compare_signed ? BPF_JSGE : BPF_JGE; break;
+	case OP_EQ: op_code = BPF_JEQ; break;
+	case OP_NE: op_code = BPF_JNE; break;
+	default:
+		printf("unrecognized op %d\n", op);
+		return -ENOTSUP;
+	}
+	/* ; BEFORE conditiona, r0/w0 = {r6/w6,r7/w7} is to extract verifier state reliably
+	 * ; this is used for debugging, as verifier doesn't always print
+	 * ; registers states as of condition jump instruction (e.g., when
+	 * ; precision marking happens)
+	 * r0 = r6;               | w0 = w6;
+	 * r0 = r7;               | w0 = w7;
+	 */
+	if (spec.compare_subregs) {
+		emit(BPF_MOV32_REG(BPF_REG_0, BPF_REG_6));
+		emit(BPF_MOV32_REG(BPF_REG_0, BPF_REG_7));
+	} else {
+		emit(BPF_MOV64_REG(BPF_REG_0, BPF_REG_6));
+		emit(BPF_MOV64_REG(BPF_REG_0, BPF_REG_7));
+	}
+	if (spec.compare_subregs)
+		emit(BPF_JMP32_REG(op_code, BPF_REG_6, BPF_REG_7, 3));
+	else
+		emit(BPF_JMP_REG(op_code, BPF_REG_6, BPF_REG_7, 3));
+	/* ; FALSE branch, r0/w0 = {r6/w6,r7/w7} is to extract verifier state reliably
+	 * r0 = r6;               | w0 = w6;
+	 * r0 = r7;               | w0 = w7;
+	 * exit;
+	 */
+	*false_pos = cur_pos;
+	if (spec.compare_subregs) {
+		emit(BPF_MOV32_REG(BPF_REG_0, BPF_REG_6));
+		emit(BPF_MOV32_REG(BPF_REG_0, BPF_REG_7));
+	} else {
+		emit(BPF_MOV64_REG(BPF_REG_0, BPF_REG_6));
+		emit(BPF_MOV64_REG(BPF_REG_0, BPF_REG_7));
+	}
+	if (branch_taken == 1) /* false branch is never taken */
+		emit(BPF_EMIT_CALL(0xDEAD)); /* poison this branch */
+	else
+		emit(BPF_EXIT_INSN());
+	/* ; TRUE branch, r0/w0 = {r6/w6,r7/w7} is to extract verifier state reliably
+	 * r0 = r6;               | w0 = w6;
+	 * r0 = r7;               | w0 = w7;
+	 * exit;
+	 */
+	*true_pos = cur_pos;
+	if (spec.compare_subregs) {
+		emit(BPF_MOV32_REG(BPF_REG_0, BPF_REG_6));
+		emit(BPF_MOV32_REG(BPF_REG_0, BPF_REG_7));
+	} else {
+		emit(BPF_MOV64_REG(BPF_REG_0, BPF_REG_6));
+		emit(BPF_MOV64_REG(BPF_REG_0, BPF_REG_7));
+	}
+	if (branch_taken == 0) /* true branch is never taken */
+		emit(BPF_EMIT_CALL(0xDEAD)); /* poison this branch */
+	emit(BPF_EXIT_INSN()); /* last instruction has to be exit */
+
+	fd = bpf_prog_load(BPF_PROG_TYPE_RAW_TRACEPOINT, "reg_bounds_test",
+			   "GPL", insns, cur_pos, &opts);
+	if (fd < 0)
+		return fd;
+
+	close(fd);
+	return 0;
+#undef emit
+#undef JMP_TO
+}
+
+#define str_has_pfx(str, pfx) \
+	(strncmp(str, pfx, __builtin_constant_p(pfx) ? sizeof(pfx) - 1 : strlen(pfx)) == 0)
+
+/* Parse register state from verifier log.
+ * `s` should point to the start of "Rx = ..." substring in the verifier log.
+ */
+static int parse_reg_state(const char *s, struct reg_state *reg)
+{
+	/* There are two generic forms for SCALAR register:
+	 * - known constant: R6_rwD=P%lld
+	 * - range: R6_rwD=scalar(id=1,...), where "..." is a comma-separated
+	 *   list of optional range specifiers:
+	 *     - umin=%llu, if missing, assumed 0;
+	 *     - umax=%llu, if missing, assumed U64_MAX;
+	 *     - smin=%lld, if missing, assumed S64_MIN;
+	 *     - smax=%lld, if missing, assummed S64_MAX;
+	 *     - umin32=%d, if missing, assumed 0;
+	 *     - umax32=%d, if missing, assumed U32_MAX;
+	 *     - smin32=%d, if missing, assumed S32_MIN;
+	 *     - smax32=%d, if missing, assummed S32_MAX;
+	 *     - var_off=(%#llx; %#llx), tnum part, we don't care about it.
+	 *
+	 * If some of the values are equal, they will be grouped (but min/max
+	 * are not mixed together, and similarly negative values are not
+	 * grouped with non-negative ones). E.g.:
+	 *
+	 *   R6_w=Pscalar(smin=smin32=0, smax=umax=umax32=1000)
+	 *
+	 * _rwD part is optional (and any of the letters can be missing).
+	 * P (precision mark) is optional as well.
+	 *
+	 * Anything inside scalar() is optional, including id, of course.
+	 */
+	struct {
+		const char *pfx;
+		const char *fmt;
+		u64 *dst, def;
+		bool is_32, is_set;
+	} *f, fields[8] = {
+		{"smin=", "%lld", &reg->r[S64].a, S64_MIN},
+		{"smax=", "%lld", &reg->r[S64].b, S64_MAX},
+		{"umin=", "%llu", &reg->r[U64].a, 0},
+		{"umax=", "%llu", &reg->r[U64].b, U64_MAX},
+		{"smin32=", "%lld", &reg->r[S32].a, (u32)S32_MIN, true},
+		{"smax32=", "%lld", &reg->r[S32].b, (u32)S32_MAX, true},
+		{"umin32=", "%llu", &reg->r[U32].a, 0,            true},
+		{"umax32=", "%llu", &reg->r[U32].b, U32_MAX,      true},
+	};
+	const char *p, *fmt;
+	int i;
+
+	p = strchr(s, '=');
+	if (!p)
+		return -EINVAL;
+	p++;
+	if (*p == 'P')
+		p++;
+
+	if (!str_has_pfx(p, "scalar(")) {
+		long long sval;
+		enum num_t t;
+
+		if (sscanf(p, "%lld", &sval) != 1)
+			return -EINVAL;
+
+		reg->valid = true;
+		for (t = MIN_T; t <= MAX_T; t++) {
+			reg->r[t] = range(t, sval, sval);
+		}
+		return 0;
+	}
+
+	p += sizeof("scalar");
+	while (p) {
+		int midxs[ARRAY_SIZE(fields)], mcnt = 0;
+		u64 val;
+
+		for (i = 0; i < ARRAY_SIZE(fields); i++) {
+			f = &fields[i];
+			if (!str_has_pfx(p, f->pfx))
+				continue;
+			midxs[mcnt++] = i;
+			p += strlen(f->pfx);
+		}
+
+		if (mcnt) {
+			/* populate all matched fields */
+			fmt = fields[midxs[0]].fmt;
+			if (sscanf(p, fmt, &val) != 1)
+				return -EINVAL;
+
+			for (i = 0; i < mcnt; i++) {
+				f = &fields[midxs[i]];
+				f->is_set = true;
+				*f->dst = f->is_32 ? (u64)(u32)val : val;
+			}
+		} else if (str_has_pfx(p, "var_off")) {
+			/* skip "var_off=(0x0; 0x3f)" part completely */
+			p = strchr(p, ')');
+			if (!p)
+				return -EINVAL;
+			p++;
+		}
+
+		p = strpbrk(p, ",)");
+		if (*p == ')')
+			break;
+		if (p)
+			p++;
+	}
+
+	reg->valid = true;
+
+	for (i = 0; i < ARRAY_SIZE(fields); i++) {
+		f = &fields[i];
+		if (!f->is_set)
+			*f->dst = f->def;
+	}
+
+	return 0;
+}
+
+
+/* Parse all register states (TRUE/FALSE branches and DST/SRC registers)
+ * out of the verifier log for a corresponding test case BPF program.
+ */
+static int parse_range_cmp_log(const char *log_buf, struct case_spec spec,
+			       int false_pos, int true_pos,
+			       struct reg_state *false1_reg, struct reg_state *false2_reg,
+			       struct reg_state *true1_reg, struct reg_state *true2_reg)
+{
+	struct {
+		int insn_idx;
+		int reg_idx;
+		const char *reg_upper;
+		struct reg_state *state;
+	} specs[] = {
+		{false_pos,     6, "R6=", false1_reg},
+		{false_pos + 1, 7, "R7=", false2_reg},
+		{true_pos,      6, "R6=", true1_reg},
+		{true_pos + 1,  7, "R7=", true2_reg},
+	};
+	char buf[32];
+	const char *p = log_buf, *q;
+	int i, err;
+
+	for (i = 0; i < 4; i++) {
+		sprintf(buf, "%d: (%s) %s = %s%d", specs[i].insn_idx,
+			spec.compare_subregs ? "bc" : "bf",
+			spec.compare_subregs ? "w0" : "r0",
+			spec.compare_subregs ? "w" : "r", specs[i].reg_idx);
+
+		q = strstr(p, buf);
+		if (!q) {
+			*specs[i].state = (struct reg_state){.valid = false};
+			continue;
+		}
+		p = strstr(q, specs[i].reg_upper);
+		if (!p)
+			return -EINVAL;
+		err = parse_reg_state(p, specs[i].state);
+		if (err)
+			return -EINVAL;
+	}
+	return 0;
+}
+
+/* Validate ranges match, and print details if they don't */
+static bool assert_range_eq(enum num_t t, struct range x, struct range y,
+			    const char *ctx1, const char *ctx2)
+{
+	DEFINE_STRBUF(sb, 512);
+
+	if (range_eq(x, y))
+		return true;
+
+	snappendf(sb, "MISMATCH %s.%s: ", ctx1, ctx2);
+	snprintf_range(t, sb, x);
+	snappendf(sb, " != ");
+	snprintf_range(t, sb, y);
+
+	printf("%s\n", sb->buf);
+
+	return false;
+}
+
+/* Validate that register states match, and print details if they don't */
+static bool assert_reg_state_eq(struct reg_state *r, struct reg_state *e, const char *ctx)
+{
+	bool ok = true;
+	enum num_t t;
+
+	if (r->valid != e->valid) {
+		printf("MISMATCH %s: actual %s != expected %s\n", ctx,
+		       r->valid ? "<valid>" : "<invalid>",
+		       e->valid ? "<valid>" : "<invalid>");
+		return false;
+	}
+
+	if (!r->valid)
+		return true;
+
+	for (t = MIN_T; t <= MAX_T; t++) {
+		if (!assert_range_eq(t, r->r[t], e->r[t], ctx, t_str(t)))
+			ok = false;
+	}
+
+	return ok;
+}
+
+/* Printf verifier log, filtering out irrelevant noise */
+static void print_verifier_log(const char *buf)
+{
+	const char *p;
+
+	while (buf[0]) {
+		p = strchrnul(buf, '\n');
+
+		/* filter out irrelevant precision backtracking logs */
+		if (str_has_pfx(buf, "mark_precise: "))
+			goto skip_line;
+
+		printf("%.*s\n", (int)(p - buf), buf);
+
+skip_line:
+		buf = *p == '\0' ? p : p + 1;
+	}
+}
+
+/* Simulate provided test case purely with our own range-based logic.
+ * This is done to set up expectations for verifier's branch_taken logic and
+ * verifier's register states in the verifier log.
+ */
+static void sim_case(enum num_t init_t, enum num_t cond_t,
+		     struct range x, struct range y, enum op op,
+		     struct reg_state *fr1, struct reg_state *fr2,
+		     struct reg_state *tr1, struct reg_state *tr2,
+		     int *branch_taken)
+{
+	const u64 A = x.a;
+	const u64 B = x.b;
+	const u64 C = y.a;
+	const u64 D = y.b;
+	struct reg_state rc;
+	enum op rev_op = complement_op(op);
+	enum num_t t;
+
+	fr1->valid = fr2->valid = true;
+	tr1->valid = tr2->valid = true;
+	for (t = MIN_T; t <= MAX_T; t++) {
+		/* if we are initializing using 32-bit subregisters,
+		 * full registers get upper 32 bits zeroed automatically
+		 */
+		struct range z = t_is_32(init_t) ? unkn_subreg(t) : unkn[t];
+
+		fr1->r[t] = fr2->r[t] = tr1->r[t] = tr2->r[t] = z;
+	}
+
+	/* step 1: r1 >= A, r2 >= C */
+	reg_state_set_const(&rc, init_t, A);
+	reg_state_cond(init_t, fr1, &rc, OP_GE, fr1, NULL, "r1>=A");
+	reg_state_set_const(&rc, init_t, C);
+	reg_state_cond(init_t, fr2, &rc, OP_GE, fr2, NULL, "r2>=C");
+	*tr1 = *fr1;
+	*tr2 = *fr2;
+	if (env.verbosity >= VERBOSE_VERY) {
+		printf("STEP1 (%s) R1: ", t_str(init_t)); print_reg_state(fr1, "\n");
+		printf("STEP1 (%s) R2: ", t_str(init_t)); print_reg_state(fr2, "\n");
+	}
+
+	/* step 2: r1 <= B, r2 <= D */
+	reg_state_set_const(&rc, init_t, B);
+	reg_state_cond(init_t, fr1, &rc, OP_LE, fr1, NULL, "r1<=B");
+	reg_state_set_const(&rc, init_t, D);
+	reg_state_cond(init_t, fr2, &rc, OP_LE, fr2, NULL, "r2<=D");
+	*tr1 = *fr1;
+	*tr2 = *fr2;
+	if (env.verbosity >= VERBOSE_VERY) {
+		printf("STEP2 (%s) R1: ", t_str(init_t)); print_reg_state(fr1, "\n");
+		printf("STEP2 (%s) R2: ", t_str(init_t)); print_reg_state(fr2, "\n");
+	}
+
+	/* step 3: r1 <op> r2 */
+	*branch_taken = reg_state_branch_taken_op(cond_t, fr1, fr2, op);
+	fr1->valid = fr2->valid = false;
+	tr1->valid = tr2->valid = false;
+	if (*branch_taken != 1) { /* FALSE is possible */
+		fr1->valid = fr2->valid = true;
+		reg_state_cond(cond_t, fr1, fr2, rev_op, fr1, fr2, "FALSE");
+	}
+	if (*branch_taken != 0) { /* TRUE is possible */
+		tr1->valid = tr2->valid = true;
+		reg_state_cond(cond_t, tr1, tr2, op, tr1, tr2, "TRUE");
+	}
+	if (env.verbosity >= VERBOSE_VERY) {
+		printf("STEP3 (%s) FALSE R1:", t_str(cond_t)); print_reg_state(fr1, "\n");
+		printf("STEP3 (%s) FALSE R2:", t_str(cond_t)); print_reg_state(fr2, "\n");
+		printf("STEP3 (%s) TRUE  R1:", t_str(cond_t)); print_reg_state(tr1, "\n");
+		printf("STEP3 (%s) TRUE  R2:", t_str(cond_t)); print_reg_state(tr2, "\n");
+	}
+}
+
+/* ===============================
+ * HIGH-LEVEL TEST CASE VALIDATION
+ * ===============================
+ */
+static u32 upper_seeds[] = {
+	0,
+	1,
+	U32_MAX,
+	U32_MAX - 1,
+	S32_MAX,
+	(u32)S32_MIN,
+};
+
+static u32 lower_seeds[] = {
+	0,
+	1,
+	2, (u32)-2,
+	255, (u32)-255,
+	UINT_MAX,
+	UINT_MAX - 1,
+	INT_MAX,
+	(u32)INT_MIN,
+};
+
+struct ctx {
+	int val_cnt, subval_cnt, range_cnt, subrange_cnt;
+	u64 uvals[ARRAY_SIZE(upper_seeds) * ARRAY_SIZE(lower_seeds)];
+	s64 svals[ARRAY_SIZE(upper_seeds) * ARRAY_SIZE(lower_seeds)];
+	u32 usubvals[ARRAY_SIZE(lower_seeds)];
+	s32 ssubvals[ARRAY_SIZE(lower_seeds)];
+	struct range *uranges, *sranges;
+	struct range *usubranges, *ssubranges;
+	int max_failure_cnt, cur_failure_cnt;
+	int total_case_cnt, case_cnt;
+	__u64 start_ns;
+	char progress_ctx[32];
+};
+
+static void cleanup_ctx(struct ctx *ctx)
+{
+	free(ctx->uranges);
+	free(ctx->sranges);
+	free(ctx->usubranges);
+	free(ctx->ssubranges);
+}
+
+struct subtest_case {
+	enum num_t init_t;
+	enum num_t cond_t;
+	struct range x;
+	struct range y;
+	enum op op;
+};
+
+static void subtest_case_str(struct strbuf *sb, struct subtest_case *t)
+{
+	snappendf(sb, "(%s)", t_str(t->init_t));
+	snprintf_range(t->init_t, sb, t->x);
+	snappendf(sb, " (%s)%s ", t_str(t->cond_t), op_str(t->op));
+	snprintf_range(t->init_t, sb, t->y);
+}
+
+/* Generate and validate test case based on specific combination of setup
+ * register ranges (including their expected num_t domain), and conditional
+ * operation to perform (including num_t domain in which it has to be
+ * performed)
+ */
+static int verify_case_op(enum num_t init_t, enum num_t cond_t,
+			  struct range x, struct range y, enum op op)
+{
+	char log_buf[256 * 1024];
+	size_t log_sz = sizeof(log_buf);
+	int err, false_pos = 0, true_pos = 0, branch_taken;
+	struct reg_state fr1, fr2, tr1, tr2;
+	struct reg_state fe1, fe2, te1, te2;
+	bool failed = false;
+	struct case_spec spec = {
+		.init_subregs = (init_t == U32 || init_t == S32),
+		.setup_subregs = (init_t == U32 || init_t == S32),
+		.setup_signed = (init_t == S64 || init_t == S32),
+		.compare_subregs = (cond_t == U32 || cond_t == S32),
+		.compare_signed = (cond_t == S64 || cond_t == S32),
+	};
+
+	log_buf[0] = '\0';
+
+	sim_case(init_t, cond_t, x, y, op, &fe1, &fe2, &te1, &te2, &branch_taken);
+
+	err = load_range_cmp_prog(x, y, op, branch_taken, spec,
+				  log_buf, log_sz, &false_pos, &true_pos);
+	if (err) {
+		ASSERT_OK(err, "load_range_cmp_prog");
+		failed = true;
+	}
+
+	err = parse_range_cmp_log(log_buf, spec, false_pos, true_pos,
+				  &fr1, &fr2, &tr1, &tr2);
+	if (err) {
+		ASSERT_OK(err, "parse_range_cmp_log");
+		failed = true;
+	}
+
+	if (!assert_reg_state_eq(&fr1, &fe1, "false_reg1") ||
+	    !assert_reg_state_eq(&fr2, &fe2, "false_reg2") ||
+	    !assert_reg_state_eq(&tr1, &te1, "true_reg1") ||
+	    !assert_reg_state_eq(&tr2, &te2, "true_reg2")) {
+		failed = true;
+	}
+
+	if (failed || env.verbosity >= VERBOSE_NORMAL) {
+		if (failed || env.verbosity >= VERBOSE_VERY) {
+			printf("VERIFIER LOG:\n========================\n");
+			print_verifier_log(log_buf);
+			printf("=====================\n");
+		}
+		printf("ACTUAL   FALSE1: "); print_reg_state(&fr1, "\n");
+		printf("EXPECTED FALSE1: "); print_reg_state(&fe1, "\n");
+		printf("ACTUAL   FALSE2: "); print_reg_state(&fr2, "\n");
+		printf("EXPECTED FALSE2: "); print_reg_state(&fe2, "\n");
+		printf("ACTUAL   TRUE1:  "); print_reg_state(&tr1, "\n");
+		printf("EXPECTED TRUE1:  "); print_reg_state(&te1, "\n");
+		printf("ACTUAL   TRUE2:  "); print_reg_state(&tr2, "\n");
+		printf("EXPECTED TRUE2:  "); print_reg_state(&te2, "\n");
+
+		return failed ? -EINVAL : 0;
+	}
+
+	return 0;
+}
+
+/* Given setup ranges and number types, go over all supported operations,
+ * generating individual subtest for each allowed combination
+ */
+static int verify_case(struct ctx *ctx, enum num_t init_t, enum num_t cond_t,
+		       struct range x, struct range y)
+{
+	DEFINE_STRBUF(sb, 256);
+	int err;
+	struct subtest_case sub = {
+		.init_t = init_t,
+		.cond_t = cond_t,
+		.x = x,
+		.y = y,
+	};
+
+	for (sub.op = MIN_OP; sub.op <= MAX_OP; sub.op++) {
+		sb->pos = 0; /* reset position in strbuf */
+		subtest_case_str(sb, &sub);
+		if (!test__start_subtest(sb->buf))
+			continue;
+
+		if (env.verbosity >= VERBOSE_NORMAL) /* this speeds up debugging */
+			printf("TEST CASE: %s\n", sb->buf);
+
+		err = verify_case_op(init_t, cond_t, x, y, sub.op);
+		if (err || env.verbosity >= VERBOSE_NORMAL)
+			ASSERT_OK(err, sb->buf);
+		if (err) {
+			ctx->cur_failure_cnt++;
+			if (ctx->cur_failure_cnt > ctx->max_failure_cnt)
+				return err;
+			return 0; /* keep testing other cases */
+		}
+		ctx->case_cnt++;
+		if ((ctx->case_cnt % 10000) == 0) {
+			double progress = (ctx->case_cnt + 0.0) / ctx->total_case_cnt;
+			u64 elapsed_ns = get_time_ns() - ctx->start_ns;
+			double remain_ns = elapsed_ns / progress * (1 - progress);
+
+			fprintf(env.stderr, "PROGRESS (%s): %d/%d (%.2lf%%), "
+					    "elapsed %llu mins (%.2lf hrs), "
+					    "ETA %.0lf mins (%.2lf hrs)\n",
+				ctx->progress_ctx,
+				ctx->case_cnt, ctx->total_case_cnt, 100.0 * progress,
+				elapsed_ns / 1000000000 / 60,
+				elapsed_ns / 1000000000.0 / 3600,
+				remain_ns / 1000000000.0 / 60,
+				remain_ns / 1000000000.0 / 3600);
+		}
+	}
+
+	return 0;
+}
+
+/* ================================
+ * GENERATED CASES FROM SEED VALUES
+ * ================================
+ */
+static int u64_cmp(const void *p1, const void *p2)
+{
+	u64 x1 = *(const u64 *)p1, x2 = *(const u64 *)p2;
+
+	return x1 != x2 ? (x1 < x2 ? -1 : 1) : 0;
+}
+
+static int u32_cmp(const void *p1, const void *p2)
+{
+	u32 x1 = *(const u32 *)p1, x2 = *(const u32 *)p2;
+
+	return x1 != x2 ? (x1 < x2 ? -1 : 1) : 0;
+}
+
+static int s64_cmp(const void *p1, const void *p2)
+{
+	s64 x1 = *(const s64 *)p1, x2 = *(const s64 *)p2;
+
+	return x1 != x2 ? (x1 < x2 ? -1 : 1) : 0;
+}
+
+static int s32_cmp(const void *p1, const void *p2)
+{
+	s32 x1 = *(const s32 *)p1, x2 = *(const s32 *)p2;
+
+	return x1 != x2 ? (x1 < x2 ? -1 : 1) : 0;
+}
+
+/* Generate valid unique constants from seeds, both signed and unsigned */
+static void gen_vals(struct ctx *ctx)
+{
+	int i, j, cnt = 0;
+
+	for (i = 0; i < ARRAY_SIZE(upper_seeds); i++) {
+		for (j = 0; j < ARRAY_SIZE(lower_seeds); j++) {
+			ctx->uvals[cnt++] = (((u64)upper_seeds[i]) << 32) | lower_seeds[j];
+		}
+	}
+
+	/* sort and compact uvals (i.e., it's `sort | uniq`) */
+	qsort(ctx->uvals, cnt, sizeof(*ctx->uvals), u64_cmp);
+	for (i = 1, j = 0; i < cnt; i++) {
+		if (ctx->uvals[j] == ctx->uvals[i])
+			continue;
+		j++;
+		ctx->uvals[j] = ctx->uvals[i];
+	}
+	ctx->val_cnt = j + 1;
+
+	/* we have exactly the same number of s64 values, they are just in
+	 * a different order than u64s, so just sort them differently
+	 */
+	for (i = 0; i < ctx->val_cnt; i++)
+		ctx->svals[i] = ctx->uvals[i];
+	qsort(ctx->svals, ctx->val_cnt, sizeof(*ctx->svals), s64_cmp);
+
+	if (env.verbosity >= VERBOSE_SUPER) {
+		DEFINE_STRBUF(sb1, 256);
+		DEFINE_STRBUF(sb2, 256);
+
+		for (i = 0; i < ctx->val_cnt; i++) {
+			sb1->pos = sb2->pos = 0;
+			snprintf_num(U64, sb1, ctx->uvals[i]);
+			snprintf_num(S64, sb2, ctx->svals[i]);
+			printf("SEED #%d: u64=%-20s s64=%-20s\n", i, sb1->buf, sb2->buf);
+		}
+	}
+
+	/* 32-bit values are generated separately */
+	cnt = 0;
+	for (i = 0; i < ARRAY_SIZE(lower_seeds); i++) {
+		ctx->usubvals[cnt++] = lower_seeds[i];
+	}
+
+	/* sort and compact usubvals (i.e., it's `sort | uniq`) */
+	qsort(ctx->usubvals, cnt, sizeof(*ctx->usubvals), u32_cmp);
+	for (i = 1, j = 0; i < cnt; i++) {
+		if (ctx->usubvals[j] == ctx->usubvals[i])
+			continue;
+		j++;
+		ctx->usubvals[j] = ctx->usubvals[i];
+	}
+	ctx->subval_cnt = j + 1;
+
+	for (i = 0; i < ctx->subval_cnt; i++)
+		ctx->ssubvals[i] = ctx->usubvals[i];
+	qsort(ctx->ssubvals, ctx->subval_cnt, sizeof(*ctx->ssubvals), s32_cmp);
+
+	if (env.verbosity >= VERBOSE_SUPER) {
+		DEFINE_STRBUF(sb1, 256);
+		DEFINE_STRBUF(sb2, 256);
+
+		for (i = 0; i < ctx->subval_cnt; i++) {
+			sb1->pos = sb2->pos = 0;
+			snprintf_num(U32, sb1, ctx->usubvals[i]);
+			snprintf_num(S32, sb2, ctx->ssubvals[i]);
+			printf("SUBSEED #%d: u32=%-10s s32=%-10s\n", i, sb1->buf, sb2->buf);
+		}
+	}
+}
+
+/* Generate valid ranges from upper/lower seeds */
+static int gen_ranges(struct ctx *ctx)
+{
+	int i, j, cnt = 0;
+
+	for (i = 0; i < ctx->val_cnt; i++) {
+		for (j = i; j < ctx->val_cnt; j++) {
+			if (env.verbosity >= VERBOSE_SUPER) {
+				DEFINE_STRBUF(sb1, 256);
+				DEFINE_STRBUF(sb2, 256);
+
+				sb1->pos = sb2->pos = 0;
+				snprintf_range(U64, sb1, range(U64, ctx->uvals[i], ctx->uvals[j]));
+				snprintf_range(S64, sb2, range(S64, ctx->svals[i], ctx->svals[j]));
+				printf("RANGE #%d: u64=%-40s s64=%-40s\n", cnt, sb1->buf, sb2->buf);
+			}
+			cnt++;
+		}
+	}
+	ctx->range_cnt = cnt;
+
+	ctx->uranges = calloc(ctx->range_cnt, sizeof(*ctx->uranges));
+	if (!ASSERT_OK_PTR(ctx->uranges, "uranges_calloc"))
+		return -EINVAL;
+	ctx->sranges = calloc(ctx->range_cnt, sizeof(*ctx->sranges));
+	if (!ASSERT_OK_PTR(ctx->sranges, "sranges_calloc"))
+		return -EINVAL;
+
+	cnt = 0;
+	for (i = 0; i < ctx->val_cnt; i++) {
+		for (j = i; j < ctx->val_cnt; j++) {
+			ctx->uranges[cnt] = range(U64, ctx->uvals[i], ctx->uvals[j]);
+			ctx->sranges[cnt] = range(S64, ctx->svals[i], ctx->svals[j]);
+			cnt++;
+		}
+	}
+
+	cnt = 0;
+	for (i = 0; i < ctx->subval_cnt; i++) {
+		for (j = i; j < ctx->subval_cnt; j++) {
+			if (env.verbosity >= VERBOSE_SUPER) {
+				DEFINE_STRBUF(sb1, 256);
+				DEFINE_STRBUF(sb2, 256);
+
+				sb1->pos = sb2->pos = 0;
+				snprintf_range(U32, sb1, range(U32, ctx->usubvals[i], ctx->usubvals[j]));
+				snprintf_range(S32, sb2, range(S32, ctx->ssubvals[i], ctx->ssubvals[j]));
+				printf("SUBRANGE #%d: u32=%-20s s32=%-20s\n", cnt, sb1->buf, sb2->buf);
+			}
+			cnt++;
+		}
+	}
+	ctx->subrange_cnt = cnt;
+
+	ctx->usubranges = calloc(ctx->subrange_cnt, sizeof(*ctx->usubranges));
+	if (!ASSERT_OK_PTR(ctx->usubranges, "usubranges_calloc"))
+		return -EINVAL;
+	ctx->ssubranges = calloc(ctx->subrange_cnt, sizeof(*ctx->ssubranges));
+	if (!ASSERT_OK_PTR(ctx->ssubranges, "ssubranges_calloc"))
+		return -EINVAL;
+
+	cnt = 0;
+	for (i = 0; i < ctx->subval_cnt; i++) {
+		for (j = i; j < ctx->subval_cnt; j++) {
+			ctx->usubranges[cnt] = range(U32, ctx->usubvals[i], ctx->usubvals[j]);
+			ctx->ssubranges[cnt] = range(S32, ctx->ssubvals[i], ctx->ssubvals[j]);
+			cnt++;
+		}
+	}
+
+	return 0;
+}
+
+static int parse_env_vars(struct ctx *ctx)
+{
+	const char *s;
+
+	if (!(s = getenv("SLOW_TESTS")) || strcmp(s, "1") != 0) {
+		test__skip();
+		return -ENOTSUP;
+	}
+
+	if ((s = getenv("REG_BOUNDS_MAX_FAILURE_CNT"))) {
+		errno = 0;
+		ctx->max_failure_cnt = strtol(s, NULL, 10);
+		if (errno || ctx->max_failure_cnt < 0) {
+			ASSERT_OK(-errno, "REG_BOUNDS_MAX_FAILURE_CNT");
+			return -EINVAL;
+		}
+	}
+
+	return 0;
+}
+
+static int prepare_gen_tests(struct ctx *ctx)
+{
+	int err;
+
+	err = parse_env_vars(ctx);
+	if (err)
+		return err;
+
+	gen_vals(ctx);
+	err = gen_ranges(ctx);
+	if (err) {
+		ASSERT_OK(err, "gen_ranges");
+		return err;
+	}
+
+	return 0;
+}
+
+/* Go over generated constants and ranges and validate various supported
+ * combinations of them
+ */
+static void validate_gen_range_vs_const_64(enum num_t init_t, enum num_t cond_t)
+{
+	struct ctx ctx;
+	struct range rconst;
+	const struct range *ranges;
+	const u64 *vals;
+	int i, j;
+
+	memset(&ctx, 0, sizeof(ctx));
+
+	if (prepare_gen_tests(&ctx))
+		goto cleanup;
+
+	ranges = init_t == U64 ? ctx.uranges : ctx.sranges;
+	vals = init_t == U64 ? ctx.uvals : (const u64 *)ctx.svals;
+
+	ctx.total_case_cnt = (MAX_OP - MIN_OP + 1) * (2 * ctx.range_cnt * ctx.val_cnt);
+	ctx.start_ns = get_time_ns();
+	snprintf(ctx.progress_ctx, sizeof(ctx.progress_ctx),
+		 "RANGE x CONST, %s -> %s",
+		 t_str(init_t), t_str(cond_t));
+
+	for (i = 0; i < ctx.val_cnt; i++) {
+		for (j = 0; j < ctx.range_cnt; j++) {
+			rconst = range(init_t, vals[i], vals[i]);
+
+			/* (u64|s64)(<range> x <const>) */
+			if (verify_case(&ctx, init_t, cond_t, ranges[j], rconst))
+				goto cleanup;
+			/* (u64|s64)(<const> x <range>) */
+			if (verify_case(&ctx, init_t, cond_t, rconst, ranges[j]))
+				goto cleanup;
+		}
+	}
+
+cleanup:
+	cleanup_ctx(&ctx);
+}
+
+static void validate_gen_range_vs_const_32(enum num_t init_t, enum num_t cond_t)
+{
+	struct ctx ctx;
+	struct range rconst;
+	const struct range *ranges;
+	const u32 *vals;
+	int i, j;
+
+	memset(&ctx, 0, sizeof(ctx));
+
+	if (prepare_gen_tests(&ctx))
+		goto cleanup;
+
+	ranges = init_t == U32 ? ctx.usubranges : ctx.ssubranges;
+	vals = init_t == U32 ? ctx.usubvals : (const u32 *)ctx.ssubvals;
+
+	ctx.total_case_cnt = (MAX_OP - MIN_OP + 1) * (2 * ctx.subrange_cnt * ctx.subval_cnt);
+	ctx.start_ns = get_time_ns();
+	snprintf(ctx.progress_ctx, sizeof(ctx.progress_ctx),
+		 "RANGE x CONST, %s -> %s",
+		 t_str(init_t), t_str(cond_t));
+
+	for (i = 0; i < ctx.subval_cnt; i++) {
+		for (j = 0; j < ctx.subrange_cnt; j++) {
+			rconst = range(init_t, vals[i], vals[i]);
+
+			/* (u32|s32)(<range> x <const>) */
+			if (verify_case(&ctx, init_t, cond_t, ranges[j], rconst))
+				goto cleanup;
+			/* (u32|s32)(<const> x <range>) */
+			if (verify_case(&ctx, init_t, cond_t, rconst, ranges[j]))
+				goto cleanup;
+		}
+	}
+
+cleanup:
+	cleanup_ctx(&ctx);
+}
+
+/* Go over thousands of test cases generated from initial seed values.
+ * Given this take a long time, guard this begind SLOW_TESTS=1 envvar. If
+ * envvar is not set, this test is skipped during test_progs testing.
+ *
+ * We split this up into smaller subsets based on initialization and
+ * conditiona numeric domains to get an easy parallelization with test_progs'
+ * -j argument.
+ */
+
+/* RANGE x CONST, U64 initial range */
+void test_reg_bounds_gen_consts_u64_u64(void) { validate_gen_range_vs_const_64(U64, U64); }
+void test_reg_bounds_gen_consts_u64_s64(void) { validate_gen_range_vs_const_64(U64, S64); }
+void test_reg_bounds_gen_consts_u64_u32(void) { validate_gen_range_vs_const_64(U64, U32); }
+void test_reg_bounds_gen_consts_u64_s32(void) { validate_gen_range_vs_const_64(U64, S32); }
+/* RANGE x CONST, S64 initial range */
+void test_reg_bounds_gen_consts_s64_u64(void) { validate_gen_range_vs_const_64(S64, U64); }
+void test_reg_bounds_gen_consts_s64_s64(void) { validate_gen_range_vs_const_64(S64, S64); }
+void test_reg_bounds_gen_consts_s64_u32(void) { validate_gen_range_vs_const_64(S64, U32); }
+void test_reg_bounds_gen_consts_s64_s32(void) { validate_gen_range_vs_const_64(S64, S32); }
+/* RANGE x CONST, U32 initial range */
+void test_reg_bounds_gen_consts_u32_u64(void) { validate_gen_range_vs_const_32(U32, U64); }
+void test_reg_bounds_gen_consts_u32_s64(void) { validate_gen_range_vs_const_32(U32, S64); }
+void test_reg_bounds_gen_consts_u32_u32(void) { validate_gen_range_vs_const_32(U32, U32); }
+void test_reg_bounds_gen_consts_u32_s32(void) { validate_gen_range_vs_const_32(U32, S32); }
+/* RANGE x CONST, S32 initial range */
+void test_reg_bounds_gen_consts_s32_u64(void) { validate_gen_range_vs_const_32(S32, U64); }
+void test_reg_bounds_gen_consts_s32_s64(void) { validate_gen_range_vs_const_32(S32, S64); }
+void test_reg_bounds_gen_consts_s32_u32(void) { validate_gen_range_vs_const_32(S32, U32); }
+void test_reg_bounds_gen_consts_s32_s32(void) { validate_gen_range_vs_const_32(S32, S32); }
+
+/* A set of hard-coded "interesting" cases to validate as part of normal
+ * test_progs test runs
+ */
+static struct subtest_case crafted_cases[] = {
+	{U64, U64, {0, 0xffffffff}, {0, 0}},
+	{U64, U64, {0, 0x80000000}, {0, 0}},
+	{U64, U64, {0x100000000ULL, 0x100000100ULL}, {0, 0}},
+	{U64, U64, {0x100000000ULL, 0x180000000ULL}, {0, 0}},
+	{U64, U64, {0x100000000ULL, 0x1ffffff00ULL}, {0, 0}},
+	{U64, U64, {0x100000000ULL, 0x1ffffff01ULL}, {0, 0}},
+	{U64, U64, {0x100000000ULL, 0x1fffffffeULL}, {0, 0}},
+	{U64, U64, {0x100000001ULL, 0x1000000ffULL}, {0, 0}},
+
+	{U64, S64, {0, 0xffffffff00000000ULL}, {0, 0}},
+	{U64, S64, {0x7fffffffffffffffULL, 0xffffffff00000000ULL}, {0, 0}},
+	{U64, S64, {0x7fffffff00000001ULL, 0xffffffff00000000ULL}, {0, 0}},
+	{U64, S64, {0, 0xffffffffULL}, {1, 1}},
+	{U64, S64, {0, 0xffffffffULL}, {0x7fffffff, 0x7fffffff}},
+
+	{U64, U32, {0, 0x100000000}, {0, 0}},
+	{U64, U32, {0xfffffffe, 0x100000000}, {0x80000000, 0x80000000}},
+
+	{U64, S32, {0, 0xffffffff00000000ULL}, {0, 0}},
+	/* these are tricky cases where lower 32 bits allow to tighten 64
+	 * bit boundaries based on tightened lower 32 bit boundaries
+	 */
+	{U64, S32, {0, 0x0ffffffffULL}, {0, 0}},
+	{U64, S32, {0, 0x100000000ULL}, {0, 0}},
+	{U64, S32, {0, 0x100000001ULL}, {0, 0}},
+	{U64, S32, {0, 0x180000000ULL}, {0, 0}},
+	{U64, S32, {0, 0x17fffffffULL}, {0, 0}},
+	{U64, S32, {0, 0x180000001ULL}, {0, 0}},
+
+	/* verifier knows about [-1, 0] range for s32 for this case already */
+	{S64, S64, {0xffffffffffffffffULL, 0}, {0xffffffff00000000ULL, 0xffffffff00000000ULL}},
+	/* but didn't know about these cases initially */
+	{U64, U64, {0xffffffff, 0x100000000ULL}, {0, 0}}, /* s32: [-1, 0] */
+	{U64, U64, {0xffffffff, 0x100000001ULL}, {0, 0}}, /* s32: [-1, 1] */
+
+	/* longer convergence case: learning from u64 -> s64 -> u64 -> u32,
+	 * arriving at u32: [1, U32_MAX] (instead of more pessimistic [0, U32_MAX])
+	 */
+	{S64, U64, {0xffffffff00000001ULL, 0}, {0xffffffff00000000ULL, 0xffffffff00000000ULL}},
+
+	{U32, U32, {1, U32_MAX}, {0, 0}},
+
+	{U32, S32, {0, U32_MAX}, {U32_MAX, U32_MAX}},
+};
+
+/* Go over crafted hard-coded cases. This is fast, so we do it as part of
+ * normal test_progs run.
+ */
+void test_reg_bounds_crafted(void)
+{
+	struct ctx ctx;
+	int i;
+
+	memset(&ctx, 0, sizeof(ctx));
+
+	for (i = 0; i < ARRAY_SIZE(crafted_cases); i++) {
+		struct subtest_case *c = &crafted_cases[i];
+
+		verify_case(&ctx, c->init_t, c->cond_t, c->x, c->y);
+		verify_case(&ctx, c->init_t, c->cond_t, c->y, c->x);
+	}
+
+	cleanup_ctx(&ctx);
+}
-- 
2.34.1

[PATCH v5 bpf-next 11/23] bpf: rename is_branch_taken reg arguments to prepare for the second one

[Andrii Nakryiko]

Just taking mundane refactoring bits out into a separate patch. No
functional changes.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 107 +++++++++++++++++++++---------------------
 1 file changed, 53 insertions(+), 54 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index f5fcb7fb2c67..aa13f32751a1 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -14169,26 +14169,25 @@ static void find_good_pkt_pointers(struct bpf_verifier_state *vstate,
 	}));
 }
 
-static int is_branch32_taken(struct bpf_reg_state *reg, u32 val, u8 opcode)
+static int is_branch32_taken(struct bpf_reg_state *reg1, u32 val, u8 opcode)
 {
-	struct tnum subreg = tnum_subreg(reg->var_off);
 	s32 sval = (s32)val;
 
 	switch (opcode) {
 	case BPF_JEQ:
 		if (tnum_is_const(subreg))
 			return !!tnum_equals_const(subreg, val);
-		else if (val < reg->u32_min_value || val > reg->u32_max_value)
+		else if (val < reg1->u32_min_value || val > reg1->u32_max_value)
 			return 0;
-		else if (sval < reg->s32_min_value || sval > reg->s32_max_value)
+		else if (sval < reg1->s32_min_value || sval > reg1->s32_max_value)
 			return 0;
 		break;
 	case BPF_JNE:
 		if (tnum_is_const(subreg))
 			return !tnum_equals_const(subreg, val);
-		else if (val < reg->u32_min_value || val > reg->u32_max_value)
+		else if (val < reg1->u32_min_value || val > reg1->u32_max_value)
 			return 1;
-		else if (sval < reg->s32_min_value || sval > reg->s32_max_value)
+		else if (sval < reg1->s32_min_value || sval > reg1->s32_max_value)
 			return 1;
 		break;
 	case BPF_JSET:
@@ -14198,51 +14197,51 @@ static int is_branch32_taken(struct bpf_reg_state *reg, u32 val, u8 opcode)
 			return 0;
 		break;
 	case BPF_JGT:
-		if (reg->u32_min_value > val)
+		if (reg1->u32_min_value > val)
 			return 1;
-		else if (reg->u32_max_value <= val)
+		else if (reg1->u32_max_value <= val)
 			return 0;
 		break;
 	case BPF_JSGT:
-		if (reg->s32_min_value > sval)
+		if (reg1->s32_min_value > sval)
 			return 1;
-		else if (reg->s32_max_value <= sval)
+		else if (reg1->s32_max_value <= sval)
 			return 0;
 		break;
 	case BPF_JLT:
-		if (reg->u32_max_value < val)
+		if (reg1->u32_max_value < val)
 			return 1;
-		else if (reg->u32_min_value >= val)
+		else if (reg1->u32_min_value >= val)
 			return 0;
 		break;
 	case BPF_JSLT:
-		if (reg->s32_max_value < sval)
+		if (reg1->s32_max_value < sval)
 			return 1;
-		else if (reg->s32_min_value >= sval)
+		else if (reg1->s32_min_value >= sval)
 			return 0;
 		break;
 	case BPF_JGE:
-		if (reg->u32_min_value >= val)
+		if (reg1->u32_min_value >= val)
 			return 1;
-		else if (reg->u32_max_value < val)
+		else if (reg1->u32_max_value < val)
 			return 0;
 		break;
 	case BPF_JSGE:
-		if (reg->s32_min_value >= sval)
+		if (reg1->s32_min_value >= sval)
 			return 1;
-		else if (reg->s32_max_value < sval)
+		else if (reg1->s32_max_value < sval)
 			return 0;
 		break;
 	case BPF_JLE:
-		if (reg->u32_max_value <= val)
+		if (reg1->u32_max_value <= val)
 			return 1;
-		else if (reg->u32_min_value > val)
+		else if (reg1->u32_min_value > val)
 			return 0;
 		break;
 	case BPF_JSLE:
-		if (reg->s32_max_value <= sval)
+		if (reg1->s32_max_value <= sval)
 			return 1;
-		else if (reg->s32_min_value > sval)
+		else if (reg1->s32_min_value > sval)
 			return 0;
 		break;
 	}
@@ -14251,79 +14250,79 @@ static int is_branch32_taken(struct bpf_reg_state *reg, u32 val, u8 opcode)
 }
 
 
-static int is_branch64_taken(struct bpf_reg_state *reg, u64 val, u8 opcode)
+static int is_branch64_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode)
 {
 	s64 sval = (s64)val;
 
 	switch (opcode) {
 	case BPF_JEQ:
-		if (tnum_is_const(reg->var_off))
-			return !!tnum_equals_const(reg->var_off, val);
-		else if (val < reg->umin_value || val > reg->umax_value)
+		if (tnum_is_const(reg1->var_off))
+			return !!tnum_equals_const(reg1->var_off, val);
+		else if (val < reg1->umin_value || val > reg1->umax_value)
 			return 0;
-		else if (sval < reg->smin_value || sval > reg->smax_value)
+		else if (sval < reg1->smin_value || sval > reg1->smax_value)
 			return 0;
 		break;
 	case BPF_JNE:
-		if (tnum_is_const(reg->var_off))
-			return !tnum_equals_const(reg->var_off, val);
-		else if (val < reg->umin_value || val > reg->umax_value)
+		if (tnum_is_const(reg1->var_off))
+			return !tnum_equals_const(reg1->var_off, val);
+		else if (val < reg1->umin_value || val > reg1->umax_value)
 			return 1;
-		else if (sval < reg->smin_value || sval > reg->smax_value)
+		else if (sval < reg1->smin_value || sval > reg1->smax_value)
 			return 1;
 		break;
 	case BPF_JSET:
-		if ((~reg->var_off.mask & reg->var_off.value) & val)
+		if ((~reg1->var_off.mask & reg1->var_off.value) & val)
 			return 1;
-		if (!((reg->var_off.mask | reg->var_off.value) & val))
+		if (!((reg1->var_off.mask | reg1->var_off.value) & val))
 			return 0;
 		break;
 	case BPF_JGT:
-		if (reg->umin_value > val)
+		if (reg1->umin_value > val)
 			return 1;
-		else if (reg->umax_value <= val)
+		else if (reg1->umax_value <= val)
 			return 0;
 		break;
 	case BPF_JSGT:
-		if (reg->smin_value > sval)
+		if (reg1->smin_value > sval)
 			return 1;
-		else if (reg->smax_value <= sval)
+		else if (reg1->smax_value <= sval)
 			return 0;
 		break;
 	case BPF_JLT:
-		if (reg->umax_value < val)
+		if (reg1->umax_value < val)
 			return 1;
-		else if (reg->umin_value >= val)
+		else if (reg1->umin_value >= val)
 			return 0;
 		break;
 	case BPF_JSLT:
-		if (reg->smax_value < sval)
+		if (reg1->smax_value < sval)
 			return 1;
-		else if (reg->smin_value >= sval)
+		else if (reg1->smin_value >= sval)
 			return 0;
 		break;
 	case BPF_JGE:
-		if (reg->umin_value >= val)
+		if (reg1->umin_value >= val)
 			return 1;
-		else if (reg->umax_value < val)
+		else if (reg1->umax_value < val)
 			return 0;
 		break;
 	case BPF_JSGE:
-		if (reg->smin_value >= sval)
+		if (reg1->smin_value >= sval)
 			return 1;
-		else if (reg->smax_value < sval)
+		else if (reg1->smax_value < sval)
 			return 0;
 		break;
 	case BPF_JLE:
-		if (reg->umax_value <= val)
+		if (reg1->umax_value <= val)
 			return 1;
-		else if (reg->umin_value > val)
+		else if (reg1->umin_value > val)
 			return 0;
 		break;
 	case BPF_JSLE:
-		if (reg->smax_value <= sval)
+		if (reg1->smax_value <= sval)
 			return 1;
-		else if (reg->smin_value > sval)
+		else if (reg1->smin_value > sval)
 			return 0;
 		break;
 	}
@@ -14338,11 +14337,11 @@ static int is_branch64_taken(struct bpf_reg_state *reg, u64 val, u8 opcode)
  * -1 - unknown. Example: "if (reg < 5)" is unknown when register value
  *      range [0,10]
  */
-static int is_branch_taken(struct bpf_reg_state *reg, u64 val, u8 opcode,
+static int is_branch_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode,
 			   bool is_jmp32)
 {
-	if (__is_pointer_value(false, reg)) {
-		if (!reg_not_null(reg))
+	if (__is_pointer_value(false, reg1)) {
+		if (!reg_not_null(reg1))
 			return -1;
 
 		/* If pointer is valid tests against zero will fail so we can
@@ -14362,8 +14361,8 @@ static int is_branch_taken(struct bpf_reg_state *reg, u64 val, u8 opcode,
 	}
 
 	if (is_jmp32)
-		return is_branch32_taken(reg, val, opcode);
-	return is_branch64_taken(reg, val, opcode);
+		return is_branch32_taken(reg1, val, opcode);
+	return is_branch64_taken(reg1, val, opcode);
 }
 
 static int flip_opcode(u32 opcode)
-- 
2.34.1

[PATCH v5 bpf-next 12/23] bpf: generalize is_branch_taken() to work with two registers

[Andrii Nakryiko]

While still assuming that second register is a constant, generalize
is_branch_taken-related code to accept two registers instead of register
plus explicit constant value. This also, as a side effect, allows to
simplify check_cond_jmp_op() by unifying BPF_K case with BPF_X case, for
which we use a fake register to represent BPF_K's imm constant as
a register.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 58 ++++++++++++++++++++++++-------------------
 1 file changed, 33 insertions(+), 25 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index aa13f32751a1..fd328c579f10 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -14169,8 +14169,13 @@ static void find_good_pkt_pointers(struct bpf_verifier_state *vstate,
 	}));
 }
 
-static int is_branch32_taken(struct bpf_reg_state *reg1, u32 val, u8 opcode)
+/*
+ * <reg1> <op> <reg2>, currently assuming reg2 is a constant
+ */
+static int is_branch32_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, u8 opcode)
 {
+	struct tnum subreg = tnum_subreg(reg1->var_off);
+	u32 val = (u32)tnum_subreg(reg2->var_off).value;
 	s32 sval = (s32)val;
 
 	switch (opcode) {
@@ -14250,8 +14255,12 @@ static int is_branch32_taken(struct bpf_reg_state *reg1, u32 val, u8 opcode)
 }
 
 
-static int is_branch64_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode)
+/*
+ * <reg1> <op> <reg2>, currently assuming reg2 is a constant
+ */
+static int is_branch64_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, u8 opcode)
 {
+	u64 val = reg2->var_off.value;
 	s64 sval = (s64)val;
 
 	switch (opcode) {
@@ -14330,16 +14339,23 @@ static int is_branch64_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode)
 	return -1;
 }
 
-/* compute branch direction of the expression "if (reg opcode val) goto target;"
+/* compute branch direction of the expression "if (<reg1> opcode <reg2>) goto target;"
  * and return:
  *  1 - branch will be taken and "goto target" will be executed
  *  0 - branch will not be taken and fall-through to next insn
- * -1 - unknown. Example: "if (reg < 5)" is unknown when register value
+ * -1 - unknown. Example: "if (reg1 < 5)" is unknown when register value
  *      range [0,10]
  */
-static int is_branch_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode,
-			   bool is_jmp32)
+static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
+			   u8 opcode, bool is_jmp32)
 {
+	struct tnum reg2_tnum = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
+	u64 val;
+
+	if (!tnum_is_const(reg2_tnum))
+		return -1;
+	val = reg2_tnum.value;
+
 	if (__is_pointer_value(false, reg1)) {
 		if (!reg_not_null(reg1))
 			return -1;
@@ -14361,8 +14377,8 @@ static int is_branch_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode,
 	}
 
 	if (is_jmp32)
-		return is_branch32_taken(reg1, val, opcode);
-	return is_branch64_taken(reg1, val, opcode);
+		return is_branch32_taken(reg1, reg2, opcode);
+	return is_branch64_taken(reg1, reg2, opcode);
 }
 
 static int flip_opcode(u32 opcode)
@@ -14833,6 +14849,7 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
 	struct bpf_reg_state *regs = this_branch->frame[this_branch->curframe]->regs;
 	struct bpf_reg_state *dst_reg, *other_branch_regs, *src_reg = NULL;
 	struct bpf_reg_state *eq_branch_regs;
+	struct bpf_reg_state fake_reg;
 	u8 opcode = BPF_OP(insn->code);
 	bool is_jmp32;
 	int pred = -1;
@@ -14873,36 +14890,27 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
 			verbose(env, "BPF_JMP/JMP32 uses reserved fields\n");
 			return -EINVAL;
 		}
+		src_reg = &fake_reg;
+		src_reg->type = SCALAR_VALUE;
+		__mark_reg_known(src_reg, insn->imm);
 	}
 
 	is_jmp32 = BPF_CLASS(insn->code) == BPF_JMP32;
 
 	if (BPF_SRC(insn->code) == BPF_K) {
-		pred = is_branch_taken(dst_reg, insn->imm, opcode, is_jmp32);
+		pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
 	} else if (src_reg->type == SCALAR_VALUE &&
 		   is_jmp32 && tnum_is_const(tnum_subreg(src_reg->var_off))) {
-		pred = is_branch_taken(dst_reg,
-				       tnum_subreg(src_reg->var_off).value,
-				       opcode,
-				       is_jmp32);
+		pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
 	} else if (src_reg->type == SCALAR_VALUE &&
 		   !is_jmp32 && tnum_is_const(src_reg->var_off)) {
-		pred = is_branch_taken(dst_reg,
-				       src_reg->var_off.value,
-				       opcode,
-				       is_jmp32);
+		pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
 	} else if (dst_reg->type == SCALAR_VALUE &&
 		   is_jmp32 && tnum_is_const(tnum_subreg(dst_reg->var_off))) {
-		pred = is_branch_taken(src_reg,
-				       tnum_subreg(dst_reg->var_off).value,
-				       flip_opcode(opcode),
-				       is_jmp32);
+		pred = is_branch_taken(src_reg, dst_reg, flip_opcode(opcode), is_jmp32);
 	} else if (dst_reg->type == SCALAR_VALUE &&
 		   !is_jmp32 && tnum_is_const(dst_reg->var_off)) {
-		pred = is_branch_taken(src_reg,
-				       dst_reg->var_off.value,
-				       flip_opcode(opcode),
-				       is_jmp32);
+		pred = is_branch_taken(src_reg, dst_reg, flip_opcode(opcode), is_jmp32);
 	} else if (reg_is_pkt_pointer_any(dst_reg) &&
 		   reg_is_pkt_pointer_any(src_reg) &&
 		   !is_jmp32) {
-- 
2.34.1

[PATCH v5 bpf-next 13/23] bpf: move is_branch_taken() down

[Andrii Nakryiko]

Move is_branch_taken() slightly down. In subsequent patched we'll need
both flip_opcode() and is_pkt_ptr_branch_taken() for is_branch_taken(),
but instead of sprinkling forward declarations around, it makes more
sense to move is_branch_taken() lower below is_pkt_ptr_branch_taken(),
and also keep it closer to very tightly related reg_set_min_max(), as
they are two critical parts of the same SCALAR range tracking logic.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 84 +++++++++++++++++++++----------------------
 1 file changed, 42 insertions(+), 42 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index fd328c579f10..25b5234ebda3 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -14339,48 +14339,6 @@ static int is_branch64_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *r
 	return -1;
 }
 
-/* compute branch direction of the expression "if (<reg1> opcode <reg2>) goto target;"
- * and return:
- *  1 - branch will be taken and "goto target" will be executed
- *  0 - branch will not be taken and fall-through to next insn
- * -1 - unknown. Example: "if (reg1 < 5)" is unknown when register value
- *      range [0,10]
- */
-static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
-			   u8 opcode, bool is_jmp32)
-{
-	struct tnum reg2_tnum = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
-	u64 val;
-
-	if (!tnum_is_const(reg2_tnum))
-		return -1;
-	val = reg2_tnum.value;
-
-	if (__is_pointer_value(false, reg1)) {
-		if (!reg_not_null(reg1))
-			return -1;
-
-		/* If pointer is valid tests against zero will fail so we can
-		 * use this to direct branch taken.
-		 */
-		if (val != 0)
-			return -1;
-
-		switch (opcode) {
-		case BPF_JEQ:
-			return 0;
-		case BPF_JNE:
-			return 1;
-		default:
-			return -1;
-		}
-	}
-
-	if (is_jmp32)
-		return is_branch32_taken(reg1, reg2, opcode);
-	return is_branch64_taken(reg1, reg2, opcode);
-}
-
 static int flip_opcode(u32 opcode)
 {
 	/* How can we transform "a <op> b" into "b <op> a"? */
@@ -14442,6 +14400,48 @@ static int is_pkt_ptr_branch_taken(struct bpf_reg_state *dst_reg,
 	return -1;
 }
 
+/* compute branch direction of the expression "if (<reg1> opcode <reg2>) goto target;"
+ * and return:
+ *  1 - branch will be taken and "goto target" will be executed
+ *  0 - branch will not be taken and fall-through to next insn
+ * -1 - unknown. Example: "if (reg1 < 5)" is unknown when register value
+ *      range [0,10]
+ */
+static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
+			   u8 opcode, bool is_jmp32)
+{
+	struct tnum reg2_tnum = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
+	u64 val;
+
+	if (!tnum_is_const(reg2_tnum))
+		return -1;
+	val = reg2_tnum.value;
+
+	if (__is_pointer_value(false, reg1)) {
+		if (!reg_not_null(reg1))
+			return -1;
+
+		/* If pointer is valid tests against zero will fail so we can
+		 * use this to direct branch taken.
+		 */
+		if (val != 0)
+			return -1;
+
+		switch (opcode) {
+		case BPF_JEQ:
+			return 0;
+		case BPF_JNE:
+			return 1;
+		default:
+			return -1;
+		}
+	}
+
+	if (is_jmp32)
+		return is_branch32_taken(reg1, reg2, opcode);
+	return is_branch64_taken(reg1, reg2, opcode);
+}
+
 /* Adjusts the register min/max values in the case that the dst_reg is the
  * variable register that we are working on, and src_reg is a constant or we're
  * simply doing a BPF_K check.
-- 
2.34.1

[PATCH v5 bpf-next 14/23] bpf: generalize is_branch_taken to handle all conditional jumps in one place

[Andrii Nakryiko]

Make is_branch_taken() a single entry point for branch pruning decision
making, handling both pointer vs pointer, pointer vs scalar, and scalar
vs scalar cases in one place. This also nicely cleans up check_cond_jmp_op().

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 49 ++++++++++++++++++++++---------------------
 1 file changed, 25 insertions(+), 24 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 25b5234ebda3..fedd6d0e76e5 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -14169,6 +14169,19 @@ static void find_good_pkt_pointers(struct bpf_verifier_state *vstate,
 	}));
 }
 
+/* check if register is a constant scalar value */
+static bool is_reg_const(struct bpf_reg_state *reg, bool subreg32)
+{
+	return reg->type == SCALAR_VALUE &&
+	       tnum_is_const(subreg32 ? tnum_subreg(reg->var_off) : reg->var_off);
+}
+
+/* assuming is_reg_const() is true, return constant value of a register */
+static u64 reg_const_value(struct bpf_reg_state *reg, bool subreg32)
+{
+	return subreg32 ? tnum_subreg(reg->var_off).value : reg->var_off.value;
+}
+
 /*
  * <reg1> <op> <reg2>, currently assuming reg2 is a constant
  */
@@ -14410,12 +14423,20 @@ static int is_pkt_ptr_branch_taken(struct bpf_reg_state *dst_reg,
 static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
 			   u8 opcode, bool is_jmp32)
 {
-	struct tnum reg2_tnum = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
 	u64 val;
 
-	if (!tnum_is_const(reg2_tnum))
+	if (reg_is_pkt_pointer_any(reg1) && reg_is_pkt_pointer_any(reg2) && !is_jmp32)
+		return is_pkt_ptr_branch_taken(reg1, reg2, opcode);
+
+	/* try to make sure reg2 is a constant SCALAR_VALUE */
+	if (!is_reg_const(reg2, is_jmp32)) {
+		opcode = flip_opcode(opcode);
+		swap(reg1, reg2);
+	}
+	/* for now we expect reg2 to be a constant to make any useful decisions */
+	if (!is_reg_const(reg2, is_jmp32))
 		return -1;
-	val = reg2_tnum.value;
+	val = reg_const_value(reg2, is_jmp32);
 
 	if (__is_pointer_value(false, reg1)) {
 		if (!reg_not_null(reg1))
@@ -14896,27 +14917,7 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
 	}
 
 	is_jmp32 = BPF_CLASS(insn->code) == BPF_JMP32;
-
-	if (BPF_SRC(insn->code) == BPF_K) {
-		pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
-	} else if (src_reg->type == SCALAR_VALUE &&
-		   is_jmp32 && tnum_is_const(tnum_subreg(src_reg->var_off))) {
-		pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
-	} else if (src_reg->type == SCALAR_VALUE &&
-		   !is_jmp32 && tnum_is_const(src_reg->var_off)) {
-		pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
-	} else if (dst_reg->type == SCALAR_VALUE &&
-		   is_jmp32 && tnum_is_const(tnum_subreg(dst_reg->var_off))) {
-		pred = is_branch_taken(src_reg, dst_reg, flip_opcode(opcode), is_jmp32);
-	} else if (dst_reg->type == SCALAR_VALUE &&
-		   !is_jmp32 && tnum_is_const(dst_reg->var_off)) {
-		pred = is_branch_taken(src_reg, dst_reg, flip_opcode(opcode), is_jmp32);
-	} else if (reg_is_pkt_pointer_any(dst_reg) &&
-		   reg_is_pkt_pointer_any(src_reg) &&
-		   !is_jmp32) {
-		pred = is_pkt_ptr_branch_taken(dst_reg, src_reg, opcode);
-	}
-
+	pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
 	if (pred >= 0) {
 		/* If we get here with a dst_reg pointer type it is because
 		 * above is_branch_taken() special cased the 0 comparison.
-- 
2.34.1

[PATCH v5 bpf-next 15/23] bpf: unify 32-bit and 64-bit is_branch_taken logic

[Andrii Nakryiko]

Combine 32-bit and 64-bit is_branch_taken logic for SCALAR_VALUE
registers. It makes it easier to see parallels between two domains
(32-bit and 64-bit), and makes subsequent refactoring more
straightforward.

No functional changes.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 154 ++++++++++--------------------------------
 1 file changed, 36 insertions(+), 118 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index fedd6d0e76e5..b911d1111fad 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -14185,166 +14185,86 @@ static u64 reg_const_value(struct bpf_reg_state *reg, bool subreg32)
 /*
  * <reg1> <op> <reg2>, currently assuming reg2 is a constant
  */
-static int is_branch32_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, u8 opcode)
+static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
+				  u8 opcode, bool is_jmp32)
 {
-	struct tnum subreg = tnum_subreg(reg1->var_off);
-	u32 val = (u32)tnum_subreg(reg2->var_off).value;
-	s32 sval = (s32)val;
+	struct tnum t1 = is_jmp32 ? tnum_subreg(reg1->var_off) : reg1->var_off;
+	u64 umin1 = is_jmp32 ? (u64)reg1->u32_min_value : reg1->umin_value;
+	u64 umax1 = is_jmp32 ? (u64)reg1->u32_max_value : reg1->umax_value;
+	s64 smin1 = is_jmp32 ? (s64)reg1->s32_min_value : reg1->smin_value;
+	s64 smax1 = is_jmp32 ? (s64)reg1->s32_max_value : reg1->smax_value;
+	u64 val = is_jmp32 ? (u32)tnum_subreg(reg2->var_off).value : reg2->var_off.value;
+	s64 sval = is_jmp32 ? (s32)val : (s64)val;
 
 	switch (opcode) {
 	case BPF_JEQ:
-		if (tnum_is_const(subreg))
-			return !!tnum_equals_const(subreg, val);
-		else if (val < reg1->u32_min_value || val > reg1->u32_max_value)
+		if (tnum_is_const(t1))
+			return !!tnum_equals_const(t1, val);
+		else if (val < umin1 || val > umax1)
 			return 0;
-		else if (sval < reg1->s32_min_value || sval > reg1->s32_max_value)
+		else if (sval < smin1 || sval > smax1)
 			return 0;
 		break;
 	case BPF_JNE:
-		if (tnum_is_const(subreg))
-			return !tnum_equals_const(subreg, val);
-		else if (val < reg1->u32_min_value || val > reg1->u32_max_value)
+		if (tnum_is_const(t1))
+			return !tnum_equals_const(t1, val);
+		else if (val < umin1 || val > umax1)
 			return 1;
-		else if (sval < reg1->s32_min_value || sval > reg1->s32_max_value)
+		else if (sval < smin1 || sval > smax1)
 			return 1;
 		break;
 	case BPF_JSET:
-		if ((~subreg.mask & subreg.value) & val)
+		if ((~t1.mask & t1.value) & val)
 			return 1;
-		if (!((subreg.mask | subreg.value) & val))
+		if (!((t1.mask | t1.value) & val))
 			return 0;
 		break;
 	case BPF_JGT:
-		if (reg1->u32_min_value > val)
+		if (umin1 > val )
 			return 1;
-		else if (reg1->u32_max_value <= val)
+		else if (umax1 <= val)
 			return 0;
 		break;
 	case BPF_JSGT:
-		if (reg1->s32_min_value > sval)
+		if (smin1 > sval)
 			return 1;
-		else if (reg1->s32_max_value <= sval)
+		else if (smax1 <= sval)
 			return 0;
 		break;
 	case BPF_JLT:
-		if (reg1->u32_max_value < val)
+		if (umax1 < val)
 			return 1;
-		else if (reg1->u32_min_value >= val)
+		else if (umin1 >= val)
 			return 0;
 		break;
 	case BPF_JSLT:
-		if (reg1->s32_max_value < sval)
+		if (smax1 < sval)
 			return 1;
-		else if (reg1->s32_min_value >= sval)
+		else if (smin1 >= sval)
 			return 0;
 		break;
 	case BPF_JGE:
-		if (reg1->u32_min_value >= val)
+		if (umin1 >= val)
 			return 1;
-		else if (reg1->u32_max_value < val)
+		else if (umax1 < val)
 			return 0;
 		break;
 	case BPF_JSGE:
-		if (reg1->s32_min_value >= sval)
+		if (smin1 >= sval)
 			return 1;
-		else if (reg1->s32_max_value < sval)
+		else if (smax1 < sval)
 			return 0;
 		break;
 	case BPF_JLE:
-		if (reg1->u32_max_value <= val)
+		if (umax1 <= val)
 			return 1;
-		else if (reg1->u32_min_value > val)
+		else if (umin1 > val)
 			return 0;
 		break;
 	case BPF_JSLE:
-		if (reg1->s32_max_value <= sval)
+		if (smax1 <= sval)
 			return 1;
-		else if (reg1->s32_min_value > sval)
-			return 0;
-		break;
-	}
-
-	return -1;
-}
-
-
-/*
- * <reg1> <op> <reg2>, currently assuming reg2 is a constant
- */
-static int is_branch64_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, u8 opcode)
-{
-	u64 val = reg2->var_off.value;
-	s64 sval = (s64)val;
-
-	switch (opcode) {
-	case BPF_JEQ:
-		if (tnum_is_const(reg1->var_off))
-			return !!tnum_equals_const(reg1->var_off, val);
-		else if (val < reg1->umin_value || val > reg1->umax_value)
-			return 0;
-		else if (sval < reg1->smin_value || sval > reg1->smax_value)
-			return 0;
-		break;
-	case BPF_JNE:
-		if (tnum_is_const(reg1->var_off))
-			return !tnum_equals_const(reg1->var_off, val);
-		else if (val < reg1->umin_value || val > reg1->umax_value)
-			return 1;
-		else if (sval < reg1->smin_value || sval > reg1->smax_value)
-			return 1;
-		break;
-	case BPF_JSET:
-		if ((~reg1->var_off.mask & reg1->var_off.value) & val)
-			return 1;
-		if (!((reg1->var_off.mask | reg1->var_off.value) & val))
-			return 0;
-		break;
-	case BPF_JGT:
-		if (reg1->umin_value > val)
-			return 1;
-		else if (reg1->umax_value <= val)
-			return 0;
-		break;
-	case BPF_JSGT:
-		if (reg1->smin_value > sval)
-			return 1;
-		else if (reg1->smax_value <= sval)
-			return 0;
-		break;
-	case BPF_JLT:
-		if (reg1->umax_value < val)
-			return 1;
-		else if (reg1->umin_value >= val)
-			return 0;
-		break;
-	case BPF_JSLT:
-		if (reg1->smax_value < sval)
-			return 1;
-		else if (reg1->smin_value >= sval)
-			return 0;
-		break;
-	case BPF_JGE:
-		if (reg1->umin_value >= val)
-			return 1;
-		else if (reg1->umax_value < val)
-			return 0;
-		break;
-	case BPF_JSGE:
-		if (reg1->smin_value >= sval)
-			return 1;
-		else if (reg1->smax_value < sval)
-			return 0;
-		break;
-	case BPF_JLE:
-		if (reg1->umax_value <= val)
-			return 1;
-		else if (reg1->umin_value > val)
-			return 0;
-		break;
-	case BPF_JSLE:
-		if (reg1->smax_value <= sval)
-			return 1;
-		else if (reg1->smin_value > sval)
+		else if (smin1 > sval)
 			return 0;
 		break;
 	}
@@ -14458,9 +14378,7 @@ static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg
 		}
 	}
 
-	if (is_jmp32)
-		return is_branch32_taken(reg1, reg2, opcode);
-	return is_branch64_taken(reg1, reg2, opcode);
+	return is_scalar_branch_taken(reg1, reg2, opcode, is_jmp32);
 }
 
 /* Adjusts the register min/max values in the case that the dst_reg is the
-- 
2.34.1

[PATCH v5 bpf-next 16/23] bpf: prepare reg_set_min_max for second set of registers

[Andrii Nakryiko]

Similarly to is_branch_taken()-related refactorings, start preparing
reg_set_min_max() to handle more generic case of two non-const
registers. Start with renaming arguments to accommodate later addition
of second register as an input argument.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 80 +++++++++++++++++++++----------------------
 1 file changed, 40 insertions(+), 40 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index b911d1111fad..dde04b17c3a3 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -14386,25 +14386,25 @@ static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg
  * simply doing a BPF_K check.
  * In JEQ/JNE cases we also adjust the var_off values.
  */
-static void reg_set_min_max(struct bpf_reg_state *true_reg,
-			    struct bpf_reg_state *false_reg,
+static void reg_set_min_max(struct bpf_reg_state *true_reg1,
+			    struct bpf_reg_state *false_reg1,
 			    u64 val, u32 val32,
 			    u8 opcode, bool is_jmp32)
 {
-	struct tnum false_32off = tnum_subreg(false_reg->var_off);
-	struct tnum false_64off = false_reg->var_off;
-	struct tnum true_32off = tnum_subreg(true_reg->var_off);
-	struct tnum true_64off = true_reg->var_off;
+	struct tnum false_32off = tnum_subreg(false_reg1->var_off);
+	struct tnum false_64off = false_reg1->var_off;
+	struct tnum true_32off = tnum_subreg(true_reg1->var_off);
+	struct tnum true_64off = true_reg1->var_off;
 	s64 sval = (s64)val;
 	s32 sval32 = (s32)val32;
 
 	/* If the dst_reg is a pointer, we can't learn anything about its
 	 * variable offset from the compare (unless src_reg were a pointer into
 	 * the same object, but we don't bother with that.
-	 * Since false_reg and true_reg have the same type by construction, we
+	 * Since false_reg1 and true_reg1 have the same type by construction, we
 	 * only need to check one of them for pointerness.
 	 */
-	if (__is_pointer_value(false, false_reg))
+	if (__is_pointer_value(false, false_reg1))
 		return;
 
 	switch (opcode) {
@@ -14419,20 +14419,20 @@ static void reg_set_min_max(struct bpf_reg_state *true_reg,
 	 */
 	case BPF_JEQ:
 		if (is_jmp32) {
-			__mark_reg32_known(true_reg, val32);
-			true_32off = tnum_subreg(true_reg->var_off);
+			__mark_reg32_known(true_reg1, val32);
+			true_32off = tnum_subreg(true_reg1->var_off);
 		} else {
-			___mark_reg_known(true_reg, val);
-			true_64off = true_reg->var_off;
+			___mark_reg_known(true_reg1, val);
+			true_64off = true_reg1->var_off;
 		}
 		break;
 	case BPF_JNE:
 		if (is_jmp32) {
-			__mark_reg32_known(false_reg, val32);
-			false_32off = tnum_subreg(false_reg->var_off);
+			__mark_reg32_known(false_reg1, val32);
+			false_32off = tnum_subreg(false_reg1->var_off);
 		} else {
-			___mark_reg_known(false_reg, val);
-			false_64off = false_reg->var_off;
+			___mark_reg_known(false_reg1, val);
+			false_64off = false_reg1->var_off;
 		}
 		break;
 	case BPF_JSET:
@@ -14455,16 +14455,16 @@ static void reg_set_min_max(struct bpf_reg_state *true_reg,
 			u32 false_umax = opcode == BPF_JGT ? val32  : val32 - 1;
 			u32 true_umin = opcode == BPF_JGT ? val32 + 1 : val32;
 
-			false_reg->u32_max_value = min(false_reg->u32_max_value,
+			false_reg1->u32_max_value = min(false_reg1->u32_max_value,
 						       false_umax);
-			true_reg->u32_min_value = max(true_reg->u32_min_value,
+			true_reg1->u32_min_value = max(true_reg1->u32_min_value,
 						      true_umin);
 		} else {
 			u64 false_umax = opcode == BPF_JGT ? val    : val - 1;
 			u64 true_umin = opcode == BPF_JGT ? val + 1 : val;
 
-			false_reg->umax_value = min(false_reg->umax_value, false_umax);
-			true_reg->umin_value = max(true_reg->umin_value, true_umin);
+			false_reg1->umax_value = min(false_reg1->umax_value, false_umax);
+			true_reg1->umin_value = max(true_reg1->umin_value, true_umin);
 		}
 		break;
 	}
@@ -14475,14 +14475,14 @@ static void reg_set_min_max(struct bpf_reg_state *true_reg,
 			s32 false_smax = opcode == BPF_JSGT ? sval32    : sval32 - 1;
 			s32 true_smin = opcode == BPF_JSGT ? sval32 + 1 : sval32;
 
-			false_reg->s32_max_value = min(false_reg->s32_max_value, false_smax);
-			true_reg->s32_min_value = max(true_reg->s32_min_value, true_smin);
+			false_reg1->s32_max_value = min(false_reg1->s32_max_value, false_smax);
+			true_reg1->s32_min_value = max(true_reg1->s32_min_value, true_smin);
 		} else {
 			s64 false_smax = opcode == BPF_JSGT ? sval    : sval - 1;
 			s64 true_smin = opcode == BPF_JSGT ? sval + 1 : sval;
 
-			false_reg->smax_value = min(false_reg->smax_value, false_smax);
-			true_reg->smin_value = max(true_reg->smin_value, true_smin);
+			false_reg1->smax_value = min(false_reg1->smax_value, false_smax);
+			true_reg1->smin_value = max(true_reg1->smin_value, true_smin);
 		}
 		break;
 	}
@@ -14493,16 +14493,16 @@ static void reg_set_min_max(struct bpf_reg_state *true_reg,
 			u32 false_umin = opcode == BPF_JLT ? val32  : val32 + 1;
 			u32 true_umax = opcode == BPF_JLT ? val32 - 1 : val32;
 
-			false_reg->u32_min_value = max(false_reg->u32_min_value,
+			false_reg1->u32_min_value = max(false_reg1->u32_min_value,
 						       false_umin);
-			true_reg->u32_max_value = min(true_reg->u32_max_value,
+			true_reg1->u32_max_value = min(true_reg1->u32_max_value,
 						      true_umax);
 		} else {
 			u64 false_umin = opcode == BPF_JLT ? val    : val + 1;
 			u64 true_umax = opcode == BPF_JLT ? val - 1 : val;
 
-			false_reg->umin_value = max(false_reg->umin_value, false_umin);
-			true_reg->umax_value = min(true_reg->umax_value, true_umax);
+			false_reg1->umin_value = max(false_reg1->umin_value, false_umin);
+			true_reg1->umax_value = min(true_reg1->umax_value, true_umax);
 		}
 		break;
 	}
@@ -14513,14 +14513,14 @@ static void reg_set_min_max(struct bpf_reg_state *true_reg,
 			s32 false_smin = opcode == BPF_JSLT ? sval32    : sval32 + 1;
 			s32 true_smax = opcode == BPF_JSLT ? sval32 - 1 : sval32;
 
-			false_reg->s32_min_value = max(false_reg->s32_min_value, false_smin);
-			true_reg->s32_max_value = min(true_reg->s32_max_value, true_smax);
+			false_reg1->s32_min_value = max(false_reg1->s32_min_value, false_smin);
+			true_reg1->s32_max_value = min(true_reg1->s32_max_value, true_smax);
 		} else {
 			s64 false_smin = opcode == BPF_JSLT ? sval    : sval + 1;
 			s64 true_smax = opcode == BPF_JSLT ? sval - 1 : sval;
 
-			false_reg->smin_value = max(false_reg->smin_value, false_smin);
-			true_reg->smax_value = min(true_reg->smax_value, true_smax);
+			false_reg1->smin_value = max(false_reg1->smin_value, false_smin);
+			true_reg1->smax_value = min(true_reg1->smax_value, true_smax);
 		}
 		break;
 	}
@@ -14529,17 +14529,17 @@ static void reg_set_min_max(struct bpf_reg_state *true_reg,
 	}
 
 	if (is_jmp32) {
-		false_reg->var_off = tnum_or(tnum_clear_subreg(false_64off),
+		false_reg1->var_off = tnum_or(tnum_clear_subreg(false_64off),
 					     tnum_subreg(false_32off));
-		true_reg->var_off = tnum_or(tnum_clear_subreg(true_64off),
+		true_reg1->var_off = tnum_or(tnum_clear_subreg(true_64off),
 					    tnum_subreg(true_32off));
-		reg_bounds_sync(false_reg);
-		reg_bounds_sync(true_reg);
+		reg_bounds_sync(false_reg1);
+		reg_bounds_sync(true_reg1);
 	} else {
-		false_reg->var_off = false_64off;
-		true_reg->var_off = true_64off;
-		reg_bounds_sync(false_reg);
-		reg_bounds_sync(true_reg);
+		false_reg1->var_off = false_64off;
+		true_reg1->var_off = true_64off;
+		reg_bounds_sync(false_reg1);
+		reg_bounds_sync(true_reg1);
 	}
 }
 
-- 
2.34.1

[PATCH v5 bpf-next 17/23] bpf: generalize reg_set_min_max() to handle two sets of two registers

[Andrii Nakryiko]

Change reg_set_min_max() to take FALSE/TRUE sets of two registers each,
instead of assuming that we are always comparing to a constant. For now
we still assume that right-hand side registers are constants (and make
sure that's the case by swapping src/dst regs, if necessary), but
subsequent patches will remove this limitation.

Taking two by two registers allows to further unify and simplify
check_cond_jmp_op() logic. We utilize fake register for BPF_K
conditional jump case, just like with is_branch_taken() part.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 112 ++++++++++++++++++------------------------
 1 file changed, 49 insertions(+), 63 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index dde04b17c3a3..522566699fbe 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -14387,26 +14387,43 @@ static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg
  * In JEQ/JNE cases we also adjust the var_off values.
  */
 static void reg_set_min_max(struct bpf_reg_state *true_reg1,
+			    struct bpf_reg_state *true_reg2,
 			    struct bpf_reg_state *false_reg1,
-			    u64 val, u32 val32,
+			    struct bpf_reg_state *false_reg2,
 			    u8 opcode, bool is_jmp32)
 {
-	struct tnum false_32off = tnum_subreg(false_reg1->var_off);
-	struct tnum false_64off = false_reg1->var_off;
-	struct tnum true_32off = tnum_subreg(true_reg1->var_off);
-	struct tnum true_64off = true_reg1->var_off;
-	s64 sval = (s64)val;
-	s32 sval32 = (s32)val32;
-
-	/* If the dst_reg is a pointer, we can't learn anything about its
-	 * variable offset from the compare (unless src_reg were a pointer into
-	 * the same object, but we don't bother with that.
-	 * Since false_reg1 and true_reg1 have the same type by construction, we
-	 * only need to check one of them for pointerness.
+	struct tnum false_32off, false_64off;
+	struct tnum true_32off, true_64off;
+	u64 val;
+	u32 val32;
+	s64 sval;
+	s32 sval32;
+
+	/* If either register is a pointer, we can't learn anything about its
+	 * variable offset from the compare (unless they were a pointer into
+	 * the same object, but we don't bother with that).
 	 */
-	if (__is_pointer_value(false, false_reg1))
+	if (false_reg1->type != SCALAR_VALUE || false_reg2->type != SCALAR_VALUE)
+		return;
+
+	/* we expect right-hand registers (src ones) to be constants, for now */
+	if (!is_reg_const(false_reg2, is_jmp32)) {
+		opcode = flip_opcode(opcode);
+		swap(true_reg1, true_reg2);
+		swap(false_reg1, false_reg2);
+	}
+	if (!is_reg_const(false_reg2, is_jmp32))
 		return;
 
+	false_32off = tnum_subreg(false_reg1->var_off);
+	false_64off = false_reg1->var_off;
+	true_32off = tnum_subreg(true_reg1->var_off);
+	true_64off = true_reg1->var_off;
+	val = false_reg2->var_off.value;
+	val32 = (u32)tnum_subreg(false_reg2->var_off).value;
+	sval = (s64)val;
+	sval32 = (s32)val32;
+
 	switch (opcode) {
 	/* JEQ/JNE comparison doesn't change the register equivalence.
 	 *
@@ -14543,22 +14560,6 @@ static void reg_set_min_max(struct bpf_reg_state *true_reg1,
 	}
 }
 
-/* Same as above, but for the case that dst_reg holds a constant and src_reg is
- * the variable reg.
- */
-static void reg_set_min_max_inv(struct bpf_reg_state *true_reg,
-				struct bpf_reg_state *false_reg,
-				u64 val, u32 val32,
-				u8 opcode, bool is_jmp32)
-{
-	opcode = flip_opcode(opcode);
-	/* This uses zero as "not present in table"; luckily the zero opcode,
-	 * BPF_JA, can't get here.
-	 */
-	if (opcode)
-		reg_set_min_max(true_reg, false_reg, val, val32, opcode, is_jmp32);
-}
-
 /* Regs are known to be equal, so intersect their min/max/var_off */
 static void __reg_combine_min_max(struct bpf_reg_state *src_reg,
 				  struct bpf_reg_state *dst_reg)
@@ -14891,45 +14892,30 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
 	 * comparable.
 	 */
 	if (BPF_SRC(insn->code) == BPF_X) {
-		struct bpf_reg_state *src_reg = &regs[insn->src_reg];
+		reg_set_min_max(&other_branch_regs[insn->dst_reg],
+				&other_branch_regs[insn->src_reg],
+				dst_reg, src_reg, opcode, is_jmp32);
 
 		if (dst_reg->type == SCALAR_VALUE &&
-		    src_reg->type == SCALAR_VALUE) {
-			if (tnum_is_const(src_reg->var_off) ||
-			    (is_jmp32 &&
-			     tnum_is_const(tnum_subreg(src_reg->var_off))))
-				reg_set_min_max(&other_branch_regs[insn->dst_reg],
-						dst_reg,
-						src_reg->var_off.value,
-						tnum_subreg(src_reg->var_off).value,
-						opcode, is_jmp32);
-			else if (tnum_is_const(dst_reg->var_off) ||
-				 (is_jmp32 &&
-				  tnum_is_const(tnum_subreg(dst_reg->var_off))))
-				reg_set_min_max_inv(&other_branch_regs[insn->src_reg],
-						    src_reg,
-						    dst_reg->var_off.value,
-						    tnum_subreg(dst_reg->var_off).value,
-						    opcode, is_jmp32);
-			else if (!is_jmp32 &&
-				 (opcode == BPF_JEQ || opcode == BPF_JNE))
-				/* Comparing for equality, we can combine knowledge */
-				reg_combine_min_max(&other_branch_regs[insn->src_reg],
-						    &other_branch_regs[insn->dst_reg],
-						    src_reg, dst_reg, opcode);
-			if (src_reg->id &&
-			    !WARN_ON_ONCE(src_reg->id != other_branch_regs[insn->src_reg].id)) {
-				find_equal_scalars(this_branch, src_reg);
-				find_equal_scalars(other_branch, &other_branch_regs[insn->src_reg]);
-			}
-
+		    src_reg->type == SCALAR_VALUE &&
+		    !is_jmp32 && (opcode == BPF_JEQ || opcode == BPF_JNE)) {
+			/* Comparing for equality, we can combine knowledge */
+			reg_combine_min_max(&other_branch_regs[insn->src_reg],
+					    &other_branch_regs[insn->dst_reg],
+					    src_reg, dst_reg, opcode);
 		}
 	} else if (dst_reg->type == SCALAR_VALUE) {
-		reg_set_min_max(&other_branch_regs[insn->dst_reg],
-					dst_reg, insn->imm, (u32)insn->imm,
-					opcode, is_jmp32);
+		reg_set_min_max(&other_branch_regs[insn->dst_reg], src_reg, /* fake one */
+				dst_reg, src_reg /* same fake one */,
+				opcode, is_jmp32);
 	}
 
+	if (BPF_SRC(insn->code) == BPF_X &&
+	    src_reg->type == SCALAR_VALUE && src_reg->id &&
+	    !WARN_ON_ONCE(src_reg->id != other_branch_regs[insn->src_reg].id)) {
+		find_equal_scalars(this_branch, src_reg);
+		find_equal_scalars(other_branch, &other_branch_regs[insn->src_reg]);
+	}
 	if (dst_reg->type == SCALAR_VALUE && dst_reg->id &&
 	    !WARN_ON_ONCE(dst_reg->id != other_branch_regs[insn->dst_reg].id)) {
 		find_equal_scalars(this_branch, dst_reg);
-- 
2.34.1

[PATCH v5 bpf-next 18/23] bpf: generalize reg_set_min_max() to handle non-const register comparisons

[Andrii Nakryiko]

Generalize bounds adjustment logic of reg_set_min_max() to handle not
just register vs constant case, but in general any register vs any
register cases. For most of the operations it's trivial extension based
on range vs range comparison logic, we just need to properly pick
min/max of a range to compare against min/max of the other range.

For BPF_JSET we keep the original capabilities, just make sure JSET is
integrated in the common framework. This is manifested in the
internal-only BPF_KSET + BPF_X "opcode" to allow for simpler and more
uniform rev_opcode() handling. See the code for details. This allows to
reuse the same code exactly both for TRUE and FALSE branches without
explicitly handling both conditions with custom code.

Note also that now we don't need a special handling of BPF_JEQ/BPF_JNE
case none of the registers are constants. This is now just a normal
generic case handled by reg_set_min_max().

To make tnum handling cleaner, tnum_with_subreg() helper is added, as
that's a common operator when dealing with 32-bit subregister bounds.
This keeps the overall logic much less noisy when it comes to tnums.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 include/linux/tnum.h  |   4 +
 kernel/bpf/tnum.c     |   7 +-
 kernel/bpf/verifier.c | 321 +++++++++++++++++++-----------------------
 3 files changed, 157 insertions(+), 175 deletions(-)

diff --git a/include/linux/tnum.h b/include/linux/tnum.h
index 1c3948a1d6ad..3c13240077b8 100644
--- a/include/linux/tnum.h
+++ b/include/linux/tnum.h
@@ -106,6 +106,10 @@ int tnum_sbin(char *str, size_t size, struct tnum a);
 struct tnum tnum_subreg(struct tnum a);
 /* Returns the tnum with the lower 32-bit subreg cleared */
 struct tnum tnum_clear_subreg(struct tnum a);
+/* Returns the tnum with the lower 32-bit subreg in *reg* set to the lower
+ * 32-bit subreg in *subreg*
+ */
+struct tnum tnum_with_subreg(struct tnum reg, struct tnum subreg);
 /* Returns the tnum with the lower 32-bit subreg set to value */
 struct tnum tnum_const_subreg(struct tnum a, u32 value);
 /* Returns true if 32-bit subreg @a is a known constant*/
diff --git a/kernel/bpf/tnum.c b/kernel/bpf/tnum.c
index 3d7127f439a1..f4c91c9b27d7 100644
--- a/kernel/bpf/tnum.c
+++ b/kernel/bpf/tnum.c
@@ -208,7 +208,12 @@ struct tnum tnum_clear_subreg(struct tnum a)
 	return tnum_lshift(tnum_rshift(a, 32), 32);
 }
 
+struct tnum tnum_with_subreg(struct tnum reg, struct tnum subreg)
+{
+	return tnum_or(tnum_clear_subreg(reg), tnum_subreg(subreg));
+}
+
 struct tnum tnum_const_subreg(struct tnum a, u32 value)
 {
-	return tnum_or(tnum_clear_subreg(a), tnum_const(value));
+	return tnum_with_subreg(a, tnum_const(value));
 }
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 522566699fbe..4c974296127b 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -14381,217 +14381,201 @@ static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg
 	return is_scalar_branch_taken(reg1, reg2, opcode, is_jmp32);
 }
 
-/* Adjusts the register min/max values in the case that the dst_reg is the
- * variable register that we are working on, and src_reg is a constant or we're
- * simply doing a BPF_K check.
- * In JEQ/JNE cases we also adjust the var_off values.
+/* Opcode that corresponds to a *false* branch condition.
+ * E.g., if r1 < r2, then reverse (false) condition is r1 >= r2
  */
-static void reg_set_min_max(struct bpf_reg_state *true_reg1,
-			    struct bpf_reg_state *true_reg2,
-			    struct bpf_reg_state *false_reg1,
-			    struct bpf_reg_state *false_reg2,
-			    u8 opcode, bool is_jmp32)
+static u8 rev_opcode(u8 opcode)
 {
-	struct tnum false_32off, false_64off;
-	struct tnum true_32off, true_64off;
-	u64 val;
-	u32 val32;
-	s64 sval;
-	s32 sval32;
-
-	/* If either register is a pointer, we can't learn anything about its
-	 * variable offset from the compare (unless they were a pointer into
-	 * the same object, but we don't bother with that).
+	switch (opcode) {
+	case BPF_JEQ:		return BPF_JNE;
+	case BPF_JNE:		return BPF_JEQ;
+	/* JSET doesn't have it's reverse opcode in BPF, so add
+	 * BPF_X flag to denote the reverse of that operation
 	 */
-	if (false_reg1->type != SCALAR_VALUE || false_reg2->type != SCALAR_VALUE)
-		return;
-
-	/* we expect right-hand registers (src ones) to be constants, for now */
-	if (!is_reg_const(false_reg2, is_jmp32)) {
-		opcode = flip_opcode(opcode);
-		swap(true_reg1, true_reg2);
-		swap(false_reg1, false_reg2);
+	case BPF_JSET:		return BPF_JSET | BPF_X;
+	case BPF_JSET | BPF_X:	return BPF_JSET;
+	case BPF_JGE:		return BPF_JLT;
+	case BPF_JGT:		return BPF_JLE;
+	case BPF_JLE:		return BPF_JGT;
+	case BPF_JLT:		return BPF_JGE;
+	case BPF_JSGE:		return BPF_JSLT;
+	case BPF_JSGT:		return BPF_JSLE;
+	case BPF_JSLE:		return BPF_JSGT;
+	case BPF_JSLT:		return BPF_JSGE;
+	default:		return 0;
 	}
-	if (!is_reg_const(false_reg2, is_jmp32))
-		return;
+}
 
-	false_32off = tnum_subreg(false_reg1->var_off);
-	false_64off = false_reg1->var_off;
-	true_32off = tnum_subreg(true_reg1->var_off);
-	true_64off = true_reg1->var_off;
-	val = false_reg2->var_off.value;
-	val32 = (u32)tnum_subreg(false_reg2->var_off).value;
-	sval = (s64)val;
-	sval32 = (s32)val32;
+/* Refine range knowledge for <reg1> <op> <reg>2 conditional operation. */
+static void regs_refine_cond_op(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
+				u8 opcode, bool is_jmp32)
+{
+	struct tnum t;
 
 	switch (opcode) {
-	/* JEQ/JNE comparison doesn't change the register equivalence.
-	 *
-	 * r1 = r2;
-	 * if (r1 == 42) goto label;
-	 * ...
-	 * label: // here both r1 and r2 are known to be 42.
-	 *
-	 * Hence when marking register as known preserve it's ID.
-	 */
 	case BPF_JEQ:
 		if (is_jmp32) {
-			__mark_reg32_known(true_reg1, val32);
-			true_32off = tnum_subreg(true_reg1->var_off);
+			reg1->u32_min_value = max(reg1->u32_min_value, reg2->u32_min_value);
+			reg1->u32_max_value = min(reg1->u32_max_value, reg2->u32_max_value);
+			reg1->s32_min_value = max(reg1->s32_min_value, reg2->s32_min_value);
+			reg1->s32_max_value = min(reg1->s32_max_value, reg2->s32_max_value);
+			reg2->u32_min_value = reg1->u32_min_value;
+			reg2->u32_max_value = reg1->u32_max_value;
+			reg2->s32_min_value = reg1->s32_min_value;
+			reg2->s32_max_value = reg1->s32_max_value;
+
+			t = tnum_intersect(tnum_subreg(reg1->var_off), tnum_subreg(reg2->var_off));
+			reg1->var_off = tnum_with_subreg(reg1->var_off, t);
+			reg2->var_off = tnum_with_subreg(reg2->var_off, t);
 		} else {
-			___mark_reg_known(true_reg1, val);
-			true_64off = true_reg1->var_off;
+			reg1->umin_value = max(reg1->umin_value, reg2->umin_value);
+			reg1->umax_value = min(reg1->umax_value, reg2->umax_value);
+			reg1->smin_value = max(reg1->smin_value, reg2->smin_value);
+			reg1->smax_value = min(reg1->smax_value, reg2->smax_value);
+			reg2->umin_value = reg1->umin_value;
+			reg2->umax_value = reg1->umax_value;
+			reg2->smin_value = reg1->smin_value;
+			reg2->smax_value = reg1->smax_value;
+
+			reg1->var_off = tnum_intersect(reg1->var_off, reg2->var_off);
+			reg2->var_off = reg1->var_off;
 		}
 		break;
 	case BPF_JNE:
+		/* we don't derive any new information for inequality yet */
+		break;
+	case BPF_JSET:
+	case BPF_JSET | BPF_X: { /* BPF_JSET and its reverse, see rev_opcode() */
+		u64 val;
+
+		if (!is_reg_const(reg2, is_jmp32))
+			swap(reg1, reg2);
+		if (!is_reg_const(reg2, is_jmp32))
+			break;
+
+		val = reg_const_value(reg2, is_jmp32);
+		/* BPF_JSET requires single bit to learn something useful */
+		if (!(opcode & BPF_X) && !is_power_of_2(val))
+			break;
+
 		if (is_jmp32) {
-			__mark_reg32_known(false_reg1, val32);
-			false_32off = tnum_subreg(false_reg1->var_off);
+			if (opcode & BPF_X)
+				t = tnum_and(tnum_subreg(reg1->var_off), tnum_const(~val));
+			else
+				t = tnum_or(tnum_subreg(reg1->var_off), tnum_const(val));
+			reg1->var_off = tnum_with_subreg(reg1->var_off, t);
 		} else {
-			___mark_reg_known(false_reg1, val);
-			false_64off = false_reg1->var_off;
+			if (opcode & BPF_X)
+				reg1->var_off = tnum_and(reg1->var_off, tnum_const(~val));
+			else
+				reg1->var_off = tnum_or(reg1->var_off, tnum_const(val));
 		}
 		break;
-	case BPF_JSET:
+	}
+	case BPF_JGE:
 		if (is_jmp32) {
-			false_32off = tnum_and(false_32off, tnum_const(~val32));
-			if (is_power_of_2(val32))
-				true_32off = tnum_or(true_32off,
-						     tnum_const(val32));
+			reg1->u32_min_value = max(reg1->u32_min_value, reg2->u32_min_value);
+			reg2->u32_max_value = min(reg1->u32_max_value, reg2->u32_max_value);
 		} else {
-			false_64off = tnum_and(false_64off, tnum_const(~val));
-			if (is_power_of_2(val))
-				true_64off = tnum_or(true_64off,
-						     tnum_const(val));
+			reg1->umin_value = max(reg1->umin_value, reg2->umin_value);
+			reg2->umax_value = min(reg1->umax_value, reg2->umax_value);
 		}
 		break;
-	case BPF_JGE:
 	case BPF_JGT:
-	{
 		if (is_jmp32) {
-			u32 false_umax = opcode == BPF_JGT ? val32  : val32 - 1;
-			u32 true_umin = opcode == BPF_JGT ? val32 + 1 : val32;
-
-			false_reg1->u32_max_value = min(false_reg1->u32_max_value,
-						       false_umax);
-			true_reg1->u32_min_value = max(true_reg1->u32_min_value,
-						      true_umin);
+			reg1->u32_min_value = max(reg1->u32_min_value, reg2->u32_min_value + 1);
+			reg2->u32_max_value = min(reg1->u32_max_value - 1, reg2->u32_max_value);
 		} else {
-			u64 false_umax = opcode == BPF_JGT ? val    : val - 1;
-			u64 true_umin = opcode == BPF_JGT ? val + 1 : val;
-
-			false_reg1->umax_value = min(false_reg1->umax_value, false_umax);
-			true_reg1->umin_value = max(true_reg1->umin_value, true_umin);
+			reg1->umin_value = max(reg1->umin_value, reg2->umin_value + 1);
+			reg2->umax_value = min(reg1->umax_value - 1, reg2->umax_value);
 		}
 		break;
-	}
 	case BPF_JSGE:
+		if (is_jmp32) {
+			reg1->s32_min_value = max(reg1->s32_min_value, reg2->s32_min_value);
+			reg2->s32_max_value = min(reg1->s32_max_value, reg2->s32_max_value);
+		} else {
+			reg1->smin_value = max(reg1->smin_value, reg2->smin_value);
+			reg2->smax_value = min(reg1->smax_value, reg2->smax_value);
+		}
+		break;
 	case BPF_JSGT:
-	{
 		if (is_jmp32) {
-			s32 false_smax = opcode == BPF_JSGT ? sval32    : sval32 - 1;
-			s32 true_smin = opcode == BPF_JSGT ? sval32 + 1 : sval32;
-
-			false_reg1->s32_max_value = min(false_reg1->s32_max_value, false_smax);
-			true_reg1->s32_min_value = max(true_reg1->s32_min_value, true_smin);
+			reg1->s32_min_value = max(reg1->s32_min_value, reg2->s32_min_value + 1);
+			reg2->s32_max_value = min(reg1->s32_max_value - 1, reg2->s32_max_value);
 		} else {
-			s64 false_smax = opcode == BPF_JSGT ? sval    : sval - 1;
-			s64 true_smin = opcode == BPF_JSGT ? sval + 1 : sval;
-
-			false_reg1->smax_value = min(false_reg1->smax_value, false_smax);
-			true_reg1->smin_value = max(true_reg1->smin_value, true_smin);
+			reg1->smin_value = max(reg1->smin_value, reg2->smin_value + 1);
+			reg2->smax_value = min(reg1->smax_value - 1, reg2->smax_value);
 		}
 		break;
-	}
 	case BPF_JLE:
+		if (is_jmp32) {
+			reg1->u32_max_value = min(reg1->u32_max_value, reg2->u32_max_value);
+			reg2->u32_min_value = max(reg1->u32_min_value, reg2->u32_min_value);
+		} else {
+			reg1->umax_value = min(reg1->umax_value, reg2->umax_value);
+			reg2->umin_value = max(reg1->umin_value, reg2->umin_value);
+		}
+		break;
 	case BPF_JLT:
-	{
 		if (is_jmp32) {
-			u32 false_umin = opcode == BPF_JLT ? val32  : val32 + 1;
-			u32 true_umax = opcode == BPF_JLT ? val32 - 1 : val32;
-
-			false_reg1->u32_min_value = max(false_reg1->u32_min_value,
-						       false_umin);
-			true_reg1->u32_max_value = min(true_reg1->u32_max_value,
-						      true_umax);
+			reg1->u32_max_value = min(reg1->u32_max_value, reg2->u32_max_value - 1);
+			reg2->u32_min_value = max(reg1->u32_min_value + 1, reg2->u32_min_value);
 		} else {
-			u64 false_umin = opcode == BPF_JLT ? val    : val + 1;
-			u64 true_umax = opcode == BPF_JLT ? val - 1 : val;
-
-			false_reg1->umin_value = max(false_reg1->umin_value, false_umin);
-			true_reg1->umax_value = min(true_reg1->umax_value, true_umax);
+			reg1->umax_value = min(reg1->umax_value, reg2->umax_value - 1);
+			reg2->umin_value = max(reg1->umin_value + 1, reg2->umin_value);
 		}
 		break;
-	}
 	case BPF_JSLE:
+		if (is_jmp32) {
+			reg1->s32_max_value = min(reg1->s32_max_value, reg2->s32_max_value);
+			reg2->s32_min_value = max(reg1->s32_min_value, reg2->s32_min_value);
+		} else {
+			reg1->smax_value = min(reg1->smax_value, reg2->smax_value);
+			reg2->smin_value = max(reg1->smin_value, reg2->smin_value);
+		}
+		break;
 	case BPF_JSLT:
-	{
 		if (is_jmp32) {
-			s32 false_smin = opcode == BPF_JSLT ? sval32    : sval32 + 1;
-			s32 true_smax = opcode == BPF_JSLT ? sval32 - 1 : sval32;
-
-			false_reg1->s32_min_value = max(false_reg1->s32_min_value, false_smin);
-			true_reg1->s32_max_value = min(true_reg1->s32_max_value, true_smax);
+			reg1->s32_max_value = min(reg1->s32_max_value, reg2->s32_max_value - 1);
+			reg2->s32_min_value = max(reg1->s32_min_value + 1, reg2->s32_min_value);
 		} else {
-			s64 false_smin = opcode == BPF_JSLT ? sval    : sval + 1;
-			s64 true_smax = opcode == BPF_JSLT ? sval - 1 : sval;
-
-			false_reg1->smin_value = max(false_reg1->smin_value, false_smin);
-			true_reg1->smax_value = min(true_reg1->smax_value, true_smax);
+			reg1->smax_value = min(reg1->smax_value, reg2->smax_value - 1);
+			reg2->smin_value = max(reg1->smin_value + 1, reg2->smin_value);
 		}
 		break;
-	}
 	default:
 		return;
 	}
-
-	if (is_jmp32) {
-		false_reg1->var_off = tnum_or(tnum_clear_subreg(false_64off),
-					     tnum_subreg(false_32off));
-		true_reg1->var_off = tnum_or(tnum_clear_subreg(true_64off),
-					    tnum_subreg(true_32off));
-		reg_bounds_sync(false_reg1);
-		reg_bounds_sync(true_reg1);
-	} else {
-		false_reg1->var_off = false_64off;
-		true_reg1->var_off = true_64off;
-		reg_bounds_sync(false_reg1);
-		reg_bounds_sync(true_reg1);
-	}
-}
-
-/* Regs are known to be equal, so intersect their min/max/var_off */
-static void __reg_combine_min_max(struct bpf_reg_state *src_reg,
-				  struct bpf_reg_state *dst_reg)
-{
-	src_reg->umin_value = dst_reg->umin_value = max(src_reg->umin_value,
-							dst_reg->umin_value);
-	src_reg->umax_value = dst_reg->umax_value = min(src_reg->umax_value,
-							dst_reg->umax_value);
-	src_reg->smin_value = dst_reg->smin_value = max(src_reg->smin_value,
-							dst_reg->smin_value);
-	src_reg->smax_value = dst_reg->smax_value = min(src_reg->smax_value,
-							dst_reg->smax_value);
-	src_reg->var_off = dst_reg->var_off = tnum_intersect(src_reg->var_off,
-							     dst_reg->var_off);
-	reg_bounds_sync(src_reg);
-	reg_bounds_sync(dst_reg);
 }
 
-static void reg_combine_min_max(struct bpf_reg_state *true_src,
-				struct bpf_reg_state *true_dst,
-				struct bpf_reg_state *false_src,
-				struct bpf_reg_state *false_dst,
-				u8 opcode)
+/* Adjusts the register min/max values in the case that the dst_reg is the
+ * variable register that we are working on, and src_reg is a constant or we're
+ * simply doing a BPF_K check.
+ * In JEQ/JNE cases we also adjust the var_off values.
+ */
+static void reg_set_min_max(struct bpf_reg_state *true_reg1,
+			    struct bpf_reg_state *true_reg2,
+			    struct bpf_reg_state *false_reg1,
+			    struct bpf_reg_state *false_reg2,
+			    u8 opcode, bool is_jmp32)
 {
-	switch (opcode) {
-	case BPF_JEQ:
-		__reg_combine_min_max(true_src, true_dst);
-		break;
-	case BPF_JNE:
-		__reg_combine_min_max(false_src, false_dst);
-		break;
-	}
+	/* If either register is a pointer, we can't learn anything about its
+	 * variable offset from the compare (unless they were a pointer into
+	 * the same object, but we don't bother with that).
+	 */
+	if (false_reg1->type != SCALAR_VALUE || false_reg2->type != SCALAR_VALUE)
+		return;
+
+	/* fallthrough (FALSE) branch */
+	regs_refine_cond_op(false_reg1, false_reg2, rev_opcode(opcode), is_jmp32);
+	reg_bounds_sync(false_reg1);
+	reg_bounds_sync(false_reg2);
+
+	/* jump (TRUE) branch */
+	regs_refine_cond_op(true_reg1, true_reg2, opcode, is_jmp32);
+	reg_bounds_sync(true_reg1);
+	reg_bounds_sync(true_reg2);
 }
 
 static void mark_ptr_or_null_reg(struct bpf_func_state *state,
@@ -14895,21 +14879,10 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
 		reg_set_min_max(&other_branch_regs[insn->dst_reg],
 				&other_branch_regs[insn->src_reg],
 				dst_reg, src_reg, opcode, is_jmp32);
-
-		if (dst_reg->type == SCALAR_VALUE &&
-		    src_reg->type == SCALAR_VALUE &&
-		    !is_jmp32 && (opcode == BPF_JEQ || opcode == BPF_JNE)) {
-			/* Comparing for equality, we can combine knowledge */
-			reg_combine_min_max(&other_branch_regs[insn->src_reg],
-					    &other_branch_regs[insn->dst_reg],
-					    src_reg, dst_reg, opcode);
-		}
 	} else if (dst_reg->type == SCALAR_VALUE) {
-		reg_set_min_max(&other_branch_regs[insn->dst_reg], src_reg, /* fake one */
-				dst_reg, src_reg /* same fake one */,
-				opcode, is_jmp32);
+		reg_set_min_max(&other_branch_regs[insn->dst_reg], src_reg /* fake*/,
+				dst_reg, src_reg, opcode, is_jmp32);
 	}
-
 	if (BPF_SRC(insn->code) == BPF_X &&
 	    src_reg->type == SCALAR_VALUE && src_reg->id &&
 	    !WARN_ON_ONCE(src_reg->id != other_branch_regs[insn->src_reg].id)) {
-- 
2.34.1

[PATCH v5 bpf-next 19/23] bpf: generalize is_scalar_branch_taken() logic

[Andrii Nakryiko]

Generalize is_branch_taken logic for SCALAR_VALUE register to handle
cases when both registers are not constants. Previously supported
<range> vs <scalar> cases are a natural subset of more generic <range>
vs <range> set of cases.

Generalized logic relies on straightforward segment intersection checks.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 104 ++++++++++++++++++++++++++----------------
 1 file changed, 64 insertions(+), 40 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 4c974296127b..f18a8247e5e2 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -14189,82 +14189,105 @@ static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_sta
 				  u8 opcode, bool is_jmp32)
 {
 	struct tnum t1 = is_jmp32 ? tnum_subreg(reg1->var_off) : reg1->var_off;
+	struct tnum t2 = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
 	u64 umin1 = is_jmp32 ? (u64)reg1->u32_min_value : reg1->umin_value;
 	u64 umax1 = is_jmp32 ? (u64)reg1->u32_max_value : reg1->umax_value;
 	s64 smin1 = is_jmp32 ? (s64)reg1->s32_min_value : reg1->smin_value;
 	s64 smax1 = is_jmp32 ? (s64)reg1->s32_max_value : reg1->smax_value;
-	u64 val = is_jmp32 ? (u32)tnum_subreg(reg2->var_off).value : reg2->var_off.value;
-	s64 sval = is_jmp32 ? (s32)val : (s64)val;
+	u64 umin2 = is_jmp32 ? (u64)reg2->u32_min_value : reg2->umin_value;
+	u64 umax2 = is_jmp32 ? (u64)reg2->u32_max_value : reg2->umax_value;
+	s64 smin2 = is_jmp32 ? (s64)reg2->s32_min_value : reg2->smin_value;
+	s64 smax2 = is_jmp32 ? (s64)reg2->s32_max_value : reg2->smax_value;
 
 	switch (opcode) {
 	case BPF_JEQ:
-		if (tnum_is_const(t1))
-			return !!tnum_equals_const(t1, val);
-		else if (val < umin1 || val > umax1)
+		/* const tnums */
+		if (tnum_is_const(t1) && tnum_is_const(t2))
+			return t1.value == t2.value;
+		/* const ranges */
+		if (umin1 == umax1 && umin2 == umax2)
+			return umin1 == umin2;
+		if (smin1 == smax1 && smin2 == smax2)
+			return umin1 == umin2;
+		/* non-overlapping ranges */
+		if (umin1 > umax2 || umax1 < umin2)
 			return 0;
-		else if (sval < smin1 || sval > smax1)
+		if (smin1 > smax2 || smax1 < smin2)
 			return 0;
 		break;
 	case BPF_JNE:
-		if (tnum_is_const(t1))
-			return !tnum_equals_const(t1, val);
-		else if (val < umin1 || val > umax1)
+		/* const tnums */
+		if (tnum_is_const(t1) && tnum_is_const(t2))
+			return t1.value != t2.value;
+		/* const ranges */
+		if (umin1 == umax1 && umin2 == umax2)
+			return umin1 != umin2;
+		if (smin1 == smax1 && smin2 == smax2)
+			return umin1 != umin2;
+		/* non-overlapping ranges */
+		if (umin1 > umax2 || umax1 < umin2)
 			return 1;
-		else if (sval < smin1 || sval > smax1)
+		if (smin1 > smax2 || smax1 < smin2)
 			return 1;
 		break;
 	case BPF_JSET:
-		if ((~t1.mask & t1.value) & val)
+		if (!is_reg_const(reg2, is_jmp32)) {
+			swap(reg1, reg2);
+			swap(t1, t2);
+		}
+		if (!is_reg_const(reg2, is_jmp32))
+			return -1;
+		if ((~t1.mask & t1.value) & t2.value)
 			return 1;
-		if (!((t1.mask | t1.value) & val))
+		if (!((t1.mask | t1.value) & t2.value))
 			return 0;
 		break;
 	case BPF_JGT:
-		if (umin1 > val )
+		if (umin1 > umax2)
 			return 1;
-		else if (umax1 <= val)
+		else if (umax1 <= umin2)
 			return 0;
 		break;
 	case BPF_JSGT:
-		if (smin1 > sval)
+		if (smin1 > smax2)
 			return 1;
-		else if (smax1 <= sval)
+		else if (smax1 <= smin2)
 			return 0;
 		break;
 	case BPF_JLT:
-		if (umax1 < val)
+		if (umax1 < umin2)
 			return 1;
-		else if (umin1 >= val)
+		else if (umin1 >= umax2)
 			return 0;
 		break;
 	case BPF_JSLT:
-		if (smax1 < sval)
+		if (smax1 < smin2)
 			return 1;
-		else if (smin1 >= sval)
+		else if (smin1 >= smax2)
 			return 0;
 		break;
 	case BPF_JGE:
-		if (umin1 >= val)
+		if (umin1 >= umax2)
 			return 1;
-		else if (umax1 < val)
+		else if (umax1 < umin2)
 			return 0;
 		break;
 	case BPF_JSGE:
-		if (smin1 >= sval)
+		if (smin1 >= smax2)
 			return 1;
-		else if (smax1 < sval)
+		else if (smax1 < smin2)
 			return 0;
 		break;
 	case BPF_JLE:
-		if (umax1 <= val)
+		if (umax1 <= umin2)
 			return 1;
-		else if (umin1 > val)
+		else if (umin1 > umax2)
 			return 0;
 		break;
 	case BPF_JSLE:
-		if (smax1 <= sval)
+		if (smax1 <= smin2)
 			return 1;
-		else if (smin1 > sval)
+		else if (smin1 > smax2)
 			return 0;
 		break;
 	}
@@ -14343,28 +14366,28 @@ static int is_pkt_ptr_branch_taken(struct bpf_reg_state *dst_reg,
 static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
 			   u8 opcode, bool is_jmp32)
 {
-	u64 val;
-
 	if (reg_is_pkt_pointer_any(reg1) && reg_is_pkt_pointer_any(reg2) && !is_jmp32)
 		return is_pkt_ptr_branch_taken(reg1, reg2, opcode);
 
-	/* try to make sure reg2 is a constant SCALAR_VALUE */
-	if (!is_reg_const(reg2, is_jmp32)) {
-		opcode = flip_opcode(opcode);
-		swap(reg1, reg2);
-	}
-	/* for now we expect reg2 to be a constant to make any useful decisions */
-	if (!is_reg_const(reg2, is_jmp32))
-		return -1;
-	val = reg_const_value(reg2, is_jmp32);
+	if (__is_pointer_value(false, reg1) || __is_pointer_value(false, reg2)) {
+		u64 val;
+
+		/* arrange that reg2 is a scalar, and reg1 is a pointer */
+		if (!is_reg_const(reg2, is_jmp32)) {
+			opcode = flip_opcode(opcode);
+			swap(reg1, reg2);
+		}
+		/* and ensure that reg2 is a constant */
+		if (!is_reg_const(reg2, is_jmp32))
+			return -1;
 
-	if (__is_pointer_value(false, reg1)) {
 		if (!reg_not_null(reg1))
 			return -1;
 
 		/* If pointer is valid tests against zero will fail so we can
 		 * use this to direct branch taken.
 		 */
+		val = reg_const_value(reg2, is_jmp32);
 		if (val != 0)
 			return -1;
 
@@ -14378,6 +14401,7 @@ static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg
 		}
 	}
 
+	/* now deal with two scalars, but not necessarily constants */
 	return is_scalar_branch_taken(reg1, reg2, opcode, is_jmp32);
 }
 
-- 
2.34.1

[PATCH v5 bpf-next 20/23] bpf: enhance BPF_JEQ/BPF_JNE is_branch_taken logic

[Andrii Nakryiko]

Use 32-bit subranges to prune some 64-bit BPF_JEQ/BPF_JNE conditions
that otherwise would be "inconclusive" (i.e., is_branch_taken() would
return -1). This can happen, for example, when registers are initialized
as 64-bit u64/s64, then compared for inequality as 32-bit subregisters,
and then followed by 64-bit equality/inequality check. That 32-bit
inequality can establish some pattern for lower 32 bits of a register
(e.g., s< 0 condition determines whether the bit #31 is zero or not),
while overall 64-bit value could be anything (according to a value range
representation).

This is not a fancy quirky special case, but actually a handling that's
necessary to prevent correctness issue with BPF verifier's range
tracking: set_range_min_max() assumes that register ranges are
non-overlapping, and if that condition is not guaranteed by
is_branch_taken() we can end up with invalid ranges, where min > max.

  [0] https://lore.kernel.org/bpf/CACkBjsY2q1_fUohD7hRmKGqv1MV=eP2f6XK8kjkYNw7BaiF8iQ@mail.gmail.com/

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/verifier.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index f18a8247e5e2..cf5bf7ab4410 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -14214,6 +14214,18 @@ static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_sta
 			return 0;
 		if (smin1 > smax2 || smax1 < smin2)
 			return 0;
+		if (!is_jmp32) {
+			/* if 64-bit ranges are inconclusive, see if we can
+			 * utilize 32-bit subrange knowledge to eliminate
+			 * branches that can't be taken a priori
+			 */
+			if (reg1->u32_min_value > reg2->u32_max_value ||
+			    reg1->u32_max_value < reg2->u32_min_value)
+				return 0;
+			if (reg1->s32_min_value > reg2->s32_max_value ||
+			    reg1->s32_max_value < reg2->s32_min_value)
+				return 0;
+		}
 		break;
 	case BPF_JNE:
 		/* const tnums */
@@ -14229,6 +14241,18 @@ static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_sta
 			return 1;
 		if (smin1 > smax2 || smax1 < smin2)
 			return 1;
+		if (!is_jmp32) {
+			/* if 64-bit ranges are inconclusive, see if we can
+			 * utilize 32-bit subrange knowledge to eliminate
+			 * branches that can't be taken a priori
+			 */
+			if (reg1->u32_min_value > reg2->u32_max_value ||
+			    reg1->u32_max_value < reg2->u32_min_value)
+				return 1;
+			if (reg1->s32_min_value > reg2->s32_max_value ||
+			    reg1->s32_max_value < reg2->s32_min_value)
+				return 1;
+		}
 		break;
 	case BPF_JSET:
 		if (!is_reg_const(reg2, is_jmp32)) {
-- 
2.34.1

[PATCH v5 bpf-next 21/23] selftests/bpf: adjust OP_EQ/OP_NE handling to use subranges for branch taken

[Andrii Nakryiko]

Similar to kernel-side BPF verifier logic enhancements, use 32-bit
subrange knowledge for is_branch_taken() logic in reg_bounds selftests.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 .../selftests/bpf/prog_tests/reg_bounds.c     | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
index ac7354cfe139..330618cc12e7 100644
--- a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
+++ b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
@@ -750,16 +750,27 @@ static int reg_state_branch_taken_op(enum num_t t, struct reg_state *x, struct r
 		/* OP_EQ and OP_NE are sign-agnostic */
 		enum num_t tu = t_unsigned(t);
 		enum num_t ts = t_signed(t);
-		int br_u, br_s;
+		int br_u, br_s, br;
 
 		br_u = range_branch_taken_op(tu, x->r[tu], y->r[tu], op);
 		br_s = range_branch_taken_op(ts, x->r[ts], y->r[ts], op);
 
 		if (br_u >= 0 && br_s >= 0 && br_u != br_s)
 			ASSERT_FALSE(true, "branch taken inconsistency!\n");
-		if (br_u >= 0)
-			return br_u;
-		return br_s;
+
+		/* if 64-bit ranges are indecisive, use 32-bit subranges to
+		 * eliminate always/never taken branches, if possible
+		 */
+		if (br_u == -1 && (t == U64 || t == S64)) {
+			br = range_branch_taken_op(U32, x->r[U32], y->r[U32], op);
+			if (br != -1)
+				return br;
+			br = range_branch_taken_op(S32, x->r[S32], y->r[S32], op);
+			if (br != -1)
+				return br;
+		}
+
+		return br_u >= 0 ? br_u : br_s;
 	}
 	return range_branch_taken_op(t, x->r[t], y->r[t], op);
 }
-- 
2.34.1

[PATCH v5 bpf-next 22/23] selftests/bpf: add range x range test to reg_bounds

[Andrii Nakryiko]

Now that verifier supports range vs range bounds adjustments, validate
that by checking each generated range against every other generated
range, across all supported operators (everything by JSET).

We also add few cases that were problematic during development either
for verifier or for selftest's range tracking implementation.

Note that we utilize the same trick with splitting everything into
multiple independent parallelizable tests, but init_t and cond_t. This
brings down verification time in parallel mode from more than 8 hours
down to less that 1.5 hours. 106 million cases were successfully
validate for range vs range logic, in addition to about 7 million range
vs const cases, added in earlier patch.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 .../selftests/bpf/prog_tests/reg_bounds.c     | 86 +++++++++++++++++++
 1 file changed, 86 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
index 330618cc12e7..16864c940548 100644
--- a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
+++ b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
@@ -1752,6 +1752,60 @@ static void validate_gen_range_vs_const_32(enum num_t init_t, enum num_t cond_t)
 	cleanup_ctx(&ctx);
 }
 
+static void validate_gen_range_vs_range(enum num_t init_t, enum num_t cond_t)
+{
+	struct ctx ctx;
+	const struct range *ranges;
+	int i, j, rcnt;
+
+	memset(&ctx, 0, sizeof(ctx));
+
+	if (prepare_gen_tests(&ctx))
+		goto cleanup;
+
+	switch (init_t)
+	{
+	case U64:
+		ranges = ctx.uranges;
+		rcnt = ctx.range_cnt;
+		break;
+	case U32:
+		ranges = ctx.usubranges;
+		rcnt = ctx.subrange_cnt;
+		break;
+	case S64:
+		ranges = ctx.sranges;
+		rcnt = ctx.range_cnt;
+		break;
+	case S32:
+		ranges = ctx.ssubranges;
+		rcnt = ctx.subrange_cnt;
+		break;
+	default:
+		printf("validate_gen_range_vs_range!\n");
+		exit(1);
+	}
+
+	ctx.total_case_cnt = (MAX_OP - MIN_OP + 1) * (2 * rcnt * (rcnt + 1) / 2);
+	ctx.start_ns = get_time_ns();
+	snprintf(ctx.progress_ctx, sizeof(ctx.progress_ctx),
+		 "RANGE x RANGE, %s -> %s",
+		 t_str(init_t), t_str(cond_t));
+
+	for (i = 0; i < rcnt; i++) {
+		for (j = i; j < rcnt; j++) {
+			/* (<range> x <range>) */
+			if (verify_case(&ctx, init_t, cond_t, ranges[i], ranges[j]))
+				goto cleanup;
+			if (verify_case(&ctx, init_t, cond_t, ranges[j], ranges[i]))
+				goto cleanup;
+		}
+	}
+
+cleanup:
+	cleanup_ctx(&ctx);
+}
+
 /* Go over thousands of test cases generated from initial seed values.
  * Given this take a long time, guard this begind SLOW_TESTS=1 envvar. If
  * envvar is not set, this test is skipped during test_progs testing.
@@ -1782,6 +1836,27 @@ void test_reg_bounds_gen_consts_s32_s64(void) { validate_gen_range_vs_const_32(S
 void test_reg_bounds_gen_consts_s32_u32(void) { validate_gen_range_vs_const_32(S32, U32); }
 void test_reg_bounds_gen_consts_s32_s32(void) { validate_gen_range_vs_const_32(S32, S32); }
 
+/* RANGE x RANGE, U64 initial range */
+void test_reg_bounds_gen_ranges_u64_u64(void) { validate_gen_range_vs_range(U64, U64); }
+void test_reg_bounds_gen_ranges_u64_s64(void) { validate_gen_range_vs_range(U64, S64); }
+void test_reg_bounds_gen_ranges_u64_u32(void) { validate_gen_range_vs_range(U64, U32); }
+void test_reg_bounds_gen_ranges_u64_s32(void) { validate_gen_range_vs_range(U64, S32); }
+/* RANGE x RANGE, S64 initial range */
+void test_reg_bounds_gen_ranges_s64_u64(void) { validate_gen_range_vs_range(S64, U64); }
+void test_reg_bounds_gen_ranges_s64_s64(void) { validate_gen_range_vs_range(S64, S64); }
+void test_reg_bounds_gen_ranges_s64_u32(void) { validate_gen_range_vs_range(S64, U32); }
+void test_reg_bounds_gen_ranges_s64_s32(void) { validate_gen_range_vs_range(S64, S32); }
+/* RANGE x RANGE, U32 initial range */
+void test_reg_bounds_gen_ranges_u32_u64(void) { validate_gen_range_vs_range(U32, U64); }
+void test_reg_bounds_gen_ranges_u32_s64(void) { validate_gen_range_vs_range(U32, S64); }
+void test_reg_bounds_gen_ranges_u32_u32(void) { validate_gen_range_vs_range(U32, U32); }
+void test_reg_bounds_gen_ranges_u32_s32(void) { validate_gen_range_vs_range(U32, S32); }
+/* RANGE x RANGE, S32 initial range */
+void test_reg_bounds_gen_ranges_s32_u64(void) { validate_gen_range_vs_range(S32, U64); }
+void test_reg_bounds_gen_ranges_s32_s64(void) { validate_gen_range_vs_range(S32, S64); }
+void test_reg_bounds_gen_ranges_s32_u32(void) { validate_gen_range_vs_range(S32, U32); }
+void test_reg_bounds_gen_ranges_s32_s32(void) { validate_gen_range_vs_range(S32, S32); }
+
 /* A set of hard-coded "interesting" cases to validate as part of normal
  * test_progs test runs
  */
@@ -1795,6 +1870,12 @@ static struct subtest_case crafted_cases[] = {
 	{U64, U64, {0x100000000ULL, 0x1fffffffeULL}, {0, 0}},
 	{U64, U64, {0x100000001ULL, 0x1000000ffULL}, {0, 0}},
 
+	/* single point overlap, interesting BPF_EQ and BPF_NE interactions */
+	{U64, U64, {0, 1}, {1, 0x80000000}},
+	{U64, S64, {0, 1}, {1, 0x80000000}},
+	{U64, U32, {0, 1}, {1, 0x80000000}},
+	{U64, S32, {0, 1}, {1, 0x80000000}},
+
 	{U64, S64, {0, 0xffffffff00000000ULL}, {0, 0}},
 	{U64, S64, {0x7fffffffffffffffULL, 0xffffffff00000000ULL}, {0, 0}},
 	{U64, S64, {0x7fffffff00000001ULL, 0xffffffff00000000ULL}, {0, 0}},
@@ -1829,6 +1910,11 @@ static struct subtest_case crafted_cases[] = {
 	{U32, U32, {1, U32_MAX}, {0, 0}},
 
 	{U32, S32, {0, U32_MAX}, {U32_MAX, U32_MAX}},
+
+	{S32, U64, {(u32)(s32)S32_MIN, (u32)(s32)S32_MIN}, {(u32)(s32)-255, 0}},
+	{S32, S64, {(u32)(s32)S32_MIN, (u32)(s32)-255}, {(u32)(s32)-2, 0}},
+	{S32, S64, {0, 1}, {(u32)(s32)S32_MIN, (u32)(s32)S32_MIN}},
+	{S32, U32, {(u32)(s32)S32_MIN, (u32)(s32)S32_MIN}, {(u32)(s32)S32_MIN, (u32)(s32)S32_MIN}},
 };
 
 /* Go over crafted hard-coded cases. This is fast, so we do it as part of
-- 
2.34.1

[PATCH v5 bpf-next 23/23] selftests/bpf: add iter test requiring range x range logic

[Andrii Nakryiko]

Add a simple verifier test that requires deriving reg bounds for one
register from another register that's not a constant. This is
a realistic example of iterating elements of an array with fixed maximum
number of elements, but smaller actual number of elements.

This small example was an original motivation for doing this whole patch
set in the first place, yes.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 tools/testing/selftests/bpf/progs/iters.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/tools/testing/selftests/bpf/progs/iters.c b/tools/testing/selftests/bpf/progs/iters.c
index c20c4e38b71c..b2181f850d3e 100644
--- a/tools/testing/selftests/bpf/progs/iters.c
+++ b/tools/testing/selftests/bpf/progs/iters.c
@@ -1411,4 +1411,26 @@ __naked int checkpoint_states_deletion(void)
 	);
 }
 
+struct {
+	int data[32];
+	int n;
+} loop_data;
+
+SEC("raw_tp")
+__success
+int iter_arr_with_actual_elem_count(const void *ctx)
+{
+	int i, n = loop_data.n, sum = 0;
+
+	if (n > ARRAY_SIZE(loop_data.data))
+		return 0;
+
+	bpf_for(i, 0, n) {
+		/* no rechecking of i against ARRAY_SIZE(loop_data.n) */
+		sum += loop_data.data[i];
+	}
+
+	return sum;
+}
+
 char _license[] SEC("license") = "GPL";
-- 
2.34.1

Re: [PATCH v5 bpf-next 00/23] BPF register bounds logic and testing improvements

[Alexei Starovoitov]

On Fri, Oct 27, 2023 at 11:13:23AM -0700, Andrii Nakryiko wrote:
> 
> Note, this is not unique to <range> vs <range> logic. Just recently ([0])
> a related issue was reported for existing verifier logic. This patch set does
> fix that issues as well, as pointed out on the mailing list.
> 
>   [0] https://lore.kernel.org/bpf/CAEf4Bzbgf-WQSCz8D4Omh3zFdS4oWS6XELnE7VeoUWgKf3cpig@mail.gmail.com/

Quick comment regarding shift out of bound issue.
I think this patch set makes Hao Sun's repro not working, but I don't think
the range vs range improvement fixes the underlying issue.
Currently we do:
if (umax_val >= insn_bitness)
  mark_reg_unknown
else
  here were use src_reg->u32_max_value or src_reg->umax_value
I suspect the insn_bitness check is buggy and it's still possible to hit UBSAN splat with
out of bounds shift. Just need to try harder.
if w8 < 0xffffffff goto +2;
if r8 != r6 goto +1;
w0 >>= w8;
won't be enough anymore.

Re: [PATCH v5 bpf-next 11/23] bpf: rename is_branch_taken reg arguments to prepare for the second one

[Alexei Starovoitov]

On Fri, Oct 27, 2023 at 11:13:34AM -0700, Andrii Nakryiko wrote:
> Just taking mundane refactoring bits out into a separate patch. No
> functional changes.
> 
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> ---
>  kernel/bpf/verifier.c | 107 +++++++++++++++++++++---------------------
>  1 file changed, 53 insertions(+), 54 deletions(-)
> 
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index f5fcb7fb2c67..aa13f32751a1 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -14169,26 +14169,25 @@ static void find_good_pkt_pointers(struct bpf_verifier_state *vstate,
>  	}));
>  }
>  
> -static int is_branch32_taken(struct bpf_reg_state *reg, u32 val, u8 opcode)
> +static int is_branch32_taken(struct bpf_reg_state *reg1, u32 val, u8 opcode)
>  {
> -	struct tnum subreg = tnum_subreg(reg->var_off);

Looks like accidental removal that breaks build.

>  	s32 sval = (s32)val;
>  
>  	switch (opcode) {
>  	case BPF_JEQ:
>  		if (tnum_is_const(subreg))
>  			return !!tnum_equals_const(subreg, val);

Re: [PATCH v5 bpf-next 15/23] bpf: unify 32-bit and 64-bit is_branch_taken logic

[Alexei Starovoitov]

On Fri, Oct 27, 2023 at 11:13:38AM -0700, Andrii Nakryiko wrote:
> Combine 32-bit and 64-bit is_branch_taken logic for SCALAR_VALUE
> registers. It makes it easier to see parallels between two domains
> (32-bit and 64-bit), and makes subsequent refactoring more
> straightforward.
> 
> No functional changes.
> 
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> ---
>  kernel/bpf/verifier.c | 154 ++++++++++--------------------------------
>  1 file changed, 36 insertions(+), 118 deletions(-)
> 
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index fedd6d0e76e5..b911d1111fad 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -14185,166 +14185,86 @@ static u64 reg_const_value(struct bpf_reg_state *reg, bool subreg32)
>  /*
>   * <reg1> <op> <reg2>, currently assuming reg2 is a constant
>   */
> -static int is_branch32_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, u8 opcode)
> +static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
> +				  u8 opcode, bool is_jmp32)
>  {
> -	struct tnum subreg = tnum_subreg(reg1->var_off);
> -	u32 val = (u32)tnum_subreg(reg2->var_off).value;
> -	s32 sval = (s32)val;
> +	struct tnum t1 = is_jmp32 ? tnum_subreg(reg1->var_off) : reg1->var_off;
> +	u64 umin1 = is_jmp32 ? (u64)reg1->u32_min_value : reg1->umin_value;
> +	u64 umax1 = is_jmp32 ? (u64)reg1->u32_max_value : reg1->umax_value;
> +	s64 smin1 = is_jmp32 ? (s64)reg1->s32_min_value : reg1->smin_value;
> +	s64 smax1 = is_jmp32 ? (s64)reg1->s32_max_value : reg1->smax_value;
> +	u64 val = is_jmp32 ? (u32)tnum_subreg(reg2->var_off).value : reg2->var_off.value;
> +	s64 sval = is_jmp32 ? (s32)val : (s64)val;

Maybe use uval and sval to be consisten with umin/smin ?

Re: [PATCH v5 bpf-next 17/23] bpf: generalize reg_set_min_max() to handle two sets of two registers

[Alexei Starovoitov]

On Fri, Oct 27, 2023 at 11:13:40AM -0700, Andrii Nakryiko wrote:
>  static void reg_set_min_max(struct bpf_reg_state *true_reg1,
> +			    struct bpf_reg_state *true_reg2,
>  			    struct bpf_reg_state *false_reg1,
> -			    u64 val, u32 val32,
> +			    struct bpf_reg_state *false_reg2,
>  			    u8 opcode, bool is_jmp32)
>  {
> -	struct tnum false_32off = tnum_subreg(false_reg1->var_off);
> -	struct tnum false_64off = false_reg1->var_off;
> -	struct tnum true_32off = tnum_subreg(true_reg1->var_off);
> -	struct tnum true_64off = true_reg1->var_off;
> -	s64 sval = (s64)val;
> -	s32 sval32 = (s32)val32;
> -
> -	/* If the dst_reg is a pointer, we can't learn anything about its
> -	 * variable offset from the compare (unless src_reg were a pointer into
> -	 * the same object, but we don't bother with that.
> -	 * Since false_reg1 and true_reg1 have the same type by construction, we
> -	 * only need to check one of them for pointerness.
> +	struct tnum false_32off, false_64off;
> +	struct tnum true_32off, true_64off;
> +	u64 val;
> +	u32 val32;
> +	s64 sval;
> +	s32 sval32;
> +
> +	/* If either register is a pointer, we can't learn anything about its
> +	 * variable offset from the compare (unless they were a pointer into
> +	 * the same object, but we don't bother with that).
>  	 */
> -	if (__is_pointer_value(false, false_reg1))

The removal of the above check, but not the comment was surprising and concerning,
so I did a bit of git-archaeology.
It was added in commit f1174f77b50c ("bpf/verifier: rework value tracking")
back in 2017 !
and in that commit reg_set_min_max() was always called with reg == scalar.
It looked like premature check. Then I spotted a comment in that commit:
  * this is only legit if both are scalars (or pointers to the same
  * object, I suppose, but we don't support that right now), because
  * otherwise the different base pointers mean the offsets aren't
  * comparable.
so the intent back then was to generalize reg_set_min_max() to be used with pointers too,
but we never got around to do that and the comment now reads:
  * this is only legit if both are scalars (or pointers to the same
  * object, I suppose, see the PTR_MAYBE_NULL related if block below),
  * because otherwise the different base pointers mean the offsets aren't
  * comparable.

So please remove is_pointer check and remove the comment,
and fixup the comment in check_cond_jmp_op() where reg_set_min_max().

Re: [PATCH v5 bpf-next 19/23] bpf: generalize is_scalar_branch_taken() logic

[Alexei Starovoitov]

On Fri, Oct 27, 2023 at 11:13:42AM -0700, Andrii Nakryiko wrote:
> Generalize is_branch_taken logic for SCALAR_VALUE register to handle
> cases when both registers are not constants. Previously supported
> <range> vs <scalar> cases are a natural subset of more generic <range>
> vs <range> set of cases.
> 
> Generalized logic relies on straightforward segment intersection checks.
> 
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> ---
>  kernel/bpf/verifier.c | 104 ++++++++++++++++++++++++++----------------
>  1 file changed, 64 insertions(+), 40 deletions(-)
> 
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 4c974296127b..f18a8247e5e2 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -14189,82 +14189,105 @@ static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_sta
>  				  u8 opcode, bool is_jmp32)
>  {
>  	struct tnum t1 = is_jmp32 ? tnum_subreg(reg1->var_off) : reg1->var_off;
> +	struct tnum t2 = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
>  	u64 umin1 = is_jmp32 ? (u64)reg1->u32_min_value : reg1->umin_value;
>  	u64 umax1 = is_jmp32 ? (u64)reg1->u32_max_value : reg1->umax_value;
>  	s64 smin1 = is_jmp32 ? (s64)reg1->s32_min_value : reg1->smin_value;
>  	s64 smax1 = is_jmp32 ? (s64)reg1->s32_max_value : reg1->smax_value;
> -	u64 val = is_jmp32 ? (u32)tnum_subreg(reg2->var_off).value : reg2->var_off.value;
> -	s64 sval = is_jmp32 ? (s32)val : (s64)val;
> +	u64 umin2 = is_jmp32 ? (u64)reg2->u32_min_value : reg2->umin_value;
> +	u64 umax2 = is_jmp32 ? (u64)reg2->u32_max_value : reg2->umax_value;
> +	s64 smin2 = is_jmp32 ? (s64)reg2->s32_min_value : reg2->smin_value;
> +	s64 smax2 = is_jmp32 ? (s64)reg2->s32_max_value : reg2->smax_value;
>  
>  	switch (opcode) {
>  	case BPF_JEQ:
> -		if (tnum_is_const(t1))
> -			return !!tnum_equals_const(t1, val);
> -		else if (val < umin1 || val > umax1)
> +		/* const tnums */
> +		if (tnum_is_const(t1) && tnum_is_const(t2))
> +			return t1.value == t2.value;
> +		/* const ranges */
> +		if (umin1 == umax1 && umin2 == umax2)
> +			return umin1 == umin2;

I don't follow this logic.
umin1 == umax1 means that it's a single constant and
it should have been handled by earlier tnum_is_const check.

> +		if (smin1 == smax1 && smin2 == smax2)
> +			return umin1 == umin2;

here it's even more confusing. smin == smax -> singel const,
but then compare umin1 with umin2 ?!

> +		/* non-overlapping ranges */
> +		if (umin1 > umax2 || umax1 < umin2)
>  			return 0;
> -		else if (sval < smin1 || sval > smax1)
> +		if (smin1 > smax2 || smax1 < smin2)
>  			return 0;

this part makes sense.

Re: [PATCH v5 bpf-next 20/23] bpf: enhance BPF_JEQ/BPF_JNE is_branch_taken logic

[Alexei Starovoitov]

On Fri, Oct 27, 2023 at 11:13:43AM -0700, Andrii Nakryiko wrote:
> Use 32-bit subranges to prune some 64-bit BPF_JEQ/BPF_JNE conditions
> that otherwise would be "inconclusive" (i.e., is_branch_taken() would
> return -1). This can happen, for example, when registers are initialized
> as 64-bit u64/s64, then compared for inequality as 32-bit subregisters,
> and then followed by 64-bit equality/inequality check. That 32-bit
> inequality can establish some pattern for lower 32 bits of a register
> (e.g., s< 0 condition determines whether the bit #31 is zero or not),
> while overall 64-bit value could be anything (according to a value range
> representation).
> 
> This is not a fancy quirky special case, but actually a handling that's
> necessary to prevent correctness issue with BPF verifier's range
> tracking: set_range_min_max() assumes that register ranges are
> non-overlapping, and if that condition is not guaranteed by
> is_branch_taken() we can end up with invalid ranges, where min > max.

This is_scalar_branch_taken() logic makes sense,
but if set_range_min_max() is delicate, it should have its own sanity
check for ranges.
Shouldn't be difficult to check for that dangerous overlap case.

Re: [PATCH v5 bpf-next 00/23] BPF register bounds logic and testing improvements

[Andrii Nakryiko]

On Mon, Oct 30, 2023 at 10:55 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Fri, Oct 27, 2023 at 11:13:23AM -0700, Andrii Nakryiko wrote:
> >
> > Note, this is not unique to <range> vs <range> logic. Just recently ([0])
> > a related issue was reported for existing verifier logic. This patch set does
> > fix that issues as well, as pointed out on the mailing list.
> >
> >   [0] https://lore.kernel.org/bpf/CAEf4Bzbgf-WQSCz8D4Omh3zFdS4oWS6XELnE7VeoUWgKf3cpig@mail.gmail.com/
>
> Quick comment regarding shift out of bound issue.
> I think this patch set makes Hao Sun's repro not working, but I don't think
> the range vs range improvement fixes the underlying issue.

Correct, yes, I think adjust_reg_min_max_vals() might still need some fixing.

> Currently we do:
> if (umax_val >= insn_bitness)
>   mark_reg_unknown
> else
>   here were use src_reg->u32_max_value or src_reg->umax_value
> I suspect the insn_bitness check is buggy and it's still possible to hit UBSAN splat with
> out of bounds shift. Just need to try harder.
> if w8 < 0xffffffff goto +2;
> if r8 != r6 goto +1;
> w0 >>= w8;
> won't be enough anymore.

Agreed, but I felt that fixing adjust_reg_min_max_vals() is out of
scope for this already large patch set. If someone can take a deeper
look into reg bounds for arithmetic operations, it would be great.

On the other hand, one of those academic papers claimed to verify
soundness of verifier's reg bounds, so I wonder why they missed this?
cc Paul, maybe he can clarify (and also, Paul, please try to run all
that formal verification machinery against this patch set, thanks!)

Re: [PATCH v5 bpf-next 11/23] bpf: rename is_branch_taken reg arguments to prepare for the second one

[Andrii Nakryiko]

On Mon, Oct 30, 2023 at 12:40 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Fri, Oct 27, 2023 at 11:13:34AM -0700, Andrii Nakryiko wrote:
> > Just taking mundane refactoring bits out into a separate patch. No
> > functional changes.
> >
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > ---
> >  kernel/bpf/verifier.c | 107 +++++++++++++++++++++---------------------
> >  1 file changed, 53 insertions(+), 54 deletions(-)
> >
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index f5fcb7fb2c67..aa13f32751a1 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -14169,26 +14169,25 @@ static void find_good_pkt_pointers(struct bpf_verifier_state *vstate,
> >       }));
> >  }
> >
> > -static int is_branch32_taken(struct bpf_reg_state *reg, u32 val, u8 opcode)
> > +static int is_branch32_taken(struct bpf_reg_state *reg1, u32 val, u8 opcode)
> >  {
> > -     struct tnum subreg = tnum_subreg(reg->var_off);
>
> Looks like accidental removal that breaks build.
>

Yeah, sorry, there was *a lot* of rebasing involved to split all this
up. I'll fix it, thanks for spotting!

> >       s32 sval = (s32)val;
> >
> >       switch (opcode) {
> >       case BPF_JEQ:
> >               if (tnum_is_const(subreg))
> >                       return !!tnum_equals_const(subreg, val);

Re: [PATCH v5 bpf-next 15/23] bpf: unify 32-bit and 64-bit is_branch_taken logic

[Andrii Nakryiko]

On Mon, Oct 30, 2023 at 12:52 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Fri, Oct 27, 2023 at 11:13:38AM -0700, Andrii Nakryiko wrote:
> > Combine 32-bit and 64-bit is_branch_taken logic for SCALAR_VALUE
> > registers. It makes it easier to see parallels between two domains
> > (32-bit and 64-bit), and makes subsequent refactoring more
> > straightforward.
> >
> > No functional changes.
> >
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > ---
> >  kernel/bpf/verifier.c | 154 ++++++++++--------------------------------
> >  1 file changed, 36 insertions(+), 118 deletions(-)
> >
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index fedd6d0e76e5..b911d1111fad 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -14185,166 +14185,86 @@ static u64 reg_const_value(struct bpf_reg_state *reg, bool subreg32)
> >  /*
> >   * <reg1> <op> <reg2>, currently assuming reg2 is a constant
> >   */
> > -static int is_branch32_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, u8 opcode)
> > +static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
> > +                               u8 opcode, bool is_jmp32)
> >  {
> > -     struct tnum subreg = tnum_subreg(reg1->var_off);
> > -     u32 val = (u32)tnum_subreg(reg2->var_off).value;
> > -     s32 sval = (s32)val;
> > +     struct tnum t1 = is_jmp32 ? tnum_subreg(reg1->var_off) : reg1->var_off;
> > +     u64 umin1 = is_jmp32 ? (u64)reg1->u32_min_value : reg1->umin_value;
> > +     u64 umax1 = is_jmp32 ? (u64)reg1->u32_max_value : reg1->umax_value;
> > +     s64 smin1 = is_jmp32 ? (s64)reg1->s32_min_value : reg1->smin_value;
> > +     s64 smax1 = is_jmp32 ? (s64)reg1->s32_max_value : reg1->smax_value;
> > +     u64 val = is_jmp32 ? (u32)tnum_subreg(reg2->var_off).value : reg2->var_off.value;
> > +     s64 sval = is_jmp32 ? (s32)val : (s64)val;
>
> Maybe use uval and sval to be consisten with umin/smin ?

Sure, I will update val to uval for consistency.

Re: [PATCH v5 bpf-next 17/23] bpf: generalize reg_set_min_max() to handle two sets of two registers

[Andrii Nakryiko]

On Mon, Oct 30, 2023 at 7:02 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Fri, Oct 27, 2023 at 11:13:40AM -0700, Andrii Nakryiko wrote:
> >  static void reg_set_min_max(struct bpf_reg_state *true_reg1,
> > +                         struct bpf_reg_state *true_reg2,
> >                           struct bpf_reg_state *false_reg1,
> > -                         u64 val, u32 val32,
> > +                         struct bpf_reg_state *false_reg2,
> >                           u8 opcode, bool is_jmp32)
> >  {
> > -     struct tnum false_32off = tnum_subreg(false_reg1->var_off);
> > -     struct tnum false_64off = false_reg1->var_off;
> > -     struct tnum true_32off = tnum_subreg(true_reg1->var_off);
> > -     struct tnum true_64off = true_reg1->var_off;
> > -     s64 sval = (s64)val;
> > -     s32 sval32 = (s32)val32;
> > -
> > -     /* If the dst_reg is a pointer, we can't learn anything about its
> > -      * variable offset from the compare (unless src_reg were a pointer into
> > -      * the same object, but we don't bother with that.
> > -      * Since false_reg1 and true_reg1 have the same type by construction, we
> > -      * only need to check one of them for pointerness.
> > +     struct tnum false_32off, false_64off;
> > +     struct tnum true_32off, true_64off;
> > +     u64 val;
> > +     u32 val32;
> > +     s64 sval;
> > +     s32 sval32;
> > +
> > +     /* If either register is a pointer, we can't learn anything about its
> > +      * variable offset from the compare (unless they were a pointer into
> > +      * the same object, but we don't bother with that).
> >        */
> > -     if (__is_pointer_value(false, false_reg1))
>
> The removal of the above check, but not the comment was surprising and concerning,
> so I did a bit of git-archaeology.
> It was added in commit f1174f77b50c ("bpf/verifier: rework value tracking")
> back in 2017 !
> and in that commit reg_set_min_max() was always called with reg == scalar.
> It looked like premature check. Then I spotted a comment in that commit:
>   * this is only legit if both are scalars (or pointers to the same
>   * object, I suppose, but we don't support that right now), because
>   * otherwise the different base pointers mean the offsets aren't
>   * comparable.
> so the intent back then was to generalize reg_set_min_max() to be used with pointers too,
> but we never got around to do that and the comment now reads:

Yeah, it shouldn't be too hard to "generalize" to pointer vs pointer,
if we ensure they point to exactly the same thing (I haven't thought
much about how), because beyond that it's still basically SCALAR
offsets. But I figured it's out of scope for these changes :)

>   * this is only legit if both are scalars (or pointers to the same
>   * object, I suppose, see the PTR_MAYBE_NULL related if block below),
>   * because otherwise the different base pointers mean the offsets aren't
>   * comparable.
>
> So please remove is_pointer check and remove the comment,

So I'm a bit confused. I did remove __is_pointer_value() check, but I
still need to guard against having pointers, which is why I have:

if (false_reg1->type != SCALAR_VALUE || false_reg2->type != SCALAR_VALUE).
    return;

I think I need this check, because reg_set_min_max() can be called
from check_cond_jmp_op() with pointer regs, and we shouldn't try to
adjust them. Or am I missing something? And the comment I have here
now:

+       /* If either register is a pointer, we can't learn anything about its
+        * variable offset from the compare (unless they were a pointer into
+        * the same object, but we don't bother with that).
         */

is trying to explain that we don't really adjust two pointers.

> and fixup the comment in check_cond_jmp_op() where reg_set_min_max().

I have this locally for now, please let me know if this is fine or you
had something else in mind:

-/* Adjusts the register min/max values in the case that the dst_reg is the
- * variable register that we are working on, and src_reg is a constant or we're
- * simply doing a BPF_K check.
- * In JEQ/JNE cases we also adjust the var_off values.
+/* Adjusts the register min/max values in the case that the dst_reg and
+ * src_reg are both SCALAR_VALUE registers (or we are simply doing a BPF_K
+ * check, in which case we havea fake SCALAR_VALUE representing insn->imm).
+ * Technically we can do similar adjustments for pointers to the same object,
+ * but we don't support that right now.
  */
 static void reg_set_min_max(struct bpf_reg_state *true_reg1,
                            struct bpf_reg_state *true_reg2,
@@ -14884,13 +14885,6 @@ static int check_cond_jmp_op(struct
bpf_verifier_env *env,
                return -EFAULT;
        other_branch_regs = other_branch->frame[other_branch->curframe]->regs;

-       /* detect if we are comparing against a constant value so we can adjust
-        * our min/max values for our dst register.
-        * this is only legit if both are scalars (or pointers to the same
-        * object, I suppose, see the PTR_MAYBE_NULL related if block below),
-        * because otherwise the different base pointers mean the offsets aren't
-        * comparable.
-        */
        if (BPF_SRC(insn->code) == BPF_X) {
                reg_set_min_max(&other_branch_regs[insn->dst_reg],
                                &other_branch_regs[insn->src_reg],

Re: [PATCH v5 bpf-next 19/23] bpf: generalize is_scalar_branch_taken() logic

[Andrii Nakryiko]

On Mon, Oct 30, 2023 at 7:12 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Fri, Oct 27, 2023 at 11:13:42AM -0700, Andrii Nakryiko wrote:
> > Generalize is_branch_taken logic for SCALAR_VALUE register to handle
> > cases when both registers are not constants. Previously supported
> > <range> vs <scalar> cases are a natural subset of more generic <range>
> > vs <range> set of cases.
> >
> > Generalized logic relies on straightforward segment intersection checks.
> >
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > ---
> >  kernel/bpf/verifier.c | 104 ++++++++++++++++++++++++++----------------
> >  1 file changed, 64 insertions(+), 40 deletions(-)
> >
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index 4c974296127b..f18a8247e5e2 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -14189,82 +14189,105 @@ static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_sta
> >                                 u8 opcode, bool is_jmp32)
> >  {
> >       struct tnum t1 = is_jmp32 ? tnum_subreg(reg1->var_off) : reg1->var_off;
> > +     struct tnum t2 = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
> >       u64 umin1 = is_jmp32 ? (u64)reg1->u32_min_value : reg1->umin_value;
> >       u64 umax1 = is_jmp32 ? (u64)reg1->u32_max_value : reg1->umax_value;
> >       s64 smin1 = is_jmp32 ? (s64)reg1->s32_min_value : reg1->smin_value;
> >       s64 smax1 = is_jmp32 ? (s64)reg1->s32_max_value : reg1->smax_value;
> > -     u64 val = is_jmp32 ? (u32)tnum_subreg(reg2->var_off).value : reg2->var_off.value;
> > -     s64 sval = is_jmp32 ? (s32)val : (s64)val;
> > +     u64 umin2 = is_jmp32 ? (u64)reg2->u32_min_value : reg2->umin_value;
> > +     u64 umax2 = is_jmp32 ? (u64)reg2->u32_max_value : reg2->umax_value;
> > +     s64 smin2 = is_jmp32 ? (s64)reg2->s32_min_value : reg2->smin_value;
> > +     s64 smax2 = is_jmp32 ? (s64)reg2->s32_max_value : reg2->smax_value;
> >
> >       switch (opcode) {
> >       case BPF_JEQ:
> > -             if (tnum_is_const(t1))
> > -                     return !!tnum_equals_const(t1, val);
> > -             else if (val < umin1 || val > umax1)
> > +             /* const tnums */
> > +             if (tnum_is_const(t1) && tnum_is_const(t2))
> > +                     return t1.value == t2.value;
> > +             /* const ranges */
> > +             if (umin1 == umax1 && umin2 == umax2)
> > +                     return umin1 == umin2;
>
> I don't follow this logic.
> umin1 == umax1 means that it's a single constant and
> it should have been handled by earlier tnum_is_const check.

I think you follow the logic, you just think it's redundant. Yes, it's
basically the same as

          if (tnum_is_const(t1) && tnum_is_const(t2))
                return t1.value == t2.value;

but based on ranges. I didn't feel comfortable to assume that if umin1
== umax1 then tnum_is_const(t1) will always be true. At worst we'll
perform one redundant check.

In short, I don't trust tnum to be as precise as umin/umax and other ranges.

>
> > +             if (smin1 == smax1 && smin2 == smax2)
> > +                     return umin1 == umin2;
>
> here it's even more confusing. smin == smax -> singel const,
> but then compare umin1 with umin2 ?!

Eagle eyes! Typo, sorry :( it should be `smin1 == smin2`, of course.

What saves us is reg_bounds_sync(), and if we have umin1 == umax1 then
we'll have also smin1 == smax1 == umin1 == umax1 (and corresponding
relation for second register). But I fixed these typos in both BPF_JEQ
and BPF_JNE branches.


>
> > +             /* non-overlapping ranges */
> > +             if (umin1 > umax2 || umax1 < umin2)
> >                       return 0;
> > -             else if (sval < smin1 || sval > smax1)
> > +             if (smin1 > smax2 || smax1 < smin2)
> >                       return 0;
>
> this part makes sense.

Re: [PATCH v5 bpf-next 20/23] bpf: enhance BPF_JEQ/BPF_JNE is_branch_taken logic

[Andrii Nakryiko]

On Mon, Oct 30, 2023 at 7:20 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Fri, Oct 27, 2023 at 11:13:43AM -0700, Andrii Nakryiko wrote:
> > Use 32-bit subranges to prune some 64-bit BPF_JEQ/BPF_JNE conditions
> > that otherwise would be "inconclusive" (i.e., is_branch_taken() would
> > return -1). This can happen, for example, when registers are initialized
> > as 64-bit u64/s64, then compared for inequality as 32-bit subregisters,
> > and then followed by 64-bit equality/inequality check. That 32-bit
> > inequality can establish some pattern for lower 32 bits of a register
> > (e.g., s< 0 condition determines whether the bit #31 is zero or not),
> > while overall 64-bit value could be anything (according to a value range
> > representation).
> >
> > This is not a fancy quirky special case, but actually a handling that's
> > necessary to prevent correctness issue with BPF verifier's range
> > tracking: set_range_min_max() assumes that register ranges are
> > non-overlapping, and if that condition is not guaranteed by
> > is_branch_taken() we can end up with invalid ranges, where min > max.
>
> This is_scalar_branch_taken() logic makes sense,
> but if set_range_min_max() is delicate, it should have its own sanity
> check for ranges.
> Shouldn't be difficult to check for that dangerous overlap case.

So let me clarify. As far as I'm concerned, is_branch_taken() is such
a check for set_reg_min_max, and so duplicating such checks in
set_reg_min_max() is just that a duplication of code and logic, and
just a chance for more typos and subtle bugs.

But the concern about invalid ranges is valid, so I don't know,
perhaps we should just do a quick check after adjustment to validate
that umin<=umax and so on? E.g., we can do that outside of
reg_set_min_max(), to keep reg_set_min_max() non-failing. WDYT?

Re: [PATCH v5 bpf-next 03/23] bpf: derive smin/smax from umin/max bounds

[Eduard Zingerman]

On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> Add smin/smax derivation from appropriate umin/umax values. Previously the
> logic was surprisingly asymmetric, trying to derive umin/umax from smin/smax
> (if possible), but not trying to do the same in the other direction. A simple
> addition to __reg64_deduce_bounds() fixes this.
> 
> Added also generic comment about u64/s64 ranges and their relationship.
> Hopefully that helps readers to understand all the bounds deductions
> a bit better.
> 
> Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

Nice comment, thank you. I noticed two typos, see below.

> ---
>  kernel/bpf/verifier.c | 70 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 70 insertions(+)
> 
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 857d76694517..bf4193706744 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -2358,6 +2358,76 @@ static void __reg32_deduce_bounds(struct bpf_reg_state *reg)
>  
>  static void __reg64_deduce_bounds(struct bpf_reg_state *reg)
>  {
> +	/* If u64 range forms a valid s64 range (due to matching sign bit),
> +	 * try to learn from that. Let's do a bit of ASCII art to see when
> +	 * this is happening. Let's take u64 range first:
> +	 *
> +	 * 0             0x7fffffffffffffff 0x8000000000000000        U64_MAX
> +	 * |-------------------------------|--------------------------------|
> +	 *
> +	 * Valid u64 range is formed when umin and umax are anywhere in this
> +	 * range [0, U64_MAX] and umin <= umax. u64 is simple and
> +	 * straightforward. Let's where s64 range maps to this simple [0,
> +	 * U64_MAX] range, annotated below the line for comparison:

Nit: this sentence sounds a bit weird, probably some word is missing
     between "let's" and "where".

> +	 *
> +	 * 0             0x7fffffffffffffff 0x8000000000000000        U64_MAX
> +	 * |-------------------------------|--------------------------------|
> +	 * 0                        S64_MAX S64_MIN                        -1
> +	 *
> +	 * So s64 values basically start in the middle and then are contiguous
> +	 * to the right of it, wrapping around from -1 to 0, and then
> +	 * finishing as S64_MAX (0x7fffffffffffffff) right before S64_MIN.
> +	 * We can try drawing more visually continuity of u64 vs s64 values as
> +	 * mapped to just actual hex valued range of values.
> +	 *
> +	 *  u64 start                                               u64 end
> +	 *  _______________________________________________________________
> +	 * /                                                               \
> +	 * 0             0x7fffffffffffffff 0x8000000000000000        U64_MAX
> +	 * |-------------------------------|--------------------------------|
> +	 * 0                        S64_MAX S64_MIN                        -1
> +	 *                                / \
> +	 * >------------------------------   ------------------------------->
> +	 * s64 continues...        s64 end   s64 start          s64 "midpoint"
> +	 *
> +	 * What this means is that in general, we can't always derive
> +	 * something new about u64 from any random s64 range, and vice versa.
> +	 * But we can do that in two particular cases. One is when entire
> +	 * u64/s64 range is *entirely* contained within left half of the above
> +	 * diagram or when it is *entirely* contained in the right half. I.e.:
> +	 *
> +	 * |-------------------------------|--------------------------------|
> +	 *     ^                   ^            ^                 ^
> +	 *     A                   B            C                 D
> +	 *
> +	 * [A, B] and [C, D] are contained entirely in their respective halves
> +	 * and form valid contiguous ranges as both u64 and s64 values. [A, B]
> +	 * will be non-negative both as u64 and s64 (and in fact it will be
> +	 * identical ranges no matter the signedness). [C, D] treated as s64
> +	 * will be a range of negative values, while in u64 it will be
> +	 * non-negative range of values larger than 0x8000000000000000.
> +	 *
> +	 * Now, any other range here can't be represented in both u64 and s64
> +	 * simultaneously. E.g., [A, C], [A, D], [B, C], [B, D] are valid
> +	 * contiguous u64 ranges, but they are discontinuous in s64. [B, C]
> +	 * in s64 would be properly presented as [S64_MIN, C] and [B, S64_MAX],
> +	 * for example. Similarly, valid s64 range [D, A] (going from negative
> +	 * to positive values), would be two separate [D, U64_MAX] and [0, A]
> +	 * ranges as u64. Currently reg_state can't represent two segments per
> +	 * numeric domain, so in such situations we can only derive maximal
> +	 * possible range ([0, U64_MAX] for u64, and [S64_MIN, S64_MAX) for s64).
                                                                  ^
Nit:                                                      missing bracket

> +	 *
> +	 * So we use these facts to derive umin/umax from smin/smax and vice
> +	 * versa only if they stay within the same "half". This is equivalent
> +	 * to checking sign bit: lower half will have sign bit as zero, upper
> +	 * half have sign bit 1. Below in code we simplify this by just
> +	 * casting umin/umax as smin/smax and checking if they form valid
> +	 * range, and vice versa. Those are equivalent checks.
> +	 */
> +	if ((s64)reg->umin_value <= (s64)reg->umax_value) {
> +		reg->smin_value = max_t(s64, reg->smin_value, reg->umin_value);
> +		reg->smax_value = min_t(s64, reg->smax_value, reg->umax_value);
> +	}
>  	/* Learn sign from signed bounds.
>  	 * If we cannot cross the sign boundary, then signed and unsigned bounds
>  	 * are the same, so combine.  This works even in the negative case, e.g.

Re: [PATCH v5 bpf-next 04/23] bpf: derive smin32/smax32 from umin32/umax32 bounds

[Eduard Zingerman]

On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> > All the logic that applies to u64 vs s64, equally applies for u32 vs s32
> > relationships (just taken in a smaller 32-bit numeric space). So do the
> > same deduction of smin32/smax32 from umin32/umax32, if we can.
> > 
> > Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

> > ---
> >  kernel/bpf/verifier.c | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index bf4193706744..0f66e9092c38 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -2324,6 +2324,13 @@ static void __update_reg_bounds(struct bpf_reg_state *reg)
> >  /* Uses signed min/max values to inform unsigned, and vice-versa */
> >  static void __reg32_deduce_bounds(struct bpf_reg_state *reg)
> >  {
> > +	/* if u32 range forms a valid s32 range (due to matching sign bit),
> > +	 * try to learn from that
> > +	 */
> > +	if ((s32)reg->u32_min_value <= (s32)reg->u32_max_value) {
> > +		reg->s32_min_value = max_t(s32, reg->s32_min_value, reg->u32_min_value);
> > +		reg->s32_max_value = min_t(s32, reg->s32_max_value, reg->u32_max_value);
> > +	}
> >  	/* Learn sign from signed bounds.
> >  	 * If we cannot cross the sign boundary, then signed and unsigned bounds
> >  	 * are the same, so combine.  This works even in the negative case, e.g.

Re: [PATCH v5 bpf-next 05/23] bpf: derive subreg bounds from full bounds when upper 32 bits are constant

[Eduard Zingerman]

On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> > Comments in code try to explain the idea behind why this is correct.
> > Please check the code and comments.
> > 
> > Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

> > ---
> >  kernel/bpf/verifier.c | 45 +++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 45 insertions(+)
> > 
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index 0f66e9092c38..5082ca1ea5dc 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -2324,6 +2324,51 @@ static void __update_reg_bounds(struct bpf_reg_state *reg)
> >  /* Uses signed min/max values to inform unsigned, and vice-versa */
> >  static void __reg32_deduce_bounds(struct bpf_reg_state *reg)
> >  {
> > +	/* If upper 32 bits of u64/s64 range don't change, we can use lower 32
> > +	 * bits to improve our u32/s32 boundaries.
> > +	 *
> > +	 * E.g., the case where we have upper 32 bits as zero ([10, 20] in
> > +	 * u64) is pretty trivial, it's obvious that in u32 we'll also have
> > +	 * [10, 20] range. But this property holds for any 64-bit range as
> > +	 * long as upper 32 bits in that entire range of values stay the same.
> > +	 *
> > +	 * E.g., u64 range [0x10000000A, 0x10000000F] ([4294967306, 4294967311]
> > +	 * in decimal) has the same upper 32 bits throughout all the values in
> > +	 * that range. As such, lower 32 bits form a valid [0xA, 0xF] ([10, 15])
> > +	 * range.
> > +	 *
> > +	 * Note also, that [0xA, 0xF] is a valid range both in u32 and in s32,
> > +	 * following the rules outlined below about u64/s64 correspondence
> > +	 * (which equally applies to u32 vs s32 correspondence). In general it
> > +	 * depends on actual hexadecimal values of 32-bit range. They can form
> > +	 * only valid u32, or only valid s32 ranges in some cases.
> > +	 *
> > +	 * So we use all these insights to derive bounds for subregisters here.
> > +	 */
> > +	if ((reg->umin_value >> 32) == (reg->umax_value >> 32)) {
> > +		/* u64 to u32 casting preserves validity of low 32 bits as
> > +		 * a range, if upper 32 bits are the same
> > +		 */
> > +		reg->u32_min_value = max_t(u32, reg->u32_min_value, (u32)reg->umin_value);
> > +		reg->u32_max_value = min_t(u32, reg->u32_max_value, (u32)reg->umax_value);
> > +
> > +		if ((s32)reg->umin_value <= (s32)reg->umax_value) {
> > +			reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->umin_value);
> > +			reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->umax_value);
> > +		}
> > +	}
> > +	if ((reg->smin_value >> 32) == (reg->smax_value >> 32)) {
> > +		/* low 32 bits should form a proper u32 range */
> > +		if ((u32)reg->smin_value <= (u32)reg->smax_value) {
> > +			reg->u32_min_value = max_t(u32, reg->u32_min_value, (u32)reg->smin_value);
> > +			reg->u32_max_value = min_t(u32, reg->u32_max_value, (u32)reg->smax_value);
> > +		}
> > +		/* low 32 bits should form a proper s32 range */
> > +		if ((s32)reg->smin_value <= (s32)reg->smax_value) {
> > +			reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->smin_value);
> > +			reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->smax_value);
> > +		}
> > +	}
> >  	/* if u32 range forms a valid s32 range (due to matching sign bit),
> >  	 * try to learn from that
> >  	 */

Re: [PATCH v5 bpf-next 06/23] bpf: add special smin32/smax32 derivation from 64-bit bounds

[Eduard Zingerman]

On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> Add a special case where we can derive valid s32 bounds from umin/umax
> or smin/smax by stitching together negative s32 subrange and
> non-negative s32 subrange. That requires upper 32 bits to form a [N, N+1]
> range in u32 domain (taking into account wrap around, so 0xffffffff
> to 0x00000000 is a valid [N, N+1] range in this sense). See code comment
> for concrete examples.
> 
> Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

fwiw, an alternative explanation might be arithmetic based.
Suppose:
. there are numbers a, b, c
. 2**31 <= b < 2**32
. 0 <= c < 2**31
. umin = 2**32 * a + b
. umax = 2**32 * (a + 1) + c

The number of values in the range represented by [umin; umax] is:
. N = umax - umin + 1 = 2**32 + c - b + 1
. min(N) = 2**32 + 0 - (2**32-1) + 1 = 2
. max(N) = 2**32 + (2**31 - 1) - 2**31 + 1 = 2**32
Hence [(s32)b; (s32)c] form a valid range.

At-least that's how I convinced myself.

> ---
>  kernel/bpf/verifier.c | 23 +++++++++++++++++++++++
>  1 file changed, 23 insertions(+)
> 
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 5082ca1ea5dc..38d21d0e46bd 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -2369,6 +2369,29 @@ static void __reg32_deduce_bounds(struct bpf_reg_state *reg)
>  			reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->smax_value);
>  		}
>  	}
> +	/* Special case where upper bits form a small sequence of two
> +	 * sequential numbers (in 32-bit unsigned space, so 0xffffffff to
> +	 * 0x00000000 is also valid), while lower bits form a proper s32 range
> +	 * going from negative numbers to positive numbers. E.g., let's say we
> +	 * have s64 range [-1, 1] ([0xffffffffffffffff, 0x0000000000000001]).
> +	 * Possible s64 values are {-1, 0, 1} ({0xffffffffffffffff,
> +	 * 0x0000000000000000, 0x00000000000001}). Ignoring upper 32 bits,
> +	 * we still get a valid s32 range [-1, 1] ([0xffffffff, 0x00000001]).
> +	 * Note that it doesn't have to be 0xffffffff going to 0x00000000 in
> +	 * upper 32 bits. As a random example, s64 range
> +	 * [0xfffffff0ffffff00; 0xfffffff100000010], forms a valid s32 range
> +	 * [-16, 16] ([0xffffff00; 0x00000010]) in its 32 bit subregister.
> +	 */
> +	if ((u32)(reg->umin_value >> 32) + 1 == (u32)(reg->umax_value >> 32) &&
> +	    (s32)reg->umin_value < 0 && (s32)reg->umax_value >= 0) {
> +		reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->umin_value);
> +		reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->umax_value);
> +	}
> +	if ((u32)(reg->smin_value >> 32) + 1 == (u32)(reg->smax_value >> 32) &&
> +	    (s32)reg->smin_value < 0 && (s32)reg->smax_value >= 0) {
> +		reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->smin_value);
> +		reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->smax_value);
> +	}
>  	/* if u32 range forms a valid s32 range (due to matching sign bit),
>  	 * try to learn from that
>  	 */

Re: [PATCH v5 bpf-next 07/23] bpf: improve deduction of 64-bit bounds from 32-bit bounds

[Eduard Zingerman]

On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> > Add a few interesting cases in which we can tighten 64-bit bounds based
> > on newly learnt information about 32-bit bounds. E.g., when full u64/s64
> > registers are used in BPF program, and then eventually compared as
> > u32/s32. The latter comparison doesn't change the value of full
> > register, but it does impose new restrictions on possible lower 32 bits
> > of such full registers. And we can use that to derive additional full
> > register bounds information.
> > 
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

> > ---
> >  kernel/bpf/verifier.c | 47 +++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 47 insertions(+)
> > 
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index 38d21d0e46bd..768247e3d667 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -2535,10 +2535,57 @@ static void __reg64_deduce_bounds(struct bpf_reg_state *reg)
> >  	}
> >  }
> >  
> > +static void __reg_deduce_mixed_bounds(struct bpf_reg_state *reg)
> > +{
> > +	/* Try to tighten 64-bit bounds from 32-bit knowledge, using 32-bit
> > +	 * values on both sides of 64-bit range in hope to have tigher range.
> > +	 * E.g., if r1 is [0x1'00000000, 0x3'80000000], and we learn from
> > +	 * 32-bit signed > 0 operation that s32 bounds are now [1; 0x7fffffff].
> > +	 * With this, we can substitute 1 as low 32-bits of _low_ 64-bit bound
> > +	 * (0x100000000 -> 0x100000001) and 0x7fffffff as low 32-bits of
> > +	 * _high_ 64-bit bound (0x380000000 -> 0x37fffffff) and arrive at a
> > +	 * better overall bounds for r1 as [0x1'000000001; 0x3'7fffffff].
> > +	 * We just need to make sure that derived bounds we are intersecting
> > +	 * with are well-formed ranges in respecitve s64 or u64 domain, just
> > +	 * like we do with similar kinds of 32-to-64 or 64-to-32 adjustments.
> > +	 */
> > +	__u64 new_umin, new_umax;
> > +	__s64 new_smin, new_smax;
> > +
> > +	/* u32 -> u64 tightening, it's always well-formed */
> > +	new_umin = (reg->umin_value & ~0xffffffffULL) | reg->u32_min_value;
> > +	new_umax = (reg->umax_value & ~0xffffffffULL) | reg->u32_max_value;
> > +	reg->umin_value = max_t(u64, reg->umin_value, new_umin);
> > +	reg->umax_value = min_t(u64, reg->umax_value, new_umax);
> > +
> > +	/* s32 -> u64 tightening, s32 should be a valid u32 range (same sign) */
> > +	if ((u32)reg->s32_min_value <= (u32)reg->s32_max_value) {
> > +		new_umin = (reg->umin_value & ~0xffffffffULL) | (u32)reg->s32_min_value;
> > +		new_umax = (reg->umax_value & ~0xffffffffULL) | (u32)reg->s32_max_value;
> > +		reg->umin_value = max_t(u64, reg->umin_value, new_umin);
> > +		reg->umax_value = min_t(u64, reg->umax_value, new_umax);
> > +	}
> > +
> > +	/* u32 -> s64 tightening, u32 range embedded into s64 preserves range validity */
> > +	new_smin = (reg->smin_value & ~0xffffffffULL) | reg->u32_min_value;
> > +	new_smax = (reg->smax_value & ~0xffffffffULL) | reg->u32_max_value;
> > +	reg->smin_value = max_t(s64, reg->smin_value, new_smin);
> > +	reg->smax_value = min_t(s64, reg->smax_value, new_smax);
> > +
> > +	/* s32 -> s64 tightening, check that s32 range behaves as u32 range */
> > +	if ((u32)reg->s32_min_value <= (u32)reg->s32_max_value) {
> > +		new_smin = (reg->smin_value & ~0xffffffffULL) | (u32)reg->s32_min_value;
> > +		new_smax = (reg->smax_value & ~0xffffffffULL) | (u32)reg->s32_max_value;
> > +		reg->smin_value = max_t(s64, reg->smin_value, new_smin);
> > +		reg->smax_value = min_t(s64, reg->smax_value, new_smax);
> > +	}
> > +}
> > +
> >  static void __reg_deduce_bounds(struct bpf_reg_state *reg)
> >  {
> >  	__reg32_deduce_bounds(reg);
> >  	__reg64_deduce_bounds(reg);
> > +	__reg_deduce_mixed_bounds(reg);
> >  }
> >  
> >  /* Attempts to improve var_off based on unsigned min/max information */

Re: [PATCH v5 bpf-next 09/23] bpf: drop knowledge-losing __reg_combine_{32,64}_into_{64,32} logic

[Eduard Zingerman]

On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> > When performing 32-bit conditional operation operating on lower 32 bits
> > of a full 64-bit register, register full value isn't changed. We just
> > potentially gain new knowledge about that register's lower 32 bits.
> > 
> > Unfortunately, __reg_combine_{32,64}_into_{64,32} logic that
> > reg_set_min_max() performs as a last step, can lose information in some
> > cases due to __mark_reg64_unbounded() and __reg_assign_32_into_64().
> > That's bad and completely unnecessary. Especially __reg_assign_32_into_64()
> > looks completely out of place here, because we are not performing
> > zero-extending subregister assignment during conditional jump.
> > 
> > So this patch replaced __reg_combine_* with just a normal
> > reg_bounds_sync() which will do a proper job of deriving u64/s64 bounds
> > from u32/s32, and vice versa (among all other combinations).
> > 
> > __reg_combine_64_into_32() is also used in one more place,
> > coerce_reg_to_size(), while handling 1- and 2-byte register loads.
> > Looking into this, it seems like besides marking subregister as
> > unbounded before performing reg_bounds_sync(), we were also performing
> > deduction of smin32/smax32 and umin32/umax32 bounds from respective
> > smin/smax and umin/umax bounds. It's now redundant as reg_bounds_sync()
> > performs all the same logic more generically (e.g., without unnecessary
> > assumption that upper 32 bits of full register should be zero).
> > 
> > Long story short, we remove __reg_combine_64_into_32() completely, and
> > coerce_reg_to_size() now only does resetting subreg to unbounded and then
> > performing reg_bounds_sync() to recover as much information as possible
> > from 64-bit umin/umax and smin/smax bounds, set explicitly in
> > coerce_reg_to_size() earlier.
> > 
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

> > ---
> >  kernel/bpf/verifier.c | 60 ++++++-------------------------------------
> >  1 file changed, 8 insertions(+), 52 deletions(-)
> > 
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index 6b0736c04ebe..f5fcb7fb2c67 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -2641,51 +2641,6 @@ static void __reg_assign_32_into_64(struct bpf_reg_state *reg)
> >  	}
> >  }
> >  
> > -static void __reg_combine_32_into_64(struct bpf_reg_state *reg)
> > -{
> > -	/* special case when 64-bit register has upper 32-bit register
> > -	 * zeroed. Typically happens after zext or <<32, >>32 sequence
> > -	 * allowing us to use 32-bit bounds directly,
> > -	 */
> > -	if (tnum_equals_const(tnum_clear_subreg(reg->var_off), 0)) {
> > -		__reg_assign_32_into_64(reg);
> > -	} else {
> > -		/* Otherwise the best we can do is push lower 32bit known and
> > -		 * unknown bits into register (var_off set from jmp logic)
> > -		 * then learn as much as possible from the 64-bit tnum
> > -		 * known and unknown bits. The previous smin/smax bounds are
> > -		 * invalid here because of jmp32 compare so mark them unknown
> > -		 * so they do not impact tnum bounds calculation.
> > -		 */
> > -		__mark_reg64_unbounded(reg);
> > -	}
> > -	reg_bounds_sync(reg);
> > -}
> > -
> > -static bool __reg64_bound_s32(s64 a)
> > -{
> > -	return a >= S32_MIN && a <= S32_MAX;
> > -}
> > -
> > -static bool __reg64_bound_u32(u64 a)
> > -{
> > -	return a >= U32_MIN && a <= U32_MAX;
> > -}
> > -
> > -static void __reg_combine_64_into_32(struct bpf_reg_state *reg)
> > -{
> > -	__mark_reg32_unbounded(reg);
> > -	if (__reg64_bound_s32(reg->smin_value) && __reg64_bound_s32(reg->smax_value)) {
> > -		reg->s32_min_value = (s32)reg->smin_value;
> > -		reg->s32_max_value = (s32)reg->smax_value;
> > -	}
> > -	if (__reg64_bound_u32(reg->umin_value) && __reg64_bound_u32(reg->umax_value)) {
> > -		reg->u32_min_value = (u32)reg->umin_value;
> > -		reg->u32_max_value = (u32)reg->umax_value;
> > -	}
> > -	reg_bounds_sync(reg);
> > -}
> > -
> >  /* Mark a register as having a completely unknown (scalar) value. */
> >  static void __mark_reg_unknown(const struct bpf_verifier_env *env,
> >  			       struct bpf_reg_state *reg)
> > @@ -6382,9 +6337,10 @@ static void coerce_reg_to_size(struct bpf_reg_state *reg, int size)
> >  	 * values are also truncated so we push 64-bit bounds into
> >  	 * 32-bit bounds. Above were truncated < 32-bits already.
> >  	 */
> > -	if (size >= 4)
> > -		return;
> > -	__reg_combine_64_into_32(reg);
> > +	if (size < 4) {
> > +		__mark_reg32_unbounded(reg);
> > +		reg_bounds_sync(reg);
> > +	}
> >  }
> >  
> >  static void set_sext64_default_val(struct bpf_reg_state *reg, int size)
> > @@ -14623,13 +14579,13 @@ static void reg_set_min_max(struct bpf_reg_state *true_reg,
> >  					     tnum_subreg(false_32off));
> >  		true_reg->var_off = tnum_or(tnum_clear_subreg(true_64off),
> >  					    tnum_subreg(true_32off));
> > -		__reg_combine_32_into_64(false_reg);
> > -		__reg_combine_32_into_64(true_reg);
> > +		reg_bounds_sync(false_reg);
> > +		reg_bounds_sync(true_reg);
> >  	} else {
> >  		false_reg->var_off = false_64off;
> >  		true_reg->var_off = true_64off;
> > -		__reg_combine_64_into_32(false_reg);
> > -		__reg_combine_64_into_32(true_reg);
> > +		reg_bounds_sync(false_reg);
> > +		reg_bounds_sync(true_reg);
> >  	}
> >  }
> >  

Re: [PATCH v5 bpf-next 12/23] bpf: generalize is_branch_taken() to work with two registers

[Eduard Zingerman]

On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> > While still assuming that second register is a constant, generalize
> > is_branch_taken-related code to accept two registers instead of register
> > plus explicit constant value. This also, as a side effect, allows to
> > simplify check_cond_jmp_op() by unifying BPF_K case with BPF_X case, for
> > which we use a fake register to represent BPF_K's imm constant as
> > a register.
> > 
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

Please see a nitpick below.

> > ---
> >  kernel/bpf/verifier.c | 58 ++++++++++++++++++++++++-------------------
> >  1 file changed, 33 insertions(+), 25 deletions(-)
> > 
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index aa13f32751a1..fd328c579f10 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -14169,8 +14169,13 @@ static void find_good_pkt_pointers(struct bpf_verifier_state *vstate,
> >  	}));
> >  }
> >  
> > -static int is_branch32_taken(struct bpf_reg_state *reg1, u32 val, u8 opcode)
> > +/*
> > + * <reg1> <op> <reg2>, currently assuming reg2 is a constant
> > + */
> > +static int is_branch32_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, u8 opcode)
> >  {
> > +	struct tnum subreg = tnum_subreg(reg1->var_off);
> > +	u32 val = (u32)tnum_subreg(reg2->var_off).value;
> >  	s32 sval = (s32)val;
> >  
> >  	switch (opcode) {
> > @@ -14250,8 +14255,12 @@ static int is_branch32_taken(struct bpf_reg_state *reg1, u32 val, u8 opcode)
> >  }
> >  
> >  
> > -static int is_branch64_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode)
> > +/*
> > + * <reg1> <op> <reg2>, currently assuming reg2 is a constant
> > + */
> > +static int is_branch64_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, u8 opcode)
> >  {
> > +	u64 val = reg2->var_off.value;
> >  	s64 sval = (s64)val;
> >  
> >  	switch (opcode) {
> > @@ -14330,16 +14339,23 @@ static int is_branch64_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode)
> >  	return -1;
> >  }
> >  
> > -/* compute branch direction of the expression "if (reg opcode val) goto target;"
> > +/* compute branch direction of the expression "if (<reg1> opcode <reg2>) goto target;"
> >   * and return:
> >   *  1 - branch will be taken and "goto target" will be executed
> >   *  0 - branch will not be taken and fall-through to next insn
> > - * -1 - unknown. Example: "if (reg < 5)" is unknown when register value
> > + * -1 - unknown. Example: "if (reg1 < 5)" is unknown when register value
> >   *      range [0,10]
> >   */
> > -static int is_branch_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode,
> > -			   bool is_jmp32)
> > +static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
> > +			   u8 opcode, bool is_jmp32)
> >  {
> > +	struct tnum reg2_tnum = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
> > +	u64 val;
> > +
> > +	if (!tnum_is_const(reg2_tnum))
> > +		return -1;
> > +	val = reg2_tnum.value;
> > +
> >  	if (__is_pointer_value(false, reg1)) {
> >  		if (!reg_not_null(reg1))
> >  			return -1;
> > @@ -14361,8 +14377,8 @@ static int is_branch_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode,
> >  	}
> >  
> >  	if (is_jmp32)
> > -		return is_branch32_taken(reg1, val, opcode);
> > -	return is_branch64_taken(reg1, val, opcode);
> > +		return is_branch32_taken(reg1, reg2, opcode);
> > +	return is_branch64_taken(reg1, reg2, opcode);
> >  }
> >  
> >  static int flip_opcode(u32 opcode)
> > @@ -14833,6 +14849,7 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
> >  	struct bpf_reg_state *regs = this_branch->frame[this_branch->curframe]->regs;
> >  	struct bpf_reg_state *dst_reg, *other_branch_regs, *src_reg = NULL;
> >  	struct bpf_reg_state *eq_branch_regs;
> > +	struct bpf_reg_state fake_reg;

Nitpick:
bpf_reg_state has a lot of fields, e.g. 'parent' pointer. While it looks like
the use within this patch-set is safe, I suggest to change the declaration to
include '= {}' initializer. Just to err on a safe side for future modifications.

> >  	u8 opcode = BPF_OP(insn->code);
> >  	bool is_jmp32;
> >  	int pred = -1;
> > @@ -14873,36 +14890,27 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
> >  			verbose(env, "BPF_JMP/JMP32 uses reserved fields\n");
> >  			return -EINVAL;
> >  		}
> > +		src_reg = &fake_reg;
> > +		src_reg->type = SCALAR_VALUE;
> > +		__mark_reg_known(src_reg, insn->imm);
> >  	}
> >  
> >  	is_jmp32 = BPF_CLASS(insn->code) == BPF_JMP32;
> >  
> >  	if (BPF_SRC(insn->code) == BPF_K) {
> > -		pred = is_branch_taken(dst_reg, insn->imm, opcode, is_jmp32);
> > +		pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
> >  	} else if (src_reg->type == SCALAR_VALUE &&
> >  		   is_jmp32 && tnum_is_const(tnum_subreg(src_reg->var_off))) {
> > -		pred = is_branch_taken(dst_reg,
> > -				       tnum_subreg(src_reg->var_off).value,
> > -				       opcode,
> > -				       is_jmp32);
> > +		pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
> >  	} else if (src_reg->type == SCALAR_VALUE &&
> >  		   !is_jmp32 && tnum_is_const(src_reg->var_off)) {
> > -		pred = is_branch_taken(dst_reg,
> > -				       src_reg->var_off.value,
> > -				       opcode,
> > -				       is_jmp32);
> > +		pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
> >  	} else if (dst_reg->type == SCALAR_VALUE &&
> >  		   is_jmp32 && tnum_is_const(tnum_subreg(dst_reg->var_off))) {
> > -		pred = is_branch_taken(src_reg,
> > -				       tnum_subreg(dst_reg->var_off).value,
> > -				       flip_opcode(opcode),
> > -				       is_jmp32);
> > +		pred = is_branch_taken(src_reg, dst_reg, flip_opcode(opcode), is_jmp32);
> >  	} else if (dst_reg->type == SCALAR_VALUE &&
> >  		   !is_jmp32 && tnum_is_const(dst_reg->var_off)) {
> > -		pred = is_branch_taken(src_reg,
> > -				       dst_reg->var_off.value,
> > -				       flip_opcode(opcode),
> > -				       is_jmp32);
> > +		pred = is_branch_taken(src_reg, dst_reg, flip_opcode(opcode), is_jmp32);
> >  	} else if (reg_is_pkt_pointer_any(dst_reg) &&
> >  		   reg_is_pkt_pointer_any(src_reg) &&
> >  		   !is_jmp32) {

Re: [PATCH v5 bpf-next 14/23] bpf: generalize is_branch_taken to handle all conditional jumps in one place

[Eduard Zingerman]

On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> > Make is_branch_taken() a single entry point for branch pruning decision
> > making, handling both pointer vs pointer, pointer vs scalar, and scalar
> > vs scalar cases in one place. This also nicely cleans up check_cond_jmp_op().
> > 
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

> > ---
> >  kernel/bpf/verifier.c | 49 ++++++++++++++++++++++---------------------
> >  1 file changed, 25 insertions(+), 24 deletions(-)
> > 
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index 25b5234ebda3..fedd6d0e76e5 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -14169,6 +14169,19 @@ static void find_good_pkt_pointers(struct bpf_verifier_state *vstate,
> >  	}));
> >  }
> >  
> > +/* check if register is a constant scalar value */
> > +static bool is_reg_const(struct bpf_reg_state *reg, bool subreg32)
> > +{
> > +	return reg->type == SCALAR_VALUE &&
> > +	       tnum_is_const(subreg32 ? tnum_subreg(reg->var_off) : reg->var_off);
> > +}
> > +
> > +/* assuming is_reg_const() is true, return constant value of a register */
> > +static u64 reg_const_value(struct bpf_reg_state *reg, bool subreg32)
> > +{
> > +	return subreg32 ? tnum_subreg(reg->var_off).value : reg->var_off.value;
> > +}
> > +
> >  /*
> >   * <reg1> <op> <reg2>, currently assuming reg2 is a constant
> >   */
> > @@ -14410,12 +14423,20 @@ static int is_pkt_ptr_branch_taken(struct bpf_reg_state *dst_reg,
> >  static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
> >  			   u8 opcode, bool is_jmp32)
> >  {
> > -	struct tnum reg2_tnum = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
> >  	u64 val;
> >  
> > -	if (!tnum_is_const(reg2_tnum))
> > +	if (reg_is_pkt_pointer_any(reg1) && reg_is_pkt_pointer_any(reg2) && !is_jmp32)
> > +		return is_pkt_ptr_branch_taken(reg1, reg2, opcode);
> > +
> > +	/* try to make sure reg2 is a constant SCALAR_VALUE */
> > +	if (!is_reg_const(reg2, is_jmp32)) {
> > +		opcode = flip_opcode(opcode);
> > +		swap(reg1, reg2);
> > +	}
> > +	/* for now we expect reg2 to be a constant to make any useful decisions */
> > +	if (!is_reg_const(reg2, is_jmp32))
> >  		return -1;
> > -	val = reg2_tnum.value;
> > +	val = reg_const_value(reg2, is_jmp32);
> >  
> >  	if (__is_pointer_value(false, reg1)) {
> >  		if (!reg_not_null(reg1))
> > @@ -14896,27 +14917,7 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
> >  	}
> >  
> >  	is_jmp32 = BPF_CLASS(insn->code) == BPF_JMP32;
> > -
> > -	if (BPF_SRC(insn->code) == BPF_K) {
> > -		pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
> > -	} else if (src_reg->type == SCALAR_VALUE &&
> > -		   is_jmp32 && tnum_is_const(tnum_subreg(src_reg->var_off))) {
> > -		pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
> > -	} else if (src_reg->type == SCALAR_VALUE &&
> > -		   !is_jmp32 && tnum_is_const(src_reg->var_off)) {
> > -		pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
> > -	} else if (dst_reg->type == SCALAR_VALUE &&
> > -		   is_jmp32 && tnum_is_const(tnum_subreg(dst_reg->var_off))) {
> > -		pred = is_branch_taken(src_reg, dst_reg, flip_opcode(opcode), is_jmp32);
> > -	} else if (dst_reg->type == SCALAR_VALUE &&
> > -		   !is_jmp32 && tnum_is_const(dst_reg->var_off)) {
> > -		pred = is_branch_taken(src_reg, dst_reg, flip_opcode(opcode), is_jmp32);
> > -	} else if (reg_is_pkt_pointer_any(dst_reg) &&
> > -		   reg_is_pkt_pointer_any(src_reg) &&
> > -		   !is_jmp32) {
> > -		pred = is_pkt_ptr_branch_taken(dst_reg, src_reg, opcode);
> > -	}
> > -
> > +	pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
> >  	if (pred >= 0) {
> >  		/* If we get here with a dst_reg pointer type it is because
> >  		 * above is_branch_taken() special cased the 0 comparison.

Re: [PATCH v5 bpf-next 17/23] bpf: generalize reg_set_min_max() to handle two sets of two registers

[Alexei Starovoitov]

On Mon, Oct 30, 2023 at 11:03 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Mon, Oct 30, 2023 at 7:02 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Fri, Oct 27, 2023 at 11:13:40AM -0700, Andrii Nakryiko wrote:
> > >  static void reg_set_min_max(struct bpf_reg_state *true_reg1,
> > > +                         struct bpf_reg_state *true_reg2,
> > >                           struct bpf_reg_state *false_reg1,
> > > -                         u64 val, u32 val32,
> > > +                         struct bpf_reg_state *false_reg2,
> > >                           u8 opcode, bool is_jmp32)
> > >  {
> > > -     struct tnum false_32off = tnum_subreg(false_reg1->var_off);
> > > -     struct tnum false_64off = false_reg1->var_off;
> > > -     struct tnum true_32off = tnum_subreg(true_reg1->var_off);
> > > -     struct tnum true_64off = true_reg1->var_off;
> > > -     s64 sval = (s64)val;
> > > -     s32 sval32 = (s32)val32;
> > > -
> > > -     /* If the dst_reg is a pointer, we can't learn anything about its
> > > -      * variable offset from the compare (unless src_reg were a pointer into
> > > -      * the same object, but we don't bother with that.
> > > -      * Since false_reg1 and true_reg1 have the same type by construction, we
> > > -      * only need to check one of them for pointerness.
> > > +     struct tnum false_32off, false_64off;
> > > +     struct tnum true_32off, true_64off;
> > > +     u64 val;
> > > +     u32 val32;
> > > +     s64 sval;
> > > +     s32 sval32;
> > > +
> > > +     /* If either register is a pointer, we can't learn anything about its
> > > +      * variable offset from the compare (unless they were a pointer into
> > > +      * the same object, but we don't bother with that).
> > >        */
> > > -     if (__is_pointer_value(false, false_reg1))
> >
> > The removal of the above check, but not the comment was surprising and concerning,
> > so I did a bit of git-archaeology.
> > It was added in commit f1174f77b50c ("bpf/verifier: rework value tracking")
> > back in 2017 !
> > and in that commit reg_set_min_max() was always called with reg == scalar.
> > It looked like premature check. Then I spotted a comment in that commit:
> >   * this is only legit if both are scalars (or pointers to the same
> >   * object, I suppose, but we don't support that right now), because
> >   * otherwise the different base pointers mean the offsets aren't
> >   * comparable.
> > so the intent back then was to generalize reg_set_min_max() to be used with pointers too,
> > but we never got around to do that and the comment now reads:
>
> Yeah, it shouldn't be too hard to "generalize" to pointer vs pointer,
> if we ensure they point to exactly the same thing (I haven't thought
> much about how), because beyond that it's still basically SCALAR
> offsets. But I figured it's out of scope for these changes :)
>
> >   * this is only legit if both are scalars (or pointers to the same
> >   * object, I suppose, see the PTR_MAYBE_NULL related if block below),
> >   * because otherwise the different base pointers mean the offsets aren't
> >   * comparable.
> >
> > So please remove is_pointer check and remove the comment,
>
> So I'm a bit confused. I did remove __is_pointer_value() check, but I
> still need to guard against having pointers, which is why I have:
>
> if (false_reg1->type != SCALAR_VALUE || false_reg2->type != SCALAR_VALUE).
>     return;
>
> I think I need this check, because reg_set_min_max() can be called
> from check_cond_jmp_op() with pointer regs, and we shouldn't try to
> adjust them. Or am I missing something? And the comment I have here
> now:

I don't see a code path where reg_set_min_max() is called
with pointers. At least not in the current code base.
Are you saying somewhere in your later patch it happens?

Then the question is whether to do this is_scalar check inside
reg_set_min_max() or outside. Both options are probably fine.

>
> +       /* If either register is a pointer, we can't learn anything about its
> +        * variable offset from the compare (unless they were a pointer into
> +        * the same object, but we don't bother with that).
>          */
>
> is trying to explain that we don't really adjust two pointers.
>
> > and fixup the comment in check_cond_jmp_op() where reg_set_min_max().
>
> I have this locally for now, please let me know if this is fine or you
> had something else in mind:
>
> -/* Adjusts the register min/max values in the case that the dst_reg is the
> - * variable register that we are working on, and src_reg is a constant or we're
> - * simply doing a BPF_K check.
> - * In JEQ/JNE cases we also adjust the var_off values.
> +/* Adjusts the register min/max values in the case that the dst_reg and
> + * src_reg are both SCALAR_VALUE registers (or we are simply doing a BPF_K
> + * check, in which case we havea fake SCALAR_VALUE representing insn->imm).
> + * Technically we can do similar adjustments for pointers to the same object,
> + * but we don't support that right now.

Looks fine. I'm trying to say that we had such comment forever and
it never lead to actually doing the work.
So I'd just remove the last sentence about pointers ...

>   */
>  static void reg_set_min_max(struct bpf_reg_state *true_reg1,
>                             struct bpf_reg_state *true_reg2,
> @@ -14884,13 +14885,6 @@ static int check_cond_jmp_op(struct
> bpf_verifier_env *env,
>                 return -EFAULT;
>         other_branch_regs = other_branch->frame[other_branch->curframe]->regs;
>
> -       /* detect if we are comparing against a constant value so we can adjust
> -        * our min/max values for our dst register.
> -        * this is only legit if both are scalars (or pointers to the same
> -        * object, I suppose, see the PTR_MAYBE_NULL related if block below),
> -        * because otherwise the different base pointers mean the offsets aren't
> -        * comparable.
> -        */

... and removing this comment is good thing too.
In general the comments should be in front of the function body
(as you're doing) instead of the callsite.

Re: [PATCH v5 bpf-next 19/23] bpf: generalize is_scalar_branch_taken() logic

[Alexei Starovoitov]

On Mon, Oct 30, 2023 at 11:12 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Mon, Oct 30, 2023 at 7:12 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Fri, Oct 27, 2023 at 11:13:42AM -0700, Andrii Nakryiko wrote:
> > > Generalize is_branch_taken logic for SCALAR_VALUE register to handle
> > > cases when both registers are not constants. Previously supported
> > > <range> vs <scalar> cases are a natural subset of more generic <range>
> > > vs <range> set of cases.
> > >
> > > Generalized logic relies on straightforward segment intersection checks.
> > >
> > > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > > ---
> > >  kernel/bpf/verifier.c | 104 ++++++++++++++++++++++++++----------------
> > >  1 file changed, 64 insertions(+), 40 deletions(-)
> > >
> > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > > index 4c974296127b..f18a8247e5e2 100644
> > > --- a/kernel/bpf/verifier.c
> > > +++ b/kernel/bpf/verifier.c
> > > @@ -14189,82 +14189,105 @@ static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_sta
> > >                                 u8 opcode, bool is_jmp32)
> > >  {
> > >       struct tnum t1 = is_jmp32 ? tnum_subreg(reg1->var_off) : reg1->var_off;
> > > +     struct tnum t2 = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
> > >       u64 umin1 = is_jmp32 ? (u64)reg1->u32_min_value : reg1->umin_value;
> > >       u64 umax1 = is_jmp32 ? (u64)reg1->u32_max_value : reg1->umax_value;
> > >       s64 smin1 = is_jmp32 ? (s64)reg1->s32_min_value : reg1->smin_value;
> > >       s64 smax1 = is_jmp32 ? (s64)reg1->s32_max_value : reg1->smax_value;
> > > -     u64 val = is_jmp32 ? (u32)tnum_subreg(reg2->var_off).value : reg2->var_off.value;
> > > -     s64 sval = is_jmp32 ? (s32)val : (s64)val;
> > > +     u64 umin2 = is_jmp32 ? (u64)reg2->u32_min_value : reg2->umin_value;
> > > +     u64 umax2 = is_jmp32 ? (u64)reg2->u32_max_value : reg2->umax_value;
> > > +     s64 smin2 = is_jmp32 ? (s64)reg2->s32_min_value : reg2->smin_value;
> > > +     s64 smax2 = is_jmp32 ? (s64)reg2->s32_max_value : reg2->smax_value;
> > >
> > >       switch (opcode) {
> > >       case BPF_JEQ:
> > > -             if (tnum_is_const(t1))
> > > -                     return !!tnum_equals_const(t1, val);
> > > -             else if (val < umin1 || val > umax1)
> > > +             /* const tnums */
> > > +             if (tnum_is_const(t1) && tnum_is_const(t2))
> > > +                     return t1.value == t2.value;
> > > +             /* const ranges */
> > > +             if (umin1 == umax1 && umin2 == umax2)
> > > +                     return umin1 == umin2;
> >
> > I don't follow this logic.
> > umin1 == umax1 means that it's a single constant and
> > it should have been handled by earlier tnum_is_const check.
>
> I think you follow the logic, you just think it's redundant. Yes, it's
> basically the same as
>
>           if (tnum_is_const(t1) && tnum_is_const(t2))
>                 return t1.value == t2.value;
>
> but based on ranges. I didn't feel comfortable to assume that if umin1
> == umax1 then tnum_is_const(t1) will always be true. At worst we'll
> perform one redundant check.
>
> In short, I don't trust tnum to be as precise as umin/umax and other ranges.
>
> >
> > > +             if (smin1 == smax1 && smin2 == smax2)
> > > +                     return umin1 == umin2;
> >
> > here it's even more confusing. smin == smax -> singel const,
> > but then compare umin1 with umin2 ?!
>
> Eagle eyes! Typo, sorry :( it should be `smin1 == smin2`, of course.
>
> What saves us is reg_bounds_sync(), and if we have umin1 == umax1 then
> we'll have also smin1 == smax1 == umin1 == umax1 (and corresponding
> relation for second register). But I fixed these typos in both BPF_JEQ
> and BPF_JNE branches.

Not just 'saves us'. The tnum <-> bounds sync is mandatory.
I think we have a test where a function returns [-errno, 0]
and then we do if (ret < 0) check. At this point the reg has
to be tnum_is_const and zero.
So if smin1 == smax1 == umin1 == umax1 it should be tnum_is_const.
Otherwise it's a bug in sync logic.
I think instead of doing redundant and confusing check may be
add WARN either here or in sync logic to make sure it's all good ?

Re: [PATCH v5 bpf-next 20/23] bpf: enhance BPF_JEQ/BPF_JNE is_branch_taken logic

[Alexei Starovoitov]

On Mon, Oct 30, 2023 at 11:16 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Mon, Oct 30, 2023 at 7:20 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Fri, Oct 27, 2023 at 11:13:43AM -0700, Andrii Nakryiko wrote:
> > > Use 32-bit subranges to prune some 64-bit BPF_JEQ/BPF_JNE conditions
> > > that otherwise would be "inconclusive" (i.e., is_branch_taken() would
> > > return -1). This can happen, for example, when registers are initialized
> > > as 64-bit u64/s64, then compared for inequality as 32-bit subregisters,
> > > and then followed by 64-bit equality/inequality check. That 32-bit
> > > inequality can establish some pattern for lower 32 bits of a register
> > > (e.g., s< 0 condition determines whether the bit #31 is zero or not),
> > > while overall 64-bit value could be anything (according to a value range
> > > representation).
> > >
> > > This is not a fancy quirky special case, but actually a handling that's
> > > necessary to prevent correctness issue with BPF verifier's range
> > > tracking: set_range_min_max() assumes that register ranges are
> > > non-overlapping, and if that condition is not guaranteed by
> > > is_branch_taken() we can end up with invalid ranges, where min > max.
> >
> > This is_scalar_branch_taken() logic makes sense,
> > but if set_range_min_max() is delicate, it should have its own sanity
> > check for ranges.
> > Shouldn't be difficult to check for that dangerous overlap case.
>
> So let me clarify. As far as I'm concerned, is_branch_taken() is such
> a check for set_reg_min_max, and so duplicating such checks in
> set_reg_min_max() is just that a duplication of code and logic, and
> just a chance for more typos and subtle bugs.
>
> But the concern about invalid ranges is valid, so I don't know,
> perhaps we should just do a quick check after adjustment to validate
> that umin<=umax and so on? E.g., we can do that outside of
> reg_set_min_max(), to keep reg_set_min_max() non-failing. WDYT?

Sounds like a good option too.
Just trying to minimize breakage in the future.
Sanity check before or after should catch it.

Re: [PATCH v5 bpf-next 03/23] bpf: derive smin/smax from umin/max bounds

[Andrii Nakryiko]

On Tue, Oct 31, 2023 at 8:37 AM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> > Add smin/smax derivation from appropriate umin/umax values. Previously the
> > logic was surprisingly asymmetric, trying to derive umin/umax from smin/smax
> > (if possible), but not trying to do the same in the other direction. A simple
> > addition to __reg64_deduce_bounds() fixes this.
> >
> > Added also generic comment about u64/s64 ranges and their relationship.
> > Hopefully that helps readers to understand all the bounds deductions
> > a bit better.
> >
> > Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
>
> Acked-by: Eduard Zingerman <eddyz87@gmail.com>
>
> Nice comment, thank you. I noticed two typos, see below.
>
> > ---
> >  kernel/bpf/verifier.c | 70 +++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 70 insertions(+)
> >
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index 857d76694517..bf4193706744 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -2358,6 +2358,76 @@ static void __reg32_deduce_bounds(struct bpf_reg_state *reg)
> >
> >  static void __reg64_deduce_bounds(struct bpf_reg_state *reg)
> >  {
> > +     /* If u64 range forms a valid s64 range (due to matching sign bit),
> > +      * try to learn from that. Let's do a bit of ASCII art to see when
> > +      * this is happening. Let's take u64 range first:
> > +      *
> > +      * 0             0x7fffffffffffffff 0x8000000000000000        U64_MAX
> > +      * |-------------------------------|--------------------------------|
> > +      *
> > +      * Valid u64 range is formed when umin and umax are anywhere in this
> > +      * range [0, U64_MAX] and umin <= umax. u64 is simple and
> > +      * straightforward. Let's where s64 range maps to this simple [0,
> > +      * U64_MAX] range, annotated below the line for comparison:
>
> Nit: this sentence sounds a bit weird, probably some word is missing
>      between "let's" and "where".
>

I don't know what's going on here, I wasn't drunk when I wrote this
and I don't remember it being so incoherent :) Will re-read and try to
make it clearer.

> > +      *
> > +      * 0             0x7fffffffffffffff 0x8000000000000000        U64_MAX
> > +      * |-------------------------------|--------------------------------|
> > +      * 0                        S64_MAX S64_MIN                        -1
> > +      *
> > +      * So s64 values basically start in the middle and then are contiguous
> > +      * to the right of it, wrapping around from -1 to 0, and then
> > +      * finishing as S64_MAX (0x7fffffffffffffff) right before S64_MIN.
> > +      * We can try drawing more visually continuity of u64 vs s64 values as
> > +      * mapped to just actual hex valued range of values.
> > +      *
> > +      *  u64 start                                               u64 end
> > +      *  _______________________________________________________________
> > +      * /                                                               \
> > +      * 0             0x7fffffffffffffff 0x8000000000000000        U64_MAX
> > +      * |-------------------------------|--------------------------------|
> > +      * 0                        S64_MAX S64_MIN                        -1
> > +      *                                / \
> > +      * >------------------------------   ------------------------------->
> > +      * s64 continues...        s64 end   s64 start          s64 "midpoint"
> > +      *
> > +      * What this means is that in general, we can't always derive
> > +      * something new about u64 from any random s64 range, and vice versa.
> > +      * But we can do that in two particular cases. One is when entire
> > +      * u64/s64 range is *entirely* contained within left half of the above
> > +      * diagram or when it is *entirely* contained in the right half. I.e.:
> > +      *
> > +      * |-------------------------------|--------------------------------|
> > +      *     ^                   ^            ^                 ^
> > +      *     A                   B            C                 D
> > +      *
> > +      * [A, B] and [C, D] are contained entirely in their respective halves
> > +      * and form valid contiguous ranges as both u64 and s64 values. [A, B]
> > +      * will be non-negative both as u64 and s64 (and in fact it will be
> > +      * identical ranges no matter the signedness). [C, D] treated as s64
> > +      * will be a range of negative values, while in u64 it will be
> > +      * non-negative range of values larger than 0x8000000000000000.
> > +      *
> > +      * Now, any other range here can't be represented in both u64 and s64
> > +      * simultaneously. E.g., [A, C], [A, D], [B, C], [B, D] are valid
> > +      * contiguous u64 ranges, but they are discontinuous in s64. [B, C]
> > +      * in s64 would be properly presented as [S64_MIN, C] and [B, S64_MAX],
> > +      * for example. Similarly, valid s64 range [D, A] (going from negative
> > +      * to positive values), would be two separate [D, U64_MAX] and [0, A]
> > +      * ranges as u64. Currently reg_state can't represent two segments per
> > +      * numeric domain, so in such situations we can only derive maximal
> > +      * possible range ([0, U64_MAX] for u64, and [S64_MIN, S64_MAX) for s64).
>                                                                   ^
> Nit:                                                      missing bracket
>

it's actually a typo, ) -> ], which is now fixed as well, thanks

> > +      *
> > +      * So we use these facts to derive umin/umax from smin/smax and vice
> > +      * versa only if they stay within the same "half". This is equivalent
> > +      * to checking sign bit: lower half will have sign bit as zero, upper
> > +      * half have sign bit 1. Below in code we simplify this by just
> > +      * casting umin/umax as smin/smax and checking if they form valid
> > +      * range, and vice versa. Those are equivalent checks.
> > +      */
> > +     if ((s64)reg->umin_value <= (s64)reg->umax_value) {
> > +             reg->smin_value = max_t(s64, reg->smin_value, reg->umin_value);
> > +             reg->smax_value = min_t(s64, reg->smax_value, reg->umax_value);
> > +     }
> >       /* Learn sign from signed bounds.
> >        * If we cannot cross the sign boundary, then signed and unsigned bounds
> >        * are the same, so combine.  This works even in the negative case, e.g.
>
>
>

Re: [PATCH v5 bpf-next 15/23] bpf: unify 32-bit and 64-bit is_branch_taken logic

[Eduard Zingerman]

On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> Combine 32-bit and 64-bit is_branch_taken logic for SCALAR_VALUE
> registers. It makes it easier to see parallels between two domains
> (32-bit and 64-bit), and makes subsequent refactoring more
> straightforward.
> 
> No functional changes.
> 
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

> ---
>  kernel/bpf/verifier.c | 154 ++++++++++--------------------------------
>  1 file changed, 36 insertions(+), 118 deletions(-)
> 
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index fedd6d0e76e5..b911d1111fad 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -14185,166 +14185,86 @@ static u64 reg_const_value(struct bpf_reg_state *reg, bool subreg32)
>  /*
>   * <reg1> <op> <reg2>, currently assuming reg2 is a constant
>   */
> -static int is_branch32_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, u8 opcode)
> +static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
> +				  u8 opcode, bool is_jmp32)
>  {
> -	struct tnum subreg = tnum_subreg(reg1->var_off);
> -	u32 val = (u32)tnum_subreg(reg2->var_off).value;
> -	s32 sval = (s32)val;
> +	struct tnum t1 = is_jmp32 ? tnum_subreg(reg1->var_off) : reg1->var_off;
> +	u64 umin1 = is_jmp32 ? (u64)reg1->u32_min_value : reg1->umin_value;
> +	u64 umax1 = is_jmp32 ? (u64)reg1->u32_max_value : reg1->umax_value;
> +	s64 smin1 = is_jmp32 ? (s64)reg1->s32_min_value : reg1->smin_value;
> +	s64 smax1 = is_jmp32 ? (s64)reg1->s32_max_value : reg1->smax_value;
> +	u64 val = is_jmp32 ? (u32)tnum_subreg(reg2->var_off).value : reg2->var_off.value;
> +	s64 sval = is_jmp32 ? (s32)val : (s64)val;
>  
>  	switch (opcode) {
>  	case BPF_JEQ:
> -		if (tnum_is_const(subreg))
> -			return !!tnum_equals_const(subreg, val);
> -		else if (val < reg1->u32_min_value || val > reg1->u32_max_value)
> +		if (tnum_is_const(t1))
> +			return !!tnum_equals_const(t1, val);
> +		else if (val < umin1 || val > umax1)
>  			return 0;
> -		else if (sval < reg1->s32_min_value || sval > reg1->s32_max_value)
> +		else if (sval < smin1 || sval > smax1)
>  			return 0;
>  		break;
>  	case BPF_JNE:
> -		if (tnum_is_const(subreg))
> -			return !tnum_equals_const(subreg, val);
> -		else if (val < reg1->u32_min_value || val > reg1->u32_max_value)
> +		if (tnum_is_const(t1))
> +			return !tnum_equals_const(t1, val);
> +		else if (val < umin1 || val > umax1)
>  			return 1;
> -		else if (sval < reg1->s32_min_value || sval > reg1->s32_max_value)
> +		else if (sval < smin1 || sval > smax1)
>  			return 1;
>  		break;
>  	case BPF_JSET:
> -		if ((~subreg.mask & subreg.value) & val)
> +		if ((~t1.mask & t1.value) & val)
>  			return 1;
> -		if (!((subreg.mask | subreg.value) & val))
> +		if (!((t1.mask | t1.value) & val))
>  			return 0;
>  		break;
>  	case BPF_JGT:
> -		if (reg1->u32_min_value > val)
> +		if (umin1 > val )
>  			return 1;
> -		else if (reg1->u32_max_value <= val)
> +		else if (umax1 <= val)
>  			return 0;
>  		break;
>  	case BPF_JSGT:
> -		if (reg1->s32_min_value > sval)
> +		if (smin1 > sval)
>  			return 1;
> -		else if (reg1->s32_max_value <= sval)
> +		else if (smax1 <= sval)
>  			return 0;
>  		break;
>  	case BPF_JLT:
> -		if (reg1->u32_max_value < val)
> +		if (umax1 < val)
>  			return 1;
> -		else if (reg1->u32_min_value >= val)
> +		else if (umin1 >= val)
>  			return 0;
>  		break;
>  	case BPF_JSLT:
> -		if (reg1->s32_max_value < sval)
> +		if (smax1 < sval)
>  			return 1;
> -		else if (reg1->s32_min_value >= sval)
> +		else if (smin1 >= sval)
>  			return 0;
>  		break;
>  	case BPF_JGE:
> -		if (reg1->u32_min_value >= val)
> +		if (umin1 >= val)
>  			return 1;
> -		else if (reg1->u32_max_value < val)
> +		else if (umax1 < val)
>  			return 0;
>  		break;
>  	case BPF_JSGE:
> -		if (reg1->s32_min_value >= sval)
> +		if (smin1 >= sval)
>  			return 1;
> -		else if (reg1->s32_max_value < sval)
> +		else if (smax1 < sval)
>  			return 0;
>  		break;
>  	case BPF_JLE:
> -		if (reg1->u32_max_value <= val)
> +		if (umax1 <= val)
>  			return 1;
> -		else if (reg1->u32_min_value > val)
> +		else if (umin1 > val)
>  			return 0;
>  		break;
>  	case BPF_JSLE:
> -		if (reg1->s32_max_value <= sval)
> +		if (smax1 <= sval)
>  			return 1;
> -		else if (reg1->s32_min_value > sval)
> -			return 0;
> -		break;
> -	}
> -
> -	return -1;
> -}
> -
> -
> -/*
> - * <reg1> <op> <reg2>, currently assuming reg2 is a constant
> - */
> -static int is_branch64_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, u8 opcode)
> -{
> -	u64 val = reg2->var_off.value;
> -	s64 sval = (s64)val;
> -
> -	switch (opcode) {
> -	case BPF_JEQ:
> -		if (tnum_is_const(reg1->var_off))
> -			return !!tnum_equals_const(reg1->var_off, val);
> -		else if (val < reg1->umin_value || val > reg1->umax_value)
> -			return 0;
> -		else if (sval < reg1->smin_value || sval > reg1->smax_value)
> -			return 0;
> -		break;
> -	case BPF_JNE:
> -		if (tnum_is_const(reg1->var_off))
> -			return !tnum_equals_const(reg1->var_off, val);
> -		else if (val < reg1->umin_value || val > reg1->umax_value)
> -			return 1;
> -		else if (sval < reg1->smin_value || sval > reg1->smax_value)
> -			return 1;
> -		break;
> -	case BPF_JSET:
> -		if ((~reg1->var_off.mask & reg1->var_off.value) & val)
> -			return 1;
> -		if (!((reg1->var_off.mask | reg1->var_off.value) & val))
> -			return 0;
> -		break;
> -	case BPF_JGT:
> -		if (reg1->umin_value > val)
> -			return 1;
> -		else if (reg1->umax_value <= val)
> -			return 0;
> -		break;
> -	case BPF_JSGT:
> -		if (reg1->smin_value > sval)
> -			return 1;
> -		else if (reg1->smax_value <= sval)
> -			return 0;
> -		break;
> -	case BPF_JLT:
> -		if (reg1->umax_value < val)
> -			return 1;
> -		else if (reg1->umin_value >= val)
> -			return 0;
> -		break;
> -	case BPF_JSLT:
> -		if (reg1->smax_value < sval)
> -			return 1;
> -		else if (reg1->smin_value >= sval)
> -			return 0;
> -		break;
> -	case BPF_JGE:
> -		if (reg1->umin_value >= val)
> -			return 1;
> -		else if (reg1->umax_value < val)
> -			return 0;
> -		break;
> -	case BPF_JSGE:
> -		if (reg1->smin_value >= sval)
> -			return 1;
> -		else if (reg1->smax_value < sval)
> -			return 0;
> -		break;
> -	case BPF_JLE:
> -		if (reg1->umax_value <= val)
> -			return 1;
> -		else if (reg1->umin_value > val)
> -			return 0;
> -		break;
> -	case BPF_JSLE:
> -		if (reg1->smax_value <= sval)
> -			return 1;
> -		else if (reg1->smin_value > sval)
> +		else if (smin1 > sval)
>  			return 0;
>  		break;
>  	}
> @@ -14458,9 +14378,7 @@ static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg
>  		}
>  	}
>  
> -	if (is_jmp32)
> -		return is_branch32_taken(reg1, reg2, opcode);
> -	return is_branch64_taken(reg1, reg2, opcode);
> +	return is_scalar_branch_taken(reg1, reg2, opcode, is_jmp32);
>  }
>  
>  /* Adjusts the register min/max values in the case that the dst_reg is the

Re: [PATCH v5 bpf-next 06/23] bpf: add special smin32/smax32 derivation from 64-bit bounds

[Andrii Nakryiko]

On Tue, Oct 31, 2023 at 8:37 AM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> > Add a special case where we can derive valid s32 bounds from umin/umax
> > or smin/smax by stitching together negative s32 subrange and
> > non-negative s32 subrange. That requires upper 32 bits to form a [N, N+1]
> > range in u32 domain (taking into account wrap around, so 0xffffffff
> > to 0x00000000 is a valid [N, N+1] range in this sense). See code comment
> > for concrete examples.
> >
> > Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
>
> Acked-by: Eduard Zingerman <eddyz87@gmail.com>
>
> fwiw, an alternative explanation might be arithmetic based.
> Suppose:
> . there are numbers a, b, c
> . 2**31 <= b < 2**32
> . 0 <= c < 2**31
> . umin = 2**32 * a + b
> . umax = 2**32 * (a + 1) + c
>
> The number of values in the range represented by [umin; umax] is:
> . N = umax - umin + 1 = 2**32 + c - b + 1
> . min(N) = 2**32 + 0 - (2**32-1) + 1 = 2
> . max(N) = 2**32 + (2**31 - 1) - 2**31 + 1 = 2**32
> Hence [(s32)b; (s32)c] form a valid range.
>
> At-least that's how I convinced myself.

So the logic here follows the (visual) intuition how s64 and u64 (and
also u32 and s32) correlate. That's how I saw it. TBH, the above
mathematical way seems scary and not so straightforward to follow, so
I'm hesitant to add it to comments to not scare anyone away :)

I did try to visually represent it, but I'm not creative enough ASCII
artist to pull this off, apparently. I'll just leave it as it is for
now.

>
> > ---
> >  kernel/bpf/verifier.c | 23 +++++++++++++++++++++++
> >  1 file changed, 23 insertions(+)
> >
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index 5082ca1ea5dc..38d21d0e46bd 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -2369,6 +2369,29 @@ static void __reg32_deduce_bounds(struct bpf_reg_state *reg)
> >                       reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->smax_value);
> >               }
> >       }
> > +     /* Special case where upper bits form a small sequence of two
> > +      * sequential numbers (in 32-bit unsigned space, so 0xffffffff to
> > +      * 0x00000000 is also valid), while lower bits form a proper s32 range
> > +      * going from negative numbers to positive numbers. E.g., let's say we
> > +      * have s64 range [-1, 1] ([0xffffffffffffffff, 0x0000000000000001]).
> > +      * Possible s64 values are {-1, 0, 1} ({0xffffffffffffffff,
> > +      * 0x0000000000000000, 0x00000000000001}). Ignoring upper 32 bits,
> > +      * we still get a valid s32 range [-1, 1] ([0xffffffff, 0x00000001]).
> > +      * Note that it doesn't have to be 0xffffffff going to 0x00000000 in
> > +      * upper 32 bits. As a random example, s64 range
> > +      * [0xfffffff0ffffff00; 0xfffffff100000010], forms a valid s32 range
> > +      * [-16, 16] ([0xffffff00; 0x00000010]) in its 32 bit subregister.
> > +      */
> > +     if ((u32)(reg->umin_value >> 32) + 1 == (u32)(reg->umax_value >> 32) &&
> > +         (s32)reg->umin_value < 0 && (s32)reg->umax_value >= 0) {
> > +             reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->umin_value);
> > +             reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->umax_value);
> > +     }
> > +     if ((u32)(reg->smin_value >> 32) + 1 == (u32)(reg->smax_value >> 32) &&
> > +         (s32)reg->smin_value < 0 && (s32)reg->smax_value >= 0) {
> > +             reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->smin_value);
> > +             reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->smax_value);
> > +     }
> >       /* if u32 range forms a valid s32 range (due to matching sign bit),
> >        * try to learn from that
> >        */
>
>
>

Re: [PATCH v5 bpf-next 12/23] bpf: generalize is_branch_taken() to work with two registers

[Andrii Nakryiko]

On Tue, Oct 31, 2023 at 8:38 AM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> > > While still assuming that second register is a constant, generalize
> > > is_branch_taken-related code to accept two registers instead of register
> > > plus explicit constant value. This also, as a side effect, allows to
> > > simplify check_cond_jmp_op() by unifying BPF_K case with BPF_X case, for
> > > which we use a fake register to represent BPF_K's imm constant as
> > > a register.
> > >
> > > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
>
> Acked-by: Eduard Zingerman <eddyz87@gmail.com>
>
> Please see a nitpick below.
>
> > > ---
> > >  kernel/bpf/verifier.c | 58 ++++++++++++++++++++++++-------------------
> > >  1 file changed, 33 insertions(+), 25 deletions(-)
> > >
> > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > > index aa13f32751a1..fd328c579f10 100644
> > > --- a/kernel/bpf/verifier.c
> > > +++ b/kernel/bpf/verifier.c
> > > @@ -14169,8 +14169,13 @@ static void find_good_pkt_pointers(struct bpf_verifier_state *vstate,
> > >     }));
> > >  }
> > >
> > > -static int is_branch32_taken(struct bpf_reg_state *reg1, u32 val, u8 opcode)
> > > +/*
> > > + * <reg1> <op> <reg2>, currently assuming reg2 is a constant
> > > + */
> > > +static int is_branch32_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, u8 opcode)
> > >  {
> > > +   struct tnum subreg = tnum_subreg(reg1->var_off);
> > > +   u32 val = (u32)tnum_subreg(reg2->var_off).value;
> > >     s32 sval = (s32)val;
> > >
> > >     switch (opcode) {
> > > @@ -14250,8 +14255,12 @@ static int is_branch32_taken(struct bpf_reg_state *reg1, u32 val, u8 opcode)
> > >  }
> > >
> > >
> > > -static int is_branch64_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode)
> > > +/*
> > > + * <reg1> <op> <reg2>, currently assuming reg2 is a constant
> > > + */
> > > +static int is_branch64_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, u8 opcode)
> > >  {
> > > +   u64 val = reg2->var_off.value;
> > >     s64 sval = (s64)val;
> > >
> > >     switch (opcode) {
> > > @@ -14330,16 +14339,23 @@ static int is_branch64_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode)
> > >     return -1;
> > >  }
> > >
> > > -/* compute branch direction of the expression "if (reg opcode val) goto target;"
> > > +/* compute branch direction of the expression "if (<reg1> opcode <reg2>) goto target;"
> > >   * and return:
> > >   *  1 - branch will be taken and "goto target" will be executed
> > >   *  0 - branch will not be taken and fall-through to next insn
> > > - * -1 - unknown. Example: "if (reg < 5)" is unknown when register value
> > > + * -1 - unknown. Example: "if (reg1 < 5)" is unknown when register value
> > >   *      range [0,10]
> > >   */
> > > -static int is_branch_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode,
> > > -                      bool is_jmp32)
> > > +static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
> > > +                      u8 opcode, bool is_jmp32)
> > >  {
> > > +   struct tnum reg2_tnum = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
> > > +   u64 val;
> > > +
> > > +   if (!tnum_is_const(reg2_tnum))
> > > +           return -1;
> > > +   val = reg2_tnum.value;
> > > +
> > >     if (__is_pointer_value(false, reg1)) {
> > >             if (!reg_not_null(reg1))
> > >                     return -1;
> > > @@ -14361,8 +14377,8 @@ static int is_branch_taken(struct bpf_reg_state *reg1, u64 val, u8 opcode,
> > >     }
> > >
> > >     if (is_jmp32)
> > > -           return is_branch32_taken(reg1, val, opcode);
> > > -   return is_branch64_taken(reg1, val, opcode);
> > > +           return is_branch32_taken(reg1, reg2, opcode);
> > > +   return is_branch64_taken(reg1, reg2, opcode);
> > >  }
> > >
> > >  static int flip_opcode(u32 opcode)
> > > @@ -14833,6 +14849,7 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
> > >     struct bpf_reg_state *regs = this_branch->frame[this_branch->curframe]->regs;
> > >     struct bpf_reg_state *dst_reg, *other_branch_regs, *src_reg = NULL;
> > >     struct bpf_reg_state *eq_branch_regs;
> > > +   struct bpf_reg_state fake_reg;
>
> Nitpick:
> bpf_reg_state has a lot of fields, e.g. 'parent' pointer. While it looks like
> the use within this patch-set is safe, I suggest to change the declaration to
> include '= {}' initializer. Just to err on a safe side for future modifications.

yes, good point. One other place where we use "fake_reg" doesn
zero-initialize with = {}, will fix.

>
> > >     u8 opcode = BPF_OP(insn->code);
> > >     bool is_jmp32;
> > >     int pred = -1;
> > > @@ -14873,36 +14890,27 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
> > >                     verbose(env, "BPF_JMP/JMP32 uses reserved fields\n");
> > >                     return -EINVAL;
> > >             }
> > > +           src_reg = &fake_reg;
> > > +           src_reg->type = SCALAR_VALUE;
> > > +           __mark_reg_known(src_reg, insn->imm);
> > >     }
> > >
> > >     is_jmp32 = BPF_CLASS(insn->code) == BPF_JMP32;
> > >
> > >     if (BPF_SRC(insn->code) == BPF_K) {
> > > -           pred = is_branch_taken(dst_reg, insn->imm, opcode, is_jmp32);
> > > +           pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
> > >     } else if (src_reg->type == SCALAR_VALUE &&
> > >                is_jmp32 && tnum_is_const(tnum_subreg(src_reg->var_off))) {
> > > -           pred = is_branch_taken(dst_reg,
> > > -                                  tnum_subreg(src_reg->var_off).value,
> > > -                                  opcode,
> > > -                                  is_jmp32);
> > > +           pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
> > >     } else if (src_reg->type == SCALAR_VALUE &&
> > >                !is_jmp32 && tnum_is_const(src_reg->var_off)) {
> > > -           pred = is_branch_taken(dst_reg,
> > > -                                  src_reg->var_off.value,
> > > -                                  opcode,
> > > -                                  is_jmp32);
> > > +           pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32);
> > >     } else if (dst_reg->type == SCALAR_VALUE &&
> > >                is_jmp32 && tnum_is_const(tnum_subreg(dst_reg->var_off))) {
> > > -           pred = is_branch_taken(src_reg,
> > > -                                  tnum_subreg(dst_reg->var_off).value,
> > > -                                  flip_opcode(opcode),
> > > -                                  is_jmp32);
> > > +           pred = is_branch_taken(src_reg, dst_reg, flip_opcode(opcode), is_jmp32);
> > >     } else if (dst_reg->type == SCALAR_VALUE &&
> > >                !is_jmp32 && tnum_is_const(dst_reg->var_off)) {
> > > -           pred = is_branch_taken(src_reg,
> > > -                                  dst_reg->var_off.value,
> > > -                                  flip_opcode(opcode),
> > > -                                  is_jmp32);
> > > +           pred = is_branch_taken(src_reg, dst_reg, flip_opcode(opcode), is_jmp32);
> > >     } else if (reg_is_pkt_pointer_any(dst_reg) &&
> > >                reg_is_pkt_pointer_any(src_reg) &&
> > >                !is_jmp32) {
>

Re: [PATCH v5 bpf-next 17/23] bpf: generalize reg_set_min_max() to handle two sets of two registers

[Andrii Nakryiko]

On Tue, Oct 31, 2023 at 9:24 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Mon, Oct 30, 2023 at 11:03 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Mon, Oct 30, 2023 at 7:02 PM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Fri, Oct 27, 2023 at 11:13:40AM -0700, Andrii Nakryiko wrote:
> > > >  static void reg_set_min_max(struct bpf_reg_state *true_reg1,
> > > > +                         struct bpf_reg_state *true_reg2,
> > > >                           struct bpf_reg_state *false_reg1,
> > > > -                         u64 val, u32 val32,
> > > > +                         struct bpf_reg_state *false_reg2,
> > > >                           u8 opcode, bool is_jmp32)
> > > >  {
> > > > -     struct tnum false_32off = tnum_subreg(false_reg1->var_off);
> > > > -     struct tnum false_64off = false_reg1->var_off;
> > > > -     struct tnum true_32off = tnum_subreg(true_reg1->var_off);
> > > > -     struct tnum true_64off = true_reg1->var_off;
> > > > -     s64 sval = (s64)val;
> > > > -     s32 sval32 = (s32)val32;
> > > > -
> > > > -     /* If the dst_reg is a pointer, we can't learn anything about its
> > > > -      * variable offset from the compare (unless src_reg were a pointer into
> > > > -      * the same object, but we don't bother with that.
> > > > -      * Since false_reg1 and true_reg1 have the same type by construction, we
> > > > -      * only need to check one of them for pointerness.
> > > > +     struct tnum false_32off, false_64off;
> > > > +     struct tnum true_32off, true_64off;
> > > > +     u64 val;
> > > > +     u32 val32;
> > > > +     s64 sval;
> > > > +     s32 sval32;
> > > > +
> > > > +     /* If either register is a pointer, we can't learn anything about its
> > > > +      * variable offset from the compare (unless they were a pointer into
> > > > +      * the same object, but we don't bother with that).
> > > >        */
> > > > -     if (__is_pointer_value(false, false_reg1))
> > >
> > > The removal of the above check, but not the comment was surprising and concerning,
> > > so I did a bit of git-archaeology.
> > > It was added in commit f1174f77b50c ("bpf/verifier: rework value tracking")
> > > back in 2017 !
> > > and in that commit reg_set_min_max() was always called with reg == scalar.
> > > It looked like premature check. Then I spotted a comment in that commit:
> > >   * this is only legit if both are scalars (or pointers to the same
> > >   * object, I suppose, but we don't support that right now), because
> > >   * otherwise the different base pointers mean the offsets aren't
> > >   * comparable.
> > > so the intent back then was to generalize reg_set_min_max() to be used with pointers too,
> > > but we never got around to do that and the comment now reads:
> >
> > Yeah, it shouldn't be too hard to "generalize" to pointer vs pointer,
> > if we ensure they point to exactly the same thing (I haven't thought
> > much about how), because beyond that it's still basically SCALAR
> > offsets. But I figured it's out of scope for these changes :)
> >
> > >   * this is only legit if both are scalars (or pointers to the same
> > >   * object, I suppose, see the PTR_MAYBE_NULL related if block below),
> > >   * because otherwise the different base pointers mean the offsets aren't
> > >   * comparable.
> > >
> > > So please remove is_pointer check and remove the comment,
> >
> > So I'm a bit confused. I did remove __is_pointer_value() check, but I
> > still need to guard against having pointers, which is why I have:
> >
> > if (false_reg1->type != SCALAR_VALUE || false_reg2->type != SCALAR_VALUE).
> >     return;
> >
> > I think I need this check, because reg_set_min_max() can be called
> > from check_cond_jmp_op() with pointer regs, and we shouldn't try to
> > adjust them. Or am I missing something? And the comment I have here
> > now:
>
> I don't see a code path where reg_set_min_max() is called
> with pointers. At least not in the current code base.
> Are you saying somewhere in your later patch it happens?
>

Hm.. no, it's all in this patch. Check check_cond_jmp_op(). We at
least allow `(reg_is_pkt_pointer_any(dst_reg) &&
reg_is_pkt_pointer_any(src_reg)` case to get into is_branch_taken(),
which, btw, does handle pointer x pointer, pointer x scalar, and
scalar x scalar cases. Then, we go straight to reg_set_min_max(), both
for BPF_X and BPF_K cases. So reg_set_min_max() has to guard itself
against pointers.


> Then the question is whether to do this is_scalar check inside
> reg_set_min_max() or outside. Both options are probably fine.

Given we have two separate calls to reg_set_min_max(), BPF_X and
BPF_K, it seems cleaner to do it once at the beginning of
reg_set_min_max(). And if in the future we do support pointer
variants, I'd handle them inside reg_set_min_max(), just like
is_branch_taken() handles different situations in one place
transparently to the caller.

>
> >
> > +       /* If either register is a pointer, we can't learn anything about its
> > +        * variable offset from the compare (unless they were a pointer into
> > +        * the same object, but we don't bother with that).
> >          */
> >
> > is trying to explain that we don't really adjust two pointers.
> >
> > > and fixup the comment in check_cond_jmp_op() where reg_set_min_max().
> >
> > I have this locally for now, please let me know if this is fine or you
> > had something else in mind:
> >
> > -/* Adjusts the register min/max values in the case that the dst_reg is the
> > - * variable register that we are working on, and src_reg is a constant or we're
> > - * simply doing a BPF_K check.
> > - * In JEQ/JNE cases we also adjust the var_off values.
> > +/* Adjusts the register min/max values in the case that the dst_reg and
> > + * src_reg are both SCALAR_VALUE registers (or we are simply doing a BPF_K
> > + * check, in which case we havea fake SCALAR_VALUE representing insn->imm).
> > + * Technically we can do similar adjustments for pointers to the same object,
> > + * but we don't support that right now.
>
> Looks fine. I'm trying to say that we had such comment forever and
> it never lead to actually doing the work.
> So I'd just remove the last sentence about pointers ...

Ah, ok, yep, sure.

>
> >   */
> >  static void reg_set_min_max(struct bpf_reg_state *true_reg1,
> >                             struct bpf_reg_state *true_reg2,
> > @@ -14884,13 +14885,6 @@ static int check_cond_jmp_op(struct
> > bpf_verifier_env *env,
> >                 return -EFAULT;
> >         other_branch_regs = other_branch->frame[other_branch->curframe]->regs;
> >
> > -       /* detect if we are comparing against a constant value so we can adjust
> > -        * our min/max values for our dst register.
> > -        * this is only legit if both are scalars (or pointers to the same
> > -        * object, I suppose, see the PTR_MAYBE_NULL related if block below),
> > -        * because otherwise the different base pointers mean the offsets aren't
> > -        * comparable.
> > -        */
>
> ... and removing this comment is good thing too.
> In general the comments should be in front of the function body
> (as you're doing) instead of the callsite.

yep, sounds good

Re: [PATCH v5 bpf-next 17/23] bpf: generalize reg_set_min_max() to handle two sets of two registers

[Andrii Nakryiko]

On Tue, Oct 31, 2023 at 10:50 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Tue, Oct 31, 2023 at 9:24 AM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Mon, Oct 30, 2023 at 11:03 PM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Mon, Oct 30, 2023 at 7:02 PM Alexei Starovoitov
> > > <alexei.starovoitov@gmail.com> wrote:
> > > >
> > > > On Fri, Oct 27, 2023 at 11:13:40AM -0700, Andrii Nakryiko wrote:
> > > > >  static void reg_set_min_max(struct bpf_reg_state *true_reg1,
> > > > > +                         struct bpf_reg_state *true_reg2,
> > > > >                           struct bpf_reg_state *false_reg1,
> > > > > -                         u64 val, u32 val32,
> > > > > +                         struct bpf_reg_state *false_reg2,
> > > > >                           u8 opcode, bool is_jmp32)
> > > > >  {
> > > > > -     struct tnum false_32off = tnum_subreg(false_reg1->var_off);
> > > > > -     struct tnum false_64off = false_reg1->var_off;
> > > > > -     struct tnum true_32off = tnum_subreg(true_reg1->var_off);
> > > > > -     struct tnum true_64off = true_reg1->var_off;
> > > > > -     s64 sval = (s64)val;
> > > > > -     s32 sval32 = (s32)val32;
> > > > > -
> > > > > -     /* If the dst_reg is a pointer, we can't learn anything about its
> > > > > -      * variable offset from the compare (unless src_reg were a pointer into
> > > > > -      * the same object, but we don't bother with that.
> > > > > -      * Since false_reg1 and true_reg1 have the same type by construction, we
> > > > > -      * only need to check one of them for pointerness.
> > > > > +     struct tnum false_32off, false_64off;
> > > > > +     struct tnum true_32off, true_64off;
> > > > > +     u64 val;
> > > > > +     u32 val32;
> > > > > +     s64 sval;
> > > > > +     s32 sval32;
> > > > > +
> > > > > +     /* If either register is a pointer, we can't learn anything about its
> > > > > +      * variable offset from the compare (unless they were a pointer into
> > > > > +      * the same object, but we don't bother with that).
> > > > >        */
> > > > > -     if (__is_pointer_value(false, false_reg1))
> > > >
> > > > The removal of the above check, but not the comment was surprising and concerning,
> > > > so I did a bit of git-archaeology.
> > > > It was added in commit f1174f77b50c ("bpf/verifier: rework value tracking")
> > > > back in 2017 !
> > > > and in that commit reg_set_min_max() was always called with reg == scalar.
> > > > It looked like premature check. Then I spotted a comment in that commit:
> > > >   * this is only legit if both are scalars (or pointers to the same
> > > >   * object, I suppose, but we don't support that right now), because
> > > >   * otherwise the different base pointers mean the offsets aren't
> > > >   * comparable.
> > > > so the intent back then was to generalize reg_set_min_max() to be used with pointers too,
> > > > but we never got around to do that and the comment now reads:
> > >
> > > Yeah, it shouldn't be too hard to "generalize" to pointer vs pointer,
> > > if we ensure they point to exactly the same thing (I haven't thought
> > > much about how), because beyond that it's still basically SCALAR
> > > offsets. But I figured it's out of scope for these changes :)
> > >
> > > >   * this is only legit if both are scalars (or pointers to the same
> > > >   * object, I suppose, see the PTR_MAYBE_NULL related if block below),
> > > >   * because otherwise the different base pointers mean the offsets aren't
> > > >   * comparable.
> > > >
> > > > So please remove is_pointer check and remove the comment,
> > >
> > > So I'm a bit confused. I did remove __is_pointer_value() check, but I
> > > still need to guard against having pointers, which is why I have:
> > >
> > > if (false_reg1->type != SCALAR_VALUE || false_reg2->type != SCALAR_VALUE).
> > >     return;
> > >
> > > I think I need this check, because reg_set_min_max() can be called
> > > from check_cond_jmp_op() with pointer regs, and we shouldn't try to
> > > adjust them. Or am I missing something? And the comment I have here
> > > now:
> >
> > I don't see a code path where reg_set_min_max() is called
> > with pointers. At least not in the current code base.
> > Are you saying somewhere in your later patch it happens?
> >
>
> Hm.. no, it's all in this patch. Check check_cond_jmp_op(). We at
> least allow `(reg_is_pkt_pointer_any(dst_reg) &&
> reg_is_pkt_pointer_any(src_reg)` case to get into is_branch_taken(),
> which, btw, does handle pointer x pointer, pointer x scalar, and
> scalar x scalar cases. Then, we go straight to reg_set_min_max(), both
> for BPF_X and BPF_K cases. So reg_set_min_max() has to guard itself
> against pointers.

Correction, BPF_K branch does check for dst_reg->type == SCALAR_VALUE.
But BPF_X doesn't. I stared at this code for so long that I don't even
notice those checks anymore :(

I'd rather drop this SCALAR check for the BPF_K case and keep
reg_set_min_max() as generic as is_branch_taken(), if that's ok. I
think it's less error-prone and a more consistent approach.

>
>
> > Then the question is whether to do this is_scalar check inside
> > reg_set_min_max() or outside. Both options are probably fine.
>
> Given we have two separate calls to reg_set_min_max(), BPF_X and
> BPF_K, it seems cleaner to do it once at the beginning of
> reg_set_min_max(). And if in the future we do support pointer
> variants, I'd handle them inside reg_set_min_max(), just like
> is_branch_taken() handles different situations in one place
> transparently to the caller.
>
> >
> > >
> > > +       /* If either register is a pointer, we can't learn anything about its
> > > +        * variable offset from the compare (unless they were a pointer into
> > > +        * the same object, but we don't bother with that).
> > >          */
> > >
> > > is trying to explain that we don't really adjust two pointers.
> > >
> > > > and fixup the comment in check_cond_jmp_op() where reg_set_min_max().
> > >
> > > I have this locally for now, please let me know if this is fine or you
> > > had something else in mind:
> > >
> > > -/* Adjusts the register min/max values in the case that the dst_reg is the
> > > - * variable register that we are working on, and src_reg is a constant or we're
> > > - * simply doing a BPF_K check.
> > > - * In JEQ/JNE cases we also adjust the var_off values.
> > > +/* Adjusts the register min/max values in the case that the dst_reg and
> > > + * src_reg are both SCALAR_VALUE registers (or we are simply doing a BPF_K
> > > + * check, in which case we havea fake SCALAR_VALUE representing insn->imm).
> > > + * Technically we can do similar adjustments for pointers to the same object,
> > > + * but we don't support that right now.
> >
> > Looks fine. I'm trying to say that we had such comment forever and
> > it never lead to actually doing the work.
> > So I'd just remove the last sentence about pointers ...
>
> Ah, ok, yep, sure.
>
> >
> > >   */
> > >  static void reg_set_min_max(struct bpf_reg_state *true_reg1,
> > >                             struct bpf_reg_state *true_reg2,
> > > @@ -14884,13 +14885,6 @@ static int check_cond_jmp_op(struct
> > > bpf_verifier_env *env,
> > >                 return -EFAULT;
> > >         other_branch_regs = other_branch->frame[other_branch->curframe]->regs;
> > >
> > > -       /* detect if we are comparing against a constant value so we can adjust
> > > -        * our min/max values for our dst register.
> > > -        * this is only legit if both are scalars (or pointers to the same
> > > -        * object, I suppose, see the PTR_MAYBE_NULL related if block below),
> > > -        * because otherwise the different base pointers mean the offsets aren't
> > > -        * comparable.
> > > -        */
> >
> > ... and removing this comment is good thing too.
> > In general the comments should be in front of the function body
> > (as you're doing) instead of the callsite.
>
> yep, sounds good

Re: [PATCH v5 bpf-next 19/23] bpf: generalize is_scalar_branch_taken() logic

[Andrii Nakryiko]

On Tue, Oct 31, 2023 at 9:35 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Mon, Oct 30, 2023 at 11:12 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Mon, Oct 30, 2023 at 7:12 PM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Fri, Oct 27, 2023 at 11:13:42AM -0700, Andrii Nakryiko wrote:
> > > > Generalize is_branch_taken logic for SCALAR_VALUE register to handle
> > > > cases when both registers are not constants. Previously supported
> > > > <range> vs <scalar> cases are a natural subset of more generic <range>
> > > > vs <range> set of cases.
> > > >
> > > > Generalized logic relies on straightforward segment intersection checks.
> > > >
> > > > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > > > ---
> > > >  kernel/bpf/verifier.c | 104 ++++++++++++++++++++++++++----------------
> > > >  1 file changed, 64 insertions(+), 40 deletions(-)
> > > >
> > > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > > > index 4c974296127b..f18a8247e5e2 100644
> > > > --- a/kernel/bpf/verifier.c
> > > > +++ b/kernel/bpf/verifier.c
> > > > @@ -14189,82 +14189,105 @@ static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_sta
> > > >                                 u8 opcode, bool is_jmp32)
> > > >  {
> > > >       struct tnum t1 = is_jmp32 ? tnum_subreg(reg1->var_off) : reg1->var_off;
> > > > +     struct tnum t2 = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
> > > >       u64 umin1 = is_jmp32 ? (u64)reg1->u32_min_value : reg1->umin_value;
> > > >       u64 umax1 = is_jmp32 ? (u64)reg1->u32_max_value : reg1->umax_value;
> > > >       s64 smin1 = is_jmp32 ? (s64)reg1->s32_min_value : reg1->smin_value;
> > > >       s64 smax1 = is_jmp32 ? (s64)reg1->s32_max_value : reg1->smax_value;
> > > > -     u64 val = is_jmp32 ? (u32)tnum_subreg(reg2->var_off).value : reg2->var_off.value;
> > > > -     s64 sval = is_jmp32 ? (s32)val : (s64)val;
> > > > +     u64 umin2 = is_jmp32 ? (u64)reg2->u32_min_value : reg2->umin_value;
> > > > +     u64 umax2 = is_jmp32 ? (u64)reg2->u32_max_value : reg2->umax_value;
> > > > +     s64 smin2 = is_jmp32 ? (s64)reg2->s32_min_value : reg2->smin_value;
> > > > +     s64 smax2 = is_jmp32 ? (s64)reg2->s32_max_value : reg2->smax_value;
> > > >
> > > >       switch (opcode) {
> > > >       case BPF_JEQ:
> > > > -             if (tnum_is_const(t1))
> > > > -                     return !!tnum_equals_const(t1, val);
> > > > -             else if (val < umin1 || val > umax1)
> > > > +             /* const tnums */
> > > > +             if (tnum_is_const(t1) && tnum_is_const(t2))
> > > > +                     return t1.value == t2.value;
> > > > +             /* const ranges */
> > > > +             if (umin1 == umax1 && umin2 == umax2)
> > > > +                     return umin1 == umin2;
> > >
> > > I don't follow this logic.
> > > umin1 == umax1 means that it's a single constant and
> > > it should have been handled by earlier tnum_is_const check.
> >
> > I think you follow the logic, you just think it's redundant. Yes, it's
> > basically the same as
> >
> >           if (tnum_is_const(t1) && tnum_is_const(t2))
> >                 return t1.value == t2.value;
> >
> > but based on ranges. I didn't feel comfortable to assume that if umin1
> > == umax1 then tnum_is_const(t1) will always be true. At worst we'll
> > perform one redundant check.
> >
> > In short, I don't trust tnum to be as precise as umin/umax and other ranges.
> >
> > >
> > > > +             if (smin1 == smax1 && smin2 == smax2)
> > > > +                     return umin1 == umin2;
> > >
> > > here it's even more confusing. smin == smax -> singel const,
> > > but then compare umin1 with umin2 ?!
> >
> > Eagle eyes! Typo, sorry :( it should be `smin1 == smin2`, of course.
> >
> > What saves us is reg_bounds_sync(), and if we have umin1 == umax1 then
> > we'll have also smin1 == smax1 == umin1 == umax1 (and corresponding
> > relation for second register). But I fixed these typos in both BPF_JEQ
> > and BPF_JNE branches.
>
> Not just 'saves us'. The tnum <-> bounds sync is mandatory.
> I think we have a test where a function returns [-errno, 0]
> and then we do if (ret < 0) check. At this point the reg has
> to be tnum_is_const and zero.
> So if smin1 == smax1 == umin1 == umax1 it should be tnum_is_const.
> Otherwise it's a bug in sync logic.
> I think instead of doing redundant and confusing check may be
> add WARN either here or in sync logic to make sure it's all good ?

Ok, let's add it as part of register state sanity checks we discussed
on another patch. I'll drop the checks and will re-run all the test to
make sure we are not missing anything.

Re: [PATCH v5 bpf-next 17/23] bpf: generalize reg_set_min_max() to handle two sets of two registers

[Alexei Starovoitov]

On Tue, Oct 31, 2023 at 10:56 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
> > >
> > > I don't see a code path where reg_set_min_max() is called
> > > with pointers. At least not in the current code base.
> > > Are you saying somewhere in your later patch it happens?
> > >
> >
> > Hm.. no, it's all in this patch. Check check_cond_jmp_op(). We at
> > least allow `(reg_is_pkt_pointer_any(dst_reg) &&
> > reg_is_pkt_pointer_any(src_reg)` case to get into is_branch_taken(),
> > which, btw, does handle pointer x pointer, pointer x scalar, and
> > scalar x scalar cases. Then, we go straight to reg_set_min_max(), both
> > for BPF_X and BPF_K cases. So reg_set_min_max() has to guard itself
> > against pointers.
>
> Correction, BPF_K branch does check for dst_reg->type == SCALAR_VALUE.
> But BPF_X doesn't. I stared at this code for so long that I don't even
> notice those checks anymore :(
>
> I'd rather drop this SCALAR check for the BPF_K case and keep
> reg_set_min_max() as generic as is_branch_taken(), if that's ok. I
> think it's less error-prone and a more consistent approach.

Ahh. Now I see that the patch is doing reg_set_min_max()
right after BPF_X check.
So before the patch all !scalar checks were done outside
and extra __is_pointer_value() inside was useless (reserved for future).
With this change the !scalar change inside is necessary.
Makes sense now. Commit log could have explained that bit
and avoided this back and forth ;)

And yeah dropping !scalar from BPF_K path makes sense as well.

Re: [PATCH v5 bpf-next 20/23] bpf: enhance BPF_JEQ/BPF_JNE is_branch_taken logic

[Andrii Nakryiko]

On Tue, Oct 31, 2023 at 9:36 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Mon, Oct 30, 2023 at 11:16 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Mon, Oct 30, 2023 at 7:20 PM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Fri, Oct 27, 2023 at 11:13:43AM -0700, Andrii Nakryiko wrote:
> > > > Use 32-bit subranges to prune some 64-bit BPF_JEQ/BPF_JNE conditions
> > > > that otherwise would be "inconclusive" (i.e., is_branch_taken() would
> > > > return -1). This can happen, for example, when registers are initialized
> > > > as 64-bit u64/s64, then compared for inequality as 32-bit subregisters,
> > > > and then followed by 64-bit equality/inequality check. That 32-bit
> > > > inequality can establish some pattern for lower 32 bits of a register
> > > > (e.g., s< 0 condition determines whether the bit #31 is zero or not),
> > > > while overall 64-bit value could be anything (according to a value range
> > > > representation).
> > > >
> > > > This is not a fancy quirky special case, but actually a handling that's
> > > > necessary to prevent correctness issue with BPF verifier's range
> > > > tracking: set_range_min_max() assumes that register ranges are
> > > > non-overlapping, and if that condition is not guaranteed by
> > > > is_branch_taken() we can end up with invalid ranges, where min > max.
> > >
> > > This is_scalar_branch_taken() logic makes sense,
> > > but if set_range_min_max() is delicate, it should have its own sanity
> > > check for ranges.
> > > Shouldn't be difficult to check for that dangerous overlap case.
> >
> > So let me clarify. As far as I'm concerned, is_branch_taken() is such
> > a check for set_reg_min_max, and so duplicating such checks in
> > set_reg_min_max() is just that a duplication of code and logic, and
> > just a chance for more typos and subtle bugs.
> >
> > But the concern about invalid ranges is valid, so I don't know,
> > perhaps we should just do a quick check after adjustment to validate
> > that umin<=umax and so on? E.g., we can do that outside of
> > reg_set_min_max(), to keep reg_set_min_max() non-failing. WDYT?
>
> Sounds like a good option too.
> Just trying to minimize breakage in the future.
> Sanity check before or after should catch it.

Sounds good, I'll have a separate register state sanity check and will
see what minimal amount of places where we should call it.

I'm assuming we are ok with returning -EFAULT and failing validation
whenever we detect violation, right?

Re: [PATCH v5 bpf-next 17/23] bpf: generalize reg_set_min_max() to handle two sets of two registers

[Andrii Nakryiko]

On Tue, Oct 31, 2023 at 11:04 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Tue, Oct 31, 2023 at 10:56 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> > > >
> > > > I don't see a code path where reg_set_min_max() is called
> > > > with pointers. At least not in the current code base.
> > > > Are you saying somewhere in your later patch it happens?
> > > >
> > >
> > > Hm.. no, it's all in this patch. Check check_cond_jmp_op(). We at
> > > least allow `(reg_is_pkt_pointer_any(dst_reg) &&
> > > reg_is_pkt_pointer_any(src_reg)` case to get into is_branch_taken(),
> > > which, btw, does handle pointer x pointer, pointer x scalar, and
> > > scalar x scalar cases. Then, we go straight to reg_set_min_max(), both
> > > for BPF_X and BPF_K cases. So reg_set_min_max() has to guard itself
> > > against pointers.
> >
> > Correction, BPF_K branch does check for dst_reg->type == SCALAR_VALUE.
> > But BPF_X doesn't. I stared at this code for so long that I don't even
> > notice those checks anymore :(
> >
> > I'd rather drop this SCALAR check for the BPF_K case and keep
> > reg_set_min_max() as generic as is_branch_taken(), if that's ok. I
> > think it's less error-prone and a more consistent approach.
>
> Ahh. Now I see that the patch is doing reg_set_min_max()
> right after BPF_X check.
> So before the patch all !scalar checks were done outside
> and extra __is_pointer_value() inside was useless (reserved for future).
> With this change the !scalar change inside is necessary.
> Makes sense now. Commit log could have explained that bit
> and avoided this back and forth ;)

Ok, I'll call this out in the commit message, np.

>
> And yeah dropping !scalar from BPF_K path makes sense as well.

Done.

Re: [PATCH v5 bpf-next 20/23] bpf: enhance BPF_JEQ/BPF_JNE is_branch_taken logic

[Alexei Starovoitov]

On Tue, Oct 31, 2023 at 11:04 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Tue, Oct 31, 2023 at 9:36 AM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Mon, Oct 30, 2023 at 11:16 PM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Mon, Oct 30, 2023 at 7:20 PM Alexei Starovoitov
> > > <alexei.starovoitov@gmail.com> wrote:
> > > >
> > > > On Fri, Oct 27, 2023 at 11:13:43AM -0700, Andrii Nakryiko wrote:
> > > > > Use 32-bit subranges to prune some 64-bit BPF_JEQ/BPF_JNE conditions
> > > > > that otherwise would be "inconclusive" (i.e., is_branch_taken() would
> > > > > return -1). This can happen, for example, when registers are initialized
> > > > > as 64-bit u64/s64, then compared for inequality as 32-bit subregisters,
> > > > > and then followed by 64-bit equality/inequality check. That 32-bit
> > > > > inequality can establish some pattern for lower 32 bits of a register
> > > > > (e.g., s< 0 condition determines whether the bit #31 is zero or not),
> > > > > while overall 64-bit value could be anything (according to a value range
> > > > > representation).
> > > > >
> > > > > This is not a fancy quirky special case, but actually a handling that's
> > > > > necessary to prevent correctness issue with BPF verifier's range
> > > > > tracking: set_range_min_max() assumes that register ranges are
> > > > > non-overlapping, and if that condition is not guaranteed by
> > > > > is_branch_taken() we can end up with invalid ranges, where min > max.
> > > >
> > > > This is_scalar_branch_taken() logic makes sense,
> > > > but if set_range_min_max() is delicate, it should have its own sanity
> > > > check for ranges.
> > > > Shouldn't be difficult to check for that dangerous overlap case.
> > >
> > > So let me clarify. As far as I'm concerned, is_branch_taken() is such
> > > a check for set_reg_min_max, and so duplicating such checks in
> > > set_reg_min_max() is just that a duplication of code and logic, and
> > > just a chance for more typos and subtle bugs.
> > >
> > > But the concern about invalid ranges is valid, so I don't know,
> > > perhaps we should just do a quick check after adjustment to validate
> > > that umin<=umax and so on? E.g., we can do that outside of
> > > reg_set_min_max(), to keep reg_set_min_max() non-failing. WDYT?
> >
> > Sounds like a good option too.
> > Just trying to minimize breakage in the future.
> > Sanity check before or after should catch it.
>
> Sounds good, I'll have a separate register state sanity check and will
> see what minimal amount of places where we should call it.
>
> I'm assuming we are ok with returning -EFAULT and failing validation
> whenever we detect violation, right?

Yep and I'll take back WARN suggestion. Let's not add any WARN to avoid
triggering panic_on_warn.

Re: [PATCH v5 bpf-next 17/23] bpf: generalize reg_set_min_max() to handle two sets of two registers

[Eduard Zingerman]

On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> Change reg_set_min_max() to take FALSE/TRUE sets of two registers each,
> instead of assuming that we are always comparing to a constant. For now
> we still assume that right-hand side registers are constants (and make
> sure that's the case by swapping src/dst regs, if necessary), but
> subsequent patches will remove this limitation.
> 
> Taking two by two registers allows to further unify and simplify
> check_cond_jmp_op() logic. We utilize fake register for BPF_K
> conditional jump case, just like with is_branch_taken() part.
> 
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

> ---
>  kernel/bpf/verifier.c | 112 ++++++++++++++++++------------------------
>  1 file changed, 49 insertions(+), 63 deletions(-)
> 
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index dde04b17c3a3..522566699fbe 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -14387,26 +14387,43 @@ static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg
>   * In JEQ/JNE cases we also adjust the var_off values.
>   */
>  static void reg_set_min_max(struct bpf_reg_state *true_reg1,
> +			    struct bpf_reg_state *true_reg2,
>  			    struct bpf_reg_state *false_reg1,
> -			    u64 val, u32 val32,
> +			    struct bpf_reg_state *false_reg2,
>  			    u8 opcode, bool is_jmp32)
>  {
> -	struct tnum false_32off = tnum_subreg(false_reg1->var_off);
> -	struct tnum false_64off = false_reg1->var_off;
> -	struct tnum true_32off = tnum_subreg(true_reg1->var_off);
> -	struct tnum true_64off = true_reg1->var_off;
> -	s64 sval = (s64)val;
> -	s32 sval32 = (s32)val32;
> -
> -	/* If the dst_reg is a pointer, we can't learn anything about its
> -	 * variable offset from the compare (unless src_reg were a pointer into
> -	 * the same object, but we don't bother with that.
> -	 * Since false_reg1 and true_reg1 have the same type by construction, we
> -	 * only need to check one of them for pointerness.
> +	struct tnum false_32off, false_64off;
> +	struct tnum true_32off, true_64off;
> +	u64 val;
> +	u32 val32;
> +	s64 sval;
> +	s32 sval32;
> +
> +	/* If either register is a pointer, we can't learn anything about its
> +	 * variable offset from the compare (unless they were a pointer into
> +	 * the same object, but we don't bother with that).
>  	 */
> -	if (__is_pointer_value(false, false_reg1))
> +	if (false_reg1->type != SCALAR_VALUE || false_reg2->type != SCALAR_VALUE)
> +		return;
> +
> +	/* we expect right-hand registers (src ones) to be constants, for now */
> +	if (!is_reg_const(false_reg2, is_jmp32)) {
> +		opcode = flip_opcode(opcode);
> +		swap(true_reg1, true_reg2);
> +		swap(false_reg1, false_reg2);
> +	}
> +	if (!is_reg_const(false_reg2, is_jmp32))
>  		return;
>  
> +	false_32off = tnum_subreg(false_reg1->var_off);
> +	false_64off = false_reg1->var_off;
> +	true_32off = tnum_subreg(true_reg1->var_off);
> +	true_64off = true_reg1->var_off;
> +	val = false_reg2->var_off.value;
> +	val32 = (u32)tnum_subreg(false_reg2->var_off).value;
> +	sval = (s64)val;
> +	sval32 = (s32)val32;
> +
>  	switch (opcode) {
>  	/* JEQ/JNE comparison doesn't change the register equivalence.
>  	 *
> @@ -14543,22 +14560,6 @@ static void reg_set_min_max(struct bpf_reg_state *true_reg1,
>  	}
>  }
>  
> -/* Same as above, but for the case that dst_reg holds a constant and src_reg is
> - * the variable reg.
> - */
> -static void reg_set_min_max_inv(struct bpf_reg_state *true_reg,
> -				struct bpf_reg_state *false_reg,
> -				u64 val, u32 val32,
> -				u8 opcode, bool is_jmp32)
> -{
> -	opcode = flip_opcode(opcode);
> -	/* This uses zero as "not present in table"; luckily the zero opcode,
> -	 * BPF_JA, can't get here.
> -	 */
> -	if (opcode)
> -		reg_set_min_max(true_reg, false_reg, val, val32, opcode, is_jmp32);
> -}
> -
>  /* Regs are known to be equal, so intersect their min/max/var_off */
>  static void __reg_combine_min_max(struct bpf_reg_state *src_reg,
>  				  struct bpf_reg_state *dst_reg)
> @@ -14891,45 +14892,30 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
>  	 * comparable.
>  	 */
>  	if (BPF_SRC(insn->code) == BPF_X) {
> -		struct bpf_reg_state *src_reg = &regs[insn->src_reg];
> +		reg_set_min_max(&other_branch_regs[insn->dst_reg],
> +				&other_branch_regs[insn->src_reg],
> +				dst_reg, src_reg, opcode, is_jmp32);
>  
>  		if (dst_reg->type == SCALAR_VALUE &&
> -		    src_reg->type == SCALAR_VALUE) {
> -			if (tnum_is_const(src_reg->var_off) ||
> -			    (is_jmp32 &&
> -			     tnum_is_const(tnum_subreg(src_reg->var_off))))
> -				reg_set_min_max(&other_branch_regs[insn->dst_reg],
> -						dst_reg,
> -						src_reg->var_off.value,
> -						tnum_subreg(src_reg->var_off).value,
> -						opcode, is_jmp32);
> -			else if (tnum_is_const(dst_reg->var_off) ||
> -				 (is_jmp32 &&
> -				  tnum_is_const(tnum_subreg(dst_reg->var_off))))
> -				reg_set_min_max_inv(&other_branch_regs[insn->src_reg],
> -						    src_reg,
> -						    dst_reg->var_off.value,
> -						    tnum_subreg(dst_reg->var_off).value,
> -						    opcode, is_jmp32);
> -			else if (!is_jmp32 &&
> -				 (opcode == BPF_JEQ || opcode == BPF_JNE))
> -				/* Comparing for equality, we can combine knowledge */
> -				reg_combine_min_max(&other_branch_regs[insn->src_reg],
> -						    &other_branch_regs[insn->dst_reg],
> -						    src_reg, dst_reg, opcode);
> -			if (src_reg->id &&
> -			    !WARN_ON_ONCE(src_reg->id != other_branch_regs[insn->src_reg].id)) {
> -				find_equal_scalars(this_branch, src_reg);
> -				find_equal_scalars(other_branch, &other_branch_regs[insn->src_reg]);
> -			}
> -
> +		    src_reg->type == SCALAR_VALUE &&
> +		    !is_jmp32 && (opcode == BPF_JEQ || opcode == BPF_JNE)) {
> +			/* Comparing for equality, we can combine knowledge */
> +			reg_combine_min_max(&other_branch_regs[insn->src_reg],
> +					    &other_branch_regs[insn->dst_reg],
> +					    src_reg, dst_reg, opcode);
>  		}
>  	} else if (dst_reg->type == SCALAR_VALUE) {
> -		reg_set_min_max(&other_branch_regs[insn->dst_reg],
> -					dst_reg, insn->imm, (u32)insn->imm,
> -					opcode, is_jmp32);
> +		reg_set_min_max(&other_branch_regs[insn->dst_reg], src_reg, /* fake one */
> +				dst_reg, src_reg /* same fake one */,
> +				opcode, is_jmp32);
>  	}
>  
> +	if (BPF_SRC(insn->code) == BPF_X &&
> +	    src_reg->type == SCALAR_VALUE && src_reg->id &&
> +	    !WARN_ON_ONCE(src_reg->id != other_branch_regs[insn->src_reg].id)) {
> +		find_equal_scalars(this_branch, src_reg);
> +		find_equal_scalars(other_branch, &other_branch_regs[insn->src_reg]);
> +	}
>  	if (dst_reg->type == SCALAR_VALUE && dst_reg->id &&
>  	    !WARN_ON_ONCE(dst_reg->id != other_branch_regs[insn->dst_reg].id)) {
>  		find_equal_scalars(this_branch, dst_reg);

Re: [PATCH v5 bpf-next 06/23] bpf: add special smin32/smax32 derivation from 64-bit bounds

[Alexei Starovoitov]

On Tue, Oct 31, 2023 at 10:39 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
> > fwiw, an alternative explanation might be arithmetic based.
> > Suppose:
> > . there are numbers a, b, c
> > . 2**31 <= b < 2**32
> > . 0 <= c < 2**31
> > . umin = 2**32 * a + b
> > . umax = 2**32 * (a + 1) + c
> >
> > The number of values in the range represented by [umin; umax] is:
> > . N = umax - umin + 1 = 2**32 + c - b + 1
> > . min(N) = 2**32 + 0 - (2**32-1) + 1 = 2
> > . max(N) = 2**32 + (2**31 - 1) - 2**31 + 1 = 2**32
> > Hence [(s32)b; (s32)c] form a valid range.
> >
> > At-least that's how I convinced myself.
>
> So the logic here follows the (visual) intuition how s64 and u64 (and
> also u32 and s32) correlate. That's how I saw it. TBH, the above
> mathematical way seems scary and not so straightforward to follow, so
> I'm hesitant to add it to comments to not scare anyone away :)

Actually Ed's math carried me across the line.
Could you add it to the commit log at least?

> I did try to visually represent it, but I'm not creative enough ASCII
> artist to pull this off, apparently. I'll just leave it as it is for
> now.

Your comment is also good, keep it as-is,
but it's harder to see that it's correct without the math part.

> > > +      * upper 32 bits. As a random example, s64 range
> > > +      * [0xfffffff0ffffff00; 0xfffffff100000010], forms a valid s32 range
> > > +      * [-16, 16] ([0xffffff00; 0x00000010]) in its 32 bit subregister.
> > > +      */

typo. It's [-256, 16]

Re: [PATCH v5 bpf-next 06/23] bpf: add special smin32/smax32 derivation from 64-bit bounds

[Andrii Nakryiko]

On Tue, Oct 31, 2023 at 11:41 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Tue, Oct 31, 2023 at 10:39 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> > > fwiw, an alternative explanation might be arithmetic based.
> > > Suppose:
> > > . there are numbers a, b, c
> > > . 2**31 <= b < 2**32
> > > . 0 <= c < 2**31
> > > . umin = 2**32 * a + b
> > > . umax = 2**32 * (a + 1) + c
> > >
> > > The number of values in the range represented by [umin; umax] is:
> > > . N = umax - umin + 1 = 2**32 + c - b + 1
> > > . min(N) = 2**32 + 0 - (2**32-1) + 1 = 2
> > > . max(N) = 2**32 + (2**31 - 1) - 2**31 + 1 = 2**32
> > > Hence [(s32)b; (s32)c] form a valid range.
> > >
> > > At-least that's how I convinced myself.
> >
> > So the logic here follows the (visual) intuition how s64 and u64 (and
> > also u32 and s32) correlate. That's how I saw it. TBH, the above
> > mathematical way seems scary and not so straightforward to follow, so
> > I'm hesitant to add it to comments to not scare anyone away :)
>
> Actually Ed's math carried me across the line.
> Could you add it to the commit log at least?

Sure, whatever it takes :) Will add to the commit message.

>
> > I did try to visually represent it, but I'm not creative enough ASCII
> > artist to pull this off, apparently. I'll just leave it as it is for
> > now.
>
> Your comment is also good, keep it as-is,
> but it's harder to see that it's correct without the math part.
>
> > > > +      * upper 32 bits. As a random example, s64 range
> > > > +      * [0xfffffff0ffffff00; 0xfffffff100000010], forms a valid s32 range
> > > > +      * [-16, 16] ([0xffffff00; 0x00000010]) in its 32 bit subregister.
> > > > +      */
>
> typo. It's [-256, 16]

I think I wanted to have 0xfffffff0 (one more f) for [-16, 16], but I
can also leave asymmetrical bounds [-256, 16]

Re: [PATCH v5 bpf-next 07/23] bpf: improve deduction of 64-bit bounds from 32-bit bounds

[Alexei Starovoitov]

On Fri, Oct 27, 2023 at 11:17 AM Andrii Nakryiko <andrii@kernel.org> wrote:
>
> Add a few interesting cases in which we can tighten 64-bit bounds based
> on newly learnt information about 32-bit bounds. E.g., when full u64/s64
> registers are used in BPF program, and then eventually compared as
> u32/s32. The latter comparison doesn't change the value of full
> register, but it does impose new restrictions on possible lower 32 bits
> of such full registers. And we can use that to derive additional full
> register bounds information.
>
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> ---
>  kernel/bpf/verifier.c | 47 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 47 insertions(+)
>
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 38d21d0e46bd..768247e3d667 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -2535,10 +2535,57 @@ static void __reg64_deduce_bounds(struct bpf_reg_state *reg)
>         }
>  }
>
> +static void __reg_deduce_mixed_bounds(struct bpf_reg_state *reg)
> +{
> +       /* Try to tighten 64-bit bounds from 32-bit knowledge, using 32-bit
> +        * values on both sides of 64-bit range in hope to have tigher range.
> +        * E.g., if r1 is [0x1'00000000, 0x3'80000000], and we learn from
> +        * 32-bit signed > 0 operation that s32 bounds are now [1; 0x7fffffff].
> +        * With this, we can substitute 1 as low 32-bits of _low_ 64-bit bound
> +        * (0x100000000 -> 0x100000001) and 0x7fffffff as low 32-bits of
> +        * _high_ 64-bit bound (0x380000000 -> 0x37fffffff) and arrive at a
> +        * better overall bounds for r1 as [0x1'000000001; 0x3'7fffffff].
> +        * We just need to make sure that derived bounds we are intersecting
> +        * with are well-formed ranges in respecitve s64 or u64 domain, just
> +        * like we do with similar kinds of 32-to-64 or 64-to-32 adjustments.
> +        */
> +       __u64 new_umin, new_umax;
> +       __s64 new_smin, new_smax;
> +
> +       /* u32 -> u64 tightening, it's always well-formed */
> +       new_umin = (reg->umin_value & ~0xffffffffULL) | reg->u32_min_value;
> +       new_umax = (reg->umax_value & ~0xffffffffULL) | reg->u32_max_value;
> +       reg->umin_value = max_t(u64, reg->umin_value, new_umin);
> +       reg->umax_value = min_t(u64, reg->umax_value, new_umax);
> +
> +       /* s32 -> u64 tightening, s32 should be a valid u32 range (same sign) */
> +       if ((u32)reg->s32_min_value <= (u32)reg->s32_max_value) {
> +               new_umin = (reg->umin_value & ~0xffffffffULL) | (u32)reg->s32_min_value;
> +               new_umax = (reg->umax_value & ~0xffffffffULL) | (u32)reg->s32_max_value;
> +               reg->umin_value = max_t(u64, reg->umin_value, new_umin);
> +               reg->umax_value = min_t(u64, reg->umax_value, new_umax);
> +       }
> +
> +       /* u32 -> s64 tightening, u32 range embedded into s64 preserves range validity */
> +       new_smin = (reg->smin_value & ~0xffffffffULL) | reg->u32_min_value;
> +       new_smax = (reg->smax_value & ~0xffffffffULL) | reg->u32_max_value;
> +       reg->smin_value = max_t(s64, reg->smin_value, new_smin);
> +       reg->smax_value = min_t(s64, reg->smax_value, new_smax);
> +
> +       /* s32 -> s64 tightening, check that s32 range behaves as u32 range */
> +       if ((u32)reg->s32_min_value <= (u32)reg->s32_max_value) {

There is no typo in this check, right?
To make sure somebody doesn't ask this question again can we
combine the same 'if'-s into one?
In order:
u32->u64
u32->s64
if ((u32)reg->s32_min_value <= (u32)reg->s32_max_value) {
  s32->u64
  s32->s64
}
?
imo will be easier to follow and the same end result?

Re: [PATCH v5 bpf-next 07/23] bpf: improve deduction of 64-bit bounds from 32-bit bounds

[Andrii Nakryiko]

On Tue, Oct 31, 2023 at 1:26 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Fri, Oct 27, 2023 at 11:17 AM Andrii Nakryiko <andrii@kernel.org> wrote:
> >
> > Add a few interesting cases in which we can tighten 64-bit bounds based
> > on newly learnt information about 32-bit bounds. E.g., when full u64/s64
> > registers are used in BPF program, and then eventually compared as
> > u32/s32. The latter comparison doesn't change the value of full
> > register, but it does impose new restrictions on possible lower 32 bits
> > of such full registers. And we can use that to derive additional full
> > register bounds information.
> >
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > ---
> >  kernel/bpf/verifier.c | 47 +++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 47 insertions(+)
> >
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index 38d21d0e46bd..768247e3d667 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -2535,10 +2535,57 @@ static void __reg64_deduce_bounds(struct bpf_reg_state *reg)
> >         }
> >  }
> >
> > +static void __reg_deduce_mixed_bounds(struct bpf_reg_state *reg)
> > +{
> > +       /* Try to tighten 64-bit bounds from 32-bit knowledge, using 32-bit
> > +        * values on both sides of 64-bit range in hope to have tigher range.
> > +        * E.g., if r1 is [0x1'00000000, 0x3'80000000], and we learn from
> > +        * 32-bit signed > 0 operation that s32 bounds are now [1; 0x7fffffff].
> > +        * With this, we can substitute 1 as low 32-bits of _low_ 64-bit bound
> > +        * (0x100000000 -> 0x100000001) and 0x7fffffff as low 32-bits of
> > +        * _high_ 64-bit bound (0x380000000 -> 0x37fffffff) and arrive at a
> > +        * better overall bounds for r1 as [0x1'000000001; 0x3'7fffffff].
> > +        * We just need to make sure that derived bounds we are intersecting
> > +        * with are well-formed ranges in respecitve s64 or u64 domain, just
> > +        * like we do with similar kinds of 32-to-64 or 64-to-32 adjustments.
> > +        */
> > +       __u64 new_umin, new_umax;
> > +       __s64 new_smin, new_smax;
> > +
> > +       /* u32 -> u64 tightening, it's always well-formed */
> > +       new_umin = (reg->umin_value & ~0xffffffffULL) | reg->u32_min_value;
> > +       new_umax = (reg->umax_value & ~0xffffffffULL) | reg->u32_max_value;
> > +       reg->umin_value = max_t(u64, reg->umin_value, new_umin);
> > +       reg->umax_value = min_t(u64, reg->umax_value, new_umax);
> > +
> > +       /* s32 -> u64 tightening, s32 should be a valid u32 range (same sign) */
> > +       if ((u32)reg->s32_min_value <= (u32)reg->s32_max_value) {
> > +               new_umin = (reg->umin_value & ~0xffffffffULL) | (u32)reg->s32_min_value;
> > +               new_umax = (reg->umax_value & ~0xffffffffULL) | (u32)reg->s32_max_value;
> > +               reg->umin_value = max_t(u64, reg->umin_value, new_umin);
> > +               reg->umax_value = min_t(u64, reg->umax_value, new_umax);
> > +       }
> > +
> > +       /* u32 -> s64 tightening, u32 range embedded into s64 preserves range validity */
> > +       new_smin = (reg->smin_value & ~0xffffffffULL) | reg->u32_min_value;
> > +       new_smax = (reg->smax_value & ~0xffffffffULL) | reg->u32_max_value;
> > +       reg->smin_value = max_t(s64, reg->smin_value, new_smin);
> > +       reg->smax_value = min_t(s64, reg->smax_value, new_smax);
> > +
> > +       /* s32 -> s64 tightening, check that s32 range behaves as u32 range */
> > +       if ((u32)reg->s32_min_value <= (u32)reg->s32_max_value) {
>
> There is no typo in this check, right?

I don't think so.

> To make sure somebody doesn't ask this question again can we
> combine the same 'if'-s into one?
> In order:
> u32->u64
> u32->s64
> if ((u32)reg->s32_min_value <= (u32)reg->s32_max_value) {
>   s32->u64
>   s32->s64
> }
> ?
> imo will be easier to follow and the same end result?

yep, absolutely, will regroup

Re: [PATCH v5 bpf-next 19/23] bpf: generalize is_scalar_branch_taken() logic

[Andrii Nakryiko]

On Tue, Oct 31, 2023 at 11:01 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Tue, Oct 31, 2023 at 9:35 AM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Mon, Oct 30, 2023 at 11:12 PM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Mon, Oct 30, 2023 at 7:12 PM Alexei Starovoitov
> > > <alexei.starovoitov@gmail.com> wrote:
> > > >
> > > > On Fri, Oct 27, 2023 at 11:13:42AM -0700, Andrii Nakryiko wrote:
> > > > > Generalize is_branch_taken logic for SCALAR_VALUE register to handle
> > > > > cases when both registers are not constants. Previously supported
> > > > > <range> vs <scalar> cases are a natural subset of more generic <range>
> > > > > vs <range> set of cases.
> > > > >
> > > > > Generalized logic relies on straightforward segment intersection checks.
> > > > >
> > > > > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > > > > ---
> > > > >  kernel/bpf/verifier.c | 104 ++++++++++++++++++++++++++----------------
> > > > >  1 file changed, 64 insertions(+), 40 deletions(-)
> > > > >
> > > > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > > > > index 4c974296127b..f18a8247e5e2 100644
> > > > > --- a/kernel/bpf/verifier.c
> > > > > +++ b/kernel/bpf/verifier.c
> > > > > @@ -14189,82 +14189,105 @@ static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_sta
> > > > >                                 u8 opcode, bool is_jmp32)
> > > > >  {
> > > > >       struct tnum t1 = is_jmp32 ? tnum_subreg(reg1->var_off) : reg1->var_off;
> > > > > +     struct tnum t2 = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
> > > > >       u64 umin1 = is_jmp32 ? (u64)reg1->u32_min_value : reg1->umin_value;
> > > > >       u64 umax1 = is_jmp32 ? (u64)reg1->u32_max_value : reg1->umax_value;
> > > > >       s64 smin1 = is_jmp32 ? (s64)reg1->s32_min_value : reg1->smin_value;
> > > > >       s64 smax1 = is_jmp32 ? (s64)reg1->s32_max_value : reg1->smax_value;
> > > > > -     u64 val = is_jmp32 ? (u32)tnum_subreg(reg2->var_off).value : reg2->var_off.value;
> > > > > -     s64 sval = is_jmp32 ? (s32)val : (s64)val;
> > > > > +     u64 umin2 = is_jmp32 ? (u64)reg2->u32_min_value : reg2->umin_value;
> > > > > +     u64 umax2 = is_jmp32 ? (u64)reg2->u32_max_value : reg2->umax_value;
> > > > > +     s64 smin2 = is_jmp32 ? (s64)reg2->s32_min_value : reg2->smin_value;
> > > > > +     s64 smax2 = is_jmp32 ? (s64)reg2->s32_max_value : reg2->smax_value;
> > > > >
> > > > >       switch (opcode) {
> > > > >       case BPF_JEQ:
> > > > > -             if (tnum_is_const(t1))
> > > > > -                     return !!tnum_equals_const(t1, val);
> > > > > -             else if (val < umin1 || val > umax1)
> > > > > +             /* const tnums */
> > > > > +             if (tnum_is_const(t1) && tnum_is_const(t2))
> > > > > +                     return t1.value == t2.value;
> > > > > +             /* const ranges */
> > > > > +             if (umin1 == umax1 && umin2 == umax2)
> > > > > +                     return umin1 == umin2;
> > > >
> > > > I don't follow this logic.
> > > > umin1 == umax1 means that it's a single constant and
> > > > it should have been handled by earlier tnum_is_const check.
> > >
> > > I think you follow the logic, you just think it's redundant. Yes, it's
> > > basically the same as
> > >
> > >           if (tnum_is_const(t1) && tnum_is_const(t2))
> > >                 return t1.value == t2.value;
> > >
> > > but based on ranges. I didn't feel comfortable to assume that if umin1
> > > == umax1 then tnum_is_const(t1) will always be true. At worst we'll
> > > perform one redundant check.
> > >
> > > In short, I don't trust tnum to be as precise as umin/umax and other ranges.
> > >
> > > >
> > > > > +             if (smin1 == smax1 && smin2 == smax2)
> > > > > +                     return umin1 == umin2;
> > > >
> > > > here it's even more confusing. smin == smax -> singel const,
> > > > but then compare umin1 with umin2 ?!
> > >
> > > Eagle eyes! Typo, sorry :( it should be `smin1 == smin2`, of course.
> > >
> > > What saves us is reg_bounds_sync(), and if we have umin1 == umax1 then
> > > we'll have also smin1 == smax1 == umin1 == umax1 (and corresponding
> > > relation for second register). But I fixed these typos in both BPF_JEQ
> > > and BPF_JNE branches.
> >
> > Not just 'saves us'. The tnum <-> bounds sync is mandatory.
> > I think we have a test where a function returns [-errno, 0]
> > and then we do if (ret < 0) check. At this point the reg has
> > to be tnum_is_const and zero.
> > So if smin1 == smax1 == umin1 == umax1 it should be tnum_is_const.
> > Otherwise it's a bug in sync logic.
> > I think instead of doing redundant and confusing check may be
> > add WARN either here or in sync logic to make sure it's all good ?
>
> Ok, let's add it as part of register state sanity checks we discussed
> on another patch. I'll drop the checks and will re-run all the test to
> make sure we are not missing anything.

So I have this as one more patch for the next revision (pending local
testing). If you hate any part of it, I'd appreciate early feedback :)
I'll wait for Eduard to finish going through the series (probably
tomorrow), and then will post the next version based on all the
feedback I got (and whatever might still come).

Note, in the below, I don't output the actual register state on
violation, which is unfortunate. But to make this happen I need to
refactor print_verifier_state() to allow me to print register state.
I've been wanting to move print_verifier_state() into kernel/bpf/log.c
for a while now, and fix how we print the state of spilled registers
(and maybe few more small things), so I'll do that separately, and
then add register state printing to sanity check error.


Author: Andrii Nakryiko <andrii@kernel.org>
Date:   Tue Oct 31 13:34:33 2023 -0700

    bpf: add register bounds sanity checks

    Add simple sanity checks that validate well-formed ranges (min <= max)
    across u64, s64, u32, and s32 ranges. Also for cases when the value is
    constant (either 64-bit or 32-bit), we validate that ranges and tnums
    are in agreement.

    These bounds checks are performed at the end of BPF_ALU/BPF_ALU64
    operations, on conditional jumps, and for LDX instructions (where subreg
    zero/sign extension is probably the most important to check). This
    covers most of the interesting cases.

    Also, we validate the sanity of the return register when manually
adjusting it
    for some special helpers.

    Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index c85d974ba21f..b29c85089bc9 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2615,6 +2615,46 @@ static void reg_bounds_sync(struct bpf_reg_state *reg)
        __update_reg_bounds(reg);
 }

+static int reg_bounds_sanity_check(struct bpf_verifier_env *env,
struct bpf_reg_state *reg)
+{
+       const char *msg;
+
+       if (reg->umin_value > reg->umax_value ||
+           reg->smin_value > reg->smax_value ||
+           reg->u32_min_value > reg->u32_max_value ||
+           reg->s32_min_value > reg->s32_max_value) {
+                   msg = "range bounds violation";
+                   goto out;
+       }
+
+       if (tnum_is_const(reg->var_off)) {
+               u64 uval = reg->var_off.value;
+               s64 sval = (s64)uval;
+
+               if (reg->umin_value != uval || reg->umax_value != uval ||
+                   reg->smin_value != sval || reg->smax_value != sval) {
+                       msg = "const tnum out of sync with range bounds";
+                       goto out;
+               }
+       }
+
+       if (tnum_subreg_is_const(reg->var_off)) {
+               u32 uval32 = tnum_subreg(reg->var_off).value;
+               s32 sval32 = (s32)uval32;
+
+               if (reg->u32_min_value != uval32 || reg->u32_max_value
!= uval32 ||
+                   reg->s32_min_value != sval32 || reg->s32_max_value
!= sval32) {
+                       msg = "const tnum (subreg) out of sync with
range bounds";
+                       goto out;
+               }
+       }
+
+       return 0;
+out:
+       verbose(env, "%s\n", msg);
+       return -EFAULT;
+}
+
 static bool __reg32_bound_s64(s32 a)
 {
        return a >= 0 && a <= S32_MAX;
@@ -9928,14 +9968,15 @@ static int prepare_func_exit(struct
bpf_verifier_env *env, int *insn_idx)
        return 0;
 }

-static void do_refine_retval_range(struct bpf_reg_state *regs, int ret_type,
-                                  int func_id,
-                                  struct bpf_call_arg_meta *meta)
+static int do_refine_retval_range(struct bpf_verifier_env *env,
+                                 struct bpf_reg_state *regs, int ret_type,
+                                 int func_id,
+                                 struct bpf_call_arg_meta *meta)
 {
        struct bpf_reg_state *ret_reg = &regs[BPF_REG_0];

        if (ret_type != RET_INTEGER)
-               return;
+               return 0;

        switch (func_id) {
        case BPF_FUNC_get_stack:
@@ -9961,6 +10002,8 @@ static void do_refine_retval_range(struct
bpf_reg_state *regs, int ret_type,
                reg_bounds_sync(ret_reg);
                break;
        }
+
+       return reg_bounds_sanity_check(env, ret_reg);
 }

 static int
@@ -10612,7 +10655,9 @@ static int check_helper_call(struct
bpf_verifier_env *env, struct bpf_insn *insn
                regs[BPF_REG_0].ref_obj_id = id;
        }

-       do_refine_retval_range(regs, fn->ret_type, func_id, &meta);
+       err = do_refine_retval_range(env, regs, fn->ret_type, func_id, &meta);
+       if (err)
+               return err;

        err = check_map_func_compatibility(env, meta.map_ptr, func_id);
        if (err)
@@ -14079,13 +14124,12 @@ static int check_alu_op(struct
bpf_verifier_env *env, struct bpf_insn *insn)

                /* check dest operand */
                err = check_reg_arg(env, insn->dst_reg, DST_OP_NO_MARK);
+               err = err ?: adjust_reg_min_max_vals(env, insn);
                if (err)
                        return err;
-
-               return adjust_reg_min_max_vals(env, insn);
        }

-       return 0;
+       return reg_bounds_sanity_check(env, &regs[insn->dst_reg]);
 }

 static void find_good_pkt_pointers(struct bpf_verifier_state *vstate,
@@ -14600,18 +14644,21 @@ static void regs_refine_cond_op(struct
bpf_reg_state *reg1, struct bpf_reg_state
  * Technically we can do similar adjustments for pointers to the same object,
  * but we don't support that right now.
  */
-static void reg_set_min_max(struct bpf_reg_state *true_reg1,
-                           struct bpf_reg_state *true_reg2,
-                           struct bpf_reg_state *false_reg1,
-                           struct bpf_reg_state *false_reg2,
-                           u8 opcode, bool is_jmp32)
+static int reg_set_min_max(struct bpf_verifier_env *env,
+                          struct bpf_reg_state *true_reg1,
+                          struct bpf_reg_state *true_reg2,
+                          struct bpf_reg_state *false_reg1,
+                          struct bpf_reg_state *false_reg2,
+                          u8 opcode, bool is_jmp32)
 {
+       int err;
+
        /* If either register is a pointer, we can't learn anything about its
         * variable offset from the compare (unless they were a pointer into
         * the same object, but we don't bother with that).
         */
        if (false_reg1->type != SCALAR_VALUE || false_reg2->type !=
SCALAR_VALUE)
-               return;
+               return 0;

        /* fallthrough (FALSE) branch */
        regs_refine_cond_op(false_reg1, false_reg2,
rev_opcode(opcode), is_jmp32);
@@ -14622,6 +14669,12 @@ static void reg_set_min_max(struct
bpf_reg_state *true_reg1,
        regs_refine_cond_op(true_reg1, true_reg2, opcode, is_jmp32);
        reg_bounds_sync(true_reg1);
        reg_bounds_sync(true_reg2);
+
+       err = reg_bounds_sanity_check(env, true_reg1);
+       err = err ?: reg_bounds_sanity_check(env, true_reg2);
+       err = err ?: reg_bounds_sanity_check(env, false_reg1);
+       err = err ?: reg_bounds_sanity_check(env, false_reg2);
+       return err;
 }

 static void mark_ptr_or_null_reg(struct bpf_func_state *state,
@@ -14915,15 +14968,20 @@ static int check_cond_jmp_op(struct
bpf_verifier_env *env,
        other_branch_regs = other_branch->frame[other_branch->curframe]->regs;

        if (BPF_SRC(insn->code) == BPF_X) {
-               reg_set_min_max(&other_branch_regs[insn->dst_reg],
-                               &other_branch_regs[insn->src_reg],
-                               dst_reg, src_reg, opcode, is_jmp32);
+               err = reg_set_min_max(env,
+                                     &other_branch_regs[insn->dst_reg],
+                                     &other_branch_regs[insn->src_reg],
+                                     dst_reg, src_reg, opcode, is_jmp32);
        } else /* BPF_SRC(insn->code) == BPF_K */ {
-               reg_set_min_max(&other_branch_regs[insn->dst_reg],
-                               src_reg /* fake one */,
-                               dst_reg, src_reg /* same fake one */,
-                               opcode, is_jmp32);
+               err = reg_set_min_max(env,
+                                     &other_branch_regs[insn->dst_reg],
+                                     src_reg /* fake one */,
+                                     dst_reg, src_reg /* same fake one */,
+                                     opcode, is_jmp32);
        }
+       if (err)
+               return err;
+
        if (BPF_SRC(insn->code) == BPF_X &&
            src_reg->type == SCALAR_VALUE && src_reg->id &&
            !WARN_ON_ONCE(src_reg->id != other_branch_regs[insn->src_reg].id)) {
@@ -17426,10 +17484,8 @@ static int do_check(struct bpf_verifier_env *env)
                                               insn->off, BPF_SIZE(insn->code),
                                               BPF_READ, insn->dst_reg, false,
                                               BPF_MODE(insn->code) ==
BPF_MEMSX);
-                       if (err)
-                               return err;
-
-                       err = save_aux_ptr_type(env, src_reg_type, true);
+                       err = err ?: save_aux_ptr_type(env, src_reg_type, true);
+                       err = reg_bounds_sanity_check(env,
&regs[insn->dst_reg]);
                        if (err)
                                return err;
                } else if (class == BPF_STX) {

Re: [PATCH v5 bpf-next 19/23] bpf: generalize is_scalar_branch_taken() logic

[Andrii Nakryiko]

On Tue, Oct 31, 2023 at 1:53 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Tue, Oct 31, 2023 at 11:01 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Tue, Oct 31, 2023 at 9:35 AM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Mon, Oct 30, 2023 at 11:12 PM Andrii Nakryiko
> > > <andrii.nakryiko@gmail.com> wrote:
> > > >
> > > > On Mon, Oct 30, 2023 at 7:12 PM Alexei Starovoitov
> > > > <alexei.starovoitov@gmail.com> wrote:
> > > > >
> > > > > On Fri, Oct 27, 2023 at 11:13:42AM -0700, Andrii Nakryiko wrote:
> > > > > > Generalize is_branch_taken logic for SCALAR_VALUE register to handle
> > > > > > cases when both registers are not constants. Previously supported
> > > > > > <range> vs <scalar> cases are a natural subset of more generic <range>
> > > > > > vs <range> set of cases.
> > > > > >
> > > > > > Generalized logic relies on straightforward segment intersection checks.
> > > > > >
> > > > > > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > > > > > ---
> > > > > >  kernel/bpf/verifier.c | 104 ++++++++++++++++++++++++++----------------
> > > > > >  1 file changed, 64 insertions(+), 40 deletions(-)
> > > > > >
> > > > > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > > > > > index 4c974296127b..f18a8247e5e2 100644
> > > > > > --- a/kernel/bpf/verifier.c
> > > > > > +++ b/kernel/bpf/verifier.c
> > > > > > @@ -14189,82 +14189,105 @@ static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_sta
> > > > > >                                 u8 opcode, bool is_jmp32)
> > > > > >  {
> > > > > >       struct tnum t1 = is_jmp32 ? tnum_subreg(reg1->var_off) : reg1->var_off;
> > > > > > +     struct tnum t2 = is_jmp32 ? tnum_subreg(reg2->var_off) : reg2->var_off;
> > > > > >       u64 umin1 = is_jmp32 ? (u64)reg1->u32_min_value : reg1->umin_value;
> > > > > >       u64 umax1 = is_jmp32 ? (u64)reg1->u32_max_value : reg1->umax_value;
> > > > > >       s64 smin1 = is_jmp32 ? (s64)reg1->s32_min_value : reg1->smin_value;
> > > > > >       s64 smax1 = is_jmp32 ? (s64)reg1->s32_max_value : reg1->smax_value;
> > > > > > -     u64 val = is_jmp32 ? (u32)tnum_subreg(reg2->var_off).value : reg2->var_off.value;
> > > > > > -     s64 sval = is_jmp32 ? (s32)val : (s64)val;
> > > > > > +     u64 umin2 = is_jmp32 ? (u64)reg2->u32_min_value : reg2->umin_value;
> > > > > > +     u64 umax2 = is_jmp32 ? (u64)reg2->u32_max_value : reg2->umax_value;
> > > > > > +     s64 smin2 = is_jmp32 ? (s64)reg2->s32_min_value : reg2->smin_value;
> > > > > > +     s64 smax2 = is_jmp32 ? (s64)reg2->s32_max_value : reg2->smax_value;
> > > > > >
> > > > > >       switch (opcode) {
> > > > > >       case BPF_JEQ:
> > > > > > -             if (tnum_is_const(t1))
> > > > > > -                     return !!tnum_equals_const(t1, val);
> > > > > > -             else if (val < umin1 || val > umax1)
> > > > > > +             /* const tnums */
> > > > > > +             if (tnum_is_const(t1) && tnum_is_const(t2))
> > > > > > +                     return t1.value == t2.value;
> > > > > > +             /* const ranges */
> > > > > > +             if (umin1 == umax1 && umin2 == umax2)
> > > > > > +                     return umin1 == umin2;
> > > > >
> > > > > I don't follow this logic.
> > > > > umin1 == umax1 means that it's a single constant and
> > > > > it should have been handled by earlier tnum_is_const check.
> > > >
> > > > I think you follow the logic, you just think it's redundant. Yes, it's
> > > > basically the same as
> > > >
> > > >           if (tnum_is_const(t1) && tnum_is_const(t2))
> > > >                 return t1.value == t2.value;
> > > >
> > > > but based on ranges. I didn't feel comfortable to assume that if umin1
> > > > == umax1 then tnum_is_const(t1) will always be true. At worst we'll
> > > > perform one redundant check.
> > > >
> > > > In short, I don't trust tnum to be as precise as umin/umax and other ranges.
> > > >
> > > > >
> > > > > > +             if (smin1 == smax1 && smin2 == smax2)
> > > > > > +                     return umin1 == umin2;
> > > > >
> > > > > here it's even more confusing. smin == smax -> singel const,
> > > > > but then compare umin1 with umin2 ?!
> > > >
> > > > Eagle eyes! Typo, sorry :( it should be `smin1 == smin2`, of course.
> > > >
> > > > What saves us is reg_bounds_sync(), and if we have umin1 == umax1 then
> > > > we'll have also smin1 == smax1 == umin1 == umax1 (and corresponding
> > > > relation for second register). But I fixed these typos in both BPF_JEQ
> > > > and BPF_JNE branches.
> > >
> > > Not just 'saves us'. The tnum <-> bounds sync is mandatory.
> > > I think we have a test where a function returns [-errno, 0]
> > > and then we do if (ret < 0) check. At this point the reg has
> > > to be tnum_is_const and zero.
> > > So if smin1 == smax1 == umin1 == umax1 it should be tnum_is_const.
> > > Otherwise it's a bug in sync logic.
> > > I think instead of doing redundant and confusing check may be
> > > add WARN either here or in sync logic to make sure it's all good ?
> >
> > Ok, let's add it as part of register state sanity checks we discussed
> > on another patch. I'll drop the checks and will re-run all the test to
> > make sure we are not missing anything.
>
> So I have this as one more patch for the next revision (pending local
> testing). If you hate any part of it, I'd appreciate early feedback :)
> I'll wait for Eduard to finish going through the series (probably
> tomorrow), and then will post the next version based on all the
> feedback I got (and whatever might still come).
>
> Note, in the below, I don't output the actual register state on
> violation, which is unfortunate. But to make this happen I need to
> refactor print_verifier_state() to allow me to print register state.
> I've been wanting to move print_verifier_state() into kernel/bpf/log.c
> for a while now, and fix how we print the state of spilled registers
> (and maybe few more small things), so I'll do that separately, and
> then add register state printing to sanity check error.
>
>
> Author: Andrii Nakryiko <andrii@kernel.org>
> Date:   Tue Oct 31 13:34:33 2023 -0700
>
>     bpf: add register bounds sanity checks
>
>     Add simple sanity checks that validate well-formed ranges (min <= max)
>     across u64, s64, u32, and s32 ranges. Also for cases when the value is
>     constant (either 64-bit or 32-bit), we validate that ranges and tnums
>     are in agreement.
>
>     These bounds checks are performed at the end of BPF_ALU/BPF_ALU64
>     operations, on conditional jumps, and for LDX instructions (where subreg
>     zero/sign extension is probably the most important to check). This
>     covers most of the interesting cases.
>
>     Also, we validate the sanity of the return register when manually
> adjusting it
>     for some special helpers.
>
>     Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
>
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index c85d974ba21f..b29c85089bc9 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -2615,6 +2615,46 @@ static void reg_bounds_sync(struct bpf_reg_state *reg)
>         __update_reg_bounds(reg);
>  }
>
> +static int reg_bounds_sanity_check(struct bpf_verifier_env *env,
> struct bpf_reg_state *reg)
> +{
> +       const char *msg;
> +
> +       if (reg->umin_value > reg->umax_value ||
> +           reg->smin_value > reg->smax_value ||
> +           reg->u32_min_value > reg->u32_max_value ||
> +           reg->s32_min_value > reg->s32_max_value) {
> +                   msg = "range bounds violation";
> +                   goto out;
> +       }
> +
> +       if (tnum_is_const(reg->var_off)) {
> +               u64 uval = reg->var_off.value;
> +               s64 sval = (s64)uval;
> +
> +               if (reg->umin_value != uval || reg->umax_value != uval ||
> +                   reg->smin_value != sval || reg->smax_value != sval) {
> +                       msg = "const tnum out of sync with range bounds";
> +                       goto out;
> +               }
> +       }
> +
> +       if (tnum_subreg_is_const(reg->var_off)) {
> +               u32 uval32 = tnum_subreg(reg->var_off).value;
> +               s32 sval32 = (s32)uval32;
> +
> +               if (reg->u32_min_value != uval32 || reg->u32_max_value
> != uval32 ||
> +                   reg->s32_min_value != sval32 || reg->s32_max_value
> != sval32) {
> +                       msg = "const tnum (subreg) out of sync with
> range bounds";
> +                       goto out;
> +               }
> +       }
> +
> +       return 0;
> +out:
> +       verbose(env, "%s\n", msg);
> +       return -EFAULT;
> +}
> +
>  static bool __reg32_bound_s64(s32 a)
>  {
>         return a >= 0 && a <= S32_MAX;
> @@ -9928,14 +9968,15 @@ static int prepare_func_exit(struct
> bpf_verifier_env *env, int *insn_idx)
>         return 0;
>  }
>
> -static void do_refine_retval_range(struct bpf_reg_state *regs, int ret_type,
> -                                  int func_id,
> -                                  struct bpf_call_arg_meta *meta)
> +static int do_refine_retval_range(struct bpf_verifier_env *env,
> +                                 struct bpf_reg_state *regs, int ret_type,
> +                                 int func_id,
> +                                 struct bpf_call_arg_meta *meta)
>  {
>         struct bpf_reg_state *ret_reg = &regs[BPF_REG_0];
>
>         if (ret_type != RET_INTEGER)
> -               return;
> +               return 0;
>
>         switch (func_id) {
>         case BPF_FUNC_get_stack:
> @@ -9961,6 +10002,8 @@ static void do_refine_retval_range(struct
> bpf_reg_state *regs, int ret_type,
>                 reg_bounds_sync(ret_reg);
>                 break;
>         }
> +
> +       return reg_bounds_sanity_check(env, ret_reg);
>  }
>
>  static int
> @@ -10612,7 +10655,9 @@ static int check_helper_call(struct
> bpf_verifier_env *env, struct bpf_insn *insn
>                 regs[BPF_REG_0].ref_obj_id = id;
>         }
>
> -       do_refine_retval_range(regs, fn->ret_type, func_id, &meta);
> +       err = do_refine_retval_range(env, regs, fn->ret_type, func_id, &meta);
> +       if (err)
> +               return err;
>
>         err = check_map_func_compatibility(env, meta.map_ptr, func_id);
>         if (err)
> @@ -14079,13 +14124,12 @@ static int check_alu_op(struct
> bpf_verifier_env *env, struct bpf_insn *insn)
>
>                 /* check dest operand */
>                 err = check_reg_arg(env, insn->dst_reg, DST_OP_NO_MARK);
> +               err = err ?: adjust_reg_min_max_vals(env, insn);
>                 if (err)
>                         return err;
> -
> -               return adjust_reg_min_max_vals(env, insn);
>         }
>
> -       return 0;
> +       return reg_bounds_sanity_check(env, &regs[insn->dst_reg]);
>  }
>
>  static void find_good_pkt_pointers(struct bpf_verifier_state *vstate,
> @@ -14600,18 +14644,21 @@ static void regs_refine_cond_op(struct
> bpf_reg_state *reg1, struct bpf_reg_state
>   * Technically we can do similar adjustments for pointers to the same object,
>   * but we don't support that right now.
>   */
> -static void reg_set_min_max(struct bpf_reg_state *true_reg1,
> -                           struct bpf_reg_state *true_reg2,
> -                           struct bpf_reg_state *false_reg1,
> -                           struct bpf_reg_state *false_reg2,
> -                           u8 opcode, bool is_jmp32)
> +static int reg_set_min_max(struct bpf_verifier_env *env,
> +                          struct bpf_reg_state *true_reg1,
> +                          struct bpf_reg_state *true_reg2,
> +                          struct bpf_reg_state *false_reg1,
> +                          struct bpf_reg_state *false_reg2,
> +                          u8 opcode, bool is_jmp32)
>  {
> +       int err;
> +
>         /* If either register is a pointer, we can't learn anything about its
>          * variable offset from the compare (unless they were a pointer into
>          * the same object, but we don't bother with that).
>          */
>         if (false_reg1->type != SCALAR_VALUE || false_reg2->type !=
> SCALAR_VALUE)
> -               return;
> +               return 0;
>
>         /* fallthrough (FALSE) branch */
>         regs_refine_cond_op(false_reg1, false_reg2,
> rev_opcode(opcode), is_jmp32);
> @@ -14622,6 +14669,12 @@ static void reg_set_min_max(struct
> bpf_reg_state *true_reg1,
>         regs_refine_cond_op(true_reg1, true_reg2, opcode, is_jmp32);
>         reg_bounds_sync(true_reg1);
>         reg_bounds_sync(true_reg2);
> +
> +       err = reg_bounds_sanity_check(env, true_reg1);
> +       err = err ?: reg_bounds_sanity_check(env, true_reg2);
> +       err = err ?: reg_bounds_sanity_check(env, false_reg1);
> +       err = err ?: reg_bounds_sanity_check(env, false_reg2);
> +       return err;
>  }
>
>  static void mark_ptr_or_null_reg(struct bpf_func_state *state,
> @@ -14915,15 +14968,20 @@ static int check_cond_jmp_op(struct
> bpf_verifier_env *env,
>         other_branch_regs = other_branch->frame[other_branch->curframe]->regs;
>
>         if (BPF_SRC(insn->code) == BPF_X) {
> -               reg_set_min_max(&other_branch_regs[insn->dst_reg],
> -                               &other_branch_regs[insn->src_reg],
> -                               dst_reg, src_reg, opcode, is_jmp32);
> +               err = reg_set_min_max(env,
> +                                     &other_branch_regs[insn->dst_reg],
> +                                     &other_branch_regs[insn->src_reg],
> +                                     dst_reg, src_reg, opcode, is_jmp32);
>         } else /* BPF_SRC(insn->code) == BPF_K */ {
> -               reg_set_min_max(&other_branch_regs[insn->dst_reg],
> -                               src_reg /* fake one */,
> -                               dst_reg, src_reg /* same fake one */,
> -                               opcode, is_jmp32);
> +               err = reg_set_min_max(env,
> +                                     &other_branch_regs[insn->dst_reg],
> +                                     src_reg /* fake one */,
> +                                     dst_reg, src_reg /* same fake one */,
> +                                     opcode, is_jmp32);
>         }
> +       if (err)
> +               return err;
> +
>         if (BPF_SRC(insn->code) == BPF_X &&
>             src_reg->type == SCALAR_VALUE && src_reg->id &&
>             !WARN_ON_ONCE(src_reg->id != other_branch_regs[insn->src_reg].id)) {
> @@ -17426,10 +17484,8 @@ static int do_check(struct bpf_verifier_env *env)
>                                                insn->off, BPF_SIZE(insn->code),
>                                                BPF_READ, insn->dst_reg, false,
>                                                BPF_MODE(insn->code) ==
> BPF_MEMSX);
> -                       if (err)
> -                               return err;
> -
> -                       err = save_aux_ptr_type(env, src_reg_type, true);
> +                       err = err ?: save_aux_ptr_type(env, src_reg_type, true);
> +                       err = reg_bounds_sanity_check(env,
> &regs[insn->dst_reg]);

this should obviously be `err = err ?: reg_bounds_sanity_check(...)`
(somehow it gets obvious in the email, not locally)

>                         if (err)
>                                 return err;
>                 } else if (class == BPF_STX) {

Re: [PATCH v5 bpf-next 18/23] bpf: generalize reg_set_min_max() to handle non-const register comparisons

[Eduard Zingerman]

On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> Generalize bounds adjustment logic of reg_set_min_max() to handle not
> just register vs constant case, but in general any register vs any
> register cases. For most of the operations it's trivial extension based
> on range vs range comparison logic, we just need to properly pick
> min/max of a range to compare against min/max of the other range.
> 
> For BPF_JSET we keep the original capabilities, just make sure JSET is
> integrated in the common framework. This is manifested in the
> internal-only BPF_KSET + BPF_X "opcode" to allow for simpler and more
> uniform rev_opcode() handling. See the code for details. This allows to
> reuse the same code exactly both for TRUE and FALSE branches without
> explicitly handling both conditions with custom code.
> 
> Note also that now we don't need a special handling of BPF_JEQ/BPF_JNE
> case none of the registers are constants. This is now just a normal
> generic case handled by reg_set_min_max().
> 
> To make tnum handling cleaner, tnum_with_subreg() helper is added, as
> that's a common operator when dealing with 32-bit subregister bounds.
> This keeps the overall logic much less noisy when it comes to tnums.
> 
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> ---
>  include/linux/tnum.h  |   4 +
>  kernel/bpf/tnum.c     |   7 +-
>  kernel/bpf/verifier.c | 321 +++++++++++++++++++-----------------------
>  3 files changed, 157 insertions(+), 175 deletions(-)
> 
> diff --git a/include/linux/tnum.h b/include/linux/tnum.h
> index 1c3948a1d6ad..3c13240077b8 100644
> --- a/include/linux/tnum.h
> +++ b/include/linux/tnum.h
> @@ -106,6 +106,10 @@ int tnum_sbin(char *str, size_t size, struct tnum a);
>  struct tnum tnum_subreg(struct tnum a);
>  /* Returns the tnum with the lower 32-bit subreg cleared */
>  struct tnum tnum_clear_subreg(struct tnum a);
> +/* Returns the tnum with the lower 32-bit subreg in *reg* set to the lower
> + * 32-bit subreg in *subreg*
> + */
> +struct tnum tnum_with_subreg(struct tnum reg, struct tnum subreg);
>  /* Returns the tnum with the lower 32-bit subreg set to value */
>  struct tnum tnum_const_subreg(struct tnum a, u32 value);
>  /* Returns true if 32-bit subreg @a is a known constant*/
> diff --git a/kernel/bpf/tnum.c b/kernel/bpf/tnum.c
> index 3d7127f439a1..f4c91c9b27d7 100644
> --- a/kernel/bpf/tnum.c
> +++ b/kernel/bpf/tnum.c
> @@ -208,7 +208,12 @@ struct tnum tnum_clear_subreg(struct tnum a)
>  	return tnum_lshift(tnum_rshift(a, 32), 32);
>  }
>  
> +struct tnum tnum_with_subreg(struct tnum reg, struct tnum subreg)
> +{
> +	return tnum_or(tnum_clear_subreg(reg), tnum_subreg(subreg));
> +}
> +
>  struct tnum tnum_const_subreg(struct tnum a, u32 value)
>  {
> -	return tnum_or(tnum_clear_subreg(a), tnum_const(value));
> +	return tnum_with_subreg(a, tnum_const(value));
>  }
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 522566699fbe..4c974296127b 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -14381,217 +14381,201 @@ static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg
>  	return is_scalar_branch_taken(reg1, reg2, opcode, is_jmp32);
>  }
>  
> -/* Adjusts the register min/max values in the case that the dst_reg is the
> - * variable register that we are working on, and src_reg is a constant or we're
> - * simply doing a BPF_K check.
> - * In JEQ/JNE cases we also adjust the var_off values.
> +/* Opcode that corresponds to a *false* branch condition.
> + * E.g., if r1 < r2, then reverse (false) condition is r1 >= r2
>   */
> -static void reg_set_min_max(struct bpf_reg_state *true_reg1,
> -			    struct bpf_reg_state *true_reg2,
> -			    struct bpf_reg_state *false_reg1,
> -			    struct bpf_reg_state *false_reg2,
> -			    u8 opcode, bool is_jmp32)
> +static u8 rev_opcode(u8 opcode)

Note: this duplicates flip_opcode() (modulo BPF_JSET).

>  {
> -	struct tnum false_32off, false_64off;
> -	struct tnum true_32off, true_64off;
> -	u64 val;
> -	u32 val32;
> -	s64 sval;
> -	s32 sval32;
> -
> -	/* If either register is a pointer, we can't learn anything about its
> -	 * variable offset from the compare (unless they were a pointer into
> -	 * the same object, but we don't bother with that).
> +	switch (opcode) {
> +	case BPF_JEQ:		return BPF_JNE;
> +	case BPF_JNE:		return BPF_JEQ;
> +	/* JSET doesn't have it's reverse opcode in BPF, so add
> +	 * BPF_X flag to denote the reverse of that operation
>  	 */
> -	if (false_reg1->type != SCALAR_VALUE || false_reg2->type != SCALAR_VALUE)
> -		return;
> -
> -	/* we expect right-hand registers (src ones) to be constants, for now */
> -	if (!is_reg_const(false_reg2, is_jmp32)) {
> -		opcode = flip_opcode(opcode);
> -		swap(true_reg1, true_reg2);
> -		swap(false_reg1, false_reg2);
> +	case BPF_JSET:		return BPF_JSET | BPF_X;
> +	case BPF_JSET | BPF_X:	return BPF_JSET;
> +	case BPF_JGE:		return BPF_JLT;
> +	case BPF_JGT:		return BPF_JLE;
> +	case BPF_JLE:		return BPF_JGT;
> +	case BPF_JLT:		return BPF_JGE;
> +	case BPF_JSGE:		return BPF_JSLT;
> +	case BPF_JSGT:		return BPF_JSLE;
> +	case BPF_JSLE:		return BPF_JSGT;
> +	case BPF_JSLT:		return BPF_JSGE;
> +	default:		return 0;
>  	}
> -	if (!is_reg_const(false_reg2, is_jmp32))
> -		return;
> +}
>  
> -	false_32off = tnum_subreg(false_reg1->var_off);
> -	false_64off = false_reg1->var_off;
> -	true_32off = tnum_subreg(true_reg1->var_off);
> -	true_64off = true_reg1->var_off;
> -	val = false_reg2->var_off.value;
> -	val32 = (u32)tnum_subreg(false_reg2->var_off).value;
> -	sval = (s64)val;
> -	sval32 = (s32)val32;
> +/* Refine range knowledge for <reg1> <op> <reg>2 conditional operation. */
> +static void regs_refine_cond_op(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2,
> +				u8 opcode, bool is_jmp32)
> +{
> +	struct tnum t;
>  
>  	switch (opcode) {
> -	/* JEQ/JNE comparison doesn't change the register equivalence.
> -	 *
> -	 * r1 = r2;
> -	 * if (r1 == 42) goto label;
> -	 * ...
> -	 * label: // here both r1 and r2 are known to be 42.
> -	 *
> -	 * Hence when marking register as known preserve it's ID.
> -	 */
>  	case BPF_JEQ:
>  		if (is_jmp32) {
> -			__mark_reg32_known(true_reg1, val32);
> -			true_32off = tnum_subreg(true_reg1->var_off);
> +			reg1->u32_min_value = max(reg1->u32_min_value, reg2->u32_min_value);
> +			reg1->u32_max_value = min(reg1->u32_max_value, reg2->u32_max_value);
> +			reg1->s32_min_value = max(reg1->s32_min_value, reg2->s32_min_value);
> +			reg1->s32_max_value = min(reg1->s32_max_value, reg2->s32_max_value);
> +			reg2->u32_min_value = reg1->u32_min_value;
> +			reg2->u32_max_value = reg1->u32_max_value;
> +			reg2->s32_min_value = reg1->s32_min_value;
> +			reg2->s32_max_value = reg1->s32_max_value;
> +
> +			t = tnum_intersect(tnum_subreg(reg1->var_off), tnum_subreg(reg2->var_off));
> +			reg1->var_off = tnum_with_subreg(reg1->var_off, t);
> +			reg2->var_off = tnum_with_subreg(reg2->var_off, t);
>  		} else {
> -			___mark_reg_known(true_reg1, val);
> -			true_64off = true_reg1->var_off;
> +			reg1->umin_value = max(reg1->umin_value, reg2->umin_value);
> +			reg1->umax_value = min(reg1->umax_value, reg2->umax_value);
> +			reg1->smin_value = max(reg1->smin_value, reg2->smin_value);
> +			reg1->smax_value = min(reg1->smax_value, reg2->smax_value);
> +			reg2->umin_value = reg1->umin_value;
> +			reg2->umax_value = reg1->umax_value;
> +			reg2->smin_value = reg1->smin_value;
> +			reg2->smax_value = reg1->smax_value;
> +
> +			reg1->var_off = tnum_intersect(reg1->var_off, reg2->var_off);
> +			reg2->var_off = reg1->var_off;
>  		}
>  		break;
>  	case BPF_JNE:
> +		/* we don't derive any new information for inequality yet */
> +		break;
> +	case BPF_JSET:
> +	case BPF_JSET | BPF_X: { /* BPF_JSET and its reverse, see rev_opcode() */
> +		u64 val;
> +
> +		if (!is_reg_const(reg2, is_jmp32))
> +			swap(reg1, reg2);
> +		if (!is_reg_const(reg2, is_jmp32))
> +			break;
> +
> +		val = reg_const_value(reg2, is_jmp32);
> +		/* BPF_JSET requires single bit to learn something useful */
> +		if (!(opcode & BPF_X) && !is_power_of_2(val))

Could you please extend comment a bit, e.g. as follows:

		/* For BPF_JSET true branch (!(opcode & BPF_X)) a single bit
         * is needed to learn something useful.
         */

For some reason it took me a while to understand this condition :(

> +			break;
> +
>  		if (is_jmp32) {
> -			__mark_reg32_known(false_reg1, val32);
> -			false_32off = tnum_subreg(false_reg1->var_off);
> +			if (opcode & BPF_X)
> +				t = tnum_and(tnum_subreg(reg1->var_off), tnum_const(~val));
> +			else
> +				t = tnum_or(tnum_subreg(reg1->var_off), tnum_const(val));
> +			reg1->var_off = tnum_with_subreg(reg1->var_off, t);
>  		} else {
> -			___mark_reg_known(false_reg1, val);
> -			false_64off = false_reg1->var_off;
> +			if (opcode & BPF_X)
> +				reg1->var_off = tnum_and(reg1->var_off, tnum_const(~val));
> +			else
> +				reg1->var_off = tnum_or(reg1->var_off, tnum_const(val));
>  		}
>  		break;
> -	case BPF_JSET:
> +	}
> +	case BPF_JGE:
>  		if (is_jmp32) {
> -			false_32off = tnum_and(false_32off, tnum_const(~val32));
> -			if (is_power_of_2(val32))
> -				true_32off = tnum_or(true_32off,
> -						     tnum_const(val32));
> +			reg1->u32_min_value = max(reg1->u32_min_value, reg2->u32_min_value);
> +			reg2->u32_max_value = min(reg1->u32_max_value, reg2->u32_max_value);
>  		} else {
> -			false_64off = tnum_and(false_64off, tnum_const(~val));
> -			if (is_power_of_2(val))
> -				true_64off = tnum_or(true_64off,
> -						     tnum_const(val));
> +			reg1->umin_value = max(reg1->umin_value, reg2->umin_value);
> +			reg2->umax_value = min(reg1->umax_value, reg2->umax_value);
>  		}
>  		break;
> -	case BPF_JGE:
>  	case BPF_JGT:
> -	{
>  		if (is_jmp32) {
> -			u32 false_umax = opcode == BPF_JGT ? val32  : val32 - 1;
> -			u32 true_umin = opcode == BPF_JGT ? val32 + 1 : val32;
> -
> -			false_reg1->u32_max_value = min(false_reg1->u32_max_value,
> -						       false_umax);
> -			true_reg1->u32_min_value = max(true_reg1->u32_min_value,
> -						      true_umin);
> +			reg1->u32_min_value = max(reg1->u32_min_value, reg2->u32_min_value + 1);

Question: This branch means that reg1 > reg2, right?
          If so, why not use reg2->u32_MAX_value, e.g.:

			reg1->u32_min_value = max(reg1->u32_min_value, reg2->u32_max_value + 1);

          Do I miss something?

> +			reg2->u32_max_value = min(reg1->u32_max_value - 1, reg2->u32_max_value);
>  		} else {
> -			u64 false_umax = opcode == BPF_JGT ? val    : val - 1;
> -			u64 true_umin = opcode == BPF_JGT ? val + 1 : val;
> -
> -			false_reg1->umax_value = min(false_reg1->umax_value, false_umax);
> -			true_reg1->umin_value = max(true_reg1->umin_value, true_umin);
> +			reg1->umin_value = max(reg1->umin_value, reg2->umin_value + 1);
> +			reg2->umax_value = min(reg1->umax_value - 1, reg2->umax_value);
>  		}
>  		break;
> -	}
>  	case BPF_JSGE:
> +		if (is_jmp32) {
> +			reg1->s32_min_value = max(reg1->s32_min_value, reg2->s32_min_value);
> +			reg2->s32_max_value = min(reg1->s32_max_value, reg2->s32_max_value);
> +		} else {
> +			reg1->smin_value = max(reg1->smin_value, reg2->smin_value);
> +			reg2->smax_value = min(reg1->smax_value, reg2->smax_value);
> +		}
> +		break;
>  	case BPF_JSGT:
> -	{
>  		if (is_jmp32) {
> -			s32 false_smax = opcode == BPF_JSGT ? sval32    : sval32 - 1;
> -			s32 true_smin = opcode == BPF_JSGT ? sval32 + 1 : sval32;
> -
> -			false_reg1->s32_max_value = min(false_reg1->s32_max_value, false_smax);
> -			true_reg1->s32_min_value = max(true_reg1->s32_min_value, true_smin);
> +			reg1->s32_min_value = max(reg1->s32_min_value, reg2->s32_min_value + 1);
> +			reg2->s32_max_value = min(reg1->s32_max_value - 1, reg2->s32_max_value);
>  		} else {
> -			s64 false_smax = opcode == BPF_JSGT ? sval    : sval - 1;
> -			s64 true_smin = opcode == BPF_JSGT ? sval + 1 : sval;
> -
> -			false_reg1->smax_value = min(false_reg1->smax_value, false_smax);
> -			true_reg1->smin_value = max(true_reg1->smin_value, true_smin);
> +			reg1->smin_value = max(reg1->smin_value, reg2->smin_value + 1);
> +			reg2->smax_value = min(reg1->smax_value - 1, reg2->smax_value);
>  		}
>  		break;
> -	}
>  	case BPF_JLE:
> +		if (is_jmp32) {
> +			reg1->u32_max_value = min(reg1->u32_max_value, reg2->u32_max_value);
> +			reg2->u32_min_value = max(reg1->u32_min_value, reg2->u32_min_value);
> +		} else {
> +			reg1->umax_value = min(reg1->umax_value, reg2->umax_value);
> +			reg2->umin_value = max(reg1->umin_value, reg2->umin_value);
> +		}
> +		break;
>  	case BPF_JLT:
> -	{
>  		if (is_jmp32) {
> -			u32 false_umin = opcode == BPF_JLT ? val32  : val32 + 1;
> -			u32 true_umax = opcode == BPF_JLT ? val32 - 1 : val32;
> -
> -			false_reg1->u32_min_value = max(false_reg1->u32_min_value,
> -						       false_umin);
> -			true_reg1->u32_max_value = min(true_reg1->u32_max_value,
> -						      true_umax);
> +			reg1->u32_max_value = min(reg1->u32_max_value, reg2->u32_max_value - 1);
> +			reg2->u32_min_value = max(reg1->u32_min_value + 1, reg2->u32_min_value);
>  		} else {
> -			u64 false_umin = opcode == BPF_JLT ? val    : val + 1;
> -			u64 true_umax = opcode == BPF_JLT ? val - 1 : val;
> -
> -			false_reg1->umin_value = max(false_reg1->umin_value, false_umin);
> -			true_reg1->umax_value = min(true_reg1->umax_value, true_umax);
> +			reg1->umax_value = min(reg1->umax_value, reg2->umax_value - 1);
> +			reg2->umin_value = max(reg1->umin_value + 1, reg2->umin_value);
>  		}
>  		break;
> -	}
>  	case BPF_JSLE:
> +		if (is_jmp32) {
> +			reg1->s32_max_value = min(reg1->s32_max_value, reg2->s32_max_value);
> +			reg2->s32_min_value = max(reg1->s32_min_value, reg2->s32_min_value);
> +		} else {
> +			reg1->smax_value = min(reg1->smax_value, reg2->smax_value);
> +			reg2->smin_value = max(reg1->smin_value, reg2->smin_value);
> +		}
> +		break;
>  	case BPF_JSLT:
> -	{
>  		if (is_jmp32) {
> -			s32 false_smin = opcode == BPF_JSLT ? sval32    : sval32 + 1;
> -			s32 true_smax = opcode == BPF_JSLT ? sval32 - 1 : sval32;
> -
> -			false_reg1->s32_min_value = max(false_reg1->s32_min_value, false_smin);
> -			true_reg1->s32_max_value = min(true_reg1->s32_max_value, true_smax);
> +			reg1->s32_max_value = min(reg1->s32_max_value, reg2->s32_max_value - 1);
> +			reg2->s32_min_value = max(reg1->s32_min_value + 1, reg2->s32_min_value);
>  		} else {
> -			s64 false_smin = opcode == BPF_JSLT ? sval    : sval + 1;
> -			s64 true_smax = opcode == BPF_JSLT ? sval - 1 : sval;
> -
> -			false_reg1->smin_value = max(false_reg1->smin_value, false_smin);
> -			true_reg1->smax_value = min(true_reg1->smax_value, true_smax);
> +			reg1->smax_value = min(reg1->smax_value, reg2->smax_value - 1);
> +			reg2->smin_value = max(reg1->smin_value + 1, reg2->smin_value);
>  		}
>  		break;
> -	}
>  	default:
>  		return;
>  	}
> -
> -	if (is_jmp32) {
> -		false_reg1->var_off = tnum_or(tnum_clear_subreg(false_64off),
> -					     tnum_subreg(false_32off));
> -		true_reg1->var_off = tnum_or(tnum_clear_subreg(true_64off),
> -					    tnum_subreg(true_32off));
> -		reg_bounds_sync(false_reg1);
> -		reg_bounds_sync(true_reg1);
> -	} else {
> -		false_reg1->var_off = false_64off;
> -		true_reg1->var_off = true_64off;
> -		reg_bounds_sync(false_reg1);
> -		reg_bounds_sync(true_reg1);
> -	}
> -}
> -
> -/* Regs are known to be equal, so intersect their min/max/var_off */
> -static void __reg_combine_min_max(struct bpf_reg_state *src_reg,
> -				  struct bpf_reg_state *dst_reg)
> -{
> -	src_reg->umin_value = dst_reg->umin_value = max(src_reg->umin_value,
> -							dst_reg->umin_value);
> -	src_reg->umax_value = dst_reg->umax_value = min(src_reg->umax_value,
> -							dst_reg->umax_value);
> -	src_reg->smin_value = dst_reg->smin_value = max(src_reg->smin_value,
> -							dst_reg->smin_value);
> -	src_reg->smax_value = dst_reg->smax_value = min(src_reg->smax_value,
> -							dst_reg->smax_value);
> -	src_reg->var_off = dst_reg->var_off = tnum_intersect(src_reg->var_off,
> -							     dst_reg->var_off);
> -	reg_bounds_sync(src_reg);
> -	reg_bounds_sync(dst_reg);
>  }
>  
> -static void reg_combine_min_max(struct bpf_reg_state *true_src,
> -				struct bpf_reg_state *true_dst,
> -				struct bpf_reg_state *false_src,
> -				struct bpf_reg_state *false_dst,
> -				u8 opcode)
> +/* Adjusts the register min/max values in the case that the dst_reg is the
> + * variable register that we are working on, and src_reg is a constant or we're
> + * simply doing a BPF_K check.
> + * In JEQ/JNE cases we also adjust the var_off values.
> + */
> +static void reg_set_min_max(struct bpf_reg_state *true_reg1,
> +			    struct bpf_reg_state *true_reg2,
> +			    struct bpf_reg_state *false_reg1,
> +			    struct bpf_reg_state *false_reg2,
> +			    u8 opcode, bool is_jmp32)
>  {
> -	switch (opcode) {
> -	case BPF_JEQ:
> -		__reg_combine_min_max(true_src, true_dst);
> -		break;
> -	case BPF_JNE:
> -		__reg_combine_min_max(false_src, false_dst);
> -		break;
> -	}
> +	/* If either register is a pointer, we can't learn anything about its
> +	 * variable offset from the compare (unless they were a pointer into
> +	 * the same object, but we don't bother with that).
> +	 */
> +	if (false_reg1->type != SCALAR_VALUE || false_reg2->type != SCALAR_VALUE)
> +		return;
> +
> +	/* fallthrough (FALSE) branch */
> +	regs_refine_cond_op(false_reg1, false_reg2, rev_opcode(opcode), is_jmp32);
> +	reg_bounds_sync(false_reg1);
> +	reg_bounds_sync(false_reg2);
> +
> +	/* jump (TRUE) branch */
> +	regs_refine_cond_op(true_reg1, true_reg2, opcode, is_jmp32);
> +	reg_bounds_sync(true_reg1);
> +	reg_bounds_sync(true_reg2);
>  }
>  
>  static void mark_ptr_or_null_reg(struct bpf_func_state *state,
> @@ -14895,21 +14879,10 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
>  		reg_set_min_max(&other_branch_regs[insn->dst_reg],
>  				&other_branch_regs[insn->src_reg],
>  				dst_reg, src_reg, opcode, is_jmp32);
> -
> -		if (dst_reg->type == SCALAR_VALUE &&
> -		    src_reg->type == SCALAR_VALUE &&
> -		    !is_jmp32 && (opcode == BPF_JEQ || opcode == BPF_JNE)) {
> -			/* Comparing for equality, we can combine knowledge */
> -			reg_combine_min_max(&other_branch_regs[insn->src_reg],
> -					    &other_branch_regs[insn->dst_reg],
> -					    src_reg, dst_reg, opcode);
> -		}
>  	} else if (dst_reg->type == SCALAR_VALUE) {
> -		reg_set_min_max(&other_branch_regs[insn->dst_reg], src_reg, /* fake one */
> -				dst_reg, src_reg /* same fake one */,
> -				opcode, is_jmp32);
> +		reg_set_min_max(&other_branch_regs[insn->dst_reg], src_reg /* fake*/,
> +				dst_reg, src_reg, opcode, is_jmp32);
>  	}
> -
>  	if (BPF_SRC(insn->code) == BPF_X &&
>  	    src_reg->type == SCALAR_VALUE && src_reg->id &&
>  	    !WARN_ON_ONCE(src_reg->id != other_branch_regs[insn->src_reg].id)) {

Re: [PATCH v5 bpf-next 00/23] BPF register bounds logic and testing improvements

[Paul Chaignon]

On Mon, Oct 30, 2023 at 10:19:01PM -0700, Andrii Nakryiko wrote:
> On Mon, Oct 30, 2023 at 10:55 AM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Fri, Oct 27, 2023 at 11:13:23AM -0700, Andrii Nakryiko wrote:
> > >
> > > Note, this is not unique to <range> vs <range> logic. Just recently ([0])
> > > a related issue was reported for existing verifier logic. This patch set does
> > > fix that issues as well, as pointed out on the mailing list.
> > >
> > >   [0] https://lore.kernel.org/bpf/CAEf4Bzbgf-WQSCz8D4Omh3zFdS4oWS6XELnE7VeoUWgKf3cpig@mail.gmail.com/
> >
> > Quick comment regarding shift out of bound issue.
> > I think this patch set makes Hao Sun's repro not working, but I don't think
> > the range vs range improvement fixes the underlying issue.
> 
> Correct, yes, I think adjust_reg_min_max_vals() might still need some fixing.
> 
> > Currently we do:
> > if (umax_val >= insn_bitness)
> >   mark_reg_unknown
> > else
> >   here were use src_reg->u32_max_value or src_reg->umax_value
> > I suspect the insn_bitness check is buggy and it's still possible to hit UBSAN splat with
> > out of bounds shift. Just need to try harder.
> > if w8 < 0xffffffff goto +2;
> > if r8 != r6 goto +1;
> > w0 >>= w8;
> > won't be enough anymore.
> 
> Agreed, but I felt that fixing adjust_reg_min_max_vals() is out of
> scope for this already large patch set. If someone can take a deeper
> look into reg bounds for arithmetic operations, it would be great.
> 
> On the other hand, one of those academic papers claimed to verify
> soundness of verifier's reg bounds, so I wonder why they missed this?

AFAICS, it should have been able to detect this bug. Equation (3) from
[1, page 10] encodes the soundness condition for conditional jumps and
the implementation definitely covers BPF_JEQ/JNE and the logic in
check_cond_jmp_op. So either there's a bug in the implementation or I'm
missing something about how it works. Let me cc two of the paper's
authors :)

Hari, Srinivas: Hao Sun recently discovered a bug in the range analysis
logic of the verifier, when comparing two unknown scalars with
non-overlapping ranges. See [2] for Eduard Zingerman's explanation. It
seems to have existed for a while. Any idea why Agni didn't uncover it?

1 - https://harishankarv.github.io/assets/files/agni-cav23.pdf
2 - https://lore.kernel.org/bpf/8731196c9a847ff35073a2034662d3306cea805f.camel@gmail.com/

> cc Paul, maybe he can clarify (and also, Paul, please try to run all
> that formal verification machinery against this patch set, thanks!)

I tried it yesterday but am running into what looks like a bug in the
LLVM IR to SMT conversion. Probably not something I can fix myself
quickly so I'll need help from Hari & co.

That said, even without your patchset, I'm running into another issue
where the formal verification takes several times longer (up to weeks
/o\) since v6.4.

Re: [PATCH v5 bpf-next 18/23] bpf: generalize reg_set_min_max() to handle non-const register comparisons

[Andrii Nakryiko]

On Tue, Oct 31, 2023 at 4:25 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> > Generalize bounds adjustment logic of reg_set_min_max() to handle not
> > just register vs constant case, but in general any register vs any
> > register cases. For most of the operations it's trivial extension based
> > on range vs range comparison logic, we just need to properly pick
> > min/max of a range to compare against min/max of the other range.
> >
> > For BPF_JSET we keep the original capabilities, just make sure JSET is
> > integrated in the common framework. This is manifested in the
> > internal-only BPF_KSET + BPF_X "opcode" to allow for simpler and more
> > uniform rev_opcode() handling. See the code for details. This allows to
> > reuse the same code exactly both for TRUE and FALSE branches without
> > explicitly handling both conditions with custom code.
> >
> > Note also that now we don't need a special handling of BPF_JEQ/BPF_JNE
> > case none of the registers are constants. This is now just a normal
> > generic case handled by reg_set_min_max().
> >
> > To make tnum handling cleaner, tnum_with_subreg() helper is added, as
> > that's a common operator when dealing with 32-bit subregister bounds.
> > This keeps the overall logic much less noisy when it comes to tnums.
> >
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > ---
> >  include/linux/tnum.h  |   4 +
> >  kernel/bpf/tnum.c     |   7 +-
> >  kernel/bpf/verifier.c | 321 +++++++++++++++++++-----------------------
> >  3 files changed, 157 insertions(+), 175 deletions(-)
> >
> > diff --git a/include/linux/tnum.h b/include/linux/tnum.h
> > index 1c3948a1d6ad..3c13240077b8 100644
> > --- a/include/linux/tnum.h
> > +++ b/include/linux/tnum.h
> > @@ -106,6 +106,10 @@ int tnum_sbin(char *str, size_t size, struct tnum a);
> >  struct tnum tnum_subreg(struct tnum a);
> >  /* Returns the tnum with the lower 32-bit subreg cleared */
> >  struct tnum tnum_clear_subreg(struct tnum a);
> > +/* Returns the tnum with the lower 32-bit subreg in *reg* set to the lower
> > + * 32-bit subreg in *subreg*
> > + */
> > +struct tnum tnum_with_subreg(struct tnum reg, struct tnum subreg);
> >  /* Returns the tnum with the lower 32-bit subreg set to value */
> >  struct tnum tnum_const_subreg(struct tnum a, u32 value);
> >  /* Returns true if 32-bit subreg @a is a known constant*/
> > diff --git a/kernel/bpf/tnum.c b/kernel/bpf/tnum.c
> > index 3d7127f439a1..f4c91c9b27d7 100644
> > --- a/kernel/bpf/tnum.c
> > +++ b/kernel/bpf/tnum.c
> > @@ -208,7 +208,12 @@ struct tnum tnum_clear_subreg(struct tnum a)
> >       return tnum_lshift(tnum_rshift(a, 32), 32);
> >  }
> >
> > +struct tnum tnum_with_subreg(struct tnum reg, struct tnum subreg)
> > +{
> > +     return tnum_or(tnum_clear_subreg(reg), tnum_subreg(subreg));
> > +}
> > +
> >  struct tnum tnum_const_subreg(struct tnum a, u32 value)
> >  {
> > -     return tnum_or(tnum_clear_subreg(a), tnum_const(value));
> > +     return tnum_with_subreg(a, tnum_const(value));
> >  }
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index 522566699fbe..4c974296127b 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -14381,217 +14381,201 @@ static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg
> >       return is_scalar_branch_taken(reg1, reg2, opcode, is_jmp32);
> >  }
> >
> > -/* Adjusts the register min/max values in the case that the dst_reg is the
> > - * variable register that we are working on, and src_reg is a constant or we're
> > - * simply doing a BPF_K check.
> > - * In JEQ/JNE cases we also adjust the var_off values.
> > +/* Opcode that corresponds to a *false* branch condition.
> > + * E.g., if r1 < r2, then reverse (false) condition is r1 >= r2
> >   */
> > -static void reg_set_min_max(struct bpf_reg_state *true_reg1,
> > -                         struct bpf_reg_state *true_reg2,
> > -                         struct bpf_reg_state *false_reg1,
> > -                         struct bpf_reg_state *false_reg2,
> > -                         u8 opcode, bool is_jmp32)
> > +static u8 rev_opcode(u8 opcode)
>
> Note: this duplicates flip_opcode() (modulo BPF_JSET).

Not at all! flip_opcode() is for swapping argument order, so JEQ stays
JEQ, but <= becomes >=. While rev_opcode() is for the true/false
branch. So JEQ in the true branch becomes JNE in the false branch, <
is true is complemented by >= in the false branch.

>
> >  {
> > -     struct tnum false_32off, false_64off;
> > -     struct tnum true_32off, true_64off;
> > -     u64 val;
> > -     u32 val32;
> > -     s64 sval;
> > -     s32 sval32;
> > -

[...]

> > +             /* we don't derive any new information for inequality yet */
> > +             break;
> > +     case BPF_JSET:
> > +     case BPF_JSET | BPF_X: { /* BPF_JSET and its reverse, see rev_opcode() */
> > +             u64 val;
> > +
> > +             if (!is_reg_const(reg2, is_jmp32))
> > +                     swap(reg1, reg2);
> > +             if (!is_reg_const(reg2, is_jmp32))
> > +                     break;
> > +
> > +             val = reg_const_value(reg2, is_jmp32);
> > +             /* BPF_JSET requires single bit to learn something useful */
> > +             if (!(opcode & BPF_X) && !is_power_of_2(val))
>
> Could you please extend comment a bit, e.g. as follows:
>
>                 /* For BPF_JSET true branch (!(opcode & BPF_X)) a single bit
>          * is needed to learn something useful.
>          */
>
> For some reason it took me a while to understand this condition :(

ok, sure

>
> > +                     break;
> > +

[...]

> > -     case BPF_JGE:
> >       case BPF_JGT:
> > -     {
> >               if (is_jmp32) {
> > -                     u32 false_umax = opcode == BPF_JGT ? val32  : val32 - 1;
> > -                     u32 true_umin = opcode == BPF_JGT ? val32 + 1 : val32;
> > -
> > -                     false_reg1->u32_max_value = min(false_reg1->u32_max_value,
> > -                                                    false_umax);
> > -                     true_reg1->u32_min_value = max(true_reg1->u32_min_value,
> > -                                                   true_umin);
> > +                     reg1->u32_min_value = max(reg1->u32_min_value, reg2->u32_min_value + 1);
>
> Question: This branch means that reg1 > reg2, right?
>           If so, why not use reg2->u32_MAX_value, e.g.:
>
>                         reg1->u32_min_value = max(reg1->u32_min_value, reg2->u32_max_value + 1);
>
>           Do I miss something?

Let's say reg1 can be anything in [10, 20], while reg2 is in [15, 30].
if reg1 > reg2, then we can only guarantee that reg1 can be [16, 20],
because worst case reg2 = 15, not 30, right?

>
> > +                     reg2->u32_max_value = min(reg1->u32_max_value - 1, reg2->u32_max_value);
> >               } else {
> > -                     u64 false_umax = opcode == BPF_JGT ? val    : val - 1;
> > -                     u64 true_umin = opcode == BPF_JGT ? val + 1 : val;
> > -
> > -                     false_reg1->umax_value = min(false_reg1->umax_value, false_umax);
> > -                     true_reg1->umin_value = max(true_reg1->umin_value, true_umin);
> > +                     reg1->umin_value = max(reg1->umin_value, reg2->umin_value + 1);
> > +                     reg2->umax_value = min(reg1->umax_value - 1, reg2->umax_value);
> >               }
> >               break;

[...]

Re: [PATCH v5 bpf-next 18/23] bpf: generalize reg_set_min_max() to handle non-const register comparisons

[Eduard Zingerman]

On Wed, 2023-11-01 at 09:35 -0700, Andrii Nakryiko wrote:
> On Tue, Oct 31, 2023 at 4:25 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > 
> > On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> > > Generalize bounds adjustment logic of reg_set_min_max() to handle not
> > > just register vs constant case, but in general any register vs any
> > > register cases. For most of the operations it's trivial extension based
> > > on range vs range comparison logic, we just need to properly pick
> > > min/max of a range to compare against min/max of the other range.
> > > 
> > > For BPF_JSET we keep the original capabilities, just make sure JSET is
> > > integrated in the common framework. This is manifested in the
> > > internal-only BPF_KSET + BPF_X "opcode" to allow for simpler and more
> > > uniform rev_opcode() handling. See the code for details. This allows to
> > > reuse the same code exactly both for TRUE and FALSE branches without
> > > explicitly handling both conditions with custom code.
> > > 
> > > Note also that now we don't need a special handling of BPF_JEQ/BPF_JNE
> > > case none of the registers are constants. This is now just a normal
> > > generic case handled by reg_set_min_max().
> > > 
> > > To make tnum handling cleaner, tnum_with_subreg() helper is added, as
> > > that's a common operator when dealing with 32-bit subregister bounds.
> > > This keeps the overall logic much less noisy when it comes to tnums.
> > > 
> > > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > > ---
> > >  include/linux/tnum.h  |   4 +
> > >  kernel/bpf/tnum.c     |   7 +-
> > >  kernel/bpf/verifier.c | 321 +++++++++++++++++++-----------------------
> > >  3 files changed, 157 insertions(+), 175 deletions(-)
> > > 
> > > diff --git a/include/linux/tnum.h b/include/linux/tnum.h
> > > index 1c3948a1d6ad..3c13240077b8 100644
> > > --- a/include/linux/tnum.h
> > > +++ b/include/linux/tnum.h
> > > @@ -106,6 +106,10 @@ int tnum_sbin(char *str, size_t size, struct tnum a);
> > >  struct tnum tnum_subreg(struct tnum a);
> > >  /* Returns the tnum with the lower 32-bit subreg cleared */
> > >  struct tnum tnum_clear_subreg(struct tnum a);
> > > +/* Returns the tnum with the lower 32-bit subreg in *reg* set to the lower
> > > + * 32-bit subreg in *subreg*
> > > + */
> > > +struct tnum tnum_with_subreg(struct tnum reg, struct tnum subreg);
> > >  /* Returns the tnum with the lower 32-bit subreg set to value */
> > >  struct tnum tnum_const_subreg(struct tnum a, u32 value);
> > >  /* Returns true if 32-bit subreg @a is a known constant*/
> > > diff --git a/kernel/bpf/tnum.c b/kernel/bpf/tnum.c
> > > index 3d7127f439a1..f4c91c9b27d7 100644
> > > --- a/kernel/bpf/tnum.c
> > > +++ b/kernel/bpf/tnum.c
> > > @@ -208,7 +208,12 @@ struct tnum tnum_clear_subreg(struct tnum a)
> > >       return tnum_lshift(tnum_rshift(a, 32), 32);
> > >  }
> > > 
> > > +struct tnum tnum_with_subreg(struct tnum reg, struct tnum subreg)
> > > +{
> > > +     return tnum_or(tnum_clear_subreg(reg), tnum_subreg(subreg));
> > > +}
> > > +
> > >  struct tnum tnum_const_subreg(struct tnum a, u32 value)
> > >  {
> > > -     return tnum_or(tnum_clear_subreg(a), tnum_const(value));
> > > +     return tnum_with_subreg(a, tnum_const(value));
> > >  }
> > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > > index 522566699fbe..4c974296127b 100644
> > > --- a/kernel/bpf/verifier.c
> > > +++ b/kernel/bpf/verifier.c
> > > @@ -14381,217 +14381,201 @@ static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg
> > >       return is_scalar_branch_taken(reg1, reg2, opcode, is_jmp32);
> > >  }
> > > 
> > > -/* Adjusts the register min/max values in the case that the dst_reg is the
> > > - * variable register that we are working on, and src_reg is a constant or we're
> > > - * simply doing a BPF_K check.
> > > - * In JEQ/JNE cases we also adjust the var_off values.
> > > +/* Opcode that corresponds to a *false* branch condition.
> > > + * E.g., if r1 < r2, then reverse (false) condition is r1 >= r2
> > >   */
> > > -static void reg_set_min_max(struct bpf_reg_state *true_reg1,
> > > -                         struct bpf_reg_state *true_reg2,
> > > -                         struct bpf_reg_state *false_reg1,
> > > -                         struct bpf_reg_state *false_reg2,
> > > -                         u8 opcode, bool is_jmp32)
> > > +static u8 rev_opcode(u8 opcode)
> > 
> > Note: this duplicates flip_opcode() (modulo BPF_JSET).
> 
> Not at all! flip_opcode() is for swapping argument order, so JEQ stays
> JEQ, but <= becomes >=. While rev_opcode() is for the true/false
> branch. So JEQ in the true branch becomes JNE in the false branch, <
> is true is complemented by >= in the false branch.

Right, my bad, sorry.

> 
> > 
> > >  {
> > > -     struct tnum false_32off, false_64off;
> > > -     struct tnum true_32off, true_64off;
> > > -     u64 val;
> > > -     u32 val32;
> > > -     s64 sval;
> > > -     s32 sval32;
> > > -
> 
> [...]
> 
> > > +             /* we don't derive any new information for inequality yet */
> > > +             break;
> > > +     case BPF_JSET:
> > > +     case BPF_JSET | BPF_X: { /* BPF_JSET and its reverse, see rev_opcode() */
> > > +             u64 val;
> > > +
> > > +             if (!is_reg_const(reg2, is_jmp32))
> > > +                     swap(reg1, reg2);
> > > +             if (!is_reg_const(reg2, is_jmp32))
> > > +                     break;
> > > +
> > > +             val = reg_const_value(reg2, is_jmp32);
> > > +             /* BPF_JSET requires single bit to learn something useful */
> > > +             if (!(opcode & BPF_X) && !is_power_of_2(val))
> > 
> > Could you please extend comment a bit, e.g. as follows:
> > 
> >                 /* For BPF_JSET true branch (!(opcode & BPF_X)) a single bit
> >          * is needed to learn something useful.
> >          */
> > 
> > For some reason it took me a while to understand this condition :(
> 
> ok, sure
> 
> > 
> > > +                     break;
> > > +
> 
> [...]
> 
> > > -     case BPF_JGE:
> > >       case BPF_JGT:
> > > -     {
> > >               if (is_jmp32) {
> > > -                     u32 false_umax = opcode == BPF_JGT ? val32  : val32 - 1;
> > > -                     u32 true_umin = opcode == BPF_JGT ? val32 + 1 : val32;
> > > -
> > > -                     false_reg1->u32_max_value = min(false_reg1->u32_max_value,
> > > -                                                    false_umax);
> > > -                     true_reg1->u32_min_value = max(true_reg1->u32_min_value,
> > > -                                                   true_umin);
> > > +                     reg1->u32_min_value = max(reg1->u32_min_value, reg2->u32_min_value + 1);
> > 
> > Question: This branch means that reg1 > reg2, right?
> >           If so, why not use reg2->u32_MAX_value, e.g.:
> > 
> >                         reg1->u32_min_value = max(reg1->u32_min_value, reg2->u32_max_value + 1);
> > 
> >           Do I miss something?
> 
> Let's say reg1 can be anything in [10, 20], while reg2 is in [15, 30].
> if reg1 > reg2, then we can only guarantee that reg1 can be [16, 20],
> because worst case reg2 = 15, not 30, right?

Right, thank you.
I should probably refrain from sending comments after midnight.

> 
> > 
> > > +                     reg2->u32_max_value = min(reg1->u32_max_value - 1, reg2->u32_max_value);
> > >               } else {
> > > -                     u64 false_umax = opcode == BPF_JGT ? val    : val - 1;
> > > -                     u64 true_umin = opcode == BPF_JGT ? val + 1 : val;
> > > -
> > > -                     false_reg1->umax_value = min(false_reg1->umax_value, false_umax);
> > > -                     true_reg1->umin_value = max(true_reg1->umin_value, true_umin);
> > > +                     reg1->umin_value = max(reg1->umin_value, reg2->umin_value + 1);
> > > +                     reg2->umax_value = min(reg1->umax_value - 1, reg2->umax_value);
> > >               }
> > >               break;
> 
> [...]

Re: [PATCH v5 bpf-next 00/23] BPF register bounds logic and testing improvements

[Andrii Nakryiko]

On Wed, Nov 1, 2023 at 5:37 AM Paul Chaignon <paul.chaignon@gmail.com> wrote:
>
> On Mon, Oct 30, 2023 at 10:19:01PM -0700, Andrii Nakryiko wrote:
> > On Mon, Oct 30, 2023 at 10:55 AM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Fri, Oct 27, 2023 at 11:13:23AM -0700, Andrii Nakryiko wrote:
> > > >
> > > > Note, this is not unique to <range> vs <range> logic. Just recently ([0])
> > > > a related issue was reported for existing verifier logic. This patch set does
> > > > fix that issues as well, as pointed out on the mailing list.
> > > >
> > > >   [0] https://lore.kernel.org/bpf/CAEf4Bzbgf-WQSCz8D4Omh3zFdS4oWS6XELnE7VeoUWgKf3cpig@mail.gmail.com/
> > >
> > > Quick comment regarding shift out of bound issue.
> > > I think this patch set makes Hao Sun's repro not working, but I don't think
> > > the range vs range improvement fixes the underlying issue.
> >
> > Correct, yes, I think adjust_reg_min_max_vals() might still need some fixing.
> >
> > > Currently we do:
> > > if (umax_val >= insn_bitness)
> > >   mark_reg_unknown
> > > else
> > >   here were use src_reg->u32_max_value or src_reg->umax_value
> > > I suspect the insn_bitness check is buggy and it's still possible to hit UBSAN splat with
> > > out of bounds shift. Just need to try harder.
> > > if w8 < 0xffffffff goto +2;
> > > if r8 != r6 goto +1;
> > > w0 >>= w8;
> > > won't be enough anymore.
> >
> > Agreed, but I felt that fixing adjust_reg_min_max_vals() is out of
> > scope for this already large patch set. If someone can take a deeper
> > look into reg bounds for arithmetic operations, it would be great.
> >
> > On the other hand, one of those academic papers claimed to verify
> > soundness of verifier's reg bounds, so I wonder why they missed this?
>
> AFAICS, it should have been able to detect this bug. Equation (3) from
> [1, page 10] encodes the soundness condition for conditional jumps and
> the implementation definitely covers BPF_JEQ/JNE and the logic in
> check_cond_jmp_op. So either there's a bug in the implementation or I'm
> missing something about how it works. Let me cc two of the paper's
> authors :)
>
> Hari, Srinivas: Hao Sun recently discovered a bug in the range analysis
> logic of the verifier, when comparing two unknown scalars with
> non-overlapping ranges. See [2] for Eduard Zingerman's explanation. It
> seems to have existed for a while. Any idea why Agni didn't uncover it?
>
> 1 - https://harishankarv.github.io/assets/files/agni-cav23.pdf
> 2 - https://lore.kernel.org/bpf/8731196c9a847ff35073a2034662d3306cea805f.camel@gmail.com/
>
> > cc Paul, maybe he can clarify (and also, Paul, please try to run all
> > that formal verification machinery against this patch set, thanks!)
>
> I tried it yesterday but am running into what looks like a bug in the
> LLVM IR to SMT conversion. Probably not something I can fix myself
> quickly so I'll need help from Hari & co.
>
> That said, even without your patchset, I'm running into another issue
> where the formal verification takes several times longer (up to weeks
> /o\) since v6.4.
>

That's unfortunate. If you figure this out, I'd still be interested in
doing an extra check. Meanwhile I'm working on doing more sanity
checks in the kernel (and inevitably having to debug and fix issues,
still working on this).

Re: [PATCH v5 bpf-next 00/23] BPF register bounds logic and testing improvements

[Harishankar Vishwanathan]

On Wed, Nov 1, 2023 1:13 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
> On Wed, Nov 1, 2023 at 5:37 AM Paul Chaignon <paul.chaignon@gmail.com> wrote:
> >
> > On Mon, Oct 30, 2023 at 10:19:01PM -0700, Andrii Nakryiko wrote:
> > > On Mon, Oct 30, 2023 at 10:55 AM Alexei Starovoitov
> > > <alexei.starovoitov@gmail.com> wrote:
> > > >
> > > > On Fri, Oct 27, 2023 at 11:13:23AM -0700, Andrii Nakryiko wrote:
> > > > >
> > > > > Note, this is not unique to <range> vs <range> logic. Just recently ([0])
> > > > > a related issue was reported for existing verifier logic. This patch set does
> > > > > fix that issues as well, as pointed out on the mailing list.
> > > > >
> > > > >   [0] https://lore.kernel.org/bpf/CAEf4Bzbgf-WQSCz8D4Omh3zFdS4oWS6XELnE7VeoUWgKf3cpig@mail.gmail.com/
> > > >
> > > > Quick comment regarding shift out of bound issue.
> > > > I think this patch set makes Hao Sun's repro not working, but I don't think
> > > > the range vs range improvement fixes the underlying issue.
> > >
> > > Correct, yes, I think adjust_reg_min_max_vals() might still need some fixing.
> > >
> > > > Currently we do:
> > > > if (umax_val >= insn_bitness)
> > > >   mark_reg_unknown
> > > > else
> > > >   here were use src_reg->u32_max_value or src_reg->umax_value
> > > > I suspect the insn_bitness check is buggy and it's still possible to hit UBSAN splat with
> > > > out of bounds shift. Just need to try harder.
> > > > if w8 < 0xffffffff goto +2;
> > > > if r8 != r6 goto +1;
> > > > w0 >>= w8;
> > > > won't be enough anymore.
> > >
> > > Agreed, but I felt that fixing adjust_reg_min_max_vals() is out of
> > > scope for this already large patch set. If someone can take a deeper
> > > look into reg bounds for arithmetic operations, it would be great.
> > >
> > > On the other hand, one of those academic papers claimed to verify
> > > soundness of verifier's reg bounds, so I wonder why they missed this?
> >
> > AFAICS, it should have been able to detect this bug. Equation (3) from
> > [1, page 10] encodes the soundness condition for conditional jumps and
> > the implementation definitely covers BPF_JEQ/JNE and the logic in
> > check_cond_jmp_op. So either there's a bug in the implementation or I'm
> > missing something about how it works. Let me cc two of the paper's
> > authors :)
> >
> > Hari, Srinivas: Hao Sun recently discovered a bug in the range analysis
> > logic of the verifier, when comparing two unknown scalars with
> > non-overlapping ranges. See [2] for Eduard Zingerman's explanation. It
> > seems to have existed for a while. Any idea why Agni didn't uncover it?
> >
> > 1 - https://harishankarv.github.io/assets/files/agni-cav23.pdf
> > 2 - https://lore.kernel.org/bpf/8731196c9a847ff35073a2034662d3306cea805f.camel@gmail.com/
> >
> > > cc Paul, maybe he can clarify (and also, Paul, please try to run all
> > > that formal verification machinery against this patch set, thanks!)
> >

Thanks Paul for bringing this to our notice, and for the valuable clarifications
you provided. The bug discovered by Hao Sun occurs only during verificaiton,
when the verifier follows what is essentially dead code. An execution of the
example eBPF program cannot manifest a mismatch between the verifier's beliefs
about the values in registers and the actual values during execution. As such,
the example eBPF program cannot be used to achieve an actual verifier bypass.

As pointed out by Eduard Zingerman in the mailing list thread, the issue arises
when the verifier follows the false (similarly true) branch of a
jump-if-not-equal (similarly jump-if-equal) instruction, when it is never
possible that the jump condition is false (similarly true). While it is okay for
the verifier to follow dead code generally, it so happens that the logic it uses
to update the registers ranges does not work in this specific case, and ends up
violating one of the invariants it is supposed to maintain (a <= b for a range
[a, b]).

Agni's verification condition [1] is stricter. It follows the false (similarly
true) branch of a jump-if-not-equal (similarly jump-if-equal) instruction *only*
when it is possible that the registers are equal (similarly not equal). In
essence, Agni discards the reported verifier bug as a false positive.

We can easily weaken Agni's verification condition to detect such bugs. We
modified Agni's verification condition [2] to follow both the branches of a
jump-if-not-equal instruction, regardless of whether it is possible that the
registers can be equal. Indeed, the modified verification condition produced the
umin > umax verifier bug from Hao's example. The example produced by Agni, and
an extended discussion can be found at Agni's issue tracker [3].

[1] https://user-images.githubusercontent.com/8588645/280917882-dc97090d-040a-43b0-9bf8-806081992716.png
[2] https://user-images.githubusercontent.com/8588645/280925756-19336087-836f-45e5-87fb-c2453558df06.png
[3] https://github.com/bpfverif/ebpf-range-analysis-verification-cav23/issues/15#issuecomment-1797858245

> > I tried it yesterday but am running into what looks like a bug in the
> > LLVM IR to SMT conversion. Probably not something I can fix myself
> > quickly so I'll need help from Hari & co.
> >
> > That said, even without your patchset, I'm running into another issue
> > where the formal verification takes several times longer (up to weeks
> > /o\) since v6.4.
> >

I'm looking into this next, thanks for the heads up!

> That's unfortunate. If you figure this out, I'd still be interested in
> doing an extra check. Meanwhile I'm working on doing more sanity
> checks in the kernel (and inevitably having to debug and fix issues,
> still working on this).

Re: [PATCH v5 bpf-next 00/23] BPF register bounds logic and testing improvements

[Paul Chaignon]

On Tue, Nov 07, 2023 at 06:37:46AM +0000, Harishankar Vishwanathan wrote:
> On Wed, Nov 1, 2023 1:13 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> > On Wed, Nov 1, 2023 at 5:37 AM Paul Chaignon <paul.chaignon@gmail.com> wrote:
> > >
> > > On Mon, Oct 30, 2023 at 10:19:01PM -0700, Andrii Nakryiko wrote:
> > > > On Mon, Oct 30, 2023 at 10:55 AM Alexei Starovoitov
> > > > <alexei.starovoitov@gmail.com> wrote:
> > > > >
> > > > > On Fri, Oct 27, 2023 at 11:13:23AM -0700, Andrii Nakryiko wrote:
> > > > > >
> > > > > > Note, this is not unique to <range> vs <range> logic. Just recently ([0])
> > > > > > a related issue was reported for existing verifier logic. This patch set does
> > > > > > fix that issues as well, as pointed out on the mailing list.
> > > > > >
> > > > > >   [0] https://lore.kernel.org/bpf/CAEf4Bzbgf-WQSCz8D4Omh3zFdS4oWS6XELnE7VeoUWgKf3cpig@mail.gmail.com/
> > > > >
> > > > > Quick comment regarding shift out of bound issue.
> > > > > I think this patch set makes Hao Sun's repro not working, but I don't think
> > > > > the range vs range improvement fixes the underlying issue.
> > > >
> > > > Correct, yes, I think adjust_reg_min_max_vals() might still need some fixing.
> > > >
> > > > > Currently we do:
> > > > > if (umax_val >= insn_bitness)
> > > > >   mark_reg_unknown
> > > > > else
> > > > >   here were use src_reg->u32_max_value or src_reg->umax_value
> > > > > I suspect the insn_bitness check is buggy and it's still possible to hit UBSAN splat with
> > > > > out of bounds shift. Just need to try harder.
> > > > > if w8 < 0xffffffff goto +2;
> > > > > if r8 != r6 goto +1;
> > > > > w0 >>= w8;
> > > > > won't be enough anymore.
> > > >
> > > > Agreed, but I felt that fixing adjust_reg_min_max_vals() is out of
> > > > scope for this already large patch set. If someone can take a deeper
> > > > look into reg bounds for arithmetic operations, it would be great.
> > > >
> > > > On the other hand, one of those academic papers claimed to verify
> > > > soundness of verifier's reg bounds, so I wonder why they missed this?
> > >
> > > AFAICS, it should have been able to detect this bug. Equation (3) from
> > > [1, page 10] encodes the soundness condition for conditional jumps and
> > > the implementation definitely covers BPF_JEQ/JNE and the logic in
> > > check_cond_jmp_op. So either there's a bug in the implementation or I'm
> > > missing something about how it works. Let me cc two of the paper's
> > > authors :)
> > >
> > > Hari, Srinivas: Hao Sun recently discovered a bug in the range analysis
> > > logic of the verifier, when comparing two unknown scalars with
> > > non-overlapping ranges. See [2] for Eduard Zingerman's explanation. It
> > > seems to have existed for a while. Any idea why Agni didn't uncover it?
> > >
> > > 1 - https://harishankarv.github.io/assets/files/agni-cav23.pdf
> > > 2 - https://lore.kernel.org/bpf/8731196c9a847ff35073a2034662d3306cea805f.camel@gmail.com/
> > >
> > > > cc Paul, maybe he can clarify (and also, Paul, please try to run all
> > > > that formal verification machinery against this patch set, thanks!)
> > >
> 
> Thanks Paul for bringing this to our notice, and for the valuable clarifications
> you provided. The bug discovered by Hao Sun occurs only during verificaiton,
> when the verifier follows what is essentially dead code. An execution of the
> example eBPF program cannot manifest a mismatch between the verifier's beliefs
> about the values in registers and the actual values during execution. As such,
> the example eBPF program cannot be used to achieve an actual verifier bypass.

There's one caveat here I wanted to double check: speculative execution.
I discussed this a bit with Daniel and it seems likely that the
false/"impossible" path could happen at runtime under speculative
execution. Daniel provided [1, slide 69] as an example of a similar
case. In the verifier, such cases are covered by
sanitize_speculative_path [2].

So this speculative execution case would be a good motivation to cover
both paths in Agni. At the same time, it may not be enough to claim
that Agni also verifies all the verifier's protections against
speculative execution. And I'm also a bit worried about the runtime
cost of weakening Agni's verification condition to detect such bugs.

1 - https://popl22.sigplan.org/details/prisc-2022-papers/11/BPF-and-Spectre-Mitigating-transient-execution-attacks
2 - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9183671af6db

> 
> As pointed out by Eduard Zingerman in the mailing list thread, the issue arises
> when the verifier follows the false (similarly true) branch of a
> jump-if-not-equal (similarly jump-if-equal) instruction, when it is never
> possible that the jump condition is false (similarly true). While it is okay for
> the verifier to follow dead code generally, it so happens that the logic it uses
> to update the registers ranges does not work in this specific case, and ends up
> violating one of the invariants it is supposed to maintain (a <= b for a range
> [a, b]).
> 
> Agni's verification condition [1] is stricter. It follows the false (similarly
> true) branch of a jump-if-not-equal (similarly jump-if-equal) instruction *only*
> when it is possible that the registers are equal (similarly not equal). In
> essence, Agni discards the reported verifier bug as a false positive.
> 
> We can easily weaken Agni's verification condition to detect such bugs. We
> modified Agni's verification condition [2] to follow both the branches of a
> jump-if-not-equal instruction, regardless of whether it is possible that the
> registers can be equal. Indeed, the modified verification condition produced the
> umin > umax verifier bug from Hao's example. The example produced by Agni, and
> an extended discussion can be found at Agni's issue tracker [3].

Nice! Thanks for trying this out so quickly!
Did you notice any difference on the time it took to verify the
conditional jumps?

> 
> [1] https://user-images.githubusercontent.com/8588645/280917882-dc97090d-040a-43b0-9bf8-806081992716.png
> [2] https://user-images.githubusercontent.com/8588645/280925756-19336087-836f-45e5-87fb-c2453558df06.png
> [3] https://github.com/bpfverif/ebpf-range-analysis-verification-cav23/issues/15#issuecomment-1797858245
> 
> > > I tried it yesterday but am running into what looks like a bug in the
> > > LLVM IR to SMT conversion. Probably not something I can fix myself
> > > quickly so I'll need help from Hari & co.
> > >
> > > That said, even without your patchset, I'm running into another issue
> > > where the formal verification takes several times longer (up to weeks
> > > /o\) since v6.4.
> > >
> 
> I'm looking into this next, thanks for the heads up!
> 
> > That's unfortunate. If you figure this out, I'd still be interested in
> > doing an extra check. Meanwhile I'm working on doing more sanity
> > checks in the kernel (and inevitably having to debug and fix issues,
> > still working on this).

Re: [PATCH v5 bpf-next 21/23] selftests/bpf: adjust OP_EQ/OP_NE handling to use subranges for branch taken

[Eduard Zingerman]

On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> Similar to kernel-side BPF verifier logic enhancements, use 32-bit
> subrange knowledge for is_branch_taken() logic in reg_bounds selftests.
> 
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> ---
>  .../selftests/bpf/prog_tests/reg_bounds.c     | 19 +++++++++++++++----
>  1 file changed, 15 insertions(+), 4 deletions(-)
> 
> diff --git a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
> index ac7354cfe139..330618cc12e7 100644
> --- a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
> +++ b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
> @@ -750,16 +750,27 @@ static int reg_state_branch_taken_op(enum num_t t, struct reg_state *x, struct r
>  		/* OP_EQ and OP_NE are sign-agnostic */
>  		enum num_t tu = t_unsigned(t);
>  		enum num_t ts = t_signed(t);
> -		int br_u, br_s;
> +		int br_u, br_s, br;
>  
>  		br_u = range_branch_taken_op(tu, x->r[tu], y->r[tu], op);
>  		br_s = range_branch_taken_op(ts, x->r[ts], y->r[ts], op);
>  
>  		if (br_u >= 0 && br_s >= 0 && br_u != br_s)
>  			ASSERT_FALSE(true, "branch taken inconsistency!\n");
> -		if (br_u >= 0)
> -			return br_u;
> -		return br_s;
> +
> +		/* if 64-bit ranges are indecisive, use 32-bit subranges to
> +		 * eliminate always/never taken branches, if possible
> +		 */
> +		if (br_u == -1 && (t == U64 || t == S64)) {
> +			br = range_branch_taken_op(U32, x->r[U32], y->r[U32], op);
> +			if (br != -1)
> +				return br;
> +			br = range_branch_taken_op(S32, x->r[S32], y->r[S32], op);
> +			if (br != -1)
> +				return br;

I'm not sure that these two checks are consistent with kernel side.
In kernel:
- for BPF_JEQ we can derive "won't happen" from u32/s32 ranges;
- for BPF_JNE we can derive "will happen" from u32/s32 ranges.

But here we seem to accept "will happen" for OP_EQ, which does not
seem right. E.g. it is possible to have inconclusive upper 32 bits and
equal lower 32 bits. What am I missing?

Re: [PATCH v5 bpf-next 21/23] selftests/bpf: adjust OP_EQ/OP_NE handling to use subranges for branch taken

[Andrii Nakryiko]

On Wed, Nov 8, 2023 at 10:22 AM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
> > Similar to kernel-side BPF verifier logic enhancements, use 32-bit
> > subrange knowledge for is_branch_taken() logic in reg_bounds selftests.
> >
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > ---
> >  .../selftests/bpf/prog_tests/reg_bounds.c     | 19 +++++++++++++++----
> >  1 file changed, 15 insertions(+), 4 deletions(-)
> >
> > diff --git a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
> > index ac7354cfe139..330618cc12e7 100644
> > --- a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
> > +++ b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
> > @@ -750,16 +750,27 @@ static int reg_state_branch_taken_op(enum num_t t, struct reg_state *x, struct r
> >               /* OP_EQ and OP_NE are sign-agnostic */
> >               enum num_t tu = t_unsigned(t);
> >               enum num_t ts = t_signed(t);
> > -             int br_u, br_s;
> > +             int br_u, br_s, br;
> >
> >               br_u = range_branch_taken_op(tu, x->r[tu], y->r[tu], op);
> >               br_s = range_branch_taken_op(ts, x->r[ts], y->r[ts], op);
> >
> >               if (br_u >= 0 && br_s >= 0 && br_u != br_s)
> >                       ASSERT_FALSE(true, "branch taken inconsistency!\n");
> > -             if (br_u >= 0)
> > -                     return br_u;
> > -             return br_s;
> > +
> > +             /* if 64-bit ranges are indecisive, use 32-bit subranges to
> > +              * eliminate always/never taken branches, if possible
> > +              */
> > +             if (br_u == -1 && (t == U64 || t == S64)) {
> > +                     br = range_branch_taken_op(U32, x->r[U32], y->r[U32], op);
> > +                     if (br != -1)
> > +                             return br;
> > +                     br = range_branch_taken_op(S32, x->r[S32], y->r[S32], op);
> > +                     if (br != -1)
> > +                             return br;
>
> I'm not sure that these two checks are consistent with kernel side.
> In kernel:
> - for BPF_JEQ we can derive "won't happen" from u32/s32 ranges;
> - for BPF_JNE we can derive "will happen" from u32/s32 ranges.
>
> But here we seem to accept "will happen" for OP_EQ, which does not
> seem right. E.g. it is possible to have inconclusive upper 32 bits and
> equal lower 32 bits. What am I missing?

I think you are right, I should take into account OP_EQ vs OP_NE here
and the specific value of br. Will update. Nice catch!

Re: [PATCH v5 bpf-next 10/23] selftests/bpf: BPF register range bounds tester

[Eduard Zingerman]

> On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:

I read through whole program and it seems to be a good specimen of an
idiomatic C program. Aside from two nitpicks below I can't really complain
about the code.

On the other hand, I'm a bit at odds with the core idea.
The algorithm for ranges computation repeats the one used in kernel
(arguably in a bit more elegant way). So, effectively we check if two
implementations of the same algorithm end up with the same answers.

It is a maintenance burden in a way.
What are the benefits of such approach? 
Simpler userspace prototyping for new range tracking features?

[...]

> +__printf(2, 3)
> +static inline void snappendf(struct strbuf *s, const char *fmt, ...)
> +{
> +	va_list args;
> +
> +	va_start(args, fmt);
> +	s->pos += vsnprintf(s->buf + s->pos, s->buf_sz - s->pos, fmt, args);
> +	va_end(args);
> +}

The manpage for vsnprintf says the following about it's return value:

  ... If the output was truncated due to this limit, then the return
  value is the number of characters (excluding the terminating null
  byte) which would have been written to the final string if enough
  space had been available ...

Which is definitely a footgun to say the least. So, I picked strbuf,
DEFINE_STRBUF, snappendf definitions to a separate file and tried the
following [0]:

    $ cat test.c
    ...
    int main(int argc, char *argv)
    {
      DEFINE_STRBUF(buf, 2);
      snappendf(buf, "kinda long string...");
      printf("buf->pos=%d\n", buf->pos);
      snappendf(buf, "will this overflow buf?");
    }

    $ gcc -O0 -g test.c && valgrind -q ./a.out
    buf->pos=20
    ==27408== Jump to the invalid address stated on the next line
    ==27408==    at 0x6C667265766F2073: ???
    ==27408==    by 0x66756220776E: ???
    ==27408==    by 0x401244: snappendf (test.c:24)
    ==27408==    by 0x10040003F: ???
    ==27408==    by 0x1FFEFFF837: ???
    ==27408==    by 0x1FFEFFF837: ???
    ==27408==    by 0x3C9D8E3B01CC2FB5: ???
    ==27408==  Address 0x6c667265766f2073 is not stack'd, malloc'd or (recently) free'd
    ...

[0] https://gist.github.com/eddyz87/251e5f0f676a0f954d4f604c83b4922d

[...]

> +#define str_has_pfx(str, pfx) \
> +	(strncmp(str, pfx, __builtin_constant_p(pfx) ? sizeof(pfx) - 1 : strlen(pfx)) == 0)

Nitpick: both gcc and clang optimize away strlen("foobar"),
         __builtin_constant_p check is not necessary.

[...]

Re: [PATCH v5 bpf-next 10/23] selftests/bpf: BPF register range bounds tester

[Andrii Nakryiko]

On Wed, Nov 8, 2023 at 2:09 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> > On Fri, 2023-10-27 at 11:13 -0700, Andrii Nakryiko wrote:
>
> I read through whole program and it seems to be a good specimen of an
> idiomatic C program. Aside from two nitpicks below I can't really complain
> about the code.
>
> On the other hand, I'm a bit at odds with the core idea.
> The algorithm for ranges computation repeats the one used in kernel
> (arguably in a bit more elegant way). So, effectively we check if two
> implementations of the same algorithm end up with the same answers.

Yes, that was the point. It's a way to cross-check a delicate and
complicated logic that is very hard to cover through manual tests. So
generated testing seemed like a better approach, for which we have to
have an alternative implementation.

Note also that it's not really just a reimplementation. While the core
idea is the same, how we get there is quite different. Kernel
implementation doesn't have any notion of casting ranges. It also
employs and keeps in sync tnum, which selftests implementation doesn't
do.

So while they do strive to implement the same behavior, they do it in
quite a different way.

>
> It is a maintenance burden in a way.

It's unlikely to need changes, so I hope it won't be. But it's also
good to have as a cross-check for future work, like Shung-Hsi Yu's
attempt to make range tracking sign-agnostic.

If this becomes a burden to maintain, it's simple to remove the
selftest. But given it's written and debugged, it adds value and makes
verifier changes a bit less scary.

> What are the benefits of such approach?
> Simpler userspace prototyping for new range tracking features?
>
> [...]
>
> > +__printf(2, 3)
> > +static inline void snappendf(struct strbuf *s, const char *fmt, ...)
> > +{
> > +     va_list args;
> > +
> > +     va_start(args, fmt);
> > +     s->pos += vsnprintf(s->buf + s->pos, s->buf_sz - s->pos, fmt, args);
> > +     va_end(args);
> > +}
>
> The manpage for vsnprintf says the following about it's return value:
>
>   ... If the output was truncated due to this limit, then the return
>   value is the number of characters (excluding the terminating null
>   byte) which would have been written to the final string if enough
>   space had been available ...
>
> Which is definitely a footgun to say the least. So, I picked strbuf,
> DEFINE_STRBUF, snappendf definitions to a separate file and tried the
> following [0]:
>
>     $ cat test.c
>     ...
>     int main(int argc, char *argv)
>     {
>       DEFINE_STRBUF(buf, 2);
>       snappendf(buf, "kinda long string...");
>       printf("buf->pos=%d\n", buf->pos);
>       snappendf(buf, "will this overflow buf?");
>     }
>
>     $ gcc -O0 -g test.c && valgrind -q ./a.out
>     buf->pos=20
>     ==27408== Jump to the invalid address stated on the next line
>     ==27408==    at 0x6C667265766F2073: ???
>     ==27408==    by 0x66756220776E: ???
>     ==27408==    by 0x401244: snappendf (test.c:24)
>     ==27408==    by 0x10040003F: ???
>     ==27408==    by 0x1FFEFFF837: ???
>     ==27408==    by 0x1FFEFFF837: ???
>     ==27408==    by 0x3C9D8E3B01CC2FB5: ???
>     ==27408==  Address 0x6c667265766f2073 is not stack'd, malloc'd or (recently) free'd
>     ...
>
> [0] https://gist.github.com/eddyz87/251e5f0f676a0f954d4f604c83b4922d
>
> [...]
>

I didn't bother to bullet-proofing it given it was added for
selftests, but you are right, it's trivial to guard against this:

diff --git a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
index a6b972036bfa..f446432bd776 100644
--- a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
+++ b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c
@@ -49,7 +49,9 @@ static inline void snappendf(struct strbuf *s, const
char *fmt, ...)
        va_list args;

        va_start(args, fmt);
-       s->pos += vsnprintf(s->buf + s->pos, s->buf_sz - s->pos, fmt, args);
+       s->pos += vsnprintf(s->buf + s->pos,
+                           s->pos < s->buf_sz ? s->buf_sz - s->pos : 0,
+                           fmt, args);
        va_end(args);
 }

I'll do this in the next revision.

> > +#define str_has_pfx(str, pfx) \
> > +     (strncmp(str, pfx, __builtin_constant_p(pfx) ? sizeof(pfx) - 1 : strlen(pfx)) == 0)
>
> Nitpick: both gcc and clang optimize away strlen("foobar"),
>          __builtin_constant_p check is not necessary.

it's a copy/paste from libbpf, but good to know it's actually
optimized! I can simplify it in the selftest, no problem.

>
> [...]

Re: [PATCH v5 bpf-next 10/23] selftests/bpf: BPF register range bounds tester

[Eduard Zingerman]

On Wed, 2023-11-08 at 15:23 -0800, Andrii Nakryiko wrote:
[...]

> > It is a maintenance burden in a way.
> 
> It's unlikely to need changes, so I hope it won't be. But it's also
> good to have as a cross-check for future work, like Shung-Hsi Yu's
> attempt to make range tracking sign-agnostic.

Testing for non-functional changes is a good point.
Anyways, I don't have a better alternatives to suggest at the moment.

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

[...]