* [PATCH bpf] bpf: Ensure 8-byte alignment for lpm_trie map values
@ 2026-11-19 15:53 Tobias Philipp
0 siblings, 0 replies; only message in thread
From: Tobias Philipp @ 2026-11-19 15:53 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Daniel Mack, linux-kernel, Tobias Philipp
The BPF verifier validates the alignment of PTR_TO_MAP_VALUE accesses in
check_ptr_alignment() only against the offset relative to the start of
the map value, assuming that the map value base address itself is at
least 8-byte aligned.
While hash maps enforce this invariant by rounding the key size up to 8
bytes (l->key + round_up(key_size, 8)), lpm_trie places the map value
immediately after the key bytes at node->data + trie->data_size without
padding. Because trie->data_size (key_size - 4) can be any value from 1
to 256, the value pointer returned by trie_lookup_elem() is only aligned
to (trie->data_size % 8).
In particular, for an IPv4 LPM trie (key_size = 8, data_size = 4), the
map value base address is always 4-byte aligned (4 mod 8, and either 4
or 12 mod 16). The verifier accepts 8-byte atomic operations
(BPF_ATOMIC | BPF_DW) at 8-byte aligned offsets within the value (e.g.
offset 0 and offset 8), but at runtime on arm64 the JITed 64-bit atomic
(stadd) faults with an unhandled hardware alignment fault (ESR.FSC =
0x21, crossing a 16-byte granule boundary on FEAT_LSE2 CPUs, or on any
4-byte aligned address without FEAT_LSE2) and panics the kernel:
Unable to handle kernel paging request at virtual address
0xffff00000159db6c
Mem abort info:
ESR = 0x0000000096000021
EC = 0x25: DABT (current EL), IL = 32 bits
FSC = 0x21: alignment fault
Internal error: Oops: 0x96000021 [#1] SMP
pc : bpf_prog_cc9155b98965fc5e+0x64/0x90
Call trace:
bpf_prog_cc9155b98965fc5e+0x64/0x90 (P)
bpf_dispatcher_xdp_func+0x14/0x28
bpf_test_run+0x11c/0x360
bpf_prog_test_run_xdp+0x358/0x548
Kernel panic - not syncing: Oops: Fatal exception in interrupt
This issue was found while formally specifying eBPF map-value alignment
invariants on Linux 7.3.5.
Fix this by marking struct lpm_trie_node's data[] flexible array as
__aligned(8) and rounding the value offset up to 8 bytes with
round_up(trie->data_size, 8) in trie_alloc(), lpm_trie_node_alloc(),
trie_lookup_elem(), and trie_mem_usage(), matching hashtab. Verified on
Linux 7.3.5 (arm64 under QEMU) with an XDP program executing 64-bit
atomic adds at offsets 0 and 8 of an IPv4 lpm_trie map value.
Fixes: b95a5c4db09b ("bpf: add a longest prefix match trie map implementation")
Assisted-by: LLM
Signed-off-by: Tobias Philipp <tphilipp@gmail.com>
---
A standalone reproducer is available on request.
kernel/bpf/lpm_trie.c | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c
index 4ef9c8c9f3a2..9b4e38d16f21 100644
--- a/kernel/bpf/lpm_trie.c
+++ b/kernel/bpf/lpm_trie.c
@@ -27,7 +27,7 @@ struct lpm_trie_node {
struct lpm_trie_node __rcu *child[2];
u32 prefixlen;
u32 flags;
- u8 data[];
+ u8 data[] __aligned(8);
};
struct lpm_trie {
@@ -286,7 +286,7 @@ static void *trie_lookup_elem(struct bpf_map *map, void *_key)
if (!found)
return NULL;
- return found->data + trie->data_size;
+ return found->data + round_up(trie->data_size, 8);
}
static struct lpm_trie_node *lpm_trie_node_alloc(struct lpm_trie *trie,
@@ -302,7 +302,7 @@ static struct lpm_trie_node *lpm_trie_node_alloc(struct lpm_trie *trie,
node->flags = 0;
if (value)
- memcpy(node->data + trie->data_size, value,
+ memcpy(node->data + round_up(trie->data_size, 8), value,
trie->map.value_size);
return node;
@@ -598,7 +598,8 @@ static struct bpf_map *trie_alloc(union bpf_attr *attr)
raw_res_spin_lock_init(&trie->lock);
/* Allocate intermediate and leaf nodes from the same allocator */
- leaf_size = sizeof(struct lpm_trie_node) + trie->data_size +
+ leaf_size = sizeof(struct lpm_trie_node) +
+ round_up(trie->data_size, 8) +
trie->map.value_size;
err = bpf_mem_alloc_init(&trie->ma, leaf_size, false);
if (err)
@@ -766,8 +767,9 @@ static u64 trie_mem_usage(const struct bpf_map *map)
struct lpm_trie *trie = container_of(map, struct lpm_trie, map);
u64 elem_size;
- elem_size = sizeof(struct lpm_trie_node) + trie->data_size +
- trie->map.value_size;
+ elem_size = sizeof(struct lpm_trie_node) +
+ round_up(trie->data_size, 8) +
+ trie->map.value_size;
return elem_size * READ_ONCE(trie->n_entries);
}
--
2.46.0
^ permalink raw reply related [flat|nested] only message in thread
only message in thread, other threads:[~2026-11-19 15:54 UTC | newest]
Thread overview: (only message) (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-19 15:53 [PATCH bpf] bpf: Ensure 8-byte alignment for lpm_trie map values Tobias Philipp
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.