Re: [LSF/MM/BPF TOPIC] BPF local storage for every packet

[Jakub Sitnicki <jakub@cloudflare.com>]

On Tue, Mar 03, 2026 at 07:06 AM -08, Zhu Yanjun wrote:
> 在 2026/2/20 6:56, Jakub Sitnicki 写道:
>> In the upcoming days we are going to post an RFC which proposes to
>> extend the concept of BPF local storage to socket buffers (sk_buff, skb)
>> as means to attach arbitrary metadata to packets from BPF programs [1]
>> (slides 41-55).
>> Design wise, BPF local storage is a great fit for a packet metadata
>> container, as it that avoids some of the shortcoming of the the XDP
>> metadata interface:
>> 1. Users interact with storage through BPF maps and can take advantage
>>     of existing built-in BPF map types, while still being able to
>>     implement a custom data format,
>> 2. Maps within local storage can have different properties controlled by
>>     map flags. For example, maps with BPF_F_CLONE set can survive packet
>>     cloning. Other flags could allow map contents to survive sk_buff
>>     scrubbing during encapsulation/decapsulation or pass across network
>>     namespace boundaries.
>> 3. Local storage supports multiple users out of the box - each user
>>     creates their own map, eliminating the need to coordinate data
>>     layout,
>> 4. Local storage has its own backing memory, so persisting it across
>>     network stack layers requires no changes to the network stack.
>> However, this flexibility comes at a cost. While XDP metadata requires
>> no allocations [2], an initial write to BPF local storage requires two:
>> one for bpf_local_storage_elem, and one for bpf_local_storage itself.
>> We would like to align this work with the needs of other BPF local
>> storage users (socks, cgroups, tasks, inodes), where allocation overhead
>> has been a concern as well [2].
>> Optimization ideas we would like to put up for discussion:
>> - slimming down bpf_local_storage so it can be embedded as an skb
>>    extension chunk,
>
> Interested in this topic. I hope to join this meeting.

Thanks for the interest. I've since posted the RFC for that [1] and the
topic is, at least partially, no longer relevant. We won't be adding new
users of BPF local storage [2].

I've proposed to the PC that we can change it to:

1) How to make regular BPF maps work as a stash-away storage for skb
metadata. I've highlighted my initial concerns [3] and will give it a
try to get hands-on experience with this approach.

2) Or if we decide to go with a secondary skb metadata embedded in the
skb_ext - which is another direction I wanted to explore, then we could
discuss how to optimize skb_ext (this overlaps with the original
proposal).

Thanks,
-jkbs

[1] https://lore.kernel.org/all/20260226-skb-local-storage-v1-0-4ca44f0dd9d1@cloudflare.com/
[2] https://lore.kernel.org/all/CAADnVQKVfyh3_OZshvYf7GJUF-ph2eMfmaQsxNgwBJd1AJgXTQ@mail.gmail.com/
[3] https://lore.kernel.org/all/87wlzydk12.fsf@cloudflare.com/