Accessing userspace created persistent eBPF maps from kernel bpf restricted C

Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[Kiran T]

Hello,
The samples and test cases have examples showing how user space
programs can pin maps and get pinned maps as FDs.  However, how would
one access a user-space created pinned map from kernel space, without
having to statically have a 'maps' section defined (and depend on the
elf loader)?

Thanks,
Kiran

Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[Kiran T]

Hello,
The samples and test cases have examples showing how user space
programs can pin maps and get pinned maps as FDs.  However, how would
one access a user-space created pinned map from kernel space, without
having to statically have a 'maps' section defined (and depend on the
elf loader)?

Thanks,
Kiran

Re: Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[Zvi Effron]

From what I've learned building my own XDP loader, it seems like the
kernel only takes a bock of eBPF instructions. For any references to
maps, the instructions need to have been "fixed up" (if they came from
a source like an elf file) to appropriately reference the file
descriptors for any maps. As far as I can tell, those file descriptors
are the userspace file descriptors for the maps in the loading program

> The samples and test cases have examples showing how user space
> programs can pin maps and get pinned maps as FDs.  However, how would
> one access a user-space created pinned map from kernel space, without
> having to statically have a 'maps' section defined (and depend on the
> elf loader)?

You should be able to write a small userspace program that opens the
pinned maps and uses those file descriptors to create or fixup an eBPF
program from any source (elf or not), so long as you have a way to
tell which instructions need to be fixed up. (ELF puts this
information into the relocation sections.)

samples/bpf/bpf_load.c has an example of how to do the fixing up using
the relocation information from an elf file. Take a look at the
parse_relo_and_apply() function for how to adjust the eBPF
instructions.

--Zvi

Re: Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[Kiran T]

>
> You should be able to write a small userspace program that opens the
> pinned maps and uses those file descriptors to create or fixup an eBPF
> program from any source (elf or not), so long as you have a way to
> tell which instructions need to be fixed up. (ELF puts this
> information into the relocation sections.)
>

What I am really trying to do is to be able to access an eBPF hashmap or array
created and pinned by a userspace program *after* the bpf program has
been loaded, from
the bpf code.
Looks like this has not been implemented?
If maps can be made persistent from userspace, but the kernel side bpf
can't quite
access it, it doesn't make much sense :(
(I understand userspace<->userspace can access these maps, but kernel
side should be
able to as well)

Re: Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[Zvi Effron]

> What I am really trying to do is to be able to access an eBPF hashmap or array
> created and pinned by a userspace program *after* the bpf program has
> been loaded, from
> the bpf code.

Can you elaborate on your use case a little bit?

Thanks!
--Zvi

Re: Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[David Miller]

From: Kiran T <kiran.lkml@gmail.com>
Date: Thu, 7 Sep 2017 17:46:47 -0700

> If maps can be made persistent from userspace, but the kernel side
> bpf can't quite access it, it doesn't make much sense :(

Pinning is so that an unload of a BPF program doesn't cause it's maps
to go away.

It's designed so that things like statistics can be persistent across
BPF program loads.

So you can load and unload a BPF program multiple times, and you won't
lose the statistic increments done by earlier instances.

Re: Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[Kiran T]

On Thu, Sep 7, 2017 at 7:12 PM, David Miller <davem@davemloft.net> wrote:
...
>
> Pinning is so that an unload of a BPF program doesn't cause it's maps
> to go away.
>
> It's designed so that things like statistics can be persistent across
> BPF program loads.
>
> So you can load and unload a BPF program multiple times, and you won't
> lose the statistic increments done by earlier instances.

I understand.
However, I have a use case where  data used to filter perf events can
change and can
be loaded as a full map,  much after the bpf program is loaded.  The
map will be pinned,
but the bpf program currently has no way to access this map. Can this use case
be addressed?

Thanks,
Kiran

Re: Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[David Miller]

From: Kiran T <kiran.lkml@gmail.com>
Date: Thu, 7 Sep 2017 22:28:32 -0700

> On Thu, Sep 7, 2017 at 7:12 PM, David Miller <davem@davemloft.net> wrote:
> ...
>>
>> Pinning is so that an unload of a BPF program doesn't cause it's maps
>> to go away.
>>
>> It's designed so that things like statistics can be persistent across
>> BPF program loads.
>>
>> So you can load and unload a BPF program multiple times, and you won't
>> lose the statistic increments done by earlier instances.
> 
> I understand.
> However, I have a use case where  data used to filter perf events can
> change and can
> be loaded as a full map,  much after the bpf program is loaded.  The
> map will be pinned,
> but the bpf program currently has no way to access this map. Can this use case
> be addressed?

The BPF program has to know the type of the map and what it looks
like at compile time.  So what you are suggesting seems to be
impossible.

Re: Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[Zvi Effron]

> However, I have a use case where  data used to filter perf events can
> change and can
> be loaded as a full map,  much after the bpf program is loaded.  The
> map will be pinned,
> but the bpf program currently has no way to access this map. Can this use case
> be addressed?

I'm not sure entirely how, since I have only just started playing with
this map type, but I suspect a workaround is possible with
BPF_TYPE_ARRAY_OF_MAPS. My suspicion is that you could store the map
you want to replace at index 0 in a pinned array of maps. The XDP
program would load the map out of the array to use it.

To change the map, have a userspace program load the new map, then set
it as index 0 in the pinned array of maps. The next time the XDP
program runs, it will pull the new map from index 0.

Maybe that will work for you?
--Zvi

Re: Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[Jesper Dangaard Brouer]

On Thu, 7 Sep 2017 14:37:42 -0700 Kiran T <kiran.lkml@gmail.com> wrote:

> The samples and test cases have examples showing how user space
> programs can pin maps and get pinned maps as FDs.  However, how would
> one access a user-space created pinned map from kernel space, without
> having to statically have a 'maps' section defined (and depend on the
> elf loader)?

Reading further in the thread, I do realize that you have special or
strange use case that is hard/impossible with bpf, as the map type is
needed at bpf load/verifier time.

(IMHO) A more common issue is: asking how can I change/update an
BPF-elf-object-file to use another bpf-map at load-time?

The use-case behind this question is:  I have pinned bpf-map accessible
via the filesystem (bpffs), and I want several BPF programs to
share-and-use this map (on the kernel side).

This is what I'm doing in my xdp_ddos01 example[1].  You can load
xdp_ddos01 on several NICs and they will all share and use the same
blacklist-maps. I've highlighted the code here[1]:

[1] https://github.com/netoptimizer/prototype-kernel/blob/master/kernel/samples/bpf/xdp_ddos01_blacklist_user.c#L186-L218

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer

Re: Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[Daniel Borkmann]

On 09/08/2017 07:28 AM, Kiran T wrote:
> On Thu, Sep 7, 2017 at 7:12 PM, David Miller <davem@davemloft.net> wrote:
> ...
>>
>> Pinning is so that an unload of a BPF program doesn't cause it's maps
>> to go away.
>>
>> It's designed so that things like statistics can be persistent across
>> BPF program loads.
>>
>> So you can load and unload a BPF program multiple times, and you won't
>> lose the statistic increments done by earlier instances.
>
> I understand.
> However, I have a use case where  data used to filter perf events can
> change and can
> be loaded as a full map,  much after the bpf program is loaded.  The
> map will be pinned,

If I understand you correctly, you have a map that contains some
kind of filter policy that you check before proceeding. The map
that contains this data is pinned in user space, but not immediately
available at load time, and you want to be able to insert or also
potentially replace this map with a different map when you need to
change the policy. Is the assumption correct that the key/value
format of the policy would be the same and known a-priori, plus the
map types will also be the same?

If I didn't get this wrong, then just use a map in map that was
suggested earlier. So you have a BPF_TYPE_ARRAY_OF_MAPS that is
pinned as well, and when you need to replace the maps at runtime,
then you fetch the fd of the pinned BPF_TYPE_ARRAY_OF_MAPS map
and update it with the inner fd of the pinned map containing the
new policy data. That way, you don't need to reload the BPF program
(which would be atomic as well, though).

In the case where the map could change also in terms of format,
then probably best would be (if you don't want to reload the whole
program and have all contained in it) that you make a tail call
from the main program to a program that processes this specific
map and that program makes another tail call to a different program
that continues processing. So you have a flow where prog A calls
prog B which calls prog C, and prog B is replaceable atomically
during runtime. If there is some need to pass data from A to B
or B to C, then just use a per CPU map as a scratch buffer.

Re: Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[Kiran T]

On Thu, Sep 7, 2017 at 11:34 PM, Zvi Effron <zeffron@riotgames.com> wrote:
...
>
> I'm not sure entirely how, since I have only just started playing with
> this map type, but I suspect a workaround is possible with
> BPF_TYPE_ARRAY_OF_MAPS. My suspicion is that you could store the map
> you want to replace at index 0 in a pinned array of maps. The XDP
> program would load the map out of the array to use it.
>
> To change the map, have a userspace program load the new map, then set
> it as index 0 in the pinned array of maps. The next time the XDP
> program runs, it will pull the new map from index 0.
>

Well, I have to switch between maps based on an "entity switch" in the
producing the event.
The map type and format is fixed and pinned.  Say it could be one per
pid, if pid is the entity
switch.  /sys/bpf/pidno could be the pinned location.  Based on the
pid at the time of event,
I want to consult the map of that pid so I can
filter/aggregate/histogram events sent to user space.
The "entity switch" could be  major:minor:inode, or other such as well
(not easy to get from bpf now, but
you get the idea).  I can't have one or just a few maps because the
key value pairs will be
different for different entity switches.  I might not filter if
/sys/pid/pidno map is not present.

I guess this is not easy right now because of the way maps are
implemented right now,
with the verifier doing the magic fixup at the load time.  But as BPF
usage increases, these
kind of use cases will start popping up :)

Thanks for your suggestion (Zvi and Daniel).  I will explore
BPF_TYPE_ARRAY_OF_MAPS  if it
can fit this case.

Thanks,
Kiran

Re: Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[Zvi Effron]

> Well, I have to switch between maps based on an "entity switch" in the
> producing the event.
> The map type and format is fixed and pinned.  Say it could be one per
> pid, if pid is the entity
> switch.  /sys/bpf/pidno could be the pinned location.  Based on the
> pid at the time of event,
> I want to consult the map of that pid so I can
> filter/aggregate/histogram events sent to user space.
> The "entity switch" could be  major:minor:inode, or other such as well
> (not easy to get from bpf now, but
> you get the idea).  I can't have one or just a few maps because the
> key value pairs will be
> different for different entity switches.  I might not filter if
> /sys/pid/pidno map is not present.

> Thanks for your suggestion (Zvi and Daniel).  I will explore
> BPF_TYPE_ARRAY_OF_MAPS  if it
> can fit this case.

It sounds like BPF_TYPE_HASH_OF_MAPS might be more of what you need,
then. It's the same as BPF_TYPE_ARRAY_OF_MAPS, only it allows for a
key instead of an index. You could use the major:minor:inode or
whatever as the key, and then your problem becomes how the eBPF
program can calculate the key.

Good luck!
--Zvi

Re: Accessing userspace created persistent eBPF maps from kernel bpf restricted C

[Y Song]

On Fri, Sep 8, 2017 at 10:37 AM, Zvi Effron <zeffron@riotgames.com> wrote:
>> Well, I have to switch between maps based on an "entity switch" in the
>> producing the event.
>> The map type and format is fixed and pinned.  Say it could be one per
>> pid, if pid is the entity
>> switch.  /sys/bpf/pidno could be the pinned location.  Based on the
>> pid at the time of event,
>> I want to consult the map of that pid so I can
>> filter/aggregate/histogram events sent to user space.
>> The "entity switch" could be  major:minor:inode, or other such as well
>> (not easy to get from bpf now, but
>> you get the idea).  I can't have one or just a few maps because the
>> key value pairs will be
>> different for different entity switches.  I might not filter if
>> /sys/pid/pidno map is not present.
>
>> Thanks for your suggestion (Zvi and Daniel).  I will explore
>> BPF_TYPE_ARRAY_OF_MAPS  if it
>> can fit this case.
>
> It sounds like BPF_TYPE_HASH_OF_MAPS might be more of what you need,
> then. It's the same as BPF_TYPE_ARRAY_OF_MAPS, only it allows for a
> key instead of an index. You could use the major:minor:inode or
> whatever as the key, and then your problem becomes how the eBPF
> program can calculate the key.

If you know the details of each map (map type, key/value format), you
can have an app specific map
(one element to indicate (active map fd, active map property)) and you
can have a giant if statement

  /* fd is the map fd */
  if (active_map == map1) {
    ...
  } else if (active_map == map2) {
    ...
  }

>
> Good luck!
> --Zvi