libbpf packaging

libbpf packaging

[Jiri Olsa]

hi guys,
we want to package libbpf and I'd like to coordinate
with you on some issues I've met on this:

1) I think libbpf should be part of kernel-tools-libs and kernel-tools-libs-devel,
   which would look like below (from early rpm build):

   $ rpm -qpl kernel-tools-libs-5.0.0-1.fc31.x86_64.rpm
   /usr/lib/.build-id
   /usr/lib/.build-id/ca
   /usr/lib/.build-id/ca/654da1e5ea553f985e28b8d98ad24e51f19e88
   /usr/lib/.build-id/f6
   /usr/lib/.build-id/f6/a788b316f26fbe70db47bfc0ef500348117023
   /usr/lib64/libbpf.so.0
   /usr/lib64/libbpf.so.0.0.1
   /usr/lib64/libcpupower.so.0
   /usr/lib64/libcpupower.so.0.0.1
   /usr/share/licenses/kernel-tools-libs
   /usr/share/licenses/kernel-tools-libs/COPYING

   $ rpm -qpl kernel-tools-libs-devel-5.0.0-1.fc31.x86_64.rpm
   /usr/include/bpf/bpf.h
   /usr/include/bpf/btf.h
   /usr/include/bpf/libbpf.h
   /usr/include/cpufreq.h
   /usr/include/cpuidle.h
   /usr/lib64/libbpf.a
   /usr/lib64/libbpf.so
   /usr/lib64/libcpupower.so

   Do you see libbpf as a standalone package or kernel-tools-libs* wuold be ok for you?

2) There's already bcc-devel's libbpf library packaged:

   $ rpm -qf /usr/lib64/libbpf.so
   bcc-devel-0.8.0-1.fc28.x86_64

   so there's a conflict.. any chance we could rename libbpf to
   something else like:

   libbpf2.so
   libbpfobject.so
   libbpfbest.so
   ...?

   I checked and I think those 2 conflicting libraries don't make
   a valid case for using 'alternatives'.
  
   Also the libbpf.so from bcc-devel has been there for some time
   so we can't just remove/rename it.. ;-)

thoughts?


thanks,
jirka

Re: libbpf packaging

[Daniel Borkmann]

On 03/25/2019 01:21 PM, Jiri Olsa wrote:
> hi guys,
> we want to package libbpf and I'd like to coordinate
> with you on some issues I've met on this:
> 
> 1) I think libbpf should be part of kernel-tools-libs and kernel-tools-libs-devel,
>    which would look like below (from early rpm build):
> 
>    $ rpm -qpl kernel-tools-libs-5.0.0-1.fc31.x86_64.rpm
>    /usr/lib/.build-id
>    /usr/lib/.build-id/ca
>    /usr/lib/.build-id/ca/654da1e5ea553f985e28b8d98ad24e51f19e88
>    /usr/lib/.build-id/f6
>    /usr/lib/.build-id/f6/a788b316f26fbe70db47bfc0ef500348117023
>    /usr/lib64/libbpf.so.0
>    /usr/lib64/libbpf.so.0.0.1
>    /usr/lib64/libcpupower.so.0
>    /usr/lib64/libcpupower.so.0.0.1
>    /usr/share/licenses/kernel-tools-libs
>    /usr/share/licenses/kernel-tools-libs/COPYING
> 
>    $ rpm -qpl kernel-tools-libs-devel-5.0.0-1.fc31.x86_64.rpm
>    /usr/include/bpf/bpf.h
>    /usr/include/bpf/btf.h
>    /usr/include/bpf/libbpf.h
>    /usr/include/cpufreq.h
>    /usr/include/cpuidle.h
>    /usr/lib64/libbpf.a
>    /usr/lib64/libbpf.so
>    /usr/lib64/libcpupower.so
> 
>    Do you see libbpf as a standalone package or kernel-tools-libs* wuold be ok for you?

My preference is definitely on making libbpf a stand-alone package, so
people can just install 'libbpf' or 'libbpf-dev{,el}' and are good to
go. Also given the pace it's growing these days, it absolutely qualifies
as a stand-alone package.

> 2) There's already bcc-devel's libbpf library packaged:
> 
>    $ rpm -qf /usr/lib64/libbpf.so
>    bcc-devel-0.8.0-1.fc28.x86_64
> 
>    so there's a conflict.. any chance we could rename libbpf to
>    something else like:
> 
>    libbpf2.so
>    libbpfobject.so
>    libbpfbest.so
>    ...?

I don't think we should rename the official libbpf package, this will
just create plain confusion and will make it much harder for potential
users to adapt in the long-term since we aim for /everyone/ to consume
official libbpf library instead of hacking their own.

I think bcc folks are migrating to official libbpf as well, at least
that was my impression. Imho, this would need fixing on bcc side then.

>    I checked and I think those 2 conflicting libraries don't make
>    a valid case for using 'alternatives'.
>   
>    Also the libbpf.so from bcc-devel has been there for some time
>    so we can't just remove/rename it.. ;-)
> thoughts?
> 
> 
> thanks,
> jirka
> 

Re: libbpf packaging

[Alexei Starovoitov]

On Mon, Mar 25, 2019 at 5:53 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>
> On 03/25/2019 01:21 PM, Jiri Olsa wrote:
> > hi guys,
> > we want to package libbpf and I'd like to coordinate
> > with you on some issues I've met on this:
> >
> > 1) I think libbpf should be part of kernel-tools-libs and kernel-tools-libs-devel,
> >    which would look like below (from early rpm build):
> >
> >    $ rpm -qpl kernel-tools-libs-5.0.0-1.fc31.x86_64.rpm
> >    /usr/lib/.build-id
> >    /usr/lib/.build-id/ca
> >    /usr/lib/.build-id/ca/654da1e5ea553f985e28b8d98ad24e51f19e88
> >    /usr/lib/.build-id/f6
> >    /usr/lib/.build-id/f6/a788b316f26fbe70db47bfc0ef500348117023
> >    /usr/lib64/libbpf.so.0
> >    /usr/lib64/libbpf.so.0.0.1
> >    /usr/lib64/libcpupower.so.0
> >    /usr/lib64/libcpupower.so.0.0.1
> >    /usr/share/licenses/kernel-tools-libs
> >    /usr/share/licenses/kernel-tools-libs/COPYING
> >
> >    $ rpm -qpl kernel-tools-libs-devel-5.0.0-1.fc31.x86_64.rpm
> >    /usr/include/bpf/bpf.h
> >    /usr/include/bpf/btf.h
> >    /usr/include/bpf/libbpf.h
> >    /usr/include/cpufreq.h
> >    /usr/include/cpuidle.h
> >    /usr/lib64/libbpf.a
> >    /usr/lib64/libbpf.so
> >    /usr/lib64/libcpupower.so
> >
> >    Do you see libbpf as a standalone package or kernel-tools-libs* wuold be ok for you?
>
> My preference is definitely on making libbpf a stand-alone package, so
> people can just install 'libbpf' or 'libbpf-dev{,el}' and are good to
> go. Also given the pace it's growing these days, it absolutely qualifies
> as a stand-alone package.

+1
libbpf should be standalone package and not part of kernel-tools.

> > 2) There's already bcc-devel's libbpf library packaged:
> >
> >    $ rpm -qf /usr/lib64/libbpf.so
> >    bcc-devel-0.8.0-1.fc28.x86_64
> >
> >    so there's a conflict.. any chance we could rename libbpf to
> >    something else like:
> >
> >    libbpf2.so
> >    libbpfobject.so
> >    libbpfbest.so
> >    ...?
>
> I don't think we should rename the official libbpf package, this will
> just create plain confusion and will make it much harder for potential
> users to adapt in the long-term since we aim for /everyone/ to consume
> official libbpf library instead of hacking their own.
>
> I think bcc folks are migrating to official libbpf as well, at least
> that was my impression. Imho, this would need fixing on bcc side then.

bcc migrated to libbpf some time ago.

Re: libbpf packaging

[Jiri Olsa]

On Mon, Mar 25, 2019 at 09:03:11AM -0700, Alexei Starovoitov wrote:
> On Mon, Mar 25, 2019 at 5:53 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >
> > On 03/25/2019 01:21 PM, Jiri Olsa wrote:
> > > hi guys,
> > > we want to package libbpf and I'd like to coordinate
> > > with you on some issues I've met on this:
> > >
> > > 1) I think libbpf should be part of kernel-tools-libs and kernel-tools-libs-devel,
> > >    which would look like below (from early rpm build):
> > >
> > >    $ rpm -qpl kernel-tools-libs-5.0.0-1.fc31.x86_64.rpm
> > >    /usr/lib/.build-id
> > >    /usr/lib/.build-id/ca
> > >    /usr/lib/.build-id/ca/654da1e5ea553f985e28b8d98ad24e51f19e88
> > >    /usr/lib/.build-id/f6
> > >    /usr/lib/.build-id/f6/a788b316f26fbe70db47bfc0ef500348117023
> > >    /usr/lib64/libbpf.so.0
> > >    /usr/lib64/libbpf.so.0.0.1
> > >    /usr/lib64/libcpupower.so.0
> > >    /usr/lib64/libcpupower.so.0.0.1
> > >    /usr/share/licenses/kernel-tools-libs
> > >    /usr/share/licenses/kernel-tools-libs/COPYING
> > >
> > >    $ rpm -qpl kernel-tools-libs-devel-5.0.0-1.fc31.x86_64.rpm
> > >    /usr/include/bpf/bpf.h
> > >    /usr/include/bpf/btf.h
> > >    /usr/include/bpf/libbpf.h
> > >    /usr/include/cpufreq.h
> > >    /usr/include/cpuidle.h
> > >    /usr/lib64/libbpf.a
> > >    /usr/lib64/libbpf.so
> > >    /usr/lib64/libcpupower.so
> > >
> > >    Do you see libbpf as a standalone package or kernel-tools-libs* wuold be ok for you?
> >
> > My preference is definitely on making libbpf a stand-alone package, so
> > people can just install 'libbpf' or 'libbpf-dev{,el}' and are good to
> > go. Also given the pace it's growing these days, it absolutely qualifies
> > as a stand-alone package.
> 
> +1
> libbpf should be standalone package and not part of kernel-tools.

ok

> 
> > > 2) There's already bcc-devel's libbpf library packaged:
> > >
> > >    $ rpm -qf /usr/lib64/libbpf.so
> > >    bcc-devel-0.8.0-1.fc28.x86_64
> > >
> > >    so there's a conflict.. any chance we could rename libbpf to
> > >    something else like:
> > >
> > >    libbpf2.so
> > >    libbpfobject.so
> > >    libbpfbest.so
> > >    ...?
> >
> > I don't think we should rename the official libbpf package, this will
> > just create plain confusion and will make it much harder for potential
> > users to adapt in the long-term since we aim for /everyone/ to consume
> > official libbpf library instead of hacking their own.
> >
> > I think bcc folks are migrating to official libbpf as well, at least
> > that was my impression. Imho, this would need fixing on bcc side then.
> 
> bcc migrated to libbpf some time ago.

yea, but looks like it still exports its own libbpf.so with some helpers:

[jolsa@krava ~]$ nm -D /usr/lib64/libbpf.so.0
...
0000000000005500 T bpf_attach_kprobe
0000000000005340 T bpf_attach_perf_event
00000000000052a0 T bpf_attach_perf_event_raw
0000000000004b50 T bpf_attach_raw_tracepoint
0000000000004af0 T bpf_attach_socket
0000000000005d10 T bpf_attach_tracepoint
0000000000005810 T bpf_attach_uprobe
0000000000004ea0 T bpf_attach_xdp
0000000000005480 T bpf_close_perf_event_fd
0000000000003990 T bpf_create_map
0000000000003cb0 T bpf_delete_elem
0000000000004b20 T bpf_detach_kprobe
0000000000004b40 T bpf_detach_tracepoint
0000000000004b30 T bpf_detach_uprobe
0000000000003d30 T bpf_get_first_key
0000000000003e70 T bpf_get_next_key
0000000000003c30 T bpf_lookup_elem
0000000000005fb0 T bpf_map_get_fd_by_id
0000000000005e40 T bpf_obj_get
0000000000003ef0 T bpf_obj_get_info
0000000000005dc0 T bpf_obj_pin
0000000000004c10 T bpf_open_perf_buffer
0000000000004d80 T bpf_open_perf_event
0000000000004980 T bpf_open_raw_sock
0000000000003f70 T bpf_prog_compute_tag
0000000000005f30 T bpf_prog_get_fd_by_id
0000000000005eb0 T bpf_prog_get_next_id
00000000000042f0 T bpf_prog_get_tag
0000000000004430 T bpf_prog_load
0000000000003ba0 T bpf_update_elem
00000000000061b0 T perf_reader_event_read
0000000000006520 T perf_reader_fd
0000000000006090 T perf_reader_free
0000000000006120 T perf_reader_mmap
0000000000006030 T perf_reader_new
00000000000063f0 T perf_reader_poll
0000000000006510 T perf_reader_set_fd
...

jirka

Re: libbpf packaging

[Michal Rostecki]

On Mon, Mar 25, 2019 at 09:03:11AM -0700, Alexei Starovoitov wrote:
> > > 2) There's already bcc-devel's libbpf library packaged:
> > >
> > >    $ rpm -qf /usr/lib64/libbpf.so
> > >    bcc-devel-0.8.0-1.fc28.x86_64
> > >
> > >    so there's a conflict.. any chance we could rename libbpf to
> > >    something else like:
> > >
> > >    libbpf2.so
> > >    libbpfobject.so
> > >    libbpfbest.so
> > >    ...?
> >
> > I don't think we should rename the official libbpf package, this will
> > just create plain confusion and will make it much harder for potential
> > users to adapt in the long-term since we aim for /everyone/ to consume
> > official libbpf library instead of hacking their own.
> >
> > I think bcc folks are migrating to official libbpf as well, at least
> > that was my impression. Imho, this would need fixing on bcc side then.
> 
> bcc migrated to libbpf some time ago.

And the libbpf.so file which is installed with bcc is "our" libbpf. bcc
simply uses libbpf (from the auto-synced standalone repo[0]) as a
submodule[1]. To package libbpf and bcc properly in Linux distros we
need a possibility to use bcc with shared libbpf library - which
probably can be achiveved by small change in bcc's CMakeLists.txt.

The standalone repo[0] might be a better base for creating a package
than full kernel sources. And using %soversion%+git%sha% (i.e.
0.0.2+git33b017) might be a better version pattern of that package.

Cheers,
Michal

[0] https://github.com/libbpf/libbpf
[1] https://github.com/iovisor/bcc/blob/master/CMakeLists.txt#L12-L16

Re: libbpf packaging

[Yonghong Song]

On 3/25/19 10:21 AM, Michal Rostecki wrote:
> On Mon, Mar 25, 2019 at 09:03:11AM -0700, Alexei Starovoitov wrote:
>>>> 2) There's already bcc-devel's libbpf library packaged:
>>>>
>>>>     $ rpm -qf /usr/lib64/libbpf.so
>>>>     bcc-devel-0.8.0-1.fc28.x86_64
>>>>
>>>>     so there's a conflict.. any chance we could rename libbpf to
>>>>     something else like:
>>>>
>>>>     libbpf2.so
>>>>     libbpfobject.so
>>>>     libbpfbest.so
>>>>     ...?
>>>
>>> I don't think we should rename the official libbpf package, this will
>>> just create plain confusion and will make it much harder for potential
>>> users to adapt in the long-term since we aim for /everyone/ to consume
>>> official libbpf library instead of hacking their own.
>>>
>>> I think bcc folks are migrating to official libbpf as well, at least
>>> that was my impression. Imho, this would need fixing on bcc side then.
>>
>> bcc migrated to libbpf some time ago.
> 
> And the libbpf.so file which is installed with bcc is "our" libbpf. bcc
> simply uses libbpf (from the auto-synced standalone repo[0]) as a
> submodule[1]. To package libbpf and bcc properly in Linux distros we
> need a possibility to use bcc with shared libbpf library - which
> probably can be achiveved by small change in bcc's CMakeLists.txt.

I think we should rename bcc libbpf.so to a different name (maybe 
libbcc_bpf.so) to avoid confusions between bcc libbpf and libbpf repo.
The bcc libbpf.so is different from libbpf repo it contains some wrappers...

I will propose to iovisor/bcc mailing list.

> 
> The standalone repo[0] might be a better base for creating a package
> than full kernel sources. And using %soversion%+git%sha% (i.e.
> 0.0.2+git33b017) might be a better version pattern of that package.
> 
> Cheers,
> Michal
> 
> [0] https://github.com/libbpf/libbpf
> [1] https://github.com/iovisor/bcc/blob/master/CMakeLists.txt#L12-L16
> 

Re: libbpf packaging

[Jiri Olsa]

On Mon, Mar 25, 2019 at 07:34:42PM +0000, Yonghong Song wrote:
> 
> 
> On 3/25/19 10:21 AM, Michal Rostecki wrote:
> > On Mon, Mar 25, 2019 at 09:03:11AM -0700, Alexei Starovoitov wrote:
> >>>> 2) There's already bcc-devel's libbpf library packaged:
> >>>>
> >>>>     $ rpm -qf /usr/lib64/libbpf.so
> >>>>     bcc-devel-0.8.0-1.fc28.x86_64
> >>>>
> >>>>     so there's a conflict.. any chance we could rename libbpf to
> >>>>     something else like:
> >>>>
> >>>>     libbpf2.so
> >>>>     libbpfobject.so
> >>>>     libbpfbest.so
> >>>>     ...?
> >>>
> >>> I don't think we should rename the official libbpf package, this will
> >>> just create plain confusion and will make it much harder for potential
> >>> users to adapt in the long-term since we aim for /everyone/ to consume
> >>> official libbpf library instead of hacking their own.
> >>>
> >>> I think bcc folks are migrating to official libbpf as well, at least
> >>> that was my impression. Imho, this would need fixing on bcc side then.
> >>
> >> bcc migrated to libbpf some time ago.
> > 
> > And the libbpf.so file which is installed with bcc is "our" libbpf. bcc
> > simply uses libbpf (from the auto-synced standalone repo[0]) as a
> > submodule[1]. To package libbpf and bcc properly in Linux distros we
> > need a possibility to use bcc with shared libbpf library - which
> > probably can be achiveved by small change in bcc's CMakeLists.txt.
> 
> I think we should rename bcc libbpf.so to a different name (maybe 
> libbcc_bpf.so) to avoid confusions between bcc libbpf and libbpf repo.
> The bcc libbpf.so is different from libbpf repo it contains some wrappers...

that'd be great first step.. then we could add libbpf
package and make bcc depend on it as suggested by Michal

> 
> I will propose to iovisor/bcc mailing list.

please keep me in cc for that

thanks,
jirka

> 
> > 
> > The standalone repo[0] might be a better base for creating a package
> > than full kernel sources. And using %soversion%+git%sha% (i.e.
> > 0.0.2+git33b017) might be a better version pattern of that package.
> > 
> > Cheers,
> > Michal
> > 
> > [0] https://github.com/libbpf/libbpf
> > [1] https://github.com/iovisor/bcc/blob/master/CMakeLists.txt#L12-L16
> > 

Re: libbpf packaging

[Yonghong Song]

Jiri,

>>>>>> 2) There's already bcc-devel's libbpf library packaged:
>>>>>>
>>>>>>      $ rpm -qf /usr/lib64/libbpf.so
>>>>>>      bcc-devel-0.8.0-1.fc28.x86_64
>>>>>>
>>>>>>      so there's a conflict.. any chance we could rename libbpf to
>>>>>>      something else like:
>>>>>>
>>>>>>      libbpf2.so
>>>>>>      libbpfobject.so
>>>>>>      libbpfbest.so
>>>>>>      ...?
>>>>>
>>>>> I don't think we should rename the official libbpf package, this will
>>>>> just create plain confusion and will make it much harder for potential
>>>>> users to adapt in the long-term since we aim for /everyone/ to consume
>>>>> official libbpf library instead of hacking their own.
>>>>>
>>>>> I think bcc folks are migrating to official libbpf as well, at least
>>>>> that was my impression. Imho, this would need fixing on bcc side then.
>>>>
>>>> bcc migrated to libbpf some time ago.
>>>
>>> And the libbpf.so file which is installed with bcc is "our" libbpf. bcc
>>> simply uses libbpf (from the auto-synced standalone repo[0]) as a
>>> submodule[1]. To package libbpf and bcc properly in Linux distros we
>>> need a possibility to use bcc with shared libbpf library - which
>>> probably can be achiveved by small change in bcc's CMakeLists.txt.
>>
>> I think we should rename bcc libbpf.so to a different name (maybe
>> libbcc_bpf.so) to avoid confusions between bcc libbpf and libbpf repo.
>> The bcc libbpf.so is different from libbpf repo it contains some wrappers...
> 
> that'd be great first step.. then we could add libbpf
> package and make bcc depend on it as suggested by Michal
> 
>>
>> I will propose to iovisor/bcc mailing list.
> 
> please keep me in cc for that

Do not know your github name, so not able to cc you on the bcc pull 
request which changes library from libbpf.{a,so} to libbcc_bpf.{a,so}.
The below is the link:
    https://github.com/iovisor/bcc/pull/2290

> 
> thanks,
> jirka
> 
>>
>>>
>>> The standalone repo[0] might be a better base for creating a package
>>> than full kernel sources. And using %soversion%+git%sha% (i.e.
>>> 0.0.2+git33b017) might be a better version pattern of that package.
>>>
>>> Cheers,
>>> Michal
>>>
>>> [0] https://github.com/libbpf/libbpf
>>> [1] https://github.com/iovisor/bcc/blob/master/CMakeLists.txt#L12-L16
>>>

Re: libbpf packaging

[Jiri Olsa]

On Wed, Mar 27, 2019 at 05:30:44AM +0000, Yonghong Song wrote:
> Jiri,
> 
> >>>>>> 2) There's already bcc-devel's libbpf library packaged:
> >>>>>>
> >>>>>>      $ rpm -qf /usr/lib64/libbpf.so
> >>>>>>      bcc-devel-0.8.0-1.fc28.x86_64
> >>>>>>
> >>>>>>      so there's a conflict.. any chance we could rename libbpf to
> >>>>>>      something else like:
> >>>>>>
> >>>>>>      libbpf2.so
> >>>>>>      libbpfobject.so
> >>>>>>      libbpfbest.so
> >>>>>>      ...?
> >>>>>
> >>>>> I don't think we should rename the official libbpf package, this will
> >>>>> just create plain confusion and will make it much harder for potential
> >>>>> users to adapt in the long-term since we aim for /everyone/ to consume
> >>>>> official libbpf library instead of hacking their own.
> >>>>>
> >>>>> I think bcc folks are migrating to official libbpf as well, at least
> >>>>> that was my impression. Imho, this would need fixing on bcc side then.
> >>>>
> >>>> bcc migrated to libbpf some time ago.
> >>>
> >>> And the libbpf.so file which is installed with bcc is "our" libbpf. bcc
> >>> simply uses libbpf (from the auto-synced standalone repo[0]) as a
> >>> submodule[1]. To package libbpf and bcc properly in Linux distros we
> >>> need a possibility to use bcc with shared libbpf library - which
> >>> probably can be achiveved by small change in bcc's CMakeLists.txt.
> >>
> >> I think we should rename bcc libbpf.so to a different name (maybe
> >> libbcc_bpf.so) to avoid confusions between bcc libbpf and libbpf repo.
> >> The bcc libbpf.so is different from libbpf repo it contains some wrappers...
> > 
> > that'd be great first step.. then we could add libbpf
> > package and make bcc depend on it as suggested by Michal
> > 
> >>
> >> I will propose to iovisor/bcc mailing list.
> > 
> > please keep me in cc for that
> 
> Do not know your github name, so not able to cc you on the bcc pull 
> request which changes library from libbpf.{a,so} to libbcc_bpf.{a,so}.
> The below is the link:
>     https://github.com/iovisor/bcc/pull/2290

it's olsajiri, that should do it, thanks

jirka