[PATCH] selftests: bpf: Support dynamic linking LLVM if static not available

[PATCH] selftests: bpf: Support dynamic linking LLVM if static not available

[Daniel Xu]

Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
libraries"), only statically linking test_progs is supported. However,
some distros only provide a dynamically linkable LLVM.

This commit adds a fallback for dynamically linking LLVM if static
linking is not available. If both options are available, static linking
is chosen.

Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
---
 tools/testing/selftests/bpf/Makefile | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 6722080b2107..da514030a153 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -184,9 +184,14 @@ ifeq ($(feature-llvm),1)
   LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
   # both llvm-config and lib.mk add -D_GNU_SOURCE, which ends up as conflict
   LLVM_CFLAGS  += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
-  LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
-  LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
-  LLVM_LDLIBS  += -lstdc++
+  # Prefer linking statically if it's available, otherwise fallback to shared
+  ifeq ($(shell $(LLVM_CONFIG) --link-static --libs &> /dev/null && echo static),static)
+    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
+    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
+    LLVM_LDLIBS  += -lstdc++
+  else
+    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-shared --libs $(LLVM_CONFIG_LIB_COMPONENTS))
+  endif
   LLVM_LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
 endif
 
-- 
2.47.1

Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available

[Eduard Zingerman]

On Thu, 2025-01-30 at 15:33 -0700, Daniel Xu wrote:
> Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
> libraries"), only statically linking test_progs is supported. However,
> some distros only provide a dynamically linkable LLVM.
> 
> This commit adds a fallback for dynamically linking LLVM if static
> linking is not available. If both options are available, static linking
> is chosen.
> 
> Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
> ---

Tested in two configuration:
- when both static and dynamic libraries are available
  (linked against static);
- when only static libraries are available.

Tested-by: Eduard Zingerman <eddyz87@gmail.com>

[...]

Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available

[Yonghong Song]

On 1/30/25 2:33 PM, Daniel Xu wrote:
> Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
> libraries"), only statically linking test_progs is supported. However,
> some distros only provide a dynamically linkable LLVM.
>
> This commit adds a fallback for dynamically linking LLVM if static
> linking is not available. If both options are available, static linking
> is chosen.
>
> Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>

LGTM.

Acked-by: Yonghong Song <yonghong.song@linux.dev>

Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available

[Yonghong Song]

On 1/30/25 2:33 PM, Daniel Xu wrote:
> Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
> libraries"), only statically linking test_progs is supported. However,
> some distros only provide a dynamically linkable LLVM.
>
> This commit adds a fallback for dynamically linking LLVM if static
> linking is not available. If both options are available, static linking
> is chosen.
>
> Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
> ---
>   tools/testing/selftests/bpf/Makefile | 11 ++++++++---
>   1 file changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 6722080b2107..da514030a153 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -184,9 +184,14 @@ ifeq ($(feature-llvm),1)
>     LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
>     # both llvm-config and lib.mk add -D_GNU_SOURCE, which ends up as conflict
>     LLVM_CFLAGS  += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
> -  LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> -  LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> -  LLVM_LDLIBS  += -lstdc++
> +  # Prefer linking statically if it's available, otherwise fallback to shared
> +  ifeq ($(shell $(LLVM_CONFIG) --link-static --libs &> /dev/null && echo static),static)
> +    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> +    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> +    LLVM_LDLIBS  += -lstdc++
> +  else
> +    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-shared --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> +  endif
>     LLVM_LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
>   endif

Although your change looks good, but maybe you can look at bpftool Makefile?

   # If LLVM is available, use it for JIT disassembly
   CFLAGS  += -DHAVE_LLVM_SUPPORT
   LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
   # llvm-config always adds -D_GNU_SOURCE, however, it may already be in CFLAGS
   # (e.g. when bpftool build is called from selftests build as selftests
   # Makefile includes lib.mk which sets -D_GNU_SOURCE) which would cause
   # compilation error due to redefinition. Let's filter it out here.
   CFLAGS  += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
   LIBS    += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
   ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static)
     LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
     LIBS += -lstdc++
   endif
   LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)

It would be great if the selftests shared library handling to be the same as bpftool's.

Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available

[Daniel Xu]

Hi Yonghong,

On Thu, Jan 30, 2025 at 10:28:11PM -0800, Yonghong Song wrote:
> 
> 
> 
> On 1/30/25 2:33 PM, Daniel Xu wrote:
> > Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
> > libraries"), only statically linking test_progs is supported. However,
> > some distros only provide a dynamically linkable LLVM.
> > 
> > This commit adds a fallback for dynamically linking LLVM if static
> > linking is not available. If both options are available, static linking
> > is chosen.
> > 
> > Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
> > ---
> >   tools/testing/selftests/bpf/Makefile | 11 ++++++++---
> >   1 file changed, 8 insertions(+), 3 deletions(-)
> > 
> > diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> > index 6722080b2107..da514030a153 100644
> > --- a/tools/testing/selftests/bpf/Makefile
> > +++ b/tools/testing/selftests/bpf/Makefile
> > @@ -184,9 +184,14 @@ ifeq ($(feature-llvm),1)
> >     LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
> >     # both llvm-config and lib.mk add -D_GNU_SOURCE, which ends up as conflict
> >     LLVM_CFLAGS  += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
> > -  LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> > -  LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> > -  LLVM_LDLIBS  += -lstdc++
> > +  # Prefer linking statically if it's available, otherwise fallback to shared
> > +  ifeq ($(shell $(LLVM_CONFIG) --link-static --libs &> /dev/null && echo static),static)
> > +    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> > +    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> > +    LLVM_LDLIBS  += -lstdc++
> > +  else
> > +    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-shared --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> > +  endif
> >     LLVM_LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
> >   endif
> 
> Although your change looks good, but maybe you can look at bpftool Makefile?
> 
>   # If LLVM is available, use it for JIT disassembly
>   CFLAGS  += -DHAVE_LLVM_SUPPORT
>   LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
>   # llvm-config always adds -D_GNU_SOURCE, however, it may already be in CFLAGS
>   # (e.g. when bpftool build is called from selftests build as selftests
>   # Makefile includes lib.mk which sets -D_GNU_SOURCE) which would cause
>   # compilation error due to redefinition. Let's filter it out here.
>   CFLAGS  += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
>   LIBS    += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
>   ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static)
>     LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
>     LIBS += -lstdc++
>   endif
>   LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
> 
> It would be great if the selftests shared library handling to be the same as bpftool's.

So bpftool is both an internally consumed (from selftests) dependency as
well as a tool packaged up by distros. For the latter case, distros
prefer dynamic linking.

So unfortunately, I think these probably need to be defined separately.
The code looks similar but the use cases are different.

Thanks,
Daniel

Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available

[Yonghong Song]

On 2/1/25 12:23 AM, Daniel Xu wrote:
> Hi Yonghong,
>
> On Thu, Jan 30, 2025 at 10:28:11PM -0800, Yonghong Song wrote:
>>
>>
>> On 1/30/25 2:33 PM, Daniel Xu wrote:
>>> Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
>>> libraries"), only statically linking test_progs is supported. However,
>>> some distros only provide a dynamically linkable LLVM.
>>>
>>> This commit adds a fallback for dynamically linking LLVM if static
>>> linking is not available. If both options are available, static linking
>>> is chosen.
>>>
>>> Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
>>> ---
>>>    tools/testing/selftests/bpf/Makefile | 11 ++++++++---
>>>    1 file changed, 8 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
>>> index 6722080b2107..da514030a153 100644
>>> --- a/tools/testing/selftests/bpf/Makefile
>>> +++ b/tools/testing/selftests/bpf/Makefile
>>> @@ -184,9 +184,14 @@ ifeq ($(feature-llvm),1)
>>>      LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
>>>      # both llvm-config and lib.mk add -D_GNU_SOURCE, which ends up as conflict
>>>      LLVM_CFLAGS  += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
>>> -  LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
>>> -  LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
>>> -  LLVM_LDLIBS  += -lstdc++
>>> +  # Prefer linking statically if it's available, otherwise fallback to shared
>>> +  ifeq ($(shell $(LLVM_CONFIG) --link-static --libs &> /dev/null && echo static),static)
>>> +    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
>>> +    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
>>> +    LLVM_LDLIBS  += -lstdc++
>>> +  else
>>> +    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-shared --libs $(LLVM_CONFIG_LIB_COMPONENTS))
>>> +  endif
>>>      LLVM_LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
>>>    endif
>> Although your change looks good, but maybe you can look at bpftool Makefile?
>>
>>    # If LLVM is available, use it for JIT disassembly
>>    CFLAGS  += -DHAVE_LLVM_SUPPORT
>>    LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
>>    # llvm-config always adds -D_GNU_SOURCE, however, it may already be in CFLAGS
>>    # (e.g. when bpftool build is called from selftests build as selftests
>>    # Makefile includes lib.mk which sets -D_GNU_SOURCE) which would cause
>>    # compilation error due to redefinition. Let's filter it out here.
>>    CFLAGS  += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
>>    LIBS    += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
>>    ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static)
>>      LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
>>      LIBS += -lstdc++
>>    endif
>>    LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
>>
>> It would be great if the selftests shared library handling to be the same as bpftool's.
> So bpftool is both an internally consumed (from selftests) dependency as
> well as a tool packaged up by distros. For the latter case, distros
> prefer dynamic linking.

I hacked llvm to have both static and shared libraries installed and indeed
`llvm-config --shared-mode` prefers shared mode.

So yes, your existing change looks good. Thanks.

>
> So unfortunately, I think these probably need to be defined separately.
> The code looks similar but the use cases are different.
>
> Thanks,
> Daniel

Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available

[Andrii Nakryiko]

On Thu, Jan 30, 2025 at 2:34 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
>
> Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
> libraries"), only statically linking test_progs is supported. However,
> some distros only provide a dynamically linkable LLVM.
>
> This commit adds a fallback for dynamically linking LLVM if static
> linking is not available. If both options are available, static linking
> is chosen.
>
> Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
> ---
>  tools/testing/selftests/bpf/Makefile | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
>

Note, we expect [PATCH bpf-next] as subject prefix and "selftests/bpf:
" prefix for BPF selftest-related patches. Please help with
consistency. I fixed it up while applying.

> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 6722080b2107..da514030a153 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -184,9 +184,14 @@ ifeq ($(feature-llvm),1)
>    LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
>    # both llvm-config and lib.mk add -D_GNU_SOURCE, which ends up as conflict
>    LLVM_CFLAGS  += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
> -  LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> -  LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> -  LLVM_LDLIBS  += -lstdc++
> +  # Prefer linking statically if it's available, otherwise fallback to shared
> +  ifeq ($(shell $(LLVM_CONFIG) --link-static --libs &> /dev/null && echo static),static)
> +    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> +    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> +    LLVM_LDLIBS  += -lstdc++
> +  else
> +    LLVM_LDLIBS  += $(shell $(LLVM_CONFIG) --link-shared --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> +  endif
>    LLVM_LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
>  endif
>
> --
> 2.47.1
>

Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to bpf/bpf-next.git (master)
by Andrii Nakryiko <andrii@kernel.org>:

On Thu, 30 Jan 2025 15:33:45 -0700 you wrote:
> Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
> libraries"), only statically linking test_progs is supported. However,
> some distros only provide a dynamically linkable LLVM.
> 
> This commit adds a fallback for dynamically linking LLVM if static
> linking is not available. If both options are available, static linking
> is chosen.
> 
> [...]

Here is the summary with links:
  - selftests: bpf: Support dynamic linking LLVM if static not available
    https://git.kernel.org/bpf/bpf-next/c/2a9d30fac818

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html