* Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available
2025-01-30 22:33 [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available Daniel Xu
@ 2025-01-31 0:48 ` Eduard Zingerman
2025-01-31 6:22 ` Yonghong Song
` (3 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Eduard Zingerman @ 2025-01-31 0:48 UTC (permalink / raw)
To: Daniel Xu, shuah, andrii, ast, nathan, daniel
Cc: martin.lau, song, yonghong.song, john.fastabend, kpsingh, sdf,
haoluo, jolsa, mykolal, ndesaulniers, morbo, justinstitt, bpf,
linux-kselftest, linux-kernel, llvm
On Thu, 2025-01-30 at 15:33 -0700, Daniel Xu wrote:
> Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
> libraries"), only statically linking test_progs is supported. However,
> some distros only provide a dynamically linkable LLVM.
>
> This commit adds a fallback for dynamically linking LLVM if static
> linking is not available. If both options are available, static linking
> is chosen.
>
> Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
> ---
Tested in two configuration:
- when both static and dynamic libraries are available
(linked against static);
- when only static libraries are available.
Tested-by: Eduard Zingerman <eddyz87@gmail.com>
[...]
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available
2025-01-30 22:33 [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available Daniel Xu
2025-01-31 0:48 ` Eduard Zingerman
@ 2025-01-31 6:22 ` Yonghong Song
2025-01-31 6:28 ` Yonghong Song
` (2 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Yonghong Song @ 2025-01-31 6:22 UTC (permalink / raw)
To: Daniel Xu, shuah, andrii, eddyz87, ast, nathan, daniel
Cc: martin.lau, song, john.fastabend, kpsingh, sdf, haoluo, jolsa,
mykolal, ndesaulniers, morbo, justinstitt, bpf, linux-kselftest,
linux-kernel, llvm
On 1/30/25 2:33 PM, Daniel Xu wrote:
> Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
> libraries"), only statically linking test_progs is supported. However,
> some distros only provide a dynamically linkable LLVM.
>
> This commit adds a fallback for dynamically linking LLVM if static
> linking is not available. If both options are available, static linking
> is chosen.
>
> Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
LGTM.
Acked-by: Yonghong Song <yonghong.song@linux.dev>
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available
2025-01-30 22:33 [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available Daniel Xu
2025-01-31 0:48 ` Eduard Zingerman
2025-01-31 6:22 ` Yonghong Song
@ 2025-01-31 6:28 ` Yonghong Song
2025-02-01 8:23 ` Daniel Xu
2025-02-06 0:50 ` Andrii Nakryiko
2025-02-06 1:00 ` patchwork-bot+netdevbpf
4 siblings, 1 reply; 8+ messages in thread
From: Yonghong Song @ 2025-01-31 6:28 UTC (permalink / raw)
To: Daniel Xu, shuah, andrii, eddyz87, ast, nathan, daniel
Cc: martin.lau, song, john.fastabend, kpsingh, sdf, haoluo, jolsa,
mykolal, ndesaulniers, morbo, justinstitt, bpf, linux-kselftest,
linux-kernel, llvm
On 1/30/25 2:33 PM, Daniel Xu wrote:
> Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
> libraries"), only statically linking test_progs is supported. However,
> some distros only provide a dynamically linkable LLVM.
>
> This commit adds a fallback for dynamically linking LLVM if static
> linking is not available. If both options are available, static linking
> is chosen.
>
> Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
> ---
> tools/testing/selftests/bpf/Makefile | 11 ++++++++---
> 1 file changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 6722080b2107..da514030a153 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -184,9 +184,14 @@ ifeq ($(feature-llvm),1)
> LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
> # both llvm-config and lib.mk add -D_GNU_SOURCE, which ends up as conflict
> LLVM_CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
> - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> - LLVM_LDLIBS += -lstdc++
> + # Prefer linking statically if it's available, otherwise fallback to shared
> + ifeq ($(shell $(LLVM_CONFIG) --link-static --libs &> /dev/null && echo static),static)
> + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> + LLVM_LDLIBS += -lstdc++
> + else
> + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-shared --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> + endif
> LLVM_LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
> endif
Although your change looks good, but maybe you can look at bpftool Makefile?
# If LLVM is available, use it for JIT disassembly
CFLAGS += -DHAVE_LLVM_SUPPORT
LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
# llvm-config always adds -D_GNU_SOURCE, however, it may already be in CFLAGS
# (e.g. when bpftool build is called from selftests build as selftests
# Makefile includes lib.mk which sets -D_GNU_SOURCE) which would cause
# compilation error due to redefinition. Let's filter it out here.
CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
LIBS += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static)
LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
LIBS += -lstdc++
endif
LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
It would be great if the selftests shared library handling to be the same as bpftool's.
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available
2025-01-31 6:28 ` Yonghong Song
@ 2025-02-01 8:23 ` Daniel Xu
2025-02-02 6:22 ` Yonghong Song
0 siblings, 1 reply; 8+ messages in thread
From: Daniel Xu @ 2025-02-01 8:23 UTC (permalink / raw)
To: Yonghong Song
Cc: shuah, andrii, eddyz87, ast, nathan, daniel, martin.lau, song,
john.fastabend, kpsingh, sdf, haoluo, jolsa, mykolal,
ndesaulniers, morbo, justinstitt, bpf, linux-kselftest,
linux-kernel, llvm
Hi Yonghong,
On Thu, Jan 30, 2025 at 10:28:11PM -0800, Yonghong Song wrote:
>
>
>
> On 1/30/25 2:33 PM, Daniel Xu wrote:
> > Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
> > libraries"), only statically linking test_progs is supported. However,
> > some distros only provide a dynamically linkable LLVM.
> >
> > This commit adds a fallback for dynamically linking LLVM if static
> > linking is not available. If both options are available, static linking
> > is chosen.
> >
> > Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
> > ---
> > tools/testing/selftests/bpf/Makefile | 11 ++++++++---
> > 1 file changed, 8 insertions(+), 3 deletions(-)
> >
> > diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> > index 6722080b2107..da514030a153 100644
> > --- a/tools/testing/selftests/bpf/Makefile
> > +++ b/tools/testing/selftests/bpf/Makefile
> > @@ -184,9 +184,14 @@ ifeq ($(feature-llvm),1)
> > LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
> > # both llvm-config and lib.mk add -D_GNU_SOURCE, which ends up as conflict
> > LLVM_CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
> > - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> > - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> > - LLVM_LDLIBS += -lstdc++
> > + # Prefer linking statically if it's available, otherwise fallback to shared
> > + ifeq ($(shell $(LLVM_CONFIG) --link-static --libs &> /dev/null && echo static),static)
> > + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> > + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> > + LLVM_LDLIBS += -lstdc++
> > + else
> > + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-shared --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> > + endif
> > LLVM_LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
> > endif
>
> Although your change looks good, but maybe you can look at bpftool Makefile?
>
> # If LLVM is available, use it for JIT disassembly
> CFLAGS += -DHAVE_LLVM_SUPPORT
> LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
> # llvm-config always adds -D_GNU_SOURCE, however, it may already be in CFLAGS
> # (e.g. when bpftool build is called from selftests build as selftests
> # Makefile includes lib.mk which sets -D_GNU_SOURCE) which would cause
> # compilation error due to redefinition. Let's filter it out here.
> CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
> LIBS += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static)
> LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> LIBS += -lstdc++
> endif
> LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
>
> It would be great if the selftests shared library handling to be the same as bpftool's.
So bpftool is both an internally consumed (from selftests) dependency as
well as a tool packaged up by distros. For the latter case, distros
prefer dynamic linking.
So unfortunately, I think these probably need to be defined separately.
The code looks similar but the use cases are different.
Thanks,
Daniel
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available
2025-02-01 8:23 ` Daniel Xu
@ 2025-02-02 6:22 ` Yonghong Song
0 siblings, 0 replies; 8+ messages in thread
From: Yonghong Song @ 2025-02-02 6:22 UTC (permalink / raw)
To: Daniel Xu
Cc: shuah, andrii, eddyz87, ast, nathan, daniel, martin.lau, song,
john.fastabend, kpsingh, sdf, haoluo, jolsa, mykolal,
ndesaulniers, morbo, justinstitt, bpf, linux-kselftest,
linux-kernel, llvm
On 2/1/25 12:23 AM, Daniel Xu wrote:
> Hi Yonghong,
>
> On Thu, Jan 30, 2025 at 10:28:11PM -0800, Yonghong Song wrote:
>>
>>
>> On 1/30/25 2:33 PM, Daniel Xu wrote:
>>> Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
>>> libraries"), only statically linking test_progs is supported. However,
>>> some distros only provide a dynamically linkable LLVM.
>>>
>>> This commit adds a fallback for dynamically linking LLVM if static
>>> linking is not available. If both options are available, static linking
>>> is chosen.
>>>
>>> Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
>>> ---
>>> tools/testing/selftests/bpf/Makefile | 11 ++++++++---
>>> 1 file changed, 8 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
>>> index 6722080b2107..da514030a153 100644
>>> --- a/tools/testing/selftests/bpf/Makefile
>>> +++ b/tools/testing/selftests/bpf/Makefile
>>> @@ -184,9 +184,14 @@ ifeq ($(feature-llvm),1)
>>> LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
>>> # both llvm-config and lib.mk add -D_GNU_SOURCE, which ends up as conflict
>>> LLVM_CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
>>> - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
>>> - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
>>> - LLVM_LDLIBS += -lstdc++
>>> + # Prefer linking statically if it's available, otherwise fallback to shared
>>> + ifeq ($(shell $(LLVM_CONFIG) --link-static --libs &> /dev/null && echo static),static)
>>> + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
>>> + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
>>> + LLVM_LDLIBS += -lstdc++
>>> + else
>>> + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-shared --libs $(LLVM_CONFIG_LIB_COMPONENTS))
>>> + endif
>>> LLVM_LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
>>> endif
>> Although your change looks good, but maybe you can look at bpftool Makefile?
>>
>> # If LLVM is available, use it for JIT disassembly
>> CFLAGS += -DHAVE_LLVM_SUPPORT
>> LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
>> # llvm-config always adds -D_GNU_SOURCE, however, it may already be in CFLAGS
>> # (e.g. when bpftool build is called from selftests build as selftests
>> # Makefile includes lib.mk which sets -D_GNU_SOURCE) which would cause
>> # compilation error due to redefinition. Let's filter it out here.
>> CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
>> LIBS += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
>> ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static)
>> LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
>> LIBS += -lstdc++
>> endif
>> LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
>>
>> It would be great if the selftests shared library handling to be the same as bpftool's.
> So bpftool is both an internally consumed (from selftests) dependency as
> well as a tool packaged up by distros. For the latter case, distros
> prefer dynamic linking.
I hacked llvm to have both static and shared libraries installed and indeed
`llvm-config --shared-mode` prefers shared mode.
So yes, your existing change looks good. Thanks.
>
> So unfortunately, I think these probably need to be defined separately.
> The code looks similar but the use cases are different.
>
> Thanks,
> Daniel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available
2025-01-30 22:33 [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available Daniel Xu
` (2 preceding siblings ...)
2025-01-31 6:28 ` Yonghong Song
@ 2025-02-06 0:50 ` Andrii Nakryiko
2025-02-06 1:00 ` patchwork-bot+netdevbpf
4 siblings, 0 replies; 8+ messages in thread
From: Andrii Nakryiko @ 2025-02-06 0:50 UTC (permalink / raw)
To: Daniel Xu
Cc: shuah, andrii, eddyz87, ast, nathan, daniel, martin.lau, song,
yonghong.song, john.fastabend, kpsingh, sdf, haoluo, jolsa,
mykolal, ndesaulniers, morbo, justinstitt, bpf, linux-kselftest,
linux-kernel, llvm
On Thu, Jan 30, 2025 at 2:34 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
>
> Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
> libraries"), only statically linking test_progs is supported. However,
> some distros only provide a dynamically linkable LLVM.
>
> This commit adds a fallback for dynamically linking LLVM if static
> linking is not available. If both options are available, static linking
> is chosen.
>
> Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
> ---
> tools/testing/selftests/bpf/Makefile | 11 ++++++++---
> 1 file changed, 8 insertions(+), 3 deletions(-)
>
Note, we expect [PATCH bpf-next] as subject prefix and "selftests/bpf:
" prefix for BPF selftest-related patches. Please help with
consistency. I fixed it up while applying.
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 6722080b2107..da514030a153 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -184,9 +184,14 @@ ifeq ($(feature-llvm),1)
> LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
> # both llvm-config and lib.mk add -D_GNU_SOURCE, which ends up as conflict
> LLVM_CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
> - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> - LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> - LLVM_LDLIBS += -lstdc++
> + # Prefer linking statically if it's available, otherwise fallback to shared
> + ifeq ($(shell $(LLVM_CONFIG) --link-static --libs &> /dev/null && echo static),static)
> + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-static --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> + LLVM_LDLIBS += -lstdc++
> + else
> + LLVM_LDLIBS += $(shell $(LLVM_CONFIG) --link-shared --libs $(LLVM_CONFIG_LIB_COMPONENTS))
> + endif
> LLVM_LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
> endif
>
> --
> 2.47.1
>
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available
2025-01-30 22:33 [PATCH] selftests: bpf: Support dynamic linking LLVM if static not available Daniel Xu
` (3 preceding siblings ...)
2025-02-06 0:50 ` Andrii Nakryiko
@ 2025-02-06 1:00 ` patchwork-bot+netdevbpf
4 siblings, 0 replies; 8+ messages in thread
From: patchwork-bot+netdevbpf @ 2025-02-06 1:00 UTC (permalink / raw)
To: Daniel Xu
Cc: shuah, andrii, eddyz87, ast, nathan, daniel, martin.lau, song,
yonghong.song, john.fastabend, kpsingh, sdf, haoluo, jolsa,
mykolal, ndesaulniers, morbo, justinstitt, bpf, linux-kselftest,
linux-kernel, llvm
Hello:
This patch was applied to bpf/bpf-next.git (master)
by Andrii Nakryiko <andrii@kernel.org>:
On Thu, 30 Jan 2025 15:33:45 -0700 you wrote:
> Since 67ab80a01886 ("selftests/bpf: Prefer static linking for LLVM
> libraries"), only statically linking test_progs is supported. However,
> some distros only provide a dynamically linkable LLVM.
>
> This commit adds a fallback for dynamically linking LLVM if static
> linking is not available. If both options are available, static linking
> is chosen.
>
> [...]
Here is the summary with links:
- selftests: bpf: Support dynamic linking LLVM if static not available
https://git.kernel.org/bpf/bpf-next/c/2a9d30fac818
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 8+ messages in thread