bpf-restrict-fs fails to load without DYNAMIC_FTRACE_WITH_DIRECT_CALLS on arm64

[Nathan Chancellor <nathan@kernel.org>]

Hi all,

I recently adjusted my kernel configuration for my arm64 systems that
boot Fedora to enable debug information so that BTF could be generated
so that systemd's bpf-restrict-fs program [1] can run, as it would show

  systemd[1]: bpf-restrict-fs: Failed to load BPF object: No such process

in the kernel log. After doing so though, I still get an error when the
program is loaded:

  systemd[1]: bpf-restrict-fs: Failed to link program; assuming BPF LSM is not available.

With Fedora's configuration from upstream, I see:

  systemd[1]: bpf-restrict-fs: LSM BPF program attached

I was able to figure out that enabling CONFIG_CFI_CLANG was the culprit
for the change in behavior but it does not appear to be the root cause,
as I can get the same error with GCC and the following diff (which
happens with CFI_CLANG because of the CALL_OPS dependency):

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 55fc331af337..a55754e54cd8 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -210,8 +210,8 @@ config ARM64
        select HAVE_DYNAMIC_FTRACE_WITH_ARGS \
                if (GCC_SUPPORTS_DYNAMIC_FTRACE_WITH_ARGS || \
                    CLANG_SUPPORTS_DYNAMIC_FTRACE_WITH_ARGS)
-       select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS \
-               if DYNAMIC_FTRACE_WITH_ARGS && DYNAMIC_FTRACE_WITH_CALL_OPS
+       #select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS \
+       #       if DYNAMIC_FTRACE_WITH_ARGS && DYNAMIC_FTRACE_WITH_CALL_OPS
        select HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS \
                if (DYNAMIC_FTRACE_WITH_ARGS && !CFI_CLANG && \
                    (CC_IS_CLANG || !CC_OPTIMIZE_FOR_SIZE))

which results in the following diff between the good and bad
configurations (and I already ruled out HID-BPF being involved here):

diff --git a/good-config b/bad-config
index 252f730..539e8fd 100644
--- a/good-config
+++ b/bad-config
@@ -4882,7 +4882,6 @@ CONFIG_HID_NTRIG=y
 #
 # HID-BPF support
 #
-CONFIG_HID_BPF=y
 # end of HID-BPF support

 CONFIG_I2C_HID=y
@@ -7534,7 +7533,6 @@ CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
 CONFIG_HAVE_FUNCTION_GRAPH_FREGS=y
 CONFIG_HAVE_FTRACE_GRAPH_FUNC=y
 CONFIG_HAVE_DYNAMIC_FTRACE=y
-CONFIG_HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y
 CONFIG_HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS=y
 CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS=y
 CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
@@ -7558,7 +7556,6 @@ CONFIG_FUNCTION_GRAPH_RETVAL=y
 # CONFIG_FUNCTION_GRAPH_RETADDR is not set
 CONFIG_FUNCTION_TRACE_ARGS=y
 CONFIG_DYNAMIC_FTRACE=y
-CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y
 CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS=y
 CONFIG_DYNAMIC_FTRACE_WITH_ARGS=y
 CONFIG_FPROBE=y

Is this expected behavior or is there some other issue here? I have not
tried different kernel versions yet but I certainly can if it would be
worthwhile. If it is not expected, I am happy to provide any information
that would be helpful for narrowing this down or test patches. This is
reproducible for me in a Fedora VM in QEMU as well, if it makes
reproducing easy.

[1]: https://github.com/systemd/systemd/blob/abe149d669c68bbf2a8dd4fab325c7e715f1fd85/src/core/bpf-restrict-fs.c

Cheers,
Nathan