Re: bpf-restrict-fs fails to load without DYNAMIC_FTRACE_WITH_DIRECT_CALLS on arm64

[Nathan Chancellor <nathan@kernel.org>]

On Tue, Jun 10, 2025 at 07:25:52PM -0700, Alexei Starovoitov wrote:
> On Tue, Jun 10, 2025 at 7:05 PM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > On Tue, Jun 10, 2025 at 04:37:24PM -0700, Alexei Starovoitov wrote:
> > > On Tue, Jun 10, 2025 at 4:24 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > I was able to figure out that enabling CONFIG_CFI_CLANG was the culprit
> > > > for the change in behavior but it does not appear to be the root cause,
> > > > as I can get the same error with GCC and the following diff (which
> > > > happens with CFI_CLANG because of the CALL_OPS dependency):
> > ...
> > > > -       select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS \
> > > > -               if DYNAMIC_FTRACE_WITH_ARGS && DYNAMIC_FTRACE_WITH_CALL_OPS
> > > >         select HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS \
> > > >                 if (DYNAMIC_FTRACE_WITH_ARGS && !CFI_CLANG && \
> > > >                     (CC_IS_CLANG || !CC_OPTIMIZE_FOR_SIZE))
> > > >
> > ...
> > > That's expected.
> > > See how kernel/bpf/trampoline.c is using DYNAMIC_FTRACE_WITH_DIRECT_CALLS.
> > >
> > > Theoretically we can make bpf trampoline work without it,
> > > but why bother? Just enable this config.
> >
> > As I note above, this is incompatible with CONFIG_CFI_CLANG, which is
> > more important for my particular area of testing and maintenance. Since
> > you note this is expected, I will just go back to ignoring the warning
> > in my kernel logs :) thank you for the quick response!
> 
> Somebody probably needs to fix CFI_CLANG on arm64 then.
> It's not clear to me why dynamic ftrace has to be disabled in such a case.

Commit baaf553d3bc3 ("arm64: Implement HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS")
says:

  Currently, this approach is not compatible with CLANG_CFI, as the
  presence/absence of pre-function NOPs changes the offset of the
  pre-function type hash, and there's no existing mechanism to ensure a
  consistent offset for instrumented and uninstrumented functions. When
  CLANG_CFI is enabled, the existing scheme with a global ops->func
  pointer is used, and there should be no functional change. I am
  currently working with others to allow the two to work together in
  future (though this will liekly require updated compiler support).

Mark, did anything ever come to fruition from the "currently working
with others to allow the two to work together in the future"? If you
need more context, the top of the thread is
https://lore.kernel.org/20250610232418.GA3544567@ax162/.

> It's not disabled for CFI_CLANG on x86, right?

No, it is not but as far as I can tell, DYNAMIC_FTRACE_WITH_DIRECT_CALLS
on x86 does not appear to depend on DYNAMIC_FTRACE_WITH_CALL_OPS like it
does on arm64.

Cheers,
Nathan