[PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile

[PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile

[Douglas Anderson]

From: Han Wang <nesthanwang@google.com>

AutoFDO profiles are commonly collected on one kernel revision and
reused across subsequent commits or stable merges before a fresh
profile is generated. While AutoFDO uses function-relative line
numbers to tolerate edits above a function, adding or removing lines
inside a function (or an inlined helper) shifts all subsequent
relative line offsets within that function. Without stale profile
salvaging, shifted callsites fail to match their nested inlinee
profiles in SampleProfileLoader (dropping profile-guided inlining for
those callees) and basic block sample counts are misattributed to the
wrong instructions, degrading branch probability estimation and
machine function splitting.

Enable LLVM stale profile salvaging (-salvage-stale-profile, supported
since LLVM 17) for C, Rust, and ThinLTO builds when
CLANG_AUTOFDO_PROFILE is specified. This instructs SampleProfileMatcher
to use lexical callsite anchors within functions to fuzzy-match and
remap shifted line offsets when querying the profile.

While -salvage-stale-profile is not enabled by default in LLVM due to
the minor compile-time/memory cost of sequence-matching callsite
anchors (which is unnecessary when profiles are freshly generated on
the exact commit being built) and the heuristic nature of fuzzy
matching on heavily rewritten functions, enabling it makes sense for
the kernel: hardware PMU profiles are rarely regenerated on every
commit, and anchoring on callsite names is strictly more resilient
than blind line-offset lookup when functions are edited.

As a side benefit, remapping shifted line offsets also avoids feeding
mismatched weights from stale profiles into MIRProfileLoader's flow
inference solver (applyFlowInference), working around a compiler bug
[1] where a small intra-function line shift (such as in
__vm_normal_page() in mm/memory.c) can cause applyFlowInference to
spend 30+ minutes compiling unmap_page_range().

Link: https://github.com/ClangBuiltLinux/linux/issues/2188 [1]
Signed-off-by: Han Wang <nesthanwang@google.com>
Signed-off-by: Douglas Anderson <dianders@chromium.org>
---
Much of the commit-message justification here is AI-generated. Sorry!
:( Folks who know better, feel free to call bullshit.

Honestly, this patch came about because of AI analysis compiles taking
30 minutes to run (see the clang bug [1]). It seems to work around the
problem. Obviously, we should fix the compiler. That being said, upon
further analysis it seemed like adding this flag was beneficial
anyway, so I'm posting it upstream. If folks disagree, we can simply
drop this patch.

 scripts/Makefile.autofdo | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/scripts/Makefile.autofdo b/scripts/Makefile.autofdo
index 57a3aba6ffc2..3b1e65a13ffa 100644
--- a/scripts/Makefile.autofdo
+++ b/scripts/Makefile.autofdo
@@ -13,13 +13,16 @@ endif
 ifdef CLANG_AUTOFDO_PROFILE
   CFLAGS_AUTOFDO_CLANG += -fprofile-sample-use=$(CLANG_AUTOFDO_PROFILE) -ffunction-sections
   CFLAGS_AUTOFDO_CLANG += -fsplit-machine-functions
+  CFLAGS_AUTOFDO_CLANG += -mllvm -salvage-stale-profile
   RUSTFLAGS_AUTOFDO_CLANG += -Zprofile-sample-use=$(CLANG_AUTOFDO_PROFILE) -Zfunction-sections=y
   RUSTFLAGS_AUTOFDO_CLANG += -Cllvm-args=-split-machine-functions
+  RUSTFLAGS_AUTOFDO_CLANG += -Cllvm-args=-salvage-stale-profile
 endif
 
 ifdef CONFIG_LTO_CLANG_THIN
   ifdef CLANG_AUTOFDO_PROFILE
     KBUILD_LDFLAGS += --lto-sample-profile=$(CLANG_AUTOFDO_PROFILE)
+    KBUILD_LDFLAGS += --mllvm=-salvage-stale-profile
   endif
   KBUILD_LDFLAGS += --mllvm=-enable-fs-discriminator=true --mllvm=-improved-fs-discriminator=true -plugin-opt=thinlto
   KBUILD_LDFLAGS += $(call ld-option,-plugin-opt=-function-splitting=all,-plugin-opt=-split-machine-functions)
-- 
2.56.0.455.g6c5a503b38-goog

Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile

[Nathan Chancellor]

On Wed, Nov 04, 2026 at 05:45:55PM -0800, Douglas Anderson wrote:
> From: Han Wang <nesthanwang@google.com>
> 
> AutoFDO profiles are commonly collected on one kernel revision and
> reused across subsequent commits or stable merges before a fresh
> profile is generated. While AutoFDO uses function-relative line
> numbers to tolerate edits above a function, adding or removing lines
> inside a function (or an inlined helper) shifts all subsequent
> relative line offsets within that function. Without stale profile
> salvaging, shifted callsites fail to match their nested inlinee
> profiles in SampleProfileLoader (dropping profile-guided inlining for
> those callees) and basic block sample counts are misattributed to the
> wrong instructions, degrading branch probability estimation and
> machine function splitting.
> 
> Enable LLVM stale profile salvaging (-salvage-stale-profile, supported
> since LLVM 17) for C, Rust, and ThinLTO builds when
> CLANG_AUTOFDO_PROFILE is specified. This instructs SampleProfileMatcher
> to use lexical callsite anchors within functions to fuzzy-match and
> remap shifted line offsets when querying the profile.
> 
> While -salvage-stale-profile is not enabled by default in LLVM due to
> the minor compile-time/memory cost of sequence-matching callsite
> anchors (which is unnecessary when profiles are freshly generated on
> the exact commit being built) and the heuristic nature of fuzzy
> matching on heavily rewritten functions, enabling it makes sense for
> the kernel: hardware PMU profiles are rarely regenerated on every
> commit, and anchoring on callsite names is strictly more resilient
> than blind line-offset lookup when functions are edited.
> 
> As a side benefit, remapping shifted line offsets also avoids feeding
> mismatched weights from stale profiles into MIRProfileLoader's flow
> inference solver (applyFlowInference), working around a compiler bug
> [1] where a small intra-function line shift (such as in
> __vm_normal_page() in mm/memory.c) can cause applyFlowInference to
> spend 30+ minutes compiling unmap_page_range().
> 
> Link: https://github.com/ClangBuiltLinux/linux/issues/2188 [1]
> Signed-off-by: Han Wang <nesthanwang@google.com>
> Signed-off-by: Douglas Anderson <dianders@chromium.org>
> ---
> Much of the commit-message justification here is AI-generated. Sorry!
> :( Folks who know better, feel free to call bullshit.

The justification does not seem wrong to me on initial read but it
definitely feels like the first and third paragraphs could be reduced in
verbosity since this can basically be summarized by "AutoFDO profiles
are often reused, which can be problematic without profile salvaging
enabled, which is designed for this exact situation".

> Honestly, this patch came about because of AI analysis compiles taking
> 30 minutes to run (see the clang bug [1]). It seems to work around the
> problem. Obviously, we should fix the compiler. That being said, upon
> further analysis it seemed like adding this flag was beneficial
> anyway, so I'm posting it upstream. If folks disagree, we can simply
> drop this patch.

I would definitely like to see that issue triaged but like you say, it
probably does not need to done before this is merged since it has
reasonable justification on its own. I would love to get thoughts from
other folks though.

>  scripts/Makefile.autofdo | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/scripts/Makefile.autofdo b/scripts/Makefile.autofdo
> index 57a3aba6ffc2..3b1e65a13ffa 100644
> --- a/scripts/Makefile.autofdo
> +++ b/scripts/Makefile.autofdo
> @@ -13,13 +13,16 @@ endif
>  ifdef CLANG_AUTOFDO_PROFILE
>    CFLAGS_AUTOFDO_CLANG += -fprofile-sample-use=$(CLANG_AUTOFDO_PROFILE) -ffunction-sections
>    CFLAGS_AUTOFDO_CLANG += -fsplit-machine-functions
> +  CFLAGS_AUTOFDO_CLANG += -mllvm -salvage-stale-profile
>    RUSTFLAGS_AUTOFDO_CLANG += -Zprofile-sample-use=$(CLANG_AUTOFDO_PROFILE) -Zfunction-sections=y
>    RUSTFLAGS_AUTOFDO_CLANG += -Cllvm-args=-split-machine-functions
> +  RUSTFLAGS_AUTOFDO_CLANG += -Cllvm-args=-salvage-stale-profile
>  endif
>  
>  ifdef CONFIG_LTO_CLANG_THIN
>    ifdef CLANG_AUTOFDO_PROFILE
>      KBUILD_LDFLAGS += --lto-sample-profile=$(CLANG_AUTOFDO_PROFILE)
> +    KBUILD_LDFLAGS += --mllvm=-salvage-stale-profile
>    endif
>    KBUILD_LDFLAGS += --mllvm=-enable-fs-discriminator=true --mllvm=-improved-fs-discriminator=true -plugin-opt=thinlto
>    KBUILD_LDFLAGS += $(call ld-option,-plugin-opt=-function-splitting=all,-plugin-opt=-split-machine-functions)
> -- 
> 2.56.0.455.g6c5a503b38-goog
> 

-- 
Cheers,
Nathan

Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile

[Doug Anderson]

Hi Nathan,

On Wed, Nov 4, 2026 at 9:24 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> On Wed, Nov 04, 2026 at 05:45:55PM -0800, Douglas Anderson wrote:
> > From: Han Wang <nesthanwang@google.com>
> >
> > AutoFDO profiles are commonly collected on one kernel revision and
> > reused across subsequent commits or stable merges before a fresh
> > profile is generated. While AutoFDO uses function-relative line
> > numbers to tolerate edits above a function, adding or removing lines
> > inside a function (or an inlined helper) shifts all subsequent
> > relative line offsets within that function. Without stale profile
> > salvaging, shifted callsites fail to match their nested inlinee
> > profiles in SampleProfileLoader (dropping profile-guided inlining for
> > those callees) and basic block sample counts are misattributed to the
> > wrong instructions, degrading branch probability estimation and
> > machine function splitting.
> >
> > Enable LLVM stale profile salvaging (-salvage-stale-profile, supported
> > since LLVM 17) for C, Rust, and ThinLTO builds when
> > CLANG_AUTOFDO_PROFILE is specified. This instructs SampleProfileMatcher
> > to use lexical callsite anchors within functions to fuzzy-match and
> > remap shifted line offsets when querying the profile.
> >
> > While -salvage-stale-profile is not enabled by default in LLVM due to
> > the minor compile-time/memory cost of sequence-matching callsite
> > anchors (which is unnecessary when profiles are freshly generated on
> > the exact commit being built) and the heuristic nature of fuzzy
> > matching on heavily rewritten functions, enabling it makes sense for
> > the kernel: hardware PMU profiles are rarely regenerated on every
> > commit, and anchoring on callsite names is strictly more resilient
> > than blind line-offset lookup when functions are edited.
> >
> > As a side benefit, remapping shifted line offsets also avoids feeding
> > mismatched weights from stale profiles into MIRProfileLoader's flow
> > inference solver (applyFlowInference), working around a compiler bug
> > [1] where a small intra-function line shift (such as in
> > __vm_normal_page() in mm/memory.c) can cause applyFlowInference to
> > spend 30+ minutes compiling unmap_page_range().
> >
> > Link: https://github.com/ClangBuiltLinux/linux/issues/2188 [1]
> > Signed-off-by: Han Wang <nesthanwang@google.com>
> > Signed-off-by: Douglas Anderson <dianders@chromium.org>
> > ---
> > Much of the commit-message justification here is AI-generated. Sorry!
> > :( Folks who know better, feel free to call bullshit.
>
> The justification does not seem wrong to me on initial read but it
> definitely feels like the first and third paragraphs could be reduced in
> verbosity since this can basically be summarized by "AutoFDO profiles
> are often reused, which can be problematic without profile salvaging
> enabled, which is designed for this exact situation".

Happy to reword or happy for upstream maintainers to reword themselves
when landing. If you want a v2 sent with shorter wording, let me know.

FWIW, I actually asked the AI to add some of this text. Specifically,
I felt it was important to explain why LLVM would not make this the
default behavior but why it was still the correct behavior for the
kernel. ;-)


> > Honestly, this patch came about because of AI analysis compiles taking
> > 30 minutes to run (see the clang bug [1]). It seems to work around the
> > problem. Obviously, we should fix the compiler. That being said, upon
> > further analysis it seemed like adding this flag was beneficial
> > anyway, so I'm posting it upstream. If folks disagree, we can simply
> > drop this patch.
>
> I would definitely like to see that issue triaged but like you say, it
> probably does not need to done before this is merged since it has
> reasonable justification on its own. I would love to get thoughts from
> other folks though.

Sounds good. Thanks for looking. Let's see what others say. :-)

-Doug

Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile

[Rong Xu]

The issue you mentioned regarding excessive time in applyFlowInference
comes from Profi.

The AutoFDO implementation used in the kernel is FS-AFDO (which uses
FS discriminators). There are known issues when using Profi with
FS-AFDO; enabling Profi actually degrades performance compared to
original profile propagation. When we tested and worked on AutoFDO for
the kernel, we did not enable Profi.

While I haven't looked into this specific bug yet and am unsure if it
is unique to FS-AFDO, I recommend using -fno-sample-profile-use-profi
to disable Profi for now.

Regarding the "--mllvm=-salvage-stale-profile" flag, I also have
reservations about adding it for the following reasons:
1.  It is heuristic-based and not enabled by default. I am unsure how
thoroughly it has been tested, so we should assess it further before
enabling it.
2.  The recommended usage model for kernel AutoFDO (as documented) is
to use the same source. While it might help in an iterative build
model, I don't see a strong reason to use it if there is no source
drift.

Best regards,

Rong

On Thu, Nov 5, 2026 at 8:24 AM Doug Anderson <dianders@chromium.org> wrote:
>
> Hi Nathan,
>
> On Wed, Nov 4, 2026 at 9:24 PM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > On Wed, Nov 04, 2026 at 05:45:55PM -0800, Douglas Anderson wrote:
> > > From: Han Wang <nesthanwang@google.com>
> > >
> > > AutoFDO profiles are commonly collected on one kernel revision and
> > > reused across subsequent commits or stable merges before a fresh
> > > profile is generated. While AutoFDO uses function-relative line
> > > numbers to tolerate edits above a function, adding or removing lines
> > > inside a function (or an inlined helper) shifts all subsequent
> > > relative line offsets within that function. Without stale profile
> > > salvaging, shifted callsites fail to match their nested inlinee
> > > profiles in SampleProfileLoader (dropping profile-guided inlining for
> > > those callees) and basic block sample counts are misattributed to the
> > > wrong instructions, degrading branch probability estimation and
> > > machine function splitting.
> > >
> > > Enable LLVM stale profile salvaging (-salvage-stale-profile, supported
> > > since LLVM 17) for C, Rust, and ThinLTO builds when
> > > CLANG_AUTOFDO_PROFILE is specified. This instructs SampleProfileMatcher
> > > to use lexical callsite anchors within functions to fuzzy-match and
> > > remap shifted line offsets when querying the profile.
> > >
> > > While -salvage-stale-profile is not enabled by default in LLVM due to
> > > the minor compile-time/memory cost of sequence-matching callsite
> > > anchors (which is unnecessary when profiles are freshly generated on
> > > the exact commit being built) and the heuristic nature of fuzzy
> > > matching on heavily rewritten functions, enabling it makes sense for
> > > the kernel: hardware PMU profiles are rarely regenerated on every
> > > commit, and anchoring on callsite names is strictly more resilient
> > > than blind line-offset lookup when functions are edited.
> > >
> > > As a side benefit, remapping shifted line offsets also avoids feeding
> > > mismatched weights from stale profiles into MIRProfileLoader's flow
> > > inference solver (applyFlowInference), working around a compiler bug
> > > [1] where a small intra-function line shift (such as in
> > > __vm_normal_page() in mm/memory.c) can cause applyFlowInference to
> > > spend 30+ minutes compiling unmap_page_range().
> > >
> > > Link: https://github.com/ClangBuiltLinux/linux/issues/2188 [1]
> > > Signed-off-by: Han Wang <nesthanwang@google.com>
> > > Signed-off-by: Douglas Anderson <dianders@chromium.org>
> > > ---
> > > Much of the commit-message justification here is AI-generated. Sorry!
> > > :( Folks who know better, feel free to call bullshit.
> >
> > The justification does not seem wrong to me on initial read but it
> > definitely feels like the first and third paragraphs could be reduced in
> > verbosity since this can basically be summarized by "AutoFDO profiles
> > are often reused, which can be problematic without profile salvaging
> > enabled, which is designed for this exact situation".
>
> Happy to reword or happy for upstream maintainers to reword themselves
> when landing. If you want a v2 sent with shorter wording, let me know.
>
> FWIW, I actually asked the AI to add some of this text. Specifically,
> I felt it was important to explain why LLVM would not make this the
> default behavior but why it was still the correct behavior for the
> kernel. ;-)
>
>
> > > Honestly, this patch came about because of AI analysis compiles taking
> > > 30 minutes to run (see the clang bug [1]). It seems to work around the
> > > problem. Obviously, we should fix the compiler. That being said, upon
> > > further analysis it seemed like adding this flag was beneficial
> > > anyway, so I'm posting it upstream. If folks disagree, we can simply
> > > drop this patch.
> >
> > I would definitely like to see that issue triaged but like you say, it
> > probably does not need to done before this is merged since it has
> > reasonable justification on its own. I would love to get thoughts from
> > other folks though.
>
> Sounds good. Thanks for looking. Let's see what others say. :-)
>
> -Doug

Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile

[Miguel Ojeda]

On Thu, Nov 5, 2026 at 2:46 AM Douglas Anderson <dianders@chromium.org> wrote:
>
> Enable LLVM stale profile salvaging (-salvage-stale-profile, supported
> since LLVM 17) for C, Rust, and ThinLTO builds when

> +  RUSTFLAGS_AUTOFDO_CLANG += -Cllvm-args=-salvage-stale-profile

I double-checked that the flag was introduced in LLVM 17 as the cover
letter mentions, and so it is likely fine given the Rust 1.85.0
compiler (i.e. our minimum) supports LLVM 18 as the minimum major
version, but if the patch goes ahead in the end, then we should make
sure it works well with that one (or probably you already did, in
which case, please ignore this!).

Cheers,
Miguel

Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile

[Doug Anderson]

Hi,

On Thu, Nov 5, 2026 at 10:23 AM Rong Xu <xur@google.com> wrote:
>
> The issue you mentioned regarding excessive time in applyFlowInference
> comes from Profi.
>
> The AutoFDO implementation used in the kernel is FS-AFDO (which uses
> FS discriminators). There are known issues when using Profi with
> FS-AFDO; enabling Profi actually degrades performance compared to
> original profile propagation. When we tested and worked on AutoFDO for
> the kernel, we did not enable Profi.
>
> While I haven't looked into this specific bug yet and am unsure if it
> is unique to FS-AFDO, I recommend using -fno-sample-profile-use-profi
> to disable Profi for now.

Interesting. I can suggest that to the team digging into it and see if
it solves the problem. My AI agent says it will though. It claims that
the default flipped recently and "-fno-sample-profile-use-profi" will
restore the old (tested) behavior.

Should I send a patch adding that flag?


> Regarding the "--mllvm=-salvage-stale-profile" flag, I also have
> reservations about adding it for the following reasons:
> 1.  It is heuristic-based and not enabled by default. I am unsure how
> thoroughly it has been tested, so we should assess it further before
> enabling it.

FWIW, we did find that Android seems to enable this globally across
its userspace, which suggests it has been fairly well tested. You can
see the change here, which landed about 1.5 years ago.

https://android-review.git.corp.google.com/c/platform/build/soong/+/3524291

A few notes:

* That change _also_ adds
"--salvage-stale-profile-max-callsites=2000". Internally, there has
been debate about whether adding that would be a good idea, too.

* This kernel change _also_ adds the flag to rust compiling.


> 2.  The recommended usage model for kernel AutoFDO (as documented) is
> to use the same source. While it might help in an iterative build
> model, I don't see a strong reason to use it if there is no source
> drift.

Hmmm. I guess in the purist sense we'd want the AudoFDO profile to be
for the exact source. ...but from my experience, that's not what
happens in reality. If nothing else, developers locally working on the
kernel still want to keep the AutoFDO optimizations so they're testing
as close to what's in the field as possible. However, the act of
working on the kernel will touch _some_ of the source. Keeping AFDO
working as much as possible seems nice to me. Of course, I'm by no
means an AutoFDO expert and mostly just got dragged into this issue,
so happy to defer to the experts. ;-)

-Doug

Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile

[Doug Anderson]

Hi,

On Thu, Nov 5, 2026 at 11:47 AM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> On Thu, Nov 5, 2026 at 2:46 AM Douglas Anderson <dianders@chromium.org> wrote:
> >
> > Enable LLVM stale profile salvaging (-salvage-stale-profile, supported
> > since LLVM 17) for C, Rust, and ThinLTO builds when
>
> > +  RUSTFLAGS_AUTOFDO_CLANG += -Cllvm-args=-salvage-stale-profile
>
> I double-checked that the flag was introduced in LLVM 17 as the cover
> letter mentions, and so it is likely fine given the Rust 1.85.0
> compiler (i.e. our minimum) supports LLVM 18 as the minimum major
> version, but if the patch goes ahead in the end, then we should make
> sure it works well with that one (or probably you already did, in
> which case, please ignore this!).

I haven't extensively tested the behavior across many versions of
rust. If you are worried, I guess we could remove the rustflags part
of the change? ...or, of course, we could just fully drop this patch
(and maybe add `-fno-sample-profile-use-profi` instead) ?

-Doug

Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile

[Nick Desaulniers]

On Thu, Nov 5, 2026 at 1:30 PM Doug Anderson <dianders@chromium.org> wrote:
>
> Hi,
>
> On Thu, Nov 5, 2026 at 10:23 AM Rong Xu <xur@google.com> wrote:
> >
> > 2.  The recommended usage model for kernel AutoFDO (as documented) is
> > to use the same source. While it might help in an iterative build
> > model, I don't see a strong reason to use it if there is no source
> > drift.
>
> Hmmm. I guess in the purist sense we'd want the AudoFDO profile to be
> for the exact source. ...but from my experience, that's not what
> happens in reality. If nothing else, developers locally working on the

Right. Rong's recommendation makes sense from a google3 perspective
(who I suspect is continuously regenerating profile data), but isn't
feasible for Android or ChromeOS (who are not).

> kernel still want to keep the AutoFDO optimizations so they're testing
> as close to what's in the field as possible. However, the act of
> working on the kernel will touch _some_ of the source. Keeping AFDO
> working as much as possible seems nice to me. Of course, I'm by no
> means an AutoFDO expert and mostly just got dragged into this issue,
> so happy to defer to the experts. ;-)
>
> -Doug



-- 
Thanks,
~Nick Desaulniers

Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile

[Nick Desaulniers]

On Thu, Nov 5, 2026 at 1:30 PM Doug Anderson <dianders@chromium.org> wrote:
>
> Hi,
>
> On Thu, Nov 5, 2026 at 10:23 AM Rong Xu <xur@google.com> wrote:
> >
> > The issue you mentioned regarding excessive time in applyFlowInference
> > comes from Profi.
> >
> > The AutoFDO implementation used in the kernel is FS-AFDO (which uses
> > FS discriminators). There are known issues when using Profi with
> > FS-AFDO; enabling Profi actually degrades performance compared to
> > original profile propagation. When we tested and worked on AutoFDO for
> > the kernel, we did not enable Profi.
> >
> > While I haven't looked into this specific bug yet and am unsure if it
> > is unique to FS-AFDO, I recommend using -fno-sample-profile-use-profi
> > to disable Profi for now.
>
> Interesting. I can suggest that to the team digging into it and see if
> it solves the problem. My AI agent says it will though. It claims that
> the default flipped recently and "-fno-sample-profile-use-profi" will

I wonder who flipped the default? Perhaps they're not aware of _all_
of the existing users of AutoFDO, such as the Linux kernel.

> restore the old (tested) behavior.
>
> Should I send a patch adding that flag?

-- 
Thanks,
~Nick Desaulniers

Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile

[Doug Anderson]

Hi,

On Thu, Nov 5, 2026 at 2:40 PM Nick Desaulniers <ndesaulniers@google.com> wrote:
>
> On Thu, Nov 5, 2026 at 1:30 PM Doug Anderson <dianders@chromium.org> wrote:
> >
> > Hi,
> >
> > On Thu, Nov 5, 2026 at 10:23 AM Rong Xu <xur@google.com> wrote:
> > >
> > > The issue you mentioned regarding excessive time in applyFlowInference
> > > comes from Profi.
> > >
> > > The AutoFDO implementation used in the kernel is FS-AFDO (which uses
> > > FS discriminators). There are known issues when using Profi with
> > > FS-AFDO; enabling Profi actually degrades performance compared to
> > > original profile propagation. When we tested and worked on AutoFDO for
> > > the kernel, we did not enable Profi.
> > >
> > > While I haven't looked into this specific bug yet and am unsure if it
> > > is unique to FS-AFDO, I recommend using -fno-sample-profile-use-profi
> > > to disable Profi for now.
> >
> > Interesting. I can suggest that to the team digging into it and see if
> > it solves the problem. My AI agent says it will though. It claims that
> > the default flipped recently and "-fno-sample-profile-use-profi" will
>
> I wonder who flipped the default? Perhaps they're not aware of _all_
> of the existing users of AutoFDO, such as the Linux kernel.

I've been preparing a patch to add -fno-sample-profile-use-profi. As
part of that, my commit message will include this:

Starting in Clang 21 (LLVM commit 2fc4a4a9d3a8 ("[Driver][SamplePGO]
Enable -fsample-profile-use-profi (#146795)") [1]), the Clang driver
automatically passes -mllvm -sample-profile-use-profi whenever
-fprofile-sample-use is specified, enabling LLVM's flow-based profile
inference algorithm (Profi) by default.

Link: https://github.com/llvm/llvm-project/commit/2fc4a4a9d3a87a9e11c48ef25522aa17c2a71003
[1]

-Doug

Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile

[Doug Anderson]

Hi,

On Thu, Nov 5, 2026 at 1:30 PM Doug Anderson <dianders@chromium.org> wrote:
>
> > While I haven't looked into this specific bug yet and am unsure if it
> > is unique to FS-AFDO, I recommend using -fno-sample-profile-use-profi
> > to disable Profi for now.
>
> Interesting. I can suggest that to the team digging into it and see if
> it solves the problem. My AI agent says it will though. It claims that
> the default flipped recently and "-fno-sample-profile-use-profi" will
> restore the old (tested) behavior.
>
> Should I send a patch adding that flag?

I went ahead and posted the patch:

https://lore.kernel.org/r/20261105151159.1.I0b27923381ef34cc99cdf8b65fea2e1bafce6370@changeid

Obviously that patch and this patch conflict because they touch the
same lines of code. If folks think we should still move forward with
both patches, I'm happy to rebase.


> > Regarding the "--mllvm=-salvage-stale-profile" flag, I also have
> > reservations about adding it for the following reasons:
> > 1.  It is heuristic-based and not enabled by default. I am unsure how
> > thoroughly it has been tested, so we should assess it further before
> > enabling it.
>
> FWIW, we did find that Android seems to enable this globally across
> its userspace, which suggests it has been fairly well tested. You can
> see the change here, which landed about 1.5 years ago.
>
> https://android-review.git.corp.google.com/c/platform/build/soong/+/3524291
>
> A few notes:
>
> * That change _also_ adds
> "--salvage-stale-profile-max-callsites=2000". Internally, there has
> been debate about whether adding that would be a good idea, too.

The conclusion of the internal debate is that, yes, we should add
`--salvage-stale-profile-max-callsites=2000` too. So if we decide to
move forward with this patch, I'll send a v2 that adds that flag as
well. For now I'll hold off until there is consensus. :-)

-Doug

Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile

[Nick Desaulniers]

On Thu, Nov 5, 2026 at 3:17 PM Doug Anderson <dianders@chromium.org> wrote:
>
> Hi,
>
> On Thu, Nov 5, 2026 at 1:30 PM Doug Anderson <dianders@chromium.org> wrote:
> >
> > > While I haven't looked into this specific bug yet and am unsure if it
> > > is unique to FS-AFDO, I recommend using -fno-sample-profile-use-profi
> > > to disable Profi for now.
> >
> > Interesting. I can suggest that to the team digging into it and see if
> > it solves the problem. My AI agent says it will though. It claims that
> > the default flipped recently and "-fno-sample-profile-use-profi" will
> > restore the old (tested) behavior.
> >
> > Should I send a patch adding that flag?
>
> I went ahead and posted the patch:
>
> https://lore.kernel.org/r/20261105151159.1.I0b27923381ef34cc99cdf8b65fea2e1bafce6370@changeid
>
> Obviously that patch and this patch conflict because they touch the
> same lines of code. If folks think we should still move forward with
> both patches, I'm happy to rebase.
>
>
> > > Regarding the "--mllvm=-salvage-stale-profile" flag, I also have
> > > reservations about adding it for the following reasons:
> > > 1.  It is heuristic-based and not enabled by default. I am unsure how
> > > thoroughly it has been tested, so we should assess it further before
> > > enabling it.
> >
> > FWIW, we did find that Android seems to enable this globally across
> > its userspace, which suggests it has been fairly well tested. You can
> > see the change here, which landed about 1.5 years ago.
> >
> > https://android-review.git.corp.google.com/c/platform/build/soong/+/3524291
> >
> > A few notes:
> >
> > * That change _also_ adds
> > "--salvage-stale-profile-max-callsites=2000". Internally, there has
> > been debate about whether adding that would be a good idea, too.
>
> The conclusion of the internal debate is that, yes, we should add

After Doug sent this email, him, I, Yabin, Rong, and a few others met
and decided we probably want

-fno-sample-profile-use-profi

...but then I stepped out of that meeting and ran into Yi who
extensively tested AutoFDO for Android userspace and suggested _the
opposite_.

i.e. Rong suggested:
yes: -fno-sample-profile-use-profi
no: --mllvm=-salvage-stale-profile

Yi suggested:
no: -fno-sample-profile-use-profi
yes: --mllvm=-salvage-stale-profile

Yi, can you share with the list why you hold that opinion?

I suspect we'll want to disable "profi" for now, but likely will end
up revisiting these pairs of flags in the near future.

> `--salvage-stale-profile-max-callsites=2000` too. So if we decide to
> move forward with this patch, I'll send a v2 that adds that flag as
> well. For now I'll hold off until there is consensus. :-)
>
> -Doug



-- 
Thanks,
~Nick Desaulniers