* [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile @ 2026-11-05 1:45 Douglas Anderson 2026-11-05 5:24 ` Nathan Chancellor 2026-11-05 19:47 ` Miguel Ojeda 0 siblings, 2 replies; 11+ messages in thread From: Douglas Anderson @ 2026-11-05 1:45 UTC (permalink / raw) To: llvm, linux-kbuild, Rong Xu, Han Shen, Nathan Chancellor, Nicolas Schier, Nick Desaulniers Cc: Han Wang, Douglas Anderson, Alexandre Courbot, Alice Ryhl, Andreas Hindborg, Benno Lossin, Bill Wendling, Björn Roy Baron, Boqun Feng, Daniel Almeida, Danilo Krummrich, Gary Guo, Justin Stitt, Miguel Ojeda, Onur Özkan, Tamir Duberstein, Trevor Gross, linux-kernel, rust-for-linux From: Han Wang <nesthanwang@google.com> AutoFDO profiles are commonly collected on one kernel revision and reused across subsequent commits or stable merges before a fresh profile is generated. While AutoFDO uses function-relative line numbers to tolerate edits above a function, adding or removing lines inside a function (or an inlined helper) shifts all subsequent relative line offsets within that function. Without stale profile salvaging, shifted callsites fail to match their nested inlinee profiles in SampleProfileLoader (dropping profile-guided inlining for those callees) and basic block sample counts are misattributed to the wrong instructions, degrading branch probability estimation and machine function splitting. Enable LLVM stale profile salvaging (-salvage-stale-profile, supported since LLVM 17) for C, Rust, and ThinLTO builds when CLANG_AUTOFDO_PROFILE is specified. This instructs SampleProfileMatcher to use lexical callsite anchors within functions to fuzzy-match and remap shifted line offsets when querying the profile. While -salvage-stale-profile is not enabled by default in LLVM due to the minor compile-time/memory cost of sequence-matching callsite anchors (which is unnecessary when profiles are freshly generated on the exact commit being built) and the heuristic nature of fuzzy matching on heavily rewritten functions, enabling it makes sense for the kernel: hardware PMU profiles are rarely regenerated on every commit, and anchoring on callsite names is strictly more resilient than blind line-offset lookup when functions are edited. As a side benefit, remapping shifted line offsets also avoids feeding mismatched weights from stale profiles into MIRProfileLoader's flow inference solver (applyFlowInference), working around a compiler bug [1] where a small intra-function line shift (such as in __vm_normal_page() in mm/memory.c) can cause applyFlowInference to spend 30+ minutes compiling unmap_page_range(). Link: https://github.com/ClangBuiltLinux/linux/issues/2188 [1] Signed-off-by: Han Wang <nesthanwang@google.com> Signed-off-by: Douglas Anderson <dianders@chromium.org> --- Much of the commit-message justification here is AI-generated. Sorry! :( Folks who know better, feel free to call bullshit. Honestly, this patch came about because of AI analysis compiles taking 30 minutes to run (see the clang bug [1]). It seems to work around the problem. Obviously, we should fix the compiler. That being said, upon further analysis it seemed like adding this flag was beneficial anyway, so I'm posting it upstream. If folks disagree, we can simply drop this patch. scripts/Makefile.autofdo | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scripts/Makefile.autofdo b/scripts/Makefile.autofdo index 57a3aba6ffc2..3b1e65a13ffa 100644 --- a/scripts/Makefile.autofdo +++ b/scripts/Makefile.autofdo @@ -13,13 +13,16 @@ endif ifdef CLANG_AUTOFDO_PROFILE CFLAGS_AUTOFDO_CLANG += -fprofile-sample-use=$(CLANG_AUTOFDO_PROFILE) -ffunction-sections CFLAGS_AUTOFDO_CLANG += -fsplit-machine-functions + CFLAGS_AUTOFDO_CLANG += -mllvm -salvage-stale-profile RUSTFLAGS_AUTOFDO_CLANG += -Zprofile-sample-use=$(CLANG_AUTOFDO_PROFILE) -Zfunction-sections=y RUSTFLAGS_AUTOFDO_CLANG += -Cllvm-args=-split-machine-functions + RUSTFLAGS_AUTOFDO_CLANG += -Cllvm-args=-salvage-stale-profile endif ifdef CONFIG_LTO_CLANG_THIN ifdef CLANG_AUTOFDO_PROFILE KBUILD_LDFLAGS += --lto-sample-profile=$(CLANG_AUTOFDO_PROFILE) + KBUILD_LDFLAGS += --mllvm=-salvage-stale-profile endif KBUILD_LDFLAGS += --mllvm=-enable-fs-discriminator=true --mllvm=-improved-fs-discriminator=true -plugin-opt=thinlto KBUILD_LDFLAGS += $(call ld-option,-plugin-opt=-function-splitting=all,-plugin-opt=-split-machine-functions) -- 2.56.0.455.g6c5a503b38-goog ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile 2026-11-05 1:45 [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile Douglas Anderson @ 2026-11-05 5:24 ` Nathan Chancellor 2026-11-05 16:24 ` Doug Anderson 2026-11-05 19:47 ` Miguel Ojeda 1 sibling, 1 reply; 11+ messages in thread From: Nathan Chancellor @ 2026-11-05 5:24 UTC (permalink / raw) To: Douglas Anderson Cc: llvm, linux-kbuild, Rong Xu, Han Shen, Nicolas Schier, Nick Desaulniers, Han Wang, Alexandre Courbot, Alice Ryhl, Andreas Hindborg, Benno Lossin, Bill Wendling, Björn Roy Baron, Boqun Feng, Daniel Almeida, Danilo Krummrich, Gary Guo, Justin Stitt, Miguel Ojeda, Onur Özkan, Tamir Duberstein, Trevor Gross, linux-kernel, rust-for-linux On Wed, Nov 04, 2026 at 05:45:55PM -0800, Douglas Anderson wrote: > From: Han Wang <nesthanwang@google.com> > > AutoFDO profiles are commonly collected on one kernel revision and > reused across subsequent commits or stable merges before a fresh > profile is generated. While AutoFDO uses function-relative line > numbers to tolerate edits above a function, adding or removing lines > inside a function (or an inlined helper) shifts all subsequent > relative line offsets within that function. Without stale profile > salvaging, shifted callsites fail to match their nested inlinee > profiles in SampleProfileLoader (dropping profile-guided inlining for > those callees) and basic block sample counts are misattributed to the > wrong instructions, degrading branch probability estimation and > machine function splitting. > > Enable LLVM stale profile salvaging (-salvage-stale-profile, supported > since LLVM 17) for C, Rust, and ThinLTO builds when > CLANG_AUTOFDO_PROFILE is specified. This instructs SampleProfileMatcher > to use lexical callsite anchors within functions to fuzzy-match and > remap shifted line offsets when querying the profile. > > While -salvage-stale-profile is not enabled by default in LLVM due to > the minor compile-time/memory cost of sequence-matching callsite > anchors (which is unnecessary when profiles are freshly generated on > the exact commit being built) and the heuristic nature of fuzzy > matching on heavily rewritten functions, enabling it makes sense for > the kernel: hardware PMU profiles are rarely regenerated on every > commit, and anchoring on callsite names is strictly more resilient > than blind line-offset lookup when functions are edited. > > As a side benefit, remapping shifted line offsets also avoids feeding > mismatched weights from stale profiles into MIRProfileLoader's flow > inference solver (applyFlowInference), working around a compiler bug > [1] where a small intra-function line shift (such as in > __vm_normal_page() in mm/memory.c) can cause applyFlowInference to > spend 30+ minutes compiling unmap_page_range(). > > Link: https://github.com/ClangBuiltLinux/linux/issues/2188 [1] > Signed-off-by: Han Wang <nesthanwang@google.com> > Signed-off-by: Douglas Anderson <dianders@chromium.org> > --- > Much of the commit-message justification here is AI-generated. Sorry! > :( Folks who know better, feel free to call bullshit. The justification does not seem wrong to me on initial read but it definitely feels like the first and third paragraphs could be reduced in verbosity since this can basically be summarized by "AutoFDO profiles are often reused, which can be problematic without profile salvaging enabled, which is designed for this exact situation". > Honestly, this patch came about because of AI analysis compiles taking > 30 minutes to run (see the clang bug [1]). It seems to work around the > problem. Obviously, we should fix the compiler. That being said, upon > further analysis it seemed like adding this flag was beneficial > anyway, so I'm posting it upstream. If folks disagree, we can simply > drop this patch. I would definitely like to see that issue triaged but like you say, it probably does not need to done before this is merged since it has reasonable justification on its own. I would love to get thoughts from other folks though. > scripts/Makefile.autofdo | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/scripts/Makefile.autofdo b/scripts/Makefile.autofdo > index 57a3aba6ffc2..3b1e65a13ffa 100644 > --- a/scripts/Makefile.autofdo > +++ b/scripts/Makefile.autofdo > @@ -13,13 +13,16 @@ endif > ifdef CLANG_AUTOFDO_PROFILE > CFLAGS_AUTOFDO_CLANG += -fprofile-sample-use=$(CLANG_AUTOFDO_PROFILE) -ffunction-sections > CFLAGS_AUTOFDO_CLANG += -fsplit-machine-functions > + CFLAGS_AUTOFDO_CLANG += -mllvm -salvage-stale-profile > RUSTFLAGS_AUTOFDO_CLANG += -Zprofile-sample-use=$(CLANG_AUTOFDO_PROFILE) -Zfunction-sections=y > RUSTFLAGS_AUTOFDO_CLANG += -Cllvm-args=-split-machine-functions > + RUSTFLAGS_AUTOFDO_CLANG += -Cllvm-args=-salvage-stale-profile > endif > > ifdef CONFIG_LTO_CLANG_THIN > ifdef CLANG_AUTOFDO_PROFILE > KBUILD_LDFLAGS += --lto-sample-profile=$(CLANG_AUTOFDO_PROFILE) > + KBUILD_LDFLAGS += --mllvm=-salvage-stale-profile > endif > KBUILD_LDFLAGS += --mllvm=-enable-fs-discriminator=true --mllvm=-improved-fs-discriminator=true -plugin-opt=thinlto > KBUILD_LDFLAGS += $(call ld-option,-plugin-opt=-function-splitting=all,-plugin-opt=-split-machine-functions) > -- > 2.56.0.455.g6c5a503b38-goog > -- Cheers, Nathan ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile 2026-11-05 5:24 ` Nathan Chancellor @ 2026-11-05 16:24 ` Doug Anderson 2026-11-05 18:23 ` Rong Xu 0 siblings, 1 reply; 11+ messages in thread From: Doug Anderson @ 2026-11-05 16:24 UTC (permalink / raw) To: Nathan Chancellor Cc: llvm, linux-kbuild, Rong Xu, Han Shen, Nicolas Schier, Nick Desaulniers, Han Wang, Alexandre Courbot, Alice Ryhl, Andreas Hindborg, Benno Lossin, Bill Wendling, Björn Roy Baron, Boqun Feng, Daniel Almeida, Danilo Krummrich, Gary Guo, Justin Stitt, Miguel Ojeda, Onur Özkan, Tamir Duberstein, Trevor Gross, linux-kernel, rust-for-linux Hi Nathan, On Wed, Nov 4, 2026 at 9:24 PM Nathan Chancellor <nathan@kernel.org> wrote: > > On Wed, Nov 04, 2026 at 05:45:55PM -0800, Douglas Anderson wrote: > > From: Han Wang <nesthanwang@google.com> > > > > AutoFDO profiles are commonly collected on one kernel revision and > > reused across subsequent commits or stable merges before a fresh > > profile is generated. While AutoFDO uses function-relative line > > numbers to tolerate edits above a function, adding or removing lines > > inside a function (or an inlined helper) shifts all subsequent > > relative line offsets within that function. Without stale profile > > salvaging, shifted callsites fail to match their nested inlinee > > profiles in SampleProfileLoader (dropping profile-guided inlining for > > those callees) and basic block sample counts are misattributed to the > > wrong instructions, degrading branch probability estimation and > > machine function splitting. > > > > Enable LLVM stale profile salvaging (-salvage-stale-profile, supported > > since LLVM 17) for C, Rust, and ThinLTO builds when > > CLANG_AUTOFDO_PROFILE is specified. This instructs SampleProfileMatcher > > to use lexical callsite anchors within functions to fuzzy-match and > > remap shifted line offsets when querying the profile. > > > > While -salvage-stale-profile is not enabled by default in LLVM due to > > the minor compile-time/memory cost of sequence-matching callsite > > anchors (which is unnecessary when profiles are freshly generated on > > the exact commit being built) and the heuristic nature of fuzzy > > matching on heavily rewritten functions, enabling it makes sense for > > the kernel: hardware PMU profiles are rarely regenerated on every > > commit, and anchoring on callsite names is strictly more resilient > > than blind line-offset lookup when functions are edited. > > > > As a side benefit, remapping shifted line offsets also avoids feeding > > mismatched weights from stale profiles into MIRProfileLoader's flow > > inference solver (applyFlowInference), working around a compiler bug > > [1] where a small intra-function line shift (such as in > > __vm_normal_page() in mm/memory.c) can cause applyFlowInference to > > spend 30+ minutes compiling unmap_page_range(). > > > > Link: https://github.com/ClangBuiltLinux/linux/issues/2188 [1] > > Signed-off-by: Han Wang <nesthanwang@google.com> > > Signed-off-by: Douglas Anderson <dianders@chromium.org> > > --- > > Much of the commit-message justification here is AI-generated. Sorry! > > :( Folks who know better, feel free to call bullshit. > > The justification does not seem wrong to me on initial read but it > definitely feels like the first and third paragraphs could be reduced in > verbosity since this can basically be summarized by "AutoFDO profiles > are often reused, which can be problematic without profile salvaging > enabled, which is designed for this exact situation". Happy to reword or happy for upstream maintainers to reword themselves when landing. If you want a v2 sent with shorter wording, let me know. FWIW, I actually asked the AI to add some of this text. Specifically, I felt it was important to explain why LLVM would not make this the default behavior but why it was still the correct behavior for the kernel. ;-) > > Honestly, this patch came about because of AI analysis compiles taking > > 30 minutes to run (see the clang bug [1]). It seems to work around the > > problem. Obviously, we should fix the compiler. That being said, upon > > further analysis it seemed like adding this flag was beneficial > > anyway, so I'm posting it upstream. If folks disagree, we can simply > > drop this patch. > > I would definitely like to see that issue triaged but like you say, it > probably does not need to done before this is merged since it has > reasonable justification on its own. I would love to get thoughts from > other folks though. Sounds good. Thanks for looking. Let's see what others say. :-) -Doug ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile 2026-11-05 16:24 ` Doug Anderson @ 2026-11-05 18:23 ` Rong Xu 2026-11-05 21:30 ` Doug Anderson 0 siblings, 1 reply; 11+ messages in thread From: Rong Xu @ 2026-11-05 18:23 UTC (permalink / raw) To: Doug Anderson Cc: Nathan Chancellor, llvm, linux-kbuild, Han Shen, Nicolas Schier, Nick Desaulniers, Han Wang, Alexandre Courbot, Alice Ryhl, Andreas Hindborg, Benno Lossin, Bill Wendling, Björn Roy Baron, Boqun Feng, Daniel Almeida, Danilo Krummrich, Gary Guo, Justin Stitt, Miguel Ojeda, Onur Özkan, Tamir Duberstein, Trevor Gross, linux-kernel, rust-for-linux The issue you mentioned regarding excessive time in applyFlowInference comes from Profi. The AutoFDO implementation used in the kernel is FS-AFDO (which uses FS discriminators). There are known issues when using Profi with FS-AFDO; enabling Profi actually degrades performance compared to original profile propagation. When we tested and worked on AutoFDO for the kernel, we did not enable Profi. While I haven't looked into this specific bug yet and am unsure if it is unique to FS-AFDO, I recommend using -fno-sample-profile-use-profi to disable Profi for now. Regarding the "--mllvm=-salvage-stale-profile" flag, I also have reservations about adding it for the following reasons: 1. It is heuristic-based and not enabled by default. I am unsure how thoroughly it has been tested, so we should assess it further before enabling it. 2. The recommended usage model for kernel AutoFDO (as documented) is to use the same source. While it might help in an iterative build model, I don't see a strong reason to use it if there is no source drift. Best regards, Rong On Thu, Nov 5, 2026 at 8:24 AM Doug Anderson <dianders@chromium.org> wrote: > > Hi Nathan, > > On Wed, Nov 4, 2026 at 9:24 PM Nathan Chancellor <nathan@kernel.org> wrote: > > > > On Wed, Nov 04, 2026 at 05:45:55PM -0800, Douglas Anderson wrote: > > > From: Han Wang <nesthanwang@google.com> > > > > > > AutoFDO profiles are commonly collected on one kernel revision and > > > reused across subsequent commits or stable merges before a fresh > > > profile is generated. While AutoFDO uses function-relative line > > > numbers to tolerate edits above a function, adding or removing lines > > > inside a function (or an inlined helper) shifts all subsequent > > > relative line offsets within that function. Without stale profile > > > salvaging, shifted callsites fail to match their nested inlinee > > > profiles in SampleProfileLoader (dropping profile-guided inlining for > > > those callees) and basic block sample counts are misattributed to the > > > wrong instructions, degrading branch probability estimation and > > > machine function splitting. > > > > > > Enable LLVM stale profile salvaging (-salvage-stale-profile, supported > > > since LLVM 17) for C, Rust, and ThinLTO builds when > > > CLANG_AUTOFDO_PROFILE is specified. This instructs SampleProfileMatcher > > > to use lexical callsite anchors within functions to fuzzy-match and > > > remap shifted line offsets when querying the profile. > > > > > > While -salvage-stale-profile is not enabled by default in LLVM due to > > > the minor compile-time/memory cost of sequence-matching callsite > > > anchors (which is unnecessary when profiles are freshly generated on > > > the exact commit being built) and the heuristic nature of fuzzy > > > matching on heavily rewritten functions, enabling it makes sense for > > > the kernel: hardware PMU profiles are rarely regenerated on every > > > commit, and anchoring on callsite names is strictly more resilient > > > than blind line-offset lookup when functions are edited. > > > > > > As a side benefit, remapping shifted line offsets also avoids feeding > > > mismatched weights from stale profiles into MIRProfileLoader's flow > > > inference solver (applyFlowInference), working around a compiler bug > > > [1] where a small intra-function line shift (such as in > > > __vm_normal_page() in mm/memory.c) can cause applyFlowInference to > > > spend 30+ minutes compiling unmap_page_range(). > > > > > > Link: https://github.com/ClangBuiltLinux/linux/issues/2188 [1] > > > Signed-off-by: Han Wang <nesthanwang@google.com> > > > Signed-off-by: Douglas Anderson <dianders@chromium.org> > > > --- > > > Much of the commit-message justification here is AI-generated. Sorry! > > > :( Folks who know better, feel free to call bullshit. > > > > The justification does not seem wrong to me on initial read but it > > definitely feels like the first and third paragraphs could be reduced in > > verbosity since this can basically be summarized by "AutoFDO profiles > > are often reused, which can be problematic without profile salvaging > > enabled, which is designed for this exact situation". > > Happy to reword or happy for upstream maintainers to reword themselves > when landing. If you want a v2 sent with shorter wording, let me know. > > FWIW, I actually asked the AI to add some of this text. Specifically, > I felt it was important to explain why LLVM would not make this the > default behavior but why it was still the correct behavior for the > kernel. ;-) > > > > > Honestly, this patch came about because of AI analysis compiles taking > > > 30 minutes to run (see the clang bug [1]). It seems to work around the > > > problem. Obviously, we should fix the compiler. That being said, upon > > > further analysis it seemed like adding this flag was beneficial > > > anyway, so I'm posting it upstream. If folks disagree, we can simply > > > drop this patch. > > > > I would definitely like to see that issue triaged but like you say, it > > probably does not need to done before this is merged since it has > > reasonable justification on its own. I would love to get thoughts from > > other folks though. > > Sounds good. Thanks for looking. Let's see what others say. :-) > > -Doug ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile 2026-11-05 18:23 ` Rong Xu @ 2026-11-05 21:30 ` Doug Anderson 2026-11-05 22:37 ` Nick Desaulniers ` (2 more replies) 0 siblings, 3 replies; 11+ messages in thread From: Doug Anderson @ 2026-11-05 21:30 UTC (permalink / raw) To: Rong Xu Cc: Nathan Chancellor, llvm, linux-kbuild, Han Shen, Nicolas Schier, Nick Desaulniers, Han Wang, Alexandre Courbot, Alice Ryhl, Andreas Hindborg, Benno Lossin, Bill Wendling, Björn Roy Baron, Boqun Feng, Daniel Almeida, Danilo Krummrich, Gary Guo, Justin Stitt, Miguel Ojeda, Onur Özkan, Tamir Duberstein, Trevor Gross, linux-kernel, rust-for-linux Hi, On Thu, Nov 5, 2026 at 10:23 AM Rong Xu <xur@google.com> wrote: > > The issue you mentioned regarding excessive time in applyFlowInference > comes from Profi. > > The AutoFDO implementation used in the kernel is FS-AFDO (which uses > FS discriminators). There are known issues when using Profi with > FS-AFDO; enabling Profi actually degrades performance compared to > original profile propagation. When we tested and worked on AutoFDO for > the kernel, we did not enable Profi. > > While I haven't looked into this specific bug yet and am unsure if it > is unique to FS-AFDO, I recommend using -fno-sample-profile-use-profi > to disable Profi for now. Interesting. I can suggest that to the team digging into it and see if it solves the problem. My AI agent says it will though. It claims that the default flipped recently and "-fno-sample-profile-use-profi" will restore the old (tested) behavior. Should I send a patch adding that flag? > Regarding the "--mllvm=-salvage-stale-profile" flag, I also have > reservations about adding it for the following reasons: > 1. It is heuristic-based and not enabled by default. I am unsure how > thoroughly it has been tested, so we should assess it further before > enabling it. FWIW, we did find that Android seems to enable this globally across its userspace, which suggests it has been fairly well tested. You can see the change here, which landed about 1.5 years ago. https://android-review.git.corp.google.com/c/platform/build/soong/+/3524291 A few notes: * That change _also_ adds "--salvage-stale-profile-max-callsites=2000". Internally, there has been debate about whether adding that would be a good idea, too. * This kernel change _also_ adds the flag to rust compiling. > 2. The recommended usage model for kernel AutoFDO (as documented) is > to use the same source. While it might help in an iterative build > model, I don't see a strong reason to use it if there is no source > drift. Hmmm. I guess in the purist sense we'd want the AudoFDO profile to be for the exact source. ...but from my experience, that's not what happens in reality. If nothing else, developers locally working on the kernel still want to keep the AutoFDO optimizations so they're testing as close to what's in the field as possible. However, the act of working on the kernel will touch _some_ of the source. Keeping AFDO working as much as possible seems nice to me. Of course, I'm by no means an AutoFDO expert and mostly just got dragged into this issue, so happy to defer to the experts. ;-) -Doug ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile 2026-11-05 21:30 ` Doug Anderson @ 2026-11-05 22:37 ` Nick Desaulniers 2026-11-05 22:40 ` Nick Desaulniers 2026-11-05 23:16 ` Doug Anderson 2 siblings, 0 replies; 11+ messages in thread From: Nick Desaulniers @ 2026-11-05 22:37 UTC (permalink / raw) To: Doug Anderson Cc: Rong Xu, Nathan Chancellor, llvm, linux-kbuild, Han Shen, Nicolas Schier, Han Wang, Alexandre Courbot, Alice Ryhl, Andreas Hindborg, Benno Lossin, Bill Wendling, Björn Roy Baron, Boqun Feng, Daniel Almeida, Danilo Krummrich, Gary Guo, Justin Stitt, Miguel Ojeda, Onur Özkan, Tamir Duberstein, Trevor Gross, linux-kernel, rust-for-linux, Yi Kong On Thu, Nov 5, 2026 at 1:30 PM Doug Anderson <dianders@chromium.org> wrote: > > Hi, > > On Thu, Nov 5, 2026 at 10:23 AM Rong Xu <xur@google.com> wrote: > > > > 2. The recommended usage model for kernel AutoFDO (as documented) is > > to use the same source. While it might help in an iterative build > > model, I don't see a strong reason to use it if there is no source > > drift. > > Hmmm. I guess in the purist sense we'd want the AudoFDO profile to be > for the exact source. ...but from my experience, that's not what > happens in reality. If nothing else, developers locally working on the Right. Rong's recommendation makes sense from a google3 perspective (who I suspect is continuously regenerating profile data), but isn't feasible for Android or ChromeOS (who are not). > kernel still want to keep the AutoFDO optimizations so they're testing > as close to what's in the field as possible. However, the act of > working on the kernel will touch _some_ of the source. Keeping AFDO > working as much as possible seems nice to me. Of course, I'm by no > means an AutoFDO expert and mostly just got dragged into this issue, > so happy to defer to the experts. ;-) > > -Doug -- Thanks, ~Nick Desaulniers ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile 2026-11-05 21:30 ` Doug Anderson 2026-11-05 22:37 ` Nick Desaulniers @ 2026-11-05 22:40 ` Nick Desaulniers 2026-11-05 22:54 ` Doug Anderson 2026-11-05 23:16 ` Doug Anderson 2 siblings, 1 reply; 11+ messages in thread From: Nick Desaulniers @ 2026-11-05 22:40 UTC (permalink / raw) To: Doug Anderson Cc: Rong Xu, Nathan Chancellor, llvm, linux-kbuild, Han Shen, Nicolas Schier, Han Wang, Alexandre Courbot, Alice Ryhl, Andreas Hindborg, Benno Lossin, Bill Wendling, Björn Roy Baron, Boqun Feng, Daniel Almeida, Danilo Krummrich, Gary Guo, Justin Stitt, Miguel Ojeda, Onur Özkan, Tamir Duberstein, Trevor Gross, linux-kernel, rust-for-linux On Thu, Nov 5, 2026 at 1:30 PM Doug Anderson <dianders@chromium.org> wrote: > > Hi, > > On Thu, Nov 5, 2026 at 10:23 AM Rong Xu <xur@google.com> wrote: > > > > The issue you mentioned regarding excessive time in applyFlowInference > > comes from Profi. > > > > The AutoFDO implementation used in the kernel is FS-AFDO (which uses > > FS discriminators). There are known issues when using Profi with > > FS-AFDO; enabling Profi actually degrades performance compared to > > original profile propagation. When we tested and worked on AutoFDO for > > the kernel, we did not enable Profi. > > > > While I haven't looked into this specific bug yet and am unsure if it > > is unique to FS-AFDO, I recommend using -fno-sample-profile-use-profi > > to disable Profi for now. > > Interesting. I can suggest that to the team digging into it and see if > it solves the problem. My AI agent says it will though. It claims that > the default flipped recently and "-fno-sample-profile-use-profi" will I wonder who flipped the default? Perhaps they're not aware of _all_ of the existing users of AutoFDO, such as the Linux kernel. > restore the old (tested) behavior. > > Should I send a patch adding that flag? -- Thanks, ~Nick Desaulniers ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile 2026-11-05 22:40 ` Nick Desaulniers @ 2026-11-05 22:54 ` Doug Anderson 0 siblings, 0 replies; 11+ messages in thread From: Doug Anderson @ 2026-11-05 22:54 UTC (permalink / raw) To: Nick Desaulniers Cc: Rong Xu, Nathan Chancellor, llvm, linux-kbuild, Han Shen, Nicolas Schier, Han Wang, Alexandre Courbot, Alice Ryhl, Andreas Hindborg, Benno Lossin, Bill Wendling, Björn Roy Baron, Boqun Feng, Daniel Almeida, Danilo Krummrich, Gary Guo, Justin Stitt, Miguel Ojeda, Onur Özkan, Tamir Duberstein, Trevor Gross, linux-kernel, rust-for-linux Hi, On Thu, Nov 5, 2026 at 2:40 PM Nick Desaulniers <ndesaulniers@google.com> wrote: > > On Thu, Nov 5, 2026 at 1:30 PM Doug Anderson <dianders@chromium.org> wrote: > > > > Hi, > > > > On Thu, Nov 5, 2026 at 10:23 AM Rong Xu <xur@google.com> wrote: > > > > > > The issue you mentioned regarding excessive time in applyFlowInference > > > comes from Profi. > > > > > > The AutoFDO implementation used in the kernel is FS-AFDO (which uses > > > FS discriminators). There are known issues when using Profi with > > > FS-AFDO; enabling Profi actually degrades performance compared to > > > original profile propagation. When we tested and worked on AutoFDO for > > > the kernel, we did not enable Profi. > > > > > > While I haven't looked into this specific bug yet and am unsure if it > > > is unique to FS-AFDO, I recommend using -fno-sample-profile-use-profi > > > to disable Profi for now. > > > > Interesting. I can suggest that to the team digging into it and see if > > it solves the problem. My AI agent says it will though. It claims that > > the default flipped recently and "-fno-sample-profile-use-profi" will > > I wonder who flipped the default? Perhaps they're not aware of _all_ > of the existing users of AutoFDO, such as the Linux kernel. I've been preparing a patch to add -fno-sample-profile-use-profi. As part of that, my commit message will include this: Starting in Clang 21 (LLVM commit 2fc4a4a9d3a8 ("[Driver][SamplePGO] Enable -fsample-profile-use-profi (#146795)") [1]), the Clang driver automatically passes -mllvm -sample-profile-use-profi whenever -fprofile-sample-use is specified, enabling LLVM's flow-based profile inference algorithm (Profi) by default. Link: https://github.com/llvm/llvm-project/commit/2fc4a4a9d3a87a9e11c48ef25522aa17c2a71003 [1] -Doug ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile 2026-11-05 21:30 ` Doug Anderson 2026-11-05 22:37 ` Nick Desaulniers 2026-11-05 22:40 ` Nick Desaulniers @ 2026-11-05 23:16 ` Doug Anderson 2 siblings, 0 replies; 11+ messages in thread From: Doug Anderson @ 2026-11-05 23:16 UTC (permalink / raw) To: Rong Xu Cc: Nathan Chancellor, llvm, linux-kbuild, Han Shen, Nicolas Schier, Nick Desaulniers, Han Wang, Alexandre Courbot, Alice Ryhl, Andreas Hindborg, Benno Lossin, Bill Wendling, Björn Roy Baron, Boqun Feng, Daniel Almeida, Danilo Krummrich, Gary Guo, Justin Stitt, Miguel Ojeda, Onur Özkan, Tamir Duberstein, Trevor Gross, linux-kernel, rust-for-linux Hi, On Thu, Nov 5, 2026 at 1:30 PM Doug Anderson <dianders@chromium.org> wrote: > > > While I haven't looked into this specific bug yet and am unsure if it > > is unique to FS-AFDO, I recommend using -fno-sample-profile-use-profi > > to disable Profi for now. > > Interesting. I can suggest that to the team digging into it and see if > it solves the problem. My AI agent says it will though. It claims that > the default flipped recently and "-fno-sample-profile-use-profi" will > restore the old (tested) behavior. > > Should I send a patch adding that flag? I went ahead and posted the patch: https://lore.kernel.org/r/20261105151159.1.I0b27923381ef34cc99cdf8b65fea2e1bafce6370@changeid Obviously that patch and this patch conflict because they touch the same lines of code. If folks think we should still move forward with both patches, I'm happy to rebase. > > Regarding the "--mllvm=-salvage-stale-profile" flag, I also have > > reservations about adding it for the following reasons: > > 1. It is heuristic-based and not enabled by default. I am unsure how > > thoroughly it has been tested, so we should assess it further before > > enabling it. > > FWIW, we did find that Android seems to enable this globally across > its userspace, which suggests it has been fairly well tested. You can > see the change here, which landed about 1.5 years ago. > > https://android-review.git.corp.google.com/c/platform/build/soong/+/3524291 > > A few notes: > > * That change _also_ adds > "--salvage-stale-profile-max-callsites=2000". Internally, there has > been debate about whether adding that would be a good idea, too. The conclusion of the internal debate is that, yes, we should add `--salvage-stale-profile-max-callsites=2000` too. So if we decide to move forward with this patch, I'll send a v2 that adds that flag as well. For now I'll hold off until there is consensus. :-) -Doug ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile 2026-11-05 1:45 [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile Douglas Anderson 2026-11-05 5:24 ` Nathan Chancellor @ 2026-11-05 19:47 ` Miguel Ojeda 2026-11-05 21:36 ` Doug Anderson 1 sibling, 1 reply; 11+ messages in thread From: Miguel Ojeda @ 2026-11-05 19:47 UTC (permalink / raw) To: Douglas Anderson Cc: llvm, linux-kbuild, Rong Xu, Han Shen, Nathan Chancellor, Nicolas Schier, Nick Desaulniers, Han Wang, Alexandre Courbot, Alice Ryhl, Andreas Hindborg, Benno Lossin, Bill Wendling, Björn Roy Baron, Boqun Feng, Daniel Almeida, Danilo Krummrich, Gary Guo, Justin Stitt, Miguel Ojeda, Onur Özkan, Tamir Duberstein, Trevor Gross, linux-kernel, rust-for-linux On Thu, Nov 5, 2026 at 2:46 AM Douglas Anderson <dianders@chromium.org> wrote: > > Enable LLVM stale profile salvaging (-salvage-stale-profile, supported > since LLVM 17) for C, Rust, and ThinLTO builds when > + RUSTFLAGS_AUTOFDO_CLANG += -Cllvm-args=-salvage-stale-profile I double-checked that the flag was introduced in LLVM 17 as the cover letter mentions, and so it is likely fine given the Rust 1.85.0 compiler (i.e. our minimum) supports LLVM 18 as the minimum major version, but if the patch goes ahead in the end, then we should make sure it works well with that one (or probably you already did, in which case, please ignore this!). Cheers, Miguel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile 2026-11-05 19:47 ` Miguel Ojeda @ 2026-11-05 21:36 ` Doug Anderson 0 siblings, 0 replies; 11+ messages in thread From: Doug Anderson @ 2026-11-05 21:36 UTC (permalink / raw) To: Miguel Ojeda Cc: llvm, linux-kbuild, Rong Xu, Han Shen, Nathan Chancellor, Nicolas Schier, Nick Desaulniers, Han Wang, Alexandre Courbot, Alice Ryhl, Andreas Hindborg, Benno Lossin, Bill Wendling, Björn Roy Baron, Boqun Feng, Daniel Almeida, Danilo Krummrich, Gary Guo, Justin Stitt, Miguel Ojeda, Onur Özkan, Tamir Duberstein, Trevor Gross, linux-kernel, rust-for-linux Hi, On Thu, Nov 5, 2026 at 11:47 AM Miguel Ojeda <miguel.ojeda.sandonis@gmail.com> wrote: > > On Thu, Nov 5, 2026 at 2:46 AM Douglas Anderson <dianders@chromium.org> wrote: > > > > Enable LLVM stale profile salvaging (-salvage-stale-profile, supported > > since LLVM 17) for C, Rust, and ThinLTO builds when > > > + RUSTFLAGS_AUTOFDO_CLANG += -Cllvm-args=-salvage-stale-profile > > I double-checked that the flag was introduced in LLVM 17 as the cover > letter mentions, and so it is likely fine given the Rust 1.85.0 > compiler (i.e. our minimum) supports LLVM 18 as the minimum major > version, but if the patch goes ahead in the end, then we should make > sure it works well with that one (or probably you already did, in > which case, please ignore this!). I haven't extensively tested the behavior across many versions of rust. If you are worried, I guess we could remove the rustflags part of the change? ...or, of course, we could just fully drop this patch (and maybe add `-fno-sample-profile-use-profi` instead) ? -Doug ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2026-11-05 23:17 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-11-05 1:45 [PATCH] kbuild: AutoFDO: Enable -salvage-stale-profile Douglas Anderson 2026-11-05 5:24 ` Nathan Chancellor 2026-11-05 16:24 ` Doug Anderson 2026-11-05 18:23 ` Rong Xu 2026-11-05 21:30 ` Doug Anderson 2026-11-05 22:37 ` Nick Desaulniers 2026-11-05 22:40 ` Nick Desaulniers 2026-11-05 22:54 ` Doug Anderson 2026-11-05 23:16 ` Doug Anderson 2026-11-05 19:47 ` Miguel Ojeda 2026-11-05 21:36 ` Doug Anderson
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox