All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>
To: Josh Poimboeuf <jpoimboe@kernel.org>
Cc: "Linus Torvalds" <torvalds@linux-foundation.org>,
	"Nathan Chancellor" <nathan@kernel.org>,
	"Nicolas Schier" <nsc@kernel.org>,
	"Nick Desaulniers" <ndesaulniers@google.com>,
	"Bill Wendling" <morbo@google.com>,
	"Justin Stitt" <justinstitt@google.com>,
	"Geert Uytterhoeven" <geert@linux-m68k.org>,
	"Luis Chamberlain" <mcgrof@kernel.org>,
	"Petr Pavlu" <petr.pavlu@suse.com>,
	"Daniel Gomez" <da.gomez@kernel.org>,
	"Sami Tolvanen" <samitolvanen@google.com>,
	"Aaron Tomlin" <atomlin@atomlin.com>,
	"Peter Zijlstra" <peterz@infradead.org>,
	"Miguel Ojeda" <ojeda@kernel.org>,
	"Boqun Feng" <boqun@kernel.org>, "Gary Guo" <gary@garyguo.net>,
	"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
	"Benno Lossin" <lossin@kernel.org>,
	"Andreas Hindborg" <a.hindborg@kernel.org>,
	"Alice Ryhl" <aliceryhl@google.com>,
	"Trevor Gross" <tmgross@umich.edu>,
	"Danilo Krummrich" <dakr@kernel.org>,
	"Daniel Almeida" <daniel.almeida@collabora.com>,
	"Tamir Duberstein" <tamird@kernel.org>,
	"Alexandre Courbot" <acourbot@nvidia.com>,
	"Onur Özkan" <work@onurozkan.dev>,
	linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org,
	llvm@lists.linux.dev, linux-m68k@lists.linux-m68k.org,
	linux-modules@vger.kernel.org, rust-for-linux@vger.kernel.org
Subject: Re: [PATCH v5 07/11] objtool: cache relocations, do less work, eliminate relocation hash
Date: Thu, 12 Nov 2026 08:00:19 +0000	[thread overview]
Message-ID: <avVwzFMkkDWCeWxN@gremlin> (raw)
In-Reply-To: <avVQy5UBOlu7IeQu@jpoimboe>

On Wed, Nov 11, 2026 at 09:45:28PM -0800, Josh Poimboeuf wrote:
> On Wed, Nov 04, 2026 at 06:20:25PM +0000, Lorenzo Stoakes (ARM) wrote:
> > Instruction relocations are looked up by destination in objtool via a hash
> > which is keyed on a 16-byte (OFFSET_STRIDE) window within the section being
> > walked.
> >
> > It iterates through each 16-byte window, looking up relocations over
> > several passes, before moving on to the next 16-byte window, caching only
> > when a relocation is not found saving further lookups in this case.
> >
> > Improve upon this by introducing a per-section relocation cache storing the
> > first relocation at or after each 64-byte window of data (an empirically
> > determined index range), indexed by chunk.
> >
> > The lookup is implemented an array lookup and touches no shared state, so
> > can be used from multiple threads.
> >
> > The index runs over the relocations in offset order. Usually this is how
> > they are input, however if not, each section is sorted once, storing
> > their offsets in reloc_order[].
> >
> > This is done lazily on lookup, so nothing pays the cost that doesn't have
> > to, and if everything is sorted nothing does.
> >
> > The output of objtool before and after this change was confirmed to be
> > byte-for-byte identical both for x86_64 defconfig and allmodconfig with
> > gcc and clang.
> >
> > objtool on the gcc allmodconfig vmlinux.o goes from 10.5s to 9.0s, clang
> > from 9.8s to 8.3s and gcc defconfig from 2.08s to 1.79s, with peak memory
> > down ~90 MiB on allmodconfig.
> >
> > Similar performance was also noted on a worst case scenario for unsorted
> > input (gcc but lld for linking), which confirms that the sorting mechanism
> > works correctly with little overhead.
> >
> > objtool on vmlinux.o is on the serial tail of every build that links
> > vmlinux, no-op builds are unchanged.
> >
> > Whole build, 128-thread Threadripper 9980X, best of N runs:
> >
> >                                          before   after     delta
> >                                          -------------------------------
> >   x86 defconfig, touch mm/vma.c, gcc        8.3s     8.2s    -0.14s (-2%)
> >   x86 defconfig, touch mm/vma.c, clang      7.6s     7.3s    -0.25s (-3%)
> >   x86 defconfig, clean, gcc                28.9s    28.6s    -0.22s (-1%)
> >   x86 defconfig, clean, clang              29.4s    29.1s    -0.28s (-1%)
> >   x86 allmodconfig, touch mm/vma.c, gcc    29.4s    28.0s     -1.4s (-5%)
> >   x86 allmodconfig, touch mm/vma.c, clang  27.9s    26.2s     -1.7s (-6%)
> >
> > Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> > ---
> >  tools/objtool/elf.c                 | 280 ++++++++++++++++++++++++++++--------
> >  tools/objtool/include/objtool/elf.h |  10 +-
> >  2 files changed, 221 insertions(+), 69 deletions(-)
>
> Urgh... so the closer I looked, the more this patch revealed itself to
> be an AI slop mess.  So I ended up completely rewriting it.

Well I took time to understand, cleanup and make the patches as good as I
could (you should have seen what it came up with originally etc.), but I
fully get it and sorry about that! :)

>
> You can go ahead and drop all the objtool patches from this series, I
> will repost them after some more cleanups.

From my side will obviously drop them on any respin!

>
> And I'm dropping patch 9 altogether, it's just not worth the pain...

Sure, again understood, sorry for subjecting you to it :P there's a win
there but obviously it's all rather a trade-off!

--
Cheers, Lorenzo

  reply	other threads:[~2026-11-12  8:00 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-04 18:20 [PATCH v5 00/11] kbuild: significantly speed up kernel builds Lorenzo Stoakes (ARM)
2026-11-04 18:20 ` [PATCH v5 01/11] kbuild: calculate list, composite object state per object Lorenzo Stoakes (ARM)
2026-11-12  4:13   ` Nathan Chancellor
2026-11-04 18:20 ` [PATCH v5 02/11] kbuild: implement and use depcheck to check dependency timestamps Lorenzo Stoakes (ARM)
2026-11-05 18:22   ` sashiko-bot
2026-11-04 18:20 ` [PATCH v5 03/11] modpost: cache section relocation mismatch state Lorenzo Stoakes (ARM)
2026-11-09 16:43   ` Petr Pavlu
2026-11-12  4:13   ` Nathan Chancellor
2026-11-04 18:20 ` [PATCH v5 04/11] modpost: emit module descriptors as assembly Lorenzo Stoakes (ARM)
2026-11-09 16:45   ` Petr Pavlu
2026-11-12  4:13   ` Nathan Chancellor
2026-11-04 18:20 ` [PATCH v5 05/11] kbuild: batch module finalisation Lorenzo Stoakes (ARM)
2026-11-12  4:13   ` Nathan Chancellor
2026-11-04 18:20 ` [PATCH v5 06/11] objtool: do not hash DWARF relocations Lorenzo Stoakes (ARM)
2026-11-04 18:20 ` [PATCH v5 07/11] objtool: cache relocations, do less work, eliminate relocation hash Lorenzo Stoakes (ARM)
2026-11-05 18:22   ` sashiko-bot
2026-11-12  5:45   ` Josh Poimboeuf
2026-11-12  8:00     ` Lorenzo Stoakes (ARM) [this message]
2026-11-04 18:20 ` [PATCH v5 08/11] objtool: size the instruction hash to the text Lorenzo Stoakes (ARM)
2026-11-05 18:22   ` sashiko-bot
2026-11-04 18:20 ` [PATCH v5 09/11] objtool: decode instructions and resolve branch targets in parallel Lorenzo Stoakes (ARM)
2026-11-04 18:20 ` [PATCH v5 10/11] rust: make exports.o depend on the headers generated for it Lorenzo Stoakes (ARM)
2026-11-11 16:04   ` Gary Guo
2026-11-04 18:20 ` [PATCH v5 11/11] kbuild: build rust crates in parallel with the rest of the build Lorenzo Stoakes (ARM)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=avVwzFMkkDWCeWxN@gremlin \
    --to=ljs@kernel.org \
    --cc=a.hindborg@kernel.org \
    --cc=acourbot@nvidia.com \
    --cc=aliceryhl@google.com \
    --cc=atomlin@atomlin.com \
    --cc=bjorn3_gh@protonmail.com \
    --cc=boqun@kernel.org \
    --cc=da.gomez@kernel.org \
    --cc=dakr@kernel.org \
    --cc=daniel.almeida@collabora.com \
    --cc=gary@garyguo.net \
    --cc=geert@linux-m68k.org \
    --cc=jpoimboe@kernel.org \
    --cc=justinstitt@google.com \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-m68k@lists.linux-m68k.org \
    --cc=linux-modules@vger.kernel.org \
    --cc=llvm@lists.linux.dev \
    --cc=lossin@kernel.org \
    --cc=mcgrof@kernel.org \
    --cc=morbo@google.com \
    --cc=nathan@kernel.org \
    --cc=ndesaulniers@google.com \
    --cc=nsc@kernel.org \
    --cc=ojeda@kernel.org \
    --cc=peterz@infradead.org \
    --cc=petr.pavlu@suse.com \
    --cc=rust-for-linux@vger.kernel.org \
    --cc=samitolvanen@google.com \
    --cc=tamird@kernel.org \
    --cc=tmgross@umich.edu \
    --cc=torvalds@linux-foundation.org \
    --cc=work@onurozkan.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.