From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C75E3E00A3 for ; Thu, 10 Sep 2026 09:29:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.41 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789032550; cv=none; b=eUFpN5r2PrjHVpoQ9D9cBX8FEkhwy7qiskrFzHBDC/b0/yc6+TmVlAEqhEsdeTxbSF50/igy1lECMzqOpmXtYED++VWGwT7gxIupE97d45ZFaJMKFhjkDFh0upd81AM8mPWX8XfDamUEhYkB+6g+SUg444l5VwcOosF+/yT/tLA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789032550; c=relaxed/simple; bh=YmZhILYI1Ap/QhHEbws/tgVKUcLX7+iRsfdsJ1tAFSs=; h=Date:From:To:Cc:Subject:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hbMmciJ+pEPUPjQGlbgq2ykHuJi8IhvmanFJGDzn5LclsXnrs/5BJ+XV88mFMt1R673ekuqUt1TVjI/yeosp0g3HK7UYIusAYlfU7I5v4DyKhXIV6dWbGA3Sn1wL0J4ME3Ek39OB7BKQ4RnL9/yfBIxNrBiMKU2At/jifdFRRiU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=juxANAUC; arc=none smtp.client-ip=209.85.128.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="juxANAUC" Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-49b0d78a801so67379405e9.2 for ; Thu, 10 Sep 2026 02:29:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789032546; x=1789637346; darn=lists.linux.dev; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:subject:cc:to:from:date:from:to:cc:subject :date:message-id:reply-to:content-type; bh=xKhavT2ReA2W483I2xKkmSEJ1FM7bdVhooTyy/B0hnc=; b=juxANAUCe9vfgms4nuYcZOGArNnkiAQf5mdVXS4o/+OVB228lr4dAFBtCJwy1KduG0 evqcWtS0v4ctIVLg5DzD5RJ/S11lHSzyL5VR6f5wNhjpVJpkX7vslXyY9nUueVyhZSH7 MdExTw1Y8zybGkrRdkyiZAkgubl3KYd+q+dbpjYZ5Z+IUI3zJf2cG56npH6vEDpiHMVH T4X3IfeQpH9fiTszGDHUFf5jJtXSRKhO8Ncz3VQm3pkqOKSWIsjdxBuSwZoiL+N6CpXX 4y9LuN+FplIdaGEu2SprLoIkZIJCmGAYK41PG6TEGbASwy1ZniReffKoXW/Uj0aWtPZp 0oVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789032546; x=1789637346; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=xKhavT2ReA2W483I2xKkmSEJ1FM7bdVhooTyy/B0hnc=; b=pCB2rO+k7Z9GkqTDiJg8H0KBbSkMOckkP3GyB8+jyf+1fZ0thXkYfiTGBYPuylTGU4 3TILQ2Gm4sPgc/17b2ZQOFcrA4VTXT/vc9YTtTBr1QyLW+ZyXX6TvutzocOb80D4OGBc pVMpv7Z6QIeorOR9oGm2RXSahjh6uUG4ABaYtiv5vV6fWeTKmTXRH3QK6m6FMNe6kqdN 3d+4P2l5dP1z6EbtrUV2jo5c8CrD8VwUvjIgTHvOIzwg9w1PGeLo3BxJjM9qWNOAMS5e SS1ju096a6tpUihpBGHzBksbOmasvdsERWlPSu+Y26SZwdRS+Ui43ytVcDYNTy+mpIe9 jpZw== X-Forwarded-Encrypted: i=1; AKwUvBwxRn5LyzwzNPfsDhXu6kH927GWLYciaH7cJHkpPoEEvw1/FTlZ4ZM1fad+yVMVy9wrOBV7@lists.linux.dev X-Gm-Message-State: AFuF++l9nCOdrZ4ZZUf2oJhXnaX96/S/+2HD7huVOfI0taz+fKYXh4Ma mNznlEiVhqBQruZjev0OLb27Q/fgiClFPMHMHuEVKz3ggIfYcrh9rJe8 X-Gm-Gg: AYBFou2nRYC7t+f05KL1bRJu8k2nyIdgQHtb6Lde1014tXclQpK/OiSSSiuoiOM2RDb Gf64fbhvyCr+CuxWngmdXs+sBf1Ak6DYCUG5LmD3/i1SvPsXHph7MxX8kLENEoKMs67YJ8UuNil Vs5fkyS41QBZGXJcomvO0lVO1cRseRcvSSUwgIuT1CWXMuzFSDP0LqcUi1rgv8x72/D/6DRNH9w fEPyvZ5LLI5rujbj4FDHaHq1YxXBPGcHIBFVLvdEdelmzqe+YQKYzqtMpqQXbbNb/RKTBWMsNsz DDHTqzl9tDlsD/Tn0PpxmZAQXR+csR2Y/3NsdTBOH2MWrvrPIzAn8DIn2XxqvMLJmmgw6Sel/F+ 5c7JQ1yFlWo+NM6E3mSOAuOAR2KgY8Q2hZvlzi2XANkoiCs8S7NIRm/JsG8Lq3NmXJS1Gkj5arA RE9WaXfDWpcFdmv/8LLlrqQ4TI8zIxGwks87McjU1JWL8zF4frx/WmMhKyVWv5pS2pI1YSQ1jlt irQ/LKFIID6mcjAwXR52xgl3lzJTz1TUTO6 X-Received: by 2002:a05:600c:138b:b0:49e:2a61:a17b with SMTP id 5b1f17b1804b1-49e2a61a41emr9166915e9.5.1789032546119; Thu, 10 Sep 2026 02:29:06 -0700 (PDT) Received: from pumpkin (82-69-66-36.dsl.in-addr.zen.co.uk. [82.69.66.36]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49d20fc23fcsm106027125e9.3.2026.09.10.02.29.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2026 02:29:05 -0700 (PDT) Date: Thu, 10 Sep 2026 10:29:03 +0100 From: David Laight To: "Lorenzo Stoakes (ARM)" Cc: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?UTF-8?B?QmrDtnJu?= Roy Baron , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , Onur =?UTF-8?B?w5Z6a2Fu?= , Jonathan Corbet , Randy Dunlap , linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe Subject: Re: [PATCH 04/23] kallsyms: output binary data to speed output and kallsyms assembly Message-ID: <20260910102903.1b211f6b@pumpkin> In-Reply-To: <20260908-build-speedup-v1-4-5dc1ac01672d@kernel.org> References: <20260908-build-speedup-v1-0-5dc1ac01672d@kernel.org> <20260908-build-speedup-v1-4-5dc1ac01672d@kernel.org> X-Mailer: Claws Mail 4.1.1 (GTK 3.24.38; arm-unknown-linux-gnueabihf) Precedence: bulk X-Mailing-List: llvm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit On Tue, 08 Sep 2026 21:55:04 +0100 "Lorenzo Stoakes (ARM)" wrote: > kallsyms generates an assembly file that consists mostly of .byte entries > containing compressed names, token strings and name-sorted sequence > numbers. > > For an x86-64 build with 158k symbols that is a 37 MiB .S file which takes > 0.57s to assemble each of the two to three times it is built over a kernel > build. > > Each time it is generated it also takes kallsyms a similar amount of time > to output it. > > Avoid this overhead by instead outputting this data as binary and importing > it into the assembly using the .incbin directive. > > Tables that are wider than a byte remain part of the assembly to ensure > endianness and relative relocations are performed correctly. > > With this change, the output assembly file shrinks from 37 MiB to 9.8 MiB, > with a 2.6 MiB binary data file alongside it, and the object remains > identical. > > The generated binary file is deleted correctly on build clean along with > all other ephemeral data. > > On an x86-64 system with CONFIG_KALLSYMS_ALL set: > > before after delta > scripts/kallsyms 0.24s 0.18s 0.06s > assemble 0.57s 0.16s 0.41s > > Per kallsyms invocation/assembly, for a total of 0.47s time saving upon > invocation. > > An incremental build on the same system was reduced from 11.15s to 9.65s, > indicating a total of 1.5 seconds saved over the build. > > The kallsyms runs and their assembly are on the serial tail of every build > that links vmlinux, no-op builds are unchanged. > > Whole build, 128-thread Threadripper 9980X, best of N runs: > > before after delta > ------------------------------- > x86 defconfig, touch mm/vma.c, gcc 10.8s 9.9s -0.92s (-8%) > x86 defconfig, touch mm/vma.c, clang 10.7s 9.5s -1.2s (-11%) > x86 defconfig, clean, gcc 29.5s 28.7s -0.81s (-3%) > x86 defconfig, clean, clang 29.7s 28.6s -1.1s (-4%) > x86 allmodconfig, touch mm/vma.c, gcc 45.3s 44.0s -1.3s (-3%) > x86 allmodconfig, touch mm/vma.c, clang 42.9s 40.2s -2.7s (-6%) > > Assisted-by: LLM > Signed-off-by: Lorenzo Stoakes (ARM) > --- > scripts/kallsyms.c | 97 ++++++++++++++++++++++++++++++++++++++----------- > scripts/link-vmlinux.sh | 2 +- > 2 files changed, 77 insertions(+), 22 deletions(-) > > diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c > index 350d118c3b9e..61c5eb537ed4 100644 > --- a/scripts/kallsyms.c > +++ b/scripts/kallsyms.c > @@ -5,7 +5,10 @@ > * This software may be used and distributed according to the terms > * of the GNU General Public License, incorporated herein by reference. > * > - * Usage: kallsyms [--all-symbols] in.map > out.S > + * Usage: kallsyms [--all-symbols] [--pc-relative] in.map out.bin > out.S > + * > + * The byte tables go to out.bin and are pulled into out.S with .incbin; > + * wider tables stay assembler source for endianness and relocations. > * > * Table compression uses all the unused char codes on the symbols and > * maps these to the most used substrings (tokens). For instance, it might > @@ -102,7 +105,7 @@ static void sym_arr_free(struct sym_arr *arr) > > static void usage(void) > { > - fprintf(stderr, "Usage: kallsyms [--all-symbols] in.map > out.S\n"); > + fprintf(stderr, "Usage: kallsyms [--all-symbols] [--pc-relative] in.map out.bin > out.S\n"); > exit(1); > } > > @@ -319,6 +322,40 @@ static void output_label(const char *label) > printf("%s:\n", label); > } > > +static void write_bin(FILE *file, const void *data, size_t len) > +{ > + if (fwrite(data, 1, len, file) == len) > + return; > + > + perror("kallsyms: write"); > + exit(EXIT_FAILURE); > +} It is pretty pointless checking the return value from fwrite(). Most of the time it is just doing a memcpy(). Instead call fflush() and the ferror() prior to the fclose(). (Or just rely on fclose() giving you that error status.) David