From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FSL_HELO_FAKE, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 189E2C433FE for ; Wed, 9 Dec 2020 04:58:03 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B75D1238EE for ; Wed, 9 Dec 2020 04:58:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B75D1238EE Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Kz4p7hPykn69igWx7P2tmj6VpuLYM9iacoZrqjkud8U=; b=HbutvWeSfwpdHvn8Sp1Bnxft6 4YCv/FLTqQsgCsyaTQDd0ZE7fRvhiT0k/WPcPftEQMj2w92e9cJQ8nuFnYWwsjtwX4FP0HgDeELZ7 Wfjn6Nrq/fUAFzT4OEZJ7DvM+KmznIYvvWwA8U/8D7XBe69HWRl0lXKtFlA1Cfu5RCJTf2lck5yhN kVILhXe0Zw2FbUxd0e56uvl1FnjUjxADr/BeeU3ImN9VTUsX/GpEiY1Ys2g0iXspeTjfQTmSvs9WT qfDVr3DTiufcACyCNw9XHHOOxAlhznnjaI2EIRMZWYOmVTd0q0SN5sUrHYh+99+6vzU+D8/VF0nzv 1sbZZN0KQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmrWN-0005SC-GX; Wed, 09 Dec 2020 04:56:11 +0000 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmrWF-0005Q2-0c for linux-arm-kernel@lists.infradead.org; Wed, 09 Dec 2020 04:56:07 +0000 Received: by mail-pf1-x435.google.com with SMTP id t7so223103pfh.7 for ; Tue, 08 Dec 2020 20:56:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=yNxPOOK8GUbsNe5moby382MnhA6+vA6WAhboDqo7Axs=; b=gVlT6d0N/3N0A44YyPBRrdAxvqWSB7tInBfZYe89aKi6EzHIs7gP38E5CuWbaquJ9Q lC1TB2BxZUYHFRIwXjsuezaszTqBj7bKmpzzgY40z0e7LLAFdthLmtnn3YPf/IEXMkLc uuaBS0tglR6qOsch4B7MSYVIDFITtcPTJxHeU5A6u96w8NBHLL2cRAa1oSHG0o+gDjES LRBnSQST8QVgjW7urPuaJ13hoWEbi3iuyJLUcTkEcPCwBaydpA8W9h+RHDRqv/3BC5QO aT3Tn0QBJmvAL3K/fKFdgT/IIZVcAKEdJKnXkXMb7a2aG8QVPIhjtldRqQ/aGIBinoxs 9hsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=yNxPOOK8GUbsNe5moby382MnhA6+vA6WAhboDqo7Axs=; b=XTPKRxPsegNPdV0djlVbQN94unztJ1EtNN769TqDSGC2/gAaxCCpg1kKFyGKOuTNsI ZtCjnzXYq7YBAjNiwaWgT2/pCB2zYdFG3lUM6Ewmx9SaCA3bjYAgnBR42Xd+vTNEYNwO iU4ql2aTJ1pGzB3Joxq55avuqv3doQSnCcG69L6n7JmxZ+X8kQgLGNgJh7gZrj5NwmaI fw5tJ+mKAleGIfHB8tJmdR0+fLT5An6cTof493cjN3KwzW5k3S5TBhZdL5wSCKebdbAT j8JxQ6I8PAEjZX/nFQGjJ3A4EF0PvjofbyGvlsa5nI0M37rsN6IQTTZlDnm+9RCMFMAo tFtw== X-Gm-Message-State: AOAM53044fGjQ/TJ/7ZnVrU9ADUu8npMXfDWsh621pXCEtsU9B2xbxzQ zUuIr0IdYbkcTlyujMH3Wz9wDUlaymeyMlz6 X-Google-Smtp-Source: ABdhPJyVhevtlV182GztPlTk2U6DWLN8px/r+M03xp8sndkyhgLZY0CoPxIqSa+brsrwqARmVo17sA== X-Received: by 2002:a63:494f:: with SMTP id y15mr405121pgk.364.1607489758368; Tue, 08 Dec 2020 20:55:58 -0800 (PST) Received: from google.com ([2620:15c:2ce:0:a6ae:11ff:fe11:4abb]) by smtp.gmail.com with ESMTPSA id s17sm508448pge.37.2020.12.08.20.55.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Dec 2020 20:55:57 -0800 (PST) Date: Tue, 8 Dec 2020 20:55:54 -0800 From: Fangrui Song To: Sami Tolvanen Subject: Re: [PATCH v8 00/16] Add support for Clang LTO Message-ID: <20201209045554.fxlzrmxknakl2gdr@google.com> References: <20201201213707.541432-1-samitolvanen@google.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201208_235603_123645_6DA21941 X-CRM114-Status: GOOD ( 36.87 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch , Arnd Bergmann , Kees Cook , "Paul E. McKenney" , Kernel Hardening , Peter Zijlstra , Greg Kroah-Hartman , Masahiro Yamada , Linux Kbuild mailing list , Nick Desaulniers , "linux-kernel@vger.kernel.org" , Steven Rostedt , clang-built-linux , linux-pci , Josh Poimboeuf , Will Deacon , Linux ARM Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 2020-12-08, 'Sami Tolvanen' via Clang Built Linux wrote: >On Tue, Dec 8, 2020 at 4:15 AM Arnd Bergmann wrote: >> >> On Tue, Dec 1, 2020 at 10:37 PM 'Sami Tolvanen' via Clang Built Linux >> wrote: >> > >> > This patch series adds support for building the kernel with Clang's >> > Link Time Optimization (LTO). In addition to performance, the primary >> > motivation for LTO is to allow Clang's Control-Flow Integrity (CFI) >> > to be used in the kernel. Google has shipped millions of Pixel >> > devices running three major kernel versions with LTO+CFI since 2018. >> > >> > Most of the patches are build system changes for handling LLVM >> > bitcode, which Clang produces with LTO instead of ELF object files, >> > postponing ELF processing until a later stage, and ensuring initcall >> > ordering. >> > >> > Note that arm64 support depends on Will's memory ordering patches >> > [1]. I will post x86_64 patches separately after we have fixed the >> > remaining objtool warnings [2][3]. >> > >> > [1] https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/log/?h=for-next/lto >> > [2] https://lore.kernel.org/lkml/20201120040424.a3wctajzft4ufoiw@treble/ >> > [3] https://git.kernel.org/pub/scm/linux/kernel/git/jpoimboe/linux.git/log/?h=objtool-vmlinux >> > >> > You can also pull this series from >> > >> > https://github.com/samitolvanen/linux.git lto-v8 >> >> I've tried pull this into my randconfig test tree to give it a spin. > >Great, thank you for testing this! > >> So far I have >> not managed to get a working build out of it, the main problem so far being >> that it is really slow to build because the link stage only uses one CPU. >> These are the other issues I've seen so far: ld.lld ThinLTO uses the number of (physical cores enabled by affinity) by default. >You may want to limit your testing only to ThinLTO at first, because >full LTO is going to be extremely slow with larger configs, especially >when building arm64 kernels. > >> - one build seems to take even longer to link. It's currently at 35GB RAM >> usage and 40 minutes into the final link, but I'm worried it might >> not complete >> before it runs out of memory. I only have 128GB installed, and google-chrome >> uses another 30GB of that, and I'm also doing some other builds in parallel. >> Is there a minimum recommended amount of memory for doing LTO builds? > >When building arm64 defconfig, the maximum memory usage I measured >with ThinLTO was 3.5 GB, and with full LTO 20.3 GB. I haven't measured >larger configurations, but I believe LLD can easily consume 3-4x that >much with full LTO allyesconfig. > >> - One build failed with >> ld.lld -EL -maarch64elf -mllvm -import-instr-limit=5 -r -o vmlinux.o >> -T .tmp_initcalls.lds --whole-archive arch/arm64/kernel/head.o >> init/built-in.a usr/built-in.a arch/arm64/built-in.a kernel/built-in.a >> certs/built-in.a mm/built-in.a fs/built-in.a ipc/built-in.a >> security/built-in.a crypto/built-in.a block/built-in.a >> arch/arm64/lib/built-in.a lib/built-in.a drivers/built-in.a >> sound/built-in.a net/built-in.a virt/built-in.a --no-whole-archive >> --start-group arch/arm64/lib/lib.a lib/lib.a >> ./drivers/firmware/efi/libstub/lib.a --end-group >> "ld.lld: error: arch/arm64/kernel/head.o: invalid symbol index" >> after about 30 minutes > >That's interesting. Did you use LLVM_IAS=1? May be worth checking which relocation or (SHT_GROUP section's sh_info) in arch/arm64/kernel/head.o is incorrect. >> - CONFIG_CPU_BIG_ENDIAN doesn't seem to work with lld, and LTO >> doesn't work with ld.bfd. >> I've added a CPU_LITTLE_ENDIAN dependency to >> ARCH_SUPPORTS_LTO_CLANG{,THIN} > >Ah, good point. I'll fix this in v9. Full/Thin LTO should work with GNU ld and gold with LLVMgold.so built from llvm-project (https://llvm.org/docs/GoldPlugin.html ). You'll need to make sure that LLVMgold.so is newer than clang. (Newer clang may introduce bitcode attributes which are unrecognizable by older LLVMgold.so/ld.lld) >[...] >> Not sure if these are all known issues. If there is one you'd like me try >> take a closer look at for finding which config options break it, I can try > >No, none of these are known issues. I would be happy to take a closer >look if you can share configs that reproduce these. > >Sami > >-- >You received this message because you are subscribed to the Google Groups "Clang Built Linux" group. >To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com. >To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CABCJKueCHo2RYfx_A21m%2B%3Dd1gQLR9QsOOxCsHFeicCqyHkb-Kg%40mail.gmail.com. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel