From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D08CC433FE for ; Tue, 8 Dec 2020 12:17:27 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4B12C239D0 for ; Tue, 8 Dec 2020 12:17:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4B12C239D0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=X1f/xtM1738pX0MRQVomDhewjClXihzpzcNg65IBQP0=; b=Khu2xsUT1JHHrwbbH1OD2okPD /lXz1fv8czfI3B2B/XB6TJdyExaHpKyDjXAfGHxv5F1MUvlTaSv+hlgRyUU/QtDfph2wTuGEWtFkV f2Bootnqt9GM3U3gvjadQUSR2bwNTRX6+V1j3yy2q7HkH8zg9lcdliFNbL/pyzr+W5lm0HlLAW7Fm pxfkPaV1FFJTGOVGeIl+GlcWXTJEAwKKQtefyWq2IBq95Jpe02qIdEgt+n3hA5hqXao0P6BRzPxjZ jkoSMRXLuPXhjb/KBeEU1ZFuXkROygn097YLecGE0HYAzPql5+OOsgwsYh9A6lZhDnEUqP/8lBe9P B2u1tm1zw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmbuN-0003gC-Ob; Tue, 08 Dec 2020 12:15:55 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmbuJ-0003fE-12 for linux-arm-kernel@lists.infradead.org; Tue, 08 Dec 2020 12:15:53 +0000 X-Gm-Message-State: AOAM533VQExnxY3CwoOEP7br4GsMQWpDCVKP2IsHyC1sCT7EpphgRjSE vh2rjP24Tu9WnqNAb3Ic0HfiRCRZwTMW/kJobzI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1607429749; bh=K94MQ6q50UGiEvhj+VB0lG6N0jZujxUWTttppsr8gwM=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=Ns4p/v7H7ScAMGSZF+uiL0GLrzuK92X4fsakgAnSLuAme/K5fEsEi38bY3jw803Wy audkXq/WYB7RTMZ3yOuTK6onrxqVBtnoYqkkvLnvpNRPrQxrYcO6XAo70lbZp3Wa4T 9qu4teHx4OR4D7tI19aZUR6LPG7nQ4wU5rtl2vcxyqBk81XPqwKBpVKsGaiwHXIJuy PuNNq9XDDyWTO1GKhaqTq1XoK2qrf9wdaOmxVotAI6FaDwC0nVzQIN5BYHWvkh+7Pl TIDC46XqhgPtl/QkCcf02gs7jdiOHkHzlufEr692R+gOJKVmFAwvvzdy3POnIF8teL 9TV0RUdUG0tPw== X-Google-Smtp-Source: ABdhPJwYXzEbw5wojirREdrWvTDFdS68u6lSRc1TxQHV80oZ08BdPpLLWGJpTl+WAoAqOweiNLoKVC/CXI7Rr5nYmRU= X-Received: by 2002:a05:6808:9a9:: with SMTP id e9mr2451121oig.4.1607429748222; Tue, 08 Dec 2020 04:15:48 -0800 (PST) MIME-Version: 1.0 References: <20201201213707.541432-1-samitolvanen@google.com> In-Reply-To: <20201201213707.541432-1-samitolvanen@google.com> From: Arnd Bergmann Date: Tue, 8 Dec 2020 13:15:31 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v8 00/16] Add support for Clang LTO To: Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201208_071551_218061_F41BCD67 X-CRM114-Status: GOOD ( 27.10 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch , Kees Cook , "Paul E. McKenney" , Kernel Hardening , Peter Zijlstra , Greg Kroah-Hartman , Masahiro Yamada , Linux Kbuild mailing list , Nick Desaulniers , "linux-kernel@vger.kernel.org" , Steven Rostedt , clang-built-linux , linux-pci , Josh Poimboeuf , Will Deacon , Linux ARM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Dec 1, 2020 at 10:37 PM 'Sami Tolvanen' via Clang Built Linux wrote: > > This patch series adds support for building the kernel with Clang's > Link Time Optimization (LTO). In addition to performance, the primary > motivation for LTO is to allow Clang's Control-Flow Integrity (CFI) > to be used in the kernel. Google has shipped millions of Pixel > devices running three major kernel versions with LTO+CFI since 2018. > > Most of the patches are build system changes for handling LLVM > bitcode, which Clang produces with LTO instead of ELF object files, > postponing ELF processing until a later stage, and ensuring initcall > ordering. > > Note that arm64 support depends on Will's memory ordering patches > [1]. I will post x86_64 patches separately after we have fixed the > remaining objtool warnings [2][3]. > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/log/?h=for-next/lto > [2] https://lore.kernel.org/lkml/20201120040424.a3wctajzft4ufoiw@treble/ > [3] https://git.kernel.org/pub/scm/linux/kernel/git/jpoimboe/linux.git/log/?h=objtool-vmlinux > > You can also pull this series from > > https://github.com/samitolvanen/linux.git lto-v8 I've tried pull this into my randconfig test tree to give it a spin. So far I have not managed to get a working build out of it, the main problem so far being that it is really slow to build because the link stage only uses one CPU. These are the other issues I've seen so far: - one build seems to take even longer to link. It's currently at 35GB RAM usage and 40 minutes into the final link, but I'm worried it might not complete before it runs out of memory. I only have 128GB installed, and google-chrome uses another 30GB of that, and I'm also doing some other builds in parallel. Is there a minimum recommended amount of memory for doing LTO builds? - One build failed with ld.lld -EL -maarch64elf -mllvm -import-instr-limit=5 -r -o vmlinux.o -T .tmp_initcalls.lds --whole-archive arch/arm64/kernel/head.o init/built-in.a usr/built-in.a arch/arm64/built-in.a kernel/built-in.a certs/built-in.a mm/built-in.a fs/built-in.a ipc/built-in.a security/built-in.a crypto/built-in.a block/built-in.a arch/arm64/lib/built-in.a lib/built-in.a drivers/built-in.a sound/built-in.a net/built-in.a virt/built-in.a --no-whole-archive --start-group arch/arm64/lib/lib.a lib/lib.a ./drivers/firmware/efi/libstub/lib.a --end-group "ld.lld: error: arch/arm64/kernel/head.o: invalid symbol index" after about 30 minutes - CONFIG_CPU_BIG_ENDIAN doesn't seem to work with lld, and LTO doesn't work with ld.bfd. I've added a CPU_LITTLE_ENDIAN dependency to ARCH_SUPPORTS_LTO_CLANG{,THIN} - one build failed with "ld.lld: error: Never resolved function from blockaddress (Producer: 'LLVM12.0.0' Reader: 'LLVM 12.0.0')" Not sure how to debug this - one build seems to have dropped all symbols the string operations from vmlinux, so while the link goes through, modules cannot be loaded: ERROR: modpost: "memmove" [drivers/media/rc/rc-core.ko] undefined! ERROR: modpost: "memcpy" [net/wireless/cfg80211.ko] undefined! ERROR: modpost: "memcpy" [net/8021q/8021q.ko] undefined! ERROR: modpost: "memset" [net/8021q/8021q.ko] undefined! ERROR: modpost: "memcpy" [net/unix/unix.ko] undefined! ERROR: modpost: "memset" [net/sched/cls_u32.ko] undefined! ERROR: modpost: "memcpy" [net/sched/cls_u32.ko] undefined! ERROR: modpost: "memset" [net/sched/sch_skbprio.ko] undefined! ERROR: modpost: "memcpy" [net/802/garp.ko] undefined! I first thought this was related to a clang-12 bug I saw the other day, but this also happens with clang-11 - many builds complain about thousands of duplicate symbols in the kernel, e.g. ld.lld: error: duplicate symbol: qrtr_endpoint_post >>> defined in net/qrtr/qrtr.lto.o >>> defined in net/qrtr/qrtr.o ld.lld: error: duplicate symbol: init_module >>> defined in crypto/842.lto.o >>> defined in crypto/842.o ld.lld: error: duplicate symbol: init_module >>> defined in net/netfilter/nfnetlink_log.lto.o >>> defined in net/netfilter/nfnetlink_log.o ld.lld: error: duplicate symbol: vli_from_be64 >>> defined in crypto/ecc.lto.o >>> defined in crypto/ecc.o ld.lld: error: duplicate symbol: __mod_of__plldig_clk_id_device_table >>> defined in drivers/clk/clk-plldig.lto.o >>> defined in drivers/clk/clk-plldig.o Not sure if these are all known issues. If there is one you'd like me try take a closer look at for finding which config options break it, I can try Arnd _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel