From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC4F0C4332F for ; Thu, 13 Oct 2022 11:09:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id E3B734B27C; Thu, 13 Oct 2022 07:09:10 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@kernel.org Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QrHEudO89k45; Thu, 13 Oct 2022 07:09:09 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 86E5149F55; Thu, 13 Oct 2022 07:09:09 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id C34DB40FAC for ; Thu, 13 Oct 2022 07:09:07 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9jWRmNo-SC6p for ; Thu, 13 Oct 2022 07:09:06 -0400 (EDT) Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 8225D40C1F for ; Thu, 13 Oct 2022 07:09:06 -0400 (EDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 93E96B8162C; Thu, 13 Oct 2022 11:09:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4A66EC433C1; Thu, 13 Oct 2022 11:09:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1665659343; bh=mJSG3Ydey0D8JU11fE/SlVcGP6owZzI1lHl/NbqcfTo=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=je2mEfiKTkOYfvV0MhUFOdagygdELrWt1GjDnOiWcenKEQ8XqOMCovkIG88IO+vYi HcOakqRS0q8ILHHQrBqi/+hZ0LHgP+sgmXFSqqg7mcIppwSw16vyQ725u8YVH0jH// GqHZLEHJSWYMFgNU6jN2M1xd0htll1Xry/lkRqUoTNDLIDR8M6DiNeAJtoZFqvN32j 51PszE1f0DxJnYeRyZ88CQSrfCesVlhcDPMOwivHfeywoXJXLivrEcxhqLojCfLcvj RGkC51wFcwVyAxiaWLhabUvvHuTjOLr93siCuuz+3oyJEPCtV4wvHe97SX/Wf3ttGm z5yw8AJkTXhoQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oiw5F-00GHc3-0t; Thu, 13 Oct 2022 12:09:01 +0100 Date: Thu, 13 Oct 2022 12:09:00 +0100 Message-ID: <86fsfsf0ab.wl-maz@kernel.org> From: Marc Zyngier To: Denis Nikitin Subject: Re: [PATCH v2] KVM: arm64: nvhe: Fix build with profile optimization In-Reply-To: References: <20220920082005.2459826-1-denik@chromium.org> <20220922053145.944786-1-denik@chromium.org> <87h70zk83g.wl-maz@kernel.org> <87sfjxhh4k.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: denik@chromium.org, catalin.marinas@arm.com, will@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, ndesaulniers@google.com, dbrazdil@google.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, manojgupta@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Cc: Catalin Marinas , Nick Desaulniers , linux-kernel@vger.kernel.org, Manoj Gupta , Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Tue, 11 Oct 2022 03:15:36 +0100, Denis Nikitin wrote: > > On Sat, Oct 8, 2022 at 7:22 PM Marc Zyngier wrote: > > > > On Thu, 06 Oct 2022 17:28:17 +0100, > > Denis Nikitin wrote: > > > > > > Hi Mark, > > > > s/k/c/ > > > > > > > > This problem currently blocks the PGO roll on the ChromeOS kernel and > > > we need some kind of a solution. > > > > I'm sorry, but I don't feel constrained by your internal deadlines. I > > have my own... > > > > > Could you please take a look? > > > > I have asked for a reproducer. All I got for an answer is "this is > > hard". Providing a profiling file would help, for example. > > Could you please try the following profile on the 5.15 branch? > > $ cat < prof.txt > kvm_pgtable_walk:100:10 > 2: 5 > 3: 5 > 5: 5 > 6: 5 > 10: 5 > 10: _kvm_pgtable_walk:50 > 5: 5 > 7: 5 > 10: 5 > 13.2: 5 > 14: 5 > 16: 5 __kvm_pgtable_walk:5 > 13: kvm_pgd_page_idx:30 > 2: __kvm_pgd_page_idx:30 > 2: 5 > 3: 5 > 5: 5 > 2: kvm_granule_shift:5 > 3: 5 > EOF > > $ make LLVM=1 ARCH=arm64 KCFLAGS=-fprofile-sample-use=prof.txt -j8 vmlinux Thanks, this was helpful, as I was able to reproduce the build failure. FWIW, it seems pretty easy to work around by filtering out the offending option, making it consistent with the mechanism we already use for tracing and the like. I came up with the hack below, which does the trick and is IMHO better than dropping the section (extra work) or adding the negation of this option (which depends on the compiler option evaluation order). M. diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 48f6ae7cc6e6..7df1b6afca7f 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -91,7 +91,7 @@ quiet_cmd_hypcopy = HYPCOPY $@ # Remove ftrace, Shadow Call Stack, and CFI CFLAGS. # This is equivalent to the 'notrace', '__noscs', and '__nocfi' annotations. -KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI), $(KBUILD_CFLAGS)) +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI) -fprofile-sample-use=%, $(KBUILD_CFLAGS)) # KVM nVHE code is run at a different exception code with a different map, so # compiler instrumentation that inserts callbacks or checks into the code may -- Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 945A4C433FE for ; Thu, 13 Oct 2022 11:10:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Py7rx/Gp+ZKctCQzRjPdqUhjTfL/O8UB/t+CpmO0Sfs=; b=N81qn9lhPHmdhl AcVEMXwav1Kdu97FGaZlvDthrxgy0DkstUL4G5TvkTeTyywhFfWMt3zD+GcOfTS9IrlAosVhHwwX1 qDYHRpCseh+xblsLOT0+ulps+aVOk6JEMiy4Jbd2X937uE+odiR7qzQNI4Ndk6Y+DfVLx+J5gKhyK Stvm1Sqs29lXWL2Ids/Z1IUgOqRI9ntZj5/1CoarS2yuPEuMqQ4pmDztBEh8HOvb7YJseCoB2olGh PY3v8Pf7XMYl8QrXURuCEoivIOFRMw9Qp10kpPHXmy0VC9jtyo0cvfV0BM3k4QEkMDRjM0l1lE9xu A6hyO6bnWO4Q8cayAHGA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oiw5N-00BVha-2c; Thu, 13 Oct 2022 11:09:09 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oiw5J-00BVgO-Sf for linux-arm-kernel@lists.infradead.org; Thu, 13 Oct 2022 11:09:07 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 93E96B8162C; Thu, 13 Oct 2022 11:09:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4A66EC433C1; Thu, 13 Oct 2022 11:09:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1665659343; bh=mJSG3Ydey0D8JU11fE/SlVcGP6owZzI1lHl/NbqcfTo=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=je2mEfiKTkOYfvV0MhUFOdagygdELrWt1GjDnOiWcenKEQ8XqOMCovkIG88IO+vYi HcOakqRS0q8ILHHQrBqi/+hZ0LHgP+sgmXFSqqg7mcIppwSw16vyQ725u8YVH0jH// GqHZLEHJSWYMFgNU6jN2M1xd0htll1Xry/lkRqUoTNDLIDR8M6DiNeAJtoZFqvN32j 51PszE1f0DxJnYeRyZ88CQSrfCesVlhcDPMOwivHfeywoXJXLivrEcxhqLojCfLcvj RGkC51wFcwVyAxiaWLhabUvvHuTjOLr93siCuuz+3oyJEPCtV4wvHe97SX/Wf3ttGm z5yw8AJkTXhoQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oiw5F-00GHc3-0t; Thu, 13 Oct 2022 12:09:01 +0100 Date: Thu, 13 Oct 2022 12:09:00 +0100 Message-ID: <86fsfsf0ab.wl-maz@kernel.org> From: Marc Zyngier To: Denis Nikitin Cc: Catalin Marinas , Will Deacon , James Morse , Alexandru Elisei , Nick Desaulniers , David Brazdil , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, Manoj Gupta Subject: Re: [PATCH v2] KVM: arm64: nvhe: Fix build with profile optimization In-Reply-To: References: <20220920082005.2459826-1-denik@chromium.org> <20220922053145.944786-1-denik@chromium.org> <87h70zk83g.wl-maz@kernel.org> <87sfjxhh4k.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: denik@chromium.org, catalin.marinas@arm.com, will@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, ndesaulniers@google.com, dbrazdil@google.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, manojgupta@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221013_040906_236903_2747BD89 X-CRM114-Status: GOOD ( 26.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, 11 Oct 2022 03:15:36 +0100, Denis Nikitin wrote: > > On Sat, Oct 8, 2022 at 7:22 PM Marc Zyngier wrote: > > > > On Thu, 06 Oct 2022 17:28:17 +0100, > > Denis Nikitin wrote: > > > > > > Hi Mark, > > > > s/k/c/ > > > > > > > > This problem currently blocks the PGO roll on the ChromeOS kernel and > > > we need some kind of a solution. > > > > I'm sorry, but I don't feel constrained by your internal deadlines. I > > have my own... > > > > > Could you please take a look? > > > > I have asked for a reproducer. All I got for an answer is "this is > > hard". Providing a profiling file would help, for example. > > Could you please try the following profile on the 5.15 branch? > > $ cat < prof.txt > kvm_pgtable_walk:100:10 > 2: 5 > 3: 5 > 5: 5 > 6: 5 > 10: 5 > 10: _kvm_pgtable_walk:50 > 5: 5 > 7: 5 > 10: 5 > 13.2: 5 > 14: 5 > 16: 5 __kvm_pgtable_walk:5 > 13: kvm_pgd_page_idx:30 > 2: __kvm_pgd_page_idx:30 > 2: 5 > 3: 5 > 5: 5 > 2: kvm_granule_shift:5 > 3: 5 > EOF > > $ make LLVM=1 ARCH=arm64 KCFLAGS=-fprofile-sample-use=prof.txt -j8 vmlinux Thanks, this was helpful, as I was able to reproduce the build failure. FWIW, it seems pretty easy to work around by filtering out the offending option, making it consistent with the mechanism we already use for tracing and the like. I came up with the hack below, which does the trick and is IMHO better than dropping the section (extra work) or adding the negation of this option (which depends on the compiler option evaluation order). M. diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 48f6ae7cc6e6..7df1b6afca7f 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -91,7 +91,7 @@ quiet_cmd_hypcopy = HYPCOPY $@ # Remove ftrace, Shadow Call Stack, and CFI CFLAGS. # This is equivalent to the 'notrace', '__noscs', and '__nocfi' annotations. -KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI), $(KBUILD_CFLAGS)) +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI) -fprofile-sample-use=%, $(KBUILD_CFLAGS)) # KVM nVHE code is run at a different exception code with a different map, so # compiler instrumentation that inserts callbacks or checks into the code may -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66CBFC43217 for ; Thu, 13 Oct 2022 11:09:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229665AbiJMLJQ (ORCPT ); Thu, 13 Oct 2022 07:09:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48770 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229620AbiJMLJH (ORCPT ); Thu, 13 Oct 2022 07:09:07 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5257E110B0C for ; Thu, 13 Oct 2022 04:09:06 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id AFDCAB81E1D for ; Thu, 13 Oct 2022 11:09:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4A66EC433C1; Thu, 13 Oct 2022 11:09:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1665659343; bh=mJSG3Ydey0D8JU11fE/SlVcGP6owZzI1lHl/NbqcfTo=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=je2mEfiKTkOYfvV0MhUFOdagygdELrWt1GjDnOiWcenKEQ8XqOMCovkIG88IO+vYi HcOakqRS0q8ILHHQrBqi/+hZ0LHgP+sgmXFSqqg7mcIppwSw16vyQ725u8YVH0jH// GqHZLEHJSWYMFgNU6jN2M1xd0htll1Xry/lkRqUoTNDLIDR8M6DiNeAJtoZFqvN32j 51PszE1f0DxJnYeRyZ88CQSrfCesVlhcDPMOwivHfeywoXJXLivrEcxhqLojCfLcvj RGkC51wFcwVyAxiaWLhabUvvHuTjOLr93siCuuz+3oyJEPCtV4wvHe97SX/Wf3ttGm z5yw8AJkTXhoQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oiw5F-00GHc3-0t; Thu, 13 Oct 2022 12:09:01 +0100 Date: Thu, 13 Oct 2022 12:09:00 +0100 Message-ID: <86fsfsf0ab.wl-maz@kernel.org> From: Marc Zyngier To: Denis Nikitin Cc: Catalin Marinas , Will Deacon , James Morse , Alexandru Elisei , Nick Desaulniers , David Brazdil , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, Manoj Gupta Subject: Re: [PATCH v2] KVM: arm64: nvhe: Fix build with profile optimization In-Reply-To: References: <20220920082005.2459826-1-denik@chromium.org> <20220922053145.944786-1-denik@chromium.org> <87h70zk83g.wl-maz@kernel.org> <87sfjxhh4k.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: denik@chromium.org, catalin.marinas@arm.com, will@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, ndesaulniers@google.com, dbrazdil@google.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, manojgupta@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 11 Oct 2022 03:15:36 +0100, Denis Nikitin wrote: > > On Sat, Oct 8, 2022 at 7:22 PM Marc Zyngier wrote: > > > > On Thu, 06 Oct 2022 17:28:17 +0100, > > Denis Nikitin wrote: > > > > > > Hi Mark, > > > > s/k/c/ > > > > > > > > This problem currently blocks the PGO roll on the ChromeOS kernel and > > > we need some kind of a solution. > > > > I'm sorry, but I don't feel constrained by your internal deadlines. I > > have my own... > > > > > Could you please take a look? > > > > I have asked for a reproducer. All I got for an answer is "this is > > hard". Providing a profiling file would help, for example. > > Could you please try the following profile on the 5.15 branch? > > $ cat < prof.txt > kvm_pgtable_walk:100:10 > 2: 5 > 3: 5 > 5: 5 > 6: 5 > 10: 5 > 10: _kvm_pgtable_walk:50 > 5: 5 > 7: 5 > 10: 5 > 13.2: 5 > 14: 5 > 16: 5 __kvm_pgtable_walk:5 > 13: kvm_pgd_page_idx:30 > 2: __kvm_pgd_page_idx:30 > 2: 5 > 3: 5 > 5: 5 > 2: kvm_granule_shift:5 > 3: 5 > EOF > > $ make LLVM=1 ARCH=arm64 KCFLAGS=-fprofile-sample-use=prof.txt -j8 vmlinux Thanks, this was helpful, as I was able to reproduce the build failure. FWIW, it seems pretty easy to work around by filtering out the offending option, making it consistent with the mechanism we already use for tracing and the like. I came up with the hack below, which does the trick and is IMHO better than dropping the section (extra work) or adding the negation of this option (which depends on the compiler option evaluation order). M. diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 48f6ae7cc6e6..7df1b6afca7f 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -91,7 +91,7 @@ quiet_cmd_hypcopy = HYPCOPY $@ # Remove ftrace, Shadow Call Stack, and CFI CFLAGS. # This is equivalent to the 'notrace', '__noscs', and '__nocfi' annotations. -KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI), $(KBUILD_CFLAGS)) +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI) -fprofile-sample-use=%, $(KBUILD_CFLAGS)) # KVM nVHE code is run at a different exception code with a different map, so # compiler instrumentation that inserts callbacks or checks into the code may -- Without deviation from the norm, progress is not possible.