From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B61C9C47422 for ; Fri, 26 Jan 2024 16:59:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=M4qAsZke8qz/YuOFtAtA4+lrn0t04WOL6D8kTZzLMAA=; b=Wsy1IKs56wafHP G7+BfysWAyYeBLFV3HlapzuLT3kPlYPrXowiRJTLaHvgq3Mf5yJDUrzVMZiAbtnq+W82r3ZqIQY7w AhmmOkHp4R2t3/iw22RCyjvDK0TT8mtvPbu4zDEImw7VsSR4v7hqqic8mxb8hFaUHh9yeajGNDWm2 uI/VNsY1NiDbo9IXxKp/zE7URP9C9cu62AJok3o9sWIgbGUqcb+wDb1mk4m3MaOAmqUY9LaUlSWPb Se1SDh9P96W7flA5FT6VVCXCJWc1F90QuAGpNui7AuJfiyKxFZ2RGMGZFy7+kHV3Uhka7g071xDgt DtJ9Hv7lVNCMJaU+BCcQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rTPXo-00000004m58-37SB; Fri, 26 Jan 2024 16:59:08 +0000 Received: from sin.source.kernel.org ([145.40.73.55]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rTPXl-00000004m3a-3Dex for linux-arm-kernel@lists.infradead.org; Fri, 26 Jan 2024 16:59:07 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id B96F3CE3715; Fri, 26 Jan 2024 16:58:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 41C47C43609; Fri, 26 Jan 2024 16:58:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706288329; bh=woc6NP6/I5JGQMib9Xx4frAMCOAZe+QuU+N6MaJyDeI=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=raWhaQgXExrO6gcnVpugMnW9QUxUfEbahNJMGhy0HREMLQdiPe1zNVtCb5DAgUODD pFTkUhFF1mC0wlA3WE/gwJ3bYD621TRy/dAykKAQvgiotFKX77GcwnkATD36C0vTSX nJaXcgdRUDQML/vYkoqOxAE2fKi8t8CDBJTWiqCobJCIQRaHAAGjKo0mKSDIEkZ3V2 NDiB9QextWlqx3mpF16UsYTIKh5XDrYXBDXPgxiY0qvl8JxKi3KraL/K0MnMBtB1po 1SWkoER6muOpyl+XGF0tx46OUvRgT4C313YnyAaeuwqzKt3dNsWdXpcHY10L5uqWZt qKyCKRL6PwJhw== Date: Fri, 26 Jan 2024 09:58:47 -0700 From: Nathan Chancellor To: Nafees Ahmed Abdul Cc: catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org Subject: Re: [RFC] arm64: use different compiler inlining options for arm64 kernel builds Message-ID: <20240126165847.GA4116262@dev-arch.thelio-3990X> References: <20240126161330.1483-1-nafeabd@amazon.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240126161330.1483-1-nafeabd@amazon.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240126_085906_187993_8BF75232 X-CRM114-Status: GOOD ( 30.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Jan 26, 2024 at 04:13:30PM +0000, Nafees Ahmed Abdul wrote: > It was observed that compiling the kernel with following compiler inlining > options has showed performance improvement up to 4% across tested > workloads. > > Below is the table with performance gains compared to default kernel > on few of the workloads we tested. > > | Workload | Improvement | > |-----------|-------------| > | Nginx | 4% | > | MongoDB | 3% | > | Memcached | 3% | > | Redis | 2% | > > The selected options increase inlining of the kernel functions which is > positively impacting workload performance. We have arrived at these values > by reviewing the options used with the gcc compiler at various optimization > levels such as O1, O2 and O3. > > Benchmarking was performed on AWS Graviton 2/3 generation instances > on 5.10 and 6.1 longterm supported kernels with GCC versions 10 and 11. These values will be applied to every single arm64 kernel build, regardless of hardware and GCC version (the kernel supports GCC 5.1.0 as a minimum version), are you sure that these values won't regress other people's workloads? Furthermore, this basically ties the arm64 kernel build to GCC internals, which the documentation notes "The names of specific parameters, and the meaning of the values, are tied to the internals of the compiler, and are subject to change without notice in future releases.". This does not really feel like something the upstream kernel should take but I am not an arm64 maintainer so I can't comment further. > Patch also adds noinline attribute to couple of functions for existing > kselftests to pass on ftrace target. That seems like a bandaid to me and probably warrants further investigation/explanation if adding these flags is the only thing that it takes to expose that, as it means there is either a latent compiler bug or the kernel has some dodgy code that happens to work now but some future internal compiler change could expose. > Signed-off-by: Nafees Ahmed Abdul > --- > arch/arm64/Makefile | 6 ++++++ > fs/namei.c | 2 +- > fs/open.c | 2 +- > 3 files changed, 8 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile > index 47ecc4cff9d2..4ad2c7d0008d 100644 > --- a/arch/arm64/Makefile > +++ b/arch/arm64/Makefile > @@ -101,6 +101,12 @@ KBUILD_CFLAGS += -Wa,-march=$(asm-arch) \ > -DARM64_ASM_ARCH='"$(asm-arch)"' > endif > > +ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE > +KBUILD_CFLAGS += --param=early-inlining-insns=14 --param=inline-heuristics-hint-percent=600 \ > + --param=inline-min-speedup=15 --param=max-inline-insns-auto=30 \ > + --param=max-inline-insns-single=200 > +endif Regardless, the Linux kernel can also be built with clang, so this block needs to have a dependency on CONFIG_CC_IS_GCC as well, but I'd wait to send any follow up before others have a chance to comment on that. > + > ifeq ($(CONFIG_SHADOW_CALL_STACK), y) > KBUILD_CFLAGS += -ffixed-x18 > endif > diff --git a/fs/namei.c b/fs/namei.c > index 71c13b2990b4..58e8f462c137 100644 > --- a/fs/namei.c > +++ b/fs/namei.c > @@ -4445,7 +4445,7 @@ SYSCALL_DEFINE1(unlink, const char __user *, pathname) > * On non-idmapped mounts or if permission checking is to be performed on the > * raw inode simply passs @nop_mnt_idmap. > */ > -int vfs_symlink(struct mnt_idmap *idmap, struct inode *dir, > +noinline vfs_symlink(struct mnt_idmap *idmap, struct inode *dir, > struct dentry *dentry, const char *oldname) > { > int error; > diff --git a/fs/open.c b/fs/open.c > index 02dc608d40d8..afec5d0e30d3 100644 > --- a/fs/open.c > +++ b/fs/open.c > @@ -1421,7 +1421,7 @@ struct file *file_open_root(const struct path *root, > } > EXPORT_SYMBOL(file_open_root); > > -static long do_sys_openat2(int dfd, const char __user *filename, > +static noinline long do_sys_openat2(int dfd, const char __user *filename, > struct open_how *how) > { > struct open_flags op; > -- > 2.40.1 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel