From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 89AA4C433EF for ; Tue, 5 Apr 2022 12:53:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=OhZIppYQA5161jWP7u5C0ib9cjzqREHcD74lKBoxQpY=; b=mZLYMZvH41hBz7 mqZHpYRuCQKvYkQXQ/z2uNFzLB3PUYyCxfwweyMEoUiE6wp2Als/vNZuafxeFFFLnjMrGykPOzED9 qaMVh33u3wDS+gFEvT6fDD/iKbn2HVMh6EFiU9ZU7gWxzZp5hk/ny/GTyIaN8y/oB9ptBzpUkrHmD 5k8si8xCKoJ/nd8q0IZs/UZgC8phhe2gFPMkSmcoNesGwlAjMtElrbGJ1SxlmDfag0lIPNbMJJdwz 678GZGi4eZzHdbl4SibJXvM9zrRhya2/Jxc600f0a+tG0+2JA39cdbZbEJ3ljvS1lo6J5z4xFCgj5 NwzIYVR4zqtY3pmXextg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbiex-0010qo-IQ; Tue, 05 Apr 2022 12:51:47 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbieu-0010q9-2y for linux-arm-kernel@lists.infradead.org; Tue, 05 Apr 2022 12:51:46 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 07A13D6E; Tue, 5 Apr 2022 05:51:42 -0700 (PDT) Received: from FVFF77S0Q05N (unknown [10.57.8.234]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5DFE03F5A1; Tue, 5 Apr 2022 05:51:39 -0700 (PDT) Date: Tue, 5 Apr 2022 13:51:30 +0100 From: Mark Rutland To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: linux-arch@vger.kernel.org, gcc@gcc.gnu.org, catalin.marinas@arm.com, will@kernel.org, marcan@marcan.st, maz@kernel.org, szabolcs.nagy@arm.com, f.fainelli@gmail.com, opendmb@gmail.com, Andrew Pinski , Ard Biesheuvel , Peter Zijlstra , x86@kernel.org, andrew.cooper3@citrix.com, Jeremy Linton Subject: GCC 12 miscompilation of volatile asm (was: Re: [PATCH] arm64/io: Remind compiler that there is a memory side effect) Message-ID: References: <20220401164406.61583-1-jeremy.linton@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_055144_270266_AD2A676E X-CRM114-Status: GOOD ( 13.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi all, [adding kernel folk who work on asm stuff] As a heads-up, GCC 12 (not yet released) appears to erroneously optimize away calls to functions with volatile asm. Szabolcs has raised an issue on the GCC bugzilla: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105160 ... which is a P1 release blocker, and is currently being investigated. Jemery originally reported this as an issue with {readl,writel}_relaxed(), but the underlying problem doesn't have anything to do with those specifically. I'm dumping a bunch of info here largely for posterity / archival, and to find out who (from the kernel side) is willing and able to test proposed compiler fixes, once those are available. I'm happy to do so for aarch64; Peter, I assume you'd be happy to look at the x86 side? This is a generic issue, and I wrote test cases for aarch64 and x86_64. Those are inline later in this mail, and currently you can see them on compiler explorer: aarch64: https://godbolt.org/z/vMczqjYvs x86_64: https://godbolt.org/z/cveff9hq5 My aarch64 test case is: | #define sysreg_read(regname) \ | ({ \ | unsigned long __sr_val; \ | asm volatile( \ | "mrs %0, " #regname "\n" \ | : "=r" (__sr_val)); \ | \ | __sr_val; \ | }) | | #define sysreg_write(regname, __sw_val) \ | do { \ | asm volatile( \ | "msr " #regname ", %0\n" \ | : \ | : "r" (__sw_val)); \ | } while (0) | | #define isb() \ | do { \ | asm volatile( \ | "isb" \ | : \ | : \ | : "memory"); \ | } while (0) | | static unsigned long sctlr_read(void) | { | return sysreg_read(sctlr_el1); | } | | static void sctlr_write(unsigned long val) | { | sysreg_write(sctlr_el1, val); | } | | static void sctlr_rmw(void) | { | unsigned long val; | | val = sctlr_read(); | val |= 1UL << 7; | sctlr_write(val); | } | | void sctlr_read_multiple(void) | { | sctlr_read(); | sctlr_read(); | sctlr_read(); | sctlr_read(); | } | | void sctlr_write_multiple(void) | { | sctlr_write(0); | sctlr_write(0); | sctlr_write(0); | sctlr_write(0); | sctlr_write(0); | } | | void sctlr_rmw_multiple(void) | { | sctlr_rmw(); | sctlr_rmw(); | sctlr_rmw(); | sctlr_rmw(); | } | | void function(void) | { | sctlr_read_multiple(); | sctlr_write_multiple(); | sctlr_rmw_multiple(); | | isb(); | } Per compiler explorer (https://godbolt.org/z/vMczqjYvs) GCC trunk currently compiles this as: | sctlr_rmw: | mrs x0, sctlr_el1 | orr x0, x0, 128 | msr sctlr_el1, x0 | ret | sctlr_read_multiple: | mrs x0, sctlr_el1 | mrs x0, sctlr_el1 | mrs x0, sctlr_el1 | mrs x0, sctlr_el1 | ret | sctlr_write_multiple: | mov x0, 0 | msr sctlr_el1, x0 | msr sctlr_el1, x0 | msr sctlr_el1, x0 | msr sctlr_el1, x0 | msr sctlr_el1, x0 | ret | sctlr_rmw_multiple: | ret | function: | isb | ret Whereas GCC 11.2 compiles this as: | sctlr_rmw: | mrs x0, sctlr_el1 | orr x0, x0, 128 | msr sctlr_el1, x0 | ret | sctlr_read_multiple: | mrs x0, sctlr_el1 | mrs x0, sctlr_el1 | mrs x0, sctlr_el1 | mrs x0, sctlr_el1 | ret | sctlr_write_multiple: | mov x0, 0 | msr sctlr_el1, x0 | msr sctlr_el1, x0 | msr sctlr_el1, x0 | msr sctlr_el1, x0 | msr sctlr_el1, x0 | ret | sctlr_rmw_multiple: | stp x29, x30, [sp, -16]! | mov x29, sp | bl sctlr_rmw | bl sctlr_rmw | bl sctlr_rmw | bl sctlr_rmw | ldp x29, x30, [sp], 16 | ret | function: | stp x29, x30, [sp, -16]! | mov x29, sp | bl sctlr_read_multiple | bl sctlr_write_multiple | bl sctlr_rmw_multiple | isb | ldp x29, x30, [sp], 16 | ret My x86_64 test case is: | unsigned long rdmsr(unsigned long reg) | { | unsigned int lo, hi; | | asm volatile( | "rdmsr" | : "=d" (hi), "=a" (lo) | : "c" (reg) | ); | | return ((unsigned long)hi << 32) | lo; | } | | void wrmsr(unsigned long reg, unsigned long val) | { | unsigned int lo = val; | unsigned int hi = val >> 32; | | asm volatile( | "wrmsr" | : | : "d" (hi), "a" (lo), "c" (reg) | ); | } | | void msr_rmw_set_bits(unsigned long reg, unsigned long bits) | { | unsigned long val; | | val = rdmsr(reg); | val |= bits; | wrmsr(reg, val); | } | | void func_with_msr_side_effects(unsigned long reg) | { | msr_rmw_set_bits(reg, 1UL << 0); | msr_rmw_set_bits(reg, 1UL << 1); | msr_rmw_set_bits(reg, 1UL << 2); | msr_rmw_set_bits(reg, 1UL << 3); | } Per compiler explorer (https://godbolt.org/z/cveff9hq5) GCC trunk currently compiles this as: | msr_rmw_set_bits: | mov rcx, rdi | rdmsr | sal rdx, 32 | mov eax, eax | or rax, rsi | or rax, rdx | mov rdx, rax | shr rdx, 32 | wrmsr | ret | func_with_msr_side_effects: | ret While GCC 11.2 compiles that as: | msr_rmw_set_bits: | mov rcx, rdi | rdmsr | sal rdx, 32 | mov eax, eax | or rax, rsi | or rax, rdx | mov rdx, rax | shr rdx, 32 | wrmsr | ret | func_with_msr_side_effects: | push rbp | push rbx | mov rbx, rdi | mov rbp, rsi | call msr_rmw_set_bits | mov rsi, rbp | mov rdi, rbx | call msr_rmw_set_bits | mov rsi, rbp | mov rdi, rbx | call msr_rmw_set_bits | mov rsi, rbp | mov rdi, rbx | call msr_rmw_set_bits | pop rbx | pop rbp | ret Thanks, Mark. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel