From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D922C32751 for ; Wed, 31 Jul 2019 16:13:51 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D5297206A3 for ; Wed, 31 Jul 2019 16:13:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="BhIxCuuq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D5297206A3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=VDFTLYmAGowouCRJBOU7+fZUWTdRmWed9vsyfIf4kSA=; b=BhIxCuuqBdvsWs foEdczfEKNIIwxAQu6858Gk2/00oyhwOrAH5y3jwzwzgNleukZ+yyaU7bwpT8qA/GVou22KXlxz/M Crkn7uFk6x/RNE66ZIuODp+CWufFPdSdoMogmbDGarZEcAA/Emo4I1usDWOboPCGx90YO+w7zPLi/ +FqjxjpX5HT+3yS1sGX1HOLxgRAlR4umTI2jipd6dsJqzYEdL9Ua6dfbrg8BKxEsSFDs22IpRRPjH s7yRpWCwtRqSh5AT+aopP1P2Sftaw6bvYDtpkuPO8HoJtQbGO7nhXjxWM74JU53o4izAy21PUyaV9 IgAvE4Re3KQx3frRomRg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hsrEc-0000cI-1Y; Wed, 31 Jul 2019 16:13:50 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hsrE0-00088n-Ue for linux-arm-kernel@lists.infradead.org; Wed, 31 Jul 2019 16:13:14 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A5CEF1576; Wed, 31 Jul 2019 09:13:12 -0700 (PDT) Received: from e119886-lin.cambridge.arm.com (unknown [10.37.6.20]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 45B3C3F71F; Wed, 31 Jul 2019 09:13:11 -0700 (PDT) From: Andrew Murray To: Catalin Marinas , Will Deacon , Peter Zijlstra , Ard.Biesheuvel@arm.com Subject: [PATCH v2 4/5] arm64: avoid using hard-coded registers for LSE atomics Date: Wed, 31 Jul 2019 17:12:55 +0100 Message-Id: <20190731161256.22210-5-andrew.murray@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190731161256.22210-1-andrew.murray@arm.com> References: <20190731161256.22210-1-andrew.murray@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190731_091313_080386_C61DF332 X-CRM114-Status: GOOD ( 11.10 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Boqun Feng , linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that we have removed the out-of-line ll/sc atomics we can give the compiler the freedom to choose its own register allocation. Let's remove the hard-coded use of x30. Signed-off-by: Andrew Murray --- arch/arm64/include/asm/atomic_lse.h | 70 +++++++++++++++++------------ 1 file changed, 41 insertions(+), 29 deletions(-) diff --git a/arch/arm64/include/asm/atomic_lse.h b/arch/arm64/include/asm/atomic_lse.h index 7dce5e1f074e..c6bd87d2915b 100644 --- a/arch/arm64/include/asm/atomic_lse.h +++ b/arch/arm64/include/asm/atomic_lse.h @@ -55,12 +55,14 @@ ATOMIC_FETCH_OPS(add, ldadd) #define ATOMIC_OP_ADD_RETURN(name, mb, cl...) \ static inline int __lse_atomic_add_return##name(int i, atomic_t *v) \ { \ + u32 tmp; \ + \ asm volatile( \ - " ldadd" #mb " %w[i], w30, %[v]\n" \ - " add %w[i], %w[i], w30" \ - : [i] "+r" (i), [v] "+Q" (v->counter) \ + " ldadd" #mb " %w[i], %w[tmp], %[v]\n" \ + " add %w[i], %w[i], %w[tmp]" \ + : [i] "+r" (i), [v] "+Q" (v->counter), [tmp] "=&r" (tmp) \ : "r" (v) \ - : "x30", ##cl); \ + : cl); \ \ return i; \ } @@ -113,13 +115,15 @@ static inline void __lse_atomic_sub(int i, atomic_t *v) #define ATOMIC_OP_SUB_RETURN(name, mb, cl...) \ static inline int __lse_atomic_sub_return##name(int i, atomic_t *v) \ { \ + u32 tmp; \ + \ asm volatile( \ " neg %w[i], %w[i]\n" \ - " ldadd" #mb " %w[i], w30, %[v]\n" \ - " add %w[i], %w[i], w30" \ - : [i] "+&r" (i), [v] "+Q" (v->counter) \ + " ldadd" #mb " %w[i], %w[tmp], %[v]\n" \ + " add %w[i], %w[i], %w[tmp]" \ + : [i] "+&r" (i), [v] "+Q" (v->counter), [tmp] "=&r" (tmp) \ : "r" (v) \ - : "x30", ##cl); \ + : cl); \ \ return i; \ } @@ -196,12 +200,14 @@ ATOMIC64_FETCH_OPS(add, ldadd) #define ATOMIC64_OP_ADD_RETURN(name, mb, cl...) \ static inline long __lse_atomic64_add_return##name(s64 i, atomic64_t *v)\ { \ + unsigned long tmp; \ + \ asm volatile( \ - " ldadd" #mb " %[i], x30, %[v]\n" \ - " add %[i], %[i], x30" \ - : [i] "+r" (i), [v] "+Q" (v->counter) \ + " ldadd" #mb " %[i], %x[tmp], %[v]\n" \ + " add %[i], %[i], %x[tmp]" \ + : [i] "+r" (i), [v] "+Q" (v->counter), [tmp] "=&r" (tmp) \ : "r" (v) \ - : "x30", ##cl); \ + : cl); \ \ return i; \ } @@ -254,13 +260,15 @@ static inline void __lse_atomic64_sub(s64 i, atomic64_t *v) #define ATOMIC64_OP_SUB_RETURN(name, mb, cl...) \ static inline long __lse_atomic64_sub_return##name(s64 i, atomic64_t *v) \ { \ + unsigned long tmp; \ + \ asm volatile( \ " neg %[i], %[i]\n" \ - " ldadd" #mb " %[i], x30, %[v]\n" \ - " add %[i], %[i], x30" \ - : [i] "+&r" (i), [v] "+Q" (v->counter) \ + " ldadd" #mb " %[i], %x[tmp], %[v]\n" \ + " add %[i], %[i], %x[tmp]" \ + : [i] "+&r" (i), [v] "+Q" (v->counter), [tmp] "=&r" (tmp) \ : "r" (v) \ - : "x30", ##cl); \ + : cl); \ \ return i; \ } @@ -294,18 +302,20 @@ ATOMIC64_FETCH_OP_SUB( , al, "memory") static inline s64 __lse_atomic64_dec_if_positive(atomic64_t *v) { + unsigned long tmp; + asm volatile( - "1: ldr x30, %[v]\n" - " subs %[ret], x30, #1\n" + "1: ldr %x[tmp], %[v]\n" + " subs %[ret], %x[tmp], #1\n" " b.lt 2f\n" - " casal x30, %[ret], %[v]\n" - " sub x30, x30, #1\n" - " sub x30, x30, %[ret]\n" - " cbnz x30, 1b\n" + " casal %x[tmp], %[ret], %[v]\n" + " sub %x[tmp], %x[tmp], #1\n" + " sub %x[tmp], %x[tmp], %[ret]\n" + " cbnz %x[tmp], 1b\n" "2:" - : [ret] "+&r" (v), [v] "+Q" (v->counter) + : [ret] "+&r" (v), [v] "+Q" (v->counter), [tmp] "=&r" (tmp) : - : "x30", "cc", "memory"); + : "cc", "memory"); return (long)v; } @@ -318,14 +328,16 @@ static inline u##sz __lse__cmpxchg_case_##name##sz(volatile void *ptr, \ register unsigned long x0 asm ("x0") = (unsigned long)ptr; \ register u##sz x1 asm ("x1") = old; \ register u##sz x2 asm ("x2") = new; \ + unsigned long tmp; \ \ asm volatile( \ - " mov " #w "30, %" #w "[old]\n" \ - " cas" #mb #sfx "\t" #w "30, %" #w "[new], %[v]\n" \ - " mov %" #w "[ret], " #w "30" \ - : [ret] "+r" (x0), [v] "+Q" (*(unsigned long *)ptr) \ + " mov %" #w "[tmp], %" #w "[old]\n" \ + " cas" #mb #sfx "\t%" #w "[tmp], %" #w "[new], %[v]\n" \ + " mov %" #w "[ret], %" #w "[tmp]" \ + : [ret] "+r" (x0), [v] "+Q" (*(unsigned long *)ptr), \ + [tmp] "=&r" (tmp) \ : [old] "r" (x1), [new] "r" (x2) \ - : "x30", ##cl); \ + : cl); \ \ return x0; \ } -- 2.21.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel