From: guoren@kernel.org
To: guoren@kernel.org
Cc: linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org,
Guo Ren <guoren@linux.alibaba.com>,
Peter Zijlstra <peterz@infradead.org>,
Arnd Bergmann <arnd@arndb.de>,
"Paul E . McKenney" <paulmck@kernel.org>
Subject: [PATCH 07/29] csky: Fixup asm/cmpxchg.h with correct ordering barrier
Date: Thu, 21 Jan 2021 14:53:27 +0800 [thread overview]
Message-ID: <20210121065349.3188251-7-guoren@kernel.org> (raw)
In-Reply-To: <20210121065349.3188251-1-guoren@kernel.org>
From: Guo Ren <guoren@linux.alibaba.com>
Optimize the performance of cmpxchg by using more fine-grained
acquire/release barriers.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Paul E. McKenney <paulmck@kernel.org>
---
arch/csky/include/asm/cmpxchg.h | 27 +++++++++++++++++----------
1 file changed, 17 insertions(+), 10 deletions(-)
diff --git a/arch/csky/include/asm/cmpxchg.h b/arch/csky/include/asm/cmpxchg.h
index 89224530a0ee..dabc8e46ce7b 100644
--- a/arch/csky/include/asm/cmpxchg.h
+++ b/arch/csky/include/asm/cmpxchg.h
@@ -3,12 +3,12 @@
#ifndef __ASM_CSKY_CMPXCHG_H
#define __ASM_CSKY_CMPXCHG_H
-#ifdef CONFIG_CPU_HAS_LDSTEX
+#ifdef CONFIG_SMP
#include <asm/barrier.h>
extern void __bad_xchg(void);
-#define __xchg(new, ptr, size) \
+#define __xchg_relaxed(new, ptr, size) \
({ \
__typeof__(ptr) __ptr = (ptr); \
__typeof__(new) __new = (new); \
@@ -16,7 +16,6 @@ extern void __bad_xchg(void);
unsigned long tmp; \
switch (size) { \
case 4: \
- smp_mb(); \
asm volatile ( \
"1: ldex.w %0, (%3) \n" \
" mov %1, %2 \n" \
@@ -25,7 +24,6 @@ extern void __bad_xchg(void);
: "=&r" (__ret), "=&r" (tmp) \
: "r" (__new), "r"(__ptr) \
:); \
- smp_mb(); \
break; \
default: \
__bad_xchg(); \
@@ -33,9 +31,10 @@ extern void __bad_xchg(void);
__ret; \
})
-#define xchg(ptr, x) (__xchg((x), (ptr), sizeof(*(ptr))))
+#define xchg_relaxed(ptr, x) \
+ (__xchg_relaxed((x), (ptr), sizeof(*(ptr))))
-#define __cmpxchg(ptr, old, new, size) \
+#define __cmpxchg_relaxed(ptr, old, new, size) \
({ \
__typeof__(ptr) __ptr = (ptr); \
__typeof__(new) __new = (new); \
@@ -44,7 +43,6 @@ extern void __bad_xchg(void);
__typeof__(*(ptr)) __ret; \
switch (size) { \
case 4: \
- smp_mb(); \
asm volatile ( \
"1: ldex.w %0, (%3) \n" \
" cmpne %0, %4 \n" \
@@ -56,7 +54,6 @@ extern void __bad_xchg(void);
: "=&r" (__ret), "=&r" (__tmp) \
: "r" (__new), "r"(__ptr), "r"(__old) \
:); \
- smp_mb(); \
break; \
default: \
__bad_xchg(); \
@@ -64,8 +61,18 @@ extern void __bad_xchg(void);
__ret; \
})
-#define cmpxchg(ptr, o, n) \
- (__cmpxchg((ptr), (o), (n), sizeof(*(ptr))))
+#define cmpxchg_relaxed(ptr, o, n) \
+ (__cmpxchg_relaxed((ptr), (o), (n), sizeof(*(ptr))))
+
+#define cmpxchg(ptr, o, n) \
+({ \
+ __typeof__(*(ptr)) __ret; \
+ __smp_release_fence(); \
+ __ret = cmpxchg_relaxed(ptr, o, n); \
+ __smp_acquire_fence(); \
+ __ret; \
+})
+
#else
#include <asm-generic/cmpxchg.h>
#endif
--
2.17.1
next prev parent reply other threads:[~2021-01-21 20:39 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-21 6:53 [PATCH 01/29] csky: Add memory layout 2.5G(user):1.5G(kernel) guoren
2021-01-21 6:53 ` [PATCH 02/29] csky: Fixup perf probe failed guoren
2021-01-21 6:53 ` [PATCH 03/29] csky: Fixup show_regs doesn't contain regs->usp guoren
2021-01-21 6:53 ` [PATCH 04/29] csky: Remove custom asm/atomic.h implementation guoren
2021-01-21 6:53 ` [PATCH 05/29] csky: Fixup barrier design guoren
2021-01-21 6:53 ` [PATCH 06/29] csky: Fixup futex SMP implementation guoren
2021-01-21 6:53 ` guoren [this message]
2021-01-21 6:53 ` [PATCH 08/29] csky: Cleanup asm/spinlock.h guoren
2021-01-21 6:53 ` [PATCH 09/29] csky: Fixup PTE global for 2.5:1.5 virtual memory guoren
2021-01-21 6:53 ` [PATCH 10/29] csky: Remove prologue of page fault handler in entry.S guoren
2021-01-21 6:53 ` [PATCH 11/29] csky: Add kmemleak support guoren
2021-01-21 6:53 ` [PATCH 12/29] csky: Fix TLB maintenance synchronization problem guoren
2021-01-21 6:53 ` [PATCH 13/29] csky: Add show_tlb for CPU_CK860 debug guoren
2021-01-21 6:53 ` [PATCH 14/29] csky: Fixup FAULT_FLAG_XXX param for handle_mm_fault guoren
2021-01-21 6:53 ` [PATCH 15/29] csky: Fixup update_mmu_cache called with user io mapping guoren
2021-01-21 6:53 ` [PATCH 16/29] csky: Add faulthandler_disabled() check guoren
2021-01-21 6:53 ` [PATCH 17/29] csky: Fixup do_page_fault parent irq status guoren
2021-01-21 6:53 ` [PATCH 18/29] csky: Sync riscv mm/fault.c for easy maintenance guoren
2021-01-21 6:53 ` [PATCH 19/29] csky: mm: abort uaccess retries upon fatal signal guoren
2021-01-21 11:42 ` Mark Rutland
2021-01-21 6:53 ` [PATCH 20/29] csky: Reconstruct VDSO framework guoren
2021-01-21 6:53 ` [PATCH 21/29] csky: Fix a size determination in gpr_get() guoren
2021-01-21 6:53 ` [PATCH 22/29] csky: remove unused including <linux/version.h> guoren
2021-01-21 6:53 ` [PATCH 23/29] csky: Fixup _PAGE_ACCESSED for default pgprot guoren
2021-01-21 6:53 ` [PATCH 24/29] csky: Coding convention del unnecessary definition guoren
2021-01-21 6:53 ` [PATCH 25/29] csky: Fixup swapon guoren
2021-01-21 6:53 ` [PATCH 26/29] csky: kprobe: fix code in simulate without 'long' guoren
2021-01-21 6:53 ` [PATCH 27/29] csky: Add VDSO with GENERIC_GETTIMEOFDAY, GENERIC_TIME_VSYSCALL, HAVE_GENERIC_VDSO guoren
2021-01-21 6:53 ` [PATCH 28/29] csky: Using set_max_mapnr api guoren
2021-01-21 6:53 ` [PATCH 29/29] csky: Fixup pfn_valid error with wrong max_mapnr guoren
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210121065349.3188251-7-guoren@kernel.org \
--to=guoren@kernel.org \
--cc=arnd@arndb.de \
--cc=guoren@linux.alibaba.com \
--cc=linux-csky@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).