[PATCH 0/6] Enable queued rwlock and queued spinlock for SPARC

[PATCH 0/6] Enable queued rwlock and queued spinlock for SPARC

[Babu Moger]

This series of patches enables queued rwlock and queued spinlock support
for SPARC. These features were introduced some time ago in upstream.
Here are some of the earlier discussions.
https://lwn.net/Articles/572765/
https://lwn.net/Articles/582200/
https://lwn.net/Articles/561775/
https://lwn.net/Articles/590243/

Tests: Ran AIM7 benchmark to verify the performance on various workloads.
https://github.com/davidlohr/areaim. Same benchmark was used when this
feature was introduced and enabled on x86. Here are the test results.

Kernel				4.11.0-rc6     4.11.0-rc6 + 	Change
				baseline	queued locks
			      (Avg No.of jobs) (Avg No.of jobs)
Workload
High systime 10-100 user	 17290.48	 17295.18	+0.02
High systime 200-1000 users	109814.95	110248.87	+0.39
High systime 1200-2000 users	107912.40	127923.16	+18.54

Disk IO 10-100 users		168910.16	158834.17	-5.96
Disk IO 200-1000 users		242781.74	281285.80	+15.85
Disk IO 1200-2000 users		228518.23	218421.23	-4.41

Disk IO 10-100 users		183933.77	207928.67	+13.04
Disk IO 200-1000 users		491981.56	500162.33	+1.66
Disk IO 1200-2000 users		463395.66	467312.70	+0.84

fserver 10-100 users		254177.53	270283.08	+6.33
fserver IO 200-1000 users	269017.35	324812.2	+20.74
fserver IO 1200-2000 users	229538.87	284713.77	+24.03

Disk I/O results are little bit in negative territory. But majority of the 
performance changes are in positive and it is significant in some cases.

Babu Moger (6):
  kernel/locking: Fix compile error with qrwlock.c
  arch/sparc: Define config parameter CPU_BIG_ENDIAN
  arch/sparc: Introduce cmpxchg_u8 SPARC
  arch/sparc: Enable queued rwlocks for SPARC
  arch/sparc: Introduce xchg16 for SPARC
  arch/sparc: Enable queued spinlock support for SPARC

 arch/sparc/Kconfig                      |    6 +++
 arch/sparc/include/asm/cmpxchg_64.h     |   76 +++++++++++++++++++++++++++----
 arch/sparc/include/asm/qrwlock.h        |    7 +++
 arch/sparc/include/asm/qspinlock.h      |    7 +++
 arch/sparc/include/asm/spinlock_64.h    |   13 ++++-
 arch/sparc/include/asm/spinlock_types.h |   10 ++++-
 include/asm-generic/qrwlock_types.h     |    6 +-
 kernel/locking/qrwlock.c                |    1 +
 8 files changed, 110 insertions(+), 16 deletions(-)
 create mode 100644 arch/sparc/include/asm/qrwlock.h
 create mode 100644 arch/sparc/include/asm/qspinlock.h

[PATCH 0/6] Enable queued rwlock and queued spinlock for SPARC

[Babu Moger]

This series of patches enables queued rwlock and queued spinlock support
for SPARC. These features were introduced some time ago in upstream.
Here are some of the earlier discussions.
https://lwn.net/Articles/572765/
https://lwn.net/Articles/582200/
https://lwn.net/Articles/561775/
https://lwn.net/Articles/590243/

Tests: Ran AIM7 benchmark to verify the performance on various workloads.
https://github.com/davidlohr/areaim. Same benchmark was used when this
feature was introduced and enabled on x86. Here are the test results.

Kernel				4.11.0-rc6     4.11.0-rc6 + 	Change
				baseline	queued locks
			      (Avg No.of jobs) (Avg No.of jobs)
Workload
High systime 10-100 user	 17290.48	 17295.18	+0.02
High systime 200-1000 users	109814.95	110248.87	+0.39
High systime 1200-2000 users	107912.40	127923.16	+18.54

Disk IO 10-100 users		168910.16	158834.17	-5.96
Disk IO 200-1000 users		242781.74	281285.80	+15.85
Disk IO 1200-2000 users		228518.23	218421.23	-4.41

Disk IO 10-100 users		183933.77	207928.67	+13.04
Disk IO 200-1000 users		491981.56	500162.33	+1.66
Disk IO 1200-2000 users		463395.66	467312.70	+0.84

fserver 10-100 users		254177.53	270283.08	+6.33
fserver IO 200-1000 users	269017.35	324812.2	+20.74
fserver IO 1200-2000 users	229538.87	284713.77	+24.03

Disk I/O results are little bit in negative territory. But majority of the 
performance changes are in positive and it is significant in some cases.

Babu Moger (6):
  kernel/locking: Fix compile error with qrwlock.c
  arch/sparc: Define config parameter CPU_BIG_ENDIAN
  arch/sparc: Introduce cmpxchg_u8 SPARC
  arch/sparc: Enable queued rwlocks for SPARC
  arch/sparc: Introduce xchg16 for SPARC
  arch/sparc: Enable queued spinlock support for SPARC

 arch/sparc/Kconfig                      |    6 +++
 arch/sparc/include/asm/cmpxchg_64.h     |   76 +++++++++++++++++++++++++++----
 arch/sparc/include/asm/qrwlock.h        |    7 +++
 arch/sparc/include/asm/qspinlock.h      |    7 +++
 arch/sparc/include/asm/spinlock_64.h    |   13 ++++-
 arch/sparc/include/asm/spinlock_types.h |   10 ++++-
 include/asm-generic/qrwlock_types.h     |    6 +-
 kernel/locking/qrwlock.c                |    1 +
 8 files changed, 110 insertions(+), 16 deletions(-)
 create mode 100644 arch/sparc/include/asm/qrwlock.h
 create mode 100644 arch/sparc/include/asm/qspinlock.h

[PATCH 1/6] kernel/locking: Fix compile error with qrwlock.c

[Babu Moger]

Seeing this error on SPARC while compiling qrwlock.c.

  CC      kernel/locking/qrwlock.o
In file included from ./include/asm-generic/qrwlock_types.h:5,
                 from ./arch/sparc/include/asm/qrwlock.h:4,
                 from kernel/locking/qrwlock.c:24:
./arch/sparc/include/asm/spinlock_types.h:5:3: error:
        #error "please don't include this file directly"

Re-arrange the includes in qrwlock_types.h and also include spinlock.h
in qrwlock.c to fix it. This should also help other architectures when
queued rwlock is enabled.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 include/asm-generic/qrwlock_types.h |    6 +++---
 kernel/locking/qrwlock.c            |    1 +
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/include/asm-generic/qrwlock_types.h b/include/asm-generic/qrwlock_types.h
index 0abc6b6..3988c7d 100644
--- a/include/asm-generic/qrwlock_types.h
+++ b/include/asm-generic/qrwlock_types.h
@@ -1,9 +1,6 @@
 #ifndef __ASM_GENERIC_QRWLOCK_TYPES_H
 #define __ASM_GENERIC_QRWLOCK_TYPES_H
 
-#include <linux/types.h>
-#include <asm/spinlock_types.h>
-
 /*
  * The queue read/write lock data structure
  */
@@ -18,4 +15,7 @@
 	.wait_lock = __ARCH_SPIN_LOCK_UNLOCKED,	\
 }
 
+#include <linux/types.h>
+#include <linux/spinlock_types.h>
+
 #endif /* __ASM_GENERIC_QRWLOCK_TYPES_H */
diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c
index cc3ed0c..2655f26 100644
--- a/kernel/locking/qrwlock.c
+++ b/kernel/locking/qrwlock.c
@@ -20,6 +20,7 @@
 #include <linux/cpumask.h>
 #include <linux/percpu.h>
 #include <linux/hardirq.h>
+#include <linux/spinlock.h>
 #include <asm/qrwlock.h>
 
 /*
-- 
1.7.1

[PATCH 1/6] kernel/locking: Fix compile error with qrwlock.c

[Babu Moger]

Seeing this error on SPARC while compiling qrwlock.c.

  CC      kernel/locking/qrwlock.o
In file included from ./include/asm-generic/qrwlock_types.h:5,
                 from ./arch/sparc/include/asm/qrwlock.h:4,
                 from kernel/locking/qrwlock.c:24:
./arch/sparc/include/asm/spinlock_types.h:5:3: error:
        #error "please don't include this file directly"

Re-arrange the includes in qrwlock_types.h and also include spinlock.h
in qrwlock.c to fix it. This should also help other architectures when
queued rwlock is enabled.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 include/asm-generic/qrwlock_types.h |    6 +++---
 kernel/locking/qrwlock.c            |    1 +
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/include/asm-generic/qrwlock_types.h b/include/asm-generic/qrwlock_types.h
index 0abc6b6..3988c7d 100644
--- a/include/asm-generic/qrwlock_types.h
+++ b/include/asm-generic/qrwlock_types.h
@@ -1,9 +1,6 @@
 #ifndef __ASM_GENERIC_QRWLOCK_TYPES_H
 #define __ASM_GENERIC_QRWLOCK_TYPES_H
 
-#include <linux/types.h>
-#include <asm/spinlock_types.h>
-
 /*
  * The queue read/write lock data structure
  */
@@ -18,4 +15,7 @@
 	.wait_lock = __ARCH_SPIN_LOCK_UNLOCKED,	\
 }
 
+#include <linux/types.h>
+#include <linux/spinlock_types.h>
+
 #endif /* __ASM_GENERIC_QRWLOCK_TYPES_H */
diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c
index cc3ed0c..2655f26 100644
--- a/kernel/locking/qrwlock.c
+++ b/kernel/locking/qrwlock.c
@@ -20,6 +20,7 @@
 #include <linux/cpumask.h>
 #include <linux/percpu.h>
 #include <linux/hardirq.h>
+#include <linux/spinlock.h>
 #include <asm/qrwlock.h>
 
 /*
-- 
1.7.1

Re: [PATCH 1/6] kernel/locking: Fix compile error with qrwlock.c

[David Miller]

From: Babu Moger <babu.moger@oracle.com>
Date: Thu, 18 May 2017 18:36:05 -0600

> Seeing this error on SPARC while compiling qrwlock.c.
> 
>   CC      kernel/locking/qrwlock.o
> In file included from ./include/asm-generic/qrwlock_types.h:5,
>                  from ./arch/sparc/include/asm/qrwlock.h:4,
>                  from kernel/locking/qrwlock.c:24:
> ./arch/sparc/include/asm/spinlock_types.h:5:3: error:
>         #error "please don't include this file directly"
> 
> Re-arrange the includes in qrwlock_types.h and also include spinlock.h
> in qrwlock.c to fix it. This should also help other architectures when
> queued rwlock is enabled.
> 
> Signed-off-by: Babu Moger <babu.moger@oracle.com>
> Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
> Reviewed-by: Jane Chu <jane.chu@oracle.com>
> Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
> Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>

I think you can simply remove the:

#ifndef __LINUX_SPINLOCK_TYPES_H
# error "please don't include this file directly"
#endif

stanza from the sparc header file instead.  Other architectures
don't use this guard.

Re: [PATCH 1/6] kernel/locking: Fix compile error with qrwlock.c

[David Miller]

From: Babu Moger <babu.moger@oracle.com>
Date: Thu, 18 May 2017 18:36:05 -0600

> Seeing this error on SPARC while compiling qrwlock.c.
> 
>   CC      kernel/locking/qrwlock.o
> In file included from ./include/asm-generic/qrwlock_types.h:5,
>                  from ./arch/sparc/include/asm/qrwlock.h:4,
>                  from kernel/locking/qrwlock.c:24:
> ./arch/sparc/include/asm/spinlock_types.h:5:3: error:
>         #error "please don't include this file directly"
> 
> Re-arrange the includes in qrwlock_types.h and also include spinlock.h
> in qrwlock.c to fix it. This should also help other architectures when
> queued rwlock is enabled.
> 
> Signed-off-by: Babu Moger <babu.moger@oracle.com>
> Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
> Reviewed-by: Jane Chu <jane.chu@oracle.com>
> Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
> Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>

I think you can simply remove the:

#ifndef __LINUX_SPINLOCK_TYPES_H
# error "please don't include this file directly"
#endif

stanza from the sparc header file instead.  Other architectures
don't use this guard.

Re: [PATCH 1/6] kernel/locking: Fix compile error with qrwlock.c

[Babu Moger]

On 5/18/2017 9:27 PM, David Miller wrote:
> From: Babu Moger <babu.moger@oracle.com>
> Date: Thu, 18 May 2017 18:36:05 -0600
>
>> Seeing this error on SPARC while compiling qrwlock.c.
>>
>>    CC      kernel/locking/qrwlock.o
>> In file included from ./include/asm-generic/qrwlock_types.h:5,
>>                   from ./arch/sparc/include/asm/qrwlock.h:4,
>>                   from kernel/locking/qrwlock.c:24:
>> ./arch/sparc/include/asm/spinlock_types.h:5:3: error:
>>          #error "please don't include this file directly"
>>
>> Re-arrange the includes in qrwlock_types.h and also include spinlock.h
>> in qrwlock.c to fix it. This should also help other architectures when
>> queued rwlock is enabled.
>>
>> Signed-off-by: Babu Moger <babu.moger@oracle.com>
>> Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
>> Reviewed-by: Jane Chu <jane.chu@oracle.com>
>> Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
>> Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
> I think you can simply remove the:
>
> #ifndef __LINUX_SPINLOCK_TYPES_H
> # error "please don't include this file directly"
> #endif
>
> stanza from the sparc header file instead.  Other architectures
> don't use this guard.

Sure. I can remove this for SPARC.  Will make this as a separate patch.
But, there are still some architectures use this gaurd.

arch/xtensa/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H
arch/arm/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H
arch/powerpc/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H
arch/metag/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H
arch/hexagon/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H
arch/alpha/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H
arch/sh/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H
arch/mn10300/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H
arch/ia64/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H
arch/blackfin/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H
arch/m32r/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H
arch/s390/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H
arch/arm64/include/asm/spinlock_types.h:#if 
!defined(__LINUX_SPINLOCK_TYPES_H) && !defined(__ASM_SPINLOCK_H)
arch/tile/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H
arch/mips/include/asm/spinlock_types.h:#ifndef __LINUX_SPINLOCK_TYPES_H

I think we need  still need this patch as it re-arrages headers 
properly. Will send  v2 series soon.

[PATCH 2/6] arch/sparc: Define config parameter CPU_BIG_ENDIAN

[Babu Moger]

Found this problem while enabling queued rwlock on SPARC.
The parameter CONFIG_CPU_BIG_ENDIAN is used to clear the
specific byte in qrwlock structure. Without this parameter,
we clear the wrong byte. Here is the code.

static inline u8 *__qrwlock_write_byte(struct qrwlock *lock)
 {
	return (u8 *)lock + 3 * IS_BUILTIN(CONFIG_CPU_BIG_ENDIAN);
 }

Define CPU_BIG_ENDIAN for SPARC to fix it.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/Kconfig |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index a2ad946..8787fc4 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -91,6 +91,10 @@ config ARCH_DEFCONFIG
 config ARCH_PROC_KCORE_TEXT
 	def_bool y
 
+config CPU_BIG_ENDIAN
+	bool
+	default y if SPARC64
+
 config ARCH_ATU
 	bool
 	default y if SPARC64
-- 
1.7.1

[PATCH 2/6] arch/sparc: Define config parameter CPU_BIG_ENDIAN

[Babu Moger]

Found this problem while enabling queued rwlock on SPARC.
The parameter CONFIG_CPU_BIG_ENDIAN is used to clear the
specific byte in qrwlock structure. Without this parameter,
we clear the wrong byte. Here is the code.

static inline u8 *__qrwlock_write_byte(struct qrwlock *lock)
 {
	return (u8 *)lock + 3 * IS_BUILTIN(CONFIG_CPU_BIG_ENDIAN);
 }

Define CPU_BIG_ENDIAN for SPARC to fix it.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/Kconfig |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index a2ad946..8787fc4 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -91,6 +91,10 @@ config ARCH_DEFCONFIG
 config ARCH_PROC_KCORE_TEXT
 	def_bool y
 
+config CPU_BIG_ENDIAN
+	bool
+	default y if SPARC64
+
 config ARCH_ATU
 	bool
 	default y if SPARC64
-- 
1.7.1

Re: [PATCH 2/6] arch/sparc: Define config parameter CPU_BIG_ENDIAN

[David Miller]

From: Babu Moger <babu.moger@oracle.com>
Date: Thu, 18 May 2017 18:36:06 -0600

> @@ -91,6 +91,10 @@ config ARCH_DEFCONFIG
>  config ARCH_PROC_KCORE_TEXT
>  	def_bool y
>  
> +config CPU_BIG_ENDIAN
> +	bool
> +	default y if SPARC64
> +

32-bit sparc is big-endian too :-)

So please don't use SPARC64 for the setting, use plain SPARC instead.

Re: [PATCH 2/6] arch/sparc: Define config parameter CPU_BIG_ENDIAN

[Babu Moger]

On 5/18/2017 9:26 PM, David Miller wrote:
> From: Babu Moger <babu.moger@oracle.com>
> Date: Thu, 18 May 2017 18:36:06 -0600
>
>> @@ -91,6 +91,10 @@ config ARCH_DEFCONFIG
>>   config ARCH_PROC_KCORE_TEXT
>>   	def_bool y
>>   
>> +config CPU_BIG_ENDIAN
>> +	bool
>> +	default y if SPARC64
>> +
> 32-bit sparc is big-endian too :-)
>
> So please don't use SPARC64 for the setting, use plain SPARC instead.

Ok. Sure. Will do.

[PATCH 3/6] arch/sparc: Introduce cmpxchg_u8 SPARC

[Babu Moger]

SPARC supports 32 bit and 64 bit cmpxchg right now. Add support
for 8 bit (1 byte) cmpxchg. This is required to support queued
rwlocks feature  which uses 1 byte cmpxchg.

The function __cmpxchg_u8 here uses the 4 byte cas instruction with a
byte manipulation to achieve 1 byte cmpxchg.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Steve Sistare <steven.sistare@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/include/asm/cmpxchg_64.h |   29 +++++++++++++++++++++++++++++
 1 files changed, 29 insertions(+), 0 deletions(-)

diff --git a/arch/sparc/include/asm/cmpxchg_64.h b/arch/sparc/include/asm/cmpxchg_64.h
index faa2f61..65bfae0 100644
--- a/arch/sparc/include/asm/cmpxchg_64.h
+++ b/arch/sparc/include/asm/cmpxchg_64.h
@@ -87,6 +87,33 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr,
 	return new;
 }
 
+/*
+ * Use 4 byte cas instruction to achieve 1 byte cmpxchg. Main logic
+ * here is to get the bit shift of the byte we are interested in.
+ * The XOR is handy for reversing the bits for big-endian byte order
+ */
+static inline unsigned long
+__cmpxchg_u8(volatile unsigned char *m, unsigned int old, unsigned int new)
+{
+	unsigned long maddr = (unsigned long)m;
+	int bit_shift = (((unsigned long)m & 3) ^ 3) << 3;
+	unsigned int mask = 0xff << bit_shift;
+	unsigned int *ptr = (unsigned int *) (maddr & ~3);
+	unsigned int old32, new32, load;
+	unsigned int load32 = *ptr;
+
+	do {
+		new32 = (load32 & ~mask) | (new << bit_shift);
+		old32 = (load32 & ~mask) | (old << bit_shift);
+		load32 = __cmpxchg_u32(ptr, old32, new32);
+		if (load32 == old32)
+			return old;
+		load = (load32 & mask) >> bit_shift;
+	} while (load == old);
+
+	return load;
+}
+
 /* This function doesn't exist, so you'll get a linker error
    if something tries to do an invalid cmpxchg().  */
 void __cmpxchg_called_with_bad_pointer(void);
@@ -95,6 +122,8 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr,
 __cmpxchg(volatile void *ptr, unsigned long old, unsigned long new, int size)
 {
 	switch (size) {
+		case 1:
+			return __cmpxchg_u8(ptr, old, new);
 		case 4:
 			return __cmpxchg_u32(ptr, old, new);
 		case 8:
-- 
1.7.1

[PATCH 4/6] arch/sparc: Enable queued rwlocks for SPARC

[Babu Moger]

Enable queued rwlocks for SPARC. Here are the discussions on this feature
when this was introduced.
https://lwn.net/Articles/572765/
https://lwn.net/Articles/582200/

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/Kconfig                      |    1 +
 arch/sparc/include/asm/qrwlock.h        |    7 +++++++
 arch/sparc/include/asm/spinlock_64.h    |    9 ++++++---
 arch/sparc/include/asm/spinlock_types.h |    5 ++++-
 4 files changed, 18 insertions(+), 4 deletions(-)
 create mode 100644 arch/sparc/include/asm/qrwlock.h

diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index 8787fc4..caeda1c 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -82,6 +82,7 @@ config SPARC64
 	select HAVE_ARCH_AUDITSYSCALL
 	select ARCH_SUPPORTS_ATOMIC_RMW
 	select HAVE_NMI
+	select ARCH_USE_QUEUED_RWLOCKS
 
 config ARCH_DEFCONFIG
 	string
diff --git a/arch/sparc/include/asm/qrwlock.h b/arch/sparc/include/asm/qrwlock.h
new file mode 100644
index 0000000..d68a4b1
--- /dev/null
+++ b/arch/sparc/include/asm/qrwlock.h
@@ -0,0 +1,7 @@
+#ifndef _ASM_SPARC_QRWLOCK_H
+#define _ASM_SPARC_QRWLOCK_H
+
+#include <asm-generic/qrwlock_types.h>
+#include <asm-generic/qrwlock.h>
+
+#endif /* _ASM_SPARC_QRWLOCK_H */
diff --git a/arch/sparc/include/asm/spinlock_64.h b/arch/sparc/include/asm/spinlock_64.h
index 07c9f2e..562dbd8 100644
--- a/arch/sparc/include/asm/spinlock_64.h
+++ b/arch/sparc/include/asm/spinlock_64.h
@@ -94,6 +94,7 @@ static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long fla
 	: "memory");
 }
 
+#ifndef CONFIG_QUEUED_RWLOCKS
 /* Multi-reader locks, these are much saner than the 32-bit Sparc ones... */
 
 static inline void arch_read_lock(arch_rwlock_t *lock)
@@ -214,12 +215,14 @@ static inline int arch_write_trylock(arch_rwlock_t *lock)
 	return result;
 }
 
-#define arch_read_lock_flags(p, f) arch_read_lock(p)
-#define arch_write_lock_flags(p, f) arch_write_lock(p)
-
 #define arch_read_can_lock(rw)		(!((rw)->lock & 0x80000000UL))
 #define arch_write_can_lock(rw)	(!(rw)->lock)
 
+#endif /* #ifndef CONFIG_QUEUED_RWLOCKS */
+#include <asm/qrwlock.h>
+#define arch_read_lock_flags(p, f) arch_read_lock(p)
+#define arch_write_lock_flags(p, f) arch_write_lock(p)
+
 #define arch_spin_relax(lock)	cpu_relax()
 #define arch_read_relax(lock)	cpu_relax()
 #define arch_write_relax(lock)	cpu_relax()
diff --git a/arch/sparc/include/asm/spinlock_types.h b/arch/sparc/include/asm/spinlock_types.h
index 9c454fd..e052d28 100644
--- a/arch/sparc/include/asm/spinlock_types.h
+++ b/arch/sparc/include/asm/spinlock_types.h
@@ -11,10 +11,13 @@
 
 #define __ARCH_SPIN_LOCK_UNLOCKED	{ 0 }
 
+#ifdef CONFIG_QUEUED_RWLOCKS
+#include <asm-generic/qrwlock_types.h>
+#else
 typedef struct {
 	volatile unsigned int lock;
 } arch_rwlock_t;
 
 #define __ARCH_RW_LOCK_UNLOCKED		{ 0 }
-
+#endif /* CONFIG_QUEUED_RWLOCKS */
 #endif
-- 
1.7.1

[PATCH 4/6] arch/sparc: Enable queued rwlocks for SPARC

[Babu Moger]

Enable queued rwlocks for SPARC. Here are the discussions on this feature
when this was introduced.
https://lwn.net/Articles/572765/
https://lwn.net/Articles/582200/

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/Kconfig                      |    1 +
 arch/sparc/include/asm/qrwlock.h        |    7 +++++++
 arch/sparc/include/asm/spinlock_64.h    |    9 ++++++---
 arch/sparc/include/asm/spinlock_types.h |    5 ++++-
 4 files changed, 18 insertions(+), 4 deletions(-)
 create mode 100644 arch/sparc/include/asm/qrwlock.h

diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index 8787fc4..caeda1c 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -82,6 +82,7 @@ config SPARC64
 	select HAVE_ARCH_AUDITSYSCALL
 	select ARCH_SUPPORTS_ATOMIC_RMW
 	select HAVE_NMI
+	select ARCH_USE_QUEUED_RWLOCKS
 
 config ARCH_DEFCONFIG
 	string
diff --git a/arch/sparc/include/asm/qrwlock.h b/arch/sparc/include/asm/qrwlock.h
new file mode 100644
index 0000000..d68a4b1
--- /dev/null
+++ b/arch/sparc/include/asm/qrwlock.h
@@ -0,0 +1,7 @@
+#ifndef _ASM_SPARC_QRWLOCK_H
+#define _ASM_SPARC_QRWLOCK_H
+
+#include <asm-generic/qrwlock_types.h>
+#include <asm-generic/qrwlock.h>
+
+#endif /* _ASM_SPARC_QRWLOCK_H */
diff --git a/arch/sparc/include/asm/spinlock_64.h b/arch/sparc/include/asm/spinlock_64.h
index 07c9f2e..562dbd8 100644
--- a/arch/sparc/include/asm/spinlock_64.h
+++ b/arch/sparc/include/asm/spinlock_64.h
@@ -94,6 +94,7 @@ static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long fla
 	: "memory");
 }
 
+#ifndef CONFIG_QUEUED_RWLOCKS
 /* Multi-reader locks, these are much saner than the 32-bit Sparc ones... */
 
 static inline void arch_read_lock(arch_rwlock_t *lock)
@@ -214,12 +215,14 @@ static inline int arch_write_trylock(arch_rwlock_t *lock)
 	return result;
 }
 
-#define arch_read_lock_flags(p, f) arch_read_lock(p)
-#define arch_write_lock_flags(p, f) arch_write_lock(p)
-
 #define arch_read_can_lock(rw)		(!((rw)->lock & 0x80000000UL))
 #define arch_write_can_lock(rw)	(!(rw)->lock)
 
+#endif /* #ifndef CONFIG_QUEUED_RWLOCKS */
+#include <asm/qrwlock.h>
+#define arch_read_lock_flags(p, f) arch_read_lock(p)
+#define arch_write_lock_flags(p, f) arch_write_lock(p)
+
 #define arch_spin_relax(lock)	cpu_relax()
 #define arch_read_relax(lock)	cpu_relax()
 #define arch_write_relax(lock)	cpu_relax()
diff --git a/arch/sparc/include/asm/spinlock_types.h b/arch/sparc/include/asm/spinlock_types.h
index 9c454fd..e052d28 100644
--- a/arch/sparc/include/asm/spinlock_types.h
+++ b/arch/sparc/include/asm/spinlock_types.h
@@ -11,10 +11,13 @@
 
 #define __ARCH_SPIN_LOCK_UNLOCKED	{ 0 }
 
+#ifdef CONFIG_QUEUED_RWLOCKS
+#include <asm-generic/qrwlock_types.h>
+#else
 typedef struct {
 	volatile unsigned int lock;
 } arch_rwlock_t;
 
 #define __ARCH_RW_LOCK_UNLOCKED		{ 0 }
-
+#endif /* CONFIG_QUEUED_RWLOCKS */
 #endif
-- 
1.7.1

Re: [PATCH 4/6] arch/sparc: Enable queued rwlocks for SPARC

[David Miller]

From: Babu Moger <babu.moger@oracle.com>
Date: Thu, 18 May 2017 18:36:08 -0600

> @@ -82,6 +82,7 @@ config SPARC64
>  	select HAVE_ARCH_AUDITSYSCALL
>  	select ARCH_SUPPORTS_ATOMIC_RMW
>  	select HAVE_NMI
> +	select ARCH_USE_QUEUED_RWLOCKS
>  

If you are selecting this on SPARC64 all the time, then:

> @@ -94,6 +94,7 @@ static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long fla
>  	: "memory");
>  }
>  
> +#ifndef CONFIG_QUEUED_RWLOCKS
>  /* Multi-reader locks, these are much saner than the 32-bit Sparc ones... */

You can remove this segment of ifdef'd code altogether since it is in
a sparc64 specific header file.

Re: [PATCH 4/6] arch/sparc: Enable queued rwlocks for SPARC

[David Miller]

From: Babu Moger <babu.moger@oracle.com>
Date: Thu, 18 May 2017 18:36:08 -0600

> @@ -82,6 +82,7 @@ config SPARC64
>  	select HAVE_ARCH_AUDITSYSCALL
>  	select ARCH_SUPPORTS_ATOMIC_RMW
>  	select HAVE_NMI
> +	select ARCH_USE_QUEUED_RWLOCKS
>  

If you are selecting this on SPARC64 all the time, then:

> @@ -94,6 +94,7 @@ static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long fla
>  	: "memory");
>  }
>  
> +#ifndef CONFIG_QUEUED_RWLOCKS
>  /* Multi-reader locks, these are much saner than the 32-bit Sparc ones... */

You can remove this segment of ifdef'd code altogether since it is in
a sparc64 specific header file.

Re: [PATCH 4/6] arch/sparc: Enable queued rwlocks for SPARC

[Peter Zijlstra]

On Thu, May 18, 2017 at 10:31:13PM -0400, David Miller wrote:
> From: Babu Moger <babu.moger@oracle.com>
> Date: Thu, 18 May 2017 18:36:08 -0600
> 
> > @@ -82,6 +82,7 @@ config SPARC64
> >  	select HAVE_ARCH_AUDITSYSCALL
> >  	select ARCH_SUPPORTS_ATOMIC_RMW
> >  	select HAVE_NMI
> > +	select ARCH_USE_QUEUED_RWLOCKS
> >  
> 
> If you are selecting this on SPARC64 all the time, then:
> 
> > @@ -94,6 +94,7 @@ static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long fla
> >  	: "memory");
> >  }
> >  
> > +#ifndef CONFIG_QUEUED_RWLOCKS
> >  /* Multi-reader locks, these are much saner than the 32-bit Sparc ones... */
> 
> You can remove this segment of ifdef'd code altogether since it is in
> a sparc64 specific header file.


So IIRC Sparc v8 only has that single byte load-and-set (or swap)
instruction, right? That means you can only make test-and-set spinlocks
and then have to build the world on top of that.

I don't see qrwlock -- which assumes the spinlock implementation is fair
-- making much sense for that.

Also, IIRC Sparc-v8 didn't really have very big SMP systems, those came
with v9. And qspinlock only really makes sense on the bigger systems
(not to mention that building the qspinlock on top of atomic operations
build on test-and-set spinlocks just seems extremely dysfunctional).


In any case, I think what I'm saying is that it makes sense to make this
a Sparcv9 only feature.

Re: [PATCH 4/6] arch/sparc: Enable queued rwlocks for SPARC

[Babu Moger]

On 5/19/2017 4:03 AM, Peter Zijlstra wrote:
> On Thu, May 18, 2017 at 10:31:13PM -0400, David Miller wrote:
>> From: Babu Moger <babu.moger@oracle.com>
>> Date: Thu, 18 May 2017 18:36:08 -0600
>>
>>> @@ -82,6 +82,7 @@ config SPARC64
>>>   	select HAVE_ARCH_AUDITSYSCALL
>>>   	select ARCH_SUPPORTS_ATOMIC_RMW
>>>   	select HAVE_NMI
>>> +	select ARCH_USE_QUEUED_RWLOCKS
>>>   
>> If you are selecting this on SPARC64 all the time, then:
>>
>>> @@ -94,6 +94,7 @@ static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long fla
>>>   	: "memory");
>>>   }
>>>   
>>> +#ifndef CONFIG_QUEUED_RWLOCKS
>>>   /* Multi-reader locks, these are much saner than the 32-bit Sparc ones... */
>> You can remove this segment of ifdef'd code altogether since it is in
>> a sparc64 specific header file.
>
> So IIRC Sparc v8 only has that single byte load-and-set (or swap)
> instruction, right? That means you can only make test-and-set spinlocks
> and then have to build the world on top of that.
>
> I don't see qrwlock -- which assumes the spinlock implementation is fair
> -- making much sense for that.
>
> Also, IIRC Sparc-v8 didn't really have very big SMP systems, those came
> with v9. And qspinlock only really makes sense on the bigger systems
> (not to mention that building the qspinlock on top of atomic operations
> build on test-and-set spinlocks just seems extremely dysfunctional).
>
>
> In any case, I think what I'm saying is that it makes sense to make this
> a Sparcv9 only feature.
>

Agree.  Lets keep this as Sparcv9 only feature.

Re: [PATCH 4/6] arch/sparc: Enable queued rwlocks for SPARC

[David Miller]

From: Peter Zijlstra <peterz@infradead.org>
Date: Fri, 19 May 2017 11:03:02 +0200

> On Thu, May 18, 2017 at 10:31:13PM -0400, David Miller wrote:
>> From: Babu Moger <babu.moger@oracle.com>
>> Date: Thu, 18 May 2017 18:36:08 -0600
>> 
>> > @@ -82,6 +82,7 @@ config SPARC64
>> >  	select HAVE_ARCH_AUDITSYSCALL
>> >  	select ARCH_SUPPORTS_ATOMIC_RMW
>> >  	select HAVE_NMI
>> > +	select ARCH_USE_QUEUED_RWLOCKS
>> >  
>> 
>> If you are selecting this on SPARC64 all the time, then:
>> 
>> > @@ -94,6 +94,7 @@ static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long fla
>> >  	: "memory");
>> >  }
>> >  
>> > +#ifndef CONFIG_QUEUED_RWLOCKS
>> >  /* Multi-reader locks, these are much saner than the 32-bit Sparc ones... */
>> 
>> You can remove this segment of ifdef'd code altogether since it is in
>> a sparc64 specific header file.
> 
> 
> So IIRC Sparc v8 only has that single byte load-and-set (or swap)
> instruction, right? That means you can only make test-and-set spinlocks
> and then have to build the world on top of that.
> 
> I don't see qrwlock -- which assumes the spinlock implementation is fair
> -- making much sense for that.
> 
> Also, IIRC Sparc-v8 didn't really have very big SMP systems, those came
> with v9. And qspinlock only really makes sense on the bigger systems
> (not to mention that building the qspinlock on top of atomic operations
> build on test-and-set spinlocks just seems extremely dysfunctional).
> 
> 
> In any case, I think what I'm saying is that it makes sense to make this
> a Sparcv9 only feature.

I agree with you, there is no reason to try and support
queued locks on 32-bit sparc.

However, I don't see what any of this has to do with the feedback
I was giving the patch author :-)

Re: [PATCH 4/6] arch/sparc: Enable queued rwlocks for SPARC

[David Miller]

From: Peter Zijlstra <peterz@infradead.org>
Date: Fri, 19 May 2017 11:03:02 +0200

> On Thu, May 18, 2017 at 10:31:13PM -0400, David Miller wrote:
>> From: Babu Moger <babu.moger@oracle.com>
>> Date: Thu, 18 May 2017 18:36:08 -0600
>> 
>> > @@ -82,6 +82,7 @@ config SPARC64
>> >  	select HAVE_ARCH_AUDITSYSCALL
>> >  	select ARCH_SUPPORTS_ATOMIC_RMW
>> >  	select HAVE_NMI
>> > +	select ARCH_USE_QUEUED_RWLOCKS
>> >  
>> 
>> If you are selecting this on SPARC64 all the time, then:
>> 
>> > @@ -94,6 +94,7 @@ static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long fla
>> >  	: "memory");
>> >  }
>> >  
>> > +#ifndef CONFIG_QUEUED_RWLOCKS
>> >  /* Multi-reader locks, these are much saner than the 32-bit Sparc ones... */
>> 
>> You can remove this segment of ifdef'd code altogether since it is in
>> a sparc64 specific header file.
> 
> 
> So IIRC Sparc v8 only has that single byte load-and-set (or swap)
> instruction, right? That means you can only make test-and-set spinlocks
> and then have to build the world on top of that.
> 
> I don't see qrwlock -- which assumes the spinlock implementation is fair
> -- making much sense for that.
> 
> Also, IIRC Sparc-v8 didn't really have very big SMP systems, those came
> with v9. And qspinlock only really makes sense on the bigger systems
> (not to mention that building the qspinlock on top of atomic operations
> build on test-and-set spinlocks just seems extremely dysfunctional).
> 
> 
> In any case, I think what I'm saying is that it makes sense to make this
> a Sparcv9 only feature.

I agree with you, there is no reason to try and support
queued locks on 32-bit sparc.

However, I don't see what any of this has to do with the feedback
I was giving the patch author :-)

Re: [PATCH 4/6] arch/sparc: Enable queued rwlocks for SPARC

[Peter Zijlstra]

On Fri, May 19, 2017 at 03:15:53PM -0400, David Miller wrote:
> However, I don't see what any of this has to do with the feedback
> I was giving the patch author :-)

Uhm,... I think my morning brain read things like you having doubts
about making it sparc64 only. But I could have easily misread things.
Ignore my ramblings :-)

Re: [PATCH 4/6] arch/sparc: Enable queued rwlocks for SPARC

[Peter Zijlstra]

On Fri, May 19, 2017 at 03:15:53PM -0400, David Miller wrote:
> However, I don't see what any of this has to do with the feedback
> I was giving the patch author :-)

Uhm,... I think my morning brain read things like you having doubts
about making it sparc64 only. But I could have easily misread things.
Ignore my ramblings :-)

Re: [PATCH 4/6] arch/sparc: Enable queued rwlocks for SPARC

[David Miller]

From: Peter Zijlstra <peterz@infradead.org>
Date: Fri, 19 May 2017 21:31:26 +0200

> On Fri, May 19, 2017 at 03:15:53PM -0400, David Miller wrote:
>> However, I don't see what any of this has to do with the feedback
>> I was giving the patch author :-)
> 
> Uhm,... I think my morning brain read things like you having doubts
> about making it sparc64 only. But I could have easily misread things.
> Ignore my ramblings :-)

He was editing a sparc64-specific header, adding "queued locks" ifdefs,
which makes no sense if for SPARC64 the queued locks will always be
enabled.

Re: [PATCH 4/6] arch/sparc: Enable queued rwlocks for SPARC

[David Miller]

From: Peter Zijlstra <peterz@infradead.org>
Date: Fri, 19 May 2017 21:31:26 +0200

> On Fri, May 19, 2017 at 03:15:53PM -0400, David Miller wrote:
>> However, I don't see what any of this has to do with the feedback
>> I was giving the patch author :-)
> 
> Uhm,... I think my morning brain read things like you having doubts
> about making it sparc64 only. But I could have easily misread things.
> Ignore my ramblings :-)

He was editing a sparc64-specific header, adding "queued locks" ifdefs,
which makes no sense if for SPARC64 the queued locks will always be
enabled.

Re: [PATCH 4/6] arch/sparc: Enable queued rwlocks for SPARC

[Babu Moger]

On 5/18/2017 9:31 PM, David Miller wrote:
> From: Babu Moger <babu.moger@oracle.com>
> Date: Thu, 18 May 2017 18:36:08 -0600
>
>> @@ -82,6 +82,7 @@ config SPARC64
>>   	select HAVE_ARCH_AUDITSYSCALL
>>   	select ARCH_SUPPORTS_ATOMIC_RMW
>>   	select HAVE_NMI
>> +	select ARCH_USE_QUEUED_RWLOCKS
>>   
> If you are selecting this on SPARC64 all the time, then:
>
>> @@ -94,6 +94,7 @@ static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long fla
>>   	: "memory");
>>   }
>>   
>> +#ifndef CONFIG_QUEUED_RWLOCKS
>>   /* Multi-reader locks, these are much saner than the 32-bit Sparc ones... */
> You can remove this segment of ifdef'd code altogether since it is in
> a sparc64 specific header file.

Sure. will do.

[PATCH 5/6] arch/sparc: Introduce xchg16 for SPARC

[Babu Moger]

SPARC supports 32 bit and 64 bit xchg right now. Add the support
for 16 bit (2 byte) xchg. This is required to support queued spinlock
feature which uses 2 byte xchg. This is achieved using 4 byte cas
instructions with byte manipulations.

Also re-arranged the code to call __cmpxchg_u32 inside xchg16.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Steven Sistare <steven.sistare@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/include/asm/cmpxchg_64.h |   49 +++++++++++++++++++++++++++-------
 1 files changed, 39 insertions(+), 10 deletions(-)

diff --git a/arch/sparc/include/asm/cmpxchg_64.h b/arch/sparc/include/asm/cmpxchg_64.h
index 65bfae0..a6c9fe3 100644
--- a/arch/sparc/include/asm/cmpxchg_64.h
+++ b/arch/sparc/include/asm/cmpxchg_64.h
@@ -6,6 +6,17 @@
 #ifndef __ARCH_SPARC64_CMPXCHG__
 #define __ARCH_SPARC64_CMPXCHG__
 
+static inline unsigned long
+__cmpxchg_u32(volatile int *m, int old, int new)
+{
+	__asm__ __volatile__("cas [%2], %3, %0"
+			     : "=&r" (new)
+			     : "0" (new), "r" (m), "r" (old)
+			     : "memory");
+
+	return new;
+}
+
 static inline unsigned long xchg32(__volatile__ unsigned int *m, unsigned int val)
 {
 	unsigned long tmp1, tmp2;
@@ -44,10 +55,38 @@ static inline unsigned long xchg64(__volatile__ unsigned long *m, unsigned long 
 
 void __xchg_called_with_bad_pointer(void);
 
+/*
+ * Use 4 byte cas instruction to achieve 2 byte xchg. Main logic
+ * here is to get the bit shift of the byte we are interested in.
+ * The XOR is handy for reversing the bits for big-endian byte order.
+ */
+static inline unsigned long
+xchg16(__volatile__ unsigned short *m, unsigned int val)
+{
+	unsigned long maddr = (unsigned long)m;
+	int bit_shift = (((unsigned long)m & 2) ^ 2) << 3;
+	unsigned int mask = 0xffff << bit_shift;
+	unsigned int *ptr = (unsigned int  *) (maddr & ~2);
+	unsigned int old32, new32, load32;
+
+	/* Read the old value */
+	load32 = *ptr;
+
+	do {
+		old32 = load32;
+		new32 = (load32 & (~mask)) | val << bit_shift;
+		load32 = __cmpxchg_u32(ptr, old32, new32);
+	} while (load32 != old32);
+
+	return (load32 & mask) >> bit_shift;
+}
+
 static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr,
 				       int size)
 {
 	switch (size) {
+	case 2:
+		return xchg16(ptr, x);
 	case 4:
 		return xchg32(ptr, x);
 	case 8:
@@ -65,16 +104,6 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr,
 
 #include <asm-generic/cmpxchg-local.h>
 
-static inline unsigned long
-__cmpxchg_u32(volatile int *m, int old, int new)
-{
-	__asm__ __volatile__("cas [%2], %3, %0"
-			     : "=&r" (new)
-			     : "0" (new), "r" (m), "r" (old)
-			     : "memory");
-
-	return new;
-}
 
 static inline unsigned long
 __cmpxchg_u64(volatile long *m, unsigned long old, unsigned long new)
-- 
1.7.1

[PATCH 6/6] arch/sparc: Enable queued spinlock support for SPARC

[Babu Moger]

This patch makes the necessary changes in SPARC architecture to enable
queued spinlock support. Here are some of the earlier discussions about
this feature.
https://lwn.net/Articles/561775/
https://lwn.net/Articles/590243/

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/Kconfig                      |    1 +
 arch/sparc/include/asm/qspinlock.h      |    7 +++++++
 arch/sparc/include/asm/spinlock_64.h    |    4 ++++
 arch/sparc/include/asm/spinlock_types.h |    5 +++++
 4 files changed, 17 insertions(+), 0 deletions(-)
 create mode 100644 arch/sparc/include/asm/qspinlock.h

diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index caeda1c..785bf3e 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -83,6 +83,7 @@ config SPARC64
 	select ARCH_SUPPORTS_ATOMIC_RMW
 	select HAVE_NMI
 	select ARCH_USE_QUEUED_RWLOCKS
+	select ARCH_USE_QUEUED_SPINLOCKS
 
 config ARCH_DEFCONFIG
 	string
diff --git a/arch/sparc/include/asm/qspinlock.h b/arch/sparc/include/asm/qspinlock.h
new file mode 100644
index 0000000..5ae9a28
--- /dev/null
+++ b/arch/sparc/include/asm/qspinlock.h
@@ -0,0 +1,7 @@
+#ifndef _ASM_SPARC_QSPINLOCK_H
+#define _ASM_SPARC_QSPINLOCK_H
+
+#include <asm-generic/qspinlock_types.h>
+#include <asm-generic/qspinlock.h>
+
+#endif /* _ASM_SPARC_QSPINLOCK_H */
diff --git a/arch/sparc/include/asm/spinlock_64.h b/arch/sparc/include/asm/spinlock_64.h
index 562dbd8..e2044e3 100644
--- a/arch/sparc/include/asm/spinlock_64.h
+++ b/arch/sparc/include/asm/spinlock_64.h
@@ -11,6 +11,9 @@
 #include <asm/processor.h>
 #include <asm/barrier.h>
 
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#include <asm/qspinlock.h>
+#else
 /* To get debugging spinlocks which detect and catch
  * deadlock situations, set CONFIG_DEBUG_SPINLOCK
  * and rebuild your kernel.
@@ -93,6 +96,7 @@ static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long fla
 	: "r"(lock), "r"(flags)
 	: "memory");
 }
+#endif /*ifdef CONFIG_QUEUED_SPINLOCKS */
 
 #ifndef CONFIG_QUEUED_RWLOCKS
 /* Multi-reader locks, these are much saner than the 32-bit Sparc ones... */
diff --git a/arch/sparc/include/asm/spinlock_types.h b/arch/sparc/include/asm/spinlock_types.h
index e052d28..93c8c5a 100644
--- a/arch/sparc/include/asm/spinlock_types.h
+++ b/arch/sparc/include/asm/spinlock_types.h
@@ -5,11 +5,16 @@
 # error "please don't include this file directly"
 #endif
 
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#include <asm-generic/qspinlock_types.h>
+#else
+
 typedef struct {
 	volatile unsigned char lock;
 } arch_spinlock_t;
 
 #define __ARCH_SPIN_LOCK_UNLOCKED	{ 0 }
+#endif /* CONFIG_QUEUED_SPINLOCKS */
 
 #ifdef CONFIG_QUEUED_RWLOCKS
 #include <asm-generic/qrwlock_types.h>
-- 
1.7.1

Re: [PATCH 6/6] arch/sparc: Enable queued spinlock support for SPARC

[David Miller]

From: Babu Moger <babu.moger@oracle.com>
Date: Thu, 18 May 2017 18:36:10 -0600

> @@ -83,6 +83,7 @@ config SPARC64
>  	select ARCH_SUPPORTS_ATOMIC_RMW
>  	select HAVE_NMI
>  	select ARCH_USE_QUEUED_RWLOCKS
> +	select ARCH_USE_QUEUED_SPINLOCKS
>  
>  config ARCH_DEFCONFIG
>  	string

Like the queued spinlock enabling patch, if this will be on
all the time for SPARC64 then:

> diff --git a/arch/sparc/include/asm/spinlock_64.h b/arch/sparc/include/asm/spinlock_64.h
> index 562dbd8..e2044e3 100644
> --- a/arch/sparc/include/asm/spinlock_64.h
> +++ b/arch/sparc/include/asm/spinlock_64.h
> @@ -11,6 +11,9 @@
>  #include <asm/processor.h>
>  #include <asm/barrier.h>
>  
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> +#include <asm/qspinlock.h>
> +#else

This ifdef doesn't make any sense, by definition it will always be on.

Re: [PATCH 6/6] arch/sparc: Enable queued spinlock support for SPARC

[Babu Moger]

On 5/18/2017 9:32 PM, David Miller wrote:
> From: Babu Moger <babu.moger@oracle.com>
> Date: Thu, 18 May 2017 18:36:10 -0600
>
>> @@ -83,6 +83,7 @@ config SPARC64
>>   	select ARCH_SUPPORTS_ATOMIC_RMW
>>   	select HAVE_NMI
>>   	select ARCH_USE_QUEUED_RWLOCKS
>> +	select ARCH_USE_QUEUED_SPINLOCKS
>>   
>>   config ARCH_DEFCONFIG
>>   	string
> Like the queued spinlock enabling patch, if this will be on
> all the time for SPARC64 then:
>
>> diff --git a/arch/sparc/include/asm/spinlock_64.h b/arch/sparc/include/asm/spinlock_64.h
>> index 562dbd8..e2044e3 100644
>> --- a/arch/sparc/include/asm/spinlock_64.h
>> +++ b/arch/sparc/include/asm/spinlock_64.h
>> @@ -11,6 +11,9 @@
>>   #include <asm/processor.h>
>>   #include <asm/barrier.h>
>>   
>> +#ifdef CONFIG_QUEUED_SPINLOCKS
>> +#include <asm/qspinlock.h>
>> +#else
> This ifdef doesn't make any sense, by definition it will always be on.

Ok. Will do. thanks

Re: [PATCH 6/6] arch/sparc: Enable queued spinlock support for SPARC

[Babu Moger]

On 5/18/2017 9:32 PM, David Miller wrote:
> From: Babu Moger <babu.moger@oracle.com>
> Date: Thu, 18 May 2017 18:36:10 -0600
>
>> @@ -83,6 +83,7 @@ config SPARC64
>>   	select ARCH_SUPPORTS_ATOMIC_RMW
>>   	select HAVE_NMI
>>   	select ARCH_USE_QUEUED_RWLOCKS
>> +	select ARCH_USE_QUEUED_SPINLOCKS
>>   
>>   config ARCH_DEFCONFIG
>>   	string
> Like the queued spinlock enabling patch, if this will be on
> all the time for SPARC64 then:
>
>> diff --git a/arch/sparc/include/asm/spinlock_64.h b/arch/sparc/include/asm/spinlock_64.h
>> index 562dbd8..e2044e3 100644
>> --- a/arch/sparc/include/asm/spinlock_64.h
>> +++ b/arch/sparc/include/asm/spinlock_64.h
>> @@ -11,6 +11,9 @@
>>   #include <asm/processor.h>
>>   #include <asm/barrier.h>
>>   
>> +#ifdef CONFIG_QUEUED_SPINLOCKS
>> +#include <asm/qspinlock.h>
>> +#else
> This ifdef doesn't make any sense, by definition it will always be on.

Ok. Will do. thanks