[PATCH 07/29] x86, tsx: Don't abort immediately in __read/write_lock_failed

Andi Kleen
2013-03-23T01:32:16Z

__read/write_lock_failed did execute a PAUSE first thing before
checking the lock. This aborts transactions. Check the lock
state again before executing the pause. This avoids a small
number of extra aborts, and is slightly cheaper too.

Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
arch/x86/lib/rwlock.S | 8 ++++++--
1 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/x86/lib/rwlock.S b/arch/x86/lib/rwlock.S
index 1cad221..9a6bc12 100644
--- a/arch/x86/lib/rwlock.S
+++ b/arch/x86/lib/rwlock.S
@@ -16,10 +16,12 @@ ENTRY(__write_lock_failed)
FRAME
0: LOCK_PREFIX
WRITE_LOCK_ADD($RW_LOCK_BIAS) (%__lock_ptr)
+ cmpl $WRITE_LOCK_CMP, (%__lock_ptr)
+ je 2f
1: rep; nop
cmpl $WRITE_LOCK_CMP, (%__lock_ptr)
jne 1b
- LOCK_PREFIX
+2: LOCK_PREFIX
WRITE_LOCK_SUB($RW_LOCK_BIAS) (%__lock_ptr)
jnz 0b
ENDFRAME
@@ -32,10 +34,12 @@ ENTRY(__read_lock_failed)
FRAME
0: LOCK_PREFIX
READ_LOCK_SIZE(inc) (%__lock_ptr)
+ READ_LOCK_SIZE(cmp) $1, (%__lock_ptr)
+ jns 2f
1: rep; nop
READ_LOCK_SIZE(cmp) $1, (%__lock_ptr)
js 1b
- LOCK_PREFIX
+2: LOCK_PREFIX
READ_LOCK_SIZE(dec) (%__lock_ptr)
js 0b
ENDFRAME
--
1.7.7.6