[PATCH 06/12] rwsem: wake queued readers when other readers are active

[Michel Lespinasse <walken@google.com>]

This change addresses the following situation:

- Thread A holds the rwsem for write
- Thread B tries to acquire the rwsem for read; blocks & gets queued
- Thread A releases the rwsem, notices active count goes back to 0
- Thread C acquires the rwsem for read
- Thread A acquires the spinlock & tries to wake thread B, but fails because
  active count is not zero anymore.

In this situation, it would be perfectly fine to let threads B and C work
in parallel as they each only want a read acquire on the rwsem. We can
recognize this situation and let A wake B as long as there are no active
writers on the rwsem.

Signed-off-by: Michel Lespinasse <walken@google.com>
---
 lib/rwsem.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/rwsem.c b/lib/rwsem.c
index 92c8f8e..9d0899b 100644
--- a/lib/rwsem.c
+++ b/lib/rwsem.c
@@ -115,8 +115,8 @@ __rwsem_do_wake(struct rw_semaphore *sem, int downgrading)
 
  retry_readers:
 	oldcount = rwsem_atomic_update(adjustment, sem) - adjustment;
-	if (!downgrading && (oldcount & RWSEM_ACTIVE_MASK))
-		/* Someone grabbed the sem already */
+	if (!downgrading && (oldcount < RWSEM_WAITING_BIAS))
+		/* Someone grabbed the sem for write already */
 		goto undo_readers;
 
 	next = sem->wait_list.next;
@@ -143,7 +143,7 @@ __rwsem_do_wake(struct rw_semaphore *sem, int downgrading)
 	goto retry_writer;
 
  undo_readers:
-	if (rwsem_atomic_update(-adjustment, sem) & RWSEM_ACTIVE_MASK)
+	if (rwsem_atomic_update(-adjustment, sem) < RWSEM_WAITING_BIAS)
 		goto out;
 	goto retry_readers;
 }
-- 
1.7.0.1