[PATCH 1/2 percpu/for-3.20] percpu_ref: remove unnecessary ACCESS_ONCE() in percpu_ref_tryget_live()

[PATCH 1/2 percpu/for-3.20] percpu_ref: remove unnecessary ACCESS_ONCE() in percpu_ref_tryget_live()

[Tejun Heo]

__ref_is_percpu() needs the implied ACCESS_ONCE() in
lockless_dereference() on @ref->percpu_count_ptr because the value is
tested for !__PERCPU_REF_ATOMIC, which may be set asynchronously, and
then used as a pointer.  If the compiler generates a separate fetch
when using it as a pointer, __PERCPU_REF_ATOMIC may be set in between
contaminating the pointer value.

percpu_ref_tryget_live() also uses ACCESS_ONCE() to test
__PERCPU_REF_DEAD; however, there's no reason for this.  I just copied
ACCESS_ONCE() usage blindly from __ref_is_percpu().  All it does is
confusing people trying to understand what's going on.

This patch removes the unnecessary ACCESS_ONCE() usage from
percpu_ref_tryget_live() and adds a comment explaining why
__ref_is_percpu() needs it.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Kent Overstreet <kmo@daterainc.com>
---
 include/linux/percpu-refcount.h |   20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

--- a/include/linux/percpu-refcount.h
+++ b/include/linux/percpu-refcount.h
@@ -128,8 +128,22 @@ static inline void percpu_ref_kill(struc
 static inline bool __ref_is_percpu(struct percpu_ref *ref,
 					  unsigned long __percpu **percpu_countp)
 {
-	/* paired with smp_store_release() in percpu_ref_reinit() */
-	unsigned long percpu_ptr = lockless_dereference(ref->percpu_count_ptr);
+	unsigned long percpu_ptr;
+
+	/*
+	 * The value of @ref->percpu_count_ptr is tested for
+	 * !__PERCPU_REF_ATOMIC, which may be set asynchronously, and then
+	 * used as a pointer.  If the compiler generates a separate fetch
+	 * when using it as a pointer, __PERCPU_REF_ATOMIC may be set in
+	 * between contaminating the pointer value, meaning that
+	 * ACCESS_ONCE() is required when fetching it.
+	 *
+	 * Also, we need a data dependency barrier to be paired with
+	 * smp_store_release() in __percpu_ref_switch_to_percpu().
+	 *
+	 * Use lockless deref which contains both.
+	 */
+	percpu_ptr = lockless_dereference(ref->percpu_count_ptr);
 
 	/*
 	 * Theoretically, the following could test just ATOMIC; however,
@@ -233,7 +247,7 @@ static inline bool percpu_ref_tryget_liv
 	if (__ref_is_percpu(ref, &percpu_count)) {
 		this_cpu_inc(*percpu_count);
 		ret = true;
-	} else if (!(ACCESS_ONCE(ref->percpu_count_ptr) & __PERCPU_REF_DEAD)) {
+	} else if (!(ref->percpu_count_ptr & __PERCPU_REF_DEAD)) {
 		ret = atomic_long_inc_not_zero(&ref->count);
 	}
 

[PATCH 2/2 percpu/for-3.20] percpu_ref: implement percpu_ref_is_dying()

[Tejun Heo]

Implement percpu_ref_is_dying() which tests whether the ref is dying
or dead.  This is useful to determine the current state when a
percpu_ref is used as a cyclic on/off switch via kill and reinit.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Kent Overstreet <kmo@daterainc.com>
---
Hello,

This will be used by the scheduled cgroup writeback support.  I'm
applying these two to percpu/for-3.20.

Thanks.

 include/linux/percpu-refcount.h |   14 ++++++++++++++
 1 file changed, 14 insertions(+)

--- a/include/linux/percpu-refcount.h
+++ b/include/linux/percpu-refcount.h
@@ -295,6 +295,20 @@ static inline void percpu_ref_put(struct
 }
 
 /**
+ * percpu_ref_is_dying - test whether a percpu refcount is dying or dead
+ * @ref: percpu_ref to test
+ *
+ * Returns %true if @ref is dying or dead.
+ *
+ * This function is safe to call as long as @ref is between init and exit
+ * and the caller is responsible for synchronizing against state changes.
+ */
+static inline bool percpu_ref_is_dying(struct percpu_ref *ref)
+{
+	return ref->percpu_count_ptr & __PERCPU_REF_DEAD;
+}
+
+/**
  * percpu_ref_is_zero - test whether a percpu refcount reached zero
  * @ref: percpu_ref to test
  *

Re: [PATCH 1/2 percpu/for-3.20] percpu_ref: remove unnecessary ACCESS_ONCE() in percpu_ref_tryget_live()

[Kent Overstreet]

On Tue, Jan 06, 2015 at 10:23:47AM -0500, Tejun Heo wrote:
> __ref_is_percpu() needs the implied ACCESS_ONCE() in
> lockless_dereference() on @ref->percpu_count_ptr because the value is
> tested for !__PERCPU_REF_ATOMIC, which may be set asynchronously, and
> then used as a pointer.  If the compiler generates a separate fetch
> when using it as a pointer, __PERCPU_REF_ATOMIC may be set in between
> contaminating the pointer value.
> 
> percpu_ref_tryget_live() also uses ACCESS_ONCE() to test
> __PERCPU_REF_DEAD; however, there's no reason for this.  I just copied
> ACCESS_ONCE() usage blindly from __ref_is_percpu().  All it does is
> confusing people trying to understand what's going on.
> 
> This patch removes the unnecessary ACCESS_ONCE() usage from
> percpu_ref_tryget_live() and adds a comment explaining why
> __ref_is_percpu() needs it.

lockless_dereference() is new - cool! I've been wanting that without even
realizing it.

Acked-by: Kent Overstreet <kmo@daterainc.com>

Re: [PATCH 2/2 percpu/for-3.20] percpu_ref: implement percpu_ref_is_dying()

[Kent Overstreet]

On Tue, Jan 06, 2015 at 10:24:59AM -0500, Tejun Heo wrote:
> Implement percpu_ref_is_dying() which tests whether the ref is dying
> or dead.  This is useful to determine the current state when a
> percpu_ref is used as a cyclic on/off switch via kill and reinit.

Acked-by: Kent Overstreet <kmo@daterainc.com>