[PATCH v13 06/15] asm-generic: barrier: Add smp_cond_load_acquire_timeout()

[Ankur Arora <ankur.a.arora@oracle.com>]

Add the acquire variant of smp_cond_load_relaxed_timeout(). This
reuses the relaxed variant, with additional LOAD->LOAD ordering
via smp_acquire__after_ctrl_dep().

To ensure that the necessary control dependency on the dereference
of @ptr exists (which does not in the timeout path), re-evaluate
the cond_expr branch.

Cc: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Will Deacon <will@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: linux-arch@vger.kernel.org
Cc: bpf@vger.kernel.org
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Haris Okanovic <harisokn@amazon.com>
Tested-by: Haris Okanovic <harisokn@amazon.com>
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
---
Notes: addresses missed control dependency in the timeout case [1] by
documenting that the acquire ordering is only in the success
(non-timeout) case.
However, we do need to re-evaluate the cond_expr branch to handle the
case where we hit both the timeout and the loop condition.

AFAICT, rqspinlock only cares about acquire ordering in the success
case so this change should be okay. 
Kumar, Alexei: please let me know if otherwise.

Catalin, Haris: I've retained your R-by on this. Please let me know if
you aren't okay with this change.

[1] https://lore.kernel.org/all/20260608082746.399BB1F00898@smtp.kernel.org/
---
 include/asm-generic/barrier.h | 40 +++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/include/asm-generic/barrier.h b/include/asm-generic/barrier.h
index d53e75a92598..5d21ad9f358f 100644
--- a/include/asm-generic/barrier.h
+++ b/include/asm-generic/barrier.h
@@ -356,6 +356,46 @@ do {									\
 })
 #endif
 
+/**
+ * smp_cond_load_acquire_timeout() - (Spin) wait for cond until a timeout
+ * expires. ACQUIRE ordering when @cond_expr is satisfied.
+ * @ptr: pointer to the variable to wait on.
+ * @cond_expr: boolean expression to wait for.
+ * @time_expr_ns: monotonic expression that evaluates to time in ns or,
+ *  on failure, returns a negative value.
+ * @timeout_ns: timeout value in ns
+ * (Both of the above are assumed to be compatible with s64.)
+ *
+ * Equivalent to using smp_cond_load_acquire() on the condition variable with
+ * a timeout.
+ */
+#ifndef smp_cond_load_acquire_timeout
+#define smp_cond_load_acquire_timeout(ptr, cond_expr,			\
+				      time_expr_ns, timeout_ns)		\
+({									\
+	__unqual_scalar_typeof(*(ptr)) VAL;				\
+	VAL = smp_cond_load_relaxed_timeout(ptr, cond_expr,		\
+					     time_expr_ns,		\
+					     timeout_ns);		\
+	/*								\
+	 * We arrive here once the loop condition is hit, on timeout,	\
+	 * or, if we hit both the timeout and the loop condition.	\
+	 *								\
+	 * The last case is low probability, but possible in the last	\
+	 * iteration, especially on architectures with waiting		\
+	 * cpu_poll_relax() implementations (ex. arm64).		\
+	 * Now since the loop condition is not evaluated on timeout,	\
+	 * we have a missed control dependency.				\
+	 *								\
+	 * So, force a re-evaluation of the control dependency to	\
+	 * provide an ACQUIRE ordering for that case as well.		\
+	 */								\
+	if (cond_expr)							\
+		smp_acquire__after_ctrl_dep();				\
+	(typeof(*(ptr)))VAL;						\
+})
+#endif
+
 /*
  * pmem_wmb() ensures that all stores for which the modification
  * are written to persistent storage by preceding instructions have
-- 
2.43.7