[PATCH v3 00/25] context_tracking, rcu: Spring cleaning of dynticks references

[PATCH v3 00/25] context_tracking, rcu: Spring cleaning of dynticks references

[Valentin Schneider]

Git info
========

The series is based on rcu/next at:
f395ae346be5 ("Merge branches 'doc.2024.06.06a', 'fixes.2024.06.06a', 'mb.2024.06.03a', 'nocb.2024.06.03a', 'rcu-tasks.2024.06.06a' and 'rcutorture.2024.06.06a' into HEAD")

It's also available as a git tree at:
https://gitlab.com/vschneid/linux.git -b redhat/isolirq/defer/v3-rcu-v3


Series description
==================

I've done one patch per renaming for my own sanity, a lot of these probably
deserve to be squashed together (e.g. all of the nmi_nesting into one).

At the end of the renaming exercise, the remaining RCU references to "dyntick"
are:
- Comments in core RCU code:
  kernel/rcu/:
  tree.h:208:8:	/* 3) dynticks interface. */
  tree.h:209:46:	int  watching_snap;		/* Per-GP tracking for dynticks. */
  tree_nocb.h:55:19: * about entering dyntick-idle mode.
  tree_plugin.h:873:4: * dyntick-idle quiescent state visible to other CPUs, which will in
  Kconfig:11:4:	# Dynticks-idle tracking
  tree.c:358:56: * We inform the RCU core by emulating a zero-duration dyntick-idle period.
  tree.c:773:10: * is in dynticks idle mode, which is an extended quiescent state.
  tree.c:793:54: * by virtue of being in or having passed through an dynticks idle state since
  tree.c:808:44:	 * If the CPU passed through or entered a dynticks idle phase with
  tree.c:1986:14:		/* Collect dyntick-idle snapshots. */
  tree.c:1989:13:		/* Handle dyntick-idle and offline CPUs. */
  tree.c:2683:54: * Otherwise, invoke the specified function to check dyntick state for
  tree.c:2743:16: * CPUs are in dyntick-idle mode.


- Stale documentation in RCU/Design/Data-Structures/Data-Structures.rst
  referencing fields that have been moved out of RCU after
  904e600e60f4 ("rcu/context_tracking: Move dynticks_nesting to context tracking")

Testing
=======

Ran through TREE01 and TREE04.

Revisions
=========

v2 -> v3
++++++++

o Collected Reviewed-By
o Rebased onto latest rcu/next

o Made it clear rcu_dynticks_in_eqs_since() is to be used in wait-for-EQS loops
o Slight rename variations following v2 suggestions

Cheers,
Valentin


Valentin Schneider (25):
  treewide: context_tracking: Rename CONTEXT_* into CT_STATE_*
  context_tracking, rcu: Rename RCU_DYNTICKS_IDX into CT_RCU_WATCHING
  context_tracking, rcu: Rename ct_dynticks() into ct_rcu_watching()
  context_tracking, rcu: Rename ct_dynticks_cpu() into
    ct_rcu_watching_cpu()
  context_tracking, rcu: Rename ct_dynticks_cpu_acquire() into
    ct_rcu_watching_cpu_acquire()
  context_tracking, rcu: Rename struct context_tracking
    .dynticks_nesting into .nesting
  context_tracking, rcu: Rename ct_dynticks_nesting() into ct_nesting()
  context_tracking, rcu: Rename ct_dynticks_nesting_cpu() into
    ct_nesting_cpu()
  context_tracking, rcu: Rename struct context_tracking
    .dynticks_nmi_nesting into .nmi_nesting
  context_tracking, rcu: Rename ct_dynticks_nmi_nesting() into
    ct_nmi_nesting()
  context_tracking, rcu: Rename ct_dynticks_nmi_nesting_cpu() into
    ct_nmi_nesting_cpu()
  context_tracking, rcu: Rename DYNTICK_IRQ_NONIDLE into
    CT_NESTING_IRQ_NONIDLE
  context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*()
  context_tracking, rcu: Rename rcu_dynticks_curr_cpu_in_eqs() into
    rcu_watching_curr_cpu()
  rcu: Rename rcu_dynticks_eqs_online() into rcu_watching_online()
  rcu: Rename rcu_dynticks_in_eqs() into rcu_watching_snap_in_eqs()
  rcu: Rename rcu_dynticks_in_eqs_since() into
    rcu_watching_snap_stopped_since()
  rcu: Rename rcu_dynticks_zero_in_eqs() into rcu_watching_zero_in_eqs()
  rcu: Rename struct rcu_data .dynticks_snap into .watching_snap
  rcu: Rename struct rcu_data .exp_dynticks_snap into .exp_watching_snap
  rcu: Rename dyntick_save_progress_counter() into
    rcu_watching_snap_save()
  rcu: Rename rcu_implicit_dynticks_qs() into rcu_implicit_eqs()
  rcu: Rename rcu_momentary_dyntick_idle() into rcu_momentary_eqs()
  rcu: Update stray documentation references to rcu_dynticks_eqs_{enter,
    exit}()
  context_tracking, rcu: Rename rcu_dyntick trace event into
    rcu_watching

 .../Data-Structures/Data-Structures.rst       |  28 ++--
 .../Tree-RCU-Memory-Ordering.rst              |   8 +-
 .../Memory-Ordering/TreeRCU-dyntick.svg       |   8 +-
 .../Design/Memory-Ordering/TreeRCU-gp-fqs.svg |   8 +-
 .../RCU/Design/Memory-Ordering/TreeRCU-gp.svg |   8 +-
 .../Memory-Ordering/TreeRCU-hotplug.svg       |   4 +-
 arch/Kconfig                                  |   2 +-
 arch/arm64/kernel/entry-common.c              |   2 +-
 arch/powerpc/include/asm/interrupt.h          |   6 +-
 arch/powerpc/kernel/interrupt.c               |   6 +-
 arch/powerpc/kernel/syscall.c                 |   2 +-
 arch/x86/entry/common.c                       |   2 +-
 include/linux/context_tracking.h              |  32 ++--
 include/linux/context_tracking_state.h        |  58 ++++----
 include/linux/entry-common.h                  |   2 +-
 include/linux/rcutiny.h                       |   2 +-
 include/linux/rcutree.h                       |   2 +-
 include/trace/events/rcu.h                    |  18 +--
 kernel/context_tracking.c                     | 140 +++++++++---------
 kernel/entry/common.c                         |   2 +-
 kernel/rcu/rcu.h                              |   4 +-
 kernel/rcu/rcutorture.c                       |   4 +-
 kernel/rcu/tasks.h                            |   2 +-
 kernel/rcu/tree.c                             | 111 +++++++-------
 kernel/rcu/tree.h                             |   4 +-
 kernel/rcu/tree_exp.h                         |   8 +-
 kernel/rcu/tree_nocb.h                        |   2 +-
 kernel/rcu/tree_plugin.h                      |   6 +-
 kernel/rcu/tree_stall.h                       |   6 +-
 kernel/sched/core.c                           |   4 +-
 kernel/stop_machine.c                         |   2 +-
 kernel/trace/trace_osnoise.c                  |   4 +-
 32 files changed, 256 insertions(+), 241 deletions(-)

--
2.43.0

[PATCH v3 01/25] treewide: context_tracking: Rename CONTEXT_* into CT_STATE_*

[Valentin Schneider]

Context tracking state related symbols currently use a mix of the
CONTEXT_ (e.g. CONTEXT_KERNEL) and CT_SATE_ (e.g. CT_STATE_MASK) prefixes.

Clean up the naming and make the ctx_state enum use the CT_STATE_ prefix.

Suggested-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Acked-by: Frederic Weisbecker <frederic@kernel.org>
---
 arch/Kconfig                           |  2 +-
 arch/arm64/kernel/entry-common.c       |  2 +-
 arch/powerpc/include/asm/interrupt.h   |  6 +++---
 arch/powerpc/kernel/interrupt.c        |  6 +++---
 arch/powerpc/kernel/syscall.c          |  2 +-
 arch/x86/entry/common.c                |  2 +-
 include/linux/context_tracking.h       | 16 ++++++++--------
 include/linux/context_tracking_state.h | 20 ++++++++++----------
 include/linux/entry-common.h           |  2 +-
 kernel/context_tracking.c              | 12 ++++++------
 kernel/entry/common.c                  |  2 +-
 kernel/sched/core.c                    |  4 ++--
 12 files changed, 38 insertions(+), 38 deletions(-)

diff --git a/arch/Kconfig b/arch/Kconfig
index 975dd22a2dbd2..4e2eaba9e3052 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -862,7 +862,7 @@ config HAVE_CONTEXT_TRACKING_USER_OFFSTACK
 	  Architecture neither relies on exception_enter()/exception_exit()
 	  nor on schedule_user(). Also preempt_schedule_notrace() and
 	  preempt_schedule_irq() can't be called in a preemptible section
-	  while context tracking is CONTEXT_USER. This feature reflects a sane
+	  while context tracking is CT_STATE_USER. This feature reflects a sane
 	  entry implementation where the following requirements are met on
 	  critical entry code, ie: before user_exit() or after user_enter():
 
diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
index b77a15955f28b..3fcd9d080bf2a 100644
--- a/arch/arm64/kernel/entry-common.c
+++ b/arch/arm64/kernel/entry-common.c
@@ -103,7 +103,7 @@ static void noinstr exit_to_kernel_mode(struct pt_regs *regs)
 static __always_inline void __enter_from_user_mode(void)
 {
 	lockdep_hardirqs_off(CALLER_ADDR0);
-	CT_WARN_ON(ct_state() != CONTEXT_USER);
+	CT_WARN_ON(ct_state() != CT_STATE_USER);
 	user_exit_irqoff();
 	trace_hardirqs_off_finish();
 	mte_disable_tco_entry(current);
diff --git a/arch/powerpc/include/asm/interrupt.h b/arch/powerpc/include/asm/interrupt.h
index 2d6c886b40f44..23638d4e73ac0 100644
--- a/arch/powerpc/include/asm/interrupt.h
+++ b/arch/powerpc/include/asm/interrupt.h
@@ -177,7 +177,7 @@ static inline void interrupt_enter_prepare(struct pt_regs *regs)
 
 	if (user_mode(regs)) {
 		kuap_lock();
-		CT_WARN_ON(ct_state() != CONTEXT_USER);
+		CT_WARN_ON(ct_state() != CT_STATE_USER);
 		user_exit_irqoff();
 
 		account_cpu_user_entry();
@@ -189,8 +189,8 @@ static inline void interrupt_enter_prepare(struct pt_regs *regs)
 		 * so avoid recursion.
 		 */
 		if (TRAP(regs) != INTERRUPT_PROGRAM)
-			CT_WARN_ON(ct_state() != CONTEXT_KERNEL &&
-				   ct_state() != CONTEXT_IDLE);
+			CT_WARN_ON(ct_state() != CT_STATE_KERNEL &&
+				   ct_state() != CT_STATE_IDLE);
 		INT_SOFT_MASK_BUG_ON(regs, is_implicit_soft_masked(regs));
 		INT_SOFT_MASK_BUG_ON(regs, arch_irq_disabled_regs(regs) &&
 					   search_kernel_restart_table(regs->nip));
diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c
index eca293794a1e8..af62ec974b970 100644
--- a/arch/powerpc/kernel/interrupt.c
+++ b/arch/powerpc/kernel/interrupt.c
@@ -266,7 +266,7 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3,
 	unsigned long ret = 0;
 	bool is_not_scv = !IS_ENABLED(CONFIG_PPC_BOOK3S_64) || !scv;
 
-	CT_WARN_ON(ct_state() == CONTEXT_USER);
+	CT_WARN_ON(ct_state() == CT_STATE_USER);
 
 	kuap_assert_locked();
 
@@ -344,7 +344,7 @@ notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs)
 
 	BUG_ON(regs_is_unrecoverable(regs));
 	BUG_ON(arch_irq_disabled_regs(regs));
-	CT_WARN_ON(ct_state() == CONTEXT_USER);
+	CT_WARN_ON(ct_state() == CT_STATE_USER);
 
 	/*
 	 * We don't need to restore AMR on the way back to userspace for KUAP.
@@ -386,7 +386,7 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs)
 	if (!IS_ENABLED(CONFIG_PPC_BOOK3E_64) &&
 	    TRAP(regs) != INTERRUPT_PROGRAM &&
 	    TRAP(regs) != INTERRUPT_PERFMON)
-		CT_WARN_ON(ct_state() == CONTEXT_USER);
+		CT_WARN_ON(ct_state() == CT_STATE_USER);
 
 	kuap = kuap_get_and_assert_locked();
 
diff --git a/arch/powerpc/kernel/syscall.c b/arch/powerpc/kernel/syscall.c
index f6f868e817e63..be159ad4b77bd 100644
--- a/arch/powerpc/kernel/syscall.c
+++ b/arch/powerpc/kernel/syscall.c
@@ -27,7 +27,7 @@ notrace long system_call_exception(struct pt_regs *regs, unsigned long r0)
 
 	trace_hardirqs_off(); /* finish reconciling */
 
-	CT_WARN_ON(ct_state() == CONTEXT_KERNEL);
+	CT_WARN_ON(ct_state() == CT_STATE_KERNEL);
 	user_exit_irqoff();
 
 	BUG_ON(regs_is_unrecoverable(regs));
diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
index 51cc9c7cb9bdc..94941c5a10ac1 100644
--- a/arch/x86/entry/common.c
+++ b/arch/x86/entry/common.c
@@ -150,7 +150,7 @@ early_param("ia32_emulation", ia32_emulation_override_cmdline);
 #endif
 
 /*
- * Invoke a 32-bit syscall.  Called with IRQs on in CONTEXT_KERNEL.
+ * Invoke a 32-bit syscall.  Called with IRQs on in CT_STATE_KERNEL.
  */
 static __always_inline void do_syscall_32_irqs_on(struct pt_regs *regs, int nr)
 {
diff --git a/include/linux/context_tracking.h b/include/linux/context_tracking.h
index 6e76b9dba00e7..28fcfa1849032 100644
--- a/include/linux/context_tracking.h
+++ b/include/linux/context_tracking.h
@@ -26,26 +26,26 @@ extern void user_exit_callable(void);
 static inline void user_enter(void)
 {
 	if (context_tracking_enabled())
-		ct_user_enter(CONTEXT_USER);
+		ct_user_enter(CT_STATE_USER);
 
 }
 static inline void user_exit(void)
 {
 	if (context_tracking_enabled())
-		ct_user_exit(CONTEXT_USER);
+		ct_user_exit(CT_STATE_USER);
 }
 
 /* Called with interrupts disabled.  */
 static __always_inline void user_enter_irqoff(void)
 {
 	if (context_tracking_enabled())
-		__ct_user_enter(CONTEXT_USER);
+		__ct_user_enter(CT_STATE_USER);
 
 }
 static __always_inline void user_exit_irqoff(void)
 {
 	if (context_tracking_enabled())
-		__ct_user_exit(CONTEXT_USER);
+		__ct_user_exit(CT_STATE_USER);
 }
 
 static inline enum ctx_state exception_enter(void)
@@ -57,7 +57,7 @@ static inline enum ctx_state exception_enter(void)
 		return 0;
 
 	prev_ctx = __ct_state();
-	if (prev_ctx != CONTEXT_KERNEL)
+	if (prev_ctx != CT_STATE_KERNEL)
 		ct_user_exit(prev_ctx);
 
 	return prev_ctx;
@@ -67,7 +67,7 @@ static inline void exception_exit(enum ctx_state prev_ctx)
 {
 	if (!IS_ENABLED(CONFIG_HAVE_CONTEXT_TRACKING_USER_OFFSTACK) &&
 	    context_tracking_enabled()) {
-		if (prev_ctx != CONTEXT_KERNEL)
+		if (prev_ctx != CT_STATE_KERNEL)
 			ct_user_enter(prev_ctx);
 	}
 }
@@ -75,7 +75,7 @@ static inline void exception_exit(enum ctx_state prev_ctx)
 static __always_inline bool context_tracking_guest_enter(void)
 {
 	if (context_tracking_enabled())
-		__ct_user_enter(CONTEXT_GUEST);
+		__ct_user_enter(CT_STATE_GUEST);
 
 	return context_tracking_enabled_this_cpu();
 }
@@ -83,7 +83,7 @@ static __always_inline bool context_tracking_guest_enter(void)
 static __always_inline void context_tracking_guest_exit(void)
 {
 	if (context_tracking_enabled())
-		__ct_user_exit(CONTEXT_GUEST);
+		__ct_user_exit(CT_STATE_GUEST);
 }
 
 #define CT_WARN_ON(cond) WARN_ON(context_tracking_enabled() && (cond))
diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h
index bbff5f7f88030..f1c53125edee2 100644
--- a/include/linux/context_tracking_state.h
+++ b/include/linux/context_tracking_state.h
@@ -10,18 +10,18 @@
 #define DYNTICK_IRQ_NONIDLE	((LONG_MAX / 2) + 1)
 
 enum ctx_state {
-	CONTEXT_DISABLED	= -1,	/* returned by ct_state() if unknown */
-	CONTEXT_KERNEL		= 0,
-	CONTEXT_IDLE		= 1,
-	CONTEXT_USER		= 2,
-	CONTEXT_GUEST		= 3,
-	CONTEXT_MAX		= 4,
+	CT_STATE_DISABLED	= -1,	/* returned by ct_state() if unknown */
+	CT_STATE_KERNEL		= 0,
+	CT_STATE_IDLE		= 1,
+	CT_STATE_USER		= 2,
+	CT_STATE_GUEST		= 3,
+	CT_STATE_MAX		= 4,
 };
 
 /* Even value for idle, else odd. */
-#define RCU_DYNTICKS_IDX CONTEXT_MAX
+#define RCU_DYNTICKS_IDX CT_STATE_MAX
 
-#define CT_STATE_MASK (CONTEXT_MAX - 1)
+#define CT_STATE_MASK (CT_STATE_MAX - 1)
 #define CT_DYNTICKS_MASK (~CT_STATE_MASK)
 
 struct context_tracking {
@@ -123,14 +123,14 @@ static inline bool context_tracking_enabled_this_cpu(void)
  *
  * Returns the current cpu's context tracking state if context tracking
  * is enabled.  If context tracking is disabled, returns
- * CONTEXT_DISABLED.  This should be used primarily for debugging.
+ * CT_STATE_DISABLED.  This should be used primarily for debugging.
  */
 static __always_inline int ct_state(void)
 {
 	int ret;
 
 	if (!context_tracking_enabled())
-		return CONTEXT_DISABLED;
+		return CT_STATE_DISABLED;
 
 	preempt_disable();
 	ret = __ct_state();
diff --git a/include/linux/entry-common.h b/include/linux/entry-common.h
index b0fb775a600d9..1e50cdb83ae50 100644
--- a/include/linux/entry-common.h
+++ b/include/linux/entry-common.h
@@ -108,7 +108,7 @@ static __always_inline void enter_from_user_mode(struct pt_regs *regs)
 	arch_enter_from_user_mode(regs);
 	lockdep_hardirqs_off(CALLER_ADDR0);
 
-	CT_WARN_ON(__ct_state() != CONTEXT_USER);
+	CT_WARN_ON(__ct_state() != CT_STATE_USER);
 	user_exit_irqoff();
 
 	instrumentation_begin();
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index 24b1e11432608..4bb5751af994f 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -317,7 +317,7 @@ void noinstr ct_nmi_enter(void)
 void noinstr ct_idle_enter(void)
 {
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !raw_irqs_disabled());
-	ct_kernel_exit(false, RCU_DYNTICKS_IDX + CONTEXT_IDLE);
+	ct_kernel_exit(false, RCU_DYNTICKS_IDX + CT_STATE_IDLE);
 }
 EXPORT_SYMBOL_GPL(ct_idle_enter);
 
@@ -335,7 +335,7 @@ void noinstr ct_idle_exit(void)
 	unsigned long flags;
 
 	raw_local_irq_save(flags);
-	ct_kernel_enter(false, RCU_DYNTICKS_IDX - CONTEXT_IDLE);
+	ct_kernel_enter(false, RCU_DYNTICKS_IDX - CT_STATE_IDLE);
 	raw_local_irq_restore(flags);
 }
 EXPORT_SYMBOL_GPL(ct_idle_exit);
@@ -485,7 +485,7 @@ void noinstr __ct_user_enter(enum ctx_state state)
 			 * user_exit() or ct_irq_enter(). Let's remove RCU's dependency
 			 * on the tick.
 			 */
-			if (state == CONTEXT_USER) {
+			if (state == CT_STATE_USER) {
 				instrumentation_begin();
 				trace_user_enter(0);
 				vtime_user_enter(current);
@@ -621,7 +621,7 @@ void noinstr __ct_user_exit(enum ctx_state state)
 			 * run a RCU read side critical section anytime.
 			 */
 			ct_kernel_enter(true, RCU_DYNTICKS_IDX - state);
-			if (state == CONTEXT_USER) {
+			if (state == CT_STATE_USER) {
 				instrumentation_begin();
 				vtime_user_exit(current);
 				trace_user_exit(0);
@@ -634,12 +634,12 @@ void noinstr __ct_user_exit(enum ctx_state state)
 			 * In this we case we don't care about any concurrency/ordering.
 			 */
 			if (!IS_ENABLED(CONFIG_CONTEXT_TRACKING_IDLE))
-				raw_atomic_set(&ct->state, CONTEXT_KERNEL);
+				raw_atomic_set(&ct->state, CT_STATE_KERNEL);
 
 		} else {
 			if (!IS_ENABLED(CONFIG_CONTEXT_TRACKING_IDLE)) {
 				/* Tracking for vtime only, no concurrent RCU EQS accounting */
-				raw_atomic_set(&ct->state, CONTEXT_KERNEL);
+				raw_atomic_set(&ct->state, CT_STATE_KERNEL);
 			} else {
 				/*
 				 * Tracking for vtime and RCU EQS. Make sure we don't race
diff --git a/kernel/entry/common.c b/kernel/entry/common.c
index 90843cc385880..5b6934e23c21d 100644
--- a/kernel/entry/common.c
+++ b/kernel/entry/common.c
@@ -182,7 +182,7 @@ static void syscall_exit_to_user_mode_prepare(struct pt_regs *regs)
 	unsigned long work = READ_ONCE(current_thread_info()->syscall_work);
 	unsigned long nr = syscall_get_nr(current, regs);
 
-	CT_WARN_ON(ct_state() != CONTEXT_KERNEL);
+	CT_WARN_ON(ct_state() != CT_STATE_KERNEL);
 
 	if (IS_ENABLED(CONFIG_PROVE_LOCKING)) {
 		if (WARN(irqs_disabled(), "syscall %lu left IRQs disabled", nr))
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 05afa2932b5e4..b0af25c4dd707 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5979,7 +5979,7 @@ static inline void schedule_debug(struct task_struct *prev, bool preempt)
 		preempt_count_set(PREEMPT_DISABLED);
 	}
 	rcu_sleep_check();
-	SCHED_WARN_ON(ct_state() == CONTEXT_USER);
+	SCHED_WARN_ON(ct_state() == CT_STATE_USER);
 
 	profile_hit(SCHED_PROFILING, __builtin_return_address(0));
 
@@ -6874,7 +6874,7 @@ asmlinkage __visible void __sched schedule_user(void)
 	 * we find a better solution.
 	 *
 	 * NB: There are buggy callers of this function.  Ideally we
-	 * should warn if prev_state != CONTEXT_USER, but that will trigger
+	 * should warn if prev_state != CT_STATE_USER, but that will trigger
 	 * too frequently to make sense yet.
 	 */
 	enum ctx_state prev_state = exception_enter();
-- 
2.43.0

[PATCH v3 02/25] context_tracking, rcu: Rename RCU_DYNTICKS_IDX into CT_RCU_WATCHING

[Valentin Schneider]

The symbols relating to the CT_STATE part of context_tracking.state are now
all prefixed with CT_STATE.

The RCU dynticks counter part of that atomic variable still involves
symbols with different prefixes, align them all to be prefixed with
CT_RCU_WATCHING.

Suggested-by: "Paul E. McKenney" <paulmck@kernel.org>
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 include/linux/context_tracking.h       |  6 +++---
 include/linux/context_tracking_state.h | 12 ++++++------
 kernel/context_tracking.c              | 22 +++++++++++-----------
 kernel/rcu/tree.c                      | 12 ++++++------
 4 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/include/linux/context_tracking.h b/include/linux/context_tracking.h
index 28fcfa1849032..a6c36780cc3bd 100644
--- a/include/linux/context_tracking.h
+++ b/include/linux/context_tracking.h
@@ -119,7 +119,7 @@ extern void ct_idle_exit(void);
  */
 static __always_inline bool rcu_dynticks_curr_cpu_in_eqs(void)
 {
-	return !(raw_atomic_read(this_cpu_ptr(&context_tracking.state)) & RCU_DYNTICKS_IDX);
+	return !(raw_atomic_read(this_cpu_ptr(&context_tracking.state)) & CT_RCU_WATCHING);
 }
 
 /*
@@ -142,7 +142,7 @@ static __always_inline bool warn_rcu_enter(void)
 	preempt_disable_notrace();
 	if (rcu_dynticks_curr_cpu_in_eqs()) {
 		ret = true;
-		ct_state_inc(RCU_DYNTICKS_IDX);
+		ct_state_inc(CT_RCU_WATCHING);
 	}
 
 	return ret;
@@ -151,7 +151,7 @@ static __always_inline bool warn_rcu_enter(void)
 static __always_inline void warn_rcu_exit(bool rcu)
 {
 	if (rcu)
-		ct_state_inc(RCU_DYNTICKS_IDX);
+		ct_state_inc(CT_RCU_WATCHING);
 	preempt_enable_notrace();
 }
 
diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h
index f1c53125edee2..94d6a935af3be 100644
--- a/include/linux/context_tracking_state.h
+++ b/include/linux/context_tracking_state.h
@@ -18,11 +18,11 @@ enum ctx_state {
 	CT_STATE_MAX		= 4,
 };
 
-/* Even value for idle, else odd. */
-#define RCU_DYNTICKS_IDX CT_STATE_MAX
+/* Odd value for watching, else even. */
+#define CT_RCU_WATCHING CT_STATE_MAX
 
 #define CT_STATE_MASK (CT_STATE_MAX - 1)
-#define CT_DYNTICKS_MASK (~CT_STATE_MASK)
+#define CT_RCU_WATCHING_MASK (~CT_STATE_MASK)
 
 struct context_tracking {
 #ifdef CONFIG_CONTEXT_TRACKING_USER
@@ -58,21 +58,21 @@ static __always_inline int __ct_state(void)
 #ifdef CONFIG_CONTEXT_TRACKING_IDLE
 static __always_inline int ct_dynticks(void)
 {
-	return atomic_read(this_cpu_ptr(&context_tracking.state)) & CT_DYNTICKS_MASK;
+	return atomic_read(this_cpu_ptr(&context_tracking.state)) & CT_RCU_WATCHING_MASK;
 }
 
 static __always_inline int ct_dynticks_cpu(int cpu)
 {
 	struct context_tracking *ct = per_cpu_ptr(&context_tracking, cpu);
 
-	return atomic_read(&ct->state) & CT_DYNTICKS_MASK;
+	return atomic_read(&ct->state) & CT_RCU_WATCHING_MASK;
 }
 
 static __always_inline int ct_dynticks_cpu_acquire(int cpu)
 {
 	struct context_tracking *ct = per_cpu_ptr(&context_tracking, cpu);
 
-	return atomic_read_acquire(&ct->state) & CT_DYNTICKS_MASK;
+	return atomic_read_acquire(&ct->state) & CT_RCU_WATCHING_MASK;
 }
 
 static __always_inline long ct_dynticks_nesting(void)
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index 4bb5751af994f..b2589bc59e186 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -31,7 +31,7 @@ DEFINE_PER_CPU(struct context_tracking, context_tracking) = {
 	.dynticks_nesting = 1,
 	.dynticks_nmi_nesting = DYNTICK_IRQ_NONIDLE,
 #endif
-	.state = ATOMIC_INIT(RCU_DYNTICKS_IDX),
+	.state = ATOMIC_INIT(CT_RCU_WATCHING),
 };
 EXPORT_SYMBOL_GPL(context_tracking);
 
@@ -90,7 +90,7 @@ static noinstr void ct_kernel_exit_state(int offset)
 	rcu_dynticks_task_trace_enter();  // Before ->dynticks update!
 	seq = ct_state_inc(offset);
 	// RCU is no longer watching.  Better be in extended quiescent state!
-	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && (seq & RCU_DYNTICKS_IDX));
+	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && (seq & CT_RCU_WATCHING));
 }
 
 /*
@@ -110,7 +110,7 @@ static noinstr void ct_kernel_enter_state(int offset)
 	seq = ct_state_inc(offset);
 	// RCU is now watching.  Better not be in an extended quiescent state!
 	rcu_dynticks_task_trace_exit();  // After ->dynticks update!
-	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !(seq & RCU_DYNTICKS_IDX));
+	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !(seq & CT_RCU_WATCHING));
 }
 
 /*
@@ -236,7 +236,7 @@ void noinstr ct_nmi_exit(void)
 	instrumentation_end();
 
 	// RCU is watching here ...
-	ct_kernel_exit_state(RCU_DYNTICKS_IDX);
+	ct_kernel_exit_state(CT_RCU_WATCHING);
 	// ... but is no longer watching here.
 
 	if (!in_nmi())
@@ -277,7 +277,7 @@ void noinstr ct_nmi_enter(void)
 			rcu_dynticks_task_exit();
 
 		// RCU is not watching here ...
-		ct_kernel_enter_state(RCU_DYNTICKS_IDX);
+		ct_kernel_enter_state(CT_RCU_WATCHING);
 		// ... but is watching here.
 
 		instrumentation_begin();
@@ -317,7 +317,7 @@ void noinstr ct_nmi_enter(void)
 void noinstr ct_idle_enter(void)
 {
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !raw_irqs_disabled());
-	ct_kernel_exit(false, RCU_DYNTICKS_IDX + CT_STATE_IDLE);
+	ct_kernel_exit(false, CT_RCU_WATCHING + CT_STATE_IDLE);
 }
 EXPORT_SYMBOL_GPL(ct_idle_enter);
 
@@ -335,7 +335,7 @@ void noinstr ct_idle_exit(void)
 	unsigned long flags;
 
 	raw_local_irq_save(flags);
-	ct_kernel_enter(false, RCU_DYNTICKS_IDX - CT_STATE_IDLE);
+	ct_kernel_enter(false, CT_RCU_WATCHING - CT_STATE_IDLE);
 	raw_local_irq_restore(flags);
 }
 EXPORT_SYMBOL_GPL(ct_idle_exit);
@@ -504,7 +504,7 @@ void noinstr __ct_user_enter(enum ctx_state state)
 			 * CPU doesn't need to maintain the tick for RCU maintenance purposes
 			 * when the CPU runs in userspace.
 			 */
-			ct_kernel_exit(true, RCU_DYNTICKS_IDX + state);
+			ct_kernel_exit(true, CT_RCU_WATCHING + state);
 
 			/*
 			 * Special case if we only track user <-> kernel transitions for tickless
@@ -534,7 +534,7 @@ void noinstr __ct_user_enter(enum ctx_state state)
 				/*
 				 * Tracking for vtime and RCU EQS. Make sure we don't race
 				 * with NMIs. OTOH we don't care about ordering here since
-				 * RCU only requires RCU_DYNTICKS_IDX increments to be fully
+				 * RCU only requires CT_RCU_WATCHING increments to be fully
 				 * ordered.
 				 */
 				raw_atomic_add(state, &ct->state);
@@ -620,7 +620,7 @@ void noinstr __ct_user_exit(enum ctx_state state)
 			 * Exit RCU idle mode while entering the kernel because it can
 			 * run a RCU read side critical section anytime.
 			 */
-			ct_kernel_enter(true, RCU_DYNTICKS_IDX - state);
+			ct_kernel_enter(true, CT_RCU_WATCHING - state);
 			if (state == CT_STATE_USER) {
 				instrumentation_begin();
 				vtime_user_exit(current);
@@ -644,7 +644,7 @@ void noinstr __ct_user_exit(enum ctx_state state)
 				/*
 				 * Tracking for vtime and RCU EQS. Make sure we don't race
 				 * with NMIs. OTOH we don't care about ordering here since
-				 * RCU only requires RCU_DYNTICKS_IDX increments to be fully
+				 * RCU only requires CT_RCU_WATCHING increments to be fully
 				 * ordered.
 				 */
 				raw_atomic_sub(state, &ct->state);
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 10474fedcd181..c0d7fbe8f6756 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -294,9 +294,9 @@ void rcu_softirq_qs(void)
  */
 static void rcu_dynticks_eqs_online(void)
 {
-	if (ct_dynticks() & RCU_DYNTICKS_IDX)
+	if (ct_dynticks() & CT_RCU_WATCHING)
 		return;
-	ct_state_inc(RCU_DYNTICKS_IDX);
+	ct_state_inc(CT_RCU_WATCHING);
 }
 
 /*
@@ -305,7 +305,7 @@ static void rcu_dynticks_eqs_online(void)
  */
 static bool rcu_dynticks_in_eqs(int snap)
 {
-	return !(snap & RCU_DYNTICKS_IDX);
+	return !(snap & CT_RCU_WATCHING);
 }
 
 /*
@@ -327,7 +327,7 @@ bool rcu_dynticks_zero_in_eqs(int cpu, int *vp)
 	int snap;
 
 	// If not quiescent, force back to earlier extended quiescent state.
-	snap = ct_dynticks_cpu(cpu) & ~RCU_DYNTICKS_IDX;
+	snap = ct_dynticks_cpu(cpu) & ~CT_RCU_WATCHING;
 	smp_rmb(); // Order ->dynticks and *vp reads.
 	if (READ_ONCE(*vp))
 		return false;  // Non-zero, so report failure;
@@ -353,9 +353,9 @@ notrace void rcu_momentary_dyntick_idle(void)
 	int seq;
 
 	raw_cpu_write(rcu_data.rcu_need_heavy_qs, false);
-	seq = ct_state_inc(2 * RCU_DYNTICKS_IDX);
+	seq = ct_state_inc(2 * CT_RCU_WATCHING);
 	/* It is illegal to call this from idle state. */
-	WARN_ON_ONCE(!(seq & RCU_DYNTICKS_IDX));
+	WARN_ON_ONCE(!(seq & CT_RCU_WATCHING));
 	rcu_preempt_deferred_qs(current);
 }
 EXPORT_SYMBOL_GPL(rcu_momentary_dyntick_idle);
-- 
2.43.0

[PATCH v3 03/25] context_tracking, rcu: Rename ct_dynticks() into ct_rcu_watching()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, reflect that change in the related helpers.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 include/linux/context_tracking_state.h |  2 +-
 kernel/context_tracking.c              | 10 +++++-----
 kernel/rcu/tree.c                      |  2 +-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h
index 94d6a935af3be..cb90d8c178104 100644
--- a/include/linux/context_tracking_state.h
+++ b/include/linux/context_tracking_state.h
@@ -56,7 +56,7 @@ static __always_inline int __ct_state(void)
 #endif
 
 #ifdef CONFIG_CONTEXT_TRACKING_IDLE
-static __always_inline int ct_dynticks(void)
+static __always_inline int ct_rcu_watching(void)
 {
 	return atomic_read(this_cpu_ptr(&context_tracking.state)) & CT_RCU_WATCHING_MASK;
 }
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index b2589bc59e186..868ae0bcd4bed 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -137,7 +137,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
 
 	instrumentation_begin();
 	lockdep_assert_irqs_disabled();
-	trace_rcu_dyntick(TPS("Start"), ct_dynticks_nesting(), 0, ct_dynticks());
+	trace_rcu_dyntick(TPS("Start"), ct_dynticks_nesting(), 0, ct_rcu_watching());
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current));
 	rcu_preempt_deferred_qs(current);
 
@@ -182,7 +182,7 @@ static void noinstr ct_kernel_enter(bool user, int offset)
 	// instrumentation for the noinstr ct_kernel_enter_state()
 	instrument_atomic_write(&ct->state, sizeof(ct->state));
 
-	trace_rcu_dyntick(TPS("End"), ct_dynticks_nesting(), 1, ct_dynticks());
+	trace_rcu_dyntick(TPS("End"), ct_dynticks_nesting(), 1, ct_rcu_watching());
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current));
 	WRITE_ONCE(ct->dynticks_nesting, 1);
 	WARN_ON_ONCE(ct_dynticks_nmi_nesting());
@@ -220,7 +220,7 @@ void noinstr ct_nmi_exit(void)
 	 */
 	if (ct_dynticks_nmi_nesting() != 1) {
 		trace_rcu_dyntick(TPS("--="), ct_dynticks_nmi_nesting(), ct_dynticks_nmi_nesting() - 2,
-				  ct_dynticks());
+				  ct_rcu_watching());
 		WRITE_ONCE(ct->dynticks_nmi_nesting, /* No store tearing. */
 			   ct_dynticks_nmi_nesting() - 2);
 		instrumentation_end();
@@ -228,7 +228,7 @@ void noinstr ct_nmi_exit(void)
 	}
 
 	/* This NMI interrupted an RCU-idle CPU, restore RCU-idleness. */
-	trace_rcu_dyntick(TPS("Startirq"), ct_dynticks_nmi_nesting(), 0, ct_dynticks());
+	trace_rcu_dyntick(TPS("Startirq"), ct_dynticks_nmi_nesting(), 0, ct_rcu_watching());
 	WRITE_ONCE(ct->dynticks_nmi_nesting, 0); /* Avoid store tearing. */
 
 	// instrumentation for the noinstr ct_kernel_exit_state()
@@ -296,7 +296,7 @@ void noinstr ct_nmi_enter(void)
 
 	trace_rcu_dyntick(incby == 1 ? TPS("Endirq") : TPS("++="),
 			  ct_dynticks_nmi_nesting(),
-			  ct_dynticks_nmi_nesting() + incby, ct_dynticks());
+			  ct_dynticks_nmi_nesting() + incby, ct_rcu_watching());
 	instrumentation_end();
 	WRITE_ONCE(ct->dynticks_nmi_nesting, /* Prevent store tearing. */
 		   ct_dynticks_nmi_nesting() + incby);
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index c0d7fbe8f6756..ae478e13fc837 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -294,7 +294,7 @@ void rcu_softirq_qs(void)
  */
 static void rcu_dynticks_eqs_online(void)
 {
-	if (ct_dynticks() & CT_RCU_WATCHING)
+	if (ct_rcu_watching() & CT_RCU_WATCHING)
 		return;
 	ct_state_inc(CT_RCU_WATCHING);
 }
-- 
2.43.0

[PATCH v3 04/25] context_tracking, rcu: Rename ct_dynticks_cpu() into ct_rcu_watching_cpu()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, reflect that change in the related helpers.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 include/linux/context_tracking_state.h |  2 +-
 kernel/rcu/tree.c                      | 10 +++++-----
 kernel/rcu/tree_stall.h                |  4 ++--
 3 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h
index cb90d8c178104..ad5a06a42b4a0 100644
--- a/include/linux/context_tracking_state.h
+++ b/include/linux/context_tracking_state.h
@@ -61,7 +61,7 @@ static __always_inline int ct_rcu_watching(void)
 	return atomic_read(this_cpu_ptr(&context_tracking.state)) & CT_RCU_WATCHING_MASK;
 }
 
-static __always_inline int ct_dynticks_cpu(int cpu)
+static __always_inline int ct_rcu_watching_cpu(int cpu)
 {
 	struct context_tracking *ct = per_cpu_ptr(&context_tracking, cpu);
 
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index ae478e13fc837..e60f878444704 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -327,14 +327,14 @@ bool rcu_dynticks_zero_in_eqs(int cpu, int *vp)
 	int snap;
 
 	// If not quiescent, force back to earlier extended quiescent state.
-	snap = ct_dynticks_cpu(cpu) & ~CT_RCU_WATCHING;
-	smp_rmb(); // Order ->dynticks and *vp reads.
+	snap = ct_rcu_watching_cpu(cpu) & ~CT_RCU_WATCHING;
+	smp_rmb(); // Order CT state and *vp reads.
 	if (READ_ONCE(*vp))
 		return false;  // Non-zero, so report failure;
-	smp_rmb(); // Order *vp read and ->dynticks re-read.
+	smp_rmb(); // Order *vp read and CT state re-read.
 
 	// If still in the same extended quiescent state, we are good!
-	return snap == ct_dynticks_cpu(cpu);
+	return snap == ct_rcu_watching_cpu(cpu);
 }
 
 /*
@@ -4793,7 +4793,7 @@ rcu_boot_init_percpu_data(int cpu)
 	rdp->grpmask = leaf_node_cpu_bit(rdp->mynode, cpu);
 	INIT_WORK(&rdp->strict_work, strict_work_handler);
 	WARN_ON_ONCE(ct->dynticks_nesting != 1);
-	WARN_ON_ONCE(rcu_dynticks_in_eqs(ct_dynticks_cpu(cpu)));
+	WARN_ON_ONCE(rcu_dynticks_in_eqs(ct_rcu_watching_cpu(cpu)));
 	rdp->barrier_seq_snap = rcu_state.barrier_sequence;
 	rdp->rcu_ofl_gp_seq = rcu_state.gp_seq;
 	rdp->rcu_ofl_gp_state = RCU_GP_CLEANED;
diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h
index 4b0e9d7c4c68e..d65974448e813 100644
--- a/kernel/rcu/tree_stall.h
+++ b/kernel/rcu/tree_stall.h
@@ -501,7 +501,7 @@ static void print_cpu_stall_info(int cpu)
 	}
 	delta = rcu_seq_ctr(rdp->mynode->gp_seq - rdp->rcu_iw_gp_seq);
 	falsepositive = rcu_is_gp_kthread_starving(NULL) &&
-			rcu_dynticks_in_eqs(ct_dynticks_cpu(cpu));
+			rcu_dynticks_in_eqs(ct_rcu_watching_cpu(cpu));
 	rcuc_starved = rcu_is_rcuc_kthread_starving(rdp, &j);
 	if (rcuc_starved)
 		// Print signed value, as negative values indicate a probable bug.
@@ -515,7 +515,7 @@ static void print_cpu_stall_info(int cpu)
 			rdp->rcu_iw_pending ? (int)min(delta, 9UL) + '0' :
 				"!."[!delta],
 	       ticks_value, ticks_title,
-	       ct_dynticks_cpu(cpu) & 0xffff,
+	       ct_rcu_watching_cpu(cpu) & 0xffff,
 	       ct_dynticks_nesting_cpu(cpu), ct_dynticks_nmi_nesting_cpu(cpu),
 	       rdp->softirq_snap, kstat_softirqs_cpu(RCU_SOFTIRQ, cpu),
 	       data_race(rcu_state.n_force_qs) - rcu_state.n_force_qs_gpstart,
-- 
2.43.0

[PATCH v3 05/25] context_tracking, rcu: Rename ct_dynticks_cpu_acquire() into ct_rcu_watching_cpu_acquire()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, reflect that change in the related helpers.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
---
 .../RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst   | 2 +-
 include/linux/context_tracking_state.h                        | 2 +-
 kernel/rcu/tree.c                                             | 4 ++--
 kernel/rcu/tree_exp.h                                         | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst b/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
index 728b1e690c646..2d7036ad74761 100644
--- a/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
+++ b/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
@@ -149,7 +149,7 @@ This case is handled by calls to the strongly ordered
 ``atomic_add_return()`` read-modify-write atomic operation that
 is invoked within ``rcu_dynticks_eqs_enter()`` at idle-entry
 time and within ``rcu_dynticks_eqs_exit()`` at idle-exit time.
-The grace-period kthread invokes first ``ct_dynticks_cpu_acquire()``
+The grace-period kthread invokes first ``ct_rcu_watching_cpu_acquire()``
 (preceded by a full memory barrier) and ``rcu_dynticks_in_eqs_since()``
 (both of which rely on acquire semantics) to detect idle CPUs.
 
diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h
index ad5a06a42b4a0..ad6570ffeff3c 100644
--- a/include/linux/context_tracking_state.h
+++ b/include/linux/context_tracking_state.h
@@ -68,7 +68,7 @@ static __always_inline int ct_rcu_watching_cpu(int cpu)
 	return atomic_read(&ct->state) & CT_RCU_WATCHING_MASK;
 }
 
-static __always_inline int ct_dynticks_cpu_acquire(int cpu)
+static __always_inline int ct_rcu_watching_cpu_acquire(int cpu)
 {
 	struct context_tracking *ct = per_cpu_ptr(&context_tracking, cpu);
 
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index e60f878444704..273d223bd8efc 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -315,7 +315,7 @@ static bool rcu_dynticks_in_eqs(int snap)
  */
 static bool rcu_dynticks_in_eqs_since(struct rcu_data *rdp, int snap)
 {
-	return snap != ct_dynticks_cpu_acquire(rdp->cpu);
+	return snap != ct_rcu_watching_cpu_acquire(rdp->cpu);
 }
 
 /*
@@ -768,7 +768,7 @@ static int dyntick_save_progress_counter(struct rcu_data *rdp)
 	 * current GP sequence number is enforced by current rnp locking
 	 * with chained smp_mb__after_unlock_lock().
 	 */
-	rdp->dynticks_snap = ct_dynticks_cpu_acquire(rdp->cpu);
+	rdp->dynticks_snap = ct_rcu_watching_cpu_acquire(rdp->cpu);
 	if (rcu_dynticks_in_eqs(rdp->dynticks_snap)) {
 		trace_rcu_fqs(rcu_state.name, rdp->gp_seq, rdp->cpu, TPS("dti"));
 		rcu_gpnum_ovf(rdp->mynode, rdp);
diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h
index 721cb93b1fece..89cbadb15a4e7 100644
--- a/kernel/rcu/tree_exp.h
+++ b/kernel/rcu/tree_exp.h
@@ -368,7 +368,7 @@ static void __sync_rcu_exp_select_node_cpus(struct rcu_exp_work *rewp)
 			 * by current rnp locking with chained
 			 * smp_mb__after_unlock_lock().
 			 */
-			snap = ct_dynticks_cpu_acquire(cpu);
+			snap = ct_rcu_watching_cpu_acquire(cpu);
 			if (rcu_dynticks_in_eqs(snap))
 				mask_ofl_test |= mask;
 			else
-- 
2.43.0

[PATCH v3 06/25] context_tracking, rcu: Rename struct context_tracking .dynticks_nesting into .nesting

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, reflect that change in the related helpers.

Suggested-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 .../RCU/Design/Data-Structures/Data-Structures.rst     | 10 +++++-----
 include/linux/context_tracking_state.h                 |  6 +++---
 include/trace/events/rcu.h                             |  2 +-
 kernel/context_tracking.c                              | 10 +++++-----
 kernel/rcu/tree.c                                      |  8 ++++----
 5 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/Documentation/RCU/Design/Data-Structures/Data-Structures.rst b/Documentation/RCU/Design/Data-Structures/Data-Structures.rst
index b34990c7c3778..57ffc33d3cce2 100644
--- a/Documentation/RCU/Design/Data-Structures/Data-Structures.rst
+++ b/Documentation/RCU/Design/Data-Structures/Data-Structures.rst
@@ -935,7 +935,7 @@ This portion of the rcu_data structure is declared as follows:
 
 ::
 
-     1   long dynticks_nesting;
+     1   long nesting;
      2   long dynticks_nmi_nesting;
      3   atomic_t dynticks;
      4   bool rcu_need_heavy_qs;
@@ -945,7 +945,7 @@ These fields in the rcu_data structure maintain the per-CPU dyntick-idle
 state for the corresponding CPU. The fields may be accessed only from
 the corresponding CPU (and from tracing) unless otherwise stated.
 
-The ``->dynticks_nesting`` field counts the nesting depth of process
+The ``->nesting`` field counts the nesting depth of process
 execution, so that in normal circumstances this counter has value zero
 or one. NMIs, irqs, and tracers are counted by the
 ``->dynticks_nmi_nesting`` field. Because NMIs cannot be masked, changes
@@ -960,9 +960,9 @@ process-level transitions.
 However, it turns out that when running in non-idle kernel context, the
 Linux kernel is fully capable of entering interrupt handlers that never
 exit and perhaps also vice versa. Therefore, whenever the
-``->dynticks_nesting`` field is incremented up from zero, the
+``->nesting`` field is incremented up from zero, the
 ``->dynticks_nmi_nesting`` field is set to a large positive number, and
-whenever the ``->dynticks_nesting`` field is decremented down to zero,
+whenever the ``->nesting`` field is decremented down to zero,
 the ``->dynticks_nmi_nesting`` field is set to zero. Assuming that
 the number of misnested interrupts is not sufficient to overflow the
 counter, this approach corrects the ``->dynticks_nmi_nesting`` field
@@ -992,7 +992,7 @@ code.
 +-----------------------------------------------------------------------+
 | **Quick Quiz**:                                                       |
 +-----------------------------------------------------------------------+
-| Why not simply combine the ``->dynticks_nesting`` and                 |
+| Why not simply combine the ``->nesting`` and                 |
 | ``->dynticks_nmi_nesting`` counters into a single counter that just   |
 | counts the number of reasons that the corresponding CPU is non-idle?  |
 +-----------------------------------------------------------------------+
diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h
index ad6570ffeff3c..65290e7677e6c 100644
--- a/include/linux/context_tracking_state.h
+++ b/include/linux/context_tracking_state.h
@@ -39,7 +39,7 @@ struct context_tracking {
 	atomic_t state;
 #endif
 #ifdef CONFIG_CONTEXT_TRACKING_IDLE
-	long dynticks_nesting;		/* Track process nesting level. */
+	long nesting;		/* Track process nesting level. */
 	long dynticks_nmi_nesting;	/* Track irq/NMI nesting level. */
 #endif
 };
@@ -77,14 +77,14 @@ static __always_inline int ct_rcu_watching_cpu_acquire(int cpu)
 
 static __always_inline long ct_dynticks_nesting(void)
 {
-	return __this_cpu_read(context_tracking.dynticks_nesting);
+	return __this_cpu_read(context_tracking.nesting);
 }
 
 static __always_inline long ct_dynticks_nesting_cpu(int cpu)
 {
 	struct context_tracking *ct = per_cpu_ptr(&context_tracking, cpu);
 
-	return ct->dynticks_nesting;
+	return ct->nesting;
 }
 
 static __always_inline long ct_dynticks_nmi_nesting(void)
diff --git a/include/trace/events/rcu.h b/include/trace/events/rcu.h
index 31b3e0d3e65f7..4066b6d51e46a 100644
--- a/include/trace/events/rcu.h
+++ b/include/trace/events/rcu.h
@@ -469,7 +469,7 @@ TRACE_EVENT(rcu_stall_warning,
  * polarity: "Start", "End", "StillNonIdle" for entering, exiting or still not
  *            being in dyntick-idle mode.
  * context: "USER" or "IDLE" or "IRQ".
- * NMIs nested in IRQs are inferred with dynticks_nesting > 1 in IRQ context.
+ * NMIs nested in IRQs are inferred with nesting > 1 in IRQ context.
  *
  * These events also take a pair of numbers, which indicate the nesting
  * depth before and after the event of interest, and a third number that is
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index 868ae0bcd4bed..5cfdfc03b4018 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -28,7 +28,7 @@
 
 DEFINE_PER_CPU(struct context_tracking, context_tracking) = {
 #ifdef CONFIG_CONTEXT_TRACKING_IDLE
-	.dynticks_nesting = 1,
+	.nesting = 1,
 	.dynticks_nmi_nesting = DYNTICK_IRQ_NONIDLE,
 #endif
 	.state = ATOMIC_INIT(CT_RCU_WATCHING),
@@ -131,7 +131,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
 		     ct_dynticks_nesting() == 0);
 	if (ct_dynticks_nesting() != 1) {
 		// RCU will still be watching, so just do accounting and leave.
-		ct->dynticks_nesting--;
+		ct->nesting--;
 		return;
 	}
 
@@ -145,7 +145,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
 	instrument_atomic_write(&ct->state, sizeof(ct->state));
 
 	instrumentation_end();
-	WRITE_ONCE(ct->dynticks_nesting, 0); /* Avoid irq-access tearing. */
+	WRITE_ONCE(ct->nesting, 0); /* Avoid irq-access tearing. */
 	// RCU is watching here ...
 	ct_kernel_exit_state(offset);
 	// ... but is no longer watching here.
@@ -170,7 +170,7 @@ static void noinstr ct_kernel_enter(bool user, int offset)
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && oldval < 0);
 	if (oldval) {
 		// RCU was already watching, so just do accounting and leave.
-		ct->dynticks_nesting++;
+		ct->nesting++;
 		return;
 	}
 	rcu_dynticks_task_exit();
@@ -184,7 +184,7 @@ static void noinstr ct_kernel_enter(bool user, int offset)
 
 	trace_rcu_dyntick(TPS("End"), ct_dynticks_nesting(), 1, ct_rcu_watching());
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current));
-	WRITE_ONCE(ct->dynticks_nesting, 1);
+	WRITE_ONCE(ct->nesting, 1);
 	WARN_ON_ONCE(ct_dynticks_nmi_nesting());
 	WRITE_ONCE(ct->dynticks_nmi_nesting, DYNTICK_IRQ_NONIDLE);
 	instrumentation_end();
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 273d223bd8efc..16007911c7890 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -381,7 +381,7 @@ static int rcu_is_cpu_rrupt_from_idle(void)
 
 	/* Check for counter underflows */
 	RCU_LOCKDEP_WARN(ct_dynticks_nesting() < 0,
-			 "RCU dynticks_nesting counter underflow!");
+			 "RCU nesting counter underflow!");
 	RCU_LOCKDEP_WARN(ct_dynticks_nmi_nesting() <= 0,
 			 "RCU dynticks_nmi_nesting counter underflow/zero!");
 
@@ -589,7 +589,7 @@ void rcu_irq_exit_check_preempt(void)
 	lockdep_assert_irqs_disabled();
 
 	RCU_LOCKDEP_WARN(ct_dynticks_nesting() <= 0,
-			 "RCU dynticks_nesting counter underflow/zero!");
+			 "RCU nesting counter underflow/zero!");
 	RCU_LOCKDEP_WARN(ct_dynticks_nmi_nesting() !=
 			 DYNTICK_IRQ_NONIDLE,
 			 "Bad RCU  dynticks_nmi_nesting counter\n");
@@ -4792,7 +4792,7 @@ rcu_boot_init_percpu_data(int cpu)
 	/* Set up local state, ensuring consistent view of global state. */
 	rdp->grpmask = leaf_node_cpu_bit(rdp->mynode, cpu);
 	INIT_WORK(&rdp->strict_work, strict_work_handler);
-	WARN_ON_ONCE(ct->dynticks_nesting != 1);
+	WARN_ON_ONCE(ct->nesting != 1);
 	WARN_ON_ONCE(rcu_dynticks_in_eqs(ct_rcu_watching_cpu(cpu)));
 	rdp->barrier_seq_snap = rcu_state.barrier_sequence;
 	rdp->rcu_ofl_gp_seq = rcu_state.gp_seq;
@@ -4886,7 +4886,7 @@ int rcutree_prepare_cpu(unsigned int cpu)
 	rdp->qlen_last_fqs_check = 0;
 	rdp->n_force_qs_snap = READ_ONCE(rcu_state.n_force_qs);
 	rdp->blimit = blimit;
-	ct->dynticks_nesting = 1;	/* CPU not up, no tearing. */
+	ct->nesting = 1;	/* CPU not up, no tearing. */
 	raw_spin_unlock_rcu_node(rnp);		/* irqs remain disabled. */
 
 	/*
-- 
2.43.0

[PATCH v3 07/25] context_tracking, rcu: Rename ct_dynticks_nesting() into ct_nesting()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, reflect that change in the related helpers.

Suggested-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 include/linux/context_tracking_state.h |  2 +-
 kernel/context_tracking.c              | 10 +++++-----
 kernel/rcu/tree.c                      |  6 +++---
 3 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h
index 65290e7677e6c..586c1ff22c2e9 100644
--- a/include/linux/context_tracking_state.h
+++ b/include/linux/context_tracking_state.h
@@ -75,7 +75,7 @@ static __always_inline int ct_rcu_watching_cpu_acquire(int cpu)
 	return atomic_read_acquire(&ct->state) & CT_RCU_WATCHING_MASK;
 }
 
-static __always_inline long ct_dynticks_nesting(void)
+static __always_inline long ct_nesting(void)
 {
 	return __this_cpu_read(context_tracking.nesting);
 }
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index 5cfdfc03b4018..a951bde0bbcbb 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -128,8 +128,8 @@ static void noinstr ct_kernel_exit(bool user, int offset)
 	WARN_ON_ONCE(ct_dynticks_nmi_nesting() != DYNTICK_IRQ_NONIDLE);
 	WRITE_ONCE(ct->dynticks_nmi_nesting, 0);
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) &&
-		     ct_dynticks_nesting() == 0);
-	if (ct_dynticks_nesting() != 1) {
+		     ct_nesting() == 0);
+	if (ct_nesting() != 1) {
 		// RCU will still be watching, so just do accounting and leave.
 		ct->nesting--;
 		return;
@@ -137,7 +137,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
 
 	instrumentation_begin();
 	lockdep_assert_irqs_disabled();
-	trace_rcu_dyntick(TPS("Start"), ct_dynticks_nesting(), 0, ct_rcu_watching());
+	trace_rcu_dyntick(TPS("Start"), ct_nesting(), 0, ct_rcu_watching());
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current));
 	rcu_preempt_deferred_qs(current);
 
@@ -166,7 +166,7 @@ static void noinstr ct_kernel_enter(bool user, int offset)
 	long oldval;
 
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !raw_irqs_disabled());
-	oldval = ct_dynticks_nesting();
+	oldval = ct_nesting();
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && oldval < 0);
 	if (oldval) {
 		// RCU was already watching, so just do accounting and leave.
@@ -182,7 +182,7 @@ static void noinstr ct_kernel_enter(bool user, int offset)
 	// instrumentation for the noinstr ct_kernel_enter_state()
 	instrument_atomic_write(&ct->state, sizeof(ct->state));
 
-	trace_rcu_dyntick(TPS("End"), ct_dynticks_nesting(), 1, ct_rcu_watching());
+	trace_rcu_dyntick(TPS("End"), ct_nesting(), 1, ct_rcu_watching());
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current));
 	WRITE_ONCE(ct->nesting, 1);
 	WARN_ON_ONCE(ct_dynticks_nmi_nesting());
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 16007911c7890..35042ffeca73b 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -380,7 +380,7 @@ static int rcu_is_cpu_rrupt_from_idle(void)
 	lockdep_assert_irqs_disabled();
 
 	/* Check for counter underflows */
-	RCU_LOCKDEP_WARN(ct_dynticks_nesting() < 0,
+	RCU_LOCKDEP_WARN(ct_nesting() < 0,
 			 "RCU nesting counter underflow!");
 	RCU_LOCKDEP_WARN(ct_dynticks_nmi_nesting() <= 0,
 			 "RCU dynticks_nmi_nesting counter underflow/zero!");
@@ -396,7 +396,7 @@ static int rcu_is_cpu_rrupt_from_idle(void)
 	WARN_ON_ONCE(!nesting && !is_idle_task(current));
 
 	/* Does CPU appear to be idle from an RCU standpoint? */
-	return ct_dynticks_nesting() == 0;
+	return ct_nesting() == 0;
 }
 
 #define DEFAULT_RCU_BLIMIT (IS_ENABLED(CONFIG_RCU_STRICT_GRACE_PERIOD) ? 1000 : 10)
@@ -588,7 +588,7 @@ void rcu_irq_exit_check_preempt(void)
 {
 	lockdep_assert_irqs_disabled();
 
-	RCU_LOCKDEP_WARN(ct_dynticks_nesting() <= 0,
+	RCU_LOCKDEP_WARN(ct_nesting() <= 0,
 			 "RCU nesting counter underflow/zero!");
 	RCU_LOCKDEP_WARN(ct_dynticks_nmi_nesting() !=
 			 DYNTICK_IRQ_NONIDLE,
-- 
2.43.0

[PATCH v3 08/25] context_tracking, rcu: Rename ct_dynticks_nesting_cpu() into ct_nesting_cpu()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, and the 'dynticks' prefix can be dropped without losing any
meaning.

Suggested-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 include/linux/context_tracking_state.h | 2 +-
 kernel/rcu/tree_stall.h                | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h
index 586c1ff22c2e9..fd42d8120ac29 100644
--- a/include/linux/context_tracking_state.h
+++ b/include/linux/context_tracking_state.h
@@ -80,7 +80,7 @@ static __always_inline long ct_nesting(void)
 	return __this_cpu_read(context_tracking.nesting);
 }
 
-static __always_inline long ct_dynticks_nesting_cpu(int cpu)
+static __always_inline long ct_nesting_cpu(int cpu)
 {
 	struct context_tracking *ct = per_cpu_ptr(&context_tracking, cpu);
 
diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h
index d65974448e813..59b1d84a47493 100644
--- a/kernel/rcu/tree_stall.h
+++ b/kernel/rcu/tree_stall.h
@@ -516,7 +516,7 @@ static void print_cpu_stall_info(int cpu)
 				"!."[!delta],
 	       ticks_value, ticks_title,
 	       ct_rcu_watching_cpu(cpu) & 0xffff,
-	       ct_dynticks_nesting_cpu(cpu), ct_dynticks_nmi_nesting_cpu(cpu),
+	       ct_nesting_cpu(cpu), ct_dynticks_nmi_nesting_cpu(cpu),
 	       rdp->softirq_snap, kstat_softirqs_cpu(RCU_SOFTIRQ, cpu),
 	       data_race(rcu_state.n_force_qs) - rcu_state.n_force_qs_gpstart,
 	       rcuc_starved ? buf : "",
-- 
2.43.0

[PATCH v3 09/25] context_tracking, rcu: Rename struct context_tracking .dynticks_nmi_nesting into .nmi_nesting

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, and the 'dynticks' prefix can be dropped without losing any
meaning.

Suggested-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 .../Data-Structures/Data-Structures.rst       | 14 ++++-----
 include/linux/context_tracking_state.h        |  6 ++--
 kernel/context_tracking.c                     | 30 +++++++++----------
 kernel/rcu/tree.c                             |  4 +--
 4 files changed, 27 insertions(+), 27 deletions(-)

diff --git a/Documentation/RCU/Design/Data-Structures/Data-Structures.rst b/Documentation/RCU/Design/Data-Structures/Data-Structures.rst
index 57ffc33d3cce2..5389cc49bea13 100644
--- a/Documentation/RCU/Design/Data-Structures/Data-Structures.rst
+++ b/Documentation/RCU/Design/Data-Structures/Data-Structures.rst
@@ -936,7 +936,7 @@ This portion of the rcu_data structure is declared as follows:
 ::
 
      1   long nesting;
-     2   long dynticks_nmi_nesting;
+     2   long nmi_nesting;
      3   atomic_t dynticks;
      4   bool rcu_need_heavy_qs;
      5   bool rcu_urgent_qs;
@@ -948,11 +948,11 @@ the corresponding CPU (and from tracing) unless otherwise stated.
 The ``->nesting`` field counts the nesting depth of process
 execution, so that in normal circumstances this counter has value zero
 or one. NMIs, irqs, and tracers are counted by the
-``->dynticks_nmi_nesting`` field. Because NMIs cannot be masked, changes
+``->nmi_nesting`` field. Because NMIs cannot be masked, changes
 to this variable have to be undertaken carefully using an algorithm
 provided by Andy Lutomirski. The initial transition from idle adds one,
 and nested transitions add two, so that a nesting level of five is
-represented by a ``->dynticks_nmi_nesting`` value of nine. This counter
+represented by a ``->nmi_nesting`` value of nine. This counter
 can therefore be thought of as counting the number of reasons why this
 CPU cannot be permitted to enter dyntick-idle mode, aside from
 process-level transitions.
@@ -961,11 +961,11 @@ However, it turns out that when running in non-idle kernel context, the
 Linux kernel is fully capable of entering interrupt handlers that never
 exit and perhaps also vice versa. Therefore, whenever the
 ``->nesting`` field is incremented up from zero, the
-``->dynticks_nmi_nesting`` field is set to a large positive number, and
+``->nmi_nesting`` field is set to a large positive number, and
 whenever the ``->nesting`` field is decremented down to zero,
-the ``->dynticks_nmi_nesting`` field is set to zero. Assuming that
+the ``->nmi_nesting`` field is set to zero. Assuming that
 the number of misnested interrupts is not sufficient to overflow the
-counter, this approach corrects the ``->dynticks_nmi_nesting`` field
+counter, this approach corrects the ``->nmi_nesting`` field
 every time the corresponding CPU enters the idle loop from process
 context.
 
@@ -993,7 +993,7 @@ code.
 | **Quick Quiz**:                                                       |
 +-----------------------------------------------------------------------+
 | Why not simply combine the ``->nesting`` and                 |
-| ``->dynticks_nmi_nesting`` counters into a single counter that just   |
+| ``->nmi_nesting`` counters into a single counter that just   |
 | counts the number of reasons that the corresponding CPU is non-idle?  |
 +-----------------------------------------------------------------------+
 | **Answer**:                                                           |
diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h
index fd42d8120ac29..12d00adf29e1e 100644
--- a/include/linux/context_tracking_state.h
+++ b/include/linux/context_tracking_state.h
@@ -40,7 +40,7 @@ struct context_tracking {
 #endif
 #ifdef CONFIG_CONTEXT_TRACKING_IDLE
 	long nesting;		/* Track process nesting level. */
-	long dynticks_nmi_nesting;	/* Track irq/NMI nesting level. */
+	long nmi_nesting;	/* Track irq/NMI nesting level. */
 #endif
 };
 
@@ -89,14 +89,14 @@ static __always_inline long ct_nesting_cpu(int cpu)
 
 static __always_inline long ct_dynticks_nmi_nesting(void)
 {
-	return __this_cpu_read(context_tracking.dynticks_nmi_nesting);
+	return __this_cpu_read(context_tracking.nmi_nesting);
 }
 
 static __always_inline long ct_dynticks_nmi_nesting_cpu(int cpu)
 {
 	struct context_tracking *ct = per_cpu_ptr(&context_tracking, cpu);
 
-	return ct->dynticks_nmi_nesting;
+	return ct->nmi_nesting;
 }
 #endif /* #ifdef CONFIG_CONTEXT_TRACKING_IDLE */
 
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index a951bde0bbcbb..ae94215aa132a 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -29,7 +29,7 @@
 DEFINE_PER_CPU(struct context_tracking, context_tracking) = {
 #ifdef CONFIG_CONTEXT_TRACKING_IDLE
 	.nesting = 1,
-	.dynticks_nmi_nesting = DYNTICK_IRQ_NONIDLE,
+	.nmi_nesting = DYNTICK_IRQ_NONIDLE,
 #endif
 	.state = ATOMIC_INIT(CT_RCU_WATCHING),
 };
@@ -117,7 +117,7 @@ static noinstr void ct_kernel_enter_state(int offset)
  * Enter an RCU extended quiescent state, which can be either the
  * idle loop or adaptive-tickless usermode execution.
  *
- * We crowbar the ->dynticks_nmi_nesting field to zero to allow for
+ * We crowbar the ->nmi_nesting field to zero to allow for
  * the possibility of usermode upcalls having messed up our count
  * of interrupt nesting level during the prior busy period.
  */
@@ -126,7 +126,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
 	struct context_tracking *ct = this_cpu_ptr(&context_tracking);
 
 	WARN_ON_ONCE(ct_dynticks_nmi_nesting() != DYNTICK_IRQ_NONIDLE);
-	WRITE_ONCE(ct->dynticks_nmi_nesting, 0);
+	WRITE_ONCE(ct->nmi_nesting, 0);
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) &&
 		     ct_nesting() == 0);
 	if (ct_nesting() != 1) {
@@ -156,7 +156,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
  * Exit an RCU extended quiescent state, which can be either the
  * idle loop or adaptive-tickless usermode execution.
  *
- * We crowbar the ->dynticks_nmi_nesting field to DYNTICK_IRQ_NONIDLE to
+ * We crowbar the ->nmi_nesting field to DYNTICK_IRQ_NONIDLE to
  * allow for the possibility of usermode upcalls messing up our count of
  * interrupt nesting level during the busy period that is just now starting.
  */
@@ -186,7 +186,7 @@ static void noinstr ct_kernel_enter(bool user, int offset)
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current));
 	WRITE_ONCE(ct->nesting, 1);
 	WARN_ON_ONCE(ct_dynticks_nmi_nesting());
-	WRITE_ONCE(ct->dynticks_nmi_nesting, DYNTICK_IRQ_NONIDLE);
+	WRITE_ONCE(ct->nmi_nesting, DYNTICK_IRQ_NONIDLE);
 	instrumentation_end();
 }
 
@@ -194,7 +194,7 @@ static void noinstr ct_kernel_enter(bool user, int offset)
  * ct_nmi_exit - inform RCU of exit from NMI context
  *
  * If we are returning from the outermost NMI handler that interrupted an
- * RCU-idle period, update ct->state and ct->dynticks_nmi_nesting
+ * RCU-idle period, update ct->state and ct->nmi_nesting
  * to let the RCU grace-period handling know that the CPU is back to
  * being RCU-idle.
  *
@@ -207,7 +207,7 @@ void noinstr ct_nmi_exit(void)
 
 	instrumentation_begin();
 	/*
-	 * Check for ->dynticks_nmi_nesting underflow and bad ->dynticks.
+	 * Check for ->nmi_nesting underflow and bad ->dynticks.
 	 * (We are exiting an NMI handler, so RCU better be paying attention
 	 * to us!)
 	 */
@@ -221,7 +221,7 @@ void noinstr ct_nmi_exit(void)
 	if (ct_dynticks_nmi_nesting() != 1) {
 		trace_rcu_dyntick(TPS("--="), ct_dynticks_nmi_nesting(), ct_dynticks_nmi_nesting() - 2,
 				  ct_rcu_watching());
-		WRITE_ONCE(ct->dynticks_nmi_nesting, /* No store tearing. */
+		WRITE_ONCE(ct->nmi_nesting, /* No store tearing. */
 			   ct_dynticks_nmi_nesting() - 2);
 		instrumentation_end();
 		return;
@@ -229,7 +229,7 @@ void noinstr ct_nmi_exit(void)
 
 	/* This NMI interrupted an RCU-idle CPU, restore RCU-idleness. */
 	trace_rcu_dyntick(TPS("Startirq"), ct_dynticks_nmi_nesting(), 0, ct_rcu_watching());
-	WRITE_ONCE(ct->dynticks_nmi_nesting, 0); /* Avoid store tearing. */
+	WRITE_ONCE(ct->nmi_nesting, 0); /* Avoid store tearing. */
 
 	// instrumentation for the noinstr ct_kernel_exit_state()
 	instrument_atomic_write(&ct->state, sizeof(ct->state));
@@ -247,7 +247,7 @@ void noinstr ct_nmi_exit(void)
  * ct_nmi_enter - inform RCU of entry to NMI context
  *
  * If the CPU was idle from RCU's viewpoint, update ct->state and
- * ct->dynticks_nmi_nesting to let the RCU grace-period handling know
+ * ct->nmi_nesting to let the RCU grace-period handling know
  * that the CPU is active.  This implementation permits nested NMIs, as
  * long as the nesting level does not overflow an int.  (You will probably
  * run out of stack space first.)
@@ -264,10 +264,10 @@ void noinstr ct_nmi_enter(void)
 	WARN_ON_ONCE(ct_dynticks_nmi_nesting() < 0);
 
 	/*
-	 * If idle from RCU viewpoint, atomically increment ->dynticks
-	 * to mark non-idle and increment ->dynticks_nmi_nesting by one.
-	 * Otherwise, increment ->dynticks_nmi_nesting by two.  This means
-	 * if ->dynticks_nmi_nesting is equal to one, we are guaranteed
+	 * If idle from RCU viewpoint, atomically increment CT state
+	 * to mark non-idle and increment ->nmi_nesting by one.
+	 * Otherwise, increment ->nmi_nesting by two.  This means
+	 * if ->nmi_nesting is equal to one, we are guaranteed
 	 * to be in the outermost NMI handler that interrupted an RCU-idle
 	 * period (observation due to Andy Lutomirski).
 	 */
@@ -298,7 +298,7 @@ void noinstr ct_nmi_enter(void)
 			  ct_dynticks_nmi_nesting(),
 			  ct_dynticks_nmi_nesting() + incby, ct_rcu_watching());
 	instrumentation_end();
-	WRITE_ONCE(ct->dynticks_nmi_nesting, /* Prevent store tearing. */
+	WRITE_ONCE(ct->nmi_nesting, /* Prevent store tearing. */
 		   ct_dynticks_nmi_nesting() + incby);
 	barrier();
 }
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 35042ffeca73b..4ccb4bb8df1d3 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -383,7 +383,7 @@ static int rcu_is_cpu_rrupt_from_idle(void)
 	RCU_LOCKDEP_WARN(ct_nesting() < 0,
 			 "RCU nesting counter underflow!");
 	RCU_LOCKDEP_WARN(ct_dynticks_nmi_nesting() <= 0,
-			 "RCU dynticks_nmi_nesting counter underflow/zero!");
+			 "RCU nmi_nesting counter underflow/zero!");
 
 	/* Are we at first interrupt nesting level? */
 	nesting = ct_dynticks_nmi_nesting();
@@ -592,7 +592,7 @@ void rcu_irq_exit_check_preempt(void)
 			 "RCU nesting counter underflow/zero!");
 	RCU_LOCKDEP_WARN(ct_dynticks_nmi_nesting() !=
 			 DYNTICK_IRQ_NONIDLE,
-			 "Bad RCU  dynticks_nmi_nesting counter\n");
+			 "Bad RCU  nmi_nesting counter\n");
 	RCU_LOCKDEP_WARN(rcu_dynticks_curr_cpu_in_eqs(),
 			 "RCU in extended quiescent state!");
 }
-- 
2.43.0

[PATCH v3 10/25] context_tracking, rcu: Rename ct_dynticks_nmi_nesting() into ct_nmi_nesting()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, and the 'dynticks' prefix can be dropped without losing any
meaning.

Suggested-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 include/linux/context_tracking_state.h |  2 +-
 kernel/context_tracking.c              | 24 ++++++++++++------------
 kernel/rcu/tree.c                      |  6 +++---
 3 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h
index 12d00adf29e1e..8f32fe599c5c0 100644
--- a/include/linux/context_tracking_state.h
+++ b/include/linux/context_tracking_state.h
@@ -87,7 +87,7 @@ static __always_inline long ct_nesting_cpu(int cpu)
 	return ct->nesting;
 }
 
-static __always_inline long ct_dynticks_nmi_nesting(void)
+static __always_inline long ct_nmi_nesting(void)
 {
 	return __this_cpu_read(context_tracking.nmi_nesting);
 }
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index ae94215aa132a..115843eeb0309 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -125,7 +125,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
 {
 	struct context_tracking *ct = this_cpu_ptr(&context_tracking);
 
-	WARN_ON_ONCE(ct_dynticks_nmi_nesting() != DYNTICK_IRQ_NONIDLE);
+	WARN_ON_ONCE(ct_nmi_nesting() != DYNTICK_IRQ_NONIDLE);
 	WRITE_ONCE(ct->nmi_nesting, 0);
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) &&
 		     ct_nesting() == 0);
@@ -185,7 +185,7 @@ static void noinstr ct_kernel_enter(bool user, int offset)
 	trace_rcu_dyntick(TPS("End"), ct_nesting(), 1, ct_rcu_watching());
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current));
 	WRITE_ONCE(ct->nesting, 1);
-	WARN_ON_ONCE(ct_dynticks_nmi_nesting());
+	WARN_ON_ONCE(ct_nmi_nesting());
 	WRITE_ONCE(ct->nmi_nesting, DYNTICK_IRQ_NONIDLE);
 	instrumentation_end();
 }
@@ -207,28 +207,28 @@ void noinstr ct_nmi_exit(void)
 
 	instrumentation_begin();
 	/*
-	 * Check for ->nmi_nesting underflow and bad ->dynticks.
+	 * Check for ->nmi_nesting underflow and bad CT state.
 	 * (We are exiting an NMI handler, so RCU better be paying attention
 	 * to us!)
 	 */
-	WARN_ON_ONCE(ct_dynticks_nmi_nesting() <= 0);
+	WARN_ON_ONCE(ct_nmi_nesting() <= 0);
 	WARN_ON_ONCE(rcu_dynticks_curr_cpu_in_eqs());
 
 	/*
 	 * If the nesting level is not 1, the CPU wasn't RCU-idle, so
 	 * leave it in non-RCU-idle state.
 	 */
-	if (ct_dynticks_nmi_nesting() != 1) {
-		trace_rcu_dyntick(TPS("--="), ct_dynticks_nmi_nesting(), ct_dynticks_nmi_nesting() - 2,
+	if (ct_nmi_nesting() != 1) {
+		trace_rcu_dyntick(TPS("--="), ct_nmi_nesting(), ct_nmi_nesting() - 2,
 				  ct_rcu_watching());
 		WRITE_ONCE(ct->nmi_nesting, /* No store tearing. */
-			   ct_dynticks_nmi_nesting() - 2);
+			   ct_nmi_nesting() - 2);
 		instrumentation_end();
 		return;
 	}
 
 	/* This NMI interrupted an RCU-idle CPU, restore RCU-idleness. */
-	trace_rcu_dyntick(TPS("Startirq"), ct_dynticks_nmi_nesting(), 0, ct_rcu_watching());
+	trace_rcu_dyntick(TPS("Startirq"), ct_nmi_nesting(), 0, ct_rcu_watching());
 	WRITE_ONCE(ct->nmi_nesting, 0); /* Avoid store tearing. */
 
 	// instrumentation for the noinstr ct_kernel_exit_state()
@@ -261,7 +261,7 @@ void noinstr ct_nmi_enter(void)
 	struct context_tracking *ct = this_cpu_ptr(&context_tracking);
 
 	/* Complain about underflow. */
-	WARN_ON_ONCE(ct_dynticks_nmi_nesting() < 0);
+	WARN_ON_ONCE(ct_nmi_nesting() < 0);
 
 	/*
 	 * If idle from RCU viewpoint, atomically increment CT state
@@ -295,11 +295,11 @@ void noinstr ct_nmi_enter(void)
 	}
 
 	trace_rcu_dyntick(incby == 1 ? TPS("Endirq") : TPS("++="),
-			  ct_dynticks_nmi_nesting(),
-			  ct_dynticks_nmi_nesting() + incby, ct_rcu_watching());
+			  ct_nmi_nesting(),
+			  ct_nmi_nesting() + incby, ct_rcu_watching());
 	instrumentation_end();
 	WRITE_ONCE(ct->nmi_nesting, /* Prevent store tearing. */
-		   ct_dynticks_nmi_nesting() + incby);
+		   ct_nmi_nesting() + incby);
 	barrier();
 }
 
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 4ccb4bb8df1d3..22322387cefb7 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -382,11 +382,11 @@ static int rcu_is_cpu_rrupt_from_idle(void)
 	/* Check for counter underflows */
 	RCU_LOCKDEP_WARN(ct_nesting() < 0,
 			 "RCU nesting counter underflow!");
-	RCU_LOCKDEP_WARN(ct_dynticks_nmi_nesting() <= 0,
+	RCU_LOCKDEP_WARN(ct_nmi_nesting() <= 0,
 			 "RCU nmi_nesting counter underflow/zero!");
 
 	/* Are we at first interrupt nesting level? */
-	nesting = ct_dynticks_nmi_nesting();
+	nesting = ct_nmi_nesting();
 	if (nesting > 1)
 		return false;
 
@@ -590,7 +590,7 @@ void rcu_irq_exit_check_preempt(void)
 
 	RCU_LOCKDEP_WARN(ct_nesting() <= 0,
 			 "RCU nesting counter underflow/zero!");
-	RCU_LOCKDEP_WARN(ct_dynticks_nmi_nesting() !=
+	RCU_LOCKDEP_WARN(ct_nmi_nesting() !=
 			 DYNTICK_IRQ_NONIDLE,
 			 "Bad RCU  nmi_nesting counter\n");
 	RCU_LOCKDEP_WARN(rcu_dynticks_curr_cpu_in_eqs(),
-- 
2.43.0

[PATCH v3 11/25] context_tracking, rcu: Rename ct_dynticks_nmi_nesting_cpu() into ct_nmi_nesting_cpu()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, and the 'dynticks' prefix can be dropped without losing any
meaning.

Suggested-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 include/linux/context_tracking_state.h | 2 +-
 kernel/rcu/tree_stall.h                | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h
index 8f32fe599c5c0..34fd504e53a86 100644
--- a/include/linux/context_tracking_state.h
+++ b/include/linux/context_tracking_state.h
@@ -92,7 +92,7 @@ static __always_inline long ct_nmi_nesting(void)
 	return __this_cpu_read(context_tracking.nmi_nesting);
 }
 
-static __always_inline long ct_dynticks_nmi_nesting_cpu(int cpu)
+static __always_inline long ct_nmi_nesting_cpu(int cpu)
 {
 	struct context_tracking *ct = per_cpu_ptr(&context_tracking, cpu);
 
diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h
index 59b1d84a47493..ec49f0155becc 100644
--- a/kernel/rcu/tree_stall.h
+++ b/kernel/rcu/tree_stall.h
@@ -516,7 +516,7 @@ static void print_cpu_stall_info(int cpu)
 				"!."[!delta],
 	       ticks_value, ticks_title,
 	       ct_rcu_watching_cpu(cpu) & 0xffff,
-	       ct_nesting_cpu(cpu), ct_dynticks_nmi_nesting_cpu(cpu),
+	       ct_nesting_cpu(cpu), ct_nmi_nesting_cpu(cpu),
 	       rdp->softirq_snap, kstat_softirqs_cpu(RCU_SOFTIRQ, cpu),
 	       data_race(rcu_state.n_force_qs) - rcu_state.n_force_qs_gpstart,
 	       rcuc_starved ? buf : "",
-- 
2.43.0

[PATCH v3 12/25] context_tracking, rcu: Rename DYNTICK_IRQ_NONIDLE into CT_NESTING_IRQ_NONIDLE

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, and the 'dynticks' prefix can be dropped without losing any
meaning.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 include/linux/context_tracking_state.h | 2 +-
 kernel/context_tracking.c              | 8 ++++----
 kernel/rcu/tree.c                      | 2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h
index 34fd504e53a86..0dbda59c9f372 100644
--- a/include/linux/context_tracking_state.h
+++ b/include/linux/context_tracking_state.h
@@ -7,7 +7,7 @@
 #include <linux/context_tracking_irq.h>
 
 /* Offset to allow distinguishing irq vs. task-based idle entry/exit. */
-#define DYNTICK_IRQ_NONIDLE	((LONG_MAX / 2) + 1)
+#define CT_NESTING_IRQ_NONIDLE	((LONG_MAX / 2) + 1)
 
 enum ctx_state {
 	CT_STATE_DISABLED	= -1,	/* returned by ct_state() if unknown */
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index 115843eeb0309..8262f57a43636 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -29,7 +29,7 @@
 DEFINE_PER_CPU(struct context_tracking, context_tracking) = {
 #ifdef CONFIG_CONTEXT_TRACKING_IDLE
 	.nesting = 1,
-	.nmi_nesting = DYNTICK_IRQ_NONIDLE,
+	.nmi_nesting = CT_NESTING_IRQ_NONIDLE,
 #endif
 	.state = ATOMIC_INIT(CT_RCU_WATCHING),
 };
@@ -125,7 +125,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
 {
 	struct context_tracking *ct = this_cpu_ptr(&context_tracking);
 
-	WARN_ON_ONCE(ct_nmi_nesting() != DYNTICK_IRQ_NONIDLE);
+	WARN_ON_ONCE(ct_nmi_nesting() != CT_NESTING_IRQ_NONIDLE);
 	WRITE_ONCE(ct->nmi_nesting, 0);
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) &&
 		     ct_nesting() == 0);
@@ -156,7 +156,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
  * Exit an RCU extended quiescent state, which can be either the
  * idle loop or adaptive-tickless usermode execution.
  *
- * We crowbar the ->nmi_nesting field to DYNTICK_IRQ_NONIDLE to
+ * We crowbar the ->nmi_nesting field to CT_NESTING_IRQ_NONIDLE to
  * allow for the possibility of usermode upcalls messing up our count of
  * interrupt nesting level during the busy period that is just now starting.
  */
@@ -186,7 +186,7 @@ static void noinstr ct_kernel_enter(bool user, int offset)
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current));
 	WRITE_ONCE(ct->nesting, 1);
 	WARN_ON_ONCE(ct_nmi_nesting());
-	WRITE_ONCE(ct->nmi_nesting, DYNTICK_IRQ_NONIDLE);
+	WRITE_ONCE(ct->nmi_nesting, CT_NESTING_IRQ_NONIDLE);
 	instrumentation_end();
 }
 
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 22322387cefb7..8a88d278e198e 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -591,7 +591,7 @@ void rcu_irq_exit_check_preempt(void)
 	RCU_LOCKDEP_WARN(ct_nesting() <= 0,
 			 "RCU nesting counter underflow/zero!");
 	RCU_LOCKDEP_WARN(ct_nmi_nesting() !=
-			 DYNTICK_IRQ_NONIDLE,
+			 CT_NESTING_IRQ_NONIDLE,
 			 "Bad RCU  nmi_nesting counter\n");
 	RCU_LOCKDEP_WARN(rcu_dynticks_curr_cpu_in_eqs(),
 			 "RCU in extended quiescent state!");
-- 
2.43.0

[PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, and the 'dynticks' prefix can be dropped without losing any
meaning.

While at it, flip the suffixes of these helpers. We are not telling
that we are entering dynticks mode from an RCU-task perspective anymore; we
are telling that we are exiting RCU-tasks because we are in eqs mode.

Suggested-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
---
 kernel/context_tracking.c | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index 8262f57a43636..1c16a7336360f 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -38,24 +38,24 @@ EXPORT_SYMBOL_GPL(context_tracking);
 #ifdef CONFIG_CONTEXT_TRACKING_IDLE
 #define TPS(x)  tracepoint_string(x)
 
-/* Record the current task on dyntick-idle entry. */
-static __always_inline void rcu_dynticks_task_enter(void)
+/* Record the current task on exiting RCU-tasks (dyntick-idle entry). */
+static __always_inline void rcu_task_exit(void)
 {
 #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL)
 	WRITE_ONCE(current->rcu_tasks_idle_cpu, smp_processor_id());
 #endif /* #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL) */
 }
 
-/* Record no current task on dyntick-idle exit. */
-static __always_inline void rcu_dynticks_task_exit(void)
+/* Record no current task on entering RCU-tasks (dyntick-idle exit). */
+static __always_inline void rcu_task_enter(void)
 {
 #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL)
 	WRITE_ONCE(current->rcu_tasks_idle_cpu, -1);
 #endif /* #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL) */
 }
 
-/* Turn on heavyweight RCU tasks trace readers on idle/user entry. */
-static __always_inline void rcu_dynticks_task_trace_enter(void)
+/* Turn on heavyweight RCU tasks trace readers on kernel exit. */
+static __always_inline void rcu_task_trace_exit(void)
 {
 #ifdef CONFIG_TASKS_TRACE_RCU
 	if (IS_ENABLED(CONFIG_TASKS_TRACE_RCU_READ_MB))
@@ -63,8 +63,8 @@ static __always_inline void rcu_dynticks_task_trace_enter(void)
 #endif /* #ifdef CONFIG_TASKS_TRACE_RCU */
 }
 
-/* Turn off heavyweight RCU tasks trace readers on idle/user exit. */
-static __always_inline void rcu_dynticks_task_trace_exit(void)
+/* Turn off heavyweight RCU tasks trace readers on kernel entry. */
+static __always_inline void rcu_task_trace_enter(void)
 {
 #ifdef CONFIG_TASKS_TRACE_RCU
 	if (IS_ENABLED(CONFIG_TASKS_TRACE_RCU_READ_MB))
@@ -87,7 +87,7 @@ static noinstr void ct_kernel_exit_state(int offset)
 	 * critical sections, and we also must force ordering with the
 	 * next idle sojourn.
 	 */
-	rcu_dynticks_task_trace_enter();  // Before ->dynticks update!
+	rcu_task_trace_exit();  // Before CT state update!
 	seq = ct_state_inc(offset);
 	// RCU is no longer watching.  Better be in extended quiescent state!
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && (seq & CT_RCU_WATCHING));
@@ -109,7 +109,7 @@ static noinstr void ct_kernel_enter_state(int offset)
 	 */
 	seq = ct_state_inc(offset);
 	// RCU is now watching.  Better not be in an extended quiescent state!
-	rcu_dynticks_task_trace_exit();  // After ->dynticks update!
+	rcu_task_trace_enter();  // After CT state update!
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !(seq & CT_RCU_WATCHING));
 }
 
@@ -149,7 +149,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
 	// RCU is watching here ...
 	ct_kernel_exit_state(offset);
 	// ... but is no longer watching here.
-	rcu_dynticks_task_enter();
+	rcu_task_exit();
 }
 
 /*
@@ -173,7 +173,7 @@ static void noinstr ct_kernel_enter(bool user, int offset)
 		ct->nesting++;
 		return;
 	}
-	rcu_dynticks_task_exit();
+	rcu_task_enter();
 	// RCU is not watching here ...
 	ct_kernel_enter_state(offset);
 	// ... but is watching here.
@@ -240,7 +240,7 @@ void noinstr ct_nmi_exit(void)
 	// ... but is no longer watching here.
 
 	if (!in_nmi())
-		rcu_dynticks_task_enter();
+		rcu_task_exit();
 }
 
 /**
@@ -274,7 +274,7 @@ void noinstr ct_nmi_enter(void)
 	if (rcu_dynticks_curr_cpu_in_eqs()) {
 
 		if (!in_nmi())
-			rcu_dynticks_task_exit();
+			rcu_task_enter();
 
 		// RCU is not watching here ...
 		ct_kernel_enter_state(CT_RCU_WATCHING);
-- 
2.43.0

[PATCH v3 14/25] context_tracking, rcu: Rename rcu_dynticks_curr_cpu_in_eqs() into rcu_watching_curr_cpu()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, reflect that change in the related helpers.

Note that "watching" is the opposite of "in EQS", so the negation is lifted
out of the helper and into the callsites.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 include/linux/context_tracking.h | 12 ++++++++----
 kernel/context_tracking.c        |  6 +++---
 kernel/rcu/tree.c                |  6 +++---
 3 files changed, 14 insertions(+), 10 deletions(-)

diff --git a/include/linux/context_tracking.h b/include/linux/context_tracking.h
index a6c36780cc3bd..d53092ffa9dba 100644
--- a/include/linux/context_tracking.h
+++ b/include/linux/context_tracking.h
@@ -113,13 +113,17 @@ extern void ct_idle_enter(void);
 extern void ct_idle_exit(void);
 
 /*
- * Is the current CPU in an extended quiescent state?
+ * Is RCU watching the current CPU (IOW, it is not in an extended quiescent state)?
+ *
+ * Note that this returns the actual boolean data (watching / not watching),
+ * whereas ct_rcu_watching() returns the RCU_WATCHING subvariable of
+ * context_tracking.state.
  *
  * No ordering, as we are sampling CPU-local information.
  */
-static __always_inline bool rcu_dynticks_curr_cpu_in_eqs(void)
+static __always_inline bool rcu_is_watching_curr_cpu(void)
 {
-	return !(raw_atomic_read(this_cpu_ptr(&context_tracking.state)) & CT_RCU_WATCHING);
+	return raw_atomic_read(this_cpu_ptr(&context_tracking.state)) & CT_RCU_WATCHING;
 }
 
 /*
@@ -140,7 +144,7 @@ static __always_inline bool warn_rcu_enter(void)
 	 * lots of the actual reporting also relies on RCU.
 	 */
 	preempt_disable_notrace();
-	if (rcu_dynticks_curr_cpu_in_eqs()) {
+	if (!rcu_is_watching_curr_cpu()) {
 		ret = true;
 		ct_state_inc(CT_RCU_WATCHING);
 	}
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index 1c16a7336360f..e64bb746cd9b3 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -212,7 +212,7 @@ void noinstr ct_nmi_exit(void)
 	 * to us!)
 	 */
 	WARN_ON_ONCE(ct_nmi_nesting() <= 0);
-	WARN_ON_ONCE(rcu_dynticks_curr_cpu_in_eqs());
+	WARN_ON_ONCE(!rcu_is_watching_curr_cpu());
 
 	/*
 	 * If the nesting level is not 1, the CPU wasn't RCU-idle, so
@@ -271,7 +271,7 @@ void noinstr ct_nmi_enter(void)
 	 * to be in the outermost NMI handler that interrupted an RCU-idle
 	 * period (observation due to Andy Lutomirski).
 	 */
-	if (rcu_dynticks_curr_cpu_in_eqs()) {
+	if (!rcu_is_watching_curr_cpu()) {
 
 		if (!in_nmi())
 			rcu_task_enter();
@@ -281,7 +281,7 @@ void noinstr ct_nmi_enter(void)
 		// ... but is watching here.
 
 		instrumentation_begin();
-		// instrumentation for the noinstr rcu_dynticks_curr_cpu_in_eqs()
+		// instrumentation for the noinstr rcu_is_watching_curr_cpu()
 		instrument_atomic_read(&ct->state, sizeof(ct->state));
 		// instrumentation for the noinstr ct_kernel_enter_state()
 		instrument_atomic_write(&ct->state, sizeof(ct->state));
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 8a88d278e198e..90a0c14f80542 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -593,7 +593,7 @@ void rcu_irq_exit_check_preempt(void)
 	RCU_LOCKDEP_WARN(ct_nmi_nesting() !=
 			 CT_NESTING_IRQ_NONIDLE,
 			 "Bad RCU  nmi_nesting counter\n");
-	RCU_LOCKDEP_WARN(rcu_dynticks_curr_cpu_in_eqs(),
+	RCU_LOCKDEP_WARN(!rcu_is_watching_curr_cpu(),
 			 "RCU in extended quiescent state!");
 }
 #endif /* #ifdef CONFIG_PROVE_RCU */
@@ -633,7 +633,7 @@ void __rcu_irq_enter_check_tick(void)
 	if (in_nmi())
 		return;
 
-	RCU_LOCKDEP_WARN(rcu_dynticks_curr_cpu_in_eqs(),
+	RCU_LOCKDEP_WARN(!rcu_is_watching_curr_cpu(),
 			 "Illegal rcu_irq_enter_check_tick() from extended quiescent state");
 
 	if (!tick_nohz_full_cpu(rdp->cpu) ||
@@ -715,7 +715,7 @@ notrace bool rcu_is_watching(void)
 	bool ret;
 
 	preempt_disable_notrace();
-	ret = !rcu_dynticks_curr_cpu_in_eqs();
+	ret = rcu_is_watching_curr_cpu();
 	preempt_enable_notrace();
 	return ret;
 }
-- 
2.43.0

[PATCH v3 15/25] rcu: Rename rcu_dynticks_eqs_online() into rcu_watching_online()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, reflect that change in the related helpers.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
---
 kernel/rcu/tree.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 90a0c14f80542..ffec0e7513466 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -283,16 +283,16 @@ void rcu_softirq_qs(void)
 }
 
 /*
- * Reset the current CPU's ->dynticks counter to indicate that the
+ * Reset the current CPU's RCU_WATCHING counter to indicate that the
  * newly onlined CPU is no longer in an extended quiescent state.
  * This will either leave the counter unchanged, or increment it
  * to the next non-quiescent value.
  *
  * The non-atomic test/increment sequence works because the upper bits
- * of the ->dynticks counter are manipulated only by the corresponding CPU,
+ * of the ->state variable are manipulated only by the corresponding CPU,
  * or when the corresponding CPU is offline.
  */
-static void rcu_dynticks_eqs_online(void)
+static void rcu_watching_online(void)
 {
 	if (ct_rcu_watching() & CT_RCU_WATCHING)
 		return;
@@ -5046,7 +5046,7 @@ void rcutree_report_cpu_starting(unsigned int cpu)
 	rnp = rdp->mynode;
 	mask = rdp->grpmask;
 	arch_spin_lock(&rcu_state.ofl_lock);
-	rcu_dynticks_eqs_online();
+	rcu_watching_online();
 	raw_spin_lock(&rcu_state.barrier_lock);
 	raw_spin_lock_rcu_node(rnp);
 	WRITE_ONCE(rnp->qsmaskinitnext, rnp->qsmaskinitnext | mask);
-- 
2.43.0

[PATCH v3 16/25] rcu: Rename rcu_dynticks_in_eqs() into rcu_watching_snap_in_eqs()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, reflect that change in the related helpers.

While at it, update a comment that still refers to rcu_dynticks_snap(),
which was removed by commit:

  7be2e6323b9b ("rcu: Remove full memory barrier on RCU stall printout")

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 kernel/rcu/tree.c       | 8 ++++----
 kernel/rcu/tree_exp.h   | 2 +-
 kernel/rcu/tree_stall.h | 2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index ffec0e7513466..7c2bd7e511579 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -300,10 +300,10 @@ static void rcu_watching_online(void)
 }
 
 /*
- * Return true if the snapshot returned from rcu_dynticks_snap()
+ * Return true if the snapshot returned from ct_rcu_watching()
  * indicates that RCU is in an extended quiescent state.
  */
-static bool rcu_dynticks_in_eqs(int snap)
+static bool rcu_watching_snap_in_eqs(int snap)
 {
 	return !(snap & CT_RCU_WATCHING);
 }
@@ -769,7 +769,7 @@ static int dyntick_save_progress_counter(struct rcu_data *rdp)
 	 * with chained smp_mb__after_unlock_lock().
 	 */
 	rdp->dynticks_snap = ct_rcu_watching_cpu_acquire(rdp->cpu);
-	if (rcu_dynticks_in_eqs(rdp->dynticks_snap)) {
+	if (rcu_watching_snap_in_eqs(rdp->dynticks_snap)) {
 		trace_rcu_fqs(rcu_state.name, rdp->gp_seq, rdp->cpu, TPS("dti"));
 		rcu_gpnum_ovf(rdp->mynode, rdp);
 		return 1;
@@ -4793,7 +4793,7 @@ rcu_boot_init_percpu_data(int cpu)
 	rdp->grpmask = leaf_node_cpu_bit(rdp->mynode, cpu);
 	INIT_WORK(&rdp->strict_work, strict_work_handler);
 	WARN_ON_ONCE(ct->nesting != 1);
-	WARN_ON_ONCE(rcu_dynticks_in_eqs(ct_rcu_watching_cpu(cpu)));
+	WARN_ON_ONCE(rcu_watching_snap_in_eqs(ct_rcu_watching_cpu(cpu)));
 	rdp->barrier_seq_snap = rcu_state.barrier_sequence;
 	rdp->rcu_ofl_gp_seq = rcu_state.gp_seq;
 	rdp->rcu_ofl_gp_state = RCU_GP_CLEANED;
diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h
index 89cbadb15a4e7..8453da73a7df2 100644
--- a/kernel/rcu/tree_exp.h
+++ b/kernel/rcu/tree_exp.h
@@ -369,7 +369,7 @@ static void __sync_rcu_exp_select_node_cpus(struct rcu_exp_work *rewp)
 			 * smp_mb__after_unlock_lock().
 			 */
 			snap = ct_rcu_watching_cpu_acquire(cpu);
-			if (rcu_dynticks_in_eqs(snap))
+			if (rcu_watching_snap_in_eqs(snap))
 				mask_ofl_test |= mask;
 			else
 				rdp->exp_dynticks_snap = snap;
diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h
index ec49f0155becc..e933c6a58d5fd 100644
--- a/kernel/rcu/tree_stall.h
+++ b/kernel/rcu/tree_stall.h
@@ -501,7 +501,7 @@ static void print_cpu_stall_info(int cpu)
 	}
 	delta = rcu_seq_ctr(rdp->mynode->gp_seq - rdp->rcu_iw_gp_seq);
 	falsepositive = rcu_is_gp_kthread_starving(NULL) &&
-			rcu_dynticks_in_eqs(ct_rcu_watching_cpu(cpu));
+			rcu_watching_snap_in_eqs(ct_rcu_watching_cpu(cpu));
 	rcuc_starved = rcu_is_rcuc_kthread_starving(rdp, &j);
 	if (rcuc_starved)
 		// Print signed value, as negative values indicate a probable bug.
-- 
2.43.0

[PATCH v3 17/25] rcu: Rename rcu_dynticks_in_eqs_since() into rcu_watching_snap_stopped_since()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, the dynticks prefix can go.

While at it, this helper is only meant to be called after failing an
earlier call to rcu_watching_snap_in_eqs(), document this in the comments
and add a WARN_ON_ONCE() for good measure.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
---
 .../Tree-RCU-Memory-Ordering.rst              |  2 +-
 kernel/rcu/tree.c                             | 23 ++++++++++++++-----
 kernel/rcu/tree_exp.h                         |  2 +-
 3 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst b/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
index 2d7036ad74761..7163d0def34e6 100644
--- a/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
+++ b/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
@@ -150,7 +150,7 @@ This case is handled by calls to the strongly ordered
 is invoked within ``rcu_dynticks_eqs_enter()`` at idle-entry
 time and within ``rcu_dynticks_eqs_exit()`` at idle-exit time.
 The grace-period kthread invokes first ``ct_rcu_watching_cpu_acquire()``
-(preceded by a full memory barrier) and ``rcu_dynticks_in_eqs_since()``
+(preceded by a full memory barrier) and ``rcu_watching_snap_stopped_since()``
 (both of which rely on acquire semantics) to detect idle CPUs.
 
 +-----------------------------------------------------------------------+
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 7c2bd7e511579..3fd9f0fc4d558 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -308,13 +308,24 @@ static bool rcu_watching_snap_in_eqs(int snap)
 	return !(snap & CT_RCU_WATCHING);
 }
 
-/*
- * Return true if the CPU corresponding to the specified rcu_data
- * structure has spent some time in an extended quiescent state since
- * rcu_dynticks_snap() returned the specified snapshot.
+/**
+ * rcu_watching_snap_stopped_since() - Has RCU stopped watching a given CPU
+ * since the specified @snap?
+ *
+ * @rdp: The rcu_data corresponding to the CPU for which to check EQS.
+ * @snap: rcu_watching snapshot taken when the CPU wasn't in an EQS.
+ *
+ * Returns true if the CPU corresponding to @rdp has spent some time in an
+ * extended quiescent state since @snap. Note that this doesn't check if it
+ * /still/ is in an EQS, just that it went through one since @snap.
+ *
+ * This is meant to be used in a loop waiting for a CPU to go through an EQS.
  */
-static bool rcu_dynticks_in_eqs_since(struct rcu_data *rdp, int snap)
+static bool rcu_watching_snap_stopped_since(struct rcu_data *rdp, int snap)
 {
+	if (WARN_ON_ONCE(rcu_watching_snap_in_eqs(snap)))
+		return true;
+
 	return snap != ct_rcu_watching_cpu_acquire(rdp->cpu);
 }
 
@@ -801,7 +812,7 @@ static int rcu_implicit_dynticks_qs(struct rcu_data *rdp)
 	 * read-side critical section that started before the beginning
 	 * of the current RCU grace period.
 	 */
-	if (rcu_dynticks_in_eqs_since(rdp, rdp->dynticks_snap)) {
+	if (rcu_watching_snap_stopped_since(rdp, rdp->dynticks_snap)) {
 		trace_rcu_fqs(rcu_state.name, rdp->gp_seq, rdp->cpu, TPS("dti"));
 		rcu_gpnum_ovf(rnp, rdp);
 		return 1;
diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h
index 8453da73a7df2..29f94bcf34033 100644
--- a/kernel/rcu/tree_exp.h
+++ b/kernel/rcu/tree_exp.h
@@ -392,7 +392,7 @@ static void __sync_rcu_exp_select_node_cpus(struct rcu_exp_work *rewp)
 		unsigned long mask = rdp->grpmask;
 
 retry_ipi:
-		if (rcu_dynticks_in_eqs_since(rdp, rdp->exp_dynticks_snap)) {
+		if (rcu_watching_snap_stopped_since(rdp, rdp->exp_dynticks_snap)) {
 			mask_ofl_test |= mask;
 			continue;
 		}
-- 
2.43.0

[PATCH v3 18/25] rcu: Rename rcu_dynticks_zero_in_eqs() into rcu_watching_zero_in_eqs()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, reflect that change in the related helpers.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 kernel/rcu/rcu.h   | 4 ++--
 kernel/rcu/tasks.h | 2 +-
 kernel/rcu/tree.c  | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h
index 38238e595a61a..5564402af4cbd 100644
--- a/kernel/rcu/rcu.h
+++ b/kernel/rcu/rcu.h
@@ -606,7 +606,7 @@ void srcutorture_get_gp_data(struct srcu_struct *sp, int *flags,
 #endif
 
 #ifdef CONFIG_TINY_RCU
-static inline bool rcu_dynticks_zero_in_eqs(int cpu, int *vp) { return false; }
+static inline bool rcu_watching_zero_in_eqs(int cpu, int *vp) { return false; }
 static inline unsigned long rcu_get_gp_seq(void) { return 0; }
 static inline unsigned long rcu_exp_batches_completed(void) { return 0; }
 static inline unsigned long
@@ -619,7 +619,7 @@ static inline void rcu_fwd_progress_check(unsigned long j) { }
 static inline void rcu_gp_slow_register(atomic_t *rgssp) { }
 static inline void rcu_gp_slow_unregister(atomic_t *rgssp) { }
 #else /* #ifdef CONFIG_TINY_RCU */
-bool rcu_dynticks_zero_in_eqs(int cpu, int *vp);
+bool rcu_watching_zero_in_eqs(int cpu, int *vp);
 unsigned long rcu_get_gp_seq(void);
 unsigned long rcu_exp_batches_completed(void);
 unsigned long srcu_batches_completed(struct srcu_struct *sp);
diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h
index ba3440a45b6dd..2484c1a8e051a 100644
--- a/kernel/rcu/tasks.h
+++ b/kernel/rcu/tasks.h
@@ -1613,7 +1613,7 @@ static int trc_inspect_reader(struct task_struct *t, void *bhp_in)
 		// However, we cannot safely change its state.
 		n_heavy_reader_attempts++;
 		// Check for "running" idle tasks on offline CPUs.
-		if (!rcu_dynticks_zero_in_eqs(cpu, &t->trc_reader_nesting))
+		if (!rcu_watching_zero_in_eqs(cpu, &t->trc_reader_nesting))
 			return -EINVAL; // No quiescent state, do it the hard way.
 		n_heavy_reader_updates++;
 		nesting = 0;
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 3fd9f0fc4d558..d05bd65fbfdc2 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -333,7 +333,7 @@ static bool rcu_watching_snap_stopped_since(struct rcu_data *rdp, int snap)
  * Return true if the referenced integer is zero while the specified
  * CPU remains within a single extended quiescent state.
  */
-bool rcu_dynticks_zero_in_eqs(int cpu, int *vp)
+bool rcu_watching_zero_in_eqs(int cpu, int *vp)
 {
 	int snap;
 
-- 
2.43.0

[PATCH v3 19/25] rcu: Rename struct rcu_data .dynticks_snap into .watching_snap

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, and the snapshot helpers are now prefix by
"rcu_watching". Reflect that change into the storage variables for these
snapshots.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
---
 .../RCU/Design/Data-Structures/Data-Structures.rst          | 4 ++--
 kernel/rcu/tree.c                                           | 6 +++---
 kernel/rcu/tree.h                                           | 2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/Documentation/RCU/Design/Data-Structures/Data-Structures.rst b/Documentation/RCU/Design/Data-Structures/Data-Structures.rst
index 5389cc49bea13..de76c624fe93e 100644
--- a/Documentation/RCU/Design/Data-Structures/Data-Structures.rst
+++ b/Documentation/RCU/Design/Data-Structures/Data-Structures.rst
@@ -921,10 +921,10 @@ This portion of the ``rcu_data`` structure is declared as follows:
 
 ::
 
-     1   int dynticks_snap;
+     1   int watching_snap;
      2   unsigned long dynticks_fqs;
 
-The ``->dynticks_snap`` field is used to take a snapshot of the
+The ``->watching_snap`` field is used to take a snapshot of the
 corresponding CPU's dyntick-idle state when forcing quiescent states,
 and is therefore accessed from other CPUs. Finally, the
 ``->dynticks_fqs`` field is used to count the number of times this CPU
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index d05bd65fbfdc2..e1dd424f8a4c0 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -779,8 +779,8 @@ static int dyntick_save_progress_counter(struct rcu_data *rdp)
 	 * current GP sequence number is enforced by current rnp locking
 	 * with chained smp_mb__after_unlock_lock().
 	 */
-	rdp->dynticks_snap = ct_rcu_watching_cpu_acquire(rdp->cpu);
-	if (rcu_watching_snap_in_eqs(rdp->dynticks_snap)) {
+	rdp->watching_snap = ct_rcu_watching_cpu_acquire(rdp->cpu);
+	if (rcu_watching_snap_in_eqs(rdp->watching_snap)) {
 		trace_rcu_fqs(rcu_state.name, rdp->gp_seq, rdp->cpu, TPS("dti"));
 		rcu_gpnum_ovf(rdp->mynode, rdp);
 		return 1;
@@ -812,7 +812,7 @@ static int rcu_implicit_dynticks_qs(struct rcu_data *rdp)
 	 * read-side critical section that started before the beginning
 	 * of the current RCU grace period.
 	 */
-	if (rcu_watching_snap_stopped_since(rdp, rdp->dynticks_snap)) {
+	if (rcu_watching_snap_stopped_since(rdp, rdp->watching_snap)) {
 		trace_rcu_fqs(rcu_state.name, rdp->gp_seq, rdp->cpu, TPS("dti"));
 		rcu_gpnum_ovf(rnp, rdp);
 		return 1;
diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h
index fcf2b4aa34417..f5361a7d7269c 100644
--- a/kernel/rcu/tree.h
+++ b/kernel/rcu/tree.h
@@ -206,7 +206,7 @@ struct rcu_data {
 	long		blimit;		/* Upper limit on a processed batch */
 
 	/* 3) dynticks interface. */
-	int dynticks_snap;		/* Per-GP tracking for dynticks. */
+	int  watching_snap;		/* Per-GP tracking for dynticks. */
 	bool rcu_need_heavy_qs;		/* GP old, so heavy quiescent state! */
 	bool rcu_urgent_qs;		/* GP old need light quiescent state. */
 	bool rcu_forced_tick;		/* Forced tick to provide QS. */
-- 
2.43.0

[PATCH v3 20/25] rcu: Rename struct rcu_data .exp_dynticks_snap into .exp_watching_snap

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, and the snapshot helpers are now prefix by
"rcu_watching". Reflect that change into the storage variables for these
snapshots.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 kernel/rcu/tree.h     | 2 +-
 kernel/rcu/tree_exp.h | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h
index f5361a7d7269c..13fdcc2aa0812 100644
--- a/kernel/rcu/tree.h
+++ b/kernel/rcu/tree.h
@@ -215,7 +215,7 @@ struct rcu_data {
 	/* 4) rcu_barrier(), OOM callbacks, and expediting. */
 	unsigned long barrier_seq_snap;	/* Snap of rcu_state.barrier_sequence. */
 	struct rcu_head barrier_head;
-	int exp_dynticks_snap;		/* Double-check need for IPI. */
+	int exp_watching_snap;		/* Double-check need for IPI. */
 
 	/* 5) Callback offloading. */
 #ifdef CONFIG_RCU_NOCB_CPU
diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h
index 29f94bcf34033..c481e440de6bd 100644
--- a/kernel/rcu/tree_exp.h
+++ b/kernel/rcu/tree_exp.h
@@ -372,7 +372,7 @@ static void __sync_rcu_exp_select_node_cpus(struct rcu_exp_work *rewp)
 			if (rcu_watching_snap_in_eqs(snap))
 				mask_ofl_test |= mask;
 			else
-				rdp->exp_dynticks_snap = snap;
+				rdp->exp_watching_snap = snap;
 		}
 	}
 	mask_ofl_ipi = rnp->expmask & ~mask_ofl_test;
@@ -392,7 +392,7 @@ static void __sync_rcu_exp_select_node_cpus(struct rcu_exp_work *rewp)
 		unsigned long mask = rdp->grpmask;
 
 retry_ipi:
-		if (rcu_watching_snap_stopped_since(rdp, rdp->exp_dynticks_snap)) {
+		if (rcu_watching_snap_stopped_since(rdp, rdp->exp_watching_snap)) {
 			mask_ofl_test |= mask;
 			continue;
 		}
-- 
2.43.0

[PATCH v3 21/25] rcu: Rename dyntick_save_progress_counter() into rcu_watching_snap_save()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, and the 'dynticks' prefix can be dropped without losing any
meaning.

Suggested-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
---
 .../RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg        | 2 +-
 .../RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg         | 2 +-
 Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg   | 2 +-
 .../RCU/Design/Memory-Ordering/TreeRCU-hotplug.svg        | 2 +-
 kernel/rcu/tree.c                                         | 8 ++++----
 5 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg
index 423df00c4df9d..13956baf748ad 100644
--- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg
+++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg
@@ -528,7 +528,7 @@
        font-style="normal"
        y="-8652.5312"
        x="2466.7822"
-       xml:space="preserve">dyntick_save_progress_counter()</text>
+       xml:space="preserve">rcu_watching_snap_save()</text>
     <text
        style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier"
        id="text202-7-2-7-2-0"
diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg
index d82a77d03d8cc..9de55fbf5be49 100644
--- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg
+++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg
@@ -844,7 +844,7 @@
      font-style="normal"
      y="1547.8876"
      x="4417.6396"
-     xml:space="preserve">dyntick_save_progress_counter()</text>
+     xml:space="preserve">rcu_watching_snap_save()</text>
   <g
      style="fill:none;stroke-width:0.025in"
      transform="translate(6501.9719,-10685.904)"
diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg
index 53e0dc2a2c793..286a098b9c42f 100644
--- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg
+++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg
@@ -2974,7 +2974,7 @@
        font-style="normal"
        y="38114.047"
        x="-334.33856"
-       xml:space="preserve">dyntick_save_progress_counter()</text>
+       xml:space="preserve">rcu_watching_snap_save()</text>
     <g
        style="fill:none;stroke-width:0.025in"
        transform="translate(1749.9916,25880.249)"
diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-hotplug.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-hotplug.svg
index 4fa7506082bfe..bab69a15c9c22 100644
--- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-hotplug.svg
+++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-hotplug.svg
@@ -516,7 +516,7 @@
        font-style="normal"
        y="-8652.5312"
        x="2466.7822"
-       xml:space="preserve">dyntick_save_progress_counter()</text>
+       xml:space="preserve">rcu_watching_snap_save()</text>
     <text
        style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier"
        id="text202-7-2-7-2-0"
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index e1dd424f8a4c0..e43d4e1ac4e18 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -768,11 +768,11 @@ static void rcu_gpnum_ovf(struct rcu_node *rnp, struct rcu_data *rdp)
 }
 
 /*
- * Snapshot the specified CPU's dynticks counter so that we can later
+ * Snapshot the specified CPU's RCU_WATCHING counter so that we can later
  * credit them with an implicit quiescent state.  Return 1 if this CPU
  * is in dynticks idle mode, which is an extended quiescent state.
  */
-static int dyntick_save_progress_counter(struct rcu_data *rdp)
+static int rcu_watching_snap_save(struct rcu_data *rdp)
 {
 	/*
 	 * Full ordering against accesses prior current GP and also against
@@ -791,7 +791,7 @@ static int dyntick_save_progress_counter(struct rcu_data *rdp)
 /*
  * Returns positive if the specified CPU has passed through a quiescent state
  * by virtue of being in or having passed through an dynticks idle state since
- * the last call to dyntick_save_progress_counter() for this same CPU, or by
+ * the last call to rcu_watching_snap_save() for this same CPU, or by
  * virtue of having been offline.
  *
  * Returns negative if the specified CPU needs a force resched.
@@ -1984,7 +1984,7 @@ static void rcu_gp_fqs(bool first_time)
 
 	if (first_time) {
 		/* Collect dyntick-idle snapshots. */
-		force_qs_rnp(dyntick_save_progress_counter);
+		force_qs_rnp(rcu_watching_snap_save);
 	} else {
 		/* Handle dyntick-idle and offline CPUs. */
 		force_qs_rnp(rcu_implicit_dynticks_qs);
-- 
2.43.0

[PATCH v3 22/25] rcu: Rename rcu_implicit_dynticks_qs() into rcu_implicit_eqs()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, replace "dynticks_qs" into "eqs" to drop the dyntick
reference.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
---
 Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg | 2 +-
 Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg  | 2 +-
 Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg      | 2 +-
 Documentation/RCU/Design/Memory-Ordering/TreeRCU-hotplug.svg | 2 +-
 kernel/rcu/tree.c                                            | 4 ++--
 5 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg
index 13956baf748ad..ab9707f04e666 100644
--- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg
+++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg
@@ -537,7 +537,7 @@
        font-style="normal"
        y="-8368.1475"
        x="2463.3262"
-       xml:space="preserve">rcu_implicit_dynticks_qs()</text>
+       xml:space="preserve">rcu_watching_snap_recheck()</text>
   </g>
   <g
      id="g4504"
diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg
index 9de55fbf5be49..e17c3356041aa 100644
--- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg
+++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg
@@ -899,7 +899,7 @@
      font-style="normal"
      y="1858.8729"
      x="4414.1836"
-     xml:space="preserve">rcu_implicit_dynticks_qs()</text>
+     xml:space="preserve">rcu_watching_snap_recheck()</text>
   <text
      xml:space="preserve"
      x="14659.87"
diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg
index 286a098b9c42f..c43e1ec10774b 100644
--- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg
+++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg
@@ -3029,7 +3029,7 @@
        font-style="normal"
        y="38425.035"
        x="-337.79462"
-       xml:space="preserve">rcu_implicit_dynticks_qs()</text>
+       xml:space="preserve">rcu_watching_snap_recheck()</text>
     <text
        xml:space="preserve"
        x="9907.8887"
diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-hotplug.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-hotplug.svg
index bab69a15c9c22..a92356ce40110 100644
--- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-hotplug.svg
+++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-hotplug.svg
@@ -525,7 +525,7 @@
        font-style="normal"
        y="-8368.1475"
        x="2463.3262"
-       xml:space="preserve">rcu_implicit_dynticks_qs()</text>
+       xml:space="preserve">rcu_watching_snap_recheck()</text>
     <text
        sodipodi:linespacing="125%"
        style="font-size:192px;font-style:normal;font-weight:bold;line-height:125%;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier"
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index e43d4e1ac4e18..3c607e45532bd 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -798,7 +798,7 @@ static int rcu_watching_snap_save(struct rcu_data *rdp)
  *
  * Returns zero otherwise.
  */
-static int rcu_implicit_dynticks_qs(struct rcu_data *rdp)
+static int rcu_watching_snap_recheck(struct rcu_data *rdp)
 {
 	unsigned long jtsq;
 	int ret = 0;
@@ -1987,7 +1987,7 @@ static void rcu_gp_fqs(bool first_time)
 		force_qs_rnp(rcu_watching_snap_save);
 	} else {
 		/* Handle dyntick-idle and offline CPUs. */
-		force_qs_rnp(rcu_implicit_dynticks_qs);
+		force_qs_rnp(rcu_watching_snap_recheck);
 	}
 	/* Clear flag to prevent immediate re-entry. */
 	if (READ_ONCE(rcu_state.gp_flags) & RCU_GP_FLAG_FQS) {
-- 
2.43.0

[PATCH v3 23/25] rcu: Rename rcu_momentary_dyntick_idle() into rcu_momentary_eqs()

[Valentin Schneider]

The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
RCU_WATCHING, replace "dyntick_idle" into "eqs" to drop the dyntick
reference.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
---
 include/linux/rcutiny.h      | 2 +-
 include/linux/rcutree.h      | 2 +-
 kernel/rcu/rcutorture.c      | 4 ++--
 kernel/rcu/tree.c            | 4 ++--
 kernel/rcu/tree_nocb.h       | 2 +-
 kernel/rcu/tree_plugin.h     | 6 +++---
 kernel/stop_machine.c        | 2 +-
 kernel/trace/trace_osnoise.c | 4 ++--
 8 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h
index d9ac7b136aeab..cf2b5a188f783 100644
--- a/include/linux/rcutiny.h
+++ b/include/linux/rcutiny.h
@@ -158,7 +158,7 @@ void rcu_scheduler_starting(void);
 static inline void rcu_end_inkernel_boot(void) { }
 static inline bool rcu_inkernel_boot_has_ended(void) { return true; }
 static inline bool rcu_is_watching(void) { return true; }
-static inline void rcu_momentary_dyntick_idle(void) { }
+static inline void rcu_momentary_eqs(void) { }
 static inline void kfree_rcu_scheduler_running(void) { }
 static inline bool rcu_gp_might_be_stalled(void) { return false; }
 
diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h
index 254244202ea97..7dbde2b6f714a 100644
--- a/include/linux/rcutree.h
+++ b/include/linux/rcutree.h
@@ -37,7 +37,7 @@ void synchronize_rcu_expedited(void);
 void kvfree_call_rcu(struct rcu_head *head, void *ptr);
 
 void rcu_barrier(void);
-void rcu_momentary_dyntick_idle(void);
+void rcu_momentary_eqs(void);
 void kfree_rcu_scheduler_running(void);
 bool rcu_gp_might_be_stalled(void);
 
diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index 08bf7c669dd3d..ef757c3e84a27 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -2680,7 +2680,7 @@ static unsigned long rcu_torture_fwd_prog_cbfree(struct rcu_fwd *rfp)
 		rcu_torture_fwd_prog_cond_resched(freed);
 		if (tick_nohz_full_enabled()) {
 			local_irq_save(flags);
-			rcu_momentary_dyntick_idle();
+			rcu_momentary_eqs();
 			local_irq_restore(flags);
 		}
 	}
@@ -2830,7 +2830,7 @@ static void rcu_torture_fwd_prog_cr(struct rcu_fwd *rfp)
 		rcu_torture_fwd_prog_cond_resched(n_launders + n_max_cbs);
 		if (tick_nohz_full_enabled()) {
 			local_irq_save(flags);
-			rcu_momentary_dyntick_idle();
+			rcu_momentary_eqs();
 			local_irq_restore(flags);
 		}
 	}
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 3c607e45532bd..2c9482b962a99 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -359,7 +359,7 @@ bool rcu_watching_zero_in_eqs(int cpu, int *vp)
  *
  * The caller must have disabled interrupts and must not be idle.
  */
-notrace void rcu_momentary_dyntick_idle(void)
+notrace void rcu_momentary_eqs(void)
 {
 	int seq;
 
@@ -369,7 +369,7 @@ notrace void rcu_momentary_dyntick_idle(void)
 	WARN_ON_ONCE(!(seq & CT_RCU_WATCHING));
 	rcu_preempt_deferred_qs(current);
 }
-EXPORT_SYMBOL_GPL(rcu_momentary_dyntick_idle);
+EXPORT_SYMBOL_GPL(rcu_momentary_eqs);
 
 /**
  * rcu_is_cpu_rrupt_from_idle - see if 'interrupted' from idle
diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h
index 3ce30841119ad..e629646ac6f52 100644
--- a/kernel/rcu/tree_nocb.h
+++ b/kernel/rcu/tree_nocb.h
@@ -917,7 +917,7 @@ static void nocb_cb_wait(struct rcu_data *rdp)
 	WARN_ON_ONCE(!rcu_rdp_is_offloaded(rdp));
 
 	local_irq_save(flags);
-	rcu_momentary_dyntick_idle();
+	rcu_momentary_eqs();
 	local_irq_restore(flags);
 	/*
 	 * Disable BH to provide the expected environment.  Also, when
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 51125f4130fd4..5fc7309187e1f 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -869,7 +869,7 @@ static void rcu_qs(void)
 
 /*
  * Register an urgently needed quiescent state.  If there is an
- * emergency, invoke rcu_momentary_dyntick_idle() to do a heavy-weight
+ * emergency, invoke rcu_momentary_eqs() to do a heavy-weight
  * dyntick-idle quiescent state visible to other CPUs, which will in
  * some cases serve for expedited as well as normal grace periods.
  * Either way, register a lightweight quiescent state.
@@ -889,7 +889,7 @@ void rcu_all_qs(void)
 	this_cpu_write(rcu_data.rcu_urgent_qs, false);
 	if (unlikely(raw_cpu_read(rcu_data.rcu_need_heavy_qs))) {
 		local_irq_save(flags);
-		rcu_momentary_dyntick_idle();
+		rcu_momentary_eqs();
 		local_irq_restore(flags);
 	}
 	rcu_qs();
@@ -909,7 +909,7 @@ void rcu_note_context_switch(bool preempt)
 		goto out;
 	this_cpu_write(rcu_data.rcu_urgent_qs, false);
 	if (unlikely(raw_cpu_read(rcu_data.rcu_need_heavy_qs)))
-		rcu_momentary_dyntick_idle();
+		rcu_momentary_eqs();
 out:
 	rcu_tasks_qs(current, preempt);
 	trace_rcu_utilization(TPS("End context switch"));
diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c
index cedb17ba158a9..da821ce258ea7 100644
--- a/kernel/stop_machine.c
+++ b/kernel/stop_machine.c
@@ -251,7 +251,7 @@ static int multi_cpu_stop(void *data)
 			 */
 			touch_nmi_watchdog();
 		}
-		rcu_momentary_dyntick_idle();
+		rcu_momentary_eqs();
 	} while (curstate != MULTI_STOP_EXIT);
 
 	local_irq_restore(flags);
diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c
index a8e28f9b9271c..a41e8a8212647 100644
--- a/kernel/trace/trace_osnoise.c
+++ b/kernel/trace/trace_osnoise.c
@@ -1535,7 +1535,7 @@ static int run_osnoise(void)
 		 * This will eventually cause unwarranted noise as PREEMPT_RCU
 		 * will force preemption as the means of ending the current
 		 * grace period. We avoid this problem by calling
-		 * rcu_momentary_dyntick_idle(), which performs a zero duration
+		 * rcu_momentary_eqs(), which performs a zero duration
 		 * EQS allowing PREEMPT_RCU to end the current grace period.
 		 * This call shouldn't be wrapped inside an RCU critical
 		 * section.
@@ -1547,7 +1547,7 @@ static int run_osnoise(void)
 			if (!disable_irq)
 				local_irq_disable();
 
-			rcu_momentary_dyntick_idle();
+			rcu_momentary_eqs();
 
 			if (!disable_irq)
 				local_irq_enable();
-- 
2.43.0

[PATCH v3 24/25] rcu: Update stray documentation references to rcu_dynticks_eqs_{enter, exit}()

[Valentin Schneider]

rcu_dynticks_eqs_{enter, exit}() have been replaced by their
context-tracking counterparts since commit:

  171476775d32 ("context_tracking: Convert state to atomic_t")

Update the stray documentation references.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
---
 .../RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst   | 4 ++--
 Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg  | 4 ++--
 Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg   | 4 ++--
 Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg       | 4 ++--
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst b/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
index 7163d0def34e6..1a5ff1a9f02e3 100644
--- a/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
+++ b/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
@@ -147,8 +147,8 @@ RCU read-side critical sections preceding and following the current
 idle sojourn.
 This case is handled by calls to the strongly ordered
 ``atomic_add_return()`` read-modify-write atomic operation that
-is invoked within ``rcu_dynticks_eqs_enter()`` at idle-entry
-time and within ``rcu_dynticks_eqs_exit()`` at idle-exit time.
+is invoked within ``ct_kernel_exit_state()`` at idle-entry
+time and within ``ct_kernel_enter_state()`` at idle-exit time.
 The grace-period kthread invokes first ``ct_rcu_watching_cpu_acquire()``
 (preceded by a full memory barrier) and ``rcu_watching_snap_stopped_since()``
 (both of which rely on acquire semantics) to detect idle CPUs.
diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg
index ab9707f04e666..3fbc19c48a584 100644
--- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg
+++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-dyntick.svg
@@ -607,7 +607,7 @@
        font-weight="bold"
        font-size="192"
        id="text202-7-5-3-27-6"
-       style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcu_dynticks_eqs_enter()</text>
+       style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">ct_kernel_exit_state()</text>
     <text
        xml:space="preserve"
        x="3745.7725"
@@ -638,7 +638,7 @@
        font-weight="bold"
        font-size="192"
        id="text202-7-5-3-27-6-1"
-       style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcu_dynticks_eqs_exit()</text>
+       style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">ct_kernel_enter_state()</text>
     <text
        xml:space="preserve"
        x="3745.7725"
diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg
index e17c3356041aa..25c7acc8a4c25 100644
--- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg
+++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg
@@ -977,7 +977,7 @@
        font-weight="bold"
        font-size="192"
        id="text202-7-5-3-27-6"
-       style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcu_dynticks_eqs_enter()</text>
+       style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">ct_kernel_exit_state()</text>
     <text
        xml:space="preserve"
        x="3745.7725"
@@ -1008,7 +1008,7 @@
        font-weight="bold"
        font-size="192"
        id="text202-7-5-3-27-6-1"
-       style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcu_dynticks_eqs_exit()</text>
+       style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">ct_kernel_enter_state()</text>
     <text
        xml:space="preserve"
        x="3745.7725"
diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg
index c43e1ec10774b..d05bc7b27edb7 100644
--- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg
+++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg
@@ -3107,7 +3107,7 @@
          font-weight="bold"
          font-size="192"
          id="text202-7-5-3-27-6"
-         style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcu_dynticks_eqs_enter()</text>
+         style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">ct_kernel_exit_state()</text>
       <text
          xml:space="preserve"
          x="3745.7725"
@@ -3138,7 +3138,7 @@
          font-weight="bold"
          font-size="192"
          id="text202-7-5-3-27-6-1"
-         style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcu_dynticks_eqs_exit()</text>
+         style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">ct_kernel_enter_state()</text>
       <text
          xml:space="preserve"
          x="3745.7725"
-- 
2.43.0

[PATCH v3 25/25] context_tracking, rcu: Rename rcu_dyntick trace event into rcu_watching

[Valentin Schneider]

The "rcu_dyntick" naming convention has been turned into "rcu_watching" for
all helpers now, align the trace event to that.

To add to the confusion, the strings passed to the trace event are now
reversed: when RCU "starts" the dyntick / EQS state, it "stops" watching.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
---
 include/trace/events/rcu.h | 16 ++++++++--------
 kernel/context_tracking.c  | 10 +++++-----
 2 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/include/trace/events/rcu.h b/include/trace/events/rcu.h
index 4066b6d51e46a..e51ef658437f0 100644
--- a/include/trace/events/rcu.h
+++ b/include/trace/events/rcu.h
@@ -466,40 +466,40 @@ TRACE_EVENT(rcu_stall_warning,
 /*
  * Tracepoint for dyntick-idle entry/exit events.  These take 2 strings
  * as argument:
- * polarity: "Start", "End", "StillNonIdle" for entering, exiting or still not
+ * polarity: "Start", "End", "StillWatching" for entering, exiting or still not
  *            being in dyntick-idle mode.
  * context: "USER" or "IDLE" or "IRQ".
  * NMIs nested in IRQs are inferred with nesting > 1 in IRQ context.
  *
  * These events also take a pair of numbers, which indicate the nesting
  * depth before and after the event of interest, and a third number that is
- * the ->dynticks counter.  Note that task-related and interrupt-related
+ * the RCU_WATCHING counter.  Note that task-related and interrupt-related
  * events use two separate counters, and that the "++=" and "--=" events
  * for irq/NMI will change the counter by two, otherwise by one.
  */
-TRACE_EVENT_RCU(rcu_dyntick,
+TRACE_EVENT_RCU(rcu_watching,
 
-	TP_PROTO(const char *polarity, long oldnesting, long newnesting, int dynticks),
+	TP_PROTO(const char *polarity, long oldnesting, long newnesting, int counter),
 
-	TP_ARGS(polarity, oldnesting, newnesting, dynticks),
+	TP_ARGS(polarity, oldnesting, newnesting, counter),
 
 	TP_STRUCT__entry(
 		__field(const char *, polarity)
 		__field(long, oldnesting)
 		__field(long, newnesting)
-		__field(int, dynticks)
+		__field(int, counter)
 	),
 
 	TP_fast_assign(
 		__entry->polarity = polarity;
 		__entry->oldnesting = oldnesting;
 		__entry->newnesting = newnesting;
-		__entry->dynticks = dynticks;
+		__entry->counter = counter;
 	),
 
 	TP_printk("%s %lx %lx %#3x", __entry->polarity,
 		  __entry->oldnesting, __entry->newnesting,
-		  __entry->dynticks & 0xfff)
+		  __entry->counter & 0xfff)
 );
 
 /*
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index e64bb746cd9b3..152b485a62db5 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -137,7 +137,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
 
 	instrumentation_begin();
 	lockdep_assert_irqs_disabled();
-	trace_rcu_dyntick(TPS("Start"), ct_nesting(), 0, ct_rcu_watching());
+	trace_rcu_watching(TPS("End"), ct_nesting(), 0, ct_rcu_watching());
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current));
 	rcu_preempt_deferred_qs(current);
 
@@ -182,7 +182,7 @@ static void noinstr ct_kernel_enter(bool user, int offset)
 	// instrumentation for the noinstr ct_kernel_enter_state()
 	instrument_atomic_write(&ct->state, sizeof(ct->state));
 
-	trace_rcu_dyntick(TPS("End"), ct_nesting(), 1, ct_rcu_watching());
+	trace_rcu_watching(TPS("Start"), ct_nesting(), 1, ct_rcu_watching());
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current));
 	WRITE_ONCE(ct->nesting, 1);
 	WARN_ON_ONCE(ct_nmi_nesting());
@@ -219,7 +219,7 @@ void noinstr ct_nmi_exit(void)
 	 * leave it in non-RCU-idle state.
 	 */
 	if (ct_nmi_nesting() != 1) {
-		trace_rcu_dyntick(TPS("--="), ct_nmi_nesting(), ct_nmi_nesting() - 2,
+		trace_rcu_watching(TPS("--="), ct_nmi_nesting(), ct_nmi_nesting() - 2,
 				  ct_rcu_watching());
 		WRITE_ONCE(ct->nmi_nesting, /* No store tearing. */
 			   ct_nmi_nesting() - 2);
@@ -228,7 +228,7 @@ void noinstr ct_nmi_exit(void)
 	}
 
 	/* This NMI interrupted an RCU-idle CPU, restore RCU-idleness. */
-	trace_rcu_dyntick(TPS("Startirq"), ct_nmi_nesting(), 0, ct_rcu_watching());
+	trace_rcu_watching(TPS("Endirq"), ct_nmi_nesting(), 0, ct_rcu_watching());
 	WRITE_ONCE(ct->nmi_nesting, 0); /* Avoid store tearing. */
 
 	// instrumentation for the noinstr ct_kernel_exit_state()
@@ -294,7 +294,7 @@ void noinstr ct_nmi_enter(void)
 		instrumentation_begin();
 	}
 
-	trace_rcu_dyntick(incby == 1 ? TPS("Endirq") : TPS("++="),
+	trace_rcu_watching(incby == 1 ? TPS("Startirq") : TPS("++="),
 			  ct_nmi_nesting(),
 			  ct_nmi_nesting() + incby, ct_rcu_watching());
 	instrumentation_end();
-- 
2.43.0

Re: [PATCH v3 05/25] context_tracking, rcu: Rename ct_dynticks_cpu_acquire() into ct_rcu_watching_cpu_acquire()

[Frederic Weisbecker]

Le Wed, Jul 24, 2024 at 04:43:05PM +0200, Valentin Schneider a écrit :
> The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
> RCU_WATCHING, reflect that change in the related helpers.
> 
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>

Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

Re: [PATCH v3 14/25] context_tracking, rcu: Rename rcu_dynticks_curr_cpu_in_eqs() into rcu_watching_curr_cpu()

[Frederic Weisbecker]

Le Wed, Jul 24, 2024 at 04:43:14PM +0200, Valentin Schneider a écrit :
> The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
> RCU_WATCHING, reflect that change in the related helpers.
> 
> Note that "watching" is the opposite of "in EQS", so the negation is lifted
> out of the helper and into the callsites.
> 
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>
> Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

You may need to update the patch title on this one.

Thanks.

Re: [PATCH v3 15/25] rcu: Rename rcu_dynticks_eqs_online() into rcu_watching_online()

[Frederic Weisbecker]

Le Wed, Jul 24, 2024 at 04:43:15PM +0200, Valentin Schneider a écrit :
> The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
> RCU_WATCHING, reflect that change in the related helpers.
> 
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>

Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

Re: [PATCH v3 17/25] rcu: Rename rcu_dynticks_in_eqs_since() into rcu_watching_snap_stopped_since()

[Frederic Weisbecker]

Le Wed, Jul 24, 2024 at 04:43:17PM +0200, Valentin Schneider a écrit :
> The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
> RCU_WATCHING, the dynticks prefix can go.
> 
> While at it, this helper is only meant to be called after failing an
> earlier call to rcu_watching_snap_in_eqs(), document this in the comments
> and add a WARN_ON_ONCE() for good measure.
> 
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>

Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

Re: [PATCH v3 19/25] rcu: Rename struct rcu_data .dynticks_snap into .watching_snap

[Frederic Weisbecker]

Le Wed, Jul 24, 2024 at 04:43:19PM +0200, Valentin Schneider a écrit :
> The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
> RCU_WATCHING, and the snapshot helpers are now prefix by
> "rcu_watching". Reflect that change into the storage variables for these
> snapshots.
> 
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>

Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

Re: [PATCH v3 21/25] rcu: Rename dyntick_save_progress_counter() into rcu_watching_snap_save()

[Frederic Weisbecker]

Le Wed, Jul 24, 2024 at 04:43:21PM +0200, Valentin Schneider a écrit :
> The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
> RCU_WATCHING, and the 'dynticks' prefix can be dropped without losing any
> meaning.
> 
> Suggested-by: Frederic Weisbecker <frederic@kernel.org>
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>

Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

Re: [PATCH v3 22/25] rcu: Rename rcu_implicit_dynticks_qs() into rcu_implicit_eqs()

[Frederic Weisbecker]

Le Wed, Jul 24, 2024 at 04:43:22PM +0200, Valentin Schneider a écrit :
> The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
> RCU_WATCHING, replace "dynticks_qs" into "eqs" to drop the dyntick
> reference.
> 
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>

You'll need to update the patch title and changelog for this one.
Other than that:

Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

Re: [PATCH v3 24/25] rcu: Update stray documentation references to rcu_dynticks_eqs_{enter, exit}()

[Frederic Weisbecker]

Le Wed, Jul 24, 2024 at 04:43:24PM +0200, Valentin Schneider a écrit :
> rcu_dynticks_eqs_{enter, exit}() have been replaced by their
> context-tracking counterparts since commit:
> 
>   171476775d32 ("context_tracking: Convert state to atomic_t")
> 
> Update the stray documentation references.
> 
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>

Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

Re: [PATCH v3 25/25] context_tracking, rcu: Rename rcu_dyntick trace event into rcu_watching

[Frederic Weisbecker]

Le Wed, Jul 24, 2024 at 04:43:25PM +0200, Valentin Schneider a écrit :
> The "rcu_dyntick" naming convention has been turned into "rcu_watching" for
> all helpers now, align the trace event to that.
> 
> To add to the confusion, the strings passed to the trace event are now
> reversed: when RCU "starts" the dyntick / EQS state, it "stops" watching.
> 
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>
> ---
>  include/trace/events/rcu.h | 16 ++++++++--------
>  kernel/context_tracking.c  | 10 +++++-----
>  2 files changed, 13 insertions(+), 13 deletions(-)
> 
> diff --git a/include/trace/events/rcu.h b/include/trace/events/rcu.h
> index 4066b6d51e46a..e51ef658437f0 100644
> --- a/include/trace/events/rcu.h
> +++ b/include/trace/events/rcu.h
> @@ -466,40 +466,40 @@ TRACE_EVENT(rcu_stall_warning,
>  /*
>   * Tracepoint for dyntick-idle entry/exit events.  These take 2 strings
>   * as argument:
> - * polarity: "Start", "End", "StillNonIdle" for entering, exiting or still not
> + * polarity: "Start", "End", "StillWatching" for entering, exiting or still not
>   *            being in dyntick-idle mode.
>   * context: "USER" or "IDLE" or "IRQ".

So that should be:

 * polarity: "Start", "End", "StillWatching" for exiting, entering or still not
             being in EQS mode.

Right?

Other than that:

Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

RCU-Task[-Trace] VS EQS (was Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*())

[Frederic Weisbecker]

Le Wed, Jul 24, 2024 at 04:43:13PM +0200, Valentin Schneider a écrit :
> -/* Turn on heavyweight RCU tasks trace readers on idle/user entry. */
> -static __always_inline void rcu_dynticks_task_trace_enter(void)
> +/* Turn on heavyweight RCU tasks trace readers on kernel exit. */
> +static __always_inline void rcu_task_trace_exit(void)

Before I proceed on this last one, a few questions for Paul and others:

1) Why is rcu_dynticks_task_exit() not called while entering in NMI?
   Does that mean that NMIs aren't RCU-Task read side critical sections?

2) Looking further into CONFIG_TASKS_TRACE_RCU_READ_MB=y, it seems to
   allow for uses of rcu_read_[un]lock_trace() while RCU is not watching
   (EQS). Is it really a good idea to support that? Are we aware of any
   such potential usecase?

Thanks!
   

Re: [PATCH v3 14/25] context_tracking, rcu: Rename rcu_dynticks_curr_cpu_in_eqs() into rcu_watching_curr_cpu()

[Valentin Schneider]

On 25/07/24 14:10, Frederic Weisbecker wrote:
> Le Wed, Jul 24, 2024 at 04:43:14PM +0200, Valentin Schneider a écrit :
>> The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
>> RCU_WATCHING, reflect that change in the related helpers.
>>
>> Note that "watching" is the opposite of "in EQS", so the negation is lifted
>> out of the helper and into the callsites.
>>
>> Signed-off-by: Valentin Schneider <vschneid@redhat.com>
>> Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
>
> You may need to update the patch title on this one.
>

Duh, there's a few of these, sorry about & thanks for spotting them.

> Thanks.

Re: [PATCH v3 25/25] context_tracking, rcu: Rename rcu_dyntick trace event into rcu_watching

[Valentin Schneider]

On 25/07/24 14:39, Frederic Weisbecker wrote:
> Le Wed, Jul 24, 2024 at 04:43:25PM +0200, Valentin Schneider a écrit :
>> The "rcu_dyntick" naming convention has been turned into "rcu_watching" for
>> all helpers now, align the trace event to that.
>>
>> To add to the confusion, the strings passed to the trace event are now
>> reversed: when RCU "starts" the dyntick / EQS state, it "stops" watching.
>>
>> Signed-off-by: Valentin Schneider <vschneid@redhat.com>
>> ---
>>  include/trace/events/rcu.h | 16 ++++++++--------
>>  kernel/context_tracking.c  | 10 +++++-----
>>  2 files changed, 13 insertions(+), 13 deletions(-)
>>
>> diff --git a/include/trace/events/rcu.h b/include/trace/events/rcu.h
>> index 4066b6d51e46a..e51ef658437f0 100644
>> --- a/include/trace/events/rcu.h
>> +++ b/include/trace/events/rcu.h
>> @@ -466,40 +466,40 @@ TRACE_EVENT(rcu_stall_warning,
>>  /*
>>   * Tracepoint for dyntick-idle entry/exit events.  These take 2 strings
>>   * as argument:
>> - * polarity: "Start", "End", "StillNonIdle" for entering, exiting or still not
>> + * polarity: "Start", "End", "StillWatching" for entering, exiting or still not
>>   *            being in dyntick-idle mode.
>>   * context: "USER" or "IDLE" or "IRQ".
>
> So that should be:
>
>  * polarity: "Start", "End", "StillWatching" for exiting, entering or still not
>              being in EQS mode.
>
> Right?
>

Yep that's better, thanks!

> Other than that:
>
> Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

Re: [PATCH v3 00/25] context_tracking, rcu: Spring cleaning of dynticks references

[Neeraj Upadhyay]

On Wed, Jul 24, 2024 at 04:43:00PM +0200, Valentin Schneider wrote:
> Git info
> ========
> 
> The series is based on rcu/next at:
> f395ae346be5 ("Merge branches 'doc.2024.06.06a', 'fixes.2024.06.06a', 'mb.2024.06.03a', 'nocb.2024.06.03a', 'rcu-tasks.2024.06.06a' and 'rcutorture.2024.06.06a' into HEAD")
>

Hi Valentin,

I see that this series is based on paulmck/linux-rcu.git next branch,
whereas the RCU tree has moved to shared tree now [1] and the next
branch there is pulled for v6.11 (tag: rcu.2024.07.12a). I get merge
conflicts while applying it. Can you please rebase?


[1] https://git.kernel.org/pub/scm/linux/kernel/git/rcu/linux.git/log/?h=next


- Neeraj

> It's also available as a git tree at:
> https://gitlab.com/vschneid/linux.git -b redhat/isolirq/defer/v3-rcu-v3
> 
> 
> Series description
> ==================
> 
> I've done one patch per renaming for my own sanity, a lot of these probably
> deserve to be squashed together (e.g. all of the nmi_nesting into one).
> 
> At the end of the renaming exercise, the remaining RCU references to "dyntick"
> are:
> - Comments in core RCU code:
>   kernel/rcu/:
>   tree.h:208:8:	/* 3) dynticks interface. */
>   tree.h:209:46:	int  watching_snap;		/* Per-GP tracking for dynticks. */
>   tree_nocb.h:55:19: * about entering dyntick-idle mode.
>   tree_plugin.h:873:4: * dyntick-idle quiescent state visible to other CPUs, which will in
>   Kconfig:11:4:	# Dynticks-idle tracking
>   tree.c:358:56: * We inform the RCU core by emulating a zero-duration dyntick-idle period.
>   tree.c:773:10: * is in dynticks idle mode, which is an extended quiescent state.
>   tree.c:793:54: * by virtue of being in or having passed through an dynticks idle state since
>   tree.c:808:44:	 * If the CPU passed through or entered a dynticks idle phase with
>   tree.c:1986:14:		/* Collect dyntick-idle snapshots. */
>   tree.c:1989:13:		/* Handle dyntick-idle and offline CPUs. */
>   tree.c:2683:54: * Otherwise, invoke the specified function to check dyntick state for
>   tree.c:2743:16: * CPUs are in dyntick-idle mode.
> 
> 
> - Stale documentation in RCU/Design/Data-Structures/Data-Structures.rst
>   referencing fields that have been moved out of RCU after
>   904e600e60f4 ("rcu/context_tracking: Move dynticks_nesting to context tracking")
> 
> Testing
> =======
> 
> Ran through TREE01 and TREE04.
> 
> Revisions
> =========
> 
> v2 -> v3
> ++++++++
> 
> o Collected Reviewed-By
> o Rebased onto latest rcu/next
> 
> o Made it clear rcu_dynticks_in_eqs_since() is to be used in wait-for-EQS loops
> o Slight rename variations following v2 suggestions
> 
> Cheers,
> Valentin
> 
> 
> Valentin Schneider (25):
>   treewide: context_tracking: Rename CONTEXT_* into CT_STATE_*
>   context_tracking, rcu: Rename RCU_DYNTICKS_IDX into CT_RCU_WATCHING
>   context_tracking, rcu: Rename ct_dynticks() into ct_rcu_watching()
>   context_tracking, rcu: Rename ct_dynticks_cpu() into
>     ct_rcu_watching_cpu()
>   context_tracking, rcu: Rename ct_dynticks_cpu_acquire() into
>     ct_rcu_watching_cpu_acquire()
>   context_tracking, rcu: Rename struct context_tracking
>     .dynticks_nesting into .nesting
>   context_tracking, rcu: Rename ct_dynticks_nesting() into ct_nesting()
>   context_tracking, rcu: Rename ct_dynticks_nesting_cpu() into
>     ct_nesting_cpu()
>   context_tracking, rcu: Rename struct context_tracking
>     .dynticks_nmi_nesting into .nmi_nesting
>   context_tracking, rcu: Rename ct_dynticks_nmi_nesting() into
>     ct_nmi_nesting()
>   context_tracking, rcu: Rename ct_dynticks_nmi_nesting_cpu() into
>     ct_nmi_nesting_cpu()
>   context_tracking, rcu: Rename DYNTICK_IRQ_NONIDLE into
>     CT_NESTING_IRQ_NONIDLE
>   context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*()
>   context_tracking, rcu: Rename rcu_dynticks_curr_cpu_in_eqs() into
>     rcu_watching_curr_cpu()
>   rcu: Rename rcu_dynticks_eqs_online() into rcu_watching_online()
>   rcu: Rename rcu_dynticks_in_eqs() into rcu_watching_snap_in_eqs()
>   rcu: Rename rcu_dynticks_in_eqs_since() into
>     rcu_watching_snap_stopped_since()
>   rcu: Rename rcu_dynticks_zero_in_eqs() into rcu_watching_zero_in_eqs()
>   rcu: Rename struct rcu_data .dynticks_snap into .watching_snap
>   rcu: Rename struct rcu_data .exp_dynticks_snap into .exp_watching_snap
>   rcu: Rename dyntick_save_progress_counter() into
>     rcu_watching_snap_save()
>   rcu: Rename rcu_implicit_dynticks_qs() into rcu_implicit_eqs()
>   rcu: Rename rcu_momentary_dyntick_idle() into rcu_momentary_eqs()
>   rcu: Update stray documentation references to rcu_dynticks_eqs_{enter,
>     exit}()
>   context_tracking, rcu: Rename rcu_dyntick trace event into
>     rcu_watching
> 
>  .../Data-Structures/Data-Structures.rst       |  28 ++--
>  .../Tree-RCU-Memory-Ordering.rst              |   8 +-
>  .../Memory-Ordering/TreeRCU-dyntick.svg       |   8 +-
>  .../Design/Memory-Ordering/TreeRCU-gp-fqs.svg |   8 +-
>  .../RCU/Design/Memory-Ordering/TreeRCU-gp.svg |   8 +-
>  .../Memory-Ordering/TreeRCU-hotplug.svg       |   4 +-
>  arch/Kconfig                                  |   2 +-
>  arch/arm64/kernel/entry-common.c              |   2 +-
>  arch/powerpc/include/asm/interrupt.h          |   6 +-
>  arch/powerpc/kernel/interrupt.c               |   6 +-
>  arch/powerpc/kernel/syscall.c                 |   2 +-
>  arch/x86/entry/common.c                       |   2 +-
>  include/linux/context_tracking.h              |  32 ++--
>  include/linux/context_tracking_state.h        |  58 ++++----
>  include/linux/entry-common.h                  |   2 +-
>  include/linux/rcutiny.h                       |   2 +-
>  include/linux/rcutree.h                       |   2 +-
>  include/trace/events/rcu.h                    |  18 +--
>  kernel/context_tracking.c                     | 140 +++++++++---------
>  kernel/entry/common.c                         |   2 +-
>  kernel/rcu/rcu.h                              |   4 +-
>  kernel/rcu/rcutorture.c                       |   4 +-
>  kernel/rcu/tasks.h                            |   2 +-
>  kernel/rcu/tree.c                             | 111 +++++++-------
>  kernel/rcu/tree.h                             |   4 +-
>  kernel/rcu/tree_exp.h                         |   8 +-
>  kernel/rcu/tree_nocb.h                        |   2 +-
>  kernel/rcu/tree_plugin.h                      |   6 +-
>  kernel/rcu/tree_stall.h                       |   6 +-
>  kernel/sched/core.c                           |   4 +-
>  kernel/stop_machine.c                         |   2 +-
>  kernel/trace/trace_osnoise.c                  |   4 +-
>  32 files changed, 256 insertions(+), 241 deletions(-)
> 
> --
> 2.43.0
> 

Re: [PATCH v3 00/25] context_tracking, rcu: Spring cleaning of dynticks references

[Valentin Schneider]

On 25/07/24 20:52, Neeraj Upadhyay wrote:
> On Wed, Jul 24, 2024 at 04:43:00PM +0200, Valentin Schneider wrote:
>> Git info
>> ========
>>
>> The series is based on rcu/next at:
>> f395ae346be5 ("Merge branches 'doc.2024.06.06a', 'fixes.2024.06.06a', 'mb.2024.06.03a', 'nocb.2024.06.03a', 'rcu-tasks.2024.06.06a' and 'rcutorture.2024.06.06a' into HEAD")
>>
>
> Hi Valentin,
>
> I see that this series is based on paulmck/linux-rcu.git next branch,
> whereas the RCU tree has moved to shared tree now [1] and the next
> branch there is pulled for v6.11 (tag: rcu.2024.07.12a). I get merge
> conflicts while applying it. Can you please rebase?
>

My bad, thanks for pointing this out!

Re: [PATCH v3 00/25] context_tracking, rcu: Spring cleaning of dynticks references

[Paul E. McKenney]

On Thu, Jul 25, 2024 at 06:07:49PM +0200, Valentin Schneider wrote:
> On 25/07/24 20:52, Neeraj Upadhyay wrote:
> > On Wed, Jul 24, 2024 at 04:43:00PM +0200, Valentin Schneider wrote:
> >> Git info
> >> ========
> >>
> >> The series is based on rcu/next at:
> >> f395ae346be5 ("Merge branches 'doc.2024.06.06a', 'fixes.2024.06.06a', 'mb.2024.06.03a', 'nocb.2024.06.03a', 'rcu-tasks.2024.06.06a' and 'rcutorture.2024.06.06a' into HEAD")
> >>
> >
> > Hi Valentin,
> >
> > I see that this series is based on paulmck/linux-rcu.git next branch,
> > whereas the RCU tree has moved to shared tree now [1] and the next
> > branch there is pulled for v6.11 (tag: rcu.2024.07.12a). I get merge
> > conflicts while applying it. Can you please rebase?
> 
> My bad, thanks for pointing this out!

Actually my bad, as I haven't done much of anything to inform people of
this change.  Huh.  I do have this LWN RCU API article in preparation,
and that sounds like as good a place to announce this as any.  ;-)

							Thanx, Paul

Re: [PATCH v3 00/25] context_tracking, rcu: Spring cleaning of dynticks references

[Valentin Schneider]

On 25/07/24 20:52, Neeraj Upadhyay wrote:
> On Wed, Jul 24, 2024 at 04:43:00PM +0200, Valentin Schneider wrote:
>> Git info
>> ========
>>
>> The series is based on rcu/next at:
>> f395ae346be5 ("Merge branches 'doc.2024.06.06a', 'fixes.2024.06.06a', 'mb.2024.06.03a', 'nocb.2024.06.03a', 'rcu-tasks.2024.06.06a' and 'rcutorture.2024.06.06a' into HEAD")
>>
>
> Hi Valentin,
>
> I see that this series is based on paulmck/linux-rcu.git next branch,
> whereas the RCU tree has moved to shared tree now [1] and the next
> branch there is pulled for v6.11 (tag: rcu.2024.07.12a). I get merge
> conflicts while applying it. Can you please rebase?
>

FYI I've stashed what will be v4 at the below, based on rcu.2024.07.12a:

https://gitlab.com/vschneid/linux.git -b redhat/isolirq/defer/v3-rcu-v4

Re: [PATCH v3 01/25] treewide: context_tracking: Rename CONTEXT_* into CT_STATE_*

[Thomas Gleixner]

On Wed, Jul 24 2024 at 16:43, Valentin Schneider wrote:

> Context tracking state related symbols currently use a mix of the
> CONTEXT_ (e.g. CONTEXT_KERNEL) and CT_SATE_ (e.g. CT_STATE_MASK) prefixes.
>
> Clean up the naming and make the ctx_state enum use the CT_STATE_ prefix.
>
> Suggested-by: Frederic Weisbecker <frederic@kernel.org>
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>
> Acked-by: Frederic Weisbecker <frederic@kernel.org>

Acked-by: Thomas Gleixner <tglx@linutronix.de>

Re: [PATCH v3 02/25] context_tracking, rcu: Rename RCU_DYNTICKS_IDX into CT_RCU_WATCHING

[Thomas Gleixner]

On Wed, Jul 24 2024 at 16:43, Valentin Schneider wrote:
> The symbols relating to the CT_STATE part of context_tracking.state are now
> all prefixed with CT_STATE.
>
> The RCU dynticks counter part of that atomic variable still involves
> symbols with different prefixes, align them all to be prefixed with
> CT_RCU_WATCHING.
>
> Suggested-by: "Paul E. McKenney" <paulmck@kernel.org>
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>
> Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

Acked-by: Thomas Gleixner <tglx@linutronix.de>

Re: [PATCH v3 00/25] context_tracking, rcu: Spring cleaning of dynticks references

[Neeraj Upadhyay]

On Fri, Jul 26, 2024 at 04:43:21PM +0200, Valentin Schneider wrote:
> On 25/07/24 20:52, Neeraj Upadhyay wrote:
> > On Wed, Jul 24, 2024 at 04:43:00PM +0200, Valentin Schneider wrote:
> >> Git info
> >> ========
> >>
> >> The series is based on rcu/next at:
> >> f395ae346be5 ("Merge branches 'doc.2024.06.06a', 'fixes.2024.06.06a', 'mb.2024.06.03a', 'nocb.2024.06.03a', 'rcu-tasks.2024.06.06a' and 'rcutorture.2024.06.06a' into HEAD")
> >>
> >
> > Hi Valentin,
> >
> > I see that this series is based on paulmck/linux-rcu.git next branch,
> > whereas the RCU tree has moved to shared tree now [1] and the next
> > branch there is pulled for v6.11 (tag: rcu.2024.07.12a). I get merge
> > conflicts while applying it. Can you please rebase?
> >
> 
> FYI I've stashed what will be v4 at the below, based on rcu.2024.07.12a:
> 
> https://gitlab.com/vschneid/linux.git -b redhat/isolirq/defer/v3-rcu-v4

Thanks!

- Neeraj

> 

Re: RCU-Task[-Trace] VS EQS (was Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*())

[Paul E. McKenney]

On Thu, Jul 25, 2024 at 04:32:46PM +0200, Frederic Weisbecker wrote:
> Le Wed, Jul 24, 2024 at 04:43:13PM +0200, Valentin Schneider a écrit :
> > -/* Turn on heavyweight RCU tasks trace readers on idle/user entry. */
> > -static __always_inline void rcu_dynticks_task_trace_enter(void)
> > +/* Turn on heavyweight RCU tasks trace readers on kernel exit. */
> > +static __always_inline void rcu_task_trace_exit(void)
> 
> Before I proceed on this last one, a few questions for Paul and others:
> 
> 1) Why is rcu_dynticks_task_exit() not called while entering in NMI?
>    Does that mean that NMIs aren't RCU-Task read side critical sections?

Because Tasks RCU Rude handles that case currently.  So good catch,
because this might need adjustment when we get rid of Tasks RCU Rude.
And both rcu_dynticks_task_enter() and rcu_dynticks_task_exit() look safe
to invoke from NMI handlers.  Memory ordering needs checking, of course.

Except that on architectures defining CONFIG_ARCH_WANTS_NO_INSTR, Tasks
RCU should instead check the ct_kernel_enter_state(RCU_DYNTICKS_IDX)
state, right?  And on those architectures, I believe that
rcu_dynticks_task_enter() and rcu_dynticks_task_exit() can just be no-ops.
Or am I missing something here?

> 2) Looking further into CONFIG_TASKS_TRACE_RCU_READ_MB=y, it seems to
>    allow for uses of rcu_read_[un]lock_trace() while RCU is not watching
>    (EQS). Is it really a good idea to support that? Are we aware of any
>    such potential usecase?

I hope that in the longer term, there will be no reason to support this.
Right now, architectures not defining CONFIG_ARCH_WANTS_NO_INSTR must
support this because tracers really can attach probes where RCU is
not watching.

And even now, in architectures defining CONFIG_ARCH_WANTS_NO_INSTR, I
am not convinced that the early incoming and late outgoing CPU-hotplug
paths are handled correctly.  RCU is not watching them, but I am not so
sure that they are all marked noinstr as needed.

Or am I missing some implicit reason why this all works?

							Thanx, Paul

Re: RCU-Task[-Trace] VS EQS (was Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*())

[Frederic Weisbecker]

Le Tue, Jul 30, 2024 at 07:23:58AM -0700, Paul E. McKenney a écrit :
> On Thu, Jul 25, 2024 at 04:32:46PM +0200, Frederic Weisbecker wrote:
> > Le Wed, Jul 24, 2024 at 04:43:13PM +0200, Valentin Schneider a écrit :
> > > -/* Turn on heavyweight RCU tasks trace readers on idle/user entry. */
> > > -static __always_inline void rcu_dynticks_task_trace_enter(void)
> > > +/* Turn on heavyweight RCU tasks trace readers on kernel exit. */
> > > +static __always_inline void rcu_task_trace_exit(void)
> > 
> > Before I proceed on this last one, a few questions for Paul and others:
> > 
> > 1) Why is rcu_dynticks_task_exit() not called while entering in NMI?
> >    Does that mean that NMIs aren't RCU-Task read side critical sections?
> 
> Because Tasks RCU Rude handles that case currently.  So good catch,
> because this might need adjustment when we get rid of Tasks RCU Rude.
> And both rcu_dynticks_task_enter() and rcu_dynticks_task_exit() look safe
> to invoke from NMI handlers.  Memory ordering needs checking, of course.
> 
> Except that on architectures defining CONFIG_ARCH_WANTS_NO_INSTR, Tasks
> RCU should instead check the ct_kernel_enter_state(RCU_DYNTICKS_IDX)
> state, right?  And on those architectures, I believe that
> rcu_dynticks_task_enter() and rcu_dynticks_task_exit() can just be no-ops.
> Or am I missing something here?

I think rcu_dynticks_task_enter() and rcu_dynticks_task_exit() are
still needed anyway because the target task can migrate. So unless the rq is locked,
it's hard to match a stable task_cpu() with the corresponding RCU_DYNTICKS_IDX.

> 
> > 2) Looking further into CONFIG_TASKS_TRACE_RCU_READ_MB=y, it seems to
> >    allow for uses of rcu_read_[un]lock_trace() while RCU is not watching
> >    (EQS). Is it really a good idea to support that? Are we aware of any
> >    such potential usecase?
> 
> I hope that in the longer term, there will be no reason to support this.
> Right now, architectures not defining CONFIG_ARCH_WANTS_NO_INSTR must
> support this because tracers really can attach probes where RCU is
> not watching.
> 
> And even now, in architectures defining CONFIG_ARCH_WANTS_NO_INSTR, I
> am not convinced that the early incoming and late outgoing CPU-hotplug
> paths are handled correctly.  RCU is not watching them, but I am not so
> sure that they are all marked noinstr as needed.

Ok I see...

Thanks.

Re: RCU-Task[-Trace] VS EQS (was Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*())

[Paul E. McKenney]

On Wed, Jul 31, 2024 at 12:17:49AM +0200, Frederic Weisbecker wrote:
> Le Tue, Jul 30, 2024 at 07:23:58AM -0700, Paul E. McKenney a écrit :
> > On Thu, Jul 25, 2024 at 04:32:46PM +0200, Frederic Weisbecker wrote:
> > > Le Wed, Jul 24, 2024 at 04:43:13PM +0200, Valentin Schneider a écrit :
> > > > -/* Turn on heavyweight RCU tasks trace readers on idle/user entry. */
> > > > -static __always_inline void rcu_dynticks_task_trace_enter(void)
> > > > +/* Turn on heavyweight RCU tasks trace readers on kernel exit. */
> > > > +static __always_inline void rcu_task_trace_exit(void)
> > > 
> > > Before I proceed on this last one, a few questions for Paul and others:
> > > 
> > > 1) Why is rcu_dynticks_task_exit() not called while entering in NMI?
> > >    Does that mean that NMIs aren't RCU-Task read side critical sections?
> > 
> > Because Tasks RCU Rude handles that case currently.  So good catch,
> > because this might need adjustment when we get rid of Tasks RCU Rude.
> > And both rcu_dynticks_task_enter() and rcu_dynticks_task_exit() look safe
> > to invoke from NMI handlers.  Memory ordering needs checking, of course.
> > 
> > Except that on architectures defining CONFIG_ARCH_WANTS_NO_INSTR, Tasks
> > RCU should instead check the ct_kernel_enter_state(RCU_DYNTICKS_IDX)
> > state, right?  And on those architectures, I believe that
> > rcu_dynticks_task_enter() and rcu_dynticks_task_exit() can just be no-ops.
> > Or am I missing something here?
> 
> I think rcu_dynticks_task_enter() and rcu_dynticks_task_exit() are
> still needed anyway because the target task can migrate. So unless the rq is locked,
> it's hard to match a stable task_cpu() with the corresponding RCU_DYNTICKS_IDX.

Can it really migrate while in entry/exit or deep idle code?  Or am I
missing a trick here?

> > > 2) Looking further into CONFIG_TASKS_TRACE_RCU_READ_MB=y, it seems to
> > >    allow for uses of rcu_read_[un]lock_trace() while RCU is not watching
> > >    (EQS). Is it really a good idea to support that? Are we aware of any
> > >    such potential usecase?
> > 
> > I hope that in the longer term, there will be no reason to support this.
> > Right now, architectures not defining CONFIG_ARCH_WANTS_NO_INSTR must
> > support this because tracers really can attach probes where RCU is
> > not watching.
> > 
> > And even now, in architectures defining CONFIG_ARCH_WANTS_NO_INSTR, I
> > am not convinced that the early incoming and late outgoing CPU-hotplug
> > paths are handled correctly.  RCU is not watching them, but I am not so
> > sure that they are all marked noinstr as needed.
> 
> Ok I see...

If need be, the outgoing-CPU transition to RCU-not-watching could be
delayed into arch-specific code.  We already allow this for the incoming
transition.

							Thanx, Paul

Re: RCU-Task[-Trace] VS EQS (was Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*())

[Frederic Weisbecker]

Le Tue, Jul 30, 2024 at 03:39:44PM -0700, Paul E. McKenney a écrit :
> On Wed, Jul 31, 2024 at 12:17:49AM +0200, Frederic Weisbecker wrote:
> > Le Tue, Jul 30, 2024 at 07:23:58AM -0700, Paul E. McKenney a écrit :
> > > On Thu, Jul 25, 2024 at 04:32:46PM +0200, Frederic Weisbecker wrote:
> > > > Le Wed, Jul 24, 2024 at 04:43:13PM +0200, Valentin Schneider a écrit :
> > > > > -/* Turn on heavyweight RCU tasks trace readers on idle/user entry. */
> > > > > -static __always_inline void rcu_dynticks_task_trace_enter(void)
> > > > > +/* Turn on heavyweight RCU tasks trace readers on kernel exit. */
> > > > > +static __always_inline void rcu_task_trace_exit(void)
> > > > 
> > > > Before I proceed on this last one, a few questions for Paul and others:
> > > > 
> > > > 1) Why is rcu_dynticks_task_exit() not called while entering in NMI?
> > > >    Does that mean that NMIs aren't RCU-Task read side critical sections?
> > > 
> > > Because Tasks RCU Rude handles that case currently.  So good catch,
> > > because this might need adjustment when we get rid of Tasks RCU Rude.
> > > And both rcu_dynticks_task_enter() and rcu_dynticks_task_exit() look safe
> > > to invoke from NMI handlers.  Memory ordering needs checking, of course.
> > > 
> > > Except that on architectures defining CONFIG_ARCH_WANTS_NO_INSTR, Tasks
> > > RCU should instead check the ct_kernel_enter_state(RCU_DYNTICKS_IDX)
> > > state, right?  And on those architectures, I believe that
> > > rcu_dynticks_task_enter() and rcu_dynticks_task_exit() can just be no-ops.
> > > Or am I missing something here?
> > 
> > I think rcu_dynticks_task_enter() and rcu_dynticks_task_exit() are
> > still needed anyway because the target task can migrate. So unless the rq is locked,
> > it's hard to match a stable task_cpu() with the corresponding RCU_DYNTICKS_IDX.
> 
> Can it really migrate while in entry/exit or deep idle code?  Or am I
> missing a trick here?

No but it can migrate before or after EQS. So we need to handle situations like:

 == CPU 0 ==                      == CPU 1 ==
                                  // TASK A is on rq
                                  set_task_cpu(TASK A, 0)
// TASK B runs
ct_user_enter()
ct_user_exit()

//TASK A runs


It could be something like the following:


int rcu_tasks_nohz_full_holdout(struct task_struct *t)
{
	int cpu;
	int snap;

	cpu = task_cpu(t);

	/* Don't miss EQS exit if the task migrated out and in */
	smp_rmb()

	snap = ct_dynticks_cpu(cpu);
	if (snap & RCU_DYNTICKS_IDX)
		return true;

	/* Check if it's the actual task running */
	smp_rmb()

	if (rcu_dereference_raw(cpu_curr(cpu)) != t)
		return true;

	/* Make sure the task hasn't migrated in after the above EQS */
	smp_rmb()

	
	return ct_dynticks_cpu(cpu) != snap;
}

But there is still a risk that ct_dynticks wraps before the last test. So
we would need task_call_func() if task_cpu() is in nohz_full mode.

> 
> > > > 2) Looking further into CONFIG_TASKS_TRACE_RCU_READ_MB=y, it seems to
> > > >    allow for uses of rcu_read_[un]lock_trace() while RCU is not watching
> > > >    (EQS). Is it really a good idea to support that? Are we aware of any
> > > >    such potential usecase?
> > > 
> > > I hope that in the longer term, there will be no reason to support this.
> > > Right now, architectures not defining CONFIG_ARCH_WANTS_NO_INSTR must
> > > support this because tracers really can attach probes where RCU is
> > > not watching.
> > > 
> > > And even now, in architectures defining CONFIG_ARCH_WANTS_NO_INSTR, I
> > > am not convinced that the early incoming and late outgoing CPU-hotplug
> > > paths are handled correctly.  RCU is not watching them, but I am not so
> > > sure that they are all marked noinstr as needed.
> > 
> > Ok I see...
> 
> If need be, the outgoing-CPU transition to RCU-not-watching could be
> delayed into arch-specific code.  We already allow this for the incoming
> transition.

That's a lot of scary architectures code to handle :-)
And how do we determine which place is finally safe to stop watching?

Thanks.

> 
> 							Thanx, Paul

Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*()

[Frederic Weisbecker]

Le Wed, Jul 24, 2024 at 04:43:13PM +0200, Valentin Schneider a écrit :
> The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
> RCU_WATCHING, and the 'dynticks' prefix can be dropped without losing any
> meaning.
> 
> While at it, flip the suffixes of these helpers. We are not telling
> that we are entering dynticks mode from an RCU-task perspective anymore; we
> are telling that we are exiting RCU-tasks because we are in eqs mode.
> 
> Suggested-by: Frederic Weisbecker <frederic@kernel.org>
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>
> ---
>  kernel/context_tracking.c | 28 ++++++++++++++--------------
>  1 file changed, 14 insertions(+), 14 deletions(-)
> 
> diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
> index 8262f57a43636..1c16a7336360f 100644
> --- a/kernel/context_tracking.c
> +++ b/kernel/context_tracking.c
> @@ -38,24 +38,24 @@ EXPORT_SYMBOL_GPL(context_tracking);
>  #ifdef CONFIG_CONTEXT_TRACKING_IDLE
>  #define TPS(x)  tracepoint_string(x)
>  
> -/* Record the current task on dyntick-idle entry. */
> -static __always_inline void rcu_dynticks_task_enter(void)
> +/* Record the current task on exiting RCU-tasks (dyntick-idle entry). */
> +static __always_inline void rcu_task_exit(void)

So this makes sense.

>  {
>  #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL)
>  	WRITE_ONCE(current->rcu_tasks_idle_cpu, smp_processor_id());
>  #endif /* #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL) */
>  }
>  
> -/* Record no current task on dyntick-idle exit. */
> -static __always_inline void rcu_dynticks_task_exit(void)
> +/* Record no current task on entering RCU-tasks (dyntick-idle exit). */
> +static __always_inline void rcu_task_enter(void)

That too.

>  {
>  #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL)
>  	WRITE_ONCE(current->rcu_tasks_idle_cpu, -1);
>  #endif /* #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL) */
>  }
>  
> -/* Turn on heavyweight RCU tasks trace readers on idle/user entry. */
> -static __always_inline void rcu_dynticks_task_trace_enter(void)
> +/* Turn on heavyweight RCU tasks trace readers on kernel exit. */
> +static __always_inline void rcu_task_trace_exit(void)

But that eventually doesn't, because it's not about not wathing anymore from
an RCU-TASKS-TRACE perspective. It's actually about adding more heavyweight
ordering to track down RCU-TASKS-TRACE read side while traditional RCU is not
watching. Sorry for understanding it that late.

Oh well. So a more accurate name here would be rcu_task_trace_heavyweight_enter().

>  {
>  #ifdef CONFIG_TASKS_TRACE_RCU
>  	if (IS_ENABLED(CONFIG_TASKS_TRACE_RCU_READ_MB))
> @@ -63,8 +63,8 @@ static __always_inline void rcu_dynticks_task_trace_enter(void)
>  #endif /* #ifdef CONFIG_TASKS_TRACE_RCU */
>  }
>  
> -/* Turn off heavyweight RCU tasks trace readers on idle/user exit. */
> -static __always_inline void rcu_dynticks_task_trace_exit(void)
> +/* Turn off heavyweight RCU tasks trace readers on kernel entry. */
> +static __always_inline void rcu_task_trace_enter(void)

And rcu_task_trace_heavyweight_exit().

Thanks!

>  {
>  #ifdef CONFIG_TASKS_TRACE_RCU
>  	if (IS_ENABLED(CONFIG_TASKS_TRACE_RCU_READ_MB))
> @@ -87,7 +87,7 @@ static noinstr void ct_kernel_exit_state(int offset)
>  	 * critical sections, and we also must force ordering with the
>  	 * next idle sojourn.
>  	 */
> -	rcu_dynticks_task_trace_enter();  // Before ->dynticks update!
> +	rcu_task_trace_exit();  // Before CT state update!
>  	seq = ct_state_inc(offset);
>  	// RCU is no longer watching.  Better be in extended quiescent state!
>  	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && (seq & CT_RCU_WATCHING));
> @@ -109,7 +109,7 @@ static noinstr void ct_kernel_enter_state(int offset)
>  	 */
>  	seq = ct_state_inc(offset);
>  	// RCU is now watching.  Better not be in an extended quiescent state!
> -	rcu_dynticks_task_trace_exit();  // After ->dynticks update!
> +	rcu_task_trace_enter();  // After CT state update!
>  	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !(seq & CT_RCU_WATCHING));
>  }
>  
> @@ -149,7 +149,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
>  	// RCU is watching here ...
>  	ct_kernel_exit_state(offset);
>  	// ... but is no longer watching here.
> -	rcu_dynticks_task_enter();
> +	rcu_task_exit();
>  }
>  
>  /*
> @@ -173,7 +173,7 @@ static void noinstr ct_kernel_enter(bool user, int offset)
>  		ct->nesting++;
>  		return;
>  	}
> -	rcu_dynticks_task_exit();
> +	rcu_task_enter();
>  	// RCU is not watching here ...
>  	ct_kernel_enter_state(offset);
>  	// ... but is watching here.
> @@ -240,7 +240,7 @@ void noinstr ct_nmi_exit(void)
>  	// ... but is no longer watching here.
>  
>  	if (!in_nmi())
> -		rcu_dynticks_task_enter();
> +		rcu_task_exit();
>  }
>  
>  /**
> @@ -274,7 +274,7 @@ void noinstr ct_nmi_enter(void)
>  	if (rcu_dynticks_curr_cpu_in_eqs()) {
>  
>  		if (!in_nmi())
> -			rcu_dynticks_task_exit();
> +			rcu_task_enter();
>  
>  		// RCU is not watching here ...
>  		ct_kernel_enter_state(CT_RCU_WATCHING);
> -- 
> 2.43.0
> 

Re: RCU-Task[-Trace] VS EQS (was Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*())

[Paul E. McKenney]

On Wed, Jul 31, 2024 at 02:28:16PM +0200, Frederic Weisbecker wrote:
> Le Tue, Jul 30, 2024 at 03:39:44PM -0700, Paul E. McKenney a écrit :
> > On Wed, Jul 31, 2024 at 12:17:49AM +0200, Frederic Weisbecker wrote:
> > > Le Tue, Jul 30, 2024 at 07:23:58AM -0700, Paul E. McKenney a écrit :
> > > > On Thu, Jul 25, 2024 at 04:32:46PM +0200, Frederic Weisbecker wrote:
> > > > > Le Wed, Jul 24, 2024 at 04:43:13PM +0200, Valentin Schneider a écrit :
> > > > > > -/* Turn on heavyweight RCU tasks trace readers on idle/user entry. */
> > > > > > -static __always_inline void rcu_dynticks_task_trace_enter(void)
> > > > > > +/* Turn on heavyweight RCU tasks trace readers on kernel exit. */
> > > > > > +static __always_inline void rcu_task_trace_exit(void)
> > > > > 
> > > > > Before I proceed on this last one, a few questions for Paul and others:
> > > > > 
> > > > > 1) Why is rcu_dynticks_task_exit() not called while entering in NMI?
> > > > >    Does that mean that NMIs aren't RCU-Task read side critical sections?
> > > > 
> > > > Because Tasks RCU Rude handles that case currently.  So good catch,
> > > > because this might need adjustment when we get rid of Tasks RCU Rude.
> > > > And both rcu_dynticks_task_enter() and rcu_dynticks_task_exit() look safe
> > > > to invoke from NMI handlers.  Memory ordering needs checking, of course.
> > > > 
> > > > Except that on architectures defining CONFIG_ARCH_WANTS_NO_INSTR, Tasks
> > > > RCU should instead check the ct_kernel_enter_state(RCU_DYNTICKS_IDX)
> > > > state, right?  And on those architectures, I believe that
> > > > rcu_dynticks_task_enter() and rcu_dynticks_task_exit() can just be no-ops.
> > > > Or am I missing something here?
> > > 
> > > I think rcu_dynticks_task_enter() and rcu_dynticks_task_exit() are
> > > still needed anyway because the target task can migrate. So unless the rq is locked,
> > > it's hard to match a stable task_cpu() with the corresponding RCU_DYNTICKS_IDX.
> > 
> > Can it really migrate while in entry/exit or deep idle code?  Or am I
> > missing a trick here?
> 
> No but it can migrate before or after EQS. So we need to handle situations like:
> 
>  == CPU 0 ==                      == CPU 1 ==
>                                   // TASK A is on rq
>                                   set_task_cpu(TASK A, 0)
> // TASK B runs
> ct_user_enter()
> ct_user_exit()
> 
> //TASK A runs
> 
> 
> It could be something like the following:
> 
> 
> int rcu_tasks_nohz_full_holdout(struct task_struct *t)
> {
> 	int cpu;
> 	int snap;
> 
> 	cpu = task_cpu(t);
> 
> 	/* Don't miss EQS exit if the task migrated out and in */
> 	smp_rmb()
> 
> 	snap = ct_dynticks_cpu(cpu);
> 	if (snap & RCU_DYNTICKS_IDX)
> 		return true;
> 
> 	/* Check if it's the actual task running */
> 	smp_rmb()
> 
> 	if (rcu_dereference_raw(cpu_curr(cpu)) != t)
> 		return true;
> 
> 	/* Make sure the task hasn't migrated in after the above EQS */
> 	smp_rmb()
> 
> 	
> 	return ct_dynticks_cpu(cpu) != snap;
> }
> 
> But there is still a risk that ct_dynticks wraps before the last test. So
> we would need task_call_func() if task_cpu() is in nohz_full mode.

Good point, migration just before or just after can look much the same
as migrating during..

> > > > > 2) Looking further into CONFIG_TASKS_TRACE_RCU_READ_MB=y, it seems to
> > > > >    allow for uses of rcu_read_[un]lock_trace() while RCU is not watching
> > > > >    (EQS). Is it really a good idea to support that? Are we aware of any
> > > > >    such potential usecase?
> > > > 
> > > > I hope that in the longer term, there will be no reason to support this.
> > > > Right now, architectures not defining CONFIG_ARCH_WANTS_NO_INSTR must
> > > > support this because tracers really can attach probes where RCU is
> > > > not watching.
> > > > 
> > > > And even now, in architectures defining CONFIG_ARCH_WANTS_NO_INSTR, I
> > > > am not convinced that the early incoming and late outgoing CPU-hotplug
> > > > paths are handled correctly.  RCU is not watching them, but I am not so
> > > > sure that they are all marked noinstr as needed.
> > > 
> > > Ok I see...
> > 
> > If need be, the outgoing-CPU transition to RCU-not-watching could be
> > delayed into arch-specific code.  We already allow this for the incoming
> > transition.
> 
> That's a lot of scary architectures code to handle :-)
> And how do we determine which place is finally safe to stop watching?

Huh.  One reason for the current smp_call_function_single() in
cpuhp_report_idle_dead() was some ARM32 CPUs that shut down caching on
their way out.  this made it impossible to use shared-variable-based
CPU-dead notification.  I wonder if Arnd's deprecation schedule
for ARM32-based platforms will allow us to go back to shared-memory
notification, which might make this sort of thing easier.

							Thanx, Paul

Re: RCU-Task[-Trace] VS EQS (was Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*())

[Peter Zijlstra]

On Fri, Aug 02, 2024 at 09:32:08PM -0700, Paul E. McKenney wrote:

> Huh.  One reason for the current smp_call_function_single() in
> cpuhp_report_idle_dead() was some ARM32 CPUs that shut down caching on
> their way out.  this made it impossible to use shared-variable-based
> CPU-dead notification.  I wonder if Arnd's deprecation schedule
> for ARM32-based platforms will allow us to go back to shared-memory
> notification, which might make this sort of thing easier.

All those idle paths should be doing ct_cpuidle_enter(), which includes
telling RCU the CPU is going idle, no?

I cleaned all that up a while back.

Re: RCU-Task[-Trace] VS EQS (was Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*())

[Frederic Weisbecker]

Le Mon, Aug 05, 2024 at 11:01:22AM +0200, Peter Zijlstra a écrit :
> On Fri, Aug 02, 2024 at 09:32:08PM -0700, Paul E. McKenney wrote:
> 
> > Huh.  One reason for the current smp_call_function_single() in
> > cpuhp_report_idle_dead() was some ARM32 CPUs that shut down caching on
> > their way out.  this made it impossible to use shared-variable-based
> > CPU-dead notification.  I wonder if Arnd's deprecation schedule
> > for ARM32-based platforms will allow us to go back to shared-memory
> > notification, which might make this sort of thing easier.
> 
> All those idle paths should be doing ct_cpuidle_enter(), which includes
> telling RCU the CPU is going idle, no?
> 
> I cleaned all that up a while back.

On every architectures? That would be a good news, as we could then remove
the CONFIG_TASKS_TRACE_RCU_READ_MB dance.

There is still some vulnerable path on CPU hotplug down path once
cpuhp_complete_idle_dead() is executed. The dead CPU may still call a bunch
of general purpose functions (the exit path of smp_call_function_single()
and the exit of cpuhp_report_idle_dead()). And this will become worse once
we remove TASK-RUDE and rely only on rcutree_report_cpu_dead().

Thanks.

Re: RCU-Task[-Trace] VS EQS (was Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*())

[Frederic Weisbecker]

Le Fri, Aug 02, 2024 at 09:32:08PM -0700, Paul E. McKenney a écrit :
> On Wed, Jul 31, 2024 at 02:28:16PM +0200, Frederic Weisbecker wrote:
> > Le Tue, Jul 30, 2024 at 03:39:44PM -0700, Paul E. McKenney a écrit :
> > > On Wed, Jul 31, 2024 at 12:17:49AM +0200, Frederic Weisbecker wrote:
> > > > Le Tue, Jul 30, 2024 at 07:23:58AM -0700, Paul E. McKenney a écrit :
> > > > > On Thu, Jul 25, 2024 at 04:32:46PM +0200, Frederic Weisbecker wrote:
> > > > > > Le Wed, Jul 24, 2024 at 04:43:13PM +0200, Valentin Schneider a écrit :
> > > > > > > -/* Turn on heavyweight RCU tasks trace readers on idle/user entry. */
> > > > > > > -static __always_inline void rcu_dynticks_task_trace_enter(void)
> > > > > > > +/* Turn on heavyweight RCU tasks trace readers on kernel exit. */
> > > > > > > +static __always_inline void rcu_task_trace_exit(void)
> > > > > > 
> > > > > > Before I proceed on this last one, a few questions for Paul and others:
> > > > > > 
> > > > > > 1) Why is rcu_dynticks_task_exit() not called while entering in NMI?
> > > > > >    Does that mean that NMIs aren't RCU-Task read side critical sections?
> > > > > 
> > > > > Because Tasks RCU Rude handles that case currently.  So good catch,
> > > > > because this might need adjustment when we get rid of Tasks RCU Rude.
> > > > > And both rcu_dynticks_task_enter() and rcu_dynticks_task_exit() look safe
> > > > > to invoke from NMI handlers.  Memory ordering needs checking, of course.
> > > > > 
> > > > > Except that on architectures defining CONFIG_ARCH_WANTS_NO_INSTR, Tasks
> > > > > RCU should instead check the ct_kernel_enter_state(RCU_DYNTICKS_IDX)
> > > > > state, right?  And on those architectures, I believe that
> > > > > rcu_dynticks_task_enter() and rcu_dynticks_task_exit() can just be no-ops.
> > > > > Or am I missing something here?
> > > > 
> > > > I think rcu_dynticks_task_enter() and rcu_dynticks_task_exit() are
> > > > still needed anyway because the target task can migrate. So unless the rq is locked,
> > > > it's hard to match a stable task_cpu() with the corresponding RCU_DYNTICKS_IDX.
> > > 
> > > Can it really migrate while in entry/exit or deep idle code?  Or am I
> > > missing a trick here?
> > 
> > No but it can migrate before or after EQS. So we need to handle situations like:
> > 
> >  == CPU 0 ==                      == CPU 1 ==
> >                                   // TASK A is on rq
> >                                   set_task_cpu(TASK A, 0)
> > // TASK B runs
> > ct_user_enter()
> > ct_user_exit()
> > 
> > //TASK A runs
> > 
> > 
> > It could be something like the following:
> > 
> > 
> > int rcu_tasks_nohz_full_holdout(struct task_struct *t)
> > {
> > 	int cpu;
> > 	int snap;
> > 
> > 	cpu = task_cpu(t);
> > 
> > 	/* Don't miss EQS exit if the task migrated out and in */
> > 	smp_rmb()
> > 
> > 	snap = ct_dynticks_cpu(cpu);
> > 	if (snap & RCU_DYNTICKS_IDX)
> > 		return true;
> > 
> > 	/* Check if it's the actual task running */
> > 	smp_rmb()
> > 
> > 	if (rcu_dereference_raw(cpu_curr(cpu)) != t)
> > 		return true;
> > 
> > 	/* Make sure the task hasn't migrated in after the above EQS */
> > 	smp_rmb()
> > 
> > 	
> > 	return ct_dynticks_cpu(cpu) != snap;
> > }
> > 
> > But there is still a risk that ct_dynticks wraps before the last test. So
> > we would need task_call_func() if task_cpu() is in nohz_full mode.
> 
> Good point, migration just before or just after can look much the same
> as migrating during..
> 
> > > > > > 2) Looking further into CONFIG_TASKS_TRACE_RCU_READ_MB=y, it seems to
> > > > > >    allow for uses of rcu_read_[un]lock_trace() while RCU is not watching
> > > > > >    (EQS). Is it really a good idea to support that? Are we aware of any
> > > > > >    such potential usecase?
> > > > > 
> > > > > I hope that in the longer term, there will be no reason to support this.
> > > > > Right now, architectures not defining CONFIG_ARCH_WANTS_NO_INSTR must
> > > > > support this because tracers really can attach probes where RCU is
> > > > > not watching.
> > > > > 
> > > > > And even now, in architectures defining CONFIG_ARCH_WANTS_NO_INSTR, I
> > > > > am not convinced that the early incoming and late outgoing CPU-hotplug
> > > > > paths are handled correctly.  RCU is not watching them, but I am not so
> > > > > sure that they are all marked noinstr as needed.
> > > > 
> > > > Ok I see...
> > > 
> > > If need be, the outgoing-CPU transition to RCU-not-watching could be
> > > delayed into arch-specific code.  We already allow this for the incoming
> > > transition.
> > 
> > That's a lot of scary architectures code to handle :-)
> > And how do we determine which place is finally safe to stop watching?
> 
> Huh.  One reason for the current smp_call_function_single() in
> cpuhp_report_idle_dead() was some ARM32 CPUs that shut down caching on
> their way out.  this made it impossible to use shared-variable-based
> CPU-dead notification.  I wonder if Arnd's deprecation schedule
> for ARM32-based platforms will allow us to go back to shared-memory
> notification, which might make this sort of thing easier.

git blame retrieved that:

71f87b2fc64c (cpu/hotplug: Plug death reporting race)

For the reason why using smp_call_function_single():

    """
    We cant call complete after rcu_report_dead(), so instead of going back to
    busy polling, simply issue a function call to do the completion.
    """

This doesn't state exactly why as I don't see obvious use of RCU there. But
RCU-TASK-TRACE definetly doesn't want to stop watching while complete() is
called.

Another thing to consider is that rcutree_migrate_callbacks() may execute
concurrently with rcutree_report_cpu_dead() otherwise. This might work ok,
despite the WARN_ON_ONCE(rcu_rdp_cpu_online(rdp)) in
rcutree_migrate_callbacks(). But that's doesn't make it more a good idea :-)

As for the reason why calling complete() _after_ rcutree_report_cpu_dead():

    """
    Paul noticed that the conversion of the death reporting introduced a race
    where the outgoing cpu might be delayed after waking the controll processor,
    so it might not be able to call rcu_report_dead() before being physically
    removed, leading to RCU stalls.
    """

I'm lacking details but perhaps the call to __cpu_die() by the control CPU on
some archs may kill the CPU before it used to reach rcu_report_dead() ? No idea.
But we need to know if __cpu_die() can kill the CPU before it reaches
arch_cpu_idle_dead(). If it's not the case, then we can do something like this:

diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index 24b1e1143260..fb5b866c2fb3 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -31,7 +31,6 @@ DEFINE_PER_CPU(struct context_tracking, context_tracking) = {
 	.dynticks_nesting = 1,
 	.dynticks_nmi_nesting = DYNTICK_IRQ_NONIDLE,
 #endif
-	.state = ATOMIC_INIT(RCU_DYNTICKS_IDX),
 };
 EXPORT_SYMBOL_GPL(context_tracking);
 
diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
index 6e78d071beb5..7ebaf04af37a 100644
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -307,6 +307,7 @@ static void do_idle(void)
 
 		if (cpu_is_offline(cpu)) {
 			cpuhp_report_idle_dead();
+			ct_state_inc(RCU_DYNTICKS_IDX);
 			arch_cpu_idle_dead();
 		}
 
This mirrors rcutree_report_cpu_starting() -> rcu_dynticks_eqs_online()
And then we can make RCU-TASKS not worry about rcu_cpu_online() anymore
and only care about the context tracking.

The CPU up path looks saner with rcutree_report_cpu_starting() called early
enough on sane archs... Otherwise it's easy to fix case by case.

Thoughts?

Re: RCU-Task[-Trace] VS EQS (was Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*())

[Paul E. McKenney]

On Mon, Aug 05, 2024 at 03:26:38PM +0200, Frederic Weisbecker wrote:
> Le Fri, Aug 02, 2024 at 09:32:08PM -0700, Paul E. McKenney a écrit :
> > On Wed, Jul 31, 2024 at 02:28:16PM +0200, Frederic Weisbecker wrote:
> > > Le Tue, Jul 30, 2024 at 03:39:44PM -0700, Paul E. McKenney a écrit :
> > > > On Wed, Jul 31, 2024 at 12:17:49AM +0200, Frederic Weisbecker wrote:
> > > > > Le Tue, Jul 30, 2024 at 07:23:58AM -0700, Paul E. McKenney a écrit :
> > > > > > On Thu, Jul 25, 2024 at 04:32:46PM +0200, Frederic Weisbecker wrote:
> > > > > > > Le Wed, Jul 24, 2024 at 04:43:13PM +0200, Valentin Schneider a écrit :
> > > > > > > > -/* Turn on heavyweight RCU tasks trace readers on idle/user entry. */
> > > > > > > > -static __always_inline void rcu_dynticks_task_trace_enter(void)
> > > > > > > > +/* Turn on heavyweight RCU tasks trace readers on kernel exit. */
> > > > > > > > +static __always_inline void rcu_task_trace_exit(void)
> > > > > > > 
> > > > > > > Before I proceed on this last one, a few questions for Paul and others:
> > > > > > > 
> > > > > > > 1) Why is rcu_dynticks_task_exit() not called while entering in NMI?
> > > > > > >    Does that mean that NMIs aren't RCU-Task read side critical sections?
> > > > > > 
> > > > > > Because Tasks RCU Rude handles that case currently.  So good catch,
> > > > > > because this might need adjustment when we get rid of Tasks RCU Rude.
> > > > > > And both rcu_dynticks_task_enter() and rcu_dynticks_task_exit() look safe
> > > > > > to invoke from NMI handlers.  Memory ordering needs checking, of course.
> > > > > > 
> > > > > > Except that on architectures defining CONFIG_ARCH_WANTS_NO_INSTR, Tasks
> > > > > > RCU should instead check the ct_kernel_enter_state(RCU_DYNTICKS_IDX)
> > > > > > state, right?  And on those architectures, I believe that
> > > > > > rcu_dynticks_task_enter() and rcu_dynticks_task_exit() can just be no-ops.
> > > > > > Or am I missing something here?
> > > > > 
> > > > > I think rcu_dynticks_task_enter() and rcu_dynticks_task_exit() are
> > > > > still needed anyway because the target task can migrate. So unless the rq is locked,
> > > > > it's hard to match a stable task_cpu() with the corresponding RCU_DYNTICKS_IDX.
> > > > 
> > > > Can it really migrate while in entry/exit or deep idle code?  Or am I
> > > > missing a trick here?
> > > 
> > > No but it can migrate before or after EQS. So we need to handle situations like:
> > > 
> > >  == CPU 0 ==                      == CPU 1 ==
> > >                                   // TASK A is on rq
> > >                                   set_task_cpu(TASK A, 0)
> > > // TASK B runs
> > > ct_user_enter()
> > > ct_user_exit()
> > > 
> > > //TASK A runs
> > > 
> > > 
> > > It could be something like the following:
> > > 
> > > 
> > > int rcu_tasks_nohz_full_holdout(struct task_struct *t)
> > > {
> > > 	int cpu;
> > > 	int snap;
> > > 
> > > 	cpu = task_cpu(t);
> > > 
> > > 	/* Don't miss EQS exit if the task migrated out and in */
> > > 	smp_rmb()
> > > 
> > > 	snap = ct_dynticks_cpu(cpu);
> > > 	if (snap & RCU_DYNTICKS_IDX)
> > > 		return true;
> > > 
> > > 	/* Check if it's the actual task running */
> > > 	smp_rmb()
> > > 
> > > 	if (rcu_dereference_raw(cpu_curr(cpu)) != t)
> > > 		return true;
> > > 
> > > 	/* Make sure the task hasn't migrated in after the above EQS */
> > > 	smp_rmb()
> > > 
> > > 	
> > > 	return ct_dynticks_cpu(cpu) != snap;
> > > }
> > > 
> > > But there is still a risk that ct_dynticks wraps before the last test. So
> > > we would need task_call_func() if task_cpu() is in nohz_full mode.
> > 
> > Good point, migration just before or just after can look much the same
> > as migrating during..
> > 
> > > > > > > 2) Looking further into CONFIG_TASKS_TRACE_RCU_READ_MB=y, it seems to
> > > > > > >    allow for uses of rcu_read_[un]lock_trace() while RCU is not watching
> > > > > > >    (EQS). Is it really a good idea to support that? Are we aware of any
> > > > > > >    such potential usecase?
> > > > > > 
> > > > > > I hope that in the longer term, there will be no reason to support this.
> > > > > > Right now, architectures not defining CONFIG_ARCH_WANTS_NO_INSTR must
> > > > > > support this because tracers really can attach probes where RCU is
> > > > > > not watching.
> > > > > > 
> > > > > > And even now, in architectures defining CONFIG_ARCH_WANTS_NO_INSTR, I
> > > > > > am not convinced that the early incoming and late outgoing CPU-hotplug
> > > > > > paths are handled correctly.  RCU is not watching them, but I am not so
> > > > > > sure that they are all marked noinstr as needed.
> > > > > 
> > > > > Ok I see...
> > > > 
> > > > If need be, the outgoing-CPU transition to RCU-not-watching could be
> > > > delayed into arch-specific code.  We already allow this for the incoming
> > > > transition.
> > > 
> > > That's a lot of scary architectures code to handle :-)
> > > And how do we determine which place is finally safe to stop watching?
> > 
> > Huh.  One reason for the current smp_call_function_single() in
> > cpuhp_report_idle_dead() was some ARM32 CPUs that shut down caching on
> > their way out.  this made it impossible to use shared-variable-based
> > CPU-dead notification.  I wonder if Arnd's deprecation schedule
> > for ARM32-based platforms will allow us to go back to shared-memory
> > notification, which might make this sort of thing easier.
> 
> git blame retrieved that:
> 
> 71f87b2fc64c (cpu/hotplug: Plug death reporting race)
> 
> For the reason why using smp_call_function_single():
> 
>     """
>     We cant call complete after rcu_report_dead(), so instead of going back to
>     busy polling, simply issue a function call to do the completion.
>     """
> 
> This doesn't state exactly why as I don't see obvious use of RCU there. But
> RCU-TASK-TRACE definetly doesn't want to stop watching while complete() is
> called.

With lockdep, the lock acquisition and release in complete_with_flags()
would splat, though this could be handled by using a low-level lock
acquisition and release that didn't mess with lockdep.  With the resulting
risks of missed deadlock detection, of course!

With tracing enabled, there is the trace_sched_waking() within
try_to_wake_up() along with the trace_sched_wakeup() within
ttwu_do_wakeup().  And more beyond that, but I am lazy this morning.  ;-)

> Another thing to consider is that rcutree_migrate_callbacks() may execute
> concurrently with rcutree_report_cpu_dead() otherwise. This might work ok,
> despite the WARN_ON_ONCE(rcu_rdp_cpu_online(rdp)) in
> rcutree_migrate_callbacks(). But that's doesn't make it more a good idea :-)
> 
> As for the reason why calling complete() _after_ rcutree_report_cpu_dead():
> 
>     """
>     Paul noticed that the conversion of the death reporting introduced a race
>     where the outgoing cpu might be delayed after waking the controll processor,
>     so it might not be able to call rcu_report_dead() before being physically
>     removed, leading to RCU stalls.
>     """

There were also ARM32 splats that have aged out of my mental cache.

> I'm lacking details but perhaps the call to __cpu_die() by the control CPU on
> some archs may kill the CPU before it used to reach rcu_report_dead() ? No idea.
> But we need to know if __cpu_die() can kill the CPU before it reaches
> arch_cpu_idle_dead(). If it's not the case, then we can do something like this:
> 
> diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
> index 24b1e1143260..fb5b866c2fb3 100644
> --- a/kernel/context_tracking.c
> +++ b/kernel/context_tracking.c
> @@ -31,7 +31,6 @@ DEFINE_PER_CPU(struct context_tracking, context_tracking) = {
>  	.dynticks_nesting = 1,
>  	.dynticks_nmi_nesting = DYNTICK_IRQ_NONIDLE,
>  #endif
> -	.state = ATOMIC_INIT(RCU_DYNTICKS_IDX),

Don't you still need the initial state for the boot CPU?

>  };
>  EXPORT_SYMBOL_GPL(context_tracking);
>  
> diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
> index 6e78d071beb5..7ebaf04af37a 100644
> --- a/kernel/sched/idle.c
> +++ b/kernel/sched/idle.c
> @@ -307,6 +307,7 @@ static void do_idle(void)
>  
>  		if (cpu_is_offline(cpu)) {
>  			cpuhp_report_idle_dead();
> +			ct_state_inc(RCU_DYNTICKS_IDX);
>  			arch_cpu_idle_dead();
>  		}
>  
> This mirrors rcutree_report_cpu_starting() -> rcu_dynticks_eqs_online()
> And then we can make RCU-TASKS not worry about rcu_cpu_online() anymore
> and only care about the context tracking.

For that to work, we need all architectures to be OK for Tasks Rude RCU
being removed, right?  Plus that would make all of arch_cpu_idle_dead()
untraceable.  Is that a good thing?

> The CPU up path looks saner with rcutree_report_cpu_starting() called early
> enough on sane archs... Otherwise it's easy to fix case by case.
> 
> Thoughts?

Not entirly sure yet, but I agree some fix would be good!

							Thanx, Paul

Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*()

[Neeraj Upadhyay]

On Wed, Jul 31, 2024 at 06:07:32PM +0200, Frederic Weisbecker wrote:
> Le Wed, Jul 24, 2024 at 04:43:13PM +0200, Valentin Schneider a écrit :
> > The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
> > RCU_WATCHING, and the 'dynticks' prefix can be dropped without losing any
> > meaning.
> > 
> > While at it, flip the suffixes of these helpers. We are not telling
> > that we are entering dynticks mode from an RCU-task perspective anymore; we
> > are telling that we are exiting RCU-tasks because we are in eqs mode.
> > 
> > Suggested-by: Frederic Weisbecker <frederic@kernel.org>
> > Signed-off-by: Valentin Schneider <vschneid@redhat.com>
> > ---
> >  kernel/context_tracking.c | 28 ++++++++++++++--------------
> >  1 file changed, 14 insertions(+), 14 deletions(-)
> > 
> > diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
> > index 8262f57a43636..1c16a7336360f 100644
> > --- a/kernel/context_tracking.c
> > +++ b/kernel/context_tracking.c
> > @@ -38,24 +38,24 @@ EXPORT_SYMBOL_GPL(context_tracking);
> >  #ifdef CONFIG_CONTEXT_TRACKING_IDLE
> >  #define TPS(x)  tracepoint_string(x)
> >  
> > -/* Record the current task on dyntick-idle entry. */
> > -static __always_inline void rcu_dynticks_task_enter(void)
> > +/* Record the current task on exiting RCU-tasks (dyntick-idle entry). */
> > +static __always_inline void rcu_task_exit(void)
> 
> So this makes sense.
> 
> >  {
> >  #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL)
> >  	WRITE_ONCE(current->rcu_tasks_idle_cpu, smp_processor_id());
> >  #endif /* #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL) */
> >  }
> >  
> > -/* Record no current task on dyntick-idle exit. */
> > -static __always_inline void rcu_dynticks_task_exit(void)
> > +/* Record no current task on entering RCU-tasks (dyntick-idle exit). */
> > +static __always_inline void rcu_task_enter(void)
> 
> That too.
> 
> >  {
> >  #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL)
> >  	WRITE_ONCE(current->rcu_tasks_idle_cpu, -1);
> >  #endif /* #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL) */
> >  }
> >  
> > -/* Turn on heavyweight RCU tasks trace readers on idle/user entry. */
> > -static __always_inline void rcu_dynticks_task_trace_enter(void)
> > +/* Turn on heavyweight RCU tasks trace readers on kernel exit. */
> > +static __always_inline void rcu_task_trace_exit(void)
> 
> But that eventually doesn't, because it's not about not wathing anymore from
> an RCU-TASKS-TRACE perspective. It's actually about adding more heavyweight
> ordering to track down RCU-TASKS-TRACE read side while traditional RCU is not
> watching. Sorry for understanding it that late.
> 
> Oh well. So a more accurate name here would be rcu_task_trace_heavyweight_enter().
> 
> >  {
> >  #ifdef CONFIG_TASKS_TRACE_RCU
> >  	if (IS_ENABLED(CONFIG_TASKS_TRACE_RCU_READ_MB))
> > @@ -63,8 +63,8 @@ static __always_inline void rcu_dynticks_task_trace_enter(void)
> >  #endif /* #ifdef CONFIG_TASKS_TRACE_RCU */
> >  }
> >  
> > -/* Turn off heavyweight RCU tasks trace readers on idle/user exit. */
> > -static __always_inline void rcu_dynticks_task_trace_exit(void)
> > +/* Turn off heavyweight RCU tasks trace readers on kernel entry. */
> > +static __always_inline void rcu_task_trace_enter(void)
> 
> And rcu_task_trace_heavyweight_exit().
> 

I have updated it here [1]. Please let me know if something looks
incorrect.


[1] https://git.kernel.org/pub/scm/linux/kernel/git/neeraj.upadhyay/linux-rcu.git/commit/?h=next.14.08.24a&id=cfc22b9f1572b137dd9f36da831dd7b69c9fe352

- Neeraj

> Thanks!
> 
> >  {
> >  #ifdef CONFIG_TASKS_TRACE_RCU
> >  	if (IS_ENABLED(CONFIG_TASKS_TRACE_RCU_READ_MB))
> > @@ -87,7 +87,7 @@ static noinstr void ct_kernel_exit_state(int offset)
> >  	 * critical sections, and we also must force ordering with the
> >  	 * next idle sojourn.
> >  	 */
> > -	rcu_dynticks_task_trace_enter();  // Before ->dynticks update!
> > +	rcu_task_trace_exit();  // Before CT state update!
> >  	seq = ct_state_inc(offset);
> >  	// RCU is no longer watching.  Better be in extended quiescent state!
> >  	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && (seq & CT_RCU_WATCHING));
> > @@ -109,7 +109,7 @@ static noinstr void ct_kernel_enter_state(int offset)
> >  	 */
> >  	seq = ct_state_inc(offset);
> >  	// RCU is now watching.  Better not be in an extended quiescent state!
> > -	rcu_dynticks_task_trace_exit();  // After ->dynticks update!
> > +	rcu_task_trace_enter();  // After CT state update!
> >  	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !(seq & CT_RCU_WATCHING));
> >  }
> >  
> > @@ -149,7 +149,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
> >  	// RCU is watching here ...
> >  	ct_kernel_exit_state(offset);
> >  	// ... but is no longer watching here.
> > -	rcu_dynticks_task_enter();
> > +	rcu_task_exit();
> >  }
> >  
> >  /*
> > @@ -173,7 +173,7 @@ static void noinstr ct_kernel_enter(bool user, int offset)
> >  		ct->nesting++;
> >  		return;
> >  	}
> > -	rcu_dynticks_task_exit();
> > +	rcu_task_enter();
> >  	// RCU is not watching here ...
> >  	ct_kernel_enter_state(offset);
> >  	// ... but is watching here.
> > @@ -240,7 +240,7 @@ void noinstr ct_nmi_exit(void)
> >  	// ... but is no longer watching here.
> >  
> >  	if (!in_nmi())
> > -		rcu_dynticks_task_enter();
> > +		rcu_task_exit();
> >  }
> >  
> >  /**
> > @@ -274,7 +274,7 @@ void noinstr ct_nmi_enter(void)
> >  	if (rcu_dynticks_curr_cpu_in_eqs()) {
> >  
> >  		if (!in_nmi())
> > -			rcu_dynticks_task_exit();
> > +			rcu_task_enter();
> >  
> >  		// RCU is not watching here ...
> >  		ct_kernel_enter_state(CT_RCU_WATCHING);
> > -- 
> > 2.43.0
> > 

Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*()

[Frederic Weisbecker]

Le Wed, Aug 14, 2024 at 05:36:55PM +0530, Neeraj Upadhyay a écrit :
> On Wed, Jul 31, 2024 at 06:07:32PM +0200, Frederic Weisbecker wrote:
> > Le Wed, Jul 24, 2024 at 04:43:13PM +0200, Valentin Schneider a écrit :
> > > The context_tracking.state RCU_DYNTICKS subvariable has been renamed to
> > > RCU_WATCHING, and the 'dynticks' prefix can be dropped without losing any
> > > meaning.
> > > 
> > > While at it, flip the suffixes of these helpers. We are not telling
> > > that we are entering dynticks mode from an RCU-task perspective anymore; we
> > > are telling that we are exiting RCU-tasks because we are in eqs mode.
> > > 
> > > Suggested-by: Frederic Weisbecker <frederic@kernel.org>
> > > Signed-off-by: Valentin Schneider <vschneid@redhat.com>
> > > ---
> > >  kernel/context_tracking.c | 28 ++++++++++++++--------------
> > >  1 file changed, 14 insertions(+), 14 deletions(-)
> > > 
> > > diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
> > > index 8262f57a43636..1c16a7336360f 100644
> > > --- a/kernel/context_tracking.c
> > > +++ b/kernel/context_tracking.c
> > > @@ -38,24 +38,24 @@ EXPORT_SYMBOL_GPL(context_tracking);
> > >  #ifdef CONFIG_CONTEXT_TRACKING_IDLE
> > >  #define TPS(x)  tracepoint_string(x)
> > >  
> > > -/* Record the current task on dyntick-idle entry. */
> > > -static __always_inline void rcu_dynticks_task_enter(void)
> > > +/* Record the current task on exiting RCU-tasks (dyntick-idle entry). */
> > > +static __always_inline void rcu_task_exit(void)
> > 
> > So this makes sense.
> > 
> > >  {
> > >  #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL)
> > >  	WRITE_ONCE(current->rcu_tasks_idle_cpu, smp_processor_id());
> > >  #endif /* #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL) */
> > >  }
> > >  
> > > -/* Record no current task on dyntick-idle exit. */
> > > -static __always_inline void rcu_dynticks_task_exit(void)
> > > +/* Record no current task on entering RCU-tasks (dyntick-idle exit). */
> > > +static __always_inline void rcu_task_enter(void)
> > 
> > That too.
> > 
> > >  {
> > >  #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL)
> > >  	WRITE_ONCE(current->rcu_tasks_idle_cpu, -1);
> > >  #endif /* #if defined(CONFIG_TASKS_RCU) && defined(CONFIG_NO_HZ_FULL) */
> > >  }
> > >  
> > > -/* Turn on heavyweight RCU tasks trace readers on idle/user entry. */
> > > -static __always_inline void rcu_dynticks_task_trace_enter(void)
> > > +/* Turn on heavyweight RCU tasks trace readers on kernel exit. */
> > > +static __always_inline void rcu_task_trace_exit(void)
> > 
> > But that eventually doesn't, because it's not about not wathing anymore from
> > an RCU-TASKS-TRACE perspective. It's actually about adding more heavyweight
> > ordering to track down RCU-TASKS-TRACE read side while traditional RCU is not
> > watching. Sorry for understanding it that late.
> > 
> > Oh well. So a more accurate name here would be rcu_task_trace_heavyweight_enter().
> > 
> > >  {
> > >  #ifdef CONFIG_TASKS_TRACE_RCU
> > >  	if (IS_ENABLED(CONFIG_TASKS_TRACE_RCU_READ_MB))
> > > @@ -63,8 +63,8 @@ static __always_inline void rcu_dynticks_task_trace_enter(void)
> > >  #endif /* #ifdef CONFIG_TASKS_TRACE_RCU */
> > >  }
> > >  
> > > -/* Turn off heavyweight RCU tasks trace readers on idle/user exit. */
> > > -static __always_inline void rcu_dynticks_task_trace_exit(void)
> > > +/* Turn off heavyweight RCU tasks trace readers on kernel entry. */
> > > +static __always_inline void rcu_task_trace_enter(void)
> > 
> > And rcu_task_trace_heavyweight_exit().
> > 
> 
> I have updated it here [1]. Please let me know if something looks
> incorrect.
> 
> 
> [1]
> https://git.kernel.org/pub/scm/linux/kernel/git/neeraj.upadhyay/linux-rcu.git/commit/?h=next.14.08.24a&id=cfc22b9f1572b137dd9f36da831dd7b69c9fe352

Looks good!

Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

Thanks!

Re: [PATCH v3 13/25] context_tracking, rcu: Rename rcu_dynticks_task*() into rcu_task*()

[Valentin Schneider]

On 14/08/24 17:36, Neeraj Upadhyay wrote:
>
> I have updated it here [1]. Please let me know if something looks
> incorrect.
>
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/neeraj.upadhyay/linux-rcu.git/commit/?h=next.14.08.24a&id=cfc22b9f1572b137dd9f36da831dd7b69c9fe352
>
> - Neeraj

LGTM, thank you for making the changes!