Re: [RFC PATCH v2 1/2] tick/nohz: Add a policy dependency for full dynticks

[Peng Wang <peng_wang@linux.alibaba.com>]

On Fri, Oct 23, 2026 at 05:00:22PM +0200, Frederic Weisbecker wrote:
> Le Sat, Oct 17, 2026 at 09:13:41PM +0800, Peng Wang a 殂rit :
> > Cloud hosts may pin vCPU threads one-to-one to host CPUs. At full
> > allocation this includes the housekeeping CPUs, which always retain
> > their non-idle tick. Operators want comparable performance for otherwise
> > equivalent vCPUs. Retaining the non-idle tick on nohz_full CPUs as well
> > avoids one placement-dependent difference in host tick interference.
> > With spare capacity, selected CPUs may stop the tick to reduce host
> > scheduler-tick interference with vCPUs.
> > 
> > Allocation changes as VMs are created or destroyed while other VMs
> > remain running. A vCPU thread may be the only runnable task in either
> > state, so existing tick dependencies do not express this policy.
> 
> I'm not sure I understand correctly what you are trying to solve.
> Are you saying that you want to keep the tick running when there
> is more than one vCPU per CPU? Because the scheduler already takes
> care of that.

Hi Frederic,

No. We pin vCPU threads one-to-one to host CPUs, without
oversubscription. By "full allocation" I meant that all allocatable
host CPUs are in use, including CPUs outside the nohz_full set.

For example, on an eight-CPU host booted with isolcpus=nohz,1-7,
CPU0 is outside the nohz_full set:

- With eight pinned vCPUs, CPU0 retains its non-idle tick while
  CPUs1-7 may stop theirs. Retaining the tick on CPUs1-7 too avoids
  this placement-dependent difference in tick interference between
  otherwise equivalent vCPUs.

- With seven or fewer pinned vCPUs, we can leave CPU0 without a
  vCPU and keep them on CPUs1-7. We can then allow tick suppression
  on CPUs1-7, subject to the existing dependencies. This gives
  these vCPUs a common tick policy while reducing host tick
  interference.

If the vCPU on CPU0 is released while the other seven remain
running, we allow CPUs1-7 to suppress their ticks. If a new vCPU
is later assigned to CPU0, we retain the ticks on CPUs1-7 again
to avoid the difference in tick interference.

Throughout these transitions, CPU1 can still have its original
pinned vCPU thread as its only runnable task. Its runnable task
count does not change merely because CPU0 gains or loses a vCPU,
so the scheduler's existing tick dependency cannot express this
allocation policy.

> 
> > 
> > Add a per-CPU POLICY dependency within the boot-time nohz_full set.
> > Permission defaults to enabled. The policy can be changed without
> > taking CPUs offline. Revoking permission uses the existing asynchronous
> > kick; granting it leaves all other dependencies effective. Preserve
> > POLICY across CPU hotplug and report it in tick_stop tracing.
> > 
> > Idle tick suppression, nohz_full membership, context tracking, RCU
> > offloading and housekeeping configuration are unchanged.
> 
> This still keeps the overhead of nohz_full even if the tick stays on, right?
> (RCU nocb, kernel <-> user overhead, etc...). Isn't it a problem for you?

Yes. Our setup uses only the nohz flag of isolcpus, without
additional housekeeping affinity tuning. Context tracking and
RCU callback offloading remain enabled when the tick is kept
running.

We have observed a performance benefit from this nohz_full setup
with our database workload running on pinned vCPUs. We retain the
boot-time nohz_full configuration so that tick suppression remains
available as vCPU allocation changes, even though its other costs
remain when the tick is kept running.

This control changes only whether non-idle ticks may be suppressed;
it does not change the nohz_full CPU set or housekeeping CPU masks.

Thanks,
Peng

> 
> Thanks.
> 
> > 
> > Co-developed-by: He Rongguang <herongguang@linux.alibaba.com>
> > Signed-off-by: He Rongguang <herongguang@linux.alibaba.com>
> > Signed-off-by: Peng Wang <peng_wang@linux.alibaba.com>
> > ---
> >  include/linux/tick.h         |  8 +++++--
> >  include/trace/events/timer.h |  3 ++-
> >  kernel/time/tick-sched.c     | 42 ++++++++++++++++++++++++++++++++++++
> >  3 files changed, 50 insertions(+), 3 deletions(-)
> > 
> > diff --git a/include/linux/tick.h b/include/linux/tick.h
> > index b121c5d53e308..8b250f245ff31 100644
> > --- a/include/linux/tick.h
> > +++ b/include/linux/tick.h
> > @@ -111,9 +111,10 @@ enum tick_dep_bits {
> >  	TICK_DEP_BIT_SCHED		= 2,
> >  	TICK_DEP_BIT_CLOCK_UNSTABLE	= 3,
> >  	TICK_DEP_BIT_RCU		= 4,
> > -	TICK_DEP_BIT_RCU_EXP		= 5
> > +	TICK_DEP_BIT_RCU_EXP		= 5,
> > +	TICK_DEP_BIT_POLICY		= 6
> >  };
> > -#define TICK_DEP_BIT_MAX TICK_DEP_BIT_RCU_EXP
> > +#define TICK_DEP_BIT_MAX TICK_DEP_BIT_POLICY
> >  
> >  #define TICK_DEP_MASK_NONE		0
> >  #define TICK_DEP_MASK_POSIX_TIMER	(1 << TICK_DEP_BIT_POSIX_TIMER)
> > @@ -122,6 +123,7 @@ enum tick_dep_bits {
> >  #define TICK_DEP_MASK_CLOCK_UNSTABLE	(1 << TICK_DEP_BIT_CLOCK_UNSTABLE)
> >  #define TICK_DEP_MASK_RCU		(1 << TICK_DEP_BIT_RCU)
> >  #define TICK_DEP_MASK_RCU_EXP		(1 << TICK_DEP_BIT_RCU_EXP)
> > +#define TICK_DEP_MASK_POLICY		(1 << TICK_DEP_BIT_POLICY)
> >  
> >  #ifdef CONFIG_NO_HZ_COMMON
> >  extern bool tick_nohz_enabled;
> > @@ -205,6 +207,8 @@ extern void tick_nohz_dep_set_signal(struct task_struct *tsk,
> >  extern void tick_nohz_dep_clear_signal(struct signal_struct *signal,
> >  				       enum tick_dep_bits bit);
> >  extern bool tick_nohz_cpu_hotpluggable(unsigned int cpu);
> > +extern bool tick_nohz_full_get_allowed(unsigned int cpu);
> > +extern int tick_nohz_full_set_allowed(unsigned int cpu, bool allowed);
> >  
> >  /*
> >   * The below are tick_nohz_[set,clear]_dep() wrappers that optimize off-cases
> > diff --git a/include/trace/events/timer.h b/include/trace/events/timer.h
> > index 3aa0608c63611..dff9579cd90eb 100644
> > --- a/include/trace/events/timer.h
> > +++ b/include/trace/events/timer.h
> > @@ -435,7 +435,8 @@ TRACE_EVENT(itimer_expire,
> >  		tick_dep_name(SCHED)			\
> >  		tick_dep_name(CLOCK_UNSTABLE)		\
> >  		tick_dep_name(RCU)			\
> > -		tick_dep_name_end(RCU_EXP)
> > +		tick_dep_name(RCU_EXP)			\
> > +		tick_dep_name_end(POLICY)
> >  
> >  #undef tick_dep_name
> >  #undef tick_dep_mask_name
> > diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
> > index 87e2a7a875265..0d6fcc3736bae 100644
> > --- a/kernel/time/tick-sched.c
> > +++ b/kernel/time/tick-sched.c
> > @@ -375,6 +375,11 @@ static bool check_tick_dependency(atomic_t *dep)
> >  		return true;
> >  	}
> >  
> > +	if (val & TICK_DEP_MASK_POLICY) {
> > +		trace_tick_stop(0, TICK_DEP_MASK_POLICY);
> > +		return true;
> > +	}
> > +
> >  	return false;
> >  }
> >  
> > @@ -561,6 +566,39 @@ void tick_nohz_dep_clear_cpu(int cpu, enum tick_dep_bits bit)
> >  }
> >  EXPORT_SYMBOL_GPL(tick_nohz_dep_clear_cpu);
> >  
> > +/* Read policy, not whether a CPU currently has its tick stopped. */
> > +bool tick_nohz_full_get_allowed(unsigned int cpu)
> > +{
> > +	struct tick_sched *ts = per_cpu_ptr(&tick_cpu_sched, cpu);
> > +
> > +	/* CPU teardown resets and restores this bit under stop_machine(). */
> > +	return !(atomic_read(&ts->tick_dep_mask) & TICK_DEP_MASK_POLICY);
> > +}
> > +
> > +int tick_nohz_full_set_allowed(unsigned int cpu, bool allowed)
> > +{
> > +	struct tick_sched *ts = per_cpu_ptr(&tick_cpu_sched, cpu);
> > +
> > +	if (!tick_nohz_full_cpu(cpu))
> > +		return -EINVAL;
> > +
> > +	/* CPU device removal may wait for an active sysfs callback. */
> > +	if (!cpus_read_trylock())
> > +		return -EBUSY;
> > +
> > +	if (allowed) {
> > +		tick_nohz_dep_clear_cpu(cpu, TICK_DEP_BIT_POLICY);
> > +	} else if (cpu_online(cpu)) {
> > +		/* The existing kick requests an asynchronous restart. */
> > +		tick_nohz_dep_set_cpu(cpu, TICK_DEP_BIT_POLICY);
> > +	} else {
> > +		/* Record policy without queuing irq_work on an offline CPU. */
> > +		atomic_or(TICK_DEP_MASK_POLICY, &ts->tick_dep_mask);
> > +	}
> > +	cpus_read_unlock();
> > +	return 0;
> > +}
> > +
> >  /*
> >   * Set a per-task tick dependency. RCU needs this. Also posix CPU timers
> >   * in order to elapse per task timers.
> > @@ -1533,6 +1571,7 @@ void tick_sched_timer_dying(int cpu)
> >  {
> >  	struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu);
> >  	unsigned long idle_calls, idle_sleeps;
> > +	int policy;
> >  
> >  	/* This must happen before hrtimers are migrated! */
> >  	if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES))
> > @@ -1540,7 +1579,10 @@ void tick_sched_timer_dying(int cpu)
> >  
> >  	idle_calls = ts->idle_calls;
> >  	idle_sleeps = ts->idle_sleeps;
> > +	/* Preserve only the policy dependency across CPU offline/online. */
> > +	policy = atomic_read(&ts->tick_dep_mask) & TICK_DEP_MASK_POLICY;
> >  	memset(ts, 0, sizeof(*ts));
> > +	atomic_set(&ts->tick_dep_mask, policy);
> >  	ts->idle_calls = idle_calls;
> >  	ts->idle_sleeps = idle_sleeps;
> >  }
> > -- 
> > 2.47.0
> > 
> 
> -- 
> Frederic Weisbecker
> SUSE Labs