Re: [PATCH RESEND] sched/nohz: Affine unpinned timers to housekeepers

[Frederic Weisbecker <fweisbec@gmail.com>]

On Fri, Aug 28, 2015 at 10:32:59AM +0200, Ingo Molnar wrote:
> 
> * Frederic Weisbecker <fweisbec@gmail.com> wrote:
> 
> > On Tue, Aug 25, 2015 at 10:29:04AM +0200, Ingo Molnar wrote:
> > > 
> > > * Frederic Weisbecker <fweisbec@gmail.com> wrote:
> > > 
> > > > On Mon, Aug 24, 2015 at 08:44:12AM +0200, Ingo Molnar wrote:
> > > > > 2)
> > > > > 
> > > > > What happens if the boot CPU is offlined? (under CONFIG_BOOTPARAM_HOTPLUG_CPU0=y)
> > > > > 
> > > > > I don't see CPU hotplug callbacks fixing up the housekeeping_mask if the boot CPU 
> > > > > is offlined.
> > > > 
> > > > We have tick_nohz_cpu_down_callback() which makes sure that the timekeeper, which
> > > > is the boot CPU in nohz full, never gets offlined.
> > > 
> > > That solution really sucks - it essentially regresses a feature the user 
> > > explicitly asked for! I also see no way for the user to migrate the timekeeping 
> > > functionality over to another CPU without rebooting.
> > > 
> > > If this is the last timekeeping CPU then it should migrate the timekeeping 
> > > functionality to another CPU, and perhaps printk a warning if all other CPUs are 
> > > nohz-full and we have to mark one of them as the timekeeper.
> > > 
> > > Also, the nohz-full and timekeeper functionality should not be a boot parameter 
> > > only thing, but should be runtime configurable.
> > 
> > When I tried to allow moving the timekeeping duty over all housekeeping CPUs, 
> > Thomas got angry because it broke the KISS current nohz full code. Indeed, there 
> > must be at least one running all the time on behalf of nohz full CPUs that can 
> > run anytime. Thus balancing the timekeeping duty over housekeepers is a bit more 
> > complicated than in normal configurations.
> > 
> > Now surely we can do that using an IPI from CPU_DOWN_PREPARE to a housekeeper if 
> > any remains or to a nohz full one. Then we must make sure the new timekeeper 
> > never goes to idle.
> > 
> > But nohz_full is a corner usecase and I'm not sure it's worth the complexity. If 
> > a nohz full user came and complained about CPU0 hotplog not working, I would 
> > definetly retry it but I haven't heard about that yet. Besides, hotplug is very 
> > isolation-unfriendly in general due to stop machine.
> 
> Ok, I guess we can live with this.

Now this will likely evolve in the future, I can easily imagine that timekeeping becomes
balanced among housekeepers when we'll have one per node. It's not yet the priority but
we may come to that one day.

> 
> Mind sending an updated series with all patches?

Sure, I'm cooking that.

Thanks!