Re: [PATCH] i386 No-Idle-Hz aka Dynamic-Ticks 3

[Srivatsa Vaddagiri <vatsa@in.ibm.com>]

On Wed, Aug 03, 2005 at 06:05:28AM +0000, Con Kolivas wrote:
> This is the dynamic ticks patch for i386 as written by Tony Lindgen 
> <tony@atomide.com> and Tuukka Tikkanen <tuukka.tikkanen@elektrobit.com>. 
> Patch for 2.6.13-rc5
> 
> There were a couple of things that I wanted to change so here is an updated 
> version. This code should have stabilised enough for general testing now.

Con,
	I have been looking at some of the requirement of tickless idle CPUs in
core kernel areas like scheduler and RCU. Basically, both power management and 
virtualization benefit if idle CPUs can cut off useless timer ticks. Especially 
from a virtualization standpoint, I think it makes sense that we enable this 
feature on a per-CPU basis i.e let individual CPUs cut off their ticks as and 
when they become idle. The benefit of this is more visible in platforms that 
host lot of (SMP) VMs on the same machine. Most of the time, these VMs may be 
partially idle (some CPUs in it are idle, some not) and it is good that we 
quiesce the timer ticks on the partial set of idle CPUs. Both S390 and Xen ports
of Linux kernel have this ability today (S390 has it in mainline already and 
Xen has it out of tree).

>From this viewpoint, I think the current implementation of dynamic tick
falls short of this requirement. It cuts of the timer ticks only when 
all CPUs go idle.

Apart from this observation, I have some others about the current dynamic tick
patch:

- All CPUs seem to cut off the same number of ticks (dyn_tick->skip). Isn't
  this wrong, considering that the timer list is per-CPU? This will cause
  some timers to be serviced much later than usual.

- The fact that dyn_tick_state is global and accessed from all CPUs
  is probably a scalability concern, especially if we allow the ticks
  to be cut off on per-CPU basis.

- Again, when we allow this on a per-CPU basis, subsystems like
  RCU need to know the partial set of idle CPUs. RCU already does
  that thr' nohz_cpu_mask (which will need to replace dyn_cpu_map).

- Looking at dyn_tick_timer_interrupt, would it be nice if we avoid calling 
  do_timer_interrupt so many times and instead update jiffies to
  (skipped_ticks - 1) and then call do_timer_interrupt once? I think
  VST does it that way.

- dyn_tick->max_skip = 0xffffff / apic_timer_val;
	From my reading of Intel docs, APIC_TMICT is 32-bit. So why does the
  above calculation take only 24-bits into account? What am I missing here?


I can take a shot at addressing these concerns in dynamic_tick patch, but it 
seems to me that VST has already addressed all these to a big extent. Had you 
considered VST before? The biggest bottleneck I see in VST going mainline is 
its dependency on HRT patch but IMO it should be possible to write a small patch
to support VST w/o HRT. 

George, what do you think?


-- 


Thanks and Regards,
Srivatsa Vaddagiri,
Linux Technology Center,
IBM Software Labs,
Bangalore, INDIA - 560017