Re: RFC: THE OFFLINE SCHEDULER

Thomas Gleixner
2009-08-28T10:26:55Z

On Fri, 28 Aug 2009, Gregory Haskins wrote:
> > However, and to what I believe is your point: its not entirely clear to
> > me what impact, if any, there would be w.r.t. any _other_ events that
> > may be driven off of the scheduler tick (i.e. events other than
> > scheduling policies, like timeslice expiration, etc). Perhaps someone
> > else like Thomas, Ingo, or Peter have some input here.
> >
> > I guess the specific question to ask is: Does the scheduler tick code
> > have any role other than timeslice policies and updating accounting
> > information? Examples would include timer-expiry, for instance. I
> > would think most of this logic is handled by finer grained components
> > like HRT, but I am admittedly ignorant of the actual timer voodoo ;)

There is not much happening in the scheduler tick:

- accounting of CPU time. this can be delegated to some other CPU
as long as the user space task is running and consuming 100%

- timer list timers. If there is no service/device active on that CPU
then there are no timers to run

- rcu call backs. Same as above, but might need some tweaking.

- printk tick. Not really interesting

- scheduler time slicing. Not necessary in such a context

- posix cpu timers. Only interesting when the application uses them

So there is not much which needs the tick in such a scenario.

Of course we'd need to exclude that CPU from the do_timer duty as
well.

> Thinking about this idea some more: I can't see why this isn't just a
> trivial variation of the nohz idle code already in mainline. In both
> cases (idle and FIFO tasks) the cpu is "consumed" 100% by some arbitrary
> job (spinning/HLT for idle, RT thread for FIFO) while we have the
> scheduler tick disabled. The only real difference is a matter of
> power-management (HLT/mwait go to sleep-states, whereas spinning/rt-task
> run full tilt).
>
> Therefore the answer may be as simple as bracketing the FIFO task with
> tick_nohz_stop_sched_tick() + tick_nohz_restart_sched_tick(). The nohz
> code will probably need some minor adjustments so it is not assuming
> things about the state being "idle" (e.g. "isidle") for places when it
> matters (idle_calls++ stat is one example).

Yeah, it's similar to what we do in nohz idle already, but we'd need
to split out some of the functions very carefully to reuse them.

> Potential problems:
>
> a) disabling/renabling the tick on a per-RT task schedule() may prove to
> be prohibitively expensive.

For a single taks consuming 100% CPU it is a non problem. You disable
it once. But yes on a standard system this needs to be investigated.

> b) we will need to make sure the rt-bandwidth protection mechanism is
> defeated so the task is allowed to consume 100% bandwidth.
>
> Perhaps these states should be in the cpuset/root-domain, and configured
> when you create the partition (e.g. "tick=off", "bandwidth=off" makes it
> an "offline" set).

That makes sense and should not be rocket science to implement.

Thanks,

tglx