Re: [RFC/PATCH] cpuset: cpuset irq affinities

Peter Zijlstra
2008-02-29T21:04:05Z

On Fri, 2008-02-29 at 12:52 -0800, Max Krasnyanskiy wrote:
> Ingo Molnar wrote:
> > * Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
> >
> >> @@ -174,11 +174,20 @@ struct irq_desc {
> >> #ifdef CONFIG_PROC_FS
> >> struct proc_dir_entry *dir;
> >> #endif
> >> +#ifdef CONFIG_CPUSETS
> >> + struct cpuset *cs;
> >> +#endif
> >
> > i like this approach - it makes irqs more resource-alike and attaches
> > them to a specific resource control group.
> >
> > So if /cgroup/boot is changed to have less CPUs then the "default" irqs
> > move along with it.
> >
> > but if an isolated RT domain has specific irqs attached to it (say the
> > IRQ of some high-speed data capture device), then the irqs would move
> > together with that domain.
> >
> > irqs are no longer a bolted-upon concept, but more explicitly managed.
> >
> > [ If you boot-test it and if Paul agrees with the general approach then
> > i could even apply it to sched-devel.git ;-) ]
>
> Believe it or not but I like it too :).
> Now we're talking different approach compared to the cpu_isolated_map since
> with this patch cpu_system_map is no longer needed.
> I've been playing with latest sched-devel tree and while I think we'll endup
> adding a lot more code, doing it with the cpuset is definitely more flexible.
> This way we can provide more fine grain control of what part of the "system"
> services are allowed to run on a cpuset. Rather that "catch all" system flag.
>
> Current sched-devel tree does not provide complete isolation at this point.
> There are still many things here and there that need to be added/fixed.
> Having finer control here helps.
>
> One concern I have is that this API conflicts with /proc/irq/X/smp_affinity.
> ie Setting smp_affinity manually will override affinity set by the cpuset.
> In other words I think
> int irq_set_affinity(unsigned int irq, cpumask_t cpumask)
> now needs to make sure that cpumask does not have cpus that do not belong to
> the cpuset this irq belongs to. Just like sched_setaffinity() does for the tasks.

The patch also needs to handle group destruction too; currently it
leaves cpuset pointers dangling. So it would either have to refuse
removing a group when there are still irqs associated, or move them to
the parent.

But yeah, this was just a quick hack to show the idea, glad you like it.
Will try to flesh it out a bit in the coming week.