Re: [PATCH] local_irq_disable removal

[Ingo Molnar <mingo@elte.hu>]

* Esben Nielsen <simlo@phys.au.dk> wrote:

> > Plus take into
> > account that the average interrupt disable section is very small .. I
> > also think it's possible to extend my version to allow those section to
> > be preemptible but keep the cost equally low.
> > 
> 
> The more I think about it the more dangerous I think it is. What does 
> local_irq_disable() protect against? All local threads as well as 
> irq-handlers. If these sections keeped mutual exclusive but preemtible 
> we will not have protected against a irq-handler.

one way to make it safe/reviewable is to runtime warn if 
local_irq_disable() is called from a !preempt_count() section. But this 
will uncover quite some code. There's some code in the VM, in the 
buffer-cache, in the RCU code - etc. that uses per-CPU data structures 
and assumes non-preemptability of local_irq_disable().

> I will start to play around with the following:
> 1) Make local_irq_disable() stop compiling to see how many we are really
> talking about.

there are roughly 100 places:

 $ objdump -d vmlinux | grep -w call |
      grep -wE 'local_irq_disable|local_irq_save' | wc -l
 116

the advantage of having such primitives as out-of-line function calls :)

> 2) Make local_cpu_lock, which on PREEMPT_RT is a rt_mutex and on
> !PREEMPT_RT turns into local_irq_disable()/enable() pairs. To introduce
> this will demand some code-analyzing for each case but I am afraid there
> is no general one-size solution to all the places.

I'm not sure we'd gain much from this. Lets assume we have a highprio RT 
task that is waiting for an external event. Will it be able to preempt 
the IRQ mutex?  Yes. Will it be able to make any progress: no, because 
it needs an IRQ thread to run to get the wakeup in the first place, and 
the IRQ thread needs to take the IRQ mutex => serialization.

what seems a better is to rewrite per-CPU-local-irq-disable users to 
make use of the DEFINE_PER_CPU_LOCKED/per_cpu_locked/get_cpu_lock 
primitives to use preemptible per-CPU data structures. In this case 
these sections would be truly preemptible. I've done this for a couple 
of cases already, where it was unavoidable for lock-dependency reasons.

	Ingo