Re: Reconciling rcu_irq_enter()/rcu_nmi_enter() with context tracking

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Thu, Jul 16, 2015 at 06:53:15PM -0700, Andy Lutomirski wrote:
> For reasons that mystify me a bit, we currently track context tracking
> state separately from rcu's watching state.  This results in strange
> artifacts: nothing generic cause IRQs to enter CONTEXT_KERNEL, and we
> can nest exceptions inside the IRQ handler (an example would be
> wrmsr_safe failing), and, in -next, we splat a warning:
> 
> https://gist.github.com/sashalevin/a006a44989312f6835e7
> 
> I'm trying to make context tracking more exact, which will fix this
> issue (the particular splat that Sasha hit shouldn't be possible when
> I'm done), but I think it would be nice to unify all of this stuff.
> Would it be plausible for us to guarantee that RCU state is always in
> sync with context tracking state?  If so, we could maybe simplify
> things and have fewer state variables.

A noble goal.  Might even be possible, and maybe even advantageous.

But it is usually easier to say than to do.  RCU really does need to make
some adjustments when the state changes, as do the other subsystems.
It might or might not be possible to do the transitions atomically.
And if the transitions are not atomic, there will still be weird code
paths where (say) the processor is considered non-idle, but RCU doesn't
realize it yet.  Such a code path could not safely use rcu_read_lock(),
so you still need RCU to be able to scream if someone tries it.
Contrariwise, if there is a code path where the processor is considered
idle, but RCU thinks it is non-idle, that code path can stall
grace periods.  (Yes, not a problem if the code path is short enough.
At least if the underlying VCPU is making progres...)

Still, I cannot prove that it is impossible, and if it is possible,
then as you say, there might well be benefits.

> Doing this for NMIs might be weird.  Would it make sense to have a
> CONTEXT_NMI that's somehow valid even if the NMI happened while
> changing context tracking state.

Face it, NMIs are weird.  ;-)

> Thoughts?  As it stands, I think we might already be broken for real:
> 
> Syscall -> user_exit.  Perf NMI hits *during* user_exit.  Perf does
> copy_from_user_nmi, which can fault, causing do_page_fault to get
> called, which calls exception_enter(), which can't be a good thing.
> 
> RCU is okay (sort of) because of rcu_nmi_enter, but this seems very fragile.

Actually, I see more cases where people forget irq_enter() than
rcu_nmi_enter().  "We will just nip in quickly and do something without
actually letting the irq system know.  Oh, and we want some event tracing
in that code path."  Boom!

> Thoughts?  As it stands, I need to do something because -tip and thus
> -next spews occasional warnings.

Tell me more?

							Thanx, Paul