Re: [PATCH v4 2/5] x86, traps: Track entry into and exit from IST context

[Frederic Weisbecker <fweisbec@gmail.com>]

On Fri, Nov 21, 2014 at 02:07:04PM -0800, Paul E. McKenney wrote:
> On Fri, Nov 21, 2014 at 01:32:50PM -0800, Andy Lutomirski wrote:
> > On Fri, Nov 21, 2014 at 1:26 PM, Andy Lutomirski <luto@amacapital.net> wrote:
> > > We currently pretend that IST context is like standard exception
> > > context, but this is incorrect.  IST entries from userspace are like
> > > standard exceptions except that they use per-cpu stacks, so they are
> > > atomic.  IST entries from kernel space are like NMIs from RCU's
> > > perspective -- they are not quiescent states even if they
> > > interrupted the kernel during a quiescent state.
> > >
> > > Add and use ist_enter and ist_exit to track IST context.  Even
> > > though x86_32 has no IST stacks, we track these interrupts the same
> > > way.
> > 
> > I should add:
> > 
> > I have no idea why RCU read-side critical sections are safe inside
> > __do_page_fault today.  It's guarded by exception_enter(), but that
> > doesn't do anything if context tracking is off, and context tracking
> > is usually off. What am I missing here?
> 
> Ah!  There are three cases:
> 
> 1.	Context tracking is off on a non-idle CPU.  In this case, RCU is
> 	still paying attention to CPUs running in both userspace and in
> 	the kernel.  So if a page fault happens, RCU will be set up to
> 	notice any RCU read-side critical sections.
> 
> 2.	Context tracking is on on a non-idle CPU.  In this case, RCU
> 	might well be ignoring userspace execution: NO_HZ_FULL and
> 	all that.  However, as you pointed out, in this case the
> 	context-tracking code lets RCU know that we have entered the
> 	kernel, which means that RCU will again be paying attention to
> 	RCU read-side critical sections.
> 
> 3.	The CPU is idle.  In this case, RCU is ignoring the CPU, so
> 	if we take a page fault when context tracking is off, life
> 	will be hard.  But the kernel is not supposed to take page
> 	faults in the idle loop, so this is not a problem.
> 
> Make sense?

To zoom out the picture for Andy, context tracking is never used 99%
of all workloads. It's only used for NO_HZ_FULL. RCU needs to tick
to poll on RCU uses. But when we run in userspace, RCU isn't used
and thus doesn't need the tick which we can stop. So context tracking
is there to tell RCU about CPUs crossing user/kernel boundaries.

Also these hooks account the cputime spent in userspace and kernelspace
in the absence of a tick.