Re: RCU+KVM: making CPU guest mode a quiescent state.

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Wed, Apr 27, 2011 at 10:47:04AM +0300, Avi Kivity wrote:
> On 04/26/2011 06:55 PM, Paul E. McKenney wrote:
> >On Tue, Apr 26, 2011 at 03:38:24PM +0300, Gleb Natapov wrote:
> >>  Hello Paul,
> >>
> >>  I have a question about RCU + KVM. KVM does not hold any references to RCU
> >>  protected data when it switches CPU into a guest mode. In fact switching
> >>  to a guest mode is very similar to exiting to userspase from RCU point
> >>  of view. In addition CPU may stay in a guest mode for quite a long time
> >>  (up to one time slice). It looks like it will be beneficial to treat guest
> >>  mode as quiescent state, just like user-mode execution. How can this be
> >>  done? I was trying to find how RCU knows about cpu entering user-mode,
> >>  but it seems that it does this by checking CPU mode in a timer interrupt
> >>  (update_process_times()->rcu_check_callbacks()). This will not work for
> >>  guest mode detection since timer interrupt will kick CPU out of a guest
> >>  mode and timer interrupt will always see CPU in kernel mode. Do we have
> >>  a simple function to call to notify RCU that CPU passed quiescent state
> >>  which we can call just before entering guest?
> >
> >Hello, Gleb,
> >
> >You could call rcu_note_context_switch(), passing it the current
> >CPU.  Please note that preemption -must- be disabled when calling
> >this.  You could call this just after exiting the guest as well
> >as just before entering guest.
> 
> It's expected that after exiting, we'd spend a very short time in
> the kernel, and then either re-enter the guest, exit to userspace,
> or switch to another task.  So I think calling it just before entry
> should be sufficient.

Agreed, sorry for my confusion!

> Looking at the code, I see rcu_note_context_switch() calls
> rcu_sched_qs(), which does
> 
>     rdp->passed_quiesc_completed = rdp->gpnum - 1;
>     barrier();
>     rdp->passed_quiesc = 1;
> 
> and also calls rcu_preempt_note_context_switch(), which calls
> rcu_preempt_qs(), which does
> 
>     rdp->passed_quiesc_completed = rdp->gpnum - 1;
>     barrier();
>     rdp->passed_quiesc = 1;
>     current->rcu_read_unlock_special &= ~RCU_READ_UNLOCK_NEED_QS;
> 
> the similarity is remarkable.  Is this intended?  Or did I get lost
> in a maze of #ifdefs?

The "rdp" is different in the two cases.  In the first case, it is
one of rcu_sched's per-CPU rcu_data structures, in the second case,
it is one of rcu_preempt's per-CPU rcu_data structures.  I considered
making the first three lines common code, but the extra function
bloated more than the duplicate three lines.  Perhaps I should have
tried harder.

							Thanx, Paul