[PATCH 0/2] make kvm guest mode to be rcu quiescent state

[PATCH 0/2] make kvm guest mode to be rcu quiescent state

[Gleb Natapov]

CPU may spend a lot of time in a guest mode while other CPUs wait for
rcu grace period to elapse. This patch series makes guest mode into
quiescent state to shorten wait time.

Gleb Natapov (2):
  rcu: provide rcu_virt_note_context_switch() function.
  KVM: make guest mode entry to be rcu quiescent state

 include/linux/kvm_host.h |    9 +++++++++
 include/linux/rcutiny.h  |    8 ++++++++
 include/linux/rcutree.h  |   10 ++++++++++
 kernel/rcutree.c         |    1 +
 4 files changed, 28 insertions(+), 0 deletions(-)

-- 
1.7.4.4

[PATCH 1/2] rcu: provide rcu_virt_note_context_switch() function.

[Gleb Natapov]

Provide rcu_virt_note_context_switch() for vitalization use to note
quiescent state during guest entry.

Signed-off-by: Gleb Natapov <gleb@redhat.com>
---
 include/linux/rcutiny.h |    8 ++++++++
 include/linux/rcutree.h |   10 ++++++++++
 kernel/rcutree.c        |    1 +
 3 files changed, 19 insertions(+), 0 deletions(-)

diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h
index 30ebd7c..52b3e02 100644
--- a/include/linux/rcutiny.h
+++ b/include/linux/rcutiny.h
@@ -100,6 +100,14 @@ static inline void rcu_note_context_switch(int cpu)
 }
 
 /*
+ * Take advantage of the fact that there is only one CPU, which
+ * allows us to ignore virtualization-based context switches.
+ */
+static inline void rcu_virt_note_context_switch(int cpu)
+{
+}
+
+/*
  * Return the number of grace periods.
  */
 static inline long rcu_batches_completed(void)
diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h
index 3a93348..b2a906e 100644
--- a/include/linux/rcutree.h
+++ b/include/linux/rcutree.h
@@ -35,6 +35,16 @@ extern void rcu_note_context_switch(int cpu);
 extern int rcu_needs_cpu(int cpu);
 extern void rcu_cpu_stall_reset(void);
 
+/*
+ * Note a virtualization-based context switch.  This is simply a
+ * wrapper around rcu_note_context_switch(), which allows TINY_RCU
+ * to save a few bytes.
+ */
+static inline void rcu_virt_note_context_switch(int cpu)
+{
+	rcu_note_context_switch(cpu);
+}
+
 #ifdef CONFIG_TREE_PREEMPT_RCU
 
 extern void exit_rcu(void);
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index dd4aea8..0837d63 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -124,6 +124,7 @@ void rcu_note_context_switch(int cpu)
 	rcu_sched_qs(cpu);
 	rcu_preempt_note_context_switch(cpu);
 }
+EXPORT_SYMBOL_GPL(rcu_note_context_switch);
 
 #ifdef CONFIG_NO_HZ
 DEFINE_PER_CPU(struct rcu_dynticks, rcu_dynticks) = {
-- 
1.7.4.4

Re: [PATCH 1/2] rcu: provide rcu_virt_note_context_switch() function.

[Paul E. McKenney]

On Wed, May 04, 2011 at 04:31:03PM +0300, Gleb Natapov wrote:
> Provide rcu_virt_note_context_switch() for vitalization use to note
> quiescent state during guest entry.

Very good, queued on -rcu.

Unless you tell me otherwise, I will assume that you want to carry the
patch modifying KVM to use this.

							Thanx, Paul

> Signed-off-by: Gleb Natapov <gleb@redhat.com>
> ---
>  include/linux/rcutiny.h |    8 ++++++++
>  include/linux/rcutree.h |   10 ++++++++++
>  kernel/rcutree.c        |    1 +
>  3 files changed, 19 insertions(+), 0 deletions(-)
> 
> diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h
> index 30ebd7c..52b3e02 100644
> --- a/include/linux/rcutiny.h
> +++ b/include/linux/rcutiny.h
> @@ -100,6 +100,14 @@ static inline void rcu_note_context_switch(int cpu)
>  }
> 
>  /*
> + * Take advantage of the fact that there is only one CPU, which
> + * allows us to ignore virtualization-based context switches.
> + */
> +static inline void rcu_virt_note_context_switch(int cpu)
> +{
> +}
> +
> +/*
>   * Return the number of grace periods.
>   */
>  static inline long rcu_batches_completed(void)
> diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h
> index 3a93348..b2a906e 100644
> --- a/include/linux/rcutree.h
> +++ b/include/linux/rcutree.h
> @@ -35,6 +35,16 @@ extern void rcu_note_context_switch(int cpu);
>  extern int rcu_needs_cpu(int cpu);
>  extern void rcu_cpu_stall_reset(void);
> 
> +/*
> + * Note a virtualization-based context switch.  This is simply a
> + * wrapper around rcu_note_context_switch(), which allows TINY_RCU
> + * to save a few bytes.
> + */
> +static inline void rcu_virt_note_context_switch(int cpu)
> +{
> +	rcu_note_context_switch(cpu);
> +}
> +
>  #ifdef CONFIG_TREE_PREEMPT_RCU
> 
>  extern void exit_rcu(void);
> diff --git a/kernel/rcutree.c b/kernel/rcutree.c
> index dd4aea8..0837d63 100644
> --- a/kernel/rcutree.c
> +++ b/kernel/rcutree.c
> @@ -124,6 +124,7 @@ void rcu_note_context_switch(int cpu)
>  	rcu_sched_qs(cpu);
>  	rcu_preempt_note_context_switch(cpu);
>  }
> +EXPORT_SYMBOL_GPL(rcu_note_context_switch);
> 
>  #ifdef CONFIG_NO_HZ
>  DEFINE_PER_CPU(struct rcu_dynticks, rcu_dynticks) = {
> -- 
> 1.7.4.4
> 

Re: [PATCH 1/2] rcu: provide rcu_virt_note_context_switch() function.

[Avi Kivity]

On 05/04/2011 07:35 PM, Paul E. McKenney wrote:
> On Wed, May 04, 2011 at 04:31:03PM +0300, Gleb Natapov wrote:
> >  Provide rcu_virt_note_context_switch() for vitalization use to note
> >  quiescent state during guest entry.
>
> Very good, queued on -rcu.
>
> Unless you tell me otherwise, I will assume that you want to carry the
> patch modifying KVM to use this.

Is -rcu a fast-forward-only tree (like tip)?  If so I'll merge it and 
apply patch 2.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [PATCH 1/2] rcu: provide rcu_virt_note_context_switch() function.

[Paul E. McKenney]

On Mon, May 09, 2011 at 11:51:34AM +0300, Avi Kivity wrote:
> On 05/04/2011 07:35 PM, Paul E. McKenney wrote:
> >On Wed, May 04, 2011 at 04:31:03PM +0300, Gleb Natapov wrote:
> >>  Provide rcu_virt_note_context_switch() for vitalization use to note
> >>  quiescent state during guest entry.
> >
> >Very good, queued on -rcu.
> >
> >Unless you tell me otherwise, I will assume that you want to carry the
> >patch modifying KVM to use this.
> 
> Is -rcu a fast-forward-only tree (like tip)?  If so I'll merge it
> and apply patch 2.

Yep, -rcu is subject to rebase and feeds into -tip.  The patch is
SHA 29ce831000081dd757d3116bf774aafffc4b6b20 in

	git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git

Branch is rcu/next.  My guess is that this commit will show up in
-tip soon.

							Thanx, Paul

[PATCH 2/2] KVM: make guest mode entry to be rcu quiescent state

[Gleb Natapov]

KVM does not hold any references to rcu protected data when it switches
CPU into a guest mode. In fact switching to a guest mode is very similar
to exiting to userspase from rcu point of view. In addition CPU may stay
in a guest mode for quite a long time (up to one time slice). Lets treat
guest mode as quiescent state, just like we do with user-mode execution.

Signed-off-by: Gleb Natapov <gleb@redhat.com>
---
 include/linux/kvm_host.h |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 0bc3d37..244413f 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -591,8 +591,17 @@ static inline int kvm_deassign_device(struct kvm *kvm,
 
 static inline void kvm_guest_enter(void)
 {
+	BUG_ON(preemptible());
 	account_system_vtime(current);
 	current->flags |= PF_VCPU;
+	/* KVM does not hold any references to rcu protected data when it
+ 	 * switches CPU into a guest mode. In fact switching to a guest mode
+ 	 * is very similar to exiting to userspase from rcu point of view. In
+ 	 * addition CPU may stay in a guest mode for quite a long time (up to
+ 	 * one time slice). Lets treat guest mode as quiescent state, just like
+ 	 * we do with user-mode execution.
+ 	 */
+	rcu_virt_note_context_switch(smp_processor_id());
 }
 
 static inline void kvm_guest_exit(void)
-- 
1.7.4.4

Re: [PATCH 2/2] KVM: make guest mode entry to be rcu quiescent state

[Avi Kivity]

On 05/04/2011 04:31 PM, Gleb Natapov wrote:
> KVM does not hold any references to rcu protected data when it switches
> CPU into a guest mode. In fact switching to a guest mode is very similar
> to exiting to userspase from rcu point of view. In addition CPU may stay
> in a guest mode for quite a long time (up to one time slice). Lets treat
> guest mode as quiescent state, just like we do with user-mode execution.

Applied, thanks.

-- 
error compiling committee.c: too many arguments to function

[PATCH 0/2] make kvm guest mode to be rcu quiescent state

[Gleb Natapov]

CPU may spend a lot of time in a guest mode while other CPUs wait for
rcu grace period to elapse. This patch series makes guest mode into
quiescent state to shorten wait time.

Gleb Natapov (2):
  rcu: export rcu_note_context_switch() function
  KVM: make guest mode entry to be rcu quiescent state

 include/linux/kvm_host.h |    1 +
 include/linux/rcutiny.h  |    6 +-----
 kernel/rcutiny.c         |    7 +++++++
 kernel/rcutree.c         |    1 +
 4 files changed, 10 insertions(+), 5 deletions(-)

-- 
1.7.2.3

[PATCH 2/2] KVM: make guest mode entry to be rcu quiescent state

[Gleb Natapov]

KVM does not hold any references to rcu protected data when it switches
CPU into a guest mode. In fact switching to a guest mode is very similar
to exiting to userspase from rcu point of view. In addition CPU may stay
in a guest mode for quite a long time (up to one time slice). Lets treat
guest mode as quiescent state, just like we do with user-mode execution.

Signed-off-by: Gleb Natapov <gleb@redhat.com>
---
 include/linux/kvm_host.h |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 0bc3d37..a347bce 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -593,6 +593,7 @@ static inline void kvm_guest_enter(void)
 {
 	account_system_vtime(current);
 	current->flags |= PF_VCPU;
+	rcu_note_context_switch(smp_processor_id());
 }
 
 static inline void kvm_guest_exit(void)
-- 
1.7.2.3

Re: [PATCH 2/2] KVM: make guest mode entry to be rcu quiescent state

[Marcelo Tosatti]

Hi Gleb,

Thats a neat idea.

On Thu, Apr 28, 2011 at 12:52:03PM +0300, Gleb Natapov wrote:
> KVM does not hold any references to rcu protected data when it switches
> CPU into a guest mode. In fact switching to a guest mode is very similar
> to exiting to userspase from rcu point of view. In addition CPU may stay
> in a guest mode for quite a long time (up to one time slice). Lets treat
> guest mode as quiescent state, just like we do with user-mode execution.
> 
> Signed-off-by: Gleb Natapov <gleb@redhat.com>
> ---
>  include/linux/kvm_host.h |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> index 0bc3d37..a347bce 100644
> --- a/include/linux/kvm_host.h
> +++ b/include/linux/kvm_host.h
> @@ -593,6 +593,7 @@ static inline void kvm_guest_enter(void)
>  {
>  	account_system_vtime(current);
>  	current->flags |= PF_VCPU;
> +	rcu_note_context_switch(smp_processor_id());
>  }
>  
>  static inline void kvm_guest_exit(void)
> -- 
> 1.7.2.3

Please have it in x86's vcpu_enter_guest, then its more explicit
(uncertain about the context of kvm_guest_enter call in other arches).

Re: [PATCH 2/2] KVM: make guest mode entry to be rcu quiescent state

[Avi Kivity]

On 05/03/2011 04:13 PM, Marcelo Tosatti wrote:
> Please have it in x86's vcpu_enter_guest, then its more explicit
> (uncertain about the context of kvm_guest_enter call in other arches).

We do need it for other archs, don't we?


-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 2/2] KVM: make guest mode entry to be rcu quiescent state

[Gleb Natapov]

On Tue, May 03, 2011 at 10:13:17AM -0300, Marcelo Tosatti wrote:
> Hi Gleb,
> 
> Thats a neat idea.
> 
> On Thu, Apr 28, 2011 at 12:52:03PM +0300, Gleb Natapov wrote:
> > KVM does not hold any references to rcu protected data when it switches
> > CPU into a guest mode. In fact switching to a guest mode is very similar
> > to exiting to userspase from rcu point of view. In addition CPU may stay
> > in a guest mode for quite a long time (up to one time slice). Lets treat
> > guest mode as quiescent state, just like we do with user-mode execution.
> > 
> > Signed-off-by: Gleb Natapov <gleb@redhat.com>
> > ---
> >  include/linux/kvm_host.h |    1 +
> >  1 files changed, 1 insertions(+), 0 deletions(-)
> > 
> > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> > index 0bc3d37..a347bce 100644
> > --- a/include/linux/kvm_host.h
> > +++ b/include/linux/kvm_host.h
> > @@ -593,6 +593,7 @@ static inline void kvm_guest_enter(void)
> >  {
> >  	account_system_vtime(current);
> >  	current->flags |= PF_VCPU;
> > +	rcu_note_context_switch(smp_processor_id());
> >  }
> >  
> >  static inline void kvm_guest_exit(void)
> > -- 
> > 1.7.2.3
> 
> Please have it in x86's vcpu_enter_guest, then its more explicit
> (uncertain about the context of kvm_guest_enter call in other arches).
> 
I checked all of them and kvm_guest_enter() is always called with local
irq disabled. Paul confirmed that rcu_note_context_switch() can be
called in such context.

--
			Gleb.

Re: [PATCH 2/2] KVM: make guest mode entry to be rcu quiescent state

[Marcelo Tosatti]

On Tue, May 03, 2011 at 04:21:23PM +0300, Gleb Natapov wrote:
> On Tue, May 03, 2011 at 10:13:17AM -0300, Marcelo Tosatti wrote:
> > Hi Gleb,
> > 
> > Thats a neat idea.
> > 
> > On Thu, Apr 28, 2011 at 12:52:03PM +0300, Gleb Natapov wrote:
> > > KVM does not hold any references to rcu protected data when it switches
> > > CPU into a guest mode. In fact switching to a guest mode is very similar
> > > to exiting to userspase from rcu point of view. In addition CPU may stay
> > > in a guest mode for quite a long time (up to one time slice). Lets treat
> > > guest mode as quiescent state, just like we do with user-mode execution.
> > > 
> > > Signed-off-by: Gleb Natapov <gleb@redhat.com>
> > > ---
> > >  include/linux/kvm_host.h |    1 +
> > >  1 files changed, 1 insertions(+), 0 deletions(-)
> > > 
> > > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> > > index 0bc3d37..a347bce 100644
> > > --- a/include/linux/kvm_host.h
> > > +++ b/include/linux/kvm_host.h
> > > @@ -593,6 +593,7 @@ static inline void kvm_guest_enter(void)
> > >  {
> > >  	account_system_vtime(current);
> > >  	current->flags |= PF_VCPU;
> > > +	rcu_note_context_switch(smp_processor_id());
> > >  }
> > >  
> > >  static inline void kvm_guest_exit(void)
> > > -- 
> > > 1.7.2.3
> > 
> > Please have it in x86's vcpu_enter_guest, then its more explicit
> > (uncertain about the context of kvm_guest_enter call in other arches).
> > 
> I checked all of them and kvm_guest_enter() is always called with local
> irq disabled. Paul confirmed that rcu_note_context_switch() can be
> called in such context.

OK then. Perhaps have an assert to verify interrupts are disabled?

Re: [PATCH 2/2] KVM: make guest mode entry to be rcu quiescent state

[Gleb Natapov]

On Tue, May 03, 2011 at 10:29:52AM -0300, Marcelo Tosatti wrote:
> On Tue, May 03, 2011 at 04:21:23PM +0300, Gleb Natapov wrote:
> > On Tue, May 03, 2011 at 10:13:17AM -0300, Marcelo Tosatti wrote:
> > > Hi Gleb,
> > > 
> > > Thats a neat idea.
> > > 
> > > On Thu, Apr 28, 2011 at 12:52:03PM +0300, Gleb Natapov wrote:
> > > > KVM does not hold any references to rcu protected data when it switches
> > > > CPU into a guest mode. In fact switching to a guest mode is very similar
> > > > to exiting to userspase from rcu point of view. In addition CPU may stay
> > > > in a guest mode for quite a long time (up to one time slice). Lets treat
> > > > guest mode as quiescent state, just like we do with user-mode execution.
> > > > 
> > > > Signed-off-by: Gleb Natapov <gleb@redhat.com>
> > > > ---
> > > >  include/linux/kvm_host.h |    1 +
> > > >  1 files changed, 1 insertions(+), 0 deletions(-)
> > > > 
> > > > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> > > > index 0bc3d37..a347bce 100644
> > > > --- a/include/linux/kvm_host.h
> > > > +++ b/include/linux/kvm_host.h
> > > > @@ -593,6 +593,7 @@ static inline void kvm_guest_enter(void)
> > > >  {
> > > >  	account_system_vtime(current);
> > > >  	current->flags |= PF_VCPU;
> > > > +	rcu_note_context_switch(smp_processor_id());
> > > >  }
> > > >  
> > > >  static inline void kvm_guest_exit(void)
> > > > -- 
> > > > 1.7.2.3
> > > 
> > > Please have it in x86's vcpu_enter_guest, then its more explicit
> > > (uncertain about the context of kvm_guest_enter call in other arches).
> > > 
> > I checked all of them and kvm_guest_enter() is always called with local
> > irq disabled. Paul confirmed that rcu_note_context_switch() can be
> > called in such context.
> 
> OK then. Perhaps have an assert to verify interrupts are disabled?
Yes. Can add BUG_ON(preemptible()).

--
			Gleb.

Re: [PATCH 2/2] KVM: make guest mode entry to be rcu quiescent state

[Marcelo Tosatti]

On Tue, May 03, 2011 at 04:39:01PM +0300, Gleb Natapov wrote:
> > > > On Thu, Apr 28, 2011 at 12:52:03PM +0300, Gleb Natapov wrote:
> > > > > KVM does not hold any references to rcu protected data when it switches
> > > > > CPU into a guest mode. In fact switching to a guest mode is very similar
> > > > > to exiting to userspase from rcu point of view. In addition CPU may stay
> > > > > in a guest mode for quite a long time (up to one time slice). Lets treat
> > > > > guest mode as quiescent state, just like we do with user-mode execution.
> > > > > 
> > > > > Signed-off-by: Gleb Natapov <gleb@redhat.com>
> > > > > ---
> > > > >  include/linux/kvm_host.h |    1 +
> > > > >  1 files changed, 1 insertions(+), 0 deletions(-)
> > > > > 
> > > > > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> > > > > index 0bc3d37..a347bce 100644
> > > > > --- a/include/linux/kvm_host.h
> > > > > +++ b/include/linux/kvm_host.h
> > > > > @@ -593,6 +593,7 @@ static inline void kvm_guest_enter(void)
> > > > >  {
> > > > >  	account_system_vtime(current);
> > > > >  	current->flags |= PF_VCPU;
> > > > > +	rcu_note_context_switch(smp_processor_id());
> > > > >  }
> > > > >  
> > > > >  static inline void kvm_guest_exit(void)
> > > > > -- 
> > > > > 1.7.2.3
> > > > 
> > > > Please have it in x86's vcpu_enter_guest, then its more explicit
> > > > (uncertain about the context of kvm_guest_enter call in other arches).
> > > > 
> > > I checked all of them and kvm_guest_enter() is always called with local
> > > irq disabled. Paul confirmed that rcu_note_context_switch() can be
> > > called in such context.
> > 
> > OK then. Perhaps have an assert to verify interrupts are disabled?
> Yes. Can add BUG_ON(preemptible()).

Also please add a comment to explain whats going on. The commit message
above seems appropriate.