[PATCH] kvm uses smp_call_function_mask() in kvm_flush_remote_tlbs()

[PATCH] kvm uses smp_call_function_mask() in kvm_flush_remote_tlbs()

[Laurent Vivier]

This patches can be applied only on kvm-updates-2.6.24 and kvm-updates-2.6.25
as it needs smp_call_function_mask().

In kvm_flush_remote_tlbs(), it replaces a loop using smp_call_function_single() 
by a single call to smp_call_function_mask().

Signed-off-by: Laurent Vivier <Laurent.Vivier-6ktuUTfB/bM@public.gmane.org>
---
 drivers/kvm/kvm_main.c |   26 +++-----------------------
 1 files changed, 3 insertions(+), 23 deletions(-)

diff --git a/drivers/kvm/kvm_main.c b/drivers/kvm/kvm_main.c
index bf6a86c..6f7b31e 100644
--- a/drivers/kvm/kvm_main.c
+++ b/drivers/kvm/kvm_main.c
@@ -200,21 +200,15 @@ void vcpu_put(struct kvm_vcpu *vcpu)
 
 static void ack_flush(void *_completed)
 {
-	atomic_t *completed = _completed;
-
-	atomic_inc(completed);
 }
 
 void kvm_flush_remote_tlbs(struct kvm *kvm)
 {
-	int i, cpu, needed;
+	int i, cpu;
 	cpumask_t cpus;
 	struct kvm_vcpu *vcpu;
-	atomic_t completed;
 
-	atomic_set(&completed, 0);
 	cpus_clear(cpus);
-	needed = 0;
 	for (i = 0; i < KVM_MAX_VCPUS; ++i) {
 		vcpu = kvm->vcpus[i];
 		if (!vcpu)
@@ -223,23 +217,9 @@ void kvm_flush_remote_tlbs(struct kvm *kvm)
 			continue;
 		cpu = vcpu->cpu;
 		if (cpu != -1 && cpu != raw_smp_processor_id())
-			if (!cpu_isset(cpu, cpus)) {
-				cpu_set(cpu, cpus);
-				++needed;
-			}
-	}
-
-	/*
-	 * We really want smp_call_function_mask() here.  But that's not
-	 * available, so ipi all cpus in parallel and wait for them
-	 * to complete.
-	 */
-	for (cpu = first_cpu(cpus); cpu != NR_CPUS; cpu = next_cpu(cpu, cpus))
-		smp_call_function_single(cpu, ack_flush, &completed, 1, 0);
-	while (atomic_read(&completed) != needed) {
-		cpu_relax();
-		barrier();
+			cpu_set(cpu, cpus);
 	}
+	smp_call_function_mask(cpus, ack_flush, NULL, 1);
 }
 
 int kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id)
-- 
1.5.2.4


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Re: [PATCH] kvm uses smp_call_function_mask() inkvm_flush_remote_tlbs()

[Dor Laor]

Laurent Vivier wrote:
>
> This patches can be applied only on kvm-updates-2.6.24 and 
> kvm-updates-2.6.25
> as it needs smp_call_function_mask().
>
> In kvm_flush_remote_tlbs(), it replaces a loop using 
> smp_call_function_single()
> by a single call to smp_call_function_mask().
>
Would you be kind enough to copy the implementation of 
smp_call_function_mask
into external_module_compat.h so older kernel's will still work after 
this commited.
Thanks, Dor.
>
> Signed-off-by: Laurent Vivier <Laurent.Vivier-6ktuUTfB/bM@public.gmane.org>
> ---
>  drivers/kvm/kvm_main.c |   26 +++-----------------------
>  1 files changed, 3 insertions(+), 23 deletions(-)
>
> diff --git a/drivers/kvm/kvm_main.c b/drivers/kvm/kvm_main.c
> index bf6a86c..6f7b31e 100644
> --- a/drivers/kvm/kvm_main.c
> +++ b/drivers/kvm/kvm_main.c
> @@ -200,21 +200,15 @@ void vcpu_put(struct kvm_vcpu *vcpu)
>
>  static void ack_flush(void *_completed)
>  {
> -       atomic_t *completed = _completed;
> -
> -       atomic_inc(completed);
>  }
>
>  void kvm_flush_remote_tlbs(struct kvm *kvm)
>  {
> -       int i, cpu, needed;
> +       int i, cpu;
>         cpumask_t cpus;
>         struct kvm_vcpu *vcpu;
> -       atomic_t completed;
>
> -       atomic_set(&completed, 0);
>         cpus_clear(cpus);
> -       needed = 0;
>         for (i = 0; i < KVM_MAX_VCPUS; ++i) {
>                 vcpu = kvm->vcpus[i];
>                 if (!vcpu)
> @@ -223,23 +217,9 @@ void kvm_flush_remote_tlbs(struct kvm *kvm)
>                         continue;
>                 cpu = vcpu->cpu;
>                 if (cpu != -1 && cpu != raw_smp_processor_id())
> -                       if (!cpu_isset(cpu, cpus)) {
> -                               cpu_set(cpu, cpus);
> -                               ++needed;
> -                       }
> -       }
> -
> -       /*
> -        * We really want smp_call_function_mask() here.  But that's not
> -        * available, so ipi all cpus in parallel and wait for them
> -        * to complete.
> -        */
> -       for (cpu = first_cpu(cpus); cpu != NR_CPUS; cpu = 
> next_cpu(cpu, cpus))
> -               smp_call_function_single(cpu, ack_flush, &completed, 
> 1, 0);
> -       while (atomic_read(&completed) != needed) {
> -               cpu_relax();
> -               barrier();
> +                       cpu_set(cpu, cpus);
>         }
> +       smp_call_function_mask(cpus, ack_flush, NULL, 1);
>  }
>
>  int kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id)
> --
> 1.5.2.4
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> kvm-devel mailing list
> kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/kvm-devel
>


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Re: [PATCH] kvm uses smp_call_function_mask() inkvm_flush_remote_tlbs()

[Laurent Vivier]

Dor Laor a écrit :
> Laurent Vivier wrote:
>> This patches can be applied only on kvm-updates-2.6.24 and 
>> kvm-updates-2.6.25
>> as it needs smp_call_function_mask().
>>
>> In kvm_flush_remote_tlbs(), it replaces a loop using 
>> smp_call_function_single()
>> by a single call to smp_call_function_mask().
>>
> Would you be kind enough to copy the implementation of 
> smp_call_function_mask
> into external_module_compat.h so older kernel's will still work after 
> this commited.

Yes, I can, I think the best solution is to move the old KVM code using 
smp_function_call_single() to external_module_compat.h.

Laurent
-- 
---------------- Laurent.Vivier-6ktuUTfB/bM@public.gmane.org  -----------------
"Given enough eyeballs, all bugs are shallow" E. S. Raymond


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Re: [PATCH] kvm uses smp_call_function_mask() inkvm_flush_remote_tlbs()

[Avi Kivity]

Laurent Vivier wrote:
> Dor Laor a écrit :
>   
>> Laurent Vivier wrote:
>>     
>>> This patches can be applied only on kvm-updates-2.6.24 and 
>>> kvm-updates-2.6.25
>>> as it needs smp_call_function_mask().
>>>
>>> In kvm_flush_remote_tlbs(), it replaces a loop using 
>>> smp_call_function_single()
>>> by a single call to smp_call_function_mask().
>>>
>>>       
>> Would you be kind enough to copy the implementation of 
>> smp_call_function_mask
>> into external_module_compat.h so older kernel's will still work after 
>> this commited.
>>     
>
> Yes, I can, I think the best solution is to move the old KVM code using 
> smp_function_call_single() to external_module_compat.h.
>   

Agree, s_c_f_m() is too low-level to go into e_m_c.h.


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Re: [PATCH] kvm uses smp_call_function_mask() in kvm_flush_remote_tlbs()

[Avi Kivity]

Laurent Vivier wrote:
> This patches can be applied only on kvm-updates-2.6.24 and kvm-updates-2.6.25
> as it needs smp_call_function_mask().
>
> In kvm_flush_remote_tlbs(), it replaces a loop using smp_call_function_single() 
> by a single call to smp_call_function_mask().
>
>   

Thanks; I pulled from Linus and applied this on master as well.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Re: [PATCH] kvm uses smp_call_function_mask() in kvm_flush_remote_tlbs()

[Carsten Otte]

Laurent Vivier wrote:
> This patches can be applied only on kvm-updates-2.6.24 and kvm-updates-2.6.25
> as it needs smp_call_function_mask().
There's one thing I don't understand: How is this locked versus cpu 
hotplug? Isn't there an obvious race involved where a cpu unplugged 
after cpu_set()?

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Re: [PATCH] kvm uses smp_call_function_mask() in kvm_flush_remote_tlbs()

[Avi Kivity]

Carsten Otte wrote:
> Laurent Vivier wrote:
>   
>> This patches can be applied only on kvm-updates-2.6.24 and kvm-updates-2.6.25
>> as it needs smp_call_function_mask().
>>     
> There's one thing I don't understand: How is this locked versus cpu 
> hotplug? Isn't there an obvious race involved where a cpu unplugged 
> after cpu_set()?
>
>   

A cpu unplug first evicts all tasks from the victim cpu.  During this 
process, it calls kvm via a hotplug notifier, which notifier calls 
vcpu_clear() for all vcpus that are resident on that cpu.  So after it 
runs, there won't be any vcpus on the dead cpu.

The notifier is called from within stop_machine_run(), which ensures 
that no task is running on any cpu, so by the time the cpu is being 
unplugged, any concurrent call to kvm_flush_remote_tlbs() will have ended.

It's a miracle that it works.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Re: [PATCH] kvm uses smp_call_function_mask() in kvm_flush_remote_tlbs()

[Avi Kivity]

Avi Kivity wrote:
> Carsten Otte wrote:
>   
>> Laurent Vivier wrote:
>>   
>>     
>>> This patches can be applied only on kvm-updates-2.6.24 and kvm-updates-2.6.25
>>> as it needs smp_call_function_mask().
>>>     
>>>       
>> There's one thing I don't understand: How is this locked versus cpu 
>> hotplug? Isn't there an obvious race involved where a cpu unplugged 
>> after cpu_set()?
>>
>>   
>>     
>
> A cpu unplug first evicts all tasks from the victim cpu.  During this 
> process, it calls kvm via a hotplug notifier, which notifier calls 
> vcpu_clear() for all vcpus that are resident on that cpu.  So after it 
> runs, there won't be any vcpus on the dead cpu.
>
> The notifier is called from within stop_machine_run(), which ensures 
> that no task is running on any cpu, so by the time the cpu is being 
> unplugged, any concurrent call to kvm_flush_remote_tlbs() will have ended.
>
> It's a miracle that it works.
>
>   

Miracle indeed.  We need to disable preemption in 
kvm_flush_remote_tlbs(), otherwise the whole story falls apart.


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Re: [PATCH] kvm uses smp_call_function_mask() in kvm_flush_remote_tlbs()

[Carsten Otte]

Avi Kivity wrote:
> A cpu unplug first evicts all tasks from the victim cpu.  During this 
> process, it calls kvm via a hotplug notifier, which notifier calls 
> vcpu_clear() for all vcpus that are resident on that cpu.  So after it 
> runs, there won't be any vcpus on the dead cpu.
> 
> The notifier is called from within stop_machine_run(), which ensures 
> that no task is running on any cpu, so by the time the cpu is being 
> unplugged, any concurrent call to kvm_flush_remote_tlbs() will have ended.
> 
> It's a miracle that it works.
Really interresting. I put it on the "neat code I want to read" list 
for long and dark winter evenings :-).

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/