* [PATCH] Maybe avoid a IPI between the cpu cores
@ 2012-12-04 3:08 yi li
2012-12-04 3:49 ` yi li
2012-12-04 11:20 ` Gleb Natapov
0 siblings, 2 replies; 6+ messages in thread
From: yi li @ 2012-12-04 3:08 UTC (permalink / raw)
To: kvm; +Cc: Alex Williamson
The cpu inject the interrupt to vcpu which vcpu->cpu is the same as it.
And it maybe avoid a IPI between the cpu core.
Signed-off-by: Yi Li <yilikernel@gmail.com>
--- linux/virt/kvm/irq_comm.c 2012-12-04 10:14:57.711024619 +0800
+++ linux/virt/kvm/irq_comm.c 2012-12-04 11:01:27.728859597 +0800
@@ -64,9 +64,10 @@ inline static bool kvm_is_dm_lowest_prio
int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
struct kvm_lapic_irq *irq)
{
- int i, r = -1;
+ int i, cpu, r = -1;
struct kvm_vcpu *vcpu, *lowest = NULL;
+ cpu = get_cpu();
if (irq->dest_mode == 0 && irq->dest_id == 0xff &&
kvm_is_dm_lowest_prio(irq)) {
printk(KERN_INFO "kvm: apic: phys broadcast and lowest prio\n");
@@ -89,13 +90,17 @@ int kvm_irq_delivery_to_apic(struct kvm
r = 0;
r += kvm_apic_set_irq(vcpu, irq);
} else if (kvm_lapic_enabled(vcpu)) {
- if (!lowest)
+ if(vcpu->cpu == cpu) {
+ lowest = vcpu;
+ break;
+ }
+ else if (!lowest)
lowest = vcpu;
else if (kvm_apic_compare_prio(vcpu, lowest) < 0)
lowest = vcpu;
}
}
-
+ put_cpu();
if (lowest)
r = kvm_apic_set_irq(lowest, irq);
YiLi
Thanks
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] Maybe avoid a IPI between the cpu cores
2012-12-04 3:08 [PATCH] Maybe avoid a IPI between the cpu cores yi li
@ 2012-12-04 3:49 ` yi li
2012-12-04 11:20 ` Gleb Natapov
1 sibling, 0 replies; 6+ messages in thread
From: yi li @ 2012-12-04 3:49 UTC (permalink / raw)
To: kvm; +Cc: Alex Williamson, weikong.cn
Maybe the patch have a risk to break the process of lapic.just avoid
a IPI between the cpu core asap.
Thanks
YiLi
2012/12/4 yi li <yilikernel@gmail.com>:
> The cpu inject the interrupt to vcpu which vcpu->cpu is the same as it.
> And it maybe avoid a IPI between the cpu core.
>
> Signed-off-by: Yi Li <yilikernel@gmail.com>
>
> --- linux/virt/kvm/irq_comm.c 2012-12-04 10:14:57.711024619 +0800
> +++ linux/virt/kvm/irq_comm.c 2012-12-04 11:01:27.728859597 +0800
> @@ -64,9 +64,10 @@ inline static bool kvm_is_dm_lowest_prio
> int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
> struct kvm_lapic_irq *irq)
> {
> - int i, r = -1;
> + int i, cpu, r = -1;
> struct kvm_vcpu *vcpu, *lowest = NULL;
>
> + cpu = get_cpu();
> if (irq->dest_mode == 0 && irq->dest_id == 0xff &&
> kvm_is_dm_lowest_prio(irq)) {
> printk(KERN_INFO "kvm: apic: phys broadcast and lowest prio\n");
> @@ -89,13 +90,17 @@ int kvm_irq_delivery_to_apic(struct kvm
> r = 0;
> r += kvm_apic_set_irq(vcpu, irq);
> } else if (kvm_lapic_enabled(vcpu)) {
> - if (!lowest)
> + if(vcpu->cpu == cpu) {
> + lowest = vcpu;
> + break;
> + }
> + else if (!lowest)
> lowest = vcpu;
> else if (kvm_apic_compare_prio(vcpu, lowest) < 0)
> lowest = vcpu;
> }
> }
> -
> + put_cpu();
> if (lowest)
> r = kvm_apic_set_irq(lowest, irq);
>
>
> YiLi
> Thanks
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] Maybe avoid a IPI between the cpu cores
2012-12-04 3:08 [PATCH] Maybe avoid a IPI between the cpu cores yi li
2012-12-04 3:49 ` yi li
@ 2012-12-04 11:20 ` Gleb Natapov
2012-12-04 17:32 ` yi li
1 sibling, 1 reply; 6+ messages in thread
From: Gleb Natapov @ 2012-12-04 11:20 UTC (permalink / raw)
To: yi li; +Cc: kvm, Alex Williamson
On Tue, Dec 04, 2012 at 11:08:41AM +0800, yi li wrote:
> The cpu inject the interrupt to vcpu which vcpu->cpu is the same as it.
> And it maybe avoid a IPI between the cpu core.
>
How often this happens in your testing? Regardless we cannot disable
preemption while iterating over all vcpus. Low latency people will kill
us.
> Signed-off-by: Yi Li <yilikernel@gmail.com>
>
> --- linux/virt/kvm/irq_comm.c 2012-12-04 10:14:57.711024619 +0800
> +++ linux/virt/kvm/irq_comm.c 2012-12-04 11:01:27.728859597 +0800
> @@ -64,9 +64,10 @@ inline static bool kvm_is_dm_lowest_prio
> int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
> struct kvm_lapic_irq *irq)
> {
> - int i, r = -1;
> + int i, cpu, r = -1;
> struct kvm_vcpu *vcpu, *lowest = NULL;
>
> + cpu = get_cpu();
> if (irq->dest_mode == 0 && irq->dest_id == 0xff &&
> kvm_is_dm_lowest_prio(irq)) {
> printk(KERN_INFO "kvm: apic: phys broadcast and lowest prio\n");
> @@ -89,13 +90,17 @@ int kvm_irq_delivery_to_apic(struct kvm
> r = 0;
> r += kvm_apic_set_irq(vcpu, irq);
> } else if (kvm_lapic_enabled(vcpu)) {
> - if (!lowest)
> + if(vcpu->cpu == cpu) {
> + lowest = vcpu;
> + break;
> + }
> + else if (!lowest)
> lowest = vcpu;
> else if (kvm_apic_compare_prio(vcpu, lowest) < 0)
> lowest = vcpu;
> }
> }
> -
> + put_cpu();
> if (lowest)
> r = kvm_apic_set_irq(lowest, irq);
>
>
> YiLi
> Thanks
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Gleb.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] Maybe avoid a IPI between the cpu cores
2012-12-04 11:20 ` Gleb Natapov
@ 2012-12-04 17:32 ` yi li
2012-12-04 17:48 ` yi li
2012-12-05 10:31 ` Gleb Natapov
0 siblings, 2 replies; 6+ messages in thread
From: yi li @ 2012-12-04 17:32 UTC (permalink / raw)
To: Gleb Natapov; +Cc: kvm, Alex Williamson, weikong.cn
My test:
1:guest os has 2 vcpu and has a virtio netcard, also host os just has
2 cpu cores, and do not do any smp_affinity
2:using the jprobe to probe the function kvm_vcpu_kick on the host
.......
static int i=0;
int jkvm_vcpu_kick(struct kvm_vcpu *vcpu)
{
int cpu = vcpu->cpu;
printk("vcpu->cpu is %d\n",cpu);
printk("the pcpu to handle the inject irq is %d\n", smp_processor_id());
if(cpu != smp_processor_id())
{
i = i+ 1;
}
printk("the ipi sum is %d\n", i);
/* Always end with a call to jprobe_return(). */
jprobe_return();
return 0;
}
..........................
run the command ping the guestos virtio netcard
3: test result,
sometimes, it is very bad, you see blow, there has 14 ipi (64-50)
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.009937] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.009940] the pcpu to
handle the inject irq is 0
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.009942] the ipi sum is 50
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010008] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010011] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010013] the ipi sum is 50
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010051] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010052] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010055] the ipi sum is 50
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010093] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010094] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010096] the ipi sum is 50
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.035829] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.035831] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.035833] the ipi sum is 50
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106226] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106229] the pcpu to
handle the inject irq is 0
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106232] the ipi sum is 51
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106309] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106311] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106313] the ipi sum is 51
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106359] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106360] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106361] the ipi sum is 51
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106385] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106386] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106387] the ipi sum is 51
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129867] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129869] the pcpu to
handle the inject irq is 0
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129871] the ipi sum is 52
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129937] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129939] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129942] the ipi sum is 52
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129979] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129981] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129983] the ipi sum is 52
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.130024] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.130025] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.130026] the ipi sum is 52
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.137952] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.137954] the pcpu to
handle the inject irq is 0
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.137956] the ipi sum is 53
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138022] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138024] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138026] the ipi sum is 53
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138064] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138066] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138068] the ipi sum is 53
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138106] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138107] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138108] the ipi sum is 53
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145844] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145846] the pcpu to
handle the inject irq is 0
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145848] the ipi sum is 54
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145911] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145913] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145915] the ipi sum is 54
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145952] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145954] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145956] the ipi sum is 54
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145993] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145995] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.146002] the ipi sum is 54
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187846] vcpu->cpu is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187848] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187850] the ipi sum is 54
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187898] vcpu->cpu is 0
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187900] the pcpu to
handle the inject irq is 1
Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187902] the ipi sum is 55
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.265951] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.265954] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.265956] the ipi sum is 56
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266037] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266039] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266041] the ipi sum is 56
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266082] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266083] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266084] the ipi sum is 56
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266109] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266110] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266111] the ipi sum is 56
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274248] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274250] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274252] the ipi sum is 57
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274318] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274320] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274322] the ipi sum is 57
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274370] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274372] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274374] the ipi sum is 57
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274413] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274414] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274415] the ipi sum is 57
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.281978] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.281981] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.281983] the ipi sum is 58
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282046] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282048] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282050] the ipi sum is 58
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282100] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282102] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282104] the ipi sum is 58
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282144] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282145] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282146] the ipi sum is 58
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289860] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289862] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289864] the ipi sum is 59
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289929] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289931] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289933] the ipi sum is 59
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289972] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289974] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289976] the ipi sum is 59
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.290018] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.290020] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.290021] the ipi sum is 59
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.305924] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.305926] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.305928] the ipi sum is 60
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.305997] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306000] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306002] the ipi sum is 60
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306040] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306042] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306044] the ipi sum is 60
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306082] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306083] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306085] the ipi sum is 60
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.496784] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.496787] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.496790] the ipi sum is 61
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.498717] vcpu->cpu is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.498719] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.498720] the ipi sum is 62
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499806] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499808] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499810] the ipi sum is 62
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499844] vcpu->cpu is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499847] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499849] the ipi sum is 62
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503811] vcpu->cpu is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503814] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503816] the ipi sum is 62
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503848] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503850] the pcpu to
handle the inject irq is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503852] the ipi sum is 62
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.507808] vcpu->cpu is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.507811] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.507813] the ipi sum is 62
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.511805] vcpu->cpu is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.511807] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.511809] the ipi sum is 62
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.515807] vcpu->cpu is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.515809] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.515811] the ipi sum is 62
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.596396] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.596399] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.596402] the ipi sum is 63
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687796] vcpu->cpu is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687799] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687801] the ipi sum is 63
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687862] vcpu->cpu is 1
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687864] the pcpu to
handle the inject irq is 0
Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687866] the ipi sum is 64
4: the above result just happen once, when i reboot the host, the IPI
is not increase occured by kvm_vcpu_kick, i will check it more detail.
YiLi
Thanks
2012/12/4 Gleb Natapov <gleb@redhat.com>:
> On Tue, Dec 04, 2012 at 11:08:41AM +0800, yi li wrote:
>> The cpu inject the interrupt to vcpu which vcpu->cpu is the same as it.
>> And it maybe avoid a IPI between the cpu core.
>>
> How often this happens in your testing? Regardless we cannot disable
> preemption while iterating over all vcpus. Low latency people will kill
> us.
>
>> Signed-off-by: Yi Li <yilikernel@gmail.com>
>>
>> --- linux/virt/kvm/irq_comm.c 2012-12-04 10:14:57.711024619 +0800
>> +++ linux/virt/kvm/irq_comm.c 2012-12-04 11:01:27.728859597 +0800
>> @@ -64,9 +64,10 @@ inline static bool kvm_is_dm_lowest_prio
>> int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
>> struct kvm_lapic_irq *irq)
>> {
>> - int i, r = -1;
>> + int i, cpu, r = -1;
>> struct kvm_vcpu *vcpu, *lowest = NULL;
>>
>> + cpu = get_cpu();
>> if (irq->dest_mode == 0 && irq->dest_id == 0xff &&
>> kvm_is_dm_lowest_prio(irq)) {
>> printk(KERN_INFO "kvm: apic: phys broadcast and lowest prio\n");
>> @@ -89,13 +90,17 @@ int kvm_irq_delivery_to_apic(struct kvm
>> r = 0;
>> r += kvm_apic_set_irq(vcpu, irq);
>> } else if (kvm_lapic_enabled(vcpu)) {
>> - if (!lowest)
>> + if(vcpu->cpu == cpu) {
>> + lowest = vcpu;
>> + break;
>> + }
>> + else if (!lowest)
>> lowest = vcpu;
>> else if (kvm_apic_compare_prio(vcpu, lowest) < 0)
>> lowest = vcpu;
>> }
>> }
>> -
>> + put_cpu();
>> if (lowest)
>> r = kvm_apic_set_irq(lowest, irq);
>>
>>
>> YiLi
>> Thanks
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> --
> Gleb.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] Maybe avoid a IPI between the cpu cores
2012-12-04 17:32 ` yi li
@ 2012-12-04 17:48 ` yi li
2012-12-05 10:31 ` Gleb Natapov
1 sibling, 0 replies; 6+ messages in thread
From: yi li @ 2012-12-04 17:48 UTC (permalink / raw)
To: Gleb Natapov; +Cc: kvm, Alex Williamson, weikong.cn
1: i have do ti again, result as blow:
..........................................................................
[ 681.297899] the pcpu to handle the inject irq is 0
[ 681.297901] the ipi sum is 10432
root@liyi-ThinkPad-X200:~/myprobe# dmesg | grep 680.
[ 680.065465] vcpu->cpu is 0
[ 680.065469] the pcpu to handle the inject irq is 0
[ 680.065472] the ipi sum is 10427
[ 680.065612] vcpu->cpu is 0
[ 680.065614] the pcpu to handle the inject irq is 0
[ 680.065616] the ipi sum is 10427
[ 680.065671] vcpu->cpu is 1
[ 680.065672] the pcpu to handle the inject irq is 0
[ 680.065674] the ipi sum is 10428
[ 680.297680] vcpu->cpu is 1
[ 680.297683] the pcpu to handle the inject irq is 1
[ 680.297685] the ipi sum is 10428
[ 680.297972] vcpu->cpu is 1
[ 680.297974] the pcpu to handle the inject irq is 0
[ 680.297975] the ipi sum is 10429
[ 680.549575] vcpu->cpu is 1
[ 680.549579] the pcpu to handle the inject irq is 1
[ 680.549581] the ipi sum is 10429
[ 680.565505] vcpu->cpu is 1
[ 680.565508] the pcpu to handle the inject irq is 1
[ 680.565510] the ipi sum is 10429
[ 680.565555] vcpu->cpu is 0
[ 680.565557] the pcpu to handle the inject irq is 1
[ 680.565559] the ipi sum is 10430
so we can see the IPI num is 3 between [680.065465] and [
680.565559], the guest os has guest two vcpus, so if we have more, i
think the ipi num will a little more.
2: i think your thinking is right about preempt disable problem,and i
have no good idea for it.
Thanks
YiLi
2012/12/5 yi li <yilikernel@gmail.com>:
> My test:
>
> 1:guest os has 2 vcpu and has a virtio netcard, also host os just has
> 2 cpu cores, and do not do any smp_affinity
>
> 2:using the jprobe to probe the function kvm_vcpu_kick on the host
> .......
> static int i=0;
> int jkvm_vcpu_kick(struct kvm_vcpu *vcpu)
> {
> int cpu = vcpu->cpu;
> printk("vcpu->cpu is %d\n",cpu);
> printk("the pcpu to handle the inject irq is %d\n", smp_processor_id());
> if(cpu != smp_processor_id())
> {
> i = i+ 1;
> }
> printk("the ipi sum is %d\n", i);
> /* Always end with a call to jprobe_return(). */
> jprobe_return();
> return 0;
> }
> ..........................
>
> run the command ping the guestos virtio netcard
>
> 3: test result,
> sometimes, it is very bad, you see blow, there has 14 ipi (64-50)
>
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.009937] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.009940] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.009942] the ipi sum is 50
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010008] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010011] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010013] the ipi sum is 50
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010051] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010052] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010055] the ipi sum is 50
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010093] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010094] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010096] the ipi sum is 50
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.035829] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.035831] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.035833] the ipi sum is 50
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106226] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106229] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106232] the ipi sum is 51
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106309] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106311] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106313] the ipi sum is 51
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106359] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106360] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106361] the ipi sum is 51
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106385] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106386] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106387] the ipi sum is 51
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129867] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129869] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129871] the ipi sum is 52
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129937] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129939] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129942] the ipi sum is 52
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129979] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129981] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129983] the ipi sum is 52
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.130024] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.130025] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.130026] the ipi sum is 52
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.137952] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.137954] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.137956] the ipi sum is 53
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138022] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138024] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138026] the ipi sum is 53
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138064] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138066] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138068] the ipi sum is 53
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138106] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138107] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138108] the ipi sum is 53
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145844] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145846] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145848] the ipi sum is 54
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145911] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145913] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145915] the ipi sum is 54
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145952] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145954] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145956] the ipi sum is 54
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145993] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145995] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.146002] the ipi sum is 54
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187846] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187848] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187850] the ipi sum is 54
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187898] vcpu->cpu is 0
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187900] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187902] the ipi sum is 55
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.265951] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.265954] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.265956] the ipi sum is 56
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266037] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266039] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266041] the ipi sum is 56
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266082] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266083] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266084] the ipi sum is 56
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266109] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266110] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266111] the ipi sum is 56
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274248] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274250] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274252] the ipi sum is 57
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274318] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274320] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274322] the ipi sum is 57
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274370] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274372] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274374] the ipi sum is 57
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274413] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274414] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274415] the ipi sum is 57
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.281978] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.281981] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.281983] the ipi sum is 58
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282046] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282048] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282050] the ipi sum is 58
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282100] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282102] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282104] the ipi sum is 58
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282144] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282145] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282146] the ipi sum is 58
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289860] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289862] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289864] the ipi sum is 59
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289929] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289931] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289933] the ipi sum is 59
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289972] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289974] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289976] the ipi sum is 59
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.290018] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.290020] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.290021] the ipi sum is 59
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.305924] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.305926] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.305928] the ipi sum is 60
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.305997] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306000] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306002] the ipi sum is 60
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306040] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306042] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306044] the ipi sum is 60
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306082] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306083] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306085] the ipi sum is 60
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.496784] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.496787] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.496790] the ipi sum is 61
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.498717] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.498719] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.498720] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499806] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499808] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499810] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499844] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499847] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499849] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503811] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503814] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503816] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503848] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503850] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503852] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.507808] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.507811] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.507813] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.511805] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.511807] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.511809] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.515807] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.515809] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.515811] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.596396] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.596399] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.596402] the ipi sum is 63
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687796] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687799] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687801] the ipi sum is 63
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687862] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687864] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687866] the ipi sum is 64
>
> 4: the above result just happen once, when i reboot the host, the IPI
> is not increase occured by kvm_vcpu_kick, i will check it more detail.
>
> YiLi
> Thanks
>
>
> 2012/12/4 Gleb Natapov <gleb@redhat.com>:
>> On Tue, Dec 04, 2012 at 11:08:41AM +0800, yi li wrote:
>>> The cpu inject the interrupt to vcpu which vcpu->cpu is the same as it.
>>> And it maybe avoid a IPI between the cpu core.
>>>
>> How often this happens in your testing? Regardless we cannot disable
>> preemption while iterating over all vcpus. Low latency people will kill
>> us.
>>
>>> Signed-off-by: Yi Li <yilikernel@gmail.com>
>>>
>>> --- linux/virt/kvm/irq_comm.c 2012-12-04 10:14:57.711024619 +0800
>>> +++ linux/virt/kvm/irq_comm.c 2012-12-04 11:01:27.728859597 +0800
>>> @@ -64,9 +64,10 @@ inline static bool kvm_is_dm_lowest_prio
>>> int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
>>> struct kvm_lapic_irq *irq)
>>> {
>>> - int i, r = -1;
>>> + int i, cpu, r = -1;
>>> struct kvm_vcpu *vcpu, *lowest = NULL;
>>>
>>> + cpu = get_cpu();
>>> if (irq->dest_mode == 0 && irq->dest_id == 0xff &&
>>> kvm_is_dm_lowest_prio(irq)) {
>>> printk(KERN_INFO "kvm: apic: phys broadcast and lowest prio\n");
>>> @@ -89,13 +90,17 @@ int kvm_irq_delivery_to_apic(struct kvm
>>> r = 0;
>>> r += kvm_apic_set_irq(vcpu, irq);
>>> } else if (kvm_lapic_enabled(vcpu)) {
>>> - if (!lowest)
>>> + if(vcpu->cpu == cpu) {
>>> + lowest = vcpu;
>>> + break;
>>> + }
>>> + else if (!lowest)
>>> lowest = vcpu;
>>> else if (kvm_apic_compare_prio(vcpu, lowest) < 0)
>>> lowest = vcpu;
>>> }
>>> }
>>> -
>>> + put_cpu();
>>> if (lowest)
>>> r = kvm_apic_set_irq(lowest, irq);
>>>
>>>
>>> YiLi
>>> Thanks
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>> --
>> Gleb.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] Maybe avoid a IPI between the cpu cores
2012-12-04 17:32 ` yi li
2012-12-04 17:48 ` yi li
@ 2012-12-05 10:31 ` Gleb Natapov
1 sibling, 0 replies; 6+ messages in thread
From: Gleb Natapov @ 2012-12-05 10:31 UTC (permalink / raw)
To: yi li; +Cc: kvm, Alex Williamson, weikong.cn
On Wed, Dec 05, 2012 at 01:32:15AM +0800, yi li wrote:
> My test:
>
> 1:guest os has 2 vcpu and has a virtio netcard, also host os just has
> 2 cpu cores, and do not do any smp_affinity
>
What about running the same test but on 32 host cpus. I expect result to
be much different.
> 2:using the jprobe to probe the function kvm_vcpu_kick on the host
> .......
> static int i=0;
> int jkvm_vcpu_kick(struct kvm_vcpu *vcpu)
> {
> int cpu = vcpu->cpu;
> printk("vcpu->cpu is %d\n",cpu);
> printk("the pcpu to handle the inject irq is %d\n", smp_processor_id());
> if(cpu != smp_processor_id())
> {
> i = i+ 1;
> }
> printk("the ipi sum is %d\n", i);
> /* Always end with a call to jprobe_return(). */
> jprobe_return();
> return 0;
> }
> ..........................
>
> run the command ping the guestos virtio netcard
>
> 3: test result,
> sometimes, it is very bad, you see blow, there has 14 ipi (64-50)
>
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.009937] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.009940] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.009942] the ipi sum is 50
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010008] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010011] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010013] the ipi sum is 50
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010051] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010052] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010055] the ipi sum is 50
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010093] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010094] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.010096] the ipi sum is 50
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.035829] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.035831] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.035833] the ipi sum is 50
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106226] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106229] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106232] the ipi sum is 51
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106309] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106311] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106313] the ipi sum is 51
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106359] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106360] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106361] the ipi sum is 51
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106385] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106386] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.106387] the ipi sum is 51
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129867] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129869] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129871] the ipi sum is 52
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129937] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129939] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129942] the ipi sum is 52
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129979] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129981] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.129983] the ipi sum is 52
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.130024] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.130025] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.130026] the ipi sum is 52
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.137952] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.137954] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.137956] the ipi sum is 53
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138022] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138024] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138026] the ipi sum is 53
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138064] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138066] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138068] the ipi sum is 53
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138106] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138107] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.138108] the ipi sum is 53
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145844] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145846] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145848] the ipi sum is 54
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145911] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145913] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145915] the ipi sum is 54
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145952] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145954] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145956] the ipi sum is 54
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145993] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.145995] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.146002] the ipi sum is 54
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187846] vcpu->cpu is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187848] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187850] the ipi sum is 54
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187898] vcpu->cpu is 0
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187900] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:52 liyi-ThinkPad-X200 kernel: [11470.187902] the ipi sum is 55
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.265951] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.265954] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.265956] the ipi sum is 56
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266037] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266039] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266041] the ipi sum is 56
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266082] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266083] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266084] the ipi sum is 56
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266109] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266110] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.266111] the ipi sum is 56
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274248] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274250] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274252] the ipi sum is 57
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274318] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274320] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274322] the ipi sum is 57
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274370] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274372] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274374] the ipi sum is 57
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274413] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274414] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.274415] the ipi sum is 57
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.281978] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.281981] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.281983] the ipi sum is 58
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282046] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282048] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282050] the ipi sum is 58
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282100] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282102] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282104] the ipi sum is 58
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282144] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282145] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.282146] the ipi sum is 58
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289860] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289862] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289864] the ipi sum is 59
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289929] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289931] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289933] the ipi sum is 59
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289972] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289974] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.289976] the ipi sum is 59
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.290018] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.290020] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.290021] the ipi sum is 59
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.305924] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.305926] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.305928] the ipi sum is 60
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.305997] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306000] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306002] the ipi sum is 60
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306040] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306042] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306044] the ipi sum is 60
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306082] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306083] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.306085] the ipi sum is 60
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.496784] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.496787] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.496790] the ipi sum is 61
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.498717] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.498719] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.498720] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499806] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499808] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499810] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499844] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499847] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.499849] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503811] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503814] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503816] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503848] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503850] the pcpu to
> handle the inject irq is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.503852] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.507808] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.507811] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.507813] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.511805] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.511807] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.511809] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.515807] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.515809] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.515811] the ipi sum is 62
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.596396] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.596399] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.596402] the ipi sum is 63
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687796] vcpu->cpu is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687799] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687801] the ipi sum is 63
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687862] vcpu->cpu is 1
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687864] the pcpu to
> handle the inject irq is 0
> Dec 5 00:04:53 liyi-ThinkPad-X200 kernel: [11470.687866] the ipi sum is 64
>
> 4: the above result just happen once, when i reboot the host, the IPI
> is not increase occured by kvm_vcpu_kick, i will check it more detail.
>
> YiLi
> Thanks
>
>
> 2012/12/4 Gleb Natapov <gleb@redhat.com>:
> > On Tue, Dec 04, 2012 at 11:08:41AM +0800, yi li wrote:
> >> The cpu inject the interrupt to vcpu which vcpu->cpu is the same as it.
> >> And it maybe avoid a IPI between the cpu core.
> >>
> > How often this happens in your testing? Regardless we cannot disable
> > preemption while iterating over all vcpus. Low latency people will kill
> > us.
> >
> >> Signed-off-by: Yi Li <yilikernel@gmail.com>
> >>
> >> --- linux/virt/kvm/irq_comm.c 2012-12-04 10:14:57.711024619 +0800
> >> +++ linux/virt/kvm/irq_comm.c 2012-12-04 11:01:27.728859597 +0800
> >> @@ -64,9 +64,10 @@ inline static bool kvm_is_dm_lowest_prio
> >> int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
> >> struct kvm_lapic_irq *irq)
> >> {
> >> - int i, r = -1;
> >> + int i, cpu, r = -1;
> >> struct kvm_vcpu *vcpu, *lowest = NULL;
> >>
> >> + cpu = get_cpu();
> >> if (irq->dest_mode == 0 && irq->dest_id == 0xff &&
> >> kvm_is_dm_lowest_prio(irq)) {
> >> printk(KERN_INFO "kvm: apic: phys broadcast and lowest prio\n");
> >> @@ -89,13 +90,17 @@ int kvm_irq_delivery_to_apic(struct kvm
> >> r = 0;
> >> r += kvm_apic_set_irq(vcpu, irq);
> >> } else if (kvm_lapic_enabled(vcpu)) {
> >> - if (!lowest)
> >> + if(vcpu->cpu == cpu) {
> >> + lowest = vcpu;
> >> + break;
> >> + }
> >> + else if (!lowest)
> >> lowest = vcpu;
> >> else if (kvm_apic_compare_prio(vcpu, lowest) < 0)
> >> lowest = vcpu;
> >> }
> >> }
> >> -
> >> + put_cpu();
> >> if (lowest)
> >> r = kvm_apic_set_irq(lowest, irq);
> >>
> >>
> >> YiLi
> >> Thanks
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe kvm" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at http://vger.kernel.org/majordomo-info.html
> >
> > --
> > Gleb.
--
Gleb.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-12-05 10:31 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-12-04 3:08 [PATCH] Maybe avoid a IPI between the cpu cores yi li
2012-12-04 3:49 ` yi li
2012-12-04 11:20 ` Gleb Natapov
2012-12-04 17:32 ` yi li
2012-12-04 17:48 ` yi li
2012-12-05 10:31 ` Gleb Natapov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).