[PATCH] kvm: x86: Fix debug typo error in lapic

[PATCH] kvm: x86: Fix debug typo error in lapic

[Chen Fan]

fix the 'vcpi' typos when apic_debug is enabled.

Signed-off-by: Chen Fan <chen.fan.fnst@cn.fujitsu.com>
---
 arch/x86/kvm/lapic.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index dec48bf..ce736ec 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -435,7 +435,7 @@ static bool pv_eoi_get_pending(struct kvm_vcpu *vcpu)
 	u8 val;
 	if (pv_eoi_get_user(vcpu, &val) < 0)
 		apic_debug("Can't read EOI MSR value: 0x%llx\n",
-			   (unsigned long long)vcpi->arch.pv_eoi.msr_val);
+			   (unsigned long long)vcpu->arch.pv_eoi.msr_val);
 	return val & 0x1;
 }
 
@@ -443,7 +443,7 @@ static void pv_eoi_set_pending(struct kvm_vcpu *vcpu)
 {
 	if (pv_eoi_put_user(vcpu, KVM_PV_EOI_ENABLED) < 0) {
 		apic_debug("Can't set EOI MSR value: 0x%llx\n",
-			   (unsigned long long)vcpi->arch.pv_eoi.msr_val);
+			   (unsigned long long)vcpu->arch.pv_eoi.msr_val);
 		return;
 	}
 	__set_bit(KVM_APIC_PV_EOI_PENDING, &vcpu->arch.apic_attention);
@@ -453,7 +453,7 @@ static void pv_eoi_clr_pending(struct kvm_vcpu *vcpu)
 {
 	if (pv_eoi_put_user(vcpu, KVM_PV_EOI_DISABLED) < 0) {
 		apic_debug("Can't clear EOI MSR value: 0x%llx\n",
-			   (unsigned long long)vcpi->arch.pv_eoi.msr_val);
+			   (unsigned long long)vcpu->arch.pv_eoi.msr_val);
 		return;
 	}
 	__clear_bit(KVM_APIC_PV_EOI_PENDING, &vcpu->arch.apic_attention);
-- 
1.8.1.4

Re: [PATCH] kvm: x86: Fix debug typo error in lapic

[Marcelo Tosatti]

On Thu, Jan 02, 2014 at 05:14:11PM +0800, Chen Fan wrote:
> fix the 'vcpi' typos when apic_debug is enabled.
> 
> Signed-off-by: Chen Fan <chen.fan.fnst@cn.fujitsu.com>
> ---
>  arch/x86/kvm/lapic.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)

Applied, thanks.

Re: [PATCH] kvm: x86: Fix debug typo error in lapic

[Hu Yaohui]

Hi guys,
I think you should be pretty familiar with lapic. I would really
appreciate it if someone could shed some lights on my problem
regarding Guest TLB flush IPI.
Supposed we get two vcpus 0 and 1.
When vcpu#0 wants to invalidate the tlb entry on vcpu#1. An IPI will
be generated by lapic on vcpu#0 by writing to ICR which will cause a
vmexit.
apic_send_ipi->kvm_irq_delivery_to_apic->kvm_apic_set_irq->__apic_accept_irq
In __apic_accept_irq, it will call kvm_make_request, kvm_vcpu_kick.
If vcpu#1 in guest mode, how can it receives this IPI immediately, or
the stale tlb entry could be accessed. Thanks for your time!

Best Wishes,
Yaohui Hu

On Wed, Jan 8, 2014 at 4:10 PM, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> On Thu, Jan 02, 2014 at 05:14:11PM +0800, Chen Fan wrote:
>> fix the 'vcpi' typos when apic_debug is enabled.
>>
>> Signed-off-by: Chen Fan <chen.fan.fnst@cn.fujitsu.com>
>> ---
>>  arch/x86/kvm/lapic.c | 6 +++---
>>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> Applied, thanks.
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] kvm: x86: Fix debug typo error in lapic

[Marcelo Tosatti]

On Wed, Jan 08, 2014 at 06:14:15PM -0500, Hu Yaohui wrote:
> Hi guys,
> I think you should be pretty familiar with lapic. I would really
> appreciate it if someone could shed some lights on my problem
> regarding Guest TLB flush IPI.
> Supposed we get two vcpus 0 and 1.
> When vcpu#0 wants to invalidate the tlb entry on vcpu#1. An IPI will
> be generated by lapic on vcpu#0 by writing to ICR which will cause a
> vmexit.
> apic_send_ipi->kvm_irq_delivery_to_apic->kvm_apic_set_irq->__apic_accept_irq
> In __apic_accept_irq, it will call kvm_make_request, kvm_vcpu_kick.
> If vcpu#1 in guest mode, how can it receives this IPI immediately, or
> the stale tlb entry could be accessed. Thanks for your time!

Two possibilities: 

2) Hardware does not support APIC virtualization: kvm_vcpu_kick sends an
host-IPI to the remote vcpu, and if that vcpu is in guest mode, a VM-exit
(exit reason: external interrupt) will be triggered due to the host-IPI.
Then on VM-entry (inject_pending_event) the guest-IPI is injected.

2) Host CPU supports APIC virtualization (see commit 83d4c286931c and
Intel's documentation): 
A bit is set in the posted interrupt section, and a special host-IPI is
delivered to the target cpu where the guest vcpu is scheduled 
(vmx_deliver_posted_interrupt) which causes the hardware to 
inject the vector (without VM-exit).

Re: [PATCH] kvm: x86: Fix debug typo error in lapic

[Hu Yaohui]

Thanks a lot Marcelo!

On Wed, Jan 8, 2014 at 6:25 PM, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> On Wed, Jan 08, 2014 at 06:14:15PM -0500, Hu Yaohui wrote:
>> Hi guys,
>> I think you should be pretty familiar with lapic. I would really
>> appreciate it if someone could shed some lights on my problem
>> regarding Guest TLB flush IPI.
>> Supposed we get two vcpus 0 and 1.
>> When vcpu#0 wants to invalidate the tlb entry on vcpu#1. An IPI will
>> be generated by lapic on vcpu#0 by writing to ICR which will cause a
>> vmexit.
>> apic_send_ipi->kvm_irq_delivery_to_apic->kvm_apic_set_irq->__apic_accept_irq
>> In __apic_accept_irq, it will call kvm_make_request, kvm_vcpu_kick.
>> If vcpu#1 in guest mode, how can it receives this IPI immediately, or
>> the stale tlb entry could be accessed. Thanks for your time!
>
I am using kvm-kmod-3.2
> Two possibilities:
>
> 2) Hardware does not support APIC virtualization: kvm_vcpu_kick sends an
> host-IPI to the remote vcpu, and if that vcpu is in guest mode, a VM-exit
> (exit reason: external interrupt) will be triggered due to the host-IPI.
> Then on VM-entry (inject_pending_event) the guest-IPI is injected.
>
if vcpu#1 is not on the same pcpu as the vcpu#0, a host-IPI will be sent.
But if they are on the same pcpu, if vcpu#1 is in guest mode. Then the
guest tlb flush IPI
will wait until the next vcpu#1 vmexit. If that's the case. they are
some time that the tlb entry has been
invalidated in vcpu#0, but the corresponding entry in vcpu#1 could
still been accessed, which seems cause some problem.

> 2) Host CPU supports APIC virtualization (see commit 83d4c286931c and
> Intel's documentation):
> A bit is set in the posted interrupt section, and a special host-IPI is
> delivered to the target cpu where the guest vcpu is scheduled
> (vmx_deliver_posted_interrupt) which causes the hardware to
> inject the vector (without VM-exit).
>
>
I did not find this function (vmx_deliver_posted_interrupt) in my kvm
kernel module.
Does that mean my hardware doesn't support APIC virtualization?

Thanks for your time!

Best Wishes,
Yaohui Hu

Re: [PATCH] kvm: x86: Fix debug typo error in lapic

[Hu Yaohui]

Hi Macelo,
I am sorry to bother you again. In your first possibility,
kvm_vcpu_kick sends an host-IPI to the remote vcpu,
and if that vcpu is in guest mode, a VM-exit will be triggered due to
the host-IPI. My question is if the vcpu has
accessed the stale tlb entry before the host-IPI arrives, what will
happen?  Thanks for your time!

Best Wishes,
Yaohui Hu


On Wed, Jan 8, 2014 at 6:35 PM, Hu Yaohui <loki2441@gmail.com> wrote:
> Thanks a lot Marcelo!
>
> On Wed, Jan 8, 2014 at 6:25 PM, Marcelo Tosatti <mtosatti@redhat.com> wrote:
>> On Wed, Jan 08, 2014 at 06:14:15PM -0500, Hu Yaohui wrote:
>>> Hi guys,
>>> I think you should be pretty familiar with lapic. I would really
>>> appreciate it if someone could shed some lights on my problem
>>> regarding Guest TLB flush IPI.
>>> Supposed we get two vcpus 0 and 1.
>>> When vcpu#0 wants to invalidate the tlb entry on vcpu#1. An IPI will
>>> be generated by lapic on vcpu#0 by writing to ICR which will cause a
>>> vmexit.
>>> apic_send_ipi->kvm_irq_delivery_to_apic->kvm_apic_set_irq->__apic_accept_irq
>>> In __apic_accept_irq, it will call kvm_make_request, kvm_vcpu_kick.
>>> If vcpu#1 in guest mode, how can it receives this IPI immediately, or
>>> the stale tlb entry could be accessed. Thanks for your time!
>>
> I am using kvm-kmod-3.2
>> Two possibilities:
>>
>> 2) Hardware does not support APIC virtualization: kvm_vcpu_kick sends an
>> host-IPI to the remote vcpu, and if that vcpu is in guest mode, a VM-exit
>> (exit reason: external interrupt) will be triggered due to the host-IPI.
>> Then on VM-entry (inject_pending_event) the guest-IPI is injected.
>>
> if vcpu#1 is not on the same pcpu as the vcpu#0, a host-IPI will be sent.
> But if they are on the same pcpu, if vcpu#1 is in guest mode. Then the
> guest tlb flush IPI
> will wait until the next vcpu#1 vmexit. If that's the case. they are
> some time that the tlb entry has been
> invalidated in vcpu#0, but the corresponding entry in vcpu#1 could
> still been accessed, which seems cause some problem.
>
>> 2) Host CPU supports APIC virtualization (see commit 83d4c286931c and
>> Intel's documentation):
>> A bit is set in the posted interrupt section, and a special host-IPI is
>> delivered to the target cpu where the guest vcpu is scheduled
>> (vmx_deliver_posted_interrupt) which causes the hardware to
>> inject the vector (without VM-exit).
>>
>>
> I did not find this function (vmx_deliver_posted_interrupt) in my kvm
> kernel module.
> Does that mean my hardware doesn't support APIC virtualization?
>
> Thanks for your time!
>
> Best Wishes,
> Yaohui Hu

Re: [PATCH] kvm: x86: Fix debug typo error in lapic

[Hu Yaohui]

Hi Marcelo,
Thanks for your replying!
I hope you have a good day! I am sorry that it's not that obvious to
me after I checked that function.
If the remote vcpu is not in the same pcpu as the sender which calls
kvm_vpcu_kick.
Before the remote vcpu received the Host-IPI, it could be in guest
mode which could possibly access the stale tlb entry that have been
modifed by the sender.

For smp_call_function_wait, the wait parameter will only let the
sender side wait until the function has been executed on all the
target cpus, then return.
For TLB Flush IPI, the sender has already invalidate one tlb entry,
the received side should get the IPI ASAP to invalidate the specified
tlb before it's accessed, or there should have some mechanisms to
enusre that even the stale tlb entry is accessed, it's still ok. I am
not sure what's the situation here after I reviewed all the source
codes.

Thanks for your time!

Best Wishes,
Yaohui Hu


On Thu, Jan 9, 2014 at 1:47 PM, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> On Thu, Jan 09, 2014 at 11:28:40AM -0500, Hu Yaohui wrote:
>> Hi Macelo,
>> I am sorry to bother you again. In your first possibility,
>> kvm_vcpu_kick sends an host-IPI to the remote vcpu,
>> and if that vcpu is in guest mode, a VM-exit will be triggered due to
>> the host-IPI. My question is if the vcpu has
>> accessed the stale tlb entry before the host-IPI arrives, what will
>> happen?  Thanks for your time!
>>
>> Best Wishes,
>> Yaohui Hu
>
> Yaohui Hu, the reply from a few seconds ago should clarify that (wait
> parameter).
>

Re: [PATCH] kvm: x86: Fix debug typo error in lapic

[Marcelo Tosatti]

On Thu, Jan 09, 2014 at 03:08:25PM -0500, Hu Yaohui wrote:
> Hi Marcelo,
> Thanks for your replying!
> I hope you have a good day! I am sorry that it's not that obvious to
> me after I checked that function.
> If the remote vcpu is not in the same pcpu as the sender which calls
> kvm_vpcu_kick.
> Before the remote vcpu received the Host-IPI, it could be in guest
> mode which could possibly access the stale tlb entry that have been
> modifed by the sender.
> 
> For smp_call_function_wait, the wait parameter will only let the
> sender side wait until the function has been executed on all the
> target cpus, then return.
> For TLB Flush IPI, the sender has already invalidate one tlb entry,
> the received side should get the IPI ASAP to invalidate the specified
> tlb before it's accessed, or there should have some mechanisms to
> enusre that even the stale tlb entry is accessed, it's still ok. I am
> not sure what's the situation here after I reviewed all the source
> codes.

The sender only considers the TLB entry (or entries) flushed when
smp_call_function_wait finishes, that is when the receiver cpu acknowledges
it has flushed its TLB.

Re: [PATCH] kvm: x86: Fix debug typo error in lapic

[Hu Yaohui]

Thank you Marcelo!
I really appreciate your explanation.

On Sat, Jan 11, 2014 at 7:27 AM, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> On Thu, Jan 09, 2014 at 03:08:25PM -0500, Hu Yaohui wrote:
>> Hi Marcelo,
>> Thanks for your replying!
>> I hope you have a good day! I am sorry that it's not that obvious to
>> me after I checked that function.
>> If the remote vcpu is not in the same pcpu as the sender which calls
>> kvm_vpcu_kick.
>> Before the remote vcpu received the Host-IPI, it could be in guest
>> mode which could possibly access the stale tlb entry that have been
>> modifed by the sender.
>>
>> For smp_call_function_wait, the wait parameter will only let the
>> sender side wait until the function has been executed on all the
>> target cpus, then return.
>> For TLB Flush IPI, the sender has already invalidate one tlb entry,
>> the received side should get the IPI ASAP to invalidate the specified
>> tlb before it's accessed, or there should have some mechanisms to
>> enusre that even the stale tlb entry is accessed, it's still ok. I am
>> not sure what's the situation here after I reviewed all the source
>> codes.
>
> The sender only considers the TLB entry (or entries) flushed when
> smp_call_function_wait finishes, that is when the receiver cpu acknowledges
> it has flushed its TLB.
>

Re: [PATCH] kvm: x86: Fix debug typo error in lapic

[Marcelo Tosatti]

On Wed, Jan 08, 2014 at 06:35:00PM -0500, Hu Yaohui wrote:
> Thanks a lot Marcelo!
> 
> On Wed, Jan 8, 2014 at 6:25 PM, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > On Wed, Jan 08, 2014 at 06:14:15PM -0500, Hu Yaohui wrote:
> >> Hi guys,
> >> I think you should be pretty familiar with lapic. I would really
> >> appreciate it if someone could shed some lights on my problem
> >> regarding Guest TLB flush IPI.
> >> Supposed we get two vcpus 0 and 1.
> >> When vcpu#0 wants to invalidate the tlb entry on vcpu#1. An IPI will
> >> be generated by lapic on vcpu#0 by writing to ICR which will cause a
> >> vmexit.
> >> apic_send_ipi->kvm_irq_delivery_to_apic->kvm_apic_set_irq->__apic_accept_irq
> >> In __apic_accept_irq, it will call kvm_make_request, kvm_vcpu_kick.
> >> If vcpu#1 in guest mode, how can it receives this IPI immediately, or
> >> the stale tlb entry could be accessed. Thanks for your time!
> >
> I am using kvm-kmod-3.2
> > Two possibilities:
> >
> > 2) Hardware does not support APIC virtualization: kvm_vcpu_kick sends an
> > host-IPI to the remote vcpu, and if that vcpu is in guest mode, a VM-exit
> > (exit reason: external interrupt) will be triggered due to the host-IPI.
> > Then on VM-entry (inject_pending_event) the guest-IPI is injected.
> >
> if vcpu#1 is not on the same pcpu as the vcpu#0, a host-IPI will be sent.

Yes.

> But if they are on the same pcpu, if vcpu#1 is in guest mode. 

If vcpu#0 and vcpu#1 are on the same pcpu, then either one of them 
is guest mode at one given moment, but not both.

> Then the
> guest tlb flush IPI
> will wait until the next vcpu#1 vmexit. If that's the case. they are
> some time that the tlb entry has been
> invalidated in vcpu#0, but the corresponding entry in vcpu#1 could
> still been accessed, which seems cause some problem.

The TLB flush is performed synchronously, see the effect of the wait
parameter to the smp_call_function_many function, and how that function
is called at arch/x86/mm/tlb.c.

> > 2) Host CPU supports APIC virtualization (see commit 83d4c286931c and
> > Intel's documentation):
> > A bit is set in the posted interrupt section, and a special host-IPI is
> > delivered to the target cpu where the guest vcpu is scheduled
> > (vmx_deliver_posted_interrupt) which causes the hardware to
> > inject the vector (without VM-exit).
> >
> >
> I did not find this function (vmx_deliver_posted_interrupt) in my kvm
> kernel module.
> Does that mean my hardware doesn't support APIC virtualization?

No, it means the kvm codebase you are looking at does not support it.

> Thanks for your time!
> 
> Best Wishes,
> Yaohui Hu
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] kvm: x86: Fix debug typo error in lapic

[Hu Yaohui]

Thanks a lot Marcelo!

On Thu, Jan 9, 2014 at 1:46 PM, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> On Wed, Jan 08, 2014 at 06:35:00PM -0500, Hu Yaohui wrote:
>> Thanks a lot Marcelo!
>>
>> On Wed, Jan 8, 2014 at 6:25 PM, Marcelo Tosatti <mtosatti@redhat.com> wrote:
>> > On Wed, Jan 08, 2014 at 06:14:15PM -0500, Hu Yaohui wrote:
>> >> Hi guys,
>> >> I think you should be pretty familiar with lapic. I would really
>> >> appreciate it if someone could shed some lights on my problem
>> >> regarding Guest TLB flush IPI.
>> >> Supposed we get two vcpus 0 and 1.
>> >> When vcpu#0 wants to invalidate the tlb entry on vcpu#1. An IPI will
>> >> be generated by lapic on vcpu#0 by writing to ICR which will cause a
>> >> vmexit.
>> >> apic_send_ipi->kvm_irq_delivery_to_apic->kvm_apic_set_irq->__apic_accept_irq
>> >> In __apic_accept_irq, it will call kvm_make_request, kvm_vcpu_kick.
>> >> If vcpu#1 in guest mode, how can it receives this IPI immediately, or
>> >> the stale tlb entry could be accessed. Thanks for your time!
>> >
>> I am using kvm-kmod-3.2
>> > Two possibilities:
>> >
>> > 2) Hardware does not support APIC virtualization: kvm_vcpu_kick sends an
>> > host-IPI to the remote vcpu, and if that vcpu is in guest mode, a VM-exit
>> > (exit reason: external interrupt) will be triggered due to the host-IPI.
>> > Then on VM-entry (inject_pending_event) the guest-IPI is injected.
>> >
>> if vcpu#1 is not on the same pcpu as the vcpu#0, a host-IPI will be sent.
>
> Yes.
>
>> But if they are on the same pcpu, if vcpu#1 is in guest mode.
>
> If vcpu#0 and vcpu#1 are on the same pcpu, then either one of them
> is guest mode at one given moment, but not both.
>
>> Then the
>> guest tlb flush IPI
>> will wait until the next vcpu#1 vmexit. If that's the case. they are
>> some time that the tlb entry has been
>> invalidated in vcpu#0, but the corresponding entry in vcpu#1 could
>> still been accessed, which seems cause some problem.
>
> The TLB flush is performed synchronously, see the effect of the wait
> parameter to the smp_call_function_many function, and how that function
> is called at arch/x86/mm/tlb.c.
>
if kvm_vcpu_kick sends an host-IPI to the remote vcpu, and if that
vcpu is in guest mode.
Is it possible that the remote vcpu has accessed the stale tlb entry
before the host-IPI arrives?
If that's the case, how this problem to be solved in KVM? Thanks for your time!
>> > 2) Host CPU supports APIC virtualization (see commit 83d4c286931c and
>> > Intel's documentation):
>> > A bit is set in the posted interrupt section, and a special host-IPI is
>> > delivered to the target cpu where the guest vcpu is scheduled
>> > (vmx_deliver_posted_interrupt) which causes the hardware to
>> > inject the vector (without VM-exit).
>> >
>> >
>> I did not find this function (vmx_deliver_posted_interrupt) in my kvm
>> kernel module.
>> Does that mean my hardware doesn't support APIC virtualization?
>
> No, it means the kvm codebase you are looking at does not support it.
>
>> Thanks for your time!
>>
>> Best Wishes,
>> Yaohui Hu
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html