[v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[Tiejun Chen]

We also can direct ISI exception to Guest like DSI.

Signed-off-by: Tiejun Chen <tiejun.chen@windriver.com>
---
 arch/powerpc/kvm/booke_emulate.c |    3 +++
 arch/powerpc/kvm/e500mc.c        |    3 ++-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/kvm/booke_emulate.c b/arch/powerpc/kvm/booke_emulate.c
index 27a4b28..33b14e9 100644
--- a/arch/powerpc/kvm/booke_emulate.c
+++ b/arch/powerpc/kvm/booke_emulate.c
@@ -197,6 +197,9 @@ int kvmppc_booke_emulate_mtspr(struct kvm_vcpu *vcpu, int sprn, ulong spr_val)
 		break;
 	case SPRN_IVOR3:
 		vcpu->arch.ivor[BOOKE_IRQPRIO_INST_STORAGE] = spr_val;
+#ifdef CONFIG_KVM_BOOKE_HV
+		mtspr(SPRN_GIVOR3, spr_val);
+#endif
 		break;
 	case SPRN_IVOR4:
 		vcpu->arch.ivor[BOOKE_IRQPRIO_EXTERNAL] = spr_val;
diff --git a/arch/powerpc/kvm/e500mc.c b/arch/powerpc/kvm/e500mc.c
index c3bdc0a..acf546a 100644
--- a/arch/powerpc/kvm/e500mc.c
+++ b/arch/powerpc/kvm/e500mc.c
@@ -123,6 +123,7 @@ void kvmppc_core_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 
 	mtspr(SPRN_GIVPR, vcpu->arch.ivpr);
 	mtspr(SPRN_GIVOR2, vcpu->arch.ivor[BOOKE_IRQPRIO_DATA_STORAGE]);
+	mtspr(SPRN_GIVOR3, vcpu->arch.ivor[BOOKE_IRQPRIO_INST_STORAGE]);
 	mtspr(SPRN_GIVOR8, vcpu->arch.ivor[BOOKE_IRQPRIO_SYSCALL]);
 	mtspr(SPRN_GSPRG0, (unsigned long)vcpu->arch.shared->sprg0);
 	mtspr(SPRN_GSPRG1, (unsigned long)vcpu->arch.shared->sprg1);
@@ -185,7 +186,7 @@ int kvmppc_core_vcpu_setup(struct kvm_vcpu *vcpu)
 	struct kvmppc_vcpu_e500 *vcpu_e500 = to_e500(vcpu);
 
 	vcpu->arch.shadow_epcr = SPRN_EPCR_DSIGS | SPRN_EPCR_DGTMI | \
-				 SPRN_EPCR_DUVD;
+				 SPRN_EPCR_DUVD | SPRN_EPCR_ISIGS;
 #ifdef CONFIG_64BIT
 	vcpu->arch.shadow_epcr |= SPRN_EPCR_ICM;
 #endif
-- 
1.7.9.5

Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[Scott Wood]

On 05/07/2013 06:06:30 AM, Tiejun Chen wrote:
> We also can direct ISI exception to Guest like DSI.
> 
> Signed-off-by: Tiejun Chen <tiejun.chen@windriver.com>
> ---
>  arch/powerpc/kvm/booke_emulate.c |    3 +++
>  arch/powerpc/kvm/e500mc.c        |    3 ++-
>  2 files changed, 5 insertions(+), 1 deletion(-)

Are you seeing a real performance improvement from this?  This will  
interfere somewhat with using the VF bit, if we were to ever do so,  
since VF only affects data accesses (and so the guest would see an ISI  
storm rather than a machine check if it tries to execute from such an  
address).

-Scott

Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[tiejun.chen]

On 05/08/2013 07:40 AM, Scott Wood wrote:
> On 05/07/2013 06:06:30 AM, Tiejun Chen wrote:
>> We also can direct ISI exception to Guest like DSI.
>>
>> Signed-off-by: Tiejun Chen <tiejun.chen@windriver.com>
>> ---
>>  arch/powerpc/kvm/booke_emulate.c |    3 +++
>>  arch/powerpc/kvm/e500mc.c        |    3 ++-
>>  2 files changed, 5 insertions(+), 1 deletion(-)
>
> Are you seeing a real performance improvement from this?  This will interfere

No. But after we reduce the exit to host, shouldn't this improve performance?

> somewhat with using the VF bit, if we were to ever do so, since VF only affects

Sorry, what is the VF you said?

Tiejun

RE: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[Caraman Mihai Claudiu-B02008]

> -----Original Message-----
> From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On
> Behalf Of tiejun.chen
> Sent: Wednesday, May 08, 2013 4:54 AM
> To: Wood Scott-B07421
> Cc: agraf@suse.de; kvm-ppc@vger.kernel.org; kvm@vger.kernel.org;
> linuxppc-dev@lists.ozlabs.org
> Subject: Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to
> Guest
> 
> On 05/08/2013 07:40 AM, Scott Wood wrote:
> > On 05/07/2013 06:06:30 AM, Tiejun Chen wrote:
> >> We also can direct ISI exception to Guest like DSI.
> >>
> >> Signed-off-by: Tiejun Chen <tiejun.chen@windriver.com>
> >> ---
> >>  arch/powerpc/kvm/booke_emulate.c |    3 +++
> >>  arch/powerpc/kvm/e500mc.c        |    3 ++-
> >>  2 files changed, 5 insertions(+), 1 deletion(-)
> >
> > Are you seeing a real performance improvement from this?  This will
> interfere
> 
> No. But after we reduce the exit to host, shouldn't this improve
> performance?

We lose some flexibility for this so it make sense only if we gain
measurable improvements.

> 
> > somewhat with using the VF bit, if we were to ever do so, since VF only
> affects
> 
> Sorry, what is the VF you said?

VF stands for virtualization fault see MAS8[VF] and we may use it for virtualized
MMIO. The hypervisor should deny execute access on pages marked with VF. Accordingly
in this case guest ISI exceptions should be handled by the hypervisor.

-Mike

Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[tiejun.chen]

On 05/08/2013 05:20 PM, Caraman Mihai Claudiu-B02008 wrote:
>> -----Original Message-----
>> From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On
>> Behalf Of tiejun.chen
>> Sent: Wednesday, May 08, 2013 4:54 AM
>> To: Wood Scott-B07421
>> Cc: agraf@suse.de; kvm-ppc@vger.kernel.org; kvm@vger.kernel.org;
>> linuxppc-dev@lists.ozlabs.org
>> Subject: Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to
>> Guest
>>
>> On 05/08/2013 07:40 AM, Scott Wood wrote:
>>> On 05/07/2013 06:06:30 AM, Tiejun Chen wrote:
>>>> We also can direct ISI exception to Guest like DSI.
>>>>
>>>> Signed-off-by: Tiejun Chen <tiejun.chen@windriver.com>
>>>> ---
>>>>   arch/powerpc/kvm/booke_emulate.c |    3 +++
>>>>   arch/powerpc/kvm/e500mc.c        |    3 ++-
>>>>   2 files changed, 5 insertions(+), 1 deletion(-)
>>>
>>> Are you seeing a real performance improvement from this?  This will
>> interfere
>>
>> No. But after we reduce the exit to host, shouldn't this improve
>> performance?
>
> We lose some flexibility for this so it make sense only if we gain
> measurable improvements.

Sounds we have much more works to do.

>
>>
>>> somewhat with using the VF bit, if we were to ever do so, since VF only
>> affects
>>
>> Sorry, what is the VF you said?
>
> VF stands for virtualization fault see MAS8[VF] and we may use it for virtualized

I almost forget this point :)

> MMIO. The hypervisor should deny execute access on pages marked with VF. Accordingly
> in this case guest ISI exceptions should be handled by the hypervisor.

Thanks for your information.

Tiejun

Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[Scott Wood]

On 05/07/2013 08:53:50 PM, tiejun.chen wrote:
> On 05/08/2013 07:40 AM, Scott Wood wrote:
>> On 05/07/2013 06:06:30 AM, Tiejun Chen wrote:
>>> We also can direct ISI exception to Guest like DSI.
>>> 
>>> Signed-off-by: Tiejun Chen <tiejun.chen@windriver.com>
>>> ---
>>>  arch/powerpc/kvm/booke_emulate.c |    3 +++
>>>  arch/powerpc/kvm/e500mc.c        |    3 ++-
>>>  2 files changed, 5 insertions(+), 1 deletion(-)
>> 
>> Are you seeing a real performance improvement from this?  This will  
>> interfere
> 
> No. But after we reduce the exit to host, shouldn't this improve  
> performance?

Not if ISIs are too rare to matter.

-Scott

Re: [v1][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[tiejun.chen]

On 05/08/2013 05:28 PM, tiejun.chen wrote:
> On 05/08/2013 05:20 PM, Caraman Mihai Claudiu-B02008 wrote:
>>> -----Original Message-----
>>> From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On
>>> Behalf Of tiejun.chen
>>> Sent: Wednesday, May 08, 2013 4:54 AM
>>> To: Wood Scott-B07421
>>> Cc: agraf@suse.de; kvm-ppc@vger.kernel.org; kvm@vger.kernel.org;
>>> linuxppc-dev@lists.ozlabs.org
>>> Subject: Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to
>>> Guest
>>>
>>> On 05/08/2013 07:40 AM, Scott Wood wrote:
>>>> On 05/07/2013 06:06:30 AM, Tiejun Chen wrote:
>>>>> We also can direct ISI exception to Guest like DSI.
>>>>>
>>>>> Signed-off-by: Tiejun Chen <tiejun.chen@windriver.com>
>>>>> ---
>>>>>   arch/powerpc/kvm/booke_emulate.c |    3 +++
>>>>>   arch/powerpc/kvm/e500mc.c        |    3 ++-
>>>>>   2 files changed, 5 insertions(+), 1 deletion(-)
>>>>
>>>> Are you seeing a real performance improvement from this?  This will
>>> interfere
>>>
>>> No. But after we reduce the exit to host, shouldn't this improve
>>> performance?
>>
>> We lose some flexibility for this so it make sense only if we gain
>> measurable improvements.
>
> Sounds we have much more works to do.
>
>>
>>>
>>>> somewhat with using the VF bit, if we were to ever do so, since VF only
>>> affects
>>>
>>> Sorry, what is the VF you said?
>>
>> VF stands for virtualization fault see MAS8[VF] and we may use it for virtualized
>
> I almost forget this point :)

Looks KVM PPC have no this mechanism currently since I don't find MAS8_VF is 
used in kernel, right?

If I'm missing something please correct me.

Tiejun

>
>> MMIO. The hypervisor should deny execute access on pages marked with VF.
>> Accordingly
>> in this case guest ISI exceptions should be handled by the hypervisor.
>

RE: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[Caraman Mihai Claudiu-B02008]

> >> VF stands for virtualization fault see MAS8[VF] and we may use it for
> virtualized
> 
> Looks KVM PPC have no this mechanism currently since I don't find MAS8_VF
> is
> used in kernel, right?

Yes but 'we may use it' in the feature, I have a functional POC with VF.
Now we capture virtualized MMIO accesses as TLB misses (we don't write
into HW TLB guest translation outside visible memslots).

-Mike

Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[tiejun.chen]

On 05/09/2013 07:34 PM, Caraman Mihai Claudiu-B02008 wrote:
>>>> VF stands for virtualization fault see MAS8[VF] and we may use it for
>> virtualized
>>
>> Looks KVM PPC have no this mechanism currently since I don't find MAS8_VF
>> is
>> used in kernel, right?
>
> Yes but 'we may use it' in the feature, I have a functional POC with VF.

Any IO performance to be improved with this POC?

> Now we capture virtualized MMIO accesses as TLB misses (we don't write
> into HW TLB guest translation outside visible memslots).

Yes.

Tiejun

RE: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[Caraman Mihai Claudiu-B02008]

> -----Original Message-----
> From: tiejun.chen [mailto:tiejun.chen@windriver.com]
> Sent: Thursday, May 09, 2013 2:40 PM
> To: Caraman Mihai Claudiu-B02008
> Cc: Wood Scott-B07421; linuxppc-dev@lists.ozlabs.org; agraf@suse.de; kvm-
> ppc@vger.kernel.org; kvm@vger.kernel.org
> Subject: Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to
> Guest
> 
> On 05/09/2013 07:34 PM, Caraman Mihai Claudiu-B02008 wrote:
> >>>> VF stands for virtualization fault see MAS8[VF] and we may use it
> for
> >> virtualized
> >>
> >> Looks KVM PPC have no this mechanism currently since I don't find
> MAS8_VF
> >> is
> >> used in kernel, right?
> >
> > Yes but 'we may use it' in the feature, I have a functional POC with
> VF.
> 
> Any IO performance to be improved with this POC?

VF approach puts more stress on HW TLB so I did not advance with performance
measurements though it may worth to do it.

-Mike

Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[Alexander Graf]

On 09.05.2013, at 14:36, Caraman Mihai Claudiu-B02008 wrote:

>> -----Original Message-----
>> From: tiejun.chen [mailto:tiejun.chen@windriver.com]
>> Sent: Thursday, May 09, 2013 2:40 PM
>> To: Caraman Mihai Claudiu-B02008
>> Cc: Wood Scott-B07421; linuxppc-dev@lists.ozlabs.org; agraf@suse.de; kvm-
>> ppc@vger.kernel.org; kvm@vger.kernel.org
>> Subject: Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to
>> Guest
>> 
>> On 05/09/2013 07:34 PM, Caraman Mihai Claudiu-B02008 wrote:
>>>>>> VF stands for virtualization fault see MAS8[VF] and we may use it
>> for
>>>> virtualized
>>>> 
>>>> Looks KVM PPC have no this mechanism currently since I don't find
>> MAS8_VF
>>>> is
>>>> used in kernel, right?
>>> 
>>> Yes but 'we may use it' in the feature, I have a functional POC with
>> VF.
>> 
>> Any IO performance to be improved with this POC?
> 
> VF approach puts more stress on HW TLB so I did not advance with performance
> measurements though it may worth to do it.

Could you guys please collect performance data during the next weeks on both guest-directed ISIs as well as VF MMIOs (preferably with in-kernel MMIO), so that we can decide on the direction that's worth going towards?


Thanks!

Alex

Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[Scott Wood]

On 05/10/2013 12:57:33 PM, Alexander Graf wrote:
> Could you guys please collect performance data during the next weeks  
> on both guest-directed ISIs as well as VF MMIOs (preferably with  
> in-kernel MMIO), so that we can decide on the direction that's worth  
> going towards?

Collecting data on VF MMIO would require implementing it (or at least  
salvaging and fixing some old code), which is not a high priority at  
the moment.  If we do implement VF in the future we could always undo  
the direct ISI change, but it would still be nice to know if there's  
any real benefit in the first place.

FWIW, I doubt that the "more stress on HW TLB" will be significant.

-Scott

Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[Alexander Graf]

Am 10.05.2013 um 21:22 schrieb Scott Wood <scottwood@freescale.com>:

> On 05/10/2013 12:57:33 PM, Alexander Graf wrote:
>> Could you guys please collect performance data during the next weeks on both guest-directed ISIs as well as VF MMIOs (preferably with in-kernel MMIO), so that we can decide on the direction that's worth going towards?
> 
> Collecting data on VF MMIO would require implementing it (or at least salvaging and fixing some old code), which is not a high priority at the moment.  If we do implement VF in the future we could always undo the direct ISI change, but it would still be nice to know if there's any real benefit in the first place.

Mike sounded like he had an almost working poc, which is good enough to collect rough numbers.

And yes, changes like these should always get at least basic performance numbers along with them, regardless of drawbacks.


Alex

> 
> FWIW, I doubt that the "more stress on HW TLB" will be significant.
> 
> -Scott

Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

[tiejun.chen]

On 05/11/2013 03:39 AM, Alexander Graf wrote:
>
>
> Am 10.05.2013 um 21:22 schrieb Scott Wood <scottwood@freescale.com>:
>
>> On 05/10/2013 12:57:33 PM, Alexander Graf wrote:
>>> Could you guys please collect performance data during the next weeks on both guest-directed ISIs as well as VF MMIOs (preferably with in-kernel MMIO), so that we can decide on the direction that's worth going towards?
>>
>> Collecting data on VF MMIO would require implementing it (or at least salvaging and fixing some old code), which is not a high priority at the moment.  If we do implement VF in the future we could always undo the direct ISI change, but it would still be nice to know if there's any real benefit in the first place.
>
> Mike sounded like he had an almost working poc, which is good enough to collect rough numbers.

Which can the test case be adopted?

Mike,

If you already have a good case for your poc, please share that with me. Then 
I'd like to run that.

Tiejun

>
> And yes, changes like these should always get at least basic performance numbers along with them, regardless of drawbacks.
>
>
> Alex
>
>>
>> FWIW, I doubt that the "more stress on HW TLB" will be significant.
>>
>> -Scott
>