[RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Tiejun Chen]

For the external interrupt, the decrementer exception and the doorbell
excpetion, we also need to soft-disable interrupts while doing as host
interrupt handlers since the DO_KVM hook is always performed to skip
EXCEPTION_COMMON then miss this original chance with the 'ints' (INTS_DISABLE).

Signed-off-by: Tiejun Chen <tiejun.chen@windriver.com>
---
 arch/powerpc/kvm/bookehv_interrupts.S |    9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/powerpc/kvm/bookehv_interrupts.S b/arch/powerpc/kvm/bookehv_interrupts.S
index e8ed7d6..2fd62bf 100644
--- a/arch/powerpc/kvm/bookehv_interrupts.S
+++ b/arch/powerpc/kvm/bookehv_interrupts.S
@@ -33,6 +33,8 @@
 
 #ifdef CONFIG_64BIT
 #include <asm/exception-64e.h>
+#include <asm/hw_irq.h>
+#include <asm/irqflags.h>
 #else
 #include "../kernel/head_booke.h" /* for THREAD_NORMSAVE() */
 #endif
@@ -469,6 +471,13 @@ _GLOBAL(kvmppc_resume_host)
 	PPC_LL	r3, HOST_RUN(r1)
 	mr	r5, r14 /* intno */
 	mr	r14, r4 /* Save vcpu pointer. */
+#ifdef CONFIG_64BIT
+	/* Should we soft-disable interrupts? */
+	andi.	r6, r5, BOOKE_INTERRUPT_EXTERNAL | BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
+	beq	skip_soft_dis
+	SOFT_DISABLE_INTS(r7,r8)
+skip_soft_dis:
+#endif
 	bl	kvmppc_handle_exit
 
 	/* Restore vcpu pointer and the nonvolatiles we used. */
-- 
1.7.9.5

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[tiejun.chen]

On 05/06/2013 11:10 AM, Tiejun Chen wrote:
> For the external interrupt, the decrementer exception and the doorbell
> excpetion, we also need to soft-disable interrupts while doing as host
> interrupt handlers since the DO_KVM hook is always performed to skip
> EXCEPTION_COMMON then miss this original chance with the 'ints' (INTS_DISABLE).

Sorry, miss to send Ben.

Tiejun

>
> Signed-off-by: Tiejun Chen <tiejun.chen@windriver.com>
> ---
>   arch/powerpc/kvm/bookehv_interrupts.S |    9 +++++++++
>   1 file changed, 9 insertions(+)
>
> diff --git a/arch/powerpc/kvm/bookehv_interrupts.S b/arch/powerpc/kvm/bookehv_interrupts.S
> index e8ed7d6..2fd62bf 100644
> --- a/arch/powerpc/kvm/bookehv_interrupts.S
> +++ b/arch/powerpc/kvm/bookehv_interrupts.S
> @@ -33,6 +33,8 @@
>
>   #ifdef CONFIG_64BIT
>   #include <asm/exception-64e.h>
> +#include <asm/hw_irq.h>
> +#include <asm/irqflags.h>
>   #else
>   #include "../kernel/head_booke.h" /* for THREAD_NORMSAVE() */
>   #endif
> @@ -469,6 +471,13 @@ _GLOBAL(kvmppc_resume_host)
>   	PPC_LL	r3, HOST_RUN(r1)
>   	mr	r5, r14 /* intno */
>   	mr	r14, r4 /* Save vcpu pointer. */
> +#ifdef CONFIG_64BIT
> +	/* Should we soft-disable interrupts? */
> +	andi.	r6, r5, BOOKE_INTERRUPT_EXTERNAL | BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
> +	beq	skip_soft_dis
> +	SOFT_DISABLE_INTS(r7,r8)
> +skip_soft_dis:
> +#endif
>   	bl	kvmppc_handle_exit
>
>   	/* Restore vcpu pointer and the nonvolatiles we used. */
>

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Scott Wood]

On 05/05/2013 10:13:17 PM, tiejun.chen wrote:
> On 05/06/2013 11:10 AM, Tiejun Chen wrote:
>> For the external interrupt, the decrementer exception and the =20
>> doorbell
>> excpetion, we also need to soft-disable interrupts while doing as =20
>> host
>> interrupt handlers since the DO_KVM hook is always performed to skip
>> EXCEPTION_COMMON then miss this original chance with the 'ints' =20
>> (INTS_DISABLE).

http://patchwork.ozlabs.org/patch/241344/
http://patchwork.ozlabs.org/patch/241412/

:-)

>> Signed-off-by: Tiejun Chen <tiejun.chen@windriver.com>
>> ---
>>   arch/powerpc/kvm/bookehv_interrupts.S |    9 +++++++++
>>   1 file changed, 9 insertions(+)
>>=20
>> diff --git a/arch/powerpc/kvm/bookehv_interrupts.S =20
>> b/arch/powerpc/kvm/bookehv_interrupts.S
>> index e8ed7d6..2fd62bf 100644
>> --- a/arch/powerpc/kvm/bookehv_interrupts.S
>> +++ b/arch/powerpc/kvm/bookehv_interrupts.S
>> @@ -33,6 +33,8 @@
>>=20
>>   #ifdef CONFIG_64BIT
>>   #include <asm/exception-64e.h>
>> +#include <asm/hw_irq.h>
>> +#include <asm/irqflags.h>
>>   #else
>>   #include "../kernel/head_booke.h" /* for THREAD_NORMSAVE() */
>>   #endif
>> @@ -469,6 +471,13 @@ _GLOBAL(kvmppc_resume_host)
>>   	PPC_LL	r3, HOST_RUN(r1)
>>   	mr	r5, r14 /* intno */
>>   	mr	r14, r4 /* Save vcpu pointer. */
>> +#ifdef CONFIG_64BIT
>> +	/* Should we soft-disable interrupts? */
>> +	andi.	r6, r5, BOOKE_INTERRUPT_EXTERNAL | =20
>> BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
>> +	beq	skip_soft_dis
>> +	SOFT_DISABLE_INTS(r7,r8)
>> +skip_soft_dis:
>> +#endif

Why wouldn't we always disable them?  kvmppc_handle_exit() will enable =20
interrupts when it's ready.

-Scott=

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[tiejun.chen]

On 05/07/2013 07:50 AM, Scott Wood wrote:
> On 05/05/2013 10:13:17 PM, tiejun.chen wrote:
>> On 05/06/2013 11:10 AM, Tiejun Chen wrote:
>>> For the external interrupt, the decrementer exception and the doorbell
>>> excpetion, we also need to soft-disable interrupts while doing as host
>>> interrupt handlers since the DO_KVM hook is always performed to skip
>>> EXCEPTION_COMMON then miss this original chance with the 'ints' (INTS_DISABLE).
>
> http://patchwork.ozlabs.org/patch/241344/
> http://patchwork.ozlabs.org/patch/241412/
>
> :-)

I'm observing the same behaviour as well:

	WARN_ON_ONCE(!irqs_disabled());

>
>>> Signed-off-by: Tiejun Chen <tiejun.chen@windriver.com>
>>> ---
>>>   arch/powerpc/kvm/bookehv_interrupts.S |    9 +++++++++
>>>   1 file changed, 9 insertions(+)
>>>
>>> diff --git a/arch/powerpc/kvm/bookehv_interrupts.S
>>> b/arch/powerpc/kvm/bookehv_interrupts.S
>>> index e8ed7d6..2fd62bf 100644
>>> --- a/arch/powerpc/kvm/bookehv_interrupts.S
>>> +++ b/arch/powerpc/kvm/bookehv_interrupts.S
>>> @@ -33,6 +33,8 @@
>>>
>>>   #ifdef CONFIG_64BIT
>>>   #include <asm/exception-64e.h>
>>> +#include <asm/hw_irq.h>
>>> +#include <asm/irqflags.h>
>>>   #else
>>>   #include "../kernel/head_booke.h" /* for THREAD_NORMSAVE() */
>>>   #endif
>>> @@ -469,6 +471,13 @@ _GLOBAL(kvmppc_resume_host)
>>>       PPC_LL    r3, HOST_RUN(r1)
>>>       mr    r5, r14 /* intno */
>>>       mr    r14, r4 /* Save vcpu pointer. */
>>> +#ifdef CONFIG_64BIT
>>> +    /* Should we soft-disable interrupts? */
>>> +    andi.    r6, r5, BOOKE_INTERRUPT_EXTERNAL | BOOKE_INTERRUPT_DECREMENTER
>>> | BOOKE_INTERRUPT_DOORBELL
>>> +    beq    skip_soft_dis
>>> +    SOFT_DISABLE_INTS(r7,r8)
>>> +skip_soft_dis:
>>> +#endif
>
> Why wouldn't we always disable them?  kvmppc_handle_exit() will enable
> interrupts when it's ready.

This only disable soft interrupt for kvmppc_restart_interrupt() that restarts 
interrupts if they were meant for the host:

a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL | 
BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL

b. bl      kvmppc_handle_exit

c. kvmppc_handle_exit()
{
         int r = RESUME_HOST;
         int s;

         /* update before a new last_exit_type is rewritten */
         kvmppc_update_timing_stats(vcpu);

         /* restart interrupts if they were meant for the host */
         kvmppc_restart_interrupt(vcpu, exit_nr);

         local_irq_enable();	==> Enable again.
....

And shouldn't we handle kvmppc_restart_interrupt() like the original HOST flow?

#define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr, ack)           \
         START_EXCEPTION(label);                                         \
         NORMAL_EXCEPTION_PROLOG(trapnum, intnum, PROLOG_ADDITION_MASKABLE)\
         EXCEPTION_COMMON(trapnum, PACA_EXGEN, *INTS_DISABLE*)             \
	...

So I think this should be reasonable :)

Tiejun

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Scott Wood]

On 05/06/2013 08:56:25 PM, tiejun.chen wrote:
> On 05/07/2013 07:50 AM, Scott Wood wrote:
>> On 05/05/2013 10:13:17 PM, tiejun.chen wrote:
>>> On 05/06/2013 11:10 AM, Tiejun Chen wrote:
>>>> For the external interrupt, the decrementer exception and the =20
>>>> doorbell
>>>> excpetion, we also need to soft-disable interrupts while doing as =20
>>>> host
>>>> interrupt handlers since the DO_KVM hook is always performed to =20
>>>> skip
>>>> EXCEPTION_COMMON then miss this original chance with the 'ints' =20
>>>> (INTS_DISABLE).
>>=20
>> http://patchwork.ozlabs.org/patch/241344/
>> http://patchwork.ozlabs.org/patch/241412/
>>=20
>> :-)
>=20
> I'm observing the same behaviour as well:
>=20
> 	WARN_ON_ONCE(!irqs_disabled());

So, could you explain the benefits of your approach over what's being =20
discussed in those threads?

>> Why wouldn't we always disable them?  kvmppc_handle_exit() will =20
>> enable
>> interrupts when it's ready.
>=20
> This only disable soft interrupt for kvmppc_restart_interrupt() that =20
> restarts interrupts if they were meant for the host:
>=20
> a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL | =20
> BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL

Those aren't the only exceptions that can end up going to the host.  We =20
could get a TLB miss that results in a heavyweight MMIO exit, etc.

And I'd rather see any fix for this problem stay out of the asm code.

> b. bl      kvmppc_handle_exit
>=20
> c. kvmppc_handle_exit()
> {
>         int r =3D RESUME_HOST;
>         int s;
>=20
>         /* update before a new last_exit_type is rewritten */
>         kvmppc_update_timing_stats(vcpu);
>=20
>         /* restart interrupts if they were meant for the host */
>         kvmppc_restart_interrupt(vcpu, exit_nr);
>=20
>         local_irq_enable();	=3D=3D> Enable again.
> ....
>=20
> And shouldn't we handle kvmppc_restart_interrupt() like the original =20
> HOST flow?
>=20
> #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr, =20
> ack)           \
>         =20
> START_EXCEPTION(label);                                         \
>         NORMAL_EXCEPTION_PROLOG(trapnum, intnum, =20
> PROLOG_ADDITION_MASKABLE)\
>         EXCEPTION_COMMON(trapnum, PACA_EXGEN, =20
> *INTS_DISABLE*)             \
> 	...

Could you elaborate on what you mean?

-Scott=

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[tiejun.chen]

On 05/07/2013 10:06 AM, Scott Wood wrote:
> On 05/06/2013 08:56:25 PM, tiejun.chen wrote:
>> On 05/07/2013 07:50 AM, Scott Wood wrote:
>>> On 05/05/2013 10:13:17 PM, tiejun.chen wrote:
>>>> On 05/06/2013 11:10 AM, Tiejun Chen wrote:
>>>>> For the external interrupt, the decrementer exception and the doorbell
>>>>> excpetion, we also need to soft-disable interrupts while doing as host
>>>>> interrupt handlers since the DO_KVM hook is always performed to skip
>>>>> EXCEPTION_COMMON then miss this original chance with the 'ints'
>>>>> (INTS_DISABLE).
>>>
>>> http://patchwork.ozlabs.org/patch/241344/
>>> http://patchwork.ozlabs.org/patch/241412/
>>>
>>> :-)
>>
>> I'm observing the same behaviour as well:
>>
>>     WARN_ON_ONCE(!irqs_disabled());
>
> So, could you explain the benefits of your approach over what's being discussed
> in those threads?

They're a long thread so I think I need to take time to see :)

>
>>> Why wouldn't we always disable them?  kvmppc_handle_exit() will enable
>>> interrupts when it's ready.
>>
>> This only disable soft interrupt for kvmppc_restart_interrupt() that restarts
>> interrupts if they were meant for the host:
>>
>> a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
>> BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
>
> Those aren't the only exceptions that can end up going to the host.  We could
> get a TLB miss that results in a heavyweight MMIO exit, etc.

This is like host handler, so I'm just disabling soft interrupt during 
kvmppc_restart_interrupt() for Doorbell interrupt/Decrementer Interrupt/External 
Input Interrupt.

I don't see anything should be disabled for any TLB exception in host handler.

>
> And I'd rather see any fix for this problem stay out of the asm code.

We already have an appropriate SOFT_DISABLE_INTS so I think we can take this 
easily :)

>
>> b. bl      kvmppc_handle_exit
>>
>> c. kvmppc_handle_exit()
>> {
>>         int r = RESUME_HOST;
>>         int s;
>>
>>         /* update before a new last_exit_type is rewritten */
>>         kvmppc_update_timing_stats(vcpu);
>>
>>         /* restart interrupts if they were meant for the host */
>>         kvmppc_restart_interrupt(vcpu, exit_nr);
>>
>>         local_irq_enable();    ==> Enable again.
>> ....
>>
>> And shouldn't we handle kvmppc_restart_interrupt() like the original HOST flow?
>>
>> #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr, ack)           \
>>         START_EXCEPTION(label);                                         \
>>         NORMAL_EXCEPTION_PROLOG(trapnum, intnum, PROLOG_ADDITION_MASKABLE)\
>>         EXCEPTION_COMMON(trapnum, PACA_EXGEN, *INTS_DISABLE*)             \
>>     ...
>
> Could you elaborate on what you mean?

In host handler, we always use MASKABLE_EXCEPTION() to define-to-handle some 
exceptions: Doorbell interrupt/Decrementer Interrupt/External Input Interrupt:

#define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr, ack)           \
         START_EXCEPTION(label);                                         \
         NORMAL_EXCEPTION_PROLOG(trapnum, intnum, PROLOG_ADDITION_MASKABLE)\
         EXCEPTION_COMMON(trapnum, PACA_EXGEN, *INTS_DISABLE*)             \

This would call INTS_DISABLE, which is equal to SOFT_DISABLE_INTS(), to disable 
soft interrupt before call all associated handlers: 
do_IRQ()/timer_interrupt()/doorbell_exception().

But DO_KVM hook always skips INTS_DISABLE.

So I think we also need to do INTS_DISABLE for kvmppc_restart_interrupt() since 
actually that restarts interrupts for the host with a similar way as they are 
called by host.

Tiejun

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Scott Wood]

On 05/06/2013 09:43:37 PM, tiejun.chen wrote:
> On 05/07/2013 10:06 AM, Scott Wood wrote:
>> On 05/06/2013 08:56:25 PM, tiejun.chen wrote:
>>> On 05/07/2013 07:50 AM, Scott Wood wrote:
>>>> On 05/05/2013 10:13:17 PM, tiejun.chen wrote:
>>>>> On 05/06/2013 11:10 AM, Tiejun Chen wrote:
>>>>>> For the external interrupt, the decrementer exception and the =20
>>>>>> doorbell
>>>>>> excpetion, we also need to soft-disable interrupts while doing =20
>>>>>> as host
>>>>>> interrupt handlers since the DO_KVM hook is always performed to =20
>>>>>> skip
>>>>>> EXCEPTION_COMMON then miss this original chance with the 'ints'
>>>>>> (INTS_DISABLE).
>>>>=20
>>>> http://patchwork.ozlabs.org/patch/241344/
>>>> http://patchwork.ozlabs.org/patch/241412/
>>>>=20
>>>> :-)
>>>=20
>>> I'm observing the same behaviour as well:
>>>=20
>>>     WARN_ON_ONCE(!irqs_disabled());
>>=20
>> So, could you explain the benefits of your approach over what's =20
>> being discussed
>> in those threads?
>=20
> They're a long thread so I think I need to take time to see :)
>=20
>>=20
>>>> Why wouldn't we always disable them?  kvmppc_handle_exit() will =20
>>>> enable
>>>> interrupts when it's ready.
>>>=20
>>> This only disable soft interrupt for kvmppc_restart_interrupt() =20
>>> that restarts
>>> interrupts if they were meant for the host:
>>>=20
>>> a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
>>> BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
>>=20
>> Those aren't the only exceptions that can end up going to the host.  =20
>> We could
>> get a TLB miss that results in a heavyweight MMIO exit, etc.
>=20
> This is like host handler, so I'm just disabling soft interrupt =20
> during kvmppc_restart_interrupt() for Doorbell interrupt/Decrementer =20
> Interrupt/External Input Interrupt.
>=20
> I don't see anything should be disabled for any TLB exception in host =20
> handler.

Every exception type needs consistent lazy EE state once we hit the =20
local_irq_enable() (or any other C code that cares).

Plus, if you're going to add code to make something conditional, you =20
should have a good reason for making it conditional.  Being more like =20
the 64-bit host handler for its own sake isn't good enough, especially =20
if you introduce differences between 32 and 64 bit in the process.

>> And I'd rather see any fix for this problem stay out of the asm code.
>=20
> We already have an appropriate SOFT_DISABLE_INTS so I think we can =20
> take this easily :)

An unconditional call to hard_irq_disable() at the beginning of =20
kvmppc_handle_exit() should suffice.  I already have this in the next =20
version of my patch that I'll be posting shortly.

Note that we need this on 32-bit as well, so that trace_hardirqs_off() =20
gets called.

-Scott=

RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Caraman Mihai Claudiu-B02008]

> > This only disable soft interrupt for kvmppc_restart_interrupt() that
> > restarts interrupts if they were meant for the host:
> >
> > a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
> > BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
>=20
> Those aren't the only exceptions that can end up going to the host.  We
> could get a TLB miss that results in a heavyweight MMIO exit, etc.
>
> > And shouldn't we handle kvmppc_restart_interrupt() like the original
> > HOST flow?
> >
> > #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr,
> > ack)           \
> >
> > START_EXCEPTION(label);                                         \
> >         NORMAL_EXCEPTION_PROLOG(trapnum, intnum,
> > PROLOG_ADDITION_MASKABLE)\
> >         EXCEPTION_COMMON(trapnum, PACA_EXGEN,
> > *INTS_DISABLE*)             \
> > 	...
>=20
> Could you elaborate on what you mean?

I think Tiejun was saying that host has flags and replays only EE/DEC/DBELL
interrupts. There is special macro masked_interrupt_book3e in those excepti=
on
handlers that sets paca->irq_happened.

The list of replied interrupts is limited to asynchronous noncritical
interrupts which can be masked by MSR[EE] (therefore no TLB miss). Now
on KVM book3e we don't want to put them in the irq_happened lazy state
but rather to execute them directly, so there is no reason for exception
handling symmetry between host and guest.

-Mike

RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Bhushan Bharat-R65777]

> -----Original Message-----
> From: Linuxppc-dev [mailto:linuxppc-dev-
> bounces+bharat.bhushan=3Dfreescale.com@lists.ozlabs.org] On Behalf Of Car=
aman
> Mihai Claudiu-B02008
> Sent: Wednesday, May 08, 2013 6:44 PM
> To: Wood Scott-B07421; tiejun.chen
> Cc: linuxppc-dev@lists.ozlabs.org; agraf@suse.de; kvm-ppc@vger.kernel.org=
;
> kvm@vger.kernel.org
> Subject: RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interru=
pts
>=20
> > > This only disable soft interrupt for kvmppc_restart_interrupt() that
> > > restarts interrupts if they were meant for the host:
> > >
> > > a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
> > > BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
> >
> > Those aren't the only exceptions that can end up going to the host.
> > We could get a TLB miss that results in a heavyweight MMIO exit, etc.
> >
> > > And shouldn't we handle kvmppc_restart_interrupt() like the original
> > > HOST flow?
> > >
> > > #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr,
> > > ack)           \
> > >
> > > START_EXCEPTION(label);                                         \
> > >         NORMAL_EXCEPTION_PROLOG(trapnum, intnum,
> > > PROLOG_ADDITION_MASKABLE)\
> > >         EXCEPTION_COMMON(trapnum, PACA_EXGEN,
> > > *INTS_DISABLE*)             \
> > > 	...
> >
> > Could you elaborate on what you mean?
>=20
> I think Tiejun was saying that host has flags and replays only EE/DEC/DBE=
LL
> interrupts. There is special macro masked_interrupt_book3e in those excep=
tion
> handlers that sets paca->irq_happened.
>=20
> The list of replied interrupts is limited to asynchronous noncritical int=
errupts
> which can be masked by MSR[EE] (therefore no TLB miss).

Embedded Perfmon interrupt is also asynchronous, Why that is not in the lis=
t of masked interruts.

-Bharat
=20
> Now on KVM book3e we
> don't want to put them in the irq_happened lazy state but rather to execu=
te them
> directly, so there is no reason for exception handling symmetry between h=
ost and
> guest.
>=20
> -Mike
>=20
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[tiejun.chen]

On 05/09/2013 03:33 PM, Bhushan Bharat-R65777 wrote:
>
>
>> -----Original Message-----
>> From: Linuxppc-dev [mailto:linuxppc-dev-
>> bounces+bharat.bhushan=freescale.com@lists.ozlabs.org] On Behalf Of Caraman
>> Mihai Claudiu-B02008
>> Sent: Wednesday, May 08, 2013 6:44 PM
>> To: Wood Scott-B07421; tiejun.chen
>> Cc: linuxppc-dev@lists.ozlabs.org; agraf@suse.de; kvm-ppc@vger.kernel.org;
>> kvm@vger.kernel.org
>> Subject: RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts
>>
>>>> This only disable soft interrupt for kvmppc_restart_interrupt() that
>>>> restarts interrupts if they were meant for the host:
>>>>
>>>> a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
>>>> BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
>>>
>>> Those aren't the only exceptions that can end up going to the host.
>>> We could get a TLB miss that results in a heavyweight MMIO exit, etc.
>>>
>>>> And shouldn't we handle kvmppc_restart_interrupt() like the original
>>>> HOST flow?
>>>>
>>>> #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr,
>>>> ack)           \
>>>>
>>>> START_EXCEPTION(label);                                         \
>>>>          NORMAL_EXCEPTION_PROLOG(trapnum, intnum,
>>>> PROLOG_ADDITION_MASKABLE)\
>>>>          EXCEPTION_COMMON(trapnum, PACA_EXGEN,
>>>> *INTS_DISABLE*)             \
>>>> 	...
>>>
>>> Could you elaborate on what you mean?
>>
>> I think Tiejun was saying that host has flags and replays only EE/DEC/DBELL
>> interrupts. There is special macro masked_interrupt_book3e in those exception
>> handlers that sets paca->irq_happened.
>>
>> The list of replied interrupts is limited to asynchronous noncritical interrupts
>> which can be masked by MSR[EE] (therefore no TLB miss).
>
> Embedded Perfmon interrupt is also asynchronous, Why that is not in the list of masked interruts.

Are you saying perfmon? If so, its also in that list:

         START_EXCEPTION(perfmon);
         NORMAL_EXCEPTION_PROLOG(0x260, BOOKE_INTERRUPT_PERFORMANCE_MONITOR,
                                 PROLOG_ADDITION_NONE)
         EXCEPTION_COMMON(0x260, PACA_EXGEN, INTS_DISABLE)

Tiejun

>
> -Bharat
>
>> Now on KVM book3e we
>> don't want to put them in the irq_happened lazy state but rather to execute them
>> directly, so there is no reason for exception handling symmetry between host and
>> guest.
>>
>> -Mike

RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Bhushan Bharat-R65777]

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogdGllanVuLmNoZW4gW21h
aWx0bzp0aWVqdW4uY2hlbkB3aW5kcml2ZXIuY29tXQ0KPiBTZW50OiBUaHVyc2RheSwgTWF5IDA5
LCAyMDEzIDE6MTggUE0NCj4gVG86IEJodXNoYW4gQmhhcmF0LVI2NTc3Nw0KPiBDYzogQ2FyYW1h
biBNaWhhaSBDbGF1ZGl1LUIwMjAwODsgV29vZCBTY290dC1CMDc0MjE7IGxpbnV4cHBjLQ0KPiBk
ZXZAbGlzdHMub3psYWJzLm9yZzsgYWdyYWZAc3VzZS5kZTsga3ZtLXBwY0B2Z2VyLmtlcm5lbC5v
cmc7DQo+IGt2bUB2Z2VyLmtlcm5lbC5vcmcNCj4gU3ViamVjdDogUmU6IFtSRkNdW0tWTV1bUEFU
Q0ggMS8xXSBrdm06cHBjOmJvb2tlLTY0OiBzb2Z0LWRpc2FibGUgaW50ZXJydXB0cw0KPiANCj4g
T24gMDUvMDkvMjAxMyAwMzozMyBQTSwgQmh1c2hhbiBCaGFyYXQtUjY1Nzc3IHdyb3RlOg0KPiA+
DQo+ID4NCj4gPj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPj4gRnJvbTogTGludXhw
cGMtZGV2IFttYWlsdG86bGludXhwcGMtZGV2LQ0KPiA+PiBib3VuY2VzK2JoYXJhdC5iaHVzaGFu
PWZyZWVzY2FsZS5jb21AbGlzdHMub3psYWJzLm9yZ10gT24gQmVoYWxmIE9mDQo+ID4+IGJvdW5j
ZXMrQ2FyYW1hbg0KPiA+PiBNaWhhaSBDbGF1ZGl1LUIwMjAwOA0KPiA+PiBTZW50OiBXZWRuZXNk
YXksIE1heSAwOCwgMjAxMyA2OjQ0IFBNDQo+ID4+IFRvOiBXb29kIFNjb3R0LUIwNzQyMTsgdGll
anVuLmNoZW4NCj4gPj4gQ2M6IGxpbnV4cHBjLWRldkBsaXN0cy5vemxhYnMub3JnOyBhZ3JhZkBz
dXNlLmRlOw0KPiA+PiBrdm0tcHBjQHZnZXIua2VybmVsLm9yZzsga3ZtQHZnZXIua2VybmVsLm9y
Zw0KPiA+PiBTdWJqZWN0OiBSRTogW1JGQ11bS1ZNXVtQQVRDSCAxLzFdIGt2bTpwcGM6Ym9va2Ut
NjQ6IHNvZnQtZGlzYWJsZQ0KPiA+PiBpbnRlcnJ1cHRzDQo+ID4+DQo+ID4+Pj4gVGhpcyBvbmx5
IGRpc2FibGUgc29mdCBpbnRlcnJ1cHQgZm9yIGt2bXBwY19yZXN0YXJ0X2ludGVycnVwdCgpDQo+
ID4+Pj4gdGhhdCByZXN0YXJ0cyBpbnRlcnJ1cHRzIGlmIHRoZXkgd2VyZSBtZWFudCBmb3IgdGhl
IGhvc3Q6DQo+ID4+Pj4NCj4gPj4+PiBhLiBTT0ZUX0RJU0FCTEVfSU5UUygpIG9ubHkgZm9yIEJP
T0tFX0lOVEVSUlVQVF9FWFRFUk5BTCB8DQo+ID4+Pj4gQk9PS0VfSU5URVJSVVBUX0RFQ1JFTUVO
VEVSIHwgQk9PS0VfSU5URVJSVVBUX0RPT1JCRUxMDQo+ID4+Pg0KPiA+Pj4gVGhvc2UgYXJlbid0
IHRoZSBvbmx5IGV4Y2VwdGlvbnMgdGhhdCBjYW4gZW5kIHVwIGdvaW5nIHRvIHRoZSBob3N0Lg0K
PiA+Pj4gV2UgY291bGQgZ2V0IGEgVExCIG1pc3MgdGhhdCByZXN1bHRzIGluIGEgaGVhdnl3ZWln
aHQgTU1JTyBleGl0LCBldGMuDQo+ID4+Pg0KPiA+Pj4+IEFuZCBzaG91bGRuJ3Qgd2UgaGFuZGxl
IGt2bXBwY19yZXN0YXJ0X2ludGVycnVwdCgpIGxpa2UgdGhlDQo+ID4+Pj4gb3JpZ2luYWwgSE9T
VCBmbG93Pw0KPiA+Pj4+DQo+ID4+Pj4gI2RlZmluZSBNQVNLQUJMRV9FWENFUFRJT04odHJhcG51
bSwgaW50bnVtLCBsYWJlbCwgaGRsciwNCj4gPj4+PiBhY2spICAgICAgICAgICBcDQo+ID4+Pj4N
Cj4gPj4+PiBTVEFSVF9FWENFUFRJT04obGFiZWwpOyAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgXA0KPiA+Pj4+ICAgICAgICAgIE5PUk1BTF9FWENFUFRJT05fUFJPTE9H
KHRyYXBudW0sIGludG51bSwNCj4gPj4+PiBQUk9MT0dfQURESVRJT05fTUFTS0FCTEUpXA0KPiA+
Pj4+ICAgICAgICAgIEVYQ0VQVElPTl9DT01NT04odHJhcG51bSwgUEFDQV9FWEdFTiwNCj4gPj4+
PiAqSU5UU19ESVNBQkxFKikgICAgICAgICAgICAgXA0KPiA+Pj4+IAkuLi4NCj4gPj4+DQo+ID4+
PiBDb3VsZCB5b3UgZWxhYm9yYXRlIG9uIHdoYXQgeW91IG1lYW4/DQo+ID4+DQo+ID4+IEkgdGhp
bmsgVGllanVuIHdhcyBzYXlpbmcgdGhhdCBob3N0IGhhcyBmbGFncyBhbmQgcmVwbGF5cyBvbmx5
DQo+ID4+IEVFL0RFQy9EQkVMTCBpbnRlcnJ1cHRzLiBUaGVyZSBpcyBzcGVjaWFsIG1hY3JvDQo+
ID4+IG1hc2tlZF9pbnRlcnJ1cHRfYm9vazNlIGluIHRob3NlIGV4Y2VwdGlvbiBoYW5kbGVycyB0
aGF0IHNldHMgcGFjYS0NCj4gPmlycV9oYXBwZW5lZC4NCj4gPj4NCj4gPj4gVGhlIGxpc3Qgb2Yg
cmVwbGllZCBpbnRlcnJ1cHRzIGlzIGxpbWl0ZWQgdG8gYXN5bmNocm9ub3VzIG5vbmNyaXRpY2Fs
DQo+ID4+IGludGVycnVwdHMgd2hpY2ggY2FuIGJlIG1hc2tlZCBieSBNU1JbRUVdICh0aGVyZWZv
cmUgbm8gVExCIG1pc3MpLg0KPiA+DQo+ID4gRW1iZWRkZWQgUGVyZm1vbiBpbnRlcnJ1cHQgaXMg
YWxzbyBhc3luY2hyb25vdXMsIFdoeSB0aGF0IGlzIG5vdCBpbiB0aGUgbGlzdA0KPiBvZiBtYXNr
ZWQgaW50ZXJydXRzLg0KPiANCj4gQXJlIHlvdSBzYXlpbmcgcGVyZm1vbj8gSWYgc28sIGl0cyBh
bHNvIGluIHRoYXQgbGlzdDoNCj4gDQo+ICAgICAgICAgIFNUQVJUX0VYQ0VQVElPTihwZXJmbW9u
KTsNCj4gICAgICAgICAgTk9STUFMX0VYQ0VQVElPTl9QUk9MT0coMHgyNjAsIEJPT0tFX0lOVEVS
UlVQVF9QRVJGT1JNQU5DRV9NT05JVE9SLA0KPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBQUk9MT0dfQURESVRJT05fTk9ORSkNCj4gICAgICAgICAgRVhDRVBUSU9OX0NPTU1PTigw
eDI2MCwgUEFDQV9FWEdFTiwgSU5UU19ESVNBQkxFKQ0KDQpXaGVyZSBpdCBpcyByZWNvcmRlZCBp
biBwYWNhLT5pcnFfaGFwcG5lZCB0byBiZSByZXBsYXllZCBsYXRlciA/DQoNCj4gDQo+IFRpZWp1
bg0KPiANCj4gPg0KPiA+IC1CaGFyYXQNCj4gPg0KPiA+PiBOb3cgb24gS1ZNIGJvb2szZSB3ZQ0K
PiA+PiBkb24ndCB3YW50IHRvIHB1dCB0aGVtIGluIHRoZSBpcnFfaGFwcGVuZWQgbGF6eSBzdGF0
ZSBidXQgcmF0aGVyIHRvDQo+ID4+IGV4ZWN1dGUgdGhlbSBkaXJlY3RseSwgc28gdGhlcmUgaXMg
bm8gcmVhc29uIGZvciBleGNlcHRpb24gaGFuZGxpbmcNCj4gPj4gc3ltbWV0cnkgYmV0d2VlbiBo
b3N0IGFuZCBndWVzdC4NCj4gPj4NCj4gPj4gLU1pa2UNCj4gDQoNCg==

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[tiejun.chen]

On 05/09/2013 03:51 PM, Bhushan Bharat-R65777 wrote:
>
>
>> -----Original Message-----
>> From: tiejun.chen [mailto:tiejun.chen@windriver.com]
>> Sent: Thursday, May 09, 2013 1:18 PM
>> To: Bhushan Bharat-R65777
>> Cc: Caraman Mihai Claudiu-B02008; Wood Scott-B07421; linuxppc-
>> dev@lists.ozlabs.org; agraf@suse.de; kvm-ppc@vger.kernel.org;
>> kvm@vger.kernel.org
>> Subject: Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts
>>
>> On 05/09/2013 03:33 PM, Bhushan Bharat-R65777 wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Linuxppc-dev [mailto:linuxppc-dev-
>>>> bounces+bharat.bhushan=freescale.com@lists.ozlabs.org] On Behalf Of
>>>> bounces+Caraman
>>>> Mihai Claudiu-B02008
>>>> Sent: Wednesday, May 08, 2013 6:44 PM
>>>> To: Wood Scott-B07421; tiejun.chen
>>>> Cc: linuxppc-dev@lists.ozlabs.org; agraf@suse.de;
>>>> kvm-ppc@vger.kernel.org; kvm@vger.kernel.org
>>>> Subject: RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable
>>>> interrupts
>>>>
>>>>>> This only disable soft interrupt for kvmppc_restart_interrupt()
>>>>>> that restarts interrupts if they were meant for the host:
>>>>>>
>>>>>> a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
>>>>>> BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
>>>>>
>>>>> Those aren't the only exceptions that can end up going to the host.
>>>>> We could get a TLB miss that results in a heavyweight MMIO exit, etc.
>>>>>
>>>>>> And shouldn't we handle kvmppc_restart_interrupt() like the
>>>>>> original HOST flow?
>>>>>>
>>>>>> #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr,
>>>>>> ack)           \
>>>>>>
>>>>>> START_EXCEPTION(label);                                         \
>>>>>>           NORMAL_EXCEPTION_PROLOG(trapnum, intnum,
>>>>>> PROLOG_ADDITION_MASKABLE)\
>>>>>>           EXCEPTION_COMMON(trapnum, PACA_EXGEN,
>>>>>> *INTS_DISABLE*)             \
>>>>>> 	...
>>>>>
>>>>> Could you elaborate on what you mean?
>>>>
>>>> I think Tiejun was saying that host has flags and replays only
>>>> EE/DEC/DBELL interrupts. There is special macro
>>>> masked_interrupt_book3e in those exception handlers that sets paca-
>>> irq_happened.
>>>>
>>>> The list of replied interrupts is limited to asynchronous noncritical
>>>> interrupts which can be masked by MSR[EE] (therefore no TLB miss).
>>>
>>> Embedded Perfmon interrupt is also asynchronous, Why that is not in the list
>> of masked interruts.
>>
>> Are you saying perfmon? If so, its also in that list:
>>
>>           START_EXCEPTION(perfmon);
>>           NORMAL_EXCEPTION_PROLOG(0x260, BOOKE_INTERRUPT_PERFORMANCE_MONITOR,
>>                                   PROLOG_ADDITION_NONE)
>>           EXCEPTION_COMMON(0x260, PACA_EXGEN, INTS_DISABLE)
>
> Where it is recorded in paca->irq_happned to be replayed later ?

In stead of PROLOG_ADDITION_MASKABLE, actually we have nothing to do with 
PROLOG_ADDITION_NONE for perfmon, so perfmon can be executed without lazy state.

Tiejun

>
>>
>> Tiejun
>>
>>>
>>> -Bharat
>>>
>>>> Now on KVM book3e we
>>>> don't want to put them in the irq_happened lazy state but rather to
>>>> execute them directly, so there is no reason for exception handling
>>>> symmetry between host and guest.
>>>>
>>>> -Mike
>>
>

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Kevin Hao]

[-- Attachment #1: Type: text/plain, Size: 4122 bytes --]

On Thu, May 09, 2013 at 07:51:09AM +0000, Bhushan Bharat-R65777 wrote:
> 
> 
> > -----Original Message-----
> > From: tiejun.chen [mailto:tiejun.chen@windriver.com]
> > Sent: Thursday, May 09, 2013 1:18 PM
> > To: Bhushan Bharat-R65777
> > Cc: Caraman Mihai Claudiu-B02008; Wood Scott-B07421; linuxppc-
> > dev@lists.ozlabs.org; agraf@suse.de; kvm-ppc@vger.kernel.org;
> > kvm@vger.kernel.org
> > Subject: Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts
> > 
> > On 05/09/2013 03:33 PM, Bhushan Bharat-R65777 wrote:
> > >
> > >
> > >> -----Original Message-----
> > >> From: Linuxppc-dev [mailto:linuxppc-dev-
> > >> bounces+bharat.bhushan=freescale.com@lists.ozlabs.org] On Behalf Of
> > >> bounces+Caraman
> > >> Mihai Claudiu-B02008
> > >> Sent: Wednesday, May 08, 2013 6:44 PM
> > >> To: Wood Scott-B07421; tiejun.chen
> > >> Cc: linuxppc-dev@lists.ozlabs.org; agraf@suse.de;
> > >> kvm-ppc@vger.kernel.org; kvm@vger.kernel.org
> > >> Subject: RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable
> > >> interrupts
> > >>
> > >>>> This only disable soft interrupt for kvmppc_restart_interrupt()
> > >>>> that restarts interrupts if they were meant for the host:
> > >>>>
> > >>>> a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
> > >>>> BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
> > >>>
> > >>> Those aren't the only exceptions that can end up going to the host.
> > >>> We could get a TLB miss that results in a heavyweight MMIO exit, etc.
> > >>>
> > >>>> And shouldn't we handle kvmppc_restart_interrupt() like the
> > >>>> original HOST flow?
> > >>>>
> > >>>> #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr,
> > >>>> ack)           \
> > >>>>
> > >>>> START_EXCEPTION(label);                                         \
> > >>>>          NORMAL_EXCEPTION_PROLOG(trapnum, intnum,
> > >>>> PROLOG_ADDITION_MASKABLE)\
> > >>>>          EXCEPTION_COMMON(trapnum, PACA_EXGEN,
> > >>>> *INTS_DISABLE*)             \
> > >>>> 	...
> > >>>
> > >>> Could you elaborate on what you mean?
> > >>
> > >> I think Tiejun was saying that host has flags and replays only
> > >> EE/DEC/DBELL interrupts. There is special macro
> > >> masked_interrupt_book3e in those exception handlers that sets paca-
> > >irq_happened.
> > >>
> > >> The list of replied interrupts is limited to asynchronous noncritical
> > >> interrupts which can be masked by MSR[EE] (therefore no TLB miss).
> > >
> > > Embedded Perfmon interrupt is also asynchronous, Why that is not in the list
> > of masked interruts.
> > 
> > Are you saying perfmon? If so, its also in that list:
> > 
> >          START_EXCEPTION(perfmon);
> >          NORMAL_EXCEPTION_PROLOG(0x260, BOOKE_INTERRUPT_PERFORMANCE_MONITOR,
> >                                  PROLOG_ADDITION_NONE)
> >          EXCEPTION_COMMON(0x260, PACA_EXGEN, INTS_DISABLE)
> 
> Where it is recorded in paca->irq_happned to be replayed later ?

Actually we don't want replay the perfmon interrupt later. We would run it
even soft irq is disabled and just treat it as NMI. Please see the following
function quoted from arch/powerpc/perf/core-fsl-emb.c:
  /*
   * If interrupts were soft-disabled when a PMU interrupt occurs, treat
   * it as an NMI.
   */
  static inline int perf_intr_is_nmi(struct pt_regs *regs)
  {
  #ifdef __powerpc64__                                                                                                                                                         
          return !regs->softe;
  #else
          return 0;
  #endif
  }

Thanks,
Kevin

> 
> > 
> > Tiejun
> > 
> > >
> > > -Bharat
> > >
> > >> Now on KVM book3e we
> > >> don't want to put them in the irq_happened lazy state but rather to
> > >> execute them directly, so there is no reason for exception handling
> > >> symmetry between host and guest.
> > >>
> > >> -Mike
> > 
> 
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/linuxppc-dev

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Bhushan Bharat-R65777]

> -----Original Message-----
> From: Kevin Hao [mailto:haokexin@gmail.com]
> Sent: Thursday, May 09, 2013 1:38 PM
> To: Bhushan Bharat-R65777
> Cc: tiejun.chen; Caraman Mihai Claudiu-B02008; kvm@vger.kernel.org; Wood =
Scott-
> B07421; agraf@suse.de; kvm-ppc@vger.kernel.org; linuxppc-dev@lists.ozlabs=
.org
> Subject: Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interru=
pts
>=20
> On Thu, May 09, 2013 at 07:51:09AM +0000, Bhushan Bharat-R65777 wrote:
> >
> >
> > > -----Original Message-----
> > > From: tiejun.chen [mailto:tiejun.chen@windriver.com]
> > > Sent: Thursday, May 09, 2013 1:18 PM
> > > To: Bhushan Bharat-R65777
> > > Cc: Caraman Mihai Claudiu-B02008; Wood Scott-B07421; linuxppc-
> > > dev@lists.ozlabs.org; agraf@suse.de; kvm-ppc@vger.kernel.org;
> > > kvm@vger.kernel.org
> > > Subject: Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable
> > > interrupts
> > >
> > > On 05/09/2013 03:33 PM, Bhushan Bharat-R65777 wrote:
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Linuxppc-dev [mailto:linuxppc-dev-
> > > >> bounces+bharat.bhushan=3Dfreescale.com@lists.ozlabs.org] On Behalf
> > > >> bounces+Of Caraman
> > > >> Mihai Claudiu-B02008
> > > >> Sent: Wednesday, May 08, 2013 6:44 PM
> > > >> To: Wood Scott-B07421; tiejun.chen
> > > >> Cc: linuxppc-dev@lists.ozlabs.org; agraf@suse.de;
> > > >> kvm-ppc@vger.kernel.org; kvm@vger.kernel.org
> > > >> Subject: RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable
> > > >> interrupts
> > > >>
> > > >>>> This only disable soft interrupt for kvmppc_restart_interrupt()
> > > >>>> that restarts interrupts if they were meant for the host:
> > > >>>>
> > > >>>> a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
> > > >>>> BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
> > > >>>
> > > >>> Those aren't the only exceptions that can end up going to the hos=
t.
> > > >>> We could get a TLB miss that results in a heavyweight MMIO exit, =
etc.
> > > >>>
> > > >>>> And shouldn't we handle kvmppc_restart_interrupt() like the
> > > >>>> original HOST flow?
> > > >>>>
> > > >>>> #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr,
> > > >>>> ack)           \
> > > >>>>
> > > >>>> START_EXCEPTION(label);                                         =
\
> > > >>>>          NORMAL_EXCEPTION_PROLOG(trapnum, intnum,
> > > >>>> PROLOG_ADDITION_MASKABLE)\
> > > >>>>          EXCEPTION_COMMON(trapnum, PACA_EXGEN,
> > > >>>> *INTS_DISABLE*)             \
> > > >>>> 	...
> > > >>>
> > > >>> Could you elaborate on what you mean?
> > > >>
> > > >> I think Tiejun was saying that host has flags and replays only
> > > >> EE/DEC/DBELL interrupts. There is special macro
> > > >> masked_interrupt_book3e in those exception handlers that sets
> > > >> paca-
> > > >irq_happened.
> > > >>
> > > >> The list of replied interrupts is limited to asynchronous
> > > >> noncritical interrupts which can be masked by MSR[EE] (therefore n=
o TLB
> miss).
> > > >
> > > > Embedded Perfmon interrupt is also asynchronous, Why that is not
> > > > in the list
> > > of masked interruts.
> > >
> > > Are you saying perfmon? If so, its also in that list:
> > >
> > >          START_EXCEPTION(perfmon);
> > >          NORMAL_EXCEPTION_PROLOG(0x260, BOOKE_INTERRUPT_PERFORMANCE_M=
ONITOR,
> > >                                  PROLOG_ADDITION_NONE)
> > >          EXCEPTION_COMMON(0x260, PACA_EXGEN, INTS_DISABLE)
> >
> > Where it is recorded in paca->irq_happned to be replayed later ?
>=20
> Actually we don't want replay the perfmon interrupt later. We would run i=
t even
> soft irq is disabled and just treat it as NMI. Please see the following f=
unction
> quoted from arch/powerpc/perf/core-fsl-emb.c:
>   /*
>    * If interrupts were soft-disabled when a PMU interrupt occurs, treat
>    * it as an NMI.
>    */
>   static inline int perf_intr_is_nmi(struct pt_regs *regs)
>   {
>   #ifdef __powerpc64__
>           return !regs->softe;
>   #else
>           return 0;
>   #endif
>   }

Is it because that we cannot afford to lose perfmon interrupt for more accu=
rate capturing of data ?

-Bharat

>=20
> Thanks,
> Kevin
>=20
> >
> > >
> > > Tiejun
> > >
> > > >
> > > > -Bharat
> > > >
> > > >> Now on KVM book3e we
> > > >> don't want to put them in the irq_happened lazy state but rather
> > > >> to execute them directly, so there is no reason for exception
> > > >> handling symmetry between host and guest.
> > > >>
> > > >> -Mike
> > >
> >
> > _______________________________________________
> > Linuxppc-dev mailing list
> > Linuxppc-dev@lists.ozlabs.org
> > https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[tiejun.chen]

On 05/09/2013 04:12 PM, Bhushan Bharat-R65777 wrote:
>
>
>> -----Original Message-----
>> From: Kevin Hao [mailto:haokexin@gmail.com]
>> Sent: Thursday, May 09, 2013 1:38 PM
>> To: Bhushan Bharat-R65777
>> Cc: tiejun.chen; Caraman Mihai Claudiu-B02008; kvm@vger.kernel.org; Wood Scott-
>> B07421; agraf@suse.de; kvm-ppc@vger.kernel.org; linuxppc-dev@lists.ozlabs.org
>> Subject: Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts
>>
>> On Thu, May 09, 2013 at 07:51:09AM +0000, Bhushan Bharat-R65777 wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: tiejun.chen [mailto:tiejun.chen@windriver.com]
>>>> Sent: Thursday, May 09, 2013 1:18 PM
>>>> To: Bhushan Bharat-R65777
>>>> Cc: Caraman Mihai Claudiu-B02008; Wood Scott-B07421; linuxppc-
>>>> dev@lists.ozlabs.org; agraf@suse.de; kvm-ppc@vger.kernel.org;
>>>> kvm@vger.kernel.org
>>>> Subject: Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable
>>>> interrupts
>>>>
>>>> On 05/09/2013 03:33 PM, Bhushan Bharat-R65777 wrote:
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Linuxppc-dev [mailto:linuxppc-dev-
>>>>>> bounces+bharat.bhushan=freescale.com@lists.ozlabs.org] On Behalf
>>>>>> bounces+Of Caraman
>>>>>> Mihai Claudiu-B02008
>>>>>> Sent: Wednesday, May 08, 2013 6:44 PM
>>>>>> To: Wood Scott-B07421; tiejun.chen
>>>>>> Cc: linuxppc-dev@lists.ozlabs.org; agraf@suse.de;
>>>>>> kvm-ppc@vger.kernel.org; kvm@vger.kernel.org
>>>>>> Subject: RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable
>>>>>> interrupts
>>>>>>
>>>>>>>> This only disable soft interrupt for kvmppc_restart_interrupt()
>>>>>>>> that restarts interrupts if they were meant for the host:
>>>>>>>>
>>>>>>>> a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
>>>>>>>> BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
>>>>>>>
>>>>>>> Those aren't the only exceptions that can end up going to the host.
>>>>>>> We could get a TLB miss that results in a heavyweight MMIO exit, etc.
>>>>>>>
>>>>>>>> And shouldn't we handle kvmppc_restart_interrupt() like the
>>>>>>>> original HOST flow?
>>>>>>>>
>>>>>>>> #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr,
>>>>>>>> ack)           \
>>>>>>>>
>>>>>>>> START_EXCEPTION(label);                                         \
>>>>>>>>           NORMAL_EXCEPTION_PROLOG(trapnum, intnum,
>>>>>>>> PROLOG_ADDITION_MASKABLE)\
>>>>>>>>           EXCEPTION_COMMON(trapnum, PACA_EXGEN,
>>>>>>>> *INTS_DISABLE*)             \
>>>>>>>> 	...
>>>>>>>
>>>>>>> Could you elaborate on what you mean?
>>>>>>
>>>>>> I think Tiejun was saying that host has flags and replays only
>>>>>> EE/DEC/DBELL interrupts. There is special macro
>>>>>> masked_interrupt_book3e in those exception handlers that sets
>>>>>> paca-
>>>>> irq_happened.
>>>>>>
>>>>>> The list of replied interrupts is limited to asynchronous
>>>>>> noncritical interrupts which can be masked by MSR[EE] (therefore no TLB
>> miss).
>>>>>
>>>>> Embedded Perfmon interrupt is also asynchronous, Why that is not
>>>>> in the list
>>>> of masked interruts.
>>>>
>>>> Are you saying perfmon? If so, its also in that list:
>>>>
>>>>           START_EXCEPTION(perfmon);
>>>>           NORMAL_EXCEPTION_PROLOG(0x260, BOOKE_INTERRUPT_PERFORMANCE_MONITOR,
>>>>                                   PROLOG_ADDITION_NONE)
>>>>           EXCEPTION_COMMON(0x260, PACA_EXGEN, INTS_DISABLE)
>>>
>>> Where it is recorded in paca->irq_happned to be replayed later ?
>>
>> Actually we don't want replay the perfmon interrupt later. We would run it even
>> soft irq is disabled and just treat it as NMI. Please see the following function
>> quoted from arch/powerpc/perf/core-fsl-emb.c:
>>    /*
>>     * If interrupts were soft-disabled when a PMU interrupt occurs, treat
>>     * it as an NMI.
>>     */
>>    static inline int perf_intr_is_nmi(struct pt_regs *regs)
>>    {
>>    #ifdef __powerpc64__
>>            return !regs->softe;
>>    #else
>>            return 0;
>>    #endif
>>    }
>
> Is it because that we cannot afford to lose perfmon interrupt for more accurate capturing of data ?

     powerpc/perf: e500 support

     This implements perf_event support for the Freescale embedded performance
     monitor, based on the existing perf_event.c that supports server/classic
     chips.

     Some limitations:
     - Performance monitor interrupts are regular EE interrupts, and thus you
       can't profile places with interrupts disabled.  We may want to implement
       soft IRQ-disabling, with perfmon interrupts exempted and treated as NMIs.

Tiejun

>
> -Bharat
>
>>
>> Thanks,
>> Kevin
>>
>>>
>>>>
>>>> Tiejun
>>>>
>>>>>
>>>>> -Bharat
>>>>>
>>>>>> Now on KVM book3e we
>>>>>> don't want to put them in the irq_happened lazy state but rather
>>>>>> to execute them directly, so there is no reason for exception
>>>>>> handling symmetry between host and guest.
>>>>>>
>>>>>> -Mike
>>>>
>>>
>>> _______________________________________________
>>> Linuxppc-dev mailing list
>>> Linuxppc-dev@lists.ozlabs.org
>>> https://lists.ozlabs.org/listinfo/linuxppc-dev
>
>
>

RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Bhushan Bharat-R65777]

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogdGllanVuLmNoZW4gW21h
aWx0bzp0aWVqdW4uY2hlbkB3aW5kcml2ZXIuY29tXQ0KPiBTZW50OiBUaHVyc2RheSwgTWF5IDA5
LCAyMDEzIDE6NDggUE0NCj4gVG86IEJodXNoYW4gQmhhcmF0LVI2NTc3Nw0KPiBDYzogS2V2aW4g
SGFvOyBDYXJhbWFuIE1paGFpIENsYXVkaXUtQjAyMDA4OyBrdm1Admdlci5rZXJuZWwub3JnOyBX
b29kIFNjb3R0LQ0KPiBCMDc0MjE7IGFncmFmQHN1c2UuZGU7IGt2bS1wcGNAdmdlci5rZXJuZWwu
b3JnOyBsaW51eHBwYy1kZXZAbGlzdHMub3psYWJzLm9yZw0KPiBTdWJqZWN0OiBSZTogW1JGQ11b
S1ZNXVtQQVRDSCAxLzFdIGt2bTpwcGM6Ym9va2UtNjQ6IHNvZnQtZGlzYWJsZSBpbnRlcnJ1cHRz
DQo+IA0KPiBPbiAwNS8wOS8yMDEzIDA0OjEyIFBNLCBCaHVzaGFuIEJoYXJhdC1SNjU3Nzcgd3Jv
dGU6DQo+ID4NCj4gPg0KPiA+PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+PiBGcm9t
OiBLZXZpbiBIYW8gW21haWx0bzpoYW9rZXhpbkBnbWFpbC5jb21dDQo+ID4+IFNlbnQ6IFRodXJz
ZGF5LCBNYXkgMDksIDIwMTMgMTozOCBQTQ0KPiA+PiBUbzogQmh1c2hhbiBCaGFyYXQtUjY1Nzc3
DQo+ID4+IENjOiB0aWVqdW4uY2hlbjsgQ2FyYW1hbiBNaWhhaSBDbGF1ZGl1LUIwMjAwODsga3Zt
QHZnZXIua2VybmVsLm9yZzsNCj4gPj4gV29vZCBTY290dC0gQjA3NDIxOyBhZ3JhZkBzdXNlLmRl
OyBrdm0tcHBjQHZnZXIua2VybmVsLm9yZzsNCj4gPj4gbGludXhwcGMtZGV2QGxpc3RzLm96bGFi
cy5vcmcNCj4gPj4gU3ViamVjdDogUmU6IFtSRkNdW0tWTV1bUEFUQ0ggMS8xXSBrdm06cHBjOmJv
b2tlLTY0OiBzb2Z0LWRpc2FibGUNCj4gPj4gaW50ZXJydXB0cw0KPiA+Pg0KPiA+PiBPbiBUaHUs
IE1heSAwOSwgMjAxMyBhdCAwNzo1MTowOUFNICswMDAwLCBCaHVzaGFuIEJoYXJhdC1SNjU3Nzcg
d3JvdGU6DQo+ID4+Pg0KPiA+Pj4NCj4gPj4+PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0K
PiA+Pj4+IEZyb206IHRpZWp1bi5jaGVuIFttYWlsdG86dGllanVuLmNoZW5Ad2luZHJpdmVyLmNv
bV0NCj4gPj4+PiBTZW50OiBUaHVyc2RheSwgTWF5IDA5LCAyMDEzIDE6MTggUE0NCj4gPj4+PiBU
bzogQmh1c2hhbiBCaGFyYXQtUjY1Nzc3DQo+ID4+Pj4gQ2M6IENhcmFtYW4gTWloYWkgQ2xhdWRp
dS1CMDIwMDg7IFdvb2QgU2NvdHQtQjA3NDIxOyBsaW51eHBwYy0NCj4gPj4+PiBkZXZAbGlzdHMu
b3psYWJzLm9yZzsgYWdyYWZAc3VzZS5kZTsga3ZtLXBwY0B2Z2VyLmtlcm5lbC5vcmc7DQo+ID4+
Pj4ga3ZtQHZnZXIua2VybmVsLm9yZw0KPiA+Pj4+IFN1YmplY3Q6IFJlOiBbUkZDXVtLVk1dW1BB
VENIIDEvMV0ga3ZtOnBwYzpib29rZS02NDogc29mdC1kaXNhYmxlDQo+ID4+Pj4gaW50ZXJydXB0
cw0KPiA+Pj4+DQo+ID4+Pj4gT24gMDUvMDkvMjAxMyAwMzozMyBQTSwgQmh1c2hhbiBCaGFyYXQt
UjY1Nzc3IHdyb3RlOg0KPiA+Pj4+Pg0KPiA+Pj4+Pg0KPiA+Pj4+Pj4gLS0tLS1PcmlnaW5hbCBN
ZXNzYWdlLS0tLS0NCj4gPj4+Pj4+IEZyb206IExpbnV4cHBjLWRldiBbbWFpbHRvOmxpbnV4cHBj
LWRldi0NCj4gPj4+Pj4+IGJvdW5jZXMrYmhhcmF0LmJodXNoYW49ZnJlZXNjYWxlLmNvbUBsaXN0
cy5vemxhYnMub3JnXSBPbiBCZWhhbGYNCj4gPj4+Pj4+IGJvdW5jZXMrT2YgQ2FyYW1hbg0KPiA+
Pj4+Pj4gTWloYWkgQ2xhdWRpdS1CMDIwMDgNCj4gPj4+Pj4+IFNlbnQ6IFdlZG5lc2RheSwgTWF5
IDA4LCAyMDEzIDY6NDQgUE0NCj4gPj4+Pj4+IFRvOiBXb29kIFNjb3R0LUIwNzQyMTsgdGllanVu
LmNoZW4NCj4gPj4+Pj4+IENjOiBsaW51eHBwYy1kZXZAbGlzdHMub3psYWJzLm9yZzsgYWdyYWZA
c3VzZS5kZTsNCj4gPj4+Pj4+IGt2bS1wcGNAdmdlci5rZXJuZWwub3JnOyBrdm1Admdlci5rZXJu
ZWwub3JnDQo+ID4+Pj4+PiBTdWJqZWN0OiBSRTogW1JGQ11bS1ZNXVtQQVRDSCAxLzFdIGt2bTpw
cGM6Ym9va2UtNjQ6IHNvZnQtZGlzYWJsZQ0KPiA+Pj4+Pj4gaW50ZXJydXB0cw0KPiA+Pj4+Pj4N
Cj4gPj4+Pj4+Pj4gVGhpcyBvbmx5IGRpc2FibGUgc29mdCBpbnRlcnJ1cHQgZm9yIGt2bXBwY19y
ZXN0YXJ0X2ludGVycnVwdCgpDQo+ID4+Pj4+Pj4+IHRoYXQgcmVzdGFydHMgaW50ZXJydXB0cyBp
ZiB0aGV5IHdlcmUgbWVhbnQgZm9yIHRoZSBob3N0Og0KPiA+Pj4+Pj4+Pg0KPiA+Pj4+Pj4+PiBh
LiBTT0ZUX0RJU0FCTEVfSU5UUygpIG9ubHkgZm9yIEJPT0tFX0lOVEVSUlVQVF9FWFRFUk5BTCB8
DQo+ID4+Pj4+Pj4+IEJPT0tFX0lOVEVSUlVQVF9ERUNSRU1FTlRFUiB8IEJPT0tFX0lOVEVSUlVQ
VF9ET09SQkVMTA0KPiA+Pj4+Pj4+DQo+ID4+Pj4+Pj4gVGhvc2UgYXJlbid0IHRoZSBvbmx5IGV4
Y2VwdGlvbnMgdGhhdCBjYW4gZW5kIHVwIGdvaW5nIHRvIHRoZSBob3N0Lg0KPiA+Pj4+Pj4+IFdl
IGNvdWxkIGdldCBhIFRMQiBtaXNzIHRoYXQgcmVzdWx0cyBpbiBhIGhlYXZ5d2VpZ2h0IE1NSU8g
ZXhpdCwgZXRjLg0KPiA+Pj4+Pj4+DQo+ID4+Pj4+Pj4+IEFuZCBzaG91bGRuJ3Qgd2UgaGFuZGxl
IGt2bXBwY19yZXN0YXJ0X2ludGVycnVwdCgpIGxpa2UgdGhlDQo+ID4+Pj4+Pj4+IG9yaWdpbmFs
IEhPU1QgZmxvdz8NCj4gPj4+Pj4+Pj4NCj4gPj4+Pj4+Pj4gI2RlZmluZSBNQVNLQUJMRV9FWENF
UFRJT04odHJhcG51bSwgaW50bnVtLCBsYWJlbCwgaGRsciwNCj4gPj4+Pj4+Pj4gYWNrKSAgICAg
ICAgICAgXA0KPiA+Pj4+Pj4+Pg0KPiA+Pj4+Pj4+PiBTVEFSVF9FWENFUFRJT04obGFiZWwpOyAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXA0KPiA+Pj4+Pj4+PiAgICAg
ICAgICAgTk9STUFMX0VYQ0VQVElPTl9QUk9MT0codHJhcG51bSwgaW50bnVtLA0KPiA+Pj4+Pj4+
PiBQUk9MT0dfQURESVRJT05fTUFTS0FCTEUpXA0KPiA+Pj4+Pj4+PiAgICAgICAgICAgRVhDRVBU
SU9OX0NPTU1PTih0cmFwbnVtLCBQQUNBX0VYR0VOLA0KPiA+Pj4+Pj4+PiAqSU5UU19ESVNBQkxF
KikgICAgICAgICAgICAgXA0KPiA+Pj4+Pj4+PiAJLi4uDQo+ID4+Pj4+Pj4NCj4gPj4+Pj4+PiBD
b3VsZCB5b3UgZWxhYm9yYXRlIG9uIHdoYXQgeW91IG1lYW4/DQo+ID4+Pj4+Pg0KPiA+Pj4+Pj4g
SSB0aGluayBUaWVqdW4gd2FzIHNheWluZyB0aGF0IGhvc3QgaGFzIGZsYWdzIGFuZCByZXBsYXlz
IG9ubHkNCj4gPj4+Pj4+IEVFL0RFQy9EQkVMTCBpbnRlcnJ1cHRzLiBUaGVyZSBpcyBzcGVjaWFs
IG1hY3JvDQo+ID4+Pj4+PiBtYXNrZWRfaW50ZXJydXB0X2Jvb2szZSBpbiB0aG9zZSBleGNlcHRp
b24gaGFuZGxlcnMgdGhhdCBzZXRzDQo+ID4+Pj4+PiBwYWNhLQ0KPiA+Pj4+PiBpcnFfaGFwcGVu
ZWQuDQo+ID4+Pj4+Pg0KPiA+Pj4+Pj4gVGhlIGxpc3Qgb2YgcmVwbGllZCBpbnRlcnJ1cHRzIGlz
IGxpbWl0ZWQgdG8gYXN5bmNocm9ub3VzDQo+ID4+Pj4+PiBub25jcml0aWNhbCBpbnRlcnJ1cHRz
IHdoaWNoIGNhbiBiZSBtYXNrZWQgYnkgTVNSW0VFXSAodGhlcmVmb3JlDQo+ID4+Pj4+PiBubyBU
TEINCj4gPj4gbWlzcykuDQo+ID4+Pj4+DQo+ID4+Pj4+IEVtYmVkZGVkIFBlcmZtb24gaW50ZXJy
dXB0IGlzIGFsc28gYXN5bmNocm9ub3VzLCBXaHkgdGhhdCBpcyBub3QNCj4gPj4+Pj4gaW4gdGhl
IGxpc3QNCj4gPj4+PiBvZiBtYXNrZWQgaW50ZXJydXRzLg0KPiA+Pj4+DQo+ID4+Pj4gQXJlIHlv
dSBzYXlpbmcgcGVyZm1vbj8gSWYgc28sIGl0cyBhbHNvIGluIHRoYXQgbGlzdDoNCj4gPj4+Pg0K
PiA+Pj4+ICAgICAgICAgICBTVEFSVF9FWENFUFRJT04ocGVyZm1vbik7DQo+ID4+Pj4gICAgICAg
ICAgIE5PUk1BTF9FWENFUFRJT05fUFJPTE9HKDB4MjYwLA0KPiBCT09LRV9JTlRFUlJVUFRfUEVS
Rk9STUFOQ0VfTU9OSVRPUiwNCj4gPj4+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgUFJPTE9HX0FERElUSU9OX05PTkUpDQo+ID4+Pj4gICAgICAgICAgIEVYQ0VQVElPTl9DT01N
T04oMHgyNjAsIFBBQ0FfRVhHRU4sIElOVFNfRElTQUJMRSkNCj4gPj4+DQo+ID4+PiBXaGVyZSBp
dCBpcyByZWNvcmRlZCBpbiBwYWNhLT5pcnFfaGFwcG5lZCB0byBiZSByZXBsYXllZCBsYXRlciA/
DQo+ID4+DQo+ID4+IEFjdHVhbGx5IHdlIGRvbid0IHdhbnQgcmVwbGF5IHRoZSBwZXJmbW9uIGlu
dGVycnVwdCBsYXRlci4gV2Ugd291bGQNCj4gPj4gcnVuIGl0IGV2ZW4gc29mdCBpcnEgaXMgZGlz
YWJsZWQgYW5kIGp1c3QgdHJlYXQgaXQgYXMgTk1JLiBQbGVhc2Ugc2VlDQo+ID4+IHRoZSBmb2xs
b3dpbmcgZnVuY3Rpb24gcXVvdGVkIGZyb20gYXJjaC9wb3dlcnBjL3BlcmYvY29yZS1mc2wtZW1i
LmM6DQo+ID4+ICAgIC8qDQo+ID4+ICAgICAqIElmIGludGVycnVwdHMgd2VyZSBzb2Z0LWRpc2Fi
bGVkIHdoZW4gYSBQTVUgaW50ZXJydXB0IG9jY3VycywgdHJlYXQNCj4gPj4gICAgICogaXQgYXMg
YW4gTk1JLg0KPiA+PiAgICAgKi8NCj4gPj4gICAgc3RhdGljIGlubGluZSBpbnQgcGVyZl9pbnRy
X2lzX25taShzdHJ1Y3QgcHRfcmVncyAqcmVncykNCj4gPj4gICAgew0KPiA+PiAgICAjaWZkZWYg
X19wb3dlcnBjNjRfXw0KPiA+PiAgICAgICAgICAgIHJldHVybiAhcmVncy0+c29mdGU7DQo+ID4+
ICAgICNlbHNlDQo+ID4+ICAgICAgICAgICAgcmV0dXJuIDA7DQo+ID4+ICAgICNlbmRpZg0KPiA+
PiAgICB9DQo+ID4NCj4gPiBJcyBpdCBiZWNhdXNlIHRoYXQgd2UgY2Fubm90IGFmZm9yZCB0byBs
b3NlIHBlcmZtb24gaW50ZXJydXB0IGZvciBtb3JlDQo+IGFjY3VyYXRlIGNhcHR1cmluZyBvZiBk
YXRhID8NCj4gDQo+ICAgICAgcG93ZXJwYy9wZXJmOiBlNTAwIHN1cHBvcnQNCj4gDQo+ICAgICAg
VGhpcyBpbXBsZW1lbnRzIHBlcmZfZXZlbnQgc3VwcG9ydCBmb3IgdGhlIEZyZWVzY2FsZSBlbWJl
ZGRlZCBwZXJmb3JtYW5jZQ0KPiAgICAgIG1vbml0b3IsIGJhc2VkIG9uIHRoZSBleGlzdGluZyBw
ZXJmX2V2ZW50LmMgdGhhdCBzdXBwb3J0cyBzZXJ2ZXIvY2xhc3NpYw0KPiAgICAgIGNoaXBzLg0K
PiANCj4gICAgICBTb21lIGxpbWl0YXRpb25zOg0KPiAgICAgIC0gUGVyZm9ybWFuY2UgbW9uaXRv
ciBpbnRlcnJ1cHRzIGFyZSByZWd1bGFyIEVFIGludGVycnVwdHMsIGFuZCB0aHVzIHlvdQ0KPiAg
ICAgICAgY2FuJ3QgcHJvZmlsZSBwbGFjZXMgd2l0aCBpbnRlcnJ1cHRzIGRpc2FibGVkLiAgV2Ug
bWF5IHdhbnQgdG8gaW1wbGVtZW50DQo+ICAgICAgICBzb2Z0IElSUS1kaXNhYmxpbmcsIHdpdGgg
cGVyZm1vbiBpbnRlcnJ1cHRzIGV4ZW1wdGVkIGFuZCB0cmVhdGVkIGFzIE5NSXMuDQoNCkFoaCwg
dGhhdCBnaXZlcyB0aGUgYW5zd2VyIGFuZCBzYW1lIGFzIEkgZXhwZWN0ZWQgOikNCg0KLUJoYXJh
dA0KDQo+IA0KPiBUaWVqdW4NCj4gDQo+ID4NCj4gPiAtQmhhcmF0DQo+ID4NCj4gPj4NCj4gPj4g
VGhhbmtzLA0KPiA+PiBLZXZpbg0KPiA+Pg0KPiA+Pj4NCj4gPj4+Pg0KPiA+Pj4+IFRpZWp1bg0K
PiA+Pj4+DQo+ID4+Pj4+DQo+ID4+Pj4+IC1CaGFyYXQNCj4gPj4+Pj4NCj4gPj4+Pj4+IE5vdyBv
biBLVk0gYm9vazNlIHdlDQo+ID4+Pj4+PiBkb24ndCB3YW50IHRvIHB1dCB0aGVtIGluIHRoZSBp
cnFfaGFwcGVuZWQgbGF6eSBzdGF0ZSBidXQgcmF0aGVyDQo+ID4+Pj4+PiB0byBleGVjdXRlIHRo
ZW0gZGlyZWN0bHksIHNvIHRoZXJlIGlzIG5vIHJlYXNvbiBmb3IgZXhjZXB0aW9uDQo+ID4+Pj4+
PiBoYW5kbGluZyBzeW1tZXRyeSBiZXR3ZWVuIGhvc3QgYW5kIGd1ZXN0Lg0KPiA+Pj4+Pj4NCj4g
Pj4+Pj4+IC1NaWtlDQo+ID4+Pj4NCj4gPj4+DQo+ID4+PiBfX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fXw0KPiA+Pj4gTGludXhwcGMtZGV2IG1haWxpbmcgbGlz
dA0KPiA+Pj4gTGludXhwcGMtZGV2QGxpc3RzLm96bGFicy5vcmcNCj4gPj4+IGh0dHBzOi8vbGlz
dHMub3psYWJzLm9yZy9saXN0aW5mby9saW51eHBwYy1kZXYNCj4gPg0KPiA+DQo+ID4NCj4gDQoN
Cg==

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Kevin Hao]

[-- Attachment #1: Type: text/plain, Size: 5141 bytes --]

On Thu, May 09, 2013 at 08:12:51AM +0000, Bhushan Bharat-R65777 wrote:
> 
> 
> > -----Original Message-----
> > From: Kevin Hao [mailto:haokexin@gmail.com]
> > Sent: Thursday, May 09, 2013 1:38 PM
> > To: Bhushan Bharat-R65777
> > Cc: tiejun.chen; Caraman Mihai Claudiu-B02008; kvm@vger.kernel.org; Wood Scott-
> > B07421; agraf@suse.de; kvm-ppc@vger.kernel.org; linuxppc-dev@lists.ozlabs.org
> > Subject: Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts
> > 
> > On Thu, May 09, 2013 at 07:51:09AM +0000, Bhushan Bharat-R65777 wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: tiejun.chen [mailto:tiejun.chen@windriver.com]
> > > > Sent: Thursday, May 09, 2013 1:18 PM
> > > > To: Bhushan Bharat-R65777
> > > > Cc: Caraman Mihai Claudiu-B02008; Wood Scott-B07421; linuxppc-
> > > > dev@lists.ozlabs.org; agraf@suse.de; kvm-ppc@vger.kernel.org;
> > > > kvm@vger.kernel.org
> > > > Subject: Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable
> > > > interrupts
> > > >
> > > > On 05/09/2013 03:33 PM, Bhushan Bharat-R65777 wrote:
> > > > >
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Linuxppc-dev [mailto:linuxppc-dev-
> > > > >> bounces+bharat.bhushan=freescale.com@lists.ozlabs.org] On Behalf
> > > > >> bounces+Of Caraman
> > > > >> Mihai Claudiu-B02008
> > > > >> Sent: Wednesday, May 08, 2013 6:44 PM
> > > > >> To: Wood Scott-B07421; tiejun.chen
> > > > >> Cc: linuxppc-dev@lists.ozlabs.org; agraf@suse.de;
> > > > >> kvm-ppc@vger.kernel.org; kvm@vger.kernel.org
> > > > >> Subject: RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable
> > > > >> interrupts
> > > > >>
> > > > >>>> This only disable soft interrupt for kvmppc_restart_interrupt()
> > > > >>>> that restarts interrupts if they were meant for the host:
> > > > >>>>
> > > > >>>> a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
> > > > >>>> BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
> > > > >>>
> > > > >>> Those aren't the only exceptions that can end up going to the host.
> > > > >>> We could get a TLB miss that results in a heavyweight MMIO exit, etc.
> > > > >>>
> > > > >>>> And shouldn't we handle kvmppc_restart_interrupt() like the
> > > > >>>> original HOST flow?
> > > > >>>>
> > > > >>>> #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr,
> > > > >>>> ack)           \
> > > > >>>>
> > > > >>>> START_EXCEPTION(label);                                         \
> > > > >>>>          NORMAL_EXCEPTION_PROLOG(trapnum, intnum,
> > > > >>>> PROLOG_ADDITION_MASKABLE)\
> > > > >>>>          EXCEPTION_COMMON(trapnum, PACA_EXGEN,
> > > > >>>> *INTS_DISABLE*)             \
> > > > >>>> 	...
> > > > >>>
> > > > >>> Could you elaborate on what you mean?
> > > > >>
> > > > >> I think Tiejun was saying that host has flags and replays only
> > > > >> EE/DEC/DBELL interrupts. There is special macro
> > > > >> masked_interrupt_book3e in those exception handlers that sets
> > > > >> paca-
> > > > >irq_happened.
> > > > >>
> > > > >> The list of replied interrupts is limited to asynchronous
> > > > >> noncritical interrupts which can be masked by MSR[EE] (therefore no TLB
> > miss).
> > > > >
> > > > > Embedded Perfmon interrupt is also asynchronous, Why that is not
> > > > > in the list
> > > > of masked interruts.
> > > >
> > > > Are you saying perfmon? If so, its also in that list:
> > > >
> > > >          START_EXCEPTION(perfmon);
> > > >          NORMAL_EXCEPTION_PROLOG(0x260, BOOKE_INTERRUPT_PERFORMANCE_MONITOR,
> > > >                                  PROLOG_ADDITION_NONE)
> > > >          EXCEPTION_COMMON(0x260, PACA_EXGEN, INTS_DISABLE)
> > >
> > > Where it is recorded in paca->irq_happned to be replayed later ?
> > 
> > Actually we don't want replay the perfmon interrupt later. We would run it even
> > soft irq is disabled and just treat it as NMI. Please see the following function
> > quoted from arch/powerpc/perf/core-fsl-emb.c:
> >   /*
> >    * If interrupts were soft-disabled when a PMU interrupt occurs, treat
> >    * it as an NMI.
> >    */
> >   static inline int perf_intr_is_nmi(struct pt_regs *regs)
> >   {
> >   #ifdef __powerpc64__
> >           return !regs->softe;
> >   #else
> >           return 0;
> >   #endif
> >   }
> 
> Is it because that we cannot afford to lose perfmon interrupt for more accurate capturing of data ?

Yes, I think this will definitely improve the perf sample quality.

Thanks,
Kevin

> 
> -Bharat
> 
> > 
> > Thanks,
> > Kevin
> > 
> > >
> > > >
> > > > Tiejun
> > > >
> > > > >
> > > > > -Bharat
> > > > >
> > > > >> Now on KVM book3e we
> > > > >> don't want to put them in the irq_happened lazy state but rather
> > > > >> to execute them directly, so there is no reason for exception
> > > > >> handling symmetry between host and guest.
> > > > >>
> > > > >> -Mike
> > > >
> > >
> > > _______________________________________________
> > > Linuxppc-dev mailing list
> > > Linuxppc-dev@lists.ozlabs.org
> > > https://lists.ozlabs.org/listinfo/linuxppc-dev
> 

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Benjamin Herrenschmidt]

On Thu, 2013-05-09 at 16:21 +0800, Kevin Hao wrote:
> > Is it because that we cannot afford to lose perfmon interrupt for
> more accurate capturing of data ?
> 
> Yes, I think this will definitely improve the perf sample quality.

This is one of the primary reason why we implemented lazy disabling in
the first place and why I recently reworked it to decrease the periods
where we are hard disabled.

The other reasons are that storing bytes to the PACA is faster than
manipulating EE on many processors.

Cheers,
Ben.

RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Bhushan Bharat-R65777]

> -----Original Message-----
> From: Linuxppc-dev [mailto:linuxppc-dev-
> bounces+bharat.bhushan=3Dfreescale.com@lists.ozlabs.org] On Behalf Of Car=
aman
> Mihai Claudiu-B02008
> Sent: Wednesday, May 08, 2013 6:44 PM
> To: Wood Scott-B07421; tiejun.chen
> Cc: linuxppc-dev@lists.ozlabs.org; agraf@suse.de; kvm-ppc@vger.kernel.org=
;
> kvm@vger.kernel.org
> Subject: RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interru=
pts
>=20
> > > This only disable soft interrupt for kvmppc_restart_interrupt() that
> > > restarts interrupts if they were meant for the host:
> > >
> > > a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
> > > BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
> >
> > Those aren't the only exceptions that can end up going to the host.
> > We could get a TLB miss that results in a heavyweight MMIO exit, etc.
> >
> > > And shouldn't we handle kvmppc_restart_interrupt() like the original
> > > HOST flow?
> > >
> > > #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr,
> > > ack)           \
> > >
> > > START_EXCEPTION(label);                                         \
> > >         NORMAL_EXCEPTION_PROLOG(trapnum, intnum,
> > > PROLOG_ADDITION_MASKABLE)\
> > >         EXCEPTION_COMMON(trapnum, PACA_EXGEN,
> > > *INTS_DISABLE*)             \
> > > 	...
> >
> > Could you elaborate on what you mean?
>=20
> I think Tiejun was saying that host has flags and replays only EE/DEC/DBE=
LL
> interrupts. There is special macro masked_interrupt_book3e in those excep=
tion
> handlers that sets paca->irq_happened.
>=20
> The list of replied interrupts is limited to asynchronous noncritical int=
errupts
> which can be masked by MSR[EE] (therefore no TLB miss). Now on KVM book3e=
 we
> don't want to put them in the irq_happened lazy state but rather to execu=
te them
> directly, so there is no reason for exception handling symmetry between h=
ost and
> guest.


Another Question:=20

The case is:=20

Case 1)
 -> Local_irq_disable()  will set soft_enabled =3D 0
 -> Now Externel interrupt happens, there we set PACA_IRQ_EE in irq_happene=
d, Also clears EE in SRR1 and rfi. So interrupts are hard disabled. No more=
 other interrupt gated by MSR.EE can happen. Looks like the idea here is to=
 not let a device keep on inserting interrupt till the interrupt condition =
on device is cleared, right?
 -> local_irq_enable() - This checks that irq_happened is set, and replays

Now the case 2)
Case 2)
-> Local_irq_disable()  will set soft_enabled =3D 0
 -> Now DEC interrupt happens. We set PACA_IRQ_DEC in irq_happened, But do =
not clear EE in SRR1 and rfi. So interrupts are not hard disabled.=20
 -> Now say EE interrupt happens, there we set PACA_IRQ_EE in irq_happened,=
 Also clears EE in SRR1 and rfi. So interrupts are hard disabled.=20
 -> local_irq_enable() - This checks that irq_happened is set.
IIUC, it replays only one interrupt? is not it?

-Bharat

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[tiejun.chen]

On 05/09/2013 04:23 PM, Bhushan Bharat-R65777 wrote:
>
>
>> -----Original Message-----
>> From: Linuxppc-dev [mailto:linuxppc-dev-
>> bounces+bharat.bhushan=freescale.com@lists.ozlabs.org] On Behalf Of Caraman
>> Mihai Claudiu-B02008
>> Sent: Wednesday, May 08, 2013 6:44 PM
>> To: Wood Scott-B07421; tiejun.chen
>> Cc: linuxppc-dev@lists.ozlabs.org; agraf@suse.de; kvm-ppc@vger.kernel.org;
>> kvm@vger.kernel.org
>> Subject: RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts
>>
>>>> This only disable soft interrupt for kvmppc_restart_interrupt() that
>>>> restarts interrupts if they were meant for the host:
>>>>
>>>> a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
>>>> BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
>>>
>>> Those aren't the only exceptions that can end up going to the host.
>>> We could get a TLB miss that results in a heavyweight MMIO exit, etc.
>>>
>>>> And shouldn't we handle kvmppc_restart_interrupt() like the original
>>>> HOST flow?
>>>>
>>>> #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr,
>>>> ack)           \
>>>>
>>>> START_EXCEPTION(label);                                         \
>>>>          NORMAL_EXCEPTION_PROLOG(trapnum, intnum,
>>>> PROLOG_ADDITION_MASKABLE)\
>>>>          EXCEPTION_COMMON(trapnum, PACA_EXGEN,
>>>> *INTS_DISABLE*)             \
>>>> 	...
>>>
>>> Could you elaborate on what you mean?
>>
>> I think Tiejun was saying that host has flags and replays only EE/DEC/DBELL
>> interrupts. There is special macro masked_interrupt_book3e in those exception
>> handlers that sets paca->irq_happened.
>>
>> The list of replied interrupts is limited to asynchronous noncritical interrupts
>> which can be masked by MSR[EE] (therefore no TLB miss). Now on KVM book3e we
>> don't want to put them in the irq_happened lazy state but rather to execute them
>> directly, so there is no reason for exception handling symmetry between host and
>> guest.
>
>
> Another Question:
>
> The case is:
>

Actually in the case GS=1 even if EE=0, EXT/DEC/DBELL still occur as I recall.

> Case 1)
>   -> Local_irq_disable()  will set soft_enabled = 0
>   -> Now Externel interrupt happens, there we set PACA_IRQ_EE in irq_happened, Also clears EE in SRR1 and rfi. So interrupts are hard disabled. No more other interrupt gated by MSR.EE can happen. Looks like the idea here is to not let a device keep on inserting interrupt till the interrupt condition on device is cleared, right?

I don't understand "the interrupt condition on device is cleared" here.

I think regardless if you clear the device interrupt status, the system still 
receive a pending interrupt once EE or GS = 1.

>   -> local_irq_enable() - This checks that irq_happened is set, and replays

ret_from_except also check to replay.

>
> Now the case 2)
> Case 2)
> -> Local_irq_disable()  will set soft_enabled = 0
>   -> Now DEC interrupt happens. We set PACA_IRQ_DEC in irq_happened, But do not clear EE in SRR1 and rfi. So interrupts are not hard disabled.
>   -> Now say EE interrupt happens, there we set PACA_IRQ_EE in irq_happened, Also clears EE in SRR1 and rfi. So interrupts are hard disabled.
>   -> local_irq_enable() - This checks that irq_happened is set.
> IIUC, it replays only one interrupt? is not it?

After anyone is replayed in arch_local_irq_restore(), we will set soft/hard 
interrupt there:

set_soft_enabled(1);
__hard_irq_enable();

Then any pending interrupt can be executed now.

Additionally, ret_from_except probably check to replay all.

Tiejun

RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Bhushan Bharat-R65777]

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogdGllanVuLmNoZW4gW21h
aWx0bzp0aWVqdW4uY2hlbkB3aW5kcml2ZXIuY29tXQ0KPiBTZW50OiBUaHVyc2RheSwgTWF5IDA5
LCAyMDEzIDM6MTUgUE0NCj4gVG86IEJodXNoYW4gQmhhcmF0LVI2NTc3Nw0KPiBDYzogQ2FyYW1h
biBNaWhhaSBDbGF1ZGl1LUIwMjAwODsgV29vZCBTY290dC1CMDc0MjE7IGxpbnV4cHBjLQ0KPiBk
ZXZAbGlzdHMub3psYWJzLm9yZzsgYWdyYWZAc3VzZS5kZTsga3ZtLXBwY0B2Z2VyLmtlcm5lbC5v
cmc7DQo+IGt2bUB2Z2VyLmtlcm5lbC5vcmcNCj4gU3ViamVjdDogUmU6IFtSRkNdW0tWTV1bUEFU
Q0ggMS8xXSBrdm06cHBjOmJvb2tlLTY0OiBzb2Z0LWRpc2FibGUgaW50ZXJydXB0cw0KPiANCj4g
T24gMDUvMDkvMjAxMyAwNDoyMyBQTSwgQmh1c2hhbiBCaGFyYXQtUjY1Nzc3IHdyb3RlOg0KPiA+
DQo+ID4NCj4gPj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPj4gRnJvbTogTGludXhw
cGMtZGV2IFttYWlsdG86bGludXhwcGMtZGV2LQ0KPiA+PiBib3VuY2VzK2JoYXJhdC5iaHVzaGFu
PWZyZWVzY2FsZS5jb21AbGlzdHMub3psYWJzLm9yZ10gT24gQmVoYWxmIE9mDQo+ID4+IGJvdW5j
ZXMrQ2FyYW1hbg0KPiA+PiBNaWhhaSBDbGF1ZGl1LUIwMjAwOA0KPiA+PiBTZW50OiBXZWRuZXNk
YXksIE1heSAwOCwgMjAxMyA2OjQ0IFBNDQo+ID4+IFRvOiBXb29kIFNjb3R0LUIwNzQyMTsgdGll
anVuLmNoZW4NCj4gPj4gQ2M6IGxpbnV4cHBjLWRldkBsaXN0cy5vemxhYnMub3JnOyBhZ3JhZkBz
dXNlLmRlOw0KPiA+PiBrdm0tcHBjQHZnZXIua2VybmVsLm9yZzsga3ZtQHZnZXIua2VybmVsLm9y
Zw0KPiA+PiBTdWJqZWN0OiBSRTogW1JGQ11bS1ZNXVtQQVRDSCAxLzFdIGt2bTpwcGM6Ym9va2Ut
NjQ6IHNvZnQtZGlzYWJsZQ0KPiA+PiBpbnRlcnJ1cHRzDQo+ID4+DQo+ID4+Pj4gVGhpcyBvbmx5
IGRpc2FibGUgc29mdCBpbnRlcnJ1cHQgZm9yIGt2bXBwY19yZXN0YXJ0X2ludGVycnVwdCgpDQo+
ID4+Pj4gdGhhdCByZXN0YXJ0cyBpbnRlcnJ1cHRzIGlmIHRoZXkgd2VyZSBtZWFudCBmb3IgdGhl
IGhvc3Q6DQo+ID4+Pj4NCj4gPj4+PiBhLiBTT0ZUX0RJU0FCTEVfSU5UUygpIG9ubHkgZm9yIEJP
T0tFX0lOVEVSUlVQVF9FWFRFUk5BTCB8DQo+ID4+Pj4gQk9PS0VfSU5URVJSVVBUX0RFQ1JFTUVO
VEVSIHwgQk9PS0VfSU5URVJSVVBUX0RPT1JCRUxMDQo+ID4+Pg0KPiA+Pj4gVGhvc2UgYXJlbid0
IHRoZSBvbmx5IGV4Y2VwdGlvbnMgdGhhdCBjYW4gZW5kIHVwIGdvaW5nIHRvIHRoZSBob3N0Lg0K
PiA+Pj4gV2UgY291bGQgZ2V0IGEgVExCIG1pc3MgdGhhdCByZXN1bHRzIGluIGEgaGVhdnl3ZWln
aHQgTU1JTyBleGl0LCBldGMuDQo+ID4+Pg0KPiA+Pj4+IEFuZCBzaG91bGRuJ3Qgd2UgaGFuZGxl
IGt2bXBwY19yZXN0YXJ0X2ludGVycnVwdCgpIGxpa2UgdGhlDQo+ID4+Pj4gb3JpZ2luYWwgSE9T
VCBmbG93Pw0KPiA+Pj4+DQo+ID4+Pj4gI2RlZmluZSBNQVNLQUJMRV9FWENFUFRJT04odHJhcG51
bSwgaW50bnVtLCBsYWJlbCwgaGRsciwNCj4gPj4+PiBhY2spICAgICAgICAgICBcDQo+ID4+Pj4N
Cj4gPj4+PiBTVEFSVF9FWENFUFRJT04obGFiZWwpOyAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgXA0KPiA+Pj4+ICAgICAgICAgIE5PUk1BTF9FWENFUFRJT05fUFJPTE9H
KHRyYXBudW0sIGludG51bSwNCj4gPj4+PiBQUk9MT0dfQURESVRJT05fTUFTS0FCTEUpXA0KPiA+
Pj4+ICAgICAgICAgIEVYQ0VQVElPTl9DT01NT04odHJhcG51bSwgUEFDQV9FWEdFTiwNCj4gPj4+
PiAqSU5UU19ESVNBQkxFKikgICAgICAgICAgICAgXA0KPiA+Pj4+IAkuLi4NCj4gPj4+DQo+ID4+
PiBDb3VsZCB5b3UgZWxhYm9yYXRlIG9uIHdoYXQgeW91IG1lYW4/DQo+ID4+DQo+ID4+IEkgdGhp
bmsgVGllanVuIHdhcyBzYXlpbmcgdGhhdCBob3N0IGhhcyBmbGFncyBhbmQgcmVwbGF5cyBvbmx5
DQo+ID4+IEVFL0RFQy9EQkVMTCBpbnRlcnJ1cHRzLiBUaGVyZSBpcyBzcGVjaWFsIG1hY3JvDQo+
ID4+IG1hc2tlZF9pbnRlcnJ1cHRfYm9vazNlIGluIHRob3NlIGV4Y2VwdGlvbiBoYW5kbGVycyB0
aGF0IHNldHMgcGFjYS0NCj4gPmlycV9oYXBwZW5lZC4NCj4gPj4NCj4gPj4gVGhlIGxpc3Qgb2Yg
cmVwbGllZCBpbnRlcnJ1cHRzIGlzIGxpbWl0ZWQgdG8gYXN5bmNocm9ub3VzIG5vbmNyaXRpY2Fs
DQo+ID4+IGludGVycnVwdHMgd2hpY2ggY2FuIGJlIG1hc2tlZCBieSBNU1JbRUVdICh0aGVyZWZv
cmUgbm8gVExCIG1pc3MpLg0KPiA+PiBOb3cgb24gS1ZNIGJvb2szZSB3ZSBkb24ndCB3YW50IHRv
IHB1dCB0aGVtIGluIHRoZSBpcnFfaGFwcGVuZWQgbGF6eQ0KPiA+PiBzdGF0ZSBidXQgcmF0aGVy
IHRvIGV4ZWN1dGUgdGhlbSBkaXJlY3RseSwgc28gdGhlcmUgaXMgbm8gcmVhc29uIGZvcg0KPiA+
PiBleGNlcHRpb24gaGFuZGxpbmcgc3ltbWV0cnkgYmV0d2VlbiBob3N0IGFuZCBndWVzdC4NCj4g
Pg0KPiA+DQo+ID4gQW5vdGhlciBRdWVzdGlvbjoNCj4gPg0KPiA+IFRoZSBjYXNlIGlzOg0KPiA+
DQo+IA0KPiBBY3R1YWxseSBpbiB0aGUgY2FzZSBHUz0xIGV2ZW4gaWYgRUU9MCwgRVhUL0RFQy9E
QkVMTCBzdGlsbCBvY2N1ciBhcyBJIHJlY2FsbC4NCj4gDQo+ID4gQ2FzZSAxKQ0KPiA+ICAgLT4g
TG9jYWxfaXJxX2Rpc2FibGUoKSAgd2lsbCBzZXQgc29mdF9lbmFibGVkID0gMA0KPiA+ICAgLT4g
Tm93IEV4dGVybmVsIGludGVycnVwdCBoYXBwZW5zLCB0aGVyZSB3ZSBzZXQgUEFDQV9JUlFfRUUg
aW4gaXJxX2hhcHBlbmVkLA0KPiBBbHNvIGNsZWFycyBFRSBpbiBTUlIxIGFuZCByZmkuIFNvIGlu
dGVycnVwdHMgYXJlIGhhcmQgZGlzYWJsZWQuIE5vIG1vcmUgb3RoZXINCj4gaW50ZXJydXB0IGdh
dGVkIGJ5IE1TUi5FRSBjYW4gaGFwcGVuLiBMb29rcyBsaWtlIHRoZSBpZGVhIGhlcmUgaXMgdG8g
bm90IGxldCBhDQo+IGRldmljZSBrZWVwIG9uIGluc2VydGluZyBpbnRlcnJ1cHQgdGlsbCB0aGUg
aW50ZXJydXB0IGNvbmRpdGlvbiBvbiBkZXZpY2UgaXMNCj4gY2xlYXJlZCwgcmlnaHQ/DQo+IA0K
PiBJIGRvbid0IHVuZGVyc3RhbmQgInRoZSBpbnRlcnJ1cHQgY29uZGl0aW9uIG9uIGRldmljZSBp
cyBjbGVhcmVkIiBoZXJlLg0KPiANCj4gSSB0aGluayByZWdhcmRsZXNzIGlmIHlvdSBjbGVhciB0
aGUgZGV2aWNlIGludGVycnVwdCBzdGF0dXMsIHRoZSBzeXN0ZW0gc3RpbGwNCj4gcmVjZWl2ZSBh
IHBlbmRpbmcgaW50ZXJydXB0IG9uY2UgRUUgb3IgR1MgPSAxLg0KDQpPbmNlIHllcywgYnV0IEkg
dGhpbmsgdG8gYXZvaWQgZmxvb2Qgb2YgZGV2aWNlIGludGVycnVwdCB3ZSBkaXNhYmxlIE1TUi5F
RSB3aGVuIHNvZnQtZGlzYWJsZWQuDQoNCj4gDQo+ID4gICAtPiBsb2NhbF9pcnFfZW5hYmxlKCkg
LSBUaGlzIGNoZWNrcyB0aGF0IGlycV9oYXBwZW5lZCBpcyBzZXQsIGFuZA0KPiA+IHJlcGxheXMN
Cj4gDQo+IHJldF9mcm9tX2V4Y2VwdCBhbHNvIGNoZWNrIHRvIHJlcGxheS4NCj4gDQo+ID4NCj4g
PiBOb3cgdGhlIGNhc2UgMikNCj4gPiBDYXNlIDIpDQo+ID4gLT4gTG9jYWxfaXJxX2Rpc2FibGUo
KSAgd2lsbCBzZXQgc29mdF9lbmFibGVkID0gMA0KPiA+ICAgLT4gTm93IERFQyBpbnRlcnJ1cHQg
aGFwcGVucy4gV2Ugc2V0IFBBQ0FfSVJRX0RFQyBpbiBpcnFfaGFwcGVuZWQsIEJ1dCBkbw0KPiBu
b3QgY2xlYXIgRUUgaW4gU1JSMSBhbmQgcmZpLiBTbyBpbnRlcnJ1cHRzIGFyZSBub3QgaGFyZCBk
aXNhYmxlZC4NCj4gPiAgIC0+IE5vdyBzYXkgRUUgaW50ZXJydXB0IGhhcHBlbnMsIHRoZXJlIHdl
IHNldCBQQUNBX0lSUV9FRSBpbiBpcnFfaGFwcGVuZWQsDQo+IEFsc28gY2xlYXJzIEVFIGluIFNS
UjEgYW5kIHJmaS4gU28gaW50ZXJydXB0cyBhcmUgaGFyZCBkaXNhYmxlZC4NCj4gPiAgIC0+IGxv
Y2FsX2lycV9lbmFibGUoKSAtIFRoaXMgY2hlY2tzIHRoYXQgaXJxX2hhcHBlbmVkIGlzIHNldC4N
Cj4gPiBJSVVDLCBpdCByZXBsYXlzIG9ubHkgb25lIGludGVycnVwdD8gaXMgbm90IGl0Pw0KPiAN
Cj4gQWZ0ZXIgYW55b25lIGlzIHJlcGxheWVkIGluIGFyY2hfbG9jYWxfaXJxX3Jlc3RvcmUoKSwg
d2Ugd2lsbCBzZXQgc29mdC9oYXJkDQo+IGludGVycnVwdCB0aGVyZToNCj4gDQo+IHNldF9zb2Z0
X2VuYWJsZWQoMSk7DQo+IF9faGFyZF9pcnFfZW5hYmxlKCk7DQo+IA0KPiBUaGVuIGFueSBwZW5k
aW5nIGludGVycnVwdCBjYW4gYmUgZXhlY3V0ZWQgbm93Lg0KDQpEbyB5b3UgbWVhbiB0aGF0IHRo
ZSBpbnRlcnJ1cHQgc2hvdWxkIGZpcmUgYWdhaW4/DQoNCj4gDQo+IEFkZGl0aW9uYWxseSwgcmV0
X2Zyb21fZXhjZXB0IHByb2JhYmx5IGNoZWNrIHRvIHJlcGxheSBhbGwuDQoNCkxvY2FsX2lycV9l
bmFibGUoKSB3aWxsIG5vdCB0YWtlIHVzIHRvIHJldF9mcm9tX2V4Y2VwdC4NCg0KLUJoYXJhdA0K
DQo+IA0KPiBUaWVqdW4NCg0K

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[tiejun.chen]

On 05/09/2013 06:00 PM, Bhushan Bharat-R65777 wrote:
>
>
>> -----Original Message-----
>> From: tiejun.chen [mailto:tiejun.chen@windriver.com]
>> Sent: Thursday, May 09, 2013 3:15 PM
>> To: Bhushan Bharat-R65777
>> Cc: Caraman Mihai Claudiu-B02008; Wood Scott-B07421; linuxppc-
>> dev@lists.ozlabs.org; agraf@suse.de; kvm-ppc@vger.kernel.org;
>> kvm@vger.kernel.org
>> Subject: Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts
>>
>> On 05/09/2013 04:23 PM, Bhushan Bharat-R65777 wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Linuxppc-dev [mailto:linuxppc-dev-
>>>> bounces+bharat.bhushan=freescale.com@lists.ozlabs.org] On Behalf Of
>>>> bounces+Caraman
>>>> Mihai Claudiu-B02008
>>>> Sent: Wednesday, May 08, 2013 6:44 PM
>>>> To: Wood Scott-B07421; tiejun.chen
>>>> Cc: linuxppc-dev@lists.ozlabs.org; agraf@suse.de;
>>>> kvm-ppc@vger.kernel.org; kvm@vger.kernel.org
>>>> Subject: RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable
>>>> interrupts
>>>>
>>>>>> This only disable soft interrupt for kvmppc_restart_interrupt()
>>>>>> that restarts interrupts if they were meant for the host:
>>>>>>
>>>>>> a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
>>>>>> BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
>>>>>
>>>>> Those aren't the only exceptions that can end up going to the host.
>>>>> We could get a TLB miss that results in a heavyweight MMIO exit, etc.
>>>>>
>>>>>> And shouldn't we handle kvmppc_restart_interrupt() like the
>>>>>> original HOST flow?
>>>>>>
>>>>>> #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr,
>>>>>> ack)           \
>>>>>>
>>>>>> START_EXCEPTION(label);                                         \
>>>>>>           NORMAL_EXCEPTION_PROLOG(trapnum, intnum,
>>>>>> PROLOG_ADDITION_MASKABLE)\
>>>>>>           EXCEPTION_COMMON(trapnum, PACA_EXGEN,
>>>>>> *INTS_DISABLE*)             \
>>>>>> 	...
>>>>>
>>>>> Could you elaborate on what you mean?
>>>>
>>>> I think Tiejun was saying that host has flags and replays only
>>>> EE/DEC/DBELL interrupts. There is special macro
>>>> masked_interrupt_book3e in those exception handlers that sets paca-
>>> irq_happened.
>>>>
>>>> The list of replied interrupts is limited to asynchronous noncritical
>>>> interrupts which can be masked by MSR[EE] (therefore no TLB miss).
>>>> Now on KVM book3e we don't want to put them in the irq_happened lazy
>>>> state but rather to execute them directly, so there is no reason for
>>>> exception handling symmetry between host and guest.
>>>
>>>
>>> Another Question:
>>>
>>> The case is:
>>>
>>
>> Actually in the case GS=1 even if EE=0, EXT/DEC/DBELL still occur as I recall.
>>
>>> Case 1)
>>>    -> Local_irq_disable()  will set soft_enabled = 0
>>>    -> Now Externel interrupt happens, there we set PACA_IRQ_EE in irq_happened,
>> Also clears EE in SRR1 and rfi. So interrupts are hard disabled. No more other
>> interrupt gated by MSR.EE can happen. Looks like the idea here is to not let a
>> device keep on inserting interrupt till the interrupt condition on device is
>> cleared, right?
>>
>> I don't understand "the interrupt condition on device is cleared" here.
>>
>> I think regardless if you clear the device interrupt status, the system still
>> receive a pending interrupt once EE or GS = 1.
>
> Once yes, but I think to avoid flood of device interrupt we disable MSR.EE when soft-disabled.

But we neither ACK nor send EOI to that irq in the interrupt controller, so that 
should be in pending state.

>
>>
>>>    -> local_irq_enable() - This checks that irq_happened is set, and
>>> replays
>>
>> ret_from_except also check to replay.
>>
>>>
>>> Now the case 2)
>>> Case 2)
>>> -> Local_irq_disable()  will set soft_enabled = 0
>>>    -> Now DEC interrupt happens. We set PACA_IRQ_DEC in irq_happened, But do
>> not clear EE in SRR1 and rfi. So interrupts are not hard disabled.
>>>    -> Now say EE interrupt happens, there we set PACA_IRQ_EE in irq_happened,
>> Also clears EE in SRR1 and rfi. So interrupts are hard disabled.
>>>    -> local_irq_enable() - This checks that irq_happened is set.
>>> IIUC, it replays only one interrupt? is not it?
>>
>> After anyone is replayed in arch_local_irq_restore(), we will set soft/hard
>> interrupt there:
>>
>> set_soft_enabled(1);
>> __hard_irq_enable();
>>
>> Then any pending interrupt can be executed now.
>
> Do you mean that the interrupt should fire again?

I means the pending exception including external interrupt, the decrementer 
exception and the doorbell exception, can trap CPU once EE=1 with 
__hard_irq_enable() here. Then the kernel can handle those exception since soft 
enable is also 1 now.

>
>>
>> Additionally, ret_from_except probably check to replay all.
>
> Local_irq_enable() will not take us to ret_from_except.

Yes. I just say ret_from_except can provide an approach to replay all :)

Tiejun

RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Bhushan Bharat-R65777]

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogdGllanVuLmNoZW4gW21h
aWx0bzp0aWVqdW4uY2hlbkB3aW5kcml2ZXIuY29tXQ0KPiBTZW50OiBUaHVyc2RheSwgTWF5IDA5
LCAyMDEzIDM6NDggUE0NCj4gVG86IEJodXNoYW4gQmhhcmF0LVI2NTc3Nw0KPiBDYzogQ2FyYW1h
biBNaWhhaSBDbGF1ZGl1LUIwMjAwODsgV29vZCBTY290dC1CMDc0MjE7IGxpbnV4cHBjLQ0KPiBk
ZXZAbGlzdHMub3psYWJzLm9yZzsgYWdyYWZAc3VzZS5kZTsga3ZtLXBwY0B2Z2VyLmtlcm5lbC5v
cmc7DQo+IGt2bUB2Z2VyLmtlcm5lbC5vcmcNCj4gU3ViamVjdDogUmU6IFtSRkNdW0tWTV1bUEFU
Q0ggMS8xXSBrdm06cHBjOmJvb2tlLTY0OiBzb2Z0LWRpc2FibGUgaW50ZXJydXB0cw0KPiANCj4g
T24gMDUvMDkvMjAxMyAwNjowMCBQTSwgQmh1c2hhbiBCaGFyYXQtUjY1Nzc3IHdyb3RlOg0KPiA+
DQo+ID4NCj4gPj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPj4gRnJvbTogdGllanVu
LmNoZW4gW21haWx0bzp0aWVqdW4uY2hlbkB3aW5kcml2ZXIuY29tXQ0KPiA+PiBTZW50OiBUaHVy
c2RheSwgTWF5IDA5LCAyMDEzIDM6MTUgUE0NCj4gPj4gVG86IEJodXNoYW4gQmhhcmF0LVI2NTc3
Nw0KPiA+PiBDYzogQ2FyYW1hbiBNaWhhaSBDbGF1ZGl1LUIwMjAwODsgV29vZCBTY290dC1CMDc0
MjE7IGxpbnV4cHBjLQ0KPiA+PiBkZXZAbGlzdHMub3psYWJzLm9yZzsgYWdyYWZAc3VzZS5kZTsg
a3ZtLXBwY0B2Z2VyLmtlcm5lbC5vcmc7DQo+ID4+IGt2bUB2Z2VyLmtlcm5lbC5vcmcNCj4gPj4g
U3ViamVjdDogUmU6IFtSRkNdW0tWTV1bUEFUQ0ggMS8xXSBrdm06cHBjOmJvb2tlLTY0OiBzb2Z0
LWRpc2FibGUNCj4gPj4gaW50ZXJydXB0cw0KPiA+Pg0KPiA+PiBPbiAwNS8wOS8yMDEzIDA0OjIz
IFBNLCBCaHVzaGFuIEJoYXJhdC1SNjU3Nzcgd3JvdGU6DQo+ID4+Pg0KPiA+Pj4NCj4gPj4+PiAt
LS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+Pj4+IEZyb206IExpbnV4cHBjLWRldiBbbWFp
bHRvOmxpbnV4cHBjLWRldi0NCj4gPj4+PiBib3VuY2VzK2JoYXJhdC5iaHVzaGFuPWZyZWVzY2Fs
ZS5jb21AbGlzdHMub3psYWJzLm9yZ10gT24gQmVoYWxmIE9mDQo+ID4+Pj4gYm91bmNlcytDYXJh
bWFuDQo+ID4+Pj4gTWloYWkgQ2xhdWRpdS1CMDIwMDgNCj4gPj4+PiBTZW50OiBXZWRuZXNkYXks
IE1heSAwOCwgMjAxMyA2OjQ0IFBNDQo+ID4+Pj4gVG86IFdvb2QgU2NvdHQtQjA3NDIxOyB0aWVq
dW4uY2hlbg0KPiA+Pj4+IENjOiBsaW51eHBwYy1kZXZAbGlzdHMub3psYWJzLm9yZzsgYWdyYWZA
c3VzZS5kZTsNCj4gPj4+PiBrdm0tcHBjQHZnZXIua2VybmVsLm9yZzsga3ZtQHZnZXIua2VybmVs
Lm9yZw0KPiA+Pj4+IFN1YmplY3Q6IFJFOiBbUkZDXVtLVk1dW1BBVENIIDEvMV0ga3ZtOnBwYzpi
b29rZS02NDogc29mdC1kaXNhYmxlDQo+ID4+Pj4gaW50ZXJydXB0cw0KPiA+Pj4+DQo+ID4+Pj4+
PiBUaGlzIG9ubHkgZGlzYWJsZSBzb2Z0IGludGVycnVwdCBmb3Iga3ZtcHBjX3Jlc3RhcnRfaW50
ZXJydXB0KCkNCj4gPj4+Pj4+IHRoYXQgcmVzdGFydHMgaW50ZXJydXB0cyBpZiB0aGV5IHdlcmUg
bWVhbnQgZm9yIHRoZSBob3N0Og0KPiA+Pj4+Pj4NCj4gPj4+Pj4+IGEuIFNPRlRfRElTQUJMRV9J
TlRTKCkgb25seSBmb3IgQk9PS0VfSU5URVJSVVBUX0VYVEVSTkFMIHwNCj4gPj4+Pj4+IEJPT0tF
X0lOVEVSUlVQVF9ERUNSRU1FTlRFUiB8IEJPT0tFX0lOVEVSUlVQVF9ET09SQkVMTA0KPiA+Pj4+
Pg0KPiA+Pj4+PiBUaG9zZSBhcmVuJ3QgdGhlIG9ubHkgZXhjZXB0aW9ucyB0aGF0IGNhbiBlbmQg
dXAgZ29pbmcgdG8gdGhlIGhvc3QuDQo+ID4+Pj4+IFdlIGNvdWxkIGdldCBhIFRMQiBtaXNzIHRo
YXQgcmVzdWx0cyBpbiBhIGhlYXZ5d2VpZ2h0IE1NSU8gZXhpdCwgZXRjLg0KPiA+Pj4+Pg0KPiA+
Pj4+Pj4gQW5kIHNob3VsZG4ndCB3ZSBoYW5kbGUga3ZtcHBjX3Jlc3RhcnRfaW50ZXJydXB0KCkg
bGlrZSB0aGUNCj4gPj4+Pj4+IG9yaWdpbmFsIEhPU1QgZmxvdz8NCj4gPj4+Pj4+DQo+ID4+Pj4+
PiAjZGVmaW5lIE1BU0tBQkxFX0VYQ0VQVElPTih0cmFwbnVtLCBpbnRudW0sIGxhYmVsLCBoZGxy
LA0KPiA+Pj4+Pj4gYWNrKSAgICAgICAgICAgXA0KPiA+Pj4+Pj4NCj4gPj4+Pj4+IFNUQVJUX0VY
Q0VQVElPTihsYWJlbCk7ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBc
DQo+ID4+Pj4+PiAgICAgICAgICAgTk9STUFMX0VYQ0VQVElPTl9QUk9MT0codHJhcG51bSwgaW50
bnVtLA0KPiA+Pj4+Pj4gUFJPTE9HX0FERElUSU9OX01BU0tBQkxFKVwNCj4gPj4+Pj4+ICAgICAg
ICAgICBFWENFUFRJT05fQ09NTU9OKHRyYXBudW0sIFBBQ0FfRVhHRU4sDQo+ID4+Pj4+PiAqSU5U
U19ESVNBQkxFKikgICAgICAgICAgICAgXA0KPiA+Pj4+Pj4gCS4uLg0KPiA+Pj4+Pg0KPiA+Pj4+
PiBDb3VsZCB5b3UgZWxhYm9yYXRlIG9uIHdoYXQgeW91IG1lYW4/DQo+ID4+Pj4NCj4gPj4+PiBJ
IHRoaW5rIFRpZWp1biB3YXMgc2F5aW5nIHRoYXQgaG9zdCBoYXMgZmxhZ3MgYW5kIHJlcGxheXMg
b25seQ0KPiA+Pj4+IEVFL0RFQy9EQkVMTCBpbnRlcnJ1cHRzLiBUaGVyZSBpcyBzcGVjaWFsIG1h
Y3JvDQo+ID4+Pj4gbWFza2VkX2ludGVycnVwdF9ib29rM2UgaW4gdGhvc2UgZXhjZXB0aW9uIGhh
bmRsZXJzIHRoYXQgc2V0cyBwYWNhLQ0KPiA+Pj4gaXJxX2hhcHBlbmVkLg0KPiA+Pj4+DQo+ID4+
Pj4gVGhlIGxpc3Qgb2YgcmVwbGllZCBpbnRlcnJ1cHRzIGlzIGxpbWl0ZWQgdG8gYXN5bmNocm9u
b3VzDQo+ID4+Pj4gbm9uY3JpdGljYWwgaW50ZXJydXB0cyB3aGljaCBjYW4gYmUgbWFza2VkIGJ5
IE1TUltFRV0gKHRoZXJlZm9yZSBubyBUTEINCj4gbWlzcykuDQo+ID4+Pj4gTm93IG9uIEtWTSBi
b29rM2Ugd2UgZG9uJ3Qgd2FudCB0byBwdXQgdGhlbSBpbiB0aGUgaXJxX2hhcHBlbmVkDQo+ID4+
Pj4gbGF6eSBzdGF0ZSBidXQgcmF0aGVyIHRvIGV4ZWN1dGUgdGhlbSBkaXJlY3RseSwgc28gdGhl
cmUgaXMgbm8NCj4gPj4+PiByZWFzb24gZm9yIGV4Y2VwdGlvbiBoYW5kbGluZyBzeW1tZXRyeSBi
ZXR3ZWVuIGhvc3QgYW5kIGd1ZXN0Lg0KPiA+Pj4NCj4gPj4+DQo+ID4+PiBBbm90aGVyIFF1ZXN0
aW9uOg0KPiA+Pj4NCj4gPj4+IFRoZSBjYXNlIGlzOg0KPiA+Pj4NCj4gPj4NCj4gPj4gQWN0dWFs
bHkgaW4gdGhlIGNhc2UgR1M9MSBldmVuIGlmIEVFPTAsIEVYVC9ERUMvREJFTEwgc3RpbGwgb2Nj
dXIgYXMgSQ0KPiByZWNhbGwuDQo+ID4+DQo+ID4+PiBDYXNlIDEpDQo+ID4+PiAgICAtPiBMb2Nh
bF9pcnFfZGlzYWJsZSgpICB3aWxsIHNldCBzb2Z0X2VuYWJsZWQgPSAwDQo+ID4+PiAgICAtPiBO
b3cgRXh0ZXJuZWwgaW50ZXJydXB0IGhhcHBlbnMsIHRoZXJlIHdlIHNldCBQQUNBX0lSUV9FRSBp
bg0KPiA+Pj4gaXJxX2hhcHBlbmVkLA0KPiA+PiBBbHNvIGNsZWFycyBFRSBpbiBTUlIxIGFuZCBy
ZmkuIFNvIGludGVycnVwdHMgYXJlIGhhcmQgZGlzYWJsZWQuIE5vDQo+ID4+IG1vcmUgb3RoZXIg
aW50ZXJydXB0IGdhdGVkIGJ5IE1TUi5FRSBjYW4gaGFwcGVuLiBMb29rcyBsaWtlIHRoZSBpZGVh
DQo+ID4+IGhlcmUgaXMgdG8gbm90IGxldCBhIGRldmljZSBrZWVwIG9uIGluc2VydGluZyBpbnRl
cnJ1cHQgdGlsbCB0aGUNCj4gPj4gaW50ZXJydXB0IGNvbmRpdGlvbiBvbiBkZXZpY2UgaXMgY2xl
YXJlZCwgcmlnaHQ/DQo+ID4+DQo+ID4+IEkgZG9uJ3QgdW5kZXJzdGFuZCAidGhlIGludGVycnVw
dCBjb25kaXRpb24gb24gZGV2aWNlIGlzIGNsZWFyZWQiIGhlcmUuDQo+ID4+DQo+ID4+IEkgdGhp
bmsgcmVnYXJkbGVzcyBpZiB5b3UgY2xlYXIgdGhlIGRldmljZSBpbnRlcnJ1cHQgc3RhdHVzLCB0
aGUNCj4gPj4gc3lzdGVtIHN0aWxsIHJlY2VpdmUgYSBwZW5kaW5nIGludGVycnVwdCBvbmNlIEVF
IG9yIEdTID0gMS4NCj4gPg0KPiA+IE9uY2UgeWVzLCBidXQgSSB0aGluayB0byBhdm9pZCBmbG9v
ZCBvZiBkZXZpY2UgaW50ZXJydXB0IHdlIGRpc2FibGUgTVNSLkVFDQo+IHdoZW4gc29mdC1kaXNh
YmxlZC4NCj4gDQo+IEJ1dCB3ZSBuZWl0aGVyIEFDSyBub3Igc2VuZCBFT0kgdG8gdGhhdCBpcnEg
aW4gdGhlIGludGVycnVwdCBjb250cm9sbGVyLCBzbyB0aGF0DQo+IHNob3VsZCBiZSBpbiBwZW5k
aW5nIHN0YXRlLg0KPiANCj4gPg0KPiA+Pg0KPiA+Pj4gICAgLT4gbG9jYWxfaXJxX2VuYWJsZSgp
IC0gVGhpcyBjaGVja3MgdGhhdCBpcnFfaGFwcGVuZWQgaXMgc2V0LCBhbmQNCj4gPj4+IHJlcGxh
eXMNCj4gPj4NCj4gPj4gcmV0X2Zyb21fZXhjZXB0IGFsc28gY2hlY2sgdG8gcmVwbGF5Lg0KPiA+
Pg0KPiA+Pj4NCj4gPj4+IE5vdyB0aGUgY2FzZSAyKQ0KPiA+Pj4gQ2FzZSAyKQ0KPiA+Pj4gLT4g
TG9jYWxfaXJxX2Rpc2FibGUoKSAgd2lsbCBzZXQgc29mdF9lbmFibGVkID0gMA0KPiA+Pj4gICAg
LT4gTm93IERFQyBpbnRlcnJ1cHQgaGFwcGVucy4gV2Ugc2V0IFBBQ0FfSVJRX0RFQyBpbg0KPiA+
Pj4gaXJxX2hhcHBlbmVkLCBCdXQgZG8NCj4gPj4gbm90IGNsZWFyIEVFIGluIFNSUjEgYW5kIHJm
aS4gU28gaW50ZXJydXB0cyBhcmUgbm90IGhhcmQgZGlzYWJsZWQuDQo+ID4+PiAgICAtPiBOb3cg
c2F5IEVFIGludGVycnVwdCBoYXBwZW5zLCB0aGVyZSB3ZSBzZXQgUEFDQV9JUlFfRUUgaW4NCj4g
Pj4+IGlycV9oYXBwZW5lZCwNCj4gPj4gQWxzbyBjbGVhcnMgRUUgaW4gU1JSMSBhbmQgcmZpLiBT
byBpbnRlcnJ1cHRzIGFyZSBoYXJkIGRpc2FibGVkLg0KPiA+Pj4gICAgLT4gbG9jYWxfaXJxX2Vu
YWJsZSgpIC0gVGhpcyBjaGVja3MgdGhhdCBpcnFfaGFwcGVuZWQgaXMgc2V0Lg0KPiA+Pj4gSUlV
QywgaXQgcmVwbGF5cyBvbmx5IG9uZSBpbnRlcnJ1cHQ/IGlzIG5vdCBpdD8NCj4gPj4NCj4gPj4g
QWZ0ZXIgYW55b25lIGlzIHJlcGxheWVkIGluIGFyY2hfbG9jYWxfaXJxX3Jlc3RvcmUoKSwgd2Ug
d2lsbCBzZXQNCj4gPj4gc29mdC9oYXJkIGludGVycnVwdCB0aGVyZToNCj4gPj4NCj4gPj4gc2V0
X3NvZnRfZW5hYmxlZCgxKTsNCj4gPj4gX19oYXJkX2lycV9lbmFibGUoKTsNCj4gPj4NCj4gPj4g
VGhlbiBhbnkgcGVuZGluZyBpbnRlcnJ1cHQgY2FuIGJlIGV4ZWN1dGVkIG5vdy4NCj4gPg0KPiA+
IERvIHlvdSBtZWFuIHRoYXQgdGhlIGludGVycnVwdCBzaG91bGQgZmlyZSBhZ2Fpbj8NCj4gDQo+
IEkgbWVhbnMgdGhlIHBlbmRpbmcgZXhjZXB0aW9uIGluY2x1ZGluZyBleHRlcm5hbCBpbnRlcnJ1
cHQsIHRoZSBkZWNyZW1lbnRlcg0KPiBleGNlcHRpb24gYW5kIHRoZSBkb29yYmVsbCBleGNlcHRp
b24sIGNhbiB0cmFwIENQVSBvbmNlIEVFPTEgd2l0aA0KPiBfX2hhcmRfaXJxX2VuYWJsZSgpIGhl
cmUuIFRoZW4gdGhlIGtlcm5lbCBjYW4gaGFuZGxlIHRob3NlIGV4Y2VwdGlvbiBzaW5jZSBzb2Z0
DQo+IGVuYWJsZSBpcyBhbHNvIDEgbm93Lg0KPiANCj4gPg0KPiA+Pg0KPiA+PiBBZGRpdGlvbmFs
bHksIHJldF9mcm9tX2V4Y2VwdCBwcm9iYWJseSBjaGVjayB0byByZXBsYXkgYWxsLg0KPiA+DQo+
ID4gTG9jYWxfaXJxX2VuYWJsZSgpIHdpbGwgbm90IHRha2UgdXMgdG8gcmV0X2Zyb21fZXhjZXB0
Lg0KPiANCj4gWWVzLiBJIGp1c3Qgc2F5IHJldF9mcm9tX2V4Y2VwdCBjYW4gcHJvdmlkZSBhbiBh
cHByb2FjaCB0byByZXBsYXkgYWxsIDopDQoNCl9fcmVwbGF5X2ludGVycnVwdCgpIGZyb20gYXJj
aF9sb2NhbF9pcnFfZW5hYmxlKCkgd2lsbCB0YWtlIHVzIHRvIHJldF9mcm9tX2V4Y2VwdC9saXRl
IDopDQpUaGVyZSBhbGwgcGVuZGluZyBpbnRlcnJ1cHRzIGFyZSByZXBsYXllZCBvbmUgYnkgb25l
IGJlZm9yZSB3ZSBoYXJkLWVuYWJsZSBhbmQgc29mdC1lbmFibGUgaW50ZXJydXB0cy4NCg0KLUJo
YXJhdA0KDQo+IA0KPiBUaWVqdW4NCg0K

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[tiejun.chen]

On 05/09/2013 07:21 PM, Bhushan Bharat-R65777 wrote:
>
>
>> -----Original Message-----
>> From: tiejun.chen [mailto:tiejun.chen@windriver.com]
>> Sent: Thursday, May 09, 2013 3:48 PM
>> To: Bhushan Bharat-R65777
>> Cc: Caraman Mihai Claudiu-B02008; Wood Scott-B07421; linuxppc-
>> dev@lists.ozlabs.org; agraf@suse.de; kvm-ppc@vger.kernel.org;
>> kvm@vger.kernel.org
>> Subject: Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts
>>
>> On 05/09/2013 06:00 PM, Bhushan Bharat-R65777 wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: tiejun.chen [mailto:tiejun.chen@windriver.com]
>>>> Sent: Thursday, May 09, 2013 3:15 PM
>>>> To: Bhushan Bharat-R65777
>>>> Cc: Caraman Mihai Claudiu-B02008; Wood Scott-B07421; linuxppc-
>>>> dev@lists.ozlabs.org; agraf@suse.de; kvm-ppc@vger.kernel.org;
>>>> kvm@vger.kernel.org
>>>> Subject: Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable
>>>> interrupts
>>>>
>>>> On 05/09/2013 04:23 PM, Bhushan Bharat-R65777 wrote:
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Linuxppc-dev [mailto:linuxppc-dev-
>>>>>> bounces+bharat.bhushan=freescale.com@lists.ozlabs.org] On Behalf Of
>>>>>> bounces+Caraman
>>>>>> Mihai Claudiu-B02008
>>>>>> Sent: Wednesday, May 08, 2013 6:44 PM
>>>>>> To: Wood Scott-B07421; tiejun.chen
>>>>>> Cc: linuxppc-dev@lists.ozlabs.org; agraf@suse.de;
>>>>>> kvm-ppc@vger.kernel.org; kvm@vger.kernel.org
>>>>>> Subject: RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable
>>>>>> interrupts
>>>>>>
>>>>>>>> This only disable soft interrupt for kvmppc_restart_interrupt()
>>>>>>>> that restarts interrupts if they were meant for the host:
>>>>>>>>
>>>>>>>> a. SOFT_DISABLE_INTS() only for BOOKE_INTERRUPT_EXTERNAL |
>>>>>>>> BOOKE_INTERRUPT_DECREMENTER | BOOKE_INTERRUPT_DOORBELL
>>>>>>>
>>>>>>> Those aren't the only exceptions that can end up going to the host.
>>>>>>> We could get a TLB miss that results in a heavyweight MMIO exit, etc.
>>>>>>>
>>>>>>>> And shouldn't we handle kvmppc_restart_interrupt() like the
>>>>>>>> original HOST flow?
>>>>>>>>
>>>>>>>> #define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr,
>>>>>>>> ack)           \
>>>>>>>>
>>>>>>>> START_EXCEPTION(label);                                         \
>>>>>>>>            NORMAL_EXCEPTION_PROLOG(trapnum, intnum,
>>>>>>>> PROLOG_ADDITION_MASKABLE)\
>>>>>>>>            EXCEPTION_COMMON(trapnum, PACA_EXGEN,
>>>>>>>> *INTS_DISABLE*)             \
>>>>>>>> 	...
>>>>>>>
>>>>>>> Could you elaborate on what you mean?
>>>>>>
>>>>>> I think Tiejun was saying that host has flags and replays only
>>>>>> EE/DEC/DBELL interrupts. There is special macro
>>>>>> masked_interrupt_book3e in those exception handlers that sets paca-
>>>>> irq_happened.
>>>>>>
>>>>>> The list of replied interrupts is limited to asynchronous
>>>>>> noncritical interrupts which can be masked by MSR[EE] (therefore no TLB
>> miss).
>>>>>> Now on KVM book3e we don't want to put them in the irq_happened
>>>>>> lazy state but rather to execute them directly, so there is no
>>>>>> reason for exception handling symmetry between host and guest.
>>>>>
>>>>>
>>>>> Another Question:
>>>>>
>>>>> The case is:
>>>>>
>>>>
>>>> Actually in the case GS=1 even if EE=0, EXT/DEC/DBELL still occur as I
>> recall.
>>>>
>>>>> Case 1)
>>>>>     -> Local_irq_disable()  will set soft_enabled = 0
>>>>>     -> Now Externel interrupt happens, there we set PACA_IRQ_EE in
>>>>> irq_happened,
>>>> Also clears EE in SRR1 and rfi. So interrupts are hard disabled. No
>>>> more other interrupt gated by MSR.EE can happen. Looks like the idea
>>>> here is to not let a device keep on inserting interrupt till the
>>>> interrupt condition on device is cleared, right?
>>>>
>>>> I don't understand "the interrupt condition on device is cleared" here.
>>>>
>>>> I think regardless if you clear the device interrupt status, the
>>>> system still receive a pending interrupt once EE or GS = 1.
>>>
>>> Once yes, but I think to avoid flood of device interrupt we disable MSR.EE
>> when soft-disabled.
>>
>> But we neither ACK nor send EOI to that irq in the interrupt controller, so that
>> should be in pending state.
>>
>>>
>>>>
>>>>>     -> local_irq_enable() - This checks that irq_happened is set, and
>>>>> replays
>>>>
>>>> ret_from_except also check to replay.
>>>>
>>>>>
>>>>> Now the case 2)
>>>>> Case 2)
>>>>> -> Local_irq_disable()  will set soft_enabled = 0
>>>>>     -> Now DEC interrupt happens. We set PACA_IRQ_DEC in
>>>>> irq_happened, But do
>>>> not clear EE in SRR1 and rfi. So interrupts are not hard disabled.
>>>>>     -> Now say EE interrupt happens, there we set PACA_IRQ_EE in
>>>>> irq_happened,
>>>> Also clears EE in SRR1 and rfi. So interrupts are hard disabled.
>>>>>     -> local_irq_enable() - This checks that irq_happened is set.
>>>>> IIUC, it replays only one interrupt? is not it?
>>>>
>>>> After anyone is replayed in arch_local_irq_restore(), we will set
>>>> soft/hard interrupt there:
>>>>
>>>> set_soft_enabled(1);
>>>> __hard_irq_enable();
>>>>
>>>> Then any pending interrupt can be executed now.
>>>
>>> Do you mean that the interrupt should fire again?
>>
>> I means the pending exception including external interrupt, the decrementer
>> exception and the doorbell exception, can trap CPU once EE=1 with
>> __hard_irq_enable() here. Then the kernel can handle those exception since soft
>> enable is also 1 now.
>>
>>>
>>>>
>>>> Additionally, ret_from_except probably check to replay all.
>>>
>>> Local_irq_enable() will not take us to ret_from_except.
>>
>> Yes. I just say ret_from_except can provide an approach to replay all :)
>
> __replay_interrupt() from arch_local_irq_enable() will take us to ret_from_except/lite :)
> There all pending interrupts are replayed one by one before we hard-enable and soft-enable interrupts.

Yes, but a point needs to be corrected,

_replay_interrupt() is following set_soft_enabled(1), so __replay_interrupt() 
can go the exception entry to call the handler.

Tiejun

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Benjamin Herrenschmidt]

On Thu, 2013-05-09 at 17:44 +0800, tiejun.chen wrote:
> 
> Actually in the case GS=1 even if EE=0, EXT/DEC/DBELL still occur as I
> recall.

Only if directed to the hypervisor.

> > Case 1)
> >   -> Local_irq_disable()  will set soft_enabled = 0
> >   -> Now Externel interrupt happens, there we set PACA_IRQ_EE in
> irq_happened, Also clears EE in SRR1 and rfi. So interrupts are hard
> disabled. No more other interrupt gated by MSR.EE can happen. Looks
> like the idea here is to not let a device keep on inserting interrupt
> till the interrupt condition on device is cleared, right?

The external interrupt line is level sensitive normally, so we have to
mask MSR:EE in that case or the interrupt would keep re-occurring (note
that FSL has this concept of auto-acked interrupts via the on die MPIC
for which you can potentially use PACA_IRQ_EE_EDGE instead and avoid
having to mask MSR:EE).

> I don't understand "the interrupt condition on device is cleared"
> here.

Well, the external interrupt line will remain asserted until the device
(via the PIC) stops interrupting the processor :-) Either because we
mask at the PIC level (or raise the priority) or because the condition
goes away.
 
> I think regardless if you clear the device interrupt status, the
> system still  receive a pending interrupt once EE or GS = 1.

Why ? Depends on your PIC actually but if it's a sane one, it won't
"remember" a level interrupt that is gone. An edge interrupt is a
different matter and is latched.

Some MPIC implementations tend to generate a spurrious IRQ in the case
of level IRQs going away. IE. they still remember an event occurred and
interrupt the processor, but on IACK return the spurious vector. However
that isn't guaranteed to be the case and it is perfectly ok (and a good
idea) for the PIC to just stop asserting the external interrupt line if
the original device interrupt goes away (and it's configured as level
sensitive).
 
You might notice that we try to re-hard-enable (while still soft
disabled) as soon as we have gone get_irq in do_IRQ. This is because
fetching the interrupt normally also masks it (either by masking at the
source or by raisin the processor interrupt priority depending on the
specific PIC) so we know we can re-hard enable. 

> >   -> local_irq_enable() - This checks that irq_happened is set, and
> replays
> 
> ret_from_except also check to replay.

ret_from_except checks because it essentially can act as an implicit
local_irq_enable.

> > Now the case 2)
> > Case 2)
> > -> Local_irq_disable()  will set soft_enabled = 0
> >   -> Now DEC interrupt happens. We set PACA_IRQ_DEC in irq_happened,
> But do not clear EE in SRR1 and rfi. So interrupts are not hard
> disabled.

Right. We move the decrementer to its maximum value however since on
most implementations, it will continue interrupting the processor while
it's top bit is set (and effectively act as a level sensitive
interrupt).

Again, the goal here is to run as much as possible with interrupts hard
enabled which allow better perf sampling.

> >   -> Now say EE interrupt happens, there we set PACA_IRQ_EE in
> irq_happened, Also clears EE in SRR1 and rfi. So interrupts are hard
> disabled.
> >   -> local_irq_enable() - This checks that irq_happened is set.
> > IIUC, it replays only one interrupt? is not it?

It replays one interrupt, but interrupt are still disabled during the
replay. Upon return from the replayed interrupt, the ret_from_except
will see the other one and replay it too.

> After anyone is replayed in arch_local_irq_restore(), we will set
> soft/hard 
> interrupt there:
> 
> set_soft_enabled(1);
> __hard_irq_enable();
> 
> Then any pending interrupt can be executed now.
> 
> Additionally, ret_from_except probably check to replay all.
> 
> Tiejun
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[David Laight]

> Some MPIC implementations tend to generate a spurrious IRQ in the case
> of level IRQs going away. IE. they still remember an event occurred =
and
> interrupt the processor, but on IACK return the spurious vector. =
However
> that isn't guaranteed to be the case and it is perfectly ok (and a =
good
> idea) for the PIC to just stop asserting the external interrupt line =
if
> the original device interrupt goes away (and it's configured as level
> sensitive).

That will happen if the IRQ goes away while the cpu is performing
the IACK sequence.
If the IRQ goes away while the cpu has interrupts masked then
the cpu won't start the interrupt sequence and then try to
read a vector when no interrupt is pending.

	David

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Benjamin Herrenschmidt]

On Thu, 2013-05-09 at 14:28 +0100, David Laight wrote:
> That will happen if the IRQ goes away while the cpu is performing
> the IACK sequence.
> If the IRQ goes away while the cpu has interrupts masked then
> the cpu won't start the interrupt sequence and then try to
> read a vector when no interrupt is pending.

Right, and get a spurrious vector which shouldn't be a big deal.

We tend to call that a "short interrupt". There have been many other
cases of short interrupts in the past, for example on some MPICs, when
distribution is enabled, they occasionally shoot an interrupt to more
than one CPU at once :-)

Cheers,
Ben.

RE: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Chen, Tiejun]

> -----Original Message-----
> From: Benjamin Herrenschmidt [mailto:benh@kernel.crashing.org]=20
> Sent: Thursday, May 09, 2013 8:38 PM
> To: Chen, Tiejun
> Cc: Bhushan Bharat-R65777; Caraman Mihai Claudiu-B02008; Wood=20
> Scott-B07421; linuxppc-dev@lists.ozlabs.org; agraf@suse.de;=20
> kvm-ppc@vger.kernel.org; kvm@vger.kernel.org
> Subject: Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64:=20
> soft-disable interrupts
>=20
> On Thu, 2013-05-09 at 17:44 +0800, tiejun.chen wrote:
> >=20
> > Actually in the case GS=3D1 even if EE=3D0, EXT/DEC/DBELL still=20
> occur as I=20
> > recall.
>=20
> Only if directed to the hypervisor.

Yes, this is our current booehv design with EPCR[EXTGS] =3D 0.

>=20
> > > Case 1)
> > >   -> Local_irq_disable()  will set soft_enabled =3D 0
> > >   -> Now Externel interrupt happens, there we set PACA_IRQ_EE in
> > irq_happened, Also clears EE in SRR1 and rfi. So interrupts=20
> are hard=20
> > disabled. No more other interrupt gated by MSR.EE can happen. Looks=20
> > like the idea here is to not let a device keep on inserting=20
> interrupt=20
> > till the interrupt condition on device is cleared, right?
>=20
> The external interrupt line is level sensitive normally, so=20
> we have to mask MSR:EE in that case or the interrupt would=20
> keep re-occurring (note that FSL has this concept of=20
> auto-acked interrupts via the on die MPIC for which you can=20
> potentially use PACA_IRQ_EE_EDGE instead and avoid having to=20
> mask MSR:EE).
>=20
> > I don't understand "the interrupt condition on device is cleared"
> > here.
>=20
> Well, the external interrupt line will remain asserted until=20
> the device (via the PIC) stops interrupting the processor :-)=20

Yes, the device keeps on interrupting the interrupt until the device clear =
its interrupted condition.

> Either because we mask at the PIC level (or raise the=20
> priority) or because the condition goes away.
> =20
> > I think regardless if you clear the device interrupt status, the=20
> > system still  receive a pending interrupt once EE or GS =3D 1.
>=20
> Why ? Depends on your PIC actually but if it's a sane one, it=20
> won't "remember" a level interrupt that is gone. An edge=20
> interrupt is a different matter and is latched.

But the interrupt controller like MPIC should leave this irq as pending if =
we don't ACK/EOI this irq at all if we just clear the device, right?

>=20
> Some MPIC implementations tend to generate a spurrious IRQ in=20
> the case of level IRQs going away. IE. they still remember an=20
> event occurred and interrupt the processor, but on IACK=20
> return the spurious vector. However that isn't guaranteed to=20

Yes, this is just what I mean. No matter if this vector is still valid, the=
 external interrupt exception always occur once EE =3D 1 again.

> be the case and it is perfectly ok (and a good
> idea) for the PIC to just stop asserting the external=20
> interrupt line if the original device interrupt goes away=20
> (and it's configured as level sensitive).

I don't remember MPIC can work with this way. So I'd like to look the manua=
l again :)

Tiejun=

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Scott Wood]

On 05/09/2013 07:37:42 AM, Benjamin Herrenschmidt wrote:
> On Thu, 2013-05-09 at 17:44 +0800, tiejun.chen wrote:
> >
> > Actually in the case GS=3D1 even if EE=3D0, EXT/DEC/DBELL still occur =20
> as I
> > recall.
>=20
> Only if directed to the hypervisor.

This is always the case with KVM, right?  At least on booke...

> > > Case 1)
> > >   -> Local_irq_disable()  will set soft_enabled =3D 0
> > >   -> Now Externel interrupt happens, there we set PACA_IRQ_EE in
> > irq_happened, Also clears EE in SRR1 and rfi. So interrupts are hard
> > disabled. No more other interrupt gated by MSR.EE can happen. Looks
> > like the idea here is to not let a device keep on inserting =20
> interrupt
> > till the interrupt condition on device is cleared, right?
>=20
> The external interrupt line is level sensitive normally, so we have to
> mask MSR:EE in that case or the interrupt would keep re-occurring =20
> (note
> that FSL has this concept of auto-acked interrupts via the on die MPIC
> for which you can potentially use PACA_IRQ_EE_EDGE instead and avoid
> having to mask MSR:EE).

Note that if we do this, we can no longer leave the interrupt vector in =20
EPR, and would have to track (potentially multiple different) pending =20
external interrupts in software.

-Scott=

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Benjamin Herrenschmidt]

On Thu, 2013-05-09 at 16:27 -0500, Scott Wood wrote:
> On 05/09/2013 07:37:42 AM, Benjamin Herrenschmidt wrote:
> > On Thu, 2013-05-09 at 17:44 +0800, tiejun.chen wrote:
> > >
> > > Actually in the case GS=1 even if EE=0, EXT/DEC/DBELL still occur  
> > as I
> > > recall.
> > 
> > Only if directed to the hypervisor.
> 
> This is always the case with KVM, right?  At least on booke...

Hrm, on A2 we could choose iirc. Well not DEC but EXT at least, I don't
remember about DBELL.

> > > > Case 1)
> > > >   -> Local_irq_disable()  will set soft_enabled = 0
> > > >   -> Now Externel interrupt happens, there we set PACA_IRQ_EE in
> > > irq_happened, Also clears EE in SRR1 and rfi. So interrupts are hard
> > > disabled. No more other interrupt gated by MSR.EE can happen. Looks
> > > like the idea here is to not let a device keep on inserting  
> > interrupt
> > > till the interrupt condition on device is cleared, right?
> > 
> > The external interrupt line is level sensitive normally, so we have to
> > mask MSR:EE in that case or the interrupt would keep re-occurring  
> > (note
> > that FSL has this concept of auto-acked interrupts via the on die MPIC
> > for which you can potentially use PACA_IRQ_EE_EDGE instead and avoid
> > having to mask MSR:EE).
> 
> Note that if we do this, we can no longer leave the interrupt vector in  
> EPR, and would have to track (potentially multiple different) pending  
> external interrupts in software.

Right, you can have a little queue in the PACA and leave EE disabled if
it's full. You can probably get away with a queue of 1 though :-)

Cheers,
Ben.

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Scott Wood]

On 05/09/2013 05:07:00 PM, Benjamin Herrenschmidt wrote:
> On Thu, 2013-05-09 at 16:27 -0500, Scott Wood wrote:
> > On 05/09/2013 07:37:42 AM, Benjamin Herrenschmidt wrote:
> > > On Thu, 2013-05-09 at 17:44 +0800, tiejun.chen wrote:
> > > >
> > > > Actually in the case GS=3D1 even if EE=3D0, EXT/DEC/DBELL still =20
> occur
> > > as I
> > > > recall.
> > >
> > > Only if directed to the hypervisor.
> >
> > This is always the case with KVM, right?  At least on booke...
>=20
> Hrm, on A2 we could choose iirc. Well not DEC but EXT at least, I =20
> don't
> remember about DBELL.

As for as the hardware goes we can choose on e500mc as well -- we just =20
don't support it in KVM, because we can't distinguish between guest and =20
host interrupts without routing the latter to critical, which is way =20
too much of a pain for Linux (we did it in our standalone hypervisor, =20
though).

-Scott=

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Kevin Hao]

[-- Attachment #1: Type: text/plain, Size: 2403 bytes --]

On Fri, May 10, 2013 at 08:07:00AM +1000, Benjamin Herrenschmidt wrote:
> On Thu, 2013-05-09 at 16:27 -0500, Scott Wood wrote:
> > On 05/09/2013 07:37:42 AM, Benjamin Herrenschmidt wrote:
> > > On Thu, 2013-05-09 at 17:44 +0800, tiejun.chen wrote:
> > > >
> > > > Actually in the case GS=1 even if EE=0, EXT/DEC/DBELL still occur  
> > > as I
> > > > recall.
> > > 
> > > Only if directed to the hypervisor.
> > 
> > This is always the case with KVM, right?  At least on booke...
> 
> Hrm, on A2 we could choose iirc. Well not DEC but EXT at least, I don't
> remember about DBELL.
> 
> > > > > Case 1)
> > > > >   -> Local_irq_disable()  will set soft_enabled = 0
> > > > >   -> Now Externel interrupt happens, there we set PACA_IRQ_EE in
> > > > irq_happened, Also clears EE in SRR1 and rfi. So interrupts are hard
> > > > disabled. No more other interrupt gated by MSR.EE can happen. Looks
> > > > like the idea here is to not let a device keep on inserting  
> > > interrupt
> > > > till the interrupt condition on device is cleared, right?
> > > 
> > > The external interrupt line is level sensitive normally, so we have to
> > > mask MSR:EE in that case or the interrupt would keep re-occurring  
> > > (note
> > > that FSL has this concept of auto-acked interrupts via the on die MPIC
> > > for which you can potentially use PACA_IRQ_EE_EDGE instead and avoid
> > > having to mask MSR:EE).
> > 
> > Note that if we do this, we can no longer leave the interrupt vector in  
> > EPR, and would have to track (potentially multiple different) pending  
> > external interrupts in software.
> 
> Right, you can have a little queue in the PACA and leave EE disabled if
> it's full. You can probably get away with a queue of 1 though :-)

So I would assume you will not pick up these two patches, right?
http://patchwork.ozlabs.org/patch/235530/
http://patchwork.ozlabs.org/patch/235532/

Anyway it is more easier to enable the external proxy by using this method.
But if you insist, I can respin a patch to use the method you suggested
since it will definitely reduce the window where the irq is hard disabled.

Thanks,
Kevin

> 
> Cheers,
> Ben.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Benjamin Herrenschmidt]

On Fri, 2013-05-10 at 22:12 +0800, Kevin Hao wrote:
> So I would assume you will not pick up these two patches, right?
> http://patchwork.ozlabs.org/patch/235530/
> http://patchwork.ozlabs.org/patch/235532/
> 
> Anyway it is more easier to enable the external proxy by using this method.
> But if you insist, I can respin a patch to use the method you suggested
> since it will definitely reduce the window where the irq is hard disabled.

I would keep the EE_EDGE bit definition. I have no objection to a gradual
approach however for the other one where we apply it as is now to enable
coreint while you do a rework to make it better :-)

Cheers,
Ben.

Re: [RFC][KVM][PATCH 1/1] kvm:ppc:booke-64: soft-disable interrupts

[Benjamin Herrenschmidt]

On Sat, 2013-05-11 at 07:49 +1000, Benjamin Herrenschmidt wrote:
> I would keep the EE_EDGE bit definition. I have no objection to a gradual
> approach however for the other one where we apply it as is now to enable
> coreint while you do a rework to make it better :-)

Note also that I generally don't apply FSL related patches directly, I
rely on Kumar Gala picking them up so he's the one ultimately making
that choice.

Cheers,
Ben.