[PATCH 0/4] KVM: PPC: BookE: Add EPR user space support

[PATCH 0/4] KVM: PPC: BookE: Add EPR user space support

[Alexander Graf]

The FSL MPIC implementation contains a feature called "external proxy
facility" which allows for interrupts to be acknowledged in the MPIC
as soon as a core accepts its pending external interrupt.

This patch set implements all the necessary pieces to support this
from the kernel space side.

Alexander Graf (4):
  KVM: PPC: BookE: Allow irq deliveries to inject requests
  KVM: PPC: BookE: Emulate mfspr on EPR
  KVM: PPC: BookE: Implement EPR exit
  KVM: PPC: BookE: Add EPR ONE_REG sync

 Documentation/virtual/kvm/api.txt   |   24 +++++++++++++++-
 arch/powerpc/include/asm/kvm_host.h |    2 +
 arch/powerpc/include/asm/kvm_ppc.h  |    9 ++++++
 arch/powerpc/include/uapi/asm/kvm.h |    6 +++-
 arch/powerpc/kvm/booke.c            |   51 +++++++++++++++++++++++++++++++---
 arch/powerpc/kvm/booke_emulate.c    |    3 ++
 arch/powerpc/kvm/powerpc.c          |   10 +++++++
 include/linux/kvm_host.h            |    1 +
 include/uapi/linux/kvm.h            |    6 ++++
 9 files changed, 104 insertions(+), 8 deletions(-)

[PATCH 1/4] KVM: PPC: BookE: Allow irq deliveries to inject requests

[Alexander Graf]

When injecting an interrupt into guest context, we usually don't need
to check for requests anymore. At least not until today.

With the introduction of EPR, we will have to create a request when the
guest has successfully accepted an external interrupt though.

So we need to prepare the interrupt delivery to abort guest entry
gracefully. Otherwise we'd delay the EPR request.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 arch/powerpc/kvm/booke.c |   14 ++++++++++----
 1 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
index 69f1140..4ae83f9 100644
--- a/arch/powerpc/kvm/booke.c
+++ b/arch/powerpc/kvm/booke.c
@@ -554,14 +554,16 @@ static void update_timer_ints(struct kvm_vcpu *vcpu)
 		kvmppc_core_dequeue_watchdog(vcpu);
 }
 
-static void kvmppc_core_check_exceptions(struct kvm_vcpu *vcpu)
+static int kvmppc_core_check_exceptions(struct kvm_vcpu *vcpu)
 {
 	unsigned long *pending = &vcpu->arch.pending_exceptions;
 	unsigned int priority;
+	int r = 0;
 
 	priority = __ffs(*pending);
 	while (priority < BOOKE_IRQPRIO_MAX) {
-		if (kvmppc_booke_irqprio_deliver(vcpu, priority))
+		r = kvmppc_booke_irqprio_deliver(vcpu, priority);
+		if (r)
 			break;
 
 		priority = find_next_bit(pending,
@@ -571,15 +573,19 @@ static void kvmppc_core_check_exceptions(struct kvm_vcpu *vcpu)
 
 	/* Tell the guest about our interrupt status */
 	vcpu->arch.shared->int_pending = !!*pending;
+
+	return r > 1;
 }
 
 /* Check pending exceptions and deliver one, if possible. */
 int kvmppc_core_prepare_to_enter(struct kvm_vcpu *vcpu)
 {
-	int r = 0;
+	int r;
 	WARN_ON_ONCE(!irqs_disabled());
 
-	kvmppc_core_check_exceptions(vcpu);
+	r = kvmppc_core_check_exceptions(vcpu);
+	if (r)
+		return r;
 
 	if (vcpu->arch.shared->msr & MSR_WE) {
 		local_irq_enable();
-- 
1.6.0.2

[PATCH 2/4] KVM: PPC: BookE: Emulate mfspr on EPR

[Alexander Graf]

The EPR register is potentially valid for PR KVM as well, so we need
to emulate accesses to it. It's only defined for reading, so only
handle the mfspr case.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 arch/powerpc/kvm/booke_emulate.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/arch/powerpc/kvm/booke_emulate.c b/arch/powerpc/kvm/booke_emulate.c
index 4685b8c..27a4b28 100644
--- a/arch/powerpc/kvm/booke_emulate.c
+++ b/arch/powerpc/kvm/booke_emulate.c
@@ -269,6 +269,9 @@ int kvmppc_booke_emulate_mfspr(struct kvm_vcpu *vcpu, int sprn, ulong *spr_val)
 	case SPRN_ESR:
 		*spr_val = vcpu->arch.shared->esr;
 		break;
+	case SPRN_EPR:
+		*spr_val = vcpu->arch.epr;
+		break;
 	case SPRN_CSRR0:
 		*spr_val = vcpu->arch.csrr0;
 		break;
-- 
1.6.0.2

[PATCH 3/4] KVM: PPC: BookE: Implement EPR exit

[Alexander Graf]

The External Proxy Facility in FSL BookE chips allows the interrupt
controller to automatically acknowledge an interrupt as soon as a
core gets its pending external interrupt delivered.

Today, user space implements the interrupt controller, so we need to
check on it during such a cycle.

This patch implements logic for user space to enable EPR exiting,
disable EPR exiting and EPR exiting itself, so that user space can
acknowledge an interrupt when an external interrupt has successfully
been delivered into the guest vcpu.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 Documentation/virtual/kvm/api.txt   |   23 +++++++++++++++++++++--
 arch/powerpc/include/asm/kvm_host.h |    2 ++
 arch/powerpc/include/asm/kvm_ppc.h  |    9 +++++++++
 arch/powerpc/kvm/booke.c            |   16 +++++++++++++++-
 arch/powerpc/kvm/powerpc.c          |   10 ++++++++++
 include/linux/kvm_host.h            |    1 +
 include/uapi/linux/kvm.h            |    6 ++++++
 7 files changed, 64 insertions(+), 3 deletions(-)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 9cf591d..9137b7f 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2238,8 +2238,8 @@ executed a memory-mapped I/O instruction which could not be satisfied
 by kvm.  The 'data' member contains the written data if 'is_write' is
 true, and should be filled by application code otherwise.
 
-NOTE: For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_DCR
-      and KVM_EXIT_PAPR the corresponding
+NOTE: For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_DCR,
+      KVM_EXIT_PAPR and KVM_EXIT_EPR the corresponding
 operations are complete (and guest state is consistent) only after userspace
 has re-entered the kernel with KVM_RUN.  The kernel side will first finish
 incomplete operations and then check for pending signals.  Userspace
@@ -2342,6 +2342,25 @@ The possible hypercalls are defined in the Power Architecture Platform
 Requirements (PAPR) document available from www.power.org (free
 developer registration required to access it).
 
+		/* KVM_EXIT_EPR */
+		struct {
+			__u32 epr;
+		} epr;
+
+On FSL BookE PowerPC chips, the interrupt controller has a fast patch
+interrupt acknowledge path to the core. When the core successfully
+delivers an interrupt, it automatically populates the EPR register with
+the interrupt vector number and acknowledges the interrupt inside
+the interrupt controller.
+
+In case the interrupt controller lives in user space, we need to do
+the interrupt acknowledge cycle through it to fetch the next to be
+delivered interrupt vector using this exit.
+
+It gets triggered whenever both KVM_CAP_PPC_EPR are enabled and an
+external interrupt has just been delivered into the guest. User space
+should put the acknowledged interrupt vector into the 'epr' field.
+
 		/* Fix the size of the union. */
 		char padding[256];
 	};
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index ab49c6c..8a72d59 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -520,6 +520,8 @@ struct kvm_vcpu_arch {
 	u8 sane;
 	u8 cpu_type;
 	u8 hcall_needed;
+	u8 epr_enabled;
+	u8 epr_needed;
 
 	u32 cpr0_cfgaddr; /* holds the last set cpr0_cfgaddr */
 
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index 5f5f69a..493630e 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -264,6 +264,15 @@ static inline void kvm_linear_init(void)
 {}
 #endif
 
+static inline void kvmppc_set_epr(struct kvm_vcpu *vcpu, u32 epr)
+{
+#ifdef CONFIG_KVM_BOOKE_HV
+	mtspr(SPRN_GEPR, epr);
+#elif defined(CONFIG_BOOKE)
+	vcpu->arch.epr = epr;
+#endif
+}
+
 int kvm_vcpu_ioctl_config_tlb(struct kvm_vcpu *vcpu,
 			      struct kvm_config_tlb *cfg);
 int kvm_vcpu_ioctl_dirty_tlb(struct kvm_vcpu *vcpu,
diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
index 4ae83f9..363301f 100644
--- a/arch/powerpc/kvm/booke.c
+++ b/arch/powerpc/kvm/booke.c
@@ -306,7 +306,7 @@ static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
 {
 	int allowed = 0;
 	ulong msr_mask = 0;
-	bool update_esr = false, update_dear = false;
+	bool update_esr = false, update_dear = false, update_epr = false;
 	ulong crit_raw = vcpu->arch.shared->critical;
 	ulong crit_r1 = kvmppc_get_gpr(vcpu, 1);
 	bool crit;
@@ -369,6 +369,8 @@ static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
 		keep_irq = true;
 		/* fall through */
 	case BOOKE_IRQPRIO_EXTERNAL:
+		if (vcpu->arch.epr_enabled)
+			update_epr = true;
 	case BOOKE_IRQPRIO_DBELL:
 		allowed = vcpu->arch.shared->msr & MSR_EE;
 		allowed = allowed && !crit;
@@ -408,6 +410,11 @@ static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
 			set_guest_esr(vcpu, vcpu->arch.queued_esr);
 		if (update_dear == true)
 			set_guest_dear(vcpu, vcpu->arch.queued_dear);
+		if (update_epr == true) {
+			kvm_make_request(KVM_REQ_EPR_EXIT, vcpu);
+			/* Indicate that we want to recheck requests */
+			allowed = 2;
+		}
 
 		new_msr &= msr_mask;
 #if defined(CONFIG_64BIT)
@@ -616,6 +623,13 @@ int kvmppc_core_check_requests(struct kvm_vcpu *vcpu)
 		r = 0;
 	}
 
+	if (kvm_check_request(KVM_REQ_EPR_EXIT, vcpu)) {
+		vcpu->run->epr.epr = 0;
+		vcpu->arch.epr_needed = true;
+		vcpu->run->exit_reason = KVM_EXIT_EPR;
+		r = 0;
+	}
+
 	return r;
 }
 
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index e2225e5..934413c 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -306,6 +306,7 @@ int kvm_dev_ioctl_check_extension(long ext)
 #ifdef CONFIG_BOOKE
 	case KVM_CAP_PPC_BOOKE_SREGS:
 	case KVM_CAP_PPC_BOOKE_WATCHDOG:
+	case KVM_CAP_PPC_EPR:
 #else
 	case KVM_CAP_PPC_SEGSTATE:
 	case KVM_CAP_PPC_HIOR:
@@ -721,6 +722,11 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
 		for (i = 0; i < 9; ++i)
 			kvmppc_set_gpr(vcpu, 4 + i, run->papr_hcall.args[i]);
 		vcpu->arch.hcall_needed = 0;
+#ifdef CONFIG_BOOKE
+	} else if (vcpu->arch.epr_needed) {
+		kvmppc_set_epr(vcpu, run->epr.epr);
+		vcpu->arch.epr_needed = 0;
+#endif
 	}
 
 	r = kvmppc_vcpu_run(run, vcpu);
@@ -762,6 +768,10 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
 		r = 0;
 		vcpu->arch.papr_enabled = true;
 		break;
+	case KVM_CAP_PPC_EPR:
+		r = 0;
+		vcpu->arch.epr_enabled = cap->args[0];
+		break;
 #ifdef CONFIG_BOOKE
 	case KVM_CAP_PPC_BOOKE_WATCHDOG:
 		r = 0;
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 32fdc45..789f365 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -122,6 +122,7 @@ static inline bool is_error_page(struct page *page)
 #define KVM_REQ_WATCHDOG          18
 #define KVM_REQ_MASTERCLOCK_UPDATE 19
 #define KVM_REQ_MCLOCK_INPROGRESS 20
+#define KVM_REQ_EPR_EXIT          21
 
 #define KVM_USERSPACE_IRQ_SOURCE_ID		0
 #define KVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID	1
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index e6e5d4b..d1fca03 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -168,6 +168,7 @@ struct kvm_pit_config {
 #define KVM_EXIT_PAPR_HCALL	  19
 #define KVM_EXIT_S390_UCONTROL	  20
 #define KVM_EXIT_WATCHDOG         21
+#define KVM_EXIT_EPR              22
 
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
@@ -285,6 +286,10 @@ struct kvm_run {
 			__u64 ret;
 			__u64 args[9];
 		} papr_hcall;
+		/* KVM_EXIT_EPR */
+		struct {
+			__u32 epr;
+		} epr;
 		/* Fix the size of the union. */
 		char padding[256];
 	};
@@ -635,6 +640,7 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_IRQFD_RESAMPLE 82
 #define KVM_CAP_PPC_BOOKE_WATCHDOG 83
 #define KVM_CAP_PPC_HTAB_FD 84
+#define KVM_CAP_PPC_EPR 85
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
-- 
1.6.0.2

[PATCH 4/4] KVM: PPC: BookE: Add EPR ONE_REG sync

[Alexander Graf]

We need to be able to read and write the contents of the EPR register
from user space.

This patch implements that logic through the ONE_REG API and declares
its (never implemented) SREGS counterpart as deprecated.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 Documentation/virtual/kvm/api.txt   |    1 +
 arch/powerpc/include/uapi/asm/kvm.h |    6 +++++-
 arch/powerpc/kvm/booke.c            |   21 +++++++++++++++++++++
 3 files changed, 27 insertions(+), 1 deletions(-)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 9137b7f..de525b4 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1774,6 +1774,7 @@ registers, find a list below:
   PPC   | KVM_REG_PPC_VPA_SLB   | 128
   PPC   | KVM_REG_PPC_VPA_DTL   | 128
   PPC   | KVM_REG_PPC_EPCR	| 32
+  PPC   | KVM_REG_PPC_EPR	| 32
 
 4.69 KVM_GET_ONE_REG
 
diff --git a/arch/powerpc/include/uapi/asm/kvm.h b/arch/powerpc/include/uapi/asm/kvm.h
index 2fba8a6..16064d0 100644
--- a/arch/powerpc/include/uapi/asm/kvm.h
+++ b/arch/powerpc/include/uapi/asm/kvm.h
@@ -114,7 +114,10 @@ struct kvm_regs {
 /* Embedded Floating Point (SPE) -- IVOR32-34 if KVM_SREGS_E_IVOR */
 #define KVM_SREGS_E_SPE			(1 << 9)
 
-/* External Proxy (EXP) -- EPR */
+/*
+ * DEPRECATED! USE ONE_REG FOR THIS ONE!
+ * External Proxy (EXP) -- EPR
+ */
 #define KVM_SREGS_EXP			(1 << 10)
 
 /* External PID (E.PD) -- EPSC/EPLC */
@@ -412,5 +415,6 @@ struct kvm_get_htab_header {
 #define KVM_REG_PPC_VPA_DTL	(KVM_REG_PPC | KVM_REG_SIZE_U128 | 0x84)
 
 #define KVM_REG_PPC_EPCR	(KVM_REG_PPC | KVM_REG_SIZE_U32 | 0x85)
+#define KVM_REG_PPC_EPR		(KVM_REG_PPC | KVM_REG_SIZE_U32 | 0x86)
 
 #endif /* __LINUX_KVM_POWERPC_H */
diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
index 363301f..2b79c44 100644
--- a/arch/powerpc/kvm/booke.c
+++ b/arch/powerpc/kvm/booke.c
@@ -300,6 +300,15 @@ static void set_guest_esr(struct kvm_vcpu *vcpu, u32 esr)
 #endif
 }
 
+static unsigned long get_guest_epr(struct kvm_vcpu *vcpu)
+{
+#ifdef CONFIG_KVM_BOOKE_HV
+	return mfspr(SPRN_GEPR);
+#else
+	return vcpu->arch.epr;
+#endif
+}
+
 /* Deliver the interrupt of the corresponding priority, if possible. */
 static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
                                         unsigned int priority)
@@ -1408,6 +1417,11 @@ int kvm_vcpu_ioctl_get_one_reg(struct kvm_vcpu *vcpu, struct kvm_one_reg *reg)
 				 &vcpu->arch.dbg_reg.dac[dac], sizeof(u64));
 		break;
 	}
+	case KVM_REG_PPC_EPR: {
+		u32 epr = get_guest_epr(vcpu);
+		r = put_user(epr, (u32 __user *)(long)reg->addr);
+		break;
+	}
 #if defined(CONFIG_64BIT)
 	case KVM_REG_PPC_EPCR:
 		r = put_user(vcpu->arch.epcr, (u32 __user *)(long)reg->addr);
@@ -1440,6 +1454,13 @@ int kvm_vcpu_ioctl_set_one_reg(struct kvm_vcpu *vcpu, struct kvm_one_reg *reg)
 			     (u64 __user *)(long)reg->addr, sizeof(u64));
 		break;
 	}
+	case KVM_REG_PPC_EPR: {
+		u32 new_epr;
+		r = get_user(new_epr, (u32 __user *)(long)reg->addr);
+		if (!r)
+			kvmppc_set_epr(vcpu, new_epr);
+		break;
+	}
 #if defined(CONFIG_64BIT)
 	case KVM_REG_PPC_EPCR: {
 		u32 new_epcr;
-- 
1.6.0.2

Re: [PATCH 1/4] KVM: PPC: BookE: Allow irq deliveries to inject requests

[Scott Wood]

On 01/04/2013 11:36:37 AM, Alexander Graf wrote:
> When injecting an interrupt into guest context, we usually don't need
> to check for requests anymore. At least not until today.
> 
> With the introduction of EPR, we will have to create a request when  
> the
> guest has successfully accepted an external interrupt though.
> 
> So we need to prepare the interrupt delivery to abort guest entry
> gracefully. Otherwise we'd delay the EPR request.
> 
> Signed-off-by: Alexander Graf <agraf@suse.de>
> ---
>  arch/powerpc/kvm/booke.c |   14 ++++++++++----
>  1 files changed, 10 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
> index 69f1140..4ae83f9 100644
> --- a/arch/powerpc/kvm/booke.c
> +++ b/arch/powerpc/kvm/booke.c
> @@ -554,14 +554,16 @@ static void update_timer_ints(struct kvm_vcpu  
> *vcpu)
>  		kvmppc_core_dequeue_watchdog(vcpu);
>  }
> 
> -static void kvmppc_core_check_exceptions(struct kvm_vcpu *vcpu)
> +static int kvmppc_core_check_exceptions(struct kvm_vcpu *vcpu)
>  {
>  	unsigned long *pending = &vcpu->arch.pending_exceptions;
>  	unsigned int priority;
> +	int r = 0;
> 
>  	priority = __ffs(*pending);
>  	while (priority < BOOKE_IRQPRIO_MAX) {
> -		if (kvmppc_booke_irqprio_deliver(vcpu, priority))
> +		r = kvmppc_booke_irqprio_deliver(vcpu, priority);
> +		if (r)
>  			break;
> 
>  		priority = find_next_bit(pending,
> @@ -571,15 +573,19 @@ static void kvmppc_core_check_exceptions(struct  
> kvm_vcpu *vcpu)
> 
>  	/* Tell the guest about our interrupt status */
>  	vcpu->arch.shared->int_pending = !!*pending;
> +
> +	return r > 1;
>  }

It isn't until patch 3/4 that the reason for "r > 1" appears...

Why not just:

diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
index 69f1140..964f447 100644
--- a/arch/powerpc/kvm/booke.c
+++ b/arch/powerpc/kvm/booke.c
@@ -581,6 +581,11 @@ int kvmppc_core_prepare_to_enter(struct kvm_vcpu  
*vcpu)

  	kvmppc_core_check_exceptions(vcpu);

+	if (vcpu->requests) {
+		/* Exception delivery raised request; start over */
+		return 1;
+	}
+
  	if (vcpu->arch.shared->msr & MSR_WE) {
  		local_irq_enable();
  		kvm_vcpu_block(vcpu);

...or if you really want to do it through return values, make it an  
enum rather than a magic "2".

-Scott

Re: [PATCH 2/4] KVM: PPC: BookE: Emulate mfspr on EPR

[Scott Wood]

On 01/04/2013 11:36:38 AM, Alexander Graf wrote:
> The EPR register is potentially valid for PR KVM as well, so we need
> to emulate accesses to it. It's only defined for reading, so only
> handle the mfspr case.
> 
> Signed-off-by: Alexander Graf <agraf@suse.de>
> ---
>  arch/powerpc/kvm/booke_emulate.c |    3 +++
>  1 files changed, 3 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/powerpc/kvm/booke_emulate.c  
> b/arch/powerpc/kvm/booke_emulate.c
> index 4685b8c..27a4b28 100644
> --- a/arch/powerpc/kvm/booke_emulate.c
> +++ b/arch/powerpc/kvm/booke_emulate.c
> @@ -269,6 +269,9 @@ int kvmppc_booke_emulate_mfspr(struct kvm_vcpu  
> *vcpu, int sprn, ulong *spr_val)
>  	case SPRN_ESR:
>  		*spr_val = vcpu->arch.shared->esr;
>  		break;
> +	case SPRN_EPR:
> +		*spr_val = vcpu->arch.epr;
> +		break;

It's not just potentially valid -- in our internal tree we do paravirt  
EPR on e500v2 even though the hardware doesn't implement it.

-Scott

Re: [PATCH 4/4] KVM: PPC: BookE: Add EPR ONE_REG sync

[Scott Wood]

On 01/04/2013 11:36:40 AM, Alexander Graf wrote:
> We need to be able to read and write the contents of the EPR register
> from user space.
> 
> This patch implements that logic through the ONE_REG API and declares
> its (never implemented) SREGS counterpart as deprecated.

QEMU already uses SREGS to read this...  I'm not sure what happened on  
the KVM side.

-Scott

Re: [PATCH 3/4] KVM: PPC: BookE: Implement EPR exit

[Scott Wood]

On 01/04/2013 11:36:39 AM, Alexander Graf wrote:
> diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
> index 4ae83f9..363301f 100644
> --- a/arch/powerpc/kvm/booke.c
> +++ b/arch/powerpc/kvm/booke.c
> @@ -306,7 +306,7 @@ static int kvmppc_booke_irqprio_deliver(struct  
> kvm_vcpu *vcpu,
>  {
>  	int allowed = 0;
>  	ulong msr_mask = 0;
> -	bool update_esr = false, update_dear = false;
> +	bool update_esr = false, update_dear = false, update_epr =  
> false;
>  	ulong crit_raw = vcpu->arch.shared->critical;
>  	ulong crit_r1 = kvmppc_get_gpr(vcpu, 1);
>  	bool crit;
> @@ -369,6 +369,8 @@ static int kvmppc_booke_irqprio_deliver(struct  
> kvm_vcpu *vcpu,
>  		keep_irq = true;
>  		/* fall through */
>  	case BOOKE_IRQPRIO_EXTERNAL:
> +		if (vcpu->arch.epr_enabled)
> +			update_epr = true;
>  	case BOOKE_IRQPRIO_DBELL:

We don't want to update EPR on decrementer/FIT.  Also, missing  
fall-through comment.

> @@ -762,6 +768,10 @@ static int kvm_vcpu_ioctl_enable_cap(struct  
> kvm_vcpu *vcpu,
>  		r = 0;
>  		vcpu->arch.papr_enabled = true;
>  		break;
> +	case KVM_CAP_PPC_EPR:
> +		r = 0;
> +		vcpu->arch.epr_enabled = cap->args[0];
> +		break;

I don't see where the docs say that this capability uses args[0] as an
enable/disable flag.

-Scott

Re: [PATCH 4/4] KVM: PPC: BookE: Add EPR ONE_REG sync

[Alexander Graf]

On 04.01.2013, at 21:08, Scott Wood wrote:

> On 01/04/2013 11:36:40 AM, Alexander Graf wrote:
>> We need to be able to read and write the contents of the EPR register
>> from user space.
>> This patch implements that logic through the ONE_REG API and declares
>> its (never implemented) SREGS counterpart as deprecated.
> 
> QEMU already uses SREGS to read this...  I'm not sure what happened on the KVM side.

Since it's never been implemented on the KVM side, I don't see why it'd be an issue to force ONE_REG on any user :). In the long run, I'd like to move as much of the state synchronization to ONE_REG.


Alex

Re: [PATCH 1/4] KVM: PPC: BookE: Allow irq deliveries to inject requests

[Alexander Graf]

On 04.01.2013, at 20:40, Scott Wood wrote:

> On 01/04/2013 11:36:37 AM, Alexander Graf wrote:
>> When injecting an interrupt into guest context, we usually don't need
>> to check for requests anymore. At least not until today.
>> With the introduction of EPR, we will have to create a request when the
>> guest has successfully accepted an external interrupt though.
>> So we need to prepare the interrupt delivery to abort guest entry
>> gracefully. Otherwise we'd delay the EPR request.
>> Signed-off-by: Alexander Graf <agraf@suse.de>
>> ---
>> arch/powerpc/kvm/booke.c |   14 ++++++++++----
>> 1 files changed, 10 insertions(+), 4 deletions(-)
>> diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
>> index 69f1140..4ae83f9 100644
>> --- a/arch/powerpc/kvm/booke.c
>> +++ b/arch/powerpc/kvm/booke.c
>> @@ -554,14 +554,16 @@ static void update_timer_ints(struct kvm_vcpu *vcpu)
>> 		kvmppc_core_dequeue_watchdog(vcpu);
>> }
>> -static void kvmppc_core_check_exceptions(struct kvm_vcpu *vcpu)
>> +static int kvmppc_core_check_exceptions(struct kvm_vcpu *vcpu)
>> {
>> 	unsigned long *pending = &vcpu->arch.pending_exceptions;
>> 	unsigned int priority;
>> +	int r = 0;
>> 	priority = __ffs(*pending);
>> 	while (priority < BOOKE_IRQPRIO_MAX) {
>> -		if (kvmppc_booke_irqprio_deliver(vcpu, priority))
>> +		r = kvmppc_booke_irqprio_deliver(vcpu, priority);
>> +		if (r)
>> 			break;
>> 		priority = find_next_bit(pending,
>> @@ -571,15 +573,19 @@ static void kvmppc_core_check_exceptions(struct kvm_vcpu *vcpu)
>> 	/* Tell the guest about our interrupt status */
>> 	vcpu->arch.shared->int_pending = !!*pending;
>> +
>> +	return r > 1;
>> }
> 
> It isn't until patch 3/4 that the reason for "r > 1" appears...
> 
> Why not just:
> 
> diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
> index 69f1140..964f447 100644
> --- a/arch/powerpc/kvm/booke.c
> +++ b/arch/powerpc/kvm/booke.c
> @@ -581,6 +581,11 @@ int kvmppc_core_prepare_to_enter(struct kvm_vcpu *vcpu)
> 
> 	kvmppc_core_check_exceptions(vcpu);
> 
> +	if (vcpu->requests) {

The main reason I didn't do it this way was that until now, we keep the guest runnable with requests pending that we never check for. With this applied, we would end up in an endless loop during guest entry.

I'm not sure which way is better though. It does have a certain appeal to fail hard when we're doing something stupid :).


Alex

> +		/* Exception delivery raised request; start over */
> +		return 1;
> +	}
> +
> 	if (vcpu->arch.shared->msr & MSR_WE) {
> 		local_irq_enable();
> 		kvm_vcpu_block(vcpu);
> 
> ...or if you really want to do it through return values, make it an enum rather than a magic "2".
> 
> -Scott
> --
> To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 4/4] KVM: PPC: BookE: Add EPR ONE_REG sync

[Scott Wood]

On 01/04/2013 04:55:34 PM, Alexander Graf wrote:
> 
> On 04.01.2013, at 21:08, Scott Wood wrote:
> 
> > On 01/04/2013 11:36:40 AM, Alexander Graf wrote:
> >> We need to be able to read and write the contents of the EPR  
> register
> >> from user space.
> >> This patch implements that logic through the ONE_REG API and  
> declares
> >> its (never implemented) SREGS counterpart as deprecated.
> >
> > QEMU already uses SREGS to read this...  I'm not sure what happened  
> on the KVM side.
> 
> Since it's never been implemented on the KVM side, I don't see why  
> it'd be an issue to force ONE_REG on any user :). In the long run,  
> I'd like to move as much of the state synchronization to ONE_REG.

That's fine -- but we should make sure not to forget to update QEMU.

-Scott

Re: [PATCH 4/4] KVM: PPC: BookE: Add EPR ONE_REG sync

[Alexander Graf]

On 05.01.2013, at 00:03, Scott Wood wrote:

> On 01/04/2013 04:55:34 PM, Alexander Graf wrote:
>> On 04.01.2013, at 21:08, Scott Wood wrote:
>> > On 01/04/2013 11:36:40 AM, Alexander Graf wrote:
>> >> We need to be able to read and write the contents of the EPR register
>> >> from user space.
>> >> This patch implements that logic through the ONE_REG API and declares
>> >> its (never implemented) SREGS counterpart as deprecated.
>> >
>> > QEMU already uses SREGS to read this...  I'm not sure what happened on the KVM side.
>> Since it's never been implemented on the KVM side, I don't see why it'd be an issue to force ONE_REG on any user :). In the long run, I'd like to move as much of the state synchronization to ONE_REG.
> 
> That's fine -- but we should make sure not to forget to update QEMU.

Yes. We need to wait for these patches to trickle down to kvm.git first though anyway.


Alex

Re: [PATCH 2/4] KVM: PPC: BookE: Emulate mfspr on EPR

[Alexander Graf]

On 04.01.2013, at 20:50, Scott Wood wrote:

> On 01/04/2013 11:36:38 AM, Alexander Graf wrote:
>> The EPR register is potentially valid for PR KVM as well, so we need
>> to emulate accesses to it. It's only defined for reading, so only
>> handle the mfspr case.
>> Signed-off-by: Alexander Graf <agraf@suse.de>
>> ---
>> arch/powerpc/kvm/booke_emulate.c |    3 +++
>> 1 files changed, 3 insertions(+), 0 deletions(-)
>> diff --git a/arch/powerpc/kvm/booke_emulate.c b/arch/powerpc/kvm/booke_emulate.c
>> index 4685b8c..27a4b28 100644
>> --- a/arch/powerpc/kvm/booke_emulate.c
>> +++ b/arch/powerpc/kvm/booke_emulate.c
>> @@ -269,6 +269,9 @@ int kvmppc_booke_emulate_mfspr(struct kvm_vcpu *vcpu, int sprn, ulong *spr_val)
>> 	case SPRN_ESR:
>> 		*spr_val = vcpu->arch.shared->esr;
>> 		break;
>> +	case SPRN_EPR:
>> +		*spr_val = vcpu->arch.epr;
>> +		break;
> 
> It's not just potentially valid -- in our internal tree we do paravirt EPR on e500v2 even though the hardware doesn't implement it.

I want to later add a flag to the QEMU e500 pv machine indicating that we support EPR. Once we get that, any e500 target cpu can use EPR.


Alex

[PATCH 3/4] KVM: PPC: BookE: Implement EPR exit

[Alexander Graf]

The External Proxy Facility in FSL BookE chips allows the interrupt
controller to automatically acknowledge an interrupt as soon as a
core gets its pending external interrupt delivered.

Today, user space implements the interrupt controller, so we need to
check on it during such a cycle.

This patch implements logic for user space to enable EPR exiting,
disable EPR exiting and EPR exiting itself, so that user space can
acknowledge an interrupt when an external interrupt has successfully
been delivered into the guest vcpu.

Signed-off-by: Alexander Graf <agraf@suse.de>

---

v1 -> v2:

  - rework update_epr logic
  - add documentation for ENABLE_CAP on EPR cap
---
 Documentation/virtual/kvm/api.txt   |   40 +++++++++++++++++++++++++++++++++-
 arch/powerpc/include/asm/kvm_host.h |    2 +
 arch/powerpc/include/asm/kvm_ppc.h  |    9 +++++++
 arch/powerpc/kvm/booke.c            |   17 ++++++++++++++-
 arch/powerpc/kvm/powerpc.c          |   10 ++++++++
 include/linux/kvm_host.h            |    1 +
 include/uapi/linux/kvm.h            |    6 +++++
 7 files changed, 82 insertions(+), 3 deletions(-)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 9cf591d..66bf7cf 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2238,8 +2238,8 @@ executed a memory-mapped I/O instruction which could not be satisfied
 by kvm.  The 'data' member contains the written data if 'is_write' is
 true, and should be filled by application code otherwise.
 
-NOTE: For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_DCR
-      and KVM_EXIT_PAPR the corresponding
+NOTE: For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_DCR,
+      KVM_EXIT_PAPR and KVM_EXIT_EPR the corresponding
 operations are complete (and guest state is consistent) only after userspace
 has re-entered the kernel with KVM_RUN.  The kernel side will first finish
 incomplete operations and then check for pending signals.  Userspace
@@ -2342,6 +2342,25 @@ The possible hypercalls are defined in the Power Architecture Platform
 Requirements (PAPR) document available from www.power.org (free
 developer registration required to access it).
 
+		/* KVM_EXIT_EPR */
+		struct {
+			__u32 epr;
+		} epr;
+
+On FSL BookE PowerPC chips, the interrupt controller has a fast patch
+interrupt acknowledge path to the core. When the core successfully
+delivers an interrupt, it automatically populates the EPR register with
+the interrupt vector number and acknowledges the interrupt inside
+the interrupt controller.
+
+In case the interrupt controller lives in user space, we need to do
+the interrupt acknowledge cycle through it to fetch the next to be
+delivered interrupt vector using this exit.
+
+It gets triggered whenever both KVM_CAP_PPC_EPR are enabled and an
+external interrupt has just been delivered into the guest. User space
+should put the acknowledged interrupt vector into the 'epr' field.
+
 		/* Fix the size of the union. */
 		char padding[256];
 	};
@@ -2463,3 +2482,20 @@ For mmu types KVM_MMU_FSL_BOOKE_NOHV and KVM_MMU_FSL_BOOKE_HV:
    where "num_sets" is the tlb_sizes[] value divided by the tlb_ways[] value.
  - The tsize field of mas1 shall be set to 4K on TLB0, even though the
    hardware ignores this value for TLB0.
+
+6.4 KVM_CAP_PPC_EPR
+
+Architectures: ppc
+Parameters: args[0] defines whether the proxy facility is active
+Returns: 0 on success; -1 on error
+
+This capability enables or disables the delivery of interrupts through the
+external proxy facility.
+
+When enabled (args[0] != 0), every time the guest gets an external interrupt
+delivered, it automatically exits into user space with a KVM_EXIT_EPR exit
+to receive the topmost interrupt vector.
+
+When disabled (args[0] == 0), behavior is as if this facility is unsupported.
+
+When this capability is enabled, KVM_EXIT_EPR can occur.
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index ab49c6c..8a72d59 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -520,6 +520,8 @@ struct kvm_vcpu_arch {
 	u8 sane;
 	u8 cpu_type;
 	u8 hcall_needed;
+	u8 epr_enabled;
+	u8 epr_needed;
 
 	u32 cpr0_cfgaddr; /* holds the last set cpr0_cfgaddr */
 
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index 5f5f69a..493630e 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -264,6 +264,15 @@ static inline void kvm_linear_init(void)
 {}
 #endif
 
+static inline void kvmppc_set_epr(struct kvm_vcpu *vcpu, u32 epr)
+{
+#ifdef CONFIG_KVM_BOOKE_HV
+	mtspr(SPRN_GEPR, epr);
+#elif defined(CONFIG_BOOKE)
+	vcpu->arch.epr = epr;
+#endif
+}
+
 int kvm_vcpu_ioctl_config_tlb(struct kvm_vcpu *vcpu,
 			      struct kvm_config_tlb *cfg);
 int kvm_vcpu_ioctl_dirty_tlb(struct kvm_vcpu *vcpu,
diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
index 964f447..1dc8b8a 100644
--- a/arch/powerpc/kvm/booke.c
+++ b/arch/powerpc/kvm/booke.c
@@ -306,7 +306,7 @@ static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
 {
 	int allowed = 0;
 	ulong msr_mask = 0;
-	bool update_esr = false, update_dear = false;
+	bool update_esr = false, update_dear = false, update_epr = false;
 	ulong crit_raw = vcpu->arch.shared->critical;
 	ulong crit_r1 = kvmppc_get_gpr(vcpu, 1);
 	bool crit;
@@ -330,6 +330,9 @@ static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
 		keep_irq = true;
 	}
 
+	if ((priority == BOOKE_IRQPRIO_EXTERNAL) && vcpu->arch.epr_enabled)
+		update_epr = true;
+
 	switch (priority) {
 	case BOOKE_IRQPRIO_DTLB_MISS:
 	case BOOKE_IRQPRIO_DATA_STORAGE:
@@ -408,6 +411,11 @@ static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
 			set_guest_esr(vcpu, vcpu->arch.queued_esr);
 		if (update_dear == true)
 			set_guest_dear(vcpu, vcpu->arch.queued_dear);
+		if (update_epr == true) {
+			kvm_make_request(KVM_REQ_EPR_EXIT, vcpu);
+			/* Indicate that we want to recheck requests */
+			allowed = 2;
+		}
 
 		new_msr &= msr_mask;
 #if defined(CONFIG_64BIT)
@@ -615,6 +623,13 @@ int kvmppc_core_check_requests(struct kvm_vcpu *vcpu)
 		r = 0;
 	}
 
+	if (kvm_check_request(KVM_REQ_EPR_EXIT, vcpu)) {
+		vcpu->run->epr.epr = 0;
+		vcpu->arch.epr_needed = true;
+		vcpu->run->exit_reason = KVM_EXIT_EPR;
+		r = 0;
+	}
+
 	return r;
 }
 
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index e2225e5..934413c 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -306,6 +306,7 @@ int kvm_dev_ioctl_check_extension(long ext)
 #ifdef CONFIG_BOOKE
 	case KVM_CAP_PPC_BOOKE_SREGS:
 	case KVM_CAP_PPC_BOOKE_WATCHDOG:
+	case KVM_CAP_PPC_EPR:
 #else
 	case KVM_CAP_PPC_SEGSTATE:
 	case KVM_CAP_PPC_HIOR:
@@ -721,6 +722,11 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
 		for (i = 0; i < 9; ++i)
 			kvmppc_set_gpr(vcpu, 4 + i, run->papr_hcall.args[i]);
 		vcpu->arch.hcall_needed = 0;
+#ifdef CONFIG_BOOKE
+	} else if (vcpu->arch.epr_needed) {
+		kvmppc_set_epr(vcpu, run->epr.epr);
+		vcpu->arch.epr_needed = 0;
+#endif
 	}
 
 	r = kvmppc_vcpu_run(run, vcpu);
@@ -762,6 +768,10 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
 		r = 0;
 		vcpu->arch.papr_enabled = true;
 		break;
+	case KVM_CAP_PPC_EPR:
+		r = 0;
+		vcpu->arch.epr_enabled = cap->args[0];
+		break;
 #ifdef CONFIG_BOOKE
 	case KVM_CAP_PPC_BOOKE_WATCHDOG:
 		r = 0;
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 32fdc45..789f365 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -122,6 +122,7 @@ static inline bool is_error_page(struct page *page)
 #define KVM_REQ_WATCHDOG          18
 #define KVM_REQ_MASTERCLOCK_UPDATE 19
 #define KVM_REQ_MCLOCK_INPROGRESS 20
+#define KVM_REQ_EPR_EXIT          21
 
 #define KVM_USERSPACE_IRQ_SOURCE_ID		0
 #define KVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID	1
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index e6e5d4b..d1fca03 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -168,6 +168,7 @@ struct kvm_pit_config {
 #define KVM_EXIT_PAPR_HCALL	  19
 #define KVM_EXIT_S390_UCONTROL	  20
 #define KVM_EXIT_WATCHDOG         21
+#define KVM_EXIT_EPR              22
 
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
@@ -285,6 +286,10 @@ struct kvm_run {
 			__u64 ret;
 			__u64 args[9];
 		} papr_hcall;
+		/* KVM_EXIT_EPR */
+		struct {
+			__u32 epr;
+		} epr;
 		/* Fix the size of the union. */
 		char padding[256];
 	};
@@ -635,6 +640,7 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_IRQFD_RESAMPLE 82
 #define KVM_CAP_PPC_BOOKE_WATCHDOG 83
 #define KVM_CAP_PPC_HTAB_FD 84
+#define KVM_CAP_PPC_EPR 85
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
-- 
1.6.0.2

Re: [PATCH 3/4] KVM: PPC: BookE: Implement EPR exit

[Scott Wood]

On 01/04/2013 05:41:42 PM, Alexander Graf wrote:
> @@ -408,6 +411,11 @@ static int kvmppc_booke_irqprio_deliver(struct  
> kvm_vcpu *vcpu,
>  			set_guest_esr(vcpu, vcpu->arch.queued_esr);
>  		if (update_dear == true)
>  			set_guest_dear(vcpu, vcpu->arch.queued_dear);
> +		if (update_epr == true) {
> +			kvm_make_request(KVM_REQ_EPR_EXIT, vcpu);
> +			/* Indicate that we want to recheck requests */
> +			allowed = 2;
> +		}

We shouldn't need "allowed = 2" anymore.

-Scott

Re: [PATCH 3/4] KVM: PPC: BookE: Implement EPR exit

[Alexander Graf]

On 07.01.2013, at 18:47, Scott Wood wrote:

> On 01/04/2013 05:41:42 PM, Alexander Graf wrote:
>> @@ -408,6 +411,11 @@ static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
>> 			set_guest_esr(vcpu, vcpu->arch.queued_esr);
>> 		if (update_dear == true)
>> 			set_guest_dear(vcpu, vcpu->arch.queued_dear);
>> +		if (update_epr == true) {
>> +			kvm_make_request(KVM_REQ_EPR_EXIT, vcpu);
>> +			/* Indicate that we want to recheck requests */
>> +			allowed = 2;
>> +		}
> 
> We shouldn't need "allowed = 2" anymore.

Thanks for noticing :). I already removed this one locally today but wanted to wait for further comments on the series before sending out a v3.


Alex

[PATCH 3/4] KVM: PPC: BookE: Implement EPR exit

[Alexander Graf]

The External Proxy Facility in FSL BookE chips allows the interrupt
controller to automatically acknowledge an interrupt as soon as a
core gets its pending external interrupt delivered.

Today, user space implements the interrupt controller, so we need to
check on it during such a cycle.

This patch implements logic for user space to enable EPR exiting,
disable EPR exiting and EPR exiting itself, so that user space can
acknowledge an interrupt when an external interrupt has successfully
been delivered into the guest vcpu.

Signed-off-by: Alexander Graf <agraf@suse.de>

---

v1 -> v2:

  - rework update_epr logic
  - add documentation for ENABLE_CAP on EPR cap

v2 -> v3:

  - remove leftover 'allowed==2' logic
---
 Documentation/virtual/kvm/api.txt   |   40 +++++++++++++++++++++++++++++++++-
 arch/powerpc/include/asm/kvm_host.h |    2 +
 arch/powerpc/include/asm/kvm_ppc.h  |    9 +++++++
 arch/powerpc/kvm/booke.c            |   14 +++++++++++-
 arch/powerpc/kvm/powerpc.c          |   10 ++++++++
 include/linux/kvm_host.h            |    1 +
 include/uapi/linux/kvm.h            |    6 +++++
 7 files changed, 79 insertions(+), 3 deletions(-)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 9cf591d..66bf7cf 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2238,8 +2238,8 @@ executed a memory-mapped I/O instruction which could not be satisfied
 by kvm.  The 'data' member contains the written data if 'is_write' is
 true, and should be filled by application code otherwise.
 
-NOTE: For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_DCR
-      and KVM_EXIT_PAPR the corresponding
+NOTE: For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_DCR,
+      KVM_EXIT_PAPR and KVM_EXIT_EPR the corresponding
 operations are complete (and guest state is consistent) only after userspace
 has re-entered the kernel with KVM_RUN.  The kernel side will first finish
 incomplete operations and then check for pending signals.  Userspace
@@ -2342,6 +2342,25 @@ The possible hypercalls are defined in the Power Architecture Platform
 Requirements (PAPR) document available from www.power.org (free
 developer registration required to access it).
 
+		/* KVM_EXIT_EPR */
+		struct {
+			__u32 epr;
+		} epr;
+
+On FSL BookE PowerPC chips, the interrupt controller has a fast patch
+interrupt acknowledge path to the core. When the core successfully
+delivers an interrupt, it automatically populates the EPR register with
+the interrupt vector number and acknowledges the interrupt inside
+the interrupt controller.
+
+In case the interrupt controller lives in user space, we need to do
+the interrupt acknowledge cycle through it to fetch the next to be
+delivered interrupt vector using this exit.
+
+It gets triggered whenever both KVM_CAP_PPC_EPR are enabled and an
+external interrupt has just been delivered into the guest. User space
+should put the acknowledged interrupt vector into the 'epr' field.
+
 		/* Fix the size of the union. */
 		char padding[256];
 	};
@@ -2463,3 +2482,20 @@ For mmu types KVM_MMU_FSL_BOOKE_NOHV and KVM_MMU_FSL_BOOKE_HV:
    where "num_sets" is the tlb_sizes[] value divided by the tlb_ways[] value.
  - The tsize field of mas1 shall be set to 4K on TLB0, even though the
    hardware ignores this value for TLB0.
+
+6.4 KVM_CAP_PPC_EPR
+
+Architectures: ppc
+Parameters: args[0] defines whether the proxy facility is active
+Returns: 0 on success; -1 on error
+
+This capability enables or disables the delivery of interrupts through the
+external proxy facility.
+
+When enabled (args[0] != 0), every time the guest gets an external interrupt
+delivered, it automatically exits into user space with a KVM_EXIT_EPR exit
+to receive the topmost interrupt vector.
+
+When disabled (args[0] == 0), behavior is as if this facility is unsupported.
+
+When this capability is enabled, KVM_EXIT_EPR can occur.
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index ab49c6c..8a72d59 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -520,6 +520,8 @@ struct kvm_vcpu_arch {
 	u8 sane;
 	u8 cpu_type;
 	u8 hcall_needed;
+	u8 epr_enabled;
+	u8 epr_needed;
 
 	u32 cpr0_cfgaddr; /* holds the last set cpr0_cfgaddr */
 
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index 5f5f69a..493630e 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -264,6 +264,15 @@ static inline void kvm_linear_init(void)
 {}
 #endif
 
+static inline void kvmppc_set_epr(struct kvm_vcpu *vcpu, u32 epr)
+{
+#ifdef CONFIG_KVM_BOOKE_HV
+	mtspr(SPRN_GEPR, epr);
+#elif defined(CONFIG_BOOKE)
+	vcpu->arch.epr = epr;
+#endif
+}
+
 int kvm_vcpu_ioctl_config_tlb(struct kvm_vcpu *vcpu,
 			      struct kvm_config_tlb *cfg);
 int kvm_vcpu_ioctl_dirty_tlb(struct kvm_vcpu *vcpu,
diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
index 964f447..940ec80 100644
--- a/arch/powerpc/kvm/booke.c
+++ b/arch/powerpc/kvm/booke.c
@@ -306,7 +306,7 @@ static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
 {
 	int allowed = 0;
 	ulong msr_mask = 0;
-	bool update_esr = false, update_dear = false;
+	bool update_esr = false, update_dear = false, update_epr = false;
 	ulong crit_raw = vcpu->arch.shared->critical;
 	ulong crit_r1 = kvmppc_get_gpr(vcpu, 1);
 	bool crit;
@@ -330,6 +330,9 @@ static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
 		keep_irq = true;
 	}
 
+	if ((priority == BOOKE_IRQPRIO_EXTERNAL) && vcpu->arch.epr_enabled)
+		update_epr = true;
+
 	switch (priority) {
 	case BOOKE_IRQPRIO_DTLB_MISS:
 	case BOOKE_IRQPRIO_DATA_STORAGE:
@@ -408,6 +411,8 @@ static int kvmppc_booke_irqprio_deliver(struct kvm_vcpu *vcpu,
 			set_guest_esr(vcpu, vcpu->arch.queued_esr);
 		if (update_dear == true)
 			set_guest_dear(vcpu, vcpu->arch.queued_dear);
+		if (update_epr == true)
+			kvm_make_request(KVM_REQ_EPR_EXIT, vcpu);
 
 		new_msr &= msr_mask;
 #if defined(CONFIG_64BIT)
@@ -615,6 +620,13 @@ int kvmppc_core_check_requests(struct kvm_vcpu *vcpu)
 		r = 0;
 	}
 
+	if (kvm_check_request(KVM_REQ_EPR_EXIT, vcpu)) {
+		vcpu->run->epr.epr = 0;
+		vcpu->arch.epr_needed = true;
+		vcpu->run->exit_reason = KVM_EXIT_EPR;
+		r = 0;
+	}
+
 	return r;
 }
 
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index e2225e5..934413c 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -306,6 +306,7 @@ int kvm_dev_ioctl_check_extension(long ext)
 #ifdef CONFIG_BOOKE
 	case KVM_CAP_PPC_BOOKE_SREGS:
 	case KVM_CAP_PPC_BOOKE_WATCHDOG:
+	case KVM_CAP_PPC_EPR:
 #else
 	case KVM_CAP_PPC_SEGSTATE:
 	case KVM_CAP_PPC_HIOR:
@@ -721,6 +722,11 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
 		for (i = 0; i < 9; ++i)
 			kvmppc_set_gpr(vcpu, 4 + i, run->papr_hcall.args[i]);
 		vcpu->arch.hcall_needed = 0;
+#ifdef CONFIG_BOOKE
+	} else if (vcpu->arch.epr_needed) {
+		kvmppc_set_epr(vcpu, run->epr.epr);
+		vcpu->arch.epr_needed = 0;
+#endif
 	}
 
 	r = kvmppc_vcpu_run(run, vcpu);
@@ -762,6 +768,10 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
 		r = 0;
 		vcpu->arch.papr_enabled = true;
 		break;
+	case KVM_CAP_PPC_EPR:
+		r = 0;
+		vcpu->arch.epr_enabled = cap->args[0];
+		break;
 #ifdef CONFIG_BOOKE
 	case KVM_CAP_PPC_BOOKE_WATCHDOG:
 		r = 0;
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 32fdc45..789f365 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -122,6 +122,7 @@ static inline bool is_error_page(struct page *page)
 #define KVM_REQ_WATCHDOG          18
 #define KVM_REQ_MASTERCLOCK_UPDATE 19
 #define KVM_REQ_MCLOCK_INPROGRESS 20
+#define KVM_REQ_EPR_EXIT          21
 
 #define KVM_USERSPACE_IRQ_SOURCE_ID		0
 #define KVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID	1
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index e6e5d4b..d1fca03 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -168,6 +168,7 @@ struct kvm_pit_config {
 #define KVM_EXIT_PAPR_HCALL	  19
 #define KVM_EXIT_S390_UCONTROL	  20
 #define KVM_EXIT_WATCHDOG         21
+#define KVM_EXIT_EPR              22
 
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
@@ -285,6 +286,10 @@ struct kvm_run {
 			__u64 ret;
 			__u64 args[9];
 		} papr_hcall;
+		/* KVM_EXIT_EPR */
+		struct {
+			__u32 epr;
+		} epr;
 		/* Fix the size of the union. */
 		char padding[256];
 	};
@@ -635,6 +640,7 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_IRQFD_RESAMPLE 82
 #define KVM_CAP_PPC_BOOKE_WATCHDOG 83
 #define KVM_CAP_PPC_HTAB_FD 84
+#define KVM_CAP_PPC_EPR 85
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
-- 
1.6.0.2