[PATCH 0/2] KVM: PPC: Support ioeventfd

[PATCH 0/2] KVM: PPC: Support ioeventfd

[Alexander Graf]

In order to support vhost, we need to be able to support ioeventfd.

This patch set adds support for ioeventfd to PPC and makes it possible to
do so without implementing irqfd along the way, as it requires an in-kernel
irqchip which we don't have yet.

Alex

Alexander Graf (2):
  KVM: Distangle eventfd code from irqchip
  KVM: PPC: Support eventfd

 arch/powerpc/kvm/Kconfig   |    1 +
 arch/powerpc/kvm/Makefile  |    4 +++-
 arch/powerpc/kvm/powerpc.c |   17 ++++++++++++++++-
 include/linux/kvm_host.h   |   12 +++++++++++-
 virt/kvm/eventfd.c         |    6 ++++++
 5 files changed, 37 insertions(+), 3 deletions(-)

[PATCH 1/2] KVM: Distangle eventfd code from irqchip

[Alexander Graf]

The current eventfd code assumes that when we have eventfd, we also have
irqfd for in-kernel interrupt delivery. This is not necessarily true. On
PPC we don't have an in-kernel irqchip yet, but we can still support easily
support eventfd.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 include/linux/kvm_host.h |   12 +++++++++++-
 virt/kvm/eventfd.c       |    6 ++++++
 2 files changed, 17 insertions(+), 1 deletions(-)

diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 6afc5be..f2f5880 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -884,10 +884,20 @@ static inline void kvm_free_irq_routing(struct kvm *kvm) {}
 #ifdef CONFIG_HAVE_KVM_EVENTFD
 
 void kvm_eventfd_init(struct kvm *kvm);
+int kvm_ioeventfd(struct kvm *kvm, struct kvm_ioeventfd *args);
+
+#ifdef CONFIG_HAVE_KVM_IRQCHIP
 int kvm_irqfd(struct kvm *kvm, struct kvm_irqfd *args);
 void kvm_irqfd_release(struct kvm *kvm);
 void kvm_irq_routing_update(struct kvm *, struct kvm_irq_routing_table *);
-int kvm_ioeventfd(struct kvm *kvm, struct kvm_ioeventfd *args);
+#else
+static inline int kvm_irqfd(struct kvm *kvm, struct kvm_irqfd *args)
+{
+	return -EINVAL;
+}
+
+static inline void kvm_irqfd_release(struct kvm *kvm) {}
+#endif
 
 #else
 
diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
index 9718e98..d7424c8 100644
--- a/virt/kvm/eventfd.c
+++ b/virt/kvm/eventfd.c
@@ -35,6 +35,7 @@
 
 #include "iodev.h"
 
+#ifdef __KVM_HAVE_IOAPIC
 /*
  * --------------------------------------------------------------------
  * irqfd: Allows an fd to be used to inject an interrupt to the guest
@@ -425,17 +426,21 @@ fail:
 	kfree(irqfd);
 	return ret;
 }
+#endif
 
 void
 kvm_eventfd_init(struct kvm *kvm)
 {
+#ifdef __KVM_HAVE_IOAPIC
 	spin_lock_init(&kvm->irqfds.lock);
 	INIT_LIST_HEAD(&kvm->irqfds.items);
 	INIT_LIST_HEAD(&kvm->irqfds.resampler_list);
 	mutex_init(&kvm->irqfds.resampler_lock);
+#endif
 	INIT_LIST_HEAD(&kvm->ioeventfds);
 }
 
+#ifdef __KVM_HAVE_IOAPIC
 /*
  * shutdown any irqfd's that match fd+gsi
  */
@@ -555,6 +560,7 @@ static void __exit irqfd_module_exit(void)
 
 module_init(irqfd_module_init);
 module_exit(irqfd_module_exit);
+#endif
 
 /*
  * --------------------------------------------------------------------
-- 
1.6.0.2

Re: [PATCH 1/2] KVM: Distangle eventfd code from irqchip

[Avi Kivity]

On 10/15/2012 02:02 PM, Alexander Graf wrote:
> The current eventfd code assumes that when we have eventfd, we also have
> irqfd for in-kernel interrupt delivery. This is not necessarily true. On
> PPC we don't have an in-kernel irqchip yet, but we can still support easily
> support eventfd.
> 

Don't you need, in addition, to reject PIO space for ioeventfd?


-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 1/2] KVM: Distangle eventfd code from irqchip

[Alexander Graf]

On 16.10.2012, at 12:57, Avi Kivity wrote:

> On 10/15/2012 02:02 PM, Alexander Graf wrote:
>> The current eventfd code assumes that when we have eventfd, we also have
>> irqfd for in-kernel interrupt delivery. This is not necessarily true. On
>> PPC we don't have an in-kernel irqchip yet, but we can still support easily
>> support eventfd.
>> 
> 
> Don't you need, in addition, to reject PIO space for ioeventfd?

Yeah, we could tell user space that it's doing something stupid. Not sure how critical it is - right now it would just register it and never get triggered, because the bus doesn't exist. But sanity checks are a good thing usually.


Alex

[PATCH 2/2] KVM: PPC: Support eventfd

[Alexander Graf]

In order to support the generic eventfd infrastructure on PPC, we need
to call into the generic KVM in-kernel device mmio code.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 arch/powerpc/kvm/Kconfig   |    1 +
 arch/powerpc/kvm/Makefile  |    4 +++-
 arch/powerpc/kvm/powerpc.c |   17 ++++++++++++++++-
 3 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
index 71f0cd9..4730c95 100644
--- a/arch/powerpc/kvm/Kconfig
+++ b/arch/powerpc/kvm/Kconfig
@@ -20,6 +20,7 @@ config KVM
 	bool
 	select PREEMPT_NOTIFIERS
 	select ANON_INODES
+	select HAVE_KVM_EVENTFD
 
 config KVM_BOOK3S_HANDLER
 	bool
diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
index c2a0863..cd89658 100644
--- a/arch/powerpc/kvm/Makefile
+++ b/arch/powerpc/kvm/Makefile
@@ -6,7 +6,8 @@ subdir-ccflags-$(CONFIG_PPC_WERROR) := -Werror
 
 ccflags-y := -Ivirt/kvm -Iarch/powerpc/kvm
 
-common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o)
+common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o \
+						eventfd.o)
 
 CFLAGS_44x_tlb.o  := -I.
 CFLAGS_e500_tlb.o := -I.
@@ -76,6 +77,7 @@ kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HV) := \
 
 kvm-book3s_64-module-objs := \
 	../../../virt/kvm/kvm_main.o \
+	../../../virt/kvm/eventfd.o \
 	powerpc.o \
 	emulate.o \
 	book3s.o \
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index deb0d59..900d8fc 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -314,6 +314,7 @@ int kvm_dev_ioctl_check_extension(long ext)
 	case KVM_CAP_PPC_IRQ_LEVEL:
 	case KVM_CAP_ENABLE_CAP:
 	case KVM_CAP_ONE_REG:
+	case KVM_CAP_IOEVENTFD:
 		r = 1;
 		break;
 #ifndef CONFIG_KVM_BOOK3S_64_HV
@@ -613,6 +614,13 @@ int kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu *vcpu,
 	vcpu->mmio_is_write = 0;
 	vcpu->arch.mmio_sign_extend = 0;
 
+	if (!kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+			     bytes, &run->mmio.data)) {
+		kvmppc_complete_mmio_load(vcpu, run);
+		vcpu->mmio_needed = 0;
+		return EMULATE_DONE;
+	}
+
 	return EMULATE_DO_MMIO;
 }
 
@@ -622,8 +630,8 @@ int kvmppc_handle_loads(struct kvm_run *run, struct kvm_vcpu *vcpu,
 {
 	int r;
 
-	r = kvmppc_handle_load(run, vcpu, rt, bytes, is_bigendian);
 	vcpu->arch.mmio_sign_extend = 1;
+	r = kvmppc_handle_load(run, vcpu, rt, bytes, is_bigendian);
 
 	return r;
 }
@@ -661,6 +669,13 @@ int kvmppc_handle_store(struct kvm_run *run, struct kvm_vcpu *vcpu,
 		}
 	}
 
+	if (!kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+			      bytes, &run->mmio.data)) {
+		kvmppc_complete_mmio_load(vcpu, run);
+		vcpu->mmio_needed = 0;
+		return EMULATE_DONE;
+	}
+
 	return EMULATE_DO_MMIO;
 }
 
-- 
1.6.0.2

Re: [PATCH 0/2] KVM: PPC: Support ioeventfd

[Avi Kivity]

On 10/15/2012 02:02 PM, Alexander Graf wrote:
> In order to support vhost, we need to be able to support ioeventfd.
> 
> This patch set adds support for ioeventfd to PPC and makes it possible to
> do so without implementing irqfd along the way, as it requires an in-kernel
> irqchip which we don't have yet.

It's not strictly required.  You have an interrupt line leading to the
core, no?  You could have your irqfd trigger that.


-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 0/2] KVM: PPC: Support ioeventfd

[Alexander Graf]

On 16.10.2012, at 12:56, Avi Kivity wrote:

> On 10/15/2012 02:02 PM, Alexander Graf wrote:
>> In order to support vhost, we need to be able to support ioeventfd.
>> 
>> This patch set adds support for ioeventfd to PPC and makes it possible to
>> do so without implementing irqfd along the way, as it requires an in-kernel
>> irqchip which we don't have yet.
> 
> It's not strictly required.  You have an interrupt line leading to the
> core, no?  You could have your irqfd trigger that.

The irqfd code in KVM directly triggers the in-kernel irqchip. That's what this patch set is cleaning up: enable you to build ioeventfd support without in-kernel irqchip irqfd support.

Vhost can work without in-kernel irqchip too, by listening to an eventfd (iiuc) in user space. That path works just fine with these patches applied.


Alex

Re: [PATCH 0/2] KVM: PPC: Support ioeventfd

[Avi Kivity]

On 10/16/2012 12:59 PM, Alexander Graf wrote:
> 
> On 16.10.2012, at 12:56, Avi Kivity wrote:
> 
>> On 10/15/2012 02:02 PM, Alexander Graf wrote:
>>> In order to support vhost, we need to be able to support ioeventfd.
>>> 
>>> This patch set adds support for ioeventfd to PPC and makes it possible to
>>> do so without implementing irqfd along the way, as it requires an in-kernel
>>> irqchip which we don't have yet.
>> 
>> It's not strictly required.  You have an interrupt line leading to the
>> core, no?  You could have your irqfd trigger that.
> 
> The irqfd code in KVM directly triggers the in-kernel irqchip. That's what this patch set is cleaning up: enable you to build ioeventfd support without in-kernel irqchip irqfd support.
> 
> Vhost can work without in-kernel irqchip too, by listening to an eventfd (iiuc) in user space. That path works just fine with these patches applied.

That's all true but it wasn't my point.  I was asking whether you can
enable irqfd without in-kernel irqchip.  If your irq input is edge
triggered then it's trivial.  If it's level triggered then you can use
the new resampling irqfd.


-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 0/2] KVM: PPC: Support ioeventfd

[Alexander Graf]

On 16.10.2012, at 13:01, Avi Kivity wrote:

> On 10/16/2012 12:59 PM, Alexander Graf wrote:
>> 
>> On 16.10.2012, at 12:56, Avi Kivity wrote:
>> 
>>> On 10/15/2012 02:02 PM, Alexander Graf wrote:
>>>> In order to support vhost, we need to be able to support ioeventfd.
>>>> 
>>>> This patch set adds support for ioeventfd to PPC and makes it possible to
>>>> do so without implementing irqfd along the way, as it requires an in-kernel
>>>> irqchip which we don't have yet.
>>> 
>>> It's not strictly required.  You have an interrupt line leading to the
>>> core, no?  You could have your irqfd trigger that.
>> 
>> The irqfd code in KVM directly triggers the in-kernel irqchip. That's what this patch set is cleaning up: enable you to build ioeventfd support without in-kernel irqchip irqfd support.
>> 
>> Vhost can work without in-kernel irqchip too, by listening to an eventfd (iiuc) in user space. That path works just fine with these patches applied.
> 
> That's all true but it wasn't my point.  I was asking whether you can
> enable irqfd without in-kernel irqchip.  If your irq input is edge
> triggered then it's trivial.  If it's level triggered then you can use
> the new resampling irqfd.

I'm not sure I fully grasp what you're trying to say :).

We have a single interrupt line on the core. So whenever any external interrupt gets injected (same thing for MSI), we need to go to the MPIC / XICS and ask it which line is active.

So yes, we could create a direct fd channel between vhost and the user space MPIC, but it wouldn't buy us anything. The interrupt injection path would be as long as it is with the current mechanism.


Alex

Re: [PATCH 0/2] KVM: PPC: Support ioeventfd

[Avi Kivity]

On 10/16/2012 01:06 PM, Alexander Graf wrote:
> 
> On 16.10.2012, at 13:01, Avi Kivity wrote:
> 
>> On 10/16/2012 12:59 PM, Alexander Graf wrote:
>>> 
>>> On 16.10.2012, at 12:56, Avi Kivity wrote:
>>> 
>>>> On 10/15/2012 02:02 PM, Alexander Graf wrote:
>>>>> In order to support vhost, we need to be able to support ioeventfd.
>>>>> 
>>>>> This patch set adds support for ioeventfd to PPC and makes it possible to
>>>>> do so without implementing irqfd along the way, as it requires an in-kernel
>>>>> irqchip which we don't have yet.
>>>> 
>>>> It's not strictly required.  You have an interrupt line leading to the
>>>> core, no?  You could have your irqfd trigger that.
>>> 
>>> The irqfd code in KVM directly triggers the in-kernel irqchip. That's what this patch set is cleaning up: enable you to build ioeventfd support without in-kernel irqchip irqfd support.
>>> 
>>> Vhost can work without in-kernel irqchip too, by listening to an eventfd (iiuc) in user space. That path works just fine with these patches applied.
>> 
>> That's all true but it wasn't my point.  I was asking whether you can
>> enable irqfd without in-kernel irqchip.  If your irq input is edge
>> triggered then it's trivial.  If it's level triggered then you can use
>> the new resampling irqfd.
> 
> I'm not sure I fully grasp what you're trying to say :).
> 
> We have a single interrupt line on the core. So whenever any external interrupt gets injected (same thing for MSI), we need to go to the MPIC / XICS and ask it which line is active.

Couldn't you attach that payload to the irqfd?  On x86 an irqfd is
associated with a gsi, and a gsi with extra information, including all
that is needed to queue an MSI.

> 
> So yes, we could create a direct fd channel between vhost and the user space MPIC, but it wouldn't buy us anything. The interrupt injection path would be as long as it is with the current mechanism.
> 

If there is a lot of prioritization and/or queuing logic, then yes.  But
what about MSI?  Doesn't that have a direct path?

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 0/2] KVM: PPC: Support ioeventfd

[Alexander Graf]

On 10/16/2012 03:47 PM, Avi Kivity wrote:
> On 10/16/2012 01:06 PM, Alexander Graf wrote:
>> On 16.10.2012, at 13:01, Avi Kivity wrote:
>>
>>> On 10/16/2012 12:59 PM, Alexander Graf wrote:
>>>> On 16.10.2012, at 12:56, Avi Kivity wrote:
>>>>
>>>>> On 10/15/2012 02:02 PM, Alexander Graf wrote:
>>>>>> In order to support vhost, we need to be able to support ioeventfd.
>>>>>>
>>>>>> This patch set adds support for ioeventfd to PPC and makes it possible to
>>>>>> do so without implementing irqfd along the way, as it requires an in-kernel
>>>>>> irqchip which we don't have yet.
>>>>> It's not strictly required.  You have an interrupt line leading to the
>>>>> core, no?  You could have your irqfd trigger that.
>>>> The irqfd code in KVM directly triggers the in-kernel irqchip. That's what this patch set is cleaning up: enable you to build ioeventfd support without in-kernel irqchip irqfd support.
>>>>
>>>> Vhost can work without in-kernel irqchip too, by listening to an eventfd (iiuc) in user space. That path works just fine with these patches applied.
>>> That's all true but it wasn't my point.  I was asking whether you can
>>> enable irqfd without in-kernel irqchip.  If your irq input is edge
>>> triggered then it's trivial.  If it's level triggered then you can use
>>> the new resampling irqfd.
>> I'm not sure I fully grasp what you're trying to say :).
>>
>> We have a single interrupt line on the core. So whenever any external interrupt gets injected (same thing for MSI), we need to go to the MPIC / XICS and ask it which line is active.
> Couldn't you attach that payload to the irqfd?  On x86 an irqfd is
> associated with a gsi, and a gsi with extra information, including all
> that is needed to queue an MSI.
>
>> So yes, we could create a direct fd channel between vhost and the user space MPIC, but it wouldn't buy us anything. The interrupt injection path would be as long as it is with the current mechanism.
>>
> If there is a lot of prioritization and/or queuing logic, then yes.  But
> what about MSI?  Doesn't that have a direct path?

Nope. Well, yes, in a certain special case where the MPIC pushes the 
interrupt vector on interrupt delivery into a special register. But not 
for the "normal" case.


Alex

Re: [PATCH 0/2] KVM: PPC: Support ioeventfd

[Avi Kivity]

On 10/16/2012 04:49 PM, Alexander Graf wrote:

>> If there is a lot of prioritization and/or queuing logic, then yes.  But
>> what about MSI?  Doesn't that have a direct path?
> 
> Nope. Well, yes, in a certain special case where the MPIC pushes the
> interrupt vector on interrupt delivery into a special register. But not
> for the "normal" case.

Ok.  The patches are fine then, but would be good to add the PIO check.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 0/2] KVM: PPC: Support ioeventfd

[Alexander Graf]

On 10/17/2012 04:50 PM, Avi Kivity wrote:
> On 10/16/2012 04:49 PM, Alexander Graf wrote:
>
>>> If there is a lot of prioritization and/or queuing logic, then yes.  But
>>> what about MSI?  Doesn't that have a direct path?
>> Nope. Well, yes, in a certain special case where the MPIC pushes the
>> interrupt vector on interrupt delivery into a special register. But not
>> for the "normal" case.
> Ok.  The patches are fine then, but would be good to add the PIO check.

Yup, will do as a separate patch.


Alex