[PATCH 0/2] Allow to use KVM without in-kernel irqchip

[PATCH 0/2] Allow to use KVM without in-kernel irqchip

[Pavel Fedin]

This patch set brings back functionality which was broken in v4.1. The
overall goal is to eventually enable using virtual timer too, but for now
changes affect only a possibility to run KVM itself. The guest currently
has to use another timer because there's no API to propagate interrupts
from in-kernel virtual timer to the interrupt controller emulated in
userspace.

Pavel Fedin (2):
  Fix NULL pointer dereferences if KVM is used without in-kernel irqchip
  Detect vGIC presence at runtime

 arch/arm/kvm/arm.c  | 19 ++++++++++++++++++-
 virt/kvm/arm/vgic.c |  5 ++++-
 2 files changed, 22 insertions(+), 2 deletions(-)

-- 
2.4.4

[PATCH 1/2] Fix NULL pointer dereferences if KVM is used without in-kernel irqchip

[Pavel Fedin]

Makes qemu working again with kernel-irqchip=off option

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
---
 virt/kvm/arm/vgic.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 78fb820..3420657 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -100,6 +100,9 @@ static bool queue_sgi(struct kvm_vcpu *vcpu, int irq)
 
 int kvm_vgic_map_resources(struct kvm *kvm)
 {
+	if (!kvm->arch.vgic.vm_ops.map_resources)
+		return 0;
+
 	return kvm->arch.vgic.vm_ops.map_resources(kvm, vgic);
 }
 
@@ -1637,7 +1640,7 @@ static int vgic_vcpu_init_maps(struct kvm_vcpu *vcpu, int nr_irqs)
  */
 int kvm_vgic_get_max_vcpus(void)
 {
-	return vgic->max_gic_vcpus;
+	return vgic ? vgic->max_gic_vcpus : KVM_MAX_VCPUS;
 }
 
 void kvm_vgic_destroy(struct kvm *kvm)
-- 
2.4.4

Re: [PATCH 1/2] Fix NULL pointer dereferences if KVM is used without in-kernel irqchip

[Christoffer Dall]

On Tue, Jul 07, 2015 at 01:38:48PM +0300, Pavel Fedin wrote:
> Makes qemu working again with kernel-irqchip=off option
> 
> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> ---
>  virt/kvm/arm/vgic.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 78fb820..3420657 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -100,6 +100,9 @@ static bool queue_sgi(struct kvm_vcpu *vcpu, int irq)
>  
>  int kvm_vgic_map_resources(struct kvm *kvm)
>  {
> +	if (!kvm->arch.vgic.vm_ops.map_resources)
> +		return 0;
> +
>  	return kvm->arch.vgic.vm_ops.map_resources(kvm, vgic);
>  }
>  
> @@ -1637,7 +1640,7 @@ static int vgic_vcpu_init_maps(struct kvm_vcpu *vcpu, int nr_irqs)
>   */
>  int kvm_vgic_get_max_vcpus(void)
>  {
> -	return vgic->max_gic_vcpus;
> +	return vgic ? vgic->max_gic_vcpus : KVM_MAX_VCPUS;
>  }
>  
>  void kvm_vgic_destroy(struct kvm *kvm)
> -- 
> 2.4.4
> 

This feels like defensive programming and it's a bit weird to return
something from kvm_vgic_get_max_vcpus when there is no vgic...

I would probably guard the callers with irqchip_in_kernel instead.

-Christoffer

[PATCH 2/2] Detect vGIC presence at runtime

[Pavel Fedin]

Allows to use KVM on hardware without vGIC. Interrupt controller has to be
emulated in userspace in this case.

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
---
 arch/arm/kvm/arm.c | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index d9631ec..f2f7a13 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -61,6 +61,8 @@ static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1);
 static u8 kvm_next_vmid;
 static DEFINE_SPINLOCK(kvm_vmid_lock);
 
+static bool vgic_present;
+
 static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu)
 {
 	BUG_ON(preemptible());
@@ -172,6 +174,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	switch (ext) {
 	case KVM_CAP_IRQCHIP:
 	case KVM_CAP_IRQFD:
+		r = vgic_present;
+		break;
 	case KVM_CAP_IOEVENTFD:
 	case KVM_CAP_DEVICE_CTRL:
 	case KVM_CAP_USER_MEMORY:
@@ -850,6 +854,8 @@ static int kvm_vm_ioctl_set_device_addr(struct kvm *kvm,
 
 	switch (dev_id) {
 	case KVM_ARM_DEVICE_VGIC_V2:
+		if (!vgic_present)
+			return -ENXIO;
 		return kvm_vgic_addr(kvm, type, &dev_addr->addr, true);
 	default:
 		return -ENODEV;
@@ -864,6 +870,8 @@ long kvm_arch_vm_ioctl(struct file *filp,
 
 	switch (ioctl) {
 	case KVM_CREATE_IRQCHIP: {
+		if (!vgic_present)
+			return -ENXIO;
 		return kvm_vgic_create(kvm, KVM_DEV_TYPE_ARM_VGIC_V2);
 	}
 	case KVM_ARM_SET_DEVICE_ADDR: {
@@ -1046,8 +1054,17 @@ static int init_hyp_mode(void)
 	 * Init HYP view of VGIC
 	 */
 	err = kvm_vgic_hyp_init();
-	if (err)
+	switch (err) {
+	case 0:
+		vgic_present = true;
+		break;
+	case -ENODEV:
+	case -ENXIO:
+		vgic_present = false;
+		break;
+	default:
 		goto out_free_context;
+	}
 
 	/*
 	 * Init HYP architected timer support
-- 
2.4.4

Re: [PATCH 2/2] Detect vGIC presence at runtime

[Christoffer Dall]

On Tue, Jul 07, 2015 at 02:11:01PM +0300, Pavel Fedin wrote:
> Allows to use KVM on hardware without vGIC. Interrupt controller has to be
> emulated in userspace in this case.
> 
> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> ---
>  arch/arm/kvm/arm.c | 19 ++++++++++++++++++-
>  1 file changed, 18 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> index d9631ec..f2f7a13 100644
> --- a/arch/arm/kvm/arm.c
> +++ b/arch/arm/kvm/arm.c
> @@ -61,6 +61,8 @@ static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1);
>  static u8 kvm_next_vmid;
>  static DEFINE_SPINLOCK(kvm_vmid_lock);
>  
> +static bool vgic_present;
> +
>  static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu)
>  {
>  	BUG_ON(preemptible());
> @@ -172,6 +174,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
>  	switch (ext) {
>  	case KVM_CAP_IRQCHIP:
>  	case KVM_CAP_IRQFD:
> +		r = vgic_present;
> +		break;
>  	case KVM_CAP_IOEVENTFD:
>  	case KVM_CAP_DEVICE_CTRL:
>  	case KVM_CAP_USER_MEMORY:
> @@ -850,6 +854,8 @@ static int kvm_vm_ioctl_set_device_addr(struct kvm *kvm,
>  
>  	switch (dev_id) {
>  	case KVM_ARM_DEVICE_VGIC_V2:
> +		if (!vgic_present)
> +			return -ENXIO;
>  		return kvm_vgic_addr(kvm, type, &dev_addr->addr, true);
>  	default:
>  		return -ENODEV;
> @@ -864,6 +870,8 @@ long kvm_arch_vm_ioctl(struct file *filp,
>  
>  	switch (ioctl) {
>  	case KVM_CREATE_IRQCHIP: {
> +		if (!vgic_present)
> +			return -ENXIO;
>  		return kvm_vgic_create(kvm, KVM_DEV_TYPE_ARM_VGIC_V2);
>  	}
>  	case KVM_ARM_SET_DEVICE_ADDR: {
> @@ -1046,8 +1054,17 @@ static int init_hyp_mode(void)
>  	 * Init HYP view of VGIC
>  	 */
>  	err = kvm_vgic_hyp_init();
> -	if (err)
> +	switch (err) {
> +	case 0:
> +		vgic_present = true;
> +		break;
> +	case -ENODEV:
> +	case -ENXIO:
> +		vgic_present = false;

why not report ENXIO as an error?  If probing the vgic fails due to
being unable to request the irq or something similar, then surely your
system has and error and this should be reported.

This may be more nicely implemented by letting the vgic init/probe
functions set the vgic_present, or maybe better yet, just export a
function from vgic.c:

bool kvm_vgic_present(void)
{
	return vgic_ops != NULL;
}

> +		break;
> +	default:
>  		goto out_free_context;
> +	}
>  
>  	/*
>  	 * Init HYP architected timer support
> -- 
> 2.4.4
> 

-Christoffer

RE: [PATCH 2/2] Detect vGIC presence at runtime

[Pavel Fedin]

 Hello!

> why not report ENXIO as an error?  If probing the vgic fails due to
> being unable to request the irq or something similar, then surely your
> system has and error and this should be reported.

 It is reported by probe function itself.
 -ENODEV here means there's no GIC at all. -ENXIO happens when, for example, there is GIC node in
the device tree, but it does not specify vGIC resources. Normally this means that vGIC is defunct on
the machine.

> This may be more nicely implemented by letting the vgic init/probe
> functions set the vgic_present, or maybe better yet, just export a
> function from vgic.c:
> 
> bool kvm_vgic_present(void)
> {
> 	return vgic_ops != NULL;
> }

 Is it necessary? Actually this flag is not needed anywhere else except arch/arm/kvm/arm.c, only at
init time. Runtime should, i believe, use irqchip_in_kernel(), because userland can choose just not
to use vGIC for some reason (testing for example).

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

Re: [PATCH 2/2] Detect vGIC presence at runtime

[Christoffer Dall]

On Thu, Jul 09, 2015 at 03:50:49PM +0300, Pavel Fedin wrote:
>  Hello!
> 
> > why not report ENXIO as an error?  If probing the vgic fails due to
> > being unable to request the irq or something similar, then surely your
> > system has and error and this should be reported.
> 
>  It is reported by probe function itself.
>  -ENODEV here means there's no GIC at all. -ENXIO happens when, for example, there is GIC node in
> the device tree, but it does not specify vGIC resources. Normally this means that vGIC is defunct on
> the machine.

I'd like to distinguish between the 'missing vgic' and 'something bad
happened when trying to initialize the vgic' cases, which I don't think
we do currently, because the ENXIO code is used in various situations.

> 
> > This may be more nicely implemented by letting the vgic init/probe
> > functions set the vgic_present, or maybe better yet, just export a
> > function from vgic.c:
> > 
> > bool kvm_vgic_present(void)
> > {
> > 	return vgic_ops != NULL;
> > }
> 
>  Is it necessary? Actually this flag is not needed anywhere else except arch/arm/kvm/arm.c, only at
> init time. Runtime should, i believe, use irqchip_in_kernel(), because userland can choose just not
> to use vGIC for some reason (testing for example).
> 
I feel the init flow is relatively difficult to follow and adding a
bunch of flags here and there doesn't seem to help.  By adding a
function with a proper comment, it should be more clear, and I don't
like the switch statement on the error return values.

-Christoffer

RE: [PATCH 2/2] Detect vGIC presence at runtime

[Pavel Fedin]

 Hello!

> I'd like to distinguish between the 'missing vgic' and 'something bad
> happened when trying to initialize the vgic' cases, which I don't think
> we do currently, because the ENXIO code is used in various situations.

 It is done. Check, for example, vgic_v2_probe(). -ENXIO is returned when some of resources are
either missing from DT specification or wrongly given (not page-aligned). In the rest of cases error
code is taken from underlying functions, which are more supposed to return things like -EINVAL or
-ENOMEM.
 If you are doubtful, i could suggest to replace -EINVAL with -ENODEV in cases where resources are
not present. This would give more clear indication of "we don't have vGIC" condition.

> I feel the init flow is relatively difficult to follow and adding a
> bunch of flags here and there doesn't seem to help.  By adding a
> function with a proper comment, it should be more clear, and I don't
> like the switch statement on the error return values.

 Well, the alternative is:
 1. If GIC node or vGIC resources are not present, return 0 instead of error code.
 2. Implement a function which you suggest.
 Then:
 1. Situation with missing vGIC is just considered to be normal; it's not a to-be-ignored error any
more.
 2. "vGIC present" situation is automatically determined by vgic_ops != NULL; this means that probe
function completely worked and vGIC implementation has been chosen.

 Agree?

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia


> -----Original Message-----
> From: Christoffer Dall [mailto:christoffer.dall@linaro.org]
> Sent: Thursday, July 09, 2015 4:25 PM
> To: Pavel Fedin
> Cc: kvmarm@lists.cs.columbia.edu; kvm@vger.kernel.org; 'Marc Zyngier'
> Subject: Re: [PATCH 2/2] Detect vGIC presence at runtime
> 
> On Thu, Jul 09, 2015 at 03:50:49PM +0300, Pavel Fedin wrote:
> >  Hello!
> >
> > > why not report ENXIO as an error?  If probing the vgic fails due to
> > > being unable to request the irq or something similar, then surely your
> > > system has and error and this should be reported.
> >
> >  It is reported by probe function itself.
> >  -ENODEV here means there's no GIC at all. -ENXIO happens when, for example, there is GIC node
in
> > the device tree, but it does not specify vGIC resources. Normally this means that vGIC is
defunct on
> > the machine.
> 
> >
> > > This may be more nicely implemented by letting the vgic init/probe
> > > functions set the vgic_present, or maybe better yet, just export a
> > > function from vgic.c:
> > >
> > > bool kvm_vgic_present(void)
> > > {
> > > 	return vgic_ops != NULL;
> > > }
> >
> >  Is it necessary? Actually this flag is not needed anywhere else except arch/arm/kvm/arm.c, only
at
> > init time. Runtime should, i believe, use irqchip_in_kernel(), because userland can choose just
not
> > to use vGIC for some reason (testing for example).
> >
> I feel the init flow is relatively difficult to follow and adding a
> bunch of flags here and there doesn't seem to help.  By adding a
> function with a proper comment, it should be more clear, and I don't
> like the switch statement on the error return values.
> 
> -Christoffer