[Qemu-devel] [PATCH 0/2] arm_gic: Initialize VGIC on machine init done

[Qemu-devel] [PATCH 0/2] arm_gic: Initialize VGIC on machine init done

[Eric Auger]

With current implementation the vgic is initialized very late,
when the VM is started, on the first VCPU run execution. This
series kicks the initialization off on machine init done instead.

Among other things, this earlier VGIC initialization allows to
have a simpler handling of VFIO. VFIO platform signaling can be
setup on reset handler, making sure the VGIC is ready, at that time,
to accept injected virtual IRQs.

Depends on the following kernel patch:
- [PATCH] KVM: arm/arm64: vgic: add init entry to VGIC KVM device

Tested on Calxeda Miday with VFIO platform use case

Eric Auger (2):
  linux-headers: Update KVM headers from linux-next tag ToBeFilled
  hw/intc/arm_gic: Initialize the vgic on machine init done

 hw/intc/arm_gic_kvm.c            | 21 +++++++++++++++++++++
 include/hw/intc/arm_gic_common.h |  1 +
 linux-headers/asm-arm/kvm.h      |  2 ++
 linux-headers/asm-arm64/kvm.h    |  2 ++
 4 files changed, 26 insertions(+)

-- 
1.8.3.2

[Qemu-devel] [PATCH 1/2] linux-headers: Update KVM headers from linux-next tag ToBeFilled

[Eric Auger]

Syncup KVM related linux headers from linux-next tree using
scripts/update-linux-headers.sh.

Add a new group/attribute in VGIC KVM device enabling to force
vgic init: KVM_DEV_ARM_VGIC_GRP_CTRL/KVM_DEV_ARM_VGIC_CTRL_INIT

Signed-off-by: Eric Auger <eric.auger@linaro.org>
---
 linux-headers/asm-arm/kvm.h   | 2 ++
 linux-headers/asm-arm64/kvm.h | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/linux-headers/asm-arm/kvm.h b/linux-headers/asm-arm/kvm.h
index 09ee408..0db25bc 100644
--- a/linux-headers/asm-arm/kvm.h
+++ b/linux-headers/asm-arm/kvm.h
@@ -175,6 +175,8 @@ struct kvm_arch_memory_slot {
 #define   KVM_DEV_ARM_VGIC_OFFSET_SHIFT	0
 #define   KVM_DEV_ARM_VGIC_OFFSET_MASK	(0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
 #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS	3
+#define KVM_DEV_ARM_VGIC_GRP_CTRL       4
+#define   KVM_DEV_ARM_VGIC_CTRL_INIT    0
 
 /* KVM_IRQ_LINE irq field index values */
 #define KVM_ARM_IRQ_TYPE_SHIFT		24
diff --git a/linux-headers/asm-arm64/kvm.h b/linux-headers/asm-arm64/kvm.h
index 8e38878..cbce625 100644
--- a/linux-headers/asm-arm64/kvm.h
+++ b/linux-headers/asm-arm64/kvm.h
@@ -161,6 +161,8 @@ struct kvm_arch_memory_slot {
 #define   KVM_DEV_ARM_VGIC_OFFSET_SHIFT	0
 #define   KVM_DEV_ARM_VGIC_OFFSET_MASK	(0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
 #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS	3
+#define KVM_DEV_ARM_VGIC_GRP_CTRL       4
+#define   KVM_DEV_ARM_VGIC_CTRL_INIT    0
 
 /* KVM_IRQ_LINE irq field index values */
 #define KVM_ARM_IRQ_TYPE_SHIFT		24
-- 
1.8.3.2

[Qemu-devel] [PATCH 2/2] hw/intc/arm_gic: Initialize the vgic on machine init done

[Eric Auger]

With current implementation the vgic is instantiated very late,
when the VM is started, on the first VCPU run execution. This patch
forces its initialization in a machine init done notifier. This
latter is registered in the vgic realize function. The notifier uses
a new group/attribute that allows such operation. In practice, vgic
initialization can be done as soon as those dimensioning parameters
are known:
- the number of IRQs,
- the number of vcpus
- start address of dist and CPU interface (set in a memory listener).

This earlier initialization allows, for example, to setup VFIO
signaling after vgic initialization, on a reset notifier.

Signed-off-by: Eric Auger <eric.auger@linaro.org>
---
 hw/intc/arm_gic_kvm.c            | 21 +++++++++++++++++++++
 include/hw/intc/arm_gic_common.h |  1 +
 2 files changed, 22 insertions(+)

diff --git a/hw/intc/arm_gic_kvm.c b/hw/intc/arm_gic_kvm.c
index 5038885..24fdf98 100644
--- a/hw/intc/arm_gic_kvm.c
+++ b/hw/intc/arm_gic_kvm.c
@@ -23,6 +23,7 @@
 #include "sysemu/kvm.h"
 #include "kvm_arm.h"
 #include "gic_internal.h"
+#include "sysemu/sysemu.h"
 
 //#define DEBUG_GIC_KVM
 
@@ -511,6 +512,17 @@ static void kvm_arm_gic_reset(DeviceState *dev)
     kvm_arm_gic_put(s);
 }
 
+static void kvm_arm_vgic_init_notify(Notifier *notifier, void *data)
+{
+    GICState *s = container_of(notifier, GICState, init_notifier);
+    struct kvm_device_attr attr = {
+         .group = KVM_DEV_ARM_VGIC_GRP_CTRL,
+         .attr = KVM_DEV_ARM_VGIC_CTRL_INIT};
+    if (s->dev_fd >= 0) {
+        kvm_device_ioctl(s->dev_fd, KVM_SET_DEVICE_ATTR, &attr);
+    }
+}
+
 static void kvm_arm_gic_realize(DeviceState *dev, Error **errp)
 {
     int i;
@@ -553,6 +565,15 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp)
         return;
     }
 
+    /*
+     * Register a notifier that will request the VGIC initialization.
+     * It will be executed after the notifier registered by
+     * kvm_arm_register_device whose role is to set dist and cpu interface
+     * base address. Also IRQ and VCPU number is known.
+     */
+    s->init_notifier.notify = kvm_arm_vgic_init_notify;
+    qemu_add_machine_init_done_notifier(&s->init_notifier);
+
     /* Distributor */
     memory_region_init_reservation(&s->iomem, OBJECT(s),
                                    "kvm-gic_dist", 0x1000);
diff --git a/include/hw/intc/arm_gic_common.h b/include/hw/intc/arm_gic_common.h
index f6887ed..98653c9 100644
--- a/include/hw/intc/arm_gic_common.h
+++ b/include/hw/intc/arm_gic_common.h
@@ -105,6 +105,7 @@ typedef struct GICState {
     uint32_t num_irq;
     uint32_t revision;
     int dev_fd; /* kvm device fd if backed by kvm vgic support */
+    Notifier init_notifier; /* notifier used to init the vgic */
 } GICState;
 
 #define TYPE_ARM_GIC_COMMON "arm_gic_common"
-- 
1.8.3.2

Re: [Qemu-devel] [PATCH 1/2] linux-headers: Update KVM headers from linux-next tag ToBeFilled

[Christoffer Dall]

On Tue, Dec 02, 2014 at 06:28:13PM +0000, Eric Auger wrote:
> Syncup KVM related linux headers from linux-next tree using
> scripts/update-linux-headers.sh.
> 
> Add a new group/attribute in VGIC KVM device enabling to force
> vgic init: KVM_DEV_ARM_VGIC_GRP_CTRL/KVM_DEV_ARM_VGIC_CTRL_INIT
> 
> Signed-off-by: Eric Auger <eric.auger@linaro.org>

How can this be from linux-next?  I'm pretty sure we didn't merge this
patch yet.

Also, if you synced your headers from your local tree, I think that
generally makes such a QEMU patches series an RFC one.

-Christoffer

Re: [Qemu-devel] [PATCH 1/2] linux-headers: Update KVM headers from linux-next tag ToBeFilled

[Eric Auger]

On 12/03/2014 11:49 AM, Christoffer Dall wrote:
> On Tue, Dec 02, 2014 at 06:28:13PM +0000, Eric Auger wrote:
>> Syncup KVM related linux headers from linux-next tree using
>> scripts/update-linux-headers.sh.
>>
>> Add a new group/attribute in VGIC KVM device enabling to force
>> vgic init: KVM_DEV_ARM_VGIC_GRP_CTRL/KVM_DEV_ARM_VGIC_CTRL_INIT
>>
>> Signed-off-by: Eric Auger <eric.auger@linaro.org>
> 
> How can this be from linux-next?  I'm pretty sure we didn't merge this
> patch yet.

Hi Christoffer,

yes that's correct . I don't know how to format that kind of patch that
refers to linux headers (I mentioned in the title tag ToBeFilled). I
looked at other similar ones and I thought this was the wording that is
generally used. I can move it to RFC but it will stay as is until kernel
patches are upstreamed. In that case I should also move the QEMU VFIO
patch back to RFC due to forwarding stuff;-)

Best Regards

Eric
> 
> Also, if you synced your headers from your local tree, I think that
> generally makes such a QEMU patches series an RFC one.
> 
> -Christoffer
> 

Re: [Qemu-devel] [PATCH 1/2] linux-headers: Update KVM headers from linux-next tag ToBeFilled

[Eric Auger]

On 12/03/2014 12:03 PM, Christoffer Dall wrote:
> On Wed, Dec 03, 2014 at 11:53:58AM +0100, Eric Auger wrote:
>> On 12/03/2014 11:49 AM, Christoffer Dall wrote:
>>> On Tue, Dec 02, 2014 at 06:28:13PM +0000, Eric Auger wrote:
>>>> Syncup KVM related linux headers from linux-next tree using
>>>> scripts/update-linux-headers.sh.
>>>>
>>>> Add a new group/attribute in VGIC KVM device enabling to force
>>>> vgic init: KVM_DEV_ARM_VGIC_GRP_CTRL/KVM_DEV_ARM_VGIC_CTRL_INIT
>>>>
>>>> Signed-off-by: Eric Auger <eric.auger@linaro.org>
>>>
>>> How can this be from linux-next?  I'm pretty sure we didn't merge this
>>> patch yet.
>>
>> Hi Christoffer,
>>
>> yes that's correct . I don't know how to format that kind of patch that
>> refers to linux headers (I mentioned in the title tag ToBeFilled). I
>> looked at other similar ones and I thought this was the wording that is
>> generally used. I can move it to RFC but it will stay as is until kernel
>> patches are upstreamed. In that case I should also move the QEMU VFIO
>> patch back to RFC due to forwarding stuff;-)
>>
> This patch subject should just be something like this:
> 
> Subject: [RFC PATCH 1/2] linux-headers: Update KVM headers for KVM_DEV_ARM_VGIC_GRP_CTRL
> 
> And in the commit message you point to a personal git tree that contains
> the patches you're syncing against.
OK thanks

I will update accordingly

Eric
> 
> -Christoffer
> 

Re: [Qemu-devel] [PATCH 1/2] linux-headers: Update KVM headers from linux-next tag ToBeFilled

[Christoffer Dall]

On Wed, Dec 03, 2014 at 11:53:58AM +0100, Eric Auger wrote:
> On 12/03/2014 11:49 AM, Christoffer Dall wrote:
> > On Tue, Dec 02, 2014 at 06:28:13PM +0000, Eric Auger wrote:
> >> Syncup KVM related linux headers from linux-next tree using
> >> scripts/update-linux-headers.sh.
> >>
> >> Add a new group/attribute in VGIC KVM device enabling to force
> >> vgic init: KVM_DEV_ARM_VGIC_GRP_CTRL/KVM_DEV_ARM_VGIC_CTRL_INIT
> >>
> >> Signed-off-by: Eric Auger <eric.auger@linaro.org>
> > 
> > How can this be from linux-next?  I'm pretty sure we didn't merge this
> > patch yet.
> 
> Hi Christoffer,
> 
> yes that's correct . I don't know how to format that kind of patch that
> refers to linux headers (I mentioned in the title tag ToBeFilled). I
> looked at other similar ones and I thought this was the wording that is
> generally used. I can move it to RFC but it will stay as is until kernel
> patches are upstreamed. In that case I should also move the QEMU VFIO
> patch back to RFC due to forwarding stuff;-)
> 
This patch subject should just be something like this:

Subject: [RFC PATCH 1/2] linux-headers: Update KVM headers for KVM_DEV_ARM_VGIC_GRP_CTRL

And in the commit message you point to a personal git tree that contains
the patches you're syncing against.

-Christoffer