[Qemu-devel] [PULL 0/8] s390x: various patches

[Qemu-devel] [PULL 0/8] s390x: various patches

[Cornelia Huck]

The following changes since commit f8340b360b9bc29d48716ba8aca79df2b9544979:

  hw/ptimer: Do not artificially limit timers when using icount (2015-05-08 17:15:23 +1000)

are available in the git repository at:

  git://github.com/cohuck/qemu tags/s390x-20150508

for you to fetch changes up to 3cda44f7bae5c9feddc11630ba6eecb2e3bed425:

  s390x/kvm: migrate vcpu interrupt state (2015-05-08 10:36:19 +0200)

----------------------------------------------------------------
Assorted s390x patches:
- updates for virtio-ccw and s390-virtio, making them more similar
  to virtio-pci
- improvements regarding per-vcpu interrupts and migration

----------------------------------------------------------------

Christian Borntraeger (2):
  s390-virtio: Accommodate guests using virtqueues too early
  s390-virtio: clear {used,avail}_event_idx on reset as well

Cornelia Huck (3):
  s390-virtio: use common features
  virtio-ccw: change realization sequence
  virtio-ccw: implement ->device_plugged

David Hildenbrand (1):
  s390x: move fpu regs into a subsection of the vmstate

Jens Freimann (2):
  s390x/kvm: use ioctl KVM_S390_IRQ for vcpu interrupts
  s390x/kvm: migrate vcpu interrupt state

 hw/s390x/s390-virtio-bus.c | 28 ++++++++---------
 hw/s390x/s390-virtio.c     | 10 ++++++
 hw/s390x/virtio-ccw.c      | 72 +++++++++++++++++++++++--------------------
 target-s390x/cpu-qom.h     |  3 ++
 target-s390x/cpu.c         |  1 +
 target-s390x/cpu.h         |  9 ++++++
 target-s390x/kvm.c         | 77 ++++++++++++++++++++++++++++++++++++++++++++--
 target-s390x/machine.c     | 49 ++++++++++++++++++++++++-----
 8 files changed, 192 insertions(+), 57 deletions(-)

-- 
2.4.0

[Qemu-devel] [PULL 1/8] s390-virtio: Accommodate guests using virtqueues too early

[Cornelia Huck]

From: Christian Borntraeger <borntraeger@de.ibm.com>

Feature updates are not a synchronuous operation for the legacy
s390-virtio transport. This transport syncs the guest feature bits
(those from finalize) on the set_status hypercall. Before that qemu
thinks that features are zero, which means QEMU will misbehave, e.g.
it will not write the event index, even if the guest asks for it.

Let's detect the case where a kick happens before the driver is ready
and force sync the features.
With this workaround, it is now safe to switch to the common feature
bit handling code as used by all other transports.

Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 hw/s390x/s390-virtio.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/hw/s390x/s390-virtio.c b/hw/s390x/s390-virtio.c
index 3a1b9ee..59750db 100644
--- a/hw/s390x/s390-virtio.c
+++ b/hw/s390x/s390-virtio.c
@@ -77,6 +77,16 @@ static int s390_virtio_hcall_notify(const uint64_t *args)
     if (mem > ram_size) {
         VirtIOS390Device *dev = s390_virtio_bus_find_vring(s390_bus, mem, &i);
         if (dev) {
+            /*
+             * Older kernels will use the virtqueue before setting DRIVER_OK.
+             * In this case the feature bits are not yet up to date, meaning
+             * that several funny things can happen, e.g. the guest thinks
+             * EVENT_IDX is on and QEMU thinks it is off. Let's force a feature
+             * and status sync.
+             */
+            if (!(dev->vdev->status & VIRTIO_CONFIG_S_DRIVER_OK)) {
+                s390_virtio_device_update_status(dev);
+            }
             virtio_queue_notify(dev->vdev, i);
         } else {
             r = -EINVAL;
-- 
2.4.0

[Qemu-devel] [PULL 2/8] s390-virtio: use common features

[Cornelia Huck]

We used to avoid enabling event_idx for virtio-blk devices via
s390-virtio, but we now have a workaround in place for guests trying
to use the device before setting DRIVER_OK. Therefore, let's add
DEFINE_VIRTIO_COMMON_FEATURES to the base device so all devices get
those common features - and make s390-virtio use the same mechanism
as the other transports do.

Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Shannon Zhao <shannon.zhao@linaro.org>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 hw/s390x/s390-virtio-bus.c | 20 ++++++--------------
 1 file changed, 6 insertions(+), 14 deletions(-)

diff --git a/hw/s390x/s390-virtio-bus.c b/hw/s390x/s390-virtio-bus.c
index c27f8a5..4a33c6a 100644
--- a/hw/s390x/s390-virtio-bus.c
+++ b/hw/s390x/s390-virtio-bus.c
@@ -530,7 +530,6 @@ static unsigned virtio_s390_get_features(DeviceState *d)
 /**************** S390 Virtio Bus Device Descriptions *******************/
 
 static Property s390_virtio_net_properties[] = {
-    DEFINE_VIRTIO_COMMON_FEATURES(VirtIOS390Device, host_features),
     DEFINE_VIRTIO_NET_FEATURES(VirtIOS390Device, host_features),
     DEFINE_PROP_END_OF_LIST(),
 };
@@ -592,18 +591,12 @@ static const TypeInfo s390_virtio_serial = {
     .class_init    = s390_virtio_serial_class_init,
 };
 
-static Property s390_virtio_rng_properties[] = {
-    DEFINE_VIRTIO_COMMON_FEATURES(VirtIOS390Device, host_features),
-    DEFINE_PROP_END_OF_LIST(),
-};
-
 static void s390_virtio_rng_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
     VirtIOS390DeviceClass *k = VIRTIO_S390_DEVICE_CLASS(klass);
 
     k->realize = s390_virtio_rng_realize;
-    dc->props = s390_virtio_rng_properties;
     set_bit(DEVICE_CATEGORY_MISC, dc->categories);
 }
 
@@ -632,10 +625,16 @@ static void s390_virtio_busdev_reset(DeviceState *dev)
     virtio_reset(_dev->vdev);
 }
 
+static Property virtio_s390_properties[] = {
+    DEFINE_VIRTIO_COMMON_FEATURES(VirtIOS390Device, host_features),
+    DEFINE_PROP_END_OF_LIST(),
+};
+
 static void virtio_s390_device_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
 
+    dc->props = virtio_s390_properties;
     dc->realize = s390_virtio_busdev_realize;
     dc->bus_type = TYPE_S390_VIRTIO_BUS;
     dc->reset = s390_virtio_busdev_reset;
@@ -651,7 +650,6 @@ static const TypeInfo virtio_s390_device_info = {
 };
 
 static Property s390_virtio_scsi_properties[] = {
-    DEFINE_VIRTIO_COMMON_FEATURES(VirtIOS390Device, host_features),
     DEFINE_VIRTIO_SCSI_FEATURES(VirtIOS390Device, host_features),
     DEFINE_PROP_END_OF_LIST(),
 };
@@ -675,18 +673,12 @@ static const TypeInfo s390_virtio_scsi = {
 };
 
 #ifdef CONFIG_VHOST_SCSI
-static Property s390_vhost_scsi_properties[] = {
-    DEFINE_VIRTIO_COMMON_FEATURES(VirtIOS390Device, host_features),
-    DEFINE_PROP_END_OF_LIST(),
-};
-
 static void s390_vhost_scsi_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
     VirtIOS390DeviceClass *k = VIRTIO_S390_DEVICE_CLASS(klass);
 
     k->realize = s390_vhost_scsi_realize;
-    dc->props = s390_vhost_scsi_properties;
     set_bit(DEVICE_CATEGORY_STORAGE, dc->categories);
 }
 
-- 
2.4.0

[Qemu-devel] [PULL 3/8] s390-virtio: clear {used, avail}_event_idx on reset as well

[Cornelia Huck]

From: Christian Borntraeger <borntraeger@de.ibm.com>

The old s390-virtio transport clears the vring used/avail indices in
the shared area on reset. When we enabled event_idx for virtio-blk, we
noticed that this is not enough: We also need to clear the published
used/avail event indices, or reboot will fail.

Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 hw/s390x/s390-virtio-bus.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/hw/s390x/s390-virtio-bus.c b/hw/s390x/s390-virtio-bus.c
index 4a33c6a..4f69cbb 100644
--- a/hw/s390x/s390-virtio-bus.c
+++ b/hw/s390x/s390-virtio-bus.c
@@ -77,10 +77,18 @@ void s390_virtio_reset_idx(VirtIOS390Device *dev)
             VIRTIO_VRING_AVAIL_IDX_OFFS;
         address_space_stw(&address_space_memory, idx_addr, 0,
                           MEMTXATTRS_UNSPECIFIED, NULL);
+        idx_addr = virtio_queue_get_avail_addr(dev->vdev, i) +
+            virtio_queue_get_avail_size(dev->vdev, i);
+        address_space_stw(&address_space_memory, idx_addr, 0,
+                          MEMTXATTRS_UNSPECIFIED, NULL);
         idx_addr = virtio_queue_get_used_addr(dev->vdev, i) +
             VIRTIO_VRING_USED_IDX_OFFS;
         address_space_stw(&address_space_memory, idx_addr, 0,
                           MEMTXATTRS_UNSPECIFIED, NULL);
+        idx_addr = virtio_queue_get_used_addr(dev->vdev, i) +
+            virtio_queue_get_used_size(dev->vdev, i);
+        address_space_stw(&address_space_memory, idx_addr, 0,
+                          MEMTXATTRS_UNSPECIFIED, NULL);
     }
 }
 
-- 
2.4.0

[Qemu-devel] [PULL 4/8] virtio-ccw: change realization sequence

[Cornelia Huck]

virtio-ccw has an odd sequence of realizing devices: first the
device-specific relization (net, block, ...), then the generic
realization. It feels less odd to have the generic realization
callback trigger the device-specific realization instead (and this
also matches what virtio-pci does).

One thing to note: We need to defer initializing the cu model in the
sense id data until after the device-specific realization has been
performed, as we need to refer to the virtio device's device_id.

Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Message-Id: <1429627016-30656-2-git-send-email-cornelia.huck@de.ibm.com>
---
 hw/s390x/virtio-ccw.c | 41 +++++++++++++++++------------------------
 1 file changed, 17 insertions(+), 24 deletions(-)

diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index c1d8288..1c2bd9d 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -642,8 +642,7 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
     return ret;
 }
 
-static void virtio_ccw_device_realize(VirtioCcwDevice *dev,
-                                      VirtIODevice *vdev, Error **errp)
+static void virtio_ccw_device_realize(VirtioCcwDevice *dev, Error **errp)
 {
     unsigned int cssid = 0;
     unsigned int ssid = 0;
@@ -654,6 +653,9 @@ static void virtio_ccw_device_realize(VirtioCcwDevice *dev,
     SubchDev *sch;
     int num;
     DeviceState *parent = DEVICE(dev);
+    Error *err = NULL;
+    VirtIOCCWDeviceClass *k = VIRTIO_CCW_DEVICE_GET_CLASS(dev);
+    VirtIODevice *vdev;
 
     sch = g_malloc0(sizeof(SubchDev));
 
@@ -766,6 +768,18 @@ static void virtio_ccw_device_realize(VirtioCcwDevice *dev,
     memset(&sch->id, 0, sizeof(SenseId));
     sch->id.reserved = 0xff;
     sch->id.cu_type = VIRTIO_CCW_CU_TYPE;
+
+    if (k->realize) {
+        k->realize(dev, &err);
+    }
+    if (err) {
+        error_propagate(errp, err);
+        css_subch_assign(cssid, ssid, schid, devno, NULL);
+        goto out_err;
+    }
+
+    /* device_id is only set after vdev has been realized */
+    vdev = virtio_ccw_get_vdev(sch);
     sch->id.cu_model = vdev->device_id;
 
     /* Only the first 32 feature bits are used. */
@@ -813,10 +827,7 @@ static void virtio_ccw_net_realize(VirtioCcwDevice *ccw_dev, Error **errp)
     object_property_set_bool(OBJECT(vdev), true, "realized", &err);
     if (err) {
         error_propagate(errp, err);
-        return;
     }
-
-    virtio_ccw_device_realize(ccw_dev, VIRTIO_DEVICE(vdev), errp);
 }
 
 static void virtio_ccw_net_instance_init(Object *obj)
@@ -839,10 +850,7 @@ static void virtio_ccw_blk_realize(VirtioCcwDevice *ccw_dev, Error **errp)
     object_property_set_bool(OBJECT(vdev), true, "realized", &err);
     if (err) {
         error_propagate(errp, err);
-        return;
     }
-
-    virtio_ccw_device_realize(ccw_dev, VIRTIO_DEVICE(vdev), errp);
 }
 
 static void virtio_ccw_blk_instance_init(Object *obj)
@@ -879,10 +887,7 @@ static void virtio_ccw_serial_realize(VirtioCcwDevice *ccw_dev, Error **errp)
     object_property_set_bool(OBJECT(vdev), true, "realized", &err);
     if (err) {
         error_propagate(errp, err);
-        return;
     }
-
-    virtio_ccw_device_realize(ccw_dev, VIRTIO_DEVICE(vdev), errp);
 }
 
 
@@ -904,10 +909,7 @@ static void virtio_ccw_balloon_realize(VirtioCcwDevice *ccw_dev, Error **errp)
     object_property_set_bool(OBJECT(vdev), true, "realized", &err);
     if (err) {
         error_propagate(errp, err);
-        return;
     }
-
-    virtio_ccw_device_realize(ccw_dev, VIRTIO_DEVICE(vdev), errp);
 }
 
 static void balloon_ccw_stats_get_all(Object *obj, struct Visitor *v,
@@ -972,10 +974,7 @@ static void virtio_ccw_scsi_realize(VirtioCcwDevice *ccw_dev, Error **errp)
     object_property_set_bool(OBJECT(vdev), true, "realized", &err);
     if (err) {
         error_propagate(errp, err);
-        return;
     }
-
-    virtio_ccw_device_realize(ccw_dev, VIRTIO_DEVICE(vdev), errp);
 }
 
 static void virtio_ccw_scsi_instance_init(Object *obj)
@@ -999,10 +998,7 @@ static void vhost_ccw_scsi_realize(VirtioCcwDevice *ccw_dev, Error **errp)
     object_property_set_bool(OBJECT(vdev), true, "realized", &err);
     if (err) {
         error_propagate(errp, err);
-        return;
     }
-
-    virtio_ccw_device_realize(ccw_dev, VIRTIO_DEVICE(vdev), errp);
 }
 
 static void vhost_ccw_scsi_instance_init(Object *obj)
@@ -1030,8 +1026,6 @@ static void virtio_ccw_rng_realize(VirtioCcwDevice *ccw_dev, Error **errp)
     object_property_set_link(OBJECT(dev),
                              OBJECT(dev->vdev.conf.rng), "rng",
                              NULL);
-
-    virtio_ccw_device_realize(ccw_dev, VIRTIO_DEVICE(vdev), errp);
 }
 
 /* DeviceState to VirtioCcwDevice. Note: used on datapath,
@@ -1640,10 +1634,9 @@ static const TypeInfo virtio_ccw_rng = {
 static void virtio_ccw_busdev_realize(DeviceState *dev, Error **errp)
 {
     VirtioCcwDevice *_dev = (VirtioCcwDevice *)dev;
-    VirtIOCCWDeviceClass *_info = VIRTIO_CCW_DEVICE_GET_CLASS(dev);
 
     virtio_ccw_bus_new(&_dev->bus, sizeof(_dev->bus), _dev);
-    _info->realize(_dev, errp);
+    virtio_ccw_device_realize(_dev, errp);
 }
 
 static int virtio_ccw_busdev_exit(DeviceState *dev)
-- 
2.4.0

[Qemu-devel] [PULL 5/8] virtio-ccw: implement ->device_plugged

[Cornelia Huck]

Let's move operations that are only valid after the backend has been
realized to a ->device_plugged callback, just as virtio-pci does.
Also reorder setting up the host feature bits to the sequence used
by virtio-pci.

While we're at it, also add a ->device_unplugged callback to stop
ioeventfd, just to be on the safe side.

Reviewed-by: Shannon Zhao <shannon.zhao@linaro.org>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Message-Id: <1429627016-30656-3-git-send-email-cornelia.huck@de.ibm.com>
---
 hw/s390x/virtio-ccw.c | 41 ++++++++++++++++++++++++++---------------
 1 file changed, 26 insertions(+), 15 deletions(-)

diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index 1c2bd9d..430cc6f 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -652,10 +652,8 @@ static void virtio_ccw_device_realize(VirtioCcwDevice *dev, Error **errp)
     bool found = false;
     SubchDev *sch;
     int num;
-    DeviceState *parent = DEVICE(dev);
     Error *err = NULL;
     VirtIOCCWDeviceClass *k = VIRTIO_CCW_DEVICE_GET_CLASS(dev);
-    VirtIODevice *vdev;
 
     sch = g_malloc0(sizeof(SubchDev));
 
@@ -778,19 +776,6 @@ static void virtio_ccw_device_realize(VirtioCcwDevice *dev, Error **errp)
         goto out_err;
     }
 
-    /* device_id is only set after vdev has been realized */
-    vdev = virtio_ccw_get_vdev(sch);
-    sch->id.cu_model = vdev->device_id;
-
-    /* Only the first 32 feature bits are used. */
-    dev->host_features[0] = virtio_bus_get_vdev_features(&dev->bus,
-                                                         dev->host_features[0]);
-
-    virtio_add_feature(&dev->host_features[0], VIRTIO_F_NOTIFY_ON_EMPTY);
-    virtio_add_feature(&dev->host_features[0], VIRTIO_F_BAD_FEATURE);
-
-    css_generate_sch_crws(sch->cssid, sch->ssid, sch->schid,
-                          parent->hotplugged, 1);
     return;
 
 out_err:
@@ -1428,6 +1413,30 @@ static int virtio_ccw_load_config(DeviceState *d, QEMUFile *f)
     return 0;
 }
 
+/* This is called by virtio-bus just after the device is plugged. */
+static void virtio_ccw_device_plugged(DeviceState *d)
+{
+    VirtioCcwDevice *dev = VIRTIO_CCW_DEVICE(d);
+    SubchDev *sch = dev->sch;
+
+    sch->id.cu_model = virtio_bus_get_vdev_id(&dev->bus);
+
+    /* Only the first 32 feature bits are used. */
+    virtio_add_feature(&dev->host_features[0], VIRTIO_F_NOTIFY_ON_EMPTY);
+    virtio_add_feature(&dev->host_features[0], VIRTIO_F_BAD_FEATURE);
+    dev->host_features[0] = virtio_bus_get_vdev_features(&dev->bus,
+                                                         dev->host_features[0]);
+
+    css_generate_sch_crws(sch->cssid, sch->ssid, sch->schid,
+                          d->hotplugged, 1);
+}
+
+static void virtio_ccw_device_unplugged(DeviceState *d)
+{
+    VirtioCcwDevice *dev = VIRTIO_CCW_DEVICE(d);
+
+    virtio_ccw_stop_ioeventfd(dev);
+}
 /**************** Virtio-ccw Bus Device Descriptions *******************/
 
 static Property virtio_ccw_net_properties[] = {
@@ -1752,6 +1761,8 @@ static void virtio_ccw_bus_class_init(ObjectClass *klass, void *data)
     k->load_queue = virtio_ccw_load_queue;
     k->save_config = virtio_ccw_save_config;
     k->load_config = virtio_ccw_load_config;
+    k->device_plugged = virtio_ccw_device_plugged;
+    k->device_unplugged = virtio_ccw_device_unplugged;
 }
 
 static const TypeInfo virtio_ccw_bus_info = {
-- 
2.4.0

[Qemu-devel] [PULL 6/8] s390x/kvm: use ioctl KVM_S390_IRQ for vcpu interrupts

[Cornelia Huck]

From: Jens Freimann <jfrei@linux.vnet.ibm.com>

KVM_S390_INT uses only two parameter fields. This is not
enough to pass all required information for certain interrupts.

A new ioctl KVM_S390_IRQ is available which allows us to
inject all local interrupts as defined in the Principles of
Operation. It takes a struct kvm_s390_irq as a parameter
which can store interrupt payload data for all interrupts.

Let's use the new ioctl for injecting vcpu interrupts.

Tested-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Reviewed-by: David Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: Jens Freimann <jfrei@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 target-s390x/kvm.c | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c
index 8e65e43..43ad009 100644
--- a/target-s390x/kvm.c
+++ b/target-s390x/kvm.c
@@ -124,6 +124,7 @@ const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
 static int cap_sync_regs;
 static int cap_async_pf;
 static int cap_mem_op;
+static int cap_s390_irq;
 
 static void *legacy_s390_alloc(size_t size, uint64_t *align);
 
@@ -249,6 +250,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     cap_sync_regs = kvm_check_extension(s, KVM_CAP_SYNC_REGS);
     cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF);
     cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP);
+    cap_s390_irq = kvm_check_extension(s, KVM_CAP_S390_INJECT_IRQ);
 
     kvm_s390_enable_cmma(s);
 
@@ -827,10 +829,9 @@ static int s390_kvm_irq_to_interrupt(struct kvm_s390_irq *irq,
     return r;
 }
 
-void kvm_s390_vcpu_interrupt(S390CPU *cpu, struct kvm_s390_irq *irq)
+static void inject_vcpu_irq_legacy(CPUState *cs, struct kvm_s390_irq *irq)
 {
     struct kvm_s390_interrupt kvmint = {};
-    CPUState *cs = CPU(cpu);
     int r;
 
     r = s390_kvm_irq_to_interrupt(irq, &kvmint);
@@ -846,6 +847,23 @@ void kvm_s390_vcpu_interrupt(S390CPU *cpu, struct kvm_s390_irq *irq)
     }
 }
 
+void kvm_s390_vcpu_interrupt(S390CPU *cpu, struct kvm_s390_irq *irq)
+{
+    CPUState *cs = CPU(cpu);
+    int r;
+
+    if (cap_s390_irq) {
+        r = kvm_vcpu_ioctl(cs, KVM_S390_IRQ, irq);
+        if (!r) {
+            return;
+        }
+        error_report("KVM failed to inject interrupt %llx", irq->type);
+        exit(1);
+    }
+
+    inject_vcpu_irq_legacy(cs, irq);
+}
+
 static void __kvm_s390_floating_interrupt(struct kvm_s390_irq *irq)
 {
     struct kvm_s390_interrupt kvmint = {};
-- 
2.4.0

[Qemu-devel] [PULL 7/8] s390x: move fpu regs into a subsection of the vmstate

[Cornelia Huck]

From: David Hildenbrand <dahi@linux.vnet.ibm.com>

Let's move the floating point registers into a seperate subsection and
bump up the version id. This cleans up the current vmstate and will
allow for a future extension with vector registers in a compatible way.

This patch is based on a patch from Eric Farman.

Reviewed-by: Eric Farman <farman@linux.vnet.ibm.com>
Signed-off-by: David Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 target-s390x/machine.c | 36 +++++++++++++++++++++++++++++-------
 1 file changed, 29 insertions(+), 7 deletions(-)

diff --git a/target-s390x/machine.c b/target-s390x/machine.c
index bd4cea7..a034423 100644
--- a/target-s390x/machine.c
+++ b/target-s390x/machine.c
@@ -33,12 +33,11 @@ static int cpu_post_load(void *opaque, int version_id)
     return 0;
 }
 
-const VMStateDescription vmstate_s390_cpu = {
-    .name = "cpu",
-    .post_load = cpu_post_load,
-    .version_id = 2,
-    .minimum_version_id = 2,
-    .fields      = (VMStateField[]) {
+const VMStateDescription vmstate_fpu = {
+    .name = "cpu/fpu",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
         VMSTATE_UINT64(env.fregs[0].ll, S390CPU),
         VMSTATE_UINT64(env.fregs[1].ll, S390CPU),
         VMSTATE_UINT64(env.fregs[2].ll, S390CPU),
@@ -55,11 +54,26 @@ const VMStateDescription vmstate_s390_cpu = {
         VMSTATE_UINT64(env.fregs[13].ll, S390CPU),
         VMSTATE_UINT64(env.fregs[14].ll, S390CPU),
         VMSTATE_UINT64(env.fregs[15].ll, S390CPU),
+        VMSTATE_UINT32(env.fpc, S390CPU),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static inline bool fpu_needed(void *opaque)
+{
+    return true;
+}
+
+const VMStateDescription vmstate_s390_cpu = {
+    .name = "cpu",
+    .post_load = cpu_post_load,
+    .version_id = 3,
+    .minimum_version_id = 3,
+    .fields      = (VMStateField[]) {
         VMSTATE_UINT64_ARRAY(env.regs, S390CPU, 16),
         VMSTATE_UINT64(env.psw.mask, S390CPU),
         VMSTATE_UINT64(env.psw.addr, S390CPU),
         VMSTATE_UINT64(env.psa, S390CPU),
-        VMSTATE_UINT32(env.fpc, S390CPU),
         VMSTATE_UINT32(env.todpr, S390CPU),
         VMSTATE_UINT64(env.pfault_token, S390CPU),
         VMSTATE_UINT64(env.pfault_compare, S390CPU),
@@ -74,4 +88,12 @@ const VMStateDescription vmstate_s390_cpu = {
         VMSTATE_UINT8(env.sigp_order, S390CPU),
         VMSTATE_END_OF_LIST()
      },
+    .subsections = (VMStateSubsection[]) {
+        {
+            .vmsd = &vmstate_fpu,
+            .needed = fpu_needed,
+        } , {
+            /* empty */
+        }
+    },
 };
-- 
2.4.0

[Qemu-devel] [PULL 8/8] s390x/kvm: migrate vcpu interrupt state

[Cornelia Huck]

From: Jens Freimann <jfrei@linux.vnet.ibm.com>

This patch adds support to migrate vcpu interrupts.
We use ioctl KVM_S390_GET_IRQ_STATE and _SET_IRQ_STATE
to get/set the complete interrupt state for a vcpu.

Reviewed-by: David Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: Jens Freimann <jfrei@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 target-s390x/cpu-qom.h |  3 +++
 target-s390x/cpu.c     |  1 +
 target-s390x/cpu.h     |  9 +++++++++
 target-s390x/kvm.c     | 55 ++++++++++++++++++++++++++++++++++++++++++++++++++
 target-s390x/machine.c | 15 +++++++++++++-
 5 files changed, 82 insertions(+), 1 deletion(-)

diff --git a/target-s390x/cpu-qom.h b/target-s390x/cpu-qom.h
index 8b376df..936ae21 100644
--- a/target-s390x/cpu-qom.h
+++ b/target-s390x/cpu-qom.h
@@ -66,6 +66,9 @@ typedef struct S390CPU {
     /*< public >*/
 
     CPUS390XState env;
+    /* needed for live migration */
+    void *irqstate;
+    uint32_t irqstate_saved_size;
 } S390CPU;
 
 static inline S390CPU *s390_env_get_cpu(CPUS390XState *env)
diff --git a/target-s390x/cpu.c b/target-s390x/cpu.c
index e0537fa..d2f9836 100644
--- a/target-s390x/cpu.c
+++ b/target-s390x/cpu.c
@@ -213,6 +213,7 @@ static void s390_cpu_finalize(Object *obj)
     S390CPU *cpu = S390_CPU(obj);
 
     qemu_unregister_reset(s390_cpu_machine_reset_cb, cpu);
+    g_free(cpu->irqstate);
 #endif
 }
 
diff --git a/target-s390x/cpu.h b/target-s390x/cpu.h
index ba7d250..c557211 100644
--- a/target-s390x/cpu.h
+++ b/target-s390x/cpu.h
@@ -1079,6 +1079,8 @@ void kvm_s390_clear_cmma_callback(void *opaque);
 int kvm_s390_set_cpu_state(S390CPU *cpu, uint8_t cpu_state);
 void kvm_s390_reset_vcpu(S390CPU *cpu);
 int kvm_s390_set_mem_limit(KVMState *s, uint64_t new_limit, uint64_t *hw_limit);
+void kvm_s390_vcpu_interrupt_pre_save(S390CPU *cpu);
+int kvm_s390_vcpu_interrupt_post_load(S390CPU *cpu);
 #else
 static inline void kvm_s390_io_interrupt(uint16_t subchannel_id,
                                         uint16_t subchannel_nr,
@@ -1121,6 +1123,13 @@ static inline int kvm_s390_set_mem_limit(KVMState *s, uint64_t new_limit,
 {
     return 0;
 }
+static inline void kvm_s390_vcpu_interrupt_pre_save(S390CPU *cpu)
+{
+}
+static inline int kvm_s390_vcpu_interrupt_post_load(S390CPU *cpu)
+{
+    return 0;
+}
 #endif
 
 static inline int s390_set_memory_limit(uint64_t new_limit, uint64_t *hw_limit)
diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c
index 43ad009..aba1265 100644
--- a/target-s390x/kvm.c
+++ b/target-s390x/kvm.c
@@ -109,6 +109,14 @@
 #define ICPT_CPU_STOP                   0x28
 #define ICPT_IO                         0x40
 
+#define NR_LOCAL_IRQS 32
+/*
+ * Needs to be big enough to contain max_cpus emergency signals
+ * and in addition NR_LOCAL_IRQS interrupts
+ */
+#define VCPU_IRQ_BUF_SIZE (sizeof(struct kvm_s390_irq) * \
+                           (max_cpus + NR_LOCAL_IRQS))
+
 static CPUWatchpoint hw_watchpoint;
 /*
  * We don't use a list because this structure is also used to transmit the
@@ -274,6 +282,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
 {
     S390CPU *cpu = S390_CPU(cs);
     kvm_s390_set_cpu_state(cpu, cpu->env.cpu_state);
+    cpu->irqstate = g_malloc0(VCPU_IRQ_BUF_SIZE);
     return 0;
 }
 
@@ -2059,6 +2068,52 @@ int kvm_s390_set_cpu_state(S390CPU *cpu, uint8_t cpu_state)
     return ret;
 }
 
+void kvm_s390_vcpu_interrupt_pre_save(S390CPU *cpu)
+{
+    struct kvm_s390_irq_state irq_state;
+    CPUState *cs = CPU(cpu);
+    int32_t bytes;
+
+    if (!kvm_check_extension(kvm_state, KVM_CAP_S390_IRQ_STATE)) {
+        return;
+    }
+
+    irq_state.buf = (uint64_t) cpu->irqstate;
+    irq_state.len = VCPU_IRQ_BUF_SIZE;
+
+    bytes = kvm_vcpu_ioctl(cs, KVM_S390_GET_IRQ_STATE, &irq_state);
+    if (bytes < 0) {
+        cpu->irqstate_saved_size = 0;
+        error_report("Migration of interrupt state failed");
+        return;
+    }
+
+    cpu->irqstate_saved_size = bytes;
+}
+
+int kvm_s390_vcpu_interrupt_post_load(S390CPU *cpu)
+{
+    CPUState *cs = CPU(cpu);
+    struct kvm_s390_irq_state irq_state;
+    int r;
+
+    if (!kvm_check_extension(kvm_state, KVM_CAP_S390_IRQ_STATE)) {
+        return -ENOSYS;
+    }
+
+    if (cpu->irqstate_saved_size == 0) {
+        return 0;
+    }
+    irq_state.buf = (uint64_t) cpu->irqstate;
+    irq_state.len = cpu->irqstate_saved_size;
+
+    r = kvm_vcpu_ioctl(cs, KVM_S390_SET_IRQ_STATE, &irq_state);
+    if (r) {
+        error_report("Setting interrupt state failed %d", r);
+    }
+    return r;
+}
+
 int kvm_arch_fixup_msi_route(struct kvm_irq_routing_entry *route,
                               uint64_t address, uint32_t data)
 {
diff --git a/target-s390x/machine.c b/target-s390x/machine.c
index a034423..7853e3c 100644
--- a/target-s390x/machine.c
+++ b/target-s390x/machine.c
@@ -28,10 +28,19 @@ static int cpu_post_load(void *opaque, int version_id)
      */
     if (kvm_enabled()) {
         kvm_s390_set_cpu_state(cpu, cpu->env.cpu_state);
+        return kvm_s390_vcpu_interrupt_post_load(cpu);
     }
 
     return 0;
 }
+static void cpu_pre_save(void *opaque)
+{
+    S390CPU *cpu = opaque;
+
+    if (kvm_enabled()) {
+        kvm_s390_vcpu_interrupt_pre_save(cpu);
+    }
+}
 
 const VMStateDescription vmstate_fpu = {
     .name = "cpu/fpu",
@@ -67,7 +76,8 @@ static inline bool fpu_needed(void *opaque)
 const VMStateDescription vmstate_s390_cpu = {
     .name = "cpu",
     .post_load = cpu_post_load,
-    .version_id = 3,
+    .pre_save = cpu_pre_save,
+    .version_id = 4,
     .minimum_version_id = 3,
     .fields      = (VMStateField[]) {
         VMSTATE_UINT64_ARRAY(env.regs, S390CPU, 16),
@@ -86,6 +96,9 @@ const VMStateDescription vmstate_s390_cpu = {
         VMSTATE_UINT64_ARRAY(env.cregs, S390CPU, 16),
         VMSTATE_UINT8(env.cpu_state, S390CPU),
         VMSTATE_UINT8(env.sigp_order, S390CPU),
+        VMSTATE_UINT32_V(irqstate_saved_size, S390CPU, 4),
+        VMSTATE_VBUFFER_UINT32(irqstate, S390CPU, 4, NULL, 0,
+                               irqstate_saved_size),
         VMSTATE_END_OF_LIST()
      },
     .subsections = (VMStateSubsection[]) {
-- 
2.4.0

Re: [Qemu-devel] [PULL 0/8] s390x: various patches

[Peter Maydell]

On 8 May 2015 at 10:08, Cornelia Huck <cornelia.huck@de.ibm.com> wrote:
> The following changes since commit f8340b360b9bc29d48716ba8aca79df2b9544979:
>
>   hw/ptimer: Do not artificially limit timers when using icount (2015-05-08 17:15:23 +1000)
>
> are available in the git repository at:
>
>   git://github.com/cohuck/qemu tags/s390x-20150508
>
> for you to fetch changes up to 3cda44f7bae5c9feddc11630ba6eecb2e3bed425:
>
>   s390x/kvm: migrate vcpu interrupt state (2015-05-08 10:36:19 +0200)
>
> ----------------------------------------------------------------
> Assorted s390x patches:
> - updates for virtio-ccw and s390-virtio, making them more similar
>   to virtio-pci
> - improvements regarding per-vcpu interrupts and migration
>
> ----------------------------------------------------------------

Applied, thanks.

-- PMM