[Qemu-devel] [PATCH v13 0/4] machvirt dynamic sysbus device instantiation

[Qemu-devel] [PATCH v13 0/4] machvirt dynamic sysbus device instantiation

[Eric Auger]

This series enables machvirt to dynamically instantiate sysbus
devices from command line (using -device option).

All those sysbus devices are plugged onto a platform bus. This latter
device is instantiated in machvirt and takes care of the binding of
children sysbus devices on a machine init done notifier. The device
tree node generation for children dynamic sysbus device also happens
on a subsequent notifier that must be executed after the above one.
machvirt registers that notifier before the platform bus creation to
make sure notifiers are executed in the right order: dt generation
after actual QOM binding.

Few sysbus devices are supposed to be instantiated that way. VFIO
devices belong to them.

Node creation really is architecture specific. On ARM the dynamic
sysbus device node creation is implemented in a new C module,
hw/arm/sysbus-fdt.c and not in the machine file.

Machvirt transformations and sysbus-fdt are largely inspired from Alex work.

The patch series can be found at:
http://git.linaro.org/people/eric.auger/qemu.git,
branch official_dynsysbus_v13-target-arm.next

Best Regards

Eric

v12 -> v13:
- rebase on target-arm.next
- first platform bus IRQ moved at 112 after GICv2m addition.
- take into account Shannon's advice about increasing indentation in
  a15memmap

v11 -> v12:
- rebase on 2.3.0
- add Alex BennéR-b assuming following changes do not alter his review
- take into account Alex Graf comments
  - resize PLATFORM_BUS_NUM_IRQS to 64 instead of 32
  - overall NUM_IRQS changed to 256 since there was no room left for other
    additions
  - VIRT_PLATFORM_BUS size changed from 4MB to 32MB and base set at
    0xc000000

v10 -> v11:
- rebase on v2.2.0 and especially Alex'
  "arm: Add PCIe host bridge in virt machine"

v9 -> v10:
- add assert and fix return value in add_fdt_node

v8 -> v9:
Take into account Peter's comments:
hw/arm/sysbus-fdt:
- s/Fdt/FDT in struct type names
- reorder fields in PlatformBusFdtNotifierParams and use DO_UPCAST
- use assert() when relevant (board model issue)
- g_free the ARMPlatformBusFDTParams and PlatformBusFDTNotifierParams
  pointers in platform_bus_fdt_notify
hw/arm/boot.c
- fix compilation with arm-linux-user
- reorder fields in ArmLoadKernelNotifier
hw/arm/virt.c
- PLATFORM_BUS_NUM_IRQS set to 32 instead of 20
- platform bus irq now start at 64 instead of 48
- remove change of indentation in  a15memmap
- correct misc style issues

add a separate patch file for re-indentation in virt a15memmap

v7 -> v8:
- rebase on 2.2.0 (boot.c and virt.c)
- in virt.c machvirt_init, create_platform_bus simply is added
  after the arm_load_kernel call instead of moving this latter.
  should ease the review.
- Add Alex & Shannon Reviewed-by

v6 -> v7:
Take into account Shannon's comments:
- hw/arm/sysbus-fdt.c: revert indentation in add_fdt_node_functions
- hw/arm/virt.c:
  - add an additional comment in a15irqmap related to PLATFORM_BUS_NUM_IRQS
  - correct platform bus size to 0x400000
  - remove PLATFORM_BUS_FIRST_IRQ macro

v5 -> v6:
Take into account Peter's comments:
- dtb overload mechanism rewritten: arm_load_kernel code is moved into a
  machine init done notifier notify instead. arm_load_kernel only registers
  that notifier. As a consequence the dtb is loaded once.
- v5 1-4 patch files are removed and replaced by a single patch file moving
  arm_load_kernel in the notifier (2).
- as a consequence arm_load_kernel must be called before sysbus-fdt
  arm_register_platform_bus_fdt_creator.
- In virt, platform_bus_params not a const anymore since its fields are
  initialized from vbi->memmap and vbi->irqmap. Hence create_platform_bus
  proto can be simplified.
- In sysbus-fdt add_all_platform_bus_fdt_nodes now takes a handle to
  an ARMPlatformBusFdtParams. This is not a modify_dtb function anymore
  fdt pointer is checked in case the callback is called after the load_dtb
  (this latter deallocated fdt pointer). Check of fdt_filename moved in here.
  upgrade_dtb is removed. copyright aligned between .h and .c.

v4 -> v5:
- in virt.c: platform_bus_params becomes static const
- sysbus-fdt: change indentation in add_fdt_node_functions array init
- s/load_dtb/arm_load_dtb in one boot.c comment

v3 -> v4:
- dyn_sysbus_binding removed since binding stuff now are implemented by
  the platform bus device
- due to a change in ARM load_dtb implementation using rom_add_blob_fixed,
  the dt no more is generated in a reset notifier but is generated on a
  machine init done notifier
- the augmented device tree is not generated from scratch anymore but is
  added using a modify_dtb function. This required some small change in
  boot.c
- the case where the user provides a dtb file now is handled
- some cleanup in virt additions
- implement a list of dyanmically instantiable devices in sysbus-fdt

v2 -> v3:
- patch now applies on top of Alex full patchset
- dyn_sysbus_devtree: add arm_prefix to emphasize the fact those
  functions are arm specific; arm_sysbus_device_create_devtree
  becomes static
- load_dtb renamed into arm_load_dtb
- add copyright in hw/arm/dyn_sysbus_devtree.c

v1 -> v2:
- device node generation no more in sysbus device but in
  dyn_sysbus_devtree
- VFIO region shrinked to 4MB and relocated in machvirt to avoid PCI
  shrink (dynamic vfio-mmio support might come latter)
- platform_bus_base removed from PlatformDevtreeData


Eric Auger (4):
  hw/arm/sysbus-fdt: helpers for platform bus nodes addition
  hw/arm/boot: arm_load_kernel implemented as a machine init done
    notifier
  hw/arm/virt: add dynamic sysbus device support
  hw/arm/virt: change indentation in a15memmap

 hw/arm/Makefile.objs        |   1 +
 hw/arm/boot.c               |  14 +++-
 hw/arm/sysbus-fdt.c         | 174 ++++++++++++++++++++++++++++++++++++++++++++
 hw/arm/virt.c               |  86 ++++++++++++++++++----
 include/hw/arm/arm.h        |  28 +++++++
 include/hw/arm/sysbus-fdt.h |  60 +++++++++++++++
 include/hw/arm/virt.h       |   1 +
 7 files changed, 349 insertions(+), 15 deletions(-)
 create mode 100644 hw/arm/sysbus-fdt.c
 create mode 100644 include/hw/arm/sysbus-fdt.h

-- 
1.8.3.2

[Qemu-devel] [PATCH v13 1/4] hw/arm/sysbus-fdt: helpers for platform bus nodes addition

[Eric Auger]

This new C module will be used by ARM machine files to generate
platform bus node and their dynamic sysbus device tree nodes.

Dynamic sysbus device node addition is done in a machine init
done notifier. arm_register_platform_bus_fdt_creator does the
registration of this latter and is supposed to be called by
ARM machine files that support platform bus and their dynamic
sysbus. Addition of dynamic sysbus nodes is done only if the
user did not provide any dtb.

Signed-off-by: Alexander Graf <agraf@suse.de>
Signed-off-by: Eric Auger <eric.auger@linaro.org>
Reviewed-by: Shannon Zhao <zhaoshenglong@huawei.com>
Reviewed-by: Alexander Graf <agraf@suse.de>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

---
v11 -> v12:
- resolve conflict introduced by netduino2.o addition in
  hw/arm/Makefile.objs
- Add Alex R-b

v9 -> v10:
- add assert and exit in add_fdt_node

v8 -> v9:
- s/Fdt/FDT in struct type names
- reorder fields in PlatformBusFdtNotifierParams and use DO_UPCAST
  instead of container_of
- use assert() when relevant (board model issue)
- g_free the ARMPlatformBusFDTParams and PlatformBusFDTNotifierParams
  pointers in platform_bus_fdt_notify

v7 -> v8:
add Reviewed-by from Alex and Shannon

v6 -> v7:
- revert indentation in add_fdt_node_functions

v5 -> v6:
- add_all_platform_bus_fdt_nodes is not a modify_dtb function anymore
- it now takes a handle to an ARMPlatformBusFdtParams.
- fdt pointer is checked in case this notifier is executed after the
  one that executes the load_dtb (this latter deallocates the fdt pointer)
- check of fdt_filename moved in here.
- upgrade_dtb is removed
- copyright aligned between .h and .c

v4 -> v5:
- change indentation in add_fdt_node_functions. Also becomes a
  static const.
- ARMPlatformBusFdtParams.system_params becomes a pointer to
  a const ARMPlatformBusSystemParams
- removes platform-bus.h second inclusion

v3 -> v4:
- dyn_sysbus_devtree.c renamed into sysbus-fdt.c
- use new PlatformBusDevice object
- the dtb upgrade is done through modify_dtb. Before the fdt
  was recreated from scratch. When the user provided a dtb this
  latter was overwritten which was not correct.
- an array contains the association between device type names
  and their node creation function
- I must aknowledge I did not find any cleaner way to implement
  a FDT_BUILDER interface, as suggested by Paolo. The class method
  would need to be initialized somewhere and since it cannot
  happen in the device itself - according to Alex & Peter comments -,
  I don't see when I shall associate the device type and its
  interface implementation.

v2 -> v3:
- add arm_ prefix
- arm_sysbus_device_create_devtree becomes static

v1 -> v2:
- Code moved in an arch specific file to accomodate architecture
  dependent specificities.
- remove platform_bus_base from PlatformDevtreeData

v1: code originally written by Alex Graf in e500.c and reused for
ARM [Eric Auger]
---
 hw/arm/Makefile.objs        |   1 +
 hw/arm/sysbus-fdt.c         | 174 ++++++++++++++++++++++++++++++++++++++++++++
 include/hw/arm/sysbus-fdt.h |  60 +++++++++++++++
 3 files changed, 235 insertions(+)
 create mode 100644 hw/arm/sysbus-fdt.c
 create mode 100644 include/hw/arm/sysbus-fdt.h

diff --git a/hw/arm/Makefile.objs b/hw/arm/Makefile.objs
index 4b09caf..cf346c1 100644
--- a/hw/arm/Makefile.objs
+++ b/hw/arm/Makefile.objs
@@ -5,6 +5,7 @@ obj-y += omap_sx1.o palm.o realview.o spitz.o stellaris.o
 obj-y += tosa.o versatilepb.o vexpress.o virt.o xilinx_zynq.o z2.o
 obj-$(CONFIG_ACPI) += virt-acpi-build.o
 obj-y += netduino2.o
+obj-y += sysbus-fdt.o
 
 obj-y += armv7m.o exynos4210.o pxa2xx.o pxa2xx_gpio.o pxa2xx_pic.o
 obj-$(CONFIG_DIGIC) += digic.o
diff --git a/hw/arm/sysbus-fdt.c b/hw/arm/sysbus-fdt.c
new file mode 100644
index 0000000..3038b94
--- /dev/null
+++ b/hw/arm/sysbus-fdt.c
@@ -0,0 +1,174 @@
+/*
+ * ARM Platform Bus device tree generation helpers
+ *
+ * Copyright (c) 2014 Linaro Limited
+ *
+ * Authors:
+ *  Alex Graf <agraf@suse.de>
+ *  Eric Auger <eric.auger@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "hw/arm/sysbus-fdt.h"
+#include "qemu/error-report.h"
+#include "sysemu/device_tree.h"
+#include "hw/platform-bus.h"
+#include "sysemu/sysemu.h"
+
+/*
+ * internal struct that contains the information to create dynamic
+ * sysbus device node
+ */
+typedef struct PlatformBusFDTData {
+    void *fdt; /* device tree handle */
+    int irq_start; /* index of the first IRQ usable by platform bus devices */
+    const char *pbus_node_name; /* name of the platform bus node */
+    PlatformBusDevice *pbus;
+} PlatformBusFDTData;
+
+/*
+ * struct used when calling the machine init done notifier
+ * that constructs the fdt nodes of platform bus devices
+ */
+typedef struct PlatformBusFDTNotifierParams {
+    Notifier notifier;
+    ARMPlatformBusFDTParams *fdt_params;
+} PlatformBusFDTNotifierParams;
+
+/* struct that associates a device type name and a node creation function */
+typedef struct NodeCreationPair {
+    const char *typename;
+    int (*add_fdt_node_fn)(SysBusDevice *sbdev, void *opaque);
+} NodeCreationPair;
+
+/* list of supported dynamic sysbus devices */
+static const NodeCreationPair add_fdt_node_functions[] = {
+    {"", NULL}, /* last element */
+};
+
+/**
+ * add_fdt_node - add the device tree node of a dynamic sysbus device
+ *
+ * @sbdev: handle to the sysbus device
+ * @opaque: handle to the PlatformBusFDTData
+ *
+ * Checks the sysbus type belongs to the list of device types that
+ * are dynamically instantiable and if so call the node creation
+ * function.
+ */
+static int add_fdt_node(SysBusDevice *sbdev, void *opaque)
+{
+    int i, ret;
+
+    for (i = 0; i < ARRAY_SIZE(add_fdt_node_functions); i++) {
+        if (!strcmp(object_get_typename(OBJECT(sbdev)),
+                    add_fdt_node_functions[i].typename)) {
+            ret = add_fdt_node_functions[i].add_fdt_node_fn(sbdev, opaque);
+            assert(!ret);
+            return 0;
+        }
+    }
+    error_report("Device %s can not be dynamically instantiated",
+                     qdev_fw_name(DEVICE(sbdev)));
+    exit(1);
+}
+
+/**
+ * add_all_platform_bus_fdt_nodes - create all the platform bus nodes
+ *
+ * builds the parent platform bus node and all the nodes of dynamic
+ * sysbus devices attached to it.
+ */
+static void add_all_platform_bus_fdt_nodes(ARMPlatformBusFDTParams *fdt_params)
+{
+    const char platcomp[] = "qemu,platform\0simple-bus";
+    PlatformBusDevice *pbus;
+    DeviceState *dev;
+    gchar *node;
+    uint64_t addr, size;
+    int irq_start, dtb_size;
+    struct arm_boot_info *info = fdt_params->binfo;
+    const ARMPlatformBusSystemParams *params = fdt_params->system_params;
+    const char *intc = fdt_params->intc;
+    void *fdt = info->get_dtb(info, &dtb_size);
+
+    /*
+     * If the user provided a dtb, we assume the dynamic sysbus nodes
+     * already are integrated there. This corresponds to a use case where
+     * the dynamic sysbus nodes are complex and their generation is not yet
+     * supported. In that case the user can take charge of the guest dt
+     * while qemu takes charge of the qom stuff.
+     */
+    if (info->dtb_filename) {
+        return;
+    }
+
+    assert(fdt);
+
+    node = g_strdup_printf("/platform@%"PRIx64, params->platform_bus_base);
+    addr = params->platform_bus_base;
+    size = params->platform_bus_size;
+    irq_start = params->platform_bus_first_irq;
+
+    /* Create a /platform node that we can put all devices into */
+    qemu_fdt_add_subnode(fdt, node);
+    qemu_fdt_setprop(fdt, node, "compatible", platcomp, sizeof(platcomp));
+
+    /* Our platform bus region is less than 32bits, so 1 cell is enough for
+     * address and size
+     */
+    qemu_fdt_setprop_cells(fdt, node, "#size-cells", 1);
+    qemu_fdt_setprop_cells(fdt, node, "#address-cells", 1);
+    qemu_fdt_setprop_cells(fdt, node, "ranges", 0, addr >> 32, addr, size);
+
+    qemu_fdt_setprop_phandle(fdt, node, "interrupt-parent", intc);
+
+    dev = qdev_find_recursive(sysbus_get_default(), TYPE_PLATFORM_BUS_DEVICE);
+    pbus = PLATFORM_BUS_DEVICE(dev);
+
+    /* We can only create dt nodes for dynamic devices when they're ready */
+    assert(pbus->done_gathering);
+
+    PlatformBusFDTData data = {
+        .fdt = fdt,
+        .irq_start = irq_start,
+        .pbus_node_name = node,
+        .pbus = pbus,
+    };
+
+    /* Loop through all dynamic sysbus devices and create their node */
+    foreach_dynamic_sysbus_device(add_fdt_node, &data);
+
+    g_free(node);
+}
+
+static void platform_bus_fdt_notify(Notifier *notifier, void *data)
+{
+    PlatformBusFDTNotifierParams *p = DO_UPCAST(PlatformBusFDTNotifierParams,
+                                                notifier, notifier);
+
+    add_all_platform_bus_fdt_nodes(p->fdt_params);
+    g_free(p->fdt_params);
+    g_free(p);
+}
+
+void arm_register_platform_bus_fdt_creator(ARMPlatformBusFDTParams *fdt_params)
+{
+    PlatformBusFDTNotifierParams *p = g_new(PlatformBusFDTNotifierParams, 1);
+
+    p->fdt_params = fdt_params;
+    p->notifier.notify = platform_bus_fdt_notify;
+    qemu_add_machine_init_done_notifier(&p->notifier);
+}
diff --git a/include/hw/arm/sysbus-fdt.h b/include/hw/arm/sysbus-fdt.h
new file mode 100644
index 0000000..e15bb81
--- /dev/null
+++ b/include/hw/arm/sysbus-fdt.h
@@ -0,0 +1,60 @@
+/*
+ * Dynamic sysbus device tree node generation API
+ *
+ * Copyright Linaro Limited, 2014
+ *
+ * Authors:
+ *  Alex Graf <agraf@suse.de>
+ *  Eric Auger <eric.auger@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef HW_ARM_SYSBUS_FDT_H
+#define HW_ARM_SYSBUS_FDT_H
+
+#include "hw/arm/arm.h"
+#include "qemu-common.h"
+#include "hw/sysbus.h"
+
+/*
+ * struct that contains dimensioning parameters of the platform bus
+ */
+typedef struct {
+    hwaddr platform_bus_base; /* start address of the bus */
+    hwaddr platform_bus_size; /* size of the bus */
+    int platform_bus_first_irq; /* first hwirq assigned to the bus */
+    int platform_bus_num_irqs; /* number of hwirq assigned to the bus */
+} ARMPlatformBusSystemParams;
+
+/*
+ * struct that contains all relevant info to build the fdt nodes of
+ * platform bus and attached dynamic sysbus devices
+ * in the future might be augmented with additional info
+ * such as PHY, CLK handles ...
+ */
+typedef struct {
+    const ARMPlatformBusSystemParams *system_params;
+    struct arm_boot_info *binfo;
+    const char *intc; /* parent interrupt controller name */
+} ARMPlatformBusFDTParams;
+
+/**
+ * arm_register_platform_bus_fdt_creator - register a machine init done
+ * notifier that creates the device tree nodes of the platform bus and
+ * associated dynamic sysbus devices
+ */
+void arm_register_platform_bus_fdt_creator(ARMPlatformBusFDTParams *fdt_params);
+
+#endif
-- 
1.8.3.2

[Qemu-devel] [PATCH v13 2/4] hw/arm/boot: arm_load_kernel implemented as a machine init done notifier

[Eric Auger]

Device tree nodes for the platform bus and its children dynamic sysbus
devices are added in a machine init done notifier. To load the dtb once,
after those latter nodes are built and before ROM freeze, the actual
arm_load_kernel existing code is moved into a notifier notify function,
arm_load_kernel_notify. arm_load_kernel now only registers the
corresponding notifier.

Machine files that do not support platform bus stay unchanged. Machine
files willing to support dynamic sysbus devices must call arm_load_kernel
before sysbus-fdt arm_register_platform_bus_fdt_creator to make sure
dynamic sysbus device nodes are integrated in the dtb.

Signed-off-by: Eric Auger <eric.auger@linaro.org>
Reviewed-by: Shannon Zhao <zhaoshenglong@huawei.com>
Reviewed-by: Alexander Graf <agraf@suse.de>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

---

v11 -> v12:
- Add Alex R-b

v8 -> v9:
- fix compilation with arm-linux-user
- reorder fields in ArmLoadKernelNotifier and use DO_UPCAST

v7 -> v8:
- Add Reviewed-by from Alex & Shannon
- rebase on 2.2.0

v6: creation of this patch file
---
 hw/arm/boot.c        | 14 +++++++++++++-
 include/hw/arm/arm.h | 28 ++++++++++++++++++++++++++++
 2 files changed, 41 insertions(+), 1 deletion(-)

diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index fa69503..d036624 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -557,7 +557,7 @@ static void load_image_to_fw_cfg(FWCfgState *fw_cfg, uint16_t size_key,
     fw_cfg_add_bytes(fw_cfg, data_key, data, size);
 }
 
-void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
+static void arm_load_kernel_notify(Notifier *notifier, void *data)
 {
     CPUState *cs;
     int kernel_size;
@@ -568,6 +568,11 @@ void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
     hwaddr entry, kernel_load_offset;
     int big_endian;
     static const ARMInsnFixup *primary_loader;
+    ArmLoadKernelNotifier *n = DO_UPCAST(ArmLoadKernelNotifier,
+                                         notifier, notifier);
+    ARMCPU *cpu = n->cpu;
+    struct arm_boot_info *info =
+        container_of(n, struct arm_boot_info, load_kernel_notifier);
 
     /* CPU objects (unlike devices) are not automatically reset on system
      * reset, so we must always register a handler to do so. If we're
@@ -775,3 +780,10 @@ void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
         ARM_CPU(cs)->env.boot_info = info;
     }
 }
+
+void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
+{
+    info->load_kernel_notifier.cpu = cpu;
+    info->load_kernel_notifier.notifier.notify = arm_load_kernel_notify;
+    qemu_add_machine_init_done_notifier(&info->load_kernel_notifier.notifier);
+}
diff --git a/include/hw/arm/arm.h b/include/hw/arm/arm.h
index 5c940eb..760804c 100644
--- a/include/hw/arm/arm.h
+++ b/include/hw/arm/arm.h
@@ -13,11 +13,21 @@
 
 #include "exec/memory.h"
 #include "hw/irq.h"
+#include "qemu/notify.h"
 
 /* armv7m.c */
 qemu_irq *armv7m_init(MemoryRegion *system_memory, int mem_size, int num_irq,
                       const char *kernel_filename, const char *cpu_model);
 
+/*
+ * struct used as a parameter of the arm_load_kernel machine init
+ * done notifier
+ */
+typedef struct {
+    Notifier notifier; /* actual notifier */
+    ARMCPU *cpu; /* handle to the first cpu object */
+} ArmLoadKernelNotifier;
+
 /* arm_boot.c */
 struct arm_boot_info {
     uint64_t ram_size;
@@ -64,6 +74,8 @@ struct arm_boot_info {
      * the user it should implement this hook.
      */
     void (*modify_dtb)(const struct arm_boot_info *info, void *fdt);
+    /* machine init done notifier executing arm_load_dtb */
+    ArmLoadKernelNotifier load_kernel_notifier;
     /* Used internally by arm_boot.c */
     int is_linux;
     hwaddr initrd_start;
@@ -75,6 +87,22 @@ struct arm_boot_info {
      */
     bool firmware_loaded;
 };
+
+/**
+ * arm_load_kernel - Loads memory with everything needed to boot
+ *
+ * @cpu: handle to the first CPU object
+ * @info: handle to the boot info struct
+ * Registers a machine init done notifier that copies to memory
+ * everything needed to boot, depending on machine and user options:
+ * kernel image, boot loaders, initrd, dtb. Also registers the CPU
+ * reset handler.
+ *
+ * In case the machine file supports the platform bus device and its
+ * dynamically instantiable sysbus devices, this function must be called
+ * before sysbus-fdt arm_register_platform_bus_fdt_creator. Indeed the
+ * machine init done notifiers are called in registration reverse order.
+ */
 void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info);
 
 /* Multiplication factor to convert from system clock ticks to qemu timer
-- 
1.8.3.2

[Qemu-devel] [PATCH v13 3/4] hw/arm/virt: add dynamic sysbus device support

[Eric Auger]

Allows sysbus devices to be instantiated from command line by
using -device option. Machvirt creates a platform bus at init.
The dynamic sysbus devices are attached to this platform bus device.

The platform bus device registers a machine init done notifier
whose role will be to bind the dynamic sysbus devices. Indeed
dynamic sysbus devices are created after machine init.

machvirt also registers a notifier that will build the device
tree nodes for the platform bus and its children dynamic sysbus
devices.

Signed-off-by: Eric Auger <eric.auger@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

---
v12 -> v13:
- Changed start IRQ number of platform bus to 112 after the addition
  of MSI range

v11 -> v12:
- resize PLATFORM_BUS_NUM_IRQS to 64 instead of 32
- overall NUM_IRQS changed to 256. This leaves space for MSI
  looming addition
- VIRT_PLATFORM_BUS size changed from 4MB to 32MB and base set at
  0xc000000
- Add Alex R-b

v8 -> v9:
- PLATFORM_BUS_NUM_IRQS set to 32 instead of 20
- platform bus irq now start at 64 instead of 48
- remove change of indentation in a15memmap
- correct misc style issues

v7 -> v8:
- rebase on 2.2.0
- in machvirt_init, create_platform_bus simply is added
  after the arm_load_kernel call instead of moving this latter.
  Related comment slighly reworded.
- Due to those changes I dropped Alex and Shannon's Reviewed-by

v6 -> v7:
Take into account Shannon comments:
- remove PLATFORM_BUS_FIRST_IRQ macro
- correct platform bus size to 0x400000
- add an additional comment in a15irqmap related to
  PLATFORM_BUS_NUM_IRQS

v5 -> v6:
- Take into account Peter's comments:
  - platform_bus_params initialized from vbi->memmap and vbi->irqmap.
    As a consequence, const is removed. Also alignment in a15memmap
    is slightly changed.
  - ARMPlatformBusSystemParams handle removed from create_platform_bus
    prototype
- arm_load_kernel has become a machine init done notifier registration.
  It must be called before platform_bus_create to guarantee the correct
  notifier execution sequence

v4 -> v5:
- platform_bus_params becomes static const
- reword comment in create_platform_bus
- reword the commit message

v3 -> v4:
- use platform bus object, instantiated in create_platform_bus
- device tree generation for platform bus and children dynamic
  sysbus devices is no more handled at reset but in a
  machine_init_done_notifier (due to the change in implementaion
  of ARM load dtb using rom_add_blob_fixed).
- device tree enhancement now takes into account the case of
  user provided dtb. Before the user dtb was overwritten which
  was wrong. However in case the dtb is provided by the user,
  dynamic sysbus nodes are not added there.
- renaming of MACHVIRT_PLATFORM defines
- MACHVIRT_PLATFORM_PAGE_SHIFT and SIZE_PAGES not needed anymore,
  hence removed.
- DynSysbusParams struct renamed into ARMPlatformBusSystemParams
  and above params removed.
- separation of dt creation and QEMU binding is not mandated anymore
  since the device tree is not created from scratch anymore. Instead
  the modify_dtb function is used.
- create_platform_bus registers another machine init done notifier
  to start VFIO IRQ handling. This latter executes after the
  dynamic sysbus device binding.

v2 -> v3:
- renaming of arm_platform_bus_create_devtree and arm_load_dtb
- add copyright in hw/arm/dyn_sysbus_devtree.c

v1 -> v2:
- remove useless vfio-platform.h include file
- s/MACHVIRT_PLATFORM_HOLE/MACHVIRT_PLATFORM_SIZE
- use dyn_sysbus_binding and dyn_sysbus_devtree
- dynamic sysbus platform buse size shrinked to 4MB and
  moved between RTC and MMIO

v1:

Inspired from what Alex Graf did in ppc e500
https://lists.gnu.org/archive/html/qemu-ppc/2014-07/msg00012.html
---
 hw/arm/virt.c         | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 include/hw/arm/virt.h |  1 +
 2 files changed, 60 insertions(+), 1 deletion(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 4bb7175..02b91ba 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -45,9 +45,11 @@
 #include "qemu/error-report.h"
 #include "hw/pci-host/gpex.h"
 #include "hw/arm/virt-acpi-build.h"
+#include "hw/arm/sysbus-fdt.h"
+#include "hw/platform-bus.h"
 
 /* Number of external interrupt lines to configure the GIC with */
-#define NUM_IRQS 128
+#define NUM_IRQS 256
 
 #define GIC_FDT_IRQ_TYPE_SPI 0
 #define GIC_FDT_IRQ_TYPE_PPI 1
@@ -60,6 +62,10 @@
 #define GIC_FDT_IRQ_PPI_CPU_START 8
 #define GIC_FDT_IRQ_PPI_CPU_WIDTH 8
 
+#define PLATFORM_BUS_NUM_IRQS 64
+
+static ARMPlatformBusSystemParams platform_bus_params;
+
 typedef struct VirtBoardInfo {
     struct arm_boot_info bootinfo;
     const char *cpu_model;
@@ -116,6 +122,7 @@ static const MemMapEntry a15memmap[] = {
     [VIRT_FW_CFG] =     { 0x09020000, 0x0000000a },
     [VIRT_MMIO] =       { 0x0a000000, 0x00000200 },
     /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */
+    [VIRT_PLATFORM_BUS] =   { 0x0c000000, 0x02000000 },
     [VIRT_PCIE_MMIO] =  { 0x10000000, 0x2eff0000 },
     [VIRT_PCIE_PIO] =   { 0x3eff0000, 0x00010000 },
     [VIRT_PCIE_ECAM] =  { 0x3f000000, 0x01000000 },
@@ -128,6 +135,7 @@ static const int a15irqmap[] = {
     [VIRT_PCIE] = 3, /* ... to 6 */
     [VIRT_MMIO] = 16, /* ...to 16 + NUM_VIRTIO_TRANSPORTS - 1 */
     [VIRT_GIC_V2M] = 48, /* ...to 48 + NUM_GICV2M_SPIS - 1 */
+    [VIRT_PLATFORM_BUS] = 112, /* ...to 112 + PLATFORM_BUS_NUM_IRQS -1 */
 };
 
 static VirtBoardInfo machines[] = {
@@ -728,6 +736,47 @@ static void create_pcie(const VirtBoardInfo *vbi, qemu_irq *pic)
     g_free(nodename);
 }
 
+static void create_platform_bus(VirtBoardInfo *vbi, qemu_irq *pic)
+{
+    DeviceState *dev;
+    SysBusDevice *s;
+    int i;
+    ARMPlatformBusFDTParams *fdt_params = g_new(ARMPlatformBusFDTParams, 1);
+    MemoryRegion *sysmem = get_system_memory();
+
+    platform_bus_params.platform_bus_base = vbi->memmap[VIRT_PLATFORM_BUS].base;
+    platform_bus_params.platform_bus_size = vbi->memmap[VIRT_PLATFORM_BUS].size;
+    platform_bus_params.platform_bus_first_irq = vbi->irqmap[VIRT_PLATFORM_BUS];
+    platform_bus_params.platform_bus_num_irqs = PLATFORM_BUS_NUM_IRQS;
+
+    fdt_params->system_params = &platform_bus_params;
+    fdt_params->binfo = &vbi->bootinfo;
+    fdt_params->intc = "/intc";
+    /*
+     * register a machine init done notifier that creates the device tree
+     * nodes of the platform bus and its children dynamic sysbus devices
+     */
+    arm_register_platform_bus_fdt_creator(fdt_params);
+
+    dev = qdev_create(NULL, TYPE_PLATFORM_BUS_DEVICE);
+    dev->id = TYPE_PLATFORM_BUS_DEVICE;
+    qdev_prop_set_uint32(dev, "num_irqs",
+        platform_bus_params.platform_bus_num_irqs);
+    qdev_prop_set_uint32(dev, "mmio_size",
+        platform_bus_params.platform_bus_size);
+    qdev_init_nofail(dev);
+    s = SYS_BUS_DEVICE(dev);
+
+    for (i = 0; i < platform_bus_params.platform_bus_num_irqs; i++) {
+        int irqn = platform_bus_params.platform_bus_first_irq + i;
+        sysbus_connect_irq(s, i, pic[irqn]);
+    }
+
+    memory_region_add_subregion(sysmem,
+                                platform_bus_params.platform_bus_base,
+                                sysbus_mmio_get_region(s, 0));
+}
+
 static void *machvirt_dtb(const struct arm_boot_info *binfo, int *fdt_size)
 {
     const VirtBoardInfo *board = (const VirtBoardInfo *)binfo;
@@ -865,6 +914,14 @@ static void machvirt_init(MachineState *machine)
     vbi->bootinfo.get_dtb = machvirt_dtb;
     vbi->bootinfo.firmware_loaded = bios_name || drive_get(IF_PFLASH, 0, 0);
     arm_load_kernel(ARM_CPU(first_cpu), &vbi->bootinfo);
+
+    /*
+     * arm_load_kernel machine init done notifier registration must
+     * happen before the platform_bus_create call. In this latter,
+     * another notifier is registered which adds platform bus nodes.
+     * Notifiers are executed in registration reverse order.
+     */
+    create_platform_bus(vbi, pic);
 }
 
 static bool virt_get_secure(Object *obj, Error **errp)
@@ -903,6 +960,7 @@ static void virt_class_init(ObjectClass *oc, void *data)
     mc->desc = "ARM Virtual Machine",
     mc->init = machvirt_init;
     mc->max_cpus = 8;
+    mc->has_dynamic_sysbus = true;
 }
 
 static const TypeInfo machvirt_info = {
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index 003ef29..d22fd8e 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -55,6 +55,7 @@ enum {
     VIRT_PCIE_PIO,
     VIRT_PCIE_ECAM,
     VIRT_GIC_V2M,
+    VIRT_PLATFORM_BUS,
 };
 
 typedef struct MemMapEntry {
-- 
1.8.3.2

[Qemu-devel] [PATCH v13 4/4] hw/arm/virt: change indentation in a15memmap

[Eric Auger]

Re-indent in a15memmap after VIRT_PLATFORM_BUS introduction

Signed-off-by: Eric Auger <eric.auger@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

---

v11 -> v12:
- Add Alex R-b
---
 hw/arm/virt.c | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 02b91ba..0a75cc8 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -111,22 +111,22 @@ typedef struct {
  */
 static const MemMapEntry a15memmap[] = {
     /* Space up to 0x8000000 is reserved for a boot ROM */
-    [VIRT_FLASH] =      {          0, 0x08000000 },
-    [VIRT_CPUPERIPHS] = { 0x08000000, 0x00020000 },
+    [VIRT_FLASH] =              {          0, 0x08000000 },
+    [VIRT_CPUPERIPHS] =         { 0x08000000, 0x00020000 },
     /* GIC distributor and CPU interfaces sit inside the CPU peripheral space */
-    [VIRT_GIC_DIST] =   { 0x08000000, 0x00010000 },
-    [VIRT_GIC_CPU] =    { 0x08010000, 0x00010000 },
-    [VIRT_GIC_V2M] =    { 0x08020000, 0x00001000 },
-    [VIRT_UART] =       { 0x09000000, 0x00001000 },
-    [VIRT_RTC] =        { 0x09010000, 0x00001000 },
-    [VIRT_FW_CFG] =     { 0x09020000, 0x0000000a },
-    [VIRT_MMIO] =       { 0x0a000000, 0x00000200 },
+    [VIRT_GIC_DIST] =           { 0x08000000, 0x00010000 },
+    [VIRT_GIC_CPU] =            { 0x08010000, 0x00010000 },
+    [VIRT_GIC_V2M] =            { 0x08020000, 0x00001000 },
+    [VIRT_UART] =               { 0x09000000, 0x00001000 },
+    [VIRT_RTC] =                { 0x09010000, 0x00001000 },
+    [VIRT_FW_CFG] =             { 0x09020000, 0x0000000a },
+    [VIRT_MMIO] =               { 0x0a000000, 0x00000200 },
     /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */
-    [VIRT_PLATFORM_BUS] =   { 0x0c000000, 0x02000000 },
-    [VIRT_PCIE_MMIO] =  { 0x10000000, 0x2eff0000 },
-    [VIRT_PCIE_PIO] =   { 0x3eff0000, 0x00010000 },
-    [VIRT_PCIE_ECAM] =  { 0x3f000000, 0x01000000 },
-    [VIRT_MEM] =        { 0x40000000, 30ULL * 1024 * 1024 * 1024 },
+    [VIRT_PLATFORM_BUS] =       { 0x0c000000, 0x02000000 },
+    [VIRT_PCIE_MMIO] =          { 0x10000000, 0x2eff0000 },
+    [VIRT_PCIE_PIO] =           { 0x3eff0000, 0x00010000 },
+    [VIRT_PCIE_ECAM] =          { 0x3f000000, 0x01000000 },
+    [VIRT_MEM] =                { 0x40000000, 30ULL * 1024 * 1024 * 1024 },
 };
 
 static const int a15irqmap[] = {
-- 
1.8.3.2

Re: [Qemu-devel] [PATCH v13 4/4] hw/arm/virt: change indentation in a15memmap

[Shannon Zhao]

On 2015/6/2 19:29, Eric Auger wrote:
> Re-indent in a15memmap after VIRT_PLATFORM_BUS introduction
> 
> Signed-off-by: Eric Auger <eric.auger@linaro.org>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> 
Reviewed-by: Shannon Zhao <shannon.zhao@linaro.org>

> ---
> 
> v11 -> v12:
> - Add Alex R-b
> ---
>  hw/arm/virt.c | 28 ++++++++++++++--------------
>  1 file changed, 14 insertions(+), 14 deletions(-)
> 
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 02b91ba..0a75cc8 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -111,22 +111,22 @@ typedef struct {
>   */
>  static const MemMapEntry a15memmap[] = {
>      /* Space up to 0x8000000 is reserved for a boot ROM */
> -    [VIRT_FLASH] =      {          0, 0x08000000 },
> -    [VIRT_CPUPERIPHS] = { 0x08000000, 0x00020000 },
> +    [VIRT_FLASH] =              {          0, 0x08000000 },
> +    [VIRT_CPUPERIPHS] =         { 0x08000000, 0x00020000 },
>      /* GIC distributor and CPU interfaces sit inside the CPU peripheral space */
> -    [VIRT_GIC_DIST] =   { 0x08000000, 0x00010000 },
> -    [VIRT_GIC_CPU] =    { 0x08010000, 0x00010000 },
> -    [VIRT_GIC_V2M] =    { 0x08020000, 0x00001000 },
> -    [VIRT_UART] =       { 0x09000000, 0x00001000 },
> -    [VIRT_RTC] =        { 0x09010000, 0x00001000 },
> -    [VIRT_FW_CFG] =     { 0x09020000, 0x0000000a },
> -    [VIRT_MMIO] =       { 0x0a000000, 0x00000200 },
> +    [VIRT_GIC_DIST] =           { 0x08000000, 0x00010000 },
> +    [VIRT_GIC_CPU] =            { 0x08010000, 0x00010000 },
> +    [VIRT_GIC_V2M] =            { 0x08020000, 0x00001000 },
> +    [VIRT_UART] =               { 0x09000000, 0x00001000 },
> +    [VIRT_RTC] =                { 0x09010000, 0x00001000 },
> +    [VIRT_FW_CFG] =             { 0x09020000, 0x0000000a },
> +    [VIRT_MMIO] =               { 0x0a000000, 0x00000200 },
>      /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */
> -    [VIRT_PLATFORM_BUS] =   { 0x0c000000, 0x02000000 },
> -    [VIRT_PCIE_MMIO] =  { 0x10000000, 0x2eff0000 },
> -    [VIRT_PCIE_PIO] =   { 0x3eff0000, 0x00010000 },
> -    [VIRT_PCIE_ECAM] =  { 0x3f000000, 0x01000000 },
> -    [VIRT_MEM] =        { 0x40000000, 30ULL * 1024 * 1024 * 1024 },
> +    [VIRT_PLATFORM_BUS] =       { 0x0c000000, 0x02000000 },
> +    [VIRT_PCIE_MMIO] =          { 0x10000000, 0x2eff0000 },
> +    [VIRT_PCIE_PIO] =           { 0x3eff0000, 0x00010000 },
> +    [VIRT_PCIE_ECAM] =          { 0x3f000000, 0x01000000 },
> +    [VIRT_MEM] =                { 0x40000000, 30ULL * 1024 * 1024 * 1024 },
>  };
>  
>  static const int a15irqmap[] = {
> 

-- 
Shannon

Re: [Qemu-devel] [PATCH v13 0/4] machvirt dynamic sysbus device instantiation

[Peter Maydell]

On 2 June 2015 at 12:29, Eric Auger <eric.auger@linaro.org> wrote:
> This series enables machvirt to dynamically instantiate sysbus
> devices from command line (using -device option).
>
> All those sysbus devices are plugged onto a platform bus. This latter
> device is instantiated in machvirt and takes care of the binding of
> children sysbus devices on a machine init done notifier. The device
> tree node generation for children dynamic sysbus device also happens
> on a subsequent notifier that must be executed after the above one.
> machvirt registers that notifier before the platform bus creation to
> make sure notifiers are executed in the right order: dt generation
> after actual QOM binding.
>
> Few sysbus devices are supposed to be instantiated that way. VFIO
> devices belong to them.
>
> Node creation really is architecture specific. On ARM the dynamic
> sysbus device node creation is implemented in a new C module,
> hw/arm/sysbus-fdt.c and not in the machine file.
>
> Machvirt transformations and sysbus-fdt are largely inspired from Alex work.
>
> The patch series can be found at:
> http://git.linaro.org/people/eric.auger/qemu.git,
> branch official_dynsysbus_v13-target-arm.next
>
> Best Regards

This has been on list a fair while and got plenty of review
(thanks to everybody who helped with that). I'm not really
very happy about the whole concept of mmio passthrough, but
some people need it, so we're stuck with it.

Applied to target-arm.next, thanks.

-- PMM