Re: [PATCH v15 00/10] KVM platform device passthrough

Baptiste Reynal
2015-05-11T12:28:41Z

Hi Eric,

I rebased amba patches on this serie. Everything is working fine with
the PL330 device.

Regards,
Baptiste

On Wed, May 6, 2015 at 8:37 AM, Eric Auger <eric.auger@linaro.org> wrote:
> Dear All,
>
> Please ignore the previous void message. For unknown reason the reply
> systematically ignores the content of the message? Retrying breaking the
> history... Content was:
>
> I am looking for "Tested-by" for this series and related Machvirt
> dynamic sysbus device instantiation. Did anyone try the last versions?
>
> I know those series are not plug & play since additions are needed for
> your device:
> - device tree node addition in sysbus device
> - creation of a specialized QEMU VFIO platform device since base class
> is abstract
>
> For those willing to try and assess that code, please do not hesitate to
> contact me. I can and I am willing to help. If you already did, with
> success or facing issues, please send your Tested-by or report any issue
> for quick fix. This will definitively help in the upstream of this code.
>
> Thanks
>
> Best Regards
>
> Eric
>
>
>
> This series aims at enabling KVM platform device passthrough.
>
> On kernel side, the vfio platform driver is needed, available from
> 4.1-rc1 onwards.
>
> This series now only relies on the following QEMU series, for
> dynamic instantiation of the VFIO platform device from qemu command
> line:
>
> [1] [PATCH v12 0/4] machvirt dynamic sysbus device instantiation
> http://comments.gmane.org/gmane.comp.emulators.kvm.arm.devel/886
>
> Both series are candidate for QEMU 2.4 and available at
>
> http://git.linaro.org/people/eric.auger/qemu.git
> (branch vfio_integ_v15)
>
> The series was tested on Calxeda Midway (ARMv7) where one xgmac
> is assigned to KVM host while the second one is assigned to the guest.
>
> Wiki for Calxeda Midway setup:
> https://wiki.linaro.org/LEG/Engineering/Virtualization/Platform_Device_Passthrough_on_Midway
>
> History:
>
> v14 -> v15:
> - add Peter R-b on sysbus: add irq_routing_notifier
> - correct g_malloc0_n usage in skeleton
> - correct return values of reset related functions
> - include Cornelia's patch for header update
>
> v13 -> v14:
> - remove v13 9, 10, 11 patch files and replace them by a single patch file
> "sysbus: add irq_routing_notifier".
> - in skeleton, fix ENAMETOOLONG sign
> - remove VFIOINTp virtualID in "add irq assignment" patch file
> - removed trace_vfio_platform_start_eventfd
>
> v12 -> v13:
> - header update but same update was already sent by Cornelia
> - Rework VFIO signaling & irqfd setup: restored 2-step setup featuring
> eventfd setup on realize and then irqfd setup on irq binding.
> - irqfd setup now uses kvm_irqchip_add_irqfd_notifier and
> sysbus irq_set_hook override. This leads to the introduction of 6 patch
> files enabling those 2 features. Paolo advised to introduce
> kvm_irqchip_add_irqfd_notifier series in the VFIO one. I did the
> same for irq_set_hook series but if it is better I can submit it aside.
> - above changes made possible to remove
> x hw/vfio/platform: add capability to start IRQ propagation"
> x hw/arm/virt: start VFIO IRQ propagation
> - in sysbus-fdt.c, use platform_bus_get_mmio_addr instead of deprecated
> mmio[0] property. Thanks to Bharat who pointed this issue out. also
> cpu_to_be32 was used for size and base (Vikram input) .
> - in skeleton misc corrections following Alex review.
>
> v11->v12:
> - add x-mmap property definition, without which the default value of
> vbasedev.allow_mmap is false, hence preventing the reg space from
> being mmapped.
>
> v10->v11:
> - rebase onto v2.3.0-rc0 (mainly related to PCIe support in virt)
> - add dma-coherent property for calxeda midway (fix revealed by removal
> of kernel-side "vfio: type1: support for ARM SMMUS with VFIO_IOMMU_TYPE1")
> - virt modifications to start VFIO IRQ forwarding are now in a separate
> patch
> - rearrange linux header exports (those are still partial exports
> waiting for definitive 4.1-rc0)
> - take into account Alex Bennée comments:
> - use g_malloc0_n instead of g_malloc0
> - use block declarations when possible
> - rework readlink returned value treatment
> - use g_strlcat in place strncat
> - re-arrange mutex locking for multiple IRQ support (user-side handled
> eventfds)
> - use g_snprintf instead of snprintf
> - change the order of functions to avoid pre-declaration in platform.c
> - add flags in VFIOINTp struct to detect whether the IRQ is automasked
> - some comment rewriting
>
> v9->v10:
> - rebase on "vfio: cleanup vfio_get_device error path, remove
> vfio_populate_device": vfio_populate_device no more called in
> vfio_get_device but in vfio_base_device_init
> - update VFIO header according to vfio platform driver v13 (no AMBA)
>
> v8->v9:
> - rebase on 2.2.0 and machvirt dynamic sysbus instantiation v10
> - v8 1-11 were pulled
> - patch files related to forwarding are moved in a seperate series since
> it depends on kernel series still in RFC.
> - introduction of basic VFIO platform device split into 3 patch files to
> ease the review (hope it will help).
> - add an author in platform.c
> - add deallocation in vfio_populate_device error case
> - add patch file doing the VFIO header sync
> - use VFIO_DEVICE_FLAGS_PLATFORM in vfio_populate_device
> - rename calxeda_xgmac.c into calxeda-xgmac.c
> - sysbus-fdt: add_calxeda_midway_xgmac_fdt_node g_free in case of errors
> - reword of linux-headers patch files
>
> v7->v8:
> - rebase on v2.2.0-rc3 and integrate
> "Add skip_dump flag to ignore memory region during dump"
> - KVM header evolution with subindex addition in kvm_arch_forwarded_irq
> - split [PATCH v7 03/16] hw/vfio/pci: introduce VFIODevice into 4 patches
> - vfio_compute_needs_reset does not return bool anymore
> - add some comments about exposed MMIO region and IRQ in calxeda xgmac
> device
> - vfio_[un]mask_irqindex renamed into vfio_[un]mask_single_irqindex
> - rework IRQ startup: former machine init done notifier is replaced by a
> reset notifier. machine file passes the interrupt controller
> DeviceState handle (not the platform bus first irq parameter).
> - sysbus-fdt:
> - move the add_fdt_node_functions array declaration between the device
> specific code and the generic code to avoid forward declarations of
> decice specific functions
> - rename add_basic_vfio_fdt_node into add_calxeda_midway_xgmac_fdt_node
> emphasizing the fact it is xgmac specific
>
> v6->v7:
> - fake injection test modality removed
> - VFIO_DEVICE_TYPE_PLATFORM only introduced with VFIO platform
> - new helper functions to start VFIO IRQ on machine init done notifier
> (introduced in hw/vfio/platform: add vfio-platform support and notifier
> registration invoked in hw/arm/virt: add support for VFIO devices).
> vfio_start_irq_injection is replaced by vfio_register_irq_starter.
>
> v5->v6:
> - rebase on 2.1rc5 PCI code
> - forwarded IRQ first integraton
> - vfio_device property renamed into host property
> - split IRQ setup in different functions that match the 3 supported
> injection techniques (user handled eventfd, irqfd, forwarded IRQ):
> removes dynamic switch between injection methods
> - introduce fake interrupts as a test modality:
> x makes possible to test multiple IRQ user-side handling.
> x this is a test feature only: enable to trigger a fd as if the
> real physical IRQ hit. No virtual IRQ is injected into the guest
> but handling is simulated so that the state machine can be tested
> - user handled eventfd:
> x add mutex to protect IRQ state & list manipulation,
> x correct misleading comment in vfio_intp_interrupt.
> x Fix bugs using fake interrupt modality
> - irqfd no more advertised in this patchset (handled in [3])
> - VFIOPlatformDeviceClass becomes abstract and Calxeda xgmac device
> and class is re-introduced (as per v4)
> - all DPRINTF removed in platform and replaced by trace-points
> - corrects compilation with configure --disable-kvm
> - simplifies the split for vfio_get_device and introduce a unique
> specialized function named vfio_populate_device
> - group_list renamed into vfio_group_list
> - hw/arm/dyn_sysbus_devtree.c currently only support vfio-calxeda-xgmac
> instantiation. Needs to be specialized for other VFIO devices
> - fix 2 bugs in dyn_sysbus_devtree(reg_attr index and compat)
>
> v4->v5:
> - rebase on v2.1.0 PCI code
> - take into account Alex Williamson comments on PCI code rework
> - trace updates in vfio_region_write/read
> - remove fd from VFIORegion
> - get/put ckeanup
> - bug fix: bar region's vbasedev field duly initialization
> - misc cleanups in platform device
> - device tree node generation removed from device and handled in
> hw/arm/dyn_sysbus_devtree.c
> - remove "hw/vfio: add an example calxeda_xgmac": with removal of
> device tree node generation we do not have so many things to
> implement in that derived device yet. May be re-introduced later
> on if needed typically for reset/migration.
> - no GSI routing table anymore
>
> v3->v4 changes (Eric Auger, Alvise Rigo)
> - rebase on last VFIO PCI code (v2.1.0-rc0)
> - full git history rework to ease PCI code change review
> - mv include files in hw/vfio
> - DPRINTF reformatting temporarily moved out
> - support of VFIO virq (removal of resamplefd handler on user-side)
> - integration with sysbus dynamic instantiation framwork
> - removal of unrealize and cleanup routines until it is better
> understood what is really needed
> - Support of VFIO for Amba devices should be handled in an inherited
> device to specialize the device tree generation (clock handle currently
> missing in framework however)
> - "Always use eventfd as notifying mechanism" temporarily moved out
> - static instantiation is not mainstream (although it remains possible)
> note if static instantiation is used, irqfd must be setup in machine file
> when virtual IRQ is known
> - create the GSI routing table on qemu side
>
> v2->v3 changes (Alvise Rigo, Eric Auger):
> - Following Alex W recommandations, further efforts to factorize the
> code between PCI:introduction of VFIODevice and VFIORegion
> as base classes
> - unique reset handler for platform and PCI
> - cleanup following Kim's comments
> - multiple IRQ support mechanics should be in place although not
> tested
> - Better handling of MMIO multiple regions
> - New features and fixes by Alvise (multiple compat string, exec
> flag, force eventfd usage, amba device tree support)
> - irqfd support
>
> v1->v2 changes (Kim Phillips, Eric Auger):
> - IRQ initial support (legacy mode where eventfds are handled on
> user side)
> - hacked dynamic instantiation
>
> v1 (Kim Phillips):
> - initial split between PCI and platform
> - MMIO support only
> - static instantiation
>
> Best Regards
>
> Eric
>
>
> Cornelia Huck (1):
> linux-headers: update
>
> Eric Auger (9):
> hw/vfio/platform: vfio-platform skeleton
> hw/vfio/platform: add irq assignment
> hw/vfio/platform: calxeda xgmac device
> hw/arm/sysbus-fdt: enable vfio-calxeda-xgmac dynamic instantiation
> kvm: rename kvm_irqchip_[add,remove]_irqfd_notifier with gsi suffix
> kvm-all.c: add qemu_irq/gsi hash table and utility routines
> intc: arm_gic_kvm: set the qemu_irq/gsi mapping
> sysbus: add irq_routing_notifier
> hw/vfio/platform: add irqfd support
>
> hw/arm/sysbus-fdt.c | 72 +++
> hw/core/sysbus.c | 6 +
> hw/intc/arm_gic_kvm.c | 5 +
> hw/s390x/virtio-ccw.c | 8 +-
> hw/vfio/Makefile.objs | 2 +
> hw/vfio/calxeda-xgmac.c | 54 ++
> hw/vfio/pci.c | 6 +-
> hw/vfio/platform.c | 723
> ++++++++++++++++++++++++
> hw/virtio/virtio-pci.c | 4 +-
> include/hw/sysbus.h | 1 +
> include/hw/vfio/vfio-calxeda-xgmac.h | 46 ++
> include/hw/vfio/vfio-common.h | 1 +
> include/hw/vfio/vfio-platform.h | 77 +++
> include/standard-headers/linux/virtio_balloon.h | 28 +-
> include/standard-headers/linux/virtio_blk.h | 8 +-
> include/standard-headers/linux/virtio_ids.h | 1 +
> include/standard-headers/linux/virtio_input.h | 76 +++
> include/sysemu/kvm.h | 11 +-
> kvm-all.c | 44 +-
> kvm-stub.c | 7 +-
> linux-headers/asm-arm/kvm.h | 9 +-
> linux-headers/asm-arm64/kvm.h | 9 +-
> linux-headers/asm-mips/kvm.h | 164 +++---
> linux-headers/asm-s390/kvm.h | 4 +
> linux-headers/asm-x86/hyperv.h | 2 +
> linux-headers/linux/kvm.h | 66 ++-
> linux-headers/linux/vfio.h | 2 +
> trace-events | 13 +
> 28 files changed, 1362 insertions(+), 87 deletions(-)
> create mode 100644 hw/vfio/calxeda-xgmac.c
> create mode 100644 hw/vfio/platform.c
> create mode 100644 include/hw/vfio/vfio-calxeda-xgmac.h
> create mode 100644 include/hw/vfio/vfio-platform.h
> create mode 100644 include/standard-headers/linux/virtio_input.h
>
> -- 1.8.3.2
>
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu