Re: [PATCH for-5.0 v11 00/20] VIRTIO-IOMMU device

[Auger Eric <eric.auger@redhat.com>]

Hi Michael,

On 12/11/19 9:40 PM, Michael S. Tsirkin wrote:
> On Wed, Dec 11, 2019 at 05:48:05PM +0100, Auger Eric wrote:
>> Hi Michael,
>>
>> On 12/11/19 5:40 PM, Michael S. Tsirkin wrote:
>>> On Fri, Nov 22, 2019 at 07:29:23PM +0100, Eric Auger wrote:
>>>> This series implements the QEMU virtio-iommu device.
>>>>
>>>> This matches the v0.12 spec and the corresponding virtio-iommu
>>>> driver upstreamed in 5.3.
>>>>
>>>> The pci proxy for the virtio-iommu device is instantiated using
>>>> "-device virtio-iommu-pci". This series still relies on ACPI IORT/DT
>>>> integration. Note the ACPI IORT integration is not yet upstreamed
>>>> and testing needs to be based on Jean-Philippe's additional
>>>> kernel patches [1].
>>>
>>> Or the config space approach? I really liked that one.
>> Yes this corresponds to the paragraph below.
>>>
>>>>
>>>> Work is ongoing to remove IORT adherence and allow the
>>>> bindings between the IOMMU and the root complex to be defined
>>>> and written into the PCI device configuration space. The outcome
>>>> of this work is uncertain at this stage though. See [2].
> 
> Oh right. Why is it uncertain? Anything can be done to help?

Jean's series was sent on the same day as this QEMU respin. My
understanding is we still need a way to handle platform devices. Also
the binding info layout needs to be revised and integrated into the spec
+ voted. Those are the uncertainties I wanted to point out.

Thanks

Eric
> 
>> Thanks
>>
>> Eric
>>
>>>>
>>>> So only patches 1-11 fully rely on upstreamed kernel code. Others
>>>> should be considered as RFC.
>>>>
>>>> This respin allows people to test on ARM and x86. It also
>>>> brings migration support (tested on ARM) and various cleanups.
>>>> Reserved regions are now passed through an array of properties.
>>>> A libqos test also is introduced to test the virtio-iommu API.
>>>>
>>>> Note integration with vhost devices and vfio devices is not part
>>>> of this series. Please follow Bharat's respins [3].
>>>>
>>>> The 1st Patch ("migration: Support QLIST migration") was sent
>>>> separately [4].
>>>>
>>>> Best Regards
>>>>
>>>> Eric
>>>>
>>>> This series can be found at:
>>>> https://github.com/eauger/qemu/tree/v4.2-rc2-virtio-iommu-v11
>>>>
>>>> [1] kernel branch to be used for guest
>>>>     https://github.com/eauger/linux/tree/v5.4-rc8-virtio-iommu-iort
>>>> [2] [RFC 00/13] virtio-iommu on non-devicetree platforms
>>>> [3] VFIO/VHOST integration is not part of this series. Please follow
>>>>     [PATCH RFC v5 0/5] virtio-iommu: VFIO integration respins
>>>> [4] [PATCH v6] migration: Support QLIST migration
>>>>
>>>> Testing:
>>>> - tested with guest using virtio-net-pci
>>>>   (,vhost=off,iommu_platform,disable-modern=off,disable-legacy=on)
>>>>   and virtio-blk-pci
>>>> - migration on ARM
>>>> - on x86 PC machine I get some AHCI non translated transactions,
>>>>   very early. This does not prevent the guest from boot and behaving
>>>>   properly. Warnings look like:
>>>> qemu-system-x86_64: virtio_iommu_translate sid=250 is not known!!
>>>> qemu-system-x86_64: no buffer available in event queue to report event
>>>> qemu-system-x86_64: AHCI: Failed to start FIS receive engine: bad FIS
>>>> receive buffer address
>>>>
>>>> History:
>>>>
>>>> v10 -> v11:
>>>> - introduce virtio_iommu_handle_req macro
>>>> - migration support
>>>> - introduce DEFINE_PROP_INTERVAL and pass reserved regions
>>>>   through an array of those
>>>> - domain gtree simplification
>>>>
>>>> v9 -> v10:
>>>> - rebase on 4.1.0-rc2, compliance with 0.12 spec
>>>> - removed ACPI part
>>>> - cleanup (see individual change logs)
>>>> - moved to a PATCH series
>>>>
>>>> v8 -> v9:
>>>> - virtio-iommu-pci device needs to be instantiated from the command
>>>>   line (RID is not imposed anymore).
>>>> - tail structure properly initialized
>>>>
>>>> v7 -> v8:
>>>> - virtio-iommu-pci added
>>>> - virt instantiation modified
>>>> - DT and ACPI modified to exclude the iommu RID from the mapping
>>>> - VIRTIO_IOMMU_F_BYPASS, VIRTIO_F_VERSION_1 features exposed
>>>>
>>>> v6 -> v7:
>>>> - rebase on qemu 3.0.0-rc3
>>>> - minor update against v0.7
>>>> - fix issue with EP not on pci.0 and ACPI probing
>>>> - change the instantiation method
>>>>
>>>> v5 -> v6:
>>>> - minor update against v0.6 spec
>>>> - fix g_hash_table_lookup in virtio_iommu_find_add_as
>>>> - replace some error_reports by qemu_log_mask(LOG_GUEST_ERROR, ...)
>>>>
>>>> v4 -> v5:
>>>> - event queue and fault reporting
>>>> - we now return the IOAPIC MSI region if the virtio-iommu is instantiated
>>>>   in a PC machine.
>>>> - we bypass transactions on MSI HW region and fault on reserved ones.
>>>> - We support ACPI boot with mach-virt (based on IORT proposal)
>>>> - We moved to the new driver naming conventions
>>>> - simplified mach-virt instantiation
>>>> - worked around the disappearing of pci_find_primary_bus
>>>> - in virtio_iommu_translate, check the dev->as is not NULL
>>>> - initialize as->device_list in virtio_iommu_get_as
>>>> - initialize bufstate.error to false in virtio_iommu_probe
>>>>
>>>> v3 -> v4:
>>>> - probe request support although no reserved region is returned at
>>>>   the moment
>>>> - unmap semantics less strict, as specified in v0.4
>>>> - device registration, attach/detach revisited
>>>> - split into smaller patches to ease review
>>>> - propose a way to inform the IOMMU mr about the page_size_mask
>>>>   of underlying HW IOMMU, if any
>>>> - remove warning associated with the translation of the MSI doorbell
>>>>
>>>> v2 -> v3:
>>>> - rebase on top of 2.10-rc0 and especially
>>>>   [PATCH qemu v9 0/2] memory/iommu: QOM'fy IOMMU MemoryRegion
>>>> - add mutex init
>>>> - fix as->mappings deletion using g_tree_ref/unref
>>>> - when a dev is attached whereas it is already attached to
>>>>   another address space, first detach it
>>>> - fix some error values
>>>> - page_sizes = TARGET_PAGE_MASK;
>>>> - I haven't changed the unmap() semantics yet, waiting for the
>>>>   next virtio-iommu spec revision.
>>>>
>>>> v1 -> v2:
>>>> - fix redefinition of viommu_as typedef
>>>>
>>>>
>>>>
>>>> Eric Auger (20):
>>>>   migration: Support QLIST migration
>>>>   virtio-iommu: Add skeleton
>>>>   virtio-iommu: Decode the command payload
>>>>   virtio-iommu: Add the iommu regions
>>>>   virtio-iommu: Endpoint and domains structs and helpers
>>>>   virtio-iommu: Implement attach/detach command
>>>>   virtio-iommu: Implement map/unmap
>>>>   virtio-iommu: Implement translate
>>>>   virtio-iommu: Implement fault reporting
>>>>   virtio-iommu-pci: Add virtio iommu pci support
>>>>   hw/arm/virt: Add the virtio-iommu device tree mappings
>>>>   qapi: Introduce DEFINE_PROP_INTERVAL
>>>>   virtio-iommu: Implement probe request
>>>>   virtio-iommu: Handle reserved regions in the translation process
>>>>   virtio-iommu-pci: Add array of Interval properties
>>>>   hw/arm/virt-acpi-build: Introduce fill_iort_idmap helper
>>>>   hw/arm/virt-acpi-build: Add virtio-iommu node in IORT table
>>>>   virtio-iommu: Support migration
>>>>   pc: Add support for virtio-iommu-pci
>>>>   tests: Add virtio-iommu test
>>>>
>>>>  hw/arm/virt-acpi-build.c         |  91 ++-
>>>>  hw/arm/virt.c                    |  53 +-
>>>>  hw/core/qdev-properties.c        |  90 +++
>>>>  hw/i386/acpi-build.c             |  72 +++
>>>>  hw/i386/pc.c                     |  15 +-
>>>>  hw/virtio/Kconfig                |   5 +
>>>>  hw/virtio/Makefile.objs          |   2 +
>>>>  hw/virtio/trace-events           |  22 +
>>>>  hw/virtio/virtio-iommu-pci.c     |  91 +++
>>>>  hw/virtio/virtio-iommu.c         | 952 +++++++++++++++++++++++++++++++
>>>>  include/exec/memory.h            |   6 +
>>>>  include/hw/acpi/acpi-defs.h      |  21 +-
>>>>  include/hw/arm/virt.h            |   2 +
>>>>  include/hw/i386/pc.h             |   2 +
>>>>  include/hw/pci/pci.h             |   1 +
>>>>  include/hw/qdev-properties.h     |   3 +
>>>>  include/hw/virtio/virtio-iommu.h |  67 +++
>>>>  include/migration/vmstate.h      |  21 +
>>>>  include/qemu/queue.h             |  39 ++
>>>>  include/qemu/typedefs.h          |   1 +
>>>>  migration/trace-events           |   5 +
>>>>  migration/vmstate-types.c        |  70 +++
>>>>  qdev-monitor.c                   |   1 +
>>>>  tests/Makefile.include           |   2 +
>>>>  tests/libqos/virtio-iommu.c      | 177 ++++++
>>>>  tests/libqos/virtio-iommu.h      |  45 ++
>>>>  tests/test-vmstate.c             | 170 ++++++
>>>>  tests/virtio-iommu-test.c        | 261 +++++++++
>>>>  28 files changed, 2253 insertions(+), 34 deletions(-)
>>>>  create mode 100644 hw/virtio/virtio-iommu-pci.c
>>>>  create mode 100644 hw/virtio/virtio-iommu.c
>>>>  create mode 100644 include/hw/virtio/virtio-iommu.h
>>>>  create mode 100644 tests/libqos/virtio-iommu.c
>>>>  create mode 100644 tests/libqos/virtio-iommu.h
>>>>  create mode 100644 tests/virtio-iommu-test.c
>>>>
>>>> -- 
>>>> 2.20.1
>>>
>>>
> 
>