Re: [PATCH 0/9] vfio/fsl-mc: VFIO support for FSL-MC devices

[Diana Craciun OSS <diana.craciun@oss.nxp.com>]

Thanks for looking into this.

On 3/27/2020 11:11 PM, Alex Williamson wrote:
> On Mon, 23 Mar 2020 19:19:02 +0200
> Diana Craciun <diana.craciun@oss.nxp.com> wrote:
>
>> DPAA2 (Data Path Acceleration Architecture) consists in
>> mechanisms for processing Ethernet packets, queue management,
>> accelerators, etc.
>>
>> The Management Complex (mc) is a hardware entity that manages the DPAA2
>> hardware resources. It provides an object-based abstraction for software
>> drivers to use the DPAA2 hardware. The MC mediates operations such as
>> create, discover, destroy of DPAA2 objects.
>> The MC provides memory-mapped I/O command interfaces (MC portals) which
>> DPAA2 software drivers use to operate on DPAA2 objects.
>>
>> A DPRC is a container object that holds other types of DPAA2 objects.
>> Each object in the DPRC is a Linux device and bound to a driver.
>> The MC-bus driver is a platform driver (different from PCI or platform
>> bus). The DPRC driver does runtime management of a bus instance. It
>> performs the initial scan of the DPRC and handles changes in the DPRC
>> configuration (adding/removing objects).
>>
>> All objects inside a container share the same hardware isolation
>> context, meaning that only an entire DPRC can be assigned to
>> a virtual machine.
>> When a container is assigned to a virtual machine, all the objects
>> within that container are assigned to that virtual machine.
>> The DPRC container assigned to the virtual machine is not allowed
>> to change contents (add/remove objects) by the guest. The restriction
>> is set by the host and enforced by the mc hardware.
>>
>> The DPAA2 objects can be directly assigned to the guest. However
>> the MC portals (the memory mapped command interface to the MC) need
>> to be emulated because there are commands that configure the
>> interrupts and the isolation IDs which are virtual in the guest.
>>
>> Example:
>> echo vfio-fsl-mc > /sys/bus/fsl-mc/devices/dprc.2/driver_override
>> echo dprc.2 > /sys/bus/fsl-mc/drivers/vfio-fsl-mc/bind
>>
>> The dprc.2 is bound to the VFIO driver and all the objects within
>> dprc.2 are going to be bound to the VFIO driver.
> What's the composition of the IOMMU group, does it start with the DPRC
> and each of the objects within the container are added to the same
> group as they're created?

Yes, the IOMMU group starts with the DPRC and the other objects are then 
added to that group.

>
> For an alternative to the driver_override mechanism used in this series
> of passing the override through various scan/create callbacks, you
> might consider something like I did for PCI SR-IOV:
>
> https://lore.kernel.org/lkml/158396395214.5601.11207416598267070486.stgit@gimli.home/
>
> ie. using the bus notifier to setup the driver_override before driver
> matching is done.  Thanks,

Thanks, I like your approach. I will give it a try.

Diana


>
> Alex
>
>> More details about the DPAA2 objects can be found here:
>> Documentation/networking/device_drivers/freescale/dpaa2/overview.rst
>>
>> The patches are dependent on some changes in the mc-bus (bus/fsl-mc)
>> driver. The changes were needed in order to re-use code and to export
>> some more functions that are needed by the VFIO driver.
>> Currenlty the mc-bus patches are under review:
>> https://www.spinics.net/lists/kernel/msg3447567.html
>>
>> Bharat Bhushan (1):
>>    vfio/fsl-mc: Add VFIO framework skeleton for fsl-mc devices
>>
>> Diana Craciun (8):
>>    vfio/fsl-mc: Scan DPRC objects on vfio-fsl-mc driver bind
>>    vfio/fsl-mc: Implement VFIO_DEVICE_GET_INFO ioctl
>>    vfio/fsl-mc: Implement VFIO_DEVICE_GET_REGION_INFO ioctl call
>>    vfio/fsl-mc: Allow userspace to MMAP fsl-mc device MMIO regions
>>    vfio/fsl-mc: Added lock support in preparation for interrupt handling
>>    vfio/fsl-mc: Add irq infrastructure for fsl-mc devices
>>    vfio/fsl-mc: trigger an interrupt via eventfd
>>    vfio/fsl-mc: Add read/write support for fsl-mc devices
>>
>>   MAINTAINERS                               |   6 +
>>   drivers/vfio/Kconfig                      |   1 +
>>   drivers/vfio/Makefile                     |   1 +
>>   drivers/vfio/fsl-mc/Kconfig               |   9 +
>>   drivers/vfio/fsl-mc/Makefile              |   4 +
>>   drivers/vfio/fsl-mc/vfio_fsl_mc.c         | 660 ++++++++++++++++++++++
>>   drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c    | 221 ++++++++
>>   drivers/vfio/fsl-mc/vfio_fsl_mc_private.h |  56 ++
>>   include/uapi/linux/vfio.h                 |   1 +
>>   9 files changed, 959 insertions(+)
>>   create mode 100644 drivers/vfio/fsl-mc/Kconfig
>>   create mode 100644 drivers/vfio/fsl-mc/Makefile
>>   create mode 100644 drivers/vfio/fsl-mc/vfio_fsl_mc.c
>>   create mode 100644 drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c
>>   create mode 100644 drivers/vfio/fsl-mc/vfio_fsl_mc_private.h
>>


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel