Re: [PATCH v3 0/4] Support dynamic MSI-X allocation

["Cédric Le Goater" <clg@redhat.com>]

On 9/26/23 04:14, Jing Liu wrote:
> Changes since v2:
> - v2: https://www.mail-archive.com/qemu-devel@nongnu.org/msg989852.html
> - Use a bool type to test (vdev->nr_vectors < nr + 1). (Alex)
> - Revise comments. (Alex)
> - Apply Cédric's and Alex's Reviewed-by.
> 
> Changes since v1:
> - v1: https://www.mail-archive.com/qemu-devel@nongnu.org/msg982842.html
> - Revise Qemu to QEMU. (Cédric)
> - Add g_free when failure of getting MSI-X irq info. (Cédric)
> - Apply Cédric's Reviewed-by. (Cédric)
> - Use g_autofree to automatically release. (Cédric)
> - Remove the failure message in vfio_enable_msix_no_vec(). (Cédric)
> 
> Changes since RFC v1:
> - RFC v1: https://www.mail-archive.com/qemu-devel@nongnu.org/msg978637.html
> - Revise the comments. (Alex)
> - Report error of getting irq info and remove the trace of failure
>    case. (Alex, Cédric)
> - Only store dynamic allocation flag as a bool type and test
>    accordingly. (Alex)
> - Move dynamic allocation detection to vfio_msix_early_setup(). (Alex)
> - Change the condition logic in vfio_msix_vector_do_use() that moving
>    the defer_kvm_irq_routing test out and create a common place to update
>    nr_vectors. (Alex)
> - Consolidate the way of MSI-X enabling during device initialization and
>    interrupt restoring that uses fd = -1 trick. Create a function doing
>    that. (Alex)
> 
> Before kernel v6.5, dynamic allocation of MSI-X interrupts was not
> supported. QEMU therefore when allocating a new interrupt, should first
> release all previously allocated interrupts (including disable of MSI-X)
> and re-allocate all interrupts that includes the new one.
> 
> The kernel series [1] adds the support of dynamic MSI-X allocation to
> vfio-pci and uses the existing flag VFIO_IRQ_INFO_NORESIZE to guide user
> space, that when dynamic MSI-X is supported the flag is cleared.
> 
> This series makes the behavior for VFIO PCI devices when dynamic MSI-X
> allocation is supported. When guest unmasks an interrupt, QEMU can
> directly allocate an interrupt on host for this and has nothing to do
> with the previously allocated ones. Therefore, host only allocates
> interrupts for those unmasked (enabled) interrupts inside guest when
> dynamic MSI-X allocation is supported by device.
> 
> When guests enable MSI-X with all of the vectors masked, QEMU need match
> the state to enable MSI-X with no vector enabled. During migration
> restore, QEMU also need enable MSI-X first in dynamic allocation mode,
> to avoid the guest unused vectors being allocated on host. To
> consolidate them, we use vector 0 with an invalid fd to get MSI-X
> enabled and create a common function for this. This is cleaner than
> setting userspace triggering and immediately release.
> 
> Any feedback is appreciated.
> 
> Jing
> 
> [1] https://lwn.net/Articles/931679/
> 
> Jing Liu (4):
>    vfio/pci: detect the support of dynamic MSI-X allocation
>    vfio/pci: enable vector on dynamic MSI-X allocation
>    vfio/pci: use an invalid fd to enable MSI-X
>    vfio/pci: enable MSI-X in interrupt restoring on dynamic allocation
> 
>   hw/vfio/pci.c        | 123 +++++++++++++++++++++++++++++++++----------
>   hw/vfio/pci.h        |   1 +
>   hw/vfio/trace-events |   2 +-
>   3 files changed, 97 insertions(+), 29 deletions(-)
> 

Applied to vfio-next.

Thanks,

C.