Re: [PATCH v11 5/5] vpci/msix: Implement cleanup function for MSI-X

["Roger Pau Monné" <roger.pau@citrix.com>]

On Fri, Aug 08, 2025 at 04:03:37PM +0800, Jiqian Chen wrote:
> When MSI-X initialization fails, vPCI hides the capability, but
> removing handlers and datas won't be performed until the device is
> deassigned. So, implement MSI-X cleanup hook that will be called
> to cleanup MSI-X related handlers and free it's associated data when
> initialization fails.
> 
> Since cleanup function of MSI-X is implemented, delete the open-code
> in vpci_deassign_device().
> 
> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> ---
> cc: "Roger Pau Monné" <roger.pau@citrix.com>
> ---
> v10->v11 changes:
> * Move calling all cleanup hook in vpci_deassign_device() out of this patch.
> * Add hide parameter to cleanup_msix().
> * Check hide, if it is false, return directly instead of letting ctrl RO.
> 
> v9->v10 changes:
> * Call all cleanup hook in vpci_deassign_device() instead of cleanup_msix().
> 
> v8->v9 changes:
> * Modify commit message.
> * Call cleanup_msix() in vpci_deassign_device() to remove the open-code to cleanup msix datas.
> * In cleanup_msix(), move "list_del(&vpci->msix->next);" above for loop of iounmap msix tables.
> 
> v7->v8 changes:
> * Given the code in vpci_remove_registers() an error in the removal of
>   registers would likely imply memory corruption, at which point it's
>   best to fully disable the device. So, Rollback the last two modifications of v7.
> 
> v6->v7 changes:
> * Change the pointer parameter of cleanup_msix() to be const.
> * When vpci_remove_registers() in cleanup_msix() fails, not to return
>   directly, instead try to free msix and re-add ctrl handler.
> * Pass pdev->vpci into vpci_add_register() instead of pdev->vpci->msix in
>   init_msix() since we need that every handler realize that msix is NULL
>   when msix is freed but handlers are still in there.
> 
> v5->v6 changes:
> * Change the logic to add dummy handler when !vpci->msix in cleanup_msix().
> 
> v4->v5 changes:
> * Change definition "static void cleanup_msix" to "static int cf_check cleanup_msix"
>   since cleanup hook is changed to be int.
> * Add a read-only register for MSIX Control Register in the end of cleanup_msix().
> 
> v3->v4 changes:
> * Change function name from fini_msix() to cleanup_msix().
> * Change to use XFREE to free vpci->msix.
> * In cleanup function, change the sequence of check and remove action according to
>   init_msix().
> 
> v2->v3 changes:
> * Remove unnecessary clean operations in fini_msix().
> 
> v1->v2 changes:
> new patch.
> 
> Best regards,
> Jiqian Chen.
> ---
>  xen/drivers/vpci/msix.c | 47 ++++++++++++++++++++++++++++++++++++++++-
>  xen/drivers/vpci/vpci.c |  8 -------
>  2 files changed, 46 insertions(+), 9 deletions(-)
> 
> diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c
> index 54a5070733aa..287aafda9157 100644
> --- a/xen/drivers/vpci/msix.c
> +++ b/xen/drivers/vpci/msix.c
> @@ -655,6 +655,51 @@ int vpci_make_msix_hole(const struct pci_dev *pdev)
>      return 0;
>  }
>  
> +static int cf_check cleanup_msix(const struct pci_dev *pdev, bool hide)
> +{
> +    int rc;
> +    struct vpci *vpci = pdev->vpci;
> +    const unsigned int msix_pos = pdev->msix_pos;
> +
> +    if ( !msix_pos )
> +        return 0;

Like with the MSI capability, is it possible to get called and
pdev->msix_pos be 0?

I would also avoid the call to vpci_remove_registers() if !hide, I
think you can change the order of the cleanup, so it's:

if ( vpci->msix )
{
    list_del(&vpci->msix->next);
    for ( unsigned int i = 0; i < ARRAY_SIZE(vpci->msix->table); i++ )
        if ( vpci->msix->table[i] )
            iounmap(vpci->msix->table[i]);

    XFREE(vpci->msix);
}

if ( !hide )
    return 0;

rc = vpci_remove_registers(vpci, msix_control_reg(msix_pos), 2);
...

Thanks, Roger.