Re: [Qemu-devel] [PATCH v2] msix: don't mask already masked vectors on reset

[Marcel Apfelbaum <marcel@redhat.com>]

On 21/12/2017 17:08, Marcel Apfelbaum wrote:
> On 21/12/2017 16:01, Michael S. Tsirkin wrote:
>> On Fri, Dec 08, 2017 at 09:09:06AM -0500, Marcel Apfelbaum wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Ladi Prosek" <lprosek@redhat.com>
>>>> To: qemu-devel@nongnu.org
>>>> Cc: marcel@redhat.com, geoff@hostfission.com, mst@redhat.com, "alex williamson" <alex.williamson@redhat.com>
>>>> Sent: Friday, December 8, 2017 8:31:13 AM
>>>> Subject: [PATCH v2] msix: don't mask already masked vectors on reset
>>>>
>>>> msix_mask_all() is supposed to invoke the release vector notifier if the
>>>> state of the
>>>> respective vector changed from unmasked to masked. The way it's currently
>>>> called from
>>>> msix_reset(), though, may result in calling the release notifier even if the
>>>> vector
>>>> is already masked.
>>>>
>>>> 1) msix_reset() clears out the msix_cap field and the msix_table.
>>>> 2) msix_mask_all() runs with was_masked=false for all vectors because of 1),
>>>> which
>>>>     results in calling the release notifier on all vectors.
>>>> 3) if msix_reset() is subsequently called again, it goes through the same
>>>> steps and
>>>>     calls the release notifier on all vectors again.
>>>>
>>>> This commit moves msix_mask_all() up so it runs before the device state is
>>>> lost. And
>>>> it adds an assignment to msix_function_masked so that the device remembers
>>>> that
>>>> MSI-X is masked.
>>>>
>>>> This is likely a low impact issue, found while debugging an already broken
>>>> device. It
>>>> is however easy to fix and the expectation that the use and release notifier
>>>> invocations
>>>> are always balanced is very natural.
>>>>
>>>> Signed-off-by: Ladi Prosek <lprosek@redhat.com>
>>>> ---
>>>>
>>>> v1->v2:
>>>> * fixed typo in commit message "or" -> "to" (Marcel)
>>>> * directly set msix_function_masked to true instead of calling
>>>>    msix_update_function_masked() (Marcel)
>>>>
>>>>
>>>>   hw/pci/msix.c | 3 ++-
>>>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/hw/pci/msix.c b/hw/pci/msix.c
>>>> index c944c02135..d6a4dbdb6b 100644
>>>> --- a/hw/pci/msix.c
>>>> +++ b/hw/pci/msix.c
>>>> @@ -500,11 +500,12 @@ void msix_reset(PCIDevice *dev)
>>>>           return;
>>>>       }
>>>>       msix_clear_all_vectors(dev);
>>>> +    msix_mask_all(dev, dev->msix_entries_nr);
>>>>       dev->config[dev->msix_cap + MSIX_CONTROL_OFFSET] &=
>>>>           ~dev->wmask[dev->msix_cap + MSIX_CONTROL_OFFSET];
>>>>       memset(dev->msix_table, 0, dev->msix_entries_nr * PCI_MSIX_ENTRY_SIZE);
>>>>       memset(dev->msix_pba, 0, QEMU_ALIGN_UP(dev->msix_entries_nr, 64) / 8);
>>>> -    msix_mask_all(dev, dev->msix_entries_nr);
>>>> +    dev->msix_function_masked = true;
>>>>   }
>>>>   /* PCI spec suggests that devices make it possible for software to configure
>>>> -- 
>>>> 2.13.6
>>>>
>>>>
>>>
>>> Thanks Ladi!
>>>
>>> Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
>>
>> This breaks make check though.
>>
>> Marcel - could you take a look please?
>>
> 
> Sure

We have a problem with the patch.

Moving msix_mask_all before resetting the msix_table to 0
modifies behavior since we don't set the PCI_MSIX_ENTRY_CTRL_MASKBIT
for each vector anymore (the information is lost).

What we need (I think) is to run msix_mask_all again,
this time without calling msix_handle_mask_update (only setting the bit).
(I tested it and the make check passes after it)

I think is risky to use the patch for now.
Thanks,
Marcel


> 
> Thanks,
> Marcel