Re: [Qemu-devel] [PATCH 3/3] pcie: work around for racy guest init

[Marcel Apfelbaum <marcel.apfelbaum@gmail.com>]

CCing qemu-devel

On 7/1/19 11:57 AM, Igor Mammedov wrote:
> On Mon, 1 Jul 2019 10:04:01 +0300
> Marcel Apfelbaum <marcel.apfelbaum@gmail.com> wrote:
>
>> On 6/21/19 9:46 AM, Michael S. Tsirkin wrote:
>>> During boot, linux guests tend to clear all bits in pcie slot status
>>> register which is used for hotplug.
>>> If they clear bits that weren't set this is racy and will lose events:
>>> not a big problem for manual hotplug on bare-metal, but a problem for us.
>>>
>>> For example, the following is broken ATM:
>>>
>>> /x86_64-softmmu/qemu-system-x86_64 -enable-kvm -S -machine q35  \
>>>       -device pcie-root-port,id=pcie_root_port_0,slot=2,chassis=2,addr=0x2,bus=pcie.0 \
>>>       -device virtio-balloon-pci,id=balloon,bus=pcie_root_port_0 \
>>>       -monitor stdio disk.qcow2
>>> (qemu)device_del balloon
>>> (qemu)cont
>>>
>>> Balloon isn't deleted as it should.
>>>
>>> As a work-around, detect this attempt to clear slot status and revert
>>> status to what it was before the write.
>>>
>>> Note: in theory this can be detected as a duplicate button press
>>> which cancels the previous press. Does not seem to happen in
>>> practice as guests seem to only have this bug during init.
>>>
>>> Note2: the right thing to do is probably to fix Linux to
>>> read status before clearing it, and act on the bits that are set.
>>>
>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>> ---
>>>    hw/pci/pcie.c | 19 +++++++++++++++++++
>>>    1 file changed, 19 insertions(+)
>>>
>>> diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c
>>> index f8490a00de..c605d32dd4 100644
>>> --- a/hw/pci/pcie.c
>>> +++ b/hw/pci/pcie.c
>>> @@ -610,6 +610,25 @@ void pcie_cap_slot_write_config(PCIDevice *dev, uint16_t slt_ctl, uint16_t slt_s
>>>        uint16_t sltsta = pci_get_word(exp_cap + PCI_EXP_SLTSTA);
>>>    
>>>        if (ranges_overlap(addr, len, pos + PCI_EXP_SLTSTA, 2)) {
>>> +        /*
>>> +         * Guests tend to clears all bits during init.
>>> +         * If they clear bits that weren't set this is racy and will lose events:
>>> +         * not a big problem for manual button presses, but a problem for us.
>>> +         * As a work-around, detect this and revert status to what it was
>>> +         * before the write.
>>> +         *
>>> +         * Note: in theory this can be detected as a duplicate button press
>>> +         * which cancels the previous press. Does not seem to happen in
>>> +         * practice as guests seem to only have this bug during init.
>>> +         */
>>> +#define PCIE_SLOT_EVENTS (PCI_EXP_SLTSTA_ABP | PCI_EXP_SLTSTA_PFD | \
>>> +                          PCI_EXP_SLTSTA_MRLSC | PCI_EXP_SLTSTA_PDC | \
>>> +                          PCI_EXP_SLTSTA_CC)
>>> +
>>> +        if (val & ~slt_sta & PCIE_SLOT_EVENTS) {
>>> +            sltsta = (sltsta & ~PCIE_SLOT_EVENTS) | (slt_sta & PCIE_SLOT_EVENTS);
>>> +            pci_set_word(exp_cap + PCI_EXP_SLTSTA, sltsta);
>>> +        }
>>>            hotplug_event_clear(dev);
>>>        }
>>>      
>> Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
> Hi Marcel,
>
> What about my question about
>     sltsta = (sltsta & ~PCIE_SLOT_EVENTS) | (slt_sta & PCIE_SLOT_EVENTS);
> being nop like
>     sltsta = LOWER_PART(sltsta) | UPPER_PART(sltsta)
>
> Can you point out what I'm missing here?

Oops, I missed that, maybe it should looks like:

     if (...) {
       sltsta = (val & ~PCIE_SLOT_EVENTS) | (sltsta & PCIE_SLOT_EVENTS);
       ....
   }

to keep the PCIE_SLOT_EVENTS that were set before the write.


Thanks,
Marcel

>> Thanks,
>> Marcel
>>