Re: [PATCH V3 2/2] Xen/PCIback: Implement PCI flr/slot/bus reset with 'reset' SysFS attribute

Govinda Tatti
2017-12-12T15:01:20Z

Thanks Jan for your review comments. Please see below for my comments.

On 12/8/2017 3:34 AM, Jan Beulich wrote:
>>>> On 07.12.17 at 23:21,<Govinda.Tatti@Oracle.COM> wrote:
>> Due to the complexity with the PCI lock we cannot do the reset when a
>> device is bound ('echo $BDF > bind') or when unbound ('echo $BDF > unbind')
>> as the pci_[slot|bus]_reset also takes the same lock resulting in a
>> dead-lock.
> It took me a moment to figure that here you're referring to the
> process of (un)binding, not the state. To avoid that ambiguity in
> wording, how about "... we cannot do the reset while a device is
> being bound (...) or while it is being unbound ..."?
Sure, I will fix it.
>> --- a/Documentation/ABI/testing/sysfs-driver-pciback
>> +++ b/Documentation/ABI/testing/sysfs-driver-pciback
>> @@ -11,3 +11,18 @@ Description:
>> #echo 00:19.0-E0:2:FF > /sys/bus/pci/drivers/pciback/quirks
>> will allow the guest to read and write to the configuration
>> register 0x0E.
>> +
>> +What: /sys/bus/pci/drivers/pciback/reset
>> +Date: Dec 2017
>> +KernelVersion: 4.15
>> +Contact:xen-devel@lists.xenproject.org
>> +Description:
>> + An option to perform a flr/slot/bus reset when a PCI device
>> + is owned by Xen PCI backend. Writing a string of DDDD:BB:DD.F
> SSSS:BB:DD.F (or else the D-s are ambiguous, the more that "domain"
> in Xen code is ambiguous anyway - I continue to be mislead by struct
> pcistub_device_id's domain field)
Thanks for catching this issue. I will fix it.
> Also I assume the SSSS part is optional (default zero), which
> probably can and should be expressed in some way.
SSSS can be 0 or non-zero, subject to system configuration.
>> --- a/drivers/xen/xen-pciback/pci_stub.c
>> +++ b/drivers/xen/xen-pciback/pci_stub.c
>> @@ -313,6 +313,102 @@ void pcistub_put_pci_dev(struct pci_dev *dev)
>> up_write(&pcistub_sem);
>> }
>>
>> +struct pcistub_args {
>> + const struct pci_dev *dev;
>> + unsigned int dcount;
> The sole use of this field is for a debug message. Why not drop it
> and make "dev" the "data" argument without further indirection?
I prefer to keep this data structure since it will be helpful to debug
any issues
orfor future enhancements.
>> +static int pcistub_device_search(struct pci_dev *dev, void *data)
>> +{
>> + struct pcistub_device *psdev;
>> + struct pcistub_args *arg = data;
>> + bool found = false;
>> + unsigned long flags;
>> +
>> + spin_lock_irqsave(&pcistub_devices_lock, flags);
>> +
>> + list_for_each_entry(psdev, &pcistub_devices, dev_list) {
>> + if (psdev->dev == dev) {
>> + found = true;
>> + arg->dcount++;
>> + break;
> Neither here nor in the caller I can see a check whether the device
> is currently assigned to a guest. Ownership by pciback alone imo is
> not sufficient to allow a reset to be performed.
I can add the following check

if ((psdev->dev == dev) && (pci_is_dev_assigned(dev)))
>> +static int pcistub_device_reset(struct pci_dev *dev)
>> +{
>> + struct xen_pcibk_dev_data *dev_data;
>> + bool slot = false, bus = false;
>> + struct pcistub_args arg = {};
>> +
>> + if (!dev)
>> + return -EINVAL;
>> +
>> + dev_dbg(&dev->dev, "[%s]\n", __func__);
>> +
>> + /* First check and try FLR */
>> + if (pcie_has_flr(dev)) {
>> + dev_dbg(&dev->dev, "resetting %s device using FLR\n",
>> + pci_name(dev));
>> + pcie_flr(dev);
> The lack of error check here puzzled me, but I see the function
> indeed returns void right now. I think the prereq patch should
> change this along with exporting the function - you really don't
> want the device to be handed to a guest when the FLR timed
> out.
We will change pcie_flr() to return error code. I will make this change
in the next version of this patch.
>> + return 0;
>> + }
>> +
>> + if (!pci_probe_reset_slot(dev->slot))
>> + slot = true;
>> + else if ((!pci_probe_reset_bus(dev->bus)) &&
>> + (!pci_is_root_bus(dev->bus)))
> Too many parentheses for my taste.
I will fix it.
>> +static ssize_t reset_store(struct device_driver *drv, const char *buf,
>> + size_t count)
>> +{
>> + struct pcistub_device *psdev;
>> + int domain, bus, slot, func;
>> + int err;
>> +
>> + err = str_to_slot(buf, &domain, &bus, &slot, &func);
>> + if (err)
>> + return err;
>> +
>> + psdev = pcistub_device_find(domain, bus, slot, func);
>> + if (psdev) {
>> + err = pcistub_device_reset(psdev->dev);
>> + pcistub_device_put(psdev);
>> + } else {
>> + err = -ENODEV;
>> + }
>> +
>> + if (!err)
>> + err = count;
>> +
>> + return err;
>> +}
>> +static DRIVER_ATTR_WO(reset);
> Would it be worth for reads of the file to return whether the device
> can be reset this way (i.e. the result of the checks you do before
> actually doing the reset)?
I don't think so. Plus, it makes this interface and its usage more
complicated.

Cheers
GOVINDA