[PATCH] device-assignment: Reset device on system reset

[PATCH] device-assignment: Reset device on system reset

[Alex Williamson]

On system reset, we currently try to quiesce DMA by clearing the
command register.  This assumes that nothing re-enables bus master
support without first de-programming the device.  Use a bigger
hammer to help the guest not shoot itself by issuing a function
reset via sysfs on each system reset.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---

 hw/device-assignment.c |   24 ++++++++++++++++++++++--
 1 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/hw/device-assignment.c b/hw/device-assignment.c
index db82e73..f1c2a3c 100644
--- a/hw/device-assignment.c
+++ b/hw/device-assignment.c
@@ -1693,13 +1693,33 @@ static const VMStateDescription vmstate_assigned_device = {
 
 static void reset_assigned_device(DeviceState *dev)
 {
-    PCIDevice *d = DO_UPCAST(PCIDevice, qdev, dev);
+    PCIDevice *pci_dev = DO_UPCAST(PCIDevice, qdev, dev);
+    AssignedDevice *adev = DO_UPCAST(AssignedDevice, dev, pci_dev);
+    char reset_file[64];
+    const char reset[] = "1";
+    int fd, ret;
+
+    snprintf(reset_file, sizeof(reset_file),
+             "/sys/bus/pci/devices/0000:%02x:%02x.%01x/reset",
+             adev->host.bus, adev->host.dev, adev->host.func);
+
+    /*
+     * Issue a device reset via pci-sysfs.  Note that we use write(2) here
+     * and ignore the return value because some kernels have a bug that
+     * returns 0 rather than bytes written on success, sending us into an
+     * infinite retry loop using other write mechanisms.
+     */
+    fd = open(reset_file, O_WRONLY);
+    if (fd != -1) {
+        ret = write(fd, reset, strlen(reset));
+        close(fd);
+    }
 
     /*
      * When a 0 is written to the command register, the device is logically
      * disconnected from the PCI bus. This avoids further DMA transfers.
      */
-    assigned_dev_pci_write_config(d, PCI_COMMAND, 0, 2);
+    assigned_dev_pci_write_config(pci_dev, PCI_COMMAND, 0, 2);
 }
 
 static int assigned_initfn(struct PCIDevice *pci_dev)

Re: [PATCH] device-assignment: Reset device on system reset

[Chris Wright]

* Alex Williamson (alex.williamson@redhat.com) wrote:
>  static void reset_assigned_device(DeviceState *dev)
>  {
> -    PCIDevice *d = DO_UPCAST(PCIDevice, qdev, dev);
> +    PCIDevice *pci_dev = DO_UPCAST(PCIDevice, qdev, dev);
> +    AssignedDevice *adev = DO_UPCAST(AssignedDevice, dev, pci_dev);
> +    char reset_file[64];
> +    const char reset[] = "1";
> +    int fd, ret;
> +
> +    snprintf(reset_file, sizeof(reset_file),
> +             "/sys/bus/pci/devices/0000:%02x:%02x.%01x/reset",
> +             adev->host.bus, adev->host.dev, adev->host.func);

need to consider segment: %04x:..., adev->host.seg, ...

> +    /*
> +     * Issue a device reset via pci-sysfs.  Note that we use write(2) here
> +     * and ignore the return value because some kernels have a bug that
> +     * returns 0 rather than bytes written on success, sending us into an
> +     * infinite retry loop using other write mechanisms.
> +     */
> +    fd = open(reset_file, O_WRONLY);
> +    if (fd != -1) {
> +        ret = write(fd, reset, strlen(reset));
> +        close(fd);
> +    }

This will probably fail when it's managed by libvirt.  I expect it
will need some file ownership and security label mgmt added to device
assignement path I expect.

Re: [PATCH] device-assignment: Reset device on system reset

[Alex Williamson]

On Thu, 2011-03-17 at 14:12 -0700, Chris Wright wrote:
> * Alex Williamson (alex.williamson@redhat.com) wrote:
> >  static void reset_assigned_device(DeviceState *dev)
> >  {
> > -    PCIDevice *d = DO_UPCAST(PCIDevice, qdev, dev);
> > +    PCIDevice *pci_dev = DO_UPCAST(PCIDevice, qdev, dev);
> > +    AssignedDevice *adev = DO_UPCAST(AssignedDevice, dev, pci_dev);
> > +    char reset_file[64];
> > +    const char reset[] = "1";
> > +    int fd, ret;
> > +
> > +    snprintf(reset_file, sizeof(reset_file),
> > +             "/sys/bus/pci/devices/0000:%02x:%02x.%01x/reset",
> > +             adev->host.bus, adev->host.dev, adev->host.func);
> 
> need to consider segment: %04x:..., adev->host.seg, ...

Shoot, I ported this over from a code base w/o seg support.  Thanks for
the catch.

> > +    /*
> > +     * Issue a device reset via pci-sysfs.  Note that we use write(2) here
> > +     * and ignore the return value because some kernels have a bug that
> > +     * returns 0 rather than bytes written on success, sending us into an
> > +     * infinite retry loop using other write mechanisms.
> > +     */
> > +    fd = open(reset_file, O_WRONLY);
> > +    if (fd != -1) {
> > +        ret = write(fd, reset, strlen(reset));
> > +        close(fd);
> > +    }
> 
> This will probably fail when it's managed by libvirt.  I expect it
> will need some file ownership and security label mgmt added to device
> assignement path I expect.

Already posted a patch for adding file rights, seems to be sufficient:

https://www.redhat.com/archives/libvir-list/2011-March/msg00823.html

Thanks,

Alex

[PATCH v2] device-assignment: Reset device on system reset

[Alex Williamson]

On system reset, we currently try to quiesce DMA by clearing the
command register.  This assumes that nothing re-enables bus master
support without first de-programming the device.  Use a bigger
hammer to help the guest not shoot itself by issuing a function
reset via sysfs on each system reset.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---

 v2: Fix segment support

 hw/device-assignment.c |   24 ++++++++++++++++++++++--
 1 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/hw/device-assignment.c b/hw/device-assignment.c
index db82e73..4997b6e 100644
--- a/hw/device-assignment.c
+++ b/hw/device-assignment.c
@@ -1693,13 +1693,33 @@ static const VMStateDescription vmstate_assigned_device = {
 
 static void reset_assigned_device(DeviceState *dev)
 {
-    PCIDevice *d = DO_UPCAST(PCIDevice, qdev, dev);
+    PCIDevice *pci_dev = DO_UPCAST(PCIDevice, qdev, dev);
+    AssignedDevice *adev = DO_UPCAST(AssignedDevice, dev, pci_dev);
+    char reset_file[64];
+    const char reset[] = "1";
+    int fd, ret;
+
+    snprintf(reset_file, sizeof(reset_file),
+             "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/reset",
+             adev->host.seg, adev->host.bus, adev->host.dev, adev->host.func);
+
+    /*
+     * Issue a device reset via pci-sysfs.  Note that we use write(2) here
+     * and ignore the return value because some kernels have a bug that
+     * returns 0 rather than bytes written on success, sending us into an
+     * infinite retry loop using other write mechanisms.
+     */
+    fd = open(reset_file, O_WRONLY);
+    if (fd != -1) {
+        ret = write(fd, reset, strlen(reset));
+        close(fd);
+    }
 
     /*
      * When a 0 is written to the command register, the device is logically
      * disconnected from the PCI bus. This avoids further DMA transfers.
      */
-    assigned_dev_pci_write_config(d, PCI_COMMAND, 0, 2);
+    assigned_dev_pci_write_config(pci_dev, PCI_COMMAND, 0, 2);
 }
 
 static int assigned_initfn(struct PCIDevice *pci_dev)

Re: [PATCH v2] device-assignment: Reset device on system reset

[Chris Wright]

* Alex Williamson (alex.williamson@redhat.com) wrote:
> On system reset, we currently try to quiesce DMA by clearing the
> command register.  This assumes that nothing re-enables bus master
> support without first de-programming the device.  Use a bigger
> hammer to help the guest not shoot itself by issuing a function
> reset via sysfs on each system reset.
> 
> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

Looks good.

Acked-by: Chris Wright <chrisw@redhat.com>

Re: [PATCH] device-assignment: Reset device on system reset

[Chris Wright]

* Alex Williamson (alex.williamson@redhat.com) wrote:
> On Thu, 2011-03-17 at 14:12 -0700, Chris Wright wrote:
> > * Alex Williamson (alex.williamson@redhat.com) wrote:
> > > +    fd = open(reset_file, O_WRONLY);
> > > +    if (fd != -1) {
> > > +        ret = write(fd, reset, strlen(reset));
> > > +        close(fd);
> > > +    }
> > 
> > This will probably fail when it's managed by libvirt.  I expect it
> > will need some file ownership and security label mgmt added to device
> > assignement path I expect.
> 
> Already posted a patch for adding file rights, seems to be sufficient:
> 
> https://www.redhat.com/archives/libvir-list/2011-March/msg00823.html

Awesome, I missed that path, thanks Alex!

thanks,
-chris

Re: [PATCH v2] device-assignment: Reset device on system reset

[Marcelo Tosatti]

On Thu, Mar 17, 2011 at 02:55:42PM -0700, Chris Wright wrote:
> * Alex Williamson (alex.williamson@redhat.com) wrote:
> > On system reset, we currently try to quiesce DMA by clearing the
> > command register.  This assumes that nothing re-enables bus master
> > support without first de-programming the device.  Use a bigger
> > hammer to help the guest not shoot itself by issuing a function
> > reset via sysfs on each system reset.
> > 
> > Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
> 
> Looks good.
> 
> Acked-by: Chris Wright <chrisw@redhat.com>

Applied, thanks.

Re: [PATCH v2] device-assignment: Reset device on system reset

[Marcelo Tosatti]

On Thu, Mar 17, 2011 at 03:24:31PM -0600, Alex Williamson wrote:
> On system reset, we currently try to quiesce DMA by clearing the
> command register.  This assumes that nothing re-enables bus master
> support without first de-programming the device.  Use a bigger
> hammer to help the guest not shoot itself by issuing a function
> reset via sysfs on each system reset.
> 
> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
> ---
> 
>  v2: Fix segment support
> 
>  hw/device-assignment.c |   24 ++++++++++++++++++++++--

Applied, thanks.