[Bug 88671] New: BUG at drivers/pci/ats.c:62! the second time booting a kvm guest with pci passthrough

[Bug 88671] New: BUG at drivers/pci/ats.c:62! the second time booting a kvm guest with pci passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=88671

            Bug ID: 88671
           Summary: BUG at drivers/pci/ats.c:62! the second time booting a
                    kvm guest with pci passthrough
           Product: Virtualization
           Version: unspecified
    Kernel Version: 3.17.3
          Hardware: All
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: kvm
          Assignee: virtualization_kvm@kernel-bugs.osdl.org
          Reporter: tstellar@gmail.com
        Regression: No

Created attachment 158381
  --> https://bugzilla.kernel.org/attachment.cgi?id=158381&action=edit
lspci

I'm running into this bug while trying to use pci passthrough of an AMD BONAIRE
XT (Radeon HD 7790)

Steps to reproduce:

1. virsh start vm
2. virsh destroy vm
3. virsh start vm

This bug only appears after starting the vm for the second time.  The first
time the vm boots normally and passthrough works as expected.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 88671] BUG at drivers/pci/ats.c:62! the second time booting a kvm guest with pci passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=88671

--- Comment #1 from Tom Stellard <tstellar@gmail.com> ---
Created attachment 158391
  --> https://bugzilla.kernel.org/attachment.cgi?id=158391&action=edit
Backtrace from BUG_ON

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 88671] BUG at drivers/pci/ats.c:62! the second time booting a kvm guest with pci passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=88671

--- Comment #2 from Tom Stellard <tstellar@gmail.com> ---
Created attachment 158401
  --> https://bugzilla.kernel.org/attachment.cgi?id=158401&action=edit
Virtual machine definition

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 88671] BUG at drivers/pci/ats.c:62! the second time booting a kvm guest with pci passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=88671

--- Comment #3 from Tom Stellard <tstellar@gmail.com> ---
I should also mention that I have this hook executing when the machine starts
up:

if [ "$2" = "prepare" ]; then
        virsh nodedev-detach pci_0000_01_00_1
fi

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 88671] BUG at drivers/pci/ats.c:62! the second time booting a kvm guest with pci passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=88671

Alex Williamson <alex.williamson@redhat.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |alex.williamson@redhat.com

--- Comment #4 from Alex Williamson <alex.williamson@redhat.com> ---
I'm not sure how you're getting to this BUG_ON, but (a) legacy KVM device
assignment is deprecated and (b) the card you've chosen has known reset issues.
 You might want to try vfio-pci, which I know can make this card work at least
once per host boot, but you're likely to get a BSOD and IOMMU faults on
subsequent guest [re]boots.  The reset problem with this card has been reported
to AMD, but there is no solution at this time.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 88671] BUG at drivers/pci/ats.c:62! the second time booting a kvm guest with pci passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=88671

--- Comment #5 from Tom Stellard <tstellar@gmail.com> ---
Thanks for the tip about vfio-pci.  I can now sometimes get two working guest
boots per host boot.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 88671] BUG at drivers/pci/ats.c:62! the second time booting a kvm guest with pci passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=88671

Tom Stellard <tstellar@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|kvm                         |Video(DRI - non Intel)
            Version|unspecified                 |2.5
            Product|Virtualization              |Drivers

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 88671] Radeon driver fails to reset hardware properly after kvm guest reboot

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=88671

Tom Stellard <tstellar@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|BUG at                      |Radeon driver fails to
                   |drivers/pci/ats.c:62! the   |reset hardware properly
                   |second time booting a kvm   |after kvm guest reboot
                   |guest with pci passthrough  |

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 88671] Radeon driver fails to reset hardware properly after kvm guest reboot

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=88671

--- Comment #6 from Tom Stellard <tstellar@gmail.com> ---
I've been playing with this a little more and it seems to be working correctly,
but radeon dynamic power management (dpm) always fails to initialize on the
second guest boot.  My questions are:

1. What methods are being used by kvm/qemu/libvirt to reset the GPU on guest
shutdown?

2. Is the problem only cuased by the fact that GPU reset is not implemented
correctly in the radeon driver of are there improvements that are needed in
kvm/qemu/libvirt in order to get this working?

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 88671] Radeon driver fails to reset hardware properly after kvm guest reboot

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=88671

--- Comment #7 from Alex Williamson <alex.williamson@redhat.com> ---
(In reply to Tom Stellard from comment #6)
> I've been playing with this a little more and it seems to be working
> correctly,
> but radeon dynamic power management (dpm) always fails to initialize on the
> second guest boot.  My questions are:
> 
> 1. What methods are being used by kvm/qemu/libvirt to reset the GPU on guest
> shutdown?

Secondary PCI bus reset from the parent bridge.

> 2. Is the problem only cuased by the fact that GPU reset is not implemented
> correctly in the radeon driver of are there improvements that are needed in
> kvm/qemu/libvirt in order to get this working?

Without a device specific reset, we're doing to most thorough standard reset
available to us.  I've also tried to use some of the reset mechanisms
implemented in the radeon FOSS driver but they offered no improvement over the
bus reset.  There seems to be some state retained in the device that is not
cleared via bus reset.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.