[Bug 60271] New: Kernelpanic since 3.9.6 with qemu-kvm and pci-passthrough

[Bug 60271] New: Kernelpanic since 3.9.6 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271

           Summary: Kernelpanic since 3.9.6 with qemu-kvm and
                    pci-passthrough
           Product: Virtualization
           Version: unspecified
    Kernel Version: 3.9.8
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: kvm
        AssignedTo: virtualization_kvm@kernel-bugs.osdl.org
        ReportedBy: dev.faz@googlemail.com
        Regression: No


I'm (and at least one other user -
https://bugs.archlinux.org/index.php?do=details&task_id=35980) getting
kernelpanics since 3.9.8 if I start my VM which is using pci-passthrough. This
is new in 3.9.8 - I already can confirm 3.9.6 and 3.9.7 are *not* affected.

Steps to reproduce:

* Boot into kernel 3.9.8
* Start VM - pci-passthrough enabled
* Wait till device is first accessed/used
* Panic/Freeze

I would be glad to help debugging this. Just tell me, which information you
need.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 60271] Kernelpanic since 3.9.6 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271





--- Comment #1 from Fabian Zimmermann <dev.faz@googlemail.com>  2013-07-01 17:01:57 ---
Created an attachment (id=106521)
 --> (https://bugzilla.kernel.org/attachment.cgi?id=106521)
screenshot of panic

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 60271] Kernelpanic since 3.9.6 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271





--- Comment #2 from Fabian Zimmermann <dev.faz@googlemail.com>  2013-07-01 17:02:23 ---
Created an attachment (id=106531)
 --> (https://bugzilla.kernel.org/attachment.cgi?id=106531)
dmesg of affected system

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 60271] Kernelpanic since 3.9.6 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271


Gleb <gleb@redhat.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |gleb@redhat.com




--- Comment #3 from Gleb <gleb@redhat.com>  2013-07-01 17:33:36 ---
You need to catch entire crash message with netconsole or by other means.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 60271] Kernelpanic since 3.9.6 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271


Alex Williamson <alex.williamson@redhat.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |alex.williamson@redhat.com




--- Comment #4 from Alex Williamson <alex.williamson@redhat.com>  2013-07-01 17:50:29 ---
There are only two KVM patches between 3.9.7 and 3.9.8, perhaps you could try
reverting each of them:

https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/patch/?id=c78078398d9c86a7b36b02726399d07013b1e732
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/patch/?id=65f918123bd108a9b50bceaf4b32ac173b9e3f3e

Save to a $FILE, patch -p1 -R < $FILE.  I don't see any IOMMU or KVM device
assignment specific changes between those stable versions (there are a couple
radeon patches too and the archlinux report seems to point at using radeon as
an assigned device).  As Gleb says, we're just stabbing in the dark w/o seeing
the beginning of panic message.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 60271] Kernelpanic since 3.9.6 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271





--- Comment #5 from Fabian Zimmermann <dev.faz@googlemail.com>  2013-07-01 18:56:45 ---
tried to get dmesg with netconsole, but didn't got any "verbose" msg in
dmesg/console, only:

[  214.717070] input: PC Speaker as /devices/platform/pcspkr/input/input7
[  220.578854] input: PC Speaker as /devices/platform/pcspkr/input/input8
[  222.689153] input: PC Speaker as /devices/platform/pcspkr/input/input9
[  237.756584] cgroup: libvirtd (797) created nested cgroup for controller
"memory" which has incomplete hierarchy support. Nested cgroups may change
behavior in the future.
[  237.756595] cgroup: "memory" requires setting use_hierarchy to 1 on the
root.
[  237.756651] cgroup: libvirtd (797) created nested cgroup for controller
"devices" which has incomplete hierarchy support. Nested cgroups may change
behavior in the future.
[  237.756688] cgroup: libvirtd (797) created nested cgroup for controller
"blkio" which has incomplete hierarchy support. Nested cgroups may change
behavior in the future.
[  237.758818] tun: Universal TUN/TAP device driver, 1.6
[  237.758825] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[  237.778667] device vnet0 entered promiscuous mode
[  237.801933] br0: port 2(vnet0) entered forwarding state
[  237.801946] br0: port 2(vnet0) entered forwarding state
[  237.828833] qemu-system-x86: sending ioctl 5326 to a partition!
[  237.828851] qemu-system-x86: sending ioctl 80200204 to a partition!
[  237.949768] pci-stub 0000:01:00.0: enabling device (0000 -> 0003)
[  238.586632] assign device 0:1:0.0
[  238.734338] assign device 0:1:0.1
[  238.837611] assign device 0:0:1b.0
[  238.940852] assign device 0:0:1a.0
[  252.827533] br0: port 2(vnet0) entered forwarding state
[  267.843359] pci-stub 0000:01:00.0: irq 47 for MSI/MSI-X
... general protection fault: 0000 [#1] 

will now try to reverse above patches.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 60271] Kernelpanic since 3.9.8 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271

Fabian Zimmermann <dev.faz@googlemail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Kernelpanic since 3.9.6     |Kernelpanic since 3.9.8
                   |with qemu-kvm and           |with qemu-kvm and
                   |pci-passthrough             |pci-passthrough

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 60271] Kernelpanic since 3.9.8 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271

--- Comment #6 from Fabian Zimmermann <dev.faz@googlemail.com> ---
* reversed both above patches - problem still there
* disabled radeon-module (in kernel-config) - problem still there

Attached you will find the dmesg.txt (netconsole-output of panic). Don't
hesitate to ask if I can provide further information

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 60271] Kernelpanic since 3.9.8 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271

--- Comment #7 from Fabian Zimmermann <dev.faz@googlemail.com> ---
Created attachment 106838
  --> https://bugzilla.kernel.org/attachment.cgi?id=106838&action=edit
netconsole / dmesg of panic

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 60271] Kernelpanic since 3.9.8 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271

Fabian Zimmermann <dev.faz@googlemail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Kernel Version|3.9.8                       |3.9.8, 3.9.9

--- Comment #8 from Fabian Zimmermann <dev.faz@googlemail.com> ---
3.9.9 is affected, too.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 60271] Kernelpanic since 3.9.8 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271

Michael S. Tsirkin <m.s.tsirkin@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |m.s.tsirkin@gmail.com

--- Comment #9 from Michael S. Tsirkin <m.s.tsirkin@gmail.com> ---
can you pls try disabling zero copy tx in vhost_net?
it's a module parameter for this module

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 60271] Kernelpanic since 3.9.8 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271

--- Comment #10 from Fabian Zimmermann <dev.faz@googlemail.com> ---
adding "vhost_net.experimental_zcopytx=0" to kernel-cmdline solved the issue!

Anything else I may do/test?

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 60271] Kernelpanic since 3.9.8 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271

--- Comment #11 from Michael S. Tsirkin <m.s.tsirkin@gmail.com> ---
could you test latest upstream please?
A fix from there is in the process of being backported to stable.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 60271] Kernelpanic since 3.9.8 with qemu-kvm and pci-passthrough

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=60271

Fabian Zimmermann <dev.faz@googlemail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |PATCH_ALREADY_AVAILABLE

--- Comment #12 from Fabian Zimmermann <dev.faz@googlemail.com> ---
Bug is gone in 3.11-rc1.

Thanks a lot!

-- 
You are receiving this mail because:
You are watching the assignee of the bug.