Re: [PATCH] Xen 4.1.2 HVM guest realtek nic problems (eth0 8139cp transmit queue timed out)

[Keir Fraser <keir.xen@gmail.com>]

On 03/11/2011 18:07, "Pasi Kärkkäinen" <pasik@iki.fi> wrote:

> On Tue, Nov 01, 2011 at 10:56:06PM +0200, Pasi Kärkkäinen wrote:
>> On Mon, Oct 31, 2011 at 09:29:24PM +0200, Pasi Kärkkäinen wrote:
>>> On Mon, Oct 31, 2011 at 12:24:14PM -0700, Boris Derzhavets wrote:
>>>>    Seems to related
>>>> 
>>>>    https://bugs.launchpad.net/ubuntu/+source/xen/+bug/854829
>>>> 
>>> 
>>> Thanks, that seems to be the same bug.
>>> 
>>> Is the bugfix patch from xen-unstable going to backported to
>>> xen-4.1-testing.hg ?
>>> (4.1 backported patch available on ubuntu's launchpad above..)
>>> 
>> 
>> So the Ubuntu backport from xen-unstable to Xen 4.1.1 is here:
>> https://launchpadlibrarian.net/81948978/xen-pirq-resubmit-irq.patch
>> 
>> It seems to be shipping in Ubuntu 11.10 xen 4.1.1-2ubuntu4.1 packages.
>> 
>> Does that patch look suitable to be applied to xen-4.1-testing.hg ?
>> This bug should be fixed for Xen 4.1.3.
> 
> Any comments? 

This looks like a backport of Stefano's xen-unstable c/s 24007. I would like
him to submit/ack the backport, as it is not a trivial backport of the
xen-unstable patch.

 -- Keir

> -- Pasi
> 
> 
>> 
>>> 
>>> 
>>>>    Boris.
>>>> 
>>>>    --- On Mon, 10/31/11, Pasi Kärkkäinen <pasik@iki.fi> wrote:
>>>> 
>>>>      From: Pasi Kärkkäinen <pasik@iki.fi>
>>>>      Subject: [Xen-devel] Xen 4.1.2 HVM guest realtek nic problems (eth0
>>>>      8139cp transmit queue timed out)
>>>>      To: xen-devel@lists.xensource.com
>>>>      Date: Monday, October 31, 2011, 2:49 PM
>>>> 
>>>>      Hello,
>>>> 
>>>>      While testing Xen 4.1.2 and HVM guests I noticed the following problem
>>>>      with Fedora 16 HVM guests (using Linux 3.1.0 kernel in the VM):
>>>> 
>>>>      The errors (call trace) happens pretty much immediately when there's
>>>>      some network traffic going on..
>>>> 
>>>>      Simple "yum update" in the VM triggers the problem..
>>>> 
>>>>      [    0.000000] Linux version 3.1.0-5.fc16.x86_64
>>>>      ([1]mockbuild@x86-10.phx2.fedoraproject.org) (gcc version 4.6.1
>>>> 20111003
>>>>      (Red Hat 4.6.1-10) (GCC) ) #1 SMP Thu Oct 27 03:46:50 UTC 2011
>>>>      [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.1.0-5.fc16.x86_64
>>>>      root=/dev/mapper/vg_f16test64hvm-lv_root ro
>>>>      rd.lvm.lv=vg_f16test64hvm/lv_root rd.dm=0 SYSFONT=latarcyrheb-sun16
>>>> rhgb
>>>>      KEYTABLE=fi rd.md=0 rd.luks=0 rd.lvm.lv=vg_f16test64hvm/lv_swap
>>>>      LANG=en_US.UTF-8 console=ttyS0,38400 console=tty0
>>>>      <snip>
>>>> 
>>>>      [   28.998481] 8139cp 0000:00:03.0: eth0: link up, 100Mbps,
>>>> full-duplex,
>>>>      lpa 0x05E1
>>>>      [  149.712071] ------------[ cut here ]------------
>>>>      [  149.717216] WARNING: at net/sched/sch_generic.c:255
>>>>      dev_watchdog+0xf0/0x150()
>>>>      [  149.724709] Hardware name: HVM domU
>>>>      [  149.728738] NETDEV WATCHDOG: eth0 (8139cp): transmit queue 0 timed
>>>>      out
>>>>      [  149.735537] Modules linked in: ip6t_REJECT nf_conntrack_ipv6
>>>>      nf_defrag_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 ip6table_filter
>>>> xt_state
>>>>      ip6_tables nf_conntrack 81
>>>>      39too 8139cp ppdev parport_pc mii parport i2c_piix4 i2c_core joydev
>>>>      [last unloaded: scsi_wait_scan]
>>>>      [  149.768028] Pid: 0, comm: swapper Not tainted 3.1.0-5.fc16.x86_64
>>>> #1
>>>>      [  149.774639] Call Trace:
>>>>      [  149.777765]  <IRQ>  [<ffffffff81057a56>]
>>>>      warn_slowpath_common+0x83/0x9b
>>>>      [  149.784024]  [<ffffffff81057b11>] warn_slowpath_fmt+0x46/0x48
>>>>      [  149.790141]  [<ffffffff813ef49d>] ? netif_tx_lock+0x4a/0x7c
>>>>      [  149.799007]  [<ffffffff813ef613>] dev_watchdog+0xf0/0x150
>>>>      [  149.806361]  [<ffffffff81064b51>] run_timer_softirq+0x19b/0x280
>>>>      [  149.814392]  [<ffffffff81014fec>] ? sched_clock+0x9/0xd
>>>>      [  149.821650]  [<ffffffff813ef523>] ? netif_tx_unlock+0x54/0x54
>>>>      [  149.828926]  [<ffffffff8105d6b3>] __do_softirq+0xc9/0x1b5
>>>>      [  149.836803]  [<ffffffff81014fec>] ? sched_clock+0x9/0xd
>>>>      [  149.843422]  [<ffffffff814be5ec>] call_softirq+0x1c/0x30
>>>>      [  149.850067]  [<ffffffff81010b45>] do_softirq+0x46/0x81
>>>>      [  149.856760]  [<ffffffff8105d97b>] irq_exit+0x57/0xb1
>>>>      [  149.863035]  [<ffffffff812a39d3>] xen_evtchn_do_upcall+0x31/0x3e
>>>>      [  149.871144]  [<ffffffff814be76e>] xen_hvm_callback_vector+0x6e/0x80
>>>>      [  149.879494]  <EOI>  [<ffffffff8102f2f1>] ? native_safe_halt+0xb/0xd
>>>>      [  149.888220]  [<ffffffff81015b7e>] default_idle+0x4e/0x86
>>>>      [  149.894962]  [<ffffffff8100e2ed>] cpu_idle+0xae/0xe8
>>>>      [  149.901461]  [<ffffffff814934ee>] rest_init+0x72/0x74
>>>>      [  149.908949]  [<ffffffff81b76b7d>] start_kernel+0x3ab/0x3b6
>>>>      [  149.916617]  [<ffffffff81b762c4>]
>>>> x86_64_start_reservations+0xaf/0xb3
>>>>      [  149.929148]  [<ffffffff81b76140>] ? early_idt_handlers+0x140/0x140
>>>>      [  149.936797]  [<ffffffff81b763ca>] x86_64_start_kernel+0x102/0x111
>>>>      [  149.944336] ---[ end trace d8786cb7d6a57f8a ]---
>>>>      [  149.950406] 8139cp 0000:00:03.0: eth0: Transmit timeout, status
>>>>      d   3b   15 80ff
>>>>      [  149.961879] ------------[ cut here ]------------
>>>>      [  149.962245] WARNING: at kernel/softirq.c:159
>>>>      _local_bh_enable_ip+0x44/0x8e()
>>>>      [  149.962245] Hardware name: HVM domU
>>>>      [  149.962245] Modules linked in: ip6t_REJECT nf_conntrack_ipv6
>>>>      nf_defrag_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 ip6table_filter
>>>> xt_state
>>>>      ip6_tables nf_conntrack 8139too 8139cp ppdev parport_pc mii parport
>>>>      i2c_piix4 i2c_core joydev [last unloaded: scsi_wait_scan]
>>>>      [  149.962245] Pid: 0, comm: swapper Tainted: G
>>>>      W   3.1.0-5.fc16.x86_64 #1
>>>>      [  149.962245] Call Trace:
>>>>      [  149.962245]  <IRQ>  [<ffffffff81057a56>]
>>>>      warn_slowpath_common+0x83/0x9b
>>>>      [  149.962245]  [<ffffffff813ce599>] ? skb_release_data+0xca/0xcf
>>>>      [  149.962245]  [<ffffffff81057a88>] warn_slowpath_null+0x1a/0x1c
>>>>      [  149.962245]  [<ffffffff8105d462>] _local_bh_enable_ip+0x44/0x8e
>>>>      [  149.962245]  [<ffffffff8105d4ba>] local_bh_enable_ip+0xe/0x10
>>>>      [  149.962245]  [<ffffffff814b5db4>] _raw_spin_unlock_bh+0x15/0x17
>>>>      [  149.962245]  [<ffffffffa0053969>] destroy_conntrack+0x9d/0xdc
>>>>      [nf_conntrack]
>>>>      [  149.962245]  [<ffffffff813fa343>] nf_conntrack_destroy+0x19/0x1b
>>>>      [  149.962245]  [<ffffffff813ce7ad>] skb_release_head_state+0xa7/0xef
>>>>      [  149.962245]  [<ffffffff813ce5b1>] __kfree_skb+0x13/0x83
>>>>      [  149.962245]  [<ffffffff813ce677>] consume_skb+0x56/0x6b
>>>>      [  149.962245]  [<ffffffffa003c1b9>] cp_clean_rings+0xb4/0x114
>>>> [8139cp]
>>>>      [  149.962245]  [<ffffffffa003c371>] cp_tx_timeout+0x88/0x10e [8139cp]
>>>>      [  149.962245]  [<ffffffff813ef627>] dev_watchdog+0x104/0x150
>>>>      [  149.962245]  [<ffffffff81064b51>] run_timer_softirq+0x19b/0x280
>>>>      [  149.962245]  [<ffffffff81014fec>] ? sched_clock+0x9/0xd
>>>>      [  149.962245]  [<ffffffff813ef523>] ? netif_tx_unlock+0x54/0x54
>>>>      [  149.962245]  [<ffffffff8105d6b3>] __do_softirq+0xc9/0x1b5
>>>>      [  149.962245]  [<ffffffff81014fec>] ? sched_clock+0x9/0xd
>>>>      [  149.962245]  [<ffffffff814be5ec>] call_softirq+0x1c/0x30
>>>>      [  149.962245]  [<ffffffff81010b45>] do_softirq+0x46/0x81
>>>>      [  149.962245]  [<ffffffff8105d97b>] irq_exit+0x57/0xb1
>>>>      [  149.962245]  [<ffffffff812a39d3>] xen_evtchn_do_upcall+0x31/0x3e
>>>>      [  149.962245]  [<ffffffff814be76e>] xen_hvm_callback_vector+0x6e/0x80
>>>>      [  149.962245]  <EOI>  [<ffffffff8102f2f1>] ? native_safe_halt+0xb/0xd
>>>>      [  149.962245]  [<ffffffff81015b7e>] default_idle+0x4e/0x86
>>>>      [  149.962245]  [<ffffffff8100e2ed>] cpu_idle+0xae/0xe8
>>>>      [  149.962245]  [<ffffffff814934ee>] rest_init+0x72/0x74
>>>>      [  149.962245]  [<ffffffff81b76b7d>] start_kernel+0x3ab/0x3b6
>>>>      [  149.962245]  [<ffffffff81b762c4>]
>>>> x86_64_start_reservations+0xaf/0xb3
>>>>      [  149.962245]  [<ffffffff81b76140>] ? early_idt_handlers+0x140/0x140
>>>>      [  149.962245]  [<ffffffff81b763ca>] x86_64_start_kernel+0x102/0x111
>>>>      [  149.962245] ---[ end trace d8786cb7d6a57f8b ]---
>>>> 
>>>>      Full guest kernel dmesg attached to this email.
>>>>      The host is running F16 with Xen 4.1.2 and Linux 3.1.0 dom0 kernel.
>>>> 
>>>>      Xen cfgfile for the HVM domain:
>>>> 
>>>>      kernel = "hvmloader"
>>>>      builder='hvm'
>>>>      device_model = 'qemu-dm'
>>>>      name = "f16test64hvm"
>>>>      memory = 1024
>>>>      vcpus=1
>>>>      pae=1
>>>>      acpi=1
>>>>      apic=1
>>>>      vif = [ 'type=ioemu, mac=00:16:3f:03:01:14, bridge=virbr0' ]
>>>>      disk = [ 'phy:/dev/vg_f16/f16test64hvm,hda,w',
>>>>      'file:/root/iso/Fedora-16-Final-RC2-x86_64-DVD.iso,hdc:cdrom,r' ]
>>>>      boot='cd'
>>>>      xen_platform_pci=0
>>>>      on_poweroff = 'destroy'
>>>>      on_reboot   = 'restart'
>>>>      on_crash    = 'restart'
>>>>      sdl=0
>>>>      vnc=1
>>>>      vncpasswd=''
>>>>      stdvga=0
>>>>      serial='pty'
>>>>      tsc_mode=0
>>>>      usb=1
>>>>      usbdevice='tablet'
>>>>      keymap='fi'
>>>> 
>>>>      Using "model=e1000" instead for the vif works OK.. no problems with
>>>> the
>>>>      emulated intel nic.
>>>> 
>>>>      Any ideas what the problem with the emulated realtek nic?
>>>> 
>>>>      Thanks,
>>>> 
>>>>      -- Pasi
>>>> 
>>>>      -----Inline Attachment Follows-----
>>>> 
>>>>      _______________________________________________
>>>>      Xen-devel mailing list
>>>>      [2]Xen-devel@lists.xensource.com
>>>>      [3]http://lists.xensource.com/xen-devel
>>>> 
>>>> References
>>>> 
>>>>    Visible links
>>>>    1. file:///mc/compose?to=mockbuild@x86-10.phx2.fedoraproject.org
>>>>    2. file:///mc/compose?to=Xen-devel@lists.xensource.com
>>>>    3. http://lists.xensource.com/xen-devel
>>> 
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel