[Bug 45941] BUG: soft lockup - CPU#1 stuck for 21s! [kworker/1:1:64] BUG: soft lockup - CPU#4 stuck for 22s! [flush-252:0:384]

[Bug 45941] BUG: soft lockup - CPU#1 stuck for 21s! [kworker/1:1:64] BUG: soft lockup - CPU#4 stuck for 22s! [flush-252:0:384]

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=45941


Alan <alan@lxorguk.ukuu.org.uk> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |alan@lxorguk.ukuu.org.uk
          Component|Other                       |ext4
     Kernel Version|Linux 3.5.1-030501-generic  |3.5.1
                   |#201208091310 SMP           |
         AssignedTo|other_other@kernel-bugs.osd |fs_ext4@kernel-bugs.osdl.or
                   |l.org                       |g
            Product|Other                       |File System




--- Comment #1 from Alan <alan@lxorguk.ukuu.org.uk>  2012-08-14 11:43:28 ---
This looks like an I/O never completed but it may be a pointer to something
else (eg sys_readahead/ext4 interaction). Assigning to the ext4 folk see if
they have any thoughts but thats half guesswork 8)

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 45941] BUG: soft lockup - CPU#1 stuck for 21s! [kworker/1:1:64] BUG: soft lockup - CPU#4 stuck for 22s! [flush-252:0:384]

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=45941





--- Comment #2 from Loris Luise <loris.luise@gmail.com>  2012-08-16 13:56:24 ---
can Another oop dump be useful? Thanks

Aug 16 07:55:28 h3oserver2 kernel: [147758.832781] BUG: soft lockup - CPU#2
stuck for 23s! [netstat:18634]
Aug 16 07:55:28 h3oserver2 kernel: [147759.046380] Modules linked in: btrfs
zlib_deflate libcrc32c ufs qnx4 hfsplus hfs minix ntfs msdos jfs xfs reiserfs
xt_multiport pcnet32 ext2 xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6
dm_crypt ipt_REJECT xt_LOG xt_limit xt_tcpudp xt_addrtype xt_state
ip6table_filter ip6_tables ppdev nf_conntrack_netbios_ns nf_conntrack_broadcast
vmw_balloon nf_nat_ftp nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack_ftp
coretemp nf_conntrack microcode psmouse iptable_filter ip_tables x_tables
serio_raw parport_pc lp i2c_piix4 mac_hid shpchp parport vmxnet3 floppy
vmw_pvscsi
Aug 16 07:55:28 h3oserver2 kernel: [147759.630769] CPU 2
Aug 16 07:55:28 h3oserver2 kernel: [147759.630772] Modules linked in: btrfs
zlib_deflate libcrc32c ufs qnx4 hfsplus hfs minix ntfs msdos jfs xfs reiserfs
xt_multiport pcnet32 ext2 xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6
dm_crypt ipt_REJECT xt_LOG xt_limit xt_tcpudp xt_addrtype xt_state
ip6table_filter ip6_tables ppdev nf_conntrack_netbios_ns nf_conntrack_broadcast
vmw_balloon nf_nat_ftp nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack_ftp
coretemp nf_conntrack microcode psmouse iptable_filter ip_tables x_tables
serio_raw parport_pc lp i2c_piix4 mac_hid shpchp parport vmxnet3 floppy
vmw_pvscsi
Aug 16 07:55:28 h3oserver2 kernel: [147759.652655]
Aug 16 07:55:28 h3oserver2 kernel: [147759.678170] Pid: 18634, comm: netstat
Not tainted 3.5.1-030501-generic #201208091310 VMware, Inc. VMware Virtual
Platform/440BX Desktop Reference Platform
Aug 16 07:55:28 h3oserver2 kernel: [147759.688391] RIP:
0010:[<ffffffff815d0b5e>]  [<ffffffff815d0b5e>] established_get_next+0xfe/0x190
Aug 16 07:55:28 h3oserver2 kernel: [147759.928918] RSP: 0018:ffff880161f8fde8 
EFLAGS: 00010286
Aug 16 07:55:28 h3oserver2 kernel: [147759.928923] RAX: 0000000000060500 RBX:
ffff880018139400 RCX: 000000000007ffff
Aug 16 07:55:28 h3oserver2 kernel: [147759.928925] RDX: ffffc90010e82000 RSI:
ffffc90011487000 RDI: ffffffff815d0b28
Aug 16 07:55:28 h3oserver2 kernel: [147759.928926] RBP: ffff880161f8fdf8 R08:
0000000000000014 R09: 000000000000ffff
Aug 16 07:55:28 h3oserver2 kernel: [147759.928928] R10: 0000000000000000 R11:
000000000000000f R12: ffff0014ff0a0000
Aug 16 07:55:28 h3oserver2 kernel: [147759.928930] R13: ffff8801ffffffff R14:
ffff8801630898b5 R15: 000000000000074b
Aug 16 07:55:28 h3oserver2 kernel: [147759.928984] FS:  00007f5d094bf700(0000)
GS:ffff88017fc40000(0000) knlGS:0000000000000000
Aug 16 07:55:28 h3oserver2 kernel: [147759.928987] CS:  0010 DS: 0000 ES: 0000
CR0: 0000000080050033
Aug 16 07:55:28 h3oserver2 kernel: [147759.928988] CR2: 0000000000415f95 CR3:
000000011ccfb000 CR4: 00000000000006e0
Aug 16 07:55:28 h3oserver2 kernel: [147759.974228] DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Aug 16 07:55:28 h3oserver2 kernel: [147759.974266] DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Aug 16 07:55:28 h3oserver2 kernel: [147760.003496] Process netstat (pid: 18634,
threadinfo ffff880161f8e000, task ffff880173125c00)
Aug 16 07:55:28 h3oserver2 kernel: [147760.003505] Stack:
Aug 16 07:55:28 h3oserver2 kernel: [147760.003507]  ffff880018139400
ffff8801745c7440 ffff880161f8fe28 ffffffff815d0cef
Aug 16 07:55:28 h3oserver2 kernel: [147760.003514]  ffff880161f8fe28
ffff8801622b0d00 ffff880018139400 ffff8801684c0000
Aug 16 07:55:28 h3oserver2 kernel: [147760.003518]  ffff880161f8fea8
ffffffff811a8d08 ffff88013ffa4530 0002000000000001
Aug 16 07:55:28 h3oserver2 kernel: [147760.003522] Call Trace:
Aug 16 07:55:28 h3oserver2 kernel: [147760.003535]  [<ffffffff815d0cef>]
tcp_seq_next+0x3f/0xa0
Aug 16 07:55:28 h3oserver2 kernel: [147760.087408]  [<ffffffff811a8d08>]
seq_read+0x238/0x400
Aug 16 07:55:28 h3oserver2 kernel: [147760.087434]  [<ffffffff811a8ad0>] ?
seq_put_decimal_ll+0x60/0x60
Aug 16 07:55:28 h3oserver2 kernel: [147760.158567]  [<ffffffff811e7402>]
proc_reg_read+0x82/0xc0
Aug 16 07:55:28 h3oserver2 kernel: [147760.178084]  [<ffffffff811873c0>]
vfs_read+0xb0/0x180
Aug 16 07:55:28 h3oserver2 kernel: [147760.178108]  [<ffffffff811874da>]
sys_read+0x4a/0x90
Aug 16 07:55:28 h3oserver2 kernel: [147760.291955]  [<ffffffff8169fe69>]
system_call_fastpath+0x16/0x1b
Aug 16 07:55:28 h3oserver2 kernel: [147760.291966] Code: 0d 58 bb 95 00 48 8b
15 41 bb 95 00 c7 43 1c 00 00 00 00 83 c0 01 39 c8 89 43 18 0f 87 8e 00 00 00
48 63 f0 48 c1 e6 04 48 01 d6 <f6> 06 01 75 73 23 05 2b bb 95 00 48 8d 3c 85 00
00 00 00 48 03

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 45941] BUG: soft lockup - CPU#1 stuck for 21s! [kworker/1:1:64] BUG: soft lockup - CPU#4 stuck for 22s! [flush-252:0:384]

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=45941





--- Comment #3 from Loris Luise <loris.luise@gmail.com>  2012-08-24 10:05:12 ---
I made 2 modifications on the server running linux and currently no more soft
lockup has happened

1) removed irqbalance daemon from ubuntu server
2) unchecked "Synchronize guest time with host" from VM setting (VMWare esx)

Modification 1) most probably solved the problem.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 45941] BUG: soft lockup - CPU#1 stuck for 21s! [kworker/1:1:64] BUG: soft lockup - CPU#4 stuck for 22s! [flush-252:0:384]

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=45941

Alan <alan@lxorguk.ukuu.org.uk> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |OBSOLETE

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 45941] BUG: soft lockup - CPU#1 stuck for 21s! [kworker/1:1:64] BUG: soft lockup - CPU#4 stuck for 22s! [flush-252:0:384]

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=45941

Alain Brossard <vmware@alain.homeunix.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |vmware@alain.homeunix.org

--- Comment #4 from Alain Brossard <vmware@alain.homeunix.org> ---
Hi,
  I had the same issue with Ubuntu 14.04 + Xen-4.4 and the same solution
worked: remove irqbalance. Though I suspect it could be a question of lock
timing and this problem could come back.

3.13.0-32-generic
Aug 16 17:11:42 mytv kernel: [   59.284157] BUG: soft lockup - CPU#2 stuck for
23s! [Xorg:3631]
Aug 16 17:11:42 mytv kernel: [   59.284160] Modules linked in: xen_gntdev
xen_evtchn xenfs xen_privcmd nfsv
3 nfsv4 bridge stp llc dm_crypt nfsd auth_rpcgss nfs_acl nfs lockd sunrpc
fscache nls_iso8859_1 eeepc_wmi a
sus_wmi sparse_keymap video mxm_wmi snd_hda_codec_realtek crct10dif_pclmul
crc32_pclmul ghash_clmulni_intel
 snd_hda_intel snd_hda_codec snd_hwdep snd_pcm aesni_intel aes_x86_64 lrw
gf128mul snd_page_alloc glue_help
er ablk_helper cryptd snd_seq_midi snd_seq_midi_event serio_raw snd_rawmidi
fam15h_power k10temp edac_core 
edac_mce_amd snd_seq joydev snd_seq_device snd_timer snd sp5100_tco i2c_piix4
soundcore nvidia(POF) parport
_pc ppdev lp parport wmi mac_hid dm_mirror dm_region_hash dm_log hid_generic
usbhid hid psmouse e1000e r816
9 mii ptp ahci pps_core libahci
Aug 16 17:11:42 mytv kernel: [   59.284210] CPU: 2 PID: 3631 Comm: Xorg
Tainted: PF          O 3.13.0-32-ge
neric #57-Ubuntu
Aug 16 17:11:42 mytv kernel: [   59.284212] Hardware name: To be filled by
O.E.M. To be filled by O.E.M./SA
BERTOOTH 990FX R2.0, BIOS 2501 04/08/2014
Aug 16 17:11:42 mytv kernel: [   59.284216] task: ffff880425155fc0 ti:
ffff8804265f6000 task.ti: ffff880426
5f6000
Aug 16 17:11:42 mytv kernel: [   59.284218] RIP: e030:[<ffffffffa077d0da>] 
[<ffffffffa077d0da>] rm_shutdow
n_gvi_device+0x187/0x295 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.284315] RSP: e02b:ffff8804265f78d0  EFLAGS:
00000282
Aug 16 17:11:42 mytv kernel: [   59.284317] RAX: ffff8800000dc97c RBX:
ffff880415192e24 RCX: ffffffffa0b616
70
Aug 16 17:11:42 mytv kernel: [   59.284318] RDX: 00000000000000ff RSI:
00000000000dc97c RDI: 00000000000dc9
7c
Aug 16 17:11:42 mytv kernel: [   59.284319] RBP: ffff880415192e20 R08:
0000000000000001 R09: ffffffffa0b6d1
40
Aug 16 17:11:42 mytv kernel: [   59.284320] R10: ffff880425ace808 R11:
ffffffffa079c8db R12: ffff880415192e
28
Aug 16 17:11:42 mytv kernel: [   59.284321] R13: ffff880415192e2c R14:
0000000000009bed R15: ffff880415192e
90
Aug 16 17:11:42 mytv kernel: [   59.284325] FS:  00007fc4235f79c0(0000)
GS:ffff880447280000(0000) knlGS:000
0000000000000
Aug 16 17:11:42 mytv kernel: [   59.284327] CS:  e033 DS: 0000 ES: 0000 CR0:
000000008005003b
Aug 16 17:11:42 mytv kernel: [   59.284328] CR2: 00007fd1e4edcff0 CR3:
0000000425a8a000 CR4: 00000000000406
60
Aug 16 17:11:42 mytv kernel: [   59.284329] Stack:
Aug 16 17:11:42 mytv kernel: [   59.284330]  ffffffffa079c8db ffffffffa077f9da
ffff8804151b8008 00000000000
0cfde
Aug 16 17:11:42 mytv kernel: [   59.284333]  ffff880415192e8c ffffffffa0789d58
ffff8804265f7910 ffff8804151
b8008
Aug 16 17:11:42 mytv kernel: [   59.284336]  000000000000cfde ffffffffa077f976
ffff8804151b8008 ffffffffa07
7d639
Aug 16 17:11:42 mytv kernel: [   59.284338] Call Trace:
Aug 16 17:11:42 mytv kernel: [   59.284384]  [<ffffffffa079c8db>] ?
os_free_mem+0x1b/0x30 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.284428]  [<ffffffffa077f9da>] ?
_nv007991rm+0x33/0x56 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.284472]  [<ffffffffa0789d58>] ?
_nv019222rm+0x970b/0xcee3 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.284517]  [<ffffffffa077f976>] ?
_nv001242rm+0x83/0xa4 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.284560]  [<ffffffffa077d639>] ?
_nv014418rm+0x1f0/0xc97 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.284605]  [<ffffffffa076ad9e>] ?
_nv014859rm+0xce/0x3da [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.284653]  [<ffffffffa076b176>] ?
_nv014923rm+0x4c/0x59 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.284716]  [<ffffffffa02e083c>] ?
_nv018571rm+0x35/0x79 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.284765]  [<ffffffffa072e0ab>] ?
_nv018447rm+0x46/0xbf [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.284827]  [<ffffffffa02a3ce1>] ?
_nv012729rm+0x551/0x13cb [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.284890]  [<ffffffffa02c05a5>] ?
_nv017460rm+0x405/0x42f [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.284954]  [<ffffffffa03111f5>] ?
_nv005690rm+0x1e5/0x1f3 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.285033]  [<ffffffffa03df9ee>] ?
_nv005044rm+0x9a/0xc4 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.285129]  [<ffffffffa0539d22>] ?
_nv004050rm+0x8858/0xaef1 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.285225]  [<ffffffffa0538573>] ?
_nv004050rm+0x70a9/0xaef1 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.285264]  [<ffffffffa0166e80>] ?
_nv010039rm+0x25/0x40 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.285310]  [<ffffffffa0779558>] ?
_nv015014rm+0x808/0x982 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.285354]  [<ffffffffa077a538>] ?
_nv001097rm+0x483/0x6b8 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.285398]  [<ffffffffa0772aa4>] ?
rm_init_adapter+0xac/0x146 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.285443]  [<ffffffffa0792b71>] ?
nv_kern_open+0x191/0x810 [nvidia]
Aug 16 17:11:42 mytv kernel: [   59.285448]  [<ffffffff811c14ef>] ?
chrdev_open+0x9f/0x1d0
Aug 16 17:11:42 mytv kernel: [   59.285451]  [<ffffffff811ba033>] ?
do_dentry_open+0x233/0x2e0
Aug 16 17:11:42 mytv kernel: [   59.285453]  [<ffffffff811c1450>] ?
cdev_put+0x30/0x30
Aug 16 17:11:42 mytv kernel: [   59.285455]  [<ffffffff811ba369>] ?
vfs_open+0x49/0x50
Aug 16 17:11:42 mytv kernel: [   59.285464]  [<ffffffff811c8f04>] ?
do_last+0x554/0x1200
Aug 16 17:11:42 mytv kernel: [   59.285467]  [<ffffffff81311bdb>] ?
apparmor_file_alloc_security+0x5b/0x180
Aug 16 17:11:42 mytv kernel: [   59.285470]  [<ffffffff811cc38b>] ?
path_openat+0xbb/0x640
Aug 16 17:11:42 mytv kernel: [   59.285473]  [<ffffffff812d3d5e>] ?
security_inode_alloc+0x1e/0x20
Aug 16 17:11:42 mytv kernel: [   59.285476]  [<ffffffff811e2988>] ?
simple_xattr_get+0x68/0xb0
Aug 16 17:11:42 mytv kernel: [   59.285478]  [<ffffffff811cd76a>] ?
do_filp_open+0x3a/0x90
Aug 16 17:11:42 mytv kernel: [   59.285481]  [<ffffffff811da527>] ?
__alloc_fd+0xa7/0x130
Aug 16 17:11:42 mytv kernel: [   59.285484]  [<ffffffff811bbe89>] ?
do_sys_open+0x129/0x280
Aug 16 17:11:42 mytv kernel: [   59.285487]  [<ffffffff81020d45>] ?
syscall_trace_enter+0x145/0x250
Aug 16 17:11:42 mytv kernel: [   59.285489]  [<ffffffff811bbffe>] ?
SyS_open+0x1e/0x20
Aug 16 17:11:42 mytv kernel: [   59.285492]  [<ffffffff8172c87f>] ?
tracesys+0xe1/0xe6
Aug 16 17:11:42 mytv kernel: [   59.285493] Code: c3 e8 e2 28 00 00 b8 00 00 00
00 48 83 c4 08 c3 48 83 ec 
08 be 01 00 00 00 e8 96 ff ff ff ba 00 00 00 00 48 85 c0 74 03 0f b6 10 <89> d0
48 83 c4 08 c3 48 83 ec 08 
be 02 00 00 00 e8 74 ff ff ff 
Aug 16 17:12:10 mytv kernel: [   87.283580] BUG: soft lockup - CPU#2 stuck for
23s! [Xorg:3631]
 and loop...  Xorg never starts properly though ssh through works. All attempts
to shutdown cleanly hang as does tcpdump if it matters.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.