From: Nikola Ciprich <extmaillist@linuxbox.cz>
To: cluster-devel.redhat.com
Subject: [Cluster-devel] 2.6.37 GFS/CLVM/DLM trouble II
Date: Sun, 20 Mar 2011 20:01:16 +0100 [thread overview]
Message-ID: <20110320190116.GA29048@nik-comp.lan> (raw)
Hello Stephen et al,
some time ago, I reported GFS2 hangs. You asked me to obtain DLM lock
dumps, I weren't able to reproduce till now.
Today, the on my testing machine, GFS got stuck again. I also noticed
that clustered LVM is also stuck on it, so I guess the problem is
somewhere in the DLM code, not GFS.
Here are kernel backtraces:
[182189.107631] INFO: task clvmd:17723 blocked for more than 120
seconds.
[182189.107633] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[182189.107634] clvmd D ffffffff8140a4c0 0 17723 1
0x00000000
[182189.107637] ffff8800853c1ca0 0000000000000086 0000000000000000
00000000000116c0
[182189.107641] ffff88013b7348d8 0000000000000001 ffff88013b734530
ffff88013fcd0000
[182189.107644] ffff8800853c1fd8 0000000000000001 0000000001c225b8
ffff8800853c1c98
[182189.107647] Call Trace:
[182189.107651] [<ffffffff810d5025>] ?
get_page_from_freelist+0x3b5/0x510
[182189.107654] [<ffffffff8103aee1>] ? get_parent_ip+0x11/0x50
[182189.107656] [<ffffffff8103aee1>] ? get_parent_ip+0x11/0x50
[182189.107659] [<ffffffff8136d205>]
rwsem_down_failed_common+0xb5/0x130
[182189.107663] [<ffffffff8136d2b5>] rwsem_down_read_failed+0x15/0x17
[182189.107665] [<ffffffff811d4c44>]
call_rwsem_down_read_failed+0x14/0x30
[182189.107668] [<ffffffff8136c65d>] ? down_read+0x2d/0x40
[182189.107673] [<ffffffffa0548aa2>] dlm_user_request+0x42/0x260 [dlm]
[182189.107676] [<ffffffff8103aee1>] ? get_parent_ip+0x11/0x50
[182189.107679] [<ffffffff8110e23e>] ?
kmem_cache_alloc_notrace+0x9e/0xc0
[182189.107684] [<ffffffffa0551b04>] device_write+0x684/0x880 [dlm]
[182189.107687] [<ffffffff811a9cde>] ?
security_file_permission+0x1e/0x90
[182189.107689] [<ffffffff8111a894>] ? rw_verify_area+0x74/0xf0
[182189.107691] [<ffffffff8111aef9>] vfs_write+0xc9/0x190
[182189.107694] [<ffffffff8111b640>] sys_write+0x50/0x90
[182189.107697] [<ffffffff810024fb>] system_call_fastpath+0x16/0x1b
[182189.107705] INFO: task gfs2_quotad:22599 blocked for more than 120
seconds.
[182189.107706] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[182189.107707] gfs2_quotad D ffffffff8140a4c0 0 22599 2
0x00000000
[182189.107711] ffff880113d0ba88 0000000000000046 00000000000116c0
00000000000116c0
[182189.107714] ffff8801141bb1c8 0000000000000002 ffff8801141bae20
ffff88013fcd5c40
[182189.107717] ffff880113d0bfd8 ffff880113d0b9b0 0000000081046cd4
ffff88013fcd0000
[182189.107720] Call Trace:
[182189.107723] [<ffffffff8103aee1>] ? get_parent_ip+0x11/0x50
[182189.107726] [<ffffffff8103aee1>] ? get_parent_ip+0x11/0x50
[182189.107729] [<ffffffff8136d205>]
rwsem_down_failed_common+0xb5/0x130
[182189.107731] [<ffffffff81035fb1>] ? cpuacct_charge+0x61/0x70
[182189.107734] [<ffffffff8136d2b5>] rwsem_down_read_failed+0x15/0x17
[182189.107737] [<ffffffff811d4c44>]
call_rwsem_down_read_failed+0x14/0x30
[182189.107740] [<ffffffff8136c65d>] ? down_read+0x2d/0x40
[182189.107745] [<ffffffffa0547039>] dlm_lock+0x59/0x180 [dlm]
[182189.107747] [<ffffffff81045ae2>] ? update_curr+0xb2/0x170
[182189.107750] [<ffffffff810374df>] ? hrtick_update+0x2f/0x40
[182189.107760] [<ffffffffa05885d3>] gdlm_lock+0xd3/0x120 [gfs2]
[182189.107769] [<ffffffffa05887f0>] ? gdlm_ast+0x0/0x160 [gfs2]
[182189.107777] [<ffffffffa0588620>] ? gdlm_bast+0x0/0x50 [gfs2]
[182189.107783] [<ffffffffa056a62c>] do_xmote+0x18c/0x280 [gfs2]
[182189.107789] [<ffffffffa056a7b1>] run_queue+0x91/0x260 [gfs2]
[182189.107796] [<ffffffffa056aac3>] gfs2_glock_nq+0xc3/0x3a0 [gfs2]
[182189.107804] [<ffffffffa0584f49>] gfs2_statfs_sync+0x59/0x1a0 [gfs2]
[182189.107812] [<ffffffffa0584f41>] ? gfs2_statfs_sync+0x51/0x1a0
[gfs2]
[182189.107815] [<ffffffff8103c64d>] ? sub_preempt_count+0x9d/0xd0
[182189.107823] [<ffffffffa057dbf7>] quotad_check_timeo+0x57/0x90
[gfs2]
[182189.107831] [<ffffffffa057f637>] gfs2_quotad+0x207/0x240 [gfs2]
[182189.107834] [<ffffffff8106b130>] ?
autoremove_wake_function+0x0/0x40
[182189.107837] [<ffffffff8136d77d>] ?
_raw_spin_unlock_irqrestore+0x1d/0x50
[182189.107846] [<ffffffffa057f430>] ? gfs2_quotad+0x0/0x240 [gfs2]
[182189.107848] [<ffffffff8106ac06>] kthread+0x96/0xa0
[182189.107851] [<ffffffff810032d4>] kernel_thread_helper+0x4/0x10
[182189.107854] [<ffffffff8106ab70>] ? kthread+0x0/0xa0
[182189.107857] [<ffffffff810032d0>] ? kernel_thread_helper+0x0/0x10
and here debugfs DLM lock dumps:
[root at vbox5 pcmk:lvs]# cat glocks
G: s:EX n:2/20188 f:yIq t:EX d:EX/0 a:1 r:3
I: n:22/131464 t:8 f:0x00 d:0x00000000 s:1434
G: s:EX n:2/25d24 f:yIq t:EX d:EX/0 a:1 r:3
I: n:25/154916 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:5/102b7 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22575 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/3017c f:yIq t:EX d:EX/0 a:1 r:3
I: n:101/196988 t:8 f:0x00 d:0x00000000 s:941
G: s:SH n:5/102b8 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:4613 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/20185 f:yIq t:EX d:EX/0 a:1 r:3
I: n:19/131461 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:5/20189 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:2/18 f:Iq t:SH d:EX/0 a:0 r:3
I: n:3/24 t:4 f:0x00 d:0x00000201 s:3864
G: s:UN n:2/25d0c f: t:UN d:EX/0 a:0 r:2
G: s:EX n:2/2017a f:yIq t:EX d:EX/0 a:1 r:3
I: n:8/131450 t:8 f:0x00 d:0x00000000 s:1822
G: s:EX n:2/2018b f:yIq t:EX d:EX/0 a:1 r:3
I: n:25/131467 t:8 f:0x00 d:0x00000000 s:1434
G: s:EX n:2/3017b f:yIq t:EX d:EX/0 a:1 r:3
I: n:75/196987 t:8 f:0x00 d:0x00000000 s:1170
G: s:SH n:5/3017f f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:29119 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/20180 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/25d32 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/20184 f:yIq t:EX d:EX/0 a:1 r:3
I: n:18/131460 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:5/2018b f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/2018a f:yIq t:EX d:EX/0 a:1 r:3
I: n:24/131466 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:5/10839 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22615 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:1/2 f:Iq t:SH d:EX/0 a:0 r:3
G: s:UN n:2/102ab f:lIq t:EX d:EX/0 a:0 r:4
H: s:EX f:cW e:0 p:22599 [gfs2_quotad] gfs2_statfs_sync+0x51/0x1a0
[gfs2]
G: s:EX n:2/1053a f:yIq t:EX d:EX/0 a:1 r:3
I: n:3/66874 t:8 f:0x00 d:0x00000000 s:3126995
G: s:SH n:5/25d30 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/25d30 f:yIq t:EX d:EX/0 a:1 r:3
I: n:37/154928 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:5/25d38 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/102ab f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22575 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/25d38 f:yIq t:EX d:EX/0 a:1 r:3
I: n:45/154936 t:8 f:0x00 d:0x00000000 s:1434
G: s:EX n:2/25d26 f:yIq t:EX d:EX/0 a:1 r:3
I: n:27/154918 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:2/16 f:Iq t:SH d:EX/0 a:0 r:6
H: s:SH f:H e:0 p:17736 [mc] gfs2_lookupi+0xbc/0x1c0 [gfs2]
H: s:EX f:W e:0 p:17711 [flush-253:6] gfs2_write_inode+0x7a/0x170
[gfs2]
H: s:SH f:AW e:0 p:18238 [ls] gfs2_getattr+0x89/0xf0 [gfs2]
I: n:1/22 t:4 f:0x00 d:0x00000001 s:3864
G: s:EX n:2/20180 f:yIq t:EX d:EX/0 a:1 r:3
I: n:14/131456 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:5/3017c f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:29119 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:4/0 f:Iq t:SH d:EX/0 a:0 r:2
G: s:SH n:5/3017d f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/2017b f:yIq t:EX d:EX/0 a:1 r:3
I: n:9/131451 t:8 f:0x00 d:0x00000000 s:1621
G: s:SH n:5/25d24 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/25d3a f:yIq t:EX d:EX/0 a:1 r:3
I: n:47/154938 t:8 f:0x00 d:0x00000000 s:1434
G: s:EX n:2/10839 f:yIq t:EX d:EX/0 a:1 r:3
I: n:4/67641 t:4 f:0x00 d:0x00000001 s:3864
G: s:EX n:2/25d11 f:yIq t:EX d:EX/0 a:1 r:3
I: n:6/154897 t:8 f:0x00 d:0x00000000 s:392
G: s:SH n:2/19 f:Iq t:SH d:EX/0 a:0 r:4
H: s:SH f:eEcH e:0 p:22575 [(ended)] init_journal+0x63f/0x9d0 [gfs2]
I: n:4/25 t:8 f:0x01 d:0x00000200 s:134217728
G: s:EX n:2/25d10 f:yIq t:EX d:EX/0 a:1 r:3
I: n:5/154896 t:8 f:0x00 d:0x00000000 s:1423
G: s:SH n:5/17 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22575 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/25d10 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:29119 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/1083a f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22615 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/2017c f:yIq t:EX d:EX/0 a:1 r:3
I: n:10/131452 t:8 f:0x00 d:0x00000000 s:1621
G: s:SH n:5/20186 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/25d2c f:yIq t:EX d:EX/0 a:1 r:3
I: n:33/154924 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:5/25d2e f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/25d0f f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:29119 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/1009d f:Iq t:SH d:EX/0 a:0 r:2
G: s:SH n:5/2017c f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/102b9 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:4613 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/2018a f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/102ac f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22575 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:UN n:2/102ac f: t:UN d:EX/0 a:0 r:2
G: s:SH n:5/20185 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:2/1083a f:Iq t:SH d:EX/0 a:0 r:3
I: n:5/67642 t:4 f:0x00 d:0x00000001 s:3864
G: s:SH n:5/2017f f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/805b f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22575 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:UN n:2/805b f:Iq t:UN d:EX/0 a:0 r:2
G: s:SH n:5/25d0e f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:29119 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/3017d f:yIq t:EX d:EX/0 a:1 r:3
I: n:103/196989 t:8 f:0x00 d:0x00000000 s:1065
G: s:EX n:2/20187 f:yIq t:EX d:EX/0 a:1 r:3
I: n:21/131463 t:8 f:0x00 d:0x00000000 s:1434
G: s:EX n:2/20186 f:yIq t:EX d:EX/0 a:1 r:3
I: n:20/131462 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:5/3017b f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/2017b f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/25d2a f:yIq t:EX d:EX/0 a:1 r:3
I: n:31/154922 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:5/1053a f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:4613 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/20181 f:yIq t:EX d:EX/0 a:1 r:3
I: n:15/131457 t:8 f:0x00 d:0x00000000 s:1434
G: s:EX n:2/25d2e f:yIq t:EX d:EX/0 a:1 r:3
I: n:35/154926 t:8 f:0x00 d:0x00000000 s:1434
G: s:EX n:2/30179 f:yIq t:EX d:EX/0 a:1 r:3
I: n:73/196985 t:8 f:0x00 d:0x00000000 s:1084
G: s:UN n:2/102b7 f: t:UN d:EX/0 a:0 r:2
G: s:EX n:2/2017f f:yIq t:EX d:EX/0 a:1 r:3
I: n:13/131455 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:2/102b8 f:Iq t:SH d:EX/0 a:0 r:3
I: n:1/66232 t:4 f:0x00 d:0x00000001 s:3864
G: s:SH n:1/1 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:eEH e:0 p:22575 [(ended)] gfs2_glock_nq_num+0x62/0x90 [gfs2]
G: s:EX n:2/2017d f:yIq t:EX d:EX/0 a:1 r:3
I: n:11/131453 t:8 f:0x00 d:0x00000000 s:1434
G: s:EX n:2/20189 f:yIq t:EX d:EX/0 a:1 r:3
I: n:23/131465 t:8 f:0x00 d:0x00000000 s:1434
G: s:EX n:2/25d34 f:yIq t:EX d:EX/0 a:1 r:3
I: n:41/154932 t:8 f:0x00 d:0x00000000 s:1434
G: s:EX n:2/25d0f f:yIq t:EX d:EX/0 a:1 r:3
I: n:4/154895 t:10 f:0x00 d:0x00000000 s:10
G: s:EX n:2/25d28 f:yIq t:EX d:EX/0 a:1 r:3
I: n:29/154920 t:8 f:0x00 d:0x00000000 s:1434
G: s:EX n:2/20182 f:yIq t:EX d:EX/0 a:1 r:3
I: n:16/131458 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:5/20182 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/3017f f:yIq t:EX d:EX/0 a:1 r:3
I: n:85/196991 t:8 f:0x00 d:0x00000000 s:1102
G: s:SH n:5/2017a f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22615 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/25d2c f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:UN n:1/3 f: t:UN d:EX/0 a:0 r:2
G: s:SH n:5/2017d f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/20183 f:yIq t:EX d:EX/0 a:1 r:3
I: n:17/131459 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:2/1009d f:Iq t:SH d:EX/0 a:0 r:2
G: s:EX n:2/100a0 f:Iq t:EX d:EX/0 a:0 r:4
H: s:EX f:H e:0 p:22575 [(ended)] init_per_node+0x181/0x250 [gfs2]
I: n:9/65696 t:8 f:0x00 d:0x00000200 s:1048576
G: s:EX n:2/25d0e f:yIq t:EX d:EX/0 a:1 r:3
I: n:3/154894 t:4 f:0x00 d:0x00000001 s:3864
G: s:SH n:5/2017e f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/25d26 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:2/17 f:Iq t:SH d:EX/0 a:0 r:3
I: n:2/23 t:4 f:0x00 d:0x00000201 s:3864
G: s:SH n:5/25d11 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22615 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/1009f f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22575 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:9/0 f:Iq t:EX d:EX/0 a:0 r:3
H: s:EX f:eH e:0 p:22575 [(ended)] gfs2_glock_nq_num+0x62/0x90 [gfs2]
G: s:SH n:5/3017e f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/20184 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/20187 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/16 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22575 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/25d32 f:yIq t:EX d:EX/0 a:1 r:3
I: n:39/154930 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:5/25d34 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/25d36 f:yIq t:EX d:EX/0 a:1 r:3
I: n:43/154934 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:5/20183 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/1009f f:Iq t:EX d:EX/0 a:0 r:4
H: s:EX f:H e:0 p:22575 [(ended)] init_per_node+0x14e/0x250 [gfs2]
I: n:8/65695 t:8 f:0x00 d:0x00000201 s:24
G: s:SH n:5/30179 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:29119 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/25d3a f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:UN n:5/25d0c f:lq t:SH d:EX/0 a:0 r:4
H: s:SH f:EW e:0 p:17736 [mc] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/18 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22575 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/25d28 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/100a0 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22575 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/25d36 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/20181 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/102b9 f:yIq t:EX d:EX/0 a:1 r:3
I: n:2/66233 t:8 f:0x00 d:0x00000000 s:2612512
G: s:EX n:2/2017e f:yIq t:EX d:EX/0 a:1 r:3
I: n:12/131454 t:8 f:0x00 d:0x00000000 s:1434
G: s:SH n:5/20188 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:EX n:2/3017e f:yIq t:EX d:EX/0 a:1 r:3
I: n:84/196990 t:8 f:0x00 d:0x00000000 s:1115
G: s:SH n:5/19 f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:22575 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
G: s:SH n:5/25d2a f:Iq t:SH d:EX/0 a:0 r:3
H: s:SH f:EH e:0 p:23817 [(ended)] gfs2_inode_lookup+0x117/0x250 [gfs2]
The machine is SMP x86_64 running 2.6.37.4 now. DLM, CLVMD as well as
GFS is handled by corosync/pacemaker cluster.
Could somebody please help me to debug it? I can keep the machine in
hung state for some time as it's testing box...
Thanks a lot in advance!
with best regards
nik
--
-------------------------------------
Ing. Nikola CIPRICH
LinuxBox.cz, s.r.o.
28. rijna 168, 709 01 Ostrava
tel.: +420 596 603 142
fax: +420 596 621 273
mobil: +420 777 093 799
www.linuxbox.cz
mobil servis: +420 737 238 656
email servis: servis at linuxbox.cz
-------------------------------------
next reply other threads:[~2011-03-20 19:01 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-03-20 19:01 Nikola Ciprich [this message]
2011-03-21 9:50 ` [Cluster-devel] 2.6.37 GFS/CLVM/DLM trouble II Steven Whitehouse
[not found] ` <20110321160804.GA2177@nik-comp.lan>
2011-03-21 16:47 ` Steven Whitehouse
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110320190116.GA29048@nik-comp.lan \
--to=extmaillist@linuxbox.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.