From mboxrd@z Thu Jan 1 00:00:00 1970 From: Zbyszek Żółkiewski Date: Thu, 8 Feb 2007 18:21:01 +0100 Subject: [Cluster-devel] Kernel panic on mounting gfs2: kernel 2.6.19 and 2.6.20. In-Reply-To: <1170951992.11001.509.camel@quoit.chygwyn.com> References: <1170939465.11001.491.camel@quoit.chygwyn.com> <1170941171.11001.496.camel@quoit.chygwyn.com> <1170951992.11001.509.camel@quoit.chygwyn.com> Message-ID: List-Id: To: cluster-devel.redhat.com MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit are you using the same glibc and distro? well i have patched gfs2 and here is output: Feb 8 18:16:28 xmpp-alt2 kernel: GFS2: fsid=xmpp-alt2:test.0: Joined cluster. Now mounting FS... Feb 8 18:16:28 xmpp-alt2 kernel: ------------[ cut here ]------------ Feb 8 18:16:28 xmpp-alt2 kernel: kernel BUG at fs/gfs2/glock.c:632! Feb 8 18:16:28 xmpp-alt2 kernel: invalid opcode: 0000 [#1] Feb 8 18:16:28 xmpp-alt2 kernel: Modules linked in: lock_nolock lock_dlm gfs2 dlm configfs Feb 8 18:16:28 xmpp-alt2 kernel: CPU: 0 Feb 8 18:16:28 xmpp-alt2 kernel: EIP: 0060:[] Not tainted VLI Feb 8 18:16:28 xmpp-alt2 kernel: EFLAGS: 00000292 (2.6.20-xmpp2 #3) Feb 8 18:16:28 xmpp-alt2 kernel: EIP is at run_queue+0x0/0x4 [gfs2] Feb 8 18:16:28 xmpp-alt2 kernel: eax: f5f119a4 ebx: f5ef9e34 ecx: f5f119e8 edx: f5f119e8 Feb 8 18:16:28 xmpp-alt2 kernel: esi: f5f119a4 edi: 00000000 ebp: f5e7c000 esp: f5ef9dd4 Feb 8 18:16:28 xmpp-alt2 kernel: ds: 007b es: 007b ss: 0068 Feb 8 18:16:28 xmpp-alt2 kernel: Process mount.gfs2 (pid: 1907, ti=f5ef8000 task=f5e1b550 task.ti=f5ef8000) Feb 8 18:16:28 xmpp-alt2 kernel: Stack: f8950d03 00000000 f5e7c000 00000000 f5ef9e34 f895101f f5ef9e34 00000001 Feb 8 18:16:28 xmpp-alt2 kernel: f5ef9df8 f5f119a4 f5eff550 f895c359 f89687a0 00000001 00000404 f5ef9e34 Feb 8 18:16:28 xmpp-alt2 kernel: 00000000 f5e7c000 f5c6ae00 00000000 f895d2bc f5ef9e7c c02d572f f5ef9e5c Feb 8 18:16:28 xmpp-alt2 kernel: Call Trace: Feb 8 18:16:28 xmpp-alt2 kernel: [] gfs2_glock_nq+0x42/0x63 [gfs2] Feb 8 18:16:28 xmpp-alt2 kernel: [] gfs2_glock_nq_num+0x4e/0x6f [gfs2] Feb 8 18:16:28 xmpp-alt2 kernel: [] init_locking+0xe1/0x2b0 [gfs2] Feb 8 18:16:28 xmpp-alt2 kernel: [] fill_super+0xd6/0x231 [gfs2] Feb 8 18:16:28 xmpp-alt2 kernel: [] gfs2_glock_nq_num+0x45/0x6f [gfs2] Feb 8 18:16:28 xmpp-alt2 kernel: [] get_sb_bdev+0xe1/0x119 Feb 8 18:16:28 xmpp-alt2 kernel: [] gfs2_get_sb+0x27/0x4a [gfs2] Feb 8 18:16:28 xmpp-alt2 kernel: [] fill_super+0x0/0x231 [gfs2] Feb 8 18:16:28 xmpp-alt2 kernel: [] vfs_kern_mount+0x4c/0x99 Feb 8 18:16:28 xmpp-alt2 kernel: [] do_kern_mount+0x35/0x50 Feb 8 18:16:28 xmpp-alt2 kernel: [] do_new_mount+0x6b/0xb7 Feb 8 18:16:28 xmpp-alt2 kernel: [] do_mount+0x1ca/0x1ec Feb 8 18:16:28 xmpp-alt2 kernel: [] __alloc_pages+0x54/0x2d4 Feb 8 18:16:28 xmpp-alt2 kernel: [] copy_mount_options+0x56/0xac Feb 8 18:16:28 xmpp-alt2 kernel: [] sys_mount+0x74/0xab Feb 8 18:16:28 xmpp-alt2 kernel: [] syscall_call+0x7/0xb Feb 8 18:16:28 xmpp-alt2 kernel: ======================= Feb 8 18:16:28 xmpp-alt2 kernel: Code: 08 8b 48 04 8b 00 89 48 04 89 01 89 5b 04 89 1b b8 07 00 00 00 0f b3 42 08 89 d8 e8 5b fd ff ff 89 d8 e8 07 79 7f c7 31 c0 5b c3 <0f> 0b eb fe 56 31 c9 31 d2 53 89 c3 83 ec 34 8d 74 24 04 89 34 Feb 8 18:16:28 xmpp-alt2 kernel: EIP: [] run_queue+0x0/0x4 [gfs2] SS:ESP 0068:f5ef9dd4 Feb 8 18:16:42 xmpp-alt2 kernel: <0>------------[ cut here ]------------ Feb 8 18:16:42 xmpp-alt2 kernel: kernel BUG at fs/gfs2/glock.c:632! Feb 8 18:16:42 xmpp-alt2 kernel: invalid opcode: 0000 [#2] Feb 8 18:16:42 xmpp-alt2 kernel: Modules linked in: lock_nolock lock_dlm gfs2 dlm configfs Feb 8 18:16:42 xmpp-alt2 kernel: CPU: 0 Feb 8 18:16:42 xmpp-alt2 kernel: EIP: 0060:[] Not tainted VLI Feb 8 18:16:42 xmpp-alt2 kernel: EFLAGS: 00000247 (2.6.20-xmpp2 #3) Feb 8 18:16:42 xmpp-alt2 kernel: EIP is at run_queue+0x0/0x4 [gfs2] Feb 8 18:16:42 xmpp-alt2 kernel: eax: f5f119a4 ebx: f5f119a4 ecx: 00000001 edx: f5f119a4 Feb 8 18:16:42 xmpp-alt2 kernel: esi: f5e7c000 edi: f5f119a4 ebp: 00000001 esp: f5f33f94 Feb 8 18:16:42 xmpp-alt2 kernel: ds: 007b es: 007b ss: 0068 Feb 8 18:16:42 xmpp-alt2 kernel: Process gfs2_scand (pid: 1925, ti=f5f32000 task=f5effa70 task.ti=f5f32000) Feb 8 18:16:42 xmpp-alt2 kernel: Stack: f895005f f8951881 f8951883 00001990 f5e7c000 f8948800 fffffffc f89518eb Feb 8 18:16:42 xmpp-alt2 kernel: f5e7c000 f5e7c000 f894880c f5ef9dc4 c012320b 00000001 ffffffff ffffffff Feb 8 18:16:42 xmpp-alt2 kernel: c012316e 00000000 00000000 00000000 c01034df f5ef9dbc 00000000 00000000 Feb 8 18:16:42 xmpp-alt2 kernel: Call Trace: Feb 8 18:16:42 xmpp-alt2 kernel: [] gfs2_glmutex_unlock+0x18/0x1c [gfs2] Feb 8 18:16:42 xmpp-alt2 kernel: [] examine_bucket+0x59/0x5b [gfs2] Feb 8 18:16:42 xmpp-alt2 kernel: [] scan_glock+0x0/0x51 [gfs2] Feb 8 18:16:42 xmpp-alt2 kernel: [] gfs2_scand+0x0/0x30 [gfs2] Feb 8 18:16:42 xmpp-alt2 kernel: [] gfs2_scand_internal+0x17/0x22 [gfs2] Feb 8 18:16:42 xmpp-alt2 kernel: [] gfs2_scand+0xc/0x30 [gfs2] Feb 8 18:16:42 xmpp-alt2 kernel: [] kthread+0x9d/0xce Feb 8 18:16:42 xmpp-alt2 kernel: [] kthread+0x0/0xce Feb 8 18:16:42 xmpp-alt2 kernel: [] kernel_thread_helper+0x7/0x10 Feb 8 18:16:42 xmpp-alt2 kernel: ======================= Feb 8 18:16:42 xmpp-alt2 kernel: Code: 08 8b 48 04 8b 00 89 48 04 89 01 89 5b 04 89 1b b8 07 00 00 00 0f b3 42 08 89 d8 e8 5b fd ff ff 89 d8 e8 07 79 7f c7 31 c0 5b c3 <0f> 0b eb fe 56 31 c9 31 d2 53 89 c3 83 ec 34 8d 74 24 04 89 34 Feb 8 18:16:42 xmpp-alt2 kernel: EIP: [] run_queue+0x0/0x4 [gfs2] SS:ESP 0068:f5f33f94 and? On 2/8/07, Steven Whitehouse wrote: > > Hi, > > I cannot reproduce what you are seeing, but try the attached patch to > see if we can narrow this down, > > Steve. > > On Thu, 2007-02-08 at 15:02 +0100, Zbyszek ???kiewski wrote: > > ok , i have build kernel from your git ( > > git://git.kernel.org/pub/scm/linux/kernel/git/steve/gfs2-2.6-nmw.git) > > Linux version 2.6.20-xmpp2-ga2cf8222-dirty (root at xmpp-alt2) (gcc > > version 3.4.4 20050314 (prerelease) (Debian 3.4.3-13sarge1)) #1 Thu > > Feb 8 14:51:21 CET 2007 > > > > and there is the same problem; > > kernel BUG at fs/gfs2/glock.c:704! > > invalid opcode: 0000 [#2] > > Modules linked in: lock_nolock lock_dlm gfs2 dlm configfs > > CPU: 0 > > EIP: 0060:[] Not tainted VLI > > EFLAGS: 00000282 (2.6.20-xmpp2-ga2cf8222-dirty #1) > > EIP is at gfs2_glmutex_unlock+0x18/0x1c [gfs2] > > > > and so one.... > > > > ok so waiting patiently for solution.... > > > > > > > > > > On 2/8/07, Steven Whitehouse wrote: > > Hi, > > > > On Thu, 2007-02-08 at 14:04 +0100, Zbyszek ???kiewski wrote: > > > sorry - mail went only to Steven, now to group.... > > > > > > > > > On 2/8/07, Zbyszek ???kiewski wrote: > > > well, thanks for answer, i have tried with nolock, > > and result > > > is as follow: > > > of course i made mkfs -t gfs2 -p lock_nolock -t > > xmpp-alt2:test > > > -j 1 /dev/sdb1 and then: > > > mount -t gfs2 /dev/sdb1 /mnt -v > > > > > > and yes - the device is mounted, > > It looks like what is happening is that a glmutex_unlock() is > > discovering that its spinlock has been dropped by > > glock.c:run_queue() > > which should be impossible, so something odd is happening here > > I think. > > > > The daemons implicated in this are there to demote unused > > locks on a > > periodic basis, so its presumably one of the locks used during > > mounting > > of the filesystem thats at fault. > > > > > (the changes to kernel you was talking about: you > > mean: git1 > > > for 2.6.20?) > > > > > I'm not sure if its in git1 or not, I suspect it will be git2 > > since it > > was only yesterday that the patches went in. Linus' current > > git tree > > seems to be broken (both gitweb and direct via the git tools) > > otherwise > > I'd post a URL to the changes. In the mean time you can find > > them in my > > -nmw tree which will get updated just as soon as git it > > working again at > > kernel.org, > > > > Steve. > > > > > > > > > > -- > > pozdrawiam, > > Zbyszek ???kiewski > > -- pozdrawiam, Zbyszek ???kiewski -------------- next part -------------- An HTML attachment was scrubbed... URL: