Re: 2.6.11: kernel BUG at fs/jbd/checkpoint.c:247! - Also on 2.6.12-rc5

Re: 2.6.11: kernel BUG at fs/jbd/checkpoint.c:247! - Also on 2.6.12-rc5

[John Baboval]

On Fri, 3 Jun 2005 16:33:56 -0700 Andrew Morton <akpm@osdl.org> wrote:
> Please test
> 
> ftp://ftp.kernel.org/pub/linux/kernel/v2.6/testing/linux-2.6.12-rc5.tar.bz2
> plus
> ftp://ftp.kernel.org/pub/linux/kernel/v2.6/snapshots/patch-2.6.12-rc5-git8.bz2


I've just reproduced this on 2.6.12-rc5.

It was hapening every 4 hours or so with 2.6.11.9. 2.6.12-rc5 ran for a week... The system is being used as a fairly high traffic NFS server.

If you need more information from my system, please CC my address with responses as I'm not subscribed to the list.

-John

kernel BUG at fs/jbd/checkpoint.c:247!
invalid operand: 0000 [#1]
PREEMPT SMP 
Modules linked in:
CPU:    0
EIP:    0060:[__flush_buffer+269/384]    Not tainted VLI
EFLAGS: 00010286   (2.6.12-rc5) 
EIP is at __flush_buffer+0x10d/0x180
eax: 0000005d   ebx: d72c3418   ecx: c044e8f4   edx: 00000001
esi: e8fc7ce4   edi: 00000000   ebp: e8fc7cb0   esp: e8fc7c90
ds: 007b   es: 007b   ss: 0068
Process find (pid: 6826, threadinfo=e8fc6000 task=f7f23a60)
Stack: c03f8e80 c03e0fc7 c03f4bc8 000000f7 c03f4bdc ec6791d8 d72c3418 f4cafb98 
       e8fc7df4 c01af8cd f7014544 ec6791d8 e8fc7ce8 e8fc7ce0 e8fc7ce4 e8fc6000 
       001651f8 cbd1bcd8 f72a0ce8 f7014628 00000012 00000042 d72c3418 ea265c58 
Call Trace:
 [show_stack+127/160] show_stack+0x7f/0xa0
 [show_registers+355/464] show_registers+0x163/0x1d0
 [die+257/400] die+0x101/0x190
 [do_invalid_op+188/208] do_invalid_op+0xbc/0xd0
 [error_code+79/84] error_code+0x4f/0x54
 [log_do_checkpoint+221/560] log_do_checkpoint+0xdd/0x230
 [__log_wait_for_space+178/208] __log_wait_for_space+0xb2/0xd0
 [start_this_handle+255/864] start_this_handle+0xff/0x360
 [journal_start+168/208] journal_start+0xa8/0xd0
 [ext3_dirty_inode+56/144] ext3_dirty_inode+0x38/0x90
 [__mark_inode_dirty+465/480] __mark_inode_dirty+0x1d1/0x1e0
 [update_atime+146/192] update_atime+0x92/0xc0
 [vfs_readdir+166/176] vfs_readdir+0xa6/0xb0
 [sys_getdents64+117/199] sys_getdents64+0x75/0xc7
 [syscall_call+7/11] syscall_call+0x7/0xb
Code: 3f c0 b8 dc 4b 3f c0 89 44 24 10 b8 f7 00 00 00 89 44 24 0c b8 c8 4b 3f c0 89 44 24 08 b8 c7 0f 3e c0 89 44 24 04 e8 43 b2 f6 ff <0f> 0b f7 00 c8 4b 3f c0 e9 38 ff ff ff c7 04 24 80 8e 3f c0 b9 
 <3>Debug: sleeping function called from invalid context at include/linux/rwsem.h:43
in_atomic():1, irqs_disabled():0
 [dump_stack+30/48] dump_stack+0x1e/0x30
 [__might_sleep+167/176] __might_sleep+0xa7/0xb0
 [profile_task_exit+35/96] profile_task_exit+0x23/0x60
 [do_exit+29/944] do_exit+0x1d/0x3b0
 [die+392/400] die+0x188/0x190
 [do_invalid_op+188/208] do_invalid_op+0xbc/0xd0
 [error_code+79/84] error_code+0x4f/0x54
 [log_do_checkpoint+221/560] log_do_checkpoint+0xdd/0x230
 [__log_wait_for_space+178/208] __log_wait_for_space+0xb2/0xd0
 [start_this_handle+255/864] start_this_handle+0xff/0x360
 [journal_start+168/208] journal_start+0xa8/0xd0
 [ext3_dirty_inode+56/144] ext3_dirty_inode+0x38/0x90
 [__mark_inode_dirty+465/480] __mark_inode_dirty+0x1d1/0x1e0
 [update_atime+146/192] update_atime+0x92/0xc0
 [vfs_readdir+166/176] vfs_readdir+0xa6/0xb0
 [sys_getdents64+117/199] sys_getdents64+0x75/0xc7
 [syscall_call+7/11] syscall_call+0x7/0xb

Re: 2.6.11: kernel BUG at fs/jbd/checkpoint.c:247! - Also on 2.6.12-rc5

[Andrew Morton]

John Baboval <baboval@spineless.org> wrote:
>
> On Fri, 3 Jun 2005 16:33:56 -0700 Andrew Morton <akpm@osdl.org> wrote:
>  > Please test
>  > 
>  > ftp://ftp.kernel.org/pub/linux/kernel/v2.6/testing/linux-2.6.12-rc5.tar.bz2
>  > plus
>  > ftp://ftp.kernel.org/pub/linux/kernel/v2.6/snapshots/patch-2.6.12-rc5-git8.bz2
> 
> 
>  I've just reproduced this on 2.6.12-rc5.
> 
>  It was hapening every 4 hours or so with 2.6.11.9. 2.6.12-rc5 ran for a week... The system is being used as a fairly high traffic NFS server.

Could you try this, please?


From: Jan Kara <jack@suse.cz>

On one path, cond_resched_lock() fails to return true if it dropped the lock. 
We think this might be causing the crashes in JBD's log_do_checkpoint().

Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Andrew Morton <akpm@osdl.org>
---

 kernel/sched.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff -puN kernel/sched.c~cond_resched-lock-fix kernel/sched.c
--- 25/kernel/sched.c~cond_resched-lock-fix	Mon Jun 13 15:51:22 2005
+++ 25-akpm/kernel/sched.c	Mon Jun 13 15:51:22 2005
@@ -3755,19 +3755,22 @@ EXPORT_SYMBOL(cond_resched);
  */
 int cond_resched_lock(spinlock_t * lock)
 {
+	int ret = 0;
+
 	if (need_lockbreak(lock)) {
 		spin_unlock(lock);
 		cpu_relax();
+		ret = 1;
 		spin_lock(lock);
 	}
 	if (need_resched()) {
 		_raw_spin_unlock(lock);
 		preempt_enable_no_resched();
 		__cond_resched();
+		ret = 1;
 		spin_lock(lock);
-		return 1;
 	}
-	return 0;
+	return ret;
 }
 
 EXPORT_SYMBOL(cond_resched_lock);
_