* nfs client oops, all 2.4 kernels @ 2001-09-10 15:02 ` Erik DeBill 2001-09-10 15:34 ` Stephan von Krawczynski 2001-09-11 8:45 ` Trond Myklebust 0 siblings, 2 replies; 9+ messages in thread From: Erik DeBill @ 2001-09-10 15:02 UTC (permalink / raw) To: linux-kernel, trond.myklebust I've been running into a repeatable oops in the NFS client code, apparently related to file locking. I've verified this with UP, SMP(2,4 procs), small (128M) and large (4G) memory under RedHat 7.1, SuSe 7.1 and SuSe 7.2. All on x86 hardware. The problem occurs while running DB2 EEE. DB2 EEE is a clustered database, which uses NFS to share the database home directory between nodes in the cluster. This is primarily to synchronize some config files, but by default it will also store all data and logs in that same nfs mounted directory. Oopses occur on the machine (I've only been testing on a 2 node cluster) which does the NFS client mount of the home directory. If you let it keep all it's data in the nfs mounted directory you can generate an oops very quickly once multiple clients start connecting to the db. The more clients, the faster the oops. The more thorough the job you do of removing things from NFS, the harder to reproduce. If you run it without NFS (completely unsupported) it runs fine and I haven't been able to reproduce the oops. It occurs with all 2.4 series kernels, at least up to 2.4.9-ac7 (which had some nfs client fixes) I've attached the ksymoops trace from 2 different incidents. The first occurred under 2.4.9-ac7, and is pretty typical. Death in lock related code as part of sys_close(). The second is from 2.4.9 + linux-2.4.9-NFS_ALL.dif. It's a little unusual in that several oopses happened all at once. Please let me know if there's anything I can do to help track these guys down. I can give you access to several machines with different hardware, and distros which all exhibit this bug. Thanks, Erik DeBill ksymoops 2.4.0 on i686 2.4.9-ac7. Options used -V (default) -k /proc/ksyms (default) -l /proc/modules (default) -o /lib/modules/2.4.9-ac7/ (default) -m /boot/System.map-2.4.9-ac7 (default) Warning: You did not tell me where to find symbol information. I will assume that the log matches the kernel and modules that are running right now and I'll use the default options above for symbol resolution. If the current kernel and/or modules do not match the log, you can get more accurate output by telling me the kernel version and where to find map, modules, ksyms etc. ksymoops -h explains the options. Warning (compare_maps): mismatch on symbol md_size , md says c8a0e380, /lib/modules/2.4.9-ac7/kernel/drivers/md/md.o says c8a0e1a0. Ignoring /lib/modules/2.4.9-ac7/kernel/drivers/md/md.o entry Warning (compare_maps): mismatch on symbol mddev_map , md says c8a0db80, /lib/modules/2.4.9-ac7/kernel/drivers/md/md.o says c8a0d9a0. Ignoring /lib/modules/2.4.9-ac7/kernel/drivers/md/md.o entry Sep 8 22:00:06 jerry kernel: WARNING: MP table in the EBDA can be UNSAFE, contact linux-smp@vger.kernel.org if you experience SMP problems! Sep 8 22:00:07 jerry kernel: cpu: 0, clocks: 1329124, slice: 664562 Sep 8 22:04:15 jerry kernel: Unable to handle kernel NULL pointer dereference at virtual address 00000000 Sep 8 22:04:15 jerry kernel: c0145c2b Sep 8 22:04:15 jerry kernel: *pde = 00000000 Sep 8 22:04:15 jerry kernel: Oops: 0000 Sep 8 22:04:15 jerry kernel: CPU: 0 Sep 8 22:04:15 jerry kernel: EIP: 0010:[<c0145c2b>] Using defaults from ksymoops -t elf32-i386 -a i386 Sep 8 22:04:15 jerry kernel: EFLAGS: 00010286 Sep 8 22:04:15 jerry kernel: eax: 00000000 ebx: 00000000 ecx: c72cc4e0 edx: c79155c8 Sep 8 22:04:15 jerry kernel: esi: c240b440 edi: 00000000 ebp: bfffba10 esp: c21a3f60 Sep 8 22:04:15 jerry kernel: ds: 0018 es: 0018 ss: 0018 Sep 8 22:04:15 jerry kernel: Process db2sysc (pid: 867, stackpage=c21a3000) Sep 8 22:04:15 jerry kernel: Stack: c79153fc c79155c8 c240b440 00000000 c79155c8 c240b440 c0147756 c79155c8 Sep 8 22:04:15 jerry kernel: 00000000 c21a2000 c27cc500 c0135a96 c27cc500 c240b440 00000000 c27cc500 Sep 8 22:04:15 jerry kernel: 00000000 c27cc500 00000030 00000000 c0135b0b c27cc500 c240b440 c21a2000 Sep 8 22:04:15 jerry kernel: Call Trace: [<c0147756>] [<c0135a96>] [<c0135b0b>] [<c0106efb>] Sep 8 22:04:15 jerry kernel: Code: 8b 03 8d 73 04 89 02 8b 43 04 c7 03 00 00 00 00 8b 56 04 89 >>EIP; c0145c2b <locks_delete_lock+b/d0> <===== Trace; c0147756 <locks_remove_posix+86/c0> Trace; c0135a96 <filp_close+86/a0> Trace; c0135b0b <sys_close+5b/70> Trace; c0106efb <system_call+33/38> Code; c0145c2b <locks_delete_lock+b/d0> 00000000 <_EIP>: Code; c0145c2b <locks_delete_lock+b/d0> <===== 0: 8b 03 mov (%ebx),%eax <===== Code; c0145c2d <locks_delete_lock+d/d0> 2: 8d 73 04 lea 0x4(%ebx),%esi Code; c0145c30 <locks_delete_lock+10/d0> 5: 89 02 mov %eax,(%edx) Code; c0145c32 <locks_delete_lock+12/d0> 7: 8b 43 04 mov 0x4(%ebx),%eax Code; c0145c35 <locks_delete_lock+15/d0> a: c7 03 00 00 00 00 movl $0x0,(%ebx) Code; c0145c3b <locks_delete_lock+1b/d0> 10: 8b 56 04 mov 0x4(%esi),%edx Code; c0145c3e <locks_delete_lock+1e/d0> 13: 89 00 mov %eax,(%eax) 3 warnings issued. Results may not be reliable. --------------------------------------------------------------------------- these next are from a single run under 2.4.9 + the all encompassing NFS patch. DB2 gets shut down as soon as an oops occurs, these must be from the death throes. --------------------------------------------------------------------------- ksymoops 2.4.0 on i686 2.4.9nfs. Options used -V (default) -k /proc/ksyms (default) -l /proc/modules (default) -o /lib/modules/2.4.9nfs/ (default) -m /boot/System.map-2.4.9nfs (default) Warning: You did not tell me where to find symbol information. I will assume that the log matches the kernel and modules that are running right now and I'll use the default options above for symbol resolution. If the current kernel and/or modules do not match the log, you can get more accurate output by telling me the kernel version and where to find map, modules, ksyms etc. ksymoops -h explains the options. Warning (compare_maps): mismatch on symbol md_size , md says c8a0e560, /lib/modules/2.4.9nfs/kernel/drivers/md/md.o says c8a0e380. Ignoring /lib/modules/2.4.9nfs/kernel/drivers/md/md.o entry Warning (compare_maps): mismatch on symbol mddev_map , md says c8a0dd60, /lib/modules/2.4.9nfs/kernel/drivers/md/md.o says c8a0db80. Ignoring /lib/modules/2.4.9nfs/kernel/drivers/md/md.o entry Sep 10 08:47:05 jerry kernel: WARNING: MP table in the EBDA can be UNSAFE, contact linux-smp@vger.kernel.org if you experience SMP problems! Sep 10 08:47:08 jerry kernel: cpu: 0, clocks: 1329118, slice: 664559 Sep 10 08:49:09 jerry kernel: Unable to handle kernel NULL pointer dereference at virtual address 00000000 Sep 10 08:49:09 jerry kernel: c0146cab Sep 10 08:49:09 jerry kernel: *pde = 00000000 Sep 10 08:49:09 jerry kernel: Oops: 0000 Sep 10 08:49:09 jerry kernel: CPU: 0 Sep 10 08:49:09 jerry kernel: EIP: 0010:[fcntl_setlease+171/640] Sep 10 08:49:09 jerry kernel: EIP: 0010:[<c0146cab>] Using defaults from ksymoops -t elf32-i386 -a i386 Sep 10 08:49:09 jerry kernel: EFLAGS: 00010286 Sep 10 08:49:09 jerry kernel: eax: 00000000 ebx: 00000000 ecx: c7302b60 edx: c7926d54 Sep 10 08:49:09 jerry kernel: esi: c6b76a40 edi: 00000000 ebp: bfffba10 esp: c6a4ff60 Sep 10 08:49:09 jerry kernel: ds: 0018 es: 0018 ss: 0018 Sep 10 08:49:09 jerry kernel: Process db2sysc (pid: 769, stackpage=c6a4f000) Sep 10 08:49:09 jerry kernel: Stack: c7926e0c c7926d54 c6b76a40 00000000 c7926d54 c6b76a40 c01488f6 c7926d54 Sep 10 08:49:09 jerry kernel: 00000000 c6a4e000 c6a23d40 c013512d c6a23d40 c6b76a40 00000000 c6a23d40 Sep 10 08:49:09 jerry kernel: 00000000 c6a23d40 00000030 00000000 c013519b c6a23d40 c6b76a40 c6a4e000 Sep 10 08:49:09 jerry kernel: Call Trace: [d_validate+54/224] [sys_chdir+253/256] [sys_fchdir+107/208] [system_call+19/56] Sep 10 08:49:09 jerry kernel: Call Trace: [<c01488f6>] [<c013512d>] [<c013519b>] [<c0106edb>] Sep 10 08:49:09 jerry kernel: Code: 8b 03 8d 73 04 89 02 8b 43 04 c7 03 00 00 00 00 8b 56 04 89 >>EIP; c0146cab <locks_delete_lock+b/d0> <===== Trace; c01488f6 <locks_remove_posix+86/e0> Trace; c013512d <filp_close+9d/b0> Trace; c013519b <sys_close+5b/70> Trace; c0106edb <system_call+33/38> Code; c0146cab <locks_delete_lock+b/d0> 00000000 <_EIP>: Code; c0146cab <locks_delete_lock+b/d0> <===== 0: 8b 03 mov (%ebx),%eax <===== Code; c0146cad <locks_delete_lock+d/d0> 2: 8d 73 04 lea 0x4(%ebx),%esi Code; c0146cb0 <locks_delete_lock+10/d0> 5: 89 02 mov %eax,(%edx) Code; c0146cb2 <locks_delete_lock+12/d0> 7: 8b 43 04 mov 0x4(%ebx),%eax Code; c0146cb5 <locks_delete_lock+15/d0> a: c7 03 00 00 00 00 movl $0x0,(%ebx) Code; c0146cbb <locks_delete_lock+1b/d0> 10: 8b 56 04 mov 0x4(%esi),%edx Code; c0146cbe <locks_delete_lock+1e/d0> 13: 89 00 mov %eax,(%eax) Sep 10 08:49:09 jerry kernel: Unable to handle kernel NULL pointer dereference at virtual address 00000000 Sep 10 08:49:09 jerry kernel: c0146cab Sep 10 08:49:09 jerry kernel: *pde = 00000000 Sep 10 08:49:09 jerry kernel: Oops: 0000 Sep 10 08:49:09 jerry kernel: CPU: 0 Sep 10 08:49:09 jerry kernel: EIP: 0010:[fcntl_setlease+171/640] Sep 10 08:49:10 jerry kernel: EIP: 0010:[<c0146cab>] Sep 10 08:49:10 jerry kernel: EFLAGS: 00010286 Sep 10 08:49:10 jerry kernel: eax: 00000000 ebx: 00000000 ecx: c7302b60 edx: c7926be4 Sep 10 08:49:10 jerry kernel: esi: c6b76700 edi: 00000000 ebp: bfffba10 esp: c5d15f60 Sep 10 08:49:10 jerry kernel: ds: 0018 es: 0018 ss: 0018 Sep 10 08:49:10 jerry kernel: Process db2sysc (pid: 770, stackpage=c5d15000) Sep 10 08:49:10 jerry kernel: Stack: c7926f20 c7926be4 c6b76700 00000000 c7926be4 c6b76700 c01488f6 c7926be4 Sep 10 08:49:10 jerry kernel: 00000000 c5d14000 c6a23f20 c013512d c6a23f20 c6b76700 00000000 c6a23f20 Sep 10 08:49:10 jerry kernel: 00000000 c6a23f20 00000030 00000000 c013519b c6a23f20 c6b76700 c5d14000 Sep 10 08:49:10 jerry kernel: Call Trace: [d_validate+54/224] [sys_chdir+253/256] [sys_fchdir+107/208] [system_call+19/56] Sep 10 08:49:10 jerry kernel: Call Trace: [<c01488f6>] [<c013512d>] [<c013519b>] [<c0106edb>] Sep 10 08:49:10 jerry kernel: Code: 8b 03 8d 73 04 89 02 8b 43 04 c7 03 00 00 00 00 8b 56 04 89 >>EIP; c0146cab <locks_delete_lock+b/d0> <===== Trace; c01488f6 <locks_remove_posix+86/e0> Trace; c013512d <filp_close+9d/b0> Trace; c013519b <sys_close+5b/70> Trace; c0106edb <system_call+33/38> Code; c0146cab <locks_delete_lock+b/d0> 00000000 <_EIP>: Code; c0146cab <locks_delete_lock+b/d0> <===== 0: 8b 03 mov (%ebx),%eax <===== Code; c0146cad <locks_delete_lock+d/d0> 2: 8d 73 04 lea 0x4(%ebx),%esi Code; c0146cb0 <locks_delete_lock+10/d0> 5: 89 02 mov %eax,(%edx) Code; c0146cb2 <locks_delete_lock+12/d0> 7: 8b 43 04 mov 0x4(%ebx),%eax Code; c0146cb5 <locks_delete_lock+15/d0> a: c7 03 00 00 00 00 movl $0x0,(%ebx) Code; c0146cbb <locks_delete_lock+1b/d0> 10: 8b 56 04 mov 0x4(%esi),%edx Code; c0146cbe <locks_delete_lock+1e/d0> 13: 89 00 mov %eax,(%eax) Sep 10 08:49:10 jerry kernel: Unable to handle kernel NULL pointer dereference at virtual address 00000000 Sep 10 08:49:10 jerry kernel: c0146cab Sep 10 08:49:10 jerry kernel: *pde = 00000000 Sep 10 08:49:10 jerry kernel: Oops: 0000 Sep 10 08:49:10 jerry kernel: CPU: 0 Sep 10 08:49:10 jerry kernel: EIP: 0010:[fcntl_setlease+171/640] Sep 10 08:49:10 jerry kernel: EIP: 0010:[<c0146cab>] Sep 10 08:49:10 jerry kernel: EFLAGS: 00010286 Sep 10 08:49:10 jerry kernel: eax: 00000000 ebx: 00000000 ecx: c7302b60 edx: c7926c40 Sep 10 08:49:10 jerry kernel: esi: c6439dc0 edi: 00000000 ebp: bfffba10 esp: c643bf60 Sep 10 08:49:10 jerry kernel: ds: 0018 es: 0018 ss: 0018 Sep 10 08:49:10 jerry kernel: Process db2sysc (pid: 776, stackpage=c643b000) Sep 10 08:49:10 jerry kernel: Stack: c7926db0 c7926c40 c6439dc0 00000000 c7926c40 c6439dc0 c01488f6 c7926c40 Sep 10 08:49:10 jerry kernel: 00000000 c643a000 c6a23da0 c013512d c6a23da0 c6439dc0 00000000 c6a23da0 Sep 10 08:49:10 jerry kernel: 00000000 c6a23da0 00000030 00000000 c013519b c6a23da0 c6439dc0 c643a000 Sep 10 08:49:10 jerry kernel: Call Trace: [d_validate+54/224] [sys_chdir+253/256] [sys_fchdir+107/208] [system_call+19/56] Sep 10 08:49:10 jerry kernel: Call Trace: [<c01488f6>] [<c013512d>] [<c013519b>] [<c0106edb>] Sep 10 08:49:10 jerry kernel: Code: 8b 03 8d 73 04 89 02 8b 43 04 c7 03 00 00 00 00 8b 56 04 89 >>EIP; c0146cab <locks_delete_lock+b/d0> <===== Trace; c01488f6 <locks_remove_posix+86/e0> Trace; c013512d <filp_close+9d/b0> Trace; c013519b <sys_close+5b/70> Trace; c0106edb <system_call+33/38> Code; c0146cab <locks_delete_lock+b/d0> 00000000 <_EIP>: Code; c0146cab <locks_delete_lock+b/d0> <===== 0: 8b 03 mov (%ebx),%eax <===== Code; c0146cad <locks_delete_lock+d/d0> 2: 8d 73 04 lea 0x4(%ebx),%esi Code; c0146cb0 <locks_delete_lock+10/d0> 5: 89 02 mov %eax,(%edx) Code; c0146cb2 <locks_delete_lock+12/d0> 7: 8b 43 04 mov 0x4(%ebx),%eax Code; c0146cb5 <locks_delete_lock+15/d0> a: c7 03 00 00 00 00 movl $0x0,(%ebx) Code; c0146cbb <locks_delete_lock+1b/d0> 10: 8b 56 04 mov 0x4(%esi),%edx Code; c0146cbe <locks_delete_lock+1e/d0> 13: 89 00 mov %eax,(%eax) Sep 10 08:49:10 jerry kernel: Unable to handle kernel NULL pointer dereference at virtual address 00000000 Sep 10 08:49:10 jerry kernel: c0146cab Sep 10 08:49:10 jerry kernel: *pde = 00000000 Sep 10 08:49:10 jerry kernel: Oops: 0000 Sep 10 08:49:10 jerry kernel: CPU: 0 Sep 10 08:49:10 jerry kernel: EIP: 0010:[fcntl_setlease+171/640] Sep 10 08:49:10 jerry kernel: EIP: 0010:[<c0146cab>] Sep 10 08:49:10 jerry kernel: EFLAGS: 00010286 Sep 10 08:49:10 jerry kernel: eax: 00000000 ebx: 00000000 ecx: c7302b60 edx: c7926b88 Sep 10 08:49:10 jerry kernel: esi: c4275240 edi: 00000000 ebp: bfffba10 esp: c5d8df60 Sep 10 08:49:10 jerry kernel: ds: 0018 es: 0018 ss: 0018 Sep 10 08:49:10 jerry kernel: Process db2sysc (pid: 774, stackpage=c5d8d000) Sep 10 08:49:10 jerry kernel: Stack: c7926b2c c7926b88 c4275240 00000000 c7926b88 c4275240 c01488f6 c7926b88 Sep 10 08:49:11 jerry kernel: 00000000 c5d8c000 c71e30c0 c013512d c71e30c0 c4275240 00000000 c71e30c0 Sep 10 08:49:11 jerry kernel: 00000000 c71e30c0 00000030 00000000 c013519b c71e30c0 c4275240 c5d8c000 Sep 10 08:49:11 jerry kernel: Call Trace: [d_validate+54/224] [sys_chdir+253/256] [sys_fchdir+107/208] [system_call+19/56] Sep 10 08:49:11 jerry kernel: Call Trace: [<c01488f6>] [<c013512d>] [<c013519b>] [<c0106edb>] Sep 10 08:49:11 jerry kernel: Code: 8b 03 8d 73 04 89 02 8b 43 04 c7 03 00 00 00 00 8b 56 04 89 >>EIP; c0146cab <locks_delete_lock+b/d0> <===== Trace; c01488f6 <locks_remove_posix+86/e0> Trace; c013512d <filp_close+9d/b0> Trace; c013519b <sys_close+5b/70> Trace; c0106edb <system_call+33/38> Code; c0146cab <locks_delete_lock+b/d0> 00000000 <_EIP>: Code; c0146cab <locks_delete_lock+b/d0> <===== 0: 8b 03 mov (%ebx),%eax <===== Code; c0146cad <locks_delete_lock+d/d0> 2: 8d 73 04 lea 0x4(%ebx),%esi Code; c0146cb0 <locks_delete_lock+10/d0> 5: 89 02 mov %eax,(%edx) Code; c0146cb2 <locks_delete_lock+12/d0> 7: 8b 43 04 mov 0x4(%ebx),%eax Code; c0146cb5 <locks_delete_lock+15/d0> a: c7 03 00 00 00 00 movl $0x0,(%ebx) Code; c0146cbb <locks_delete_lock+1b/d0> 10: 8b 56 04 mov 0x4(%esi),%edx Code; c0146cbe <locks_delete_lock+1e/d0> 13: 89 00 mov %eax,(%eax) 3 warnings issued. Results may not be reliable. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: nfs client oops, all 2.4 kernels 2001-09-10 15:02 ` nfs client oops, all 2.4 kernels Erik DeBill @ 2001-09-10 15:34 ` Stephan von Krawczynski 2001-09-10 19:52 ` Olivier Molteni 2001-09-11 8:45 ` Trond Myklebust 1 sibling, 1 reply; 9+ messages in thread From: Stephan von Krawczynski @ 2001-09-10 15:34 UTC (permalink / raw) To: Erik DeBill; +Cc: linux-kernel, trond.myklebust On Mon, 10 Sep 2001 10:02:02 -0500 Erik DeBill <erik@www.creditminders.com> wrote: > I've been running into a repeatable oops in the NFS client code, > apparently related to file locking. > Sep 10 08:49:10 jerry kernel: Unable to handle kernel NULL pointer dereference at virtual address 00000000 > Sep 10 08:49:10 jerry kernel: c0146cab > Sep 10 08:49:10 jerry kernel: *pde = 00000000 > Sep 10 08:49:10 jerry kernel: Oops: 0000 > Sep 10 08:49:10 jerry kernel: CPU: 0 > Sep 10 08:49:10 jerry kernel: EIP: 0010:[fcntl_setlease+171/640] > Sep 10 08:49:10 jerry kernel: EIP: 0010:[<c0146cab>] > Sep 10 08:49:10 jerry kernel: EFLAGS: 00010286 > Sep 10 08:49:10 jerry kernel: eax: 00000000 ebx: 00000000 ecx: c7302b60 edx: c7926b88 > Sep 10 08:49:10 jerry kernel: esi: c4275240 edi: 00000000 ebp: bfffba10 esp: c5d8df60 > Sep 10 08:49:10 jerry kernel: ds: 0018 es: 0018 ss: 0018 > Sep 10 08:49:10 jerry kernel: Process db2sysc (pid: 774, stackpage=c5d8d000) > Sep 10 08:49:10 jerry kernel: Stack: c7926b2c c7926b88 c4275240 00000000 c7926b88 c4275240 c01488f6 c7926b88 > Sep 10 08:49:11 jerry kernel: 00000000 c5d8c000 c71e30c0 c013512d c71e30c0 c4275240 00000000 c71e30c0 > Sep 10 08:49:11 jerry kernel: 00000000 c71e30c0 00000030 00000000 c013519b c71e30c0 c4275240 c5d8c000 > Sep 10 08:49:11 jerry kernel: Call Trace: [d_validate+54/224] [sys_chdir+253/256] [sys_fchdir+107/208] [system_call+19/56] > Sep 10 08:49:11 jerry kernel: Call Trace: [<c01488f6>] [<c013512d>] [<c013519b>] [<c0106edb>] > Sep 10 08:49:11 jerry kernel: Code: 8b 03 8d 73 04 89 02 8b 43 04 c7 03 00 00 00 00 8b 56 04 89 > > >>EIP; c0146cab <locks_delete_lock+b/d0> <===== > Trace; c01488f6 <locks_remove_posix+86/e0> > Trace; c013512d <filp_close+9d/b0> > Trace; c013519b <sys_close+5b/70> > Trace; c0106edb <system_call+33/38> > Code; c0146cab <locks_delete_lock+b/d0> > 00000000 <_EIP>: > Code; c0146cab <locks_delete_lock+b/d0> <===== > 0: 8b 03 mov (%ebx),%eax <===== > Code; c0146cad <locks_delete_lock+d/d0> > 2: 8d 73 04 lea 0x4(%ebx),%esi > Code; c0146cb0 <locks_delete_lock+10/d0> > 5: 89 02 mov %eax,(%edx) > Code; c0146cb2 <locks_delete_lock+12/d0> > 7: 8b 43 04 mov 0x4(%ebx),%eax > Code; c0146cb5 <locks_delete_lock+15/d0> > a: c7 03 00 00 00 00 movl $0x0,(%ebx) > Code; c0146cbb <locks_delete_lock+1b/d0> > 10: 8b 56 04 mov 0x4(%esi),%edx > Code; c0146cbe <locks_delete_lock+1e/d0> > 13: 89 00 mov %eax,(%eax) > Hm, I am sure no nfs or fs guru, but looking at static void locks_delete_lock(struct file_lock **thisfl_p, unsigned int wait) { struct file_lock *fl = *thisfl_p; *thisfl_p = fl->fl_next; fl->fl_next = NULL; list_del(&fl->fl_link); INIT_LIST_HEAD(&fl->fl_link); fasync_helper(0, fl->fl_file, 0, &fl->fl_fasync); if (fl->fl_fasync != NULL){ printk(KERN_ERR "locks_delete_lock: fasync == %p\n", fl->fl_fasync); fl->fl_fasync = NULL; } if (fl->fl_remove) fl->fl_remove(fl); locks_wake_up_blocks(fl, wait); locks_free_lock(fl); } in linux/fs/locks.c I would say it fails either because thisfl_p is NULL or *thisfl_p is NULL. Try securing it via: static void locks_delete_lock(struct file_lock **thisfl_p, unsigned int wait) { struct file_lock *fl; if (thisfl_p == NULL || *thisfl_p == NULL) return; fl = *thisfl_p; *thisfl_p = fl->fl_next; fl->fl_next = NULL; ... } This is for sure not the cure, but may help your setup. Regards, Stephan ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: nfs client oops, all 2.4 kernels 2001-09-10 15:34 ` Stephan von Krawczynski @ 2001-09-10 19:52 ` Olivier Molteni 2001-09-10 20:24 ` edebill 0 siblings, 1 reply; 9+ messages in thread From: Olivier Molteni @ 2001-09-10 19:52 UTC (permalink / raw) To: Stephan von Krawczynski; +Cc: Erik DeBill, linux-kernel, trond.myklebust Stephan von Krawczynski wrote: > On Mon, 10 Sep 2001 10:02:02 -0500 Erik DeBill <erik@www.creditminders.com> > wrote: > > > I've been running into a repeatable oops in the NFS client code, > > apparently related to file locking. > > > in linux/fs/locks.c I would say it fails either because thisfl_p is NULL or > *thisfl_p is NULL. Try securing it via: > > static void locks_delete_lock(struct file_lock **thisfl_p, unsigned int wait) > { > struct file_lock *fl; > > if (thisfl_p == NULL || *thisfl_p == NULL) > return; > > fl = *thisfl_p; > > *thisfl_p = fl->fl_next; > fl->fl_next = NULL; > > ... > } > > This is for sure not the cure, but may help your setup. Hi, see my post and related answers [ Oops NFS Locking in 2.4.x] I have the same Problem. Returning on *thisfl_p == NULL don't fix the trouble... Kernel no more Oops, but process stay in wait state on IO (D). See the answers from Trond Myklebust, I think he is right... Regards, Olivier. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: nfs client oops, all 2.4 kernels 2001-09-10 19:52 ` Olivier Molteni @ 2001-09-10 20:24 ` edebill 0 siblings, 0 replies; 9+ messages in thread From: edebill @ 2001-09-10 20:24 UTC (permalink / raw) To: Olivier Molteni; +Cc: linux-kernel On Mon, Sep 10, 2001 at 09:52:10PM +0200, Olivier Molteni wrote: > > Hi, see my post and related answers [ Oops NFS Locking in 2.4.x] I have the same > Problem. > Returning on *thisfl_p == NULL don't fix the trouble... Kernel no more Oops, but > process stay in wait state on IO (D). > > See the answers from Trond Myklebust, I think he is right... That's what I get for not reading LKML over the weekend :-) Sounds like he has a handle on the problem - I'll gladly test patches if one of you comes up with a (proposed) fix. Erik ^ permalink raw reply [flat|nested] 9+ messages in thread
* nfs client oops, all 2.4 kernels 2001-09-10 15:02 ` nfs client oops, all 2.4 kernels Erik DeBill 2001-09-10 15:34 ` Stephan von Krawczynski @ 2001-09-11 8:45 ` Trond Myklebust 2001-09-11 13:15 ` Trond Myklebust 1 sibling, 1 reply; 9+ messages in thread From: Trond Myklebust @ 2001-09-11 8:45 UTC (permalink / raw) To: Erik DeBill; +Cc: linux-kernel, Olivier Molteni, Francis Galiegue >>>>> " " == Erik DeBill <erik@www.creditminders.com> writes: > I've been running into a repeatable oops in the NFS client > code, apparently related to file locking. This is the same Oops as was sent in by Olivier Molteni. It's due to an incorrect assumption in the locking code. To repeat: 2 processes are calling close(), which under POSIX, tries to free all locks held by a program. The problem occurs when you `dup()' a file. In that case, both processes can call close() at the same time, thus triggering a call to filp_close(). The race boils down to: - locks_unlock_delete() assumes that the BKL (kernel_lock()) is sufficient to protect against *thisfl_p from disappearing beneath it due to some second process. BUT - The call to lock() in locks_unlock_delete() sleeps when the underlying filesystem is NFS, hence 2 processes can race despite the BKL assumption. IMHO the correct thing to do is to detach the lock from the global linked list *before* one makes the call to f_op->lock() down to the NFS code. This would probably entail splitting locks_delete_lock() into 2 procedures: one that does the unlinking from the global list: *thisfl_p = fl->fl_next; fl->fl_next = NULL; list_del(&fl->fl_link); INIT_LIST_HEAD(&fl->fl_link); the other that does the rest. Could you check if the appended patch works? Cheers, Trond --- linux-2.4.9/fs/locks.c.orig Thu Jul 5 00:39:28 2001 +++ linux-2.4.9/fs/locks.c Tue Sep 11 10:41:53 2001 @@ -473,21 +473,26 @@ fl->fl_insert(fl); } -/* Delete a lock and then free it. - * Remove our lock from the lock lists, wake up processes that are blocked - * waiting for this lock, notify the FS that the lock has been cleared and - * finally free the lock. +/* + * Remove lock from the lock lists */ -static void locks_delete_lock(struct file_lock **thisfl_p, unsigned int wait) +static inline void _unhash_lock(struct file_lock **thisfl_p) { struct file_lock *fl = *thisfl_p; *thisfl_p = fl->fl_next; fl->fl_next = NULL; - list_del(&fl->fl_link); - INIT_LIST_HEAD(&fl->fl_link); + list_del_init(&fl->fl_link); +} +/* + * Wake up processes that are blocked waiting for this lock, + * notify the FS that the lock has been cleared and + * finally free the lock. + */ +static inline void _delete_lock(struct file_lock *fl, unsigned int wait) +{ fasync_helper(0, fl->fl_file, 0, &fl->fl_fasync); if (fl->fl_fasync != NULL){ printk(KERN_ERR "locks_delete_lock: fasync == %p\n", fl->fl_fasync); @@ -502,6 +507,15 @@ } /* + * Delete a lock and then free it. + */ +static void locks_delete_lock(struct file_lock **thisfl_p, unsigned int wait) +{ + _unhash_lock(thisfl_p); + _delete_lock(*thisfl_p, wait); +} + +/* * Call back client filesystem in order to get it to unregister a lock, * then delete lock. Essentially useful only in locks_remove_*(). * Note: this must be called with the semaphore already held! @@ -511,12 +525,13 @@ struct file_lock *fl = *thisfl_p; int (*lock)(struct file *, int, struct file_lock *); + _unhash_lock(thisfl_p); if (fl->fl_file->f_op && (lock = fl->fl_file->f_op->lock) != NULL) { fl->fl_type = F_UNLCK; lock(fl->fl_file, F_SETLK, fl); } - locks_delete_lock(thisfl_p, 0); + _delete_lock(fl, 0); } /* Determine if lock sys_fl blocks lock caller_fl. Common functionality @@ -1661,6 +1676,7 @@ while ((fl = *before) != NULL) { if ((fl->fl_flags & FL_POSIX) && fl->fl_owner == owner) { locks_unlock_delete(before); + before = &inode->i_flock; continue; } before = &fl->fl_next; ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: nfs client oops, all 2.4 kernels 2001-09-11 8:45 ` Trond Myklebust @ 2001-09-11 13:15 ` Trond Myklebust 2001-09-11 17:26 ` Erik DeBill ` (2 more replies) 0 siblings, 3 replies; 9+ messages in thread From: Trond Myklebust @ 2001-09-11 13:15 UTC (permalink / raw) To: Erik DeBill, Olivier Molteni; +Cc: linux-kernel, Francis Galiegue >>>>> " " == Trond Myklebust <trond.myklebust@fys.uio.no> writes: > Could you check if the appended patch works? The previous patch had a bug in the rewrite of locks_delete_lock() which will Oops/corrupt the inode->i_flock list. Please consign to /dev/null, and try the following instead. (Note to self: Never attempt to write any patches *before* lunch...) Cheers, Trond --- linux-2.4.9/fs/locks.c.orig Thu Jul 5 00:39:28 2001 +++ linux-2.4.9/fs/locks.c Tue Sep 11 15:09:13 2001 @@ -473,21 +473,26 @@ fl->fl_insert(fl); } -/* Delete a lock and then free it. - * Remove our lock from the lock lists, wake up processes that are blocked - * waiting for this lock, notify the FS that the lock has been cleared and - * finally free the lock. +/* + * Remove lock from the lock lists */ -static void locks_delete_lock(struct file_lock **thisfl_p, unsigned int wait) +static inline void _unhash_lock(struct file_lock **thisfl_p) { struct file_lock *fl = *thisfl_p; *thisfl_p = fl->fl_next; fl->fl_next = NULL; - list_del(&fl->fl_link); - INIT_LIST_HEAD(&fl->fl_link); + list_del_init(&fl->fl_link); +} +/* + * Wake up processes that are blocked waiting for this lock, + * notify the FS that the lock has been cleared and + * finally free the lock. + */ +static inline void _delete_lock(struct file_lock *fl, unsigned int wait) +{ fasync_helper(0, fl->fl_file, 0, &fl->fl_fasync); if (fl->fl_fasync != NULL){ printk(KERN_ERR "locks_delete_lock: fasync == %p\n", fl->fl_fasync); @@ -502,6 +507,17 @@ } /* + * Delete a lock and then free it. + */ +static void locks_delete_lock(struct file_lock **thisfl_p, unsigned int wait) +{ + struct file_lock *fl = *thisfl_p; + + _unhash_lock(thisfl_p); + _delete_lock(fl, wait); +} + +/* * Call back client filesystem in order to get it to unregister a lock, * then delete lock. Essentially useful only in locks_remove_*(). * Note: this must be called with the semaphore already held! @@ -511,12 +527,13 @@ struct file_lock *fl = *thisfl_p; int (*lock)(struct file *, int, struct file_lock *); + _unhash_lock(thisfl_p); if (fl->fl_file->f_op && (lock = fl->fl_file->f_op->lock) != NULL) { fl->fl_type = F_UNLCK; lock(fl->fl_file, F_SETLK, fl); } - locks_delete_lock(thisfl_p, 0); + _delete_lock(fl, 0); } /* Determine if lock sys_fl blocks lock caller_fl. Common functionality @@ -1661,6 +1678,7 @@ while ((fl = *before) != NULL) { if ((fl->fl_flags & FL_POSIX) && fl->fl_owner == owner) { locks_unlock_delete(before); + before = &inode->i_flock; continue; } before = &fl->fl_next; ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: nfs client oops, all 2.4 kernels 2001-09-11 13:15 ` Trond Myklebust @ 2001-09-11 17:26 ` Erik DeBill 2001-09-12 1:52 ` Olivier Molteni 2001-09-13 1:05 ` Olivier Molteni 2 siblings, 0 replies; 9+ messages in thread From: Erik DeBill @ 2001-09-11 17:26 UTC (permalink / raw) To: Trond Myklebust; +Cc: linux-kernel On Tue, Sep 11, 2001 at 03:15:34PM +0200, Trond Myklebust wrote: > >>>>> " " == Trond Myklebust <trond.myklebust@fys.uio.no> writes: > > > Could you check if the appended patch works? > > The previous patch had a bug in the rewrite of locks_delete_lock() > which will Oops/corrupt the inode->i_flock list. Please consign to > /dev/null, and try the following instead. > > (Note to self: Never attempt to write any patches *before* lunch...) This patch seems to work. I'll keep testing, but it's been running well for an hour or so already. Thanks, Erik ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: nfs client oops, all 2.4 kernels 2001-09-11 13:15 ` Trond Myklebust 2001-09-11 17:26 ` Erik DeBill @ 2001-09-12 1:52 ` Olivier Molteni 2001-09-13 1:05 ` Olivier Molteni 2 siblings, 0 replies; 9+ messages in thread From: Olivier Molteni @ 2001-09-12 1:52 UTC (permalink / raw) To: Trond Myklebust; +Cc: linux-kernel Trond Myklebust wrote: > >>>>> " " == Trond Myklebust <trond.myklebust@fys.uio.no> writes: > > > Could you check if the appended patch works? Hi, First, thank you very much for posting this patch !! Because I'm awake and at work for too long, I have no had time to test it... Now, I MUST to sleep if I want to survive ;)) But, I promise I will test your patch ASAP and I will give you a feed back on it. Cheers Olivier. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: nfs client oops, all 2.4 kernels 2001-09-11 13:15 ` Trond Myklebust 2001-09-11 17:26 ` Erik DeBill 2001-09-12 1:52 ` Olivier Molteni @ 2001-09-13 1:05 ` Olivier Molteni 2 siblings, 0 replies; 9+ messages in thread From: Olivier Molteni @ 2001-09-13 1:05 UTC (permalink / raw) To: Trond Myklebust; +Cc: linux-kernel Trond Myklebust wrote: > >>>>> " " == Trond Myklebust <trond.myklebust@fys.uio.no> writes: > > > Could you check if the appended patch works? > Hi, I have just finished to test with 4 PCs doing intensive lock/unlock on several NFS files for 3 hours... it seems to work ! I will now test with the initial mail application, but if I will post again only if the test fails... Thank's a lot again !! Cheers, Olivier. ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2001-09-13 1:06 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <"from olivier"@molteni.net>
2001-09-10 15:02 ` nfs client oops, all 2.4 kernels Erik DeBill
2001-09-10 15:34 ` Stephan von Krawczynski
2001-09-10 19:52 ` Olivier Molteni
2001-09-10 20:24 ` edebill
2001-09-11 8:45 ` Trond Myklebust
2001-09-11 13:15 ` Trond Myklebust
2001-09-11 17:26 ` Erik DeBill
2001-09-12 1:52 ` Olivier Molteni
2001-09-13 1:05 ` Olivier Molteni
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox