Re: Linux v2.6.8 - Oops on NFSv3

Willy Tarreau
2004-08-14T10:34:18Z

Hi Again,

sorry for the noise, I've just found a similar report that Jeff sent
yesterday with a patch. I've applied it and it now runs reliably. It's
a bit of a shame that it didn't get into mainline just in time.

Cheers,
Willy

On Sat, Aug 14, 2004 at 12:10:39PM +0200, Willy Tarreau wrote:
> Hi Linus & All,
>
> I've just compiled and booted 2.6.8 on my dual athlon. Everything went
> OK before I logged in as a non-root user whose home is mounted from
> another linux box over NFSv3/UDP.
>
> I got this oops :
>
> <-----------
> Unable to handle kernel NULL pointer dereference at virtual address 00000014
> printing eip:
> c01f1e76
> *pde = 00000000
> Oops: 0002 [#1]
> PREEMPT SMP
> Modules linked in: usbmouse ohci_hcd usbcore e1000 snd_seq_midi snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_emu10k1 snd_rawmidi snd_ac97_codec snd_util_mem snd_hwdep snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_pcm snd_page_alloc snd_timer snd_mixer_oss snd soundcore
> w83781d i2c_sensor i2c_amd756 i2c_core bsd_comp ppp_generic slhc
> CPU: 1
> EIP: 0060:[<c01f1e76>] Not tainted
> EFLAGS: 00010296 (2.6.8)
> EIP is at nfs3_request_init+0x16/0x20
> eax: 00000000 ebx: d5496620 ecx: 00000000 edx: df614880
> esi: dc262000 edi: dbb4bd68 ebp: 00000000 esp: dc263c44
> ds: 007b es: 007b ss: 0068
> Process bash (pid: 460, threadinfo=dc262000 task=dfc3d730)
> Stack: dff96e00 c01ea7e8 d5496620 df614880 00000174 dbb4bd68 c12bd3c0 dc263cb0
> d5496620 c01ed162 df614880 dbb4bd68 c12bd3c0 00000000 00000174 c12bd3d8
> c12bd3c0 dc263dd0 dc263cb0 c014170c dc263d20 c12bd3c0 dc263d28 dc263d28
> Call Trace:
> [<c01ea7e8>] nfs_create_request+0xd8/0xf0
> [<c01ed162>] readpage_async_filler+0x52/0x100
> [<c014170c>] read_cache_pages+0xac/0x160
> [<c011e6b0>] autoremove_wake_function+0x0/0x40
> [<c012985e>] recalc_sigpending+0xe/0x10
> [<c03e3923>] rpc_call_sync+0x93/0xb0
> [<c01ed29c>] nfs_readpages+0x8c/0xc0
> [<c01ed110>] readpage_async_filler+0x0/0x100
> [<c01417f5>] read_pages+0x35/0x130
> [<c013eac4>] __alloc_pages+0xc4/0x320
> [<c013ed0f>] __alloc_pages+0x30f/0x320
> [<c0141cf2>] do_page_cache_readahead+0x1b2/0x1e0
> [<c0141e5b>] page_cache_readahead+0x13b/0x1c0
> [<c013ae88>] do_generic_mapping_read+0xb8/0x410
> [<c013b488>] __generic_file_aio_read+0x1d8/0x200
> [<c013b1e0>] file_read_actor+0x0/0xd0
> [<c013b4f5>] generic_file_aio_read+0x45/0x50
> [<c01e6128>] nfs_file_read+0xb8/0xd0
> [<c0157e6a>] do_sync_read+0x7a/0xb0
> [<c0157f55>] vfs_read+0xb5/0xf0
> [<c0158170>] sys_read+0x40/0x70
> [<c01059e3>] syscall_call+0x7/0xb
> Code: f0 ff 40 14 89 43 18 5b c3 90 8b 4c 24 04 8b 51 0c 3b 54 24
>
> >>EIP; c01f1e76 <nfs3_request_init+16/20> <=====
> Code; c01f1e76 <nfs3_request_init+16/20>
> 00000000 <_EIP>:
> Code; c01f1e76 <nfs3_request_init+16/20> <=====
> 0: f0 ff 40 14 lock incl 0x14(%eax) <=====
> Code; c01f1e7a <nfs3_request_init+1a/20>
> 4: 89 43 18 mov %eax,0x18(%ebx)
> Code; c01f1e7d <nfs3_request_init+1d/20>
> 7: 5b pop %ebx
> Code; c01f1e7e <nfs3_request_init+1e/20>
> 8: c3 ret
> Code; c01f1e7f <nfs3_request_init+1f/20>
> 9: 90 nop
> Code; c01f1e80 <nfs3_request_compatible+0/60>
> a: 8b 4c 24 04 mov 0x4(%esp,1),%ecx
> Code; c01f1e84 <nfs3_request_compatible+4/60>
> e: 8b 51 0c mov 0xc(%ecx),%edx
> Code; c01f1e87 <nfs3_request_compatible+7/60>
> 11: 3b 54 24 00 cmp 0x0(%esp,1),%edx
> ----------->
>
> Extracts from the functions where the oops occured :
>
> void
> nfs3_request_init(struct nfs_page *req, struct file *filp)
> {
> req->wb_cred = get_rpccred(nfs_cred(req->wb_inode, filp));
> }
>
> static inline
> struct rpc_cred * get_rpccred(struct rpc_cred *cred)
> {
> atomic_inc(&cred->cr_count);
> return cred;
> }
>
> static struct rpc_cred *
> nfs_cred(struct inode *inode, struct file *filp)
> {
> struct rpc_cred *cred = NULL;
>
> if (filp)
> cred = (struct rpc_cred *)filp->private_data;
> if (!cred)
> cred = NFS_I(inode)->mm_cred;
> return cred;
> }
>
> static inline struct nfs_inode *NFS_I(struct inode *inode)
> {
> return container_of(inode, struct nfs_inode, vfs_inode);
> }
>
> So it seems to me that both (filp or filp->private_data) and
> NFS_I(req->wb_inode)->mm_cred were NULL. I don't know if this
> is a permitted situation, but directly calling get_rpccred()
> from this leads to this oops.
>
> Last 2.6 kernel I tried on this machine was 2.6.7, and I did
> not notice this oops. Unfortunately, I don't have enough time
> to dig through the large patch (19MB uncompressed), nor to try
> different configurations. But this seems reproducible anytime
> I try to access a file over NFSv3. Directory listings seem OK
> at the moment. I've not tested other combinations of NFS/config
> either (I'm typing this mail from the machine, so I don't want
> to risk retyping everything). Since it's reproducible on a
> single request, I don't think that PREEMPT/SMP has anything to
> do with this.
>
> I can send other informations if needed. If anyone wants to
> suggest a patch, I can give it a try, but I won't have much
> time to help debugging this, unfortunately.
>
> Cheers,
> Willy