Linux NFS development
 help / color / mirror / Atom feed
* Re: [BUG] unable to handle kernel NULL pointer dereference
@ 2014-02-17  4:46 Dieter Nützel
  0 siblings, 0 replies; 7+ messages in thread
From: Dieter Nützel @ 2014-02-17  4:46 UTC (permalink / raw)
  To: Trond Myklebust; +Cc: linux-nfs, John

> Subject: Re: [BUG] unable to handle kernel NULL pointer dereference
> From: Trond Myklebust <trond.myklebust@xxxxxxxxxxxxxxx>
> Date: Sun, 16 Feb 2014 12:27:33 -0500
> Cc: John <da_audiophile@xxxxxxxxx>, lkml 
> <linux-kernel@xxxxxxxxxxxxxxx>, "netdev@xxxxxxxxxxxxxxx" 
> <netdev@xxxxxxxxxxxxxxx>, "stephen@xxxxxxxxxxxxxxxxxx" 
> <stephen@xxxxxxxxxxxxxxxxxx>, "mlindner@xxxxxxxxxxx" 
> <mlindner@xxxxxxxxxxx>, "J. Bruce Fields" <bfields@xxxxxxxxxxxx>
> In-reply-to: <20140215232508.GB4508@pd.tnic>
> Organization: PrimaryData Inc
> References: 
> <1392466251.41282.YahooMailNeo@web140003.mail.bf1.yahoo.com> 
> <1392494917.71728.YahooMailNeo@web140002.mail.bf1.yahoo.com>	 
> <20140215203015.GA4528@pd.tnic> 
> <1392498262.98385.YahooMailNeo@web140003.mail.bf1.yahoo.com>	 
> <20140215232508.GB4508@pd.tnic>
> 
> 
> Please ensure that you post to the linux-nfs@xxxxxxxxxxxxxxx when
> reporting NFS and RPC related bugs.
> 
> On Sun, 2014-02-16 at 00:25 +0100, Borislav Petkov wrote:
> > On Sat, Feb 15, 2014 at 01:04:22PM -0800, John wrote:
> > > Thanks for the reply, Boris.  The .config is unmodified
> > > from the Arch Distro default for 3.13.3-1 which can be found
> > > here: http://pastebin.com/LPGZ8ZqA
> >
> > Yep, it is that struct net *net argument to put_pipe_version() which is NULL:
> >
> >   12:   55                      push   %ebp
> >   13:   89 e5                   mov    %esp,%ebp
> >   15:   56                      push   %esi
> >   16:   53                      push   %ebx
> >   17:   3e 8d 74 26 00          lea    %ds:0x0(%esi,%eiz,1),%esi
> >   1c:   8b 1d 28 e9 a3 f8       mov    0xf8a3e928,%ebx
> >   22:   89 c6                   mov    %eax,%esi
> >   24:   e8 59 64 5f c8          call   0xc85f6482
> >   29:   85 db                   test   %ebx,%ebx
> >   2b:*  8b 86 58 08 00 00       mov    0x858(%esi),%eax         <-- trapping instruction
> >
> > put_pipe_version:
> > 	pushl	%ebp	#
> > 	movl	%esp, %ebp	#,
> > 	pushl	%esi	#
> > 	pushl	%ebx	#
> > 	call	mcount
> > 	movl	sunrpc_net_id, %ebx	# sunrpc_net_id, sunrpc_net_id.130
> > 	movl	%eax, %esi	# net, net
> > 	call	__rcu_read_lock	#
> > 	testl	%ebx, %ebx	# sunrpc_net_id.130
> > 	movl	2136(%esi), %eax	# MEM[(struct net_generic * const *)net_4(D) + 2136B], ng <-- trapping insn
> >
> >
> > 	[ 137.689996] ESI: 00000000 EDI: f56efc00 EBP: f568fee8 ESP: f568fee0
> > 			   ^^^^^^^^
> >
> > Here's the c/asm interleaved version:
> >
> > static void put_pipe_version(struct net *net)
> > {
> >      d80:       55                      push   %ebp
> >      d81:       89 e5                   mov    %esp,%ebp
> >      d83:       56                      push   %esi
> >      d84:       53                      push   %ebx
> >      d85:       e8 fc ff ff ff          call   d86 <put_pipe_version+0x6>
> >                         d86: R_386_PC32 mcount
> >         struct sunrpc_net *sn = net_generic(net, sunrpc_net_id);
> >      d8a:       8b 1d 00 00 00 00       mov    0x0,%ebx
> >                         d8c: R_386_32   sunrpc_net_id
> >         spin_unlock(&pipe_version_lock);
> >         return ret;
> > }
> >
> > static void put_pipe_version(struct net *net)
> > {
> >      d90:       89 c6                   mov    %eax,%esi
> >  * block, but only when acquiring spinlocks that are subject to priority
> >  * inheritance.
> >  */
> > static inline void rcu_read_lock(void)
> > {
> >         __rcu_read_lock();
> >      d92:       e8 fc ff ff ff          call   d93 <put_pipe_version+0x13>
> >                         d93: R_386_PC32 __rcu_read_lock
> >         struct net_generic *ng;
> >         void *ptr;
> >
> >         rcu_read_lock();
> >         ng = rcu_dereference(net->gen);
> >         BUG_ON(id == 0 || id > ng->len);
> >      d97:       85 db                   test   %ebx,%ebx
> > {
> >         struct net_generic *ng;
> >         void *ptr;
> >
> >         rcu_read_lock();
> >         ng = rcu_dereference(net->gen);
> >      d99:       8b 86 58 08 00 00       mov    0x858(%esi),%eax			<-- trapping insn
> >
> >
> > I guess you could avoid the crash if you did
> >
> > 	if (!net)
> > 		return;
> >
> > in put_pipe_version() but this hardly is the right solution. Someone
> > else has to make sense of this thing, not me. :-)
> 
> Does the following patch help?

Works for me on 3.14-rc3.
Feel free to add my
Tested-by: Dieter Nützel <Dieter@nuetzel-hh.de>

Thanks,
    Dieter

> 8<-------------------------------------------------------------------
> >From 0e57b109cd7b17d6e6f16c3454427372a583b18a Mon Sep 17 00:00:00 2001
> From: Trond Myklebust <trond.myklebust@xxxxxxxxxxxxxxx>
> Date: Sun, 16 Feb 2014 12:14:13 -0500
> Subject: [PATCH] SUNRPC: Ensure that gss_auth isn't freed before its 
> upcall
> messages
> 
> Fix a race in which the RPC client is shutting down while the
> gss daemon is processing a downcall. If the RPC client manages to
> shut down before the gss daemon is done, then the struct gss_auth
> used in gss_release_msg() may have already been freed.
> 
> Link: 
> http://lkml.kernel.org/r/1392494917.71728.YahooMailNeo@xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Reported-by: John <da_audiophile@xxxxxxxxx>
> Reported-by: Borislav Petkov <bp@xxxxxxxxx>
> Signed-off-by: Trond Myklebust <trond.myklebust@xxxxxxxxxxxxxxx>
> ---
>  net/sunrpc/auth_gss/auth_gss.c | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/net/sunrpc/auth_gss/auth_gss.c 
> b/net/sunrpc/auth_gss/auth_gss.c
> index 44a61e8fda6f..1ba1fd114912 100644
> --- a/net/sunrpc/auth_gss/auth_gss.c
> +++ b/net/sunrpc/auth_gss/auth_gss.c
> @@ -108,6 +108,7 @@ struct gss_auth {
>  static DEFINE_SPINLOCK(pipe_version_lock);
>  static struct rpc_wait_queue pipe_version_rpc_waitqueue;
>  static DECLARE_WAIT_QUEUE_HEAD(pipe_version_waitqueue);
> +static void gss_put_auth(struct gss_auth *gss_auth);
> 
>  static void gss_free_ctx(struct gss_cl_ctx *);
>  static const struct rpc_pipe_ops gss_upcall_ops_v0;
> @@ -320,6 +321,7 @@ gss_release_msg(struct gss_upcall_msg *gss_msg)
>  	if (gss_msg->ctx != NULL)
>  		gss_put_ctx(gss_msg->ctx);
>  	rpc_destroy_wait_queue(&gss_msg->rpc_waitqueue);
> +	gss_put_auth(gss_msg->auth);
>  	kfree(gss_msg);
>  }
> 
> @@ -500,6 +502,7 @@ gss_alloc_msg(struct gss_auth *gss_auth,
>  		if (err)
>  			goto err_free_msg;
>  	};
> +	kref_get(&gss_auth->kref);
>  	return gss_msg;
>  err_free_msg:
>  	kfree(gss_msg);
> @@ -1064,6 +1067,12 @@ gss_free_callback(struct kref *kref)
>  }
> 
>  static void
> +gss_put_auth(struct gss_auth *gss_auth)
> +{
> +	kref_put(&gss_auth->kref, gss_free_callback);
> +}
> +
> +static void
>  gss_destroy(struct rpc_auth *auth)
>  {
>  	struct gss_auth *gss_auth = container_of(auth,
> @@ -1084,7 +1093,7 @@ gss_destroy(struct rpc_auth *auth)
>  	gss_auth->gss_pipe[1] = NULL;
>  	rpcauth_destroy_credcache(auth);
> 
> -	kref_put(&gss_auth->kref, gss_free_callback);
> +	gss_put_auth(gss_auth);
>  }
> 
>  /*
> @@ -1255,7 +1264,7 @@ gss_destroy_nullcred(struct rpc_cred *cred)
>  	call_rcu(&cred->cr_rcu, gss_free_cred_callback);
>  	if (ctx)
>  		gss_put_ctx(ctx);
> -	kref_put(&gss_auth->kref, gss_free_callback);
> +	gss_put_auth(gss_auth);
>  }
> 
>  static void
> --
> 1.8.5.3
> 
> 
> --
> Trond Myklebust
> Linux NFS client maintainer, PrimaryData
> trond.myklebust@xxxxxxxxxxxxxxx
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


^ permalink raw reply	[flat|nested] 7+ messages in thread
* [BUG] unable to handle kernel NULL pointer dereference
@ 2014-02-15 15:19 John
  0 siblings, 0 replies; 7+ messages in thread
From: John @ 2014-02-15 15:19 UTC (permalink / raw)
  To: linux-nfs@vger.kernel.org

When booting into linux v3.13.3, I am unable to mount an nfs share on this particular hardware.  I get the same problem using v3.12.11.  Only the 3.10.x series allows normal operation.  Partial dmesg output shown inline, additional logs available upon request.

PLEASE cc me on my replies as I am not subscribed to lkml.

Hardware: Athlon XP 3200+ on an NVIDIA nForce2 Ultra 400 motherboard.
Distro: Arch Linux i686.

% dmesg
...
[ 137.616014] NFS: Registering the id_resolver key type
[ 137.616036] Key type id_resolver registered
[ 137.616038] Key type id_legacy registered
[ 137.686758] BUG: unable to handle kernel NULL pointer dereference at 00000858
[ 137.689996] IP: [<f8aa2d99>] put_pipe_version+0x19/0x60 [auth_rpcgss]
[ 137.689996] *pde = 00000000 
[ 137.689996] Oops: 0000 [#1] PREEMPT SMP 
[ 137.689996] Modules linked in: rpcsec_gss_krb5 auth_rpcgss oid_registry nfsv4 asb100 hwmon_vid snd_wavefront ir_mce_kbd_decoder ir_lirc_codec ir_rc5_sz_decoder ir_sony_decoder lirc_dev ir_rc5_decoder ir_jvc_decoder ir_sanyo_decoder ir_rc6_decoder ir_nec_decoder rc_streamzap streamzap mousedev snd_cs4236 rc_core snd_intel8x0 snd_wss_lib snd_opl3_lib snd_hwdep snd_ac97_codec evdev snd_mpu401 ac97_bus snd_mpu401_uart snd_pcm snd_rawmidi snd_page_alloc snd_seq_device snd_timer snd pcspkr skge shpchp i2c_nforce2 i2c_core soundcore ns558 gameport processor button nvidia_agp agpgart nfs lockd sunrpc fscache ext4 crc16 mbcache jbd2 hid_generic usbhid hid sr_mod cdrom sd_mod ata_generic pata_acpi sata_sil pata_amd libata ehci_pci ohci_pci ohci_hcd ehci_hcd scsi_mod usbcore usb_common
[ 137.689996] CPU: 0 PID: 534 Comm: rpc.gssd Not tainted 3.13.3-1-ARCH #1
[ 137.689996] Hardware name: ASUSTeK Computer INC. A7N8X-E/A7N8X-E, BIOS ASUS A7N8X-E Deluxe ACPI BIOS Rev 1013 11/12/2004
[ 137.689996] task: f4633210 ti: f568e000 task.ti: f568e000
[ 137.689996] EIP: 0060:[<f8aa2d99>] EFLAGS: 00010202 CPU: 0
[ 137.689996] EIP is at put_pipe_version+0x19/0x60 [auth_rpcgss]
[ 137.689996] EAX: f4633210 EBX: 00000001 ECX: f56efca8 EDX: 00000296
[ 137.689996] ESI: 00000000 EDI: f56efc00 EBP: f568fee8 ESP: f568fee0
[ 137.689996] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 137.689996] CR0: 8005003b CR2: 00000858 CR3: 34523000 CR4: 000007d0
[ 137.689996] Stack:
[ 137.689996] f56efc00 f6c64f78 f568fef4 f8aa2e05 00000010 f568ff40 f8aa3b38 00000374
[ 137.689996] 00000080 b74dde40 f4644a80 f568ff30 00000246 f8ac1080 ffff41c9 f6c64f78
[ 137.689996] fffffff3 00000010 f4460140 f44d5820 f44d5810 f53df7ec f57595a0 f8aa93e8
[ 137.689996] Call Trace:
[ 137.689996] [<f8aa2e05>] gss_release_msg+0x25/0x70 [auth_rpcgss]
[ 137.689996] [<f8aa3b38>] gss_pipe_downcall+0x208/0x4b0 [auth_rpcgss]
[ 137.689996] [<f8a2f9ab>] rpc_pipe_write+0x3b/0x60 [sunrpc]
[ 137.689996] [<f8a2f970>] ? rpc_pipe_poll+0x90/0x90 [sunrpc]
[ 137.689996] [<c1156bd5>] vfs_write+0x95/0x1c0
[ 137.689996] [<c11572a1>] SyS_write+0x51/0x90
[ 137.689996] [<c145cc0d>] sysenter_do_call+0x12/0x28
[ 137.689996] Code: f8 e8 4f b8 9a c8 31 c0 eb c6 90 8d b4 26 00 00 00 00 55 89 e5 56 53 3e 8d 74 26 00 8b 1d 28 e9 a3 f8 89 c6 e8 59 64 5f c8 85 db <8b> 86 58 08 00 00 74 3a 3b 18 77 36 8b 5c 98 08 e8 32 66 5f c8
[ 137.689996] EIP: [<f8aa2d99>] put_pipe_version+0x19/0x60 [auth_rpcgss] SS:ESP 0068:f568fee0
[ 137.689996] CR2: 0000000000000858
[ 138.578433] ---[ end trace 3dcb8d5c35b64fbd ]---
[ 142.979263] type=1006 audit(1392415950.632:4): pid=540 uid=0 old auid=4294967295 new auid=1000 old ses=4294967295 new ses=3 res=1

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2014-02-17 20:42 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <1392466251.41282.YahooMailNeo@web140003.mail.bf1.yahoo.com>
     [not found] ` <1392494917.71728.YahooMailNeo@web140002.mail.bf1.yahoo.com>
     [not found]   ` <20140215203015.GA4528@pd.tnic>
     [not found]     ` <1392498262.98385.YahooMailNeo@web140003.mail.bf1.yahoo.com>
     [not found]       ` <20140215232508.GB4508@pd.tnic>
2014-02-16 17:27         ` [BUG] unable to handle kernel NULL pointer dereference Trond Myklebust
2014-02-16 17:35           ` Borislav Petkov
2014-02-17 20:12           ` John
2014-02-17 20:30             ` Borislav Petkov
2014-02-17 20:35           ` John
2014-02-17  4:46 Dieter Nützel
  -- strict thread matches above, loose matches on Subject: below --
2014-02-15 15:19 John

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox