Re: Kernel OPS when using xattr

["Mkrtchyan, Tigran" <tigran.mkrtchyan@desy.de>]

Hi Trond,

unfortunately the same result. Here the output of gdb, if it helps.

(gdb) list *0x00000000000252be
0x252be is in _shift_data_right_pages (net/sunrpc/xdr.c:344).
339			if (*pgto != *pgfrom) {
340				vfrom = kmap_atomic(*pgfrom);
341				memcpy(vto + pgto_base, vfrom + pgfrom_base, copy);
342				kunmap_atomic(vfrom);
343			} else
344				memmove(vto + pgto_base, vto + pgfrom_base, copy);
345			flush_dcache_page(*pgto);
346			kunmap_atomic(vto);
347
348		} while ((len -= copy) != 0);
(gdb)


Tigran.


----- Original Message -----
> From: "trondmy" <trondmy@hammerspace.com>
> To: "Tigran Mkrtchyan" <tigran.mkrtchyan@desy.de>, "linux-nfs" <linux-nfs@vger.kernel.org>
> Cc: "Frank van der Linden" <fllinden@amazon.com>
> Sent: Thursday, 3 December, 2020 16:07:53
> Subject: Re: Kernel OPS when using xattr

> Hi Tigran,
> 
> On Thu, 2020-12-03 at 09:20 +0100, Mkrtchyan, Tigran wrote:
>> 
>> Dear NFS folk,
>> 
>> this is I got while accessing xattrs over NFS with 5.10.0-rc6 kernel
>> from Trond's testing tree (8102e956f22e710eecb3913cdd236282213812cf).
>> The 5.9 kernel works as expected.
>> 
>> Tigran.
>> 
>> [ 2803.765467] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver
>> Registering...
>> [59837.811426] general protection fault, probably for non-canonical
>> address 0x5088000000ffc: 0000 [#1] SMP PTI
>> [59837.811433] CPU: 3 PID: 3858 Comm: attr Not tainted 5.10.0-rc6+
>> #60
>> [59837.811435] Hardware name: Dell Inc. Latitude E6520/0J4TFW, BIOS
>> A06 07/11/2011
>> [59837.811442] RIP: 0010:__memmove+0xe2/0x1a0
>> [59837.811445] Code: 1f 84 00 00 00 00 00 90 48 83 fa 20 72 50 48 81
>> fa a8 02 00 00 72 05 40 38 fe 74 bc 48 01 d6 48 01 d7 48 83 ea 20 48
>> 83 ea 20 <4c> 8b 5e f8 4c 8b 56 f0 4c 8b 4e e8 4c 8b 46 e0 48 8d 76
>> e0 4c 89
>> [59837.811447] RSP: 0018:ffffc90002b4f870 EFLAGS: 00010202
>> [59837.811451] RAX: 0005088000000004 RBX: 0000000000000000 RCX:
>> 000000000000fffc
>> [59837.811453] RDX: 0000000000000fbc RSI: 0005088000000ffc RDI:
>> 0005088000001000
>> [59837.811455] RBP: ffff88810be80000 R08: 0005088000000ffc R09:
>> ffff888235aec550
>> [59837.811457] R10: 0000000000000356 R11: 000000000000016c R12:
>> 0000000000000004
>> [59837.811459] R13: 000000000000fffc R14: ffff88810be80000 R15:
>> ffffc90002b4fc58
>> [59837.811462] FS:  00007fd9303ff740(0000) GS:ffff888235ac0000(0000)
>> knlGS:0000000000000000
>> [59837.811465] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [59837.811467] CR2: 00007fd9305d2000 CR3: 000000011fc10005 CR4:
>> 00000000000606e0
>> [59837.811469] Call Trace:
>> [59837.811505]  _shift_data_right_pages+0x11e/0x150 [sunrpc]
>> [59837.811531]  xdr_shrink_bufhead+0x151/0x170 [sunrpc]
>> [59837.811555]  xdr_realign_pages+0x4c/0xa0 [sunrpc]
>> [59837.811578]  xdr_align_pages+0x49/0x120 [sunrpc]
>> [59837.811601]  xdr_read_pages+0x23/0xb0 [sunrpc]
>> [59837.811626]  nfs4_xdr_dec_getxattr+0xfa/0x120 [nfsv4]
>> [59837.811643]  call_decode+0x199/0x1f0 [sunrpc]
>> [59837.811659]  ? rpc_decode_header+0x4e0/0x4e0 [sunrpc]
>> [59837.811678]  __rpc_execute+0x71/0x420 [sunrpc]
>> [59837.811700]  ? xprt_iter_default_rewind+0x10/0x10 [sunrpc]
>> [59837.811721]  ? xprt_iter_get_next+0x4a/0x60 [sunrpc]
>> [59837.811737]  rpc_run_task+0x14c/0x180 [sunrpc]
>> [59837.811756]  nfs4_do_call_sync+0x6e/0xb0 [nfsv4]
>> [59837.811785]  _nfs42_proc_getxattr+0xb7/0x170 [nfsv4]
>> [59837.811809]  ? xprt_iter_get_next+0x4a/0x60 [sunrpc]
>> [59837.811830]  nfs42_proc_getxattr+0x86/0xb0 [nfsv4]
>> [59837.811844]  nfs4_xattr_get_nfs4_user+0xc9/0xe0 [nfsv4]
>> [59837.811849]  vfs_getxattr+0x161/0x1a0
>> [59837.811852]  getxattr+0x14f/0x230
>> [59837.811856]  ? filename_lookup+0x123/0x1b0
>> [59837.811861]  ? _cond_resched+0x16/0x40
>> [59837.811864]  ? kmem_cache_alloc+0x3c4/0x4b0
>> [59837.811866]  ? getname_flags.part.0+0x45/0x1a0
>> [59837.811869]  path_getxattr+0x62/0xb0
>> [59837.811873]  do_syscall_64+0x33/0x40
>> [59837.811876]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>> [59837.811879] RIP: 0033:0x7fd93050162e
>> [59837.811883] Code: 48 8b 0d 4d 48 0c 00 f7 d8 64 89 01 48 83 c8 ff
>> c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 c0 00 00
>> 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1a 48 0c 00 f7 d8 64
>> 89 01 48
>> [59837.811884] RSP: 002b:00007ffe178ad1c8 EFLAGS: 00000202 ORIG_RAX:
>> 00000000000000c0
>> [59837.811887] RAX: ffffffffffffffda RBX: 00007ffe178ad330 RCX:
>> 00007fd93050162e
>> [59837.811889] RDX: 0000000000000000 RSI: 00007ffe178ad330 RDI:
>> 00007ffe178bf525
>> [59837.811890] RBP: 0000000000000000 R08: 00007ffe178ad210 R09:
>> 0000000000000000
>> [59837.811892] R10: 0000000000000000 R11: 0000000000000202 R12:
>> 0000000000000001
>> [59837.811894] R13: 00007ffe178ad210 R14: 00007ffe178bf525 R15:
>> 0000000000000000
>> [59837.811896] Modules linked in: nfs_layout_flexfiles
>> rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace
>> nfs_ssc fscache nf_conntrack_netbios_ns nf_conntrack_broadcast nft_ct
>> nf_tables ebtable_nat ebtable_broute ip6table_nat ip6table_mangle
>> ip6table_raw ip6table_security iptable_nat nf_nat nf_conntrack
>> nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_raw
>> iptable_security ip_set nfnetlink ebtable_filter ebtables
>> ip6table_filter ip6_tables sunrpc snd_hda_codec_idt
>> snd_hda_codec_generic ledtrig_audio nouveau i915 iwldvm mac80211
>> btrfs snd_hda_codec_hdmi snd_hda_intel snd_intel_ds
> 
> Does this fix it?
> 
> 8<---------------------------------------------
> From 013ee77c43a4dcd468becaf2f234624433cc6fb2 Mon Sep 17 00:00:00 2001
> From: Trond Myklebust <trond.myklebust@hammerspace.com>
> Date: Thu, 3 Dec 2020 09:16:15 -0500
> Subject: [PATCH] SUNRPC: xs_alloc_sparse_pages() should set buf-
>>page_len
> 
> If the page buffer allocated in xs_alloc_sparse_pages() ends up being
> shorter than the predicted buf->page_len, then we should truncate the
> latter so that later calls to xdr_read_pages() doesn't get confused and
> trigger an Oops.
> 
> Reported-by: "Mkrtchyan, Tigran" <tigran.mkrtchyan@desy.de>
> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
> ---
> net/sunrpc/xprtsock.c | 21 ++++++++++++++-------
> 1 file changed, 14 insertions(+), 7 deletions(-)
> 
> diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
> index c93ff70da3f9..32054176786e 100644
> --- a/net/sunrpc/xprtsock.c
> +++ b/net/sunrpc/xprtsock.c
> @@ -327,21 +327,28 @@ static void xs_free_peer_addresses(struct
> rpc_xprt *xprt)
> static size_t
> xs_alloc_sparse_pages(struct xdr_buf *buf, size_t want, gfp_t gfp)
> {
> -	size_t i,n;
> +	size_t i,n, len;
> 
> -	if (!want || !(buf->flags & XDRBUF_SPARSE_PAGES))
> +	if (!(buf->flags & XDRBUF_SPARSE_PAGES))
> 		return want;
> 	n = (buf->page_base + want + PAGE_SIZE - 1) >> PAGE_SHIFT;
> 	for (i = 0; i < n; i++) {
> 		if (buf->pages[i])
> 			continue;
> 		buf->bvec[i].bv_page = buf->pages[i] =
> alloc_page(gfp);
> -		if (!buf->pages[i]) {
> -			i *= PAGE_SIZE;
> -			return i > buf->page_base ? i - buf->page_base
> : 0;
> -		}
> +		if (!buf->pages[i])
> +			break;
> +	}
> +	len = i << PAGE_SHIFT;
> +	if (len > buf->page_base)
> +		len -= buf->page_base;
> +	else
> +		len = 0;
> +	if (buf->page_len > len) {
> +		buf->buflen -= buf->page_len - len;
> +		buf->page_len = len;
> 	}
> -	return want;
> +	return want <= len ? want : len;
> }
> 
> static ssize_t
> --
> 2.28.0
> 
> 
> --
> Trond Myklebust
> Linux NFS client maintainer, Hammerspace
> trond.myklebust@hammerspace.com