From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9180528C5AA for ; Thu, 13 Nov 2025 22:20:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.52 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763072422; cv=none; b=NQuJYaQSEiNNfFzbrEPV68Fn/kvZAqj3f2Yt4ILXRKz1XHtvXs77VKB8z4tsDHG+qe5Mtc4z3p/draWJTOzEnZbp70hxTL4GpkEd7FZUtCOZce7N306VBKXo1IlyQYAgOxyUXuCPCqdRyRZ/DRpczqm1mvd5gk34KhgGs1+kuHg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763072422; c=relaxed/simple; bh=q3qpMo01f8XaBD3HWLZ1JfzQDmrbrutVRvh9hYqKids=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=NlZAXDjsO/98iXNI9M0ij2RPstKDRka0D51+zRC5pfM0PWEiUW+pbhSe4JAMJis+2FscIfaLD+ILO6aXzl6R44Jui5lNJHaSVOv6XdAH2kHGJxChf25Ge1s7FL48T7pnDKqrPFMhlW7DUWMkJgsdBiUISVxed+JIioie5O3cNwk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=cJ0Bn01F; arc=none smtp.client-ip=209.85.221.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cJ0Bn01F" Received: by mail-wr1-f52.google.com with SMTP id ffacd0b85a97d-42b38de7940so807057f8f.3 for ; Thu, 13 Nov 2025 14:20:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1763072419; x=1763677219; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:from:to:cc:subject:date:message-id :reply-to; bh=mxD6Va4nlrGdLx84AnZel4scwr9uai3z0FsuYN7KzXI=; b=cJ0Bn01FsO1/ybg+6F92/lu3Z5xKYHlZFL0ZZEGlfH8ns/bTCmRHOLetiD7t1fuXUZ x4KJfm2WwJIprceMuDGent6XFLgvNjxWZ0aovsuW+HPshf/xCGKy4hzj26nqJjSdxZlH S0RNgY8fVVdNqq3CQaNV+a4nkTb7JakPELFtgJvehKd0PFV9HvJt+AMUlhTCs/dLI4Gf RagvSwIHUlobTDN6B4YYXOM5ttf8XFtBGFI6rnHgizmd4ZnGmT9y+Nyff0W9khgHEcEk OSy3VWBjzLlPmbVl9hNV+xrrs4OsuGxQpy4QbCMqg8GzvWIdOWEXyV8p2xKB4bflGb5a ZguQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763072419; x=1763677219; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=mxD6Va4nlrGdLx84AnZel4scwr9uai3z0FsuYN7KzXI=; b=wlQj6EqyeyYpeFuBzvnlNj5mkx8TlsucTvpprUbwx5cSfZU314434KWwEkCCIHhtPP xlG4ngCKkwp9jWmwc+/QvnF4AfkamOOmZqAUU4ALaH4I7+23g4PWa66Ai5Tiz25JDWbZ J8lMEZzvPl5/TqqgelRl3dp3ITPZwoyoNU0R4FbM9BGTuQE6hYFRSpYs2i5Je00jBKZ7 qRtS6a/KSx0LF2UKP2YNqPkkMjr2ttWJoYHWC+4XppvzunFoxGnMbJSUr6beNt0MLLTC 08KH0V1FOUBb86bHfHPHxqm7yvHG2pNkEQe2EuadwMajrFPOA/feoYK8Q6mH/pmvuvzc +4jw== X-Forwarded-Encrypted: i=1; AJvYcCXoqqLT6T4D0hD8c8cBbSf8MpY9LdBZOH3A9kdi6HreTjIm7v2/ti11ih8Zg1iRM8Io4E93HR86/7c9PUc=@vger.kernel.org X-Gm-Message-State: AOJu0YzykHTas7/PZidj33URaiSRgM4uhvujERIZuHfIC5izDz6du2Dh 6ano+f5fYDHI4bpaWW+c5l0ww5j/lwGzpWB4BCv2I5BGjdZGrYbviqeH X-Gm-Gg: ASbGnctyCxtjlina0NE3UxDXKorVz+fRQNb2GNKdHJEo7sjVpF4fI9bjFDOQPvGFaHs jZ2k8vmzSNTHadDXvO6N9NyyPQjBfAzW+cDWPxOX2PX+HvTvkiectC2dYvkzuqMuu+kiYwlRTNZ AAurDVb7Ev5SNFFHCIyPqU7O0mNG1Dlr+e0W6fFifiuuqvCtTlfUTOGzxczu0UymjqxrTFLeE4n J8npiU2iXqizMwyvngGp1CBbSvh6lc4n+ehov39wRXg8gYZA2M+AL5WN8MGa3zNYEyK8dDJWmJt 1lM+zn45jYAdSYV//wN2yZoaHD69hQWIpqT4ZZL7gLxYJtnwvCoLnAZr9Bc4YeKNTu9gFT9BWCB KKWmS5R5uWWzHlyWX+CoAftDI2v4d1EnpiZ8gkFG/6k0qOiSkD7B2NSSH1p85I8umILrS7cjFWb LNLKib9TLp90eiV+q9LKWsuacBvH0clOxk6w== X-Google-Smtp-Source: AGHT+IEZr/jgTUzIzh92CEqHXY9JBpXDSThmF8hDmL5x+X3hWg9F2OP0W6O2TCfdZmtA3yVNYmboqg== X-Received: by 2002:a05:6000:2f83:b0:429:d3e9:694 with SMTP id ffacd0b85a97d-42b5932360dmr897283f8f.5.1763072418673; Thu, 13 Nov 2025 14:20:18 -0800 (PST) Received: from eldamar.lan (c-82-192-244-13.customer.ggaweb.ch. [82.192.244.13]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-42b53e982d6sm6400196f8f.21.2025.11.13.14.20.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Nov 2025 14:20:17 -0800 (PST) Sender: Salvatore Bonaccorso Received: by eldamar.lan (Postfix, from userid 1000) id D2AD4BE2EE7; Thu, 13 Nov 2025 23:20:16 +0100 (CET) Date: Thu, 13 Nov 2025 23:20:16 +0100 From: Salvatore Bonaccorso To: Chuck Lever Cc: "Tyler W. Ross" , "1120598@bugs.debian.org" <1120598@bugs.debian.org>, Jeff Layton , NeilBrown , Scott Mayhew , Steve Dickson , Olga Kornievskaia , Dai Ngo , Tom Talpey , Trond Myklebust , Anna Schumaker , linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: ls input/output error ("NFS: readdir(/) returns -5") on krb5 NFSv4 client using SHA2 Message-ID: References: <176298368872.955.14091113173156448257.reportbug@nfsclient-sid.ipa.twrlab.net> <8d873978-2df6-4b79-891d-f0fd78485551@oracle.com> <1cee1c3e-e6b9-485a-a4d4-c336072f14c3@oracle.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1cee1c3e-e6b9-485a-a4d4-c336072f14c3@oracle.com> Hi Chuck, On Thu, Nov 13, 2025 at 04:23:52PM -0500, Chuck Lever wrote: > On 11/13/25 4:21 PM, Salvatore Bonaccorso wrote: > > Hi Chuck, > > > > On Thu, Nov 13, 2025 at 12:47:23PM -0500, Chuck Lever wrote: > >> On 11/13/25 12:16 PM, Tyler W. Ross wrote: > >>> Thanks, Chunk. > >>> > >>> Suggested trace-cmd report from the client follows. Last 3 lines appear salient, but I've included the full report just in case. > >>> > >>> -0 [001] ..s2. 270.327040: xs_data_ready: peer=[10.108.2.102]:2049 > >>> kworker/u16:0-12 [001] ...1. 270.327048: xprt_lookup_rqst: peer=[10.108.2.102]:2049 xid=0x7b569c7a status=0 > >>> kworker/u16:0-12 [001] ...2. 270.327050: rpc_task_wakeup: task:00000008@00000005 flags=MOVEABLE|DYNAMIC|SENT|NORTO|CRED_NOREF runstate=0x6 status=0 timeout=15000 queue=xprt_pending > >>> kworker/u16:0-12 [001] ..... 270.327054: xs_stream_read_request: peer=[10.108.2.102]:2049 xid=0x7b569c7a copied=988 reclen=988 offset=988 > >>> kworker/u16:0-12 [001] ..... 270.327055: xs_stream_read_data: peer=[10.108.2.102]:2049 err=-11 total=992 > >>> ls-969 [003] ..... 270.327062: rpc_task_sync_wake: task:00000008@00000005 flags=MOVEABLE|DYNAMIC|SENT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=call_status > >>> ls-969 [003] ..... 270.327062: rpc_task_run_action: task:00000008@00000005 flags=MOVEABLE|DYNAMIC|SENT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=xprt_timer > >>> ls-969 [003] ..... 270.327063: rpc_task_run_action: task:00000008@00000005 flags=MOVEABLE|DYNAMIC|SENT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=call_status > >>> ls-969 [003] ..... 270.327063: rpc_task_run_action: task:00000008@00000005 flags=MOVEABLE|DYNAMIC|SENT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=call_decode > >>> ls-969 [003] ..... 270.327063: rpc_xdr_recvfrom: task:00000008@00000005 head=[0xffff8895c29fef64,140] page=4008(88) tail=[0xffff8895c29feff0,36] len=988 > >>> ls-969 [003] ..... 270.327067: rpc_xdr_overflow: task:00000008@00000005 nfsv4 READDIR requested=8 p=0xffff8895c29fefec end=0xffff8895c29feff0 xdr=[0xffff8895c29fef64,140]/4008/[0xffff8895c29feff0,36]/988 > >> > >> Here's the problem. This is a sign of an XDR decoding issue. If you > >> capture the traffic with Wireshark, does Wireshark indicate where the > >> XDR is malformed? > >> > >> If it doesn't, then there is some problem with the client code. Since > >> Fedora 43 is working as expected, I would guess there's a misapplied > >> patch on Debian 13's kernel...? > > > > if it is helpful: Debian follows the stable upstream releases (6.12.y > > for trixie/Debian 13, right now 6.17.y for Debian unstable) and we try > > to keep the patches limited which we apply on top. So far I see none > > which touches net/sunrpc/. The patches applied: > > https://salsa.debian.org/kernel-team/linux/-/tree/debian/6.17/forky/debian/patches?ref_type=heads > > (in case this could help narrowing down more the issue). > > > > But we could try here additionally, if Tylor has the possibility to do > > so, to try directly the 6.17.7 upstream version without Debian patches > > applied. > > A bisect between broken v6.12.y and working v6.17.7 could identify > what is possibly missing from v6.12.y. There seems to be a missundestanding? 6.17.7 as present in Debian unstable is neither working, at least Tyler said: > 2. Freshly installed Debian sid via mini ISO (2025-11-01). Same > configuration as 1/above. which includes a 6.17.y based kernel (6.17.7-1). Regards, Salvatore