Linux NFS development
 help / color / mirror / Atom feed
From: Jeff Layton <jlayton@kernel.org>
To: Chuck Lever III <chuck.lever@oracle.com>,
	Jiri Slaby <jirislaby@kernel.org>,
	Steve Dickson <SteveD@redhat.com>
Cc: Chuck Lever <cel@kernel.org>,
	Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
	Neil Brown <neilb@suse.de>,
	Alexander Ahring Oder Aring <aahringo@redhat.com>
Subject: Re: [PATCH v1 26/27] SUNRPC: Set rq_accept_statp inside ->accept methods
Date: Tue, 16 May 2023 15:23:41 -0400	[thread overview]
Message-ID: <569be8e6eff7af373e6baaf2170edb8a8c52f262.camel@kernel.org> (raw)
In-Reply-To: <4AB1ED03-57C8-42C1-9A04-6C224E98EDCC@oracle.com>

[-- Attachment #1: Type: text/plain, Size: 2667 bytes --]

On Tue, 2023-05-02 at 14:14 +0000, Chuck Lever III wrote:
> 
> > On May 2, 2023, at 7:01 AM, Jiri Slaby <jirislaby@kernel.org> wrote:
> > 
> > On 08. 01. 23, 17:31, Chuck Lever wrote:
> > > From: Chuck Lever <chuck.lever@oracle.com>
> > > To navigate around the space that svcauth_gss_accept() reserves
> > > for the RPC payload body length and sequence number fields,
> > > svcauth_gss_release() does a little dance with the reply's
> > > accept_stat, moving the accept_stat value in the response buffer
> > > down by two words.
> > > Instead, let's have the ->accept() methods each set the proper
> > > final location of the accept_stat to avoid having to move
> > > things.
> > 
> > Hi,
> > 
> > I bisected to this (4bcf0343e8)
> 
> Assuming you did the bisect on the NFS server's kernel?
> 
> 
> > as it breaks nfs3-only servers in 6.3. I.e. /etc/nfs.conf containing:
> > [nfsd]
> > vers4=no
> 
> Note: Changing the settings in /etc/nfs.conf had no effect
> on my server, so I effected the change by stopping the
> server and poking values into /proc/fs/nfsd/versions by
> hand.
> 
> Steve?
> 
> 
> > The client sees:
> >  mount("10.0.2.15:/tmp", "/mnt", "nfs", 0, "vers=4.2,addr=10.0.2.15,clientad"...) = -1 EIO (Input/output error)
> >  write(2, "mount.nfs: mount system call fai"..., 45
> >  mount.nfs: mount system call failed for /mnt
> > 
> > And the kernel says:
> >  nfs4_discover_server_trunking unhandled error -5. Exiting with error EIO
> > 
> > I reported in downstream as:
> > https://bugzilla.suse.com/show_bug.cgi?id=1210995
> > 
> > It cannot be reverted cleanly on the top of 6.3.
> > 
> > Any ideas?
> 
> I can reproduce a similar problem. Network capture shows
> that the server is responding with NFS4ERR_NOENT to the
> EXCHANGE_ID operation, and the client kernel log says:
> 
> >  nfs4_discover_server_trunking unhandled error -121. Exiting with error EIO
> 
> That's not the failure mode I expected given the commit
> you bisected to, so it might not be the same problem you've
> hit. I'll troubleshoot this and send a fix for testing.
> 

Alex hit this problem in testing too, and I took a quick look.

In the attached capture, the client should have gotten back a
RPC_PROG_MISMATCH error, but the server has recorded an extra successful
accept state before encoding the RPC_PROG_MISMATCH error, leading to a
malformed reply.

I think that the problem is that encoding the accept status too early
means that we can't properly handle failures from the pg_init_request
call.

Chuck, any thoughts on how you'd like to handle this?
-- 
Jeff Layton <jlayton@kernel.org>

[-- Attachment #2: bad-fallback.pcapng.gz --]
[-- Type: application/x-pcapng, Size: 2000 bytes --]

  parent reply	other threads:[~2023-05-16 19:23 UTC|newest]

Thread overview: 39+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-01-08 16:28 [PATCH v1 00/27] Server-side RPC reply header parsing overhaul Chuck Lever
2023-01-08 16:28 ` [PATCH v1 01/27] SUNRPC: Clean up svcauth_gss_release() Chuck Lever
2023-01-10 14:01   ` Jeff Layton
2023-01-08 16:28 ` [PATCH v1 02/27] SUNRPC: Rename automatic variables in svcauth_gss_wrap_resp_integ() Chuck Lever
2023-01-08 16:28 ` [PATCH v1 03/27] SUNRPC: Record gss_get_mic() errors in svcauth_gss_wrap_integ() Chuck Lever
2023-01-08 16:28 ` [PATCH v1 04/27] SUNRPC: Replace checksum construction " Chuck Lever
2023-01-08 16:28 ` [PATCH v1 05/27] SUNRPC: Convert svcauth_gss_wrap_integ() to use xdr_stream() Chuck Lever
2023-01-08 16:29 ` [PATCH v1 06/27] SUNRPC: Rename automatic variables in svcauth_gss_wrap_resp_priv() Chuck Lever
2023-01-08 16:29 ` [PATCH v1 07/27] SUNRPC: Record gss_wrap() errors in svcauth_gss_wrap_priv() Chuck Lever
2023-01-08 16:29 ` [PATCH v1 08/27] SUNRPC: Add @head and @tail variables " Chuck Lever
2023-01-08 16:29 ` [PATCH v1 09/27] SUNRPC: Convert svcauth_gss_wrap_priv() to use xdr_stream() Chuck Lever
2023-01-08 16:29 ` [PATCH v1 10/27] SUNRPC: Check rq_auth_stat when preparing to wrap a response Chuck Lever
2023-01-08 16:29 ` [PATCH v1 11/27] SUNRPC: Remove the rpc_stat variable in svc_process_common() Chuck Lever
2023-01-08 16:29 ` [PATCH v1 12/27] SUNRPC: Add XDR encoding helper for opaque_auth Chuck Lever
2023-01-08 16:29 ` [PATCH v1 13/27] SUNRPC: Push svcxdr_init_encode() into svc_process_common() Chuck Lever
2023-01-08 16:29 ` [PATCH v1 14/27] SUNRPC: Move svcxdr_init_encode() into ->accept methods Chuck Lever
2023-01-08 16:29 ` [PATCH v1 15/27] SUNRPC: Use xdr_stream to encode Reply verifier in svcauth_null_accept() Chuck Lever
2023-01-08 16:30 ` [PATCH v1 16/27] SUNRPC: Use xdr_stream to encode Reply verifier in svcauth_unix_accept() Chuck Lever
2023-01-08 16:30 ` [PATCH v1 17/27] SUNRPC: Use xdr_stream to encode Reply verifier in svcauth_tls_accept() Chuck Lever
2023-01-08 16:30 ` [PATCH v1 18/27] SUNRPC: Convert unwrap data paths to use xdr_stream for replies Chuck Lever
2023-01-08 16:30 ` [PATCH v1 19/27] SUNRPC: Use xdr_stream to encode replies in server-side GSS upcall helpers Chuck Lever
2023-01-08 16:30 ` [PATCH v1 20/27] SUNRPC: Use xdr_stream for encoding GSS reply verifiers Chuck Lever
2023-01-08 16:30 ` [PATCH v1 21/27] SUNRPC: Hoist init_encode out of svc_authenticate() Chuck Lever
2023-01-08 16:30 ` [PATCH v1 22/27] SUNRPC: Convert RPC Reply header encoding to use xdr_stream Chuck Lever
2023-01-08 16:30 ` [PATCH v1 23/27] SUNRPC: Final clean-up of svc_process_common() Chuck Lever
2023-01-08 16:30 ` [PATCH v1 24/27] SUNRPC: Remove no-longer-used helper functions Chuck Lever
2023-01-08 16:30 ` [PATCH v1 25/27] SUNRPC: Refactor RPC server dispatch method Chuck Lever
2023-01-08 16:31 ` [PATCH v1 26/27] SUNRPC: Set rq_accept_statp inside ->accept methods Chuck Lever
2023-05-02 11:01   ` Jiri Slaby
2023-05-02 14:14     ` Chuck Lever III
2023-05-02 21:29       ` NeilBrown
2023-05-16 19:23       ` Jeff Layton [this message]
2023-05-16 19:25         ` Chuck Lever III
2023-05-16 21:25           ` Jeff Layton
2023-05-16 21:27             ` Chuck Lever III
2023-05-16 22:28               ` Jeff Layton
2023-01-08 16:31 ` [PATCH v1 27/27] SUNRPC: Go back to using gsd->body_start Chuck Lever
2023-01-10 14:53 ` [PATCH v1 00/27] Server-side RPC reply header parsing overhaul Jeff Layton
2023-01-10 15:16   ` Chuck Lever III

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=569be8e6eff7af373e6baaf2170edb8a8c52f262.camel@kernel.org \
    --to=jlayton@kernel.org \
    --cc=SteveD@redhat.com \
    --cc=aahringo@redhat.com \
    --cc=cel@kernel.org \
    --cc=chuck.lever@oracle.com \
    --cc=jirislaby@kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=neilb@suse.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox