Re: NFS client to pNFS DS

["Mkrtchyan, Tigran" <tigran.mkrtchyan@desy.de>]

> 
> On 10. Aug 2024, at 23:20, Olga Kornievskaia <aglo@umich.edu> wrote:
> 
> On Fri, Aug 9, 2024 at 12:27 PM marc eshel <eshel.marc@gmail.com> wrote:
> >
> >
> > On 8/9/24 8:15 AM, Olga Kornievskaia wrote:
> > > On Fri, Aug 9, 2024 at 10:09 AM marc eshel <eshel.marc@gmail.com> wrote:
> > >> Thanks for the replies, I am a little rusty with debugging NFS but this what I see when the NFS client tried to create a session with the DS.
> > >>
> > >> Ganesha was configured for sec=sys and the client mount had the option sec=sys, I assume flavor 390004 means it was trying to use krb5i.
> > > For 4.1, the client will always try to do state operations with krb5i
> > > even when sec=sys when the client detects that it's configured to do
> > > Kerberos (ie., gssd is running). This context creation is triggered
> > > regardless of whether the rpc client is used for MDS or DS.
> > >
> > > My question to you: is the MDS configured with Kerberos but the DS
> > > isn't? And also, does this lead to a failure?
> > Both MDS DS are configured for sec=sys and it is leading to client
> > switching from DS to MDS so yes, it is pNFS failure. What I see on the
> > DS is the client creating a session and than imminently destroying it
> > before committing it. If the is something else that I can debug I will
> > be happy to.
> 
> pnfs failure is unexpected. I'm pretty confident that a non-kerberos
> configured client can do normal pnfs with sec=sys. I can help debug,

Yes, I can confirm that. All RHEL kernels and weekly -rc kernels from Linus works as expected. We mount always with sec=sys, and despite the fact, that hosts configured to support kerberos due to AFS and GPFS, the access to DSes is never an issue and never tries to use kerberos.

Tigran.

> if you want to send me a network trace and tracepoint output. Btw what
> kernel/distro are you using?
> 
> >
> > Marc.
> >
> > >
> > >> Jul 30 11:10:58 svl-marcrh-node-1 kernel: RPC: Couldn't create
> > >> auth handle (flavor 390004)
> > >>
> > >> Marc.
> > >>
> > >> On 8/9/24 6:06 AM, Anna Schumaker wrote:
> > >>
> > >> On Thu, Aug 8, 2024 at 6:07 PM Olga Kornievskaia <aglo@umich.edu> wrote:
> > >>
> > >> On Mon, Aug 5, 2024 at 5:51 PM marc eshel <eshel.marc@gmail.com> wrote:
> > >>
> > >> Hi Trond,
> > >>
> > >> Will the Linux NFS client try to us krb5i regardless of the MDS
> > >> configuration?
> > >>
> > >> Is there any option to avoid it?
> > >>
> > >> I was under the impression the linux client has no way of choosing a
> > >> different auth_gss security flavor for the DS than the MDS. Meaning
> > >>
> > >> That's a good point, I completely missed that this is specifically for the DS.
> > >>
> > >> that if mount command has say sec=krb5i then both MDS and DS
> > >> connections have to do krb5i and if say the DS isn't configured for
> > >> Kerberos, then IO would fallback to MDS. I no longer have a pnfs
> > >>
> > >> That's what I would expect, too.
> > >>
> > >> server to verify whether or not what I say is true but that is what my
> > >> memory tells me is the case.
> > >>
> > >>
> > >> Thanks, Marc.
> > >>
> > >> ul 30 11:10:58 svl-marcrh-node-1 kernel: nfs4_fl_alloc_deviceid_node
> > >> stripe count 1
> > >> Jul 30 11:10:58 svl-marcrh-node-1 kernel: nfs4_fl_alloc_deviceid_node
> > >> ds_num 1
> > >> Jul 30 11:10:58 svl-marcrh-node-1 kernel: RPC: Couldn't create
> > >> auth handle (flavor 390004)
> > >>
> > >>
> >
>