lockd GRANT_MSG RPC callback

lockd GRANT_MSG RPC callback

[Marc Eshel]

Hi,
My server sends the async GRANT_MSG RPC to the client for a blocked lock
but the client either ignoring it or not receiving it. The client wait for
30 seconds and than tries to get the locks again. Can someone explain what
might cause this problem ?
Thanks, Marc.




-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd GRANT_MSG RPC callback

[Trond Myklebust]

P=E5 on , 08/09/2004 klokka 13:33, skreiv Marc Eshel:
>=20
>=20
> Hi,
> My server sends the async GRANT_MSG RPC to the client for a blocked lock
> but the client either ignoring it or not receiving it. The client wait fo=
r
> 30 seconds and than tries to get the locks again. Can someone explain wha=
t
> might cause this problem ?
> Thanks, Marc.

Yes, but not without knowing which client you are using. ;-)

Cheers,
  Trond



-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd GRANT_MSG RPC callback

[Marc Eshel]

Sorry, the client is running on RedHat Linux 7.3 with a 2.6.7 kernel.
Marc,

> P=E5 on , 08/09/2004 klokka 13:33, skreiv Marc Eshel:
> >
> >
> > Hi,
> > My server sends the async GRANT_MSG RPC to the client for a blocked=

lock
> > but the client either ignoring it or not receiving it. The client w=
ait
for
> > 30 seconds and than tries to get the locks again. Can someone expla=
in
what
> > might cause this problem ?
> > Thanks, Marc.

> Yes, but not without knowing which client you are using. ;-)

> Cheers,
> Trond
>
> -------------------------------------------------------
> This SF.Net email is sponsored by BEA Weblogic Workshop
> FREE Java Enterprise J2EE developer tools!
> Get your free copy of BEA WebLogic Workshop 8.1 today.
> http://ads.osdn.com/?ad_idP47&alloc_id=10808&oplick
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs=




-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd GRANT_MSG RPC callback

[Trond Myklebust]

P=E5 on , 08/09/2004 klokka 15:49, skreiv Marc Eshel:
>=20
>=20
> Sorry, the client is running on RedHat Linux 7.3 with a 2.6.7 kernel.
> Marc,
>=20
> > P=E5 on , 08/09/2004 klokka 13:33, skreiv Marc Eshel:
> > >
> > >
> > > Hi,
> > > My server sends the async GRANT_MSG RPC to the client for a blocked
> lock

So if you are certain that the GRANT_MSG call is being sent by the
server (2.6.x kernels prior to 2.6.9-rc1 have a bug that prevents this),
then my first suspect is the new code in net/sunrpc/svcauth_unix.c,
which assumes that all server traffic needs to be authenticated as if it
were rpc.nfsd traffic.

If so, then a tcpdump should show you that the lockd process on the
client is returning an AUTH_BADCRED reply or something like that. Could
you check please?

Cheers,
  Trond



-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd GRANT_MSG RPC callback

[Marc Eshel]

Trond Myklebust <trond.myklebust@fys.uio.no> wrote on 09/08/2004 01:12:=
01
PM:

> P=E5 on , 08/09/2004 klokka 15:49, skreiv Marc Eshel:
> >
> >
> > Sorry, the client is running on RedHat Linux 7.3 with a 2.6.7 kerne=
l.
> > Marc,
> >
> > > P=E5 on , 08/09/2004 klokka 13:33, skreiv Marc Eshel:
> > > >
> > > >
> > > > Hi,
> > > > My server sends the async GRANT_MSG RPC to the client for a blo=
cked
> > lock

> So if you are certain that the GRANT_MSG call is being sent by the
> server (2.6.x kernels prior to 2.6.9-rc1 have a bug that prevents thi=
s),
> then my first suspect is the new code in net/sunrpc/svcauth_unix.c,
> which assumes that all server traffic needs to be authenticated as if=
 it
> were rpc.nfsd traffic.

> If so, then a tcpdump should show you that the lockd process on the
> client is returning an AUTH_BADCRED reply or something like that. Cou=
ld
> you check please?

Yes, I see the following messages on the client:

svc: authenticate (1)
svc: authentication failed (1)

Marc.

=




-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd GRANT_MSG RPC callback

[Trond Myklebust]

P=E5 on , 08/09/2004 klokka 16:35, skreiv Marc Eshel:
> Yes, I see the following messages on the client:
>=20
> svc: authenticate (1)
> svc: authentication failed (1)

OK. Bruce has promised to work on splitting out the "domain" code from
the svcauth_unix code. Once that is done, we can fix up GRANT_MSG and
NOTIFY as well as any other routines that do not need to be
authenticated by rpc.mountd.

Cheers,
  Trond



-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd GRANT_MSG RPC callback

[Greg Banks]

On Thu, 2004-09-09 at 06:50, Trond Myklebust wrote:
> P=E5 on , 08/09/2004 klokka 16:35, skreiv Marc Eshel:
> > Yes, I see the following messages on the client:
> >=20
> > svc: authenticate (1)
> > svc: authentication failed (1)
>=20

By sheer coincidence, I've had what appears to be the same problem
reported internally in the last couple of days.  I have a network
trace (partially truncated by tcpdump, sigh) which shows a GRANTED
callback from an IRIX server being rejected by a 2.6.5 (SLES9) client
with AUTH_BADCRED.  The same case works fine in 2.4 based kernels.
My suspicions were hovering around the same code as your suspicions ;-)

> OK. Bruce has promised to work on splitting out the "domain" code from
> the svcauth_unix code. Once that is done, we can fix up GRANT_MSG and
> NOTIFY as well as any other routines that do not need to be
> authenticated by rpc.mountd.

Any idea how long is this likely to take?

Greg.
--=20
Greg Banks, R&D Software Engineer, SGI Australian Software Group.
I don't speak for SGI.




-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd GRANT_MSG RPC callback

[J. Bruce Fields]

On Thu, Sep 09, 2004 at 01:08:36PM +1000, Greg Banks wrote:
> On Thu, 2004-09-09 at 06:50, Trond Myklebust wrote:
> > OK. Bruce has promised to work on splitting out the "domain" code from
> > the svcauth_unix code. Once that is done, we can fix up GRANT_MSG and
> > NOTIFY as well as any other routines that do not need to be
> > authenticated by rpc.mountd.
> 
> Any idea how long is this likely to take?

I'll try to have something out by early next week.

The problem is that the server-side rpc code is checking *all* incoming
auth_unix and auth_null requests (whether for nfsd or for some other
service) against the exports table.  So as a temporary workaround, lockd
will work if the client lists the server in the exports file.  Or
something like

echo nfsd server.ip.address.here 0x7FFFFFFF server.name.here \
	>/proc/net/rpc/auth.unix.ip/content

may also work, though the entry you add to the auth.unix.ip cache by
doing this may end up getting purged next time you run exportfs.

--Bruce Fields


-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd GRANT_MSG RPC callback

[Marc Eshel]

"J. Bruce Fields" <bfields@fieldses.org> wrote on 09/09/2004 05:56:25 AM:

> On Thu, Sep 09, 2004 at 01:08:36PM +1000, Greg Banks wrote:
> > On Thu, 2004-09-09 at 06:50, Trond Myklebust wrote:
> > > OK. Bruce has promised to work on splitting out the "domain" code
from
> > > the svcauth_unix code. Once that is done, we can fix up GRANT_MSG and
> > > NOTIFY as well as any other routines that do not need to be
> > > authenticated by rpc.mountd.
> >
> > Any idea how long is this likely to take?

> I'll try to have something out by early next week.

> The problem is that the server-side rpc code is checking *all* incoming
> auth_unix and auth_null requests (whether for nfsd or for some other
> service) against the exports table.  So as a temporary workaround, lockd
> will work if the client lists the server in the exports file.  Or
> something like

It worked only after I mounted a client export on the server side.

> echo nfsd server.ip.address.here 0x7FFFFFFF server.name.here \
> >/proc/net/rpc/auth.unix.ip/content
>
> may also work, though the entry you add to the auth.unix.ip cache by
> doing this may end up getting purged next time you run exportfs.

It doesn't look like you can write to /proc/net/rpc/auth.unix.ip/content.

> --Bruce Fields

Thanks, Marc.



-------------------------------------------------------
This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170
Project Admins to receive an Apple iPod Mini FREE for your judgement on
who ports your project to Linux PPC the best. Sponsored by IBM. 
Deadline: Sept. 13. Go here: http://sf.net/ppc_contest.php
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd GRANT_MSG RPC callback

[J. Bruce Fields]

On Thu, Sep 09, 2004 at 11:20:41AM -0700, Marc Eshel wrote:
> "J. Bruce Fields" <bfields@fieldses.org> wrote on 09/09/2004 05:56:25 AM:
> 
> > On Thu, Sep 09, 2004 at 01:08:36PM +1000, Greg Banks wrote:
> > > On Thu, 2004-09-09 at 06:50, Trond Myklebust wrote:
> > > > OK. Bruce has promised to work on splitting out the "domain" code
> from
> > > > the svcauth_unix code. Once that is done, we can fix up GRANT_MSG and
> > > > NOTIFY as well as any other routines that do not need to be
> > > > authenticated by rpc.mountd.
> > >
> > > Any idea how long is this likely to take?
> 
> > I'll try to have something out by early next week.
> 
> > The problem is that the server-side rpc code is checking *all* incoming
> > auth_unix and auth_null requests (whether for nfsd or for some other
> > service) against the exports table.  So as a temporary workaround, lockd
> > will work if the client lists the server in the exports file.  Or
> > something like
> 
> It worked only after I mounted a client export on the server side.
> 
> > echo nfsd server.ip.address.here 0x7FFFFFFF server.name.here \
> > >/proc/net/rpc/auth.unix.ip/content
> >
> > may also work, though the entry you add to the auth.unix.ip cache by
> > doing this may end up getting purged next time you run exportfs.
> 
> It doesn't look like you can write to /proc/net/rpc/auth.unix.ip/content.

Have you mounted the nfsd filesystem at /proc/fs/nfs/?  (See the
exportfs(8) man page.)

--Bruce Fields


-------------------------------------------------------
This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170
Project Admins to receive an Apple iPod Mini FREE for your judgement on
who ports your project to Linux PPC the best. Sponsored by IBM. 
Deadline: Sept. 13. Go here: http://sf.net/ppc_contest.php
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd GRANT_MSG RPC callback

[Greg Banks]

On Thu, 2004-09-09 at 22:56, J. Bruce Fields wrote:
> On Thu, Sep 09, 2004 at 01:08:36PM +1000, Greg Banks wrote:
> > On Thu, 2004-09-09 at 06:50, Trond Myklebust wrote:
> > > OK. Bruce has promised to work on splitting out the "domain" code from
> > > the svcauth_unix code. Once that is done, we can fix up GRANT_MSG and
> > > NOTIFY as well as any other routines that do not need to be
> > > authenticated by rpc.mountd.
> > 
> > Any idea how long is this likely to take?
> 
> I'll try to have something out by early next week.

Cool.  I presume you'll be adding a field to the svc_procedure struct
to indicate whether an export is necessary and fiddling with the order
of code around the call to svc_authenticate() in svc_process() ?

> [...] So as a temporary workaround, lockd
> will work if the client lists the server in the exports file.

Ok, thanks.

Greg.
-- 
Greg Banks, R&D Software Engineer, SGI Australian Software Group.
I don't speak for SGI.




-------------------------------------------------------
This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170
Project Admins to receive an Apple iPod Mini FREE for your judgement on
who ports your project to Linux PPC the best. Sponsored by IBM. 
Deadline: Sept. 13. Go here: http://sf.net/ppc_contest.php
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd GRANT_MSG RPC callback

[Trond Myklebust]

P=E5 to , 09/09/2004 klokka 22:51, skreiv Greg Banks:

> Cool.  I presume you'll be adding a field to the svc_procedure struct
> to indicate whether an export is necessary and fiddling with the order
> of code around the call to svc_authenticate() in svc_process() ?

I have already added pg_authenticate() into svc_program for this very
purpose.

The NFSv4 delegation callback makes use of it to provide its own
AUTH_UNIX authentication (for as long as the common one remains borken)
and to check for whether or not we've got an NFSv4 mount for this
server. The lockd GRANTED stuff, which wants exactly the same type of
check, can just reuse this.

nfsd and the remaining lockd code can use that same hook to add domain
checking to its AUTH_UNIX stuff.

Cheers,
  Trond



-------------------------------------------------------
This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170
Project Admins to receive an Apple iPod Mini FREE for your judgement on
who ports your project to Linux PPC the best. Sponsored by IBM. 
Deadline: Sept. 13. Go here: http://sf.net/ppc_contest.php
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd GRANT_MSG RPC callback

[Greg Banks]

On Sat, 2004-09-11 at 02:22, Trond Myklebust wrote:
> P=E5 to , 09/09/2004 klokka 22:51, skreiv Greg Banks:
>=20
> > Cool.  I presume you'll be adding a field to the svc_procedure struct
> > to indicate whether an export is necessary and fiddling with the order
> > of code around the call to svc_authenticate() in svc_process() ?
>=20
> I have already added pg_authenticate() into svc_program for this very
> purpose.

Ah, that would be=20

http://linus.bkbits.net:8080/linux-2.5/cset@1.1830.1.4

My kernel didn't have that.

> [...]
> nfsd and the remaining lockd code can use that same hook to add domain
> checking to its AUTH_UNIX stuff.

Ok.  I'm looking forward to Bruce's patch.

Greg.
--=20
Greg Banks, R&D Software Engineer, SGI Australian Software Group.
I don't speak for SGI.




-------------------------------------------------------
This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170
Project Admins to receive an Apple iPod Mini FREE for your judgement on
who ports your project to Linux PPC the best. Sponsored by IBM. 
Deadline: Sept. 13. Go here: http://sf.net/ppc_contest.php
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs