lockd and krb5

lockd and krb5

[Helge Bahmann]

Hello,

I am trying to use NFSv3+sec=krb5 for user home directories and I am still 
hitting lockd problems; server is 2.6.18.1 and client is 2.6.17.9

The symptoms are:
- withouth sec=krb5 everything works fine
- with sec=krb5 file locking breaks in strange ways

With sec=krb5:
- processes on the client "hang" waiting for locks
- /proc/locks on the client shows the corresponding files to be not locked
- /proc/locks on the server shows the corresponding files to be locked, by 
processes that do not exist on either the server nor the client (from the pid 
they must be client processes that have exited already)
- sometimes I receive the message "VFS is out of sync with lock manager!"

Unfortunately I have not been able to write a simple testcase, however KDE 
startup triggers the problem very frequently (lots of apps taking a flurry of 
locks and doing funny things with fork)

Can anyone enlighten me what difference sec=krb5 would make with regards to 
locking behaviour so I can narrow down the problem? I am unfortunately 
completely in the dark.

Best regards
-- 
Mathematicians stand on each other's shoulders while computer scientists stand 
on each other's toes.
-- Richard Hamming

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd and krb5

[Kevin Coffman]

I can't help directly with the locking behavior, but could it be
related to expiring Kerberos credentials and/or gss contexts?  Is the
KDE startup that often triggers this at initial login, or at some
other time?

On 11/16/06, Helge Bahmann <hcb@chaoticmind.net> wrote:
> Hello,
>
> I am trying to use NFSv3+sec=krb5 for user home directories and I am still
> hitting lockd problems; server is 2.6.18.1 and client is 2.6.17.9
>
> The symptoms are:
> - withouth sec=krb5 everything works fine
> - with sec=krb5 file locking breaks in strange ways
>
> With sec=krb5:
> - processes on the client "hang" waiting for locks
> - /proc/locks on the client shows the corresponding files to be not locked
> - /proc/locks on the server shows the corresponding files to be locked, by
> processes that do not exist on either the server nor the client (from the pid
> they must be client processes that have exited already)
> - sometimes I receive the message "VFS is out of sync with lock manager!"
>
> Unfortunately I have not been able to write a simple testcase, however KDE
> startup triggers the problem very frequently (lots of apps taking a flurry of
> locks and doing funny things with fork)
>
> Can anyone enlighten me what difference sec=krb5 would make with regards to
> locking behaviour so I can narrow down the problem? I am unfortunately
> completely in the dark.
>
> Best regards
> --
> Mathematicians stand on each other's shoulders while computer scientists stand
> on each other's toes.
> -- Richard Hamming
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd and krb5

[Helge Bahmann]

Am Donnerstag, 16. November 2006 15:21 schrieb Kevin Coffman:
> I can't help directly with the locking behavior, but could it be
> related to expiring Kerberos credentials and/or gss contexts? 

no credentials are valid

> Is the KDE startup that often triggers this at initial login, or at some
> other time?

seems completely erratic; it happens both at initial login as well as second 
login (after successful logout, but before credentials expire); if there is 
any regularity at all then it seems that initial login seems more likely to 
succeed

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd and krb5

[J. Bruce Fields]

On Thu, Nov 16, 2006 at 05:02:38PM +0100, Helge Bahmann wrote:
> Am Donnerstag, 16. November 2006 15:21 schrieb Kevin Coffman:
> > I can't help directly with the locking behavior, but could it be
> > related to expiring Kerberos credentials and/or gss contexts? 
> 
> no credentials are valid
> 
> > Is the KDE startup that often triggers this at initial login, or at some
> > other time?
> 
> seems completely erratic; it happens both at initial login as well as second 
> login (after successful logout, but before credentials expire); if there is 
> any regularity at all then it seems that initial login seems more likely to 
> succeed

Is the filesystem exported under both secuirty flavors (krb5 and sys),
with the export options otherwise the same?

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd and krb5

[Helge Bahmann]

> > > Is the KDE startup that often triggers this at initial login, or at
> > > some other time?
> >
> > seems completely erratic; it happens both at initial login as well as
> > second login (after successful logout, but before credentials expire); if
> > there is any regularity at all then it seems that initial login seems
> > more likely to succeed
>
> Is the filesystem exported under both secuirty flavors (krb5 and sys),
> with the export options otherwise the same?

it is exported to the following clients:
*(ro,all_sqash,fsid=9)
test.client.for.auth_unix(rw,sync,fsid=9)
gss/krb5(rw,sync,fsid=9)

-- 
Mathematicians stand on each other's shoulders while computer scientists stand 
on each other's toes.
-- Richard Hamming

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd and krb5

[Helge Bahmann]

Am Donnerstag, 16. November 2006 17:27 schrieb J. Bruce Fields:
> On Thu, Nov 16, 2006 at 05:02:38PM +0100, Helge Bahmann wrote:
> > Am Donnerstag, 16. November 2006 15:21 schrieb Kevin Coffman:
> > > I can't help directly with the locking behavior, but could it be
> > > related to expiring Kerberos credentials and/or gss contexts?
> >
> > no credentials are valid
> >
> > > Is the KDE startup that often triggers this at initial login, or at
> > > some other time?
> >
> > seems completely erratic; it happens both at initial login as well as
> > second login (after successful logout, but before credentials expire); if
> > there is any regularity at all then it seems that initial login seems
> > more likely to succeed
>
> Is the filesystem exported under both secuirty flavors (krb5 and sys),
> with the export options otherwise the same?

Is there a better tool than tcpdump to capture the rpc traffic? I can try to 
capture a failed (auth-krb5) and succesfull (auth-sys)  login session, maybe 
something stands out

best regards

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd and krb5

[J. Bruce Fields]

On Fri, Nov 17, 2006 at 09:22:40AM +0100, Helge Bahmann wrote:
> > > > Is the KDE startup that often triggers this at initial login, or at
> > > > some other time?
> > >
> > > seems completely erratic; it happens both at initial login as well as
> > > second login (after successful logout, but before credentials expire); if
> > > there is any regularity at all then it seems that initial login seems
> > > more likely to succeed
> >
> > Is the filesystem exported under both secuirty flavors (krb5 and sys),
> > with the export options otherwise the same?
> 
> it is exported to the following clients:
> *(ro,all_sqash,fsid=9)
> test.client.for.auth_unix(rw,sync,fsid=9)
> gss/krb5(rw,sync,fsid=9)

Do you see the same problems if your exports all have the same options?
E.g.

	*(rw,sync,fsid=9)
	gss/krb5(rw,sync,fsid=9)

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd and krb5

[Helge Bahmann]

Am Freitag, 17. November 2006 19:44 schrieben Sie:
> On Fri, Nov 17, 2006 at 09:22:40AM +0100, Helge Bahmann wrote:
> > > > > Is the KDE startup that often triggers this at initial login, or at
> > > > > some other time?
> > > >
> > > > seems completely erratic; it happens both at initial login as well as
> > > > second login (after successful logout, but before credentials
> > > > expire); if there is any regularity at all then it seems that initial
> > > > login seems more likely to succeed
> > >
> > > Is the filesystem exported under both secuirty flavors (krb5 and sys),
> > > with the export options otherwise the same?
> >
> > it is exported to the following clients:
> > *(ro,all_sqash,fsid=9)
> > test.client.for.auth_unix(rw,sync,fsid=9)
> > gss/krb5(rw,sync,fsid=9)
>
> Do you see the same problems if your exports all have the same options?
> E.g.
>
> 	*(rw,sync,fsid=9)
> 	gss/krb5(rw,sync,fsid=9)

this seems kind of pointless because then I would be exporting the whole 
filesystem with "sys" security which is exactly what I want to avoid

but AFAIC remember the server had an active export entry with sys security for 
the test machine with the exact same options as for gss/krb5, only "world" 
export was marked ro,all_squash during the gss test

but I will make sure to include this when I try to capture traffic logs next 
week

Best regards

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: lockd and krb5

[J. Bruce Fields]

On Sun, Nov 19, 2006 at 06:31:57PM +0100, Helge Bahmann wrote:
> Am Freitag, 17. November 2006 19:44 schrieben Sie:
> > On Fri, Nov 17, 2006 at 09:22:40AM +0100, Helge Bahmann wrote:
> > > > > > Is the KDE startup that often triggers this at initial login, or at
> > > > > > some other time?
> > > > >
> > > > > seems completely erratic; it happens both at initial login as well as
> > > > > second login (after successful logout, but before credentials
> > > > > expire); if there is any regularity at all then it seems that initial
> > > > > login seems more likely to succeed
> > > >
> > > > Is the filesystem exported under both secuirty flavors (krb5 and sys),
> > > > with the export options otherwise the same?
> > >
> > > it is exported to the following clients:
> > > *(ro,all_sqash,fsid=9)
> > > test.client.for.auth_unix(rw,sync,fsid=9)
> > > gss/krb5(rw,sync,fsid=9)
> >
> > Do you see the same problems if your exports all have the same options?
> > E.g.
> >
> > 	*(rw,sync,fsid=9)
> > 	gss/krb5(rw,sync,fsid=9)
> 
> this seems kind of pointless because then I would be exporting the whole 
> filesystem with "sys" security which is exactly what I want to avoid

I agree; but knowing whether you can reproduce the same problem with the
above configuration might help determine where exactly the bug is.

(Unfortunately, though, there's a known problem here: since the lockd
client always uses auth_sys, locking will not work on a client that
doesn't have auth_sys access to the export.  I'm not sure yet what the
right fix is for that problem.)

> but AFAIC remember the server had an active export entry with sys
> security for the test machine with the exact same options as for
> gss/krb5, only "world" export was marked ro,all_squash during the gss
> test

Yes, I'm not sure why that didn't work.

> but I will make sure to include this when I try to capture traffic
> logs next week

Thanks.--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs