nfsv3 gssapi client?

nfsv3 gssapi client?

[George Mamalakis]

   Hi all,

I have a FreeBSD nfsv3 server that exports a filesystem with sec=krb5. 
Mounting the share with sec=krb5 under a fbsd client works fine. I now 
try to mount it under linux (archlinux, upgraded today) using nfs-utils. 
Heimdal is configured on the linux-box, kinit -k linuxclient works fine. 
I am also able to kinit to my user principals from it. When I try to 
mount the nvsv3 sec=krb5 share, I get the following error:

# mount -t nfs -o sec=krb5 fbsdserver:/exports /mnt

# tail /var/log/messages
Sep 17 16:05:31 linuxclient rpc.statd[27683]: Version 1.2.2 starting
Sep 17 16:05:31 linuxclient sm-notify[27684]: Version 1.2.2 starting
Sep 17 16:05:31 linuxclient sm-notify[27684]: Already notifying clients; 
Exiting!
Sep 17 16:05:31 linuxclient rpc.statd[27683]: Running as root.  chown 
/var/lib/nfs to choose different user
Sep 17 16:05:31 linuxclient sm-notify[27687]: Version 1.2.2 starting
Sep 17 16:05:31 linuxclient sm-notify[27687]: Already notifying clients; 
Exiting!
Sep 17 16:05:53 linuxclient kernel: svc: failed to register lockdv1 RPC 
service (errno 111).
Sep 17 16:05:53 linuxclient kernel: lockd_up: makesock failed, error=-111
Sep 17 16:05:54 linuxclient kernel: svc: failed to register lockdv1 RPC 
service (errno 111).
Sep 17 16:05:57 linuxclient kernel: svc: failed to register lockdv1 RPC 
service (errno 111).
Sep 17 16:06:01 linuxclient kernel: svc: failed to register lockdv1 RPC 
service (errno 111).

and it keeps on like this.

My nfs-common.conf reads:

[root@linuxclient ~]# cat /etc/conf.d/nfs-common.conf
# Parameters to be passed to nfs-common (nfs clients & server) init script.
#

# If you do not set values for the NEED_ options, they will be attempted
# autodetected; this should be sufficient for most people. Valid 
alternatives
# for the NEED_ options are "yes" and "no".

# Do you want to start the statd daemon? It is not needed for NFSv4.
NEED_STATD=""

# Options to pass to rpc.statd.
# See rpc.statd(8) for more details.
# N.B. statd normally runs on both client and server, and run-time
# options should be specified accordingly.
# STATD_OPTS="-p 32765 -o 32766"
STATD_OPTS=""

# Options to pass to sm-notify
# e.g. SMNOTIFY_OPTS="-p 32764"
SMNOTIFY_OPTS=""

# Do you want to start the idmapd daemon? It is only needed for NFSv4.
NEED_IDMAPD=""

# Options to pass to rpc.idmapd.
# See rpc.idmapd(8) for more details.
IDMAPD_OPTS="-vvv"

# Do you want to start the gssd daemon? It is required for Kerberos mounts.
NEED_GSSD="yes"

# Options to pass to rpc.gssd.
# See rpc.gssd(8) for more details.
GSSD_OPTS="-vvv"
#RPCGSSDOPTS="-vvv"
# Where to mount rpc_pipefs filesystem; the default is 
"/var/lib/nfs/rpc_pipefs".
PIPEFS_MOUNTPOINT=""

# Options used to mount rpc_pipefs filesystem; the default is "defaults".
PIPEFS_MOUNTOPTS=""

my rpc processes are:
[root@linuxclient ~]# ps axuww  | grep -i rpc
root      1228  0.0  0.0      0     0 ?        S    14:47   0:00 [rpciod/0]
root     27670  0.0  0.0   6232   908 ?        Ss   16:05   0:00 
/usr/bin/rpcbind
root     27683  0.0  0.1   6332  1236 ?        Ss   16:05   0:00 
/usr/sbin/rpc.statd
root     27699  0.0  0.1   6264  1180 ?        Ss   16:05   0:00 
/usr/sbin/rpc.gssd -vvv
root     27720  0.0  0.0   3776   476 pts/0    S+   17:01   0:00 grep -i 
rpc

And rpcinfo shows:
root@linuxclient ~]# rpcinfo
    program version netid     address                service    owner
     100000    4    tcp6      ::.0.111               portmapper superuser
     100000    3    tcp6      ::.0.111               portmapper superuser
     100000    4    udp6      ::.0.111               portmapper superuser
     100000    3    udp6      ::.0.111               portmapper superuser
     100000    4    udp       0.0.0.0.0.111          portmapper superuser
     100000    3    udp       0.0.0.0.0.111          portmapper superuser
     100000    2    udp       0.0.0.0.0.111          portmapper superuser
     100000    4    local     /var/run/rpcbind.sock  portmapper superuser
     100000    3    local     /var/run/rpcbind.sock  portmapper superuser
     100024    1    udp       0.0.0.0.228.144        status     superuser
     100024    1    tcp       0.0.0.0.198.8          status     superuser
[root@linuxclient ~]# rpcinfo -s
    program version(s) netid(s)                         service     owner
     100000  2,3,4     local,udp,udp6,tcp6              portmapper  
superuser
     100024  1         tcp,udp                          status      
superuser

whereas on the fbsd box I have:
[root@fbsdserver ~]# rpcinfo -s
    program version(s) netid(s)                         service     owner
     100000  2,3,4     local,udp6,tcp6,udp,tcp          rpcbind     
superuser
     100024  1         tcp,udp,tcp6,udp6                status      
superuser
     100021  4,3,1,0   tcp,udp,tcp6,udp6                nlockmgr    
superuser
     100003  3,2       tcp6,tcp,udp6,udp                nfs         
superuser
     100005  3,1       tcp,udp,tcp6,udp6                mountd      
superuser


The versions I use are:
rpcbind-0.2.0-2
nfs-utils-1.2.2-3

And uname -a shows:

[root@linuxclient ~]# uname -a
Linux linuxclient 2.6.35-ARCH #1 SMP PREEMPT Fri Aug 27 16:22:18 UTC 
2010 i686 Intel(R) Xeon(R) CPU E5310 @ 1.60GHz GenuineIntel GNU/Linux

Does linux support RPCSEC_GSS security flavors over nvsv3? And if so, 
could somebody direct me on how to establish mounting the remote share?

Thank you all for your time and attention in advance,

regards,

mamalos

-- 
George Mamalakis

IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)

Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki

phone number : +30 (2310) 994379

Re: nfsv3 gssapi client?

[J. Bruce Fields]

On Fri, Sep 17, 2010 at 06:54:35PM +0300, George Mamalakis wrote:
>   Hi all,
> 
> I have a FreeBSD nfsv3 server that exports a filesystem with
> sec=krb5. Mounting the share with sec=krb5 under a fbsd client works
> fine. I now try to mount it under linux (archlinux, upgraded today)
> using nfs-utils. Heimdal is configured on the linux-box, kinit -k
> linuxclient works fine. I am also able to kinit to my user
> principals from it. When I try to mount the nvsv3 sec=krb5 share, I
> get the following error:
> 
> # mount -t nfs -o sec=krb5 fbsdserver:/exports /mnt
> 

Was there supposed to be some error output there?  Or did the mount just
hang?

> # tail /var/log/messages
> Sep 17 16:05:31 linuxclient rpc.statd[27683]: Version 1.2.2 starting
> Sep 17 16:05:31 linuxclient sm-notify[27684]: Version 1.2.2 starting
> Sep 17 16:05:31 linuxclient sm-notify[27684]: Already notifying
> clients; Exiting!
> Sep 17 16:05:31 linuxclient rpc.statd[27683]: Running as root.
> chown /var/lib/nfs to choose different user
> Sep 17 16:05:31 linuxclient sm-notify[27687]: Version 1.2.2 starting
> Sep 17 16:05:31 linuxclient sm-notify[27687]: Already notifying
> clients; Exiting!
> Sep 17 16:05:53 linuxclient kernel: svc: failed to register lockdv1
> RPC service (errno 111).
> Sep 17 16:05:53 linuxclient kernel: lockd_up: makesock failed, error=-111
> Sep 17 16:05:54 linuxclient kernel: svc: failed to register lockdv1
> RPC service (errno 111).
> Sep 17 16:05:57 linuxclient kernel: svc: failed to register lockdv1
> RPC service (errno 111).
> Sep 17 16:06:01 linuxclient kernel: svc: failed to register lockdv1
> RPC service (errno 111).

111 is ECONNREFUSED.  I'm not sure why that's failing.  (Should failure
to register lockd fail the whole mount?  I thought it would at worst
result in ENOLCK on lock requests?) Do you get better results if you
mount with nolock?

> and it keeps on like this.
> 
> My nfs-common.conf reads:
> 
> [root@linuxclient ~]# cat /etc/conf.d/nfs-common.conf
> # Parameters to be passed to nfs-common (nfs clients & server) init script.
> #
> 
> # If you do not set values for the NEED_ options, they will be attempted
> # autodetected; this should be sufficient for most people. Valid
> alternatives
> # for the NEED_ options are "yes" and "no".
> 
> # Do you want to start the statd daemon? It is not needed for NFSv4.
> NEED_STATD=""
> 
> # Options to pass to rpc.statd.
> # See rpc.statd(8) for more details.
> # N.B. statd normally runs on both client and server, and run-time
> # options should be specified accordingly.
> # STATD_OPTS="-p 32765 -o 32766"
> STATD_OPTS=""
> 
> # Options to pass to sm-notify
> # e.g. SMNOTIFY_OPTS="-p 32764"
> SMNOTIFY_OPTS=""
> 
> # Do you want to start the idmapd daemon? It is only needed for NFSv4.
> NEED_IDMAPD=""
> 
> # Options to pass to rpc.idmapd.
> # See rpc.idmapd(8) for more details.
> IDMAPD_OPTS="-vvv"
> 
> # Do you want to start the gssd daemon? It is required for Kerberos mounts.
> NEED_GSSD="yes"
> 
> # Options to pass to rpc.gssd.
> # See rpc.gssd(8) for more details.
> GSSD_OPTS="-vvv"
> #RPCGSSDOPTS="-vvv"
> # Where to mount rpc_pipefs filesystem; the default is
> "/var/lib/nfs/rpc_pipefs".
> PIPEFS_MOUNTPOINT=""
> 
> # Options used to mount rpc_pipefs filesystem; the default is "defaults".
> PIPEFS_MOUNTOPTS=""
> 
> my rpc processes are:
> [root@linuxclient ~]# ps axuww  | grep -i rpc
> root      1228  0.0  0.0      0     0 ?        S    14:47   0:00 [rpciod/0]
> root     27670  0.0  0.0   6232   908 ?        Ss   16:05   0:00
> /usr/bin/rpcbind
> root     27683  0.0  0.1   6332  1236 ?        Ss   16:05   0:00
> /usr/sbin/rpc.statd
> root     27699  0.0  0.1   6264  1180 ?        Ss   16:05   0:00
> /usr/sbin/rpc.gssd -vvv
> root     27720  0.0  0.0   3776   476 pts/0    S+   17:01   0:00
> grep -i rpc
> 
> And rpcinfo shows:
> root@linuxclient ~]# rpcinfo
>    program version netid     address                service    owner
>     100000    4    tcp6      ::.0.111               portmapper superuser
>     100000    3    tcp6      ::.0.111               portmapper superuser
>     100000    4    udp6      ::.0.111               portmapper superuser
>     100000    3    udp6      ::.0.111               portmapper superuser
>     100000    4    udp       0.0.0.0.0.111          portmapper superuser
>     100000    3    udp       0.0.0.0.0.111          portmapper superuser
>     100000    2    udp       0.0.0.0.0.111          portmapper superuser
>     100000    4    local     /var/run/rpcbind.sock  portmapper superuser
>     100000    3    local     /var/run/rpcbind.sock  portmapper superuser
>     100024    1    udp       0.0.0.0.228.144        status     superuser
>     100024    1    tcp       0.0.0.0.198.8          status     superuser
> [root@linuxclient ~]# rpcinfo -s
>    program version(s) netid(s)                         service     owner
>     100000  2,3,4     local,udp,udp6,tcp6              portmapper
> superuser
>     100024  1         tcp,udp                          status
> superuser
> 
> whereas on the fbsd box I have:
> [root@fbsdserver ~]# rpcinfo -s
>    program version(s) netid(s)                         service     owner
>     100000  2,3,4     local,udp6,tcp6,udp,tcp          rpcbind
> superuser
>     100024  1         tcp,udp,tcp6,udp6                status
> superuser
>     100021  4,3,1,0   tcp,udp,tcp6,udp6                nlockmgr
> superuser
>     100003  3,2       tcp6,tcp,udp6,udp                nfs
> superuser
>     100005  3,1       tcp,udp,tcp6,udp6                mountd
> superuser
> 
> 
> The versions I use are:
> rpcbind-0.2.0-2
> nfs-utils-1.2.2-3
> 
> And uname -a shows:
> 
> [root@linuxclient ~]# uname -a
> Linux linuxclient 2.6.35-ARCH #1 SMP PREEMPT Fri Aug 27 16:22:18 UTC
> 2010 i686 Intel(R) Xeon(R) CPU E5310 @ 1.60GHz GenuineIntel
> GNU/Linux
> 
> Does linux support RPCSEC_GSS security flavors over nvsv3?

Yes, and this is something I test regularly.

> And if
> so, could somebody direct me on how to establish mounting the remote
> share?

Looks like the server advertises ipv6.  I wonder if anyone's tested gss
in that case?

--b.

> 
> Thank you all for your time and attention in advance,
> 
> regards,
> 
> mamalos
> 
> -- 
> George Mamalakis
> 
> IT Officer
> Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
> MSc (Imperial College of London)
> 
> Department of Electrical and Computer Engineering
> Faculty of Engineering
> Aristotle University of Thessaloniki
> 
> phone number : +30 (2310) 994379
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nfsv3 gssapi client?

[Jeff Layton]

On Fri, 17 Sep 2010 13:43:28 -0400
"J. Bruce Fields" <bfields@fieldses.org> wrote:

> On Fri, Sep 17, 2010 at 06:54:35PM +0300, George Mamalakis wrote:
> >   Hi all,
> > 
> > I have a FreeBSD nfsv3 server that exports a filesystem with
> > sec=krb5. Mounting the share with sec=krb5 under a fbsd client works
> > fine. I now try to mount it under linux (archlinux, upgraded today)
> > using nfs-utils. Heimdal is configured on the linux-box, kinit -k
> > linuxclient works fine. I am also able to kinit to my user
> > principals from it. When I try to mount the nvsv3 sec=krb5 share, I
> > get the following error:
> > 
> > # mount -t nfs -o sec=krb5 fbsdserver:/exports /mnt
> > 
> 
> Was there supposed to be some error output there?  Or did the mount just
> hang?
> 
> > # tail /var/log/messages
> > Sep 17 16:05:31 linuxclient rpc.statd[27683]: Version 1.2.2 starting
> > Sep 17 16:05:31 linuxclient sm-notify[27684]: Version 1.2.2 starting
> > Sep 17 16:05:31 linuxclient sm-notify[27684]: Already notifying
> > clients; Exiting!
> > Sep 17 16:05:31 linuxclient rpc.statd[27683]: Running as root.
> > chown /var/lib/nfs to choose different user
> > Sep 17 16:05:31 linuxclient sm-notify[27687]: Version 1.2.2 starting
> > Sep 17 16:05:31 linuxclient sm-notify[27687]: Already notifying
> > clients; Exiting!
> > Sep 17 16:05:53 linuxclient kernel: svc: failed to register lockdv1
> > RPC service (errno 111).
> > Sep 17 16:05:53 linuxclient kernel: lockd_up: makesock failed, error=-111
> > Sep 17 16:05:54 linuxclient kernel: svc: failed to register lockdv1
> > RPC service (errno 111).
> > Sep 17 16:05:57 linuxclient kernel: svc: failed to register lockdv1
> > RPC service (errno 111).
> > Sep 17 16:06:01 linuxclient kernel: svc: failed to register lockdv1
> > RPC service (errno 111).
> 
> 111 is ECONNREFUSED.  I'm not sure why that's failing.  (Should failure
> to register lockd fail the whole mount?  I thought it would at worst
> result in ENOLCK on lock requests?) Do you get better results if you
> mount with nolock?
> 

Firewalling issues maybe?

> > and it keeps on like this.
> > 
> > My nfs-common.conf reads:
> > 
> > [root@linuxclient ~]# cat /etc/conf.d/nfs-common.conf
> > # Parameters to be passed to nfs-common (nfs clients & server) init script.
> > #
> > 
> > # If you do not set values for the NEED_ options, they will be attempted
> > # autodetected; this should be sufficient for most people. Valid
> > alternatives
> > # for the NEED_ options are "yes" and "no".
> > 
> > # Do you want to start the statd daemon? It is not needed for NFSv4.
> > NEED_STATD=""
> > 
> > # Options to pass to rpc.statd.
> > # See rpc.statd(8) for more details.
> > # N.B. statd normally runs on both client and server, and run-time
> > # options should be specified accordingly.
> > # STATD_OPTS="-p 32765 -o 32766"
> > STATD_OPTS=""
> > 
> > # Options to pass to sm-notify
> > # e.g. SMNOTIFY_OPTS="-p 32764"
> > SMNOTIFY_OPTS=""
> > 
> > # Do you want to start the idmapd daemon? It is only needed for NFSv4.
> > NEED_IDMAPD=""
> > 
> > # Options to pass to rpc.idmapd.
> > # See rpc.idmapd(8) for more details.
> > IDMAPD_OPTS="-vvv"
> > 
> > # Do you want to start the gssd daemon? It is required for Kerberos mounts.
> > NEED_GSSD="yes"
> > 
> > # Options to pass to rpc.gssd.
> > # See rpc.gssd(8) for more details.
> > GSSD_OPTS="-vvv"
> > #RPCGSSDOPTS="-vvv"
> > # Where to mount rpc_pipefs filesystem; the default is
> > "/var/lib/nfs/rpc_pipefs".
> > PIPEFS_MOUNTPOINT=""
> > 
> > # Options used to mount rpc_pipefs filesystem; the default is "defaults".
> > PIPEFS_MOUNTOPTS=""
> > 
> > my rpc processes are:
> > [root@linuxclient ~]# ps axuww  | grep -i rpc
> > root      1228  0.0  0.0      0     0 ?        S    14:47   0:00 [rpciod/0]
> > root     27670  0.0  0.0   6232   908 ?        Ss   16:05   0:00
> > /usr/bin/rpcbind
> > root     27683  0.0  0.1   6332  1236 ?        Ss   16:05   0:00
> > /usr/sbin/rpc.statd
> > root     27699  0.0  0.1   6264  1180 ?        Ss   16:05   0:00
> > /usr/sbin/rpc.gssd -vvv
> > root     27720  0.0  0.0   3776   476 pts/0    S+   17:01   0:00
> > grep -i rpc
> > 
> > And rpcinfo shows:
> > root@linuxclient ~]# rpcinfo
> >    program version netid     address                service    owner
> >     100000    4    tcp6      ::.0.111               portmapper superuser
> >     100000    3    tcp6      ::.0.111               portmapper superuser
> >     100000    4    udp6      ::.0.111               portmapper superuser
> >     100000    3    udp6      ::.0.111               portmapper superuser
> >     100000    4    udp       0.0.0.0.0.111          portmapper superuser
> >     100000    3    udp       0.0.0.0.0.111          portmapper superuser
> >     100000    2    udp       0.0.0.0.0.111          portmapper superuser
> >     100000    4    local     /var/run/rpcbind.sock  portmapper superuser
> >     100000    3    local     /var/run/rpcbind.sock  portmapper superuser
> >     100024    1    udp       0.0.0.0.228.144        status     superuser
> >     100024    1    tcp       0.0.0.0.198.8          status     superuser
> > [root@linuxclient ~]# rpcinfo -s
> >    program version(s) netid(s)                         service     owner
> >     100000  2,3,4     local,udp,udp6,tcp6              portmapper
> > superuser
> >     100024  1         tcp,udp                          status
> > superuser
> > 
> > whereas on the fbsd box I have:
> > [root@fbsdserver ~]# rpcinfo -s
> >    program version(s) netid(s)                         service     owner
> >     100000  2,3,4     local,udp6,tcp6,udp,tcp          rpcbind
> > superuser
> >     100024  1         tcp,udp,tcp6,udp6                status
> > superuser
> >     100021  4,3,1,0   tcp,udp,tcp6,udp6                nlockmgr
> > superuser
> >     100003  3,2       tcp6,tcp,udp6,udp                nfs
> > superuser
> >     100005  3,1       tcp,udp,tcp6,udp6                mountd
> > superuser
> > 
> > 
> > The versions I use are:
> > rpcbind-0.2.0-2
> > nfs-utils-1.2.2-3
> > 
> > And uname -a shows:
> > 
> > [root@linuxclient ~]# uname -a
> > Linux linuxclient 2.6.35-ARCH #1 SMP PREEMPT Fri Aug 27 16:22:18 UTC
> > 2010 i686 Intel(R) Xeon(R) CPU E5310 @ 1.60GHz GenuineIntel
> > GNU/Linux
> > 
> > Does linux support RPCSEC_GSS security flavors over nvsv3?
> 
> Yes, and this is something I test regularly.
> 
> > And if
> > so, could somebody direct me on how to establish mounting the remote
> > share?
> 
> Looks like the server advertises ipv6.  I wonder if anyone's tested gss
> in that case?
> 

Not in a while, but I did test it when I did the IPv6 changes for
rpc.gssd. It should work fine, AFAIK. Based on the logs above,
RPCSEC_GSS is probably a red herring. The real problem seems to be that
there are problems communicating with the portmapper.

-- 
Jeff Layton <jlayton@redhat.com>

Re: nfsv3 gssapi client?

[Chuck Lever]

On Sep 17, 2010, at 1:43 PM, J. Bruce Fields wrote:

> On Fri, Sep 17, 2010 at 06:54:35PM +0300, George Mamalakis wrote:
>>  Hi all,
>> 
>> I have a FreeBSD nfsv3 server that exports a filesystem with
>> sec=krb5. Mounting the share with sec=krb5 under a fbsd client works
>> fine. I now try to mount it under linux (archlinux, upgraded today)
>> using nfs-utils. Heimdal is configured on the linux-box, kinit -k
>> linuxclient works fine. I am also able to kinit to my user
>> principals from it. When I try to mount the nvsv3 sec=krb5 share, I
>> get the following error:
>> 
>> # mount -t nfs -o sec=krb5 fbsdserver:/exports /mnt
>> 
> 
> Was there supposed to be some error output there?  Or did the mount just
> hang?
> 
>> # tail /var/log/messages
>> Sep 17 16:05:31 linuxclient rpc.statd[27683]: Version 1.2.2 starting
>> Sep 17 16:05:31 linuxclient sm-notify[27684]: Version 1.2.2 starting
>> Sep 17 16:05:31 linuxclient sm-notify[27684]: Already notifying
>> clients; Exiting!
>> Sep 17 16:05:31 linuxclient rpc.statd[27683]: Running as root.
>> chown /var/lib/nfs to choose different user
>> Sep 17 16:05:31 linuxclient sm-notify[27687]: Version 1.2.2 starting
>> Sep 17 16:05:31 linuxclient sm-notify[27687]: Already notifying
>> clients; Exiting!
>> Sep 17 16:05:53 linuxclient kernel: svc: failed to register lockdv1
>> RPC service (errno 111).
>> Sep 17 16:05:53 linuxclient kernel: lockd_up: makesock failed, error=-111
>> Sep 17 16:05:54 linuxclient kernel: svc: failed to register lockdv1
>> RPC service (errno 111).
>> Sep 17 16:05:57 linuxclient kernel: svc: failed to register lockdv1
>> RPC service (errno 111).
>> Sep 17 16:06:01 linuxclient kernel: svc: failed to register lockdv1
>> RPC service (errno 111).
> 
> 111 is ECONNREFUSED.  I'm not sure why that's failing.  (Should failure
> to register lockd fail the whole mount?

I could be wrong, but I believe that yes, it should fail the mount operation in this case, unless you specified "nolock".

> I thought it would at worst
> result in ENOLCK on lock requests?) Do you get better results if you
> mount with nolock?

-- 
chuck[dot]lever[at]oracle[dot]com

Re: nfsv3 gssapi client?

[George Mamalakis]

  Hi Bruce and thanx for the reply,

to your questions now:

On 17/09/2010 20:43, J. Bruce Fields wrote:
>
>> # mount -t nfs -o sec=krb5 fbsdserver:/exports /mnt
>>
> Was there supposed to be some error output there?  Or did the mount just
> hang?
>
The mount command stalls and no output is presented; only the messages 
on /var/log/messages.
>> Sep 17 16:06:01 linuxclient kernel: svc: failed to register lockdv1
>> RPC service (errno 111).
> 111 is ECONNREFUSED.  I'm not sure why that's failing.  (Should failure
> to register lockd fail the whole mount?  I thought it would at worst
> result in ENOLCK on lock requests?) Do you get better results if you
> mount with nolock?
>
Now this is peculiar. If run mount -o nolock then gssd segfaults. This 
is because it creates a /tmp/krb5cc_machine_REALM ticket, and when it 
access it, it segfaults. To overcome this issue I deleted the temporary 
file, and ran gssd manually to see its output. So, mount returned the 
following:

[root@linuxclient ~]# mount -t nfs -o sec=krb5,nolock 
fbsdserver:/exports /mnt
mount.nfs: access denied by server while mounting fbsdserver:/exports

and gssd console output read:

destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt110
handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
process_krb5_upcall: service is '<null>'
getting credentials for client with uid 0 for server fbsdserver
WARNING: Failed to create krb5 context for user with uid 0 for server 
fbsdserver
doing error downcall
destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt113
handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
process_krb5_upcall: service is '<null>'
getting credentials for client with uid 0 for server fbsdserver
WARNING: Failed to create krb5 context for user with uid 0 for server 
fbsdserver
doing error downcall
handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
process_krb5_upcall: service is '<null>'
getting credentials for client with uid 0 for server fbsdclient.ee.auth.gr
WARNING: Failed to create krb5 context for user with uid 0 for server 
fbsdserver
doing error downcall
destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt112
destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt111

/var/log/messages shows nothing.


I think that's all regarding your questions.

Thanx again,

mamalos.

-- 
George Mamalakis

IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)

Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki

phone number : +30 (2310) 994379

Re: nfsv3 gssapi client?

[George Mamalakis]

  Hi Jeff, and thanx for your answer,
>> 111 is ECONNREFUSED.  I'm not sure why that's failing.  (Should failure
>> to register lockd fail the whole mount?  I thought it would at worst
>> result in ENOLCK on lock requests?) Do you get better results if you
>> mount with nolock?
>>
> Firewalling issues maybe?
>
The fbsd box runs no firewall whatsoever, and nvsv3 mounts work just 
fine if sec=sys is given in the mount command instead of sec=krb5(ip). 
On the linux box I haven't setup any iptables whatsoever. My 
/etc/hosts.allow on the linuxbox reads:
[root@linuxclient ~]# cat /etc/hosts.allow
#
# /etc/hosts.allow
#
ALL:ALL


So I guess that there is no problem from this side too.

Thanx again,

mamalos


PS. Sorry I haven't answered to your question in my previous email, I 
saw your email after sending the previous answer.

-- 
George Mamalakis

IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)

Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki

phone number : +30 (2310) 994379

Re: nfsv3 gssapi client?

[J. Bruce Fields]

On Mon, Sep 20, 2010 at 12:47:31PM +0300, George Mamalakis wrote:
>  Hi Bruce and thanx for the reply,
> 
> to your questions now:
> 
> On 17/09/2010 20:43, J. Bruce Fields wrote:
> >
> >># mount -t nfs -o sec=krb5 fbsdserver:/exports /mnt
> >>
> >Was there supposed to be some error output there?  Or did the mount just
> >hang?
> >
> The mount command stalls and no output is presented; only the
> messages on /var/log/messages.
> >>Sep 17 16:06:01 linuxclient kernel: svc: failed to register lockdv1
> >>RPC service (errno 111).
> >111 is ECONNREFUSED.  I'm not sure why that's failing.  (Should failure
> >to register lockd fail the whole mount?  I thought it would at worst
> >result in ENOLCK on lock requests?) Do you get better results if you
> >mount with nolock?
> >
> Now this is peculiar. If run mount -o nolock then gssd segfaults.
> This is because it creates a /tmp/krb5cc_machine_REALM ticket, and
> when it access it, it segfaults.

Ouch.  Well, that's a problem.

(And have you checked the log files to make sure it's gssd segfaulting
and not some kind of crash in the kernel?)

--b.

> To overcome this issue I deleted
> the temporary file, and ran gssd manually to see its output. So,
> mount returned the following:
> 
> [root@linuxclient ~]# mount -t nfs -o sec=krb5,nolock
> fbsdserver:/exports /mnt
> mount.nfs: access denied by server while mounting fbsdserver:/exports
> 
> and gssd console output read:
> 
> destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt110
> handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
> handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
> handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
> process_krb5_upcall: service is '<null>'
> getting credentials for client with uid 0 for server fbsdserver
> WARNING: Failed to create krb5 context for user with uid 0 for
> server fbsdserver
> doing error downcall
> destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt113
> handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
> handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
> handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
> process_krb5_upcall: service is '<null>'
> getting credentials for client with uid 0 for server fbsdserver
> WARNING: Failed to create krb5 context for user with uid 0 for
> server fbsdserver
> doing error downcall
> handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
> handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
> handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
> process_krb5_upcall: service is '<null>'
> getting credentials for client with uid 0 for server fbsdclient.ee.auth.gr
> WARNING: Failed to create krb5 context for user with uid 0 for
> server fbsdserver
> doing error downcall
> destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt112
> destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt111
> 
> /var/log/messages shows nothing.
> 
> 
> I think that's all regarding your questions.
> 
> Thanx again,
> 
> mamalos.
> 
> -- 
> George Mamalakis
> 
> IT Officer
> Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
> MSc (Imperial College of London)
> 
> Department of Electrical and Computer Engineering
> Faculty of Engineering
> Aristotle University of Thessaloniki
> 
> phone number : +30 (2310) 994379
> 

Re: nfsv3 gssapi client?

[George Mamalakis]

  On 20/09/2010 17:56, J. Bruce Fields wrote:
>
> Ouch.  Well, that's a problem.
>
> (And have you checked the log files to make sure it's gssd segfaulting
> and not some kind of crash in the kernel?)
>
> --b.
>

Yes, it segfaults, even the log shows it. It's a null pointer exception. 
I used gdb in gssd and this is my outcome:

0xb783f424 in __kernel_vsyscall ()
(gdb) c
Continuing.

Program received signal SIG37, Real-time event 37.
0xb783f424 in __kernel_vsyscall ()
(gdb) c
Continuing.

Program received signal SIG37, Real-time event 37.
0xb783f424 in __kernel_vsyscall ()
(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
0xb72b4c8e in __gss_get_mechanism_cred () from /usr/lib/libgssglue.so.1
(gdb) bt
#0  0xb72b4c8e in __gss_get_mechanism_cred () from /usr/lib/libgssglue.so.1
#1  0xb72b5c2c in gss_init_sec_context () from /usr/lib/libgssglue.so.1
#2  0xb776c215 in authgss_refresh () from /usr/lib/libtirpc.so.1
#3  0xb776c67d in authgss_create () from /usr/lib/libtirpc.so.1
#4  0xb776c779 in authgss_create_default () from /usr/lib/libtirpc.so.1
#5  0x0804cbdc in ?? ()
#6  0x0804cf9f in ?? ()
#7  0x0804d7c8 in ?? ()
#8  0x0804b7a8 in ?? ()
#9  0x0804b309 in ?? ()
#10 0xb761cc76 in __libc_start_main () from /lib/libc.so.6
#11 0x0804a621 in ?? ()
(gdb) info r
eax            0x0    0
ecx            0x9    9
edx            0x0    0
ebx            0xb72b9710    -1221880048
esp            0xbfe0b844    0xbfe0b844
ebp            0xbfe0b858    0xbfe0b858
esi            0x9    9
edi            0x9370928    154601768
eip            0xb72b4c8e    0xb72b4c8e <__gss_get_mechanism_cred+62>
eflags         0x210246    [ PF ZF IF RF ID ]
cs             0x73    115
ss             0x7b    123
ds             0x7b    123
es             0x7b    123
fs             0x0    0
gs             0x33    51
(gdb) disass $eip
Dump of assembler code for function __gss_get_mechanism_cred:
    0xb72b4c50 <+0>:    push   %ebp
    0xb72b4c51 <+1>:    mov    %esp,%ebp
    0xb72b4c53 <+3>:    push   %edi
    0xb72b4c54 <+4>:    push   %esi
    0xb72b4c55 <+5>:    sub    $0xc,%esp
    0xb72b4c58 <+8>:    mov    0x8(%ebp),%ecx
    0xb72b4c5b <+11>:    test   %ecx,%ecx
    0xb72b4c5d <+13>:    jne    0xb72b4c68 <__gss_get_mechanism_cred+24>
    0xb72b4c5f <+15>:    add    $0xc,%esp
    0xb72b4c62 <+18>:    xor    %eax,%eax
    0xb72b4c64 <+20>:    pop    %esi
    0xb72b4c65 <+21>:    pop    %edi
    0xb72b4c66 <+22>:    pop    %ebp
    0xb72b4c67 <+23>:    ret
    0xb72b4c68 <+24>:    mov    0x8(%ebp),%eax
    0xb72b4c6b <+27>:    mov    (%eax),%eax
    0xb72b4c6d <+29>:    test   %eax,%eax
    0xb72b4c6f <+31>:    mov    %eax,-0x10(%ebp)
    0xb72b4c72 <+34>:    jle    0xb72b4c5f <__gss_get_mechanism_cred+15>
    0xb72b4c74 <+36>:    mov    0xc(%ebp),%ecx
    0xb72b4c77 <+39>:    mov    0x8(%ebp),%eax
    0xb72b4c7a <+42>:    mov    (%ecx),%edx
    0xb72b4c7c <+44>:    mov    0x4(%eax),%eax
    0xb72b4c7f <+47>:    mov    %edx,-0x14(%ebp)
    0xb72b4c82 <+50>:    mov    %eax,-0xc(%ebp)
    0xb72b4c85 <+53>:    xor    %eax,%eax
    0xb72b4c87 <+55>:    nop
    0xb72b4c88 <+56>:    mov    -0xc(%ebp),%edx
    0xb72b4c8b <+59>:    mov    -0x14(%ebp),%ecx
=> 0xb72b4c8e <+62>:    cmp    (%edx,%eax,8),%ecx
    0xb72b4c91 <+65>:    jne    0xb72b4cab <__gss_get_mechanism_cred+91>
    0xb72b4c93 <+67>:    mov    0xc(%ebp),%edx
    0xb72b4c96 <+70>:    mov    -0xc(%ebp),%ecx
    0xb72b4c99 <+73>:    mov    0x4(%edx),%esi
    0xb72b4c9c <+76>:    mov    -0x14(%ebp),%edx
---Type <return> to continue, or q <return> to quit---q
Quit
(gdb) info r
eax            0x0    0
ecx            0x9    9
edx            0x0    0
ebx            0xb72b9710    -1221880048
esp            0xbfe0b844    0xbfe0b844
ebp            0xbfe0b858    0xbfe0b858
esi            0x9    9
edi            0x9370928    154601768
eip            0xb72b4c8e    0xb72b4c8e <__gss_get_mechanism_cred+62>
eflags         0x210246    [ PF ZF IF RF ID ]
cs             0x73    115
ss             0x7b    123
ds             0x7b    123
es             0x7b    123
fs             0x0    0
gs             0x33    51


Where you can see the null reference.

This, on the other hand, reminded me a discussion I had in the 
fbsd-stable list, where somebody advised me to use des-cbc-crc keytabs, 
and then experiment with other encryption types. Even though I didn't 
have such issues in fbsd, this might be the case on this problem. Maybe 
linux tries to compare an inexistent mech-type (or enc-type, whatever) 
with one of it's list, and hence it segfaults. So, to shed a little more 
light into the issue:

- the kdc (not the nfs-server) runs fbsd8, heimdal-1.2.1.
- on the linuxbox when I "kinit -k linuxclient" everything works fine.
- on the linux client "ktutil -k /etc/krb5.keytab list" shows:
/etc/krb5.keytab:

Vno  Type                     Principal                               
Aliases
   1  des-cbc-md5              host/linuxclient@EXAMPLE
   1  des-cbc-md4              host/linuxclient@EXAMPLE
   1  des-cbc-crc              host/linuxclient@EXAMPLE
   1  aes256-cts-hmac-sha1-96  host/linuxclient@EXAMPLE
   1  des3-cbc-sha1            host/linuxclient@EXAMPLE
   1  arcfour-hmac-md5         host/linuxclient@EXAMPLE

So maybe what I have to do next is to create a keytab of des-cbc-crc 
encryption-type only for the client (maybe also for the nfsserver as 
well..) and see how it'll behave...but I will do it tomorrow.

Until then,

mamalos

-- 
George Mamalakis

IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)

Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki

phone number : +30 (2310) 994379

Re: nfsv3 gssapi client?

[Kevin Coffman]

On Mon, Sep 20, 2010 at 11:38 AM, George Mamalakis <mamalos@eng.auth.gr> wrote:
>  On 20/09/2010 17:56, J. Bruce Fields wrote:
>>
>> Ouch.  Well, that's a problem.
>>
>> (And have you checked the log files to make sure it's gssd segfaulting
>> and not some kind of crash in the kernel?)
>>
>> --b.
>>
>
> Yes, it segfaults, even the log shows it. It's a null pointer exception. I
> used gdb in gssd and this is my outcome:
>
> 0xb783f424 in __kernel_vsyscall ()
> (gdb) c
> Continuing.
>
> Program received signal SIG37, Real-time event 37.
> 0xb783f424 in __kernel_vsyscall ()
> (gdb) c
> Continuing.
>
> Program received signal SIG37, Real-time event 37.
> 0xb783f424 in __kernel_vsyscall ()
> (gdb) c
> Continuing.
>
> Program received signal SIGSEGV, Segmentation fault.
> 0xb72b4c8e in __gss_get_mechanism_cred () from /usr/lib/libgssglue.so.1
> (gdb) bt
> #0  0xb72b4c8e in __gss_get_mechanism_cred () from /usr/lib/libgssglue.so.1
> #1  0xb72b5c2c in gss_init_sec_context () from /usr/lib/libgssglue.so.1
> #2  0xb776c215 in authgss_refresh () from /usr/lib/libtirpc.so.1
> #3  0xb776c67d in authgss_create () from /usr/lib/libtirpc.so.1
> #4  0xb776c779 in authgss_create_default () from /usr/lib/libtirpc.so.1
> #5  0x0804cbdc in ?? ()
> #6  0x0804cf9f in ?? ()
> #7  0x0804d7c8 in ?? ()
> #8  0x0804b7a8 in ?? ()
> #9  0x0804b309 in ?? ()
> #10 0xb761cc76 in __libc_start_main () from /lib/libc.so.6
> #11 0x0804a621 in ?? ()
> (gdb) info r
> eax            0x0    0
> ecx            0x9    9
> edx            0x0    0
> ebx            0xb72b9710    -1221880048
> esp            0xbfe0b844    0xbfe0b844
> ebp            0xbfe0b858    0xbfe0b858
> esi            0x9    9
> edi            0x9370928    154601768
> eip            0xb72b4c8e    0xb72b4c8e <__gss_get_mechanism_cred+62>
> eflags         0x210246    [ PF ZF IF RF ID ]
> cs             0x73    115
> ss             0x7b    123
> ds             0x7b    123
> es             0x7b    123
> fs             0x0    0
> gs             0x33    51
> (gdb) disass $eip
> Dump of assembler code for function __gss_get_mechanism_cred:
>   0xb72b4c50 <+0>:    push   %ebp
>   0xb72b4c51 <+1>:    mov    %esp,%ebp
>   0xb72b4c53 <+3>:    push   %edi
>   0xb72b4c54 <+4>:    push   %esi
>   0xb72b4c55 <+5>:    sub    $0xc,%esp
>   0xb72b4c58 <+8>:    mov    0x8(%ebp),%ecx
>   0xb72b4c5b <+11>:    test   %ecx,%ecx
>   0xb72b4c5d <+13>:    jne    0xb72b4c68 <__gss_get_mechanism_cred+24>
>   0xb72b4c5f <+15>:    add    $0xc,%esp
>   0xb72b4c62 <+18>:    xor    %eax,%eax
>   0xb72b4c64 <+20>:    pop    %esi
>   0xb72b4c65 <+21>:    pop    %edi
>   0xb72b4c66 <+22>:    pop    %ebp
>   0xb72b4c67 <+23>:    ret
>   0xb72b4c68 <+24>:    mov    0x8(%ebp),%eax
>   0xb72b4c6b <+27>:    mov    (%eax),%eax
>   0xb72b4c6d <+29>:    test   %eax,%eax
>   0xb72b4c6f <+31>:    mov    %eax,-0x10(%ebp)
>   0xb72b4c72 <+34>:    jle    0xb72b4c5f <__gss_get_mechanism_cred+15>
>   0xb72b4c74 <+36>:    mov    0xc(%ebp),%ecx
>   0xb72b4c77 <+39>:    mov    0x8(%ebp),%eax
>   0xb72b4c7a <+42>:    mov    (%ecx),%edx
>   0xb72b4c7c <+44>:    mov    0x4(%eax),%eax
>   0xb72b4c7f <+47>:    mov    %edx,-0x14(%ebp)
>   0xb72b4c82 <+50>:    mov    %eax,-0xc(%ebp)
>   0xb72b4c85 <+53>:    xor    %eax,%eax
>   0xb72b4c87 <+55>:    nop
>   0xb72b4c88 <+56>:    mov    -0xc(%ebp),%edx
>   0xb72b4c8b <+59>:    mov    -0x14(%ebp),%ecx
> => 0xb72b4c8e <+62>:    cmp    (%edx,%eax,8),%ecx
>   0xb72b4c91 <+65>:    jne    0xb72b4cab <__gss_get_mechanism_cred+91>
>   0xb72b4c93 <+67>:    mov    0xc(%ebp),%edx
>   0xb72b4c96 <+70>:    mov    -0xc(%ebp),%ecx
>   0xb72b4c99 <+73>:    mov    0x4(%edx),%esi
>   0xb72b4c9c <+76>:    mov    -0x14(%ebp),%edx
> ---Type <return> to continue, or q <return> to quit---q
> Quit
> (gdb) info r
> eax            0x0    0
> ecx            0x9    9
> edx            0x0    0
> ebx            0xb72b9710    -1221880048
> esp            0xbfe0b844    0xbfe0b844
> ebp            0xbfe0b858    0xbfe0b858
> esi            0x9    9
> edi            0x9370928    154601768
> eip            0xb72b4c8e    0xb72b4c8e <__gss_get_mechanism_cred+62>
> eflags         0x210246    [ PF ZF IF RF ID ]
> cs             0x73    115
> ss             0x7b    123
> ds             0x7b    123
> es             0x7b    123
> fs             0x0    0
> gs             0x33    51
>
>
> Where you can see the null reference.
>
> This, on the other hand, reminded me a discussion I had in the fbsd-stable
> list, where somebody advised me to use des-cbc-crc keytabs, and then
> experiment with other encryption types. Even though I didn't have such
> issues in fbsd, this might be the case on this problem. Maybe linux tries to
> compare an inexistent mech-type (or enc-type, whatever) with one of it's
> list, and hence it segfaults. So, to shed a little more light into the
> issue:
>
> - the kdc (not the nfs-server) runs fbsd8, heimdal-1.2.1.
> - on the linuxbox when I "kinit -k linuxclient" everything works fine.
> - on the linux client "ktutil -k /etc/krb5.keytab list" shows:
> /etc/krb5.keytab:
>
> Vno  Type                     Principal
> Aliases
>  1  des-cbc-md5              host/linuxclient@EXAMPLE
>  1  des-cbc-md4              host/linuxclient@EXAMPLE
>  1  des-cbc-crc              host/linuxclient@EXAMPLE
>  1  aes256-cts-hmac-sha1-96  host/linuxclient@EXAMPLE
>  1  des3-cbc-sha1            host/linuxclient@EXAMPLE
>  1  arcfour-hmac-md5         host/linuxclient@EXAMPLE
>
> So maybe what I have to do next is to create a keytab of des-cbc-crc
> encryption-type only for the client (maybe also for the nfsserver as well..)
> and see how it'll behave...but I will do it tomorrow.
>
> Until then,
>
> mamalos


Kernels before 2.6.35:
gssd has code to limit the encryption types negotiated, so limiting
its keytab to only des is not strictly required. (callbacks can
complicate this)  It is required for a linux NFS server.

Kernels 2.6.35 and after:
However I saw from your previous message that the kernel is passing up
"enctypes=18,17,16,23,3,1,2", which indicates it is a rather new
kernel.  What version is your linux kernel?  AFAIK, a version of
nfs-utils with the changes to support newer enctypes has not yet been
released.

What kerberos libraries are on the linux client machine?  (MIT or
Heimdal?)  And what version?  I've seen people successfully using
Heimdal libraries with gssd, but AFAIK it has not been tested like it
has with MIT.

Re: nfsv3 gssapi client?

[J. Bruce Fields]

On Mon, Sep 20, 2010 at 06:38:48PM +0300, George Mamalakis wrote:
>  On 20/09/2010 17:56, J. Bruce Fields wrote:
> >
> >Ouch.  Well, that's a problem.
> >
> >(And have you checked the log files to make sure it's gssd segfaulting
> >and not some kind of crash in the kernel?)
> >
> >--b.
> >
> 
> Yes, it segfaults, even the log shows it. It's a null pointer
> exception. I used gdb in gssd and this is my outcome:
> 
> 0xb783f424 in __kernel_vsyscall ()
> (gdb) c
> Continuing.
> 
> Program received signal SIG37, Real-time event 37.
> 0xb783f424 in __kernel_vsyscall ()
> (gdb) c
> Continuing.
> 
> Program received signal SIG37, Real-time event 37.
> 0xb783f424 in __kernel_vsyscall ()
> (gdb) c
> Continuing.
> 
> Program received signal SIGSEGV, Segmentation fault.
> 0xb72b4c8e in __gss_get_mechanism_cred () from /usr/lib/libgssglue.so.1
> (gdb) bt
> #0  0xb72b4c8e in __gss_get_mechanism_cred () from /usr/lib/libgssglue.so.1
> #1  0xb72b5c2c in gss_init_sec_context () from /usr/lib/libgssglue.so.1
> #2  0xb776c215 in authgss_refresh () from /usr/lib/libtirpc.so.1
> #3  0xb776c67d in authgss_create () from /usr/lib/libtirpc.so.1
> #4  0xb776c779 in authgss_create_default () from /usr/lib/libtirpc.so.1
> #5  0x0804cbdc in ?? ()
> #6  0x0804cf9f in ?? ()
> #7  0x0804d7c8 in ?? ()
> #8  0x0804b7a8 in ?? ()
> #9  0x0804b309 in ?? ()
> #10 0xb761cc76 in __libc_start_main () from /lib/libc.so.6
> #11 0x0804a621 in ?? ()
> (gdb) info r
> eax            0x0    0
> ecx            0x9    9
> edx            0x0    0
> ebx            0xb72b9710    -1221880048
> esp            0xbfe0b844    0xbfe0b844
> ebp            0xbfe0b858    0xbfe0b858
> esi            0x9    9
> edi            0x9370928    154601768
> eip            0xb72b4c8e    0xb72b4c8e <__gss_get_mechanism_cred+62>
> eflags         0x210246    [ PF ZF IF RF ID ]
> cs             0x73    115
> ss             0x7b    123
> ds             0x7b    123
> es             0x7b    123
> fs             0x0    0
> gs             0x33    51
> (gdb) disass $eip
> Dump of assembler code for function __gss_get_mechanism_cred:
>    0xb72b4c50 <+0>:    push   %ebp
>    0xb72b4c51 <+1>:    mov    %esp,%ebp
>    0xb72b4c53 <+3>:    push   %edi
>    0xb72b4c54 <+4>:    push   %esi
>    0xb72b4c55 <+5>:    sub    $0xc,%esp
>    0xb72b4c58 <+8>:    mov    0x8(%ebp),%ecx
>    0xb72b4c5b <+11>:    test   %ecx,%ecx
>    0xb72b4c5d <+13>:    jne    0xb72b4c68 <__gss_get_mechanism_cred+24>
>    0xb72b4c5f <+15>:    add    $0xc,%esp
>    0xb72b4c62 <+18>:    xor    %eax,%eax
>    0xb72b4c64 <+20>:    pop    %esi
>    0xb72b4c65 <+21>:    pop    %edi
>    0xb72b4c66 <+22>:    pop    %ebp
>    0xb72b4c67 <+23>:    ret
>    0xb72b4c68 <+24>:    mov    0x8(%ebp),%eax
>    0xb72b4c6b <+27>:    mov    (%eax),%eax
>    0xb72b4c6d <+29>:    test   %eax,%eax
>    0xb72b4c6f <+31>:    mov    %eax,-0x10(%ebp)
>    0xb72b4c72 <+34>:    jle    0xb72b4c5f <__gss_get_mechanism_cred+15>
>    0xb72b4c74 <+36>:    mov    0xc(%ebp),%ecx
>    0xb72b4c77 <+39>:    mov    0x8(%ebp),%eax
>    0xb72b4c7a <+42>:    mov    (%ecx),%edx
>    0xb72b4c7c <+44>:    mov    0x4(%eax),%eax
>    0xb72b4c7f <+47>:    mov    %edx,-0x14(%ebp)
>    0xb72b4c82 <+50>:    mov    %eax,-0xc(%ebp)
>    0xb72b4c85 <+53>:    xor    %eax,%eax
>    0xb72b4c87 <+55>:    nop
>    0xb72b4c88 <+56>:    mov    -0xc(%ebp),%edx
>    0xb72b4c8b <+59>:    mov    -0x14(%ebp),%ecx
> => 0xb72b4c8e <+62>:    cmp    (%edx,%eax,8),%ecx
>    0xb72b4c91 <+65>:    jne    0xb72b4cab <__gss_get_mechanism_cred+91>
>    0xb72b4c93 <+67>:    mov    0xc(%ebp),%edx
>    0xb72b4c96 <+70>:    mov    -0xc(%ebp),%ecx
>    0xb72b4c99 <+73>:    mov    0x4(%edx),%esi
>    0xb72b4c9c <+76>:    mov    -0x14(%ebp),%edx
> ---Type <return> to continue, or q <return> to quit---q
> Quit
> (gdb) info r
> eax            0x0    0
> ecx            0x9    9
> edx            0x0    0
> ebx            0xb72b9710    -1221880048
> esp            0xbfe0b844    0xbfe0b844
> ebp            0xbfe0b858    0xbfe0b858
> esi            0x9    9
> edi            0x9370928    154601768
> eip            0xb72b4c8e    0xb72b4c8e <__gss_get_mechanism_cred+62>
> eflags         0x210246    [ PF ZF IF RF ID ]
> cs             0x73    115
> ss             0x7b    123
> ds             0x7b    123
> es             0x7b    123
> fs             0x0    0
> gs             0x33    51
> 
> 
> Where you can see the null reference.
> 
> This, on the other hand, reminded me a discussion I had in the
> fbsd-stable list, where somebody advised me to use des-cbc-crc
> keytabs, and then experiment with other encryption types. Even
> though I didn't have such issues in fbsd, this might be the case on
> this problem. Maybe linux tries to compare an inexistent mech-type
> (or enc-type, whatever) with one of it's list, and hence it
> segfaults. So, to shed a little more light into the issue:
> 
> - the kdc (not the nfs-server) runs fbsd8, heimdal-1.2.1.
> - on the linuxbox when I "kinit -k linuxclient" everything works fine.
> - on the linux client "ktutil -k /etc/krb5.keytab list" shows:
> /etc/krb5.keytab:
> 
> Vno  Type                     Principal
> Aliases
>   1  des-cbc-md5              host/linuxclient@EXAMPLE
>   1  des-cbc-md4              host/linuxclient@EXAMPLE
>   1  des-cbc-crc              host/linuxclient@EXAMPLE
>   1  aes256-cts-hmac-sha1-96  host/linuxclient@EXAMPLE
>   1  des3-cbc-sha1            host/linuxclient@EXAMPLE
>   1  arcfour-hmac-md5         host/linuxclient@EXAMPLE
> 
> So maybe what I have to do next is to create a keytab of des-cbc-crc
> encryption-type only for the client (maybe also for the nfsserver as
> well..) and see how it'll behave...but I will do it tomorrow.

OK, thanks for the investigation so far!

Unfortunately, I don't know who's going to follow up on this soon, so
you may also want to get out the source that was built from, try to
figure exactly what line the NULL dereference is happening on, which
caller passed in the bad data, etc.

--b.

Re: nfsv3 gssapi client?

[George Mamalakis]

  Hi guys,

Kevin:
As stated in my first email, the linux box is running on a 2.6.35 
kernel, heimdal 1.3.3-1 (the version was not mentioned in my email),  
rpcbind-0.2.0-2, nfs-utils-1.2.2-3, and the distro is arch-linux.

Bruce: I would rather find an "easier" (configuration-specific) way to 
overcome my problem instead of delving into the code, since I am not a 
developer (I am an admin :)). Nevertheless, when I used 
google.com/searchcode for _gss_get_mechanism_cred I found that there is 
a libgss/g_glue.c that defines this function in the following way (the 
site is svn://svn.genunix.org/on/trunk 
<http://google.com/codesearch/p?hl=en#m0BoFGhUuEY/&q=_gss_get_mechanism_cred>):

/*
  * Glue routine for returning the mechanism-specific credential from a
  * external union credential.
  */
gss_cred_id_t
__gss_get_mechanism_cred(union_cred, mech_type)
         const  gss_union_cred_t  union_cred;
         const  gss_OID           mech_type;
{
         int                      i;

         if  (union_cred == (gss_union_cred_t)GSS_C_NO_CREDENTIAL)
                 return  (GSS_C_NO_CREDENTIAL);

         for  (i =0; i<  union_cred->count; i++) {
                 if  (g_OID_equal(mech_type,&union_cred->mechs_array[i]))
                         return  (union_cred->cred_array[i]);
         }
         return  (GSS_C_NO_CREDENTIAL);
}


where you can see that the guy returns a cred_array[i] element, that 
obviously is null, because (most probably) the if statement above did 
not match any "compatible" mechanisms during the recursion.

Maybe I am totally wrong, since I haven't checked the code thoroughly, 
but I think that the problem is located somewhere in this code segment.

As I stated before, I was rather hoping to avoid this procedure, since I 
decided to use the linux distro for my clients due to its 
'out-of-the-box' and 'everything-is-supported' benefits as 
(slightly-)opposed to fbsd (which I use in all my servers and personal 
computers, and "love" as an operating system) can be a bit more 
"demanding" when out-of-the-box-general-purpose issues are involved. The 
thing is that the client will be used as a general-purpose student pc in 
a university lab.

Thanx again for your help,

if anybody could direct me somewhere else I would be delighted to know so!

mamalos

-- 
George Mamalakis

IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)

Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki

phone number : +30 (2310) 994379

Re: nfsv3 gssapi client?

[J. Bruce Fields]

On Tue, Sep 21, 2010 at 11:36:18AM +0300, George Mamalakis wrote:
>  Hi guys,
> 
> Kevin:
> As stated in my first email, the linux box is running on a 2.6.35
> kernel, heimdal 1.3.3-1 (the version was not mentioned in my email),

As Kevin says, it would be worth trying MIT kerberos instead of heimdal
if you can, as that is generally better tested.

--b.

> rpcbind-0.2.0-2, nfs-utils-1.2.2-3, and the distro is arch-linux.
> 
> Bruce: I would rather find an "easier" (configuration-specific) way
> to overcome my problem instead of delving into the code, since I am
> not a developer (I am an admin :)). Nevertheless, when I used
> google.com/searchcode for _gss_get_mechanism_cred I found that there
> is a libgss/g_glue.c that defines this function in the following way
> (the site is svn://svn.genunix.org/on/trunk <http://google.com/codesearch/p?hl=en#m0BoFGhUuEY/&q=_gss_get_mechanism_cred>):
> 
> /*
>  * Glue routine for returning the mechanism-specific credential from a
>  * external union credential.
>  */
> gss_cred_id_t
> __gss_get_mechanism_cred(union_cred, mech_type)
>         const  gss_union_cred_t  union_cred;
>         const  gss_OID           mech_type;
> {
>         int                      i;
> 
>         if  (union_cred == (gss_union_cred_t)GSS_C_NO_CREDENTIAL)
>                 return  (GSS_C_NO_CREDENTIAL);
> 
>         for  (i =0; i<  union_cred->count; i++) {
>                 if  (g_OID_equal(mech_type,&union_cred->mechs_array[i]))
>                         return  (union_cred->cred_array[i]);
>         }
>         return  (GSS_C_NO_CREDENTIAL);
> }
> 
> 
> where you can see that the guy returns a cred_array[i] element, that
> obviously is null, because (most probably) the if statement above
> did not match any "compatible" mechanisms during the recursion.
> 
> Maybe I am totally wrong, since I haven't checked the code
> thoroughly, but I think that the problem is located somewhere in
> this code segment.
> 
> As I stated before, I was rather hoping to avoid this procedure,
> since I decided to use the linux distro for my clients due to its
> 'out-of-the-box' and 'everything-is-supported' benefits as
> (slightly-)opposed to fbsd (which I use in all my servers and
> personal computers, and "love" as an operating system) can be a bit
> more "demanding" when out-of-the-box-general-purpose issues are
> involved. The thing is that the client will be used as a
> general-purpose student pc in a university lab.
> 
> Thanx again for your help,
> 
> if anybody could direct me somewhere else I would be delighted to know so!
> 
> mamalos
> 
> -- 
> George Mamalakis
> 
> IT Officer
> Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
> MSc (Imperial College of London)
> 
> Department of Electrical and Computer Engineering
> Faculty of Engineering
> Aristotle University of Thessaloniki
> 
> phone number : +30 (2310) 994379
> 

Re: nfsv3 gssapi client?

[George Mamalakis]

  I did some more testing, so I run a new kdc (heimdal again) on the 
fbsd server (just to eliminate one candidate from my problematic-boxes); 
this one is from the fbsd base system (the previous one was from ports), 
and the version is 1.1.0. I init'd my realm, and changed the 
/etc/krb.conf files on server and client to reflect my new kerberos 
implementation.

When I tried to mount the remote share using sec=krb5, gssd segfaulted 
again; I was on the exact place as before. Then I started deleting 
encryption types from the keytab (using kadmin and del_enctype) to see 
if this would help, until only one encryption type was left 
(des3-cbc-sha1), but gssd segfaulted repeatedly. Two funny things that I 
noticed:

1) when I del_enctype'd des3-cbc-sha1 and tried to kinit -t keytab -k 
host/linclient on the linuxbox I got the error:
[root@linuxclient ]# kinit -t keytab -k host/linclient
kinit: krb5_get_init_creds: Looping 11 times while getting initial 
credentials
[root@linuxclient ]#

and on the server tail /var/heimdal/kdc.log showed:

2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from 
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED -- 
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from 
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED -- 
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from 
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED -- 
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from 
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED -- 
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from 
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED -- 
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from 
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED -- 
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from 
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED -- 
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from 
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED -- 
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from 
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED -- 
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from 
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED -- 
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from 
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED -- 
host/linclient@EXAMPLE

which means that there is
a) some strange "incompatibility" issue with the keytabs provided by 
fbsd's heimdal version and the one that linux ships with.
b) the way linux kinit renders a keytab file becomes "problematic" when 
des3-cbc-sha1 is absent (on fbsd kinit worked fine with the same 
keytab). Haven't setup a linux heimdal server (yet...) to clear this.
but this is a complete different discussion.

The thing is that gssd creates it's krb5 cache in /tmp without any 
issues, and dies only after trying to "create context with the server". 
This is illustrated below, where gssd is run manually with verbose flags:

destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt49
handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt4b)
handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt4b)
process_krb5_upcall: service is '<null>'
Full hostname for 'filesrv' is 'filesrv'
Full hostname for 'linclient' is 'linclient'
Failed to find root/linclient@EXAMPLE in keytab FILE:/etc/krb5.keytab 
(null) while getting keytab entry for 'root/linclient@EXAMPLE'
Failed to find nfs/linclient@EXAMPLE in keytab FILE:/etc/krb5.keytab 
(null) while getting keytab entry for 'nfs/linclient@EXAMPLE'
Success getting keytab entry for 'host/linclient@EXAMPLE'
Successfully obtained machine credentials for principal 
'host/linclient@EXAMPLE' stored in ccache 'FILE:/tmp/krb5cc_machine_EXAMPLE'
INFO: Credentials in CC 'FILE:/tmp/krb5cc_machine_EXAMPLE' are good 
until 1285112422
using FILE:/tmp/krb5cc_machine_EXAMPLE as credentials cache for machine 
creds
using gss_krb5_ccache_name to select krb5 ccache 
FILE:/tmp/krb5cc_machine_EXAMPLE
creating context using fsuid 0 (save_uid 0)
creating tcp client for server filesrv
DEBUG: port already set to 2049
creating context with server nfs@filesrv
Segmentation fault

The cache in /tmp/krb5cc_machine_EXAMPLE file is ok, since I am able to 
klist -c /tmp/krb5cc_machine_EXAMPLE and read its content. The problem 
comes after that step, when the client tries to find available 
mechanisms (stated in my last email).

2) The second funny thing was that when I searched the list for relevant 
problems I found a mail I had sent on the 5th March 2010, where I was 
facing some analogous problems, only this time with nfsv4 and a solaris 
nfs server. The subject was "mount.nfs4: Broken pipe", and from my 
"investigation" that far, I saw that there had to be a mix with MIT and 
heimdal on arch-linux. I think that this must be the case now, since 
when I read /etc/gssapi_mech.conf, I realized that it discussed about 
MIT kerberos5, and /usr/lib/libgssapi.so must be relevant to MIT too. 
But pacman -Qo /usr/lib/libgssapi.so showed that it is installed by 
heimdal 1.3.3-1...

...I don't know, I am confused! Is there a config 
(/etc/conf.d/nfs-common.conf, relevant packages, etc.) with which 
someone is able to mount nfsv3 shares using sec=krb5? If anyone is able 
to shed some light, it would be reaaally helpful.

Regards,

mamalos

-- 
George Mamalakis

IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)

Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki

phone number : +30 (2310) 994379

Re: nfsv3 gssapi client?

[George Mamalakis]

  On 21/09/2010 16:51, J. Bruce Fields wrote:
> On Tue, Sep 21, 2010 at 11:36:18AM +0300, George Mamalakis wrote:
>>   Hi guys,
>>
>> Kevin:
>> As stated in my first email, the linux box is running on a 2.6.35
>> kernel, heimdal 1.3.3-1 (the version was not mentioned in my email),
> As Kevin says, it would be worth trying MIT kerberos instead of heimdal
> if you can, as that is generally better tested.
>
> --b.
>
>> rpcbind-0.2.0-2, nfs-utils-1.2.2-3, and the distro is arch-linux.
>>
>> Bruce: I would rather find an "easier" (configuration-specific) way
>> to overcome my problem instead of delving into the code, since I am
>> not a developer (I am an admin :)). Nevertheless, when I used
>> google.com/searchcode for _gss_get_mechanism_cred I found that there
>> is a libgss/g_glue.c that defines this function in the following way
>> (the site is svn://svn.genunix.org/on/trunk<http://google.com/codesearch/p?hl=en#m0BoFGhUuEY/&q=_gss_get_mechanism_cred>):
>>
>> /*
>>   * Glue routine for returning the mechanism-specific credential from a
>>   * external union credential.
>>   */
>> gss_cred_id_t
>> __gss_get_mechanism_cred(union_cred, mech_type)
>>          const  gss_union_cred_t  union_cred;
>>          const  gss_OID           mech_type;
>> {
>>          int                      i;
>>
>>          if  (union_cred == (gss_union_cred_t)GSS_C_NO_CREDENTIAL)
>>                  return  (GSS_C_NO_CREDENTIAL);
>>
>>          for  (i =0; i<   union_cred->count; i++) {
>>                  if  (g_OID_equal(mech_type,&union_cred->mechs_array[i]))
>>                          return  (union_cred->cred_array[i]);
>>          }
>>          return  (GSS_C_NO_CREDENTIAL);
>> }
>>
>>
>> where you can see that the guy returns a cred_array[i] element, that
>> obviously is null, because (most probably) the if statement above
>> did not match any "compatible" mechanisms during the recursion.
>>
>> Maybe I am totally wrong, since I haven't checked the code
>> thoroughly, but I think that the problem is located somewhere in
>> this code segment.
>>
>> As I stated before, I was rather hoping to avoid this procedure,
>> since I decided to use the linux distro for my clients due to its
>> 'out-of-the-box' and 'everything-is-supported' benefits as
>> (slightly-)opposed to fbsd (which I use in all my servers and
>> personal computers, and "love" as an operating system) can be a bit
>> more "demanding" when out-of-the-box-general-purpose issues are
>> involved. The thing is that the client will be used as a
>> general-purpose student pc in a university lab.
>>
>> Thanx again for your help,
>>
>> if anybody could direct me somewhere else I would be delighted to know so!
>>
>> mamalos
>>
>> -- 
>> George Mamalakis
>>
>> IT Officer
>> Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
>> MSc (Imperial College of London)
>>
>> Department of Electrical and Computer Engineering
>> Faculty of Engineering
>> Aristotle University of Thessaloniki
>>
>> phone number : +30 (2310) 994379
>>
Ok Bruce and Kevin,

I'll try and do that. I only hope that MIT-client is compatible with 
Heimdal-server (if I manage to install it on archlinux through AUR...:)).

-- 
George Mamalakis

IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)

Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki

phone number : +30 (2310) 994379