Re: automount dies almost immediately after upgrade to autofs 5.0.1

[Dan Krejsa <dan.krejsa@alamedanet.net>]

Hi Ian,

Yes, in fact, there are multiple core.* files in / which
I didn't notice.  I can send you one of them if you like.
I checked with gdb and they are from automount.

[dlkrejsa@clifford /]$ gdb -c core.3805 /usr/sbin/automount
GNU gdb Red Hat Linux (6.5-15.fc6rh)
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...
(no debugging symbols found)
Using host libthread_db library "/lib/libthread_db.so.1".


warning: Can't read pathname for load map: Input/output error.
(no debugging symbols found)
Loaded symbols for /usr/sbin/automount
Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/libgcc_s.so.1...
(no debugging symbols found)...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /usr/lib/autofs/lookup_file.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/autofs/lookup_file.so
Reading symbols from /lib/libnsl.so.1...
(no debugging symbols found)...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /usr/lib/autofs/parse_sun.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/autofs/parse_sun.so
Reading symbols from /usr/lib/autofs/mount_nfs.so...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/autofs/mount_nfs.so
Reading symbols from /usr/lib/autofs/mount_bind.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/autofs/mount_bind.so
Reading symbols from /usr/lib/autofs/lookup_hosts.so...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/autofs/lookup_hosts.so
Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_dns.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_dns.so.2
Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libresolv.so.2
Failed to read a valid object file image from memory.

Core was generated by `automount'.
Program terminated with signal 6, Aborted.
#0  0x00739402 in ?? ()
(gdb) tt
Argument required (filename to set it to.).
(gdb) bt
#0  0x00739402 in ?? ()
#1  0x008afd40 in raise () from /lib/libc.so.6
#2  0x008b1591 in abort () from /lib/libc.so.6
#3  0x008e533b in __libc_message () from /lib/libc.so.6
#4  0x00968361 in __chk_fail () from /lib/libc.so.6
#5  0x00967817 in __strcat_chk () from /lib/libc.so.6
#6  0x0064aaad in rpc_time () from /usr/lib/autofs/lookup_hosts.so
#7  0x0064adce in rpc_time () from /usr/lib/autofs/lookup_hosts.so
#8  0x0064aff7 in rpc_exports_prune () from /usr/lib/autofs/lookup_hosts.so
#9  0x00646343 in lookup_mount () from /usr/lib/autofs/lookup_hosts.so
#10 0x80012409 in do_lookup_mount () from /usr/sbin/automount
#11 0x800133c6 in lookup_nss_mount () from /usr/sbin/automount
#12 0x8000b3d2 in expire_proc_indirect () from /usr/sbin/automount
#13 0x00d8d3db in start_thread () from /lib/libpthread.so.0
#14 0x0095426e in clone () from /lib/libc.so.6
(gdb) threads
Undefined command: "threads".  Try "help".
(gdb) help
List of classes of commands:

aliases -- Aliases of other commands
breakpoints -- Making program stop at certain points
data -- Examining data
files -- Specifying and examining files
internals -- Maintenance commands
obscure -- Obscure features
running -- Running the program
stack -- Examining the stack
status -- Status inquiries
support -- Support facilities
tracepoints -- Tracing of program execution without stopping the program
user-defined -- User-defined commands

Type "help" followed by a class name for a list of commands in that class.
Type "help" followed by command name for full documentation.
Command name abbreviations are allowed if unambiguous.
(gdb) help status
Status inquiries.

List of commands:

info -- Generic command for showing things about the program being debugged
macro -- Prefix for commands dealing with C preprocessor macros
show -- Generic command for showing things about the debugger

Type "help" followed by command name for full documentation.
Command name abbreviations are allowed if unambiguous.
(gdb) i th
  6 process 3805  0x00739402 in ?? ()
  5 process 3806  0x00739402 in ?? ()
  4 process 3807  0x00739402 in ?? ()
  3 process 3810  0x00739402 in ?? ()
  2 process 3813  0x00739402 in ?? ()
* 1 process 3841  0x00739402 in ?? ()
(gdb) th 6
Ambiguous command "th 6": thbreak, thread.
(gdb) thread 6
[Switching to thread 6 (process 3805)]#0  0x00739402 in ?? ()
(gdb) bt
#0  0x00739402 in ?? ()
#1  0x00d94ebe in do_sigwait () from /lib/libpthread.so.0
#2  0x00d94f5f in sigwait () from /lib/libpthread.so.0
#3  0x80005838 in _start () from /usr/sbin/automount
(gdb) thread 5
[Switching to thread 5 (process 3806)]#0  0x00739402 in ?? ()
(gdb) bt
#0  0x00739402 in ?? ()
#1  0x00d9142c in pthread_cond_timedwait@@GLIBC_2.3.2 ()
   from /lib/libpthread.so.0
#2  0x8001c8f6 in alarm_delete () from /usr/sbin/automount
#3  0x00d8d3db in start_thread () from /lib/libpthread.so.0
#4  0x0095426e in clone () from /lib/libc.so.6
(gdb) thread 4
[Switching to thread 4 (process 3807)]#0  0x00739402 in ?? ()
(gdb) bt
#0  0x00739402 in ?? ()
#1  0x00d9142c in pthread_cond_timedwait@@GLIBC_2.3.2 ()
   from /lib/libpthread.so.0
#2  0x8001626b in expire_cleanup () from /usr/sbin/automount
#3  0x00d8d3db in start_thread () from /lib/libpthread.so.0
#4  0x0095426e in clone () from /lib/libc.so.6
(gdb) thread 3
[Switching to thread 3 (process 3810)]#0  0x00739402 in ?? ()
(gdb) bt
#0  0x00739402 in ?? ()
#1  0x0094a6d3 in poll () from /lib/libc.so.6
#2  0x80008384 in handle_mounts () from /usr/sbin/automount
#3  0x00d8d3db in start_thread () from /lib/libpthread.so.0
#4  0x0095426e in clone () from /lib/libc.so.6
(gdb) thread 2
[Switching to thread 2 (process 3813)]#0  0x00739402 in ?? ()
(gdb) bt
#0  0x00739402 in ?? ()
#1  0x0094a6d3 in poll () from /lib/libc.so.6
#2  0x80008384 in handle_mounts () from /usr/sbin/automount
#3  0x00d8d3db in start_thread () from /lib/libpthread.so.0
#4  0x0095426e in clone () from /lib/libc.so.6
(gdb) thread 1
[Switching to thread 1 (process 3841)]#0  0x00739402 in ?? ()
(gdb) bt
#0  0x00739402 in ?? ()
#1  0x008afd40 in raise () from /lib/libc.so.6
#2  0x008b1591 in abort () from /lib/libc.so.6
#3  0x008e533b in __libc_message () from /lib/libc.so.6
#4  0x00968361 in __chk_fail () from /lib/libc.so.6
#5  0x00967817 in __strcat_chk () from /lib/libc.so.6
#6  0x0064aaad in rpc_time () from /usr/lib/autofs/lookup_hosts.so
#7  0x0064adce in rpc_time () from /usr/lib/autofs/lookup_hosts.so
#8  0x0064aff7 in rpc_exports_prune () from /usr/lib/autofs/lookup_hosts.so
#9  0x00646343 in lookup_mount () from /usr/lib/autofs/lookup_hosts.so
#10 0x80012409 in do_lookup_mount () from /usr/sbin/automount
#11 0x800133c6 in lookup_nss_mount () from /usr/sbin/automount
#12 0x8000b3d2 in expire_proc_indirect () from /usr/sbin/automount
#13 0x00d8d3db in start_thread () from /lib/libpthread.so.0
#14 0x0095426e in clone () from /lib/libc.so.6
(gdb) 

I'll try out the rc3.25 version sometime...

Thanks,


- Dan

On Wed, 2007-03-07 at 10:09 +0900, Ian Kent wrote:
> On Tue, 2007-03-06 at 10:29 -0800, Krejsa, Dan wrote:
> > Hi Ian & Folks,
> > 
> > I've included the debug log and other information asked for
> > below, and attached as separate files in an archive.
> > 
> > To collect the /var/log/debug output, I restarted autofs
> > as root, and in another shell as an ordinary user, did
> 
> Thanks.
> 
> > 
> >   cd /net
> >   cd blue
> > 
> > The 'cd blue' call hanged, and the automount died (
> > according to /etc/init.d/autofs status) after
> > fetching the export list for blue. (I provide the output
> > of 'showmount -e blue' below.)  After a bit I was able
> > to interrupt the 'cd blue' call, getting back a 'No such
> > file or directory' message.  I then restarted autofs
> > again (/etc/init.d/autofs restart), and again tried
> > 'cd blue' with the same results.
> > 
> > In the /var/log/debug output I see
> > 
> > Mar  6 09:23:52 localhost automount[3579]: lookup_read_master:
> > lookup(yp): read of master map auto.master failed: Can't bind to server
> > which serves this domain
> > Mar  6 09:23:52 localhost automount[3579]: lookup_read_master:
> > lookup(file): failed to read included master map auto.master
> 
> Map includes are only allowed in file maps, this is the return from the
> attempted include of "+auto.master".
> 
> > 
> > I'm not too surprised about the yp lookup failure, but I'm
> > not sure what to make of the lookup(file) failure, or even
> > whether this is to be worried about for /net mounts.
> > 
> > Thanks for any suggestions,
> > 
> > - Dan
> > 
> > [root@clifford etc]# uname -a
> > Linux clifford 2.6.19-1.2911.6.4.fc6 #1 SMP Sat Feb 24 14:39:04 EST 2007
> > i686 i686 i386 GNU/Linux
> > [root@clifford etc]# rpm -q autofs
> > autofs-5.0.1-0.rc3.23
> > [root@clifford etc]# lf -l auto.*
> > -rw-r--r-- 1 root root  560 Feb 19 20:39 auto.master
> > -rw-r--r-- 1 root root  581 Feb 19 20:39 auto.misc
> > -rwxr-xr-x 1 root root 1292 Feb 19 20:39 auto.net*
> > -rwxr-xr-x 1 root root  715 Feb 19 20:39 auto.smb*
> > [root@clifford etc]# cat auto.master
> > #
> > # $Id: auto.master,v 1.4 2005/01/04 14:36:54 raven Exp $
> > #
> > # Sample auto.master file
> > # This is an automounter map and it has the following format
> > # key [ -mount-options-separated-by-comma ] location
> > # For details of the format look at autofs(5).
> > #
> > /misc   /etc/auto.misc
> > /net    -hosts
> > #
> > # Include central master map if it can be found using
> > # nsswitch sources.
> > #
> > # Note that if there are entries for /net or /misc (as
> > # above) in the included master map any keys that are the
> > # same will not be seen as the first read key seen takes
> > # precedence.
> > #
> > +auto.master
> > [root@clifford etc]# cat auto.net
> 
> The auto.net script isn't used for the "-hosts" map.
> 
> snip ...
> 
> > [root@clifford etc]# showmount -e blue
> > Export list for blue:
> > /vol/vol0/fsfcvs                  (everyone)
> > /vol/vol0/sww                     (everyone)
> > /vol/vol5/prj-prlc-work-products  (everyone)
> > /vol/vol7/prj-2007-fall-release   (everyone)
> > /vol/vol0/tina                    (everyone)
> > /vol/vol0/software                (everyone)
> > /vol/vol0/rti_tools               (everyone)
> > /vol/vol7/prj-2007-hut-dev        (everyone)
> > /vol/vol1/dave-test               ala-doradb2
> > /vol/vol0/sales_video             (everyone)
> > /vol/vol0/jumpstart               (everyone)
> > /vol/vol5/ilt                     (everyone)
> > /vol/vol3/prj-2006-fall-release   (everyone)
> > /vol/vol2/prj-2006-spring-release (everyone)
> > /vol/vol0/jumpstart/redhat/BSD    (everyone)
> > /vol/vol0/vmware-test             (everyone)
> > /vol/vol5/tux                     (everyone)
> > /vol/vol0/ultravox                (everyone)
> > /vol/vol0/ecms                    web-ecms,cadacan,morpheus,wisdom
> > /vol/vol0/docs                    (everyone)
> > /vol/vol0                         morpheus
> > /vol/vol1                         morpheus
> > /vol/vol2                         morpheus
> > /vol/vol3                         morpheus
> > /vol/vol4                         morpheus,ala-toradb2,ala-doradb2
> > /vol/vol5                         morpheus
> > /vol/vol6                         (everyone)
> > /vol/vol7                         (everyone)
> > /vol/vol8                         (everyone)
> > /vol/vol7/prj-2007-hut-release    (everyone)
> > /vol/vol0/web                     (everyone)
> > /vol/vol1/it_laptop_backup        (everyone)
> > /vol/vol6/prj-2007-spring-release (everyone)
> > /vol/vol7/prj-2007-fall-dev       (everyone)
> > /vol/vol0/ithome                  (everyone)
> 
> Looks fine.
> 
> > [root@clifford etc]#
> > [root@clifford etc]# cd /var/log
> > [root@clifford log]# cat debug
> > Mar  6 09:19:31 localhost automount[3453]: Starting automounter version
> > 5.0.1-0.rc3.23, master map auto.master
> 
> There is a correction to the exports lookup code in 0.rc3.25 which you
> should find in the testing directory on the mirrors. I've just requested
> it be moved from testing to final so it should be available as an update
> soon.
> 
> snip ...
> 
> > Mar  6 09:23:53 localhost automount[3579]: mounted indirect mount on
> > /net with timeout 300, freq 75 seconds
> > Mar  6 09:24:01 localhost automount[3579]: handle_packet: type = 3
> > Mar  6 09:24:01 localhost automount[3579]:
> > handle_packet_missing_indirect: token 3, name blue, request pid 3500
> > Mar  6 09:24:01 localhost automount[3579]: attempting to mount entry
> > /net/blue
> > Mar  6 09:24:01 localhost automount[3579]: lookup_mount: lookup(hosts):
> > fetchng export list for blue
> 
> Strange you don't get a SEGV and a core.
> Do you see any core file in the root directory?
> 
> Ian
> 
>