Debugging hung autofs /home?

Debugging hung autofs /home?

[Jim Davis]

[root@lectura /]# ps ax | grep auto
 2709 ?        Ss     0:00 /usr/sbin/automount --timeout=3600 --verbose 
--debug /home file /etc/auto_home -rw,intr,nosuid
 2763 ?        Ss     0:00 /usr/sbin/automount --timeout=3600 --verbose 
--debug /cs file /etc/auto_cs -rw,intr,nosuid
 3776 ?        S      0:00 kalarmd --autostart
21284 pts/16   S+     0:00 grep automount
21827 pts/28   S+     0:00 grep auto
[root@lectura /]# kill -HUP 2709
[root@lectura /]# strace -f -p 2709
Process 2709 attached - interrupt to quit
futex(0x2aaaaaefd840, FUTEX_WAIT, 2, NULL

That process doesn't respond to kill -HUP, USR1, USR2,  or /sbin/service 
autofs reload.  (The other automount process associated with /cs seems 
to be fine.)   New mounts under /home just hang.  Mounting the 
directories by hand from the NFS server works.

This is on a 4-processor AMD system running FC4:

Linux lectura.cs.arizona.edu 2.6.14-1.1656_FC4smp #1 SMP Thu Jan 5 
22:26:33 EST 2006 x86_64 x86_64 x86_64 GNU/Linux

Re: Debugging hung autofs /home?

[Jeff Moyer]

==> Regarding [autofs] Debugging hung autofs /home?; Jim Davis <jdavis@cs.arizona.edu> adds:

jdavis> [root@lectura /]# ps ax | grep auto
jdavis>  2709 ?        Ss     0:00 /usr/sbin/automount --timeout=3600 --verbose 
jdavis> --debug /home file /etc/auto_home -rw,intr,nosuid
jdavis>  2763 ?        Ss     0:00 /usr/sbin/automount --timeout=3600 --verbose 
jdavis> --debug /cs file /etc/auto_cs -rw,intr,nosuid
jdavis>  3776 ?        S      0:00 kalarmd --autostart
jdavis> 21284 pts/16   S+     0:00 grep automount
jdavis> 21827 pts/28   S+     0:00 grep auto
jdavis> [root@lectura /]# kill -HUP 2709
jdavis> [root@lectura /]# strace -f -p 2709
jdavis> Process 2709 attached - interrupt to quit
jdavis> futex(0x2aaaaaefd840, FUTEX_WAIT, 2, NULL

jdavis> That process doesn't respond to kill -HUP, USR1, USR2,  or /sbin/service 
jdavis> autofs reload.  (The other automount process associated with /cs seems 
jdavis> to be fine.)   New mounts under /home just hang.  Mounting the 
jdavis> directories by hand from the NFS server works.

jdavis> This is on a 4-processor AMD system running FC4:

jdavis> Linux lectura.cs.arizona.edu 2.6.14-1.1656_FC4smp #1 SMP Thu Jan 5 
jdavis> 22:26:33 EST 2006 x86_64 x86_64 x86_64 GNU/Linux

And you left off the version of autofs you're running.  Most likely it's
older, and does not have the reentrant syslog patches.  I think an upgrade
will fix your wagon.  ;)

-Jeff

Re: Debugging hung autofs /home?

[Jim Davis]

Jeff Moyer wrote:

>==> Regarding [autofs] Debugging hung autofs /home?; Jim Davis <jdavis@cs.arizona.edu> adds:
>
>And you left off the version of autofs you're running. 
>
Ooops; autofs-4.1.4-5.  'yum update' doesn't seem to find a newer one.

Re: Debugging hung autofs /home?

[Jeff Moyer]

==> Regarding Re: [autofs] Debugging hung autofs /home?; Jim Davis <jdavis@CS.Arizona.EDU> adds:

jdavis> Jeff Moyer wrote:
>> ==> Regarding [autofs] Debugging hung autofs /home?; Jim Davis <jdavis@cs.arizona.edu> adds:
>> 
>> And you left off the version of autofs you're running. 
>> 
jdavis> Ooops; autofs-4.1.4-5.  'yum update' doesn't seem to find a newer one.

Interesting.  Then I'd suggest you provide the information requested in the
"Filing bug reports" section of my people page:

	http://people.redhat.com/jmoyer/

We previously saw automount processes hung in FUTEX_WAIT when making syslog
calls from a signal handler (a definite no no).  This has been fixed in the
version of autofs that you are running.  So, we'll need more information to
debug this one.

Thanks,

Jeff

Re: Debugging hung autofs /home?

[Jim Davis]

Jeff Moyer wrote:

>
>ewer one.
>
>Interesting.  Then I'd suggest you provide the information requested in the
>"Filing bug reports" section of my people page:
>
>	http://people.redhat.com/jmoyer/
>
The somewhat lengthy file from the latest hang is at 
http://www.cs.arizona.edu/people/jdavis/autofsbug.  If there's anything 
else I can provide, let me know.  I have installed the 4-1-4.8 rpms, and 
have strace running on the automount process controlling /home.

Re: Debugging hung autofs /home?

[Jeff Moyer]

==> Regarding Re: [autofs] Debugging hung autofs /home?; Jim Davis <jdavis@CS.Arizona.EDU> adds:

jdavis> Jeff Moyer wrote:
>> 
>> ewer one.
>> 
>> Interesting.  Then I'd suggest you provide the information requested in the
>> "Filing bug reports" section of my people page:
>> 
>> http://people.redhat.com/jmoyer/
>> 
jdavis> The somewhat lengthy file from the latest hang is at 
jdavis> http://www.cs.arizona.edu/people/jdavis/autofsbug.  If there's anything 
jdavis> else I can provide, let me know.  I have installed the 4-1-4.8 rpms, and 
jdavis> have strace running on the automount process controlling /home.

Turns out Ian is right.  The syslog fixes didn't go in until 4.1.4-8 (and
those fixes were a different implementation that what was committed in
4.1.4-10).  So, 4.1.4-10 is your best bet.  I'll see about updating the
Fedora packages soonish.

-Jeff