[linux-lvm] Cleaning up stale handles in /etc/mtab

[linux-lvm] Cleaning up stale handles in /etc/mtab

[Brown, Rodrick R]

Not sure what exactly happened on this server but it looks like the LUNS
were pulled w/o removing the volumes correctly or something of that
sort.  

# df 
df: `/vob/gcib/its': Too many open files in system
df: `/vob/lftech/artefacts': Too many open files in system
df: `/vob/rmt/mktrisk_portal': Too many open files in system
df: `/vob/gcib/gfx_fxpb_client_facing': Too many open files in system
df: `/vob/gcib/gfx_fxpb_workflow': Too many open files in system
df: `/vob/rmt/credittrend': Too many open files in system
df: `/vob/adv/re_reporting': Too many open files in system
df: `/vob/datamgmt/atlas_p2': Too many open files in system
df: `/vob/gcib/gfx_rad': Too many open files in system
...
...

Does anyone know of a method to fix this without rebooting the box
thanks? 

Thanks. 

---
Rodrick R. Brown 
UNIX Platform Support (SME)
Banc of America Securities LLC. 
Global Trading Infrastructure (GCIB)
100 West 33rd ST. 3rd Flr. 
New York, NY 10001
Mail Code: NY1-509-03-18
Office: 646 733 4473
Cell: 646-261-5286

[linux-lvm] Re: Cleaning up stale handles in /etc/mtab

[Matthew Galgoci]

> Date: Fri, 01 Sep 2006 11:13:41 -0400
> From: "Brown, Rodrick R" <rodrick.r.brown@bofasecurities.com>
> Reply-To: redhat-sysadmin-list@redhat.com
> To: redhat-sysadmin-list@redhat.com
> Cc: linux-lvm@redhat.com
> Subject: Cleaning up stale handles in /etc/mtab
>
> Not sure what exactly happened on this server but it looks like the LUNS
> were pulled w/o removing the volumes correctly or something of that
> sort.

If you've got processes in D waiting on those volumes I think you might
be hosed beyond recovery without reboot.

Having said that, you could try bumping up /proc/sys/fs/file-max to something
insanely high and then cat /proc/mounts > /etc/mtab

After that, I could only make wild guesses as to what needs to happen next.

-- 
Matthew Galgoci
IS Production Operations
Red Hat, Inc
919.754.3700 x44155

Re: [linux-lvm] Cleaning up stale handles in /etc/mtab

[Michael Loftis]

--On September 1, 2006 11:13:41 AM -0400 "Brown, Rodrick R" 
<rodrick.r.brown@bofasecurities.com> wrote:

> Not sure what exactly happened on this server but it looks like the LUNS
> were pulled w/o removing the volumes correctly or something of that
> sort.

Looks like you managed to hit fs.filemax actually.  What is the output of 
"sysctl fs.file-max fs.file-nr"?

>
># df
> df: `/vob/gcib/its': Too many open files in system
> df: `/vob/lftech/artefacts': Too many open files in system
> df: `/vob/rmt/mktrisk_portal': Too many open files in system
> df: `/vob/gcib/gfx_fxpb_client_facing': Too many open files in system
> df: `/vob/gcib/gfx_fxpb_workflow': Too many open files in system
> df: `/vob/rmt/credittrend': Too many open files in system
> df: `/vob/adv/re_reporting': Too many open files in system
> df: `/vob/datamgmt/atlas_p2': Too many open files in system
> df: `/vob/gcib/gfx_rad': Too many open files in system
> ...
> ...
>
> Does anyone know of a method to fix this without rebooting the box
> thanks?
>
> Thanks.
>
> ---
> Rodrick R. Brown
> UNIX Platform Support (SME)
> Banc of America Securities LLC.
> Global Trading Infrastructure (GCIB)
> 100 West 33rd ST. 3rd Flr.
> New York, NY 10001
> Mail Code: NY1-509-03-18
> Office: 646 733 4473
> Cell: 646-261-5286
>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler