XFS File System Monitor

XFS File System Monitor

[Rotem Ben Arye]

[-- Attachment #1.1: Type: text/plain, Size: 644 bytes --]

Hi, SGI Support Team.
My Name is Rotem , I am a Linux/Unix System Administrator in web company at
Israel.
I have a question I want to appeal to you to get some advice.

In the last weekend we had crisis in one of the Production server in
the comany ,the problem was defined by the Integrators as  "xfs file system
corrupted"
My question is , what are the open source tools , that we can use on
runtime at production environment , to monitor and sample to get indication
on mount XFS ,
That something is not living well, and can lead to problem.

We are working in a Linux environment on CentOS distributions server.

Thank you,

Rotem Ben Arye

[-- Attachment #1.2: Type: text/html, Size: 945 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS File System Monitor

[Stan Hoeppner]

On 1/2/2014 6:16 AM, Rotem Ben Arye wrote:
> Hi, SGI Support Team.
> My Name is Rotem , I am a Linux/Unix System Administrator in web company at
> Israel.
> I have a question I want to appeal to you to get some advice.
> 
> In the last weekend we had crisis in one of the Production server in
> the comany ,the problem was defined by the Integrators as  "xfs file system
> corrupted"
> My question is , what are the open source tools , that we can use on
> runtime at production environment , to monitor and sample to get indication
> on mount XFS ,
> That something is not living well, and can lead to problem.
> 
> We are working in a Linux environment on CentOS distributions server.

So in a nutshell you're looking for a monitor application that will in
essence give you a green, yellow, or red light informing you of the
filesystem's health.  Or some kind of SNMP logging that suggests a
corruption is imminent.

There is no such tool, and never will be.  Nearly all XFS corruption
events are caused by either software bugs in the XFS code or elsewhere
in the Linux kernel, transient or permanent hardware failures, or power
failures, at some layer in the storage stack.  It is not feasible to
predict such events.

When an XFS corruption occurs, one should report all related log
information and errors to this list so that the problem may be analyzed
and the root cause identified.  Then the proper corrective action can be
identified and implemented to fix the problem and hopefully prevent it
from reoccurring.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS File System Monitor

[Stan Hoeppner]

On 1/4/2014 2:21 AM, Rotem Ben Arye wrote:
> Hi Stan,
> Thank you for focused answer ,only to realize , i'm aware that monitor snmp
> checks On  /var/log/messages
> Of Production server in power failure ,so that after that will tell us -
> "power outage causes to a file system corruption" is useless.
> 
> But for all those cases that you mention " bugs in the XFS code or
> elsewhere in the Linux kernel , transient or permanent hardware failures"
> Is there no some suitable log that we can track to get an indication of
> kind event that you specified .

Errors due to such problems are logged in dmesg.  Hardware problems will
usually show up as IO errors generated from the device on which XFS
resides.  When these occur XFS will typically initiate automatic
shutdown of the filesystem to prevent (further) corruption.  In this
case the log entry occurs simultaneously with the shutdown, so
monitoring logs won't notify you in advance of this problem.  Monitoring
your hardware may.

If you get corruption due to a software bug, you may not see an error in
the log until after the filesystem suffers the corruption event.
Usually when you see errors of this nature in the log it is because
corruption has already occurred, possibly long ago, but is just now
being detected by code specifically added to XFS to detect such things.
 For example, say your filesystem is 3 years old, corruption occurred in
year one, and an update to XFS 2 years later looks for such corruption
whereas before it did not.  Depending on the severity of the corruption,
xfs_repair may be able to fix it, or it may not.  If not you ask for
help here.

So again, I'm not away of any proactive monitoring that would help in
these situations.  Of course it would be nice to know if something is
going to fail beforehand, but this isn't always possible, unfortunately.

--
Stan


> Thank you.
> 
> 
> 
> On Thu, Jan 2, 2014 at 5:07 PM, Stan Hoeppner <stan@hardwarefreak.com>wrote:
> 
>> On 1/2/2014 6:16 AM, Rotem Ben Arye wrote:
>>> Hi, SGI Support Team.
>>> My Name is Rotem , I am a Linux/Unix System Administrator in web company
>> at
>>> Israel.
>>> I have a question I want to appeal to you to get some advice.
>>>
>>> In the last weekend we had crisis in one of the Production server in
>>> the comany ,the problem was defined by the Integrators as  "xfs file
>> system
>>> corrupted"
>>> My question is , what are the open source tools , that we can use on
>>> runtime at production environment , to monitor and sample to get
>> indication
>>> on mount XFS ,
>>> That something is not living well, and can lead to problem.
>>>
>>> We are working in a Linux environment on CentOS distributions server.
>>
>> So in a nutshell you're looking for a monitor application that will in
>> essence give you a green, yellow, or red light informing you of the
>> filesystem's health.  Or some kind of SNMP logging that suggests a
>> corruption is imminent.
>>
>> There is no such tool, and never will be.  Nearly all XFS corruption
>> events are caused by either software bugs in the XFS code or elsewhere
>> in the Linux kernel, transient or permanent hardware failures, or power
>> failures, at some layer in the storage stack.  It is not feasible to
>> predict such events.
>>
>> When an XFS corruption occurs, one should report all related log
>> information and errors to this list so that the problem may be analyzed
>> and the root cause identified.  Then the proper corrective action can be
>> identified and implemented to fix the problem and hopefully prevent it
>> from reoccurring.
>>
>> --
>> Stan
>>
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS File System Monitor

[support]

Hello Rotem

We have received your email and wish to log your request. However we need to know the Serial Number (s/n) of your machine in order to do so.

The Serial Number is usually located on a white barcode label on the back of your machine.

Depending on your system type, you may also be able to determine the serial number by running one of the following commands:

1) "sysinfo -vv" (Irix)
2) "cat /proc/sgi_sn/system_serial_number" (Altix)
3) dmidecode |grep Serial |grep X  (Altix XE)
4) scash -p -c "dmidecode |grep Serial |grep X" (Altix XE Clusters with Scalimanage)
5) dmidecode  | grep "Serial Number" | grep Z (Altix ICE)
6) dmidecode -s system-serial-number (Altix UV 10)
7) From the OS, use the 'topology' command.  From the SMN, use the 'config' command.  (Altix UV 100 & UV 1000)
8) "wmic bios get serialnumber" from the command line (Windows 2003/2008)

Once we receive this information, we will be able to log a case.

Thank you in advance.

Regards,
SGI Global Customer Support Center

Corporate Office:
SGI, 46600 Landing Parkway, Fremont, CA 94538, USA

>
>
> To: support@sgi.com
> Date: 12:16 PM Thursday January 02nd, 2014(GMT)
> From: "Rotem Ben Arye" <rotem.benarye@gmail.com>
> CC: xfs@oss.sgi.com
> Subject: XFS File System Monitor
> 
> Hi, SGI Support Team.
>
> My Name is Rotem , I am a Linux/Unix System Administrator in web company
> at Israel. I have a question I want to appeal to you to get some advice.
> 
> In the last weekend we had crisis in one of the Production server in
> the comany ,the problem was defined by the Integrators as  "xfs file
> system corrupted"
>
> My question is , what are the open source tools , that we can use on
> runtime at production environment , to monitor and sample to get
> indication on mount XFS , That something is not living well, and can 
> lead to problem.
> 
> We are working in a Linux environment on CentOS distributions server.
> 
> Thank you,
> 
> Rotem Ben Arye
> 

-------------------------------------------------------------------------------
***** Have you tried Supportfolio? *****
Did you know that you can open cases via the SGI web portal Supportfolio?  Just click on the following link: https://support.sgi.com/login
-------------------------------------------------------------------------------

http://www.sgi.com/
Contact us: http://www.sgi.com/support/supportcenters.html

NB: Information in this message is SGI confidential. It is intended solely for the person(s) to whom it is addressed and may not be copied, used, disclosed or distributed to others without SGI consent. If you are not the intended recipient please will you notify me by email or telephone, delete the message from your system immediately and destroy any printed copies.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS File System Monitor

[Rotem Ben Arye]

[-- Attachment #1.1: Type: text/plain, Size: 3244 bytes --]

HI , I'm not sure it will help here,  we  are working central storage with
Netapp , EMC and others ,
The Using of XFS File system on volume , is expressed with an creation of
XFS, as e.g :  mkfs.xfs.
 My question about what are the tools that could assist us to monitor XFS
file system was in general.


 On Thu, Jan 2, 2014 at 3:04 PM, <support@sgi.com> wrote:

> Hello Rotem
>
> We have received your email and wish to log your request. However we need
> to know the Serial Number (s/n) of your machine in order to do so.
>
> The Serial Number is usually located on a white barcode label on the back
> of your machine.
>
> Depending on your system type, you may also be able to determine the
> serial number by running one of the following commands:
>
> 1) "sysinfo -vv" (Irix)
> 2) "cat /proc/sgi_sn/system_serial_number" (Altix)
> 3) dmidecode |grep Serial |grep X  (Altix XE)
> 4) scash -p -c "dmidecode |grep Serial |grep X" (Altix XE Clusters with
> Scalimanage)
> 5) dmidecode  | grep "Serial Number" | grep Z (Altix ICE)
> 6) dmidecode -s system-serial-number (Altix UV 10)
> 7) From the OS, use the 'topology' command.  From the SMN, use the
> 'config' command.  (Altix UV 100 & UV 1000)
> 8) "wmic bios get serialnumber" from the command line (Windows 2003/2008)
>
> Once we receive this information, we will be able to log a case.
>
> Thank you in advance.
>
> Regards,
> SGI Global Customer Support Center
>
> Corporate Office:
> SGI, 46600 Landing Parkway, Fremont, CA 94538, USA
>
> >
> >
> > To: support@sgi.com
> > Date: 12:16 PM Thursday January 02nd, 2014(GMT)
> > From: "Rotem Ben Arye" <rotem.benarye@gmail.com>
> > CC: xfs@oss.sgi.com
> > Subject: XFS File System Monitor
> >
> > Hi, SGI Support Team.
> >
> > My Name is Rotem , I am a Linux/Unix System Administrator in web company
> > at Israel. I have a question I want to appeal to you to get some advice.
> >
> > In the last weekend we had crisis in one of the Production server in
> > the comany ,the problem was defined by the Integrators as  "xfs file
> > system corrupted"
> >
> > My question is , what are the open source tools , that we can use on
> > runtime at production environment , to monitor and sample to get
> > indication on mount XFS , That something is not living well, and can
> > lead to problem.
> >
> > We are working in a Linux environment on CentOS distributions server.
> >
> > Thank you,
> >
> > Rotem Ben Arye
> >
>
>
> -------------------------------------------------------------------------------
> ***** Have you tried Supportfolio? *****
> Did you know that you can open cases via the SGI web portal Supportfolio?
>  Just click on the following link: https://support.sgi.com/login
>
> -------------------------------------------------------------------------------
>
> http://www.sgi.com/
> Contact us: http://www.sgi.com/support/supportcenters.html
>
> NB: Information in this message is SGI confidential. It is intended solely
> for the person(s) to whom it is addressed and may not be copied, used,
> disclosed or distributed to others without SGI consent. If you are not the
> intended recipient please will you notify me by email or telephone, delete
> the message from your system immediately and destroy any printed copies.
>
>

[-- Attachment #1.2: Type: text/html, Size: 4404 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS File System Monitor

[Emmanuel Florac]

Le Thu, 2 Jan 2014 15:20:58 +0200
Rotem Ben Arye <rotem.benarye@gmail.com> écrivait:

> HI , I'm not sure it will help here,  we  are working central storage
> with Netapp , EMC and others ,
> The Using of XFS File system on volume , is expressed with an
> creation of XFS, as e.g :  mkfs.xfs.
>  My question about what are the tools that could assist us to monitor
> XFS file system was in general.

Rotem, if you have no SGI supported hardware, SGI support won't help
you.

Furthermore, it's impossible to help you without more details: what is
the hardware setup, the exact OS version and linux kernel in use.

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS File System Monitor

[Eric Sandeen]

On 1/2/14, 7:27 AM, Emmanuel Florac wrote:
> Le Thu, 2 Jan 2014 15:20:58 +0200
> Rotem Ben Arye <rotem.benarye@gmail.com> écrivait:
> 
>> HI , I'm not sure it will help here,  we  are working central storage
>> with Netapp , EMC and others ,
>> The Using of XFS File system on volume , is expressed with an
>> creation of XFS, as e.g :  mkfs.xfs.
>>  My question about what are the tools that could assist us to monitor
>> XFS file system was in general.
> 
> Rotem, if you have no SGI supported hardware, SGI support won't help
> you.
> 
> Furthermore, it's impossible to help you without more details: what is
> the hardware setup, the exact OS version and linux kernel in use.
> 

http://xfs.org/index.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS File System Monitor

[Rotem Ben Arye]

[-- Attachment #1.1: Type: text/plain, Size: 979 bytes --]

Thank you Eric .
An properly Document..




On Fri, Jan 3, 2014 at 8:24 PM, Eric Sandeen <sandeen@sandeen.net> wrote:

> On 1/2/14, 7:27 AM, Emmanuel Florac wrote:
> > Le Thu, 2 Jan 2014 15:20:58 +0200
> > Rotem Ben Arye <rotem.benarye@gmail.com> écrivait:
> >
> >> HI , I'm not sure it will help here,  we  are working central storage
> >> with Netapp , EMC and others ,
> >> The Using of XFS File system on volume , is expressed with an
> >> creation of XFS, as e.g :  mkfs.xfs.
> >>  My question about what are the tools that could assist us to monitor
> >> XFS file system was in general.
> >
> > Rotem, if you have no SGI supported hardware, SGI support won't help
> > you.
> >
> > Furthermore, it's impossible to help you without more details: what is
> > the hardware setup, the exact OS version and linux kernel in use.
> >
>
>
> http://xfs.org/index.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F
>
> -Eric
>

[-- Attachment #1.2: Type: text/html, Size: 1747 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs