Re: XFS File system in trouble

[Leslie Rhorer <lrhorer@mygrande.net>]

	OK, this is goofy.  It seems to be working, now.  As usual, I've been 
doing some work on the server this weekend, but I can't think of 
anything I have done that would fix the issue.  I did replace the 
remaining good 4G RAM module with a pair of 8G RAM modules, but memtest 
reported the remaining 4G module as good, and I verified the removed 
module really was bad.  I also replaced the removable drive carrier and 
cables that were feeding the two SSDs, once of which was reporting 
failures as noted in the syslog.  It's hard for me to believe either of 
those things could have been causing the issue, though.

	I attached a 1.5T external drive to the server and formatted it as XFS 
in preparation to continue troubleshooting.  To make sure of things, I 
tried decompressing the tarball, again, and this time it worked all the 
way to the end.  I then deleted the entire directory structure created 
by the tarball and decompressed the file again twice.  I'll see if the 
rsync process works.  That will take a couple of days.

On 7/28/2015 5:11 PM, Brian Foster wrote:
> On Tue, Jul 28, 2015 at 10:13:01AM -0500, Leslie Rhorer wrote:
>> On 7/28/2015 7:33 AM, Brian Foster wrote:
>>> On Tue, Jul 28, 2015 at 02:46:45AM -0500, Leslie Rhorer wrote:
>>>> On 7/20/2015 6:17 AM, Brian Foster wrote:
>>>>> On Sat, Jul 18, 2015 at 08:02:50PM -0500, Leslie Rhorer wrote:
>>>>>>
> ...
>>>
>>>> 	I then copied both the tarball and the image over to the root, and while
>>>> the system would not let me create the image on the root, it did let me copy
>>>> the image to the root.  I then umounted the RAID array, mounted the image,
>>>> and attempted to cd to the original directory in the image mount where the
>>>> tarball was saved.  That failed with an I/O error:
>>>>
>>>
>>> It sounds a bit strange for the mdrestore to fail on root but a cp of
>>> the resulting image to work. Do the resulting images have the same file
>>> size or is the rootfs copy truncated? If the latter, you could be
>>> missing part of the fs and thus any of the following tests are probably
>>> moot.
>>
>> 	Well, it can't be as large as it is reported, let's put it that way,
>> although the reported file size is the same.  Ls claims it to be 16T in
>> size, which cannot be the case on a 100G partition.  I forgot to mention cp
>> does complain:
>>
>> RAID-Server:/# cp /RAID/TEST/RAIDfile.img ./
>> cp: cannot lseek ‘./RAIDfile.img’: Invalid argument
>>
>> 	But it does the same thing on the backup server, and it works there.  I
>> tried a cmp, and it seems to be hung.  It just may be taking a long time,
>> however.
>>
>
> Yeah, you can't really trust the resulting image. It doesn't take much
> space to create a very large sparse file, but different filesystems have
> different maximum file size limits. The problem here is that some
> metadata near the beginning of the file might reference or depend on
> something near the end, and I/Os beyond the end of the file will
> probably result in errors.
>
> I'd probably try the nouuid approach since the hardware is similar as
> well as some of the other interesting suggestions that have been made to
> try and get the image on the rootfs and see what happens there too.
>
> Brian
>
>>> Brian
>>>
>>>> RAID-Server:/# cd "/media/Server-Main/Equipment/Drive Controllers/HighPoint
>>>> Adapters/Rocket 2722/Driver/"
>>>> bash: cd: /media/Server-Main/Equipment/Drive Controllers/HighPoint
>>>> Adapters/Rocket 2722/Driver/: Input/output error
>>>>
>>>> 	I changed directories to a point two directories above the previous attempt
>>>> and did a long listing:
>>>>
>>>> RAID-Server:/# cd "/media/Server-Main/Equipment/Drive Controllers/HighPoint
>>>> Adapters"
>>>> RAID-Server:/media/Server-Main/Equipment/Drive Controllers/HighPoint
>>>> Adapters# ll
>>>> ls: cannot access RocketRAID 2722: Input/output error
>>>> total 4
>>>> drwxr-xr-x 6 root lrhorer 4096 Jul 18 19:26 Rocket 2722
>>>> ?????????? ? ?    ?          ?            ? RocketRAID 2722
>>>>
>>>> 	As you can see, Rocket 2722 is still there, but RocketRAID 2722 is very
>>>> sick.  Rocket 2722 is the parent of where the tarbal was, however, so I did
>>>> a cd and an ll again:
>>>>
>>>> RAID-Server:/media/Server-Main/Equipment/Drive Controllers/HighPoint
>>>> Adapters# cd "Rocket 2722"/
>>>> RAID-Server:/media/Server-Main/Equipment/Drive Controllers/HighPoint
>>>> Adapters/Rocket 2722# ll
>>>> ls: cannot access BIOS: Input/output error
>>>> ls: cannot access Driver: Input/output error
>>>> ls: cannot access HighPoint RAID Management Software: Input/output error
>>>> ls: cannot access Manual: Input/output error
>>>> total 248
>>>> -rwxr--r-- 1 root lrhorer 245760 Nov 20  2008 autorun.exe
>>>> -rwxr--r-- 1 root lrhorer     51 Mar 21  2001 autorun.inf
>>>> ?????????? ? ?    ?            ?            ? BIOS
>>>> ?????????? ? ?    ?            ?            ? Driver
>>>> ?????????? ? ?    ?            ?            ? HighPoint RAID Management
>>>> Software
>>>> ?????????? ? ?    ?            ?            ? Manual
>>>> -rwxr--r-- 1 root lrhorer   1134 Feb  5  2012 readme.txt
>>>>
>>>> 	So now, what?
>>>>
>>>> _______________________________________________
>>>> xfs mailing list
>>>> xfs@oss.sgi.com
>>>> http://oss.sgi.com/mailman/listinfo/xfs
>>>
>>
>> _______________________________________________
>> xfs mailing list
>> xfs@oss.sgi.com
>> http://oss.sgi.com/mailman/listinfo/xfs
>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs