public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
From: Andrew Ryder <tireman@shaw.ca>
To: Dave Chinner <david@fromorbit.com>
Cc: Brian Foster <bfoster@redhat.com>, xfs@oss.sgi.com
Subject: Re: xfs_repair fails after trying to format log cycle?
Date: Fri, 3 Jun 2016 22:28:54 -0400	[thread overview]
Message-ID: <57523CE6.7020906@shaw.ca> (raw)
In-Reply-To: <20160413045129.GO567@dastard>

Sorry to dig up old stuff here but after replacing hardware and pulling 
my hair out to no end, I traced the source of the problem out.

On every drive in the array, the points on the PCB which the contacts 
for the drives motors and head/actuators make contact with were oxidized 
enough to cause the issue. I ended up pulling the PCB off each drive and 
cleaning them as well as cleaning out all other cable and drive 
connectors from the HBA outward and everything is happy again.

http://s33.postimg.org/uhjmvw4dr/Not_Cleaned.jpg
http://s33.postimg.org/xo94ieii7/Partial_Cleaned_1.jpg
http://s33.postimg.org/hoqgyumgf/Partial_Cleaned_2.jpg
http://s33.postimg.org/68k20t8a7/Partial_Cleaned_3.jpg


On 04/13/2016 12:51 AM, Dave Chinner wrote:
> On Tue, Apr 12, 2016 at 11:02:37PM -0400, Andrew Ryder wrote:
>> Is it possible the location its searching for at block
>>
>> 02:34:43.887528 pread64(4, 0x7fb8f53e0200, 2097152, 3001552175104) =
>> -1 EIO (Input/output error)
>
> so offset is 3001552175104, or roughly around the 3TB mark. Given
> the log i always placed int eh middle of the filesystem and you have
> a 6TB device, then the above definitely looks like a valid place to
> be reading from the log.
>
>> xfs_logprint:
>>      data device: 0x902
>>      log device: 0x902 daddr: 5860130880 length: 4173824
>
> daddr converted to offset is 5860130880 * 512 = 3001552175104, which
> tells us that the above pread64 failure was definitely coming from
> an attempt to read the log.
>
> That this is coming from the block device from userspace indicates a
> problem below XFS. There is something going wrong with your
> underlying block device and/or hardware here; AFAICT it's not
> related to XFS at all.
>
>>> GNU Parted 3.2
>>> Using /dev/sdk
>>> Welcome to GNU Parted! Type 'help' to view a list of commands.
>>> (parted) p
>>> Model: ATA ST2000DL001-9VT1 (scsi)
>>> Disk /dev/sdk: 2000GB
>>> Sector size (logical/physical): 512B/512B
>>> Partition Table: msdos
>>> Disk Flags:
>>>
>>> Number  Start  End     Size    Type     File system  Flags
>>>   1      512B   2000GB  2000GB  primary               raid
>>> Number  Start  End          Size         Type     File system  Flags
>>>   1      1s     3907029167s  3907029167s  primary               raid
>
> Compared to the other devices, it has a different start sector, a
> different size, and an msdos partition table rather than gpt.
> Definitely a red flag...
>
>>>>> This all began when the RR2722 driver running under 3.18.15
>>>>> complained and
>
> Reported physical IO errors to a write command. Really, this looks
> like a hardware issue, not something that can be fixed by running
> xfs_repair...
>
> Cheers,
>
> Dave.
>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

  reply	other threads:[~2016-06-04  2:28 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-03-26 19:09 xfs_repair fails after trying to format log cycle? Andrew Ryder
2016-03-28  8:55 ` Brian Foster
2016-04-12  5:53   ` Andrew Ryder
2016-04-12 14:05     ` Brian Foster
2016-04-12 20:16       ` Andrew Ryder
2016-04-13  3:02         ` Andrew Ryder
2016-04-13  4:51           ` Dave Chinner
2016-06-04  2:28             ` Andrew Ryder [this message]
2016-06-06 15:33               ` Emmanuel Florac
2016-06-06 19:19                 ` Andrew Ryder
2016-06-06 19:37                   ` Andrew Ryder
2016-04-13 12:12         ` Brian Foster
2016-04-13 22:34           ` Andrew Ryder
2016-04-14  4:32             ` Andrew Ryder
2016-04-15  4:24             ` Andrew Ryder

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=57523CE6.7020906@shaw.ca \
    --to=tireman@shaw.ca \
    --cc=bfoster@redhat.com \
    --cc=david@fromorbit.com \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox