From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751470AbZHZPit (ORCPT ); Wed, 26 Aug 2009 11:38:49 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1751167AbZHZPis (ORCPT ); Wed, 26 Aug 2009 11:38:48 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57674 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750828AbZHZPis (ORCPT ); Wed, 26 Aug 2009 11:38:48 -0400 Message-ID: <4A95573A.6090404@redhat.com> Date: Wed, 26 Aug 2009 11:39:38 -0400 From: Ric Wheeler User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.1) Gecko/20090806 Fedora/3.0-3.8.b3.fc12 Thunderbird/3.0b3 MIME-Version: 1.0 To: Andrei Tanas CC: NeilBrown , linux-kernel@vger.kernel.org, linux-ide-owner@vger.kernel.org Subject: Re: MD/RAID: what's wrong with sector 1953519935? References: <004e01ca25e4$c11a54e0$434efea0$@ca> <9cfb6af689a7010df166fdebb1ef516b.squirrel@neil.brown.name> <4A948A82.4080901@redhat.com> <4A94905F.7050705@redhat.com> <005101ca25f4$09006830$1b013890$@ca> <4A94A0E6.4020401@redhat.com> <005401ca25ff$9ac91cc0$d05b5640$@ca> <4A950FA6.4020408@redhat.com> <92cb16daad8278b0aa98125b9e1d057a@localhost> In-Reply-To: <92cb16daad8278b0aa98125b9e1d057a@localhost> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 08/26/2009 10:46 AM, Andrei Tanas wrote: > On Wed, 26 Aug 2009 06:34:14 -0400, Ric Wheeler > wrote: >> On 08/25/2009 11:45 PM, Andrei Tanas wrote: >>>>>> I would suggest that Andrei might try to write and clear the IO >>>>>> >>>> error >>>> >>>>>> at that >>>>>> offset. You can use Mark Lord's hdparm to clear a specific sector or >>>>>> just do the >>>>>> math (carefully!) and dd over it. It the write succeeds (without >>>>>> bumping your >>>>>> remapped sectors count) this is a likely match to this problem, >>>>>> >>>>> I've tried dd multiple times, it always succeeds, and the relocated >>>>> >>>> sector >>>> >>>>> count is currently 1 on this drive, even though this particular fault >>>>> happened at least 3 times so far. >>>>> >>>>> >>>> I would bump that count way up (say to 2) and see if you have an >>>> issue... >>>> >>> Not sure what you mean by this: how can I artificially bump the > relocated >>> sector count? >>> >>> >> Sorry - you need to set the tunable: >> >> /sys/block/mdX/md/safe_mode_delay >> >> to something like "2" to prevent that sector from being a hotspot... > > I did that as soon as you suggested that it's possible to tune it. The > array is still being rebuilt (it's a fairly busy machine, so rebuilding is > slow). I'll monitor it, but I don't expect to see the results soon as even > with the default value of 0.2 it used to happen once in several weeks. > > On the other note: is it possible that the drive was actually working > properly but was not given enough time to complete the write request? These > newer drives have 32MB cache but the same rotational speed and seek times > as the older ones so they must need more time to flush their cache? > > Andrei. > Timeouts on IO requests are pretty large, usually drives won't fail an IO unless there is a real problem but I will add the linux-ide list to this response so they can weigh in. I suspect that the error was real, but might be this "repairable" type of adjacent track issue I mentioned before. Interesting to note that just following the error, you see that it was indeed the super block that did not get updated... The error you referenced was: 90307.328266] ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen [90307.328275] ata2.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 0 [90307.328277] res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout) [90307.328280] ata2.00: status: { DRDY } [90307.328288] ata2: hard resetting link [90313.218511] ata2: link is slow to respond, please be patient (ready=0) [90317.377711] ata2: SRST failed (errno=-16) [90317.377720] ata2: hard resetting link [90318.251720] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300) [90318.338026] ata2.00: configured for UDMA/133 [90318.338062] ata2: EH complete [90318.370625] end_request: I/O error, dev sdb, sector 1953519935 [90318.370632] md: super_written gets error=-5, uptodate=0 Ric