Errors on ICH10R with Seagate HDDs

Errors on ICH10R with Seagate HDDs

[Thomas Jackowski]

Hi,

I assume a problem between Seagate Drives and the ATA Drivers.

Chipset is a ICH10R, but this problems has been seen on a ICH9 also.

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda 7200.11
Device Model:     ST31000340AS
Serial Number:    6QJ03QBN
Firmware Version: SD1A
User Capacity:    1,000,203,804,160 bytes
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Fri Mar 27 10:58:37 2009 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

This is /dev/sdc. /dev/sdd and /dev/sde is the same model with the same 
Firmware. I'm using a RAID5 on these 3 HDDs.
Because of the Seagate Firmware Bug I had to upgrade the firmware from
'SD15' to 'SD1A'. The upgrade was successful.

Before the upgrade there were no problems, after the upgrade I have 
trouble like this:

[14144.875924] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 
frozen
[14144.875931] ata4.00: cmd ec/00:01:00:00:00/00:00:00:00:00/00 tag 0 
pio 512 in
[14144.875932]          res 40/00:ff:00:00:00/00:00:00:00:00/40 Emask 
0x4 (timeout)
[14144.875934] ata4.00: status: { DRDY }
[14144.875938] ata4: hard resetting link
[14145.362708] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[14145.526248] ata4.00: configured for UDMA/133
[14145.526261] ata4: EH complete
[14145.550243] sd 3:0:0:0: [sdd] 1953523055 512-byte hardware sectors 
(1000204 MB)
[14145.550263] sd 3:0:0:0: [sdd] Write Protect is off
[14145.550265] sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[14145.550282] sd 3:0:0:0: [sdd] Write cache: enabled, read cache: 
enabled, doesn't support DPO or FUA

[507139.744594] ata5.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 
0x6 frozen
[507139.744602] ata5.00: cmd e5/00:00:00:00:00/00:00:00:00:00/00 tag 0
[507139.744603]          res 40/00:00:be:ac:54/00:00:1b:00:00/e0 Emask 
0x4 (timeout)
[507139.744606] ata5.00: status: { DRDY }
[507139.744611] ata5: hard resetting link
[507140.247396] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[507140.251384] ata5.00: configured for UDMA/133
[507140.251384] ata5: EH complete
[507140.253668] sd 4:0:0:0: [sde] 1953523055 512-byte hardware sectors 
(1000204 MB)
[507140.253681] sd 4:0:0:0: [sde] Write Protect is off
[507140.253683] sd 4:0:0:0: [sde] Mode Sense: 00 3a 00 00
[507140.253701] sd 4:0:0:0: [sde] Write cache: enabled, read cache: 
enabled, doesn't suppo
rt DPO or FUA

[507453.049511] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 
0x6 frozen
[507453.049519] ata4.00: cmd e5/00:00:00:00:00/00:00:00:00:00/00 tag 0
[507453.049520]          res 40/00:00:3e:e0:54/00:00:1b:00:00/e0 Emask 
0x4 (timeout)
[507453.049523] ata4.00: status: { DRDY }
[507453.049529] ata4: hard resetting link
[507453.547144] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[507453.843855] ata4.00: configured for UDMA/133
[507453.843855] ata4: EH complete
[507453.871434] sd 3:0:0:0: [sdd] 1953523055 512-byte hardware sectors 
(1000204 MB)
[507453.872475] sd 3:0:0:0: [sdd] Write Protect is off
[507453.872475] sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[507453.872475] sd 3:0:0:0: [sdd] Write cache: enabled, read cache: 
enabled, doesn't support DPO or FUA

Where is the problem?

Bye, Thomas

Re: Errors on ICH10R with Seagate HDDs

[Tejun Heo]

Hello,

Thomas Jackowski wrote:
> This is /dev/sdc. /dev/sdd and /dev/sde is the same model with the same
> Firmware. I'm using a RAID5 on these 3 HDDs.
> Because of the Seagate Firmware Bug I had to upgrade the firmware from
> 'SD15' to 'SD1A'. The upgrade was successful.
> 
> Before the upgrade there were no problems, after the upgrade I have
> trouble like this:
...
> [14144.875931] ata4.00: cmd ec/00:01:00:00:00/00:00:00:00:00/00 tag 0

Timeout on IDENTIFY.

> [507139.744602] ata5.00: cmd e5/00:00:00:00:00/00:00:00:00:00/00 tag 0
> [507453.049519] ata4.00: cmd e5/00:00:00:00:00/00:00:00:00:00/00 tag 0

and two timeouts on CHK_POWER.

ahci, right?  Given that the problem appeared after firmware update,
it seems to indicate firmware problem.  Does libata.force=noncq make
any difference?

-- 
tejun

Re: Errors on ICH10R with Seagate HDDs

[Thomas Jackowski]

Hi,

yes this was AHCI. NCQ was enabled. I forgot to mention kernel version:
2.6.26-1-amd64 (Debian Lenny)

So here is my test:
Enabled AHCI in BIOS.

[    0.000000] Command line: root=/dev/md1 ro libata.force=noncq
[    0.000000] Kernel command line: root=/dev/md1 ro libata.force=noncq
[...]
[    3.237941] ahci 0000:00:1f.2: AHCI 0001.0200 32 slots 6 ports 3 Gbps 
0x3f impl SATA mode
[    3.238000] ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led clo 
pmp pio slum part

Ah, NCQ still activated.
/sys/block/sdc/device/queue_depth shows 31.

I guess NCQ still activated even with libata.force=noncq ?


Now here is my second test:
Disabled AHCI in BIOS.

ata_piix is used now. Did some IO, can't force any freezes.


In the meantime I found topics like this:
http://forums.seagate.com/stx/board/message?board.id=ata_drives&thread.id=2879&view=by_date_ascending&page=1

This looks like a firmware bug. I think I should contact Seagate ;) or 
any other ideas?

Bye



Tejun Heo wrote:
> Hello,
> 
> Thomas Jackowski wrote:
>> This is /dev/sdc. /dev/sdd and /dev/sde is the same model with the same
>> Firmware. I'm using a RAID5 on these 3 HDDs.
>> Because of the Seagate Firmware Bug I had to upgrade the firmware from
>> 'SD15' to 'SD1A'. The upgrade was successful.
>>
>> Before the upgrade there were no problems, after the upgrade I have
>> trouble like this:
> ...
>> [14144.875931] ata4.00: cmd ec/00:01:00:00:00/00:00:00:00:00/00 tag 0
> 
> Timeout on IDENTIFY.
> 
>> [507139.744602] ata5.00: cmd e5/00:00:00:00:00/00:00:00:00:00/00 tag 0
>> [507453.049519] ata4.00: cmd e5/00:00:00:00:00/00:00:00:00:00/00 tag 0
> 
> and two timeouts on CHK_POWER.
> 
> ahci, right?  Given that the problem appeared after firmware update,
> it seems to indicate firmware problem.  Does libata.force=noncq make
> any difference?
> 

Re: Errors on ICH10R with Seagate HDDs

[Tejun Heo]

Hello,

Thomas Jackowski wrote:
> yes this was AHCI. NCQ was enabled. I forgot to mention kernel version:
> 2.6.26-1-amd64 (Debian Lenny)

Can you please try 2.6.29?

> So here is my test:
> Enabled AHCI in BIOS.
> 
> [    0.000000] Command line: root=/dev/md1 ro libata.force=noncq
> [    0.000000] Kernel command line: root=/dev/md1 ro libata.force=noncq
> [...]
> [    3.237941] ahci 0000:00:1f.2: AHCI 0001.0200 32 slots 6 ports 3 Gbps
> 0x3f impl SATA mode
> [    3.238000] ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led clo
> pmp pio slum part
> 
> Ah, NCQ still activated.
> /sys/block/sdc/device/queue_depth shows 31.
> 
> I guess NCQ still activated even with libata.force=noncq ?

Well, if the module is loaded from initrd, you need to jump through
some loops to get the parameter passed to libata.ko.  You can also
disable NCQ by echoing 1 to queue_depth.

> Now here is my second test: Disabled AHCI in BIOS.
> 
> ata_piix is used now. Did some IO, can't force any freezes.
> 
> In the meantime I found topics like this:
> http://forums.seagate.com/stx/board/message?board.id=ata_drives&thread.id=2879&view=by_date_ascending&page=1
> 
> This looks like a firmware bug. I think I should contact Seagate ;) or
> any other ideas?

Hmmm... please verify with ahci w/o NCQ.  If that works fine too,
please go ahead and contact Seagate.

Thanks.

-- 
tejun

Re: Errors on ICH10R with Seagate HDDs

[Thomas Jackowski]

Hey,

some additional information (2.6.26-1) and ata_piix:
[653158.828848] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 
0x6 frozen
[653158.828856] ata3.00: cmd e5/00:00:00:00:00/00:00:00:00:00/00 tag 0
[653158.828857]          res 40/00:01:00:00:00/00:00:00:00:00/00 Emask 
0x4 (timeout)
[653158.828860] ata3.00: status: { DRDY }
[653158.828868] ata3: hard resetting link
[653159.331110] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[653159.355648] ata3.00: configured for UDMA/133
[653159.355666] ata3: EH complete
[653159.371885] sd 2:0:0:0: [sde] 1953523055 512-byte hardware sectors 
(1000204 MB)
[653159.374081] sd 2:0:0:0: [sde] Write Protect is off
[653159.374081] sd 2:0:0:0: [sde] Mode Sense: 00 3a 00 00
[653159.390212] sd 2:0:0:0: [sde] Write cache: enabled, read cache: 
enabled, doesn't support DPO or FUA

I tried 2.6.29 and I had no outputs in kernel.log with AHCI and w/o NCQ. 
But my system still had hiccups (system hangs for a second) so I am not 
sure if there was just no output to kernel.log (due using debian 
unstable kernel with eventually mysterious options).

I will observe this and report if any error occurs again.

Bye Thomas

Tejun Heo wrote:
> Hello,
> 
> Thomas Jackowski wrote:
>> yes this was AHCI. NCQ was enabled. I forgot to mention kernel version:
>> 2.6.26-1-amd64 (Debian Lenny)
> 
> Can you please try 2.6.29?
> 
>> So here is my test:
>> Enabled AHCI in BIOS.
>>
>> [    0.000000] Command line: root=/dev/md1 ro libata.force=noncq
>> [    0.000000] Kernel command line: root=/dev/md1 ro libata.force=noncq
>> [...]
>> [    3.237941] ahci 0000:00:1f.2: AHCI 0001.0200 32 slots 6 ports 3 Gbps
>> 0x3f impl SATA mode
>> [    3.238000] ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led clo
>> pmp pio slum part
>>
>> Ah, NCQ still activated.
>> /sys/block/sdc/device/queue_depth shows 31.
>>
>> I guess NCQ still activated even with libata.force=noncq ?
> 
> Well, if the module is loaded from initrd, you need to jump through
> some loops to get the parameter passed to libata.ko.  You can also
> disable NCQ by echoing 1 to queue_depth.
> 
>> Now here is my second test: Disabled AHCI in BIOS.
>>
>> ata_piix is used now. Did some IO, can't force any freezes.
>>
>> In the meantime I found topics like this:
>> http://forums.seagate.com/stx/board/message?board.id=ata_drives&thread.id=2879&view=by_date_ascending&page=1
>>
>> This looks like a firmware bug. I think I should contact Seagate ;) or
>> any other ideas?
> 
> Hmmm... please verify with ahci w/o NCQ.  If that works fine too,
> please go ahead and contact Seagate.
> 
> Thanks.
> 

Re: Errors on ICH10R with Seagate HDDs

[Tejun Heo]

Hello,

Thomas Jackowski wrote:
> some additional information (2.6.26-1) and ata_piix:
> [653158.828848] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action
> 0x6 frozen
> [653158.828856] ata3.00: cmd e5/00:00:00:00:00/00:00:00:00:00/00 tag 0
> [653158.828857]          res 40/00:01:00:00:00/00:00:00:00:00/00 Emask
> 0x4 (timeout)

Hmmm... that's timeout on CHECK POWER.

> [653158.828860] ata3.00: status: { DRDY }
> [653158.828868] ata3: hard resetting link
> [653159.331110] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
> [653159.355648] ata3.00: configured for UDMA/133
> [653159.355666] ata3: EH complete
> [653159.371885] sd 2:0:0:0: [sde] 1953523055 512-byte hardware sectors
> (1000204 MB)
> [653159.374081] sd 2:0:0:0: [sde] Write Protect is off
> [653159.374081] sd 2:0:0:0: [sde] Mode Sense: 00 3a 00 00
> [653159.390212] sd 2:0:0:0: [sde] Write cache: enabled, read cache:
> enabled, doesn't support DPO or FUA
> 
> I tried 2.6.29 and I had no outputs in kernel.log with AHCI and w/o NCQ.
> But my system still had hiccups (system hangs for a second) so I am not
> sure if there was just no output to kernel.log (due using debian
> unstable kernel with eventually mysterious options).
> 
> I will observe this and report if any error occurs again.

Thanks.

-- 
tejun

Re: Errors on ICH10R with Seagate HDDs

[Thomas Jackowski]

Hi,

I'm using 2.6.29-2-amd64 now and the bug still exists (see previous
mails) with AHCI and NCQ enabled.

Maybe I should contact Seagate Support now?

I wonder that nobody else reported this sort of problem?

Tejun Heo wrote:
> Hello,
> 
> Thomas Jackowski wrote:
>> some additional information (2.6.26-1) and ata_piix:
>> [653158.828848] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action
>> 0x6 frozen
>> [653158.828856] ata3.00: cmd e5/00:00:00:00:00/00:00:00:00:00/00 tag 0
>> [653158.828857]          res 40/00:01:00:00:00/00:00:00:00:00/00 Emask
>> 0x4 (timeout)
> 
> Hmmm... that's timeout on CHECK POWER.
> 
>> [653158.828860] ata3.00: status: { DRDY }
>> [653158.828868] ata3: hard resetting link
>> [653159.331110] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
>> [653159.355648] ata3.00: configured for UDMA/133
>> [653159.355666] ata3: EH complete
>> [653159.371885] sd 2:0:0:0: [sde] 1953523055 512-byte hardware sectors
>> (1000204 MB)
>> [653159.374081] sd 2:0:0:0: [sde] Write Protect is off
>> [653159.374081] sd 2:0:0:0: [sde] Mode Sense: 00 3a 00 00
>> [653159.390212] sd 2:0:0:0: [sde] Write cache: enabled, read cache:
>> enabled, doesn't support DPO or FUA
>>
>> I tried 2.6.29 and I had no outputs in kernel.log with AHCI and w/o NCQ.
>> But my system still had hiccups (system hangs for a second) so I am not
>> sure if there was just no output to kernel.log (due using debian
>> unstable kernel with eventually mysterious options).
>>
>> I will observe this and report if any error occurs again.
> 
> Thanks.
> 

-- 
Gruß,
Thomas

WWW: http://nice-pic.de
Hadifon: ~2127
E-Mail: aXe@hadiko.de

Re: Errors on ICH10R with Seagate HDDs

[Tejun Heo]

Hello,

Thomas Jackowski wrote:
> I'm using 2.6.29-2-amd64 now and the bug still exists (see previous
> mails) with AHCI and NCQ enabled.
> 
> Maybe I should contact Seagate Support now?
> 
> I wonder that nobody else reported this sort of problem?

There were several reports drives timing out some SMART commansd but
on CPM I think yours is the only one.  BTW, where is that command
coming from?  Are you using disk head parking?

Thanks.

-- 
tejun

Re: Errors on ICH10R with Seagate HDDs

[Thomas Jackowski]

Hi,

I have no clue where this command comes from. How can I determine this?

I'm not using disk head parking.

Tejun Heo wrote:
> Hello,
> 
> Thomas Jackowski wrote:
>> I'm using 2.6.29-2-amd64 now and the bug still exists (see previous
>> mails) with AHCI and NCQ enabled.
>>
>> Maybe I should contact Seagate Support now?
>>
>> I wonder that nobody else reported this sort of problem?
> 
> There were several reports drives timing out some SMART commansd but
> on CPM I think yours is the only one.  BTW, where is that command
> coming from?  Are you using disk head parking?
> 
> Thanks.
> 

Thanks Thomas

Re: Errors on ICH10R with Seagate HDDs

[Tejun Heo]

Thomas Jackowski wrote:
> Hi,
> 
> I have no clue where this command comes from. How can I determine this?
> 
> I'm not using disk head parking.

Hmmm.... busy looping fuser -v /dev/sdX would probably catch it given
enough time.  Can you please run "while true; do fuser -v /dev/sdX;
done" for a while and report the result?  If it doesn't give you
anything, please let me know, I'll prep a debug patch.

Thanks.

-- 
tejun

Re: Errors on ICH10R with Seagate HDDs

[Thomas Jackowski]

Hi,

there is no relevant output in this loop (running about 6 hours).

Tejun Heo wrote:
> Thomas Jackowski wrote:
>> Hi,
>>
>> I have no clue where this command comes from. How can I determine this?
>>
>> I'm not using disk head parking.
> 
> Hmmm.... busy looping fuser -v /dev/sdX would probably catch it given
> enough time.  Can you please run "while true; do fuser -v /dev/sdX;
> done" for a while and report the result?  If it doesn't give you
> anything, please let me know, I'll prep a debug patch.
> 
> Thanks.
> 

-- 
Gruß,
Thomas

Re: Errors on ICH10R with Seagate HDDs

[Tejun Heo]

[-- Attachment #1: Type: text/plain, Size: 201 bytes --]

Thomas Jackowski wrote:
> there is no relevant output in this loop (running about 6 hours).

Hmm... can you please try the attached patch?  It should show us who's
issuing the CPM.

Thanks.

-- 
tejun

[-- Attachment #2: whos-doing-cpm.patch --]
[-- Type: text/x-patch, Size: 1010 bytes --]

diff --git a/block/elevator.c b/block/elevator.c
index 7073a90..93e6575 100644
--- a/block/elevator.c
+++ b/block/elevator.c
@@ -618,6 +618,16 @@ void elv_insert(struct request_queue *q, struct request *rq, int where)
 	unsigned ordseq;
 	int unplug_it = 1;
 
+	if (rq->cmd[0] == 0xa1 || rq->cmd[0] == 0x85) {
+		u8 cmd = rq->cmd[0] == 0xa1 ? rq->cmd[9] : rq->cmd[14];
+
+		if (cmd == 0xe5) {
+			printk("XXX %s: elv_insert(ATA_CHK_POWER)\n",
+			       rq->rq_disk ? rq->rq_disk->disk_name : "?");
+			dump_stack();
+		}
+	}
+
 	trace_block_rq_insert(q, rq);
 
 	rq->q = q;
diff --git a/drivers/ata/libata-eh.c b/drivers/ata/libata-eh.c
index 94919ad..a56c53d 100644
--- a/drivers/ata/libata-eh.c
+++ b/drivers/ata/libata-eh.c
@@ -2722,6 +2722,7 @@ static void ata_eh_park_issue_cmd(struct ata_device *dev, int park)
 		tf.lbah = 0x55;
 	} else {
 		ehc->unloaded_mask &= ~(1 << dev->devno);
+		ata_dev_printk(dev, KERN_INFO, "XXX issuing CHK_POWER for unparking\n");
 		tf.command = ATA_CMD_CHK_POWER;
 	}
 

Re: Errors on ICH10R with Seagate HDDs

[Thomas Jackowski]

Hi,

sorry, I lost the reference in the mail headers. Please see:
http://www.spinics.net/lists/linux-ide/msg33340.html

Well, I will post now the result with Tejun's debug patch:

[15714.808535] XXX sdc: elv_insert(ATA_CHK_POWER)
[15714.808540] Pid: 22034, comm: smartctl Tainted: P           2.6.30 #1
[15714.808542] Call Trace:
[15714.808550]  [<ffffffff8033e456>] ? blk_plug_device+0x5a/0x83
[15714.808555]  [<ffffffff8033c2d2>] ? elv_insert+0x5e/0x2b0
[15714.808559]  [<ffffffff80341008>] ? blk_execute_rq_nowait+0x65/0x92
[15714.808562]  [<ffffffff803410c0>] ? blk_execute_rq+0x8b/0xab
[15714.808565]  [<ffffffff8033e831>] ? get_request_wait+0x21/0x17e
[15714.808569]  [<ffffffff8033da3d>] ? __freed_request+0x26/0x82
[15714.808572]  [<ffffffff8033dabc>] ? freed_request+0x23/0x43
[15714.808575]  [<ffffffff80344030>] ? sg_io+0x277/0x392
[15714.808578]  [<ffffffff803445df>] ? scsi_cmd_ioctl+0x1db/0x3a0
[15714.808583]  [<ffffffff80235c12>] ? check_preempt_wakeup+0x183/0x1dd
[15714.808587]  [<ffffffff802d350f>] ? __lookup_mnt+0x12/0x4b
[15714.808603]  [<ffffffffa00d9ae9>] ? sd_ioctl+0x9d/0xcb [sd_mod]
[15714.808606]  [<ffffffff803420b9>] ? __blkdev_driver_ioctl+0x69/0x7e
[15714.808615]  [<ffffffff803428d6>] ? blkdev_ioctl+0x7e4/0x81a
[15714.808618]  [<ffffffff802a41f3>] ? __do_fault+0x374/0x3af
[15714.808621]  [<ffffffff80297bbb>] ? __alloc_pages_internal+0xd2/0x427
[15714.808625]  [<ffffffff802e1a07>] ? block_ioctl+0x38/0x3c
[15714.808628]  [<ffffffff802cc0de>] ? vfs_ioctl+0x21/0x6c
[15714.808630]  [<ffffffff802cc554>] ? do_vfs_ioctl+0x42b/0x464
[15714.808634]  [<ffffffff8022cce1>] ? do_page_fault+0x1f3/0x208
[15714.808636]  [<ffffffff802cc5de>] ? sys_ioctl+0x51/0x70
[15714.808640]  [<ffffffff8020fa42>] ? system_call_fastpath+0x16/0x1b
[15715.216145] XXX sda: elv_insert(ATA_CHK_POWER)
[15715.216149] Pid: 22125, comm: hdparm Tainted: P           2.6.30 #1
[15715.216151] Call Trace:
[15715.216158]  [<ffffffff8033e456>] ? blk_plug_device+0x5a/0x83
[15715.216161]  [<ffffffff8033c2d2>] ? elv_insert+0x5e/0x2b0
[15715.216164]  [<ffffffff80341008>] ? blk_execute_rq_nowait+0x65/0x92
[15715.216167]  [<ffffffff803410c0>] ? blk_execute_rq+0x8b/0xab
[15715.216169]  [<ffffffff8033e831>] ? get_request_wait+0x21/0x17e
[15715.216173]  [<ffffffff803af118>] ? mix_pool_bytes_extract+0x57/0x14a
[15715.216176]  [<ffffffff80344030>] ? sg_io+0x277/0x392
[15715.216178]  [<ffffffff803445df>] ? scsi_cmd_ioctl+0x1db/0x3a0
[15715.216182]  [<ffffffff802d0180>] ? __d_lookup+0xc3/0x107
[15715.216185]  [<ffffffff802a1ab0>] ? zone_statistics+0x3c/0x5d
[15715.216188]  [<ffffffff802d350f>] ? __lookup_mnt+0x12/0x4b
[15715.216202]  [<ffffffffa00d9ae9>] ? sd_ioctl+0x9d/0xcb [sd_mod]
[15715.216204]  [<ffffffff803420b9>] ? __blkdev_driver_ioctl+0x69/0x7e
[15715.216207]  [<ffffffff803428d6>] ? blkdev_ioctl+0x7e4/0x81a
[15715.216210]  [<ffffffff802a41f3>] ? __do_fault+0x374/0x3af
[15715.216213]  [<ffffffff80297bbb>] ? __alloc_pages_internal+0xd2/0x427
[15715.216216]  [<ffffffff802e1a07>] ? block_ioctl+0x38/0x3c
[15715.216219]  [<ffffffff802cc0de>] ? vfs_ioctl+0x21/0x6c
[15715.216221]  [<ffffffff802cc554>] ? do_vfs_ioctl+0x42b/0x464
[15715.216224]  [<ffffffff8022cce1>] ? do_page_fault+0x1f3/0x208
[15715.216227]  [<ffffffff802cc5de>] ? sys_ioctl+0x51/0x70
[15715.216230]  [<ffffffff8020fa42>] ? system_call_fastpath+0x16/0x1b

As you can see I am using Kernel 2.6.30 (Debian 2.6.30-8).

So, what to do now?

Bye, Thomas

Re: Errors on ICH10R with Seagate HDDs

[Tejun Heo]

Thomas Jackowski wrote:
> Hi,
> 
> sorry, I lost the reference in the mail headers. Please see:
> http://www.spinics.net/lists/linux-ide/msg33340.html
> 
> Well, I will post now the result with Tejun's debug patch:
> 
> [15714.808535] XXX sdc: elv_insert(ATA_CHK_POWER)
> [15714.808540] Pid: 22034, comm: smartctl Tainted: P           2.6.30 #1
> [15714.808542] Call Trace:
> [15714.808550]  [<ffffffff8033e456>] ? blk_plug_device+0x5a/0x83
> [15714.808555]  [<ffffffff8033c2d2>] ? elv_insert+0x5e/0x2b0
> [15714.808559]  [<ffffffff80341008>] ? blk_execute_rq_nowait+0x65/0x92
> [15714.808562]  [<ffffffff803410c0>] ? blk_execute_rq+0x8b/0xab
> [15714.808565]  [<ffffffff8033e831>] ? get_request_wait+0x21/0x17e
> [15714.808569]  [<ffffffff8033da3d>] ? __freed_request+0x26/0x82
> [15714.808572]  [<ffffffff8033dabc>] ? freed_request+0x23/0x43
> [15714.808575]  [<ffffffff80344030>] ? sg_io+0x277/0x392
> [15714.808578]  [<ffffffff803445df>] ? scsi_cmd_ioctl+0x1db/0x3a0
> [15714.808583]  [<ffffffff80235c12>] ? check_preempt_wakeup+0x183/0x1dd
> [15714.808587]  [<ffffffff802d350f>] ? __lookup_mnt+0x12/0x4b
> [15714.808603]  [<ffffffffa00d9ae9>] ? sd_ioctl+0x9d/0xcb [sd_mod]
> [15714.808606]  [<ffffffff803420b9>] ? __blkdev_driver_ioctl+0x69/0x7e
> [15714.808615]  [<ffffffff803428d6>] ? blkdev_ioctl+0x7e4/0x81a
> [15714.808618]  [<ffffffff802a41f3>] ? __do_fault+0x374/0x3af
> [15714.808621]  [<ffffffff80297bbb>] ? __alloc_pages_internal+0xd2/0x427
> [15714.808625]  [<ffffffff802e1a07>] ? block_ioctl+0x38/0x3c
> [15714.808628]  [<ffffffff802cc0de>] ? vfs_ioctl+0x21/0x6c
> [15714.808630]  [<ffffffff802cc554>] ? do_vfs_ioctl+0x42b/0x464
> [15714.808634]  [<ffffffff8022cce1>] ? do_page_fault+0x1f3/0x208
> [15714.808636]  [<ffffffff802cc5de>] ? sys_ioctl+0x51/0x70
> [15714.808640]  [<ffffffff8020fa42>] ? system_call_fastpath+0x16/0x1b
> [15715.216145] XXX sda: elv_insert(ATA_CHK_POWER)
> [15715.216149] Pid: 22125, comm: hdparm Tainted: P           2.6.30 #1
> [15715.216151] Call Trace:
> [15715.216158]  [<ffffffff8033e456>] ? blk_plug_device+0x5a/0x83
> [15715.216161]  [<ffffffff8033c2d2>] ? elv_insert+0x5e/0x2b0
> [15715.216164]  [<ffffffff80341008>] ? blk_execute_rq_nowait+0x65/0x92
> [15715.216167]  [<ffffffff803410c0>] ? blk_execute_rq+0x8b/0xab
> [15715.216169]  [<ffffffff8033e831>] ? get_request_wait+0x21/0x17e
> [15715.216173]  [<ffffffff803af118>] ? mix_pool_bytes_extract+0x57/0x14a
> [15715.216176]  [<ffffffff80344030>] ? sg_io+0x277/0x392
> [15715.216178]  [<ffffffff803445df>] ? scsi_cmd_ioctl+0x1db/0x3a0
> [15715.216182]  [<ffffffff802d0180>] ? __d_lookup+0xc3/0x107
> [15715.216185]  [<ffffffff802a1ab0>] ? zone_statistics+0x3c/0x5d
> [15715.216188]  [<ffffffff802d350f>] ? __lookup_mnt+0x12/0x4b
> [15715.216202]  [<ffffffffa00d9ae9>] ? sd_ioctl+0x9d/0xcb [sd_mod]
> [15715.216204]  [<ffffffff803420b9>] ? __blkdev_driver_ioctl+0x69/0x7e
> [15715.216207]  [<ffffffff803428d6>] ? blkdev_ioctl+0x7e4/0x81a
> [15715.216210]  [<ffffffff802a41f3>] ? __do_fault+0x374/0x3af
> [15715.216213]  [<ffffffff80297bbb>] ? __alloc_pages_internal+0xd2/0x427
> [15715.216216]  [<ffffffff802e1a07>] ? block_ioctl+0x38/0x3c
> [15715.216219]  [<ffffffff802cc0de>] ? vfs_ioctl+0x21/0x6c
> [15715.216221]  [<ffffffff802cc554>] ? do_vfs_ioctl+0x42b/0x464
> [15715.216224]  [<ffffffff8022cce1>] ? do_page_fault+0x1f3/0x208
> [15715.216227]  [<ffffffff802cc5de>] ? sys_ioctl+0x51/0x70
> [15715.216230]  [<ffffffff8020fa42>] ? system_call_fastpath+0x16/0x1b
> 
> As you can see I am using Kernel 2.6.30 (Debian 2.6.30-8).
> 
> So, what to do now?

Hmm.... Probably the best thing to do is to update firmwares on those
drives as timing out on CPM is simply broken.  Another approach would
be hunting down who's invoking hdparm and smartctl and shut it down.

Thanks.

-- 
tejun

Re: Errors on ICH10R with Seagate HDDs

[Thomas Jackowski]

Tejun Heo wrote:
> Thomas Jackowski wrote:
>> Hi,
>>
>> sorry, I lost the reference in the mail headers. Please see:
>> http://www.spinics.net/lists/linux-ide/msg33340.html
>>
>> Well, I will post now the result with Tejun's debug patch:
>>
>> [15714.808535] XXX sdc: elv_insert(ATA_CHK_POWER)
>> [15714.808540] Pid: 22034, comm: smartctl Tainted: P           2.6.30 #1
>> [15714.808542] Call Trace:
>> [15714.808550]  [<ffffffff8033e456>] ? blk_plug_device+0x5a/0x83
>> [15714.808555]  [<ffffffff8033c2d2>] ? elv_insert+0x5e/0x2b0
>> [15714.808559]  [<ffffffff80341008>] ? blk_execute_rq_nowait+0x65/0x92
>> [15714.808562]  [<ffffffff803410c0>] ? blk_execute_rq+0x8b/0xab
>> [15714.808565]  [<ffffffff8033e831>] ? get_request_wait+0x21/0x17e
>> [15714.808569]  [<ffffffff8033da3d>] ? __freed_request+0x26/0x82
>> [15714.808572]  [<ffffffff8033dabc>] ? freed_request+0x23/0x43
>> [15714.808575]  [<ffffffff80344030>] ? sg_io+0x277/0x392
>> [15714.808578]  [<ffffffff803445df>] ? scsi_cmd_ioctl+0x1db/0x3a0
>> [15714.808583]  [<ffffffff80235c12>] ? check_preempt_wakeup+0x183/0x1dd
>> [15714.808587]  [<ffffffff802d350f>] ? __lookup_mnt+0x12/0x4b
>> [15714.808603]  [<ffffffffa00d9ae9>] ? sd_ioctl+0x9d/0xcb [sd_mod]
>> [15714.808606]  [<ffffffff803420b9>] ? __blkdev_driver_ioctl+0x69/0x7e
>> [15714.808615]  [<ffffffff803428d6>] ? blkdev_ioctl+0x7e4/0x81a
>> [15714.808618]  [<ffffffff802a41f3>] ? __do_fault+0x374/0x3af
>> [15714.808621]  [<ffffffff80297bbb>] ? __alloc_pages_internal+0xd2/0x427
>> [15714.808625]  [<ffffffff802e1a07>] ? block_ioctl+0x38/0x3c
>> [15714.808628]  [<ffffffff802cc0de>] ? vfs_ioctl+0x21/0x6c
>> [15714.808630]  [<ffffffff802cc554>] ? do_vfs_ioctl+0x42b/0x464
>> [15714.808634]  [<ffffffff8022cce1>] ? do_page_fault+0x1f3/0x208
>> [15714.808636]  [<ffffffff802cc5de>] ? sys_ioctl+0x51/0x70
>> [15714.808640]  [<ffffffff8020fa42>] ? system_call_fastpath+0x16/0x1b
>> [15715.216145] XXX sda: elv_insert(ATA_CHK_POWER)
>> [15715.216149] Pid: 22125, comm: hdparm Tainted: P           2.6.30 #1
>> [15715.216151] Call Trace:
>> [15715.216158]  [<ffffffff8033e456>] ? blk_plug_device+0x5a/0x83
>> [15715.216161]  [<ffffffff8033c2d2>] ? elv_insert+0x5e/0x2b0
>> [15715.216164]  [<ffffffff80341008>] ? blk_execute_rq_nowait+0x65/0x92
>> [15715.216167]  [<ffffffff803410c0>] ? blk_execute_rq+0x8b/0xab
>> [15715.216169]  [<ffffffff8033e831>] ? get_request_wait+0x21/0x17e
>> [15715.216173]  [<ffffffff803af118>] ? mix_pool_bytes_extract+0x57/0x14a
>> [15715.216176]  [<ffffffff80344030>] ? sg_io+0x277/0x392
>> [15715.216178]  [<ffffffff803445df>] ? scsi_cmd_ioctl+0x1db/0x3a0
>> [15715.216182]  [<ffffffff802d0180>] ? __d_lookup+0xc3/0x107
>> [15715.216185]  [<ffffffff802a1ab0>] ? zone_statistics+0x3c/0x5d
>> [15715.216188]  [<ffffffff802d350f>] ? __lookup_mnt+0x12/0x4b
>> [15715.216202]  [<ffffffffa00d9ae9>] ? sd_ioctl+0x9d/0xcb [sd_mod]
>> [15715.216204]  [<ffffffff803420b9>] ? __blkdev_driver_ioctl+0x69/0x7e
>> [15715.216207]  [<ffffffff803428d6>] ? blkdev_ioctl+0x7e4/0x81a
>> [15715.216210]  [<ffffffff802a41f3>] ? __do_fault+0x374/0x3af
>> [15715.216213]  [<ffffffff80297bbb>] ? __alloc_pages_internal+0xd2/0x427
>> [15715.216216]  [<ffffffff802e1a07>] ? block_ioctl+0x38/0x3c
>> [15715.216219]  [<ffffffff802cc0de>] ? vfs_ioctl+0x21/0x6c
>> [15715.216221]  [<ffffffff802cc554>] ? do_vfs_ioctl+0x42b/0x464
>> [15715.216224]  [<ffffffff8022cce1>] ? do_page_fault+0x1f3/0x208
>> [15715.216227]  [<ffffffff802cc5de>] ? sys_ioctl+0x51/0x70
>> [15715.216230]  [<ffffffff8020fa42>] ? system_call_fastpath+0x16/0x1b
>>
>> As you can see I am using Kernel 2.6.30 (Debian 2.6.30-8).
>>
>> So, what to do now?
> 
> Hmm.... Probably the best thing to do is to update firmwares on those
> drives as timing out on CPM is simply broken.  Another approach would
> be hunting down who's invoking hdparm and smartctl and shut it down.
> 
> Thanks.
> 

Hi Tejun,

thanks for your answer.

I already thought about the second approach, but the thing is: the
latest firmware upgrade broke my hdds and that is the actual situation.

So, can you please give me some hints for non-kernel-hackers so I can
tell those Seagate support? Would be very gentle.

Btw: I think smartd is invoking smartctl. The data is provided to munin.

Thanks, Thomas