VIA VT6420: SATA disconnects

VIA VT6420: SATA disconnects

[Vasily Averin]

Jeff, Tejun,

Our RHEL5-based OpenVZ linux kernel reports about SATA-related issues:
VIA VT6420 SATA RAID Controller on MSI motherboard, x86_64 kernel based on latest RHEL5 kernel,
On booting hardware initialized properly and all works fine some time, but then it detects timeout and disables devices. We have replaced SATA cables, but issue didn't go away and still present.

I've googled and found similair bugreport in linux-ide@
http://www.mail-archive.com/linux-ide@vger.kernel.org/msg06011.html

Are you know something about this issue? I've seen that you have fixed SATA reset procedure recently, probably this issue was fixed already?

thank you,
	Vasily Averin

OpenVZ/Virtuozzo Linux kernel Team

May 24 09:39:39 ts28 SCSI subsystem initialized
May 24 09:39:39 ts28 libata version 2.00 loaded.
May 24 09:39:39 ts28 sata_via 0000:00:0f.0: version 2.0
May 24 09:39:39 ts28 ACPI: PCI Interrupt 0000:00:0f.0[B] -> Link [ALKA] -> GSI 20 (level, low) -> IRQ 169
May 24 09:39:39 ts28 sata_via 0000:00:0f.0: routed to hard irq line 11
May 24 09:39:39 ts28 ata1: SATA max UDMA/133 cmd 0xC000 ctl 0xC402 bmdma 0xD000 irq 169
May 24 09:39:39 ts28 ata2: SATA max UDMA/133 cmd 0xC800 ctl 0xCC02 bmdma 0xD008 irq 169
May 24 09:39:39 ts28 scsi0 : sata_via
May 24 09:39:39 ts28 ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
May 24 09:39:40 ts28 ata1.00: ATA-7, max UDMA/133, 156301488 sectors: LBA48 NCQ (depth 0/32)
May 24 09:39:40 ts28
May 24 09:39:40 ts28 ata1.00: ata1: dev 0 multi count 16
May 24 09:39:40 ts28 ata1.00: configured for UDMA/133
May 24 09:39:40 ts28 scsi1 : sata_via
May 24 09:39:40 ts28 ata2: SATA link down 1.5 Gbps (SStatus 0 SControl 300)
May 24 09:39:40 ts28 ATA: abnormal status 0x7F on port 0xC807
May 24 09:39:40 ts28   Vendor: ATA       Model: ST380811AS        Rev: 3.AA
May 24 09:39:40 ts28   Type:   Direct-Access                      ANSI SCSI revision: 05
May 24 09:39:40 ts28 SCSI device sda: 156301488 512-byte hdwr sectors (80026 MB)
May 24 09:39:40 ts28 sda: Write Protect is off
May 24 09:39:40 ts28 sda: Mode Sense: 00 3a 00 00
May 24 09:39:40 ts28 SCSI device sda: drive cache: write back
May 24 09:39:40 ts28 SCSI device sda: 156301488 512-byte hdwr sectors (80026 MB)
May 24 09:39:40 ts28 sda: Write Protect is off
May 24 09:39:40 ts28 sda: Mode Sense: 00 3a 00 00
May 24 09:39:40 ts28
May 24 09:39:40 ts28 SCSI device sda: drive cache: write back
May 24 09:39:40 ts28  sda: sda1 sda2 sda3 sda4 < sda5 >
May 24 09:39:40 ts28 sd 0:0:0:0: Attached scsi disk sda
May 24 09:39:43 ts28 kjournald starting.  Commit interval 5 seconds
May 24 09:39:43 ts28 EXT3-fs: mounted filesystem with ordered data mode.
May 24 09:53:15 ts28 ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x2 frozen
May 24 09:53:15 ts28 ata1.00: (BMDMA stat 0x4)
May 24 09:53:15 ts28 ata1.00: tag 0 cmd 0xca Emask 0x4 stat 0x40 err 0x0 (timeout)
May 24 09:53:46 ts28 ata1.00: qc timeout (cmd 0xec)
May 24 09:53:46 ts28 ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4)
May 24 09:53:46 ts28 ata1.00: revalidation failed (errno=-5)
May 24 09:53:46 ts28 ata1: failed to recover some devices, retrying in 5 secs
May 24 09:54:23 ts28 ata1.00: qc timeout (cmd 0xec)
May 24 09:54:23 ts28 ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4)
May 24 09:54:23 ts28 ata1.00: revalidation failed (errno=-5)
May 24 09:54:23 ts28 ata1: failed to recover some devices, retrying in 5 secs
May 24 09:54:59 ts28 ata1.00: qc timeout (cmd 0xec)
May 24 09:54:59 ts28 ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4)
May 24 09:54:59 ts28 ata1.00: revalidation failed (errno=-5)
May 24 09:54:59 ts28 ata1.00: disabled


Linux ts28 2.6.18-028stab031.1 #1 SMP Fri Apr 27 18:39:46 MSD 2007 x86_64 x86_64 x86_64 GNU/Linux

00:0f.0 RAID bus controller: VIA Technologies, Inc. VIA VT6420 SATA RAID Controller (rev 80)
        Subsystem: Micro-Star International Co., Ltd. Unknown device 1300
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
        Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
        Latency: 32
        Interrupt: pin B routed to IRQ 169
        Region 0: I/O ports at c000 [size=8]
        Region 1: I/O ports at c400 [size=4]
        Region 2: I/O ports at c800 [size=8]
        Region 3: I/O ports at cc00 [size=4]
        Region 4: I/O ports at d000 [size=16]
        Region 5: I/O ports at d400 [size=256]
        Capabilities: [c0] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
00: 06 11 49 31 07 00 90 02 80 00 04 01 00 20 80 00
10: 01 c0 00 00 01 c4 00 00 01 c8 00 00 01 cc 00 00
20: 01 d0 00 00 01 d4 00 00 00 00 00 00 62 14 00 13
30: 00 00 00 00 c0 00 00 00 00 00 00 00 0b 02 00 00
40: 33 03 f1 44 06 af 00 00 10 82 65 03 00 00 00 00
50: 00 00 00 00 00 00 04 04 00 10 10 00 05 00 20 00
60: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 01 00 01 10 10 00 00 00
80: 00 00 00 00 00 00 00 00 00 30 c6 de 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 80 02 49 31 62 14 00 13 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Re: VIA VT6420: SATA disconnects

[Jeff Garzik]

Vasily Averin wrote:
> Jeff, Tejun,
> 
> Our RHEL5-based OpenVZ linux kernel reports about SATA-related issues:
> VIA VT6420 SATA RAID Controller on MSI motherboard, x86_64 kernel based on latest RHEL5 kernel,
> On booting hardware initialized properly and all works fine some time, but then it detects timeout and disables devices. We have replaced SATA cables, but issue didn't go away and still present.
> 
> I've googled and found similair bugreport in linux-ide@
> http://www.mail-archive.com/linux-ide@vger.kernel.org/msg06011.html
> 
> Are you know something about this issue? I've seen that you have fixed SATA reset procedure recently, probably this issue was fixed already?

RHEL5 SATA is unfortunately way out of date :(  The next RHEL5 update 
should include a boatload of fixes.

Try running the latest upstream kernel (2.6.21.3 or 2.6.22-rc2-git7), 
and see if the problem is reproducible.

	Jeff

Re: VIA VT6420: SATA disconnects

[Vasily Averin]

Jeff Garzik wrote:
> RHEL5 SATA is unfortunately way out of date :(  The next RHEL5 update 
> should include a boatload of fixes.

Is SATA update included into RHEL5 testkernels? And do you probably know if they are accessible somewhere (like in http://people.redhat.com/~jbaron/rhel4/ for RHEL4 testkernels)?

thank you,
	Vasily Averin

Re: VIA VT6420: SATA disconnects

[Jan Evert van Grootheest]

Jeff Garzik wrote:
> Vasily Averin wrote:
>> Jeff, Tejun,
>>
>> Our RHEL5-based OpenVZ linux kernel reports about SATA-related issues:
>> VIA VT6420 SATA RAID Controller on MSI motherboard, x86_64 kernel 
>> based on latest RHEL5 kernel,
>> On booting hardware initialized properly and all works fine some 
>> time, but then it detects timeout and disables devices. We have 
>> replaced SATA cables, but issue didn't go away and still present.
>>
>> I've googled and found similair bugreport in linux-ide@
>> http://www.mail-archive.com/linux-ide@vger.kernel.org/msg06011.html
>>
>> Are you know something about this issue? I've seen that you have 
>> fixed SATA reset procedure recently, probably this issue was fixed 
>> already?
>
> RHEL5 SATA is unfortunately way out of date :(  The next RHEL5 update 
> should include a boatload of fixes.
>
> Try running the latest upstream kernel (2.6.21.3 or 2.6.22-rc2-git7), 
> and see if the problem is reproducible.
>
Jeff,

In the meantime I've taken that disk out of use (although it is still in 
that same machine and connected). So I can easily run tests on it.
I'm not sure I want to build my own (somewhat recent) kernel, because 
that machine serves as our home server. It takes some engineering to 
find time where the family is gone and nobody needs it.
Also it uses xen and I don't know whether I can find the proper patches 
to get it to compile (I find it difficult in debian to find the patches 
that were used to produce a kernel). But if nobody uses it, I could do 
with a xen-less kernel. Hopefully bonnie or somesuch will make the 
problem appear.

Like Vasily I also had it connected to the VIA controller. But the 
problems also appeared when the disk was connected to the Promise 
controller on the same board. So I would, at first sight, not consider 
this a controller issue.

-- Jan Evert

Re: VIA VT6420: SATA disconnects

[Vasily Averin]

Jeff Garzik wrote:
> Vasily Averin wrote:
>> Jeff, Tejun,
>>
>> Our RHEL5-based OpenVZ linux kernel reports about SATA-related issues:
>> VIA VT6420 SATA RAID Controller on MSI motherboard, x86_64 kernel based on latest RHEL5 kernel,
>> On booting hardware initialized properly and all works fine some time, but then it detects timeout and disables devices. We have replaced SATA cables, but issue didn't go away and still present.
>>
>> I've googled and found similair bugreport in linux-ide@
>> http://www.mail-archive.com/linux-ide@vger.kernel.org/msg06011.html
>>
>> Are you know something about this issue? I've seen that you have fixed SATA reset procedure recently, probably this issue was fixed already?
> 
> RHEL5 SATA is unfortunately way out of date :(  The next RHEL5 update 
> should include a boatload of fixes.
> 
> Try running the latest upstream kernel (2.6.21.3 or 2.6.22-rc2-git7), 
> and see if the problem is reproducible.

I've reproduced this issue. But on this kernel EH works well and node is still alive:

Linux version 2.6.22-rc4 (vvs@vvs.work.ve) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-3)) #1 SMP Fri Jun 8 14:32:01 MSD 2007
...

hda: lost interrupt
ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x2 frozen
ata1.00: cmd ca/00:78:30:1a:24/00:00:00:00:00/e2 tag 0 cdb 0x0 data 61440 out
         res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
ata1: soft resetting port
ATA: abnormal status 0x7F on port 0x000000000001c007
ATA: abnormal status 0x7F on port 0x000000000001c007
ata1.00: qc timeout (cmd 0x27)
ata1.00: ata_hpa_resize 1: sectors = 156301488, hpa_sectors = 0
ata1.00: failed to set xfermode (err_mask=0x40)
ata1: failed to recover some devices, retrying in 5 secs
ata1: soft resetting port
ATA: abnormal status 0x7F on port 0x000000000001c007
ATA: abnormal status 0x7F on port 0x000000000001c007
ata1.00: qc timeout (cmd 0x27)
ata1.00: ata_hpa_resize 1: sectors = 156301488, hpa_sectors = 0
ata1.00: failed to set xfermode (err_mask=0x40)
ata1.00: limiting speed to UDMA/133:PIO3
ata1: failed to recover some devices, retrying in 5 secs
ata1: soft resetting port
ATA: abnormal status 0x7F on port 0x000000000001c007
ATA: abnormal status 0x7F on port 0x000000000001c007
ata1.00: qc timeout (cmd 0x27)
ata1.00: ata_hpa_resize 1: sectors = 156301488, hpa_sectors = 0
ata1.00: failed to set xfermode (err_mask=0x40)
ata1.00: disabled
ata1: EH complete
sd 0:0:0:0: [sda] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK

You can find some additional details in bug #8650
http://bugzilla.kernel.org/show_bug.cgi?id=8650
 
thank you,
	Vasily Averin