problems with sata_nv and MV88SX5081 under 2.6.28, fine under2.6.17.6

problems with sata_nv and MV88SX5081 under 2.6.28, fine under2.6.17.6

[Eamonn Hamilton]

Hi,

I look after a server with a bunch of drives hooked up to two 8 port
PCI-X sata cards, detected as above.

The server was running 2.6.17.6 which has sata_mv version 0.7, which
worked fine. I recently upgraded the kernel to 2.6.18 with version 1.24,
and started getting

ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen

for several of the disks, and performance went through the floor because
of multiple bus resets. Now, the system was running SMART monitoring,
and in the course of some googling it was suggested that this perhaps
was causing a timeout, however disabling the smartd made no difference.

Reverting to 2.6.17.6 made the problem go away, suggesting something
between 0.7 and 1.24 is causing some difficulty.

Before I launch into trying to find out where in the interim the
regression occurred, does anybody have any suggestions?

Cheers,
Eamonn

System details :

00:00.0 Host bridge: Intel Corporation E7501 Memory Controller Hub (rev
01)
00:00.1 Class ff00: Intel Corporation E7500/E7501 Host RASUM Controller
(rev 01)
00:03.0 PCI bridge: Intel Corporation E7500/E7501 Hub Interface C
PCI-to-PCI Bridge (rev 01)
00:03.1 Class ff00: Intel Corporation E7500/E7501 Hub Interface C RASUM
Controller (rev 01)
00:1d.0 USB Controller: Intel Corporation 82801CA/CAM USB (Hub #1) (rev
02)
00:1d.1 USB Controller: Intel Corporation 82801CA/CAM USB (Hub #2) (rev
02)
00:1d.2 USB Controller: Intel Corporation 82801CA/CAM USB (Hub #3) (rev
02)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 42)
00:1f.0 ISA bridge: Intel Corporation 82801CA LPC Interface Controller
(rev 02)
00:1f.1 IDE interface: Intel Corporation 82801CA Ultra ATA Storage
Controller (rev 02)
00:1f.3 SMBus: Intel Corporation 82801CA/CAM SMBus Controller (rev 02)
01:04.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
Controller (rev 02)
01:0c.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
02:1c.0 PIC: Intel Corporation 82870P2 P64H2 I/OxAPIC (rev 04)
02:1d.0 PCI bridge: Intel Corporation 82870P2 P64H2 Hub PCI Bridge (rev
04)
02:1e.0 PIC: Intel Corporation 82870P2 P64H2 I/OxAPIC (rev 04)
02:1f.0 PCI bridge: Intel Corporation 82870P2 P64H2 Hub PCI Bridge (rev
04)
03:02.0 SCSI storage controller: Marvell Technology Group Ltd.
MV88SX5081 8-port SATA I PCI-X Controller (rev 03)
03:03.0 SCSI storage controller: Adaptec AIC-7901A U320 (rev 03)
04:02.0 SCSI storage controller: Marvell Technology Group Ltd.
MV88SX5081 8-port SATA I PCI-X Controller (rev 03)


disks :Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: ST3400832AS      Rev: 3.02
  Type:   Direct-Access                    ANSI SCSI revision: 05
Host: scsi1 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: ST3400833AS      Rev: 3.AA
  Type:   Direct-Access                    ANSI SCSI revision: 05
Host: scsi2 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: ST3400832AS      Rev: 3.02
  Type:   Direct-Access                    ANSI SCSI revision: 05
Host: scsi3 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: WDC WD4000YR-01P Rev: 01.0
  Type:   Direct-Access                    ANSI SCSI revision: 05
Host: scsi5 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: ST3400832AS      Rev: 3.02
  Type:   Direct-Access                    ANSI SCSI revision: 05
Host: scsi8 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: ST3400832AS      Rev: 3.06
  Type:   Direct-Access                    ANSI SCSI revision: 05
Host: scsi10 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: ST3400832AS      Rev: 3.03
  Type:   Direct-Access                    ANSI SCSI revision: 05
Host: scsi11 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: ST3400620NS      Rev: 3.AE
  Type:   Direct-Access                    ANSI SCSI revision: 05
Host: scsi13 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: ST3400832AS      Rev: 3.03
  Type:   Direct-Access                    ANSI SCSI revision: 05

 




SAIC Limited is a private limited company registered in England and Wales. Registered number 1396396. Registered office at 120 New Cavendish Street, London, W1W 6XX. VAT number 599 5474 64.
This e-mail and any attachments are private and confidential.  If you are not the intended recipient of this e-mail, any disclosure, copying, distribution or use of its contents is strictly prohibited. Please notify the sender immediately and then delete it (including any attachments) from your system.
All emails and attachments are virus scanned. It is your responsibility to ensure that any onward transmission, opening or use of this message and any attachments will not adversely affect your or the onward recipients' systems or data. Please carry out such virus and other such checks as you consider appropriate.
SAIC Limited may monitor email traffic data and, also, the content of email for the purposes of security, staff training and compliance with SAIC policies.

Re: problems with sata_nv and MV88SX5081 under 2.6.28, fine under2.6.17.6

[Alan Cox]

On Wed, 07 Jan 2009 14:53:25 +0000
"Eamonn Hamilton" <EAMONN.HAMILTON@saic.com> wrote:

> Hi,
> 
> I look after a server with a bunch of drives hooked up to two 8 port
> PCI-X sata cards, detected as above.
> 
> The server was running 2.6.17.6 which has sata_mv version 0.7, which
> worked fine. I recently upgraded the kernel to 2.6.18 with version 1.24,
> and started getting

2.6.18 or 2.6.28 ? - if its 2.6.18 then you might want to check if 2.6.28
is ok before doing the work.

Re: problems with sata_nv and MV88SX5081 under 2.6.28, fine under2.6.17.6

[Eamonn Hamilton]

2.6.28, unfortunately :(

I've really got to get a handle on this typing thing ...


On Wed, 2009-01-07 at 15:45 +0000, Alan Cox wrote:
> On Wed, 07 Jan 2009 14:53:25 +0000
> "Eamonn Hamilton" <EAMONN.HAMILTON@saic.com> wrote:
> 
> > Hi,
> > 
> > I look after a server with a bunch of drives hooked up to two 8 port
> > PCI-X sata cards, detected as above.
> > 
> > The server was running 2.6.17.6 which has sata_mv version 0.7, which
> > worked fine. I recently upgraded the kernel to 2.6.18 with version 1.24,
> > and started getting
> 
> 2.6.18 or 2.6.28 ? - if its 2.6.18 then you might want to check if 2.6.28
> is ok before doing the work.


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Re: problems with sata_nv and MV88SX5081 under 2.6.28, fine under2.6.17.6

[Robert Hancock]

Eamonn Hamilton wrote:
> Hi,
> 
> I look after a server with a bunch of drives hooked up to two 8 port
> PCI-X sata cards, detected as above.
> 
> The server was running 2.6.17.6 which has sata_mv version 0.7, which
> worked fine. I recently upgraded the kernel to 2.6.18 with version 1.24,
> and started getting
> 
> ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen

Can you post the full output (that part of the message is not very useful..)

Re: problems with sata_nv and MV88SX5081 under 2.6.28, fineunder2.6.17.6

[Eamonn Hamilton]

Sure

Jan  5 18:37:47 talis kernel: ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Jan  5 18:37:47 talis kernel: ata4.00: cmd 35/00:08:08:90:93/00:00:2e:00:00/e0 tag 0 dma 4096 out
Jan  5 18:37:47 talis kernel:          res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Jan  5 18:37:47 talis kernel: ata4.00: status: { DRDY }
Jan  5 18:37:47 talis kernel: ata4: hard resetting link
Jan  5 18:37:47 talis kernel: ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
Jan  5 18:37:47 talis kernel: ata4.00: configured for UDMA/133
Jan  5 18:37:47 talis kernel: ata4: EH complete
Jan  5 18:37:47 talis kernel: sd 3:0:0:0: [sdd] 781422768 512-byte hardware sectors: (400 GB/372 GiB)
Jan  5 18:37:47 talis kernel: sd 3:0:0:0: [sdd] Write Protect is off
Jan  5 18:37:47 talis kernel: sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00
Jan  5 18:37:48 talis kernel: sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jan  5 18:37:48 talis kernel: svc: 127.0.0.1, port=56936: unknown version (3 for prog 100003, nfsd)
Jan  5 18:39:29 talis kernel: ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Jan  5 18:39:29 talis kernel: ata4.00: cmd 35/00:08:08:90:93/00:00:2e:00:00/e0 tag 0 dma 4096 out
Jan  5 18:39:29 talis kernel:          res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Jan  5 18:39:29 talis kernel: ata4.00: status: { DRDY }
Jan  5 18:39:29 talis kernel: ata4: hard resetting link
Jan  5 18:39:30 talis kernel: ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
Jan  5 18:39:30 talis kernel: ata4.00: configured for UDMA/133
Jan  5 18:39:30 talis kernel: ata4: EH complete

This is immediately after booting.

Regards,
Eamonn



On Wed, 2009-01-07 at 21:13 -0600, Robert Hancock wrote:
> Eamonn Hamilton wrote:
> > Hi,
> > 
> > I look after a server with a bunch of drives hooked up to two 8 port
> > PCI-X sata cards, detected as above.
> > 
> > The server was running 2.6.17.6 which has sata_mv version 0.7, which
> > worked fine. I recently upgraded the kernel to 2.6.18 with version 1.24,
> > and started getting
> > 
> > ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
> 
> Can you post the full output (that part of the message is not very useful..)

SAIC Limited is a private limited company registered in England and Wales. Registered number 1396396. Registered office at 120 New Cavendish Street, London, W1W 6XX. VAT number 599 5474 64.
This e-mail and any attachments are private and confidential.  If you are not the intended recipient of this e-mail, any disclosure, copying, distribution or use of its contents is strictly prohibited. Please notify the sender immediately and then delete it (including any attachments) from your system.
All emails and attachments are virus scanned. It is your responsibility to ensure that any onward transmission, opening or use of this message and any attachments will not adversely affect your or the onward recipients' systems or data. Please carry out such virus and other such checks as you consider appropriate.
SAIC Limited may monitor email traffic data and, also, the content of email for the purposes of security, staff training and compliance with SAIC policies.

Re: problems with sata_nv and MV88SX5081 under 2.6.28, fine under2.6.17.6

[Mark Lord]

Eamonn Hamilton wrote:
> Hi,
> 
> I look after a server with a bunch of drives hooked up to two 8 port
> PCI-X sata cards, detected as above.
> 
> The server was running 2.6.17.6 which has sata_mv version 0.7, which
> worked fine. I recently upgraded the kernel to 2.6.18 with version 1.24,
> and started getting
> 
> ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
> 
> for several of the disks, and performance went through the floor because
> of multiple bus resets. Now, the system was running SMART monitoring,
> and in the course of some googling it was suggested that this perhaps
> was causing a timeout, however disabling the smartd made no difference.
> 
> Reverting to 2.6.17.6 made the problem go away, suggesting something
> between 0.7 and 1.24 is causing some difficulty.
..

Yeah, there are vaguely similar reports from som of the 88SX6081 users as well.
I think the restructuring of the reset/probe code last year, to share more
of the libata core/sff stuff, probably messes up the older chipsets like those.

I'm looking into now, so please copy me on any extra postings on this topic.

Eamonn, are you available to test updates early, before they go upstream?

Thanks.

Mark

Re: problems with sata_nv and MV88SX5081 under 2.6.28, fine under2.6.17.6

[Eamonn Hamilton]

Hi Mark.

It bisected down to the following:

e3472cbe5c10a91c737405cd706142787736392c is first bad commit
commit e3472cbe5c10a91c737405cd706142787736392c
Author: Albert Lee <albertcc@tw.ibm.com>
Date:   Thu Dec 7 11:37:58 2006 +0800

    [PATCH] libata: let ATA_FLAG_PIO_POLLING use polling pio for
ATA_PROT_NODATA
    
    Even if ATA_FLAG_PIO_POLLING is set, libata uses irq pio for the
ATA_PROT_NODATA protocol.
    This patch let ATA_FLAG_PIO_POLLING use polling pio for the
ATA_PROT_NODATA protocol.
    
    Signed-off-by: Albert Lee <albertcc@tw.ibm.com>
    Signed-off-by: Jeff Garzik <jeff@garzik.org>


Now, why it dies after that, I don't know, it's one flag being added.

I guess I could try the latest kernel with that removed and see what
happens?

Cheers,
Eamonn








On Mon, 2009-01-12 at 15:37 -0500, Mark Lord wrote:
> Hamilton, Eamonn [OS-IE] wrote:
> > 
> > 
> > With any luck I'll be able to bisect it down in the next day or so, git 
> > willing - I've never used the thing, but I hear good reports ;)
> ..
> 
> Okay, no changes whatsoever in sata_mv between .19 and .20.
> Are you sure about those two kernel versions?
> 
> Cheers


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Re: problems with sata_mv and MV88SX5081 under 2.6.28, fine under 2.6.17.6

[Mark Lord]

Eamonn Hamilton wrote:
> Hi Mark.
> 
> It bisected down to the following:
> 
> e3472cbe5c10a91c737405cd706142787736392c is first bad commit
> commit e3472cbe5c10a91c737405cd706142787736392c
> Author: Albert Lee <albertcc@tw.ibm.com>
> Date:   Thu Dec 7 11:37:58 2006 +0800
> 
>libata: let ATA_FLAG_PIO_POLLING use polling pio for ATA_PROT_NODATA
>
>Even if ATA_FLAG_PIO_POLLING is set, libata uses irq pio for the ATA_PROT_NODATA protocol.
>This patch let ATA_FLAG_PIO_POLLING use polling pio for the ATA_PROT_NODATA protocol.
>
>Signed-off-by: Albert Lee <albertcc@tw.ibm.com>
>Signed-off-by: Jeff Garzik <jeff@garzik.org>
>---
..
>diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
>index 8816e30..d2e6863 100644
>--- a/drivers/ata/libata-core.c
>+++ b/drivers/ata/libata-core.c
>@@ -4960,6 +4960,7 @@ unsigned int ata_qc_issue_prot(struct ata_queued_cmd *qc)
> 	if (ap->flags & ATA_FLAG_PIO_POLLING) {
> 		switch (qc->tf.protocol) {
> 		case ATA_PROT_PIO:
>+		case ATA_PROT_NODATA:
> 		case ATA_PROT_ATAPI:
> 		case ATA_PROT_ATAPI_NODATA:
> 			qc->tf.flags |= ATA_TFLAG_POLLING;
> 
..
> Now, why it dies after that, I don't know, it's one flag being added.
> 
> I guess I could try the latest kernel with that removed and see what happens?
..

You certainly could.  That routine is now in libata-sff.c,
but otherwise looks the same as you found it.

Mmm.. I seem to recall seeing some funny biz in this area when
I was working on sata_mv last year.. time to fire up the test rig
and poke around more at this stuff.

Ideally, I'd like to tell sata_mv to use IRQs for all commands,
rather than the polling that the ATA_FLAG_PIO_POLLING induces.
But there was something funny about it at the time, so I left it.

Cheers

Re: problems with sata_mv and MV88SX5081 under 2.6.28, fine under 2.6.17.6

[Mark Lord]

> Eamonn Hamilton wrote:
>> Hi Mark.
>>
>> It bisected down to the following:
>>
>> e3472cbe5c10a91c737405cd706142787736392c is first bad commit
>> commit e3472cbe5c10a91c737405cd706142787736392c
>> Author: Albert Lee <albertcc@tw.ibm.com>
>> Date:   Thu Dec 7 11:37:58 2006 +0800
>>
>> libata: let ATA_FLAG_PIO_POLLING use polling pio for ATA_PROT_NODATA
>>
>> Even if ATA_FLAG_PIO_POLLING is set, libata uses irq pio for the 
>> ATA_PROT_NODATA protocol.
>> This patch let ATA_FLAG_PIO_POLLING use polling pio for the 
>> ATA_PROT_NODATA protocol.
>>
>> Signed-off-by: Albert Lee <albertcc@tw.ibm.com>
>> Signed-off-by: Jeff Garzik <jeff@garzik.org>
>> ---
> ..
>> diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
>> index 8816e30..d2e6863 100644
>> --- a/drivers/ata/libata-core.c
>> +++ b/drivers/ata/libata-core.c
>> @@ -4960,6 +4960,7 @@ unsigned int ata_qc_issue_prot(struct 
>> ata_queued_cmd *qc)
>>     if (ap->flags & ATA_FLAG_PIO_POLLING) {
>>         switch (qc->tf.protocol) {
>>         case ATA_PROT_PIO:
>> +        case ATA_PROT_NODATA:
>>         case ATA_PROT_ATAPI:
>>         case ATA_PROT_ATAPI_NODATA:
>>             qc->tf.flags |= ATA_TFLAG_POLLING;
>>
> ..
>> Now, why it dies after that, I don't know, it's one flag being added.
..

Eamonn,

Have you tried testing with the latest sata_mv fix?
This fixes interrupt problems with all 8-port sata_mv chips,
including the one you have access to.

Please give this patch a try and report back.  Thanks.
The patch is against 2.6.28, but will probably apply
to several earlier kernel versions as well.

--- linux-2.6.28/drivers/ata/sata_mv.c	2009-01-13 15:57:11.000000000 -0500
+++ linux/drivers/ata/sata_mv.c	2009-01-14 16:56:38.000000000 -0500
@@ -883,7 +883,7 @@
 		struct mv_host_priv *hpriv = ap->host->private_data;
 		int hardport = mv_hardport_from_port(ap->port_no);
 		void __iomem *hc_mmio = mv_hc_base_from_port(
-					mv_host_base(ap->host), hardport);
+					mv_host_base(ap->host), ap->port_no);
 		u32 hc_irq_cause, ipending;
 
 		/* clear EDMA event indicators, if any */

Re: problems with sata_mv and MV88SX5081 under 2.6.28, fine under 2.6.17.6

[Eamonn Hamilton]

Hi Mark,

The fix does indeed allow the cards to function normally under 2.6.28.

many thanks for all the work - now all I have to do is try and figure
out how to recover this array without going to backup if I can help
it :(

Oh well, if it as easy, everybody would be doing it :)


On Thu, 2009-01-15 at 14:37 -0500, Mark Lord wrote:
> > Eamonn Hamilton wrote:
> >> Hi Mark.
> >>
> >> It bisected down to the following:
> >>
> >> e3472cbe5c10a91c737405cd706142787736392c is first bad commit
> >> commit e3472cbe5c10a91c737405cd706142787736392c
> >> Author: Albert Lee <albertcc@tw.ibm.com>
> >> Date:   Thu Dec 7 11:37:58 2006 +0800
> >>
> >> libata: let ATA_FLAG_PIO_POLLING use polling pio for ATA_PROT_NODATA
> >>
> >> Even if ATA_FLAG_PIO_POLLING is set, libata uses irq pio for the 
> >> ATA_PROT_NODATA protocol.
> >> This patch let ATA_FLAG_PIO_POLLING use polling pio for the 
> >> ATA_PROT_NODATA protocol.
> >>
> >> Signed-off-by: Albert Lee <albertcc@tw.ibm.com>
> >> Signed-off-by: Jeff Garzik <jeff@garzik.org>
> >> ---
> > ..
> >> diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
> >> index 8816e30..d2e6863 100644
> >> --- a/drivers/ata/libata-core.c
> >> +++ b/drivers/ata/libata-core.c
> >> @@ -4960,6 +4960,7 @@ unsigned int ata_qc_issue_prot(struct 
> >> ata_queued_cmd *qc)
> >>     if (ap->flags & ATA_FLAG_PIO_POLLING) {
> >>         switch (qc->tf.protocol) {
> >>         case ATA_PROT_PIO:
> >> +        case ATA_PROT_NODATA:
> >>         case ATA_PROT_ATAPI:
> >>         case ATA_PROT_ATAPI_NODATA:
> >>             qc->tf.flags |= ATA_TFLAG_POLLING;
> >>
> > ..
> >> Now, why it dies after that, I don't know, it's one flag being added.
> ..
> 
> Eamonn,
> 
> Have you tried testing with the latest sata_mv fix?
> This fixes interrupt problems with all 8-port sata_mv chips,
> including the one you have access to.
> 
> Please give this patch a try and report back.  Thanks.
> The patch is against 2.6.28, but will probably apply
> to several earlier kernel versions as well.
> 
> --- linux-2.6.28/drivers/ata/sata_mv.c	2009-01-13 15:57:11.000000000 -0500
> +++ linux/drivers/ata/sata_mv.c	2009-01-14 16:56:38.000000000 -0500
> @@ -883,7 +883,7 @@
>  		struct mv_host_priv *hpriv = ap->host->private_data;
>  		int hardport = mv_hardport_from_port(ap->port_no);
>  		void __iomem *hc_mmio = mv_hc_base_from_port(
> -					mv_host_base(ap->host), hardport);
> +					mv_host_base(ap->host), ap->port_no);
>  		u32 hc_irq_cause, ipending;
>  
>  		/* clear EDMA event indicators, if any */
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ide" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.