Marvell MV88SX6041 SATA Driver PIO4 Mode Not DMA

Marvell MV88SX6041 SATA Driver PIO4 Mode Not DMA

[Spencer Tuttle]

I have just compiled the new 2.6.14 kernel from the gentoo-sources tree.

I can access the drives just fine, but it seems really slow.  Here is
the dmesg output when I load the kernel module

libata version 1.12 loaded.
sata_mv version 0.12
PCI: setting IRQ 9 as level-triggered
ACPI: PCI Interrupt 0000:03:04.0[A] -> GSI 9 (level, low) -> IRQ 9
ata1: SATA max PIO4 cmd 0x0 ctl 0xF89A2120 bmdma 0x0 irq 9
ata2: SATA max PIO4 cmd 0x0 ctl 0xF89A4120 bmdma 0x0 irq 9
ata3: SATA max PIO4 cmd 0x0 ctl 0xF89A6120 bmdma 0x0 irq 9
ata4: SATA max PIO4 cmd 0x0 ctl 0xF89A8120 bmdma 0x0 irq 9
ata1: dev 0 cfg 49:2f00 82:7869 83:7d09 84:4003 85:7869 86:3c01 87:4003
88:203f
ata1: dev 0 ATA, max UDMA/100, 156301488 sectors: lba48
ata1: dev 0 configured for PIO4
scsi0 : sata_mv
ata2: dev 0 cfg 49:2f00 82:7869 83:7d09 84:4003 85:7869 86:3c01 87:4003
88:203f
ata2: dev 0 ATA, max UDMA/100, 156301488 sectors: lba48
ata2: dev 0 configured for PIO4
scsi1 : sata_mv
ata3: dev 0 cfg 49:2f00 82:7869 83:7d09 84:4003 85:7869 86:3c01 87:4003
88:203f
ata3: dev 0 ATA, max UDMA/100, 156301488 sectors: lba48
ata3: dev 0 configured for PIO4
scsi2 : sata_mv
ata4: dev 0 cfg 49:2f00 82:7869 83:7d09 84:4003 85:7869 86:3c01 87:4003
88:203f
ata4: dev 0 ATA, max UDMA/100, 156301488 sectors: lba48
ata4: dev 0 configured for PIO4
scsi3 : sata_mv
blk_queue_max_hw_segments: set to minimum 1
  Vendor: ATA       Model: Maxtor 6Y080M0    Rev: YAR5
  Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sda: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sda: drive cache: write back
SCSI device sda: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sda: drive cache: write back
 sda: sda1
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
Attached scsi generic sg0 at scsi0, channel 0, id 0, lun 0,  type 0
blk_queue_max_hw_segments: set to minimum 1
=============<repeated 31 more times>==================

  Vendor: ATA       Model: Maxtor 6Y080M0    Rev: YAR5
  Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sdb: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sdb: drive cache: write back
SCSI device sdb: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sdb: drive cache: write back
 sdb: sdb1
Attached scsi disk sdb at scsi1, channel 0, id 0, lun 0
Attached scsi generic sg1 at scsi1, channel 0, id 0, lun 0,  type 0
blk_queue_max_hw_segments: set to minimum 1
=============<repeated 31 more times>==================

  Vendor: ATA       Model: Maxtor 6Y080M0    Rev: YAR5
  Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sdc: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sdc: drive cache: write back
SCSI device sdc: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sdc: drive cache: write back
 sdc: sdc1
Attached scsi disk sdc at scsi2, channel 0, id 0, lun 0
Attached scsi generic sg2 at scsi2, channel 0, id 0, lun 0,  type 0
blk_queue_max_hw_segments: set to minimum 1
=============<repeated 31 more times>==================

  Vendor: ATA       Model: Maxtor 6Y080M0    Rev: YAR5
  Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sdd: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sdd: drive cache: write back
SCSI device sdd: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sdd: drive cache: write back
 sdd: sdd1
Attached scsi disk sdd at scsi3, channel 0, id 0, lun 0
Attached scsi generic sg3 at scsi3, channel 0, id 0, lun 0,  type 0
blk_queue_max_hw_segments: set to minimum 1
=============<repeated 31 more times>==================


Anyone have any ideas for me?

-- 
http://www.fastmail.fm - Access your email from home and the web

Re: Marvell MV88SX6041 SATA Driver PIO4 Mode Not DMA

[Jeff Garzik]

Spencer Tuttle wrote:
> I have just compiled the new 2.6.14 kernel from the gentoo-sources tree.
> 
> I can access the drives just fine, but it seems really slow.  Here is
> the dmesg output when I load the kernel module

That's expected, since the driver in 2.6.14 only does PIO mode.

Try 2.6.14-gitN which supports EDMA.

	Jeff

Re: Marvell MV88SX6041 SATA Driver PIO4 Mode Not DMA

[Jens Axboe]

On Fri, Nov 11 2005, Jeff Garzik wrote:
> Spencer Tuttle wrote:
> >I have just compiled the new 2.6.14 kernel from the gentoo-sources tree.
> >
> >I can access the drives just fine, but it seems really slow.  Here is
> >the dmesg output when I load the kernel module
> 
> That's expected, since the driver in 2.6.14 only does PIO mode.
> 
> Try 2.6.14-gitN which supports EDMA.

Did you see these as well:

blk_queue_max_hw_segments: set to minimum 1

Could it be forgetting to set ->sg_tablesize as well?

-- 
Jens Axboe

Re: Marvell MV88SX6041 SATA Driver PIO4 Mode Not DMA

[Jeff Garzik]

Jens Axboe wrote:
> On Fri, Nov 11 2005, Jeff Garzik wrote:
> 
>>Spencer Tuttle wrote:
>>
>>>I have just compiled the new 2.6.14 kernel from the gentoo-sources tree.
>>>
>>>I can access the drives just fine, but it seems really slow.  Here is
>>>the dmesg output when I load the kernel module
>>
>>That's expected, since the driver in 2.6.14 only does PIO mode.
>>
>>Try 2.6.14-gitN which supports EDMA.
> 
> 
> Did you see these as well:
> 
> blk_queue_max_hw_segments: set to minimum 1
> 
> Could it be forgetting to set ->sg_tablesize as well?

[jgarzik@sata linux-2.6]$ grep MV_MAX_SG_CT drivers/scsi/sata_mv.c
         MV_MAX_SG_CT            = 176,
         MV_SG_TBL_SZ            = (16 * MV_MAX_SG_CT),
         .sg_tablesize           = MV_MAX_SG_CT,

Re: Marvell MV88SX6041 SATA Driver PIO4 Mode Not DMA

[Jens Axboe]

On Fri, Nov 11 2005, Jeff Garzik wrote:
> Jens Axboe wrote:
> >On Fri, Nov 11 2005, Jeff Garzik wrote:
> >
> >>Spencer Tuttle wrote:
> >>
> >>>I have just compiled the new 2.6.14 kernel from the gentoo-sources tree.
> >>>
> >>>I can access the drives just fine, but it seems really slow.  Here is
> >>>the dmesg output when I load the kernel module
> >>
> >>That's expected, since the driver in 2.6.14 only does PIO mode.
> >>
> >>Try 2.6.14-gitN which supports EDMA.
> >
> >
> >Did you see these as well:
> >
> >blk_queue_max_hw_segments: set to minimum 1
> >
> >Could it be forgetting to set ->sg_tablesize as well?
> 
> [jgarzik@sata linux-2.6]$ grep MV_MAX_SG_CT drivers/scsi/sata_mv.c
>         MV_MAX_SG_CT            = 176,
>         MV_SG_TBL_SZ            = (16 * MV_MAX_SG_CT),
>         .sg_tablesize           = MV_MAX_SG_CT,

It was just a guess, clearly someone is calling scsi_alloc_queue() with
an sdev->host->sg_tablesize of 0. While that wont make PIO a whole lot
faster, it's clearly a bug and would certainly hurt DMA performance.

-- 
Jens Axboe

Re: Marvell MV88SX6041 SATA Driver PIO4 Mode Not DMA

[Jens Axboe]

On Fri, Nov 11 2005, Jens Axboe wrote:
> On Fri, Nov 11 2005, Jeff Garzik wrote:
> > Jens Axboe wrote:
> > >On Fri, Nov 11 2005, Jeff Garzik wrote:
> > >
> > >>Spencer Tuttle wrote:
> > >>
> > >>>I have just compiled the new 2.6.14 kernel from the gentoo-sources tree.
> > >>>
> > >>>I can access the drives just fine, but it seems really slow.  Here is
> > >>>the dmesg output when I load the kernel module
> > >>
> > >>That's expected, since the driver in 2.6.14 only does PIO mode.
> > >>
> > >>Try 2.6.14-gitN which supports EDMA.
> > >
> > >
> > >Did you see these as well:
> > >
> > >blk_queue_max_hw_segments: set to minimum 1
> > >
> > >Could it be forgetting to set ->sg_tablesize as well?
> > 
> > [jgarzik@sata linux-2.6]$ grep MV_MAX_SG_CT drivers/scsi/sata_mv.c
> >         MV_MAX_SG_CT            = 176,
> >         MV_SG_TBL_SZ            = (16 * MV_MAX_SG_CT),
> >         .sg_tablesize           = MV_MAX_SG_CT,
> 
> It was just a guess, clearly someone is calling scsi_alloc_queue() with
> an sdev->host->sg_tablesize of 0. While that wont make PIO a whole lot
> faster, it's clearly a bug and would certainly hurt DMA performance.

Before commit 31961943e3110c5a1c36b1e0069c29f7c4380e51, sata_mv had
->sg_tablesize set to MV_UNDEF which was defined to 0. Just checking,
2.6.14 ships with that.

-- 
Jens Axboe

Re: Marvell MV88SX6041 SATA Driver PIO4 Mode Not DMA

[Spencer Tuttle]

>>>>>Spencer Tuttle wrote:
>>>>>
>>>>>
>>>>>>I have just compiled the new 2.6.14 kernel from the gentoo-sources tree.
>>>>>>
>>>>>>I can access the drives just fine, but it seems really slow.  Here is
>>>>>>the dmesg output when I load the kernel module
>>>>>
>>>>>That's expected, since the driver in 2.6.14 only does PIO mode.
>>>>>
>>>>>Try 2.6.14-gitN which supports EDMA.


Ok I will seek this out and compile it.

Its a bummer i didn't ask sooner, before i tried to create a software 
raid 5 array, (which took 5 days to build) :-)

>>>>
>>>>
>>>>Did you see these as well:
>>>>
>>>>blk_queue_max_hw_segments: set to minimum 1

Yes I did, they appeared 32 times for each drive.

Thank You,

Re: Marvell MV88SX6041 SATA Driver PIO4 Mode Not DMA

[Spencer Tuttle]

Spencer Tuttle wrote:
>>>>>> Spencer Tuttle wrote:
>>>>>>
>>>>>>
>>>>>>> I have just compiled the new 2.6.14 kernel from the 
>>>>>>> gentoo-sources tree.
>>>>>>>
>>>>>>> I can access the drives just fine, but it seems really slow.  
>>>>>>> Here is
>>>>>>> the dmesg output when I load the kernel module
>>>>>>
>>>>>>
>>>>>> That's expected, since the driver in 2.6.14 only does PIO mode.
>>>>>>
>>>>>> Try 2.6.14-gitN which supports EDMA.
> 
> 
> 
> Ok I will seek this out and compile it.

Ok, i have it compiled and running, when I modprobe sata_mv i get
libata version 1.12 loaded.
sata_mv 0000:03:04.0: version 0.25
PCI: setting IRQ 9 as level-triggered
ACPI: PCI Interrupt 0000:03:04.0[A] -> GSI 9 (level, low) -> IRQ 9
sata_mv 0000:03:04.0: 32 slots 4 ports SCSI mode IRQ via INTx
ata1: SATA max UDMA/133 cmd 0x0 ctl 0xF89A2120 bmdma 0x0 irq 9
ata2: SATA max UDMA/133 cmd 0x0 ctl 0xF89A4120 bmdma 0x0 irq 9
ata3: SATA max UDMA/133 cmd 0x0 ctl 0xF89A6120 bmdma 0x0 irq 9
ata4: SATA max UDMA/133 cmd 0x0 ctl 0xF89A8120 bmdma 0x0 irq 9
ATA: abnormal status 0x80 on port 0xF89A211C
ATA: abnormal status 0x80 on port 0xF89A211C
ata1: dev 0 cfg 49:2f00 82:7869 83:7d09 84:4003 85:7869 86:3c01 87:4003 
88:203f
ata1: dev 0 ATA-7, max UDMA/100, 156301488 sectors: LBA48
ata1: dev 0 configured for UDMA/100
scsi0 : sata_mv
ATA: abnormal status 0x80 on port 0xF89A411C
ATA: abnormal status 0x80 on port 0xF89A411C
ata2: dev 0 cfg 49:2f00 82:7869 83:7d09 84:4003 85:7869 86:3c01 87:4003 
88:203f
ata2: dev 0 ATA-7, max UDMA/100, 156301488 sectors: LBA48
ata2: dev 0 configured for UDMA/100
scsi1 : sata_mv
ATA: abnormal status 0x80 on port 0xF89A611C
ATA: abnormal status 0x80 on port 0xF89A611C
ata3: dev 0 cfg 49:2f00 82:7869 83:7d09 84:4003 85:7869 86:3c01 87:4003 
88:203f
ata3: dev 0 ATA-7, max UDMA/100, 156301488 sectors: LBA48
ata3: dev 0 configured for UDMA/100
scsi2 : sata_mv
ATA: abnormal status 0x80 on port 0xF89A811C
ATA: abnormal status 0x80 on port 0xF89A811C
ata4: dev 0 cfg 49:2f00 82:7869 83:7d09 84:4003 85:7869 86:3c01 87:4003 
88:203f
ata4: dev 0 ATA-7, max UDMA/100, 156301488 sectors: LBA48
ata4: dev 0 configured for UDMA/100
scsi3 : sata_mv
   Vendor: ATA       Model: Maxtor 6Y080M0    Rev: YAR5
   Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sda: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sda: drive cache: write back
SCSI device sda: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sda: drive cache: write back
  sda: sda1
sd 0:0:0:0: Attached scsi disk sda
   Vendor: ATA       Model: Maxtor 6Y080M0    Rev: YAR5
   Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sdb: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sdb: drive cache: write back
SCSI device sdb: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sdb: drive cache: write back
  sdb: sdb1
sd 1:0:0:0: Attached scsi disk sdb
   Vendor: ATA       Model: Maxtor 6Y080M0    Rev: YAR5
   Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sdc: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sdc: drive cache: write back
SCSI device sdc: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sdc: drive cache: write back
  sdc: sdc1
sd 2:0:0:0: Attached scsi disk sdc
   Vendor: ATA       Model: Maxtor 6Y080M0    Rev: YAR5
   Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sdd: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sdd: drive cache: write back
SCSI device sdd: 156301488 512-byte hdwr sectors (80026 MB)
SCSI device sdd: drive cache: write back
  sdd: sdd1
sd 3:0:0:0: Attached scsi disk sdd


Do I need to be concerned with the abnormal status? In the bios of my 
controller I have them setup as JBOD.

Does this driver support the RAID1 and RAID0 levels that are available 
on the controller?

Spencer

Re: Marvell MV88SX6041 SATA Driver PIO4 Mode Not DMA

[Jeff Garzik]

Jens Axboe wrote:
> On Fri, Nov 11 2005, Jens Axboe wrote:
> 
>>On Fri, Nov 11 2005, Jeff Garzik wrote:
>>
>>>Jens Axboe wrote:
>>>
>>>>On Fri, Nov 11 2005, Jeff Garzik wrote:
>>>>
>>>>
>>>>>Spencer Tuttle wrote:
>>>>>
>>>>>
>>>>>>I have just compiled the new 2.6.14 kernel from the gentoo-sources tree.
>>>>>>
>>>>>>I can access the drives just fine, but it seems really slow.  Here is
>>>>>>the dmesg output when I load the kernel module
>>>>>
>>>>>That's expected, since the driver in 2.6.14 only does PIO mode.
>>>>>
>>>>>Try 2.6.14-gitN which supports EDMA.
>>>>
>>>>
>>>>Did you see these as well:
>>>>
>>>>blk_queue_max_hw_segments: set to minimum 1
>>>>
>>>>Could it be forgetting to set ->sg_tablesize as well?
>>>
>>>[jgarzik@sata linux-2.6]$ grep MV_MAX_SG_CT drivers/scsi/sata_mv.c
>>>        MV_MAX_SG_CT            = 176,
>>>        MV_SG_TBL_SZ            = (16 * MV_MAX_SG_CT),
>>>        .sg_tablesize           = MV_MAX_SG_CT,
>>
>>It was just a guess, clearly someone is calling scsi_alloc_queue() with
>>an sdev->host->sg_tablesize of 0. While that wont make PIO a whole lot
>>faster, it's clearly a bug and would certainly hurt DMA performance.
> 
> 
> Before commit 31961943e3110c5a1c36b1e0069c29f7c4380e51, sata_mv had
> ->sg_tablesize set to MV_UNDEF which was defined to 0. Just checking,
> 2.6.14 ships with that.

OK, that version has no DMA support, so that makes sense.

I guess the gist of the thread is, sg_tablesize should never be less 
than one, even if "it would hurt DMA performance"?

	Jeff

Re: Marvell MV88SX6041 SATA Driver PIO4 Mode Not DMA

[Jens Axboe]

On Mon, Nov 14 2005, Jeff Garzik wrote:
> Jens Axboe wrote:
> >On Fri, Nov 11 2005, Jens Axboe wrote:
> >
> >>On Fri, Nov 11 2005, Jeff Garzik wrote:
> >>
> >>>Jens Axboe wrote:
> >>>
> >>>>On Fri, Nov 11 2005, Jeff Garzik wrote:
> >>>>
> >>>>
> >>>>>Spencer Tuttle wrote:
> >>>>>
> >>>>>
> >>>>>>I have just compiled the new 2.6.14 kernel from the gentoo-sources 
> >>>>>>tree.
> >>>>>>
> >>>>>>I can access the drives just fine, but it seems really slow.  Here is
> >>>>>>the dmesg output when I load the kernel module
> >>>>>
> >>>>>That's expected, since the driver in 2.6.14 only does PIO mode.
> >>>>>
> >>>>>Try 2.6.14-gitN which supports EDMA.
> >>>>
> >>>>
> >>>>Did you see these as well:
> >>>>
> >>>>blk_queue_max_hw_segments: set to minimum 1
> >>>>
> >>>>Could it be forgetting to set ->sg_tablesize as well?
> >>>
> >>>[jgarzik@sata linux-2.6]$ grep MV_MAX_SG_CT drivers/scsi/sata_mv.c
> >>>       MV_MAX_SG_CT            = 176,
> >>>       MV_SG_TBL_SZ            = (16 * MV_MAX_SG_CT),
> >>>       .sg_tablesize           = MV_MAX_SG_CT,
> >>
> >>It was just a guess, clearly someone is calling scsi_alloc_queue() with
> >>an sdev->host->sg_tablesize of 0. While that wont make PIO a whole lot
> >>faster, it's clearly a bug and would certainly hurt DMA performance.
> >
> >
> >Before commit 31961943e3110c5a1c36b1e0069c29f7c4380e51, sata_mv had
> >->sg_tablesize set to MV_UNDEF which was defined to 0. Just checking,
> >2.6.14 ships with that.
> 
> OK, that version has no DMA support, so that makes sense.

sg_tablesize isn't necessarily tied to dma support, but the performance
difference should be marginal so...

> I guess the gist of the thread is, sg_tablesize should never be less 
> than one, even if "it would hurt DMA performance"?

Precisely, a request with zero segments doesn't make sense for any
request carrying data.

-- 
Jens Axboe