Odd IDE performance drop 2.4 vs 2.6?

Odd IDE performance drop 2.4 vs 2.6?

[Grant Coady]

Hi there,

A new 'old' box, with near 3:1 hdparm -Tt /dev/hda performance drop 
comparing 2.4.31 with 2.6.11.12. pII/266 on 440LX chipset. HDD set 
to udma2 (max for h/w) with manuf. utility.  Single master on ribbon.
CDROM on other ribbon.  Two runs each via ssh login soon after boot:

Linux 2.4.31-si.
root@silly:~# hdparm -tT /dev/hda

/dev/hda:
 Timing cached reads:   344 MB in  1.99 seconds = 172.86 MB/sec
 Timing buffered disk reads:   68 MB in  3.02 seconds =  22.52 MB/sec
root@silly:~# hdparm -tT /dev/hda

/dev/hda:
 Timing cached reads:   356 MB in  2.00 seconds = 178.00 MB/sec
 Timing buffered disk reads:   68 MB in  3.04 seconds =  22.37 MB/sec
root@silly:~#

Linux 2.6.11.12a.
root@silly:~# hdparm -tT /dev/hda

/dev/hda:
 Timing cached reads:   340 MB in  2.01 seconds = 168.76 MB/sec
 Timing buffered disk reads:   26 MB in  3.02 seconds =   8.60 MB/sec
root@silly:~# hdparm -tT /dev/hda

/dev/hda:
 Timing cached reads:   340 MB in  2.01 seconds = 169.26 MB/sec
 Timing buffered disk reads:   26 MB in  3.02 seconds =   8.61 MB/sec
root@silly:~#

Hardware info, configs, etc at http://scatter.mine.nu/test/boxen/silly/
--Grant.

Re: Odd IDE performance drop 2.4 vs 2.6?

[Ondrej Zary]

Grant Coady wrote:
> Hi there,
> 
> A new 'old' box, with near 3:1 hdparm -Tt /dev/hda performance drop 
> comparing 2.4.31 with 2.6.11.12. pII/266 on 440LX chipset. HDD set 
> to udma2 (max for h/w) with manuf. utility.  Single master on ribbon.
> CDROM on other ribbon.  Two runs each via ssh login soon after boot:

I see this problem too with i430TX chipset (the south bridge and thus 
IDE controller is the same as in i440LX/EX and BX/ZX).

2.6.12-rc5:
/dev/hda:
  Timing cached reads:   180 MB in  2.02 seconds =  89.11 MB/sec
  Timing buffered disk reads:   40 MB in  3.09 seconds =  12.94 MB/sec

2.4.31:
/dev/hda:
  Timing cached reads:   180 MB in  2.03 seconds =  88.67 MB/sec
  Timing buffered disk reads:   62 MB in  3.01 seconds =  20.60 MB/sec

I also noticed that during the buffered read test on 2.6 kernel, the IDE 
activity LED is blinking (so the drive is not 100% utilised) while it's 
permanently on with 2.4.

> Linux 2.4.31-si.
> root@silly:~# hdparm -tT /dev/hda
> 
> /dev/hda:
>  Timing cached reads:   344 MB in  1.99 seconds = 172.86 MB/sec
>  Timing buffered disk reads:   68 MB in  3.02 seconds =  22.52 MB/sec
> root@silly:~# hdparm -tT /dev/hda
> 
> /dev/hda:
>  Timing cached reads:   356 MB in  2.00 seconds = 178.00 MB/sec
>  Timing buffered disk reads:   68 MB in  3.04 seconds =  22.37 MB/sec
> root@silly:~#
> 
> Linux 2.6.11.12a.
> root@silly:~# hdparm -tT /dev/hda
> 
> /dev/hda:
>  Timing cached reads:   340 MB in  2.01 seconds = 168.76 MB/sec
>  Timing buffered disk reads:   26 MB in  3.02 seconds =   8.60 MB/sec
> root@silly:~# hdparm -tT /dev/hda
> 
> /dev/hda:
>  Timing cached reads:   340 MB in  2.01 seconds = 169.26 MB/sec
>  Timing buffered disk reads:   26 MB in  3.02 seconds =   8.61 MB/sec
> root@silly:~#
> 
> Hardware info, configs, etc at http://scatter.mine.nu/test/boxen/silly/
> --Grant.
> 

-- 
Ondrej Zary

Re: Odd IDE performance drop 2.4 vs 2.6?

[Alan Cox]

On Llu, 2005-06-13 at 11:43, Ondrej Zary wrote:
> I see this problem too with i430TX chipset (the south bridge and thus 
> IDE controller is the same as in i440LX/EX and BX/ZX).

Make sure you have pre-empt disabled and the antcipatory I/O scheduler
disabled. 

Alan

Re: Odd IDE performance drop 2.4 vs 2.6?

[Nick Piggin]

Alan Cox wrote:
> On Llu, 2005-06-13 at 11:43, Ondrej Zary wrote:
> 
>>I see this problem too with i430TX chipset (the south bridge and thus 
>>IDE controller is the same as in i440LX/EX and BX/ZX).
> 
> 
> Make sure you have pre-empt disabled and the antcipatory I/O scheduler
> disabled. 
> 

I don't think that those could explain it.

Increasing readahead with the `blockdev` command has been known
to fix similar reports.

Send instant messages to your online friends http://au.messenger.yahoo.com 

Re: Odd IDE performance drop 2.4 vs 2.6?

[Alan Cox]

On Llu, 2005-06-13 at 15:06, Nick Piggin wrote:
> > Make sure you have pre-empt disabled and the antcipatory I/O scheduler
> > disabled. 
> > 
> I don't think that those could explain it.

Try it and see. The anticipatory I/O scheduler does horrible things to
my IDE streaming performance numbers and to swap performance. It tries
to merge I/O by delaying it which is deeply ungood when it comes to IDE
streaming even if its good for general I/O.

Alan

Re: Odd IDE performance drop 2.4 vs 2.6?

[Ondrej Zary]

Alan Cox wrote:
> On Llu, 2005-06-13 at 15:06, Nick Piggin wrote:
> 
>>>Make sure you have pre-empt disabled and the antcipatory I/O scheduler
>>>disabled. 
>>>
>>
>>I don't think that those could explain it.
> 
> 
> Try it and see. The anticipatory I/O scheduler does horrible things to
> my IDE streaming performance numbers and to swap performance. It tries
> to merge I/O by delaying it which is deeply ungood when it comes to IDE
> streaming even if its good for general I/O.

Changing the scheduler did not help (the results are about the same with 
any of the 4 schedulers). Read ahead is already set to 256 (increasing 
to 1024 did not help either). Kernel compilation takes too much time 
here so I didn't test with preempt disabled.
The drive is WD300BB (7200RPM) in UDMA2 mode.

root@pentium:~# cat /sys/block/hda/queue/scheduler
noop anticipatory [deadline] cfq
root@pentium:~# hdparm -tT /dev/hda

/dev/hda:
  Timing cached reads:   176 MB in  2.00 seconds =  88.00 MB/sec
  Timing buffered disk reads:   34 MB in  3.02 seconds =  11.26 MB/sec
root@pentium:~# hdparm /dev/hda

/dev/hda:
  multcount    = 16 (on)
  IO_support   =  1 (32-bit)
  unmaskirq    =  1 (on)
  using_dma    =  1 (on)
  keepsettings =  1 (on)
  readonly     =  0 (off)
  readahead    = 256 (on)
  geometry     = 58168/16/63, sectors = 58633344, start = 0


-- 
Ondrej Zary

Re: Odd IDE performance drop 2.4 vs 2.6?

[Jens Axboe]

On Mon, Jun 13 2005, Alan Cox wrote:
> On Llu, 2005-06-13 at 15:06, Nick Piggin wrote:
> > > Make sure you have pre-empt disabled and the antcipatory I/O scheduler
> > > disabled. 
> > > 
> > I don't think that those could explain it.
> 
> Try it and see. The anticipatory I/O scheduler does horrible things to
> my IDE streaming performance numbers and to swap performance. It tries
> to merge I/O by delaying it which is deeply ungood when it comes to IDE
> streaming even if its good for general I/O.

Well, AS should only intentionally delay when it has competing threads
fighting for the disk. For a single threaded io case like hdparm, it
should never idle the drive. It never delays to increase merge rate, or
anything like that - only to increase spatial locality on the drive for
two or more processes accessing it simultanously.

-- 
Jens Axboe

Re: Odd IDE performance drop 2.4 vs 2.6?

[Nick Piggin]

Alan Cox wrote:

>On Llu, 2005-06-13 at 15:06, Nick Piggin wrote:
>
>>>Make sure you have pre-empt disabled and the antcipatory I/O scheduler
>>>disabled. 
>>>
>>>
>>I don't think that those could explain it.
>>
>
>Try it and see. The anticipatory I/O scheduler does horrible things to
>my IDE streaming performance numbers and to swap performance. It tries
>to merge I/O by delaying it which is deeply ungood when it comes to IDE
>streaming even if its good for general I/O.
>
>
Sure it has regression cases here and there, as you would expect.
But I'm fairly sure this won't be one of them. If there is just a
single process submitting the IO the anticipatory scheduler should
completely turn of any delays, and degenerate basically to the
deadline scheduler.


Send instant messages to your online friends http://au.messenger.yahoo.com 

Re: Odd IDE performance drop 2.4 vs 2.6?

[Ondrej Zary]

Alan Cox wrote:
> On Llu, 2005-06-13 at 15:06, Nick Piggin wrote:
> 
>>>Make sure you have pre-empt disabled and the antcipatory I/O scheduler
>>>disabled. 
>>>
>>
>>I don't think that those could explain it.
> 
> 
> Try it and see. The anticipatory I/O scheduler does horrible things to
> my IDE streaming performance numbers and to swap performance. It tries
> to merge I/O by delaying it which is deeply ungood when it comes to IDE
> streaming even if its good for general I/O.

Now I've tested it with preempt disabled and nothing has changed. When 
fiddling around with hdparm, I got about 16MB/s max. with 2.6.12-rc5. 
With 2.4.31, I got about 21MB/s when just the DMA was enabled 
(read-ahead and multcount set to 0 - changing them does not make almost 
any difference).

-- 
Ondrej Zary

Re: Odd IDE performance drop 2.4 vs 2.6?

[Alan Cox]

On Mer, 2005-06-15 at 20:15, Ondrej Zary wrote:
> Now I've tested it with preempt disabled and nothing has changed. When 
> fiddling around with hdparm, I got about 16MB/s max. with 2.6.12-rc5. 
> With 2.4.31, I got about 21MB/s when just the DMA was enabled 
> (read-ahead and multcount set to 0 - changing them does not make almost 
> any difference).

multcount is only used for PIO so that would be expected. Similarly the
block readahead should matter but not anything drive level.

If you compare the hdparm data are both 2.4 and 2.6 selecting the same
IDE modes ?

Re: Odd IDE performance drop 2.4 vs 2.6?

[Ondrej Zary]

Alan Cox wrote:
> On Mer, 2005-06-15 at 20:15, Ondrej Zary wrote:
> 
>>Now I've tested it with preempt disabled and nothing has changed. When 
>>fiddling around with hdparm, I got about 16MB/s max. with 2.6.12-rc5. 
>>With 2.4.31, I got about 21MB/s when just the DMA was enabled 
>>(read-ahead and multcount set to 0 - changing them does not make almost 
>>any difference).
> 
> 
> multcount is only used for PIO so that would be expected. Similarly the
> block readahead should matter but not anything drive level.
> 
> If you compare the hdparm data are both 2.4 and 2.6 selecting the same
> IDE modes ?

This is in my init scripts:
/usr/sbin/hdparm -u1c1k1 /dev/hda /dev/hdc /dev/hdd 1> /dev/null
It selects UDMA2 mode in both 2.4 and 2.6.

hdparm -i /dev/hda shows exactly the same output in both 2.4 and 2.6:

/dev/hda:

  Model=WDC WD300BB-00AUA1, FwRev=18.20D18, SerialNo=WD-WMA6W1847372
  Config={ HardSect NotMFM HdSw>15uSec SpinMotCtl Fixed DTR>5Mbs FmtGapReq }
  RawCHS=16383/16/63, TrkSize=57600, SectSize=600, ECCbytes=40
  BuffType=DualPortCache, BuffSize=2048kB, MaxMultSect=16, MultSect=16
  CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=58633344
  IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
  PIO modes:  pio0 pio1 pio2 pio3 pio4
  DMA modes:  mdma0 mdma1 mdma2
  UDMA modes: udma0 udma1 *udma2
  AdvancedPM=no WriteCache=enabled
  Drive conforms to: device does not report version:

  * signifies the current active mode

-- 
Ondrej Zary

Re: Odd IDE performance drop 2.4 vs 2.6?

[Grant Coady]

On Mon, 13 Jun 2005 14:39:37 +0100, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

>On Llu, 2005-06-13 at 11:43, Ondrej Zary wrote:
>> I see this problem too with i430TX chipset (the south bridge and thus 
>> IDE controller is the same as in i440LX/EX and BX/ZX).
>
>Make sure you have pre-empt disabled and the antcipatory I/O scheduler
>disabled. 
I don't set pre-empt, not sure about scheduler, recheck that in daytime,
SATA (Via chipset) on different box doesn't have the problem, neither 
another box with SATA and ICH5, this is Via chipset, two runs after boot:

Linux 2.4.31-sp

root@sempro:~# hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   1172 MB in  2.00 seconds = 586.00 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device
 Timing buffered disk reads:  172 MB in  3.02 seconds =  56.95 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device
root@sempro:~# hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   1056 MB in  2.00 seconds = 528.00 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device
 Timing buffered disk reads:  170 MB in  3.00 seconds =  56.67 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device
root@sempro:~#

Linux 2.6.11.12a

/dev/sda:
 Timing cached reads:   1308 MB in  2.00 seconds = 653.77 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device
 Timing buffered disk reads:  170 MB in  3.00 seconds =  56.60 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device
root@sempro:~# hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   1124 MB in  2.00 seconds = 560.96 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device
 Timing buffered disk reads:  170 MB in  3.00 seconds =  56.66 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device
root@sempro:~#

HDD specified as 58MB/s at the fast end, so these figures look reasonable.
http://scatter.mine.nu/test/boxen/sempro/ for hardware / config info

--Grant.