40% IDE performance regression going from FC3 to FC5 with same kernel

40% IDE performance regression going from FC3 to FC5 with same kernel

[Alessandro Suardi]

[-- Attachment #1: Type: text/plain, Size: 2866 bytes --]

I'll be filing a FC5 performance bug for this but would like an opinion
 from the IDE kernel people just in case this has already been seen...

I just upgraded my home K7-800, 512MB RAM box from FC3 to FC5
 and noticed a disk performance slowdown while copying files around.

System has two 160GB disks, a Samsung SP1604N 2MB cache and
 a Maxtor 6Y160P0 8MB cache; both disks appear to be almost 2x
 slower both on hdparm -t tests (17-19MB/s against 33/35 MB/s) and
 on dd tests, like this:

FC3
[root@donkey tmp]# time dd if=/dev/hda of=/dev/null skip=200 bs=1024k count=200
200+0 records in
200+0 records out

real    0m4.623s
user    0m0.004s
sys     0m1.308s

FC5
[root@donkey tmp]#  time dd if=/dev/hda of=/dev/null skip=200 bs=1024k count=200
200+0 records in
200+0 records out
209715200 bytes (210 MB) copied, 9.67808 seconds, 21.7 MB/s

real    0m9.683s
user    0m0.008s
sys     0m1.400s


The initial tests were my last FC3 self-compiled kernel (2.6.16-rc5-git8)
 vs FC5's 2.6.16-1.2080_FC5 kernel; so just to be sure, I copied over
 from my FC3 partition the 2.6.16-rc5-git8 kernel and its config file,
 and rebuilt it under FC5, with just a few differences for the new USB
 2.0 disk I added to a PCI controller I just put in, namely

[root@donkey linux-2.6.16-rc5-git8]# diff .config
/fc3/usr/src/linux-2.6.16-rc5-git8/.config
4c4
< # Fri Apr  7 03:58:23 2006
---
> # Mon Mar  6 22:49:32 2006
1110,1112c1110
< CONFIG_USB_EHCI_HCD=m
< CONFIG_USB_EHCI_SPLIT_ISO=y
< CONFIG_USB_EHCI_ROOT_HUB_TT=y
---
> # CONFIG_USB_EHCI_HCD is not set
1115c1113
< CONFIG_USB_UHCI_HCD=m
---
> CONFIG_USB_UHCI_HCD=y
1218d1215
< # CONFIG_USB_SISUSBVGA is not set

The result is unexpected - performance delta is still there. Concatenating
 output from hdparm -i /dev/hda and hdparm /dev/hda for the same kernel
 under FC3 and FC5, the only difference is

[root@donkey ~]# diff /tmp/hdparm.out.2616rc2git8-fc5
/tmp/hdparm.out.2616rc2git8
14c14
<  Drive conforms to: (null):  ATA/ATAPI-1 ATA/ATAPI-2 ATA/ATAPI-3
ATA/ATAPI-4 ATA/ATAPI-5 ATA/ATAPI-6 ATA/ATAPI-7
---
>  Drive conforms to: (null):
27c27
<  geometry     = 19457/255/63, sectors = 312581808, start = 0
---
>  geometry     = 19457/255/63, sectors = 160041885696, start = 0

I'll try now and rebuild a 2.6.16-rc5-git8 kernel under FC5 with the
 FC3 GCC and see whether that is responsible for the performance
 drop... of course if anyone has any idea about what's going on, I
 will be happy to try out stuff. Attaching hdparm output from the FC5
 2.6.16-rc5-git8 just to show that there is DMA etc. all configured fine.


Thanks in advance, ciao,

--alessandro

 "Dreamer ? Each one of us is a dreamer. We just push it down deep because
   we are repeatedly told that we are not allowed to dream in real life"
     (Reinhold Ziegler)

[-- Attachment #2: hdparm.out.2616rc5git8-fc5 --]
[-- Type: application/octet-stream, Size: 954 bytes --]


/dev/hda:

 Model=SAMSUNG SP1604N, FwRev=TM100-24, SerialNo=0773J1FWC32580
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs }
 RawCHS=16383/16/63, TrkSize=34902, SectSize=554, ECCbytes=4
 BuffType=DualPortCache, BuffSize=2048kB, MaxMultSect=16, MultSect=off
 CurCHS=17475/15/63, CurSects=16513875, LBA=yes, LBAsects=268435455
 IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4 
 DMA modes:  mdma0 mdma1 mdma2 
 UDMA modes: udma0 udma1 udma2 udma3 *udma4 udma5 
 AdvancedPM=no WriteCache=enabled
 Drive conforms to: (null):  ATA/ATAPI-1 ATA/ATAPI-2 ATA/ATAPI-3 ATA/ATAPI-4 ATA/ATAPI-5 ATA/ATAPI-6 ATA/ATAPI-7

 * signifies the current active mode


/dev/hda:
 multcount    =  0 (off)
 IO_support   =  1 (32-bit)
 unmaskirq    =  1 (on)
 using_dma    =  1 (on)
 keepsettings =  0 (off)
 readonly     =  0 (off)
 readahead    = 256 (on)
 geometry     = 19457/255/63, sectors = 312581808, start = 0

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Alessandro Suardi]

On 4/8/06, Alessandro Suardi <alessandro.suardi@gmail.com> wrote:
> I'll be filing a FC5 performance bug for this but would like an opinion
>  from the IDE kernel people just in case this has already been seen...
>
> I just upgraded my home K7-800, 512MB RAM box from FC3 to FC5
>  and noticed a disk performance slowdown while copying files around.
>
> System has two 160GB disks, a Samsung SP1604N 2MB cache and
>  a Maxtor 6Y160P0 8MB cache; both disks appear to be almost 2x
>  slower both on hdparm -t tests (17-19MB/s against 33/35 MB/s) and
>  on dd tests, like this:
>
> FC3
> [root@donkey tmp]# time dd if=/dev/hda of=/dev/null skip=200 bs=1024k count=200
> 200+0 records in
> 200+0 records out
>
> real    0m4.623s
> user    0m0.004s
> sys     0m1.308s
>
> FC5
> [root@donkey tmp]#  time dd if=/dev/hda of=/dev/null skip=200 bs=1024k count=200
> 200+0 records in
> 200+0 records out
> 209715200 bytes (210 MB) copied, 9.67808 seconds, 21.7 MB/s
>
> real    0m9.683s
> user    0m0.008s
> sys     0m1.400s
>
>
> The initial tests were my last FC3 self-compiled kernel (2.6.16-rc5-git8)
>  vs FC5's 2.6.16-1.2080_FC5 kernel; so just to be sure, I copied over
>  from my FC3 partition the 2.6.16-rc5-git8 kernel and its config file,
>  and rebuilt it under FC5, with just a few differences for the new USB
>  2.0 disk I added to a PCI controller I just put in, namely
>
> [root@donkey linux-2.6.16-rc5-git8]# diff .config
> /fc3/usr/src/linux-2.6.16-rc5-git8/.config
> 4c4
> < # Fri Apr  7 03:58:23 2006
> ---
> > # Mon Mar  6 22:49:32 2006
> 1110,1112c1110
> < CONFIG_USB_EHCI_HCD=m
> < CONFIG_USB_EHCI_SPLIT_ISO=y
> < CONFIG_USB_EHCI_ROOT_HUB_TT=y
> ---
> > # CONFIG_USB_EHCI_HCD is not set
> 1115c1113
> < CONFIG_USB_UHCI_HCD=m
> ---
> > CONFIG_USB_UHCI_HCD=y
> 1218d1215
> < # CONFIG_USB_SISUSBVGA is not set
>
> The result is unexpected - performance delta is still there. Concatenating
>  output from hdparm -i /dev/hda and hdparm /dev/hda for the same kernel
>  under FC3 and FC5, the only difference is
>
> [root@donkey ~]# diff /tmp/hdparm.out.2616rc2git8-fc5
> /tmp/hdparm.out.2616rc2git8
> 14c14
> <  Drive conforms to: (null):  ATA/ATAPI-1 ATA/ATAPI-2 ATA/ATAPI-3
> ATA/ATAPI-4 ATA/ATAPI-5 ATA/ATAPI-6 ATA/ATAPI-7
> ---
> >  Drive conforms to: (null):
> 27c27
> <  geometry     = 19457/255/63, sectors = 312581808, start = 0
> ---
> >  geometry     = 19457/255/63, sectors = 160041885696, start = 0
>
> I'll try now and rebuild a 2.6.16-rc5-git8 kernel under FC5 with the
>  FC3 GCC and see whether that is responsible for the performance
>  drop... of course if anyone has any idea about what's going on, I
>  will be happy to try out stuff. Attaching hdparm output from the FC5
>  2.6.16-rc5-git8 just to show that there is DMA etc. all configured fine.

Just for the record - no, even rebuilding same kernel with same GCC
 (3.4.4) under FC5, disk performance is much slower than FC3 -
 according to hdparm _and_ dd tests.

--alessandro

 "Dreamer ? Each one of us is a dreamer. We just push it down deep because
   we are repeatedly told that we are not allowed to dream in real life"
     (Reinhold Ziegler)

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Arjan van de Ven]

> Just for the record - no, even rebuilding same kernel with same GCC
>  (3.4.4) under FC5, disk performance is much slower than FC3 -
>  according to hdparm _and_ dd tests.

what happens if you kill hald and other inotify using animals?

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Alessandro Suardi]

On 4/8/06, Arjan van de Ven <arjan@infradead.org> wrote:
>
> > Just for the record - no, even rebuilding same kernel with same GCC
> >  (3.4.4) under FC5, disk performance is much slower than FC3 -
> >  according to hdparm _and_ dd tests.
>
> what happens if you kill hald and other inotify using animals?

Thanks Arjan for looking into this.

Stopping hald brings hdparm from 18.5MB/s to 20MB/s, which is
 indeed a noticeable improvement, though still far from the FC3
 performance. I'm unsure what else can be stopped however
 from this process list:

[root@donkey init.d]# ps ax
  PID TTY      STAT   TIME COMMAND
    1 ?        S      0:00 init [3]
    2 ?        SN     0:00 [ksoftirqd/0]
    3 ?        S      0:00 [watchdog/0]
    4 ?        S<     0:00 [events/0]
    5 ?        S<     0:00 [khelper]
    6 ?        S<     0:00 [kthread]
    8 ?        S<     0:00 [kblockd/0]
   11 ?        S<     0:00 [khubd]
   40 ?        S      0:00 [kapmd]
   72 ?        S      0:00 [pdflush]
   73 ?        S      0:00 [pdflush]
   74 ?        S      0:00 [kswapd0]
   75 ?        S<     0:00 [aio/0]
   76 ?        S      0:00 [cifsoplockd]
   77 ?        S      0:00 [cifsdnotifyd]
  153 ?        S<     0:00 [kseriod]
  186 ?        S      0:00 [kedac]
  199 ?        S      0:00 [kjournald]
  274 ?        S<s    0:00 /sbin/udevd -d
  346 ?        S<     0:00 [kpsmoused]
  781 ?        S<     0:00 [scsi_eh_0]
  782 ?        S<     0:00 [usb-storage]
  836 ?        S      0:00 [kjournald]
  838 ?        S      0:00 [kjournald]
  840 ?        S      0:00 [kjournald]
  842 ?        S      0:00 [kjournald]
  844 ?        S      0:00 [kjournald]
 1170 ?        Ss     0:00 syslogd -m 0
 1173 ?        Ss     0:00 klogd -x
 1197 ?        Ss     0:00 portmap
 1216 ?        Ss     0:00 rpc.statd
 1295 ?        Ss     0:00 /usr/sbin/apmd -p 10 -w 5 -W -P
/etc/sysconfig/apm-sc 1440 ?        Ss     0:00 cupsd
 1480 ?        Ss     0:00 /usr/sbin/sshd
 1504 ?        Ss     0:00 sendmail: accepting connections
 1514 ?        Ss     0:00 sendmail: Queue runner@01:00:00 for
/var/spool/client 1524 ?        Ss     0:00 gpm -m /dev/input/mice -t
exps2
 1533 ?        Ss     0:00 crond
 1582 ?        Ss     0:00 /usr/sbin/atd
 1662 tty1     Ss+    0:00 /sbin/mingetty tty1
 1663 tty2     Ss+    0:00 /sbin/mingetty tty2
 1666 tty3     Ss+    0:00 /sbin/mingetty tty3
 1669 tty4     Ss+    0:00 /sbin/mingetty tty4
 1672 tty5     Ss+    0:00 /sbin/mingetty tty5
 1703 tty6     Ss+    0:00 /sbin/mingetty tty6
 1718 ?        Ss     0:00 sshd: root@pts/0
 1742 pts/0    Ss     0:00 -bash
 1859 pts/0    R+     0:00 ps ax

strace shows read() system time practically identical around 1.3s,
 hence I ran a ltrace comparison, which shows this:

FC3:

[root@donkey ~]#  ltrace -c dd if=/dev/hdb of=/dev/null skip=400
bs=1024k count=200
200+0 records in
200+0 records out
% time     seconds  usecs/call     calls      function
------ ----------- ----------- --------- --------------------
 99.86   14.597554       72987       200 read
  0.10    0.015150          75       200 write
  0.01    0.000947         473         2 dcgettext
  0.01    0.000871         435         2 fprintf
  0.00    0.000634         634         1 setlocale

FC5:

[root@donkey ~]# ltrace -c dd if=/dev/hdb of=/dev/null skip=400
bs=1024k count=200
200+0 records in
200+0 records out
209715200 bytes (210 MB) copied, 9.49944 seconds, 22.1 MB/s
% time     seconds  usecs/call     calls      function
------ ----------- ----------- --------- --------------------
 99.81   27.460675      137303       200 read
  0.12    0.032411       32411         1 dcgettext
  0.05    0.014069          70       200 write
  0.00    0.000908         302         3 __fprintf_chk

FC5 rebooted, with hald stopped:

[root@donkey init.d]# ltrace -c dd if=/dev/hdb of=/dev/null skip=400
bs=1024k count=200
200+0 records in
200+0 records out
209715200 bytes (210 MB) copied, 8.83107 seconds, 23.7 MB/s
% time     seconds  usecs/call     calls      function
------ ----------- ----------- --------- --------------------
 99.93   26.791861      133959       200 read
  0.05    0.014154          70       200 write
  0.00    0.000925         925         1 dcgettext
  0.00    0.000856         285         3 __fprintf_chk


Note the usecs/call comparison for read()...

[Also note how the ltrace output is wrong in terms of absolute
 timing - approximately 3x the actual elapsed time]

Thanks,

--alessandro

 "Dreamer ? Each one of us is a dreamer. We just push it down deep because
   we are repeatedly told that we are not allowed to dream in real life"
     (Reinhold Ziegler)

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Arjan van de Ven]

On Sat, 2006-04-08 at 18:27 +0200, Alessandro Suardi wrote:
> On 4/8/06, Arjan van de Ven <arjan@infradead.org> wrote:
> >
> > > Just for the record - no, even rebuilding same kernel with same GCC
> > >  (3.4.4) under FC5, disk performance is much slower than FC3 -
> > >  according to hdparm _and_ dd tests.
> >
> > what happens if you kill hald and other inotify using animals?
> 
> Thanks Arjan for looking into this.
> 
> Stopping hald brings hdparm from 18.5MB/s to 20MB/s, which is
>  indeed a noticeable improvement, though still far from the FC3
>  performance. I'm unsure what else can be stopped however
>  from this process list:
> 
>   274 ?        S<s    0:00 /sbin/udevd -d


try killing that one next; it may or may not help but it's sure worth a
try (esp given the success of the first kill :)

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Alessandro Suardi]

On 4/8/06, Arjan van de Ven <arjan@infradead.org> wrote:
> On Sat, 2006-04-08 at 18:27 +0200, Alessandro Suardi wrote:
> > On 4/8/06, Arjan van de Ven <arjan@infradead.org> wrote:
> > >
> > > > Just for the record - no, even rebuilding same kernel with same GCC
> > > >  (3.4.4) under FC5, disk performance is much slower than FC3 -
> > > >  according to hdparm _and_ dd tests.
> > >
> > > what happens if you kill hald and other inotify using animals?
> >
> > Thanks Arjan for looking into this.
> >
> > Stopping hald brings hdparm from 18.5MB/s to 20MB/s, which is
> >  indeed a noticeable improvement, though still far from the FC3
> >  performance. I'm unsure what else can be stopped however
> >  from this process list:
> >
> >   274 ?        S<s    0:00 /sbin/udevd -d
>
>
> try killing that one next; it may or may not help but it's sure worth a
> try (esp given the success of the first kill :)

killing udevd doesn't bring any improvement - still at 20MB/s.

Do you want me to file a FC5 bugzilla entry with the current info
 or do you think there is something else that can be discussed
 on lkml ?

Thanks,

--alessandro

 "Dreamer ? Each one of us is a dreamer. We just push it down deep because
   we are repeatedly told that we are not allowed to dream in real life"
     (Reinhold Ziegler)

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Jan Engelhardt]

>> try killing that one next; it may or may not help but it's sure worth a
>> try (esp given the success of the first kill :)
>
>killing udevd doesn't bring any improvement - still at 20MB/s.
>
>Do you want me to file a FC5 bugzilla entry with the current info
> or do you think there is something else that can be discussed
> on lkml ?
>

Compile a non-initrd kernel and run it with the -b parameter (it's passed 
to /sbin/init). From that shell, run your speed test. What does it show?


Jan Engelhardt
-- 

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Alessandro Suardi]

On 4/11/06, Jan Engelhardt <jengelh@linux01.gwdg.de> wrote:
> >> try killing that one next; it may or may not help but it's sure worth a
> >> try (esp given the success of the first kill :)
> >
> >killing udevd doesn't bring any improvement - still at 20MB/s.
> >
> >Do you want me to file a FC5 bugzilla entry with the current info
> > or do you think there is something else that can be discussed
> > on lkml ?
> >
>
> Compile a non-initrd kernel and run it with the -b parameter (it's passed
> to /sbin/init). From that shell, run your speed test. What does it show?

All my kernels are non-initrd (as long as I can do this in Fedora,
 I will do that). How do I pass -b to init ?

Thanks,

--alessandro

 "Dreamer ? Each one of us is a dreamer. We just push it down deep because
   we are repeatedly told that we are not allowed to dream in real life"
     (Reinhold Ziegler)

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Jan Engelhardt]

>> Compile a non-initrd kernel and run it with the -b parameter (it's passed
>> to /sbin/init). From that shell, run your speed test. What does it show?
>
>All my kernels are non-initrd (as long as I can do this in Fedora,
> I will do that). How do I pass -b to init ?

Any arguments on the kernel boot command line ("append" in lilo.conf or 
equivalent in grub) are passed down to the master process, as specified by 
the init= boot option, if any (defaults to /sbin/init for non-initrd 
kernels, defaults to /init for initramfs, and defaults to something else 
for initrds).


Jan Engelhardt
-- 

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Andreas Mohr]

Hi,

On Sat, Apr 08, 2006 at 04:47:18PM +0200, Alessandro Suardi wrote:
> I'll be filing a FC5 performance bug for this but would like an opinion
>  from the IDE kernel people just in case this has already been seen...
> 
> I just upgraded my home K7-800, 512MB RAM box from FC3 to FC5
>  and noticed a disk performance slowdown while copying files around.

Just another suggestion: try eliminating/pinpointing I/O scheduler issues
(switch e.g. to "noop" at /sys/block/hda/queue/scheduler and compare again)

Andreas Mohr
~

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Alessandro Suardi]

On 4/11/06, Andreas Mohr <andi@rhlx01.fht-esslingen.de> wrote:
> Hi,
>
> On Sat, Apr 08, 2006 at 04:47:18PM +0200, Alessandro Suardi wrote:
> > I'll be filing a FC5 performance bug for this but would like an opinion
> >  from the IDE kernel people just in case this has already been seen...
> >
> > I just upgraded my home K7-800, 512MB RAM box from FC3 to FC5
> >  and noticed a disk performance slowdown while copying files around.
>
> Just another suggestion: try eliminating/pinpointing I/O scheduler issues
> (switch e.g. to "noop" at /sys/block/hda/queue/scheduler and compare again)

Thanks Andi. Tried every scheduler (my default is anticipatory) and
 there aren't meaningful differences - 18.3 to 18.6MB/s.

As a further data point, my box can burn a 8x DVD+R at up to 7.1x
 average speed under FC3, while it barely keeps up with 4x in FC5.

--alessandro

 "Dreamer ? Each one of us is a dreamer. We just push it down deep because
   we are repeatedly told that we are not allowed to dream in real life"
     (Reinhold Ziegler)

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Jan Engelhardt]

>> > I'll be filing a FC5 performance bug for this but would like an opinion
>> >  from the IDE kernel people just in case this has already been seen...
>> >
>> > I just upgraded my home K7-800, 512MB RAM box from FC3 to FC5
>> >  and noticed a disk performance slowdown while copying files around.
>>
>> Just another suggestion: try eliminating/pinpointing I/O scheduler issues
>> (switch e.g. to "noop" at /sys/block/hda/queue/scheduler and compare again)
>
>Thanks Andi. Tried every scheduler (my default is anticipatory) and
> there aren't meaningful differences - 18.3 to 18.6MB/s.
>
>As a further data point, my box can burn a 8x DVD+R at up to 7.1x
> average speed under FC3, while it barely keeps up with 4x in FC5.

Since you said it happens with the same kernel, I think it's caused by 
userspace (do the boot-with-"-b" thing and you'll know). Possible someone 
setting DMA to speeds as low as udma4 or udma2.


Jan Engelhardt
-- 

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Alessandro Suardi]

On 4/11/06, Jan Engelhardt <jengelh@linux01.gwdg.de> wrote:
> >> > I'll be filing a FC5 performance bug for this but would like an opinion
> >> >  from the IDE kernel people just in case this has already been seen...
> >> >
> >> > I just upgraded my home K7-800, 512MB RAM box from FC3 to FC5
> >> >  and noticed a disk performance slowdown while copying files around.
> >>
> >> Just another suggestion: try eliminating/pinpointing I/O scheduler issues
> >> (switch e.g. to "noop" at /sys/block/hda/queue/scheduler and compare again)
> >
> >Thanks Andi. Tried every scheduler (my default is anticipatory) and
> > there aren't meaningful differences - 18.3 to 18.6MB/s.
> >
> >As a further data point, my box can burn a 8x DVD+R at up to 7.1x
> > average speed under FC3, while it barely keeps up with 4x in FC5.
>
> Since you said it happens with the same kernel, I think it's caused by
> userspace (do the boot-with-"-b" thing and you'll know). Possible someone
> setting DMA to speeds as low as udma4 or udma2.

Booting into /sbin/init -b indeed shows I get 33.3MB/s from hdparm,
 as in FC3. DMA is (again according to hdparm) udma4 both booted
 normally and in emergency mode. Note that at this point I'm testing
 2.6.17-rc1-git4 in FC5, not to be trailing current kernels too much
 (and yes, I already tested that normal 2.6.17-rc1-git4 boot in FC5
 gets 18.5MB/s - without IDE_GENERIC, so that is also clearly a
 non relevant factor).

As per Bill Davidsen's question in another email, blockdev --getra
 shows 256, which is the same value that hdparm shows.

--alessandro

 "Dreamer ? Each one of us is a dreamer. We just push it down deep because
   we are repeatedly told that we are not allowed to dream in real life"
     (Reinhold Ziegler)

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Alessandro Suardi]

On 4/11/06, Alessandro Suardi <alessandro.suardi@gmail.com> wrote:
> On 4/11/06, Jan Engelhardt <jengelh@linux01.gwdg.de> wrote:
> > >> > I'll be filing a FC5 performance bug for this but would like an opinion
> > >> >  from the IDE kernel people just in case this has already been seen...
> > >> >
> > >> > I just upgraded my home K7-800, 512MB RAM box from FC3 to FC5
> > >> >  and noticed a disk performance slowdown while copying files around.
> > >>
> > >> Just another suggestion: try eliminating/pinpointing I/O scheduler issues
> > >> (switch e.g. to "noop" at /sys/block/hda/queue/scheduler and compare again)
> > >
> > >Thanks Andi. Tried every scheduler (my default is anticipatory) and
> > > there aren't meaningful differences - 18.3 to 18.6MB/s.
> > >
> > >As a further data point, my box can burn a 8x DVD+R at up to 7.1x
> > > average speed under FC3, while it barely keeps up with 4x in FC5.
> >
> > Since you said it happens with the same kernel, I think it's caused by
> > userspace (do the boot-with-"-b" thing and you'll know). Possible someone
> > setting DMA to speeds as low as udma4 or udma2.
>
> Booting into /sbin/init -b indeed shows I get 33.3MB/s from hdparm,
>  as in FC3. DMA is (again according to hdparm) udma4 both booted
>  normally and in emergency mode. Note that at this point I'm testing
>  2.6.17-rc1-git4 in FC5, not to be trailing current kernels too much
>  (and yes, I already tested that normal 2.6.17-rc1-git4 boot in FC5
>  gets 18.5MB/s - without IDE_GENERIC, so that is also clearly a
>  non relevant factor).
>
> As per Bill Davidsen's question in another email, blockdev --getra
>  shows 256, which is the same value that hdparm shows.
>

Filed bugzilla #188630.

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=188630

Thanks,

--alessandro

 "Dreamer ? Each one of us is a dreamer. We just push it down deep because
   we are repeatedly told that we are not allowed to dream in real life"
     (Reinhold Ziegler)

Re: 40% IDE performance regression going from FC3 to FC5 with same kernel

[Bill Davidsen]

Alessandro Suardi wrote:
> I'll be filing a FC5 performance bug for this but would like an opinion
>  from the IDE kernel people just in case this has already been seen...
> 
> I just upgraded my home K7-800, 512MB RAM box from FC3 to FC5
>  and noticed a disk performance slowdown while copying files around.
> 
> System has two 160GB disks, a Samsung SP1604N 2MB cache and
>  a Maxtor 6Y160P0 8MB cache; both disks appear to be almost 2x
>  slower both on hdparm -t tests (17-19MB/s against 33/35 MB/s) and
>  on dd tests, like this:
> 
> FC3
> [root@donkey tmp]# time dd if=/dev/hda of=/dev/null skip=200 bs=1024k count=200
> 200+0 records in
> 200+0 records out
> 
> real    0m4.623s
> user    0m0.004s
> sys     0m1.308s
> 
> FC5
> [root@donkey tmp]#  time dd if=/dev/hda of=/dev/null skip=200 bs=1024k count=200
> 200+0 records in
> 200+0 records out
> 209715200 bytes (210 MB) copied, 9.67808 seconds, 21.7 MB/s
> 
> real    0m9.683s
> user    0m0.008s
> sys     0m1.400s
> 
> 
> The initial tests were my last FC3 self-compiled kernel (2.6.16-rc5-git8)
>  vs FC5's 2.6.16-1.2080_FC5 kernel; so just to be sure, I copied over
>  from my FC3 partition the 2.6.16-rc5-git8 kernel and its config file,
>  and rebuilt it under FC5, with just a few differences for the new USB
>  2.0 disk I added to a PCI controller I just put in, namely
> 
> [root@donkey linux-2.6.16-rc5-git8]# diff .config
> /fc3/usr/src/linux-2.6.16-rc5-git8/.config
> 4c4
> < # Fri Apr  7 03:58:23 2006
> ---
>> # Mon Mar  6 22:49:32 2006
> 1110,1112c1110
> < CONFIG_USB_EHCI_HCD=m
> < CONFIG_USB_EHCI_SPLIT_ISO=y
> < CONFIG_USB_EHCI_ROOT_HUB_TT=y
> ---
>> # CONFIG_USB_EHCI_HCD is not set
> 1115c1113
> < CONFIG_USB_UHCI_HCD=m
> ---
>> CONFIG_USB_UHCI_HCD=y
> 1218d1215
> < # CONFIG_USB_SISUSBVGA is not set
> 
> The result is unexpected - performance delta is still there. Concatenating
>  output from hdparm -i /dev/hda and hdparm /dev/hda for the same kernel
>  under FC3 and FC5, the only difference is
> 
> [root@donkey ~]# diff /tmp/hdparm.out.2616rc2git8-fc5
> /tmp/hdparm.out.2616rc2git8
> 14c14
> <  Drive conforms to: (null):  ATA/ATAPI-1 ATA/ATAPI-2 ATA/ATAPI-3
> ATA/ATAPI-4 ATA/ATAPI-5 ATA/ATAPI-6 ATA/ATAPI-7
> ---
>>  Drive conforms to: (null):
> 27c27
> <  geometry     = 19457/255/63, sectors = 312581808, start = 0
> ---
>>  geometry     = 19457/255/63, sectors = 160041885696, start = 0
> 
> I'll try now and rebuild a 2.6.16-rc5-git8 kernel under FC5 with the
>  FC3 GCC and see whether that is responsible for the performance
>  drop... of course if anyone has any idea about what's going on, I
>  will be happy to try out stuff. Attaching hdparm output from the FC5
>  2.6.16-rc5-git8 just to show that there is DMA etc. all configured fine.

Could you look at params with hdparm, and display the readahead with
"blockdev --getra" and see if this is read or write limited?