Terrible performance (<100kb/sec) in a cp -Raf

Terrible performance (<100kb/sec) in a cp -Raf

[Scott A Crosby]

Hello, as part of a planned migration of my computer from ext3 to
reiserfs I experienced terrible performance. I was running a stock
linux 2.6.6 kernel on AMD XP-2500+ CPU with 1.5gb of RAM.

I was copying about 26 gigabytes from an IDE hard drive to an external
USB HD when the bulk copy (cp -Raf) appeared to freeze. The HD did
nothing other than grind itself back and forth. Stracing the copy
operation showed the write() syscall blocking.

'vmstat 1' during the freeze reported:

 0  2  97716   9504  81208 1353328    0    0     0  1824 1989  2147  0  0  0 100
 0  2  97716   9504  81208 1353328    0    0     0  2132 2018  2235  0  3  0 97
 0  2  97716   9504  81208 1353328    0    0     0  1844 2019  2222  0  0  0 100
 0  2  97716   9520  81208 1353328    0    0     0  2108 2012  2239  0  0  0 100
 0  2  97716   9520  81208 1353328    0    0     0  1916 2037  2263  0  1  0 99

The result of running 'df' on the partitian every 10 seconds was:

/dev/sda1             26683112   4470492  22212620  17% /mnt/tmp
/dev/sda1             26683112   4484316  22198796  17% /mnt/tmp
/dev/sda1             26683112   4485976  22197136  17% /mnt/tmp
/dev/sda1             26683112   4486616  22196496  17% /mnt/tmp
/dev/sda1             26683112   4488252  22194860  17% /mnt/tmp
/dev/sda1             26683112   4488884  22194228  17% /mnt/tmp
/dev/sda1             26683112   4489524  22193588  17% /mnt/tmp
/dev/sda1             26683112   4489524  22193588  17% /mnt/tmp
/dev/sda1             26683112   4793412  21889700  18% /mnt/tmp
/dev/sda1             26683112   4873236  21809876  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873620  21809492  19% /mnt/tmp
/dev/sda1             26683112   4873748  21809364  19% /mnt/tmp

Notice I was getting under 100kb of writes/second out of the partitian
for about 30 seconds near the beginning of this log. before it seemed
to freeze completely. Normal performance for the drive under ext3 is
about 18mb/sec reading and 12mb/sec reading.

I considered that the problem might be hash collisions, so I
reformatted the partitian restarted the copy from scratch. I got about
the same freezing behavior at about the same spot in the copy
operation. A reformat for ext3 and a third copy had none of these
problems. I also checked the dmesg log for any IO errors and saw none.

After control-C'ing the copy and waiting for 5 minutes for the
grinding to stop I examined the destination to look for unique
properties at the point it appeared to freeze. The destination
filesystem tree showed an almost complete copy of the TIGER dataset
(3gb of data in 3000 files and 30 directories)

System version is:
    Linux version 2.6.6 (root@dragonlight) (gcc version 3.3.3 (Debian 20040401)) #12 Wed Jun 2 19:59:39 CDT 2004

I'm available for testing potential patches if the developers consider
this worth tracking down. If not, I'll wait for Reiser4.

Scott

Re: Terrible performance (<100kb/sec) in a cp -Raf

[Pierre Etchemaite]

Le 14 Jun 2004 08:25:24 -0500, Scott A Crosby <scrosby@cs.rice.edu> a écrit
:

> Hello, as part of a planned migration of my computer from ext3 to
> reiserfs I experienced terrible performance. I was running a stock
> linux 2.6.6 kernel on AMD XP-2500+ CPU with 1.5gb of RAM.
> 
> I was copying about 26 gigabytes from an IDE hard drive to an external
> USB HD when the bulk copy (cp -Raf) appeared to freeze. The HD did
> nothing other than grind itself back and forth. Stracing the copy
> operation showed the write() syscall blocking.

With 2.6.5, I had to apply this patch to avoid firewire misbehaving
(Oopsing, in my case) :

http://marc.theaimsgroup.com/?l=linux-kernel&m=108114119220175&w=2

Since I cannot find this code in 2.6.6 either, I guess the patch is still
relevant.

Firewire support doesn't look too solid in 2.6.x yet.

Re: Terrible performance (<100kb/sec) in a cp -Raf

[Scott A Crosby]

On 14 Jun 2004 08:25:24 -0500, Scott A Crosby <scrosby@cs.rice.edu> writes:

> Hello, as part of a planned migration of my computer from ext3 to
> reiserfs I experienced terrible performance. I was running a stock
> linux 2.6.6 kernel on AMD XP-2500+ CPU with 1.5gb of RAM.

Responding to Pierre's comments:

I appreciate the help, but I am using an external USB drive, not an
external firewire drive. The kernel patch only modifies the
ieee1394_core.c file, which is unused in my system.

Also, my error seems related to Fabian Sturm's report a couple of days
ago about 'no space left on device'. When I waited for the bulk copy
to stop, 'cp' reported no space available even though the partitian is
only 17% used. 

Scott

Re: Terrible performance (<100kb/sec) in a cp -Raf

[Pierre Etchemaite]

Le 17 Jun 2004 09:49:53 -0500, Scott A Crosby <scrosby@cs.rice.edu> a écrit
:

> I appreciate the help, but I am using an external USB drive, not an
> external firewire drive. The kernel patch only modifies the
> ieee1394_core.c file, which is unused in my system.

Uh, I wonder why I read "firewire" when it was "USB" ;)

> Also, my error seems related to Fabian Sturm's report a couple of days
> ago about 'no space left on device'. When I waited for the bulk copy
> to stop, 'cp' reported no space available even though the partitian is
> only 17% used. 

So it may not be USB related after all. Tried kernel 2.6.7 ?

Re: Terrible performance (<100kb/sec) in a cp -Raf

[Hendrik Visage]

On Fri, Jun 18, 2004 at 11:07:59AM +0200, Pierre Etchemaite wrote:
> Le 17 Jun 2004 09:49:53 -0500, Scott A Crosby <scrosby@cs.rice.edu> a écrit
> :
> 
> > I appreciate the help, but I am using an external USB drive, not an
> > external firewire drive. The kernel patch only modifies the
> > ieee1394_core.c file, which is unused in my system.
> 
> Uh, I wonder why I read "firewire" when it was "USB" ;)

And then also, USB 1.1 would only run at about 100Kbyte/sec max :(

Re: Terrible performance (<100kb/sec) in a cp -Raf

[godot]

>> Uh, I wonder why I read "firewire" when it was "USB" ;)
>
> And then also, USB 1.1 would only run at about 100Kbyte/sec max :(

That should read ~1000 KB/s.

Frank

Re: Terrible performance (<100kb/sec) in a cp -Raf

[Hendrik Visage]

On Fri, Jun 18, 2004 at 12:23:22PM +0200, godot@uni-paderborn.de wrote:
> >> Uh, I wonder why I read "firewire" when it was "USB" ;)
> >
> > And then also, USB 1.1 would only run at about 100Kbyte/sec max :(
> 
> That should read ~1000 KB/s.

USB 1.1 is 1.2Mbit/sec (give or take a few bits ;^) = ~150Kbyte/sec

Re: Terrible performance (<100kb/sec) in a cp -Raf

[Claudio Martins]

On Friday 18 June 2004 14:27, Hendrik Visage wrote:
> On Fri, Jun 18, 2004 at 12:23:22PM +0200, godot@uni-paderborn.de wrote:
> > >> Uh, I wonder why I read "firewire" when it was "USB" ;)
> > >
> > > And then also, USB 1.1 would only run at about 100Kbyte/sec max :(
> >
> > That should read ~1000 KB/s.
>
> USB 1.1 is 1.2Mbit/sec (give or take a few bits ;^) = ~150Kbyte/sec


  No.

  USB 1.1 is 12Mb/s max transfer speed. HID (mice, keyboard) devices use a low 
rate transfer mode of 1.2Mbit/sec, but USB Storage devices like external 
disks and dvd burners use up to 12Mbit/s, which gives a little above 1Mbyte/s 
transfer rate to/from a reiserfs filesystem on that external device.

  Newer USB 2.0 capable hardware can give 480Mbit/sec.

regards

Claudio