vfat <-> vfat copying of ~700MB file, so slow!

vfat <-> vfat copying of ~700MB file, so slow!

[Arkadiusz Miskiewicz]

Copying between vfat <-> vfat partitions is so slow. It seems
that it's vfat/msdos kernel driver problem because I tried to copy
this file between few partitions (all these on the same disc):

hda: IBM-DTLA-307030, ATA DISK drive
hda: 60036480 sectors (30739 MB) w/1916KiB Cache, CHS=3737/255/63, UDMA(33)

hdparm settings:
/dev/hda:
 multcount    = 16 (on)
 I/O support  =  0 (default 16-bit)
 unmaskirq    =  0 (off)
 using_dma    =  1 (on)
 keepsettings =  0 (off)
 nowerr       =  0 (off)
 readonly     =  0 (off)
 readahead    =  8 (on)
 geometry     = 3737/255/63, sectors = 60036480, start = 0

I tried it on 2.4.0 kernel, P166MMX, 96MB RAM, Soyo 5XB5 mainboard:
00:00.0 Host bridge: Intel Corporation 430TX - 82439TX MTXC (rev 01)
00:07.1 IDE interface: Intel Corporation 82371AB PIIX4 IDE (rev 01)

File was about 700MB and:

vfat -> ext2 -- by whole time, copying speed was about 3,6-3,8MB/s
ext2 -> vfat -- similar results
vfat -> vfat -- 3,4MB/s at beggining of copying,
		2,3MB/s after 1 minute of copying,
		1,7MB/s - 2 min,
		1,4MB/s - 3 min
		1,3MB/s - 4   - about 45% of file already copied
		1,2MB/s	- 5   - 51%
		1,1MB/s	- 6   - 58%
		1,0MB/s	- 7   - 63%
		1,0MB/s - 8   - 67%
		0,95MB/s - 9  - 72%
		0,91MB/s - 10 - 76%
		0,87MB/s - 11 - 81%
		0,84MB/s - 12 - 85%
		0,81MB/s - 13 -	88%
		0,78MB/s - 14 - 92%
		0,76MB/s - 15 - 96%
		0,74MB/s - 16 minute, 99% of file

so average speed is about 730kb/s only !!! I was copying under midnight
commander (4.5.51) so these speed values are from mc.

Patches or comments welcome.

-- 
Arkadiusz Miśkiewicz, AM2-6BONE    [ PLD GNU/Linux IPv6 ]
http://www.t17.ds.pwr.wroc.pl/~misiek/ipv6/   [ enabled ]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: vfat <-> vfat copying of ~700MB file, so slow!

[Brad Felmey]

On Wed, 24 Jan 2001 13:14:31 +0100, you, Arkadiusz Miskiewicz
<misiek@pld.ORG.PL>, wrote:

> I/O support  =  0 (default 16-bit)

hdparm -c1 /dev/hda, or are you running in 16-bit mode on purpose?
--
Brad Felmey
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: vfat <-> vfat copying of ~700MB file, so slow!

[Arkadiusz Miskiewicz]

On/Dnia Wed, Jan 24, 2001 at 12:23:03PM -0600, Brad Felmey wrote/napisał(a)
> > I/O support  =  0 (default 16-bit)
> 
> hdparm -c1 /dev/hda, or are you running in 16-bit mode on purpose?
no purpose. Setting this can only speed up all operations a bit but it doesn't
change nothing in vfat <-> vfat copying. It still slows down while copying.

> Brad Felmey

-- 
Arkadiusz Miśkiewicz, AM2-6BONE    [ PLD GNU/Linux IPv6 ]
http://www.t17.ds.pwr.wroc.pl/~misiek/ipv6/   [ enabled ]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: vfat <-> vfat copying of ~700MB file, so slow!

[Thunder from the hill]

Arkadiusz Miskiewicz wrote:
> 
> On/Dnia Wed, Jan 24, 2001 at 12:23:03PM -0600, Brad Felmey wrote/napisa³(a)
> > > I/O support  =  0 (default 16-bit)
> >
> > hdparm -c1 /dev/hda, or are you running in 16-bit mode on purpose?
> no purpose. Setting this can only speed up all operations a bit but it doesn't
> change nothing in vfat <-> vfat copying. It still slows down while copying.
I noticed the kernel to increase cache and let the buffers break down during long file copies, it seems like this is the wrong way if only copying one large file. This also happens on SMB connections. I don't know if this info is useful.

Thunder
---
I did a "cat /boot/vmlinuz >> /dev/audio" - and I think I heard god...
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: vfat <-> vfat copying of ~700MB file, so slow!

[Gábor Lénárt]

On Thu, Jan 25, 2001 at 11:56:35AM -0700, Thunder from the hill wrote:
> Arkadiusz Miskiewicz wrote:
> > 
> > On/Dnia Wed, Jan 24, 2001 at 12:23:03PM -0600, Brad Felmey wrote/napisa?(a)
> > > > I/O support  =  0 (default 16-bit)
> > >
> > > hdparm -c1 /dev/hda, or are you running in 16-bit mode on purpose?
> > no purpose. Setting this can only speed up all operations a bit but it doesn't
> > change nothing in vfat <-> vfat copying. It still slows down while copying.
> I noticed the kernel to increase cache and let the buffers break down during long file copies, it seems like this is the wrong way if only copying one large file. This also happens on SMB connections. I don't know if this info is useful.

Hmmm, imho this is a more general problem. Of course kernel does not know
if this is a single copy or data should be cached since it will be used
frequently in the near future. I don't know kernel internals very well
but I think cache is block cache. On higher layer (fs) it would be possible
to check out that we simply read blocks of file linear (which means
something like you mentioned especially on big files). I met similar
problems when I copy a harddisk to another (same type), with simple dd'ing
/dev/hda to /dev/hdb. Even with 192Mbs of RAM, the system goes down to
a very uninteractive state :( This copy task may be implemented on raw devices
to avoid caching nowdays. Please correct me, because these are only feelings
or what :)

-- 
 --[ Gábor Lénárt ]---[ Vivendi Telecom Hungary ]--[ lgb@supervisor.hu ]--
 U have 8 bit comp or chip of them and it's unused or to be sold? Call me!
 -------[ +36 30 2270823 ]------> LGB <-----[ Linux/UNIX/8bit 4ever ]-----
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: vfat <-> vfat copying of ~700MB file, so slow!

[Eric Lammerts]

On Wed, 24 Jan 2001, Brad Felmey wrote:
> > I/O support  =  0 (default 16-bit)
>
> hdparm -c1 /dev/hda, or are you running in 16-bit mode on purpose?

The -c option is only relevant for PIO modes. In this case DMA was on,
so it doesn't make any difference.

Eric

-- 
Eric Lammerts <eric@lammerts.org> | The best way to accelerate a computer
http://www.lammerts.org           | running Windows is at 9.8 m/s^2.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: vfat <-> vfat copying of ~700MB file, so slow!

[Albert D. Cahalan]

> Copying between vfat <-> vfat partitions is so slow. It seems
> that it's vfat/msdos kernel driver problem because I tried to copy

I reported this years ago, with a 700 kB file on a floppy and
a 4 MB file on a Zip disk. In both cases mcopy was several times
faster than the kernel code.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: vfat <-> vfat copying of ~700MB file, so slow!

[Pavel Machek]

Hi!

> > Copying between vfat <-> vfat partitions is so slow. It seems
> > that it's vfat/msdos kernel driver problem because I tried to copy
> 
> I reported this years ago, with a 700 kB file on a floppy and
> a 4 MB file on a Zip disk. In both cases mcopy was several times
> faster than the kernel code.

Perhaps linear scan of FAT?
								Pavel
-- 
I'm pavel@ucw.cz. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at discuss@linmodems.org
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: vfat <-> vfat copying of ~700MB file, so slow!

[Ion Badulescu]

On Fri, 26 Jan 2001 18:41:37 +0100, Pavel Machek <pavel@suse.cz> wrote:
> Hi!
> 
>> > Copying between vfat <-> vfat partitions is so slow. It seems
>> > that it's vfat/msdos kernel driver problem because I tried to copy
>> 
>> I reported this years ago, with a 700 kB file on a floppy and
>> a 4 MB file on a Zip disk. In both cases mcopy was several times
>> faster than the kernel code.
> 
> Perhaps linear scan of FAT?

Maybe. Quite likely, in fact. But there is no reason why fatfs can't
store the current FAT cluster number in struct file's private data,
making seeks, reads and writes O(1) (from O(n)).

You'll have to give up using generic_file_read(), however. So it's
not a trivial change.

Ion

-- 
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/