Extremely poor umass transfer rates

Extremely poor umass transfer rates

[Mark Rosenstand]

I get transfer rates at around 30 kB/s to USB mass storage devices. It
applies to both my keyring and my mp3 player. Both are running vfat.

I'm running 2.6.12-rc3 for amd64 with patches for inotify and skge. The
motherboard is an ASUS K8V-X (VIA K8T800).

It worked alright earlier (2.6.10 or 2.6.11, I'll test later if
necessary.)

Also, if I transfer more than one file at a time the music tracks start
overlapping on my mp3 player.

Any hints?

-- 
  .-.    Mark Rosenstand        (-.)
  oo|                           cc )
 /`'\    (+45) 255 31337      3-n-(
(\_;/)   mark@geekworld.org    _(|/`->

Re: Extremely poor umass transfer rates

[Dave Jones]

On Thu, Apr 28, 2005 at 06:02:22PM +0200, Mark Rosenstand wrote:
 > I get transfer rates at around 30 kB/s to USB mass storage devices. It
 > applies to both my keyring and my mp3 player. Both are running vfat.
 > 
 > I'm running 2.6.12-rc3 for amd64 with patches for inotify and skge. The
 > motherboard is an ASUS K8V-X (VIA K8T800).
 > 
 > It worked alright earlier (2.6.10 or 2.6.11, I'll test later if
 > necessary.)
 > 
 > Also, if I transfer more than one file at a time the music tracks start
 > overlapping on my mp3 player.

Are you running it on a USB 2.0 capable interface ?
Is your mp3 player USB2.0 capable ?
USB1.1 is painfully slow for storage.

		Dave

Re: Extremely poor umass transfer rates

[Randy.Dunlap]

On Thu, 28 Apr 2005 12:59:15 -0400
Dave Jones <davej@redhat.com> wrote:

> On Thu, Apr 28, 2005 at 06:02:22PM +0200, Mark Rosenstand wrote:
>  > I get transfer rates at around 30 kB/s to USB mass storage devices. It
>  > applies to both my keyring and my mp3 player. Both are running vfat.
>  > 
>  > I'm running 2.6.12-rc3 for amd64 with patches for inotify and skge. The
>  > motherboard is an ASUS K8V-X (VIA K8T800).
>  > 
>  > It worked alright earlier (2.6.10 or 2.6.11, I'll test later if
>  > necessary.)
>  > 
>  > Also, if I transfer more than one file at a time the music tracks start
>  > overlapping on my mp3 player.
> 
> Are you running it on a USB 2.0 capable interface ?
> Is your mp3 player USB2.0 capable ?
> USB1.1 is painfully slow for storage.

And which USB storage driver are you using (USB storage or
USB block driver)?  (or what device names are you mounting?)

---
~Randy

Re: Extremely poor umass transfer rates

[Mark Rosenstand]

On Thu, 2005-04-28 at 12:59 -0400, Dave Jones wrote:
> On Thu, Apr 28, 2005 at 06:02:22PM +0200, Mark Rosenstand wrote:
>  > I get transfer rates at around 30 kB/s to USB mass storage devices. It
>  > applies to both my keyring and my mp3 player. Both are running vfat.
>  > 
>  > I'm running 2.6.12-rc3 for amd64 with patches for inotify and skge. The
>  > motherboard is an ASUS K8V-X (VIA K8T800).
>  > 
>  > It worked alright earlier (2.6.10 or 2.6.11, I'll test later if
>  > necessary.)
>  > 
>  > Also, if I transfer more than one file at a time the music tracks start
>  > overlapping on my mp3 player.
> 
> Are you running it on a USB 2.0 capable interface ?

I believe so. How do I verify it?

I've tried to move it to the other on-board hub but without results.

> Is your mp3 player USB2.0 capable ?

I'm not sure. But I do know that it used to be *much* faster than this.
My keyring is USB 2.0 capable and that's slow as well.

> USB1.1 is painfully slow for storage.

Yeah, but I don't think it should be 30 kB/s.

Some more details:


The line that 'hald' puts in fstab looks like this:

	/dev/sdb /media/usbdisk vfat \
		user,exec,noauto,utf8,noatime,sync,managed 0 0


The relevant parts of my .config:

	#
	# Miscellaneous USB options
	#
	CONFIG_USB_DEVICEFS=y
	# CONFIG_USB_BANDWIDTH is not set
	# CONFIG_USB_DYNAMIC_MINORS is not set
	# CONFIG_USB_SUSPEND is not set
	# CONFIG_USB_OTG is not set

	#
	# USB Host Controller Drivers
	#
	CONFIG_USB_EHCI_HCD=y
	# CONFIG_USB_EHCI_SPLIT_ISO is not set
	# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
	# CONFIG_USB_OHCI_HCD is not set
	CONFIG_USB_UHCI_HCD=y
	# CONFIG_USB_SL811_HCD is not set

	[...]

	CONFIG_USB_STORAGE=m


What my dmesg tells me when I attach the device:

	usb 2-1: new full speed USB device using uhci_hcd and address 6
	scsi6 : SCSI emulation for USB Mass Storage devices
	usb-storage: device found at 6
	usb-storage: waiting for device to settle before scanning
	  Vendor: iriver    Model: MassStorage Disc  Rev:
	  Type:   Direct-Access                      ANSI SCSI revision: 00
	SCSI device sdb: 249857 512-byte hdwr sectors (128 MB)
	sdb: Write Protect is off
	sdb: Mode Sense: 45 00 00 08
	sdb: assuming drive cache: write through
	SCSI device sdb: 249857 512-byte hdwr sectors (128 MB)
	sdb: Write Protect is off
	sdb: Mode Sense: 45 00 00 08
	sdb: assuming drive cache: write through
	sdb:
	Attached scsi removable disk sdb at scsi6, channel 0, id 0, lun 0
	usb-storage: device scan complete

Thanks for the advices so far :-)

-- 
  .-.    Mark Rosenstand        (-.)
  oo|                           cc )
 /`'\    (+45) 255 31337      3-n-(
(\_;/)   mark@geekworld.org    _(|/`->

Re: Extremely poor umass transfer rates

[Randy.Dunlap]

On Thu, 28 Apr 2005 19:55:40 +0200 Mark Rosenstand wrote:

| On Thu, 2005-04-28 at 12:59 -0400, Dave Jones wrote:
| > On Thu, Apr 28, 2005 at 06:02:22PM +0200, Mark Rosenstand wrote:
| >  > I get transfer rates at around 30 kB/s to USB mass storage devices. It
| >  > applies to both my keyring and my mp3 player. Both are running vfat.
| >  > 
| >  > I'm running 2.6.12-rc3 for amd64 with patches for inotify and skge. The
| >  > motherboard is an ASUS K8V-X (VIA K8T800).
| >  > 
| >  > It worked alright earlier (2.6.10 or 2.6.11, I'll test later if
| >  > necessary.)
| >  > 
| >  > Also, if I transfer more than one file at a time the music tracks start
| >  > overlapping on my mp3 player.
| > 
| > Are you running it on a USB 2.0 capable interface ?

with the EHCI host controller driver loaded?

| I believe so. How do I verify it?

(see below)
and post /proc/bus/usb/devices (contents)

| I've tried to move it to the other on-board hub but without results.
| 
| > Is your mp3 player USB2.0 capable ?
| 
| I'm not sure. But I do know that it used to be *much* faster than this.
| My keyring is USB 2.0 capable and that's slow as well.
| 
| > USB1.1 is painfully slow for storage.
| 
| Yeah, but I don't think it should be 30 kB/s.
| 
| Some more details:
| 
| 
| The line that 'hald' puts in fstab looks like this:
| 
| 	/dev/sdb /media/usbdisk vfat \
| 		user,exec,noauto,utf8,noatime,sync,managed 0 0
| 
| 
| The relevant parts of my .config:
| 
| 	#
| 	# Miscellaneous USB options
| 	#
| 	CONFIG_USB_DEVICEFS=y
| 	# CONFIG_USB_BANDWIDTH is not set
| 	# CONFIG_USB_DYNAMIC_MINORS is not set
| 	# CONFIG_USB_SUSPEND is not set
| 	# CONFIG_USB_OTG is not set
| 
| 	#
| 	# USB Host Controller Drivers
| 	#
| 	CONFIG_USB_EHCI_HCD=y
OK, EHCI answer is Yes.

| 	# CONFIG_USB_EHCI_SPLIT_ISO is not set
| 	# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
| 	# CONFIG_USB_OHCI_HCD is not set
| 	CONFIG_USB_UHCI_HCD=y
| 	# CONFIG_USB_SL811_HCD is not set
| 
| 	[...]
| 
| 	CONFIG_USB_STORAGE=m
| 
| 
| What my dmesg tells me when I attach the device:
| 
| 	usb 2-1: new full speed USB device using uhci_hcd and address 6

"full speed" is USB 1.x (12 Mbps, not high-speed 480 Mbps),
so the device is reporting itself as less than high-speed
or the USB descriptor parsing is missing it somehow (but it
works for a few hundred other devices).

| 	scsi6 : SCSI emulation for USB Mass Storage devices
| 	usb-storage: device found at 6
| 	usb-storage: waiting for device to settle before scanning
| 	  Vendor: iriver    Model: MassStorage Disc  Rev:
| 	  Type:   Direct-Access                      ANSI SCSI revision: 00
| 	SCSI device sdb: 249857 512-byte hdwr sectors (128 MB)
| 	sdb: Write Protect is off
| 	sdb: Mode Sense: 45 00 00 08
| 	sdb: assuming drive cache: write through
| 	SCSI device sdb: 249857 512-byte hdwr sectors (128 MB)
| 	sdb: Write Protect is off
| 	sdb: Mode Sense: 45 00 00 08
| 	sdb: assuming drive cache: write through
| 	sdb:
| 	Attached scsi removable disk sdb at scsi6, channel 0, id 0, lun 0
| 	usb-storage: device scan complete
| 
| Thanks for the advices so far :-)


---
~Randy

Re: Extremely poor umass transfer rates

[Paulo Marques]

Randy.Dunlap wrote:
> [...]On Thu, 28 Apr 2005 19:55:40 +0200 Mark Rosenstand wrote:
> | On Thu, 2005-04-28 at 12:59 -0400, Dave Jones wrote:
 > [...]
> | > USB1.1 is painfully slow for storage.
> | 
> | Yeah, but I don't think it should be 30 kB/s.

Yes it should be much more than 30kB/s. I remember measuring about 900kB/s.

> | 
> | Some more details:
> | 
> | 
> | The line that 'hald' puts in fstab looks like this:
> | 
> | 	/dev/sdb /media/usbdisk vfat \
> | 		user,exec,noauto,utf8,noatime,sync,managed 0 0

The "sync" flag is what is killing your performance. It is needed if you 
intend to remove your usb pen without warning, but if you are going to 
unmount carefully you don't need it at all.

Try mounting the device as root somewhere else without the "sync" flag 
and measure the performance that way, to see the difference.

I hope this helps,

-- 
Paulo Marques - www.grupopie.com

All that is necessary for the triumph of evil is that good men do nothing.
Edmund Burke (1729 - 1797)

Re: Extremely poor umass transfer rates

[Mark Rosenstand]

On Thu, 2005-04-28 at 19:19 +0100, Paulo Marques wrote:
> > | The line that 'hald' puts in fstab looks like this:
> > | 
> > | 	/dev/sdb /media/usbdisk vfat \
> > | 		user,exec,noauto,utf8,noatime,sync,managed 0 0
> 
> The "sync" flag is what is killing your performance. It is needed if you 
> intend to remove your usb pen without warning, but if you are going to 
> unmount carefully you don't need it at all.
> 
> Try mounting the device as root somewhere else without the "sync" flag 
> and measure the performance that way, to see the difference.

Wow. That seems to speed things up alot. However I can't unmount it
again, I keep getting "umount: /media/usbdisk: device is
busy" (twice(?)). It's been 5 minutes since I did the transfer (4 MB
file) now.

> I hope this helps,

It sure did. Thanks!

-- 
  .-.    Mark Rosenstand        (-.)
  oo|                           cc )
 /`'\    (+45) 255 31337      3-n-(
(\_;/)   mark@geekworld.org    _(|/`->

Re: Extremely poor umass transfer rates

[Mark Rosenstand]

On Thu, 2005-04-28 at 20:47 +0200, Mark Rosenstand wrote:
> > Try mounting the device as root somewhere else without the "sync" flag 
> > and measure the performance that way, to see the difference.
> 
> Wow. That seems to speed things up alot. However I can't unmount it
> again, I keep getting "umount: /media/usbdisk: device is
> busy" (twice(?)). It's been 5 minutes since I did the transfer (4 MB
> file) now.

Err, I accidently mixed up the device names of my keyring and the mp3
player. It works allright.

(It flushes changes to the device when I unmount it now, which takes
around 8 seconds for a 4 MB file.)

Thanks a lot, Paulo!

-- 
  .-.    Mark Rosenstand        (-.)
  oo|                           cc )
 /`'\    (+45) 255 31337      3-n-(
(\_;/)   mark@geekworld.org    _(|/`->

Re: Extremely poor umass transfer rates

[Mark Rosenstand]

On Thu, 2005-04-28 at 11:06 -0700, Randy.Dunlap wrote:
[snip]
> and post /proc/bus/usb/devices (contents)

See the attached 'devices' file.

Summary:

    T:  Bus=02 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#=  7 Spd=12  MxCh= 0
    S:  Manufacturer=iriver Limited.
    S:  Product=iriver Internet Audio Player N10

    T:  Bus=01 Lev=01 Prnt=01 Port=01 Cnt=01 Dev#=  9 Spd=480 MxCh= 0
    S:  Manufacturer=SanDisk Corp.
    S:  Product=Cruzer Micro

So my mp3 player wasn't USB 2.0. (I told you I wasn't sure.)

I still don't think it should transfer at 20-30 kB/s though, and
certainly not that the keyring (Spd=480) should either. The mp3 player
used to be *much* faster.

-- 
  .-.    Mark Rosenstand        (-.)
  oo|                           cc )
 /`'\    (+45) 255 31337      3-n-(
(\_;/)   mark@geekworld.org    _(|/`->

Re: Extremely poor umass transfer rates

[Tomasz Torcz]

[-- Attachment #1: Type: text/plain, Size: 2196 bytes --]

On Thu, Apr 28, 2005 at 07:55:40PM +0200, Mark Rosenstand wrote:
> The line that 'hald' puts in fstab looks like this:
> 
> 	/dev/sdb /media/usbdisk vfat \
> 		user,exec,noauto,utf8,noatime,sync,managed 0 0

 Are you sure it's correct? I can't even mount with those options:

#v+
# mount /dev/sdb1 /mnt/other -t vfat -o user,exec,noauto,utf8,noatime,sync,managed 
mount: wrong fs type, bad option, bad superblock on /dev/sdb1
       missing codepage or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

# dmesg | tail -2 
usb-storage: device scan complete
FAT: Unrecognized mount option "managed" or missing value
#v-

 Omitting "managed" seems to work. But it's slooow:

# dd if=/dev/zero  bs=1M count=100 | pv > /mnt/other/100MB
1,17MB 0:00:28 [41,7kB/s] [      <=>                ]

It stays at about 40 kB/s during all transfer. 
Reading is as fast as it should be = about 18 MB/s (after umount, mount
again, to clear cache). 

 My device is Kingston USB memory stick with USB 2.0, connected to USB
2.0 controller of nForce2 motherboard.

 Here's dmesg after plugin:

#v+
usb 1-6: new high speed USB device using ehci_hcd and address 3
Initializing USB Mass Storage driver...
scsi2 : SCSI emulation for USB Mass Storage devices
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
usb-storage: device found at 3
usb-storage: waiting for device to settle before scanning
  Vendor: Kingston  Model: DataTraveler II+  Rev: 1.13
  Type:   Direct-Access                      ANSI SCSI revision: 00
SCSI device sdb: 502784 512-byte hdwr sectors (257 MB)
sdb: Write Protect is off
sdb: Mode Sense: 23 00 00 00
sdb: assuming drive cache: write through
SCSI device sdb: 502784 512-byte hdwr sectors (257 MB)
sdb: Write Protect is off
sdb: Mode Sense: 23 00 00 00
sdb: assuming drive cache: write through
 sdb: sdb1
Attached scsi removable disk sdb at scsi2, channel 0, id 0, lun 0
usb-storage: device scan complete
FAT: Unrecognized mount option "managed" or missing value
#v-

 .config attached

-- 
Tomasz Torcz                 "God, root, what's the difference?"
zdzichu@irc.-nie.spam-.pl         "God is more forgiving."


[-- Attachment #2: config.gz --]
[-- Type: application/x-gunzip, Size: 9865 bytes --]

Re: Extremely poor umass transfer rates

[J.A. Magallon]

On 04.28, Tomasz Torcz wrote:
> On Thu, Apr 28, 2005 at 07:55:40PM +0200, Mark Rosenstand wrote:
> > The line that 'hald' puts in fstab looks like this:
> > 
> > 	/dev/sdb /media/usbdisk vfat \
> > 		user,exec,noauto,utf8,noatime,sync,managed 0 0
> 
>  Are you sure it's correct? I can't even mount with those options:
> 
> #v+
> # mount /dev/sdb1 /mnt/other -t vfat -o user,exec,noauto,utf8,noatime,sync,managed 
> mount: wrong fs type, bad option, bad superblock on /dev/sdb1
>        missing codepage or other error
>        In some cases useful info is found in syslog - try
>        dmesg | tail  or so
> 
> # dmesg | tail -2 
> usb-storage: device scan complete
> FAT: Unrecognized mount option "managed" or missing value
> #v-
> 
>  Omitting "managed" seems to work. But it's slooow:
> 
> # dd if=/dev/zero  bs=1M count=100 | pv > /mnt/other/100MB
> 1,17MB 0:00:28 [41,7kB/s] [      <=>                ]
> 
> It stays at about 40 kB/s during all transfer. 
> Reading is as fast as it should be = about 18 MB/s (after umount, mount
> again, to clear cache). 
> 

Je, je, you're dreaming. The limit on pendrives, flash mp3 players and so
on is the flash memory read/write speed. You are lucky if you get 1Mb/s.
So for a flash based device, dont ever worry about if it is USB 1.1 or 2.0.
Things are different if you have a disk based device (iPod, iPod mini).
A disk is a disk.

The difference in total time should be minimal between sync and async mounts,
if you take into account filesystem sync/umount time.

Anyways, your 40 Kb/sec seem too sloow. But dont expect your 19Mb/s, only
about 1Mb/s (reading, writing thigs drop to 600Kb/s).

by

--
J.A. Magallon <jamagallon()able!es>     \               Software is like sex:
werewolf!able!es                         \         It's better when it's free
Mandriva Linux release 2006.0 (Cooker) for i586
Linux 2.6.11-jam14 (gcc 4.0.0 (4.0.0-1mdk for Mandriva Linux

Re: Extremely poor umass transfer rates

[Robert Hancock]

J.A. Magallon wrote:
> Je, je, you're dreaming. The limit on pendrives, flash mp3 players and so
> on is the flash memory read/write speed. You are lucky if you get 1Mb/s.
> So for a flash based device, dont ever worry about if it is USB 1.1 or 2.0.

Lots of flash is much faster than that these days. On the Kingston USB 
2.0 pen drive I have, there's a HUGE speed difference between USB 1.1 
and 2.0. I'm not sure what the throughput is exactly under 2.0, but it's 
far faster than 1 MB/sec.

-- 
Robert Hancock      Saskatoon, SK, Canada
To email, remove "nospam" from hancockr@nospamshaw.ca
Home Page: http://www.roberthancock.com/

Re: Extremely poor umass transfer rates

[Joe]

Something is definetly going on with either vfat or the USB drivers...
My ipod is unrunnable on linux now.  To put it plainly, it transfers
painfully slow (on USB 2.0 mind you), it randomly ceases to respond
during file transfers.. and will only respond if replugged in.. its
corrupted my music and the fs to the point where i've now done two
low-level formats, and every time i put the stuff back on, the same
problems persist.

Oh yeah, my CF cards do the same thing.. you'll be transferring some
files.. and then it just stops responding.  I've tested it on my USB
1.1 ports and my USB 2.0 card, also with two completley different
cardreaders.

Its just bad.. my current kernel is 2.6.12-rc3-mm2, but i've seen
these problems for awile now.. its just with this release, its much
worse.

Joe

Re: Extremely poor umass transfer rates

[Greg KH]

On Wed, May 04, 2005 at 11:13:55PM -0400, Joe wrote:
> Something is definetly going on with either vfat or the USB drivers...
> My ipod is unrunnable on linux now.  To put it plainly, it transfers
> painfully slow (on USB 2.0 mind you), it randomly ceases to respond
> during file transfers.. and will only respond if replugged in.. its
> corrupted my music and the fs to the point where i've now done two
> low-level formats, and every time i put the stuff back on, the same
> problems persist.

Are you using the ub or usb-storage driver?

thanks,

greg k-h

Re: Extremely poor umass transfer rates

[Joe]

On 5/5/05, Greg KH <greg@kroah.com> wrote:
> On Wed, May 04, 2005 at 11:13:55PM -0400, Joe wrote:
> > Something is definetly going on with either vfat or the USB drivers...
> > My ipod is unrunnable on linux now.  To put it plainly, it transfers
> > painfully slow (on USB 2.0 mind you), it randomly ceases to respond
> > during file transfers.. and will only respond if replugged in.. its
> > corrupted my music and the fs to the point where i've now done two
> > low-level formats, and every time i put the stuff back on, the same
> > problems persist.
> 
> Are you using the ub or usb-storage driver?
> 
> thanks,
> 
> greg k-h
> 

Using usb-storage.  I double checked that it wasn't a hardware failure
on the ipod by adding stuff to it in windows... It performed
perfectly.

Thanks again,

Joe

Re: Extremely poor umass transfer rates

[Joe]

On 5/5/05, Joe <joecool1029@gmail.com> wrote:
> Using usb-storage.  I double checked that it wasn't a hardware failure
> on the ipod by adding stuff to it in windows... It performed
> perfectly.
> 
> Thanks again,
> 
> Joe
> 

Upon triple checking.... I found there is some hardware issues.  So i
am sending it in for a service.  But I still would like to say that it
is very slow, and unreliable with compact flash cards (on two
different readers).

Re: Extremely poor umass transfer rates

[Pete Zaitcev]

On Thu, 5 May 2005 01:22:50 -0700, Greg KH <greg@kroah.com> wrote:
> On Wed, May 04, 2005 at 11:13:55PM -0400, Joe wrote:
> > Something is definetly going on with either vfat or the USB drivers...
> > My ipod is unrunnable on linux now.  To put it plainly, it transfers
> > painfully slow (on USB 2.0 mind you), it randomly ceases to respond
> > during file transfers.. and will only respond if replugged in.. its
> > corrupted my music and the fs to the point where i've now done two
> > low-level formats, and every time i put the stuff back on, the same
> > problems persist.
> 
> Are you using the ub or usb-storage driver?

Joe's symptoms are not consistent with ub. I use ub with my iPod, it works
just fine, only somewhat slow (my partition size on iPod is a multiply
of 4KB, so it only loses speed because of external fragmentation).
I suspect Jor has cabling problems or something like that.

If anyone observes data integrity issues with ub, I need to know
right away.

-- Pete