* USB storage: corrupted data transfers
@ 2007-11-10 21:05 Mark Lord
2007-11-10 21:20 ` [Linux-usb-users] " Alan Stern
2007-11-10 21:41 ` Mark Lord
0 siblings, 2 replies; 4+ messages in thread
From: Mark Lord @ 2007-11-10 21:05 UTC (permalink / raw)
To: linux-usb-users, Linux Kernel, mdharm-usb, dbrownell,
Andrew Morton
Something may be broken in USB / usb-storage land.
I've got a 2GB USB stick here.
I want to copy it to an image file on my hard drive:
cat /dev/sdb > usbkey.image1
Make a second copy, with or without unplugging/replugging the stick:
cat /dev/sdb > usbkey.image2
After doing this, the two copies *differ*.
So I wrote a little program to compare them and determine *how* they differ,
and the result is very interesting.
Periodically, or or the other file has 8192 *zero* bytes inserted
in place of it's data. Not replacing it's data, just inserted
into the stream, causing the real data (which follows) to be offset by 8192.
If one skips over the inserted 8192 bytes in the file, the data from that point
onward matches the other file, until another 8192 zeros are encountered.
The total sizes of the two image files match each other,
but that's probably just due to the logic in the 'cat' program.
I wonder where those extraneous pairs of zero pages are coming from?
This makes USB drives somewhat unreliable for backups and the like,
which unfortunately is exactly what just about everybody uses them for.
This (so far) is with 2.6.23.1 (w/slab allocater), and 2.6.24-rc2,
on two different machines (both Intel based, but different chipset, CPU, RAM, ...).
I suspect not all brands/models of USB sticks would give the same results,
but it is rather worrisome that it happens at all.
Mmmmmm ????
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Linux-usb-users] USB storage: corrupted data transfers
2007-11-10 21:05 USB storage: corrupted data transfers Mark Lord
@ 2007-11-10 21:20 ` Alan Stern
2007-11-10 21:41 ` Mark Lord
1 sibling, 0 replies; 4+ messages in thread
From: Alan Stern @ 2007-11-10 21:20 UTC (permalink / raw)
To: Mark Lord
Cc: linux-usb-users, Linux Kernel, mdharm-usb, dbrownell,
Andrew Morton
On Sat, 10 Nov 2007, Mark Lord wrote:
> Something may be broken in USB / usb-storage land.
>
> I've got a 2GB USB stick here.
> I want to copy it to an image file on my hard drive:
>
> cat /dev/sdb > usbkey.image1
>
> Make a second copy, with or without unplugging/replugging the stick:
>
> cat /dev/sdb > usbkey.image2
>
> After doing this, the two copies *differ*.
>
> So I wrote a little program to compare them and determine *how* they differ,
> and the result is very interesting.
>
> Periodically, or or the other file has 8192 *zero* bytes inserted
> in place of it's data. Not replacing it's data, just inserted
> into the stream, causing the real data (which follows) to be offset by 8192.
That's strange. It happens periodically, you say? What's the period?
> If one skips over the inserted 8192 bytes in the file, the data from that point
> onward matches the other file, until another 8192 zeros are encountered.
>
> The total sizes of the two image files match each other,
> but that's probably just due to the logic in the 'cat' program.
>
> I wonder where those extraneous pairs of zero pages are coming from?
Me too. It seems rather unlikely that they are coming from the USB
device or the transport, since at that level everything is addressed in
terms of sector numbers. I don't see how 16 extra sectors could just
get inserted. More believable would be a problem in the block layer,
the filesystem, or the application.
> This makes USB drives somewhat unreliable for backups and the like,
> which unfortunately is exactly what just about everybody uses them for.
Or it makes all drives somewhat unreliable, depending on just where the
problem is.
> This (so far) is with 2.6.23.1 (w/slab allocater), and 2.6.24-rc2,
> on two different machines (both Intel based, but different chipset, CPU, RAM, ...).
>
> I suspect not all brands/models of USB sticks would give the same results,
> but it is rather worrisome that it happens at all.
You could try doing a more specific test. Write a program to access
the block device sector-by-sector and store a distinct recognizable
byte pattern to each sector. (Maybe just repeat the 4-byte sector
number over and over.) Then write another program to read the
data back and see what bytes ended up in which sectors.
At the same time, use the usbmon facility (see
Documentation/usb/usbmon.txt) to record exactly what data gets sent
to/from the stick. That should at least be sufficient to determine the
problem is in the stick itself, in the SCSI/USB layers, or someplace
higher up.
Alan Stern
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: USB storage: corrupted data transfers
2007-11-10 21:05 USB storage: corrupted data transfers Mark Lord
2007-11-10 21:20 ` [Linux-usb-users] " Alan Stern
@ 2007-11-10 21:41 ` Mark Lord
2007-11-11 0:06 ` Mark Lord
1 sibling, 1 reply; 4+ messages in thread
From: Mark Lord @ 2007-11-10 21:41 UTC (permalink / raw)
To: linux-usb-users, Linux Kernel, mdharm-usb, dbrownell,
Andrew Morton
Mark Lord wrote:
> Something may be broken in USB / usb-storage land.
>
> I've got a 2GB USB stick here.
> I want to copy it to an image file on my hard drive:
>
> cat /dev/sdb > usbkey.image1
>
> Make a second copy, with or without unplugging/replugging the stick:
>
> cat /dev/sdb > usbkey.image2
>
> After doing this, the two copies *differ*.
Here's the information on the specific devices I'm trying this with.
I'm beginning to suspect a faulty stick, or maybe one that just doesn't
work reliably with the way Linux accesses it (?). The type that fails
(I have several of that model here) also gives some errors from "lsusb -v"
(see below for details).
I have another completely different brand/model of 2GB stick here
that has now been confirmed to NOT have any such issues.
Perhaps the USB experts here could comment on the differences
in information they report from lsusb --> maybe it's somethere there
that makes one of them fail, and the other one work?
First, the BAD one:
Bus 001 Device 084: ID 1307:0163
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 64
idVendor 0x1307
idProduct 0x0163
bcdDevice 1.00
iManufacturer 1 USB 2.0
iProduct 2 Flash Disk
iSerial 3 465743a79ca017
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 39
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 80mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 3
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 80 Bulk (Zip)
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x01 EP 1 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 1
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 1
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
bInterval 8
Device Qualifier (for other device speed):
bLength 10
bDescriptorType 6
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 64
bNumConfigurations 1
can't get debug descriptor: Connection timed out
cannot read device status, Connection timed out (110)
****************************
Now, the GOOD one:
Bus 001 Device 087: ID 1005:b113 Apacer Technology, Inc. Handy Steno 2.0 (256MB)
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 64
idVendor 0x1005 Apacer Technology, Inc.
idProduct 0xb113 Handy Steno 2.0 (256MB)
bcdDevice 1.00
iManufacturer 1
iProduct 2 USB FLASH DRIVE
iSerial 3 1962140002E9
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 32
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 200mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 80 Bulk (Zip)
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Device Qualifier (for other device speed):
bLength 10
bDescriptorType 6
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 64
bNumConfigurations 1
Device Status: 0x0000
(Bus Powered)
**************************
Now just a "diff -u" of the above:
--- bad 2007-11-10 16:29:35.000000000 -0500
+++ good 2007-11-10 16:36:59.000000000 -0500
@@ -1,4 +1,4 @@
-Bus 001 Device 084: ID 1307:0163
+Bus 001 Device 087: ID 1005:b113 Apacer Technology, Inc. Handy Steno 2.0 (256MB)
Device Descriptor:
bLength 18
bDescriptorType 1
@@ -7,29 +7,29 @@
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 64
- idVendor 0x1307
- idProduct 0x0163
+ idVendor 0x1005 Apacer Technology, Inc.
+ idProduct 0xb113 Handy Steno 2.0 (256MB)
bcdDevice 1.00
- iManufacturer 1 USB 2.0
- iProduct 2 Flash Disk
- iSerial 3 465743a79ca017
+ iManufacturer 1
+ iProduct 2 USB FLASH DRIVE
+ iSerial 3 1962140002E9
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
- wTotalLength 39
+ wTotalLength 32
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
- MaxPower 80mA
+ MaxPower 200mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
- bNumEndpoints 3
+ bNumEndpoints 2
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 80 Bulk (Zip)
@@ -37,33 +37,23 @@
Endpoint Descriptor:
bLength 7
bDescriptorType 5
- bEndpointAddress 0x01 EP 1 OUT
+ bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
- bInterval 1
+ bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
- bEndpointAddress 0x82 EP 2 IN
+ bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
- bInterval 1
- Endpoint Descriptor:
- bLength 7
- bDescriptorType 5
- bEndpointAddress 0x83 EP 3 IN
- bmAttributes 3
- Transfer Type Interrupt
- Synch Type None
- Usage Type Data
- wMaxPacketSize 0x0040 1x 64 bytes
- bInterval 8
+ bInterval 0
Device Qualifier (for other device speed):
bLength 10
bDescriptorType 6
@@ -73,7 +63,5 @@
bDeviceProtocol 0
bMaxPacketSize0 64
bNumConfigurations 1
-
-can't get debug descriptor: Connection timed out
-cannot read device status, Connection timed out (110)
-
+Device Status: 0x0000
+ (Bus Powered)
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: USB storage: corrupted data transfers
2007-11-10 21:41 ` Mark Lord
@ 2007-11-11 0:06 ` Mark Lord
0 siblings, 0 replies; 4+ messages in thread
From: Mark Lord @ 2007-11-11 0:06 UTC (permalink / raw)
To: linux-usb-users, Linux Kernel, mdharm-usb, dbrownell,
Andrew Morton
Mark Lord wrote:
> Mark Lord wrote:
>> Something may be broken in USB / usb-storage land.
>>
>> I've got a 2GB USB stick here.
>> I want to copy it to an image file on my hard drive:
>>
>> cat /dev/sdb > usbkey.image1
>>
>> Make a second copy, with or without unplugging/replugging the stick:
>>
>> cat /dev/sdb > usbkey.image2
>>
>> After doing this, the two copies *differ*.
>
> Here's the information on the specific devices I'm trying this with.
>
> I'm beginning to suspect a faulty stick, or maybe one that just doesn't
> work reliably with the way Linux accesses it (?). The type that fails
...
Yup, bad hardware. I've got 13 of those sticks here, and 11 are fine.
Move along folks, nothing to see here.. :)
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2007-11-11 0:06 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-11-10 21:05 USB storage: corrupted data transfers Mark Lord
2007-11-10 21:20 ` [Linux-usb-users] " Alan Stern
2007-11-10 21:41 ` Mark Lord
2007-11-11 0:06 ` Mark Lord
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox