* Re: [RFC][PATCH] fix sign extension with 1.5TB usb-storage LBD=y
2009-04-21 21:18 ` Matthew Wilcox
@ 2009-04-21 21:29 ` Randy Dunlap
2009-04-21 21:31 ` Matthew Wilcox
2009-04-21 22:00 ` [PATCH v2] " Dave Hansen
2009-04-21 22:49 ` [RFC][PATCH] " Dave Hansen
2 siblings, 1 reply; 12+ messages in thread
From: Randy Dunlap @ 2009-04-21 21:29 UTC (permalink / raw)
To: Matthew Wilcox
Cc: Dave Hansen, linux-kernel, mdharm-usb, linux-usb, usb-storage,
James Bottomley, linux-scsi
Matthew Wilcox wrote:
> On Tue, Apr 21, 2009 at 01:52:54PM -0700, Dave Hansen wrote:
>> This is with current git as of this morning, which is at v2.6.30-rc2.
>>
>> I have a 1.5TB USB device which gets a bit angry when I plug it in. It
>> ends up with a scsi_disk->capacity of ffffffffaea87b30. I tracked it
>> down to the lba calculation in read_capacity_10():
>>
>> lba = (buffer[0] << 24) | (buffer[1] << 16) |
>> (buffer[2] << 8) | buffer[3];
>>
>> lba is getting all 0xf's in its high 32 bits. It seems odd that this
>> would happen since 'buffer' is an 'unsigned char', but that is
>> apparently what is going on. Note that this isn't an issue 32-bit
>> kernels compiled with CONFIG_LBD=n since there's no more bits into which
>> the sign could be extended.
>
> I think I know ... unsigned char gets promoted to signed int since it will
> fit. then signed int gets cast to unsigned long long, sign-extending. C's
> promotion rules have always felt a bit wacky to me.
>
>> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
>> index 3fcb64b..db60e96 100644
>> --- a/drivers/scsi/sd.c
>> +++ b/drivers/scsi/sd.c
>> @@ -1402,7 +1402,7 @@ static int read_capacity_10(struct scsi_disk *sdkp, struct scsi_device *sdp,
>>
>> sector_size = (buffer[4] << 24) | (buffer[5] << 16) |
>> (buffer[6] << 8) | buffer[7];
>> - lba = (buffer[0] << 24) | (buffer[1] << 16) |
>> + lba = ((sector_t)buffer[0] << 24) | (buffer[1] << 16) |
>> (buffer[2] << 8) | buffer[3];
>
> this certainly fixes your problem. I'd prefer this patch instead, just
> because I find the cast unaesthetic ...
>
> ----
>
> Fix READ CAPACITY 10 with drives over 1TB
>
> Shifting an unsigned char implicitly casts it to a signed int. This
> caused 'lba' to sign-extend and Linux would then try READ CAPACITY 16
> which was not supported by at least one drive. Making 'lba' an unsigned
> int ensures that sign extension will not occur.
>
> Signed-off-by: Matthew Wilcox <willy@linux.intel.com>
>
> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
> index 3fcb64b..c856b1b 100644
> --- a/drivers/scsi/sd.c
> +++ b/drivers/scsi/sd.c
> @@ -1373,7 +1373,7 @@ static int read_capacity_10(struct scsi_disk *sdkp, struct scsi_device *sdp,
> int sense_valid = 0;
> int the_result;
> int retries = 3;
> - sector_t lba;
> + unsigned lba;
sector_t is either unsigned long or u64, depending on CONFIG_LBD.
Are you saying (implying) that the higher-order bits of it don't matter here?
If so, I'd just like for that to be clear(er).
> unsigned sector_size;
>
> do {
> @@ -1413,7 +1413,7 @@ static int read_capacity_10(struct scsi_disk *sdkp, struct scsi_device *sdp,
> return -EOVERFLOW;
> }
>
> - sdkp->capacity = lba + 1;
> + sdkp->capacity = (sector_t)lba + 1;
> return sector_size;
> }
>
>
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [RFC][PATCH] fix sign extension with 1.5TB usb-storage LBD=y
2009-04-21 21:29 ` Randy Dunlap
@ 2009-04-21 21:31 ` Matthew Wilcox
0 siblings, 0 replies; 12+ messages in thread
From: Matthew Wilcox @ 2009-04-21 21:31 UTC (permalink / raw)
To: Randy Dunlap
Cc: Dave Hansen, linux-kernel, mdharm-usb, linux-usb, usb-storage,
James Bottomley, linux-scsi
On Tue, Apr 21, 2009 at 02:29:54PM -0700, Randy Dunlap wrote:
> > Fix READ CAPACITY 10 with drives over 1TB
> >
> > Shifting an unsigned char implicitly casts it to a signed int. This
> > caused 'lba' to sign-extend and Linux would then try READ CAPACITY 16
> > which was not supported by at least one drive. Making 'lba' an unsigned
> > int ensures that sign extension will not occur.
> >
> > Signed-off-by: Matthew Wilcox <willy@linux.intel.com>
> >
> > diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
> > index 3fcb64b..c856b1b 100644
> > --- a/drivers/scsi/sd.c
> > +++ b/drivers/scsi/sd.c
> > @@ -1373,7 +1373,7 @@ static int read_capacity_10(struct scsi_disk *sdkp, struct scsi_device *sdp,
> > int sense_valid = 0;
> > int the_result;
> > int retries = 3;
> > - sector_t lba;
> > + unsigned lba;
>
> sector_t is either unsigned long or u64, depending on CONFIG_LBD.
> Are you saying (implying) that the higher-order bits of it don't matter here?
> If so, I'd just like for that to be clear(er).
We're reading 32 bits of information from the drive response. The top
32 bits are implicitly zero.
--
Matthew Wilcox Intel Open Source Technology Centre
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours. We can't possibly take such
a retrograde step."
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v2] fix sign extension with 1.5TB usb-storage LBD=y
2009-04-21 21:18 ` Matthew Wilcox
2009-04-21 21:29 ` Randy Dunlap
@ 2009-04-21 22:00 ` Dave Hansen
2009-04-21 23:03 ` Matthew Wilcox
2009-04-22 7:32 ` Boaz Harrosh
2009-04-21 22:49 ` [RFC][PATCH] " Dave Hansen
2 siblings, 2 replies; 12+ messages in thread
From: Dave Hansen @ 2009-04-21 22:00 UTC (permalink / raw)
To: Matthew Wilcox
Cc: linux-kernel, mdharm-usb, linux-usb, usb-storage, James Bottomley,
linux-scsi, viro
Here's a patch implementing Al's suggestion. Not quite as trivial as
Matthew's, but even nicer aesthetically. (Description stolen from
Matthew's patch). I have verified that this fixes my issue.
--
Shifting an unsigned char implicitly casts it to a signed int. This
caused 'lba' to sign-extend and Linux would then try READ CAPACITY 16
which was not supported by at least one drive. Using the
get_unaligned_be*() helpers keeps us from having to worry about how the
extension might occur.
Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 3fcb64b..c50142b 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -50,6 +50,7 @@
#include <linux/string_helpers.h>
#include <linux/async.h>
#include <asm/uaccess.h>
+#include <asm/unaligned.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
@@ -1344,12 +1345,8 @@ static int read_capacity_16(struct scsi_disk *sdkp, struct scsi_device *sdp,
return -EINVAL;
}
- sector_size = (buffer[8] << 24) | (buffer[9] << 16) |
- (buffer[10] << 8) | buffer[11];
- lba = (((u64)buffer[0] << 56) | ((u64)buffer[1] << 48) |
- ((u64)buffer[2] << 40) | ((u64)buffer[3] << 32) |
- ((u64)buffer[4] << 24) | ((u64)buffer[5] << 16) |
- ((u64)buffer[6] << 8) | (u64)buffer[7]);
+ sector_size = get_unaligned_be32(&buffer[8]);
+ lba = get_unaligned_be64(&buffer[0]);
sd_read_protection_type(sdkp, buffer);
@@ -1400,10 +1397,8 @@ static int read_capacity_10(struct scsi_disk *sdkp, struct scsi_device *sdp,
return -EINVAL;
}
- sector_size = (buffer[4] << 24) | (buffer[5] << 16) |
- (buffer[6] << 8) | buffer[7];
- lba = (buffer[0] << 24) | (buffer[1] << 16) |
- (buffer[2] << 8) | buffer[3];
+ sector_size = get_unaligned_be32(&buffer[4]);
+ lba = get_unaligned_be32(&buffer[0]);
if ((sizeof(sdkp->capacity) == 4) && (lba == 0xffffffff)) {
sd_printk(KERN_ERR, sdkp, "Too big for this kernel. Use a "
-- Dave
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH v2] fix sign extension with 1.5TB usb-storage LBD=y
2009-04-21 22:00 ` [PATCH v2] " Dave Hansen
@ 2009-04-21 23:03 ` Matthew Wilcox
2009-04-21 23:43 ` Dave Hansen
2009-04-22 7:32 ` Boaz Harrosh
1 sibling, 1 reply; 12+ messages in thread
From: Matthew Wilcox @ 2009-04-21 23:03 UTC (permalink / raw)
To: Dave Hansen
Cc: linux-kernel, mdharm-usb, linux-usb, usb-storage, James Bottomley,
linux-scsi, viro
On Tue, Apr 21, 2009 at 03:00:10PM -0700, Dave Hansen wrote:
> Here's a patch implementing Al's suggestion. Not quite as trivial as
> Matthew's, but even nicer aesthetically. (Description stolen from
> Matthew's patch). I have verified that this fixes my issue.
I prefer this approach, but I'm equally happy with whichever patch
James chooses.
> Shifting an unsigned char implicitly casts it to a signed int. This
> caused 'lba' to sign-extend and Linux would then try READ CAPACITY 16
> which was not supported by at least one drive. Using the
> get_unaligned_be*() helpers keeps us from having to worry about how the
> extension might occur.
>
> Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
Reviewed-by: Matthew Wilcox <willy@linux.intel.com>
> @@ -1344,12 +1345,8 @@ static int read_capacity_16(struct scsi_disk *sdkp, struct scsi_device *sdp,
> return -EINVAL;
> }
>
> - sector_size = (buffer[8] << 24) | (buffer[9] << 16) |
> - (buffer[10] << 8) | buffer[11];
> - lba = (((u64)buffer[0] << 56) | ((u64)buffer[1] << 48) |
> - ((u64)buffer[2] << 40) | ((u64)buffer[3] << 32) |
> - ((u64)buffer[4] << 24) | ((u64)buffer[5] << 16) |
> - ((u64)buffer[6] << 8) | (u64)buffer[7]);
> + sector_size = get_unaligned_be32(&buffer[8]);
> + lba = get_unaligned_be64(&buffer[0]);
The smallest of quibbles ... you have an excess space after the = ...
--
Matthew Wilcox Intel Open Source Technology Centre
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours. We can't possibly take such
a retrograde step."
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2] fix sign extension with 1.5TB usb-storage LBD=y
2009-04-21 23:03 ` Matthew Wilcox
@ 2009-04-21 23:43 ` Dave Hansen
0 siblings, 0 replies; 12+ messages in thread
From: Dave Hansen @ 2009-04-21 23:43 UTC (permalink / raw)
To: Matthew Wilcox
Cc: linux-kernel, mdharm-usb, linux-usb, usb-storage, James Bottomley,
linux-scsi, viro
Here's a patch implementing Al's suggestion. Not quite as trivial as
Matthew's, but even nicer aesthetically. (Description stolen from
Matthew's patch). I have verified that this fixes my issue.
v3 fixed extra whitespace.
--
Shifting an unsigned char implicitly casts it to a signed int. This
caused 'lba' to sign-extend and Linux would then try READ CAPACITY 16
which was not supported by at least one drive. Using the
get_unaligned_be*() helpers keeps us from having to worry about how the
extension might occur.
Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
Reviewed-by: Matthew Wilcox <willy@linux.intel.com>
diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 3fcb64b..c50142b 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -50,6 +50,7 @@
#include <linux/string_helpers.h>
#include <linux/async.h>
#include <asm/uaccess.h>
+#include <asm/unaligned.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
@@ -1344,12 +1345,8 @@ static int read_capacity_16(struct scsi_disk *sdkp, struct scsi_device *sdp,
return -EINVAL;
}
- sector_size = (buffer[8] << 24) | (buffer[9] << 16) |
- (buffer[10] << 8) | buffer[11];
- lba = (((u64)buffer[0] << 56) | ((u64)buffer[1] << 48) |
- ((u64)buffer[2] << 40) | ((u64)buffer[3] << 32) |
- ((u64)buffer[4] << 24) | ((u64)buffer[5] << 16) |
- ((u64)buffer[6] << 8) | (u64)buffer[7]);
+ sector_size = get_unaligned_be32(&buffer[8]);
+ lba = get_unaligned_be64(&buffer[0]);
sd_read_protection_type(sdkp, buffer);
@@ -1400,10 +1397,8 @@ static int read_capacity_10(struct scsi_disk *sdkp, struct scsi_device *sdp,
return -EINVAL;
}
- sector_size = (buffer[4] << 24) | (buffer[5] << 16) |
- (buffer[6] << 8) | buffer[7];
- lba = (buffer[0] << 24) | (buffer[1] << 16) |
- (buffer[2] << 8) | buffer[3];
+ sector_size = get_unaligned_be32(&buffer[4]);
+ lba = get_unaligned_be32(&buffer[0]);
if ((sizeof(sdkp->capacity) == 4) && (lba == 0xffffffff)) {
sd_printk(KERN_ERR, sdkp, "Too big for this kernel. Use a "
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH v2] fix sign extension with 1.5TB usb-storage LBD=y
2009-04-21 22:00 ` [PATCH v2] " Dave Hansen
2009-04-21 23:03 ` Matthew Wilcox
@ 2009-04-22 7:32 ` Boaz Harrosh
2009-04-22 11:09 ` Matthew Wilcox
1 sibling, 1 reply; 12+ messages in thread
From: Boaz Harrosh @ 2009-04-22 7:32 UTC (permalink / raw)
To: Dave Hansen
Cc: Matthew Wilcox, linux-kernel, mdharm-usb, linux-usb, usb-storage,
James Bottomley, linux-scsi, viro
On 04/22/2009 01:00 AM, Dave Hansen wrote:
> Here's a patch implementing Al's suggestion. Not quite as trivial as
> Matthew's, but even nicer aesthetically. (Description stolen from
> Matthew's patch). I have verified that this fixes my issue.
>
> --
>
> Shifting an unsigned char implicitly casts it to a signed int. This
> caused 'lba' to sign-extend and Linux would then try READ CAPACITY 16
> which was not supported by at least one drive. Using the
> get_unaligned_be*() helpers keeps us from having to worry about how the
> extension might occur.
>
> Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
>
> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
> index 3fcb64b..c50142b 100644
> --- a/drivers/scsi/sd.c
> +++ b/drivers/scsi/sd.c
> @@ -50,6 +50,7 @@
> #include <linux/string_helpers.h>
> #include <linux/async.h>
> #include <asm/uaccess.h>
> +#include <asm/unaligned.h>
>
> #include <scsi/scsi.h>
> #include <scsi/scsi_cmnd.h>
> @@ -1344,12 +1345,8 @@ static int read_capacity_16(struct scsi_disk *sdkp, struct scsi_device *sdp,
> return -EINVAL;
> }
>
> - sector_size = (buffer[8] << 24) | (buffer[9] << 16) |
> - (buffer[10] << 8) | buffer[11];
> - lba = (((u64)buffer[0] << 56) | ((u64)buffer[1] << 48) |
> - ((u64)buffer[2] << 40) | ((u64)buffer[3] << 32) |
> - ((u64)buffer[4] << 24) | ((u64)buffer[5] << 16) |
> - ((u64)buffer[6] << 8) | (u64)buffer[7]);
> + sector_size = get_unaligned_be32(&buffer[8]);
> + lba = get_unaligned_be64(&buffer[0]);
These are actually aligned access it might be worth sacrificing a cast
to be32/64 for sake of speed.
>
> sd_read_protection_type(sdkp, buffer);
>
> @@ -1400,10 +1397,8 @@ static int read_capacity_10(struct scsi_disk *sdkp, struct scsi_device *sdp,
> return -EINVAL;
> }
>
> - sector_size = (buffer[4] << 24) | (buffer[5] << 16) |
> - (buffer[6] << 8) | buffer[7];
> - lba = (buffer[0] << 24) | (buffer[1] << 16) |
> - (buffer[2] << 8) | buffer[3];
> + sector_size = get_unaligned_be32(&buffer[4]);
> + lba = get_unaligned_be32(&buffer[0]);
>
Here too, both are actually aligned.
> if ((sizeof(sdkp->capacity) == 4) && (lba == 0xffffffff)) {
> sd_printk(KERN_ERR, sdkp, "Too big for this kernel. Use a "
>
>
> -- Dave
>
I'm not 100% convinced just a thought.
Boaz
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH v2] fix sign extension with 1.5TB usb-storage LBD=y
2009-04-22 7:32 ` Boaz Harrosh
@ 2009-04-22 11:09 ` Matthew Wilcox
2009-04-22 11:27 ` Boaz Harrosh
0 siblings, 1 reply; 12+ messages in thread
From: Matthew Wilcox @ 2009-04-22 11:09 UTC (permalink / raw)
To: Boaz Harrosh
Cc: Dave Hansen, linux-kernel, mdharm-usb, linux-usb, usb-storage,
James Bottomley, linux-scsi, viro
On Wed, Apr 22, 2009 at 10:32:59AM +0300, Boaz Harrosh wrote:
> These are actually aligned access it might be worth sacrificing a cast
> to be32/64 for sake of speed.
"for sake of speed"? How often do you think we ask a device how large
it is? How much overhead do you think is incurred by the unaligned code
if the data happens to be aligned?
--
Matthew Wilcox Intel Open Source Technology Centre
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours. We can't possibly take such
a retrograde step."
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2] fix sign extension with 1.5TB usb-storage LBD=y
2009-04-22 11:09 ` Matthew Wilcox
@ 2009-04-22 11:27 ` Boaz Harrosh
0 siblings, 0 replies; 12+ messages in thread
From: Boaz Harrosh @ 2009-04-22 11:27 UTC (permalink / raw)
To: Matthew Wilcox
Cc: Dave Hansen, linux-kernel, mdharm-usb, linux-usb, usb-storage,
James Bottomley, linux-scsi, viro
On 04/22/2009 02:09 PM, Matthew Wilcox wrote:
> On Wed, Apr 22, 2009 at 10:32:59AM +0300, Boaz Harrosh wrote:
>> These are actually aligned access it might be worth sacrificing a cast
>> to be32/64 for sake of speed.
>
> "for sake of speed"? How often do you think we ask a device how large
> it is?
OK, that was the wrong choice of words, on my part. I meant for sake of
"nobleness". I calculated as a programmer that these are aligned do I make
the extra effort of stating that in code, or I get lazy because it does
not matter?
> How much overhead do you think is incurred by the unaligned code
> if the data happens to be aligned?
>
Well for BE systems we are already order of magnitude faster by just
using the accessors, so I guess we are already well in the "plus" ;)
This is such a small matter, sorry to bother you about it.
Just that it's a programming style I'm constantly debating with myself,
feel free to ignore it.
The patch looks very good to me as it is.
Thanks
Boaz
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC][PATCH] fix sign extension with 1.5TB usb-storage LBD=y
2009-04-21 21:18 ` Matthew Wilcox
2009-04-21 21:29 ` Randy Dunlap
2009-04-21 22:00 ` [PATCH v2] " Dave Hansen
@ 2009-04-21 22:49 ` Dave Hansen
2 siblings, 0 replies; 12+ messages in thread
From: Dave Hansen @ 2009-04-21 22:49 UTC (permalink / raw)
To: Matthew Wilcox
Cc: linux-kernel, mdharm-usb, linux-usb, usb-storage, James Bottomley,
linux-scsi
On Tue, 2009-04-21 at 15:18 -0600, Matthew Wilcox wrote:
>
> this certainly fixes your problem. I'd prefer this patch instead, just
> because I find the cast unaesthetic ...
>
> ----
>
> Fix READ CAPACITY 10 with drives over 1TB
>
> Shifting an unsigned char implicitly casts it to a signed int. This
> caused 'lba' to sign-extend and Linux would then try READ CAPACITY 16
> which was not supported by at least one drive. Making 'lba' an
> unsigned
> int ensures that sign extension will not occur.
>
> Signed-off-by: Matthew Wilcox <willy@linux.intel.com>
>
> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
> index 3fcb64b..c856b1b 100644
> --- a/drivers/scsi/sd.c
> +++ b/drivers/scsi/sd.c
> @@ -1373,7 +1373,7 @@ static int read_capacity_10(struct scsi_disk
> *sdkp, struct scsi_device *sdp,
> int sense_valid = 0;
> int the_result;
> int retries = 3;
> - sector_t lba;
> + unsigned lba;
> unsigned sector_size;
>
> do {
> @@ -1413,7 +1413,7 @@ static int read_capacity_10(struct scsi_disk
> *sdkp, struct scsi_device *sdp,
> return -EOVERFLOW;
> }
>
> - sdkp->capacity = lba + 1;
> + sdkp->capacity = (sector_t)lba + 1;
> return sector_size;
> }
That's fine with me too. This patch fixes my issue as well.
Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
-- Dave
^ permalink raw reply [flat|nested] 12+ messages in thread