SCSI tape block size

SCSI tape block size

[Guennadi Liakhovetski]

Hi all

I've got a problem report and a patch from John Adams, he says tmscsim 
driver under 2.6.17 and on doesn't allow him to read tapes with blocksize 
of 1MB. He fixes this with the below patch.

On Sun, 24 Sep 2006, John Adams wrote:

> --- drivers/scsi/tmscsim.c.bak	2006-09-24 12:55:08.000000000 -0400
> +++ drivers/scsi/tmscsim.c	2006-09-24 12:55:56.000000000 -0400
> @@ -2300,7 +2300,8 @@
>  	.this_id		= 7,
>  	.sg_tablesize		= SG_ALL,
>  	.cmd_per_lun		= 1,
> -	.use_clustering		= DISABLE_CLUSTERING,
> +	.use_clustering		= ENABLE_CLUSTERING,
> +	.max_sectors            = 2048,
>  };
>  
>  /***********************************************************************

It looks like

1) use_clustering doesn't directly affect maximum block size, so, he 
doesn't really need it, although, it might make sense for tmscsim to 
improve performance.

2) max_sectors is indeed what he needs, but it looks strange that now it 
has to be set by the low-level driver... For block devices you can set it 
at run time with /sys/block/sda/queue/max_sectors_kb, right? But what 
about tapes?

Sorry, didn't find too much documentation about it.

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: SCSI tape block size

[Michael Reed]

Guennadi Liakhovetski wrote:
> Hi all
> 
> I've got a problem report and a patch from John Adams, he says tmscsim 
> driver under 2.6.17 and on doesn't allow him to read tapes with blocksize 
> of 1MB. He fixes this with the below patch.
> 
> On Sun, 24 Sep 2006, John Adams wrote:
> 
>> --- drivers/scsi/tmscsim.c.bak	2006-09-24 12:55:08.000000000 -0400
>> +++ drivers/scsi/tmscsim.c	2006-09-24 12:55:56.000000000 -0400
>> @@ -2300,7 +2300,8 @@
>>  	.this_id		= 7,
>>  	.sg_tablesize		= SG_ALL,
>>  	.cmd_per_lun		= 1,
>> -	.use_clustering		= DISABLE_CLUSTERING,
>> +	.use_clustering		= ENABLE_CLUSTERING,
>> +	.max_sectors            = 2048,
>>  };
>>  
>>  /***********************************************************************
> 
> It looks like
> 
> 1) use_clustering doesn't directly affect maximum block size, so, he 
> doesn't really need it, although, it might make sense for tmscsim to 
> improve performance.
> 
> 2) max_sectors is indeed what he needs, but it looks strange that now it 
> has to be set by the low-level driver... For block devices you can set it 
> at run time with /sys/block/sda/queue/max_sectors_kb, right? But what 
> about tapes?

Thank you for posting this.  I was wondering why with 2.6.16 the maximum
tape block I could access via by BusLogic adapter dropped to 64k.  Now
I know what to change.

Does anyone know what the appropriate max sectors limit is for BusLogic?

Thanks,
 Mike

> 
> Sorry, didn't find too much documentation about it.
> 
> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: SCSI tape block size

[Kai Makisara]

On Wed, 27 Sep 2006, Guennadi Liakhovetski wrote:

> Hi all
> 
> I've got a problem report and a patch from John Adams, he says tmscsim 
> driver under 2.6.17 and on doesn't allow him to read tapes with blocksize 
> of 1MB. He fixes this with the below patch.
> 
> On Sun, 24 Sep 2006, John Adams wrote:
> 
> > --- drivers/scsi/tmscsim.c.bak	2006-09-24 12:55:08.000000000 -0400
> > +++ drivers/scsi/tmscsim.c	2006-09-24 12:55:56.000000000 -0400
> > @@ -2300,7 +2300,8 @@
> >  	.this_id		= 7,
> >  	.sg_tablesize		= SG_ALL,
> >  	.cmd_per_lun		= 1,
> > -	.use_clustering		= DISABLE_CLUSTERING,
> > +	.use_clustering		= ENABLE_CLUSTERING,
> > +	.max_sectors            = 2048,
> >  };
> >  
> >  /***********************************************************************
> 
> It looks like
> 
> 1) use_clustering doesn't directly affect maximum block size, so, he 
> doesn't really need it, although, it might make sense for tmscsim to 
> improve performance.
> 
> 2) max_sectors is indeed what he needs, but it looks strange that now it 
> has to be set by the low-level driver... For block devices you can set it 
> at run time with /sys/block/sda/queue/max_sectors_kb, right? But what 
> about tapes?
> 
Both 1 and 2 are needed for 1 MB block size. max_sectors_kb for tapes is 
not visible in /sys.

> Sorry, didn't find too much documentation about it.
> 
The limits come from the block subsystem that all scsi devices use 
nowadays. It is a rather long story but can be found from linux-scsi 
archives, e.g.,
http://marc.theaimsgroup.com/?l=linux-scsi&m=114147170831847&w=2

-- 
Kai

use_clustering again (was Re: SCSI tape block size)

[Guennadi Liakhovetski]

On Thu, 28 Sep 2006, Kai Makisara wrote:

> The limits come from the block subsystem that all scsi devices use 
> nowadays. It is a rather long story but can be found from linux-scsi 
> archives, e.g.,
> http://marc.theaimsgroup.com/?l=linux-scsi&m=114147170831847&w=2

Yes, already found the 2 later threads - one on lkml and one on scsi. I 
did look at them briefly back then, but, probably, too briefly.

One of the settings - enable_clustering - seems to be pretty critical and 
can only be verified to be safe by intensive testing of various chips / 
cards. So, how about making it a run-time parameter? A module parameter is 
the easiest, but then each lld would have to do the same thing more or 
less. A sysfs attribute - but where? It is a queue attribute, but st/sg 
don't expose their queues in /sys...

Maybe under /sys/class/scsi_host?

Similar for max_sectors?

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: use_clustering again (was Re: SCSI tape block size)

[Mike Christie]

Guennadi Liakhovetski wrote:
> On Thu, 28 Sep 2006, Kai Makisara wrote:
> 
>> The limits come from the block subsystem that all scsi devices use 
>> nowadays. It is a rather long story but can be found from linux-scsi 
>> archives, e.g.,
>> http://marc.theaimsgroup.com/?l=linux-scsi&m=114147170831847&w=2
> 
> Yes, already found the 2 later threads - one on lkml and one on scsi. I 
> did look at them briefly back then, but, probably, too briefly.
> 
> One of the settings - enable_clustering 

If you are just trying to get to 1MB, then instead of enabling
clustering we could also increase the defaul SCSI_MAX_PHYS_SEGMENTS. It
is 128 now, but there is a compile option to set it it up to 256.

Well, actually I think that gets a little short because SG_ALL is only
255. Why is SG_ALL 255, but you can set SCSI_MAX_PHYS_SEGMENTS to 256?

Re: use_clustering again

[Martin K. Petersen]

>>>>> "Mike" == Mike Christie <michaelc@cs.wisc.edu> writes:

Mike> If you are just trying to get to 1MB, then instead of enabling
Mike> clustering we could also increase the defaul
Mike> SCSI_MAX_PHYS_SEGMENTS. It is 128 now, but there is a compile
Mike> option to set it it up to 256.

I have essentially been having the same problem doing big direct I/Os
using sg.  I ended up bumping sg_tablesize to 1024 but only because I
knew the controllers used for this application support that many
scatter-gather segments.

As you mention, SG_ALL is 255 and that used to be the max value (quite
useless, just short of 1MB).  In recent kernels it's been clamped to
128 thanks to getting intimate with the block/request layer.

These short lists are quite useless for contemporary tape hardware
that need a block size of 1MB at the minimum and preferably 2MB or
more...

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: SCSI tape block size

[Guennadi Liakhovetski]

On Thu, 28 Sep 2006, Kai Makisara wrote:

> Both 1 and 2 are needed for 1 MB block size. max_sectors_kb for tapes is 
> not visible in /sys.

Kai, as st maintainer, you surely know - on my tape HP C1533A I cannot 
seem to set up any block size other than 1 byte... Is it __really__ 1 byte 
or is it just some fixed block size that the drive doesn't report and 
cannot change? And it means I cannot test setting of different block 
sizes? I have a (unconnected ATM) slightly newer C1555D, wonder if that 
could use different block sizes?...

1 byte I see in "mt status" and setting anything else with "mt setblk" 
causes "st0: Incorrect block size."

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: SCSI tape block size

[Kai Makisara]

On Sat, 30 Sep 2006, Guennadi Liakhovetski wrote:

> On Thu, 28 Sep 2006, Kai Makisara wrote:
> 
> > Both 1 and 2 are needed for 1 MB block size. max_sectors_kb for tapes is 
> > not visible in /sys.
> 
> Kai, as st maintainer, you surely know - on my tape HP C1533A I cannot 
> seem to set up any block size other than 1 byte... Is it __really__ 1 byte 
> or is it just some fixed block size that the drive doesn't report and 
> cannot change? And it means I cannot test setting of different block 
> sizes? I have a (unconnected ATM) slightly newer C1555D, wonder if that 
> could use different block sizes?...
> 
This is interesting. I also have HP C1533A (firmware 9608) and I have not 
had any problems with setting the block size. Dmesg shows that the drive 
supports blocks up to 1<<24 - 1 bytes:
st0: Block limits 1 - 16777215 bytes.

I just tested with kernel 2.6.18 that setting block size works in my 
system.

> 1 byte I see in "mt status" and setting anything else with "mt setblk" 
> causes "st0: Incorrect block size."
> 
This message comes from reading in fixed block mode when the drive tries 
to read a block and the size of the block does not match the block size 
set to the drive. If you have written the tape with block size 1, you 
can't read it with any other block size in fixed block mode.

If you are testing the limits of the LLD, it may be easier to use variable 
block mode. What matters is the size of the SCSI i/o and in this case you 
know it exactly. In fixed block mode the i/o size may be several blocks 
unless you are careful.

-- 
Kai

Re: SCSI tape block size

[Guennadi Liakhovetski]

On Sun, 1 Oct 2006, Kai Makisara wrote:

> On Sat, 30 Sep 2006, Guennadi Liakhovetski wrote:
> 
> > Kai, as st maintainer, you surely know - on my tape HP C1533A I cannot 
> > seem to set up any block size other than 1 byte... Is it __really__ 1 byte 
> > or is it just some fixed block size that the drive doesn't report and 
> > cannot change? And it means I cannot test setting of different block 
> > sizes? I have a (unconnected ATM) slightly newer C1555D, wonder if that 
> > could use different block sizes?...
> > 
> This is interesting. I also have HP C1533A (firmware 9608) and I have not 
> had any problems with setting the block size. Dmesg shows that the drive 
> supports blocks up to 1<<24 - 1 bytes:
> st0: Block limits 1 - 16777215 bytes.
> 
> I just tested with kernel 2.6.18 that setting block size works in my 
> system.

I must have done something wrong yesterday. Sorry. It does work indeed, 
tested with

mt -f /dev/st0 setblk `expr $bs \* 1024`
time tar cf /dev/st0 -b `expr $bs \* 2` *jp*
mt -f /dev/st0 status

I can set block size as high as 512k (.max_sectors defaulting to 1024) 
independent of .use_clustering. Only setting .max_sectors higher lets one 
use larger blocks.

Thanks
Guennadi
---
Guennadi Liakhovetski