blocksize limitations in scsi tape driver (st) when used with DLT1 tape drives?

blocksize limitations in scsi tape driver (st) when used with DLT1 tape drives?

[Friedrich Lobenstock]

Hi!

I'm having an odd problem here. I use SuSE Linux 8.0 with kernel
2.4.18-64GB-SMP, arkeia as backup software and a Tandberg
VS80 (DLT1) streamer.

After some trial and error I found out that I have to use a fixed block
size for DLT streamers (same applies to LTO ones). This document,
found at HP, tells you in details about which blocksize to use:
   http://www.hp.com/cposupport/information_storage/support_doc/lpg50167.html

Now I told arkeia to use a blocksize of 65536, 32768 and 16384 and it
could not write to the tape. All I got in the log was:

Sep  4 22:04:50 filesrv kernel: st: Unloaded.
Sep  4 22:08:40 filesrv kernel: st: Version 20020205, bufsize 32768, wrt 30720, max init. bufs 4, s/g segs 16
Sep  4 22:08:40 filesrv kernel: st0: Block limits 2 - 16777214 bytes.
Sep  4 22:08:41 filesrv kernel: st0: Incorrect block size.
Sep  4 22:08:45 filesrv kernel: st0: Write not multiple of tape block size.

Sep  4 23:20:06 filesrv kernel: st: Unloaded.
Sep  4 23:20:30 filesrv kernel: st: Version 20020205, bufsize 524288, wrt 522240, max init. bufs 16, s/g segs 64
Sep  4 23:20:30 filesrv kernel: st0: Block limits 2 - 16777214 bytes.
Sep  4 23:20:33 filesrv kernel: st0: Incorrect block size.
Sep  4 23:20:35 filesrv kernel: st0: Write not multiple of tape block size.

(As you might have notice in the second case I'm loading st with buffer_kbs=512,
max_buffers=16 and max_sg_segs=64 but that makes no difference compared to the
default values)


BUT now comes the cloo: at and below a blocksize of 8192 it was possible
to write to the tape. At 1024 the backup slowed down from 20 min to 4 hours,
but at 8192 the backup speed is normal (the later on with the st options
you can see above, but haven't tested if that makes a difference).

LTO drives in contrast to DLT drives are happy with a blocksize of 1024
and backup speed is normal at this blocksize.

Everytime when arkeia could not write to the tape I checked with mt to
display the blocksize and I verified that arkeia set it to the value I
told it to. Now with arkeia set to a blocksize of 8192:

filesrv:~ # mt -f /dev/st0 status
drive type = Generic SCSI-2 tape
drive status = 1073750016
sense key error = 0
residue count = 0
file number = 0
block number = 0
Tape block size 8192 bytes. Density code 0x40 (unknown).
Soft error count since last status=0
General status bits on (41010000):
  BOT ONLINE IM_REP_EN

I think there must be something wrong in the scsi tape driver(st).
Can someone shine some light on this?

PS: Reply-to set to linux-scsi.

-- 
MfG / Regards
Friedrich Lobenstock

Re: blocksize limitations in scsi tape driver (st) when used with DLT1 tape drives?

[Dag Nygren]

Hi,

I have successfully been using Legato Networker with a
blocksize of 32k with my DLT here.

But the way Legato wants to do it is to decide about the
blocksize itself.
This means that the driver should NOT decide on the BS, but
pass on anything written through it, meaning a blocksize setting
of 0 (or variable blocksize).

Perhaps Arkeia works the same way ?

Best Luck


-- 
Dag Nygren                               email: dag@newtech.fi
Oy Espoon NewTech Ab                     phone: +358 9 8024910
Träsktorpet 3                              fax: +358 9 8024916
02360 ESBO                              Mobile: +358 400 426312
FINLAND

Re: blocksize limitations in scsi tape driver (st) when used with DLT1 tape drives?

[Friedrich Lobenstock]

Dag Nygren wrote:
> I have successfully been using Legato Networker with a
> blocksize of 32k with my DLT here.
> 
> But the way Legato wants to do it is to decide about the
> blocksize itself.
> This means that the driver should NOT decide on the BS, but
> pass on anything written through it, meaning a blocksize setting
> of 0 (or variable blocksize).
> 
> Perhaps Arkeia works the same way ?

I used variable blocksize (=0) before but then after about 3.5 gigs stored
on the tape I get scsi-error and arkeia interprets this as end-of-tape.

Aug  6 02:13:50 filesrv kernel: st0: Error with sense data: Info fld=0x2000, Deferred st09:00: sns = f1  3
Aug  6 02:13:50 filesrv kernel: ASC=80 ASCQ= 1
Aug  6 02:13:50 filesrv kernel: Raw sense data:0xf1 0x00 0x03 0x00 0x00 0x20 0x00 0x16 0x00 0x00 0xe6 0xc2 0x80 0x01 0x00 0x00 0x00 0x00 0x83 0x00 0x37 0x00 0x00 0x00 0x21 0x00 0x90 0x7f 0x3a 0x00
Aug  6 02:13:50 filesrv kernel: klogd 1.4.1, ---------- state change ----------
Aug  6 02:13:50 filesrv kernel: Inspecting /boot/System.map-2.4.18-64GB-SMP
Aug  6 02:13:50 filesrv kernel: Loaded 13537 symbols from /boot/System.map-2.4.18-64GB-SMP.
Aug  6 02:13:50 filesrv kernel: Symbols match kernel version 2.4.18.
Aug  6 02:13:50 filesrv kernel: Loaded 481 symbols from 13 modules.
Aug  6 02:13:50 filesrv kernel: st0: Error with sense data: Info fld=0x1, Current st09:00: sns = f0  3
Aug  6 02:13:50 filesrv kernel: ASC= c ASCQ= 0
Aug  6 02:13:50 filesrv kernel: Raw sense data:0xf0 0x00 0x03 0x00 0x00 0x00 0x01 0x16 0x00 0x00 0xe6 0xc2 0x0c 0x00 0x00 0x00 0x00 0x00 0x83 0x00 0x37 0x00 0x00 0x00 0x21 0x00 0x90 0x7f 0x3a 0x00

I had this problem with LTO drives too and here the arkeia faq tells one
to set fixed block size.

And when you look a the HP document referenced in my last mail:

    ISSUE: Block sizes of LESS THAN 64 KB for DLT1/DLT VS80 and 32 KB for
           all other DAT and DLT drives can drastically increase the backup/restore
           time and severely affect the performance of the drive.

SOLUTION: Most backup applications allow viewing and adjusting the block
           size used for a particular device. See below for advice on how
           to achieve this for CA ARCserve, Veritas Backup Exec and Tapeware.

(I capitalized "less than" to emphasis its occurance)

I you missed the link here it is again:
   http://www.hp.com/cposupport/information_storage/support_doc/lpg50167.html

Did you check with mt after Legato Networker did a backup which blocksize
it set?

-- 
MfG / Regards
Friedrich Lobenstock

Re: blocksize limitations in scsi tape driver (st) when used with DLT1 tape drives?

[Kai Makisara]

On Thu, 5 Sep 2002, Friedrich Lobenstock wrote:

> Dag Nygren wrote:
> > I have successfully been using Legato Networker with a
> > blocksize of 32k with my DLT here.
> >
> > But the way Legato wants to do it is to decide about the
> > blocksize itself.
> > This means that the driver should NOT decide on the BS, but
> > pass on anything written through it, meaning a blocksize setting
> > of 0 (or variable blocksize).
> >
> > Perhaps Arkeia works the same way ?
>
> I used variable blocksize (=0) before but then after about 3.5 gigs stored
> on the tape I get scsi-error and arkeia interprets this as end-of-tape.
>
> Aug  6 02:13:50 filesrv kernel: st0: Error with sense data: Info fld=0x2000, Deferred st09:00: sns = f1  3
> Aug  6 02:13:50 filesrv kernel: ASC=80 ASCQ= 1

The sense code 3 is MEDIUM ERROR. The ASC/ASCQ 80 01 is vendor specific.
In other words, you have bad tape (or the drive thinks that it is bad).
This is returned as an error in writing data to tape. Most applications
(hopefully) interpret this as end of tape.

	Kai

Re: blocksize limitations in scsi tape driver (st) when used with DLT1 tape drives?

[Dag Nygren]

> Dag Nygren wrote:

> I used variable blocksize (=0) before but then after about 3.5 gigs stored
> on the tape I get scsi-error and arkeia interprets this as end-of-tape.
> 
> Aug  6 02:13:50 filesrv kernel: st0: Error with sense data: Info fld=0x2000, Deferred st09:00: sns = f1  3
> Aug  6 02:13:50 filesrv kernel: ASC=80 ASCQ= 1
> Aug  6 02:13:50 filesrv kernel: Raw sense data:0xf1 0x00 0x03 0x00 0x00 0x20 0x00 0x16 0x00 0x00 0xe6 0xc2 0x80 0x01 0x00 0x00 0x00 0x00 0x83 0x00 0x37 0x00 0x00 0x00 0x21 0x00 0x90 0x7f 0x3a 0x00
> Aug  6 02:13:50 filesrv kernel: klogd 1.4.1, ---------- state change ----------
> Aug  6 02:13:50 filesrv kernel: Inspecting /boot/System.map-2.4.18-64GB-SMP
> Aug  6 02:13:50 filesrv kernel: Loaded 13537 symbols from /boot/System.map-2.4.18-64GB-SMP.
> Aug  6 02:13:50 filesrv kernel: Symbols match kernel version 2.4.18.
> Aug  6 02:13:50 filesrv kernel: Loaded 481 symbols from 13 modules.
> Aug  6 02:13:50 filesrv kernel: st0: Error with sense data: Info fld=0x1, Current st09:00: sns = f0  3
> Aug  6 02:13:50 filesrv kernel: ASC= c ASCQ= 0
> Aug  6 02:13:50 filesrv kernel: Raw sense data:0xf0 0x00 0x03 0x00 0x00 0x00 0x01 0x16 0x00 0x00 0xe6 0xc2 0x0c 0x00 0x00 0x00 0x00 0x00 0x83 0x00 0x37 0x00 0x00 0x00 0x21 0x00 0x90 0x7f 0x3a 0x00

Have you checked what this sense data indicates ?
Do you have the SCSI manual for your DLT ?
Sense Key 0x03 and Additional Sense Key=0xe6

Sense key= 0x03 means MEDIUM_ERROR
ASK=0xe6 is Vendor Specific but could tell you more about the 
medium error

> I had this problem with LTO drives too and here the arkeia faq tells one
> to set fixed block size.

This sounds very strange........

> And when you look a the HP document referenced in my last mail:
> 
>     ISSUE: Block sizes of LESS THAN 64 KB for DLT1/DLT VS80 and 32 KB for
>            all other DAT and DLT drives can drastically increase the backup/restore
>            time and severely affect the performance of the drive.
> 
> SOLUTION: Most backup applications allow viewing and adjusting the block
>            size used for a particular device. See below for advice on how
>            to achieve this for CA ARCserve, Veritas Backup Exec and Tapeware.

And they are certainly right on that, I recently made a study on blocksizes in
Linux-tapes for a customer and the performance was significantly better up to
16 k blocks, after this the perf. increase was in the regions of 2-3 %.

> 
> (I capitalized "less than" to emphasis its occurance)
> 
> I you missed the link here it is again:
>    http://www.hp.com/cposupport/information_storage/support_doc/lpg50167.html
> 
> Did you check with mt after Legato Networker did a backup which blocksize
> it set?

It will not be set to anything else than 0. Using 0 just tells the driver that
it shall write all the data given in ONE write() call to the driver in one 
block
to the device. Thus the application (in this case Legato) can determine which
size of blocks to use.
When using fixed blocks, the driver will split the write into that big chunks 
and
write those onto the tape one by one.

> 
> -- 
> MfG / Regards
> Friedrich Lobenstock
> 
> 

Re: blocksize limitations in scsi tape driver (st) when used with DLT1 tape drives?

[Kai Makisara]

On Thu, 5 Sep 2002, Friedrich Lobenstock wrote:

> Hi!
>
> I'm having an odd problem here. I use SuSE Linux 8.0 with kernel
> 2.4.18-64GB-SMP, arkeia as backup software and a Tandberg
> VS80 (DLT1) streamer.
>
> After some trial and error I found out that I have to use a fixed block
> size for DLT streamers (same applies to LTO ones). This document,
> found at HP, tells you in details about which blocksize to use:
>    http://www.hp.com/cposupport/information_storage/support_doc/lpg50167.html
>
> Now I told arkeia to use a blocksize of 65536, 32768 and 16384 and it
> could not write to the tape. All I got in the log was:
>
> Sep  4 22:04:50 filesrv kernel: st: Unloaded.
> Sep  4 22:08:40 filesrv kernel: st: Version 20020205, bufsize 32768, wrt 30720, max init. bufs 4, s/g segs 16
> Sep  4 22:08:40 filesrv kernel: st0: Block limits 2 - 16777214 bytes.
> Sep  4 22:08:41 filesrv kernel: st0: Incorrect block size.
> Sep  4 22:08:45 filesrv kernel: st0: Write not multiple of tape block size.
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The write() byte count from Arkeia is not a multiple of the tape block
size. If you use fixed block size, you must take care that this constraint
is not violated. When you set the block size to 8192 or something that
multiplies to 8192, this constraint is obeyed. If Arkeia sets the block
size, I don't know why it does not obey the constraint. Seems to be an
application problem.

In most cases, it is best to use variable block mode
(i.e., mt setblk 0) and let the application to decide the tape block size.
In order to have good efficiency, you should make sure that the
application uses large enough byte counts in its write() calls, i.e., set
the Arkeia block size to 65536.

	Kai

Re: blocksize limitations in scsi tape driver (st) when used with DLT1 tape drives?

[Friedrich Lobenstock]

Kai Makisara wrote:
> On Thu, 5 Sep 2002, Friedrich Lobenstock wrote:
> 
> 
>>Hi!
>>
>>I'm having an odd problem here. I use SuSE Linux 8.0 with kernel
>>2.4.18-64GB-SMP, arkeia as backup software and a Tandberg
>>VS80 (DLT1) streamer.
>>
>>After some trial and error I found out that I have to use a fixed block
>>size for DLT streamers (same applies to LTO ones). This document,
>>found at HP, tells you in details about which blocksize to use:
>>   http://www.hp.com/cposupport/information_storage/support_doc/lpg50167.html
>>
>>Now I told arkeia to use a blocksize of 65536, 32768 and 16384 and it
>>could not write to the tape. All I got in the log was:
>>
>>Sep  4 22:04:50 filesrv kernel: st: Unloaded.
>>Sep  4 22:08:40 filesrv kernel: st: Version 20020205, bufsize 32768, wrt 30720, max init. bufs 4, s/g segs 16
>>Sep  4 22:08:40 filesrv kernel: st0: Block limits 2 - 16777214 bytes.
>>Sep  4 22:08:41 filesrv kernel: st0: Incorrect block size.
>>Sep  4 22:08:45 filesrv kernel: st0: Write not multiple of tape block size.
> 
>                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> The write() byte count from Arkeia is not a multiple of the tape block
> size. If you use fixed block size, you must take care that this constraint
> is not violated. When you set the block size to 8192 or something that
> multiplies to 8192, this constraint is obeyed. If Arkeia sets the block
> size, I don't know why it does not obey the constraint. Seems to be an
> application problem.
> 
> In most cases, it is best to use variable block mode
> (i.e., mt setblk 0) and let the application to decide the tape block size.
> In order to have good efficiency, you should make sure that the
> application uses large enough byte counts in its write() calls, i.e., set
> the Arkeia block size to 65536.

Hmmm...I whish I could, but LTO and as it seems DLT drives too do not
cope well with variable block sizes.

All blocksizes greater than 8192 do make troubles. At and below this
value all is fine.


-- 
MfG / Regards
Friedrich Lobenstock

Re: blocksize limitations in scsi tape driver (st) when used with DLT1 tape drives?

[Willem Riede]

On 2002.09.05 18:15 Friedrich Lobenstock wrote:
> Kai Makisara wrote:
> > On Thu, 5 Sep 2002, Friedrich Lobenstock wrote:
> > 
> >>Now I told arkeia to use a blocksize of 65536, 32768 and 16384 and it
> >>could not write to the tape. All I got in the log was:
> >>
> >>Sep  4 22:08:41 filesrv kernel: st0: Incorrect block size.
> >>Sep  4 22:08:45 filesrv kernel: st0: Write not multiple of tape block size.
> > 
> >                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > The write() byte count from Arkeia is not a multiple of the tape block
> > size. If you use fixed block size, you must take care that this constraint
> > is not violated. When you set the block size to 8192 or something that
> > multiplies to 8192, this constraint is obeyed. If Arkeia sets the block
> > size, I don't know why it does not obey the constraint. Seems to be an
> > application problem.
> > 
> > In most cases, it is best to use variable block mode
> > (i.e., mt setblk 0) and let the application to decide the tape block size.
> > In order to have good efficiency, you should make sure that the
> > application uses large enough byte counts in its write() calls, i.e., set
> > the Arkeia block size to 65536.
> 
> Hmmm...I whish I could, but LTO and as it seems DLT drives too do not
> cope well with variable block sizes.
> 
> All blocksizes greater than 8192 do make troubles. At and below this
> value all is fine.

I had a run in with Arkeia some time ago when used with osst tape devices,
and if I remember correctly, it is best to not tell Arkeia anything about
the block size to use. It will use multiples of 8192 in that case. Given
that you say the DLT drive is happy with 8192 block size that should work.

Regards, Willem Riede.