SCSI tape corruption problem

SCSI tape corruption problem

[lomarcan]

I've recently installed a SDT-9000 tape drive. Running kernel 2.4.x I've
noticed the following (critical) problem:

Apparently the data are corrupted on the way to (from?) tape. I'm sure the
DAT 
drive is good (worked good on NT, head clean, new cartridge). It doesn't
report
data errors. I've got bad CRC errors on tar (the gzip part, of course)

The drive is on an Adaptec 2904 controller, with a Yamaha CDRW on the same
bus.
I'm pretty sure it's terminated correctly. Another SCSI controller (2940)
is 
driving 2 hard drives. Underlying HW: Athlon 1GHz, on Asus board (VIA
chipset). 
It seems to happen frequently (tried four times with about 600MB of data,
three
times failed the restore :((. Tried all the 2.4.x kernel series (thru
2.4.3)

What can it be? (I'll try to compare the read data with the original...)

				-- Lorenzo Marcantonio

Re: SCSI tape corruption problem

[Mircea Damian]

This seems to happen on my system too but I have and IDE tape:

Apr  3 00:00:48 www kernel: ide-tape: hdb <-> ht0: Seagate STT8000A rev 5.02
Apr  3 00:00:48 www kernel: ide-tape: hdb <-> ht0: 600KBps, 14*26kB buffer, 5850kB pipeline, 80ms tDSC, DMA


I have managed to recover the tar archive by writing the data through a
faucet pipe on another machine.

So this seems to be a problem only when I write the data on the same IDE
interface.

On Wed, Apr 11, 2001 at 01:37:02PM +0200, lomarcan@tin.it wrote:
> I've recently installed a SDT-9000 tape drive. Running kernel 2.4.x I've
> noticed the following (critical) problem:
> 
> Apparently the data are corrupted on the way to (from?) tape. I'm sure the
> DAT 
> drive is good (worked good on NT, head clean, new cartridge). It doesn't
> report
> data errors. I've got bad CRC errors on tar (the gzip part, of course)
> 
> The drive is on an Adaptec 2904 controller, with a Yamaha CDRW on the same
> bus.
> I'm pretty sure it's terminated correctly. Another SCSI controller (2940)
> is 
> driving 2 hard drives. Underlying HW: Athlon 1GHz, on Asus board (VIA
> chipset). 
> It seems to happen frequently (tried four times with about 600MB of data,
> three
> times failed the restore :((. Tried all the 2.4.x kernel series (thru
> 2.4.3)
> 
> What can it be? (I'll try to compare the read data with the original...)
> 
> 				-- Lorenzo Marcantonio
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Mircea Damian
E-mails: dmircea@kappa.ro, dmircea@roedu.net
WebPage: http://taz.mania.k.ro/~dmircea/

Re: SCSI tape corruption problem

[lomarcan]

> This seems to happen on my system too but I have and IDE tape:

Seems uncurrelated. I'm trying this:

# cd ~archive; tar cvzf /dev/tapes/tape0 # using devfs on rewinding dev
(some 600MB of stuff...)

where ~archive is on a SCSI drive (ext2 fs on LVM volume if can help)

# tar tvzf /dev/tapes/tape0
... some stuff pass correctly...

then gzip give a CRC error (no tape errors in syslog)

tape0 is on scsi0 (2904), the disk is on scsi1 (2940UW). Seems that the
problem is writing, because I get the CRC error always at the same point

				-- Lorenzo Marcantonio

Re: SCSI tape corruption problem

[Marc SCHAEFER]

Now try this:

   cd ~archive
   mt -f /dev/tapes/tape0 rewind
   tar cvf - . | gzip -9 | dd of=/dev/tapes/tape0 bs=32k

and then:

   mt -f /dev/tapes/tape0 rewind
   dd if=/dev/tapes/tape0 bs=32k | gzip -d | tar --compare -v -f -

The above is the proper way to talk to a tape drive through gzip.

PS: if you pre-compress, it can be wise to suppress hardware compression
(mt -f /dev/nst0 datcompression 0) and maybe use a big buffer with
the buffer command.

Re: SCSI tape corruption problem

[Geert Uytterhoeven]

On Sat, 14 Apr 2001, Marc SCHAEFER wrote:
> Now try this:
> 
>    cd ~archive
>    mt -f /dev/tapes/tape0 rewind
>    tar cvf - . | gzip -9 | dd of=/dev/tapes/tape0 bs=32k
> 
> and then:
> 
>    mt -f /dev/tapes/tape0 rewind
>    dd if=/dev/tapes/tape0 bs=32k | gzip -d | tar --compare -v -f -
> 
> The above is the proper way to talk to a tape drive through gzip.

So you make gzip use blocks of 32 kB.

Do you also mean it is illegal to use blocksize 10 kB (default tar, no gzip)
with a tape drive??

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: SCSI tape corruption problem

[Marc SCHAEFER]

On Sat, 14 Apr 2001, Geert Uytterhoeven wrote:

> Do you also mean it is illegal to use blocksize 10 kB (default tar, no gzip)
> with a tape drive??

not at all. Infact, with the default settings of the st driver, any
multiple of 512 bytes is ok.  The additional dd step is just there to be
sure everything is fine. If gzip always outputs multiple of 512 bytes then
everything is fine. I do this as precaution, since on Solaris with an old
Exabyte tape if you didn't do the extra dd you had various bizarre
problems. Linux is much nicer, but ...

Now if you change st defaults to e.g. variable block mode, then it changes
a bit the equation: you need to read exactly the same size of block (e.g.
bs=32k) else you get an error.

Variable block mode is mt -f /dev/nst0 setblk 0.

Re: SCSI tape corruption problem

[Marc SCHAEFER]

In article <Pine.LNX.4.05.10104141940320.12788-100000@callisto.of.borg> you wrote:
> So you make gzip use blocks of 32 kB.

no, infact it really makes it using blocks of 4k (a PIPE_SIZE is 4k), so
it is really equivalent to bs=4k. dd doesn't re-join blocks when
they are smaller then bs=, unless you specify obs=32k.

I have been playing with buffer recently and mixed up both.
buffer is really a nice tool.

But as I said, as long as you don't play with non-fixed block size
you don't have problems.

sorry for mixup.

Re: SCSI tape corruption problem

[Lorenzo Marcantonio]

On Sat, 14 Apr 2001, Marc SCHAEFER wrote:

> Now try this:
>
>    cd ~archive
>    mt -f /dev/tapes/tape0 rewind
>    tar cvf - . | gzip -9 | dd of=/dev/tapes/tape0 bs=32k
>
> and then:
>
>    mt -f /dev/tapes/tape0 rewind
>    dd if=/dev/tapes/tape0 bs=32k | gzip -d | tar --compare -v -f -
>
> The above is the proper way to talk to a tape drive through gzip.

I see the blocking part... but in my second experiment I've used ONLY
dd to put a large file on tape...

... still, I've investigated on this because amverify gave me a ton of
crc errors... (I REALLY hope that amanda uses proper blocking :)

				-- Lorenzo Marcantonio

Re: SCSI tape corruption problem

[Marc SCHAEFER]

In article <Pine.LNX.4.31.0104142124390.1307-100000@eris.discordia.loc> you wrote:
> ... still, I've investigated on this because amverify gave me a ton of
> crc errors... (I REALLY hope that amanda uses proper blocking :)

yes, it does.

This really looks like a st problem, or kernel. Or host adapter, or :)

I have to try 2.4.x soon. Problem is for me 2.2.x is enough :)

Re: SCSI tape corruption problem

[Olaf Titz]

>    tar cvf - . | gzip -9 | dd of=/dev/tapes/tape0 bs=32k

I have a tool which compresses individual files in a tar archive
instead of the whole archive[1]. That one tries hard to write only in
standard 10kb blocks to emulate tar's behavior towards the drive.
I'll try in a few days (when I'm back from holidays) if I have this
error on my scsi tape too (bad) and if the compression program affects
it.

Olaf

[1] <URL:http://sites.inka.de/bigred/sw/tarmill-0.22.tar.gz>