Re: Linux software RAID assistance

[Phil Turmel <philip@turmel.org>]

On 02/16/2011 05:44 PM, Simon Mcnair wrote:
> I just went to bed and one last question popped in to my mind. Since
> there is a fair timezone gap I thought I'd be presumptuous and ask it
> in the hope I can turn it around a bit quicker in the morning.
> 
> My suspicion is that once I have 5 formatted 2tb drives I may be lucky
> to get 10x 1tb dd images on to it. Can I feed the dd process in to
> tar, bzip2, zip or something else which will give me enough space to
> fit the images on ?
> 
> Will I get more usable space from 5x2tb partitions or from 1xspanned
> volume ? (the thecus pretty much only allows you to create raid
> volumes so a jbod needs to be 5x1tb arrays or 1 spanned volume or
> stripe).

I'd use one spanned volume, and gzip.  I'd simultaneously generate an md5sum while streaming to your thecus.  A script like so:

#! /bin/bash
#
function usage() {
	printf "Usage:\n\t%s devname\n\n" "`basename \"$0\"`"
	printf "'devname' must be a relative path in /dev/ to the desired block device.\n"
	exit 1
}

# Verify the supplied name is a device
test -b "/dev/$1" || usage

# Convert path separators and spaces into dashes
outfile="`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"

# Create a side-stream for computing the MD5 of the data read
fifo=`mktemp -u`
mkfifo $fifo || exit
md5sum -b <$fifo >/mnt/thecus/$outfile.md5 &

# Read the device and compress it
dd if="/dev/$1" bs=1M | tee 2>$fifo | gzip >/mnt/thecus/$outfile.gz

# Wait for the background task to close
wait