The missing RAID level

[Hubert Tonneau <hubert.tonneau@fullpliant.org>]

Assuming that I want to do long term archiving (many many gigabytes of datas,
but tiny load) on disks, the cheapest and easiest solution nowdays seems to
connect large 300 GB IDE disks through USB 2 to a comodity PC.

Now the problem is how to best recover from some disks failure ?

For the production storage, I use software RAID 5 on internal disks, but
for huge external ones, it might no more be a good idea, because of two
reasons:
. having several disks failure is more likely
. in case of a catastrophy, I would like to be abble to recover at least some
  of the datas, so no RAID at all is better that RAID 5 since I have DVD
  backups and what I want to optimise is operator time

So, one very interesting possibility would be to have an extra RAID level that
would do the following:
assuming that you connect 5+1 partitions, then you get 5 md partitions, not a
single one, with the following properties:
. any read to mdX goes straight forward to reading the underlying partition.
. any write goes staight forward to writting the underlying partition, but also
  updates the parity on the extra partition.

So, at the expense of slow write capabilities, which is not a problem for long
term archiving, I get a system with very interesting properties not covered by
existing Linux software RAID levels:
. in case of one disk failure, I can plug a new one, then rebuild just as with
  classical RAID
. in case of more disk failures, I only loose part of the archives (so spend
  less operator time for recovery from DVD).
. all partitions can be read just through ignoring the RAID details, so it is
  possible to unplug any of disks and connect it sowehere else with no extra
  constrains
. adding or removing disks from the raidset is trivial: just rebuild the parity
  partition

On the 'use what's available instead of requesting new features' side, I'm also
interested with feedback from users using large (8 to 16 SATA disks) external
cheap (anything that raises the price for 8 disk from 8 x 350 euro to more than
16 x 360 euro is no solution since clustering is then the way to go) towers that
would make RAID 5 a resonable solution, and how they connect to the Linux kernel
(each disk seen individualy, RAID handled by the contoler, need for a driver
outside the stock kernel, etc)

Regards,
Hubert Tonneau