Re: [dm-crypt] Encrypted Raid1 or Raid 1 of encrypted devices?

[Laurence Darby <ldarby@tuffmail.com>]

Hi Arno, All,

Thanks very much for your comments, I'll definitely go with encrypting
the RAID.

Arno Wagner wrote:

> On Tue, Jul 12, 2011 at 02:10:03PM +0200, Milan Broz wrote:
> > On 07/12/2011 01:32 PM, Jorge F?bregas wrote:
> > > That's an interesting question:  encrypted raid1 or raid1 of
> > > encrypted disks? That also could be phrased as "dm-crypt on top
> > > of dm-raid" or "dm-raid on top of dm-crypt"?
> > > 
> > > I must admit  I would have never thought about a "raid1 of
> > > encrypted disks" (seems awkward) but apparently it works.  I'm
> > > new here (and to disk encryption at all) but here are my two
> > > cents:
> > 
> > Technically both works.
> 
> Technically, RAID and encryption are on the same layer of the
> storage system and you can do arbitrary conbinations.
>  
> > > # Performance
> > > I guess from the point of view of performance (CPU-wise) , an
> > > "encrypted RAID1" would be better as you would be only encrypting
> > > once and DM-raid will take care of copying those bits as they are
> > > to the 2nd disk.  I suggest you do some tests (copying large
> > > amount of data to the encrypted disk) and measure it.
> > 
> > This depends on kernel version and if the system is SMP/multi-cpu.
> > For <2.6.38 you may get better performance for raid over crypt,
> > for newer kernel it will be different.
> > (I am not saying better because there are still performance issues
> > with crypt over MD Raid. Depends on io pattern and if IO are issued
> > from different cpus or not. Like dd can be slower but threaded fs
> > test can have much more better performance.)
> > 

Yes, I noticed this - a single dd process has to wait for both the
encryption and then IO, so more processes are needed to fill the
pipeline.

There's not much of a performance difference anyway - I have a 2 core
machine, and found that both disks (loopback files) got encrypted at
the same time (top showed 2 kworker processes, each using 80% cpu, the
rest mostly waiting on IO.)


> > > # Management
> > > There's no doubt that an encrypted raid1 is much better (much less
> > > commands: you just need to format once, luksOpen once, luksClose
> > > once. one backup of the header)
> > 
> > yes, I would suggest crypt over MD always too.
> 
> Good point. And you ned to enter your passphrase twice, exposing 
> it more.
> 
> The main difference security-wise is that an attacker gets
> the same data-set encrypted with two different keys when
> RAIDing encrypted devices. That can be a concern. 
> 
> 


Thank you for pointing that out - it's exactly the kind of thing I
wasn't aware of but should be.


> > > # Reliability
> > > I'm not sure about this part.  Let's see what others have to say
> > > regarding this.
> > 
> > IMHO both solutions are similar here. Some errors are propagated,
> > hw failure (RAM, disk) would have similar effect.
> 
> Reliability with regard to data corryption really belongs into 
> the filesystem-layer and above and into the hysocal device layer.
> 
> RAID seems to be reliability, but in fact it is redundancy 
> only, and detection of errors (bit errors, unreadable sectors
> and dead disks) is done below (or for bit-errors possibly
> in the filesystem-layer with a checksumming FS). The RAID
> cannot detect errors, it can just react to errors reported by
> lower layers using the redundancy it provides. 
> One exception: It can do consistency checks, but it can only 
> do somethign about inconsistencioes if it is at least 3-way 
> redundant by voting, i.e. >= 3-way RAID1 or RAID6. And consistency
> checks are not a normal RAID operation, but rather an externally
> triggered RAID maintenance operation.
> 
> > RAID is not backup. You should backup LUKS header and data anyway.
> 


This RAID1 is my backup device, that I do daily rsync-snapshots to.
I'll admit, I'm not doing automated integrity checks yet, I know I
should be, but encryption is more fun :) 

I was thinking that with a RAID of encrypted devices, rather than the
encrypted RAID, that I wouldn't need a header backup - If I break one,
that mistake isn't immediately written to both drives, so I can just
reformat it, and rebuild the array.  Although, yes, just restoring a
header backup does sound a lot easier, and after what Arno said above,
it's not feasible anyway.

I'll reply to the other thread tomorrow, bedtime now.

Thanks,
Laurence