Re: [dm-crypt] LUKS in failover cluster

[Milan Broz <mbroz@redhat.com>]

On 10/08/2011 06:52 AM, Arno Wagner wrote:
>> Also, scalability is a requirement in my design, hence XFS. I was
>> thinking I needed to use multiple LUKS PVs in LVM to grow the
>> filesystem. But I would end up with multiple LUKS devices to keep track
>> of. I recently found out that LUKS can resize. 
> 
> LUKS _cannot_ resize. The thing is that LUKS does not care about
> device size. So if you enlarge a device/partition with an intact
> LUKS header, "cryptsetup luksOpen" will just map the larger 
> device.

yes, but crytsetup resize is designed that it it supports open
(active) LUKS containers. You do not need to activate/deactivate,
resize operation is enough.

> If you have multiple devices, you can "slave" the additional ones
> to a "master" device using something like "decrypt_derived". 
> This just takes the master key from the opened "master" container,
> runns it though a hash and uses this as key for the "slave"
> device. 
> 
>> Would it be better to
>> create one LUKS device on top of LVM? Then create a filesystem on that?
>> (Though that would affect resource dependencies.)
> 
> Sre you sure you need LVM at all?

LVM is very useful in such configurations, you can dynamically manage
LVs over encrypted storage.
Scripts for managing such stack is not complicated but of course
it is yet another layer to care of.

Because you mentioned cluster, I will just repeat

- if you manage it in HA LVM style (the device
stack is always active exclusively only on one node) it will work.
(You just need to create own resource for cluster suite - add LUKS
to the stack. So cluster suite will handle exclusivity and failover itself.)

For using LUKS in cluster for shared storage (visible to all nodes)
- if encryption (mapping) runs in server and plaintext device is exported
and shared to nodes (using iscsi + clvm or so) this will work

- if you want to run dm-crypt on every node (for the same device,
IOW export ciphertext device) this is not supported.

(I think that I said on IRC that with proper used cluster fs and
flush requests this may work - but it is not tested. The problem is of course
with internal crypto queues so without flushing these queues different
nodes will see different storage content. And moreover, there is no cluster
infrastructure like clvmd to manage such devices consistently.)

Milan