[dm-crypt] advice on encrypted snapshots

[dm-crypt] advice on encrypted snapshots

[Ross Boylan]

Does anyone have any advice about how to snapshot an encrypted volume so
that the snapshot won't leak information?

This is for a backup; I'm aware that the backup system involves
additional security challenges. I want to do a file-based, rather than
disk-image, backup.

Thanks.

Ross Boylan

Re: [dm-crypt] advice on encrypted snapshots

[Arno Wagner]

On Fri, Oct 30, 2009 at 01:05:54PM -0700, Ross Boylan wrote:
> Does anyone have any advice about how to snapshot an encrypted volume so
> that the snapshot won't leak information?
> 
> This is for a backup; I'm aware that the backup system involves
> additional security challenges. I want to do a file-based, rather than
> disk-image, backup.

Hmm. Use "tar -f" to do an archive file and encrypt that image
on-the-fly with pgp/GnuPG. This way you can write the backup 
directly to unencrypted space.

The command could look as follows:

  tar cf - <path_to_data> | gpg -e -r <key owner> -o backup.tar.gpg -

Compression by tar is a bit redundant, as PGP/GnuPG also compress, 
but YMMV. The "f -" part to tar causes it to write to STDOUT
and the trailing "-" causes gpg to read from STDIN. In that
case gpg needs the output file name via "-o".

The above requires you to have a key for <key owner> set up
and the security of the backup then depends on the security
of that key. Note that you do not need the passphrase for 
encryption, only later for decyption. If you wan to use a
passhprase for the backup instead (no public-key crypto
involved), use the -c option to gpg. You can supply the 
passphrase in several ways, see the --passphrase-<something>
options.

Arno 
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] advice on encrypted snapshots

[Heinz Diehl]

On 30.10.2009, Ross Boylan wrote: 

> Does anyone have any advice about how to snapshot an encrypted volume so
> that the snapshot won't leak information?

How about using rsync which copies the data to a LUKS/dmcrypt encrypted
external harddisk?

Re: [dm-crypt] advice on encrypted snapshots

[Luca Berra]

On Fri, Oct 30, 2009 at 01:05:54PM -0700, Ross Boylan wrote:
>Does anyone have any advice about how to snapshot an encrypted volume so
>that the snapshot won't leak information?
>
Do you mean linux-lvm snapshot or some storage based one?
In the first case I think the safest way is encrypting the PV.

L.

-- 
Luca Berra -- bluca@comedia.it
         Communication Media & Services S.r.l.
  /"\
  \ /     ASCII RIBBON CAMPAIGN
   X        AGAINST HTML MAIL
  / \

Re: [dm-crypt] advice on encrypted snapshots

[Ross Boylan]

On Sat, 2009-10-31 at 09:12 +0100, Luca Berra wrote:
> On Fri, Oct 30, 2009 at 01:05:54PM -0700, Ross Boylan wrote:
> >Does anyone have any advice about how to snapshot an encrypted volume so
> >that the snapshot won't leak information?
> >
> Do you mean linux-lvm snapshot 
Yes.
> or some storage based one?
I'm not sure what that means, but I don't want to rsync or tar.  The
backup is run from a central computer using something like bacula.

> In the first case I think the safest way is encrypting the PV.

I don't think I can.  Here's my setup:
V1E   encrypted volume, built on top of
V1R   raw volume, which is part of 
VGA   volume group, composed of
PVA   physical volume (which is actually software RAID).

My terminology may be non-standard: V1E is readable and looks like a
regular file system; V1R is the one that looks scrambled.  i.e.,
cyptsetup luksOpen /dev/mapper/VGA-V1R V1E.
One could say that V1R is encrypted.  Is there a conventional way to
refer to these items?  V1R is the "backing device" and V1E is ....?

So if I snapshot V1E I think I must use VGA (at any rate, I have no
other space), which exposes the readable version of my data.

Maybe I could snapshot V1R and use the same encryption key as for V1E to
make V2E?

Now that I think of it, I'm not even sure if LVM will snapshot the
product of dm-crypt (V1E).

Ross

Re: [dm-crypt] advice on encrypted snapshots

[Luca Berra]

On Sat, Oct 31, 2009 at 11:03:02AM -0700, Ross Boylan wrote:
>On Sat, 2009-10-31 at 09:12 +0100, Luca Berra wrote:
>> On Fri, Oct 30, 2009 at 01:05:54PM -0700, Ross Boylan wrote:
>> >Does anyone have any advice about how to snapshot an encrypted volume so
>> >that the snapshot won't leak information?
>> >
>> Do you mean linux-lvm snapshot 
>Yes.
>> or some storage based one?
>I'm not sure what that means, but I don't want to rsync or tar.  The
i meant a storage devices which presents disk space as one or more lun
to a host using either fibre-channel or iscasi or similar means, but
that's not your case.
>
>> In the first case I think the safest way is encrypting the PV.
>
>I don't think I can.  Here's my setup:
>V1E   encrypted volume, built on top of
>V1R   raw volume, which is part of 
>VGA   volume group, composed of
>PVA   physical volume (which is actually software RAID).
>
...
>
>So if I snapshot V1E I think I must use VGA (at any rate, I have no
>other space), which exposes the readable version of my data.
it fails on me when creating the v1e-snap device, but maybe i am just
too tired to figure it out now, see below...

>Maybe I could snapshot V1R and use the same encryption key as for V1E to
>make V2E?
when i try to luksOpen a snapshot i get "Device Busy"
and "device-mapper: ioctl: device doesn't appear to be in the dev hash
table." in dmesg

>Now that I think of it, I'm not even sure if LVM will snapshot the
>product of dm-crypt (V1E).
no, you have to do it by hand
it could be something like:

size=`blockdev --getsize /dev/mapper/v1e`
cowsize=$(( $size / 2048 * 20 / 100 )) # 20% of v1e size
chunk=8
lvcreate -n v1e-cow -l $cowsize /dev/vga
dmsetup table v1e | dmsetup create v1e-real
dmsetup suspend v1e
echo 0 $size snapshot /dev/mapper/v1e /dev/vga/v1e-cow p $chunk | dmsetup create v1e-snap
echo 0 $size snapshot-origin /dev/mapper/v1e | dmsetup create v1e-origin
dmsetup table v1e-origin | dmsetup load v1e
dmsetup resume v1e

mount /dev/mapper/v1e-snap /wherever
backup
umount /dev/mapper/v1e-snap

dmsetup suspend v1e
dmsetup remove v1e-snap
dmsetup remove v1e-origin
dmsetup table v1e-real | dmsetup load v1e
dmsetup resume v1e


-- 
Luca Berra -- bluca@comedia.it
         Communication Media & Services S.r.l.
  /"\
  \ /     ASCII RIBBON CAMPAIGN
   X        AGAINST HTML MAIL
  / \

Re: [dm-crypt] advice on encrypted snapshots

[Arno Wagner]

One pice of meta-advice may be appropriate:

Try for simplicity. If you feel your solution is getting
too complex to understand it in one go, try very hard to
find a simpler one. Complex solutions are not only a lot
more likely to fail, the chances for them to be secure
are a lot worse.

Incidentially, the ooriginal question was about file 
backups, not snapshots. If you want a snapshot and
can afford to umount the device, just use dd.

A sector-level shapshot will leak information about 
which disk areas have been changed (another reason
to do file-level backup), but other than that
it is as secure as the original partition.

Arno

On Sun, Nov 01, 2009 at 12:39:47AM +0100, Luca Berra wrote:
> On Sat, Oct 31, 2009 at 11:03:02AM -0700, Ross Boylan wrote:
>> On Sat, 2009-10-31 at 09:12 +0100, Luca Berra wrote:
>>> On Fri, Oct 30, 2009 at 01:05:54PM -0700, Ross Boylan wrote:
>>> >Does anyone have any advice about how to snapshot an encrypted volume so
>>> >that the snapshot won't leak information?
>>> >
>>> Do you mean linux-lvm snapshot 
>> Yes.
>>> or some storage based one?
>> I'm not sure what that means, but I don't want to rsync or tar.  The
> i meant a storage devices which presents disk space as one or more lun
> to a host using either fibre-channel or iscasi or similar means, but
> that's not your case.
>>
>>> In the first case I think the safest way is encrypting the PV.
>>
>> I don't think I can.  Here's my setup:
>> V1E   encrypted volume, built on top of
>> V1R   raw volume, which is part of VGA   volume group, composed of
>> PVA   physical volume (which is actually software RAID).
>>
> ...
>>
>> So if I snapshot V1E I think I must use VGA (at any rate, I have no
>> other space), which exposes the readable version of my data.
> it fails on me when creating the v1e-snap device, but maybe i am just
> too tired to figure it out now, see below...
>
>> Maybe I could snapshot V1R and use the same encryption key as for V1E to
>> make V2E?
> when i try to luksOpen a snapshot i get "Device Busy"
> and "device-mapper: ioctl: device doesn't appear to be in the dev hash
> table." in dmesg
>
>> Now that I think of it, I'm not even sure if LVM will snapshot the
>> product of dm-crypt (V1E).
> no, you have to do it by hand
> it could be something like:
>
> size=`blockdev --getsize /dev/mapper/v1e`
> cowsize=$(( $size / 2048 * 20 / 100 )) # 20% of v1e size
> chunk=8
> lvcreate -n v1e-cow -l $cowsize /dev/vga
> dmsetup table v1e | dmsetup create v1e-real
> dmsetup suspend v1e
> echo 0 $size snapshot /dev/mapper/v1e /dev/vga/v1e-cow p $chunk | dmsetup create v1e-snap
> echo 0 $size snapshot-origin /dev/mapper/v1e | dmsetup create v1e-origin
> dmsetup table v1e-origin | dmsetup load v1e
> dmsetup resume v1e
>
> mount /dev/mapper/v1e-snap /wherever
> backup
> umount /dev/mapper/v1e-snap
>
> dmsetup suspend v1e
> dmsetup remove v1e-snap
> dmsetup remove v1e-origin
> dmsetup table v1e-real | dmsetup load v1e
> dmsetup resume v1e
>
>
> -- 
> Luca Berra -- bluca@comedia.it
>         Communication Media & Services S.r.l.
>  /"\
>  \ /     ASCII RIBBON CAMPAIGN
>   X        AGAINST HTML MAIL
>  / \
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] advice on encrypted snapshots

[Ross Boylan]

On Sun, 2009-11-01 at 03:05 +0100, Arno Wagner wrote:
> Incidentially, the ooriginal question was about file 
> backups, not snapshots. If you want a snapshot and
> can afford to umount the device, just use dd.
My original question was about using snapshots to make file-level
backups.  Apparently the meaning was unclear.  I want to use lvm
snapshots, and by "file backups" I mean that I want to backup individual
files on a file system, rather than getting some kind of disk image.
File backups were referring to the source, not the destination, of the
backup.

Ross

Re: [dm-crypt] advice on encrypted snapshots

[Luca Berra]

On Sun, Nov 01, 2009 at 03:05:31AM +0100, Arno Wagner wrote:
>One pice of meta-advice may be appropriate:
>
>Try for simplicity. If you feel your solution is getting
>too complex to understand it in one go, try very hard to
>find a simpler one. Complex solutions are not only a lot
>more likely to fail, the chances for them to be secure
>are a lot worse.
The solution i proposed at the beginning, encrypt the pv, was the
simplest and more secure one, it also prevented any possible leak.
Alas it does not work in the op environment, so we have to find a better
one.
Here comes solution two, snapshot the lv, it is still quite simple,
there is the possibility of some information leak, but i don't really
believe the cow table and some modified sector are enough material for
cryptanalysis.
But solution two fails, I can luksOpen either the original lv or the
snapshot, but not both at the same time. I have no idea if this is a
intended, a bug, or if i am overlooking something obvious.

>Incidentially, the ooriginal question was about file 
>backups, not snapshots. If you want a snapshot and
>can afford to umount the device, just use dd.
incidentally it was about snapshots, but your advice about dd makes me
realize you have no idea what a snapshot is.

A snapshot is not a byte-level copy of a disk, id call that a dump.
A snapshot is a live image of a block device frozen at the point in time
it was taken. A snapshot does not usually need the same amount of disk
space as the original, and creation is almost instant.
The most common use of snapshots is doing backups while preserving data
coherence.
If you backup a large data set while an application is active you will
find some files will change between the start and end of backup. If the
files are related probably having a backup of file A at version 1001 and
file B at version 1012 is not what you want.
One solution would be stopping the application for the whole time it
takes to back-up, but what if we cannot afford the downtime?
Enter snapshots:
You just have to stop/start your app for the few seconds it would take
to snapshot, or maybe your app can quiesce io, or you just cross your
finger and go on....
Then you remount the snapshot on a different mount point and do a file
level backup of that, as you would do with the original.

Third solution was written just to show how complex it is, i did not
really mean it, but since two did not work i gave it a try, and found it
does not work either. In the form written below it is not even secure,
but if it worked it could be improved by encrypting the backing storage
of the cow table with a random key, and make the snapshot volatile. That
would even negate the info leak we have at two.

Regards,
L.

P.S. sorry for the embarrassing amount of typos in the mail below, but i
was really tired.


>On Sun, Nov 01, 2009 at 12:39:47AM +0100, Luca Berra wrote:
>> On Sat, Oct 31, 2009 at 11:03:02AM -0700, Ross Boylan wrote:
>>> On Sat, 2009-10-31 at 09:12 +0100, Luca Berra wrote:
>>>> On Fri, Oct 30, 2009 at 01:05:54PM -0700, Ross Boylan wrote:
>>>> >Does anyone have any advice about how to snapshot an encrypted volume so
>>>> >that the snapshot won't leak information?
>>>> >
>>>> Do you mean linux-lvm snapshot 
>>> Yes.
>>>> or some storage based one?
>>> I'm not sure what that means, but I don't want to rsync or tar.  The
>> i meant a storage devices which presents disk space as one or more lun
>> to a host using either fibre-channel or iscasi or similar means, but
>> that's not your case.
>>>
>>>> In the first case I think the safest way is encrypting the PV.
>>>
>>> I don't think I can.  Here's my setup:
>>> V1E   encrypted volume, built on top of
>>> V1R   raw volume, which is part of VGA   volume group, composed of
>>> PVA   physical volume (which is actually software RAID).
>>>
>> ...
>>>
>>> So if I snapshot V1E I think I must use VGA (at any rate, I have no
>>> other space), which exposes the readable version of my data.
>> it fails on me when creating the v1e-snap device, but maybe i am just
>> too tired to figure it out now, see below...
>>
>>> Maybe I could snapshot V1R and use the same encryption key as for V1E to
>>> make V2E?
>> when i try to luksOpen a snapshot i get "Device Busy"
>> and "device-mapper: ioctl: device doesn't appear to be in the dev hash
>> table." in dmesg
>>
>>> Now that I think of it, I'm not even sure if LVM will snapshot the
>>> product of dm-crypt (V1E).
>> no, you have to do it by hand
>> it could be something like:
>>
>> size=`blockdev --getsize /dev/mapper/v1e`
>> cowsize=$(( $size / 2048 * 20 / 100 )) # 20% of v1e size
>> chunk=8
>> lvcreate -n v1e-cow -l $cowsize /dev/vga
>> dmsetup table v1e | dmsetup create v1e-real
>> dmsetup suspend v1e
>> echo 0 $size snapshot /dev/mapper/v1e /dev/vga/v1e-cow p $chunk | dmsetup create v1e-snap
>> echo 0 $size snapshot-origin /dev/mapper/v1e | dmsetup create v1e-origin
>> dmsetup table v1e-origin | dmsetup load v1e
>> dmsetup resume v1e
>>
>> mount /dev/mapper/v1e-snap /wherever
>> backup
>> umount /dev/mapper/v1e-snap
>>
>> dmsetup suspend v1e
>> dmsetup remove v1e-snap
>> dmsetup remove v1e-origin
>> dmsetup table v1e-real | dmsetup load v1e
>> dmsetup resume v1e
>>
>>
>> -- 
>> Luca Berra -- bluca@comedia.it
>>         Communication Media & Services S.r.l.
>>  /"\
>>  \ /     ASCII RIBBON CAMPAIGN
>>   X        AGAINST HTML MAIL
>>  / \
>> _______________________________________________
>> dm-crypt mailing list
>> dm-crypt@saout.de
>> http://www.saout.de/mailman/listinfo/dm-crypt
>>
>
>-- 
>Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
>GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
>----
>Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans
>
>If it's in the news, don't worry about it.  The very definition of 
>"news" is "something that hardly ever happens." -- Bruce Schneier 
>_______________________________________________
>dm-crypt mailing list
>dm-crypt@saout.de
>http://www.saout.de/mailman/listinfo/dm-crypt

-- 
Luca Berra -- bluca@comedia.it
         Communication Media & Services S.r.l.
  /"\
  \ /     ASCII RIBBON CAMPAIGN
   X        AGAINST HTML MAIL
  / \

Re: [dm-crypt] advice on encrypted snapshots

[Arno Wagner]

On Sun, Nov 01, 2009 at 09:06:48AM +0100, Luca Berra wrote:
> On Sun, Nov 01, 2009 at 03:05:31AM +0100, Arno Wagner wrote:
>> One pice of meta-advice may be appropriate:
>>
>> Try for simplicity. If you feel your solution is getting
>> too complex to understand it in one go, try very hard to
>> find a simpler one. Complex solutions are not only a lot
>> more likely to fail, the chances for them to be secure
>> are a lot worse.
> The solution i proposed at the beginning, encrypt the pv, was the
> simplest and more secure one, it also prevented any possible leak.
> Alas it does not work in the op environment, so we have to find a better
> one.
> Here comes solution two, snapshot the lv, it is still quite simple,
> there is the possibility of some information leak, but i don't really
> believe the cow table and some modified sector are enough material for
> cryptanalysis.
> But solution two fails, I can luksOpen either the original lv or the
> snapshot, but not both at the same time. I have no idea if this is a
> intended, a bug, or if i am overlooking something obvious.
>
>> Incidentially, the ooriginal question was about file backups, not 
>> snapshots. If you want a snapshot and
>> can afford to umount the device, just use dd.
> incidentally it was about snapshots, but your advice about dd makes me
> realize you have no idea what a snapshot is.

Actually I do know what a snapshot is. You are thinking of a 
more complex operation it seems, namely snapshot taking and 
then conventional, file based backup of the snapshot. 

No reason why an encryption layer should add problems. Of 
course you need to snapshot the encrypted volume,
and you may want to sync or better umount the volume during
snapshot taking. 

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] advice on encrypted snapshots [solved]

[Ross Boylan]

On Sat, 2009-10-31 at 11:03 -0700, Ross Boylan wrote:
> Here's my setup:
> V1E   encrypted volume, built on top of
> V1R   raw volume, which is part of 
> VGA   volume group, composed of
> PVA   physical volume (which is actually software RAID).
> 
> My terminology may be non-standard: V1E is readable and looks like a
> regular file system; V1R is the one that looks scrambled.  i.e.,
> cyptsetup luksOpen /dev/mapper/VGA-V1R V1E.
> One could say that V1R is encrypted.  Is there a conventional way to
> refer to these items?  V1R is the "backing device" and V1E is ....?
> 
> So if I snapshot V1E I think I must use VGA (at any rate, I have no
> other space), which exposes the readable version of my data.
> 
> Maybe I could snapshot V1R and use the same encryption key as for V1E
> to
> make V2E?
That works.  Although Luca reported problems opening V1R and the
snapshot at the same time, I did not encounter problems.  However, there
was no activity on the original disk.
> 
> Now that I think of it, I'm not even sure if LVM will snapshot the
> product of dm-crypt (V1E).
I'm pretty sure this can't work, since one must snapshot a volume path.
VGA/V1E does not exist (at least, /dev/VGA/V1E does not),
though /dev/mapper/V1E does.

I'm running a Linux 2.6.30 kernel.

Ross Boylan