[linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[Jean-Marc Saffroy]

[-- Attachment #1.1: Type: text/plain, Size: 1433 bytes --]

Hello LVM experts,

I am trying to create a volume with the following properties:
- the volume can be resized
- the volume is encrypted
- the volume can be snapshotted (for online backups)

So I thought I'd create the volume with LVM, encrypt it with LUKS, and
snapshot it with LVM. However, LVM doesn't want to snapshot the unencrypted
LUKS volume, as it is not an actual logical volume known to LVM (and I am
not keen on snapshotting the encrypted volume, as that means the backup
process would need the passphrase to mount the encrypted snapshot).

Is there a good way to achieve this with LUKS and LVM, or should I look
elsewhere?

I have two ideas but I don't know if they are safe or practical:
- I could try running LVM (snapshots) on top of LUKS (encryption) itself on
top of LVM (resize)
- or I could try working with dmsetup to fill the gap between LUKS and LVM

I did simple tests with dmsetup, and that *seems* to work, however I am not
sure at all if that would be robust. An outline of my test:
- create an LVM volume (lvcreate) from a larger volume group
- make it a LUKS volume (cryptsetup lukfsFormat)
- "open" the LUKS volume (cryptsetup open)
- create a snapshot-origin volume from the open LUKS volume (dmsetup create)
- mount that as my active volume
- every time I want to do a backup:
  create a temporary snapshot volume from the origin, mount it, run the
backup, unmount it, delete it

Thoughts?

Cheers,
JM

[-- Attachment #1.2: Type: text/html, Size: 1805 bytes --]

[-- Attachment #2: Type: text/plain, Size: 202 bytes --]

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[Harald Dunkel]

What about encrypting the physical volumes, and creating the volume groups
and logical volumes on top of it?

Regards
Harri

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[Zdenek Kabelac]

Dne 25. 09. 23 v 0:09 Jean-Marc Saffroy napsal(a):
> Hello LVM experts,
> 
> I am trying to create a volume with the following properties:
> - the volume can be resized
> - the volume is encrypted
> - the volume can be snapshotted (for online backups)
> 
> So I thought I'd create the volume with LVM, encrypt it with LUKS, and 
> snapshot it with LVM. However, LVM doesn't want to snapshot the unencrypted 
> LUKS volume, as it is not an actual logical volume known to LVM (and I am not 
> keen on snapshotting the encrypted volume, as that means the backup process 
> would need the passphrase to mount the encrypted snapshot).
> 
> Is there a good way to achieve this with LUKS and LVM, or should I look elsewhere?
> 
> I have two ideas but I don't know if they are safe or practical:
> - I could try running LVM (snapshots) ontop of LUKS (encryption)itself ontop 
> of LVM (resize)

Hi


Yep typical usage is to encrypt underlying PV - and then create LVs and its 
snapshots on encrypted device.


> - or I could try working with dmsetup to fill the gap between LUKS and LVM
> 
> I did simple tests with dmsetup, and that *seems* to work, however I am not 
> sure at all if that would be robust. An outline of my test:
> - create an LVM volume (lvcreate) from a larger volume group
> - make it a LUKS volume (cryptsetup lukfsFormat)
> - "open" the LUKS volume (cryptsetup open)
> - create a snapshot-origin volume from the open LUKS volume (dmsetup create)
> - mount that as my active volume
> - every time I want to do a backup:
>    create a temporary snapshot volume from the origin, mount it, run the 
> backup, unmount it, delete it

Usually those 'into encryption' want to have encrypted everything - thus even 
layout of the whole storage.

Encrypting 'individual' LVs - while certainly 'doable' would i.e. create a 
considerable larger amount of volumes that would need individual 'unlocking' 
with each activation.

Speaking about snapshots - you should consider switching to 'thin-pools'  for 
far better performance...

Regards

Zdenek


_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[Jean-Marc Saffroy]

Hi,

On Tue, Sep 26, 2023 at 10:00 PM Zdenek Kabelac
<zdenek.kabelac@gmail.com> wrote:
> Yep typical usage is to encrypt underlying PV - and then create LVs and its
> snapshots on encrypted device.

Sure, I'd do that in other circumstances.

But in my case it would just be a waste: I am replacing several disks
on a desktop computer with a single 2TB NVME SSD for everything. Only
/home needs to be encrypted, and it's tiny, like 100-200GB. Going
through encryption for most application I/Os would use CPU time and
increase latency with no benefit.

So I prefer to manage available raw (un-encrypted) space with LVM.

Now, I also need to do backups of /home, and that's why I want
snapshots. But that first layer of LVM would only show a snapshot of
an encrypted volume, and the backup job shouldn't have the passphrase
to decrypt the volume.

Which is why I'm trying to find a way of doing snaphots of an "opened"
LUKS volume: this way, the backup job can do its job without requiring
a passphrase.

In simple tests, I could make it work, with dmsetup on LUKS on LVM,
and also (after I sent my original email) with LVM on LUKS on LVM.

But my tests don't tell me if there are other people doing similar
things on production systems, or if they are happy with the results.
Unusual setups tend to exhibit unusual bugs, and I am not super fond
of bugs in my storage systems. :-)

So that's really the core of my question: do other people run either
"raw" dmsetup or LVM on top of LUKS/LVM, and with success?

> Encrypting 'individual' LVs - while certainly 'doable' would i.e. create a
> considerable larger amount of volumes that would need individual 'unlocking'
> with each activation.

Just the one /home in my case, so no worse than prompting for the
passphrase for an entire disk.

> Speaking about snapshots - you should consider switching to 'thin-pools'  for
> far better performance...

I only need snapshots for backups: once a day, create a snapshot,
mount it, do a file-level incremental backup, unmount it, delete it.

Would the thin-pools make a difference in this case?


Cheers,
JM

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[Stuart D Gathman]

On Wed, 27 Sep 2023, Jean-Marc Saffroy wrote:

> So I prefer to manage available raw (un-encrypted) space with LVM.
>
> Now, I also need to do backups of /home, and that's why I want
> snapshots. But that first layer of LVM would only show a snapshot of
> an encrypted volume, and the backup job shouldn't have the passphrase
> to decrypt the volume.
>
> Which is why I'm trying to find a way of doing snaphots of an "opened"
> LUKS volume: this way, the backup job can do its job without requiring
> a passphrase.

Besides LVM on LUKS on LVM which you already tried, consider using
a filesystem that supports snapshots.  I use btrfs, and snapshots work
beautifully, and if you use "btrfs send" you can even do differential
backups.  Btrfs is COW, so snaps share all blocks not touched.

Pipe the output of btrfs send directly to your backup process/server
running "btrfs receive".  Note, this requires the backup server to have
btrfs.  If it doesn't, then just use rsync from the snapshot directory
to the backup server like a typical unix backup solution.  (E.g. my vm
host uses XFS on the backup drives, so it uses rsync.)

> In simple tests, I could make it work, with dmsetup on LUKS on LVM,
> and also (after I sent my original email) with LVM on LUKS on LVM.

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[Demi Marie Obenour]

[-- Attachment #1.1: Type: text/plain, Size: 1219 bytes --]

On Wed, Sep 27, 2023 at 01:10:10AM +0200, Jean-Marc Saffroy wrote:
> Hi,
> 
> On Tue, Sep 26, 2023 at 10:00 PM Zdenek Kabelac
> <zdenek.kabelac@gmail.com> wrote:
> > Yep typical usage is to encrypt underlying PV - and then create LVs and its
> > snapshots on encrypted device.
> 
> Sure, I'd do that in other circumstances.
> 
> But in my case it would just be a waste: I am replacing several disks
> on a desktop computer with a single 2TB NVME SSD for everything. Only
> /home needs to be encrypted, and it's tiny, like 100-200GB. Going
> through encryption for most application I/Os would use CPU time and
> increase latency with no benefit.

"No benefit" depends on one's threat model.  A surprising amount of
sensitive data gets put outside of /home.  For instance, SSH host keys
are in /etc, and system daemons store their data in /var.  That's why
the standard is to encrypt the entire drive, except for /boot and
/boot/efi.  It's the only way to ensure that sensitive data doesn't wind
up on the NVMe drive, from which it cannot be removed except by
destroying or (cryptographically) securely erasing the drive.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 202 bytes --]

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[Zdenek Kabelac]

Dne 27. 09. 23 v 1:10 Jean-Marc Saffroy napsal(a):
> Hi,
> 
> On Tue, Sep 26, 2023 at 10:00 PM Zdenek Kabelac
> <zdenek.kabelac@gmail.com> wrote:
>> Yep typical usage is to encrypt underlying PV - and then create LVs and its
>> snapshots on encrypted device.
> 
> Sure, I'd do that in other circumstances.
> 
> But in my case it would just be a waste: I am replacing several disks
> on a desktop computer with a single 2TB NVME SSD for everything. Only
> /home needs to be encrypted, and it's tiny, like 100-200GB. Going
> through encryption for most application I/Os would use CPU time and
> increase latency with no benefit.
> 
> So I prefer to manage available raw (un-encrypted) space with LVM.
> 
> Now, I also need to do backups of /home, and that's why I want
> snapshots. But that first layer of LVM would only show a snapshot of
> an encrypted volume, and the backup job shouldn't have the passphrase
> to decrypt the volume.
> 
> Which is why I'm trying to find a way of doing snaphots of an "opened"
> LUKS volume: this way, the backup job can do its job without requiring
> a passphrase.

well that's where you will considerably 'complicate' your life :)
As you would need to 'orchestrace' this yourself with 'dmsetup' usage.

running 'dmsetup suspend' on your home device,
that taking a snapshot of your underlying  LV.

Here the usage of 'thin-pool' would possibly help a little bit - as you get a 
control over when a snapshot LV appears in your system.

Once you have the snapshot created you 'resume'  the top-level
decrypted volume.

Then if you want to access your snapshot - you create another 'crypto' device 
- unlock it again with your key - and it should work.

But the level of complexity here is rather  high - this it might be actually
way easier to just 'partition' your device for  'encrypted'  and unecrypted' 
parts and use 2 PVs for 2 VGs....

> But my tests don't tell me if there are other people doing similar
> things on production systems, or if they are happy with the results.
> Unusual setups tend to exhibit unusual bugs, and I am not super fond
> of bugs in my storage systems. :-)

Yep - people prefer simple rock solid solutions....
That's why the above describe scenarios is not really used....
As solving then all individual errors that may appear is far from being simple.


> Just the one /home in my case, so no worse than prompting for the
> passphrase for an entire disk.

Every access to a snapshot needs then a new separate  'unlock'..

>> Speaking about snapshots - you should consider switching to 'thin-pools'  for
>> far better performance...
> 
> I only need snapshots for backups: once a day, create a snapshot,
> mount it, do a file-level incremental backup, unmount it, delete it.
> 
> Would the thin-pools make a difference in this case?

Well there are many ways how to skin a cat...
I.e. check  blk-archive  https://github.com/jthornber/blk-archive

Regards

Zdenek

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[Roberto Fastec]

[-- Attachment #1.1: Type: text/plain, Size: 4163 bytes --]

What an awful idea to encrypt data on an hardware accelerator

SSD is everything but an affordable storage for keeping safely your data

and as soon some cells will fail , and with LVM , the first ones will be where LVM tables are sitting (and being heavily modified) .. you'll loose ALL the data, because of the extents mechanism

Nevertheless, if you got damaged just the cells where encryption keys are written, again you loose everything 

Remember that the data must seat on the affordable hard disk drive.

SSD is just an hardware accelerator, if of high quality (enterprise class) it can be good as cache for RAID systems

Man that has been advised, is half saved

Kind regards

R.

⁣Ottieni BlueMail per Android ​

Il giorno 27 set 2023, 11:58, alle ore 11:58, Zdenek Kabelac <zdenek.kabelac@gmail.com> ha scritto:
>Dne 27. 09. 23 v 1:10 Jean-Marc Saffroy napsal(a):
>> Hi,
>> 
>> On Tue, Sep 26, 2023 at 10:00 PM Zdenek Kabelac
>> <zdenek.kabelac@gmail.com> wrote:
>>> Yep typical usage is to encrypt underlying PV - and then create LVs
>and its
>>> snapshots on encrypted device.
>> 
>> Sure, I'd do that in other circumstances.
>> 
>> But in my case it would just be a waste: I am replacing several disks
>> on a desktop computer with a single 2TB NVME SSD for everything. Only
>> /home needs to be encrypted, and it's tiny, like 100-200GB. Going
>> through encryption for most application I/Os would use CPU time and
>> increase latency with no benefit.
>> 
>> So I prefer to manage available raw (un-encrypted) space with LVM.
>> 
>> Now, I also need to do backups of /home, and that's why I want
>> snapshots. But that first layer of LVM would only show a snapshot of
>> an encrypted volume, and the backup job shouldn't have the passphrase
>> to decrypt the volume.
>> 
>> Which is why I'm trying to find a way of doing snaphots of an
>"opened"
>> LUKS volume: this way, the backup job can do its job without
>requiring
>> a passphrase.
>
>well that's where you will considerably 'complicate' your life :)
>As you would need to 'orchestrace' this yourself with 'dmsetup' usage.
>
>running 'dmsetup suspend' on your home device,
>that taking a snapshot of your underlying  LV.
>
>Here the usage of 'thin-pool' would possibly help a little bit - as you
>get a 
>control over when a snapshot LV appears in your system.
>
>Once you have the snapshot created you 'resume'  the top-level
>decrypted volume.
>
>Then if you want to access your snapshot - you create another 'crypto'
>device 
>- unlock it again with your key - and it should work.
>
>But the level of complexity here is rather  high - this it might be
>actually
>way easier to just 'partition' your device for  'encrypted'  and
>unecrypted' 
>parts and use 2 PVs for 2 VGs....
>
>> But my tests don't tell me if there are other people doing similar
>> things on production systems, or if they are happy with the results.
>> Unusual setups tend to exhibit unusual bugs, and I am not super fond
>> of bugs in my storage systems. :-)
>
>Yep - people prefer simple rock solid solutions....
>That's why the above describe scenarios is not really used....
>As solving then all individual errors that may appear is far from being
>simple.
>
>
>> Just the one /home in my case, so no worse than prompting for the
>> passphrase for an entire disk.
>
>Every access to a snapshot needs then a new separate  'unlock'..
>
>>> Speaking about snapshots - you should consider switching to
>'thin-pools'  for
>>> far better performance...
>> 
>> I only need snapshots for backups: once a day, create a snapshot,
>> mount it, do a file-level incremental backup, unmount it, delete it.
>> 
>> Would the thin-pools make a difference in this case?
>
>Well there are many ways how to skin a cat...
>I.e. check  blk-archive  https://github.com/jthornber/blk-archive
>
>Regards
>
>Zdenek
>
>_______________________________________________
>linux-lvm mailing list
>linux-lvm@redhat.com
>https://listman.redhat.com/mailman/listinfo/linux-lvm
>read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

[-- Attachment #1.2: Type: text/html, Size: 5850 bytes --]

[-- Attachment #2: Type: text/plain, Size: 202 bytes --]

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[Jean-Marc Saffroy]

On Wed, Sep 27, 2023 at 11:58 AM Zdenek Kabelac
<zdenek.kabelac@gmail.com> wrote:
>
> Dne 27. 09. 23 v 1:10 Jean-Marc Saffroy napsal(a):
> > Hi,
> >
> > On Tue, Sep 26, 2023 at 10:00 PM Zdenek Kabelac
> > <zdenek.kabelac@gmail.com> wrote:
> >> Yep typical usage is to encrypt underlying PV - and then create LVs and its
> >> snapshots on encrypted device.
> >
> > Sure, I'd do that in other circumstances.
> >
> > But in my case it would just be a waste: I am replacing several disks
> > on a desktop computer with a single 2TB NVME SSD for everything. Only
> > /home needs to be encrypted, and it's tiny, like 100-200GB. Going
> > through encryption for most application I/Os would use CPU time and
> > increase latency with no benefit.
> >
> > So I prefer to manage available raw (un-encrypted) space with LVM.
> >
> > Now, I also need to do backups of /home, and that's why I want
> > snapshots. But that first layer of LVM would only show a snapshot of
> > an encrypted volume, and the backup job shouldn't have the passphrase
> > to decrypt the volume.
> >
> > Which is why I'm trying to find a way of doing snaphots of an "opened"
> > LUKS volume: this way, the backup job can do its job without requiring
> > a passphrase.
>
> well that's where you will considerably 'complicate' your life :)
> As you would need to 'orchestrace' this yourself with 'dmsetup' usage.

If you mean that I'd have to script a few things, then I am perfectly
fine with that.

> running 'dmsetup suspend' on your home device,
> that taking a snapshot of your underlying  LV.

What is the role of "dmsetup suspend"? I am having trouble finding
decent documentation about its purpose and how it's related to
snapshots. I did not need it in my experiments, so I am curious.

> Here the usage of 'thin-pool' would possibly help a little bit - as you get a
> control over when a snapshot LV appears in your system.
>
> Once you have the snapshot created you 'resume'  the top-level
> decrypted volume.
>
> Then if you want to access your snapshot - you create another 'crypto' device
> - unlock it again with your key - and it should work.

At boot time (or login time, same thing), I need to give the
passphrase to open the LUKS device, but after that, backup jobs run in
the background, they are not interactive.

If I were okay with giving the passphrase to my backup script, then I
could simply have the backup script create its snapshot from the
encrypted LV, and I wouldn't have started this thread in this case.
:-)

> But the level of complexity here is rather  high - this it might be actually
> way easier to just 'partition' your device for  'encrypted'  and unecrypted'
> parts and use 2 PVs for 2 VGs....

But then I can't resize the encrypted volume/partition.

> > Just the one /home in my case, so no worse than prompting for the
> > passphrase for an entire disk.
>
> Every access to a snapshot needs then a new separate  'unlock'..

Not with the approaches I suggested, as they work to build the
snapshots on top of the "open", decrypted device.

It seems LVM cannot do it directly, but it becomes possible (at least
in my simple tests) if I use a bunch of dmsetup commands, or if I use
the decrypted device as the PV for a new VG.

But I don't know if these approaches are safe to use, and that is what
drove me here.

In the mean time, I found this page:
https://access.redhat.com/articles/2106521

Apparently, LVM on LUKS on LVM would be case of "LVM recursion", and
so not entirely unheard of.

Does anyone here have experience with "LVM recursion"?

Cheers,
JM

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[Jean-Marc Saffroy]

Wow, that was very harsh AND very wrong.

All storage devices die eventually, that's why we have backups (and
RAID, off-site replication, etc). HDDs and SSDs are the same in this
respect.

SSDs do NOT die just from repeated writes to the same sector: this can
happen with raw flash cells, but SSDs are NOT that, they all include
complex logic for wear leveling, and it works very well in practice.

As SSDs grow larger, they can withstand more writes, as wear leveling
spreads the wear over more flash cells.

For datacenter use, SSDs are gaining more and more use cases as
primary storage, and their reliability is getting ahead of HDDs:
https://www.backblaze.com/blog/ssd-drive-stats-mid-2022-review/

Home users need not worry at all about SSD longevity. They just need
to be ready for they drives to die, just like any storage device, and
use backups.

JM

On Wed, Sep 27, 2023 at 3:26 PM Roberto Fastec <roberto.fastec@gmail.com> wrote:
>
> What an awful idea to encrypt data on an hardware accelerator
>
> SSD is everything but an affordable storage for keeping safely your data
>
> and as soon some cells will fail , and with LVM , the first ones will be where LVM tables are sitting (and being heavily modified) .. you'll loose ALL the data, because of the extents mechanism
>
> Nevertheless, if you got damaged just the cells where encryption keys are written, again you loose everything
>
> Remember that the data must seat on the affordable hard disk drive.
>
> SSD is just an hardware accelerator, if of high quality (enterprise class) it can be good as cache for RAID systems
>
> Man that has been advised, is half saved
>
> Kind regards
>
> R.
>
> Ottieni BlueMail per Android
> Il giorno 27 set 2023, alle ore 11:58, Zdenek Kabelac <zdenek.kabelac@gmail.com> ha scritto:
>>
>> Dne 27. 09. 23 v 1:10 Jean-Marc Saffroy napsal(a):
>>>
>>>  Hi,
>>>
>>>  On Tue, Sep 26, 2023 at 10:00 PM Zdenek Kabelac
>>>  <zdenek.kabelac@gmail.com> wrote:
>>>>
>>>>  Yep typical usage is to encrypt underlying PV - and then create LVs and its
>>>>  snapshots on encrypted device.
>>>
>>>
>>>  Sure, I'd do that in other circumstances.
>>>
>>>  But in my case it would just be a waste: I am replacing several disks
>>>  on a desktop computer with a single 2TB NVME SSD for everything. Only
>>>  /home needs to be encrypted, and it's tiny, like 100-200GB. Going
>>>  through encryption for most application I/Os would use CPU time and
>>>  increase latency with no benefit.
>>>
>>>  So I prefer to manage available raw (un-encrypted) space with LVM.
>>>
>>>  Now, I also need to do backups of /home, and that's why I want
>>>  snapshots. But that first layer of LVM would only show a snapshot of
>>>  an encrypted volume, and the backup job shouldn't have the passphrase
>>>  to decrypt the volume.
>>>
>>>  Which is why I'm trying to find a way of doing snaphots of an "opened"
>>>  LUKS volume: this way, the backup job can do its job without requiring
>>>  a passphrase.
>>
>>
>> well that's where you will considerably 'complicate' your life :)
>> As you would need to 'orchestrace' this yourself with 'dmsetup' usage.
>>
>> running 'dmsetup suspend' on your home device,
>> that taking a snapshot of your underlying  LV.
>>
>> Here the usage of 'thin-pool' would possibly help a little bit - as you get a
>> control over when a snapshot LV appears in your system.
>>
>> Once you have the snapshot created you 'resume'  the top-level
>> decrypted volume.
>>
>> Then if you want to access your snapshot - you create another 'crypto' device
>> - unlock it again with your key - and it should work.
>>
>> But the level of complexity here is rather  high - this it might be actually
>> way easier to just 'partition' your device for  'encrypted'  and unecrypted'
>> parts and use 2 PVs for 2 VGs....
>>
>>>  But my tests don't tell me if there are other people doing similar
>>>  things on production systems, or if they are happy with the results.
>>>  Unusual setups tend to exhibit unusual bugs, and I am not super fond
>>>  of bugs in my storage systems. :-)
>>
>>
>> Yep - people prefer simple rock solid solutions....
>> That's why the above describe scenarios is not really used....
>> As solving then all individual errors that may appear is far from being simple.
>>
>>
>>>  Just the one /home in my case, so no worse than prompting for the
>>>  passphrase for an entire disk.
>>
>>
>> Every access to a snapshot needs then a new separate  'unlock'..
>>
>>>>  Speaking about snapshots - you should consider switching to 'thin-pools'  for
>>>>  far better performance...
>>>
>>>
>>>  I only need snapshots for backups: once a day, create a snapshot,
>>>  mount it, do a file-level incremental backup, unmount it, delete it.
>>>
>>>  Would the thin-pools make a difference in this case?
>>
>>
>> Well there are many ways how to skin a cat...
>> I.e. check  blk-archive  https://github.com/jthornber/blk-archive
>>
>> Regards
>>
>> Zdenek
>>
>> ________________________________
>>
>> linux-lvm mailing list
>> linux-lvm@redhat.com
>> https://listman.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[Zdenek Kabelac]

Dne 27. 09. 23 v 15:45 Jean-Marc Saffroy napsal(a):
> On Wed, Sep 27, 2023 at 11:58 AM Zdenek Kabelac
> <zdenek.kabelac@gmail.com> wrote:
>>
>> Dne 27. 09. 23 v 1:10 Jean-Marc Saffroy napsal(a):
>>> Hi,
>>>
>>> On Tue, Sep 26, 2023 at 10:00 PM Zdenek Kabelac
>>> <zdenek.kabelac@gmail.com> wrote:
>>>> Yep typical usage is to encrypt underlying PV - and then create LVs and its
>>>> snapshots on encrypted device.
>>>
>>> Sure, I'd do that in other circumstances.
>>>
>>> But in my case it would just be a waste: I am replacing several disks
>>> on a desktop computer with a single 2TB NVME SSD for everything. Only
>>> /home needs to be encrypted, and it's tiny, like 100-200GB. Going
>>> through encryption for most application I/Os would use CPU time and
>>> increase latency with no benefit.
>>>
>>> So I prefer to manage available raw (un-encrypted) space with LVM.
>>>
>>> Now, I also need to do backups of /home, and that's why I want
>>> snapshots. But that first layer of LVM would only show a snapshot of
>>> an encrypted volume, and the backup job shouldn't have the passphrase
>>> to decrypt the volume.
>>>
>>> Which is why I'm trying to find a way of doing snaphots of an "opened"
>>> LUKS volume: this way, the backup job can do its job without requiring
>>> a passphrase.
>>
>> well that's where you will considerably 'complicate' your life :)
>> As you would need to 'orchestrace' this yourself with 'dmsetup' usage.
> 
> If you mean that I'd have to script a few things, then I am perfectly
> fine with that.
> 
>> running 'dmsetup suspend' on your home device,
>> that taking a snapshot of your underlying  LV.
> 
> What is the role of "dmsetup suspend"? I am having trouble finding
> decent documentation about its purpose and how it's related to
> snapshots. I did not need it in my experiments, so I am curious.
> 


Suspend is freezing device's i/o queue  (together with freezing FS layer - so 
the snapshot should be easily mountable without requiring extensive fsck 
operation as it would be missing some important metadata to be written on disk)
So the goal of a suspend is to take a 'good point in time' where the content 
of snapshot is having all 'committed' transaction on disk in valid state.

Clearly that needs a 'top-level device - which would be a crypto DM in your 
case - and goes via 'tree' down to PV level.

Clearly lvm2 does this while taking snapshot  - and you can easily observe 
that 'magic' if you read carefully  -vvvv  trace of a command.

Then your script needs to replicate this at script level.
Fun would begin once you would start to resolve all the possible error paths...


> If I were okay with giving the passphrase to my backup script, then I
> could simply have the backup script create its snapshot from the
> encrypted LV, and I wouldn't have started this thread in this case.
> :-)

Maybe you could drop your whole disk encryption idea then and just use some 
encrypted tarballs - since if you tend to place passwords into scripts - it's 
kind of big security hole....

If all you want to have is encrypted files on disk - there are probably more 
easier approaches with the use of encrypted filesystems...

>> But the level of complexity here is rather  high - this it might be actually
>> way easier to just 'partition' your device for  'encrypted'  and unecrypted'
>> parts and use 2 PVs for 2 VGs....
> 
> But then I can't resize the encrypted volume/partition.

Not sure how often do you need to do that - surely a 'split' between those 2 
partitions is a decisive point...

But then whatever you do within those VG is fully resizable as before.


> It seems LVM cannot do it directly, but it becomes possible (at least
> in my simple tests) if I use a bunch of dmsetup commands, or if I use
> the decrypted device as the PV for a new VG.
> 
> But I don't know if these approaches are safe to use, and that is what
> drove me here.
> 
> In the mean time, I found this page:
> https://access.redhat.com/articles/2106521
> 
> Apparently, LVM on LUKS on LVM would be case of "LVM recursion", and
> so not entirely unheard of.
> 
> Does anyone here have experience with "LVM recursion"?

lvm2 does not advice to use 'stacking'  - it's very complicated and in some 
way inefficient - it's always true the less layers -> the better the 
performance will be (especially true with modern NVMe devices...)

And lvm2 itself does NOT support/count with recursion - so while it may appear 
to be working - there will be corner cases with unresolvable problems - 
although this is mostly an issue if you are running your system from such 
'stacked' solution (causing deadlocks...)

Anyway - placing VG on top of another VG is always good to avoid and use only 
as a last resource if there is no better approach existing.

Regards

Zdenek

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[Jean-Marc Saffroy]

On Wed, Sep 27, 2023 at 5:41 PM Zdenek Kabelac <zdenek.kabelac@gmail.com> wrote:
>
> > What is the role of "dmsetup suspend"? I am having trouble finding
> > decent documentation about its purpose and how it's related to
> > snapshots. I did not need it in my experiments, so I am curious.
> >
>
>
> Suspend is freezing device's i/o queue  (together with freezing FS layer - so
> the snapshot should be easily mountable without requiring extensive fsck
> operation as it would be missing some important metadata to be written on disk)
> So the goal of a suspend is to take a 'good point in time' where the content
> of snapshot is having all 'committed' transaction on disk in valid state.

Is this still required or useful with a journaling FS like ext4? It is
robust to pulling the plug at any time, so any point in time should be
good, no?

> Clearly that needs a 'top-level device - which would be a crypto DM in your
> case - and goes via 'tree' down to PV level.
>
> Clearly lvm2 does this while taking snapshot  - and you can easily observe
> that 'magic' if you read carefully  -vvvv  trace of a command.
>
> Then your script needs to replicate this at script level.
> Fun would begin once you would start to resolve all the possible error paths...

Clearly I have no plan to reimplement LVM with homegrown scripts. :-)

That said, I am curious about what can be achieved with dmsetup
commands. By any chance, do you have pointers to documentation besides
what's in the kernel (Documentation/admin-guide/device-mapper/)?

> > If I were okay with giving the passphrase to my backup script, then I
> > could simply have the backup script create its snapshot from the
> > encrypted LV, and I wouldn't have started this thread in this case.
> > :-)
>
> Maybe you could drop your whole disk encryption idea then and just use some
> encrypted tarballs - since if you tend to place passwords into scripts - it's
> kind of big security hole....

So I think I've found a decent alternative to putting passwords into
scripts or files: kernel key rings.

It turns out crypttab (at least on Debian) can leverage keyctl to
enable reusing some secrets without exposing them too much.

I half-tested a crypttab file like this:

# <target name> <source device>         <key file>      <options>
home    /dev/vg1/crypthome      home
luks,noearly,keyscript=/root/decrypt_keyctl
snap    /dev/vg1/snap           home
luks,noauto,keyscript=/root/decrypt_keyctl

The referenced script above is merely a copy Debian's decrypt_keyctl
with the "keyctl timeout" lines commented.

In this way, each re-creation of the snapshot should reuse the secret
stored in root's "user" keyring with the "cryptsetup:home"
description, and this secret should be first stored in the keyring
when prompting the user at boot time.

The backup job can later do something like this:

> + lvcreate -n snap --snapshot -L 1G /dev/vg1/crypthome
>   Logical volume "snap" created.
> + cryptdisks_start snap
> Starting crypto disk...snap (starting)...Using cached passphrase for snap.
> snap (started)...done.
> + mount /dev/mapper/snap /mnt/snap
... perform the actual file backup...
> + umount /mnt/snap
> + cryptdisks_stop snap
> Stopping crypto disk...snap (stopping)...done.
> + lvremove -y /dev/vg1/snap
>   Logical volume "snap" successfully removed.

And so in this case, LUKS on LVM can do the job and meet all my requirements.

The minor drawback is that the passphrase can still be seen by root
with a command like:
  keyctl print $(keyctl search @u user cryptsetup:home)

But that's good enough for me.

Cheers,
JM

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can I combine LUKS and LVM to achieve encryption and snapshots?

[Zdenek Kabelac]

Dne 28. 09. 23 v 14:23 Jean-Marc Saffroy napsal(a):
> On Wed, Sep 27, 2023 at 5:41 PM Zdenek Kabelac <zdenek.kabelac@gmail.com> wrote:
>>
>>> What is the role of "dmsetup suspend"? I am having trouble finding
>>> decent documentation about its purpose and how it's related to
>>> snapshots. I did not need it in my experiments, so I am curious.
>>>
>>
>>
>> Suspend is freezing device's i/o queue  (together with freezing FS layer - so
>> the snapshot should be easily mountable without requiring extensive fsck
>> operation as it would be missing some important metadata to be written on disk)
>> So the goal of a suspend is to take a 'good point in time' where the content
>> of snapshot is having all 'committed' transaction on disk in valid state.
> 
> Is this still required or useful with a journaling FS like ext4? It is
> robust to pulling the plug at any time, so any point in time should be
> good, no?

Wondering where do you came to the idea that journaling FS can rescue such 
scenario flawlessly.  Sure 'FS' should not completely broke itself if you 
avoid this suspension & fsfreeze  - but on the other hand the internal 
inconsistency within a snapshot would require some repairing operation to 
happen - and potential risk of valid data loss as even ext4 by default 
journals only it's metadata, and 'data' are journaled only in 'data=journal' 
mode - which is however used only by very small group of users who are willing 
to give-up performance for this feature.

In all other cases - you want to get FS into frozen state before taking its 
snapshot - so there is maximal consistency.


> That said, I am curious about what can be achieved with dmsetup
> commands. By any chance, do you have pointers to documentation besides
> what's in the kernel (Documentation/admin-guide/device-mapper/)?

There are some DM talks available on the net describing some target logic in 
greater details with some drawn boxes describing I/O flow - but other then 
that I'm not sure what other kind of help would be needed here?

Regards

Zdenek

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/