[dm-crypt] u?mount (8) helper script for luks encrypted disks

[dm-crypt] u?mount (8) helper script for luks encrypted disks

[Steffen Vogel]

[-- Attachment #1: Type: text/plain, Size: 1678 bytes --]

Dear list,

Today I worked on a simple way to mount/umount luks encrypted disks:

I know, there a several ways to do this: cryptmount, cryptsetup, initd
scripts etc..

But I was looking for a way to use the standard mount (8) utility for
this. I came up with mount "helper" scripts as used sometimes with
ntfs-3g, fuse or nfs filesystems. These helper scripts are located
in /sbin/mount.FSTYPE and executed in precendence if they exist.
I introduced a "virtual" FSTYPE named "luks" to identify my luks
encrypted drives.

My version a simple Bash script which is based on cryptsetup:

https://github.com/stv0g/snippets/blob/master/bash/mount.luks.sh
(Please note the comments in the script for further tech details.)

Now I'm able to mount my drives with a simple call to mount (8):

	mount -t luks /dev/sda1 /home

Or use a line in my /etc/fstab for this:

	/dev/sda/   /home   luks   defaults,compress   0 0

Followed by a std "mount /home"

At the moment my script has some minor drawbacks which could be
fixed for the future:

1. Mount has to automatically determine the real filesystem type.
   If it fails with this, my script wont work.

2. Currently, passphrases can only supplied via STDIN.



I'm curious about your feedback. And perhaps we could add this to the
cryptsetup tarball as it's a helper script based on cryptsetup.

Or do you think thats its up to the distro maintainers to include such a
enhancement?

Regards,

	Steffen

-- 
Steffen Vogel
Robensstraße 69
52070 Aachen

Mail: post@steffenvogel.de
Mobil: +49 176 34911387
Web: http://www.steffenvogel.de
Jabber: steffen.vogel@jabber.rwth-aachen.de

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [dm-crypt] u?mount (8) helper script for luks encrypted disks

[Matthias Schniedermeyer]

On 24.08.2013 17:40, Steffen Vogel wrote:
> Dear list,
> 
> Today I worked on a simple way to mount/umount luks encrypted disks:
> 
> I know, there a several ways to do this: cryptmount, cryptsetup, initd
> scripts etc..
> 
> But I was looking for a way to use the standard mount (8) utility for
> this. I came up with mount "helper" scripts as used sometimes with
> ntfs-3g, fuse or nfs filesystems. These helper scripts are located
> in /sbin/mount.FSTYPE and executed in precendence if they exist.
> I introduced a "virtual" FSTYPE named "luks" to identify my luks
> encrypted drives.
> 
> My version a simple Bash script which is based on cryptsetup:
> 
> https://github.com/stv0g/snippets/blob/master/bash/mount.luks.sh
> (Please note the comments in the script for further tech details.)
> 
> Now I'm able to mount my drives with a simple call to mount (8):
> 
> 	mount -t luks /dev/sda1 /home
> 
> Or use a line in my /etc/fstab for this:
> 
> 	/dev/sda/   /home   luks   defaults,compress   0 0
> 
> Followed by a std "mount /home"
> 
> At the moment my script has some minor drawbacks which could be
> fixed for the future:
> 
> 1. Mount has to automatically determine the real filesystem type.
>    If it fails with this, my script wont work.

Hmmm. I don't know if it works for everything, but i know it works for 
fuse

mount -t fuse.sshfs ...
Which calls /sbin/mount.fuse and it gets the information that it should 
mount a sshfs.

If it's a generic solution this should work:
mount -t luks.xfs ...

Which you maybe have to parse before you pass it to the second 
mount-process you have to be calling.

> 2. Currently, passphrases can only supplied via STDIN.
> 
> 
> 
> I'm curious about your feedback. And perhaps we could add this to the
> cryptsetup tarball as it's a helper script based on cryptsetup.
> 
> Or do you think thats its up to the distro maintainers to include such a
> enhancement?

Personally i "solved" this by renaming /bin/mount to /bin/mount.orig 
and putting a shell-script as /bin/mount that checks if i want to mount 
a /dev/mapper/XXX and does the setup of XXX before it calls 
/bin/mount.orig.

"Back then" when i implemented that about 1.5 years ago i tried to 
explain to Karel Zak (util-linux maintainer) that a generic "premount" 
and "postumount" command in (u)mount could solve this generic problem. 
The Problem that all cryptographic-setups need (at least) one more step 
to setup(/tear-down) a device. But that didn't happen and i didn't try 
to open the issue again.





-- 

Matthias

Re: [dm-crypt] u?mount (8) helper script for luks encrypted disks

[Milan Broz]

On 26.8.2013 10:23, Matthias Schniedermeyer wrote:
> Personally i "solved" this by renaming /bin/mount to /bin/mount.orig
> and putting a shell-script as /bin/mount that checks if i want to mount
> a /dev/mapper/XXX and does the setup of XXX before it calls
> /bin/mount.orig.

Underlying device construction can be very complex task sometimes
(it can be combination of lvm, mdraid, multipath, partitions and whatever.)

So while it works for your use case, it will not work for other.

> "Back then" when i implemented that about 1.5 years ago i tried to
> explain to Karel Zak (util-linux maintainer) that a generic "premount"
> and "postumount" command in (u)mount could solve this generic problem.
> The Problem that all cryptographic-setups need (at least) one more step
> to setup(/tear-down) a device. But that didn't happen and i didn't try
> to open the issue again.


For that particular case, LUKS tear down, I think we had a better approach.
Just implement auto removal on last device close (similar to loop device
autoclear flag.)

For more info see
https://bugzilla.redhat.com/show_bug.cgi?id=873734


Milan

p.s.
Be very careful with shell scripts in mount helpers here.
It will run under the same UID as mount, which means root for LUKS here.

Re: [dm-crypt] u?mount (8) helper script for luks encrypted disks

[Matthias Schniedermeyer]

On 29.08.2013 07:50, Milan Broz wrote:
> On 26.8.2013 10:23, Matthias Schniedermeyer wrote:
> >Personally i "solved" this by renaming /bin/mount to /bin/mount.orig
> >and putting a shell-script as /bin/mount that checks if i want to mount
> >a /dev/mapper/XXX and does the setup of XXX before it calls
> >/bin/mount.orig.
> 
> Underlying device construction can be very complex task sometimes
> (it can be combination of lvm, mdraid, multipath, partitions and whatever.)
> 
> So while it works for your use case, it will not work for other.

And there will never be a "one size fits all" solution for this.

Sure, someone could create a "monster" that could cope with anything.
But that wouldn't be KISS.

So for the cases that are simple, a mount-option to exec something 
before the actual mount happens would suffice.
(I aimed my solutions for autofs with ghosting.)

> >"Back then" when i implemented that about 1.5 years ago i tried to
> >explain to Karel Zak (util-linux maintainer) that a generic "premount"
> >and "postumount" command in (u)mount could solve this generic problem.
> >The Problem that all cryptographic-setups need (at least) one more step
> >to setup(/tear-down) a device. But that didn't happen and i didn't try
> >to open the issue again.
> 
> 
> For that particular case, LUKS tear down, I think we had a better approach.
> Just implement auto removal on last device close (similar to loop device
> autoclear flag.)
> 
> For more info see
> https://bugzilla.redhat.com/show_bug.cgi?id=873734

I will see if it works and can kill the umount-part if it works for me.

> 
> Milan
> 
> p.s.
> Be very careful with shell scripts in mount helpers here.
> It will run under the same UID as mount, which means root for LUKS here.

In my case that's not the case, Shell-scripts (under normal 
circumstances) can't be SUIDed. So as i've intercepted the original call 
to mount it will run with the UID of the caller and will only SUID root 
when mount.orig is execed.

So if i would call it as my user, it simply wouldn't work as it couldn't 
construct the mapper.



-- 

Matthias

Re: [dm-crypt] u?mount (8) helper script for luks encrypted disks

[.. ink ..]

[-- Attachment #1: Type: text/plain, Size: 1796 bytes --]

On 29.08.2013 07:50, Milan Broz wrote:
> > On 26.8.2013 10:23, Matthias Schniedermeyer wrote:
> > >Personally i "solved" this by renaming /bin/mount to /bin/mount.orig
> > >and putting a shell-script as /bin/mount that checks if i want to mount
> > >a /dev/mapper/XXX and does the setup of XXX before it calls
> > >/bin/mount.orig.
> >
> > Underlying device construction can be very complex task sometimes
> > (it can be combination of lvm, mdraid, multipath, partitions and
> whatever.)
> >
> > So while it works for your use case, it will not work for other.
>
> And there will never be a "one size fits all" solution for this.
>
> Sure, someone could create a "monster" that could cope with anything.
> But that wouldn't be KISS.
>
>
Its possible to have a "one size that fits all" without it being a "monster"

In your "mount" script,take a path to an arbitrary device and then do the
following.

On mounting:

1. call "blkid" and check the file system on the device,if its present and
its not "crypto_LUKS",then its a device with a normal file system,just
mount it normally.

2. if the file system is found to be "crypto_LUKS",then call cryptseup to
unlock the path with whatever tool policy you have to create the mapper
path.Then call "blkid" against the mapper path to check the file system and
then mount the mapper normally.

Its just that simple.


On unmounting.
1. Look at the path to be unmounted,if it starts with "/dev/mapper/" then
it could an mdraid path or a cryptsetup mapper path or something else.Its
easy to check which one is it.
2. If its encrypted mapper path,then unmount the mapper and then call
cryptsetup to unmap the mapper.If its not encrypted then just unmount.

The whole thing seem easy enough and can be done by adding a handful of if
statements in the script

[-- Attachment #2: Type: text/html, Size: 2566 bytes --]

Re: [dm-crypt] u?mount (8) helper script for luks encrypted disks

[Milan Broz]

Well, if you want use such mount helper script, you can.

But I do no think this should be part of upstream package.

On 30.8.2013 1:56, .. ink .. wrote:
> On unmounting.
> 1. Look at the path to be unmounted,if it starts with "/dev/mapper/"
> then it could an mdraid path or a cryptsetup mapper path or
> something else.Its easy to check which one is it.

Nope.

You have /dev/VG/LV or /dev/mpath/NAME (aka friendly mpath names) etc.
These are symlinks, or on non-udev system directly device nodes.

If you want to check for dm device, you should check for major
number of device. Ditto for MD (note md can be some fake
raid as well as replacement for dmraid.)

In shell script, I would use lsblk to detect it (that should have all
"known" tricks included :)

> 2. If its
> encrypted mapper path,then unmount the mapper and then call
> cryptsetup to unmap the mapper.If its not encrypted then just
> unmount.

Not sure if it works for lazy umount (device is still in use).

Also udev can perform blkid "scan on close" blocking the device
but this one problem should be already solved inside libdevmapper
(retrying removal).

Milan

Re: [dm-crypt] u?mount (8) helper script for luks encrypted disks

[.. ink ..]

[-- Attachment #1: Type: text/plain, Size: 909 bytes --]

> On 30.8.2013 1:56, .. ink .. wrote:
>
>> On unmounting.
>> 1. Look at the path to be unmounted,if it starts with "/dev/mapper/"
>> then it could an mdraid path or a cryptsetup mapper path or
>> something else.Its easy to check which one is it.
>>
>
> Nope.
>
> You have /dev/VG/LV or /dev/mpath/NAME (aka friendly mpath names) etc.
> These are symlinks, or on non-udev system directly device nodes.
>
> If you want to check for dm device, you should check for major
> number of device. Ditto for MD (note md can be some fake
> raid as well as replacement for dmraid.)
>
>
checking for major:minor number does not work on devices with btrfs[1]

What i do in my project to identify lvm path is to check if the path exists
in both "/dev/abc/def" and "/dev/mapper/abd-def".formats.

mdraid paths are usually in "/dev/mdX" or "/dev/md/X" format.


[1] http://article.gmane.org/gmane.comp.file-systems.btrfs/2851

[-- Attachment #2: Type: text/html, Size: 1596 bytes --]

Re: [dm-crypt] u?mount (8) helper script for luks encrypted disks

[Milan Broz]

On 30.8.2013 7:58, .. ink .. wrote:
>
>     On 30.8.2013 1:56, .. ink .. wrote:
>
>         On unmounting.
>         1. Look at the path to be unmounted,if it starts with "/dev/mapper/"
>         then it could an mdraid path or a cryptsetup mapper path or
>         something else.Its easy to check which one is it.
>
>
>     Nope.
>
>     You have /dev/VG/LV or /dev/mpath/NAME (aka friendly mpath names) etc.
>     These are symlinks, or on non-udev system directly device nodes.
>
>     If you want to check for dm device, you should check for major
>     number of device. Ditto for MD (note md can be some fake
>     raid as well as replacement for dmraid.)
>
>
> checking for major:minor number does not work on devices with btrfs[1]

Yes. But I thought you want to check if underlying device is DM device,
then major is DM major number always.

>
> What i do in my project to identify lvm path is to check if the path exists in both "/dev/abc/def" and "/dev/mapper/abd-def".formats.

That will _usually_ work but the cost of duplicating lvm/mpath naming
logic in your program (I think multipath has this configurable also).

>
> mdraid paths are usually in "/dev/mdX" or "/dev/md/X" format.

Yes. But once this will change, you will have to update all programs
which have this logic hardcoded.


But again, it it works for you, that's fine but maintaining
such script upstream means thinking about generic solution.

> [1] http://article.gmane.org/gmane.comp.file-systems.btrfs/2851

Yes, I know about it, util-linux maintainer complained several times about
it as well.

Milan

Re: [dm-crypt] u?mount (8) helper script for luks encrypted disks

[Matthias Schniedermeyer]

On 29.08.2013 19:56, .. ink .. wrote:
> On 29.08.2013 07:50, Milan Broz wrote:
> > > On 26.8.2013 10:23, Matthias Schniedermeyer wrote:
> > > >Personally i "solved" this by renaming /bin/mount to /bin/mount.orig
> > > >and putting a shell-script as /bin/mount that checks if i want to mount
> > > >a /dev/mapper/XXX and does the setup of XXX before it calls
> > > >/bin/mount.orig.
> > >
> > > Underlying device construction can be very complex task sometimes
> > > (it can be combination of lvm, mdraid, multipath, partitions and
> > whatever.)
> > >
> > > So while it works for your use case, it will not work for other.
> >
> > And there will never be a "one size fits all" solution for this.
> >
> > Sure, someone could create a "monster" that could cope with anything.
> > But that wouldn't be KISS.
> >
> >
> Its possible to have a "one size that fits all" without it being a "monster"
> 
> In your "mount" script,take a path to an arbitrary device and then do the
> following.
> 
> On mounting:
> 
> 1. call "blkid" and check the file system on the device,if its present and
> its not "crypto_LUKS",then its a device with a normal file system,just
> mount it normally.

And i would crash & burn right here. Not all encryption is LUKS!

I use loopAES v3 encryption (a.k.a. lmk3).

There are zero identifiable features in a file or block-device that is 
loopAES (any version) encrypted. Just like plain encryption. And if i 
understood it correctly, this is also true for e.g. a Truecrypt 
container.

A "monster" would natuarally support any and all encryption models, that 
includes encryption models that can't be identified.

And my personal model has also a splash of special-sauce. My "whole 
disc" encryption is from sector 8 until the end of device. So i can put 
a dummy-MBR on each HDD in which i can stamp the name. This name in turn 
is used in a udev-rule to create a symlink that identifies the connected 
HDD. And last but not least, there is the matching autofs configuration, 
so i can just cd /misc/<name> after connecting the corresponding HDD.

> 2. if the file system is found to be "crypto_LUKS",then call cryptseup to
> unlock the path with whatever tool policy you have to create the mapper
> path.Then call "blkid" against the mapper path to check the file system and
> then mount the mapper normally.
> 
> Its just that simple.

Only if you restrict it to simple cases, but that would be KISS and not 
"monster".

> On unmounting.
> 1. Look at the path to be unmounted,if it starts with "/dev/mapper/" then
> it could an mdraid path or a cryptsetup mapper path or something else.Its
> easy to check which one is it.
> 2. If its encrypted mapper path,then unmount the mapper and then call
> cryptsetup to unmap the mapper.If its not encrypted then just unmount.
> 
> The whole thing seem easy enough and can be done by adding a handful of if
> statements in the script

Umount-case (in the future(?)) should be a solved problem by the 
autoclose-feature, so the mapper is automatically destructed after the 
filesystem is umounted. The interesting question, altough not for me, is 
if that works with stacked mappings.




-- 

Matthias

Re: [dm-crypt] u?mount (8) helper script for luks encrypted disks

[.. ink ..]

[-- Attachment #1: Type: text/plain, Size: 1457 bytes --]

> > 1. call "blkid" and check the file system on the device,if its present
> and
> > its not "crypto_LUKS",then its a device with a normal file system,just
> > mount it normally.
>
> And i would crash & burn right here. Not all encryption is LUKS!
>
> I use loopAES v3 encryption (a.k.a. lmk3).
>
> i prefer PLAIN type of cryptsetup.The basic idea still stands,encrypted
volumes will cause blkid to return either "crypto_LUKS" or nothing.This is
the easiest way to identify an encrypted volume.

any particular reason why you prefer loopAES over plain type of cryptsetup?

There are zero identifiable features in a file or block-device that is
> loopAES (any version) encrypted. Just like plain encryption. And if i
> understood it correctly, this is also true for e.g. a Truecrypt
> container.
>
> yap,truecrypt works the same way.

And my personal model has also a splash of special-sauce. My "whole
> disc" encryption is from sector 8 until the end of device. So i can put
> a dummy-MBR on each HDD in which i can stamp the name. This name in turn
> is used in a udev-rule to create a symlink that identifies the connected
> HDD. And last but not least, there is the matching autofs configuration,
> so i can just cd /misc/<name> after connecting the corresponding HDD.
>
>
why dont you use udev links created in "/dev/disk/by-id/" to access your
HDD? it seem like
you are doing something udev is already doing and wasting a bit of space in
the process.

[-- Attachment #2: Type: text/html, Size: 2257 bytes --]

Re: [dm-crypt] u?mount (8) helper script for luks encrypted disks

[Matthias Schniedermeyer]

On 30.08.2013 04:24, .. ink .. wrote:
> > > 1. call "blkid" and check the file system on the device,if its present
> > and
> > > its not "crypto_LUKS",then its a device with a normal file system,just
> > > mount it normally.
> >
> > And i would crash & burn right here. Not all encryption is LUKS!
> >
> > I use loopAES v3 encryption (a.k.a. lmk3).
> >
> > i prefer PLAIN type of cryptsetup.The basic idea still stands,encrypted
> volumes will cause blkid to return either "crypto_LUKS" or nothing.This is
> the easiest way to identify an encrypted volume.
> 
> any particular reason why you prefer loopAES over plain type of cryptsetup?

It's the first encryption i used and is "good enough", since AES-NI it's 
also fast enough to not be noticable.

Also converting >100TB of HDDs isn't really fun, so i decided to not 
change anything and also keep my schema for new HDDs.

It works for me. :-)

> There are zero identifiable features in a file or block-device that is
> > loopAES (any version) encrypted. Just like plain encryption. And if i
> > understood it correctly, this is also true for e.g. a Truecrypt
> > container.
> >
> > yap,truecrypt works the same way.
> 
> And my personal model has also a splash of special-sauce. My "whole
> > disc" encryption is from sector 8 until the end of device. So i can put
> > a dummy-MBR on each HDD in which i can stamp the name. This name in turn
> > is used in a udev-rule to create a symlink that identifies the connected
> > HDD. And last but not least, there is the matching autofs configuration,
> > so i can just cd /misc/<name> after connecting the corresponding HDD.
> >
> >
> why dont you use udev links created in "/dev/disk/by-id/" to access your
> HDD? it seem like
> you are doing something udev is already doing and wasting a bit of space in
> the process.

Not stable enough, and some USB-enclosures "destroy" that information. 
As i want to be able to connect my HDD in any way, the information 
to identify a specific HDD has to be stable regardless of 
connection method (SATA, USB2, USB3).

At least "back then" when i tried that years ago it wasn't stable/usable 
for multiple connection paths, and even with USB2 alone i had enclosures 
where you couldn't differenciate between different HDDs because it 
didn't pass through the serial-number of the HDDs, so all different HDDs 
had the same link in by-id.

Also i would have to make and keep up to data a list of what is what.
That list wouldn't be "fun".
Counting historical HDDs i'm at something beyond 150 HDDs.
About 60 currently in use, after i migrated all HDDs <1.5TB to 3TB 
HDDs a little while.





-- 

Matthias