[dm-crypt] help mounting partitions in an encrypted disk after first reboot

[dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Julio Gago]

Hi there!

I'm facing an issue I've been unable to solve by myself. I've been exploring around a lot without success. I'm a newbie in this topic so please accept my apologies if this is a silly question.

I am using cryptsetup 1.6.6 in Ubuntu 16.04 on kernel 4.8.0.

I successfully encrypted a disk with LUKS and then created a couple of partitions and mounted them with:

cryptsetup luksFormat --cipher aes-xts /dev/sdc
cryptsetup luksOpen /dev/sdc sdc
fdisk /dev/mapper/sdc
(created partitions manually)
mkfs.ext4 /dev/mapper/sdc1
mkfs.ext4 /dev/mapper/sdc2
mount /dev/mapper/sdc1 /part1_dmcrypt
mount /dev/mapper/sdc2 /part2_dmcrypt

fdisk gave the usual complaint about updating partition list to kernel, which I ignored and assumed it would be fixed by the next reboot. But it didn't!

After reboot, I can open the luks volume normally:

cryptsetup luksOpen /dev/sdc sdc
cryptsetup status sdc
/dev/mapper/sdc is active and is in use.
  type:    LUKS1
  cipher:  aes-xts-plain64
  keysize: 256 bits
  device:  /dev/sdc
  offset:  4096 sectors
  size:    11719929856 sectors
  mode:    read/write

And I can see the partitions if I use fdisk:

Disk /dev/mapper/sdc: 5.5 TiB, 6000604086272 bytes, 11719929856 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 2F089102-C3CE-4C64-BA09-A19FCC49CFF8

Device                     Start         End    Sectors  Size Type
/dev/mapper/sdc-part1       2048  6442452991 6442450944    3T Linux filesystem
/dev/mapper/sdc-part2 6442452992 11719929822 5277476831  2.5T Linux filesystem

However, the kernel does not seem to see the partitions, since the block devices are not present in /dev:

ls -la /dev/mapper
total 0
drwxr-xr-x  2 root root      80 Jun 16 19:07 .
drwxr-xr-x 19 root root    4560 Jun 16 18:38 ..
crw-------  1 root root 10, 236 Jun 16 18:35 control
lrwxrwxrwx  1 root root       7 Jun 17 12:15 sdc -> ../dm-0

So I cannot mount the volumes normally. I tried partx and other methods to tell the kernel about the partitions without success. The corresponding IOCTL calls return with EINVAL error.

Decryption seems to be working alright (since I can see the partitions) and I can in fact mount the volumes doing something like this:

losetup /dev/loop0 /dev/mapper/sdc -o 1048576
mount /dev/loop0 /part1_dmcrypt

The volumes seem to have the right contents. So I am actually ok to proceed, I have backups of both volumes and the LUKS header.

However, I would like to understand what is wrong and how could I fix the issue and mount the volumes canonicaly.

Thanks in advance for your help and my apologies if I did anything really silly :)

Regards,
Julio

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Michael Kjörling]

On 18 Jun 2017 08:49 +0200, from julio.gago@metempsy.com (Julio Gago):
> cryptsetup luksFormat --cipher aes-xts /dev/sdc
> cryptsetup luksOpen /dev/sdc sdc
> fdisk /dev/mapper/sdc
> (created partitions manually)
> mkfs.ext4 /dev/mapper/sdc1
> mkfs.ext4 /dev/mapper/sdc2
> mount /dev/mapper/sdc1 /part1_dmcrypt
> mount /dev/mapper/sdc2 /part2_dmcrypt

What you have done here is to sub-partition a LUKS container using MBR
partitions.

You can do that, _technically_ (as you have found out), but I dare say
that it's not a typical setup. You are therefore likely to run into
edge cases that have seen relatively little testing, and some things
that might just be plain difficult to get to work reliably. I suspect
that what you are seeing here is more the latter than the former.

A typical setup would more likely be to partition the disk, then set
up a separate LUKS container (possibly with derived keys, which would
allow you to open all LUKS containers by opening just one) on each
partition. An alternative typical setup would be to create a LUKS
container over the whole disk and use that container as a single file
system, with no partitioning (in the sense of MBR or GPT) involved.

If you have your heart set on sub-partitioning the LUKS container (in
order to conceal the fact that the LUKS container is divided into
distinct portions), I suppose you could create a LVM container within
the LUKS container, then create logical volumes within the LVM
container, then file systems on those, for an end result of something
like:

Physical storage
    LUKS
        LVM
            LV
                FS
            LV
                FS
            LV
                FS
            ...

That would probably have better chances of working reliably than what
you have now. You'd open the LUKS container, then import the LVM
container, which will import the LVs within it and make the file
systems within those available for mounting. The LVM metadata would be
encrypted on disk by virtue of the LVM container existing fully within
the LUKS container, so the data security properties should be very
similar to those you'd get with your proposed partitions-within-LUKS
scheme.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Julio Gago]

 ---- On Sun, 18 Jun 2017 09:25:28 +0200 Michael Kjörling <michael@kjorling.se> wrote ---- 

 > What you have done here is to sub-partition a LUKS container using MBR
 > partitions.
 > 
 > You can do that, _technically_ (as you have found out), but I dare say
 > that it's not a typical setup. You are therefore likely to run into
 > edge cases that have seen relatively little testing, and some things
 > that might just be plain difficult to get to work reliably. I suspect
 > that what you are seeing here is more the latter than the former.
 > 
 > A typical setup would more likely be to partition the disk, then set
 > up a separate LUKS container (possibly with derived keys, which would
 > allow you to open all LUKS containers by opening just one) on each
 > partition. An alternative typical setup would be to create a LUKS
 > container over the whole disk and use that container as a single file
 > system, with no partitioning (in the sense of MBR or GPT) involved.
 > 
 > If you have your heart set on sub-partitioning the LUKS container [...]

Well, I din't have a strong opinion between encrypting first or partitioning first. Because I was going to encrypt all partitions inside the disk, it just seemed natural to encrypt once and then partition, as opposed to partition and encrypting several times. Less containers and less operations looked better :). In retrospective, and re-reading the FAQ, I can now see clearly that I cornered myself into one alternative that was not listed in there. I have really no intentions to enter the LVM world, I really just wanted some security with encryption :).

For what you say, I assume there is no easy path to go to a more typical setup without reformatting the volumes, hence I will need to schedule some downtime for that. I will stay with the trick to keep the partitions alive via the loop trick until then.

Is that reasonable or do you think I am going to hit reliability issues too?

Thanks a lot for your help,
Julio.

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Carl-Daniel Hailfinger]

On 18.06.2017 09:25, Michael Kjörling wrote:
> On 18 Jun 2017 08:49 +0200, from julio.gago@metempsy.com (Julio Gago):
>> cryptsetup luksFormat --cipher aes-xts /dev/sdc
>> cryptsetup luksOpen /dev/sdc sdc
>> fdisk /dev/mapper/sdc
>> (created partitions manually)
>> mkfs.ext4 /dev/mapper/sdc1
>> mkfs.ext4 /dev/mapper/sdc2
>> mount /dev/mapper/sdc1 /part1_dmcrypt
>> mount /dev/mapper/sdc2 /part2_dmcrypt
> What you have done here is to sub-partition a LUKS container using MBR
> partitions.
> [...]
> A typical setup would more likely be to partition the disk, then set
> up a separate LUKS container (possibly with derived keys, which would
> allow you to open all LUKS containers by opening just one) on each
> partition. An alternative typical setup would be to create a LUKS
> container over the whole disk and use that container as a single file
> system, with no partitioning (in the sense of MBR or GPT) involved.
>
> If you have your heart set on sub-partitioning the LUKS container (in
> order to conceal the fact that the LUKS container is divided into
> distinct portions), I suppose you could create a LVM container within
> the LUKS container, then create logical volumes within the LVM
> container, then file systems on those, for an end result of something
> like:
>
> Physical storage
>     LUKS
>         LVM
>             LV
>                 FS
>             LV
>                 FS
>             LV
>                 FS
>             ...
>
> That would probably have better chances of working reliably than what
> you have now. You'd open the LUKS container, then import the LVM
> container, which will import the LVs within it and make the file
> systems within those available for mounting. The LVM metadata would be
> encrypted on disk by virtue of the LVM container existing fully within
> the LUKS container, so the data security properties should be very
> similar to those you'd get with your proposed partitions-within-LUKS
> scheme.

That (LVM inside a LUKS container) is the standard scheme proposed by
Ubuntu for an encrypted installation. It works out of the box (needs
just a single click in the Ubuntu installer), is well-tested and
supports resizing the encrypted logical volumes at a later date.

If you don't need an unencrypted boot partition on that disk and if you
want to do all of this manually instead of trusting the installer, you
can try these commands which pretty much exactly replicate what Ubuntu
does by default.

cfdisk /dev/sdc
(create a single partition spanning the whole disk)
cryptsetup luksFormat --cipher aes-xts /dev/sdc1
cryptsetup luksOpen /dev/sdc1 sdc1_crypt
pvcreate /dev/mapper/sdc1_crypt
vgcreate encryptedvolumegroup /dev/mapper/sdc1_crypt
lvcreate --size 500G --name backups encryptedvolumegroup
lvcreate --size 100G --name documents encryptedvolumegroup
lvcreate --size 50G --name email encryptedvolumegroup
mkfs.xfs /dev/mapper/encryptedvolumegroup-backups
mkfs.ext3 /dev/mapper/encryptedvolumegroup-documents
mkfs.reiserfs /dev/mapper/encryptedvolumegroup-email
mount /dev/mapper/encryptedvolumegroup-backups /backups
mount /dev/mapper/encryptedvolumegroup-documents /documents
mount /dev/mapper/encryptedvolumegroup-email /email

Regards,
Carl-Daniel

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Arno Wagner]

On Sun, Jun 18, 2017 at 17:25:41 CEST, Carl-Daniel Hailfinger wrote:
> On 18.06.2017 09:25, Michael Kjörling wrote:
[...]
> That (LVM inside a LUKS container) is the standard scheme proposed by
> Ubuntu for an encrypted installation. It works out of the box (needs
> just a single click in the Ubuntu installer), is well-tested and
> supports resizing the encrypted logical volumes at a later date.

But keep in mind that it makes things a lot more complicated,
hence violating KISS. It is easier for doing fully automated 
stuff, like a distro-installer would do, but as soon as you 
do things manually, LVM is more of a problem than a solution.

We have had many people here on the list that killed their
LUKS containers by overwriting the headers with LVM or
as a result of LVM misconfiguration and we had others that
managed to change the LVM setup and then were unable to
find their LUKS containers afterwards.
 
My advice would be to stay away from LVM. In this scenario
it does not do more than a "partprobe" would do and it has
no advantages. It is a case of something that looks simple,
but is not, and that is the worst kind. If the ritual fails
(and complex things that look simple are usually done by
ritual, not by understanding), you are screwed.

Of course, in the Windows-world, that approach is standard
and it has been creeping into Linux for a while now (see,
e.g. systemd, LVM, udev, etc.). This is probably due to people
comming into the Linux community that never understood what
the problem with the Windows-approach is.

Sorry for the rant, I just ran into a problem with udev 
(again) an hour ago that makes me want to rip this whole
crappy "automess" stuff out. 

Regards,
Arno

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Carl-Daniel Hailfinger]

On 18.06.2017 17:51, Arno Wagner wrote:
> On Sun, Jun 18, 2017 at 17:25:41 CEST, Carl-Daniel Hailfinger wrote:
>> On 18.06.2017 09:25, Michael Kjörling wrote:
> [...]
>> That (LVM inside a LUKS container) is the standard scheme proposed by
>> Ubuntu for an encrypted installation. It works out of the box (needs
>> just a single click in the Ubuntu installer), is well-tested and
>> supports resizing the encrypted logical volumes at a later date.
> But keep in mind that it makes things a lot more complicated,
> hence violating KISS.

A single click is the very definition of KISS.


> It is easier for doing fully automated 
> stuff, like a distro-installer would do, but as soon as you 
> do things manually, LVM is more of a problem than a solution.
>
> We have had many people here on the list that killed their
> LUKS containers by overwriting the headers with LVM or
> as a result of LVM misconfiguration and we had others that
> managed to change the LVM setup and then were unable to
> find their LUKS containers afterwards.

(Not intended as a response to the OP, but rather a general observation.)
To be honest, if people want to work as root on the command line, they
should read all the man pages of the tools they are using as root, and
they should read the man pages in full instead of just copy-pasting
snippets from the EXAMPLES section. Reading the supplementary literature
about the design of those tools is strongly recommended well. At least
that's what I'm doing when I'm using tools to manipulate my disks.

Various graphical tools are available for people who are unable to
understand man pages or who are for some reason unwilling to read man
pages (the latter case applies to me when I'm in a hurry) and those
tools usually have quite a few sanity checks. Last time I checked (~2
years ago), the most user friendly one was the openSUSE YaST Partitioner
<https://doc.opensuse.org/documentation/leap/reference/html/book.opensuse.reference/cha.advdisk.html>,
followed by the KDE Volume and Partition Manager
<https://sourceforge.net/projects/kvpm/> and finally GParted
<http://gparted.org/>. I haven't checked the KDE Partition Manager,
QTParted and other tools.


> My advice would be to stay away from LVM. In this scenario
> it does not do more than a "partprobe" would do and it has
> no advantages. It is a case of something that looks simple,
> but is not, and that is the worst kind. If the ritual fails
> (and complex things that look simple are usually done by
> ritual, not by understanding), you are screwed.

Oh, traditional MBR partitioning is not simple at all, and it gets worse
with >2 TB disks. Try explaining to someone how primary partitions,
extended partitions and logical partitions work together. The
interaction with GPT partitioning and how the MBR partitioning looks if
GPT is active is the stuff of nightmares. Oh, and repartitioning a disk
with some partitions currently in use is still not really going to yield
satisfying results. Besides that, the students in my IT forensics
lectures usually complain that traditional MBR partitioning is the
weirdest scheme ever.
That said, I think that traditional MBR partitioning is the best choice
if all you want is static partitions without nesting.

Yes, the design of LVM takes some getting used to, but IMHO it is a
pretty good design. That said, I wouldn't use it in scenarios where the
requirement is a simple non-encrypted installation without any plans to
resize partitions.


> Of course, in the Windows-world, that approach is standard
> and it has been creeping into Linux for a while now (see,
> e.g. systemd, LVM, udev, etc.). This is probably due to people
> comming into the Linux community that never understood what
> the problem with the Windows-approach is.

I've done my fair share of Linux kernel development, firmware
development, reverse engineering and other fun stuff, and use Linux
exclusively since around 2000. In 1998, I thought people not using the
command line on Linux were heretics. I have changed since then. With the
old MS Windows versions, it was impossible to fix things which were not
supported by graphical tools, and I loved having the chance to fix (and
break) anything on the Linux command line. Back then, I thought
attracting all the new users with "bloatware" like desktop environments
was a bad idea. It took me a while to learn that this stuff was useful
after all. Nowadays I'm not afraid of people who want/use a GUI because
the GUIs have matured to a point where I'd even call them mostly failsafe.
I'm fine with LVM because it's a choice, and I use it on most of my
systems because it's useful for me. I haven't seen zealots trying to get
MBR support removed from Linux distros.
udev made my life easier in some cases, and back then you could use it
standalone.
systemd and git are useful for some people, but I've lost more time to
each of then than I can ever recover. I could accept that (sunk cost),
were it not for the zealots who will flame anyone questioning the
superiority of their sacred tool. The flaming also serves as a pretty
effective way of killing the motivation of anybody working on competing
solutions. The "you're holding it wrong" mentality of many of those
zealots isn't making them many friends either.


> Sorry for the rant, I just ran into a problem with udev 
> (again) an hour ago that makes me want to rip this whole
> crappy "automess" stuff out.

I completely agree that automation can mess things up a lot. I've seen
my forensics students desperately trying to disable the automounter (it
feels like any given distro has a dozen of them, interacting in weird
ways), and you can pretty much rely on any documentation to be either
non-existent, wrong or simply outdated.

That said, I strongly advocate that people should stick as closely as
possible to what their distro has as default because that's where other
more experienced people can easily help.

Regards,
Carl-Daniel

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Arno Wagner]

On Mon, Jun 19, 2017 at 00:26:42 CEST, Carl-Daniel Hailfinger wrote:
> On 18.06.2017 17:51, Arno Wagner wrote:
> > On Sun, Jun 18, 2017 at 17:25:41 CEST, Carl-Daniel Hailfinger wrote:
> >> On 18.06.2017 09:25, Michael Kjörling wrote:
> > [...]
> >> That (LVM inside a LUKS container) is the standard scheme proposed by
> >> Ubuntu for an encrypted installation. It works out of the box (needs
> >> just a single click in the Ubuntu installer), is well-tested and
> >> supports resizing the encrypted logical volumes at a later date.
> > But keep in mind that it makes things a lot more complicated,
> > hence violating KISS.
> 
> A single click is the very definition of KISS.

Not at all. KISS does not refer to what the user does, it
refers to what the system does. Hence a single click triggering
a complex operation is about at the largest distance from KISS 
that you can get.

I though that was obvious.  

Pressing a button is a ritual. Simplicity of the rituals 
used has no benefit  when you need to understand what is 
actually happening. A ritual does not give you any insight 
into the nature of what is going on and computing is not 
(and will not be for a long time) at a point where basically 
everything can be done by ritual in a secure and reliable 
manner.
 
> > It is easier for doing fully automated 
> > stuff, like a distro-installer would do, but as soon as you 
> > do things manually, LVM is more of a problem than a solution.
> >
> > We have had many people here on the list that killed their
> > LUKS containers by overwriting the headers with LVM or
> > as a result of LVM misconfiguration and we had others that
> > managed to change the LVM setup and then were unable to
> > find their LUKS containers afterwards.
> 
> (Not intended as a response to the OP, but rather a general observation.)
> To be honest, if people want to work as root on the command line, they
> should read all the man pages of the tools they are using as root, and
> they should read the man pages in full instead of just copy-pasting
> snippets from the EXAMPLES section. Reading the supplementary literature
> about the design of those tools is strongly recommended well. At least
> that's what I'm doing when I'm using tools to manipulate my disks.


Well, yes. And that is were KISS comes in: A system that is simple 
and clear is far easier to understand than one that is convoluted, 
complex and full of details and exceptions.

Good engineering makes understanding things easy, bad engineering
makes it hard. And bad engineering is full of surprises and
"emergent properties", and hence unreliable, insecure, hard
to maintain and use. That is the basis of where the KISS
requirement for good engineering comes from. 

There is a school of though that seems to think things should 
be complex in order to show off the superior skill of the
person that created them and then blame the user if they
have trouble to understand what is going on.

I consider that juvenile and a sign of gross incompetence. It 
is far harder to design a system to be simple than to be complex. 
For complex you just throw in everything and the kitchen sink. 
For simple, you actually need to understand the problem you are 
trying to solve and its very core nature.

People that do not try for simplicity have no business creating 
infrastructure in general use. Also, these people waste my time 
because of their mismatch between ego and skill. And I will
freely admit that their lack of skill offends me.

I will stop answering here, because I have no clue what you 
want to say with the rest below. 

Also, raving about your credentials does nothing for me, except 
making me think they are not nearly as good as you try to make 
them sound. That is the usual case.

Does give me a laugh though, as it reminds me of the one time
where some guy in a newsgroup actually looked me up and then 
accused me of being the janitor using my computer to post 
things after having broken into my office. Ah, good times ;-)

Regards,
Arno


 
> Various graphical tools are available for people who are unable to
> understand man pages or who are for some reason unwilling to read man
> pages (the latter case applies to me when I'm in a hurry) and those
> tools usually have quite a few sanity checks. Last time I checked (~2
> years ago), the most user friendly one was the openSUSE YaST Partitioner
> <https://doc.opensuse.org/documentation/leap/reference/html/book.opensuse.reference/cha.advdisk.html>,
> followed by the KDE Volume and Partition Manager
> <https://sourceforge.net/projects/kvpm/> and finally GParted
> <http://gparted.org/>. I haven't checked the KDE Partition Manager,
> QTParted and other tools.
> 
> 
> > My advice would be to stay away from LVM. In this scenario
> > it does not do more than a "partprobe" would do and it has
> > no advantages. It is a case of something that looks simple,
> > but is not, and that is the worst kind. If the ritual fails
> > (and complex things that look simple are usually done by
> > ritual, not by understanding), you are screwed.
> 
> Oh, traditional MBR partitioning is not simple at all, and it gets worse
> with >2 TB disks. Try explaining to someone how primary partitions,
> extended partitions and logical partitions work together. The
> interaction with GPT partitioning and how the MBR partitioning looks if
> GPT is active is the stuff of nightmares. Oh, and repartitioning a disk
> with some partitions currently in use is still not really going to yield
> satisfying results. Besides that, the students in my IT forensics
> lectures usually complain that traditional MBR partitioning is the
> weirdest scheme ever.
> That said, I think that traditional MBR partitioning is the best choice
> if all you want is static partitions without nesting.
> 
> Yes, the design of LVM takes some getting used to, but IMHO it is a
> pretty good design. That said, I wouldn't use it in scenarios where the
> requirement is a simple non-encrypted installation without any plans to
> resize partitions.
> 
> 
> > Of course, in the Windows-world, that approach is standard
> > and it has been creeping into Linux for a while now (see,
> > e.g. systemd, LVM, udev, etc.). This is probably due to people
> > comming into the Linux community that never understood what
> > the problem with the Windows-approach is.
> 
> I've done my fair share of Linux kernel development, firmware
> development, reverse engineering and other fun stuff, and use Linux
> exclusively since around 2000. In 1998, I thought people not using the
> command line on Linux were heretics. I have changed since then. With the
> old MS Windows versions, it was impossible to fix things which were not
> supported by graphical tools, and I loved having the chance to fix (and
> break) anything on the Linux command line. Back then, I thought
> attracting all the new users with "bloatware" like desktop environments
> was a bad idea. It took me a while to learn that this stuff was useful
> after all. Nowadays I'm not afraid of people who want/use a GUI because
> the GUIs have matured to a point where I'd even call them mostly failsafe.
> I'm fine with LVM because it's a choice, and I use it on most of my
> systems because it's useful for me. I haven't seen zealots trying to get
> MBR support removed from Linux distros.
> udev made my life easier in some cases, and back then you could use it
> standalone.
> systemd and git are useful for some people, but I've lost more time to
> each of then than I can ever recover. I could accept that (sunk cost),
> were it not for the zealots who will flame anyone questioning the
> superiority of their sacred tool. The flaming also serves as a pretty
> effective way of killing the motivation of anybody working on competing
> solutions. The "you're holding it wrong" mentality of many of those
> zealots isn't making them many friends either.
> 
> 
> > Sorry for the rant, I just ran into a problem with udev 
> > (again) an hour ago that makes me want to rip this whole
> > crappy "automess" stuff out.
> 
> I completely agree that automation can mess things up a lot. I've seen
> my forensics students desperately trying to disable the automounter (it
> feels like any given distro has a dozen of them, interacting in weird
> ways), and you can pretty much rely on any documentation to be either
> non-existent, wrong or simply outdated.
> 
> That said, I strongly advocate that people should stick as closely as
> possible to what their distro has as default because that's where other
> more experienced people can easily help.
> 
> Regards,
> Carl-Daniel
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Carl-Daniel Hailfinger]

On 19.06.2017 01:01, Arno Wagner wrote:
> On Mon, Jun 19, 2017 at 00:26:42 CEST, Carl-Daniel Hailfinger wrote:
>> On 18.06.2017 17:51, Arno Wagner wrote:
>>> On Sun, Jun 18, 2017 at 17:25:41 CEST, Carl-Daniel Hailfinger wrote:
>>>> On 18.06.2017 09:25, Michael Kjörling wrote:
>>> [...]
>>>> That (LVM inside a LUKS container) is the standard scheme proposed by
>>>> Ubuntu for an encrypted installation. It works out of the box (needs
>>>> just a single click in the Ubuntu installer), is well-tested and
>>>> supports resizing the encrypted logical volumes at a later date.
>>> But keep in mind that it makes things a lot more complicated,
>>> hence violating KISS.
>> A single click is the very definition of KISS.
> Not at all. KISS does not refer to what the user does, it
> refers to what the system does. Hence a single click triggering
> a complex operation is about at the largest distance from KISS 
> that you can get.
>
> I though that was obvious.

Actually, no, I don't think that's obvious at all. According to your
definition, using a hex editor for editing the MBR partition definitions
could be KISS because (I'm quoting you) "KISS does not refer to what the
user does".

However, according to your definition "KISS [...] refers to what the
system does" the on-disk representation of MBR partitions is definitely
not KISS. There are complex rules about whether a partition definition
is an entry in an array, or an entry in a singly linked list pointed to
by an array entry, how the on-disk ordering of the partitions has to
correspond with the type of entry, and of course (and totally
non-obvious) that one type of partition is assumed to never host a file
system, and all partitioning tools will declare said type of partition
as free space although it differs from actual free space not covered by
a partition. Oh, and try explaining to people why the second usable
partition on a Linux system usually is sda5 and not sda2.

> Pressing a button is a ritual. Simplicity of the rituals 
> used has no benefit  when you need to understand what is 
> actually happening. A ritual does not give you any insight 
> into the nature of what is going on and computing is not 
> (and will not be for a long time) at a point where basically 
> everything can be done by ritual in a secure and reliable 
> manner.

Insight ist often not necessary to use something. In many cases, it
doesn't even help. A newborn baby drinking from a bottle neither needs
to understand the complex chemical processes involved in creation of
milk and digestion of milk, nor would such a baby benefit from that
knowledge.

If insight is desired by someone, more power to them.


>>> It is easier for doing fully automated 
>>> stuff, like a distro-installer would do, but as soon as you 
>>> do things manually, LVM is more of a problem than a solution.
>>>
>>> We have had many people here on the list that killed their
>>> LUKS containers by overwriting the headers with LVM or
>>> as a result of LVM misconfiguration and we had others that
>>> managed to change the LVM setup and then were unable to
>>> find their LUKS containers afterwards.
>> (Not intended as a response to the OP, but rather a general observation.)
>> To be honest, if people want to work as root on the command line, they
>> should read all the man pages of the tools they are using as root, and
>> they should read the man pages in full instead of just copy-pasting
>> snippets from the EXAMPLES section. Reading the supplementary literature
>> about the design of those tools is strongly recommended well. At least
>> that's what I'm doing when I'm using tools to manipulate my disks.
> Well, yes. And that is were KISS comes in: A system that is simple 
> and clear is far easier to understand than one that is convoluted, 
> complex and full of details and exceptions.
>
> Good engineering makes understanding things easy, bad engineering
> makes it hard. And bad engineering is full of surprises and
> "emergent properties", and hence unreliable, insecure, hard
> to maintain and use. That is the basis of where the KISS
> requirement for good engineering comes from.

I think you're conflating "complex" and "convoluted" here. High
precision clockwork movements can be really complex, but being
convoluted is not something I'd associate with them.


> There is a school of though that seems to think things should 
> be complex in order to show off the superior skill of the
> person that created them and then blame the user if they
> have trouble to understand what is going on.
>
> I consider that juvenile and a sign of gross incompetence. It 
> is far harder to design a system to be simple than to be complex. 
> For complex you just throw in everything and the kitchen sink. 
> For simple, you actually need to understand the problem you are 
> trying to solve and its very core nature.

Yes, and sometimes problems are complex and need complex solutions.

> People that do not try for simplicity have no business creating 
> infrastructure in general use. Also, these people waste my time 
> because of their mismatch between ego and skill. And I will
> freely admit that their lack of skill offends me.
>
> I will stop answering here, because I have no clue what you 
> want to say with the rest below. 
>
> Also, raving about your credentials

I didn't rave about my credentials, I tried to explain that although my
opinion a few years ago was pretty close to yours right now, there are
reasons I've changed since then.

> does nothing for me, except 
> making me think they are not nearly as good as you try to make 
> them sound. That is the usual case.

My apologies. I thought you value such credentials greatly because you
list yours in your email signature of every email. Thank you for your
insight about "the usual case". Does it apply to you as well?


> Does give me a laugh though, as it reminds me of the one time
> where some guy in a newsgroup actually looked me up and then 
> accused me of being the janitor using my computer to post 
> things after having broken into my office. Ah, good times ;-)

I fail to see how this is related to our conversation.


> Regards,
> Arno
> -- 
> Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
> GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718


Regards,
Carl-Daniel


>> Various graphical tools are available for people who are unable to
>> understand man pages or who are for some reason unwilling to read man
>> pages (the latter case applies to me when I'm in a hurry) and those
>> tools usually have quite a few sanity checks. Last time I checked (~2
>> years ago), the most user friendly one was the openSUSE YaST Partitioner
>> <https://doc.opensuse.org/documentation/leap/reference/html/book.opensuse.reference/cha.advdisk.html>,
>> followed by the KDE Volume and Partition Manager
>> <https://sourceforge.net/projects/kvpm/> and finally GParted
>> <http://gparted.org/>. I haven't checked the KDE Partition Manager,
>> QTParted and other tools.
>>
>>
>>> My advice would be to stay away from LVM. In this scenario
>>> it does not do more than a "partprobe" would do and it has
>>> no advantages. It is a case of something that looks simple,
>>> but is not, and that is the worst kind. If the ritual fails
>>> (and complex things that look simple are usually done by
>>> ritual, not by understanding), you are screwed.
>> Oh, traditional MBR partitioning is not simple at all, and it gets worse
>> with >2 TB disks. Try explaining to someone how primary partitions,
>> extended partitions and logical partitions work together. The
>> interaction with GPT partitioning and how the MBR partitioning looks if
>> GPT is active is the stuff of nightmares. Oh, and repartitioning a disk
>> with some partitions currently in use is still not really going to yield
>> satisfying results. Besides that, the students in my IT forensics
>> lectures usually complain that traditional MBR partitioning is the
>> weirdest scheme ever.
>> That said, I think that traditional MBR partitioning is the best choice
>> if all you want is static partitions without nesting.
>>
>> Yes, the design of LVM takes some getting used to, but IMHO it is a
>> pretty good design. That said, I wouldn't use it in scenarios where the
>> requirement is a simple non-encrypted installation without any plans to
>> resize partitions.
>>
>>
>>> Of course, in the Windows-world, that approach is standard
>>> and it has been creeping into Linux for a while now (see,
>>> e.g. systemd, LVM, udev, etc.). This is probably due to people
>>> comming into the Linux community that never understood what
>>> the problem with the Windows-approach is.
>> I've done my fair share of Linux kernel development, firmware
>> development, reverse engineering and other fun stuff, and use Linux
>> exclusively since around 2000. In 1998, I thought people not using the
>> command line on Linux were heretics. I have changed since then. With the
>> old MS Windows versions, it was impossible to fix things which were not
>> supported by graphical tools, and I loved having the chance to fix (and
>> break) anything on the Linux command line. Back then, I thought
>> attracting all the new users with "bloatware" like desktop environments
>> was a bad idea. It took me a while to learn that this stuff was useful
>> after all. Nowadays I'm not afraid of people who want/use a GUI because
>> the GUIs have matured to a point where I'd even call them mostly failsafe.
>> I'm fine with LVM because it's a choice, and I use it on most of my
>> systems because it's useful for me. I haven't seen zealots trying to get
>> MBR support removed from Linux distros.
>> udev made my life easier in some cases, and back then you could use it
>> standalone.
>> systemd and git are useful for some people, but I've lost more time to
>> each of then than I can ever recover. I could accept that (sunk cost),
>> were it not for the zealots who will flame anyone questioning the
>> superiority of their sacred tool. The flaming also serves as a pretty
>> effective way of killing the motivation of anybody working on competing
>> solutions. The "you're holding it wrong" mentality of many of those
>> zealots isn't making them many friends either.
>>
>>
>>> Sorry for the rant, I just ran into a problem with udev 
>>> (again) an hour ago that makes me want to rip this whole
>>> crappy "automess" stuff out.
>> I completely agree that automation can mess things up a lot. I've seen
>> my forensics students desperately trying to disable the automounter (it
>> feels like any given distro has a dozen of them, interacting in weird
>> ways), and you can pretty much rely on any documentation to be either
>> non-existent, wrong or simply outdated.
>>
>> That said, I strongly advocate that people should stick as closely as
>> possible to what their distro has as default because that's where other
>> more experienced people can easily help.
>>
>> Regards,
>> Carl-Daniel

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Arno Wagner]

On Mon, Jun 19, 2017 at 21:02:24 CEST, Carl-Daniel Hailfinger wrote:
> On 19.06.2017 01:01, Arno Wagner wrote:
> 
> My apologies. I thought you value such credentials greatly because you
> list yours in your email signature of every email. Thank you for your
> insight about "the usual case". Does it apply to you as well?

I actually have a few more. What I list is to shut up people that
try to use theirs offensively and it is in the _signature_ (i.e. 
attached automatically), not the text. Most people do get the hint.

Regards,
Arno

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Arno Wagner]

Hi Julio,

the partitions do not show up on reboot as the kernel cannot
see them. The partition table it would need to scan is
in the encrypted LUKS container and as that is still
closed on boot, hence the kernel has no chance to look at it.

A fix might be to just call "partprobe" after opening the
LUKS container. That scans all reachable devices for
partition tables. It also works directly after repartitioning,
i.e. without a reboot.

partptobe is part of the parted package (at least on Debian).

Regards,
Arno


On Sun, Jun 18, 2017 at 08:49:33 CEST, Julio Gago wrote:
> Hi there!
> 
> I'm facing an issue I've been unable to solve by myself. I've been exploring around a lot without success. I'm a newbie in this topic so please accept my apologies if this is a silly question.
> 
> I am using cryptsetup 1.6.6 in Ubuntu 16.04 on kernel 4.8.0.
> 
> I successfully encrypted a disk with LUKS and then created a couple of partitions and mounted them with:
> 
> cryptsetup luksFormat --cipher aes-xts /dev/sdc
> cryptsetup luksOpen /dev/sdc sdc
> fdisk /dev/mapper/sdc
> (created partitions manually)
> mkfs.ext4 /dev/mapper/sdc1
> mkfs.ext4 /dev/mapper/sdc2
> mount /dev/mapper/sdc1 /part1_dmcrypt
> mount /dev/mapper/sdc2 /part2_dmcrypt
> 
> fdisk gave the usual complaint about updating partition list to kernel, which I ignored and assumed it would be fixed by the next reboot. But it didn't!
> 
> After reboot, I can open the luks volume normally:
> 
> cryptsetup luksOpen /dev/sdc sdc
> cryptsetup status sdc
> /dev/mapper/sdc is active and is in use.
>   type:    LUKS1
>   cipher:  aes-xts-plain64
>   keysize: 256 bits
>   device:  /dev/sdc
>   offset:  4096 sectors
>   size:    11719929856 sectors
>   mode:    read/write
> 
> And I can see the partitions if I use fdisk:
> 
> Disk /dev/mapper/sdc: 5.5 TiB, 6000604086272 bytes, 11719929856 sectors
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> Disklabel type: gpt
> Disk identifier: 2F089102-C3CE-4C64-BA09-A19FCC49CFF8
> 
> Device                     Start         End    Sectors  Size Type
> /dev/mapper/sdc-part1       2048  6442452991 6442450944    3T Linux filesystem
> /dev/mapper/sdc-part2 6442452992 11719929822 5277476831  2.5T Linux filesystem
> 
> However, the kernel does not seem to see the partitions, since the block devices are not present in /dev:
> 
> ls -la /dev/mapper
> total 0
> drwxr-xr-x  2 root root      80 Jun 16 19:07 .
> drwxr-xr-x 19 root root    4560 Jun 16 18:38 ..
> crw-------  1 root root 10, 236 Jun 16 18:35 control
> lrwxrwxrwx  1 root root       7 Jun 17 12:15 sdc -> ../dm-0
> 
> So I cannot mount the volumes normally. I tried partx and other methods to tell the kernel about the partitions without success. The corresponding IOCTL calls return with EINVAL error.
> 
> Decryption seems to be working alright (since I can see the partitions) and I can in fact mount the volumes doing something like this:
> 
> losetup /dev/loop0 /dev/mapper/sdc -o 1048576
> mount /dev/loop0 /part1_dmcrypt
> 
> The volumes seem to have the right contents.  So I am actually ok to
> proceed, I have backups of both volumes and the LUKS header.
> 
> However, I would like to understand what is wrong and how could I fix the
> issue and mount the volumes canonicaly.
> 
> Thanks in advance for your help and my apologies if I did anything really
> silly :)
> 
> Regards,
> Julio
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Julio Gago]

 ---- On Sun, 18 Jun 2017 17:40:54 +0200 Arno Wagner <arno@wagner.name> wrote ---- 
 > Hi Julio, 
 >  
 > the partitions do not show up on reboot as the kernel cannot 
 > see them. The partition table it would need to scan is 
 > in the encrypted LUKS container and as that is still 
 > closed on boot, hence the kernel has no chance to look at it. 
 >  
 > A fix might be to just call "partprobe" after opening the 
 > LUKS container. That scans all reachable devices for 
 > partition tables. It also works directly after repartitioning, 
 > i.e. without a reboot. 
 >  
 > partptobe is part of the parted package (at least on Debian). 

Thanks Arno!!!!

That fixed it!!!!

I was so close to finding this :). I can see in my history that I tried with "partx" (it does the same, in theory, or something similar in the realm of my understanding) and that I did "man partprobe", probably saw it in the "see also".

I am wondering if we should add a section about this in the FAQ. Or maybe this is not frequent enough?

I volunteer myself to do the update to the FAQ if you guys want.

Julio

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Arno Wagner]

On Sun, Jun 18, 2017 at 19:21:57 CEST, Julio Gago wrote:
>  ---- On Sun, 18 Jun 2017 17:40:54 +0200 Arno Wagner <arno@wagner.name> wrote ---- 
>  > Hi Julio, 
>  >  
>  > the partitions do not show up on reboot as the kernel cannot 
>  > see them. The partition table it would need to scan is 
>  > in the encrypted LUKS container and as that is still 
>  > closed on boot, hence the kernel has no chance to look at it. 
>  >  
>  > A fix might be to just call "partprobe" after opening the 
>  > LUKS container. That scans all reachable devices for 
>  > partition tables. It also works directly after repartitioning, 
>  > i.e. without a reboot. 
>  >  
>  > partptobe is part of the parted package (at least on Debian). 
> 
> Thanks Arno!!!!
> 
> That fixed it!!!!
> 
> I was so close to finding this :).  I can see in my history that I tried
> with "partx" (it does the same, in theory, or something similar in the
> realm of my understanding) and that I did "man partprobe", probably saw it
> in the "see also".
> 
> I am wondering if we should add a section about this in the FAQ.  Or maybe
> this is not frequent enough?
> 
> I volunteer myself to do the update to the FAQ if you guys want.
> 
> Julio

You are welcome.

I think I will do an  FAQ item on "Partitioning a LUKS container", 
that would also describe this effect and the need for partprobe or 
something like it.

And yes, this is rare. I believe this is the first time somebody
had this problem. Most people will use unpartitioned LUKS
containers or use LVM.

Regards,
Arno

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Julio Gago]

 ---- On Sun, 18 Jun 2017 20:03:24 +0200 Arno Wagner <arno@wagner.name> wrote ---- 
 >  
 > And yes, this is rare. I believe this is the first time somebody 
 > had this problem. Most people will use unpartitioned LUKS 
 > containers or use LVM. 

I see. As I told Michael, this looked just natural to me. I was planning to encrypt everything in the disk, so it totally made sense to encrypt once and forget about the detail. Then deal with partitions and filesystems normally. It looked beautiful to me :).

Regards,
Julio.

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Arno Wagner]

On Sun, Jun 18, 2017 at 21:13:26 CEST, Julio Gago wrote:
>  ---- On Sun, 18 Jun 2017 20:03:24 +0200 Arno Wagner <arno@wagner.name> wrote ---- 
>  >  
>  > And yes, this is rare. I believe this is the first time somebody 
>  > had this problem. Most people will use unpartitioned LUKS 
>  > containers or use LVM. 
> 
> I see.  As I told Michael, this looked just natural to me.  I was planning
> to encrypt everything in the disk, so it totally made sense to encrypt
> once and forget about the detail.  Then deal with partitions and
> filesystems normally.  It looked beautiful to me :).

I don't think it is a bad approach. It is just uncommon.

Regards,
Arno

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Michael Kjörling]

On 18 Jun 2017 17:40 +0200, from arno@wagner.name (Arno Wagner):
> A fix might be to just call "partprobe" after opening the
> LUKS container.

That is indeed a _far_ better solution than what I had in mind.

As for LVM vs non-LVM, while I don't use LVM myself except for one
system which I could wipe and reinstall right now if needed and not
lose much more than some web browser settings; LVM _does_ offer a few
niceties that are hard to get with plain partitions, and it is in
widespread use. But I absolutely agree that it is an additional
complex layer of things that can go wrong.

As a famous scientist once allegedly said: always make things as
simple as possible, but never simpler.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Arno Wagner]

On Sun, Jun 18, 2017 at 20:45:17 CEST, Michael Kjörling wrote:
> On 18 Jun 2017 17:40 +0200, from arno@wagner.name (Arno Wagner):
> > A fix might be to just call "partprobe" after opening the
> > LUKS container.
> 
> That is indeed a _far_ better solution than what I had in mind.
> 
> As for LVM vs non-LVM, while I don't use LVM myself except for one
> system which I could wipe and reinstall right now if needed and not
> lose much more than some web browser settings; LVM _does_ offer a few
> niceties that are hard to get with plain partitions, and it is in
> widespread use. But I absolutely agree that it is an additional
> complex layer of things that can go wrong.

I agree to that. For a "throw-away" installtion that you just
wipe if things break, automated use of LVM is perfectly fine. 
For a complex installation that is carefully crafted due to 
special needs and with real understanding of the LVM details, 
it is perfectly fine too. 

Just in the middle, I do not think it has a place and I do think 
it is used far too often without a real need and increasing 
complexity without any real benefit. It is like it is used 
to get a longer "feature list" to sell.
 
> As a famous scientist once allegedly said: always make things as
> simple as possible, but never simpler.

And that is the trick. It is not easy, becuse making things simpler
as possible is about as bad as making them too complex and it
is pretty easy to mess up in that direction as well (and later
find out that you did shoot yourself in the foot...). 

As some other smart guy wrote: "There is no silver bullet". 
Every situation is difficult and there is no replacement for
understanding what is going on. One reason I like the discussions
here: People comming at it from different directions and backgrounds
and you get to compare different approaches.

Regards,
Arno
-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Sven Eschenberg]

Hi Arno,

as an addition, kpartx (not partx) should be able to do that aswell.
It will however not tell the kernel about partitions on a device, but 
rather setup dm mappings for each partition.

Just wanted to add this in. And I am somehwat surprised that partprobe 
actually works this way. That makes me wonder, why the kernel does 
detect partitions on physical disks, but not on other block devices (be 
it the current case, or NBDs, iSCSI BDs or whatever)? Does the kernel 
simply decide to only scan 'real' block devices by itself. Because it 
does so even after boot, when a disk comes up by hotplugging etc. .

Regards

-Sven

Am 18.06.2017 um 17:40 schrieb Arno Wagner:
> Hi Julio,
> 
> the partitions do not show up on reboot as the kernel cannot
> see them. The partition table it would need to scan is
> in the encrypted LUKS container and as that is still
> closed on boot, hence the kernel has no chance to look at it.
> 
> A fix might be to just call "partprobe" after opening the
> LUKS container. That scans all reachable devices for
> partition tables. It also works directly after repartitioning,
> i.e. without a reboot.
> 
> partptobe is part of the parted package (at least on Debian).
> 
> Regards,
> Arno
> 
> 
> On Sun, Jun 18, 2017 at 08:49:33 CEST, Julio Gago wrote:
>> Hi there!
>>
>> I'm facing an issue I've been unable to solve by myself. I've been exploring around a lot without success. I'm a newbie in this topic so please accept my apologies if this is a silly question.
>>
>> I am using cryptsetup 1.6.6 in Ubuntu 16.04 on kernel 4.8.0.
>>
>> I successfully encrypted a disk with LUKS and then created a couple of partitions and mounted them with:
>>
>> cryptsetup luksFormat --cipher aes-xts /dev/sdc
>> cryptsetup luksOpen /dev/sdc sdc
>> fdisk /dev/mapper/sdc
>> (created partitions manually)
>> mkfs.ext4 /dev/mapper/sdc1
>> mkfs.ext4 /dev/mapper/sdc2
>> mount /dev/mapper/sdc1 /part1_dmcrypt
>> mount /dev/mapper/sdc2 /part2_dmcrypt
>>
>> fdisk gave the usual complaint about updating partition list to kernel, which I ignored and assumed it would be fixed by the next reboot. But it didn't!
>>
>> After reboot, I can open the luks volume normally:
>>
>> cryptsetup luksOpen /dev/sdc sdc
>> cryptsetup status sdc
>> /dev/mapper/sdc is active and is in use.
>>    type:    LUKS1
>>    cipher:  aes-xts-plain64
>>    keysize: 256 bits
>>    device:  /dev/sdc
>>    offset:  4096 sectors
>>    size:    11719929856 sectors
>>    mode:    read/write
>>
>> And I can see the partitions if I use fdisk:
>>
>> Disk /dev/mapper/sdc: 5.5 TiB, 6000604086272 bytes, 11719929856 sectors
>> Units: sectors of 1 * 512 = 512 bytes
>> Sector size (logical/physical): 512 bytes / 4096 bytes
>> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> Disklabel type: gpt
>> Disk identifier: 2F089102-C3CE-4C64-BA09-A19FCC49CFF8
>>
>> Device                     Start         End    Sectors  Size Type
>> /dev/mapper/sdc-part1       2048  6442452991 6442450944    3T Linux filesystem
>> /dev/mapper/sdc-part2 6442452992 11719929822 5277476831  2.5T Linux filesystem
>>
>> However, the kernel does not seem to see the partitions, since the block devices are not present in /dev:
>>
>> ls -la /dev/mapper
>> total 0
>> drwxr-xr-x  2 root root      80 Jun 16 19:07 .
>> drwxr-xr-x 19 root root    4560 Jun 16 18:38 ..
>> crw-------  1 root root 10, 236 Jun 16 18:35 control
>> lrwxrwxrwx  1 root root       7 Jun 17 12:15 sdc -> ../dm-0
>>
>> So I cannot mount the volumes normally. I tried partx and other methods to tell the kernel about the partitions without success. The corresponding IOCTL calls return with EINVAL error.
>>
>> Decryption seems to be working alright (since I can see the partitions) and I can in fact mount the volumes doing something like this:
>>
>> losetup /dev/loop0 /dev/mapper/sdc -o 1048576
>> mount /dev/loop0 /part1_dmcrypt
>>
>> The volumes seem to have the right contents.  So I am actually ok to
>> proceed, I have backups of both volumes and the LUKS header.
>>
>> However, I would like to understand what is wrong and how could I fix the
>> issue and mount the volumes canonicaly.
>>
>> Thanks in advance for your help and my apologies if I did anything really
>> silly :)
>>
>> Regards,
>> Julio
>>
>> _______________________________________________
>> dm-crypt mailing list
>> dm-crypt@saout.de
>> http://www.saout.de/mailman/listinfo/dm-crypt
> 

Re: [dm-crypt] help mounting partitions in an encrypted disk after first reboot

[Arno Wagner]

Hi Sven,

On Tue, Jun 20, 2017 at 00:04:36 CEST, Sven Eschenberg wrote:
> Hi Arno,
> 
> as an addition, kpartx (not partx) should be able to do that aswell.  It
> will however not tell the kernel about partitions on a device, but rather
> setup dm mappings for each partition.

Interesting. That may be useful in some scenarios, exspecially
for experiments and temporary mounts. Seems also to be used
by some (most?) life-CDs for the disks found.

> Just wanted to add this in.  And I am somehwat surprised that partprobe
> actually works this way.  That makes me wonder, why the kernel does detect
> partitions on physical disks, but not on other block devices (be it the
> current case, or NBDs, iSCSI BDs or whatever)?  Does the kernel simply
> decide to only scan 'real' block devices by itself.  Because it does so
> even after boot, when a disk comes up by hotplugging etc.  .

I think the kernel does not do any partition detection except
on boot. When you hotplug a disk, udev may be what actually 
does the partition detection on it, probably using a similar
mechanism as partprobe or kpartx.

I may be off here, but so far partprobe has worked well for me,
due care stipulated (i.e. do not start to change mounted or
mapped partitions and the like).

Regards,
Arno

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier