[dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Christoph Anton Mitterer]

Hi dm-crypt experts.

I've posted a question
(http://thread.gmane.org/gmane.linux.kernel/1003210) on linux-kernel
regarding stacked block layers (including dm-crypt) and if you didn't
already read it, I'd like to draw your attention on it.
Perhaps you can comment :)


Thanks,
Chris.

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Arno Wagner]

As far as I know, the "remount,ro" does a complete flush, just
as an umount does. So, yes, the fs-layer "umount"/"remount,ro" 
should make sure everything is on disk when it returns. 

ON addition, any meta-information (RAID superblock, LVM superblock, 
LUKS header) should be written to disk immediately after a change.

Arno

On Sat, Jun 26, 2010 at 01:59:40PM +0200, Christoph Anton Mitterer wrote:
> Hi dm-crypt experts.
> 
> I've posted a question
> (http://thread.gmane.org/gmane.linux.kernel/1003210) on linux-kernel
> regarding stacked block layers (including dm-crypt) and if you didn't
> already read it, I'd like to draw your attention on it.
> Perhaps you can comment :)
> 
> 
> Thanks,
> Chris.
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Christoph Anton Mitterer]

On Sat, 2010-06-26 at 14:52 +0200, Arno Wagner wrote:
> As far as I know, the "remount,ro" does a complete flush, just
> as an umount does.
But that doesn't answer, whether the flush propagates through all
layers... it could be that one layer has to wait (e.g. lvm, when using
clustering)...

And I'm not even sure if flushing alone is enough,... or where one needs
something like barriers...

Cheers,
Chris.

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Arno Wagner]

On Sat, Jun 26, 2010 at 04:21:52PM +0200, Christoph Anton Mitterer wrote:
> On Sat, 2010-06-26 at 14:52 +0200, Arno Wagner wrote:
> > As far as I know, the "remount,ro" does a complete flush, just
> > as an umount does.
> But that doesn't answer, whether the flush propagates through all
> layers... it could be that one layer has to wait (e.g. lvm, when using
> clustering)...

Umount does propagate. If it ever does not, the disk layer will
be basically unusable. Until then, don't worry about it.
 
> And I'm not even sure if flushing alone is enough,... or where one needs
> something like barriers...

Barriesrs are a filesystem concept and play no role here.

Flushing ("fdatasync" or "fsunc") is not enough in some cases,
hence the ro-remount, which is.

Arno

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Milan Broz]

On 06/26/2010 08:36 PM, Arno Wagner wrote:
> On Sat, Jun 26, 2010 at 04:21:52PM +0200, Christoph Anton Mitterer wrote:

>> And I'm not even sure if flushing alone is enough,... or where one needs
>> something like barriers...
>
> Barriesrs are a filesystem concept and play no role here.

Nope, "IO barriers" here are block level functionality.

See Documentation/block/barrier.txt  in kernel
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob_plain;f=Documentation/block/barrier.txt;hb=HEAD

(yes, it is confusing:-)

> Flushing ("fdatasync" or "fsunc") is not enough in some cases,
> hence the ro-remount, which is.

fsycn currently in kerenl issues barrier in block layer and then waits for it.

But if because ever reads can cause some metadata updates (last access on file stamp),
remounting read-only is there.

Milan

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Christoph Anton Mitterer]

On Sat, 2010-06-26 at 21:24 +0200, Milan Broz wrote:
> fsycn currently in kerenl issues barrier in block layer and then waits for it.
> 
> But if because ever reads can cause some metadata updates (last access on file stamp),
> remounting read-only is there.
Just hope that init-systems like sysvinit are really guaranteed to wait
for each of their scripts (and therefore for the blocked umount).
Otherwise they could kill it,.. and while the umount would be still
there and blocked,... it could go on to the next (halt/reboot).

:/


Cheers,
Chris.

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Arno Wagner]

On Sun, Jun 27, 2010 at 01:13:01AM +0200, Christoph Anton Mitterer wrote:
> On Sat, 2010-06-26 at 21:24 +0200, Milan Broz wrote:
> > fsycn currently in kerenl issues barrier in block layer and then waits for it.
> > 
> > But if because ever reads can cause some metadata updates (last access on file stamp),
> > remounting read-only is there.
> Just hope that init-systems like sysvinit are really guaranteed to wait
> for each of their scripts (and therefore for the blocked umount).
> Otherwise they could kill it,.. and while the umount would be still
> there and blocked,... it could go on to the next (halt/reboot).
> 
> :/

Hmm. You know, encrypted root is a problem and pretty difficult
to do in the rfirt place. Why not just encrypt the critical 
parts, like /var /home /root? The rest only holds binaries
and config files anyways, which are not that sensitive...

However, from my experience you cannot actually kill -9
umount/remoute-ro (had a problem with some drive...), 
but either have to wait it out or do a physical reset or
power cycle.

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[markus reichelt]

[-- Attachment #1: Type: text/plain, Size: 375 bytes --]

* Arno Wagner <arno@wagner.name> wrote:

> Hmm. You know, encrypted root is a problem and pretty difficult to
> do in the rfirt place. Why not just encrypt the critical parts,
> like /var /home /root? The rest only holds binaries and config
> files anyways, which are not that sensitive...

Are you serious?

btw, rfirt? -v please.


-- 
left blank, right bald

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Milan Broz]

On 06/27/2010 02:20 AM, markus reichelt wrote:
> * Arno Wagner<arno@wagner.name>  wrote:
>
>> Hmm. You know, encrypted root is a problem and pretty difficult to
>> do in the rfirt place. Why not just encrypt the critical parts,
>> like /var /home /root? The rest only holds binaries and config
>> files anyways, which are not that sensitive...
>
> Are you serious?

Usually encrypting everything is better, otherwise we add many problems here.

Just to randomly pick two of them:

- User must think and know which data are sensitive and avoid to copy them
to unencrypted space. It can happen even without his knowledge
- temporary file somewhere, coredump, whatever.  

- using "social engineering"
how many people will set the same password to disk encryption and
his account? If I have /etc/shadow visible, why I should bother
with attacking disc encryption with all its barriers?
I'll run dictionary search for passwords there, pretty good tools
already here.

...


I think that for laptop, encrypting everything is better. And I expect
that after clean shutdown my machine is safe.
All used tools currently providing methods how to do it properly
(I mean dm-crypt/LUKS, loop-aes or Truecrypt).

It is just about properly written init/shutdown scripts. I do not think it is
so complicated to fix it - just reverse initramfs root-fs mapping.

Several similar parts of problems "cutting own throat" are there
(like pvmove on root-fs in LVM, multipath solving the situation when all paths
to underlying device are temporarily gone).
This is nothing completely new.

(And yes, I a quite intentionally hijacked this thread to focus on this shutdown
& encrypted root-fs problem, sorry:-)

Milan

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Christoph Anton Mitterer]

On Sun, 2010-06-27 at 10:35 +0200, Milan Broz wrote:
> (And yes, I a quite intentionally hijacked this thread to focus on this shutdown
> & encrypted root-fs problem, sorry:-)
I guess we're already all on some watch-list of NSA and friends,... so
hijacking email threads is probably your least problem ;P

Cheers,
Chris.

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[markus reichelt]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]

* Milan Broz <mbroz@redhat.com> wrote:

> On 06/27/2010 02:20 AM, markus reichelt wrote:
> >* Arno Wagner<arno@wagner.name>  wrote:
> >
> >>Hmm. You know, encrypted root is a problem and pretty difficult
> >>to do in the rfirt place. Why not just encrypt the critical
> >>parts, like /var /home /root? The rest only holds binaries and
> >>config files anyways, which are not that sensitive...
> >
> >Are you serious?
> 
> Usually encrypting everything is better, otherwise we add many
> problems here.

I phrased the question badly. My point is: Leaving /etc in plain but
encrypting /home (at least) leaves you wide wide open these days to
identity theft; wpa_supplicant and related foo, openvpn, ssh host
keys, ... just to name a few. That's what I was after.

-- 
left blank, right bald

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Christoph Anton Mitterer]

On Fri, 2010-07-02 at 20:48 +0200, markus reichelt wrote:
> I phrased the question badly. My point is: Leaving /etc in plain but
> encrypting /home (at least) leaves you wide wide open these days to
> identity theft; wpa_supplicant and related foo, openvpn, ssh host
> keys, ... just to name a few. That's what I was after.
Well,... leaving anything unencrypted,... makes you more vulnerable
against attacks when people have physical access to your system.
(And yes of course, their are always weaker elements in the chain,..
keyloggers, etc).


Chris.

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Christoph Anton Mitterer]

On Sun, 2010-06-27 at 01:34 +0200, Arno Wagner wrote:
> Hmm. You know, encrypted root is a problem and pretty difficult
> to do in the rfirt place. Why not just encrypt the critical 
> parts, like /var /home /root? The rest only holds binaries
> and config files anyways, which are not that sensitive...
They're actually very sensitive, against compromise "when I'm not there"
and the device is e.g. shut down (or even running).
An attacker with access to my device could easily add e.g. a rootkit
when I'm not there, which just waits until I once decrypt the "important
stuff" and sends the key/data back home.

dm-crypte largely protects you from this. Even if it doesn't give you
mathematical integrity/authenticity, it's still very difficult for an
attacker to do reasonable attacks (other then destroying your data)
because he neither know where to change, nor to which value.



Cheers,
Chris.

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Arno Wagner]

On Sun, Jun 27, 2010 at 04:28:35AM +0200, Christoph Anton Mitterer wrote:
> On Sun, 2010-06-27 at 01:34 +0200, Arno Wagner wrote:
> > Hmm. You know, encrypted root is a problem and pretty difficult
> > to do in the rfirt place. Why not just encrypt the critical 
> > parts, like /var /home /root? The rest only holds binaries
> > and config files anyways, which are not that sensitive...
> They're actually very sensitive, against compromise "when I'm not there"
> and the device is e.g. shut down (or even running).

For running, I recomend looking up physical memory freezing.
It allows you to cool down the memory, pull the pwer plug and
read the complete memeory contents up to a few minutes later on 
an external device. The keys are in there.

For not running, there are numerous ways to still attack the system.

> An attacker with access to my device could easily add e.g. a rootkit
> when I'm not there, which just waits until I once decrypt the "important
> stuff" and sends the key/data back home.

The current consensus in much of the security community is that
if an attacker has that level of physical access, you are screwed
anyways.  

> dm-crypte largely protects you from this. 

Only against very low-powered attackers. Against these I reccomend
a better lock on the door.

> Even if it doesn't give you
> mathematical integrity/authenticity, it's still very difficult for an
> attacker to do reasonable attacks (other then destroying your data)
> because he neither know where to change, nor to which value.

There are all kinds of possibilities to install keyloggers and other 
malicious software. Your kernel, for example, cannot be encrypted.
Keyloggers in all sizes and shapes, including inside your keyboard
can be installed. Other things can be done. 

Face it, you are using the wrong tool if protection against 
manipulation with physical access is your goal. For that 
I would recommend a safe that is intended to have a PC running
inside it. Not too cheap, but tamper-obvious. 

Arno

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Christoph Anton Mitterer]

On Sun, 2010-06-27 at 04:53 +0200, Arno Wagner wrote:
> For running, I recomend looking up physical memory freezing.
> It allows you to cool down the memory, pull the pwer plug and
> read the complete memeory contents up to a few minutes later on 
> an external device. The keys are in there.
Yeah of course,.. I know...
People have to keep this in mind when that want that level of security,
and never leave their PCs alone too long when they're on (or suspended
to RAM).

I for example, always power it off, when I leave my faculty office for
longer time (e.g. during lunch)... (all my colleagues are evil ;) *G* )


> For not running, there are numerous ways to still attack the system.
Including, "inviting" me to Guantanmo an beat me so long until I give
them the passphrases with pleasure ;)

Seriously,... of course there are still ways (which do you think of?)...
but this should not mean, that we do not harden everything as much as we
can..


> The current consensus in much of the security community is that
> if an attacker has that level of physical access, you are screwed
> anyways.  
Uhm.... must secure my office with some anti-person-mines ;)


> Your kernel, for example, cannot be encrypted.
But I can take that one always with me.


> Keyloggers in all sizes and shapes, including inside your keyboard
> can be installed. Other things can be done. 
Of course,... and mighty attackers could replace your CPU with one that
looks like normal, but that hast a "root-kid" and sends everything via
radio signals...
But again my argument,... we could then drop most of our security
efforts, as there are always much weaker parts in the chain.


Cheers,
Chris.

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Arno Wagner]

On Sat, Jun 26, 2010 at 09:24:27PM +0200, Milan Broz wrote:
> On 06/26/2010 08:36 PM, Arno Wagner wrote:
>> On Sat, Jun 26, 2010 at 04:21:52PM +0200, Christoph Anton Mitterer wrote:
>
>>> And I'm not even sure if flushing alone is enough,... or where one needs
>>> something like barriers...
>>
>> Barriesrs are a filesystem concept and play no role here.
>
> Nope, "IO barriers" here are block level functionality.
>
> See Documentation/block/barrier.txt  in kernel
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob_plain;f=Documentation/block/barrier.txt;hb=HEAD
>
> (yes, it is confusing:-)

Ah, I see. Must have confused it with something. Thanks and
sorry for my misstatement.

>> Flushing ("fdatasync" or "fsunc") is not enough in some cases,
>> hence the ro-remount, which is.
>
> fsycn currently in kerenl issues barrier in block layer and 
> then waits for it.

Good. My information here is admittedly pretty old.  

> But if because ever reads can cause some metadata updates (last 
> access on file stamp), remounting read-only is there.

Ok.Thanks again.

Arno

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Christoph Anton Mitterer]

FYI

I've put up a wiki site which tries to describe and discuss this and
related issues.

It's on the Debian wiki, but I guess (as most of them are probably
inter-distro-issues ^^) all people are happiliy invited to join
discussion :) .

I also tried to separate the generic discussion from every Debian
related.


Best wishes,
Chris.

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Christoph Anton Mitterer]

Argl,... forgot to include the URI ;)

http://wiki.debian.org/AdvancedStartupShutdownWithMultilayeredBlockDevices

Cheers,
Chris.

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Arno Wagner]

Incidentially, I just found an article that talks about the
state of barriers in the last few kernels. Seems they are
only fully implemented as of 2.6.33 (md) and 2.6.29 (LVNM). 
The umount/rmount should still work reliably on older kernels.
See http://www.linux-mag.com/id/7773

Do you have a link to the wiki page?

Arno


On Sun, Jun 27, 2010 at 04:31:40AM +0200, Christoph Anton Mitterer wrote:
> FYI
> 
> I've put up a wiki site which tries to describe and discuss this and
> related issues.
> 
> It's on the Debian wiki, but I guess (as most of them are probably
> inter-distro-issues ^^) all people are happiliy invited to join
> discussion :) .
> 
> I also tried to separate the generic discussion from every Debian
> related.
> 
> 
> Best wishes,
> Chris.
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] FYI: how to (really) cleanly shutdown the system when root is on multiple stacked block devices

[Christoph Anton Mitterer]

On Sun, 2010-06-27 at 04:56 +0200, Arno Wagner wrote:
> Incidentially, I just found an article that talks about the
> state of barriers in the last few kernels. Seems they are
> only fully implemented as of 2.6.33 (md) and 2.6.29 (LVNM). 
> The umount/rmount should still work reliably on older kernels.
> See http://www.linux-mag.com/id/7773
Ah nice :) thanks!
I'll put it on my to-read-list ;)


> Do you have a link to the wiki page?
Have you got it in the meantime? :)
http://www.saout.de/pipermail/dm-crypt/2010-June/000906.html


Cheers,
Chris.