[dm-crypt] [RFC] dm-crypt and hardware-optimized crypto modules

[dm-crypt] [RFC] dm-crypt and hardware-optimized crypto modules

[Jonas Meurer]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,

In the Debian bugreport #639832 [1], Simon Mackinlay pointed out, that
hardware-optimized crypto driver modules aren't loaded automatically
at cryptsetup invokation in the boot process (initramfs) in Debian.

I verified this. At least for setups with aes support compiled into
the kernel, and hardware-optimized aes drivers (aes-x86_64,
aesni-intel) built as modules (which is the default for Debian and
Ubuntu kernels), the hardware-optimized aes modules aren't loaded at
cryptsetup invokation. (Sure, this is tested with aes-encrypted
volumes.) I didn't have time to check other setups (e.g. everything
built as modules) yet.

Is this behaviour intended, or should the kernel select
hardware-optimized drivers by default in case they're available (even
as modules) and supported by hardware?

I'm happy to extend the initramfs scripts to load hardware-optimized
modules in case they're available before cryptsetup is invoked. But
that an implementation would be ugly and hard to maintain as it needs
to be updated for possible kernel crypto driver changes. I would
prefer a solution where the kernel crypto api took responsibility for
this task.

Greetings,
 jonas

[1] http://bugs.debian.org/639832
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJOpKOwAAoJEFJi5/9JEEn+NEMP/RYIryt4pw/4ROxHUIZVJEi7
gsPNzLbqCjwQ89uRsbN6BHrwrLeR7/s8xNUU2e4ppFj34069PI//0/KkY5BwnrQt
LTxg9yiyWeQc0ajVfxTClq2ZtgL3pcN6FclXKTE+ffjYMxbfBbM5AT1ATAkI4AOy
A3KhiT3KzG0sXs0P1o+sP+/JB//QBj6uJcIfMy2lbI0yqhl39rs5U97eueSImwEw
6ZuRVm4MD4zXQVfufAeEAHrXcPJAUd4DQyINQcFv+Ar75bSVtqsyzxsfaJXABOjZ
E+o06Zfs0i4rCfH5S08Wsqdcywua69NSnsmYJZlaOgA16uxVqcLvjw9M8DYFnBWT
clUuEXIquqQInX/bEgtq6eLgJR/BHSxv4GRJ07fVeQMae5xYPNelnEaiov6ikRs2
jNODQ2EHr3vgXsR4k6VDN/ffXybtUkndulk42vEi+a5u9eNBXYLNe1tYwmX/Ylyx
qQmAbKZ0KS3jbBFKU8rUJxlL4BetYpH1+EUDmRAIX99y4SjxdV2/SG5WZHK+scOF
cf/durQK13jGu5SGT7cSodmnFo6lgdvFUm1F+fTz4Kou0Jlgl031byG0aX6cETQD
oTMyr1z2ToUKD0hI6HMdOXYQLydlnE3U94OwV7ph+CQsvcv02qCrKBTX6zUuZNeL
cqYw/o7Yao9lVKmeCNkk
=86Vz
-----END PGP SIGNATURE-----

Re: [dm-crypt] [RFC] dm-crypt and hardware-optimized crypto modules

[Arno Wagner]

Hi Jonas,

the definite authority on this is Milan, but as far as I
understand module autoloading, as long as an implementation 
for a requested cipher is already loaded, that will be used.
Now, I expect it would be possible to not build the normal
AES module and thereby have the HW-supported AES module loade
automatically when needed. As the Debian distro-kernel cannot
know HW-support would be there, it obviously defaults to the
software implementation.

AFAIK, if both HW and SW support are loaded, HW support
is used as default. I think there is some kind of priority 
system in place. But I am really only guessing here.

I see two ways around this: 

1. Load the HW module manually (or scripted). 
   While I have not used a Debian Distro kernel for a long
   time, I think adding the HW-module to /etc/modules
   should accomplish that. Noneed to mess with the initrd,
   unless possibly if you have encrypted root.

2. Roll your own kernel, possibly with HW support statically
   compiled in. I have used Debian with kernels from kernel.org
   and module-support turned off with good success for about
   10 years now. (I don't like initrds. Good for distros, but 
   they complicate things and complexity is the enemy of reliablity
   and efficiency. Also, I like to mess around with my installatons
   and initrds make that harder. I also do not like to use kernel 
   modules very much, although it is definitely good that they 
   are there.)

   To use your own kernel with Debian, just boot it and tell it
   the root partition. Of course you have to make sure it somehow 
   has the drivers it needs to fnd and mount the root partition.
   

Arno


On Mon, Oct 24, 2011 at 01:30:56AM +0200, Jonas Meurer wrote:
> Hello,
> 
> In the Debian bugreport #639832 [1], Simon Mackinlay pointed out, that
> hardware-optimized crypto driver modules aren't loaded automatically
> at cryptsetup invokation in the boot process (initramfs) in Debian.
> 
> I verified this. At least for setups with aes support compiled into
> the kernel, and hardware-optimized aes drivers (aes-x86_64,
> aesni-intel) built as modules (which is the default for Debian and
> Ubuntu kernels), the hardware-optimized aes modules aren't loaded at
> cryptsetup invokation. (Sure, this is tested with aes-encrypted
> volumes.) I didn't have time to check other setups (e.g. everything
> built as modules) yet.
> 
> Is this behaviour intended, or should the kernel select
> hardware-optimized drivers by default in case they're available (even
> as modules) and supported by hardware?
> 
> I'm happy to extend the initramfs scripts to load hardware-optimized
> modules in case they're available before cryptsetup is invoked. But
> that an implementation would be ugly and hard to maintain as it needs
> to be updated for possible kernel crypto driver changes. I would
> prefer a solution where the kernel crypto api took responsibility for
> this task.
> 
> Greetings,
>  jonas
> 
> [1] http://bugs.debian.org/639832
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] [RFC] dm-crypt and hardware-optimized crypto modules

[Milan Broz]

On 10/24/2011 01:30 AM, Jonas Meurer wrote:

> In the Debian bugreport #639832 [1], Simon Mackinlay pointed out, that
> hardware-optimized crypto driver modules aren't loaded automatically
> at cryptsetup invokation in the boot process (initramfs) in Debian.
> 
> I verified this. At least for setups with aes support compiled into
> the kernel, and hardware-optimized aes drivers (aes-x86_64,
> aesni-intel) built as modules (which is the default for Debian and
> Ubuntu kernels), the hardware-optimized aes modules aren't loaded at
> cryptsetup invokation. (Sure, this is tested with aes-encrypted
> volumes.) I didn't have time to check other setups (e.g. everything
> built as modules) yet.

If the modules are present at this time (either compiled-in or as separate
modules) this seems to be kernel cryptoAPI bug.

If it is not present (in intramfs) then available module is used and later
it is not replaced by hw accelerated driver.

Anyway, I am using aesni_intel loaded from Debian initramfs and it works
with no hacks. Wonder what is the difference...
(kernel 3.0.3 but compiled with own config to own kernel deb package.)

> I'm happy to extend the initramfs scripts to load hardware-optimized
> modules in case they're available before cryptsetup is invoked. But
> that an implementation would be ugly and hard to maintain as it needs
> to be updated for possible kernel crypto driver changes. I would
> prefer a solution where the kernel crypto api took responsibility for
> this task.

I think it should load modules automatically according to its priorities
(hw has always higher priority). Anyway, this is the question
for linux-crypto (kernel) list.

There is no way how to force dm-crypt load specific driver.

Milan

Re: [dm-crypt] [RFC] dm-crypt and hardware-optimized crypto modules

[Arno Wagner]

On Mon, Oct 24, 2011 at 08:29:57AM +0200, Milan Broz wrote:
> On 10/24/2011 01:30 AM, Jonas Meurer wrote:
[...]
> If the modules are present at this time (either compiled-in or as separate
> modules) this seems to be kernel cryptoAPI bug.

So if it is loaded by the autoloader it should try the HW module first
and only if that fails load the SW module? Makes sense.

> If it is not present (in intramfs) then available module is used and later
> it is not replaced by hw accelerated driver.
> 
> Anyway, I am using aesni_intel loaded from Debian initramfs and it works
> with no hacks. Wonder what is the difference...
> (kernel 3.0.3 but compiled with own config to own kernel deb package.)

Ah, yes, I fogot: There is an organized and documented process for 
rolling your own Debian kernel package. I have not used it though.

> > I'm happy to extend the initramfs scripts to load hardware-optimized
> > modules in case they're available before cryptsetup is invoked. But
> > that an implementation would be ugly and hard to maintain as it needs
> > to be updated for possible kernel crypto driver changes. I would
> > prefer a solution where the kernel crypto api took responsibility for
> > this task.
> 
> I think it should load modules automatically according to its priorities
> (hw has always higher priority). Anyway, this is the question
> for linux-crypto (kernel) list.
> 
> There is no way how to force dm-crypt load specific driver.

You could do a wrapper and unload the software AES module if loaded. 
That would be a pretty dirty hack though.

BTW, nice to see RH folks using Debian ;-)

Arno
 
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] [RFC] dm-crypt and hardware-optimized crypto modules

[Jonas Meurer]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hey Milan,

Am 24.10.2011 08:29, schrieb Milan Broz:
> On 10/24/2011 01:30 AM, Jonas Meurer wrote:
> 
>> In the Debian bugreport #639832 [1], Simon Mackinlay pointed out,
>> that hardware-optimized crypto driver modules aren't loaded
>> automatically at cryptsetup invokation in the boot process
>> (initramfs) in Debian.
>> 
>> I verified this. At least for setups with aes support compiled
>> into the kernel, and hardware-optimized aes drivers (aes-x86_64, 
>> aesni-intel) built as modules (which is the default for Debian
>> and Ubuntu kernels), the hardware-optimized aes modules aren't
>> loaded at cryptsetup invokation. (Sure, this is tested with
>> aes-encrypted volumes.) I didn't have time to check other setups
>> (e.g. everything built as modules) yet.
> 
> If the modules are present at this time (either compiled-in or as
> separate modules) this seems to be kernel cryptoAPI bug.

It seems like this is the case, yes. I verified that
hardware-optimized modules are present in the initramfs both in Debian
and Ubuntu. I tested the 3.0.0-12-generic kernel in Ubuntu so far,
will check other kernels and setups later.

> If it is not present (in intramfs) then available module is used
> and later it is not replaced by hw accelerated driver.

Yes, that makes a lot of sense to me. But as written above, the
hardware-optimized drivers are available as modules at the time of
cryptsetup invokation.

> Anyway, I am using aesni_intel loaded from Debian initramfs and it
> works with no hacks. Wonder what is the difference... (kernel 3.0.3
> but compiled with own config to own kernel deb package.)

Do you have crypto drivers compiled into the kernel? Or built as
modules? I guess that software drivers built into the kernel and
hardware drivers available as modules is the only setup with problems,
but didn't test it yet.

>> I'm happy to extend the initramfs scripts to load
>> hardware-optimized modules in case they're available before
>> cryptsetup is invoked. But that an implementation would be ugly
>> and hard to maintain as it needs to be updated for possible
>> kernel crypto driver changes. I would prefer a solution where the
>> kernel crypto api took responsibility for this task.
> 
> I think it should load modules automatically according to its
> priorities (hw has always higher priority). Anyway, this is the
> question for linux-crypto (kernel) list.
> 
> There is no way how to force dm-crypt load specific driver.

Yes, I see the point that this is a issue for linux-crypto, and will
move the discussion to this list as soon as I did further investigation.

Thanks for your answers!

Greetings,
 jonas
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJOpVRwAAoJEFJi5/9JEEn+LAMQAKLyIr8YZZMF2vYC/2pwN9WG
PI295FhABcdXCMuaD2GFbbW4euF7DSaknQF0uOFpxevm1wpXtlxOPFDPb6cD6YS2
9/n12quqVnfcgCsUo7cyWmZqZQylfQyuA6Xs/iamoaF7Y8SKXzLcazlNSRYHhCt9
lT03CdkTSGAR0g4Kbek8CT/lEjcjZ/DMO4OBCaPPZi9GppauW5eTu3yRvLZexZe7
xtiD2ZZoVu7YHIimMs/zbOvzi3Yo+nEPj6uQOeFkFjxHX/eMScKOcPzKX+KqvYqO
mDSMiMeDyxv5AVc8jdvgJUftbAIZ9mOPGxvIrI61v006KMHftC0NOlnlIz7xC7RG
E0XW+956sHLfDBRnfTe4dxuZYPHy4RjgwVJVBHvacSHl6IKu/jZHowadDglaF8NT
EJGdKRgnlkgAK3rb0APmBzd4WM/PY2Cew43Z5Ux1vLyH7/ZtXv6NlK6l7k6SBkoB
q4QChUlVzpLTKgZ5QCesMtyI/TVqjSHv3WEVOOwW3FLTT6riexYe6BzaHvoJUQXq
1DqmzCHhNjr6Fq5f++PuiKQSvb0MPn4dk+ZK7gXHshoNG05uSmXgTKr3l13oP9/5
XdiecNJF0eQjfSttLkc+T/LYVRlTanbyWODwlgPZaugDyDgBmUJsSyGV5xTt2w23
mZ4Rl1Au3UofuudPqf10
=Cu6i
-----END PGP SIGNATURE-----

Re: [dm-crypt] [RFC] dm-crypt and hardware-optimized crypto modules

[Jonas Meurer]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 24.10.2011 08:21, schrieb Arno Wagner:
> Hi Jonas,

Hey Arno,

> the definite authority on this is Milan, but as far as I understand
> module autoloading, as long as an implementation for a requested
> cipher is already loaded, that will be used. Now, I expect it would
> be possible to not build the normal AES module and thereby have the
> HW-supported AES module loade automatically when needed. As the
> Debian distro-kernel cannot know HW-support would be there, it
> obviously defaults to the software implementation.

Nope, the Debian distro-kernel has software implementation built into
the kernel, and hardware-accelerated drivers built as modules. So
according to Milans answers, the kernel crypto engine should load and
use the hardware-optimised drivers in case they're supported.

> AFAIK, if both HW and SW support are loaded, HW support is used as
> default. I think there is some kind of priority system in place.
> But I am really only guessing here.

I guess you're correct here ;)

> I see two ways around this:
> 
> 1. Load the HW module manually (or scripted). While I have not used
> a Debian Distro kernel for a long time, I think adding the
> HW-module to /etc/modules should accomplish that. Noneed to mess
> with the initrd, unless possibly if you have encrypted root.
> 
> 2. Roll your own kernel, possibly with HW support statically 
> compiled in. I have used Debian with kernels from kernel.org and
> module-support turned off with good success for about 10 years now.
> (I don't like initrds. Good for distros, but they complicate things
> and complexity is the enemy of reliablity and efficiency. Also, I
> like to mess around with my installatons and initrds make that
> harder. I also do not like to use kernel modules very much,
> although it is definitely good that they are there.)
> 
> To use your own kernel with Debian, just boot it and tell it the
> root partition. Of course you have to make sure it somehow has the
> drivers it needs to fnd and mount the root partition.

As I'm the maintainer of cryptsetup in Debian, I'm searching for a
solution for default setups. I know how to manually tweak setups to
use the hardware-optimized crypto drivers. But I need a solution for
the default setup with default distro-kernel. Thus building custom
kernels is out of scope in my case.

Greetings,
 jonas
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJOpVXxAAoJEFJi5/9JEEn+rAcP+gMbUNbx4/YkZzD2CEEUfJsr
QFgUth+B5znHs7YCo9ndR+uDNyiYu0/EQA9xvjvp/lz8ageynrCiawaClTfytIJW
lpA+qL89WX3gPtKIK/W8TKVor1ArWS6ZqzLZvO4avxt0bqTxfHRR7dilWgvtlWQt
AcA2VeiBwDp7JtTTyKSPgIFYaVDsWo/GhPShQ4fMMQOH0HeuyLOYUgcuP4TlKrPN
17U7AlfkMPwDc1asoMdyAev+0G+3NT8vY+0ppd+aiQygpEJgafJj+UrXjlEb8qBl
s8Byff44+FtyKVbG5q6njS6EWlTygwkVH2YJs5pSqNyJG+EALuj/Mwv8JAzefoYE
GoF1xImNJPLdWf5SfuWw8t+6pOEydtkSKIBAxvaNpTuB8T122iei+GI33RIkH8eR
q6cmdP9Kxau+Hsa6WEMB0fqjzNdekNdtzQHLKqEHjW9Fu4UekzKcV1bslU4hntvh
KA2UTCOxkmopLmWSty2dAfqgVALzSRLBJjC/V6bjWoY/vUCQDOFjexv2vIf4F4Zq
jISypLfsFytZQqOSTcd2gEBOOXEyua+D02Wq6H3SRzxqPRzxAPMFxAO1aVD/Y7lh
64cLd2bfEiDW+IgTUOQAAouOIVIWYRAsmFwEsmP+NDvLDf6b14cHs+Q2m5JY8exX
WP7+vC+GW4zyfxWEKoS8
=elNz
-----END PGP SIGNATURE-----

Re: [dm-crypt] [RFC] dm-crypt and hardware-optimized crypto modules

[Arno Wagner]

On Mon, Oct 24, 2011 at 02:11:29PM +0200, Jonas Meurer wrote:
> Am 24.10.2011 08:21, schrieb Arno Wagner:
> > Hi Jonas,
> 
> Hey Arno,
> 
> > the definite authority on this is Milan, but as far as I understand
> > module autoloading, as long as an implementation for a requested
> > cipher is already loaded, that will be used. Now, I expect it would
> > be possible to not build the normal AES module and thereby have the
> > HW-supported AES module loade automatically when needed. As the
> > Debian distro-kernel cannot know HW-support would be there, it
> > obviously defaults to the software implementation.
> 
> Nope, the Debian distro-kernel has software implementation built into
> the kernel, and hardware-accelerated drivers built as modules. So
> according to Milans answers, the kernel crypto engine should load and
> use the hardware-optimised drivers in case they're supported.

Hmm. If the software-version is already compiled-in, that could
prevent auto-loading of the hw-version. I would expect that you 
need both as modules or both compiled-in. Should be easy to test
though. 

> > AFAIK, if both HW and SW support are loaded, HW support is used as
> > default. I think there is some kind of priority system in place.
> > But I am really only guessing here.
> 
> I guess you're correct here ;)
> 
> > I see two ways around this:
> > 
> > 1. Load the HW module manually (or scripted). While I have not used
> > a Debian Distro kernel for a long time, I think adding the
> > HW-module to /etc/modules should accomplish that. Noneed to mess
> > with the initrd, unless possibly if you have encrypted root.
> > 
> > 2. Roll your own kernel, possibly with HW support statically 
> > compiled in. I have used Debian with kernels from kernel.org and
> > module-support turned off with good success for about 10 years now.
> > (I don't like initrds. Good for distros, but they complicate things
> > and complexity is the enemy of reliablity and efficiency. Also, I
> > like to mess around with my installatons and initrds make that
> > harder. I also do not like to use kernel modules very much,
> > although it is definitely good that they are there.)
> > 
> > To use your own kernel with Debian, just boot it and tell it the
> > root partition. Of course you have to make sure it somehow has the
> > drivers it needs to fnd and mount the root partition.
> 
> As I'm the maintainer of cryptsetup in Debian, I'm searching for a
> solution for default setups. 

Ah, sorry. That gives you a different perspective obviously.

> I know how to manually tweak setups to
> use the hardware-optimized crypto drivers. But I need a solution for
> the default setup with default distro-kernel. Thus building custom
> kernels is out of scope in my case.

I can see that, yes. 

Arno

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier