[dm-crypt] Recommended modes for performance (SMP+AES-NI)

[dm-crypt] Recommended modes for performance (SMP+AES-NI)

[Brad House]

We're in the process of building a new fileserver which will
be using dm-crypt, and are trying to get a game plan together
on what mode of operation will be best for a good ratio of
performance and security.

Initially the machine will be a 6-core Xeon which supports
the AES-NI instruction set, but a second identical CPU may be
dropped-in, in the future.  It will be connected to the network
by at least one 10Gbps NIC.

Obviously, we'll be making sure to use 2.6.38 or higher in
order to utilize the multi-cpu scaling enhancements to
dm-crypt:
http://kernelnewbies.org/Linux_2_6_38#head-49f5f735853f8cc7c4d89e5c266fe07316b49f4c

I think we've settled on AES-256, but may entertain AES-128
if there is a huge performance difference as I think AES-128
is still considered sufficiently safe for our purposes.

So, the question is mainly what mode of operation would be
best?
  - cbc-essiv
  - ctr-{plain64|essiv}
  - xts-{plain64|essiv}
  - are there any others I should be considering?
NOTE: I'm not sure if essiv is even an option for CTR or XTS
       modes, I'd like feedback on that, as well as what the
       security implications are...

At this point, I'm leaning towards CTR mode, mainly because it
was designed explicitly to be parallelizable:
http://en.wikipedia.org/wiki/Block_cipher_modes_of_operation#Counter_.28CTR.29

And it appears Intel has explicitly submitted a patch to optimize
dm-crypt for AES-NI with this mode of operation:
http://lwn.net/Articles/376562/

I know "test it" is going to be the obvious answer, and we will,
but I don't want to make any decisions that could severely impact
security for a little extra speed.  Well, that, and our hardware
is on order and probably won't be in for 3 weeks ;)

Any suggestions/feedback would be greatly appreciated.

Thanks!
-Brad

Re: [dm-crypt] Recommended modes for performance (SMP+AES-NI)

[Arno Wagner]

On Mon, Jun 27, 2011 at 11:38:44AM -0400, Brad House wrote:
> We're in the process of building a new fileserver which will
> be using dm-crypt, and are trying to get a game plan together
> on what mode of operation will be best for a good ratio of
> performance and security.
> 
> Initially the machine will be a 6-core Xeon which supports
> the AES-NI instruction set, but a second identical CPU may be
> dropped-in, in the future.  It will be connected to the network
> by at least one 10Gbps NIC.
> 
> Obviously, we'll be making sure to use 2.6.38 or higher in
> order to utilize the multi-cpu scaling enhancements to
> dm-crypt:
> http://kernelnewbies.org/Linux_2_6_38#head-49f5f735853f8cc7c4d89e5c266fe07316b49f4c
> 
> I think we've settled on AES-256, but may entertain AES-128
> if there is a huge performance difference as I think AES-128
> is still considered sufficiently safe for our purposes.

The performance difference should be relatively small.
 
> So, the question is mainly what mode of operation would be
> best?
>  - cbc-essiv
>  - ctr-{plain64|essiv}
>  - xts-{plain64|essiv}
>  - are there any others I should be considering?
> NOTE: I'm not sure if essiv is even an option for CTR or XTS
>       modes, I'd like feedback on that, as well as what the
>       security implications are...

ESSIV is only for CBC.

> At this point, I'm leaning towards CTR mode, mainly because it
> was designed explicitly to be parallelizable:
> http://en.wikipedia.org/wiki/Block_cipher_modes_of_operation#Counter_.28CTR.29

That is only for fine-grained paralellism, and hence not 
applicable here. I am also not sure whether you can even use it
with dm-crypt as it needs a nonce in addition to the counter.
And that needs to be stored somewhere.
 
> And it appears Intel has explicitly submitted a patch to optimize
> dm-crypt for AES-NI with this mode of operation:
> http://lwn.net/Articles/376562/
> 
> I know "test it" is going to be the obvious answer, and we will,
> but I don't want to make any decisions that could severely impact
> security for a little extra speed.  Well, that, and our hardware
> is on order and probably won't be in for 3 weeks ;)
> 
> Any suggestions/feedback would be greatly appreciated.

Unless you have any specific security requirements beyond
the standard, go with the defaults. I think you are 
overthinking this. The defaults are what is maintained best 
and also what will get the fastest fixes and problem detection.

If you have special requiremenrts, any deviation from the
defaults should have a strong justification comming from
these requirements. As you have not given any, I cannot
comment on them.

As for speed, AFAIK, basically you are still limited to
one CPU per process that accesses an encrypted disk.
But keep in mind that with the defaults you get something
like 100MB/s crypto speed in pure software. Unless this thing
has an 2.5 or 10Gb/sec interface, and SSDs or mostly 
linear, large-file accesses, that is quite fast enough
in most cases.

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] Recommended modes for performance (SMP+AES-NI)

[Brad House]

>> I think we've settled on AES-256, but may entertain AES-128
>> if there is a huge performance difference as I think AES-128
>> is still considered sufficiently safe for our purposes.
>
> The performance difference should be relatively small.


That's my thought too, especially with AES-NI.


>> So, the question is mainly what mode of operation would be
>> best?
>>   - cbc-essiv
>>   - ctr-{plain64|essiv}
>>   - xts-{plain64|essiv}
>>   - are there any others I should be considering?
>> NOTE: I'm not sure if essiv is even an option for CTR or XTS
>>        modes, I'd like feedback on that, as well as what the
>>        security implications are...
>
> ESSIV is only for CBC.


Ah, ok, that's good to know.  Thanks.


>> At this point, I'm leaning towards CTR mode, mainly because it
>> was designed explicitly to be parallelizable:
>> http://en.wikipedia.org/wiki/Block_cipher_modes_of_operation#Counter_.28CTR.29
>
> That is only for fine-grained paralellism, and hence not
> applicable here. I am also not sure whether you can even use it
> with dm-crypt as it needs a nonce in addition to the counter.
> And that needs to be stored somewhere.


Well, since Intel provided a specific CTR mode AES-NI patch and
it referenced testing it _using_ dm-crypt
(http://lwn.net/Articles/376562/), I'd assume it is possible to at
least use it with dm-crypt ;)


> Unless you have any specific security requirements beyond
> the standard, go with the defaults. I think you are
> overthinking this. The defaults are what is maintained best
> and also what will get the fastest fixes and problem detection.


No specific security requirements.  I'm not worried about
governments targeting breaking this encryption or anything
like that.   Just want to make sure the actual storage of this
data isn't the weakest link in the security chain.


> If you have special requiremenrts, any deviation from the
> defaults should have a strong justification comming from
> these requirements. As you have not given any, I cannot
> comment on them.


Sorry, I should have provided more information...

The requirements are basically if the box leaves the building,
that it be infeasible that the plaintext data be retrieved.
Not worried about leakage of statistics or even modification
of data, though if those too can be prevented for little cost,
it'd be nice to have ;)


> As for speed, AFAIK, basically you are still limited to
> one CPU per process that accesses an encrypted disk.
> But keep in mind that with the defaults you get something
> like 100MB/s crypto speed in pure software. Unless this thing
> has an 2.5 or 10Gb/sec interface, and SSDs or mostly
> linear, large-file accesses, that is quite fast enough
> in most cases.


This is a multi-user fileserver hosting remote home directories
(just across the LAN) for Linux and Mac users, primarily developers,
who could be doing anything from compiling code, to tar'ing files,
to heck, even running a VM from the network filesystem (though
that last one won't exactly be recommended).

It _will_ be a pure SSD RAID subsystem (LSI 9260-8i RAID 5 using
8 Intel 320series SSD drives), and interconnected to our network backbone
with a 10Gb/sec interface (SFP+ twinaxial direct-connect).  Though
each developer's workstation will be limited to gigabit speeds, the
aggregate of multiple workstations could be higher, and we don't
want the remote share to be a bottleneck.

A combination of NFSv3 and NFSv4 will be used, I'd need to research
this point, but I'd assume each workstation's connection would be
handled via its own process or thread within NFS, so it should be
able to scale multi-core.

Thanks.
-Brad

Re: [dm-crypt] Recommended modes for performance (SMP+AES-NI)

[Arno Wagner]

On Mon, Jun 27, 2011 at 01:00:11PM -0400, Brad House wrote:
> >>I think we've settled on AES-256, but may entertain AES-128
> >>if there is a huge performance difference as I think AES-128
> >>is still considered sufficiently safe for our purposes.
> >
> >The performance difference should be relatively small.
> 
> 
> That's my thought too, especially with AES-NI.
> 
> 
> >>So, the question is mainly what mode of operation would be
> >>best?
> >>  - cbc-essiv
> >>  - ctr-{plain64|essiv}
> >>  - xts-{plain64|essiv}
> >>  - are there any others I should be considering?
> >>NOTE: I'm not sure if essiv is even an option for CTR or XTS
> >>       modes, I'd like feedback on that, as well as what the
> >>       security implications are...
> >
> >ESSIV is only for CBC.
> 
> 
> Ah, ok, that's good to know.  Thanks.
> 
> 
> >>At this point, I'm leaning towards CTR mode, mainly because it
> >>was designed explicitly to be parallelizable:
> >>http://en.wikipedia.org/wiki/Block_cipher_modes_of_operation#Counter_.28CTR.29
> >
> >That is only for fine-grained paralellism, and hence not
> >applicable here. I am also not sure whether you can even use it
> >with dm-crypt as it needs a nonce in addition to the counter.
> >And that needs to be stored somewhere.
> 
> 
> Well, since Intel provided a specific CTR mode AES-NI patch and
> it referenced testing it _using_ dm-crypt
> (http://lwn.net/Articles/376562/), I'd assume it is possible to at
> least use it with dm-crypt ;)

If you drop the nonce, sure. But that would be a bad idea
security-wise, especially in disk encryption. Looks like 
they primarily wanted a good benchmark result.
  
> >Unless you have any specific security requirements beyond
> >the standard, go with the defaults. I think you are
> >overthinking this. The defaults are what is maintained best
> >and also what will get the fastest fixes and problem detection.
> 
> 
> No specific security requirements.  I'm not worried about
> governments targeting breaking this encryption or anything
> like that.   Just want to make sure the actual storage of this
> data isn't the weakest link in the security chain.

"Government" does not qualify as special ;-) 

What I meant was atypical usage scenarios. You have good
security as long as you use a high-entropy password 
(ideally in the > 80 bit range) and do not do bad
things with backup. Best read the Cryptsetup FAQ as
well, as if you use LUKS, you will want a secured 
header backup. 

> >If you have special requiremenrts, any deviation from the
> >defaults should have a strong justification comming from
> >these requirements. As you have not given any, I cannot
> >comment on them.
> 
> 
> Sorry, I should have provided more information...
> 
> The requirements are basically if the box leaves the building,
> that it be infeasible that the plaintext data be retrieved.

In this case, take into account that competent attackers will
bring an UPS and will not switch the box off. Switchover from
mains to UPS is possible, if a little tricky. If the box still 
runs, that leaves your data exposed. If that attacker competence
level (relatively high) is not a concern, then you are covered.

> Not worried about leakage of statistics or even modification
> of data, though if those too can be prevented for little cost,
> it'd be nice to have ;)

All basically covered with the defaults ;-)
 
> >As for speed, AFAIK, basically you are still limited to
> >one CPU per process that accesses an encrypted disk.
> >But keep in mind that with the defaults you get something
> >like 100MB/s crypto speed in pure software. Unless this thing
> >has an 2.5 or 10Gb/sec interface, and SSDs or mostly
> >linear, large-file accesses, that is quite fast enough
> >in most cases.
> 
> 
> This is a multi-user fileserver hosting remote home directories
> (just across the LAN) for Linux and Mac users, primarily developers,
> who could be doing anything from compiling code, to tar'ing files,
> to heck, even running a VM from the network filesystem (though
> that last one won't exactly be recommended).
> 
> It _will_ be a pure SSD RAID subsystem (LSI 9260-8i RAID 5 using
> 8 Intel 320series SSD drives), and interconnected to our network backbone
> with a 10Gb/sec interface (SFP+ twinaxial direct-connect).  Though
> each developer's workstation will be limited to gigabit speeds, the
> aggregate of multiple workstations could be higher, and we don't
> want the remote share to be a bottleneck.
> 
> A combination of NFSv3 and NFSv4 will be used, I'd need to research
> this point, but I'd assume each workstation's connection would be
> handled via its own process or thread within NFS, so it should be
> able to scale multi-core.

I don't actually know what the NFS-Server does. If it splits
off multiple children, you should get multiple CPUs
on the crypto. Best try it. However this impacts only
throughput anyways. Latency should basically be the same, so
people should get SSD-like performance at least for small files.

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] Recommended modes for performance (SMP+AES-NI)

[Milan Broz]

On 06/27/2011 07:00 PM, Brad House wrote:
>> ESSIV is only for CBC.

yes, but nothing will stop you to use it for other
mode (even if it is needed or redundant, like for XTS)

>>> At this point, I'm leaning towards CTR mode, mainly because it
>>> was designed explicitly to be parallelizable:
>>> http://en.wikipedia.org/wiki/Block_cipher_modes_of_operation#Counter_.28CTR.29
>>
>> That is only for fine-grained paralellism, and hence not
>> applicable here. I am also not sure whether you can even use it
>> with dm-crypt as it needs a nonce in addition to the counter.
>> And that needs to be stored somewhere.
> 
> 
> Well, since Intel provided a specific CTR mode AES-NI patch and
> it referenced testing it _using_ dm-crypt
> (http://lwn.net/Articles/376562/), I'd assume it is possible to at
> least use it with dm-crypt ;)

You can "use" it, again - dmcrypt will not stop you when doing that.
(try e.g. -c aes-ctr-plain64 -s 128)

Internally, it should use generated IV (plain64 - sector number here)
as concatenated nonce + counter, crypto API CTR implementation
then increases counter part when walking through block device sector
using cipher block steps.

So it "works" somehow. It is not tested at all though for use
with full disk encryption:)


>> Unless you have any specific security requirements beyond
>> the standard, go with the defaults. I think you are
>> overthinking this. The defaults are what is maintained best
>> and also what will get the fastest fixes and problem detection.

Exactly. Default is still CBC mode with ESSIV,
XTS is more and more used, so it is possible that in future
XTS mode will be become default in cryptsetup.

Milan