[dm-crypt] LUKS & TrueCrypt - Speed Test

[dm-crypt] LUKS & TrueCrypt - Speed Test

[Jorge Fábregas]

Hello everyone,

Inspired by this old blog post:

http://movingparts.net/2007/10/26/truecrypt-versus-luks-speed-test/

...I decided to perform some tests on my Fedora 14 box.   This is not a
pro benchmark so be warned :)

Common Facts for both tests:

- source & destination filesystems were ext4
- destination is an external USB drive
- source data size is 143GB (a folder with lots of files & directories,
small & large files, regular data...)
- rsync was used to perform the actual copy
- I'm using an "encrypted partition " (against an encrypted file)
- I did a test first with TrueCrypt and then with LUKS
- Between the above tests, I shut down the machine (to flush filesystem
cache).
- my system kernel: 2.6.35.13-92.fc14.i686

### TrueCrypt Results ####
I used AES-256 (XTS operation mode), hash algorithm: ripemd-160 and the
package was realcrypt-7.0a-1.fc14.i686

Output of time command after rsync finished:

real	105m22.211s
user	28m10.471s
sys	41m35.319s


### DM-Crypt LUKS Results ###
I used the defaults:  AES-256 (CBC), sha1 for header hashing and the
package cryptsetup-luks-1.1.3-1.fc14.i686

Output of time command after rsync finished:

real	108m55.291s
user	28m6.534s
sys	42m53.400s


As you can see, there's almost a 4 minute difference.  I was expecting
LUKS to be faster (as dm-crypt is a kernel module) and TrueCrypt runs
mainly in user space isn't it?  Do you think the cipher operation modes
(XTS vs CBC) played a role in this difference? Have any of you performed
a similar test?

Regards,
Jorge

Re: [dm-crypt] LUKS & TrueCrypt - Speed Test

[Arno Wagner]

On Wed, Jul 27, 2011 at 07:18:24PM -0400, Jorge F?bregas wrote:
> Hello everyone,
> 
> Inspired by this old blog post:
> 
> http://movingparts.net/2007/10/26/truecrypt-versus-luks-speed-test/
> 
> ...I decided to perform some tests on my Fedora 14 box.   This is not a
> pro benchmark so be warned :)
> 
> Common Facts for both tests:
> 
> - source & destination filesystems were ext4
> - destination is an external USB drive
> - source data size is 143GB (a folder with lots of files & directories,
> small & large files, regular data...)
> - rsync was used to perform the actual copy
> - I'm using an "encrypted partition " (against an encrypted file)
> - I did a test first with TrueCrypt and then with LUKS
> - Between the above tests, I shut down the machine (to flush filesystem
> cache).
> - my system kernel: 2.6.35.13-92.fc14.i686
> 
> ### TrueCrypt Results ####
> I used AES-256 (XTS operation mode), hash algorithm: ripemd-160 and the
> package was realcrypt-7.0a-1.fc14.i686
> 
> Output of time command after rsync finished:
> 
> real	105m22.211s
> user	28m10.471s
> sys	41m35.319s
> 
> 
> ### DM-Crypt LUKS Results ###
> I used the defaults:  AES-256 (CBC), sha1 for header hashing and the
> package cryptsetup-luks-1.1.3-1.fc14.i686
> 
> Output of time command after rsync finished:
> 
> real	108m55.291s
> user	28m6.534s
> sys	42m53.400s
> 
> 
> As you can see, there's almost a 4 minute difference.  I was expecting
> LUKS to be faster (as dm-crypt is a kernel module) and TrueCrypt runs
> mainly in user space isn't it?  Do you think the cipher operation modes
> (XTS vs CBC) played a role in this difference? Have any of you performed
> a similar test?

There is an old gemran egineering saying:

"wer mist mist mist" 

(along the lines of "Those who measure measure crap")
I think it applies here.

Real-time is tricky. It does not reflect effort invested. If you 
look at the sys itime, you see that the crypto-effort is only about
90 seconds more. Even that is pretty much below the measurement
error. Very likely the differences are due to storage differences
and do not show crypto-speed differences.

I suggest you run both tests at least 3 times and make sure
your storage is significantly faster than the crypto, e.g. 
by doing this between RAM disks or SSD storage. Also a complex
disk access patterhn like rsync is not suitable as it may
have complex interactions with caching and buffering.

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] LUKS & TrueCrypt - Speed Test

[Milan Broz]

On 07/28/2011 01:18 AM, Jorge Fábregas wrote:

> - Between the above tests, I shut down the machine (to flush filesystem
> cache).

echo 3 > /proc/sys/vm/drop_caches

does the same in more controlled way

> - my system kernel: 2.6.35.13-92.fc14.i686

Can you use something more recent? There were huge changes.
The same for userspace.

BTW do you know that TC7 uses dm-crypt as backend? So with the same
mode and alignemnt you should get the same results
(except chained ciphers in TC).

> As you can see, there's almost a 4 minute difference.  I was expecting
> LUKS to be faster (as dm-crypt is a kernel module) and TrueCrypt runs
> mainly in user space isn't it?  Do you think the cipher operation modes
> (XTS vs CBC) played a role in this difference? Have any of you performed
> a similar test?

Perhaps because you tested different modes... Use the same mode
in cryptsetup

cryptstup luksFormat -c aes-xts-plain64 -s 512 ... 

I would really wonder if speed differs, TC7 uses dmsetup to configure
dm-crypt mapping on Linux :-) It uses userspace (and FUSE)  only for old
containers and some magic for hidden container.

Milan

Re: [dm-crypt] LUKS & TrueCrypt - Speed Test

[Jorge Fábregas]

On 07/28/2011 01:11 AM, Arno Wagner wrote:
> There is an old gemran egineering saying:
> 
> "wer mist mist mist" 
> 
> (along the lines of "Those who measure measure crap")
> I think it applies here.

Hello Amo,

I warned everyone that this wasn't a pro test :)  At least, I laid down
the specifics involved.


> Real-time is tricky. It does not reflect effort invested. If you 
> look at the sys itime, you see that the crypto-effort is only about
> 90 seconds more. Even that is pretty much below the measurement
> error. 

I agree here. I shouldn't have paid much attention to real time.
Nonetheless I'm still curious about the little difference...

> Very likely the differences are due to storage differences
> and do not show crypto-speed differences.


I used the same external drive for both tests.

> I suggest you run both tests at least 3 times and make sure
> your storage is significantly faster than the crypto, e.g. 
> by doing this between RAM disks or SSD storage. Also a complex
> disk access patterhn like rsync is not suitable as it may
> have complex interactions with caching and buffering.


I didn't want to go with sequential & random read/writes (with different
block sizes etc) as I wanted a rough test out of the very same tool I
use every day (rsync) with the same data on the same disk.  I understand
the crypto involved (CPU-wise) is much faster than the slow I/O of my
external drive but that's what I have.   Regarding repeating the test, I
totally agree with that.

Thanks for the input.

Regards,
Jorge

Re: [dm-crypt] LUKS & TrueCrypt - Speed Test

[Jorge Fábregas]

On 07/28/2011 03:40 AM, Milan Broz wrote:
> echo 3 > /proc/ss/cm/drop_caches
> 
> does the same in more controlled way

Hello Milan,

Arraign I didn't know that one.  Good to know!

> Can you use something more recent? There were huge changes.
> The same for user space.

I always stay with the stock kernel (Fedora 14) so I don't have to take
care of bug-fixes & security-updates for the kernel :)   ...but I'll try
to run a newer kernel If I do the test again.

> BTW do you know that TC7 uses dm-crypt as backend? So with the same
> mode and alignemnt you should get the same results
> (except chained ciphers in TC).

I had no idea it used dm-crypt!  That's great.

> Perhaps because you tested different modes... Use the same mode
> in cryptsetup
>
> cryptstup luksFormat -c aes-xts-plain64 -s 512 ... 
> 
> I would really wonder if speed differs, TC7 uses dmsetup to configure
> dm-crypt mapping on Linux :-) It uses user space (and FUSE)  only for old
> containers and some magic for hidden container.

Great,  I'll do that.  I'll perform the tests again (everything equal:
kernel, disk, data, empty buffers, rsync) but I'll use xts this time for
LUKS.  I'll report back.

Thanks Milan!
Jorge

Re: [dm-crypt] LUKS & TrueCrypt - Speed Test

[Jorge Fábregas]

On 07/27/2011 07:18 PM, Jorge Fábregas wrote:
> I decided to perform some tests on my Fedora 14 box.   This is not a
> pro benchmark so be warned :)

Hi everyone,

I performed the tests again (twice for each test) but this time I
formatted the LUKS partition using:

cryptsetup luksFormat -c aes-xts-plain -s 256 -h ripemd160 /dev/sdd1

...so that I was more _similar_ to the TrueCrypt setup.  Also, between
each test I run:

echo 3 > /proc/sys/vm/drop_caches

Here are the new results for the same payload (143 GB of data):

### TRUECRYPT ####

1st round:
real	105m39.547s
user	28m17.667s
sys	42m25.300s


2nd round:
real	105m40.271s
user	28m21.893s
sys	42m19.672s


### LUKS ###

1st round:
real	104m33.901s
user	27m41.362s
sys	41m0.339s


2nd round:
real	104m44.913s
user	27m42.364s
sys	40m57.655s


Now as you may see, LUKS is roughly around 1 minute ahead (sytem-time)
compared to TrueCrypt.  It appears the change in cipher operation mode
definitely affected the results (thing I should have done on the first
place).

Cheers and thank your for the feedback!
Jorge

Re: [dm-crypt] LUKS & TrueCrypt - Speed Test

[Arno Wagner]

Hi Jorge,

you can see that running the test several times gives you some 
appreciation for the variation in the measurement. Anyways,
this is what I would expect, i.e. TrueCrypt having a little
bit lower speed due to the userspace interface. 

Thanks for posting these!

Arno


On Thu, Jul 28, 2011 at 07:58:58PM -0400, Jorge F?bregas wrote:
> On 07/27/2011 07:18 PM, Jorge F?bregas wrote:
> > I decided to perform some tests on my Fedora 14 box.   This is not a
> > pro benchmark so be warned :)
> 
> Hi everyone,
> 
> I performed the tests again (twice for each test) but this time I
> formatted the LUKS partition using:
> 
> cryptsetup luksFormat -c aes-xts-plain -s 256 -h ripemd160 /dev/sdd1
> 
> ...so that I was more _similar_ to the TrueCrypt setup.  Also, between
> each test I run:
> 
> echo 3 > /proc/sys/vm/drop_caches
> 
> Here are the new results for the same payload (143 GB of data):
> 
> ### TRUECRYPT ####
> 
> 1st round:
> real	105m39.547s
> user	28m17.667s
> sys	42m25.300s
> 
> 
> 2nd round:
> real	105m40.271s
> user	28m21.893s
> sys	42m19.672s
> 
> 
> ### LUKS ###
> 
> 1st round:
> real	104m33.901s
> user	27m41.362s
> sys	41m0.339s
> 
> 
> 2nd round:
> real	104m44.913s
> user	27m42.364s
> sys	40m57.655s
> 
> 
> Now as you may see, LUKS is roughly around 1 minute ahead (sytem-time)
> compared to TrueCrypt.  It appears the change in cipher operation mode
> definitely affected the results (thing I should have done on the first
> place).
> 
> Cheers and thank your for the feedback!
> Jorge
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] LUKS & TrueCrypt - Speed Test

[Roscoe]

[-- Attachment #1: Type: text/plain, Size: 2676 bytes --]

Hey Jorge,

If you're in the benchmarking mood and have a modern CPU...Different ciphers
would be interesting :)

A short script that given a block device would try a variety of ciphers [and
no encryption too] and print the results in a nice format would be neat.

-- Roscoe

On Fri, Jul 29, 2011 at 3:20 PM, Arno Wagner <arno@wagner.name> wrote:

> Hi Jorge,
>
> you can see that running the test several times gives you some
> appreciation for the variation in the measurement. Anyways,
> this is what I would expect, i.e. TrueCrypt having a little
> bit lower speed due to the userspace interface.
>
> Thanks for posting these!
>
> Arno
>
>
> On Thu, Jul 28, 2011 at 07:58:58PM -0400, Jorge F?bregas wrote:
> > On 07/27/2011 07:18 PM, Jorge F?bregas wrote:
> > > I decided to perform some tests on my Fedora 14 box.   This is not a
> > > pro benchmark so be warned :)
> >
> > Hi everyone,
> >
> > I performed the tests again (twice for each test) but this time I
> > formatted the LUKS partition using:
> >
> > cryptsetup luksFormat -c aes-xts-plain -s 256 -h ripemd160 /dev/sdd1
> >
> > ...so that I was more _similar_ to the TrueCrypt setup.  Also, between
> > each test I run:
> >
> > echo 3 > /proc/sys/vm/drop_caches
> >
> > Here are the new results for the same payload (143 GB of data):
> >
> > ### TRUECRYPT ####
> >
> > 1st round:
> > real  105m39.547s
> > user  28m17.667s
> > sys   42m25.300s
> >
> >
> > 2nd round:
> > real  105m40.271s
> > user  28m21.893s
> > sys   42m19.672s
> >
> >
> > ### LUKS ###
> >
> > 1st round:
> > real  104m33.901s
> > user  27m41.362s
> > sys   41m0.339s
> >
> >
> > 2nd round:
> > real  104m44.913s
> > user  27m42.364s
> > sys   40m57.655s
> >
> >
> > Now as you may see, LUKS is roughly around 1 minute ahead (sytem-time)
> > compared to TrueCrypt.  It appears the change in cipher operation mode
> > definitely affected the results (thing I should have done on the first
> > place).
> >
> > Cheers and thank your for the feedback!
> > Jorge
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> >
>
> --
> Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email:
> arno@wagner.name
> GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25
> 338F
> ----
> Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans
>
> If it's in the news, don't worry about it.  The very definition of
> "news" is "something that hardly ever happens." -- Bruce Schneier
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>

[-- Attachment #2: Type: text/html, Size: 3805 bytes --]