Re: [dm-crypt] aes-xts-plain with aes_x86_64 makes my SSD 5x slower than my encrypted HD

[Marc MERLIN <marc@merlins.org>]

On Sun, Jul 22, 2012 at 01:39:29PM -0700, Marc MERLIN wrote:
> Result is the same:
> gandalfthegreat:~# hdparm -tT /dev/mapper/ssdcrypt 
> 
> /dev/mapper/ssdcrypt:
>  Timing cached reads:   16614 MB in  2.00 seconds = 8317.50 MB/sec
>  Timing buffered disk reads:  68 MB in  3.08 seconds =  22.09 MB/sec
> 
> Grumble.
> 
> It shouldn't be a hardware problem since I do see 400MB/s before encryption.

Ok, I'm not sure I understand. I mounted /dev/mapper/ssdcrypt (btrfs)
and when going through the filesystem I get much higher speeds than
using dd on the raw block device.
But... when I run 

The test below shows that I can access the encrypted SSD 10x faster by talking
to it through a btrfs filesystem than doing a dd read from the device node.

So I suppose I could just ignore the dd device node thing, however not really.
Let's take a directory with some source files inside:

gandalfthegreat:/var/local# find src | wc -l
15261
gandalfthegreat:/var/local# echo 3 > /proc/sys/vm/drop_caches; time du -sh src
514M	src
real	0m5.865s

So on an encrypted spinning disk, it takes 6 seconds.
On my SSD in the same machine with the same encryption and the same btrfs filesystem,
I get 5 times slower:
gandalfthegreat:/mnt/btrfs_pool1/var/local# time du -sh src
514M	src
real	0m24.937s

Incidently 5x is also the speed difference between my encrypted HD and encrypted SSD
with dd.
Now, why du is 5x slower and dd of a file from the filesystem is 2.5x
faster, I have no idea :-/


See below:

1) drop caches
gandalfthegreat:/mnt/btrfs_pool1/var/local/VirtualBox VMs/w2k_virtual# echo 3 > /proc/sys/vm/drop_caches

gandalfthegreat:/mnt/btrfs_pool1/var/local/VirtualBox VMs/w2k_virtual# dd if=w2k-s001.vmdk of=/dev/null
2146631680 bytes (2.1 GB) copied, 8.03898 s, 267 MB/s
-> 267MB/s reading from the file through the encrypted filesystem. That's good.

For comparison
gandalfthegreat:/mnt/mnt2# dd if=w2k-s001.vmdk of=/dev/null 
2146631680 bytes (2.1 GB) copied, 4.33393 s, 495 MB/s
-> almost 500MB/s reading through another unencrypted filesystem on the same SSD

gandalfthegreat:/mnt/btrfs_pool1/var/local/VirtualBox VMs/w2k_virtual# dd if=/dev/mapper/ssdcrypt of=/dev/null bs=1M count=1000
1048576000 bytes (1.0 GB) copied, 45.1234 s, 23.2 MB/s

-> 23MB/s reading from the block device that my FS is mounted from. WTF?

gandalfthegreat:/mnt/btrfs_pool1/var/local/VirtualBox VMs/w2k_virtual# echo 3 > /proc/sys/vm/drop_caches; dd if=w2k-s001.vmdk of=test
2146631680 bytes (2.1 GB) copied, 17.9129 s, 120 MB/s

-> 120MB/s copying a file from the SSD to itself. That's not bad.

gandalfthegreat:/mnt/btrfs_pool1/var/local/VirtualBox VMs/w2k_virtual# echo 3 > /proc/sys/vm/drop_caches; dd if=test of=/dev/null
2146631680 bytes (2.1 GB) copied, 8.4907 s, 253 MB/s

-> reading the new copied file still shows 253MB/s, good.

gandalfthegreat:/mnt/btrfs_pool1/var/local/VirtualBox VMs/w2k_virtual# dd if=test of=/dev/null
2146631680 bytes (2.1 GB) copied, 2.11001 s, 1.0 GB/s

-> reading without dropping the cache shows 1GB/s 

I'm very lost now, any idea what's going on?

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/