Re: [dm-crypt] aes-xts-plain with aes_x86_64 makes my SSD 5x slower than my encrypted HD

[Milan Broz <mbroz@redhat.com>]

>> So those speeds are expected/believable, but I still don't know why
>> hdparm is so slow on /dev/mapper/cryptroot.

First, hdparm basically run 2M read in direct-io mode for test,
all IOs are sumbitted from one process.

AES-NI helps a lot and because it is prioritised in crypto api,
you usually using it without any additional config if hw supports it.
(Also I see some patches whit run XTS blocks in parallel on crypto api list.)

But that's not the problem you are seeing now. (I will explain it on recent
kernel code, note that RHEL5/6 have older code and dmcrypt behaves slightly
differently when processing requests.)

Current in-kernel dmcrypt code tries to keep IO processing on CPU which
submitted request. It means, that if one process generates all requests,
all requests are usually processed by one core where it is this process
scheduled not using other CPU cores.
(Saying usually because page cache can submit request on different cpu.)

So no wonder that you get slow operation - in dd/hdparm usually only
one cpu is processing the request. If CPU is fast enough, no problem.
If not you will see slowdown. AES-NI will speed up this on that cpu core,
but will not help run request in parallel on multi-core systems.

In real fs use case, more applications submitting io requests, you will
get much better throughput. Also if not using directio, page cache can help
and fs are usually more clever too (writes are usually better
in performance - try it).

I do not like this dmcrypt mode a we tried to fix it. There is a bunch of patches
from Mikulas Patocka which switches parallelization to use all available
cpus (if not limited by paramater).
In my tests it improved performance in some cases but not in all situations
(there were some slow downs which scares me).
(You can see patches here http://people.redhat.com/mpatocka/patches/kernel/dm-crypt-paralelizace/)

Unfortunately discussion stopped and device-mapper maintainer forgot about it.

Well, your mails apparently caused some head-ups here, so I'll try to return
to this. (Will post them to dm-devel directly this time.)

I think fast SSD case is exactly situation where it should help.
We eill need some testers though :)
(I have currently no fast SSD available for testing.)

Milan