Re: [dm-crypt] aes-xts-plain with aes_x86_64 makes my SSD 5x slower than my encrypted HD

[Marc MERLIN <marc@merlins.org>]

On Mon, Jul 23, 2012 at 11:31:28PM +0200, Milan Broz wrote:
> On 07/23/2012 07:51 PM, Marc MERLIN wrote:
> > On Mon, Jul 23, 2012 at 07:15:24PM +0200, Milan Broz wrote:
> >>> Mmmh, I have one possible thing. I have a preempt kernel. Could that be it?
> >>> http://marc.merlins.org/tmp/config-3.4.4-amd64-preempt.txt
> >>
> >> Can you send me your kernel .config then? Preempt should not be problem.
> >> Which kernel version? which architecture? Any additional patches over
> >> mainline code?
> > 
> > I just sent you my config, it was in the URL above :)
> > No patches, kernel 3.4.4 from kernel.org, see above.
> 
> Ehm... sorry, I completely missed that. Thanks.
 
Mmmh, so I installed "standard" linux-image-3.2.0-3-amd64 from debian.
And....
nothing changed :(

/dev/mapper/cryptroot:
 Timing cached reads:   19642 MB in  2.00 seconds = 9833.88 MB/sec
 Timing buffered disk reads:  72 MB in  3.05 seconds =  23.59 MB/sec

Did you find anything more useful here:
http://marc.merlins.org/tmp/blktrace.txt

Or can I take another blktrace that helps?

> > Really?
> > I used fdisk -H32 -S32 /dev/sda as recomended on
> > http://www.void.gr/kargig/blog/2012/01/11/linux-ssd-partition-alignment-tips/
> 
> Do not use -H32 -S32. It is crazy and obsolete way how to align it...
> Someone is wrong in the internet seems http://xkcd.com/386/ ;-)

Yes, I know the cartoon :)
Mmmh, so I'll have to reformat everything so that all my partition start numbers
are multiple of 512.
Maybe I can get parted to move at least partition #4 without losing all my
data. I'll try that.

However is using
cryptsetup luksFormat --align-payload=8192
still the right thing for me?
(with the understanding that alignment shouldn't really an issue for reads,
but for writes)

> Disk driver should set topology parameters which fdisk uses. But for your
> case all is set to 512 bytes...
> 
> Whatever, there was a bug in fdisk, fixed now thanks to your report :)
> https://github.com/karelzak/util-linux/commit/c0175e6185ac81843cbad33cbea75abd033f0e66

Cool, thanks for that.

Ok, so I repartitioned my first 3 partitions, which I could do without
losing data:
Disk /dev/sda: 512.1 GB, 512110190592 bytes
255 heads, 63 sectors/track, 62260 cylinders, total 1000215216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x09aaf50a

  Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048      502271      250112   83  Linux
/dev/sda2          502272    52930559    26214144   83  Linux
/dev/sda3        52930560    73902079    10485760   82  Linux swap / Solaris
/dev/sda4        73902368  1000215215   463156424   83  Linux

OMG, that actually helped:
gandalfthegreat:~# echo test | cryptsetup create -c null test_crypt /dev/sda2
gandalfthegreat:~# hdparm -t -T /dev/mapper/test_crypt 
/dev/mapper/test_crypt:
 Timing cached reads:   18186 MB in  2.00 seconds = 9103.83 MB/sec
 Timing buffered disk reads: 524 MB in  3.04 seconds = 172.63 MB/sec

But with LUKS, it falls apart:
gandalfthegreat:~# cryptsetup luksFormat --align-payload=8192 -c aes-xts-plain -s 256 /dev/sda2 
(...)
gandalfthegreat:~# cryptsetup luksOpen --allow-discards /dev/sda2 test
(...)
gandalfthegreat:~# hdparm -t -T /dev/mapper/test
/dev/mapper/test:
 Timing cached reads:   17436 MB in  2.00 seconds = 8728.61 MB/sec
 Timing buffered disk reads:  74 MB in  3.03 seconds =  24.44 MB/sec

Grumble.

So
1) alignement has some effect and I'm not sure how to get luksFormat aligned right
2) Even in the better case above, 172.63 MB/sec is too slow. I was getting faster 
speeds by dding a file from potentially the encrypted filesystem.

If blktrace doesn't show anything useful, is there anything else I can capture?

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/