From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.saout.de ([127.0.0.1]) by localhost (mail.saout.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZQ78niaGj9FI for ; Mon, 23 Jul 2012 19:51:34 +0200 (CEST) Received: from mail1.merlins.org (magic.merlins.org [209.81.13.136]) (using TLSv1.2 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.saout.de (Postfix) with ESMTPS for ; Mon, 23 Jul 2012 19:51:34 +0200 (CEST) Date: Mon, 23 Jul 2012 10:51:29 -0700 From: Marc MERLIN Message-ID: <20120723175129.GA15867@merlins.org> References: <20120722190757.GB10089@merlins.org> <20120722202213.GA7306@fancy-poultry.org> <20120722190757.GB10089@merlins.org> <1342986452.26887.3.camel@scapa> <20120722203929.GB3925@merlins.org> <20120722214757.GA16793@tansi.org> <20120723062850.GA6931@merlins.org> <20120723081407.GA872@tansi.org> <20120723161242.GB27727@merlins.org> <500D86AC.7090100@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1343060350.26887.20.camel@scapa> <500D86AC.7090100@redhat.com> Subject: Re: [dm-crypt] aes-xts-plain with aes_x86_64 makes my SSD 5x slower than my encrypted HD List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Milan Broz , Yves-Alexis Perez Cc: dm-crypt@saout.de On Mon, Jul 23, 2012 at 07:15:24PM +0200, Milan Broz wrote: > > Mmmh, I have one possible thing. I have a preempt kernel. Could that be it? > > http://marc.merlins.org/tmp/config-3.4.4-amd64-preempt.txt > > Can you send me your kernel .config then? Preempt should not be problem. > Which kernel version? which architecture? Any additional patches over > mainline code? I just sent you my config, it was in the URL above :) No patches, kernel 3.4.4 from kernel.org, see above. I tried without preempt and with volprempt and duplicated the same slow speeds. > > Disk /dev/sda: 512.1 GB, 512110190592 bytes > > 255 heads, 63 sectors/track, 62260 cylinders, total 1000215216 sectors > > Units = sectors of 1 * 512 = 512 bytes > > Sector size (logical/physical): 512 bytes / 512 bytes > > I/O size (minimum/optimal): 512 bytes / 512 bytes > > Disk identifier: 0x09aaf50a > > > > Device Boot Start End Blocks Id System > > /dev/sda1 * 2048 502047 250000 83 Linux > > /dev/sda2 502048 52930847 26214400 83 Linux > > /dev/sda3 52930848 73902367 10485760 82 Linux swap / Solaris > > /dev/sda4 73902368 1000215215 463156424 83 Linux > > Hm. sda1 is apparently aligned. But I think > other partitions are not aligned properly. Really? I used fdisk -H32 -S32 /dev/sda as recomended on http://www.void.gr/kargig/blog/2012/01/11/linux-ssd-partition-alignment-tips/ > No idea which block size/page size your SSD internaly use, but to be safe, > let's assume ideal is 256KiB (= 512 * 512 byte sectors). > 73902368 seems not to be multiple of 512... > > Well, your sda2 configuration is created with +256MB, which is SI unit, > so not multiple of 1024 but 1000! Oh my, I can't believe they changed that (been using linux since 1993 back then +1MB in fdisk was a real MB). > It should be created with +256M, so you end with: > Device Boot Start End Blocks Id System > /dev/sda1 2048 526335 262144 83 Linux > /dev/sda2 526336 ... > > But I think this is not the core problem but you should try it to fix. > (I hope I did not mixed up numbers above :) It would be a problem for erase blocks if it is wrong, but not for reading, especially when I can do filesystem reads at 260MB/s on the filesystem created on the encrypted device. But I'm confused, I thought fdisk -H32 -S32 would protect me from mis-alignment and fdisk/util-linux 2.20.1 was smart enough to use proper boundaries on its own on SSDs anyway. > ... Thinking about it, we can test it without partition change (for reads). > > This test is perhaps useless but maybe someone can find it useful later. > > Let's map properly aligned device to sda4 (read only). Result will be just > garbage, but it is just for speed testing. > > For mapping above, properly aligned (1MiB alignment = 2048 sectors) > should start on sector 7390412. This is +1760 sector shift for sda4. > > Map 1GiB device there: > # dmsetup create test_plain -r --table "0 2097152 linear /dev/sda4 1760" > > Map null cryptsetup over it > # echo "password" | cryptsetup create -c null test_crypt /dev/mapper/test_plain > > Now repeat read dd test for /dev/mapper/test_plain and /dev/mapper/test_crypt Same thing: gandalfthegreat:~# dmsetup create test_plain -r --table "0 2097152 linear /dev/sda4 1760" gandalfthegreat:~# echo "password" | cryptsetup create -c null test_crypt /dev/mapper/test_plain gandalfthegreat:~# dmsetup status test_crypt 0 2097152 crypt gandalfthegreat:~# cryptsetup status test_crypt /dev/mapper/test_crypt is active. type: PLAIN cipher: cipher_null-ecb keysize: 0 bits device: /dev/mapper/test_plain offset: 0 sectors size: 2097152 sectors mode: read/write gandalfthegreat:~# hdparm -t -T /dev/mapper/test_plain /dev/mapper/test_plain: Timing cached reads: 13978 MB in 2.00 seconds = 6997.30 MB/sec Timing buffered disk reads: 958 MB in 3.00 seconds = 319.00 MB/sec gandalfthegreat:~# hdparm -t -T /dev/mapper/test_crypt /dev/mapper/test_crypt: Timing cached reads: 15434 MB in 2.00 seconds = 7725.50 MB/sec Timing buffered disk reads: 76 MB in 3.03 seconds = 25.05 MB/sec gandalfthegreat:~# > > I also used: > > cryptsetup luksFormat --align-payload=8192 > > It will not help if underlying partition is misaligned. That's correct of course :) Now I'm going to have to do some more reading to see whether I'm really misaligned or not. I had done all the relevant reading on SSDs to make sure I did use proper alignment. I wonder why fdisk -H32 -S32 failed me. > I think this one, try to increase it to 8192 or so... > > > nr_requests:128 Didn't help. gandalfthegreat:/sys/block/sda/queue# echo 8192 > nr_requests gandalfthegreat:/sys/block/sda/queue# hdparm -t -T /dev/mapper/cryptroot /dev/mapper/cryptroot: Timing cached reads: 14970 MB in 2.00 seconds = 7492.34 MB/sec Timing buffered disk reads: 72 MB in 3.08 seconds = 23.35 MB/sec gandalfthegreat:/sys/block/sda/queue# Thanks for the other suggestions. Hopefully we'll nail this somehow :) Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/