From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.saout.de ([127.0.0.1]) by localhost (mail.saout.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yFbfuqD1itSq for ; Mon, 23 Jul 2012 12:46:42 +0200 (CEST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mail.saout.de (Postfix) with ESMTP for ; Mon, 23 Jul 2012 12:46:41 +0200 (CEST) Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q6NAkdRq028306 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Mon, 23 Jul 2012 06:46:40 -0400 Received: from [10.34.2.19] (tawny.brq.redhat.com [10.34.2.19]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id q6NAkcVJ019835 for ; Mon, 23 Jul 2012 06:46:39 -0400 Message-ID: <500D2B8E.3000908@redhat.com> Date: Mon, 23 Jul 2012 12:46:38 +0200 From: Milan Broz MIME-Version: 1.0 References: <20120722190757.GB10089@merlins.org> <20120722202213.GA7306@fancy-poultry.org> <20120722190757.GB10089@merlins.org> <1342986452.26887.3.camel@scapa> <20120722203929.GB3925@merlins.org> <20120722214757.GA16793@tansi.org> <20120723062850.GA6931@merlins.org> <20120723081407.GA872@tansi.org> In-Reply-To: <20120723081407.GA872@tansi.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: [dm-crypt] aes-xts-plain with aes_x86_64 makes my SSD 5x slower than my encrypted HD List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: dm-crypt@saout.de >> So those speeds are expected/believable, but I still don't know why >> hdparm is so slow on /dev/mapper/cryptroot. First, hdparm basically run 2M read in direct-io mode for test, all IOs are sumbitted from one process. AES-NI helps a lot and because it is prioritised in crypto api, you usually using it without any additional config if hw supports it. (Also I see some patches whit run XTS blocks in parallel on crypto api list.) But that's not the problem you are seeing now. (I will explain it on recent kernel code, note that RHEL5/6 have older code and dmcrypt behaves slightly differently when processing requests.) Current in-kernel dmcrypt code tries to keep IO processing on CPU which submitted request. It means, that if one process generates all requests, all requests are usually processed by one core where it is this process scheduled not using other CPU cores. (Saying usually because page cache can submit request on different cpu.) So no wonder that you get slow operation - in dd/hdparm usually only one cpu is processing the request. If CPU is fast enough, no problem. If not you will see slowdown. AES-NI will speed up this on that cpu core, but will not help run request in parallel on multi-core systems. In real fs use case, more applications submitting io requests, you will get much better throughput. Also if not using directio, page cache can help and fs are usually more clever too (writes are usually better in performance - try it). I do not like this dmcrypt mode a we tried to fix it. There is a bunch of patches from Mikulas Patocka which switches parallelization to use all available cpus (if not limited by paramater). In my tests it improved performance in some cases but not in all situations (there were some slow downs which scares me). (You can see patches here http://people.redhat.com/mpatocka/patches/kernel/dm-crypt-paralelizace/) Unfortunately discussion stopped and device-mapper maintainer forgot about it. Well, your mails apparently caused some head-ups here, so I'll try to return to this. (Will post them to dm-devel directly this time.) I think fast SSD case is exactly situation where it should help. We eill need some testers though :) (I have currently no fast SSD available for testing.) Milan