From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.saout.de ([127.0.0.1]) by localhost (mail.saout.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c9Jv4tpHcFa6 for ; Tue, 5 Feb 2013 22:09:37 +0100 (CET) Received: from ngcobalt23.manitu.net (ngcobalt23.manitu.net [217.11.48.123]) by mail.saout.de (Postfix) with ESMTP for ; Tue, 5 Feb 2013 22:09:37 +0100 (CET) Received: from ngcobalt23.manitu.net (localhost [127.0.0.1]) by ngcobalt23.manitu.net (8.10.2/8.10.2) with ESMTP id r15L9Xc22347 for ; Tue, 5 Feb 2013 22:09:33 +0100 Received: from server.passau (188-192-73-15-dynip.superkabel.de [188.192.73.15]) (Authenticated sender: pwendler) by ngcobalt23.manitu.net (Postfix) with ESMTPSA id 428EB2F0061 for ; Tue, 5 Feb 2013 22:09:33 +0100 (CET) Received: from [IPv6:2001:6f8:1c32::3] (mail.philippw.xdns.eu [IPv6:2001:6f8:1c32::3]) by server.passau (Postfix) with ESMTPS id A66078013F for ; Tue, 5 Feb 2013 22:09:36 +0100 (CET) Message-ID: <51117510.3090105@philippwendler.de> Date: Tue, 05 Feb 2013 22:09:36 +0100 From: Philipp Wendler MIME-Version: 1.0 References: <4E12B2EC.6000209@philippwendler.de> <5111654D.4000702@philippwendler.de> <20130205204528.GB15447@tansi.org> In-Reply-To: <20130205204528.GB15447@tansi.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: [dm-crypt] Bad performance with software RAID5 and LUKS encryption List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: dm-crypt@saout.de Hi, Am 05.02.2013 21:45, schrieb Arno Wagner: > thanks for the info. I think I may add a "performance" > section to the FAQ, and this could be one of the items. > Do you have the reference where you found the info on the > stripe_cache_size? Actually, I first read about the concept of a stripe cache in this btrfs announcement post: http://article.gmane.org/gmane.comp.file-systems.btrfs/23006 Then I googled it and found some usages: http://h3x.no/2011/07/09/tuning-ubuntu-mdadm-raid56 http://ubuntuforums.org/showthread.php?t=1494846 Unfortunately, I didn't find a real citable source. The kernel documentation (Documentation/md.txt) also only contains 3 lines saying nothing about what it actually is. Greetings, Philipp > On Tue, Feb 05, 2013 at 09:02:21PM +0100, Philipp Wendler wrote: >> Hello, >> >> Am 05.07.2011 08:45, schrieb Philipp Wendler: >>> I have set up a Linux software RAID5 on three hard drives and want to >>> encrypt it with cryptsetup/LUKS. My tests showed that the encryption >>> leads to a massive performance decrease that I cannot explain. >>> >>> The RAID5 is able to write 187 MB/s [1] without encryption. With >>> encryption on top of it, write speed is down to about 40 MB/s. >> >> Sorry for answering to a mail in this very old thread, >> but it seems I finally found a solution, >> and I know that there are some other people interested in the solution >> as well (so if you got this email directly from me, I BCC'ed you because >> you contacted me about this). >> If you want to read up the full story, here's the link: >> http://www.saout.de/pipermail/dm-crypt/2011-July/001773.html >> >> Today I read about the stripe_cache_size setting of md RAID, and tried >> it out. With the default value of 256, the performance is slow as >> described. With a value of 4096, I get a performance increase from about >> 40-50 MB/s to 123 MB/s. For values >= 8192, I get 140 MB/s out of it. >> >> Background: The stripe cache stores recently written blocks. If data is >> written continuously, it might happen that during a first write only a >> part of one stripe is written. This means, the RAID code has to read the >> complete stripe from disk, update it, and write it completely again. If >> a second write comes in for another part of the same stripe, all this >> would have to be done again. Now, if the cache is used and still >> contains the data written by the first write, the read that was >> necessary before the second write can be omitted. >> >> >> Now it seems that dm-crypt always writes with small block size to the >> underlying disk, even when I write with a big block size. >> Could this be true? >> Could this perhaps be improved? While I have found a solution for me, >> this could probably solve performance problems for many people. >> >> Furthermore, dm-crypt write is still slower than unencrypted write, >> although for reads the performance of encrypted and unencrypted are the >> same. So I guess the small block size still has a performance penalty >> (probably when first writing to a stripe and it is not yet in cache). >> >> My current setup is Ubuntu 12.04 (Linux 3.2). >> Nothing else has changed compared to when I first asked about this. >> >> Greetings, Philipp >> _______________________________________________ >> dm-crypt mailing list >> dm-crypt@saout.de >> http://www.saout.de/mailman/listinfo/dm-crypt >