Re: [dm-crypt] Bad performance with software RAID5 and LUKS encryption

[Philipp Wendler <ml@philippwendler.de>]

Hello,

Am 05.07.2011 08:45, schrieb Philipp Wendler:
> I have set up a Linux software RAID5 on three hard drives and want to
> encrypt it with cryptsetup/LUKS. My tests showed that the encryption
> leads to a massive performance decrease that I cannot explain.
>
> The RAID5 is able to write 187 MB/s [1] without encryption. With
> encryption on top of it, write speed is down to about 40 MB/s.

Sorry for answering to a mail in this very old thread,
but it seems I finally found a solution,
and I know that there are some other people interested in the solution
as well (so if you got this email directly from me, I BCC'ed you because
you contacted me about this).
If you want to read up the full story, here's the link:
http://www.saout.de/pipermail/dm-crypt/2011-July/001773.html

Today I read about the stripe_cache_size setting of md RAID, and tried
it out. With the default value of 256, the performance is slow as
described. With a value of 4096, I get a performance increase from about
40-50 MB/s to 123 MB/s. For values >= 8192, I get 140 MB/s out of it.

Background: The stripe cache stores recently written blocks. If data is
written continuously, it might happen that during a first write only a
part of one stripe is written. This means, the RAID code has to read the
complete stripe from disk, update it, and write it completely again. If
a second write comes in for another part of the same stripe, all this
would have to be done again. Now, if the cache is used and still
contains the data written by the first write, the read that was
necessary before the second write can be omitted.


Now it seems that dm-crypt always writes with small block size to the
underlying disk, even when I write with a big block size.
Could this be true?
Could this perhaps be improved? While I have found a solution for me,
this could probably solve performance problems for many people.

Furthermore, dm-crypt write is still slower than unencrypted write,
although for reads the performance of encrypted and unencrypted are the
same. So I guess the small block size still has a performance penalty
(probably when first writing to a stripe and it is not yet in cache).

My current setup is Ubuntu 12.04 (Linux 3.2).
Nothing else has changed compared to when I first asked about this.

Greetings, Philipp