[dm-crypt] LUKS/ dm-crypt/ ext4 appears to be single threaded

[dm-crypt] LUKS/ dm-crypt/ ext4 appears to be single threaded

[David Christensen]

dm-crypt:

Thread moved from debian-user.


Please comment.


TIA,

David



I have a 1.5 TB SATA hard drive I use for back-up's.  It has a single
large partition encrypted with LUKS/ dm-crypt and formatted with ext4.
I've noticed what appears to be single-threaded behavior when one
process is performing a long-lived write to the disk (notably 'ssh
user@host tar ... > backupfile.tar.gz') and another process attempts to
access the disk (either read or write).  This is tolerable for a back-up 
application, but would not be acceptable for multi-user, multi-process, 
and/or multi-threaded applications (file server, terminal server, web 
server, etc.).


Is this a fundamental limitation of LUKS, dm-crypt, and/or ext4, or
something I've configured/ misconfigured?


If a fundamental limitation, is there something I can substitute to
eliminate the problem?


Some manufacturers make hard drives with built-in encryption.  Are these 
supported by Debian, Linux, or BSD?

     http://www.seagate.com/www/en-us/products/self-encrypting-drives/


Any other comments or suggestions?


TIA,

David

Re: [dm-crypt] LUKS/ dm-crypt/ ext4 appears to be single threaded

[Roscoe]

[-- Attachment #1: Type: text/plain, Size: 532 bytes --]

Hey David,

I've frequently found performance in Linux while running an IO intensive
process to leave something to be desired, though I've only delved as deep
as changing IO schedulers to resolve this. I'll be interested to hear what
you find.

As for hard drives with built-in encryption, I'd suggest avoiding them
unless you have some reliable assurance that they work as you believe they
do. It's just too easy for an invisible implemention detail (mistake or
intentional) to result in compromised security.

Regards,

-- Roscoe

[-- Attachment #2: Type: text/html, Size: 668 bytes --]

Re: [dm-crypt] LUKS/ dm-crypt/ ext4 appears to be single threaded

[Milan Broz]

On 03/23/2012 05:11 AM, David Christensen wrote:
> I have a 1.5 TB SATA hard drive I use for back-up's.  It has a single
> large partition encrypted with LUKS/ dm-crypt and formatted with ext4.
> I've noticed what appears to be single-threaded behavior when one
> process is performing a long-lived write to the disk (notably 'ssh
> user@host tar ...>  backupfile.tar.gz') and another process attempts to
> access the disk (either read or write).  This is tolerable for a back-up
> application, but would not be acceptable for multi-user, multi-process,
> and/or multi-threaded applications (file server, terminal server, web
> server, etc.).

You forgot to mention kernel version, it changed in various versions.

For recent kernel 3.x, if multiple processes access the encrypted disk,
it tries to keep submitted work on cpu which submitted it (so it run in parallel).

There are some patches to change parallel operations to another model
in dmcrypt, if you want to experiment, see
http://people.redhat.com/mpatocka/patches/kernel/dm-crypt-paralelizace/current/

I have several performance tests, and some of these patches perhaps
appear in 3.5 kernel but it need still more work.

Milan

Re: [dm-crypt] LUKS/ dm-crypt/ ext4 appears to be single threaded

[David Christensen]

On 03/28/2012 02:33 PM, Milan Broz wrote:

Thank you for your reply.  :-)


I've been trying to pay more attention, and it appears that the blocking 
is limited to directories (and their subdirectories) where long-lived 
writes are occurring (?).  Am I understanding the issue correctly?  Has 
anyone else seen this?


Is "New Methods in Hard Disk Encryption" still relevant?

     http://clemens.endorphin.org/cryptography


> You forgot to mention kernel version, it changed in various versions.

$ uname -a
Linux p43400e 2.6.32-5-686 #1 SMP Mon Jan 16 16:04:25 UTC 2012 i686 
GNU/Linux

$ sudo cryptsetup --version
cryptsetup 1.1.3

$ cat /etc/debian_version
6.0.4


Are there any tunable parameters, options, alternatives, etc., that I 
can adjust/ change to improve parallelism with what I have?


> For recent kernel 3.x, if multiple processes access the encrypted disk,
> it tries to keep submitted work on cpu which submitted it (so it run in
> parallel).
> There are some patches to change parallel operations to another model
> in dmcrypt, if you want to experiment, see
> http://people.redhat.com/mpatocka/patches/kernel/dm-crypt-paralelizace/current/
> I have several performance tests, and some of these patches perhaps
> appear in 3.5 kernel but it need still more work.

My kernel skills are far too rusty, and I need stability/ finished 
product.  So, I'll have to wait for those patches to get into Debian 
stable (or testing).


David