From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mexico.limpoc.com (mexico.limpoc.com [207.241.238.23]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.saout.de (Postfix) with ESMTPS for ; Thu, 11 Nov 2010 20:17:11 +0100 (CET) Date: Thu, 11 Nov 2010 11:10:01 -0800 From: epvdm@limpoc.com Message-ID: <20101111191000.GA18175@limpoc.com> References: <4CDC2F13.3050602@gmx.at> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4CDC2F13.3050602@gmx.at> Subject: Re: [dm-crypt] Improving performance? List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Markus Krainz Cc: dm-crypt@saout.de, Lasse Jensen On Thu, Nov 11, 2010 at 06:59:47PM +0100, Markus Krainz wrote: > Hi. > > On 2010-11-11 11:49, Lasse Jensen wrote: > >Hi. I have a RAID 5 array with 3 (soon upgrading to 4 + hotspare = > >5) encrypted drives connected to a system with a Core 2 Duo @ 2.5 > >ghz running Debian Squeeze. For what it's worth I use luks on top of mirrored pairs of drives and have frequently lost and replaced one drive of a pair without problems. eric > >Each drive has been formatted with > > > >cryptsetup luksFormat /path/to/device > > > >And put together in a array with > > > >mdamd -C /dev/md0 --raid-level=5 /path/to/first-device > >/path/to/third-device /path/to/third-device > > > >It works great, and encrypting the devices separately allows me to > >run more than one instance of kcryptd, thus using both cores in my > >server. It compensates for the overhead of encrypting the > >checksumming data seperately, compared to raw devices -> RAID -> > >encryption and still give me improved speed. > > Have you ever done the test and unplugged a drive from your raid and > assessed if the raid5 still works? > My experience with this setup is that cryptsetup and mdadm do not > work well, if the underlying device suddenly disappears. > > First unplug 1 of your 3 drives, look if it still works and mdadm > recognises the missing drive using mdadm --detail /dev/raidname > Then reconnect the drive without restarting the computer simulating > a new device to replace the old one. > Try if you can still open it with cryptsetup (using the same name). > Try if you can rebuild the array. > > Could you please try it and poste the results here? > > > > > >At the moment, i get 70 mb/s sequential read speed locally. I > >would like to boost it to at least 100 or even more, as 1) the raw > >drives support way more and 2) i would like to fill my gigabit > >ethernet when copying files over the network. > > > >Now, what are my options? > > > >A quadcore CPU like the Q6600 would double the number of cores and > >theoretically double the throughput, but at cost of idle power. > >Note that the server is idle most of the time. > >A core i5. They have AES support in hardware, but it's an > >expensive solution and i'm not even sure it has Linux support. > >A PCI or PCIe based card, like the HiFN cards, but what card > >should i look for and what speed should i expect? > >Using the CUDA cores of my nVidia card, but no driver seems to > >exists for that. > > > >The first option is pretty straight forward, but what about the > >rest? Or are there any other options i havent thought of? > > > > I have a setup with an i5 and another one with q6600. Notice that > the q6600 does not fit on the same motherboards as the i5. > dmcrypt/luks is used on top of the raid. The performance of the i5 > is not great, despite hardware aes. Should not be this numbers a bit > higher than 158 MB/sec? > > ~/httptunnel-3.3/ hdparm -t --direct /dev/md1 > > /dev/md1: > Timing O_DIRECT disk reads: 936 MB in 3.00 seconds = 311.67 MB/sec > > ~/httptunnel-3.3/ hdparm -t --direct /dev/mapper/evol > > /dev/mapper/evol: > Timing O_DIRECT disk reads: 476 MB in 3.01 seconds = 158.30 MB/sec > > cat /proc/cpuinfo | grep -E (model name|aes) > model name : Intel(R) Core(TM) i5 CPU 660 @ 3.33GHz > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr > pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm > pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good > xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl > vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt aes > lahf_lm ida arat dts tpr_shadow vnmi flexpriority ept vpid > > > Regards, > Markus Krainz > > >-- > >Lasse Jensen (fafler at gmail dot com) > > > > > >_______________________________________________ > >dm-crypt mailing list > >dm-crypt@saout.de > >http://www.saout.de/mailman/listinfo/dm-crypt > > _______________________________________________ > dm-crypt mailing list > dm-crypt@saout.de > http://www.saout.de/mailman/listinfo/dm-crypt