Re: [dm-crypt] Improving performance?

[epvdm@limpoc.com]

On Thu, Nov 11, 2010 at 06:59:47PM +0100, Markus Krainz wrote:
> Hi.
> 
> On 2010-11-11 11:49, Lasse Jensen wrote:
> >Hi. I have a RAID 5 array with 3 (soon upgrading to 4 + hotspare =
> >5) encrypted drives connected to a system with a Core 2 Duo @ 2.5
> >ghz  running Debian Squeeze.

For what it's worth I use luks on top of mirrored pairs of drives and have 
frequently lost and replaced one drive of a pair without problems. 

eric

> >Each drive has been formatted with
> >
> >cryptsetup luksFormat /path/to/device
> >
> >And put together in a array with
> >
> >mdamd -C /dev/md0 --raid-level=5 /path/to/first-device
> >/path/to/third-device /path/to/third-device
> >
> >It works great, and encrypting the devices separately allows me to
> >run more than one instance of kcryptd, thus using both cores in my
> >server. It compensates for the overhead of encrypting the
> >checksumming data seperately, compared to raw devices -> RAID ->
> >encryption and still give me improved speed.
> 
> Have you ever done the test and unplugged a drive from your raid and
> assessed if the raid5 still works?
> My experience with this setup is that cryptsetup and mdadm do not
> work well, if the underlying device suddenly disappears.
> 
> First unplug 1 of your 3 drives, look if it still works and mdadm
> recognises the missing drive using mdadm --detail /dev/raidname
> Then reconnect the drive without restarting the computer simulating
> a new device to replace the old one.
> Try if you can still open it with cryptsetup (using the same name).
> Try if you can rebuild the array.
> 
> Could you please try it and poste the results here?
> 
> 
> >
> >At the moment, i get 70 mb/s sequential read speed locally. I
> >would like to boost it to at least 100 or even more, as 1) the raw
> >drives support way more and 2) i would like to fill my gigabit
> >ethernet when copying files over the network.
> >
> >Now, what are my options?
> >
> >A quadcore CPU like the Q6600 would double the number of cores and
> >theoretically double the throughput, but at cost of idle power.
> >Note that the server is idle most of the time.
> >A core i5. They have AES support in hardware, but it's an
> >expensive solution and i'm not even sure it has Linux support.
> >A PCI or PCIe based card, like the HiFN cards, but what card
> >should i look for and what speed should i expect?
> >Using the CUDA cores of my nVidia card, but no driver seems to
> >exists for that.
> >
> >The first option is pretty straight forward, but what about the
> >rest? Or are there any other options i havent thought of?
> >
> 
> I have a setup with an i5 and another one with q6600. Notice that
> the q6600 does not fit on the same motherboards as the i5.
> dmcrypt/luks is used on top of the raid. The performance of the i5
> is not great, despite hardware aes. Should not be this numbers a bit
> higher than 158 MB/sec?
> 
> ~/httptunnel-3.3/ hdparm -t --direct /dev/md1
> 
> /dev/md1:
>  Timing O_DIRECT disk reads:  936 MB in  3.00 seconds = 311.67 MB/sec
> 
> ~/httptunnel-3.3/ hdparm -t --direct /dev/mapper/evol
> 
> /dev/mapper/evol:
>  Timing O_DIRECT disk reads:  476 MB in  3.01 seconds = 158.30 MB/sec
> 
> cat /proc/cpuinfo | grep -E (model name|aes)
> model name      : Intel(R) Core(TM) i5 CPU         660  @ 3.33GHz
> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
> pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
> pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good
> xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl
> vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt aes
> lahf_lm ida arat dts tpr_shadow vnmi flexpriority ept vpid
> 
> 
> Regards,
> Markus Krainz
> 
> >-- 
> >Lasse Jensen (fafler at gmail dot com)
> >
> >
> >_______________________________________________
> >dm-crypt mailing list
> >dm-crypt@saout.de
> >http://www.saout.de/mailman/listinfo/dm-crypt
> 

> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt