Re: Intel Erasure Code library

[Loic Dachary <loic@dachary.org>]

[-- Attachment #1: Type: text/plain, Size: 5097 bytes --]



On 16/12/2013 14:10, Andreas Joachim Peters wrote:
> Hi Loic, 
> this looks excellent on new INTEL hardware .... certainly ;-)

:-) For sure.

> I was just running some benchmarks on different platforms. It should be extremely simple to make a plugin ... the API is simple ...
> 
> I configured (10+4):

I'd like to try the same command you did. Could you include the command you ran ? And if you have modified the source the corresponding patch ? 


> 
> 1)
> model name	: Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
> stepping	: 7
> cpu MHz		: 2001.000
> cache size	: 20480 KB
> 
> (cold cache)
> erasure_code_encode_cold: runtime =     684410 usecs, bandwidth 3199 MB in 0.6844 sec = 4902.61 MB/s
> erasure_code_decode_cold: runtime =     340002 usecs, bandwidth 2239 MB in 0.3400 sec = 6908.12 MB/s (recover 3)
> erasure_code_decode_cold: runtime =     176811 usecs, bandwidth 1599 MB in 0.1768 sec = 9488.64 MB/s (recover 1)
> 
> (warm cache)
> erasure_code_encode_warm: runtime =    1744615 usecs, bandwidth 9985 MB in 1.7446 sec = 6001.55 MB/s
> erasure_code_decode_warm: runtime =     860179 usecs, bandwidth 6989 MB in 0.8602 sec = 8520.65 MB/s (recover 3)
> 
> 
> 2)
> model name      : AMD Opteron(TM) Processor 6276                 
> stepping        : 2
> cpu MHz         : 2300.000
> cache size      : 2048 KB
> 
> (cold cache)
> erasure_code_encode_cold: runtime =    1769806 usecs, bandwidth 3199 MB in 1.7698 sec = 1895.91 MB/s
> erasure_code_decode_cold: runtime =    1135461 usecs, bandwidth 2239 MB in 1.1355 sec = 2068.56 MB/s (recover 3)
> erasure_code_decode_cold: runtime =     553315 usecs, bandwidth 1599 MB in 0.5533 sec = 3032.08 MB/s (recover 1)
> 
> (warm cache)
> erasure_code_encode_warm: runtime =    2158180 usecs, bandwidth 9985 MB in 2.1582 sec = 4851.50 MB/s
> erasure_code_decode_warm: runtime =    1133059 usecs, bandwidth 6989 MB in 1.1331 sec = 6468.58 MB/s (recover 3)
> 
> 
> 3) [running in a virtual machine!!!]
> 
> model name	: Intel Core i7 9xx (Nehalem Class Core i7)
> stepping	: 3
> cpu MHz		: 2266.746
> cache size	: 4096 KB
> 
> (cold cache)
> erasure_code_encode_cold: runtime =    1125543 usecs, bandwidth 3199 MB in 1.1255 sec = 2981.13 MB/s
> erasure_code_decode_cold: runtime =     554727 usecs, bandwidth 2239 MB in 0.5547 sec = 4234.11 MB/s (recover 3)
> erasure_code_decode_cold: runtime =     324933 usecs, bandwidth 1599 MB in 0.3249 sec = 5163.21 MB/s (recover 1)
> 
> (warm cache)
> erasure_code_encode_warm: runtime =    2636439 usecs, bandwidth 9985 MB in 2.6364 sec = 3971.42 MB/s
> erasure_code_decode_warm: runtime =    1263597 usecs, bandwidth 6989 MB in 1.2636 sec = 5800.33 MB/s (recover 3)
> 
> 4)
> model name      : Intel(R) Xeon(R) CPU           L5520  @ 2.27GHz
> stepping        : 5
> cpu MHz         : 1600.000
> cache size      : 8192 KB
> 
> (cold cache)
> erasure_code_encode_cold: runtime =    1166904 usecs, bandwidth 3199 MB in 1.1669 sec = 2875.47 MB/s
> erasure_code_decode_cold: runtime =     345337 usecs, bandwidth 1599 MB in 0.3453 sec = 4858.14 MB/s (recover 1)
> erasure_code_decode_cold: runtime =     575194 usecs, bandwidth 2239 MB in 0.5752 sec = 4083.45 MB/s (recover 3)
> 
> (warm cache)
> erasure_code_encode_warm: runtime =    2686134 usecs, bandwidth 9985 MB in 2.6861 sec = 3897.94 MB/s
> erasure_code_decode_warm: runtime =     777904 usecs, bandwidth 4992 MB in 0.7779 sec = 6729.88 MB/s (recover 1)
> erasure_code_decode_warm: runtime =    1278117 usecs, bandwidth 6989 MB in 1.2781 sec = 5734.44 MB/s (recover 3)
> 
> 5)
> model name	: Intel(R) Xeon(R) CPU           L5520  @ 2.27GHz
> stepping	: 5
> cpu MHz		: 1600.000
> cache size	: 8192 KB
> 
> (cold cache)
> erasure_code_encode_cold: runtime =     956274 usecs, bandwidth 3199 MB in 0.9563 sec = 3508.82 MB/s
> erasure_code_decode_cold: runtime =     264835 usecs, bandwidth 1599 MB in 0.2648 sec = 6334.87 MB/s (recover 1)
> erasure_code_decode_cold: runtime =     463120 usecs, bandwidth 2239 MB in 0.4631 sec = 5071.63 MB/s (recover 3)
> 
> 
> (warm cache)
> erasure_code_encode_warm: runtime =    2601232 usecs, bandwidth 9985 MB in 2.6012 sec = 4025.17 MB/s
> erasure_code_decode_warm: runtime =     683921 usecs, bandwidth 4992 MB in 0.6839 sec = 7654.69 MB/s (recover 1)
> erasure_code_decode_warm: runtime =    1258408 usecs, bandwidth 6989 MB in 1.2584 sec = 5824.25 MB/s (recover 3)
> 
> ___________________________________
> From: ceph-devel-owner@vger.kernel.org [ceph-devel-owner@vger.kernel.org] on behalf of Loic Dachary [loic@dachary.org]
> Sent: 16 December 2013 10:56
> To: Ceph Development
> Subject: Intel Erasure Code library
> 
> Hi,
> 
> A new Erasure Code library has been released as Free Software
> 
> https://01.org/storage-acceleration-library/downloads/2013/isa-l-open-source-release
> 
> It is quite tempting to write a Ceph erasure code plugin for it and see how it performs :-)
> 
> Cheers
> 
> --
> Loïc Dachary, Artisan Logiciel Libre
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]