* fecpp C++ forward error correction library @ 2013-04-22 13:19 ` Loic Dachary 2013-04-29 14:06 ` Jimmy Tang 2013-04-30 0:08 ` Christopher LILJENSTOLPE 0 siblings, 2 replies; 6+ messages in thread From: Loic Dachary @ 2013-04-22 13:19 UTC (permalink / raw) To: Christopher LILJENSTOLPE; +Cc: Ceph Development, Jack Lloyd [-- Attachment #1: Type: text/plain, Size: 406 bytes --] Hi Christopher, Jack Lloyd is the author of fecpp ( http://www.randombit.net/code/fecpp/ ) and he tells me someone sent him a new SIMD approach a few weeks ago. I'm not sure what SIMD means yet, but I'll figure it out ;-). I tend to favor fecpp because it is more self contained and may be easier to embed than https://pypi.python.org/pypi/zfec Cheers -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 262 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: fecpp C++ forward error correction library 2013-04-22 13:19 ` fecpp C++ forward error correction library Loic Dachary @ 2013-04-29 14:06 ` Jimmy Tang 2013-04-29 14:23 ` Loic Dachary 2013-04-30 0:08 ` Christopher LILJENSTOLPE 1 sibling, 1 reply; 6+ messages in thread From: Jimmy Tang @ 2013-04-29 14:06 UTC (permalink / raw) To: Loic Dachary; +Cc: Christopher LILJENSTOLPE, Ceph Development, Jack Lloyd On 22 Apr 2013, at 14:19, Loic Dachary wrote: > Hi Christopher, > > Jack Lloyd is the author of fecpp ( http://www.randombit.net/code/fecpp/ ) and he tells me someone sent him a new SIMD approach a few weeks ago. I'm not sure what SIMD means yet, but I'll figure it out ;-). I tend to favor fecpp because it is more self contained and may be easier to embed than https://pypi.python.org/pypi/zfec > > Cheers > > -- > Loïc Dachary, Artisan Logiciel Libre > Having played with zfec (more the underlying fec.[c,h] file) it's well written and self-contained as I remember it; but you will need to write a bunch of wrapper functions to chunk up data and pad out blocks and of course turn it into a library before you can use the underlying zfec implementation for ceph. If my memory is right, zfec's c code requires a C90 or C99 compiler to build it, I'm not sure if you care about that. Regards, Jimmy Tang -- Senior Software Engineer, Digital Repository of Ireland (DRI) High Performance & Research Computing, IS Services Lloyd Building, Trinity College Dublin, Dublin 2, Ireland. http://www.tchpc.tcd.ie/ | jtang@tchpc.tcd.ie Tel: +353-1-896-3847 -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: fecpp C++ forward error correction library 2013-04-29 14:06 ` Jimmy Tang @ 2013-04-29 14:23 ` Loic Dachary 2013-04-30 0:12 ` Christopher LILJENSTOLPE 0 siblings, 1 reply; 6+ messages in thread From: Loic Dachary @ 2013-04-29 14:23 UTC (permalink / raw) To: Jimmy Tang; +Cc: Christopher LILJENSTOLPE, Ceph Development, Jack Lloyd [-- Attachment #1: Type: text/plain, Size: 1496 bytes --] On 04/29/2013 04:06 PM, Jimmy Tang wrote: > > On 22 Apr 2013, at 14:19, Loic Dachary wrote: > >> Hi Christopher, >> >> Jack Lloyd is the author of fecpp ( http://www.randombit.net/code/fecpp/ ) and he tells me someone sent him a new SIMD approach a few weeks ago. I'm not sure what SIMD means yet, but I'll figure it out ;-). I tend to favor fecpp because it is more self contained and may be easier to embed than https://pypi.python.org/pypi/zfec >> >> Cheers >> >> -- >> Loïc Dachary, Artisan Logiciel Libre >> > > > Having played with zfec (more the underlying fec.[c,h] file) it's well written and self-contained as I remember it; but you will need to write a bunch of wrapper functions to chunk up data and pad out blocks and of course turn it into a library before you can use the underlying zfec implementation for ceph. If my memory is right, zfec's c code requires a C90 or C99 compiler to build it, I'm not sure if you care about that. Hi Jimmy, fecpp and zfec compile and run using g++ 4.7 . The requirements for http://code.google.com/p/holostor/ may be different but I suspect it also works. Cheers > > Regards, > Jimmy Tang > > -- > Senior Software Engineer, Digital Repository of Ireland (DRI) > High Performance & Research Computing, IS Services > Lloyd Building, Trinity College Dublin, Dublin 2, Ireland. > http://www.tchpc.tcd.ie/ | jtang@tchpc.tcd.ie > Tel: +353-1-896-3847 > -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 262 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: fecpp C++ forward error correction library 2013-04-29 14:23 ` Loic Dachary @ 2013-04-30 0:12 ` Christopher LILJENSTOLPE 0 siblings, 0 replies; 6+ messages in thread From: Christopher LILJENSTOLPE @ 2013-04-30 0:12 UTC (permalink / raw) To: Loic Dachary; +Cc: Jimmy Tang, Ceph Development, Jack Lloyd [-- Attachment #1: Type: text/plain, Size: 2046 bytes --] Supposedly, on 2013-Apr-29, at 07.23 PDT(-0700), someone claiming to be Loic Dachary scribed: > On 04/29/2013 04:06 PM, Jimmy Tang wrote: >> >> On 22 Apr 2013, at 14:19, Loic Dachary wrote: >> >>> Hi Christopher, >>> >>> Jack Lloyd is the author of fecpp ( http://www.randombit.net/code/fecpp/ ) and he tells me someone sent him a new SIMD approach a few weeks ago. I'm not sure what SIMD means yet, but I'll figure it out ;-). I tend to favor fecpp because it is more self contained and may be easier to embed than https://pypi.python.org/pypi/zfec >>> >>> Cheers >>> >>> -- >>> Loïc Dachary, Artisan Logiciel Libre >>> >> >> >> Having played with zfec (more the underlying fec.[c,h] file) it's well written and self-contained as I remember it; but you will need to write a bunch of wrapper functions to chunk up data and pad out blocks and of course turn it into a library before you can use the underlying zfec implementation for ceph. If my memory is right, zfec's c code requires a C90 or C99 compiler to build it, I'm not sure if you care about that. > > Hi Jimmy, > > fecpp and zfec compile and run using g++ 4.7 . The requirements for http://code.google.com/p/holostor/ may be different but I suspect it also works. Greetings - just looked at holostor - some concerns, it looks like it caps out (per the comments) at 17:21. As I know of at least one case where I needed to do something like 10:18, that might already violate holostor (n-m > 4) or definitely push up to its m boundary. Christopher > > Cheers > >> >> Regards, >> Jimmy Tang >> >> -- >> Senior Software Engineer, Digital Repository of Ireland (DRI) >> High Performance & Research Computing, IS Services >> Lloyd Building, Trinity College Dublin, Dublin 2, Ireland. >> http://www.tchpc.tcd.ie/ | jtang@tchpc.tcd.ie >> Tel: +353-1-896-3847 >> > > -- > Loïc Dachary, Artisan Logiciel Libre -- 李柯睿 Check my PGP key here: http://www.asgaard.org/cdl/cdl.asc Current vCard here: http://www.asgaard.org/cdl/cdl.vcf [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 495 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: fecpp C++ forward error correction library 2013-04-22 13:19 ` fecpp C++ forward error correction library Loic Dachary 2013-04-29 14:06 ` Jimmy Tang @ 2013-04-30 0:08 ` Christopher LILJENSTOLPE 2013-04-30 8:11 ` Loic Dachary 1 sibling, 1 reply; 6+ messages in thread From: Christopher LILJENSTOLPE @ 2013-04-30 0:08 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development, Jack Lloyd [-- Attachment #1: Type: text/plain, Size: 1187 bytes --] Supposedly, on 2013-Apr-22, at 06.19 PDT(-0700), someone claiming to be Loic Dachary scribed: > Hi Christopher, > > Jack Lloyd is the author of fecpp ( http://www.randombit.net/code/fecpp/ ) and he tells me someone sent him a new SIMD approach a few weeks ago. I'm not sure what SIMD means yet, but I'll figure it out ;-). I tend to favor fecpp because it is more self contained and may be easier to embed than https://pypi.python.org/pypi/zfec I'll defer, provided that we don't incur a performance penalty (which is one of the negatives in sharding). As far as SIMD: Single Instruction Multiple Data. Basically, if you think of a tiled processor (like a GPU), then you set all the tiles to do the same instruction, and stream different data to each tile. As opposed to MIMD, where each tile runs a different instruction on different data. You don't need parallel processing CPUs to make use of SIMD, but you can get them going screamingly fast if you do… Christopher > > Cheers > > -- > Loïc Dachary, Artisan Logiciel Libre -- 李柯睿 Check my PGP key here: http://www.asgaard.org/cdl/cdl.asc Current vCard here: http://www.asgaard.org/cdl/cdl.vcf [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 495 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: fecpp C++ forward error correction library 2013-04-30 0:08 ` Christopher LILJENSTOLPE @ 2013-04-30 8:11 ` Loic Dachary 0 siblings, 0 replies; 6+ messages in thread From: Loic Dachary @ 2013-04-30 8:11 UTC (permalink / raw) To: Christopher LILJENSTOLPE; +Cc: Ceph Development, Jack Lloyd [-- Attachment #1: Type: text/plain, Size: 1521 bytes --] On 04/30/2013 02:08 AM, Christopher LILJENSTOLPE wrote: > Supposedly, on 2013-Apr-22, at 06.19 PDT(-0700), someone claiming to be Loic Dachary scribed: > >> Hi Christopher, >> >> Jack Lloyd is the author of fecpp ( http://www.randombit.net/code/fecpp/ ) and he tells me someone sent him a new SIMD approach a few weeks ago. I'm not sure what SIMD means yet, but I'll figure it out ;-). I tend to favor fecpp because it is more self contained and may be easier to embed than https://pypi.python.org/pypi/zfec > > I'll defer, provided that we don't incur a performance penalty (which is one of the negatives in sharding). As far as SIMD: Single Instruction Multiple Data. Basically, if you think of a tiled processor (like a GPU), then you set all the tiles to do the same instruction, and stream different data to each tile. As opposed to MIMD, where each tile runs a different instruction on different data. You don't need parallel processing CPUs to make use of SIMD, but you can get them going screamingly fast if you do… Hi Christopher, Thanks for explaining :-) Although it seems disk / network I/O are the bottleneck, it's good to be able to reduce the processing footprint to a minimum. Cheers > > Christopher > >> >> Cheers >> >> -- >> Loïc Dachary, Artisan Logiciel Libre > > > -- > 李柯睿 > Check my PGP key here: http://www.asgaard.org/cdl/cdl.asc > Current vCard here: http://www.asgaard.org/cdl/cdl.vcf -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 262 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-04-30 8:12 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <sfid-H20130422-141929-+033.82-1@marduk.tchpc.tcd.ie>
2013-04-22 13:19 ` fecpp C++ forward error correction library Loic Dachary
2013-04-29 14:06 ` Jimmy Tang
2013-04-29 14:23 ` Loic Dachary
2013-04-30 0:12 ` Christopher LILJENSTOLPE
2013-04-30 0:08 ` Christopher LILJENSTOLPE
2013-04-30 8:11 ` Loic Dachary
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.