Jerasure 1.2A plugin for Ceph

Jerasure 1.2A plugin for Ceph

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 709 bytes --]

Hi James,

The first version of the jerasure 1.2A plugin for Ceph is complete at

https://github.com/ceph/ceph/pull/538#commits-pushed-763275e

This commit introduces the main part:
ErasureCodeJerasure: base class for jerasure ErasureCodeInterface
https://github.com/dachary/ceph/commit/76d2842358465e560a4929d60131762f8c93804f

and each technique is derived from it in six successive commits, starting from here
ErasureCodeJerasure: define technique ReedSolomonVandermonde

It would be great if you could take a look and let us know if you see anything odd.

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]

Re: Jerasure 1.2A plugin for Ceph

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1611 bytes --]

[cc'ing the devel mailing list]

Hi James,

Thanks for the positive feedback :-) I found the API fairly straightforward and I don't see what could be improved. However, I've learnt about erasure code and designed the Ceph plugin abstract API mostly by studying jerasure. I'm probably biased : let hear what others have to suggest.

Cheers

On 11/09/2013 17:04, James Plank wrote:
> Hi Loic -- I took a scan, and it looks like a nice usage of the library.  Here's a question -- for the next revision of jerasure, are there any functionalities that you would like to see, that would make your life as a developer easier?  Thanks!  Jim
> ----------
> 
> On Aug 30, 2013, at 4:20 AM, Loic Dachary wrote:
> 
>> Hi James,
>>
>> The first version of the jerasure 1.2A plugin for Ceph is complete at
>>
>> https://github.com/ceph/ceph/pull/538#commits-pushed-763275e
>>
>> This commit introduces the main part:
>> ErasureCodeJerasure: base class for jerasure ErasureCodeInterface
>> https://github.com/dachary/ceph/commit/76d2842358465e560a4929d60131762f8c93804f
>>
>> and each technique is derived from it in six successive commits, starting from here
>> ErasureCodeJerasure: define technique ReedSolomonVandermonde
>>
>> It would be great if you could take a look and let us know if you see anything odd.
>>
>> Cheers
>>
>> -- 
>> Loïc Dachary, Artisan Logiciel Libre
>> All that is necessary for the triumph of evil is that good people do nothing.
>>
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]

Re: Jerasure 1.2A plugin for Ceph

[Sage Weil]

On Wed, 11 Sep 2013, Loic Dachary wrote:
> [cc'ing the devel mailing list]
> 
> Hi James,
> 
> Thanks for the positive feedback :-) I found the API fairly 
> straightforward and I don't see what could be improved. However, I've 
> learnt about erasure code and designed the Ceph plugin abstract API 
> mostly by studying jerasure. I'm probably biased : let hear what others 
> have to suggest.
> 
> Cheers
> 
> On 11/09/2013 17:04, James Plank wrote:
> > Hi Loic -- I took a scan, and it looks like a nice usage of the 
> > library.  Here's a question -- for the next revision of jerasure, are 
> > there any functionalities that you would like to see, that would make 
> > your life as a developer easier?  Thanks!  Jim ----------

Hi James-

Late to this discussion, and this may have already been covered, but: 
there is a bunch of Intel-optimized code floating around for various 
erasure codes.  I would love to see this incorporated into jerasure (with 
automatic arch and CPU feature detection) so that users can transparently 
take advantage of it.

sage


> > 
> > On Aug 30, 2013, at 4:20 AM, Loic Dachary wrote:
> > 
> >> Hi James,
> >>
> >> The first version of the jerasure 1.2A plugin for Ceph is complete at
> >>
> >> https://github.com/ceph/ceph/pull/538#commits-pushed-763275e
> >>
> >> This commit introduces the main part:
> >> ErasureCodeJerasure: base class for jerasure ErasureCodeInterface
> >> https://github.com/dachary/ceph/commit/76d2842358465e560a4929d60131762f8c93804f
> >>
> >> and each technique is derived from it in six successive commits, starting from here
> >> ErasureCodeJerasure: define technique ReedSolomonVandermonde
> >>
> >> It would be great if you could take a look and let us know if you see anything odd.
> >>
> >> Cheers
> >>
> >> -- 
> >> Lo?c Dachary, Artisan Logiciel Libre
> >> All that is necessary for the triumph of evil is that good people do nothing.
> >>
> > 
> 
> -- 
> Lo?c Dachary, Artisan Logiciel Libre
> All that is necessary for the triumph of evil is that good people do nothing.
> 
> 

Re: Jerasure 1.2A plugin for Ceph

[Mark Nelson]

On 09/11/2013 11:29 AM, Sage Weil wrote:
> On Wed, 11 Sep 2013, Loic Dachary wrote:
>> [cc'ing the devel mailing list]
>>
>> Hi James,
>>
>> Thanks for the positive feedback :-) I found the API fairly
>> straightforward and I don't see what could be improved. However, I've
>> learnt about erasure code and designed the Ceph plugin abstract API
>> mostly by studying jerasure. I'm probably biased : let hear what others
>> have to suggest.
>>
>> Cheers
>>
>> On 11/09/2013 17:04, James Plank wrote:
>>> Hi Loic -- I took a scan, and it looks like a nice usage of the
>>> library.  Here's a question -- for the next revision of jerasure, are
>>> there any functionalities that you would like to see, that would make
>>> your life as a developer easier?  Thanks!  Jim ----------
>
> Hi James-
>
> Late to this discussion, and this may have already been covered, but:
> there is a bunch of Intel-optimized code floating around for various
> erasure codes.  I would love to see this incorporated into jerasure (with
> automatic arch and CPU feature detection) so that users can transparently
> take advantage of it.
>
> sage
>

+1.  Same for ARM down the road. :D

Mark

>
>>>
>>> On Aug 30, 2013, at 4:20 AM, Loic Dachary wrote:
>>>
>>>> Hi James,
>>>>
>>>> The first version of the jerasure 1.2A plugin for Ceph is complete at
>>>>
>>>> https://github.com/ceph/ceph/pull/538#commits-pushed-763275e
>>>>
>>>> This commit introduces the main part:
>>>> ErasureCodeJerasure: base class for jerasure ErasureCodeInterface
>>>> https://github.com/dachary/ceph/commit/76d2842358465e560a4929d60131762f8c93804f
>>>>
>>>> and each technique is derived from it in six successive commits, starting from here
>>>> ErasureCodeJerasure: define technique ReedSolomonVandermonde
>>>>
>>>> It would be great if you could take a look and let us know if you see anything odd.
>>>>
>>>> Cheers
>>>>
>>>> --
>>>> Lo?c Dachary, Artisan Logiciel Libre
>>>> All that is necessary for the triumph of evil is that good people do nothing.
>>>>
>>>
>>
>> --
>> Lo?c Dachary, Artisan Logiciel Libre
>> All that is necessary for the triumph of evil is that good people do nothing.
>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Jerasure 1.2A plugin for Ceph

[James Plank]

That's definitely down the road, once I get gf_complete, well, complete.   Thanks for the feedback -- Jim
----------

On Sep 11, 2013, at 12:32 PM, Mark Nelson wrote:

> On 09/11/2013 11:29 AM, Sage Weil wrote:
>> On Wed, 11 Sep 2013, Loic Dachary wrote:
>>> [cc'ing the devel mailing list]
>>> 
>>> Hi James,
>>> 
>>> Thanks for the positive feedback :-) I found the API fairly
>>> straightforward and I don't see what could be improved. However, I've
>>> learnt about erasure code and designed the Ceph plugin abstract API
>>> mostly by studying jerasure. I'm probably biased : let hear what others
>>> have to suggest.
>>> 
>>> Cheers
>>> 
>>> On 11/09/2013 17:04, James Plank wrote:
>>>> Hi Loic -- I took a scan, and it looks like a nice usage of the
>>>> library.  Here's a question -- for the next revision of jerasure, are
>>>> there any functionalities that you would like to see, that would make
>>>> your life as a developer easier?  Thanks!  Jim ----------
>> 
>> Hi James-
>> 
>> Late to this discussion, and this may have already been covered, but:
>> there is a bunch of Intel-optimized code floating around for various
>> erasure codes.  I would love to see this incorporated into jerasure (with
>> automatic arch and CPU feature detection) so that users can transparently
>> take advantage of it.
>> 
>> sage
>> 
> 
> +1.  Same for ARM down the road. :D
> 
> Mark
> 
>> 
>>>> 
>>>> On Aug 30, 2013, at 4:20 AM, Loic Dachary wrote:
>>>> 
>>>>> Hi James,
>>>>> 
>>>>> The first version of the jerasure 1.2A plugin for Ceph is complete at
>>>>> 
>>>>> https://github.com/ceph/ceph/pull/538#commits-pushed-763275e
>>>>> 
>>>>> This commit introduces the main part:
>>>>> ErasureCodeJerasure: base class for jerasure ErasureCodeInterface
>>>>> https://github.com/dachary/ceph/commit/76d2842358465e560a4929d60131762f8c93804f
>>>>> 
>>>>> and each technique is derived from it in six successive commits, starting from here
>>>>> ErasureCodeJerasure: define technique ReedSolomonVandermonde
>>>>> 
>>>>> It would be great if you could take a look and let us know if you see anything odd.
>>>>> 
>>>>> Cheers
>>>>> 
>>>>> --
>>>>> Lo?c Dachary, Artisan Logiciel Libre
>>>>> All that is necessary for the triumph of evil is that good people do nothing.
>>>>> 
>>>> 
>>> 
>>> --
>>> Lo?c Dachary, Artisan Logiciel Libre
>>> All that is necessary for the triumph of evil is that good people do nothing.
>>> 
>>> 
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 
>