Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure (or not)

[Loic Dachary <loic@dachary.org>]

On 20/04/2016 23:48, Chandan Kumar Singh wrote:
> Hi
> 
> As expected, Your observation in interesting. I had just glanced
> through the paper initially but will go through it again. I started
> looking at alternatives after reading about StreamScale Inc's claims
> on jerasure. As of now, I suppose ISA is the best bet.

Where did you read that StreamScale has claims on jerasure ? Again, the devil is in the details ;-)

> On Wed, Apr 20, 2016 at 10:01 PM, Loic Dachary <loic@dachary.org> wrote:
>>
>>
>> On 20/04/2016 20:10, Chandan Kumar Singh wrote:
>>> Hi
>>>
>>> Here is the link for the benchmark study :
>>> http://arxiv.org/pdf/1504.07038.pdf. Initially, the mailing list was
>>> blocking it because of HTML subparts.
>>
>> I meant that the first step would be to repeat the benchmark as described in the paper. Unless I missed something the details of how the results were obtained have not been published. It is enough to get just one detail wrong and see significantly different results.
>>
>> Here is an example. From an implementor point of view, the performance difference that matters is when reading data that has not been damaged (i.e. what happens 99.99% of the time). Mojette does not have systematic code. What it means is that reading data always requires decoding chunks. With systematic code, which is what jerasure / isa-l offer and what Ceph uses, reading erasure coded data does not require decoding. This is the Achilles' heel of this erasure code method. It is supposed to be measured by the graph in figure 3. For instance the bottom of figure 3 states that ISA-L requires 640 cycles to read 8KB while Mojette requires 1750 when there is no erasure (i.e. no chunk lost). That makes little sense to me as you don't need ISA-L at all to read the original 8KB (that's what systematic codes are good for). This detail would need to be clarified before the experiment can be repeated.
>>
>> Mojette is definitely interesting and it would be nice to see Free Software minded people working on it. There is a long way to go before it's useable, but exploring new avenues is fascinating :-)
>>
>> Cheers
>>
>>> Regards
>>> Chandan
>>>
>>> On Wed, Apr 20, 2016 at 6:28 PM, Loic Dachary <loic@dachary.org> wrote:
>>>> Hi,
>>>>
>>>> On 20/04/2016 15:52, Chandan Kumar Singh wrote:
>>>>> Hi
>>>>>
>>>>> The authors of  Mojette Erasure Encoding claim that this EC has higher
>>>>> performance than ISA-L and Jerasure and can be used for hot storage
>>>>> use cases. Their EC is used in open source RozoFS. What is your
>>>>> evaluation of it? It will be nice if it is available as another EC
>>>>> plugin.
>>>>
>>>> RozoFS is no longer developped (last commit was july last year). It would be nice to have an alternative implementation of the same algorithm, well maintained. It's sad to see Free Software being frozen. But Free Software is never dead, all it needs is good will and a little work to be resurected ! I've never seen any benchmark regarding performances, that would be the first step.
>>>>
>>>> Cheers
>>>>
>>>> --
>>>> Loïc Dachary, Artisan Logiciel Libre
>>>>
>>>
>>
>> --
>> Loïc Dachary, Artisan Logiciel Libre
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html