RE: Question on Ceph LRC design

RE: Question on Ceph LRC design

[Zhou, Yuan]

[resend with plain format, sorry for the duplicated mail]

Hi Loic/Anderas,

I was trying to understand the LRC design in Ceph EC. Per my understanding, it seems Ceph was using a slightly different design with the Microsoft LRC: the local parities were calculated with the global parities included. Is there any special consideration on this change? 
I was asking because in a typical MS LRC design the global and local parities could be calculated at the same time actually(I mean inside the Erasure Code library). But with this new design, we lost this potential optimization.

Thanks, -Yuan

Re: Question on Ceph LRC design

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1521 bytes --]

Hi,

I believe Andreas has a more elaborate answer on this topic. The current implementation is not as good as what is described in the paper you mention, this is correct. My incentive for chosing this path is that I was able to understand it, mainly. It is not much more than stacking layers of erasure coded chunks on top of each other ;-)

Now that we have this plugin, it would be nice to have another implementation that uses less space and possible even less network when reconstructing. During the OpenStack summit we discussed this with Kevin Greenan and there are promising directions. It would help a lot to have a sample code to study so that it can be adapted to what we have currently in Ceph. Do you know of such an implementation of LRC or other similar code designed to reduce the network bandwidth during reconstruction ?

Cheers

On 17/11/2014 01:52, Zhou, Yuan wrote:
> Hi Loic/Anderas,
> 
>  
> 
> I was trying to understand the LRC design in Ceph EC. Per my understanding, it seems Ceph was using a slightly different design with the Microsoft LRC: the local parities were calculated with the global parities included. Is there any special consideration on this change?
> 
> I was asking because in a typical MS LRC design the global and local parities could be calculated at the same time actually(I mean inside the Erasure Code library). But with this new design, we lost this potential optimization.
> 
>  
> 
> Thanks, -Yuan
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

RE: Question on Ceph LRC design

[Andreas Joachim Peters]

Hi,

if you want to compute local parities within a plug-in library by adding rows to the encoding matrix, you can still do that. If you can do LRC inside the plug-in more efficienty, one should do it in the plug-in and skip the LRC layer. I have a patch to the ISA plug-in which can do LRC like HDFS Xorbas encoding, but i didn't manage to express that in Loic's generic LRC layer  ... I think it is hard to formulate a generic framework to define LRC as a layer on top and doing the encoding/decoding only in the lowest layer.  Same problem for the shingled EC, which you cannot express in the simple LRC way ...

I see the current LRC layering as a generic tool to use in multi-datacenter setups honouring the locality of chunks because some  of the LRC optimization codes break locality by cross-talk between locations, they reduce the traffic but do not restrict traffic within data centers.

Cheers Andreas.
________________________________________
From: Loic Dachary [loic@dachary.org]
Sent: 17 November 2014 02:04
To: Zhou, Yuan; Andreas Joachim Peters
Cc: ceph-devel@vger.kernel.org
Subject: Re: Question on Ceph LRC design

Hi,

I believe Andreas has a more elaborate answer on this topic. The current implementation is not as good as what is described in the paper you mention, this is correct. My incentive for chosing this path is that I was able to understand it, mainly. It is not much more than stacking layers of erasure coded chunks on top of each other ;-)

Now that we have this plugin, it would be nice to have another implementation that uses less space and possible even less network when reconstructing. During the OpenStack summit we discussed this with Kevin Greenan and there are promising directions. It would help a lot to have a sample code to study so that it can be adapted to what we have currently in Ceph. Do you know of such an implementation of LRC or other similar code designed to reduce the network bandwidth during reconstruction ?

Cheers

On 17/11/2014 01:52, Zhou, Yuan wrote:
> Hi Loic/Anderas,
>
>
>
> I was trying to understand the LRC design in Ceph EC. Per my understanding, it seems Ceph was using a slightly different design with the Microsoft LRC: the local parities were calculated with the global parities included. Is there any special consideration on this change?
>
> I was asking because in a typical MS LRC design the global and local parities could be calculated at the same time actually(I mean inside the Erasure Code library). But with this new design, we lost this potential optimization.
>
>
>
> Thanks, -Yuan
>

--
Loïc Dachary, Artisan Logiciel Libre

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question on Ceph LRC design

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 3054 bytes --]


Hi Andreas,

On 17/11/2014 11:07, Andreas Joachim Peters wrote:> Hi,
> 
> if you want to compute local parities within a plug-in library by adding rows to the encoding matrix, you can still do that. If you can do LRC inside the plug-in more efficienty, one should do it in the plug-in and skip the LRC layer. I have a patch to the ISA plug-in which can do LRC like HDFS Xorbas encoding, but i didn't manage to express that in Loic's generic LRC layer  ... I think it is hard to formulate a generic framework to define LRC as a layer on top and doing the encoding/decoding only in the lowest layer.  Same problem for the shingled EC, which you cannot express in the simple LRC way ...

Could you please remind us the URL of this draft ?

> 
> I see the current LRC layering as a generic tool to use in multi-datacenter setups honouring the locality of chunks because some  of the LRC optimization codes break locality by cross-talk between locations, they reduce the traffic but do not restrict traffic within data centers.

It certainly has its use but it would be nice to have a more specialized and better optimized version that implements most subtle algorithms ;-)

Cheers

> 
> Cheers Andreas.
> ________________________________________
> From: Loic Dachary [loic@dachary.org]
> Sent: 17 November 2014 02:04
> To: Zhou, Yuan; Andreas Joachim Peters
> Cc: ceph-devel@vger.kernel.org
> Subject: Re: Question on Ceph LRC design
> 
> Hi,
> 
> I believe Andreas has a more elaborate answer on this topic. The current implementation is not as good as what is described in the paper you mention, this is correct. My incentive for chosing this path is that I was able to understand it, mainly. It is not much more than stacking layers of erasure coded chunks on top of each other ;-)
> 
> Now that we have this plugin, it would be nice to have another implementation that uses less space and possible even less network when reconstructing. During the OpenStack summit we discussed this with Kevin Greenan and there are promising directions. It would help a lot to have a sample code to study so that it can be adapted to what we have currently in Ceph. Do you know of such an implementation of LRC or other similar code designed to reduce the network bandwidth during reconstruction ?
> 
> Cheers
> 
> On 17/11/2014 01:52, Zhou, Yuan wrote:
>> Hi Loic/Anderas,
>>
>>
>>
>> I was trying to understand the LRC design in Ceph EC. Per my understanding, it seems Ceph was using a slightly different design with the Microsoft LRC: the local parities were calculated with the global parities included. Is there any special consideration on this change?
>>
>> I was asking because in a typical MS LRC design the global and local parities could be calculated at the same time actually(I mean inside the Erasure Code library). But with this new design, we lost this potential optimization.
>>
>>
>>
>> Thanks, -Yuan
>>
> 
> --
> Loïc Dachary, Artisan Logiciel Libre
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]