What crush ruleset for a given SHEC configuration ?

What crush ruleset for a given SHEC configuration ?

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1198 bytes --]

Hi Takeshi,

In the context of http://ceph.com/docs/master/rados/operations/erasure-code-shec/ it would be useful to have a more detailed explanation of why SHEC is more efficient during recovery (in the introduction). 

Am I correct to assume that SHEC does not provide a way to control the locality of the chunks ? For instance in the following scenario:

rack 1 has 10 OSDs
rack 2 has 10 OSDs

a crush ruleset is made to provide 15 OSDs with 7 in the first rack, 8 in the last rack: the first 7 are in rack 1, the last 8 in rack 2. When SHEC is used with such a crush ruleset, it cannot guarantee that the loss of one chunk in rack 2 can always be recovered with chunks from rack 2. When reading at figure 3 of

https://wiki.ceph.com/Planning/Blueprints/Hammer/Shingled_Erasure_Code_%28SHEC%29

with D1 to D5, P1 and P2 in rack 1 and D6 to D10, P3, P4, P5 in rack 2, my understanding is that to recover D6 which is in rack 2 it may be necessary to use P2 from rack 1. And to recover D5 which is in rack 1 it may be necessary to use P3 from rack 2.

Maybe I'm missing something ? Thanks in advance for your explanations :-)

Cheers
-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

RE: What crush ruleset for a given SHEC configuration ?

[Miyamae, Takeshi]

Hi Loic,

> to recover D6 which is in rack 2 it may be necessary to use P2 from rack 1

Because D6 can be recovered from P3 as well, P2 is not necessarily used to recover D6.
However, when D6 is recovered from P3, D5 which is in rack 1 must be read.
Therefore, I believe Ceph-LRC is more efficient when network resources is poor such as
in case of Geo-Replication. SHEC should be used in a single data center that has enough
network resources.

Best regards,
Takeshi Miyamae

-----Original Message-----
From: Loic Dachary [mailto:loic@dachary.org] 
Sent: Tuesday, May 19, 2015 3:39 PM
To: Miyamae, Takeshi/宮前 剛
Cc: Ceph Development
Subject: What crush ruleset for a given SHEC configuration ?

Hi Takeshi,

In the context of http://ceph.com/docs/master/rados/operations/erasure-code-shec/ it would be useful to have a more detailed explanation of why SHEC is more efficient during recovery (in the introduction). 

Am I correct to assume that SHEC does not provide a way to control the locality of the chunks ? For instance in the following scenario:

rack 1 has 10 OSDs
rack 2 has 10 OSDs

a crush ruleset is made to provide 15 OSDs with 7 in the first rack, 8 in the last rack: the first 7 are in rack 1, the last 8 in rack 2. When SHEC is used with such a crush ruleset, it cannot guarantee that the loss of one chunk in rack 2 can always be recovered with chunks from rack 2. When reading at figure 3 of

https://wiki.ceph.com/Planning/Blueprints/Hammer/Shingled_Erasure_Code_%28SHEC%29

with D1 to D5, P1 and P2 in rack 1 and D6 to D10, P3, P4, P5 in rack 2, my understanding is that to recover D6 which is in rack 2 it may be necessary to use P2 from rack 1. And to recover D5 which is in rack 1 it may be necessary to use P3 from rack 2.

Maybe I'm missing something ? Thanks in advance for your explanations :-)

Cheers
-- 
Loïc Dachary, Artisan Logiciel Libre

Re: What crush ruleset for a given SHEC configuration ?

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2121 bytes --]



On 19/05/2015 11:11, Miyamae, Takeshi wrote:
> Hi Loic,
> 
>> to recover D6 which is in rack 2 it may be necessary to use P2 from rack 1
> 
> Because D6 can be recovered from P3 as well, P2 is not necessarily used to recover D6.
> However, when D6 is recovered from P3, D5 which is in rack 1 must be read.
> Therefore, I believe Ceph-LRC is more efficient when network resources is poor such as
> in case of Geo-Replication. SHEC should be used in a single data center that has enough
> network resources.

Thanks for the detailed explanation, it is clear now :-)

Cheers

> 
> Best regards,
> Takeshi Miyamae
> 
> -----Original Message-----
> From: Loic Dachary [mailto:loic@dachary.org] 
> Sent: Tuesday, May 19, 2015 3:39 PM
> To: Miyamae, Takeshi/宮前 剛
> Cc: Ceph Development
> Subject: What crush ruleset for a given SHEC configuration ?
> 
> Hi Takeshi,
> 
> In the context of http://ceph.com/docs/master/rados/operations/erasure-code-shec/ it would be useful to have a more detailed explanation of why SHEC is more efficient during recovery (in the introduction). 
> 
> Am I correct to assume that SHEC does not provide a way to control the locality of the chunks ? For instance in the following scenario:
> 
> rack 1 has 10 OSDs
> rack 2 has 10 OSDs
> 
> a crush ruleset is made to provide 15 OSDs with 7 in the first rack, 8 in the last rack: the first 7 are in rack 1, the last 8 in rack 2. When SHEC is used with such a crush ruleset, it cannot guarantee that the loss of one chunk in rack 2 can always be recovered with chunks from rack 2. When reading at figure 3 of
> 
> https://wiki.ceph.com/Planning/Blueprints/Hammer/Shingled_Erasure_Code_%28SHEC%29
> 
> with D1 to D5, P1 and P2 in rack 1 and D6 to D10, P3, P4, P5 in rack 2, my understanding is that to recover D6 which is in rack 2 it may be necessary to use P2 from rack 1. And to recover D5 which is in rack 1 it may be necessary to use P3 from rack 2.
> 
> Maybe I'm missing something ? Thanks in advance for your explanations :-)
> 
> Cheers
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]