teuthology and erasure code pool creation

teuthology and erasure code pool creation

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1287 bytes --]

Hi Zack,

In order to test locally repairable codes ( described at https://wiki.ceph.com/Planning/Blueprints/Giant/Pyramid_Erasure_Code and documented at https://github.com/dachary/ceph/commit/2ae645438243232392c11fd80fd960f6221fd2a0#diff-5518964bc98a094a784ce2d17a5b0cc1R20 ) it would be a convenient to set the erasure code profile without modifying the teuthology code. 

There are not too many instances of hardcoded parameters:

  https://github.com/ceph/teuthology/blob/master/teuthology/task_util/rados.py#L27
  https://github.com/ceph/teuthology/blob/master/teuthology/task/ceph_manager.py#L732

and it could be generalized. In a nutshell, the definition of an erasure coded profile is a set of key value which should default to:

  k: 2
  m: 1
  plugin: jerasure
  ruleset-failure-domain: osd

and 

  self.raw_cluster_cmd('osd', 'erasure-code-profile', 'set', 'teuthologyprofile', 'ruleset-failure-domain=osd', 'm='+str(ec_m), 'k='+str(ec_k))

could be replaced with:
   
   self.raw_cluster_cmd('osd', 'erasure-code-profile', 'set', 'teuthologyprofile', **kwargs)

assuming kwargs is the above map parsed from the yaml. 

Any advice you may have on the implementation would be very welcome :-)

Cheers
-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: teuthology and erasure code pool creation

[Zack Cerza]

Hi Loic,

Without spending too much time investigating, it seems to me it would
be preferable to just add the missing arguments to the functions you
pointed out. That way they could even be fully documented! ;)

Zack

On Sun, Jun 8, 2014 at 11:00 AM, Loic Dachary <loic@dachary.org> wrote:
> Hi Zack,
>
> In order to test locally repairable codes ( described at https://wiki.ceph.com/Planning/Blueprints/Giant/Pyramid_Erasure_Code and documented at https://github.com/dachary/ceph/commit/2ae645438243232392c11fd80fd960f6221fd2a0#diff-5518964bc98a094a784ce2d17a5b0cc1R20 ) it would be a convenient to set the erasure code profile without modifying the teuthology code.
>
> There are not too many instances of hardcoded parameters:
>
>   https://github.com/ceph/teuthology/blob/master/teuthology/task_util/rados.py#L27
>   https://github.com/ceph/teuthology/blob/master/teuthology/task/ceph_manager.py#L732
>
> and it could be generalized. In a nutshell, the definition of an erasure coded profile is a set of key value which should default to:
>
>   k: 2
>   m: 1
>   plugin: jerasure
>   ruleset-failure-domain: osd
>
> and
>
>   self.raw_cluster_cmd('osd', 'erasure-code-profile', 'set', 'teuthologyprofile', 'ruleset-failure-domain=osd', 'm='+str(ec_m), 'k='+str(ec_k))
>
> could be replaced with:
>
>    self.raw_cluster_cmd('osd', 'erasure-code-profile', 'set', 'teuthologyprofile', **kwargs)
>
> assuming kwargs is the above map parsed from the yaml.
>
> Any advice you may have on the implementation would be very welcome :-)
>
> Cheers
> --
> Loïc Dachary, Artisan Logiciel Libre
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: teuthology and erasure code pool creation

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2015 bytes --]



On 09/06/2014 19:15, Zack Cerza wrote:
> Hi Loic,
> 
> Without spending too much time investigating, it seems to me it would
> be preferable to just add the missing arguments to the functions you
> pointed out. That way they could even be fully documented! ;)

Ok, I'll do that, thanks !

> 
> Zack
> 
> On Sun, Jun 8, 2014 at 11:00 AM, Loic Dachary <loic@dachary.org> wrote:
>> Hi Zack,
>>
>> In order to test locally repairable codes ( described at https://wiki.ceph.com/Planning/Blueprints/Giant/Pyramid_Erasure_Code and documented at https://github.com/dachary/ceph/commit/2ae645438243232392c11fd80fd960f6221fd2a0#diff-5518964bc98a094a784ce2d17a5b0cc1R20 ) it would be a convenient to set the erasure code profile without modifying the teuthology code.
>>
>> There are not too many instances of hardcoded parameters:
>>
>>   https://github.com/ceph/teuthology/blob/master/teuthology/task_util/rados.py#L27
>>   https://github.com/ceph/teuthology/blob/master/teuthology/task/ceph_manager.py#L732
>>
>> and it could be generalized. In a nutshell, the definition of an erasure coded profile is a set of key value which should default to:
>>
>>   k: 2
>>   m: 1
>>   plugin: jerasure
>>   ruleset-failure-domain: osd
>>
>> and
>>
>>   self.raw_cluster_cmd('osd', 'erasure-code-profile', 'set', 'teuthologyprofile', 'ruleset-failure-domain=osd', 'm='+str(ec_m), 'k='+str(ec_k))
>>
>> could be replaced with:
>>
>>    self.raw_cluster_cmd('osd', 'erasure-code-profile', 'set', 'teuthologyprofile', **kwargs)
>>
>> assuming kwargs is the above map parsed from the yaml.
>>
>> Any advice you may have on the implementation would be very welcome :-)
>>
>> Cheers
>> --
>> Loïc Dachary, Artisan Logiciel Libre
>>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]