caching get_chunk_size()

caching get_chunk_size()

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1515 bytes --]

[cc' ceph-devel]

Hi Sam,

Assuming https://github.com/ceph/ceph/pull/1144 gets merged, MONs can query the plugin for a given pool or crush ruleset conveniently:

https://github.com/dachary/ceph/blob/15c90fbf61445063967fc80ee1c916d70c593a54/src/mon/OSDMonitor.cc#L2803

      ErasureCodeInterfaceRef erasure_code;
      err = get_erasure_code(properties, &erasure_code, ss);
      if (err == 0)
         *size = erasure_code->get_chunk_count();

And OSDs can do something similar (and must whenever an erasure coded pool is instantiated). If I understood correctly, you would like the result of erasure_code->get_chunk_size(some_osd_config_opt_value_defining_the_stripe_size) to be set as a new value in pg_pool_t https://github.com/dachary/ceph/blob/15c90fbf61445063967fc80ee1c916d70c593a54/src/osd/osd_types.h#L700 when the pool is created. 

The benefit of having pg_pool_t::chunk_size would be 

a) the value is shared cluster wide and does not depend on the specific version of the OSD/MON
b) if the value is needed frequently in a context where the plugin would have to be loaded, this will improve performances

However, whenever an OSD instantiates a PG that belongs to a pool, it must assert that the get_chunk_count() returned by the plugin matches the value stored in pg_pool_t::chunk_size otherwise it will encode inconsistently. 

If my reasoning matches yours, I'll implement the change. Otherwise, please correct me :-)

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: caching get_chunk_size()

[Samuel Just]

That looks about right to me.  It's handy to have a "ground truth" for
everyone to ensure that they agree with.
-Sam

On Sun, Feb 2, 2014 at 7:06 AM, Loic Dachary <loic@dachary.org> wrote:
> [cc' ceph-devel]
>
> Hi Sam,
>
> Assuming https://github.com/ceph/ceph/pull/1144 gets merged, MONs can query the plugin for a given pool or crush ruleset conveniently:
>
> https://github.com/dachary/ceph/blob/15c90fbf61445063967fc80ee1c916d70c593a54/src/mon/OSDMonitor.cc#L2803
>
>       ErasureCodeInterfaceRef erasure_code;
>       err = get_erasure_code(properties, &erasure_code, ss);
>       if (err == 0)
>          *size = erasure_code->get_chunk_count();
>
> And OSDs can do something similar (and must whenever an erasure coded pool is instantiated). If I understood correctly, you would like the result of erasure_code->get_chunk_size(some_osd_config_opt_value_defining_the_stripe_size) to be set as a new value in pg_pool_t https://github.com/dachary/ceph/blob/15c90fbf61445063967fc80ee1c916d70c593a54/src/osd/osd_types.h#L700 when the pool is created.
>
> The benefit of having pg_pool_t::chunk_size would be
>
> a) the value is shared cluster wide and does not depend on the specific version of the OSD/MON
> b) if the value is needed frequently in a context where the plugin would have to be loaded, this will improve performances
>
> However, whenever an OSD instantiates a PG that belongs to a pool, it must assert that the get_chunk_count() returned by the plugin matches the value stored in pg_pool_t::chunk_size otherwise it will encode inconsistently.
>
> If my reasoning matches yours, I'll implement the change. Otherwise, please correct me :-)
>
> Cheers
>
> --
> Loïc Dachary, Artisan Logiciel Libre
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: caching get_chunk_size()

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2219 bytes --]

Hi Sam,

Is there a branch somewhere where you have defined a config_opts for the desired strip size / chunk size ?

Cheers

On 02/02/2014 23:28, Samuel Just wrote:
> That looks about right to me.  It's handy to have a "ground truth" for
> everyone to ensure that they agree with.
> -Sam
> 
> On Sun, Feb 2, 2014 at 7:06 AM, Loic Dachary <loic@dachary.org> wrote:
>> [cc' ceph-devel]
>>
>> Hi Sam,
>>
>> Assuming https://github.com/ceph/ceph/pull/1144 gets merged, MONs can query the plugin for a given pool or crush ruleset conveniently:
>>
>> https://github.com/dachary/ceph/blob/15c90fbf61445063967fc80ee1c916d70c593a54/src/mon/OSDMonitor.cc#L2803
>>
>>       ErasureCodeInterfaceRef erasure_code;
>>       err = get_erasure_code(properties, &erasure_code, ss);
>>       if (err == 0)
>>          *size = erasure_code->get_chunk_count();
>>
>> And OSDs can do something similar (and must whenever an erasure coded pool is instantiated). If I understood correctly, you would like the result of erasure_code->get_chunk_size(some_osd_config_opt_value_defining_the_stripe_size) to be set as a new value in pg_pool_t https://github.com/dachary/ceph/blob/15c90fbf61445063967fc80ee1c916d70c593a54/src/osd/osd_types.h#L700 when the pool is created.
>>
>> The benefit of having pg_pool_t::chunk_size would be
>>
>> a) the value is shared cluster wide and does not depend on the specific version of the OSD/MON
>> b) if the value is needed frequently in a context where the plugin would have to be loaded, this will improve performances
>>
>> However, whenever an OSD instantiates a PG that belongs to a pool, it must assert that the get_chunk_count() returned by the plugin matches the value stored in pg_pool_t::chunk_size otherwise it will encode inconsistently.
>>
>> If my reasoning matches yours, I'll implement the change. Otherwise, please correct me :-)
>>
>> Cheers
>>
>> --
>> Loïc Dachary, Artisan Logiciel Libre
>>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]