Re: Deadline of Github pull request for Hammer release (question)

[Loic Dachary <loic@dachary.org>]

[-- Attachment #1: Type: text/plain, Size: 8731 bytes --]

Hi,

Thanks for the update. To speed up things, I could review what's already published. Did you add the tests already ?

Cheers

On 20/01/2015 01:48, Miyamae, Takeshi wrote:
> Hi Loic,
> 
> Thank you for your advice which suggested providing SHEC as an extra-package.
> We believe it's generally a good idea, but we are hoping SHEC would be merged
> into Hammer because that would have an apparent advantage from a viewpoint of
> maintenance support.
> 
> As for the thread safety issue, we totally agree with you and we are trying to solve it.
> We will complete it in a few days and we'd like to ask you to review it again so that
> it might be in time for Hammer's feature freeze (v0.93).
> 
> Best regards,
> Takeshi Miyamae
> 
> -----Original Message-----
> From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Loic Dachary
> Sent: Wednesday, January 14, 2015 3:03 AM
> To: Miyamae, Takeshi/宮前 剛
> Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾 鷹詔; Paul Von-Stamwitz (PVonStamwitz@us.fujitsu.com)
> Subject: Re: Deadline of Github pull request for Hammer release (question)
> 
> Hi,
> 
> On 13/01/2015 18:05, Miyamae, Takeshi wrote:
>> Hi Loic,
>>
>> Thank you for your quick review.
>>
>>> Could you reference the jerasure files (they are in the jerasure 
>>> plugin already) instead of including them in your patch ?
>>
>> We had used the reference of jerasure v2.0 files before, but changed 
>> to including v1.2 files after the patent issue.
> 
> If you are refering to http://erasure-code-patents.xyz/, it can safely be ignored.
> 
>> However, we can restore the reference right away if we are needed.
>>
>>> In ::prepare you reuse the matrix, if possible. If your intention is 
>>> to improve performances, you should consider using the same approach as the isa plugin.
>>
>> We have found a performance issue caused by redundant initializations 
>> of the module, and solved it by caching the initialized variables in memory.
>> If that is the same approach as isa-plugin you mentioned, we also do 
>> not have the same issue any more.
> 
> The ISA plugin approach is thread safe, that's the important part.
> 
>>> I'll have more comments once you have unit / functional tests
>>
>> We do have the those unit/functional tests, but the server is unreachable at this moment.
>> Please let us upload them to the repository tomorrow.
> 
> Great.
> 
>>> Regarding your initial question: I think it is too late for the Hammer release.
>>
>> We are so disappointed to hear that, but we must apologize for the late request.
>> If there would be any chance that SHEC be pulled into hammer, we are 
>> very happy to conduct all the necessary tests by ourselves.
>> Please let us know the detailed schedule if this is a feasible option.
> 
> Note that since this is a plugin it will be easy for someone to install it on a Hammer cluster, simply by copying the plugin files to each OSD / MON and restarting them. If there is some kind of urgency for you to be able to install this new plugin, I would advise making a package that contains just this file, restart the OSD once installed and recommend that the installation is done on all machines of the cluster (because every OSD / MON need to know about the plugin). 
> 
> Cheers
> 
>> Best regards,
>> Takeshi Miyamae
>>
>> -----Original Message-----
>> From: ceph-devel-owner@vger.kernel.org 
>> [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Loic Dachary
>> Sent: Tuesday, January 13, 2015 9:46 PM
>> To: Miyamae, Takeshi/宮前 剛
>> Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾 鷹詔; 
>> Paul Von-Stamwitz (PVonStamwitz@us.fujitsu.com)
>> Subject: Re: Deadline of Github pull request for Hammer release 
>> (question)
>>
>> Hi,
>>
>> It's a great start :-) A few comments:
>>
>> Could you reference the jerasure files (they are in the jerasure plugin already) instead of including them in your patch ?
>>
>> In ::prepare you reuse the matrix, if possible. If your intention is 
>> to improve performances, you should consider using the same approach 
>> as the isa plugin. See 
>> http://workbench.dachary.org/ceph/ceph/blob/giant/src/erasure-code/isa
>> /ErasureCodePluginIsa.cc#L36 which is and independent type 
>> http://workbench.dachary.org/ceph/ceph/blob/giant/src/erasure-code/isa
>> /ErasureCodeIsaTableCache.h with only one instance and is populated as 
>> needed and protected by a lock to make it thread safe : 
>> http://workbench.dachary.org/ceph/ceph/blob/giant/src/erasure-code/isa
>> /ErasureCodeIsaTableCache.h#L101
>>
>> I'll have more comments once you have unit / functional tests ( similar to http://workbench.dachary.org/ceph/ceph/blob/giant/src/test/erasure-code/TestErasureCodeJerasure.cc ). 
>>
>> Regarding your initial question: I think it is too late for the Hammer release. But from what I read it looks like we'll be able to merge in the early stages of the next release and that will give us time to properly test it.
>>
>> Cheers
>>
>> On 13/01/2015 11:34, Miyamae, Takeshi wrote:
>>> Hi Loic,
>>>
>>> I'm so sorry. The following is the correct repository.
>>>
>>> https://github.com/t-miyamae/ceph
>>>
>>> Best regards,
>>> Takeshi Miyamae
>>>
>>> -----Original Message-----
>>> From: Loic Dachary [mailto:loic@dachary.org]
>>> Sent: Tuesday, January 13, 2015 7:26 PM
>>> To: Miyamae, Takeshi/宮前 剛
>>> Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾 鷹詔;
>>> Paul Von-Stamwitz (PVonStamwitz@us.fujitsu.com)
>>> Subject: Re: Deadline of Github pull request for Hammer release
>>> (question)
>>>
>>> Hi,
>>>
>>> On 13/01/2015 11:24, Miyamae, Takeshi wrote:
>>>> Hi Loic,
>>>>
>>>>> Although we're late in the Hammer roadmap, it's a good time for an early preview. It will help show what needs to be changed to accomodate the SHEC plugin.
>>>>
>>>> Thank you for your advices.
>>>> We have uploaded our latest codes to the following folk repository for an early review.
>>>>
>>>> https://github.com/miyamae-takeshi/multiple-shec
>>>
>>> It's 404 ? Is it a private repository maybe ?
>>>
>>>>
>>>> SHEC is located in src/erasure-code/shec directory.
>>>>
>>>> We are verifying SHEC's advantages on our Ceph cluster. It takes a little bit more.
>>>> Would you please start the review before that?
>>>>
>>>> Best regards,
>>>> Takeshi Miyamae
>>>>
>>>> -----Original Message-----
>>>> From: ceph-devel-owner@vger.kernel.org 
>>>> [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Loic Dachary
>>>> Sent: Wednesday, January 7, 2015 12:52 AM
>>>> To: Miyamae, Takeshi/宮前 剛
>>>> Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾 鷹詔
>>>> Subject: Re: Deadline of Github pull request for Hammer release
>>>> (question)
>>>>
>>>> Hi,
>>>>
>>>> On 06/01/2015 12:49, Miyamae, Takeshi wrote:
>>>>> Dear Loic,
>>>>>
>>>>> I'm Takeshi Miyamae, one of the authors of SHEC's blueprint.
>>>>>
>>>>> Shingled Erasure Code (SHEC)
>>>>> https://wiki.ceph.com/Planning/Blueprints/Hammer/Shingled_Erasure_C
>>>>> o
>>>>> d
>>>>> e
>>>>> _(SHEC)
>>>>
>>>> The work you have done is quite impressive :-)
>>>>
>>>>> We have revised our blueprint shown in the last CDS to extend our 
>>>>> erasure code layouts and describe the guideline for choosing SHEC among various EC plugins.
>>>>> We believe the blueprint now answers all the comments given at the CDS.
>>>>
>>>> Great.
>>>>
>>>>> In addition, we would like to ask for your advice on the schedule 
>>>>> of our github pull request. More specifically, we would like to 
>>>>> know its deadline for Hammer release.
>>>>> (As we have not really completed our verification of SHEC, we are 
>>>>> wondering if we should make it open for early preview.)
>>>>
>>>> Although we're late in the Hammer roadmap, it's a good time for an early preview. It will help show what needs to be changed to accomodate the SHEC plugin.
>>>>
>>>> Cheers
>>>>
>>>>> Thank you in advance,
>>>>> Takeshi Miyamae
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" 
>>>>> in the body of a message to majordomo@vger.kernel.org More 
>>>>> majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>
>>>> --
>>>> Loïc Dachary, Artisan Logiciel Libre
>>>>
>>>
>>> --
>>> Loïc Dachary, Artisan Logiciel Libre
>>>
>>
>> --
>> Loïc Dachary, Artisan Logiciel Libre
>>
> 
> --
> Loïc Dachary, Artisan Logiciel Libre
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]