Re: summit: erasure coding

[Loic Dachary <loic@dachary.org>]

[-- Attachment #1: Type: text/plain, Size: 2435 bytes --]

Hi Sage,

I've created "PG/ReplicatedPG API" ( http://tracker.ceph.com/issues/4928 ) for the first action item:

>  - clean up the OSD -> PG interface

and started with IPG class to be used in place of PG/ReplicatedPG ( https://github.com/dachary/ceph/commit/66d798753fc90e0daa7d8ce92ef7b692e259484f ).

I'll join the standup tomorrow ( past two days were hollidays in France ;-)

Cheers

On 05/09/2013 05:23 PM, Sage Weil wrote:
> We have a great session with Loic, Christopher, Sam, and Greg that 
> discussed how to move forward with erasure coding support.  The high-level 
> consensus on approach:
> 
>  - it is possible to do erasure coding above rados across distinct pools, 
> but it is harder, and less useful.
>  - we should have an ErasureCodedPG that takes advantage of CRUSH's 
> placement to place shards
>  - we can support a limited subset of rados operations for such pools and 
> still be useful (write_full, or write/append on block boundaries)
>  - this will be used in conjuction with a replicated pool as a second tier 
> of storage, or by applications that are happy with a limited subset of 
> commands.
> 
> That said, the implementation will be non-trivial.  But, we identified 
> several areas where code cleanup will move us down the right path.  By 
> factoring our useful components of PG and ReplicatedPG into separate 
> classes, we clean up the current interfaces and can also build unit tests 
> for them as we do so for immediate benefit.
> 
> Initial focus areas:
> 
>  - clean up the OSD -> PG interface (PG -> OSD is already reasonably well 
>    captured by teh OSDService class)
>  - ObjectContext tracking
>  - PG log handling
>  - PG missing
>  - RepOp state
>  - Peering state machine
> 
> The last one willb e most tricky (and saved for last).  In each case, 
> we'll have to think carefully about how well things generalize from 
> replication to erasure coding.
> 
> Loic has volunteered to own this work, and Sam and I will be supporting.  
> He'll also be joining our daily core standup.  Yay!
> 
> sage
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]