Re: PGLog API : missing

[Loic Dachary <loic@dachary.org>]

[-- Attachment #1: Type: text/plain, Size: 2967 bytes --]

For the record:

On 06/21/2013 06:42 PM, Samuel Just wrote:
> That seems like a reasonable break down for the missing set.  Rather
> than representing a full object, the missing set could be understood
> to represent whether an OSD has or is missing it's own piece of an
> object, whether that is a full copy or an erasure coded chunk.  The
> primary would use the missing sets to determine which osds are missing
> which chunks.


On 20/06/2013 16:38, Loic Dachary wrote:
> Hi Sam,
> 
> I tried to consider the part of the API of PGLog related to pg_missing_t, looking at what it currently is and what it should be. At the moment it is ( not considering the methods that have a side effect on both missing and log ):
> 
> Accessor:
> 
>   const pg_missing_t& get_missing() const { return missing; }
> 
> Actors:
> 
>   void missing_add(const hobject_t& oid, eversion_t need, eversion_t have)
>   void revise_need(hobject_t oid, eversion_t need)   void revise_have(hobject_t oid, eversion_t have)
> 
>   void missing_got(map<hobject_t, pg_missing_t::item>::const_iterator m)
>   void missing_rm(map<hobject_t, pg_missing_t::item>::const_iterator m)
> 
> The get_missing read only accessor is used in PG.{cc,h}, OSD.{cc,h}, Replicated.{cc,h} It is used in dozen of different ways and I'm not sure if it would make sense to define an API to replace them. I fail to see a pattern that would be useful.
> 
> The missing_add method is used by ReplicatedPG::recover_object_replicas and  PG::repair_object The revise_have method is used by ReplicatedPG::submit_push_data
> The revise_need method is used by ReplicatedPG::mark_all_unfound_lost when handling pg_log_entry_t::LOST_REVERT
> The missing_got method is used by ReplicatedPG::mark_all_unfound_lost when handling pg_log_entry_t::LOST_MARK
> The missing_rm method is used by ReplicatedPG::mark_all_unfound_lost when handling pg_log_entry_t::LOST_DELETE
> 
> All these make sense to me. But I'm looking at what is currently in place rather than thinking about what it should become to accomodate for erasure coding.
> 
> With erasure_coding one OSD contains a chunk. If it is lost, it will not be copied from another OSD, it will be repaired by using the content of other OSDs. For instance; if the chunk object 4 version 1,2 is not in OSD 3 the other chunks objects will be fetched from the remaining OSDs, the repair function will then be called using their content to create the chunk object 4 version 1,2. Instead of fetching a copy of the missing object from one OSD, the code would fetch chunk objects from multiple OSDs to be able to repair.
> Because pg_missing_t is dealing with the missing objects for a single OSD, it may not need to be modified when used in the context of erasure coding.
> 
> Am I making any sense ? I'm not sure ;-)
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]