pg_missing_t::is_missing semantics

pg_missing_t::is_missing semantics

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1537 bytes --]

Hi,

When testing if a specific version of an object is missing with pg_missing_t::is_missing
https://github.com/ceph/ceph/blob/master/src/osd/osd_types.cc#L2153
if the version needed is greater than the version given in argument, false is returned.

Each object in pg_missing_t *have* a version and *need* another and *have* is lower than *need*. 
https://github.com/ceph/ceph/blob/master/src/osd/osd_types.h#L1470

Here is what gets me confused:

  if (item.need > v)
    return false;
  return true;

A version lower than *need* is considered to not be missing ( by is_missing ) 
where a version greater than *need* is considered to be missing. 

This prototype of is_missing is used at

https://github.com/ceph/ceph/blob/master/src/osd/ReplicatedPG.cc#L5910

  if (missing.is_missing(oid, v)) {
    dout(10) << "got missing " << oid << " v " << v << dendl;
    missing.got(oid, v);
    if (is_primary())
      missing_loc.erase(oid);

If the version is greater than the one needed ( i.e. it is more recent ) then it is not considered missing and therefore not removed from missing ( which is what pg_missing_t::got does ). 

Does it mean that when an object version is in missing, it will always be removed because the PG will receive an object that is either of the same version or lower ? And never receive an object of a version greater than the one in missing ?

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: pg_missing_t::is_missing semantics

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2821 bytes --]



On 05/27/2013 05:29 PM, Loic Dachary wrote:
> Hi,
> 
> When testing if a specific version of an object is missing with pg_missing_t::is_missing
> https://github.com/ceph/ceph/blob/master/src/osd/osd_types.cc#L2153
> if the version needed is greater than the version given in argument, false is returned.
> 
> Each object in pg_missing_t *have* a version and *need* another and *have* is lower than *need*. 
> https://github.com/ceph/ceph/blob/master/src/osd/osd_types.h#L1470
> 
> Here is what gets me confused:
> 
>   if (item.need > v)
>     return false;
>   return true;
> 
> A version lower than *need* is considered to not be missing ( by is_missing ) 
> where a version greater than *need* is considered to be missing. 
> 
> This prototype of is_missing is used at
> 
> https://github.com/ceph/ceph/blob/master/src/osd/ReplicatedPG.cc#L5910
> 
>   if (missing.is_missing(oid, v)) {
>     dout(10) << "got missing " << oid << " v " << v << dendl;
>     missing.got(oid, v);
>     if (is_primary())
>       missing_loc.erase(oid);
> 
> If the version is greater than the one needed ( i.e. it is more recent ) then it is not considered missing and therefore not removed from missing ( which is what pg_missing_t::got does ). 
> 
> Does it mean that when an object version is in missing, it will always be removed because the PG will receive an object that is either of the same version or lower ? And never receive an object of a version greater than the one in missing ?
> 

https://github.com/ceph/ceph/blob/master/src/osd/ReplicatedPG.cc#L5319

  if (missing.is_missing(recovery_info.soid) &&
      missing.missing[recovery_info.soid].need > recovery_info.version) {
    assert(is_primary());
    pg_log_entry_t *latest = log.objects[recovery_info.soid];
    if (latest->op == pg_log_entry_t::LOST_REVERT &&
latest->reverting_to == recovery_info.version) {
      dout(10) << " got old revert version " << recovery_info.version
<< " for " << *latest << dendl;
      recovery_info.version = latest->version;
      // update the attr to the revert event version
      recovery_info.oi.prior_version = recovery_info.oi.version;
      recovery_info.oi.version = latest->version;
      bufferlist bl;
      ::encode(recovery_info.oi, bl);
      t->setattr(coll, recovery_info.soid, OI_ATTR, bl);
    }
  }
  recover_got(recovery_info.soid, recovery_info.version);

Before calling recover_got, if the pg_log_entry is a LOST_REVERT operation for an object which version is lower than *need*, then it is modified to spoof the pg_log_entry as if it was of the same version as the object currently in the logs. 

/me still digging :-)

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: pg_missing_t::is_missing semantics

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 4131 bytes --]



On 05/27/2013 09:11 PM, Loic Dachary wrote:
> 
> 
> On 05/27/2013 05:29 PM, Loic Dachary wrote:
>> Hi,
>>
>> When testing if a specific version of an object is missing with pg_missing_t::is_missing
>> https://github.com/ceph/ceph/blob/master/src/osd/osd_types.cc#L2153
>> if the version needed is greater than the version given in argument, false is returned.
>>
>> Each object in pg_missing_t *have* a version and *need* another and *have* is lower than *need*. 
>> https://github.com/ceph/ceph/blob/master/src/osd/osd_types.h#L1470
>>
>> Here is what gets me confused:
>>
>>   if (item.need > v)
>>     return false;
>>   return true;
>>
>> A version lower than *need* is considered to not be missing ( by is_missing ) 
>> where a version greater than *need* is considered to be missing. 
>>
>> This prototype of is_missing is used at
>>
>> https://github.com/ceph/ceph/blob/master/src/osd/ReplicatedPG.cc#L5910
>>
>>   if (missing.is_missing(oid, v)) {
>>     dout(10) << "got missing " << oid << " v " << v << dendl;
>>     missing.got(oid, v);
>>     if (is_primary())
>>       missing_loc.erase(oid);
>>
>> If the version is greater than the one needed ( i.e. it is more recent ) then it is not considered missing and therefore not removed from missing ( which is what pg_missing_t::got does ). 
>>
>> Does it mean that when an object version is in missing, it will always be removed because the PG will receive an object that is either of the same version or lower ? And never receive an object of a version greater than the one in missing ?
>>
> 
> https://github.com/ceph/ceph/blob/master/src/osd/ReplicatedPG.cc#L5319
> 
>   if (missing.is_missing(recovery_info.soid) &&
>       missing.missing[recovery_info.soid].need > recovery_info.version) {
>     assert(is_primary());
>     pg_log_entry_t *latest = log.objects[recovery_info.soid];
>     if (latest->op == pg_log_entry_t::LOST_REVERT &&
> latest->reverting_to == recovery_info.version) {
>       dout(10) << " got old revert version " << recovery_info.version
> << " for " << *latest << dendl;
>       recovery_info.version = latest->version;
>       // update the attr to the revert event version
>       recovery_info.oi.prior_version = recovery_info.oi.version;
>       recovery_info.oi.version = latest->version;
>       bufferlist bl;
>       ::encode(recovery_info.oi, bl);
>       t->setattr(coll, recovery_info.soid, OI_ATTR, bl);
>     }
>   }
>   recover_got(recovery_info.soid, recovery_info.version);
> 
> Before calling recover_got, if the pg_log_entry is a LOST_REVERT operation for an object which version is lower than *need*, then it is modified to spoof the pg_log_entry as if it was of the same version as the object currently in the logs. 
> 
> /me still digging :-)
> 

is_missing is also called from ReplicatedPG::pull

https://github.com/ceph/ceph/blob/master/src/osd/ReplicatedPG.cc#L5022

If the desired object version is missing from the peer (i.e. soid is known to peer_missing[fromosd] and the *need*ed version is greater than version v) , the versions it has ( *have* ) is pulled instead:

if (peer_missing[fromosd].is_missing(soid, v)) {
    assert(peer_missing[fromosd].missing[soid].have != v);
    dout(10) << "pulling soid " << soid << " from osd " << fromosd
<< " at version " << peer_missing[fromosd].missing[soid].have
<< " rather than at version " << v << dendl;
    v = peer_missing[fromosd].missing[soid].have;

If soid is not known to peer_missing[fromosd] or the *need*ed version is lower or equal than version v, it is not missing, meaning the version v will be pulled. 

https://github.com/ceph/ceph/blob/master/src/osd/ReplicatedPG.cc#L5101

I'm unsure what will happen if v is pulled although soid is known to peer_missing[fromosd]. That is, in the case where the *need*ed version is lower or equal than version v. Because the peer does not have the required version. Or does it ? 

/me out for the night

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]