Re: OSDMap / osd_state questions

[Jesse Williamson <jwilliamson@suse.de>]

On Tue, 4 Apr 2017, Jesse Williamson wrote:

Ok, sorry to add extra traffic, but a partial answer to my own question leads to more questions:

> ...is there some condition where an osd id will appear in osd_state and yet 
> CEPH_OSD_EXISTS be false? It looks like the other two conditions, UP and 
> OUT[1] depend on that being true.

OSDMonitor::get_health() appears to circumvent the is_up()/is_down() 
methods from OSDMap, instead checking manually-- and slightly 
differently[1].

One of the checks is:
       if (!osdmap.exists(i)) {
         if (osdmap.crush->item_exists(i)) {
           down_osds.insert(i);
         }

...so, to be "down", something must NOT exist in the osd_state AND must 
exist in the CRUSH map. Looking at definition of OSDMap::exists() and 
OSDMap::is_down(), they differ in that get_health() is checking the CRUSH 
map. Who's right..?

It also looks like in get_health() when something's added to down_osds, 
num_down_in_osds isn't incremented. Is this intentional?

Thanks again,

-Jesse

1. From OSDMonitor::get_health():

     int num_in_osds = 0;
     set<int> down_osds;
     for (int i = 0; i < osdmap.get_max_osd(); i++) {
       if (!osdmap.exists(i)) {
         if (osdmap.crush->item_exists(i)) {
           down_osds.insert(i);
         }
      continue;
       }
       if (osdmap.is_out(i))
         continue;
       ++num_in_osds;
       if (!osdmap.is_up(i)) {
     ++num_down_in_osds;

2. exists(), etc.:

   bool exists(int osd) const {
     //assert(osd >= 0);
     return osd >= 0 && osd < max_osd && (osd_state[osd] & 
CEPH_OSD_EXISTS);
   }

   bool is_up(int osd) const {
     return exists(osd) && (osd_state[osd] & CEPH_OSD_UP);
   }

   bool has_been_up_since(int osd, epoch_t epoch) const {
     return is_up(osd) && get_up_from(osd) <= epoch;
   }

   bool is_down(int osd) const {
     return !is_up(osd);
   }