OSD not coming up after being set down

[Willem Jan Withagen <wjw@digiware.nl>]

Hi,

Any handholding is welcomed!!

In test/cephtool-mon-test.sh part of the excuted code is:
 ceph osd down 0
  ceph osd dump | grep 'osd.0 down'
  ceph osd unset noup
  for ((i=0; i < 120; i++)); do
    if ! ceph osd dump | grep 'osd.0 up'; then
      echo "waiting for osd.0 to come back up"
      sleep 1
    else
      break
    fi
  done
  ceph osd dump | grep 'osd.0 up'

But the OSD refused to come back up.
Below the output of the dump.

How would I start analyzing this issue?
What kind of things would I expect to see in the logfile?
  What if the OSD does come up
  What if the OSD stays down

Thanx,
--WjW


*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
epoch 170
fsid 8b5c0b4b-e08c-11e5-8cd4-1c6f6582ec12
created 2016-03-02 16:36:35.001700
modified 2016-03-02 16:45:17.802073
flags sortbitwise
pool 0 'rbd' replicated size 3 min_size 1 crush_ruleset 0 object_hash
rjenkins pg_num 8 pgp_num 8 last_change 1 flags hashpspool stripe_width 0
max_osd 3
osd.0 down out weight 0 up_from 4 up_thru 163 down_at 166
last_clean_interval [0,0) 127.0.0.1:6804/2455 127.0.0.1:6805/2455
127.0.0.1:6806/2455 127.0.0.1:6807/2455 autoout,exists
8bc29c74-e08c-11e5-8cd4-1c6f6582ec12
osd.1 up   in  weight 1 up_from 8 up_thru 166 down_at 0
last_clean_interval [0,0) 127.0.0.1:6808/2475 127.0.0.1:6811/2475
127.0.0.1:6813/2475 127.0.0.1:6816/2475 exists,up
8d7a2cb5-e08c-11e5-8cd4-1c6f6582ec12
osd.2 up   in  weight 1 up_from 13 up_thru 166 down_at 0
last_clean_interval [0,0) 127.0.0.1:6817/2495 127.0.0.1:6818/2495
127.0.0.1:6819/2495 127.0.0.1:6820/2495 exists,up
8f46df05-e08c-11e5-8cd4-1c6f6582ec12
pg_temp 0.0 [0,2,1]
pg_temp 0.1 [2,0,1]
pg_temp 0.2 [0,1,2]
pg_temp 0.3 [2,0,1]
pg_temp 0.4 [0,2,1]
pg_temp 0.5 [0,2,1]
pg_temp 0.6 [0,1,2]
pg_temp 0.7 [1,0,2]
2016-03-02 16:56:11.027977 8021d7800  0 lockdep stop