Hi, Sorry I did not get a chance to answer in time to your question on IRC. loicd: a ruleset having {step take osd.0 step emit } should choose osd.0 for all pgs right wats the error with this crushmap ruleset http://pastebin.com/yXn1t4sU i want all pgs to be on [0,1,3] which is working as expected but if 3 goesdown i want it to be [0,1,4] .. this is not happening shylesh: hi ! shylesh: looking at http://pastebin.com/yXn1t4sU (answering here because that's the channel where non dev questions are best archived) shylesh: that's a very unusual crush map. You want to precisely control the order in which the OSDs are assigned to a placement group, is that right ? If that's so, maybe your use case would be better served by other means. What are you trying to achieve ? Measure the bandwidth between OSDs during repair maybe ? loicd: I want one group of PG mapped to [0,1,3] and another group of pgs(may be from different pool) mapped to [0,1,2] and I will mark 3 as out and 2 as down so that new group becomes [0,1,4] , osd4 will have degraded and misplaced pgs to get mapped there is a fix i want to test , degraded pgs should get priority over misplaced pgs so now i want to see on osd4 degraded will be recovered first and then misplaced now the mapping is working fine but if suppose 2 goes down i will be left with [0,1] for some reason 4 is not joining the replica group Could you show the fix that you want to test ? I may be able to figure out a simpler strategy to verify that it does what is expected. Cheers -- Loïc Dachary, Artisan Logiciel Libre