Re: Crush and Monitor questions

[Joao Eduardo Luis <joao.luis@inktank.com>]

Hello Bryant,

On 12/11/2012 08:23 PM, Bryant Ng wrote:
> Hi,
>
> I'm pretty new to Ceph and am just learning about it.
>
> Where are the CRUSH maps stored in Ceph? In the documentation I see you
> use the 'crushtool' to compile and decompile the crush map.

The crushmap is kept alongside with the osdmap, and shared by the monitors.

> I understand that if a single monitor comes online, it can talk to the
> other existing monitors to get the cluster map but how does it work on
> initial startup?  Or if the entire Ceph clusters goes down b/c of power
> failure or something.

If you add a new monitor to an existing cluster, it will synchronize 
with the existing monitors and will obtain all their infos, including 
the crushmap. Updates to the crushmap will also be shared among the 
monitors in the quorum.

If you are starting up fresh, you will have to either add your custom 
crushmap to the monitors (using the ceph tool), or stick with the 
default crushmap (which only defines something along the lines of a 
'default' root, a 'defaultrack' rack and a 'localhost' host).

If the entire cluster goes down... well, if the monitors are not up they 
won't be able to share the crushmap. When they are brought back up, then 
they will pick up where they left. But I'm not sure if I understand what 
your question is.

   -Joao