Re: domino-style OSD crash

[Yann Dupont <Yann.Dupont@univ-nantes.fr>]

Le 10/07/2012 19:11, Tommi Virtanen a écrit :
> On Tue, Jul 10, 2012 at 9:39 AM, Yann Dupont <Yann.Dupont@univ-nantes.fr> wrote:
>>> The cluster mechanism was never intended for moving existing osds to
>>> other clusters. Trying that might not be a good idea.
>> Ok, good to know. I saw that the remaining maps could lead to problem, but
>> in 2 words, what are the other associated risks ? Basically If I use 2
>> distincts config files,
>> with differents & non-overlapping paths, and different ports for OSD, MDS &
>> MON, we basically have 2 distincts and independant instances ?
> Fundamentally, it comes down to this: the two clusters will still have
> the same fsid, and you won't be isolated from configuration errors or

Ah I understand. This is not the case : see :

root@chichibu:~# cat /CEPH/data/osd.0/fsid
f00139fe-478e-4c50-80e2-f7cb359100d4
root@chichibu:~# cat /CEPH-PROD/data/osd.0/fsid
43afd025-330e-4aa8-9324-3e9b0afce794

(CEPH-PROD is the old btrfs volume ). /CEPH is new xfs volume, 
completely redone & reformatted with mkcephfs. The volumes are totally 
independant :

if you want the gore details :

root@chichibu:~# lvs
   LV              VG             Attr   LSize   Origin Snap%  Move Log 
Copy%  Convert
   ceph-osd        LocalDisk      -wi-a- 225,00g
   mon-btrfs       LocalDisk      -wi-ao 10,00g
   mon-xfs         LocalDisk      -wi-ao 10,00g
   data            ceph-chichibu  -wi-ao   5,00t    <- OLD btrfs, 
mounted on /CEPH-PROD
   data            xceph-chichibu -wi-ao   4,50t   <- NEW xfs, mounted 
on /CEPH

> leftover state (such as the monmap) in any way. There's a high chance
> that your "let's poke around and debug" cluster wrecks your healthy
> cluster.

Yes I understand the risk.

>> By the way, is using 2 mon instance with different ports supported ?
> Monitors are identified by ip:port. You can have multiple bind to the
> same IP address, as long as they get separate ports.
>
> Naturally, this practically means giving up on high availability.

The idea is not just having 2 mon. I'll still use 3 differents machines 
for mon, but with 2 mon instance on each. One for the current ceph, the 
other for the old ceph.
2x3 Mon.

Cheers,

-- 
Yann Dupont - Service IRTS, DSI Université de Nantes
Tel : 02.53.48.49.20 - Mail/Jabber : Yann.Dupont@univ-nantes.fr

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html