From mboxrd@z Thu Jan 1 00:00:00 1970 From: Greg Farnum Subject: Re: pgs stuck inactive Date: Wed, 18 Apr 2012 13:41:36 -0700 Message-ID: <454EEF491D1B41C3B0382F1A101BABE0@dreamhost.com> References: <3154FE08568349E9B9EB0EBBCE4EDFB2@dreamhost.com> <8260F00ADA384FEDBA6AA9873273BB38@dreamhost.com> <90B1EE3CB1C7494B9B097AD5E4BCF272@dreamhost.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Return-path: Received: from mail-ob0-f174.google.com ([209.85.214.174]:42507 "EHLO mail-ob0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754457Ab2DRUlk (ORCPT ); Wed, 18 Apr 2012 16:41:40 -0400 Received: by obbta14 with SMTP id ta14so7075715obb.19 for ; Wed, 18 Apr 2012 13:41:39 -0700 (PDT) In-Reply-To: Content-Disposition: inline Sender: ceph-devel-owner@vger.kernel.org List-ID: To: Damien Churchill Cc: ceph-devel@vger.kernel.org On Wednesday, April 18, 2012 at 12:04 PM, Damien Churchill wrote: > On 18 April 2012 19:41, Greg Farnum wrote: > > > > Are you building your binaries from source, and can you run a patched version of the monitors? If you can I'll give you a patch to enable a simple command that should make things work; otherwise we'll need to start editing things by hand. (Yucky) > > -Greg > > > > I was using the Ubuntu packages but I can quite happily build my own > packages if you give me the patch :-) > > I agree it's a waste of time if it's not obvious what's caused it, > could be some obscure cause occurred due to upgrading between older > versions. Okay, assuming you're still on 0.41.1, can you checkout the git branch "for-damien" and build it? Then shut down your monitors, replace their executables with the freshly-built ones, and run "ceph osd pool set vmimages pg_num 320 --a-dev-told-me-to" and "ceph osd pool set vmimages pgp_num 320" That should get everything back up and running. The one sour note is that due to the bug in the past, your data (ie, filesystem) and vmimages pools have gotten conflated. It shouldn't cause any issues (they use very different naming schemes), but they're tied together in terms of replication and the raw pool statistics. (If that's important you can create a new pool and move the rbd images to it.) -Greg