CEPH filesystem development
 help / color / mirror / Atom feed
From: Greg Farnum <gregory.farnum@dreamhost.com>
To: Damien Churchill <damoxc@gmail.com>
Cc: Samuel Just <sam.just@dreamhost.com>, ceph-devel@vger.kernel.org
Subject: Re: pgs stuck inactive
Date: Fri, 13 Apr 2012 12:30:32 -0700	[thread overview]
Message-ID: <8260F00ADA384FEDBA6AA9873273BB38@dreamhost.com> (raw)
In-Reply-To: <CAFtEh-eEH5Hon5o2hXiEXA6FewB05Uj8=fR4L3HtkLw9zTxccA@mail.gmail.com>

On Thursday, April 12, 2012 at 8:29 AM, Damien Churchill wrote:
> On 11 April 2012 00:40, Greg Farnum <gregory.farnum@dreamhost.com (mailto:gregory.farnum@dreamhost.com)> wrote:
> >  
> > A quick glance through these shows that all the pg_temp requests aren't actually requesting any changes from the monitor. It's either a very serious mon bug which happened a while ago (unlikely, given the restarts and ongoing map changes, etc), or an OSD bug. I think we want logs from both osd.0 and osd.3 at the same time, from what I'm seeing. :)
> > -Greg
>  
>  
>  
> Just to make sure all bases are covered:
>  
> http://damoxc.net/ceph/ceph-logs-20120412142537.tar.gz
>  
> This contains all 5 osd logs and all 3 monitor logs, everything
> restarted with debug logging prior to capturing the logs.

I (and Sam) spent some time looking at this very closely. It continues to tell me that the OSD and the monitor are disagreeing on whether osd 3 should be in the pg temp set for some things, but they seem to agree on everything else….  
Can you zip up for me:
1) The files matching osdmap* of osd0's store from the current/meta/ directory,
2) The contents of your lead monitor's osdmap and osdmap_full directories?

We can check these for differences and then run them through some of our tools and stuff to try and identify the issue.
Thanks!
-Greg

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

  reply	other threads:[~2012-04-13 19:30 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-04-04  8:12 pgs stuck inactive Damien Churchill
2012-04-04 20:51 ` Samuel Just
2012-04-04 21:44   ` Damien Churchill
2012-04-06 16:50     ` Samuel Just
2012-04-06 16:55       ` Damien Churchill
2012-04-06 17:30         ` Samuel Just
2012-04-06 17:53           ` Damien Churchill
     [not found]             ` <CACLRD_3hqoOqbZWnyft3B+pEaPG=pmiDt5HeQASUC=OPS-uA7Q@mail.gmail.com>
     [not found]               ` <CAFtEh-dnR0zu=ct-gG0L+76BTYHVFimjFKNhHjnDNZWA=Scs1g@mail.gmail.com>
2012-04-06 18:20                 ` Samuel Just
     [not found]                   ` <CAFtEh-cx2by6yG_xiQbq5e2fbViOWyHoaOLFPBCZgX7x-7gwFA@mail.gmail.com>
     [not found]                     ` <CACLRD_0UrwKJuELiZqrL7Q_5AjB1EvOd5gqLRzrxw_7AFMcjxg@mail.gmail.com>
     [not found]                       ` <CAFtEh-fwt9BwZ7NwZaJVnwq3hgzvkrb1XvKT0uF81vePRcu7oA@mail.gmail.com>
2012-04-10 21:49                         ` Samuel Just
2012-04-10 22:03                           ` Damien Churchill
2012-04-10 23:00                             ` Samuel Just
2012-04-10 23:40                               ` Greg Farnum
2012-04-12 15:29                                 ` Damien Churchill
2012-04-13 19:30                                   ` Greg Farnum [this message]
2012-04-13 19:42                                     ` Damien Churchill
2012-04-17  0:06                                       ` Greg Farnum
2012-04-17  5:32                                         ` Damien Churchill
2012-04-17 16:49                                           ` Greg Farnum
2012-04-18  4:35                                             ` Martin Wilderoth
2012-04-18  6:41                                             ` Damien Churchill
2012-04-18 18:41                                               ` Greg Farnum
2012-04-18 19:04                                                 ` Damien Churchill
2012-04-18 20:41                                                   ` Greg Farnum
2012-04-19  9:07                                                     ` Damien Churchill
2012-04-19 17:09                                                       ` Greg Farnum

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=8260F00ADA384FEDBA6AA9873273BB38@dreamhost.com \
    --to=gregory.farnum@dreamhost.com \
    --cc=ceph-devel@vger.kernel.org \
    --cc=damoxc@gmail.com \
    --cc=sam.just@dreamhost.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox