From: "Jens Kristian Søgaard" <jens@mermaidconsulting.dk>
To: Sage Weil <sage@inktank.com>
Cc: Stefan Priebe <s.priebe@profihost.ag>,
Wido den Hollander <wido@widodh.nl>,
"ceph-devel@vger.kernel.org" <ceph-devel@vger.kernel.org>
Subject: Re: Hit suicide timeout after adding new osd
Date: Sat, 19 Jan 2013 18:56:53 +0100 [thread overview]
Message-ID: <50FADE65.5050403@mermaidconsulting.dk> (raw)
In-Reply-To: <alpine.DEB.2.00.1301190737110.29915@cobra.newdream.net>
Hi Sage,
> a I dropped the wip branch. I just repushed the patches (on top of
> bobtail) for you.
Thanks for the new build!
I have finished testing it on one osd - sadly the osd crashed again, but
now with a new stack trace.
0> 2013-01-19 18:48:44.636938 7fb5597f2700 -1 ./osd/OSDMap.h: In
function 'const epoch_t& OSDMap::get_up_thru(int) const' thread
7fb5597f2700 time 2013-01-19 18:48:44.499494
./osd/OSDMap.h: 367: FAILED assert(exists(osd))
ceph version 0.56.1-25-g25a6b1b (25a6b1b325db2a2b45963f83623c447ec577c5ef)
1: /usr/bin/ceph-osd() [0x60db42]
2: /usr/bin/ceph-osd() [0x6e3b35]
3: (pg_interval_t::check_new_interval(std::vector<int,
std::allocator<int> > const&, std::vector<int, std::allocator<int> >
const&, std::vector<int, std::allocator<int> > const&, std::vector<int,
std::allocator<int> > const&, unsigned int, unsigned int,
std::tr1::shared_ptr<OSDMap const>, std::tr1::shared_ptr<OSDMap const>,
long, pg_t, std::map<unsigned int, pg_interval_t, std::less<unsigned
int>, std::allocator<std::pair<unsigned int const, pg_interval_t> > >*,
std::ostream*)+0x250) [0x935590]
4: (PG::start_peering_interval(std::tr1::shared_ptr<OSDMap const>,
std::vector<int, std::allocator<int> > const&, std::vector<int,
std::allocator<int> > const&)+0x353) [0x7563c3]
5: (PG::RecoveryState::Reset::react(PG::AdvMap const&)+0x21e) [0x75887e]
6: (boost::statechart::detail::reaction_result
boost::statechart::simple_state<PG::RecoveryState::Reset,
PG::RecoveryState::RecoveryMachine, boost::mpl::list<mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na>,
(boost::statechart::history_mode)0>::local_react_impl_non_empty::local_react_impl<boost::mpl::list5<boost::statechart::custom_reaction<PG::AdvMap>,
boost::statechart::custom_reaction<PG::ActMap>,
boost::statechart::custom_reaction<PG::NullEvt>,
boost::statechart::custom_reaction<PG::FlushedEvt>,
boost::statechart::transition<boost::statechart::event_base,
PG::RecoveryState::Crashed,
boost::statechart::detail::no_context<boost::statechart::event_base>,
&boost::statechart::detail::no_context<boost::statechart::event_base>::no_function>
>, boost::statechart::simple_state<PG::RecoveryState::Reset,
PG::RecoveryState::RecoveryMachine, boost::mpl::list<mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na>,
(boost::statechart::history_mode)0>
>(boost::statechart::simple_state<PG::RecoveryState::Reset,
PG::RecoveryState::RecoveryMachine, boost::mpl::list<mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na>,
(boost::statechart::history_mode)0>&, boost::statechart::event_base
const&, void const*)+0x86) [0x78abb6]
7: (boost::statechart::detail::reaction_result
boost::statechart::simple_state<PG::RecoveryState::Reset,
PG::RecoveryState::RecoveryMachine, boost::mpl::list<mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na>,
(boost::statechart::history_mode)0>::local_react_impl_non_empty::local_react_impl<boost::mpl::list<boost::statechart::custom_reaction<PG::QueryState>,
boost::statechart::custom_reaction<PG::AdvMap>,
boost::statechart::custom_reaction<PG::ActMap>,
boost::statechart::custom_reaction<PG::NullEvt>,
boost::statechart::custom_reaction<PG::FlushedEvt>,
boost::statechart::transition<boost::statechart::event_base,
PG::RecoveryState::Crashed,
boost::statechart::detail::no_context<boost::statechart::event_base>,
&boost::statechart::detail::no_context<boost::statechart::event_base>::no_function>,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>,
boost::statechart::simple_state<PG::RecoveryState::Reset,
PG::RecoveryState::RecoveryMachine, boost::mpl::list<mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na>,
(boost::statechart::history_mode)0>
>(boost::statechart::simple_state<PG::RecoveryState::Reset,
PG::RecoveryState::RecoveryMachine, boost::mpl::list<mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na,
mpl_::na, mpl_::na, mpl_::na, mpl_::na>,
(boost::statechart::history_mode)0>&, boost::statechart::event_base
const&, void const*)+0x53) [0x78ac33]
8:
(boost::statechart::state_machine<PG::RecoveryState::RecoveryMachine,
PG::RecoveryState::Initial, std::allocator<void>,
boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base
const&)+0x5b) [0x76f58b]
9:
(boost::statechart::state_machine<PG::RecoveryState::RecoveryMachine,
PG::RecoveryState::Initial, std::allocator<void>,
boost::statechart::null_exception_translator>::process_event(boost::statechart::event_base
const&)+0x19) [0x76f619]
10: (PG::RecoveryState::handle_event(boost::statechart::event_base
const&, PG::RecoveryCtx*)+0x4d) [0x76f6cd]
11: (PG::handle_advance_map(std::tr1::shared_ptr<OSDMap const>,
std::tr1::shared_ptr<OSDMap const>, std::vector<int, std::allocator<int>
>&, std::vector<int, std::allocator<int> >&, PG::RecoveryCtx*)+0x196)
[0x72bf46]
12: (OSD::advance_pg(unsigned int, PG*, PG::RecoveryCtx*,
std::set<boost::intrusive_ptr<PG>, std::less<boost::intrusive_ptr<PG> >,
std::allocator<boost::intrusive_ptr<PG> > >*)+0x48b) [0x6cf14b]
13: (OSD::process_peering_events(std::list<PG*, std::allocator<PG*> >
const&)+0x2a6) [0x6cf7f6]
14: (OSD::PeeringWQ::_process(std::list<PG*, std::allocator<PG*> >
const&)+0x17) [0x70a3f7]
15: (ThreadPool::worker(ThreadPool::WorkThread*)+0x95c) [0x8ccccc]
16: (ThreadPool::WorkThread::entry()+0x10) [0x8cdc40]
17: /lib64/libpthread.so.0() [0x360de07d14]
18: (clone()+0x6d) [0x360d6f167d]
--
Jens Kristian Søgaard, Mermaid Consulting ApS,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2013-01-19 17:56 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-01-17 14:35 Hit suicide timeout after adding new osd Jens Kristian Søgaard
2013-01-17 14:47 ` Wido den Hollander
2013-01-17 14:50 ` Stefan Priebe
2013-01-17 15:33 ` Wido den Hollander
2013-01-17 15:37 ` Stefan Priebe
2013-01-17 17:17 ` Sage Weil
2013-01-17 20:32 ` Jens Kristian Søgaard
2013-01-17 22:03 ` Sage Weil
2013-01-18 11:24 ` Jens Kristian Søgaard
2013-01-18 21:28 ` Sage Weil
2013-01-18 21:36 ` Jens Kristian Søgaard
2013-01-18 21:44 ` Sage Weil
2013-01-19 9:25 ` Jens Kristian Søgaard
2013-01-19 16:44 ` Sage Weil
2013-01-19 17:56 ` Jens Kristian Søgaard [this message]
2013-01-19 18:19 ` Sage Weil
2013-01-19 18:40 ` Jens Kristian Søgaard
2013-01-19 20:08 ` Sage Weil
2013-01-19 20:29 ` Jens Kristian Søgaard
2013-01-19 22:04 ` Sage Weil
2013-01-21 0:14 ` Sage Weil
2013-01-21 6:59 ` Jens Kristian Søgaard
2013-01-21 7:11 ` Sage Weil
2013-01-23 12:14 ` Jens Kristian Søgaard
2013-01-23 12:26 ` Wido den Hollander
2013-01-23 12:29 ` Jens Kristian Søgaard
2013-01-23 13:13 ` Sage Weil
2013-01-23 20:59 ` Jens Kristian Søgaard
2013-01-23 22:56 ` Andrey Korolyov
2013-01-24 4:39 ` Sage Weil
2013-01-24 7:44 ` Andrey Korolyov
2013-01-24 18:01 ` Sage Weil
2013-02-17 11:21 ` Andrey Korolyov
2013-02-17 17:52 ` Sage Weil
2013-01-24 4:28 ` Sage Weil
2013-01-24 10:08 ` Jens Kristian Søgaard
2013-01-24 18:06 ` Sage Weil
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=50FADE65.5050403@mermaidconsulting.dk \
--to=jens@mermaidconsulting.dk \
--cc=ceph-devel@vger.kernel.org \
--cc=s.priebe@profihost.ag \
--cc=sage@inktank.com \
--cc=wido@widodh.nl \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.