From: "Jens Kristian Søgaard" <jens@mermaidconsulting.dk>
To: Sage Weil <sage@inktank.com>
Cc: Stefan Priebe <s.priebe@profihost.ag>,
Wido den Hollander <wido@widodh.nl>,
"ceph-devel@vger.kernel.org" <ceph-devel@vger.kernel.org>
Subject: Re: Hit suicide timeout after adding new osd
Date: Fri, 18 Jan 2013 12:24:30 +0100 [thread overview]
Message-ID: <50F930EE.9070201@mermaidconsulting.dk> (raw)
In-Reply-To: <alpine.DEB.2.00.1301171402130.21470@cobra.newdream.net>
Hi Sage,
> Unfortunately I can't see what the thread gets stuck doing after it stops
> doing work (at an apparently normal point). Is there any chance you can
> attach to it with gdb as soon as the log slows down and the initial
> timeout messages appear? Or check the core file and see what thread
> 7fd8c1ff3700 is up to?
Here's the info from the core file:
(gdb) thread 45
[Switching to thread 45 (Thread 0x7fd8c1ff3700 (LWP 995))]
#0 0x000000360de0acb4 in pthread_rwlock_rdlock () from
/lib64/libpthread.so.0
(gdb) bt
#0 0x000000360de0acb4 in pthread_rwlock_rdlock () from
/lib64/libpthread.so.0
#1 0x00000000006e34a9 in RWLock::get_read (this=0x1aa54c8) at
common/RWLock.h:51
#2 0x00000000006a0857 in OSD::queue_want_up_thru
(this=this@entry=0x1aa44f0, want=want@entry=460) at osd/OSD.cc:2585
Python Exception <type 'exceptions.IndexError'> list index out of range:
#3 0x00000000006ce175 in OSD::process_peering_events (this=0x1aa44f0,
pgs=std::list) at osd/OSD.cc:6193
Python Exception <type 'exceptions.IndexError'> list index out of range:
#4 0x0000000000709617 in OSD::PeeringWQ::_process (this=<optimized
out>, pgs=std::list) at osd/OSD.h:718
#5 0x00000000008cbefc in ThreadPool::worker (this=0x1aa4938,
wt=0x3c539a0) at common/WorkQueue.cc:113
#6 0x00000000008cce70 in ThreadPool::WorkThread::entry (this=<optimized
out>) at common/WorkQueue.h:288
#7 0x000000360de07d14 in start_thread () from /lib64/libpthread.so.0
#8 0x000000360d6f167d in clone () from /lib64/libc.so.6
Does this help?
--
Jens Kristian Søgaard, Mermaid Consulting ApS,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2013-01-18 11:24 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-01-17 14:35 Hit suicide timeout after adding new osd Jens Kristian Søgaard
2013-01-17 14:47 ` Wido den Hollander
2013-01-17 14:50 ` Stefan Priebe
2013-01-17 15:33 ` Wido den Hollander
2013-01-17 15:37 ` Stefan Priebe
2013-01-17 17:17 ` Sage Weil
2013-01-17 20:32 ` Jens Kristian Søgaard
2013-01-17 22:03 ` Sage Weil
2013-01-18 11:24 ` Jens Kristian Søgaard [this message]
2013-01-18 21:28 ` Sage Weil
2013-01-18 21:36 ` Jens Kristian Søgaard
2013-01-18 21:44 ` Sage Weil
2013-01-19 9:25 ` Jens Kristian Søgaard
2013-01-19 16:44 ` Sage Weil
2013-01-19 17:56 ` Jens Kristian Søgaard
2013-01-19 18:19 ` Sage Weil
2013-01-19 18:40 ` Jens Kristian Søgaard
2013-01-19 20:08 ` Sage Weil
2013-01-19 20:29 ` Jens Kristian Søgaard
2013-01-19 22:04 ` Sage Weil
2013-01-21 0:14 ` Sage Weil
2013-01-21 6:59 ` Jens Kristian Søgaard
2013-01-21 7:11 ` Sage Weil
2013-01-23 12:14 ` Jens Kristian Søgaard
2013-01-23 12:26 ` Wido den Hollander
2013-01-23 12:29 ` Jens Kristian Søgaard
2013-01-23 13:13 ` Sage Weil
2013-01-23 20:59 ` Jens Kristian Søgaard
2013-01-23 22:56 ` Andrey Korolyov
2013-01-24 4:39 ` Sage Weil
2013-01-24 7:44 ` Andrey Korolyov
2013-01-24 18:01 ` Sage Weil
2013-02-17 11:21 ` Andrey Korolyov
2013-02-17 17:52 ` Sage Weil
2013-01-24 4:28 ` Sage Weil
2013-01-24 10:08 ` Jens Kristian Søgaard
2013-01-24 18:06 ` Sage Weil
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=50F930EE.9070201@mermaidconsulting.dk \
--to=jens@mermaidconsulting.dk \
--cc=ceph-devel@vger.kernel.org \
--cc=s.priebe@profihost.ag \
--cc=sage@inktank.com \
--cc=wido@widodh.nl \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.