* OSDs crashing with latest master
@ 2011-12-02 4:54 Noah Watkins
2011-12-02 5:07 ` Sage Weil
0 siblings, 1 reply; 2+ messages in thread
From: Noah Watkins @ 2011-12-02 4:54 UTC (permalink / raw)
To: ceph-devel
Running a new master install on 13 nodes. Half the OSDs crashed with the
following stack trace.
Thanks,
Noah
2011-12-01 20:31:17.974984 7f0826e07700 osd.4 11 OSD::ms_handle_reset()
2011-12-01 20:31:17.975030 7f0826e07700 osd.4 11 OSD::ms_handle_reset()
s=0x27c86c0
2011-12-01 20:35:45.729921 7f0826e07700 osd.4 11 OSD::ms_handle_reset()
2011-12-01 20:35:45.729987 7f0826e07700 osd.4 11 OSD::ms_handle_reset()
s=0x2097900
osd/ReplicatedPG.cc: In function 'void
ReplicatedPG::sub_op_modify(MOSDSubOp*)', in thread '7f08234ff700'
osd/ReplicatedPG.cc: 3373: FAILED assert(0)
ceph version 0.38-273-g813523a
(commit:813523a6816417c75cae0d6aac6e3222c5d6c605)
1: (ReplicatedPG::sub_op_modify(MOSDSubOp*)+0x1402) [0x4b4672]
2: (ReplicatedPG::do_sub_op(MOSDSubOp*)+0x83) [0x4d6d13]
3: (OSD::dequeue_op(PG*)+0x3e4) [0x541304]
4: (ThreadPool::worker()+0xa28) [0x5bef08]
5: (ThreadPool::WorkThread::entry()+0xd) [0x565f7d]
6: (()+0x7971) [0x7f0832a90971]
7: (clone()+0x6d) [0x7f083111b92d]
ceph version 0.38-273-g813523a
(commit:813523a6816417c75cae0d6aac6e3222c5d6c605)
1: (ReplicatedPG::sub_op_modify(MOSDSubOp*)+0x1402) [0x4b4672]
2: (ReplicatedPG::do_sub_op(MOSDSubOp*)+0x83) [0x4d6d13]
3: (OSD::dequeue_op(PG*)+0x3e4) [0x541304]
4: (ThreadPool::worker()+0xa28) [0x5bef08]
5: (ThreadPool::WorkThread::entry()+0xd) [0x565f7d]
6: (()+0x7971) [0x7f0832a90971]
7: (clone()+0x6d) [0x7f083111b92d]
*** Caught signal (Aborted) **
in thread 7f08234ff700
ceph version 0.38-273-g813523a
(commit:813523a6816417c75cae0d6aac6e3222c5d6c605)
1: /usr/bin/ceph-osd() [0x6b1d14]
2: (()+0xfb40) [0x7f0832a98b40]
3: (gsignal()+0x35) [0x7f0831068ba5]
4: (abort()+0x180) [0x7f083106c6b0]
5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f083190c6bd]
6: (()+0xb9906) [0x7f083190a906]
7: (()+0xb9933) [0x7f083190a933]
8: (()+0xb9a3e) [0x7f083190aa3e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char
const*)+0x39f) [0x5cce4f]
10: (ReplicatedPG::sub_op_modify(MOSDSubOp*)+0x1402) [0x4b4672]
11: (ReplicatedPG::do_sub_op(MOSDSubOp*)+0x83) [0x4d6d13]
12: (OSD::dequeue_op(PG*)+0x3e4) [0x541304]
13: (ThreadPool::worker()+0xa28) [0x5bef08]
14: (ThreadPool::WorkThread::entry()+0xd) [0x565f7d]
15: (()+0x7971) [0x7f0832a90971]
16: (clone()+0x6d) [0x7f083111b92d]
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: OSDs crashing with latest master
2011-12-02 4:54 OSDs crashing with latest master Noah Watkins
@ 2011-12-02 5:07 ` Sage Weil
0 siblings, 0 replies; 2+ messages in thread
From: Sage Weil @ 2011-12-02 5:07 UTC (permalink / raw)
To: Noah Watkins; +Cc: ceph-devel
On Thu, 1 Dec 2011, Noah Watkins wrote:
> Running a new master install on 13 nodes. Half the OSDs crashed with the
> following stack trace.
Fixed by commit 2f5bd5f737e831a03beb93c3928c74b59a59052e.
Thanks!
sage
>
> Thanks,
> Noah
>
> 2011-12-01 20:31:17.974984 7f0826e07700 osd.4 11 OSD::ms_handle_reset()
> 2011-12-01 20:31:17.975030 7f0826e07700 osd.4 11 OSD::ms_handle_reset()
> s=0x27c86c0
> 2011-12-01 20:35:45.729921 7f0826e07700 osd.4 11 OSD::ms_handle_reset()
> 2011-12-01 20:35:45.729987 7f0826e07700 osd.4 11 OSD::ms_handle_reset()
> s=0x2097900
> osd/ReplicatedPG.cc: In function 'void
> ReplicatedPG::sub_op_modify(MOSDSubOp*)', in thread '7f08234ff700'
> osd/ReplicatedPG.cc: 3373: FAILED assert(0)
> ceph version 0.38-273-g813523a
> (commit:813523a6816417c75cae0d6aac6e3222c5d6c605)
> 1: (ReplicatedPG::sub_op_modify(MOSDSubOp*)+0x1402) [0x4b4672]
> 2: (ReplicatedPG::do_sub_op(MOSDSubOp*)+0x83) [0x4d6d13]
> 3: (OSD::dequeue_op(PG*)+0x3e4) [0x541304]
> 4: (ThreadPool::worker()+0xa28) [0x5bef08]
> 5: (ThreadPool::WorkThread::entry()+0xd) [0x565f7d]
> 6: (()+0x7971) [0x7f0832a90971]
> 7: (clone()+0x6d) [0x7f083111b92d]
> ceph version 0.38-273-g813523a
> (commit:813523a6816417c75cae0d6aac6e3222c5d6c605)
> 1: (ReplicatedPG::sub_op_modify(MOSDSubOp*)+0x1402) [0x4b4672]
> 2: (ReplicatedPG::do_sub_op(MOSDSubOp*)+0x83) [0x4d6d13]
> 3: (OSD::dequeue_op(PG*)+0x3e4) [0x541304]
> 4: (ThreadPool::worker()+0xa28) [0x5bef08]
> 5: (ThreadPool::WorkThread::entry()+0xd) [0x565f7d]
> 6: (()+0x7971) [0x7f0832a90971]
> 7: (clone()+0x6d) [0x7f083111b92d]
> *** Caught signal (Aborted) **
> in thread 7f08234ff700
> ceph version 0.38-273-g813523a
> (commit:813523a6816417c75cae0d6aac6e3222c5d6c605)
> 1: /usr/bin/ceph-osd() [0x6b1d14]
> 2: (()+0xfb40) [0x7f0832a98b40]
> 3: (gsignal()+0x35) [0x7f0831068ba5]
> 4: (abort()+0x180) [0x7f083106c6b0]
> 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f083190c6bd]
> 6: (()+0xb9906) [0x7f083190a906]
> 7: (()+0xb9933) [0x7f083190a933]
> 8: (()+0xb9a3e) [0x7f083190aa3e]
> 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char
> const*)+0x39f) [0x5cce4f]
> 10: (ReplicatedPG::sub_op_modify(MOSDSubOp*)+0x1402) [0x4b4672]
> 11: (ReplicatedPG::do_sub_op(MOSDSubOp*)+0x83) [0x4d6d13]
> 12: (OSD::dequeue_op(PG*)+0x3e4) [0x541304]
> 13: (ThreadPool::worker()+0xa28) [0x5bef08]
> 14: (ThreadPool::WorkThread::entry()+0xd) [0x565f7d]
> 15: (()+0x7971) [0x7f0832a90971]
> 16: (clone()+0x6d) [0x7f083111b92d]
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2011-12-02 5:07 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-12-02 4:54 OSDs crashing with latest master Noah Watkins
2011-12-02 5:07 ` Sage Weil
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.