* Ceph-mon crashing FAILED assert(addr_name.count(p->second) == 0) @ 2015-07-10 22:31 kernel neophyte 2015-07-12 8:25 ` Joao Eduardo Luis 0 siblings, 1 reply; 4+ messages in thread From: kernel neophyte @ 2015-07-10 22:31 UTC (permalink / raw) To: ceph-devel@vger.kernel.org Hi, I am seeing the following error every time I am trying to manually deploy ceph cluster and do a ceph -s : mon/MonMap.h: In function 'void MonMap::calc_ranks()' thread 7fb3ccfb6700 time 2015-07-10 15:27:56.004148 mon/MonMap.h: 47: FAILED assert(addr_name.count(p->second) == 0) ceph version 9.0.1-1445-g4a179ee (4a179eea527f7cbcf45eed4a63ad0fa8f744fc4a) 1: (()+0x12716b) [0x7fb3d53ec16b] 2: (()+0x1c0a6d) [0x7fb3d5485a6d] 3: (()+0x1b7909) [0x7fb3d547c909] 4: (()+0x1b8377) [0x7fb3d547d377] 5: (()+0x2fbbe7) [0x7fb3d55c0be7] 6: (()+0x310f1d) [0x7fb3d55d5f1d] 7: (()+0x3112a0) [0x7fb3d55d62a0] 8: (()+0x8182) [0x7fb3d9e05182] 9: (clone()+0x6d) [0x7fb3d9b3247d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. terminate called after throwing an instance of 'ceph::FailedAssertion' Aborted (core dumped) Any help greatly appreciated :-) -Neo ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Ceph-mon crashing FAILED assert(addr_name.count(p->second) == 0) 2015-07-10 22:31 Ceph-mon crashing FAILED assert(addr_name.count(p->second) == 0) kernel neophyte @ 2015-07-12 8:25 ` Joao Eduardo Luis 2015-07-15 17:12 ` kernel neophyte 0 siblings, 1 reply; 4+ messages in thread From: Joao Eduardo Luis @ 2015-07-12 8:25 UTC (permalink / raw) To: kernel neophyte, ceph-devel@vger.kernel.org On 07/10/2015 11:31 PM, kernel neophyte wrote: > Hi, > > I am seeing the following error every time I am trying to manually > deploy ceph cluster and do a ceph -s : > > mon/MonMap.h: In function 'void MonMap::calc_ranks()' thread > 7fb3ccfb6700 time 2015-07-10 15:27:56.004148 > > mon/MonMap.h: 47: FAILED assert(addr_name.count(p->second) == 0) Please send us a copy of your monmap and ceph.conf. You should also open a ticket on the tracker. Thanks! -Joao > > ceph version 9.0.1-1445-g4a179ee (4a179eea527f7cbcf45eed4a63ad0fa8f744fc4a) > > 1: (()+0x12716b) [0x7fb3d53ec16b] > > 2: (()+0x1c0a6d) [0x7fb3d5485a6d] > > 3: (()+0x1b7909) [0x7fb3d547c909] > > 4: (()+0x1b8377) [0x7fb3d547d377] > > 5: (()+0x2fbbe7) [0x7fb3d55c0be7] > > 6: (()+0x310f1d) [0x7fb3d55d5f1d] > > 7: (()+0x3112a0) [0x7fb3d55d62a0] > > 8: (()+0x8182) [0x7fb3d9e05182] > > 9: (clone()+0x6d) [0x7fb3d9b3247d] > > NOTE: a copy of the executable, or `objdump -rdS <executable>` is > needed to interpret this. > > terminate called after throwing an instance of 'ceph::FailedAssertion' > > Aborted (core dumped) > > Any help greatly appreciated :-) > > -Neo > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Ceph-mon crashing FAILED assert(addr_name.count(p->second) == 0) 2015-07-12 8:25 ` Joao Eduardo Luis @ 2015-07-15 17:12 ` kernel neophyte 2015-07-15 19:19 ` Vu Pham 0 siblings, 1 reply; 4+ messages in thread From: kernel neophyte @ 2015-07-15 17:12 UTC (permalink / raw) To: raju.kurunkad, vu, Sage Weil Cc: ceph-devel@vger.kernel.org, Joao Eduardo Luis I was able to dig bit further, I see this happening when using XIO as messenger (Simple & Async works fine). My Stack details are: Linux Distro: Ubuntu Kernel: 3.13.0-24-generic OFED: I see it happen in both MLNX_OFED_LINUX-2.4-1.0.4-ubuntu14.04-x86_64 & MLNX_OFED_LINUX-3.0-1.0.1-ubuntu14.04-x86_64 Accelio: I see it happen in both https://github.com/accelio/accelio Branch: master & https://github.com/vuhuong/accelio Branch: master-v1.3-fix monmap: monmaptool -p /tmp/monmap monmaptool: monmap file /tmp/monmap epoch 0 fsid 41e024e8-b224-41c7-ab13-9d9b681f3b61 last_changed 2015-07-15 09:48:38.037222 created 2015-07-15 09:48:38.037222 0: 10.13.10.189:6789/0 mon.abc-def-ghij07 Debug/Log: ceph -s 2015-07-15 09:54:07.763512 7f6df0c12700 -1 WARNING: the following dangerous and experimental features are enabled: ms-type-xio 2015-07-15 09:54:07.766830 7f6df0c12700 -1 WARNING: the following dangerous and experimental features are enabled: ms-type-xio 2015-07-15 09:54:07.766864 7f6df0c12700 -1 WARNING: experimental feature 'ms-type-xio' is enabled Please be aware that this feature is experimental, untested, unsupported, and may result in data corruption, data loss, and/or irreparable damage to your cluster. Do not use feature with important data. 2015-07-15 09:54:07.767331 7f6df0c12700 2 [debug] xio_mempool.c:500 xio_mempool_create - mempool: using regular allocator 2015-07-15 09:54:07.768458 7f6df0c12700 4 XioMessenger 0x7f6de4018040 get_connection: xio_uri rdma://10.13.10.189:6789 2015-07-15 09:54:07.768466 7f6df0c12700 4 Peer type: mon throttle_msgs: 1024 throttle_bytes: 536870912 2015-07-15 09:54:07.768483 7f6df0c12700 2 [debug] xio_mempool.c:494 xio_mempool_create - mempool: using huge pages allocator 2015-07-15 09:54:07.768581 7f6df0c12700 20 [trace] xio_rdma_management.c:2646 xio_rdma_open - xio_rdma_open: [new] handle:0x7f6de4055d40 2015-07-15 09:54:07.768587 7f6df0c12700 20 [trace] xio_nexus.c:1903 xio_nexus_open - nexus: [new] nexus:0x7f6de4055b20, transport_hndl:0x7f6de4055d40 2015-07-15 09:54:07.768595 7f6df0c12700 20 [trace] xio_nexus.c:1993 xio_nexus_connect - xio_nexus_connect: nexus:0x7f6de4055b20, rdma_hndl:0x7f6de4055d40, portal:rdma://10.13.10.189:6789 2015-07-15 09:54:07.768759 7f6df0c12700 2 [debug] xio_session_client.c:1022 xio_connect - xio_connect: session:0x7f6de40558f0, connection:0x7f6de4056e20, ctx:0x7f6de40179d0, nexus:0x7f6de4055b20 2015-07-15 09:54:07.768767 7f6df0c12700 2 new connection xcon: 0x7f6de4055460 up_ready on session 0x7f6de40558f0 2015-07-15 09:54:07.768860 7f6df0c12700 4 _send_message_impl 0x7f6de40578e0 new XioMsg 0x7f6df014f040 req_0 0x7f6df014f160 msg type 17 features: 0 conn 0x7f6de4056e20 sess 0x7f6de40558f0 2015-07-15 09:54:07.768867 7f6df0c12700 10 ex_cnt 0, req_off -1, msg_cnt 1 2015-07-15 09:54:07.768863 7f6de9ac4700 2 [debug] xio_rdma_management.c:2386 xio_handle_cm_event - cm event: [RDMA_CM_EVENT_ADDR_RESOLVED], hndl:0x7f6de4055d40, status:0 2015-07-15 09:54:07.769112 7f6de9ac4700 2 [debug] xio_rdma_management.c:2386 xio_handle_cm_event - cm event: [RDMA_CM_EVENT_ROUTE_RESOLVED], hndl:0x7f6de4055d40, status:0 2015-07-15 09:54:07.769707 7f6de9ac4700 20 [trace] xio_rdma_management.c:469 xio_cq_get - comp_vec:17 2015-07-15 09:54:07.770201 7f6de9ac4700 2 [debug] xio_rdma_management.c:961 xio_qp_create - rdma qp: [new] handle:0x7f6de4055d40, qp:0x260, max inline:448 2015-07-15 09:54:07.770231 7f6de9ac4700 4 xio_send_msg xio msg: sn: 0 timestamp: 256752506178532 2015-07-15 09:54:07.770232 7f6de9ac4700 4 xio_send_msg ceph header: front_len: 60 seq: 1 tid: 0 type: 17 prio: 0 name type: 8 name num: -1 version: 1 compat_version: 1 front_len: 60 middle_len: 0 data_len: 0 xio header: msg_cnt: 1 2015-07-15 09:54:07.770235 7f6de9ac4700 4 xio_send_msg ceph footer: front_crc: 0 middle_crc: 0 data_crc: 0 sig: 0 flags: 3 2015-07-15 09:54:07.778297 7f6de9ac4700 2 [debug] xio_rdma_management.c:2386 xio_handle_cm_event - cm event: [RDMA_CM_EVENT_ESTABLISHED], hndl:0x7f6de4055d40, status:0 2015-07-15 09:54:07.778308 7f6de9ac4700 2 [debug] xio_nexus.c:1632 xio_nexus_on_transport_event - nexus: [notification] - transport established. nexus:0x7f6de4055b20, transport:0x7f6de4055d40 2015-07-15 09:54:07.778330 7f6de9ac4700 20 [trace] xio_rdma_verbs.c:240 xio_reg_mr_ex_dev - before ibv_reg_mr 2015-07-15 09:54:07.778399 7f6de9ac4700 20 [trace] xio_rdma_verbs.c:242 xio_reg_mr_ex_dev - after ibv_reg_mr 2015-07-15 09:54:07.778404 7f6de9ac4700 20 [trace] xio_rdma_verbs.c:240 xio_reg_mr_ex_dev - before ibv_reg_mr 2015-07-15 09:54:07.778462 7f6de9ac4700 20 [trace] xio_rdma_verbs.c:242 xio_reg_mr_ex_dev - after ibv_reg_mr 2015-07-15 09:54:07.778480 7f6de9ac4700 2 [debug] xio_rdma_management.c:1274 xio_rdma_initial_pool_post_create - post_recv conn_setup rx task:0x7f6ddc009400 2015-07-15 09:54:07.778489 7f6de9ac4700 20 [trace] xio_nexus.c:384 xio_nexus_send_setup_req - send setup request 2015-07-15 09:54:07.778492 7f6de9ac4700 20 [trace] xio_nexus.c:426 xio_nexus_send_setup_req - xio_nexus_send_setup_req: nexus:0x7f6de4055b20, rdma_hndl:0x7f6de4055d40 2015-07-15 09:54:07.778497 7f6de9ac4700 20 [trace] xio_rdma_datapath.c:4180 xio_rdma_send_setup_req - rdma send setup request 2015-07-15 09:54:07.782250 7f6de9ac4700 20 [trace] xio_rdma_datapath.c:4319 xio_rdma_on_setup_msg - setup complete. send_buf_sz:17408 2015-07-15 09:54:07.782258 7f6de9ac4700 20 [trace] xio_nexus.c:661 xio_nexus_on_recv_setup_rsp - receiving setup response. nexus:0x7f6de4055b20 2015-07-15 09:54:07.782261 7f6de9ac4700 20 [trace] xio_nexus.c:714 xio_nexus_on_recv_setup_rsp - xio_nexus_on_recv_setup_rsp: nexus:0x7f6de4055b20, trans_hndl:0x7f6de4055d40 2015-07-15 09:54:07.782866 7f6de9ac4700 20 [trace] xio_rdma_verbs.c:240 xio_reg_mr_ex_dev - before ibv_reg_mr 2015-07-15 09:54:07.785038 7f6de9ac4700 20 [trace] xio_rdma_verbs.c:242 xio_reg_mr_ex_dev - after ibv_reg_mr 2015-07-15 09:54:07.785045 7f6de9ac4700 20 [trace] xio_rdma_verbs.c:240 xio_reg_mr_ex_dev - before ibv_reg_mr 2015-07-15 09:54:07.785247 7f6de9ac4700 20 [trace] xio_rdma_verbs.c:242 xio_reg_mr_ex_dev - after ibv_reg_mr 2015-07-15 09:54:07.785257 7f6de9ac4700 2 [debug] xio_rdma_management.c:1572 xio_rdma_primary_pool_slab_pre_create - pool buf:0x7f6de31e7000, mr:0x7f6ddc0093b0 2015-07-15 09:54:07.785495 7f6de9ac4700 2 [debug] xio_session_client.c:793 xio_client_on_nexus_event - session: [notification] - nexus established. session:0x7f6de40558f0, nexus:0x7f6de4055b20 2015-07-15 09:54:07.785692 7f6de9ac4700 2 [debug] xio_session_client.c:503 xio_on_setup_rsp_recv - task recycled 2015-07-15 09:54:07.785700 7f6de9ac4700 20 [trace] xio_session_client.c:517 xio_on_setup_rsp_recv - session state is now ONLINE. session:0x7f6de40558f0 2015-07-15 09:54:07.785703 7f6de9ac4700 20 [trace] xio_session_client.c:547 xio_on_setup_rsp_recv - session state is now ACCEPT. session:0x7f6de40558f0 2015-07-15 09:54:07.785711 7f6de9ac4700 2 [debug] xio_connection.c:1719 xio_disconnect_initial_connection - send fin request. session:0x7f6de40558f0, connection:0x7f6ddc015aa0 2015-07-15 09:54:07.785715 7f6de9ac4700 20 [trace] xio_connection.c:1726 xio_disconnect_initial_connection - connection 0x7f6ddc015aa0 state change: current_state:ONLINE, next_state:FIN_WAIT_1 2015-07-15 09:54:07.785725 7f6de9ac4700 20 [trace] xio_rdma_management.c:2646 xio_rdma_open - xio_rdma_open: [new] handle:0x7f6ddc0162d0 2015-07-15 09:54:07.785727 7f6de9ac4700 20 [trace] xio_nexus.c:1903 xio_nexus_open - nexus: [new] nexus:0x7f6ddc0160b0, transport_hndl:0x7f6ddc0162d0 2015-07-15 09:54:07.785728 7f6de9ac4700 2 [debug] xio_session_client.c:196 xio_session_accept_connections - reconnecting to rdma://10.13.10.189:6800. connection:0x7f6de4056e20, nexus:0x7f6ddc0160b0 2015-07-15 09:54:07.785734 7f6de9ac4700 20 [trace] xio_nexus.c:1993 xio_nexus_connect - xio_nexus_connect: nexus:0x7f6ddc0160b0, rdma_hndl:0x7f6ddc0162d0, portal:rdma://10.13.10.189:6800 2015-07-15 09:54:07.785761 7f6de9ac4700 2 [debug] xio_connection.c:2526 xio_on_fin_req_recv - fin request received. session:0x7f6de40558f0, connection:0x7f6ddc015aa0 2015-07-15 09:54:07.785764 7f6de9ac4700 2 [debug] xio_connection.c:1682 xio_send_fin_ack - send fin response. session:0x7f6de40558f0, connection:0x7f6ddc015aa0 2015-07-15 09:54:07.785767 7f6de9ac4700 2 [debug] xio_connection.c:2367 xio_on_fin_req_send_comp - got fin request send completion. session:0x7f6de40558f0, connection:0x7f6ddc015aa0 2015-07-15 09:54:07.785769 7f6de9ac4700 2 [debug] xio_connection.c:2454 xio_on_fin_ack_recv - got fin ack. session:0x7f6de40558f0, connection:0x7f6ddc015aa0 2015-07-15 09:54:07.785771 7f6de9ac4700 2 [debug] xio_connection.c:2495 xio_on_fin_ack_recv - connection 0x7f6ddc015aa0 state change: current_state:FIN_WAIT_1, next_state:FIN_WAIT_2 2015-07-15 09:54:07.785772 7f6de9ac4700 2 [debug] xio_connection.c:2560 xio_on_fin_ack_send_comp - fin ack send completion received. session:0x7f6de40558f0, connection:0x7f6ddc015aa0 2015-07-15 09:54:07.785773 7f6de9ac4700 2 [debug] xio_connection.c:2583 xio_on_fin_ack_send_comp - connection 0x7f6ddc015aa0 state change: current_state:FIN_WAIT_2, next_state:TIME_WAIT 2015-07-15 09:54:07.785784 7f6de9ac4700 2 [debug] xio_rdma_management.c:2386 xio_handle_cm_event - cm event: [RDMA_CM_EVENT_ADDR_RESOLVED], hndl:0x7f6ddc0162d0, status:0 2015-07-15 09:54:07.786008 7f6de9ac4700 2 [debug] xio_rdma_management.c:2386 xio_handle_cm_event - cm event: [RDMA_CM_EVENT_ROUTE_RESOLVED], hndl:0x7f6ddc0162d0, status:0 2015-07-15 09:54:07.786468 7f6de9ac4700 2 [debug] xio_rdma_management.c:961 xio_qp_create - rdma qp: [new] handle:0x7f6ddc0162d0, qp:0x262, max inline:448 2015-07-15 09:54:07.787793 7f6de9ac4700 2 [debug] xio_connection.c:2383 xio_close_time_wait - connection 0x7f6ddc015aa0 state change: current_state:TIME_WAIT, next_state:CLOSED 2015-07-15 09:54:07.787804 7f6de9ac4700 2 [debug] xio_connection.c:2183 xio_connection_destroy - xio_connection_destroy. session:0x7f6de40558f0, connection:0x7f6ddc015aa0 nexus:0x7f6de4055b20 nr:1, state:CLOSED 2015-07-15 09:54:07.787807 7f6de9ac4700 2 [debug] xio_connection.c:2108 xio_connection_post_destroy - xio_connection_post_destroy. session:0x7f6de40558f0, connection:0x7f6ddc015aa0 conn:0x7f6de4055b20 nr:1 2015-07-15 09:54:07.787810 7f6de9ac4700 20 [trace] xio_nexus.c:2139 xio_nexus_close - nexus: [putref] ptr:0x7f6de4055b20, refcnt:1 2015-07-15 09:54:07.787812 7f6de9ac4700 2 [debug] xio_session_client.c:808 xio_client_on_nexus_event - session: [notification] - nexus closed. session:0x7f6de40558f0, nexus:0x7f6de4055b20 2015-07-15 09:54:07.787814 7f6de9ac4700 20 [trace] xio_session.c:987 xio_on_nexus_closed - session:0x7f6de40558f0 - nexus:0x7f6de4055b20 close complete 2015-07-15 09:54:07.787816 7f6de9ac4700 20 [trace] xio_nexus.c:2111 xio_nexus_delayed_close - xio_nexus_deleyed close. nexus:0x7f6de4055b20, state:4 2015-07-15 09:54:07.787817 7f6de9ac4700 20 [trace] xio_connection.c:2122 xio_connection_post_destroy - lead connection is closed 2015-07-15 09:54:07.790664 7f6de9ac4700 2 [debug] xio_rdma_management.c:2386 xio_handle_cm_event - cm event: [RDMA_CM_EVENT_ESTABLISHED], hndl:0x7f6ddc0162d0, status:0 2015-07-15 09:54:07.790672 7f6de9ac4700 2 [debug] xio_nexus.c:1632 xio_nexus_on_transport_event - nexus: [notification] - transport established. nexus:0x7f6ddc0160b0, transport:0x7f6ddc0162d0 2015-07-15 09:54:07.790675 7f6de9ac4700 2 [debug] xio_rdma_management.c:1274 xio_rdma_initial_pool_post_create - post_recv conn_setup rx task:0x7f6ddc009400 2015-07-15 09:54:07.790680 7f6de9ac4700 20 [trace] xio_nexus.c:384 xio_nexus_send_setup_req - send setup request 2015-07-15 09:54:07.790682 7f6de9ac4700 20 [trace] xio_nexus.c:426 xio_nexus_send_setup_req - xio_nexus_send_setup_req: nexus:0x7f6ddc0160b0, rdma_hndl:0x7f6ddc0162d0 2015-07-15 09:54:07.790683 7f6de9ac4700 20 [trace] xio_rdma_datapath.c:4180 xio_rdma_send_setup_req - rdma send setup request 2015-07-15 09:54:07.794359 7f6de9ac4700 20 [trace] xio_rdma_datapath.c:4319 xio_rdma_on_setup_msg - setup complete. send_buf_sz:17408 2015-07-15 09:54:07.794367 7f6de9ac4700 20 [trace] xio_nexus.c:661 xio_nexus_on_recv_setup_rsp - receiving setup response. nexus:0x7f6ddc0160b0 2015-07-15 09:54:07.794369 7f6de9ac4700 20 [trace] xio_nexus.c:714 xio_nexus_on_recv_setup_rsp - xio_nexus_on_recv_setup_rsp: nexus:0x7f6ddc0160b0, trans_hndl:0x7f6ddc0162d0 2015-07-15 09:54:07.795445 7f6de9ac4700 20 [trace] xio_rdma_verbs.c:240 xio_reg_mr_ex_dev - before ibv_reg_mr 2015-07-15 09:54:07.798625 7f6de9ac4700 20 [trace] xio_rdma_verbs.c:242 xio_reg_mr_ex_dev - after ibv_reg_mr 2015-07-15 09:54:07.798633 7f6de9ac4700 20 [trace] xio_rdma_verbs.c:240 xio_reg_mr_ex_dev - before ibv_reg_mr 2015-07-15 09:54:07.798929 7f6de9ac4700 20 [trace] xio_rdma_verbs.c:242 xio_reg_mr_ex_dev - after ibv_reg_mr 2015-07-15 09:54:07.798940 7f6de9ac4700 2 [debug] xio_rdma_management.c:1572 xio_rdma_primary_pool_slab_pre_create - pool buf:0x7f6de25a1000, mr:0x7f6ddc015d40 2015-07-15 09:54:07.799371 7f6de9ac4700 2 [debug] xio_session_client.c:793 xio_client_on_nexus_event - session: [notification] - nexus established. session:0x7f6de40558f0, nexus:0x7f6ddc0160b0 2015-07-15 09:54:07.799376 7f6de9ac4700 2 [debug] xio_connection.c:2030 xio_connection_send_hello_req - send hello request. session:0x7f6de40558f0, connection:0x7f6de4056e20 2015-07-15 09:54:07.799522 7f6de9ac4700 2 [debug] xio_connection.c:2643 xio_on_connection_hello_rsp_recv - recv hello response. session:0x7f6de40558f0, connection:0x7f6de4056e20 2015-07-15 09:54:07.799530 7f6de9ac4700 2 [debug] xio_connection.c:2647 xio_on_connection_hello_rsp_recv - got hello response. session:0x7f6de40558f0, connection:0x7f6de4056e20 2015-07-15 09:54:07.799533 7f6de9ac4700 4 session event: connection established. reason: Success 2015-07-15 09:54:07.799538 7f6de9ac4700 2 connection established 0x7f6de4056e20 session 0x7f6de40558f0 xcon 0x7f6de4055460 2015-07-15 09:54:07.799541 7f6de9ac4700 2 learned my addr 10.13.10.189:0/1003368 2015-07-15 09:54:07.799546 7f6de9ac4700 2 client: connected from 10.13.10.189:49134/0 to 10.13.10.189:6800/0 2015-07-15 09:54:07.799576 7f6de9ac4700 11 on_msg_delivered xcon: 0x7f6de4055460 session: 0x7f6de40558f0 msg: 0x7f6df014f160 sn: 0 type: 17 tid: 0 seq: 1 2015-07-15 09:54:07.804553 7f6de9ac4700 10 on_msg_req receive req treq 0x7f6de8162598 msg_cnt 1 iov_base 0x7f6de31f1060 iov_len 216 nents 1 conn 0x7f6de4056e20 sess 0x7f6de40558f0 sn 0 2015-07-15 09:54:07.804564 7f6de9ac4700 4 on_msg_req msg_seq.size()=1 2015-07-15 09:54:07.804590 7f6de9ac4700 10 on_msg_req receive req treq 0x7f6de8160178 msg_cnt 1 iov_base 0x7f6de31e7060 iov_len 216 nents 1 conn 0x7f6de4056e20 sess 0x7f6de40558f0 sn 1 2015-07-15 09:54:07.804594 7f6de9ac4700 4 on_msg_req msg_seq.size()=1 mon/MonMap.h: In function 'void MonMap::calc_ranks()' thread 7f6de92c3700 time 2015-07-15 09:54:07.804654 mon/MonMap.h: 47: FAILED assert(addr_name.count(p->second) == 0) ceph version 9.0.1-1494-g8fc0496 (8fc049664bc798432e1750da86b1f216f85a842d) 1: (()+0x12637b) [0x7f6df585037b] 2: (()+0x1bf86d) [0x7f6df58e986d] 3: (()+0x1b6709) [0x7f6df58e0709] 4: (()+0x1b7177) [0x7f6df58e1177] 5: (()+0x2faa27) [0x7f6df5a24a27] 6: (()+0x30fd2d) [0x7f6df5a39d2d] 7: (()+0x3100b0) [0x7f6df5a3a0b0] 8: (()+0x8182) [0x7f6dfa785182] 9: (clone()+0x6d) [0x7f6dfa4b247d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. terminate called after throwing an instance of 'ceph::FailedAssertion' Aborted (core dumped) -> No Idea where this core is! looked at PWD, & /var/crash Any help / pointers greatly appreciated (-: -Neo On Sun, Jul 12, 2015 at 1:25 AM, Joao Eduardo Luis <joao@suse.de> wrote: > On 07/10/2015 11:31 PM, kernel neophyte wrote: >> Hi, >> >> I am seeing the following error every time I am trying to manually >> deploy ceph cluster and do a ceph -s : >> >> mon/MonMap.h: In function 'void MonMap::calc_ranks()' thread >> 7fb3ccfb6700 time 2015-07-10 15:27:56.004148 >> >> mon/MonMap.h: 47: FAILED assert(addr_name.count(p->second) == 0) > > Please send us a copy of your monmap and ceph.conf. > > You should also open a ticket on the tracker. > > Thanks! > > -Joao > >> >> ceph version 9.0.1-1445-g4a179ee (4a179eea527f7cbcf45eed4a63ad0fa8f744fc4a) >> >> 1: (()+0x12716b) [0x7fb3d53ec16b] >> >> 2: (()+0x1c0a6d) [0x7fb3d5485a6d] >> >> 3: (()+0x1b7909) [0x7fb3d547c909] >> >> 4: (()+0x1b8377) [0x7fb3d547d377] >> >> 5: (()+0x2fbbe7) [0x7fb3d55c0be7] >> >> 6: (()+0x310f1d) [0x7fb3d55d5f1d] >> >> 7: (()+0x3112a0) [0x7fb3d55d62a0] >> >> 8: (()+0x8182) [0x7fb3d9e05182] >> >> 9: (clone()+0x6d) [0x7fb3d9b3247d] >> >> NOTE: a copy of the executable, or `objdump -rdS <executable>` is >> needed to interpret this. >> >> terminate called after throwing an instance of 'ceph::FailedAssertion' >> >> Aborted (core dumped) >> >> Any help greatly appreciated :-) >> >> -Neo >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Ceph-mon crashing FAILED assert(addr_name.count(p->second) == 0) 2015-07-15 17:12 ` kernel neophyte @ 2015-07-15 19:19 ` Vu Pham 0 siblings, 0 replies; 4+ messages in thread From: Vu Pham @ 2015-07-15 19:19 UTC (permalink / raw) To: kernel neophyte, raju.kurunkad@sandisk.com, Sage Weil Cc: ceph-devel@vger.kernel.org, Joao Eduardo Luis Looking at debug log, I don't see any unusual thing in accelio & xio messenger. I guess the problem happened because you installed MLNX_OFED_LINUX packages after compiling & installing accelio and ceph Installing MLNX_OFED package (with xen & kdm supports) will look for and install (re-install) distributions default librados and librbd. This will mess up your librados & librbd from ceph v9.0.1 package. Could you re-compile & re-install accelio & ceph v9.0.1 again and reproduce? -vu On 7/15/2015 10:12:03 AM, "kernel neophyte" <neophyte.hacker001@gmail.com> wrote: >I was able to dig bit further, I see this happening when using XIO as >messenger (Simple & Async works fine). > >My Stack details are: > >Linux Distro: Ubuntu >Kernel: 3.13.0-24-generic >OFED: I see it happen in both >MLNX_OFED_LINUX-2.4-1.0.4-ubuntu14.04-x86_64 & >MLNX_OFED_LINUX-3.0-1.0.1-ubuntu14.04-x86_64 >Accelio: I see it happen in both https://github.com/accelio/accelio >Branch: master & https://github.com/vuhuong/accelio Branch: >master-v1.3-fix > > >monmap: > >monmaptool -p /tmp/monmap > >monmaptool: monmap file /tmp/monmap > >epoch 0 > >fsid 41e024e8-b224-41c7-ab13-9d9b681f3b61 > >last_changed 2015-07-15 09:48:38.037222 > >created 2015-07-15 09:48:38.037222 > >0: 10.13.10.189:6789/0 mon.abc-def-ghij07 > > > >Debug/Log: > >ceph -s > >2015-07-15 09:54:07.763512 7f6df0c12700 -1 WARNING: the following >dangerous and experimental features are enabled: ms-type-xio > >2015-07-15 09:54:07.766830 7f6df0c12700 -1 WARNING: the following >dangerous and experimental features are enabled: ms-type-xio > >2015-07-15 09:54:07.766864 7f6df0c12700 -1 WARNING: experimental >feature 'ms-type-xio' is enabled > >Please be aware that this feature is experimental, untested, > >unsupported, and may result in data corruption, data loss, > >and/or irreparable damage to your cluster. Do not use > >feature with important data. > > >2015-07-15 09:54:07.767331 7f6df0c12700 2 [debug] xio_mempool.c:500 >xio_mempool_create - mempool: using regular allocator > > >2015-07-15 09:54:07.768458 7f6df0c12700 4 XioMessenger 0x7f6de4018040 >get_connection: xio_uri rdma://10.13.10.189:6789 > >2015-07-15 09:54:07.768466 7f6df0c12700 4 Peer type: mon >throttle_msgs: 1024 throttle_bytes: 536870912 > >2015-07-15 09:54:07.768483 7f6df0c12700 2 [debug] xio_mempool.c:494 >xio_mempool_create - mempool: using huge pages allocator > > >2015-07-15 09:54:07.768581 7f6df0c12700 20 [trace] >xio_rdma_management.c:2646 xio_rdma_open - xio_rdma_open: [new] >handle:0x7f6de4055d40 > > >2015-07-15 09:54:07.768587 7f6df0c12700 20 [trace] xio_nexus.c:1903 >xio_nexus_open - nexus: [new] nexus:0x7f6de4055b20, >transport_hndl:0x7f6de4055d40 > > >2015-07-15 09:54:07.768595 7f6df0c12700 20 [trace] xio_nexus.c:1993 >xio_nexus_connect - xio_nexus_connect: nexus:0x7f6de4055b20, >rdma_hndl:0x7f6de4055d40, portal:rdma://10.13.10.189:6789 > > >2015-07-15 09:54:07.768759 7f6df0c12700 2 [debug] >xio_session_client.c:1022 xio_connect - xio_connect: >session:0x7f6de40558f0, connection:0x7f6de4056e20, ctx:0x7f6de40179d0, >nexus:0x7f6de4055b20 > > >2015-07-15 09:54:07.768767 7f6df0c12700 2 new connection xcon: >0x7f6de4055460 up_ready on session 0x7f6de40558f0 > >2015-07-15 09:54:07.768860 7f6df0c12700 4 _send_message_impl >0x7f6de40578e0 new XioMsg 0x7f6df014f040 req_0 0x7f6df014f160 msg type >17 features: 0 conn 0x7f6de4056e20 sess 0x7f6de40558f0 > >2015-07-15 09:54:07.768867 7f6df0c12700 10 ex_cnt 0, req_off -1, >msg_cnt 1 > >2015-07-15 09:54:07.768863 7f6de9ac4700 2 [debug] >xio_rdma_management.c:2386 xio_handle_cm_event - cm event: >[RDMA_CM_EVENT_ADDR_RESOLVED], hndl:0x7f6de4055d40, status:0 > > >2015-07-15 09:54:07.769112 7f6de9ac4700 2 [debug] >xio_rdma_management.c:2386 xio_handle_cm_event - cm event: >[RDMA_CM_EVENT_ROUTE_RESOLVED], hndl:0x7f6de4055d40, status:0 > > >2015-07-15 09:54:07.769707 7f6de9ac4700 20 [trace] >xio_rdma_management.c:469 xio_cq_get - comp_vec:17 > > >2015-07-15 09:54:07.770201 7f6de9ac4700 2 [debug] >xio_rdma_management.c:961 xio_qp_create - rdma qp: [new] >handle:0x7f6de4055d40, qp:0x260, max inline:448 > > >2015-07-15 09:54:07.770231 7f6de9ac4700 4 xio_send_msg xio msg: sn: 0 >timestamp: 256752506178532 > >2015-07-15 09:54:07.770232 7f6de9ac4700 4 xio_send_msg ceph header: >front_len: 60 seq: 1 tid: 0 type: 17 prio: 0 name type: 8 name num: -1 >version: 1 compat_version: 1 front_len: 60 middle_len: 0 data_len: 0 >xio header: msg_cnt: 1 > >2015-07-15 09:54:07.770235 7f6de9ac4700 4 xio_send_msg ceph footer: >front_crc: 0 middle_crc: 0 data_crc: 0 sig: 0 flags: 3 > >2015-07-15 09:54:07.778297 7f6de9ac4700 2 [debug] >xio_rdma_management.c:2386 xio_handle_cm_event - cm event: >[RDMA_CM_EVENT_ESTABLISHED], hndl:0x7f6de4055d40, status:0 > > >2015-07-15 09:54:07.778308 7f6de9ac4700 2 [debug] xio_nexus.c:1632 >xio_nexus_on_transport_event - nexus: [notification] - transport >established. nexus:0x7f6de4055b20, transport:0x7f6de4055d40 > > >2015-07-15 09:54:07.778330 7f6de9ac4700 20 [trace] >xio_rdma_verbs.c:240 xio_reg_mr_ex_dev - before ibv_reg_mr > > >2015-07-15 09:54:07.778399 7f6de9ac4700 20 [trace] >xio_rdma_verbs.c:242 xio_reg_mr_ex_dev - after ibv_reg_mr > > >2015-07-15 09:54:07.778404 7f6de9ac4700 20 [trace] >xio_rdma_verbs.c:240 xio_reg_mr_ex_dev - before ibv_reg_mr > > >2015-07-15 09:54:07.778462 7f6de9ac4700 20 [trace] >xio_rdma_verbs.c:242 xio_reg_mr_ex_dev - after ibv_reg_mr > > >2015-07-15 09:54:07.778480 7f6de9ac4700 2 [debug] >xio_rdma_management.c:1274 xio_rdma_initial_pool_post_create - >post_recv conn_setup rx task:0x7f6ddc009400 > > >2015-07-15 09:54:07.778489 7f6de9ac4700 20 [trace] xio_nexus.c:384 >xio_nexus_send_setup_req - send setup request > > >2015-07-15 09:54:07.778492 7f6de9ac4700 20 [trace] xio_nexus.c:426 >xio_nexus_send_setup_req - xio_nexus_send_setup_req: >nexus:0x7f6de4055b20, rdma_hndl:0x7f6de4055d40 > > >2015-07-15 09:54:07.778497 7f6de9ac4700 20 [trace] >xio_rdma_datapath.c:4180 xio_rdma_send_setup_req - rdma send setup >request > > >2015-07-15 09:54:07.782250 7f6de9ac4700 20 [trace] >xio_rdma_datapath.c:4319 xio_rdma_on_setup_msg - setup complete. >send_buf_sz:17408 > > >2015-07-15 09:54:07.782258 7f6de9ac4700 20 [trace] xio_nexus.c:661 >xio_nexus_on_recv_setup_rsp - receiving setup response. >nexus:0x7f6de4055b20 > > >2015-07-15 09:54:07.782261 7f6de9ac4700 20 [trace] xio_nexus.c:714 >xio_nexus_on_recv_setup_rsp - xio_nexus_on_recv_setup_rsp: >nexus:0x7f6de4055b20, trans_hndl:0x7f6de4055d40 > > >2015-07-15 09:54:07.782866 7f6de9ac4700 20 [trace] >xio_rdma_verbs.c:240 xio_reg_mr_ex_dev - before ibv_reg_mr > > >2015-07-15 09:54:07.785038 7f6de9ac4700 20 [trace] >xio_rdma_verbs.c:242 xio_reg_mr_ex_dev - after ibv_reg_mr > > >2015-07-15 09:54:07.785045 7f6de9ac4700 20 [trace] >xio_rdma_verbs.c:240 xio_reg_mr_ex_dev - before ibv_reg_mr > > >2015-07-15 09:54:07.785247 7f6de9ac4700 20 [trace] >xio_rdma_verbs.c:242 xio_reg_mr_ex_dev - after ibv_reg_mr > > >2015-07-15 09:54:07.785257 7f6de9ac4700 2 [debug] >xio_rdma_management.c:1572 xio_rdma_primary_pool_slab_pre_create - >pool buf:0x7f6de31e7000, mr:0x7f6ddc0093b0 > > >2015-07-15 09:54:07.785495 7f6de9ac4700 2 [debug] >xio_session_client.c:793 xio_client_on_nexus_event - session: >[notification] - nexus established. session:0x7f6de40558f0, >nexus:0x7f6de4055b20 > > >2015-07-15 09:54:07.785692 7f6de9ac4700 2 [debug] >xio_session_client.c:503 xio_on_setup_rsp_recv - task recycled > > >2015-07-15 09:54:07.785700 7f6de9ac4700 20 [trace] >xio_session_client.c:517 xio_on_setup_rsp_recv - session state is now >ONLINE. session:0x7f6de40558f0 > > >2015-07-15 09:54:07.785703 7f6de9ac4700 20 [trace] >xio_session_client.c:547 xio_on_setup_rsp_recv - session state is now >ACCEPT. session:0x7f6de40558f0 > > >2015-07-15 09:54:07.785711 7f6de9ac4700 2 [debug] >xio_connection.c:1719 xio_disconnect_initial_connection - send fin >request. session:0x7f6de40558f0, connection:0x7f6ddc015aa0 > > >2015-07-15 09:54:07.785715 7f6de9ac4700 20 [trace] >xio_connection.c:1726 xio_disconnect_initial_connection - connection >0x7f6ddc015aa0 state change: current_state:ONLINE, >next_state:FIN_WAIT_1 > > >2015-07-15 09:54:07.785725 7f6de9ac4700 20 [trace] >xio_rdma_management.c:2646 xio_rdma_open - xio_rdma_open: [new] >handle:0x7f6ddc0162d0 > > >2015-07-15 09:54:07.785727 7f6de9ac4700 20 [trace] xio_nexus.c:1903 >xio_nexus_open - nexus: [new] nexus:0x7f6ddc0160b0, >transport_hndl:0x7f6ddc0162d0 > > >2015-07-15 09:54:07.785728 7f6de9ac4700 2 [debug] >xio_session_client.c:196 xio_session_accept_connections - reconnecting >to rdma://10.13.10.189:6800. connection:0x7f6de4056e20, >nexus:0x7f6ddc0160b0 > > >2015-07-15 09:54:07.785734 7f6de9ac4700 20 [trace] xio_nexus.c:1993 >xio_nexus_connect - xio_nexus_connect: nexus:0x7f6ddc0160b0, >rdma_hndl:0x7f6ddc0162d0, portal:rdma://10.13.10.189:6800 > > >2015-07-15 09:54:07.785761 7f6de9ac4700 2 [debug] >xio_connection.c:2526 xio_on_fin_req_recv - fin request received. >session:0x7f6de40558f0, connection:0x7f6ddc015aa0 > > >2015-07-15 09:54:07.785764 7f6de9ac4700 2 [debug] >xio_connection.c:1682 xio_send_fin_ack - send fin response. >session:0x7f6de40558f0, connection:0x7f6ddc015aa0 > > >2015-07-15 09:54:07.785767 7f6de9ac4700 2 [debug] >xio_connection.c:2367 xio_on_fin_req_send_comp - got fin request send >completion. session:0x7f6de40558f0, connection:0x7f6ddc015aa0 > > >2015-07-15 09:54:07.785769 7f6de9ac4700 2 [debug] >xio_connection.c:2454 xio_on_fin_ack_recv - got fin ack. >session:0x7f6de40558f0, connection:0x7f6ddc015aa0 > > >2015-07-15 09:54:07.785771 7f6de9ac4700 2 [debug] >xio_connection.c:2495 xio_on_fin_ack_recv - connection 0x7f6ddc015aa0 >state change: current_state:FIN_WAIT_1, next_state:FIN_WAIT_2 > > >2015-07-15 09:54:07.785772 7f6de9ac4700 2 [debug] >xio_connection.c:2560 xio_on_fin_ack_send_comp - fin ack send >completion received. session:0x7f6de40558f0, connection:0x7f6ddc015aa0 > > >2015-07-15 09:54:07.785773 7f6de9ac4700 2 [debug] >xio_connection.c:2583 xio_on_fin_ack_send_comp - connection >0x7f6ddc015aa0 state change: current_state:FIN_WAIT_2, >next_state:TIME_WAIT > > >2015-07-15 09:54:07.785784 7f6de9ac4700 2 [debug] >xio_rdma_management.c:2386 xio_handle_cm_event - cm event: >[RDMA_CM_EVENT_ADDR_RESOLVED], hndl:0x7f6ddc0162d0, status:0 > > >2015-07-15 09:54:07.786008 7f6de9ac4700 2 [debug] >xio_rdma_management.c:2386 xio_handle_cm_event - cm event: >[RDMA_CM_EVENT_ROUTE_RESOLVED], hndl:0x7f6ddc0162d0, status:0 > > >2015-07-15 09:54:07.786468 7f6de9ac4700 2 [debug] >xio_rdma_management.c:961 xio_qp_create - rdma qp: [new] >handle:0x7f6ddc0162d0, qp:0x262, max inline:448 > > >2015-07-15 09:54:07.787793 7f6de9ac4700 2 [debug] >xio_connection.c:2383 xio_close_time_wait - connection 0x7f6ddc015aa0 >state change: current_state:TIME_WAIT, next_state:CLOSED > > >2015-07-15 09:54:07.787804 7f6de9ac4700 2 [debug] >xio_connection.c:2183 xio_connection_destroy - xio_connection_destroy. >session:0x7f6de40558f0, connection:0x7f6ddc015aa0 nexus:0x7f6de4055b20 >nr:1, state:CLOSED > > >2015-07-15 09:54:07.787807 7f6de9ac4700 2 [debug] >xio_connection.c:2108 xio_connection_post_destroy - >xio_connection_post_destroy. session:0x7f6de40558f0, >connection:0x7f6ddc015aa0 conn:0x7f6de4055b20 nr:1 > > >2015-07-15 09:54:07.787810 7f6de9ac4700 20 [trace] xio_nexus.c:2139 >xio_nexus_close - nexus: [putref] ptr:0x7f6de4055b20, refcnt:1 > > >2015-07-15 09:54:07.787812 7f6de9ac4700 2 [debug] >xio_session_client.c:808 xio_client_on_nexus_event - session: >[notification] - nexus closed. session:0x7f6de40558f0, >nexus:0x7f6de4055b20 > > >2015-07-15 09:54:07.787814 7f6de9ac4700 20 [trace] xio_session.c:987 >xio_on_nexus_closed - session:0x7f6de40558f0 - nexus:0x7f6de4055b20 >close complete > > >2015-07-15 09:54:07.787816 7f6de9ac4700 20 [trace] xio_nexus.c:2111 >xio_nexus_delayed_close - xio_nexus_deleyed close. >nexus:0x7f6de4055b20, state:4 > > >2015-07-15 09:54:07.787817 7f6de9ac4700 20 [trace] >xio_connection.c:2122 xio_connection_post_destroy - lead connection is >closed > > >2015-07-15 09:54:07.790664 7f6de9ac4700 2 [debug] >xio_rdma_management.c:2386 xio_handle_cm_event - cm event: >[RDMA_CM_EVENT_ESTABLISHED], hndl:0x7f6ddc0162d0, status:0 > > >2015-07-15 09:54:07.790672 7f6de9ac4700 2 [debug] xio_nexus.c:1632 >xio_nexus_on_transport_event - nexus: [notification] - transport >established. nexus:0x7f6ddc0160b0, transport:0x7f6ddc0162d0 > > >2015-07-15 09:54:07.790675 7f6de9ac4700 2 [debug] >xio_rdma_management.c:1274 xio_rdma_initial_pool_post_create - >post_recv conn_setup rx task:0x7f6ddc009400 > > >2015-07-15 09:54:07.790680 7f6de9ac4700 20 [trace] xio_nexus.c:384 >xio_nexus_send_setup_req - send setup request > > >2015-07-15 09:54:07.790682 7f6de9ac4700 20 [trace] xio_nexus.c:426 >xio_nexus_send_setup_req - xio_nexus_send_setup_req: >nexus:0x7f6ddc0160b0, rdma_hndl:0x7f6ddc0162d0 > > >2015-07-15 09:54:07.790683 7f6de9ac4700 20 [trace] >xio_rdma_datapath.c:4180 xio_rdma_send_setup_req - rdma send setup >request > > >2015-07-15 09:54:07.794359 7f6de9ac4700 20 [trace] >xio_rdma_datapath.c:4319 xio_rdma_on_setup_msg - setup complete. >send_buf_sz:17408 > > >2015-07-15 09:54:07.794367 7f6de9ac4700 20 [trace] xio_nexus.c:661 >xio_nexus_on_recv_setup_rsp - receiving setup response. >nexus:0x7f6ddc0160b0 > > >2015-07-15 09:54:07.794369 7f6de9ac4700 20 [trace] xio_nexus.c:714 >xio_nexus_on_recv_setup_rsp - xio_nexus_on_recv_setup_rsp: >nexus:0x7f6ddc0160b0, trans_hndl:0x7f6ddc0162d0 > > >2015-07-15 09:54:07.795445 7f6de9ac4700 20 [trace] >xio_rdma_verbs.c:240 xio_reg_mr_ex_dev - before ibv_reg_mr > > >2015-07-15 09:54:07.798625 7f6de9ac4700 20 [trace] >xio_rdma_verbs.c:242 xio_reg_mr_ex_dev - after ibv_reg_mr > > >2015-07-15 09:54:07.798633 7f6de9ac4700 20 [trace] >xio_rdma_verbs.c:240 xio_reg_mr_ex_dev - before ibv_reg_mr > > >2015-07-15 09:54:07.798929 7f6de9ac4700 20 [trace] >xio_rdma_verbs.c:242 xio_reg_mr_ex_dev - after ibv_reg_mr > > >2015-07-15 09:54:07.798940 7f6de9ac4700 2 [debug] >xio_rdma_management.c:1572 xio_rdma_primary_pool_slab_pre_create - >pool buf:0x7f6de25a1000, mr:0x7f6ddc015d40 > > >2015-07-15 09:54:07.799371 7f6de9ac4700 2 [debug] >xio_session_client.c:793 xio_client_on_nexus_event - session: >[notification] - nexus established. session:0x7f6de40558f0, >nexus:0x7f6ddc0160b0 > > >2015-07-15 09:54:07.799376 7f6de9ac4700 2 [debug] >xio_connection.c:2030 xio_connection_send_hello_req - send hello >request. session:0x7f6de40558f0, connection:0x7f6de4056e20 > > >2015-07-15 09:54:07.799522 7f6de9ac4700 2 [debug] >xio_connection.c:2643 xio_on_connection_hello_rsp_recv - recv hello >response. session:0x7f6de40558f0, connection:0x7f6de4056e20 > > >2015-07-15 09:54:07.799530 7f6de9ac4700 2 [debug] >xio_connection.c:2647 xio_on_connection_hello_rsp_recv - got hello >response. session:0x7f6de40558f0, connection:0x7f6de4056e20 > > >2015-07-15 09:54:07.799533 7f6de9ac4700 4 session event: connection >established. reason: Success > >2015-07-15 09:54:07.799538 7f6de9ac4700 2 connection established >0x7f6de4056e20 session 0x7f6de40558f0 xcon 0x7f6de4055460 > >2015-07-15 09:54:07.799541 7f6de9ac4700 2 learned my addr >10.13.10.189:0/1003368 > >2015-07-15 09:54:07.799546 7f6de9ac4700 2 client: connected from >10.13.10.189:49134/0 to 10.13.10.189:6800/0 > >2015-07-15 09:54:07.799576 7f6de9ac4700 11 on_msg_delivered xcon: >0x7f6de4055460 session: 0x7f6de40558f0 msg: 0x7f6df014f160 sn: 0 type: >17 tid: 0 seq: 1 > >2015-07-15 09:54:07.804553 7f6de9ac4700 10 on_msg_req receive req treq >0x7f6de8162598 msg_cnt 1 iov_base 0x7f6de31f1060 iov_len 216 nents 1 >conn 0x7f6de4056e20 sess 0x7f6de40558f0 sn 0 > >2015-07-15 09:54:07.804564 7f6de9ac4700 4 on_msg_req msg_seq.size()=1 > >2015-07-15 09:54:07.804590 7f6de9ac4700 10 on_msg_req receive req treq >0x7f6de8160178 msg_cnt 1 iov_base 0x7f6de31e7060 iov_len 216 nents 1 >conn 0x7f6de4056e20 sess 0x7f6de40558f0 sn 1 > >2015-07-15 09:54:07.804594 7f6de9ac4700 4 on_msg_req msg_seq.size()=1 > >mon/MonMap.h: In function 'void MonMap::calc_ranks()' thread >7f6de92c3700 time 2015-07-15 09:54:07.804654 > >mon/MonMap.h: 47: FAILED assert(addr_name.count(p->second) == 0) > > ceph version 9.0.1-1494-g8fc0496 >(8fc049664bc798432e1750da86b1f216f85a842d) > > 1: (()+0x12637b) [0x7f6df585037b] > > 2: (()+0x1bf86d) [0x7f6df58e986d] > > 3: (()+0x1b6709) [0x7f6df58e0709] > > 4: (()+0x1b7177) [0x7f6df58e1177] > > 5: (()+0x2faa27) [0x7f6df5a24a27] > > 6: (()+0x30fd2d) [0x7f6df5a39d2d] > > 7: (()+0x3100b0) [0x7f6df5a3a0b0] > > 8: (()+0x8182) [0x7f6dfa785182] > > 9: (clone()+0x6d) [0x7f6dfa4b247d] > > NOTE: a copy of the executable, or `objdump -rdS <executable>` is >needed to interpret this. > >terminate called after throwing an instance of 'ceph::FailedAssertion' > >Aborted (core dumped) -> No Idea where this core is! looked at PWD, & >/var/crash > > > >Any help / pointers greatly appreciated (-: > >-Neo > > > >On Sun, Jul 12, 2015 at 1:25 AM, Joao Eduardo Luis <joao@suse.de> >wrote: >> On 07/10/2015 11:31 PM, kernel neophyte wrote: >>> Hi, >>> >>> I am seeing the following error every time I am trying to manually >>> deploy ceph cluster and do a ceph -s : >>> >>> mon/MonMap.h: In function 'void MonMap::calc_ranks()' thread >>> 7fb3ccfb6700 time 2015-07-10 15:27:56.004148 >>> >>> mon/MonMap.h: 47: FAILED assert(addr_name.count(p->second) == 0) >> >> Please send us a copy of your monmap and ceph.conf. >> >> You should also open a ticket on the tracker. >> >> Thanks! >> >> -Joao >> >>> >>> ceph version 9.0.1-1445-g4a179ee >>>(4a179eea527f7cbcf45eed4a63ad0fa8f744fc4a) >>> >>> 1: (()+0x12716b) [0x7fb3d53ec16b] >>> >>> 2: (()+0x1c0a6d) [0x7fb3d5485a6d] >>> >>> 3: (()+0x1b7909) [0x7fb3d547c909] >>> >>> 4: (()+0x1b8377) [0x7fb3d547d377] >>> >>> 5: (()+0x2fbbe7) [0x7fb3d55c0be7] >>> >>> 6: (()+0x310f1d) [0x7fb3d55d5f1d] >>> >>> 7: (()+0x3112a0) [0x7fb3d55d62a0] >>> >>> 8: (()+0x8182) [0x7fb3d9e05182] >>> >>> 9: (clone()+0x6d) [0x7fb3d9b3247d] >>> >>> NOTE: a copy of the executable, or `objdump -rdS <executable>` is >>> needed to interpret this. >>> >>> terminate called after throwing an instance of >>>'ceph::FailedAssertion' >>> >>> Aborted (core dumped) >>> >>> Any help greatly appreciated :-) >>> >>> -Neo >>> -- >>> To unsubscribe from this list: send the line "unsubscribe >>>ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >> ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2015-07-15 19:33 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-07-10 22:31 Ceph-mon crashing FAILED assert(addr_name.count(p->second) == 0) kernel neophyte 2015-07-12 8:25 ` Joao Eduardo Luis 2015-07-15 17:12 ` kernel neophyte 2015-07-15 19:19 ` Vu Pham
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.