From: Steve Freyder <steve@freyder.net>
To: "xenomai@xenomai.org" <xenomai@xenomai.org>
Subject: Possible Cobalt mqueue issue
Date: Mon, 25 Feb 2019 10:53:15 -0600 [thread overview]
Message-ID: <5C741D7B.5080108@freyder.net> (raw)
Greetings again,
Recently I have converted my codebase from using Alchemy-based queues
(rt_queue_xx) to Cobalt (Posix) mqueues for all inter-process
communication, and using rt_queue queues only for communication between
threads in the same process.
This is running on Xenomai 3.0.7 built from -next (our vendor does the
Xenomai/kernel builds):
Linux g3l-36 4.1.18_C01571-15S01-00.002.zimg+83fdace666 #4 SMP Tue Aug
21 11:22:31 CDT 2018 armv7l GNU/Linux
This happened as my main process was starting during boot. I have not
been able to reproduce this, but I thought maybe the output would be useful.
Thanks in advance,
Best regards,
Steve
=============================================================================
[ 13.056376] I-pipe: Detected stalled head domain, probably caused by
a bug.
[ 13.056376] A critical section may have been left unterminated.
[ 13.069983] CPU: 1 PID: 1259 Comm: g2ld-main Not tainted
4.1.18_C01571-15S01-00.002.zimg+83fdace666 #4
[ 13.079309] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
[ 13.085854] Backtrace:
[ 13.088362] [<80014a64>] (dump_backtrace) from [<80014c9c>]
(show_stack+0x20/0x24)
[ 13.095948] r7:00000000 r6:00000080 r5:00000000 r4:80b85c94
[ 13.101747] [<80014c7c>] (show_stack) from [<806b679c>]
(dump_stack+0xa0/0xc4)
[ 13.109003] [<806b66fc>] (dump_stack) from [<800ab000>]
(ipipe_root_only+0x11c/0x188)
[ 13.116848] r9:80c49380 r8:00000000 r7:80c49380 r6:80b38e6c
r5:600b0113 r4:809afba4
[ 13.124758] [<800aaee4>] (ipipe_root_only) from [<80021bc0>]
(do_page_fault+0x2fc/0x4a8)
[ 13.132864] r10:bc34da84 r9:bba50000 r8:00000004 r7:bc34da40
r6:00000817 r5:bb291d58
[ 13.140845] r4:600b0093 r3:00000000
[ 13.144498] [<800218c4>] (do_page_fault) from [<800093ec>]
(do_DataAbort+0x44/0x1b4)
[ 13.152257] r10:80b3da3c r9:80b38e6c r8:00000004 r7:bb291d58
r6:800218c4 r5:00000817
[ 13.160238] r4:80b3dbbc
[ 13.162822] [<800093a8>] (do_DataAbort) from [<80015838>]
(__dabt_svc+0x58/0x80)
[ 13.170237] Exception stack(0xbb291d58 to 0xbb291da0)
[ 13.175309] 1d40: 00000000 00100100
[ 13.183510] 1d60: 00000000 00000000 bb8efc00 00000000 809ae9a0
00000001 00000000 80b38e6c
[ 13.191710] 1d80: 80b3da3c bb291dd4 bb291d58 bb291da0 800eee1c
801107dc 600b0093 ffffffff
[ 13.199902] r10:80b3da3c r9:80b38e6c r8:00000000 r7:bb291d8c
r6:ffffffff r5:600b0093
[ 13.207884] r4:801107dc
[ 13.210475] [<801106dc>] (mq_unref_inner) from [<80110ab8>]
(mq_unref+0x78/0xd8)
[ 13.217886] r10:00000000 r9:80b3da3c r8:80b38e6c r7:80c49380
r6:809ae9a0 r5:bb8efc00
[ 13.225867] r4:00000000
[ 13.228449] [<80110a40>] (mq_unref) from [<80110b3c>]
(mqd_close+0x24/0x28)
[ 13.235426] r7:80c49380 r6:809ae9a0 r5:600b0013 r4:bb8efc00
[ 13.241218] [<80110b18>] (mqd_close) from [<80106c58>]
(__put_fd+0x35c/0x3b8)
[ 13.248369] r5:600b0013 r4:bda592c4
[ 13.252020] [<801068fc>] (__put_fd) from [<80107a84>]
(rtdm_fd_close+0x190/0x2f0)
[ 13.259518] r10:bda592c4 r9:80b3da3c r8:86860b0b r7:80b38e6c
r6:809ae9a0 r5:00000003
[ 13.267499] r4:bda594c8
[ 13.270086] [<801078f4>] (rtdm_fd_close) from [<801119a8>]
(__cobalt_mq_open+0x730/0xc4c)
[ 13.278278] r10:ffffffef r9:80b3da3c r8:80b38e6c r7:809ae9a0
r6:bb8efc00 r5:00000042
[ 13.286259] r4:00000000
[ 13.288844] [<80111278>] (__cobalt_mq_open) from [<80111f54>]
(CoBaLt_mq_open+0x90/0xa0)
[ 13.296949] r10:80111ec4 r9:80c5c300 r8:80c5c300 r7:c0943808
r6:00000000 r5:7ee16b4c
[ 13.304930] r4:00000042
[ 13.307517] [<80111ec4>] (CoBaLt_mq_open) from [<8011ebb8>]
(handle_head_syscall+0xf8/0x3a4)
[ 13.315969] r6:00000001 r5:00000001 r4:bb291fb0
[ 13.320691] [<8011eac0>] (handle_head_syscall) from [<8011f498>]
(ipipe_fastcall_hook+0x20/0x28)
[ 13.329493] r10:00000006 r9:bb290000 r8:80010928 r7:000f0042
r6:7ee16b4c r5:00000000
[ 13.337474] r4:7ee16b8c
[ 13.340065] [<8011f478>] (ipipe_fastcall_hook) from [<80010808>]
(local_restart+0x20/0x44)
[ 13.348359] Unable to handle kernel NULL pointer dereference at
virtual address 00000004
[ 13.356468] pgd = bb2fc000
[ 13.359192] [00000004] *pgd=4da75831, *pte=00000000, *ppte=00000000
[ 13.365562] Internal error: Oops: 817 [#1] SMP ARM
[ 13.370371] Modules linked in: xeno_imx_uart xeno_gpio_mxc rt_e1000e
rtnet fec e1000e
[ 13.378412] CPU: 1 PID: 1259 Comm: g2ld-main Not tainted
4.1.18_C01571-15S01-00.002.zimg+83fdace666 #4
[ 13.387735] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
[ 13.394281] task: bba50000 ti: bb290000 task.ti: bb290000
[ 13.399699] PC is at mq_unref_inner+0x100/0x364
[ 13.404254] LR is at xnsynch_flush+0x154/0x2a0
[ 13.408717] pc : [<801107dc>] lr : [<800eee1c>] psr: 600b0093
[ 13.408717] sp : bb291da0 ip : bb291d58 fp : bb291dd4
[ 13.420213] r10: 80b3da3c r9 : 80b38e6c r8 : 00000000
[ 13.425454] r7 : 00000001 r6 : 809ae9a0 r5 : 00000000 r4 : bb8efc00
[ 13.432000] r3 : 00000000 r2 : 00000000 r1 : 00100100 r0 : 00000000
[ 13.438549] Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM
Segment user
[ 13.445789] Control: 10c5387d Table: 4b2fc04a DAC: 00000015
[ 13.451552] Process g2ld-main (pid: 1259, stack limit = 0xbb290220)
[ 13.457836] Stack: (0xbb291da0 to 0xbb292000)
[ 13.462218] 1da0: bb291df4 00000000 80b4d7cc 00000000 bb8efc00
809ae9a0 80c49380 80b38e6c
[ 13.470420] 1dc0: 80b3da3c 00000000 bb291df4 bb291dd8 80110ab8
801106e8 bb8efc00 600b0013
[ 13.478620] 1de0: 809ae9a0 80c49380 bb291e0c bb291df8 80110b3c
80110a4c bda592c4 600b0013
[ 13.486822] 1e00: bb291e4c bb291e10 80106c58 80110b24 80169c3c
00000000 80b4d7cc bb8efc00
[ 13.495022] 1e20: 80b50384 bda594c8 00000003 809ae9a0 80b38e6c
86860b0b 80b3da3c bda592c4
[ 13.503223] 1e40: bb291e94 bb291e50 80107a84 80106908 bb291e74
bb291e60 800a9e54 bda594c0
[ 13.511423] 1e60: bc39b40c 00000000 bb291e94 00000000 00000042
bb8efc00 809ae9a0 80b38e6c
[ 13.519624] 1e80: 80b3da3c ffffffef bb291f34 bb291e98 801119a8
80107900 00000000 bb291ec4
[ 13.527824] 1ea0: bb8efc30 bb8efc0c 00000000 809ae9a0 809ae9a0
809ae9a0 bb291f38 00000003
[ 13.536024] 1ec0: 0000000c 800f00f4 6c32672f 65645f64 00677562
00000000 00000000 00000102
[ 13.544225] 1ee0: 800ac9ec 800b05d8 809ae9a0 bb291fb0 00000006
00000001 80b38e6c 80114408
[ 13.552426] 1f00: c0943808 bf648800 bb291f24 00000042 7ee16b4c
00000000 c0943808 80c5c300
[ 13.560626] 1f20: 80c5c300 80111ec4 bb291f64 bb291f38 80111f54
80111284 00000000 00000800
[ 13.568827] 1f40: 00000200 00000000 7ee16b4c bb291fb0 00000001
00000001 bb291f9c bb291f68
[ 13.577027] 1f60: 8011ebb8 80111ed0 00000000 809afb40 809ad800
7ee16b8c 00000000 7ee16b4c
[ 13.585228] 1f80: 000f0042 80010928 bb290000 00000006 bb291fac
bb291fa0 8011f498 8011eacc
[ 13.593429] 1fa0: 00000000 bb291fb0 80010808 8011f484 10000025
7ee16b4c 00000042 00000000
[ 13.601630] 1fc0: 7ee16b8c 00000000 7ee16b4c 000f0042 00000000
00000000 4b310000 7ee16bbc
[ 13.609831] 1fe0: 7ee16b8c 7ee16b00 00000042 4b5db418 200b0010
10000025 00000000 00000000
[ 13.618021] Backtrace:
[ 13.620518] [<801106dc>] (mq_unref_inner) from [<80110ab8>]
(mq_unref+0x78/0xd8)
[ 13.627929] r10:00000000 r9:80b3da3c r8:80b38e6c r7:80c49380
r6:809ae9a0 r5:bb8efc00
[ 13.635912] r4:00000000
[ 13.638495] [<80110a40>] (mq_unref) from [<80110b3c>]
(mqd_close+0x24/0x28)
[ 13.645472] r7:80c49380 r6:809ae9a0 r5:600b0013 r4:bb8efc00
[ 13.651262] [<80110b18>] (mqd_close) from [<80106c58>]
(__put_fd+0x35c/0x3b8)
[ 13.658412] r5:600b0013 r4:bda592c4
[ 13.662064] [<801068fc>] (__put_fd) from [<80107a84>]
(rtdm_fd_close+0x190/0x2f0)
[ 13.669562] r10:bda592c4 r9:80b3da3c r8:86860b0b r7:80b38e6c
r6:809ae9a0 r5:00000003
[ 13.677544] r4:bda594c8
[ 13.680127] [<801078f4>] (rtdm_fd_close) from [<801119a8>]
(__cobalt_mq_open+0x730/0xc4c)
[ 13.688320] r10:ffffffef r9:80b3da3c r8:80b38e6c r7:809ae9a0
r6:bb8efc00 r5:00000042
[ 13.696301] r4:00000000
[ 13.698885] [<80111278>] (__cobalt_mq_open) from [<80111f54>]
(CoBaLt_mq_open+0x90/0xa0)
[ 13.706990] r10:80111ec4 r9:80c5c300 r8:80c5c300 r7:c0943808
r6:00000000 r5:7ee16b4c
[ 13.714971] r4:00000042
[ 13.717553] [<80111ec4>] (CoBaLt_mq_open) from [<8011ebb8>]
(handle_head_syscall+0xf8/0x3a4)
[ 13.726007] r6:00000001 r5:00000001 r4:bb291fb0
[ 13.730728] [<8011eac0>] (handle_head_syscall) from [<8011f498>]
(ipipe_fastcall_hook+0x20/0x28)
[ 13.739528] r10:00000006 r9:bb290000 r8:80010928 r7:000f0042
r6:7ee16b4c r5:00000000
[ 13.747510] r4:7ee16b8c
[ 13.750095] [<8011f478>] (ipipe_fastcall_hook) from [<80010808>]
(local_restart+0x20/0x44)
[ 13.758382] Code: e2840030 ebff79e4 e994000c e59f1244 (e5823004)
[ 13.764501] ---[ end trace a39fc1ccbff4481b ]---
[ 13.769156] ------------[ cut here ]------------
[ 13.773800] WARNING: CPU: 1 PID: 1259 at
/home/developer/oe/build_c01571-15/tmp/work-shared/c01571-15/kernel-source/kernel/ipipe/core.c:1764
__ipipe_spin_unlock_debug+0x4c/0x64()
[ 13.789717] Modules linked in: xeno_imx_uart xeno_gpio_mxc rt_e1000e
rtnet fec e1000e
[ 13.797748] CPU: 1 PID: 1259 Comm: g2ld-main Tainted: G D
4.1.18_C01571-15S01-00.002.zimg+83fdace666 #4
[ 13.808285] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
[ 13.814825] Backtrace:
[ 13.817317] [<80014a64>] (dump_backtrace) from [<80014c9c>]
(show_stack+0x20/0x24)
[ 13.824903] r7:00000000 r6:00000080 r5:00000000 r4:80b85c94
[ 13.830690] [<80014c7c>] (show_stack) from [<806b679c>]
(dump_stack+0xa0/0xc4)
[ 13.837943] [<806b66fc>] (dump_stack) from [<80031080>]
(warn_slowpath_common+0x88/0xc4)
[ 13.846049] r9:bf7cbf70 r8:800ab1fc r7:80867500 r6:000006e4
r5:00000009 r4:00000000
[ 13.853943] [<80030ff8>] (warn_slowpath_common) from [<80031178>]
(warn_slowpath_null+0x2c/0x34)
[ 13.862742] r8:80b3da3c r7:80b38e6c r6:00000000 r5:c0943aa0 r4:80bbbee8
[ 13.869597] [<8003114c>] (warn_slowpath_null) from [<800ab1fc>]
(__ipipe_spin_unlock_debug+0x4c/0x64)
[ 13.878841] [<800ab1b0>] (__ipipe_spin_unlock_debug) from
[<80067fa8>] (complete+0x4c/0x5c)
[ 13.887206] r5:c0943aa0 r4:c0943aa4
[ 13.890858] [<80067f5c>] (complete) from [<800f3704>]
(__xnthread_cleanup+0x1ac/0x338)
[ 13.898788] r7:80b38e6c r6:809ae9a0 r5:00000001 r4:c0943808
[ 13.904578] [<800f3558>] (__xnthread_cleanup) from [<801184e0>]
(ipipe_kevent_hook+0x498/0x870)
[ 13.913291] r9:bba50000 r8:00000004 r7:00000000 r6:600b0113
r5:600b0113 r4:bba50000
[ 13.921193] [<80118048>] (ipipe_kevent_hook) from [<800abe54>]
(__ipipe_notify_kevent+0x84/0x148)
[ 13.930079] r10:801107de r9:bba50000 r8:00000004 r7:00000000
r6:600b0113 r5:600b0113
[ 13.938055] r4:bf649800 r3:00000000
[ 13.941707] [<800abdd0>] (__ipipe_notify_kevent) from [<80032ca8>]
(do_exit+0x764/0x9cc)
[ 13.949812] r10:801107de r9:00000001 r8:801107e0 r7:0000000b
r6:00000080 r5:0000000b
[ 13.957786] r4:bba50000 r3:00000002
[ 13.961434] [<80032544>] (do_exit) from [<80015070>] (die+0x3d0/0x420)
[ 13.967975] r7:0000000b
[ 13.970557] [<80014ca0>] (die) from [<806b53a0>]
(__do_kernel_fault.part.0+0x64/0x84)
[ 13.978402] r10:bc34da84 r9:bba50000 r8:00000004 r7:bc34da40
r6:00000817 r5:bb291d58
[ 13.986376] r4:00000004
[ 13.988960] [<806b533c>] (__do_kernel_fault.part.0) from [<80021b8c>]
(do_page_fault+0x2c8/0x4a8)
[ 13.997845] r7:bc34da40 r4:600b0093
[ 14.001495] [<800218c4>] (do_page_fault) from [<800093ec>]
(do_DataAbort+0x44/0x1b4)
[ 14.009252] r10:80b3da3c r9:80b38e6c r8:00000004 r7:bb291d58
r6:800218c4 r5:00000817
[ 14.017227] r4:80b3dbbc
[ 14.019807] [<800093a8>] (do_DataAbort) from [<80015838>]
(__dabt_svc+0x58/0x80)
[ 14.027219] Exception stack(0xbb291d58 to 0xbb291da0)
[ 14.032289] 1d40: 00000000 00100100
[ 14.040489] 1d60: 00000000 00000000 bb8efc00 00000000 809ae9a0
00000001 00000000 80b38e6c
[ 14.048688] 1d80: 80b3da3c bb291dd4 bb291d58 bb291da0 800eee1c
801107dc 600b0093 ffffffff
[ 14.056879] r10:80b3da3c r9:80b38e6c r8:00000000 r7:bb291d8c
r6:ffffffff r5:600b0093
[ 14.064854] r4:801107dc
[ 14.067436] [<801106dc>] (mq_unref_inner) from [<80110ab8>]
(mq_unref+0x78/0xd8)
[ 14.074846] r10:00000000 r9:80b3da3c r8:80b38e6c r7:80c49380
r6:809ae9a0 r5:bb8efc00
[ 14.082822] r4:00000000
[ 14.085403] [<80110a40>] (mq_unref) from [<80110b3c>]
(mqd_close+0x24/0x28)
[ 14.092378] r7:80c49380 r6:809ae9a0 r5:600b0013 r4:bb8efc00
[ 14.098164] [<80110b18>] (mqd_close) from [<80106c58>]
(__put_fd+0x35c/0x3b8)
[ 14.105313] r5:600b0013 r4:bda592c4
[ 14.108961] [<801068fc>] (__put_fd) from [<80107a84>]
(rtdm_fd_close+0x190/0x2f0)
[ 14.116458] r10:bda592c4 r9:80b3da3c r8:86860b0b r7:80b38e6c
r6:809ae9a0 r5:00000003
[ 14.124433] r4:bda594c8
[ 14.127014] [<801078f4>] (rtdm_fd_close) from [<801119a8>]
(__cobalt_mq_open+0x730/0xc4c)
[ 14.135205] r10:ffffffef r9:80b3da3c r8:80b38e6c r7:809ae9a0
r6:bb8efc00 r5:00000042
[ 14.143179] r4:00000000
[ 14.145760] [<80111278>] (__cobalt_mq_open) from [<80111f54>]
(CoBaLt_mq_open+0x90/0xa0)
[ 14.153864] r10:80111ec4 r9:80c5c300 r8:80c5c300 r7:c0943808
r6:00000000 r5:7ee16b4c
[ 14.161839] r4:00000042
[ 14.164421] [<80111ec4>] (CoBaLt_mq_open) from [<8011ebb8>]
(handle_head_syscall+0xf8/0x3a4)
[ 14.172872] r6:00000001 r5:00000001 r4:bb291fb0
[ 14.177589] [<8011eac0>] (handle_head_syscall) from [<8011f498>]
(ipipe_fastcall_hook+0x20/0x28)
[ 14.186387] r10:00000006 r9:bb290000 r8:80010928 r7:000f0042
r6:7ee16b4c r5:00000000
[ 14.194362] r4:7ee16b8c
[ 14.196943] [<8011f478>] (ipipe_fastcall_hook) from [<80010808>]
(local_restart+0x20/0x44)
[ 14.205222] ---[ end trace a39fc1ccbff4481c ]---
next reply other threads:[~2019-02-25 16:53 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-25 16:53 Steve Freyder [this message]
2019-02-25 17:15 ` Possible Cobalt mqueue issue Jan Kiszka
2019-02-25 18:38 ` Steve Freyder
2019-02-25 18:56 ` Jan Kiszka
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5C741D7B.5080108@freyder.net \
--to=steve@freyder.net \
--cc=xenomai@xenomai.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.