public inbox for trinity@vger.kernel.org
 help / color / mirror / Atom feed
* msgrcv (70) returned ENOSYS, marking as inactive.
@ 2013-06-13 19:41 Tommi Rantala
  2013-06-13 20:15 ` Dave Jones
  0 siblings, 1 reply; 3+ messages in thread
From: Tommi Rantala @ 2013-06-13 19:41 UTC (permalink / raw)
  To: trinity

Latest trinity from git is constantly complaining:

[32492] Random reseed: 3901037224
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
[watchdog] 200253 iterations. [F:161784 S:38468]
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
trinity: malloc.c:3616: _int_malloc: Assertion `(unsigned long)(size)
>= (unsigned long)(nb)' failed.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.
msgrcv (70) returned ENOSYS, marking as inactive.

Whats up?

I collected bunch of cores with -D and see this a lot, related perhaps:

Core was generated by `./trinity -q -l off -C40 -D'.
Program terminated with signal 11, Segmentation fault.
#0  mkcall (childno=childno@entry=1) at syscall.c:292
292                             syscalls_32bit[call32].entry->flags &= ~ACTIVE;
#0  mkcall (childno=childno@entry=1) at syscall.c:292
#1  0x0000000000407e8c in do_random_syscalls (childno=childno@entry=1)
at random-syscalls.c:125

Tommi

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: msgrcv (70) returned ENOSYS, marking as inactive.
  2013-06-13 19:41 msgrcv (70) returned ENOSYS, marking as inactive Tommi Rantala
@ 2013-06-13 20:15 ` Dave Jones
  2013-06-13 20:33   ` Dave Jones
  0 siblings, 1 reply; 3+ messages in thread
From: Dave Jones @ 2013-06-13 20:15 UTC (permalink / raw)
  To: Tommi Rantala; +Cc: trinity

On Thu, Jun 13, 2013 at 10:41:30PM +0300, Tommi Rantala wrote:
 > Latest trinity from git is constantly complaining:

weird.

 > [32492] Random reseed: 3901037224
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > [watchdog] 200253 iterations. [F:161784 S:38468]
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.
 > msgrcv (70) returned ENOSYS, marking as inactive.

puzzling.  this should (obviously) only occur once.
Maybe it's happening once per thread at the same time,
so they're all racing. (I should add some locking around
the modifications of the syscall table I suppose).

 > trinity: malloc.c:3616: _int_malloc: Assertion `(unsigned long)(size)
 > >= (unsigned long)(nb)' failed.
 
This is trinity corrupting itself. Seen this occasionally for a while now,
but haven't had time to track it down.  Need to spend some time
with scripts/test-all-sequentially.sh to figure out a) which syscall
is causing it, and b) what memory it's scribbling over.

 > I collected bunch of cores with -D and see this a lot, related perhaps:
 > 
 > Core was generated by `./trinity -q -l off -C40 -D'.
 > Program terminated with signal 11, Segmentation fault.
 > #0  mkcall (childno=childno@entry=1) at syscall.c:292
 > 292                             syscalls_32bit[call32].entry->flags &= ~ACTIVE;
 > #0  mkcall (childno=childno@entry=1) at syscall.c:292
 > #1  0x0000000000407e8c in do_random_syscalls (childno=childno@entry=1)
 > at random-syscalls.c:125

Can you see what syscalls_32bit, and call32 are at that point ?
Also a backtrace might be useful.

Is it always the 32bit path ?

	Dave

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: msgrcv (70) returned ENOSYS, marking as inactive.
  2013-06-13 20:15 ` Dave Jones
@ 2013-06-13 20:33   ` Dave Jones
  0 siblings, 0 replies; 3+ messages in thread
From: Dave Jones @ 2013-06-13 20:33 UTC (permalink / raw)
  To: Tommi Rantala; +Cc: trinity

On Thu, Jun 13, 2013 at 04:15:53PM -0400, Dave Jones wrote:
 > On Thu, Jun 13, 2013 at 10:41:30PM +0300, Tommi Rantala wrote:
 >  > Latest trinity from git is constantly complaining:
 > 
 > weird.
 > 
 >  > [32492] Random reseed: 3901037224
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > [watchdog] 200253 iterations. [F:161784 S:38468]
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 >  > msgrcv (70) returned ENOSYS, marking as inactive.
 > 
 > puzzling.  this should (obviously) only occur once.
 > Maybe it's happening once per thread at the same time,
 > so they're all racing. (I should add some locking around
 > the modifications of the syscall table I suppose).

Something that's even more puzzling about this..
What are the odds all threads hit that at the same time
(unless you did -C msgrcv, which it doesn't look like you
 did from the core dump)

I wonder if I've broken seeding again.
Leave logging enabled, and compare a couple of the child logs,
just to make sure they're doing different syscalls.

Would be surprised though, the only thing remotely relevant
that changed is yesterdays rand32 changes in 7d86b1ca5ddc1ac04468d6ee7a0a74e931b67dae

	Dave

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2013-06-13 20:33 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-06-13 19:41 msgrcv (70) returned ENOSYS, marking as inactive Tommi Rantala
2013-06-13 20:15 ` Dave Jones
2013-06-13 20:33   ` Dave Jones

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox