post 3.5, phantom signals.

post 3.5, phantom signals.

[Dave Jones]

Since 3.5, I've started noticing weird things happening with signal delivery.
Things I've seen so far..

- long running tasks SIGINT, even though I wasn't even anywhere near the keyboard.
- processes running inside screen/tmux disappearing (and taking the whole session with them).

And now, while waiting for a kernel to build I just saw..

/bin/sh: line 1:  3274 Hangup                  gcc
-Wp,-MD,net/netfilter/.xt_HL.o.d -nostdinc -isystem
/usr/lib/gcc/x86_64-redhat-linux/4.7.0/include
-I/home/davej/src/linux-dj/arch/x86/include -Iarch/x86/include/generated
-Iinclude -include /home/davej/src/linux-dj/include/linux/kconfig.h
-D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs
-fno-strict-aliasing -fno-common -Werror-implicit-function-declaration
-Wno-format-security -fno-delete-null-pointer-checks -O2 -m64
-mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time
-maccumulate-outgoing-args -fstack-protector -DCONFIG_X86_X32_ABI
-DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1
-DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_AVX=1 -pipe
-Wno-sign-compare -fno-asynchronous-unwind-tables -mno-sse -mno-mmx
-mno-sse2 -mno-3dnow -mno-avx -Wframe-larger-than=2048
-Wno-unused-but-set-variable -fno-omit-frame-pointer
-fno-optimize-sibling-calls -g -pg -Wdeclaration-after-statement
-Wno-pointer-sign -fno-strict-overflow -fconserve-stack
-DCC_HAVE_ASM_GOTO -DMODULE -D"KBUILD_STR(s)=#s"
-D"KBUILD_BASENAME=KBUILD_STR(xt_HL)"
-D"KBUILD_MODNAME=KBUILD_STR(xt_HL)" -c -o net/netfilter/xt_HL.o
net/netfilter/xt_HL.c

Any thoughts on specific commits that might be guilty ?

	Dave

Re: post 3.5, phantom signals.

[Linus Torvalds]

On Tue, Jul 31, 2012 at 12:17 PM, Dave Jones <davej@redhat.com> wrote:
> Since 3.5, I've started noticing weird things happening with signal delivery.
> Things I've seen so far..
>
> - long running tasks SIGINT, even though I wasn't even anywhere near the keyboard.
> - processes running inside screen/tmux disappearing (and taking the whole session with them).

I can't even *begin* to guess how something like that could happen due
to kernel issues.

The signals you see are TTY signals, but SIGINT/HUP really don't make
sense as a tty layer bug either. If it was *just* SIGHUP I'd think it
was some tty refcounting issue, but with different signals that sounds
very very unlikely.

Quite frankly, from those symptoms I'd suspect the terminal program
being confused over a kernel issue. Of course, it could be a kernel
change that in turn confuses a terminal program.

             Linus

Re: post 3.5, phantom signals.

[Oleg Nesterov]

On 07/31, Dave Jones wrote:
>
> Since 3.5, I've started noticing weird things happening with signal delivery.
> Things I've seen so far..
>
> - long running tasks SIGINT, even though I wasn't even anywhere near the keyboard.
> - processes running inside screen/tmux disappearing (and taking the whole session with them).
>
> And now, while waiting for a kernel to build I just saw..
>
> /bin/sh: line 1:  3274 Hangup                  gcc

Currently I have no idea where this change could come from...

Well, not sure this will help... but if you can reproduce this,
may be you can identify the sender of the wrong SIGHUP at least.

	cd /sys/kernel/debug/tracing/
	echo "sig==1" >> events/signal/signal_generate/filter
	echo 1 >> options/stacktrace
	echo 1 >> events/signal/signal_generate/enable
	cat trace_pipe > SIGHUP_TRACE

and then rebuild the kernel.

Oleg.

Re: post 3.5, phantom signals.

[Dave Jones]

On Wed, Aug 01, 2012 at 05:11:34PM +0200, Oleg Nesterov wrote:
 > On 07/31, Dave Jones wrote:
 > >
 > > Since 3.5, I've started noticing weird things happening with signal delivery.
 > > Things I've seen so far..
 > >
 > > - long running tasks SIGINT, even though I wasn't even anywhere near the keyboard.
 > > - processes running inside screen/tmux disappearing (and taking the whole session with them).
 > >
 > > And now, while waiting for a kernel to build I just saw..
 > >
 > > /bin/sh: line 1:  3274 Hangup                  gcc
 > 
 > Currently I have no idea where this change could come from...
 > 
 > Well, not sure this will help... but if you can reproduce this,
 > may be you can identify the sender of the wrong SIGHUP at least.
 > 
 > 	cd /sys/kernel/debug/tracing/
 > 	echo "sig==1" >> events/signal/signal_generate/filter
 > 	echo 1 >> options/stacktrace
 > 	echo 1 >> events/signal/signal_generate/enable
 > 	cat trace_pipe > SIGHUP_TRACE
 > 
 > and then rebuild the kernel.

I'll give that a shot when I'm back on running kernel of the day.
Right now, I'm bisecting something that requires an hour or so of testing
between each step, so that's pretty much written off this week.

	Dave

Re: post 3.5, phantom signals.

[Frank Ch. Eigler]

Oleg Nesterov <oleg@redhat.com> writes:

> [...]
> Well, not sure this will help... but if you can reproduce this,
> may be you can identify the sender of the wrong SIGHUP at least.
>
> 	cd /sys/kernel/debug/tracing/
> 	echo "sig==1" >> events/signal/signal_generate/filter
> 	echo 1 >> options/stacktrace
> 	echo 1 >> events/signal/signal_generate/enable
> 	cat trace_pipe > SIGHUP_TRACE

FWIW, just in case your ftrace subsystem is already busy, this
two-liner is equivalent:

# stap -o SIGHUP_TRACE -e 'probe kernel.trace("signal_generate")
                              { if ($sig==1) print_backtrace() }'

- FChE