Orphaned Processes and TCSETSW

Orphaned Processes and TCSETSW

[linuxassembly]

I've made a new program for Softer, and when it starts up, if it's not run under Softer it will start Softer itself, so that people can just type 'gpsmap' instead of 'softer gpsmap'.  It works well, until it exits.  Upon exit, Softer gives this error message:

Error calling TCSETSW to restore settings.

EAX Error Code: EIO -- I/O error
EAX: FFFFFFFB  EBX: 00000000  ECX: 00005403  EDX: 08054A96
ESI: 0806938E  EDI: 40002000  EBP: 00000000  ESP: BFFFFB24
EIP: 08048781  EFLAGS: 00000000 00000000 00000010 00000110
                                           ..OD   SZ A P C

And the console is left in raw mode.  (not keyboard raw, rather 'stty raw -echo' raw)  The line it's having trouble with is this:

sys sys_ioctl, 0, TCSETSW, old_termios; systrap "calling TCSETSW to restore settings."

...or in other words...

mov eax, sys_ioctl    ; sys_ioctl == 54
mov ebx, 0
mov ecx, TCSETSW      ; TCSETSW == 0x5403
mov edx, old_termios
int 0x80

Searching google all I could come up with was this:

     If  TOSTOP is set and a member of a background process group
     attempts   to   ioctl  its  controlling  terminal,  and that
     ioctl will modify terminal parameters (for example,  TCSETA,
     TCSETAW,  TCSETAF, or  TIOCSPGRP), its process group will be
     sent a   SIGTTOU  signal,  which  will  normally  cause  the
     members  of  that  process  group to stop.  If, however, the
     process is ignoring  or  holding  SIGTTOU,  the  ioctl  will
     succeed.  If the process is not ignoring or holding  SIGTTOU
     and is a member of an orphaned process group, the write will
     fail with  errno set to  EIO, and no signal will be sent.

However, TOSTOP isn't set, so it shouldn't happen.  Regardless, I tried adding signals 20, 21, and 22 to Softer's mile long ignore list, and although that stopped the EIO error, the console settings wern't being restored.  So I added a handler for those signals that just returns, but then the EIO error still occours, so apparently the kernel is rather set on not allowing Softer to change the terminal parameters.

If Softer exits first, there is no problem.  (e.g., if I press control-alt-backspace to make softer exit, or if I make the gpsmap program, upon exit, close it's stdin and stdout (which are just pipes to Softer), then sleep 0.2 seconds to give Softer a second to see the pipes are closed and exit, it works just fine.)  So I'm pretty sure it's something to do with this 'background process group' thing, since the process chain looks like this:

	shell -> gpsmap -> softer

And if gpsmap exits first, there's a link missing in the chain.  If I execute it with 'softer gpsmap' then it's like this:

	shell -> softer -> gpsmap

And there's never any problem regardless of which exits first.

If I can't find a solution to the problem, I'll probably just make it call wait on Softer to make sure Softer exits first, but I'd rather fix the problem than just work around it.

Any suggestions?  Anyone know more than I do about this controlling terminal orphaned background process group stuff?

Re: Orphaned Processes and TCSETSW

[willy meier]

linuxassembly@evobsyniva.com wrote:
> 
> I've made a new program for Softer, and when it starts up, if it's not run under Softer it will start Softer itself, so that people can just type 'gpsmap' instead of 'softer gpsmap'.  It works well, until it exits.  Upon exit, Softer gives this error message:
> 
> Error calling TCSETSW to restore settings.
	[ ... ]
> 
>         shell -> gpsmap -> softer
> 
> And if gpsmap exits first, there's a link missing in the chain.  If I execute it with 'softer gpsmap' then it's like this:
> 
>         shell -> softer -> gpsmap
> 
> And there's never any problem regardless of which exits first.
> 
> If I can't find a solution to the problem, I'll probably just make it call wait on Softer to make sure Softer exits first, but I'd rather fix the problem than just work around it.
> 
> Any suggestions?  Anyone know more than I do about this controlling terminal orphaned background process group stuff?

your problem may be due to interleaved saveing/restoreing of the rsp.
console state.

i.e. let program-1 save the console state, then modify; now program-2
starts & saves current state & modifies... which would restore _only_ if
the jobs would terminate in the same sequence as they were started.
many programs apparently save, modify and (attempt to) restore console
state!
I'd encounter that problem with 'less' and 'lib4th'/'F4'.


best,
	hp


-- 
Linux,Assembly,Forth: http://www.lxhp.in-berlin.de/index-lx.shtml
  >> xxxx -at- lxhp -dot- in-berlin -dot- de <<

correction: Re: Orphaned Processes and TCSETSW]

[willy meier]

> Any suggestions?  Anyone know more than I do about this controlling terminal orphaned background process group stuff?

your problem may be due to interleaved saveing/restoreing of the rsp.
console state.

i.e. let program-1 save the console state, then modify; now program-2
starts & saves current state & modifies... which would restore _only_ if
the jobs would terminate in the same sequence as they were started.
				^-- REVERSED (i.e. last restores first)
many programs apparently save, modify and (attempt to) restore console
state!
I'd encounter that problem with 'less' and 'lib4th'/'F4'.


best,
	hp


-- 
Linux,Assembly,Forth: http://www.lxhp.in-berlin.de/index-lx.shtml
  >> xxxx -at- lxhp -dot- in-berlin -dot- de <<

Re: Orphaned Processes and TCSETSW

[Philip Jacob Smith]

> i.e. let program-1 save the console state, then modify; now program-2
> starts & saves current state & modifies... which would restore _only_ if
> the jobs would terminate in the same sequence as they were started.
> 		            		^-- REVERSED (i.e. last restores first)
> many programs apparently save, modify and (attempt to) restore console
> state!

Yes, I think this is related.  Program-1 (gpsmap) doesn't modify any terminal settings, but rather it executes program-2 (softer) to do it, but I think the kernel may have something built into it to prevent what you described from happening, and so when it sees program-1 exit it disallows program-2 from changing the terminal settings under the expectation that it doing so would mess things up.  However, that's not the case this time, and in fact not allowing Softer to change the settings messes things up.

It's also possible that you can't change the settings when a program is currently blocking while reading the terminal, and since gpsmap exits, bash begins reading the terminal, and Linux then isn't able to allow Softer to change the settings while bash is reading.  I think I'm out of luck if that's the case.

- Pj

--- special one time signature line ---
(yes, blatant off-topic advertising)

Sucko's, the operating system so cool it...err...
(I don't want to use bad words on a public mailing list.)
http://www.evobsyniva.com/suckos/
It's probably a good thing it doesn't actually exist.

Re: Orphaned Processes and TCSETSW

[willy meier]

Philip Jacob Smith wrote:
> settings under the expectation that it doing so would mess things up. 
> However, that's not the case this time, and in fact not allowing Softer 
> to change the settings messes things up.

my solution, not at all perfect, but at least providing some last
resort, is the aequivalent to <stty sane> coded into the final clean-up
procedure plus, a (sort of) switch by which that part could be
dis-abled, at runtime.
at least, it returns the keyboard in an accessible state and, it's
results are something which can easily be managed w. the standard tools.

works well w. my forth/lesspipe example, but wasn't tested, further.

hp

-- 
Linux,Assembly,Forth: http://www.lxhp.in-berlin.de/index-lx.shtml
  >> xxxx -at- lxhp -dot- in-berlin -dot- de <<

Re: Orphaned Processes and TCSETSW

[Philip Jacob Smith]

> my solution, not at all perfect, but at least providing some last
> resort, is the aequivalent to <stty sane> coded into the final clean-up
> procedure plus, a (sort of) switch by which that part could be
> dis-abled, at runtime.
> at least, it returns the keyboard in an accessible state and, it's
> results are something which can easily be managed w. the standard tools.

I tried a few perl scripts...  (I'm retying these from memory, so if I forget a semicolon and perl won't accept them, forgive me.)

if (fork == 0) {
	sleep 1;
	`stty sane`;
};

For those who don't know perl, this script forks, then if it's the child, it sleeps for a second then executes the command 'stty sane', or if it's the parent, it'll just exit.

This script causes the error.  You'll see stty complain about an I/O error.

So I went back to the softer source and coded it so that when it does the final equivelent of 'stty sane' it first opens /dev/tty4 and does it on the file descriptor there, thinking maybe a new open might clear up the problem, but that didn't work, the error occoured even with the new file handle.

So I went back to perl and tried this:

if (fork == 0) {
	if (fork == 0) {
		if (fork == 0) {
			sleep 1;
			`stty sane`;
		};
	};
} else {
	sleep 4;
};

That's probably more forks than are necissary, but anyway this script works.  If you comment out the 'sleep 4' then it stops working.

This made me think that maybe it was failing because a process was reading from stdin, so I tried this script.

if (fork == 0) {
	if (fork == 0) {
		if (fork == 0) {
			sleep 1;
			`stty sane`;
		};
	};
} else {
	<>;
};
	
In perl, <> will read a line from standard input.

This script had no problems, so it's not reading from stdin that does it.  So then I tried this.

if (fork == 0) {
	if (fork == 0) {
		if (fork == 0) {
			sleep 3;
			`stty sane`;
		};
	};
} else {
	sleep 1;
	`stty raw`;
	sleep 4;
};

This script didn't have any problems either.

So now I'm thinking maybe it's bash, or that readline library it uses.  So I try a few other shells, they all do it too.  Then I take the first script, lengthen it's sleep time to 5 seconds, then after running it I execute 'sleep 10' before it gets around to doing the stty command, so that bash won't be reading the keyboard when it does the stty command.  This still causes the error.  So this makes me think that it's probably not the shell.

So I'm lost at this point.  As best I can tell, the problem occours when you have a parent-child chain like

shell -> process_1 -> process_2 -> process_3 -> process_4 -> process_5

and any process 2 or greater tries to modify terminal settings when process_1 has already exited.
There's no problem if process 3 or greater does the same after process 2 has exited, but rather the whole chain can do whatever it wants so long as process_1 hasn't exited, and once it has, none of them can do anything, regardless of which others have or haven't exited.

Interestingly, if I start another shell by typing 'bash', it now works as if the new bash is 'shell' in the above diagram, even though it's 'process_1'.  So I don't think it's something to do with who opened that particular TTY first.

So I guess that either bash (or the readline libaray) is making a system call after exit to cause this to happen, or when bash starts up it makes one system call to make this happen automatically.

So I tried one of the scripts that fail, but ran it like this:

	./fork_test.pl | sleep 5

This way it works just fine.  So I'm thinking maybe it's bash making a system call.

Doing an strace on bash, I see it make an ioctl 0x5410, which turns out to be TIOCSPGRP.  As best I can tell, this has something to do with changing the process group of the controlling terminal, or something like that.

So I go to console number 4, and type this:

stty raw -echo < /dev/tty1

Then I switch over to console 1, and sure enough, it did just what it was supposed to.

So it seems that it's not that Softer isn't in the process group, but that it was at one time in the process group, and isn't any longer.

So then I make softer call setsid to make it's own process group.  This fixes the problem, but now we have the problem you described before, as bash saves the incorrect settings when gpsmap exits, then softer fixes them, but of course when bash later goes to restore them, they're incorrect again.  All that trouble for something that didn't even work...

I also noticed that if softer is executed from the shell, it's setsid call fails, which I guess means that softer is the process group leader, not the shell.  So I guess once the process group leader is dead, none of it's child processes can change the terminal settings.

Anyway, I guess I could just make gpsmap execute softer with gpsmap on it's command line, so that softer will start gpsmap and the processes will be lined up in the right order.  It's odd how obvious solutions like this completly slip my mind until I've come to realize that the way I wanted to do it is impossible.  The thought didn't cross my mind until I was typing "when bash later goes to restore them..."

Thanks for helping out, I had pretty much given up until I read that you were writing code to try to help.

 - Pj

Re: Orphaned Processes and TCSETSW

[willy meier]

Philip Jacob Smith wrote:
> 
> This script causes the error.  You'll see stty complain about an I/O error.
> 

I forgot mentioning another -imho- quite odd behaviour which I didn't
find a single word of 'documentation' about, wrt the 'dup' and 'dup2'
syscalls, which I found a source of the difficulties which you
mentioned:

those syscalls do not duplicate the rsp. channels' descriptions but,
return just another index ('fd') to the rsp. single one description
array. thus, any modification to a 'dup'-ed channel will appear working
with its siblings, in the same manner.
apparently, only the settings which can be modified with the <fcntl>
syscall will be unique to the rsp. fd.

now, since stdin etc when passed as console channels (almost?) always
were set up with the 'dup' syscalls, there is no real distinction
between stdin/out/err and, the ioctl-ed modifications could easily
conflict with the rsp. channel's (copy) purpose.

my solution was reading the supposed to be modified channel's
description and its path+name (with <fstat> and, <readlink> - which
requires the /proc-fs), than closeing and newly opening the channel with
the required settings, with the <open> syscall, thus constituting a
truely independent console channel. 
<close>ing won't affect the parents channels, and the new one can't...


best,
	hp

-- 
Linux,Assembly,Forth: http://www.lxhp.in-berlin.de/index-lx.shtml
  >> hp -at- lxhp -dot- in-berlin -dot- de <<