Remotely rebooting a machine with state 'D' processes, how?

Remotely rebooting a machine with state 'D' processes, how?

[Axel Thimm]

How can I reboot a stuck machine remotely, when there are uninterruptable
processes arround? shutdown -r, reboot [-n] [-f], telinit 6 do not give the
intended results. Localy I can use Alt-SysRq-S/U/B, but what if I still have a
remote ssh connection and don't want to have to get to the machines location?

Of course the real problem are the processes themselves, but being able to
revive a machine is also nice ;)

Regards, Axel.
-- 
Axel.Thimm@physik.fu-berlin.de

Re: Remotely rebooting a machine with state 'D' processes, how?

[Brian]

It has been awhile since I had this problem, but I recall
	shutdown -r -n now
doing the job.

	-- Brian

On Friday 10 August 2001 05:19 pm, Axel Thimm wrote:
> How can I reboot a stuck machine remotely, when there are
> uninterruptable processes arround? shutdown -r, reboot [-n] [-f],
> telinit 6 do not give the intended results. Localy I can use
> Alt-SysRq-S/U/B, but what if I still have a remote ssh connection and
> don't want to have to get to the machines location?
>
> Of course the real problem are the processes themselves, but being able
> to revive a machine is also nice ;)
>
> Regards, Axel.

Re: Remotely rebooting a machine with state 'D' processes, how?

[Linus Torvalds]

In article <20010810231906.A21435@bonzo.nirvana> you write:
>How can I reboot a stuck machine remotely, when there are uninterruptable
>processes arround? shutdown -r, reboot [-n] [-f], telinit 6 do not give the
>intended results. Localy I can use Alt-SysRq-S/U/B, but what if I still have a
>remote ssh connection and don't want to have to get to the machines location?
>
>Of course the real problem are the processes themselves, but being able to
>revive a machine is also nice ;)

You have to use the reboot() system call directly as root, with the
proper arguments to make it avoid doing even any sync. See

	man 2 reboot

for details.

		Linus

Re: Remotely rebooting a machine with state 'D' processes, how?

[Herbert Xu]

Linus Torvalds <torvalds@transmeta.com> wrote:
> In article <20010810231906.A21435@bonzo.nirvana> you write:

> You have to use the reboot() system call directly as root, with the
> proper arguments to make it avoid doing even any sync. See

>        man 2 reboot

How do you do this when the process in the D state is holding the BKL?
-- 
Debian GNU/Linux 2.2 is out! ( http://www.debian.org/ )
Email:  Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Remotely rebooting a machine with state 'D' processes, how?

[Andi Kleen]

In article <E15VL6x-0007Jm-00@gondolin.me.apana.org.au>,
herbert@gondor.apana.org.au (Herbert Xu) writes:
> Linus Torvalds <torvalds@transmeta.com> wrote:
>> In article <20010810231906.A21435@bonzo.nirvana> you write:

>> You have to use the reboot() system call directly as root, with the
>> proper arguments to make it avoid doing even any sync. See

>> man 2 reboot

> How do you do this when the process in the D state is holding the BKL?

A process in D state sleeps and the BKL is always automatically dropped
when a process sleeps.

Bigger problems are other semaphores like inode sems, especially
when they belong to important shared libraries or directories.


-Andi

Re: Remotely rebooting a machine with state 'D' processes, how?

[Herbert Xu]

On Sat, Aug 11, 2001 at 12:55:46AM +0200, Andi Kleen wrote:
> 
> A process in D state sleeps and the BKL is always automatically dropped
> when a process sleeps.

OK.  What about console_lock since reboot(2) insists on doing a printk?
-- 
Debian GNU/Linux 2.2 is out! ( http://www.debian.org/ )
Email:  Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Remotely rebooting a machine with state 'D' processes, how?

[Robert Love]

On 11 Aug 2001 08:50:35 +1000, Herbert Xu wrote:
> Linus Torvalds <torvalds@transmeta.com> wrote:
> > In article <20010810231906.A21435@bonzo.nirvana> you write:
> 
> > You have to use the reboot() system call directly as root, with the
> > proper arguments to make it avoid doing even any sync. See
> 
> >        man 2 reboot
> 
> How do you do this when the process in the D state is holding the BKL?

the kernel should not sleep when it is holding the BKL (or any
semaphore).

-- 
Robert M. Love
rml at ufl.edu
rml at tech9.net

Re: Remotely rebooting a machine with state 'D' processes, how?

[Linus Torvalds]

On Sat, 11 Aug 2001, Herbert Xu wrote:
>
> Linus Torvalds <torvalds@transmeta.com> wrote:
> > In article <20010810231906.A21435@bonzo.nirvana> you write:
>
> > You have to use the reboot() system call directly as root, with the
> > proper arguments to make it avoid doing even any sync. See
>
> >        man 2 reboot
>
> How do you do this when the process in the D state is holding the BKL?

If it's in D state, it will be sleeping, and will have released the BKL.

Besides, does the reboot system call actually get the BKL? I don't think
it should need it..

		Linus

Re: Remotely rebooting a machine with state 'D' processes, how?

[Herbert Xu]

On Fri, Aug 10, 2001 at 03:58:02PM -0700, Linus Torvalds wrote:
> 
> Besides, does the reboot system call actually get the BKL? I don't think
> it should need it..

Actually, the machine in question turned out to be UP :)

However, it does have RAID 1 and the notifier call chain stuff looks like
a killer to me since it leads to do_md_stop.

Perhaps we need a RESTART3 that restarts without notifying?
-- 
Debian GNU/Linux 2.2 is out! ( http://www.debian.org/ )
Email:  Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Remotely rebooting a machine with state 'D' processes, how?

[Mike Fedyk]

On Sat, Aug 11, 2001 at 09:50:51AM +1000, Herbert Xu wrote:
> On Fri, Aug 10, 2001 at 03:58:02PM -0700, Linus Torvalds wrote:
> > 
> > Besides, does the reboot system call actually get the BKL? I don't think
> > it should need it..
> 
> Actually, the machine in question turned out to be UP :)
> 
> However, it does have RAID 1 and the notifier call chain stuff looks like
> a killer to me since it leads to do_md_stop.
> 
> Perhaps we need a RESTART3 that restarts without notifying?

Interesting...

I have an oldworld ppc mac with RAID1 compiled on 2.2.19, and a bad floppy
made badblocks unkillable.

Is there another way to kill that will work when kill -9 won't?

Re: Remotely rebooting a machine with state 'D' processes, how?

[Troy Benjegerdes]

On Fri, Aug 10, 2001 at 05:04:07PM -0700, Mike Fedyk wrote:
> On Sat, Aug 11, 2001 at 09:50:51AM +1000, Herbert Xu wrote:
> > On Fri, Aug 10, 2001 at 03:58:02PM -0700, Linus Torvalds wrote:
> > > 
> > > Besides, does the reboot system call actually get the BKL? I don't think
> > > it should need it..
> > 
> > Actually, the machine in question turned out to be UP :)
> > 
> > However, it does have RAID 1 and the notifier call chain stuff looks like
> > a killer to me since it leads to do_md_stop.
> > 
> > Perhaps we need a RESTART3 that restarts without notifying?
> 
> Interesting...
> 
> I have an oldworld ppc mac with RAID1 compiled on 2.2.19, and a bad floppy
> made badblocks unkillable.

I think that was probably because the pmac floppy driver has some issues..


-- 
Troy Benjegerdes | master of mispeeling | 'da hozer' |  hozer@drgw.net
-----"If this message isn't misspelled, I didn't write it" -- Me -----
"Why do musicians compose symphonies and poets write poems? They do it
because life wouldn't have any meaning for them if they didn't. That's 
why I draw cartoons. It's my life." -- Charles Shulz

Re: Remotely rebooting a machine with state 'D' processes, how?

[Chris Abbey]

Tomorrow, Herbert Xu wrote:
> Perhaps we need a RESTART3 that restarts without notifying?

That would be the one line outb that Chris Wedgewood posted a few notes
back in the thread. If my memory of x86 bios is correct he simple poked
the control register that equates to pushing the reset button.

-- 
now the forces of openness have a powerful and
  unexpected new ally - http://ibm.com/linux

Re: Remotely rebooting a machine with state 'D' processes, how?

[Chris Wedgwood]

On Sat, Aug 11, 2001 at 08:50:35AM +1000, Herbert Xu wrote:

    How do you do this when the process in the D state is holding the BKL?

On ia32:

        outb(0xfe, 0x64);

I have a command-line proglet for doing this, and also a patch to add
to SysReq is people really want such a thing.



  --cw

Re: Remotely rebooting a machine with state 'D' processes, how?

[Axel Thimm]

On Fri, Aug 10, 2001 at 02:59:37PM -0700, Linus Torvalds wrote:
> In article <20010810231906.A21435@bonzo.nirvana> you write:
> > How can I reboot a stuck machine remotely, when there are uninterruptable
> > processes arround? shutdown -r, reboot [-n] [-f], telinit 6 do not give
> > the intended results. Localy I can use Alt-SysRq-S/U/B, but what if I
> > still have a remote ssh connection and don't want to have to get to the
> > machines location?
> >
> > Of course the real problem are the processes themselves, but being able to
> > revive a machine is also nice ;)
> You have to use the reboot() system call directly as root, with the proper
> arguments to make it avoid doing even any sync. See man 2 reboot for
> details.

Would there be a way to also simulate the effects of Alt-SysRq-S and
Alt-SysRq-U? A simple sync falls also into D-state, Alt-SysRq-S does not, as
far as I have had to use it.

Would an `emergency-reboot' binary that calls these three kernel calls be
possible and also good-to-have if it's not a too bad idea?

Is there a way to have this automated in some kind of a kernel software
watchdog? Given certain conditions (like processes in D-state for longer than
a specified time) the kernel might first try to call a userland-reboot and
this timing out the kernel might use the sync;umount;reboot procedure of
Alt-SysRq-S/U/B.

But even when I write this I realize the security hole potentials. If someone
could simulate the watchdog conditions for a reboot, he might be able to have
this machine reboot all the time.
-- 
Axel.Thimm@physik.fu-berlin.de

Re: Remotely rebooting a machine with state 'D' processes, how?

[Kai Henningsen]

torvalds@transmeta.com (Linus Torvalds)  wrote on 10.08.01 in <200108102159.f7ALxb908284@penguin.transmeta.com>:

> In article <20010810231906.A21435@bonzo.nirvana> you write:
> >How can I reboot a stuck machine remotely, when there are uninterruptable
> >processes arround? shutdown -r, reboot [-n] [-f], telinit 6 do not give the
> >intended results. Localy I can use Alt-SysRq-S/U/B, but what if I still
> >have a remote ssh connection and don't want to have to get to the machines
> >location?
> >Of course the real problem are the processes themselves, but being able to
> >revive a machine is also nice ;)
>
> You have to use the reboot() system call directly as root, with the
> proper arguments to make it avoid doing even any sync. See
>
> 	man 2 reboot
>
> for details.

I thought that was exactly what reboot -n -f does: don't sync, don't call  
shutdown, just reboot immediately. That's certainly what I have  
(successfully) used that for in the past.

MfG Kai

Re: Remotely rebooting a machine with state 'D' processes, how?

[Ricardo Galli]

> > >man 2 reboot
> > How do you do this when the process in the D state is holding the BKL?
> 
> OK. What about console_lock since reboot(2) insists on doing a printk?

reboot -n -f ?

--ricardo
http://m3d.uib.es/~gallir/