/dev/watchdog for onchip MPC860 watchdog?

/dev/watchdog for onchip MPC860 watchdog?

[Graham Stoney]

Dear Embeddees,

I'm wondering if anyone is working on a driver to provide the /dev/watchdog
interface to the MPC860's on-chip watchdog?

Thanks,
Graham

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: /dev/watchdog for onchip MPC860 watchdog?

[Dan Malek]

Graham Stoney wrote:
> 
> Dear Embeddees,
> 
> I'm wondering if anyone is working on a driver to provide the /dev/watchdog
> interface to the MPC860's on-chip watchdog?

The difficulty with the on-chip watchdog isn't the driver but
the coordination with the boot roms.  The "auto-reset" features
of the 8xx (watch dog, bus monitor, etc.) are part of a write once
configuration register.  Once disabled, they can't be enabled
again until the processor is reset.

If the boot rom enables these features (especially the watch dog),
you must service this as part of the Linux kernel processing.  You
can't wait around for some application at a later time to determine
it now wants to enable and process the watchdog.


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: /dev/watchdog for onchip MPC860 watchdog?

[Graham Stoney]

Dan Malek writes:
> Graham Stoney wrote:
> > 
> > Dear Embeddees,
> > 
> > I'm wondering if anyone is working on a driver to provide the /dev/watchdog
> > interface to the MPC860's on-chip watchdog?
> 
> The difficulty with the on-chip watchdog isn't the driver but
> the coordination with boot roms.  The "auto-reset" features
> of the 8xx (watch dog, bus monitor, etc.) are part of a write once
> configuration register.  Once disabled, they can't be enabled
> again until the processor is reset.
> 
> If the boot rom enables these features (especially the watch dog),
> you must service this as part of the Linux kernel processing.  You
> can't wait around for some application at a later time to determine
> it now wants to enable and process the watchdog.

A couple of people mailed me off the list about my query, and as always, Dan's
right -- it's nowhere near as simple as I first imagined. The watchdog can't
be disabled/reenabled, and its maximum period is only a couple of seconds; 
there's no guarantee that the kernel will have booted and loaded the
application by then, so the watchdog is likely to go off during the boot
process. I think this probably puts my plans to use it on hold for now...

Thanks,
Graham

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: /dev/watchdog for onchip MPC860 watchdog?

[Dan Malek]

Graham Stoney wrote:


> .... The watchdog can't
> be disabled/reenabled, and its maximum period is only a couple of seconds;
> there's no guarantee that the kernel will have booted and loaded the
> application by then, so the watchdog is likely to go off during the boot
> process. I think this probably puts my plans to use it on hold for now...

Maybe not.  I was thinking about this (when I should have been doing
other things :-) earlier today.  What we could do when the internal
watchdog is configured is to service it during the Linux timer
interrupt when an application doesn't have it "enabled".  This
doesn't provide a watchdog service, just ensures it doesn't
unexpectedly expire.  When an application decides it wants the
watchdog, we don't service it in the timer ISR.

We would have to litter the boot code with some service calls,
like when you wait at the Linux boot prompt, and make sure
you can uncompress and enable timer interrupts before the next
expiration.  Beyond that, we wouldn't have to look for any
watchdog service points in the kernel.

I may play with it later tonight........


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: /dev/watchdog for onchip MPC860 watchdog?

[Jeff Millar]

I've also been thinking about this...

If the boot process goes bad, then the interrupt driven watchdog will
keep getting reset and prevent the watchdog from firing...leading to a
unrecoverable hang.  So, what's the likely hood of the boot process
going bad?

Our application requires downloading new kernels and applications and this
may
increase the likely hood that a kernel fails to complete the boot process.

This idea has a window of vunerability but how much and how likely???

Maybe the timer can be set longer than the longest bootup time or at least
until the first daemon gets going?

jeff

----- Original Message -----
From: Dan Malek <dan@netx4.com>
To: Graham Stoney <greyham@research.canon.com.au>
Cc: Dan Malek <dan@penguin.netx4.com>;
<linuxppc-embedded@lists.linuxppc.org>
Sent: Tuesday, October 19, 1999 3:51 PM
Subject: Re: /dev/watchdog for onchip MPC860 watchdog?


>
> Graham Stoney wrote:
>
>
> > .... The watchdog can't
> > be disabled/reenabled, and its maximum period is only a couple of
seconds;
> > there's no guarantee that the kernel will have booted and loaded the
> > application by then, so the watchdog is likely to go off during the boot
> > process. I think this probably puts my plans to use it on hold for
now...
>
> Maybe not.  I was thinking about this (when I should have been doing
> other things :-) earlier today.  What we could do when the internal
> watchdog is configured is to service it during the Linux timer
> interrupt when an application doesn't have it "enabled".  This
> doesn't provide a watchdog service, just ensures it doesn't
> unexpectedly expire.  When an application decides it wants the
> watchdog, we don't service it in the timer ISR.
>
> We would have to litter the boot code with some service calls,
> like when you wait at the Linux boot prompt, and make sure
> you can uncompress and enable timer interrupts before the next
> expiration.  Beyond that, we wouldn't have to look for any
> watchdog service points in the kernel.
>
> I may play with it later tonight........
>
>
> -- Dan
>
>


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: /dev/watchdog for onchip MPC860 watchdog?

[Raphael Massin]

> 
> 
> I've also been thinking about this...
> 
> If the boot process goes bad, then the interrupt driven watchdog will
> keep getting reset and prevent the watchdog from firing...leading to a
> unrecoverable hang.  So, what's the likely hood of the boot process
> going bad?
> 
> Our application requires downloading new kernels and applications and this
> may
> increase the likely hood that a kernel fails to complete the boot process.
> 
> This idea has a window of vunerability but how much and how likely???
> 
> Maybe the timer can be set longer than the longest bootup time or at least
> until the first daemon gets going?
> 
> jeff
> 

Hello,

I'm using the watchdog timer on a custom board since several months. To
prevent a CPU reset during the kernel is uncompressing, i prorammed it
to generate a Non Maskable Interrupt instead of system reset. Those NMIs
are handled by a special function in the Linux kernel. This function
can reset the board, using a specific I/O, when too many NMIs occurred
without any watchdog timer (SWSR) servicing.

Today, i refresh the timer in my real time application that is launched
immediately after the kernel init. This scheme must work for any board
launching an appli/real time module after kernel init. For other configurations,
Dan's idea seems perfect.

	Raphael

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: /dev/watchdog for onchip MPC860 watchdog?

[Dan Malek]

Jeff Millar wrote:

> If the boot process goes bad, then the interrupt driven watchdog will
> keep getting reset and prevent the watchdog from firing...


Yes....The alternative is finding all of the places in the kernel
where you would have to service the watchdog during the boot process.
Probably not many.....Depends upon the timeout value.


> .......  So, what's the likely hood of the boot process
> going bad?

My experience has been that unless the hardware is broken, it
tends to either work every time or fail every time.

> Our application requires downloading new kernels and applications and this
> may
> increase the likely hood that a kernel fails to complete the boot process.

Yes, it will.  But....what do you do after the new software
load fails to boot?  It will just keep resetting and running the
same broken software......


> Maybe the timer can be set longer than the longest bootup time or at least
> until the first daemon gets going?

All of the timeout and other controls are in the write-once SYPCR.
Once you select the timeout and behavior, you have it until the
next reset.



	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: /dev/watchdog for onchip MPC860 watchdog?

[Dan Malek]

Raphael Massin wrote:

> ... i prorammed it
> to generate a Non Maskable Interrupt instead of system reset. Those NMIs
> are handled by a special function in the Linux kernel. This function
> can reset the board, using a specific I/O, when too many NMIs occurred
> without any watchdog timer (SWSR) servicing.


That's not exactly the same thing.  You still rely upon some,
although smaller, piece of software and proper operation of your
counter (a memory location that could be corrupted by errant
software) to perform the watchdog service.


> ..... configurations,
> Dan's idea seems perfect.

A far from perfect alternative.......


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: /dev/watchdog for onchip MPC860 watchdog?

[Graham Stoney]

> From: Dan Malek <dan@netx4.com>
> To: Graham Stoney <greyham@research.canon.com.au>
> Cc: Dan Malek <dan@penguin.netx4.com>;
> <linuxppc-embedded@lists.linuxppc.org>
> Sent: Tuesday, October 19, 1999 3:51 PM
> Subject: Re: /dev/watchdog for onchip MPC860 watchdog?
> 
> > Graham Stoney wrote:
> > > .... The watchdog can't
> > > be disabled/reenabled, and its maximum period is only a couple of
> seconds;
> > > there's no guarantee that the kernel will have booted and loaded the
> > > application by then, so the watchdog is likely to go off during the boot
> > > process. I think this probably puts my plans to use it on hold for
> now...
> >
> > Maybe not.  I was thinking about this (when I should have been doing
> > other things :-) earlier today.  What we could do when the internal
> > watchdog is configured is to service it during the Linux timer
> > interrupt when an application doesn't have it "enabled".  This
> > doesn't provide a watchdog service, just ensures it doesn't
> > unexpectedly expire.  When an application decides it wants the
> > watchdog, we don't service it in the timer ISR.
> >
> > We would have to litter the boot code with some service calls,
> > like when you wait at the Linux boot prompt, and make sure
> > you can uncompress and enable timer interrupts before the next
> > expiration.  Beyond that, we wouldn't have to look for any
> > watchdog service points in the kernel.

Our email & file server died big time yesterday, so I only saw this this
morning, and co-incidentally had exactly the same idea. To the user app,
/dev/watchdog still looks much like the interface in watchdog.txt.

Sounds pretty good to me!

Thanks,
Graham

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: /dev/watchdog for onchip MPC860 watchdog?

[Cort Dougan]

I've added an arch-specific heartbeat as of 2.3 to the timer handler that
could easily be setup to handle this.

I don't want the boot code 'littered' with calls to this, though.  It'd be
nice if the timer handler could take care of this until the user app could
take it over.  If the time between time_init() and the sti() is too great
in start_kernel() that may not be possible.

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: /dev/watchdog for onchip MPC860 watchdog?

[Tania Boak]

Hi Raphael

I'm interested in the application you mentioned in your message:

http://lists.linuxppc.org/listarcs/linuxppc-embedded/199910/msg00028.html



> I'm using the watchdog timer on a custom board since several months.
> To
> prevent a CPU reset during the kernel is uncompressing, i prorammed it
> to generate a Non Maskable Interrupt instead of system reset. Those
> NMIs
> are handled by a special function in the Linux kernel. This function
> can reset the board, using a specific I/O, when too many NMIs occurred
> without any watchdog timer (SWSR) servicing.
>
> Today, i refresh the timer in my real time application that is
> launched
> immediately after the kernel init. This scheme must work for any board
> launching an appli/real time module after kernel init. For other
> configurations,
> Dan's idea seems perfect.

I'm trying to find a way to use the on chip watchdog to provide an
automatic reboot should the software on our custom board go to never
never land.  At the moment I don't need to provide the /dev/watchdog
interface, but I may in the future.  Dan's idea about not servicing the
SWSR while /dev/watchdog is open sounds good for that.  But for now, I
just need a way to keep the watchdog timer from expiring as long as
things are working "normally".  I like the idea of using an application
to write to the SWSR rather than doing it from the timer interrupt in
the kernel because I can envision cases where interrupts but not much
else may be working.

But I don't know how to guarantee that the application will be able to
service the watchdog in any given interval of time.  How do you
distinguish between the case where the system has stopped working and
the case where the system is busy but still functioning?

Any help you can give would be appreciated.  If anyone else has an
opinion as to the best place to service the watchdog in order to provide
protection from a hung system, I'd love to hear it.

Thanks


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: /dev/watchdog for onchip MPC860 watchdog?

[Graham Stoney]

Tania Boak writes:
> But I don't know how to guarantee that the application will be able to
> service the watchdog in any given interval of time.  How do you
> distinguish between the case where the system has stopped working and
> the case where the system is busy but still functioning?

Your system needs to have some periodic task which you can guarantee will
run sufficiently often during normal operation, and hopefully will clag if
something catastrophic happens.  If you're doing simple regular data
aquisition, that's reasonably easy, but if it's a more complex system, there
may not be such a periodic task.  In this case you can run a real-time thread
at high priority which sleeps most of the time, just to tickle the watchdog;
but it still won't guarantee that the lower priority threads are running OK.
Still, some protection may be better than none, depending on your application.

Regards,
Graham
--
Graham Stoney
Principal Hardware/Software Engineer
Canon Information Systems Research Australia
Ph: +61 2 9805 2909  Fax: +61 2 9805 2929

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/