[Buildroot] RaspberryPi U-Boot watchdog

[Buildroot] RaspberryPi U-Boot watchdog

[José Mendes]

[-- Attachment #1.1: Type: text/plain, Size: 1713 bytes --]

Hi all,

I'm trying to build a system with redundant rootfs partitions for a
Raspberry Pi 4 with U-Boot.

I'm trying to test the recovery of this system but I can't seem to get the
raspberry pi to reset after a kernel panic on a non-working rootfs.

To provoke this kernel panic I am simply deleting all the files in the
particular rootfs. The system seems to just hang there after the panic:

.....
[    3.374273] Kernel panic - not syncing: No working init found.  Try
passing init= option to kernel. See Linux
Documentation/admin-guide/init.rst for guidance.
[    3.388444] CPU: 3 PID: 1 Comm: swapper/0 Not tainted 6.1.61-v8 #2
[    3.394627] Hardware name: Raspberry Pi 4 Model B Rev 1.5 (DT)
[    3.400456] Call trace:
[    3.402897]  dump_backtrace.part.0+0xe8/0x100
[    3.407264]  show_stack+0x20/0x30
[    3.410579]  dump_stack_lvl+0x88/0xb4
[    3.414246]  dump_stack+0x18/0x34
[    3.417561]  panic+0x1a0/0x370
[    3.420618]  kernel_init+0x138/0x140
[    3.424197]  ret_from_fork+0x10/0x20
[    3.427776] SMP: stopping secondary CPUs
[    3.431701] Kernel Offset: 0x1353c00000 from 0xffffffc008000000
[    3.437617] PHYS_OFFSET: 0x0
[    3.440492] CPU features: 0x80000,2013c080,0000421b
[    3.445368] Memory Limit: none
[    3.448423] ---[ end Kernel panic - not syncing: No working init found.
Try passing init= option to kernel. See Linux
Documentation/admin-guide/init.rst for guidance. ]---

What I've tried:
- From `Target packages`: watchdog, watchdogd
- Busybox watchdog
- From linux-menuconfig > device drivers > watchdog
- From uboot-menuconfig > device drivers > watchdog

None of these seem to be working for me.

Best regards,

José Mendes

[-- Attachment #1.2: Type: text/html, Size: 1877 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] RaspberryPi U-Boot watchdog

[Waldemar Brodkorb]

Hi,

José Mendes wrote,

> Hi all,
> 
> I'm trying to build a system with redundant rootfs partitions for a Raspberry
> Pi 4 with U-Boot.
> 
> I'm trying to test the recovery of this system but I can't seem to get the
> raspberry pi to reset after a kernel panic on a non-working rootfs.
> 
> To provoke this kernel panic I am simply deleting all the files in the
> particular rootfs. The system seems to just hang there after the panic:
> 
> .....
> [    3.374273] Kernel panic - not syncing: No working init found.  Try passing
> init= option to kernel. See Linux Documentation/admin-guide/init.rst for
> guidance.
> [    3.388444] CPU: 3 PID: 1 Comm: swapper/0 Not tainted 6.1.61-v8 #2
> [    3.394627] Hardware name: Raspberry Pi 4 Model B Rev 1.5 (DT)
> [    3.400456] Call trace:
> [    3.402897]  dump_backtrace.part.0+0xe8/0x100
> [    3.407264]  show_stack+0x20/0x30
> [    3.410579]  dump_stack_lvl+0x88/0xb4
> [    3.414246]  dump_stack+0x18/0x34
> [    3.417561]  panic+0x1a0/0x370
> [    3.420618]  kernel_init+0x138/0x140
> [    3.424197]  ret_from_fork+0x10/0x20
> [    3.427776] SMP: stopping secondary CPUs
> [    3.431701] Kernel Offset: 0x1353c00000 from 0xffffffc008000000
> [    3.437617] PHYS_OFFSET: 0x0
> [    3.440492] CPU features: 0x80000,2013c080,0000421b
> [    3.445368] Memory Limit: none
> [    3.448423] ---[ end Kernel panic - not syncing: No working init found.  Try
> passing init= option to kernel. See Linux Documentation/admin-guide/init.rst
> for guidance. ]---
> 
> What I've tried:
> - From `Target packages`: watchdog, watchdogd
> - Busybox watchdog
> - From linux-menuconfig > device drivers > watchdog
> - From uboot-menuconfig > device drivers > watchdog
> 
> None of these seem to be working for me.

Have you tried to add panic=5 to your kernel commandline? I am not
sure what the default for panic is.
Looks like it is 0, so waiting forever.

best regards
 Waldemar
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] RaspberryPi U-Boot watchdog

[Fiona Klute via buildroot]

Hi José!

Am 24.07.24 um 12:16 schrieb José Mendes:
> Hi all,
>
> I'm trying to build a system with redundant rootfs partitions for a
> Raspberry Pi 4 with U-Boot.
>
> I'm trying to test the recovery of this system but I can't seem to get the
> raspberry pi to reset after a kernel panic on a non-working rootfs.
>
> To provoke this kernel panic I am simply deleting all the files in the
> particular rootfs. The system seems to just hang there after the panic:
>
> .....
> [    3.374273] Kernel panic - not syncing: No working init found.  Try
> passing init= option to kernel. See Linux
> Documentation/admin-guide/init.rst for guidance.
> [    3.388444] CPU: 3 PID: 1 Comm: swapper/0 Not tainted 6.1.61-v8 #2
> [    3.394627] Hardware name: Raspberry Pi 4 Model B Rev 1.5 (DT)
> [    3.400456] Call trace:
> [    3.402897]  dump_backtrace.part.0+0xe8/0x100
> [    3.407264]  show_stack+0x20/0x30
> [    3.410579]  dump_stack_lvl+0x88/0xb4
> [    3.414246]  dump_stack+0x18/0x34
> [    3.417561]  panic+0x1a0/0x370
> [    3.420618]  kernel_init+0x138/0x140
> [    3.424197]  ret_from_fork+0x10/0x20
> [    3.427776] SMP: stopping secondary CPUs
> [    3.431701] Kernel Offset: 0x1353c00000 from 0xffffffc008000000
> [    3.437617] PHYS_OFFSET: 0x0
> [    3.440492] CPU features: 0x80000,2013c080,0000421b
> [    3.445368] Memory Limit: none
> [    3.448423] ---[ end Kernel panic - not syncing: No working init found.
> Try passing init= option to kernel. See Linux
> Documentation/admin-guide/init.rst for guidance. ]---
>
> What I've tried:
> - From `Target packages`: watchdog, watchdogd
> - Busybox watchdog
> - From linux-menuconfig > device drivers > watchdog
> - From uboot-menuconfig > device drivers > watchdog
>
> None of these seem to be working for me.

The config options in menuconfig (both for U-Boot and Linux) only enable
the drivers to support the watchdog. The kernel can handle watchdogs
internally (see CONFIG_WATCHDOG_HANDLE_BOOT_ENABLED and
CONFIG_WATCHDOG_OPEN_TIMEOUT in the kernel config), but generally you
want that only as a transitory thing during boot before a watchdog
service (whether watchdogd or Busybox watchdog) takes over.

If you want the watchdog to reset no matter how broken the kernel or
userspace may be, enable the watchdog in U-Boot *before* starting the
kernel using the "wdt" command:
https://docs.u-boot.org/en/latest/usage/cmd/wdt.html

Excerpt from my boot.scr that configures the watchdog of an RPi CM4 (so,
the device may be different for you, and you need the matching driver
enabled in the U-Boot config) to expect a pet every 15 seconds:

wdt dev watchdog@7e100000
wdt start 15000

Of course the system then needs to boot up far enough to handle the
watchdog before that time is over.

Best regards,
Fiona

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] RaspberryPi U-Boot watchdog

[José Mendes]

[-- Attachment #1.1: Type: text/plain, Size: 3239 bytes --]

Hi Fiona,

Your suggestion worked really well.

It helped me figure out I needed to enable the correct watchdog timing
drivers on uboot-buildroot.

Best regards,

José Mendes

On Wed, 24 Jul 2024 at 12:58, Fiona Klute <fiona.klute@gmx.de> wrote:

> Hi José!
>
> Am 24.07.24 um 12:16 schrieb José Mendes:
> > Hi all,
> >
> > I'm trying to build a system with redundant rootfs partitions for a
> > Raspberry Pi 4 with U-Boot.
> >
> > I'm trying to test the recovery of this system but I can't seem to get
> the
> > raspberry pi to reset after a kernel panic on a non-working rootfs.
> >
> > To provoke this kernel panic I am simply deleting all the files in the
> > particular rootfs. The system seems to just hang there after the panic:
> >
> > .....
> > [    3.374273] Kernel panic - not syncing: No working init found.  Try
> > passing init= option to kernel. See Linux
> > Documentation/admin-guide/init.rst for guidance.
> > [    3.388444] CPU: 3 PID: 1 Comm: swapper/0 Not tainted 6.1.61-v8 #2
> > [    3.394627] Hardware name: Raspberry Pi 4 Model B Rev 1.5 (DT)
> > [    3.400456] Call trace:
> > [    3.402897]  dump_backtrace.part.0+0xe8/0x100
> > [    3.407264]  show_stack+0x20/0x30
> > [    3.410579]  dump_stack_lvl+0x88/0xb4
> > [    3.414246]  dump_stack+0x18/0x34
> > [    3.417561]  panic+0x1a0/0x370
> > [    3.420618]  kernel_init+0x138/0x140
> > [    3.424197]  ret_from_fork+0x10/0x20
> > [    3.427776] SMP: stopping secondary CPUs
> > [    3.431701] Kernel Offset: 0x1353c00000 from 0xffffffc008000000
> > [    3.437617] PHYS_OFFSET: 0x0
> > [    3.440492] CPU features: 0x80000,2013c080,0000421b
> > [    3.445368] Memory Limit: none
> > [    3.448423] ---[ end Kernel panic - not syncing: No working init
> found.
> > Try passing init= option to kernel. See Linux
> > Documentation/admin-guide/init.rst for guidance. ]---
> >
> > What I've tried:
> > - From `Target packages`: watchdog, watchdogd
> > - Busybox watchdog
> > - From linux-menuconfig > device drivers > watchdog
> > - From uboot-menuconfig > device drivers > watchdog
> >
> > None of these seem to be working for me.
>
> The config options in menuconfig (both for U-Boot and Linux) only enable
> the drivers to support the watchdog. The kernel can handle watchdogs
> internally (see CONFIG_WATCHDOG_HANDLE_BOOT_ENABLED and
> CONFIG_WATCHDOG_OPEN_TIMEOUT in the kernel config), but generally you
> want that only as a transitory thing during boot before a watchdog
> service (whether watchdogd or Busybox watchdog) takes over.
>
> If you want the watchdog to reset no matter how broken the kernel or
> userspace may be, enable the watchdog in U-Boot *before* starting the
> kernel using the "wdt" command:
> https://docs.u-boot.org/en/latest/usage/cmd/wdt.html
>
> Excerpt from my boot.scr that configures the watchdog of an RPi CM4 (so,
> the device may be different for you, and you need the matching driver
> enabled in the U-Boot config) to expect a pet every 15 seconds:
>
> wdt dev watchdog@7e100000
> wdt start 15000
>
> Of course the system then needs to boot up far enough to handle the
> watchdog before that time is over.
>
> Best regards,
> Fiona
>
>

[-- Attachment #1.2: Type: text/html, Size: 3973 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot