[Bug 207129] New: PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[Bug 207129] New: PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=207129

            Bug ID: 207129
           Summary: PowerMac G4 DP (5.6.2 debug kernel + inline KASAN)
                    freezes shortly after booting with "do_IRQ: stack
                    overflow: 1760"
           Product: Platform Specific/Hardware
           Version: 2.5
    Kernel Version: 5.6.2
          Hardware: PPC-32
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: PPC-32
          Assignee: platform_ppc-32@kernel-bugs.osdl.org
          Reporter: erhard_f@mailbox.org
                CC: christophe.leroy@c-s.fr
        Regression: No

Created attachment 288221
  --> https://bugzilla.kernel.org/attachment.cgi?id=288221&action=edit
kernel .config (5.6.2, INLINE KASAN, PowerMac G4 DP)

Was trying to do some testing with the PowerMac G4 DP again, running a 5.6.2
debug kernel w. KASAN INLINE. The G4 boots fine, but crashes shortly afterwards
when using it, leaving no stack trace, but only this message on the screen:

do_IRQ: stack overflow: 1760
CPU: 0 PID: 209 Comm: rsync Tained: G        W        5.6.2-PowerMacG4+ #3
Call Trace:


120 seconds panic timer does not kick in. I have to manually switch off/switch
on the G4.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=207129

--- Comment #1 from Christophe Leroy (christophe.leroy@c-s.fr) ---
So it hands in show_stack().

Does it also hang without CONFIG_DEBUG_STACKOVERFLOW ? If not, it means we have
a problem with check_stack_overflow()

Regardless of the result above, can you try increasing CONFIG_THREAD_SHIFT ?

Can you maybe also do a test without CONFIG_VMAP_STACK ?

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=207129

--- Comment #2 from Erhard F. (erhard_f@mailbox.org) ---
Created attachment 288229
  --> https://bugzilla.kernel.org/attachment.cgi?id=288229&action=edit
screenshot01.jpg

Without CONFIG_DEBUG_STACKOVERFLOW things are better. The rsync completes, the
G4 was building stuff for 2 hours or so until I got these errors and a hard
freeze:

[...]
Oops: kernel stack overflow, sig: 11 [#1]
BE PAGE_SIZE=4K MMU=Hash SMP NR_CPUS=2 PowerMac
Modules linked in: ...
CPU: 1 PID: 17105 Comm: kworker/u4:5 Tainted: G        W       
5.6.2-PowerMacG4+ #5
------------[ cut here  ]------------
kernel BUG at mm/usercopy.c:99!
Oops: Exception in kernel mode, sig: 5 [#2]
BE PAGE_SIZE=4K MMU=Hash SMP NR_CPUS=2 PowerMac
Modules linked in: ...
CPU: 1 PID: 17185 Comm: kworker/u4:5 Tainted: G        W       
5.6.2-PowerMacG4+ #5
usercopy: Kernel memory overwrite attempt detected to kernel text (offset 6336,
size 4)!
------------[ cut here  ]------------
kernel BUG at mm/usercopy.c:99!
Oops: Exception in kernel mode, sig: 5 [#3]
BE PAGE_SIZE=4K MMU=Hash SMP NR_CPUS=2 PowerMac
Modules linked in: ...
CPU: 1 PID: 17185 Comm: kworker/u4:5 Tainted: G        W       
5.6.2-PowerMacG4+ #5
usercopy: Kernel memory overwrite attempt detected to kernel text (offset 5336,
size 4)!
------------[ cut here  ]------------
kernel BUG at mm/usercopy.c:99!
Oops: Exception in kernel mode, sig: 5 [#4]
BE PAGE_SIZE=4K MMU=Hash SMP NR_CPUS=2 PowerMac
Modules linked in: ...
CPU: 1 PID: 17185 Comm: kworker/u4:5 Tainted: G        W       
5.6.2-PowerMacG4+ #5
usercopy: Kernel memory overwrite attempt detected to kernel text (offset 4336,
size 4)!
------------[ cut here  ]------------
kernel BUG at mm/usercopy.c:99!
Oops: Exception in kernel mode, sig: 5 [#5]
BE PAGE_SIZE=4K MMU=Hash SMP NR_CPUS=2 PowerMac
Modules linked in: ...
Unrecoverable FP Unavailable Exception 801 at 9b8
CPU: 1 PID: 17185 Comm: kworker/u4:5 Tainted: G        W       
5.6.2-PowerMacG4+ #5
usercopy: Kernel memory overwrite attempt detected to kernel text (offset 3336,
size 4)!
------------[ cut here  ]------------

Now running with CONFIG_THREAD_SHIFT=14 which runs fine so far... Did not try
without CONFIG_VMAP_STACK yet.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=207129

--- Comment #3 from Erhard F. (erhard_f@mailbox.org) ---
Created attachment 288231
  --> https://bugzilla.kernel.org/attachment.cgi?id=288231&action=edit
screenshot02.jpg

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=207129

--- Comment #4 from Erhard F. (erhard_f@mailbox.org) ---
Without CONFIG_VMAP_STACK I had one crash after 2-3 hours of building but the
panic timer kicked in and rebooted the machine. Now it has been building
packages for hours again without any anomalies.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=207129

--- Comment #5 from Christophe Leroy (christophe.leroy@c-s.fr) ---
Ok, so as a summary:
- With CONFIG_THREAD_SHIFT = 13 and CONFIG_DEBUG_STACKOVERFLOW, the system gets
stuck
- With CONFIG_THREAD_SHIFT = 13 and without CONFIG_DEBUG_STACKOVERFLOW, stack
overflow is not really detected until it gets into kernel text !!!
- With CONFIG_THREAD_SHIFT = 14 it runs fine
- With CONFIG_VMAP_STACK, the automatic restart doesn't work
- Without CONFIG_VMAP_STACK, the automatic restart works

So I'll send a patch to set CONFIG_THREAD_SHIFT to 14 when CONFIG_KASAN is
selected. x86 and arm64 already do that.

And I'll try to investigate the other points when I have time.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=207129

--- Comment #6 from Erhard F. (erhard_f@mailbox.org) ---
Yes, precisely summarized! Thanks for your efforts!

CONFIG_KASAN though only is x86_64 not x86 AFAIK.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=207129

Michael Ellerman (michael@ellerman.id.au) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |michael@ellerman.id.au
         Resolution|---                         |CODE_FIX

--- Comment #7 from Michael Ellerman (michael@ellerman.id.au) ---
I think this was resolved by increasing the stack size for KASAN builds.

ie.
edbadaf06710 ("powerpc/kasan: Fix stack overflow by increasing THREAD_SHIFT")
and later
3e8635fb2e07 ("powerpc/kasan: Force thread size increase with KASAN")

If not feel free to reopen.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=207129

Michael Ellerman (michael@ellerman.id.au) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=207129

Erhard F. (erhard_f@mailbox.org) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|CLOSED                      |REOPENED
         Resolution|CODE_FIX                    |---

--- Comment #8 from Erhard F. (erhard_f@mailbox.org) ---
Currently (as of v6.8-rc6) and since quite a while my G4 does not boot at all
with CONFIG_KASAN_INLINE=y.

When I try booting an KASAN_INLINE enabled kernel it fails with an invalid
memory access and I get dropped out to the OpenFirmaware console.

A small difference is when I boot the 56M vmlinux-6.8.0-rc6-PMacG4:

Please wait, loading kernel...
   Elf32 kernel loaded...

Invalid memory access at   %SRR0: 00000013   %SRR1: 00001300

Apple PowerMac3,6 4.6.0f1 BootROM built on 02/20/03 at 13:52:27
[...]


vs. booting the 16M arch/powerpc/boot/zImage:


Please wait, loading kernel...
   Elf32 kernel loaded...

zImage starting: loaded at 0x00400000 (sp: 0x012eefb0)
OF version = 'OpenFirmware 3'
Allocating 0x2c337e0 bytes for kernel...
Trying to claim from 0x400000 to 0x12ef5d8 (0xeef5d8) got ffffffff
Decompressing (0x01414000 <- 0x00410000:0x12ea9c9)...
Done! Decompressed 0x2bcc80c bytes

Linux/PowerPC load: ro root=/dev/sda5 slub_debug=FZP page_poison=1
netconsole=6666@192.168.2.8/eth0,6666@192.168.2.3/A8:A1:59:16:4F:EA debug

Finalizing device tree... using OF tree (promptr=ff847240)

Invalid memory access at   %SRR0: 40000000   %SRR1: 00000000

Apple PowerMac3,6 4.6.0f1 BootROM built on 02/20/03 at 13:52:27
[...]


Same kernel with CONFIG_KASAN_OUTLINE=y instead of KASAN_INLINE boots and runs
ok.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=207129

--- Comment #9 from Erhard F. (erhard_f@mailbox.org) ---
Created attachment 305910
  --> https://bugzilla.kernel.org/attachment.cgi?id=305910&action=edit
kernel .config (6.8-rc6, KASAN_INLINE=y, PowerMac G4 DP)

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=207129

--- Comment #10 from Christophe Leroy (christophe.leroy@csgroup.eu) ---
I built a kernel with your .config, the problem is a size problem.

PPC32 kernels are not designed to be that big.

Extract from generated System.map:
  c2394000 D _etext
  c2800000 T _sinittext
  c2bf5000 B _end

You need to keep the size of the kernel below 32Mbytes, or a deep work is
required to enable the kernel to perform far jumps before the kernel is
relocated.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=207129

Erhard F. (erhard_f@mailbox.org) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|---                         |CODE_FIX

--- Comment #11 from Erhard F. (erhard_f@mailbox.org) ---
You were correct! I forgot about that...

I shrunk the size by using -Os and disabling some debugging stuff and changing
some statically built-in stuff to 'M' without sacrificing debugging
capabilities too much until it fit < 32 MiB:

KASAN_OUTLINE vs.
 # size vmlinux-6.8.0-rc6-PMacG4 
   text    data     bss     dec     hex filename
12367737 6652440 426336 19446513 128baf1 vmlinux-6.8.0-rc6-PMacG4

KASAN_INLINE
 # size vmlinux-6.8.0-rc6-PMacG4 
   text    data     bss     dec     hex filename
24660169 6652440  426336 31738945 1e44c41 vmlinux-6.8.0-rc6-PMacG4


Apart from that I can confirm inline KASAN runs fine now and I really no longer
get this stack overflow when using it.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.