Re: stack smashing detected

[Michael Schmitz <schmitzmic@gmail.com>]

Hi Stan,

Am 03.02.2023 um 12:16 schrieb Stan Johnson:
> On 2/1/23 11:51 AM, Michael Schmitz wrote:
>> ...
>>
>> The stack canary mechanism pushes a token on the stack at function
>> entry, and compares against that token's value at function exit. This is
>> all code generated by gcc in the user binary.
>>
>> The kernel is not involved in function calls other than syscalls. For
>> syscalls, we could try to find the user mode stack, and do a similar
>> canary trick, but I don't think that would be necessary for all
>> syscalls. Might be easier to instrument copy_to_user() instead if you're
>> worried about a syscall receiving result data that way to a variable on
>> the stack.
>>
>> But since we're touching on copy_to_user() here - the 'remove set_fs'
>> patch set by Christoph Hellwig refactored the m68k inline helpers around
>> July 2021. Can you test a kernel prior to those patches (5.15-rc2)?
>> ...
> I updated my m68k rootfs in QEMU to the latest Debian SID and installed
> the updated rootfs on my Mac IIci. And it's now restoring to my SE/30,
> so I'll be able to test on that system again eventually.
>
> I tested 5.15.0-rc3, 5.15.0-rc2 and 5.15.0-rc1, with inconclusive
> results on the IIci (3 tests per kernel; e.g. "0, 1, 5" in
> "Stack-Smashing" indicates that test 1 had zero smashing errors, test 2
> had 1, and test 3 had 5):
>
> Kernel          Stack-Smashing
> 5.15.0-rc3	    0, 1, 5
> 5.15.0-rc2          2, 3, 2
> 5.15.0-rc1          7, 1, 3

I think we can rule out those changes to usermode copy routines now.

Good idea to test on some other hardware. I'd also consider Geert's 
suggestion to enable kernel level debugging for the kernel's memory 
allocators. That does look a lot easier than debugging usermode copy.

Cheers,

	Michael


> I saved the serial console logs, but they're probably not too useful at
> this point. Next, I'll confirm a similar failure on the SE/30, then I'll
> check 4.0 and 5.0 (on the IIci) to see whether the issue started in that
> range, then use git bisect in an attempt to isolate the issue further.
> I'll also do a spot check on a different IIci to lessen the chance that
> the issue is being caused by a hardware problem. If it's not caused by a
> kernel bug, the next step will be to isolate the specific offending
> executable(s) in the sysvinit scripts.
>
> Or I could test using systemd instead of sysvinit; that would take
> longer but it might be worthwhile if it doesn't look like a kernel bug.
>
> -Stan
>