Re: reliable reproducer, was Re: core dump analysis

[Michael Schmitz <schmitzmic@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4804 bytes --]

Hi Finn,

Am 21.04.2023 um 21:18 schrieb Michael Schmitz:
> Hi Finn,
>
> Am 21.04.2023 um 20:30 schrieb Finn Thain:
>> On Fri, 21 Apr 2023, Michael Schmitz wrote:
>>
>>>
>>> How often did a page fault happen when executing moveml, in other
>>> programs?
>>>
>>
>> The printk() I placed in bus_error030() was conditional on the short word
>> at the instruction pointer. It didn't consider all forms of movem, just
>> 0x48e7 which is the start of "moveml X,%sp@-". This matched page
>> faults in
>> many of the programs that executed while booting to single user mode. I
>> suppose most of them don't use signal handlers in the same way dash does
>> otherwise they would probably be unreliable too.
>
> OK; so too much noise unless filtered on the command name...
>
> I'll try first to get register state at the time of signal delivery from
> the sa_sigaction handler's ucontext parameter to see where the signal
> stack falls in relation to the call frames from your rec() function on
> the stack (and what the register contents were). Hope that won't be too

Took a little while to figure out that the ucontext format changed in 
the decade or two since my userland's libc headers were generated. With 
the correct format, the information stored on th signal frame made a lot 
more sense.

Log of your test program (attached), instrumented to keep track of user 
stack pointer in the parent process, user stack pointer in the signal 
handler, and stack pointer, pc and exceptiopn frame format from the 
signal stack (only the last few signals shown):

parent usp  : 0xef97beb8
handler tos : 0xef97bdc4
handler usp : 0xef97bbe0
signal usp  : 0xef97bea8
signal pc   : 0xc009f37a
signal fmtv : 0x800006ca

parent usp  : 0xef969eb8
handler tos : 0xef969dc4
handler usp : 0xef969be0
signal usp  : 0xef969ea8
signal pc   : 0xc009f37a
signal fmtv : 0x800006ca

parent usp  : 0xef9530d0
handler tos : 0xef952fec
handler usp : 0xef952e08
signal usp  : 0xef9530d0
signal pc   : 0x800006dc
signal fmtv : 0x91929394

parent usp  : 0xef945eb8
handler tos : 0xef945dc4
handler usp : 0xef945be0
signal usp  : 0xef945ea8
signal pc   : 0xc009f37a
signal fmtv : 0x800006ca

parent usp  : 0xef933eb8
handler tos : 0xef933dc4
handler usp : 0xef933be0
signal usp  : 0xef933ea8
signal pc   : 0xc009f37a
signal fmtv : 0x800006ca

parent usp  : 0xef921edc
handler tos : 0xef997984
handler stack overwrote usp!
handler usp : 0xef9977a0
signal usp  : 0xef997a64
signal pc   : 0x80000768
signal fmtv : 0xa1a2a3a4

Illegal instruction (core dumped)

usp from the signal stack is below that of the parent process (before 
calling fork()).

usp from the signal handler is below both of those. So far, so good.

The top of the signal frame, however, is getting quite close to these 
stack pointers. In the last log, it has grown above the user stack pointer.

Two things to note:

- pc in the signal frame (from struct uc_mcontext) is either the return 
pc from the stack stuffing function, or something else I cannot work 
out. That part of ucontext appears valid.

- what ought to be the frame format and vector offset does in fact hold 
varying longwords from the user stack. This information is not from 
struct uc_mcontext, but from extra information copied after struct 
ucontext ends. That wouldn't be there if at time of signal delivery, 
nothing had yet written to the area where the signal frame is stored.

It appears that neither format/vector nor other exception frame 
information is stored there by the kernel signal code, so the registers 
that rec() had saved to the stack remain there. At time of signal 
delivery, the user stack appears to have grown into the area used for 
the signal stack, in spite of the user stack pointer saved after setting 
up the registers saying otherwise. I can't figure that out.

At any rate, the constant part of the extra signal stack information 
does not take into account that we may end up storing additional 
information beyond that, in the case that the exception frame that got 
us into kernel mode is larger than four words. Maybe that is why the 
last stack frames logged do have the end of the signal stack go beyond 
the user stack pointer.

The FPU state and exception frame part stored above the ucontext struct 
may well have a different size in your libc headers, or I may have made 
a mistake calculating the size of the extra information stored on the 
stack. You may want to undo my hack to use a local copy of the struct 
declarations and compile with the correct headers, maybe that does 
change the overall picture.

> noisy ... Then see how that changes with bus error handling forcing
> signal delivery.

Haven't got to that bit yet ...

Cheers,

	Michael


> Shame we don't have similar code to find the MMU descriptors on 040.
>
> Cheers,
>
>     Michael
>
>

[-- Attachment #2: movemlrtas.c --]
[-- Type: text/x-csrc, Size: 4554 bytes --]

#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <signal.h>
#include <string.h>
#include <ucontext.h>

/* copies of ucontext struct matching 6.x kernels */
/* no extra stack contents */
typedef struct k_ucontext {
        unsigned long     uc_flags;
	struct ucontext  *uc_link;
	stack_t           uc_stack;
	mcontext_t	  uc_mcontext;
	sigset_t          uc_sigmask;   /* mask last for extensibility */
} k_ucontext_t;
/* extra stack contents (fpu state, exception frame part (> 4 words) */                                        
typedef struct k_ucontextf {
        unsigned long     uc_flags;
	struct ucontext  *uc_link;
	stack_t           uc_stack;
	mcontext_t	  uc_mcontext;
	sigset_t          uc_sigmask;   /* mask last for extensibility */
	long int uc_filler[174];
} k_ucontextf_t;
                                        
int depth = 200000 /* 32768 ; 200000 */;

const unsigned long i0 = 0x91929394;
const unsigned long i1 = 0xa1a2a3a4;
const unsigned long i2 = 0xb1b2b3b4;
const unsigned long i3 = 0xc1c2c3c4;
const unsigned long i4 = 0xd1d2d3d4;
const unsigned long i5 = 0xe1e2e3e4;
const unsigned long i6 = 0xf1f2f3f4;

unsigned long o0;
unsigned long o1;
unsigned long o2;
unsigned long o3;
unsigned long o4;
unsigned long o5;
unsigned long o6;

unsigned long parent_usp, child_usp;

static void rec(void)
{
	// initialize registers
	asm(	"	move.l %0, %%a2\n" 
		"	move.l %1, %%a3\n"
		"	move.l %2, %%a4\n"
		"	move.l %3, %%a5\n"
		"	move.l %4, %%d2\n"
		"	move.l %5, %%d3\n"
		"	move.l %6, %%d4\n"
		:
		: "m" (i0), "m" (i1), "m" (i2),
		  "m" (i3), "m" (i4), "m" (i5), "m" (i6)
		: "a2", "a3", "a4", "a5", "d2", "d3", "d4"
	);

	// note current usp
	asm(	"	move.l %%sp, %0\n" 
		: "=m" (parent_usp)
		:
		: "memory" 
	);

	// maybe fork a short-lived process
	if ((depth & 0x7ff) == 0)
		if (fork() == 0) {
			// note current usp
			asm(	"	move.l %%sp, %0\n" 
				: "=m" (child_usp)
				:
				: "memory" 
			);
			exit(0);
		}

	if (--depth)
		rec();	// callee to save & restore registers

	// compare register contents
	asm(	"	move.l %%a2, %0\n" 
		"	move.l %%a3, %1\n"
		"	move.l %%a4, %2\n"
		"	move.l %%a5, %3\n"
		"	move.l %%d2, %4\n"
		"	move.l %%d3, %5\n"
		"	move.l %%d4, %6\n"
		: "=m" (o0), "=m" (o1), "=m" (o2),
		  "=m" (o3), "=m" (o4), "=m" (o5), "=m" (o6)
		:
		: "memory" /* "a2", "a3", "a4", "a5", "d2", "d3", "d4" */
	);
	if (o0 != i0 || o1 != i1 || o2 != i2 ||
	    o3 != i3 || o4 != i4 || o5 != i5 || o6 != i6)
		asm("illegal");
}

static void handler(int nevermind)
{
}
            
static void rthandler(int signo, siginfo_t *info, void *ucontext)
{
	/* use struct ucontext from libc headers */
	// ucontext_t *ctx = (ucontext_t *) ucontext;
	/* use struct ucontext from kernel source */
	k_ucontext_t *ctx = (k_ucontext_t *) ucontext;
	k_ucontextf_t *ctxf = (k_ucontextf_t *) ucontext;

	if (signo == SIGCHLD) {
		unsigned long usp, pc;
		int i;

		asm(	"	move.l %%sp, %0\n" 
			: "=m" (usp)
			:
			: "memory" 
		);
		fprintf(stderr, "parent usp  : 0x%lx\n", parent_usp);
		// fprintf(stderr, "child usp   : 0x%lx\n", child_usp);
		/* top of signal stack - using ucontext w/o uc_filler
		 * Adding filler (which will take additional exception
		 * frame contents for longer tha four word frames),
		 * top of stack may end up above parent usp! */
		fprintf(stderr, "handler tos : 0x%lx\n", usp + sizeof(struct siginfo) + sizeof(struct k_ucontext) + 24);
                if (usp + sizeof(struct siginfo) + sizeof(struct k_ucontext) + 24 > parent_usp)
                        fprintf(stderr, "handler stack overwrote usp!\n");
		fprintf(stderr, "handler usp : 0x%lx\n", usp);
		fprintf(stderr, "signal usp  : 0x%lx\n", ctxf->uc_mcontext.gregs[15]);
		fprintf(stderr, "signal pc   : 0x%lx\n", ctxf->uc_mcontext.gregs[16]);
		fprintf(stderr, "signal fmtv : 0x%lx\n", ctxf->uc_filler[54]);
		fprintf(stderr, "\n");
	}
}

int main(void)

{
	struct sigaction act;
	stack_t ss;

	ss.ss_sp = malloc(SIGSTKSZ);
	if (ss.ss_sp == NULL) {
		perror("malloc");
		exit(EXIT_FAILURE);
	}
	
	ss.ss_size = SIGSTKSZ;
	ss.ss_flags = 0;
	if (sigaltstack(&ss, NULL) == -1) {
		perror("sigaltstack");
		exit(EXIT_FAILURE);
	}
	
#ifdef USE_ALTSTACK
	fprintf(stderr, "alt sig stack: 0x%lx - 0x%lx\n", ss.ss_sp, ss.ss_sp + ss.ss_size);

	act.sa_flags = SA_ONSTACK;
	act.sa_flags |= SA_SIGINFO;
	act.sa_sigaction = rthandler;
#else
	act.sa_flags = SA_SIGINFO;
	act.sa_handler = handler;
#endif
	sigemptyset(&act.sa_mask);
	if (sigaction(SIGCHLD, &act, NULL) == -1) {
		perror("sigaction");
		exit(EXIT_FAILURE);
	}

	rec();
}