somewhat OT -- trying to build code on the fly then run it

somewhat OT -- trying to build code on the fly then run it

[Chris Friesen]

I'm writing a testcase to test some code we wrote to allow userspace to 
flush the whole dcache and invalidate the whole icache.  This requires 
me to write self-modifying code.  For the first stage I'm just trying to 
build a routine (that just increments r3 and returns) on the heap and 
then call it.

Everything seems to be fine right up until I jump to the code that I've 
written, then I get a segfault.  The debugger shows my registers and 
data values are as expected, and the page of memory has xwr permissions.

Can anyone tell me what I'm missing?  I'm sure its something simple.

Thanks,

Chris.


PS.  Here's my current test code.

#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <stdio.h>
#include <stdio.h>
#include <sys/mman.h>

/* these next two lines are the hex equivalents of the instructions:
  * addi r3,r3,1
  * blr
  */
unsigned int incr_code[] = {
	0x38630001,
	0x4e800020
};

int dotest(unsigned long *addr)
{
         int i=0;
         asm volatile (" \n\
                 mr 3,%1 \n\
                 bla %2  \n\
                 mr %0,3 \n"
                 : "=r" (i)
                 : "r" (i), "r" (addr));
         return i;
}

void alter_opcode(unsigned long *addr, unsigned long opcode)
{
	unsigned long offset = 0;
	
	asm volatile(
                 "stw    %1,0(%0)  \n\t"
                 "dcbf   %0,%2     \n\t"
                 "sync             \n\t"
                 "icbi   %0,%2     \n\t"
                 "sync             \n\t"
                 "isync            \n\t"
                     :: "r" (addr), "r" (opcode), "r" (offset));
}

int main()
{
	unsigned long *addr;
	void *p = mmap(0, 4096, PROT_EXEC|PROT_WRITE|PROT_READ,
		MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
	if (p == MAP_FAILED) {
		perror("mmap");
		goto error;
	}
	
	addr = (unsigned long *)p;
	
	alter_opcode(addr, incr_code[0]);
	alter_opcode(addr+1, incr_code[1]);
	
	printf("%d\n", dotest(addr));
	return 0;
	
error:
	return -1;
}

Re: somewhat OT -- trying to build code on the fly then run it

[Benjamin Herrenschmidt]

On Mon, 2005-04-04 at 17:32 -0600, Chris Friesen wrote:
> I'm writing a testcase to test some code we wrote to allow userspace to 
> flush the whole dcache and invalidate the whole icache.  This requires 
> me to write self-modifying code.  For the first stage I'm just trying to 
> build a routine (that just increments r3 and returns) on the heap and 
> then call it.
> 
> Everything seems to be fine right up until I jump to the code that I've 
> written, then I get a segfault.  The debugger shows my registers and 
> data values are as expected, and the page of memory has xwr permissions.
> 
> Can anyone tell me what I'm missing?  I'm sure its something simple.
> 
> Thanks,
> 
> Chris.
> 
> 
> PS.  Here's my current test code.
> 
> #include <errno.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> #include <sys/ioctl.h>
> #include <unistd.h>
> #include <stdio.h>
> #include <stdio.h>
> #include <sys/mman.h>
> 
> /* these next two lines are the hex equivalents of the instructions:
>   * addi r3,r3,1
>   * blr
>   */
> unsigned int incr_code[] = {
> 	0x38630001,
> 	0x4e800020
> };
> 
> int dotest(unsigned long *addr)
> {
>          int i=0;
>          asm volatile (" \n\
>                  mr 3,%1 \n\
>                  bla %2  \n\
>                  mr %0,3 \n"
>                  : "=r" (i)
>                  : "r" (i), "r" (addr));
>          return i;
> }

bla can only be used for small addresses (or very high addresses), and
it doesn't take a register argument but an absolute address. You want
something different, more like

	mtctr %2
	bctrl

Though you also need to add proper "clobber" constraints to indicate to
the compiler what will be clobbered by the routine you are calling (look
at the syscall macros of the kernel for an example of rather standard
clobber lists).

> void alter_opcode(unsigned long *addr, unsigned long opcode)
> {
> 	unsigned long offset = 0;
> 	
> 	asm volatile(
>                  "stw    %1,0(%0)  \n\t"
>                  "dcbf   %0,%2     \n\t"
>                  "sync             \n\t"
>                  "icbi   %0,%2     \n\t"
>                  "sync             \n\t"
>                  "isync            \n\t"
>                      :: "r" (addr), "r" (opcode), "r" (offset));
> }
> 
> int main()
> {
> 	unsigned long *addr;
> 	void *p = mmap(0, 4096, PROT_EXEC|PROT_WRITE|PROT_READ,
> 		MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
> 	if (p == MAP_FAILED) {
> 		perror("mmap");
> 		goto error;
> 	}
> 	
> 	addr = (unsigned long *)p;
> 	
> 	alter_opcode(addr, incr_code[0]);
> 	alter_opcode(addr+1, incr_code[1]);
> 	
> 	printf("%d\n", dotest(addr));
> 	return 0;
> 	
> error:
> 	return -1;
> }
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-dev
-- 
Benjamin Herrenschmidt <benh@kernel.crashing.org>

Re: somewhat OT -- trying to build code on the fly then run it

[Chris Friesen]

Benjamin Herrenschmidt wrote:

> bla can only be used for small addresses (or very high addresses), and
> it doesn't take a register argument but an absolute address. You want
> something different, more like
> 
> 	mtctr %2
> 	bctrl
> 
> Though you also need to add proper "clobber" constraints to indicate to
> the compiler what will be clobbered by the routine you are calling (look
> at the syscall macros of the kernel for an example of rather standard
> clobber lists).


Thanks for the tip.  With those changes, the code runs perfectly.  For 
posterity, the new function looks like this:

int dotest(void *p)
{
         int i=0;
         asm volatile (" \n\
                 mr 3,%1 \n\
		mtctr %2  \n\
		bctrl  \n\
                 mr %0,3 \n"
                 : "=r" (i)
                 : "r" (i), "r" (p)
		: "ctr", "lr");
         return i;
}


The bad news is that it seems to indicate that our kernel code to flush 
the icache doesn't seem to work.  I'll open a new thread for that.

Chris

RE: somewhat OT -- trying to build code on the fly then run it

[Fillod Stephane]

Chris Friesen wrote:=20
> I'm writing a testcase to test some code we wrote to allow userspace
to=20
> flush the whole dcache and invalidate the whole icache. =20

Your testcase sounds like this kernel module for RT load generator:

	http://rtai.dk/cgi-bin/gratiswiki.pl?Latency_Killer/Flushy


Comments welcome.
--=20
Stephane

Re: somewhat OT -- trying to build code on the fly then run it

[Chris Friesen]

Fillod Stephane wrote:
> Chris Friesen wrote: 
> 
>>I'm writing a testcase to test some code we wrote to allow userspace
> to flush the whole dcache and invalidate the whole icache.  
> 
> 
> Your testcase sounds like this kernel module for RT load generator:
> 
> 	http://rtai.dk/cgi-bin/gratiswiki.pl?Latency_Killer/Flushy

Unfortunately that code calls flush_instruction_cache(), which doesn't 
work on the 970 in the ppc tree, and doesn't even exist anymore in the 
ppc64 tree.

Chris