using ptrace to cancel a syscall on sparc

[Mike Frysinger <vapier@gentoo.org>]

[-- Attachment #1: Type: text/plain, Size: 1266 bytes --]

i've been playing with ptrace on sparc and trying to use it to watch and
cancel specific syscalls.  i have this working for other arches already.

the test is pretty simple:
 - call open("f", O_CREAT)
 - call unlink("f")
the tracer will watch for the unlink, and when it gets notified, stuffs
the syscall # with -1 (so it'll get skipped).  then when it gets called
a second time, it stuffs the exit with -1/EPERM.

i'm using PTRACE_GETREGS/PTRACE_SETREGS to read/write the regsets (and
i've swapped the addr/data args specifically for sparc).  i extract the
syscall # from g1 (u_regs[0]) and syscalls args from o0..o5 (u_regs[7]
through u_regs[12]).  this seems to be working fine.

when setting the return value, i set PSR_C in the psr register, and i
set o0 to EPERM.  seems to be working (the unlink function returns a
-1 and errno is EPERM).

i'm having trouble with canceling of the syscall itself.  seems like
no matter what i stuff into o0, the kernel executes the unlink.  i've
tried tracing arch/sparc/kernel/syscalls.S and kernel/head_64.S, the
the entry is linux_sparc_syscall32 which calls linux_syscall_trace32,
but it seems like the o0 stuff doesn't seem to work for me.  my sparc
asm foo isn't strong enough to figure out what's going wrong :/.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]