New capabilities for plugins

[Florian Hofhammer <florian.hofhammer@epfl.ch>]

Hello,

I'm currently working a lot with QEMU plugins for dynamic analysis of 
userspace binaries (i.e., running under qemu-user). While working on 
that, I found that the QEMU plugin API luckily has been getting more and 
more capabilities with recent versions but that I'm still lacking some 
functionality for my use cases.
More specifically, the "vcpu_syscall_cb" and "vcpu_syscall_ret" 
callbacks already allow me to instrument syscall translation entry and 
exit points. While the register read/write APIs also allow me to modify 
register contents in my syscall callback implementations, there is 
currently no good way to emulate a syscall myself in the plugin or 
explicitly set the syscall return value (as it will be overwritten with 
the original syscall's return value again, even if I set the 
corresponding guest register).

I was wondering whether the QEMU community would be open to extending 
the plugin API so that a plugin can fully emulate a syscall without the 
original syscall being executed by QEMU. I had multiple approaches for 
that in mind, with some working patches locally that I'd be happy to 
share and build such a feature on:

1. Change the API of the existing callbacks so that the syscall entry 
point callback returns "bool" instead of "void" and if any of the 
registered callbacks returns true, execution of the actual syscall is 
skipped.
2. Introduce a new API function that sets a flag for a specific syscall 
to be skipped:
2a. A function that's called once in the manner of "always skip the 
syscall with this specific syscall number" or
2b. a function that's called every time in the syscall entry point 
callback in the manner of "skip this specific instance of the syscall".

I'd be happy to get your opinion on those proposals and to 
develop/submit the corresponding patches!

Thanks in advance and best regards,
Florian