Re: New capabilities for plugins

["Alex Bennée" <alex.bennee@linaro.org>]

Florian Hofhammer <florian.hofhammer@epfl.ch> writes:

> Hi Daniel,
>
> My apologies for the late reply, I've been out of office the past two
> weeks.
>
>> Yeah, this sounds like it is potentially going a step too far in enabling
>> fully out of tree extension of core QEMU functionality.
>> If something conceptually is in scope of the core QEMU codebase,
>> then
>> IMHO, our plugin system should aim to avoid enabling external
>> implementations as far as is practical.  That was easy when plugins
>> were limited to observability, but the more we enable in terms of
>> state modification the wider pandora's box is opened.
>> Where to draw the line is a hard problem.
>
> As I'm new to the QEMU mailing list (not to tinkering with QEMU /
> implementing things in or on top of QEMU, though), I'm not fully
> familiar with the requirement of preventing out of tree extensions
> (or at least certain functionality in such plugins).

I don't think we can prevent out-of-tree extensions. We do however say
the plugin API is not considered a stable interface - we want to be able
to evolve the interface without worrying about supporting code we can't
see.

There is basic versioning support in the plugin interface to catch
mismatches but there is certainly no backward compatibility - unlike for
example QEMU's approach to machine model versioning.

> I personally (of course with a somewhat biased view :)) see features
> that allow modifying internal state (such as syscall behavior) as
> beneficial or even required for certain dynamic binary analysis use
> cases. Currently, the situation in academic research is that researchers
> (i.e., typically PhD students) for dynamic analysis use cases fork QEMU
> and implement their use case directly in the core logic. Such patches
> can of course never be upstreamed, as they're very use-case-specific and
> typically don't generalize to the full QEMU functionality (or might even
> break / deteriorate other functionality that's not relevant to this
> specific use case).
> As soon as the corresponding project is considered "done" / "finished",
> the code basically just rots and never gets rebased onto newer versions
> of QEMU anymore, basically freezing it in time with all current bugs and
> without potential future improvements to QEMU's core.

There is a long history of academic forks of QEMU which have been done
to support various papers. Usually these changes never get upstreamed
and as you say get left to bitrot on old branches. This is unfortunate
because potential improvements to upstream QEMU never get made.

I should point out there are some cases of academic work being
upstreamed - the MTTCG changes led to a number of papers being published
and I like to think the overall quality of the academic work was
improved by having early review of the patches on the list as it was
being developed.

> That's why I personally actually really love QEMU's plugin interface: it
> allows me to introspect the system state without adding custom hooks
> everywhere in the code that might not be portable to newer QEMU
> versions! Consequently, my code is reusable either by myself later on or
> other researchers with any version of QEMU that supports at least the
> specified plugin API version.

Absolutely - the aim of the plugin interface was to remove the need for
custom hooks which inevitably bitrot when the instruction frontends get
re-factored. The unicorn engine fork for example will never be as up to
date for emulating AArch64 as the upstream because it is an actively
maintained architecture forever adding new CPU features.

> However, I often wish to modify QEMU's state as well, such as in the
> provided example with modifying syscall return values or skipping
> over a syscall altogether.
>
> Do I understand correctly that handling syscalls is considered in scope
> of the core QEMU codebase and therefore shouldn't be possible to do via
> a plugin?

I'm not sure. As I understand it the communities general worry is
splitting development unnecessarily and leaving QEMU as an "open core"
to which proprietary extensions became the only way to emulate certain
hardware or syscalls.

The security use-case certainly seems like a new novel use case that I'm
personally minded to support. It doesn't sound like much else is needed
other than the ability to reset the PC while executing.

> If I understand correctly, as it stands, there's no way to
> modify syscall behavior on an emulator-level with QEMU instead of at the
> kernel-level via seccomp, eBPF, kernel modules, ...
>
> Best regards,
> Florian

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro