QEMU Rust Crates + Plugins in Rust

QEMU Rust Crates + Plugins in Rust

[Rowan Hart]

Hi all, this is my first post to the mailing list! I've spent the last
couple weeks building QEMU Rust crates to enable a couple goals:

- Install QEMU binaries using cargo, the Rust package manager
- Use git-latest QEMU binaries in Rust projects as a dependency
- Write QEMU TCG plugins entirely in Rust

The main crate is https://crates.io/crates/qemu, a thin wrapper around
the existing QEMU build system, using crate features to toggle
configure options. There are also crates for each binary build target,
for example https://crates.io/crates/qemu-system-x86_64. These
crates allow users to run `cargo install qemu-system-x86_64`, which
installs a small wrapper binary with a dependency on the built
binaries from the qemu crate. These binary installs run exactly as a
normal QEMU installation via package manager would, and essentially
just provide another alternative distro-agnostic packaging mechanism.

More interesting (I think) is the crate
https://github.com/novafacing/cannonball, which depends on the qemu
crate and enables developers
to write QEMU TCG plugins entirely in Rust. This enables a very nice
workflow for Rust developers interested in building tracers,
profilers, and
other tooling already supported by the TCG Plugin API. The cannonball
repository has one example of such a tracer that outputs various
events
as JSON.

This is a pretty rough implementation I've hacked together over a week
or two, but I've heard from IRC there is some interest in
experimenting
with Rust as part of QEMU and I'm interested in feedback on this
attempt in that direction.

There's a bit more information in my post about this on cohost:
https://cohost.org/novafacing/post/240422-some-qemu-and-plugi

Finally, I am acutely aware in doing this I've taken control of almost
70 crate names the QEMU project might like to use at some point. I'm
not
here to step on toes, so if the decision makers would like the QEMU
project to get control of these crate names, just let me know and I'll
email
the crates.io team about getting them moved. Otherwise, comments,
discussion about Rust-ifying QEMU and its plugins, ideas, and PRs
welcome,
as an author of a decent number of TCG plugins the last few years I'm
pretty excited about what even this rudimentary approach has enabled
with regards to ease of use and development speed.

-Rowan

Re: QEMU Rust Crates + Plugins in Rust

[Alex Bennée]

Rowan Hart <rowanbhart+devel@gmail.com> writes:

> Hi all, this is my first post to the mailing list!

Apologies on the delay in responding. I've been mostly focused on the
recent release.

> I've spent the last
> couple weeks building QEMU Rust crates to enable a couple goals:
>
> - Install QEMU binaries using cargo, the Rust package manager
> - Use git-latest QEMU binaries in Rust projects as a dependency
> - Write QEMU TCG plugins entirely in Rust
>
> The main crate is https://crates.io/crates/qemu, a thin wrapper around
> the existing QEMU build system, using crate features to toggle
> configure options. There are also crates for each binary build target,
> for example https://crates.io/crates/qemu-system-x86_64. These
> crates allow users to run `cargo install qemu-system-x86_64`, which
> installs a small wrapper binary with a dependency on the built
> binaries from the qemu crate. These binary installs run exactly as a
> normal QEMU installation via package manager would, and essentially
> just provide another alternative distro-agnostic packaging mechanism.

I find it a little confusing that the numbering scheme doesn't follow
the upstream. Is it possible to mirror the upstream version numbers?
Does rust's semver have any concept of an edge or HEAD build?

I tried:

  cargo install qemu-aarch64

but it fell over with complains about various rust bits:


     Compiling nix v0.25.1
  error[E0658]: `let...else` statements are unstable
     --> /home/alex/.cargo/registry/src/github.com-1ecc6299db9ec823/memfd-exec-0.1.4/src/executable.rs:347:13
      |
  347 |             let Err(err) = (unsafe { self.do_exec(theirs, envp) }) else { unreachable!("..."); };
      |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |
      = note: see issue #87335 <https://github.com/rust-lang/rust/issues/87335> for more information

  error[E0658]: `let...else` statements are unstable
     --> /home/alex/.cargo/registry/src/github.com-1ecc6299db9ec823/memfd-exec-0.1.4/src/executable.rs:468:17
      |
  468 |                 let Err(e) = self.do_exec(theirs, envp) else { unreachable!("..."); };
      |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |
      = note: see issue #87335 <https://github.com/rust-lang/rust/issues/87335> for more information

  error[E0658]: use of unstable library feature 'io_safety'
   --> /home/alex/.cargo/registry/src/github.com-1ecc6299db9ec823/memfd-exec-0.1.4/src/anon_pipe.rs:4:25
    |
  4 |     os::unix::prelude::{AsFd, AsRawFd, BorrowedFd, FromRawFd, IntoRawFd, RawFd},
    |                         ^^^^
    |
    = note: see issue #87074 <https://github.com/rust-lang/rust/issues/87074> for more information

  error[E0658]: use of unstable library feature 'io_safety'
   --> /home/alex/.cargo/registry/src/github.com-1ecc6299db9ec823/memfd-exec-0.1.4/src/anon_pipe.rs:4:40
    |
  4 |     os::unix::prelude::{AsFd, AsRawFd, BorrowedFd, FromRawFd, IntoRawFd, RawFd},
    |                                        ^^^^^^^^^^
    |
    = note: see issue #87074 <https://github.com/rust-lang/rust/issues/87074> for more information

  error[E0658]: use of unstable library feature 'io_safety'
    --> /home/alex/.cargo/registry/src/github.com-1ecc6299db9ec823/memfd-exec-0.1.4/src/anon_pipe.rs:68:5
     |
  68 | /     fn as_fd(&self) -> BorrowedFd<'_> {
  69 | |         self.0.as_fd()
  70 | |     }
     | |_____^
     |
     = note: see issue #87074 <https://github.com/rust-lang/rust/issues/87074> for more information

Running:

  cargo build

in a checked out repo completes but I don't see any binaries in the
output. This is perhaps a misunderstanding on my part on how the repo is
meant to be used?

> More interesting (I think) is the crate
> https://github.com/novafacing/cannonball, which depends on the qemu
> crate and enables developers
> to write QEMU TCG plugins entirely in Rust. This enables a very nice
> workflow for Rust developers interested in building tracers,
> profilers, and
> other tooling already supported by the TCG Plugin API. The cannonball
> repository has one example of such a tracer that outputs various
> events
> as JSON.

I ran into the same build problems with this.

> This is a pretty rough implementation I've hacked together over a week
> or two, but I've heard from IRC there is some interest in
> experimenting
> with Rust as part of QEMU and I'm interested in feedback on this
> attempt in that direction.

I expect the biggest thing would be how to incorporate the build of any
rust bits in the main build environment. I don't think we are at the
point we would want to dump the entire build environment for a native
rust based one.

From a plugin point of view I'm certainly keen to enable the building of
plugins written in rust. However currently having everything wrapped up
in cargo makes it confusing as to what exactly is going on. For example
if I just pick up libjaivana.so like:

  ./qemu-aarch64 -d plugin -plugin
    ~/lsrc/cannonball.git/target/debug/libjaivana.so \
    ./tests/tcg/aarch64-linux-user/sysregs

I don't see anything output (btw qemu_plugin_outs uses the logging
system so you need to enable the output with -d plugin).


> There's a bit more information in my post about this on cohost:
> https://cohost.org/novafacing/post/240422-some-qemu-and-plugi
>
> Finally, I am acutely aware in doing this I've taken control of almost
> 70 crate names the QEMU project might like to use at some point. I'm
> not
> here to step on toes, so if the decision makers would like the QEMU
> project to get control of these crate names, just let me know and I'll
> email
> the crates.io team about getting them moved. Otherwise, comments,
> discussion about Rust-ifying QEMU and its plugins, ideas, and PRs
> welcome,
> as an author of a decent number of TCG plugins the last few years I'm
> pretty excited about what even this rudimentary approach has enabled
> with regards to ease of use and development speed.

Me too. I'm hoping to get register access sorted next year although
I have some clean-up for the gdbstub to do first.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro