Re: [RFC PATCH v1 0/6] Implement ARM PL011 in Rust

[Manos Pitsidianakis <manos.pitsidianakis@linaro.org>]

On Mon, 10 Jun 2024 22:59, Stefan Hajnoczi <stefanha@gmail.com> wrote:
>> What are the issues with not using the compiler, rustc, directly?
>> -----------------------------------------------------------------
>> [whataretheissueswith] Back to [TOC]
>>
>> 1. Tooling
>>    Mostly writing up the build-sys tooling to do so. Ideally we'd
>>    compile everything without cargo but rustc directly.
>
>Why would that be ideal?

It remove the indirection level of meson<->cargo<->rustc. I don't have a 
concrete idea on how to tackle this, but if cargo ends up not strictly 
necessary, I don't see why we cannot use one build system.

>
>>
>>    If we decide we need Rust's `std` library support, we could
>>    investigate whether building it from scratch is a good solution. This
>>    will only build the bits we need in our devices.
>
>Whether or not to use std is a fundamental decision. It might be
>difficult to back from std later on. This is something that should be
>discussed in more detail.
>
>Do you want to avoid std for maximum flexibility in the future, or are
>there QEMU use cases today where std is unavailable?

For flexibility, and for being compatible with more versions.

But I do not want to avoid it, what I am saying is we can do a custom 
build of it instead of linking to the rust toolchain's prebuilt version.

>
>>
>> 2. Rust dependencies
>>    We could go without them completely. I chose deliberately to include
>>    one dependency in my UART implementation, `bilge`[0], because it has
>>    an elegant way of representing typed bitfields for the UART's
>>    registers.
>>
>> [0]: Article: https://hecatia-elegua.github.io/blog/no-more-bit-fiddling/
>>      Crates.io page: https://crates.io/crates/bilge
>>      Repository: https://github.com/hecatia-elegua/bilge
>
>I guess there will be interest in using rust-vmm crates in some way.
>
>Bindings to platform features that are not available in core or std
>will also be desirable. We probably don't want to reinvent them.


Agreed.

>
>>
>> Should QEMU use third-party dependencies?
>> -----------------------------------------
>> [shouldqemuusethirdparty] Back to [TOC]
>>
>> In my personal opinion, if we need a dependency we need a strong
>> argument for it. A dependency needs a trusted upstream source, a QEMU
>> maintainer to make sure it us up-to-date in QEMU etc.
>>
>> We already fetch some projects with meson subprojects, so this is not a
>> new reality. Cargo allows you to define "locked" dependencies which is
>> the same as only fetching specific commits by SHA. No suspicious
>> tarballs, and no disappearing dependencies a la left-pad in npm.
>>
>> However, I believe it's worth considering vendoring every dependency by
>> default, if they prove to be few, for the sake of having a local QEMU
>> git clone buildable without network access.
>
>Do you mean vendoring by committing them to qemu.git or just the
>practice of running `cargo vendor` locally for users who decide they
>want to keep a copy of the dependencies?


Committing, with an option to opt-out. They are generally not big in 
size. I am not of strong opinion on this one, I'm very open to 
alternatives.


>>
>> Should QEMU provide wrapping Rust APIs over QEMU internals?
>> -----------------------------------------------------------
>> [qemuprovidewrappingrustapis] Back to [TOC]
>>
>> My personal opinion is no, with the reasoning being that QEMU internals
>> are not documented or stable. However I do not see why creating stable
>> opt-in interfaces is bad. It just needs someone to volunteer to maintain
>> it and ensure there are no breakages through versions.
>
>Rust code will need to interface with QEMU's C APIs, so Rust wrappers
>seem unavoidable. Using a protocol like vhost-user might be possible
>in some cases. It separates the two codebases so they can both be
>native and without bindings, but that won't work for all parts of the
>QEMU source tree.
>
>Stable APIs aren't necessary if most developers in the QEMU community
>are willing to work in both languages. They can adjust both C and Rust
>code when making changes to APIs. I find this preferable to having
>Rust maintainers whose job is to keep wrappers up-to-date. Those Rust
>maintainers would probably burn out. This seems like a question of
>which approach the developer community is comfortable with.


Me too.

>
>>
>> Will QEMU now depend on Rust and thus not build on my XYZ platform?
>> -------------------------------------------------------------------
>> [qemudependonrustnotbuildonxyz] Back to [TOC]
>>
>> No, worry about this in some years if this experiment takes off. Rust
>> has broad platform support and is present in most distro package
>> managers. In the future we might have gcc support for it as well.
>>
>> For now, Rust will have an experimental status, and will be aimed to
>> those who wish to try it. I leave it to the project leaders to make
>> proper decisions and statements on this if necessary.
>
>This can be discussed in a separate email thread if you prefer, but I
>do think it needs agreement soon so that people have the confidence to
>invest their time in writing Rust. They need to know that the code
>they develop will be available on most platforms where QEMU is
>available and that others in the community won't object or insist on a
>C implementation for platform support reasons.

Definitely, also it's out of scope for this RFC since we're not writing 
and rules/guidelines yet.

>
>>
>>
>> How is the compilation structured?
>> ==================================
>> [howisthecompilationstructured] Back to [TOC]
>>
>> First, a meson target that runs `bindgen` on a bunch of header files
>> (defined in `rust/wrapper.h`) is created as a target and as a dependency
>> for any rust hardware device that needs it. You can see the generated
>> bindings by running
>>
>>   ninja generated.rs
>>
>> inside your build directory.
>>
>> The devices are defined as dictionaries in rust/meson.build because they
>> depend on the bindgen dependency, which is available much later in the
>> meson process (when the static qemu lib and target emulator executables
>> are defined).
>>
>> A cargo wrapper python script under scripts/ exists to build the crate
>> library, by providing the path to the generated.rs bindings via the
>> environment. Then, the qemu-system-aarch64 binary links against the
>> staticlib archive (i.e. libpl011.a)
>>
>> The generated.rs rust file includes a bunch of junk definitions?
>> ================================================================
>> [generatedrsincludesjunk] Back to [TOC]
>>
>> Yes, bindgen allows you to block certain types and identifiers from
>> being generated but they are simply too many. I have trimmed some of the
>> fat but vast improvements can be made.
>>
>> The staticlib artifact contains a bunch of mangled .o objects?
>> ==============================================================
>> [staticlibmangledobjects] Back to [TOC]
>>
>> Yes, until we compile without the `std` module library or we compile it
>> manually instead of linking it, we will have some junk in it.
>
>What is the consequence of this? As long as the linker is bringing in
>.o files from the .a only through symbol dependencies, then unused .o
>files in the .a won't be linked into the final QEMU binary.

No consequence, I just want to warn anyone peeking into the rust output 
(not the final qemu binary) to expect junk.