Re: Rust kernel policy

[Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>]

On Wed, Feb 19, 2025 at 5:03 PM James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
>
> Just so we're on the same page, kernel API stability can't be the goal.
> We can debate how valuable the current API instability is, but it's a
> fact of life.  The point of the proposal is not to stabilise the C API
> but to allow the instability to propagate more easily to the rust side.

Sure, I didn't mean to imply that -- I am only trying to say that,
even if you add a lot of information to the C headers, you would still
have to update callers (both C and Rust ones).

Now, there are C APIs that even if they are not guaranteed to be
stable, they are fairly stable in practice, so the pain can be fairly
low in some cases.

But please see below on what "Rust callers" mean here -- it is not
every Rust module, but rather just the "abstractions".

> This very much depends on how the callers are coded, I think.  When I
> looked at Wedson's ideas on this, the C API contracts were encoded in
> the headers, so mostly only the headers not the body of the code had to
> change (so the headers needed updating when the C API contract
> changed). If the enhanced bindgen produces new headers then code like
> this will just update without breaking (I admit not all code will work
> like that, but it's still a useful property).

Hmm... I am not sure exactly what you mean here. Are you referring to
Wedson's FS slides from LSF/MM/BPF? i.e are you referring to Rust
signatures?

If yes, those signatures are manually written, they are not the
generated bindings. We typically refer to those as "abstractions", to
differentiate from the generated stuff.

The Rust callers (i.e. the users of those abstractions) definitely do
not need to change if the C APIs change (unless they change in a major
way that you need to redesign your Rust abstractions layer, of
course).

So, for instance, if your C API gains a parameter, then you should
update all your C callers as usual, plus the Rust abstraction that
calls C (which could be just a single call). But you don't need to
update all the Rust modules that call Rust abstractions.

In other words, we do not call C directly from Rust modules, in fact,
we forbid it (modulo exceptional/justified cases). There is a bit more
on that here, with a diagram:

    https://docs.kernel.org/rust/general-information.html#abstractions-vs-bindings

In summary, those abstractions give you several things: the ability to
provide safe APIs for Rust modules (instead of unsafe calls
everywhere), the ability to write idiomatic Rust in your callers
(instead of FFI) and the ability to reduce breaks like I think you are
suggesting.

Now, generating those safe abstractions automatically would be quite
an achievement, and it would require more than just a few simple
annotations in the header. Typically, it requires understanding the C
implementation, and even then, it is hard for a human to do, i.e. we
are talking about an open problem.

Perhaps you could approximate it with an AI that you give the C
implementation, plus the C headers, plus the C headers and
implementations that those call, and so on, up to some layer. Even
then, it is a problem that typically has many different valid
solutions, i.e. you can design your safe Rust API in different ways
and with different tradeoffs.

I hope that clarifies.

Cheers,
Miguel