Re: Rust kernel policy

[James Bottomley <James.Bottomley@HansenPartnership.com>]

On Wed, 2025-02-19 at 16:08 +0100, Miguel ol9 wrote:
> On Wed, Feb 19, 2025 at 3:05 PM James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> > 
> > So here's a proposal to fix this: could we not annotate the C
> > headers with the API information in such a way that a much improved
> > rust bindgen can simply generate the whole cloth API binding from
> > the C code?  We would also need an enhanced sparse like tool for C
> > that checked the annotations and made sure they got updated. 
> > Something like this wouldn't solve every unintentional rust build
> > break, but it would fix quite a few of them.  And more to the
> > point, it would allow non-rust developers to update the kernel API
> > with much less fear of breaking rust.
> 
> This has come up a few times, and we indeed would like to have some
> annotations in the C headers so that we can generate more (and to
> keep the information local).
> 
> For instance, it would be nice to have bindgen's `__opaque` near the
> C items, or being able to mark functions as `__safe`, or to have
> other `enum`s-related annotations, or even custom attributes, as well
> as "formatted-formally-enough" docs so that can be rendered properly
> on the Rust side, or even references/lifetimes with an eventual "Safe
> C"-like approach, and so on and so forth.
> 
> However, even if we automate more and even reach a point where most C
> APIs are e.g. "safe" (which would be great),

I wouldn't say C API safety would be the main goal, although it might
be a nice add on feature.

>  it wouldn't prevent breakage -- the C APIs would still need to be
> stable enough so that you don't break callers,

Just so we're on the same page, kernel API stability can't be the goal.
We can debate how valuable the current API instability is, but it's a
fact of life.  The point of the proposal is not to stabilise the C API
but to allow the instability to propagate more easily to the rust side.

>  including C ones. It would still be great to have that information
> formally expressed, though, of course, and it would help maintain the
> Rust sid

This very much depends on how the callers are coded, I think.  When I
looked at Wedson's ideas on this, the C API contracts were encoded in
the headers, so mostly only the headers not the body of the code had to
change (so the headers needed updating when the C API contract
changed). If the enhanced bindgen produces new headers then code like
this will just update without breaking (I admit not all code will work
like that, but it's still a useful property).

> We have also discussed at times is documenting the C side more, e.g.
> the pre/post/invariants we use on the Rust side. That would be useful
> for the C side to know something is being relied upon from Rust (and
> other C callers) and for the Rust side to document why something is
> sound. Of course, it is a lot of work, and the more we can express as
> code instead of as documentation, the better.

So I do think this feeds into the documentation project as well.  We've
already decided that the best way to document an API is in the code for
it, so adding annotations that can be checked is better than adding
docbook that not many people check; although hopefully we could still
generate documentation from the annotations.

Regards,

James