Re: [PATCH] chore(util): minimal Rust build example

Re: [PATCH] chore(util): minimal Rust build example

[Kent Overstreet]

On Tue, May 02, 2023 at 11:53:54PM -0700, TruongSinh Tran-Nguyen wrote:
> This is a minimal demonstration of how to use Rust in kernel side
> of bcachefs. Kernel side rust does not have Rust `std`, thus
> formating is a bit more challenging than doing from user space

So if I'm following, it looks like everything is basically there. It
looks like we can add Rust source files to fs/bcachefs/ and have them
built as part of the bcachefs module, and calling back and forth just
works as expected - nice.

The kbuild stuff looks nice and clean - congrats to those responsible :)

We pretty heavily use to_text() functions kernel side; these output to
printbufs, and in Rust they map nicely to the Display trait - do we have
that available in the kernel or is that in std?

> This minimal demo implement `uuid_le_cmp`, but eventually,
> in Rust, we probably do not even need to use it, just simply
> `u1.b != u2.b` is good enough. Furthermore, we might consider
> (even in C code) to use uuid_t instead of uuid_le

Sounds like that would be a welcome cleanup :)

> Signed-off-by: TruongSinh Tran-Nguyen <i@truongsinh.pro>
> ---
>  fs/bcachefs/Makefile              |  3 +++
>  fs/bcachefs/util.h                |  8 ++++++++
>  fs/bcachefs/util_rust.rs          | 17 +++++++++++++++++
>  scripts/generate_rust_analyzer.py |  2 +-
>  4 files changed, 29 insertions(+), 1 deletion(-)
>  create mode 100644 fs/bcachefs/util_rust.rs
> 
> diff --git a/fs/bcachefs/Makefile b/fs/bcachefs/Makefile
> index a71956048a027..972976aac6921 100644
> --- a/fs/bcachefs/Makefile
> +++ b/fs/bcachefs/Makefile
> @@ -72,3 +72,6 @@ bcachefs-y		:=	\
>  	xattr.o
>  
>  bcachefs-$(CONFIG_BCACHEFS_POSIX_ACL) += acl.o
> +
> +bcachefs-$(CONFIG_RUST) +=	\
> +	util_rust.o	\
> diff --git a/fs/bcachefs/util.h b/fs/bcachefs/util.h
> index 9eb86c58e5509..6e87baee760a9 100644
> --- a/fs/bcachefs/util.h
> +++ b/fs/bcachefs/util.h
> @@ -839,9 +839,17 @@ static inline int u8_cmp(u8 l, u8 r)
>  
>  #include <linux/uuid.h>
>  
> +#ifdef CONFIG_RUST
> +int deprecated_uuid_le_cmp(const uuid_le *b1, const uuid_le *b2);
> +#endif
> +
>  static inline int uuid_le_cmp(const uuid_le u1, const uuid_le u2)
>  {
> +	#ifdef CONFIG_RUST
> +	return deprecated_uuid_le_cmp(&u1, &u2);
> +	#else
>  	return memcmp(&u1, &u2, sizeof(uuid_le));
> +	#endif
>  }
>  
>  #endif /* _BCACHEFS_UTIL_H */
> diff --git a/fs/bcachefs/util_rust.rs b/fs/bcachefs/util_rust.rs
> new file mode 100644
> index 0000000000000..23338c29384c0
> --- /dev/null
> +++ b/fs/bcachefs/util_rust.rs
> @@ -0,0 +1,17 @@
> +/* SPDX-License-Identifier: MIT */
> +
> +#[repr(C)]
> +pub struct UuidLe {
> +  b: [u8; 16],
> +}
> +
> +// this function is prefixed `deprecated` because it uses
> +// uuid_le from /usr/include/linux/uuid.h, which itself is
> +// deprecated in favor of uuid_t from include/linux/uuid.h
> +// it also does not exactly matches memcmp
> +#[no_mangle]
> +pub extern "C" fn deprecated_uuid_le_cmp( u1 : &UuidLe, u2 : &UuidLe) -> bool
> +{
> +  // pr_info!("using deprecated_uuid_le_cmp");
> +  return u1.b != u2.b;
> +}
> diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
> index 946e250c1b2a6..d258d10bb6e31 100755
> --- a/scripts/generate_rust_analyzer.py
> +++ b/scripts/generate_rust_analyzer.py
> @@ -98,7 +98,7 @@ def generate_crates(srctree, objtree, sysroot_src):
>      # Then, the rest outside of `rust/`.
>      #
>      # We explicitly mention the top-level folders we want to cover.
> -    for folder in ("samples", "drivers"):
> +    for folder in ("samples", "drivers", 'fs'):
>          for path in (srctree / folder).rglob("*.rs"):
>              logging.info("Checking %s", path)
>              name = path.name.replace(".rs", "")
> -- 
> 2.30.2
> 

Re: [PATCH] chore(util): minimal Rust build example

[TruongSinh Tran-Nguyen]

On Tue, 9 May 2023 at 10:47, Kent Overstreet <kent.overstreet@linux.dev> wrote:

> We pretty heavily use to_text() functions kernel side; these output to
> printbufs, and in Rust they map nicely to the Display trait - do we have
> that available in the kernel or is that in std?
>

https://doc.rust-lang.org/std/fmt/trait.Display.html Display trait is definitely
in Rust `std`. As for the alternative from kernel, I am still searching.
Worst case, there is nothing right now, Rust-for-linux folks would
happy to take PR
to include it.

> Sounds like that would be a welcome cleanup :)

By clean up, do you mean moving from `uuid_le` to `uuid_t`?

Also, I don't expect this PR to be merge, as it is for demonstration
purposes only,
however, what would be a minimum mergeable PR (related to Rust build)
that you would consider?

Re: [PATCH] chore(util): minimal Rust build example

[Kent Overstreet]

On Tue, May 09, 2023 at 10:57:18AM -0700, TruongSinh Tran-Nguyen wrote:
> On Tue, 9 May 2023 at 10:47, Kent Overstreet <kent.overstreet@linux.dev> wrote:
> 
> > We pretty heavily use to_text() functions kernel side; these output to
> > printbufs, and in Rust they map nicely to the Display trait - do we have
> > that available in the kernel or is that in std?
> >
> 
> https://doc.rust-lang.org/std/fmt/trait.Display.html Display trait is definitely
> in Rust `std`. As for the alternative from kernel, I am still searching.
> Worst case, there is nothing right now, Rust-for-linux folks would
> happy to take PR
> to include it.

I included them on the CC, maybe they'll have something to say.

> 
> > Sounds like that would be a welcome cleanup :)
> 
> By clean up, do you mean moving from `uuid_le` to `uuid_t`?
> 
> Also, I don't expect this PR to be merge, as it is for demonstration
> purposes only,
> however, what would be a minimum mergeable PR (related to Rust build)
> that you would consider?

First thing to look at is going to be moving all the code under
bch_bindgen in -tools into the kernel tree...

Re: [PATCH] chore(util): minimal Rust build example

[Wedson Almeida Filho]

On Tue, 9 May 2023 at 15:13, Kent Overstreet <kent.overstreet@linux.dev> wrote:
>
> On Tue, May 09, 2023 at 10:57:18AM -0700, TruongSinh Tran-Nguyen wrote:
> > On Tue, 9 May 2023 at 10:47, Kent Overstreet <kent.overstreet@linux.dev> wrote:
> >
> > > We pretty heavily use to_text() functions kernel side; these output to
> > > printbufs, and in Rust they map nicely to the Display trait - do we have
> > > that available in the kernel or is that in std?
> > >
> >
> > https://doc.rust-lang.org/std/fmt/trait.Display.html Display trait is definitely
> > in Rust `std`. As for the alternative from kernel, I am still searching.
> > Worst case, there is nothing right now, Rust-for-linux folks would
> > happy to take PR
> > to include it.
>
> I included them on the CC, maybe they'll have something to say.

`Display` is in core: https://doc.rust-lang.org/core/fmt/trait.Display.html

Here's an example of it being implemented (and used) in the kernel:
https://elixir.bootlin.com/linux/v6.4-rc1/source/rust/kernel/sync/arc.rs#L604

>
> >
> > > Sounds like that would be a welcome cleanup :)
> >
> > By clean up, do you mean moving from `uuid_le` to `uuid_t`?
> >
> > Also, I don't expect this PR to be merge, as it is for demonstration
> > purposes only,
> > however, what would be a minimum mergeable PR (related to Rust build)
> > that you would consider?
>
> First thing to look at is going to be moving all the code under
> bch_bindgen in -tools into the kernel tree...

Re: [PATCH] chore(util): minimal Rust build example

[TruongSinh Tran-Nguyen]

Wonderful, thanks!



On Tue, 9 May 2023 at 11:42, Wedson Almeida Filho <wedsonaf@gmail.com> wrote:
>
> On Tue, 9 May 2023 at 15:13, Kent Overstreet <kent.overstreet@linux.dev> wrote:
> >
> > On Tue, May 09, 2023 at 10:57:18AM -0700, TruongSinh Tran-Nguyen wrote:
> > > On Tue, 9 May 2023 at 10:47, Kent Overstreet <kent.overstreet@linux.dev> wrote:
> > >
> > > > We pretty heavily use to_text() functions kernel side; these output to
> > > > printbufs, and in Rust they map nicely to the Display trait - do we have
> > > > that available in the kernel or is that in std?
> > > >
> > >
> > > https://doc.rust-lang.org/std/fmt/trait.Display.html Display trait is definitely
> > > in Rust `std`. As for the alternative from kernel, I am still searching.
> > > Worst case, there is nothing right now, Rust-for-linux folks would
> > > happy to take PR
> > > to include it.
> >
> > I included them on the CC, maybe they'll have something to say.
>
> `Display` is in core: https://doc.rust-lang.org/core/fmt/trait.Display.html
>
> Here's an example of it being implemented (and used) in the kernel:
> https://elixir.bootlin.com/linux/v6.4-rc1/source/rust/kernel/sync/arc.rs#L604
>
> >
> > >
> > > > Sounds like that would be a welcome cleanup :)
> > >
> > > By clean up, do you mean moving from `uuid_le` to `uuid_t`?
> > >
> > > Also, I don't expect this PR to be merge, as it is for demonstration
> > > purposes only,
> > > however, what would be a minimum mergeable PR (related to Rust build)
> > > that you would consider?
> >
> > First thing to look at is going to be moving all the code under
> > bch_bindgen in -tools into the kernel tree...

Re: [PATCH] chore(util): minimal Rust build example

[TruongSinh Tran-Nguyen]

Hi Rust for Linux folks,

I have another question. When working on user-space, I'm used to cargo
dependency management, but I assume it's not possible when doing
kernel development. Specifically, for the immediately next PR, I need
to use these 2 crates (and their artifacts)

`
use bitfield::bitfield;
use byteorder::{LittleEndian, ReadBytesExt};
`

Is there something similar in Rust part of kernel already
(https://docs.kernel.org/core-api/packing.html# and
https://man7.org/linux/man-pages/man3/htons.3.html respectively, may
be) ? Or if not, is it normal to just copy-paste the needed code
together our code?

Kind regards,
Sinh

TruongSinh Tran-Nguyen, PMI-ACP, PSM III, PSPO III

Tech Visionary, Craftsman, Coach and Investor

+1-6127-TR-SINH (+1-6127-87-7464) | @truongsinhtn

Find me on LinkedIn | Twitter | Github | StackOverflow

IBAN: LT76 3250 0264 6898 1177 | BIC: REVOLT21

Currently Reading

Get your own email signature




On Tue, 9 May 2023 at 13:57, TruongSinh Tran-Nguyen <i@truongsinh.pro> wrote:
>
> Wonderful, thanks!
>
>
>
> On Tue, 9 May 2023 at 11:42, Wedson Almeida Filho <wedsonaf@gmail.com> wrote:
> >
> > On Tue, 9 May 2023 at 15:13, Kent Overstreet <kent.overstreet@linux.dev> wrote:
> > >
> > > On Tue, May 09, 2023 at 10:57:18AM -0700, TruongSinh Tran-Nguyen wrote:
> > > > On Tue, 9 May 2023 at 10:47, Kent Overstreet <kent.overstreet@linux.dev> wrote:
> > > >
> > > > > We pretty heavily use to_text() functions kernel side; these output to
> > > > > printbufs, and in Rust they map nicely to the Display trait - do we have
> > > > > that available in the kernel or is that in std?
> > > > >
> > > >
> > > > https://doc.rust-lang.org/std/fmt/trait.Display.html Display trait is definitely
> > > > in Rust `std`. As for the alternative from kernel, I am still searching.
> > > > Worst case, there is nothing right now, Rust-for-linux folks would
> > > > happy to take PR
> > > > to include it.
> > >
> > > I included them on the CC, maybe they'll have something to say.
> >
> > `Display` is in core: https://doc.rust-lang.org/core/fmt/trait.Display.html
> >
> > Here's an example of it being implemented (and used) in the kernel:
> > https://elixir.bootlin.com/linux/v6.4-rc1/source/rust/kernel/sync/arc.rs#L604
> >
> > >
> > > >
> > > > > Sounds like that would be a welcome cleanup :)
> > > >
> > > > By clean up, do you mean moving from `uuid_le` to `uuid_t`?
> > > >
> > > > Also, I don't expect this PR to be merge, as it is for demonstration
> > > > purposes only,
> > > > however, what would be a minimum mergeable PR (related to Rust build)
> > > > that you would consider?
> > >
> > > First thing to look at is going to be moving all the code under
> > > bch_bindgen in -tools into the kernel tree...

Re: [PATCH] chore(util): minimal Rust build example

[Miguel Ojeda]

On Wed, May 10, 2023 at 5:30 AM TruongSinh Tran-Nguyen <i@truongsinh.pro> wrote:
>
> I have another question. When working on user-space, I'm used to cargo
> dependency management, but I assume it's not possible when doing
> kernel development. Specifically, for the immediately next PR, I need
> to use these 2 crates (and their artifacts)

Please see https://rust-for-linux.com/third-party-crates (I just uploaded it).

Cheers,
Miguel

Re: [PATCH] chore(util): minimal Rust build example

[TruongSinh Tran-Nguyen]

Thanks, Miguel,

I just read through it, and it makes sense. I did expect this to be
the case, so the problem would go back to our needs of wrapper or
re-written of  https://docs.kernel.org/core-api/packing.html# (for
bitfield::bitfield macro) and
https://man7.org/linux/man-pages/man3/htons.3.html for byteorder
utility. I cannot find them anywhere in:

- https://rust-for-linux.github.io/docs/macros/all.html?search=bitfield
- https://rust-for-linux.github.io/docs/macros/all.html?search=byteorder

So I assume there must be PR for these 2 first before I can move
forward, is that correct?

One more question. Assuming that we have some Linux kernel API
exposing Rust interface (similar to `linux/*.h`), when writing
user-space tool, how do I `use`/`import` them? I have not been able to
find any answer or even good keywords to describe what I mean yet.

Kind regards,
Sinh

TruongSinh Tran-Nguyen, PMI-ACP, PSM III, PSPO III

Tech Visionary, Craftsman, Coach and Investor

+1-6127-TR-SINH (+1-6127-87-7464) | @truongsinhtn

Find me on LinkedIn | Twitter | Github | StackOverflow

IBAN: LT76 3250 0264 6898 1177 | BIC: REVOLT21

Currently Reading

Get your own email signature

Kind regards,
Sinh

TruongSinh Tran-Nguyen, PMI-ACP, PSM III, PSPO III

Tech Visionary, Craftsman, Coach and Investor

+1-6127-TR-SINH (+1-6127-87-7464) | @truongsinhtn

Find me on LinkedIn | Twitter | Github | StackOverflow

IBAN: LT76 3250 0264 6898 1177 | BIC: REVOLT21

Currently Reading

Get your own email signature




On Wed, 10 May 2023 at 10:59, Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> On Wed, May 10, 2023 at 5:30 AM TruongSinh Tran-Nguyen <i@truongsinh.pro> wrote:
> >
> > I have another question. When working on user-space, I'm used to cargo
> > dependency management, but I assume it's not possible when doing
> > kernel development. Specifically, for the immediately next PR, I need
> > to use these 2 crates (and their artifacts)
>
> Please see https://rust-for-linux.com/third-party-crates (I just uploaded it).
>
> Cheers,
> Miguel

Re: [PATCH] chore(util): minimal Rust build example

[Miguel Ojeda]

On Wed, May 10, 2023 at 8:21 PM TruongSinh Tran-Nguyen <i@truongsinh.pro> wrote:
>
> So I assume there must be PR for these 2 first before I can move
> forward, is that correct?

I can look into adding those two to the experiment PR, so that you can
move forward, and so that we can try them out in general (which is the
purpose of the experiment).

It is not a guarantee that they will land in mainline, but it is a
first step to determine whether we want them, or whether we want
something else.

> One more question. Assuming that we have some Linux kernel API
> exposing Rust interface (similar to `linux/*.h`), when writing
> user-space tool, how do I `use`/`import` them? I have not been able to
> find any answer or even good keywords to describe what I mean yet.

If you mean libraries in general that you want to reuse in both
userspace and the kernel, then I think this is the case that Kent
wanted (though as far as I remember from Kangrejos, he wanted to share
the code in crates.io, rather than the kernel tree).

If you mean UAPI in "native" Rust, then we don't provide Rust ones
yet. We may need to wait until Rust is not an experiment anymore
within the kernel to do something like that.

Cheers,
Miguel

Re: [PATCH] chore(util): minimal Rust build example

[TruongSinh Tran-Nguyen]

> If you mean UAPI in "native" Rust, then we don't provide Rust ones
> yet. We may need to wait until Rust is not an experiment anymore
> within the kernel to do something like that.

Yes I mean UAPI in "native" Rust. If its not provided right now, and
there is a kernel module written fully in Rust, then is it recommended
to have it have `repr(c)/[no_mangle]` and use cbindgen to generate
UAPI in C (linux/*.h)?

Kind regards,
Sinh

TruongSinh Tran-Nguyen, PMI-ACP, PSM III, PSPO III

Tech Visionary, Craftsman, Coach and Investor

+1-6127-TR-SINH (+1-6127-87-7464) | @truongsinhtn

Find me on LinkedIn | Twitter | Github | StackOverflow

IBAN: LT76 3250 0264 6898 1177 | BIC: REVOLT21

Currently Reading

Get your own email signature




On Thu, 11 May 2023 at 16:37, Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> On Wed, May 10, 2023 at 8:21 PM TruongSinh Tran-Nguyen <i@truongsinh.pro> wrote:
> >
> > So I assume there must be PR for these 2 first before I can move
> > forward, is that correct?
>
> I can look into adding those two to the experiment PR, so that you can
> move forward, and so that we can try them out in general (which is the
> purpose of the experiment).
>
> It is not a guarantee that they will land in mainline, but it is a
> first step to determine whether we want them, or whether we want
> something else.
>
> > One more question. Assuming that we have some Linux kernel API
> > exposing Rust interface (similar to `linux/*.h`), when writing
> > user-space tool, how do I `use`/`import` them? I have not been able to
> > find any answer or even good keywords to describe what I mean yet.
>
> If you mean libraries in general that you want to reuse in both
> userspace and the kernel, then I think this is the case that Kent
> wanted (though as far as I remember from Kangrejos, he wanted to share
> the code in crates.io, rather than the kernel tree).
>
> If you mean UAPI in "native" Rust, then we don't provide Rust ones
> yet. We may need to wait until Rust is not an experiment anymore
> within the kernel to do something like that.
>
> Cheers,
> Miguel

Re: [PATCH] chore(util): minimal Rust build example

[Miguel Ojeda]

On Fri, May 12, 2023 at 1:54 AM TruongSinh Tran-Nguyen <i@truongsinh.pro> wrote:
>
> Yes I mean UAPI in "native" Rust. If its not provided right now, and
> there is a kernel module written fully in Rust, then is it recommended
> to have it have `repr(c)/[no_mangle]` and use cbindgen to generate
> UAPI in C (linux/*.h)?

We would need to see how to do it when the need comes. Are you
planning to upstream your code soon? Could you please show the Rust
code you need to provide an UAPI for?

Cheers,
Miguel

Re: [PATCH] chore(util): minimal Rust build example

[Kent Overstreet]

On Thu, May 11, 2023 at 04:54:34PM -0700, TruongSinh Tran-Nguyen wrote:
> > If you mean UAPI in "native" Rust, then we don't provide Rust ones
> > yet. We may need to wait until Rust is not an experiment anymore
> > within the kernel to do something like that.
> 
> Yes I mean UAPI in "native" Rust. If its not provided right now, and
> there is a kernel module written fully in Rust, then is it recommended
> to have it have `repr(c)/[no_mangle]` and use cbindgen to generate
> UAPI in C (linux/*.h)?

This doesn't sound like a uapi question, it sounds more like - how do we
expose C interfaces from Rust?

That's a good question. Do we have a sort of reverse-bindgen, that can
emit a C header file for repr(c) functions?

Re: [PATCH] chore(util): minimal Rust build example

[TruongSinh Tran-Nguyen]

> This doesn't sound like a uapi question, it sounds more like - how do we
> expose C interfaces from Rust?

No, I do mean UAPI, as exposing C interfaces from Rust (to be consumed
by C code) is known. I'm interested in exposing Rust API (from kernel)
to be consumed by user-space Rust code.

> That's a good question. Do we have a sort of reverse-bindgen, that can
> emit a C header file for repr(c) functions?

There is, see https://docs.rust-embedded.org/book/interoperability/rust-with-c.html
which recommends https://github.com/mozilla/cbindgen

Re: [PATCH] chore(util): minimal Rust build example

[Kent Overstreet]

On Thu, May 11, 2023 at 09:16:19PM -0700, TruongSinh Tran-Nguyen wrote:
> > This doesn't sound like a uapi question, it sounds more like - how do we
> > expose C interfaces from Rust?
> 
> No, I do mean UAPI, as exposing C interfaces from Rust (to be consumed
> by C code) is known. I'm interested in exposing Rust API (from kernel)
> to be consumed by user-space Rust code.

What's your use case?

Re: [PATCH] chore(util): minimal Rust build example

[TruongSinh Tran-Nguyen]

> What's your use case?

https://github.com/koverstreet/bcachefs-tools/blob/master/cmd_migrate.c
for example, includes several syscall and kernel uAPIs. Assuming in
the far future,
some/all of the APIs are written in Rust. It is then unnecessary for
user-space Rust code
to bindgen from a header files, which in turn are cbindgen-ed from Rust code,
and wrap them in `unsafe` block. It would be much more straight
forward just directly
use Rust API.

That being said, I'm interested in Rust uAPI in general, not
necessarily in the scope of
bcachefs only.

Re: [PATCH] chore(util): minimal Rust build example

[Kent Overstreet]

On Thu, May 11, 2023 at 09:39:34PM -0700, TruongSinh Tran-Nguyen wrote:
> > What's your use case?
> 
> https://github.com/koverstreet/bcachefs-tools/blob/master/cmd_migrate.c
> for example, includes several syscall and kernel uAPIs. Assuming in
> the far future,
> some/all of the APIs are written in Rust.

This seems like something for quite a long ways off, unless I'm
misunderstanding what you're trying to do right now?

Re: [PATCH] chore(util): minimal Rust build example

[TruongSinh Tran-Nguyen]

> This seems like something for quite a long ways off, unless I'm
> misunderstanding what you're trying to do right now?

Correct, it is not my immediate need (neither for bcachefs nor other
user-space tools), just want to look out for the future. There is no
kernel module written fully in Rust yet anyway.

Re: [PATCH] chore(util): minimal Rust build example

[Kent Overstreet]

On Wed, May 10, 2023 at 07:59:03PM +0200, Miguel Ojeda wrote:
> On Wed, May 10, 2023 at 5:30 AM TruongSinh Tran-Nguyen <i@truongsinh.pro> wrote:
> >
> > I have another question. When working on user-space, I'm used to cargo
> > dependency management, but I assume it's not possible when doing
> > kernel development. Specifically, for the immediately next PR, I need
> > to use these 2 crates (and their artifacts)
> 
> Please see https://rust-for-linux.com/third-party-crates (I just uploaded it).

Has cargo vendor been getting discussed?

Re: [PATCH] chore(util): minimal Rust build example

[Miguel Ojeda]

On Wed, May 10, 2023 at 8:38 PM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> Has cargo vendor been getting discussed?

If you mean for dynamically fetching things, yeah, that is an option
(I think we discussed it in Kangrejos?); but the implementation
specifics are not too important: what we need is getting the agreement
from kernel maintainers on dynamically fetching them.

Cheers,
Miguel

Re: [PATCH] chore(util): minimal Rust build example

[Björn Roy Baron]

On Wednesday, May 10th, 2023 at 05:10, TruongSinh Tran-Nguyen <i@truongsinh.pro> wrote:

> Hi Rust for Linux folks,
> 
> I have another question. When working on user-space, I'm used to cargo
> dependency management, but I assume it's not possible when doing
> kernel development. Specifically, for the immediately next PR, I need
> to use these 2 crates (and their artifacts)
> 
> `use bitfield::bitfield; use byteorder::{LittleEndian, ReadBytesExt};`
> 
> Is there something similar in Rust part of kernel already
> (https://docs.kernel.org/core-api/packing.html# and
> https://man7.org/linux/man-pages/man3/htons.3.html respectively, may
> be) ? Or if not, is it normal to just copy-paste the needed code
> together our code?
> 
> Kind regards,
> Sinh
> 

For byteorder you can use the builtin {to,from}_{le,be,ne}_bytes and {from,to}_{be,le} methods. The former return and accepts a [u8; N] array where N is the size in bytes of the integer you called it on. The _be variants of the later are equivalent to the htons family.

For bitfields if you have a struct definition with bitfields in a C header already, bindgen should generate a type with getters and setters for the individual fields as well as a constructor. See https://rust-lang.github.io/rust-bindgen/using-bitfields.html If the type isn't defined on the C side I guess you could define it there as helper, but having a rust macro for this like the bitfield crate provides is of course nicer.

Cheers,
Bjorn

Re: [PATCH] chore(util): minimal Rust build example

[TruongSinh Tran-Nguyen]

Hi Björn,

Thanks for the help. About bitfields, we might consider using bindgen
for now. However, in the future, if we have a rust-only module (no C
code), it would no longer be an option. If I understand correctly,
there is currently no similar macro like the bitfield crate provides,
and thus one of the other options (if the license permits), is to copy
the macros code directly to the module. Is that correct?

Kind regards,
Sinh

TruongSinh Tran-Nguyen, PMI-ACP, PSM III, PSPO III

Tech Visionary, Craftsman, Coach and Investor

+1-6127-TR-SINH (+1-6127-87-7464) | @truongsinhtn

Find me on LinkedIn | Twitter | Github | StackOverflow

IBAN: LT76 3250 0264 6898 1177 | BIC: REVOLT21

Currently Reading

Get your own email signature

Kind regards,
Sinh

TruongSinh Tran-Nguyen, PMI-ACP, PSM III, PSPO III

Tech Visionary, Craftsman, Coach and Investor

+1-6127-TR-SINH (+1-6127-87-7464) | @truongsinhtn

Find me on LinkedIn | Twitter | Github | StackOverflow

IBAN: LT76 3250 0264 6898 1177 | BIC: REVOLT21

Currently Reading

Get your own email signature




On Thu, 11 May 2023 at 09:16, Björn Roy Baron <bjorn3_gh@protonmail.com> wrote:
>
> On Wednesday, May 10th, 2023 at 05:10, TruongSinh Tran-Nguyen <i@truongsinh.pro> wrote:
>
> > Hi Rust for Linux folks,
> >
> > I have another question. When working on user-space, I'm used to cargo
> > dependency management, but I assume it's not possible when doing
> > kernel development. Specifically, for the immediately next PR, I need
> > to use these 2 crates (and their artifacts)
> >
> > `use bitfield::bitfield; use byteorder::{LittleEndian, ReadBytesExt};`
> >
> > Is there something similar in Rust part of kernel already
> > (https://docs.kernel.org/core-api/packing.html# and
> > https://man7.org/linux/man-pages/man3/htons.3.html respectively, may
> > be) ? Or if not, is it normal to just copy-paste the needed code
> > together our code?
> >
> > Kind regards,
> > Sinh
> >
>
> For byteorder you can use the builtin {to,from}_{le,be,ne}_bytes and {from,to}_{be,le} methods. The former return and accepts a [u8; N] array where N is the size in bytes of the integer you called it on. The _be variants of the later are equivalent to the htons family.
>
> For bitfields if you have a struct definition with bitfields in a C header already, bindgen should generate a type with getters and setters for the individual fields as well as a constructor. See https://rust-lang.github.io/rust-bindgen/using-bitfields.html If the type isn't defined on the C side I guess you could define it there as helper, but having a rust macro for this like the bitfield crate provides is of course nicer.
>
> Cheers,
> Bjorn

Re: [PATCH] chore(util): minimal Rust build example

[Björn Roy Baron]

On Thursday, May 11th, 2023 at 18:46, TruongSinh Tran-Nguyen <i@truongsinh.pro> wrote:

> Hi Björn,
> 
> Thanks for the help. About bitfields, we might consider using bindgen
> for now. However, in the future, if we have a rust-only module (no C
> code), it would no longer be an option. If I understand correctly,
> there is currently no similar macro like the bitfield crate provides,

Correct

> and thus one of the other options (if the license permits), is to copy
> the macros code directly to the module. Is that correct?

I think you will have to vendor the bitfield crate into a separate file and add an SPDX-License-Identifier header with the license of the crate. Given how small this specific crate is, I think it would be ok to add it to the kernel crate. If it was bigger it would likely have to be a separate crate.

> 
> Kind regards,
> Sinh

Cheers,
Bjorn

Re: [PATCH] chore(util): minimal Rust build example

[TruongSinh Tran-Nguyen]

> I think you will have to vendor the bitfield crate into a separate file
> and add an SPDX-License-Identifier header with the license of the crate

I just send and patch to rust-for-linux@vger.kernel.org to include
bitfield macro into kernel crate.

Kind regards,
Sinh

TruongSinh Tran-Nguyen, PMI-ACP, PSM III, PSPO III

Tech Visionary, Craftsman, Coach and Investor

+1-6127-TR-SINH (+1-6127-87-7464) | @truongsinhtn

Find me on LinkedIn | Twitter | Github | StackOverflow

IBAN: LT76 3250 0264 6898 1177 | BIC: REVOLT21

Currently Reading

Re: [PATCH] chore(util): minimal Rust build example

[Andreas Hindborg]

TruongSinh Tran-Nguyen <i@truongsinh.pro> writes:

> On Tue, 9 May 2023 at 10:47, Kent Overstreet <kent.overstreet@linux.dev> wrote:
>
>> We pretty heavily use to_text() functions kernel side; these output to
>> printbufs, and in Rust they map nicely to the Display trait - do we have
>> that available in the kernel or is that in std?
>>
>
> https://doc.rust-lang.org/std/fmt/trait.Display.html Display trait is definitely
> in Rust `std`. As for the alternative from kernel, I am still searching.
> Worst case, there is nothing right now, Rust-for-linux folks would
> happy to take PR
> to include it.

Many traits and types in `std` are reexports from `core` or `alloc`.
To my knowledge, there is no way to see if an item is reexported when
browsing the documentation. Some ways discover reexports when browsing docs:

 - hit the `source` link at see where it is going
 - when searching for an item, see if it is present in multiple crates.

Best regards
Andreas

Re: [PATCH] chore(util): minimal Rust build example

[Miguel Ojeda]

On Tue, May 9, 2023 at 8:07 PM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> So if I'm following, it looks like everything is basically there. It
> looks like we can add Rust source files to fs/bcachefs/ and have them
> built as part of the bcachefs module, and calling back and forth just
> works as expected - nice.

Indeed, that should work, and it may be useful, though typically you
will want to avoid crossing the C/Rust boundary all the time, so that
you can work with Rust types as much as possible.

For modules/drivers, we provide Rust abstractions for the C APIs so
that one can write drivers in Rust (and thus use Rust facilities and
avoid `unsafe` as much as possible), rather than mixing C and Rust
source code in the "leaves". Perhaps there are some parts within
bcachefs amenable to a similar approach.

> The kbuild stuff looks nice and clean - congrats to those responsible :)

Thanks for the kind words, though a rework of how that works is
planned, so hopefully you will also like the new way too... :)

Cheers,
Miguel

Re: [PATCH] chore(util): minimal Rust build example

[Kent Overstreet]

On Wed, May 10, 2023 at 07:56:07PM +0200, Miguel Ojeda wrote:
> On Tue, May 9, 2023 at 8:07 PM Kent Overstreet
> <kent.overstreet@linux.dev> wrote:
> >
> > So if I'm following, it looks like everything is basically there. It
> > looks like we can add Rust source files to fs/bcachefs/ and have them
> > built as part of the bcachefs module, and calling back and forth just
> > works as expected - nice.
> 
> Indeed, that should work, and it may be useful, though typically you
> will want to avoid crossing the C/Rust boundary all the time, so that
> you can work with Rust types as much as possible.
> 
> For modules/drivers, we provide Rust abstractions for the C APIs so
> that one can write drivers in Rust (and thus use Rust facilities and
> avoid `unsafe` as much as possible), rather than mixing C and Rust
> source code in the "leaves". Perhaps there are some parts within
> bcachefs amenable to a similar approach.

I've been doing the same thing.

Right now in bcachefs-tools we've got two different crates, one for
bcachefs rust code proper and another for bindings, with a build.rs for
bindgen.

Any pointers to how we'll be interacting with bindgen kernel side? We
need to pass it some stuff, see
https://evilpiepirate.org/git/bcachefs-tools.git/tree/rust-src/bch_bindgen/build.rs

> 
> > The kbuild stuff looks nice and clean - congrats to those responsible :)
> 
> Thanks for the kind words, though a rework of how that works is
> planned, so hopefully you will also like the new way too... :)

Can't wait... :)

Re: [PATCH] chore(util): minimal Rust build example

[Miguel Ojeda]

On Wed, May 10, 2023 at 8:45 PM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> I've been doing the same thing.
>
> Right now in bcachefs-tools we've got two different crates, one for
> bcachefs rust code proper and another for bindings, with a build.rs for
> bindgen.

Sounds great. (I was mentioning it due to the patch above, mostly,
i.e. I wouldn't recommend reimplementing a single function and writing
the bindings manually etc. -- the patch also was not follow either the
C or Rust styles).

> Any pointers to how we'll be interacting with bindgen kernel side? We
> need to pass it some stuff, see
> https://evilpiepirate.org/git/bcachefs-tools.git/tree/rust-src/bch_bindgen/build.rs

Currently we put the `#include`s in `rust/bindings/bindings_helper.h`,
and options if needed (e.g. allows/blocks like you also have) in
`rust/bindgen_parameters`.

When the `kernel` crate split happens, we may have
per-crate/per-subsystem/... configuration for this, too.

Cheers,
Miguel