[PATCH v2 0/2] rust: fix rust-analyzer configuration for generated files

[PATCH v2 0/2] rust: fix rust-analyzer configuration for generated files

[Tamir Duberstein]

The individual patches should be descriptive on their own. They are
included in a single series because the second patch uses a function
introduced in the first.

I've confirmed this allows me to navigate to symbols defined in
generated files as well as to the generated files themselves. I am using
an out-of-source build.

Signed-off-by: Tamir Duberstein <tamird@gmail.com>
---
Changes in v2:
- Add missing core dependency on uapi. Fixes navigation in
  rust/kernel/ioctl.rs.
- Link to v1: https://lore.kernel.org/r/20250210-rust-analyzer-bindings-include-v1-0-6cbf420e95b6@gmail.com

---
Tamir Duberstein (2):
      scripts: generate_rust_analyzer.py: add missing include_dirs
      scripts: generate_rust_analyzer.py: add uapi crate

 scripts/generate_rust_analyzer.py | 41 ++++++++++++++++++++-------------------
 1 file changed, 21 insertions(+), 20 deletions(-)
---
base-commit: a64dcfb451e254085a7daee5fe51bf22959d52d3
change-id: 20250210-rust-analyzer-bindings-include-531e9dacec8d

Best regards,
-- 
Tamir Duberstein <tamird@gmail.com>

[PATCH v2 1/2] scripts: generate_rust_analyzer.py: add missing include_dirs

[Tamir Duberstein]

Commit 8c4555ccc55c ("scripts: add `generate_rust_analyzer.py`")
specified OBJTREE for the bindings crate, and `source.include_dirs` for
the kernel crate, likely in an attempt to support out-of-source builds
for those crates where the generated files reside in `objtree` rather
than `srctree`. This was insufficient because both bits of configuration
are required for each crate; the result is that rust-analyzer is unable
to resolve generated files for either crate in an out-of-source build.

Add the missing bits to improve the developer experience.

Fixes: 8c4555ccc55c ("scripts: add `generate_rust_analyzer.py`")
Signed-off-by: Tamir Duberstein <tamird@gmail.com>
---
 scripts/generate_rust_analyzer.py | 40 +++++++++++++++++++--------------------
 1 file changed, 20 insertions(+), 20 deletions(-)

diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
index aa8ea1a4dbe5..1f573d19cd99 100755
--- a/scripts/generate_rust_analyzer.py
+++ b/scripts/generate_rust_analyzer.py
@@ -85,27 +85,27 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs):
         ["core", "compiler_builtins"],
     )
 
-    append_crate(
-        "bindings",
-        srctree / "rust"/ "bindings" / "lib.rs",
-        ["core"],
-        cfg=cfg,
-    )
-    crates[-1]["env"]["OBJTREE"] = str(objtree.resolve(True))
+    def append_crate_with_generated(
+        display_name,
+        deps,
+    ):
+        append_crate(
+            display_name,
+            srctree / "rust"/ display_name / "lib.rs",
+            deps,
+            cfg=cfg,
+        )
+        crates[-1]["env"]["OBJTREE"] = str(objtree.resolve(True))
+        crates[-1]["source"] = {
+            "include_dirs": [
+                str(srctree / "rust" / display_name),
+                str(objtree / "rust")
+            ],
+            "exclude_dirs": [],
+        }
 
-    append_crate(
-        "kernel",
-        srctree / "rust" / "kernel" / "lib.rs",
-        ["core", "macros", "build_error", "bindings"],
-        cfg=cfg,
-    )
-    crates[-1]["source"] = {
-        "include_dirs": [
-            str(srctree / "rust" / "kernel"),
-            str(objtree / "rust")
-        ],
-        "exclude_dirs": [],
-    }
+    append_crate_with_generated("bindings", ["core"])
+    append_crate_with_generated("kernel", ["core", "macros", "build_error", "bindings"])
 
     def is_root_crate(build_file, target):
         try:

-- 
2.48.1

Re: [PATCH v2 1/2] scripts: generate_rust_analyzer.py: add missing include_dirs

[Miguel Ojeda]

On Mon, Feb 10, 2025 at 7:04 PM Tamir Duberstein <tamird@gmail.com> wrote:
>
> Commit 8c4555ccc55c ("scripts: add `generate_rust_analyzer.py`")
> specified OBJTREE for the bindings crate, and `source.include_dirs` for
> the kernel crate, likely in an attempt to support out-of-source builds
> for those crates where the generated files reside in `objtree` rather
> than `srctree`. This was insufficient because both bits of configuration
> are required for each crate; the result is that rust-analyzer is unable
> to resolve generated files for either crate in an out-of-source build.

Originally we were not using `OBJTREE` in the `kernel` crate, but we
did pass it anyway, so conceptually it could have been there. So I am
not sure if it counts as a fix for that commit, but it shouldn't hurt
even if backported.

Regarding `include_dirs`, it started in `kernel` before being in
mainline because we included the bindings there (i.e. there was not
`bindings` crate), but it should have been probably moved when it was
split. Nowadays, I guess we still need it for
`generated_arch_static_branch_asm.rs`, or is it something else that
needs it? I assume it shouldn't hurt, in any case, so it looks OK.

Cheers,
Miguel

Re: [PATCH v2 1/2] scripts: generate_rust_analyzer.py: add missing include_dirs

[Tamir Duberstein]

On Tue, Mar 11, 2025 at 6:00 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> On Mon, Feb 10, 2025 at 7:04 PM Tamir Duberstein <tamird@gmail.com> wrote:
> >
> > Commit 8c4555ccc55c ("scripts: add `generate_rust_analyzer.py`")
> > specified OBJTREE for the bindings crate, and `source.include_dirs` for
> > the kernel crate, likely in an attempt to support out-of-source builds
> > for those crates where the generated files reside in `objtree` rather
> > than `srctree`. This was insufficient because both bits of configuration
> > are required for each crate; the result is that rust-analyzer is unable
> > to resolve generated files for either crate in an out-of-source build.
>
> Originally we were not using `OBJTREE` in the `kernel` crate, but we
> did pass it anyway, so conceptually it could have been there. So I am
> not sure if it counts as a fix for that commit, but it shouldn't hurt
> even if backported.

Ah, I see. The reference to `OBJTREE` in the `kernel` crate was
introduced in commit 169484ab6677 ("rust: add arch_static_branch").
I'm not able to build at commit 8c4555ccc55c ("scripts: add
`generate_rust_analyzer.py`") but I would expect that rust-analyzer
didn't work properly there for the bindings crate for out of tree
builds because of the missing `include_dirs`.

>
> Regarding `include_dirs`, it started in `kernel` before being in
> mainline because we included the bindings there (i.e. there was not
> `bindings` crate),

I don't follow this; there's a bindings crate at 8c4555ccc55c
("scripts: add `generate_rust_analyzer.py`") - at least as far as RA
is concerned.

but it should have been probably moved when it was
> split. Nowadays, I guess we still need it for
> `generated_arch_static_branch_asm.rs`, or is it something else that
> needs it? I assume it shouldn't hurt, in any case, so it looks OK.

Yes, without it `generated_arch_static_branch_asm.rs` is not found by RA.

Re: [PATCH v2 1/2] scripts: generate_rust_analyzer.py: add missing include_dirs

[Miguel Ojeda]

On Tue, Mar 11, 2025 at 11:48 PM Tamir Duberstein <tamird@gmail.com> wrote:
>
> I don't follow this; there's a bindings crate at 8c4555ccc55c
> ("scripts: add `generate_rust_analyzer.py`") - at least as far as RA
> is concerned.

By "before being in mainline" I mean before Rust was in Linus' tree,
i.e. a long time ago in the old `rust` branch.

Back then, there was a period of time when there was no `bindings`
crate. At some point it was split from the `kernel` crate, which did
have the `include_dirs` for that reason, and at that moment the
`include_dirs` should have been moved too, but it wasn't.

So we ended up with `include_dirs` in the `kernel` crate for no reason
(because AFAICS it was not needed until we got
`generated_arch_static_branch_asm.rs`, but I haven't actually tested
that in RA) and missing it in the `bindings` one where it was actually
needed.

(I was trying to understand how we ended up here just to double-check
why we had things split like that)

> Yes, without it `generated_arch_static_branch_asm.rs` is not found by RA.

Thanks for confirming!

Cheers,
Miguel

Re: [PATCH v2 1/2] scripts: generate_rust_analyzer.py: add missing include_dirs

[Tamir Duberstein]

\On Tue, Mar 11, 2025 at 7:01 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> On Tue, Mar 11, 2025 at 11:48 PM Tamir Duberstein <tamird@gmail.com> wrote:
> >
> > I don't follow this; there's a bindings crate at 8c4555ccc55c
> > ("scripts: add `generate_rust_analyzer.py`") - at least as far as RA
> > is concerned.
>
> By "before being in mainline" I mean before Rust was in Linus' tree,
> i.e. a long time ago in the old `rust` branch.
>
> Back then, there was a period of time when there was no `bindings`
> crate. At some point it was split from the `kernel` crate, which did
> have the `include_dirs` for that reason, and at that moment the
> `include_dirs` should have been moved too, but it wasn't.
>
> So we ended up with `include_dirs` in the `kernel` crate for no reason
> (because AFAICS it was not needed until we got
> `generated_arch_static_branch_asm.rs`, but I haven't actually tested
> that in RA) and missing it in the `bindings` one where it was actually
> needed.
>
> (I was trying to understand how we ended up here just to double-check
> why we had things split like that)

Ack, thanks for explaining.

> > Yes, without it `generated_arch_static_branch_asm.rs` is not found by RA.
>
> Thanks for confirming!

You're welcome!

[PATCH v2 2/2] scripts: generate_rust_analyzer.py: add uapi crate

[Tamir Duberstein]

Commit 4e1746656839 ("rust: uapi: Add UAPI crate") did not update
rust-analyzer to include the new crate.

Add the missing definition to improve the developer experience.

Fixes: 4e1746656839 ("rust: uapi: Add UAPI crate")
Signed-off-by: Tamir Duberstein <tamird@gmail.com>
---
 scripts/generate_rust_analyzer.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
index 1f573d19cd99..d3fe091a55dd 100755
--- a/scripts/generate_rust_analyzer.py
+++ b/scripts/generate_rust_analyzer.py
@@ -105,7 +105,8 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs):
         }
 
     append_crate_with_generated("bindings", ["core"])
-    append_crate_with_generated("kernel", ["core", "macros", "build_error", "bindings"])
+    append_crate_with_generated("uapi", ["core"])
+    append_crate_with_generated("kernel", ["core", "macros", "build_error", "bindings", "uapi"])
 
     def is_root_crate(build_file, target):
         try:

-- 
2.48.1

Re: [PATCH v2 0/2] rust: fix rust-analyzer configuration for generated files

[Andreas Hindborg]

"Tamir Duberstein" <tamird@gmail.com> writes:

> The individual patches should be descriptive on their own. They are
> included in a single series because the second patch uses a function
> introduced in the first.
>
> I've confirmed this allows me to navigate to symbols defined in
> generated files as well as to the generated files themselves. I am using
> an out-of-source build.
>
> Signed-off-by: Tamir Duberstein <tamird@gmail.com>

This is amazing, I have been missing this for so long 😆 With this patch
I get auto completion of symbols in `bindings` and `uapi`, without it,
no completions. I build out of tree.

For the whole series:

Tested-by: Andreas Hindborg <a.hindborg@kernel.org>


Best regards,
Andreas Hindborg

Re: [PATCH v2 0/2] rust: fix rust-analyzer configuration for generated files

[Tamir Duberstein]

On Tue, Feb 11, 2025 at 10:21 AM Andreas Hindborg <a.hindborg@kernel.org> wrote:
>
> "Tamir Duberstein" <tamird@gmail.com> writes:
>
> > The individual patches should be descriptive on their own. They are
> > included in a single series because the second patch uses a function
> > introduced in the first.
> >
> > I've confirmed this allows me to navigate to symbols defined in
> > generated files as well as to the generated files themselves. I am using
> > an out-of-source build.
> >
> > Signed-off-by: Tamir Duberstein <tamird@gmail.com>

Hi folks, gentle ping here.

> This is amazing, I have been missing this for so long 😆 With this patch
> I get auto completion of symbols in `bindings` and `uapi`, without it,
> no completions. I build out of tree.
>
> For the whole series:
>
> Tested-by: Andreas Hindborg <a.hindborg@kernel.org>

Thanks for testing!

Re: [PATCH v2 0/2] rust: fix rust-analyzer configuration for generated files

[Miguel Ojeda]

On Mon, Feb 10, 2025 at 7:04 PM Tamir Duberstein <tamird@gmail.com> wrote:
>
> The individual patches should be descriptive on their own. They are
> included in a single series because the second patch uses a function
> introduced in the first.
>
> I've confirmed this allows me to navigate to symbols defined in
> generated files as well as to the generated files themselves. I am using
> an out-of-source build.
>
> Signed-off-by: Tamir Duberstein <tamird@gmail.com>

Applied to `rust-fixes` -- thanks everyone!

      [ Originally we were not using `OBJTREE` in the `kernel` crate, but
        we did pass the variable anyway, so conceptually it could have been
        there since then.

        Regarding `include_dirs`, it started in `kernel` before being in
        mainline because we included the bindings directly there (i.e.
        there was no `bindings` crate). However, when that crate got
        created, we moved the `OBJTREE` there but not the `include_dirs`.
        Nowadays, though, we happen to need the `include_dirs` also in
        the `kernel` crate for `generated_arch_static_branch_asm.rs` which
        was not there back then -- Tamir confirms it is indeed required
        for that reason. - Miguel ]

    [ Slightly reworded title. - Miguel ]

    [ Slightly reworded title. - Miguel ]

Cheers,
Miguel