Re: [PATCH] rust: fix regexp in scripts/is_rust_module.sh

Re: [PATCH] rust: fix regexp in scripts/is_rust_module.sh

[Martin Rodriguez Reboredo]

On Mon, Feb 13, 2023 at 1:19 PM Miguel Ojeda <ojeda@kernel.org> wrote:
> On Fri, Feb 10, 2023 at 4:26 PM Andrea Righi <andrea.righi@canonical.com> wrote:
> >
> > nm can use "R" or "r" to show read-only data sections, but
> > scripts/is_rust_module.sh can only recognize "r", so with some versions
> > of binutils it can fail to detect if a module is a Rust module or not.
>
> Do you know which versions? If so, it would be nice to document it here.
>
> > Moreover, with this patch applied I can also relax the constraint of
> > "RUST depends on !DEBUG_INFO_BTF" and build a kernel with Rust and BTF
> > enabled at the same time (of course BTF generation is still skipped for
> > Rust modules).
>
> Even if that build succeeds, can you load the modules? i.e. the
> constraint was there due to
> https://github.com/Rust-for-Linux/linux/issues/735.

Issue was that the kernel couldn't even load C modules, Rust CUs must be
skipped in vmlinux for it to work, and because of that the constraint
!DEBUG_INFO_BTF || PAHOLE_HAS_LANG_EXCLUDE [1] cannot be relaxed until
version 1.24+ of Pahole is the minimum requirement by the kernel. I'm OK
with this change but keep in mind that it won't remedy that condition.

Link: https://lore.kernel.org/bpf/20230111152050.559334-1-yakoyoku@gmail.com/ [1]

Re: [PATCH] rust: fix regexp in scripts/is_rust_module.sh

[Andrea Righi]

On Mon, Feb 13, 2023 at 12:13:39PM -0300, Martin Rodriguez Reboredo wrote:
> On Mon, Feb 13, 2023 at 1:19 PM Miguel Ojeda <ojeda@kernel.org> wrote:
> > On Fri, Feb 10, 2023 at 4:26 PM Andrea Righi <andrea.righi@canonical.com> wrote:
> > >
> > > nm can use "R" or "r" to show read-only data sections, but
> > > scripts/is_rust_module.sh can only recognize "r", so with some versions
> > > of binutils it can fail to detect if a module is a Rust module or not.
> >
> > Do you know which versions? If so, it would be nice to document it here.
> >
> > > Moreover, with this patch applied I can also relax the constraint of
> > > "RUST depends on !DEBUG_INFO_BTF" and build a kernel with Rust and BTF
> > > enabled at the same time (of course BTF generation is still skipped for
> > > Rust modules).
> >
> > Even if that build succeeds, can you load the modules? i.e. the
> > constraint was there due to
> > https://github.com/Rust-for-Linux/linux/issues/735.
> 
> Issue was that the kernel couldn't even load C modules, Rust CUs must be
> skipped in vmlinux for it to work, and because of that the constraint
> !DEBUG_INFO_BTF || PAHOLE_HAS_LANG_EXCLUDE [1] cannot be relaxed until
> version 1.24+ of Pahole is the minimum requirement by the kernel. I'm OK
> with this change but keep in mind that it won't remedy that condition.
> 
> Link: https://lore.kernel.org/bpf/20230111152050.559334-1-yakoyoku@gmail.com/ [1]

Oh yes, I'm using pahole 1.24, that's probably why it's all working for
me. So we can't just drop !DEBUG_INFO_BTF, but we can definitely relax
the constraint to "!DEBUG_INFO_BTF || PAHOLE_HAS_LANG_EXCLUDE".

-Andrea

Re: [PATCH] rust: fix regexp in scripts/is_rust_module.sh

[Eric Curtin]

On Mon, 13 Feb 2023 at 15:48, Andrea Righi <andrea.righi@canonical.com> wrote:
>
> On Mon, Feb 13, 2023 at 12:13:39PM -0300, Martin Rodriguez Reboredo wrote:
> > On Mon, Feb 13, 2023 at 1:19 PM Miguel Ojeda <ojeda@kernel.org> wrote:
> > > On Fri, Feb 10, 2023 at 4:26 PM Andrea Righi <andrea.righi@canonical.com> wrote:
> > > >
> > > > nm can use "R" or "r" to show read-only data sections, but
> > > > scripts/is_rust_module.sh can only recognize "r", so with some versions
> > > > of binutils it can fail to detect if a module is a Rust module or not.
> > >
> > > Do you know which versions? If so, it would be nice to document it here.
> > >
> > > > Moreover, with this patch applied I can also relax the constraint of
> > > > "RUST depends on !DEBUG_INFO_BTF" and build a kernel with Rust and BTF
> > > > enabled at the same time (of course BTF generation is still skipped for
> > > > Rust modules).
> > >
> > > Even if that build succeeds, can you load the modules? i.e. the
> > > constraint was there due to
> > > https://github.com/Rust-for-Linux/linux/issues/735.
> >
> > Issue was that the kernel couldn't even load C modules, Rust CUs must be
> > skipped in vmlinux for it to work, and because of that the constraint
> > !DEBUG_INFO_BTF || PAHOLE_HAS_LANG_EXCLUDE [1] cannot be relaxed until
> > version 1.24+ of Pahole is the minimum requirement by the kernel. I'm OK
> > with this change but keep in mind that it won't remedy that condition.
> >
> > Link: https://lore.kernel.org/bpf/20230111152050.559334-1-yakoyoku@gmail.com/ [1]
>
> Oh yes, I'm using pahole 1.24, that's probably why it's all working for
> me. So we can't just drop !DEBUG_INFO_BTF, but we can definitely relax
> the constraint to "!DEBUG_INFO_BTF || PAHOLE_HAS_LANG_EXCLUDE".

Yup the patch is ok, but we can only build with RUST and
DEBUG_INFO_BTF if we have pahole 1.24+ because that is the version
smart enough to exclude the Rust CUs.

If you have a version of pahole < 1.24 and you want to include RUST
you basically should not have DEBUG_INFO_BTF on at all because your
build will fail (unless this change alters the list of CUs passed to
pahole).

What is a little confusing is scripts/is_rust_module.sh and pahole
"--lang_exclude=rust" feature both attempt to do the same thing,
recognize and skip Rust CUs

>
> -Andrea
>

Re: [PATCH] rust: fix regexp in scripts/is_rust_module.sh

[Andrea Righi]

On Mon, Feb 13, 2023 at 04:03:46PM +0000, Eric Curtin wrote:
> On Mon, 13 Feb 2023 at 15:48, Andrea Righi <andrea.righi@canonical.com> wrote:
> >
> > On Mon, Feb 13, 2023 at 12:13:39PM -0300, Martin Rodriguez Reboredo wrote:
> > > On Mon, Feb 13, 2023 at 1:19 PM Miguel Ojeda <ojeda@kernel.org> wrote:
> > > > On Fri, Feb 10, 2023 at 4:26 PM Andrea Righi <andrea.righi@canonical.com> wrote:
> > > > >
> > > > > nm can use "R" or "r" to show read-only data sections, but
> > > > > scripts/is_rust_module.sh can only recognize "r", so with some versions
> > > > > of binutils it can fail to detect if a module is a Rust module or not.
> > > >
> > > > Do you know which versions? If so, it would be nice to document it here.
> > > >
> > > > > Moreover, with this patch applied I can also relax the constraint of
> > > > > "RUST depends on !DEBUG_INFO_BTF" and build a kernel with Rust and BTF
> > > > > enabled at the same time (of course BTF generation is still skipped for
> > > > > Rust modules).
> > > >
> > > > Even if that build succeeds, can you load the modules? i.e. the
> > > > constraint was there due to
> > > > https://github.com/Rust-for-Linux/linux/issues/735.
> > >
> > > Issue was that the kernel couldn't even load C modules, Rust CUs must be
> > > skipped in vmlinux for it to work, and because of that the constraint
> > > !DEBUG_INFO_BTF || PAHOLE_HAS_LANG_EXCLUDE [1] cannot be relaxed until
> > > version 1.24+ of Pahole is the minimum requirement by the kernel. I'm OK
> > > with this change but keep in mind that it won't remedy that condition.
> > >
> > > Link: https://lore.kernel.org/bpf/20230111152050.559334-1-yakoyoku@gmail.com/ [1]
> >
> > Oh yes, I'm using pahole 1.24, that's probably why it's all working for
> > me. So we can't just drop !DEBUG_INFO_BTF, but we can definitely relax
> > the constraint to "!DEBUG_INFO_BTF || PAHOLE_HAS_LANG_EXCLUDE".
> 
> Yup the patch is ok, but we can only build with RUST and
> DEBUG_INFO_BTF if we have pahole 1.24+ because that is the version
> smart enough to exclude the Rust CUs.
> 
> If you have a version of pahole < 1.24 and you want to include RUST
> you basically should not have DEBUG_INFO_BTF on at all because your
> build will fail (unless this change alters the list of CUs passed to
> pahole).
> 
> What is a little confusing is scripts/is_rust_module.sh and pahole
> "--lang_exclude=rust" feature both attempt to do the same thing,
> recognize and skip Rust CUs

Yep, "pahole --lang_exclude=rust" makes scripts/is_rust_module.sh
obsolete, so with this applied:
https://lore.kernel.org/bpf/20230111152050.559334-1-yakoyoku@gmail.com/

I can just drop scripts/is_rust_module.sh and everything is still
working.

So, if that patch is applied we can do another patch to get rid of
is_rust_module.sh completely (because to my knowledge this script is
only used to exclude the BTF generation for the Rust modules).

-Andrea

[PATCH] rust: fix regexp in scripts/is_rust_module.sh

[Andrea Righi]

nm can use "R" or "r" to show read-only data sections, but
scripts/is_rust_module.sh can only recognize "r", so with some versions
of binutils it can fail to detect if a module is a Rust module or not.

Right now we're using this script only to determine if we need to skip
BTF generation (that is disabled globally if CONFIG_RUST is enabled),
but it's still nice to fix this script to do the proper job.

Moreover, with this patch applied I can also relax the constraint of
"RUST depends on !DEBUG_INFO_BTF" and build a kernel with Rust and BTF
enabled at the same time (of course BTF generation is still skipped for
Rust modules).

Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
---
 scripts/is_rust_module.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/is_rust_module.sh b/scripts/is_rust_module.sh
index 28b3831a7593..464761a7cf7f 100755
--- a/scripts/is_rust_module.sh
+++ b/scripts/is_rust_module.sh
@@ -13,4 +13,4 @@ set -e
 #
 # In the future, checking for the `.comment` section may be another
 # option, see https://github.com/rust-lang/rust/pull/97550.
-${NM} "$*" | grep -qE '^[0-9a-fA-F]+ r _R[^[:space:]]+16___IS_RUST_MODULE[^[:space:]]*$'
+${NM} "$*" | grep -qE '^[0-9a-fA-F]+ [Rr] _R[^[:space:]]+16___IS_RUST_MODULE[^[:space:]]*$'
-- 
2.37.2

Re: [PATCH] rust: fix regexp in scripts/is_rust_module.sh

[Vincenzo Palazzo]

> nm can use "R" or "r" to show read-only data sections, but
> scripts/is_rust_module.sh can only recognize "r", so with some versions
> of binutils it can fail to detect if a module is a Rust module or not.
>
> Right now we're using this script only to determine if we need to skip
> BTF generation (that is disabled globally if CONFIG_RUST is enabled),
> but it's still nice to fix this script to do the proper job.
>
> Moreover, with this patch applied I can also relax the constraint of
> "RUST depends on !DEBUG_INFO_BTF" and build a kernel with Rust and BTF
> enabled at the same time (of course BTF generation is still skipped for
> Rust modules).
>
> Signed-off-by: Andrea Righi <andrea.righi@canonical.com>

Reviewed-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

Re: [PATCH] rust: fix regexp in scripts/is_rust_module.sh

[Miguel Ojeda]

On Fri, Feb 10, 2023 at 4:26 PM Andrea Righi <andrea.righi@canonical.com> wrote:
>
> nm can use "R" or "r" to show read-only data sections, but
> scripts/is_rust_module.sh can only recognize "r", so with some versions
> of binutils it can fail to detect if a module is a Rust module or not.

Do you know which versions? If so, it would be nice to document it here.

> Moreover, with this patch applied I can also relax the constraint of
> "RUST depends on !DEBUG_INFO_BTF" and build a kernel with Rust and BTF
> enabled at the same time (of course BTF generation is still skipped for
> Rust modules).

Even if that build succeeds, can you load the modules? i.e. the
constraint was there due to
https://github.com/Rust-for-Linux/linux/issues/735.

Also Cc'ing Daniel, Eric and Martin since they are the ones working on this.

Cheers,
Miguel

Re: [PATCH] rust: fix regexp in scripts/is_rust_module.sh

[Eric Curtin]

On Mon, 13 Feb 2023 at 13:19, Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> On Fri, Feb 10, 2023 at 4:26 PM Andrea Righi <andrea.righi@canonical.com> wrote:
> >
> > nm can use "R" or "r" to show read-only data sections, but
> > scripts/is_rust_module.sh can only recognize "r", so with some versions
> > of binutils it can fail to detect if a module is a Rust module or not.
>
> Do you know which versions? If so, it would be nice to document it here.
>
> > Moreover, with this patch applied I can also relax the constraint of
> > "RUST depends on !DEBUG_INFO_BTF" and build a kernel with Rust and BTF
> > enabled at the same time (of course BTF generation is still skipped for
> > Rust modules).
>
> Even if that build succeeds, can you load the modules? i.e. the
> constraint was there due to
> https://github.com/Rust-for-Linux/linux/issues/735.
>
> Also Cc'ing Daniel, Eric and Martin since they are the ones working on this.

Don't have any issues with the change. Seems simple enough!

Reviewed-by: Eric Curtin <ecurtin@redhat.com>

>
> Cheers,
> Miguel
>

Re: [PATCH] rust: fix regexp in scripts/is_rust_module.sh

[Andrea Righi]

On Mon, Feb 13, 2023 at 02:19:38PM +0100, Miguel Ojeda wrote:
> On Fri, Feb 10, 2023 at 4:26 PM Andrea Righi <andrea.righi@canonical.com> wrote:
> >
> > nm can use "R" or "r" to show read-only data sections, but
> > scripts/is_rust_module.sh can only recognize "r", so with some versions
> > of binutils it can fail to detect if a module is a Rust module or not.
> 
> Do you know which versions? If so, it would be nice to document it here.
> 
> > Moreover, with this patch applied I can also relax the constraint of
> > "RUST depends on !DEBUG_INFO_BTF" and build a kernel with Rust and BTF
> > enabled at the same time (of course BTF generation is still skipped for
> > Rust modules).
> 
> Even if that build succeeds, can you load the modules? i.e. the
> constraint was there due to
> https://github.com/Rust-for-Linux/linux/issues/735.

This patch simply fixes scripts/is_rust_module.sh to recognize Rust
modules from "regular" C modules with certain versions of binutils, so
that BTF generation is properly skipped for Rust modules.

In this way both C and Rust modules can be loaded correctly (at least in
my tests I'm able load both with CONFIG_DEBUG_INFO_BTF enabled).

I haven't dropped the "RUST depends on !DEBUG_INFO_BTF" yet, but I think
with this fix is applied we can relax this constraint.

-Andrea

> 
> Also Cc'ing Daniel, Eric and Martin since they are the ones working on this.
> 
> Cheers,
> Miguel

Re: [PATCH] rust: fix regexp in scripts/is_rust_module.sh

[Miguel Ojeda]

On Mon, Feb 13, 2023 at 4:01 PM Andrea Righi <andrea.righi@canonical.com> wrote:
>
> In this way both C and Rust modules can be loaded correctly (at least in
> my tests I'm able load both with CONFIG_DEBUG_INFO_BTF enabled).
>
> I haven't dropped the "RUST depends on !DEBUG_INFO_BTF" yet, but I think
> with this fix is applied we can relax this constraint.

Yeah, but the constraint was there for other reasons, so I got
surprised when I read that in the commit message.

Apparently, Martin cannot load the modules
(https://lore.kernel.org/rust-for-linux/20230213151339.661225-1-yakoyoku@gmail.com/),
but you can.

Cheers,
Miguel

Re: [PATCH] rust: fix regexp in scripts/is_rust_module.sh

[Martin Rodriguez Reboredo]

On Fri, Feb 10, 2023 at 4:26 PM Andrea Righi <andrea.righi@canonical.com> wrote:
> nm can use "R" or "r" to show read-only data sections, but
> scripts/is_rust_module.sh can only recognize "r", so with some versions
> of binutils it can fail to detect if a module is a Rust module or not.

As __IS_RUST_MODULE can be a dynamic symbol too this change seems
reasonable to merge.

Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>

Re: [PATCH] rust: fix regexp in scripts/is_rust_module.sh

[Miguel Ojeda]

On Fri, Feb 10, 2023 at 4:26 PM Andrea Righi <andrea.righi@canonical.com> wrote:
>
> nm can use "R" or "r" to show read-only data sections, but
> scripts/is_rust_module.sh can only recognize "r", so with some versions
> of binutils it can fail to detect if a module is a Rust module or not.

Applied to `rust-fixes`. Thanks!

Cheers,
Miguel