[PATCH] kbuild: use $(obj)/ instead of $(src)/ for COPY

[PATCH] kbuild: use $(obj)/ instead of $(src)/ for COPY

[HONG Yifan]

Similar to
commit 9a0ebe5011f4 ("kbuild: use $(obj)/ instead of $(src)/ for common pattern rules")

This change updates the COPY rule to use $(obj) instead of $(src). This
makes Kbuild rules like

    always-y += libfoo/.foo.o.cmd

work when the user provides libfoo/.foo.o.cmd_shipped, even when obj and
src is different and src is an absolute path. This is useful when foo.o
and .foo.o.cmd are checked-in as prebuilts.

(Admittedly, `always-y += libfoo/.foo.o.cmd` is not recommended in
kbuild/modules.rst, "Several Subdirectories".)

For example, if

    obj=.
    src=/some/path

then the original rule

    $(obj)/%: $(src)/%_shipped

expands to

    ./%: /some/path/%_shipped

And when matching against the above example, the stem is just `bar.o`
[^1] so the following is looked up:

    libfoo/.foo.o.cmd: libfoo//some/path/.foo.o.cmd_shipped

... and it cannot be matched.

With this change, the rule expands to

    ./%: ./%_shipped

... and it should work, at least for files that does not have a more
specific pattern rule.

NOTE: that after this change, code like

    bar-y += libfoo/foo.o

... with libfoo/foo.o_shipped provided still DOES NOT work, because
the pattern rule $(obj)/%.o takes priority. For .o_shipped files,
the user still needs an explicit `$(obj)/%.o: $(obj)/%.o_shipped` rule
in its own Kbuild file.

[^1]: https://www.gnu.org/software/make/manual/html_node/Pattern-Match.html

Signed-off-by: HONG Yifan <elsk@google.com>
---
 scripts/Makefile.lib | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 1d581ba5df66..e066b7b00bcc 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -272,7 +272,7 @@ endef
 quiet_cmd_copy = COPY    $@
       cmd_copy = cat $< > $@
 
-$(obj)/%: $(src)/%_shipped
+$(obj)/%: $(obj)/%_shipped
 	$(call cmd,copy)
 
 # Touch a file
-- 
2.51.0.618.g983fd99d29-goog

Re: [PATCH] kbuild: use $(obj)/ instead of $(src)/ for COPY

[Nicolas Schier]

On Mon, Oct 06, 2025 at 07:38:38PM +0000, HONG Yifan wrote:
> Similar to
> commit 9a0ebe5011f4 ("kbuild: use $(obj)/ instead of $(src)/ for common pattern rules")
> 
> This change updates the COPY rule to use $(obj) instead of $(src). This
> makes Kbuild rules like
> 
>     always-y += libfoo/.foo.o.cmd

This is a strange example.  Why should we ship any prebuilt .*.o.cmd file?

> 
> work when the user provides libfoo/.foo.o.cmd_shipped, even when obj and
> src is different and src is an absolute path. This is useful when foo.o
> and .foo.o.cmd are checked-in as prebuilts.
> 
> (Admittedly, `always-y += libfoo/.foo.o.cmd` is not recommended in
> kbuild/modules.rst, "Several Subdirectories".)
> 
> For example, if
> 
>     obj=.
>     src=/some/path
> 
> then the original rule
> 
>     $(obj)/%: $(src)/%_shipped
> 
> expands to
> 
>     ./%: /some/path/%_shipped
> 
> And when matching against the above example, the stem is just `bar.o`
> [^1] so the following is looked up:
> 
>     libfoo/.foo.o.cmd: libfoo//some/path/.foo.o.cmd_shipped
> 
> ... and it cannot be matched.
> 
> With this change, the rule expands to
> 
>     ./%: ./%_shipped
> 
> ... and it should work, at least for files that does not have a more
> specific pattern rule.
> 
> NOTE: that after this change, code like
> 
>     bar-y += libfoo/foo.o
> 
> ... with libfoo/foo.o_shipped provided still DOES NOT work, because
> the pattern rule $(obj)/%.o takes priority. For .o_shipped files,
> the user still needs an explicit `$(obj)/%.o: $(obj)/%.o_shipped` rule
> in its own Kbuild file.
> 
> [^1]: https://www.gnu.org/software/make/manual/html_node/Pattern-Match.html
> 
> Signed-off-by: HONG Yifan <elsk@google.com>
> ---
>  scripts/Makefile.lib | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index 1d581ba5df66..e066b7b00bcc 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -272,7 +272,7 @@ endef
>  quiet_cmd_copy = COPY    $@
>        cmd_copy = cat $< > $@
>  
> -$(obj)/%: $(src)/%_shipped
> +$(obj)/%: $(obj)/%_shipped
>  	$(call cmd,copy)

No, I don't see a reason to support *_shipped files from the build tree.
The purpose of *_shipped is to deliver prebuilt files with the source
tree to allow or simplify building the corresponding output files.

Kind regards,
Nicolas

Re: [PATCH] kbuild: use $(obj)/ instead of $(src)/ for COPY

[Hong, Yifan]

On Thu, Oct 9, 2025 at 2:16 AM Nicolas Schier <nsc@kernel.org> wrote:
>
> On Mon, Oct 06, 2025 at 07:38:38PM +0000, HONG Yifan wrote:
> > Similar to
> > commit 9a0ebe5011f4 ("kbuild: use $(obj)/ instead of $(src)/ for common pattern rules")
> >
> > This change updates the COPY rule to use $(obj) instead of $(src). This
> > makes Kbuild rules like
> >
> >     always-y += libfoo/.foo.o.cmd
>
> This is a strange example.  Why should we ship any prebuilt .*.o.cmd file?

When one ships the .o file, it might be beneficial to also ship the
accompanying .o.cmd file
so that compdb may work, I guess. Though, I just get this example from one
of the SoC manufacturers, so I actually don't know the true reasoning
behind it. I agree that it
isn't a good example.

Still, this applies to any file that does NOT match any existing
pattern rules in Kbuild. It might be more
generic if I had said instead

    always-y += libfoo/foo.xyz

... and we were providing a libfoo/foo.xyz_shipped in the source tree.

>
> >
> > work when the user provides libfoo/.foo.o.cmd_shipped, even when obj and
> > src is different and src is an absolute path. This is useful when foo.o
> > and .foo.o.cmd are checked-in as prebuilts.
> >
> > (Admittedly, `always-y += libfoo/.foo.o.cmd` is not recommended in
> > kbuild/modules.rst, "Several Subdirectories".)
> >
> > For example, if
> >
> >     obj=.
> >     src=/some/path
> >
> > then the original rule
> >
> >     $(obj)/%: $(src)/%_shipped
> >
> > expands to
> >
> >     ./%: /some/path/%_shipped
> >
> > And when matching against the above example, the stem is just `bar.o`
> > [^1] so the following is looked up:
> >
> >     libfoo/.foo.o.cmd: libfoo//some/path/.foo.o.cmd_shipped
> >
> > ... and it cannot be matched.
> >
> > With this change, the rule expands to
> >
> >     ./%: ./%_shipped
> >
> > ... and it should work, at least for files that does not have a more
> > specific pattern rule.
> >
> > NOTE: that after this change, code like
> >
> >     bar-y += libfoo/foo.o
> >
> > ... with libfoo/foo.o_shipped provided still DOES NOT work, because
> > the pattern rule $(obj)/%.o takes priority. For .o_shipped files,
> > the user still needs an explicit `$(obj)/%.o: $(obj)/%.o_shipped` rule
> > in its own Kbuild file.
> >
> > [^1]: https://www.gnu.org/software/make/manual/html_node/Pattern-Match.html
> >
> > Signed-off-by: HONG Yifan <elsk@google.com>
> > ---
> >  scripts/Makefile.lib | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > index 1d581ba5df66..e066b7b00bcc 100644
> > --- a/scripts/Makefile.lib
> > +++ b/scripts/Makefile.lib
> > @@ -272,7 +272,7 @@ endef
> >  quiet_cmd_copy = COPY    $@
> >        cmd_copy = cat $< > $@
> >
> > -$(obj)/%: $(src)/%_shipped
> > +$(obj)/%: $(obj)/%_shipped
> >       $(call cmd,copy)
>
> No, I don't see a reason to support *_shipped files from the build tree.
> The purpose of *_shipped is to deliver prebuilt files with the source
> tree to allow or simplify building the corresponding output files.

The goal is not to support .o_shipped files from the build tree,
though it is indeed one
side effect of the patch. My goal is that we support .o_shipped files
from the source tree when all
of the following is true:

- We are building with O= / MO=, so $(obj) and $(src) are different,
and in this case $(obj) is `.` and
  $(src) is an absolute path);
- We have a `xxx-y += libfoo/bar.o` line; in other words, the
bar.o_shipped file is in a **subdirectory**
  libfoo below the directory of the Kbuild file. As I said in the
commit message, this is NOT recommended
  (kbuild/modules.rst, "Several Subdirectories"), but it is a
documented way to include a .o file from
  elsewhere.

And as I said in the commit message at the end, unfortunately this
patch still can't achieve this goal for .o files,
only for files that don't match any existing pattern rules, like .cmd
files for example.

Would you please suggest how we can support .o_shipped files in a
subdirectory when building with
$(obj) = . and $(src) = <some absolute path>? Thank you!

Yifan

>
> Kind regards,
> Nicolas

Re: [PATCH] kbuild: use $(obj)/ instead of $(src)/ for COPY

[Nicolas Schier]

On Thu, Oct 09, 2025 at 02:16:20PM -0700, Hong, Yifan wrote:
> On Thu, Oct 9, 2025 at 2:16 AM Nicolas Schier <nsc@kernel.org> wrote:
> >
> > On Mon, Oct 06, 2025 at 07:38:38PM +0000, HONG Yifan wrote:
> > > Similar to
> > > commit 9a0ebe5011f4 ("kbuild: use $(obj)/ instead of $(src)/ for common pattern rules")
> > >
> > > This change updates the COPY rule to use $(obj) instead of $(src). This
> > > makes Kbuild rules like
> > >
> > >     always-y += libfoo/.foo.o.cmd
> >
> > This is a strange example.  Why should we ship any prebuilt .*.o.cmd file?
> 
> When one ships the .o file, it might be beneficial to also ship the
> accompanying .o.cmd file
> so that compdb may work, I guess. Though, I just get this example from one
> of the SoC manufacturers, so I actually don't know the true reasoning
> behind it. I agree that it
> isn't a good example.
> 
> Still, this applies to any file that does NOT match any existing
> pattern rules in Kbuild. It might be more
> generic if I had said instead
> 
>     always-y += libfoo/foo.xyz
> 
> ... and we were providing a libfoo/foo.xyz_shipped in the source tree.
> 
> >
> > >
> > > work when the user provides libfoo/.foo.o.cmd_shipped, even when obj and
> > > src is different and src is an absolute path. This is useful when foo.o
> > > and .foo.o.cmd are checked-in as prebuilts.
> > >
> > > (Admittedly, `always-y += libfoo/.foo.o.cmd` is not recommended in
> > > kbuild/modules.rst, "Several Subdirectories".)
> > >
> > > For example, if
> > >
> > >     obj=.
> > >     src=/some/path
> > >
> > > then the original rule
> > >
> > >     $(obj)/%: $(src)/%_shipped
> > >
> > > expands to
> > >
> > >     ./%: /some/path/%_shipped
> > >
> > > And when matching against the above example, the stem is just `bar.o`
> > > [^1] so the following is looked up:
> > >
> > >     libfoo/.foo.o.cmd: libfoo//some/path/.foo.o.cmd_shipped
> > >
> > > ... and it cannot be matched.
> > >
> > > With this change, the rule expands to
> > >
> > >     ./%: ./%_shipped
> > >
> > > ... and it should work, at least for files that does not have a more
> > > specific pattern rule.
> > >
> > > NOTE: that after this change, code like
> > >
> > >     bar-y += libfoo/foo.o
> > >
> > > ... with libfoo/foo.o_shipped provided still DOES NOT work, because
> > > the pattern rule $(obj)/%.o takes priority. For .o_shipped files,
> > > the user still needs an explicit `$(obj)/%.o: $(obj)/%.o_shipped` rule
> > > in its own Kbuild file.
> > >
> > > [^1]: https://www.gnu.org/software/make/manual/html_node/Pattern-Match.html
> > >
> > > Signed-off-by: HONG Yifan <elsk@google.com>
> > > ---
> > >  scripts/Makefile.lib | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > > index 1d581ba5df66..e066b7b00bcc 100644
> > > --- a/scripts/Makefile.lib
> > > +++ b/scripts/Makefile.lib
> > > @@ -272,7 +272,7 @@ endef
> > >  quiet_cmd_copy = COPY    $@
> > >        cmd_copy = cat $< > $@
> > >
> > > -$(obj)/%: $(src)/%_shipped
> > > +$(obj)/%: $(obj)/%_shipped
> > >       $(call cmd,copy)
> >
> > No, I don't see a reason to support *_shipped files from the build tree.
> > The purpose of *_shipped is to deliver prebuilt files with the source
> > tree to allow or simplify building the corresponding output files.
> 
> The goal is not to support .o_shipped files from the build tree,
> though it is indeed one
> side effect of the patch. My goal is that we support .o_shipped files
> from the source tree when all
> of the following is true:
> 
> - We are building with O= / MO=, so $(obj) and $(src) are different,
> and in this case $(obj) is `.` and
>   $(src) is an absolute path);
> - We have a `xxx-y += libfoo/bar.o` line; in other words, the
> bar.o_shipped file is in a **subdirectory**
>   libfoo below the directory of the Kbuild file. As I said in the
> commit message, this is NOT recommended
>   (kbuild/modules.rst, "Several Subdirectories"), but it is a
> documented way to include a .o file from
>   elsewhere.
> 
> And as I said in the commit message at the end, unfortunately this
> patch still can't achieve this goal for .o files,
> only for files that don't match any existing pattern rules, like .cmd
> files for example.
> 
> Would you please suggest how we can support .o_shipped files in a
> subdirectory when building with
> $(obj) = . and $(src) = <some absolute path>? Thank you!

Thanks for the clarifications.  Please note that shipping .o_shipped
files with upstream Linux will not happen out of obvious reasons, and I
am not willing to invest time into supporting any closed-source
approach.

You might want to try to add the missing rules to your local modules'
Kbuild file, e.g. something like

$(obj)/%.o: $(obj)/%.o.cmd

$(obj)/%: $(src)/%_shipped | $(obj)/%.o.cmd
	$(call cmd,copy)

(not tested, no support).

Good luck and kind regards,
Nicolas

Re: [PATCH] kbuild: use $(obj)/ instead of $(src)/ for COPY

[Hong, Yifan]

On Fri, Oct 10, 2025 at 11:57 AM Nicolas Schier <nsc@kernel.org> wrote:
>
> On Thu, Oct 09, 2025 at 02:16:20PM -0700, Hong, Yifan wrote:
> > On Thu, Oct 9, 2025 at 2:16 AM Nicolas Schier <nsc@kernel.org> wrote:
> > >
> > > On Mon, Oct 06, 2025 at 07:38:38PM +0000, HONG Yifan wrote:
> > > > Similar to
> > > > commit 9a0ebe5011f4 ("kbuild: use $(obj)/ instead of $(src)/ for common pattern rules")
> > > >
> > > > This change updates the COPY rule to use $(obj) instead of $(src). This
> > > > makes Kbuild rules like
> > > >
> > > >     always-y += libfoo/.foo.o.cmd
> > >
> > > This is a strange example.  Why should we ship any prebuilt .*.o.cmd file?
> >
> > When one ships the .o file, it might be beneficial to also ship the
> > accompanying .o.cmd file
> > so that compdb may work, I guess. Though, I just get this example from one
> > of the SoC manufacturers, so I actually don't know the true reasoning
> > behind it. I agree that it
> > isn't a good example.
> >
> > Still, this applies to any file that does NOT match any existing
> > pattern rules in Kbuild. It might be more
> > generic if I had said instead
> >
> >     always-y += libfoo/foo.xyz
> >
> > ... and we were providing a libfoo/foo.xyz_shipped in the source tree.
> >
> > >
> > > >
> > > > work when the user provides libfoo/.foo.o.cmd_shipped, even when obj and
> > > > src is different and src is an absolute path. This is useful when foo.o
> > > > and .foo.o.cmd are checked-in as prebuilts.
> > > >
> > > > (Admittedly, `always-y += libfoo/.foo.o.cmd` is not recommended in
> > > > kbuild/modules.rst, "Several Subdirectories".)
> > > >
> > > > For example, if
> > > >
> > > >     obj=.
> > > >     src=/some/path
> > > >
> > > > then the original rule
> > > >
> > > >     $(obj)/%: $(src)/%_shipped
> > > >
> > > > expands to
> > > >
> > > >     ./%: /some/path/%_shipped
> > > >
> > > > And when matching against the above example, the stem is just `bar.o`
> > > > [^1] so the following is looked up:
> > > >
> > > >     libfoo/.foo.o.cmd: libfoo//some/path/.foo.o.cmd_shipped
> > > >
> > > > ... and it cannot be matched.
> > > >
> > > > With this change, the rule expands to
> > > >
> > > >     ./%: ./%_shipped
> > > >
> > > > ... and it should work, at least for files that does not have a more
> > > > specific pattern rule.
> > > >
> > > > NOTE: that after this change, code like
> > > >
> > > >     bar-y += libfoo/foo.o
> > > >
> > > > ... with libfoo/foo.o_shipped provided still DOES NOT work, because
> > > > the pattern rule $(obj)/%.o takes priority. For .o_shipped files,
> > > > the user still needs an explicit `$(obj)/%.o: $(obj)/%.o_shipped` rule
> > > > in its own Kbuild file.
> > > >
> > > > [^1]: https://www.gnu.org/software/make/manual/html_node/Pattern-Match.html
> > > >
> > > > Signed-off-by: HONG Yifan <elsk@google.com>
> > > > ---
> > > >  scripts/Makefile.lib | 2 +-
> > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > > > index 1d581ba5df66..e066b7b00bcc 100644
> > > > --- a/scripts/Makefile.lib
> > > > +++ b/scripts/Makefile.lib
> > > > @@ -272,7 +272,7 @@ endef
> > > >  quiet_cmd_copy = COPY    $@
> > > >        cmd_copy = cat $< > $@
> > > >
> > > > -$(obj)/%: $(src)/%_shipped
> > > > +$(obj)/%: $(obj)/%_shipped
> > > >       $(call cmd,copy)
> > >
> > > No, I don't see a reason to support *_shipped files from the build tree.
> > > The purpose of *_shipped is to deliver prebuilt files with the source
> > > tree to allow or simplify building the corresponding output files.
> >
> > The goal is not to support .o_shipped files from the build tree,
> > though it is indeed one
> > side effect of the patch. My goal is that we support .o_shipped files
> > from the source tree when all
> > of the following is true:
> >
> > - We are building with O= / MO=, so $(obj) and $(src) are different,
> > and in this case $(obj) is `.` and
> >   $(src) is an absolute path);
> > - We have a `xxx-y += libfoo/bar.o` line; in other words, the
> > bar.o_shipped file is in a **subdirectory**
> >   libfoo below the directory of the Kbuild file. As I said in the
> > commit message, this is NOT recommended
> >   (kbuild/modules.rst, "Several Subdirectories"), but it is a
> > documented way to include a .o file from
> >   elsewhere.
> >
> > And as I said in the commit message at the end, unfortunately this
> > patch still can't achieve this goal for .o files,
> > only for files that don't match any existing pattern rules, like .cmd
> > files for example.
> >
> > Would you please suggest how we can support .o_shipped files in a
> > subdirectory when building with
> > $(obj) = . and $(src) = <some absolute path>? Thank you!
>
> Thanks for the clarifications.  Please note that shipping .o_shipped
> files with upstream Linux will not happen out of obvious reasons, and I
> am not willing to invest time into supporting any closed-source
> approach.

Thanks for your reply. This is primarily for out-of-tree modules which
are not yet ready to be upstreamed. I agree that shipping prebuilt .o
files does not make sense in upstream Linux.

>
> You might want to try to add the missing rules to your local modules'
> Kbuild file, e.g. something like
>
> $(obj)/%.o: $(obj)/%.o.cmd
>
> $(obj)/%: $(src)/%_shipped | $(obj)/%.o.cmd
>         $(call cmd,copy)

Thanks. This is my current workaround now. I'll treat this as an
acceptable solution for now.

Please kindly disregard this patch.

>
> (not tested, no support).
>
> Good luck and kind regards,
> Nicolas