Re: [PATCH v2 4/4] kbuild: use $(src) instead of $(srctree)/$(src) for source directory

Re: [PATCH v2 4/4] kbuild: use $(src) instead of $(srctree)/$(src) for source directory

[Krzysztof Kozlowski]

On 27/04/2024 16:55, Masahiro Yamada wrote:
> Kbuild conventionally uses $(obj)/ for generated files, and $(src)/ for
> checked-in source files. It is merely a convention without any functional
> difference. In fact, $(obj) and $(src) are exactly the same, as defined
> in scripts/Makefile.build:
> 
>     src := $(obj)
> 
> When the kernel is built in a separate output directory, $(src) does
> not accurately reflect the source directory location. While Kbuild
> resolves this discrepancy by specifying VPATH=$(srctree) to search for
> source files, it does not cover all cases. For example, when adding a
> header search path for local headers, -I$(srctree)/$(src) is typically
> passed to the compiler.
> 
> This introduces inconsistency between upstream and downstream Makefiles
> because $(src) is used instead of $(srctree)/$(src) for the latter.
> 
> To address this inconsistency, this commit changes the semantics of
> $(src) so that it always points to the directory in the source tree.
> 
> Going forward, the variables used in Makefiles will have the following
> meanings:
> 
>   $(obj)     - directory in the object tree
>   $(src)     - directory in the source tree  (changed by this commit)
>   $(objtree) - the top of the kernel object tree
>   $(srctree) - the top of the kernel source tree
> 
> Consequently, $(srctree)/$(src) in upstream Makefiles need to be replaced
> with $(src).
> 

This patch, since yesterday's next, causes build issues with my
(completely) standard and typical workflow:

alias crosc64='CROSS_COMPILE="ccache aarch64-linux-gnu-" ARCH=arm64 KBUILD_OUTPUT=out/'
crosc64 make defconfig
crosc64 make dt_binding_check

Errors:

No rule to make target 'Documentation/devicetree/bindings/i3c/snps,dw-i3c-master.example.dtb', needed by 'dt_binding_check'. Stop.

Please revert/drop from next.

Best regards,
Krzysztof

Re: [PATCH v2 4/4] kbuild: use $(src) instead of $(srctree)/$(src) for source directory

[Masahiro Yamada]

On Thu, May 9, 2024 at 5:59 PM Krzysztof Kozlowski <krzk@kernel.org> wrote:
>
> On 27/04/2024 16:55, Masahiro Yamada wrote:
> > Kbuild conventionally uses $(obj)/ for generated files, and $(src)/ for
> > checked-in source files. It is merely a convention without any functional
> > difference. In fact, $(obj) and $(src) are exactly the same, as defined
> > in scripts/Makefile.build:
> >
> >     src := $(obj)
> >
> > When the kernel is built in a separate output directory, $(src) does
> > not accurately reflect the source directory location. While Kbuild
> > resolves this discrepancy by specifying VPATH=$(srctree) to search for
> > source files, it does not cover all cases. For example, when adding a
> > header search path for local headers, -I$(srctree)/$(src) is typically
> > passed to the compiler.
> >
> > This introduces inconsistency between upstream and downstream Makefiles
> > because $(src) is used instead of $(srctree)/$(src) for the latter.
> >
> > To address this inconsistency, this commit changes the semantics of
> > $(src) so that it always points to the directory in the source tree.
> >
> > Going forward, the variables used in Makefiles will have the following
> > meanings:
> >
> >   $(obj)     - directory in the object tree
> >   $(src)     - directory in the source tree  (changed by this commit)
> >   $(objtree) - the top of the kernel object tree
> >   $(srctree) - the top of the kernel source tree
> >
> > Consequently, $(srctree)/$(src) in upstream Makefiles need to be replaced
> > with $(src).
> >
>
> This patch, since yesterday's next, causes build issues with my
> (completely) standard and typical workflow:
>
> alias crosc64='CROSS_COMPILE="ccache aarch64-linux-gnu-" ARCH=arm64 KBUILD_OUTPUT=out/'
> crosc64 make defconfig
> crosc64 make dt_binding_check
>
> Errors:
>
> No rule to make target 'Documentation/devicetree/bindings/i3c/snps,dw-i3c-master.example.dtb', needed by 'dt_binding_check'. Stop.
>
> Please revert/drop from next.


Or, fix the code.


Now fixed. Thanks for the report.




-- 
Best Regards
Masahiro Yamada

Re: [PATCH v2 4/4] kbuild: use $(src) instead of $(srctree)/$(src) for source directory

[Edgecombe, Rick P]

On Fri, 2024-05-10 at 04:36 +0900, Masahiro Yamada wrote:
> 
> Or, fix the code.
> 
> 
> Now fixed. Thanks for the report.
> 

Hi,

This commit broke my build in a similar way:
make ARCH=x86_64 O=../linux-tdx-kvm-out/ -j36 M=arch/x86/kvm/
make[1]: Entering directory '/home/rpedgeco/repos/linux-tdx-kvm-out'
/home/rpedgeco/repos/linux/scripts/Makefile.build:41: arch/x86/kvm/Makefile: No
such file or directory
make[3]: *** No rule to make target 'arch/x86/kvm/Makefile'.  Stop.
make[2]: *** [/home/rpedgeco/repos/linux/Makefile:1934: arch/x86/kvm] Error 2
make[1]: *** [/home/rpedgeco/repos/linux/Makefile:240: __sub-make] Error 2
make[1]: Leaving directory '/home/rpedgeco/repos/linux-tdx-kvm-out'
make: *** [Makefile:240: __sub-make] Error 2

This patch didn’t fix it:
https://lore.kernel.org/lkml/20240624-kbuild-fix-xconfig-v1-1-7c06eae6d3aa@avm.de/

But reverting b1992c3772e6 ("kbuild: use $(src) instead of $(srctree)/$(src) for
source directory") does.

I'm guess it has something to do with the "M=" option because:
make ARCH=x86_64 O=../linux-tdx-kvm-out/ -j36 arch/x86/kvm/kvm.ko
arch/x86/kvm/kvm-intel.ko

...works.

Re: [PATCH v2 4/4] kbuild: use $(src) instead of $(srctree)/$(src) for source directory

[Masahiro Yamada]

On Fri, Jun 28, 2024 at 2:14 AM Edgecombe, Rick P
<rick.p.edgecombe@intel.com> wrote:
>
> On Fri, 2024-05-10 at 04:36 +0900, Masahiro Yamada wrote:
> >
> > Or, fix the code.
> >
> >
> > Now fixed. Thanks for the report.
> >
>
> Hi,
>
> This commit broke my build in a similar way:
> make ARCH=x86_64 O=../linux-tdx-kvm-out/ -j36 M=arch/x86/kvm/
> make[1]: Entering directory '/home/rpedgeco/repos/linux-tdx-kvm-out'
> /home/rpedgeco/repos/linux/scripts/Makefile.build:41: arch/x86/kvm/Makefile: No
> such file or directory
> make[3]: *** No rule to make target 'arch/x86/kvm/Makefile'.  Stop.
> make[2]: *** [/home/rpedgeco/repos/linux/Makefile:1934: arch/x86/kvm] Error 2
> make[1]: *** [/home/rpedgeco/repos/linux/Makefile:240: __sub-make] Error 2
> make[1]: Leaving directory '/home/rpedgeco/repos/linux-tdx-kvm-out'
> make: *** [Makefile:240: __sub-make] Error 2
>
> This patch didn’t fix it:
> https://lore.kernel.org/lkml/20240624-kbuild-fix-xconfig-v1-1-7c06eae6d3aa@avm.de/
>
> But reverting b1992c3772e6 ("kbuild: use $(src) instead of $(srctree)/$(src) for
> source directory") does.
>
> I'm guess it has something to do with the "M=" option because:
> make ARCH=x86_64 O=../linux-tdx-kvm-out/ -j36 arch/x86/kvm/kvm.ko
> arch/x86/kvm/kvm-intel.ko
>
> ...works.



There are two solutions, depending on what you want to achieve.

The official way is to pass the absolute path to M=
(or relative path to ../linux-tdm-kvm-out)

The other unofficial way is to pass VPATH.
The external module build does not officially support
the separate output directory, but you can still
do it in this case.

[1] will work like before.



[1] Output module objects to linux-tdm-kvm-out/arch/x86/kvm/ (unofficial)


$ make ARCH=x86_64 O=../linux-tdx-kvm-out/ -j36 M=arch/x86/kvm VPATH=$(pwd)


[2] Output the module objects into arch/x86/kvm/ of the source tree (official)


$ make ARCH=x86_64 O=../linux-tdx-kvm-out/ -j36 M=$(realpath arch/x86/kvm)



-- 
Best Regards
Masahiro Yamada

Re: [PATCH v2 4/4] kbuild: use $(src) instead of $(srctree)/$(src) for source directory

[Edgecombe, Rick P]

On Fri, 2024-06-28 at 11:13 +0900, Masahiro Yamada wrote:
> There are two solutions, depending on what you want to achieve.
> 
> The official way is to pass the absolute path to M=
> (or relative path to ../linux-tdm-kvm-out)
> 
> The other unofficial way is to pass VPATH.
> The external module build does not officially support
> the separate output directory, but you can still
> do it in this case.
> 
> [1] will work like before.

The absolute path worked, but why not make it use the relative path by default
in this case? "arch/x86/kvm/" shouldn't be confused with an absolute path...

Re: [PATCH v2 4/4] kbuild: use $(src) instead of $(srctree)/$(src) for source directory

[Edgecombe, Rick P]

On Fri, 2024-06-28 at 14:23 -0700, Rick Edgecombe wrote:
> On Fri, 2024-06-28 at 11:13 +0900, Masahiro Yamada wrote:
> > There are two solutions, depending on what you want to achieve.
> > 
> > The official way is to pass the absolute path to M=
> > (or relative path to ../linux-tdm-kvm-out)
> > 
> > The other unofficial way is to pass VPATH.
> > The external module build does not officially support
> > the separate output directory, but you can still
> > do it in this case.
> > 
> > [1] will work like before.
> 
> The absolute path worked, but why not make it use the relative path by default
> in this case? "arch/x86/kvm/" shouldn't be confused with an absolute path...

Argh, I missed that make ARCH=x86_64 O=../linux-tdx-kvm-out/ -j36 M=$(realpath
arch/x86/kvm) will output the the source tree instead of the output directory.

So there is no official way to build just a module to the output directory? And
it just accidentally worked all these years? IMO it's a nice feature to have.
I've especially found it useful when doing virtualization development where you
need to build/test guest and host kernels as the same time.

Or was it just now delegated to unofficial support after this patch?

Re: [PATCH v2 4/4] kbuild: use $(src) instead of $(srctree)/$(src) for source directory

[Masahiro Yamada]

On Sat, Jun 29, 2024 at 6:24 AM Edgecombe, Rick P
<rick.p.edgecombe@intel.com> wrote:
>
> On Fri, 2024-06-28 at 11:13 +0900, Masahiro Yamada wrote:
> > There are two solutions, depending on what you want to achieve.
> >
> > The official way is to pass the absolute path to M=
> > (or relative path to ../linux-tdm-kvm-out)
> >
> > The other unofficial way is to pass VPATH.
> > The external module build does not officially support
> > the separate output directory, but you can still
> > do it in this case.
> >
> > [1] will work like before.
>
> The absolute path worked, but why not make it use the relative path by default
> in this case? "arch/x86/kvm/" shouldn't be confused with an absolute path...




If you give a relative path to the M= option,
it is relative to the top of the build tree,
not relative to the top of the source tree.


If you use the relative path correctly,
it is interchangeable with the absolute path.


It did not work as you expected because
you used it in the wrong way.




-- 
Best Regards
Masahiro Yamada

Re: [PATCH v2 4/4] kbuild: use $(src) instead of $(srctree)/$(src) for source directory

[Masahiro Yamada]

On Sat, Jun 29, 2024 at 7:54 AM Edgecombe, Rick P
<rick.p.edgecombe@intel.com> wrote:
>
> On Fri, 2024-06-28 at 14:23 -0700, Rick Edgecombe wrote:
> > On Fri, 2024-06-28 at 11:13 +0900, Masahiro Yamada wrote:
> > > There are two solutions, depending on what you want to achieve.
> > >
> > > The official way is to pass the absolute path to M=
> > > (or relative path to ../linux-tdm-kvm-out)
> > >
> > > The other unofficial way is to pass VPATH.
> > > The external module build does not officially support
> > > the separate output directory, but you can still
> > > do it in this case.
> > >
> > > [1] will work like before.
> >
> > The absolute path worked, but why not make it use the relative path by default
> > in this case? "arch/x86/kvm/" shouldn't be confused with an absolute path...
>
> Argh, I missed that make ARCH=x86_64 O=../linux-tdx-kvm-out/ -j36 M=$(realpath
> arch/x86/kvm) will output the the source tree instead of the output directory.
>
> So there is no official way to build just a module to the output directory?


You already know how to do it:

#I'm guess it has something to do with the "M=" option because:
#make ARCH=x86_64 O=../linux-tdx-kvm-out/ -j36 arch/x86/kvm/kvm.ko
#arch/x86/kvm/kvm-intel.ko
#
#...works.




> And
> it just accidentally worked all these years? IMO it's a nice feature to have.
> I've especially found it useful when doing virtualization development where you
> need to build/test guest and host kernels as the same time.
>
> Or was it just now delegated to unofficial support after this patch?


No.
The M= option is intended for building external modules.
You just abused it to build the upstream modules.

Kbuild has never supported building external modules in a separate
output directory.


"delegated to unofficial support" is a wrong statement,
as it has never officially supported in the past.


Your way is a workaround that people found happened to work.






-- 
Best Regards
Masahiro Yamada

Re: [PATCH v2 4/4] kbuild: use $(src) instead of $(srctree)/$(src) for source directory

[Edgecombe, Rick P]

On Tue, 2024-07-02 at 18:15 +0900, Masahiro Yamada wrote:
> > So there is no official way to build just a module to the output directory?
> 
> 
> You already know how to do it:
> 
> #I'm guess it has something to do with the "M=" option because:
> #make ARCH=x86_64 O=../linux-tdx-kvm-out/ -j36 arch/x86/kvm/kvm.ko
> #arch/x86/kvm/kvm-intel.ko
> #
> #...works.

This can build just the module, but not clean just the module.

> 
> 
> 
> 
> > And
> > it just accidentally worked all these years? IMO it's a nice feature to
> > have.
> > I've especially found it useful when doing virtualization development where
> > you
> > need to build/test guest and host kernels as the same time.
> > 
> > Or was it just now delegated to unofficial support after this patch?
> 
> 
> No.
> The M= option is intended for building external modules.
> You just abused it to build the upstream modules.
> 
> Kbuild has never supported building external modules in a separate
> output directory.
> 
> 
> "delegated to unofficial support" is a wrong statement,
> as it has never officially supported in the past.
> 
> 
> Your way is a workaround that people found happened to work.

The main point is that the old behavior was useful. It seems the change that
broke my workflow was intended to make it easier to maintain out-of-tree
modules? If so then aren't we taking away useful behavior for in-tree module
development to help out-of-tree modules?

Can you at least support VPATH method going forward? Although I am surprised I'm
the only one that hit this. I assumed this was a common usage.

Re: [PATCH v2 4/4] kbuild: use $(src) instead of $(srctree)/$(src) for source directory

[Masahiro Yamada]

On Wed, Jul 3, 2024 at 1:15 AM Edgecombe, Rick P
<rick.p.edgecombe@intel.com> wrote:
>
> On Tue, 2024-07-02 at 18:15 +0900, Masahiro Yamada wrote:
> > > So there is no official way to build just a module to the output directory?
> >
> >
> > You already know how to do it:
> >
> > #I'm guess it has something to do with the "M=" option because:
> > #make ARCH=x86_64 O=../linux-tdx-kvm-out/ -j36 arch/x86/kvm/kvm.ko
> > #arch/x86/kvm/kvm-intel.ko
> > #
> > #...works.
>
> This can build just the module, but not clean just the module.
>
> >
> >
> >
> >
> > > And
> > > it just accidentally worked all these years? IMO it's a nice feature to
> > > have.
> > > I've especially found it useful when doing virtualization development where
> > > you
> > > need to build/test guest and host kernels as the same time.
> > >
> > > Or was it just now delegated to unofficial support after this patch?
> >
> >
> > No.
> > The M= option is intended for building external modules.
> > You just abused it to build the upstream modules.
> >
> > Kbuild has never supported building external modules in a separate
> > output directory.
> >
> >
> > "delegated to unofficial support" is a wrong statement,
> > as it has never officially supported in the past.
> >
> >
> > Your way is a workaround that people found happened to work.
>
> The main point is that the old behavior was useful. It seems the change that
> broke my workflow was intended to make it easier to maintain out-of-tree
> modules? If so then aren't we taking away useful behavior for in-tree module
> development to help out-of-tree modules?


No.  I never meant to prioritize the external module builds.

I believe b1992c3772e69a6fd0e3fc81cd4d2820c8b6eca0 is the
right thing to do. That's all.



> Can you at least support VPATH method going forward? Although I am surprised I'm
> the only one that hit this. I assumed this was a common usage.


VPATH will be broken when a big change happens again
for external module builds.
That is what "unsupported" and "unofficial" mean.




-- 
Best Regards
Masahiro Yamada