[PATCH] kbuild: mark "FORCE" target as secondary

[PATCH] kbuild: mark "FORCE" target as secondary

[Arnd Bergmann]

From: Arnd Bergmann <arnd@arndb.de>

Starting with make-4.4.1, Kbuild uses the special .NOTINTERMEDIATE
target to mark all targets as not intermediate, which slightly changes
the behavior compared to older versions of make.

This causes a build regression with my change to the system call table
scripts now in scripts/Makefile.asm-headers, forcing a rebuild of the
generated files with every make invocation and effectively breaking
incremental builds.

I have narrowed down the change in behavior to the way that the
'FORCE' target is treated: If this is marked as not intermediate,
the $(if_changed) macro always evaluates it as a missing prerequisite,
but if it is marked as .SECONDARY, it works like before.

Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: linux-kbuild@vger.kernel.org
Fixes: 875ef1a57f32 ("kbuild: use .NOTINTERMEDIATE for future GNU Make versions")
Fixes: fbb5c0606fa4 ("kbuild: add syscall table generation to scripts/Makefile.asm-headers")
Closes: https://lore.kernel.org/lkml/91b10591-1554-4860-8843-01c6cfd7de13@app.fastmail.com/T/#m4c979c42d0c086f616e41b4ca76f2873902b8a25
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
----
I'm still a bit confused by the way this works in detail, hopefully
Masahiro Yamada can either confirm that this is a correct fix or provide
a better one.
---
 scripts/Kbuild.include | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
index faf37bafa3f8..aa1ffaeb8fc0 100644
--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -267,6 +267,7 @@ endif
 # deleted files.
 ifneq ($(and $(filter notintermediate, $(.FEATURES)),$(filter-out 4.4,$(MAKE_VERSION))),)
 .NOTINTERMEDIATE:
+.SECONDARY: FORCE
 else
 .SECONDARY:
 endif
-- 
2.39.2

Re: [PATCH] kbuild: mark "FORCE" target as secondary

[Masahiro Yamada]

On Wed, Jul 17, 2024 at 9:43 PM Arnd Bergmann <arnd@kernel.org> wrote:
>
> From: Arnd Bergmann <arnd@arndb.de>
>
> Starting with make-4.4.1, Kbuild uses the special .NOTINTERMEDIATE
> target to mark all targets as not intermediate, which slightly changes
> the behavior compared to older versions of make.
>
> This causes a build regression with my change to the system call table
> scripts now in scripts/Makefile.asm-headers, forcing a rebuild of the
> generated files with every make invocation and effectively breaking
> incremental builds.
>
> I have narrowed down the change in behavior to the way that the
> 'FORCE' target is treated: If this is marked as not intermediate,
> the $(if_changed) macro always evaluates it as a missing prerequisite,
> but if it is marked as .SECONDARY, it works like before.
>
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Cc: linux-kbuild@vger.kernel.org
> Fixes: 875ef1a57f32 ("kbuild: use .NOTINTERMEDIATE for future GNU Make versions")
> Fixes: fbb5c0606fa4 ("kbuild: add syscall table generation to scripts/Makefile.asm-headers")
> Closes: https://lore.kernel.org/lkml/91b10591-1554-4860-8843-01c6cfd7de13@app.fastmail.com/T/#m4c979c42d0c086f616e41b4ca76f2873902b8a25
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ----
> I'm still a bit confused by the way this works in detail, hopefully
> Masahiro Yamada can either confirm that this is a correct fix or provide
> a better one.


Of course, NACK.

Now I am surprised that this broken series was quickly merged
into the mainline.
I really dislike how you changed scripts/Makefile.asm-headers


I will send a correct fix.
Please do not break Kbuild even further.




> ---
>  scripts/Kbuild.include | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
> index faf37bafa3f8..aa1ffaeb8fc0 100644
> --- a/scripts/Kbuild.include
> +++ b/scripts/Kbuild.include
> @@ -267,6 +267,7 @@ endif
>  # deleted files.
>  ifneq ($(and $(filter notintermediate, $(.FEATURES)),$(filter-out 4.4,$(MAKE_VERSION))),)
>  .NOTINTERMEDIATE:
> +.SECONDARY: FORCE
>  else
>  .SECONDARY:
>  endif
> --
> 2.39.2
>
>


-- 
Best Regards
Masahiro Yamada

Re: [PATCH] kbuild: mark "FORCE" target as secondary

[Arnd Bergmann]

On Wed, Jul 17, 2024, at 17:18, Masahiro Yamada wrote:
>
> Of course, NACK.
>
> Now I am surprised that this broken series was quickly merged
> into the mainline.
> I really dislike how you changed scripts/Makefile.asm-headers
>
>
> I will send a correct fix.
> Please do not break Kbuild even further.

Should we maybe revert ff96f5c6971c..3db80c999deb then
and find a better way to do it in 6.12?

      Arnd

Re: [PATCH] kbuild: mark "FORCE" target as secondary

[Masahiro Yamada]

On Thu, Jul 18, 2024 at 1:25 AM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Wed, Jul 17, 2024, at 17:18, Masahiro Yamada wrote:
> >
> > Of course, NACK.
> >
> > Now I am surprised that this broken series was quickly merged
> > into the mainline.
> > I really dislike how you changed scripts/Makefile.asm-headers
> >
> >
> > I will send a correct fix.
> > Please do not break Kbuild even further.
>
> Should we maybe revert ff96f5c6971c..3db80c999deb then
> and find a better way to do it in 6.12?
>
>       Arnd
>
>

I do not think so.

I pretty much like 4fe53bf2ba0a45cd708dcd4c3e8e1950731b3d4d,
and I believe generating all syscall headers from a table
is the right thing to do.
So, it must happen.


Currently, the new syscall-y syntax can be used only for
architectures using the generic syscall table.
I wonder what we should do for the other legacy architectures.

Some years ago, I tried to unify 'archheaders' and 'asm-generic'
into a more generic rule, but I did not find a nice solution,
so I did not do anything. I need to find some way when I have time.




For the rebuild problem, I think this should work:
https://lore.kernel.org/all/20240717162421.1402773-1-masahiroy@kernel.org/T/#u

(I only compile-tested arch64 and riscv, but the other architectires
should be the same pattern.)



--
Best Regards
Masahiro Yamada

Re: [PATCH] kbuild: mark "FORCE" target as secondary

[Arnd Bergmann]

On Wed, Jul 17, 2024, at 18:45, Masahiro Yamada wrote:
>> On Wed, Jul 17, 2024, at 17:18, Masahiro Yamada wrote:
>>
>> Should we maybe revert ff96f5c6971c..3db80c999deb then
>> and find a better way to do it in 6.12?
>
> I do not think so.
>
> I pretty much like 4fe53bf2ba0a45cd708dcd4c3e8e1950731b3d4d,
> and I believe generating all syscall headers from a table
> is the right thing to do.
> So, it must happen.

Right, I meant to leave that part here and just revert
the Makefile changes until we have something that has
seen enough testing.

> Currently, the new syscall-y syntax can be used only for
> architectures using the generic syscall table.
> I wonder what we should do for the other legacy architectures.

I have patches for all architectures at

https://git.kernel.org/pub/scm/linux/kernel/git/arnd/asm-generic.git/log/?h=syscall-tbl-6.11&id=92499ba526eac435

but I stopped after the ones using the generic syscall
table since I had spent more time making sure the actual
table contents are unchanged.

> For the rebuild problem, I think this should work:
> https://lore.kernel.org/all/20240717162421.1402773-1-masahiroy@kernel.org/T/#u
>
> (I only compile-tested arch64 and riscv, but the other architectires
> should be the same pattern.)

Right, I completely missed the "existing-targets" bit, and
without that I couldn't figure out the rest correctly.

I have checked this on the other architectures now and
tried out the bit with the changed command line I got
wrong.

Sorry for causing these problems and thanks a lot for
the fix!

     Arnd