No progress output when make 4.4.1 builds Linux 4.19 and earlier

No progress output when make 4.4.1 builds Linux 4.19 and earlier

[Eric Biggers]

After upgrading to make v4.4.1 (released last week), there's no longer any
progress output from builds of the Linux kernel v4.19 and earlier.  It seems the
actual build still works, but it's now silent except for warnings and errors.

It bisects to the following 'make' commit:

    commit dc2d963989b96161472b2cd38cef5d1f4851ea34
    Author: Dmitry Goncharov <dgoncharov@users.sf.net>
    Date:   Sun Nov 27 14:09:17 2022 -0500

        [SV 63347] Always add command line variable assignments to MAKEFLAGS

Is this an intentional breakage from the 'make' side?

- Eric

Re: No progress output when make 4.4.1 builds Linux 4.19 and earlier

[Greg KH]

On Tue, Mar 07, 2023 at 10:13:45PM -0800, Eric Biggers wrote:
> After upgrading to make v4.4.1 (released last week), there's no longer any
> progress output from builds of the Linux kernel v4.19 and earlier.  It seems the
> actual build still works, but it's now silent except for warnings and errors.
> 
> It bisects to the following 'make' commit:
> 
>     commit dc2d963989b96161472b2cd38cef5d1f4851ea34
>     Author: Dmitry Goncharov <dgoncharov@users.sf.net>
>     Date:   Sun Nov 27 14:09:17 2022 -0500
> 
>         [SV 63347] Always add command line variable assignments to MAKEFLAGS
> 
> Is this an intentional breakage from the 'make' side?

Ah, thanks for figuring this out, it's been bugging me locally for a bit
as well!  The fact that kernels 5.4 and newer imply to me that there is
a kernel build fix that should resolve this if someone can take the time
to bisect it...

thanks,

greg k-h

Re: No progress output when make 4.4.1 builds Linux 4.19 and earlier

[Eric Biggers]

On Wed, Mar 08, 2023 at 07:17:37AM +0100, Greg KH wrote:
> On Tue, Mar 07, 2023 at 10:13:45PM -0800, Eric Biggers wrote:
> > After upgrading to make v4.4.1 (released last week), there's no longer any
> > progress output from builds of the Linux kernel v4.19 and earlier.  It seems the
> > actual build still works, but it's now silent except for warnings and errors.
> > 
> > It bisects to the following 'make' commit:
> > 
> >     commit dc2d963989b96161472b2cd38cef5d1f4851ea34
> >     Author: Dmitry Goncharov <dgoncharov@users.sf.net>
> >     Date:   Sun Nov 27 14:09:17 2022 -0500
> > 
> >         [SV 63347] Always add command line variable assignments to MAKEFLAGS
> > 
> > Is this an intentional breakage from the 'make' side?
> 
> Ah, thanks for figuring this out, it's been bugging me locally for a bit
> as well!  The fact that kernels 5.4 and newer imply to me that there is
> a kernel build fix that should resolve this if someone can take the time
> to bisect it...
> 

Fix bisection comes up with the following kernel commit:

    commit 80463f1b7bf9f822fd3495139bcf3ef254fdca10
    Author: Masahiro Yamada <yamada.masahiro@socionext.com>
    Date:   Fri Sep 14 15:33:23 2018 +0900
    
        kbuild: add --include-dir flag only for out-of-tree build

But only for in-tree builds.  Out-of-tree builds need:

    commit 3812b8c5c5d527239ac015f1f2c7654da7fcfbba
    Author: Masahiro Yamada <yamada.masahiro@socionext.com>
    Date:   Fri Feb 22 16:40:07 2019 +0900

        kbuild: make -r/-R effective in top Makefile for old Make versions

Masahiro, what dependencies (if any) do the above two commits have in order to
be backported to 4.19 and 4.14?

- Eric

Re: No progress output when make 4.4.1 builds Linux 4.19 and earlier

[Dmitry Goncharov]

On Wed, Mar 8, 2023 at 1:37 AM Greg KH <gregkh@linuxfoundation.org> wrote:
> On Tue, Mar 07, 2023 at 10:13:45PM -0800, Eric Biggers wrote:
...
> > Is this an intentional breakage from the 'make' side?
No it is not an intentional breakage.
This is a fix for https://savannah.gnu.org/bugs/?63347.

> The fact that kernels 5.4 and newer imply to me that there is
> a kernel build fix that should resolve this if someone can take the time
> to bisect it...

Kernel makefile was updated to work with old and new make in
4bf73588165ba7d32131a043775557a54b6e1db5.
If you wanted to backport, try this commit.

regards, Dmitry

Re: No progress output when make 4.4.1 builds Linux 4.19 and earlier

[Greg KH]

On Wed, Mar 08, 2023 at 08:12:57AM -0500, Dmitry Goncharov wrote:
> On Wed, Mar 8, 2023 at 1:37 AM Greg KH <gregkh@linuxfoundation.org> wrote:
> > On Tue, Mar 07, 2023 at 10:13:45PM -0800, Eric Biggers wrote:
> ...
> > > Is this an intentional breakage from the 'make' side?
> No it is not an intentional breakage.
> This is a fix for https://savannah.gnu.org/bugs/?63347.
> 
> > The fact that kernels 5.4 and newer imply to me that there is
> > a kernel build fix that should resolve this if someone can take the time
> > to bisect it...
> 
> Kernel makefile was updated to work with old and new make in
> 4bf73588165ba7d32131a043775557a54b6e1db5.
> If you wanted to backport, try this commit.

Nice, that worked for me!  I'll go queue that up for the stable kernels
now, thank you for the quick response.

greg k-h

Re: No progress output when make 4.4.1 builds Linux 4.19 and earlier

[Paul Smith]

On Wed, 2023-03-08 at 08:12 -0500, Dmitry Goncharov wrote:
> > > Is this an intentional breakage from the 'make' side?
> No it is not an intentional breakage.
> This is a fix for https://savannah.gnu.org/bugs/?63347.

Just to note, it was possible to run into this problem with earlier
versions of GNU Make as well, it just became much simpler once the
variables were available since it's easier to have an "s" in some
variable.  But it is possible to have an "s" in a MAKEFLAGS flag which
is not introduced with a "--", and doesn't represent the short option.

I give some examples in that Savannah bug.

> > The fact that kernels 5.4 and newer imply to me that there is
> > a kernel build fix that should resolve this if someone can take the
> > time to bisect it...
> 
> Kernel makefile was updated to work with old and new make in
> 4bf73588165ba7d32131a043775557a54b6e1db5.
> If you wanted to backport, try this commit.

Does anyone know why this commit is using a make version comparison? 
That seems totally unnecessary to me; am I forgetting something?  As
far as I remember,

    silence := $(findstring s,$(firstword -$(MAKEFLAGS)))

has always been the proper way to check for the short option "s", and
has always worked in every version of GNU Make.

https://github.com/torvalds/linux/commit/4bf73588165ba7d32131a043775557a54b6e1db5

Re: No progress output when make 4.4.1 builds Linux 4.19 and earlier

[Greg KH]

On Wed, Mar 08, 2023 at 08:57:34AM -0500, Paul Smith wrote:
> On Wed, 2023-03-08 at 08:12 -0500, Dmitry Goncharov wrote:
> > > > Is this an intentional breakage from the 'make' side?
> > No it is not an intentional breakage.
> > This is a fix for https://savannah.gnu.org/bugs/?63347.
> 
> Just to note, it was possible to run into this problem with earlier
> versions of GNU Make as well, it just became much simpler once the
> variables were available since it's easier to have an "s" in some
> variable.  But it is possible to have an "s" in a MAKEFLAGS flag which
> is not introduced with a "--", and doesn't represent the short option.
> 
> I give some examples in that Savannah bug.
> 
> > > The fact that kernels 5.4 and newer imply to me that there is
> > > a kernel build fix that should resolve this if someone can take the
> > > time to bisect it...
> > 
> > Kernel makefile was updated to work with old and new make in
> > 4bf73588165ba7d32131a043775557a54b6e1db5.
> > If you wanted to backport, try this commit.
> 
> Does anyone know why this commit is using a make version comparison? 
> That seems totally unnecessary to me; am I forgetting something?  As
> far as I remember,
> 
>     silence := $(findstring s,$(firstword -$(MAKEFLAGS)))
> 
> has always been the proper way to check for the short option "s", and
> has always worked in every version of GNU Make.
> 
> https://github.com/torvalds/linux/commit/4bf73588165ba7d32131a043775557a54b6e1db5

No idea, sorry, submit a patch to the kbuild maintainer and they will
probably accept it.

thanks,

greg k-h

Re: No progress output when make 4.4.1 builds Linux 4.19 and earlier

[Dmitry Goncharov]

On Wed, Mar 8, 2023 at 8:58 AM Paul Smith <psmith@gnu.org> wrote:
> Does anyone know why this commit is using a make version comparison?

Kernel build system is expected to work with gnu make-3.82.
make-3.82 puts long options before short ones.

$ cat makefile
$(info at parse time makeflags=$(MAKEFLAGS))
all:; $(info at build time makeflags=$(MAKEFLAGS))
$ make-3.82 --warn-undefined-variables -sR
at parse time makeflags= --warn-undefined-variables -sRr
at build time makeflags= --warn-undefined-variables -sRr

regards, Dmitry

RE: No progress output when make 4.4.1 builds Linux 4.19 and earlier

[David Laight]

...
> Does anyone know why this commit is using a make version comparison?
> That seems totally unnecessary to me; am I forgetting something?  As
> far as I remember,
> 
>     silence := $(findstring s,$(firstword -$(MAKEFLAGS)))

Adding a $(filter-out --%,...) should help with old makes:
Probably:

	silence := $(findstring s,$(firstword $(filter-out --%,$(MAKEFLAGS))))

    David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

Re: No progress output when make 4.4.1 builds Linux 4.19 and earlier

[Paul Smith]

On Wed, 2023-03-08 at 15:53 +0000, David Laight wrote:
> Adding a $(filter-out --%,...) should help with old makes:

Yes that's what is there.  See the link to the commit I posted.

I just couldn't remember why it was necessary to treat things
differently between different versions.  Dmitry reminded me.