[PATCH] Fix compiler message generation

[PATCH] Fix compiler message generation

[David Howells]

The following commit:

	commit 9da0763bdd82572be243fcf5161734f11568960f
	Author: Michal Marek <mmarek@suse.cz>
	Date:   Fri Apr 25 23:25:18 2014 +0200
	Subject: kbuild: Use relative path when building in a subdir of the source tree

makes compiler messages relative to the *build* tree if the build tree is a
subdirectory at the root of the source tree.

This is the wrong thing to do since the make command is issued in the *source*
tree and so any editor or IDE that issues the make command will likely expect
paths in warnings and errors to be relative either to the current directory at
the time the make was issued or to the directory in which make was run.

Certainly, this is something I'm seeing with emacs.  I do:

	LANG=C nice -19 make O=build -C /data/fs/linux-2.6-fscache -j4

And then I get error messages that look like:

	../fs/namei.c: In function 'SYSC_linkat':
	../fs/namei.c:4836:57: error: expected declaration specifiers before 'x'
		int, newdfd, const char __user *, newname, int, flags)x

which emacs can't find the source for because it doesn't relate to anything
emacs knows about.

As a temporary measure, fix this by substituting the full path of the source
as make knows it.

I suspect the correct fix from make's point of view is to issue the build
command in the build tree and use VPATH to refer to the source, but that would
likely involve making a lot of Makefiles and would involve a step equivalent
to autoconf - which is probably not what we want.

Signed-off-by: David Howells <dhowells@redhat.com>
cc: Michal Marek <mmarek@suse.cz>
cc: Sam Ravnborg <sam@ravnborg.org>
---

 Makefile |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/Makefile b/Makefile
index 1317563..def4a75 100644
--- a/Makefile
+++ b/Makefile
@@ -155,7 +155,7 @@ ifeq ($(KBUILD_SRC),)
 else
         ifeq ($(KBUILD_SRC)/,$(dir $(CURDIR)))
                 # building in a subdirectory of the source tree
-                srctree := ..
+                srctree := $(realpath ..)
         else
                 srctree := $(KBUILD_SRC)
         endif

Re: [PATCH] Fix compiler message generation

[Michal Marek]

On Wed, Jul 02, 2014 at 11:56:46AM +0100, David Howells wrote:
> The following commit:
> 
> 	commit 9da0763bdd82572be243fcf5161734f11568960f
> 	Author: Michal Marek <mmarek@suse.cz>
> 	Date:   Fri Apr 25 23:25:18 2014 +0200
> 	Subject: kbuild: Use relative path when building in a subdir of the source tree
> 
> makes compiler messages relative to the *build* tree if the build tree is a
> subdirectory at the root of the source tree.
> 
> This is the wrong thing to do since the make command is issued in the *source*
> tree and so any editor or IDE that issues the make command will likely expect
> paths in warnings and errors to be relative either to the current directory at
> the time the make was issued or to the directory in which make was run.
> 
> Certainly, this is something I'm seeing with emacs.  I do:
> 
> 	LANG=C nice -19 make O=build -C /data/fs/linux-2.6-fscache -j4
> 
> And then I get error messages that look like:
> 
> 	../fs/namei.c: In function 'SYSC_linkat':
> 	../fs/namei.c:4836:57: error: expected declaration specifiers before 'x'
> 		int, newdfd, const char __user *, newname, int, flags)x
> 
> which emacs can't find the source for because it doesn't relate to anything
> emacs knows about.
> 
> As a temporary measure, fix this by substituting the full path of the source
> as make knows it.

Boaz Harrosh hit the same problem with kdevelop and posted a patch that
adds a variable to turn off relative paths:
https://lkml.org/lkml/2014/6/19/295. This has the downside that one has
to remember to set the variable.

However, VPATH builds are not that uncommon, so I guess the editors and
IDEs can actually handle them if make prints the "Entering directory
`blah/blah'" messages. We disable this in kbuild to not pollute the log,
but how about the following patch? I tried it with vim and it worked:


>From 5b59dcacf358f143b9fb39d2f788142ab9ba3e00 Mon Sep 17 00:00:00 2001
From: Michal Marek <mmarek@suse.cz>
Date: Wed, 2 Jul 2014 14:28:26 +0200
Subject: [PATCH] kbuild: Print the name of the build directory

With commit 9da0763b (kbuild: Use relative path when building in a
subdir of the source tree), the compiler messages include relative
paths. These are however relative to the build directory, not the
directory where make was started. Print the "Entering directory ..."
message once, so that IDEs/editors can find the source files.

Signed-off-by: Michal Marek <mmarek@suse.cz>
---
 Makefile | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Makefile b/Makefile
index 97b2861..40544a0 100644
--- a/Makefile
+++ b/Makefile
@@ -126,7 +126,10 @@ PHONY += $(MAKECMDGOALS) sub-make
 $(filter-out _all sub-make $(CURDIR)/Makefile, $(MAKECMDGOALS)) _all: sub-make
 	@:
 
+# Fake the "Entering directory" message once, so that IDEs/editors are
+# able to understand relative filenames.
 sub-make: FORCE
+	@echo "make[1]: Entering directory \`$(KBUILD_OUTPUT)'"
 	$(if $(KBUILD_VERBOSE:1=),@)$(MAKE) -C $(KBUILD_OUTPUT) \
 	KBUILD_SRC=$(CURDIR) \
 	KBUILD_EXTMOD="$(KBUILD_EXTMOD)" -f $(CURDIR)/Makefile \
-- 
1.8.4.5



> I suspect the correct fix from make's point of view is to issue the build
> command in the build tree and use VPATH to refer to the source, but that would
> likely involve making a lot of Makefiles and would involve a step equivalent
> to autoconf - which is probably not what we want.

The build does run in the build tree with VPATH = $(srctree). The first
make just passes over to a make -C $(objtree).

Michal

Re: [PATCH] Fix compiler message generation

[David Howells]

Michal Marek <mmarek@suse.cz> wrote:

> From 5b59dcacf358f143b9fb39d2f788142ab9ba3e00 Mon Sep 17 00:00:00 2001
> From: Michal Marek <mmarek@suse.cz>
> Date: Wed, 2 Jul 2014 14:28:26 +0200
> Subject: [PATCH] kbuild: Print the name of the build directory
> 
> With commit 9da0763b (kbuild: Use relative path when building in a
> subdir of the source tree), the compiler messages include relative
> paths. These are however relative to the build directory, not the
> directory where make was started. Print the "Entering directory ..."
> message once, so that IDEs/editors can find the source files.
> 
> Signed-off-by: Michal Marek <mmarek@suse.cz>
> ---
>  Makefile | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/Makefile b/Makefile
> index 97b2861..40544a0 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -126,7 +126,10 @@ PHONY += $(MAKECMDGOALS) sub-make
>  $(filter-out _all sub-make $(CURDIR)/Makefile, $(MAKECMDGOALS)) _all: sub-make
>  	@:
>  
> +# Fake the "Entering directory" message once, so that IDEs/editors are
> +# able to understand relative filenames.
>  sub-make: FORCE
> +	@echo "make[1]: Entering directory \`$(KBUILD_OUTPUT)'"
>  	$(if $(KBUILD_VERBOSE:1=),@)$(MAKE) -C $(KBUILD_OUTPUT) \
>  	KBUILD_SRC=$(CURDIR) \
>  	KBUILD_EXTMOD="$(KBUILD_EXTMOD)" -f $(CURDIR)/Makefile \

Works for me with emacs.

Acked-by: David Howells <dhowells@redhat.com>

Re: [PATCH] Fix compiler message generation

[Michal Marek]

On Wed, Jul 02, 2014 at 02:34:10PM +0100, David Howells wrote:
> Michal Marek <mmarek@suse.cz> wrote:
> 
> > From 5b59dcacf358f143b9fb39d2f788142ab9ba3e00 Mon Sep 17 00:00:00 2001
> > From: Michal Marek <mmarek@suse.cz>
> > Date: Wed, 2 Jul 2014 14:28:26 +0200
> > Subject: [PATCH] kbuild: Print the name of the build directory
> > 
> > With commit 9da0763b (kbuild: Use relative path when building in a
> > subdir of the source tree), the compiler messages include relative
> > paths. These are however relative to the build directory, not the
> > directory where make was started. Print the "Entering directory ..."
> > message once, so that IDEs/editors can find the source files.
> > 
> > Signed-off-by: Michal Marek <mmarek@suse.cz>
> > ---
> >  Makefile | 3 +++
> >  1 file changed, 3 insertions(+)
> > 
> > diff --git a/Makefile b/Makefile
> > index 97b2861..40544a0 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -126,7 +126,10 @@ PHONY += $(MAKECMDGOALS) sub-make
> >  $(filter-out _all sub-make $(CURDIR)/Makefile, $(MAKECMDGOALS)) _all: sub-make
> >  	@:
> >  
> > +# Fake the "Entering directory" message once, so that IDEs/editors are
> > +# able to understand relative filenames.
> >  sub-make: FORCE
> > +	@echo "make[1]: Entering directory \`$(KBUILD_OUTPUT)'"
> >  	$(if $(KBUILD_VERBOSE:1=),@)$(MAKE) -C $(KBUILD_OUTPUT) \
> >  	KBUILD_SRC=$(CURDIR) \
> >  	KBUILD_EXTMOD="$(KBUILD_EXTMOD)" -f $(CURDIR)/Makefile \
> 
> Works for me with emacs.
> 
> Acked-by: David Howells <dhowells@redhat.com>

I found a regression today... after I had sent a pull request to Linus
yesterday :-/. The message is also printed by 'make -s', which is not
what the user expects from a silent mode. The following patch fixes it:


>From 066b7ed9558087a7957a1128f27d7a3462ff117f Mon Sep 17 00:00:00 2001
From: Michal Marek <mmarek@suse.cz>
Date: Fri, 4 Jul 2014 14:29:30 +0200
Subject: [PATCH] kbuild: Do not print the build directory with make -s

Commit c2e28dc9 (kbuild: Print the name of the build directory) prints
the name of the build directory for O= builds, but we should not be
doing this in make -s mode, so that commands like

  make -s O=<dir> kernelrelease

can be used by scripts. This matches the behavior of make itself, where
the -s option implies --no-print-directory.

Signed-off-by: Michal Marek <mmarek@suse.cz>
---
 Makefile | 97 +++++++++++++++++++++++++++++++++-------------------------------
 1 file changed, 50 insertions(+), 47 deletions(-)

diff --git a/Makefile b/Makefile
index 40544a0..1a5f9f3 100644
--- a/Makefile
+++ b/Makefile
@@ -41,6 +41,29 @@ unexport GREP_OPTIONS
 # descending is started. They are now explicitly listed as the
 # prepare rule.
 
+# Beautify output
+# ---------------------------------------------------------------------------
+#
+# Normally, we echo the whole command before executing it. By making
+# that echo $($(quiet)$(cmd)), we now have the possibility to set
+# $(quiet) to choose other forms of output instead, e.g.
+#
+#         quiet_cmd_cc_o_c = Compiling $(RELDIR)/$@
+#         cmd_cc_o_c       = $(CC) $(c_flags) -c -o $@ $<
+#
+# If $(quiet) is empty, the whole command will be printed.
+# If it is set to "quiet_", only the short version will be printed.
+# If it is set to "silent_", nothing will be printed at all, since
+# the variable $(silent_cmd_cc_o_c) doesn't exist.
+#
+# A simple variant is to prefix commands with $(Q) - that's useful
+# for commands that shall be hidden in non-verbose mode.
+#
+#	$(Q)ln $@ :<
+#
+# If KBUILD_VERBOSE equals 0 then the above command will be hidden.
+# If KBUILD_VERBOSE equals 1 then the above command is displayed.
+#
 # To put more focus on warnings, be less verbose as default
 # Use 'make V=1' to see the full commands
 
@@ -51,6 +74,29 @@ ifndef KBUILD_VERBOSE
   KBUILD_VERBOSE = 0
 endif
 
+ifeq ($(KBUILD_VERBOSE),1)
+  quiet =
+  Q =
+else
+  quiet=quiet_
+  Q = @
+endif
+
+# If the user is running make -s (silent mode), suppress echoing of
+# commands
+
+ifneq ($(filter 4.%,$(MAKE_VERSION)),)	# make-4
+ifneq ($(filter %s ,$(firstword x$(MAKEFLAGS))),)
+  quiet=silent_
+endif
+else					# make-3.8x
+ifneq ($(filter s% -s%,$(MAKEFLAGS)),)
+  quiet=silent_
+endif
+endif
+
+export quiet Q KBUILD_VERBOSE
+
 # Call a source code checker (by default, "sparse") as part of the
 # C compilation.
 #
@@ -128,8 +174,11 @@ $(filter-out _all sub-make $(CURDIR)/Makefile, $(MAKECMDGOALS)) _all: sub-make
 
 # Fake the "Entering directory" message once, so that IDEs/editors are
 # able to understand relative filenames.
+       echodir := @echo
+ quiet_echodir := @echo
+silent_echodir := @:
 sub-make: FORCE
-	@echo "make[1]: Entering directory \`$(KBUILD_OUTPUT)'"
+	$($(quiet)echodir) "make[1]: Entering directory \`$(KBUILD_OUTPUT)'"
 	$(if $(KBUILD_VERBOSE:1=),@)$(MAKE) -C $(KBUILD_OUTPUT) \
 	KBUILD_SRC=$(CURDIR) \
 	KBUILD_EXTMOD="$(KBUILD_EXTMOD)" -f $(CURDIR)/Makefile \
@@ -292,52 +341,6 @@ endif
 export KBUILD_MODULES KBUILD_BUILTIN
 export KBUILD_CHECKSRC KBUILD_SRC KBUILD_EXTMOD
 
-# Beautify output
-# ---------------------------------------------------------------------------
-#
-# Normally, we echo the whole command before executing it. By making
-# that echo $($(quiet)$(cmd)), we now have the possibility to set
-# $(quiet) to choose other forms of output instead, e.g.
-#
-#         quiet_cmd_cc_o_c = Compiling $(RELDIR)/$@
-#         cmd_cc_o_c       = $(CC) $(c_flags) -c -o $@ $<
-#
-# If $(quiet) is empty, the whole command will be printed.
-# If it is set to "quiet_", only the short version will be printed.
-# If it is set to "silent_", nothing will be printed at all, since
-# the variable $(silent_cmd_cc_o_c) doesn't exist.
-#
-# A simple variant is to prefix commands with $(Q) - that's useful
-# for commands that shall be hidden in non-verbose mode.
-#
-#	$(Q)ln $@ :<
-#
-# If KBUILD_VERBOSE equals 0 then the above command will be hidden.
-# If KBUILD_VERBOSE equals 1 then the above command is displayed.
-
-ifeq ($(KBUILD_VERBOSE),1)
-  quiet =
-  Q =
-else
-  quiet=quiet_
-  Q = @
-endif
-
-# If the user is running make -s (silent mode), suppress echoing of
-# commands
-
-ifneq ($(filter 4.%,$(MAKE_VERSION)),)	# make-4
-ifneq ($(filter %s ,$(firstword x$(MAKEFLAGS))),)
-  quiet=silent_
-endif
-else					# make-3.8x
-ifneq ($(filter s% -s%,$(MAKEFLAGS)),)
-  quiet=silent_
-endif
-endif
-
-export quiet Q KBUILD_VERBOSE
-
 ifneq ($(CC),)
 ifeq ($(shell $(CC) -v 2>&1 | grep -c "clang version"), 1)
 COMPILER := clang
-- 
1.8.4.5

Re: [PATCH] Fix compiler message generation

[Doug Anderson]

Michal

On Fri, Jul 4, 2014 at 5:38 AM, Michal Marek <mmarek@suse.cz> wrote:
> On Wed, Jul 02, 2014 at 02:34:10PM +0100, David Howells wrote:
>> Michal Marek <mmarek@suse.cz> wrote:
>>
>> > From 5b59dcacf358f143b9fb39d2f788142ab9ba3e00 Mon Sep 17 00:00:00 2001
>> > From: Michal Marek <mmarek@suse.cz>
>> > Date: Wed, 2 Jul 2014 14:28:26 +0200
>> > Subject: [PATCH] kbuild: Print the name of the build directory
>> >
>> > With commit 9da0763b (kbuild: Use relative path when building in a
>> > subdir of the source tree), the compiler messages include relative
>> > paths. These are however relative to the build directory, not the
>> > directory where make was started. Print the "Entering directory ..."
>> > message once, so that IDEs/editors can find the source files.
>> >
>> > Signed-off-by: Michal Marek <mmarek@suse.cz>
>> > ---
>> >  Makefile | 3 +++
>> >  1 file changed, 3 insertions(+)
>> >
>> > diff --git a/Makefile b/Makefile
>> > index 97b2861..40544a0 100644
>> > --- a/Makefile
>> > +++ b/Makefile
>> > @@ -126,7 +126,10 @@ PHONY += $(MAKECMDGOALS) sub-make
>> >  $(filter-out _all sub-make $(CURDIR)/Makefile, $(MAKECMDGOALS)) _all: sub-make
>> >     @:
>> >
>> > +# Fake the "Entering directory" message once, so that IDEs/editors are
>> > +# able to understand relative filenames.
>> >  sub-make: FORCE
>> > +   @echo "make[1]: Entering directory \`$(KBUILD_OUTPUT)'"
>> >     $(if $(KBUILD_VERBOSE:1=),@)$(MAKE) -C $(KBUILD_OUTPUT) \
>> >     KBUILD_SRC=$(CURDIR) \
>> >     KBUILD_EXTMOD="$(KBUILD_EXTMOD)" -f $(CURDIR)/Makefile \
>>
>> Works for me with emacs.
>>
>> Acked-by: David Howells <dhowells@redhat.com>
>
> I found a regression today... after I had sent a pull request to Linus
> yesterday :-/. The message is also printed by 'make -s', which is not
> what the user expects from a silent mode. The following patch fixes it:
>
>
> From 066b7ed9558087a7957a1128f27d7a3462ff117f Mon Sep 17 00:00:00 2001
> From: Michal Marek <mmarek@suse.cz>
> Date: Fri, 4 Jul 2014 14:29:30 +0200
> Subject: [PATCH] kbuild: Do not print the build directory with make -s
>
> Commit c2e28dc9 (kbuild: Print the name of the build directory) prints
> the name of the build directory for O= builds, but we should not be
> doing this in make -s mode, so that commands like
>
>   make -s O=<dir> kernelrelease
>
> can be used by scripts. This matches the behavior of make itself, where
> the -s option implies --no-print-directory.
>
> Signed-off-by: Michal Marek <mmarek@suse.cz>
> ---
>  Makefile | 97 +++++++++++++++++++++++++++++++++-------------------------------
>  1 file changed, 50 insertions(+), 47 deletions(-)

This fixes the build error I was running into.  Our build system
effectively calls "make -s kernelrelease" and was getting tripped up.
I haven't extensively reviewed your fix but I have tested it.

Tested-by: Doug Anderson <dianders@chromium.org>

Re: [PATCH] Fix compiler message generation

[Boaz Harrosh]

On 07/02/2014 03:40 PM, Michal Marek wrote:
<>
>>From 5b59dcacf358f143b9fb39d2f788142ab9ba3e00 Mon Sep 17 00:00:00 2001
> From: Michal Marek <mmarek@suse.cz>
> Date: Wed, 2 Jul 2014 14:28:26 +0200
> Subject: [PATCH] kbuild: Print the name of the build directory
> 
> With commit 9da0763b (kbuild: Use relative path when building in a
> subdir of the source tree), the compiler messages include relative
> paths. These are however relative to the build directory, not the
> directory where make was started. Print the "Entering directory ..."
> message once, so that IDEs/editors can find the source files.
> 
> Signed-off-by: Michal Marek <mmarek@suse.cz>
> ---
>  Makefile | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/Makefile b/Makefile
> index 97b2861..40544a0 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -126,7 +126,10 @@ PHONY += $(MAKECMDGOALS) sub-make
>  $(filter-out _all sub-make $(CURDIR)/Makefile, $(MAKECMDGOALS)) _all: sub-make
>  	@:
>  
> +# Fake the "Entering directory" message once, so that IDEs/editors are
> +# able to understand relative filenames.
>  sub-make: FORCE
> +	@echo "make[1]: Entering directory \`$(KBUILD_OUTPUT)'"
>  	$(if $(KBUILD_VERBOSE:1=),@)$(MAKE) -C $(KBUILD_OUTPUT) \
>  	KBUILD_SRC=$(CURDIR) \
>  	KBUILD_EXTMOD="$(KBUILD_EXTMOD)" -f $(CURDIR)/Makefile \
> 

Sir Michal, sorry for the late response

This patch by itself works with kdevelop so I hope you did not submit my crap
KBUILD_FULL_PATH patch at all and only added this one (and I know, the fix to this
one with the -s)

For me I do not have use for KBUILD_FULL_PATH so I do not see any point for it.

Also not that I like your patches because now I can compile the same directory
from different machines with different absolute paths to the source and the
compilation will come out the same. (before changed absolute path would cause
a rebuild)

Thanks
Boaz

Re: [PATCH] Fix compiler message generation

[Michal Marek]

On 2014-07-16 18:15, Boaz Harrosh wrote:
> This patch by itself works with kdevelop so I hope you did not submit my crap
> KBUILD_FULL_PATH patch at all and only added this one (and I know, the fix to this
> one with the -s)

Yes, I did not merge the KBUILD_FULL_PATH patch, because the approach
with the make message works out of the box.


> Also not that I like your patches because now I can compile the same directory
> from different machines with different absolute paths to the source and the
> compilation will come out the same. (before changed absolute path would cause
> a rebuild)

That's a nice side effect which I did not realize initially ;).

Michal

Re: [PATCH] Fix compiler message generation

[Boaz Harrosh]

On 07/17/2014 12:09 PM, Michal Marek wrote:
> On 2014-07-16 18:15, Boaz Harrosh wrote:
>> This patch by itself works with kdevelop so I hope you did not submit my crap
>> KBUILD_FULL_PATH patch at all and only added this one (and I know, the fix to this
>> one with the -s)
> 
> Yes, I did not merge the KBUILD_FULL_PATH patch, because the approach
> with the make message works out of the box.
> 
> 
>> Also not that I like your patches because now I can compile the same directory
>> from different machines with different absolute paths to the source and the
>> compilation will come out the same. (before changed absolute path would cause
>> a rebuild)
> 

Rrrr I hope you understood that not => Note above. I *do* like your patch a lot

> That's a nice side effect which I did not realize initially ;).
> 

I guess this is everywhere that has __FILE__ in code needs to be rebuilt
So it is more in debug builds than ndebug.

> Michal
> 

Ye cool thanks, all is well
Boaz