'make headers_check' failed to install headers to arbitrary location

'make headers_check' failed to install headers to arbitrary location

[Sergei Poselenov]

Hello,

I'm trying to install kernel headers to build a cross-toolchain, but got
the following:

make ARCH=arm 
INSTALL_HDR_PATH=/work/psl/eldk-builds/arm-2009-04-21/work/var/tmp/crosstool-0.43-3-root/usr/crosstool/gcc-4.2.2-glibc-20070515T2025-eldk/arm-linux-gnueabi/arm-linux-gnueabi/ 
headers_check
...
   CHECK   include/linux/raid (2 files)
   CHECK   include/linux/spi (1 files)
   CHECK   include/linux/sunrpc (1 files)
   CHECK   include/linux/tc_act (6 files)
   CHECK   include/linux/tc_ematch (4 files)
   CHECK   include/linux/usb (8 files)
make[2]: execvp: /bin/sh: Argument list too long
make[2]: *** 
[/work/psl/eldk-builds/arm-2009-04-21/work/var/tmp/crosstool-0.43-3-root/usr/crosstool/gcc-4.2.2-glibc-20070515T2025-eldk/arm-linux-gnueabi/arm-linux-gnueabi//include/linux/.check] 
Error 127
make[1]: *** [linux] Error 2
make: *** [headers_check] Error 2
->


The kernel version is 2.6.28

However, this works fine for 2.6.24.2

Running make with '-n', I've found the problem is that the filelist 
passed as argument to scripts/headers_check.pl apparently is too big.

The error depends also on a bash version, it failed on 2.05a, but passed
on 3.2.39.  Unfortunately, bash (or the hole host system) upgrade is not
an option. Also, I'm sure I could find a path length big enough to 
reproduce the error on a modern bash.

What would you suggest to resolve the problem?

Thanks for any help.

Regards,
Sergei

Re: 'make headers_check' failed to install headers to arbitrary location

[Arnd Bergmann]

On Thursday 14 May 2009, Sergei Poselenov wrote:
>    CHECK   include/linux/tc_ematch (4 files)
>    CHECK   include/linux/usb (8 files)
> make[2]: execvp: /bin/sh: Argument list too long
> make[2]: *** 

Can you try this patch?

	Arnd <><

--- a/scripts/Makefile.headersinst
+++ b/scripts/Makefile.headersinst
@@ -54,8 +54,8 @@ quiet_cmd_remove = REMOVE  $(unwanted)
       cmd_remove = rm -f $(unwanted-file)
 
 quiet_cmd_check = CHECK   $(printdir) ($(words $(all-files)) files)
-      cmd_check = $(PERL) $< $(INSTALL_HDR_PATH)/include $(SRCARCH) \
-                  $(addprefix $(install)/, $(all-files));           \
+      cmd_check = echo $(addprefix $(install)/, $(all-files)) | \
+		  xargs $(PERL) $< $(INSTALL_HDR_PATH)/include $(SRCARCH) ; \
 	          touch $@
 
 PHONY += __headersinst __headerscheck

Re: 'make headers_check' failed to install headers to arbitrary location

[Sergei Poselenov]

Hello Arnd,

Thanks, but it doesn't help. The same error.

This is actually not a perl error, but the shell's
passing a list of files with added $(install) prefix.

Regards,
Sergei

Arnd Bergmann wrote:
> On Thursday 14 May 2009, Sergei Poselenov wrote:
>>    CHECK   include/linux/tc_ematch (4 files)
>>    CHECK   include/linux/usb (8 files)
>> make[2]: execvp: /bin/sh: Argument list too long
>> make[2]: *** 
> 
> Can you try this patch?
> 
> 	Arnd <><
> 
> --- a/scripts/Makefile.headersinst
> +++ b/scripts/Makefile.headersinst
> @@ -54,8 +54,8 @@ quiet_cmd_remove = REMOVE  $(unwanted)
>        cmd_remove = rm -f $(unwanted-file)
>  
>  quiet_cmd_check = CHECK   $(printdir) ($(words $(all-files)) files)
> -      cmd_check = $(PERL) $< $(INSTALL_HDR_PATH)/include $(SRCARCH) \
> -                  $(addprefix $(install)/, $(all-files));           \
> +      cmd_check = echo $(addprefix $(install)/, $(all-files)) | \
> +		  xargs $(PERL) $< $(INSTALL_HDR_PATH)/include $(SRCARCH) ; \
>  	          touch $@
>  
>  PHONY += __headersinst __headerscheck

[PATCH] Re: 'make headers_check' failed to install headers to arbitrary location

[Sergei Poselenov]

Hello all,

A time ago I noted a 'make headers_check' failure, see the
original message here:
http://lkml.org/lkml/2009/5/14/248

The proposed patch fixes this error, please review.
Tested on 2.6.30-rc5.

Signed-off-by: Sergei Poselenov <sposelenov@emcraft.com>
---
  scripts/Makefile.headersinst |    5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/scripts/Makefile.headersinst b/scripts/Makefile.headersinst
index 095cfc8..2bd5fe4 100644
--- a/scripts/Makefile.headersinst
+++ b/scripts/Makefile.headersinst
@@ -54,8 +54,9 @@ quiet_cmd_remove = REMOVE  $(unwanted)
        cmd_remove = rm -f $(unwanted-file)

  quiet_cmd_check = CHECK   $(printdir) ($(words $(all-files)) files)
-      cmd_check = $(PERL) $< $(INSTALL_HDR_PATH)/include $(SRCARCH) \
-                  $(addprefix $(install)/, $(all-files));           \
+      cmd_check = (for f in $(all-files); do                        \
+                  $(PERL) $< $(INSTALL_HDR_PATH)/include $(SRCARCH) \
+                  $(install)/$${f}; done);                          \
  	          touch $@

  PHONY += __headersinst __headerscheck
-- 
1.6.0.6

Re: [PATCH] Re: 'make headers_check' failed to install headers to arbitrary location

[Sergei Poselenov]

Hello all,

On Thu, 21 May 2009 17:46:59 +0400
Sergei Poselenov <sposelenov@emcraft.com> wrote:

> Hello all,
> 
> A time ago I noted a 'make headers_check' failure, see the
> original message here:
> http://lkml.org/lkml/2009/5/14/248
> 
> The proposed patch fixes this error, please review.
> Tested on 2.6.30-rc5.
> 

Here is another version of the patch which fixes
the "Argument list too long" error for "make headers_check", used xargs
to minimize the perl calls (thanks, Wolfgang).

Signed-off-by: Sergei Poselenov <sposelenov@emcraft.com>
---
 scripts/Makefile.headersinst |   10 ++++++++--
 1 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/scripts/Makefile.headersinst b/scripts/Makefile.headersinst
index 095cfc8..4c54d22 100644
--- a/scripts/Makefile.headersinst
+++ b/scripts/Makefile.headersinst
@@ -54,8 +54,14 @@ quiet_cmd_remove = REMOVE  $(unwanted)
       cmd_remove = rm -f $(unwanted-file)
 
 quiet_cmd_check = CHECK   $(printdir) ($(words $(all-files)) files)
-      cmd_check = $(PERL) $< $(INSTALL_HDR_PATH)/include $(SRCARCH) \
-                  $(addprefix $(install)/, $(all-files));           \
+# Headers list can be pretty long, xargs helps to avoid
+# the "Argument list too long" error.
+      cmd_check = rm -f hdrlist; touch hdrlist;                      \
+                  for f in $(all-files); do                          \
+                  echo "$(install)/$${f} " >> hdrlist; done;         \
+                  cat hdrlist | xargs                                \
+                  $(PERL) $< $(INSTALL_HDR_PATH)/include $(SRCARCH); \
+                  rm -f hdrlist;                                     \
 	          touch $@
 
 PHONY += __headersinst __headerscheck
-- 
1.6.0.6

Re: [PATCH] Re: 'make headers_check' failed to install headers to arbitrary location

[Wolfgang Denk]

Dear Sergei,

In message <20090604175536.2a1d0add@emcraft.com> you wrote:
> 
> Here is another version of the patch which fixes
> the "Argument list too long" error for "make headers_check", used xargs
> to minimize the perl calls (thanks, Wolfgang).

Hm... but now we have a useless use of cat and a temp file.

> -      cmd_check = $(PERL) $< $(INSTALL_HDR_PATH)/include $(SRCARCH) \
> -                  $(addprefix $(install)/, $(all-files));           \
> +# Headers list can be pretty long, xargs helps to avoid
> +# the "Argument list too long" error.
> +      cmd_check = rm -f hdrlist; touch hdrlist;                      \
> +                  for f in $(all-files); do                          \
> +                  echo "$(install)/$${f} " >> hdrlist; done;         \
> +                  cat hdrlist | xargs                                \
> +                  $(PERL) $< $(INSTALL_HDR_PATH)/include $(SRCARCH); \
> +                  rm -f hdrlist;                                     \

Why not simply:

	for f in $(all-files); do
		echo "$(install)/$${f}"
	done | xargs $(PERL) ...

?

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
A witty saying proves nothing, but saying  something  pointless  gets
people's attention.

Re: [PATCH] Re: 'make headers_check' failed to install headers to arbitrary location

[Sergei Poselenov]

On Thu, 04 Jun 2009 19:07:15 +0200
Dear Wolfgang,

Wolfgang Denk <wd@denx.de> wrote:

> Why not simply:
> 
> 	for f in $(all-files); do
> 		echo "$(install)/$${f}"
> 	done | xargs $(PERL) ...
> 
> ?
> 

Ah,right, I missed this.

Here is the modified version of the patch, tested on 2.6.30-rc8,
ARCH=arm

Subject: [PATCH] Fixed the "Argument list too long" error for "make headers_check",
 used xargs to minimize the perl calls.

Signed-off-by: Sergei Poselenov <sposelenov@emcraft.com>
---
 scripts/Makefile.headersinst |    8 ++++++--
 1 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/scripts/Makefile.headersinst b/scripts/Makefile.headersinst
index 095cfc8..0fcd838 100644
--- a/scripts/Makefile.headersinst
+++ b/scripts/Makefile.headersinst
@@ -54,8 +54,12 @@ quiet_cmd_remove = REMOVE  $(unwanted)
       cmd_remove = rm -f $(unwanted-file)
 
 quiet_cmd_check = CHECK   $(printdir) ($(words $(all-files)) files)
-      cmd_check = $(PERL) $< $(INSTALL_HDR_PATH)/include $(SRCARCH) \
-                  $(addprefix $(install)/, $(all-files));           \
+# Headers list can be pretty long, xargs helps to avoid
+# the "Argument list too long" error.
+      cmd_check = for f in $(all-files); do                          \
+                  echo "$(install)/$${f}"; done                      \
+                  | xargs                                            \
+                  $(PERL) $< $(INSTALL_HDR_PATH)/include $(SRCARCH); \
 	          touch $@
 
 PHONY += __headersinst __headerscheck
-- 
1.6.0.6

Re: [PATCH] Re: 'make headers_check' failed to install headers to arbitrary location

[Sam Ravnborg]

On Fri, Jun 05, 2009 at 04:11:09PM +0400, Sergei Poselenov wrote:
> On Thu, 04 Jun 2009 19:07:15 +0200
> Dear Wolfgang,
> 
> Wolfgang Denk <wd@denx.de> wrote:
> 
> > Why not simply:
> > 
> > 	for f in $(all-files); do
> > 		echo "$(install)/$${f}"
> > 	done | xargs $(PERL) ...
> > 
> > ?
> > 
> 
> Ah,right, I missed this.
> 
> Here is the modified version of the patch, tested on 2.6.30-rc8,
> ARCH=arm
> 
> Subject: [PATCH] Fixed the "Argument list too long" error for "make headers_check",
>  used xargs to minimize the perl calls.
> 
> Signed-off-by: Sergei Poselenov <sposelenov@emcraft.com>

I extended the chagelog with the original error message
and applied this.

	Sam