[PATCH 0/1] coreutils: fix do_configure error with long TMPDIR

[PATCH 0/1] coreutils: fix do_configure error with long TMPDIR

[wenzong.fan]

From: Wenzong Fan <wenzong.fan@windriver.com>

By default the 'm4/*' files will be extended with absolute path while
autoreconf running, if the length of TMPDIR is too long such as 410
which is the maximum value allowed by 'sanity.bbclass', autoreconf
will fail with error:
    
    Can't exec "/bin/sh": Argument list too long ...
    
Fix this issue by specific the 'acpaths' to '-I ./m4'.

The following changes since commit 91ece5d5668ee1adb5c5e3d757426a0d5187bd84:

  libpam: Fix missing DESTDIR for a mkdir causing build failures (2012-08-19 13:32:06 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib wenzong/2766
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=wenzong/2766

Wenzong Fan (1):
  coreutils: fix do_configure error with long TMPDIR

 meta/recipes-core/coreutils/coreutils_8.14.bb |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

-- 
1.7.9.5

[PATCH 1/1] coreutils: fix do_configure error with long TMPDIR

[wenzong.fan]

From: Wenzong Fan <wenzong.fan@windriver.com>

By default the 'm4/*' files will be extended with absolute path while
autoreconf running, if the length of TMPDIR is too long such as 410
which is the maximum value allowed by 'sanity.bbclass', autoreconf
will fail with error:

    Can't exec "/bin/sh": Argument list too long ...

Fix this issue by specific the 'acpaths' to '-I ./m4'.

[YOCTO #2766]

Signed-off-by: Wenzong Fan <wenzong.fan@windriver.com>
---
 meta/recipes-core/coreutils/coreutils_8.14.bb |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/coreutils/coreutils_8.14.bb b/meta/recipes-core/coreutils/coreutils_8.14.bb
index 9a714a9..c7c8e13 100644
--- a/meta/recipes-core/coreutils/coreutils_8.14.bb
+++ b/meta/recipes-core/coreutils/coreutils_8.14.bb
@@ -7,7 +7,7 @@ BUGTRACKER = "http://debbugs.gnu.org/coreutils"
 LICENSE = "GPLv3+"
 LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504\
                     file://src/ls.c;startline=5;endline=16;md5=e1a509558876db58fb6667ba140137ad"
-PR = "r5"
+PR = "r6"
 DEPENDS = "gmp libcap"
 DEPENDS_virtclass-native = ""
 
@@ -36,6 +36,8 @@ base_bindir_progs = "cat chgrp chmod chown cp date dd echo false kill ln ls mkdi
 
 sbindir_progs= "chroot"
 
+acpaths = "-I ./m4"
+
 do_install_append() {
 	for i in df mktemp base64; do mv ${D}${bindir}/$i ${D}${bindir}/$i.${BPN}; done
 
-- 
1.7.9.5

Re: [PATCH 1/1] coreutils: fix do_configure error with long TMPDIR

[Saul Wold]

On 08/19/2012 07:55 PM, wenzong.fan@windriver.com wrote:
> From: Wenzong Fan <wenzong.fan@windriver.com>
>
> By default the 'm4/*' files will be extended with absolute path while
> autoreconf running, if the length of TMPDIR is too long such as 410
> which is the maximum value allowed by 'sanity.bbclass', autoreconf
> will fail with error:
>
>      Can't exec "/bin/sh": Argument list too long ...
>
> Fix this issue by specific the 'acpaths' to '-I ./m4'.
>
> [YOCTO #2766]
>
> Signed-off-by: Wenzong Fan <wenzong.fan@windriver.com>
> ---
>   meta/recipes-core/coreutils/coreutils_8.14.bb |    4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/meta/recipes-core/coreutils/coreutils_8.14.bb b/meta/recipes-core/coreutils/coreutils_8.14.bb
> index 9a714a9..c7c8e13 100644
> --- a/meta/recipes-core/coreutils/coreutils_8.14.bb
> +++ b/meta/recipes-core/coreutils/coreutils_8.14.bb
> @@ -7,7 +7,7 @@ BUGTRACKER = "http://debbugs.gnu.org/coreutils"
>   LICENSE = "GPLv3+"
>   LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504\
>                       file://src/ls.c;startline=5;endline=16;md5=e1a509558876db58fb6667ba140137ad"
> -PR = "r5"
> +PR = "r6"
>   DEPENDS = "gmp libcap"
>   DEPENDS_virtclass-native = ""
>
> @@ -36,6 +36,8 @@ base_bindir_progs = "cat chgrp chmod chown cp date dd echo false kill ln ls mkdi
>
>   sbindir_progs= "chroot"
>
> +acpaths = "-I ./m4"
> +
This is a poor work-around to the problem, and does not give any idea as 
to what your fixing.  Please investigate a proper fix not just a 
work-around.

Thanks
	Sau!

>   do_install_append() {
>   	for i in df mktemp base64; do mv ${D}${bindir}/$i ${D}${bindir}/$i.${BPN}; done
>
>

Re: [PATCH 1/1] coreutils: fix do_configure error with long TMPDIR

[wenzong fan]

On 08/24/2012 12:28 AM, Saul Wold wrote:
> On 08/19/2012 07:55 PM, wenzong.fan@windriver.com wrote:
>> From: Wenzong Fan <wenzong.fan@windriver.com>
>>
>> By default the 'm4/*' files will be extended with absolute path while
>> autoreconf running, if the length of TMPDIR is too long such as 410
>> which is the maximum value allowed by 'sanity.bbclass', autoreconf
>> will fail with error:
>>
>> Can't exec "/bin/sh": Argument list too long ...
>>
>> Fix this issue by specific the 'acpaths' to '-I ./m4'.
>>
>> [YOCTO #2766]
>>
>> Signed-off-by: Wenzong Fan <wenzong.fan@windriver.com>
>> ---
>> meta/recipes-core/coreutils/coreutils_8.14.bb | 4 +++-
>> 1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/meta/recipes-core/coreutils/coreutils_8.14.bb
>> b/meta/recipes-core/coreutils/coreutils_8.14.bb
>> index 9a714a9..c7c8e13 100644
>> --- a/meta/recipes-core/coreutils/coreutils_8.14.bb
>> +++ b/meta/recipes-core/coreutils/coreutils_8.14.bb
>> @@ -7,7 +7,7 @@ BUGTRACKER = "http://debbugs.gnu.org/coreutils"
>> LICENSE = "GPLv3+"
>> LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504\
>> file://src/ls.c;startline=5;endline=16;md5=e1a509558876db58fb6667ba140137ad"
>>
>> -PR = "r5"
>> +PR = "r6"
>> DEPENDS = "gmp libcap"
>> DEPENDS_virtclass-native = ""
>>
>> @@ -36,6 +36,8 @@ base_bindir_progs = "cat chgrp chmod chown cp date
>> dd echo false kill ln ls mkdi
>>
>> sbindir_progs= "chroot"
>>
>> +acpaths = "-I ./m4"
>> +
> This is a poor work-around to the problem, and does not give any idea as
> to what your fixing. Please investigate a proper fix not just a
> work-around.

Hi Saul,

Because a too long TMPDIR will caused the failure:

Can't exec "/bin/sh": Argument list too long at 
/usr/lib/perl/5.10/IO/File.pm line 66.

So we have a limitation in the sanity.bbclass that the tmpdir can't be 
longer than 410. See commit 2dee999ab2731c2a96e82762ae04fe44f27edac6.

This limitation works for mostly packages but except for coreutils. 
There are 378 m4 files under '.../coreutils-8.14/m4', so if all of them 
extended with absolute path while the TMPDIR is long enough, it will be 
a very long argument list and over the ARG_MAX value that kernel defined.

I think the best solution is reducing the length of argument list, 
because autoreconf will be run in $S dir, so absolute path is not 
necessary here. The acpaths is generated by autotools.bbclass and allow 
to overwrite by a pre-defined value. Another option is getting 
autotools.bbclass generate acpatchs with relative path directly, this 
might work, but will effect all autotool packages instead of coreutils only.


Thanks
Wenzong

>
> Thanks
> Sau!
>
>> do_install_append() {
>> for i in df mktemp base64; do mv ${D}${bindir}/$i
>> ${D}${bindir}/$i.${BPN}; done
>>
>>
>

Re: [PATCH 1/1] coreutils: fix do_configure error with long TMPDIR

[wenzong fan]

On 08/24/2012 12:33 PM, wenzong fan wrote:
> On 08/24/2012 12:28 AM, Saul Wold wrote:
>> On 08/19/2012 07:55 PM, wenzong.fan@windriver.com wrote:
>>> From: Wenzong Fan <wenzong.fan@windriver.com>
>>>
>>> By default the 'm4/*' files will be extended with absolute path while
>>> autoreconf running, if the length of TMPDIR is too long such as 410
>>> which is the maximum value allowed by 'sanity.bbclass', autoreconf
>>> will fail with error:
>>>
>>> Can't exec "/bin/sh": Argument list too long ...
>>>
>>> Fix this issue by specific the 'acpaths' to '-I ./m4'.
>>>
>>> [YOCTO #2766]
>>>
>>> Signed-off-by: Wenzong Fan <wenzong.fan@windriver.com>
>>> ---
>>> meta/recipes-core/coreutils/coreutils_8.14.bb | 4 +++-
>>> 1 file changed, 3 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/meta/recipes-core/coreutils/coreutils_8.14.bb
>>> b/meta/recipes-core/coreutils/coreutils_8.14.bb
>>> index 9a714a9..c7c8e13 100644
>>> --- a/meta/recipes-core/coreutils/coreutils_8.14.bb
>>> +++ b/meta/recipes-core/coreutils/coreutils_8.14.bb
>>> @@ -7,7 +7,7 @@ BUGTRACKER = "http://debbugs.gnu.org/coreutils"
>>> LICENSE = "GPLv3+"
>>> LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504\
>>> file://src/ls.c;startline=5;endline=16;md5=e1a509558876db58fb6667ba140137ad"
>>>
>>>
>>> -PR = "r5"
>>> +PR = "r6"
>>> DEPENDS = "gmp libcap"
>>> DEPENDS_virtclass-native = ""
>>>
>>> @@ -36,6 +36,8 @@ base_bindir_progs = "cat chgrp chmod chown cp date
>>> dd echo false kill ln ls mkdi
>>>
>>> sbindir_progs= "chroot"
>>>
>>> +acpaths = "-I ./m4"
>>> +
>> This is a poor work-around to the problem, and does not give any idea as
>> to what your fixing. Please investigate a proper fix not just a
>> work-around.
>
> Hi Saul,
>
> Because a too long TMPDIR will caused the failure:
>
> Can't exec "/bin/sh": Argument list too long at
> /usr/lib/perl/5.10/IO/File.pm line 66.
>
> So we have a limitation in the sanity.bbclass that the tmpdir can't be
> longer than 410. See commit 2dee999ab2731c2a96e82762ae04fe44f27edac6.
>
> This limitation works for mostly packages but except for coreutils.
> There are 378 m4 files under '.../coreutils-8.14/m4', so if all of them
> extended with absolute path while the TMPDIR is long enough, it will be
> a very long argument list and over the ARG_MAX value that kernel defined.

The ARG_MAX from '/usr/include/linux/limits.h' is '131072'. Only the 
m4/* extended will over than 378 * 410 = 154980 bytes, it's indeed a 
very long argument list.

Wenzong

>
> I think the best solution is reducing the length of argument list,
> because autoreconf will be run in $S dir, so absolute path is not
> necessary here. The acpaths is generated by autotools.bbclass and allow
> to overwrite by a pre-defined value. Another option is getting
> autotools.bbclass generate acpatchs with relative path directly, this
> might work, but will effect all autotool packages instead of coreutils
> only.
>
>
> Thanks
> Wenzong
>
>>
>> Thanks
>> Sau!
>>
>>> do_install_append() {
>>> for i in df mktemp base64; do mv ${D}${bindir}/$i
>>> ${D}${bindir}/$i.${BPN}; done
>>>
>>>
>>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>

Re: [PATCH 1/1] coreutils: fix do_configure error with long TMPDIR

[Robert Yang]

On 08/24/2012 12:28 AM, Saul Wold wrote:
> On 08/19/2012 07:55 PM, wenzong.fan@windriver.com wrote:
>> From: Wenzong Fan <wenzong.fan@windriver.com>
>>
>> By default the 'm4/*' files will be extended with absolute path while
>> autoreconf running, if the length of TMPDIR is too long such as 410
>> which is the maximum value allowed by 'sanity.bbclass', autoreconf
>> will fail with error:
>>
>>      Can't exec "/bin/sh": Argument list too long ...
>>
>> Fix this issue by specific the 'acpaths' to '-I ./m4'.
>>
>> [YOCTO #2766]
>>
>> Signed-off-by: Wenzong Fan <wenzong.fan@windriver.com>
>> ---
>>   meta/recipes-core/coreutils/coreutils_8.14.bb |    4 +++-
>>   1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/meta/recipes-core/coreutils/coreutils_8.14.bb
>> b/meta/recipes-core/coreutils/coreutils_8.14.bb
>> index 9a714a9..c7c8e13 100644
>> --- a/meta/recipes-core/coreutils/coreutils_8.14.bb
>> +++ b/meta/recipes-core/coreutils/coreutils_8.14.bb
>> @@ -7,7 +7,7 @@ BUGTRACKER = "http://debbugs.gnu.org/coreutils"
>>   LICENSE = "GPLv3+"
>>   LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504\
>>
>> file://src/ls.c;startline=5;endline=16;md5=e1a509558876db58fb6667ba140137ad"
>> -PR = "r5"
>> +PR = "r6"
>>   DEPENDS = "gmp libcap"
>>   DEPENDS_virtclass-native = ""
>>
>> @@ -36,6 +36,8 @@ base_bindir_progs = "cat chgrp chmod chown cp date dd echo
>> false kill ln ls mkdi
>>
>>   sbindir_progs= "chroot"
>>
>> +acpaths = "-I ./m4"
>> +
> This is a poor work-around to the problem, and does not give any idea as to what
> your fixing.  Please investigate a proper fix not just a work-around.
>

Hi Saul,

This is because the aclocal's argument list is very long when building
coreutils, it has many arguments (378 m4 files under 'coreutils-8.14/m4'),
when the length of the TMPDIR is short this is OK, but when it is longer,
there would be a argument list too long error.

I think that we have two solutions to fix this problem:
1) Make aclocal accept longer arguments, unfortunately, the native aclocal is
    running on the host, the length of the argument depends on the host's kernel
    (The ARG_MAX which is defined in '/usr/include/linux/limits.h'), so this way
    doesn't work.

2) Make coreutils' acpaths shorter, this is what this patch does, it changes the
    absolute path to the relative path, so it doesn't care the length of the
    TMPDIR anymore.

// Robert

> Thanks
>      Sau!
>
>>   do_install_append() {
>>       for i in df mktemp base64; do mv ${D}${bindir}/$i
>> ${D}${bindir}/$i.${BPN}; done
>>
>>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>

Re: [PATCH 1/1] coreutils: fix do_configure error with long TMPDIR

[Richard Purdie]

On Thu, 2012-09-20 at 12:37 +0800, Robert Yang wrote:
> 
> On 08/24/2012 12:28 AM, Saul Wold wrote:
> > On 08/19/2012 07:55 PM, wenzong.fan@windriver.com wrote:
> >> From: Wenzong Fan <wenzong.fan@windriver.com>
> >>
> >> By default the 'm4/*' files will be extended with absolute path while
> >> autoreconf running, if the length of TMPDIR is too long such as 410
> >> which is the maximum value allowed by 'sanity.bbclass', autoreconf
> >> will fail with error:
> >>
> >>      Can't exec "/bin/sh": Argument list too long ...
> >>
> >> Fix this issue by specific the 'acpaths' to '-I ./m4'.
> >>
> >> [YOCTO #2766]
> >>
> >> Signed-off-by: Wenzong Fan <wenzong.fan@windriver.com>
> >> ---
> >>   meta/recipes-core/coreutils/coreutils_8.14.bb |    4 +++-
> >>   1 file changed, 3 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/meta/recipes-core/coreutils/coreutils_8.14.bb
> >> b/meta/recipes-core/coreutils/coreutils_8.14.bb
> >> index 9a714a9..c7c8e13 100644
> >> --- a/meta/recipes-core/coreutils/coreutils_8.14.bb
> >> +++ b/meta/recipes-core/coreutils/coreutils_8.14.bb
> >> @@ -7,7 +7,7 @@ BUGTRACKER = "http://debbugs.gnu.org/coreutils"
> >>   LICENSE = "GPLv3+"
> >>   LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504\
> >>
> >> file://src/ls.c;startline=5;endline=16;md5=e1a509558876db58fb6667ba140137ad"
> >> -PR = "r5"
> >> +PR = "r6"
> >>   DEPENDS = "gmp libcap"
> >>   DEPENDS_virtclass-native = ""
> >>
> >> @@ -36,6 +36,8 @@ base_bindir_progs = "cat chgrp chmod chown cp date dd echo
> >> false kill ln ls mkdi
> >>
> >>   sbindir_progs= "chroot"
> >>
> >> +acpaths = "-I ./m4"
> >> +
> > This is a poor work-around to the problem, and does not give any idea as to what
> > your fixing.  Please investigate a proper fix not just a work-around.
> >
> 
> Hi Saul,
> 
> This is because the aclocal's argument list is very long when building
> coreutils, it has many arguments (378 m4 files under 'coreutils-8.14/m4'),
> when the length of the TMPDIR is short this is OK, but when it is longer,
> there would be a argument list too long error.
> 
> I think that we have two solutions to fix this problem:
> 1) Make aclocal accept longer arguments, unfortunately, the native aclocal is
>     running on the host, the length of the argument depends on the host's kernel
>     (The ARG_MAX which is defined in '/usr/include/linux/limits.h'), so this way
>     doesn't work.
> 
> 2) Make coreutils' acpaths shorter, this is what this patch does, it changes the
>     absolute path to the relative path, so it doesn't care the length of the
>     TMPDIR anymore.

There is solution

3) Fix autotools.bbclass to use relative paths for acpaths instead of
absolute ones. Something like:

for i in `find ${S} -maxdepth 2 -name \*.m4|grep -v 'aclocal.m4'| grep
-v 'acinclude.m4' | sed -e 's,\(.*/\).*$,\1,'|sort -u | sed -e 's,${S},
\.,'`; do
					acpaths="$acpaths -I $i"

The reason this hasn't gone in yet as it hacks around it on a per recipe
basis rather than fixes an underlying problem.

Cheers,

Richard