[PATCH 0/1] ncurses: Disable parallel make

[PATCH 0/1] ncurses: Disable parallel make

[xiaofeng.yan]

From: Xiaofeng Yan <xiaofeng.yan@windriver.com>

I can't reproduce bug 2298 after building ncurses successfully many time because Richard said "Its a race issue and sometimes occurs with high values of PARALLEL_MAKE. This means it will sometimes appear and sometimes not appear". I didn't know whether this patch is appropriate or not? I will appreciate him very much if anyone has the resolution.

The following changes since commit d4e265661517f8dd4e1648fdc56bac5973f986f6:
  Saul Wold (1):
        poky.conf: Change WARNS -> ERRORS

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib xiaofeng/2298
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=xiaofeng/2298

Xiaofeng Yan (1):
  ncurses: Disable parallel make

 meta/recipes-core/ncurses/ncurses.inc |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

[PATCH 1/1] ncurses: Disable parallel make

[xiaofeng.yan]

From: Xiaofeng Yan <xiaofeng.yan@windriver.com>

Ncurses failure non-gplv3 build by race issue. So disable parallel \
make when building this package.

Signed-off-by: Xiaofeng Yan <xiaofeng.yan@windriver.com>
---
 meta/recipes-core/ncurses/ncurses.inc |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/meta/recipes-core/ncurses/ncurses.inc b/meta/recipes-core/ncurses/ncurses.inc
index ae99e2c..b115e46 100644
--- a/meta/recipes-core/ncurses/ncurses.inc
+++ b/meta/recipes-core/ncurses/ncurses.inc
@@ -29,6 +29,8 @@ BUILD_CPPFLAGS += "-D_GNU_SOURCE"
 # natives don't generally look in base_libdir
 base_libdir_virtclass-native = "${libdir}"
 
+PARALLEL_MAKE = ""
+
 # Helper function for do_configure to allow multiple configurations
 # $1 the directory to run configure in
 # $@ the arguments to pass to configure
-- 
1.7.0.4

Re: [PATCH 1/1] ncurses: Disable parallel make

[Saul Wold]

On 05/16/2012 01:10 PM, xiaofeng.yan@windriver.com wrote:
> From: Xiaofeng Yan<xiaofeng.yan@windriver.com>
>
> Ncurses failure non-gplv3 build by race issue. So disable parallel \
> make when building this package.
>
This is not the best approach as you disable PARALLEL_MAKE for both 
non-gplv3 and gplv3 versions.  Further, we want to get rid of 
PARALLLEL_MAKE setting as much as possible, so this patch is not helping 
that.

Did you try running on a large many core machine?  It might help if you 
have some other builds going also to stress the machine.

Sau!

> Signed-off-by: Xiaofeng Yan<xiaofeng.yan@windriver.com>
> ---
>   meta/recipes-core/ncurses/ncurses.inc |    2 ++
>   1 files changed, 2 insertions(+), 0 deletions(-)
>
> diff --git a/meta/recipes-core/ncurses/ncurses.inc b/meta/recipes-core/ncurses/ncurses.inc
> index ae99e2c..b115e46 100644
> --- a/meta/recipes-core/ncurses/ncurses.inc
> +++ b/meta/recipes-core/ncurses/ncurses.inc
> @@ -29,6 +29,8 @@ BUILD_CPPFLAGS += "-D_GNU_SOURCE"
>   # natives don't generally look in base_libdir
>   base_libdir_virtclass-native = "${libdir}"
>
> +PARALLEL_MAKE = ""
> +
>   # Helper function for do_configure to allow multiple configurations
>   # $1 the directory to run configure in
>   # $@ the arguments to pass to configure

Re: [PATCH 1/1] ncurses: Disable parallel make

[Xiaofeng Yan]

On 2012年05月16日 19:02, Saul Wold wrote:
> On 05/16/2012 01:10 PM, xiaofeng.yan@windriver.com wrote:
>> From: Xiaofeng Yan<xiaofeng.yan@windriver.com>
>>
>> Ncurses failure non-gplv3 build by race issue. So disable parallel \
>> make when building this package.
>>
> This is not the best approach as you disable PARALLEL_MAKE for both 
> non-gplv3 and gplv3 versions. Further, we want to get rid of [M1] 
> setting as much as possible, so this patch is not helping that.
>
> Did you try running on a large many core machine? It might help if you 
> have some other builds going also to stress the machine.
>
> Sau!
Thanks for your reply. The most cores I have are eight. I also set 
PARALLEL_MAKE=j1000 and 10000. I think I need try to find new way for 
fixing bugs.

Thanks
Yan
>
>> Signed-off-by: Xiaofeng Yan<xiaofeng.yan@windriver.com>
>> ---
>> meta/recipes-core/ncurses/ncurses.inc | 2 ++
>> 1 files changed, 2 insertions(+), 0 deletions(-)
>>
>> diff --git a/meta/recipes-core/ncurses/ncurses.inc 
>> b/meta/recipes-core/ncurses/ncurses.inc
>> index ae99e2c..b115e46 100644
>> --- a/meta/recipes-core/ncurses/ncurses.inc
>> +++ b/meta/recipes-core/ncurses/ncurses.inc
>> @@ -29,6 +29,8 @@ BUILD_CPPFLAGS += "-D_GNU_SOURCE"
>> # natives don't generally look in base_libdir
>> base_libdir_virtclass-native = "${libdir}"
>>
>> +PARALLEL_MAKE = ""
>> +
>> # Helper function for do_configure to allow multiple configurations
>> # $1 the directory to run configure in
>> # $@ the arguments to pass to configure
>

Re: [PATCH 1/1] ncurses: Disable parallel make

[Khem Raj]

On Wed, May 16, 2012 at 7:01 PM, Xiaofeng Yan
<xiaofeng.yan@windriver.com> wrote:
> Thanks for your reply. The most cores I have are eight. I also set
> PARALLEL_MAKE=j1000 and 10000. I think I need try to find new way for fixing
> bugs.
>

I think disabling parallel make should be last resort. So yes any
other solution is a good thing to try.

Re: [PATCH 1/1] ncurses: Disable parallel make

[Jason Wessel]

On 05/16/2012 09:01 PM, Xiaofeng Yan wrote:
> On 2012年05月16日 19:02, Saul Wold wrote:
>> On 05/16/2012 01:10 PM, xiaofeng.yan@windriver.com wrote:
>>> From: Xiaofeng Yan<xiaofeng.yan@windriver.com>
>>>
>>> Ncurses failure non-gplv3 build by race issue. So disable parallel \
>>> make when building this package.
>>>
>> This is not the best approach as you disable PARALLEL_MAKE for both 
>> non-gplv3 and gplv3 versions. Further, we want to get rid of [M1] 
>> setting as much as possible, so this patch is not helping that.
>>
>> Did you try running on a large many core machine? It might help if you 
>> have some other builds going also to stress the machine.
>>
>> Sau!
> Thanks for your reply. The most cores I have are eight. I also set 
> PARALLEL_MAKE=j1000 and 10000. I think I need try to find new way for 
> fixing bugs.
> 

Do you have an error file from a failed build (and ideally the failed build directory)?  Having diagnosed many problems like this in the past, it is easiest to look for the failure case and add some sleep statement in the Makefile to get it to trigger every time in the same way. 

The two most common problems are:
  1) autoconf re-runs due to time stamps or partially patched files
  2) a generated file is reported as missing

In the first case it, it will often be some error with a .h missing or some other strange error about a header in the compilation and it is a result of only having a partial file because it is getting regenerated at the time.

In the second case you just find the file's rule in the Makefile and add an if statement in the Make target goal if it is a multi-object rule to look for the problem object and sleep a bit.  I have yet to see a case I couldn't reproduce the results by following the strategy of some forcing some extra delay.  You probably won't have to go to this length, but there was one time I even wrote a C wrapper around a command to add some sleep controlled by an environment variable to prove config.h was getting removed and regenerated.   Example:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int main(int argc, char *const argv[]) {
   char *lookfor;
   if (argc >= 2) { 
     lookfor = getenv("LOOKFORSLEEP");
     if (lookfor && strcmp(argv[1], lookfor) == 0) {
         if (argc >= 3 && strcmp(argv[2], "config.h") == 0) {
            unlink("config.h");
            printf("Special sleep on command %s\n", lookfor);
            sleep(2);
         }
     }
   }
   execv("/bin/sh", argv);
   return 0;
}


Best of luck,
Jason.

Re: [PATCH 1/1] ncurses: Disable parallel make

[Xiaofeng Yan]

On 2012年05月17日 20:02, Jason Wessel wrote:
> On 05/16/2012 09:01 PM, Xiaofeng Yan wrote:
>> On 2012年05月16日 19:02, Saul Wold wrote:
>>> On 05/16/2012 01:10 PM, xiaofeng.yan@windriver.com wrote:
>>>> From: Xiaofeng Yan<xiaofeng.yan@windriver.com>
>>>>
>>>> Ncurses failure non-gplv3 build by race issue. So disable parallel \
>>>> make when building this package.
>>>>
>>> This is not the best approach as you disable PARALLEL_MAKE for both
>>> non-gplv3 and gplv3 versions. Further, we want to get rid of [M1]
>>> setting as much as possible, so this patch is not helping that.
>>>
>>> Did you try running on a large many core machine? It might help if you
>>> have some other builds going also to stress the machine.
>>>
>>> Sau!
>> Thanks for your reply. The most cores I have are eight. I also set
>> PARALLEL_MAKE=j1000 and 10000. I think I need try to find new way for
>> fixing bugs.
>>
> Do you have an error file from a failed build (and ideally the failed build directory)?  Having diagnosed many problems like this in the past, it is easiest to look for the failure case and add some sleep statement in the Makefile to get it to trigger every time in the same way.
>
> The two most common problems are:
>    1) autoconf re-runs due to time stamps or partially patched files
>    2) a generated file is reported as missing
>
> In the first case it, it will often be some error with a .h missing or some other strange error about a header in the compilation and it is a result of only having a partial file because it is getting regenerated at the time.
>
> In the second case you just find the file's rule in the Makefile and add an if statement in the Make target goal if it is a multi-object rule to look for the problem object and sleep a bit.  I have yet to see a case I couldn't reproduce the results by following the strategy of some forcing some extra delay.  You probably won't have to go to this length, but there was one time I even wrote a C wrapper around a command to add some sleep controlled by an environment variable to prove config.h was getting removed and regenerated.   Example:
>
> #include<stdio.h>
> #include<stdlib.h>
> #include<string.h>
> #include<unistd.h>
>
> int main(int argc, char *const argv[]) {
>     char *lookfor;
>     if (argc>= 2) {
>       lookfor = getenv("LOOKFORSLEEP");
>       if (lookfor&&  strcmp(argv[1], lookfor) == 0) {
>           if (argc>= 3&&  strcmp(argv[2], "config.h") == 0) {
>              unlink("config.h");
>              printf("Special sleep on command %s\n", lookfor);
>              sleep(2);
>           }
>       }
>     }
>     execv("/bin/sh", argv);
>     return 0;
> }
>
>
> Best of luck,
> Jason.
>
Hi Jason,

Thank you to share your good experience with me very much.  I will apply 
your methods to fix this bug.

Thanks
Yan

Re: [PATCH 1/1] ncurses: Disable parallel make

[Xiaofeng Yan]

[-- Attachment #1: Type: text/plain, Size: 3009 bytes --]

On 2012年05月17日 20:02, Jason Wessel wrote:
> On 05/16/2012 09:01 PM, Xiaofeng Yan wrote:
>> On 2012年05月16日 19:02, Saul Wold wrote:
>>> On 05/16/2012 01:10 PM, xiaofeng.yan@windriver.com wrote:
>>>> From: Xiaofeng Yan<xiaofeng.yan@windriver.com>
>>>>
>>>> Ncurses failure non-gplv3 build by race issue. So disable parallel \
>>>> make when building this package.
>>>>
>>> This is not the best approach as you disable PARALLEL_MAKE for both
>>> non-gplv3 and gplv3 versions. Further, we want to get rid of [M1]
>>> setting as much as possible, so this patch is not helping that.
>>>
>>> Did you try running on a large many core machine? It might help if you
>>> have some other builds going also to stress the machine.
>>>
>>> Sau!
>> Thanks for your reply. The most cores I have are eight. I also set
>> PARALLEL_MAKE=j1000 and 10000. I think I need try to find new way for
>> fixing bugs.
>>
> Do you have an error file from a failed build (and ideally the failed build directory)?  Having diagnosed many problems like this in the past, it is easiest to look for the failure case and add some sleep statement in the Makefile to get it to trigger every time in the same way.
Hi jason,
The failed build information is in *Bug 2298* 
<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2298>.  The error 
appear in the stage of install, not configure and compiling.
Do you any ideas after reading bug information?

Thanks
Yan



> The two most common problems are:
>    1) autoconf re-runs due to time stamps or partially patched files
>    2) a generated file is reported as missing
>
> In the first case it, it will often be some error with a .h missing or some other strange error about a header in the compilation and it is a result of only having a partial file because it is getting regenerated at the time.
>
> In the second case you just find the file's rule in the Makefile and add an if statement in the Make target goal if it is a multi-object rule to look for the problem object and sleep a bit.  I have yet to see a case I couldn't reproduce the results by following the strategy of some forcing some extra delay.  You probably won't have to go to this length, but there was one time I even wrote a C wrapper around a command to add some sleep controlled by an environment variable to prove config.h was getting removed and regenerated.   Example:
>
> #include<stdio.h>
> #include<stdlib.h>
> #include<string.h>
> #include<unistd.h>
>
> int main(int argc, char *const argv[]) {
>     char *lookfor;
>     if (argc>= 2) {
>       lookfor = getenv("LOOKFORSLEEP");
>       if (lookfor&&  strcmp(argv[1], lookfor) == 0) {
>           if (argc>= 3&&  strcmp(argv[2], "config.h") == 0) {
>              unlink("config.h");
>              printf("Special sleep on command %s\n", lookfor);
>              sleep(2);
>           }
>       }
>     }
>     execv("/bin/sh", argv);
>     return 0;
> }
>
>
> Best of luck,
> Jason.
>


[-- Attachment #2: Type: text/html, Size: 5147 bytes --]

Re: [PATCH 1/1] ncurses: Disable parallel make

[Robert Yang]

I've looked into the code, this is a race issue which is caused by
install.libs and install.data:

1) install.data needs run tic
2) tic needs libtinfo.so
3) install.libs would regenerate libtinfo.so
4) but install.data doesn't depend on install.libs, and they can run
    parallelly

So there would be errors in a very critical condition: tic is begining
to run at the same time when install.libs is generating libtinfo.so, and
this libtinfo.so is not integrity, then there would be the error:

tic: error while loading shared libraries: /path/to/lib/libtinfo.so.5: file too 
short

Here are the related code:
1) The do_install in meta/recipes-core/ncurses/ncurses.inc:

make install.libs ... install.data

2) The install.libs and install.data in ncurses-5.9/narrowc/Makefile, note that
    it uses the double-colon which means that the all the targets in different
    rules will be run, and if there are no prerequisites for that rule, its
    recipe is always executed (even if the target already exists). The
    install.libs is in 9 rules (just paste 2 here):

install.libs uninstall.libs \
install.data uninstall.data ::
         cd misc && ${MAKE} ${CF_MFLAGS} $@
...
install.libs \
uninstall.libs \
install.ncurses \
uninstall.ncurses ::
         cd ncurses && ${MAKE} ${CF_MFLAGS} $@
...


3) The install.libs in ncurses-5.9/narrowc/ncurses/Makefile:

../lib/libtinfo.so.$(REL_VERSION) : \
                 ../lib \
                 $(SHARED_T_OBJS)
         @echo linking $@
         $(MK_SHARED_LIB) $(SHARED_T_OBJS) $(TINFO_LIST) $(LDFLAGS)
         cd ../lib && ($(LN_S) libtinfo.so.$(REL_VERSION) 
libtinfo.so.$(ABI_VERSION); $(LN_S) libtinfo.so.$(ABI_VERSION) libtinfo.so; )


The ../lib/libtinfo.so has been generated in the compile stage, but it would be
regenerated since its prerequisites (../lib) has changed.

The easiest way to fix this is first run install.libs, then install.data, here
is the patch:
diff --git a/meta/recipes-core/ncurses/ncurses.inc 
b/meta/recipes-core/ncurses/ncurses.inc
index ae99e2c..6309b69 100644
--- a/meta/recipes-core/ncurses/ncurses.inc
+++ b/meta/recipes-core/ncurses/ncurses.inc
@@ -122,8 +122,17 @@ shell_do_install() {
          # Order of installation is important; widec installs a 'curses.h'
          # header with more definitions and must be installed last hence.
          # Compatibility of these headers will be checked in 'do_test()'.
+
+        # The install.data should run after install.libs, otherwise
+        # there would be a race issue in a very critical conditon, since
+        # tic will be run by install.data, and tic needs libtinfo.so
+        # which would be regenerated by install.libs.
          oe_runmake -C narrowc ${_install_opts} \
-                install.data install.progs
+                install.progs
+
+        oe_runmake -C narrowc DESTDIR='${D}'  \
+                PKG_CONFIG_LIBDIR='${libdir}/pkgconfig' \
+                install.data

          ! ${ENABLE_WIDEC} || \
              oe_runmake -C widec ${_install_opts}


Another solution is modify the Makefiles, but that is not as simple as modify
the ncurses.inc.

Xiaofeng will send the V2 after enough testing.

// Robert

On 05/18/2012 03:33 PM, Xiaofeng Yan wrote:
> On 2012年05月17日 20:02, Jason Wessel wrote:
>> On 05/16/2012 09:01 PM, Xiaofeng Yan wrote:
>>> On 2012年05月16日 19:02, Saul Wold wrote:
>>>> On 05/16/2012 01:10 PM, xiaofeng.yan@windriver.com wrote:
>>>>> From: Xiaofeng Yan<xiaofeng.yan@windriver.com>
>>>>>
>>>>> Ncurses failure non-gplv3 build by race issue. So disable parallel \
>>>>> make when building this package.
>>>>>
>>>> This is not the best approach as you disable PARALLEL_MAKE for both
>>>> non-gplv3 and gplv3 versions. Further, we want to get rid of [M1]
>>>> setting as much as possible, so this patch is not helping that.
>>>>
>>>> Did you try running on a large many core machine? It might help if you
>>>> have some other builds going also to stress the machine.
>>>>
>>>> Sau!
>>> Thanks for your reply. The most cores I have are eight. I also set
>>> PARALLEL_MAKE=j1000 and 10000. I think I need try to find new way for
>>> fixing bugs.
>>>
>> Do you have an error file from a failed build (and ideally the failed build
>> directory)? Having diagnosed many problems like this in the past, it is
>> easiest to look for the failure case and add some sleep statement in the
>> Makefile to get it to trigger every time in the same way.
> Hi jason,
> The failed build information is in *Bug 2298*
> <https://bugzilla.yoctoproject.org/show_bug.cgi?id=2298>. The error appear in
> the stage of install, not configure and compiling.
> Do you any ideas after reading bug information?
>
> Thanks
> Yan
>
>
>
>> The two most common problems are:
>> 1) autoconf re-runs due to time stamps or partially patched files
>> 2) a generated file is reported as missing
>>
>> In the first case it, it will often be some error with a .h missing or some
>> other strange error about a header in the compilation and it is a result of
>> only having a partial file because it is getting regenerated at the time.
>>
>> In the second case you just find the file's rule in the Makefile and add an if
>> statement in the Make target goal if it is a multi-object rule to look for the
>> problem object and sleep a bit. I have yet to see a case I couldn't reproduce
>> the results by following the strategy of some forcing some extra delay. You
>> probably won't have to go to this length, but there was one time I even wrote
>> a C wrapper around a command to add some sleep controlled by an environment
>> variable to prove config.h was getting removed and regenerated. Example:
>>
>> #include<stdio.h>
>> #include<stdlib.h>
>> #include<string.h>
>> #include<unistd.h>
>>
>> int main(int argc, char *const argv[]) {
>> char *lookfor;
>> if (argc>= 2) {
>> lookfor = getenv("LOOKFORSLEEP");
>> if (lookfor&& strcmp(argv[1], lookfor) == 0) {
>> if (argc>= 3&& strcmp(argv[2], "config.h") == 0) {
>> unlink("config.h");
>> printf("Special sleep on command %s\n", lookfor);
>> sleep(2);
>> }
>> }
>> }
>> execv("/bin/sh", argv);
>> return 0;
>> }
>>
>>
>> Best of luck,
>> Jason.
>>
>
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core