[PATCH] gcc: add flex-native explicit dependency

[PATCH] gcc: add flex-native explicit dependency

[Denys Dmytriyenko]

From: Denys Dmytriyenko <denys@ti.com>

It seems flex is required to build gcc:

| .../work-shared/gcc-7.2.0-r0/gcc-7.2.0/missing: line 81: flex: command not found
| WARNING: 'flex' is missing on your system.
|          You should only need it if you modified a '.l' file.
|          You may want to install the Fast Lexical Analyzer package:
|          <http://flex.sourceforge.net/>
| Makefile:2799: recipe for target 'gengtype-lex.c' failed
| make[1]: [gengtype-lex.c] Error 127 (ignored)

Normally this is handled indirectly throught binutils-cross dependency
pulling in flex-native implicitly. For deterministic builds, this should
be specified explicitly.

Signed-off-by: Denys Dmytriyenko <denys@ti.com>
---
 meta/recipes-devtools/gcc/gcc-7.2.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/gcc/gcc-7.2.inc b/meta/recipes-devtools/gcc/gcc-7.2.inc
index 1d40cba..d1fb6de 100644
--- a/meta/recipes-devtools/gcc/gcc-7.2.inc
+++ b/meta/recipes-devtools/gcc/gcc-7.2.inc
@@ -10,7 +10,7 @@ BINV = "7.2.0"
 
 FILESEXTRAPATHS =. "${FILE_DIRNAME}/gcc-7.2:${FILE_DIRNAME}/gcc-7.2/backport:"
 
-DEPENDS =+ "mpfr gmp libmpc zlib"
+DEPENDS =+ "mpfr gmp libmpc zlib flex-native"
 NATIVEDEPS = "mpfr-native gmp-native libmpc-native zlib-native"
 
 LICENSE = "GPL-3.0-with-GCC-exception & GPLv3"
-- 
2.7.4

Re: [PATCH] gcc: add flex-native explicit dependency

[Richard Purdie]

On Wed, 2018-01-10 at 22:15 -0500, Denys Dmytriyenko wrote:
> From: Denys Dmytriyenko <denys@ti.com>
> 
> It seems flex is required to build gcc:
> 
> > 
> > .../work-shared/gcc-7.2.0-r0/gcc-7.2.0/missing: line 81: flex:
> > command not found
> > WARNING: 'flex' is missing on your system.
> >          You should only need it if you modified a '.l' file.
> >          You may want to install the Fast Lexical Analyzer package:
> >          <http://flex.sourceforge.net/>
> > Makefile:2799: recipe for target 'gengtype-lex.c' failed
> > make[1]: [gengtype-lex.c] Error 127 (ignored)
> Normally this is handled indirectly throught binutils-cross
> dependency
> pulling in flex-native implicitly. For deterministic builds, this
> should
> be specified explicitly.
> 
> Signed-off-by: Denys Dmytriyenko <denys@ti.com>
> ---
>  meta/recipes-devtools/gcc/gcc-7.2.inc | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-devtools/gcc/gcc-7.2.inc b/meta/recipes-
> devtools/gcc/gcc-7.2.inc
> index 1d40cba..d1fb6de 100644
> --- a/meta/recipes-devtools/gcc/gcc-7.2.inc
> +++ b/meta/recipes-devtools/gcc/gcc-7.2.inc
> @@ -10,7 +10,7 @@ BINV = "7.2.0"
>  
>  FILESEXTRAPATHS =. "${FILE_DIRNAME}/gcc-7.2:${FILE_DIRNAME}/gcc-
> 7.2/backport:"
>  
> -DEPENDS =+ "mpfr gmp libmpc zlib"
> +DEPENDS =+ "mpfr gmp libmpc zlib flex-native"
>  NATIVEDEPS = "mpfr-native gmp-native libmpc-native zlib-native"
>  
>  LICENSE = "GPL-3.0-with-GCC-exception & GPLv3"

Agreed, however I think we have bigger problems. I'm testing with this:

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 65f51430ee2..a5c4a73963e 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -921,6 +921,13 @@ def setscene_depvalid(task, taskdependees, notneeded, d, log=None):
     if taskdependees[task][1] == "do_stash_locale" or taskdependees[task][1] == "do_gcc_stash_builddir":
         return True
 
+
+    if taskdependees[task][1] == 'do_populate_sysroot':
+        if taskdependees[task][0] == "flex-native" or taskdependees[task][0] == "bison-native":
+            #bb.warn("Skipping %s" % str(taskdependees[dep]))
+            bb.warn("Skipping")
+            return True
+
     # We only need to trigger packagedata through direct dependencies
     # but need to preserve packagedata on packagedata links
     if taskdependees[task][1] == "do_packagedata":


and test results so far imply that we need:

diff --git a/meta/recipes-devtools/gcc/gcc-7.2.inc b/meta/recipes-devtools/gcc/gcc-7.2.inc
index 1d40cba7317..90e4a990cb3 100644
--- a/meta/recipes-devtools/gcc/gcc-7.2.inc
+++ b/meta/recipes-devtools/gcc/gcc-7.2.inc
@@ -11,7 +11,7 @@ BINV = "7.2.0"
 FILESEXTRAPATHS =. "${FILE_DIRNAME}/gcc-7.2:${FILE_DIRNAME}/gcc-7.2/backport:"
 
 DEPENDS =+ "mpfr gmp libmpc zlib"
-NATIVEDEPS = "mpfr-native gmp-native libmpc-native zlib-native"
+NATIVEDEPS = "mpfr-native gmp-native libmpc-native zlib-native flex-native"
 
 LICENSE = "GPL-3.0-with-GCC-exception & GPLv3"
 

probably in addition to your patch. I'll continue to run some test
builds and see how much breakage the above change shows up.

Cheers,

Richard

Re: [PATCH] gcc: add flex-native explicit dependency

[Denys Dmytriyenko]

On Thu, Jan 11, 2018 at 02:41:15PM +0000, Richard Purdie wrote:
> On Wed, 2018-01-10 at 22:15 -0500, Denys Dmytriyenko wrote:
> > From: Denys Dmytriyenko <denys@ti.com>
> > 
> > It seems flex is required to build gcc:
> > 
> > > 
> > > .../work-shared/gcc-7.2.0-r0/gcc-7.2.0/missing: line 81: flex:
> > > command not found
> > > WARNING: 'flex' is missing on your system.
> > >          You should only need it if you modified a '.l' file.
> > >          You may want to install the Fast Lexical Analyzer package:
> > >          <http://flex.sourceforge.net/>
> > > Makefile:2799: recipe for target 'gengtype-lex.c' failed
> > > make[1]: [gengtype-lex.c] Error 127 (ignored)
> > Normally this is handled indirectly throught binutils-cross
> > dependency
> > pulling in flex-native implicitly. For deterministic builds, this
> > should
> > be specified explicitly.
> > 
> > Signed-off-by: Denys Dmytriyenko <denys@ti.com>
> > ---
> >  meta/recipes-devtools/gcc/gcc-7.2.inc | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/meta/recipes-devtools/gcc/gcc-7.2.inc b/meta/recipes-
> > devtools/gcc/gcc-7.2.inc
> > index 1d40cba..d1fb6de 100644
> > --- a/meta/recipes-devtools/gcc/gcc-7.2.inc
> > +++ b/meta/recipes-devtools/gcc/gcc-7.2.inc
> > @@ -10,7 +10,7 @@ BINV = "7.2.0"
> >  
> >  FILESEXTRAPATHS =. "${FILE_DIRNAME}/gcc-7.2:${FILE_DIRNAME}/gcc-
> > 7.2/backport:"
> >  
> > -DEPENDS =+ "mpfr gmp libmpc zlib"
> > +DEPENDS =+ "mpfr gmp libmpc zlib flex-native"
> >  NATIVEDEPS = "mpfr-native gmp-native libmpc-native zlib-native"
> >  
> >  LICENSE = "GPL-3.0-with-GCC-exception & GPLv3"
> 
> Agreed, however I think we have bigger problems. I'm testing with this:
> 
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index 65f51430ee2..a5c4a73963e 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -921,6 +921,13 @@ def setscene_depvalid(task, taskdependees, notneeded, d, log=None):
>      if taskdependees[task][1] == "do_stash_locale" or taskdependees[task][1] == "do_gcc_stash_builddir":
>          return True
>  
> +
> +    if taskdependees[task][1] == 'do_populate_sysroot':
> +        if taskdependees[task][0] == "flex-native" or taskdependees[task][0] == "bison-native":
> +            #bb.warn("Skipping %s" % str(taskdependees[dep]))
> +            bb.warn("Skipping")
> +            return True
> +
>      # We only need to trigger packagedata through direct dependencies
>      # but need to preserve packagedata on packagedata links
>      if taskdependees[task][1] == "do_packagedata":
> 
> 
> and test results so far imply that we need:
> 
> diff --git a/meta/recipes-devtools/gcc/gcc-7.2.inc b/meta/recipes-devtools/gcc/gcc-7.2.inc
> index 1d40cba7317..90e4a990cb3 100644
> --- a/meta/recipes-devtools/gcc/gcc-7.2.inc
> +++ b/meta/recipes-devtools/gcc/gcc-7.2.inc
> @@ -11,7 +11,7 @@ BINV = "7.2.0"
>  FILESEXTRAPATHS =. "${FILE_DIRNAME}/gcc-7.2:${FILE_DIRNAME}/gcc-7.2/backport:"
>  
>  DEPENDS =+ "mpfr gmp libmpc zlib"
> -NATIVEDEPS = "mpfr-native gmp-native libmpc-native zlib-native"
> +NATIVEDEPS = "mpfr-native gmp-native libmpc-native zlib-native flex-native"
>  
>  LICENSE = "GPL-3.0-with-GCC-exception & GPLv3"
>  
> 
> probably in addition to your patch. I'll continue to run some test
> builds and see how much breakage the above change shows up.

Thanks. I was building native gcc for the target, while using external 
prebuilt cross toolchain. I first tried adding flex-native to NATIVEDEPS 
list, but that didn't help - looks like this list is only used for cross and 
crosssdk builds.

-- 
Denys

Re: [PATCH] gcc: add flex-native explicit dependency

[Richard Purdie]

On Thu, 2018-01-11 at 10:02 -0500, Denys Dmytriyenko wrote:
> On Thu, Jan 11, 2018 at 02:41:15PM +0000, Richard Purdie wrote:
> > and test results so far imply that we need:
> > 
> > diff --git a/meta/recipes-devtools/gcc/gcc-7.2.inc b/meta/recipes-
> > devtools/gcc/gcc-7.2.inc
> > index 1d40cba7317..90e4a990cb3 100644
> > --- a/meta/recipes-devtools/gcc/gcc-7.2.inc
> > +++ b/meta/recipes-devtools/gcc/gcc-7.2.inc
> > @@ -11,7 +11,7 @@ BINV = "7.2.0"
> >  FILESEXTRAPATHS =. "${FILE_DIRNAME}/gcc-7.2:${FILE_DIRNAME}/gcc-
> > 7.2/backport:"
> >  
> >  DEPENDS =+ "mpfr gmp libmpc zlib"
> > -NATIVEDEPS = "mpfr-native gmp-native libmpc-native zlib-native"
> > +NATIVEDEPS = "mpfr-native gmp-native libmpc-native zlib-native
> > flex-native"
> >  
> >  LICENSE = "GPL-3.0-with-GCC-exception & GPLv3"
> >  
> > 
> > probably in addition to your patch. I'll continue to run some test
> > builds and see how much breakage the above change shows up.
> Thanks. I was building native gcc for the target, while using
> external 
> prebuilt cross toolchain. I first tried adding flex-native to
> NATIVEDEPS 
> list, but that didn't help - looks like this list is only used for
> cross and 
> crosssdk builds.

I'll probably take your at and perf patches, my gcc one which covers
the cross and candadian variants and merge a change to the sstate
dependency code which means we'll explicitly see these dependency
issues in future. I'll queue this in -next and test.

Cheers,

Richard