[PATCH v2 1/2] waf: don't assume the waf intepretter is good

[PATCH v2 1/2] waf: don't assume the waf intepretter is good

[Ross Burton]

Waf typically uses `python` as the intepretter but inside a task this
does not exist.  Typically this is solved by patching waf (see the
glmark2 recipe) but not all versionf of Waf support Python 3 so we can't
assume a specific interpretter.

Instead, create a new variable WAF_PYTHON for the correct interpretter,
and default this to `python3`.  If the user has a recipe that needs
Python 2 then this can be changed in the recipe.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 meta/classes/waf.bbclass | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/meta/classes/waf.bbclass b/meta/classes/waf.bbclass
index 309f625a40..8fa5063645 100644
--- a/meta/classes/waf.bbclass
+++ b/meta/classes/waf.bbclass
@@ -1,6 +1,10 @@
 # avoids build breaks when using no-static-libs.inc
 DISABLE_STATIC = ""
 
+# What Python interpretter to use.  Defaults to Python 3 but can be
+# overridden if required.
+WAF_PYTHON ?= "python3"
+
 B = "${WORKDIR}/build"
 
 EXTRA_OECONF_append = " ${PACKAGECONFIG_CONFARGS}"
@@ -40,9 +44,10 @@ python waf_preconfigure() {
     import subprocess
     from distutils.version import StrictVersion
     subsrcdir = d.getVar('S')
+    python = d.getVar('WAF_PYTHON')
     wafbin = os.path.join(subsrcdir, 'waf')
     try:
-        result = subprocess.check_output([wafbin, '--version'], cwd=subsrcdir, stderr=subprocess.STDOUT)
+        result = subprocess.check_output([python, wafbin, '--version'], cwd=subsrcdir, stderr=subprocess.STDOUT)
         version = result.decode('utf-8').split()[1]
         if StrictVersion(version) >= StrictVersion("1.8.7"):
             d.setVar("WAF_EXTRA_CONF", "--bindir=${bindir} --libdir=${libdir}")
@@ -55,16 +60,16 @@ python waf_preconfigure() {
 do_configure[prefuncs] += "waf_preconfigure"
 
 waf_do_configure() {
-	(cd ${S} && ./waf configure -o ${B} --prefix=${prefix} ${WAF_EXTRA_CONF} ${EXTRA_OECONF})
+	(cd ${S} && ${WAF_PYTHON} ./waf configure -o ${B} --prefix=${prefix} ${WAF_EXTRA_CONF} ${EXTRA_OECONF})
 }
 
 do_compile[progress] = "outof:^\[\s*(\d+)/\s*(\d+)\]\s+"
 waf_do_compile()  {
-	(cd ${S} && ./waf build ${@oe.utils.parallel_make_argument(d, '-j%d', limit=64)} ${EXTRA_OEWAF_BUILD})
+	(cd ${S} && ${WAF_PYTHON} ./waf build ${@oe.utils.parallel_make_argument(d, '-j%d', limit=64)} ${EXTRA_OEWAF_BUILD})
 }
 
 waf_do_install() {
-	(cd ${S} && ./waf install --destdir=${D} ${EXTRA_OEWAF_INSTALL})
+	(cd ${S} && ${WAF_PYTHON} ./waf install --destdir=${D} ${EXTRA_OEWAF_INSTALL})
 }
 
 EXPORT_FUNCTIONS do_configure do_compile do_install
-- 
2.25.1

[PATCH v2 2/2] waf: add ${B} to do_configure[cleandirs]

[Ross Burton]

As waf is always out-of-tree, we can delete ${B} before every build.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 meta/classes/waf.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/waf.bbclass b/meta/classes/waf.bbclass
index 8fa5063645..188119f356 100644
--- a/meta/classes/waf.bbclass
+++ b/meta/classes/waf.bbclass
@@ -6,6 +6,7 @@ DISABLE_STATIC = ""
 WAF_PYTHON ?= "python3"
 
 B = "${WORKDIR}/build"
+do_configure[cleandirs] += "${B}"
 
 EXTRA_OECONF_append = " ${PACKAGECONFIG_CONFARGS}"
 
-- 
2.25.1

Re: [OE-core] [PATCH v2 1/2] waf: don't assume the waf intepretter is good

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3199 bytes --]

should it also inherit ${WAF_PYTHON}native, so that people who need to use
python2 don't need to inherit it in the recipe (as python is no longer in
HOSTTOOLS) or is python3-native considered unwanted dependency for the
default behavior?

I don't personally use waf, so no opinion either way from me.

On Fri, Oct 16, 2020 at 11:21 AM Ross Burton <ross@burtonini.com> wrote:

> Waf typically uses `python` as the intepretter but inside a task this
> does not exist.  Typically this is solved by patching waf (see the
> glmark2 recipe) but not all versionf of Waf support Python 3 so we can't
> assume a specific interpretter.
>
> Instead, create a new variable WAF_PYTHON for the correct interpretter,
> and default this to `python3`.  If the user has a recipe that needs
> Python 2 then this can be changed in the recipe.
>
> Signed-off-by: Ross Burton <ross.burton@arm.com>
> ---
>  meta/classes/waf.bbclass | 13 +++++++++----
>  1 file changed, 9 insertions(+), 4 deletions(-)
>
> diff --git a/meta/classes/waf.bbclass b/meta/classes/waf.bbclass
> index 309f625a40..8fa5063645 100644
> --- a/meta/classes/waf.bbclass
> +++ b/meta/classes/waf.bbclass
> @@ -1,6 +1,10 @@
>  # avoids build breaks when using no-static-libs.inc
>  DISABLE_STATIC = ""
>
> +# What Python interpretter to use.  Defaults to Python 3 but can be
> +# overridden if required.
> +WAF_PYTHON ?= "python3"
> +
>  B = "${WORKDIR}/build"
>
>  EXTRA_OECONF_append = " ${PACKAGECONFIG_CONFARGS}"
> @@ -40,9 +44,10 @@ python waf_preconfigure() {
>      import subprocess
>      from distutils.version import StrictVersion
>      subsrcdir = d.getVar('S')
> +    python = d.getVar('WAF_PYTHON')
>      wafbin = os.path.join(subsrcdir, 'waf')
>      try:
> -        result = subprocess.check_output([wafbin, '--version'],
> cwd=subsrcdir, stderr=subprocess.STDOUT)
> +        result = subprocess.check_output([python, wafbin, '--version'],
> cwd=subsrcdir, stderr=subprocess.STDOUT)
>          version = result.decode('utf-8').split()[1]
>          if StrictVersion(version) >= StrictVersion("1.8.7"):
>              d.setVar("WAF_EXTRA_CONF", "--bindir=${bindir}
> --libdir=${libdir}")
> @@ -55,16 +60,16 @@ python waf_preconfigure() {
>  do_configure[prefuncs] += "waf_preconfigure"
>
>  waf_do_configure() {
> -       (cd ${S} && ./waf configure -o ${B} --prefix=${prefix}
> ${WAF_EXTRA_CONF} ${EXTRA_OECONF})
> +       (cd ${S} && ${WAF_PYTHON} ./waf configure -o ${B}
> --prefix=${prefix} ${WAF_EXTRA_CONF} ${EXTRA_OECONF})
>  }
>
>  do_compile[progress] = "outof:^\[\s*(\d+)/\s*(\d+)\]\s+"
>  waf_do_compile()  {
> -       (cd ${S} && ./waf build ${@oe.utils.parallel_make_argument(d,
> '-j%d', limit=64)} ${EXTRA_OEWAF_BUILD})
> +       (cd ${S} && ${WAF_PYTHON} ./waf build
> ${@oe.utils.parallel_make_argument(d, '-j%d', limit=64)}
> ${EXTRA_OEWAF_BUILD})
>  }
>
>  waf_do_install() {
> -       (cd ${S} && ./waf install --destdir=${D} ${EXTRA_OEWAF_INSTALL})
> +       (cd ${S} && ${WAF_PYTHON} ./waf install --destdir=${D}
> ${EXTRA_OEWAF_INSTALL})
>  }
>
>  EXPORT_FUNCTIONS do_configure do_compile do_install
> --
> 2.25.1
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 3975 bytes --]

Re: [OE-core] [PATCH v2 1/2] waf: don't assume the waf intepretter is good

[Richard Purdie]

On Fri, 2020-10-16 at 11:45 +0200, Martin Jansa wrote:
> should it also inherit ${WAF_PYTHON}native, so that people who need
> to use python2 don't need to inherit it in the recipe (as python is
> no longer in HOSTTOOLS) or is python3-native considered unwanted
> dependency for the default behavior?
> 
> I don't personally use waf, so no opinion either way from me.

We need python3 for bitbake so I think we don't want to include the
native python chain here if we can help it...

Cheers,

Richard