[PATCH v2] layer.conf: fix sanity error for PATH variable in extensible SDK workflow

[PATCH v2] layer.conf: fix sanity error for PATH variable in extensible SDK workflow

[sbanerje]

Sanity checker reports following error for the PATH variable,
when bitbake -e <recipe> command is run in an extensible SDK workspace.
   PATH contains '.', './' or '' (empty element), which will break the build

In case of extensible SDK, PATH variable is formed with two consecutive ':'
as bb.utils.which(d.getVar('PATH'),'bitbake') call returns an empty string.

This change adds the ':' only if the bb.utils.which(d.getVar('PATH'),'bitbake')
returns a valid path.

Signed-off-by: Sourabh Banerjee <sbanerje@codeaurora.org>
---
 meta/conf/layer.conf | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
index 38df0f3..75d2ee0 100644
--- a/meta/conf/layer.conf
+++ b/meta/conf/layer.conf
@@ -102,4 +102,12 @@ SSTATE_EXCLUDEDEPS_SYSROOT += "\
 SSTATE_EXCLUDEDEPS_SYSROOT += ".*->autoconf-archive-native"
 
 # We need to keep bitbake tools in PATH
-PATH := "${@os.path.dirname(bb.utils.which(d.getVar('PATH'),'bitbake'))}:${HOSTTOOLS_DIR}"
+# The ':' separator is conditionally added only if 'bitbake' is in environment.
+# In case of extensible SDK, bitbake is not in environment, which results in
+# two consecutive ':'. Sanity checker reports "empty element" error when it
+# encounters two consecutive ':' for PATH variable in the extensible SDK.
+PATH := "\
+${@os.path.dirname(bb.utils.which(d.getVar('PATH'),'bitbake'))}\
+${@(':', '')[bb.utils.which(d.getVar('PATH'),'bitbake') is ""]}\
+${HOSTTOOLS_DIR}\
+"
-- 
$(echo -e 'The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project')



-- 
Regards,
Sourabh

Re: [OE-core] [PATCH v2] layer.conf: fix sanity error for PATH variable in extensible SDK workflow

[Richard Purdie]

On Sun, 2020-10-11 at 13:26 +0530, Sourabh Banerjee wrote:
> Sanity checker reports following error for the PATH variable,
> when bitbake -e <recipe> command is run in an extensible SDK workspace.
>    PATH contains '.', './' or '' (empty element), which will break the build
> 
> In case of extensible SDK, PATH variable is formed with two consecutive ':'
> as bb.utils.which(d.getVar('PATH'),'bitbake') call returns an empty string.
> 
> This change adds the ':' only if the bb.utils.which(d.getVar('PATH'),'bitbake')
> returns a valid path.
> 
> Signed-off-by: Sourabh Banerjee <sbanerje@codeaurora.org>
> ---
>  meta/conf/layer.conf | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
> index 38df0f3..75d2ee0 100644
> --- a/meta/conf/layer.conf
> +++ b/meta/conf/layer.conf
> @@ -102,4 +102,12 @@ SSTATE_EXCLUDEDEPS_SYSROOT += "\
>  SSTATE_EXCLUDEDEPS_SYSROOT += ".*->autoconf-archive-native"
>  
>  # We need to keep bitbake tools in PATH
> -PATH := "${@os.path.dirname(bb.utils.which(d.getVar('PATH'),'bitbake'))}:${HOSTTOOLS_DIR}"
> +# The ':' separator is conditionally added only if 'bitbake' is in environment.
> +# In case of extensible SDK, bitbake is not in environment, which results in
> +# two consecutive ':'. Sanity checker reports "empty element" error when it
> +# encounters two consecutive ':' for PATH variable in the extensible SDK.
> +PATH := "\
> +${@os.path.dirname(bb.utils.which(d.getVar('PATH'),'bitbake'))}\
> +${@(':', '')[bb.utils.which(d.getVar('PATH'),'bitbake') is ""]}\
> +${HOSTTOOLS_DIR}\
> +"

I think we need to try and find something a little neater than this.
How about:

# Need to avoid empty path entries
BITBAKEPATH := "${@os.path.dirname(bb.utils.which(d.getVar('PATH'),'bitbake'))}"
PATH := "${@'${BITBAKEPATH}:' if ${BITBAKEPATH} is not ""}${HOSTTOOLS_DIR}"

Cheers,

Richard

Re: [OE-core] [PATCH v2] layer.conf: fix sanity error for PATH variable in extensible SDK workflow

[Sourabh Banerjee]

On 2020-10-13 14:32, Richard Purdie wrote:
> On Sun, 2020-10-11 at 13:26 +0530, Sourabh Banerjee wrote:
>> Sanity checker reports following error for the PATH variable,
>> when bitbake -e <recipe> command is run in an extensible SDK 
>> workspace.
>>    PATH contains '.', './' or '' (empty element), which will break the 
>> build
>> 
>> In case of extensible SDK, PATH variable is formed with two 
>> consecutive ':'
>> as bb.utils.which(d.getVar('PATH'),'bitbake') call returns an empty 
>> string.
>> 
>> This change adds the ':' only if the 
>> bb.utils.which(d.getVar('PATH'),'bitbake')
>> returns a valid path.
>> 
>> Signed-off-by: Sourabh Banerjee <sbanerje@codeaurora.org>
>> ---
>>  meta/conf/layer.conf | 10 +++++++++-
>>  1 file changed, 9 insertions(+), 1 deletion(-)
>> 
>> diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
>> index 38df0f3..75d2ee0 100644
>> --- a/meta/conf/layer.conf
>> +++ b/meta/conf/layer.conf
>> @@ -102,4 +102,12 @@ SSTATE_EXCLUDEDEPS_SYSROOT += "\
>>  SSTATE_EXCLUDEDEPS_SYSROOT += ".*->autoconf-archive-native"
>> 
>>  # We need to keep bitbake tools in PATH
>> -PATH := 
>> "${@os.path.dirname(bb.utils.which(d.getVar('PATH'),'bitbake'))}:${HOSTTOOLS_DIR}"
>> +# The ':' separator is conditionally added only if 'bitbake' is in 
>> environment.
>> +# In case of extensible SDK, bitbake is not in environment, which 
>> results in
>> +# two consecutive ':'. Sanity checker reports "empty element" error 
>> when it
>> +# encounters two consecutive ':' for PATH variable in the extensible 
>> SDK.
>> +PATH := "\
>> +${@os.path.dirname(bb.utils.which(d.getVar('PATH'),'bitbake'))}\
>> +${@(':', '')[bb.utils.which(d.getVar('PATH'),'bitbake') is ""]}\
>> +${HOSTTOOLS_DIR}\
>> +"
> 
> I think we need to try and find something a little neater than this.
> How about:
> 
> # Need to avoid empty path entries
> BITBAKEPATH := 
> "${@os.path.dirname(bb.utils.which(d.getVar('PATH'),'bitbake'))}"
> PATH := "${@'${BITBAKEPATH}:' if ${BITBAKEPATH} is not 
> ""}${HOSTTOOLS_DIR}"
> 

Thanks Richard, sounds good, let me upload patch v3 with this.

> Cheers,
> 
> Richard


-- 
Regards,
Sourabh

Re: [OE-core] [PATCH v2] layer.conf: fix sanity error for PATH variable in extensible SDK workflow

[Sourabh Banerjee]

On 2020-10-13 14:32, Richard Purdie wrote:
> On Sun, 2020-10-11 at 13:26 +0530, Sourabh Banerjee wrote:
>> Sanity checker reports following error for the PATH variable,
>> when bitbake -e <recipe> command is run in an extensible SDK 
>> workspace.
>>    PATH contains '.', './' or '' (empty element), which will break the 
>> build
>> 
>> In case of extensible SDK, PATH variable is formed with two 
>> consecutive ':'
>> as bb.utils.which(d.getVar('PATH'),'bitbake') call returns an empty 
>> string.
>> 
>> This change adds the ':' only if the 
>> bb.utils.which(d.getVar('PATH'),'bitbake')
>> returns a valid path.
>> 
>> Signed-off-by: Sourabh Banerjee <sbanerje@codeaurora.org>
>> ---
>>  meta/conf/layer.conf | 10 +++++++++-
>>  1 file changed, 9 insertions(+), 1 deletion(-)
>> 
>> diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
>> index 38df0f3..75d2ee0 100644
>> --- a/meta/conf/layer.conf
>> +++ b/meta/conf/layer.conf
>> @@ -102,4 +102,12 @@ SSTATE_EXCLUDEDEPS_SYSROOT += "\
>>  SSTATE_EXCLUDEDEPS_SYSROOT += ".*->autoconf-archive-native"
>> 
>>  # We need to keep bitbake tools in PATH
>> -PATH := 
>> "${@os.path.dirname(bb.utils.which(d.getVar('PATH'),'bitbake'))}:${HOSTTOOLS_DIR}"
>> +# The ':' separator is conditionally added only if 'bitbake' is in 
>> environment.
>> +# In case of extensible SDK, bitbake is not in environment, which 
>> results in
>> +# two consecutive ':'. Sanity checker reports "empty element" error 
>> when it
>> +# encounters two consecutive ':' for PATH variable in the extensible 
>> SDK.
>> +PATH := "\
>> +${@os.path.dirname(bb.utils.which(d.getVar('PATH'),'bitbake'))}\
>> +${@(':', '')[bb.utils.which(d.getVar('PATH'),'bitbake') is ""]}\
>> +${HOSTTOOLS_DIR}\
>> +"
> 
> I think we need to try and find something a little neater than this.
> How about:
> 
> # Need to avoid empty path entries
> BITBAKEPATH := 
> "${@os.path.dirname(bb.utils.which(d.getVar('PATH'),'bitbake'))}"
> PATH := "${@'${BITBAKEPATH}:' if ${BITBAKEPATH} is not 
> ""}${HOSTTOOLS_DIR}"

Hi Richard,

Uploaded patch v3 here 
https://lists.openembedded.org/g/openembedded-core/message/143292
I observed following error if I used your suggestion as is.

		ERROR: Unable to parse 
/workspace/sbanerje/MASTER/poky/bitbake/lib/bb/data_smart.py
		Traceback (most recent call last):
		  File "/workspace/sbanerje/MASTER/poky/bitbake/lib/bb/data_smart.py", 
line 114, in VariableParse.python_sub(match=<_sre.SRE_Match object; 
span=(0, 127), 
match='${@\'/local/mnt/workspace/sbanerje/MASTER/poky/bi>):
							 varname = '<expansion>'
			>            codeobj = compile(code.strip(), varname, "eval")

		bb.data_smart.ExpansionError: Failure expanding variable PATH[:=], 
expression was ${@'/workspace/sbanerje/MASTER/poky/bitbake/bin:' if 
/workspace/sbanerje/MASTER/poky/bitbake/bin is not ""}${HOSTTOOLS_DIR} 
which triggered exception SyntaxError: invalid syntax (Var <PATH[:=]>, 
line 1)

I figured, only way to get around this is to add 'else'.

> 
> Cheers,
> 
> Richard


-- 
Regards,
Sourabh