Bash parser

Bash parser

[Isak Lichtenstein]

Hi all,

I'm creating my own image by implementing IMAGE_CMD_<myImage> in my custom .bbclass file.

In this method I'm using the bash syntax. But a lot of time the parser doesn't manage to parse my file properly. Examples:

	TMP="file1 file2"
	read -a scripts <<< $tmp
generates
	ShellSyntaxError: expecting here-document name, got '<'

Or 

	TMP="file1 file2"
	scripts=(${TMP})
generate
	ShellSyntaxError: LexToken(TOKEN,'${TMP}',0,0)


Other bash commands are parsed properly, but generate an error while executing them. Example:
	TMP="file1, file2"
	tmp=${TMP//,/ }
generates
	Bad substitution
	| WARNING: exit code 2 from a shell command.


Does a page exist somewhere describing the bash features supported by the parser and also the execution environment? 
Are arrays supported at all?

Best regards

Isak Lichtenstein

Re: Bash parser

[Olof Johansson]

On 14-07-16 11:36 +0200, Isak Lichtenstein wrote:
> In this method I'm using the bash syntax. But a lot of time the
> parser doesn't manage to parse my file properly. Examples:
> 
> 	TMP="file1 file2"
> 	read -a scripts <<< $tmp
> generates
> 	ShellSyntaxError: expecting here-document name, got '<'
> 
> Or 
> 
> 	TMP="file1 file2"
> 	scripts=(${TMP})
> generate
> 	ShellSyntaxError: LexToken(TOKEN,'${TMP}',0,0)
> 
> 
> Other bash commands are parsed properly, but generate an error
> while executing them. Example:
> 	TMP="file1, file2"
> 	tmp=${TMP//,/ }
> generates
> 	Bad substitution
> 	| WARNING: exit code 2 from a shell command.

Note that these features you describe here are all bash
extensions. For Debian users (and I think Ubuntu users as well?),
the default /bin/sh is dash and does not support either of these
extensions. There are cases where the bitbake parser will refuse
valid portable shell script features as well though, like shell
arithmetics, e.g.:

 n=$((n+1))

> Does a page exist somewhere describing the bash features
> supported by the parser and also the execution environment?
> Are arrays supported at all?

I don't know of any such documentation, but if you stick to
portable shell script features, you should be mostly fine.

-- 
olofjn

Re: Bash parser

[Isak Lichtenstein]

Hi Olof,

Thank you very much for your prompt answer
> 
> On 14-07-16 11:36 +0200, Isak Lichtenstein wrote:
> > In this method I'm using the bash syntax. But a lot of time the parser
> > doesn't manage to parse my file properly. Examples:
> >
> > 	TMP="file1 file2"
> > 	read -a scripts <<< $tmp
> > generates
> > 	ShellSyntaxError: expecting here-document name, got '<'
> >
> > Or
> >
> > 	TMP="file1 file2"
> > 	scripts=(${TMP})
> > generate
> > 	ShellSyntaxError: LexToken(TOKEN,'${TMP}',0,0)
> >
> >
> > Other bash commands are parsed properly, but generate an error while
> > executing them. Example:
> > 	TMP="file1, file2"
> > 	tmp=${TMP//,/ }
> > generates
> > 	Bad substitution
> > 	| WARNING: exit code 2 from a shell command.
> 
> Note that these features you describe here are all bash extensions. For Debian
> users (and I think Ubuntu users as well?), the default /bin/sh is dash and does not
> support either of these extensions. There are cases where the bitbake parser will
> refuse valid portable shell script features as well though, like shell arithmetics, e.g.:

Ubuntu default is actually bash.
> 
>  n=$((n+1))
> 
> > Does a page exist somewhere describing the bash features supported by
> > the parser and also the execution environment?
> > Are arrays supported at all?
> 
> I don't know of any such documentation, but if you stick to portable shell script
> features, you should be mostly fine.
> 
Thanks for the advice. Will try to stick to it.

Isak

Re: Bash parser

[William Mills]

On 07/16/2014 06:32 AM, Isak Lichtenstein wrote:
> Hi Olof,
>
> Thank you very much for your prompt answer
>>
>> On 14-07-16 11:36 +0200, Isak Lichtenstein wrote:
>>> In this method I'm using the bash syntax. But a lot of time the parser
>>> doesn't manage to parse my file properly. Examples:
>>>
>>> 	TMP="file1 file2"
>>> 	read -a scripts <<< $tmp
>>> generates
>>> 	ShellSyntaxError: expecting here-document name, got '<'
>>>
>>> Or
>>>
>>> 	TMP="file1 file2"
>>> 	scripts=(${TMP})
>>> generate
>>> 	ShellSyntaxError: LexToken(TOKEN,'${TMP}',0,0)
>>>
>>>
>>> Other bash commands are parsed properly, but generate an error while
>>> executing them. Example:
>>> 	TMP="file1, file2"
>>> 	tmp=${TMP//,/ }
>>> generates
>>> 	Bad substitution
>>> 	| WARNING: exit code 2 from a shell command.
>>
>> Note that these features you describe here are all bash extensions. For Debian
>> users (and I think Ubuntu users as well?), the default /bin/sh is dash and does not
>> support either of these extensions. There are cases where the bitbake parser will
>> refuse valid portable shell script features as well though, like shell arithmetics, e.g.:
>
> Ubuntu default is actually bash.

Ubuntu default login shell is bash.
Default for /bin/sh is to link to dash.
So shell scripts that use
	#!/bin/sh
will execute with dash.

You may have changed this on your machine and don't remember.
Many developers do.
I just checked a fresh 14.04 to make sure my knowledge is up to date.
	ls -l /bin/sh

>>
>>   n=$((n+1))
>>
>>> Does a page exist somewhere describing the bash features supported by
>>> the parser and also the execution environment?
>>> Are arrays supported at all?
>>
>> I don't know of any such documentation, but if you stick to portable shell script
>> features, you should be mostly fine.
>>
> Thanks for the advice. Will try to stick to it.
>
> Isak
>

Re: Bash parser

[Paul Eggleton]

On Wednesday 16 July 2014 11:53:30 Olof Johansson wrote:
> On 14-07-16 11:36 +0200, Isak Lichtenstein wrote:
> > Does a page exist somewhere describing the bash features
> > supported by the parser and also the execution environment?
> > Are arrays supported at all?
> 
> I don't know of any such documentation, but if you stick to
> portable shell script features, you should be mostly fine.

The following may be helpful:

  http://mywiki.wooledge.org/Bashism

  http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre