trouble during parsing

trouble during parsing

[Andreas Müller]

Hi,

with current bitbake master I get

ERROR: Command execution failed: Traceback (most recent call
last):#########################################################
                              | ETA:  00:00:17
  File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
92, in runAsyncCommand
    self.cooker.updateCache()
  File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
1330, in updateCache
    if not self.parser.parse_next():
  File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
1703, in parse_next
    self.virtuals += len(result)
TypeError: object of type 'ExpansionError' has no len()

Any ideas?

Andreas

Re: trouble during parsing

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 1016 bytes --]

On Wed, Feb 20, 2013 at 5:54 AM, Andreas Müller <
schnitzeltony@googlemail.com> wrote:

> with current bitbake master I get
>
> ERROR: Command execution failed: Traceback (most recent call
> last):#########################################################
>                               | ETA:  00:00:17
>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
> 92, in runAsyncCommand
>     self.cooker.updateCache()
>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> 1330, in updateCache
>     if not self.parser.parse_next():
>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> 1703, in parse_next
>     self.virtuals += len(result)
> TypeError: object of type 'ExpansionError' has no len()
>

Hmm, looks like it's returning the exceptions rather than raising them, for
some reason, but that doesn't make much sense — the pool code always raises
any exceptions from its imap iterator's next() method.
-- 
Christopher Larson

[-- Attachment #2: Type: text/html, Size: 1441 bytes --]

Re: trouble during parsing

[Flanagan, Elizabeth]

On Wed, Feb 20, 2013 at 6:31 AM, Chris Larson <clarson@kergoth.com> wrote:
>
> On Wed, Feb 20, 2013 at 5:54 AM, Andreas Müller
> <schnitzeltony@googlemail.com> wrote:
>>
>> with current bitbake master I get
>>
>> ERROR: Command execution failed: Traceback (most recent call
>> last):#########################################################
>>                               | ETA:  00:00:17
>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
>> 92, in runAsyncCommand
>>     self.cooker.updateCache()
>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>> 1330, in updateCache
>>     if not self.parser.parse_next():
>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>> 1703, in parse_next
>>     self.virtuals += len(result)
>> TypeError: object of type 'ExpansionError' has no len()
>
>
> Hmm, looks like it's returning the exceptions rather than raising them, for
> some reason, but that doesn't make much sense — the pool code always raises
> any exceptions from its imap iterator's next() method.
> --
> Christopher Larson
>

I'm hitting this as well with
poky:dde7a481354d5b0539762109bdfaaba6f85f879b, meta-ti and pandaboard
as my machine.

-b

> _______________________________________________
> bitbake-devel mailing list
> bitbake-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/bitbake-devel
>

-- 
Elizabeth Flanagan
Yocto Project
Build and Release

Re: trouble during parsing

[Flanagan, Elizabeth]

On Sun, Feb 24, 2013 at 6:12 PM, Flanagan, Elizabeth
<elizabeth.flanagan@intel.com> wrote:
> On Wed, Feb 20, 2013 at 6:31 AM, Chris Larson <clarson@kergoth.com> wrote:
>>
>> On Wed, Feb 20, 2013 at 5:54 AM, Andreas Müller
>> <schnitzeltony@googlemail.com> wrote:
>>>
>>> with current bitbake master I get
>>>
>>> ERROR: Command execution failed: Traceback (most recent call
>>> last):#########################################################
>>>                               | ETA:  00:00:17
>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
>>> 92, in runAsyncCommand
>>>     self.cooker.updateCache()
>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>> 1330, in updateCache
>>>     if not self.parser.parse_next():
>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>> 1703, in parse_next
>>>     self.virtuals += len(result)
>>> TypeError: object of type 'ExpansionError' has no len()
>>
>>
>> Hmm, looks like it's returning the exceptions rather than raising them, for
>> some reason, but that doesn't make much sense — the pool code always raises
>> any exceptions from its imap iterator's next() method.
>> --
>> Christopher Larson
>>
>
> I'm hitting this as well with
> poky:dde7a481354d5b0539762109bdfaaba6f85f879b, meta-ti and pandaboard
> as my machine.
>

Ach, should have updated my email. Seems reverting
0a99563a4ea270594fd9a61da46f9387fb79dc66 cleared up the issue.

-b

-- 
Elizabeth Flanagan
Yocto Project
Build and Release

Re: trouble during parsing

[Andreas Müller]

On Mon, Feb 25, 2013 at 3:28 AM, Flanagan, Elizabeth
<elizabeth.flanagan@intel.com> wrote:
> On Sun, Feb 24, 2013 at 6:12 PM, Flanagan, Elizabeth
> <elizabeth.flanagan@intel.com> wrote:
>> On Wed, Feb 20, 2013 at 6:31 AM, Chris Larson <clarson@kergoth.com> wrote:
>>>
>>> On Wed, Feb 20, 2013 at 5:54 AM, Andreas Müller
>>> <schnitzeltony@googlemail.com> wrote:
>>>>
>>>> with current bitbake master I get
>>>>
>>>> ERROR: Command execution failed: Traceback (most recent call
>>>> last):#########################################################
>>>>                               | ETA:  00:00:17
>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
>>>> 92, in runAsyncCommand
>>>>     self.cooker.updateCache()
>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>> 1330, in updateCache
>>>>     if not self.parser.parse_next():
>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>> 1703, in parse_next
>>>>     self.virtuals += len(result)
>>>> TypeError: object of type 'ExpansionError' has no len()
>>>
>>>
>>> Hmm, looks like it's returning the exceptions rather than raising them, for
>>> some reason, but that doesn't make much sense — the pool code always raises
>>> any exceptions from its imap iterator's next() method.
>>> --
>>> Christopher Larson
>>>
>>
>> I'm hitting this as well with
>> poky:dde7a481354d5b0539762109bdfaaba6f85f879b, meta-ti and pandaboard
>> as my machine.
>>
>
> Ach, should have updated my email. Seems reverting
> 0a99563a4ea270594fd9a61da46f9387fb79dc66 cleared up the issue.
>
> -b
>
I have two machines (Fedora 14  / Fedora 15)  both reverted
0a99563a4ea270594fd9a61da46f9387fb79dc66 and both worked fine during
weekend.

Just a guess: Last week when arago git was down I saw errors during
parsing for a recipe I never used. This was with a revision of bitbake
a bit earlier than HEAD - that caused me to update. After sending a
message to meta-ti I could work during weekend. Now it seems arago is
down again:

$ git clone git://arago-project.org/git/projects/u-boot-keystone.git
Cloning into u-boot-keystone...
fatal: read error: Connection reset by peer

and bitbake -DD gives

DEBUG: Fetcher accessed the network with the command git ls-remote
git://arago-project.org/git/projects/u-boot-keystone.git
DEV.MCSDK-03.00.00.07
DEBUG: Running export SSH_AUTH_SOCK="/tmp/keyring-Pob6mG/ssh"; export
PATH="/home/andreas/data/oe-core/sources/openembedded-core/scripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin/armv7a-vfp-neon-angstrom-linux-gnueabi:/home/andreas/tmp/oe-core-eglibc/sysroots/overo/usr/bin/crossscripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux//bin:/home/andreas/oe-core/sources/bitbake/bin:/home/andreas/data/OpenEmbedded/bitbake/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:./:/home/andreas/bin:/sbin:/usr/sbin";
export HOME="/home/andreas"; git ls-remote
git://arago-project.org/git/projects/u-boot-keystone.git
DEV.MCSDK-03.00.00.07
DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
argument 'preinst' is not a string literal
DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
argument 'postinst' is not a string literal

...

DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
argument 'preinst' is not a string literal
DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
argument 'postinst' is not a string literal
DEBUG: while parsing do_clean, in call of bb.build.exec_func, argument
'f' is not a string literal
DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
argument 'preinst' is not a string literal
ERROR: Command execution failed: Traceback (most recent call last):
  File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
92, in runAsyncCommand
    self.cooker.updateCache()
  File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
1329, in updateCache
    if not self.parser.parse_next():
  File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
1702, in parse_next
    self.virtuals += len(result)
TypeError: object of type 'ExpansionError' has no len()


In the recipe u-boot_2013.01.bb I find SRCREV = "DEV.MCSDK-03.00.00.07"

which is not a commit ID and causes a git ls-remote during parse

Is it possible that this error occurs when a git SRC_URI server closes
connection unexpected?

Andreas

Re: [bitbake-devel] trouble during parsing

[Andreas Müller]

On Mon, Feb 25, 2013 at 3:28 AM, Flanagan, Elizabeth
<elizabeth.flanagan@intel.com> wrote:
> On Sun, Feb 24, 2013 at 6:12 PM, Flanagan, Elizabeth
> <elizabeth.flanagan@intel.com> wrote:
>> On Wed, Feb 20, 2013 at 6:31 AM, Chris Larson <clarson@kergoth.com> wrote:
>>>
>>> On Wed, Feb 20, 2013 at 5:54 AM, Andreas Müller
>>> <schnitzeltony@googlemail.com> wrote:
>>>>
>>>> with current bitbake master I get
>>>>
>>>> ERROR: Command execution failed: Traceback (most recent call
>>>> last):#########################################################
>>>>                               | ETA:  00:00:17
>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
>>>> 92, in runAsyncCommand
>>>>     self.cooker.updateCache()
>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>> 1330, in updateCache
>>>>     if not self.parser.parse_next():
>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>> 1703, in parse_next
>>>>     self.virtuals += len(result)
>>>> TypeError: object of type 'ExpansionError' has no len()
>>>
>>>
>>> Hmm, looks like it's returning the exceptions rather than raising them, for
>>> some reason, but that doesn't make much sense — the pool code always raises
>>> any exceptions from its imap iterator's next() method.
>>> --
>>> Christopher Larson
>>>
>>
>> I'm hitting this as well with
>> poky:dde7a481354d5b0539762109bdfaaba6f85f879b, meta-ti and pandaboard
>> as my machine.
>>
>
> Ach, should have updated my email. Seems reverting
> 0a99563a4ea270594fd9a61da46f9387fb79dc66 cleared up the issue.
>
> -b
>
I have two machines (Fedora 14  / Fedora 15)  both reverted
0a99563a4ea270594fd9a61da46f9387fb79dc66 and both worked fine during
weekend.

Just a guess: Last week when arago git was down I saw errors during
parsing for a recipe I never used. This was with a revision of bitbake
a bit earlier than HEAD - that caused me to update. After sending a
message to meta-ti I could work during weekend. Now it seems arago is
down again:

$ git clone git://arago-project.org/git/projects/u-boot-keystone.git
Cloning into u-boot-keystone...
fatal: read error: Connection reset by peer

and bitbake -DD gives

DEBUG: Fetcher accessed the network with the command git ls-remote
git://arago-project.org/git/projects/u-boot-keystone.git
DEV.MCSDK-03.00.00.07
DEBUG: Running export SSH_AUTH_SOCK="/tmp/keyring-Pob6mG/ssh"; export
PATH="/home/andreas/data/oe-core/sources/openembedded-core/scripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin/armv7a-vfp-neon-angstrom-linux-gnueabi:/home/andreas/tmp/oe-core-eglibc/sysroots/overo/usr/bin/crossscripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux//bin:/home/andreas/oe-core/sources/bitbake/bin:/home/andreas/data/OpenEmbedded/bitbake/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:./:/home/andreas/bin:/sbin:/usr/sbin";
export HOME="/home/andreas"; git ls-remote
git://arago-project.org/git/projects/u-boot-keystone.git
DEV.MCSDK-03.00.00.07
DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
argument 'preinst' is not a string literal
DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
argument 'postinst' is not a string literal

...

DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
argument 'preinst' is not a string literal
DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
argument 'postinst' is not a string literal
DEBUG: while parsing do_clean, in call of bb.build.exec_func, argument
'f' is not a string literal
DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
argument 'preinst' is not a string literal
ERROR: Command execution failed: Traceback (most recent call last):
  File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
92, in runAsyncCommand
    self.cooker.updateCache()
  File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
1329, in updateCache
    if not self.parser.parse_next():
  File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
1702, in parse_next
    self.virtuals += len(result)
TypeError: object of type 'ExpansionError' has no len()


In the recipe u-boot_2013.01.bb I find SRCREV = "DEV.MCSDK-03.00.00.07"

which is not a commit ID and causes a git ls-remote during parse

Is it possible that this error occurs when a git SRC_URI server closes
connection unexpected?

Andreas

Re: [meta-ti] trouble during parsing

[Javier Martinez Canillas]

On Mon, Feb 25, 2013 at 10:32 AM, Andreas Müller
<schnitzeltony@googlemail.com> wrote:
> On Mon, Feb 25, 2013 at 3:28 AM, Flanagan, Elizabeth
> <elizabeth.flanagan@intel.com> wrote:
>> On Sun, Feb 24, 2013 at 6:12 PM, Flanagan, Elizabeth
>> <elizabeth.flanagan@intel.com> wrote:
>>> On Wed, Feb 20, 2013 at 6:31 AM, Chris Larson <clarson@kergoth.com> wrote:
>>>>
>>>> On Wed, Feb 20, 2013 at 5:54 AM, Andreas Müller
>>>> <schnitzeltony@googlemail.com> wrote:
>>>>>
>>>>> with current bitbake master I get
>>>>>
>>>>> ERROR: Command execution failed: Traceback (most recent call
>>>>> last):#########################################################
>>>>>                               | ETA:  00:00:17
>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
>>>>> 92, in runAsyncCommand
>>>>>     self.cooker.updateCache()
>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>>> 1330, in updateCache
>>>>>     if not self.parser.parse_next():
>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>>> 1703, in parse_next
>>>>>     self.virtuals += len(result)
>>>>> TypeError: object of type 'ExpansionError' has no len()
>>>>
>>>>
>>>> Hmm, looks like it's returning the exceptions rather than raising them, for
>>>> some reason, but that doesn't make much sense — the pool code always raises
>>>> any exceptions from its imap iterator's next() method.
>>>> --
>>>> Christopher Larson
>>>>
>>>
>>> I'm hitting this as well with
>>> poky:dde7a481354d5b0539762109bdfaaba6f85f879b, meta-ti and pandaboard
>>> as my machine.
>>>
>>
>> Ach, should have updated my email. Seems reverting
>> 0a99563a4ea270594fd9a61da46f9387fb79dc66 cleared up the issue.
>>
>> -b
>>
> I have two machines (Fedora 14  / Fedora 15)  both reverted
> 0a99563a4ea270594fd9a61da46f9387fb79dc66 and both worked fine during
> weekend.
>
> Just a guess: Last week when arago git was down I saw errors during
> parsing for a recipe I never used. This was with a revision of bitbake
> a bit earlier than HEAD - that caused me to update. After sending a
> message to meta-ti I could work during weekend. Now it seems arago is
> down again:
>
> $ git clone git://arago-project.org/git/projects/u-boot-keystone.git
> Cloning into u-boot-keystone...
> fatal: read error: Connection reset by peer
>
> and bitbake -DD gives
>
> DEBUG: Fetcher accessed the network with the command git ls-remote
> git://arago-project.org/git/projects/u-boot-keystone.git
> DEV.MCSDK-03.00.00.07
> DEBUG: Running export SSH_AUTH_SOCK="/tmp/keyring-Pob6mG/ssh"; export
> PATH="/home/andreas/data/oe-core/sources/openembedded-core/scripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin/armv7a-vfp-neon-angstrom-linux-gnueabi:/home/andreas/tmp/oe-core-eglibc/sysroots/overo/usr/bin/crossscripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux//bin:/home/andreas/oe-core/sources/bitbake/bin:/home/andreas/data/OpenEmbedded/bitbake/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:./:/home/andreas/bin:/sbin:/usr/sbin";
> export HOME="/home/andreas"; git ls-remote
> git://arago-project.org/git/projects/u-boot-keystone.git
> DEV.MCSDK-03.00.00.07
> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
> argument 'preinst' is not a string literal
> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
> argument 'postinst' is not a string literal
>
> ...
>
> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
> argument 'preinst' is not a string literal
> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
> argument 'postinst' is not a string literal
> DEBUG: while parsing do_clean, in call of bb.build.exec_func, argument
> 'f' is not a string literal
> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
> argument 'preinst' is not a string literal
> ERROR: Command execution failed: Traceback (most recent call last):
>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
> 92, in runAsyncCommand
>     self.cooker.updateCache()
>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> 1329, in updateCache
>     if not self.parser.parse_next():
>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> 1702, in parse_next
>     self.virtuals += len(result)
> TypeError: object of type 'ExpansionError' has no len()
>
>
> In the recipe u-boot_2013.01.bb I find SRCREV = "DEV.MCSDK-03.00.00.07"
>
> which is not a commit ID and causes a git ls-remote during parse
>
> Is it possible that this error occurs when a git SRC_URI server closes
> connection unexpected?
>
> Andreas

Hi Andreas,

I have the same issue and your guess is correct. The problem is that
Arago servers are down so "git ls-remote" is failing and this makes
the BitBake task to fail as well.

Now about the why is been used tag names instead of just commits IDs
so a "git ls-remote" call is needed, this is because some git trees in
arago are been rebased overwriting the commit IDs.

You may take a look to Robert P. J. Day email "[meta-ti] objections to
replacing git SRCREV tag names with actual	hash IDs?" [1] to get a
better understanding of this.

Best regards,
javier

[1]: http://www.mail-archive.com/meta-ti@yoctoproject.org/msg01304.html

Re: [bitbake-devel] trouble during parsing

[Javier Martinez Canillas]

On Mon, Feb 25, 2013 at 10:32 AM, Andreas Müller
<schnitzeltony@googlemail.com> wrote:
> On Mon, Feb 25, 2013 at 3:28 AM, Flanagan, Elizabeth
> <elizabeth.flanagan@intel.com> wrote:
>> On Sun, Feb 24, 2013 at 6:12 PM, Flanagan, Elizabeth
>> <elizabeth.flanagan@intel.com> wrote:
>>> On Wed, Feb 20, 2013 at 6:31 AM, Chris Larson <clarson@kergoth.com> wrote:
>>>>
>>>> On Wed, Feb 20, 2013 at 5:54 AM, Andreas Müller
>>>> <schnitzeltony@googlemail.com> wrote:
>>>>>
>>>>> with current bitbake master I get
>>>>>
>>>>> ERROR: Command execution failed: Traceback (most recent call
>>>>> last):#########################################################
>>>>>                               | ETA:  00:00:17
>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
>>>>> 92, in runAsyncCommand
>>>>>     self.cooker.updateCache()
>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>>> 1330, in updateCache
>>>>>     if not self.parser.parse_next():
>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>>> 1703, in parse_next
>>>>>     self.virtuals += len(result)
>>>>> TypeError: object of type 'ExpansionError' has no len()
>>>>
>>>>
>>>> Hmm, looks like it's returning the exceptions rather than raising them, for
>>>> some reason, but that doesn't make much sense — the pool code always raises
>>>> any exceptions from its imap iterator's next() method.
>>>> --
>>>> Christopher Larson
>>>>
>>>
>>> I'm hitting this as well with
>>> poky:dde7a481354d5b0539762109bdfaaba6f85f879b, meta-ti and pandaboard
>>> as my machine.
>>>
>>
>> Ach, should have updated my email. Seems reverting
>> 0a99563a4ea270594fd9a61da46f9387fb79dc66 cleared up the issue.
>>
>> -b
>>
> I have two machines (Fedora 14  / Fedora 15)  both reverted
> 0a99563a4ea270594fd9a61da46f9387fb79dc66 and both worked fine during
> weekend.
>
> Just a guess: Last week when arago git was down I saw errors during
> parsing for a recipe I never used. This was with a revision of bitbake
> a bit earlier than HEAD - that caused me to update. After sending a
> message to meta-ti I could work during weekend. Now it seems arago is
> down again:
>
> $ git clone git://arago-project.org/git/projects/u-boot-keystone.git
> Cloning into u-boot-keystone...
> fatal: read error: Connection reset by peer
>
> and bitbake -DD gives
>
> DEBUG: Fetcher accessed the network with the command git ls-remote
> git://arago-project.org/git/projects/u-boot-keystone.git
> DEV.MCSDK-03.00.00.07
> DEBUG: Running export SSH_AUTH_SOCK="/tmp/keyring-Pob6mG/ssh"; export
> PATH="/home/andreas/data/oe-core/sources/openembedded-core/scripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin/armv7a-vfp-neon-angstrom-linux-gnueabi:/home/andreas/tmp/oe-core-eglibc/sysroots/overo/usr/bin/crossscripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux//bin:/home/andreas/oe-core/sources/bitbake/bin:/home/andreas/data/OpenEmbedded/bitbake/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:./:/home/andreas/bin:/sbin:/usr/sbin";
> export HOME="/home/andreas"; git ls-remote
> git://arago-project.org/git/projects/u-boot-keystone.git
> DEV.MCSDK-03.00.00.07
> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
> argument 'preinst' is not a string literal
> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
> argument 'postinst' is not a string literal
>
> ...
>
> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
> argument 'preinst' is not a string literal
> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
> argument 'postinst' is not a string literal
> DEBUG: while parsing do_clean, in call of bb.build.exec_func, argument
> 'f' is not a string literal
> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
> argument 'preinst' is not a string literal
> ERROR: Command execution failed: Traceback (most recent call last):
>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
> 92, in runAsyncCommand
>     self.cooker.updateCache()
>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> 1329, in updateCache
>     if not self.parser.parse_next():
>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> 1702, in parse_next
>     self.virtuals += len(result)
> TypeError: object of type 'ExpansionError' has no len()
>
>
> In the recipe u-boot_2013.01.bb I find SRCREV = "DEV.MCSDK-03.00.00.07"
>
> which is not a commit ID and causes a git ls-remote during parse
>
> Is it possible that this error occurs when a git SRC_URI server closes
> connection unexpected?
>
> Andreas

Hi Andreas,

I have the same issue and your guess is correct. The problem is that
Arago servers are down so "git ls-remote" is failing and this makes
the BitBake task to fail as well.

Now about the why is been used tag names instead of just commits IDs
so a "git ls-remote" call is needed, this is because some git trees in
arago are been rebased overwriting the commit IDs.

You may take a look to Robert P. J. Day email "[meta-ti] objections to
replacing git SRCREV tag names with actual	hash IDs?" [1] to get a
better understanding of this.

Best regards,
javier

[1]: http://www.mail-archive.com/meta-ti@yoctoproject.org/msg01304.html

Re: [meta-ti] trouble during parsing

[Andreas Müller]

On Mon, Feb 25, 2013 at 10:46 AM, Javier Martinez Canillas
<javier@dowhile0.org> wrote:
> On Mon, Feb 25, 2013 at 10:32 AM, Andreas Müller
> <schnitzeltony@googlemail.com> wrote:
>> On Mon, Feb 25, 2013 at 3:28 AM, Flanagan, Elizabeth
>> <elizabeth.flanagan@intel.com> wrote:
>>> On Sun, Feb 24, 2013 at 6:12 PM, Flanagan, Elizabeth
>>> <elizabeth.flanagan@intel.com> wrote:
>>>> On Wed, Feb 20, 2013 at 6:31 AM, Chris Larson <clarson@kergoth.com> wrote:
>>>>>
>>>>> On Wed, Feb 20, 2013 at 5:54 AM, Andreas Müller
>>>>> <schnitzeltony@googlemail.com> wrote:
>>>>>>
>>>>>> with current bitbake master I get
>>>>>>
>>>>>> ERROR: Command execution failed: Traceback (most recent call
>>>>>> last):#########################################################
>>>>>>                               | ETA:  00:00:17
>>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
>>>>>> 92, in runAsyncCommand
>>>>>>     self.cooker.updateCache()
>>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>>>> 1330, in updateCache
>>>>>>     if not self.parser.parse_next():
>>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>>>> 1703, in parse_next
>>>>>>     self.virtuals += len(result)
>>>>>> TypeError: object of type 'ExpansionError' has no len()
>>>>>
>>>>>
>>>>> Hmm, looks like it's returning the exceptions rather than raising them, for
>>>>> some reason, but that doesn't make much sense — the pool code always raises
>>>>> any exceptions from its imap iterator's next() method.
>>>>> --
>>>>> Christopher Larson
>>>>>
>>>>
>>>> I'm hitting this as well with
>>>> poky:dde7a481354d5b0539762109bdfaaba6f85f879b, meta-ti and pandaboard
>>>> as my machine.
>>>>
>>>
>>> Ach, should have updated my email. Seems reverting
>>> 0a99563a4ea270594fd9a61da46f9387fb79dc66 cleared up the issue.
>>>
>>> -b
>>>
>> I have two machines (Fedora 14  / Fedora 15)  both reverted
>> 0a99563a4ea270594fd9a61da46f9387fb79dc66 and both worked fine during
>> weekend.
>>
>> Just a guess: Last week when arago git was down I saw errors during
>> parsing for a recipe I never used. This was with a revision of bitbake
>> a bit earlier than HEAD - that caused me to update. After sending a
>> message to meta-ti I could work during weekend. Now it seems arago is
>> down again:
>>
>> $ git clone git://arago-project.org/git/projects/u-boot-keystone.git
>> Cloning into u-boot-keystone...
>> fatal: read error: Connection reset by peer
>>
>> and bitbake -DD gives
>>
>> DEBUG: Fetcher accessed the network with the command git ls-remote
>> git://arago-project.org/git/projects/u-boot-keystone.git
>> DEV.MCSDK-03.00.00.07
>> DEBUG: Running export SSH_AUTH_SOCK="/tmp/keyring-Pob6mG/ssh"; export
>> PATH="/home/andreas/data/oe-core/sources/openembedded-core/scripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin/armv7a-vfp-neon-angstrom-linux-gnueabi:/home/andreas/tmp/oe-core-eglibc/sysroots/overo/usr/bin/crossscripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux//bin:/home/andreas/oe-core/sources/bitbake/bin:/home/andreas/data/OpenEmbedded/bitbake/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:./:/home/andreas/bin:/sbin:/usr/sbin";
>> export HOME="/home/andreas"; git ls-remote
>> git://arago-project.org/git/projects/u-boot-keystone.git
>> DEV.MCSDK-03.00.00.07
>> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
>> argument 'preinst' is not a string literal
>> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
>> argument 'postinst' is not a string literal
>>
>> ...
>>
>> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
>> argument 'preinst' is not a string literal
>> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
>> argument 'postinst' is not a string literal
>> DEBUG: while parsing do_clean, in call of bb.build.exec_func, argument
>> 'f' is not a string literal
>> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
>> argument 'preinst' is not a string literal
>> ERROR: Command execution failed: Traceback (most recent call last):
>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
>> 92, in runAsyncCommand
>>     self.cooker.updateCache()
>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>> 1329, in updateCache
>>     if not self.parser.parse_next():
>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>> 1702, in parse_next
>>     self.virtuals += len(result)
>> TypeError: object of type 'ExpansionError' has no len()
>>
>>
>> In the recipe u-boot_2013.01.bb I find SRCREV = "DEV.MCSDK-03.00.00.07"
>>
>> which is not a commit ID and causes a git ls-remote during parse
>>
>> Is it possible that this error occurs when a git SRC_URI server closes
>> connection unexpected?
>>
>> Andreas
>
> Hi Andreas,
>
> I have the same issue and your guess is correct. The problem is that
> Arago servers are down so "git ls-remote" is failing and this makes
> the BitBake task to fail as well.
>
> Now about the why is been used tag names instead of just commits IDs
> so a "git ls-remote" call is needed, this is because some git trees in
> arago are been rebased overwriting the commit IDs.
>
> You may take a look to Robert P. J. Day email "[meta-ti] objections to
> replacing git SRCREV tag names with actual      hash IDs?" [1] to get a
> better understanding of this.
>
> Best regards,
> javier
>
> [1]: http://www.mail-archive.com/meta-ti@yoctoproject.org/msg01304.html

One additional note: With current bitbake there are configurations
which cause a full parse on every build [1]. Is it possible that
arrago-git dies for the flood of  git ls-remote?

Andreas

[1] http://lists.linuxtogo.org/pipermail/bitbake-devel/2013-February/004178.html

Re: [bitbake-devel] trouble during parsing

[Andreas Müller]

On Mon, Feb 25, 2013 at 10:46 AM, Javier Martinez Canillas
<javier@dowhile0.org> wrote:
> On Mon, Feb 25, 2013 at 10:32 AM, Andreas Müller
> <schnitzeltony@googlemail.com> wrote:
>> On Mon, Feb 25, 2013 at 3:28 AM, Flanagan, Elizabeth
>> <elizabeth.flanagan@intel.com> wrote:
>>> On Sun, Feb 24, 2013 at 6:12 PM, Flanagan, Elizabeth
>>> <elizabeth.flanagan@intel.com> wrote:
>>>> On Wed, Feb 20, 2013 at 6:31 AM, Chris Larson <clarson@kergoth.com> wrote:
>>>>>
>>>>> On Wed, Feb 20, 2013 at 5:54 AM, Andreas Müller
>>>>> <schnitzeltony@googlemail.com> wrote:
>>>>>>
>>>>>> with current bitbake master I get
>>>>>>
>>>>>> ERROR: Command execution failed: Traceback (most recent call
>>>>>> last):#########################################################
>>>>>>                               | ETA:  00:00:17
>>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
>>>>>> 92, in runAsyncCommand
>>>>>>     self.cooker.updateCache()
>>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>>>> 1330, in updateCache
>>>>>>     if not self.parser.parse_next():
>>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>>>> 1703, in parse_next
>>>>>>     self.virtuals += len(result)
>>>>>> TypeError: object of type 'ExpansionError' has no len()
>>>>>
>>>>>
>>>>> Hmm, looks like it's returning the exceptions rather than raising them, for
>>>>> some reason, but that doesn't make much sense — the pool code always raises
>>>>> any exceptions from its imap iterator's next() method.
>>>>> --
>>>>> Christopher Larson
>>>>>
>>>>
>>>> I'm hitting this as well with
>>>> poky:dde7a481354d5b0539762109bdfaaba6f85f879b, meta-ti and pandaboard
>>>> as my machine.
>>>>
>>>
>>> Ach, should have updated my email. Seems reverting
>>> 0a99563a4ea270594fd9a61da46f9387fb79dc66 cleared up the issue.
>>>
>>> -b
>>>
>> I have two machines (Fedora 14  / Fedora 15)  both reverted
>> 0a99563a4ea270594fd9a61da46f9387fb79dc66 and both worked fine during
>> weekend.
>>
>> Just a guess: Last week when arago git was down I saw errors during
>> parsing for a recipe I never used. This was with a revision of bitbake
>> a bit earlier than HEAD - that caused me to update. After sending a
>> message to meta-ti I could work during weekend. Now it seems arago is
>> down again:
>>
>> $ git clone git://arago-project.org/git/projects/u-boot-keystone.git
>> Cloning into u-boot-keystone...
>> fatal: read error: Connection reset by peer
>>
>> and bitbake -DD gives
>>
>> DEBUG: Fetcher accessed the network with the command git ls-remote
>> git://arago-project.org/git/projects/u-boot-keystone.git
>> DEV.MCSDK-03.00.00.07
>> DEBUG: Running export SSH_AUTH_SOCK="/tmp/keyring-Pob6mG/ssh"; export
>> PATH="/home/andreas/data/oe-core/sources/openembedded-core/scripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin/armv7a-vfp-neon-angstrom-linux-gnueabi:/home/andreas/tmp/oe-core-eglibc/sysroots/overo/usr/bin/crossscripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux//bin:/home/andreas/oe-core/sources/bitbake/bin:/home/andreas/data/OpenEmbedded/bitbake/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:./:/home/andreas/bin:/sbin:/usr/sbin";
>> export HOME="/home/andreas"; git ls-remote
>> git://arago-project.org/git/projects/u-boot-keystone.git
>> DEV.MCSDK-03.00.00.07
>> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
>> argument 'preinst' is not a string literal
>> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
>> argument 'postinst' is not a string literal
>>
>> ...
>>
>> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
>> argument 'preinst' is not a string literal
>> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
>> argument 'postinst' is not a string literal
>> DEBUG: while parsing do_clean, in call of bb.build.exec_func, argument
>> 'f' is not a string literal
>> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
>> argument 'preinst' is not a string literal
>> ERROR: Command execution failed: Traceback (most recent call last):
>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
>> 92, in runAsyncCommand
>>     self.cooker.updateCache()
>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>> 1329, in updateCache
>>     if not self.parser.parse_next():
>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>> 1702, in parse_next
>>     self.virtuals += len(result)
>> TypeError: object of type 'ExpansionError' has no len()
>>
>>
>> In the recipe u-boot_2013.01.bb I find SRCREV = "DEV.MCSDK-03.00.00.07"
>>
>> which is not a commit ID and causes a git ls-remote during parse
>>
>> Is it possible that this error occurs when a git SRC_URI server closes
>> connection unexpected?
>>
>> Andreas
>
> Hi Andreas,
>
> I have the same issue and your guess is correct. The problem is that
> Arago servers are down so "git ls-remote" is failing and this makes
> the BitBake task to fail as well.
>
> Now about the why is been used tag names instead of just commits IDs
> so a "git ls-remote" call is needed, this is because some git trees in
> arago are been rebased overwriting the commit IDs.
>
> You may take a look to Robert P. J. Day email "[meta-ti] objections to
> replacing git SRCREV tag names with actual      hash IDs?" [1] to get a
> better understanding of this.
>
> Best regards,
> javier
>
> [1]: http://www.mail-archive.com/meta-ti@yoctoproject.org/msg01304.html

One additional note: With current bitbake there are configurations
which cause a full parse on every build [1]. Is it possible that
arrago-git dies for the flood of  git ls-remote?

Andreas

[1] http://lists.linuxtogo.org/pipermail/bitbake-devel/2013-February/004178.html

Re: [meta-ti] trouble during parsing

[Trevor Woerner]

If the purpose of the ls-remote is to match a tag with a hash,
wouldn't it be easier to just:

    cat .git/refs/tags/<tagid>

?

And if the purpose of knowing the tagid is to use that version of the
code, wouldn't it be easier to just:

    git checkout <tagid>

?

Re: [meta-ti] trouble during parsing

[Richard Purdie]

On Mon, 2013-02-25 at 08:44 -0500, Trevor Woerner wrote:
> If the purpose of the ls-remote is to match a tag with a hash,
> wouldn't it be easier to just:
> 
>     cat .git/refs/tags/<tagid>
> 
> ?

If the git clone hasn't happened yet, this doesn't work.

> And if the purpose of knowing the tagid is to use that version of the
> code, wouldn't it be easier to just:
> 
>     git checkout <tagid>

That isn't the only purpose.

Cheers,

Richard

Re: [meta-ti] trouble during parsing

[Gary Thomas]

On 2013-02-25 07:01, Richard Purdie wrote:
> On Mon, 2013-02-25 at 08:44 -0500, Trevor Woerner wrote:
>> If the purpose of the ls-remote is to match a tag with a hash,
>> wouldn't it be easier to just:
>>
>>      cat .git/refs/tags/<tagid>
>>
>> ?
>
> If the git clone hasn't happened yet, this doesn't work.
>
>> And if the purpose of knowing the tagid is to use that version of the
>> code, wouldn't it be easier to just:
>>
>>      git checkout <tagid>
>
> That isn't the only purpose.

Additionally, the problem is that tags can move and the revid
will change.  Hence the need to probe the source GIT for the answer.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: [meta-ti] trouble during parsing

[Denys Dmytriyenko]

On Mon, Feb 25, 2013 at 11:02:21AM +0100, Andreas Müller wrote:
> On Mon, Feb 25, 2013 at 10:46 AM, Javier Martinez Canillas
> <javier@dowhile0.org> wrote:
> > On Mon, Feb 25, 2013 at 10:32 AM, Andreas Müller
> > <schnitzeltony@googlemail.com> wrote:
> >> On Mon, Feb 25, 2013 at 3:28 AM, Flanagan, Elizabeth
> >> <elizabeth.flanagan@intel.com> wrote:
> >>> On Sun, Feb 24, 2013 at 6:12 PM, Flanagan, Elizabeth
> >>> <elizabeth.flanagan@intel.com> wrote:
> >>>> On Wed, Feb 20, 2013 at 6:31 AM, Chris Larson <clarson@kergoth.com> wrote:
> >>>>>
> >>>>> On Wed, Feb 20, 2013 at 5:54 AM, Andreas Müller
> >>>>> <schnitzeltony@googlemail.com> wrote:
> >>>>>>
> >>>>>> with current bitbake master I get
> >>>>>>
> >>>>>> ERROR: Command execution failed: Traceback (most recent call
> >>>>>> last):#########################################################
> >>>>>>                               | ETA:  00:00:17
> >>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
> >>>>>> 92, in runAsyncCommand
> >>>>>>     self.cooker.updateCache()
> >>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> >>>>>> 1330, in updateCache
> >>>>>>     if not self.parser.parse_next():
> >>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> >>>>>> 1703, in parse_next
> >>>>>>     self.virtuals += len(result)
> >>>>>> TypeError: object of type 'ExpansionError' has no len()
> >>>>>
> >>>>>
> >>>>> Hmm, looks like it's returning the exceptions rather than raising them, for
> >>>>> some reason, but that doesn't make much sense — the pool code always raises
> >>>>> any exceptions from its imap iterator's next() method.
> >>>>> --
> >>>>> Christopher Larson
> >>>>>
> >>>>
> >>>> I'm hitting this as well with
> >>>> poky:dde7a481354d5b0539762109bdfaaba6f85f879b, meta-ti and pandaboard
> >>>> as my machine.
> >>>>
> >>>
> >>> Ach, should have updated my email. Seems reverting
> >>> 0a99563a4ea270594fd9a61da46f9387fb79dc66 cleared up the issue.
> >>>
> >>> -b
> >>>
> >> I have two machines (Fedora 14  / Fedora 15)  both reverted
> >> 0a99563a4ea270594fd9a61da46f9387fb79dc66 and both worked fine during
> >> weekend.
> >>
> >> Just a guess: Last week when arago git was down I saw errors during
> >> parsing for a recipe I never used. This was with a revision of bitbake
> >> a bit earlier than HEAD - that caused me to update. After sending a
> >> message to meta-ti I could work during weekend. Now it seems arago is
> >> down again:
> >>
> >> $ git clone git://arago-project.org/git/projects/u-boot-keystone.git
> >> Cloning into u-boot-keystone...
> >> fatal: read error: Connection reset by peer
> >>
> >> and bitbake -DD gives
> >>
> >> DEBUG: Fetcher accessed the network with the command git ls-remote
> >> git://arago-project.org/git/projects/u-boot-keystone.git
> >> DEV.MCSDK-03.00.00.07
> >> DEBUG: Running export SSH_AUTH_SOCK="/tmp/keyring-Pob6mG/ssh"; export
> >> PATH="/home/andreas/data/oe-core/sources/openembedded-core/scripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin/armv7a-vfp-neon-angstrom-linux-gnueabi:/home/andreas/tmp/oe-core-eglibc/sysroots/overo/usr/bin/crossscripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux//bin:/home/andreas/oe-core/sources/bitbake/bin:/home/andreas/data/OpenEmbedded/bitbake/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:./:/home/andreas/bin:/sbin:/usr/sbin";
> >> export HOME="/home/andreas"; git ls-remote
> >> git://arago-project.org/git/projects/u-boot-keystone.git
> >> DEV.MCSDK-03.00.00.07
> >> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
> >> argument 'preinst' is not a string literal
> >> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
> >> argument 'postinst' is not a string literal
> >>
> >> ...
> >>
> >> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
> >> argument 'preinst' is not a string literal
> >> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
> >> argument 'postinst' is not a string literal
> >> DEBUG: while parsing do_clean, in call of bb.build.exec_func, argument
> >> 'f' is not a string literal
> >> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
> >> argument 'preinst' is not a string literal
> >> ERROR: Command execution failed: Traceback (most recent call last):
> >>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
> >> 92, in runAsyncCommand
> >>     self.cooker.updateCache()
> >>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> >> 1329, in updateCache
> >>     if not self.parser.parse_next():
> >>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> >> 1702, in parse_next
> >>     self.virtuals += len(result)
> >> TypeError: object of type 'ExpansionError' has no len()
> >>
> >>
> >> In the recipe u-boot_2013.01.bb I find SRCREV = "DEV.MCSDK-03.00.00.07"
> >>
> >> which is not a commit ID and causes a git ls-remote during parse
> >>
> >> Is it possible that this error occurs when a git SRC_URI server closes
> >> connection unexpected?
> >>
> >> Andreas
> >
> > Hi Andreas,
> >
> > I have the same issue and your guess is correct. The problem is that
> > Arago servers are down so "git ls-remote" is failing and this makes
> > the BitBake task to fail as well.
> >
> > Now about the why is been used tag names instead of just commits IDs
> > so a "git ls-remote" call is needed, this is because some git trees in
> > arago are been rebased overwriting the commit IDs.
> >
> > You may take a look to Robert P. J. Day email "[meta-ti] objections to
> > replacing git SRCREV tag names with actual      hash IDs?" [1] to get a
> > better understanding of this.
> >
> > Best regards,
> > javier
> >
> > [1]: http://www.mail-archive.com/meta-ti@yoctoproject.org/msg01304.html
> 
> One additional note: With current bitbake there are configurations
> which cause a full parse on every build [1]. Is it possible that
> arrago-git dies for the flood of  git ls-remote?
> 
> Andreas
> 
> [1] http://lists.linuxtogo.org/pipermail/bitbake-devel/2013-February/004178.html

All,

First of all, sorry for the troubles with the server connection. It should be 
fine now.

Second, there is a hard limit of maximum allowed simultaneous git connections 
on the server due to limited resources, and the limit was maxed out. The 
server was still running fine, but it would not accept any more connections. 
It looked like a DoS attack - connections were still running and packing data. 
Most of the connections were from China - I'm wondering how long does it take 
to clone a large git tree from over there...

Anyway, there are some fixes related to this problem being discussed and will 
be implemented shortly...

And BTW, bitbake's git-ls-remote is very fragile and we've seen it failing on 
a good server connection (even with kernel.org and github.com) very easily. 
There needs to be a re-try count implemented for that code as well...

-- 
Denys

Re: [bitbake-devel] trouble during parsing

[Denys Dmytriyenko]

On Mon, Feb 25, 2013 at 11:02:21AM +0100, Andreas Müller wrote:
> On Mon, Feb 25, 2013 at 10:46 AM, Javier Martinez Canillas
> <javier@dowhile0.org> wrote:
> > On Mon, Feb 25, 2013 at 10:32 AM, Andreas Müller
> > <schnitzeltony@googlemail.com> wrote:
> >> On Mon, Feb 25, 2013 at 3:28 AM, Flanagan, Elizabeth
> >> <elizabeth.flanagan@intel.com> wrote:
> >>> On Sun, Feb 24, 2013 at 6:12 PM, Flanagan, Elizabeth
> >>> <elizabeth.flanagan@intel.com> wrote:
> >>>> On Wed, Feb 20, 2013 at 6:31 AM, Chris Larson <clarson@kergoth.com> wrote:
> >>>>>
> >>>>> On Wed, Feb 20, 2013 at 5:54 AM, Andreas Müller
> >>>>> <schnitzeltony@googlemail.com> wrote:
> >>>>>>
> >>>>>> with current bitbake master I get
> >>>>>>
> >>>>>> ERROR: Command execution failed: Traceback (most recent call
> >>>>>> last):#########################################################
> >>>>>>                               | ETA:  00:00:17
> >>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
> >>>>>> 92, in runAsyncCommand
> >>>>>>     self.cooker.updateCache()
> >>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> >>>>>> 1330, in updateCache
> >>>>>>     if not self.parser.parse_next():
> >>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> >>>>>> 1703, in parse_next
> >>>>>>     self.virtuals += len(result)
> >>>>>> TypeError: object of type 'ExpansionError' has no len()
> >>>>>
> >>>>>
> >>>>> Hmm, looks like it's returning the exceptions rather than raising them, for
> >>>>> some reason, but that doesn't make much sense — the pool code always raises
> >>>>> any exceptions from its imap iterator's next() method.
> >>>>> --
> >>>>> Christopher Larson
> >>>>>
> >>>>
> >>>> I'm hitting this as well with
> >>>> poky:dde7a481354d5b0539762109bdfaaba6f85f879b, meta-ti and pandaboard
> >>>> as my machine.
> >>>>
> >>>
> >>> Ach, should have updated my email. Seems reverting
> >>> 0a99563a4ea270594fd9a61da46f9387fb79dc66 cleared up the issue.
> >>>
> >>> -b
> >>>
> >> I have two machines (Fedora 14  / Fedora 15)  both reverted
> >> 0a99563a4ea270594fd9a61da46f9387fb79dc66 and both worked fine during
> >> weekend.
> >>
> >> Just a guess: Last week when arago git was down I saw errors during
> >> parsing for a recipe I never used. This was with a revision of bitbake
> >> a bit earlier than HEAD - that caused me to update. After sending a
> >> message to meta-ti I could work during weekend. Now it seems arago is
> >> down again:
> >>
> >> $ git clone git://arago-project.org/git/projects/u-boot-keystone.git
> >> Cloning into u-boot-keystone...
> >> fatal: read error: Connection reset by peer
> >>
> >> and bitbake -DD gives
> >>
> >> DEBUG: Fetcher accessed the network with the command git ls-remote
> >> git://arago-project.org/git/projects/u-boot-keystone.git
> >> DEV.MCSDK-03.00.00.07
> >> DEBUG: Running export SSH_AUTH_SOCK="/tmp/keyring-Pob6mG/ssh"; export
> >> PATH="/home/andreas/data/oe-core/sources/openembedded-core/scripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin/armv7a-vfp-neon-angstrom-linux-gnueabi:/home/andreas/tmp/oe-core-eglibc/sysroots/overo/usr/bin/crossscripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux//bin:/home/andreas/oe-core/sources/bitbake/bin:/home/andreas/data/OpenEmbedded/bitbake/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:./:/home/andreas/bin:/sbin:/usr/sbin";
> >> export HOME="/home/andreas"; git ls-remote
> >> git://arago-project.org/git/projects/u-boot-keystone.git
> >> DEV.MCSDK-03.00.00.07
> >> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
> >> argument 'preinst' is not a string literal
> >> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
> >> argument 'postinst' is not a string literal
> >>
> >> ...
> >>
> >> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
> >> argument 'preinst' is not a string literal
> >> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
> >> argument 'postinst' is not a string literal
> >> DEBUG: while parsing do_clean, in call of bb.build.exec_func, argument
> >> 'f' is not a string literal
> >> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
> >> argument 'preinst' is not a string literal
> >> ERROR: Command execution failed: Traceback (most recent call last):
> >>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
> >> 92, in runAsyncCommand
> >>     self.cooker.updateCache()
> >>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> >> 1329, in updateCache
> >>     if not self.parser.parse_next():
> >>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
> >> 1702, in parse_next
> >>     self.virtuals += len(result)
> >> TypeError: object of type 'ExpansionError' has no len()
> >>
> >>
> >> In the recipe u-boot_2013.01.bb I find SRCREV = "DEV.MCSDK-03.00.00.07"
> >>
> >> which is not a commit ID and causes a git ls-remote during parse
> >>
> >> Is it possible that this error occurs when a git SRC_URI server closes
> >> connection unexpected?
> >>
> >> Andreas
> >
> > Hi Andreas,
> >
> > I have the same issue and your guess is correct. The problem is that
> > Arago servers are down so "git ls-remote" is failing and this makes
> > the BitBake task to fail as well.
> >
> > Now about the why is been used tag names instead of just commits IDs
> > so a "git ls-remote" call is needed, this is because some git trees in
> > arago are been rebased overwriting the commit IDs.
> >
> > You may take a look to Robert P. J. Day email "[meta-ti] objections to
> > replacing git SRCREV tag names with actual      hash IDs?" [1] to get a
> > better understanding of this.
> >
> > Best regards,
> > javier
> >
> > [1]: http://www.mail-archive.com/meta-ti@yoctoproject.org/msg01304.html
> 
> One additional note: With current bitbake there are configurations
> which cause a full parse on every build [1]. Is it possible that
> arrago-git dies for the flood of  git ls-remote?
> 
> Andreas
> 
> [1] http://lists.linuxtogo.org/pipermail/bitbake-devel/2013-February/004178.html

All,

First of all, sorry for the troubles with the server connection. It should be 
fine now.

Second, there is a hard limit of maximum allowed simultaneous git connections 
on the server due to limited resources, and the limit was maxed out. The 
server was still running fine, but it would not accept any more connections. 
It looked like a DoS attack - connections were still running and packing data. 
Most of the connections were from China - I'm wondering how long does it take 
to clone a large git tree from over there...

Anyway, there are some fixes related to this problem being discussed and will 
be implemented shortly...

And BTW, bitbake's git-ls-remote is very fragile and we've seen it failing on 
a good server connection (even with kernel.org and github.com) very easily. 
There needs to be a re-try count implemented for that code as well...

-- 
Denys

Re: trouble during parsing

[Jason Wessel]

On 02/24/2013 08:28 PM, Flanagan, Elizabeth wrote:
> On Sun, Feb 24, 2013 at 6:12 PM, Flanagan, Elizabeth
> <elizabeth.flanagan@intel.com> wrote:
>> On Wed, Feb 20, 2013 at 6:31 AM, Chris Larson <clarson@kergoth.com> wrote:
>>>
>>> On Wed, Feb 20, 2013 at 5:54 AM, Andreas Müller
>>> <schnitzeltony@googlemail.com> wrote:
>>>>
>>>> with current bitbake master I get
>>>>
>>>> ERROR: Command execution failed: Traceback (most recent call
>>>> last):#########################################################
>>>> | ETA: 00:00:17
>>>> File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
>>>> 92, in runAsyncCommand
>>>> self.cooker.updateCache()
>>>> File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>> 1330, in updateCache
>>>> if not self.parser.parse_next():
>>>> File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>> 1703, in parse_next
>>>> self.virtuals += len(result)
>>>> TypeError: object of type 'ExpansionError' has no len()
>>>
>>>
>>> Hmm, looks like it's returning the exceptions rather than raising them, for
>>> some reason, but that doesn't make much sense — the pool code always raises
>>> any exceptions from its imap iterator's next() method.
>>
>
> Ach, should have updated my email. Seems reverting
> 0a99563a4ea270594fd9a61da46f9387fb79dc66 cleared up the issue.

The real question is what are the expansion errors about here.  This is probably not the right way to fix it, but it at least allows you to see what is behind the expansion error:

diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index 448f899..b166425 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -1661,6 +1661,8 @@ class CookerParser(object):
         parsed = None
         try:
             parsed, result = self.results.next()
+            if isinstance(result, BaseException):
+                raise result
         except StopIteration:
             self.shutdown()
             return False

Re: [meta-ti] trouble during parsing

[Trevor Woerner]

On Mon, Feb 25, 2013 at 9:07 AM, Gary Thomas <gary@mlbassoc.com> wrote:
> Additionally, the problem is that tags can move and the revid
> will change.  Hence the need to probe the source GIT for the answer.

If a tag has moved then a

    $ git fetch --tags

would update your local tag refspecs with the updated information, wouldn't it?