Re: Robert Yang : perf: fix for rebuilding

Re: Robert Yang : perf: fix for rebuilding

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1757 bytes --]

I'm still seeing issues when perf is rebuilt

The original issue was:
OSError: [Errno 2] No such file or directory: 'BUILD/work/qemux86-webos-linux/perf/1.0-r9/perf-1.0'

as described here:
http://patches.openembedded.org/patch/94719/

First part of fix is now backported to dizzy:
commit 40aeb23c427e0c56f7073fc54ea63bf0eabbe092
Author: Robert Yang <liezhi.yang@windriver.com>
Date:   Wed Mar 11 01:35:38 2015 -0700

    license.bbclass: set dirs for do_populate_lic_setscene

but perf is still failing in some builds with the same error when
do_configure and do_populate_lic are executed at the same time:

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
  File "bitbake/lib/bb/build.py", line 497, in exec_task
    return _exec_task(fn, task, d, quieterr)
  File "bitbake/lib/bb/build.py", line 440, in _exec_task
    exec_func(func, localdata)
  File "bitbake/lib/bb/build.py", line 212, in exec_func
    exec_func_python(func, d, runfile, cwd=adir)
  File "bitbake/lib/bb/build.py", line 237, in exec_func_python
    os.chdir(cwd)
OSError: [Errno 2] No such file or directory: '....../perf/1.0-r9/perf-1.0'

NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 3826 (../oe-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded

So is there some 2nd/3rd part of this fix or is it broken in master?

I haven't found any other commit related to this perf configure or
populate_lic in master and fido.

Regards,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: Robert Yang : perf: fix for rebuilding

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 4084 bytes --]

On Wed, Sep 02, 2015 at 06:00:35PM +0200, Martin Jansa wrote:
> I'm still seeing issues when perf is rebuilt
> 
> The original issue was:
> OSError: [Errno 2] No such file or directory: 'BUILD/work/qemux86-webos-linux/perf/1.0-r9/perf-1.0'
> 
> as described here:
> http://patches.openembedded.org/patch/94719/
> 
> First part of fix is now backported to dizzy:
> commit 40aeb23c427e0c56f7073fc54ea63bf0eabbe092
> Author: Robert Yang <liezhi.yang@windriver.com>
> Date:   Wed Mar 11 01:35:38 2015 -0700
> 
>     license.bbclass: set dirs for do_populate_lic_setscene
> 
> but perf is still failing in some builds with the same error when
> do_configure and do_populate_lic are executed at the same time:
> 
> NOTE: recipe perf-1.0-r9: task do_configure: Started
> NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
> ERROR: Build of do_populate_lic failed
> ERROR: Traceback (most recent call last):
>   File "bitbake/lib/bb/build.py", line 497, in exec_task
>     return _exec_task(fn, task, d, quieterr)
>   File "bitbake/lib/bb/build.py", line 440, in _exec_task
>     exec_func(func, localdata)
>   File "bitbake/lib/bb/build.py", line 212, in exec_func
>     exec_func_python(func, d, runfile, cwd=adir)
>   File "bitbake/lib/bb/build.py", line 237, in exec_func_python
>     os.chdir(cwd)
> OSError: [Errno 2] No such file or directory: '....../perf/1.0-r9/perf-1.0'
> 
> NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
> ERROR: Task 3826 (../oe-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
> NOTE: recipe perf-1.0-r9: task do_configure: Succeeded
> 
> So is there some 2nd/3rd part of this fix or is it broken in master?
> 
> I haven't found any other commit related to this perf configure or
> populate_lic in master and fido.

It's still broken in master.

Easy way to reproduce this is:
OE qemux86@ ~/build/oe-core $ for i in `seq -w 1 100`; do bitbake -c cleansstate perf; bitbake -c build perf | tee log.perf.$i; done

On slow machine I got first failure on 60th iteration, on
much faster machine I got 4 failures in first 20 iterations.

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Function failed: do_configure (log file is located at /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_configure.10700)
ERROR: Logfile of failure stored in: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_configure.10700
Log data follows:
| DEBUG: Executing python function sysroot_cleansstate
| DEBUG: Python function sysroot_cleansstate finished
| DEBUG: Executing shell function do_configure
| mkdir: cannot create directory '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0/': File exists
| WARNING: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/run.do_configure.10700:1 exit 1 from
|   mkdir /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0/
| ERROR: Function failed: do_configure (log file is located at /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_configure.10700)
NOTE: recipe perf-1.0-r9: task do_configure: Failed
ERROR: Task 5 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_configure) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_populate_lic: Succeeded
NOTE: Tasks Summary: Attempted 833 tasks of which 828 didn't need to be rerun and 1 failed.

Can you please fix it properly instead of removing whole ${B}?

do_configure_prepend () {
    # Fix for rebuilding
    rm -rf ${B}/
    mkdir ${B}/
...

I wish there is easier way to undo this commit in .bbappend, we very
rarely rebuild perf in the same build directory, but too often we see
normal clean builds failing in perf do_configure or do_populate_lic tasks :/.

Regards and thanks for feedback (it's broken for 8 months and no reply
after it was reported 3 months ago).

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

[PATCH][master][fido][dizzy] Revert "perf: fix for rebuilding"

[Martin Jansa]

* This reverts commit 9dafa571ed0a40d21a886dec7704c31150b21942.

* The "fix" is causing more issues then it's fixing and there was no
  feedback in many months.
* More info:
  http://lists.openembedded.org/pipermail/openembedded-core/2015-June/105684.html
  http://lists.openembedded.org/pipermail/openembedded-core/2015-September/109870.html
  http://lists.openembedded.org/pipermail/openembedded-core/2015-September/110338.html

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 meta/recipes-kernel/perf/perf.bb | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index adb3a2c..f1d7894 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -118,10 +118,6 @@ do_install() {
 }
 
 do_configure_prepend () {
-    # Fix for rebuilding
-    rm -rf ${B}/
-    mkdir ${B}/
-
     # If building a multlib based perf, the incorrect library path will be
     # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a 32 bit
     # build, with a 64 bit multilib, the arch won't match and the detection of a 
-- 
2.5.1

Re: [PATCH][master][fido][dizzy] Revert "perf: fix for rebuilding"

[Robert Yang]

Hi Martin,

Sorry, I'm blocked by mutitlib SDK issues recently, I will fix it sooner.
Revert the patch is fine to me if it is urgent.

// Robert

On 09/15/2015 01:29 PM, Martin Jansa wrote:
> * This reverts commit 9dafa571ed0a40d21a886dec7704c31150b21942.
>
> * The "fix" is causing more issues then it's fixing and there was no
>    feedback in many months.
> * More info:
>    http://lists.openembedded.org/pipermail/openembedded-core/2015-June/105684.html
>    http://lists.openembedded.org/pipermail/openembedded-core/2015-September/109870.html
>    http://lists.openembedded.org/pipermail/openembedded-core/2015-September/110338.html
>
> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> ---
>   meta/recipes-kernel/perf/perf.bb | 4 ----
>   1 file changed, 4 deletions(-)
>
> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
> index adb3a2c..f1d7894 100644
> --- a/meta/recipes-kernel/perf/perf.bb
> +++ b/meta/recipes-kernel/perf/perf.bb
> @@ -118,10 +118,6 @@ do_install() {
>   }
>
>   do_configure_prepend () {
> -    # Fix for rebuilding
> -    rm -rf ${B}/
> -    mkdir ${B}/
> -
>       # If building a multlib based perf, the incorrect library path will be
>       # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a 32 bit
>       # build, with a 64 bit multilib, the arch won't match and the detection of a
>

Re: [PATCH][master][fido][dizzy] Revert "perf: fix for rebuilding"

[Robert Yang]

Hi Martin,

I've figured out the reason, this is because the task's default cwd
is ${B}, the easier way to fix the problem is use mkdir -p rather than
mkdir, autotools.bbclass also has this problem, but I'd like to fix
exec_func, I will send a patch to bitbake-devel after more testing.

// Robert

On 09/15/2015 01:53 PM, Robert Yang wrote:
>
> Hi Martin,
>
> Sorry, I'm blocked by mutitlib SDK issues recently, I will fix it sooner.
> Revert the patch is fine to me if it is urgent.
>
> // Robert
>
> On 09/15/2015 01:29 PM, Martin Jansa wrote:
>> * This reverts commit 9dafa571ed0a40d21a886dec7704c31150b21942.
>>
>> * The "fix" is causing more issues then it's fixing and there was no
>>    feedback in many months.
>> * More info:
>>
>> http://lists.openembedded.org/pipermail/openembedded-core/2015-June/105684.html
>>
>> http://lists.openembedded.org/pipermail/openembedded-core/2015-September/109870.html
>>
>>
>> http://lists.openembedded.org/pipermail/openembedded-core/2015-September/110338.html
>>
>>
>> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
>> ---
>>   meta/recipes-kernel/perf/perf.bb | 4 ----
>>   1 file changed, 4 deletions(-)
>>
>> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
>> index adb3a2c..f1d7894 100644
>> --- a/meta/recipes-kernel/perf/perf.bb
>> +++ b/meta/recipes-kernel/perf/perf.bb
>> @@ -118,10 +118,6 @@ do_install() {
>>   }
>>
>>   do_configure_prepend () {
>> -    # Fix for rebuilding
>> -    rm -rf ${B}/
>> -    mkdir ${B}/
>> -
>>       # If building a multlib based perf, the incorrect library path will be
>>       # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a
>> 32 bit
>>       # build, with a 64 bit multilib, the arch won't match and the detection
>> of a
>>

Re: [PATCH][master][fido][dizzy] Revert "perf: fix for rebuilding"

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 632 bytes --]

On 16 September 2015 at 08:48, Robert Yang <liezhi.yang@windriver.com>
wrote:

> I've figured out the reason, this is because the task's default cwd
> is ${B}, the easier way to fix the problem is use mkdir -p rather than
> mkdir, autotools.bbclass also has this problem, but I'd like to fix
> exec_func, I will send a patch to bitbake-devel after more testing.
>

Changing the default $B of [dirs] is not something we'll be doing post-M3.
I've a branch that changes that default and have fixed all the obvious
breakage in oe-core already so this is something on my plan for immediately
after 2.1 branches off.

Ross

[-- Attachment #2: Type: text/html, Size: 1062 bytes --]

Re: [PATCH][master][fido][dizzy] Revert "perf: fix for rebuilding"

[Robert Yang]

On 09/16/2015 04:57 PM, Burton, Ross wrote:
>
> On 16 September 2015 at 08:48, Robert Yang <liezhi.yang@windriver.com
> <mailto:liezhi.yang@windriver.com>> wrote:
>
>     I've figured out the reason, this is because the task's default cwd
>     is ${B}, the easier way to fix the problem is use mkdir -p rather than
>     mkdir, autotools.bbclass also has this problem, but I'd like to fix
>     exec_func, I will send a patch to bitbake-devel after more testing.
>
>
> Changing the default $B of [dirs] is not something we'll be doing post-M3. I've

Thanks, so let's simply use "mkdir -p" to fix the problem, atm ?

> a branch that changes that default and have fixed all the obvious breakage in
> oe-core already so this is something on my plan for immediately after 2.1
> branches off.

I made 2 patches just now, one is for bitbake, and one for insane.bbclass,
  maybe we are doing the similar things, I can drop them then.

     build.py: default exec_func's cwd to WORKDIR

     This can fix a few problems:
     - The ${B} was nearly always created in the past after any tak runs, for
       example, the ${B} exists after do_clean or do_cleansstate (first
       removed, then created), but ${B} is useless and confused end user in
       this case, the similar to a lot of prefuncs and postfuncs which also
       create ${B}, but not used. This patch fixes the problem.

     - This can fix race issue when we use the following commands in other
       tasks:
         rm -rf ${B}
         mkdir ${B}
       such as autotools.bbclass and perf.bb.

     When the 'dirs' or 'cleandirs' is not specified by task, which means
     that they are not important, so default to WORKDIR which is more common
     and usually existed.

     Signed-off-by: Robert Yang <liezhi.yang@windriver.com>

diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py
index 948c395..a413c35 100644
--- a/bitbake/lib/bb/build.py
+++ b/bitbake/lib/bb/build.py
@@ -182,8 +182,9 @@ def exec_func(func, d, dirs = None):
              bb.utils.mkdirhier(adir)
          adir = dirs[-1]
      else:
-        adir = d.getVar('B', True)
-        bb.utils.mkdirhier(adir)
+        adir = d.getVar('WORKDIR', True)
+        if not os.path.exists(adir):
+            bb.utils.mkdirhier(adir)

      ispython = flags.get('python')

===================

     insane.bbclass: only do_qa_unpack warn when SRC_URI is not null

     If SRC_URI is null, then no source is needed to be unpacked, thus no
     warn.

     Signed-off-by: Robert Yang <liezhi.yang@windriver.com>

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 5c8629a..e22e8a0 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -1214,7 +1214,7 @@ python do_qa_unpack() {
      bb.note("Checking has ${S} been created")

      s_dir = d.getVar('S', True)
-    if not os.path.exists(s_dir):
+    if not os.path.exists(s_dir) and d.getVar('SRC_URI', True):
          bb.warn('%s: the directory %s (%s) pointed to by the S variable 
doesn\'t exist - please set S within the recipe to point to where the source has 
been unpacked to' % (d.getVar('PN', T
  }

// Robert
>
> Ross

Re: [PATCH][master][fido][dizzy] Revert "perf: fix for rebuilding"

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 6334 bytes --]

On Wed, Sep 16, 2015 at 03:48:58PM +0800, Robert Yang wrote:
> 
> Hi Martin,
> 
> I've figured out the reason, this is because the task's default cwd
> is ${B}, the easier way to fix the problem is use mkdir -p rather than
> mkdir, autotools.bbclass also has this problem, but I'd like to fix
> exec_func, I will send a patch to bitbake-devel after more testing.

Unfortunately "mkdir -p" fixes only part of this problem, it still fails
quite often when bitbake tries to "os.chdir(cwd)" in populate_lic
after do_configure removed ${B} and before do_configure re-created it
with "mkdir -p".

So 3 from 50 iterations (if I use the same for cycle to test this) fail
with:

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 553, in exec_task
    return _exec_task(fn, task, d, quieterr)
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 493, in _exec_task
    exec_func(func, localdata)
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 227, in exec_func
    exec_func_python(func, d, runfile, cwd=adir)
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 252, in exec_func_python
    os.chdir(cwd)
OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'

NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded


or even longer exception:

NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
NOTE: recipe perf-1.0-r9: task do_configure: Started
ERROR: Error executing a python function in /OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'sstate_task_postfunc', lineno: 14, function: <module>
     0010:    sstate_package(shared_state, d)
     0011:    os.umask(omask)
     0012:
     0013:
 *** 0014:sstate_task_postfunc(d)
     0015:
File: 'sstate_task_postfunc', lineno: 4, function: sstate_task_postfunc
     0001:
     0002:def sstate_task_postfunc(d):
     0003:    shared_state = sstate_state_fromvars(d)
 *** 0004:    sstate_install(shared_state, d)
     0005:    for intercept in shared_state['interceptfuncs']:
     0006:        bb.build.exec_func(intercept, d)
     0007:    omask = os.umask(002)
     0008:    if omask != 002:
File: 'sstate.bbclass', lineno: 113, function: sstate_install
     0109:        if os.path.exists(state[1]):
     0110:            oe.path.copyhardlinktree(state[1], state[2])
     0111:
     0112:    for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
 *** 0113:        bb.build.exec_func(postinst, d)
     0114:
     0115:    for lock in locks:
     0116:        bb.utils.unlockfile(lock)
     0117:
File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 227, function: exec_func
     0223:                pass
     0224:
     0225:    with bb.utils.fileslocked(lockfiles):
     0226:        if ispython:
 *** 0227:            exec_func_python(func, d, runfile, cwd=adir)
    0228:        else:
     0229:            exec_func_shell(func, d, runfile, cwd=adir)
     0230:
     0231:_functionfmt = """
File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 252, function: exec_func_python
     0248:        try:
     0249:            olddir = os.getcwd()
     0250:        except OSError:
     0251:            olddir = None
 *** 0252:        os.chdir(cwd)
     0253:
     0254:    bb.debug(2, "Executing python function %s" % func)
     0255:
     0256:    try:
Exception: OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'

ERROR: Function failed: sstate_task_postfunc
ERROR: Logfile of failure stored in: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_populate_lic.16588
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded
NOTE: Tasks Summary: Attempted 840 tasks of which 835 didn't need to be rerun and 1 failed.

So this revert (or more complete fix) is still needed in master, fido and dizzy.

Regards,

> 
> // Robert
> 
> On 09/15/2015 01:53 PM, Robert Yang wrote:
> >
> > Hi Martin,
> >
> > Sorry, I'm blocked by mutitlib SDK issues recently, I will fix it sooner.
> > Revert the patch is fine to me if it is urgent.
> >
> > // Robert
> >
> > On 09/15/2015 01:29 PM, Martin Jansa wrote:
> >> * This reverts commit 9dafa571ed0a40d21a886dec7704c31150b21942.
> >>
> >> * The "fix" is causing more issues then it's fixing and there was no
> >>    feedback in many months.
> >> * More info:
> >>
> >> http://lists.openembedded.org/pipermail/openembedded-core/2015-June/105684.html
> >>
> >> http://lists.openembedded.org/pipermail/openembedded-core/2015-September/109870.html
> >>
> >>
> >> http://lists.openembedded.org/pipermail/openembedded-core/2015-September/110338.html
> >>
> >>
> >> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> >> ---
> >>   meta/recipes-kernel/perf/perf.bb | 4 ----
> >>   1 file changed, 4 deletions(-)
> >>
> >> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
> >> index adb3a2c..f1d7894 100644
> >> --- a/meta/recipes-kernel/perf/perf.bb
> >> +++ b/meta/recipes-kernel/perf/perf.bb
> >> @@ -118,10 +118,6 @@ do_install() {
> >>   }
> >>
> >>   do_configure_prepend () {
> >> -    # Fix for rebuilding
> >> -    rm -rf ${B}/
> >> -    mkdir ${B}/
> >> -
> >>       # If building a multlib based perf, the incorrect library path will be
> >>       # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a
> >> 32 bit
> >>       # build, with a 64 bit multilib, the arch won't match and the detection
> >> of a
> >>

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

[RFC][PATCH] perf: don't re-create ${B}

[Martin Jansa]

* otherwise there is race-condition between do_configure rm+mkdir and
  bitbake trying to use it as CWD for do_populate_lic task, which
  results in errors like this:

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 553, in exec_task
    return _exec_task(fn, task, d, quieterr)
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 493, in _exec_task
    exec_func(func, localdata)
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 227, in exec_func
    exec_func_python(func, d, runfile, cwd=adir)
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 252, in exec_func_python
    os.chdir(cwd)
OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded

or even longer exception:

NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
NOTE: recipe perf-1.0-r9: task do_configure: Started
ERROR: Error executing a python function in /OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'sstate_task_postfunc', lineno: 14, function: <module>
     0010:    sstate_package(shared_state, d)
     0011:    os.umask(omask)
     0012:
     0013:
 *** 0014:sstate_task_postfunc(d)
     0015:
File: 'sstate_task_postfunc', lineno: 4, function: sstate_task_postfunc
     0001:
     0002:def sstate_task_postfunc(d):
     0003:    shared_state = sstate_state_fromvars(d)
 *** 0004:    sstate_install(shared_state, d)
     0005:    for intercept in shared_state['interceptfuncs']:
     0006:        bb.build.exec_func(intercept, d)
     0007:    omask = os.umask(002)
     0008:    if omask != 002:
File: 'sstate.bbclass', lineno: 113, function: sstate_install
     0109:        if os.path.exists(state[1]):
     0110:            oe.path.copyhardlinktree(state[1], state[2])
     0111:
     0112:    for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
 *** 0113:        bb.build.exec_func(postinst, d)
     0114:
     0115:    for lock in locks:
     0116:        bb.utils.unlockfile(lock)
     0117:
File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 227, function: exec_func
     0223:                pass
     0224:
     0225:    with bb.utils.fileslocked(lockfiles):
     0226:        if ispython:
 *** 0227:            exec_func_python(func, d, runfile, cwd=adir)
    0228:        else:
     0229:            exec_func_shell(func, d, runfile, cwd=adir)
     0230:
     0231:_functionfmt = """
File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 252, function: exec_func_python
     0248:        try:
     0249:            olddir = os.getcwd()
     0250:        except OSError:
     0251:            olddir = None
 *** 0252:        os.chdir(cwd)
     0253:
     0254:    bb.debug(2, "Executing python function %s" % func)
     0255:
     0256:    try:
Exception: OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'

ERROR: Function failed: sstate_task_postfunc
ERROR: Logfile of failure stored in: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_populate_lic.16588
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded
NOTE: Tasks Summary: Attempted 840 tasks of which 835 didn't need to be rerun and 1 failed.

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 meta/recipes-kernel/perf/perf.bb | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index 22bd3c8..7f5c3b3 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -119,8 +119,7 @@ do_install() {
 
 do_configure_prepend () {
     # Fix for rebuilding
-    rm -rf ${B}/
-    mkdir -p ${B}/
+    find ${B} -mindepth 1 -maxdepth 1 -delete
 
     # If building a multlib based perf, the incorrect library path will be
     # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a 32 bit
-- 
2.5.1

[RFC][PATCHv2] cmake.bbclass, perf: don't re-create ${B}

[Martin Jansa]

* otherwise there is race-condition between do_configure rm+mkdir and
  bitbake trying to use it as CWD for do_populate_lic task, which
  results in errors like this:

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 553, in exec_task
    return _exec_task(fn, task, d, quieterr)
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 493, in _exec_task
    exec_func(func, localdata)
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 227, in exec_func
    exec_func_python(func, d, runfile, cwd=adir)
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 252, in exec_func_python
    os.chdir(cwd)
OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded

or even longer exception:

NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
NOTE: recipe perf-1.0-r9: task do_configure: Started
ERROR: Error executing a python function in /OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'sstate_task_postfunc', lineno: 14, function: <module>
     0010:    sstate_package(shared_state, d)
     0011:    os.umask(omask)
     0012:
     0013:
 *** 0014:sstate_task_postfunc(d)
     0015:
File: 'sstate_task_postfunc', lineno: 4, function: sstate_task_postfunc
     0001:
     0002:def sstate_task_postfunc(d):
     0003:    shared_state = sstate_state_fromvars(d)
 *** 0004:    sstate_install(shared_state, d)
     0005:    for intercept in shared_state['interceptfuncs']:
     0006:        bb.build.exec_func(intercept, d)
     0007:    omask = os.umask(002)
     0008:    if omask != 002:
File: 'sstate.bbclass', lineno: 113, function: sstate_install
     0109:        if os.path.exists(state[1]):
     0110:            oe.path.copyhardlinktree(state[1], state[2])
     0111:
     0112:    for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
 *** 0113:        bb.build.exec_func(postinst, d)
     0114:
     0115:    for lock in locks:
     0116:        bb.utils.unlockfile(lock)
     0117:
File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 227, function: exec_func
     0223:                pass
     0224:
     0225:    with bb.utils.fileslocked(lockfiles):
     0226:        if ispython:
 *** 0227:            exec_func_python(func, d, runfile, cwd=adir)
    0228:        else:
     0229:            exec_func_shell(func, d, runfile, cwd=adir)
     0230:
     0231:_functionfmt = """
File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 252, function: exec_func_python
     0248:        try:
     0249:            olddir = os.getcwd()
     0250:        except OSError:
     0251:            olddir = None
 *** 0252:        os.chdir(cwd)
     0253:
     0254:    bb.debug(2, "Executing python function %s" % func)
     0255:
     0256:    try:
Exception: OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'

ERROR: Function failed: sstate_task_postfunc
ERROR: Logfile of failure stored in: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_populate_lic.16588
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded
NOTE: Tasks Summary: Attempted 840 tasks of which 835 didn't need to be rerun and 1 failed.

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 meta/classes/cmake.bbclass       | 3 +--
 meta/recipes-kernel/perf/perf.bb | 3 +--
 2 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index ae3cc02..e31799a 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -91,8 +91,7 @@ cmake_do_configure() {
 	fi
 
 	if [ "${S}" != "${B}" ]; then
-		rm -rf ${B}
-		mkdir -p ${B}
+		find ${B} -mindepth 1 -maxdepth 1 | xargs rm -rf
 		cd ${B}
 	else
 		find ${B} -name CMakeFiles -or -name Makefile -or -name cmake_install.cmake -or -name CMakeCache.txt -delete
diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index 22bd3c8..a5a04af 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -119,8 +119,7 @@ do_install() {
 
 do_configure_prepend () {
     # Fix for rebuilding
-    rm -rf ${B}/
-    mkdir -p ${B}/
+    find ${B} -mindepth 1 -maxdepth 1 | xargs rm -rf
 
     # If building a multlib based perf, the incorrect library path will be
     # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a 32 bit
-- 
2.5.1

Re: [RFC][PATCHv2] cmake.bbclass, perf: don't re-create ${B}

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 995 bytes --]

On 21 September 2015 at 21:33, Martin Jansa <martin.jansa@gmail.com> wrote:

> * otherwise there is race-condition between do_configure rm+mkdir and
>   bitbake trying to use it as CWD for do_populate_lic task, which
>   results in errors like this:
>

It took me a few minutes to understand this race:

Task A executing perf:do_configure and deleting ${B}
Task B finishing perf:do_populate_lic and so running the sstate generation
logic and the functions listed in SSTATEPOSTINSTFUNCS, which get executed
with the default cwd of ${B}.

I'll note that autotools.bbclass also does rmdir/mkdir ${B} so I'm curious
why you see it more in perf and cmake and not everything else.

Would a better long-term fix be to pass cwd=${WORKDIR} to exec_func in the
sstate code to stop the functions being executed in ${B} (after vetting the
functions, obviously).  FWIW I'm the owner of the bug to stop bitbake
knowing about ${B} which is leading to exactly this sort of problem.

Ross

[-- Attachment #2: Type: text/html, Size: 1637 bytes --]

Re: [RFC][PATCHv2] cmake.bbclass, perf: don't re-create ${B}

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2649 bytes --]

On Mon, Sep 21, 2015 at 09:55:56PM +0100, Burton, Ross wrote:
> On 21 September 2015 at 21:33, Martin Jansa <martin.jansa@gmail.com> wrote:
> 
> > * otherwise there is race-condition between do_configure rm+mkdir and
> >   bitbake trying to use it as CWD for do_populate_lic task, which
> >   results in errors like this:
> >
> 
> It took me a few minutes to understand this race:
> 
> Task A executing perf:do_configure and deleting ${B}
> Task B finishing perf:do_populate_lic and so running the sstate generation
> logic and the functions listed in SSTATEPOSTINSTFUNCS, which get executed
> with the default cwd of ${B}.

Thanks for more detailed explanation, sorry if I wasn't clear on this
one in my previous e-mails.

> I'll note that autotools.bbclass also does rmdir/mkdir ${B} so I'm curious
> why you see it more in perf and cmake and not everything else.

perf for some reason shows this most often, most our internal components are
using cmake, that's probably cmake was easier to find in our logs, but
I didn't grep all of them, maybe there are similar exceptions in
autotools based components. I'll send v3 with autotools.bbclass fix as
well.

> Would a better long-term fix be to pass cwd=${WORKDIR} to exec_func in the
> sstate code to stop the functions being executed in ${B} (after vetting the
> functions, obviously).  FWIW I'm the owner of the bug to stop bitbake
> knowing about ${B} which is leading to exactly this sort of problem.

Yes, I agree it would be better, but as you said, such change is too
late even for master branch and I need to fix this long-standing issue
even in fido and dizzy, so smaller, simpler change will be easier to get
backported.

My for cycle finished 469 iterations over night and no exception since
this fix:
OE @ ~/build/oe-core $ grep "ERROR: Task" log.perf.00*
log.perf.0003:ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
log.perf.0004:ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
log.perf.0031:ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
log.perf.0055:ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
log.perf.0062:ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'

Regards,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

[RFC][PATCHv3] autotools.bbclass, cmake.bbclass, perf: don't re-create ${B}

[Martin Jansa]

* otherwise there is race-condition between do_configure rm+mkdir and
  bitbake trying to use it as CWD for do_populate_lic task, which
  results in errors like this:

NOTE: recipe perf-1.0-r9: task do_configure: Started
NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 553, in exec_task
    return _exec_task(fn, task, d, quieterr)
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 493, in _exec_task
    exec_func(func, localdata)
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 227, in exec_func
    exec_func_python(func, d, runfile, cwd=adir)
  File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 252, in exec_func_python
    os.chdir(cwd)
OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded

or even longer exception:

NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
NOTE: recipe perf-1.0-r9: task do_configure: Started
ERROR: Error executing a python function in /OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'sstate_task_postfunc', lineno: 14, function: <module>
     0010:    sstate_package(shared_state, d)
     0011:    os.umask(omask)
     0012:
     0013:
 *** 0014:sstate_task_postfunc(d)
     0015:
File: 'sstate_task_postfunc', lineno: 4, function: sstate_task_postfunc
     0001:
     0002:def sstate_task_postfunc(d):
     0003:    shared_state = sstate_state_fromvars(d)
 *** 0004:    sstate_install(shared_state, d)
     0005:    for intercept in shared_state['interceptfuncs']:
     0006:        bb.build.exec_func(intercept, d)
     0007:    omask = os.umask(002)
     0008:    if omask != 002:
File: 'sstate.bbclass', lineno: 113, function: sstate_install
     0109:        if os.path.exists(state[1]):
     0110:            oe.path.copyhardlinktree(state[1], state[2])
     0111:
     0112:    for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
 *** 0113:        bb.build.exec_func(postinst, d)
     0114:
     0115:    for lock in locks:
     0116:        bb.utils.unlockfile(lock)
     0117:
File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 227, function: exec_func
     0223:                pass
     0224:
     0225:    with bb.utils.fileslocked(lockfiles):
     0226:        if ispython:
 *** 0227:            exec_func_python(func, d, runfile, cwd=adir)
    0228:        else:
     0229:            exec_func_shell(func, d, runfile, cwd=adir)
     0230:
     0231:_functionfmt = """
File: '/OE/build/oe-core/bitbake/lib/bb/build.py', lineno: 252, function: exec_func_python
     0248:        try:
     0249:            olddir = os.getcwd()
     0250:        except OSError:
     0251:            olddir = None
 *** 0252:        os.chdir(cwd)
     0253:
     0254:    bb.debug(2, "Executing python function %s" % func)
     0255:
     0256:    try:
Exception: OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'

ERROR: Function failed: sstate_task_postfunc
ERROR: Logfile of failure stored in: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/temp/log.do_populate_lic.16588
NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
NOTE: recipe perf-1.0-r9: task do_configure: Succeeded
NOTE: Tasks Summary: Attempted 840 tasks of which 835 didn't need to be rerun and 1 failed.

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 meta/classes/autotools.bbclass   | 3 +--
 meta/classes/cmake.bbclass       | 3 +--
 meta/recipes-kernel/perf/perf.bb | 3 +--
 3 files changed, 3 insertions(+), 6 deletions(-)

diff --git a/meta/classes/autotools.bbclass b/meta/classes/autotools.bbclass
index 819045a..67e1e3a 100644
--- a/meta/classes/autotools.bbclass
+++ b/meta/classes/autotools.bbclass
@@ -104,8 +104,7 @@ autotools_preconfigure() {
 		if [ "`cat ${CONFIGURESTAMPFILE}`" != "${BB_TASKHASH}" ]; then
 			if [ "${S}" != "${B}" ]; then
 				echo "Previously configured separate build directory detected, cleaning ${B}"
-				rm -rf ${B}
-				mkdir -p ${B}
+				find ${B} -mindepth 1 -maxdepth 1 | xargs rm -rf
 			else
 				# At least remove the .la files since automake won't automatically
 				# regenerate them even if CFLAGS/LDFLAGS are different
diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index ae3cc02..e31799a 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -91,8 +91,7 @@ cmake_do_configure() {
 	fi
 
 	if [ "${S}" != "${B}" ]; then
-		rm -rf ${B}
-		mkdir -p ${B}
+		find ${B} -mindepth 1 -maxdepth 1 | xargs rm -rf
 		cd ${B}
 	else
 		find ${B} -name CMakeFiles -or -name Makefile -or -name cmake_install.cmake -or -name CMakeCache.txt -delete
diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index 22bd3c8..a5a04af 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -119,8 +119,7 @@ do_install() {
 
 do_configure_prepend () {
     # Fix for rebuilding
-    rm -rf ${B}/
-    mkdir -p ${B}/
+    find ${B} -mindepth 1 -maxdepth 1 | xargs rm -rf
 
     # If building a multlib based perf, the incorrect library path will be
     # detected by perf, since it triggers via: ifeq ($(ARCH),x86_64). In a 32 bit
-- 
2.5.1

Re: [RFC][PATCHv2] cmake.bbclass, perf: don't re-create ${B}

[Richard Purdie]

On Mon, 2015-09-21 at 22:33 +0200, Martin Jansa wrote:
> * otherwise there is race-condition between do_configure rm+mkdir and
>   bitbake trying to use it as CWD for do_populate_lic task, which
>   results in errors like this:
> 
> NOTE: recipe perf-1.0-r9: task do_configure: Started
> NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
> ERROR: Build of do_populate_lic failed
> ERROR: Traceback (most recent call last):
>   File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 553, in exec_task
>     return _exec_task(fn, task, d, quieterr)
>   File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 493, in _exec_task
>     exec_func(func, localdata)
>   File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 227, in exec_func
>     exec_func_python(func, d, runfile, cwd=adir)
>   File "/OE/build/oe-core/bitbake/lib/bb/build.py", line 252, in exec_func_python
>     os.chdir(cwd)
> OSError: [Errno 2] No such file or directory: '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/perf/1.0-r9/perf-1.0'
> NOTE: recipe perf-1.0-r9: task do_populate_lic: Failed
> ERROR: Task 7 (/OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_populate_lic) failed with exit code '1'
> NOTE: recipe perf-1.0-r9: task do_configure: Succeeded
> 
> or even longer exception:
> 
> NOTE: recipe perf-1.0-r9: task do_populate_lic: Started
> NOTE: recipe perf-1.0-r9: task do_configure: Started
> ERROR: Error executing a python function in /OE/build/oe-core/openembedded-core/meta/recipes-kernel/perf/perf.bb:
> 
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'sstate_task_postfunc', lineno: 14, function: <module>
>      0010:    sstate_package(shared_state, d)
>      0011:    os.umask(omask)
>      0012:
>      0013:
>  *** 0014:sstate_task_postfunc(d)
>      0015:
> File: 'sstate_task_postfunc', lineno: 4, function: sstate_task_postfunc
>      0001:
>      0002:def sstate_task_postfunc(d):
>      0003:    shared_state = sstate_state_fromvars(d)
>  *** 0004:    sstate_install(shared_state, d)
>      0005:    for intercept in shared_state['interceptfuncs']:
>      0006:        bb.build.exec_func(intercept, d)
>      0007:    omask = os.umask(002)
>      0008:    if omask != 002:
> File: 'sstate.bbclass', lineno: 113, function: sstate_install
>      0109:        if os.path.exists(state[1]):
>      0110:            oe.path.copyhardlinktree(state[1], state[2])
>      0111:
>      0112:    for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
>  *** 0113:        bb.build.exec_func(postinst, d)
>      0114:
>      0115:    for lock in locks:
>      0116:        bb.utils.unlockfile(lock)
>      0117:

The longer stack trace is actually really helpful. I believe the code
here should inherit the working directory of the current task rather
than rely on the default of ${B} which bitbake assigns if there is
nothing else.

The trouble is that if the function does specify a dirs flag, it should
really use that instead of a supplied parameter and if we use the
parameter to exec_func(), it won't do that.

There aren't that many uses of SSTATEPOSTINSTFUNCS in our metadata so
the easiest way to fix this short term is probably to go through them
and add specific [dirs] options to the functions.

Cheers,

Richard

Re: [RFC][PATCHv2] cmake.bbclass, perf: don't re-create ${B}

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 1119 bytes --]

On 22 September 2015 at 09:10, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> The longer stack trace is actually really helpful. I believe the code
> here should inherit the working directory of the current task rather
> than rely on the default of ${B} which bitbake assigns if there is
> nothing else.
>
> The trouble is that if the function does specify a dirs flag, it should
> really use that instead of a supplied parameter and if we use the
> parameter to exec_func(), it won't do that.
>
> There aren't that many uses of SSTATEPOSTINSTFUNCS in our metadata so
> the easiest way to fix this short term is probably to go through them
> and add specific [dirs] options to the functions.
>

There's enough instances of SSTATEPOSTINSTFUNCS dotted around even just
oe-core, but they're all going to be using SSTATE_INSTDIR as the working
directory.  Whilst you have a valid point that the functions should respect
[dirs] if set, for the case of sstate post-install hooks I think we should
just force the right path so the recipe creators don't have to worry about
this.

Ross

[-- Attachment #2: Type: text/html, Size: 1574 bytes --]