* [RFC][PATCH] sstate: run recipe-provided hooks outside of ${B}
@ 2015-09-22 14:11 Ross Burton
2015-09-22 17:30 ` Martin Jansa
0 siblings, 1 reply; 3+ messages in thread
From: Ross Burton @ 2015-09-22 14:11 UTC (permalink / raw)
To: openembedded-core
To avoid races between the sstate tasks/hooks using ${B} as the cwd, and other
tasks such as cmake_do_configure which deletes and re-creates ${B}, ensure that
all sstate hooks are run in the right directory, and run the prefunc/postfunc in WORKDIR.
Signed-off-by: Ross Burton <ross.burton@intel.com>
---
meta/classes/sstate.bbclass | 26 ++++++++++----------------
1 file changed, 10 insertions(+), 16 deletions(-)
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index b9ad6da..695929c 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -61,16 +61,6 @@ SSTATE_SIG_PASSPHRASE ?= ""
# Whether to verify the GnUPG signatures when extracting sstate archives
SSTATE_VERIFY_SIG ?= "0"
-# Specify dirs in which the shell function is executed and don't use ${B}
-# as default dirs to avoid possible race about ${B} with other task.
-sstate_create_package[dirs] = "${SSTATE_BUILDDIR}"
-sstate_unpack_package[dirs] = "${SSTATE_INSTDIR}"
-
-# Do not run sstate_hardcode_path() in ${B}:
-# the ${B} maybe removed by cmake_do_configure() while
-# sstate_hardcode_path() running.
-sstate_hardcode_path[dirs] = "${SSTATE_BUILDDIR}"
-
python () {
if bb.data.inherits_class('native', d):
d.setVar('SSTATE_PKGARCH', d.getVar('BUILD_ARCH', False))
@@ -267,7 +257,7 @@ def sstate_install(ss, d):
oe.path.copyhardlinktree(state[1], state[2])
for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
- bb.build.exec_func(postinst, d)
+ bb.build.exec_func(postinst, d, (os.getcwd(),))
for lock in locks:
bb.utils.unlockfile(lock)
@@ -307,7 +297,8 @@ def sstate_installpkg(ss, d):
bb.warn("Cannot verify signature on sstate package %s" % sstatepkg)
for f in (d.getVar('SSTATEPREINSTFUNCS', True) or '').split() + ['sstate_unpack_package'] + (d.getVar('SSTATEPOSTUNPACKFUNCS', True) or '').split():
- bb.build.exec_func(f, d)
+ # All hooks should run in the SSTATE_INSTDIR
+ bb.build.exec_func(f, d, (sstateinst,))
for state in ss['dirs']:
prepdir(state[1])
@@ -579,8 +570,9 @@ def sstate_package(ss, d):
for f in (d.getVar('SSTATECREATEFUNCS', True) or '').split() + ['sstate_create_package'] + \
(d.getVar('SSTATEPOSTCREATEFUNCS', True) or '').split():
- bb.build.exec_func(f, d)
-
+ # All hooks should run in SSTATE_BUILDDIR.
+ bb.build.exec_func(f, d, (sstatebuild,))
+
bb.siggen.dump_this_task(sstatepkg + ".siginfo", d)
return
@@ -642,19 +634,21 @@ python sstate_task_prefunc () {
shared_state = sstate_state_fromvars(d)
sstate_clean(shared_state, d)
}
+sstate_task_prefunc[dirs] = "${WORKDIR}"
python sstate_task_postfunc () {
shared_state = sstate_state_fromvars(d)
sstate_install(shared_state, d)
for intercept in shared_state['interceptfuncs']:
- bb.build.exec_func(intercept, d)
+ bb.build.exec_func(intercept, d, (os.getcwd(),))
omask = os.umask(002)
if omask != 002:
bb.note("Using umask 002 (not %0o) for sstate packaging" % omask)
sstate_package(shared_state, d)
os.umask(omask)
}
-
+sstate_task_postfunc[dirs] = "${WORKDIR}"
+
#
# Shell function to generate a sstate package from a directory
--
2.1.4
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [RFC][PATCH] sstate: run recipe-provided hooks outside of ${B}
2015-09-22 14:11 [RFC][PATCH] sstate: run recipe-provided hooks outside of ${B} Ross Burton
@ 2015-09-22 17:30 ` Martin Jansa
2015-09-22 20:52 ` Burton, Ross
0 siblings, 1 reply; 3+ messages in thread
From: Martin Jansa @ 2015-09-22 17:30 UTC (permalink / raw)
To: Ross Burton; +Cc: openembedded-core
[-- Attachment #1: Type: text/plain, Size: 4155 bytes --]
On Tue, Sep 22, 2015 at 03:11:53PM +0100, Ross Burton wrote:
> To avoid races between the sstate tasks/hooks using ${B} as the cwd, and other
> tasks such as cmake_do_configure which deletes and re-creates ${B}, ensure that
> all sstate hooks are run in the right directory, and run the prefunc/postfunc in WORKDIR.
No error in first 126 iterations of:
for i in `seq -w 1 1000`; do bitbake -c cleansstate perf; bitbake -c build -f perf 2>&1 | tee log.perf.$i; done
so it's good improvement, thanks!
Please get this merged in fido and dizzy as well.
> Signed-off-by: Ross Burton <ross.burton@intel.com>
> ---
> meta/classes/sstate.bbclass | 26 ++++++++++----------------
> 1 file changed, 10 insertions(+), 16 deletions(-)
>
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index b9ad6da..695929c 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -61,16 +61,6 @@ SSTATE_SIG_PASSPHRASE ?= ""
> # Whether to verify the GnUPG signatures when extracting sstate archives
> SSTATE_VERIFY_SIG ?= "0"
>
> -# Specify dirs in which the shell function is executed and don't use ${B}
> -# as default dirs to avoid possible race about ${B} with other task.
> -sstate_create_package[dirs] = "${SSTATE_BUILDDIR}"
> -sstate_unpack_package[dirs] = "${SSTATE_INSTDIR}"
> -
> -# Do not run sstate_hardcode_path() in ${B}:
> -# the ${B} maybe removed by cmake_do_configure() while
> -# sstate_hardcode_path() running.
> -sstate_hardcode_path[dirs] = "${SSTATE_BUILDDIR}"
> -
> python () {
> if bb.data.inherits_class('native', d):
> d.setVar('SSTATE_PKGARCH', d.getVar('BUILD_ARCH', False))
> @@ -267,7 +257,7 @@ def sstate_install(ss, d):
> oe.path.copyhardlinktree(state[1], state[2])
>
> for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
> - bb.build.exec_func(postinst, d)
> + bb.build.exec_func(postinst, d, (os.getcwd(),))
>
> for lock in locks:
> bb.utils.unlockfile(lock)
> @@ -307,7 +297,8 @@ def sstate_installpkg(ss, d):
> bb.warn("Cannot verify signature on sstate package %s" % sstatepkg)
>
> for f in (d.getVar('SSTATEPREINSTFUNCS', True) or '').split() + ['sstate_unpack_package'] + (d.getVar('SSTATEPOSTUNPACKFUNCS', True) or '').split():
> - bb.build.exec_func(f, d)
> + # All hooks should run in the SSTATE_INSTDIR
> + bb.build.exec_func(f, d, (sstateinst,))
>
> for state in ss['dirs']:
> prepdir(state[1])
> @@ -579,8 +570,9 @@ def sstate_package(ss, d):
>
> for f in (d.getVar('SSTATECREATEFUNCS', True) or '').split() + ['sstate_create_package'] + \
> (d.getVar('SSTATEPOSTCREATEFUNCS', True) or '').split():
> - bb.build.exec_func(f, d)
> -
> + # All hooks should run in SSTATE_BUILDDIR.
> + bb.build.exec_func(f, d, (sstatebuild,))
> +
> bb.siggen.dump_this_task(sstatepkg + ".siginfo", d)
>
> return
> @@ -642,19 +634,21 @@ python sstate_task_prefunc () {
> shared_state = sstate_state_fromvars(d)
> sstate_clean(shared_state, d)
> }
> +sstate_task_prefunc[dirs] = "${WORKDIR}"
>
> python sstate_task_postfunc () {
> shared_state = sstate_state_fromvars(d)
> sstate_install(shared_state, d)
> for intercept in shared_state['interceptfuncs']:
> - bb.build.exec_func(intercept, d)
> + bb.build.exec_func(intercept, d, (os.getcwd(),))
> omask = os.umask(002)
> if omask != 002:
> bb.note("Using umask 002 (not %0o) for sstate packaging" % omask)
> sstate_package(shared_state, d)
> os.umask(omask)
> }
> -
> +sstate_task_postfunc[dirs] = "${WORKDIR}"
> +
>
> #
> # Shell function to generate a sstate package from a directory
> --
> 2.1.4
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [RFC][PATCH] sstate: run recipe-provided hooks outside of ${B}
2015-09-22 17:30 ` Martin Jansa
@ 2015-09-22 20:52 ` Burton, Ross
0 siblings, 0 replies; 3+ messages in thread
From: Burton, Ross @ 2015-09-22 20:52 UTC (permalink / raw)
To: Martin Jansa; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 428 bytes --]
On 22 September 2015 at 18:30, Martin Jansa <martin.jansa@gmail.com> wrote:
> No error in first 126 iterations of:
> for i in `seq -w 1 1000`; do bitbake -c cleansstate perf; bitbake -c build
> -f perf 2>&1 | tee log.perf.$i; done
>
> so it's good improvement, thanks!
>
> Please get this merged in fido and dizzy as well.
>
Just found a problem in it so I'll be sending a v2, but thanks for the
testing.
Ross
[-- Attachment #2: Type: text/html, Size: 856 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2015-09-22 20:52 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-09-22 14:11 [RFC][PATCH] sstate: run recipe-provided hooks outside of ${B} Ross Burton
2015-09-22 17:30 ` Martin Jansa
2015-09-22 20:52 ` Burton, Ross
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.