Openembedded Core Discussions
 help / color / mirror / Atom feed
From: Markus Lehtonen <markus.lehtonen@linux.intel.com>
To: openembedded-core@lists.openembedded.org
Subject: [PATCH 10/20] python-pgo-image: enable sstate for do_profile
Date: Mon, 20 Feb 2017 12:35:41 +0200	[thread overview]
Message-ID: <20170220103551.20022-10-markus.lehtonen@linux.intel.com> (raw)
In-Reply-To: <20170220103551.20022-1-markus.lehtonen@linux.intel.com>

There are caveats however, basically because of the circular dependency
between python-pgo-image and python (python is a build dep of
python-pgo-image, and at the same time, depends on the profile data
generated by python-pgo-image). In practice the sstate data is only used
when no profile data is yet present. Existing profile data (i.e.
non-empty PYTHON_PROFILE_DIR) changes signature hash of python which, in
turn, changes signature of python-pgo-image and causes a sstate
signature mismatch and rebuild of python-pgo-image. Thus, in order to
utilize sstate data the user should run "bitbake python-pgo-image -c clean"
(or otherwise wipe pre-existing profile data) before running
"bitbake python-pgo-image -c profile".

[YOCTO #9338]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
 meta/recipes-devtools/images/python-pgo-image.bb | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/images/python-pgo-image.bb b/meta/recipes-devtools/images/python-pgo-image.bb
index 82cb001..bab95f8 100644
--- a/meta/recipes-devtools/images/python-pgo-image.bb
+++ b/meta/recipes-devtools/images/python-pgo-image.bb
@@ -9,6 +9,7 @@ LICENSE = "MIT"
 inherit core-image
 
 PYTHON_PROFILE_DIR ?= "${TMPDIR}/work-shared/${MACHINE}/python/pgo-data"
+PROFILE_DATA_WORKDIR = "${WORKDIR}/profile-data"
 PYTHON_PROFILE_TASK_DEFAULT = "-m test.regrtest --pgo -w -x test_asyncore test_gdb test_multiprocessing test_subprocess"
 ## Exclude tests that are segfaulting on qemux86 target
 #PYTHON_PROFILE_TASK_DEFAULT += "test_bytes test_str test_string test_tuple test_unicode test_userstring test_xmlrpc"
@@ -47,7 +48,7 @@ python do_profile() {
             bb.fatal("Failed to archive profile data on target: %s" % output)
 
         # Retrieve and unpack profile data
-        profile_dir = d.getVar("PYTHON_PROFILE_DIR", True)
+        profile_dir = d.getVar("PROFILE_DATA_WORKDIR", True)
         target.copy_from('/home/root/pgo-data.tgz', profile_dir)
 
         profile_tarball = os.path.join(profile_dir, 'pgo-data.tgz')
@@ -61,4 +62,14 @@ python do_profile() {
 
 addtask profile after do_build
 do_profile[depends] += "qemu-native:do_populate_sysroot qemu-helper-native:do_populate_sysroot"
-do_profile[cleandirs] = "${PYTHON_PROFILE_DIR}"
+do_profile[cleandirs] = "${PROFILE_DATA_WORKDIR}"
+
+
+python do_profile_setscene () {
+    sstate_setscene(d)
+}
+
+SSTATETASKS += "do_profile"
+do_profile[sstate-inputdirs] = "${PROFILE_DATA_WORKDIR}"
+do_profile[sstate-outputdirs] = "${PYTHON_PROFILE_DIR}"
+addtask do_profile_setscene
-- 
2.10.2



  parent reply	other threads:[~2017-02-20 10:36 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-02-20 10:35 [PATCH 01/20] python-native: support profile optimized build Markus Lehtonen
2017-02-20 10:35 ` [PATCH 02/20] python: add python-profile-opt recipe Markus Lehtonen
2017-02-20 10:35 ` [PATCH 03/20] python: remove path hack from setup.py Markus Lehtonen
2017-02-20 10:35 ` [PATCH 04/20] python-profile-opt: rename libpython Markus Lehtonen
2017-02-20 10:35 ` [PATCH 05/20] devtools/images: add python-pgo-image Markus Lehtonen
2017-02-20 10:35 ` [PATCH 06/20] python: make profile-optimized build possible Markus Lehtonen
2017-02-20 10:35 ` [PATCH 07/20] python-pgo-image: exclude tests from the default profile target Markus Lehtonen
2017-02-20 10:35 ` [PATCH 08/20] python: add python-tools subpackage Markus Lehtonen
2017-02-20 10:35 ` [PATCH 09/20] python-pgo-image: switch python default profile task to pybench Markus Lehtonen
2017-02-20 10:35 ` Markus Lehtonen [this message]
2017-02-20 10:35 ` [PATCH 11/20] python3-native: support profile optimized build Markus Lehtonen
2017-02-20 10:35 ` [PATCH 12/20] python3: fix depends of python-tests Markus Lehtonen
2017-02-20 10:35 ` [PATCH 13/20] python3: add python-profile-opt3 recipe Markus Lehtonen
2017-02-20 10:35 ` [PATCH 14/20] python-pgo-image: profiling for python3 Markus Lehtonen
2017-02-20 10:35 ` [PATCH 15/20] python3: remove two setup.py cross-compile hacks Markus Lehtonen
2017-02-20 10:35 ` [PATCH 16/20] python3: support profile optimized build Markus Lehtonen
2017-02-20 10:35 ` [PATCH 17/20] python3: fix profile-optimized build of modules Markus Lehtonen
2017-02-20 10:35 ` [PATCH 18/20] python-pgo-image: exclude tests from the python3 profile target Markus Lehtonen
2017-02-20 10:35 ` [PATCH 19/20] python3: add python3-tools subpackage Markus Lehtonen
2017-02-20 10:35 ` [PATCH 20/20] python-pgo-image: change python3 profile target to pybench Markus Lehtonen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170220103551.20022-10-markus.lehtonen@linux.intel.com \
    --to=markus.lehtonen@linux.intel.com \
    --cc=openembedded-core@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox