From: Ross Burton <ross.burton@intel.com>
To: openembedded-core@lists.openembedded.org
Subject: [PATCH 5/5] python3: use regrtest instead of PyBench for profile-guided-optimisation
Date: Thu, 6 Sep 2018 12:15:21 +0100 [thread overview]
Message-ID: <20180906111521.20997-5-ross.burton@intel.com> (raw)
In-Reply-To: <20180906111521.20997-1-ross.burton@intel.com>
PyBench takes a long time to run, also upstream have removed it from Python and
instead use test.regrtest —pgo to profile the interpreter.
The results are good: not only does Python compile faster (~300s vs ~600s on my
machine) but Phoronix’s PyBench test runs in 2130ms compared to 2229ms when
using PyBench to train (and 2345ms with PGO disabled).
Signed-off-by: Ross Burton <ross.burton@intel.com>
---
meta/recipes-devtools/python/python3_3.5.5.bb | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/meta/recipes-devtools/python/python3_3.5.5.bb b/meta/recipes-devtools/python/python3_3.5.5.bb
index b89e3b04c86..386b46a763d 100644
--- a/meta/recipes-devtools/python/python3_3.5.5.bb
+++ b/meta/recipes-devtools/python/python3_3.5.5.bb
@@ -76,7 +76,7 @@ SDK_CC_ARCH += "-DNDEBUG -fno-inline"
EXTRA_OEMAKE += "CROSS_COMPILE=yes"
EXTRA_OECONF += "CROSSPYTHONPATH=${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dynload/ --without-ensurepip"
-PYTHON3_PROFILE_TASK ?= "${S}/Tools/pybench/pybench.py -n 1"
+PYTHON3_PROFILE_TASK ?= "./python -m test.regrtest --pgo test_grammar test_opcodes test_dict test_builtin test_exceptions test_types test_support || true"
export CROSS_COMPILE = "${TARGET_PREFIX}"
export _PYTHON_PROJECT_BASE = "${B}"
@@ -144,13 +144,15 @@ do_compile() {
if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then
run_make profile-opt
qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}"
- cat > pgo-image-qemuwrapper << EOF
+ cat >pgo-wrapper <<EOF
#!/bin/sh
-set -x
+cd ${B}
$qemu_binary "\$@"
EOF
- chmod +x pgo-image-qemuwrapper
- ./pgo-image-qemuwrapper ${B}/python ${PYTHON3_PROFILE_TASK} || true
+ chmod +x pgo-wrapper
+ bbnote Gathering profiling data
+ ./pgo-wrapper ${PYTHON3_PROFILE_TASK}
+ bbnote Profiling data gathered, rebuilding
run_make clean_and_use_profile
else
run_make libpython3.so
--
2.11.0
prev parent reply other threads:[~2018-09-06 11:15 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-09-06 11:15 [PATCH 1/5] python3: generalise make commands Ross Burton
2018-09-06 11:15 ` [PATCH 2/5] python3: add toggle for profile-guided-optimisation Ross Burton
2018-09-06 11:15 ` [PATCH 3/5] python3: trim PGO patch Ross Burton
2018-09-06 11:15 ` [PATCH 4/5] python3: don't use runtime checks to identify float endianism Ross Burton
2018-09-06 11:15 ` Ross Burton [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180906111521.20997-5-ross.burton@intel.com \
--to=ross.burton@intel.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox