From: "Aníbal Limón" <anibal.limon@linux.intel.com>
To: openembedded-core@lists.openembedded.org
Cc: joshua.g.lock@intel.com
Subject: [PATCHv3 00/30] oeqa core and oe-selftest threaded enablement
Date: Mon, 17 Jul 2017 16:45:08 -0500 [thread overview]
Message-ID: <cover.1500327589.git.anibal.limon@linux.intel.com> (raw)
This v3 makes changes on tinfoil busy status patch and adds an
environment variable to oe-seltest script to specify number of threads.
There are patches for bitbake [3] and meta-yocto-bsp [4] that was send
to their specific ML.
The first 6 patches previous enable support of threads in oeqa core
are independent and can be merged first.
The v2 address comments did by Patrick about add better explanation
on the commit messages and some typos.
Other interesting information to add is that with this series we will
able to execute oe-selftest in around half [1] of the original time [2],
(9406.782s vs 17303.014s).
[1] https://bugzilla.yoctoproject.org/attachment.cgi?id=3863
[2] https://bugzilla.yoctoproject.org/attachment.cgi?id=3864
[3] http://lists.openembedded.org/pipermail/bitbake-devel/2017-July/008794.html
[4] https://lists.yoctoproject.org/pipermail/yocto/2017-July/037140.html
This series is to enable oe-selftest threaded runs along some fixes,
* Implementation of main/end thread usage in oeqa threaded
* Adaptation of oe-selftest cases to be able to run in threaded env,
** Usage of own build directory by Test class.
** Mark the test modules that are enabled to run into a thread.
** Split some test modules because aren't support run into a thread
due to bitbake/tinfoil constraints.
The oe-selftest script now has an cmdline option (-t) to enable threaded
runs, by default is set to 1, this needs to be set manually because
depending on HW resources available can cause ran out of cpu/memory.
By default the oe-selftest cases runs on the main thread and uses
the main build directory for compatibility purposes.
The following changes since commit a4f2bf37b7eac888f37f11cd4d4606436909c507:
mpc8315e-rdb: add wic.bmap to IMAGE_FSTYPES (2017-07-17 15:14:27 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib alimon/oe_selftest_threaded
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=alimon/oe_selftest_threaded
Aníbal Limón (30):
oeqa/core/loader: Generate function _make_failed_test dynamically
selftest/cases/package: Call parent setUpClass method
oeqa/selftest/{context,case}: Handle KeyboardInterrupt/SIGINT and
SIGTERM
bb/server/process: Handle EINTR on idle_commands select
bb/tinfoil: run_command handle busy status in bitbake server
oe/copy_buildsystem: check_sstate_task_list also pop BBPATH from env
oeqa/core/threaded: Enable support to use the main thread
oeqa/core/threaded: Add support to run into a thread at end of
execution
oeqa/core/threaded: logSummary add skipped tests info
oeqa/core/tests: Update test_loader threaded to cover main thread
usage
oeqa/selftest/{case,context}: Add builddir by test class and context
oeqa/selftest/case: Add wrappers to utils.commands modules
oeqa/selftest/case: Creates meta-selftest layer per class
oeqa/selftest/case: tearDown extra commands print what actually fails
oeqa/selftest/case: Support bitbake memres mode in per build directory
oeqa/selftest/cases: Use testlayer_path instead of call
get_test_layer()
oeqa/selftest/cases: Use builddir from class instead of get from
environment
oeqa/selftest/cases: Use wrapper methods from OESelfTestCase class and
enable threaded runs
oeqa/selftest/cases: imagefeatures enable threaded runs
oeqa/selftest/cases: runqemu enable threaded runs
oeqa/selftest/cases: runtime enable threaded runs
oeqa/selftest/cases: eSDK enable threaded runs
oeqa/selftest/cases: devtool enable threaded runs
oeqa/selftest/cases: recipetool enable for threaded runs
oeqa/selftest/cases: Move devtool deploy test case to own module
selftest/cases/devtool{,end}: Move update/finish_modify tests to its
own module
seltest/cases/devtool: Build dbus on test_devtool_add_git_local
argparse_oe: Add int_positive type
oeqa/selftest/context: Enable support for threaded runs
oeqa/selftest/cases: systemd_boot enable threaded runs
bitbake/lib/bb/server/process.py | 7 +-
bitbake/lib/bb/tinfoil.py | 24 +-
.../lib/oeqa/selftest/cases/systemd_boot.py | 26 +-
meta/lib/oe/copy_buildsystem.py | 1 +
meta/lib/oeqa/core/loader.py | 25 +-
meta/lib/oeqa/core/tests/test_loader.py | 17 +-
meta/lib/oeqa/core/threaded.py | 169 +++-
meta/lib/oeqa/selftest/case.py | 251 ++++-
.../lib/oeqa/selftest/cases/_sstatetests_noauto.py | 23 +-
meta/lib/oeqa/selftest/cases/archiver.py | 21 +-
meta/lib/oeqa/selftest/cases/bblayers.py | 45 +-
meta/lib/oeqa/selftest/cases/bbtests.py | 103 +-
meta/lib/oeqa/selftest/cases/buildhistory.py | 9 +-
meta/lib/oeqa/selftest/cases/buildoptions.py | 58 +-
meta/lib/oeqa/selftest/cases/containerimage.py | 9 +-
meta/lib/oeqa/selftest/cases/devtool.py | 1014 +++++---------------
meta/lib/oeqa/selftest/cases/devtool_deploy.py | 93 ++
meta/lib/oeqa/selftest/cases/devtool_end.py | 506 ++++++++++
meta/lib/oeqa/selftest/cases/distrodata.py | 7 +-
meta/lib/oeqa/selftest/cases/eSDK.py | 64 +-
meta/lib/oeqa/selftest/cases/image_typedep.py | 8 +-
meta/lib/oeqa/selftest/cases/imagefeatures.py | 78 +-
meta/lib/oeqa/selftest/cases/imagefeatures_boot.py | 63 ++
meta/lib/oeqa/selftest/cases/layerappend.py | 22 +-
meta/lib/oeqa/selftest/cases/liboe.py | 13 +-
meta/lib/oeqa/selftest/cases/lic_checksum.py | 8 +-
meta/lib/oeqa/selftest/cases/manifest.py | 13 +-
meta/lib/oeqa/selftest/cases/oelib/buildhistory.py | 7 +-
meta/lib/oeqa/selftest/cases/oescripts.py | 5 +-
meta/lib/oeqa/selftest/cases/package.py | 12 +-
meta/lib/oeqa/selftest/cases/pkgdata.py | 73 +-
meta/lib/oeqa/selftest/cases/prservice.py | 19 +-
meta/lib/oeqa/selftest/cases/recipetool.py | 105 +-
meta/lib/oeqa/selftest/cases/runqemu.py | 7 +-
meta/lib/oeqa/selftest/cases/runtime_test.py | 225 +----
.../lib/oeqa/selftest/cases/runtime_test_export.py | 104 ++
.../oeqa/selftest/cases/runtime_test_postinsts.py | 114 +++
meta/lib/oeqa/selftest/cases/selftest.py | 2 +
meta/lib/oeqa/selftest/cases/signing.py | 42 +-
meta/lib/oeqa/selftest/cases/sstate.py | 5 +-
meta/lib/oeqa/selftest/cases/sstatetests.py | 51 +-
meta/lib/oeqa/selftest/context.py | 140 ++-
scripts/lib/argparse_oe.py | 7 +
43 files changed, 2030 insertions(+), 1565 deletions(-)
create mode 100644 meta/lib/oeqa/selftest/cases/devtool_deploy.py
create mode 100644 meta/lib/oeqa/selftest/cases/devtool_end.py
create mode 100644 meta/lib/oeqa/selftest/cases/imagefeatures_boot.py
create mode 100644 meta/lib/oeqa/selftest/cases/runtime_test_export.py
create mode 100644 meta/lib/oeqa/selftest/cases/runtime_test_postinsts.py
--
2.11.0
next reply other threads:[~2017-07-17 21:45 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-17 21:45 Aníbal Limón [this message]
2017-07-17 21:45 ` [PATCHv3 01/30] oeqa/core/loader: Generate function _make_failed_test dynamically Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 02/30] selftest/cases/package: Call parent setUpClass method Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 03/30] oeqa/selftest/{context, case}: Handle KeyboardInterrupt/SIGINT and SIGTERM Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 04/30] bb/server/process: Handle EINTR on idle_commands select Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 05/30] bb/tinfoil: run_command handle busy status in bitbake server Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 06/30] oe/copy_buildsystem: check_sstate_task_list also pop BBPATH from env Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 07/30] oeqa/core/threaded: Enable support to use the main thread Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 08/30] oeqa/core/threaded: Add support to run into a thread at end of execution Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 09/30] oeqa/core/threaded: logSummary add skipped tests info Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 10/30] oeqa/core/tests: Update test_loader threaded to cover main thread usage Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 11/30] oeqa/selftest/{case, context}: Add builddir by test class and context Aníbal Limón
2017-07-18 15:16 ` Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 12/30] oeqa/selftest/case: Add wrappers to utils.commands modules Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 13/30] oeqa/selftest/case: Creates meta-selftest layer per class Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 14/30] oeqa/selftest/case: tearDown extra commands print what actually fails Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 15/30] oeqa/selftest/case: Support bitbake memres mode in per build directory Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 16/30] oeqa/selftest/cases: Use testlayer_path instead of call get_test_layer() Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 17/30] oeqa/selftest/cases: Use builddir from class instead of get from environment Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 18/30] oeqa/selftest/cases: Use wrapper methods from OESelfTestCase class and enable threaded runs Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 19/30] oeqa/selftest/cases: imagefeatures " Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 20/30] oeqa/selftest/cases: runqemu " Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 21/30] oeqa/selftest/cases: runtime " Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 22/30] oeqa/selftest/cases: eSDK " Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 23/30] oeqa/selftest/cases: devtool " Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 24/30] oeqa/selftest/cases: recipetool enable for " Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 25/30] oeqa/selftest/cases: Move devtool deploy test case to own module Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 26/30] selftest/cases/devtool{, end}: Move update/finish_modify tests to its " Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 27/30] seltest/cases/devtool: Build dbus on test_devtool_add_git_local Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 28/30] argparse_oe: Add int_positive type Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 29/30] oeqa/selftest/context: Enable support for threaded runs Aníbal Limón
2017-07-17 21:45 ` [PATCHv3 30/30] oeqa/selftest/cases: systemd_boot enable " Aníbal Limón
2017-07-17 22:01 ` ✗ patchtest: failure for oeqa core and oe-selftest threaded enablement (rev3) Patchwork
2017-07-21 8:11 ` [PATCHv3 00/30] oeqa core and oe-selftest threaded enablement Richard Purdie
2017-07-21 16:15 ` Aníbal Limón
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1500327589.git.anibal.limon@linux.intel.com \
--to=anibal.limon@linux.intel.com \
--cc=joshua.g.lock@intel.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox