* [PATCH v2 0/3] devtool fixes
@ 2015-09-23 10:05 Paul Eggleton
2015-09-23 10:05 ` [PATCH v2 1/3] devtool: second fix for running from a different directory Paul Eggleton
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Paul Eggleton @ 2015-09-23 10:05 UTC (permalink / raw)
To: openembedded-core
A couple of patches Markus previously sent rebased on top of master
(also applicable on top of poky master-next) to fix some minor issues in
devtool, plus one minor optimisation from me.
The following changes since commit 2ad7308ee7166641eff99f3b9fe6794de143f6bc:
oeqa/utils/qemurunner.py: Remove duplicate message on LoggingThread start (2015-09-22 18:13:02 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/devtool-fixes6
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/devtool-fixes6
Markus Lehtonen (2):
devtool: second fix for running from a different directory
devtool: upgrade: use shutil.move instead of os.rename
Paul Eggleton (1):
devtool: runqemu: avoid recipe parse
scripts/devtool | 5 +----
scripts/lib/devtool/__init__.py | 6 +++++-
scripts/lib/devtool/build-image.py | 2 +-
scripts/lib/devtool/deploy.py | 2 +-
scripts/lib/devtool/package.py | 2 +-
scripts/lib/devtool/runqemu.py | 2 +-
scripts/lib/devtool/search.py | 2 +-
scripts/lib/devtool/standard.py | 10 +++++-----
scripts/lib/devtool/upgrade.py | 4 ++--
9 files changed, 18 insertions(+), 17 deletions(-)
--
2.1.0
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v2 1/3] devtool: second fix for running from a different directory 2015-09-23 10:05 [PATCH v2 0/3] devtool fixes Paul Eggleton @ 2015-09-23 10:05 ` Paul Eggleton 2015-09-23 15:09 ` Christopher Larson 2015-09-23 10:05 ` [PATCH v2 2/3] devtool: runqemu: avoid recipe parse Paul Eggleton 2015-09-23 10:05 ` [PATCH v2 3/3] devtool: upgrade: use shutil.move instead of os.rename Paul Eggleton 2 siblings, 1 reply; 7+ messages in thread From: Paul Eggleton @ 2015-09-23 10:05 UTC (permalink / raw) To: openembedded-core From: Markus Lehtonen <markus.lehtonen@linux.intel.com> Do not change change current working directory permanently, but, only for the duration of tinfoil initialization instead. The previous fix caused very unintuitive behavior where using relative paths were solved with respect to the builddir instead of the current working directory. E.g. calling "devtool extract zlib ./zlib" would always create create srctree in ${TOPDIR}/zlib, independent of the users cwd. (From OE-Core rev: 4c7f159b0e17a0475a4a4e9dc4dd012e3d2e6a1f) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> --- scripts/devtool | 5 +---- scripts/lib/devtool/__init__.py | 6 +++++- scripts/lib/devtool/build-image.py | 2 +- scripts/lib/devtool/deploy.py | 2 +- scripts/lib/devtool/package.py | 2 +- scripts/lib/devtool/runqemu.py | 2 +- scripts/lib/devtool/search.py | 2 +- scripts/lib/devtool/standard.py | 10 +++++----- scripts/lib/devtool/upgrade.py | 2 +- 9 files changed, 17 insertions(+), 16 deletions(-) diff --git a/scripts/devtool b/scripts/devtool index 87df951..e4d9db3 100755 --- a/scripts/devtool +++ b/scripts/devtool @@ -221,9 +221,6 @@ def main(): if not config.read(): return -1 - # We need to be in this directory or we won't be able to initialise tinfoil - os.chdir(basepath) - bitbake_subdir = config.get('General', 'bitbake_subdir', '') if bitbake_subdir: # Normally set for use within the SDK @@ -244,7 +241,7 @@ def main(): scriptutils.logger_setup_color(logger, global_args.color) if global_args.bbpath is None: - tinfoil = setup_tinfoil(config_only=True) + tinfoil = setup_tinfoil(config_only=True, basepath=basepath) global_args.bbpath = tinfoil.config_data.getVar('BBPATH', True) else: tinfoil = None diff --git a/scripts/lib/devtool/__init__.py b/scripts/lib/devtool/__init__.py index f815ef2..7b1ab11 100644 --- a/scripts/lib/devtool/__init__.py +++ b/scripts/lib/devtool/__init__.py @@ -96,9 +96,12 @@ def exec_fakeroot(d, cmd, **kwargs): newenv[splitval[0]] = splitval[1] return subprocess.call("%s %s" % (fakerootcmd, cmd), env=newenv, **kwargs) -def setup_tinfoil(config_only=False): +def setup_tinfoil(config_only=False, basepath=None): """Initialize tinfoil api from bitbake""" import scriptpath + orig_cwd = os.path.abspath(os.curdir) + if basepath: + os.chdir(basepath) bitbakepath = scriptpath.add_bitbake_lib_path() if not bitbakepath: logger.error("Unable to find bitbake by searching parent directory of this script or PATH") @@ -108,6 +111,7 @@ def setup_tinfoil(config_only=False): tinfoil = bb.tinfoil.Tinfoil() tinfoil.prepare(config_only) tinfoil.logger.setLevel(logger.getEffectiveLevel()) + os.chdir(orig_cwd) return tinfoil def get_recipe_file(cooker, pn): diff --git a/scripts/lib/devtool/build-image.py b/scripts/lib/devtool/build-image.py index 05c1d75..e53239d 100644 --- a/scripts/lib/devtool/build-image.py +++ b/scripts/lib/devtool/build-image.py @@ -59,7 +59,7 @@ def build_image(args, config, basepath, workspace): if os.path.isfile(appendfile): os.unlink(appendfile) - tinfoil = setup_tinfoil() + tinfoil = setup_tinfoil(basepath=basepath) rd = parse_recipe(config, tinfoil, image, True) if not rd: # Error already shown diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py index 41b666f..c90c6b1 100644 --- a/scripts/lib/devtool/deploy.py +++ b/scripts/lib/devtool/deploy.py @@ -40,7 +40,7 @@ def deploy(args, config, basepath, workspace): deploy_dir = os.path.join(basepath, 'target_deploy', args.target) deploy_file = os.path.join(deploy_dir, args.recipename + '.list') - tinfoil = setup_tinfoil() + tinfoil = setup_tinfoil(basepath=basepath) try: rd = oe.recipeutils.parse_recipe_simple(tinfoil.cooker, args.recipename, tinfoil.config_data) except Exception as e: diff --git a/scripts/lib/devtool/package.py b/scripts/lib/devtool/package.py index 28ecfed..b8d8423 100644 --- a/scripts/lib/devtool/package.py +++ b/scripts/lib/devtool/package.py @@ -34,7 +34,7 @@ def package(args, config, basepath, workspace): image_pkgtype = config.get('Package', 'image_pkgtype', '') if not image_pkgtype: - tinfoil = setup_tinfoil() + tinfoil = setup_tinfoil(basepath=basepath) try: tinfoil.prepare(config_only=True) image_pkgtype = tinfoil.config_data.getVar('IMAGE_PKGTYPE', True) diff --git a/scripts/lib/devtool/runqemu.py b/scripts/lib/devtool/runqemu.py index e7f26ab..4d08d8c 100644 --- a/scripts/lib/devtool/runqemu.py +++ b/scripts/lib/devtool/runqemu.py @@ -29,7 +29,7 @@ logger = logging.getLogger('devtool') def runqemu(args, config, basepath, workspace): """Entry point for the devtool 'runqemu' subcommand""" - tinfoil = setup_tinfoil() + tinfoil = setup_tinfoil(basepath=basepath) machine = tinfoil.config_data.getVar('MACHINE', True) bindir_native = tinfoil.config_data.getVar('STAGING_BINDIR_NATIVE', True) tinfoil.shutdown() diff --git a/scripts/lib/devtool/search.py b/scripts/lib/devtool/search.py index e6ae922..c2f420c 100644 --- a/scripts/lib/devtool/search.py +++ b/scripts/lib/devtool/search.py @@ -29,7 +29,7 @@ logger = logging.getLogger('devtool') def search(args, config, basepath, workspace): """Entry point for the devtool 'search' subcommand""" - tinfoil = setup_tinfoil(config_only=True) + tinfoil = setup_tinfoil(config_only=True, basepath=basepath) pkgdata_dir = tinfoil.config_data.getVar('PKGDATA_DIR', True) tinfoil.shutdown() diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py index 96b271c..1dcf7cd 100644 --- a/scripts/lib/devtool/standard.py +++ b/scripts/lib/devtool/standard.py @@ -111,7 +111,7 @@ def add(args, config, basepath, workspace): (stdout, _) = bb.process.run('git rev-parse HEAD', cwd=srctree) initial_rev = stdout.rstrip() - tinfoil = setup_tinfoil(config_only=True) + tinfoil = setup_tinfoil(config_only=True, basepath=basepath) rd = oe.recipeutils.parse_recipe(recipefile, None, tinfoil.config_data) if not rd: return 1 @@ -231,7 +231,7 @@ class BbTaskExecutor(object): def _prep_extract_operation(config, basepath, recipename): """HACK: Ugly workaround for making sure that requirements are met when trying to extract a package. Returns the tinfoil instance to be used.""" - tinfoil = setup_tinfoil() + tinfoil = setup_tinfoil(basepath=basepath) rd = parse_recipe(config, tinfoil, recipename, True) if bb.data.inherits_class('kernel-yocto', rd): @@ -239,7 +239,7 @@ def _prep_extract_operation(config, basepath, recipename): try: stdout, _ = exec_build_env_command(config.init_path, basepath, 'bitbake kern-tools-native') - tinfoil = setup_tinfoil() + tinfoil = setup_tinfoil(basepath=basepath) except bb.process.ExecutionError as err: raise DevtoolError("Failed to build kern-tools-native:\n%s" % err.stdout) @@ -443,7 +443,7 @@ def modify(args, config, basepath, workspace): if args.extract: tinfoil = _prep_extract_operation(config, basepath, args.recipename) else: - tinfoil = setup_tinfoil() + tinfoil = setup_tinfoil(basepath=basepath) rd = parse_recipe(config, tinfoil, args.recipename, True) if not rd: @@ -797,7 +797,7 @@ def update_recipe(args, config, basepath, workspace): raise DevtoolError('conf/layer.conf not found in bbappend ' 'destination layer "%s"' % args.append) - tinfoil = setup_tinfoil() + tinfoil = setup_tinfoil(basepath=basepath) rd = parse_recipe(config, tinfoil, args.recipename, True) if not rd: diff --git a/scripts/lib/devtool/upgrade.py b/scripts/lib/devtool/upgrade.py index 6c1dfee..aa53c8e 100644 --- a/scripts/lib/devtool/upgrade.py +++ b/scripts/lib/devtool/upgrade.py @@ -294,7 +294,7 @@ def upgrade(args, config, basepath, workspace): if reason: raise DevtoolError(reason) - tinfoil = setup_tinfoil() + tinfoil = setup_tinfoil(basepath=basepath) rd = parse_recipe(config, tinfoil, args.recipename, True) if not rd: -- 2.1.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v2 1/3] devtool: second fix for running from a different directory 2015-09-23 10:05 ` [PATCH v2 1/3] devtool: second fix for running from a different directory Paul Eggleton @ 2015-09-23 15:09 ` Christopher Larson 2015-09-23 16:04 ` Paul Eggleton 0 siblings, 1 reply; 7+ messages in thread From: Christopher Larson @ 2015-09-23 15:09 UTC (permalink / raw) To: Paul Eggleton; +Cc: Patches and discussions about the oe-core layer [-- Attachment #1: Type: text/plain, Size: 1129 bytes --] On Wed, Sep 23, 2015 at 3:05 AM, Paul Eggleton < paul.eggleton@linux.intel.com> wrote: > -def setup_tinfoil(config_only=False): > +def setup_tinfoil(config_only=False, basepath=None): > """Initialize tinfoil api from bitbake""" > import scriptpath > + orig_cwd = os.path.abspath(os.curdir) > + if basepath: > + os.chdir(basepath) > bitbakepath = scriptpath.add_bitbake_lib_path() > if not bitbakepath: > logger.error("Unable to find bitbake by searching parent > directory of this script or PATH") > @@ -108,6 +111,7 @@ def setup_tinfoil(config_only=False): > tinfoil = bb.tinfoil.Tinfoil() > tinfoil.prepare(config_only) > tinfoil.logger.setLevel(logger.getEffectiveLevel()) > + os.chdir(orig_cwd) > return tinfoil > Just from a correctness standpoint, this will not go back to orig_cwd if an exception is raised. We should be using a try/finally block for things like this. -- Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics [-- Attachment #2: Type: text/html, Size: 1702 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2 1/3] devtool: second fix for running from a different directory 2015-09-23 15:09 ` Christopher Larson @ 2015-09-23 16:04 ` Paul Eggleton 0 siblings, 0 replies; 7+ messages in thread From: Paul Eggleton @ 2015-09-23 16:04 UTC (permalink / raw) To: Christopher Larson; +Cc: Patches and discussions about the oe-core layer On Wednesday 23 September 2015 08:09:37 Christopher Larson wrote: > On Wed, Sep 23, 2015 at 3:05 AM, Paul Eggleton < > > paul.eggleton@linux.intel.com> wrote: > > -def setup_tinfoil(config_only=False): > > > > +def setup_tinfoil(config_only=False, basepath=None): > > """Initialize tinfoil api from bitbake""" > > import scriptpath > > > > + orig_cwd = os.path.abspath(os.curdir) > > + if basepath: > > + os.chdir(basepath) > > > > bitbakepath = scriptpath.add_bitbake_lib_path() > > > > if not bitbakepath: > > logger.error("Unable to find bitbake by searching parent > > > > directory of this script or PATH") > > > > @@ -108,6 +111,7 @@ def setup_tinfoil(config_only=False): > > tinfoil = bb.tinfoil.Tinfoil() > > tinfoil.prepare(config_only) > > tinfoil.logger.setLevel(logger.getEffectiveLevel()) > > > > + os.chdir(orig_cwd) > > > > return tinfoil > > Just from a correctness standpoint, this will not go back to orig_cwd if an > exception is raised. We should be using a try/finally block for things like > this. That is a good point yes. I can send out a v3. Cheers, Paul -- Paul Eggleton Intel Open Source Technology Centre ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v2 2/3] devtool: runqemu: avoid recipe parse 2015-09-23 10:05 [PATCH v2 0/3] devtool fixes Paul Eggleton 2015-09-23 10:05 ` [PATCH v2 1/3] devtool: second fix for running from a different directory Paul Eggleton @ 2015-09-23 10:05 ` Paul Eggleton 2015-09-23 10:05 ` [PATCH v2 3/3] devtool: upgrade: use shutil.move instead of os.rename Paul Eggleton 2 siblings, 0 replies; 7+ messages in thread From: Paul Eggleton @ 2015-09-23 10:05 UTC (permalink / raw) To: openembedded-core We only need the base configuration to get the variable values we want to get here, there's no need to parse recipes / load the cache. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> --- scripts/lib/devtool/runqemu.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/lib/devtool/runqemu.py b/scripts/lib/devtool/runqemu.py index 4d08d8c..5282afb 100644 --- a/scripts/lib/devtool/runqemu.py +++ b/scripts/lib/devtool/runqemu.py @@ -29,7 +29,7 @@ logger = logging.getLogger('devtool') def runqemu(args, config, basepath, workspace): """Entry point for the devtool 'runqemu' subcommand""" - tinfoil = setup_tinfoil(basepath=basepath) + tinfoil = setup_tinfoil(config_only=True, basepath=basepath) machine = tinfoil.config_data.getVar('MACHINE', True) bindir_native = tinfoil.config_data.getVar('STAGING_BINDIR_NATIVE', True) tinfoil.shutdown() -- 2.1.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 3/3] devtool: upgrade: use shutil.move instead of os.rename 2015-09-23 10:05 [PATCH v2 0/3] devtool fixes Paul Eggleton 2015-09-23 10:05 ` [PATCH v2 1/3] devtool: second fix for running from a different directory Paul Eggleton 2015-09-23 10:05 ` [PATCH v2 2/3] devtool: runqemu: avoid recipe parse Paul Eggleton @ 2015-09-23 10:05 ` Paul Eggleton 2 siblings, 0 replies; 7+ messages in thread From: Paul Eggleton @ 2015-09-23 10:05 UTC (permalink / raw) To: openembedded-core From: Markus Lehtonen <markus.lehtonen@linux.intel.com> Rename fails over filesystem boundaries. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> --- scripts/lib/devtool/upgrade.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/lib/devtool/upgrade.py b/scripts/lib/devtool/upgrade.py index aa53c8e..4f850cf 100644 --- a/scripts/lib/devtool/upgrade.py +++ b/scripts/lib/devtool/upgrade.py @@ -53,7 +53,7 @@ def _copy_source_code(orig, dest): dest_dir = os.path.join(dest, os.path.dirname(path)) bb.utils.mkdirhier(dest_dir) dest_path = os.path.join(dest, path) - os.rename(os.path.join(orig, path), dest_path) + shutil.move(os.path.join(orig, path), dest_path) def _get_checksums(rf): import re -- 2.1.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 0/3] devtool fixes @ 2022-07-13 1:41 Paul Eggleton 0 siblings, 0 replies; 7+ messages in thread From: Paul Eggleton @ 2022-07-13 1:41 UTC (permalink / raw) To: openembedded-core A few fixes for devtool, mostly relating to recipes that fetch from git where S points to a subdirectory of the repo. Changes since v1: * Fixed erroneous addition of a separator when not needed (which broke one of the other devtool tests) * Fixed missing separator before patchdir= when updating original recipe * Added extra logic to the test to ensure we can update the original recipe properly (as well creating a bbappend that was already in the test). * Rebased and dropped pn- overrides fix since that has just been merged. The following changes since commit db22dbc1f66d2f76dab719597986cfd96cf18d71: linux-yocto/5.10: fix build_OID_registry/conmakehash buildpaths warning (2022-07-12 23:56:06 +0100) are available in the git repository at: git://git.openembedded.org/openembedded-core-contrib paule/devtool37-oe http://cgit.openembedded.org/openembedded-core-contrib/log/?h=paule/devtool37-oe Paul Eggleton (3): patch: handle if S points to a subdirectory of a git repo devtool: finish: handle patching when S points to subdir of a git repo oe-selftest: devtool: test modify git recipe building from a subdir meta/lib/oe/patch.py | 8 +-- meta/lib/oe/recipeutils.py | 9 ++- meta/lib/oeqa/selftest/cases/devtool.py | 114 +++++++++++++++++++++++++++----- scripts/lib/devtool/standard.py | 25 +++++-- 4 files changed, 127 insertions(+), 29 deletions(-) -- 1.8.3.1 ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-07-13 1:41 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-09-23 10:05 [PATCH v2 0/3] devtool fixes Paul Eggleton 2015-09-23 10:05 ` [PATCH v2 1/3] devtool: second fix for running from a different directory Paul Eggleton 2015-09-23 15:09 ` Christopher Larson 2015-09-23 16:04 ` Paul Eggleton 2015-09-23 10:05 ` [PATCH v2 2/3] devtool: runqemu: avoid recipe parse Paul Eggleton 2015-09-23 10:05 ` [PATCH v2 3/3] devtool: upgrade: use shutil.move instead of os.rename Paul Eggleton -- strict thread matches above, loose matches on Subject: below -- 2022-07-13 1:41 [PATCH v2 0/3] devtool fixes Paul Eggleton
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox