* [RFC PATCH 0/7] Developer workflow improvements
@ 2014-11-25 17:28 Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 1/7] lib/oe/patch: fall back to patch if git apply fails Paul Eggleton
` (11 more replies)
0 siblings, 12 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-11-25 17:28 UTC (permalink / raw)
To: openembedded-core
[Note: this is an RFC series for the moment, and shouldn't yet be merged.]
I've been looking at how to make it easier for application and system
component developers to get their work done using the tools we provide,
and I believe this patchset is a piece of the solution. There's still a
number of other pieces to come, but this should be usable on its own.
The first three patches extend the PATCHTOOL = "git" code to ensure that
we're able to apply all patches even if "git am" and "git apply" can't
handle them, and that we do a commit to the repository per patch. This
is needed for devtool later on.
I've then added a new recipe auto-creation script, recipetool, which can
take a source tree or URL and create a skeleton recipe to build it.
(In case anyone is wondering about the existing scripts/create-recipe,
frankly I consider it a dead end - it's written in Perl, which makes it
a bit difficult to integrate with the rest of our code; it's also
GPLv3-only which makes any such integration pretty much impossible from
another angle.)
Then we add devtool. This allows you to:
* Add a new piece of software (auto-create the skeleton of a recipe
using the aforementioned recipetool and point the build to an external
source tree)
* Modify the source for an existing recipe (point the build to an
external source tree, possibly creating that tree in the same step
and managing it with git)
* Deploy the installed files from a recipe from ${D} to a target using
scp.
There will obviously be extensions to these tools, but I hope they are
already functional enough to be useful at the state they are in at the
moment.
Known issues:
* "devtool modify" can't currently deal with gcc, linux-yocto or perf
because they manage their workdir differently, this will need to be
fixed
* devtool's workspace layer triggers a bitbake warning when empty.
I suspect a mechanism to disable this warning internally will need to
be added.
* "recipetool create" ideally needs to become smarter and fill in
more details of the recipe. At some point we'll probably have to
make the process interactive and possibly have it automate parts of
the build process and examine the output; some things just can't be
practically detected from the source tree without building.
(Note that some pieces of the code which are not essential rely on a couple
of changes to bitbake, you can find these in the paule/devtool-bb branch
in poky-contrib, or if you are happy to use poky, grab the paule/devtool
branch in poky-contrib which has everything).
Comments/questions/suggestions welcome!
The following changes since commit ff2ff155ea5273b2023a1c9834b13f10249d343f:
gdk-pixbuf: use ptest-gnome (2014-11-25 12:58:21 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/devtool
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/devtool
Paul Eggleton (6):
lib/oe/patch: fall back to patch if git apply fails
lib/oe/patch: auto-commit when falling back from git am
lib/oe/patch: use --keep-cr with git am
scripts/recipetool: Add a recipe auto-creation script
lib/oe: add recipeutils module
scripts/devtool: add development helper tool
Junchun Guan (1):
scripts/devtool: Support deploy/undeploy function
meta/lib/oe/patch.py | 94 ++++++-
meta/lib/oe/recipeutils.py | 264 ++++++++++++++++++++
scripts/devtool | 223 +++++++++++++++++
scripts/lib/devtool/__init__.py | 79 ++++++
scripts/lib/devtool/deploy.py | 100 ++++++++
scripts/lib/devtool/standard.py | 390 ++++++++++++++++++++++++++++++
scripts/lib/recipetool/__init__.py | 0
scripts/lib/recipetool/create.py | 375 ++++++++++++++++++++++++++++
scripts/lib/recipetool/create_buildsys.py | 234 ++++++++++++++++++
scripts/recipetool | 124 ++++++++++
10 files changed, 1881 insertions(+), 2 deletions(-)
create mode 100644 meta/lib/oe/recipeutils.py
create mode 100755 scripts/devtool
create mode 100644 scripts/lib/devtool/__init__.py
create mode 100644 scripts/lib/devtool/deploy.py
create mode 100644 scripts/lib/devtool/standard.py
create mode 100644 scripts/lib/recipetool/__init__.py
create mode 100644 scripts/lib/recipetool/create.py
create mode 100644 scripts/lib/recipetool/create_buildsys.py
create mode 100755 scripts/recipetool
--
1.9.3
^ permalink raw reply [flat|nested] 40+ messages in thread
* [RFC PATCH 1/7] lib/oe/patch: fall back to patch if git apply fails
2014-11-25 17:28 [RFC PATCH 0/7] Developer workflow improvements Paul Eggleton
@ 2014-11-25 17:28 ` Paul Eggleton
2014-11-25 17:40 ` Paul Barker
2014-11-25 17:28 ` [RFC PATCH 2/7] lib/oe/patch: auto-commit when falling back from git am Paul Eggleton
` (10 subsequent siblings)
11 siblings, 1 reply; 40+ messages in thread
From: Paul Eggleton @ 2014-11-25 17:28 UTC (permalink / raw)
To: openembedded-core
When PATCHTOOL = "git", git apply doesn't support fuzzy application, so
if a patch requires that it's better to be able to apply it rather than
just failing.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/lib/oe/patch.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index b085c9d..788f465 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -219,7 +219,11 @@ class GitApplyTree(PatchTree):
return _applypatchhelper(shellcmd, patch, force, reverse, run)
except CmdError:
shellcmd = ["git", "--git-dir=.", "apply", "-p%s" % patch['strippath']]
- return _applypatchhelper(shellcmd, patch, force, reverse, run)
+ try:
+ output = _applypatchhelper(shellcmd, patch, force, reverse, run)
+ except CmdError:
+ output = PatchTree._applypatch(self, patch, force, reverse, run)
+ return output
class QuiltTree(PatchSet):
--
1.9.3
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [RFC PATCH 2/7] lib/oe/patch: auto-commit when falling back from git am
2014-11-25 17:28 [RFC PATCH 0/7] Developer workflow improvements Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 1/7] lib/oe/patch: fall back to patch if git apply fails Paul Eggleton
@ 2014-11-25 17:28 ` Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 3/7] lib/oe/patch: use --keep-cr with " Paul Eggleton
` (9 subsequent siblings)
11 siblings, 0 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-11-25 17:28 UTC (permalink / raw)
To: openembedded-core
When PATCHTOOL = "git", if we're not able to use "git am" to apply a
patch and fall back to "git apply" or "patch", it is desirable to
actually commit the changes, attempting to preserve (and interpret) the
patch header as part of the commit message if present. As a bonus, the
code for extracting the commit message is callable externally in case it
is useful elsewhere.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/lib/oe/patch.py | 86 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 86 insertions(+)
diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index 788f465..2d56ba4 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -202,6 +202,78 @@ class GitApplyTree(PatchTree):
def __init__(self, dir, d):
PatchTree.__init__(self, dir, d)
+ @staticmethod
+ def extractPatchHeader(patchfile):
+ """
+ Extract just the header lines from the top of a patch file
+ """
+ lines = []
+ with open(patchfile, 'r') as f:
+ for line in f.readlines():
+ if line.startswith('Index: ') or line.startswith('diff -') or line.startswith('---'):
+ break
+ lines.append(line)
+ return lines
+
+ @staticmethod
+ def prepareCommit(patchfile):
+ """
+ Prepare a git commit command line based on the header from a patch file
+ (typically this is useful for patches that cannot be applied with "git am" due to formatting)
+ """
+ import tempfile
+ import re
+ author_re = re.compile('[\S ]+ <\S+@\S+\.\S+>')
+ # Process patch header and extract useful information
+ lines = GitApplyTree.extractPatchHeader(patchfile)
+ outlines = []
+ author = None
+ date = None
+ for line in lines:
+ if line.startswith('Subject: '):
+ subject = line.split(':', 1)[1]
+ # Remove any [PATCH][oe-core] etc.
+ subject = re.sub(r'\[.+?\]\s*', '', subject)
+ outlines.insert(0, '%s\n\n' % subject.strip())
+ continue
+ if line.startswith('From: ') or line.startswith('Author: '):
+ authorval = line.split(':', 1)[1].strip().replace('"', '')
+ # git is fussy about author formatting i.e. it must be Name <email@domain>
+ if author_re.match(authorval):
+ author = authorval
+ continue
+ if line.startswith('Date: '):
+ if date is None:
+ dateval = line.split(':', 1)[1].strip()
+ # Very crude check for date format, since git will blow up if it's not in the right
+ # format. Without e.g. a python-dateutils dependency we can't do a whole lot more
+ if len(dateval) > 12:
+ date = dateval
+ continue
+ if line.startswith('Signed-off-by: '):
+ authorval = line.split(':', 1)[1].strip().replace('"', '')
+ # git is fussy about author formatting i.e. it must be Name <email@domain>
+ if author_re.match(authorval):
+ author = authorval
+ outlines.append(line)
+ # Add a pointer to the original patch file name
+ if outlines and outlines[-1].strip():
+ outlines.append('\n')
+ outlines.append('(from original patch: %s)\n' % os.path.basename(patchfile))
+ # Write out commit message to a file
+ with tempfile.NamedTemporaryFile('w', delete=False) as tf:
+ tmpfile = tf.name
+ for line in outlines:
+ tf.write(line)
+ # Prepare git command
+ cmd = ["git", "commit", "-F", tmpfile]
+ # git doesn't like plain email addresses as authors
+ if author and '<' in author:
+ cmd.append('--author="%s"' % author)
+ if date:
+ cmd.append('--date="%s"' % date)
+ return (tmpfile, cmd)
+
def _applypatch(self, patch, force = False, reverse = False, run = True):
def _applypatchhelper(shellcmd, patch, force = False, reverse = False, run = True):
if reverse:
@@ -218,11 +290,25 @@ class GitApplyTree(PatchTree):
shellcmd = ["git", "--work-tree=.", "am", "-3", "-p%s" % patch['strippath']]
return _applypatchhelper(shellcmd, patch, force, reverse, run)
except CmdError:
+ # Fall back to git apply
shellcmd = ["git", "--git-dir=.", "apply", "-p%s" % patch['strippath']]
try:
output = _applypatchhelper(shellcmd, patch, force, reverse, run)
except CmdError:
+ # Fall back to patch
output = PatchTree._applypatch(self, patch, force, reverse, run)
+ # Add all files
+ shellcmd = ["git", "add", "-f", "."]
+ output += runcmd(["sh", "-c", " ".join(shellcmd)], self.dir)
+ # Exclude the patches directory
+ shellcmd = ["git", "reset", "HEAD", self.patchdir]
+ output += runcmd(["sh", "-c", " ".join(shellcmd)], self.dir)
+ # Commit the result
+ (tmpfile, shellcmd) = self.prepareCommit(patch['file'])
+ try:
+ output += runcmd(["sh", "-c", " ".join(shellcmd)], self.dir)
+ finally:
+ os.remove(tmpfile)
return output
--
1.9.3
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [RFC PATCH 3/7] lib/oe/patch: use --keep-cr with git am
2014-11-25 17:28 [RFC PATCH 0/7] Developer workflow improvements Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 1/7] lib/oe/patch: fall back to patch if git apply fails Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 2/7] lib/oe/patch: auto-commit when falling back from git am Paul Eggleton
@ 2014-11-25 17:28 ` Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 4/7] scripts/recipetool: Add a recipe auto-creation script Paul Eggleton
` (8 subsequent siblings)
11 siblings, 0 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-11-25 17:28 UTC (permalink / raw)
To: openembedded-core
Preserving carriage returns is important where the patch contains them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/lib/oe/patch.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index 2d56ba4..2bf3065 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -287,7 +287,7 @@ class GitApplyTree(PatchTree):
return runcmd(["sh", "-c", " ".join(shellcmd)], self.dir)
try:
- shellcmd = ["git", "--work-tree=.", "am", "-3", "-p%s" % patch['strippath']]
+ shellcmd = ["git", "--work-tree=.", "am", "-3", "--keep-cr", "-p%s" % patch['strippath']]
return _applypatchhelper(shellcmd, patch, force, reverse, run)
except CmdError:
# Fall back to git apply
--
1.9.3
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [RFC PATCH 4/7] scripts/recipetool: Add a recipe auto-creation script
2014-11-25 17:28 [RFC PATCH 0/7] Developer workflow improvements Paul Eggleton
` (2 preceding siblings ...)
2014-11-25 17:28 ` [RFC PATCH 3/7] lib/oe/patch: use --keep-cr with " Paul Eggleton
@ 2014-11-25 17:28 ` Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 5/7] lib/oe: add recipeutils module Paul Eggleton
` (7 subsequent siblings)
11 siblings, 0 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-11-25 17:28 UTC (permalink / raw)
To: openembedded-core
Add a more maintainable and flexible script for creating at least the
skeleton of a recipe based on an examination of the source tree.
Commands can be added and the creation process can be extended through
plugins.
[YOCTO #6406]
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
scripts/lib/recipetool/__init__.py | 0
scripts/lib/recipetool/create.py | 375 ++++++++++++++++++++++++++++++
scripts/lib/recipetool/create_buildsys.py | 234 +++++++++++++++++++
scripts/recipetool | 124 ++++++++++
4 files changed, 733 insertions(+)
create mode 100644 scripts/lib/recipetool/__init__.py
create mode 100644 scripts/lib/recipetool/create.py
create mode 100644 scripts/lib/recipetool/create_buildsys.py
create mode 100755 scripts/recipetool
diff --git a/scripts/lib/recipetool/__init__.py b/scripts/lib/recipetool/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
new file mode 100644
index 0000000..07b33a8
--- /dev/null
+++ b/scripts/lib/recipetool/create.py
@@ -0,0 +1,375 @@
+# Recipe creation tool - create command plugin
+#
+# Copyright (C) 2014 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+import sys
+import os
+import argparse
+import glob
+import fnmatch
+import re
+import logging
+
+logger = logging.getLogger('recipetool')
+
+tinfoil = None
+plugins = None
+
+def plugin_init(pluginlist):
+ # Take a reference to the list so we can use it later
+ global plugins
+ plugins = pluginlist
+
+def tinfoil_init(instance):
+ global tinfoil
+ tinfoil = instance
+
+class RecipeHandler():
+ def checkfiles(self, path, speclist):
+ results = []
+ for spec in speclist:
+ results.extend(glob.glob(os.path.join(path, spec)))
+ return results
+
+ def genfunction(self, outlines, funcname, content):
+ outlines.append('%s () {' % funcname)
+ for line in content:
+ outlines.append('\t%s' % line)
+ outlines.append('}')
+ outlines.append('')
+
+ def process(self, srctree, classes, lines_before, lines_after, handled):
+ return False
+
+
+
+def fetch_source(uri, destdir):
+ import bb.data
+ bb.utils.mkdirhier(destdir)
+ localdata = bb.data.createCopy(tinfoil.config_data)
+ bb.data.update_data(localdata)
+ localdata.setVar('BB_STRICT_CHECKSUM', '')
+ localdata.setVar('SRCREV', '${AUTOREV}')
+ ret = (None, None)
+ olddir = os.getcwd()
+ try:
+ fetcher = bb.fetch2.Fetch([uri], localdata)
+ for u in fetcher.ud:
+ ud = fetcher.ud[u]
+ ud.ignore_checksums = True
+ fetcher.download()
+ fetcher.unpack(destdir)
+ for u in fetcher.ud:
+ ud = fetcher.ud[u]
+ if ud.method.recommends_checksum(ud):
+ md5value = bb.utils.md5_file(ud.localpath)
+ sha256value = bb.utils.sha256_file(ud.localpath)
+ ret = (md5value, sha256value)
+ except bb.fetch2.BBFetchException, e:
+ raise bb.build.FuncFailed(e)
+ finally:
+ os.chdir(olddir)
+ return ret
+
+def supports_srcrev(uri):
+ localdata = bb.data.createCopy(tinfoil.config_data)
+ bb.data.update_data(localdata)
+ fetcher = bb.fetch2.Fetch([uri], localdata)
+ urldata = fetcher.ud
+ for u in urldata:
+ if urldata[u].method.supports_srcrev():
+ return True
+ return False
+
+def create_recipe(args):
+ import bb.process
+ import tempfile
+ import shutil
+
+ pkgarch = ""
+ if args.machine:
+ pkgarch = "${MACHINE_ARCH}"
+
+ checksums = (None, None)
+ tempsrc = ''
+ srcsubdir = ''
+ if '://' in args.source:
+ # Fetch a URL
+ srcuri = args.source
+ if args.externalsrc:
+ srctree = args.externalsrc
+ else:
+ tempsrc = tempfile.mkdtemp(prefix='recipetool-')
+ srctree = tempsrc
+ logger.info('Fetching %s...' % srcuri)
+ checksums = fetch_source(args.source, srctree)
+ dirlist = os.listdir(srctree)
+ if 'git.indirectionsymlink' in dirlist:
+ dirlist.remove('git.indirectionsymlink')
+ if len(dirlist) == 1 and os.path.isdir(os.path.join(srctree, dirlist[0])):
+ # We unpacked a single directory, so we should use that
+ srcsubdir = dirlist[0]
+ srctree = os.path.join(srctree, srcsubdir)
+ else:
+ # Assume we're pointing to an existing source tree
+ if args.externalsrc:
+ logger.error('externalsrc cannot be specified if source is a directory')
+ sys.exit(1)
+ if not os.path.isdir(args.source):
+ logger.error('Invalid source directory %s' % args.source)
+ sys.exit(1)
+ srcuri = ''
+ srctree = args.source
+
+ outfile = args.outfile
+ if outfile and outfile != '-':
+ if os.path.exists(outfile):
+ logger.error('Output file %s already exists' % outfile)
+ sys.exit(1)
+
+ lines_before = []
+ lines_after = []
+
+ lines_before.append('# Recipe created by %s' % os.path.basename(sys.argv[0]))
+ lines_before.append('# This is the basis of a recipe and may need further editing in order to be fully functional.')
+ lines_before.append('# (Feel free to remove these comments when editing.)')
+ lines_before.append('#')
+
+ licvalues = guess_license(srctree)
+ licenses = []
+ lic_files_chksum = []
+ for licvalue in licvalues:
+ if not licvalue[0] in licenses:
+ licenses.append(licvalue[0])
+ lic_files_chksum.append('file://%s;md5=%s' % (licvalue[1], licvalue[2]))
+ if licvalues:
+ lines_before.append('# WARNING: the following LICENSE and LIC_FILES_CHKSUM values are best guesses - it is')
+ lines_before.append('# your responsibility to verify that the values are complete and correct.')
+ if len(licvalues) > 1:
+ lines_before.append('#')
+ lines_before.append('# NOTE: multiple licenses have been detected; if that is correct you should separate')
+ lines_before.append('# these in the LICENSE value using & if the multiple licenses all apply, or | if there')
+ lines_before.append('# is a choice between the multiple licenses. If in doubt, check the accompanying')
+ lines_before.append('# documentation to determine which situation is applicable.')
+ else:
+ lines_before.append('# Unable to find any files that looked like license statements. Check the accompanying')
+ lines_before.append('# documentation and source headers and set LICENSE and LIC_FILES_CHKSUM accordingly.')
+ lines_before.append('LICENSE = "%s"' % ' '.join(licenses))
+ lines_before.append('LIC_FILES_CHKSUM = "%s"' % ' \\\n '.join(lic_files_chksum))
+ lines_before.append('')
+
+ # FIXME This is kind of a hack, we probably ought to be using bitbake to do this
+ # we'd also want a way to automatically set outfile based upon auto-detecting these values from the source if possible
+ recipefn = os.path.splitext(os.path.basename(outfile))[0]
+ fnsplit = recipefn.split('_')
+ if len(fnsplit) > 1:
+ pn = fnsplit[0]
+ pv = fnsplit[1]
+ else:
+ pn = recipefn
+ pv = None
+
+ if srcuri:
+ if pv and pv not in 'git svn hg'.split():
+ srcuri = srcuri.replace(pv, '${PV}')
+ else:
+ lines_before.append('# No information for SRC_URI yet (only an external source tree was specified)')
+ lines_before.append('SRC_URI = "%s"' % srcuri)
+ (md5value, sha256value) = checksums
+ if md5value:
+ lines_before.append('SRC_URI[md5sum] = "%s"' % md5value)
+ if sha256value:
+ lines_before.append('SRC_URI[sha256sum] = "%s"' % sha256value)
+ if srcuri and supports_srcrev(srcuri):
+ lines_before.append('')
+ lines_before.append('# Modify these as desired')
+ lines_before.append('PV = "1.0+git${SRCPV}"')
+ lines_before.append('SRCREV = "${AUTOREV}"')
+ lines_before.append('')
+
+ if srcsubdir and pv:
+ if srcsubdir == "%s-%s" % (pn, pv):
+ # This would be the default, so we don't need to set S in the recipe
+ srcsubdir = ''
+ if srcsubdir:
+ lines_before.append('S = "${WORKDIR}/%s"' % srcsubdir)
+ lines_before.append('')
+
+ if pkgarch:
+ lines_after.append('PACKAGE_ARCH = "%s"' % pkgarch)
+ lines_after.append('')
+
+ # Find all plugins that want to register handlers
+ handlers = []
+ for plugin in plugins:
+ if hasattr(plugin, 'register_recipe_handlers'):
+ plugin.register_recipe_handlers(handlers)
+
+ # Apply the handlers
+ classes = []
+ handled = []
+ for handler in handlers:
+ handler.process(srctree, classes, lines_before, lines_after, handled)
+
+ outlines = []
+ outlines.extend(lines_before)
+ if classes:
+ outlines.append('inherit %s' % ' '.join(classes))
+ outlines.append('')
+ outlines.extend(lines_after)
+
+ if outfile == '-':
+ sys.stdout.write('\n'.join(outlines) + '\n')
+ else:
+ with open(outfile, 'w') as f:
+ f.write('\n'.join(outlines) + '\n')
+ logger.info('Recipe %s has been created; further editing may be required to make it fully functional' % outfile)
+
+ if tempsrc:
+ shutil.rmtree(tempsrc)
+
+ return 0
+
+
+def guess_license(srctree):
+ import bb
+ # Gather md5sums of license files in common license dir
+ #commonlicdir = os.path.abspath(os.path.join(os.path.basename(__file__), '../meta/files/common-licenses'))
+ commonlicdir = tinfoil.config_data.getVar('COMMON_LICENSE_DIR', True)
+ md5sums = {}
+ for fn in os.listdir(commonlicdir):
+ md5value = bb.utils.md5_file(os.path.join(commonlicdir, fn))
+ md5sums[md5value] = fn
+ # FIXME we probably want more common values of this as well
+ md5sums['94d55d512a9ba36caa9b7df079bae19f'] = 'GPLv2'
+ md5sums['b234ee4d69f5fce4486a80fdaf4a4263'] = 'GPLv2'
+ md5sums['59530bdf33659b29e73d4adb9f9f6552'] = 'GPLv2'
+ md5sums['bbb461211a33b134d42ed5ee802b37ff'] = 'LGPLv2.1'
+ md5sums['7fbc338309ac38fefcd64b04bb903e34'] = 'LGPLv2.1'
+ md5sums['d32239bcb673463ab874e80d47fae504'] = 'GPLv3'
+ md5sums['6a6a8e020838b23406c81b19c1d46df6'] = 'LGPLv3'
+ md5sums['9f604d8a4f8e74f4f5140845a21b6674'] = 'LGPLv2'
+
+ licenses = []
+ licspecs = ['LICENSE*', 'COPYING*', '*[Ll]icense*', 'LICENCE*', 'LEGAL*', '[Ll]egal*', '*GPL*', 'README.lic*', 'COPYRIGHT*', '[Cc]opyright*']
+ licfiles = []
+ for root, dirs, files in os.walk(srctree):
+ for fn in files:
+ for spec in licspecs:
+ if fnmatch.fnmatch(fn, spec):
+ licfiles.append(os.path.join(root, fn))
+ for licfile in licfiles:
+ md5value = bb.utils.md5_file(licfile)
+ license = md5sums.get(md5value, 'Unknown')
+ licenses.append((license, os.path.relpath(licfile, srctree), md5value))
+
+ # FIXME should we grab at least one source file with a license header and add that too?
+
+ return licenses
+
+def read_pkgconfig_provides(d):
+ pkgdatadir = d.getVar('PKGDATA_DIR', True)
+ pkgmap = {}
+ for fn in glob.glob(os.path.join(pkgdatadir, 'shlibs2', '*.pclist')):
+ with open(fn, 'r') as f:
+ for line in f:
+ pkgmap[os.path.basename(line.rstrip())] = os.path.splitext(os.path.basename(fn))[0]
+ recipemap = {}
+ for pc, pkg in pkgmap.iteritems():
+ pkgdatafile = os.path.join(pkgdatadir, 'runtime', pkg)
+ if os.path.exists(pkgdatafile):
+ with open(pkgdatafile, 'r') as f:
+ for line in f:
+ if line.startswith('PN: '):
+ recipemap[pc] = line.split(':', 1)[1].strip()
+ return recipemap
+
+def convert_pkginfo(pkginfofile):
+ values = {}
+ with open(pkginfofile, 'r') as f:
+ indesc = False
+ for line in f:
+ if indesc:
+ if line.strip():
+ values['DESCRIPTION'] += ' ' + line.strip()
+ else:
+ indesc = False
+ else:
+ splitline = line.split(': ', 1)
+ key = line[0]
+ value = line[1]
+ if key == 'LICENSE':
+ for dep in value.split(','):
+ dep = dep.split()[0]
+ mapped = depmap.get(dep, '')
+ if mapped:
+ depends.append(mapped)
+ elif key == 'License':
+ values['LICENSE'] = value
+ elif key == 'Summary':
+ values['SUMMARY'] = value
+ elif key == 'Description':
+ values['DESCRIPTION'] = value
+ indesc = True
+ return values
+
+def convert_debian(debpath):
+ # FIXME extend this mapping - perhaps use distro_alias.inc?
+ depmap = {'libz-dev': 'zlib'}
+
+ values = {}
+ depends = []
+ with open(os.path.join(debpath, 'control')) as f:
+ indesc = False
+ for line in f:
+ if indesc:
+ if line.strip():
+ if line.startswith(' This package contains'):
+ indesc = False
+ else:
+ values['DESCRIPTION'] += ' ' + line.strip()
+ else:
+ indesc = False
+ else:
+ splitline = line.split(':', 1)
+ key = line[0]
+ value = line[1]
+ if key == 'Build-Depends':
+ for dep in value.split(','):
+ dep = dep.split()[0]
+ mapped = depmap.get(dep, '')
+ if mapped:
+ depends.append(mapped)
+ elif key == 'Section':
+ values['SECTION'] = value
+ elif key == 'Description':
+ values['SUMMARY'] = value
+ indesc = True
+
+ if depends:
+ values['DEPENDS'] = ' '.join(depends)
+
+ return values
+
+
+def register_command(subparsers):
+ parser_create = subparsers.add_parser('create', help='Create a new recipe')
+ parser_create.add_argument('source', help='Path or URL to source')
+ parser_create.add_argument('-o', '--outfile', help='Full path and filename to recipe to add', required=True)
+ parser_create.add_argument('-m', '--machine', help='Make recipe machine-specific as opposed to architecture-specific', action='store_true')
+ parser_create.add_argument('-x', '--externalsrc', help='Assuming source is a URL, fetch it and extract it to the specified directory')
+ parser_create.set_defaults(func=create_recipe)
+
diff --git a/scripts/lib/recipetool/create_buildsys.py b/scripts/lib/recipetool/create_buildsys.py
new file mode 100644
index 0000000..98fd7f8
--- /dev/null
+++ b/scripts/lib/recipetool/create_buildsys.py
@@ -0,0 +1,234 @@
+# Recipe creation tool - create command build system handlers
+#
+# Copyright (C) 2014 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+import re
+import logging
+from recipetool.create import RecipeHandler, read_pkgconfig_provides
+
+logger = logging.getLogger('recipetool')
+
+tinfoil = None
+
+def tinfoil_init(instance):
+ global tinfoil
+ tinfoil = instance
+
+class CmakeRecipeHandler(RecipeHandler):
+ def process(self, srctree, classes, lines_before, lines_after, handled):
+ if 'buildsystem' in handled:
+ return False
+
+ if self.checkfiles(srctree, ['CMakeLists.txt']):
+ classes.append('cmake')
+ lines_after.append('# Specify any options you want to pass to cmake using EXTRA_OECMAKE:')
+ lines_after.append('EXTRA_OECMAKE = ""')
+ lines_after.append('')
+ handled.append('buildsystem')
+ return True
+ return False
+
+class SconsRecipeHandler(RecipeHandler):
+ def process(self, srctree, classes, lines_before, lines_after, handled):
+ if 'buildsystem' in handled:
+ return False
+
+ if self.checkfiles(srctree, ['SConstruct', 'Sconstruct', 'sconstruct']):
+ classes.append('scons')
+ lines_after.append('# Specify any options you want to pass to scons using EXTRA_OESCONS:')
+ lines_after.append('EXTRA_OESCONS = ""')
+ lines_after.append('')
+ handled.append('buildsystem')
+ return True
+ return False
+
+class QmakeRecipeHandler(RecipeHandler):
+ def process(self, srctree, classes, lines_before, lines_after, handled):
+ if 'buildsystem' in handled:
+ return False
+
+ if self.checkfiles(srctree, ['*.pro']):
+ classes.append('qmake2')
+ handled.append('buildsystem')
+ return True
+ return False
+
+class AutotoolsRecipeHandler(RecipeHandler):
+ def process(self, srctree, classes, lines_before, lines_after, handled):
+ if 'buildsystem' in handled:
+ return False
+
+ autoconf = False
+ if self.checkfiles(srctree, ['configure.ac', 'configure.in']):
+ autoconf = True
+ values = self._extract_autotools_deps(lines_before, srctree)
+ classes.extend(values.pop('inherit', '').split())
+ for var, value in values.iteritems():
+ lines_before.append('%s = "%s"' % (var, value))
+ else:
+ conffile = self.checkfiles(srctree, ['configure'])
+ if conffile:
+ # Check if this is just a pre-generated autoconf configure script
+ with open(conffile[0], 'r') as f:
+ for i in range(1, 10):
+ if 'Generated by GNU Autoconf' in f.readline():
+ autoconf = True
+ break
+
+ if autoconf:
+ lines_before.append('# NOTE: if this software is not capable of being built in a separate build directory')
+ lines_before.append('# from the source, you should replace autotools with autotools-brokensep in the')
+ lines_before.append('# inherit line')
+ classes.append('autotools')
+ lines_after.append('# Specify any options you want to pass to the configure script using EXTRA_OECONF:')
+ lines_after.append('EXTRA_OECONF = ""')
+ lines_after.append('')
+ handled.append('buildsystem')
+ return True
+
+ return False
+
+ def _extract_autotools_deps(self, outlines, srctree):
+ import shlex
+
+ values = {}
+ inherits = []
+
+ # FIXME this mapping is very thin
+ progmap = {'flex': 'flex-native',
+ 'bison': 'bison-native',
+ 'm4': 'm4-native'}
+ progclassmap = {'gconftool-2': 'gconf',
+ 'pkg-config': 'pkgconfig'}
+
+ pkg_re = re.compile('PKG_CHECK_MODULES\(\[?[a-zA-Z0-9]*\]?, \[?([^,\]]*)[),].*')
+ progs_re = re.compile('_PROGS?\(\[?[a-zA-Z0-9]*\]?, \[?([^,\]]*)\]?[),].*')
+ dep_re = re.compile('([^ ><=]+)( [<>=]+ [^ ><=]+)?')
+
+ # Since a configure.ac file is essentially a program, this is only ever going to be
+ # a hack unfortunately; but it ought to be enough of an approximation
+ srcfiles = self.checkfiles(srctree, ['configure.ac', 'configure.in'])
+ pcdeps = []
+ deps = []
+ unmapped = []
+ with open(srcfiles[0], 'r') as f:
+ for line in f:
+ if 'PKG_CHECK_MODULES' in line:
+ res = pkg_re.search(line)
+ if res:
+ res = dep_re.findall(res.group(1))
+ if res:
+ pcdeps.extend([x[0] for x in res])
+ inherits.append('pkgconfig')
+ if line.lstrip().startswith('AM_GNU_GETTEXT'):
+ inherits.append('gettext')
+ elif 'AC_CHECK_PROG' in line or 'AC_PATH_PROG' in line:
+ res = progs_re.search(line)
+ if res:
+ for prog in shlex.split(res.group(1)):
+ prog = prog.split()[0]
+ progclass = progclassmap.get(prog, None)
+ if progclass:
+ inherits.append(progclass)
+ else:
+ progdep = progmap.get(prog, None)
+ if progdep:
+ deps.append(progdep)
+ else:
+ unmapped.append(prog)
+
+ if unmapped:
+ outlines.append('# NOTE: the following prog dependencies are unknown, ignoring: %s' % ' '.join(unmapped))
+
+ recipemap = read_pkgconfig_provides(tinfoil.config_data)
+ unmapped = []
+ for pcdep in pcdeps:
+ recipe = recipemap.get(pcdep, None)
+ if recipe:
+ deps.append(recipe)
+ else:
+ unmapped.append(pcdep)
+
+ if unmapped:
+ outlines.append('# WARNING: unable to map the following pkg-config dependencies: %s' % ' '.join(unmapped))
+ if deps:
+ values['DEPENDS'] = ' '.join(list(set(deps)))
+
+ if inherits:
+ values['inherit'] = ' '.join(list(set(inherits)))
+
+ return values
+
+
+class MakefileRecipeHandler(RecipeHandler):
+ def process(self, srctree, classes, lines_before, lines_after, handled):
+ if 'buildsystem' in handled:
+ return False
+
+ makefile = self.checkfiles(srctree, ['Makefile'])
+ if makefile:
+ lines_after.append('# NOTE: this is a Makefile-only piece of software, so we cannot generate much of the')
+ lines_after.append('# recipe automatically - you will need to examine the Makefile yourself and ensure')
+ lines_after.append('# that the appropriate arguments are passed in.')
+ lines_after.append('')
+
+ self.genfunction(lines_after, 'do_configure', ['# Specify any needed configure commands here'])
+
+ func = []
+ func.append('# You will almost certainly need to add additional arguments here')
+ func.append('oe_runmake')
+ self.genfunction(lines_after, 'do_compile', func)
+
+ installtarget = True
+ try:
+ stdout, stderr = bb.process.run('make -qn install', cwd=srctree, shell=True)
+ except bb.process.ExecutionError as e:
+ if e.exitcode != 1:
+ installtarget = False
+ func = []
+ if installtarget:
+ func.append('# This is a guess; additional arguments may be required')
+ makeargs = ''
+ with open(makefile[0], 'r') as f:
+ for i in range(1, 100):
+ if 'DESTDIR' in f.readline():
+ makeargs += " 'DESTDIR=${D}'"
+ break
+ func.append('oe_runmake install%s' % makeargs)
+ else:
+ func.append('# NOTE: unable to determine what to put here - there is a Makefile but no')
+ func.append('# target named "install", so you will need to define this yourself')
+ self.genfunction(lines_after, 'do_install', func)
+
+ handled.append('buildsystem')
+ else:
+ lines_after.append('# NOTE: no Makefile found, unable to determine what needs to be done')
+ lines_after.append('')
+ self.genfunction(lines_after, 'do_configure', ['# Specify any needed configure commands here'])
+ self.genfunction(lines_after, 'do_compile', ['# Specify compilation commands here'])
+ self.genfunction(lines_after, 'do_install', ['# Specify install commands here'])
+
+
+def plugin_init(pluginlist):
+ pass
+
+def register_recipe_handlers(handlers):
+ # These are in a specific order so that the right one is detected first
+ handlers.append(CmakeRecipeHandler())
+ handlers.append(AutotoolsRecipeHandler())
+ handlers.append(SconsRecipeHandler())
+ handlers.append(QmakeRecipeHandler())
+ handlers.append(MakefileRecipeHandler())
diff --git a/scripts/recipetool b/scripts/recipetool
new file mode 100755
index 0000000..0e2f527
--- /dev/null
+++ b/scripts/recipetool
@@ -0,0 +1,124 @@
+#!/usr/bin/env python
+
+# Recipe creation tool
+#
+# Copyright (C) 2014 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+import sys
+import os
+import argparse
+import glob
+import logging
+
+def logger_create(name):
+ logger = logging.getLogger(name)
+ loggerhandler = logging.StreamHandler()
+ loggerhandler.setFormatter(logging.Formatter("%(levelname)s: %(message)s"))
+ logger.addHandler(loggerhandler)
+ logger.setLevel(logging.INFO)
+ return logger
+logger = logger_create('recipetool')
+
+plugins = []
+
+def load_plugins(pluginpath):
+ global plugins
+
+ def load_plugin(name):
+ logger.debug('Loading plugin %s' % name)
+ fp, pathname, description = imp.find_module(name, [pluginpath])
+ try:
+ return imp.load_module(name, fp, pathname, description)
+ finally:
+ if fp:
+ fp.close()
+
+ logger.debug('Loading plugins from %s...' % pluginpath)
+ import imp
+ for fn in glob.glob(os.path.join(pluginpath, '*.py')):
+ name = os.path.splitext(os.path.basename(fn))[0]
+ if name != '__init__':
+ plugin = load_plugin(name)
+ if hasattr(plugin, 'plugin_init'):
+ plugin.plugin_init(plugins)
+ plugins.append(plugin)
+
+def tinfoil_init():
+ import bb.tinfoil
+ import logging
+ tinfoil = bb.tinfoil.Tinfoil()
+ tinfoil.prepare(True)
+
+ for plugin in plugins:
+ if hasattr(plugin, 'tinfoil_init'):
+ plugin.tinfoil_init(tinfoil)
+ tinfoil.logger.setLevel(logging.WARNING)
+
+def main():
+
+ scripts_path = os.path.abspath(os.path.dirname(os.path.abspath(sys.argv[0])))
+ lib_path = scripts_path + '/lib'
+ sys.path = sys.path + [lib_path]
+
+ if not os.environ.get('BUILDDIR', ''):
+ logger.error("This script can only be run after initialising the build environment (e.g. by using oe-init-build-env)")
+ sys.exit(1)
+
+ parser = argparse.ArgumentParser(description="Recipe tool")
+ parser.add_argument('-d', '--debug', help='Enable debug output', action='store_true')
+ parser.add_argument('-q', '--quiet', help='Print only errors', action='store_true')
+ subparsers = parser.add_subparsers()
+
+ load_plugins(os.path.join(os.path.dirname(__file__), 'lib', 'recipetool'))
+ registered = False
+ for plugin in plugins:
+ if hasattr(plugin, 'register_command'):
+ registered = True
+ plugin.register_command(subparsers)
+
+ if not registered:
+ logger.error("No commands registered - missing plugins?")
+ sys.exit(1)
+
+ args = parser.parse_args()
+
+ if args.debug:
+ logger.setLevel(logging.DEBUG)
+ elif args.quiet:
+ logger.setLevel(logging.ERROR)
+
+ import scriptpath
+ bitbakepath = scriptpath.add_bitbake_lib_path()
+ if not bitbakepath:
+ logger.error("Unable to find bitbake by searching parent directory of this script or PATH")
+ sys.exit(1)
+ logger.debug('Found bitbake path: %s' % bitbakepath)
+
+ tinfoil_init()
+
+ ret = args.func(args)
+
+ return ret
+
+
+if __name__ == "__main__":
+ try:
+ ret = main()
+ except Exception:
+ ret = 1
+ import traceback
+ traceback.print_exc(5)
+ sys.exit(ret)
--
1.9.3
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [RFC PATCH 5/7] lib/oe: add recipeutils module
2014-11-25 17:28 [RFC PATCH 0/7] Developer workflow improvements Paul Eggleton
` (3 preceding siblings ...)
2014-11-25 17:28 ` [RFC PATCH 4/7] scripts/recipetool: Add a recipe auto-creation script Paul Eggleton
@ 2014-11-25 17:28 ` Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 6/7] scripts/devtool: add development helper tool Paul Eggleton
` (6 subsequent siblings)
11 siblings, 0 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-11-25 17:28 UTC (permalink / raw)
To: openembedded-core
Add a module to help provide utility functions for dealing with recipes.
This would typically be used by external tools.
Substantial portions of this module were borrowed from the OE Layer
index code; other functions originally contributed by
Markus Lehtonen <markus.lehtonen@intel.com>.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/lib/oe/recipeutils.py | 264 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 264 insertions(+)
create mode 100644 meta/lib/oe/recipeutils.py
diff --git a/meta/lib/oe/recipeutils.py b/meta/lib/oe/recipeutils.py
new file mode 100644
index 0000000..2597765
--- /dev/null
+++ b/meta/lib/oe/recipeutils.py
@@ -0,0 +1,264 @@
+# Utility functions for reading and modifying recipes
+#
+# Some code borrowed from the OE layer index
+#
+# Copyright (C) 2013-2014 Intel Corporation
+#
+
+import sys
+import os
+import os.path
+import tempfile
+import textwrap
+import difflib
+import utils
+import shutil
+import re
+from collections import OrderedDict, defaultdict
+
+
+# Help us to find places to insert values
+recipe_progression = ['SUMMARY', 'DESCRIPTION', 'HOMEPAGE', 'BUGTRACKER', 'SECTION', 'LICENSE', 'LIC_FILES_CHKSUM', 'PROVIDES', 'DEPENDS', 'PR', 'PV', 'SRC_URI', 'do_fetch', 'do_unpack', 'do_patch', 'EXTRA_OECONF', 'do_configure', 'EXTRA_OEMAKE', 'do_compile', 'do_install', 'do_populate_sysroot', 'INITSCRIPT', 'USERADD', 'GROUPADD', 'PACKAGES', 'FILES', 'RDEPENDS', 'RRECOMMENDS', 'RSUGGESTS', 'RPROVIDES', 'RREPLACES', 'RCONFLICTS', 'ALLOW_EMPTY', 'do_package', 'do_deploy']
+# Variables that sometimes are a bit long but shouldn't be wrapped
+nowrap_vars = ['SUMMARY', 'HOMEPAGE', 'BUGTRACKER', 'LIC_FILES_CHKSUM']
+meta_vars = ['SUMMARY', 'DESCRIPTION', 'HOMEPAGE', 'BUGTRACKER', 'SECTION']
+
+
+def pn_to_recipe(cooker, pn):
+ """Convert a recipe name (PN) to the path to the recipe file"""
+ import bb.providers
+
+ if pn in cooker.recipecache.pkg_pn:
+ filenames = cooker.recipecache.pkg_pn[pn]
+ best = bb.providers.findBestProvider(pn, cooker.data, cooker.recipecache, cooker.recipecache.pkg_pn)
+ return best[3]
+ else:
+ return None
+
+
+def parse_recipe(fn, d):
+ """Parse an individual recipe"""
+ import bb.cache
+ envdata = bb.cache.Cache.loadDataFull(fn, [], d)
+ return envdata
+
+
+def get_var_files(fn, varlist, d):
+ """Find the file in which each of a list of variables is set.
+ Note: requires variable history to be enabled when parsing.
+ """
+ envdata = parse_recipe(fn, d)
+ varfiles = {}
+ for v in varlist:
+ history = envdata.varhistory.variable(v)
+ files = []
+ for event in history:
+ if 'file' in event and not 'flag' in event:
+ files.append(event['file'])
+ if files:
+ actualfile = files[-1]
+ else:
+ actualfile = None
+ varfiles[v] = actualfile
+
+ return varfiles
+
+
+def patch_recipe_file(fn, values, patch=False, relpath=''):
+ """Update or insert variable values into a recipe file (assuming you
+ have already identified the exact file you want to update.)
+ Note that some manual inspection/intervention may be required
+ since this cannot handle all situations.
+ """
+ remainingnames = {}
+ for k in values.keys():
+ remainingnames[k] = recipe_progression.index(k) if k in recipe_progression else -1
+ remainingnames = OrderedDict(sorted(remainingnames.iteritems(), key=lambda x: x[1]))
+
+ with tempfile.NamedTemporaryFile('w', delete=False) as tf:
+ def outputvalue(name):
+ rawtext = '%s = "%s"\n' % (name, values[name])
+ if name in nowrap_vars:
+ tf.write(rawtext)
+ else:
+ wrapped = textwrap.wrap(rawtext)
+ for wrapline in wrapped[:-1]:
+ tf.write('%s \\\n' % wrapline)
+ tf.write('%s\n' % wrapped[-1])
+
+ tfn = tf.name
+ with open(fn, 'r') as f:
+ # First runthrough - find existing names (so we know not to insert based on recipe_progression)
+ # Second runthrough - make the changes
+ existingnames = []
+ for runthrough in [1, 2]:
+ currname = None
+ for line in f:
+ if not currname:
+ insert = False
+ for k in remainingnames.keys():
+ for p in recipe_progression:
+ if re.match('^%s[ ?:=]' % p, line):
+ if remainingnames[k] > -1 and recipe_progression.index(p) > remainingnames[k] and runthrough > 1 and not k in existingnames:
+ outputvalue(k)
+ del remainingnames[k]
+ break
+ for k in remainingnames.keys():
+ if re.match('^%s[ ?:=]' % k, line):
+ currname = k
+ if runthrough == 1:
+ existingnames.append(k)
+ else:
+ del remainingnames[k]
+ break
+ if currname and runthrough > 1:
+ outputvalue(currname)
+
+ if currname:
+ sline = line.rstrip()
+ if not sline.endswith('\\'):
+ currname = None
+ continue
+ if runthrough > 1:
+ tf.write(line)
+ f.seek(0)
+ if remainingnames:
+ tf.write('\n')
+ for k in remainingnames.keys():
+ outputvalue(k)
+
+ with open(tfn, 'U') as f:
+ tolines = f.readlines()
+ if patch:
+ with open(fn, 'U') as f:
+ fromlines = f.readlines()
+ relfn = os.path.relpath(fn, relpath)
+ diff = difflib.unified_diff(fromlines, tolines, 'a/%s' % relfn, 'b/%s' % relfn)
+ os.remove(tfn)
+ return diff
+ else:
+ with open(fn, 'w') as f:
+ f.writelines(tolines)
+ os.remove(tfn)
+ return None
+
+def localise_file_vars(fn, varfiles, varlist):
+ """Given a list of variables and variable history (fetched with get_var_files())
+ find where each variable should be set/changed. This handles for example where a
+ recipe includes an inc file where variables might be changed - in most cases
+ we want to update the inc file when changing the variable value rather than adding
+ it to the recipe itself.
+ """
+ fndir = os.path.dirname(fn) + os.sep
+
+ first_meta_file = None
+ for v in meta_vars:
+ f = varfiles.get(v, None)
+ if f:
+ actualdir = os.path.dirname(f) + os.sep
+ if actualdir.startswith(fndir):
+ first_meta_file = f
+ break
+
+ filevars = defaultdict(list)
+ for v in varlist:
+ f = varfiles[v]
+ # Only return files that are in the same directory as the recipe or in some directory below there
+ # (this excludes bbclass files and common inc files that wouldn't be appropriate to set the variable
+ # in if we were going to set a value specific to this recipe)
+ if f:
+ actualfile = f
+ else:
+ # Variable isn't in a file, if it's one of the "meta" vars, use the first file with a meta var in it
+ if first_meta_file:
+ actualfile = first_meta_file
+ else:
+ actualfile = fn
+
+ actualdir = os.path.dirname(actualfile) + os.sep
+ if not actualdir.startswith(fndir):
+ actualfile = fn
+ filevars[actualfile].append(v)
+
+ return filevars
+
+def patch_recipe(d, fn, varvalues, patch=False, relpath=''):
+ """Modify a list of variable values in the specified recipe. Handles inc files if
+ used by the recipe.
+ """
+ varlist = varvalues.keys()
+ varfiles = get_var_files(fn, varlist, d)
+ locs = localise_file_vars(fn, varfiles, varlist)
+ patches = []
+ for f,v in locs.iteritems():
+ vals = {k: varvalues[k] for k in v}
+ patchdata = patch_recipe_file(f, vals, patch, relpath)
+ if patch:
+ patches.append(patchdata)
+
+ if patch:
+ return patches
+ else:
+ return None
+
+
+
+def copy_recipe_files(d, tgt_dir, whole_dir=False, download=True):
+ """Copy (local) recipe files, including both files included via include/require,
+ and files referred to in the SRC_URI variable."""
+ import bb.fetch2
+ import oe.path
+
+ # FIXME need a warning if the unexpanded SRC_URI value contains variable references
+
+ uris = (d.getVar('SRC_URI', True) or "").split()
+ fetch = bb.fetch2.Fetch(uris, d)
+ if download:
+ fetch.download()
+
+ # Copy local files to target directory and gather any remote files
+ bb_dir = os.path.dirname(d.getVar('FILE', True)) + os.sep
+ remotes = []
+ includes = [path for path in d.getVar('BBINCLUDED', True).split() if
+ path.startswith(bb_dir) and os.path.exists(path)]
+ for path in fetch.localpaths() + includes:
+ # Only import files that are under the meta directory
+ if path.startswith(bb_dir):
+ if not whole_dir:
+ relpath = os.path.relpath(path, bb_dir)
+ subdir = os.path.join(tgt_dir, os.path.dirname(relpath))
+ if not os.path.exists(subdir):
+ os.makedirs(subdir)
+ shutil.copy2(path, os.path.join(tgt_dir, relpath))
+ else:
+ remotes.append(path)
+ # Simply copy whole meta dir, if requested
+ if whole_dir:
+ shutil.copytree(bb_dir, tgt_dir)
+
+ return remotes
+
+
+def get_recipe_patches(d):
+ """Get a list of the patches included in SRC_URI within a recipe."""
+ patchfiles = []
+ # Execute src_patches() defined in patch.bbclass - this works since that class
+ # is inherited globally
+ patches = bb.utils.exec_flat_python_func('src_patches', d)
+ for patch in patches:
+ _, _, local, _, _, parm = bb.fetch.decodeurl(patch)
+ patchfiles.append(local)
+ return patchfiles
+
+
+def validate_pn(pn):
+ """Perform validation on a recipe name (PN) for a new recipe."""
+ reserved_names = ['forcevariable', 'append', 'prepend', 'remove']
+ if not re.match('[0-9a-z-]+', pn):
+ return 'Recipe name "%s" is invalid: only characters 0-9, a-z and - are allowed' % pn
+ elif pn in reserved_names:
+ return 'Recipe name "%s" is invalid: is a reserved keyword' % pn
+ elif pn.startswith('pn-'):
+ return 'Recipe name "%s" is invalid: names starting with "pn-" are reserved' % pn
+ return ''
+
--
1.9.3
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [RFC PATCH 6/7] scripts/devtool: add development helper tool
2014-11-25 17:28 [RFC PATCH 0/7] Developer workflow improvements Paul Eggleton
` (4 preceding siblings ...)
2014-11-25 17:28 ` [RFC PATCH 5/7] lib/oe: add recipeutils module Paul Eggleton
@ 2014-11-25 17:28 ` Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 7/7] scripts/devtool: Support deploy/undeploy function Paul Eggleton
` (5 subsequent siblings)
11 siblings, 0 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-11-25 17:28 UTC (permalink / raw)
To: openembedded-core
Provides an easy means to work on developing applications and system
components with the build system.
You can start by creating a workspace layer (only needs to be done
once):
$ devtool create-workspace
Then you can use the other functions, for example to "modify" the source
for an existing recipe:
$ devtool modify -x pango /home/projects/pango
Parsing recipes..done.
INFO: Fetching pango...
INFO: Unpacking...
INFO: Patching...
INFO: Source tree extracted to /tmp/woot2/
INFO: Recipe pango now set up to build from /home/projects/pango
The pango source is now extracted to /home/projects/pango, managed in
git, with each patch as a commit, and a bbappend is created in the
workspace layer to use the source in /home/projects/pango when building.
Additionally, you can add a new piece of software:
$ devtool add pv /home/projects/pv
INFO: Recipe /path/to/workspace/recipes/pv/pv.bb has been
automatically created; further editing may be required to make it
fully functional
The latter uses recipetool to create a skeleton recipe and again sets up
a bbappend to use the source in /home/projects/pv when building.
[YOCTO #6561]
[YOCTO #6653]
[YOCTO #6656]
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
scripts/devtool | 223 +++++++++++++++++++++++
scripts/lib/devtool/__init__.py | 79 ++++++++
scripts/lib/devtool/standard.py | 390 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 692 insertions(+)
create mode 100755 scripts/devtool
create mode 100644 scripts/lib/devtool/__init__.py
create mode 100644 scripts/lib/devtool/standard.py
diff --git a/scripts/devtool b/scripts/devtool
new file mode 100755
index 0000000..eea93e0
--- /dev/null
+++ b/scripts/devtool
@@ -0,0 +1,223 @@
+#!/usr/bin/env python
+
+# OpenEmbedded Development tool
+#
+# Copyright (C) 2014 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+import sys
+import os
+import argparse
+import glob
+import ConfigParser
+import subprocess
+import logging
+
+basepath = ''
+workspace = {}
+config = None
+context = None
+
+def logger_create(name):
+ logger = logging.getLogger(name)
+ loggerhandler = logging.StreamHandler()
+ loggerhandler.setFormatter(logging.Formatter("%(levelname)s: %(message)s"))
+ logger.addHandler(loggerhandler)
+ logger.setLevel(logging.INFO)
+ return logger
+logger = logger_create('devtool')
+
+plugins = []
+
+def load_plugins(pluginpath):
+ global plugins
+
+ def load_plugin(name):
+ logger.debug('Loading plugin %s' % name)
+ fp, pathname, description = imp.find_module(name, [pluginpath])
+ try:
+ return imp.load_module(name, fp, pathname, description)
+ finally:
+ if fp:
+ fp.close()
+
+ logger.debug('Loading plugins from %s...' % pluginpath)
+ import imp
+ for fn in glob.glob(os.path.join(pluginpath, '*.py')):
+ name = os.path.splitext(os.path.basename(fn))[0]
+ if name != '__init__':
+ plugin = load_plugin(name)
+ if hasattr(plugin, 'plugin_init'):
+ plugin.plugin_init(plugins)
+ plugins.append(plugin)
+
+
+class ConfigHandler(object):
+ config_file = ''
+ config_obj = None
+ init_path = ''
+ workspace_path = ''
+
+ def __init__(self, filename):
+ self.config_file = filename
+ self.config_obj = ConfigParser.SafeConfigParser()
+
+ def get(self, section, option, default=None):
+ try:
+ ret = self.config_obj.get(section, option)
+ except (ConfigParser.NoOptionError, ConfigParser.NoSectionError):
+ if default != None:
+ ret = default
+ else:
+ raise
+ return ret
+
+ def read(self):
+ if os.path.exists(self.config_file):
+ self.config_obj.read(self.config_file)
+
+ if self.config_obj.has_option('General', 'init_path'):
+ pth = self.get('General', 'init_path')
+ self.init_path = os.path.join(basepath, pth)
+ if not os.path.exists(self.init_path):
+ logger.error('init_path %s specified in config file cannot be found' % pth)
+ return False
+
+ self.workspace_path = self.get('General', 'workspace_path', os.path.join(basepath, 'workspace'))
+
+ return True
+ else:
+ self.config_obj.add_section('General')
+ return True
+
+ def write(self):
+ self.config_obj.set('General', 'workspace_path', self.workspace_path)
+ print self.config_file
+ with open(self.config_file, 'w') as f:
+ self.config_obj.write(f)
+
+class Context:
+ def __init__(self, **kwargs):
+ self.__dict__.update(kwargs)
+
+
+def read_workspace():
+ global workspace
+ workspace = {}
+ if not os.path.exists(os.path.join(config.workspace_path, 'conf', 'layer.conf')):
+ if context.fixed_setup:
+ logger.error("workspace layer not set up")
+ else:
+ logger.error("workspace layer not set up - you can create one by running %s create-workspace" % os.path.basename(sys.argv[0]))
+ sys.exit(1)
+
+ logger.debug('Reading workspace in %s' % config.workspace_path)
+ for fn in glob.glob(os.path.join(config.workspace_path, 'appends', '*.bbappend')):
+ pn = os.path.splitext(os.path.basename(fn))[0].split('_')[0]
+ with open(fn, 'r') as f:
+ for line in f:
+ if line.startswith('EXTERNALSRC ='):
+ splitval = line.split('=', 2)
+ workspace[pn] = splitval[1].strip('" \n\r\t')
+ break
+
+def main():
+ global basepath
+ global config
+ global context
+
+ context = Context(fixed_setup=False)
+
+ # Default basepath
+ basepath = os.path.dirname(os.path.abspath(__file__))
+ pth = basepath
+ while pth != '' and pth != os.sep:
+ if os.path.exists(os.path.join(pth, '.devtoolbase')):
+ context.fixed_setup = True
+ basepath = pth
+ break
+ pth = os.path.dirname(pth)
+
+ scripts_path = os.path.dirname(os.path.realpath(__file__))
+ lib_path = scripts_path + '/lib'
+ sys.path = sys.path + [lib_path]
+
+ parser = argparse.ArgumentParser(description="OpenEmbedded development tool")
+ parser.add_argument('--basepath', help='Base directory of SDK / build directory')
+ parser.add_argument('-d', '--debug', help='Enable debug output', action='store_true')
+ parser.add_argument('-q', '--quiet', help='Print only errors', action='store_true')
+
+ subparsers = parser.add_subparsers(dest="subparser_name")
+
+ load_plugins(os.path.join(scripts_path, 'lib', 'devtool'))
+ for plugin in plugins:
+ if hasattr(plugin, 'register_commands'):
+ plugin.register_commands(subparsers, context)
+
+ args = parser.parse_args()
+
+ if args.debug:
+ logger.setLevel(logging.DEBUG)
+ elif args.quiet:
+ logger.setLevel(logging.ERROR)
+
+ if args.basepath:
+ # Override
+ basepath = args.basepath
+ elif not context.fixed_setup:
+ basepath = os.environ.get('BUILDDIR')
+ if not basepath:
+ logger.error("This script can only be run after initialising the build environment (e.g. by using oe-init-build-env)")
+ sys.exit(1)
+
+ logger.debug('Using basepath %s' % basepath)
+
+ config = ConfigHandler(os.path.join(basepath, 'devtool.conf'))
+ if not config.read():
+ return -1
+
+ bitbake_subdir = config.get('General', 'bitbake_subdir', '')
+ if bitbake_subdir:
+ # Normally set for use within the SDK
+ logger.debug('Using bitbake subdir %s' % bitbake_subdir)
+ sys.path.insert(0, os.path.join(basepath, bitbake_subdir, 'lib'))
+ core_meta_subdir = config.get('General', 'core_meta_subdir')
+ sys.path.insert(0, os.path.join(basepath, core_meta_subdir, 'lib'))
+ else:
+ # Standard location
+ import scriptpath
+ bitbakepath = scriptpath.add_bitbake_lib_path()
+ if not bitbakepath:
+ logger.error("Unable to find bitbake by searching parent directory of this script or PATH")
+ sys.exit(1)
+ logger.debug('Using standard bitbake path %s' % bitbakepath)
+ scriptpath.add_oe_lib_path()
+
+ if args.subparser_name != 'create-workspace':
+ read_workspace()
+
+ ret = args.func(args, config, basepath, workspace)
+
+ return ret
+
+
+if __name__ == "__main__":
+ try:
+ ret = main()
+ except Exception:
+ ret = 1
+ import traceback
+ traceback.print_exc(5)
+ sys.exit(ret)
diff --git a/scripts/lib/devtool/__init__.py b/scripts/lib/devtool/__init__.py
new file mode 100644
index 0000000..3cad708
--- /dev/null
+++ b/scripts/lib/devtool/__init__.py
@@ -0,0 +1,79 @@
+#!/usr/bin/env python
+
+# Development tool - utility functions for plugins
+#
+# Copyright (C) 2014 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+import os
+import sys
+import subprocess
+import logging
+
+logger = logging.getLogger('devtool')
+
+def exec_build_env_command(init_path, builddir, cmd, watch=False, **options):
+ import bb
+ if not 'cwd' in options:
+ options["cwd"] = builddir
+ if init_path:
+ logger.debug('Executing command: "%s" using init path %s' % (cmd, init_path))
+ init_prefix = '. %s %s > /dev/null && ' % (init_path, builddir)
+ else:
+ logger.debug('Executing command "%s"' % cmd)
+ init_prefix = ''
+ if watch:
+ return exec_watch('%s%s' % (init_prefix, cmd), **options)
+ else:
+ return bb.process.run('%s%s' % (init_prefix, cmd), **options)
+
+def exec_watch(cmd, **options):
+ if isinstance(cmd, basestring) and not "shell" in options:
+ options["shell"] = True
+
+ process = subprocess.Popen(
+ cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, **options
+ )
+
+ buf = ''
+ while True:
+ out = process.stdout.read(1)
+ if out:
+ sys.stdout.write(out)
+ sys.stdout.flush()
+ buf += out
+ elif out == '' and process.poll() != None:
+ break
+ return buf
+
+def setup_tinfoil():
+ scripts_path = os.path.abspath(os.path.dirname(os.path.abspath(sys.argv[0])))
+ lib_path = scripts_path + '/lib'
+ sys.path = sys.path + [lib_path]
+
+ import scriptpath
+ bitbakepath = scriptpath.add_bitbake_lib_path()
+ if not bitbakepath:
+ logger.error("Unable to find bitbake by searching parent directory of this script or PATH")
+ sys.exit(1)
+
+ import bb.tinfoil
+ import logging
+ tinfoil = bb.tinfoil.Tinfoil()
+ tinfoil.prepare(False)
+ tinfoil.logger.setLevel(logging.WARNING)
+ return tinfoil
+
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
new file mode 100644
index 0000000..82c5b53
--- /dev/null
+++ b/scripts/lib/devtool/standard.py
@@ -0,0 +1,390 @@
+# Development tool - standard commands plugin
+#
+# Copyright (C) 2014 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+import os
+import sys
+import re
+import shutil
+import glob
+import tempfile
+import logging
+from devtool import exec_build_env_command, setup_tinfoil
+
+logger = logging.getLogger('devtool')
+
+def plugin_init(pluginlist):
+ pass
+
+
+def create_workspace(args, config, basepath, workspace):
+ import bb
+
+ if args.directory:
+ workspacedir = os.path.abspath(args.directory)
+ else:
+ workspacedir = os.path.abspath(os.path.join(basepath, 'workspace'))
+
+ confdir = os.path.join(workspacedir, 'conf')
+ if os.path.exists(os.path.join(confdir, 'layer.conf')):
+ logger.info('Specified workspace already set up, leaving as-is')
+ else:
+ bb.utils.mkdirhier(confdir)
+ with open(os.path.join(confdir, 'layer.conf'), 'w') as f:
+ f.write('# ### workspace layer auto-generated by devtool ###\n')
+ f.write('BBPATH =. "$' + '{LAYERDIR}:"\n')
+ f.write('BBFILES += "$' + '{LAYERDIR}/recipes/*/*.bb \\\n')
+ f.write(' $' + '{LAYERDIR}/appends/*.bbappend"\n')
+ f.write('BBFILE_COLLECTIONS += "workspacelayer"\n')
+ f.write('BBFILE_PATTERN_workspacelayer = "^$' + '{LAYERDIR}/"\n')
+ f.write('BBFILE_PRIORITY_workspacelayer = "99"\n')
+ if not args.create_only:
+ # Add the workspace layer to bblayers.conf
+ bblayers_conf = os.path.join(basepath, 'conf', 'bblayers.conf')
+ if not os.path.exists(bblayers_conf):
+ logger.error('Unable to find bblayers.conf')
+ return -1
+ newlines = []
+ with open(bblayers_conf, 'r') as f:
+ bblayers_found = False
+ updated = False
+ in_bblayers = False
+ bblayers = ''
+ for line in f:
+ if in_bblayers:
+ value = line.rstrip()
+ bblayers += value[:-1]
+ if value.endswith('"'):
+ in_bblayers = False
+ bblayers = bblayers.split()
+ if workspacedir not in bblayers:
+ updated = True
+ bblayers.append(workspacedir)
+ if workspacedir != config.workspace_path and config.workspace_path in bblayers:
+ bblayers.remove(config.workspace_path)
+ newlines.append('BBLAYERS ?= " \\\n')
+ for layer in bblayers:
+ newlines.append(' %s \\\n' % layer)
+ newlines.append(' "\n')
+ else:
+ if line.startswith('BBLAYERS '):
+ bblayers_found = True
+ in_bblayers = True
+ value = line.split('"', 1)[1].rstrip()
+ if value.endswith('\\'):
+ value = value[:-1]
+ bblayers = value
+ else:
+ newlines.append(line)
+ if not bblayers_found:
+ logger.error('Found bblayers.conf but could not find BBLAYERS value to update')
+ return -1
+ if updated:
+ # Write out the new bblayers.conf
+ with open(bblayers_conf, 'w') as f:
+ f.writelines(newlines)
+ if config.workspace_path != workspacedir:
+ # Update our config to point to the new location
+ config.workspace_path = workspacedir
+ config.write()
+
+
+def add(args, config, basepath, workspace):
+ import bb
+ import oe.recipeutils
+
+ if args.recipename in workspace:
+ logger.error("recipe %s is already in your workspace" % args.recipename)
+ return -1
+
+ reason = oe.recipeutils.validate_pn(args.recipename)
+ if reason:
+ logger.error(reason)
+ return -1
+
+ srctree = os.path.abspath(args.srctree)
+ appendpath = os.path.join(config.workspace_path, 'appends')
+ if not os.path.exists(appendpath):
+ os.makedirs(appendpath)
+
+ recipedir = os.path.join(config.workspace_path, 'recipes', args.recipename)
+ bb.utils.mkdirhier(recipedir)
+ if args.version:
+ if '_' in args.version or ' ' in args.version:
+ logger.error('Invalid version string "%s"' % args.version)
+ return -1
+ bp = "%s_%s" % (args.recipename, args.version)
+ else:
+ bp = args.recipename
+ recipefile = os.path.join(recipedir, "%s.bb" % bp)
+ stdout, stderr = exec_build_env_command(config.init_path, basepath, 'recipetool create -o %s %s' % (recipefile, srctree))
+ logger.info('Recipe %s has been automatically created; further editing may be required to make it fully functional' % recipefile)
+
+ _add_md5(config, args.recipename, recipefile)
+
+ appendfile = os.path.join(appendpath, '%s.bbappend' % args.recipename)
+ with open(appendfile, 'w') as f:
+ f.write('inherit externalsrc\n')
+ f.write('EXTERNALSRC = "%s"\n' % srctree)
+ f.write('do_compile[nostamp] = "1"\n')
+
+ _add_md5(config, args.recipename, appendfile)
+
+ return 0
+
+
+def extract(args, config, basepath, workspace):
+ import bb
+ import oe.recipeutils
+
+ tinfoil = setup_tinfoil()
+
+ recipefile = oe.recipeutils.pn_to_recipe(tinfoil.cooker, args.recipename)
+ if not recipefile:
+ logger.error("Unable to find any recipe file matching %s" % args.recipename)
+ return -1
+ rd = oe.recipeutils.parse_recipe(recipefile, tinfoil.config_data)
+
+ srctree = os.path.abspath(args.srctree)
+ return _extract_source(srctree, rd)
+
+
+def _extract_source(srctree, d):
+ pn = d.getVar('PN', True)
+
+ if os.path.exists(srctree):
+ if not os.path.isdir(srctree):
+ logger.error("output path %s exists and is not a directory" % srctree)
+ return -1
+ elif os.listdir(srctree):
+ logger.error("output path %s already exists and is non-empty" % srctree)
+ return -1
+
+ # Prepare for shutil.move later on
+ bb.utils.mkdirhier(srctree)
+ os.rmdir(srctree)
+
+ tempdir = tempfile.mkdtemp(prefix='devtool')
+ try:
+ crd = d.createCopy()
+ # Make a subdir so we guard against WORKDIR==S
+ workdir = os.path.join(tempdir, 'workdir')
+ crd.setVar('WORKDIR', workdir)
+ crd.setVar('T', os.path.join(tempdir, 'temp'))
+ logger.info('Fetching %s...' % pn)
+ bb.build.exec_func('do_fetch', crd)
+ logger.info('Unpacking...')
+ bb.build.exec_func('do_unpack', crd)
+ srcsubdir = crd.getVar('S', True)
+ if srcsubdir != workdir and os.path.dirname(srcsubdir) != workdir:
+ # Handle if S is set to a subdirectory of the source
+ srcsubdir = os.path.join(workdir, os.path.relpath(srcsubdir, workdir).split(os.sep)[0])
+
+ patchdir = os.path.join(srcsubdir, 'patches')
+ haspatches = False
+ if os.path.exists(patchdir):
+ if os.listdir(patchdir):
+ haspatches = True
+ else:
+ os.rmdir(patchdir)
+
+ if not os.listdir(srcsubdir):
+ logger.error("no source unpacked to S, perhaps the %s recipe doesn't use any source?" % pn)
+ return -1
+
+ if not os.path.exists(os.path.join(srcsubdir, '.git')):
+ bb.process.run('git init', cwd=srcsubdir)
+ bb.process.run('git add .', cwd=srcsubdir)
+ bb.process.run('git commit -q -m "Initial commit from upstream at version %s"' % crd.getVar('PV', True), cwd=srcsubdir)
+ logger.info('Patching...')
+ crd.setVar('PATCHTOOL', 'git')
+ bb.build.exec_func('do_patch', crd)
+
+ if os.path.exists(patchdir):
+ shutil.rmtree(patchdir)
+ if haspatches:
+ bb.process.run('git checkout patches', cwd=srcsubdir)
+
+ shutil.move(srcsubdir, srctree)
+ logger.info('Source tree extracted to %s' % srctree)
+ finally:
+ shutil.rmtree(tempdir)
+ return 0
+
+def _add_md5(config, recipename, filename):
+ import bb.utils
+ md5 = bb.utils.md5_file(filename)
+ with open(os.path.join(config.workspace_path, '.devtool_md5'), 'a') as f:
+ f.write('%s|%s|%s\n' % (recipename, os.path.relpath(filename, config.workspace_path), md5))
+
+def _check_preserve(config, recipename):
+ import bb.utils
+ origfile = os.path.join(config.workspace_path, '.devtool_md5')
+ newfile = os.path.join(config.workspace_path, '.devtool_md5_new')
+ preservepath = os.path.join(config.workspace_path, 'attic')
+ with open(origfile, 'r') as f:
+ with open(newfile, 'w') as tf:
+ for line in f.readlines():
+ splitline = line.rstrip().split('|')
+ if splitline[0] == recipename:
+ removefile = os.path.join(config.workspace_path, splitline[1])
+ md5 = bb.utils.md5_file(removefile)
+ if splitline[2] != md5:
+ bb.utils.mkdirhier(preservepath)
+ preservefile = os.path.basename(removefile)
+ logger.warn('File %s modified since it was written, preserving in %s' % (preservefile, preservepath))
+ shutil.move(removefile, os.path.join(preservepath, preservefile))
+ else:
+ os.remove(removefile)
+ else:
+ tf.write(line)
+ os.rename(newfile, origfile)
+
+ return False
+
+
+def modify(args, config, basepath, workspace):
+ import bb
+ import oe.recipeutils
+
+ if args.recipename in workspace:
+ logger.error("recipe %s is already in your workspace" % args.recipename)
+ return -1
+
+ if not args.extract:
+ if not os.path.isdir(args.srctree):
+ logger.error("directory %s does not exist or not a directory (specify -x to extract source from recipe)" % args.srctree)
+ return -1
+
+ tinfoil = setup_tinfoil()
+
+ recipefile = oe.recipeutils.pn_to_recipe(tinfoil.cooker, args.recipename)
+ if not recipefile:
+ logger.error("Unable to find any recipe file matching %s" % args.recipename)
+ return -1
+ rd = oe.recipeutils.parse_recipe(recipefile, tinfoil.config_data)
+
+ srctree = os.path.abspath(args.srctree)
+ if args.extract:
+ ret = _extract_source(args.srctree, rd)
+ if ret:
+ return ret
+
+ # Handle if S is set to a subdirectory of the source
+ s = rd.getVar('S', True)
+ workdir = rd.getVar('WORKDIR', True)
+ if s != workdir and os.path.dirname(s) != workdir:
+ srcsubdir = os.sep.join(os.path.relpath(s, workdir).split(os.sep)[1:])
+ srctree = os.path.join(srctree, srcsubdir)
+
+ appendpath = os.path.join(config.workspace_path, 'appends')
+ if not os.path.exists(appendpath):
+ os.makedirs(appendpath)
+
+ appendname = os.path.splitext(os.path.basename(recipefile))[0]
+ if args.wildcard:
+ appendname = re.sub(r'_.*', '_%', appendname)
+ appendfile = os.path.join(appendpath, appendname + '.bbappend')
+ with open(appendfile, 'w') as f:
+ f.write('FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"\n\n')
+ f.write('inherit externalsrc\n')
+ f.write('EXTERNALSRC = "%s"\n' % srctree)
+ if bb.data.inherits_class('autotools-brokensep', rd):
+ logger.info('using source tree as build directory since original recipe inherits autotools-brokensep')
+ f.write('EXTERNALSRC_BUILD = "%s"\n' % srctree)
+ f.write('do_compile[nostamp] = "1"\n')
+
+ _add_md5(config, args.recipename, appendfile)
+
+ logger.info('Recipe %s now set up to build from %s' % (args.recipename, srctree))
+
+ return 0
+
+
+def status(args, config, basepath, workspace):
+ for recipe, value in workspace.iteritems():
+ print("%s: %s" % (recipe, value))
+ return 0
+
+
+def reset(args, config, basepath, workspace):
+ import bb.utils
+ if not args.recipename in workspace:
+ logger.error("no recipe named %s in your workspace" % args.recipename)
+ return -1
+ _check_preserve(config, args.recipename)
+
+ preservepath = os.path.join(config.workspace_path, 'attic', args.recipename)
+ def preservedir(origdir):
+ if os.path.exists(origdir):
+ for fn in os.listdir(origdir):
+ logger.warn('Preserving %s in %s' % (fn, preservepath))
+ bb.utils.mkdirhier(preservepath)
+ shutil.move(os.path.join(origdir, fn), os.path.join(preservepath, fn))
+ os.rmdir(origdir)
+
+ preservedir(os.path.join(config.workspace_path, 'recipes', args.recipename))
+ # We don't automatically create this dir next to appends, but the user can
+ preservedir(os.path.join(config.workspace_path, 'appends', args.recipename))
+ return 0
+
+
+def build(args, config, basepath, workspace):
+ import bb
+ if not args.recipename in workspace:
+ logger.error("no recipe named %s in your workspace" % args.recipename)
+ return -1
+ exec_build_env_command(config.init_path, basepath, 'bitbake -c install %s' % args.recipename, watch=True)
+
+ return 0
+
+
+def register_commands(subparsers, context):
+ if not context.fixed_setup:
+ parser_create_workspace = subparsers.add_parser('create-workspace', help='Set up a workspace')
+ parser_create_workspace.add_argument('directory', nargs='?', help='Directory for the workspace')
+ parser_create_workspace.add_argument('--create-only', action="store_true", help='Only create the workspace, do not alter configuration')
+ parser_create_workspace.set_defaults(func=create_workspace)
+
+ parser_add = subparsers.add_parser('add', help='Add a new recipe')
+ parser_add.add_argument('recipename', help='Name for new recipe to add')
+ parser_add.add_argument('srctree', help='Path to external source tree')
+ parser_add.add_argument('--version', '-V', help='Version to use within recipe (PV)')
+ parser_add.set_defaults(func=add)
+
+ parser_add = subparsers.add_parser('modify', help='Modify the source for an existing recipe')
+ parser_add.add_argument('recipename', help='Name for recipe to edit')
+ parser_add.add_argument('srctree', help='Path to external source tree')
+ parser_add.add_argument('--wildcard', '-w', action="store_true", help='Use wildcard for unversioned bbappend')
+ parser_add.add_argument('--extract', '-x', action="store_true", help='Extract source as well')
+ parser_add.set_defaults(func=modify)
+
+ parser_add = subparsers.add_parser('extract', help='Extract the source for an existing recipe')
+ parser_add.add_argument('recipename', help='Name for recipe to extract the source for')
+ parser_add.add_argument('srctree', help='Path to where to extract the source tree')
+ parser_add.set_defaults(func=extract)
+
+ parser_status = subparsers.add_parser('status', help='Show status')
+ parser_status.set_defaults(func=status)
+
+ parser_build = subparsers.add_parser('build', help='Build recipe')
+ parser_build.add_argument('recipename', help='Recipe to build')
+ parser_build.set_defaults(func=build)
+
+ parser_reset = subparsers.add_parser('reset', help='Remove a recipe from your workspace')
+ parser_reset.add_argument('recipename', help='Recipe to reset')
+ parser_reset.set_defaults(func=reset)
+
--
1.9.3
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [RFC PATCH 7/7] scripts/devtool: Support deploy/undeploy function
2014-11-25 17:28 [RFC PATCH 0/7] Developer workflow improvements Paul Eggleton
` (5 preceding siblings ...)
2014-11-25 17:28 ` [RFC PATCH 6/7] scripts/devtool: add development helper tool Paul Eggleton
@ 2014-11-25 17:28 ` Paul Eggleton
2014-11-25 17:51 ` [RFC PATCH 0/7] Developer workflow improvements Paul Barker
` (4 subsequent siblings)
11 siblings, 0 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-11-25 17:28 UTC (permalink / raw)
To: openembedded-core
From: Junchun Guan <junchunx.guan@intel.com>
Deploy recipe output files to live target machine using scp
Store the files list and target machine info in localhost if deployment
is done
Undeploy recipe output files in target machine using the previous
deployment info
[YOCTO #6654]
Signed-off-by: Junchun Guan <junchunx.guan@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
scripts/lib/devtool/deploy.py | 100 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 100 insertions(+)
create mode 100644 scripts/lib/devtool/deploy.py
diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py
new file mode 100644
index 0000000..965179c
--- /dev/null
+++ b/scripts/lib/devtool/deploy.py
@@ -0,0 +1,100 @@
+# Development tool - deploy/undeploy command plugin
+#
+# Copyright (C) 2014 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+import os
+import subprocess
+import logging
+from devtool import exec_build_env_command
+
+logger = logging.getLogger('devtool')
+
+def plugin_init(pluginlist):
+ pass
+
+
+def deploy(args, config, basepath, workspace):
+ import re
+ from devtool import exec_build_env_command
+
+ if not args.recipename in workspace:
+ logger.error("no recipe named %s in your workspace" % args.recipename)
+ return -1
+ try:
+ host, destdir = args.target.split(':')
+ except ValueError:
+ destdir = '/'
+ else:
+ args.target = host
+
+ deploy_dir = os.path.join(basepath, 'target_deploy', args.target)
+ deploy_file = os.path.join(deploy_dir, args.recipename + '.list')
+
+ if os.path.exists(deploy_file):
+ undeploy(args)
+
+ stdout, stderr = exec_build_env_command(config.init_path, basepath, 'bitbake -e %s' % args.recipename, shell=True)
+ recipe_outdir = re.search(r'^D="(.*)"', stdout, re.MULTILINE).group(1)
+ ret = subprocess.call('scp -qr %s/* %s:%s' % (recipe_outdir, args.target, destdir), shell=True)
+ if ret != 0:
+ return ret
+
+ logger.info('Successfully deployed %s' % recipe_outdir)
+
+ if not os.path.exists(deploy_dir):
+ os.makedirs(deploy_dir)
+
+ files_list = []
+ for root, _, files in os.walk(recipe_outdir):
+ for filename in files:
+ filename = os.path.relpath(os.path.join(root, filename), recipe_outdir)
+ files_list.append(os.path.join(destdir, filename))
+
+ with open(deploy_file, 'w') as fobj:
+ fobj.write('\n'.join(files_list))
+
+ return 0
+
+def undeploy(args, config, basepath, workspace):
+
+ deploy_file = os.path.join(basepath, 'target_deploy', args.target, args.recipename + '.list')
+ if not os.path.exists(deploy_file):
+ logger.error('%s has not been deployed' % args.recipename)
+ return -1
+
+ ret = subprocess.call("scp -q %s %s:/tmp" % (deploy_file, args.target), shell=True)
+ if ret != 0:
+ logger.error('Failed to copy %s to %s' % (deploy, args.target))
+ return -1
+
+ ret = subprocess.call("ssh %s 'xargs -n1 rm -f </tmp/%s'" % (args.target, os.path.basename(deploy_file)), shell=True)
+ if ret == 0:
+ logger.info('Successfully undeployed %s' % args.recipename)
+ os.remove(deploy_file)
+
+ return ret
+
+
+def register_commands(subparsers, context):
+ parser_deploy = subparsers.add_parser('deploy', help='Deploy recipe output files to live target machine')
+ parser_deploy.add_argument('recipename', help='Recipe to deploy')
+ parser_deploy.add_argument('target', help='Live target machine running an ssh server: user@hostname[:destdir]')
+ parser_deploy.set_defaults(func=deploy)
+
+ parser_undeploy = subparsers.add_parser('undeploy', help='Undeploy recipe output files in live target machine')
+ parser_undeploy.add_argument('recipename', help='Recipe to undeploy')
+ parser_undeploy.add_argument('target', help='Live target machine running an ssh server: user@hostname')
+ parser_undeploy.set_defaults(func=undeploy)
--
1.9.3
^ permalink raw reply related [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 1/7] lib/oe/patch: fall back to patch if git apply fails
2014-11-25 17:28 ` [RFC PATCH 1/7] lib/oe/patch: fall back to patch if git apply fails Paul Eggleton
@ 2014-11-25 17:40 ` Paul Barker
2014-11-25 17:58 ` Paul Eggleton
0 siblings, 1 reply; 40+ messages in thread
From: Paul Barker @ 2014-11-25 17:40 UTC (permalink / raw)
To: Paul Eggleton; +Cc: OE Core
On 25 November 2014 at 17:28, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> When PATCHTOOL = "git", git apply doesn't support fuzzy application, so
> if a patch requires that it's better to be able to apply it rather than
> just failing.
>
> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> ---
> meta/lib/oe/patch.py | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
> index b085c9d..788f465 100644
> --- a/meta/lib/oe/patch.py
> +++ b/meta/lib/oe/patch.py
> @@ -219,7 +219,11 @@ class GitApplyTree(PatchTree):
> return _applypatchhelper(shellcmd, patch, force, reverse, run)
> except CmdError:
> shellcmd = ["git", "--git-dir=.", "apply", "-p%s" % patch['strippath']]
> - return _applypatchhelper(shellcmd, patch, force, reverse, run)
> + try:
> + output = _applypatchhelper(shellcmd, patch, force, reverse, run)
> + except CmdError:
> + output = PatchTree._applypatch(self, patch, force, reverse, run)
> + return output
>
Would this give a warning or other notification if the fallback to
'patch' is used? When developing patches it'd probably be good to know
whether they're exactly correct or not.
I haven't looked at _applypatchhelper or _applypatch so I'm not sure
if there's some notification hidden in there.
Cheers,
--
Paul Barker
Email: paul@paulbarker.me.uk
http://www.paulbarker.me.uk
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-11-25 17:28 [RFC PATCH 0/7] Developer workflow improvements Paul Eggleton
` (6 preceding siblings ...)
2014-11-25 17:28 ` [RFC PATCH 7/7] scripts/devtool: Support deploy/undeploy function Paul Eggleton
@ 2014-11-25 17:51 ` Paul Barker
2014-11-25 18:34 ` Paul Eggleton
2014-11-25 19:56 ` Trevor Woerner
` (3 subsequent siblings)
11 siblings, 1 reply; 40+ messages in thread
From: Paul Barker @ 2014-11-25 17:51 UTC (permalink / raw)
To: Paul Eggleton; +Cc: OE Core
On 25 November 2014 at 17:28, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> [Note: this is an RFC series for the moment, and shouldn't yet be merged.]
>
> I've been looking at how to make it easier for application and system
> component developers to get their work done using the tools we provide,
> and I believe this patchset is a piece of the solution. There's still a
> number of other pieces to come, but this should be usable on its own.
>
> The first three patches extend the PATCHTOOL = "git" code to ensure that
> we're able to apply all patches even if "git am" and "git apply" can't
> handle them, and that we do a commit to the repository per patch. This
> is needed for devtool later on.
>
> I've then added a new recipe auto-creation script, recipetool, which can
> take a source tree or URL and create a skeleton recipe to build it.
> (In case anyone is wondering about the existing scripts/create-recipe,
> frankly I consider it a dead end - it's written in Perl, which makes it
> a bit difficult to integrate with the rest of our code; it's also
> GPLv3-only which makes any such integration pretty much impossible from
> another angle.)
>
> Then we add devtool. This allows you to:
>
> * Add a new piece of software (auto-create the skeleton of a recipe
> using the aforementioned recipetool and point the build to an external
> source tree)
> * Modify the source for an existing recipe (point the build to an
> external source tree, possibly creating that tree in the same step
> and managing it with git)
> * Deploy the installed files from a recipe from ${D} to a target using
> scp.
>
> There will obviously be extensions to these tools, but I hope they are
> already functional enough to be useful at the state they are in at the
> moment.
>
> Known issues:
>
> * "devtool modify" can't currently deal with gcc, linux-yocto or perf
> because they manage their workdir differently, this will need to be
> fixed
> * devtool's workspace layer triggers a bitbake warning when empty.
> I suspect a mechanism to disable this warning internally will need to
> be added.
> * "recipetool create" ideally needs to become smarter and fill in
> more details of the recipe. At some point we'll probably have to
> make the process interactive and possibly have it automate parts of
> the build process and examine the output; some things just can't be
> practically detected from the source tree without building.
>
>
> (Note that some pieces of the code which are not essential rely on a couple
> of changes to bitbake, you can find these in the paule/devtool-bb branch
> in poky-contrib, or if you are happy to use poky, grab the paule/devtool
> branch in poky-contrib which has everything).
>
> Comments/questions/suggestions welcome!
>
This sounds great and looks like it will make some regular tasks quite
a bit easier!
One thing which may be useful within recipetool or devtool is a single
command to create patch files (using `git format-patch`), put them
into the relevant directory and add the file names to SRC_URI.
Obviously this assumes the external source tree is a git repo and we
can trace the local modifications back to either an external master
branch or an initial state after extracting a source archive. I'm just
thinking out loud here though, depends if other people think that work
flow is common enough to be worth creating a shortcut for.
Thanks,
--
Paul Barker
Email: paul@paulbarker.me.uk
http://www.paulbarker.me.uk
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 1/7] lib/oe/patch: fall back to patch if git apply fails
2014-11-25 17:40 ` Paul Barker
@ 2014-11-25 17:58 ` Paul Eggleton
0 siblings, 0 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-11-25 17:58 UTC (permalink / raw)
To: Paul Barker; +Cc: OE Core
Hi Paul,
On Tuesday 25 November 2014 17:40:24 Paul Barker wrote:
> On 25 November 2014 at 17:28, Paul Eggleton
>
> <paul.eggleton@linux.intel.com> wrote:
> > When PATCHTOOL = "git", git apply doesn't support fuzzy application, so
> > if a patch requires that it's better to be able to apply it rather than
> > just failing.
> >
> > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> > ---
> >
> > meta/lib/oe/patch.py | 6 +++++-
> > 1 file changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
> > index b085c9d..788f465 100644
> > --- a/meta/lib/oe/patch.py
> > +++ b/meta/lib/oe/patch.py
> >
> > @@ -219,7 +219,11 @@ class GitApplyTree(PatchTree):
> > return _applypatchhelper(shellcmd, patch, force, reverse,
> > run)
> >
> > except CmdError:
> > shellcmd = ["git", "--git-dir=.", "apply", "-p%s" %
> > patch['strippath']]
> >
> > - return _applypatchhelper(shellcmd, patch, force, reverse,
> > run)
> > + try:
> > + output = _applypatchhelper(shellcmd, patch, force,
> > reverse, run) + except CmdError:
> > + output = PatchTree._applypatch(self, patch, force,
> > reverse, run) + return output
>
> Would this give a warning or other notification if the fallback to
> 'patch' is used? When developing patches it'd probably be good to know
> whether they're exactly correct or not.
In the context I'm attempting to use it I'd want it not to warn in this case,
but I can see that in normal usage if you've explicitly elected to use
PATCHTOOL = "git" it could be that you do expect all patches for the recipe to
apply that way and want to see a warning if they don't.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-11-25 17:51 ` [RFC PATCH 0/7] Developer workflow improvements Paul Barker
@ 2014-11-25 18:34 ` Paul Eggleton
0 siblings, 0 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-11-25 18:34 UTC (permalink / raw)
To: Paul Barker; +Cc: OE Core
On Tuesday 25 November 2014 17:51:05 Paul Barker wrote:
> On 25 November 2014 at 17:28, Paul Eggleton
> <paul.eggleton@linux.intel.com> wrote:
> > I've been looking at how to make it easier for application and system
> > component developers to get their work done using the tools we provide,
> > and I believe this patchset is a piece of the solution. There's still a
> > number of other pieces to come, but this should be usable on its own.
> > ...
> > Comments/questions/suggestions welcome!
>
> This sounds great and looks like it will make some regular tasks quite
> a bit easier!
>
> One thing which may be useful within recipetool or devtool is a single
> command to create patch files (using `git format-patch`), put them
> into the relevant directory and add the file names to SRC_URI.
> Obviously this assumes the external source tree is a git repo and we
> can trace the local modifications back to either an external master
> branch or an initial state after extracting a source archive. I'm just
> thinking out loud here though, depends if other people think that work
> flow is common enough to be worth creating a shortcut for.
We've been thinking in this direction and it seems like a natural extension to
me, and one that I do hope to add. The tricky part is as you say, reconciling
the commits you have in the git tree with the patches you have in the recipe,
particularly when the user might have modified one or more of those commits.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-11-25 17:28 [RFC PATCH 0/7] Developer workflow improvements Paul Eggleton
` (7 preceding siblings ...)
2014-11-25 17:51 ` [RFC PATCH 0/7] Developer workflow improvements Paul Barker
@ 2014-11-25 19:56 ` Trevor Woerner
2014-11-26 9:02 ` Paul Eggleton
2014-11-28 17:28 ` Trevor Woerner
` (2 subsequent siblings)
11 siblings, 1 reply; 40+ messages in thread
From: Trevor Woerner @ 2014-11-25 19:56 UTC (permalink / raw)
To: Paul Eggleton, openembedded-core
Wow, this is exciting!
Just thinking out loud here... is there a way to way to have the
devshell come up without having applied any patches but with the patches
already queued in, say, a quilt series?
It seems to happen often enough to me that: a recipe has 10 local
patches which applied fine yesterday but now patch 7 is failing because
the upstream developers changed something. I would be nice to be able to
devshell to the point where patch 6 has been applied and patch 7 is
waiting to go on. Then I could apply the patch myself to figure out why
it's failing. Or just having all the patches ready to apply (in a quilt
series, as mentioned above) would be great too.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-11-25 19:56 ` Trevor Woerner
@ 2014-11-26 9:02 ` Paul Eggleton
0 siblings, 0 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-11-26 9:02 UTC (permalink / raw)
To: Trevor Woerner; +Cc: openembedded-core
Hi Trevor,
On Tuesday 25 November 2014 14:56:14 Trevor Woerner wrote:
> Just thinking out loud here... is there a way to way to have the
> devshell come up without having applied any patches but with the patches
> already queued in, say, a quilt series?
>
> It seems to happen often enough to me that: a recipe has 10 local
> patches which applied fine yesterday but now patch 7 is failing because
> the upstream developers changed something. I would be nice to be able to
> devshell to the point where patch 6 has been applied and patch 7 is
> waiting to go on. Then I could apply the patch myself to figure out why
> it's failing. Or just having all the patches ready to apply (in a quilt
> series, as mentioned above) would be great too.
We do already have PATCHRESOLVE = "user" that's supposed to handle exactly
this, although to be honest I've never really felt comfortable using it (that
might have something to do with my lack of familiarity with quilt, though).
devtool is using the same code that we use for applying patches, which
supports quilt, so in theory it should be pretty easy to provide an option to
use quilt rather than git when doing "devtool modify" or "devtool extract" for
those that prefer it.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-11-25 17:28 [RFC PATCH 0/7] Developer workflow improvements Paul Eggleton
` (8 preceding siblings ...)
2014-11-25 19:56 ` Trevor Woerner
@ 2014-11-28 17:28 ` Trevor Woerner
2014-12-01 10:11 ` Paul Eggleton
2014-12-02 4:54 ` Trevor Woerner
[not found] ` <54866CD7.1050102@linaro.org>
11 siblings, 1 reply; 40+ messages in thread
From: Trevor Woerner @ 2014-11-28 17:28 UTC (permalink / raw)
To: Paul Eggleton, openembedded-core
Hi Paul,
These tools are really nice! Some thoughts/comments:
Maybe the "devtool.conf" that gets created could be placed in the
"conf/" subdirectory (along with the other configuration files such as
local.conf and bblayers.conf)?
Perhaps any recipe you're working on could be automatically included via
an IMAGE_INSTALL_append in conf/local.conf (or maybe that's too intrusive?)?
Do you envision users creating multiple workspaces? I'm wondering why
"devtool create-workspace" is required. Is there any advantage to
requiring users to create the workspace explicitly instead of just
having it be created implicitly?
Some of the commands to "devtool" include things like
- extract
- build
- deploy
- undeploy
but when a workspace is created, devtool (very intelligently) adds the
workspace to the set of BBLAYERS. So one could then just use bitbake to
build the recipe. Are there any advantages to using "devtool build
<recipe>" instead of "bitbake <recipe>"?
Best regards,
Trevor
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-11-28 17:28 ` Trevor Woerner
@ 2014-12-01 10:11 ` Paul Eggleton
2014-12-02 4:36 ` Trevor Woerner
0 siblings, 1 reply; 40+ messages in thread
From: Paul Eggleton @ 2014-12-01 10:11 UTC (permalink / raw)
To: Trevor Woerner; +Cc: openembedded-core
On Friday 28 November 2014 12:28:00 Trevor Woerner wrote:
> These tools are really nice! Some thoughts/comments:
>
> Maybe the "devtool.conf" that gets created could be placed in the
> "conf/" subdirectory (along with the other configuration files such as
> local.conf and bblayers.conf)?
Yes, that's a good idea, I'll change that.
> Perhaps any recipe you're working on could be automatically included via
> an IMAGE_INSTALL_append in conf/local.conf (or maybe that's too intrusive?)?
This is something I'd wanted to do - it's certainly something that should be
made easy, but I was concerned about causing a full reparse just because of
adding that to local.conf. (There might be a workaround through some sort of
packagegroup for containing the packages produced by the recipes in the
workspace that is added once when we create the workspace - maybe that's the
answer?)
> Do you envision users creating multiple workspaces? I'm wondering why
> "devtool create-workspace" is required. Is there any advantage to
> requiring users to create the workspace explicitly instead of just
> having it be created implicitly?
I wouldn't expect users to want to create multiple workspaces, but I did want
users to be able to (a) choose where their workspace would go and (b) know
that it has been created, so that the workspace layer showing up in the
configuration isn't a surprise.
> Some of the commands to "devtool" include things like
> - extract
> - build
> - deploy
> - undeploy
> but when a workspace is created, devtool (very intelligently) adds the
> workspace to the set of BBLAYERS. So one could then just use bitbake to
> build the recipe. Are there any advantages to using "devtool build
> <recipe>" instead of "bitbake <recipe>"?
Not at the moment, although it is a convenient shortcut for "bitbake -c
install <recipename>" (which is all you need to do for "devtool deploy" - note
that "deploy" is distinct from our do_deploy, it could perhaps benefit from a
better name). The other reason it's there is more for use as part of the SDK
where the intention is to do everything through the devtool command, although
that is a usage model that isn't enabled yet.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-01 10:11 ` Paul Eggleton
@ 2014-12-02 4:36 ` Trevor Woerner
2014-12-02 11:46 ` Paul Eggleton
0 siblings, 1 reply; 40+ messages in thread
From: Trevor Woerner @ 2014-12-02 4:36 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
On 12/01/14 05:11, Paul Eggleton wrote:
> On Friday 28 November 2014 12:28:00 Trevor Woerner wrote:
>> Perhaps any recipe you're working on could be automatically included via
>> an IMAGE_INSTALL_append in conf/local.conf (or maybe that's too intrusive?)?
> This is something I'd wanted to do - it's certainly something that should be
> made easy, but I was concerned about causing a full reparse just because of
> adding that to local.conf. (There might be a workaround through some sort of
> packagegroup for containing the packages produced by the recipes in the
> workspace that is added once when we create the workspace - maybe that's the
> answer?)
Maybe even just printing a bit of text after a successful "add" to
inform the user that the just-added project isn't part of any image and
what they might want to do to include it?
The packagegroup sounds good too. But if the user doesn't want it
included, they might be confused about how it magically appeared, and
have some trouble finding how to remove it.
>> Do you envision users creating multiple workspaces? I'm wondering why
>> "devtool create-workspace" is required. Is there any advantage to
>> requiring users to create the workspace explicitly instead of just
>> having it be created implicitly?
> I wouldn't expect users to want to create multiple workspaces, but I did want
> users to be able to (a) choose where their workspace would go and (b) know
> that it has been created, so that the workspace layer showing up in the
> configuration isn't a surprise.
I can't help think that there's no harm in an unused workspace (is
there?). Maybe the empty workspace from "create-workspace" should just
be created by default by the "oe-init-build-env" tool (or whatever a
given project is using).
If I were writing the documentation for this workflow, or giving a
presentation on it... I'm just trying to figure out how to justify the
extra, empty (but necessary) step of creating the environment before
using it. Maybe if someone tries to add a project before creating the
workspace, instead of erroring out, just run the create-workspace for
them with the defaults. If they're a power user who wants to put the
workspace somewhere specific then they're free to explicitly call
create-workspace on their own, otherwise it'll get called with the
defaults for them (or created automatically as part of the
oe-init-build-env).
To borrow the analogy from git: create-workspace is part of the pluming;
it's there if people what to call it explicitly, otherwise it gets
called implicitly with defaults?
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-11-25 17:28 [RFC PATCH 0/7] Developer workflow improvements Paul Eggleton
` (9 preceding siblings ...)
2014-11-28 17:28 ` Trevor Woerner
@ 2014-12-02 4:54 ` Trevor Woerner
2014-12-02 14:01 ` Paul Eggleton
[not found] ` <54866CD7.1050102@linaro.org>
11 siblings, 1 reply; 40+ messages in thread
From: Trevor Woerner @ 2014-12-02 4:54 UTC (permalink / raw)
To: Paul Eggleton, openembedded-core
On 11/25/14 12:28, Paul Eggleton wrote:
> I've then added a new recipe auto-creation script, recipetool, which can
> take a source tree or URL and create a skeleton recipe to build it.
...
> * "recipetool create" ideally needs to become smarter and fill in
> more details of the recipe. At some point we'll probably have to
> make the process interactive and possibly have it automate parts of
> the build process and examine the output; some things just can't be
> practically detected from the source tree without building.
There is a tool "autoscan" which might be of some use here. If nothing
else, the resulting configure.scan file could be parsed for DEPENDS
information and perhaps PACKAGECONFIG items too.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-02 4:36 ` Trevor Woerner
@ 2014-12-02 11:46 ` Paul Eggleton
2014-12-04 14:03 ` Trevor Woerner
0 siblings, 1 reply; 40+ messages in thread
From: Paul Eggleton @ 2014-12-02 11:46 UTC (permalink / raw)
To: Trevor Woerner; +Cc: openembedded-core
On Monday 01 December 2014 23:36:23 Trevor Woerner wrote:
> On 12/01/14 05:11, Paul Eggleton wrote:
> > On Friday 28 November 2014 12:28:00 Trevor Woerner wrote:
> >> Perhaps any recipe you're working on could be automatically included via
> >> an IMAGE_INSTALL_append in conf/local.conf (or maybe that's too
> >> intrusive?)?>
> > This is something I'd wanted to do - it's certainly something that should
> > be made easy, but I was concerned about causing a full reparse just
> > because of adding that to local.conf. (There might be a workaround
> > through some sort of packagegroup for containing the packages produced by
> > the recipes in the workspace that is added once when we create the
> > workspace - maybe that's the answer?)
>
> Maybe even just printing a bit of text after a successful "add" to
> inform the user that the just-added project isn't part of any image and
> what they might want to do to include it?
>
> The packagegroup sounds good too. But if the user doesn't want it
> included, they might be confused about how it magically appeared, and
> have some trouble finding how to remove it.
Right. I might defer this until we get to the SDK part where we'll need
something to handle this (since the aim in that case is to avoid any editing
of conf files or interacting with bitbake directly).
> >> Do you envision users creating multiple workspaces? I'm wondering why
> >> "devtool create-workspace" is required. Is there any advantage to
> >> requiring users to create the workspace explicitly instead of just
> >> having it be created implicitly?
> >
> > I wouldn't expect users to want to create multiple workspaces, but I did
> > want users to be able to (a) choose where their workspace would go and
> > (b) know that it has been created, so that the workspace layer showing up
> > in the configuration isn't a surprise.
>
> I can't help think that there's no harm in an unused workspace (is
> there?). Maybe the empty workspace from "create-workspace" should just
> be created by default by the "oe-init-build-env" tool (or whatever a
> given project is using).
I guess my thought was that some people are fussy about their setup, and this
is a bit of a new thing.
> If I were writing the documentation for this workflow, or giving a
> presentation on it... I'm just trying to figure out how to justify the
> extra, empty (but necessary) step of creating the environment before
> using it. Maybe if someone tries to add a project before creating the
> workspace, instead of erroring out, just run the create-workspace for
> them with the defaults. If they're a power user who wants to put the
> workspace somewhere specific then they're free to explicitly call
> create-workspace on their own, otherwise it'll get called with the
> defaults for them (or created automatically as part of the
> oe-init-build-env).
>
> To borrow the analogy from git: create-workspace is part of the pluming;
> it's there if people what to call it explicitly, otherwise it gets
> called implicitly with defaults?
Actually you make a very good case; auto-creating it (and stating so) might be
better than failing. I'll make the change.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-02 4:54 ` Trevor Woerner
@ 2014-12-02 14:01 ` Paul Eggleton
2014-12-09 15:47 ` Trevor Woerner
0 siblings, 1 reply; 40+ messages in thread
From: Paul Eggleton @ 2014-12-02 14:01 UTC (permalink / raw)
To: Trevor Woerner; +Cc: openembedded-core
On Monday 01 December 2014 23:54:03 Trevor Woerner wrote:
> On 11/25/14 12:28, Paul Eggleton wrote:
> > I've then added a new recipe auto-creation script, recipetool, which can
> > take a source tree or URL and create a skeleton recipe to build it.
>
> ...
>
> > * "recipetool create" ideally needs to become smarter and fill in
> >
> > more details of the recipe. At some point we'll probably have to
> > make the process interactive and possibly have it automate parts of
> > the build process and examine the output; some things just can't be
> > practically detected from the source tree without building.
>
> There is a tool "autoscan" which might be of some use here. If nothing
> else, the resulting configure.scan file could be parsed for DEPENDS
> information and perhaps PACKAGECONFIG items too.
Interesting - I was not aware of this tool. I've added support for running it
as well as picking converting AC_CHECK_LIB lines over to DEPENDS (with a note
about some of them possibly being optional). Thanks!
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-02 11:46 ` Paul Eggleton
@ 2014-12-04 14:03 ` Trevor Woerner
2014-12-04 15:33 ` Paul Eggleton
0 siblings, 1 reply; 40+ messages in thread
From: Trevor Woerner @ 2014-12-04 14:03 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
Hi Paul,
Are you making these changes on the existing contrib/paule/devtool or
somewhere else?
A small nit I stumbled on recently... many layers contain a
"recipes-devtools" directory, and I'm thinking that using the same word
for this feature might get confusing (?). Maybe it would be best to
replace "devtool(s)" with "workflow(s)"?
Best regards,
Trevor
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-04 14:03 ` Trevor Woerner
@ 2014-12-04 15:33 ` Paul Eggleton
0 siblings, 0 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-12-04 15:33 UTC (permalink / raw)
To: Trevor Woerner; +Cc: openembedded-core
Hi Trevor,
On Thursday 04 December 2014 09:03:37 Trevor Woerner wrote:
> Are you making these changes on the existing contrib/paule/devtool or
> somewhere else?
Actually I hadn't pushed them anywhere; I have just done that on the
paule/devtool branch in both contrib repos. I'm collecting commits on top of
the original ones, these will be squashed in when I send the first non-RFC
series.
> A small nit I stumbled on recently... many layers contain a
> "recipes-devtools" directory, and I'm thinking that using the same word
> for this feature might get confusing (?). Maybe it would be best to
> replace "devtool(s)" with "workflow(s)"?
Well, naming is always tricky. I don't think "workflow" really describes what
the tool is supposed to do however; the tool certainly doesn't force a
particular workflow, just provides tools to enable various different developer
workflows. I'm happy to take other suggestions on naming though.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
[not found] ` <7839390.odV1UMkpj8@peggleto-mobl5.ger.corp.intel.com>
@ 2014-12-09 15:00 ` Trevor Woerner
2014-12-09 15:10 ` Paul Eggleton
0 siblings, 1 reply; 40+ messages in thread
From: Trevor Woerner @ 2014-12-09 15:00 UTC (permalink / raw)
To: Paul Eggleton, openembedded-core
(oops! sorry for going off-list)
On 12/09/14 05:50, Paul Eggleton wrote:
> On Monday 08 December 2014 22:30:31 Trevor Woerner wrote:
>> This is really awesome stuff. I'm still playing around with it.
>>
>> My current HEAD is:
>>
>> commit 6ea5f5368a88317ace198f44a47cf043bc9fb6e6
>> Author: Paul Eggleton <paul.eggleton@linux.intel.com>
>> Date: Fri Dec 5 11:30:25 2014 +0000
>>
>> devtool: use event filtering to disable events if available
>>
>> on your contrib/paule/devtool branch.
>>
>> If I do a "devtool add" without a "devtool create-workspace" it succeeds
>> but I get:
>>
>> $ devtool add xannounce ~/devel/code/xannounce
>> INFO: Creating workspace layer in
>> INFO: Recipe recipes/xannounce/xannounce.bb has been automatically
>> created; further editing may be required to make it fully functional
>>
>> and find the changes have been made in the current directory. If I do
>> the "devtool create-workspace" first, I get a "workspace" directory and
>> all the stuff put in there, which is nicer.
>>
>> If it doesn't fail due to a user running "devtool add" before "devtool
>> create-workspace" then I think it would be nice if the two results were
>> the same.
> Ah, oops - I've found the cause and fixed it on the branch. Thanks!
Awesome... *very* nice. It works great now.
>
>> Would it be possible to make the output of devtool be nice and colourful
>> like the output of bitbake?
> Good idea, this is now done on the branch as well. "devtool build" doesn't
> support colour in the bitbake output, but that's because colour support in
> bitbake is switched by whether the output is going to a TTY and stdout is
> being redirected in this case. (I guess we could add a command line option for
> that as I have with devtool/recipetool.)
>
If it's not too much trouble I would like to ask that colour/curses be
possible/enabled by default. Trying to find the error in all that output
is a "needle in a haystack"-like operation :-)
I wonder if all the
NOTE: Running setscene task 31 of 33
(/home/trevor/devel/yocto/build/poky/meta-poky/meta/recipes-devtools/gcc/gcc-runtime_4.9.bb,
do_populate_sysroot_setscene)
NOTE: recipe gcc-runtime-4.9.1-r0: task
do_populate_sysroot_setscene: Started
NOTE: recipe gcc-runtime-4.9.1-r0: task
do_populate_sysroot_setscene: Succeeded
NOTE: Running setscene task 33 of 33
(/home/trevor/devel/yocto/build/poky/meta-poky/meta/recipes-devtools/gcc/gcc-cross_4.9.bb,
do_populate_sysroot_setscene)
NOTE: recipe gcc-cross-i586-4.9.1-r0: task
do_populate_sysroot_setscene: Started
NOTE: recipe gcc-cross-i586-4.9.1-r0: task
do_populate_sysroot_setscene: Succeeded
Could be (should be) done as part of the "devtool
create-workspace"/"devtool add" operation?
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-09 15:00 ` Trevor Woerner
@ 2014-12-09 15:10 ` Paul Eggleton
2014-12-09 15:39 ` Trevor Woerner
0 siblings, 1 reply; 40+ messages in thread
From: Paul Eggleton @ 2014-12-09 15:10 UTC (permalink / raw)
To: Trevor Woerner; +Cc: openembedded-core
On Tuesday 09 December 2014 10:00:51 Trevor Woerner wrote:
> (oops! sorry for going off-list)
>
> On 12/09/14 05:50, Paul Eggleton wrote:
> > On Monday 08 December 2014 22:30:31 Trevor Woerner wrote:
> >> This is really awesome stuff. I'm still playing around with it.
> >>
> >> My current HEAD is:
> >> commit 6ea5f5368a88317ace198f44a47cf043bc9fb6e6
> >> Author: Paul Eggleton <paul.eggleton@linux.intel.com>
> >> Date: Fri Dec 5 11:30:25 2014 +0000
> >>
> >> devtool: use event filtering to disable events if available
> >>
> >> on your contrib/paule/devtool branch.
> >>
> >> If I do a "devtool add" without a "devtool create-workspace" it succeeds
> >>
> >> but I get:
> >> $ devtool add xannounce ~/devel/code/xannounce
> >> INFO: Creating workspace layer in
> >> INFO: Recipe recipes/xannounce/xannounce.bb has been automatically
> >> created; further editing may be required to make it fully functional
> >>
> >> and find the changes have been made in the current directory. If I do
> >> the "devtool create-workspace" first, I get a "workspace" directory and
> >> all the stuff put in there, which is nicer.
> >>
> >> If it doesn't fail due to a user running "devtool add" before "devtool
> >> create-workspace" then I think it would be nice if the two results were
> >> the same.
> >
> > Ah, oops - I've found the cause and fixed it on the branch. Thanks!
>
> Awesome... *very* nice. It works great now.
>
> >> Would it be possible to make the output of devtool be nice and colourful
> >> like the output of bitbake?
> >
> > Good idea, this is now done on the branch as well. "devtool build" doesn't
> > support colour in the bitbake output, but that's because colour support in
> > bitbake is switched by whether the output is going to a TTY and stdout is
> > being redirected in this case. (I guess we could add a command line option
> > for that as I have with devtool/recipetool.)
>
> If it's not too much trouble I would like to ask that colour/curses be
> possible/enabled by default. Trying to find the error in all that output
> is a "needle in a haystack"-like operation :-)
I don't think this is straightforward unfortunately. I'll have to look into
it.
> I wonder if all the
>
> NOTE: Running setscene task 31 of 33
>
> (/home/trevor/devel/yocto/build/poky/meta-poky/meta/recipes-devtools/gcc/gc
> c-runtime_4.9.bb, do_populate_sysroot_setscene)
> NOTE: recipe gcc-runtime-4.9.1-r0: task
> do_populate_sysroot_setscene: Started
> NOTE: recipe gcc-runtime-4.9.1-r0: task
> do_populate_sysroot_setscene: Succeeded
> NOTE: Running setscene task 33 of 33
>
> (/home/trevor/devel/yocto/build/poky/meta-poky/meta/recipes-devtools/gcc/gc
> c-cross_4.9.bb, do_populate_sysroot_setscene)
> NOTE: recipe gcc-cross-i586-4.9.1-r0: task
> do_populate_sysroot_setscene: Started
> NOTE: recipe gcc-cross-i586-4.9.1-r0: task
> do_populate_sysroot_setscene: Succeeded
>
>
> Could be (should be) done as part of the "devtool
> create-workspace"/"devtool add" operation?
If running devtool along side someone's existing build environment, wouldn't
that be expected to be set up already? (All it does effectively is "bitbake
recipename")
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-09 15:10 ` Paul Eggleton
@ 2014-12-09 15:39 ` Trevor Woerner
2014-12-09 16:08 ` Paul Eggleton
0 siblings, 1 reply; 40+ messages in thread
From: Trevor Woerner @ 2014-12-09 15:39 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
On 12/09/14 10:10, Paul Eggleton wrote:
> On Tuesday 09 December 2014 10:00:51 Trevor Woerner wrote:
>> Would it be possible to make the output of devtool be nice and colourful
>> like the output of bitbake?
>>> Good idea, this is now done on the branch as well. "devtool build" doesn't
>>> support colour in the bitbake output, but that's because colour support in
>>> bitbake is switched by whether the output is going to a TTY and stdout is
>>> being redirected in this case. (I guess we could add a command line option
>>> for that as I have with devtool/recipetool.)
>> If it's not too much trouble I would like to ask that colour/curses be
>> possible/enabled by default. Trying to find the error in all that output
>> is a "needle in a haystack"-like operation :-)
> I don't think this is straightforward unfortunately. I'll have to look into
> it.
Okay. Maybe it'd be best to just revert to bitbake? IOW, does the
devtool tool need its own "build,deploy,..."?
>
>> I wonder if all the
>>
>> NOTE: Running setscene task 31 of 33
>>
>> (/home/trevor/devel/yocto/build/poky/meta-poky/meta/recipes-devtools/gcc/gc
>> c-runtime_4.9.bb, do_populate_sysroot_setscene)
>> NOTE: recipe gcc-runtime-4.9.1-r0: task
>> do_populate_sysroot_setscene: Started
>> NOTE: recipe gcc-runtime-4.9.1-r0: task
>> do_populate_sysroot_setscene: Succeeded
>> NOTE: Running setscene task 33 of 33
>>
>> (/home/trevor/devel/yocto/build/poky/meta-poky/meta/recipes-devtools/gcc/gc
>> c-cross_4.9.bb, do_populate_sysroot_setscene)
>> NOTE: recipe gcc-cross-i586-4.9.1-r0: task
>> do_populate_sysroot_setscene: Started
>> NOTE: recipe gcc-cross-i586-4.9.1-r0: task
>> do_populate_sysroot_setscene: Succeeded
>>
>>
>> Could be (should be) done as part of the "devtool
>> create-workspace"/"devtool add" operation?
> If running devtool along side someone's existing build environment, wouldn't
> that be expected to be set up already? (All it does effectively is "bitbake
> recipename")
>
Here are my steps:
$ source meta-poky/oe-init-build-env build
$ bitbake core-image-x11
$ devtool add <myx11app>
$ devtool build <myx11app>
at this point it does a bunch of these gcc-runtime tasks. So my first
thought is: "why is my build going off into left field and doing all
these other things when all I asked it to do is to build <myx11app>?".
I'm guessing these gcc-runtime tasks are required for something related
to devtool, and they only need to be run once. So maybe it would be best
if they were run as part of creating the workspace (either explicitly
with "devtool create-workspace" or implicitly with "devtool add")?
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-02 14:01 ` Paul Eggleton
@ 2014-12-09 15:47 ` Trevor Woerner
2014-12-11 23:10 ` Trevor Woerner
0 siblings, 1 reply; 40+ messages in thread
From: Trevor Woerner @ 2014-12-09 15:47 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
On 12/02/14 09:01, Paul Eggleton wrote:
> On Monday 01 December 2014 23:54:03 Trevor Woerner wrote:
>> On 11/25/14 12:28, Paul Eggleton wrote:
>>> I've then added a new recipe auto-creation script, recipetool, which can
>>> take a source tree or URL and create a skeleton recipe to build it.
>> ...
>>
>>> * "recipetool create" ideally needs to become smarter and fill in
>>>
>>> more details of the recipe. At some point we'll probably have to
>>> make the process interactive and possibly have it automate parts of
>>> the build process and examine the output; some things just can't be
>>> practically detected from the source tree without building.
>> There is a tool "autoscan" which might be of some use here. If nothing
>> else, the resulting configure.scan file could be parsed for DEPENDS
>> information and perhaps PACKAGECONFIG items too.
> Interesting - I was not aware of this tool. I've added support for running it
> as well as picking converting AC_CHECK_LIB lines over to DEPENDS (with a note
> about some of them possibly being optional). Thanks!
>
Finding "AC_PATH_X" in "configure.scan" is probably a good indication
there's an RDEPENDS on "libx11". Maybe "recipetool" could check for that?
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-09 15:39 ` Trevor Woerner
@ 2014-12-09 16:08 ` Paul Eggleton
2014-12-09 18:58 ` Trevor Woerner
0 siblings, 1 reply; 40+ messages in thread
From: Paul Eggleton @ 2014-12-09 16:08 UTC (permalink / raw)
To: Trevor Woerner; +Cc: openembedded-core
On Tuesday 09 December 2014 10:39:05 Trevor Woerner wrote:
> On 12/09/14 10:10, Paul Eggleton wrote:
> > On Tuesday 09 December 2014 10:00:51 Trevor Woerner wrote:
> >> Would it be possible to make the output of devtool be nice and colourful
> >> like the output of bitbake?
> >>
> >>> Good idea, this is now done on the branch as well. "devtool build"
> >>> doesn't support colour in the bitbake output, but that's because colour
> >>> support in bitbake is switched by whether the output is going to a TTY
> >>> and stdout is being redirected in this case. (I guess we could add a
> >>> command line option for that as I have with devtool/recipetool.)
> >>
> >> If it's not too much trouble I would like to ask that colour/curses be
> >> possible/enabled by default. Trying to find the error in all that output
> >> is a "needle in a haystack"-like operation :-)
> >
> > I don't think this is straightforward unfortunately. I'll have to look
> > into it.
>
> Okay. Maybe it'd be best to just revert to bitbake? IOW, does the
> devtool tool need its own "build,deploy,..."?
We did talk about this in a previous email, but just to clarify:
"devtool build" isn't strictly required in this phase, I added it primarily to
support future usage in the SDK where the intention is to do everything
through the devtool command, although that is a usage model that isn't enabled
yet. However, it is a convenient shortcut for "bitbake -c install
<recipename>" (which is all you need to do for "devtool deploy" to be able to
work.)
"devtool deploy" is not related to do_deploy - it places files for the recipe
on a nominated target machine without having to worry about packaging; we have
no other equivalent for this at the moment. Perhaps it could have a different
name in case the naming overlap is confusing.
> >> I wonder if all the
> >>
> >> NOTE: Running setscene task 31 of 33
> >>
> >> (/home/trevor/devel/yocto/build/poky/meta-poky/meta/recipes-devtools/gcc/
> >> gc
> >> c-runtime_4.9.bb, do_populate_sysroot_setscene)
> >>
> >> NOTE: recipe gcc-runtime-4.9.1-r0: task
> >> do_populate_sysroot_setscene: Started
> >> NOTE: recipe gcc-runtime-4.9.1-r0: task
> >> do_populate_sysroot_setscene: Succeeded
> >> NOTE: Running setscene task 33 of 33
> >>
> >> (/home/trevor/devel/yocto/build/poky/meta-poky/meta/recipes-devtools/gcc/
> >> gc
> >> c-cross_4.9.bb, do_populate_sysroot_setscene)
> >>
> >> NOTE: recipe gcc-cross-i586-4.9.1-r0: task
> >> do_populate_sysroot_setscene: Started
> >> NOTE: recipe gcc-cross-i586-4.9.1-r0: task
> >> do_populate_sysroot_setscene: Succeeded
> >>
> >> Could be (should be) done as part of the "devtool
> >> create-workspace"/"devtool add" operation?
> >
> > If running devtool along side someone's existing build environment,
> > wouldn't that be expected to be set up already? (All it does effectively
> > is "bitbake recipename")
>
> Here are my steps:
> $ source meta-poky/oe-init-build-env build
> $ bitbake core-image-x11
> $ devtool add <myx11app>
> $ devtool build <myx11app>
>
> at this point it does a bunch of these gcc-runtime tasks. So my first
> thought is: "why is my build going off into left field and doing all
> these other things when all I asked it to do is to build <myx11app>?".
> I'm guessing these gcc-runtime tasks are required for something related
> to devtool, and they only need to be run once. So maybe it would be best
> if they were run as part of creating the workspace (either explicitly
> with "devtool create-workspace" or implicitly with "devtool add")?
Trying to make "devtool add" or "devtool modify" do this just doesn't feel
right to me. These operations are supposed to be quick, and if they end up
running actual build tasks, they won't be.
My guess (with no other context) is that the reason you are seeing this is
that you're building from sstate, and when it builds core-image-x11 it doesn't
need some of the tasks that it then does need when it goes to build the
individual recipe. I'm not sure, but I would expect that most people would be
doing this from an existing build directory rather than starting with a new
one. Is there a reason you're creating a new build directory as part of the
steps above?
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-09 16:08 ` Paul Eggleton
@ 2014-12-09 18:58 ` Trevor Woerner
2014-12-10 17:51 ` Paul Eggleton
0 siblings, 1 reply; 40+ messages in thread
From: Trevor Woerner @ 2014-12-09 18:58 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
On 12/09/14 11:08, Paul Eggleton wrote:
> On Tuesday 09 December 2014 10:39:05 Trevor Woerner wrote:
>>
>> Okay. Maybe it'd be best to just revert to bitbake? IOW, does the
>> devtool tool need its own "build,deploy,..."?
> We did talk about this in a previous email, but just to clarify:
>
> "devtool build" isn't strictly required in this phase, I added it primarily to
> support future usage in the SDK where the intention is to do everything
> through the devtool command, although that is a usage model that isn't enabled
> yet. However, it is a convenient shortcut for "bitbake -c install
> <recipename>" (which is all you need to do for "devtool deploy" to be able to
> work.)
True, we did talk about this before. All I'm saying is that the colour
feature of bitbake is really nice, useful, and helpful (to visually
parse output) and if we've lost colour because we've wrapped bitbake
with "devtool build" then we're taking a step backwards.
But I'll concede the point if the future usage in the SDK turns out to
be really helpful. All I'm saying at this point is I have no idea what
this future usage might be or what your ideas are for this step, and for
the time-being (i.e. without knowing or working with this SDK usage)
"devtool build" is a step backwards from "bitbake".
> "devtool deploy" is not related to do_deploy - it places files for the recipe
> on a nominated target machine without having to worry about packaging; we have
> no other equivalent for this at the moment. Perhaps it could have a different
> name in case the naming overlap is confusing.
Ah yes, sorry I forgot... maybe something more explicit like
"deploy_to_target"? ;-)
push_to_target?
upload_to_target?
>
>>>> I wonder if all the
>>>>
>>>> NOTE: Running setscene task 31 of 33
>>>>
>>>> (/home/trevor/devel/yocto/build/poky/meta-poky/meta/recipes-devtools/gcc/
>>>> gc
>>>> c-runtime_4.9.bb, do_populate_sysroot_setscene)
>>>>
>>>> NOTE: recipe gcc-runtime-4.9.1-r0: task
>>>> do_populate_sysroot_setscene: Started
>>>> NOTE: recipe gcc-runtime-4.9.1-r0: task
>>>> do_populate_sysroot_setscene: Succeeded
>>>> NOTE: Running setscene task 33 of 33
>>>>
>>>> (/home/trevor/devel/yocto/build/poky/meta-poky/meta/recipes-devtools/gcc/
>>>> gc
>>>> c-cross_4.9.bb, do_populate_sysroot_setscene)
>>>>
>>>> NOTE: recipe gcc-cross-i586-4.9.1-r0: task
>>>> do_populate_sysroot_setscene: Started
>>>> NOTE: recipe gcc-cross-i586-4.9.1-r0: task
>>>> do_populate_sysroot_setscene: Succeeded
>>>>
>>>> Could be (should be) done as part of the "devtool
>>>> create-workspace"/"devtool add" operation?
>>> If running devtool along side someone's existing build environment,
>>> wouldn't that be expected to be set up already? (All it does effectively
>>> is "bitbake recipename")
>> Here are my steps:
>> $ source meta-poky/oe-init-build-env build
>> $ bitbake core-image-x11
>> $ devtool add <myx11app>
>> $ devtool build <myx11app>
>>
>> at this point it does a bunch of these gcc-runtime tasks. So my first
>> thought is: "why is my build going off into left field and doing all
>> these other things when all I asked it to do is to build <myx11app>?".
>> I'm guessing these gcc-runtime tasks are required for something related
>> to devtool, and they only need to be run once. So maybe it would be best
>> if they were run as part of creating the workspace (either explicitly
>> with "devtool create-workspace" or implicitly with "devtool add")?
> Trying to make "devtool add" or "devtool modify" do this just doesn't feel
> right to me. These operations are supposed to be quick, and if they end up
> running actual build tasks, they won't be.
>
> My guess (with no other context) is that the reason you are seeing this is
> that you're building from sstate, and when it builds core-image-x11 it doesn't
> need some of the tasks that it then does need when it goes to build the
> individual recipe. I'm not sure, but I would expect that most people would be
> doing this from an existing build directory rather than starting with a new
> one. Is there a reason you're creating a new build directory as part of the
> steps above?
The reason I included the "oe-init-build-env build" step in my
instructions above is to emphasize the fact I'm starting from a
completely fresh build environment :-)
LICENSE
If the tool that creates the recipe can't find any explicit licensing
information, it leaves LICENCE blank. Would it be better, if there's no
explicit license, to set it to "CLOSED"? Otherwise the first build will
fail and the user will have to figure out what to do in order to get the
recipe to build.
code updates
Maybe my inexperience with externalsrc is showing through here (I've
never used it before), but as I play with this new feature, I'm testing
it by trying to include a very old project I wrote years ago. As luck
would have it, that project needs updates (it doesn't quite work as-is
in qemux86/core-image-x11). So here I am going through the cycle of
working on its source, and building new images. What I've noticed is
when I create the image I keep getting the same old sources used over
and over from when I first added this project "devtool add" (and it
doesn't matter whether I use bitbake or devtool to build it, it even
keeps using the old sources even when I perform commits in the sources
and even when I perform a "bitbake -c cleansstate <recipe>"!). The only
way I can move forward is to "devtool reset" and then "devtool add".
In my scenario I have a base image I have already built
(core-image-x11). Now I want to add one package to that image from
sources found on my local machine. I also want to go through the cycle
of editing those sources, building a new image, and runqemu'ing the new
image to test my changes. I can't seem to do that without knowing
devtool and/or bitbake magic. I either have to "devtool reset" or I have
to "bitbake -c clean core-image-x11" because just modifying the sources
doesn't cause the image to be regenerated.
I think at this point I should create a detailed doc explaining my
workflow (or maybe a screencast?) so you can see exactly what I'm doing
if the above doesn't make sense.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-09 18:58 ` Trevor Woerner
@ 2014-12-10 17:51 ` Paul Eggleton
2014-12-11 17:14 ` Paul Eggleton
0 siblings, 1 reply; 40+ messages in thread
From: Paul Eggleton @ 2014-12-10 17:51 UTC (permalink / raw)
To: Trevor Woerner; +Cc: openembedded-core
On Tuesday 09 December 2014 13:58:02 Trevor Woerner wrote:
> On 12/09/14 11:08, Paul Eggleton wrote:
> > On Tuesday 09 December 2014 10:39:05 Trevor Woerner wrote:
> >> Okay. Maybe it'd be best to just revert to bitbake? IOW, does the
> >> devtool tool need its own "build,deploy,..."?
> >
> > We did talk about this in a previous email, but just to clarify:
> >
> > "devtool build" isn't strictly required in this phase, I added it
> > primarily to support future usage in the SDK where the intention is to do
> > everything through the devtool command, although that is a usage model
> > that isn't enabled yet. However, it is a convenient shortcut for "bitbake
> > -c install
> > <recipename>" (which is all you need to do for "devtool deploy" to be able
> > to work.)
>
> True, we did talk about this before. All I'm saying is that the colour
> feature of bitbake is really nice, useful, and helpful (to visually
> parse output) and if we've lost colour because we've wrapped bitbake
> with "devtool build" then we're taking a step backwards.
>
> But I'll concede the point if the future usage in the SDK turns out to
> be really helpful. All I'm saying at this point is I have no idea what
> this future usage might be or what your ideas are for this step, and for
> the time-being (i.e. without knowing or working with this SDK usage)
> "devtool build" is a step backwards from "bitbake".
OK, understood. I'll have a think about it.
> > "devtool deploy" is not related to do_deploy - it places files for the
> > recipe on a nominated target machine without having to worry about
> > packaging; we have no other equivalent for this at the moment. Perhaps it
> > could have a different name in case the naming overlap is confusing.
>
> Ah yes, sorry I forgot... maybe something more explicit like
> "deploy_to_target"? ;-)
> push_to_target?
> upload_to_target?
Hmm, perhaps deploy-target would be more in line with the git-style command
naming I've been using.
> >> Here are my steps:
> >> $ source meta-poky/oe-init-build-env build
> >> $ bitbake core-image-x11
> >> $ devtool add <myx11app>
> >> $ devtool build <myx11app>
> >>
> >> at this point it does a bunch of these gcc-runtime tasks. So my first
> >> thought is: "why is my build going off into left field and doing all
> >> these other things when all I asked it to do is to build <myx11app>?".
> >> I'm guessing these gcc-runtime tasks are required for something related
> >> to devtool, and they only need to be run once. So maybe it would be best
> >> if they were run as part of creating the workspace (either explicitly
> >> with "devtool create-workspace" or implicitly with "devtool add")?
> >
> > Trying to make "devtool add" or "devtool modify" do this just doesn't feel
> > right to me. These operations are supposed to be quick, and if they end up
> > running actual build tasks, they won't be.
> >
> > My guess (with no other context) is that the reason you are seeing this is
> > that you're building from sstate, and when it builds core-image-x11 it
> > doesn't need some of the tasks that it then does need when it goes to
> > build the individual recipe. I'm not sure, but I would expect that most
> > people would be doing this from an existing build directory rather than
> > starting with a new one. Is there a reason you're creating a new build
> > directory as part of the steps above?
>
> The reason I included the "oe-init-build-env build" step in my
> instructions above is to emphasize the fact I'm starting from a
> completely fresh build environment :-)
OK, but would you expect to be doing that on a regular basis or is it just for
the purposes of exercise?
> LICENSE
> If the tool that creates the recipe can't find any explicit licensing
> information, it leaves LICENCE blank. Would it be better, if there's no
> explicit license, to set it to "CLOSED"? Otherwise the first build will
> fail and the user will have to figure out what to do in order to get the
> recipe to build.
Possibly yes. Ideally people would take an interest in setting this properly,
but I do recognise that it's not a high priority when you are just getting
started. What might be missing is some kind of modal operation where during
initial development you aren't bothered by all of these recipe quality checks,
but then when it comes to production the checks are re-enabled.
> code updates
> Maybe my inexperience with externalsrc is showing through here (I've
> never used it before), but as I play with this new feature, I'm testing
> it by trying to include a very old project I wrote years ago. As luck
> would have it, that project needs updates (it doesn't quite work as-is
> in qemux86/core-image-x11). So here I am going through the cycle of
> working on its source, and building new images. What I've noticed is
> when I create the image I keep getting the same old sources used over
> and over from when I first added this project "devtool add" (and it
> doesn't matter whether I use bitbake or devtool to build it, it even
> keeps using the old sources even when I perform commits in the sources
> and even when I perform a "bitbake -c cleansstate <recipe>"!). The only
> way I can move forward is to "devtool reset" and then "devtool add".
>
> In my scenario I have a base image I have already built
> (core-image-x11). Now I want to add one package to that image from
> sources found on my local machine. I also want to go through the cycle
> of editing those sources, building a new image, and runqemu'ing the new
> image to test my changes. I can't seem to do that without knowing
> devtool and/or bitbake magic. I either have to "devtool reset" or I have
> to "bitbake -c clean core-image-x11" because just modifying the sources
> doesn't cause the image to be regenerated.
It's not supposed to behave this way, although I have reproduced it here - I
think I hadn't actually tested rebuilding an image with a recipe in the
workspace added to the image. The problem is that the do_compile task is being
marked as nostamp, but the way bitbake behaves now, that doesn't cause any
dependent task signatures to be changed and thus no dependent tasks get run
afterwards. Either I'll have to change that or work out a different way of
ensuring do_compile and dependent tasks get run - either way, you should only
have to run a build that calls for the recipe (be it the recipe itself or
something that depends upon it), no cleaning/resetting should be needed. I'll
try to fix this tomorrow.
> I think at this point I should create a detailed doc explaining my
> workflow (or maybe a screencast?) so you can see exactly what I'm doing
> if the above doesn't make sense.
If you think it might help, but I think I understand most of the above. Let me
fix the task signature issue and then hopefully things should work a bit more
sensibly :)
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-10 17:51 ` Paul Eggleton
@ 2014-12-11 17:14 ` Paul Eggleton
2014-12-11 18:30 ` Trevor Woerner
2014-12-14 3:47 ` Trevor Woerner
0 siblings, 2 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-12-11 17:14 UTC (permalink / raw)
To: Trevor Woerner; +Cc: openembedded-core
On Wednesday 10 December 2014 17:51:16 Paul Eggleton wrote:
> On Tuesday 09 December 2014 13:58:02 Trevor Woerner wrote:
> > On 12/09/14 11:08, Paul Eggleton wrote:
> > > "devtool build" isn't strictly required in this phase, I added it
> > > primarily to support future usage in the SDK where the intention is to
> > > do everything through the devtool command, although that is a usage
> > > model that isn't enabled yet. However, it is a convenient shortcut for
> > > "bitbake -c install <recipename>" (which is all you need to do for
> > > "devtool deploy" to be able to work.)
> >
> > True, we did talk about this before. All I'm saying is that the colour
> > feature of bitbake is really nice, useful, and helpful (to visually
> > parse output) and if we've lost colour because we've wrapped bitbake
> > with "devtool build" then we're taking a step backwards.
> >
> > But I'll concede the point if the future usage in the SDK turns out to
> > be really helpful. All I'm saying at this point is I have no idea what
> > this future usage might be or what your ideas are for this step, and for
> > the time-being (i.e. without knowing or working with this SDK usage)
> > "devtool build" is a step backwards from "bitbake".
>
> OK, understood. I'll have a think about it.
I've fixed this on the branch; turns out there is an easy way to fool processes
into thinking they are connected to a real terminal and thus have bitbake use
the normal colour and terminal handling.
> > > "devtool deploy" is not related to do_deploy - it places files for the
> > > recipe on a nominated target machine without having to worry about
> > > packaging; we have no other equivalent for this at the moment. Perhaps
> > > it
> > > could have a different name in case the naming overlap is confusing.
> >
> > Ah yes, sorry I forgot... maybe something more explicit like
> > "deploy_to_target"? ;-)
> > push_to_target?
> > upload_to_target?
>
> Hmm, perhaps deploy-target would be more in line with the git-style command
> naming I've been using.
Fixed on the branch.
> > LICENSE
> > If the tool that creates the recipe can't find any explicit licensing
> > information, it leaves LICENCE blank. Would it be better, if there's no
> > explicit license, to set it to "CLOSED"? Otherwise the first build will
> > fail and the user will have to figure out what to do in order to get the
> > recipe to build.
>
> Possibly yes. Ideally people would take an interest in setting this
> properly, but I do recognise that it's not a high priority when you are
> just getting started. What might be missing is some kind of modal operation
> where during initial development you aren't bothered by all of these recipe
> quality checks, but then when it comes to production the checks are
> re-enabled.
I've made this change on the branch (with an appropriate warning comment
preceding it).
> > code updates
> > Maybe my inexperience with externalsrc is showing through here (I've
> > never used it before), but as I play with this new feature, I'm testing
> > it by trying to include a very old project I wrote years ago. As luck
> > would have it, that project needs updates (it doesn't quite work as-is
> > in qemux86/core-image-x11). So here I am going through the cycle of
> > working on its source, and building new images. What I've noticed is
> > when I create the image I keep getting the same old sources used over
> > and over from when I first added this project "devtool add" (and it
> > doesn't matter whether I use bitbake or devtool to build it, it even
> > keeps using the old sources even when I perform commits in the sources
> > and even when I perform a "bitbake -c cleansstate <recipe>"!). The only
> > way I can move forward is to "devtool reset" and then "devtool add".
> >
> > In my scenario I have a base image I have already built
> > (core-image-x11). Now I want to add one package to that image from
> > sources found on my local machine. I also want to go through the cycle
> > of editing those sources, building a new image, and runqemu'ing the new
> > image to test my changes. I can't seem to do that without knowing
> > devtool and/or bitbake magic. I either have to "devtool reset" or I have
> > to "bitbake -c clean core-image-x11" because just modifying the sources
> > doesn't cause the image to be regenerated.
>
> It's not supposed to behave this way, although I have reproduced it here - I
> think I hadn't actually tested rebuilding an image with a recipe in the
> workspace added to the image. The problem is that the do_compile task is
> being marked as nostamp, but the way bitbake behaves now, that doesn't
> cause any dependent task signatures to be changed and thus no dependent
> tasks get run afterwards. Either I'll have to change that or work out a
> different way of ensuring do_compile and dependent tasks get run - either
> way, you should only have to run a build that calls for the recipe (be it
> the recipe itself or something that depends upon it), no cleaning/resetting
> should be needed. I'll try to fix this tomorrow.
OK, I've fixed this on the branch as well. You should now find that rebuilding
the image always triggers a compile -> install -> package etc. of the
recipe(s) in the workspace first. (It'll end up triggering other recipes to
rebuild as well if they are in the chain; we may have to address that
differently later for cases where that's annoying).
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-11 17:14 ` Paul Eggleton
@ 2014-12-11 18:30 ` Trevor Woerner
2014-12-11 18:55 ` Paul Eggleton
2014-12-14 3:47 ` Trevor Woerner
1 sibling, 1 reply; 40+ messages in thread
From: Trevor Woerner @ 2014-12-11 18:30 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
Awesome!! I'm giving this update a whirl right now...
...just one small nit... in commit
7d2c6430131ccf94b8bdc185b8cd849fe479d552 the warning message says:
+ lines_before.append('# NOTE: LICENSE is being set to "CLOSED"
to allow you to at least start building - if')
+ lines_before.append('# this is not accurate with respect to the
licensing of the software being built (it')
+ lines_before.append('# will not be for in most cases) you must
specify the correct value before using this')
+ lines_before.append('# recipe for anything other than initial
testing/development!')
Is the "for" in "it will not be for in most cases" out of place? Perhaps
replace with "so", "true" or "accurate"?
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-11 18:30 ` Trevor Woerner
@ 2014-12-11 18:55 ` Paul Eggleton
2014-12-11 19:16 ` Trevor Woerner
0 siblings, 1 reply; 40+ messages in thread
From: Paul Eggleton @ 2014-12-11 18:55 UTC (permalink / raw)
To: Trevor Woerner; +Cc: openembedded-core
On Thursday 11 December 2014 13:30:01 Trevor Woerner wrote:
> Awesome!! I'm giving this update a whirl right now...
>
> ...just one small nit... in commit
> 7d2c6430131ccf94b8bdc185b8cd849fe479d552 the warning message says:
>
> + lines_before.append('# NOTE: LICENSE is being set to "CLOSED"
> to allow you to at least start building - if')
> + lines_before.append('# this is not accurate with respect to the
> licensing of the software being built (it')
> + lines_before.append('# will not be for in most cases) you must
> specify the correct value before using this')
> + lines_before.append('# recipe for anything other than initial
> testing/development!')
>
> Is the "for" in "it will not be for in most cases" out of place? Perhaps
> replace with "so", "true" or "accurate"?
Ah yes - fixed, thanks :)
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-11 18:55 ` Paul Eggleton
@ 2014-12-11 19:16 ` Trevor Woerner
2014-12-11 19:43 ` Trevor Woerner
0 siblings, 1 reply; 40+ messages in thread
From: Trevor Woerner @ 2014-12-11 19:16 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
Oops! (this worked before the "git pull")
$ devtool add xannounce ~/devel/yocto/build/workflow/xannounce/
NOTE: Creating workspace layer in
/home/trevor/devel/yocto/build/workflow/build/workspace
NOTE: Recipe
/home/trevor/devel/yocto/build/workflow/build/workspace/recipes/xannounce/xannounce.bb
has been automatically created; further editing may be required to
make it fully functional
Traceback (most recent call last):
File
"/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/devtool",
line 274, in <module>
ret = main()
File
"/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/devtool",
line 267, in main
ret = args.func(args, config, basepath, workspace)
File
"/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/lib/devtool/standard.py",
line 68, in add
_add_md5(config, args.recipename, recipefile)
File
"/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/lib/devtool/standard.py",
line 218, in _add_md5
md5 = bb.utils.md5_file(filename)
File
"/home/trevor/devel/yocto/build/workflow/meta-poky/bitbake/lib/bb/utils.py",
line 463, in md5_file
with open(filename, "rb") as f:
IOError: [Errno 2] No such file or directory:
'/home/trevor/devel/yocto/build/workflow/build/workspace/recipes/xannounce/xannounce.bb'
Sure enough...
$ devtool-workflow> tree workspace/
workspace/
|-- appends
|-- conf
| `-- layer.conf
`-- recipes
`-- xannounce
4 directories, 1 file
whereas before it was:
$ tree workspace/
workspace/
|-- appends
| `-- xannounce.bbappend
|-- conf
| `-- layer.conf
`-- recipes
`-- xannounce
`-- xannounce.bb
4 directories, 3 files
For some reason my recipe and bbappends are not getting created :-(
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-11 19:16 ` Trevor Woerner
@ 2014-12-11 19:43 ` Trevor Woerner
2014-12-11 21:35 ` Paul Eggleton
0 siblings, 1 reply; 40+ messages in thread
From: Trevor Woerner @ 2014-12-11 19:43 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
bisecting (manually) leads to this commit being safe:
ea50ec527483eb3a094588858baebd5fb23f71c6 devtool: rename deploy/undeploy
to deploy-target/undeploy-target
and this one introducing the problem:
770f203f73f358ce3c6c5ac12fe2775b812bdc6b devtool: fool bitbake into
thinking it's outputting to a terminal
On 12/11/14 14:16, Trevor Woerner wrote:
> Oops! (this worked before the "git pull")
>
> $ devtool add xannounce ~/devel/yocto/build/workflow/xannounce/
> NOTE: Creating workspace layer in
> /home/trevor/devel/yocto/build/workflow/build/workspace
> NOTE: Recipe
> /home/trevor/devel/yocto/build/workflow/build/workspace/recipes/xannounce/xannounce.bb
> has been automatically created; further editing may be required to
> make it fully functional
> Traceback (most recent call last):
> File
> "/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/devtool",
> line 274, in <module>
> ret = main()
> File
> "/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/devtool",
> line 267, in main
> ret = args.func(args, config, basepath, workspace)
> File
> "/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/lib/devtool/standard.py",
> line 68, in add
> _add_md5(config, args.recipename, recipefile)
> File
> "/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/lib/devtool/standard.py",
> line 218, in _add_md5
> md5 = bb.utils.md5_file(filename)
> File
> "/home/trevor/devel/yocto/build/workflow/meta-poky/bitbake/lib/bb/utils.py",
> line 463, in md5_file
> with open(filename, "rb") as f:
> IOError: [Errno 2] No such file or directory:
> '/home/trevor/devel/yocto/build/workflow/build/workspace/recipes/xannounce/xannounce.bb'
>
> Sure enough...
>
> $ devtool-workflow> tree workspace/
> workspace/
> |-- appends
> |-- conf
> | `-- layer.conf
> `-- recipes
> `-- xannounce
>
> 4 directories, 1 file
>
>
> whereas before it was:
>
> $ tree workspace/
> workspace/
> |-- appends
> | `-- xannounce.bbappend
> |-- conf
> | `-- layer.conf
> `-- recipes
> `-- xannounce
> `-- xannounce.bb
>
> 4 directories, 3 files
>
> For some reason my recipe and bbappends are not getting created :-(
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-11 19:43 ` Trevor Woerner
@ 2014-12-11 21:35 ` Paul Eggleton
2014-12-11 23:14 ` Trevor Woerner
2014-12-12 13:01 ` Otavio Salvador
0 siblings, 2 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-12-11 21:35 UTC (permalink / raw)
To: Trevor Woerner; +Cc: openembedded-core
On Thursday 11 December 2014 14:43:40 Trevor Woerner wrote:
> On 12/11/14 14:16, Trevor Woerner wrote:
> > Oops! (this worked before the "git pull")
> >
> > $ devtool add xannounce ~/devel/yocto/build/workflow/xannounce/
> > NOTE: Creating workspace layer in
> > /home/trevor/devel/yocto/build/workflow/build/workspace
> > NOTE: Recipe
> > /home/trevor/devel/yocto/build/workflow/build/workspace/recipes/xannou
> > nce/xannounce.bb has been automatically created; further editing may
> > be required to make it fully functional
> >
> > Traceback (most recent call last):
> > File
> >
> > "/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/devtool",
> > line 274, in <module>
> >
> > ret = main()
> >
> > File
> >
> > "/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/devtool",
> > line 267, in main
> >
> > ret = args.func(args, config, basepath, workspace)
> >
> > File
> >
> > "/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/lib/devtool
> > /standard.py", line 68, in add
> >
> > _add_md5(config, args.recipename, recipefile)
> >
> > File
> >
> > "/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/lib/devtool
> > /standard.py", line 218, in _add_md5
> >
> > md5 = bb.utils.md5_file(filename)
> >
> > File
> >
> > "/home/trevor/devel/yocto/build/workflow/meta-poky/bitbake/lib/bb/util
> > s.py", line 463, in md5_file
> >
> > with open(filename, "rb") as f:
> > IOError: [Errno 2] No such file or directory:
> > '/home/trevor/devel/yocto/build/workflow/build/workspace/recipes/xanno
> > unce/xannounce.bb'>
> > Sure enough...
> >
> > $ devtool-workflow> tree workspace/
> > workspace/
> >
> > |-- appends
> > |-- conf
> > |
> > | `-- layer.conf
> >
> > `-- recipes
> >
> > `-- xannounce
> >
> > 4 directories, 1 file
> >
> > whereas before it was:
> > $ tree workspace/
> > workspace/
> >
> > |-- appends
> > |
> > | `-- xannounce.bbappend
> > |
> > |-- conf
> > |
> > | `-- layer.conf
> >
> > `-- recipes
> >
> > `-- xannounce
> >
> > `-- xannounce.bb
> >
> > 4 directories, 3 files
> >
> > For some reason my recipe and bbappends are not getting created :-(
>
> bisecting (manually) leads to this commit being safe:
>
> ea50ec527483eb3a094588858baebd5fb23f71c6 devtool: rename deploy/undeploy
> to deploy-target/undeploy-target
>
> and this one introducing the problem:
>
> 770f203f73f358ce3c6c5ac12fe2775b812bdc6b devtool: fool bitbake into
> thinking it's outputting to a terminal
Thanks for the quick bisect - I've fixed it so that wrapper is only used when
running bitbake through "devtool build", and now "devtool add" works properly
again. This is why I should probably write some proper tests before trying to
do anything further...
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-09 15:47 ` Trevor Woerner
@ 2014-12-11 23:10 ` Trevor Woerner
2014-12-12 12:39 ` Paul Eggleton
0 siblings, 1 reply; 40+ messages in thread
From: Trevor Woerner @ 2014-12-11 23:10 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
I think *this* thread (and question) got lost in the shuffle...
On 12/09/14 10:47, Trevor Woerner wrote:
> On 12/02/14 09:01, Paul Eggleton wrote:
>> On Monday 01 December 2014 23:54:03 Trevor Woerner wrote:
>>> On 11/25/14 12:28, Paul Eggleton wrote:
>>>> I've then added a new recipe auto-creation script, recipetool, which can
>>>> take a source tree or URL and create a skeleton recipe to build it.
>>> ...
>>>
>>>> * "recipetool create" ideally needs to become smarter and fill in
>>>>
>>>> more details of the recipe. At some point we'll probably have to
>>>> make the process interactive and possibly have it automate parts of
>>>> the build process and examine the output; some things just can't be
>>>> practically detected from the source tree without building.
>>> There is a tool "autoscan" which might be of some use here. If nothing
>>> else, the resulting configure.scan file could be parsed for DEPENDS
>>> information and perhaps PACKAGECONFIG items too.
>> Interesting - I was not aware of this tool. I've added support for running it
>> as well as picking converting AC_CHECK_LIB lines over to DEPENDS (with a note
>> about some of them possibly being optional). Thanks!
>>
> Finding "AC_PATH_X" in "configure.scan" is probably a good indication
> there's an RDEPENDS on "libx11". Maybe "recipetool" could check for that?
*ping*!!
Oddly enough, the default recipe even says:
# NOTE: the following library dependencies are unknown, ignoring: X11
Why is it ignoring x11? Coincidentally the code I'm testing this with
has a dependency on X11 and when I try building, bitbake throws out an
RDEPENDS warning.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-11 21:35 ` Paul Eggleton
@ 2014-12-11 23:14 ` Trevor Woerner
2014-12-12 13:01 ` Otavio Salvador
1 sibling, 0 replies; 40+ messages in thread
From: Trevor Woerner @ 2014-12-11 23:14 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
On 12/11/14 16:35, Paul Eggleton wrote:
> Thanks for the quick bisect - I've fixed it so that wrapper is only used when
> running bitbake through "devtool build", and now "devtool add" works properly
> again. This is why I should probably write some proper tests before trying to
> do anything further...
It works great now, thanks!
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-11 23:10 ` Trevor Woerner
@ 2014-12-12 12:39 ` Paul Eggleton
0 siblings, 0 replies; 40+ messages in thread
From: Paul Eggleton @ 2014-12-12 12:39 UTC (permalink / raw)
To: Trevor Woerner; +Cc: openembedded-core
On Thursday 11 December 2014 18:10:57 Trevor Woerner wrote:
> I think *this* thread (and question) got lost in the shuffle...
>
> On 12/09/14 10:47, Trevor Woerner wrote:
> > On 12/02/14 09:01, Paul Eggleton wrote:
> >> On Monday 01 December 2014 23:54:03 Trevor Woerner wrote:
> >>> On 11/25/14 12:28, Paul Eggleton wrote:
> >>>> I've then added a new recipe auto-creation script, recipetool, which
> >>>> can
> >>>> take a source tree or URL and create a skeleton recipe to build it.
> >>>
> >>> ...
> >>>
> >>>> * "recipetool create" ideally needs to become smarter and fill in
> >>>>
> >>>> more details of the recipe. At some point we'll probably have to
> >>>> make the process interactive and possibly have it automate parts of
> >>>> the build process and examine the output; some things just can't be
> >>>> practically detected from the source tree without building.
> >>>
> >>> There is a tool "autoscan" which might be of some use here. If nothing
> >>> else, the resulting configure.scan file could be parsed for DEPENDS
> >>> information and perhaps PACKAGECONFIG items too.
> >>
> >> Interesting - I was not aware of this tool. I've added support for
> >> running it as well as picking converting AC_CHECK_LIB lines over to
> >> DEPENDS (with a note about some of them possibly being optional).
> >> Thanks!
> >
> > Finding "AC_PATH_X" in "configure.scan" is probably a good indication
> > there's an RDEPENDS on "libx11". Maybe "recipetool" could check for that?
>
> *ping*!!
I hadn't missed this, I just hadn't got around to looking at it ;)
> Oddly enough, the default recipe even says:
>
> # NOTE: the following library dependencies are unknown, ignoring: X11
>
>
> Why is it ignoring x11? Coincidentally the code I'm testing this with
> has a dependency on X11 and when I try building, bitbake throws out an
> RDEPENDS warning.
It's ignoring anything that the extremely naive and incomplete mapping that I
initially added doesn't currently handle. It'll need to be extended, and X11
would obviously be something we should add straight away. I've added a general
task to expand the mapping to the todo list. FYI we are tracking more fine-
grained tasks than are covered in bugzilla in the following wiki pages
(especially since this is currently being worked on in a separate branch):
https://wiki.yoctoproject.org/wiki/Developer_Workflow_Improvements
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-11 21:35 ` Paul Eggleton
2014-12-11 23:14 ` Trevor Woerner
@ 2014-12-12 13:01 ` Otavio Salvador
1 sibling, 0 replies; 40+ messages in thread
From: Otavio Salvador @ 2014-12-12 13:01 UTC (permalink / raw)
To: Paul Eggleton; +Cc: Patches and discussions about the oe-core layer
On Thu, Dec 11, 2014 at 7:35 PM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> On Thursday 11 December 2014 14:43:40 Trevor Woerner wrote:
>> On 12/11/14 14:16, Trevor Woerner wrote:
>> > Oops! (this worked before the "git pull")
>> >
>> > $ devtool add xannounce ~/devel/yocto/build/workflow/xannounce/
>> > NOTE: Creating workspace layer in
>> > /home/trevor/devel/yocto/build/workflow/build/workspace
>> > NOTE: Recipe
>> > /home/trevor/devel/yocto/build/workflow/build/workspace/recipes/xannou
>> > nce/xannounce.bb has been automatically created; further editing may
>> > be required to make it fully functional
>> >
>> > Traceback (most recent call last):
>> > File
>> >
>> > "/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/devtool",
>> > line 274, in <module>
>> >
>> > ret = main()
>> >
>> > File
>> >
>> > "/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/devtool",
>> > line 267, in main
>> >
>> > ret = args.func(args, config, basepath, workspace)
>> >
>> > File
>> >
>> > "/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/lib/devtool
>> > /standard.py", line 68, in add
>> >
>> > _add_md5(config, args.recipename, recipefile)
>> >
>> > File
>> >
>> > "/home/trevor/devel/yocto/build/workflow/meta-poky/scripts/lib/devtool
>> > /standard.py", line 218, in _add_md5
>> >
>> > md5 = bb.utils.md5_file(filename)
>> >
>> > File
>> >
>> > "/home/trevor/devel/yocto/build/workflow/meta-poky/bitbake/lib/bb/util
>> > s.py", line 463, in md5_file
>> >
>> > with open(filename, "rb") as f:
>> > IOError: [Errno 2] No such file or directory:
>> > '/home/trevor/devel/yocto/build/workflow/build/workspace/recipes/xanno
>> > unce/xannounce.bb'>
>> > Sure enough...
>> >
>> > $ devtool-workflow> tree workspace/
>> > workspace/
>> >
>> > |-- appends
>> > |-- conf
>> > |
>> > | `-- layer.conf
>> >
>> > `-- recipes
>> >
>> > `-- xannounce
>> >
>> > 4 directories, 1 file
>> >
>> > whereas before it was:
>> > $ tree workspace/
>> > workspace/
>> >
>> > |-- appends
>> > |
>> > | `-- xannounce.bbappend
>> > |
>> > |-- conf
>> > |
>> > | `-- layer.conf
>> >
>> > `-- recipes
>> >
>> > `-- xannounce
>> >
>> > `-- xannounce.bb
>> >
>> > 4 directories, 3 files
>> >
>> > For some reason my recipe and bbappends are not getting created :-(
>>
>> bisecting (manually) leads to this commit being safe:
>>
>> ea50ec527483eb3a094588858baebd5fb23f71c6 devtool: rename deploy/undeploy
>> to deploy-target/undeploy-target
>>
>> and this one introducing the problem:
>>
>> 770f203f73f358ce3c6c5ac12fe2775b812bdc6b devtool: fool bitbake into
>> thinking it's outputting to a terminal
>
> Thanks for the quick bisect - I've fixed it so that wrapper is only used when
> running bitbake through "devtool build", and now "devtool add" works properly
> again. This is why I should probably write some proper tests before trying to
> do anything further...
Agreed. I think a good unittest needs to be include sooner than later
as the more it grows, the harder will be to cover all usecases with
tests.
--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [RFC PATCH 0/7] Developer workflow improvements
2014-12-11 17:14 ` Paul Eggleton
2014-12-11 18:30 ` Trevor Woerner
@ 2014-12-14 3:47 ` Trevor Woerner
1 sibling, 0 replies; 40+ messages in thread
From: Trevor Woerner @ 2014-12-14 3:47 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
On 12/11/14 12:14, Paul Eggleton wrote:
>> It's not supposed to behave this way, although I have reproduced it here - I
>> think I hadn't actually tested rebuilding an image with a recipe in the
>> workspace added to the image. The problem is that the do_compile task is
>> being marked as nostamp, but the way bitbake behaves now, that doesn't
>> cause any dependent task signatures to be changed and thus no dependent
>> tasks get run afterwards. Either I'll have to change that or work out a
>> different way of ensuring do_compile and dependent tasks get run - either
>> way, you should only have to run a build that calls for the recipe (be it
>> the recipe itself or something that depends upon it), no cleaning/resetting
>> should be needed. I'll try to fix this tomorrow.
> OK, I've fixed this on the branch as well. You should now find that rebuilding
> the image always triggers a compile -> install -> package etc. of the
> recipe(s) in the workspace first. (It'll end up triggering other recipes to
> rebuild as well if they are in the chain; we may have to address that
> differently later for cases where that's annoying).
Ok great. I've re-tested and it all seems good.
^ permalink raw reply [flat|nested] 40+ messages in thread
end of thread, other threads:[~2014-12-14 3:47 UTC | newest]
Thread overview: 40+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-11-25 17:28 [RFC PATCH 0/7] Developer workflow improvements Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 1/7] lib/oe/patch: fall back to patch if git apply fails Paul Eggleton
2014-11-25 17:40 ` Paul Barker
2014-11-25 17:58 ` Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 2/7] lib/oe/patch: auto-commit when falling back from git am Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 3/7] lib/oe/patch: use --keep-cr with " Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 4/7] scripts/recipetool: Add a recipe auto-creation script Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 5/7] lib/oe: add recipeutils module Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 6/7] scripts/devtool: add development helper tool Paul Eggleton
2014-11-25 17:28 ` [RFC PATCH 7/7] scripts/devtool: Support deploy/undeploy function Paul Eggleton
2014-11-25 17:51 ` [RFC PATCH 0/7] Developer workflow improvements Paul Barker
2014-11-25 18:34 ` Paul Eggleton
2014-11-25 19:56 ` Trevor Woerner
2014-11-26 9:02 ` Paul Eggleton
2014-11-28 17:28 ` Trevor Woerner
2014-12-01 10:11 ` Paul Eggleton
2014-12-02 4:36 ` Trevor Woerner
2014-12-02 11:46 ` Paul Eggleton
2014-12-04 14:03 ` Trevor Woerner
2014-12-04 15:33 ` Paul Eggleton
2014-12-02 4:54 ` Trevor Woerner
2014-12-02 14:01 ` Paul Eggleton
2014-12-09 15:47 ` Trevor Woerner
2014-12-11 23:10 ` Trevor Woerner
2014-12-12 12:39 ` Paul Eggleton
[not found] ` <54866CD7.1050102@linaro.org>
[not found] ` <7839390.odV1UMkpj8@peggleto-mobl5.ger.corp.intel.com>
2014-12-09 15:00 ` Trevor Woerner
2014-12-09 15:10 ` Paul Eggleton
2014-12-09 15:39 ` Trevor Woerner
2014-12-09 16:08 ` Paul Eggleton
2014-12-09 18:58 ` Trevor Woerner
2014-12-10 17:51 ` Paul Eggleton
2014-12-11 17:14 ` Paul Eggleton
2014-12-11 18:30 ` Trevor Woerner
2014-12-11 18:55 ` Paul Eggleton
2014-12-11 19:16 ` Trevor Woerner
2014-12-11 19:43 ` Trevor Woerner
2014-12-11 21:35 ` Paul Eggleton
2014-12-11 23:14 ` Trevor Woerner
2014-12-12 13:01 ` Otavio Salvador
2014-12-14 3:47 ` Trevor Woerner
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox