* [PATCH] sanity: Improve configuration upgrade capabilities (support meta-yocto -> poky transition)
@ 2016-02-28 10:53 Richard Purdie
2016-02-28 12:20 ` Andreas Müller
0 siblings, 1 reply; 5+ messages in thread
From: Richard Purdie @ 2016-02-28 10:53 UTC (permalink / raw)
To: openembedded-core
Right now, only one configuration file can be processed (conf/bblayers.conf)
and it can only have one version number. This is a cause of immense friction
between OE-Core and Poky since if one needs a version change, it shouldn't
be forced on the other.
We'd like to rename the meta-yocto layer (within the meta-yocto repository)
to meta-poky. To do this, we need to correct the bblayers.conf file and that
means changing the sanity version. After the pain this caused the last time,
Paul made me promise never to have them out of sync between OE-Core and Poky,
equally, having every distro changing config update OE-Core isn't scalable
either.
This patch changes the sanity upgrade method to list a more generic format:
<config file>:<current version variable name>:<required version variable name>:<upgrade function>
This in theory allows us to support upgrades to any of the core
configuration files, and allow layers to extend them as needed. Files
with the same name can be handled in different layers by setting a unique
version name variable in the file itself. The upgrade code is only called
if the version variable is set.
To allow us to make the poky name change and use a new configuration file
name, one last version bump is included for poky to handle the transition.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
index 49693e7..a0553ee 100644
--- a/meta/classes/sanity.bbclass
+++ b/meta/classes/sanity.bbclass
@@ -25,21 +25,70 @@ def sanity_conf_update(fn, lines, version_var_name, new_version):
with open(fn, "w") as f:
f.write(''.join(lines))
-# Functions added to this variable MUST throw an exception (or sys.exit()) unless they
-# successfully changed LCONF_VERSION in bblayers.conf
-BBLAYERS_CONF_UPDATE_FUNCS += "oecore_update_bblayers"
+# Functions added to this variable MUST throw a NotImplementedError exception unless
+# they successfully changed the config version in the config file. Exceptions
+# are used since exec_func doesn't handle return values.
+BBLAYERS_CONF_UPDATE_FUNCS += " \
+ conf/bblayers.conf:LCONF_VERSION:LAYER_CONF_VERSION:oecore_update_bblayers \
+ conf/local.conf:CONF_VERSION:LOCALCONF_VERSION:oecore_update_localconf \
+ conf/site.conf:SCONF_VERSION:SITE_CONF_VERSION:oecore_update_siteconf \
+"
+
+python oecore_update_localconf() {
+ # Check we are using a valid local.conf
+ current_conf = d.getVar('CONF_VERSION', True)
+ conf_version = d.getVar('LOCALCONF_VERSION', True)
+
+ failmsg = "Your version of local.conf was generated from an older/newer version of
+local.conf.sample and there have been updates made to this file. Please compare the two
+files and merge any changes before continuing.
+
+Matching the version numbers will remove this message.
+
+\"meld conf/local.conf ${COREBASE}/meta*/conf/local.conf.sample\"
+
+is a good way to visualise the changes."
+
+ raise NotImplementedError(failmsg)
+}
+
+python oecore_update_siteconf() {
+ # If we have a site.conf, check it's valid
+ current_sconf = d.getVar('SCONF_VERSION', True)
+ sconf_version = d.getVar('SITE_CONF_VERSION', True)
+
+ failmsg = "Your version of site.conf was generated from an older version of
+site.conf.sample and there have been updates made to this file. Please compare the two
+files and merge any changes before continuing.
+
+Matching the version numbers will remove this message.
+
+\"meld conf/site.conf ${COREBASE}/meta*/conf/site.conf.sample\"
+
+is a good way to visualise the changes."
+
+ raise NotImplementedError(failmsg)
+}
python oecore_update_bblayers() {
# bblayers.conf is out of date, so see if we can resolve that
current_lconf = int(d.getVar('LCONF_VERSION', True))
- if not current_lconf:
- sys.exit()
lconf_version = int(d.getVar('LAYER_CONF_VERSION', True))
+
+ failmsg = """Your version of bblayers.conf has the wrong LCONF_VERSION (has %s, expecting %s).
+Please compare the your file against bblayers.conf.sample and merge any changes before continuing.
+"meld conf/bblayers.conf ${COREBASE}/meta*/conf/bblayers.conf.sample"
+
+is a good way to visualise the changes.""" % (current_lconf, lconf_version)
+
+ if not current_lconf:
+ raise NotImplementedError(failmsg)
+
lines = []
if current_lconf < 4:
- sys.exit()
+ raise NotImplementedError(failmsg)
bblayers_fn = bblayers_conf_file(d)
lines = sanity_conf_read(bblayers_fn)
@@ -58,13 +107,13 @@ python oecore_update_bblayers() {
lines[index] = (bbpath_line[:start + 1] +
topdir_var + ':' + bbpath_line[start + 1:])
else:
- sys.exit()
+ raise NotImplementedError(failmsg)
else:
index, bbfiles_line = sanity_conf_find_line('BBFILES', lines)
if bbfiles_line:
lines.insert(index, 'BBPATH = "' + topdir_var + '"\n')
else:
- sys.exit()
+ raise NotImplementedError(failmsg)
current_lconf += 1
sanity_conf_update(bblayers_fn, lines, 'LCONF_VERSION', current_lconf)
@@ -76,7 +125,33 @@ python oecore_update_bblayers() {
sanity_conf_update(bblayers_fn, lines, 'LCONF_VERSION', current_lconf)
return
- sys.exit()
+ if not status.reparse:
+ status.addresult()
+
+ elif current_lconf == 6 and lconf_version > 6:
+ # Handle rename of meta-yocto -> meta-poky
+ # This marks the start of separate version numbers but code is needed in OE-Core
+ # for the migration, one last time.
+ layers = d.getVar('BBLAYERS', True)
+ if 'meta-yocto' in layers:
+ index, meta_yocto_line = sanity_conf_find_line('.*meta-yocto.*\n', lines)
+ if meta_yocto_line:
+ lines[index] = meta_yocto_line.replace('meta-yocto', 'meta-poky')
+ else:
+ raise NotImplementedError(failmsg)
+ index, meta_yocto_line = sanity_conf_find_line('LCONF_VERSION.*\n', lines)
+ if meta_yocto_line:
+ lines[index] = 'POKY_BBLAYERS_CONF_VERSION = "1"\n'
+ else:
+ raise NotImplementedError(failmsg)
+ with open(bblayers_fn, "w") as f:
+ f.write(''.join(lines))
+ return
+
+ sanity_conf_update(bblayers_fn, lines, 'LCONF_VERSION', "7")
+ return
+
+ raise NotImplementedError(failmsg)
}
def raise_sanity_error(msg, d, network_error=False):
@@ -463,37 +538,20 @@ def check_perl_modules(sanity_data):
return None
def sanity_check_conffiles(status, d):
- # Check we are using a valid local.conf
- current_conf = d.getVar('CONF_VERSION', True)
- conf_version = d.getVar('LOCALCONF_VERSION', True)
-
- if current_conf != conf_version:
- status.addresult("Your version of local.conf was generated from an older/newer version of local.conf.sample and there have been updates made to this file. Please compare the two files and merge any changes before continuing.\nMatching the version numbers will remove this message.\n\"meld conf/local.conf ${COREBASE}/meta*/conf/local.conf.sample\" is a good way to visualise the changes.\n")
-
- # Check bblayers.conf is valid
- current_lconf = d.getVar('LCONF_VERSION', True)
- lconf_version = d.getVar('LAYER_CONF_VERSION', True)
- if current_lconf != lconf_version:
- funcs = d.getVar('BBLAYERS_CONF_UPDATE_FUNCS', True).split()
- for func in funcs:
+ funcs = d.getVar('BBLAYERS_CONF_UPDATE_FUNCS', True).split()
+ for func in funcs:
+ conffile, current_version, required_version, func = func.split(":")
+ if check_conf_exists(conffile, d) and d.getVar(current_version, True) is not None and \
+ d.getVar(current_version, True) != d.getVar(required_version, True):
success = True
try:
bb.build.exec_func(func, d)
- except Exception:
+ except NotImplementedError as e:
success = False
+ status.addresult(e.msg)
if success:
bb.note("Your conf/bblayers.conf has been automatically updated.")
status.reparse = True
- if not status.reparse:
- status.addresult("Your version of bblayers.conf has the wrong LCONF_VERSION (has %s, expecting %s).\nPlease compare the your file against bblayers.conf.sample and merge any changes before continuing.\n\"meld conf/bblayers.conf ${COREBASE}/meta*/conf/bblayers.conf.sample\" is a good way to visualise the changes.\n" % (current_lconf, lconf_version))
-
- # If we have a site.conf, check it's valid
- if check_conf_exists("conf/site.conf", d):
- current_sconf = d.getVar('SCONF_VERSION', True)
- sconf_version = d.getVar('SITE_CONF_VERSION', True)
- if current_sconf != sconf_version:
- status.addresult("Your version of site.conf was generated from an older version of site.conf.sample and there have been updates made to this file. Please compare the two files and merge any changes before continuing.\nMatching the version numbers will remove this message.\n\"meld conf/site.conf ${COREBASE}/meta*/conf/site.conf.sample\" is a good way to visualise the changes.\n")
-
def sanity_handle_abichanges(status, d):
#
diff --git a/meta/conf/sanity.conf b/meta/conf/sanity.conf
index 0f852d1..0d20c99 100644
--- a/meta/conf/sanity.conf
+++ b/meta/conf/sanity.conf
@@ -9,7 +9,7 @@ SANITY_ABIFILE = "${TMPDIR}/abi_version"
SANITY_VERSION ?= "1"
LOCALCONF_VERSION ?= "1"
-LAYER_CONF_VERSION ?= "6"
+LAYER_CONF_VERSION ?= "7"
SITE_CONF_VERSION ?= "1"
INHERIT += "sanity"
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH] sanity: Improve configuration upgrade capabilities (support meta-yocto -> poky transition)
2016-02-28 10:53 [PATCH] sanity: Improve configuration upgrade capabilities (support meta-yocto -> poky transition) Richard Purdie
@ 2016-02-28 12:20 ` Andreas Müller
2016-02-28 15:59 ` Richard Purdie
0 siblings, 1 reply; 5+ messages in thread
From: Andreas Müller @ 2016-02-28 12:20 UTC (permalink / raw)
To: Richard Purdie; +Cc: openembedded-core
On Sun, Feb 28, 2016 at 11:53 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> Right now, only one configuration file can be processed (conf/bblayers.conf)
> and it can only have one version number. This is a cause of immense friction
> between OE-Core and Poky since if one needs a version change, it shouldn't
> be forced on the other.
>
> We'd like to rename the meta-yocto layer (within the meta-yocto repository)
> to meta-poky. To do this, we need to correct the bblayers.conf file and that
> means changing the sanity version. After the pain this caused the last time,
> Paul made me promise never to have them out of sync between OE-Core and Poky,
> equally, having every distro changing config update OE-Core isn't scalable
> either.
>
> This patch changes the sanity upgrade method to list a more generic format:
>
> <config file>:<current version variable name>:<required version variable name>:<upgrade function>
>
> This in theory allows us to support upgrades to any of the core
> configuration files, and allow layers to extend them as needed. Files
> with the same name can be handled in different layers by setting a unique
> version name variable in the file itself. The upgrade code is only called
> if the version variable is set.
>
> To allow us to make the poky name change and use a new configuration file
> name, one last version bump is included for poky to handle the transition.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>
> diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
> index 49693e7..a0553ee 100644
> --- a/meta/classes/sanity.bbclass
> +++ b/meta/classes/sanity.bbclass
I think this breaks parsing. Have reverted it and can build again. I got
[superandy@localhost oe-core]$ bitbake -k xfce4-games-image
ERROR: Error executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:oecore_update_bblayers(d)
0003:
File: '/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass',
lineno: 151, function: oecore_update_bblayers
0147: with open(bblayers_fn, "w") as f:
0148: f.write(''.join(lines))
0149: return
0150:
*** 0151: sanity_conf_update(bblayers_fn, lines, 'LCONF_VERSION', "7")
0152: return
0153:
0154: raise NotImplementedError(failmsg)
0155:}
File: '/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass',
lineno: 24, function: sanity_conf_update
0020: if re.search(pattern, line)), (None, None))
0021:
0022:def sanity_conf_update(fn, lines, version_var_name, new_version):
0023: index, line = sanity_conf_find_line(version_var_name, lines)
*** 0024: lines[index] = '%s = "%d"\n' % (version_var_name, new_version)
0025: with open(fn, "w") as f:
0026: f.write(''.join(lines))
0027:
0028:# Functions added to this variable MUST throw a
NotImplementedError exception unless
Exception: TypeError: %d format: a number is required, not str
ERROR: Execution of event handler 'check_sanity_eventhandler' failed
Traceback (most recent call last):
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 954, in check_sanity(sanity_data=<bb.data_smart.DataSmart object
at 0x7f932aa55890>):
> check_sanity_everybuild(status, sanity_data)
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 768, in check_sanity_everybuild(status=<SanityStatus object at
0x7f932aca3810>, d=<bb.data_smart.DataSmart object at
0x7f932aa55890>):
> sanity_check_conffiles(status, d)
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 548, in sanity_check_conffiles(status=<SanityStatus object at
0x7f932aca3810>, d=<bb.data_smart.DataSmart object at
0x7f932aa55890>):
try:
> bb.build.exec_func(func, d)
except NotImplementedError as e:
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 227, in exec_func(func='oecore_update_bblayers',
d=<bb.data_smart.DataSmart object at 0x7f932aa55890>, dirs=None):
if ispython:
> exec_func_python(func, d, runfile, cwd=adir)
else:
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 263, in exec_func_python(func='oecore_update_bblayers',
d=<bb.data_smart.DataSmart object at 0x7f932aa55890>,
runfile='/home/superandy/tmp/oe-core-glibc/work/cortexa7t2hf-neon-vfpv4-angstrom-linux-gnueabi/defaultpkgname/1.0-r0/temp/run.oecore_update_bblayers.4316',
cwd='/home/superandy/tmp/oe-core-glibc/work/cortexa7t2hf-neon-vfpv4-angstrom-linux-gnueabi/defaultpkgname/1.0-r0/defaultpkgname-1.0'):
except:
> raise FuncFailed(func, None)
finally:
FuncFailed: Function failed: oecore_update_bblayers
ERROR: Command execution failed: Traceback (most recent call last):
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/command.py",
line 101, in runAsyncCommand
self.cooker.updateCache()
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/cooker.py",
line 1538, in updateCache
bb.event.fire(bb.event.SanityCheck(False), self.data)
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/event.py",
line 171, in fire
fire_class_handlers(event, d)
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/event.py",
line 110, in fire_class_handlers
execute_handler(name, handler, event, d)
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/event.py",
line 82, in execute_handler
ret = handler(event)
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 991, in check_sanity_eventhandler
reparse = check_sanity(sanity_data)
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 954, in check_sanity
check_sanity_everybuild(status, sanity_data)
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 768, in check_sanity_everybuild
sanity_check_conffiles(status, d)
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 548, in sanity_check_conffiles
bb.build.exec_func(func, d)
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 227, in exec_func
exec_func_python(func, d, runfile, cwd=adir)
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 263, in exec_func_python
raise FuncFailed(func, None)
FuncFailed: Function failed: oecore_update_bblayers
Summary: There were 3 ERROR messages shown, returning a non-zero exit code.
[superandy@localhost oe-core]$ bitbake -k xfce4-games-image
ERROR: Error executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:oecore_update_bblayers(d)
0003:
File: '/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass',
lineno: 151, function: oecore_update_bblayers
0147: with open(bblayers_fn, "w") as f:
0148: f.write(''.join(lines))
0149: return
0150:
*** 0151: sanity_conf_update(bblayers_fn, lines, 'LCONF_VERSION', "7")
0152: return
0153:
0154: raise NotImplementedError(failmsg)
0155:}
File: '/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass',
lineno: 24, function: sanity_conf_update
0020: if re.search(pattern, line)), (None, None))
0021:
0022:def sanity_conf_update(fn, lines, version_var_name, new_version):
0023: index, line = sanity_conf_find_line(version_var_name, lines)
*** 0024: lines[index] = '%s = "%d"\n' % (version_var_name, new_version)
0025: with open(fn, "w") as f:
0026: f.write(''.join(lines))
0027:
0028:# Functions added to this variable MUST throw a
NotImplementedError exception unless
Exception: TypeError: %d format: a number is required, not str
ERROR: Execution of event handler 'check_sanity_eventhandler' failed
Traceback (most recent call last):
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 954, in check_sanity(sanity_data=<bb.data_smart.DataSmart object
at 0x7f2bb26f3cd0>):
> check_sanity_everybuild(status, sanity_data)
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 768, in check_sanity_everybuild(status=<SanityStatus object at
0x7f2bb26f31d0>, d=<bb.data_smart.DataSmart object at
0x7f2bb26f3cd0>):
> sanity_check_conffiles(status, d)
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 548, in sanity_check_conffiles(status=<SanityStatus object at
0x7f2bb26f31d0>, d=<bb.data_smart.DataSmart object at
0x7f2bb26f3cd0>):
try:
> bb.build.exec_func(func, d)
except NotImplementedError as e:
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 227, in exec_func(func='oecore_update_bblayers',
d=<bb.data_smart.DataSmart object at 0x7f2bb26f3cd0>, dirs=None):
if ispython:
> exec_func_python(func, d, runfile, cwd=adir)
else:
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 263, in exec_func_python(func='oecore_update_bblayers',
d=<bb.data_smart.DataSmart object at 0x7f2bb26f3cd0>,
runfile='/home/superandy/tmp/oe-core-glibc/work/cortexa7t2hf-neon-vfpv4-angstrom-linux-gnueabi/defaultpkgname/1.0-r0/temp/run.oecore_update_bblayers.4459',
cwd='/home/superandy/tmp/oe-core-glibc/work/cortexa7t2hf-neon-vfpv4-angstrom-linux-gnueabi/defaultpkgname/1.0-r0/defaultpkgname-1.0'):
except:
> raise FuncFailed(func, None)
finally:
FuncFailed: Function failed: oecore_update_bblayers
ERROR: Command execution failed: Traceback (most recent call last):
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/command.py",
line 101, in runAsyncCommand
self.cooker.updateCache()
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/cooker.py",
line 1538, in updateCache
bb.event.fire(bb.event.SanityCheck(False), self.data)
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/event.py",
line 171, in fire
fire_class_handlers(event, d)
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/event.py",
line 110, in fire_class_handlers
execute_handler(name, handler, event, d)
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/event.py",
line 82, in execute_handler
ret = handler(event)
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 991, in check_sanity_eventhandler
reparse = check_sanity(sanity_data)
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 954, in check_sanity
check_sanity_everybuild(status, sanity_data)
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 768, in check_sanity_everybuild
sanity_check_conffiles(status, d)
File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 548, in sanity_check_conffiles
bb.build.exec_func(func, d)
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 227, in exec_func
exec_func_python(func, d, runfile, cwd=adir)
File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 263, in exec_func_python
raise FuncFailed(func, None)
FuncFailed: Function failed: oecore_update_bblayers
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] sanity: Improve configuration upgrade capabilities (support meta-yocto -> poky transition)
2016-02-28 12:20 ` Andreas Müller
@ 2016-02-28 15:59 ` Richard Purdie
2016-02-28 21:11 ` Martin Jansa
0 siblings, 1 reply; 5+ messages in thread
From: Richard Purdie @ 2016-02-28 15:59 UTC (permalink / raw)
To: Andreas Müller; +Cc: openembedded-core
On Sun, 2016-02-28 at 13:20 +0100, Andreas Müller wrote:
> On Sun, Feb 28, 2016 at 11:53 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > Right now, only one configuration file can be processed
> > (conf/bblayers.conf)
> > and it can only have one version number. This is a cause of immense
> > friction
> > between OE-Core and Poky since if one needs a version change, it
> > shouldn't
> > be forced on the other.
> >
> > We'd like to rename the meta-yocto layer (within the meta-yocto
> > repository)
> > to meta-poky. To do this, we need to correct the bblayers.conf file
> > and that
> > means changing the sanity version. After the pain this caused the
> > last time,
> > Paul made me promise never to have them out of sync between OE-Core
> > and Poky,
> > equally, having every distro changing config update OE-Core isn't
> > scalable
> > either.
> >
> > This patch changes the sanity upgrade method to list a more generic
> > format:
> >
> > <config file>:<current version variable name>:<required version
> > variable name>:<upgrade function>
> >
> > This in theory allows us to support upgrades to any of the core
> > configuration files, and allow layers to extend them as needed.
> > Files
> > with the same name can be handled in different layers by setting a
> > unique
> > version name variable in the file itself. The upgrade code is only
> > called
> > if the version variable is set.
> >
> > To allow us to make the poky name change and use a new
> > configuration file
> > name, one last version bump is included for poky to handle the
> > transition.
> >
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> >
> > diff --git a/meta/classes/sanity.bbclass
> > b/meta/classes/sanity.bbclass
> > index 49693e7..a0553ee 100644
> > --- a/meta/classes/sanity.bbclass
> > +++ b/meta/classes/sanity.bbclass
>
> I think this breaks parsing. Have reverted it and can build again.
Sorry, I missed a version number change. I've pushed a fix.
Cheers,
Richard
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] sanity: Improve configuration upgrade capabilities (support meta-yocto -> poky transition)
2016-02-28 15:59 ` Richard Purdie
@ 2016-02-28 21:11 ` Martin Jansa
2016-02-28 22:54 ` Richard Purdie
0 siblings, 1 reply; 5+ messages in thread
From: Martin Jansa @ 2016-02-28 21:11 UTC (permalink / raw)
To: Richard Purdie; +Cc: openembedded-core
[-- Attachment #1: Type: text/plain, Size: 7896 bytes --]
On Sun, Feb 28, 2016 at 03:59:42PM +0000, Richard Purdie wrote:
> On Sun, 2016-02-28 at 13:20 +0100, Andreas Müller wrote:
> > On Sun, Feb 28, 2016 at 11:53 AM, Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > > Right now, only one configuration file can be processed
> > > (conf/bblayers.conf)
> > > and it can only have one version number. This is a cause of immense
> > > friction
> > > between OE-Core and Poky since if one needs a version change, it
> > > shouldn't
> > > be forced on the other.
> > >
> > > We'd like to rename the meta-yocto layer (within the meta-yocto
> > > repository)
> > > to meta-poky. To do this, we need to correct the bblayers.conf file
> > > and that
> > > means changing the sanity version. After the pain this caused the
> > > last time,
> > > Paul made me promise never to have them out of sync between OE-Core
> > > and Poky,
> > > equally, having every distro changing config update OE-Core isn't
> > > scalable
> > > either.
> > >
> > > This patch changes the sanity upgrade method to list a more generic
> > > format:
> > >
> > > <config file>:<current version variable name>:<required version
> > > variable name>:<upgrade function>
> > >
> > > This in theory allows us to support upgrades to any of the core
> > > configuration files, and allow layers to extend them as needed.
> > > Files
> > > with the same name can be handled in different layers by setting a
> > > unique
> > > version name variable in the file itself. The upgrade code is only
> > > called
> > > if the version variable is set.
> > >
> > > To allow us to make the poky name change and use a new
> > > configuration file
> > > name, one last version bump is included for poky to handle the
> > > transition.
> > >
> > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > >
> > > diff --git a/meta/classes/sanity.bbclass
> > > b/meta/classes/sanity.bbclass
> > > index 49693e7..a0553ee 100644
> > > --- a/meta/classes/sanity.bbclass
> > > +++ b/meta/classes/sanity.bbclass
> >
> > I think this breaks parsing. Have reverted it and can build again.
>
> Sorry, I missed a version number change. I've pushed a fix.
Even with:
commit 933088c74870b8c3aa5077d57bc85fc66a652291
Author: Richard Purdie <richard.purdie@linuxfoundation.org>
Date: Sun Feb 28 15:57:58 2016 +0000
bblayers.conf.sample: Fix missing layer version bump
The sanity.bbclass changes required the layer verison to increase,
this adds that missing component.
I'm still seeing this exception:
NOTE: Started PRServer with DBfile: /home/jenkins/oe/world/shr-core/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 42730, PID: 10040
ERROR: Error executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:oecore_update_bblayers(d)
0003:
File: '/home/jenkins/oe/world/shr-core/openembedded-core/meta/classes/sanity.bbclass', lineno: 151, function: oecore_update_bblayers
0147: with open(bblayers_fn, "w") as f:
0148: f.write(''.join(lines))
0149: return
0150:
*** 0151: sanity_conf_update(bblayers_fn, lines, 'LCONF_VERSION', "7")
0152: return
0153:
0154: raise NotImplementedError(failmsg)
0155:}
File: '/home/jenkins/oe/world/shr-core/openembedded-core/meta/classes/sanity.bbclass', lineno: 24, function: sanity_conf_update
0020: if re.search(pattern, line)), (None, None))
0021:
0022:def sanity_conf_update(fn, lines, version_var_name, new_version):
0023: index, line = sanity_conf_find_line(version_var_name, lines)
*** 0024: lines[index] = '%s = "%d"\n' % (version_var_name, new_version)
0025: with open(fn, "w") as f:
0026: f.write(''.join(lines))
0027:
0028:# Functions added to this variable MUST throw a NotImplementedError exception unless
Exception: TypeError: %d format: a number is required, not str
ERROR: Execution of event handler 'check_sanity_eventhandler' failed
Traceback (most recent call last):
File "/home/jenkins/oe/world/shr-core/openembedded-core/meta/classes/sanity.bbclass", line 954, in check_sanity(sanity_data=<bb.data_smart.DataSmart object at 0x7fb069f71110>):
> check_sanity_everybuild(status, sanity_data)
File "/home/jenkins/oe/world/shr-core/openembedded-core/meta/classes/sanity.bbclass", line 768, in check_sanity_everybuild(status=<SanityStatus object at 0x7fb069f71090>, d=<bb.data_smart.DataSmart object at 0x7fb069f71110>):
> sanity_check_conffiles(status, d)
File "/home/jenkins/oe/world/shr-core/openembedded-core/meta/classes/sanity.bbclass", line 548, in sanity_check_conffiles(status=<SanityStatus object at 0x7fb069f71090>, d=<bb.data_smart.DataSmart object at 0x7fb069f71110>):
try:
> bb.build.exec_func(func, d)
except NotImplementedError as e:
File "/home/jenkins/oe/world/shr-core/bitbake/lib/bb/build.py", line 227, in exec_func(func='oecore_update_bblayers', d=<bb.data_smart.DataSmart object at 0x7fb069f71110>, dirs=None):
if ispython:
> exec_func_python(func, d, runfile, cwd=adir)
else:
File "/home/jenkins/oe/world/shr-core/bitbake/lib/bb/build.py", line 263, in exec_func_python(func='oecore_update_bblayers', d=<bb.data_smart.DataSmart object at 0x7fb069f71110>, runfile='/home/jenkins/oe/world/shr-core/tmp-glibc/work/i586-oe-linux/defaultpkgname/1.0-r0/temp/run.oecore_update_bblayers.10038', cwd='/home/jenkins/oe/world/shr-core/tmp-glibc/work/i586-oe-linux/defaultpkgname/1.0-r0/defaultpkgname-1.0'):
except:
> raise FuncFailed(func, None)
finally:
FuncFailed: Function failed: oecore_update_bblayers
ERROR: Command execution failed: Traceback (most recent call last):
File "/home/jenkins/oe/world/shr-core/bitbake/lib/bb/command.py", line 101, in runAsyncCommand
self.cooker.updateCache()
File "/home/jenkins/oe/world/shr-core/bitbake/lib/bb/cooker.py", line 1586, in updateCache
bb.event.fire(bb.event.SanityCheck(False), self.data)
File "/home/jenkins/oe/world/shr-core/bitbake/lib/bb/event.py", line 171, in fire
fire_class_handlers(event, d)
File "/home/jenkins/oe/world/shr-core/bitbake/lib/bb/event.py", line 110, in fire_class_handlers
execute_handler(name, handler, event, d)
File "/home/jenkins/oe/world/shr-core/bitbake/lib/bb/event.py", line 82, in execute_handler
ret = handler(event)
File "/home/jenkins/oe/world/shr-core/openembedded-core/meta/classes/sanity.bbclass", line 991, in check_sanity_eventhandler
reparse = check_sanity(sanity_data)
File "/home/jenkins/oe/world/shr-core/openembedded-core/meta/classes/sanity.bbclass", line 954, in check_sanity
check_sanity_everybuild(status, sanity_data)
File "/home/jenkins/oe/world/shr-core/openembedded-core/meta/classes/sanity.bbclass", line 768, in check_sanity_everybuild
sanity_check_conffiles(status, d)
File "/home/jenkins/oe/world/shr-core/openembedded-core/meta/classes/sanity.bbclass", line 548, in sanity_check_conffiles
bb.build.exec_func(func, d)
File "/home/jenkins/oe/world/shr-core/bitbake/lib/bb/build.py", line 227, in exec_func
exec_func_python(func, d, runfile, cwd=adir)
File "/home/jenkins/oe/world/shr-core/bitbake/lib/bb/build.py", line 263, in exec_func_python
raise FuncFailed(func, None)
FuncFailed: Function failed: oecore_update_bblayers
Summary: There were 3 ERROR messages shown, returning a non-zero exit code.
--
Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] sanity: Improve configuration upgrade capabilities (support meta-yocto -> poky transition)
2016-02-28 21:11 ` Martin Jansa
@ 2016-02-28 22:54 ` Richard Purdie
0 siblings, 0 replies; 5+ messages in thread
From: Richard Purdie @ 2016-02-28 22:54 UTC (permalink / raw)
To: Martin Jansa; +Cc: openembedded-core
On Sun, 2016-02-28 at 22:11 +0100, Martin Jansa wrote:
> Even with:
> commit 933088c74870b8c3aa5077d57bc85fc66a652291
> Author: Richard Purdie <richard.purdie@linuxfoundation.org>
> Date: Sun Feb 28 15:57:58 2016 +0000
>
> bblayers.conf.sample: Fix missing layer version bump
>
> The sanity.bbclass changes required the layer verison to
> increase,
> this adds that missing component.
>
> I'm still seeing this exception:
>
> NOTE: Started PRServer with DBfile: /home/jenkins/oe/world/shr
> -core/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 42730, PID: 10040
> ERROR: Error executing a python function in exec_python_func()
> autogenerated:
Sorry, hopefully really fixed this time.
Cheers,
Richard
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2016-02-28 22:54 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-02-28 10:53 [PATCH] sanity: Improve configuration upgrade capabilities (support meta-yocto -> poky transition) Richard Purdie
2016-02-28 12:20 ` Andreas Müller
2016-02-28 15:59 ` Richard Purdie
2016-02-28 21:11 ` Martin Jansa
2016-02-28 22:54 ` Richard Purdie
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.