* [yocto-autobuilder][PATCH 0/4] autobuilder: make toaster eventlog creation optional
@ 2016-07-14 17:05 Bill Randle
2016-07-14 17:05 ` [yocto-autobuilder][PATCH 1/4] remote_kick.py: add default entry for new 'create_eventlog' property Bill Randle
` (4 more replies)
0 siblings, 5 replies; 9+ messages in thread
From: Bill Randle @ 2016-07-14 17:05 UTC (permalink / raw)
To: yocto
Currently, a toaster eventlog is generated for most all machine specific
images (e.g., nightly-arm, nightly-x86, etc.) and placed in the top-level
build directory. Having it in the top level dir causes inotify overflow in
bitbake, due to the large number of changes to the file.
This set of patches (1) make generation of the eventlog optional, by
having a user selection in the Forced Build web pages, (2) places the
eventlog in build/tmp/log, instead of build and (3) adds a timestamp to
the eventlog filename since multiple event logs can be generated during
a multi-image buildset and otherwise only the last one would otherwise
be saved.
[YOCTO #9884]
Bill Randle (4):
remote_kick.py: add default entry for new 'create_eventlog' property
BuildImages.py: build toaster eventlog only when requested
UploadToasterEventlog.py: handle multiple eventlog files
buildset-config.controller: add new 'create_eventlog' forced build
property
bin/remote_kick.py | 1 +
buildset-config.controller/nightly-arm-lsb.conf | 4 ++++
buildset-config.controller/nightly-arm.conf | 4 ++++
buildset-config.controller/nightly-arm64.conf | 4 ++++
buildset-config.controller/nightly-mips-lsb.conf | 4 ++++
buildset-config.controller/nightly-mips.conf | 4 ++++
buildset-config.controller/nightly-mips64.conf | 4 ++++
buildset-config.controller/nightly-no-x11.conf | 4 ++++
buildset-config.controller/nightly-ppc-lsb.conf | 4 ++++
buildset-config.controller/nightly-ppc.conf | 4 ++++
buildset-config.controller/nightly-x32.conf | 4 ++++
buildset-config.controller/nightly-x86-64-lsb.conf | 4 ++++
buildset-config.controller/nightly-x86-64.conf | 4 ++++
buildset-config.controller/nightly-x86-lsb.conf | 4 ++++
buildset-config.controller/nightly-x86.conf | 4 ++++
buildset-config.controller/nightly.conf | 7 ++++++-
.../autobuilder/buildsteps/BuildImages.py | 24 +++++++++++++++++-----
.../buildsteps/UploadToasterEventlog.py | 16 +++++++++++-----
18 files changed, 93 insertions(+), 11 deletions(-)
--
2.5.5
^ permalink raw reply [flat|nested] 9+ messages in thread* [yocto-autobuilder][PATCH 1/4] remote_kick.py: add default entry for new 'create_eventlog' property 2016-07-14 17:05 [yocto-autobuilder][PATCH 0/4] autobuilder: make toaster eventlog creation optional Bill Randle @ 2016-07-14 17:05 ` Bill Randle 2016-07-14 17:05 ` [yocto-autobuilder][PATCH 2/4] BuildImages.py: build toaster eventlog only when requested Bill Randle ` (3 subsequent siblings) 4 siblings, 0 replies; 9+ messages in thread From: Bill Randle @ 2016-07-14 17:05 UTC (permalink / raw) To: yocto Default eventlog creating to False when starting an AB build session from the command line. [YOCTO #9884] Signed-off-by: Bill Randle <william.c.randle@intel.com> --- bin/remote_kick.py | 1 + 1 file changed, 1 insertion(+) diff --git a/bin/remote_kick.py b/bin/remote_kick.py index 20a7b7e..7320162 100644 --- a/bin/remote_kick.py +++ b/bin/remote_kick.py @@ -44,6 +44,7 @@ class RemoteBuild(): kw['branch_eclipse-poky-luna'] = 'luna-master' kw['branch_eclipse-poky-mars'] = 'mars-master' kw['branch_oecore'] = 'master' + kw['custom_create_eventlog'] = 'False' kw['custom_deploy_artifacts'] = 'False' kw['custom_is_milestone'] = 'False' kw['custom_milestone_number'] = '' -- 2.5.5 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [yocto-autobuilder][PATCH 2/4] BuildImages.py: build toaster eventlog only when requested 2016-07-14 17:05 [yocto-autobuilder][PATCH 0/4] autobuilder: make toaster eventlog creation optional Bill Randle 2016-07-14 17:05 ` [yocto-autobuilder][PATCH 1/4] remote_kick.py: add default entry for new 'create_eventlog' property Bill Randle @ 2016-07-14 17:05 ` Bill Randle 2016-07-15 21:37 ` Joshua G Lock 2016-07-14 17:05 ` [yocto-autobuilder][PATCH 3/4] UploadToasterEventlog.py: handle multiple eventlog files Bill Randle ` (2 subsequent siblings) 4 siblings, 1 reply; 9+ messages in thread From: Bill Randle @ 2016-07-14 17:05 UTC (permalink / raw) To: yocto Qualify creation of the toaster eventlog based on the setting of the new 'custom_create_eventlog' property. Place the eventlog in build/tmp/log, instead of build and add a timestamp to the eventlog filename since multiple event logs can be generated during a multi-image buildset and otherwise only the last one would otherwise be saved. [YOCTO #9884] Signed-off-by: Bill Randle <william.c.randle@intel.com> --- .../autobuilder/buildsteps/BuildImages.py | 24 +++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/lib/python2.7/site-packages/autobuilder/buildsteps/BuildImages.py b/lib/python2.7/site-packages/autobuilder/buildsteps/BuildImages.py index 7ef8aab..9ea350f 100644 --- a/lib/python2.7/site-packages/autobuilder/buildsteps/BuildImages.py +++ b/lib/python2.7/site-packages/autobuilder/buildsteps/BuildImages.py @@ -15,7 +15,7 @@ from buildbot.steps.shell import ShellCommand from buildbot.process.buildstep import LogLineObserver from distutils.version import StrictVersion from buildbot.status.results import SUCCESS, SKIPPED -import os +import os, datetime from lib.buildsteps import BitbakeShellCommand @@ -47,8 +47,23 @@ class BuildImages(BitbakeShellCommand): self.deploycheck = self.getProperty('custom_deploy_artifacts') except: self.deploycheck = "True" + + # the eventlog capability exists only in bitbake 1.25 and newer + self.create_eventlog = "False" + if self.getProperty('bitbakeversion') \ + and StrictVersion(self.getProperty('bitbakeversion')) >= StrictVersion("1.25"): + try: + self.create_eventlog = self.getProperty("custom_create_eventlog") + if self.create_eventlog == "True": + timestamp = datetime.datetime.now().strftime("%Y%m%d%H%M%S") + self.eventlog_name = "tmp/log/bitbake_eventlog-%s.json" % timestamp + except: + pass + if self.images == "#TOASTER": - bitbakeflags = "-k -w 'bitbake_eventlog.json' " + bitbakeflags = "-k " + if self.create_eventlog == "True": + bitbakeflags += "-w '" + self.eventlog_name + "' " self.images=self.getProperty("custom_images") self.command = ". ./oe-init-build-env; bitbake " + bitbakeflags + self.images self.description = ["Building " + str(self.images)] @@ -130,9 +145,8 @@ class BuildImages(BitbakeShellCommand): self.description = ["Skipping Build. No Images to be built"] else: bitbakeflags = "-k " - # -w only exists in bitbake 1.25 and newer, use distroversion string and make sure we're on poky >1.7 - if self.getProperty('bitbakeversion') and StrictVersion(self.getProperty('bitbakeversion')) >= StrictVersion("1.25"): - bitbakeflags += "-w 'bitbake_eventlog.json' " + if self.create_eventlog == "True": + bitbakeflags += "-w '" + self.eventlog_name + "' " if self.minnowExists is None or self.minnowExists == "True": self.command = ". ./oe-init-build-env; bitbake " + bitbakeflags + self.images self.description = ["Building " + str(self.images)] -- 2.5.5 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [yocto-autobuilder][PATCH 2/4] BuildImages.py: build toaster eventlog only when requested 2016-07-14 17:05 ` [yocto-autobuilder][PATCH 2/4] BuildImages.py: build toaster eventlog only when requested Bill Randle @ 2016-07-15 21:37 ` Joshua G Lock 2016-07-15 23:54 ` Randle, William C 0 siblings, 1 reply; 9+ messages in thread From: Joshua G Lock @ 2016-07-15 21:37 UTC (permalink / raw) To: Bill Randle, yocto On Thu, 2016-07-14 at 10:05 -0700, Bill Randle wrote: > Qualify creation of the toaster eventlog based on the setting of the > new > 'custom_create_eventlog' property. Place the eventlog in > build/tmp/log, > instead of build and add a timestamp to the eventlog filename since > multiple event logs can be generated during a multi-image buildset > and > otherwise only the last one would otherwise be saved. > > [YOCTO #9884] > > Signed-off-by: Bill Randle <william.c.randle@intel.com> > --- > .../autobuilder/buildsteps/BuildImages.py | 24 > +++++++++++++++++----- > 1 file changed, 19 insertions(+), 5 deletions(-) > > diff --git a/lib/python2.7/site- > packages/autobuilder/buildsteps/BuildImages.py b/lib/python2.7/site- > packages/autobuilder/buildsteps/BuildImages.py > index 7ef8aab..9ea350f 100644 > --- a/lib/python2.7/site- > packages/autobuilder/buildsteps/BuildImages.py > +++ b/lib/python2.7/site- > packages/autobuilder/buildsteps/BuildImages.py > @@ -15,7 +15,7 @@ from buildbot.steps.shell import ShellCommand > from buildbot.process.buildstep import LogLineObserver > from distutils.version import StrictVersion > from buildbot.status.results import SUCCESS, SKIPPED > -import os > +import os, datetime > > from lib.buildsteps import BitbakeShellCommand > > @@ -47,8 +47,23 @@ class BuildImages(BitbakeShellCommand): > self.deploycheck = > self.getProperty('custom_deploy_artifacts') > except: > self.deploycheck = "True" > + > + # the eventlog capability exists only in bitbake 1.25 and > newer > + self.create_eventlog = "False" > + if self.getProperty('bitbakeversion') \ > + and StrictVersion(self.getProperty('bitbakeversion')) >= > StrictVersion("1.25"): > + try: > + self.create_eventlog = > self.getProperty("custom_create_eventlog") > + if self.create_eventlog == "True": > + timestamp = > datetime.datetime.now().strftime("%Y%m%d%H%M%S") > + self.eventlog_name = "tmp/log/bitbake_eventlog- > %s.json" % timestamp > + except: > + pass Do we really want to catch all exceptions here and ignore them? What exception might be thrown by the above code? > + > if self.images == "#TOASTER": > - bitbakeflags = "-k -w 'bitbake_eventlog.json' " > + bitbakeflags = "-k " > + if self.create_eventlog == "True": > + bitbakeflags += "-w '" + self.eventlog_name + "' " > self.images=self.getProperty("custom_images") > self.command = ". ./oe-init-build-env; bitbake " + > bitbakeflags + self.images > self.description = ["Building " + str(self.images)] > @@ -130,9 +145,8 @@ class BuildImages(BitbakeShellCommand): > self.description = ["Skipping Build. No Images to be > built"] > else: > bitbakeflags = "-k " > - # -w only exists in bitbake 1.25 and newer, use > distroversion string and make sure we're on poky >1.7 > - if self.getProperty('bitbakeversion') and > StrictVersion(self.getProperty('bitbakeversion')) >= > StrictVersion("1.25"): > - bitbakeflags += "-w 'bitbake_eventlog.json' " > + if self.create_eventlog == "True": > + bitbakeflags += "-w '" + self.eventlog_name + "' > " > if self.minnowExists is None or self.minnowExists == > "True": > self.command = ". ./oe-init-build-env; bitbake " > + bitbakeflags + self.images > self.description = ["Building " + > str(self.images)] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [yocto-autobuilder][PATCH 2/4] BuildImages.py: build toaster eventlog only when requested 2016-07-15 21:37 ` Joshua G Lock @ 2016-07-15 23:54 ` Randle, William C 2016-07-16 14:45 ` Randle, William C 0 siblings, 1 reply; 9+ messages in thread From: Randle, William C @ 2016-07-15 23:54 UTC (permalink / raw) To: joshua.g.lock@linux.intel.com, yocto@yoctoproject.org [-- Attachment #1: Type: text/plain, Size: 3881 bytes --] On Fri, 2016-07-15 at 22:37 +0100, Joshua G Lock wrote: On Thu, 2016-07-14 at 10:05 -0700, Bill Randle wrote: Qualify creation of the toaster eventlog based on the setting of the new 'custom_create_eventlog' property. Place the eventlog in build/tmp/log, instead of build and add a timestamp to the eventlog filename since multiple event logs can be generated during a multi-image buildset and otherwise only the last one would otherwise be saved. [YOCTO #9884] Signed-off-by: Bill Randle <william.c.randle@intel.com<mailto:william.c.randle@intel.com>> --- .../autobuilder/buildsteps/BuildImages.py | 24 +++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/lib/python2.7/site- packages/autobuilder/buildsteps/BuildImages.py b/lib/python2.7/site- packages/autobuilder/buildsteps/BuildImages.py index 7ef8aab..9ea350f 100644 --- a/lib/python2.7/site- packages/autobuilder/buildsteps/BuildImages.py +++ b/lib/python2.7/site- packages/autobuilder/buildsteps/BuildImages.py @@ -15,7 +15,7 @@ from buildbot.steps.shell import ShellCommand from buildbot.process.buildstep import LogLineObserver from distutils.version import StrictVersion from buildbot.status.results import SUCCESS, SKIPPED -import os +import os, datetime from lib.buildsteps import BitbakeShellCommand @@ -47,8 +47,23 @@ class BuildImages(BitbakeShellCommand): self.deploycheck = self.getProperty('custom_deploy_artifacts') except: self.deploycheck = "True" + + # the eventlog capability exists only in bitbake 1.25 and newer + self.create_eventlog = "False" + if self.getProperty('bitbakeversion') \ + and StrictVersion(self.getProperty('bitbakeversion')) >= StrictVersion("1.25"): + try: + self.create_eventlog = self.getProperty("custom_create_eventlog") + if self.create_eventlog == "True": + timestamp = datetime.datetime.now().strftime("%Y%m%d%H%M%S") + self.eventlog_name = "tmp/log/bitbake_eventlog- %s.json" % timestamp + except: + pass Do we really want to catch all exceptions here and ignore them? What exception might be thrown by the above code? The only exception I know of would be the case where the property is not defined because the code was invoked from a page that did not set it (not all pages do). + if self.images == "#TOASTER": - bitbakeflags = "-k -w 'bitbake_eventlog.json' " + bitbakeflags = "-k " + if self.create_eventlog == "True": + bitbakeflags += "-w '" + self.eventlog_name + "' " self.images=self.getProperty("custom_images") self.command = ". ./oe-init-build-env; bitbake " + bitbakeflags + self.images self.description = ["Building " + str(self.images)] @@ -130,9 +145,8 @@ class BuildImages(BitbakeShellCommand): self.description = ["Skipping Build. No Images to be built"] else: bitbakeflags = "-k " - # -w only exists in bitbake 1.25 and newer, use distroversion string and make sure we're on poky >1.7 - if self.getProperty('bitbakeversion') and StrictVersion(self.getProperty('bitbakeversion')) >= StrictVersion("1.25"): - bitbakeflags += "-w 'bitbake_eventlog.json' " + if self.create_eventlog == "True": + bitbakeflags += "-w '" + self.eventlog_name + "' " if self.minnowExists is None or self.minnowExists == "True": self.command = ". ./oe-init-build-env; bitbake " + bitbakeflags + self.images self.description = ["Building " + str(self.images)] [-- Attachment #2: Type: text/html, Size: 7279 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [yocto-autobuilder][PATCH 2/4] BuildImages.py: build toaster eventlog only when requested 2016-07-15 23:54 ` Randle, William C @ 2016-07-16 14:45 ` Randle, William C 0 siblings, 0 replies; 9+ messages in thread From: Randle, William C @ 2016-07-16 14:45 UTC (permalink / raw) To: joshua.g.lock@linux.intel.com, yocto@yoctoproject.org [-- Attachment #1: Type: text/plain, Size: 4451 bytes --] On Fri, 2016-07-15 at 16:54 -0700, Randle, William C wrote: On Fri, 2016-07-15 at 22:37 +0100, Joshua G Lock wrote: On Thu, 2016-07-14 at 10:05 -0700, Bill Randle wrote: Qualify creation of the toaster eventlog based on the setting of the new 'custom_create_eventlog' property. Place the eventlog in build/tmp/log, instead of build and add a timestamp to the eventlog filename since multiple event logs can be generated during a multi-image buildset and otherwise only the last one would otherwise be saved. [YOCTO #9884] Signed-off-by: Bill Randle <william.c.randle@intel.com<mailto:william.c.randle@intel.com>> --- .../autobuilder/buildsteps/BuildImages.py | 24 +++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/lib/python2.7/site- packages/autobuilder/buildsteps/BuildImages.py b/lib/python2.7/site- packages/autobuilder/buildsteps/BuildImages.py index 7ef8aab..9ea350f 100644 --- a/lib/python2.7/site- packages/autobuilder/buildsteps/BuildImages.py +++ b/lib/python2.7/site- packages/autobuilder/buildsteps/BuildImages.py @@ -15,7 +15,7 @@ from buildbot.steps.shell import ShellCommand from buildbot.process.buildstep import LogLineObserver from distutils.version import StrictVersion from buildbot.status.results import SUCCESS, SKIPPED -import os +import os, datetime from lib.buildsteps import BitbakeShellCommand @@ -47,8 +47,23 @@ class BuildImages(BitbakeShellCommand): self.deploycheck = self.getProperty('custom_deploy_artifacts') except: self.deploycheck = "True" + + # the eventlog capability exists only in bitbake 1.25 and newer + self.create_eventlog = "False" + if self.getProperty('bitbakeversion') \ + and StrictVersion(self.getProperty('bitbakeversion')) >= StrictVersion("1.25"): + try: + self.create_eventlog = self.getProperty("custom_create_eventlog") + if self.create_eventlog == "True": + timestamp = datetime.datetime.now().strftime("%Y%m%d%H%M%S") + self.eventlog_name = "tmp/log/bitbake_eventlog- %s.json" % timestamp + except: + pass Do we really want to catch all exceptions here and ignore them? What exception might be thrown by the above code? The only exception I know of would be the case where the property is not defined because the code was invoked from a page that did not set it (not all pages do). To clarify a bit more, if you look at the full source file (BuildImages.py), you will see that standard way to check a property is: try: somevar = self.getProperty("propname") except: somevar = default_value In the added code, it actually sets the default value first: self.create_eventlog = "False" because there are several checks made. Since the default is already set, there's no reason to set it again in the except caluse, hence I just did a 'pass'. -Bill + if self.images == "#TOASTER": - bitbakeflags = "-k -w 'bitbake_eventlog.json' " + bitbakeflags = "-k " + if self.create_eventlog == "True": + bitbakeflags += "-w '" + self.eventlog_name + "' " self.images=self.getProperty("custom_images") self.command = ". ./oe-init-build-env; bitbake " + bitbakeflags + self.images self.description = ["Building " + str(self.images)] @@ -130,9 +145,8 @@ class BuildImages(BitbakeShellCommand): self.description = ["Skipping Build. No Images to be built"] else: bitbakeflags = "-k " - # -w only exists in bitbake 1.25 and newer, use distroversion string and make sure we're on poky >1.7 - if self.getProperty('bitbakeversion') and StrictVersion(self.getProperty('bitbakeversion')) >= StrictVersion("1.25"): - bitbakeflags += "-w 'bitbake_eventlog.json' " + if self.create_eventlog == "True": + bitbakeflags += "-w '" + self.eventlog_name + "' " if self.minnowExists is None or self.minnowExists == "True": self.command = ". ./oe-init-build-env; bitbake " + bitbakeflags + self.images self.description = ["Building " + str(self.images)] [-- Attachment #2: Type: text/html, Size: 8209 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* [yocto-autobuilder][PATCH 3/4] UploadToasterEventlog.py: handle multiple eventlog files 2016-07-14 17:05 [yocto-autobuilder][PATCH 0/4] autobuilder: make toaster eventlog creation optional Bill Randle 2016-07-14 17:05 ` [yocto-autobuilder][PATCH 1/4] remote_kick.py: add default entry for new 'create_eventlog' property Bill Randle 2016-07-14 17:05 ` [yocto-autobuilder][PATCH 2/4] BuildImages.py: build toaster eventlog only when requested Bill Randle @ 2016-07-14 17:05 ` Bill Randle 2016-07-14 17:05 ` [yocto-autobuilder][PATCH 4/4] buildset-config.controller: add new 'create_eventlog' forced build property Bill Randle 2016-07-19 15:42 ` [yocto-autobuilder][PATCH 0/4] autobuilder: make toaster eventlog creation optional Beth 'pidge' Flanagan 4 siblings, 0 replies; 9+ messages in thread From: Bill Randle @ 2016-07-14 17:05 UTC (permalink / raw) To: yocto Now that multiple eventlog files can be generated, upload thyem all. Also, only upload files in user requested an eventlog be generated. [YOCTO #9884] Signed-off-by: Bill Randle <william.c.randle@intel.com> --- .../autobuilder/buildsteps/UploadToasterEventlog.py | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/lib/python2.7/site-packages/autobuilder/buildsteps/UploadToasterEventlog.py b/lib/python2.7/site-packages/autobuilder/buildsteps/UploadToasterEventlog.py index ed1cb1c..0fbb899 100644 --- a/lib/python2.7/site-packages/autobuilder/buildsteps/UploadToasterEventlog.py +++ b/lib/python2.7/site-packages/autobuilder/buildsteps/UploadToasterEventlog.py @@ -34,17 +34,23 @@ class UploadToasterEventlog(ShellCommand): self.workerdir=os.path.join(os.path.join(YOCTO_ABBASE, "yocto-worker")) self.buildername=self.getProperty("buildername") self.layerversion_core = self.getProperty("layerversion_core") - self.filepath=os.path.join(os.path.join( - self.workerdir, self.buildername), - "build/build/bitbake_eventlog.json") + try: + self.create_eventlog = self.getProperty("custom_create_eventlog") + except: + self.create_eventlog = "False" - if int(self.layerversion_core) < 5: + if int(self.layerversion_core) < 5 or self.create_eventlog == "False": self.command = "echo 'Skipping Step.'" else: if os.environ.get('TOASTER_UPLOAD_URL') is not None: + self.filepath = os.path.join(os.path.join( + self.workerdir, self.buildername), + "build/build/tmp/log/bitbake_eventlog-*.json") self.toasterserver = str(os.environ.get('TOASTER_UPLOAD_URL')) - self.command = "curl -F eventlog=@" + self.filepath + " " + self.toasterserver + self.command = "for fn in %s; do " % self.filepath + self.command += "curl -F eventlog=@$fn " + self.toasterserver + ";" + self.command += "done" else: self.command = "echo environment var TOASTER_UPLOADURL not set - Step is noop" self.description = ["Uploading toaster data"] -- 2.5.5 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [yocto-autobuilder][PATCH 4/4] buildset-config.controller: add new 'create_eventlog' forced build property 2016-07-14 17:05 [yocto-autobuilder][PATCH 0/4] autobuilder: make toaster eventlog creation optional Bill Randle ` (2 preceding siblings ...) 2016-07-14 17:05 ` [yocto-autobuilder][PATCH 3/4] UploadToasterEventlog.py: handle multiple eventlog files Bill Randle @ 2016-07-14 17:05 ` Bill Randle 2016-07-19 15:42 ` [yocto-autobuilder][PATCH 0/4] autobuilder: make toaster eventlog creation optional Beth 'pidge' Flanagan 4 siblings, 0 replies; 9+ messages in thread From: Bill Randle @ 2016-07-14 17:05 UTC (permalink / raw) To: yocto Add a new 'create_eventlog' custom property to the configurations that could potentially create a toaster event log. In the case of "nightly.conf", ensure the property is passed to the Triggered builds. [YOCTO #9884] Signed-off-by: Bill Randle <william.c.randle@intel.com> --- buildset-config.controller/nightly-arm-lsb.conf | 4 ++++ buildset-config.controller/nightly-arm.conf | 4 ++++ buildset-config.controller/nightly-arm64.conf | 4 ++++ buildset-config.controller/nightly-mips-lsb.conf | 4 ++++ buildset-config.controller/nightly-mips.conf | 4 ++++ buildset-config.controller/nightly-mips64.conf | 4 ++++ buildset-config.controller/nightly-no-x11.conf | 4 ++++ buildset-config.controller/nightly-ppc-lsb.conf | 4 ++++ buildset-config.controller/nightly-ppc.conf | 4 ++++ buildset-config.controller/nightly-x32.conf | 4 ++++ buildset-config.controller/nightly-x86-64-lsb.conf | 4 ++++ buildset-config.controller/nightly-x86-64.conf | 4 ++++ buildset-config.controller/nightly-x86-lsb.conf | 4 ++++ buildset-config.controller/nightly-x86.conf | 4 ++++ buildset-config.controller/nightly.conf | 7 ++++++- 15 files changed, 62 insertions(+), 1 deletion(-) diff --git a/buildset-config.controller/nightly-arm-lsb.conf b/buildset-config.controller/nightly-arm-lsb.conf index eb8d3eb..f94db21 100644 --- a/buildset-config.controller/nightly-arm-lsb.conf +++ b/buildset-config.controller/nightly-arm-lsb.conf @@ -10,6 +10,10 @@ repos: [{'poky': {'meta-qt3': {'repourl':'git://git.yoctoproject.org/meta-qt3', 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly-arm.conf b/buildset-config.controller/nightly-arm.conf index 2e6c829..f4c3fb8 100644 --- a/buildset-config.controller/nightly-arm.conf +++ b/buildset-config.controller/nightly-arm.conf @@ -4,6 +4,10 @@ repos: [{'poky': {'repourl':'git://git.yoctoproject.org/poky', 'layerversion':{'core':'meta', 'yoctobsp':'meta-yocto-bsp', 'yocto':'meta-yocto', 'poky':'meta-poky'}, 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly-arm64.conf b/buildset-config.controller/nightly-arm64.conf index 32a5492..3986c93 100644 --- a/buildset-config.controller/nightly-arm64.conf +++ b/buildset-config.controller/nightly-arm64.conf @@ -4,6 +4,10 @@ repos: [{'poky': {'repourl':'git://git.yoctoproject.org/poky', 'layerversion':{'core':'meta', 'yoctobsp':'meta-yocto-bsp', 'yocto':'meta-yocto', 'poky':'meta-poky'}, 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly-mips-lsb.conf b/buildset-config.controller/nightly-mips-lsb.conf index 81faaad..65fb4ec 100644 --- a/buildset-config.controller/nightly-mips-lsb.conf +++ b/buildset-config.controller/nightly-mips-lsb.conf @@ -10,6 +10,10 @@ repos: [{'poky': {'meta-qt3': {'repourl':'git://git.yoctoproject.org/meta-qt3', 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly-mips.conf b/buildset-config.controller/nightly-mips.conf index e419ce4..1dbad7e 100644 --- a/buildset-config.controller/nightly-mips.conf +++ b/buildset-config.controller/nightly-mips.conf @@ -4,6 +4,10 @@ repos: [{'poky': {'repourl':'git://git.yoctoproject.org/poky', 'layerversion':{'core':'meta', 'yoctobsp':'meta-yocto-bsp', 'yocto':'meta-yocto', 'poky':'meta-poky'}, 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly-mips64.conf b/buildset-config.controller/nightly-mips64.conf index 13f8573..73cb577 100644 --- a/buildset-config.controller/nightly-mips64.conf +++ b/buildset-config.controller/nightly-mips64.conf @@ -4,6 +4,10 @@ repos: [{'poky': {'repourl':'git://git.yoctoproject.org/poky', 'layerversion':{'core':'meta', 'yoctobsp':'meta-yocto-bsp', 'yocto':'meta-yocto', 'poky':'meta-poky'}, 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly-no-x11.conf b/buildset-config.controller/nightly-no-x11.conf index 4ef1b35..063ae89 100644 --- a/buildset-config.controller/nightly-no-x11.conf +++ b/buildset-config.controller/nightly-no-x11.conf @@ -4,6 +4,10 @@ repos: [{'poky': {'repourl':'git://git.yoctoproject.org/poky', 'layerversion':{'core':'meta', 'yoctobsp':'meta-yocto-bsp', 'yocto':'meta-yocto', 'poky':'meta-poky'}, 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly-ppc-lsb.conf b/buildset-config.controller/nightly-ppc-lsb.conf index 7cf05c1..5944240 100644 --- a/buildset-config.controller/nightly-ppc-lsb.conf +++ b/buildset-config.controller/nightly-ppc-lsb.conf @@ -10,6 +10,10 @@ repos: [{'poky': {'meta-qt3': {'repourl':'git://git.yoctoproject.org/meta-qt3', 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly-ppc.conf b/buildset-config.controller/nightly-ppc.conf index 16413c3..1e26307 100644 --- a/buildset-config.controller/nightly-ppc.conf +++ b/buildset-config.controller/nightly-ppc.conf @@ -4,6 +4,10 @@ repos: [{'poky': {'repourl':'git://git.yoctoproject.org/poky', 'layerversion':{'core':'meta', 'yoctobsp':'meta-yocto-bsp', 'yocto':'meta-yocto', 'poky':'meta-poky'}, 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly-x32.conf b/buildset-config.controller/nightly-x32.conf index 2ec5b87..e811a30 100644 --- a/buildset-config.controller/nightly-x32.conf +++ b/buildset-config.controller/nightly-x32.conf @@ -4,6 +4,10 @@ repos: [{'poky': {'repourl':'git://git.yoctoproject.org/poky', 'layerversion':{'core':'meta', 'yoctobsp':'meta-yocto-bsp', 'yocto':'meta-yocto', 'poky':'meta-poky'}, 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly-x86-64-lsb.conf b/buildset-config.controller/nightly-x86-64-lsb.conf index b0bb0a5..784f23d 100644 --- a/buildset-config.controller/nightly-x86-64-lsb.conf +++ b/buildset-config.controller/nightly-x86-64-lsb.conf @@ -10,6 +10,10 @@ repos: [{'poky': {'meta-qt3': {'repourl':'git://git.yoctoproject.org/meta-qt3', 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly-x86-64.conf b/buildset-config.controller/nightly-x86-64.conf index 92e5866..93df577 100644 --- a/buildset-config.controller/nightly-x86-64.conf +++ b/buildset-config.controller/nightly-x86-64.conf @@ -4,6 +4,10 @@ repos: [{'poky': {'repourl':'git://git.yoctoproject.org/poky', 'layerversion':{'core':'meta', 'yoctobsp':'meta-yocto-bsp', 'yocto':'meta-yocto', 'poky':'meta-poky'}, 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly-x86-lsb.conf b/buildset-config.controller/nightly-x86-lsb.conf index 34b6780..1381b4c 100644 --- a/buildset-config.controller/nightly-x86-lsb.conf +++ b/buildset-config.controller/nightly-x86-lsb.conf @@ -10,6 +10,10 @@ repos: [{'poky': {'meta-qt3': {'repourl':'git://git.yoctoproject.org/meta-qt3', 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly-x86.conf b/buildset-config.controller/nightly-x86.conf index a5c9f0a..07fb34c 100644 --- a/buildset-config.controller/nightly-x86.conf +++ b/buildset-config.controller/nightly-x86.conf @@ -4,6 +4,10 @@ repos: [{'poky': {'repourl':'git://git.yoctoproject.org/poky', 'layerversion':{'core':'meta', 'yoctobsp':'meta-yocto-bsp', 'yocto':'meta-yocto', 'poky':'meta-poky'}, 'branch':'master'}}] +props: [{'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr>Create Toaster event log as part of image creation?:'}}] steps: [{'SetDest':{}}, {'CheckOutLayers': {}}, {'RunPreamble': {}}, diff --git a/buildset-config.controller/nightly.conf b/buildset-config.controller/nightly.conf index 2ac83dc..0f9dd4e 100644 --- a/buildset-config.controller/nightly.conf +++ b/buildset-config.controller/nightly.conf @@ -74,6 +74,10 @@ props: [{'release_me':{'prop_type':'ChoiceStringParameter', 'name': 'prefered_kernel', 'default': 'linux-yocto', 'label':'<hr><h3> PREFERRED_PROVIDER_virtual/kernel (default is "linux-yocto"):</h3>'}}, + {'create_eventlog':{'prop_type':'ChoiceStringParameter', + 'choices': ['False', 'True'], + 'name': 'create_eventlog', + 'label':'<hr><h3> Should a Toaster event log be created as part of the image creation?:</h3>'}}, {'deploy_artifacts':{'prop_type':'ChoiceStringParameter', 'choices': ['True', 'False'], 'name': 'deploy_artifacts', @@ -87,7 +91,8 @@ steps: [{'SetDest':{}}, 'atextprepend': 'SOURCE_MIRROR_FETCH = "1"\n'}}, {'CreateBBLayersConf': {'buildprovider' : 'yocto'}}, {'BuildImages': {'images': 'universe -c fetch'}}, - {'TriggerBuilds': {'schedulerName': 'main-build', 'waitForFinish': 'True', + {'TriggerBuilds': {'copy_properties': ['custom_create_eventlog'], + 'schedulerName': 'main-build', 'waitForFinish': 'True', 'schedulerNames': {'nightly-arm': {}, 'nightly-arm-lsb': {}, 'nightly-mips': {}, 'nightly-mips-lsb': {}, 'nightly-multilib': {}, 'nightly-x32': {}, -- 2.5.5 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [yocto-autobuilder][PATCH 0/4] autobuilder: make toaster eventlog creation optional 2016-07-14 17:05 [yocto-autobuilder][PATCH 0/4] autobuilder: make toaster eventlog creation optional Bill Randle ` (3 preceding siblings ...) 2016-07-14 17:05 ` [yocto-autobuilder][PATCH 4/4] buildset-config.controller: add new 'create_eventlog' forced build property Bill Randle @ 2016-07-19 15:42 ` Beth 'pidge' Flanagan 4 siblings, 0 replies; 9+ messages in thread From: Beth 'pidge' Flanagan @ 2016-07-19 15:42 UTC (permalink / raw) To: Bill Randle, yocto Merged into master. -b On Thu, 2016-07-14 at 10:05 -0700, Bill Randle wrote: > Currently, a toaster eventlog is generated for most all machine > specific > images (e.g., nightly-arm, nightly-x86, etc.) and placed in the top- > level > build directory. Having it in the top level dir causes inotify > overflow in > bitbake, due to the large number of changes to the file. > > This set of patches (1) make generation of the eventlog optional, by > having a user selection in the Forced Build web pages, (2) places the > eventlog in build/tmp/log, instead of build and (3) adds a timestamp > to > the eventlog filename since multiple event logs can be generated > during > a multi-image buildset and otherwise only the last one would > otherwise > be saved. > > [YOCTO #9884] > > Bill Randle (4): > remote_kick.py: add default entry for new 'create_eventlog' > property > BuildImages.py: build toaster eventlog only when requested > UploadToasterEventlog.py: handle multiple eventlog files > buildset-config.controller: add new 'create_eventlog' forced build > property > > bin/remote_kick.py | 1 + > buildset-config.controller/nightly-arm-lsb.conf | 4 ++++ > buildset-config.controller/nightly-arm.conf | 4 ++++ > buildset-config.controller/nightly-arm64.conf | 4 ++++ > buildset-config.controller/nightly-mips-lsb.conf | 4 ++++ > buildset-config.controller/nightly-mips.conf | 4 ++++ > buildset-config.controller/nightly-mips64.conf | 4 ++++ > buildset-config.controller/nightly-no-x11.conf | 4 ++++ > buildset-config.controller/nightly-ppc-lsb.conf | 4 ++++ > buildset-config.controller/nightly-ppc.conf | 4 ++++ > buildset-config.controller/nightly-x32.conf | 4 ++++ > buildset-config.controller/nightly-x86-64-lsb.conf | 4 ++++ > buildset-config.controller/nightly-x86-64.conf | 4 ++++ > buildset-config.controller/nightly-x86-lsb.conf | 4 ++++ > buildset-config.controller/nightly-x86.conf | 4 ++++ > buildset-config.controller/nightly.conf | 7 ++++++- > .../autobuilder/buildsteps/BuildImages.py | 24 > +++++++++++++++++----- > .../buildsteps/UploadToasterEventlog.py | 16 +++++++++++- > ---- > 18 files changed, 93 insertions(+), 11 deletions(-) > -- Beth 'pidge' Flanagan <pidge@toganlabs.com> toganlabs.com ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2016-07-19 15:47 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-07-14 17:05 [yocto-autobuilder][PATCH 0/4] autobuilder: make toaster eventlog creation optional Bill Randle 2016-07-14 17:05 ` [yocto-autobuilder][PATCH 1/4] remote_kick.py: add default entry for new 'create_eventlog' property Bill Randle 2016-07-14 17:05 ` [yocto-autobuilder][PATCH 2/4] BuildImages.py: build toaster eventlog only when requested Bill Randle 2016-07-15 21:37 ` Joshua G Lock 2016-07-15 23:54 ` Randle, William C 2016-07-16 14:45 ` Randle, William C 2016-07-14 17:05 ` [yocto-autobuilder][PATCH 3/4] UploadToasterEventlog.py: handle multiple eventlog files Bill Randle 2016-07-14 17:05 ` [yocto-autobuilder][PATCH 4/4] buildset-config.controller: add new 'create_eventlog' forced build property Bill Randle 2016-07-19 15:42 ` [yocto-autobuilder][PATCH 0/4] autobuilder: make toaster eventlog creation optional Beth 'pidge' Flanagan
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.