* [PATCH 01/15] toaster: bldcontrol Add forceShutDown function to BitbakeController
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 02/15] toaster: Move xhr calls for starting and stopping builds Ed Bartosh
` (14 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
From: Sujith H <sujith.h@gmail.com>
Add forceShutDown function to BitbakeController class. This function
provides a mechanism to cancel the build from toaster. An API which
can be used safely to cancel build and hence shutdown running bitbake
server.
[YOCTO #6787]
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
bitbake/lib/toaster/bldcontrol/bbcontroller.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/bitbake/lib/toaster/bldcontrol/bbcontroller.py b/bitbake/lib/toaster/bldcontrol/bbcontroller.py
index 058e490..0f7b66d 100644
--- a/bitbake/lib/toaster/bldcontrol/bbcontroller.py
+++ b/bitbake/lib/toaster/bldcontrol/bbcontroller.py
@@ -63,6 +63,9 @@ class BitbakeController(object):
task = "build"
return self._runCommand(["buildTargets", targets, task])
+ def forceShutDown(self):
+ return self._runCommand(["stateForceShutdown"])
+
def getBuildEnvironmentController(**kwargs):
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 02/15] toaster: Move xhr calls for starting and stopping builds
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
2016-03-17 15:41 ` [PATCH 01/15] toaster: bldcontrol Add forceShutDown function to BitbakeController Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 03/15] toaster: xhr Update the implementation of the build cancellation request Ed Bartosh
` (13 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
From: Michael Wood <michael.g.wood@intel.com>
Move the backend xhr implementation of the build request changes
into it's own file and out of the ToasterTable definition.
It used to live in the views.py but in a hope of starting to collate logical
groups of views move this to a new file called api.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
bitbake/lib/toaster/toastergui/api.py | 74 ++++++++++++++++++++++++++++++++
bitbake/lib/toaster/toastergui/tables.py | 43 +------------------
bitbake/lib/toaster/toastergui/urls.py | 5 +++
3 files changed, 80 insertions(+), 42 deletions(-)
create mode 100644 bitbake/lib/toaster/toastergui/api.py
diff --git a/bitbake/lib/toaster/toastergui/api.py b/bitbake/lib/toaster/toastergui/api.py
new file mode 100644
index 0000000..1b1f598
--- /dev/null
+++ b/bitbake/lib/toaster/toastergui/api.py
@@ -0,0 +1,74 @@
+#
+# BitBake Toaster Implementation
+#
+# Copyright (C) 2016 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+# Temporary home for the UI's misc API
+
+from orm.models import Project, ProjectTarget
+from bldcontrol.models import BuildRequest
+from bldcontrol import bbcontroller
+from django.http import HttpResponse, JsonResponse
+from django.views.generic import View
+
+
+class XhrBuildRequest(View):
+
+ def get(self, request, *args, **kwargs):
+ return HttpResponse()
+
+ def post(self, request, *args, **kwargs):
+ """ Process HTTP POSTs which make build requests """
+
+ project = Project.objects.get(pk=kwargs['pid'])
+
+ if 'buildCancel' in request.POST:
+ for i in request.POST['buildCancel'].strip().split(" "):
+ try:
+ br = BuildRequest.objects.select_for_update().get(project = project, pk = i, state__lte = BuildRequest.REQ_QUEUED)
+ br.state = BuildRequest.REQ_DELETED
+ br.save()
+ except BuildRequest.DoesNotExist:
+ pass
+
+ if 'buildDelete' in request.POST:
+ for i in request.POST['buildDelete'].strip().split(" "):
+ try:
+ BuildRequest.objects.select_for_update().get(project = project, pk = i, state__lte = BuildRequest.REQ_DELETED).delete()
+ except BuildRequest.DoesNotExist:
+ pass
+
+ if 'targets' in request.POST:
+ ProjectTarget.objects.filter(project = project).delete()
+ s = str(request.POST['targets'])
+ for t in s.translate(None, ";%|\"").split(" "):
+ if ":" in t:
+ target, task = t.split(":")
+ else:
+ target = t
+ task = ""
+ ProjectTarget.objects.create(project = project,
+ target = target,
+ task = task)
+ project.schedule_build()
+
+ # redirect back to builds page so any new builds in progress etc.
+ # are visible
+ response = HttpResponse()
+ response.status_code = 302
+ response['Location'] = request.build_absolute_uri()
+ return response
diff --git a/bitbake/lib/toaster/toastergui/tables.py b/bitbake/lib/toaster/toastergui/tables.py
index 67a6592..822c7e5 100644
--- a/bitbake/lib/toaster/toastergui/tables.py
+++ b/bitbake/lib/toaster/toastergui/tables.py
@@ -22,7 +22,7 @@
from toastergui.widgets import ToasterTable
from orm.models import Recipe, ProjectLayer, Layer_Version, Machine, Project
from orm.models import CustomImageRecipe, Package, Target, Build, LogMessage, Task
-from orm.models import CustomImagePackage, ProjectTarget
+from orm.models import CustomImagePackage
from django.db.models import Q, Max, Sum, Count, When, Case, Value, IntegerField
from django.conf.urls import url
from django.core.urlresolvers import reverse, resolve
@@ -1400,47 +1400,6 @@ class BuildsTable(ToasterTable):
failed_tasks_filter.add_action(without_failed_tasks_action)
self.add_filter(failed_tasks_filter)
- def post(self, request, *args, **kwargs):
- """ Process HTTP POSTs which make build requests """
-
- project = Project.objects.get(pk=kwargs['pid'])
-
- if 'buildCancel' in request.POST:
- for i in request.POST['buildCancel'].strip().split(" "):
- try:
- br = BuildRequest.objects.select_for_update().get(project = project, pk = i, state__lte = BuildRequest.REQ_QUEUED)
- br.state = BuildRequest.REQ_DELETED
- br.save()
- except BuildRequest.DoesNotExist:
- pass
-
- if 'buildDelete' in request.POST:
- for i in request.POST['buildDelete'].strip().split(" "):
- try:
- BuildRequest.objects.select_for_update().get(project = project, pk = i, state__lte = BuildRequest.REQ_DELETED).delete()
- except BuildRequest.DoesNotExist:
- pass
-
- if 'targets' in request.POST:
- ProjectTarget.objects.filter(project = project).delete()
- s = str(request.POST['targets'])
- for t in s.translate(None, ";%|\"").split(" "):
- if ":" in t:
- target, task = t.split(":")
- else:
- target = t
- task = ""
- ProjectTarget.objects.create(project = project,
- target = target,
- task = task)
- project.schedule_build()
-
- # redirect back to builds page so any new builds in progress etc.
- # are visible
- response = HttpResponse()
- response.status_code = 302
- response['Location'] = request.build_absolute_uri()
- return response
class AllBuildsTable(BuildsTable):
""" Builds page for all builds """
diff --git a/bitbake/lib/toaster/toastergui/urls.py b/bitbake/lib/toaster/toastergui/urls.py
index 400580a..27b0baa 100644
--- a/bitbake/lib/toaster/toastergui/urls.py
+++ b/bitbake/lib/toaster/toastergui/urls.py
@@ -22,6 +22,7 @@ from django.views.generic import RedirectView, TemplateView
from django.http import HttpResponseBadRequest
from toastergui import tables
from toastergui import typeaheads
+from toastergui import api
urlpatterns = patterns('toastergui.views',
# landing page
@@ -180,6 +181,10 @@ urlpatterns = patterns('toastergui.views',
url(r'^xhr_customrecipe/', 'xhr_customrecipe',
name='xhr_customrecipe'),
+ url(r'^xhr_buildrequest/project/(?P<pid>\d+)$',
+ api.XhrBuildRequest.as_view(),
+ name='xhr_buildrequest'),
+
# default redirection
url(r'^$', RedirectView.as_view(url='landing', permanent=True)),
)
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 03/15] toaster: xhr Update the implementation of the build cancellation request
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
2016-03-17 15:41 ` [PATCH 01/15] toaster: bldcontrol Add forceShutDown function to BitbakeController Ed Bartosh
2016-03-17 15:41 ` [PATCH 02/15] toaster: Move xhr calls for starting and stopping builds Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 04/15] toaster: libtoaster Update implementation of startABuild and cancelABuild Ed Bartosh
` (12 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
From: Michael Wood <michael.g.wood@intel.com>
Update the implementation of the backend api for cancelling builds with
the new cancelling BuildRequest state and cancelled Build state.
Also added some docstring about general usage.
Co-Author: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
bitbake/lib/toaster/toastergui/api.py | 54 +++++++++++++++++++++++++++++------
1 file changed, 45 insertions(+), 9 deletions(-)
diff --git a/bitbake/lib/toaster/toastergui/api.py b/bitbake/lib/toaster/toastergui/api.py
index 1b1f598..42901f75 100644
--- a/bitbake/lib/toaster/toastergui/api.py
+++ b/bitbake/lib/toaster/toastergui/api.py
@@ -19,7 +19,7 @@
# Temporary home for the UI's misc API
-from orm.models import Project, ProjectTarget
+from orm.models import Project, ProjectTarget, Build
from bldcontrol.models import BuildRequest
from bldcontrol import bbcontroller
from django.http import HttpResponse, JsonResponse
@@ -32,18 +32,54 @@ class XhrBuildRequest(View):
return HttpResponse()
def post(self, request, *args, **kwargs):
- """ Process HTTP POSTs which make build requests """
+ """
+ Build control
+
+ Entry point: /xhr_buildrequest/<project_id>
+ Method: POST
+
+ Args:
+ id: id of build to change
+ buildCancel = build_request_id ...
+ buildDelete = id ...
+ targets = recipe_name ...
+
+ Returns:
+ {"error": "ok"}
+ or
+ {"error": <error message>}
+ """
project = Project.objects.get(pk=kwargs['pid'])
if 'buildCancel' in request.POST:
for i in request.POST['buildCancel'].strip().split(" "):
try:
- br = BuildRequest.objects.select_for_update().get(project = project, pk = i, state__lte = BuildRequest.REQ_QUEUED)
- br.state = BuildRequest.REQ_DELETED
+ br = BuildRequest.objects.get(project=project, pk=i)
+
+ try:
+ bbctrl = bbcontroller.BitbakeController(br.environment)
+ bbctrl.forceShutDown()
+ except:
+ # We catch a bunch of exceptions here because
+ # this is where the server has not had time to start up
+ # and the build request or build is in transit between
+ # processes.
+ # We can safely just set the build as cancelled
+ # already as it never got started
+ build = br.build
+ build.outcome = Build.CANCELLED
+ build.save()
+
+ # We now hand over to the buildinfohelper to update the
+ # build state once we've finished cancelling
+ br.state = BuildRequest.REQ_CANCELLING
br.save()
+
except BuildRequest.DoesNotExist:
- pass
+ return JsonResponse({'error':'No such build id %s' % i})
+
+ return JsonResponse({'error': 'ok'})
if 'buildDelete' in request.POST:
for i in request.POST['buildDelete'].strip().split(" "):
@@ -51,6 +87,7 @@ class XhrBuildRequest(View):
BuildRequest.objects.select_for_update().get(project = project, pk = i, state__lte = BuildRequest.REQ_DELETED).delete()
except BuildRequest.DoesNotExist:
pass
+ return JsonResponse({'error': 'ok' })
if 'targets' in request.POST:
ProjectTarget.objects.filter(project = project).delete()
@@ -66,9 +103,8 @@ class XhrBuildRequest(View):
task = task)
project.schedule_build()
- # redirect back to builds page so any new builds in progress etc.
- # are visible
+ return JsonResponse({'error': 'ok' })
+
response = HttpResponse()
- response.status_code = 302
- response['Location'] = request.build_absolute_uri()
+ response.status_code = 500
return response
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 04/15] toaster: libtoaster Update implementation of startABuild and cancelABuild
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
` (2 preceding siblings ...)
2016-03-17 15:41 ` [PATCH 03/15] toaster: xhr Update the implementation of the build cancellation request Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 05/15] toaster: update BuildEnvironmentController and BitbakeController Ed Bartosh
` (11 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
From: Michael Wood <michael.g.wood@intel.com>
Update the implementation of startABuild and cancelAbuild to reflect
changes to the backend api. We now have a dedicated endpoint to make
calls into so add this url to libtoaster.ctx and allow passing null in
as a url value to indicate that we want to use the current project
Also:
- Fix some documentation comments
- Add the convenience of passing in an array of targets to startABuild
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
.../toaster/toastergui/static/js/customrecipe.js | 4 +--
.../lib/toaster/toastergui/static/js/layerBtn.js | 3 +-
.../lib/toaster/toastergui/static/js/libtoaster.js | 41 ++++++++++++++--------
.../toaster/toastergui/static/js/projectpage.js | 4 +--
.../toaster/toastergui/static/js/projecttopbar.js | 6 ++--
.../toaster/toastergui/static/js/recipedetails.js | 4 +--
bitbake/lib/toaster/toastergui/templates/base.html | 1 +
7 files changed, 34 insertions(+), 29 deletions(-)
diff --git a/bitbake/lib/toaster/toastergui/static/js/customrecipe.js b/bitbake/lib/toaster/toastergui/static/js/customrecipe.js
index 3c57899..131ac2a 100644
--- a/bitbake/lib/toaster/toastergui/static/js/customrecipe.js
+++ b/bitbake/lib/toaster/toastergui/static/js/customrecipe.js
@@ -207,9 +207,7 @@ function customRecipePageInit(ctx) {
/* Trigger a build of your custom image */
$(".build-custom-image").click(function(){
- libtoaster.startABuild(libtoaster.ctx.projectBuildsUrl,
- libtoaster.ctx.projectId,
- ctx.recipe.name,
+ libtoaster.startABuild(null, ctx.recipe.name,
function(){
window.location.replace(libtoaster.ctx.projectBuildsUrl);
});
diff --git a/bitbake/lib/toaster/toastergui/static/js/layerBtn.js b/bitbake/lib/toaster/toastergui/static/js/layerBtn.js
index b2666ab..aa43284 100644
--- a/bitbake/lib/toaster/toastergui/static/js/layerBtn.js
+++ b/bitbake/lib/toaster/toastergui/static/js/layerBtn.js
@@ -60,8 +60,7 @@ function layerBtnsInit() {
e.preventDefault();
var recipe = $(this).data('recipe-name');
- libtoaster.startABuild(libtoaster.ctx.projectBuildsUrl,
- libtoaster.ctx.projectId, recipe,
+ libtoaster.startABuild(null, recipe,
function(){
/* Success */
window.location.replace(libtoaster.ctx.projectBuildsUrl);
diff --git a/bitbake/lib/toaster/toastergui/static/js/libtoaster.js b/bitbake/lib/toaster/toastergui/static/js/libtoaster.js
index b6b49b6..8d1d20f 100644
--- a/bitbake/lib/toaster/toastergui/static/js/libtoaster.js
+++ b/bitbake/lib/toaster/toastergui/static/js/libtoaster.js
@@ -90,27 +90,35 @@ var libtoaster = (function (){
jQElement.data('typeahead').render = customRenderFunc;
}
- /*
- * url - the url of the xhr build */
- function _startABuild (url, project_id, targets, onsuccess, onfail) {
+ /* startABuild:
+ * url: xhr_buildrequest or null for current project
+ * targets: an array or space separated list of targets to build
+ * onsuccess: callback for successful execution
+ * onfail: callback for failed execution
+ */
+ function _startABuild (url, targets, onsuccess, onfail) {
- var data = {
- project_id : project_id,
- targets : targets,
+ if (!url)
+ url = libtoaster.ctx.xhrBuildRequestUrl;
+
+ /* Flatten the array of targets into a space spearated list */
+ if (targets instanceof Array){
+ targets = targets.reduce(function(prevV, nextV){
+ return prev + ' ' + next;
+ });
}
$.ajax( {
type: "POST",
url: url,
- data: data,
+ data: { 'targets' : targets },
headers: { 'X-CSRFToken' : $.cookie('csrftoken')},
success: function (_data) {
- /* No proper reponse YOCTO #7995
if (_data.error !== "ok") {
console.warn(_data.error);
- } else { */
+ } else {
if (onsuccess !== undefined) onsuccess(_data);
- // }
+ }
},
error: function (_data) {
console.warn("Call failed");
@@ -120,22 +128,25 @@ var libtoaster = (function (){
}
/* cancelABuild:
- * url: projectbuilds
- * builds_ids: space separated list of build request ids
+ * url: xhr_buildrequest url or null for current project
+ * buildRequestIds: space separated list of build request ids
* onsuccess: callback for successful execution
* onfail: callback for failed execution
*/
- function _cancelABuild(url, build_ids, onsuccess, onfail){
+ function _cancelABuild(url, buildRequestIds, onsuccess, onfail){
+ if (!url)
+ url = libtoaster.ctx.xhrBuildRequestUrl;
+
$.ajax( {
type: "POST",
url: url,
- data: { 'buildCancel': build_ids },
+ data: { 'buildCancel': buildRequestIds },
headers: { 'X-CSRFToken' : $.cookie('csrftoken')},
success: function (_data) {
if (_data.error !== "ok") {
console.warn(_data.error);
} else {
- if (onsuccess !== undefined) onsuccess(_data);
+ if (onsuccess) onsuccess(_data);
}
},
error: function (_data) {
diff --git a/bitbake/lib/toaster/toastergui/static/js/projectpage.js b/bitbake/lib/toaster/toastergui/static/js/projectpage.js
index 6655a18..3013416 100644
--- a/bitbake/lib/toaster/toastergui/static/js/projectpage.js
+++ b/bitbake/lib/toaster/toastergui/static/js/projectpage.js
@@ -232,9 +232,7 @@ function projectPageInit(ctx) {
toBuild = toBuild.trim();
- libtoaster.startABuild(libtoaster.ctx.projectBuildsUrl,
- libtoaster.ctx.projectId,
- toBuild,
+ libtoaster.startABuild(null, toBuild,
function(){
/* Build request started */
window.location.replace(libtoaster.ctx.projectBuildsUrl);
diff --git a/bitbake/lib/toaster/toastergui/static/js/projecttopbar.js b/bitbake/lib/toaster/toastergui/static/js/projecttopbar.js
index 58a32a0..b09f974 100644
--- a/bitbake/lib/toaster/toastergui/static/js/projecttopbar.js
+++ b/bitbake/lib/toaster/toastergui/static/js/projecttopbar.js
@@ -82,9 +82,9 @@ function projectTopBarInit(ctx) {
selectedTarget = { name: newBuildTargetInput.val() };
/* Fire off the build */
- libtoaster.startABuild(libtoaster.ctx.projectBuildsUrl,
- null, selectedTarget.name, function(){
- window.location.replace(libtoaster.ctx.projectBuildsUrl);
+ libtoaster.startABuild(null, selectedTarget.name,
+ function(){
+ window.location.replace(libtoaster.ctx.projectBuildsUrl);
}, null);
});
}
diff --git a/bitbake/lib/toaster/toastergui/static/js/recipedetails.js b/bitbake/lib/toaster/toastergui/static/js/recipedetails.js
index 2bfd0a4..d5f9eac 100644
--- a/bitbake/lib/toaster/toastergui/static/js/recipedetails.js
+++ b/bitbake/lib/toaster/toastergui/static/js/recipedetails.js
@@ -42,9 +42,7 @@ function recipeDetailsPageInit(ctx){
/* Trigger a build of your custom image */
$(".build-recipe-btn").click(function(){
- libtoaster.startABuild(libtoaster.ctx.projectBuildsUrl,
- libtoaster.ctx.projectId,
- ctx.recipe.name,
+ libtoaster.startABuild(null, ctx.recipe.name,
function(){
window.location.replace(libtoaster.ctx.projectBuildsUrl);
});
diff --git a/bitbake/lib/toaster/toastergui/templates/base.html b/bitbake/lib/toaster/toastergui/templates/base.html
index 121a75c..192f9fb 100644
--- a/bitbake/lib/toaster/toastergui/templates/base.html
+++ b/bitbake/lib/toaster/toastergui/templates/base.html
@@ -47,6 +47,7 @@
projectBuildsUrl: {% url 'projectbuilds' project.id as pburl %}{{pburl|json}},
xhrCustomRecipeUrl : "{% url 'xhr_customrecipe' %}",
projectId : {{project.id}},
+ xhrBuildRequestUrl: "{% url 'xhr_buildrequest' project.id %}",
{% else %}
projectId : undefined,
projectPageUrl : undefined,
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 05/15] toaster: update BuildEnvironmentController and BitbakeController
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
` (3 preceding siblings ...)
2016-03-17 15:41 ` [PATCH 04/15] toaster: libtoaster Update implementation of startABuild and cancelABuild Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 06/15] toaster: models Add cancelled state to build outcome Ed Bartosh
` (10 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
From: Sujith H <sujith.h@gmail.com>
Remove getBBController function from BuildEnvironmentController.
The constructor of BitbakeController is updated appropriately so that
call can be made to connect to running server.
The call to startBBServer is removed from bbcontroller and handledin
localhostbecontroller.
[YOCTO #6787]
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
bitbake/lib/toaster/bldcontrol/bbcontroller.py | 26 ++++------------------
.../toaster/bldcontrol/localhostbecontroller.py | 2 +-
2 files changed, 5 insertions(+), 23 deletions(-)
diff --git a/bitbake/lib/toaster/bldcontrol/bbcontroller.py b/bitbake/lib/toaster/bldcontrol/bbcontroller.py
index 0f7b66d..d09ac17 100644
--- a/bitbake/lib/toaster/bldcontrol/bbcontroller.py
+++ b/bitbake/lib/toaster/bldcontrol/bbcontroller.py
@@ -37,11 +37,12 @@ class BitbakeController(object):
It is outside the scope of this class on how the server is started and aquired
"""
- def __init__(self, connection):
- self.connection = connection
+ def __init__(self, be):
+ self.connection = bb.server.xmlrpc._create_server(be.bbaddress,
+ int(be.bbport))[0]
def _runCommand(self, command):
- result, error = self.connection.connection.runCommand(command)
+ result, error = self.connection.runCommand(command)
if error:
raise Exception(error)
return result
@@ -122,25 +123,6 @@ class BuildEnvironmentController(object):
"""
raise NotImplementedError("FIXME: Must override setLayers")
-
- def getBBController(self):
- """ returns a BitbakeController to an already started server; this is the point where the server
- starts if needed; or reconnects to the server if we can
- """
- if not self.connection:
- self.be.lock = BuildEnvironment.LOCK_RUNNING
- self.be.save()
-
- server = bb.server.xmlrpc.BitBakeXMLRPCClient()
- server.initServer()
- server.saveConnectionDetails("%s:%s" % (self.be.bbaddress, self.be.bbport))
- self.connection = server.establishConnection([])
-
- self.be.bbtoken = self.connection.transport.connection_token
- self.be.save()
-
- return BitbakeController(self.connection)
-
def getArtifact(self, path):
""" This call returns an artifact identified by the 'path'. How 'path' is interpreted as
up to the implementing BEC. The return MUST be a REST URL where a GET will actually return
diff --git a/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py b/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py
index 480f945..6a6cff8 100644
--- a/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py
+++ b/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py
@@ -32,7 +32,7 @@ import subprocess
from toastermain import settings
-from bbcontroller import BuildEnvironmentController, ShellCmdException, BuildSetupException
+from bbcontroller import BuildEnvironmentController, ShellCmdException, BuildSetupException, BitbakeController
import logging
logger = logging.getLogger("toaster")
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 06/15] toaster: models Add cancelled state to build outcome
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
` (4 preceding siblings ...)
2016-03-17 15:41 ` [PATCH 05/15] toaster: update BuildEnvironmentController and BitbakeController Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 07/15] toaster: bldcontrol models Add a cancelling state the BuildRequest Ed Bartosh
` (9 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
From: Sujith H <sujith.h@gmail.com>
A new state CANCELLED is introduced to, distinguish
the state of build.
[YOCTO #6787]
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
.../orm/migrations/0006_add_cancelled_state.py | 19 +++++++++++++++++++
bitbake/lib/toaster/orm/models.py | 7 ++++++-
2 files changed, 25 insertions(+), 1 deletion(-)
create mode 100644 bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py
diff --git a/bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py b/bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py
new file mode 100644
index 0000000..f8ec65a
--- /dev/null
+++ b/bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py
@@ -0,0 +1,19 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+ dependencies = [
+ ('orm', '0004_provides'),
+ ]
+
+ operations = [
+ migrations.AlterField(
+ model_name='build',
+ name='outcome',
+ field=models.IntegerField(default=2, choices=[(0, b'Succeeded'), (1, b'Failed'), (2, b'In Progress'), (3, b'Cancelled')]),
+ ),
+ ]
diff --git a/bitbake/lib/toaster/orm/models.py b/bitbake/lib/toaster/orm/models.py
index 3c98c0b..e0c2096 100644
--- a/bitbake/lib/toaster/orm/models.py
+++ b/bitbake/lib/toaster/orm/models.py
@@ -358,11 +358,13 @@ class Build(models.Model):
SUCCEEDED = 0
FAILED = 1
IN_PROGRESS = 2
+ CANCELLED = 3
BUILD_OUTCOME = (
(SUCCEEDED, 'Succeeded'),
(FAILED, 'Failed'),
(IN_PROGRESS, 'In Progress'),
+ (CANCELLED, 'Cancelled'),
)
search_allowed_fields = ['machine', 'cooker_log_path', "target__target", "target__target_image_file__file_name"]
@@ -389,7 +391,10 @@ class Build(models.Model):
if project:
builds = builds.filter(project=project)
- finished_criteria = Q(outcome=Build.SUCCEEDED) | Q(outcome=Build.FAILED)
+ finished_criteria = \
+ Q(outcome=Build.SUCCEEDED) | \
+ Q(outcome=Build.FAILED) | \
+ Q(outcome=Build.CANCELLED)
recent_builds = list(itertools.chain(
builds.filter(outcome=Build.IN_PROGRESS).order_by("-started_on"),
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 07/15] toaster: bldcontrol models Add a cancelling state the BuildRequest
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
` (5 preceding siblings ...)
2016-03-17 15:41 ` [PATCH 06/15] toaster: models Add cancelled state to build outcome Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 08/15] buildinfohelper: Add handler for cancelling a build Ed Bartosh
` (8 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
From: Michael Wood <michael.g.wood@intel.com>
To accurately reflect the state of a build request we also need a
cancelling state. This is set when we've started a build and then for
whatever reason cancel it, cancelling is not instantaneous so we have
this state to indicate that a cancel is in progress.
Also add a state transition guard. As the state of a BuildRequest can
currently be modified by three processes; Toastergui,
Runbuilds/bldcontrol and the buildinofhelper we cannot say for sure
which process will be running at the time of cancellation so in order to
avoid one of these processes making an incorrect transition only allow
transitions of state to increase.
e.g. CREATED -> QUEUED -> INPROGRESS
And to ignore such requested changes such as
INPROGRESS -> CREATED
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
.../migrations/0002_add_cancelling_state.py | 19 +++++++++++++++
bitbake/lib/toaster/bldcontrol/models.py | 27 +++++++++++++++++++++-
2 files changed, 45 insertions(+), 1 deletion(-)
create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py
diff --git a/bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py b/bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py
new file mode 100644
index 0000000..efcdf49
--- /dev/null
+++ b/bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py
@@ -0,0 +1,19 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+ dependencies = [
+ ('bldcontrol', '0001_initial'),
+ ]
+
+ operations = [
+ migrations.AlterField(
+ model_name='buildrequest',
+ name='state',
+ field=models.IntegerField(default=0, choices=[(0, b'created'), (1, b'queued'), (2, b'in progress'), (3, b'completed'), (4, b'failed'), (5, b'deleted'), (6, b'cancelling'), (7, b'archive')]),
+ ),
+ ]
diff --git a/bitbake/lib/toaster/bldcontrol/models.py b/bitbake/lib/toaster/bldcontrol/models.py
index 9b2d0d0..cb49a58 100644
--- a/bitbake/lib/toaster/bldcontrol/models.py
+++ b/bitbake/lib/toaster/bldcontrol/models.py
@@ -4,6 +4,8 @@ from django.core.validators import MaxValueValidator, MinValueValidator
from django.utils.encoding import force_bytes
from orm.models import Project, ProjectLayer, ProjectVariable, ProjectTarget, Build, Layer_Version
+import logging
+logger = logging.getLogger("toaster")
# a BuildEnvironment is the equivalent of the "build/" directory on the localhost
class BuildEnvironment(models.Model):
SERVER_STOPPED = 0
@@ -64,7 +66,8 @@ class BuildRequest(models.Model):
REQ_COMPLETED = 3
REQ_FAILED = 4
REQ_DELETED = 5
- REQ_ARCHIVE = 6
+ REQ_CANCELLING = 6
+ REQ_ARCHIVE = 7
REQUEST_STATE = (
(REQ_CREATED, "created"),
@@ -73,6 +76,7 @@ class BuildRequest(models.Model):
(REQ_COMPLETED, "completed"),
(REQ_FAILED, "failed"),
(REQ_DELETED, "deleted"),
+ (REQ_CANCELLING, "cancelling"),
(REQ_ARCHIVE, "archive"),
)
@@ -85,6 +89,27 @@ class BuildRequest(models.Model):
created = models.DateTimeField(auto_now_add = True)
updated = models.DateTimeField(auto_now = True)
+ def __init__(self, *args, **kwargs):
+ super(BuildRequest, self).__init__(*args, **kwargs)
+ # Save the old state incase it's about to be modified
+ self.old_state = self.state
+
+ def save(self, *args, **kwargs):
+ # Check that the state we're trying to set is not going backwards
+ # e.g. from REQ_FAILED to REQ_INPROGRESS
+ if self.old_state != self.state and self.old_state > self.state:
+ logger.warn("Invalid state change requested: "
+ "Cannot go from %s to %s - ignoring request" %
+ (BuildRequest.REQUEST_STATE[self.old_state][1],
+ BuildRequest.REQUEST_STATE[self.state][1])
+ )
+ # Set property back to the old value
+ self.state = self.old_state
+ return
+
+ super(BuildRequest, self).save(*args, **kwargs)
+
+
def get_duration(self):
return (self.updated - self.created).total_seconds()
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 08/15] buildinfohelper: Add handler for cancelling a build
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
` (6 preceding siblings ...)
2016-03-17 15:41 ` [PATCH 07/15] toaster: bldcontrol models Add a cancelling state the BuildRequest Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 09/15] toaster: tables BuildsTable exclude cancelled builds Ed Bartosh
` (7 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
From: Michael Wood <michael.g.wood@intel.com>
When a build is cancelled the build (action) is complete if it has been
caused the request being cancelled then update the build outcome
accordingly.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
bitbake/lib/bb/ui/buildinfohelper.py | 19 ++++++++++++++++++-
1 file changed, 18 insertions(+), 1 deletion(-)
diff --git a/bitbake/lib/bb/ui/buildinfohelper.py b/bitbake/lib/bb/ui/buildinfohelper.py
index 79fd37d..dfbce47 100644
--- a/bitbake/lib/bb/ui/buildinfohelper.py
+++ b/bitbake/lib/bb/ui/buildinfohelper.py
@@ -215,6 +215,15 @@ class ORMWrapper(object):
assert isinstance(errors, int)
assert isinstance(warnings, int)
+ if build.outcome == Build.CANCELLED:
+ return
+ try:
+ if build.buildrequest.state == BuildRequest.REQ_CANCELLING:
+ return
+ except AttributeError:
+ # We may not have a buildrequest if this is a command line build
+ pass
+
outcome = Build.SUCCEEDED
if errors or taskfailures:
outcome = Build.FAILED
@@ -1390,9 +1399,17 @@ class BuildInfoHelper(object):
be.lock = BuildEnvironment.LOCK_LOCK
be.save()
br = BuildRequest.objects.get(pk = br_id)
+
+ # if we're 'done' because we got cancelled update the build outcome
+ if br.state == BuildRequest.REQ_CANCELLING:
+ logger.info("Build cancelled")
+ br.build.outcome = Build.CANCELLED
+ br.build.save()
+ errorcode = 0
+
if errorcode == 0:
# request archival of the project artifacts
- br.state = BuildRequest.REQ_ARCHIVE
+ br.state = BuildRequest.REQ_COMPLETED
else:
br.state = BuildRequest.REQ_FAILED
br.save()
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 09/15] toaster: tables BuildsTable exclude cancelled builds
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
` (7 preceding siblings ...)
2016-03-17 15:41 ` [PATCH 08/15] buildinfohelper: Add handler for cancelling a build Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 10/15] toaster: mrb_section template Add build cancel button Ed Bartosh
` (6 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
From: Sujith H <sujith.h@gmail.com>
Exclude cancelled builds from showing in the builds table
[YOCTO #6787]
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
bitbake/lib/toaster/toastergui/tables.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/bitbake/lib/toaster/toastergui/tables.py b/bitbake/lib/toaster/toastergui/tables.py
index 822c7e5..385d658 100644
--- a/bitbake/lib/toaster/toastergui/tables.py
+++ b/bitbake/lib/toaster/toastergui/tables.py
@@ -1080,8 +1080,9 @@ class BuildsTable(ToasterTable):
"""
queryset = self.get_builds()
- # don't include in progress builds
- queryset = queryset.exclude(outcome=Build.IN_PROGRESS)
+ # Don't include in progress builds pr cancelled builds
+ queryset = queryset.exclude(Q(outcome=Build.IN_PROGRESS) |
+ Q(outcome=Build.CANCELLED))
# sort
queryset = queryset.order_by(self.default_orderby)
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 10/15] toaster: mrb_section template Add build cancel button
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
` (8 preceding siblings ...)
2016-03-17 15:41 ` [PATCH 09/15] toaster: tables BuildsTable exclude cancelled builds Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 11/15] toaster: models Exclude the CANCELLED builds from get_number_of_builds Ed Bartosh
` (5 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
From: Michael Wood <michael.g.wood@intel.com>
Add the cancel build button to the mrb section template and add the event
handlers to cancelABuild.
Also clean up the calls to startABuild to use the updated libtoaster methods
and to make the code consistent with it's cancelABuild counterpart.
Co-Author: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
.../lib/toaster/toastergui/static/js/mrbsection.js | 95 +++++++++++++
.../toaster/toastergui/templates/mrb_section.html | 148 ++++++++-------------
2 files changed, 149 insertions(+), 94 deletions(-)
create mode 100644 bitbake/lib/toaster/toastergui/static/js/mrbsection.js
diff --git a/bitbake/lib/toaster/toastergui/static/js/mrbsection.js b/bitbake/lib/toaster/toastergui/static/js/mrbsection.js
new file mode 100644
index 0000000..09117e1
--- /dev/null
+++ b/bitbake/lib/toaster/toastergui/static/js/mrbsection.js
@@ -0,0 +1,95 @@
+
+function mrbSectionInit(ctx){
+
+ var projectBuilds;
+
+ if (ctx.mrbType === 'project')
+ projectBuilds = true;
+
+ $(".cancel-build-btn").click(function(e){
+ e.preventDefault();
+
+ var url = $(this).data('request-url');
+ var buildReqIds = $(this).data('buildrequest-id');
+ var banner = $(this).parents(".alert");
+
+ banner.find(".progress-info").fadeOut().promise().done(function(){
+ $("#cancelling-msg-" + buildReqIds).show();
+ console.log("cancel build");
+ libtoaster.cancelABuild(url, buildReqIds, function(){
+ if (projectBuilds == false){
+ /* the all builds page is not 'self updating' like thei
+ * project Builds
+ */
+ window.location.reload();
+ }
+ }, null);
+ });
+ });
+
+ $(".run-again-btn").click(function(e){
+ e.preventDefault();
+
+ var url = $(this).data('request-url');
+ var target = $(this).data('target');
+
+ libtoaster.startABuild(url, target, function(){
+ window.location.reload();
+ }, null);
+ });
+
+
+ var progressTimer;
+
+ if (projectBuilds === true){
+ progressTimer = window.setInterval(function() {
+ libtoaster.getProjectInfo(libtoaster.ctx.projectPageUrl,
+ function(prjInfo){
+ /* These two are needed because a build can be 100% and still
+ * in progress due to the fact that the % done is updated at the
+ * start of a task so it can be doing the last task at 100%
+ */
+ var inProgress = 0;
+ var allPercentDone = 0;
+ if (prjInfo.builds.length === 0)
+ return
+
+ for (var i in prjInfo.builds){
+ var build = prjInfo.builds[i];
+
+ if (build.outcome === "In Progress" ||
+ $(".progress .bar").length > 0){
+ /* Update the build progress */
+ var percentDone;
+
+ if (build.outcome !== "In Progress"){
+ /* We have to ignore the value when it's Succeeded because it
+ * goes back to 0
+ */
+ percentDone = 100;
+ } else {
+ percentDone = build.percentDone;
+ inProgress++;
+ }
+
+ $("#build-pc-done-" + build.id).text(percentDone);
+ $("#build-pc-done-title-" + build.id).attr("title", percentDone);
+ $("#build-pc-done-bar-" + build.id).css("width",
+ String(percentDone) + "%");
+
+ allPercentDone += percentDone;
+ }
+ }
+
+ if (allPercentDone === (100 * prjInfo.builds.length) && !inProgress)
+ window.location.reload();
+
+ /* Our progress bar is not still showing so shutdown the polling. */
+ if ($(".progress .bar").length === 0)
+ window.clearInterval(progressTimer);
+
+ });
+ }, 1500);
+ }
+}
+
diff --git a/bitbake/lib/toaster/toastergui/templates/mrb_section.html b/bitbake/lib/toaster/toastergui/templates/mrb_section.html
index 551e341..b5e798d 100644
--- a/bitbake/lib/toaster/toastergui/templates/mrb_section.html
+++ b/bitbake/lib/toaster/toastergui/templates/mrb_section.html
@@ -2,6 +2,21 @@
{% load projecttags %}
{% load project_url_tag %}
{% load humanize %}
+<script src="{% static 'js/mrbsection.js' %}"></script>
+<script>
+ $(document).ready(function () {
+ var ctx = {
+ mrbType : "{{mrb_type}}",
+ }
+
+ try {
+ mrbSectionInit(ctx);
+ } catch (e) {
+ document.write("Sorry, An error has occurred loading this page");
+ console.warn(e);
+ }
+ });
+</script>
{%if mru and mru.count > 0%}
@@ -99,7 +114,7 @@
" title="Builds in this project cannot be started from Toaster: they are started from the command line">
</i>
{% else %}
- <button class="btn
+ <button class="run-again-btn btn
{% if build.outcome == build.SUCCEEDED %}
btn-success
{% elif build.outcome == build.FAILED %}
@@ -108,10 +123,9 @@
btn-info
{%endif%}
pull-right"
- onclick='scheduleBuild({% url 'projectbuilds' build.project.id as bpi %}{{bpi|json}},
- {{build.project.name|json}},
- {% url 'project' build.project.id as purl %}{{purl|json}},
- {{build.target_set.all|get_tasks|json}})'>
+ data-request-url="{% url 'xhr_buildrequest' build.project.pk %}"
+ data-target='{{build.target_set.all|get_tasks|json}}'>
+
Rebuild
</button>
@@ -119,100 +133,46 @@
</div>
{%endif%}
{%if build.outcome == build.IN_PROGRESS %}
- <div class="span4 offset1">
- <div class="progress" id="build-pc-done-title-{{build.pk}}" style="margin-top:5px;" data-toggle="tooltip" title="{{build.completeper}}% of tasks complete">
- <div id="build-pc-done-bar-{{build.pk}}" style="width: {{build.completeper}}%;" class="bar"></div>
- </div>
- </div>
- <div class="lead pull-right"><span id="build-pc-done-{{build.pk}}">{{build.completeper}}</span>% of tasks complete</div>
- {%endif%}
- </div>
+ <div class="span4" style="display:none" id="cancelling-msg-{{build.buildrequest.pk}}">
+ <p class="lead">Cancelling the build ...</p>
</div>
+ <div class="span4 offset1 progress-info">
+ <div class="progress" id="build-pc-done-title-{{build.pk}}" style="margin-top:5px;" data-toggle="tooltip" title="{{build.completeper}}% of tasks complete">
+ <div id="build-pc-done-bar-{{build.pk}}" style="width: {{build.completeper}}%;" class="bar"></div>
+ </div>
+ </div>
+ <div class="lead span3 progress-info"><span id="build-pc-done-{{build.pk}}">{{build.completeper}}</span>% of tasks complete</div>
+ {# No build cancel for command line builds project #}
+ {% if build.project.is_default %}
+ <i class="icon-question-sign get-help get-help-blue pull-right" title="" data-original-title="Builds in this project cannot be cancelled from Toaster: they can only be cancalled from the command line"></i>
+ {% else %}
+ <div class="lead pull-right progress-info">
+ <button class="cancel-build-btn btn btn-info pull-right"
+ data-buildrequest-id={{build.buildrequest.pk}}
+ data-request-url="{% url 'xhr_buildrequest' build.project.pk %}" >
+ Cancel
+ </button>
+ </div>
+ {% endif %}
- {% endfor %}
- </div>
-
-<script>
-
-function scheduleBuild(url, projectName, projectUrl, buildlist) {
- console.log("scheduleBuild");
- libtoaster.startABuild(url, null, buildlist.join(" "), function(){
- console.log("reloading page");
- window.location.reload();
- }, null);
-}
-
-$(document).ready(function(){
-
- $(".cancel-build-btn").click(function (){
- var url = $(this).data('request-url');
- var buildIds = $(this).data('build-id');
- var btn = $(this);
-
- libtoaster.cancelABuild(url, buildIds, function(){
- btn.parents(".alert").fadeOut();
- }, null);
- });
-
- {%if mrb_type == 'project' %}
- var projectBuilds = true;
- {% else %}
- var projectBuilds = false;
- {% endif %}
-
- var progressTimer;
-
- if (projectBuilds === true){
- progressTimer = window.setInterval(function() {
- libtoaster.getProjectInfo(libtoaster.ctx.projectPageUrl,
- function(prjInfo){
- /* These two are needed because a build can be 100% and still
- * in progress due to the fact that the % done is updated at the
- * start of a task so it can be doing the last task at 100%
- */
- var inProgress = 0;
- var allPercentDone = 0;
-
- for (var i in prjInfo.builds){
- var build = prjInfo.builds[i];
-
- if (build.outcome === "In Progress" ||
- $(".progress .bar").length > 0){
- /* Update the build progress */
- var percentDone;
-
- if (build.outcome !== "In Progress"){
- /* We have to ignore the value when it's Succeeded because it
- * goes back to 0
- */
- percentDone = 100;
- } else {
- percentDone = build.percentDone;
- inProgress++;
- }
-
- $("#build-pc-done-" + build.id).text(percentDone);
- $("#build-pc-done-title-" + build.id).attr("title", percentDone);
- $("#build-pc-done-bar-" + build.id).css("width",
- String(percentDone) + "%");
-
- allPercentDone += percentDone;
- }
- }
-
- if (allPercentDone === (100 * prjInfo.builds.length) && !inProgress)
- window.location.reload();
+ {%endif%} {# end if in progress #}
- /* Our progress bar is not still showing so shutdown the polling. */
- if ($(".progress .bar").length === 0)
- window.clearInterval(progressTimer);
+ {% if build.outcome == build.CANCELLED %}
+ <div class="span4">
+ <p class="lead">Build cancelled</p>
+ </div>
+ <button class="btn btn-info pull-right run-again-btn"
+ data-request-url="{% url 'xhr_buildrequest' build.project.pk %}"
+ data-target='{{build.target_set.all|get_tasks|json}}'>
+ Rebuild
- });
- }, 1500);
- }
-});
+ </button>
+ {% endif %}
+ </div>
+</div>
-</script>
+ {% endfor %}
+ </div>
{%endif%}
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 11/15] toaster: models Exclude the CANCELLED builds from get_number_of_builds
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
` (9 preceding siblings ...)
2016-03-17 15:41 ` [PATCH 10/15] toaster: mrb_section template Add build cancel button Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 12/15] toaster: runbuilds Make runbuilds aware of the build CANCELLED state Ed Bartosh
` (4 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
From: Michael Wood <michael.g.wood@intel.com>
Don't count CANCELLED builds when returning the number of builds.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
bitbake/lib/toaster/orm/models.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/bitbake/lib/toaster/orm/models.py b/bitbake/lib/toaster/orm/models.py
index e0c2096..d837912 100644
--- a/bitbake/lib/toaster/orm/models.py
+++ b/bitbake/lib/toaster/orm/models.py
@@ -197,7 +197,10 @@ class Project(models.Model):
def get_number_of_builds(self):
"""Return the number of builds which have ended"""
- return self.build_set.filter(~Q(outcome=Build.IN_PROGRESS)).count()
+ return self.build_set.exclude(
+ Q(outcome=Build.IN_PROGRESS) |
+ Q(outcome=Build.CANCELLED)
+ ).count()
def get_last_build_id(self):
try:
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 12/15] toaster: runbuilds Make runbuilds aware of the build CANCELLED state
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
` (10 preceding siblings ...)
2016-03-17 15:41 ` [PATCH 11/15] toaster: models Exclude the CANCELLED builds from get_number_of_builds Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 13/15] toaster: runbuilds Clean up runbuilds Ed Bartosh
` (3 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
From: Michael Wood <michael.g.wood@intel.com>
Add handlers to make sure we remove the BuildEnvironment LOCK when we
have cancelled a build.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
.../bldcontrol/management/commands/runbuilds.py | 26 ++++++++++++++++++----
1 file changed, 22 insertions(+), 4 deletions(-)
diff --git a/bitbake/lib/toaster/bldcontrol/management/commands/runbuilds.py b/bitbake/lib/toaster/bldcontrol/management/commands/runbuilds.py
index 8ba836e..4c1c6a7 100644
--- a/bitbake/lib/toaster/bldcontrol/management/commands/runbuilds.py
+++ b/bitbake/lib/toaster/bldcontrol/management/commands/runbuilds.py
@@ -1,5 +1,7 @@
from django.core.management.base import NoArgsCommand, CommandError
from django.db import transaction
+from django.db.models import Q
+
from orm.models import Build, ToasterSetting, LogMessage, Target
from bldcontrol.bbcontroller import getBuildEnvironmentController, ShellCmdException, BuildSetupException
from bldcontrol.models import BuildRequest, BuildEnvironment, BRError, BRVariable
@@ -86,13 +88,20 @@ class Command(NoArgsCommand):
def cleanup(self):
from django.utils import timezone
from datetime import timedelta
- # environments locked for more than 30 seconds - they should be unlocked
- BuildEnvironment.objects.filter(buildrequest__state__in=[BuildRequest.REQ_FAILED, BuildRequest.REQ_COMPLETED]).filter(lock=BuildEnvironment.LOCK_LOCK).filter(updated__lt = timezone.now() - timedelta(seconds = 30)).update(lock = BuildEnvironment.LOCK_FREE)
-
-
# update all Builds that failed to start
for br in BuildRequest.objects.filter(state = BuildRequest.REQ_FAILED, build__outcome = Build.IN_PROGRESS):
+ # environments locked for more than 30 seconds
+ # they should be unlocked
+ BuildEnvironment.objects.filter(
+ Q(buildrequest__state__in=[BuildRequest.REQ_FAILED,
+ BuildRequest.REQ_COMPLETED,
+ BuildRequest.REQ_CANCELLING]) &
+ Q(lock=BuildEnvironment.LOCK_LOCK) &
+ Q(updated__lt=timezone.now() - timedelta(seconds = 30))
+ ).update(lock=BuildEnvironment.LOCK_FREE)
+
+
# transpose the launch errors in ToasterExceptions
br.build.outcome = Build.FAILED
for brerror in br.brerror_set.all():
@@ -125,6 +134,15 @@ class Command(NoArgsCommand):
br.build.save()
pass
+ # Make sure the LOCK is removed for builds which have been fully
+ # cancelled
+ for br in BuildRequest.objects.filter(
+ Q(build__outcome=Build.CANCELLED) &
+ Q(state=BuildRequest.REQ_CANCELLING) &
+ ~Q(environment=None)):
+ br.environment.lock = BuildEnvironment.LOCK_FREE
+ br.environment.save()
+
def handle_noargs(self, **options):
while True:
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 13/15] toaster: runbuilds Clean up runbuilds
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
` (11 preceding siblings ...)
2016-03-17 15:41 ` [PATCH 12/15] toaster: runbuilds Make runbuilds aware of the build CANCELLED state Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 14/15] toaster: fix conflicting migrations Ed Bartosh
` (2 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
From: Michael Wood <michael.g.wood@intel.com>
- Organise the imports into logical groups
- Fix 80 col wrapping
- Remove catch all exceptions
- Log to the toaster log
- Use QuerySet functions such as .first() and Q()
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
.../bldcontrol/management/commands/runbuilds.py | 81 ++++++++++++++--------
1 file changed, 52 insertions(+), 29 deletions(-)
diff --git a/bitbake/lib/toaster/bldcontrol/management/commands/runbuilds.py b/bitbake/lib/toaster/bldcontrol/management/commands/runbuilds.py
index 4c1c6a7..27289be 100644
--- a/bitbake/lib/toaster/bldcontrol/management/commands/runbuilds.py
+++ b/bitbake/lib/toaster/bldcontrol/management/commands/runbuilds.py
@@ -2,18 +2,25 @@ from django.core.management.base import NoArgsCommand, CommandError
from django.db import transaction
from django.db.models import Q
+from bldcontrol.bbcontroller import getBuildEnvironmentController
+from bldcontrol.bbcontroller import ShellCmdException, BuildSetupException
+from bldcontrol.models import BuildRequest, BuildEnvironment
+from bldcontrol.models import BRError, BRVariable
+
from orm.models import Build, ToasterSetting, LogMessage, Target
-from bldcontrol.bbcontroller import getBuildEnvironmentController, ShellCmdException, BuildSetupException
-from bldcontrol.models import BuildRequest, BuildEnvironment, BRError, BRVariable
+
import os
import logging
import time
+import sys
+import traceback
-logger = logging.getLogger("ToasterScheduler")
+logger = logging.getLogger("toaster")
class Command(NoArgsCommand):
args = ""
- help = "Schedules and executes build requests as possible. Does not return (interrupt with Ctrl-C)"
+ help = "Schedules and executes build requests as possible."
+ "Does not return (interrupt with Ctrl-C)"
@transaction.atomic
@@ -25,21 +32,19 @@ class Command(NoArgsCommand):
@transaction.atomic
def _selectBuildRequest(self):
- br = BuildRequest.objects.filter(state = BuildRequest.REQ_QUEUED).order_by('pk')[0]
- br.state = BuildRequest.REQ_INPROGRESS
- br.save()
+ br = BuildRequest.objects.filter(state=BuildRequest.REQ_QUEUED).first()
return br
def schedule(self):
- import traceback
try:
- br = None
- try:
- # select the build environment and the request to build
- br = self._selectBuildRequest()
- except IndexError as e:
- #logger.debug("runbuilds: No build request")
+ # select the build environment and the request to build
+ br = self._selectBuildRequest()
+ if br:
+ br.state = BuildRequest.REQ_INPROGRESS
+ br.save()
+ else:
return
+
try:
bec = self._selectBuildEnvironment()
except IndexError as e:
@@ -49,15 +54,17 @@ class Command(NoArgsCommand):
logger.debug("runbuilds: No build env")
return
- logger.debug("runbuilds: starting build %s, environment %s" % (str(br).decode('utf-8'), bec.be))
+ logger.debug("runbuilds: starting build %s, environment %s" % \
+ (str(br).decode('utf-8'), bec.be))
# let the build request know where it is being executed
br.environment = bec.be
br.save()
# this triggers an async build
- bec.triggerBuild(br.brbitbake, br.brlayer_set.all(), br.brvariable_set.all(),
- br.brtarget_set.all(), "%d:%d" % (br.pk, bec.be.pk))
+ bec.triggerBuild(br.brbitbake, br.brlayer_set.all(),
+ br.brvariable_set.all(), br.brtarget_set.all(),
+ "%d:%d" % (br.pk, bec.be.pk))
except Exception as e:
logger.error("runbuilds: Error launching build %s" % e)
@@ -88,9 +95,6 @@ class Command(NoArgsCommand):
def cleanup(self):
from django.utils import timezone
from datetime import timedelta
- # update all Builds that failed to start
-
- for br in BuildRequest.objects.filter(state = BuildRequest.REQ_FAILED, build__outcome = Build.IN_PROGRESS):
# environments locked for more than 30 seconds
# they should be unlocked
BuildEnvironment.objects.filter(
@@ -102,22 +106,30 @@ class Command(NoArgsCommand):
).update(lock=BuildEnvironment.LOCK_FREE)
+ # update all Builds that were in progress and failed to start
+ for br in BuildRequest.objects.filter(
+ state=BuildRequest.REQ_FAILED,
+ build__outcome=Build.IN_PROGRESS):
# transpose the launch errors in ToasterExceptions
br.build.outcome = Build.FAILED
for brerror in br.brerror_set.all():
logger.debug("Saving error %s" % brerror)
- LogMessage.objects.create(build = br.build, level = LogMessage.EXCEPTION, message = brerror.errmsg)
+ LogMessage.objects.create(build=br.build,
+ level=LogMessage.EXCEPTION,
+ message=brerror.errmsg)
br.build.save()
- # we don't have a true build object here; hence, toasterui didn't have a change to release the BE lock
+ # we don't have a true build object here; hence, toasterui
+ # didn't have a change to release the BE lock
br.environment.lock = BuildEnvironment.LOCK_FREE
br.environment.save()
-
# update all BuildRequests without a build created
for br in BuildRequest.objects.filter(build = None):
- br.build = Build.objects.create(project = br.project, completed_on = br.updated, started_on = br.created)
+ br.build = Build.objects.create(project=br.project,
+ completed_on=br.updated,
+ started_on=br.created)
br.build.outcome = Build.FAILED
try:
br.build.machine = br.brvariable_set.get(name='MACHINE').value
@@ -126,13 +138,16 @@ class Command(NoArgsCommand):
br.save()
# transpose target information
for brtarget in br.brtarget_set.all():
- Target.objects.create(build=br.build, target=brtarget.target, task=brtarget.task)
+ Target.objects.create(build=br.build,
+ target=brtarget.target,
+ task=brtarget.task)
# transpose the launch errors in ToasterExceptions
for brerror in br.brerror_set.all():
- LogMessage.objects.create(build = br.build, level = LogMessage.EXCEPTION, message = brerror.errmsg)
+ LogMessage.objects.create(build=br.build,
+ level=LogMessage.EXCEPTION,
+ message=brerror.errmsg)
br.build.save()
- pass
# Make sure the LOCK is removed for builds which have been fully
# cancelled
@@ -148,9 +163,17 @@ class Command(NoArgsCommand):
while True:
try:
self.cleanup()
+ except Exception as e:
+ logger.warn("runbuilds: cleanup exception %s" % str(e))
+
+ try:
self.archive()
+ except Exception as e:
+ logger.warn("runbuilds: archive exception %s" % str(e))
+
+ try:
self.schedule()
- except:
- pass
+ except Exception as e:
+ logger.warn("runbuilds: schedule exception %s" % str(e))
time.sleep(1)
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 14/15] toaster: fix conflicting migrations
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
` (12 preceding siblings ...)
2016-03-17 15:41 ` [PATCH 13/15] toaster: runbuilds Clean up runbuilds Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-17 15:41 ` [PATCH 15/15] toaster: use empty token Ed Bartosh
2016-03-19 23:13 ` [PATCH 00/15] Build cancellation Brian Avery
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
Fixed by running manage.py makemigrations --merge
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py | 15 +++++++++++++++
bitbake/lib/toaster/orm/migrations/0007_merge.py | 15 +++++++++++++++
2 files changed, 30 insertions(+)
create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py
create mode 100644 bitbake/lib/toaster/orm/migrations/0007_merge.py
diff --git a/bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py b/bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py
new file mode 100644
index 0000000..b0f6b16
--- /dev/null
+++ b/bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py
@@ -0,0 +1,15 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+ dependencies = [
+ ('bldcontrol', '0002_add_cancelling_state'),
+ ('bldcontrol', '0002_auto_20160120_1250'),
+ ]
+
+ operations = [
+ ]
diff --git a/bitbake/lib/toaster/orm/migrations/0007_merge.py b/bitbake/lib/toaster/orm/migrations/0007_merge.py
new file mode 100644
index 0000000..5d92f336
--- /dev/null
+++ b/bitbake/lib/toaster/orm/migrations/0007_merge.py
@@ -0,0 +1,15 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+ dependencies = [
+ ('orm', '0006_add_cancelled_state'),
+ ('orm', '0006_remove_build_build_name'),
+ ]
+
+ operations = [
+ ]
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 15/15] toaster: use empty token
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
` (13 preceding siblings ...)
2016-03-17 15:41 ` [PATCH 14/15] toaster: fix conflicting migrations Ed Bartosh
@ 2016-03-17 15:41 ` Ed Bartosh
2016-03-19 23:13 ` [PATCH 00/15] Build cancellation Brian Avery
15 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-17 15:41 UTC (permalink / raw)
To: toaster
If client xmlrpc token is not provided in the command line, bitbake
generates random token. Server token in --server-only mode is always
empty. This prevents clients with non-empty tokens to communicate
with the server. Specifying empty token should prevent generation of
random token and makes it possible to communicate with the server.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
bitbake/lib/toaster/bldcontrol/localhostbecontroller.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py b/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py
index 6a6cff8..2c67f1b 100644
--- a/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py
+++ b/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py
@@ -319,7 +319,7 @@ class LocalhostBEController(BuildEnvironmentController):
# run build with local bitbake. stop the server after the build.
log = os.path.join(builddir, 'toaster_ui.log')
self._shellcmd(['bash -c \"(TOASTER_BRBE="%s" BBSERVER="0.0.0.0:-1" '
- '../bitbake/bin/bitbake %s -u toasterui >>%s 2>&1;'
+ '../bitbake/bin/bitbake %s -u toasterui --token="" >>%s 2>&1;'
'BITBAKE_UI="" BBSERVER=0.0.0.0:-1 %s -m)&\"' \
% (brbe, bbtargets, log, bitbake)], builddir,
nowait=True)
--
2.1.4
^ permalink raw reply related [flat|nested] 27+ messages in thread* Re: [PATCH 00/15] Build cancellation
2016-03-17 15:41 [PATCH 00/15] Build cancellation Ed Bartosh
` (14 preceding siblings ...)
2016-03-17 15:41 ` [PATCH 15/15] toaster: use empty token Ed Bartosh
@ 2016-03-19 23:13 ` Brian Avery
2016-03-20 17:12 ` Brian Avery
` (3 more replies)
15 siblings, 4 replies; 27+ messages in thread
From: Brian Avery @ 2016-03-19 23:13 UTC (permalink / raw)
To: Ed Bartosh; +Cc: toaster
Hi,
I finally had a chance to try out the
'poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir'
branch and I had comments:
1) use project name for build dirs? This would make it clearer which
build dir has which projects in it.
2) we cannot assume I am building in ~/poky/build. There are env
variables to let us find where poky is $OEROOT for instance. My bin/sh
is bash but I still got the following when building from
$HOME/cow/test/ when poky was in $HOME/src/poky :
: less toaster_ui.log
bash: ../bitbake/bin/bitbake: No such file or directory
3) if a build fails (as above) and then I cancel it, my state stays in
Cancelling the build ...
4) when I cancel a build it says “Failed” it should probably say
“Canceled” and still be red.
5) If I change one of my base project settings (like my download
folder) the next (and following) projects should probably use my last
setting as the default rather than the hardcoded default.
For instance I change my downloads from /home/…/downloads to
/home/…./DL in cowMaster, when I make cowJethro, the default DL_DIR
should be /home/…./DL
The following were done based on Jethro:
6) built rpi-hwup-image - then made custom image based on it. CI
interface says: “Toaster has no package information for rpime. To
generate package information, build rpime”
7) same true for customizing off off core-image-minimal. for both 6/7
machine = raspberrypi2.
8) trying to download the recipe file gives me the yellow error screen
[Errno 2] No such file or directory:
u'/recipes-core/images/rpi-hwup-image.bb'
9) if I switch back to qemux86 as the machine, build
core-image-minimal, and then customize it I also get the Toaster has
no package information for cimx86me. To generate package information,
build cimx86me
so it is not just due to it being a rpi image. this also gives [Errno
2] No such file or directory:
u'meta/recipes-core/images/core-image-minimal.bb'
if it try to download it.
10) after building the cimx86me, I can now download the recipe but I
still do not see any packages.
11) I’m not personally fond of having the sstate and dl_dir being set
on the project creation page. they are available in the bitbake var
page for each project which is nice. This is more a Belén thing. just
putting my opinion down.
12) if I cancel a command line build from the command line (as the
little question mark suggests) the cancellation is not reflected in
the UI. Mine got stuck at 46% when I cancelled my dropbear cli build.
——
So as far as upstreaming, I think we need to have the following 2
things fixed first:
1) being able to run/build from an arbitrary dir and not assuming we
are in the poky tree. (should be ez)
2) fixing the custom image stuff. Since the CI is the main selling
point of the 2.1 release we need to make sure we don’t break it.
(hopefully not too hard?)
Other issues can be made as bugs on top of the upstream, i think.
Hope this helps,
Brian
an Intel employee
On Thu, Mar 17, 2016 at 8:41 AM, Ed Bartosh <ed.bartosh@linux.intel.com> wrote:
> This is build cancelation functionality implemented by Sujith and Michael
> and modified by me to work on top of 'per project build directory' patchset.
>
> NOTE: This patchset depends on 'per project build directory' patchset' v5:
> https://lists.yoctoproject.org/pipermail/toaster/2016-March/004170.html
>
> The following changes since commit e4ba8a50641b6a2ad8ccda5fc62c61fdf7cef733:
>
> toasterui: shutdown on BuildCompleted event (2016-03-17 17:31:00 +0200)
>
> are available in the git repository at:
>
> git://git.yoctoproject.org/poky-contrib ed/toaster/project-build-dir-7880-cancel
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/project-build-dir-7880-cancel
>
> Ed Bartosh (2):
> toaster: fix conflicting migrations
> toaster: use empty token
>
> Michael Wood (9):
> toaster: Move xhr calls for starting and stopping builds
> toaster: xhr Update the implementation of the build cancellation
> request
> toaster: libtoaster Update implementation of startABuild and
> cancelABuild
> toaster: bldcontrol models Add a cancelling state the BuildRequest
> buildinfohelper: Add handler for cancelling a build
> toaster: mrb_section template Add build cancel button
> toaster: models Exclude the CANCELLED builds from get_number_of_builds
> toaster: runbuilds Make runbuilds aware of the build CANCELLED state
> toaster: runbuilds Clean up runbuilds
>
> Sujith H (4):
> toaster: bldcontrol Add forceShutDown function to BitbakeController
> toaster: update BuildEnvironmentController and BitbakeController
> toaster: models Add cancelled state to build outcome
> toaster: tables BuildsTable exclude cancelled builds
>
> bitbake/lib/bb/ui/buildinfohelper.py | 19 ++-
> bitbake/lib/toaster/bldcontrol/bbcontroller.py | 29 +---
> .../toaster/bldcontrol/localhostbecontroller.py | 4 +-
> .../bldcontrol/management/commands/runbuilds.py | 107 ++++++++++-----
> .../migrations/0002_add_cancelling_state.py | 19 +++
> .../toaster/bldcontrol/migrations/0003_merge.py | 15 +++
> bitbake/lib/toaster/bldcontrol/models.py | 27 +++-
> .../orm/migrations/0006_add_cancelled_state.py | 19 +++
> bitbake/lib/toaster/orm/migrations/0007_merge.py | 15 +++
> bitbake/lib/toaster/orm/models.py | 12 +-
> bitbake/lib/toaster/toastergui/api.py | 110 +++++++++++++++
> .../toaster/toastergui/static/js/customrecipe.js | 4 +-
> .../lib/toaster/toastergui/static/js/layerBtn.js | 3 +-
> .../lib/toaster/toastergui/static/js/libtoaster.js | 41 +++---
> .../lib/toaster/toastergui/static/js/mrbsection.js | 95 +++++++++++++
> .../toaster/toastergui/static/js/projectpage.js | 4 +-
> .../toaster/toastergui/static/js/projecttopbar.js | 6 +-
> .../toaster/toastergui/static/js/recipedetails.js | 4 +-
> bitbake/lib/toaster/toastergui/tables.py | 48 +------
> bitbake/lib/toaster/toastergui/templates/base.html | 1 +
> .../toaster/toastergui/templates/mrb_section.html | 148 ++++++++-------------
> bitbake/lib/toaster/toastergui/urls.py | 5 +
> 22 files changed, 507 insertions(+), 228 deletions(-)
> create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py
> create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py
> create mode 100644 bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py
> create mode 100644 bitbake/lib/toaster/orm/migrations/0007_merge.py
> create mode 100644 bitbake/lib/toaster/toastergui/api.py
> create mode 100644 bitbake/lib/toaster/toastergui/static/js/mrbsection.js
>
> --
> Regards,
> Ed
> --
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 00/15] Build cancellation
2016-03-19 23:13 ` [PATCH 00/15] Build cancellation Brian Avery
@ 2016-03-20 17:12 ` Brian Avery
2016-03-21 10:34 ` Ed Bartosh
2016-03-21 10:14 ` Ed Bartosh
` (2 subsequent siblings)
3 siblings, 1 reply; 27+ messages in thread
From: Brian Avery @ 2016-03-20 17:12 UTC (permalink / raw)
To: Ed Bartosh; +Cc: toaster
a couple of other thoughts after looking at it a little more today:
1) pid of bitbake started in proj builds dir for cleanup? do we want
to save these?
bitbake.lock with lsof might mean we don't need these. however, having
them might negate the lsof requirement...)
2)should poky oe-init be a #!/bin/bash if dash fails?
3) toasterui event loop should probably check for liveness of server
connection…. or do we get an error on waitEvent if the server has
died (segv’d, kill -9 by user, etc)?
4) add tests for start, cancel builds... this needs to be a bugzilla
entry so we have some tests for it. We don't need to do a build,just
make sure we can start/cancel them.
- we may want to add a success.bbclass inteo meta-oe-selftest like
the error.bbclass i added awhile ago. the error class doesn’t build
anything so it’s very fast...
-b
an intel employee
On Sat, Mar 19, 2016 at 4:13 PM, Brian Avery <avery.brian@gmail.com> wrote:
> Hi,
>
> I finally had a chance to try out the
> 'poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir'
> branch and I had comments:
>
> 1) use project name for build dirs? This would make it clearer which
> build dir has which projects in it.
>
>
> 2) we cannot assume I am building in ~/poky/build. There are env
> variables to let us find where poky is $OEROOT for instance. My bin/sh
> is bash but I still got the following when building from
> $HOME/cow/test/ when poky was in $HOME/src/poky :
>
> : less toaster_ui.log
>
> bash: ../bitbake/bin/bitbake: No such file or directory
>
> 3) if a build fails (as above) and then I cancel it, my state stays in
> Cancelling the build ...
>
>
> 4) when I cancel a build it says “Failed” it should probably say
> “Canceled” and still be red.
>
>
> 5) If I change one of my base project settings (like my download
> folder) the next (and following) projects should probably use my last
> setting as the default rather than the hardcoded default.
>
> For instance I change my downloads from /home/…/downloads to
> /home/…./DL in cowMaster, when I make cowJethro, the default DL_DIR
> should be /home/…./DL
>
>
>
>
> The following were done based on Jethro:
>
> 6) built rpi-hwup-image - then made custom image based on it. CI
> interface says: “Toaster has no package information for rpime. To
> generate package information, build rpime”
>
>
> 7) same true for customizing off off core-image-minimal. for both 6/7
> machine = raspberrypi2.
>
>
> 8) trying to download the recipe file gives me the yellow error screen
> [Errno 2] No such file or directory:
> u'/recipes-core/images/rpi-hwup-image.bb'
>
>
> 9) if I switch back to qemux86 as the machine, build
> core-image-minimal, and then customize it I also get the Toaster has
> no package information for cimx86me. To generate package information,
> build cimx86me
>
> so it is not just due to it being a rpi image. this also gives [Errno
> 2] No such file or directory:
> u'meta/recipes-core/images/core-image-minimal.bb'
>
> if it try to download it.
>
>
> 10) after building the cimx86me, I can now download the recipe but I
> still do not see any packages.
>
>
> 11) I’m not personally fond of having the sstate and dl_dir being set
> on the project creation page. they are available in the bitbake var
> page for each project which is nice. This is more a Belén thing. just
> putting my opinion down.
>
>
> 12) if I cancel a command line build from the command line (as the
> little question mark suggests) the cancellation is not reflected in
> the UI. Mine got stuck at 46% when I cancelled my dropbear cli build.
>
>
> ——
>
>
> So as far as upstreaming, I think we need to have the following 2
> things fixed first:
>
> 1) being able to run/build from an arbitrary dir and not assuming we
> are in the poky tree. (should be ez)
>
> 2) fixing the custom image stuff. Since the CI is the main selling
> point of the 2.1 release we need to make sure we don’t break it.
> (hopefully not too hard?)
>
>
> Other issues can be made as bugs on top of the upstream, i think.
>
>
> Hope this helps,
> Brian
> an Intel employee
>
> On Thu, Mar 17, 2016 at 8:41 AM, Ed Bartosh <ed.bartosh@linux.intel.com> wrote:
>> This is build cancelation functionality implemented by Sujith and Michael
>> and modified by me to work on top of 'per project build directory' patchset.
>>
>> NOTE: This patchset depends on 'per project build directory' patchset' v5:
>> https://lists.yoctoproject.org/pipermail/toaster/2016-March/004170.html
>>
>> The following changes since commit e4ba8a50641b6a2ad8ccda5fc62c61fdf7cef733:
>>
>> toasterui: shutdown on BuildCompleted event (2016-03-17 17:31:00 +0200)
>>
>> are available in the git repository at:
>>
>> git://git.yoctoproject.org/poky-contrib ed/toaster/project-build-dir-7880-cancel
>> http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/project-build-dir-7880-cancel
>>
>> Ed Bartosh (2):
>> toaster: fix conflicting migrations
>> toaster: use empty token
>>
>> Michael Wood (9):
>> toaster: Move xhr calls for starting and stopping builds
>> toaster: xhr Update the implementation of the build cancellation
>> request
>> toaster: libtoaster Update implementation of startABuild and
>> cancelABuild
>> toaster: bldcontrol models Add a cancelling state the BuildRequest
>> buildinfohelper: Add handler for cancelling a build
>> toaster: mrb_section template Add build cancel button
>> toaster: models Exclude the CANCELLED builds from get_number_of_builds
>> toaster: runbuilds Make runbuilds aware of the build CANCELLED state
>> toaster: runbuilds Clean up runbuilds
>>
>> Sujith H (4):
>> toaster: bldcontrol Add forceShutDown function to BitbakeController
>> toaster: update BuildEnvironmentController and BitbakeController
>> toaster: models Add cancelled state to build outcome
>> toaster: tables BuildsTable exclude cancelled builds
>>
>> bitbake/lib/bb/ui/buildinfohelper.py | 19 ++-
>> bitbake/lib/toaster/bldcontrol/bbcontroller.py | 29 +---
>> .../toaster/bldcontrol/localhostbecontroller.py | 4 +-
>> .../bldcontrol/management/commands/runbuilds.py | 107 ++++++++++-----
>> .../migrations/0002_add_cancelling_state.py | 19 +++
>> .../toaster/bldcontrol/migrations/0003_merge.py | 15 +++
>> bitbake/lib/toaster/bldcontrol/models.py | 27 +++-
>> .../orm/migrations/0006_add_cancelled_state.py | 19 +++
>> bitbake/lib/toaster/orm/migrations/0007_merge.py | 15 +++
>> bitbake/lib/toaster/orm/models.py | 12 +-
>> bitbake/lib/toaster/toastergui/api.py | 110 +++++++++++++++
>> .../toaster/toastergui/static/js/customrecipe.js | 4 +-
>> .../lib/toaster/toastergui/static/js/layerBtn.js | 3 +-
>> .../lib/toaster/toastergui/static/js/libtoaster.js | 41 +++---
>> .../lib/toaster/toastergui/static/js/mrbsection.js | 95 +++++++++++++
>> .../toaster/toastergui/static/js/projectpage.js | 4 +-
>> .../toaster/toastergui/static/js/projecttopbar.js | 6 +-
>> .../toaster/toastergui/static/js/recipedetails.js | 4 +-
>> bitbake/lib/toaster/toastergui/tables.py | 48 +------
>> bitbake/lib/toaster/toastergui/templates/base.html | 1 +
>> .../toaster/toastergui/templates/mrb_section.html | 148 ++++++++-------------
>> bitbake/lib/toaster/toastergui/urls.py | 5 +
>> 22 files changed, 507 insertions(+), 228 deletions(-)
>> create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py
>> create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py
>> create mode 100644 bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py
>> create mode 100644 bitbake/lib/toaster/orm/migrations/0007_merge.py
>> create mode 100644 bitbake/lib/toaster/toastergui/api.py
>> create mode 100644 bitbake/lib/toaster/toastergui/static/js/mrbsection.js
>>
>> --
>> Regards,
>> Ed
>> --
>> _______________________________________________
>> toaster mailing list
>> toaster@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/toaster
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 00/15] Build cancellation
2016-03-20 17:12 ` Brian Avery
@ 2016-03-21 10:34 ` Ed Bartosh
2016-03-21 18:07 ` Brian Avery
0 siblings, 1 reply; 27+ messages in thread
From: Ed Bartosh @ 2016-03-21 10:34 UTC (permalink / raw)
To: Brian Avery; +Cc: toaster
On Sun, Mar 20, 2016 at 10:12:22AM -0700, Brian Avery wrote:
> a couple of other thoughts after looking at it a little more today:
>
> 1) pid of bitbake started in proj builds dir for cleanup? do we want
> to save these?
not until we use them.
> bitbake.lock with lsof might mean we don't need these. however, having
> them might negate the lsof requirement...)
I'll look what I can do. I was planning to reuse what I did in my old
ed/toaster/bec branch: to generate shell script for cloning all repos
and running the build. It would be much more easy to deal with
pid files in the shell script than in current python code.
However, I think it's better not to do right now as we already have
quite a bit of changes and very close to release.
> 2)should poky oe-init be a #!/bin/bash if dash fails?
No, we shouldn't as it works with zsh.
> 3) toasterui event loop should probably check for liveness of server
> connection…. or do we get an error on waitEvent if the server has
> died (segv’d, kill -9 by user, etc)?
We run sever only during the build. We probably need to set build state to
'Failed' if server is killed or crashed.
Regards,
Ed
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 00/15] Build cancellation
2016-03-21 10:34 ` Ed Bartosh
@ 2016-03-21 18:07 ` Brian Avery
2016-03-21 18:37 ` Barros Pena, Belen
0 siblings, 1 reply; 27+ messages in thread
From: Brian Avery @ 2016-03-21 18:07 UTC (permalink / raw)
To: Ed Bartosh; +Cc: toaster
Hi!
On Mon, Mar 21, 2016 at 3:34 AM, Ed Bartosh <ed.bartosh@linux.intel.com> wrote:
> On Sun, Mar 20, 2016 at 10:12:22AM -0700, Brian Avery wrote:
>> a couple of other thoughts after looking at it a little more today:
>>
>> 1) pid of bitbake started in proj builds dir for cleanup? do we want
>> to save these?
> not until we use them.
>
>> bitbake.lock with lsof might mean we don't need these. however, having
>> them might negate the lsof requirement...)
> I'll look what I can do. I was planning to reuse what I did in my old
> ed/toaster/bec branch: to generate shell script for cloning all repos
> and running the build. It would be much more easy to deal with
> pid files in the shell script than in current python code.
> However, I think it's better not to do right now as we already have
> quite a bit of changes and very close to release.
>
k. was more of a me musing about it than anything else...
>> 2)should poky oe-init be a #!/bin/bash if dash fails?
> No, we shouldn't as it works with zsh.
>
k. btw, how did it fail? I just ran oe-init and bitbake
core-image-minimal in an ubuntu:14.04 container and it seemed fine.
/bin/sh was liked to dash...
>> 3) toasterui event loop should probably check for liveness of server
>> connection…. or do we get an error on waitEvent if the server has
>> died (segv’d, kill -9 by user, etc)?
> We run sever only during the build. We probably need to set build state to
> 'Failed' if server is killed or crashed.
yep. though it can easily be a follow on bug.
>
> Regards,
> Ed
My thoughts:
since Ed has fixed the ../bitbake/bin issue and if
https://lists.yoctoproject.org/pipermail/toaster/2016-March/004162.html
fixes the CI not working I'm good with upstreaming and filing
additional bugs for the corner cases since this fixes alot.
TY!
-brian
-an intel employee
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 00/15] Build cancellation
2016-03-21 18:07 ` Brian Avery
@ 2016-03-21 18:37 ` Barros Pena, Belen
2016-03-21 21:18 ` Brian Avery
0 siblings, 1 reply; 27+ messages in thread
From: Barros Pena, Belen @ 2016-03-21 18:37 UTC (permalink / raw)
To: toaster@yoctoproject.org
On 21/03/2016 18:07, "toaster-bounces@yoctoproject.org on behalf of Brian
Avery" <toaster-bounces@yoctoproject.org on behalf of
avery.brian@gmail.com> wrote:
>My thoughts:
>since Ed has fixed the ../bitbake/bin issue and if
>https://lists.yoctoproject.org/pipermail/toaster/2016-March/004162.html
>fixes the CI not working I'm good with upstreaming and filing
>additional bugs for the corner cases since this fixes alot.
I finally managed to look into
poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir
but my looking was brief. I can't get any builds started from the web UI:
none of the 'build' buttons seem to work. I click and nothing happens.
I can build from the command line though, and the builds do show up in
Toaster.
I wonder what I am doing wrong this time ...
Cheers
Belén
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 00/15] Build cancellation
2016-03-21 18:37 ` Barros Pena, Belen
@ 2016-03-21 21:18 ` Brian Avery
2016-03-22 13:45 ` Barros Pena, Belen
0 siblings, 1 reply; 27+ messages in thread
From: Brian Avery @ 2016-03-21 21:18 UTC (permalink / raw)
To: Barros Pena, Belen; +Cc: toaster@yoctoproject.org
what happens when starting builds from toaster?
If you are seeing :
> : less toaster_ui.log
>
> bash: ../bitbake/bin/bitbake: No such file or directory
Then Ed says ->
Should be fixed by this commit:
https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir&id=a3566432d7d42939a482322abb7ead6026008d49
-b
...
On Mon, Mar 21, 2016 at 11:37 AM, Barros Pena, Belen
<belen.barros.pena@intel.com> wrote:
>
>
> On 21/03/2016 18:07, "toaster-bounces@yoctoproject.org on behalf of Brian
> Avery" <toaster-bounces@yoctoproject.org on behalf of
> avery.brian@gmail.com> wrote:
>
>>My thoughts:
>>since Ed has fixed the ../bitbake/bin issue and if
>>https://lists.yoctoproject.org/pipermail/toaster/2016-March/004162.html
>>fixes the CI not working I'm good with upstreaming and filing
>>additional bugs for the corner cases since this fixes alot.
>
> I finally managed to look into
>
> poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir
>
> but my looking was brief. I can't get any builds started from the web UI:
> none of the 'build' buttons seem to work. I click and nothing happens.
>
> I can build from the command line though, and the builds do show up in
> Toaster.
>
> I wonder what I am doing wrong this time ...
>
> Cheers
>
> Belén
>
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 00/15] Build cancellation
2016-03-21 21:18 ` Brian Avery
@ 2016-03-22 13:45 ` Barros Pena, Belen
0 siblings, 0 replies; 27+ messages in thread
From: Barros Pena, Belen @ 2016-03-22 13:45 UTC (permalink / raw)
To: Brian Avery; +Cc: toaster@yoctoproject.org
On 21/03/2016 21:18, "Brian Avery" <avery.brian@gmail.com> wrote:
>what happens when starting builds from toaster?
>
>If you are seeing :
>> : less toaster_ui.log
>>
>> bash: ../bitbake/bin/bitbake: No such file or directory
>Then Ed says ->
>Should be fixed by this commit:
>https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=ed/toast
>er/project-build-dir-7880-cancel-dldir_sstatedir&id=a3566432d7d42939a48232
>2abb7ead6026008d49
Builds are now working in the branch: I can start them and cancel them.
Unfortunately, something has gone funny when applying Michael's patch for
the image customisation problem, so that is not fixed yet. Ed is looking
into it.
Thanks!
Belén
>
>-b
>...
>
>On Mon, Mar 21, 2016 at 11:37 AM, Barros Pena, Belen
><belen.barros.pena@intel.com> wrote:
>>
>>
>> On 21/03/2016 18:07, "toaster-bounces@yoctoproject.org on behalf of
>>Brian
>> Avery" <toaster-bounces@yoctoproject.org on behalf of
>> avery.brian@gmail.com> wrote:
>>
>>>My thoughts:
>>>since Ed has fixed the ../bitbake/bin issue and if
>>>https://lists.yoctoproject.org/pipermail/toaster/2016-March/004162.html
>>>fixes the CI not working I'm good with upstreaming and filing
>>>additional bugs for the corner cases since this fixes alot.
>>
>> I finally managed to look into
>>
>> poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir
>>
>> but my looking was brief. I can't get any builds started from the web
>>UI:
>> none of the 'build' buttons seem to work. I click and nothing happens.
>>
>> I can build from the command line though, and the builds do show up in
>> Toaster.
>>
>> I wonder what I am doing wrong this time ...
>>
>> Cheers
>>
>> Belén
>>
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 00/15] Build cancellation
2016-03-19 23:13 ` [PATCH 00/15] Build cancellation Brian Avery
2016-03-20 17:12 ` Brian Avery
@ 2016-03-21 10:14 ` Ed Bartosh
2016-03-21 11:14 ` Smith, Elliot
2016-03-21 11:36 ` Barros Pena, Belen
3 siblings, 0 replies; 27+ messages in thread
From: Ed Bartosh @ 2016-03-21 10:14 UTC (permalink / raw)
To: Brian Avery; +Cc: toaster
On Sat, Mar 19, 2016 at 04:13:27PM -0700, Brian Avery wrote:
> 2) we cannot assume I am building in ~/poky/build. There are env
> variables to let us find where poky is $OEROOT for instance. My bin/sh
> is bash but I still got the following when building from
> $HOME/cow/test/ when poky was in $HOME/src/poky :
>
> : less toaster_ui.log
>
> bash: ../bitbake/bin/bitbake: No such file or directory
Should be fixed by this commit:
https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir&id=a3566432d7d42939a482322abb7ead6026008d49
...
> 11) I’m not personally fond of having the sstate and dl_dir being set
> on the project creation page. they are available in the bitbake var
> page for each project which is nice. This is more a Belén thing. just
> putting my opinion down.
This should be alread fixed by Sujith. The fix is also in
ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir
...
> So as far as upstreaming, I think we need to have the following 2
> things fixed first:
>
> 1) being able to run/build from an arbitrary dir and not assuming we
> are in the poky tree. (should be ez)
yes, it should be fixed already. see my comment above.
> 2) fixing the custom image stuff. Since the CI is the main selling
> point of the 2.1 release we need to make sure we don’t break it.
> (hopefully not too hard?)
OK, I'll try to look at it.
BTW, the full patchset has 42 patches and becomes hard to maintain.
We may need to consider upstreaming it in 2 or 3 steps if the above
fixes are hard to do.
Regards,
Ed
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 00/15] Build cancellation
2016-03-19 23:13 ` [PATCH 00/15] Build cancellation Brian Avery
2016-03-20 17:12 ` Brian Avery
2016-03-21 10:14 ` Ed Bartosh
@ 2016-03-21 11:14 ` Smith, Elliot
2016-03-21 11:36 ` Barros Pena, Belen
3 siblings, 0 replies; 27+ messages in thread
From: Smith, Elliot @ 2016-03-21 11:14 UTC (permalink / raw)
To: Brian Avery; +Cc: toaster
[-- Attachment #1: Type: text/plain, Size: 1263 bytes --]
Brian, a few comments on your comments...
On 19 March 2016 at 23:13, Brian Avery <avery.brian@gmail.com> wrote:
> Hi,
>
> I finally had a chance to try out the
> 'poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir'
> branch and I had comments:
>
> 1) use project name for build dirs? This would make it clearer which
> build dir has which projects in it.
>
This introduces the problem of what to do if a project's name changes,
which could happen in the middle of a build; it would also need additional
code to sanitise the project name to produce a valid directory name. It
seems simpler to me to just use the build ID, as it's not a massive stretch
to figure out which build goes with which numbered directory.
...
>
> 11) I’m not personally fond of having the sstate and dl_dir being set
> on the project creation page. they are available in the bitbake var
> page for each project which is nice. This is more a Belén thing. just
> putting my opinion down.
>
After discussion on the list, Sujith has provided a version of Ed's branch
which removes these fields from the new project page:
sujith/set_dldir_sstatedirv2
Elliot
--
Elliot Smith
Software Engineer
Intel Open Source Technology Centre
[-- Attachment #2: Type: text/html, Size: 2041 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 00/15] Build cancellation
2016-03-19 23:13 ` [PATCH 00/15] Build cancellation Brian Avery
` (2 preceding siblings ...)
2016-03-21 11:14 ` Smith, Elliot
@ 2016-03-21 11:36 ` Barros Pena, Belen
2016-03-21 18:04 ` Brian Avery
3 siblings, 1 reply; 27+ messages in thread
From: Barros Pena, Belen @ 2016-03-21 11:36 UTC (permalink / raw)
To: Brian Avery, Ed Bartosh; +Cc: toaster@yoctoproject.org
On 19/03/2016 23:13, "toaster-bounces@yoctoproject.org on behalf of Brian
Avery" <toaster-bounces@yoctoproject.org on behalf of
avery.brian@gmail.com> wrote:
>Hi,
>
>I finally had a chance to try out the
>'poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir'
>branch and I had comments:
>
>1) use project name for build dirs? This would make it clearer which
>build dir has which projects in it.
We had a discussion about this, part of which Elliot has summarised in his
previous email. It is true that it might be easier to identify the build
directory of a project if we use the project name, but Toaster does not
enforce unique names, it does not validate for special characters or
length limits, and it allows you change the project name at any time. The
problems all these could cause made us go with the project id. To ease
identification, we could show the project id and the path to the build
directory in the project page, unless anybody has issues with displaying
this information.
>
>
>2) we cannot assume I am building in ~/poky/build. There are env
>variables to let us find where poky is $OEROOT for instance. My bin/sh
>is bash but I still got the following when building from
>$HOME/cow/test/ when poky was in $HOME/src/poky :
>
>: less toaster_ui.log
>
>bash: ../bitbake/bin/bitbake: No such file or directory
>
>3) if a build fails (as above) and then I cancel it, my state stays in
>Cancelling the build ...
This sounds like a bug: you should never be able to cancel a failed build.
The 'cancel' button should only be available while bitbake is running.
>
>
>4) when I cancel a build it says ³Failed² it should probably say
>³Canceled² and still be red.
This is also a bug. Cancelled builds have their own presentation (they say
"cancelled"). See page 4 of this design document
https://bugzilla.yoctoproject.org/attachment.cgi?id=2706
>
>
>5) If I change one of my base project settings (like my download
>folder) the next (and following) projects should probably use my last
>setting as the default rather than the hardcoded default.
I am not sure I agree. The UI right now provides only per project
configuration: I wouldn't make exceptions to that rule. If we did, you
would be in a project context surrounded by variables that only apply to
the selected project, and all of a sudden you would have 2 variables that
look the same as all others but behave differently and override the
Toaster defaults.
If you want to override the Toaster defaults, you should change the
default value via the Django admin interface or in the toasterconf.json
file. The UI only provides access to project configuration at this time
... consistently limited ;)
>
>For instance I change my downloads from /home/Š/downloads to
>/home/Š./DL in cowMaster, when I make cowJethro, the default DL_DIR
>should be /home/Š./DL
>
>
>The following were done based on Jethro:
>
>6) built rpi-hwup-image - then made custom image based on it. CI
>interface says: ³Toaster has no package information for rpime. To
>generate package information, build rpime²
>
>
>7) same true for customizing off off core-image-minimal. for both 6/7
>machine = raspberrypi2.
I've seen this too, but I assumed is was caused by this:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=9266
Although I could be completely wrong. There is a patch under review to fix
it
https://lists.yoctoproject.org/pipermail/toaster/2016-March/004162.html
But I haven't had a chance to try it on top of Ed's branch.
>
>
>8) trying to download the recipe file gives me the yellow error screen
>[Errno 2] No such file or directory:
>u'/recipes-core/images/rpi-hwup-image.bb'
>
>
>9) if I switch back to qemux86 as the machine, build
>core-image-minimal, and then customize it I also get the Toaster has
>no package information for cimx86me. To generate package information,
>build cimx86me
>
>so it is not just due to it being a rpi image. this also gives [Errno
>2] No such file or directory:
>u'meta/recipes-core/images/core-image-minimal.bb'
>
>if it try to download it.
>
>
>10) after building the cimx86me, I can now download the recipe but I
>still do not see any packages.
>
>
>11) I¹m not personally fond of having the sstate and dl_dir being set
>on the project creation page. they are available in the bitbake var
>page for each project which is nice. This is more a Belén thing. just
>putting my opinion down.
We also discussed this (everybody complained) and decided to remove them.
>
>
>12) if I cancel a command line build from the command line (as the
>little question mark suggests) the cancellation is not reflected in
>the UI. Mine got stuck at 46% when I cancelled my dropbear cli build.
>
>
>‹‹
>
>
>So as far as upstreaming, I think we need to have the following 2
>things fixed first:
>
>1) being able to run/build from an arbitrary dir and not assuming we
>are in the poky tree. (should be ez)
>
>2) fixing the custom image stuff. Since the CI is the main selling
>point of the 2.1 release we need to make sure we don¹t break it.
>(hopefully not too hard?)
>
>
>Other issues can be made as bugs on top of the upstream, i think.
>
>
>Hope this helps,
>Brian
>an Intel employee
>
>On Thu, Mar 17, 2016 at 8:41 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
>wrote:
>> This is build cancelation functionality implemented by Sujith and
>>Michael
>> and modified by me to work on top of 'per project build directory'
>>patchset.
>>
>> NOTE: This patchset depends on 'per project build directory' patchset'
>>v5:
>>
>>https://lists.yoctoproject.org/pipermail/toaster/2016-March/004170.html
>>
>> The following changes since commit
>>e4ba8a50641b6a2ad8ccda5fc62c61fdf7cef733:
>>
>> toasterui: shutdown on BuildCompleted event (2016-03-17 17:31:00
>>+0200)
>>
>> are available in the git repository at:
>>
>> git://git.yoctoproject.org/poky-contrib
>>ed/toaster/project-build-dir-7880-cancel
>>
>>http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/proje
>>ct-build-dir-7880-cancel
>>
>> Ed Bartosh (2):
>> toaster: fix conflicting migrations
>> toaster: use empty token
>>
>> Michael Wood (9):
>> toaster: Move xhr calls for starting and stopping builds
>> toaster: xhr Update the implementation of the build cancellation
>> request
>> toaster: libtoaster Update implementation of startABuild and
>> cancelABuild
>> toaster: bldcontrol models Add a cancelling state the BuildRequest
>> buildinfohelper: Add handler for cancelling a build
>> toaster: mrb_section template Add build cancel button
>> toaster: models Exclude the CANCELLED builds from get_number_of_builds
>> toaster: runbuilds Make runbuilds aware of the build CANCELLED state
>> toaster: runbuilds Clean up runbuilds
>>
>> Sujith H (4):
>> toaster: bldcontrol Add forceShutDown function to BitbakeController
>> toaster: update BuildEnvironmentController and BitbakeController
>> toaster: models Add cancelled state to build outcome
>> toaster: tables BuildsTable exclude cancelled builds
>>
>> bitbake/lib/bb/ui/buildinfohelper.py | 19 ++-
>> bitbake/lib/toaster/bldcontrol/bbcontroller.py | 29 +---
>> .../toaster/bldcontrol/localhostbecontroller.py | 4 +-
>> .../bldcontrol/management/commands/runbuilds.py | 107
>>++++++++++-----
>> .../migrations/0002_add_cancelling_state.py | 19 +++
>> .../toaster/bldcontrol/migrations/0003_merge.py | 15 +++
>> bitbake/lib/toaster/bldcontrol/models.py | 27 +++-
>> .../orm/migrations/0006_add_cancelled_state.py | 19 +++
>> bitbake/lib/toaster/orm/migrations/0007_merge.py | 15 +++
>> bitbake/lib/toaster/orm/models.py | 12 +-
>> bitbake/lib/toaster/toastergui/api.py | 110
>>+++++++++++++++
>> .../toaster/toastergui/static/js/customrecipe.js | 4 +-
>> .../lib/toaster/toastergui/static/js/layerBtn.js | 3 +-
>> .../lib/toaster/toastergui/static/js/libtoaster.js | 41 +++---
>> .../lib/toaster/toastergui/static/js/mrbsection.js | 95 +++++++++++++
>> .../toaster/toastergui/static/js/projectpage.js | 4 +-
>> .../toaster/toastergui/static/js/projecttopbar.js | 6 +-
>> .../toaster/toastergui/static/js/recipedetails.js | 4 +-
>> bitbake/lib/toaster/toastergui/tables.py | 48 +------
>> bitbake/lib/toaster/toastergui/templates/base.html | 1 +
>> .../toaster/toastergui/templates/mrb_section.html | 148
>>++++++++-------------
>> bitbake/lib/toaster/toastergui/urls.py | 5 +
>> 22 files changed, 507 insertions(+), 228 deletions(-)
>> create mode 100644
>>bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py
>> create mode 100644
>>bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py
>> create mode 100644
>>bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py
>> create mode 100644 bitbake/lib/toaster/orm/migrations/0007_merge.py
>> create mode 100644 bitbake/lib/toaster/toastergui/api.py
>> create mode 100644
>>bitbake/lib/toaster/toastergui/static/js/mrbsection.js
>>
>> --
>> Regards,
>> Ed
>> --
>> _______________________________________________
>> toaster mailing list
>> toaster@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/toaster
>--
>_______________________________________________
>toaster mailing list
>toaster@yoctoproject.org
>https://lists.yoctoproject.org/listinfo/toaster
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 00/15] Build cancellation
2016-03-21 11:36 ` Barros Pena, Belen
@ 2016-03-21 18:04 ` Brian Avery
0 siblings, 0 replies; 27+ messages in thread
From: Brian Avery @ 2016-03-21 18:04 UTC (permalink / raw)
To: Barros Pena, Belen; +Cc: toaster@yoctoproject.org
Hi,
Responding to comments on my comments:
On Mon, Mar 21, 2016 at 4:36 AM, Barros Pena, Belen
<belen.barros.pena@intel.com> wrote:
>
>
> On 19/03/2016 23:13, "toaster-bounces@yoctoproject.org on behalf of Brian
> Avery" <toaster-bounces@yoctoproject.org on behalf of
> avery.brian@gmail.com> wrote:
>
>>Hi,
>>
>>I finally had a chance to try out the
>>'poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir'
>>branch and I had comments:
>>
>>1) use project name for build dirs? This would make it clearer which
>>build dir has which projects in it.
>
> We had a discussion about this, part of which Elliot has summarised in his
> previous email. It is true that it might be easier to identify the build
> directory of a project if we use the project name, but Toaster does not
> enforce unique names, it does not validate for special characters or
> length limits, and it allows you change the project name at any time. The
> problems all these could cause made us go with the project id. To ease
> identification, we could show the project id and the path to the build
> directory in the project page, unless anybody has issues with displaying
> this information.
>
That would work for me. I just wanted an ez way to tie them together
and i feared the proj_id would not be clear to users.
>>
>>
>>2) we cannot assume I am building in ~/poky/build. There are env
>>variables to let us find where poky is $OEROOT for instance. My bin/sh
>>is bash but I still got the following when building from
>>$HOME/cow/test/ when poky was in $HOME/src/poky :
>>
>>: less toaster_ui.log
>>
>>bash: ../bitbake/bin/bitbake: No such file or directory
>>
>>3) if a build fails (as above) and then I cancel it, my state stays in
>>Cancelling the build ...
>
> This sounds like a bug: you should never be able to cancel a failed build.
> The 'cancel' button should only be available while bitbake is running.
>
>>
>>
>>4) when I cancel a build it says ³Failed² it should probably say
>>³Canceled² and still be red.
>
> This is also a bug. Cancelled builds have their own presentation (they say
> "cancelled"). See page 4 of this design document
>
> https://bugzilla.yoctoproject.org/attachment.cgi?id=2706
>
>>
>>
>>5) If I change one of my base project settings (like my download
>>folder) the next (and following) projects should probably use my last
>>setting as the default rather than the hardcoded default.
>
> I am not sure I agree. The UI right now provides only per project
> configuration: I wouldn't make exceptions to that rule. If we did, you
> would be in a project context surrounded by variables that only apply to
> the selected project, and all of a sudden you would have 2 variables that
> look the same as all others but behave differently and override the
> Toaster defaults.
>
> If you want to override the Toaster defaults, you should change the
> default value via the Django admin interface or in the toasterconf.json
> file. The UI only provides access to project configuration at this time
> ... consistently limited ;)
>
fair point. and since Elliot said they were moving off the main proj
page i withdraw the suggestion.
>>
>>For instance I change my downloads from /home/Š/downloads to
>>/home/Š./DL in cowMaster, when I make cowJethro, the default DL_DIR
>>should be /home/Š./DL
>>
>>
>>The following were done based on Jethro:
>>
>>6) built rpi-hwup-image - then made custom image based on it. CI
>>interface says: ³Toaster has no package information for rpime. To
>>generate package information, build rpime²
>>
>>
>>7) same true for customizing off off core-image-minimal. for both 6/7
>>machine = raspberrypi2.
>
> I've seen this too, but I assumed is was caused by this:
>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=9266
>
> Although I could be completely wrong. There is a patch under review to fix
> it
>
> https://lists.yoctoproject.org/pipermail/toaster/2016-March/004162.html
>
> But I haven't had a chance to try it on top of Ed's branch.
Could be true; i hope so :).
>
>>
>>
>>8) trying to download the recipe file gives me the yellow error screen
>>[Errno 2] No such file or directory:
>>u'/recipes-core/images/rpi-hwup-image.bb'
>>
>>
>>9) if I switch back to qemux86 as the machine, build
>>core-image-minimal, and then customize it I also get the Toaster has
>>no package information for cimx86me. To generate package information,
>>build cimx86me
>>
>>so it is not just due to it being a rpi image. this also gives [Errno
>>2] No such file or directory:
>>u'meta/recipes-core/images/core-image-minimal.bb'
>>
>>if it try to download it.
>>
>>
>>10) after building the cimx86me, I can now download the recipe but I
>>still do not see any packages.
>>
>>
>>11) I¹m not personally fond of having the sstate and dl_dir being set
>>on the project creation page. they are available in the bitbake var
>>page for each project which is nice. This is more a Belén thing. just
>>putting my opinion down.
>
> We also discussed this (everybody complained) and decided to remove them.
>
>>
>>
>>12) if I cancel a command line build from the command line (as the
>>little question mark suggests) the cancellation is not reflected in
>>the UI. Mine got stuck at 46% when I cancelled my dropbear cli build.
>>
>>
>>‹‹
>>
>>
>>So as far as upstreaming, I think we need to have the following 2
>>things fixed first:
>>
>>1) being able to run/build from an arbitrary dir and not assuming we
>>are in the poky tree. (should be ez)
>>
>>2) fixing the custom image stuff. Since the CI is the main selling
>>point of the 2.1 release we need to make sure we don¹t break it.
>>(hopefully not too hard?)
>>
>>
>>Other issues can be made as bugs on top of the upstream, i think.
>>
>>
>>Hope this helps,
>>Brian
>>an Intel employee
>>
>>On Thu, Mar 17, 2016 at 8:41 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
>>wrote:
>>> This is build cancelation functionality implemented by Sujith and
>>>Michael
>>> and modified by me to work on top of 'per project build directory'
>>>patchset.
>>>
>>> NOTE: This patchset depends on 'per project build directory' patchset'
>>>v5:
>>>
>>>https://lists.yoctoproject.org/pipermail/toaster/2016-March/004170.html
>>>
>>> The following changes since commit
>>>e4ba8a50641b6a2ad8ccda5fc62c61fdf7cef733:
>>>
>>> toasterui: shutdown on BuildCompleted event (2016-03-17 17:31:00
>>>+0200)
>>>
>>> are available in the git repository at:
>>>
>>> git://git.yoctoproject.org/poky-contrib
>>>ed/toaster/project-build-dir-7880-cancel
>>>
>>>http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/proje
>>>ct-build-dir-7880-cancel
>>>
>>> Ed Bartosh (2):
>>> toaster: fix conflicting migrations
>>> toaster: use empty token
>>>
>>> Michael Wood (9):
>>> toaster: Move xhr calls for starting and stopping builds
>>> toaster: xhr Update the implementation of the build cancellation
>>> request
>>> toaster: libtoaster Update implementation of startABuild and
>>> cancelABuild
>>> toaster: bldcontrol models Add a cancelling state the BuildRequest
>>> buildinfohelper: Add handler for cancelling a build
>>> toaster: mrb_section template Add build cancel button
>>> toaster: models Exclude the CANCELLED builds from get_number_of_builds
>>> toaster: runbuilds Make runbuilds aware of the build CANCELLED state
>>> toaster: runbuilds Clean up runbuilds
>>>
>>> Sujith H (4):
>>> toaster: bldcontrol Add forceShutDown function to BitbakeController
>>> toaster: update BuildEnvironmentController and BitbakeController
>>> toaster: models Add cancelled state to build outcome
>>> toaster: tables BuildsTable exclude cancelled builds
>>>
>>> bitbake/lib/bb/ui/buildinfohelper.py | 19 ++-
>>> bitbake/lib/toaster/bldcontrol/bbcontroller.py | 29 +---
>>> .../toaster/bldcontrol/localhostbecontroller.py | 4 +-
>>> .../bldcontrol/management/commands/runbuilds.py | 107
>>>++++++++++-----
>>> .../migrations/0002_add_cancelling_state.py | 19 +++
>>> .../toaster/bldcontrol/migrations/0003_merge.py | 15 +++
>>> bitbake/lib/toaster/bldcontrol/models.py | 27 +++-
>>> .../orm/migrations/0006_add_cancelled_state.py | 19 +++
>>> bitbake/lib/toaster/orm/migrations/0007_merge.py | 15 +++
>>> bitbake/lib/toaster/orm/models.py | 12 +-
>>> bitbake/lib/toaster/toastergui/api.py | 110
>>>+++++++++++++++
>>> .../toaster/toastergui/static/js/customrecipe.js | 4 +-
>>> .../lib/toaster/toastergui/static/js/layerBtn.js | 3 +-
>>> .../lib/toaster/toastergui/static/js/libtoaster.js | 41 +++---
>>> .../lib/toaster/toastergui/static/js/mrbsection.js | 95 +++++++++++++
>>> .../toaster/toastergui/static/js/projectpage.js | 4 +-
>>> .../toaster/toastergui/static/js/projecttopbar.js | 6 +-
>>> .../toaster/toastergui/static/js/recipedetails.js | 4 +-
>>> bitbake/lib/toaster/toastergui/tables.py | 48 +------
>>> bitbake/lib/toaster/toastergui/templates/base.html | 1 +
>>> .../toaster/toastergui/templates/mrb_section.html | 148
>>>++++++++-------------
>>> bitbake/lib/toaster/toastergui/urls.py | 5 +
>>> 22 files changed, 507 insertions(+), 228 deletions(-)
>>> create mode 100644
>>>bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py
>>> create mode 100644
>>>bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py
>>> create mode 100644
>>>bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py
>>> create mode 100644 bitbake/lib/toaster/orm/migrations/0007_merge.py
>>> create mode 100644 bitbake/lib/toaster/toastergui/api.py
>>> create mode 100644
>>>bitbake/lib/toaster/toastergui/static/js/mrbsection.js
>>>
>>> --
>>> Regards,
>>> Ed
>>> --
>>> _______________________________________________
>>> toaster mailing list
>>> toaster@yoctoproject.org
>>> https://lists.yoctoproject.org/listinfo/toaster
>>--
>>_______________________________________________
>>toaster mailing list
>>toaster@yoctoproject.org
>>https://lists.yoctoproject.org/listinfo/toaster
>
-brian
an intel employee
^ permalink raw reply [flat|nested] 27+ messages in thread