* [review-request][PATCH 1/3] toaster: Always run bldcontrol migrations
2015-09-22 16:41 [review-request][PATCH 0/3] Fix error thrown by "command line builds" project page Elliot Smith
@ 2015-09-22 16:41 ` Elliot Smith
2015-09-22 16:41 ` [review-request][PATCH 2/3] toaster: Check whether buildrequest exists before using it Elliot Smith
` (2 subsequent siblings)
3 siblings, 0 replies; 9+ messages in thread
From: Elliot Smith @ 2015-09-22 16:41 UTC (permalink / raw)
To: toaster
The toaster startup script conditionally migrates the database
tables depending on whether you are in managed mode or not. This
means that if you are in analysis mode, some of the bldcontrol*
database tables used by managed mode are not available.
As a consequence, some of the code in toaster which refers to
those tables can break in analysis mode, as there's no clean
isolation of the two modes.
To prevent this from happening, always run the migrations for
managed mode and create the bldcontrol* tables, even if in
analysis mode.
Also clean up the function which starts up toaster so the
logic is easier to follow.
[YOCTO #8277]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
---
bitbake/bin/toaster | 58 +++++++++++++++++++++++++++++++++++------------------
1 file changed, 39 insertions(+), 19 deletions(-)
diff --git a/bitbake/bin/toaster b/bitbake/bin/toaster
index 411ce2c..1417d47 100755
--- a/bitbake/bin/toaster
+++ b/bitbake/bin/toaster
@@ -54,35 +54,55 @@ webserverStartAll()
fi
retval=0
- if [ "$TOASTER_MANAGED" '=' '1' ]; then
+ if [ "$TOASTER_MANAGED" = '1' ]; then
python $BBBASEDIR/lib/toaster/manage.py syncdb || retval=1
else
python $BBBASEDIR/lib/toaster/manage.py syncdb --noinput || retval=1
fi
- python $BBBASEDIR/lib/toaster/manage.py migrate orm || retval=2
+
if [ $retval -eq 1 ]; then
- echo "Failed db sync, stopping system start" 1>&2
- elif [ $retval -eq 2 ]; then
- printf "\nError on migration, trying to recover... \n"
+ echo "Failed db sync, aborting system start" 1>&2
+ return $retval
+ fi
+
+ python $BBBASEDIR/lib/toaster/manage.py migrate orm || retval=1
+
+ if [ $retval -eq 1 ]; then
+ printf "\nError on orm migration, rolling back...\n"
python $BBBASEDIR/lib/toaster/manage.py migrate orm 0001_initial --fake
- retval=0
- python $BBBASEDIR/lib/toaster/manage.py migrate orm || retval=1
+ return $retval
fi
+
+ python $BBBASEDIR/lib/toaster/manage.py migrate bldcontrol || retval=1
+
+ if [ $retval -eq 1 ]; then
+ printf "\nError on bldcontrol migration, rolling back...\n"
+ python $BBBASEDIR/lib/toaster/manage.py migrate bldcontrol 0001_initial --fake
+ return $retval
+ fi
+
if [ "$TOASTER_MANAGED" = '1' ]; then
- python $BBBASEDIR/lib/toaster/manage.py migrate bldcontrol || retval=1
- python $BBBASEDIR/lib/toaster/manage.py checksettings --traceback || retval=1
+ python $BBBASEDIR/lib/toaster/manage.py checksettings --traceback || retval=1
fi
- if [ $retval -eq 0 ]; then
- echo "Starting webserver..."
- python $BBBASEDIR/lib/toaster/manage.py runserver "0.0.0.0:$WEB_PORT" </dev/null >>${BUILDDIR}/toaster_web.log 2>&1 & echo $! >${BUILDDIR}/.toastermain.pid
- sleep 1
- if ! cat "${BUILDDIR}/.toastermain.pid" | xargs -I{} kill -0 {} ; then
- retval=1
- rm "${BUILDDIR}/.toastermain.pid"
- else
- echo "Webserver address: http://0.0.0.0:$WEB_PORT/"
- fi
+
+ if [ $retval -eq 1 ]; then
+ printf "\nError while checking settings; aborting\n"
+ return $retval
+ fi
+
+ echo "Starting webserver..."
+
+ python $BBBASEDIR/lib/toaster/manage.py runserver "0.0.0.0:$WEB_PORT" </dev/null >>${BUILDDIR}/toaster_web.log 2>&1 & echo $! >${BUILDDIR}/.toastermain.pid
+
+ sleep 1
+
+ if ! cat "${BUILDDIR}/.toastermain.pid" | xargs -I{} kill -0 {} ; then
+ retval=1
+ rm "${BUILDDIR}/.toastermain.pid"
+ else
+ echo "Webserver address: http://0.0.0.0:$WEB_PORT/"
fi
+
return $retval
}
--
Elliot Smith
Software Engineer
Intel OTC
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
^ permalink raw reply related [flat|nested] 9+ messages in thread* [review-request][PATCH 2/3] toaster: Check whether buildrequest exists before using it
2015-09-22 16:41 [review-request][PATCH 0/3] Fix error thrown by "command line builds" project page Elliot Smith
2015-09-22 16:41 ` [review-request][PATCH 1/3] toaster: Always run bldcontrol migrations Elliot Smith
@ 2015-09-22 16:41 ` Elliot Smith
2015-09-22 16:41 ` [review-request][PATCH 3/3] toaster: Test that exception isn't thrown by project page Elliot Smith
2015-09-23 3:26 ` [review-request][PATCH 0/3] Fix error thrown by "command line builds" " Brian Avery
3 siblings, 0 replies; 9+ messages in thread
From: Elliot Smith @ 2015-09-22 16:41 UTC (permalink / raw)
To: toaster
Builds initiated from the command line don't have a buildrequest
associated with them. The build.buildrequest association is
only added if a build is triggered from toaster.
Some of the code for displaying the status of a build refers
to build.buildrequest without checking whether it has been set,
which causes an error to be thrown.
Add a guard to check whether the buildrequest has been set.
[YOCTO #8277]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
---
bitbake/lib/toaster/orm/models.py | 20 ++++++++++++++++++--
1 file changed, 18 insertions(+), 2 deletions(-)
diff --git a/bitbake/lib/toaster/orm/models.py b/bitbake/lib/toaster/orm/models.py
index e4d2e87..88591bc 100644
--- a/bitbake/lib/toaster/orm/models.py
+++ b/bitbake/lib/toaster/orm/models.py
@@ -333,6 +333,9 @@ class Build(models.Model):
tgts = Target.objects.filter(build_id = self.id).order_by( 'target' );
return( tgts );
+ def get_outcome_text(self):
+ return Build.BUILD_OUTCOME[int(self.outcome)][1]
+
@property
def toaster_exceptions(self):
return self.logmessage_set.filter(level=LogMessage.EXCEPTION)
@@ -350,10 +353,23 @@ class Build(models.Model):
return (self.completed_on - self.started_on).total_seconds()
def get_current_status(self):
+ """
+ get the status string from the build request if the build
+ has one, or the text for the build outcome if it doesn't
+ """
+
from bldcontrol.models import BuildRequest
- if self.outcome == Build.IN_PROGRESS and self.buildrequest.state != BuildRequest.REQ_INPROGRESS:
+
+ build_request = None
+ if hasattr(self, 'buildrequest'):
+ build_request = self.buildrequest
+
+ if (build_request
+ and build_request.state != BuildRequest.REQ_INPROGRESS
+ and self.outcome == Build.IN_PROGRESS):
return self.buildrequest.get_state_display()
- return self.get_outcome_display()
+ else:
+ return self.get_outcome_text()
def __str__(self):
return "%d %s %s" % (self.id, self.project, ",".join([t.target for t in self.target_set.all()]))
--
Elliot Smith
Software Engineer
Intel OTC
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
^ permalink raw reply related [flat|nested] 9+ messages in thread* [review-request][PATCH 3/3] toaster: Test that exception isn't thrown by project page
2015-09-22 16:41 [review-request][PATCH 0/3] Fix error thrown by "command line builds" project page Elliot Smith
2015-09-22 16:41 ` [review-request][PATCH 1/3] toaster: Always run bldcontrol migrations Elliot Smith
2015-09-22 16:41 ` [review-request][PATCH 2/3] toaster: Check whether buildrequest exists before using it Elliot Smith
@ 2015-09-22 16:41 ` Elliot Smith
2015-10-14 14:10 ` Michael Wood
2015-09-23 3:26 ` [review-request][PATCH 0/3] Fix error thrown by "command line builds" " Brian Avery
3 siblings, 1 reply; 9+ messages in thread
From: Elliot Smith @ 2015-09-22 16:41 UTC (permalink / raw)
To: toaster
Add a test which checks that an exception is no longer thrown
for the /toastergui/project/X page for the default project.
Note that we still get a spinning dialogue box on this page
because the default project has no configuration to display,
but at least it doesn't fail altogether.
[YOCTO #8277]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
---
bitbake/lib/toaster/toastergui/tests.py | 33 ++++++++++++++++++++++++++++++++-
1 file changed, 32 insertions(+), 1 deletion(-)
diff --git a/bitbake/lib/toaster/toastergui/tests.py b/bitbake/lib/toaster/toastergui/tests.py
index 4d1549b..437a45b 100644
--- a/bitbake/lib/toaster/toastergui/tests.py
+++ b/bitbake/lib/toaster/toastergui/tests.py
@@ -376,4 +376,35 @@ class ProjectBuildsDisplayTest(TestCase):
build2b = Build.objects.create(**self.project2_build_in_progress)
build_rows = self._get_rows_for_project(self.project1.id)
- self.assertEqual(len(build_rows), 2)
\ No newline at end of file
+ self.assertEqual(len(build_rows), 2)
+
+class ProjectPageTests(TestCase):
+ """ Test project data at /project/X/ is displayed correctly """
+
+ PROJECT_NAME = 'Command line builds'
+
+ def test_command_line_builds_in_progress(self):
+ """
+ In progress builds should not cause an error to be thrown
+ when navigating to "command line builds" project page;
+ see https://bugzilla.yoctoproject.org/show_bug.cgi?id=8277
+ """
+
+ # add the "command line builds" default project; this mirrors what
+ # we do in migration 0026_set_default_project.py
+ default_project = Project.objects.create_project(self.PROJECT_NAME, None)
+ default_project.is_default = True
+ default_project.save()
+
+ # add an "in progress" build for the default project
+ now = timezone.now()
+ build = Build.objects.create(project=default_project,
+ started_on=now,
+ completed_on=now,
+ outcome=Build.IN_PROGRESS)
+
+ # navigate to the project page for the default project
+ url = reverse("project", args=(default_project.id,))
+ response = self.client.get(url, follow=True)
+
+ self.assertEqual(response.status_code, 200)
--
Elliot Smith
Software Engineer
Intel OTC
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [review-request][PATCH 3/3] toaster: Test that exception isn't thrown by project page
2015-09-22 16:41 ` [review-request][PATCH 3/3] toaster: Test that exception isn't thrown by project page Elliot Smith
@ 2015-10-14 14:10 ` Michael Wood
0 siblings, 0 replies; 9+ messages in thread
From: Michael Wood @ 2015-10-14 14:10 UTC (permalink / raw)
To: toaster
Thanks - Now pushed to toaster-next and upstream
On 22/09/15 17:41, Elliot Smith wrote:
> Add a test which checks that an exception is no longer thrown
> for the /toastergui/project/X page for the default project.
>
> Note that we still get a spinning dialogue box on this page
> because the default project has no configuration to display,
> but at least it doesn't fail altogether.
>
> [YOCTO #8277]
>
> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
> ---
> bitbake/lib/toaster/toastergui/tests.py | 33 ++++++++++++++++++++++++++++++++-
> 1 file changed, 32 insertions(+), 1 deletion(-)
>
> diff --git a/bitbake/lib/toaster/toastergui/tests.py b/bitbake/lib/toaster/toastergui/tests.py
> index 4d1549b..437a45b 100644
> --- a/bitbake/lib/toaster/toastergui/tests.py
> +++ b/bitbake/lib/toaster/toastergui/tests.py
> @@ -376,4 +376,35 @@ class ProjectBuildsDisplayTest(TestCase):
> build2b = Build.objects.create(**self.project2_build_in_progress)
>
> build_rows = self._get_rows_for_project(self.project1.id)
> - self.assertEqual(len(build_rows), 2)
> \ No newline at end of file
> + self.assertEqual(len(build_rows), 2)
> +
> +class ProjectPageTests(TestCase):
> + """ Test project data at /project/X/ is displayed correctly """
> +
> + PROJECT_NAME = 'Command line builds'
> +
> + def test_command_line_builds_in_progress(self):
> + """
> + In progress builds should not cause an error to be thrown
> + when navigating to "command line builds" project page;
> + see https://bugzilla.yoctoproject.org/show_bug.cgi?id=8277
> + """
> +
> + # add the "command line builds" default project; this mirrors what
> + # we do in migration 0026_set_default_project.py
> + default_project = Project.objects.create_project(self.PROJECT_NAME, None)
> + default_project.is_default = True
> + default_project.save()
> +
> + # add an "in progress" build for the default project
> + now = timezone.now()
> + build = Build.objects.create(project=default_project,
> + started_on=now,
> + completed_on=now,
> + outcome=Build.IN_PROGRESS)
> +
> + # navigate to the project page for the default project
> + url = reverse("project", args=(default_project.id,))
> + response = self.client.get(url, follow=True)
> +
> + self.assertEqual(response.status_code, 200)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [review-request][PATCH 0/3] Fix error thrown by "command line builds" project page
2015-09-22 16:41 [review-request][PATCH 0/3] Fix error thrown by "command line builds" project page Elliot Smith
` (2 preceding siblings ...)
2015-09-22 16:41 ` [review-request][PATCH 3/3] toaster: Test that exception isn't thrown by project page Elliot Smith
@ 2015-09-23 3:26 ` Brian Avery
2015-09-23 7:02 ` Smith, Elliot
3 siblings, 1 reply; 9+ messages in thread
From: Brian Avery @ 2015-09-23 3:26 UTC (permalink / raw)
To: Elliot Smith; +Cc: toaster
[-- Attachment #1: Type: text/plain, Size: 2430 bytes --]
Hi,
Could we add the attached patch to yours? It
1) gets rid of the annoying do you want to make a superuser now (you
can do it at anytime with :
python bitbake/lib/toaster/manage.py python manage.py createsuperuser
--username=ME
2) actually applies the value of WEB_PORT if we start in analysis mode.
if there's a preferred way for me to add a suggestion on top of your
patch, lemme know :)
-b
On Tue, Sep 22, 2015 at 9:41 AM, Elliot Smith <elliot.smith@intel.com> wrote:
> This patch series fixes an exception being thrown when viewing
> the default "command line builds" project while a bitbake command line
> build is running.
>
> It also modifies and cleans up the toaster startup script so that
> the bldcontrol migrations are applied in analysis mode, as well as
> in managed mode. This should help prevent other errors which may
> occur in analysis mode due to missing database tables.
>
> Changes since 7b86c771c80d0759c2ca0e57c46c4c966f89c49e are in
> git://git.yoctoproject.org/poky-contrib, elliot/toaster/cli_builds_error-8277
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/toaster/cli_builds_error-8277
>
> Related bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=8277
>
> Elliot Smith (3):
> toaster: Always run bldcontrol migrations
> toaster: Check whether buildrequest exists before using it
> toaster: Test that exception isn't thrown by project page
>
> bitbake/bin/toaster | 58 ++++++++++++++++++++++-----------
> bitbake/lib/toaster/orm/models.py | 20 ++++++++++--
> bitbake/lib/toaster/toastergui/tests.py | 33 ++++++++++++++++++-
> 3 files changed, 89 insertions(+), 22 deletions(-)
>
> --
> Elliot Smith
> Software Engineer
> Intel OTC
>
> ---------------------------------------------------------------------
> Intel Corporation (UK) Limited
> Registered No. 1134945 (England)
> Registered Office: Pipers Way, Swindon SN3 1RJ
> VAT No: 860 2173 47
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
> --
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
[-- Attachment #2: toaster.patch --]
[-- Type: application/octet-stream, Size: 761 bytes --]
diff --git a/bitbake/bin/toaster b/bitbake/bin/toaster
index 1417d47..e2aa610 100755
--- a/bitbake/bin/toaster
+++ b/bitbake/bin/toaster
@@ -54,11 +54,7 @@ webserverStartAll()
fi
retval=0
- if [ "$TOASTER_MANAGED" = '1' ]; then
- python $BBBASEDIR/lib/toaster/manage.py syncdb || retval=1
- else
- python $BBBASEDIR/lib/toaster/manage.py syncdb --noinput || retval=1
- fi
+ python $BBBASEDIR/lib/toaster/manage.py syncdb --noinput || retval=1
if [ $retval -eq 1 ]; then
echo "Failed db sync, aborting system start" 1>&2
@@ -183,7 +179,10 @@ RUNNING=0
NOTOASTERUI=0
WEBSERVER=1
TOASTER_BRBE=""
-WEB_PORT="8000"
+if [ "WEB_PORT" == "" ]; then
+ WEB_PORT="8000"
+fi
+
NOBROWSER=0
for param in $*; do
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [review-request][PATCH 0/3] Fix error thrown by "command line builds" project page
2015-09-23 3:26 ` [review-request][PATCH 0/3] Fix error thrown by "command line builds" " Brian Avery
@ 2015-09-23 7:02 ` Smith, Elliot
2015-09-23 13:58 ` Brian Avery
0 siblings, 1 reply; 9+ messages in thread
From: Smith, Elliot @ 2015-09-23 7:02 UTC (permalink / raw)
To: Brian Avery; +Cc: toaster
[-- Attachment #1: Type: text/plain, Size: 757 bytes --]
On 23 September 2015 at 04:26, Brian Avery <avery.brian@gmail.com> wrote:
> Could we add the attached patch to yours? It
> 1) gets rid of the annoying do you want to make a superuser now (you
> can do it at anytime with :
>
> python bitbake/lib/toaster/manage.py python manage.py createsuperuser
> --username=ME
> 2) actually applies the value of WEB_PORT if we start in analysis mode.
> if there's a preferred way for me to add a suggestion on top of your
> patch, lemme know :)
>
My preference would be to make a separate branch and review request for
your patch (it has a clear and separate scope from my branch).
I'm happy to do that if you like.
Elliot
--
Elliot Smith
Software Engineer
Intel Open Source Technology Centre
[-- Attachment #2: Type: text/html, Size: 1206 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [review-request][PATCH 0/3] Fix error thrown by "command line builds" project page
2015-09-23 7:02 ` Smith, Elliot
@ 2015-09-23 13:58 ` Brian Avery
0 siblings, 0 replies; 9+ messages in thread
From: Brian Avery @ 2015-09-23 13:58 UTC (permalink / raw)
To: Smith, Elliot; +Cc: toaster
That sounds like a better approach. I 'll do it today since I already
have the patch :)
-b
On Wed, Sep 23, 2015 at 12:02 AM, Smith, Elliot <elliot.smith@intel.com> wrote:
> On 23 September 2015 at 04:26, Brian Avery <avery.brian@gmail.com> wrote:
>>
>> Could we add the attached patch to yours? It
>> 1) gets rid of the annoying do you want to make a superuser now (you
>> can do it at anytime with :
>>
>> python bitbake/lib/toaster/manage.py python manage.py createsuperuser
>> --username=ME
>> 2) actually applies the value of WEB_PORT if we start in analysis mode.
>> if there's a preferred way for me to add a suggestion on top of your
>> patch, lemme know :)
>
>
> My preference would be to make a separate branch and review request for your
> patch (it has a clear and separate scope from my branch).
>
> I'm happy to do that if you like.
>
> Elliot
> --
> Elliot Smith
> Software Engineer
> Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 9+ messages in thread