[PATCH 0/3] toaster: Fix error thrown by command line builds project page

[PATCH 0/3] toaster: Fix error thrown by command line builds project page

[Michael Wood]

This patch series fixes an exception being thrown when viewing
the default "command line builds" project while a bitbake command line
build is running.

It also modifies and cleans up the toaster startup script so that
the bldcontrol migrations are applied in analysis mode, as well as
in managed mode. This should help prevent other errors which may
occur in analysis mode due to missing database tables.


Available in the git repository at:

  git://git.yoctoproject.org/poky-contrib michaelw/submit/elliot/toaster/cli_builds_error-8277
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=michaelw/submit/elliot/toaster/cli_builds_error-8277

Elliot Smith (3):
  toaster: Always run bldcontrol migrations
  toaster: Check whether buildrequest exists before using it
  toaster: Test that exception isn't thrown by project page

 bin/toaster                     | 56 ++++++++++++++++++++++++++++-------------
 lib/toaster/orm/models.py       | 20 +++++++++++++--
 lib/toaster/toastergui/tests.py | 30 ++++++++++++++++++++++
 3 files changed, 87 insertions(+), 19 deletions(-)

-- 
2.1.4

[PATCH 1/3] toaster: Always run bldcontrol migrations

[Michael Wood]

From: Elliot Smith <elliot.smith@intel.com>

The toaster startup script conditionally migrates the database
tables depending on whether you are in managed mode or not. This
means that if you are in analysis mode, some of the bldcontrol*
database tables used by managed mode are not available.

As a consequence, some of the code in toaster which refers to
those tables can break in analysis mode, as there's no clean
isolation of the two modes.

To prevent this from happening, always run the migrations for
managed mode and create the bldcontrol* tables, even if in
analysis mode.

Also clean up the function which starts up toaster so the
logic is easier to follow.

[YOCTO #8277]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
---
 bin/toaster | 56 +++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 39 insertions(+), 17 deletions(-)

diff --git a/bin/toaster b/bin/toaster
index bc439e6..e976604 100755
--- a/bin/toaster
+++ b/bin/toaster
@@ -59,29 +59,50 @@ webserverStartAll()
     python $BBBASEDIR/lib/toaster/manage.py syncdb --noinput || retval=1
 
     python $BBBASEDIR/lib/toaster/manage.py migrate orm || retval=2
+
     if [ $retval -eq 1 ]; then
-        echo "Failed db sync, stopping system start" 1>&2
-    elif [ $retval -eq 2 ]; then
-        printf "\nError on migration, trying to recover... \n"
+        echo "Failed db sync, aborting system start" 1>&2
+        return $retval
+    fi
+
+    python $BBBASEDIR/lib/toaster/manage.py migrate orm || retval=1
+
+    if [ $retval -eq 1 ]; then
+        printf "\nError on orm migration, rolling back...\n"
         python $BBBASEDIR/lib/toaster/manage.py migrate orm 0001_initial --fake
-        retval=0
-        python $BBBASEDIR/lib/toaster/manage.py migrate orm || retval=1
+        return $retval
     fi
+
+    python $BBBASEDIR/lib/toaster/manage.py migrate bldcontrol || retval=1
+
+    if [ $retval -eq 1 ]; then
+        printf "\nError on bldcontrol migration, rolling back...\n"
+        python $BBBASEDIR/lib/toaster/manage.py migrate bldcontrol 0001_initial --fake
+        return $retval
+    fi
+
     if [ "$TOASTER_MANAGED" = '1' ]; then
-        python $BBBASEDIR/lib/toaster/manage.py migrate bldcontrol || retval=1
-        python $BBBASEDIR/lib/toaster/manage.py checksettings  --traceback || retval=1
+        python $BBBASEDIR/lib/toaster/manage.py checksettings --traceback || retval=1
     fi
-    if [ $retval -eq 0 ]; then
-        echo "Starting webserver..."
-        python $BBBASEDIR/lib/toaster/manage.py runserver "0.0.0.0:$WEB_PORT" </dev/null >>${BUILDDIR}/toaster_web.log 2>&1 & echo $! >${BUILDDIR}/.toastermain.pid
-        sleep 1
-        if ! cat "${BUILDDIR}/.toastermain.pid" | xargs -I{} kill -0 {} ; then
-            retval=1
-            rm "${BUILDDIR}/.toastermain.pid"
-        else
-            echo "Webserver address:  http://0.0.0.0:$WEB_PORT/"
-        fi
+
+    if [ $retval -eq 1 ]; then
+        printf "\nError while checking settings; aborting\n"
+        return $retval
+    fi
+
+    echo "Starting webserver..."
+
+    python $BBBASEDIR/lib/toaster/manage.py runserver "0.0.0.0:$WEB_PORT" </dev/null >>${BUILDDIR}/toaster_web.log 2>&1 & echo $! >${BUILDDIR}/.toastermain.pid
+
+    sleep 1
+
+    if ! cat "${BUILDDIR}/.toastermain.pid" | xargs -I{} kill -0 {} ; then
+        retval=1
+        rm "${BUILDDIR}/.toastermain.pid"
+    else
+        echo "Webserver address:  http://0.0.0.0:$WEB_PORT/"
     fi
+
     return $retval
 }
 
@@ -375,3 +396,4 @@ case $CMD in
         echo "Successful ${CMD}."
     ;;
 esac
+
-- 
2.1.4

[PATCH 2/3] toaster: Check whether buildrequest exists before using it

[Michael Wood]

From: Elliot Smith <elliot.smith@intel.com>

Builds initiated from the command line don't have a buildrequest
associated with them. The build.buildrequest association is
only added if a build is triggered from toaster.

Some of the code for displaying the status of a build refers
to build.buildrequest without checking whether it has been set,
which causes an error to be thrown.

Add a guard to check whether the buildrequest has been set.

[YOCTO #8277]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
---
 lib/toaster/orm/models.py | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index 1cbf480..44a453a 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -344,6 +344,9 @@ class Build(models.Model):
         tgts = Target.objects.filter(build_id = self.id).order_by( 'target' );
         return( tgts );
 
+    def get_outcome_text(self):
+        return Build.BUILD_OUTCOME[int(self.outcome)][1]
+
     @property
     def toaster_exceptions(self):
         return self.logmessage_set.filter(level=LogMessage.EXCEPTION)
@@ -361,10 +364,23 @@ class Build(models.Model):
         return (self.completed_on - self.started_on).total_seconds()
 
     def get_current_status(self):
+        """
+        get the status string from the build request if the build
+        has one, or the text for the build outcome if it doesn't
+        """
+
         from bldcontrol.models import BuildRequest
-        if self.outcome == Build.IN_PROGRESS and self.buildrequest.state != BuildRequest.REQ_INPROGRESS:
+
+        build_request = None
+        if hasattr(self, 'buildrequest'):
+            build_request = self.buildrequest
+
+        if (build_request
+                and build_request.state != BuildRequest.REQ_INPROGRESS
+                and self.outcome == Build.IN_PROGRESS):
             return self.buildrequest.get_state_display()
-        return self.get_outcome_display()
+        else:
+            return self.get_outcome_text()
 
     def __str__(self):
         return "%d %s %s" % (self.id, self.project, ",".join([t.target for t in self.target_set.all()]))
-- 
2.1.4

[PATCH 3/3] toaster: Test that exception isn't thrown by project page

[Michael Wood]

From: Elliot Smith <elliot.smith@intel.com>

Add a test which checks that an exception is no longer thrown
for the /toastergui/project/X page for the default project.

Note that we still get a spinning dialogue box on this page
because the default project has no configuration to display,
but at least it doesn't fail altogether.

[YOCTO #8277]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
---
 lib/toaster/toastergui/tests.py | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/lib/toaster/toastergui/tests.py b/lib/toaster/toastergui/tests.py
index 95790a2..e652b98 100644
--- a/lib/toaster/toastergui/tests.py
+++ b/lib/toaster/toastergui/tests.py
@@ -574,3 +574,33 @@ class ProjectBuildsDisplayTest(TestCase):
         response = self.client.get(url, follow=True)
         result = re.findall('bash:clean', response.content, re.MULTILINE)
         self.assertEqual(len(result), 3)
+
+class ProjectPageTests(TestCase):
+    """ Test project data at /project/X/ is displayed correctly """
+    CLI_BUILDS_PROJECT_NAME = 'Command line builds'
+
+    def test_command_line_builds_in_progress(self):
+        """
+        In progress builds should not cause an error to be thrown
+        when navigating to "command line builds" project page;
+        see https://bugzilla.yoctoproject.org/show_bug.cgi?id=8277
+        """
+
+        # add the "command line builds" default project; this mirrors what
+        # we do in migration 0026_set_default_project.py
+        default_project = Project.objects.create_project(self.CLI_BUILDS_PROJECT_NAME, None)
+        default_project.is_default = True
+        default_project.save()
+
+        # add an "in progress" build for the default project
+        now = timezone.now()
+        build = Build.objects.create(project=default_project,
+                                     started_on=now,
+                                     completed_on=now,
+                                     outcome=Build.IN_PROGRESS)
+
+        # navigate to the project page for the default project
+        url = reverse("project", args=(default_project.id,))
+        response = self.client.get(url, follow=True)
+
+        self.assertEqual(response.status_code, 200)
-- 
2.1.4