[PATCH 0/4] toaster:cummulative_03272020

[PATCH 0/4] toaster:cummulative_03272020

[Reyna, David]

This cummulative patch set for Toaster is based after the previously submitted "Dango-2.2 update"
patch (though there are no direct overlaps):
  * Bugzilla #13847 Toaster needs update to Dunfell
  * Bugzilla #13170 Support environment-safe manage commands
  * Bugzilla #13717 FieldError when trying to load Reverse runtime dependencies
  * Bugzilla #13707 toaster: import build directory fails if BBLAYERS contains variable reference

The following changes since commit 127ebac4a40ff63f493afbb3629f914a0b92d22b:

  toaster: migrate to Django-2.2 (2020-03-27 15:39:24 -0700)

are available in the Git repository at:

  git://git.yoctoproject.org/poky-contrib dreyna/submit/dreyna/toaster/toaster_cummulative_03272020
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=dreyna/submit/dreyna/toaster/toaster_cummulative_03272020

David Reyna (3):
  toaster: update to Dunfell
  toaster: support environment-safe manage commands
  toaster: FieldError loading Reverse runtime dependencies

Paul Eggleton (1):
  toaster: fix for import build directory with var refs in BBLAYERS

 bin/toaster                                   | 19 ++++++++++++++++---
 lib/toaster/orm/fixtures/oe-core.xml          | 12 ++++++------
 lib/toaster/orm/fixtures/poky.xml             | 18 +++++++++---------
 lib/toaster/toastergui/views.py               |  2 +-
 .../management/commands/buildimport.py        | 12 +++++++++++-
 5 files changed, 43 insertions(+), 20 deletions(-)

-- 
2.20.1

[PATCH 1/4] toaster: update to Dunfell

[Reyna, David]

Toaster needs update to Dunfell

[YOCTO #13847]

Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
 lib/toaster/orm/fixtures/oe-core.xml | 12 ++++++------
 lib/toaster/orm/fixtures/poky.xml    | 18 +++++++++---------
 2 files changed, 15 insertions(+), 15 deletions(-)

diff --git a/lib/toaster/orm/fixtures/oe-core.xml b/lib/toaster/orm/fixtures/oe-core.xml
index a723f5a2a7..fd93f4d87b 100644
--- a/lib/toaster/orm/fixtures/oe-core.xml
+++ b/lib/toaster/orm/fixtures/oe-core.xml
@@ -8,9 +8,9 @@
 
   <!-- Bitbake versions which correspond to the metadata release -->
   <object model="orm.bitbakeversion" pk="1">
-    <field type="CharField" name="name">warrior</field>
+    <field type="CharField" name="name">dunfell</field>
     <field type="CharField" name="giturl">git://git.openembedded.org/bitbake</field>
-    <field type="CharField" name="branch">1.42</field>
+    <field type="CharField" name="branch">1.46</field>
   </object>
   <object model="orm.bitbakeversion" pk="2">
     <field type="CharField" name="name">HEAD</field>
@@ -30,11 +30,11 @@
 
   <!-- Releases available -->
   <object model="orm.release" pk="1">
-    <field type="CharField" name="name">warrior</field>
-    <field type="CharField" name="description">Openembedded Warrior</field>
+    <field type="CharField" name="name">dunfell</field>
+    <field type="CharField" name="description">Openembedded Dunfell</field>
     <field rel="ManyToOneRel" to="orm.bitbakeversion" name="bitbake_version">1</field>
-    <field type="CharField" name="branch_name">warrior</field>
-    <field type="TextField" name="helptext">Toaster will run your builds using the tip of the &lt;a href=\"http://cgit.openembedded.org/openembedded-core/log/?h=warrior\"&gt;OpenEmbedded Warrior&lt;/a&gt; branch.</field>
+    <field type="CharField" name="branch_name">dunfell</field>
+    <field type="TextField" name="helptext">Toaster will run your builds using the tip of the &lt;a href=\"http://cgit.openembedded.org/openembedded-core/log/?h=dunfell\"&gt;OpenEmbedded Dunfell&lt;/a&gt; branch.</field>
   </object>
   <object model="orm.release" pk="2">
     <field type="CharField" name="name">local</field>
diff --git a/lib/toaster/orm/fixtures/poky.xml b/lib/toaster/orm/fixtures/poky.xml
index 79923837f6..902bc88a50 100644
--- a/lib/toaster/orm/fixtures/poky.xml
+++ b/lib/toaster/orm/fixtures/poky.xml
@@ -8,9 +8,9 @@
 
   <!-- Bitbake versions which correspond to the metadata release -->
   <object model="orm.bitbakeversion" pk="1">
-    <field type="CharField" name="name">warrior</field>
+    <field type="CharField" name="name">dunfell</field>
     <field type="CharField" name="giturl">git://git.yoctoproject.org/poky</field>
-    <field type="CharField" name="branch">warrior</field>
+    <field type="CharField" name="branch">dunfell</field>
     <field type="CharField" name="dirpath">bitbake</field>
   </object>
   <object model="orm.bitbakeversion" pk="2">
@@ -35,11 +35,11 @@
 
   <!-- Releases available -->
   <object model="orm.release" pk="1">
-    <field type="CharField" name="name">warrior</field>
-    <field type="CharField" name="description">Yocto Project 2.7 "Warrior"</field>
+    <field type="CharField" name="name">dunfell</field>
+    <field type="CharField" name="description">Yocto Project 3.1 "Dunfell"</field>
     <field rel="ManyToOneRel" to="orm.bitbakeversion" name="bitbake_version">1</field>
-    <field type="CharField" name="branch_name">warrior</field>
-    <field type="TextField" name="helptext">Toaster will run your builds using the tip of the &lt;a href="http://git.yoctoproject.org/cgit/cgit.cgi/poky/log/?h=warrior"&gt;Yocto Project Warrior branch&lt;/a&gt;.</field>
+    <field type="CharField" name="branch_name">dunfell</field>
+    <field type="TextField" name="helptext">Toaster will run your builds using the tip of the &lt;a href="http://git.yoctoproject.org/cgit/cgit.cgi/poky/log/?h=dunfell"&gt;Yocto Project Dunfell branch&lt;/a&gt;.</field>
   </object>
   <object model="orm.release" pk="2">
     <field type="CharField" name="name">local</field>
@@ -130,7 +130,7 @@
     <field rel="ManyToOneRel" to="orm.layer" name="layer">1</field>
     <field type="IntegerField" name="layer_source">0</field>
     <field rel="ManyToOneRel" to="orm.release" name="release">1</field>
-    <field type="CharField" name="branch">warrior</field>
+    <field type="CharField" name="branch">dunfell</field>
     <field type="CharField" name="dirpath">meta</field>
   </object>
   <object model="orm.layer_version" pk="2">
@@ -168,7 +168,7 @@
     <field rel="ManyToOneRel" to="orm.layer" name="layer">2</field>
     <field type="IntegerField" name="layer_source">0</field>
     <field rel="ManyToOneRel" to="orm.release" name="release">1</field>
-    <field type="CharField" name="branch">warrior</field>
+    <field type="CharField" name="branch">dunfell</field>
     <field type="CharField" name="dirpath">meta-poky</field>
   </object>
   <object model="orm.layer_version" pk="6">
@@ -206,7 +206,7 @@
     <field rel="ManyToOneRel" to="orm.layer" name="layer">3</field>
     <field type="IntegerField" name="layer_source">0</field>
     <field rel="ManyToOneRel" to="orm.release" name="release">1</field>
-    <field type="CharField" name="branch">warrior</field>
+    <field type="CharField" name="branch">dunfell</field>
     <field type="CharField" name="dirpath">meta-yocto-bsp</field>
   </object>
   <object model="orm.layer_version" pk="10">
-- 
2.20.1

[PATCH 2/4] toaster: support environment-safe manage commands

[Reyna, David]

Directly support the various 'manage' commands from the Toaster
executable, so that users do not have to manually set up the required
environment and paths.

Examples:
  $ . toaster manage createsuperuser
  $ . toaster manage lsupdates

[YOCTO #13170]

Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
 bin/toaster | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/bin/toaster b/bin/toaster
index c3472dfee8..6b90ee187e 100755
--- a/bin/toaster
+++ b/bin/toaster
@@ -8,12 +8,13 @@
 #
 
 HELP="
-Usage: source toaster start|stop [webport=<address:port>] [noweb] [nobuild] [toasterdir]
+Usage 1: source toaster start|stop [webport=<address:port>] [noweb] [nobuild] [toasterdir]
     Optional arguments:
         [nobuild] Setup the environment for capturing builds with toaster but disable managed builds
         [noweb] Setup the environment for capturing builds with toaster but don't start the web server
         [webport] Set the development server (default: localhost:8000)
         [toasterdir] Set absolute path to be used as TOASTER_DIR (default: BUILDDIR/../)
+Usage 2: source toaster manage [createsuperuser|lsupdates|migrate|makemigrations|checksettings|collectstatic|...]
 "
 
 custom_extention()
@@ -208,13 +209,21 @@ for param in $*; do
     toasterdir=*)
             TOASTERDIR="${param#*=}"
     ;;
+    manage )
+            CMD=$param
+            manage_cmd=""
+    ;;
     --help)
             echo "$HELP"
             return 0
     ;;
     *)
-            echo "$HELP"
-            return 1
+            if [ "manage" == "$CMD" ] ; then
+                manage_cmd="$manage_cmd $param"
+            else
+                echo "$HELP"
+                exit 1
+            fi
     ;;
 
     esac
@@ -306,6 +315,10 @@ case $CMD in
         stop_system
         echo "Successful ${CMD}."
     ;;
+    manage )
+        cd $BBBASEDIR/lib/toaster
+        $MANAGE $manage_cmd
+    ;;
 esac
 custom_extention toaster_postpend $CMD $ADDR_PORT
 
-- 
2.20.1

[PATCH 3/4] toaster: FieldError loading Reverse runtime dependencies

[Reyna, David]

When selecting reverse runtime dependencies, filter on simply 'depends_on'
instead of the obsolete 'depends_on__name' or 'depends_on__size'.

[YOCTO #13717]

Signed-off-by: Ahmed.Hossam <Ahmed.Hossam@opensynergy.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
 lib/toaster/toastergui/views.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py
index ae1973ff24..9a5e48e3bb 100644
--- a/lib/toaster/toastergui/views.py
+++ b/lib/toaster/toastergui/views.py
@@ -1191,7 +1191,7 @@ def package_included_reverse_dependencies(request, build_id, target_id, package_
         return _redirect_parameters( 'package_included_reverse_dependencies', request.GET, mandatory_parameters, build_id = build_id, target_id = target_id, package_id = package_id)
     (filter_string, search_term, ordering_string) = _search_tuple(request, Package_File)
 
-    queryset = Package_Dependency.objects.select_related('depends_on__name', 'depends_on__size').filter(depends_on=package_id, target_id=target_id, dep_type=Package_Dependency.TYPE_TRDEPENDS)
+    queryset = Package_Dependency.objects.select_related('depends_on').filter(depends_on=package_id, target_id=target_id, dep_type=Package_Dependency.TYPE_TRDEPENDS)
     objects = _get_queryset(Package_Dependency, queryset, filter_string, search_term, ordering_string, 'package__name')
 
     package = Package.objects.get(pk=package_id)
-- 
2.20.1

[PATCH 4/4] toaster: fix for import build directory with var refs in BBLAYERS

[Reyna, David]

From: Paul Eggleton <paul.eggleton@linux.intel.com>

Update importing a build directory to support where bblayers.conf
sets BBLAYERS to a value that includes a variable reference e.g.:
  BBLAYERS = "${TOPDIR}/../meta \
              ${TOPDIR}/../meta-selftest"

[YOCTO #13707]

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
 .../toastermain/management/commands/buildimport.py   | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/lib/toaster/toastermain/management/commands/buildimport.py b/lib/toaster/toastermain/management/commands/buildimport.py
index 7718caa57a..59da6ff7ac 100644
--- a/lib/toaster/toastermain/management/commands/buildimport.py
+++ b/lib/toaster/toastermain/management/commands/buildimport.py
@@ -114,6 +114,15 @@ class Command(BaseCommand):
             help='command (configure,reconfigure,import)',
             )
 
+    def get_var(self, varname):
+        value = self.vars.get(varname, '')
+        if value:
+            varrefs = re.findall('\${([^}]*)}', value)
+            for ref in varrefs:
+                if ref in self.vars:
+                    value = value.replace('${%s}' % ref, self.vars[ref])
+        return value
+
     # Extract the bb variables from a conf file
     def scan_conf(self,fn):
         vars = self.vars
@@ -241,7 +250,7 @@ class Command(BaseCommand):
     # Apply table of all layer versions
     def extract_bblayers(self):
         # set up the constants
-        bblayer_str = self.vars['BBLAYERS']
+        bblayer_str = self.get_var('BBLAYERS')
         TOASTER_DIR = os.environ.get('TOASTER_DIR')
         INSTALL_CLONE_PREFIX = os.path.dirname(TOASTER_DIR) + "/"
         TOASTER_CLONE_PREFIX = TOASTER_DIR + "/_toaster_clones/"
@@ -421,6 +430,7 @@ class Command(BaseCommand):
 
     # Scan the project's conf files (if any)
     def scan_conf_variables(self,project_path):
+        self.vars['TOPDIR'] = project_path
         # scan the project's settings, add any new layers or variables
         if os.path.isfile("%s/conf/local.conf" % project_path):
             self.scan_conf("%s/conf/local.conf" % project_path)
-- 
2.20.1