* [PATCH 1/3] toaster: Update to Django 4.2
2023-08-28 8:45 [PATCH 0/3] toaster_cummulative_08282023 David Reyna
@ 2023-08-28 8:45 ` David Reyna
2023-08-28 8:45 ` [PATCH 2/3] toaster: import only used layers David Reyna
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: David Reyna @ 2023-08-28 8:45 UTC (permalink / raw)
To: bitbake-devel
Update Toaster to support Django 4.2, to match current
hosts and to address CVEs.
[YOCTO #15152]
Signed-off-by: Kieran McNulty <Kieran.McNulty@windriver.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
lib/toaster/bldcollector/urls.py | 2 +-
lib/toaster/bldcontrol/models.py | 4 ++--
lib/toaster/toastergui/urls.py | 2 +-
lib/toaster/toastergui/views.py | 4 ++--
lib/toaster/toastergui/widgets.py | 5 +++--
lib/toaster/toastermain/management/commands/checksocket.py | 4 ++--
lib/toaster/toastermain/urls.py | 2 +-
toaster-requirements.txt | 2 +-
8 files changed, 13 insertions(+), 12 deletions(-)
diff --git a/lib/toaster/bldcollector/urls.py b/lib/toaster/bldcollector/urls.py
index efd67a81a54..3c34070351c 100644
--- a/lib/toaster/bldcollector/urls.py
+++ b/lib/toaster/bldcollector/urls.py
@@ -6,7 +6,7 @@
# SPDX-License-Identifier: GPL-2.0-only
#
-from django.conf.urls import url
+from django.urls import re_path as url
import bldcollector.views
diff --git a/lib/toaster/bldcontrol/models.py b/lib/toaster/bldcontrol/models.py
index c2f302da24a..42750e7180a 100644
--- a/lib/toaster/bldcontrol/models.py
+++ b/lib/toaster/bldcontrol/models.py
@@ -4,7 +4,7 @@
from __future__ import unicode_literals
from django.db import models
-from django.utils.encoding import force_text
+from django.utils.encoding import force_str
from orm.models import Project, Build, Layer_Version
import logging
@@ -124,7 +124,7 @@ class BuildRequest(models.Model):
return self.brvariable_set.get(name="MACHINE").value
def __str__(self):
- return force_text('%s %s' % (self.project, self.get_state_display()))
+ return force_str('%s %s' % (self.project, self.get_state_display()))
# These tables specify the settings for running an actual build.
# They MUST be kept in sync with the tables in orm.models.Project*
diff --git a/lib/toaster/toastergui/urls.py b/lib/toaster/toastergui/urls.py
index d2df4e60488..bc3b0c79d82 100644
--- a/lib/toaster/toastergui/urls.py
+++ b/lib/toaster/toastergui/urls.py
@@ -6,7 +6,7 @@
# SPDX-License-Identifier: GPL-2.0-only
#
-from django.conf.urls import url
+from django.urls import re_path as url
from django.views.generic import RedirectView
from toastergui import tables
diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py
index a571b8cc18c..bf92510cdcd 100644
--- a/lib/toaster/toastergui/views.py
+++ b/lib/toaster/toastergui/views.py
@@ -670,11 +670,11 @@ def xhr_dirinfo(request, build_id, target_id):
return HttpResponse(_get_dir_entries(build_id, target_id, top), content_type = "application/json")
from django.utils.functional import Promise
-from django.utils.encoding import force_text
+from django.utils.encoding import force_str
class LazyEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, Promise):
- return force_text(obj)
+ return force_str(obj)
return super(LazyEncoder, self).default(obj)
from toastergui.templatetags.projecttags import filtered_filesizeformat
diff --git a/lib/toaster/toastergui/widgets.py b/lib/toaster/toastergui/widgets.py
index ceff52942ee..53696912a46 100644
--- a/lib/toaster/toastergui/widgets.py
+++ b/lib/toaster/toastergui/widgets.py
@@ -7,6 +7,7 @@
#
from django.views.generic import View, TemplateView
+from django.utils.decorators import method_decorator
from django.views.decorators.cache import cache_control
from django.shortcuts import HttpResponse
from django.core.cache import cache
@@ -63,8 +64,8 @@ class ToasterTable(TemplateView):
self.default_orderby = ""
# prevent HTTP caching of table data
- @cache_control(must_revalidate=True,
- max_age=0, no_store=True, no_cache=True)
+ @method_decorator(cache_control(must_revalidate=True,
+ max_age=0, no_store=True, no_cache=True))
def dispatch(self, *args, **kwargs):
return super(ToasterTable, self).dispatch(*args, **kwargs)
diff --git a/lib/toaster/toastermain/management/commands/checksocket.py b/lib/toaster/toastermain/management/commands/checksocket.py
index 811fd5d5161..b2c002da7aa 100644
--- a/lib/toaster/toastermain/management/commands/checksocket.py
+++ b/lib/toaster/toastermain/management/commands/checksocket.py
@@ -13,7 +13,7 @@ import errno
import socket
from django.core.management.base import BaseCommand, CommandError
-from django.utils.encoding import force_text
+from django.utils.encoding import force_str
DEFAULT_ADDRPORT = "0.0.0.0:8000"
@@ -51,7 +51,7 @@ class Command(BaseCommand):
if hasattr(err, 'errno') and err.errno in errors:
errtext = errors[err.errno]
else:
- errtext = force_text(err)
+ errtext = force_str(err)
raise CommandError(errtext)
self.stdout.write("OK")
diff --git a/lib/toaster/toastermain/urls.py b/lib/toaster/toastermain/urls.py
index 5fb520b384d..03603026688 100644
--- a/lib/toaster/toastermain/urls.py
+++ b/lib/toaster/toastermain/urls.py
@@ -6,7 +6,7 @@
# SPDX-License-Identifier: GPL-2.0-only
#
-from django.conf.urls import include, url
+from django.urls import re_path as url, include
from django.views.generic import RedirectView, TemplateView
from django.views.decorators.cache import never_cache
import bldcollector.views
diff --git a/toaster-requirements.txt b/toaster-requirements.txt
index dedd423556a..c1f433f9ec6 100644
--- a/toaster-requirements.txt
+++ b/toaster-requirements.txt
@@ -1,3 +1,3 @@
-Django>3.2,<3.3
+Django>4.2,<4.3
beautifulsoup4>=4.4.0
pytz
--
2.20.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH 2/3] toaster: import only used layers
2023-08-28 8:45 [PATCH 0/3] toaster_cummulative_08282023 David Reyna
2023-08-28 8:45 ` [PATCH 1/3] toaster: Update to Django 4.2 David Reyna
@ 2023-08-28 8:45 ` David Reyna
2023-08-28 8:45 ` [PATCH 3/3] toaster: accommodate missing 'Image Name' value in buildinfohelper David Reyna
2023-08-29 14:38 ` [bitbake-devel] [PATCH 0/3] toaster_cummulative_08282023 Alexandre Belloni
3 siblings, 0 replies; 5+ messages in thread
From: David Reyna @ 2023-08-28 8:45 UTC (permalink / raw)
To: bitbake-devel
If you import a build directory, Toaster still adds openembedded-core,
meta-poky and meta-yocto-bsp to the newly created project. Toaster
should only be including in the project the layers that it imported.
[YOCTO #13764]
Signed-off-by: Kieran McNulty <Kieran.McNulty@windriver.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
lib/toaster/orm/models.py | 28 +++++++++----------
lib/toaster/toastergui/views.py | 2 +-
.../management/commands/buildimport.py | 2 +-
3 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index 2cb7d7e0499..f9fcf9e4fbb 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -107,7 +107,7 @@ class ToasterSetting(models.Model):
class ProjectManager(models.Manager):
- def create_project(self, name, release, existing_project=None):
+ def create_project(self, name, release, existing_project=None, imported=False):
if existing_project and (release is not None):
prj = existing_project
prj.bitbake_version = release.bitbake_version
@@ -134,19 +134,19 @@ class ProjectManager(models.Manager):
if release is None:
return prj
-
- for rdl in release.releasedefaultlayer_set.all():
- lv = Layer_Version.objects.filter(
- layer__name=rdl.layer_name,
- release=release).first()
-
- if lv:
- ProjectLayer.objects.create(project=prj,
- layercommit=lv,
- optional=False)
- else:
- logger.warning("Default project layer %s not found" %
- rdl.layer_name)
+ if not imported:
+ for rdl in release.releasedefaultlayer_set.all():
+ lv = Layer_Version.objects.filter(
+ layer__name=rdl.layer_name,
+ release=release).first()
+
+ if lv:
+ ProjectLayer.objects.create(project=prj,
+ layercommit=lv,
+ optional=False)
+ else:
+ logger.warning("Default project layer %s not found" %
+ rdl.layer_name)
return prj
diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py
index bf92510cdcd..552ff1649bf 100644
--- a/lib/toaster/toastergui/views.py
+++ b/lib/toaster/toastergui/views.py
@@ -1404,7 +1404,7 @@ if True:
if not os.path.isdir('%s/conf' % request.POST['importdir']):
raise BadParameterException("Bad path or missing 'conf' directory (%s)" % request.POST['importdir'])
from django.core import management
- management.call_command('buildimport', '--command=import', '--name=%s' % request.POST['projectname'], '--path=%s' % request.POST['importdir'], interactive=False)
+ management.call_command('buildimport', '--command=import', '--name=%s' % request.POST['projectname'], '--path=%s' % request.POST['importdir'])
prj = Project.objects.get(name = request.POST['projectname'])
prj.merged_attr = True
prj.save()
diff --git a/lib/toaster/toastermain/management/commands/buildimport.py b/lib/toaster/toastermain/management/commands/buildimport.py
index e25b55e5ab9..f7139aa0419 100644
--- a/lib/toaster/toastermain/management/commands/buildimport.py
+++ b/lib/toaster/toastermain/management/commands/buildimport.py
@@ -545,7 +545,7 @@ class Command(BaseCommand):
# Find the directory's release, and promote to default_release if local paths
release = self.find_import_release(layers_list,lv_dict,default_release)
# create project, SANITY: reuse any project of same name
- project = Project.objects.create_project(project_name,release,project)
+ project = Project.objects.create_project(project_name,release,project, imported=True)
# Apply any new layers or variables
self.apply_conf_variables(project,layers_list,lv_dict,release)
# WORKAROUND: since we now derive the release, redirect 'newproject_specific' to 'project_specific'
--
2.20.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH 3/3] toaster: accommodate missing 'Image Name' value in buildinfohelper
2023-08-28 8:45 [PATCH 0/3] toaster_cummulative_08282023 David Reyna
2023-08-28 8:45 ` [PATCH 1/3] toaster: Update to Django 4.2 David Reyna
2023-08-28 8:45 ` [PATCH 2/3] toaster: import only used layers David Reyna
@ 2023-08-28 8:45 ` David Reyna
2023-08-29 14:38 ` [bitbake-devel] [PATCH 0/3] toaster_cummulative_08282023 Alexandre Belloni
3 siblings, 0 replies; 5+ messages in thread
From: David Reyna @ 2023-08-28 8:45 UTC (permalink / raw)
To: bitbake-devel
The value "Image Name" in bitbake events was missing for certain builds. Update 'buildinfohelper' to extract the
image name elsewhere in this circumstance and not crash.
[YOCTO #13191]
Signed-off-by: Kieran McNulty <Kieran.McNulty@windriver.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
lib/bb/ui/buildinfohelper.py | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/lib/bb/ui/buildinfohelper.py b/lib/bb/ui/buildinfohelper.py
index 129bb329c38..8b212b78034 100644
--- a/lib/bb/ui/buildinfohelper.py
+++ b/lib/bb/ui/buildinfohelper.py
@@ -1746,7 +1746,6 @@ class BuildInfoHelper(object):
buildname = self.server.runCommand(['getVariable', 'BUILDNAME'])[0]
machine = self.server.runCommand(['getVariable', 'MACHINE'])[0]
- image_name = self.server.runCommand(['getVariable', 'IMAGE_NAME'])[0]
# location of the manifest files for this build;
# note that this file is only produced if an image is produced
@@ -1767,6 +1766,18 @@ class BuildInfoHelper(object):
# filter out anything which isn't an image target
image_targets = [target for target in targets if target.is_image]
+ if len(image_targets) > 0:
+ #if there are image targets retrieve image_name
+ image_name = self.server.runCommand(['getVariable', 'IMAGE_NAME'])[0]
+ if not image_name:
+ #When build target is an image and image_name is not found as an environment variable
+ logger.info("IMAGE_NAME not found, extracting from bitbake command")
+ cmd = self.server.runCommand(['getVariable','BB_CMDLINE'])[0]
+ #filter out tokens that are command line options
+ cmd = [token for token in cmd if not token.startswith('-')]
+ image_name = cmd[1].split(':', 1)[0] # remove everything after : in image name
+ logger.info("IMAGE_NAME found as : %s " % image_name)
+
for image_target in image_targets:
# this is set to True if we find at least one file relating to
# this target; if this remains False after the scan, we copy the
--
2.20.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [bitbake-devel] [PATCH 0/3] toaster_cummulative_08282023
2023-08-28 8:45 [PATCH 0/3] toaster_cummulative_08282023 David Reyna
` (2 preceding siblings ...)
2023-08-28 8:45 ` [PATCH 3/3] toaster: accommodate missing 'Image Name' value in buildinfohelper David Reyna
@ 2023-08-29 14:38 ` Alexandre Belloni
3 siblings, 0 replies; 5+ messages in thread
From: Alexandre Belloni @ 2023-08-29 14:38 UTC (permalink / raw)
To: david.reyna; +Cc: bitbake-devel, Kieran McNulty
Hello David,
Please set you git configuration so your patches include a From: line,
see https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded#Fixing_your_From_identity
On 28/08/2023 01:45:17-0700, Reyna, David via lists.openembedded.org wrote:
> toaster_cummulative_08282023
>
> The following changes since commit c8e9590a376c0b63e38167dc522fac4cfbecb5d4:
>
> bitbake: Fix disk space monitoring on cephfs (2023-08-25 07:55:34 +0100)
>
> are available in the Git repository at:
>
> ssh://git@push.yoctoproject.org/poky-contrib dreyna/submit/dreyna/toaster/toaster_cummulative_08282023
>
> David Reyna (3):
> toaster: Update to Django 4.2
> toaster: import only used layers
> toaster: accommodate missing 'Image Name' value in buildinfohelper
>
> lib/bb/ui/buildinfohelper.py | 13 ++++++++-
> lib/toaster/bldcollector/urls.py | 2 +-
> lib/toaster/bldcontrol/models.py | 4 +--
> lib/toaster/orm/models.py | 28 +++++++++----------
> lib/toaster/toastergui/urls.py | 2 +-
> lib/toaster/toastergui/views.py | 6 ++--
> lib/toaster/toastergui/widgets.py | 5 ++--
> .../management/commands/buildimport.py | 2 +-
> .../management/commands/checksocket.py | 4 +--
> lib/toaster/toastermain/urls.py | 2 +-
> toaster-requirements.txt | 2 +-
> 11 files changed, 41 insertions(+), 29 deletions(-)
>
> --
> 2.20.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#14985): https://lists.openembedded.org/g/bitbake-devel/message/14985
> Mute This Topic: https://lists.openembedded.org/mt/101006437/3617179
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 5+ messages in thread