[PATCH 0/6] Fixes for toastergui

[PATCH 0/6] Fixes for toastergui

[Ed Bartosh]

Hi,

This patchset fixes creation of Target objects for command line builds
and couple of usability fixes for toastergui module.

The following changes since commit 5a51fb28dbdfcae8ceb503a5290532dd38aeb09f:

  bitbake: prserv/serv: Start/Stop daemon using ip instead of host (2015-09-24 17:55:05 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib ed/toaster/target-task
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/target-task

Ed Bartosh (6):
  event: add new property to BuildBase class
  event: add new parameter to Build* event APIs
  toaster: add task to the target_information
  toaster: add get_target_objects API
  toaster: ignore ReachableStamps event
  toaster: use meaningful logging levels

 bitbake/lib/bb/cooker.py             |  8 ++++----
 bitbake/lib/bb/event.py              | 18 +++++++++++++-----
 bitbake/lib/bb/ui/buildinfohelper.py | 20 +++++++++++++++++++-
 bitbake/lib/bb/ui/toasterui.py       | 11 ++++++-----
 4 files changed, 42 insertions(+), 15 deletions(-)

-- 
2.1.4

Re: [PATCH 0/6] Fixes for toastergui

[Smith, Elliot]

[-- Attachment #1: Type: text/plain, Size: 1905 bytes --]

On 28 September 2015 at 09:56, Ed Bartosh <ed.bartosh@linux.intel.com>
wrote:

> This patchset fixes creation of Target objects for command line builds
> and couple of usability fixes for toastergui module.
>

I've tested this in analysis and managed mode, and it seems to be working
fine.

The only thing I've noticed is that command line builds (shown in the
"Latest builds" and "Project builds" areas) always show the task (e.g.
"zlib:build"), while builds started from the UI don't show it (e.g. "zlib").

I think we should either always hide the suffix when it is ":build", or
always show it, to be consistent.

Elliot


>
> The following changes since commit
> 5a51fb28dbdfcae8ceb503a5290532dd38aeb09f:
>
>   bitbake: prserv/serv: Start/Stop daemon using ip instead of host
> (2015-09-24 17:55:05 +0100)
>
> are available in the git repository at:
>
>   git://git.yoctoproject.org/poky-contrib ed/toaster/target-task
>
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/target-task
>
> Ed Bartosh (6):
>   event: add new property to BuildBase class
>   event: add new parameter to Build* event APIs
>   toaster: add task to the target_information
>   toaster: add get_target_objects API
>   toaster: ignore ReachableStamps event
>   toaster: use meaningful logging levels
>
>  bitbake/lib/bb/cooker.py             |  8 ++++----
>  bitbake/lib/bb/event.py              | 18 +++++++++++++-----
>  bitbake/lib/bb/ui/buildinfohelper.py | 20 +++++++++++++++++++-
>  bitbake/lib/bb/ui/toasterui.py       | 11 ++++++-----
>  4 files changed, 42 insertions(+), 15 deletions(-)
>
> --
> 2.1.4
>
> --
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
>



-- 
Elliot Smith
Software Engineer
Intel Open Source Technology Centre

[-- Attachment #2: Type: text/html, Size: 3078 bytes --]

Re: [PATCH 0/6] Fixes for toastergui

[Barros Pena, Belen]

On 28/09/2015 12:14, "toaster-bounces@yoctoproject.org on behalf of Smith,
Elliot" <toaster-bounces@yoctoproject.org on behalf of
elliot.smith@intel.com> wrote:

>On 28 September 2015 at 09:56, Ed Bartosh
><ed.bartosh@linux.intel.com> wrote:
>
>This patchset fixes creation of Target objects for command line builds
>and couple of usability fixes for toastergui module.
>
>
>
>
>I've tested this in analysis and managed mode, and it seems to be working
>fine.
>
>
>The only thing I've noticed is that command line builds (shown in the
>"Latest builds" and "Project builds" areas) always show the task (e.g.
>"zlib:build"), while builds started from the UI don't show it (e.g.
>"zlib").
>
>
>I think we should either always hide the suffix when it is ":build", or
>always show it, to be consistent.

The 'build' task should not show: I suspect that would create more
confusion than clarity.

Cheers

Belén

>
>
>Elliot
> 
>
>
>The following changes since commit
>5a51fb28dbdfcae8ceb503a5290532dd38aeb09f:
>
>  bitbake: prserv/serv: Start/Stop daemon using ip instead of host
>(2015-09-24 17:55:05 +0100)
>
>are available in the git repository at:
>
>  git://git.yoctoproject.org/poky-contrib
><http://git.yoctoproject.org/poky-contrib> ed/toaster/target-task
>  
>http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/target
>-task 
><http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/targe
>t-task>
>
>Ed Bartosh (6):
>  event: add new property to BuildBase class
>  event: add new parameter to Build* event APIs
>  toaster: add task to the target_information
>  toaster: add get_target_objects API
>  toaster: ignore ReachableStamps event
>  toaster: use meaningful logging levels
>
> bitbake/lib/bb/cooker.py             |  8 ++++----
> bitbake/lib/bb/event.py              | 18 +++++++++++++-----
> bitbake/lib/bb/ui/buildinfohelper.py | 20 +++++++++++++++++++-
> bitbake/lib/bb/ui/toasterui.py       | 11 ++++++-----
> 4 files changed, 42 insertions(+), 15 deletions(-)
>
>--
>2.1.4
>
>--
>_______________________________________________
>toaster mailing list
>toaster@yoctoproject.org
>https://lists.yoctoproject.org/listinfo/toaster
>
>
>
>
>
>
>
>-- 
>Elliot Smith
>Software Engineer
>Intel Open Source Technology Centre
>
>
>
>

[PATCH 0/6] Fixes for toastergui

[Elliot Smith]

This patchset fixes creation of Target objects for command line builds
and provides a couple of usability fixes for toastergui module.

The following changes since commit 78b7cfc2d29209354a1ed3a3a787c651efd31f28:

  toaster: test get_alldeps API (2015-10-05 10:37:47 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib elliot/submit/ed/toaster/target-task
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/submit/ed/toaster/target-task

Ed Bartosh (6):
  event: add new property to BuildBase class
  event: add new parameter to Build* event APIs
  toaster: add task to the target_information
  toaster: add get_or_create_targets API
  toaster: ignore ReachableStamps event
  toaster: use meaningful logging levels

 lib/bb/cooker.py             |  8 ++++----
 lib/bb/event.py              | 18 +++++++++++++-----
 lib/bb/ui/buildinfohelper.py | 37 +++++++++++++++++++++----------------
 lib/bb/ui/toasterui.py       | 11 ++++++-----
 4 files changed, 44 insertions(+), 30 deletions(-)

--
1.9.3

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

[PATCH 1/6] event: add new property to BuildBase class

[Elliot Smith]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Added 'task' property to the base class of BuildStarted and
BuildCompleted classes to store bitbake task if it's specified.

This is done as without task Build* events can't fully represent
bitbake build. Task information is needed by UI classes to properly
show or process Build* events.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
---
 lib/bb/event.py | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/lib/bb/event.py b/lib/bb/event.py
index 366bc41..68496e5 100644
--- a/lib/bb/event.py
+++ b/lib/bb/event.py
@@ -331,6 +331,7 @@ class BuildBase(Event):
     def __init__(self, n, p, failures = 0):
         self._name = n
         self._pkgs = p
+        self._task = None
         Event.__init__(self)
         self._failures = failures
 
@@ -352,6 +353,12 @@ class BuildBase(Event):
     def setCfg(self, cfg):
         self.data = cfg
 
+    def getTask(self):
+        return self._task
+
+    def setTask(self, task):
+        self._task = task
+
     def getFailures(self):
         """
         Return the number of failed packages
@@ -361,6 +368,7 @@ class BuildBase(Event):
     pkgs = property(getPkgs, setPkgs, None, "pkgs property")
     name = property(getName, setName, None, "name property")
     cfg = property(getCfg, setCfg, None, "cfg property")
+    task = property(getTask, setTask, None, "task property")
 
 
 
-- 
1.9.3

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

[PATCH 2/6] event: add new parameter to Build* event APIs

[Elliot Smith]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Added 't'(task) parameter to BuildStarted and BuildCompleted
event APIs. Modified existing calls of above mentioned APIs:
passed bitbake task name as a parameter.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
---
 lib/bb/cooker.py |  8 ++++----
 lib/bb/event.py  | 12 ++++++------
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index a0d7d59..ef81c3a 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -1313,7 +1313,7 @@ class BBCooker:
         taskdata.add_provider(self.data, self.recipecache, item)
 
         buildname = self.data.getVar("BUILDNAME", True)
-        bb.event.fire(bb.event.BuildStarted(buildname, [item]), self.expanded_data)
+        bb.event.fire(bb.event.BuildStarted(buildname, [item], task), self.expanded_data)
 
         # Execute the runqueue
         if not task.startswith("do_"):
@@ -1345,7 +1345,7 @@ class BBCooker:
                 return False
 
             if not retval:
-                bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runq_fnid), buildname, item, failures, interrupted), self.expanded_data)
+                bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runq_fnid), buildname, item, task, failures, interrupted), self.expanded_data)
                 self.command.finishAsyncCommand(msg)
                 return False
             if retval is True:
@@ -1381,7 +1381,7 @@ class BBCooker:
                 return False
 
             if not retval:
-                bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runq_fnid), buildname, targets, failures, interrupted), self.data)
+                bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runq_fnid), buildname, targets, task, failures, interrupted), self.data)
                 self.command.finishAsyncCommand(msg)
                 return False
             if retval is True:
@@ -1394,7 +1394,7 @@ class BBCooker:
         taskdata, runlist, fulltargetlist = self.buildTaskData(targets, task, self.configuration.abort)
 
         buildname = self.data.getVar("BUILDNAME", False)
-        bb.event.fire(bb.event.BuildStarted(buildname, fulltargetlist), self.data)
+        bb.event.fire(bb.event.BuildStarted(buildname, fulltargetlist, task), self.data)
 
         rq = bb.runqueue.RunQueue(self, self.data, self.recipecache, taskdata, runlist)
         if 'universe' in targets:
diff --git a/lib/bb/event.py b/lib/bb/event.py
index 68496e5..bf67ee9 100644
--- a/lib/bb/event.py
+++ b/lib/bb/event.py
@@ -328,10 +328,10 @@ class StampUpdate(Event):
 class BuildBase(Event):
     """Base class for bbmake run events"""
 
-    def __init__(self, n, p, failures = 0):
+    def __init__(self, n, p, t, failures = 0):
         self._name = n
         self._pkgs = p
-        self._task = None
+        self._task = t
         Event.__init__(self)
         self._failures = failures
 
@@ -376,19 +376,19 @@ class BuildBase(Event):
 
 class BuildStarted(BuildBase, OperationStarted):
     """bbmake build run started"""
-    def __init__(self, n, p, failures = 0):
+    def __init__(self, n, p, t, failures = 0):
         OperationStarted.__init__(self, "Building Started")
-        BuildBase.__init__(self, n, p, failures)
+        BuildBase.__init__(self, n, p, t, failures)
 
 class BuildCompleted(BuildBase, OperationCompleted):
     """bbmake build run completed"""
-    def __init__(self, total, n, p, failures=0, interrupted=0):
+    def __init__(self, total, n, p, t, failures=0, interrupted=0):
         if not failures:
             OperationCompleted.__init__(self, total, "Building Succeeded")
         else:
             OperationCompleted.__init__(self, total, "Building Failed")
         self._interrupted = interrupted
-        BuildBase.__init__(self, n, p, failures)
+        BuildBase.__init__(self, n, p, t, failures)
 
 class DiskFull(Event):
     """Disk full case build aborted"""
-- 
1.9.3

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

[PATCH 3/6] toaster: add task to the target_information

[Elliot Smith]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Added task information from BuildStarted event to the
target_information structure. Made sure 'build' task
is not set in Target objects as it shouldn't be shown
in Toaster UI.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
---
 lib/bb/ui/buildinfohelper.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/lib/bb/ui/buildinfohelper.py b/lib/bb/ui/buildinfohelper.py
index 9fdde29..34a8694 100644
--- a/lib/bb/ui/buildinfohelper.py
+++ b/lib/bb/ui/buildinfohelper.py
@@ -925,6 +925,12 @@ class BuildInfoHelper(object):
         target_information = {}
         target_information['targets'] = event._pkgs
         target_information['build'] = build_obj
+        task = getattr(event, 'task', '')
+        if task.startswith('do_'):
+            task = task[3:]
+        if task == 'build':
+            task = ''
+        target_information['task'] = task
 
         self.internal_state['targets'] = self.orm_wrapper.create_target_objects(target_information)
 
-- 
1.9.3

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

[PATCH 4/6] toaster: add get_or_create_targets API

[Elliot Smith]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Target objects are created before the build if build is
started from UI in build mode. However, in analysis mode Target
objects don't exist and need to be created using information
from bitbake events.

Added new API call get_or_create_targets to retrive existing
target objects or create them if they don't exist yet.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
---
 lib/bb/ui/buildinfohelper.py | 29 ++++++++++++++---------------
 1 file changed, 14 insertions(+), 15 deletions(-)

diff --git a/lib/bb/ui/buildinfohelper.py b/lib/bb/ui/buildinfohelper.py
index 34a8694..793418a 100644
--- a/lib/bb/ui/buildinfohelper.py
+++ b/lib/bb/ui/buildinfohelper.py
@@ -162,8 +162,6 @@ class ORMWrapper(object):
             build.bitbake_version=build_info['bitbake_version']
             build.save()
 
-            Target.objects.filter(build = build).delete()
-
         else:
             build = Build.objects.create(
                                     project = prj,
@@ -184,18 +182,19 @@ class ORMWrapper(object):
 
         return build
 
-    def create_target_objects(self, target_info):
-        assert 'build' in target_info
-        assert 'targets' in target_info
-
-        targets = []
-        for tgt_name in target_info['targets']:
-            tgt_object = Target.objects.create( build = target_info['build'],
-                                    target = tgt_name,
-                                    is_image = False,
-                                    )
-            targets.append(tgt_object)
-        return targets
+    @staticmethod
+    def get_or_create_targets(target_info):
+        result = []
+        for target in target_info['targets']:
+            obj, created = Target.objects.get_or_create(build=target_info['build'],
+                                                        target=target)
+            if created:
+                obj.is_image = False
+                if target_info['task']:
+                    obj.task = target_info['task']
+                obj.save()
+            result.append(obj)
+        return result
 
     def update_build_object(self, build, errors, warnings, taskfailures):
         assert isinstance(build,Build)
@@ -932,7 +931,7 @@ class BuildInfoHelper(object):
             task = ''
         target_information['task'] = task
 
-        self.internal_state['targets'] = self.orm_wrapper.create_target_objects(target_information)
+        self.internal_state['targets'] = self.orm_wrapper.get_or_create_targets(target_information)
 
         # Save build configuration
         data = self.server.runCommand(["getAllKeysWithFlags", ["doc", "func"]])[0]
-- 
1.9.3

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

[PATCH 5/6] toaster: ignore ReachableStamps event

[Elliot Smith]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Added ReachableStamps event to the list of known events to
ignore. This should stop toaster throwing error message:
ERROR: Unknown event: <bb.event.ReachableStamps>

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
---
 lib/bb/ui/toasterui.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/bb/ui/toasterui.py b/lib/bb/ui/toasterui.py
index e0c278b..3891e63 100644
--- a/lib/bb/ui/toasterui.py
+++ b/lib/bb/ui/toasterui.py
@@ -308,7 +308,8 @@ def main(server, eventHandler, params ):
                                   bb.event.OperationProgress,
                                   bb.command.CommandFailed,
                                   bb.command.CommandExit,
-                                  bb.command.CommandCompleted)):
+                                  bb.command.CommandCompleted,
+                                  bb.event.ReachableStamps)):
                 continue
 
             if isinstance(event, bb.event.DepTreeGenerated):
-- 
1.9.3

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

[PATCH 6/6] toaster: use meaningful logging levels

[Elliot Smith]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Changed logging levels to more appropriate ones.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
---
 lib/bb/ui/toasterui.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/bb/ui/toasterui.py b/lib/bb/ui/toasterui.py
index 3891e63..dbe0d09 100644
--- a/lib/bb/ui/toasterui.py
+++ b/lib/bb/ui/toasterui.py
@@ -130,14 +130,14 @@ def main(server, eventHandler, params ):
 
             if isinstance(event, (bb.build.TaskStarted, bb.build.TaskSucceeded, bb.build.TaskFailedSilent)):
                 buildinfohelper.update_and_store_task(event)
-                logger.warn("Logfile for task %s", event.logfile)
+                logger.info("Logfile for task %s", event.logfile)
                 continue
 
             if isinstance(event, bb.build.TaskBase):
                 logger.info(event._message)
 
             if isinstance(event, bb.event.LogExecTTY):
-                logger.warn(event.msg)
+                logger.info(event.msg)
                 continue
 
             if isinstance(event, logging.LogRecord):
@@ -316,7 +316,7 @@ def main(server, eventHandler, params ):
                 buildinfohelper.store_dependency_information(event)
                 continue
 
-            logger.error("Unknown event: %s", event)
+            logger.warn("Unknown event: %s", event)
             return_value += 1
 
         except EnvironmentError as ioerror:
@@ -336,7 +336,7 @@ def main(server, eventHandler, params ):
             if tb is not None:
                 curr = tb
                 while curr is not None:
-                    logger.warn("Error data dump %s\n%s\n" , traceback.format_tb(curr,1), pformat(curr.tb_frame.f_locals))
+                    logger.error("Error data dump %s\n%s\n" , traceback.format_tb(curr,1), pformat(curr.tb_frame.f_locals))
                     curr = curr.tb_next
 
             # save them to database, if possible; if it fails, we already logged to console.
-- 
1.9.3

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [PATCH 1/6] event: add new property to BuildBase class

[Richard Purdie]

On Mon, 2015-10-05 at 10:43 +0100, Elliot Smith wrote:
> From: Ed Bartosh <ed.bartosh@linux.intel.com>
> 
> Added 'task' property to the base class of BuildStarted and
> BuildCompleted classes to store bitbake task if it's specified.
> 
> This is done as without task Build* events can't fully represent
> bitbake build. Task information is needed by UI classes to properly
> show or process Build* events.

This doesn't make sense I'm afraid. There is no one "task" which a build
start/complete event can correspond to. Consider something like:

"bitbake A:do_fetch B:do_rootfs"

There would be one BuildStarted event but task doesn't have a singular
value. We therefore don't want to add task information to these events
as it will ultimately mislead anything building on top of this
information.

I appreciate there is the -c option, but that just means "if there is no
specific task specified, use this", i.e. its a default.

Cheers,

Richard

Re: [PATCH 1/6] event: add new property to BuildBase class

[Ed Bartosh]

On Sun, Oct 11, 2015 at 05:22:10AM +0100, Richard Purdie wrote:
> On Mon, 2015-10-05 at 10:43 +0100, Elliot Smith wrote:
> > From: Ed Bartosh <ed.bartosh@linux.intel.com>
> > 
> > Added 'task' property to the base class of BuildStarted and
> > BuildCompleted classes to store bitbake task if it's specified.
> > 
> > This is done as without task Build* events can't fully represent
> > bitbake build. Task information is needed by UI classes to properly
> > show or process Build* events.
> 
> This doesn't make sense I'm afraid. There is no one "task" which a build
> start/complete event can correspond to. Consider something like:
> 
> "bitbake A:do_fetch B:do_rootfs"
> 
> There would be one BuildStarted event but task doesn't have a singular
> value. We therefore don't want to add task information to these events
> as it will ultimately mislead anything building on top of this
> information.
> 
> I appreciate there is the -c option, but that just means "if there is no
> specific task specified, use this", i.e. its a default.
>

OK. can we call it 'default_task' then?
You spotted it's purpose right - it's done to represent the task
specified by -c option.

--
Regards,
Ed

Re: [PATCH 1/6] event: add new property to BuildBase class

[Richard Purdie]

On Sun, 2015-10-11 at 23:02 +0300, Ed Bartosh wrote:
> On Sun, Oct 11, 2015 at 05:22:10AM +0100, Richard Purdie wrote:
> > On Mon, 2015-10-05 at 10:43 +0100, Elliot Smith wrote:
> > > From: Ed Bartosh <ed.bartosh@linux.intel.com>
> > > 
> > > Added 'task' property to the base class of BuildStarted and
> > > BuildCompleted classes to store bitbake task if it's specified.
> > > 
> > > This is done as without task Build* events can't fully represent
> > > bitbake build. Task information is needed by UI classes to properly
> > > show or process Build* events.
> > 
> > This doesn't make sense I'm afraid. There is no one "task" which a build
> > start/complete event can correspond to. Consider something like:
> > 
> > "bitbake A:do_fetch B:do_rootfs"
> > 
> > There would be one BuildStarted event but task doesn't have a singular
> > value. We therefore don't want to add task information to these events
> > as it will ultimately mislead anything building on top of this
> > information.
> > 
> > I appreciate there is the -c option, but that just means "if there is no
> > specific task specified, use this", i.e. its a default.
> >
> 
> OK. can we call it 'default_task' then?
> You spotted it's purpose right - it's done to represent the task
> specified by -c option.

That would certainly be better. I am a little worried how this would be
used in the UI though. Can you explain where and why the UI needs this
piece of information?

Cheers,

Richard

Re: [PATCH 1/6] event: add new property to BuildBase class

[Ed Bartosh]

On Mon, Oct 12, 2015 at 05:54:59AM +0100, Richard Purdie wrote:
> On Sun, 2015-10-11 at 23:02 +0300, Ed Bartosh wrote:
> > On Sun, Oct 11, 2015 at 05:22:10AM +0100, Richard Purdie wrote:
> > > On Mon, 2015-10-05 at 10:43 +0100, Elliot Smith wrote:
> > > > From: Ed Bartosh <ed.bartosh@linux.intel.com>
> > > > 
> > > > Added 'task' property to the base class of BuildStarted and
> > > > BuildCompleted classes to store bitbake task if it's specified.
> > > > 
> > > > This is done as without task Build* events can't fully represent
> > > > bitbake build. Task information is needed by UI classes to properly
> > > > show or process Build* events.
> > > 
> > > This doesn't make sense I'm afraid. There is no one "task" which a build
> > > start/complete event can correspond to. Consider something like:
> > > 
> > > "bitbake A:do_fetch B:do_rootfs"
> > > 
> > > There would be one BuildStarted event but task doesn't have a singular
> > > value. We therefore don't want to add task information to these events
> > > as it will ultimately mislead anything building on top of this
> > > information.
> > > 
> > > I appreciate there is the -c option, but that just means "if there is no
> > > specific task specified, use this", i.e. its a default.
> > >
> > 
> > OK. can we call it 'default_task' then?
> > You spotted it's purpose right - it's done to represent the task
> > specified by -c option.
> 
> That would certainly be better. I am a little worried how this would be
> used in the UI though. Can you explain where and why the UI needs this
> piece of information?

After looking at UI code I think I can implement this functionality
differently. Thank you for pointing out to this.

Please merge patches 5 and 6 from this patchset. They're not directly
related to 'task' property and can be merged independently.

--
Regards,
Ed

[PATCH] bitbake: normalize build targets

[Ed Bartosh]

BuildStarted event not fully represents build tasks for
the targets. If -c option is used to specify default task
it's not included into the event.

Made build targets to always look as <target>:do_<task>.
Consider default task (do_build or specified by -c command
line option) when normalizing.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/bb/cooker.py | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
index a0d7d59..7e84a7c 100644
--- a/bitbake/lib/bb/cooker.py
+++ b/bitbake/lib/bb/cooker.py
@@ -1391,10 +1391,28 @@ class BBCooker:
         build.reset_cache()
         self.buildSetVars()
 
+        # If we are told to do the None task then query the default task
+        if (task == None):
+            task = self.configuration.cmd
+
+        if not task.startswith("do_"):
+            task = "do_%s" % task
+
         taskdata, runlist, fulltargetlist = self.buildTaskData(targets, task, self.configuration.abort)
 
         buildname = self.data.getVar("BUILDNAME", False)
-        bb.event.fire(bb.event.BuildStarted(buildname, fulltargetlist), self.data)
+
+        # make targets to always look as <target>:do_<task>
+        ntargets = []
+        for target in fulltargetlist:
+            if ":" in target:
+                if ":do_" not in target:
+                    target = "%s:do_%s" % tuple(target.split(":"))
+            else:
+                target = "%s:%s" % (target, task)
+            ntargets.append(target)
+
+        bb.event.fire(bb.event.BuildStarted(buildname, ntargets), self.data)
 
         rq = bb.runqueue.RunQueue(self, self.data, self.recipecache, taskdata, runlist)
         if 'universe' in targets:
-- 
2.1.4

Re: [PATCH] bitbake: normalize build targets

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 714 bytes --]

On Wed, Oct 14, 2015 at 1:41 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
wrote:

> +        for target in fulltargetlist:
> +            if ":" in target:
> +                if ":do_" not in target:
> +                    target = "%s:do_%s" % tuple(target.split(":"))
>

I know it's an unlikely and theoretical case, but just in case the user
asks to build foo:: or something, we wouldn't want to explode horribly with
a traceback. It's best to do target.split(":", 1) here, so you know we'll
get 2 values for the format strings, no more.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 1232 bytes --]

[PATCH v2] bitbake: normalize build targets

[Ed Bartosh]

BuildStarted event not fully represents build tasks for
the targets. If -c option is used to specify default task
it's not included into the event.

Made build targets to always look as <target>:do_<task>.
Consider default task (do_build or specified by -c command
line option) when normalizing.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/bb/cooker.py | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
index a0d7d59..a416d2e 100644
--- a/bitbake/lib/bb/cooker.py
+++ b/bitbake/lib/bb/cooker.py
@@ -1391,10 +1391,28 @@ class BBCooker:
         build.reset_cache()
         self.buildSetVars()
 
+        # If we are told to do the None task then query the default task
+        if (task == None):
+            task = self.configuration.cmd
+
+        if not task.startswith("do_"):
+            task = "do_%s" % task
+
         taskdata, runlist, fulltargetlist = self.buildTaskData(targets, task, self.configuration.abort)
 
         buildname = self.data.getVar("BUILDNAME", False)
-        bb.event.fire(bb.event.BuildStarted(buildname, fulltargetlist), self.data)
+
+        # make targets to always look as <target>:do_<task>
+        ntargets = []
+        for target in fulltargetlist:
+            if ":" in target:
+                if ":do_" not in target:
+                    target = "%s:do_%s" % tuple(target.split(":", 1))
+            else:
+                target = "%s:%s" % (target, task)
+            ntargets.append(target)
+
+        bb.event.fire(bb.event.BuildStarted(buildname, ntargets), self.data)
 
         rq = bb.runqueue.RunQueue(self, self.data, self.recipecache, taskdata, runlist)
         if 'universe' in targets:
-- 
2.1.4

Re: [PATCH v2] bitbake: normalize build targets

[Richard Purdie]

On Thu, 2015-10-15 at 11:37 +0300, Ed Bartosh wrote:
> BuildStarted event not fully represents build tasks for
> the targets. If -c option is used to specify default task
> it's not included into the event.
> 
> Made build targets to always look as <target>:do_<task>.
> Consider default task (do_build or specified by -c command
> line option) when normalizing.
> 
> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

The correct prefix for the commit shortlog is "cooker:". I'd really like
to see toaster becoming a little more specific too such as
"toaster/xxxx:" instead of the usual "toaster:".

Cheers,

Richard

Re: [bitbake-devel] [PATCH v2] bitbake: normalize build targets

[Richard Purdie]

On Thu, 2015-10-15 at 11:37 +0300, Ed Bartosh wrote:
> BuildStarted event not fully represents build tasks for
> the targets. If -c option is used to specify default task
> it's not included into the event.
> 
> Made build targets to always look as <target>:do_<task>.
> Consider default task (do_build or specified by -c command
> line option) when normalizing.
> 
> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

The correct prefix for the commit shortlog is "cooker:". I'd really like
to see toaster becoming a little more specific too such as
"toaster/xxxx:" instead of the usual "toaster:".

Cheers,

Richard