[PATCH 0/7] toaster: allow setEventMask to be called by bitbake observer

[PATCH 0/7] toaster: allow setEventMask to be called by bitbake observer

[brian avery]

This patchset is a preparation for the #7880 fix. This will allow the toasterui to work in build mode, i.e. with
bitbake <target> -u toasterui.

If you want to test this functionality you can do the following:
 - start Toaster
 - kill bitbake server and observer processes
 - unset BBSERVER
 - add INHERIT+="toaster buildhistory" line to conf/local.conf
 - run bitbake <target> -u toasterui

Expected outcome: you should see commandline build in UI without bitbake server
                  and observer running.

Also, I've discovered nasty bug in handling setEventMask by bitbake server while
working on this. It causes ignoring of the event mask set by toasterui. The result
of this is that list of events in _ev_list variable is different from the actual
list of events processed by toasterui.

Upstreamed for Ed,
-bavery

The following changes since commit ffce6a18876c3a0c949eb75f0ce2f9dc36054c83:

  toaster: remove sshbecontroller module (2016-03-02 20:31:08 -0800)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib bavery/submit/ed/toaster/toasterui-in-build-mode
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=bavery/submit/ed/toaster/toasterui-in-build-mode

Ed Bartosh (7):
  toasterui: reformat list of events
  toasterui: update list of events
  command: make setEventMask readonly
  toasterui: check if setEventMask succeeded
  toasterui: make toasterui to work in build mode
  toasterui: exit on final events
  toasterui: fix warning 'Unknown event'

 lib/bb/command.py      |  1 +
 lib/bb/ui/toasterui.py | 90 +++++++++++++++++++++++++++++++++++---------------
 2 files changed, 64 insertions(+), 27 deletions(-)

--
1.9.1

[PATCH 1/7] toasterui: reformat list of events

[brian avery]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Reformatted and reordered list of events to make changes
easily and see them clearly in the diffs.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
 lib/bb/ui/toasterui.py | 36 +++++++++++++++++++++++++++---------
 1 file changed, 27 insertions(+), 9 deletions(-)

diff --git a/lib/bb/ui/toasterui.py b/lib/bb/ui/toasterui.py
index 32b1889..377526e 100644
--- a/lib/bb/ui/toasterui.py
+++ b/lib/bb/ui/toasterui.py
@@ -92,15 +92,33 @@ def _close_build_log(build_log):
         build_log.close()
         logger.removeHandler(build_log)
 
-_evt_list = [ "bb.runqueue.runQueueExitWait", "bb.event.LogExecTTY", "logging.LogRecord",
-              "bb.build.TaskFailed", "bb.build.TaskBase", "bb.event.ParseStarted",
-              "bb.event.ParseProgress", "bb.event.ParseCompleted", "bb.event.CacheLoadStarted",
-              "bb.event.CacheLoadProgress", "bb.event.CacheLoadCompleted", "bb.command.CommandFailed",
-              "bb.command.CommandExit", "bb.command.CommandCompleted",  "bb.cooker.CookerExit",
-              "bb.event.MultipleProviders", "bb.event.NoProvider", "bb.runqueue.sceneQueueTaskStarted",
-              "bb.runqueue.runQueueTaskStarted", "bb.runqueue.runQueueTaskFailed", "bb.runqueue.sceneQueueTaskFailed",
-              "bb.event.BuildBase", "bb.build.TaskStarted", "bb.build.TaskSucceeded", "bb.build.TaskFailedSilent",
-              "bb.event.MetadataEvent"]
+_evt_list = [
+    "bb.build.TaskBase",
+    "bb.build.TaskFailed",
+    "bb.build.TaskFailedSilent",
+    "bb.build.TaskStarted",
+    "bb.build.TaskSucceeded",
+    "bb.command.CommandCompleted",
+    "bb.command.CommandExit",
+    "bb.command.CommandFailed",
+    "bb.cooker.CookerExit",
+    "bb.event.BuildBase",
+    "bb.event.CacheLoadCompleted",
+    "bb.event.CacheLoadProgress",
+    "bb.event.CacheLoadStarted",
+    "bb.event.LogExecTTY",
+    "bb.event.MetadataEvent",
+    "bb.event.MultipleProviders",
+    "bb.event.NoProvider",
+    "bb.event.ParseCompleted",
+    "bb.event.ParseProgress",
+    "bb.event.ParseStarted",
+    "bb.runqueue.runQueueExitWait",
+    "bb.runqueue.runQueueTaskFailed",
+    "bb.runqueue.runQueueTaskStarted",
+    "bb.runqueue.sceneQueueTaskFailed",
+    "bb.runqueue.sceneQueueTaskStarted",
+    "logging.LogRecord"]
 
 def main(server, eventHandler, params):
     # set to a logging.FileHandler instance when a build starts;
-- 
1.9.1

[PATCH 2/7] toasterui: update list of events

[brian avery]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Removed events not used in the code from the list.
Added events that are used in the code.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
 lib/bb/ui/toasterui.py | 27 ++++++++++++---------------
 1 file changed, 12 insertions(+), 15 deletions(-)

diff --git a/lib/bb/ui/toasterui.py b/lib/bb/ui/toasterui.py
index 377526e..4bb33f1 100644
--- a/lib/bb/ui/toasterui.py
+++ b/lib/bb/ui/toasterui.py
@@ -102,10 +102,13 @@ _evt_list = [
     "bb.command.CommandExit",
     "bb.command.CommandFailed",
     "bb.cooker.CookerExit",
-    "bb.event.BuildBase",
+    "bb.event.BuildCompleted",
+    "bb.event.BuildStarted",
     "bb.event.CacheLoadCompleted",
     "bb.event.CacheLoadProgress",
     "bb.event.CacheLoadStarted",
+    "bb.event.ConfigParsed",
+    "bb.event.DepTreeGenerated",
     "bb.event.LogExecTTY",
     "bb.event.MetadataEvent",
     "bb.event.MultipleProviders",
@@ -113,9 +116,16 @@ _evt_list = [
     "bb.event.ParseCompleted",
     "bb.event.ParseProgress",
     "bb.event.ParseStarted",
-    "bb.runqueue.runQueueExitWait",
+    "bb.event.RecipeParsed",
+    "bb.event.SanityCheck",
+    "bb.event.SanityCheckPassed",
+    "bb.event.TreeDataPreparationCompleted",
+    "bb.event.TreeDataPreparationStarted",
+    "bb.runqueue.runQueueTaskCompleted",
     "bb.runqueue.runQueueTaskFailed",
+    "bb.runqueue.runQueueTaskSkipped",
     "bb.runqueue.runQueueTaskStarted",
+    "bb.runqueue.sceneQueueTaskCompleted",
     "bb.runqueue.sceneQueueTaskFailed",
     "bb.runqueue.sceneQueueTaskStarted",
     "logging.LogRecord"]
@@ -395,19 +405,6 @@ def main(server, eventHandler, params):
                 main.shutdown = 1
                 continue
 
-            # ignore
-            if isinstance(event, (bb.event.BuildBase,
-                                  bb.event.StampUpdate,
-                                  bb.event.RecipePreFinalise,
-                                  bb.runqueue.runQueueEvent,
-                                  bb.runqueue.runQueueExitWait,
-                                  bb.event.OperationProgress,
-                                  bb.command.CommandFailed,
-                                  bb.command.CommandExit,
-                                  bb.command.CommandCompleted,
-                                  bb.event.ReachableStamps)):
-                continue
-
             if isinstance(event, bb.event.DepTreeGenerated):
                 buildinfohelper.store_dependency_information(event)
                 continue
-- 
1.9.1

[PATCH 3/7] command: make setEventMask readonly

[brian avery]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Executing setEventMask command when bitbake server is in readonly
mode causes runCommand to fail with the following error:
 'Not able to execute not readonly commands in readonly mode'

Set readonly attribute for setEventMask command to make it working
for Toaster UI. This should not do any harm as this command doesn't
influence cooker state.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
 lib/bb/command.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/bb/command.py b/lib/bb/command.py
index 74106d1..0559ffc 100644
--- a/lib/bb/command.py
+++ b/lib/bb/command.py
@@ -279,6 +279,7 @@ class CommandsSync:
         mask = params[3]
         return bb.event.set_UIHmask(handlerNum, llevel, debug_domains, mask)
     setEventMask.needconfig = False
+    setEventMask.readonly = True
 
     def setFeatures(self, command, params):
         """
-- 
1.9.1

[PATCH 4/7] toasterui: check if setEventMask succeeded

[brian avery]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Currently toasterui ignores return value of setEventMask
command, which created confusing difference between set of
events set by this command and the real set used in the code.

Checked if setEventMask succeeded. Print error message and
exit if it's not.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
 lib/bb/ui/toasterui.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/lib/bb/ui/toasterui.py b/lib/bb/ui/toasterui.py
index 4bb33f1..fec6962 100644
--- a/lib/bb/ui/toasterui.py
+++ b/lib/bb/ui/toasterui.py
@@ -154,8 +154,10 @@ def main(server, eventHandler, params):
     logger.addHandler(console)
     logger.setLevel(logging.INFO)
     llevel, debug_domains = bb.msg.constructLogOptions()
-    server.runCommand(["setEventMask", server.getEventHandle(), llevel, debug_domains, _evt_list])
-
+    result, error = server.runCommand(["setEventMask", server.getEventHandle(), llevel, debug_domains, _evt_list])
+    if not result or error:
+        logger.error("can't set event mask: %s", error)
+        return 1
 
     # verify and warn
     build_history_enabled = True
-- 
1.9.1

[PATCH 5/7] toasterui: make toasterui to work in build mode

[brian avery]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Currently toasterui works only in observer mode. This is
artificial limitation which was made to support current toaster
design. As we decided to stop using bitbake server we'll
need to run toasterui also in build mode.

[YOCTO #7880]

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
 lib/bb/ui/toasterui.py | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/lib/bb/ui/toasterui.py b/lib/bb/ui/toasterui.py
index fec6962..2d44377 100644
--- a/lib/bb/ui/toasterui.py
+++ b/lib/bb/ui/toasterui.py
@@ -168,8 +168,23 @@ def main(server, eventHandler, params):
         build_history_enabled = False
 
     if not params.observe_only:
-        logger.error("ToasterUI can only work in observer mode")
-        return 1
+        params.updateFromServer(server)
+        params.updateToServer(server, os.environ.copy())
+        cmdline = params.parseActions()
+        if not cmdline:
+            print("Nothing to do.  Use 'bitbake world' to build everything, or run 'bitbake --help' for usage information.")
+            return 1
+        if 'msg' in cmdline and cmdline['msg']:
+            logger.error(cmdline['msg'])
+            return 1
+
+        ret, error = server.runCommand(cmdline['action'])
+        if error:
+            logger.error("Command '%s' failed: %s" % (cmdline, error))
+            return 1
+        elif ret != True:
+            logger.error("Command '%s' failed: returned %s" % (cmdline, ret))
+            return 1
 
     # set to 1 when toasterui needs to shut down
     main.shutdown = 0
-- 
1.9.1

[PATCH 6/7] toasterui: exit on final events

[brian avery]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Toasterui exits only if bitbake observer shuts down.
In build mode it should exit when build is done.

Made toasterui exit on bb.command.CommandCompleted,
bb.command.CommandFailed and bb.command.CommandExit events
when it's running in build mode.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
 lib/bb/ui/toasterui.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/bb/ui/toasterui.py b/lib/bb/ui/toasterui.py
index 2d44377..728803d 100644
--- a/lib/bb/ui/toasterui.py
+++ b/lib/bb/ui/toasterui.py
@@ -388,7 +388,10 @@ def main(server, eventHandler, params):
             if isinstance(event, (bb.command.CommandCompleted,
                                   bb.command.CommandFailed,
                                   bb.command.CommandExit)):
-                errorcode = 0
+                if params.observe_only:
+                    errorcode = 0
+                else:
+                    main.shutdown = 1
 
                 continue
 
-- 
1.9.1

[PATCH 7/7] toasterui: fix warning 'Unknown event'

[brian avery]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Continue after processing BuildStarted event to fix
WARNING: Unknown event: <bb.event.BuildStarted object at 0x2554150>

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
 lib/bb/ui/toasterui.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/bb/ui/toasterui.py b/lib/bb/ui/toasterui.py
index 728803d..eee8d14 100644
--- a/lib/bb/ui/toasterui.py
+++ b/lib/bb/ui/toasterui.py
@@ -244,6 +244,7 @@ def main(server, eventHandler, params):
                     build_log, build_log_file_path = _open_build_log(log_dir)
 
                 buildinfohelper.store_started_build(event, build_log_file_path)
+                continue
 
             if isinstance(event, (bb.build.TaskStarted, bb.build.TaskSucceeded, bb.build.TaskFailedSilent)):
                 buildinfohelper.update_and_store_task(event)
-- 
1.9.1