* [PATCH v4 01/18] process.py, build.py: Fix log truncation problems with flush()
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 02/18] knotty: Extend knotty to support "real time" log tail for tasks Jason Wessel
` (16 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
There are two problems with the _logged_communicate that are both
caused as a result of buffering I/O, instead of flushing it out to the
log files as it arrives.
1) log truncation when python dumps
I have seen the task logs missing data that was not flushed when
bitbake crashes.
2) While a bitbake task is running it is impossible to see what is
going on if it is only writing a small incremental log that is
smaller than the buffer, or you get only a partial log, up until
the task exists. It is worse in the case that stderr and stdout
are separate file handles, because previous code blocks on the read
of stdout and then stderr, serially.
The right approach is simply to use select() to determine if there is
data available and then to flush the log buffers as they arrive. This
still makes use of buffering in the cases where there is more than 1
byte of data, but the buffers are much more dynamic.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
lib/bb/build.py | 3 ++-
lib/bb/process.py | 29 +++++++++++++++++++++++++++--
2 files changed, 29 insertions(+), 3 deletions(-)
diff --git a/lib/bb/build.py b/lib/bb/build.py
index 95f1dcf..da4c911 100644
--- a/lib/bb/build.py
+++ b/lib/bb/build.py
@@ -135,7 +135,8 @@ class LogTee(object):
def __repr__(self):
return '<LogTee {0}>'.format(self.name)
-
+ def flush(self):
+ self.outfile.flush()
def exec_func(func, d, dirs = None):
"""Execute an BB 'function'"""
diff --git a/lib/bb/process.py b/lib/bb/process.py
index b74cb18..9848fc3 100644
--- a/lib/bb/process.py
+++ b/lib/bb/process.py
@@ -1,6 +1,9 @@
import logging
import signal
import subprocess
+import fcntl
+import errno
+import select
logger = logging.getLogger('BitBake.Process')
@@ -70,18 +73,40 @@ def _logged_communicate(pipe, log, input):
bufsize = 512
outdata, errdata = [], []
+ rin = []
+
+ if pipe.stdout is not None:
+ fd = pipe.stdout.fileno()
+ fl = fcntl.fcntl(fd, fcntl.F_GETFL)
+ fcntl.fcntl(fd, fcntl.F_SETFL, fl | os.O_NONBLOCK)
+ rin.append(pipe.stdout)
+ if pipe.stderr is not None:
+ fd = pipe.stderr.fileno()
+ fl = fcntl.fcntl(fd, fcntl.F_GETFL)
+ fcntl.fcntl(fd, fcntl.F_SETFL, fl | os.O_NONBLOCK)
+ rin.append(pipe.stderr)
+
while pipe.poll() is None:
- if pipe.stdout is not None:
+ rlist = rin
+ try:
+ r,w,e = select.select (rlist, [], [])
+ except OSError, e:
+ if e.errno != errno.EINTR:
+ raise
+
+ if pipe.stdout in r:
data = pipe.stdout.read(bufsize)
if data is not None:
outdata.append(data)
log.write(data)
+ log.flush()
- if pipe.stderr is not None:
+ if pipe.stderr in r:
data = pipe.stderr.read(bufsize)
if data is not None:
errdata.append(data)
log.write(data)
+ log.flush()
return ''.join(outdata), ''.join(errdata)
def run(cmd, input=None, log=None, **options):
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 02/18] knotty: Extend knotty to support "real time" log tail for tasks
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
2012-06-08 13:41 ` [PATCH v4 01/18] process.py, build.py: Fix log truncation problems with flush() Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 03/18] knotty: Add the ability to dynamically select loglevel from stdin Jason Wessel
` (15 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
This patch provides the capability to knotty to have a higher log
level where the individual tasks logs can be multiplexed to stdio.
This is highly desirable for several different use cases.
1) You can visually spot "hot spots" in building parallel tasks
2) The relative order an sequence of events can be determined in
difficult to diagnose inter-related library problems.
3) It provides a nice way to see what all is actually going on and
getting logged by various tasks such as a kernel compile where you
might want to actually look at the output in a build over build
scenario.
4) There are most certainly other aspects where this is useful.
The primary control is to set the BB_RT_LOGLEVEL = "2" in your
local.conf. Additional capabilities will follow in successive
patches. When parallel tasks are enabled, the logs will be
multiplexed and prefixed with the PID of the task. In the
non-parallel case there is no PID prefix as everything is linear.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
lib/bb/build.py | 4 ++
lib/bb/ui/crumbs/multilogtail.py | 135 ++++++++++++++++++++++++++++++++++++++
lib/bb/ui/knotty.py | 46 +++++++++++++
3 files changed, 185 insertions(+)
create mode 100644 lib/bb/ui/crumbs/multilogtail.py
diff --git a/lib/bb/build.py b/lib/bb/build.py
index da4c911..fb61b00 100644
--- a/lib/bb/build.py
+++ b/lib/bb/build.py
@@ -88,6 +88,10 @@ class TaskBase(event.Event):
class TaskStarted(TaskBase):
"""Task execution started"""
+ def __init__(self, t, d):
+ self.logfile = d.getVar("BB_LOGFILE", True) or "";
+ super(TaskStarted, self).__init__(t, d)
+
class TaskSucceeded(TaskBase):
"""Task execution completed"""
diff --git a/lib/bb/ui/crumbs/multilogtail.py b/lib/bb/ui/crumbs/multilogtail.py
new file mode 100644
index 0000000..da8ae68
--- /dev/null
+++ b/lib/bb/ui/crumbs/multilogtail.py
@@ -0,0 +1,135 @@
+#
+# Multipile Log Tail for real time logging
+#
+# Copyright (C) 2012 Wind River Systems, Inc.
+#
+# Authored by Jason Wessel <jason.wessel@windriver.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+import re
+import time
+import sys
+
+class LogTail(object):
+ """Tail a single log file"""
+
+ def __init__(self):
+ self.file = None
+ self.offset = 0
+ self.filterLines = True
+ self.limit = 8192
+ self.cnt = 0
+ self.storedLine = ""
+
+ def openLog(self, filename, filterLines=True, limit=8192):
+ try:
+ self.file = open(filename, 'rb')
+ self.limit = limit
+ self.filterLines = filterLines
+ except:
+ return False
+ return True
+
+ def getLeftLine(self):
+ (left, right) = self.storedLine.split('\r',1)
+ self.storedLine = right
+ self.cnt = len(right)
+ return left
+
+ def getLine(self):
+ if self.cnt > 0 and '\r' in self.storedLine:
+ return self.getLeftLine()
+ while 1:
+ self.offset = self.file.tell()
+ line = self.file.readline()
+ if line:
+ if self.filterLines:
+ line = line.replace('\r\n','\r');
+ line = line.replace('\n','\r');
+ self.storedLine += line
+ self.cnt = len(self.storedLine)
+ if (self.cnt > self.limit):
+ if not '\r' in self.storedLine:
+ self.storedLine += '\r'
+ return self.getLeftLine()
+ if '\r' in self.storedLine:
+ return self.getLeftLine()
+ continue
+ else:
+ return line
+ self.file.seek(self.offset)
+ return None
+
+ def closeLog(self):
+ if self.file:
+ self.file.close()
+ self.file = None
+
+ def __del__(self):
+ self.closeLog()
+
+class MultiLogTail(object):
+ """Tail mulitple log files"""
+
+ def __init__(self, printPid=True):
+ self.filelist = []
+ self.trackPid = []
+ self.printPid = printPid
+
+ def openLog(self, filename, pid=0, limit=8192):
+ logtail = LogTail()
+ if logtail.openLog(filename, True, limit):
+ self.filelist.append((logtail, filename, pid))
+ else:
+ if pid > 0:
+ self.trackPid.append(pid)
+
+ def displayLogs(self):
+ for (logtail, filename, pid) in self.filelist:
+ while 1:
+ output = logtail.getLine()
+ if output == None:
+ break
+ if self.printPid:
+ print " " + str(pid) + ":" + output
+ else:
+ print output
+
+ def closeLogs(self):
+ while 1:
+ try:
+ (logtail, filename, pid) = self.filelist.pop()
+ logtail.closeLog()
+ except:
+ break
+
+ def closeLog(self, rm_filename):
+ for (logtail, filename, pid) in self.filelist:
+ if filename == rm_filename:
+ self.filelist.remove((logtail, filename, pid))
+ logtail.closeLog()
+ return
+ raise Exception("File %s not in the list" % rm_filename)
+
+ def closeLogPid(self, rm_pid):
+ for (logtail, filename, pid) in self.filelist:
+ if pid == rm_pid:
+ self.filelist.remove((logtail, filename, pid))
+ logtail.closeLog()
+ return
+ if rm_pid in self.trackPid:
+ self.trackPid.remove(rm_pid)
+ return
+ print "ERROR: PID %s not in the list" % rm_pid
diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index 0167416..fd86ed5 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -22,11 +22,15 @@ from __future__ import division
import os
import sys
+import select
+import termios
+import copy
import xmlrpclib
import logging
import progressbar
import bb.msg
from bb.ui import uihelper
+from bb.ui.crumbs.multilogtail import MultiLogTail
logger = logging.getLogger("BitBake")
interactive = sys.stdout.isatty()
@@ -104,6 +108,25 @@ class TerminalFilter(object):
def finish(self):
return
+class RtLogLevel:
+ def __init__(self, mlt):
+ self.displaytail = False
+ self.mlt = mlt
+
+ def displayLogs(self):
+ if self.displaytail:
+ self.mlt.displayLogs()
+
+ def setLevel(self, input, verbose):
+ if input == "1":
+ if verbose:
+ print "NOTE: Turning off real time log tail"
+ self.displaytail = False
+ elif input == "2":
+ if verbose:
+ print "NOTE: Turning on real time log tail"
+ self.displaytail = True
+
def main(server, eventHandler, tf = TerminalFilter):
# Get values of variables which control our output
@@ -111,11 +134,22 @@ def main(server, eventHandler, tf = TerminalFilter):
loglines = server.runCommand(["getVariable", "BBINCLUDELOGS_LINES"])
consolelogfile = server.runCommand(["getVariable", "BB_CONSOLELOG"])
+ # MultiTail affected varialbles
+ numthreads = server.runCommand(["getVariable", "BB_NUMBER_THREADS"])
+ if numthreads is None or numthreads == "1":
+ mlt = MultiLogTail(False)
+ else:
+ mlt = MultiLogTail(True)
+
helper = uihelper.BBUIHelper()
console = logging.StreamHandler(sys.stdout)
format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
bb.msg.addDefaultlogFilter(console)
+ rtloglevel = RtLogLevel(mlt)
+ bb_rt_loglevel = server.runCommand(["getVariable", "BB_RT_LOGLEVEL"])
+ if bb_rt_loglevel and bb_rt_loglevel != "":
+ rtloglevel.setLevel(bb_rt_loglevel, False)
console.setFormatter(format)
logger.addHandler(console)
if consolelogfile:
@@ -155,11 +189,23 @@ def main(server, eventHandler, tf = TerminalFilter):
try:
termfilter.updateFooter()
event = eventHandler.waitEvent(0.25)
+ # Always try printing any accumulated log files first
+ rtloglevel.displayLogs()
if event is None:
if main.shutdown > 1:
break
continue
helper.eventHandler(event)
+ if isinstance(event, bb.build.TaskStarted):
+ mlt.openLog(event.logfile, event.pid)
+ rtloglevel.displayLogs()
+
+ if isinstance(event, bb.build.TaskSucceeded):
+ mlt.closeLogPid(event.pid)
+
+ if isinstance(event, bb.build.TaskFailed):
+ mlt.closeLogPid(event.pid)
+
if isinstance(event, bb.runqueue.runQueueExitWait):
if not main.shutdown:
main.shutdown = 1
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 03/18] knotty: Add the ability to dynamically select loglevel from stdin
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
2012-06-08 13:41 ` [PATCH v4 01/18] process.py, build.py: Fix log truncation problems with flush() Jason Wessel
2012-06-08 13:41 ` [PATCH v4 02/18] knotty: Extend knotty to support "real time" log tail for tasks Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 04/18] msg.py, knotty.py: Allow dynamic toggle of the debug log level Jason Wessel
` (14 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
If stdin is a controlling terminal, make it possible to dynamically
select a log level with the keys 1 and 2.
1 = normal bitbake knotty logs
2 = realtime multiple tail of task logs
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
lib/bb/ui/knotty.py | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)
diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index fd86ed5..f72f0ad 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -108,6 +108,32 @@ class TerminalFilter(object):
def finish(self):
return
+class StdinMgr:
+ def __init__(self):
+ self.stdinbackup = None
+ self.fd = None
+ if sys.stdin.isatty():
+ self.fd = sys.stdin.fileno()
+ self.stdinbackup = termios.tcgetattr(self.fd)
+ new = termios.tcgetattr(self.fd)
+ new[3] = new[3] & ~termios.ICANON & ~termios.ECHO
+ termios.tcsetattr(self.fd, termios.TCSANOW, new)
+
+ def poll(self):
+ if not self.stdinbackup:
+ return False
+ return select.select([sys.stdin], [], [], 0) == ([sys.stdin], [], [])
+
+ def restore(self):
+ if self.stdinbackup:
+ termios.tcsetattr(self.fd, termios.TCSADRAIN, self.stdinbackup)
+ self.stdinbackup = None
+ # Force echo back on "just in case" something went haywire with an exception
+ if sys.stdin.isatty():
+ new = termios.tcgetattr(self.fd)
+ new[3] = new[3] | termios.ECHO
+ termios.tcsetattr(self.fd, termios.TCSANOW, new)
+
class RtLogLevel:
def __init__(self, mlt):
self.displaytail = False
@@ -184,11 +210,15 @@ def main(server, eventHandler, tf = TerminalFilter):
taskfailures = []
termfilter = tf(main, helper, console, format)
+ stdin_mgr = StdinMgr()
while True:
try:
termfilter.updateFooter()
event = eventHandler.waitEvent(0.25)
+ if stdin_mgr.poll():
+ keyinput = sys.stdin.read(1)
+ rtloglevel.setLevel(keyinput, True)
# Always try printing any accumulated log files first
rtloglevel.displayLogs()
if event is None:
@@ -364,6 +394,7 @@ def main(server, eventHandler, tf = TerminalFilter):
if ioerror.args[0] == 4:
pass
except KeyboardInterrupt:
+ stdin_mgr.restore()
termfilter.clearFooter()
if main.shutdown == 1:
print("\nSecond Keyboard Interrupt, stopping...\n")
@@ -375,6 +406,7 @@ def main(server, eventHandler, tf = TerminalFilter):
main.shutdown = main.shutdown + 1
pass
+ stdin_mgr.restore()
summary = ""
if taskfailures:
summary += pluralise("\nSummary: %s task failed:",
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 04/18] msg.py, knotty.py: Allow dynamic toggle of the debug log level
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (2 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 03/18] knotty: Add the ability to dynamically select loglevel from stdin Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 05/18] knotty.py: Add the ability to spawn screen directly from knotty with OE_TERMINAL=screen_inline Jason Wessel
` (13 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
If stdin is a controlling tty, make it possible to have
log levels 1-4 where you can press a key to dynamically
change the log level, while bitbake is running to see
what is going on.
1 = default log level
2 = multiple log tail of bitbake tasks
3 = bit bake debug messages
4 = bitbake debug messages + multiple lot tail of bitbake tasks
In order to make this possible a slight modification to
the msg logfilter was needed to make it possible to get
an object so as to change the filter log level.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
lib/bb/msg.py | 9 ++++++++-
lib/bb/ui/knotty.py | 21 ++++++++++++++++++---
2 files changed, 26 insertions(+), 4 deletions(-)
diff --git a/lib/bb/msg.py b/lib/bb/msg.py
index 9b39325..4b2f90e 100644
--- a/lib/bb/msg.py
+++ b/lib/bb/msg.py
@@ -86,6 +86,13 @@ class BBLogFilter(object):
handler.setLevel(loglevel)
handler.addFilter(self)
+ def setFiltLevel(self, handler, level):
+ self.stdlevel = level
+ handler.setLevel(level)
+
+ def getFiltLevel(self):
+ return self.stdlevel
+
def filter(self, record):
if record.levelno >= self.stdlevel:
return True
@@ -131,7 +138,7 @@ def addDefaultlogFilter(handler):
dlevel = len(tuple(iterator))
debug_domains["BitBake.%s" % domainarg] = logging.DEBUG - dlevel + 1
- BBLogFilter(handler, level, debug_domains)
+ return BBLogFilter(handler, level, debug_domains)
#
# Message handling functions
diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index f72f0ad..7a7a1b2 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -135,8 +135,11 @@ class StdinMgr:
termios.tcsetattr(self.fd, termios.TCSANOW, new)
class RtLogLevel:
- def __init__(self, mlt):
+ def __init__(self, handler, logfilter, mlt):
self.displaytail = False
+ self.handler = handler
+ self.logfilter = logfilter
+ self.defaultLevel = logfilter.getFiltLevel()
self.mlt = mlt
def displayLogs(self):
@@ -147,10 +150,22 @@ class RtLogLevel:
if input == "1":
if verbose:
print "NOTE: Turning off real time log tail"
+ self.logfilter.setFiltLevel(self.handler, self.defaultLevel)
self.displaytail = False
elif input == "2":
if verbose:
print "NOTE: Turning on real time log tail"
+ self.logfilter.setFiltLevel(self.handler, self.defaultLevel)
+ self.displaytail = True
+ elif input == "3":
+ if verbose:
+ print "NOTE: Turning on DEBUG logging"
+ self.logfilter.setFiltLevel(self.handler, logging.DEBUG)
+ self.displaytail = False
+ elif input == "4":
+ if verbose:
+ print "NOTE: Turning on DEBUG logging + real time log tail"
+ self.logfilter.setFiltLevel(self.handler, logging.DEBUG)
self.displaytail = True
def main(server, eventHandler, tf = TerminalFilter):
@@ -171,8 +186,8 @@ def main(server, eventHandler, tf = TerminalFilter):
console = logging.StreamHandler(sys.stdout)
format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
- bb.msg.addDefaultlogFilter(console)
- rtloglevel = RtLogLevel(mlt)
+ logfilter = bb.msg.addDefaultlogFilter(console)
+ rtloglevel = RtLogLevel(console, logfilter, mlt)
bb_rt_loglevel = server.runCommand(["getVariable", "BB_RT_LOGLEVEL"])
if bb_rt_loglevel and bb_rt_loglevel != "":
rtloglevel.setLevel(bb_rt_loglevel, False)
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 05/18] knotty.py: Add the ability to spawn screen directly from knotty with OE_TERMINAL=screen_inline
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (3 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 04/18] msg.py, knotty.py: Allow dynamic toggle of the debug log level Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 06/18] fetch2: Fix missing output from stderr in fetcher logs Jason Wessel
` (12 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
The oe-core has a change pending to add the screen_inline class.
This change allows the simple use case of directly using a devshell
from the controlling terminal that started bitbake which is a
significant convience for some use cases.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
lib/bb/ui/knotty.py | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index 7a7a1b2..b4125f6 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -174,6 +174,10 @@ def main(server, eventHandler, tf = TerminalFilter):
includelogs = server.runCommand(["getVariable", "BBINCLUDELOGS"])
loglines = server.runCommand(["getVariable", "BBINCLUDELOGS_LINES"])
consolelogfile = server.runCommand(["getVariable", "BB_CONSOLELOG"])
+ if sys.stdin.isatty() and sys.stdout.isatty():
+ runscreen = True
+ else:
+ runscreen = False
# MultiTail affected varialbles
numthreads = server.runCommand(["getVariable", "BB_NUMBER_THREADS"])
@@ -260,6 +264,12 @@ def main(server, eventHandler, tf = TerminalFilter):
errors = errors + 1
return_value = 1
elif event.levelno == format.WARNING:
+ msg = event.getMessage()
+ if runscreen and event.name == "BitBake.OE.Terminal" and msg.startswith("Screen Connect Command: "):
+ msg = msg[24:]
+ print "Starting terminal with: %s" % msg
+ os.system(msg)
+ continue
warnings = warnings + 1
# For "normal" logging conditions, don't show note logs from tasks
# but do show them if the user has changed the default log level to
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 06/18] fetch2: Fix missing output from stderr in fetcher logs
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (4 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 05/18] knotty.py: Add the ability to spawn screen directly from knotty with OE_TERMINAL=screen_inline Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 07/18] fetch2/git.py: Use local download dir, when using an external read-only GITDIR Jason Wessel
` (11 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
There are actually two problems to fix
1) The exception for bb.process.ExecutionError must be processed first
because it is a derived from the bb.process.CmdError class and
we never reach the ExecutionError otherwise.
2) The stderr needs to be printed as well as stdout to determine
the root cause of a fetch failure.
The example I have is that I got a log that looked like:
--
ERROR: Function failed: Network access disabled through
BB_NO_NETWORK but access requested with command
/usr/bin/env wget -t 5 -nv --passive-ftp --no-check-certificate -P
/localds 'http://downloads.yoctoproject.org/[...CLIPPED...] url None)
--
That really didn't tell me much, but with this patch I get error above
plus the following:
--
STDERR: /net/[...CLIPPED...]kernel-tools.git: Read-only file system
--
Having the ability to see that the build was trying to write to a
read-only file system was a good clue that the fetcher is broken in
some other way to be fixed in a follow on patch, but let us fix the
logging problem first.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
lib/bb/fetch2/__init__.py | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index 3391e6a..ba562a8 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -421,11 +421,10 @@ def runfetchcmd(cmd, d, quiet = False, cleanup = []):
success = True
except bb.process.NotFoundError as e:
error_message = "Fetch command %s" % (e.command)
+ except bb.process.ExecutionError as e:
+ error_message = "Fetch command %s failed with exit code %s, output:\nSTDOUT: %s\nSTDERR: %s" % (e.command, e.exitcode, e.stdout, e.stderr)
except bb.process.CmdError as e:
error_message = "Fetch command %s could not be run:\n%s" % (e.command, e.msg)
- except bb.process.ExecutionError as e:
- error_message = "Fetch command %s failed with exit code %s, output:\n%s" % (e.command, e.exitcode, e.stderr)
-
if not success:
for f in cleanup:
try:
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 07/18] fetch2/git.py: Use local download dir, when using an external read-only GITDIR
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (5 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 06/18] fetch2: Fix missing output from stderr in fetcher logs Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 08/18] knotty.py, knotty2, py: Flush and update footer on dynamic log level change Jason Wessel
` (10 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
When building the yocto kernel or yocto kernel tools, if the GITDIR is
on a read-only server that is shared and updated by via some cron
jobs, the fetch2 should not try to write to that area. The end result
will be odd failures if you have download turned off, or extra
needless downloading if you have download turned on.
The fetch2 should prefer to create mirror the mirror clones in the
local "downloads" directory in the work space.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
lib/bb/fetch2/git.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
index bcc0da5..59d95cb 100644
--- a/lib/bb/fetch2/git.py
+++ b/lib/bb/fetch2/git.py
@@ -136,8 +136,11 @@ class Git(FetchMethod):
gitsrcname = gitsrcname + '_' + ud.revisions[name]
ud.mirrortarball = 'git2_%s.tar.gz' % (gitsrcname)
ud.fullmirror = os.path.join(data.getVar("DL_DIR", d, True), ud.mirrortarball)
- ud.clonedir = os.path.join(data.expand('${GITDIR}', d), gitsrcname)
+ ud.gitdir = os.path.join(data.expand('${GITDIR}', d), gitsrcname)
+ ud.clonedir = os.path.join(data.getVar("DL_DIR", d, True), gitsrcname)
+ if os.path.exists(ud.gitdir):
+ ud.clonedir = ud.gitdir
ud.localfile = ud.clonedir
def localpath(self, url, ud, d):
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 08/18] knotty.py, knotty2, py: Flush and update footer on dynamic log level change
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (6 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 07/18] fetch2/git.py: Use local download dir, when using an external read-only GITDIR Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 09/18] knotty2.py: Fix knotty2 to report something other than 0 of 0 for setscene tasks Jason Wessel
` (9 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
This patch adds a call to update the task list after changing log
modes dynamically such that you can immediately see the remaining
tasks. After adding this functionality, it became obvious there was a
problem flushing data loaded into stdout because the task list would
not appear immediately. Doing something like "bitbake IMAGE | tee
/tmp/log" is where the flush problem manifests itself the most.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
lib/bb/ui/knotty.py | 14 ++++++++++++--
lib/bb/ui/knotty2.py | 4 ++++
2 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index b4125f6..d7b1e19 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -82,9 +82,16 @@ class TerminalFilter(object):
def clearFooter(self):
return
+ def updateFooterForce(self):
+ self.printFooter(True)
+
def updateFooter(self):
- if not main.shutdown or not self.helper.needUpdate:
- return
+ self.printFooter(False)
+
+ def printFooter(self, force_update):
+ if not force_update:
+ if not main.shutdown or not self.helper.needUpdate:
+ return
activetasks = self.helper.running_tasks
runningpids = self.helper.running_pids
@@ -238,6 +245,9 @@ def main(server, eventHandler, tf = TerminalFilter):
if stdin_mgr.poll():
keyinput = sys.stdin.read(1)
rtloglevel.setLevel(keyinput, True)
+ termfilter.updateFooterForce()
+ sys.stdout.flush()
+
# Always try printing any accumulated log files first
rtloglevel.displayLogs()
if event is None:
diff --git a/lib/bb/ui/knotty2.py b/lib/bb/ui/knotty2.py
index aa6a408..01693e9 100644
--- a/lib/bb/ui/knotty2.py
+++ b/lib/bb/ui/knotty2.py
@@ -73,6 +73,10 @@ class TerminalFilter2(object):
sys.stdout.write(self.curses.tparm(self.ed))
self.footer_present = False
+ def updateFooterForce(self):
+ self.footer_present = False
+ self.updateFooter()
+
def updateFooter(self):
if not self.cuu:
return
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 09/18] knotty2.py: Fix knotty2 to report something other than 0 of 0 for setscene tasks
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (7 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 08/18] knotty.py, knotty2, py: Flush and update footer on dynamic log level change Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 10/18] knotty2: Properly adjust for lines longer than terminal size Jason Wessel
` (8 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
Any time knotty2 was running through the setscene tasks it was always
reporting the top line as follows for a core-image-minimal build.
Currently 7 running tasks (0 of 0):
With this patch the number of SetScene tasks are tracked and reported
with knotty such that it nows looks something like:
Currently 7 running SetScene tasks (342 of 479):
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
lib/bb/ui/knotty2.py | 5 ++++-
lib/bb/ui/uihelper.py | 5 +++++
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/lib/bb/ui/knotty2.py b/lib/bb/ui/knotty2.py
index 01693e9..9e3619b 100644
--- a/lib/bb/ui/knotty2.py
+++ b/lib/bb/ui/knotty2.py
@@ -97,7 +97,10 @@ class TerminalFilter2(object):
if self.main.shutdown:
print("Waiting for %s running tasks to finish:" % len(activetasks))
else:
- print("Currently %s running tasks (%s of %s):" % (len(activetasks), self.helper.tasknumber_current, self.helper.tasknumber_total))
+ if self.helper.tasknumber_current == 0:
+ print("Currently %s running SetScene tasks (%s of %s):" % (len(activetasks), self.helper.sc_tasknumber_current, self.helper.sc_tasknumber_total))
+ else:
+ print("Currently %s running tasks (%s of %s):" % (len(activetasks), self.helper.tasknumber_current, self.helper.tasknumber_total))
for tasknum, task in enumerate(tasks):
print("%s: %s" % (tasknum, task))
lines = lines + 1
diff --git a/lib/bb/ui/uihelper.py b/lib/bb/ui/uihelper.py
index 2c78695..84af5a3 100644
--- a/lib/bb/ui/uihelper.py
+++ b/lib/bb/ui/uihelper.py
@@ -28,6 +28,8 @@ class BBUIHelper:
self.failed_tasks = []
self.tasknumber_current = 0
self.tasknumber_total = 0
+ self.sc_tasknumber_current = 0
+ self.sc_tasknumber_current = 0
def eventHandler(self, event):
if isinstance(event, bb.build.TaskStarted):
@@ -48,6 +50,9 @@ class BBUIHelper:
self.running_pids.remove(event.pid)
self.failed_tasks.append( { 'title' : "%s %s" % (event._package, event._task)})
self.needUpdate = True
+ if isinstance(event, bb.runqueue.sceneQueueTaskStarted):
+ self.sc_tasknumber_current = event.stats.completed + event.stats.active + event.stats.failed + 1
+ self.sc_tasknumber_total = event.stats.total
if isinstance(event, bb.runqueue.runQueueTaskStarted):
self.tasknumber_current = event.stats.completed + event.stats.active + event.stats.failed + 1
self.tasknumber_total = event.stats.total
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 10/18] knotty2: Properly adjust for lines longer than terminal size
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (8 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 09/18] knotty2.py: Fix knotty2 to report something other than 0 of 0 for setscene tasks Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 11/18] knotty: Merge knotty2 functionality into knotty Jason Wessel
` (7 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
knotty2 does not handle being resized nor does it work properly
if a task line is longer than the terminal size.
The filter handler relies on the sigwinch to properly
handle a terminal resize. The cache and parse tasks will
try to take over the sigwinch handler, so it must be restored
when either of these tasks completes.
---
lib/bb/ui/knotty.py | 5 +++++
lib/bb/ui/knotty2.py | 42 +++++++++++++++++++++++++++++++++++++++---
2 files changed, 44 insertions(+), 3 deletions(-)
diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index d7b1e19..58fb8f1 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -82,6 +82,9 @@ class TerminalFilter(object):
def clearFooter(self):
return
+ def sigUpdate(self):
+ return
+
def updateFooterForce(self):
self.printFooter(True)
@@ -332,6 +335,7 @@ def main(server, eventHandler, tf = TerminalFilter):
parseprogress.finish()
print(("Parsing of %d .bb files complete (%d cached, %d parsed). %d targets, %d skipped, %d masked, %d errors."
% ( event.total, event.cached, event.parsed, event.virtuals, event.skipped, event.masked, event.errors)))
+ termfilter.sigUpdate()
continue
if isinstance(event, bb.event.CacheLoadStarted):
@@ -343,6 +347,7 @@ def main(server, eventHandler, tf = TerminalFilter):
if isinstance(event, bb.event.CacheLoadCompleted):
cacheprogress.finish()
print("Loaded %d entries from dependency cache." % event.num_entries)
+ termfilter.sigUpdate()
continue
if isinstance(event, bb.command.CommandFailed):
diff --git a/lib/bb/ui/knotty2.py b/lib/bb/ui/knotty2.py
index 9e3619b..cfa4bf7 100644
--- a/lib/bb/ui/knotty2.py
+++ b/lib/bb/ui/knotty2.py
@@ -35,6 +35,39 @@ class InteractConsoleLogFilter(logging.Filter):
return True
class TerminalFilter2(object):
+ columns = 80
+
+ def sigwinch_handle(self, sig, data):
+ self.columns = self.getTerminalColumns()
+
+ def sigUpdate(self):
+ import signal
+ if self.interactive and self.cuu:
+ signal.signal(signal.SIGWINCH, self.sigwinch_handle)
+
+ def getTerminalColumns(self):
+ def ioctl_GWINSZ(fd):
+ try:
+ import fcntl, termios, struct, os
+ cr = struct.unpack('hh', fcntl.ioctl(fd, termios.TIOCGWINSZ, '1234'))
+ except:
+ return None
+ return cr
+ cr = ioctl_GWINSZ(0) or ioctl_GWINSZ(1) or ioctl_GWINSZ(2)
+ if not cr:
+ try:
+ fd = os.open(os.ctermid(), os.O_RDONLY)
+ cr = ioctl_GWINSZ(fd)
+ os.close(fd)
+ except:
+ pass
+ if not cr:
+ try:
+ cr = (env['LINES'], env['COLUMNS'])
+ except:
+ cr = (25, 80)
+ return cr[1]
+
def __init__(self, main, helper, console, format):
self.main = main
self.helper = helper
@@ -62,7 +95,9 @@ class TerminalFilter2(object):
self.ed = curses.tigetstr("ed")
if self.ed:
self.cuu = curses.tigetstr("cuu")
- except:
+ self.columns = self.getTerminalColumns()
+ self.sigUpdate()
+ except Exception:
self.cuu = None
console.addFilter(InteractConsoleLogFilter(self, format))
@@ -102,8 +137,9 @@ class TerminalFilter2(object):
else:
print("Currently %s running tasks (%s of %s):" % (len(activetasks), self.helper.tasknumber_current, self.helper.tasknumber_total))
for tasknum, task in enumerate(tasks):
- print("%s: %s" % (tasknum, task))
- lines = lines + 1
+ content = "%s: %s" % (tasknum, task)
+ print content
+ lines = lines + 1 + int(len(content) / (self.columns + 1))
self.footer_present = lines
self.lastpids = runningpids[:]
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 11/18] knotty: Merge knotty2 functionality into knotty
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (9 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 10/18] knotty2: Properly adjust for lines longer than terminal size Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 12/18] knotty: Add ability to dynamically toggle log location Jason Wessel
` (6 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
The knotty 2 functionality can be activated in realtime
by pressing "t" for top mode or reverted with "0" for
normal mode.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
lib/bb/ui/knotty.py | 172 ++++++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 150 insertions(+), 22 deletions(-)
diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index 58fb8f1..765ad27 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -74,50 +74,156 @@ def pluralise(singular, plural, qty):
else:
return plural % qty
+class InteractConsoleLogFilter(logging.Filter):
+ def __init__(self, tf, format):
+ self.tf = tf
+ self.format = format
+
+ def filter(self, record):
+ if record.levelno == self.format.NOTE and (record.msg.startswith("Running") or record.msg.startswith("package ")):
+ return False
+ self.tf.clearFooter()
+ return True
+
class TerminalFilter(object):
def __init__(self, main, helper, console, format):
+ import curses
self.main = main
+ self.console = console
self.helper = helper
+ self.topMode = False
+ self.cuu = None
+ self.interactive = sys.stdout.isatty()
+ self.footer_present = False
+ self.lastpids = []
+ self.curses = curses
+ self.confilter = InteractConsoleLogFilter(self, format)
+ self.curses_init = False
+ self.columns = 80
+
+ def cursesInit(self):
+ if self.curses_init:
+ return
+ self.curses_init = True
+ if self.interactive:
+ try:
+ self.curses.setupterm()
+ self.ed = self.curses.tigetstr("ed")
+ if self.ed:
+ self.cuu = self.curses.tigetstr("cuu")
+ self.columns = self.getTerminalColumns()
+ self.sigUpdate()
+ except Exception as e:
+ self.cuu = None
+
+ def setTopMode(self):
+ if self.topMode:
+ return
+ self.cursesInit()
+ self.topMode = True
+ self.console.addFilter(self.confilter)
+
+ def setNormalMode(self):
+ if not self.topMode:
+ return
+ self.topMode = False
+ try:
+ self.console.removeFilter(self.confilter)
+ except:
+ pass
def clearFooter(self):
- return
+ if not self.topMode:
+ return
+ if self.footer_present:
+ lines = self.footer_present
+ sys.stdout.write(self.curses.tparm(self.cuu, lines))
+ sys.stdout.write(self.curses.tparm(self.ed))
+ self.footer_present = False
- def sigUpdate(self):
- return
def updateFooterForce(self):
+ self.footer_present = False
self.printFooter(True)
def updateFooter(self):
self.printFooter(False)
def printFooter(self, force_update):
- if not force_update:
- if not main.shutdown or not self.helper.needUpdate:
- return
-
activetasks = self.helper.running_tasks
+ failedtasks = self.helper.failed_tasks
runningpids = self.helper.running_pids
+ if self.topMode:
+ if not self.cuu:
+ return
+ if self.footer_present and (self.lastpids == runningpids):
+ return
+ if self.footer_present:
+ self.clearFooter()
+ else:
+ if not force_update:
+ if not main.shutdown or not self.helper.needUpdate:
+ return
+ if len(runningpids) == 0:
+ return
+ self.helper.getTasks()
- if len(runningpids) == 0:
+ if not activetasks:
return
-
- self.helper.getTasks()
-
+ lines = 1
tasks = []
for t in runningpids:
tasks.append("%s (pid %s)" % (activetasks[t]["title"], t))
- if main.shutdown:
+ if self.main.shutdown:
print("Waiting for %s running tasks to finish:" % len(activetasks))
else:
- print("Currently %s running tasks (%s of %s):" % (len(activetasks), self.helper.tasknumber_current, self.helper.tasknumber_total))
+ if self.helper.tasknumber_current == 0:
+ print("Currently %s running SetScene tasks (%s of %s):" % (len(activetasks), self.helper.sc_tasknumber_current, self.helper.sc_tasknumber_total))
+ else:
+ print("Currently %s running tasks (%s of %s):" % (len(activetasks), self.helper.tasknumber_current, self.helper.tasknumber_total))
for tasknum, task in enumerate(tasks):
- print("%s: %s" % (tasknum, task))
+ content = "%s: %s" % (tasknum, task)
+ print content
+ lines = lines + 1 + int(len(content) / (self.columns + 1))
+ self.footer_present = lines
+ self.lastpids = runningpids[:]
def finish(self):
return
+ def sigwinch_handle(self, sig, data):
+ self.columns = self.getTerminalColumns()
+
+ def sigUpdate(self):
+ import signal
+ if self.interactive and self.cuu:
+ signal.signal(signal.SIGWINCH, self.sigwinch_handle)
+
+ def getTerminalColumns(self):
+ def ioctl_GWINSZ(fd):
+ try:
+ import fcntl, termios, struct, os
+ cr = struct.unpack('hh', fcntl.ioctl(fd, termios.TIOCGWINSZ, '1234'))
+ except:
+ return None
+ return cr
+ cr = ioctl_GWINSZ(0) or ioctl_GWINSZ(1) or ioctl_GWINSZ(2)
+ if not cr:
+ try:
+ fd = os.open(os.ctermid(), os.O_RDONLY)
+ cr = ioctl_GWINSZ(fd)
+ os.close(fd)
+ except:
+ pass
+ if not cr:
+ try:
+ cr = (env['LINES'], env['COLUMNS'])
+ except:
+ cr = (25, 80)
+ return cr[1]
+
+
class StdinMgr:
def __init__(self):
self.stdinbackup = None
@@ -145,23 +251,26 @@ class StdinMgr:
termios.tcsetattr(self.fd, termios.TCSANOW, new)
class RtLogLevel:
- def __init__(self, handler, logfilter, mlt):
+ def __init__(self, handler, logfilter, mlt, tf):
self.displaytail = False
self.handler = handler
self.logfilter = logfilter
self.defaultLevel = logfilter.getFiltLevel()
self.mlt = mlt
+ self.tf = tf
def displayLogs(self):
if self.displaytail:
self.mlt.displayLogs()
def setLevel(self, input, verbose):
- if input == "1":
+ if input == "1" or input == "0":
if verbose:
print "NOTE: Turning off real time log tail"
self.logfilter.setFiltLevel(self.handler, self.defaultLevel)
self.displaytail = False
+ if input == "0" and isinstance(self.tf, TerminalFilter):
+ self.tf.setNormalMode()
elif input == "2":
if verbose:
print "NOTE: Turning on real time log tail"
@@ -177,6 +286,23 @@ class RtLogLevel:
print "NOTE: Turning on DEBUG logging + real time log tail"
self.logfilter.setFiltLevel(self.handler, logging.DEBUG)
self.displaytail = True
+ elif input == "t":
+ if verbose:
+ print "NOTE: Activing task \"top\" mode"
+ if isinstance(self.tf, TerminalFilter):
+ self.tf.setTopMode()
+ elif input == "h":
+ print "============================================="
+ print "Interaction help commands:"
+ print " 0 - default with default logs"
+ print " 1 - turn off real time log tail"
+ print " 2 - turn on real time log tail"
+ print " 3 - turn on debug logging"
+ print " 4 - turn on debug logging and real time log tail"
+ print " t - Display tasks in \"top\" mode (formerly knotty2 mode)"
+ print " h - display commands"
+ return False
+ return True
def main(server, eventHandler, tf = TerminalFilter):
@@ -201,10 +327,6 @@ def main(server, eventHandler, tf = TerminalFilter):
console = logging.StreamHandler(sys.stdout)
format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
logfilter = bb.msg.addDefaultlogFilter(console)
- rtloglevel = RtLogLevel(console, logfilter, mlt)
- bb_rt_loglevel = server.runCommand(["getVariable", "BB_RT_LOGLEVEL"])
- if bb_rt_loglevel and bb_rt_loglevel != "":
- rtloglevel.setLevel(bb_rt_loglevel, False)
console.setFormatter(format)
logger.addHandler(console)
if consolelogfile:
@@ -241,14 +363,20 @@ def main(server, eventHandler, tf = TerminalFilter):
termfilter = tf(main, helper, console, format)
stdin_mgr = StdinMgr()
+ rtloglevel = RtLogLevel(console, logfilter, mlt, termfilter)
+ bb_rt_loglevel = server.runCommand(["getVariable", "BB_RT_LOGLEVEL"])
+ if bb_rt_loglevel and bb_rt_loglevel != "":
+ for inputkey in bb_rt_loglevel:
+ rtloglevel.setLevel(inputkey, False)
while True:
try:
termfilter.updateFooter()
event = eventHandler.waitEvent(0.25)
if stdin_mgr.poll():
keyinput = sys.stdin.read(1)
- rtloglevel.setLevel(keyinput, True)
- termfilter.updateFooterForce()
+ termfilter.clearFooter()
+ if (rtloglevel.setLevel(keyinput, True)):
+ termfilter.updateFooterForce()
sys.stdout.flush()
# Always try printing any accumulated log files first
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 12/18] knotty: Add ability to dynamically toggle log location
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (10 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 11/18] knotty: Merge knotty2 functionality into knotty Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 13/18] bitbake, runqueue: Add --no-setscene to skip all setscene tasks Jason Wessel
` (5 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
Add the ability to see where all the per task logs
were written to disk.
The resulting output looks like where [...] is the full path:
NOTE: LOG: [...]/python-2.7.2-r2.20/temp/log.do_package_setscene.8719
NOTE: LOG: [...]/perl-5.14.2-r7/temp/log.do_package_setscene.8760
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
lib/bb/ui/knotty.py | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index 765ad27..e485c7e 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -253,6 +253,7 @@ class StdinMgr:
class RtLogLevel:
def __init__(self, handler, logfilter, mlt, tf):
self.displaytail = False
+ self.displayLogLocations = False
self.handler = handler
self.logfilter = logfilter
self.defaultLevel = logfilter.getFiltLevel()
@@ -291,6 +292,14 @@ class RtLogLevel:
print "NOTE: Activing task \"top\" mode"
if isinstance(self.tf, TerminalFilter):
self.tf.setTopMode()
+ elif input == "l":
+ if verbose:
+ print "NOTE: Activating log locations display"
+ self.displayLogLocations = True
+ elif input == "L":
+ if verbose:
+ print "NOTE: Disable log locations display"
+ self.displayLogLocations = False
elif input == "h":
print "============================================="
print "Interaction help commands:"
@@ -300,6 +309,7 @@ class RtLogLevel:
print " 3 - turn on debug logging"
print " 4 - turn on debug logging and real time log tail"
print " t - Display tasks in \"top\" mode (formerly knotty2 mode)"
+ print " l - emit log locations (L to turn off)"
print " h - display commands"
return False
return True
@@ -387,6 +397,9 @@ def main(server, eventHandler, tf = TerminalFilter):
continue
helper.eventHandler(event)
if isinstance(event, bb.build.TaskStarted):
+ if (rtloglevel.displayLogLocations):
+ termfilter.clearFooter()
+ print "NOTE: LOG: %s" % event.logfile
mlt.openLog(event.logfile, event.pid)
rtloglevel.displayLogs()
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 13/18] bitbake, runqueue: Add --no-setscene to skip all setscene tasks
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (11 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 12/18] knotty: Add ability to dynamically toggle log location Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 14/18] runqueue.py, build.py: Invalidate setscene tasks based on do_unpack stamps Jason Wessel
` (4 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
Mainly intended for the purpose of debugging or forcing builds
from source, the --no-setscene will prevent any setscene
tasks from running.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
bin/bitbake | 2 ++
lib/bb/runqueue.py | 13 +++++++------
2 files changed, 9 insertions(+), 6 deletions(-)
diff --git a/bin/bitbake b/bin/bitbake
index 420e69d..702d99c 100755
--- a/bin/bitbake
+++ b/bin/bitbake
@@ -170,6 +170,8 @@ Default BBFILES are the .bb files in the current directory.""")
parser.add_option("-B", "--bind", help = "The name/address for the bitbake server to bind to",
action = "store", dest = "bind", default = False)
+ parser.add_option("", "--no-setscene", help = "Do not run any setscene tasks, forces builds",
+ action = "store_true", dest = "nosetscene", default = False)
options, args = parser.parse_args(sys.argv)
configuration = BBConfiguration(options)
diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
index b870caf..da3fdf9 100644
--- a/lib/bb/runqueue.py
+++ b/lib/bb/runqueue.py
@@ -699,13 +699,14 @@ class RunQueueData:
# Interate over the task list looking for tasks with a 'setscene' function
self.runq_setscene = []
- for task in range(len(self.runq_fnid)):
- setscene = taskData.gettask_id(self.taskData.fn_index[self.runq_fnid[task]], self.runq_task[task] + "_setscene", False)
- if not setscene:
- continue
- self.runq_setscene.append(task)
+ if not self.cooker.configuration.nosetscene:
+ for task in range(len(self.runq_fnid)):
+ setscene = taskData.gettask_id(self.taskData.fn_index[self.runq_fnid[task]], self.runq_task[task] + "_setscene", False)
+ if not setscene:
+ continue
+ self.runq_setscene.append(task)
- # Interate over the task list and call into the siggen code
+ # Iterate over the task list and call into the siggen code
dealtwith = set()
todeal = set(range(len(self.runq_fnid)))
while len(todeal) > 0:
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 14/18] runqueue.py, build.py: Invalidate setscene tasks based on do_unpack stamps
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (12 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 13/18] bitbake, runqueue: Add --no-setscene to skip all setscene tasks Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-11 13:52 ` Richard Purdie
2012-06-08 13:41 ` [PATCH v4 15/18] bitbake, runqueue.py: Add -i to invalidate a stamp and rebuild the target Jason Wessel
` (3 subsequent siblings)
17 siblings, 1 reply; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
If you have a fully populated sstate cache and have used it to
execute a build, it is not possible to invalidate repackage
an intermediate build after you have forced a compiled
Example when you have build from a complete sstate cache build:
bitbake core-image-sato
bitbake -c patch acl
*** Make some changes to the C files for experimentation.
bitbake -f -c compile acl
bitbake acl
The bitbake will refuse to build the acl package at this
point and instead keep populating it from the sstate. Using
the cleanstate is no longer a good option because it will
also erase the scratch area.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
lib/bb/build.py | 8 ++++++++
lib/bb/runqueue.py | 14 ++++++++++++++
2 files changed, 22 insertions(+)
diff --git a/lib/bb/build.py b/lib/bb/build.py
index fb61b00..18c28aa 100644
--- a/lib/bb/build.py
+++ b/lib/bb/build.py
@@ -463,6 +463,14 @@ def del_stamp(task, d, file_name = None):
stamp = stamp_internal(task, d, file_name)
bb.utils.remove(stamp)
+def exists_stamp(task, d, file_name = None):
+ """
+ Removes a stamp for a given task
+ (d can be a data dict or dataCache)
+ """
+ stamp = stamp_internal(task, d, file_name)
+ return os.path.exists(stamp)
+
def stampfile(taskname, d, file_name = None):
"""
Return the stamp for a given task
diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
index da3fdf9..6c802af 100644
--- a/lib/bb/runqueue.py
+++ b/lib/bb/runqueue.py
@@ -718,6 +718,20 @@ class RunQueueData:
for dep in self.runq_depends[task]:
procdep.append(self.taskData.fn_index[self.runq_fnid[dep]] + "." + self.runq_task[dep])
self.runq_hash[task] = bb.parse.siggen.get_taskhash(self.taskData.fn_index[self.runq_fnid[task]], self.runq_task[task], procdep, self.dataCache)
+ try:
+ new_setscene = []
+ for task in self.runq_setscene:
+ try:
+ fn = self.taskData.fn_index[self.rq.rqdata.runq_fnid[task]]
+ if bb.build.exists_stamp("do_unpack", self.dataCache, fn):
+ logger.debug(2, 'Removing task %s from queue because do_unpack exists', task)
+ else:
+ new_setscene.append(task)
+ except:
+ logger.debug(2, 'Failed do_unpack check for %s', task)
+ self.runq_setscene = new_setscene
+ except:
+ logger.debug(2, 'Failed to update runq_setscene')
self.hashes = {}
self.hash_deps = {}
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* Re: [PATCH v4 14/18] runqueue.py, build.py: Invalidate setscene tasks based on do_unpack stamps
2012-06-08 13:41 ` [PATCH v4 14/18] runqueue.py, build.py: Invalidate setscene tasks based on do_unpack stamps Jason Wessel
@ 2012-06-11 13:52 ` Richard Purdie
2012-06-14 3:00 ` Jason Wessel
0 siblings, 1 reply; 21+ messages in thread
From: Richard Purdie @ 2012-06-11 13:52 UTC (permalink / raw)
To: Jason Wessel; +Cc: bitbake-devel
On Fri, 2012-06-08 at 08:41 -0500, Jason Wessel wrote:
> If you have a fully populated sstate cache and have used it to
> execute a build, it is not possible to invalidate repackage
> an intermediate build after you have forced a compiled
>
> Example when you have build from a complete sstate cache build:
> bitbake core-image-sato
> bitbake -c patch acl
> *** Make some changes to the C files for experimentation.
> bitbake -f -c compile acl
> bitbake acl
>
> The bitbake will refuse to build the acl package at this
> point and instead keep populating it from the sstate. Using
> the cleanstate is no longer a good option because it will
> also erase the scratch area.
>
> Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
> ---
> lib/bb/build.py | 8 ++++++++
> lib/bb/runqueue.py | 14 ++++++++++++++
> 2 files changed, 22 insertions(+)
>
> diff --git a/lib/bb/build.py b/lib/bb/build.py
> index fb61b00..18c28aa 100644
> --- a/lib/bb/build.py
> +++ b/lib/bb/build.py
> @@ -463,6 +463,14 @@ def del_stamp(task, d, file_name = None):
> stamp = stamp_internal(task, d, file_name)
> bb.utils.remove(stamp)
>
> +def exists_stamp(task, d, file_name = None):
> + """
> + Removes a stamp for a given task
> + (d can be a data dict or dataCache)
> + """
> + stamp = stamp_internal(task, d, file_name)
> + return os.path.exists(stamp)
> +
> def stampfile(taskname, d, file_name = None):
> """
> Return the stamp for a given task
> diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
> index da3fdf9..6c802af 100644
> --- a/lib/bb/runqueue.py
> +++ b/lib/bb/runqueue.py
> @@ -718,6 +718,20 @@ class RunQueueData:
> for dep in self.runq_depends[task]:
> procdep.append(self.taskData.fn_index[self.runq_fnid[dep]] + "." + self.runq_task[dep])
> self.runq_hash[task] = bb.parse.siggen.get_taskhash(self.taskData.fn_index[self.runq_fnid[task]], self.runq_task[task], procdep, self.dataCache)
> + try:
> + new_setscene = []
> + for task in self.runq_setscene:
> + try:
> + fn = self.taskData.fn_index[self.rq.rqdata.runq_fnid[task]]
> + if bb.build.exists_stamp("do_unpack", self.dataCache, fn):
> + logger.debug(2, 'Removing task %s from queue because do_unpack exists', task)
> + else:
> + new_setscene.append(task)
> + except:
> + logger.debug(2, 'Failed do_unpack check for %s', task)
> + self.runq_setscene = new_setscene
> + except:
> + logger.debug(2, 'Failed to update runq_setscene')
We've gone to quite a bit of trouble to keep knowledge of the specific
tasks out of bitbake. I've noticed a few of your patches are "blurring"
the separation between bitbake and the metadata.
Here, bitbake shouldn't have knowledge of "unpack" or why its special.
I'd also argue its not in fact special and there are probably other
"force" scenarios which would break with a similar problem even with
this patch. I agree there is a problem here but we need to fix it in a
more generic way. There is already a bug open on this kind of problem
(#2256).
Cheers,
Richard
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v4 14/18] runqueue.py, build.py: Invalidate setscene tasks based on do_unpack stamps
2012-06-11 13:52 ` Richard Purdie
@ 2012-06-14 3:00 ` Jason Wessel
0 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-14 3:00 UTC (permalink / raw)
To: Richard Purdie; +Cc: bitbake-devel
On 06/11/2012 08:52 AM, Richard Purdie wrote:
> On Fri, 2012-06-08 at 08:41 -0500, Jason Wessel wrote:
>> If you have a fully populated sstate cache and have used it to
>> execute a build, it is not possible to invalidate repackage
>> an intermediate build after you have forced a compiled
>>
>> Example when you have build from a complete sstate cache build:
>> bitbake core-image-sato
>> bitbake -c patch acl
>> *** Make some changes to the C files for experimentation.
>> bitbake -f -c compile acl
>> bitbake acl
>>
>> The bitbake will refuse to build the acl package at this
>> point and instead keep populating it from the sstate. Using
>> the cleanstate is no longer a good option because it will
>> also erase the scratch area.
>>
>> Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
>> ---
>> lib/bb/build.py | 8 ++++++++
>> lib/bb/runqueue.py | 14 ++++++++++++++
>> 2 files changed, 22 insertions(+)
>>
>> diff --git a/lib/bb/build.py b/lib/bb/build.py
>> index fb61b00..18c28aa 100644
>> --- a/lib/bb/build.py
>> +++ b/lib/bb/build.py
>> @@ -463,6 +463,14 @@ def del_stamp(task, d, file_name = None):
>> stamp = stamp_internal(task, d, file_name)
>> bb.utils.remove(stamp)
>>
>> +def exists_stamp(task, d, file_name = None):
>> + """
>> + Removes a stamp for a given task
>> + (d can be a data dict or dataCache)
>> + """
>> + stamp = stamp_internal(task, d, file_name)
>> + return os.path.exists(stamp)
>> +
>> def stampfile(taskname, d, file_name = None):
>> """
>> Return the stamp for a given task
>> diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
>> index da3fdf9..6c802af 100644
>> --- a/lib/bb/runqueue.py
>> +++ b/lib/bb/runqueue.py
>> @@ -718,6 +718,20 @@ class RunQueueData:
>> for dep in self.runq_depends[task]:
>> procdep.append(self.taskData.fn_index[self.runq_fnid[dep]] + "." + self.runq_task[dep])
>> self.runq_hash[task] = bb.parse.siggen.get_taskhash(self.taskData.fn_index[self.runq_fnid[task]], self.runq_task[task], procdep, self.dataCache)
>> + try:
>> + new_setscene = []
>> + for task in self.runq_setscene:
>> + try:
>> + fn = self.taskData.fn_index[self.rq.rqdata.runq_fnid[task]]
>> + if bb.build.exists_stamp("do_unpack", self.dataCache, fn):
>> + logger.debug(2, 'Removing task %s from queue because do_unpack exists', task)
>> + else:
>> + new_setscene.append(task)
>> + except:
>> + logger.debug(2, 'Failed do_unpack check for %s', task)
>> + self.runq_setscene = new_setscene
>> + except:
>> + logger.debug(2, 'Failed to update runq_setscene')
> We've gone to quite a bit of trouble to keep knowledge of the specific
> tasks out of bitbake. I've noticed a few of your patches are "blurring"
> the separation between bitbake and the metadata.
>
> Here, bitbake shouldn't have knowledge of "unpack" or why its special.
> I'd also argue its not in fact special and there are probably other
> "force" scenarios which would break with a similar problem even with
> this patch. I agree there is a problem here but we need to fix it in a
> more generic way. There is already a bug open on this kind of problem
> (#2256).
As I mentioned in the 0/18 summary this is certainly the case that there needs to be an API and this should be treated as more of an RFC.
I will break whole series down into a few smaller chunks since there are some important fixes vs the features.
Jason.
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v4 15/18] bitbake, runqueue.py: Add -i to invalidate a stamp and rebuild the target
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (13 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 14/18] runqueue.py, build.py: Invalidate setscene tasks based on do_unpack stamps Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 16/18] meta/lib/oe/terminal.py: Add an additional reference to screen called screen_inline Jason Wessel
` (2 subsequent siblings)
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
It is highly desirable to be able to invalidate a stamp used for
do_compile and to be able to continue on to build the entire
package's build rule.
If invalidating a stamp, the setscene rules for the specified targets
should be automatically canceled so as to allow a forced build
of an individual target.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
bin/bitbake | 3 +++
lib/bb/runqueue.py | 13 ++++++++++++-
2 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/bin/bitbake b/bin/bitbake
index 702d99c..bf81a0c 100755
--- a/bin/bitbake
+++ b/bin/bitbake
@@ -150,6 +150,9 @@ Default BBFILES are the .bb files in the current directory.""")
parser.add_option("-I", "--ignore-deps", help = """Assume these dependencies don't exist and are already provided (equivalent to ASSUME_PROVIDED). Useful to make dependency graphs more appealing""",
action = "append", dest = "extra_assume_provided", default = [])
+ parser.add_option("-i", "--invalidate-stamp", help = "Invalidate the specified stamp for a command such as 'compile' and build",
+ action = "store", dest = "invalidate_stamp")
+
parser.add_option("-l", "--log-domains", help = """Show debug logging for the specified logging domains""",
action = "append", dest = "debug_domains", default = [])
diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
index 6c802af..3dbfaad 100644
--- a/lib/bb/runqueue.py
+++ b/lib/bb/runqueue.py
@@ -720,10 +720,14 @@ class RunQueueData:
self.runq_hash[task] = bb.parse.siggen.get_taskhash(self.taskData.fn_index[self.runq_fnid[task]], self.runq_task[task], procdep, self.dataCache)
try:
new_setscene = []
+ test_fns = []
+ if self.cooker.configuration.invalidate_stamp:
+ for tgt in self.target_pairs:
+ test_fns.append(tgt[0])
for task in self.runq_setscene:
try:
fn = self.taskData.fn_index[self.rq.rqdata.runq_fnid[task]]
- if bb.build.exists_stamp("do_unpack", self.dataCache, fn):
+ if bb.build.exists_stamp("do_unpack", self.dataCache, fn) or (self.cooker.configuration.invalidate_stamp and fn in test_fns):
logger.debug(2, 'Removing task %s from queue because do_unpack exists', task)
else:
new_setscene.append(task)
@@ -746,6 +750,13 @@ class RunQueueData:
deps.append(depidentifier)
self.hash_deps[identifier] = deps
+ # Remove stamps for forced invalidation
+ if self.cooker.configuration.invalidate_stamp:
+ for (fn, target) in self.target_pairs:
+ for st in self.cooker.configuration.invalidate_stamp.split(','):
+ logger.verbose("Remove stamp %s, %s", st, fn)
+ bb.build.del_stamp("do_%s" % st, self.dataCache, fn)
+
# Remove stamps for targets if force mode active
if self.cooker.configuration.force:
for (fn, target) in self.target_pairs:
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 16/18] meta/lib/oe/terminal.py: Add an additional reference to screen called screen_inline
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (14 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 15/18] bitbake, runqueue.py: Add -i to invalidate a stamp and rebuild the target Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 17/18] terminal.bbclass: Allow OE_TERMINAL_PREFERRED to override the fallback order for OE_TERMINAL Jason Wessel
2012-06-08 13:41 ` [PATCH v4 18/18] terminal.py: Make screen_inline have a higher priority than screen Jason Wessel
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
This is a new mode for screen with the explicit purpose of invoking
screen on the same terminal that you started bitbake. The bitbake knotty
will attempt to launch screen directly when using this setting for
OE_TERMINAL.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
meta/lib/oe/terminal.py | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/meta/lib/oe/terminal.py b/meta/lib/oe/terminal.py
index 43639d5..ca94e73 100644
--- a/meta/lib/oe/terminal.py
+++ b/meta/lib/oe/terminal.py
@@ -92,6 +92,13 @@ class Rxvt(XTerminal):
command = 'rxvt -T "{title}" -e {command}'
priority = 1
+class Screen_inline(Terminal):
+ command = 'screen -D -m -t "{title}" -S devshell {command}'
+
+ def __init__(self, command, title=None, env=None):
+ Terminal.__init__(self, command, title, env)
+ logger.warn('Screen Connect Command: screen -r devshell')
+
class Screen(Terminal):
command = 'screen -D -m -t "{title}" -S devshell {command}'
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 17/18] terminal.bbclass: Allow OE_TERMINAL_PREFERRED to override the fallback order for OE_TERMINAL
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (15 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 16/18] meta/lib/oe/terminal.py: Add an additional reference to screen called screen_inline Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
2012-06-08 13:41 ` [PATCH v4 18/18] terminal.py: Make screen_inline have a higher priority than screen Jason Wessel
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
With the addition of two different modes for the use of screen, an end
user may prefer to set specific fall back order for an X vs tty
environment. This patch modifies the terminal.bbclass and terminal.py
to allow the variable OE_TERMINAL_PREFERRED to control the order of
trying the various supported OE_TERMINAL's.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
meta/classes/terminal.bbclass | 3 ++-
meta/lib/oe/terminal.py | 8 +++++---
2 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/meta/classes/terminal.bbclass b/meta/classes/terminal.bbclass
index 3cfc84b..2be6974 100644
--- a/meta/classes/terminal.bbclass
+++ b/meta/classes/terminal.bbclass
@@ -4,6 +4,7 @@ OE_TERMINAL[choices] = 'auto none \
${@" ".join(o.name \
for o in oe.terminal.prioritized())}'
+OE_TERMINAL_PREFERRED ?= "auto"
OE_TERMINAL_EXPORTS = 'XAUTHORITY SHELL DBUS_SESSION_BUS_ADDRESS DISPLAY EXTRA_OEMAKE SCREENDIR'
OE_TERMINAL_EXPORTS[type] = 'list'
@@ -34,7 +35,7 @@ def oe_terminal(command, title, d):
bb.fatal('Unable to spawn terminal %s: %s' % (terminal, exc))
try:
- oe.terminal.spawn_preferred(command, title)
+ oe.terminal.spawn_preferred(d.getVar('OE_TERMINAL_PREFERRED', True), command, title)
except oe.terminal.NoSupportedTerminals:
bb.fatal('No valid terminal found, unable to open devshell')
except oe.terminal.ExecutionError as exc:
diff --git a/meta/lib/oe/terminal.py b/meta/lib/oe/terminal.py
index ca94e73..943e5db 100644
--- a/meta/lib/oe/terminal.py
+++ b/meta/lib/oe/terminal.py
@@ -111,11 +111,13 @@ class Screen(Terminal):
def prioritized():
return Registry.prioritized()
-def spawn_preferred(command, title=None, env=None):
+def spawn_preferred(priorityList, command, title=None, env=None):
"""Spawn the first supported terminal, by priority"""
- for terminal in prioritized():
+ if priorityList == "auto":
+ priorityList = ' '.join(o.name for o in prioritized())
+ for terminal in priorityList.split():
try:
- spawn(terminal.name, command, title, env)
+ spawn(terminal, command, title, env)
break
except UnsupportedTerminal:
continue
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v4 18/18] terminal.py: Make screen_inline have a higher priority than screen
2012-06-08 13:41 [PATCH v4 0/18] Multiplexed task logs / bitbake fixes for fetch2 / stamp features Jason Wessel
` (16 preceding siblings ...)
2012-06-08 13:41 ` [PATCH v4 17/18] terminal.bbclass: Allow OE_TERMINAL_PREFERRED to override the fallback order for OE_TERMINAL Jason Wessel
@ 2012-06-08 13:41 ` Jason Wessel
17 siblings, 0 replies; 21+ messages in thread
From: Jason Wessel @ 2012-06-08 13:41 UTC (permalink / raw)
To: bitbake-devel
The screen_inline is far easier to use and less confusing for
beginner users of oe-core. Making this the default improves
the end user experience, and it is possible to change
the OE_TERMINAL variable as well as the default fall back list
with OE_TERMINAL_PREFERRED for anyone who desires the original
behavior.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
meta/lib/oe/terminal.py | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/meta/lib/oe/terminal.py b/meta/lib/oe/terminal.py
index 943e5db..3122d56 100644
--- a/meta/lib/oe/terminal.py
+++ b/meta/lib/oe/terminal.py
@@ -55,11 +55,11 @@ class XTerminal(Terminal):
class Gnome(XTerminal):
command = 'gnome-terminal --disable-factory -t "{title}" -x {command}'
- priority = 2
+ priority = 3
class Xfce(XTerminal):
command = 'Terminal -T "{title}" -e "{command}"'
- priority = 2
+ priority = 3
def __init__(self, command, title=None, env=None):
# Upstream binary name is Terminal but Debian/Ubuntu use
@@ -73,7 +73,7 @@ class Xfce(XTerminal):
class Konsole(XTerminal):
command = 'konsole -T "{title}" -e {command}'
- priority = 2
+ priority = 3
def __init__(self, command, title=None, env=None):
# Check version
@@ -86,14 +86,15 @@ class Konsole(XTerminal):
class XTerm(XTerminal):
command = 'xterm -T "{title}" -e {command}'
- priority = 1
+ priority = 2
class Rxvt(XTerminal):
command = 'rxvt -T "{title}" -e {command}'
- priority = 1
+ priority = 2
class Screen_inline(Terminal):
command = 'screen -D -m -t "{title}" -S devshell {command}'
+ priority = 1
def __init__(self, command, title=None, env=None):
Terminal.__init__(self, command, title, env)
--
1.7.10
^ permalink raw reply related [flat|nested] 21+ messages in thread