* [PATCH 1/2] report-error.bbclass: Support Unicode reports
2015-10-01 10:40 [PATCH 0/2] Unicode support for Report Error and Qemu mariano.lopez
@ 2015-10-01 10:40 ` mariano.lopez
2015-10-01 10:40 ` [PATCH 2/2] oeqa/utils/qemurunner: Add support for Unicode from qemu mariano.lopez
1 sibling, 0 replies; 3+ messages in thread
From: mariano.lopez @ 2015-10-01 10:40 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
Currently error-report doesn't manage Unicode because
the files are opened with the default codec.
This patch changes the codec of the files to UTF-8,
this way the reports will include Unicode characters.
This is useful for the qemu output when doing the
testimage task.
[YOCTO #8225]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/classes/report-error.bbclass | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/meta/classes/report-error.bbclass b/meta/classes/report-error.bbclass
index 040c29e..82b5bcd 100644
--- a/meta/classes/report-error.bbclass
+++ b/meta/classes/report-error.bbclass
@@ -9,22 +9,25 @@
ERR_REPORT_DIR ?= "${LOG_DIR}/error-report"
def errorreport_getdata(e):
+ import codecs
logpath = e.data.getVar('ERR_REPORT_DIR', True)
datafile = os.path.join(logpath, "error-report.txt")
- with open(datafile) as f:
+ with codecs.open(datafile, 'r', 'utf-8') as f:
data = f.read()
return data
def errorreport_savedata(e, newdata, file):
import json
+ import codecs
logpath = e.data.getVar('ERR_REPORT_DIR', True)
datafile = os.path.join(logpath, file)
- with open(datafile, "w") as f:
+ with codecs.open(datafile, 'w', 'utf-8') as f:
json.dump(newdata, f, indent=4, sort_keys=True)
return datafile
python errorreport_handler () {
import json
+ import codecs
logpath = e.data.getVar('ERR_REPORT_DIR', True)
datafile = os.path.join(logpath, "error-report.txt")
@@ -53,8 +56,8 @@ python errorreport_handler () {
taskdata['task'] = task
if log:
try:
- logFile = open(log, 'r')
- logdata = logFile.read().decode('utf-8')
+ logFile = codecs.open(log, 'r', 'utf-8')
+ logdata = logFile.read()
logFile.close()
except:
logdata = "Unable to read log file"
--
1.8.4.5
^ permalink raw reply related [flat|nested] 3+ messages in thread* [PATCH 2/2] oeqa/utils/qemurunner: Add support for Unicode from qemu
2015-10-01 10:40 [PATCH 0/2] Unicode support for Report Error and Qemu mariano.lopez
2015-10-01 10:40 ` [PATCH 1/2] report-error.bbclass: Support Unicode reports mariano.lopez
@ 2015-10-01 10:40 ` mariano.lopez
1 sibling, 0 replies; 3+ messages in thread
From: mariano.lopez @ 2015-10-01 10:40 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
The current state of qemurunner will drop the Unicode
characters received from qemu, this is because error
report web had problems with Unicode characters; now
that the server support Unicode, it is possible to
log all the output from qemu. So far the only Unicode
character seen is the copyright symbol.
This patch allows to get Unicode characters from the qemu
target and save the log in an UTF-8 file for latter use.
[YOCTO #8225]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/utils/qemurunner.py | 15 +++++++++++----
1 file changed, 11 insertions(+), 4 deletions(-)
diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index 9c878bc..6fe75b8 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -15,11 +15,18 @@ import select
import errno
import string
import threading
+import codecs
from oeqa.utils.dump import HostDumper
import logging
logger = logging.getLogger("BitBake.QemuRunner")
+# Get Unicode non printable control chars
+control_range = range(0,32)+range(127,160)
+control_chars = [unichr(x) for x in control_range
+ if unichr(x) not in string.printable]
+re_control_char = re.compile('[%s]' % re.escape("".join(control_chars)))
+
class QemuRunner:
def __init__(self, machine, rootfs, display, tmpdir, deploy_dir_image, logfile, boottime, dump_dir, dump_host_cmds):
@@ -63,9 +70,9 @@ class QemuRunner:
def log(self, msg):
if self.logfile:
# It is needed to sanitize the data received from qemu
- # because is possible to have control characters or Unicode
- msg = "".join(filter(lambda x:x in string.printable, msg))
- with open(self.logfile, "a") as f:
+ # because is possible to have control characters
+ msg = re_control_char.sub('', unicode(msg, 'utf-8'))
+ with codecs.open(self.logfile, "a", encoding="utf-8") as f:
f.write("%s" % msg)
def getOutput(self, o):
@@ -176,7 +183,7 @@ class QemuRunner:
cmdline = p.read()
# It is needed to sanitize the data received
# because is possible to have control characters
- cmdline = "".join(filter(lambda x:x in string.printable, cmdline))
+ cmdline = re_control_char.sub('', cmdline)
try:
ips = re.findall("((?:[0-9]{1,3}\.){3}[0-9]{1,3})", cmdline.split("ip=")[1])
if not ips or len(ips) != 3:
--
1.8.4.5
^ permalink raw reply related [flat|nested] 3+ messages in thread