[PATCH 0/1] Error reporting tool

[PATCH 0/1] Error reporting tool

[Andreea Proca]

Hello,

This patch together with the web app from git://git.yoctoproject.org/error-report-web 
form an error reporting system similar to kernel-oops service.
This tool proposes a way of keeping track of the errors appeared during builds
by dumping the debug information of the build into a database which can be analyzed.

The data collected consists of the log file of the tasks that failed and common configuration variables
(such as machine, distro, host distro, etc as well as the version of the build system - branch/commit id, which
is commonly missing from bug reports :) ).
The project will benefit from the multitude of build results published by quickly identifying the issues and the areas of interest for the contributors.
The objectives of the application were achieved by creating a client-server system. The client (basically this bitbake class) collects all the information needed
for solving the errors and the server receives that data and saves in a database (see the Django web app at git://git.yoctoproject.org/error-report-web).

All the debug information collected during a build which failed is stored on the user's machine, in a JSON format and its path is displayed
in the output of the build as well as the command that the contributor can run a script to send the error report file to the server. The
script will give the user a link corresponding to his entry in the database. The scripts accepts a parameter which is the hostname
to the web app instance (by default it's localhost).
Contributors can search by certain configurations (e.g. machine, component, error, task) and see statistics through the web interface.


Thanks,
Andreea


The following changes since commit abf22bed20870e6283853674a51a39f44af93936:

  Revert "nss: avoid to use the hardcode kernel version" (2014-02-13 12:06:01 +0000)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib andreeap/report-poky
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=andreeap/report-poky

Andreea Proca (1):
  report-error.bbclass: new class to save build information when errors
    occur

 meta/classes/report-error.bbclass |   67 +++++++++++++++++++++++++++++++
 scripts/send-error-report         |   78 +++++++++++++++++++++++++++++++++++++
 2 files changed, 145 insertions(+)
 create mode 100644 meta/classes/report-error.bbclass
 create mode 100755 scripts/send-error-report

-- 
1.7.9.5

[PATCH 1/1] report-error.bbclass: new class to save build information when errors occur

[Andreea Proca]

Class is used to save data about errors after every task that failed.
Debug data is saved in json files in ERROR_REPORT_DIR (defaults to tmp/log/error-report).
To use this class one has to add INHERIT += "report-error" to local.conf.

scripts/send-error-report is a simple script that sends the json file
to a HTTP server that collects data (git://git.yoctoproject.org/error-report-web
is a Django web interface that can be used to receive and visualize
the error reports). The script will give you an URL where you can
find your report.

Signed-off-by: Andreea Proca <andreea.b.proca@intel.com>
---
 meta/classes/report-error.bbclass |   67 +++++++++++++++++++++++++++++++
 scripts/send-error-report         |   78 +++++++++++++++++++++++++++++++++++++
 2 files changed, 145 insertions(+)
 create mode 100644 meta/classes/report-error.bbclass
 create mode 100755 scripts/send-error-report

diff --git a/meta/classes/report-error.bbclass b/meta/classes/report-error.bbclass
new file mode 100644
index 0000000..149b36c
--- /dev/null
+++ b/meta/classes/report-error.bbclass
@@ -0,0 +1,67 @@
+#
+# Collects debug information in order to create error report files.
+#
+# Copyright (C) 2013 Intel Corporation
+# Author: Andreea Brandusa Proca <andreea.b.proca@intel.com>
+#
+# Licensed under the MIT license, see COPYING.MIT for details
+
+ERR_REPORT_DIR ?= "${LOG_DIR}/error-report"
+
+def getdata(e):
+    logpath = e.data.getVar('ERR_REPORT_DIR', True)
+    datafile = os.path.join(logpath, "error-report.txt")
+    f = open(datafile, "r")
+    data = f.read()
+    f.close()
+    return data
+
+def savedata(e, newdata, file):
+    import json
+    logpath = e.data.getVar('ERR_REPORT_DIR', True)
+    bb.utils.mkdirhier(logpath)
+    datafile = os.path.join(logpath, file)
+    f = open(datafile, "w")
+    json.dump(newdata, f, indent=4, sort_keys=True)
+    f.close()
+    return datafile
+
+python report () {
+        import os, re, sys, json
+
+        if isinstance(e, bb.event.BuildStarted):
+            data = {}
+            machine = e.data.getVar("MACHINE")
+            data['machine'] = machine
+            data['build_sys'] = e.data.getVar("BUILD_SYS", True)
+            data['nativelsb'] = e.data.getVar("NATIVELSBSTRING")
+            data['distro'] = e.data.getVar("DISTRO")
+            data['target_sys'] = e.data.getVar("TARGET_SYS", True)
+            data['failures'] = []
+            data['component'] = e.getPkgs()[0]
+            data['branch_commit'] = base_detect_branch(e.data) + ": " + base_detect_revision(e.data)
+            savedata(e, data, "error-report.txt")
+
+        elif isinstance(e, bb.build.TaskFailed):
+            task = e.task
+            taskdata={}
+            log = e.data.getVar('BB_LOGFILE', True)
+            logFile = open(log, 'r')
+            taskdata['package'] = e.data.expand("${PF}")
+            taskdata['task'] = task
+            taskdata['log'] = logFile.read()
+            logFile.close()
+            jsondata = json.loads(getdata(e))
+            jsondata['failures'].append(taskdata)
+            savedata(e, jsondata, "error-report.txt")
+
+        elif isinstance(e, bb.event.BuildCompleted):
+            jsondata = json.loads(getdata(e))
+            failures = jsondata['failures']
+            if(len(failures) > 0):
+                filename = "error_report_" + e.data.getVar("BUILDNAME")+".txt"
+                datafile = savedata(e, jsondata, filename)
+                bb.note("The errors of this build are stored in: %s. You can send the errors to an upstream server by running: send-error-report %s [server]" % (datafile, datafile))
+}
+
+addhandler report
diff --git a/scripts/send-error-report b/scripts/send-error-report
new file mode 100755
index 0000000..0d85776
--- /dev/null
+++ b/scripts/send-error-report
@@ -0,0 +1,78 @@
+#!/usr/bin/env python
+
+# Sends an error report (if the report-error class was enabled) to a remote server.
+#
+# Copyright (C) 2013 Intel Corporation
+# Author: Andreea Proca <andreea.b.proca@intel.com>
+
+
+
+import httplib, urllib, os, sys, json
+
+
+def sendData(json_file, server):
+
+    if os.path.isfile(json_file):
+
+        home = os.path.expanduser("~")
+        userfile = os.path.join(home, ".oe-send-error")
+        if os.path.isfile(userfile):
+            with open(userfile) as g:
+                username = g.readline()
+                email = g.readline()
+        else:
+            print("Please enter your name and your email (optionally), they'll be saved in the file you send.")
+            username = raw_input("Name: ")
+            email = raw_input("E-mail (not required): ")
+            if len(username) > 0 and len(username) < 50:
+                with open(userfile, "w") as g:
+                    g.write(username + "\n")
+                    g.write(email + "\n")
+            else:
+                print("Invalid inputs, try again.")
+                return
+
+        with open(json_file) as f:
+            data = f.read()
+
+        try:
+            jsondata = json.loads(data)
+            jsondata['username'] = username.strip()
+            jsondata['email'] = email.strip()
+            data = json.dumps(jsondata, indent=4, sort_keys=True)
+        except:
+            print("Invalid json data")
+            return
+
+        try:
+            params = urllib.urlencode({'data': data})
+            headers = {"Content-type": "application/json"}
+            conn = httplib.HTTPConnection(server)
+            conn.request("POST", "/ClientPost/", params, headers)
+            response = conn.getresponse()
+            print response.status, response.reason
+            res = response.read()
+            if response.status == 200:
+                print res
+            else:
+                print("There was a problem submiting your data")
+            conn.close()
+        except:
+                print("Server connection failed")
+
+    else:
+        print("No data file found.")
+
+
+if __name__ == '__main__':
+    print ("\nSends an error report (if the report-error class was enabled) to a remote server.")
+    if len(sys.argv) < 2:
+        print("\nThis scripts sends the contents of a file to an upstream server.")
+        print("\nUsage: send-error-report <error_fileName> [server]")
+        print("\nIf this is the first when sending a report you'll be asked for your name and optionally your email address.")
+        print("They will be associated with your report.\n")
+
+    elif len(sys.argv) == 3:
+        sendData(sys.argv[1], sys.argv[2])
+    else:
+        sendData(sys.argv[1], "localhost:8000")
-- 
1.7.9.5

Re: [PATCH 1/1] report-error.bbclass: new class to save build information when errors occur

[Richard Purdie]

On Thu, 2014-02-13 at 17:28 +0200, Andreea Proca wrote:
> Class is used to save data about errors after every task that failed.
> Debug data is saved in json files in ERROR_REPORT_DIR (defaults to tmp/log/error-report).
> To use this class one has to add INHERIT += "report-error" to local.conf.
> 
> scripts/send-error-report is a simple script that sends the json file
> to a HTTP server that collects data (git://git.yoctoproject.org/error-report-web
> is a Django web interface that can be used to receive and visualize
> the error reports). The script will give you an URL where you can
> find your report.
> 
> Signed-off-by: Andreea Proca <andreea.b.proca@intel.com>
> ---
>  meta/classes/report-error.bbclass |   67 +++++++++++++++++++++++++++++++
>  scripts/send-error-report         |   78 +++++++++++++++++++++++++++++++++++++
>  2 files changed, 145 insertions(+)
>  create mode 100644 meta/classes/report-error.bbclass
>  create mode 100755 scripts/send-error-report
> 
> diff --git a/meta/classes/report-error.bbclass b/meta/classes/report-error.bbclass
> new file mode 100644
> index 0000000..149b36c
> --- /dev/null
> +++ b/meta/classes/report-error.bbclass
> @@ -0,0 +1,67 @@
> +#
> +# Collects debug information in order to create error report files.
> +#
> +# Copyright (C) 2013 Intel Corporation
> +# Author: Andreea Brandusa Proca <andreea.b.proca@intel.com>
> +#
> +# Licensed under the MIT license, see COPYING.MIT for details
> +
> +ERR_REPORT_DIR ?= "${LOG_DIR}/error-report"
> +
> +def getdata(e):
> +    logpath = e.data.getVar('ERR_REPORT_DIR', True)
> +    datafile = os.path.join(logpath, "error-report.txt")
> +    f = open(datafile, "r")
> +    data = f.read()
> +    f.close()
> +    return data
> +
> +def savedata(e, newdata, file):
> +    import json
> +    logpath = e.data.getVar('ERR_REPORT_DIR', True)
> +    bb.utils.mkdirhier(logpath)
> +    datafile = os.path.join(logpath, file)
> +    f = open(datafile, "w")
> +    json.dump(newdata, f, indent=4, sort_keys=True)
> +    f.close()
> +    return datafile

Since these functions will end up in the main namespace, could we prefix
them so something like errorreport_getdata and errorreport_savedata?
This makes it clearer when you look at the names out of context of this
class.


> +python report () {
> +        import os, re, sys, json
> +
> +        if isinstance(e, bb.event.BuildStarted):
> +            data = {}
> +            machine = e.data.getVar("MACHINE")
> +            data['machine'] = machine
> +            data['build_sys'] = e.data.getVar("BUILD_SYS", True)
> +            data['nativelsb'] = e.data.getVar("NATIVELSBSTRING")
> +            data['distro'] = e.data.getVar("DISTRO")
> +            data['target_sys'] = e.data.getVar("TARGET_SYS", True)
> +            data['failures'] = []
> +            data['component'] = e.getPkgs()[0]
> +            data['branch_commit'] = base_detect_branch(e.data) + ": " + base_detect_revision(e.data)
> +            savedata(e, data, "error-report.txt")
> +
> +        elif isinstance(e, bb.build.TaskFailed):
> +            task = e.task
> +            taskdata={}
> +            log = e.data.getVar('BB_LOGFILE', True)
> +            logFile = open(log, 'r')
> +            taskdata['package'] = e.data.expand("${PF}")
> +            taskdata['task'] = task
> +            taskdata['log'] = logFile.read()
> +            logFile.close()
> +            jsondata = json.loads(getdata(e))
> +            jsondata['failures'].append(taskdata)
> +            savedata(e, jsondata, "error-report.txt")
> +
> +        elif isinstance(e, bb.event.BuildCompleted):
> +            jsondata = json.loads(getdata(e))
> +            failures = jsondata['failures']
> +            if(len(failures) > 0):
> +                filename = "error_report_" + e.data.getVar("BUILDNAME")+".txt"
> +                datafile = savedata(e, jsondata, filename)
> +                bb.note("The errors of this build are stored in: %s. You can send the errors to an upstream server by running: send-error-report %s [server]" % (datafile, datafile))
> +}
> +
> +addhandler report

Could you name this something like errorreport_handler for the same
reason as the functions and add an eventmask:

errorreport_handler[eventmask] = "bb.event.BuildStarted bb.event.BuildCompleted bb.build.TaskFailed"

which means that handler only gets the events it is interested in. I
suspect this syntax was added after you started work on this code but it
would be nice to add.

I'm really looking forward to seeing this in action!

Cheers,

Richard

> diff --git a/scripts/send-error-report b/scripts/send-error-report
> new file mode 100755
> index 0000000..0d85776
> --- /dev/null
> +++ b/scripts/send-error-report
> @@ -0,0 +1,78 @@
> +#!/usr/bin/env python
> +
> +# Sends an error report (if the report-error class was enabled) to a remote server.
> +#
> +# Copyright (C) 2013 Intel Corporation
> +# Author: Andreea Proca <andreea.b.proca@intel.com>
> +
> +
> +
> +import httplib, urllib, os, sys, json
> +
> +
> +def sendData(json_file, server):
> +
> +    if os.path.isfile(json_file):
> +
> +        home = os.path.expanduser("~")
> +        userfile = os.path.join(home, ".oe-send-error")
> +        if os.path.isfile(userfile):
> +            with open(userfile) as g:
> +                username = g.readline()
> +                email = g.readline()
> +        else:
> +            print("Please enter your name and your email (optionally), they'll be saved in the file you send.")
> +            username = raw_input("Name: ")
> +            email = raw_input("E-mail (not required): ")
> +            if len(username) > 0 and len(username) < 50:
> +                with open(userfile, "w") as g:
> +                    g.write(username + "\n")
> +                    g.write(email + "\n")
> +            else:
> +                print("Invalid inputs, try again.")
> +                return
> +
> +        with open(json_file) as f:
> +            data = f.read()
> +
> +        try:
> +            jsondata = json.loads(data)
> +            jsondata['username'] = username.strip()
> +            jsondata['email'] = email.strip()
> +            data = json.dumps(jsondata, indent=4, sort_keys=True)
> +        except:
> +            print("Invalid json data")
> +            return
> +
> +        try:
> +            params = urllib.urlencode({'data': data})
> +            headers = {"Content-type": "application/json"}
> +            conn = httplib.HTTPConnection(server)
> +            conn.request("POST", "/ClientPost/", params, headers)
> +            response = conn.getresponse()
> +            print response.status, response.reason
> +            res = response.read()
> +            if response.status == 200:
> +                print res
> +            else:
> +                print("There was a problem submiting your data")
> +            conn.close()
> +        except:
> +                print("Server connection failed")
> +
> +    else:
> +        print("No data file found.")
> +
> +
> +if __name__ == '__main__':
> +    print ("\nSends an error report (if the report-error class was enabled) to a remote server.")
> +    if len(sys.argv) < 2:
> +        print("\nThis scripts sends the contents of a file to an upstream server.")
> +        print("\nUsage: send-error-report <error_fileName> [server]")
> +        print("\nIf this is the first when sending a report you'll be asked for your name and optionally your email address.")
> +        print("They will be associated with your report.\n")
> +
> +    elif len(sys.argv) == 3:
> +        sendData(sys.argv[1], sys.argv[2])
> +    else:
> +        sendData(sys.argv[1], "localhost:8000")

Re: [PATCH 0/1] Error reporting tool

[David Nyström]

On 2014-02-13 16:28, Andreea Proca wrote:
>
> Hello,
>
> This patch together with the web app from git://git.yoctoproject.org/error-report-web
> form an error reporting system similar to kernel-oops service.
> This tool proposes a way of keeping track of the errors appeared during builds
> by dumping the debug information of the build into a database which can be analyzed.
>
> The data collected consists of the log file of the tasks that failed and common configuration variables
> (such as machine, distro, host distro, etc as well as the version of the build system - branch/commit id, which
> is commonly missing from bug reports :) ).
> The project will benefit from the multitude of build results published by quickly identifying the issues and the areas of interest for the contributors.
> The objectives of the application were achieved by creating a client-server system. The client (basically this bitbake class) collects all the information needed
> for solving the errors and the server receives that data and saves in a database (see the Django web app at git://git.yoctoproject.org/error-report-web).
>
> All the debug information collected during a build which failed is stored on the user's machine, in a JSON format and its path is displayed
> in the output of the build as well as the command that the contributor can run a script to send the error report file to the server. The
> script will give the user a link corresponding to his entry in the database. The scripts accepts a parameter which is the hostname
> to the web app instance (by default it's localhost).
> Contributors can search by certain configurations (e.g. machine, component, error, task) and see statistics through the web interface.
>
>
> Thanks,
> Andreea
>

+1
Good initiative !

kerneloops-daemon/www.kerneloops.org for bitbake as far as I understand 
the explanation above.

Br,
David