From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dan.rpsys.net (5751f4a1.skybroadband.com [87.81.244.161]) by mail.openembedded.org (Postfix) with ESMTP id 6BC9673196 for ; Tue, 9 Feb 2016 15:08:51 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by dan.rpsys.net (8.14.4/8.14.4/Debian-4.1ubuntu1) with ESMTP id u19F8nHn022354; Tue, 9 Feb 2016 15:08:49 GMT Received: from dan.rpsys.net ([127.0.0.1]) by localhost (dan.rpsys.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id d3cqVbeA1OQB; Tue, 9 Feb 2016 15:08:48 +0000 (GMT) Received: from hex ([192.168.3.34]) (authenticated bits=0) by dan.rpsys.net (8.14.4/8.14.4/Debian-4.1ubuntu1) with ESMTP id u19F8g9f022350 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 9 Feb 2016 15:08:44 GMT Message-ID: <1455030522.16142.78.camel@linuxfoundation.org> From: Richard Purdie To: bitbake-devel Date: Tue, 09 Feb 2016 15:08:42 +0000 X-Mailer: Evolution 3.16.5-1ubuntu3.1 Mime-Version: 1.0 Cc: "Witt, Randy E" Subject: [PATCH] build: Improve python execution tracebacks X-BeenThere: bitbake-devel@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussion that advance bitbake development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Feb 2016 15:08:52 -0000 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit If something fails in a exec_func_python() the current stack trace shows incorrect filenames and linenumbers. For example: The stack trace of python calls that resulted in this exception/failure was: File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 200, function: 0196: chksum = bb.utils.sha256_file(fn) 0197: f.write('%s\t%s\n' % (chksum, os.path.relpath(fn, baseoutpath))) 0198: 0199: *** 0200:copy_buildsystem(d) 0201: File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 9, function: copy_buildsystem 0005:IMAGE_FEATURES += "splash package-management x11-base x11-sato ssh-server-dropbear hwcodecs" 0006: 0007:LICENSE = "MIT" 0008: *** 0009:inherit core-image 0010: 0011:IMAGE_INSTALL += "packagegroup-core-x11-sato-games" File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call 0531: The arguments are the same as for the Popen constructor. Example: 0532: 0533: check_call(["ls", "-l"]) 0534: """ *** 0535: retcode = call(*popenargs, **kwargs) 0536: if retcode: 0537: cmd = kwargs.get("args") 0538: if cmd is None: 0539: cmd = popenargs[0] The problem is the use of "FILE" to obtain the current filename. Instead, we therefore inject the function being executed into the methodpool which allows us to correct its linenumber and filename information. We can then clearly mark the initial piece as autogenerated and the rest of the linenumber and filename information should be correct. Afterwards the trace starts: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: 0001: *** 0002:copy_buildsystem(d) 0003: File: '/media/build1/poky/meta/classes/populate_sdk_ext.bbclass', lineno: 66, function: copy_buildsystem 0062: import glob 0063: import oe.copy_buildsystem 0064: import subprocess 0065: *** 0066: subprocess.check_call("foo") 0067: 0068: oe_init_env_script = d.getVar('OE_INIT_ENV_SCRIPT', True) 0069: 0070: conf_bbpath = '' File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call 0531: The arguments are the same as for the Popen constructor. Example: 0532: 0533: check_call(["ls", "-l"]) 0534: """ *** 0535: retcode = call(*popenargs, **kwargs) 0536: if retcode: 0537: cmd = kwargs.get("args") 0538: if cmd is None: 0539: cmd = popenargs[0] We can't inject into methodpool at parsing time, since there may be _append or other override operations against the function before its execution. Signed-off-by: Richard Purdie diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py index 3039837..005ace0 100644 --- a/bitbake/lib/bb/build.py +++ b/bitbake/lib/bb/build.py @@ -229,17 +229,13 @@ def exec_func(func, d, dirs = None): exec_func_shell(func, d, runfile, cwd=adir) _functionfmt = """ -def {function}(d): -{body} - {function}(d) """ logformatter = bb.msg.BBLogFormatter("%(levelname)s: %(message)s") def exec_func_python(func, d, runfile, cwd=None): """Execute a python BB 'function'""" - bbfile = d.getVar('FILE', True) - code = _functionfmt.format(function=func, body=d.getVar(func, False)) + code = _functionfmt.format(function=func) bb.utils.mkdirhier(os.path.dirname(runfile)) with open(runfile, 'w') as script: bb.data.emit_func_python(func, script, d) @@ -254,8 +250,13 @@ def exec_func_python(func, d, runfile, cwd=None): bb.debug(2, "Executing python function %s" % func) try: - comp = utils.better_compile(code, func, bbfile) - utils.better_exec(comp, {"d": d}, code, bbfile) + text = "def %s(d):\n%s" % (func, d.getVar(func, False)) + fn = d.getVarFlag(func, "filename", False) + lineno = int(d.getVarFlag(func, "lineno", False)) + bb.methodpool.insert_method(func, text, fn, lineno - 1) + + comp = utils.better_compile(code, func, "exec_python_func() autogenerated") + utils.better_exec(comp, {"d": d}, code, "exec_python_func() autogenerated") except (bb.parse.SkipRecipe, bb.build.FuncFailed): raise except: