From: ecordonnier@snap.com
To: bitbake-devel@lists.openembedded.org
Cc: Etienne Cordonnier <ecordonnier@snap.com>
Subject: [PATCH] data_smart: Cache compiled inline Python expressions
Date: Wed, 23 Sep 2026 16:02:08 +0200 [thread overview]
Message-ID: <20260923140208.1915740-1-ecordonnier@snap.com> (raw)
From: Etienne Cordonnier <ecordonnier@snap.com>
VariableParse.python_sub() recompiles an inline Python expression on every
expansion, although parsing evaluates a small set of expressions many times.
Cache the compiled code objects keyed on the expression and synthetic
filename, so error reporting is unchanged, and bound the cache size so a
long-lived server cannot grow it indefinitely.
This reduces a cold parse of 2847 recipes (poky, meta-oe, meta-python) from
32.6s to 28.6s (-12%), averaged over four alternating runs.
AI-Generated: Uses OpenAI Codex
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
---
lib/bb/data_smart.py | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
index 8aa7c1f99..cc87a665d 100644
--- a/lib/bb/data_smart.py
+++ b/lib/bb/data_smart.py
@@ -18,6 +18,7 @@ BitBake build tools.
import builtins
import copy
+import functools
import re
import sys
from collections.abc import MutableMapping
@@ -30,6 +31,10 @@ from bb.COW import COWDictBase
logger = logging.getLogger("BitBake.Data")
+@functools.lru_cache(maxsize=8192)
+def compile_python_expression(code, varname):
+ return compile(code, varname, "eval")
+
__setvar_keyword__ = [":append", ":prepend", ":remove"]
__setvar_regexp__ = re.compile(r'(?P<base>.*?)(?P<keyword>:append|:prepend|:remove)(:(?P<add>[^A-Z]*))?$')
__expand_var_regexp__ = re.compile(r"\${[a-zA-Z0-9\-_+./~:]+}")
@@ -133,7 +138,7 @@ class VariableParse:
varname = 'Var <%s>' % self.varname
else:
varname = '<expansion>'
- codeobj = compile(code.strip(), varname, "eval")
+ codeobj = compile_python_expression(code.strip(), varname)
parser = bb.codeparser.PythonParser(self.varname, logger)
parser.parse_python(code)
--
2.43.0
reply other threads:[~2026-09-23 14:02 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260923140208.1915740-1-ecordonnier@snap.com \
--to=ecordonnier@snap.com \
--cc=bitbake-devel@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox