Openembedded Bitbake Development
 help / color / mirror / Atom feed
From: ecordonnier@snap.com
To: bitbake-devel@lists.openembedded.org
Cc: Etienne Cordonnier <ecordonnier@snap.com>
Subject: [PATCH] data_smart: Cache compiled inline Python expressions
Date: Wed, 23 Sep 2026 16:02:08 +0200	[thread overview]
Message-ID: <20260923140208.1915740-1-ecordonnier@snap.com> (raw)

From: Etienne Cordonnier <ecordonnier@snap.com>

VariableParse.python_sub() recompiles an inline Python expression on every
expansion, although parsing evaluates a small set of expressions many times.
Cache the compiled code objects keyed on the expression and synthetic
filename, so error reporting is unchanged, and bound the cache size so a
long-lived server cannot grow it indefinitely.

This reduces a cold parse of 2847 recipes (poky, meta-oe, meta-python) from
32.6s to 28.6s (-12%), averaged over four alternating runs.

AI-Generated: Uses OpenAI Codex
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
---
 lib/bb/data_smart.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
index 8aa7c1f99..cc87a665d 100644
--- a/lib/bb/data_smart.py
+++ b/lib/bb/data_smart.py
@@ -18,6 +18,7 @@ BitBake build tools.
 
 import builtins
 import copy
+import functools
 import re
 import sys
 from collections.abc import MutableMapping
@@ -30,6 +31,10 @@ from bb.COW  import COWDictBase
 
 logger = logging.getLogger("BitBake.Data")
 
+@functools.lru_cache(maxsize=8192)
+def compile_python_expression(code, varname):
+    return compile(code, varname, "eval")
+
 __setvar_keyword__ = [":append", ":prepend", ":remove"]
 __setvar_regexp__ = re.compile(r'(?P<base>.*?)(?P<keyword>:append|:prepend|:remove)(:(?P<add>[^A-Z]*))?$')
 __expand_var_regexp__ = re.compile(r"\${[a-zA-Z0-9\-_+./~:]+}")
@@ -133,7 +138,7 @@ class VariableParse:
             varname = 'Var <%s>' % self.varname
         else:
             varname = '<expansion>'
-        codeobj = compile(code.strip(), varname, "eval")
+        codeobj = compile_python_expression(code.strip(), varname)
 
         parser = bb.codeparser.PythonParser(self.varname, logger)
         parser.parse_python(code)
-- 
2.43.0



                 reply	other threads:[~2026-09-23 14:02 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260923140208.1915740-1-ecordonnier@snap.com \
    --to=ecordonnier@snap.com \
    --cc=bitbake-devel@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox