From: <daniel.turull@ericsson.com>
To: <openembedded-core@lists.openembedded.org>
Cc: Daniel Turull <daniel.turull@ericsson.com>
Subject: [PATCH v2 2/3] devtool: upgrade: read changelog blobs via a temp file
Date: Tue, 4 Aug 2026 08:02:04 +0200 [thread overview]
Message-ID: <20260804060205.179715-3-daniel.turull@ericsson.com> (raw)
In-Reply-To: <20260804060205.179715-1-daniel.turull@ericsson.com>
From: Daniel Turull <daniel.turull@ericsson.com>
bb.process.run() always decodes command output as UTF-8, which raises
UnicodeDecodeError for changelogs containing non-UTF-8 bytes (e.g.
Latin-1 author names). Add _git_show_file(), which redirects
`git show` to a temp file and reads it back with errors='replace' to
tolerate that, and use it for the existing per-version release notes
lookup in _extract_changelog(), which had the same crash exposure.
AI-Generated: Kiro with Claude Sonnet 5
Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
---
scripts/lib/devtool/upgrade.py | 16 ++++++++++++++--
1 file changed, 14 insertions(+), 2 deletions(-)
diff --git a/scripts/lib/devtool/upgrade.py b/scripts/lib/devtool/upgrade.py
index 495a5b8217..6883c99cce 100644
--- a/scripts/lib/devtool/upgrade.py
+++ b/scripts/lib/devtool/upgrade.py
@@ -589,6 +589,18 @@ def _resolve_rst_includes(content, srctree):
return ''.join(result)
+def _git_show_file(srctree, ref, fname):
+ """Return the content of fname at ref. Changelogs occasionally contain
+ non-UTF-8 bytes (e.g. Latin-1 author names), which bb.process.run()
+ can't handle since it always decodes command output as UTF-8;
+ redirecting to a temp file and reading it back leniently avoids that
+ restriction."""
+ with tempfile.NamedTemporaryFile(prefix='devtool-changelog') as tmpf:
+ _run('git show %s > %s' % (shlex.quote('%s:%s' % (ref, fname)), shlex.quote(tmpf.name)), srctree)
+ with open(tmpf.name, 'r', errors='replace') as f:
+ return f.read()
+
+
def _extract_changelog(srctree, pn, old_ver, new_ver, old_tag, new_tag, workspace_path, is_git_source):
"""Extract changelog between old and new version using devtool git tags."""
changelog_content = None
@@ -612,10 +624,10 @@ def _extract_changelog(srctree, pn, old_ver, new_ver, old_tag, new_tag, workspac
continue
if re.search(r'(releas|relnote|change|news|migrat)', fname, re.IGNORECASE):
try:
- file_content, _ = _run('git show %s' % shlex.quote('%s:%s' % (new_tag, fname)), srctree)
+ file_content = _git_show_file(srctree, new_tag, fname)
except bb.process.ExecutionError:
try:
- file_content, _ = _run('git show %s' % shlex.quote('%s:%s' % (old_tag, fname)), srctree)
+ file_content = _git_show_file(srctree, old_tag, fname)
except bb.process.ExecutionError:
continue
if file_content.strip():
next prev parent reply other threads:[~2026-08-04 6:02 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-04 6:02 [PATCH v2 0/3] devtool: upgrade: improve changelog extraction for git-log-style ChangeLogs daniel.turull
2026-08-04 6:02 ` [PATCH v2 1/3] devtool: upgrade: ignore changelogs from 3rd party daniel.turull
2026-08-04 6:02 ` daniel.turull [this message]
2026-08-04 6:02 ` [PATCH v2 3/3] devtool: upgrade: diff git-log-style changelogs by commit hash daniel.turull
[not found] ` <18C884DEFF3686EE.1990914@lists.openembedded.org>
2026-08-04 6:06 ` [OE-core] [PATCH v2 2/3] devtool: upgrade: read changelog blobs via a temp file Daniel Turull
2026-08-06 14:05 ` Ross Burton
2026-08-06 14:09 ` Daniel Turull
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260804060205.179715-3-daniel.turull@ericsson.com \
--to=daniel.turull@ericsson.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox