Openembedded Core Discussions
 help / color / mirror / Atom feed
From: <daniel.turull@ericsson.com>
To: <openembedded-core@lists.openembedded.org>
Cc: Daniel Turull <daniel.turull@ericsson.com>
Subject: [PATCH v2 2/3] devtool: upgrade: read changelog blobs via a temp file
Date: Tue, 4 Aug 2026 08:02:04 +0200	[thread overview]
Message-ID: <20260804060205.179715-3-daniel.turull@ericsson.com> (raw)
In-Reply-To: <20260804060205.179715-1-daniel.turull@ericsson.com>

From: Daniel Turull <daniel.turull@ericsson.com>

bb.process.run() always decodes command output as UTF-8, which raises
UnicodeDecodeError for changelogs containing non-UTF-8 bytes (e.g.
Latin-1 author names). Add _git_show_file(), which redirects
`git show` to a temp file and reads it back with errors='replace' to
tolerate that, and use it for the existing per-version release notes
lookup in _extract_changelog(), which had the same crash exposure.

AI-Generated: Kiro with Claude Sonnet 5
Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
---
 scripts/lib/devtool/upgrade.py | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/devtool/upgrade.py b/scripts/lib/devtool/upgrade.py
index 495a5b8217..6883c99cce 100644
--- a/scripts/lib/devtool/upgrade.py
+++ b/scripts/lib/devtool/upgrade.py
@@ -589,6 +589,18 @@ def _resolve_rst_includes(content, srctree):
     return ''.join(result)
 
 
+def _git_show_file(srctree, ref, fname):
+    """Return the content of fname at ref. Changelogs occasionally contain
+    non-UTF-8 bytes (e.g. Latin-1 author names), which bb.process.run()
+    can't handle since it always decodes command output as UTF-8;
+    redirecting to a temp file and reading it back leniently avoids that
+    restriction."""
+    with tempfile.NamedTemporaryFile(prefix='devtool-changelog') as tmpf:
+        _run('git show %s > %s' % (shlex.quote('%s:%s' % (ref, fname)), shlex.quote(tmpf.name)), srctree)
+        with open(tmpf.name, 'r', errors='replace') as f:
+            return f.read()
+
+
 def _extract_changelog(srctree, pn, old_ver, new_ver, old_tag, new_tag, workspace_path, is_git_source):
     """Extract changelog between old and new version using devtool git tags."""
     changelog_content = None
@@ -612,10 +624,10 @@ def _extract_changelog(srctree, pn, old_ver, new_ver, old_tag, new_tag, workspac
                 continue
             if re.search(r'(releas|relnote|change|news|migrat)', fname, re.IGNORECASE):
                 try:
-                    file_content, _ = _run('git show %s' % shlex.quote('%s:%s' % (new_tag, fname)), srctree)
+                    file_content = _git_show_file(srctree, new_tag, fname)
                 except bb.process.ExecutionError:
                     try:
-                        file_content, _ = _run('git show %s' % shlex.quote('%s:%s' % (old_tag, fname)), srctree)
+                        file_content = _git_show_file(srctree, old_tag, fname)
                     except bb.process.ExecutionError:
                         continue
                 if file_content.strip():


  parent reply	other threads:[~2026-08-04  6:02 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-04  6:02 [PATCH v2 0/3] devtool: upgrade: improve changelog extraction for git-log-style ChangeLogs daniel.turull
2026-08-04  6:02 ` [PATCH v2 1/3] devtool: upgrade: ignore changelogs from 3rd party daniel.turull
2026-08-04  6:02 ` daniel.turull [this message]
2026-08-04  6:02 ` [PATCH v2 3/3] devtool: upgrade: diff git-log-style changelogs by commit hash daniel.turull
     [not found] ` <18C884DEFF3686EE.1990914@lists.openembedded.org>
2026-08-04  6:06   ` [OE-core] [PATCH v2 2/3] devtool: upgrade: read changelog blobs via a temp file Daniel Turull
2026-08-06 14:05     ` Ross Burton
2026-08-06 14:09       ` Daniel Turull

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260804060205.179715-3-daniel.turull@ericsson.com \
    --to=daniel.turull@ericsson.com \
    --cc=openembedded-core@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox