Re: [OE-core] [PATCH v2 2/3] devtool: upgrade: read changelog blobs via a temp file

[Daniel Turull <daniel.turull@ericsson.com>]

On Tue, 2026-08-04 at 08:02 +0200, Daniel Turull via lists.openembedded.org wrote:
> From: Daniel Turull <daniel.turull@ericsson.com>
> 
> bb.process.run() always decodes command output as UTF-8, which raises
> UnicodeDecodeError for changelogs containing non-UTF-8 bytes (e.g.
> Latin-1 author names). Add _git_show_file(), which redirects
> `git show` to a temp file and reads it back with errors='replace' to
> tolerate that, and use it for the existing per-version release notes
> lookup in _extract_changelog(), which had the same crash exposure.
> 
> AI-Generated: Kiro with Claude Sonnet 5
> Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
> ---
>  scripts/lib/devtool/upgrade.py | 16 ++++++++++++++--
>  1 file changed, 14 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/lib/devtool/upgrade.py b/scripts/lib/devtool/upgrade.py
> index 495a5b8217..6883c99cce 100644
> --- a/scripts/lib/devtool/upgrade.py
> +++ b/scripts/lib/devtool/upgrade.py
> @@ -589,6 +589,18 @@ def _resolve_rst_includes(content, srctree):
>      return ''.join(result)
>  
>  
> +def _git_show_file(srctree, ref, fname):
> +    """Return the content of fname at ref. Changelogs occasionally contain
> +    non-UTF-8 bytes (e.g. Latin-1 author names), which bb.process.run()
> +    can't handle since it always decodes command output as UTF-8;
> +    redirecting to a temp file and reading it back leniently avoids that
> +    restriction."""
> +    with tempfile.NamedTemporaryFile(prefix='devtool-changelog') as tmpf:
> +        _run('git show %s > %s' % (shlex.quote('%s:%s' % (ref, fname)), shlex.quote(tmpf.name)),
> srctree)

The alternative to use a tmp file was to modify _run to allow other encodings than utf-8, but it
will be a bigger refactor. Or not use _run and call directly the raw subprocess calls.

I can change it to either option or keep the current patch.

> +        with open(tmpf.name, 'r', errors='replace') as f:
> +            return f.read()
> +
> +
> 

Daniel