[OE-core][PATCH v2 1/2] package_manager/ipk: skip checksums for unsigned local feeds

[OE-core][PATCH v2 1/2] package_manager/ipk: skip checksums for unsigned local feeds

[ecordonnier]

From: Etienne Cordonnier <ecordonnier@snap.com>

Computing checksums in the opkg Packages index requires reading every
.ipk file in full.  For a large image with 7000+ packages (including
multi-gigabyte debug packages), this adds 150-300s to every do_rootfs run.

Checksums in the Packages index are only meaningful for signed feeds
(PACKAGE_FEED_SIGN=1): the GPG signature covers the Packages index
which contains the SHA256Sum of each .ipk, forming a chain of trust
that prevents tampered packages being swapped on a remote feed.

For unsigned local file:// feeds the packages are installed directly
from the build host filesystem where there is no tampering risk.  Skip
all checksum generation in that case by passing no --checksum flags to
opkg-make-index (the tool's default behaviour when given no flags).
Pass --force-checksum to opkg so it does not error on the absent
checksum fields.

On a test image with 7000+ packages (including a 2.3 GB debug
package): write_index time reduced from ~180s to ~22s (8x speedup)
when opkg-make-index is configured to produce no checksums by default.

See https://git.openembedded.org/openembedded-core/commit/?id=e462f47489f35902b6972f9837d9adfa542fc796
("Enable sha256 checksums in opkg indexer", 2019) for the original rationale.

AI-Generated: Claude Sonnet 4.6
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
---
 meta/lib/oe/package_manager/ipk/__init__.py | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oe/package_manager/ipk/__init__.py b/meta/lib/oe/package_manager/ipk/__init__.py
index 344e0852177..93a12fad6a3 100644
--- a/meta/lib/oe/package_manager/ipk/__init__.py
+++ b/meta/lib/oe/package_manager/ipk/__init__.py
@@ -12,7 +12,7 @@ from oe.package_manager import *
 from oe.package_manager.common_deb_ipk import OpkgDpkgPM
 
 class OpkgIndexer(Indexer):
-    def write_index(self):
+    def write_index(self, build_checksums=True):
         arch_vars = ["ALL_MULTILIB_PACKAGE_ARCHS",
                      "SDK_PACKAGE_ARCHS",
                      ]
@@ -44,8 +44,9 @@ class OpkgIndexer(Indexer):
                 if not os.path.exists(pkgs_file):
                     open(pkgs_file, "w").close()
 
-                index_cmds.add('%s --checksum md5 --checksum sha256 -r %s -p %s -m %s %s' %
-                                  (opkg_index_cmd, pkgs_file, pkgs_file, pkgs_dir, opkg_index_cmd_extra_params))
+                checksum_args = '--checksum md5 --checksum sha256 ' if build_checksums else ''
+                index_cmds.add('%s %s-r %s -p %s -m %s %s' %
+                                  (opkg_index_cmd, checksum_args, pkgs_file, pkgs_file, pkgs_dir, opkg_index_cmd_extra_params))
 
                 index_sign_files.add(pkgs_file)
 
@@ -103,8 +104,12 @@ class OpkgPM(OpkgDpkgPM):
         self.deploy_dir = oe.path.join(self.d.getVar('WORKDIR'), ipk_repo_workdir)
         self.deploy_lock_file = os.path.join(self.deploy_dir, "deploy.lock")
         self.opkg_cmd = bb.utils.which(os.getenv('PATH'), "opkg")
+        self.from_feeds = (self.d.getVar('BUILD_IMAGES_FROM_FEEDS') or "") == "1"
+        self._build_checksums = self.from_feeds or self.d.getVar('PACKAGE_FEED_SIGN') == '1'
         self.opkg_args = ['--volatile-cache', '-f', config_file, '-t', self.d.expand('${T}/ipktemp/'), '-o', target_rootfs]
         self.opkg_args.extend(shlex.split(self.d.getVar("OPKG_ARGS")))
+        if not self._build_checksums:
+            self.opkg_args.append('--force-checksum')
 
         if prepare_index:
             create_packages_dir(self.d, self.deploy_dir, d.getVar("DEPLOY_DIR_IPK"), "package_write_ipk", filterbydependencies)
@@ -116,7 +121,6 @@ class OpkgPM(OpkgDpkgPM):
         if not os.path.exists(self.d.expand('${T}/saved')):
             bb.utils.mkdirhier(self.d.expand('${T}/saved'))
 
-        self.from_feeds = (self.d.getVar('BUILD_IMAGES_FROM_FEEDS') or "") == "1"
         if self.from_feeds:
             self._create_custom_config()
         else:
@@ -346,7 +350,7 @@ class OpkgPM(OpkgDpkgPM):
     def write_index(self):
         self.deploy_dir_lock()
 
-        result = self.indexer.write_index()
+        result = self.indexer.write_index(build_checksums=self._build_checksums)
 
         self.deploy_dir_unlock()
 
-- 
2.43.0

[OE-core][PATCH v2 2/2] opkg: enable sha256 checksum verification by default

[ecordonnier]

From: Etienne Cordonnier <ecordonnier@snap.com>

WITH_SHA256=ON adds no external dependency: sha256.c is a self-contained
528-line pure C implementation already present in the source tree.

oe-core already generates SHA256Sum fields in the Packages index
(via --checksum sha256 in OpkgIndexer.write_index), but opkg silently
ignores them unless compiled with WITH_SHA256=ON.  Enabling sha256 by
default makes those checksums actually verified at install time,
improving package integrity checking at zero dependency cost.

Once SHA256 is universally available, --checksum md5 can also be
dropped from OpkgIndexer.write_index() -- MD5 is collision-prone and
should not be relied upon for integrity checking -- completing the
deprecation noted in https://git.openembedded.org/openembedded-core/commit/?id=e462f47489f35902b6972f9837d9adfa542fc796
(2019).

AI-Generated: Claude Sonnet 4.6
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
---
 meta/recipes-devtools/opkg/opkg_0.10.0.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/opkg/opkg_0.10.0.bb b/meta/recipes-devtools/opkg/opkg_0.10.0.bb
index debcd94a300..e3c2dd790dd 100644
--- a/meta/recipes-devtools/opkg/opkg_0.10.0.bb
+++ b/meta/recipes-devtools/opkg/opkg_0.10.0.bb
@@ -30,7 +30,7 @@ inherit cmake pkgconfig ptest
 target_localstatedir := "${localstatedir}"
 OPKGLIBDIR ??= "${target_localstatedir}/lib"
 
-PACKAGECONFIG ??= "libsolv ${@bb.utils.filter('DISTRO_FEATURES', 'acl xattr', d)}"
+PACKAGECONFIG ??= "libsolv sha256 ${@bb.utils.filter('DISTRO_FEATURES', 'acl xattr', d)}"
 
 PACKAGECONFIG[gpg] = "-DWITH_GPGME=ON,-DWITH_GPGME=OFF,\
     gnupg gpgme libgpg-error,\
-- 
2.43.0

Re: [OE-core][PATCH v2 1/2] package_manager/ipk: skip checksums for unsigned local feeds

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 1978 bytes --]

On Thu, 2026-07-30 at 11:10 +0200, Etienne Cordonnier via
lists.openembedded.org wrote:
> From: Etienne Cordonnier <ecordonnier@snap.com>
> 
> Computing checksums in the opkg Packages index requires reading every
> .ipk file in full.  For a large image with 7000+ packages (including
> multi-gigabyte debug packages), this adds 150-300s to every do_rootfs run.
> 
> Checksums in the Packages index are only meaningful for signed feeds
> (PACKAGE_FEED_SIGN=1): the GPG signature covers the Packages index
> which contains the SHA256Sum of each .ipk, forming a chain of trust
> that prevents tampered packages being swapped on a remote feed.
> 
> For unsigned local file:// feeds the packages are installed directly
> from the build host filesystem where there is no tampering risk.  Skip
> all checksum generation in that case by passing no --checksum flags to
> opkg-make-index (the tool's default behaviour when given no flags).
> Pass --force-checksum to opkg so it does not error on the absent
> checksum fields.
> 
> On a test image with 7000+ packages (including a 2.3 GB debug
> package): write_index time reduced from ~180s to ~22s (8x speedup)
> when opkg-make-index is configured to produce no checksums by default.
> 
> See https://git.openembedded.org/openembedded-core/commit/?id=e462f47489f35902b6972f9837d9adfa542fc796
> ("Enable sha256 checksums in opkg indexer", 2019) for the original rationale.
> 
> AI-Generated: Claude Sonnet 4.6
> Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>

Hi,

The package feed may be made available over a network and/or externally
signed after it has been generated, so I don't think we should disable
writing checksums into the feed by default.

How about adding a variable to control this? It can be enabled by
default in OE-core and you can disable it in a local or distro config if
you know the signatures won't be needed in your use case.

Best regards,

-- 
Paul Barker


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: [OE-core][PATCH v2 2/2] opkg: enable sha256 checksum verification by default

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 1286 bytes --]

On Thu, 2026-07-30 at 11:10 +0200, Etienne Cordonnier via
lists.openembedded.org wrote:
> From: Etienne Cordonnier <ecordonnier@snap.com>
> 
> WITH_SHA256=ON adds no external dependency: sha256.c is a self-contained
> 528-line pure C implementation already present in the source tree.
> 
> oe-core already generates SHA256Sum fields in the Packages index
> (via --checksum sha256 in OpkgIndexer.write_index), but opkg silently
> ignores them unless compiled with WITH_SHA256=ON.  Enabling sha256 by
> default makes those checksums actually verified at install time,
> improving package integrity checking at zero dependency cost.
> 
> Once SHA256 is universally available, --checksum md5 can also be
> dropped from OpkgIndexer.write_index() -- MD5 is collision-prone and
> should not be relied upon for integrity checking -- completing the
> deprecation noted in https://git.openembedded.org/openembedded-core/commit/?id=e462f47489f35902b6972f9837d9adfa542fc796
> (2019).
> 
> AI-Generated: Claude Sonnet 4.6
> Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>

Hi,

Claude likes to include irrelevant information in commit messages,
please re-write this one to focus on the relevant bits.

Otherwise this LGTM.

Best regards,

-- 
Paul Barker


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: [OE-core][PATCH v2 2/2] opkg: enable sha256 checksum verification by default

[Etienne Cordonnier]

[-- Attachment #1: Type: text/plain, Size: 1595 bytes --]

hi Paul,
I've already manually revised the commit message to include the relevant
context justifying the increase in size of opkg. What should I remove?

 Étienne

On Thu, Jul 30, 2026 at 1:52 PM Paul Barker <paul@pbarker.dev> wrote:

> On Thu, 2026-07-30 at 11:10 +0200, Etienne Cordonnier via
> lists.openembedded.org wrote:
> > From: Etienne Cordonnier <ecordonnier@snap.com>
> >
> > WITH_SHA256=ON adds no external dependency: sha256.c is a self-contained
> > 528-line pure C implementation already present in the source tree.
> >
> > oe-core already generates SHA256Sum fields in the Packages index
> > (via --checksum sha256 in OpkgIndexer.write_index), but opkg silently
> > ignores them unless compiled with WITH_SHA256=ON.  Enabling sha256 by
> > default makes those checksums actually verified at install time,
> > improving package integrity checking at zero dependency cost.
> >
> > Once SHA256 is universally available, --checksum md5 can also be
> > dropped from OpkgIndexer.write_index() -- MD5 is collision-prone and
> > should not be relied upon for integrity checking -- completing the
> > deprecation noted in
> https://git.openembedded.org/openembedded-core/commit/?id=e462f47489f35902b6972f9837d9adfa542fc796
> > (2019).
> >
> > AI-Generated: Claude Sonnet 4.6
> > Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
>
> Hi,
>
> Claude likes to include irrelevant information in commit messages,
> please re-write this one to focus on the relevant bits.
>
> Otherwise this LGTM.
>
> Best regards,
>
> --
> Paul Barker
>
>

[-- Attachment #2: Type: text/html, Size: 2381 bytes --]

Re: [OE-core][PATCH v2 1/2] package_manager/ipk: skip checksums for unsigned local feeds

[Etienne Cordonnier]

[-- Attachment #1: Type: text/plain, Size: 3419 bytes --]

Hi Paul,
my understanding is that my change does not impact the package feed.
Checksums are still always calculated for the normal package feed after my
change. There are two different code paths:

1. do_package_index (package-index.bb -> generate_index_files() ->
OpkgIndexer(d, DEPLOY_DIR_IPK).write_index()): operates on the global
DEPLOY_DIR_IPK. This is the feed that can be published, externally signed,
or served over a network. This path is not touched by this commit: it still
always writes checksums.
This is the write_index() call at
meta/lib/oe/package_manager/__init__.py:579

2. do_rootfs (OpkgPM.write_index()): operates on WORKDIR/oe-rootfs-repo, a
throwaway local copy of packages assembled exclusively for the current
rootfs build. This directory is never published or signed. This is the only
path the commit changes.
This is the write_index() call
at meta/lib/oe/package_manager/ipk/rootfs.py:281

Or are you saying that there is a use-case where
e.g. core-image-minimal/1.0/oe-rootfs-repo gets published over the network,
rather than build/tmp/deploy/ipk? I was referring to yocto_docs where the
example given is publishing packages from tmp/deploy:
$ cd bitbake-builds/build/tmp/deploy/rpm
$ python3 -m http.server

Étienne

On Thu, Jul 30, 2026 at 1:51 PM Paul Barker <paul@pbarker.dev> wrote:

> On Thu, 2026-07-30 at 11:10 +0200, Etienne Cordonnier via
> lists.openembedded.org wrote:
> > From: Etienne Cordonnier <ecordonnier@snap.com>
> >
> > Computing checksums in the opkg Packages index requires reading every
> > .ipk file in full.  For a large image with 7000+ packages (including
> > multi-gigabyte debug packages), this adds 150-300s to every do_rootfs
> run.
> >
> > Checksums in the Packages index are only meaningful for signed feeds
> > (PACKAGE_FEED_SIGN=1): the GPG signature covers the Packages index
> > which contains the SHA256Sum of each .ipk, forming a chain of trust
> > that prevents tampered packages being swapped on a remote feed.
> >
> > For unsigned local file:// feeds the packages are installed directly
> > from the build host filesystem where there is no tampering risk.  Skip
> > all checksum generation in that case by passing no --checksum flags to
> > opkg-make-index (the tool's default behaviour when given no flags).
> > Pass --force-checksum to opkg so it does not error on the absent
> > checksum fields.
> >
> > On a test image with 7000+ packages (including a 2.3 GB debug
> > package): write_index time reduced from ~180s to ~22s (8x speedup)
> > when opkg-make-index is configured to produce no checksums by default.
> >
> > See
> https://git.openembedded.org/openembedded-core/commit/?id=e462f47489f35902b6972f9837d9adfa542fc796
> > ("Enable sha256 checksums in opkg indexer", 2019) for the original
> rationale.
> >
> > AI-Generated: Claude Sonnet 4.6
> > Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
>
> Hi,
>
> The package feed may be made available over a network and/or externally
> signed after it has been generated, so I don't think we should disable
> writing checksums into the feed by default.
>
> How about adding a variable to control this? It can be enabled by
> default in OE-core and you can disable it in a local or distro config if
> you know the signatures won't be needed in your use case.
>
> Best regards,
>
> --
> Paul Barker
>
>

[-- Attachment #2: Type: text/html, Size: 4320 bytes --]