[PATCH 1/2] package_manager: ipk: add OPKG_MAKE_INDEX_EXTRA_PARAMS variable

[PATCH 1/2] package_manager: ipk: add OPKG_MAKE_INDEX_EXTRA_PARAMS variable

[Martin Jansa]

* can be used to pass e.g. -f param to preserve user-defined fields
  in the index as added in:
  https://git.yoctoproject.org/opkg-utils/commit/opkg-make-index?id=13f6281d24e17199e0fef6c2984419372ea0f86f

* otherwise it will show a lot of messages like:
  "Lost field Author <value>"
  for every package in the feed

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
---
 meta/lib/oe/package_manager/ipk/__init__.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/package_manager/ipk/__init__.py b/meta/lib/oe/package_manager/ipk/__init__.py
index 8fcbad56aa..c35c90ce42 100644
--- a/meta/lib/oe/package_manager/ipk/__init__.py
+++ b/meta/lib/oe/package_manager/ipk/__init__.py
@@ -16,6 +16,7 @@ class OpkgIndexer(Indexer):
                      ]
 
         opkg_index_cmd = bb.utils.which(os.getenv('PATH'), "opkg-make-index")
+        opkg_index_cmd_extra_params = self.d.getVar('OPKG_MAKE_INDEX_EXTRA_PARAMS')
         if self.d.getVar('PACKAGE_FEED_SIGN') == '1':
             signer = get_signer(self.d, self.d.getVar('PACKAGE_FEED_GPG_BACKEND'))
         else:
@@ -41,8 +42,8 @@ class OpkgIndexer(Indexer):
                 if not os.path.exists(pkgs_file):
                     open(pkgs_file, "w").close()
 
-                index_cmds.add('%s --checksum md5 --checksum sha256 -r %s -p %s -m %s' %
-                                  (opkg_index_cmd, pkgs_file, pkgs_file, pkgs_dir))
+                index_cmds.add('%s --checksum md5 --checksum sha256 -r %s -p %s -m %s %s' %
+                                  (opkg_index_cmd, pkgs_file, pkgs_file, pkgs_dir, opkg_index_cmd_extra_params))
 
                 index_sign_files.add(pkgs_file)
 
-- 
2.43.0

[PATCH 2/2] package_rpm: add RPMBUILD_EXTRA_PARAMS variable

[Martin Jansa]

* e.g. for DISTROs which define extra user-defined fields with
  PACKAGE_ADD_METADATA/PACKAGE_ADD_METADATA_RPM
  as undefined fields in packagedata are fatal error for rpmbuild:
  "error: line 9: Unknown tag: Author: Unspecified"
  as shown in:
  http://errors.yoctoproject.org/Errors/Details/751706/
  with "Author" field added with:

  PACKAGE_CLASSES = "package_rpm"
  PACKAGE_ADD_AUTHOR_METADATA = "test-author"
  PACKAGE_ADD_METADATA = "Author: ${PACKAGE_ADD_AUTHOR_METADATA}"

  to fix rpm build you can use:
  RPMBUILD_EXTRA_PARAMS = " --define '_Author Author'"
  keep in mind that this doesn't cause this Author field to be
  added in .rpm, it just avoids the BUILDSPEC failure.

  and for ipk build:
  OPKG_MAKE_INDEX_EXTRA_PARAMS = "-f"

  alternatively you can avoid additional packagedata fields ending
  in the package manager (if you use them only with buildhistory
  or packagedata) with:
  PACKAGE_ADD_METADATA_RPM = ""
  PACKAGE_ADD_METADATA_IPK = ""
  PACKAGE_ADD_METADATA_DEP = ""

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
---
 meta/classes-global/package_rpm.bbclass | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/classes-global/package_rpm.bbclass b/meta/classes-global/package_rpm.bbclass
index 819ee50278..f932d2c928 100644
--- a/meta/classes-global/package_rpm.bbclass
+++ b/meta/classes-global/package_rpm.bbclass
@@ -683,6 +683,7 @@ python do_package_rpm () {
     # Setup the rpmbuild arguments...
     rpmbuild = d.getVar('RPMBUILD')
     rpmbuild_compmode = d.getVar('RPMBUILD_COMPMODE')
+    rpmbuild_extra_params = d.getVar('RPMBUILD_EXTRA_PARAMS')
 
     # Too many places in dnf stack assume that arch-independent packages are "noarch".
     # Let's not fight against this.
@@ -724,6 +725,7 @@ python do_package_rpm () {
     cmd = cmd + " --define '_use_weak_usergroup_deps 1'"
     cmd = cmd + " --define '_passwd_path " + "/completely/bogus/path" + "'"
     cmd = cmd + " --define '_group_path " + "/completely/bogus/path" + "'"
+    cmd = cmd + rpmbuild_extra_params
     if d.getVarFlag('ARCHIVER_MODE', 'srpm') == '1' and bb.data.inherits_class('archiver', d):
         cmd = cmd + " --define '_sourcedir " + d.getVar('ARCHIVER_OUTDIR') + "'"
         cmdsrpm = cmd + " --define '_srcrpmdir " + d.getVar('ARCHIVER_RPMOUTDIR') + "'"
-- 
2.43.0

Re: [OE-core] [PATCH 2/2] package_rpm: add RPMBUILD_EXTRA_PARAMS variable

[Alexandre Belloni]

On 02/02/2024 14:24:34+0100, Martin Jansa wrote:
> * e.g. for DISTROs which define extra user-defined fields with
>   PACKAGE_ADD_METADATA/PACKAGE_ADD_METADATA_RPM
>   as undefined fields in packagedata are fatal error for rpmbuild:
>   "error: line 9: Unknown tag: Author: Unspecified"
>   as shown in:
>   http://errors.yoctoproject.org/Errors/Details/751706/
>   with "Author" field added with:
> 
>   PACKAGE_CLASSES = "package_rpm"
>   PACKAGE_ADD_AUTHOR_METADATA = "test-author"
>   PACKAGE_ADD_METADATA = "Author: ${PACKAGE_ADD_AUTHOR_METADATA}"
> 
>   to fix rpm build you can use:
>   RPMBUILD_EXTRA_PARAMS = " --define '_Author Author'"
>   keep in mind that this doesn't cause this Author field to be
>   added in .rpm, it just avoids the BUILDSPEC failure.
> 
>   and for ipk build:
>   OPKG_MAKE_INDEX_EXTRA_PARAMS = "-f"
> 
>   alternatively you can avoid additional packagedata fields ending
>   in the package manager (if you use them only with buildhistory
>   or packagedata) with:
>   PACKAGE_ADD_METADATA_RPM = ""
>   PACKAGE_ADD_METADATA_IPK = ""
>   PACKAGE_ADD_METADATA_DEP = ""
> 
> Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
> ---
>  meta/classes-global/package_rpm.bbclass | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/meta/classes-global/package_rpm.bbclass b/meta/classes-global/package_rpm.bbclass
> index 819ee50278..f932d2c928 100644
> --- a/meta/classes-global/package_rpm.bbclass
> +++ b/meta/classes-global/package_rpm.bbclass
> @@ -683,6 +683,7 @@ python do_package_rpm () {
>      # Setup the rpmbuild arguments...
>      rpmbuild = d.getVar('RPMBUILD')
>      rpmbuild_compmode = d.getVar('RPMBUILD_COMPMODE')
> +    rpmbuild_extra_params = d.getVar('RPMBUILD_EXTRA_PARAMS')
>  
>      # Too many places in dnf stack assume that arch-independent packages are "noarch".
>      # Let's not fight against this.
> @@ -724,6 +725,7 @@ python do_package_rpm () {
>      cmd = cmd + " --define '_use_weak_usergroup_deps 1'"
>      cmd = cmd + " --define '_passwd_path " + "/completely/bogus/path" + "'"
>      cmd = cmd + " --define '_group_path " + "/completely/bogus/path" + "'"
> +    cmd = cmd + rpmbuild_extra_params


File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_package_rpm(d)
     0003:
File: '/home/pokybuild/yocto-worker/meta-aws/build/meta/classes-global/package_rpm.bbclass', lineno: 728, function: do_package_rpm
     0724:    cmd = cmd + " --define '_tmppath " + workdir + "'"
     0725:    cmd = cmd + " --define '_use_weak_usergroup_deps 1'"
     0726:    cmd = cmd + " --define '_passwd_path " + "/completely/bogus/path" + "'"
     0727:    cmd = cmd + " --define '_group_path " + "/completely/bogus/path" + "'"
 *** 0728:    cmd = cmd + rpmbuild_extra_params
     0729:    if d.getVarFlag('ARCHIVER_MODE', 'srpm') == '1' and bb.data.inherits_class('archiver', d):
     0730:        cmd = cmd + " --define '_sourcedir " + d.getVar('ARCHIVER_OUTDIR') + "'"
     0731:        cmdsrpm = cmd + " --define '_srcrpmdir " + d.getVar('ARCHIVER_RPMOUTDIR') + "'"
     0732:        cmdsrpm = cmdsrpm + " -bs " + outspecfile
Exception: TypeError: can only concatenate str (not "NoneType") to str

>      if d.getVarFlag('ARCHIVER_MODE', 'srpm') == '1' and bb.data.inherits_class('archiver', d):
>          cmd = cmd + " --define '_sourcedir " + d.getVar('ARCHIVER_OUTDIR') + "'"
>          cmdsrpm = cmd + " --define '_srcrpmdir " + d.getVar('ARCHIVER_RPMOUTDIR') + "'"
> -- 
> 2.43.0
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#194796): https://lists.openembedded.org/g/openembedded-core/message/194796
> Mute This Topic: https://lists.openembedded.org/mt/104118974/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH 1/2] package_manager: ipk: add OPKG_MAKE_INDEX_EXTRA_PARAMS variable

[Alex Stewart]

ACK from me.

It's a little strange that we wouldn't just have a 
PACKAGE_FEED_PRESERVE_USER_FIELDS variable. But I understand that 
wouldn't work as well for how the RPM version of this parameter is defined.

On 2/2/24 08:24, Martin Jansa via lists.openembedded.org wrote:
> * can be used to pass e.g. -f param to preserve user-defined fields
>    in the index as added in:
>    https://git.yoctoproject.org/opkg-utils/commit/opkg-make-index?id=13f6281d24e17199e0fef6c2984419372ea0f86f
>
> * otherwise it will show a lot of messages like:
>    "Lost field Author <value>"
>    for every package in the feed
>
> Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
> ---
>   meta/lib/oe/package_manager/ipk/__init__.py | 5 +++--
>   1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/meta/lib/oe/package_manager/ipk/__init__.py b/meta/lib/oe/package_manager/ipk/__init__.py
> index 8fcbad56aa..c35c90ce42 100644
> --- a/meta/lib/oe/package_manager/ipk/__init__.py
> +++ b/meta/lib/oe/package_manager/ipk/__init__.py
> @@ -16,6 +16,7 @@ class OpkgIndexer(Indexer):
>                        ]
>   
>           opkg_index_cmd = bb.utils.which(os.getenv('PATH'), "opkg-make-index")
> +        opkg_index_cmd_extra_params = self.d.getVar('OPKG_MAKE_INDEX_EXTRA_PARAMS')
>           if self.d.getVar('PACKAGE_FEED_SIGN') == '1':
>               signer = get_signer(self.d, self.d.getVar('PACKAGE_FEED_GPG_BACKEND'))
>           else:
> @@ -41,8 +42,8 @@ class OpkgIndexer(Indexer):
>                   if not os.path.exists(pkgs_file):
>                       open(pkgs_file, "w").close()
>   
> -                index_cmds.add('%s --checksum md5 --checksum sha256 -r %s -p %s -m %s' %
> -                                  (opkg_index_cmd, pkgs_file, pkgs_file, pkgs_dir))
> +                index_cmds.add('%s --checksum md5 --checksum sha256 -r %s -p %s -m %s %s' %
> +                                  (opkg_index_cmd, pkgs_file, pkgs_file, pkgs_dir, opkg_index_cmd_extra_params))
>   
>                   index_sign_files.add(pkgs_file)
>   
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#194795): https://lists.openembedded.org/g/openembedded-core/message/194795
> Mute This Topic: https://lists.openembedded.org/mt/104118971/3616788
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.stewart@ni.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

-- 
Alex Stewart
Software Engineer - NI Real-Time OS
NI (National Instruments)

alex.stewart@ni.com

Re: [OE-core] [PATCH 1/2] package_manager: ipk: add OPKG_MAKE_INDEX_EXTRA_PARAMS variable

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3097 bytes --]

Alex: thanks for review

I'll send v2 shortly, because now while testing without this variable set
at all it fails with:
opkg-make-index: error: unrecognized arguments: None
so will add weak assignment to empty before the use.

On Fri, Feb 2, 2024 at 4:40 PM Alex Stewart <alex.stewart@ni.com> wrote:

> ACK from me.
>
> It's a little strange that we wouldn't just have a
> PACKAGE_FEED_PRESERVE_USER_FIELDS variable. But I understand that
> wouldn't work as well for how the RPM version of this parameter is defined.
>
> On 2/2/24 08:24, Martin Jansa via lists.openembedded.org wrote:
> > * can be used to pass e.g. -f param to preserve user-defined fields
> >    in the index as added in:
> >
> https://git.yoctoproject.org/opkg-utils/commit/opkg-make-index?id=13f6281d24e17199e0fef6c2984419372ea0f86f
> >
> > * otherwise it will show a lot of messages like:
> >    "Lost field Author <value>"
> >    for every package in the feed
> >
> > Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
> > ---
> >   meta/lib/oe/package_manager/ipk/__init__.py | 5 +++--
> >   1 file changed, 3 insertions(+), 2 deletions(-)
> >
> > diff --git a/meta/lib/oe/package_manager/ipk/__init__.py
> b/meta/lib/oe/package_manager/ipk/__init__.py
> > index 8fcbad56aa..c35c90ce42 100644
> > --- a/meta/lib/oe/package_manager/ipk/__init__.py
> > +++ b/meta/lib/oe/package_manager/ipk/__init__.py
> > @@ -16,6 +16,7 @@ class OpkgIndexer(Indexer):
> >                        ]
> >
> >           opkg_index_cmd = bb.utils.which(os.getenv('PATH'),
> "opkg-make-index")
> > +        opkg_index_cmd_extra_params =
> self.d.getVar('OPKG_MAKE_INDEX_EXTRA_PARAMS')
> >           if self.d.getVar('PACKAGE_FEED_SIGN') == '1':
> >               signer = get_signer(self.d,
> self.d.getVar('PACKAGE_FEED_GPG_BACKEND'))
> >           else:
> > @@ -41,8 +42,8 @@ class OpkgIndexer(Indexer):
> >                   if not os.path.exists(pkgs_file):
> >                       open(pkgs_file, "w").close()
> >
> > -                index_cmds.add('%s --checksum md5 --checksum sha256 -r
> %s -p %s -m %s' %
> > -                                  (opkg_index_cmd, pkgs_file,
> pkgs_file, pkgs_dir))
> > +                index_cmds.add('%s --checksum md5 --checksum sha256 -r
> %s -p %s -m %s %s' %
> > +                                  (opkg_index_cmd, pkgs_file,
> pkgs_file, pkgs_dir, opkg_index_cmd_extra_params))
> >
> >                   index_sign_files.add(pkgs_file)
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#194795):
> https://lists.openembedded.org/g/openembedded-core/message/194795
> > Mute This Topic: https://lists.openembedded.org/mt/104118971/3616788
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> alex.stewart@ni.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>
> --
> Alex Stewart
> Software Engineer - NI Real-Time OS
> NI (National Instruments)
>
> alex.stewart@ni.com
>
>

[-- Attachment #2: Type: text/html, Size: 4721 bytes --]

Re: [OE-core] [PATCH 1/2] package_manager: ipk: add OPKG_MAKE_INDEX_EXTRA_PARAMS variable

[Alexandre Belloni]

On 02/02/2024 17:06:12+0100, Martin Jansa wrote:
> Alex: thanks for review
> 
> I'll send v2 shortly, because now while testing without this variable set
> at all it fails with:
> opkg-make-index: error: unrecognized arguments: None
> so will add weak assignment to empty before the use.

Yes, I just hit that one too..

> 
> On Fri, Feb 2, 2024 at 4:40 PM Alex Stewart <alex.stewart@ni.com> wrote:
> 
> > ACK from me.
> >
> > It's a little strange that we wouldn't just have a
> > PACKAGE_FEED_PRESERVE_USER_FIELDS variable. But I understand that
> > wouldn't work as well for how the RPM version of this parameter is defined.
> >
> > On 2/2/24 08:24, Martin Jansa via lists.openembedded.org wrote:
> > > * can be used to pass e.g. -f param to preserve user-defined fields
> > >    in the index as added in:
> > >
> > https://git.yoctoproject.org/opkg-utils/commit/opkg-make-index?id=13f6281d24e17199e0fef6c2984419372ea0f86f
> > >
> > > * otherwise it will show a lot of messages like:
> > >    "Lost field Author <value>"
> > >    for every package in the feed
> > >
> > > Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
> > > ---
> > >   meta/lib/oe/package_manager/ipk/__init__.py | 5 +++--
> > >   1 file changed, 3 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/meta/lib/oe/package_manager/ipk/__init__.py
> > b/meta/lib/oe/package_manager/ipk/__init__.py
> > > index 8fcbad56aa..c35c90ce42 100644
> > > --- a/meta/lib/oe/package_manager/ipk/__init__.py
> > > +++ b/meta/lib/oe/package_manager/ipk/__init__.py
> > > @@ -16,6 +16,7 @@ class OpkgIndexer(Indexer):
> > >                        ]
> > >
> > >           opkg_index_cmd = bb.utils.which(os.getenv('PATH'),
> > "opkg-make-index")
> > > +        opkg_index_cmd_extra_params =
> > self.d.getVar('OPKG_MAKE_INDEX_EXTRA_PARAMS')
> > >           if self.d.getVar('PACKAGE_FEED_SIGN') == '1':
> > >               signer = get_signer(self.d,
> > self.d.getVar('PACKAGE_FEED_GPG_BACKEND'))
> > >           else:
> > > @@ -41,8 +42,8 @@ class OpkgIndexer(Indexer):
> > >                   if not os.path.exists(pkgs_file):
> > >                       open(pkgs_file, "w").close()
> > >
> > > -                index_cmds.add('%s --checksum md5 --checksum sha256 -r
> > %s -p %s -m %s' %
> > > -                                  (opkg_index_cmd, pkgs_file,
> > pkgs_file, pkgs_dir))
> > > +                index_cmds.add('%s --checksum md5 --checksum sha256 -r
> > %s -p %s -m %s %s' %
> > > +                                  (opkg_index_cmd, pkgs_file,
> > pkgs_file, pkgs_dir, opkg_index_cmd_extra_params))
> > >
> > >                   index_sign_files.add(pkgs_file)
> > >
> > >
> > > 
> > >
> >
> > --
> > Alex Stewart
> > Software Engineer - NI Real-Time OS
> > NI (National Instruments)
> >
> > alex.stewart@ni.com
> >
> >

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#194806): https://lists.openembedded.org/g/openembedded-core/message/194806
> Mute This Topic: https://lists.openembedded.org/mt/104118971/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com