[PATCH] perl: Fix perl-module-* being ignored via COMPLEMENTARY_GLOB

[PATCH] perl: Fix perl-module-* being ignored via COMPLEMENTARY_GLOB

[William.Lyu]

From: William Lyu <William.Lyu@windriver.com>

When perl-modules is added via COMPLEMENTARY_GLOB, all perl-module-*
packages recommended by perl-modules are ignored due to the defined
behavior of COMPLEMENTARY_GLOB.

This patch changes the relationship between perl-modules and all of its
perl-module-* from RRECOMMENDS to RDEPENDS. This makes sense as
perl-modules should represent the collection of all optional
perl-module-* packages. After this patch, perl-modules itself is being
RRECOMMENDED instead of the individual perl-module-* packages
perl-modules represents.

Signed-off-by: William Lyu <William.Lyu@windriver.com>
---
 meta/recipes-devtools/perl/perl_5.38.2.bb | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-devtools/perl/perl_5.38.2.bb b/meta/recipes-devtools/perl/perl_5.38.2.bb
index a9d684cfc5..5037816f89 100644
--- a/meta/recipes-devtools/perl/perl_5.38.2.bb
+++ b/meta/recipes-devtools/perl/perl_5.38.2.bb
@@ -306,8 +306,8 @@ ALTERNATIVE_PRIORITY = "40"
 ALTERNATIVE:${PN}-doc = "Thread.3"
 ALTERNATIVE_LINK_NAME[Thread.3] = "${mandir}/man3/Thread.3"
 
-# Create a perl-modules package recommending all the other perl
-# packages (actually the non modules packages and not created too)
+# Create a perl-modules package that represents the collection of all the
+# other perl packages (actually the non modules packages and not created too).
 ALLOW_EMPTY:${PN}-modules = "1"
 PACKAGES += "${PN}-modules "
 
@@ -322,11 +322,16 @@ python split_perl_packages () {
     do_split_packages(d, libdir, r'.*linux/([^\/].*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
     do_split_packages(d, libdir, r'(^(?!(CPAN\/|CPANPLUS\/|Module\/|unicore\/|.*linux\/)[^\/]).*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
 
-    # perl-modules should recommend every perl module, and only the
+    # perl-modules should runtime-depend on every perl module, and only the
     # modules. Don't attempt to use the result of do_split_packages() as some
     # modules are manually split (eg. perl-module-unicore).
     packages = filter(lambda p: 'perl-module-' in p, d.getVar('PACKAGES').split())
-    d.setVar(d.expand("RRECOMMENDS:${PN}-modules"), ' '.join(packages))
+    d.setVar(d.expand("RDEPENDS:${PN}-modules"), ' '.join(packages))
+
+    # As perl-modules depends on all other, potentially non-existent, perl
+    # packages, we allow these packages to be empty to prevent build errors.
+    for pkg in d.getVar(d.expand("RDEPENDS:${PN}-modules")).split():
+        d.setVar(d.expand("ALLOW_EMPTY:" + pkg), "1")
 
     # Read the pre-generated dependency file, and use it to set module dependecies
     for line in open(d.expand("${WORKDIR}") + '/perl-rdepends.txt').readlines():
@@ -352,7 +357,8 @@ python() {
         d.setVar("PACKAGES_DYNAMIC", "^nativesdk-perl-module-.*")
 }
 
-RDEPENDS:${PN}-misc += "perl perl-modules"
+RDEPENDS:${PN}-misc += "perl"
+RRECOMMENDS:${PN}-misc += "perl-modules"
 RDEPENDS:${PN}-pod += "perl"
 
 BBCLASSEXTEND = "native nativesdk"
-- 
2.43.0

Re: [OE-core] [PATCH] perl: Fix perl-module-* being ignored via COMPLEMENTARY_GLOB

[Richard Purdie]

On Mon, 2024-01-22 at 07:49 -0800, Lyu, William via
lists.openembedded.org wrote:
> From: William Lyu <William.Lyu@windriver.com>
> 
> When perl-modules is added via COMPLEMENTARY_GLOB, all perl-module-*
> packages recommended by perl-modules are ignored due to the defined
> behavior of COMPLEMENTARY_GLOB.
> 
> This patch changes the relationship between perl-modules and all of its
> perl-module-* from RRECOMMENDS to RDEPENDS. This makes sense as
> perl-modules should represent the collection of all optional
> perl-module-* packages. After this patch, perl-modules itself is being
> RRECOMMENDED instead of the individual perl-module-* packages
> perl-modules represents.
> 
> Signed-off-by: William Lyu <William.Lyu@windriver.com>
> ---
>  meta/recipes-devtools/perl/perl_5.38.2.bb | 16 +++++++++++-----
>  1 file changed, 11 insertions(+), 5 deletions(-)
> 
> diff --git a/meta/recipes-devtools/perl/perl_5.38.2.bb b/meta/recipes-devtools/perl/perl_5.38.2.bb
> index a9d684cfc5..5037816f89 100644
> --- a/meta/recipes-devtools/perl/perl_5.38.2.bb
> +++ b/meta/recipes-devtools/perl/perl_5.38.2.bb
> @@ -306,8 +306,8 @@ ALTERNATIVE_PRIORITY = "40"
>  ALTERNATIVE:${PN}-doc = "Thread.3"
>  ALTERNATIVE_LINK_NAME[Thread.3] = "${mandir}/man3/Thread.3"
>  
> -# Create a perl-modules package recommending all the other perl
> -# packages (actually the non modules packages and not created too)
> +# Create a perl-modules package that represents the collection of all the
> +# other perl packages (actually the non modules packages and not created too).
>  ALLOW_EMPTY:${PN}-modules = "1"
>  PACKAGES += "${PN}-modules "
>  
> @@ -322,11 +322,16 @@ python split_perl_packages () {
>      do_split_packages(d, libdir, r'.*linux/([^\/].*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
>      do_split_packages(d, libdir, r'(^(?!(CPAN\/|CPANPLUS\/|Module\/|unicore\/|.*linux\/)[^\/]).*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
>  
> -    # perl-modules should recommend every perl module, and only the
> +    # perl-modules should runtime-depend on every perl module, and only the
>      # modules. Don't attempt to use the result of do_split_packages() as some
>      # modules are manually split (eg. perl-module-unicore).
>      packages = filter(lambda p: 'perl-module-' in p, d.getVar('PACKAGES').split())
> -    d.setVar(d.expand("RRECOMMENDS:${PN}-modules"), ' '.join(packages))
> +    d.setVar(d.expand("RDEPENDS:${PN}-modules"), ' '.join(packages))
> +
> +    # As perl-modules depends on all other, potentially non-existent, perl
> +    # packages, we allow these packages to be empty to prevent build errors.
> +    for pkg in d.getVar(d.expand("RDEPENDS:${PN}-modules")).split():
> +        d.setVar(d.expand("ALLOW_EMPTY:" + pkg), "1")
>  
>      # Read the pre-generated dependency file, and use it to set module dependecies
>      for line in open(d.expand("${WORKDIR}") + '/perl-rdepends.txt').readlines():

Do you have some examples of these packages which don't exist?

I'd like to understand why they don't exist and perhaps not have them
listed in PACKAGES in the first place. Creating a load of empty/useless
packages with ALLOW_EMPTY seems a bit pointless/sad.

Cheers,

Richard

RE: [OE-core] [PATCH] perl: Fix perl-module-* being ignored via COMPLEMENTARY_GLOB

[Lyu, William]

> On Mon, 2024-01-22 at 07:49 -0800, Lyu, William via
> lists.openembedded.org wrote:
>> From: William Lyu <William.Lyu@windriver.com>
>>
>> When perl-modules is added via COMPLEMENTARY_GLOB, all perl-module-*
>> packages recommended by perl-modules are ignored due to the defined
>> behavior of COMPLEMENTARY_GLOB.
>>
>> This patch changes the relationship between perl-modules and all of its
>> perl-module-* from RRECOMMENDS to RDEPENDS. This makes sense as
>> perl-modules should represent the collection of all optional
>> perl-module-* packages. After this patch, perl-modules itself is being
>> RRECOMMENDED instead of the individual perl-module-* packages
>> perl-modules represents.
>>
>> Signed-off-by: William Lyu <William.Lyu@windriver.com>
>> ---
>>  meta/recipes-devtools/perl/perl_5.38.2.bb | 16 +++++++++++-----
>>  1 file changed, 11 insertions(+), 5 deletions(-)
>>
>> diff --git a/meta/recipes-devtools/perl/perl_5.38.2.bb b/meta/recipes-devtools/perl/perl_5.38.2.bb
>> index a9d684cfc5..5037816f89 100644
>> --- a/meta/recipes-devtools/perl/perl_5.38.2.bb
>> +++ b/meta/recipes-devtools/perl/perl_5.38.2.bb
>> @@ -306,8 +306,8 @@ ALTERNATIVE_PRIORITY = "40"
>>  ALTERNATIVE:${PN}-doc = "Thread.3"
>>  ALTERNATIVE_LINK_NAME[Thread.3] = "${mandir}/man3/Thread.3"
>>
>> -# Create a perl-modules package recommending all the other perl
>> -# packages (actually the non modules packages and not created too)
>> +# Create a perl-modules package that represents the collection of all the
>> +# other perl packages (actually the non modules packages and not created too).
>>  ALLOW_EMPTY:${PN}-modules = "1"
>>  PACKAGES += "${PN}-modules "
>>
>> @@ -322,11 +322,16 @@ python split_perl_packages () {
>>      do_split_packages(d, libdir, r'.*linux/([^\/].*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
>>      do_split_packages(d, libdir, r'(^(?!(CPAN\/|CPANPLUS\/|Module\/|unicore\/|.*linux\/)[^\/]).*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
>>
>> -    # perl-modules should recommend every perl module, and only the
>> +    # perl-modules should runtime-depend on every perl module, and only the
>>      # modules. Don't attempt to use the result of do_split_packages() as some
>>      # modules are manually split (eg. perl-module-unicore).
>>      packages = filter(lambda p: 'perl-module-' in p, d.getVar('PACKAGES').split())
>> -    d.setVar(d.expand("RRECOMMENDS:${PN}-modules"), ' '.join(packages))
>> +    d.setVar(d.expand("RDEPENDS:${PN}-modules"), ' '.join(packages))
>> +
>> +    # As perl-modules depends on all other, potentially non-existent, perl
>> +    # packages, we allow these packages to be empty to prevent build errors.
>> +    for pkg in d.getVar(d.expand("RDEPENDS:${PN}-modules")).split():
>> +        d.setVar(d.expand("ALLOW_EMPTY:" + pkg), "1")
>>
>>      # Read the pre-generated dependency file, and use it to set module dependecies
>>      for line in open(d.expand("${WORKDIR}") + '/perl-rdepends.txt').readlines():
> 
> Do you have some examples of these packages which don't exist?
> 
> I'd like to understand why they don't exist and perhaps not have them
> listed in PACKAGES in the first place. Creating a load of empty/useless
> packages with ALLOW_EMPTY seems a bit pointless/sad.

The following is a list of empty packages perl-modules RDEPENDS on.

tmp/work/core2-64-poky-linux/perl/5.38.2/packages-split$ find -maxdepth 1 -empty -name 'perl-module-*'
./perl-module-config
./perl-module-warnings-register
./perl-module-vars
./perl-module-strict
./perl-module-config-git
./perl-module-warnings

Best, 
William

> 
> Cheers,
> 
> Richard

Re: [OE-core] [PATCH] perl: Fix perl-module-* being ignored via COMPLEMENTARY_GLOB

[Alexander Kanavin]

On Tue, 23 Jan 2024 at 00:51, Lyu, William via lists.openembedded.org
<william.lyu=windriver.com@lists.openembedded.org> wrote:
> > Do you have some examples of these packages which don't exist?
> >
> > I'd like to understand why they don't exist and perhaps not have them
> > listed in PACKAGES in the first place. Creating a load of empty/useless
> > packages with ALLOW_EMPTY seems a bit pointless/sad.
>
> The following is a list of empty packages perl-modules RDEPENDS on.
>
> tmp/work/core2-64-poky-linux/perl/5.38.2/packages-split$ find -maxdepth 1 -empty -name 'perl-module-*'
> ./perl-module-config
> ./perl-module-warnings-register
> ./perl-module-vars
> ./perl-module-strict
> ./perl-module-config-git
> ./perl-module-warnings

All of these are special-cased by the recipe to be packaged into the
main perl package (and it sets RPROVIDES for them too). As the module
package split is done with do_split_packages(), there are two options:
- adjust file regexes in calls to that function to exclude the above items
- (easier, and fine with me) set ALLOW_EMPTY only specifically for the
above list, and not all module packages.

Alex

Re: [OE-core] [PATCH] perl: Fix perl-module-* being ignored via COMPLEMENTARY_GLOB

[Richard Purdie]

On Tue, 2024-01-23 at 10:35 +0100, Alexander Kanavin wrote:
> On Tue, 23 Jan 2024 at 00:51, Lyu, William via lists.openembedded.org
> <william.lyu=windriver.com@lists.openembedded.org> wrote:
> > > Do you have some examples of these packages which don't exist?
> > > 
> > > I'd like to understand why they don't exist and perhaps not have them
> > > listed in PACKAGES in the first place. Creating a load of empty/useless
> > > packages with ALLOW_EMPTY seems a bit pointless/sad.
> > 
> > The following is a list of empty packages perl-modules RDEPENDS on.
> > 
> > tmp/work/core2-64-poky-linux/perl/5.38.2/packages-split$ find -maxdepth 1 -empty -name 'perl-module-*'
> > ./perl-module-config
> > ./perl-module-warnings-register
> > ./perl-module-vars
> > ./perl-module-strict
> > ./perl-module-config-git
> > ./perl-module-warnings
> 
> All of these are special-cased by the recipe to be packaged into the
> main perl package (and it sets RPROVIDES for them too). As the module
> package split is done with do_split_packages(), there are two options:
> - adjust file regexes in calls to that function to exclude the above items
> - (easier, and fine with me) set ALLOW_EMPTY only specifically for the
> above list, and not all module packages.

I have a third option. After calling do_split_packages(), iterate
through RPROVIDES_${PN} and for earch perl-module- item, remove any
matching entry from PACKAGES. Setting PACKAGES is basically what
do_split_packages does so we can tweak it afterwards.

If we remove the items from PACKAGES, the RDEPENDS for perl-modules
will then be correct and we don't have bogus empty packages lying
around.

Cheers,

Richard

[PATCH] perl: Fix perl-module-* being ignored via COMPLEMENTARY_GLOB

[William.Lyu]

From: William Lyu <William.Lyu@windriver.com>

When perl-modules is added via COMPLEMENTARY_GLOB, all perl-module-*
packages recommended by perl-modules are ignored due to the defined
behavior of COMPLEMENTARY_GLOB.

This patch changes the relationship between perl-modules and all of its
perl-module-* from RRECOMMENDS to RDEPENDS. This makes sense as
perl-modules should represent the collection of all optional
perl-module-* packages. After this patch, perl-modules itself is being
RRECOMMENDED instead of the individual perl-module-* packages
perl-modules represents.

Signed-off-by: William Lyu <William.Lyu@windriver.com>
---
 meta/recipes-devtools/perl/perl_5.38.2.bb | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/meta/recipes-devtools/perl/perl_5.38.2.bb b/meta/recipes-devtools/perl/perl_5.38.2.bb
index a9d684cfc5..329652a718 100644
--- a/meta/recipes-devtools/perl/perl_5.38.2.bb
+++ b/meta/recipes-devtools/perl/perl_5.38.2.bb
@@ -306,8 +306,8 @@ ALTERNATIVE_PRIORITY = "40"
 ALTERNATIVE:${PN}-doc = "Thread.3"
 ALTERNATIVE_LINK_NAME[Thread.3] = "${mandir}/man3/Thread.3"
 
-# Create a perl-modules package recommending all the other perl
-# packages (actually the non modules packages and not created too)
+# Create a perl-modules package that represents the collection of all the
+# other perl packages (actually the non modules packages and not created too).
 ALLOW_EMPTY:${PN}-modules = "1"
 PACKAGES += "${PN}-modules "
 
@@ -322,11 +322,15 @@ python split_perl_packages () {
     do_split_packages(d, libdir, r'.*linux/([^\/].*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
     do_split_packages(d, libdir, r'(^(?!(CPAN\/|CPANPLUS\/|Module\/|unicore\/|.*linux\/)[^\/]).*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
 
-    # perl-modules should recommend every perl module, and only the
+    # perl-modules should runtime-depend on every perl module, and only the
     # modules. Don't attempt to use the result of do_split_packages() as some
-    # modules are manually split (eg. perl-module-unicore).
-    packages = filter(lambda p: 'perl-module-' in p, d.getVar('PACKAGES').split())
-    d.setVar(d.expand("RRECOMMENDS:${PN}-modules"), ' '.join(packages))
+    # modules are manually split (eg. perl-module-unicore). Also, the split
+    # packages should not include packages defined in RPROVIDES:${PN}. In
+    # addition, "perl-module-config-git" is not a valid package as
+    # Config_git.pl is listed under FILES:${PN}.
+    empty_module_pkgs = d.getVar(d.expand("RPROVIDES:${PN}")).split() + ['perl-module-config-git']
+    packages = filter(lambda p: 'perl-module-' in p and p not in empty_module_pkgs, d.getVar('PACKAGES').split())
+    d.setVar(d.expand("RDEPENDS:${PN}-modules"), ' '.join(packages))
 
     # Read the pre-generated dependency file, and use it to set module dependecies
     for line in open(d.expand("${WORKDIR}") + '/perl-rdepends.txt').readlines():
@@ -352,7 +356,8 @@ python() {
         d.setVar("PACKAGES_DYNAMIC", "^nativesdk-perl-module-.*")
 }
 
-RDEPENDS:${PN}-misc += "perl perl-modules"
+RDEPENDS:${PN}-misc += "perl"
+RRECOMMENDS:${PN}-misc += "perl-modules"
 RDEPENDS:${PN}-pod += "perl"
 
 BBCLASSEXTEND = "native nativesdk"
-- 
2.43.0

[PATCH] perl: Fix perl-module-* being ignored via COMPLEMENTARY_GLOB

[William.Lyu]

From: William Lyu <William.Lyu@windriver.com>

When perl-modules is added via COMPLEMENTARY_GLOB, all perl-module-*
packages recommended by perl-modules are ignored due to the defined
behavior of COMPLEMENTARY_GLOB.

This patch changes the relationship between perl-modules and all of its
perl-module-* from RRECOMMENDS to RDEPENDS. This makes sense as
perl-modules should represent the collection of all optional
perl-module-* packages. After this patch, perl-modules itself is being
RRECOMMENDED instead of the individual perl-module-* packages
perl-modules represents.

Signed-off-by: William Lyu <William.Lyu@windriver.com>
---
 meta/recipes-devtools/perl/perl_5.38.2.bb | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/meta/recipes-devtools/perl/perl_5.38.2.bb b/meta/recipes-devtools/perl/perl_5.38.2.bb
index a9d684cfc5..b6c9cda7ae 100644
--- a/meta/recipes-devtools/perl/perl_5.38.2.bb
+++ b/meta/recipes-devtools/perl/perl_5.38.2.bb
@@ -273,7 +273,7 @@ FILES:${PN} = "${bindir}/perl ${bindir}/perl.real ${bindir}/perl${PV} ${libdir}/
                ${libdir}/perl5/${PV}/ExtUtils/typemap \
                "
 RPROVIDES:${PN} += "perl-module-strict perl-module-vars perl-module-config perl-module-warnings \
-                    perl-module-warnings-register"
+                    perl-module-warnings-register perl-module-config-git"
 
 FILES:${PN}-staticdev:append = " ${libdir}/perl5/${PV}/*/CORE/libperl.a"
 
@@ -306,8 +306,8 @@ ALTERNATIVE_PRIORITY = "40"
 ALTERNATIVE:${PN}-doc = "Thread.3"
 ALTERNATIVE_LINK_NAME[Thread.3] = "${mandir}/man3/Thread.3"
 
-# Create a perl-modules package recommending all the other perl
-# packages (actually the non modules packages and not created too)
+# Create a perl-modules package that represents the collection of all the
+# other perl packages (actually the non modules packages and not created too).
 ALLOW_EMPTY:${PN}-modules = "1"
 PACKAGES += "${PN}-modules "
 
@@ -322,11 +322,13 @@ python split_perl_packages () {
     do_split_packages(d, libdir, r'.*linux/([^\/].*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
     do_split_packages(d, libdir, r'(^(?!(CPAN\/|CPANPLUS\/|Module\/|unicore\/|.*linux\/)[^\/]).*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
 
-    # perl-modules should recommend every perl module, and only the
+    # perl-modules should runtime-depend on every perl module, and only the
     # modules. Don't attempt to use the result of do_split_packages() as some
-    # modules are manually split (eg. perl-module-unicore).
-    packages = filter(lambda p: 'perl-module-' in p, d.getVar('PACKAGES').split())
-    d.setVar(d.expand("RRECOMMENDS:${PN}-modules"), ' '.join(packages))
+    # modules are manually split (eg. perl-module-unicore). Also, the split
+    # packages should not include packages defined in RPROVIDES:${PN}.
+    perl_sub_pkgs = d.getVar(d.expand("RPROVIDES:${PN}")).split()
+    packages = filter(lambda p: 'perl-module-' in p and p not in perl_sub_pkgs, d.getVar('PACKAGES').split())
+    d.setVar(d.expand("RDEPENDS:${PN}-modules"), ' '.join(packages))
 
     # Read the pre-generated dependency file, and use it to set module dependecies
     for line in open(d.expand("${WORKDIR}") + '/perl-rdepends.txt').readlines():
@@ -352,7 +354,8 @@ python() {
         d.setVar("PACKAGES_DYNAMIC", "^nativesdk-perl-module-.*")
 }
 
-RDEPENDS:${PN}-misc += "perl perl-modules"
+RDEPENDS:${PN}-misc += "perl"
+RRECOMMENDS:${PN}-misc += "perl-modules"
 RDEPENDS:${PN}-pod += "perl"
 
 BBCLASSEXTEND = "native nativesdk"
-- 
2.43.0