[yocto] NOT split a group kernel modules in separate packages

[yocto] NOT split a group kernel modules in separate packages

[Ferry Toth]

Hi,
I read the earlier post on how to disable package splitting entirely. 
But that is not what I want to do.

Some modules need to be packaged separate as they go into the initrd as 
well as the image. But there is a large group (800) in particular under 
kernel/drivers/iio/ that I would like in a single package as updating 
these on btrfs is very slow.

I already have a kernel recipe and was thinking to add something 
(without understanding what I am doing exactly here, just copied stuff 
from elsewhere and modified a bit) like:

python split_kernel_module_packages:prepend () {
     kernel_package_name = d.getVar("KERNEL_PACKAGE_NAME") or "kernel"
     kernel_version = d.getVar("KERNEL_VERSION")

     metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE')
     splitmods = d.getVar('KERNEL_SPLIT_MODULES')
     postinst = d.getVar('pkg_postinst:modules')
     postrm = d.getVar('pkg_postrm:modules')

     if splitmods != '1':
         d.appendVar('FILES:' + metapkg, '%s %s %s/modules' %
             (d.getVar('modulesloaddir'), d.getVar('modprobedir'), 
d.getVar("nonarch_base_libdir")))
         d.appendVar('pkg_postinst:%s' % metapkg, postinst)
         d.prependVar('pkg_postrm:%s' % metapkg, postrm);
         return

     module_pattern_prefix = d.getVar('KERNEL_MODULE_PACKAGE_PREFIX')
     module_pattern_suffix = d.getVar('KERNEL_MODULE_PACKAGE_SUFFIX')
     module_pattern = module_pattern_prefix + kernel_package_name + 
'-module-iio' + module_pattern_suffix

     modules = do_split_packages(d, 
root='${nonarch_base_libdir}/modules', file_regex=r'.*/iio(/.*)?$', 
output_pattern=module_pattern, description='iio kernel modules', 
postinst=postinst, postrm=postrm, recursive=True, allow_dirs=True, 
extra_depends='')
     if modules:
         d.appendVar('RDEPENDS:' + metapkg, ' '+' '.join(modules))
}

I was hoping this would just magically match all modules with */iio/** 
and place them in a single package. But alas.

Is it at all possible what I'm trying do?

Ideally there would of course be just 2 packages, "essential" and 
"extra" or so, similar to what Ubuntu does.

Re: [yocto] NOT split a group kernel modules in separate packages

[Ferry Toth]

Hi,

Op 06-06-2025 om 23:10 schreef Ferry Toth:
> Hi,
> I read the earlier post on how to disable package splitting entirely. 
> But that is not what I want to do.
> 
> Some modules need to be packaged separate as they go into the initrd as 
> well as the image. But there is a large group (800) in particular under 
> kernel/drivers/iio/ that I would like in a single package as updating 
> these on btrfs is very slow.
> 
> I already have a kernel recipe and was thinking to add something 
> (without understanding what I am doing exactly here, just copied stuff 
> from elsewhere and modified a bit) like:
> 
> python split_kernel_module_packages:prepend () {
>      kernel_package_name = d.getVar("KERNEL_PACKAGE_NAME") or "kernel"
>      kernel_version = d.getVar("KERNEL_VERSION")
> 
>      metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE')
>      splitmods = d.getVar('KERNEL_SPLIT_MODULES')
>      postinst = d.getVar('pkg_postinst:modules')
>      postrm = d.getVar('pkg_postrm:modules')
> 
>      if splitmods != '1':
>          d.appendVar('FILES:' + metapkg, '%s %s %s/modules' %
>              (d.getVar('modulesloaddir'), d.getVar('modprobedir'), 
> d.getVar("nonarch_base_libdir")))
>          d.appendVar('pkg_postinst:%s' % metapkg, postinst)
>          d.prependVar('pkg_postrm:%s' % metapkg, postrm);
>          return
> 
>      module_pattern_prefix = d.getVar('KERNEL_MODULE_PACKAGE_PREFIX')
>      module_pattern_suffix = d.getVar('KERNEL_MODULE_PACKAGE_SUFFIX')
>      module_pattern = module_pattern_prefix + kernel_package_name + '- 
> module-iio' + module_pattern_suffix
> 
>      modules = do_split_packages(d, root='${nonarch_base_libdir}/ 
> modules', file_regex=r'.*/iio(/.*)?$', output_pattern=module_pattern, 
> description='iio kernel modules', postinst=postinst, postrm=postrm, 
> recursive=True, allow_dirs=True, extra_depends='')
>      if modules:
>          d.appendVar('RDEPENDS:' + metapkg, ' '+' '.join(modules))
> }
> 
> I was hoping this would just magically match all modules with */iio/** 
> and place them in a single package. But alas.
> 
> Is it at all possible what I'm trying do?

Any hints much appreciated!

> Ideally there would of course be just 2 packages, "essential" and 
> "extra" or so, similar to what Ubuntu does.
> 

Re: [yocto] NOT split a group kernel modules in separate packages

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 3265 bytes --]

On Fri, Jun 6, 2025 at 5:10 PM Ferry Toth via lists.yoctoproject.org
<fntoth=gmail.com@lists.yoctoproject.org> wrote:

> Hi,
> I read the earlier post on how to disable package splitting entirely.
> But that is not what I want to do.
>
> Some modules need to be packaged separate as they go into the initrd as
> well as the image. But there is a large group (800) in particular under
> kernel/drivers/iio/ that I would like in a single package as updating
> these on btrfs is very slow.
>

There's an in-progress (under review and tweaking) series that was proposed
for a similar purpose.

https://lore.kernel.org/openembedded-core/20250404162932.447699-1-mikko.rapeli@linaro.org/

There may be some information in there that you can use to implement your
own solution while we work through something for oe-core.

Bruce


>
> I already have a kernel recipe and was thinking to add something
> (without understanding what I am doing exactly here, just copied stuff
> from elsewhere and modified a bit) like:
>
> python split_kernel_module_packages:prepend () {
>      kernel_package_name = d.getVar("KERNEL_PACKAGE_NAME") or "kernel"
>      kernel_version = d.getVar("KERNEL_VERSION")
>
>      metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE')
>      splitmods = d.getVar('KERNEL_SPLIT_MODULES')
>      postinst = d.getVar('pkg_postinst:modules')
>      postrm = d.getVar('pkg_postrm:modules')
>
>      if splitmods != '1':
>          d.appendVar('FILES:' + metapkg, '%s %s %s/modules' %
>              (d.getVar('modulesloaddir'), d.getVar('modprobedir'),
> d.getVar("nonarch_base_libdir")))
>          d.appendVar('pkg_postinst:%s' % metapkg, postinst)
>          d.prependVar('pkg_postrm:%s' % metapkg, postrm);
>          return
>
>      module_pattern_prefix = d.getVar('KERNEL_MODULE_PACKAGE_PREFIX')
>      module_pattern_suffix = d.getVar('KERNEL_MODULE_PACKAGE_SUFFIX')
>      module_pattern = module_pattern_prefix + kernel_package_name +
> '-module-iio' + module_pattern_suffix
>
>      modules = do_split_packages(d,
> root='${nonarch_base_libdir}/modules', file_regex=r'.*/iio(/.*)?$',
> output_pattern=module_pattern, description='iio kernel modules',
> postinst=postinst, postrm=postrm, recursive=True, allow_dirs=True,
> extra_depends='')
>      if modules:
>          d.appendVar('RDEPENDS:' + metapkg, ' '+' '.join(modules))
> }
>
> I was hoping this would just magically match all modules with */iio/**
> and place them in a single package. But alas.
>
> Is it at all possible what I'm trying do?
>
> Ideally there would of course be just 2 packages, "essential" and
> "extra" or so, similar to what Ubuntu does.
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#65486):
> https://lists.yoctoproject.org/g/yocto/message/65486
> Mute This Topic: https://lists.yoctoproject.org/mt/113510861/1050810
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [
> bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 5419 bytes --]

Re: [yocto] NOT split a group kernel modules in separate packages

[Ferry Toth]

Hi,

Op 07-07-2025 om 04:18 schreef Bruce Ashfield:
> 
> 
> On Fri, Jun 6, 2025 at 5:10 PM Ferry Toth via lists.yoctoproject.org 
> <http://lists.yoctoproject.org> <fntoth=gmail.com@lists.yoctoproject.org 
> <mailto:gmail.com@lists.yoctoproject.org>> wrote:
> 
>     Hi,
>     I read the earlier post on how to disable package splitting entirely.
>     But that is not what I want to do.
> 
>     Some modules need to be packaged separate as they go into the initrd as
>     well as the image. But there is a large group (800) in particular under
>     kernel/drivers/iio/ that I would like in a single package as updating
>     these on btrfs is very slow.
> 
> 
> There's an in-progress (under review and tweaking) series that was proposed
> for a similar purpose.
> 
> https://lore.kernel.org/openembedded-core/20250404162932.447699-1- 
> mikko.rapeli@linaro.org/ <https://lore.kernel.org/openembedded- 
> core/20250404162932.447699-1-mikko.rapeli@linaro.org/>

Afaiu this is about creating meta package to pull in the needed module 
packages into initrd. That is great, but as they already note, without 
it you need to pull specific module packages into the initrd. Which is 
what I do - and that list is not that long.

My main goal to have all the other kernel modules in a single package. 
As I expect `apt get install modules-extra` will be a lot faster than 
`apt get install modules*`.

> There may be some information in there that you can use to implement your
> own solution while we work through something for oe-core.
> 
> Bruce
> 
> 
>     I already have a kernel recipe and was thinking to add something
>     (without understanding what I am doing exactly here, just copied stuff
>     from elsewhere and modified a bit) like:
> 
>     python split_kernel_module_packages:prepend () {
>           kernel_package_name = d.getVar("KERNEL_PACKAGE_NAME") or "kernel"
>           kernel_version = d.getVar("KERNEL_VERSION")
> 
>           metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE')
>           splitmods = d.getVar('KERNEL_SPLIT_MODULES')
>           postinst = d.getVar('pkg_postinst:modules')
>           postrm = d.getVar('pkg_postrm:modules')
> 
>           if splitmods != '1':
>               d.appendVar('FILES:' + metapkg, '%s %s %s/modules' %
>                   (d.getVar('modulesloaddir'), d.getVar('modprobedir'),
>     d.getVar("nonarch_base_libdir")))
>               d.appendVar('pkg_postinst:%s' % metapkg, postinst)
>               d.prependVar('pkg_postrm:%s' % metapkg, postrm);
>               return
> 
>           module_pattern_prefix = d.getVar('KERNEL_MODULE_PACKAGE_PREFIX')
>           module_pattern_suffix = d.getVar('KERNEL_MODULE_PACKAGE_SUFFIX')
>           module_pattern = module_pattern_prefix + kernel_package_name +
>     '-module-iio' + module_pattern_suffix
> 
>           modules = do_split_packages(d,
>     root='${nonarch_base_libdir}/modules', file_regex=r'.*/iio(/.*)?$',
>     output_pattern=module_pattern, description='iio kernel modules',
>     postinst=postinst, postrm=postrm, recursive=True, allow_dirs=True,
>     extra_depends='')
>           if modules:
>               d.appendVar('RDEPENDS:' + metapkg, ' '+' '.join(modules))
>     }
> 
>     I was hoping this would just magically match all modules with */iio/**
>     and place them in a single package. But alas.
> 
>     Is it at all possible what I'm trying do?
> 
>     Ideally there would of course be just 2 packages, "essential" and
>     "extra" or so, similar to what Ubuntu does.
> 
>     -=-=-=-=-=-=-=-=-=-=-=-
>     Links: You receive all messages sent to this group.
>     View/Reply Online (#65486): https://lists.yoctoproject.org/g/yocto/
>     message/65486 <https://lists.yoctoproject.org/g/yocto/message/65486>
>     Mute This Topic: https://lists.yoctoproject.org/mt/113510861/1050810
>     <https://lists.yoctoproject.org/mt/113510861/1050810>
>     Group Owner: yocto+owner@lists.yoctoproject.org
>     <mailto:yocto%2Bowner@lists.yoctoproject.org>
>     Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub <https://
>     lists.yoctoproject.org/g/yocto/unsub> [bruce.ashfield@gmail.com
>     <mailto:bruce.ashfield@gmail.com>]
>     -=-=-=-=-=-=-=-=-=-=-=-
> 
> 
> 
> -- 
> - Thou shalt not follow the NULL pointer, for chaos and madness await 
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>