Re: [OE-core] [PATCH] lib/oe/package: fix LOCALE_PATHS scan to create locale packages

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Mon, 2024-02-19 at 16:19 +0000, Jonathan GUILLOT wrote:
> split_locales() must only check subdirectories in paths added to
> LOCALE_PATHS to avoid creating weird packages based on filenames also
> present in paths.
> 
> Without such a filter, cups recipe adding ${datadir}/cups/templates
> to
> LOCALE_PATHS creates the following incorrect packages:
> - cups-locale-add-class.tmpl
> - cups-locale-add-printer.tmpl
> - cups-locale-admin.tmpl
> 
> Signed-off-by: Jonathan GUILLOT <jonathan@joggee.fr>
> ---
>  meta/conf/documentation.conf |  2 +-
>  meta/lib/oe/package.py       | 14 +++++++++-----
>  2 files changed, 10 insertions(+), 6 deletions(-)
> 
> diff --git a/meta/conf/documentation.conf
> b/meta/conf/documentation.conf
> index 90d8e82932..b0591881ba 100644
> --- a/meta/conf/documentation.conf
> +++ b/meta/conf/documentation.conf
> @@ -271,7 +271,7 @@ LICENSE_PATH[doc] = "Path to additional licenses
> used during the build."
>  LINUX_KERNEL_TYPE[doc] = "Defines the kernel type to be used in
> assembling the configuration."
>  LINUX_VERSION[doc] = "The Linux version from kernel.org on which the
> Linux kernel image being built using the OpenEmbedded build system is
> based. You define this variable in the kernel recipe."
>  LINUX_VERSION_EXTENSION[doc] = "A string extension compiled into the
> version string of the Linux kernel built with the OpenEmbedded build
> system. You define this variable in the kernel recipe."
> -LOCALE_PATHS[doc] = "Whitespace separated list of paths that are
> scanned to construct locale packages. The list already contains
> ${datadir}/locale by default."
> +LOCALE_PATHS[doc] = "Whitespace separated list of paths that are
> scanned to construct locale packages. The list already contains
> ${datadir}/locale by default. Note that all subdirectories in these
> paths are assumed to be locales."
>  LOCALE_UTF8_IS_DEFAULT[doc] = "If set, locale names are renamed such
> that those lacking an explicit encoding (e.g. en_US) will always be
> UTF-8, and non-UTF-8 encodings are renamed to, e.g., en_US.ISO-8859-
> 1. Otherwise, the encoding is specified by glibc's SUPPORTED file.
> Not supported for precompiled locales."
>  LOG_DIR[doc] = "Specifies the directory to which the OpenEmbedded
> build system writes overall log files. The default directory is
> ${TMPDIR}/log"
>  
> diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
> index d1738d3b61..587810bdaf 100644
> --- a/meta/lib/oe/package.py
> +++ b/meta/lib/oe/package.py
> @@ -652,11 +652,15 @@ def split_locales(d):
>      locales = set()
>      for localepath in (d.getVar('LOCALE_PATHS') or "").split():
>          localedir = dvar + localepath
> -        if cpath.isdir(localedir):
> -            locales.update(os.listdir(localedir))
> -            localepaths.append(localepath)
> -        else:
> -            bb.debug(1, "No locale files in %s" % localepath)
> +        if not cpath.isdir(localedir):
> +            bb.debug(1, 'No locale files in %s' % localepath)
> +            continue
> +
> +        localepaths.append(localepath)
> +        with os.scandir(localedir) as it:
> +            for entry in it:
> +                if entry.is_dir():
> +                    locales.add(entry.name)
>  
>      if len(locales) == 0:
>          bb.debug(1, "No locale files in this package")
> 
> base-commit: f5e6183b9557477bef74024a587de0bfcc2b7c0d

Unfortunately this breaks glibc-locale:

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/6379/steps/14/logs/stdio

ERROR: glibc-locale-2.39+git-r0 do_package: QA Issue: glibc-locale: Files/directories were installed but not shipped in any package:
  /usr/share/locale/locale.alias
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.

Cheers,

Richard