Re: [PATCH RFC 1/3] insane: Split do_package_qa into a separate task (from do_package)

[Martin Jansa <martin.jansa@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4174 bytes --]

On Thu, Jul 10, 2014 at 11:43:02PM -0300, Otavio Salvador wrote:
> Hello Richard,
> 
> (added meta-freescale mailing list as it is related to meta-fsl-arm)
> 
> On Wed, Jul 9, 2014 at 5:15 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > Its possible to run the package QA checks as a separate task rather than
> > as part of the do_package task. This offers more parallelism but the
> > fact that made me propose this is that ideally we'd like to access
> > pkgdata to help add new tests and to do that, we need to run later in
> > the task list. We also need to add in RDEPENDS to the task which apply
> > to do_package_write_* but not do_package. See the subsequent patches
> > for why this is desireable.
> >
> > If we split into a separate task, we need to add in calls to read
> > the sub package data, build the cache structure used by do_package and
> > cover the task with sstate (which is empty and just acts as a stamp
> > saying it passed package QA). We also need to handle our own
> > dependencies.
> >
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> 
> When I see a RFC serie I expect people to wait some days /for
> comments/. I learned today those are already merged and /no comment
> time/ has been given.
> 
> The problem I found is that running the QA checks in another task
> seems to break some use-cases.
> 
> In the libfslcodec (it is a binary blob provided by Freescale for
> multimedia) we have:
> 
> ...
> python populate_packages_prepend() {
>     ...
>     # FIXME: All binaries lack GNU_HASH in elf binary but as we don't have
>     # the source we cannot fix it. Disable the insane check for now.
>     for p in d.getVar('PACKAGES', True).split():
>         d.setVar("DEBIAN_NOAUTONAME_%s" % p, "1")
> 
>         if p == 'libfslcodec-test-bin':
>             # FIXME: includes the DUT .so files so we need to deploy those
>             d.appendVar("INSANE_SKIP_%s" % p, "ldflags textrel libdir")
>         else:
>             d.appendVar("INSANE_SKIP_%s" % p, "ldflags textrel")
>     ...
> }
> 
> In this case the INSANE_SKIP var is not being set on time. If we check
> the value it is being set but it seems to be too late.

FWIW: llvm in meta-oe has similar problem

python llvm_populate_packages() {
    libdir = bb.data.expand('${libdir}', d)
    libllvm_libdir = bb.data.expand('${libdir}/${LLVM_DIR}', d)
    split_dbg_packages = do_split_packages(d, libllvm_libdir+'/.debug', '^lib(.*)\.so$', 'libllvm${LLVM_RELEASE}-%s-dbg', 'Split debug package for %s', allow_dirs=True)
    split_packages = do_split_packages(d, libdir, '^lib(.*)\.so$', 'libllvm${LLVM_RELEASE}-%s', 'Split package for %s', allow_dirs=True, allow_links=True, recursive=True)
    split_staticdev_packages = do_split_packages(d, libllvm_libdir, '^lib(.*)\.a$', 'libllvm${LLVM_RELEASE}-%s-staticdev', 'Split staticdev package for %s', allow_dirs=True)   
    if split_packages:
        pn = d.getVar('PN', True)
        for package in split_packages:
            d.appendVar('INSANE_SKIP_' + package, ' dev-so')
        d.appendVar('RDEPENDS_' + pn, ' '+' '.join(split_packages))
        d.appendVar('RDEPENDS_' + pn + '-dbg', ' '+' '.join(split_dbg_packages))
        d.appendVar('RDEPENDS_' + pn + '-staticdev', ' '+' '.join(split_staticdev_packages))
}

PACKAGESPLITFUNCS_prepend = "llvm_populate_packages "

and now it fails with:

ERROR: QA Issue: non -dev/-dbg/-nativesdk package contains symlink .so: libllvm3.3-llvm-3.3 path '/work/armv5te-oe-linux-gnueabi/llvm3.3/3.3-r0/packages-split/libllvm3.3-llvm-3.3/usr/lib/libLLVM-3.3.so' [dev-so]
ERROR: QA run found fatal errors. Please consider fixing them.
ERROR: Function failed: do_package_qa
ERROR: Logfile of failure stored in: /home/jenkins/oe/world/shr-core/tmp-eglibc/work/armv5te-oe-linux-gnueabi/llvm3.3/3.3-r0/temp/log.do_package_qa.21132
NOTE: recipe llvm3.3-3.3-r0: task do_package_qa: Failed

I'll look at solution for fsl and do the same for llvm (and also possibly merge all dbg and staticdev packages into one.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]