Re: [PATCH] hwloc: Add

[Haris Okanovic <haris.okanovic@ni.com>]

Thanks for looking over the patch. I'll post a V2 shortly to address 
these issues.

On 12/25/2016 05:55 PM, Martin Jansa wrote:
> Missing dependencies and PACKAGES_prepend breaks hwloc-dbg, because of missing space at the end of the prepended value.
>
>
>
> hwloc-1.11.5: hwloc: Files/directories were installed but not shipped in any package:
>   /usr/src
>   /usr/src/debug
>   /usr/src/debug/hwloc
>   /usr/src/debug/hwloc/1.11.5-r0
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5
>   /usr/src/debug/hwloc/1.11.5-r0/build
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/lstopo
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/lstopo/lstopo.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/lstopo/lstopo.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/lstopo/lstopo-xml.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/lstopo/lstopo-text.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/lstopo/lstopo-fig.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/lstopo/lstopo-draw.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/lstopo/lstopo-color.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/lstopo/lstopo-cairo.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/lstopo/lstopo-ascii.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/misc.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/hwloc-ps.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/hwloc-patch.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/hwloc-info.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/hwloc-dump-hwdata.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/hwloc-dump-hwdata-knl.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/hwloc-distrib.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/hwloc-distances.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/hwloc-diff.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/hwloc-calc.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/hwloc-calc.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/hwloc-bind.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/hwloc-assembler.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/utils/hwloc/hwloc-annotate.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/traversal.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/topology.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/topology-xml.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/topology-xml-nolibxml.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/topology-xml-libxml.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/topology-x86.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/topology-synthetic.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/topology-pci.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/topology-noos.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/topology-linux.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/topology-hardwired.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/topology-custom.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/pci-common.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/misc.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/distances.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/diff.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/components.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/bitmap.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/bind.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/src/base64.c
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/hwloc.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/private
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/hwloc
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/private/xml.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/private/private.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/private/misc.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/private/debug.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/private/cpuid-x86.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/private/components.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/hwloc/plugins.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/hwloc/linux.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/hwloc/inlines.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/hwloc/helper.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/hwloc/diff.h
>   /usr/src/debug/hwloc/1.11.5-r0/hwloc-1.11.5/include/hwloc/bitmap.h
>   /usr/src/debug/hwloc/1.11.5-r0/build/src
>   /usr/src/debug/hwloc/1.11.5-r0/build/include
>   /usr/src/debug/hwloc/1.11.5-r0/build/src/static-components.h
>   /usr/src/debug/hwloc/1.11.5-r0/build/include/hwloc
>   /usr/src/debug/hwloc/1.11.5-r0/build/include/hwloc/autogen
>   /usr/src/debug/hwloc/1.11.5-r0/build/include/hwloc/autogen/config.h
> Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
> hwloc: 75 installed and not shipped files. [installed-vs-shipped]
> hwloc-1.11.5: hwloc rdepends on numactl, but it isn't a build dependency, missing numactl in DEPENDS or PACKAGECONFIG? [build-deps]
> hwloc-1.11.5: hwloc rdepends on cairo, but it isn't a build dependency, missing cairo in DEPENDS or PACKAGECONFIG? [build-deps]
> hwloc-1.11.5: hwloc rdepends on libpciaccess, but it isn't a build dependency, missing libpciaccess in DEPENDS or PACKAGECONFIG? [build-deps]
> hwloc-1.11.5: hwloc rdepends on libx11, but it isn't a build dependency, missing libx11 in DEPENDS or PACKAGECONFIG? [build-deps]
> hwloc-1.11.5: hwloc rdepends on libcap, but it isn't a build dependency, missing libcap in DEPENDS or PACKAGECONFIG? [build-deps]
> hwloc-1.11.5: hwloc rdepends on zlib, but it isn't a build dependency, missing zlib in DEPENDS or PACKAGECONFIG? [build-deps]
> hwloc-1.11.5: hwloc rdepends on ncurses-libtinfo, but it isn't a build dependency, missing ncurses in DEPENDS or PACKAGECONFIG? [build-deps]
> hwloc-1.11.5: hwloc rdepends on libxml2, but it isn't a build dependency, missing libxml2 in DEPENDS or PACKAGECONFIG? [build-deps]
> hwloc-1.11.5: hwloc rdepends on libudev, but it isn't a build dependency, missing systemd in DEPENDS or PACKAGECONFIG? [build-deps]
>

I'm not really sure what happened here. `bitbake -e` shows a properly 
delimited PACKAGES when I build:

PACKAGES="libhwloc hwloc-dbg hwloc-staticdev [...]"

However, my distro has some core patches which might affect this 
particular variable. I'll pad with spaces as you suggested.

>
> On Thu, Dec 22, 2016 at 11:56 PM, Khem Raj <raj.khem@gmail.com
> <mailto:raj.khem@gmail.com>> wrote:
>
>     On Thu, Dec 22, 2016 at 2:35 PM, Christopher Larson
>     <clarson@kergoth.com <mailto:clarson@kergoth.com>> wrote:
>     > On Thu, Dec 22, 2016 at 2:34 PM, Martin Jansa
>     <martin.jansa@gmail.com <mailto:martin.jansa@gmail.com>>
>     > wrote:
>     >
>     >> SUMMARY is for short summary, put long one into DESCRIPTION and
>     create
>     >> short one for SUMMARY.
>     >>
>     >> Also missing sha265sum, so I doubt it was tested properly.

Andre pointed out that "a single hash doesn't generate a warning" in a 
separate thread. However, I added a sha256sum anyway since it seems 
pretty common.

>     >>
>     >> On Thu, Dec 22, 2016 at 9:02 PM, Haris Okanovic
>     <haris.okanovic@ni.com <mailto:haris.okanovic@ni.com>>
>     >> wrote:
>     >>
>     >> > https://www.open-mpi.org/projects/hwloc/
>     <https://www.open-mpi.org/projects/hwloc/>
>     >> >
>     >> > "The Portable Hardware Locality (hwloc) software package provides a
>     >> > portable abstraction (across OS, versions, architectures, ...)
>     of the
>     >> > hierarchical topology of modern architectures, including NUMA
>     memory
>     >> > nodes, sockets, shared caches, cores and simultaneous
>     multithreading.
>     >> > It also gathers various system attributes such as cache and memory
>     >> > information as well as the locality of I/O devices such as network
>     >> > interfaces, InfiniBand HCAs or GPUs."
>     >> >
>     >> > This recipe adds libhwloc and lstopo utility as two packages.
>     >> >
>     >> > Testing: Built in OE/Krogoth. Installed to x86_64 machine.
>     Successfully
>     >> > ran `lstopo` and verified it printed correct CPU and cache info.
>     >> >
>     >> > Signed-off-by: Haris Okanovic <haris.okanovic@ni.com
>     <mailto:haris.okanovic@ni.com>>
>     >> > ---
>     >> >  meta-oe/recipes-extended/hwloc/hwloc_1.11.5.bb
>     <http://hwloc_1.11.5.bb> | 25
>     >> > +++++++++++++++++++++++++
>     >> >  1 file changed, 25 insertions(+)
>     >> >  create mode 100644
>     meta-oe/recipes-extended/hwloc/hwloc_1.11.5.bb <http://hwloc_1.11.5.bb>
>     >> >
>     >> > diff --git a/meta-oe/recipes-extended/hwloc/hwloc_1.11.5.bb
>     <http://hwloc_1.11.5.bb>
>     >> > b/meta-oe/recipes-extended/hwloc/hwloc_1.11.5.bb
>     <http://hwloc_1.11.5.bb>
>     >> > new file mode 100644
>     >> > index 0000000..3a88f5b
>     >> > --- /dev/null
>     >> > +++ b/meta-oe/recipes-extended/hwloc/hwloc_1.11.5.bb
>     <http://hwloc_1.11.5.bb>
>     >> > @@ -0,0 +1,25 @@
>     >> > +SUMMARY = "The Portable Hardware Locality (hwloc) software
>     package \
>     >> > + provides a portable abstraction of the hierarchical topology
>     of modern
>     >> \
>     >> > + architectures."
>     >> > +HOMEPAGE = "https://www.open-mpi.org/software/hwloc/
>     <https://www.open-mpi.org/software/hwloc/>"
>     >> > +SECTION = "base"
>     >> > +LICENSE = "BSD"
>     >> > +LIC_FILES_CHKSUM =
>     "file://COPYING;md5=3282e20dc3cec311deda3c6d4b1f99
>     >> 0b"
>     >> > +
>     >> > +SRC_URI = "https://www.open-mpi.org/software/${PN}/v1.11/
>     <https://www.open-mpi.org/software/${PN}/v1.11/>
>     >> > downloads/${BP}.tar.bz2"
>     >> > +SRC_URI[md5sum] = "96c34136ff416d2b13a7821c27477bed"
>     >> > +
>     >> > +inherit autotools
>     >> > +
>     >> > +# Split hwloc library into separate subpackage
>     >> > +PACKAGES_prepend = "lib${PN}"
>     >> > +FILES_lib${PN} += "${libdir}/lib${PN}.so*"
>     >> > +${PN}_RDEPENDS += "lib${PN}"
>     >> > +
>     >> > +do_install_append() {
>     >> > +    # Let ldconfig do it's thing, no need for these symlinks
>     >> > +    test -L "${D}${libdir}/lib${PN}.so.5"
>     >> > +    test -L "${D}${libdir}/lib${PN}.so"
>     >> > +    rm "${D}${libdir}/lib${PN}.so.5"
>     >> > +    rm "${D}${libdir}/lib${PN}.so"
>     >>
>     >
>     > This link manipulation is also inconsistent with every other
>     recipe. It’s
>     > not appropriate. If your distro wants to rely on ldconfig to
>     handle the
>     > links, then your distro could do this for all recipes rather than
>     just one.
>     > This isn’t how things are done in our existing recipes.
>
>     There is global knob USE_LDCONFIG, perhaps thats what could be used
>     but generally I agree with Chris here.
>

That's fine, I can remove this. I really did it to avoid a dev-so QA 
warning, which isn't very amenable to lib* package names. It trips on 
symlink in any non *-dev or *-dbg package name. Added INSANE_SKIP instead.

Perhaps we should loosen dev-so to permit symlinks in lib*'s as well, if 
it's common practice to package soname symlinks in OE. Thoughts?

>     > --
>     > Christopher Larson
>     > clarson at kergoth dot com
>     > Founder - BitBake, OpenEmbedded, OpenZaurus
>     > Maintainer - Tslib
>     > Senior Software Engineer, Mentor Graphics
>     > --
>     > _______________________________________________
>     > Openembedded-devel mailing list
>     > Openembedded-devel@lists.openembedded.org
>     <mailto:Openembedded-devel@lists.openembedded.org>
>     > http://lists.openembedded.org/mailman/listinfo/openembedded-devel
>     <http://lists.openembedded.org/mailman/listinfo/openembedded-devel>
>     --
>     _______________________________________________
>     Openembedded-devel mailing list
>     Openembedded-devel@lists.openembedded.org
>     <mailto:Openembedded-devel@lists.openembedded.org>
>     http://lists.openembedded.org/mailman/listinfo/openembedded-devel
>     <http://lists.openembedded.org/mailman/listinfo/openembedded-devel>
>
>

Thanks,
Haris