[PATCH] boost.inc: make libboost_python3.so available

[PATCH] boost.inc: make libboost_python3.so available

[mingli.yu]

From: Mingli Yu <mingli.yu@windriver.com>

Currently boost provides ${libdir}/libboost_python35.so,
but there is some logic as below in CMakeLists.txt
of package cv-bridge under meta-ros layer.
[snip]
if(PYTHONLIBS_VERSION_STRING VERSION_LESS 3)
    find_package(Boost REQUIRED python)
else()
    find_package(Boost REQUIRED python3)
endif()
[snip]

And comes below do_configure error for cv-bridge:
meta-ros/recipes-ros/vision-opencv/cv-bridge_1.11.16.bb:do_configure
| CMake Error at $Prj/cv-bridge/1.11.16-r0/recipe-sysroot-native/usr/share/cmake-3.12/Modules/FindBoost.cmake:2048 (message):
|   Unable to find the requested Boost libraries.
|
|   Boost version: 1.68.0
|
|   Boost include path:
|   $Prj/cv-bridge/1.11.16-r0/recipe-sysroot/usr/include
|
|
|   Could not find the following Boost libraries:
|
|           boost_python3
|
|   No Boost libraries were found.  You may need to set BOOST_LIBRARYDIR to the
|   directory containing Boost libraries or BOOST_ROOT to the location of
|   Boost.
| Call Stack (most recent call first):
|   CMakeLists.txt:11 (find_package)

Can update CMakeLists.txt of cv-bridge as below
to remove the configure error:
 - find_package(Boost REQUIRED python3)
 + find_package(Boost REQUIRED python35)

But once the python version changes such as
python37, it fails again. So make boost provides
libboost_python3.so to make the fix flexibly.

[YOCTO #12833]

Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
---
 meta/recipes-support/boost/boost.inc | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/recipes-support/boost/boost.inc b/meta/recipes-support/boost/boost.inc
index c4faea211f0..98f0b71d73a 100644
--- a/meta/recipes-support/boost/boost.inc
+++ b/meta/recipes-support/boost/boost.inc
@@ -198,6 +198,10 @@ do_install() {
 			ln -s libboost_${lib}.so ${D}${libdir}/libboost_${lib}-mt.so
 		fi
 	done
+	if [ -e ${D}${libdir}/libboost_python3[0-9].so ] && [ ! -e ${D}${libdir}/libboost_python3.so ]; then
+		lib_python3_so=$(basename `ls ${D}${libdir}/libboost_python3[0-9].so`)
+		ln -s ${lib_python3_so} ${D}${libdir}/libboost_python3.so
+	fi
 
 }
 
-- 
2.17.1

Re: [PATCH] boost.inc: make libboost_python3.so available

[richard.purdie]

On Tue, 2018-10-16 at 02:00 -0700, mingli.yu@windriver.com wrote:
> From: Mingli Yu <mingli.yu@windriver.com>
> 
> Currently boost provides ${libdir}/libboost_python35.so,
> but there is some logic as below in CMakeLists.txt
> of package cv-bridge under meta-ros layer.
> [snip]
> if(PYTHONLIBS_VERSION_STRING VERSION_LESS 3)
>     find_package(Boost REQUIRED python)
> else()
>     find_package(Boost REQUIRED python3)
> endif()
> [snip]
> 
> And comes below do_configure error for cv-bridge:
> meta-ros/recipes-ros/vision-opencv/cv-bridge_1.11.16.bb:do_configure
> > CMake Error at $Prj/cv-bridge/1.11.16-r0/recipe-sysroot-
> > native/usr/share/cmake-3.12/Modules/FindBoost.cmake:2048 (message):
> >   Unable to find the requested Boost libraries.
> > 
> >   Boost version: 1.68.0
> > 
> >   Boost include path:
> >   $Prj/cv-bridge/1.11.16-r0/recipe-sysroot/usr/include
> > 
> > 
> >   Could not find the following Boost libraries:
> > 
> >           boost_python3
> > 
> >   No Boost libraries were found.  You may need to set
> > BOOST_LIBRARYDIR to the
> >   directory containing Boost libraries or BOOST_ROOT to the
> > location of
> >   Boost.
> > Call Stack (most recent call first):
> >   CMakeLists.txt:11 (find_package)
> 
> Can update CMakeLists.txt of cv-bridge as below
> to remove the configure error:
>  - find_package(Boost REQUIRED python3)
>  + find_package(Boost REQUIRED python35)
> 
> But once the python version changes such as
> python37, it fails again. So make boost provides
> libboost_python3.so to make the fix flexibly.
> 
> [YOCTO #12833]
> 
> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> ---
>  meta/recipes-support/boost/boost.inc | 4 ++++
>  1 file changed, 4 insertions(+)

I'm a little worried about why we're having to create these links when
other software seems able to expect these things by default. Is this
some standard packaging most distros do? I'm a little worried we're
creating an ABI here which doesn't exist...

Put another way, why doesn't upstream create this link if its expected
to work?

Cheers,

Richard

Re: [PATCH] boost.inc: make libboost_python3.so available

[Burton, Ross]

On Tue, 16 Oct 2018 at 11:01, <richard.purdie@linuxfoundation.org> wrote:
> I'm a little worried about why we're having to create these links when
> other software seems able to expect these things by default. Is this
> some standard packaging most distros do? I'm a little worried we're
> creating an ABI here which doesn't exist...

https://koji.fedoraproject.org/koji/rpminfo?rpmID=14904908 says that
Fedora's boost-python3 package contains just
/usr/lib64/libboost_python3.so.1.66.0.  Their spec file at
https://src.fedoraproject.org/rpms/boost/blob/master/f/boost.spec is
hairy (as one expects for Boost) but doesn't appear to rename the
file.

So the question is why does our build produce libboost_python35.so?

Ross

Re: [PATCH] boost.inc: make libboost_python3.so available

[Burton, Ross]

Right.

This was a change in Boost 1.67 onwards:

https://github.com/boostorg/python/commit/d4d41d94aecc9f8098aabd3587fcb95458451f71#diff-42dd6ec1330a7c47aaccf2ab2b8f1e02

OpenCV has a patch to "fix" (bodge) the Py2 build:

https://github.com/ros-perception/vision_opencv/pull/239

But this doesn't work for Py3.  Boost and CMake consider this change
correct and the documentation for the cmake/boost integration reflects
this:

https://gitlab.kitware.com/cmake/cmake/blob/master/Modules/FindBoost.cmake#L43

So the worst-case fix is to patch opencv to look for "python35"
instead of "python3", although this will break when we upgrade to
Python 3.  A better solution would be to upstream a patch that uses
the Python version to generate the correct name.

Ross

On Tue, 16 Oct 2018 at 11:38, Burton, Ross <ross.burton@intel.com> wrote:
>
> On Tue, 16 Oct 2018 at 11:01, <richard.purdie@linuxfoundation.org> wrote:
> > I'm a little worried about why we're having to create these links when
> > other software seems able to expect these things by default. Is this
> > some standard packaging most distros do? I'm a little worried we're
> > creating an ABI here which doesn't exist...
>
> https://koji.fedoraproject.org/koji/rpminfo?rpmID=14904908 says that
> Fedora's boost-python3 package contains just
> /usr/lib64/libboost_python3.so.1.66.0.  Their spec file at
> https://src.fedoraproject.org/rpms/boost/blob/master/f/boost.spec is
> hairy (as one expects for Boost) but doesn't appear to rename the
> file.
>
> So the question is why does our build produce libboost_python35.so?
>
> Ross

Re: [PATCH] boost.inc: make libboost_python3.so available

[Burton, Ross]

On Tue, 16 Oct 2018 at 12:18, Burton, Ross <ross.burton@intel.com> wrote:
> OpenCV has a patch to "fix" (bodge) the Py2 build:
>
> https://github.com/ros-perception/vision_opencv/pull/239
>
> But this doesn't work for Py3.

This was just pushed:
https://github.com/ros-perception/vision_opencv/commit/b57fd5b

Ross

Re: [PATCH] boost.inc: make libboost_python3.so available

[Yu, Mingli]

On 2018年10月16日 20:05, Burton, Ross wrote:
> On Tue, 16 Oct 2018 at 12:18, Burton, Ross <ross.burton@intel.com> wrote:
>> OpenCV has a patch to "fix" (bodge) the Py2 build:
>>
>> https://github.com/ros-perception/vision_opencv/pull/239
>>
>> But this doesn't work for Py3.
>
> This was just pushed:
> https://github.com/ros-perception/vision_opencv/commit/b57fd5b

Thanks Ross!
I'm fine not add the the hack fix from boost side, but as I know there 
is another package camera-calibration-parsers which has the same problem 
as below apart from cv-bridge. BTW, there may some other package also 
has this kind of problem. This to say we need to fix the issue for the 
this kind of packages one by one.

meta-ros/recipes-ros/image-common/camera-calibration-parsers_1.11.13.bb:do_configure 
failed
| CMake Error at 
/$Prj/camera-calibration-parsers/1.11.13-r0/recipe-sysroot-native/usr/share/cmake-3.12/Modules/FindBoost.cmake:2048 
(message):
|   Unable to find the requested Boost libraries.
|
|   Boost version: 1.68.0
|
|   Boost include path:
|   /$Prj/camera-calibration-parsers/1.11.13-r0/recipe-sysroot/usr/include
|
|
|   Could not find the following Boost libraries:
|
|           boost_python3
|
|   Some (but not all) of the required Boost libraries were found.  You may
|   need to install these additional Boost libraries.  Alternatively, set
|   BOOST_LIBRARYDIR to the directory containing Boost libraries or 
BOOST_ROOT
|   to the location of Boost.
| Call Stack (most recent call first):
|   CMakeLists.txt:10 (find_package)

Thanks,

>
> Ross
>

Re: [PATCH] boost.inc: make libboost_python3.so available

[Burton, Ross]

On Wed, 17 Oct 2018 at 06:48, Yu, Mingli <mingli.yu@windriver.com> wrote:
> I'm fine not add the the hack fix from boost side, but as I know there
> is another package camera-calibration-parsers which has the same problem
> as below apart from cv-bridge. BTW, there may some other package also
> has this kind of problem. This to say we need to fix the issue for the
> this kind of packages one by one.

Yes, it will need to be fixed one by one, but everyone will have to do
this because this is what Boost is now doing.  This isn't a change to
boost that we introduced, upstream changed the library names so
everyone who uses boost will be fixing their code.

Ross

Re: [PATCH] boost.inc: make libboost_python3.so available

[Yu, Mingli]

On 2018年10月17日 17:39, Burton, Ross wrote:
> On Wed, 17 Oct 2018 at 06:48, Yu, Mingli <mingli.yu@windriver.com> wrote:
>> I'm fine not add the the hack fix from boost side, but as I know there
>> is another package camera-calibration-parsers which has the same problem
>> as below apart from cv-bridge. BTW, there may some other package also
>> has this kind of problem. This to say we need to fix the issue for the
>> this kind of packages one by one.
>
> Yes, it will need to be fixed one by one, but everyone will have to do
> this because this is what Boost is now doing.  This isn't a change to
> boost that we introduced, upstream changed the library names so
> everyone who uses boost will be fixing their code.

Many thanks for Ross's clarification!
If we need to fix the packages one by one, but not add the hack at boost 
side, then https://bugzilla.yoctoproject.org/show_bug.cgi?id=12833 
should be invalid.

Thanks,

>
> Ross
>

Re: [PATCH] boost.inc: make libboost_python3.so available

[Burton, Ross]

On Thu, 18 Oct 2018 at 03:29, Yu, Mingli <mingli.yu@windriver.com> wrote:
> Many thanks for Ross's clarification!
> If we need to fix the packages one by one, but not add the hack at boost
> side, then https://bugzilla.yoctoproject.org/show_bug.cgi?id=12833
> should be invalid.

Agreed, done!

Ross