[PATCHv3 1/4] cmake.bbclass: use CMAKE_SYSTEM_LIBRARY_PATH instead of CMAKE_LIBRARY_PATH

[PATCHv3 1/4] cmake.bbclass: use CMAKE_SYSTEM_LIBRARY_PATH instead of CMAKE_LIBRARY_PATH

[Pascal Bach]

CMAKE_LIBRARY_PATH is intended to be set by projects.
CMAKE_SYSTEM_LIBRARY_PATH is better suited to be used in a toolchain
file.

Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
---
 meta/classes/cmake.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index fd40a9863e..86b1d0e9aa 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -106,7 +106,7 @@ set( CMAKE_INSTALL_RPATH ${OECMAKE_RPATH} )
 list(APPEND CMAKE_MODULE_PATH "${STAGING_DATADIR}/cmake/Modules/")
 
 # add for non /usr/lib libdir, e.g. /usr/lib64
-set( CMAKE_LIBRARY_PATH ${libdir} ${base_libdir})
+set( CMAKE_SYSTEM_LIBRARY_PATH ${libdir} ${base_libdir})
 
 EOF
 }
-- 
2.11.0

[PATCHv3 2/4] cmake.bbclass: search both sysroot-native and host for native packages

[Pascal Bach]

Certain headers and libraries like `math.h` an `-m` are only available on the
host as they are provided by the host toolchain.

This leads to issues that a find_library in CMake doesn't find the `m` library
of a find_path doesn't find `math.h`. This issue occurred in the wireshark recipe
for example.

This change enables CMake to also look on the host for libraries and includes when
building a native package.

Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
---
 meta/classes/cmake.bbclass | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index 86b1d0e9aa..ce3c0278ff 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -43,8 +43,8 @@ OECMAKE_RPATH ?= ""
 OECMAKE_PERLNATIVE_DIR ??= ""
 OECMAKE_EXTRA_ROOT_PATH ?= ""
 
-OECMAKE_FIND_ROOT_PATH_MODE_PROGRAM = "ONLY"
-OECMAKE_FIND_ROOT_PATH_MODE_PROGRAM_class-native = "BOTH"
+OECMAKE_FIND_ROOT_PATH_MODE = "ONLY"
+OECMAKE_FIND_ROOT_PATH_MODE_class-native = "BOTH"
 
 EXTRA_OECMAKE_append = " ${PACKAGECONFIG_CONFARGS}"
 
@@ -90,10 +90,10 @@ set( CMAKE_CXX_LINK_FLAGS "${OECMAKE_CXX_LINK_FLAGS}" CACHE STRING "LDFLAGS" )
 # only search in the paths provided so cmake doesnt pick
 # up libraries and tools from the native build machine
 set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE} ${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH} ${EXTERNAL_TOOLCHAIN})
-set( CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY )
-set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ${OECMAKE_FIND_ROOT_PATH_MODE_PROGRAM} )
-set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY )
-set( CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY )
+set( CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ${OECMAKE_FIND_ROOT_PATH_MODE} )
+set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ${OECMAKE_FIND_ROOT_PATH_MODE} )
+set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ${OECMAKE_FIND_ROOT_PATH_MODE} )
+set( CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ${OECMAKE_FIND_ROOT_PATH_MODE} )
 
 # Use qt.conf settings
 set( ENV{QT_CONF_PATH} ${WORKDIR}/qt.conf )
-- 
2.11.0

[PATCHv3 3/4] cmake.bbclass: move CMAKE_NO_SYSTEM_FROM_IMPORTED to toolchain.cmake

[Pascal Bach]

The setting influences the build like other settings already in toolchain.cmake.
It is more appropriate to set it there instead of providing it as a random
command line parameter to CMake.

It also makes it easier to use the toolchain.cmake file independent of bitbake.
Like the devshell for example.

Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
---
 meta/classes/cmake.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index ce3c0278ff..0ef63795eb 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -108,6 +108,9 @@ list(APPEND CMAKE_MODULE_PATH "${STAGING_DATADIR}/cmake/Modules/")
 # add for non /usr/lib libdir, e.g. /usr/lib64
 set( CMAKE_SYSTEM_LIBRARY_PATH ${libdir} ${base_libdir})
 
+# avoid treating imports as system includes
+set( CMAKE_NO_SYSTEM_FROM_IMPORTED ON)
+
 EOF
 }
 
@@ -152,7 +155,6 @@ cmake_do_configure() {
 	  -DCMAKE_INSTALL_SO_NO_EXE=0 \
 	  -DCMAKE_TOOLCHAIN_FILE=${WORKDIR}/toolchain.cmake \
 	  -DCMAKE_VERBOSE_MAKEFILE=1 \
-	  -DCMAKE_NO_SYSTEM_FROM_IMPORTED=1 \
 	  ${EXTRA_OECMAKE} \
 	  -Wno-dev
 }
-- 
2.11.0

[PATCHv3 4/4] cmake.bbclass: allow cmake to find hosttools

[Pascal Bach]

Currently the generated toolchain file is unable to find hosttools as they
do not appear in the search paths.

Just adding HOSTTOOLS_DIR is not enough as binaries are located directly under
${HOSTTOOLS_DIR}. Like ${HOSTTOOLS_DIR}/git for example.
CMake however only searches in [s]bin sub directories of the paths specified in
CMAKE_FIND_ROOT_PATH. Explicitly adding / to CMAKE_SYSTEM_PROGRAM_PATH makes
CMake look in the right location.

Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
---
 meta/classes/cmake.bbclass | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index 0ef63795eb..c8a079c417 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -89,7 +89,7 @@ set( CMAKE_CXX_LINK_FLAGS "${OECMAKE_CXX_LINK_FLAGS}" CACHE STRING "LDFLAGS" )
 
 # only search in the paths provided so cmake doesnt pick
 # up libraries and tools from the native build machine
-set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE} ${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH} ${EXTERNAL_TOOLCHAIN})
+set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE} ${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH} ${EXTERNAL_TOOLCHAIN} ${HOSTTOOLS_DIR})
 set( CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ${OECMAKE_FIND_ROOT_PATH_MODE} )
 set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ${OECMAKE_FIND_ROOT_PATH_MODE} )
 set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ${OECMAKE_FIND_ROOT_PATH_MODE} )
@@ -108,6 +108,10 @@ list(APPEND CMAKE_MODULE_PATH "${STAGING_DATADIR}/cmake/Modules/")
 # add for non /usr/lib libdir, e.g. /usr/lib64
 set( CMAKE_SYSTEM_LIBRARY_PATH ${libdir} ${base_libdir})
 
+# by default CMake only looks in [s]bin subdirectories of CMAKE_FIND_ROOT_PATH
+# adding / makes CMake look for binaries in hosttools too.
+set( CMAKE_SYSTEM_PROGRAM_PATH /)
+
 # avoid treating imports as system includes
 set( CMAKE_NO_SYSTEM_FROM_IMPORTED ON)
 
-- 
2.11.0

Re: [PATCHv3 1/4] cmake.bbclass: use CMAKE_SYSTEM_LIBRARY_PATH instead of CMAKE_LIBRARY_PATH

[Richard Purdie]

On Wed, 2018-10-10 at 10:10 +0200, Pascal Bach wrote:
> CMAKE_LIBRARY_PATH is intended to be set by projects.
> CMAKE_SYSTEM_LIBRARY_PATH is better suited to be used in a toolchain
> file.
> 
> Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
> ---
>  meta/classes/cmake.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

I have a feeling something in this series may have caused:

https://autobuilder.yoctoproject.org/typhoon/api/v2/logs/42753/raw

but haven't bisected to confirm it is the cmake changes yet (it is from
something in master-next).

Cheers,

Richard

Re: [PATCHv3 1/4] cmake.bbclass: use CMAKE_SYSTEM_LIBRARY_PATH instead of CMAKE_LIBRARY_PATH

[Pascal Bach]

On 10.10.2018 14:36, Richard Purdie wrote:
> On Wed, 2018-10-10 at 10:10 +0200, Pascal Bach wrote:
>> CMAKE_LIBRARY_PATH is intended to be set by projects.
>> CMAKE_SYSTEM_LIBRARY_PATH is better suited to be used in a toolchain
>> file.
>>
>> Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
>> ---
>>  meta/classes/cmake.bbclass | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
> I have a feeling something in this series may have caused:
>
> https://autobuilder.yoctoproject.org/typhoon/api/v2/logs/42753/raw
>
> but haven't bisected to confirm it is the cmake changes yet (it is from
> something in master-next).
You are right the recipe is using cmake. I will investigate the issue.

Re: [PATCHv3 1/4] cmake.bbclass: use CMAKE_SYSTEM_LIBRARY_PATH instead of CMAKE_LIBRARY_PATH

[Pascal Bach]

On 10.10.2018 14:36, Richard Purdie wrote:
> On Wed, 2018-10-10 at 10:10 +0200, Pascal Bach wrote:
>> CMAKE_LIBRARY_PATH is intended to be set by projects.
>> CMAKE_SYSTEM_LIBRARY_PATH is better suited to be used in a toolchain
>> file.
>>
>> Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
>> ---
>>  meta/classes/cmake.bbclass | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
> I have a feeling something in this series may have caused:
>
> https://autobuilder.yoctoproject.org/typhoon/api/v2/logs/42753/raw
>
> but haven't bisected to confirm it is the cmake changes yet (it is from
> something in master-next).
>
The problem is "cmake.bbclass: allow cmake to find hosttools" it has the side effect of making all hosttools available to all recipes.
In the case of libproxy this leads to it finding python, which is included in hosttools, and thus building it's bindings.

This kind of defeats the purpose of having recipe specific sysroots.
Is there a way to make only selected hosttools available to a recipe. Like for example git? Maybe use git-native?

I think the patch "cmake.bbclass: allow cmake to find hosttools" in this form does more harm then good. The rest of the series should be fine.
Should I resubmit the series without this specific patch?

One more thing for libproxy. It currently disables python explicitly via `-WITH_PYTHON=no`. But this option doesn't exist anymore as it got replaces
by `-WITH_PYTHON2=no` `-WITH_PYTHON3=no`. This is why the auto detection kicks in. I will submit a patch to clean this recipe up too.

Pascal

Re: [PATCHv3 1/4] cmake.bbclass: use CMAKE_SYSTEM_LIBRARY_PATH instead of CMAKE_LIBRARY_PATH

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 1325 bytes --]

On Wed, 10 Oct 2018 at 15:12, Pascal Bach <pascal.bach@siemens.com> wrote:

> The problem is "cmake.bbclass: allow cmake to find hosttools" it has the
> side effect of making all hosttools available to all recipes.
> In the case of libproxy this leads to it finding python, which is included
> in hosttools, and thus building it's bindings.
>
> This kind of defeats the purpose of having recipe specific sysroots.
> Is there a way to make only selected hosttools available to a recipe. Like
> for example git? Maybe use git-native?
>
> I think the patch "cmake.bbclass: allow cmake to find hosttools" in this
> form does more harm then good. The rest of the series should be fine.
> Should I resubmit the series without this specific patch?
>
> One more thing for libproxy. It currently disables python explicitly via
> `-WITH_PYTHON=no`. But this option doesn't exist anymore as it got replaces
> by `-WITH_PYTHON2=no` `-WITH_PYTHON3=no`. This is why the auto detection
> kicks in. I will submit a patch to clean this recipe up too.


Per-recipe hosttools is an interesting idea but I'd say it's quite likely
too invasive.  This change has actually fixed cmake, and exposed the
problem that the recipe is broken and should be using
WITH_PYTHON2/WITH_PYTHON3: fixing that is the correct fix.

Ross

[-- Attachment #2: Type: text/html, Size: 1624 bytes --]