[PATCH 0/6] Use llvmpile for mesa on x86/x86-64

[PATCH 0/6] Use llvmpile for mesa on x86/x86-64

[Khem Raj]

This series fixes llvm build to support multilib configurations
Adds dos2unix bbclass to core from meta-oe
Fixes ppc security flags world builds

The following changes since commit 176e50fb177dc40e439700d5f3f838dd7eaaa427:

  runtime selftest: limit kernel hw bp arches (2018-08-16 22:40:06 +0100)

are available in the Git repository at:

  git://git.openembedded.org/openembedded-core-contrib kraj/master
  http://cgit.openembedded.org/openembedded-core-contrib/log/?h=kraj/master

Khem Raj (6):
  mesa: Enable gallium-llvm on x86 and x86_64
  powerpc: Remove pie flags from compiler cmdline
  llvm: Point llvm-config to one built with llvm-native
  dos2unix.bbclass: Move to oe-core
  llvm: Use YOCTO_ALTERNATE_MULTILIB_NAME environment variable in
    llvm-config
  mesa: Define YOCTO_ALTERNATE_MULTILIB_NAME to consider multilib names

 meta/classes/dos2unix.bbclass                 | 14 ++++++
 meta/conf/distro/include/security_flags.inc   |  7 +--
 ...-llvm-allow-env-override-of-exe-path.patch | 47 ++++++++++++++++---
 meta/recipes-devtools/llvm/llvm_git.bb        |  4 +-
 meta/recipes-graphics/cairo/cairo.inc         |  3 +-
 meta/recipes-graphics/mesa/mesa.inc           |  4 ++
 6 files changed, 65 insertions(+), 14 deletions(-)
 create mode 100644 meta/classes/dos2unix.bbclass

-- 
2.18.0

[PATCH 1/6] mesa: Enable gallium-llvm on x86 and x86_64

[Khem Raj]

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/recipes-graphics/cairo/cairo.inc | 3 ++-
 meta/recipes-graphics/mesa/mesa.inc   | 3 +++
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-graphics/cairo/cairo.inc b/meta/recipes-graphics/cairo/cairo.inc
index 20e0d2c92a..7347f223ff 100644
--- a/meta/recipes-graphics/cairo/cairo.inc
+++ b/meta/recipes-graphics/cairo/cairo.inc
@@ -22,7 +22,8 @@ X11DEPENDS = "virtual/libx11 libsm libxrender libxext"
 DEPENDS = "libpng fontconfig pixman glib-2.0 zlib"
 
 PACKAGECONFIG ??= "${@bb.utils.contains('DISTRO_FEATURES', 'x11', 'x11 xcb', '', d)} \
-           ${@bb.utils.filter('DISTRO_FEATURES', 'directfb', d)}"
+           ${@bb.utils.filter('DISTRO_FEATURES', 'directfb', d)} \
+           ${@bb.utils.contains('DISTRO_FEATURES', 'x11 opengl', 'opengl', '', d)}"
 
 PACKAGECONFIG[x11] = "--with-x=yes -enable-xlib,--with-x=no --disable-xlib,${X11DEPENDS}"
 PACKAGECONFIG[xcb] = "--enable-xcb,--disable-xcb,libxcb"
diff --git a/meta/recipes-graphics/mesa/mesa.inc b/meta/recipes-graphics/mesa/mesa.inc
index 6aedeee9dc..2671d4de4d 100644
--- a/meta/recipes-graphics/mesa/mesa.inc
+++ b/meta/recipes-graphics/mesa/mesa.inc
@@ -47,6 +47,9 @@ PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'wayland vulkan', d)} \
                    ${@bb.utils.contains('DISTRO_FEATURES', 'x11 vulkan', 'dri3', '', d)} \
 		   "
 
+PACKAGECONFIG_append_x86 = " gallium-llvm gallium r600"
+PACKAGECONFIG_append_x86-64 = " gallium-llvm gallium r600"
+
 # "gbm" requires "dri", "opengl"
 PACKAGECONFIG[gbm] = "--enable-gbm,--disable-gbm"
 
-- 
2.18.0

[PATCH 2/6] powerpc: Remove pie flags from compiler cmdline

[Khem Raj]

Original approach to add -no-<pie> flags cause link time behavior changes
where packages start to lose the -fPIC -DPIC in compiler cmdline and this
list keeps growing as we build more and more packages,

Instead lets just remove the options we dont need from SECURITY_CFLAGS
this makes it more robust and less intrusive

This also means we do not need to re-add pic options as we started to do
for affected packages

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/conf/distro/include/security_flags.inc | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/meta/conf/distro/include/security_flags.inc b/meta/conf/distro/include/security_flags.inc
index 52e1e4ebc5..4e20a4d0f8 100644
--- a/meta/conf/distro/include/security_flags.inc
+++ b/meta/conf/distro/include/security_flags.inc
@@ -28,13 +28,10 @@ SECURITY_LDFLAGS ?= "-fstack-protector-strong -Wl,-z,relro,-z,now"
 SECURITY_X_LDFLAGS ?= "-fstack-protector-strong -Wl,-z,relro"
 
 # powerpc does not get on with pie for reasons not looked into as yet
-SECURITY_CFLAGS_powerpc = "-fstack-protector-strong ${lcl_maybe_fortify} ${SECURITY_NOPIE_CFLAGS}"
-SECURITY_CFLAGS_pn-libgcc_powerpc = ""
-SECURITY_CFLAGS_append_powerpc_pn-libdrm = " -fPIC -DPIC"
-SECURITY_CFLAGS_append_powerpc_pn-libinput = " -fPIC -DPIC"
-SECURITY_CFLAGS_append_powerpc_pn-at-spi2-core = " -fPIC -DPIC"
 GCCPIE_powerpc = ""
 GLIBCPIE_powerpc = ""
+SECURITY_CFLAGS_remove_powerpc = "${SECURITY_PIE_CFLAGS}"
+SECURITY_CFLAGS_pn-libgcc_powerpc = ""
 
 SECURITY_CFLAGS_pn-glibc = ""
 SECURITY_CFLAGS_pn-glibc-initial = ""
-- 
2.18.0

[PATCH 3/6] llvm: Point llvm-config to one built with llvm-native

[Khem Raj]

If not defined, llvm build system tries to build one
which then confuses the OE QA system since its building
native tool and target packages in same package build
moreover it is not required since we already have it via
llvm-native

Fixes
ERROR: llvm-6.0-r0 do_package_qa: QA Issue: llvm: The compile log indicates that host include and/or library paths were used.

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/recipes-devtools/llvm/llvm_git.bb | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/llvm/llvm_git.bb b/meta/recipes-devtools/llvm/llvm_git.bb
index bb83012231..d853d4151d 100644
--- a/meta/recipes-devtools/llvm/llvm_git.bb
+++ b/meta/recipes-devtools/llvm/llvm_git.bb
@@ -64,17 +64,19 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \
                   -DLLVM_ENABLE_FFI=ON \
                   -DFFI_INCLUDE_DIR=$(pkg-config --variable=includedir libffi) \
                   -DLLVM_OPTIMIZED_TABLEGEN=ON \
-                  -DLLVM_TARGETS_TO_BUILD="${LLVM_TARGETS}" \
+                  -DLLVM_TARGETS_TO_BUILD='${LLVM_TARGETS}' \
                   -G Ninja"
 
 EXTRA_OECMAKE_append_class-target = "\
                   -DCMAKE_CROSSCOMPILING:BOOL=ON \
                   -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \
+                  -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \
                  "
 
 EXTRA_OECMAKE_append_class-nativesdk = "\
                   -DCMAKE_CROSSCOMPILING:BOOL=ON \
                   -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \
+                  -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \
                  "
 
 do_configure_prepend() {
-- 
2.18.0

[PATCH 4/6] dos2unix.bbclass: Move to oe-core

[Khem Raj]

- Import from meta-oe layer

- This is useful for many packages where CR-LF
  needs to be adjusted, many recipes depend on it
  e.g. meta-multimedia libebml and so on.

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/classes/dos2unix.bbclass | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
 create mode 100644 meta/classes/dos2unix.bbclass

diff --git a/meta/classes/dos2unix.bbclass b/meta/classes/dos2unix.bbclass
new file mode 100644
index 0000000000..3fc17e2196
--- /dev/null
+++ b/meta/classes/dos2unix.bbclass
@@ -0,0 +1,14 @@
+# Class for use to convert all CRLF line terminators to LF
+# provided that some projects are being developed/maintained
+# on Windows so they have different line terminators(CRLF) vs
+# on Linux(LF), which can cause annoying patching errors during
+# git push/checkout processes.
+
+do_convert_crlf_to_lf[depends] += "dos2unix-native:do_populate_sysroot"
+
+# Convert CRLF line terminators to LF
+do_convert_crlf_to_lf () {
+	find ${S} -type f -exec dos2unix {} \;
+}
+
+addtask convert_crlf_to_lf after do_unpack before do_patch
-- 
2.18.0

[PATCH 5/6] llvm: Use YOCTO_ALTERNATE_MULTILIB_NAME environment variable in llvm-config

[Khem Raj]

llvm-config is a tool on similar veins as pkg-config but provides a lot
more information and packages which use llvm e.g. mesa use this tool to
poke for llvm related informaiton e.g. version, libpath, includepaths
to name a few, this has few challanges in cross build environments where
llvm-config is supposed to be build for buildhost but provide information
about target llvm which is addressed by building native llvm-config along
with target llvm build, but this is frowned upon by OE build system since
it detects that host paths are being used so we have to build it as part
of llvm-native but then it means install paths for llvm and llvm-native
are different and wrong paths get reported when llvm-config is used.
This is solved by providing YOCTO_ALTERNATE_EXE_PATH variable to let
llvm-config use that path instead of self-relative path to report back

Second problem is when building multi-lib packages base_libdir is different
for target packages but native llvm-config does not know about it so
it reports non-multilibbed paths as libdir and packages can not find
llvm in sysroot. This is fixed by adding another environment variable
YOCTO_ALTERNATE_MULTILIB_NAME which can be set from recipes to set
proper multilib path

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 ...-llvm-allow-env-override-of-exe-path.patch | 47 ++++++++++++++++---
 1 file changed, 40 insertions(+), 7 deletions(-)

diff --git a/meta/recipes-devtools/llvm/llvm/0002-llvm-allow-env-override-of-exe-path.patch b/meta/recipes-devtools/llvm/llvm/0002-llvm-allow-env-override-of-exe-path.patch
index 21d2f81b58..6a9283323c 100644
--- a/meta/recipes-devtools/llvm/llvm/0002-llvm-allow-env-override-of-exe-path.patch
+++ b/meta/recipes-devtools/llvm/llvm/0002-llvm-allow-env-override-of-exe-path.patch
@@ -1,4 +1,4 @@
-From 2f8ea767afdaa440c6368040630e1b3ea6a0977a Mon Sep 17 00:00:00 2001
+From aeccf16eaccdd80e4d5ecaa51673ce4b2bac1130 Mon Sep 17 00:00:00 2001
 From: Martin Kelly <mkelly@xevo.com>
 Date: Fri, 19 May 2017 00:22:57 -0700
 Subject: [PATCH 2/2] llvm: allow env override of exe path
@@ -8,16 +8,19 @@ return the libraries, include directories, etc. from inside the sysroot rather
 than from the native sysroot. Thus provide an env override for calling
 llvm-config from a target sysroot.
 
+To let it work in multilib environment, we need to provide a knob to supply
+multilib dirname as well
+
+Upstream-Status: Inappropriate [OE-Specific]
+
 Signed-off-by: Martin Kelly <mkelly@xevo.com>
 Signed-off-by: Khem Raj <raj.khem@gmail.com>
 ---
-Upstream-Status: Pending
-
- tools/llvm-config/llvm-config.cpp | 7 +++++++
- 1 file changed, 7 insertions(+)
+ tools/llvm-config/llvm-config.cpp | 17 ++++++++++++++++-
+ 1 file changed, 16 insertions(+), 1 deletion(-)
 
 diff --git a/tools/llvm-config/llvm-config.cpp b/tools/llvm-config/llvm-config.cpp
-index 08b096afb05..d8d7742744e 100644
+index 08b096afb05..360cc5abf4e 100644
 --- a/tools/llvm-config/llvm-config.cpp
 +++ b/tools/llvm-config/llvm-config.cpp
 @@ -225,6 +225,13 @@ Typical components:\n\
@@ -34,6 +37,36 @@ index 08b096afb05..d8d7742744e 100644
    // This just needs to be some symbol in the binary; C++ doesn't
    // allow taking the address of ::main however.
    void *P = (void *)(intptr_t)GetExecutablePath;
+@@ -306,12 +313,20 @@ int main(int argc, char **argv) {
+   std::string ActivePrefix, ActiveBinDir, ActiveIncludeDir, ActiveLibDir,
+               ActiveCMakeDir;
+   std::string ActiveIncludeOption;
++  // Hack for Yocto: we need to override the multilib path when we are using
++  // llvm-config from within a target sysroot.
++  std::string Multilibdir = std::getenv("YOCTO_ALTERNATE_MULTILIB_NAME");
++  if (Multilibdir.empty()) {
++    Multilibdir = "/lib" LLVM_LIBDIR_SUFFIX;
++  }
++
+   if (IsInDevelopmentTree) {
+     ActiveIncludeDir = std::string(LLVM_SRC_ROOT) + "/include";
+     ActivePrefix = CurrentExecPrefix;
+ 
+     // CMake organizes the products differently than a normal prefix style
+     // layout.
++
+     switch (DevelopmentTreeLayout) {
+     case CMakeStyle:
+       ActiveBinDir = ActiveObjRoot + "/bin";
+@@ -336,7 +351,7 @@ int main(int argc, char **argv) {
+     SmallString<256> path(StringRef(LLVM_TOOLS_INSTALL_DIR));
+     sys::fs::make_absolute(ActivePrefix, path);
+     ActiveBinDir = path.str();
+-    ActiveLibDir = ActivePrefix + "/lib" + LLVM_LIBDIR_SUFFIX;
++    ActiveLibDir = ActivePrefix + Multilibdir;
+     ActiveCMakeDir = ActiveLibDir + "/cmake/llvm";
+     ActiveIncludeOption = "-I" + ActiveIncludeDir;
+   }
 -- 
-2.16.1
+2.18.0
 
-- 
2.18.0

[PATCH 6/6] mesa: Define YOCTO_ALTERNATE_MULTILIB_NAME to consider multilib names

[Khem Raj]

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/recipes-graphics/mesa/mesa.inc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-graphics/mesa/mesa.inc b/meta/recipes-graphics/mesa/mesa.inc
index 2671d4de4d..dd626d9f00 100644
--- a/meta/recipes-graphics/mesa/mesa.inc
+++ b/meta/recipes-graphics/mesa/mesa.inc
@@ -34,6 +34,7 @@ PLATFORMS ??= "${@bb.utils.filter('PACKAGECONFIG', 'x11 wayland', d)} \
 
 export LLVM_CONFIG = "${STAGING_BINDIR_NATIVE}/llvm-config${MESA_LLVM_RELEASE}"
 export YOCTO_ALTERNATE_EXE_PATH = "${STAGING_LIBDIR}/llvm${MESA_LLVM_RELEASE}/llvm-config"
+export YOCTO_ALTERNATE_MULTILIB_NAME = "${base_libdir}"
 EXTRA_OECONF = "--enable-shared-glapi \
                 --disable-opencl \
                 --enable-glx-read-only-text \
-- 
2.18.0