[PATCH v4 1/4] lto: Add global LTO distro policy file

[PATCH v4 1/4] lto: Add global LTO distro policy file

[Khem Raj]

Distros which want to enable LTO can utilize this file, it only covers
packages from OE-Core, other layers should include there own exclusion
list for recipe which dont work with LTO

Document the needed changes in local.conf.extended

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
v2: Include feeedback for adding DISTRO_FEATURE knob
v3: Drop unused PACKAGE_DEBUG_SPLIT_STYLE settings
v4: No change

 meta/conf/distro/include/lto.inc     | 28 ++++++++++++++++++++++++++++
 meta/conf/local.conf.sample.extended |  4 ++++
 2 files changed, 32 insertions(+)
 create mode 100644 meta/conf/distro/include/lto.inc

diff --git a/meta/conf/distro/include/lto.inc b/meta/conf/distro/include/lto.inc
new file mode 100644
index 0000000000..fe0f6c9f44
--- /dev/null
+++ b/meta/conf/distro/include/lto.inc
@@ -0,0 +1,28 @@
+# To enable LTO, add following in local.conf
+# require conf/distro/include/lto.inc
+# DISTRO_FEATURES_append = " lto"
+#
+
+# Disable LTO for following packages
+LTO_pn-glibc = ""
+LTO_pn-gcc-runtime = ""
+LTO_pn-libgcc-initial = ""
+LTO_pn-libgcc = ""
+LTO_pn-libpam = ""
+LTO_pn-elfutils = ""
+LTO_pn-perl = ""
+LTO_pn-busybox = ""
+LTO_pn-libxcrypt = ""
+LTO_pn-curl = ""
+LTO_pn-libcap = ""
+LTO_pn-libproxy = ""
+LTO_pn-libbsd = ""
+
+# Override it for additional or different options if needed e.g.
+# with clang thin-lto might be better for compile speed
+LTO ?= "-flto"
+
+SELECTED_OPTIMIZATION_append = "${@bb.utils.contains('DISTRO_FEATURES', 'lto', ' ${LTO}', '', d)}"
+TARGET_LDFLAGS_append_class-target = "${@bb.utils.contains('DISTRO_FEATURES', 'lto', ' ${LTO}', '', d)}"
+
+SELECTED_OPTIMIZATION[vardeps] += "LTO"
diff --git a/meta/conf/local.conf.sample.extended b/meta/conf/local.conf.sample.extended
index 9aa226a2d0..2c4b8ad8cb 100644
--- a/meta/conf/local.conf.sample.extended
+++ b/meta/conf/local.conf.sample.extended
@@ -417,3 +417,7 @@
 # ERR_REPORT_UPLOAD_ALL = "1"
 #
 # INHERIT += "report-error"
+
+# Enable LTO Distro-wide
+# require conf/distro/include/lto.inc
+# DISTRO_FEATURES_append = " lto"
-- 
2.29.2

[PATCH v4 2/4] python3: Enable lto if its in DISTRO_FEATURES

[Khem Raj]

python3 configure compiles on object file and then greps for strings in
it for endianness for target ,when using LTO the .o files are not nomal
ELF onjects so this test fails, since we are using --enable-lto to
enable this here we dont need to inject extra paths via bitbake anyway
therefore reset LTO variable for target

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
v3: Reset LTO variable and use --with-lto instead of --enable-lto
v4: No change

 meta/recipes-devtools/python/python3_3.9.0.bb | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3_3.9.0.bb b/meta/recipes-devtools/python/python3_3.9.0.bb
index 6402760a1b..8fe60ea016 100644
--- a/meta/recipes-devtools/python/python3_3.9.0.bb
+++ b/meta/recipes-devtools/python/python3_3.9.0.bb
@@ -74,6 +74,9 @@ export CROSSPYTHONPATH="${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dynl
 
 EXTRANATIVEPATH += "python3-native"
 
+# LTO will be enabled via packageconfig depending upong distro features
+LTO_class-target = ""
+
 CACHED_CONFIGUREVARS = " \
                 ac_cv_file__dev_ptmx=yes \
                 ac_cv_file__dev_ptc=no \
@@ -88,7 +91,7 @@ def possibly_include_pgo(d):
     
     return ''
 
-PACKAGECONFIG_class-target ??= "readline ${@possibly_include_pgo(d)} gdbm"
+PACKAGECONFIG_class-target ??= "readline ${@possibly_include_pgo(d)} gdbm ${@bb.utils.filter('DISTRO_FEATURES', 'lto', d)}"
 PACKAGECONFIG_class-native ??= "readline gdbm"
 PACKAGECONFIG_class-nativesdk ??= "readline gdbm"
 PACKAGECONFIG[readline] = ",,readline"
@@ -96,6 +99,7 @@ PACKAGECONFIG[readline] = ",,readline"
 PACKAGECONFIG[pgo] = "--enable-optimizations,,qemu-native"
 PACKAGECONFIG[tk] = ",,tk"
 PACKAGECONFIG[gdbm] = ",,gdbm"
+PACKAGECONFIG[lto] = "--with-lto,,"
 
 do_configure_prepend () {
     mkdir -p ${B}/Modules
-- 
2.29.2

[PATCH v4 3/4] lto.inc: Add -ffat-lto-objects and -fuse-linker-plugin

[Khem Raj]

This helps to improve LTO and ensure the libs can be linked with non-LTO
objects too

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/conf/distro/include/lto.inc | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/meta/conf/distro/include/lto.inc b/meta/conf/distro/include/lto.inc
index fe0f6c9f44..d7fea26c64 100644
--- a/meta/conf/distro/include/lto.inc
+++ b/meta/conf/distro/include/lto.inc
@@ -20,7 +20,16 @@ LTO_pn-libbsd = ""
 
 # Override it for additional or different options if needed e.g.
 # with clang thin-lto might be better for compile speed
-LTO ?= "-flto"
+#
+# ffat-lto-objects
+# object files that contain both the intermediate
+# language and the object code. This makes them
+# usable for both LTO linking and normal linking
+#
+# -fuse-linker-plugin
+# ensures that libraries participate in LTO by supplying intermediate
+# code from .a files to linker
+LTO ?= "-flto -ffat-lto-objects -fuse-linker-plugin "
 
 SELECTED_OPTIMIZATION_append = "${@bb.utils.contains('DISTRO_FEATURES', 'lto', ' ${LTO}', '', d)}"
 TARGET_LDFLAGS_append_class-target = "${@bb.utils.contains('DISTRO_FEATURES', 'lto', ' ${LTO}', '', d)}"
-- 
2.29.2

[PATCH v4 4/4] lto: Introduce LTOEXTRA variable

[Khem Raj]

Certain packages may need additional flags to enable LTO, therefore
LTOEXTRA can be used to pass those flags

Add -flto-partition=none for alsa-libs

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/conf/distro/include/lto.inc | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/meta/conf/distro/include/lto.inc b/meta/conf/distro/include/lto.inc
index d7fea26c64..39d99b157d 100644
--- a/meta/conf/distro/include/lto.inc
+++ b/meta/conf/distro/include/lto.inc
@@ -18,6 +18,13 @@ LTO_pn-libcap = ""
 LTO_pn-libproxy = ""
 LTO_pn-libbsd = ""
 
+# Custom LTO flags
+# disable partitioning/streaming algorithm since its uses ASM
+# constructs not compatible with lto
+LTOEXTRA_pn-alsa-libs = "-flto-partition=none"
+
+LTOEXTRA ?= ""
+
 # Override it for additional or different options if needed e.g.
 # with clang thin-lto might be better for compile speed
 #
@@ -29,9 +36,9 @@ LTO_pn-libbsd = ""
 # -fuse-linker-plugin
 # ensures that libraries participate in LTO by supplying intermediate
 # code from .a files to linker
-LTO ?= "-flto -ffat-lto-objects -fuse-linker-plugin "
+LTO ?= "-flto -ffat-lto-objects -fuse-linker-plugin ${LTOEXTRA}"
 
 SELECTED_OPTIMIZATION_append = "${@bb.utils.contains('DISTRO_FEATURES', 'lto', ' ${LTO}', '', d)}"
 TARGET_LDFLAGS_append_class-target = "${@bb.utils.contains('DISTRO_FEATURES', 'lto', ' ${LTO}', '', d)}"
 
-SELECTED_OPTIMIZATION[vardeps] += "LTO"
+SELECTED_OPTIMIZATION[vardeps] += "LTO LTOEXTRA"
-- 
2.29.2