[PATCH V2 1/2] lto: Add global LTO distro policy file

[PATCH V2 1/2] lto: Add global LTO distro policy file

[Khem Raj]

Distros which want to enable LTO can utilize this file, it only covers
packages from OE-Core, other layers should include there own exclusion
list for recipe which dont work with LTO

Document the needed changes in local.conf.extended

debug file generator does  not generate debugsrc packages when lto is
enabled therefore disable it

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
v2: Include feeedback for adding DISTRO_FEATURE knob

 meta/conf/distro/include/lto.inc     | 30 ++++++++++++++++++++++++++++
 meta/conf/local.conf.sample.extended |  4 ++++
 2 files changed, 34 insertions(+)
 create mode 100644 meta/conf/distro/include/lto.inc

diff --git a/meta/conf/distro/include/lto.inc b/meta/conf/distro/include/lto.inc
new file mode 100644
index 0000000000..1f76ec7c16
--- /dev/null
+++ b/meta/conf/distro/include/lto.inc
@@ -0,0 +1,30 @@
+# To enable LTO, add following in local.conf
+# require conf/distro/include/lto.inc
+# DISTRO_FEATURES_append = " lto"
+#
+
+# Disable LTO for following packages
+LTO_pn-glibc = ""
+LTO_pn-gcc-runtime = ""
+LTO_pn-libgcc-initial = ""
+LTO_pn-libgcc = ""
+LTO_pn-libpam = ""
+LTO_pn-elfutils = ""
+LTO_pn-perl = ""
+LTO_pn-busybox = ""
+LTO_pn-libxcrypt = ""
+LTO_pn-curl = ""
+LTO_pn-libcap = ""
+LTO_pn-libproxy = ""
+LTO_pn-libbsd = ""
+
+# Override it for additional or different options if needed e.g.
+# with clang thin-lto might be better for compile speed
+LTO ?= "-flto"
+
+SELECTED_OPTIMIZATION_append = "${@bb.utils.contains('DISTRO_FEATURES', 'lto', ' ${LTO}', '', d)}"
+TARGET_LDFLAGS_append_class-target = "${@bb.utils.contains('DISTRO_FEATURES', 'lto', ' ${LTO}', '', d)}"
+
+SELECTED_OPTIMIZATION[vardeps] += "LTO"
+
+PACKAGE_DEBUG_SPLIT_STYLE_lto = "debug-without-src"
diff --git a/meta/conf/local.conf.sample.extended b/meta/conf/local.conf.sample.extended
index 9aa226a2d0..2c4b8ad8cb 100644
--- a/meta/conf/local.conf.sample.extended
+++ b/meta/conf/local.conf.sample.extended
@@ -417,3 +417,7 @@
 # ERR_REPORT_UPLOAD_ALL = "1"
 #
 # INHERIT += "report-error"
+
+# Enable LTO Distro-wide
+# require conf/distro/include/lto.inc
+# DISTRO_FEATURES_append = " lto"
-- 
2.29.2

[PATCH v2 2/2] python3: Enable lto if its in DISTRO_FEATURES

[Khem Raj]

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/recipes-devtools/python/python3_3.9.0.bb | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3_3.9.0.bb b/meta/recipes-devtools/python/python3_3.9.0.bb
index 6402760a1b..7080312ccf 100644
--- a/meta/recipes-devtools/python/python3_3.9.0.bb
+++ b/meta/recipes-devtools/python/python3_3.9.0.bb
@@ -88,7 +88,7 @@ def possibly_include_pgo(d):
     
     return ''
 
-PACKAGECONFIG_class-target ??= "readline ${@possibly_include_pgo(d)} gdbm"
+PACKAGECONFIG_class-target ??= "readline ${@possibly_include_pgo(d)} gdbm ${@bb.utils.filter('DISTRO_FEATURES', 'lto', d)}"
 PACKAGECONFIG_class-native ??= "readline gdbm"
 PACKAGECONFIG_class-nativesdk ??= "readline gdbm"
 PACKAGECONFIG[readline] = ",,readline"
@@ -96,6 +96,7 @@ PACKAGECONFIG[readline] = ",,readline"
 PACKAGECONFIG[pgo] = "--enable-optimizations,,qemu-native"
 PACKAGECONFIG[tk] = ",,tk"
 PACKAGECONFIG[gdbm] = ",,gdbm"
+PACKAGECONFIG[lto] = "--enable-lto,,"
 
 do_configure_prepend () {
     mkdir -p ${B}/Modules
-- 
2.29.2

Re: [OE-core] [PATCH V2 1/2] lto: Add global LTO distro policy file

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2583 bytes --]

Does this actually work (I don't see lto added to OVERRIDES) and why is
this needed?

PACKAGE_DEBUG_SPLIT_STYLE_lto = "debug-without-src"

On Sat, Nov 7, 2020 at 7:41 PM Khem Raj <raj.khem@gmail.com> wrote:

> Distros which want to enable LTO can utilize this file, it only covers
> packages from OE-Core, other layers should include there own exclusion
> list for recipe which dont work with LTO
>
> Document the needed changes in local.conf.extended
>
> debug file generator does  not generate debugsrc packages when lto is
> enabled therefore disable it
>
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> ---
> v2: Include feeedback for adding DISTRO_FEATURE knob
>
>  meta/conf/distro/include/lto.inc     | 30 ++++++++++++++++++++++++++++
>  meta/conf/local.conf.sample.extended |  4 ++++
>  2 files changed, 34 insertions(+)
>  create mode 100644 meta/conf/distro/include/lto.inc
>
> diff --git a/meta/conf/distro/include/lto.inc
> b/meta/conf/distro/include/lto.inc
> new file mode 100644
> index 0000000000..1f76ec7c16
> --- /dev/null
> +++ b/meta/conf/distro/include/lto.inc
> @@ -0,0 +1,30 @@
> +# To enable LTO, add following in local.conf
> +# require conf/distro/include/lto.inc
> +# DISTRO_FEATURES_append = " lto"
> +#
> +
> +# Disable LTO for following packages
> +LTO_pn-glibc = ""
> +LTO_pn-gcc-runtime = ""
> +LTO_pn-libgcc-initial = ""
> +LTO_pn-libgcc = ""
> +LTO_pn-libpam = ""
> +LTO_pn-elfutils = ""
> +LTO_pn-perl = ""
> +LTO_pn-busybox = ""
> +LTO_pn-libxcrypt = ""
> +LTO_pn-curl = ""
> +LTO_pn-libcap = ""
> +LTO_pn-libproxy = ""
> +LTO_pn-libbsd = ""
> +
> +# Override it for additional or different options if needed e.g.
> +# with clang thin-lto might be better for compile speed
> +LTO ?= "-flto"
> +
> +SELECTED_OPTIMIZATION_append = "${@bb.utils.contains('DISTRO_FEATURES',
> 'lto', ' ${LTO}', '', d)}"
> +TARGET_LDFLAGS_append_class-target =
> "${@bb.utils.contains('DISTRO_FEATURES', 'lto', ' ${LTO}', '', d)}"
> +
> +SELECTED_OPTIMIZATION[vardeps] += "LTO"
> +
> +PACKAGE_DEBUG_SPLIT_STYLE_lto = "debug-without-src"
> diff --git a/meta/conf/local.conf.sample.extended
> b/meta/conf/local.conf.sample.extended
> index 9aa226a2d0..2c4b8ad8cb 100644
> --- a/meta/conf/local.conf.sample.extended
> +++ b/meta/conf/local.conf.sample.extended
> @@ -417,3 +417,7 @@
>  # ERR_REPORT_UPLOAD_ALL = "1"
>  #
>  # INHERIT += "report-error"
> +
> +# Enable LTO Distro-wide
> +# require conf/distro/include/lto.inc
> +# DISTRO_FEATURES_append = " lto"
> --
> 2.29.2
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 3392 bytes --]

Re: [OE-core] [PATCH V2 1/2] lto: Add global LTO distro policy file

[Khem Raj]

On Sat, Nov 7, 2020 at 10:49 AM Martin Jansa <martin.jansa@gmail.com> wrote:
>
> Does this actually work (I don't see lto added to OVERRIDES) and why is this needed?
>
> PACKAGE_DEBUG_SPLIT_STYLE_lto = "debug-without-src"

Right, it is a nop, which means it's not needed anymore, it was needed
in the past as packaging failed for some packages.
Let me remove it and do the image build and perhaps remove it in v3.

>
> On Sat, Nov 7, 2020 at 7:41 PM Khem Raj <raj.khem@gmail.com> wrote:
>>
>> Distros which want to enable LTO can utilize this file, it only covers
>> packages from OE-Core, other layers should include there own exclusion
>> list for recipe which dont work with LTO
>>
>> Document the needed changes in local.conf.extended
>>
>> debug file generator does  not generate debugsrc packages when lto is
>> enabled therefore disable it
>>
>> Signed-off-by: Khem Raj <raj.khem@gmail.com>
>> ---
>> v2: Include feeedback for adding DISTRO_FEATURE knob
>>
>>  meta/conf/distro/include/lto.inc     | 30 ++++++++++++++++++++++++++++
>>  meta/conf/local.conf.sample.extended |  4 ++++
>>  2 files changed, 34 insertions(+)
>>  create mode 100644 meta/conf/distro/include/lto.inc
>>
>> diff --git a/meta/conf/distro/include/lto.inc b/meta/conf/distro/include/lto.inc
>> new file mode 100644
>> index 0000000000..1f76ec7c16
>> --- /dev/null
>> +++ b/meta/conf/distro/include/lto.inc
>> @@ -0,0 +1,30 @@
>> +# To enable LTO, add following in local.conf
>> +# require conf/distro/include/lto.inc
>> +# DISTRO_FEATURES_append = " lto"
>> +#
>> +
>> +# Disable LTO for following packages
>> +LTO_pn-glibc = ""
>> +LTO_pn-gcc-runtime = ""
>> +LTO_pn-libgcc-initial = ""
>> +LTO_pn-libgcc = ""
>> +LTO_pn-libpam = ""
>> +LTO_pn-elfutils = ""
>> +LTO_pn-perl = ""
>> +LTO_pn-busybox = ""
>> +LTO_pn-libxcrypt = ""
>> +LTO_pn-curl = ""
>> +LTO_pn-libcap = ""
>> +LTO_pn-libproxy = ""
>> +LTO_pn-libbsd = ""
>> +
>> +# Override it for additional or different options if needed e.g.
>> +# with clang thin-lto might be better for compile speed
>> +LTO ?= "-flto"
>> +
>> +SELECTED_OPTIMIZATION_append = "${@bb.utils.contains('DISTRO_FEATURES', 'lto', ' ${LTO}', '', d)}"
>> +TARGET_LDFLAGS_append_class-target = "${@bb.utils.contains('DISTRO_FEATURES', 'lto', ' ${LTO}', '', d)}"
>> +
>> +SELECTED_OPTIMIZATION[vardeps] += "LTO"
>> +
>> +PACKAGE_DEBUG_SPLIT_STYLE_lto = "debug-without-src"
>> diff --git a/meta/conf/local.conf.sample.extended b/meta/conf/local.conf.sample.extended
>> index 9aa226a2d0..2c4b8ad8cb 100644
>> --- a/meta/conf/local.conf.sample.extended
>> +++ b/meta/conf/local.conf.sample.extended
>> @@ -417,3 +417,7 @@
>>  # ERR_REPORT_UPLOAD_ALL = "1"
>>  #
>>  # INHERIT += "report-error"
>> +
>> +# Enable LTO Distro-wide
>> +# require conf/distro/include/lto.inc
>> +# DISTRO_FEATURES_append = " lto"
>> --
>> 2.29.2
>>
>>
>> 
>>