[RFC] init.h: mark init functions hot instead of cold

[RFC] init.h: mark init functions hot instead of cold

[Rasmus Villemoes]

attribute((cold)) causes gcc to optimize the function for size rather
than speed. But since __init functions will be discarded anyway, I
don't see why memory should be a concern. On the contrary, everybody
wants their box to boot faster. Using the opposite attribute, hot,
causes gcc to optimize the functions more aggressively, possibly at
the expense of larger (.init).text. A completely unscientific test
showed about 2% faster boot time: I booted a kernel in qemu with and
without this patch five times each; the boot times were very stable in
each case, so I think the 2% is ok, but of course only applies to that
specific .config running in a virtual machine on my hardware.

__cold also means any path to a call of such a function is treated as
unlikely, while __hot means the opposite. For intra-__init calls, I
don't think that matters. That leaves calls from __ref functions to
__init functions. While I also think it doesn't matter in that case,
I'm sure someone can tell me why I'm wrong.

Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
---
 include/linux/compiler-gcc4.h | 6 ++----
 include/linux/compiler.h      | 8 ++++++++
 include/linux/init.h          | 2 +-
 3 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/include/linux/compiler-gcc4.h b/include/linux/compiler-gcc4.h
index 769e19864632..b5e5c96d538e 100644
--- a/include/linux/compiler-gcc4.h
+++ b/include/linux/compiler-gcc4.h
@@ -27,11 +27,9 @@
    Early snapshots of gcc 4.3 don't support this and we can't detect this
    in the preprocessor, but we can live with this because they're unreleased.
    Maketime probing would be overkill here.
-
-   gcc also has a __attribute__((__hot__)) to move hot functions into
-   a special section, but I don't see any sense in this right now in
-   the kernel context */
+ */
 #define __cold			__attribute__((__cold__))
+#define __hot			__attribute__((__hot__))
 
 #define __UNIQUE_ID(prefix) __PASTE(__PASTE(__UNIQUE_ID_, prefix), __COUNTER__)
 
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 0e41ca0e5927..50dbdc8d570d 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -357,6 +357,14 @@ static __always_inline void __write_once_size(volatile void *p, void *res, int s
 #define __cold
 #endif
 
+/*
+ * The opposite of __cold: Tell the compiler to optimize the function
+ * more aggressively, and treat paths leading to a call as likely.
+ */
+#ifndef __hot
+#define __hot
+#endif
+
 /* Simple shorthand for a section definition */
 #ifndef __section
 # define __section(S) __attribute__ ((__section__(#S)))
diff --git a/include/linux/init.h b/include/linux/init.h
index 21b6d768edd7..b6153a612ea0 100644
--- a/include/linux/init.h
+++ b/include/linux/init.h
@@ -39,7 +39,7 @@
 
 /* These are for everybody (although not all archs will actually
    discard it in modules) */
-#define __init		__section(.init.text) __cold notrace
+#define __init		__section(.init.text) __hot notrace
 #define __initdata	__section(.init.data)
 #define __initconst	__constsection(.init.rodata)
 #define __exitdata	__section(.exit.data)
-- 
2.1.3

Re: [RFC] init.h: mark init functions hot instead of cold

[Andi Kleen]

On Sat, May 09, 2015 at 12:45:01AM +0200, Rasmus Villemoes wrote:
> attribute((cold)) causes gcc to optimize the function for size rather
> than speed. But since __init functions will be discarded anyway, I
> don't see why memory should be a concern. On the contrary, everybody

It makes the bzImage smaller.
A lot of people on smaller systems are interested in flash size.

> wants their box to boot faster. Using the opposite attribute, hot,
> causes gcc to optimize the functions more aggressively, possibly at
> the expense of larger (.init).text. A completely unscientific test
> showed about 2% faster boot time: I booted a kernel in qemu with and
> without this patch five times each; the boot times were very stable in
> each case, so I think the 2% is ok, but of course only applies to that
> specific .config running in a virtual machine on my hardware.

2% on boot is basically noise.

-Andi

Re: [RFC] init.h: mark init functions hot instead of cold

[Josh Triplett]

On Sat, May 09, 2015 at 02:17:39AM -0700, Andi Kleen wrote:
> On Sat, May 09, 2015 at 12:45:01AM +0200, Rasmus Villemoes wrote:
> > attribute((cold)) causes gcc to optimize the function for size rather
> > than speed. But since __init functions will be discarded anyway, I
> > don't see why memory should be a concern. On the contrary, everybody
> 
> It makes the bzImage smaller.
> A lot of people on smaller systems are interested in flash size.

True, but people interested in flash size can set
CONFIG_CC_OPTIMIZE_FOR_SIZE.

I would propose dropping "cold" and *not* adding "hot"; just let __init
functions get default optimizations.  People who set
CONFIG_CC_OPTIMIZE_FOR_SIZE should get similar size optimizations, and
people who don't will get GCC's normal optimizations, which should
provide much of the improved boot performance Rasmus observed.

Rasmus, can you confirm that just dropping cold 1) doesn't make the
kernel larger when building with CONFIG_CC_OPTIMIZE_FOR_SIZE, and 2)
gives approximately the same 2% performance benefit you observed?

> > wants their box to boot faster. Using the opposite attribute, hot,
> > causes gcc to optimize the functions more aggressively, possibly at
> > the expense of larger (.init).text. A completely unscientific test
> > showed about 2% faster boot time: I booted a kernel in qemu with and
> > without this patch five times each; the boot times were very stable in
> > each case, so I think the 2% is ok, but of course only applies to that
> > specific .config running in a virtual machine on my hardware.
> 
> 2% on boot is basically noise.

I disagree; there are people working on shaving milliseconds from boot.

- Josh Triplett