Re: [llvmlinux] percpu | bitmap issue? (Cannot boot on bare metal due to a kernel NULL pointer dereference)

[Ingo Molnar <mingo@kernel.org>]

* Christoph Lameter <cl@linux.com> wrote:

> On Mon, 14 Sep 2015, Austin S Hemmelgarn wrote:
> 
> >I can comment at least a little about the -Os aspect (although not I'm no
> >expert on this in particular).  In general, for _most_ use cases, a kernel
> >compiled with CONFIG_CC_OPTIMIZE_FOR_SIZE will run slower than one compiled
> >without it.  On rare occasion though, it may actually run faster, the only
> >cases I've seen where this happens are specialized uses that are very memory
> >pressure dependent and run almost entirely in userspace with almost no
> >syscalls (for example math related stuff operating on _very, very big_ (as in,
> >>1 trillion elements) multidimensional matrices, with complex memory
> >constraints), and even then it's usually a miniscule improvement in
> >performance (generally less than 1%, which can of course be significant
> >depending on how long it takes before the improvement).
> 
> Cache footprint depends on size which has a significant impact on
> performance. In our experience the kernel (and any other code) is
> generally faster if optimized for size.

Unfortunately, GCC overdoes -Os generating outright silly code, which makes the 
result generally slower - despite the reduced instruction count and reduced cache 
footprint.

We've recently applied patches to the x86 tree that give us a good chunk of the 
size savings that -Os brings:

  52648e83c9a6 x86: Pack loops tightly as well
  be6cb02779ca x86: Align jump targets to 1-byte boundaries

these two shave about 5% off from the typical distro kernel's size. That's still 
way off the 15%-20% that -Os can muster, but another ~10% are possible by not 
aligning functions to byte boundaries (instead of the default 16 bytes).

So about 70% of the -Os size win is from simple and pure alignment relaxation, not 
from any deeper compiler optimizations.

So LLVM could emulate most of the good effects of -Os by only compressing the 
various alignment parameters - and this would be a pretty safe optimization as 
well.

Thanks,

	Ingo