Dropping CONFIG_REORDER on x86-64 for 2.6.22

Dropping CONFIG_REORDER on x86-64 for 2.6.22

[Andi Kleen]

Hallo,

I'm thinking about dropping the x86-64 CONFIG_REORDER for 2.6.22. 
The function enabled -ffunction-sections and then tries to reorder
the executable

While that's in theory a worthy goal to save TLB/icache, in practice it 
didn't really work out.

Rationale:
- It cannot be enabled in normal builds because all current lds
become very slow when they have to handle thousands of sections.

- The list of hot functions it needs - arch/x86_64/kernel/functionlist -
hasn't been changed for a long time and is already out of date. 
I also don't know no sane way to keep it uptodate. Profile feedback
at runtime is not practical for the kernel and even if it's questionable
we would find a good training workload. Trying to change the manually
is imho foolish and likely not too useful anyways (if any hot function
is missed the TLB savings will be lost and it is unlikely we can
get 100% coverage) 

- The workloads of the kernel are too diverse to make this concept
really work

So I'm planning to drop the option and arch/x86_64/kernel/functionlist

Comments?

-Andi

Re: Dropping CONFIG_REORDER on x86-64 for 2.6.22

[Arjan van de Ven]

Andi Kleen wrote:
> Rationale:
> - It cannot be enabled in normal builds because all current lds
> become very slow when they have to handle thousands of sections.
> 

afaik this is only ever reported on SuSE; I've not heard it on any 
other distro...

Re: Dropping CONFIG_REORDER on x86-64 for 2.6.22

[Chuck Ebbert]

Arjan van de Ven wrote:
> Andi Kleen wrote:
>> Rationale:
>> - It cannot be enabled in normal builds because all current lds
>> become very slow when they have to handle thousands of sections.
>>
> 
> afaik this is only ever reported on SuSE; I've not heard it on any other
> distro...

It's horribly slow on Fedora.

Re: Dropping CONFIG_REORDER on x86-64 for 2.6.22

[Måns Rullgård]

Arjan van de Ven <arjan@linux.intel.com> writes:

> Andi Kleen wrote:
>> Rationale:
>> - It cannot be enabled in normal builds because all current lds
>> become very slow when they have to handle thousands of sections.
>>
>
> afaik this is only ever reported on SuSE; I've not heard it on any
> other distro...

Even with that option set, the full kernel build with my configuration
finishes in one minute flat on my Gentoo box.  Could it be that the
linker uses enormous amounts of memory?  I have 4GB so I wouldn't
immediately notice.

-- 
Måns Rullgård
mans@mansr.com

Re: [discuss] Dropping CONFIG_REORDER on x86-64 for 2.6.22

[Andi Kleen]

On Friday 20 April 2007 10:35:10 Måns Rullgård wrote:
> Arjan van de Ven <arjan@linux.intel.com> writes:
> 
> > Andi Kleen wrote:
> >> Rationale:
> >> - It cannot be enabled in normal builds because all current lds
> >> become very slow when they have to handle thousands of sections.
> >>
> >
> > afaik this is only ever reported on SuSE; I've not heard it on any
> > other distro...
> 
> Even with that option set, the full kernel build with my configuration
> finishes in one minute flat on my Gentoo box.  Could it be that the
> linker uses enormous amounts of memory?  I have 4GB so I wouldn't
> immediately notice.

What binutils version do you use? Maybe that particular problem is finally fixed.
Of course would still leave the other issues I listed.

-Andi

Re: [discuss] Dropping CONFIG_REORDER on x86-64 for 2.6.22

[Måns Rullgård]

Andi Kleen <ak@suse.de> writes:

> On Friday 20 April 2007 10:35:10 Måns Rullgård wrote:
>> Arjan van de Ven <arjan@linux.intel.com> writes:
>> 
>> > Andi Kleen wrote:
>> >> Rationale:
>> >> - It cannot be enabled in normal builds because all current lds
>> >> become very slow when they have to handle thousands of sections.
>> >>
>> >
>> > afaik this is only ever reported on SuSE; I've not heard it on any
>> > other distro...
>> 
>> Even with that option set, the full kernel build with my configuration
>> finishes in one minute flat on my Gentoo box.  Could it be that the
>> linker uses enormous amounts of memory?  I have 4GB so I wouldn't
>> immediately notice.
>
> What binutils version do you use? Maybe that particular problem is
> finally fixed.  Of course would still leave the other issues I
> listed.

I'm using the Gentoo default, which is 2.16.1 plus some patches.  A
quick scan of the patches suggests that perhaps this one has something
to do with it:
http://sources.redhat.com/ml/binutils/2005-03/msg00847.html

-- 
Måns Rullgård
mans@mansr.com

Re: Dropping CONFIG_REORDER on x86-64 for 2.6.22

[Adrian Bunk]

On Thu, Apr 19, 2007 at 01:54:52PM +0200, Andi Kleen wrote:
> 
> Hallo,
> 
> I'm thinking about dropping the x86-64 CONFIG_REORDER for 2.6.22. 
> The function enabled -ffunction-sections and then tries to reorder
> the executable
> 
> While that's in theory a worthy goal to save TLB/icache, in practice it 
> didn't really work out.
> 
> Rationale:
>...
> So I'm planning to drop the option and arch/x86_64/kernel/functionlist
> 
> Comments?

Have there been any benchmarks showing performance improvements 
with -ffunction-sections?

> -Andi

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Dropping CONFIG_REORDER on x86-64 for 2.6.22

[David Miller]

From: Andi Kleen <ak@suse.de>
Date: Thu, 19 Apr 2007 13:54:52 +0200

> So I'm planning to drop the option and arch/x86_64/kernel/functionlist

Please do so, I'm tired of editing that file every time I remove
something from the tree.

That file had alloc_skb_from_cache() in it, which nothing in the
vanilla kernel ever invoked.  How did it even get there?  If it was
put there for Xen's sake, that stinks because Xen is out of tree.

Re: Dropping CONFIG_REORDER on x86-64 for 2.6.22

[Herbert Xu]

David Miller <davem@davemloft.net> wrote:
> 
> That file had alloc_skb_from_cache() in it, which nothing in the
> vanilla kernel ever invoked.  How did it even get there?  If it was
> put there for Xen's sake, that stinks because Xen is out of tree.

I think it was included because this is a list of all functions,
so no conspiracies to see here :)

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt