Re: speedup in tag lookup using hash tables

[Arnaldo Carvalho de Melo <acme-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]

Em Tue, Feb 12, 2008 at 01:24:44PM +0200, Ilpo Järvinen escreveu:
> On Mon, 11 Feb 2008, Arnaldo Carvalho de Melo wrote:
> 
> > 	Today I optimized the dwarves a bit by using a per object file
> > hash table for tag lookup, it yelded almost 50% speedup on pahole when
> > running on a vmlinux file :-)
> >
> > 	codiff doesn't makes that much tag lookups, so we didn't got
> > much improvements there,
> 
> Because I had to do these file by file anyway, I ended up using modified 
> timestamps in my scripts to avoid loading .o files altogether when one 
> wasn't recompiled. It helped some.

This reminds me that I have to add support comparing build trees, so
that we can use 'make O=old' to build a baseline and then 'make O=new',
using ccache to reuse the old (or simply copying old to new) and then
process file after file, combining the results for tree wide
comparisions/statistics.
 
> > but I'm doing experiments on dead tag elimination
> > that probably will help a lot there, but for that I have first to grok
> > Ulrich Drepper's libdisasm to find out what are the tags that are really
> > used by looking at accesses to register indexed memory areas that use as
> > a base pointer what is in local/global variables and function
> > parameters.
> >
> > 	This also will provide the basis for detecting access patterns
> > that will ultimatelly allow libdwarves_reorganize to do struct
> > reorganizations to improve locality of reference, etc.
> 
> This sounds nice, I was thinking something similar earlier but would have 
> tried to do that with some kind of source analysis.

That is possible too, and indeed I did experiments in the past with
sparse, the library used by the kernel checker (make C=[12]), but I
think that tapping into the readily available debuginfo packages, not
just for the kernel, looking at the end, object file, result is better.
 
> > 	Please take a look at v1.6 that I pushed today and tell me your
> > impressions.
> 
> Thanks, I'll check that later on. My bruteforce inline remover run over 
> all include/ stuff just finished yesterday after ~1.5 days (ie. ~5800 
> inlines compile tested on ~90 slaves, next time I'll have to plug ccache 
> into that site-specific distribute thing I use to speed it up a lot :-)), 
> I'll post the results later on to lkml+netdev with some patches and 
> thoughts, the winner seems to be this beauty :-) :
>    -110805  869 funcs, 198 +, 111003 -, diff: -110805 --- skb_put
> ...And 22 other are in 10000+ category and 235 1kB+ (some overlap 
> exists due to __-funcs). What still remains to do are the non-include/ 
> headers which might reveal some candies too.

Excellent! Some people at some point noted that while the dwarves are
useful for showing the problems, people actually had to follow up with
patches to solve the problems, I'm very happy that you are doing just
that, thanks a lot for that.

- Arnaldo