CONFIG_KALLSYMS_EXTRA_PASS

CONFIG_KALLSYMS_EXTRA_PASS

[Steven Rostedt]

OK, I'm working on a custom kernel, and suddenly I'm getting the compile
error "Try setting CONFIG_KALLSYMS_EXTRA_PASS".  I've also just did a
debian update, but that doesn't seem to bother the vanilla kernel.

What did I screw up to cause this kind of error?  For now I'll just set
this CONFIG option, but I would like to also know how to fix this. Where
do I start to look?

Thanks,

-- Steve


FYI:

$ diff -u System.map .tmp_System.map
--- System.map  2005-05-17 17:00:45.000000000 -0400
+++ .tmp_System.map     2005-05-17 17:00:45.000000000 -0400
@@ -25485,9 +25485,9 @@
 c03a1290 D kallsyms_addresses
 c03aae14 D kallsyms_num_syms
 c03aae18 D kallsyms_names
-c03c2b98 D kallsyms_markers
-c03c2c34 D kallsyms_token_table
-c03c300c D kallsyms_token_index
+c03c2b80 D kallsyms_markers
+c03c2c1c D kallsyms_token_table
+c03c2ff4 D kallsyms_token_index
 c03c4000 D idt_table
 c03c4000 A __nosave_begin
 c03c4000 A __nosave_end

Looks to me that the kallsyms_names have changed.
System.map has 0x17d80 bytes and .tmp_System.map has 0x17d68 bytes with
the difference of 0x18 or 24 bytes (six 32bit words). 

Re: CONFIG_KALLSYMS_EXTRA_PASS

[pmarques]

Quoting Steven Rostedt <rostedt@goodmis.org>:
> OK, I'm working on a custom kernel, and suddenly I'm getting the compile
> error "Try setting CONFIG_KALLSYMS_EXTRA_PASS".  I've also just did a
> debian update, but that doesn't seem to bother the vanilla kernel.

This is probably the same problem that me and other people are having.

It seems that sometimes the symbol that marks the end of a section changes
position with the symbol that marks the beggining of the next section if they
happen to on the same address (they might be on different addresses due to
alignment issues).

In this case the compression algorithm might produce different compression
ratios and the kallsyms compressed data changes size.

You can try the very crude (but effective) way to check if this is your problem
or not. Go to scripts/kallsyms.c and change:

#define WORKING_SET             1024

to:

#define WORKING_SET             65536

This will force kallsyms to use *all* the symbols for the compression, and the
size of the result won't be affected by the symbol positions.

Don't forget to turn off KALLSYMS_EXTRA_PASS to test this.

If this turns out to be the problem _again_, I'll post a patch to fix this for
good by storing the token data from the first pass and use it on the second
pass. This will not only speed up compression, it will also guarantee that this
kind of problems will never bite us again.

--
Paulo Marques

Re: CONFIG_KALLSYMS_EXTRA_PASS

[Steven Rostedt]

On Wed, 2005-05-18 at 00:27 +0100, pmarques@grupopie.com wrote:

> You can try the very crude (but effective) way to check if this is your problem
> or not. Go to scripts/kallsyms.c and change:
> 
> #define WORKING_SET             1024
> 
> to:
> 
> #define WORKING_SET             65536
> 

Yep, that did the trick. Thanks.  And just to make sure, I put it back
to 1024, recompiled, and got the error again.

> This will force kallsyms to use *all* the symbols for the compression, and the
> size of the result won't be affected by the symbol positions.
> 
> Don't forget to turn off KALLSYMS_EXTRA_PASS to test this.
> 
> If this turns out to be the problem _again_, I'll post a patch to fix this for
> good by storing the token data from the first pass and use it on the second
> pass. This will not only speed up compression, it will also guarantee that this
> kind of problems will never bite us again.

Your patch sounds too good to not be included even if this wasn't the
case. How come it hasn't been applied before?

-- Steve

Re: CONFIG_KALLSYMS_EXTRA_PASS

[pmarques]

Quoting Steven Rostedt <rostedt@goodmis.org>:

> On Wed, 2005-05-18 at 00:27 +0100, pmarques@grupopie.com wrote:
> 
> > You can try the very crude (but effective) way to check if this is your
> problem
> > or not. Go to scripts/kallsyms.c and change:
> > 
> > #define WORKING_SET             1024
> > 
> > to:
> > 
> > #define WORKING_SET             65536
> > 
> 
> Yep, that did the trick. Thanks.  And just to make sure, I put it back
> to 1024, recompiled, and got the error again.
> 
> > This will force kallsyms to use *all* the symbols for the compression, and
> the
> > size of the result won't be affected by the symbol positions.
> > 
> > [...]
> 
> Your patch sounds too good to not be included even if this wasn't the
> case. How come it hasn't been applied before?

This is not really the patch, its just a quick fix. The problem with this fix is
that, not only it will take a lot more memory to build the kallsyms data, it
will take about 6 seconds on a P4 2.8GHz to compress 20000 symbols which is
about the number of symbols of a i386 defconfig, IIRC.

The actual patch must still use a small working set of symbols to improve
performance, but keep the token data from the first pass to use on the second.
This will actually decrease the time for the kallsyms calculation instead of
increasing it, as the previous solution did.

I've still not written the patch because I was waiting for some confirmation
that it would have some chance of acceptance. I guess I'll just write it and
send it upstream and see how that goes.

Thanks for taking the time to confirm the problem,

--
Paulo Marques