[lm-sensors] libsensors config file scanner speed

[lm-sensors] libsensors config file scanner speed

[Mark M. Hoffman]

Hi all:

I've committed all of my work to date on the libsensors config file scanner
to SVN.  It is on the branch called scanner-opt-branch.  I'm pretty happy
about the results so far...

# w/ w83627hf loaded
# also, use DEBUG:=1 in the Makefile

$ valgrind --tool llgrind sensors -u -c etc/sensors.conf.eg
$ kcachegrind callgrind.out.NNNN

Total cycles before:	~11.2M
Total cycles after:	~ 7.3M
Total speedup:		~ 1.5x

Scanner cycles before:	~ 5.9M
Scanner cycles after:	~ 2.0M
Scanner speedup:	~ 3.0x

I'm going to keep it on the branch until (1) we release 2.10.1, and (2) I
create some sort of regression test for it.  For now at least, it has no
*known* bugs.  Give it a try; let me know if you have trouble.

Regards,

-- 
Mark M. Hoffman
mhoffman at lightlink.com

[lm-sensors] libsensors config file scanner speed

[Jean Delvare]

Hi Mark,

> I've committed all of my work to date on the libsensors config file scanner
> to SVN.  It is on the branch called scanner-opt-branch.  I'm pretty happy
> about the results so far...
> 
> # w/ w83627hf loaded
> # also, use DEBUG:=1 in the Makefile
> 
> $ valgrind --toolÊllgrind sensors -u -c etc/sensors.conf.eg
> $ kcachegrind callgrind.out.NNNN
> 
> Total cycles before:		~11.2M
> Total cycles after:		~ 7.3M
> Total speedup:		~ 1.5x
> 
> Scanner cycles before:	~ 5.9M
> Scanner cycles after:		~ 2.0M
> Scanner speedup:		~ 3.0x

Really nice. I've been doing some measurements on my machine and here
are my results. I assumed that what you call "scanner cycles" is the
cycle count for sensors_yylex, and "total cycles" is the cycle count
for main.

Total cycles before:	~11.3M
Total cycles after:	~ 8.2M
Total speedup:		~ 1.4x

Scanner cycles before:	~ 5.1M
Scanner cycles after:	~ 1.9M
Scanner speedup:	~ 2.7x

This is on x86_64.

Another interesting point of comparison is when using the dedicated
configuration file I have for this test machine, instead of the
fat default one:

Total cycles before:	~ 4.7M
Total cycles after:	~ 4.5M
Total speedup:		~ none

Scanner cycles before:	~ 267k
Scanner cycles after:	~ 138k
Scanner speedup:	~ 1.9x

I guess the result isn't a surprise, if the configuration file is
smaller, the amount of time spent in the scanner has to be smaller as
well, thus the overall speed improvement is lower.

If we anaylze the benefit of the configuration file change (without
taking your scanner optimizations in consideration) we obtain:

Total cycles before:	~11.3M
Total cycles after:	~ 4.7M
Total speedup:		~ 2.4x

Scanner cycles before:	~ 5.1M
Scanner cycles after:	~ 267k
Scanner speedup:	~  19x

Not to discourage you from working on the scanner, every additional
improvement is welcome... But just to underline the well known fact
that sensors.conf.eg has grown much larger than the original authors of
libsensors thought it ever would, and as a result 95% of the
configuration file we install by default is useless for each user, and
ruins the "sensors" performance. This means that we really need
dedicated configuration files for people to use instead of the default
configuration file.

Other possible approachs:
* Having a smaller dedicated configuration file which would only
  contain the safest defaults (chip manufacturer recommended compute
  lines and labels). The rest could be moved to documentation.
* Having a separate default file for each chip, and copying it (or
  merging them) to /etc/sensors.conf when the user runs sensors-detect.
* Delaying the scanning of the configuration data until after we know
  which chips have been found of the system. So we could happily skip
  the data which has not been found.

Or maybe it's more interesting to put our energy in the configuration
files database, and let the default configuration file as is in the
hope that people won't use it anymore anyway.

BTW, if anyone wants to compile kcachegrind (very impressive tool,
thanks for making me discover it) on x86_64, the key for me was to pass
--enable-libsuffixd to the configure script. Without it, configure
would fail to find the right libraries.

> I'm going to keep it on the branch until (1) we release 2.10.1, and (2) I
> create some sort of regression test for it.  For now at least, it has no
> *known* bugs.  Give it a try; let me know if you have trouble.

Sounds like a good plan to me, thanks for working on this.

-- 
Jean Delvare

[lm-sensors] libsensors config file scanner speed

[Jim Cromie]

Jean Delvare wrote:
> Hi Mark,
>
>   
> Not to discourage you from working on the scanner, every additional
> improvement is welcome... But just to underline the well known fact
> that sensors.conf.eg has grown much larger than the original authors of
> libsensors thought it ever would, and as a result 95% of the
> configuration file we install by default is useless for each user, and
> ruins the "sensors" performance. This means that we really need
> dedicated configuration files for people to use instead of the default
> configuration file.
>
> Other possible approachs:
> * Having a smaller dedicated configuration file which would only
>   contain the safest defaults (chip manufacturer recommended compute
>   lines and labels). The rest could be moved to documentation.
> * Having a separate default file for each chip, and copying it (or
>   merging them) to /etc/sensors.conf when the user runs sensors-detect.
> * Delaying the scanning of the configuration data until after we know
>   which chips have been found of the system. So we could happily skip
>   the data which has not been found.
>
> Or maybe it's more interesting to put our energy in the configuration
> files database, and let the default configuration file as is in the
> hope that people won't use it anymore anyway.
>
>   

something like this ?

[jimc at harpo lm-sensors]$ ls etc/sensors.conf.d/
abituguru  adm9240   fscpos   k8temp  lm85c    maxilife  smsc47m192  
w83697hf
adm1021    as99127f  fscscy   lm63    lm87     mtp008    via686a     w83782d
adm1024    asb100    gl518sm  lm75    lm90     pc87366   vt1211      w83783s
adm1025    bmc       gl520sm  lm78    lm92     pcf8591   vt8231      w83792d
adm1030    f71805f   it87     lm80    lm99     sis5595   w83627ehf   w83793
adm1031    fscher    it8716   lm83    max1619  smsc47m1  w83627thf   
w83l785ts

assuming yes, attached script will do the scut-work.

Before you use it for real, the sensors.conf.eg file needs some comments 
moved below
the "chip <foo>" lines they pertain to, otherwize theyre written into 
the wrong file.
I'll do that juggling if you want to go this route.

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: split-sensors
Url: http://lists.lm-sensors.org/pipermail/lm-sensors/attachments/20060907/d76a67f6/attachment-0001.pl 

[lm-sensors] libsensors config file scanner speed

[Jean Delvare]

Hi Jim,

> > Other possible approachs:
> > * Having a smaller dedicated configuration file which would only
> >   contain the safest defaults (chip manufacturer recommended compute
> >   lines and labels). The rest could be moved to documentation.
> > * Having a separate default file for each chip, and copying it (or
> >   merging them) to /etc/sensors.conf when the user runs sensors-detect.
> > * Delaying the scanning of the configuration data until after we know
> >   which chips have been found of the system. So we could happily skip
> >   the data which has not been found.
> >
> > Or maybe it's more interesting to put our energy in the configuration
> > files database, and let the default configuration file as is in the
> > hope that people won't use it anymore anyway.
> 
> something like this ?
> 
> [jimc at harpo lm-sensors]$ ls etc/sensors.conf.d/
> abituguru  adm9240   fscpos   k8temp  lm85c    maxilife  smsc47m192  
> w83697hf
> adm1021    as99127f  fscscy   lm63    lm87     mtp008    via686a     w83782d
> adm1024    asb100    gl518sm  lm75    lm90     pc87366   vt1211      w83783s
> adm1025    bmc       gl520sm  lm78    lm92     pcf8591   vt8231      w83792d
> adm1030    f71805f   it87     lm80    lm99     sis5595   w83627ehf   w83793
> adm1031    fscher    it8716   lm83    max1619  smsc47m1  w83627thf   
> w83l785ts

Yeah, that's what I had in mind.

> assuming yes, attached script will do the scut-work.

How do we handle the files which cover several chips? Maybe we can
limit the split to one file per driver. Else we need some mapping,
possibly achieved with symbolic links.

> Before you use it for real, the sensors.conf.eg file needs some comments 
> moved below
> the "chip <foo>" lines they pertain to, otherwize theyre written into 
> the wrong file.

Good point.

> I'll do that juggling if you want to go this route.

I'm not sure where we want to go at this point. My other proposals
above need to be examined as well, and I'd like others (especially
Mark) to comment on these.

We also need to consider the two other on-going projects affecting the
configuration file:
* Configuration file database (already mentioned above).
* 2.4 drivers and 2.6 drivers will need different configuration files
  in the future.
These issues may help us make our choice, and have much higher priority.

-- 
Jean Delvare