[PCI] building PCI IDs/drivers DB from Linux kernel sources

[PCI] building PCI IDs/drivers DB from Linux kernel sources

[Thierry Laronde]

[-- Attachment #1: Type: text/plain, Size: 4105 bytes --]

[I'm sorry for crossposting, but since it could be of some interest for
Debian installation AND involves the linux sources...]

Please note that in the following, these are remarks _not_ bad criticism.
Maybe what is found by the script could be of some interest to people
coordinating the source writing.

GOAL
----

In order to allow a kind of light detection of hardware to be use during
installation, I wanted to build a database (for PCI: I start with the
easiest...) with the following format:

CLASS_ID	VENDOR_ID	DEVICE_ID	driver_name

I have decided to write a script (you will find all the stuff attached)
parsing the Linux kernel sources in order to do that.

RULES
-----

The strategy adopted is to try to find rules that apply to a great number of
files, if not all, and to treat the least possible exceptions by hand.

For retrieving the information I found 2 main rules.

R1: mainly for the net drivers, since one person is at the origin (Donald
Becker), I scan the files in order to discover pci_tbl arrays of
pci_device_id, and retrieve info from here.
	Since some files have the magic numbers hardcoded, and others use
	constant names, the files are preprocessed.
	When the preprocess fails (config.h is created, asm symlink too), I
	suspect that the driver is not buildable for the ARCH

R2: for the others, since the devices ids are spread all around, I search
for PCI_DEVICE_ID_* strings, extract them from the sources, construct
PCI_VENDOR_ID and put the driver name, and finally try to preprocess the
database, against pci_ids.h and others macro definitions found in the
sources.

PROBLEMS
--------

Macro definitions:

I have found two macros not defined anywhere (sorry, this is for 2.4.6; I
haven't try for later):

PCI_DEVICE_ID_PHILIPS_SAA9730
PCI_DEVICE_ID_V3_SEMI_V370PDC

Driver names:

I try to match the filename processed (minus the suffix) against entries in
the Makefile, and even try a substring against the Makefile in order to
guess the correct driver name. When everything fails, the log file indicates
the problem, and I have built a "drivers_aliases" giving the correct name
for these files.

Class ids:

This is one of the main problem. For some drivers, the directory tree allows
an easy matching. For example for 'net', the drivers at the root are
ethernet, tokenring is in a subdirectory, etc...
For others, for example in 'char' one must almost try to put the match by
hand [this is not a surprise since in Unix like kernels char is a default].

Completeness:

At the moment, the DB (you will find it in the tarball attached) is an
almost 600 entries one. But I have only treated the "regular" (rules 1 or 2)
files and there remains 30% of exceptions.

SOLUTIONS ?
-----------

To be able to construct almost automagically the DB could be of some
interest, since pci_ids is not exhaustive, and definitions are spread all
around.
May I suggest some possible tracks?

* Use of macro definitions for magic numbers

Could the policy be to not hardcode the magic numbers but only to use
preprocessing and put PCI_DEVICE_* etc in place of the hexa values? This
could allow to use only rule 2.

* Define the driver name in the file

The driver name is not always easy to automatically guess. Could the policy
be that the driver name is defined in the source via a macro ?

* Define the class name in the file too

Trying to organize sources following the PCI classes distinction will not be
always a good idea. IMHO, the simple way would be to define too in the
drivers source a macro defining the class.

The class id is important for example to sort the DB. If one has to
construct an installer and has a limited place (general case for initrd),
one could want to extract only all the records for net devices from the DB,
or only SCSI controlers and so on...


Thank for reading,

Note: The DB constructed is _not_ garanty to be bullet proof. For example,
since this is only alpha tests, the class id are probably wrong for some
devices. Comments absolutely welcomed.
-- 
Thierry LARONDE, Centre de Ressources Informatiques, Archamps - France
http://www.cri74.org/

[-- Attachment #2: dmodtools.tgz --]
[-- Type: application/x-gtar, Size: 12885 bytes --]

Re: [PCI] building PCI IDs/drivers DB from Linux kernel sources

[Giacomo Catenazzi]

Hello.

I've already done something and it should be included on kernel 2.5,
with my
autoprobe configuration utility.
The sources are in
http://people.debian.org/~cate/files/kautoconfigure/autoconfigure/
(note that autoconfigure.rule is a simple bash script, so you can
extract
and transform data in a few shell commands)

My format is:
   check_pci 'PCI_ID' NAME_OF_CONFIG # kernel/file
where PCI_ID is a regexp of the form:
   vendor_id, device_id, subvendor_id, subdevice_id; class, interface
(mail me for further details)

Note you should manually enter the devices, because there are a lot of
exceptions:
motherboard workaround in drivers,...


(I will chek you program and maybe I would stole some of your result, It
is GPL?)

	giacomo

Re: [PCI] building PCI IDs/drivers DB from Linux kernel sources

[Thierry Laronde]

On Mon, Jul 30, 2001 at 12:04:58PM +0200, Giacomo Catenazzi wrote:
> Hello.

Hello,
> 
> I've already done something and it should be included on kernel 2.5,
> with my
> autoprobe configuration utility.
> The sources are in
> http://people.debian.org/~cate/files/kautoconfigure/autoconfigure/
> (note that autoconfigure.rule is a simple bash script, so you can
> extract
> and transform data in a few shell commands)

OK, I will give it a look.
> 
> (I will chek you program and maybe I would stole some of your result, It
> is GPL?)

Yes! (this is indicated in cpcidb.linux --- the main script; yes I know the
name is too long...)

Cheers,
-- 
Thierry LARONDE, Centre de Ressources Informatiques, Archamps - France
http://www.cri74.org/

Re: [PCI] building PCI IDs/drivers DB from Linux kernel sources

[Tim Waugh]

[-- Attachment #1: Type: text/plain, Size: 1186 bytes --]

On Mon, Jul 30, 2001 at 11:33:19AM +0200, Thierry Laronde wrote:

> I have decided to write a script (you will find all the stuff attached)
> parsing the Linux kernel sources in order to do that.

For the MODULE_DEVICE_TABLE case, isn't it less fragile to parse the
binary modules?

If there were a program to spit out the MODULE_DEVICE_TABLE in a
format easily parseable by scripts, it would be great; is there such a
program already?

> Driver names:
> 
> I try to match the filename processed (minus the suffix) against entries in
> the Makefile, and even try a substring against the Makefile in order to
> guess the correct driver name. When everything fails, the log file indicates
> the problem, and I have built a "drivers_aliases" giving the correct name
> for these files.

Again for the MODULE_DEVICE_TABLE case, there is nothing to guess.

> May I suggest some possible tracks?
> 
> * Use of macro definitions for magic numbers
[...]
> * Define the driver name in the file
[...]
> * Define the class name in the file too
[...]

None of this is needed if people just use the MODULE_DEVICE_TABLE
mechanism that is already provided.

Tim.
*/

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [PCI] building PCI IDs/drivers DB from Linux kernel sources

[Marcus Meissner]

In article <20010730113319.A24939@pc04.cri.cur-archamps.fr> you wrote:
> Please note that in the following, these are remarks _not_ bad criticism.
> Maybe what is found by the script could be of some interest to people
> coordinating the source writing.

> GOAL
> ----

> In order to allow a kind of light detection of hardware to be use during
> installation, I wanted to build a database (for PCI: I start with the
> easiest...) with the following format:

> CLASS_ID	VENDOR_ID	DEVICE_ID	driver_name

> I have decided to write a script (you will find all the stuff attached)
> parsing the Linux kernel sources in order to do that.

Well, that was what I did 2 years ago for Caldera ;)

Howevery this is no longer needed.

Nearly all PCI kernel modules now export the ids they match for in the
MODULE_DEVICE_TABLE, for PCI, ISAPNP and USB.

So either read it from /lib/modules/<kernelver>/modules.*map, or 
use the modutils code that extracts this information from the
.o files itself.

You need to compile those, but you usually do that anyway for a kernel
build.

Ciao, Marcus

Re: [PCI] building PCI IDs/drivers DB from Linux kernel sources

[Keith Owens]

On Mon, 30 Jul 2001 11:33:19 +0200, 
Thierry Laronde <thierry@cri74.org> wrote:
>In order to allow a kind of light detection of hardware to be use during
>installation, I wanted to build a database (for PCI: I start with the
>easiest...) with the following format:
>
>CLASS_ID	VENDOR_ID	DEVICE_ID	driver_name

depmod already builds /lib/modules/`uname -r`/modules.pcimap containing
all the data required for PCI identification.  Before you reinvent too
many wheels, see the hotplug project,
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: [PCI] building PCI IDs/drivers DB from Linux kernel sources

[Thierry Laronde]

On Mon, Jul 30, 2001 at 09:06:21PM +1000, Keith Owens wrote:
> On Mon, 30 Jul 2001 11:33:19 +0200, 
> Thierry Laronde <thierry@cri74.org> wrote:
> >In order to allow a kind of light detection of hardware to be use during
> >installation, I wanted to build a database (for PCI: I start with the
> >easiest...) with the following format:
> >
> >CLASS_ID	VENDOR_ID	DEVICE_ID	driver_name
> 
> depmod already builds /lib/modules/`uname -r`/modules.pcimap containing
> all the data required for PCI identification.  Before you reinvent too
> many wheels, see the hotplug project,
> http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Thanks for the pointer. I will give a look.
-- 
Thierry LARONDE, Centre de Ressources Informatiques, Archamps - France
http://www.cri74.org/

Re: [PCI] building PCI IDs/drivers DB from Linux kernel sources

[Thierry Laronde]

On Mon, Jul 30, 2001 at 01:03:01PM +0200, Marcus Meissner wrote:
> In article <20010730113319.A24939@pc04.cri.cur-archamps.fr> you wrote:
> > Please note that in the following, these are remarks _not_ bad criticism.
> > Maybe what is found by the script could be of some interest to people
> > coordinating the source writing.
> 
> > GOAL
> > ----
> 
> > In order to allow a kind of light detection of hardware to be use during
> > installation, I wanted to build a database (for PCI: I start with the
> > easiest...) with the following format:
> 
> > CLASS_ID	VENDOR_ID	DEVICE_ID	driver_name
> 
> > I have decided to write a script (you will find all the stuff attached)
> > parsing the Linux kernel sources in order to do that.
> 
> Well, that was what I did 2 years ago for Caldera ;)
> 
> Howevery this is no longer needed.
> 
> Nearly all PCI kernel modules now export the ids they match for in the
> MODULE_DEVICE_TABLE, for PCI, ISAPNP and USB.

I might have missed something ;) The use of MODULE_DEVICE_TABLE seemed not
general. Thanks for the infos.
-- 
Thierry LARONDE, Centre de Ressources Informatiques, Archamps - France
http://www.cri74.org/

Re: [PCI] building PCI IDs/drivers DB from Linux kernel sources

[Glenn]

On Mon, 30 Jul 2001 11:33:19 +0200
"Thierry Laronde" <thierry@cri74.org> wrote:

> 
> In order to allow a kind of light detection of hardware to be use during
> installation, I wanted to build a database (for PCI: I start with the
> easiest...) with the following format:
> 
> CLASS_ID	VENDOR_ID	DEVICE_ID	driver_name
> 
> I have decided to write a script (you will find all the stuff attached)
> parsing the Linux kernel sources in order to do that.
> 
<snip>
> 
> Note: The DB constructed is _not_ garanty to be bullet proof. For example,
> since this is only alpha tests, the class id are probably wrong for some
> devices. Comments absolutely welcomed.

I imported your db into gnumeric, sorted it to put the driver first,
converted it to html (and .csv).

http://people.debian.org/~bug1/dmod/linux-2.4.6-i386-pci.html

A couple of possible problem entries

The comma in the following
53c7,8xx  256  4096  1
53c7,8xx  256  4096  2
53c7,8xx  256  4096  3
53c7,8xx  256  4096  4

and no number for device_id
saa9730  512  4401  PCI_DEVICE_ID_PHILIPS_SAA9730
pmc551  512  4528  PCI_DEVICE_ID_V3_SEMI_V370PDC

(im not subscribed to lk-ml)


Glenn

Re: [PCI] building PCI IDs/drivers DB from Linux kernel sources

[Eric Van Buggenhaut]

On Mon, Jul 30, 2001 at 11:33:19AM +0200, Thierry Laronde wrote:

[...]

> PROBLEMS
> --------
> 
> Macro definitions:
> 
> I have found two macros not defined anywhere (sorry, this is for 2.4.6; I
> haven't try for later):
> 
> PCI_DEVICE_ID_PHILIPS_SAA9730

Still not defined in 2.4.7 ...

> PCI_DEVICE_ID_V3_SEMI_V370PDC

defined in 2.4.7 in include/linux/mtd/pmc551.h:51

#ifndef PCI_DEVICE_ID_V3_SEMI_V370PDC
#define PCI_DEVICE_ID_V3_SEMI_V370PDC     0x0200
#endif

-- 
Eric VAN BUGGENHAUT

Eric.VanBuggenhaut@AdValvas.be