Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Martin Knoblauch]

> From: Dave Jones (davej@suse.de)
> Date: Wed Jan 02 2002 - 18:31:33 EST
> 
> 
> On Wed, 2 Jan 2002, Alan Cox wrote:
> 
> > > What's wrong with a startup routine that includes something like:
> > > dmidecode > /var/run/dmi
> > Absolutely nothing, and that also handily means it isnt setuid 8)
> 
> Indeed, it's perfect. Except no distro does it (yet), but it's
> definitly the best idea so far in this thread.
> 

 seeing this thread - is there any serious work being spend on something
like "hinv" on IRIX, which gives you a *complete* listing of your
hardware? I have seen some attempts at shell and perl scripts, but none
of them really is trustworthy.

Martin
-- 
+-----------------------------------------------------+
|Martin Knoblauch                                     |
|-----------------------------------------------------|
|http://www.knobisoft.de/cats                         |
|-----------------------------------------------------|
|e-mail: knobi@knobisoft.de                           |
+-----------------------------------------------------+

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Dave Jones]

On Fri, Jan 04, 2002 at 12:32:48PM +0100, Martin Knoblauch wrote:
 >  seeing this thread - is there any serious work being spend on something
 > like "hinv" on IRIX, which gives you a *complete* listing of your
 > hardware? I have seen some attempts at shell and perl scripts, but none
 > of them really is trustworthy.

 When devicefs is ready (or more to the point, the drivers become
 devicefs aware), something to the effect of ls -R /devices 
 should be possible.

 If we have ACPI fill out the tree, it enumerates pretty much
 every device you have in the system, and half dozen you probably
 didn't know you had.

Dave.

-- 
Dave Jones.                    http://www.codemonkey.org.uk
SuSE Labs.

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Paul Jakma]

On Fri, 4 Jan 2002, Dave Jones wrote:

>  When devicefs is ready (or more to the point, the drivers become
>  devicefs aware), something to the effect of ls -R /devices 
>  should be possible.

how does devicefs differ from devfs? eg, on some of my systems i mount 
devfs on /devfs and an ls -l of it shows all the devices that 
currently have drivers that registered them.

> Dave.

--paulj

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Dave Jones]

On Sat, 5 Jan 2002, Paul Jakma wrote:

> how does devicefs differ from devfs? eg, on some of my systems i mount
> devfs on /devfs and an ls -l of it shows all the devices that
> currently have drivers that registered them.

different goals. One of the reasons this has come about is for power
management, we need a tree like structure so that we for eg, power
down a network card before powering down the pci bridge it sits on.
(The idea being to power down from the leaves, and work your way back
up to the root of the tree)

devicefs is just a means of exporting this to userspace, be that for
usage with the userspace acpi tools, or for hinv like programs.
As I mentioned earlier, ACPI enumerates pretty much everything in the
system, even if theres no driver for it.
If there is a driver for it, it can register things like "I support
these power saving states" with driverfs for additional functionality.

It would be nice at some point to get some of the other (pre-ACPI)
busses registering stuff there too, for completeness.

-- 
| Dave Jones.        http://www.codemonkey.org.uk
| SuSE Labs

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Patrick Mochel]

On Sat, 5 Jan 2002, Dave Jones wrote:

> On Sat, 5 Jan 2002, Paul Jakma wrote:
>
> > how does devicefs differ from devfs? eg, on some of my systems i mount
> > devfs on /devfs and an ls -l of it shows all the devices that
> > currently have drivers that registered them.
>
> different goals. One of the reasons this has come about is for power
> management, we need a tree like structure so that we for eg, power
> down a network card before powering down the pci bridge it sits on.
> (The idea being to power down from the leaves, and work your way back
> up to the root of the tree)
>
> devicefs is just a means of exporting this to userspace, be that for
> usage with the userspace acpi tools, or for hinv like programs.
> As I mentioned earlier, ACPI enumerates pretty much everything in the
> system, even if theres no driver for it.
> If there is a driver for it, it can register things like "I support
> these power saving states" with driverfs for additional functionality.
>
> It would be nice at some point to get some of the other (pre-ACPI)
> busses registering stuff there too, for completeness.

One of the ideas that I've kicked around with some people here and the
ACPI guys is the notion of trigger device enumeration from userspace
completely.

During the initramfs stage, a program (say devmgr) figures out what type
of system you have, where the PCI buses are, etc. It tells the kernel this
information, which then probes for existence, then loads drivers.

There of course needs to be a fallback mechanism for cases where the
hardware isn't known about, the tables are buggy, or you need to do
things like make bios32 calls. But, that can be triggered via a procfs
file, a system call, etc.

But, this all fits nicely with the notion of parsing firmware tables from
userspace, be them DMI, MPS, or ACPI tables. How they're read is
unimportant (/dev/mem or /var/run/*), it's just that the logic to do so
can be moved out of the kernel and consolidated.

	-pat

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Dave Jones]

On Mon, 7 Jan 2002, Patrick Mochel wrote:

> One of the ideas that I've kicked around with some people here and the
> ACPI guys is the notion of trigger device enumeration from userspace
> completely.
>
> During the initramfs stage, a program (say devmgr) figures out what type
> of system you have, where the PCI buses are, etc. It tells the kernel this
> information, which then probes for existence, then loads drivers.

Sounds remarkably like the work that Greg has been doing with hotplug
support.

-- 
| Dave Jones.        http://www.codemonkey.org.uk
| SuSE Labs

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Greg KH]

On Mon, Jan 07, 2002 at 07:11:03PM +0100, Dave Jones wrote:
> On Mon, 7 Jan 2002, Patrick Mochel wrote:
> 
> > One of the ideas that I've kicked around with some people here and the
> > ACPI guys is the notion of trigger device enumeration from userspace
> > completely.
> >
> > During the initramfs stage, a program (say devmgr) figures out what type
> > of system you have, where the PCI buses are, etc. It tells the kernel this
> > information, which then probes for existence, then loads drivers.
> 
> Sounds remarkably like the work that Greg has been doing with hotplug
> support.

Yup :)

But I wanted to rely on the existing PCI and USB core code to do the
probing of the busses and devices, as it knows how to do this the best
right now.  Whenever it finds a new device it calls out to /sbin/hotplug
with the device info.  The userspace program at that location then loads
the proper driver for the device, if it knows about it.  This is the
same code and process that runs when the kernel is up and running today.
No code duplication is a good thing :)

And the /sbin/hotplug program knows about _all_ devices that the
currently compiled kernel can handle due to the MODULE_DEVICE_TABLE tags
in the drivers.

See the linux-hotplug project's documentation for more info on this:
	http://linux-hotplug.sf.net/
A paper and presentation about the linux-hotplug process:
	http://www.kroah.com/linux/

dietHotplug, a _very_ tiny implementation of /sbin/hotplug which is was
created exactly for the initramfs stage:
	http://www.kernel.org/pub/linux/utils/kernel/hotplug/diethotplug-0.3.tar.gz

thanks,

greg k-h

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Greg KH]

On Mon, Jan 07, 2002 at 10:50:01AM -0800, Greg KH wrote:
> 
> And the /sbin/hotplug program knows about _all_ devices that the
> currently compiled kernel can handle due to the MODULE_DEVICE_TABLE tags
> in the drivers.

Along these lines, I am very disappointed in looking at the
autoconfigure stuff in CML2.  It should be taking all of the device and
driver matching information from the kernel itself, as it is already
specified there.

Look at the modules.*map files.  They specify the kernel drivers that
specific devices work for.  They are automatically created from the
kernel that you just built.

Eric, if you are going to keep your "2000+" configuration probes up to
date by hand, good luck.  Look at all of the new USB drivers that have
been added in just the 2.5.2-pre series alone.  That's a lot of data to
keep track of.

The rest of us have decided to rely on automatic tools for this process :)

thanks,

greg k-h

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Eric S. Raymond]

Greg KH <greg@kroah.com>:
> On Mon, Jan 07, 2002 at 10:50:01AM -0800, Greg KH wrote:
> > 
> > And the /sbin/hotplug program knows about _all_ devices that the
> > currently compiled kernel can handle due to the MODULE_DEVICE_TABLE tags
> > in the drivers.
> 
> Along these lines, I am very disappointed in looking at the
> autoconfigure stuff in CML2.  It should be taking all of the device and
> driver matching information from the kernel itself, as it is already
> specified there.

I'm taking my rules file from Giacomo Catenazzi, who developed it
originally for a shellscript he wrote.  I don't know how he generates
the hardware probes; for all I know, he's got code groveling through
the module device tables.

I've been meaning to ask you about this, Giacomo.  Where *did* all
those probes come from?
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Don't think of it as `gun control', think of it as `victim
disarmament'. If we make enough laws, we can all be criminals.

Re: [kbuild-devel] Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Giacomo Catenazzi]

Greg KH wrote:

> On Mon, Jan 07, 2002 at 10:50:01AM -0800, Greg KH wrote:
> 
>>And the /sbin/hotplug program knows about _all_ devices that the
>>currently compiled kernel can handle due to the MODULE_DEVICE_TABLE tags
>>in the drivers.
>>
> 
> Along these lines, I am very disappointed in looking at the
> autoconfigure stuff in CML2.  It should be taking all of the device and
> driver matching information from the kernel itself, as it is already
> specified there.
> 
> Look at the modules.*map files.  They specify the kernel drivers that
> specific devices work for.  They are automatically created from the
> kernel that you just built.


modules.*map exist only on compiled kernel. And entry depends on
architecture and on configuration.

But don't worry. I use the kernel source to find the
MODULES_DEVICE_TABLE (with a partially automated script) to build the
new tables.
For USB and PNP this works great. For PCI I have some more problems:
the item found with MODULES_DEVICE_TABLE are the item marked with '#!'.
You see that the not all drivers use MODULES_DEVICE_TABLE.
An the most important missing dirvers are the IDE cards.

> 
> Eric, if you are going to keep your "2000+" configuration probes up to
> date by hand, good luck.  Look at all of the new USB drivers that have
> been added in just the 2.5.2-pre series alone.  That's a lot of data to
> keep track of.
> 
> The rest of us have decided to rely on automatic tools for this process :)


Slowly I make the proces more automatic (adding more 'special' cases),
i.e. I now I try to find the '#IFDEF' in the middle of the
*_device_id tables and setting the data correctly).

The sources is always the kernel.

	giacomo

Re: [kbuild-devel] Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Greg KH]

On Tue, Jan 08, 2002 at 09:04:14AM +0100, Giacomo Catenazzi wrote:
> modules.*map exist only on compiled kernel. And entry depends on
> architecture and on configuration.

I agree.  That's why something like what David has proposed would be
more helpful here.

> But don't worry. I use the kernel source to find the
> MODULES_DEVICE_TABLE (with a partially automated script) to build the
> new tables.

Do you check for devices that are now handled by different drivers?  For
example, the CDCEther and acm USB drivers have gone though a series of
different configuration changes, where one driver would claim devices
meant for the other.  It is hopefully all fixes up now, but might have
confused your scripts.

What about devices that are supported by more than one driver?  How do
you handle that?  (see the USB keyspan_pda and keyspan drivers for an
example.)

thanks,

greg k-h

Re: [kbuild-devel] Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Giacomo Catenazzi]

Greg KH wrote:
> 
> What about devices that are supported by more than one driver?  How do
> you handle that?  (see the USB keyspan_pda and keyspan drivers for an
> example.)

I check the devices supported by multiple drivers, and normally
I comment both probes (This is why I have so much probes commented.
These drivers are marked in my db '## 2x').
Sometime I arbitrary choice a device.
I.e. in FS 'autofs4' is prefered to 'autofs'.
In the case of USB, two driver 'register' the uhci cards.
I prefer USB_UHCI_ALT to USB_UHCI (arbitrary choice).

I would like comments and correction to these choices.
(The kernel is to big to know all, and the sources are not
usefull in some cases)


BTW the database is available to all. If somebody need it
I can confirm that the database is complete (for MODULES_TABLE
items).
The database have these informations:
- type (pci, usb, pnp,...)
- device ID (ev. with class) or class ID, or interface ID or function ID, or both
- configuration variable
- source (the kernel file where the device is defined)
(for non autoprobe use, you should discard the ev. first '#', CONFIG = "_" means 'always',
in pci the "!" before the source means from MODULES_TABLE, the other pci are detected by other
methods (no so accurate)



	giacomo

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Dave Jones]

On Mon, 7 Jan 2002, Greg KH wrote:

> dietHotplug, a _very_ tiny implementation of /sbin/hotplug which is was
> created exactly for the initramfs stage:
> 	http://www.kernel.org/pub/linux/utils/kernel/hotplug/diethotplug-0.3.tar.gz

which reminds me of another initramfs issue. I noticed you included
dietlibc in the diethotplug (hence the name I guess), but has any
decision been made yet as to what's going to be chosen as an
initlibc/klibc ?

There are several small libc's out there, but I've not seen any
discussion either for or against any of them.

-- 
| Dave Jones.        http://www.codemonkey.org.uk
| SuSE Labs

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Greg KH]

On Mon, Jan 07, 2002 at 07:58:49PM +0100, Dave Jones wrote:
> which reminds me of another initramfs issue. I noticed you included
> dietlibc in the diethotplug (hence the name I guess), but has any
> decision been made yet as to what's going to be chosen as an
> initlibc/klibc ?

I don't know.  I asked the dietLibc people if they would be willing to
create a stripped down version of it and help port it to the remaining
archs that Linux supports, but dietLibc doesn't, and didn't hear
anything back.

It doesn't look like much work to do the stripping (I did a bunch of it
for the latest version of dietHotplug) but the porting, I have no idea
of what is needed there.

Anyone want to start up a klibc project? :)

greg k-h

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Dave Jones]

On Mon, 7 Jan 2002, Greg KH wrote:

> I don't know.  I asked the dietLibc people if they would be willing to
> create a stripped down version of it and help port it to the remaining
> archs that Linux supports, but dietLibc doesn't, and didn't hear
> anything back.

That doesn't fill me with confidence. This thing will need
maintainence after initial merge.

> It doesn't look like much work to do the stripping (I did a bunch of it
> for the latest version of dietHotplug) but the porting, I have no idea
> of what is needed there.
> Anyone want to start up a klibc project? :)

That's not half a bad idea. If we want a _maintained_ libc for the kernel,
having it maintained by kernel folks may make sense. There's nothing
stopping us borrowing bits from dietlibc and friends after all.

-- 
| Dave Jones.        http://www.codemonkey.org.uk
| SuSE Labs

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Alexander Viro]

On Mon, 7 Jan 2002, Dave Jones wrote:

> > It doesn't look like much work to do the stripping (I did a bunch of it
> > for the latest version of dietHotplug) but the porting, I have no idea
> > of what is needed there.
> > Anyone want to start up a klibc project? :)
> 
> That's not half a bad idea. If we want a _maintained_ libc for the kernel,
> having it maintained by kernel folks may make sense. There's nothing
> stopping us borrowing bits from dietlibc and friends after all.

Count me in.  Having libc that wouldn't make me puke at every look at
the source had been my dream since waaay back...

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Alan Cox]

> That's not half a bad idea. If we want a _maintained_ libc for the kernel,
> having it maintained by kernel folks may make sense. There's nothing
> stopping us borrowing bits from dietlibc and friends after all.

Why not _work_ with the dietlibc people instead of creating the fifth or
sixth small libc project ?

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Erik Andersen]

On Tue Jan 08, 2002 at 02:00:19PM +0000, Alan Cox wrote:
> > That's not half a bad idea. If we want a _maintained_ libc for the kernel,
> > having it maintained by kernel folks may make sense. There's nothing
> > stopping us borrowing bits from dietlibc and friends after all.
> 
> Why not _work_ with the dietlibc people instead of creating the fifth or
> sixth small libc project ?

And/or the uClibc people....

 -Erik

--
Erik B. Andersen             http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Patrick Mochel]

On Mon, 7 Jan 2002, Greg KH wrote:

> On Mon, Jan 07, 2002 at 07:11:03PM +0100, Dave Jones wrote:
> > On Mon, 7 Jan 2002, Patrick Mochel wrote:
> >
> > > One of the ideas that I've kicked around with some people here and the
> > > ACPI guys is the notion of trigger device enumeration from userspace
> > > completely.
> > >
> > > During the initramfs stage, a program (say devmgr) figures out what type
> > > of system you have, where the PCI buses are, etc. It tells the kernel this
> > > information, which then probes for existence, then loads drivers.
> >
> > Sounds remarkably like the work that Greg has been doing with hotplug
> > support.
>
> Yup :)
>
> But I wanted to rely on the existing PCI and USB core code to do the
> probing of the busses and devices, as it knows how to do this the best
> right now.  Whenever it finds a new device it calls out to /sbin/hotplug
> with the device info.  The userspace program at that location then loads
> the proper driver for the device, if it knows about it.  This is the
> same code and process that runs when the kernel is up and running today.
> No code duplication is a good thing :)
>
> And the /sbin/hotplug program knows about _all_ devices that the
> currently compiled kernel can handle due to the MODULE_DEVICE_TABLE tags
> in the drivers.
>
> See the linux-hotplug project's documentation for more info on this:
> 	http://linux-hotplug.sf.net/
> A paper and presentation about the linux-hotplug process:
> 	http://www.kroah.com/linux/
>
> dietHotplug, a _very_ tiny implementation of /sbin/hotplug which is was
> created exactly for the initramfs stage:
> 	http://www.kernel.org/pub/linux/utils/kernel/hotplug/diethotplug-0.3.tar.gz

It's very closely related; kinda like kissing cousins.

/sbin/hotplug is called from the kernel only, right?

I see no reason to change that at all for notification of devices that are
plugged in/removed by suprise.

I was thinking, though, more along the lines of triggering the probe for
devices that the kernel has a tough time finding on its own. E.g. peer
Host/PCI bridges, batteries, etc.

	-pat

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Greg KH]

On Mon, Jan 07, 2002 at 11:19:06AM -0800, Patrick Mochel wrote:
> 
> It's very closely related; kinda like kissing cousins.
> 
> /sbin/hotplug is called from the kernel only, right?

Right.  But there's no reason it can't be called from any other place.
It's just a userspace program with a well documented interface :)

> I see no reason to change that at all for notification of devices that are
> plugged in/removed by suprise.

Also realize that the first scan of a bus looks just like a device was
plugged in from the subsystem's point of view.

> I was thinking, though, more along the lines of triggering the probe for
> devices that the kernel has a tough time finding on its own. E.g. peer
> Host/PCI bridges, batteries, etc.

Ah, things that do not have individual kernel module drivers right now?

greg k-h

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[David Brownell]

> > /sbin/hotplug is called from the kernel only, right?
>
> Right.  But there's no reason it can't be called from any other place.
> It's just a userspace program with a well documented interface :)

Actually, it's already called from other places ... :)
/sbin/hotplug is called from the init.d/hotplug startup script.

There's a period during initial system boot when not all filesystems
or system services are available.  When the kernel calls out to
/sbin/hotplug in such a situation, hotplugging can't yet do it's job.

It's too early, the system isn't "hot" yet ... which is why I call this
problem the "coldplug" issue.   Even simple device setup
operations like modprobing may not be possible, much less
more complex ones like alerting/starting daemons.  So the
init.d/hotplug script, invoked later, fakes hotplug events to
make sure the same setup gets done, without requiring users
to unplug/replug devices.

- Dave

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Greg KH]

On Mon, Jan 07, 2002 at 12:36:32PM -0800, David Brownell wrote:
> 
> It's too early, the system isn't "hot" yet ... which is why I call this
> problem the "coldplug" issue.   Even simple device setup
> operations like modprobing may not be possible, much less
> more complex ones like alerting/starting daemons.  So the
> init.d/hotplug script, invoked later, fakes hotplug events to
> make sure the same setup gets done, without requiring users
> to unplug/replug devices.

Hopefully, integration of /sbin/hotplug during the boot process (using
dietHotplug) will reduce the number of things the "coldplug" issue will
have to handle.

greg k-h

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[David Brownell]

> Hopefully, integration of /sbin/hotplug during the boot process (using
> dietHotplug) will reduce the number of things the "coldplug" issue will
> have to handle.

Somewhat -- though it only handles the "load a module"
subproblem.  When new devices need any more setup
than that, "dietHotplug" isn't enough.

- Dave

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Greg KH]

On Mon, Jan 07, 2002 at 02:28:38PM -0800, David Brownell wrote:
> > Hopefully, integration of /sbin/hotplug during the boot process (using
> > dietHotplug) will reduce the number of things the "coldplug" issue will
> > have to handle.
> 
> Somewhat -- though it only handles the "load a module"
> subproblem.  When new devices need any more setup
> than that, "dietHotplug" isn't enough.

Agreed.  dietHotplug doesn't want to solve that problem right now.  I'll
leave that up to the main linux-hotplug scripts :)

greg k-h

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Kevin Easton]

> > Hopefully, integration of /sbin/hotplug during the boot process (using 
> > dietHotplug) will reduce the number of things the "coldplug" issue will 
> > have to handle. 
> 
> 
> Somewhat -- though it only handles the "load a module" 
> subproblem. When new devices need any more setup 
> than that, "dietHotplug" isn't enough. 
> 
> 
> - Dave 

What if this was handled by the kernel not sending hotplug messages until it
had been told that the system was ready to load drivers etc.

init.d/hotplug tells the kernel that userspace is ready to hear about hotplug
events, and the kernel kicks off by telling /sbin/hotplug about the devices
that are already in the system at startup.

It should probably be a priority or bitmask, rather than a simple on/off
switch, so that the diethotplug in initramfs can be told about the devices
needed for booting (but the TV tuner can wait until the heavyweight hotplug
is around).

...or have I totally misunderstood the coldplug problem?

    - Kevin.

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[David Brownell]

> > > Hopefully, integration of /sbin/hotplug during the boot process (using 
> > > dietHotplug) will reduce the number of things the "coldplug" issue will 
> > > have to handle. 
> > 
> > Somewhat -- though it only handles the "load a module" 
> > subproblem. When new devices need any more setup 
> > than that, "dietHotplug" isn't enough. 
> 
> What if this was handled by the kernel not sending hotplug messages until it
> had been told that the system was ready to load drivers etc.

That would prevent dynamic loading of modules when they're needed
during the early stages of system booting, not just delay the "etc" set of
device setup subproblems.

However, there's already machinery that handles part of that "queue
up hotplug events for later delivery".  The original issue was that when
the network subsystem needed to report hotplug events, it needed to
be able to do it when some locks were held and sometimes even
in_interrupt(), so the "fork a subprocess" work had to be handed off
to some other kernel thread.


> ...or have I totally misunderstood the coldplug problem?

I suspect so.  The kernel's role with hotplug is intentionally limited:  just
tell userland policy agents "here's a device, it may need to be configured".
Configuring devices can be a complex problem, even when the driver is
already linked into the kernel.

The coldplug problem is that it's saying those things before the system
has been brought up far enough that the agents can do anything!  As in,
sometimes even before a root filesystem is mounted, and often before
other filesystems or essential system servcies are available.

So if in those cases the agent notification can't get the device set up,
then some other boot component needs to do that work.  It's better to
have one subsystem (hotplug) handling that consistently, no matter
when the device needs to be set up, than to have different code to
handle "post-boot" (hotplug) and "during-boot" (kudzu etc) cases.
Which is why the "hotplug" tools have a "coldplug" mode too.

To nudge this slightly back towards the original topic, I'll just point
out that doing this "before-boot" (as part of kernel config) is a horse
of a different color, although it needs most of the same metadata from
the kernel.  The device detection has to be done with a tool other than
the running kernel, and it never actually sets up devices for users.
All it cares about is coming up with config flags that cause the
right modules to be built and (statically or dynamically) linked.

- Dave

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Kai Germaschewski]

On Sat, 5 Jan 2002, Dave Jones wrote:

> devicefs is just a means of exporting this to userspace, be that for
> usage with the userspace acpi tools, or for hinv like programs.
> As I mentioned earlier, ACPI enumerates pretty much everything in the
> system, even if theres no driver for it.
> If there is a driver for it, it can register things like "I support
> these power saving states" with driverfs for additional functionality.
> 
> It would be nice at some point to get some of the other (pre-ACPI)
> busses registering stuff there too, for completeness.

I had a patch which showed ISAPnP devices within driverfs at one point. 
Once the switch to struct device is in the kernel, I'll dig it out and 
bring it up to date.

--Kai

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Patrick Mochel]

On Sat, 5 Jan 2002, Paul Jakma wrote:

> On Fri, 4 Jan 2002, Dave Jones wrote:
>
> >  When devicefs is ready (or more to the point, the drivers become
> >  devicefs aware), something to the effect of ls -R /devices
> >  should be possible.
>
> how does devicefs differ from devfs? eg, on some of my systems i mount
> devfs on /devfs and an ls -l of it shows all the devices that
> currently have drivers that registered them.

It's actually driverfs ;). (I know it's confusing, I wanted devfs, but it
was already taken.)

It exports devices based on their locality. On my test box, I have this
output:

sh-2.05# find pci0/ -type d
pci0/
pci0/00:1f.4
pci0/00:1f.3
pci0/00:1f.2
pci0/00:1f.1
pci0/00:1f.0
pci0/00:02.0
pci0/00:00.0

Nodes are added by the bus driver as it enumerates the bus, before
device-specific drivers are loaded.

devfs groups devices based on device class (video, net, disk, etc). Adding
primitive support for this should be pretty easy to driverfs, though there
are many nasty details to work out.

Basically, each device already has a directory. When it registers with its
class subsystem (like networking), the subsystem creates a symlink:

class/net/1 -> pci0/00:02.0

(or something like that). It doesn't seem that there needs to be any
explicit devfs_register() in the driver. I could be wrong, but that's my
initial impression...

	-pat

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Richard Gooch]

Patrick Mochel writes:
> 
> On Sat, 5 Jan 2002, Paul Jakma wrote:
> 
> > On Fri, 4 Jan 2002, Dave Jones wrote:
> >
> > >  When devicefs is ready (or more to the point, the drivers become
> > >  devicefs aware), something to the effect of ls -R /devices
> > >  should be possible.
> >
> > how does devicefs differ from devfs? eg, on some of my systems i mount
> > devfs on /devfs and an ls -l of it shows all the devices that
> > currently have drivers that registered them.
> 
> It's actually driverfs ;). (I know it's confusing, I wanted devfs, but it
> was already taken.)
> 
> It exports devices based on their locality. On my test box, I have this
> output:
> 
> sh-2.05# find pci0/ -type d
> pci0/
> pci0/00:1f.4
> pci0/00:1f.3
> pci0/00:1f.2
> pci0/00:1f.1
> pci0/00:1f.0
> pci0/00:02.0
> pci0/00:00.0
> 
> Nodes are added by the bus driver as it enumerates the bus, before
> device-specific drivers are loaded.

This is in fact something that I've had planned for devfs as well: a
/dev/bus hierarchy.

> devfs groups devices based on device class (video, net, disk,
> etc). Adding primitive support for this should be pretty easy to
> driverfs, though there are many nasty details to work out.

Well, devfs itself doesn't lay things out, it's the drivers that do
this. My plan is to have /dev/scsi/host# become a symlink to something
like /dev/bus/pci0/slot1/function1 (a directory).

It still eludes me why a new device FS was developed when devfs
already has the mechanisms that are needed.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Dave Jones]

On Mon, 7 Jan 2002, Richard Gooch wrote:

> It still eludes me why a new device FS was developed when devfs
> already has the mechanisms that are needed.

For one, driverfs can be made mandatory. Sure we could do the same for
devfs, but there are probably an army of people who don't want
a mandatory devfs.

-- 
| Dave Jones.        http://www.codemonkey.org.uk
| SuSE Labs

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Alexander Viro]

On Mon, 7 Jan 2002, Dave Jones wrote:

> On Mon, 7 Jan 2002, Richard Gooch wrote:
> 
> > It still eludes me why a new device FS was developed when devfs
> > already has the mechanisms that are needed.
> 
> For one, driverfs can be made mandatory. Sure we could do the same for
> devfs, but there are probably an army of people who don't want
> a mandatory devfs.

... for another, there's an old saying about doing one thing and doing
it well.  Devfs is _not_ following that.

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Paul Jakma]

On Mon, 7 Jan 2002, Dave Jones wrote:

> For one, driverfs can be made mandatory. Sure we could do the same for
> devfs, but there are probably an army of people who don't want
> a mandatory devfs.

but devfs internally is a driver API for registering stuff. it can be
made mandatory without requiring the user-side interface, ie /dev dir,
to be mandatory.

(indeed, i dont mount devfs on /dev).

obviously devfs must also be made to pass Al's goodness-o-meter.  :)

in terms of the kernel interface it'd be daft to have multiple device
registration interfaces.

--paulj

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Patrick Mochel]

> It still eludes me why a new device FS was developed when devfs
> already has the mechanisms that are needed.

It's really pretty simple, and I don't mean any disrespect by it, so don't
take it personally.

When I started working on this new driver model thingy, I wanted to export
the device tree, so I could individually turn devices on and off. So, I
did it via procfs.

One fine, sunny day, I'm explaining the concept to Linus, and he says
"Don't use proc."

"What do you mean, 'don't use proc'?" (Since I already had done it).

He pointed out that it was old and crufty, and already over-abused. He
suggested that I write my own fs to do it.

So I did. It was easy. I blatantly ripped off ramfs, and it worked.

I looked at devfs for inspiration, but I didn't get very far. It seemed
way too complex for what it was trying to do. And, it was taking too much
frickin' time to figure what the hell you were doing.

Besides, at the time, it was an orthogonal problem. I didn't care about
device class functionality, only the hierarchy.

That's still what I mainly care about. I realized a while back that it
would be relatively simple to add class support. But, I've stayed away
from it for political reasons. I predict there will be an integrated
solution.

Besides, everyone hates devfs. Being the image-conscious guy that I am, I
don't want to play for the team that everyone hates.

Basically, I think I'm just Linus' bitch in this whole scheme of things
for

1) creating a decent /dev replacement
2) helping motivate you to fix devfs
and/or
3) helping motivate people to modernize/fix procfs

He gets what he wants and I take the heat.

I don't really care what gets used. I like my code because I wrote it. It
might suck and be buggy; but I'm willing to fix it, and play by the common
rules. I want something that works and that other people are happy with.

	-pat

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[Andrew Morton]

Patrick Mochel wrote:
> 
> ...
> "What do you mean, 'don't use proc'?" (Since I already had done it).
> 
> He pointed out that it was old and crufty, and already over-abused. He
> suggested that I write my own fs to do it.
> 
> So I did. It was easy. I blatantly ripped off ramfs, and it worked.
> 

One little problem I have with driverfs (and with the whole "everything
is a filesystem" idea) is that the infrastructure just isn't ready for
it yet.

For example, driverfs_create_dir().  That has no driverfs-specific
content at all.  It's 100% mucking with VFS internals.

The more cut-n-paste-filesystems we create, the harder it all gets
to maintain.  We need a library layer which simplifies and abstracts
these operations.  (Says me, looking at you-know-who :)

-

Re: Hardware Inventory [was: Re: ISA slot detection on PCI systems?]

[David Brownell]

Greg, you wrote:

> > And the /sbin/hotplug program knows about _all_ devices that the
> > currently compiled kernel can handle due to the MODULE_DEVICE_TABLE
> > tags in the drivers.
>
> Along these lines, I am very disappointed in looking at the
> autoconfigure stuff in CML2.  It should be taking all of the device and
> driver matching information from the kernel itself, as it is already
> specified there.

I sent related comments to Eric last April:

> > One thing I noticed is that there's important information in the build
> > system that's discarded after a build.  That "metadata" can be handy
> > for lots of purposes.   Hotplugging drivers (including configuring them
> > after they're loaded) is one application.

So far as I can tell, nobody's trying to do much with that metadata.
Maybe in part because it's not easily accessible.  But that's exactly
something I'd expect the 2.5 build/configure system to start fixing.

MODULE_DEVICE_TABLE support is actually a good example.

Its format isn't quite tool-friendly, but the real issue is that such data
is only available _after_ a build!  In this case it's needed before
config.  And if viewed as metadata, it's also incomplete:

    - Drivers omitted from the current config are not listed.
    - It's discarded for statically linked drivers (affects hotplug)
    - Doesn't say what CONFIG_ flag corresponds to each driver
    - No driver-to-docs linkage
    - ...

For anyone configuring a kernel who's not already an expert (we're
not talking Aunt Tillie ... most technical folk shouldn't be kernel experts),
that information can be important.

(One example:  driver docs.  I did some XSLT hacks a while
back, available off the linux-hotplug "links" page.  They do a better
job for USB than PCI.  Sample output (very old now :) is at

http://linux-hotplug.sourceforge.net/kernel/kernel.html

The early text is boilerplate explanation, but sections 3 and 4 are
generated automagically from modutils output, as driven by the
MODULE_DEVICE_TABLE info.  I think such info should be far
more accessible than it is.)


> Eric, if you are going to keep your "2000+" configuration probes up to
> date by hand, good luck.  Look at all of the new USB drivers that have
> been added in just the 2.5.2-pre series alone.  That's a lot of data to
> keep track of.
>
> The rest of us have decided to rely on automatic tools for this process :)

Absolutely!  What I think is needed in this case is automatic tools to
generate such stuff from kernel sources, without needing to build the
whole thing.  Run once before "configure me a kernel", to build the
driver database.

And, maybe more in the "I have a dream ..." category, take this rare
opportunity (2.5 cycle reworking build/config) to take a look at the
forest rather than the trees.  Useful tools and documents will come
from making this metadata more generally accessible.

- Dave