Unloading drivers, start-up, shut-down and some clean-ups

Unloading drivers, start-up, shut-down and some clean-ups

[Stamatis Mitrofanis]

[-- Attachment #1: Type: text/plain, Size: 4467 bytes --]

  Hello everybody. I have the 2001_04_24 release of the hotplug scripts. 
I figured it would be good to be able to automatically unload drivers 
too so I did some work on the subject. Unfortunately, it _wasn't_ as 
simple as implementing the "remove" event in the policy agents and so 
that makes it a _lot_ of work on the subject. :-)

For properly unloading drivers there need to be some changes in the way 
the boot scripts, the agent scripts and the driver-specific 
scripts (well, that doesn't leave much else :-)) are organized. Here's what:

My idea is to put the functonality of the ".rc" boot scripts in the 
".agent" scripts and have a central subsystem which triggers all others 
(via devices of it) at startup and shutdown -- creating a hierarchy this 
way. This means adding "start" and "stop" events and getting rid of 
"/etc/hotplug/*.rc" scripts. An example is that a PCI device (the host 
controller) is what "enables" the USB subsystem (or a part of it in the 
case of multiple controlers). So it goes for all other cases as long as 
the hotplug scripts are dealing only with hardware devices. This way the 
subsystems will be "started" and "stopped" in an order which is more 
appropriate than the current lexical order.

My suggestion is also to have driver-specific "*.agent" (instead of just 
"*" scripts:-)) scripts which will be called _in place_ of  (and not in 
addition to) modprobe (of course, they will call modprobe themselves if 
necessary).
- This makes it possible for scripts to do their stuff before and/or 
after "modprobe"ing.
- It also eliminates the need for a blacklist (you (or a package) can 
simply add empty agents)
- It completes the previous hierarchical design which I was talking 
about. "uhci.agent" for example can "start" the USB subsystem when "added" .
- The driver-specific *.agent files can be enumerated very simply (with 
this glob).
- The driver-specific policy agents can have their own homes and invite 
friends and relatives as well... :-) I mean that each driver-specific 
agent can have its own driver-specific subdirectory (just like each 
subsystem-specific agent can have its own subsystem-specific subdirectory).
- Aside the previous, each driver (and this is the most useful part) can 
have its own .usbmap (or .pcimap or similar) in the subsystem-specific 
directory. This *.usbmap (as an example) file should be parsed as well 
by usbmodules. This makes usb.handmap, usb.usermap and similar files 
unnecessary.

Another thing is that the individual policy agents should be separated 
from each other (/etc/hotplug/hotplug.functions is ugly :-)) and thus 
should have very few common parts. That's because future hotplug agents 
might not be only (or at all) for (un)loading kernel modules for 
hardware devices and they just can't ignore any common functions (like 
driver-loading functions or so) that are given to them. Now, that may 
sound bad since it means duplicating a couple of lines of script but I'm 
playing safe :-)...
Here's a file layout:
- Anything matching /etc/hotplug* is hotplug specific
- Anything matching /etc/hotplug/xxx* is xxx-subsystem-specific.
- Anything matching /etc/hotplug/xxx/yyy* is yyy-driver-specific.
Here's a uniform naming scheme:
- If * corresponds to nothing then it's a directory.
- If * corresponds to ".agent" then it's a script that processes hotplug 
requests.
- If * corresponds to ".conf" then it's sourced from the corresponding 
.agent script and related ones.

So, my suggestion is also to create subsystem-specific .conf files (as 
well as a global one) that will be sourced from agent scripts and will 
define relevant variables and functions. The global /etc/hotplug.conf 
file will define the "msg" function ("mesg" is already used as a program 
name!!!) and a variable for the "central" subsystem (normally pci). It 
will also be sourced from /etc/init.d/hotplug .

I must admit now that my fingers are getting tired... :-) I hope I 
covered everything I changed. All this work I did was mostly a rewrite 
and a clean-up of the scripts. Of course, I have attached a patch for 
the 2001_04_24 release. I have tested my changes on two machines with 
two USB devices and it seems to load/unload everything fine ("works for 
me") like the drivers for sound cards, UHCIs etc. . Yet it needs a few 
FIXMEs fixed to be 100% .

Furthermore, I hope I did something positive here so please, anyone, do 
what you will to help.

All yours(!),
Stamatis


[-- Attachment #2: hotplug-patch.diff.gz --]
[-- Type: application/x-gzip, Size: 32670 bytes --]

Re: Unloading drivers, start-up, shut-down and some clean-ups

[David Brownell]

>   Hello everybody. I have the 2001_04_24 release of the hotplug scripts. 
> I figured it would be good to be able to automatically unload drivers 
> too so I did some work on the subject. Unfortunately, it _wasn't_ as 
> simple as implementing the "remove" event in the policy agents and so 
> that makes it a _lot_ of work on the subject. :-)

It was waiting for volunteers ... we seem to have a few now!  :)
Presumably you'll provide a version against the current release?


> My idea is to put the functonality of the ".rc" boot scripts in the 
> ".agent" scripts and have a central subsystem which triggers all others 
> (via devices of it) at startup and shutdown -- creating a hierarchy this 
> way. This means adding "start" and "stop" events and getting rid of 
> "/etc/hotplug/*.rc" scripts. 

The "rc" scripts were most handy for addressing the "coldplug" case,
supporting subsystem-specific startup/shutdown.  I'm sure you noticed
there was a bunch of USB-specific stuff crying out for better homes.


>    An example is that a PCI device (the host 
> controller) is what "enables" the USB subsystem (or a part of it in the 
> case of multiple controlers). So it goes for all other cases as long as 
> the hotplug scripts are dealing only with hardware devices. This way the 
> subsystems will be "started" and "stopped" in an order which is more 
> appropriate than the current lexical order.

Right.  I forget why I never implemented that; likely it was just a lack
of time to refactor things.  There should certainly be scripts that make the
USB subsystem start up when PCI hotplugs a USB controller.


> My suggestion is also to have driver-specific "*.agent" (instead of just 
> "*" scripts:-)) scripts which will be called _in place_ of  (and not in 
> addition to) modprobe (of course, they will call modprobe themselves if 
> necessary).
> - This makes it possible for scripts to do their stuff before and/or 
> after "modprobe"ing.
> - It also eliminates the need for a blacklist (you (or a package) can 
> simply add empty agents)

If there's no such agent, there needs to be a default behavior.
Most driver developers should never need to write those scripts,
they certainly would view such a requirement as a bug.

And the default behavior for network drivers needs to be a bit
sensitive to what distro it's running on.  It'd be nasty to need to
have N distro's worth of scripts for M drivers = NxM...

But otherwise a "*.agent" that's more flexible seems like a
fine update.


> - It completes the previous hierarchical design which I was talking 
> about. "uhci.agent" for example can "start" the USB subsystem when "added" .

Except that there still needs to be some common place for generic
USB setup, shared by all the host controllers (uhci, ohci, ehci, and
sometimes non-pci ones).  And as a USB developer I do want to
be able to start and stop the USB subsystem by itself...


> - The driver-specific *.agent files can be enumerated very simply (with 
> this glob).
> - The driver-specific policy agents can have their own homes and invite 
> friends and relatives as well... :-) I mean that each driver-specific 
> agent can have its own driver-specific subdirectory (just like each 
> subsystem-specific agent can have its own subsystem-specific subdirectory).

Also good.


> - Aside the previous, each driver (and this is the most useful part) can 
> have its own .usbmap (or .pcimap or similar) in the subsystem-specific 
> directory. This *.usbmap (as an example) file should be parsed as well 
> by usbmodules. This makes usb.handmap, usb.usermap and similar files 
> unnecessary.

I'd say it mostly moves them around, rather than making them become
unnecessary.  Generalizing is good.


> Another thing is that the individual policy agents should be separated 
> from each other (/etc/hotplug/hotplug.functions is ugly :-)) and thus 
> should have very few common parts. That's because future hotplug agents 
> might not be only (or at all) for (un)loading kernel modules for 
> hardware devices and they just can't ignore any common functions (like 
> driver-loading functions or so) that are given to them. Now, that may 
> sound bad since it means duplicating a couple of lines of script but I'm 
> playing safe :-)...

That's something I'll disagree with -- duplicating lines of code is
something I've usually found to be more dangerous than having
one master copy.  Bugs and features need to stay in sync.


> Here's a file layout:
> - Anything matching /etc/hotplug* is hotplug specific
> - Anything matching /etc/hotplug/xxx* is xxx-subsystem-specific.
> - Anything matching /etc/hotplug/xxx/yyy* is yyy-driver-specific.

As it is today -- except /etc/hotplug/ is always a directory, there
are no other /etc/hotplug* files.


> Here's a uniform naming scheme:
> - If * corresponds to nothing then it's a directory.
> - If * corresponds to ".agent" then it's a script that processes hotplug 
> requests.
> - If * corresponds to ".conf" then it's sourced from the corresponding 
> .agent script and related ones.

Sounds reasonable.  Would the "*conf" files be where
distro-specific magic happens?  That's a big issue; it'd
be good to have nice clean places to hang customization
for Debian/RedHat/SuSE/etc and their config policies.

- Dave



_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Unloading drivers, start-up, shut-down and some clean-ups

[Stamatis Mitrofanis]

[-- Attachment #1: Type: text/plain, Size: 8300 bytes --]

Well, firstly, sorry for delaying the discussion a bit -- I'm not in the 
'net all too often :-( . Furthermore, I'm forced to use ******* for the 
Internet because there are only ***modems around.

David Brownell wrote:

>It was waiting for volunteers ... we seem to have a few now!  :)
>Presumably you'll provide a version against the current release?
>
But I _did_ include an attachment (a .diff.gz file) with what I've 
written! You _must_ check it out because it's filled with a lot of 
useful comments. Anyway I include here the same one but in .tar.gz since 
it looks like the patch file is getting larger than the original (and 
besides, it's a rewrite). :-)

>>My idea is to put the functonality of the ".rc" boot scripts in the 
>>".agent" scripts and have a central subsystem which triggers all others 
>>(via devices of it) at startup and shutdown -- creating a hierarchy this 
>>way. This means adding "start" and "stop" events and getting rid of 
>>"/etc/hotplug/*.rc" scripts.
>>
>The "rc" scripts were most handy for addressing the "coldplug" case,
>supporting subsystem-specific startup/shutdown.  I'm sure you noticed
>there was a bunch of USB-specific stuff crying out for better homes.
>
>>   An example is that a PCI device (the host 
>>controller) is what "enables" the USB subsystem (or a part of it in the 
>>case of multiple controlers). So it goes for all other cases as long as 
>>the hotplug scripts are dealing only with hardware devices. This way the 
>>subsystems will be "started" and "stopped" in an order which is more 
>>appropriate than the current lexical order.
>>
>Right.  I forget why I never implemented that; likely it was just a lack
>of time to refactor things.  There should certainly be scripts that make the
>USB subsystem start up when PCI hotplugs a USB controller.
>
Yep. pci.agent calls uhci.agent and then uhci.agent calls usb.agent 
(giving it an extra "USB_BUS" parameter -- see below). But for this, the 
scripts must have a way to associate USB controller devices with USB bus 
numbers (and the same goes for all possible subsystems similar to USB). 
I haven't figured any ways so far.

>>My suggestion is also to have driver-specific "*.agent" (instead of just 
>>"*" scripts:-)) scripts which will be called _in place_ of  (and not in 
>>addition to) modprobe (of course, they will call modprobe themselves if 
>>necessary).
>>- This makes it possible for scripts to do their stuff before and/or 
>>after "modprobe"ing.
>>- It also eliminates the need for a blacklist (you (or a package) can 
>>simply add empty agents)
>>
>If there's no such agent, there needs to be a default behavior.
>Most driver developers should never need to write those scripts,
>they certainly would view such a requirement as a bug.
>
>And the default behavior for network drivers needs to be a bit
>sensitive to what distro it's running on.  It'd be nasty to need to
>have N distro's worth of scripts for M drivers == NxM...
>
>But otherwise a "*.agent" that's more flexible seems like a
>fine update.
>
_Of course_ if a specific "driver agent" can't be found then the module 
should get modprobe'd by the subsystem agent (I should practice 
English writing more :-) ). We consider _the driver agent_ as a special 
case.

>>- It completes the previous hierarchical design which I was talking 
>>about. "uhci.agent" for example can "start" the USB subsystem when "added" .
>>
>Except that there still needs to be some common place for generic
>USB setup, shared by all the host controllers (uhci, ohci, ehci, and
>sometimes non-pci ones).  And as a USB developer I do want to
>be able to start and stop the USB subsystem by itself...
>
Again I didn't mention everything (lack of concentration?). For the 
"start" and "stop" events, the usb.agent accepts a new USB_BUS parameter 
which is simply the name of the USB "bus" whose devices "must become 
available"/"needn't be available anymore" to users. Now, not passing 
that parameter should mean that the action applies to all USB busses. I 
think that should be ok.

>>- The driver-specific *.agent files can be enumerated very simply (with 
>>this glob).
>>- The driver-specific policy agents can have their own homes and invite 
>>friends and relatives as well... :-) I mean that each driver-specific 
>>agent can have its own driver-specific subdirectory (just like each 
>>subsystem-specific agent can have its own subsystem-specific subdirectory).
>>
>Also good.
>
>>- Aside the previous, each driver (and this is the most useful part) can 
>>have its own .usbmap (or .pcimap or similar) in the subsystem-specific 
>>directory. This *.usbmap (as an example) file should be parsed as well 
>>by usbmodules. This makes usb.handmap, usb.usermap and similar files 
>>unnecessary.
>>
>I'd say it mostly moves them around, rather than making them become
>unnecessary.  Generalizing is good.
>
Also, this way you can even have an "input.usbmap" file in the usb 
directory. But then, there's a new "input" agent coming up which makes 
this obsolete.

>>Another thing is that the individual policy agents should be separated 
>>from each other (/etc/hotplug/hotplug.functions is ugly :-)) and thus 
>>should have very few common parts. That's because future hotplug agents 
>>might not be only (or at all) for (un)loading kernel modules for 
>>hardware devices and they just can't ignore any common functions (like 
>>driver-loading functions or so) that are given to them. Now, that may 
>>sound bad since it means duplicating a couple of lines of script but I'm 
>>playing safe :-)...
>>
>That's something I'll disagree with -- duplicating lines of code is
>something I've usually found to be more dangerous than having
>one master copy.  Bugs and features need to stay in sync.
>
I didn't mention that the global ".conf" file is /etc/hotplug.conf . 
It's in the /etc directory and it's just like pci.conf which is in the 
/etc/hotplug directory and comes together with a "pci" subdirectory. Now 
isn't that uniform? :-)

>>Here's a uniform naming scheme:
>>- If * corresponds to nothing then it's a directory.
>>- If * corresponds to ".agent" then it's a script that processes hotplug 
>>requests.
>>- If * corresponds to ".conf" then it's sourced from the corresponding 
>>.agent script and related ones.
>>
>Sounds reasonable.  Would the "*conf" files be where
>distro-specific magic happens?  That's a big issue; it'd
>be good to have nice clean places to hang customization
>for Debian/RedHat/SuSE/etc and their config policies.
>
Sounds like a good idea. I'm afraid though that I never liked having to 
"support" specific distributions. Hey, they're distros: _They_ had 
better support _us_! :-) But, anyways, for PCI and USB there doesn't 
seem to be anything distro-specific to do in the subsystem layer (= from 
the subsystem agent scripts). In the driver agents though there may be 
all sorts of that stuff . But I must note this _very_ important thing: 
We definitely want drivers packaged separately and _not_ for specific 
distros. But, to cover both cases:

If it turns out that "drivers" are packaged distro-independently, then, 
for additional distro-specific files I'd recomend that they are _always_ 
named "*.distro-name.conf" and are sourced from the distro-indifferent 
"*.conf" files (and that's for both subsystem- and driver-agents). But 
to select one of those files, the (packaged separately) driver agent 
must know _simply_ the _name_ of the distro it's running on. A 
_standard_ environment variable should do the trick. But I don't 
remember if such variable is already defined.

If it turns out the monolithic way then individual distros will -- yuck 
-- MODIFY the code however they want just so that they make it more 
<insert distro name here>-friendly but actually making it totally 
incompatible with anything else. Now that's something I've had enough of 
in other parts of the system.

I'd rather see things to be more distro-independent, so I'll choose the 
first solution. In the distro-specific files, a driver or a 
subsystem-agent can put the (abstract) functions whose actual behavior 
is distro-dependent. But, once more I'll vote for distro-independence, 
so that these files are not necessary at all.

Say, did I mention that I want things to be distro-independent? :-)

[-- Attachment #2: hotplug-patch.tar.gz --]
[-- Type: application/x-gzip, Size: 13230 bytes --]

Re: Unloading drivers, start-up, shut-down and some clean-ups

[Harald Barth]

A while ago Stamatis Mitrofanis wrote something about a rewrite of the
hotplug scripts to support start/stop as the *.rc scripts do. As I
think this is the right thing to do (This $REMOVE thingie looks ugly).
I wonder what happened after the mail 10/05/2001 21:13:53.

Did someone consider the changes proposed?
Did someone get to some conclusion?

>Well, firstly, sorry for delaying the discussion a bit -- I'm not in the 
>'net all too often :-( . Furthermore, I'm forced to use ******* for the 
>Internet because there are only ***modems around.

I understand, and my Internet connection suddenly feels much better.

>Anyway I include here the same one but in .tar.gz since it looks like
>the patch file is getting larger than the original (and besides, it's
>a rewrite). :-)

Someone did save that tarball? URL? As I read a lot of lists but not
this one regularely, I'd appreciate to get a personal cc: of your
answer. Thanks folks.

Harald.

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Unloading drivers, start-up, shut-down and some clean-ups

[Stamatis Mitrofanis]

[-- Attachment #1: Type: text/plain, Size: 1834 bytes --]

Harald Barth wrote:

> A while ago Stamatis Mitrofanis wrote something about a rewrite of the
> hotplug scripts to support start/stop as the *.rc scripts do. As I
> think this is the right thing to do (This $REMOVE thingie looks ugly).
> I wonder what happened after the mail 10/05/2001 21:13:53.
> 
> Did someone consider the changes proposed?
> Did someone get to some conclusion?


If nothing else, then somebody at least do the *.agent *.rc script 
unification! (please?)

> Someone did save that tarball? ...

Well, I have... Sorry for not providing a series of patches but I started

rewriting the scripts from nothing (occasionally borrowing some code from

the old scripts, though, but mostly rewriting everything).


I have the newest tarball included as an attachment. Take a look at it. 
It doesn't want to be in just my hard disk any more.

I did all this because the scripts seemed a little ugly to me:
* redundant .rc scripts
* no uniform directory naming (.handmap .distmap .usbmap blacklist...)
* ugly indenting of scripts
* nonextensible, restrictive, RedHatish central hotplug.functions file
* no per-subsystem configuration files or subdirectories (pesky CVS 
subdirs...)
* no per-subsystem-driver map files (for drivers that don't provide the 
information internally)
* USB start event not trigerred by PCI driver-specific script
* didn't work well with a non-highly-modular kernel
* <your favorite inconvenience here>

To see the solutions I suggest, _just take a look at the scripts_. I've 
included a couple of example driver agents too.

Also on hotplug...

I set up the scripts to call wget and use a simple format for queries

to automatically download drivers from some central server. It's quite

a simple design, but flexible. See the scripts for more. I know I have

put plenty of descriptive comments.


[-- Attachment #2: linux-hotplug-2001_12_20.tar.gz --]
[-- Type: application/octet-stream, Size: 24357 bytes --]

Re: Unloading drivers, start-up, shut-down and some clean-ups

[Stamatis Mitrofanis]

I wrote:

> If nothing else, then somebody at least do the *.agent *.rc script 
> unification! (please?)


Um, actually, that 'somebody' is probably David Brownell who might 
kindly include some of my contributions in CVS, perhaps?

I have the impression that we are wasting valuable time.


_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Unloading drivers, start-up, shut-down and some clean-ups

[David Brownell]

The first time you sent things around, they weren't in
the form of digestible/reviewable patches ... just an
entire rewrite.  I asked for a more reviewable form,
but the second iteration looked like the same, and
so does this one.

What that does is magnify the time/cost needed to
look at things, and increase the risk of breaking
things when some unexpected interaction doesn't
get noticed in the review, and didn't get fixed.

Rather than complain that I'm not investing that extra
time, perhaps you would consider spending it yourself
to provide a series of MUCH smaller patches, where
each one is reasonably sized and independently
reviewable???

I don't think that's too much to ask.  There's some
good stuff in there, but it's hard to get at it past the
"gratuitous rewrite" type of changes.

- Dave


> I wrote:
> 
> > If nothing else, then somebody at least do the *.agent *.rc script 
> > unification! (please?)
> 
> 
> Um, actually, that 'somebody' is probably David Brownell who might 
> kindly include some of my contributions in CVS, perhaps?
> 
> I have the impression that we are wasting valuable time.



_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel