Re: Unloading drivers, start-up, shut-down and a rewrite (a problem)

Re: Unloading drivers, start-up, shut-down and a rewrite (a problem)

[Stamatis Mitrofanis]

Keith Owens wrote:

>On Tue, 02 Oct 2001 03:00:00 +0100, 
>stamit <ewstam@softhome.net> wrote:
>
>>Now, the final thing that remains for unloading to be complete is 
>>reference counting. I tried to do it with bash (by storing counts in 
>>*.ref files in the driver directories) and guess what I stumbled to... a 
>>race condition! Multiple hotplug events can be processed by the scripts 
>>at the same time. So I took it back. Isn't there a way to use modutils 
>>programs to play with the module's real refrence counts? It would make 
>>things a whole lot easier (and safer).
>>
>
>Without knowing exactly what you are trying to do I am guessing but it
>sounds like you need this:
>
>* kernel/module.c registers /proc/sys/kernel/mod_use_count, mode 600.
>* module.c creates a dummy module entry called mod_use_count the first
>  time that /proc/sys/kernel/mod_use_count is written to.
>* /proc/sys/kernel/mod_use_count recognises commands like
>    up module-name
>    down module-name
>  and adjusts the use counts and reference chain accordingly.  Hotplug
>  scripts use mod_use_count to bump the reference count before allowing
>  a start event to continue and after completing a stop event.
>
>If the module does not exist (already unloaded) then write fails with
>some return code.  There is no specific code for "module does not
>exist" so pick something similar, like ESRCH or ENXIO.
>
I'm trying to implement module unloading in the hotplug scripts. Aside 
that, I'm also doing a rewrite, a slight redesign and "beautification" 
of the scripts generally because thay lacked such things as readability 
and uniformity. I'm trying to make them beautiful because, hey, vendors 
might choose to release drivers with scripts for our hotplug project!

So I'm making them pretty so they'll be attractive. :-)

Of course, because, besides making scripts and framework pretty, I'm 
also implementing the ability to unload modules, I needed to have a 
method to play with the modules' reference counts(which is the only 
thing still missing from my rewrite). And since you pointed me to the 
right way, I'll try to add it and post a patch when it's ready. I didn't 
really know about that because I'm not very into the kernel's internals. 
I'm doing mostly user-space stuff so far (thus my interest for the scripts).



_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Unloading drivers, start-up, shut-down and a rewrite (a problem)

[Keith Owens]

On Thu, 04 Oct 2001 04:14:08 +0100, 
Stamatis Mitrofanis <ewstam@softhome.net> wrote:
>Of course, because, besides making scripts and framework pretty, I'm 
>also implementing the ability to unload modules, I needed to have a 
>method to play with the modules' reference counts(which is the only 
>thing still missing from my rewrite).

More details please.  Why do you need to externally adjust a module's
use count?  All other modules handle their use counts internally, what
is different about hotplug?

>And since you pointed me to the 
>right way, I'll try to add it and post a patch when it's ready. I didn't 
>really know about that because I'm not very into the kernel's internals. 

I will do the patch to kernel/module.c, once I understand exactly what
the requirements are.


_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Unloading drivers, start-up, shut-down and a rewrite (a problem)

[Stamatis Mitrofanis]

>
>
>More details please.  Why do you need to externally adjust a module's
>use count?  All other modules handle their use counts internally, what
>is different about hotplug?
>
>I will do the patch to kernel/module.c, once I understand exactly what
>the requirements are.
>
It's when you have many hotpluggable devices of the same type (handled 
by the same driver). Managing a reference (device) count for each driver 
from the hotpug scripts is necessary for unloading the driver _only_ 
after the last such device is unplugged.

It might be a bad idea to play with the modules' real reference counts 
though, but there must be at least some form of mutual exclusion for 
accessing this driver-specific device count. Therefore, it was wrong for 
me to try to implement this device count by making the scripts 
read/write"*.ref" files in the hotplug directory (possible race 
condition which would mess up the count).

If it should be implemented in the kernel then the functionality should 
be exposed through modprobe. I'm sorry, but I have no idea about the 
actual details of (un)loading modules so you'll have to do everything 
behind modprobe.

The requirement is just that modprobe gets a new command-line paremeter 
which, when given in a call to load/unload a module, will also 
increment/decrement the module's reference count. I.e. if  the parameter 
is "--ref" and the module ABC is not loaded, then the following should 
result in module ABC being in memory with a reference count of 1:
# modprobe --ref ABC
# modprobe --ref ABC
# modprobe --ref -r ABC
The first command will load the module and set its reference count to 1. 
The second command will increment the ABC's reference count to 2 and the 
third should decrement the reference count to 1 without, of course, 
unloading the module.



_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Unloading drivers, start-up, shut-down and a rewrite (a problem)

[Keith Owens]

On Fri, 05 Oct 2001 01:03:19 +0100, 
Stamatis Mitrofanis <ewstam@softhome.net> wrote:
>It's when you have many hotpluggable devices of the same type (handled 
>by the same driver). Managing a reference (device) count for each driver 
>from the hotpug scripts is necessary for unloading the driver _only_ 
>after the last such device is unplugged.

That means we are changing the meaning of the module use count on
drivers from "number of devices opened" to "number of devices present
and activated".  Not such a bad idea, there has long been a problem
with network drivers that the use count can be zero but the driver code
can be active, rmmod gets very nasty then.

Before I write the code, what needs to change in user space to match
the new semantics?  Every device activation and inactivation must
adjust the use count of the supporting module.  That is easy for
hotplug but what about PCMCIA and fixed devices?  In particular
ifconfig eth0 up must adjust the use count of the module controlling
eth0.  A partial solution that only works for hotplug is not a good
idea.  Can we get a consistent user space framework for the new
semantics?


_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Unloading drivers, start-up, shut-down and a rewrite (a problem)

[David Hinds]

On Fri, Oct 05, 2001 at 03:58:58PM +1000, Keith Owens wrote:
> 
> Before I write the code, what needs to change in user space to match
> the new semantics?  Every device activation and inactivation must
> adjust the use count of the supporting module.  That is easy for
> hotplug but what about PCMCIA and fixed devices?  In particular
> ifconfig eth0 up must adjust the use count of the module controlling
> eth0.  A partial solution that only works for hotplug is not a good
> idea.  Can we get a consistent user space framework for the new
> semantics?

Just so you know, and not saying whether it is a better or worse
solution, the way PCMCIA drivers handle this, is that there's a
separate interface, /proc/bus/pccard/drivers, that has "use counts"
for PCMCIA drivers that indicate how many physical devices are
currently associated with each driver.  These use counts are
independent of the regular module use counts.

-- Dave

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Unloading drivers, start-up, shut-down and a rewrite (a problem)

[David Brownell]

Quoting David Hinds:
> Just so you know, and not saying whether it is a better or worse
> solution, the way PCMCIA drivers handle this, is that there's a
> separate interface, /proc/bus/pccard/drivers, that has "use counts"
> for PCMCIA drivers that indicate how many physical devices are
> currently associated with each driver.  These use counts are
> independent of the regular module use counts.

And FWIW, when I've thought of how to solve this "driver unload"
problem in a clean way that improves user-friendliness, that's just
what I end up wanting:  a devices-per-driver count that's distinct
from the existing uses-per-driver refcount.

It's late, so don't hold me to this, but I think adding that is probably
sufficient for user-mode policy decisions.  Hotplugging would by
default consider the device-per-driver count when seeing if it's OK
to unload a driver module.

Likely "rmmod" needs a flag that says whether the device count is
ignored, as (effectively) done today. Driver developers will want the
flexibility to say "remove this driver even though there's hardware
attached, I've got a bugfix".  USB developers work in that mode
today, it's quite handy.

- Dave



_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Unloading drivers, start-up, shut-down and a rewrite (a problem)

[David Brownell]

> The requirement is just that modprobe gets a new command-line paremeter 
> which, when given in a call to load/unload a module, will also 
> increment/decrement the module's reference count. I.e. if  the parameter 
> is "--ref" and the module ABC is not loaded, then the following should 
> result in module ABC being in memory with a reference count of 1:

Modulo the issue about making a new "devices per driver" counter,
I'd also like to raise the issue of whether this counter shouldn't be
driven entirely by the kernel code binding drivers to devices.

That is, code the USB or PCI layers would increment/decrement
the counter when the binding was successfully achieved.  No
changes to the device drivers needed, but having the kernel
maintain those counters would be safer than doing it in user mode.

- Dave



_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Unloading drivers, start-up, shut-down and a rewrite (a problem)

[Keith Owens]

On Fri, 05 Oct 2001 00:13:15 -0700, 
David Brownell <david-b@pacbell.net> wrote:
>Quoting David Hinds:
>> Just so you know, and not saying whether it is a better or worse
>> solution, the way PCMCIA drivers handle this, is that there's a
>> separate interface, /proc/bus/pccard/drivers, that has "use counts"
>> for PCMCIA drivers that indicate how many physical devices are
>> currently associated with each driver.  These use counts are
>> independent of the regular module use counts.
>
>And FWIW, when I've thought of how to solve this "driver unload"
>problem in a clean way that improves user-friendliness, that's just
>what I end up wanting:  a devices-per-driver count that's distinct
>from the existing uses-per-driver refcount.

I am not convinced that we need a separate count, at least not at the
bottom layer.  The module use count is a pure reference count on the
module code, rmmod does not care what the types of reference are, rmmod
just cares if it is safe to unload the code.

>Likely "rmmod" needs a flag that says whether the device count is
>ignored, as (effectively) done today. Driver developers will want the
>flexibility to say "remove this driver even though there's hardware
>attached, I've got a bugfix".  USB developers work in that mode
>today, it's quite handy.

See above, I don't see any need to distinguish between references.  If
the device is up then the module cannot be unloaded.  That does assume
that all devices can be deactivated which is not currently true, to do
this properly every device driver would need an up and a down routine.

I can add a use count up/down interface to kernel/module.c without
waiting for every driver to be converted to an up/down interface.
However we have to agree on a common design for drivers, I would hate
to see some modules supporting up/down and others doing it a different
way.


_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Unloading drivers, start-up, shut-down and a rewrite (a problem)

[Oliver Neukum]

Am Freitag, 5. Oktober 2001 09:28 schrieb Keith Owens:
> On Fri, 05 Oct 2001 00:13:15 -0700,
>
> David Brownell <david-b@pacbell.net> wrote:

> >And FWIW, when I've thought of how to solve this "driver unload"
> >problem in a clean way that improves user-friendliness, that's just
> >what I end up wanting:  a devices-per-driver count that's distinct
> >from the existing uses-per-driver refcount.
>
> I am not convinced that we need a separate count, at least not at the
> bottom layer.  The module use count is a pure reference count on the
> module code, rmmod does not care what the types of reference are, rmmod
> just cares if it is safe to unload the code.

Yes. In last consequence only the kernel cares about reference counts.
Yet it is not a pure reference counter. It's possible for a module, though 
possibly not a good idea to bump the counter while eg running a timer.
In fact you need to bump it while handling probe() and disconnect().

> >Likely "rmmod" needs a flag that says whether the device count is
> >ignored, as (effectively) done today. Driver developers will want the
> >flexibility to say "remove this driver even though there's hardware
> >attached, I've got a bugfix".  USB developers work in that mode
> >today, it's quite handy.
>
> See above, I don't see any need to distinguish between references.  If
> the device is up then the module cannot be unloaded.  That does assume
> that all devices can be deactivated which is not currently true, to do
> this properly every device driver would need an up and a down routine.

The problem is not whether the driver can be unloaded without endangering
system integrity. It's about whether unloading a driver breaks the desired 
function. Else a perfectly usable device becomes driverless all of a sudden.
With the current design in unloading you need a seperate counter.

> I can add a use count up/down interface to kernel/module.c without
> waiting for every driver to be converted to an up/down interface.
> However we have to agree on a common design for drivers, I would hate
> to see some modules supporting up/down and others doing it a different
> way.

That would be helpful. And doing this in kernel makes handling errors in 
binding drivers to devices trivial.
For USB the core should handle it evaluating the return value of probe().
That needs a module* in the usb_device struct. On the other hand the 
conventional usage counters would be simplified by this.

	Regards
		Oliver

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Unloading drivers, start-up, shut-down and a rewrite (a problem)

[David Brownell]

Hi Keith,

> I am not convinced that we need a separate count, at least not at the
> bottom layer.  The module use count is a pure reference count on the
> module code, rmmod does not care what the types of reference are, rmmod
> just cares if it is safe to unload the code.

While "rmmod" may not care, there are two distinct user communities
that IMO need to be supported:

- "real users", 95+% of the computer users in the world, who want
  things to "just work" and never need root access or a sysadmin;

- "sysadmin" or "developer" ... who does troubleshooting/upgrading/...
  and so needs extra system capabilities.

I think "one use count" might work for a "real user", but not for anyone
who wants safe ways to replace drivers on running systems.  If there's
just one use count, adjusted in userland, it'd be too easy for userland
bugs to corrupt things.  (What do you mean by "need"? :)


> >Likely "rmmod" needs a flag that says whether the device count is
> >ignored, as (effectively) done today. Driver developers will want the
> >flexibility to say "remove this driver even though there's hardware
> >attached, I've got a bugfix".  USB developers work in that mode
> >today, it's quite handy.
> 
> See above, I don't see any need to distinguish between references.

How would someone SSH into a system and upgrade some of its
drivers?  With two counts, one can shutdown all code using the
drivers (use count goes to zero), then say "ignore any hardware,
just remove the driver".  And then start things up again; perfectly
safe, even a sleep-deprived and seriously aggravated sysadmin
can't make mistakes (run some command too many times :) and
thereby crash the server (consequences left to your imagination).


>  If the device is up then the module cannot be unloaded.  That does assume
> that all devices can be deactivated which is not currently true, to do
> this properly every device driver would need an up and a down routine.

If "up" is a software notion, that's what I see the current use count as
being used for.  (I can't see it being a hardware notion...)  But I'm not
sure what you're saying "is not currently true".

Hotpluggable drivers get told when the hardware comes and goes,
by USB or PCI etc.  With USB, that doesn't modify any use counter,
ensuring that drivers can safely be reloaded so long as they're not
actually in use.  If the hardware is accessed through the file system
device nodes, open/release routines apply; network drivers have up/down
analogues.  Those all (should) update the current use count, keeping
the driver from being unloaded until the device becomes inactive.


> I can add a use count up/down interface to kernel/module.c without
> waiting for every driver to be converted to an up/down interface.

The same with MOD_{INC,DEC}_DEV_COUNT() support;
it's not drivers that'd need converting, but subsystems.  Two lines
(each) in the right places of usb.c, pci.c (or wherever).
Changing every single driver would be wrong, I'd agree.


> However we have to agree on a common design for drivers, I would hate
> to see some modules supporting up/down and others doing it a different
> way.

We have USB pretty well in hand, as I described above.  It could again
serve as a guinea pig ... :)

- Dave



_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Unloading drivers, start-up, shut-down and a rewrite (a problem)

[David Hinds]

On Fri, Oct 05, 2001 at 12:21:06AM -0700, David Brownell wrote:
> 
> That is, code the USB or PCI layers would increment/decrement
> the counter when the binding was successfully achieved.  No
> changes to the device drivers needed, but having the kernel
> maintain those counters would be safer than doing it in user mode.

This is what PCMCIA does.

-- Dave

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel