Modules and the Interfaces who Love Them (Take I)

Modules and the Interfaces who Love Them (Take I)

[Rusty Russell]

Feedback appreciated.  It's aimed at driver writers.
================
Documentation/module-writing.txt

Writing Modules and the Interfaces To Be Used By Them: A Gentle Guide.
Copyright 2002, Rusty Russell IBM Corportation

Modules are running parts of the kernel which can be added, and
sometimes removed, while the kernel is operational.

There are several delicate issues involved in this procedure which
indicate special care should be taken.

There are three cases you need to be careful:

1) Any code which creates an interface for callbacks (ie. almost any
   function called register_*)
	=> See Rule #1

2) Any modules which use (old) interfaces which do not obey Rule #1
	=> See Rule #2

Rule #1: Module-safe Interfaces.  Any interface which allows
	registration of callbacks, must also allow registration of a
	"struct module *owner", either in the structure or as a
	function parameter, and it must use them to protect the
	callbacks.  See "MAKING INTERFACES SAFE".

Exception #1: As an optimization, you may skip this protection if you
	   *know* that the callbacks are non-preemtible and never
	   sleep (eg. registration of interrupt handlers).


Rule #2: Modules using unsafe interfaces.  If your module is using any
	interface which does not obey rule number 1, that means your
	module functions may be called from the rest of the kernel
	without the caller first doing a successful try_module_get().

	You must not register a "module_cleanup" handler, and your module
	cannot be unloaded except by force.  You must be especially
	careful in this case with initialization: see "INITIALIZING
	MODULES WHICH USE UNSAFE INTERFACES".

MAKING INTERFACES SAFE

A caller must always call "try_module_get()" on a function pointers's
owner before calling through that function pointer.  If
"try_module_get()" returns 0 (false), the function pointer must *not*
be called, and the caller should pretend that registration does not
exist: this means the (module) owner is closing down and doesn't want
any more calls, or in the process of starting up and isn't ready yet.

For many interfaces, this can be optimized by assuming that a
structure containing function pointers has the same owner, and knowing
that one function is always called before the others, such as the
filesystem code which knows a mount must succeed before any other
methods can be accessed.

You must call "module_put()" on the owner sometime after you have
called the function(s).

If you cannot make your interface module-safe in this way, you can at
least split registration into a "reserve" stage and an "activate"
stage, so that modules can use the interface, even if they cannot
(easily) unload.


INITIALIZING MODULES WHICH USE UNSAFE INTERFACES

Safe interfaces will never enter your module before module_init() has
successfully finished, but unsafe interfaces may.  The rule is simple:
your init_module() function *must* succeed (by returning 0) if it has
successfully used any unsafe interfaces.

So, if you are only using ONE unsafe interface, simply use that
interface last.  Otherwise you will have to use printk() to report
failure and leave the module initialized (but possibly useless).


If you have questions about how to apply this document to your own
modules, please ask rusty@rustcorp.com.au or linux-kernel@vger.kernel.org.

Thankyou,
Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: Modules and the Interfaces who Love Them (Take I)

[Roman Zippel]

Hi,

On Wed, 13 Nov 2002, Rusty Russell wrote:

> Feedback appreciated.  It's aimed at driver writers.

If that's your audience, I expect a very confused audience.
You can make it very simple: There are safe interfaces and there are 
broken interfaces and you shall never write or use broken interfaces.
For the majority of driver writers that's good enough.
Any documentation about module writing should also include/point to a 
chapter about resource management. The user has to understand anyway, by 
whom the module resources are used, otherwise he has more problems than 
just module unloading. Module management is just a small part of this 
whole picture or could be part of it, but right now the current module 
code is desperate attempt at keeping it out of it.
I would prefer if the user would be teached about proper resource 
management at kernel level. As soon as the user gets that right he will 
also have no problems to get module unloading right _if_ that would follow 
the same rules, but currently it involves lots of black magic instead.
Rusty, I'm not impressed by the new module code, maybe I'm missing 
something, but it doesn't fix anything and only encourages to write more 
broken interfaces.

bye, Roman

Re: Modules and the Interfaces who Love Them (Take I)

[Rusty Russell]

In message <Pine.LNX.4.44.0211131430190.2109-100000@serv> you write:
> Hi,
> 
> On Wed, 13 Nov 2002, Rusty Russell wrote:
> 
> > Feedback appreciated.  It's aimed at driver writers.
> 
> If that's your audience, I expect a very confused audience.

>From my feedback, I suspect you are right.  I've rewritten it.

> You can make it very simple: There are safe interfaces and there are 
> broken interfaces and you shall never write or use broken interfaces.
> For the majority of driver writers that's good enough.

Yes, maybe I should.

> Any documentation about module writing should also include/point to a 
> chapter about resource management.

Yes, it'd be nice to have good documentation on this.

> As soon as the user gets that right he will also have no problems to
> get module unloading right _if_ that would follow the same rules,
> but currently it involves lots of black magic instead.

Look, your implementation was slow, confusing, invasive, inflexible
*and* buggy.

Slow: Your approach where every interface has to do reference counts
even though they're only useful for modules makes every interface
slow, whether they are using modules or not.  You can't make them
fast, because that would make every interface NR_CPUS *
sizeof(cacheline) larger.

Confusing: Your "deregistration can fail if busy but must succeed
otherwise" is guaranteed to confuse authors who will wonder why.
Currently some fail if it wasn't registered in the first place.  To
understand why this has to change, they have to understand your
interface.  You require a massive style change of not handling failure
within the failing function.  Each module has to implement four (or
was it five?) functions for managing it.

Invasive: Every unregister interface to change, although many are very
happy with try_inc_modcount().  Worse, every module in the kernel has
to change to implement your method.

Inflexible: There is no way your scheme could allow "rmmod --wait" or
"rmmod --force" without adding yet another interface to *every*
module.

Buggy: You did not solve the "module used before init failed" problem,
but left the module loaded and uninitialized in this case.

> Rusty, I'm not impressed by the new module code,

I seriously question your taste in this matter.  You obviously hold a
personal dislike for my code.   Fine.

> maybe I'm missing something, but it doesn't fix anything

Then you don't understand the problem.

Sorry, but I'm wasting my time sending you mail. 8(
Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: Modules and the Interfaces who Love Them (Take I)

[Jeff Garzik]

Rusty Russell wrote:

> Slow: Your approach where every interface has to do reference counts
> even though they're only useful for modules makes every interface
> slow, whether they are using modules or not.  You can't make them
> fast, because that would make every interface NR_CPUS *
> sizeof(cacheline) larger.


(tangent alert)

Objects controlled by interfaces typically should be ref-counted... 
life in the kernel is slowly getting better WRT this, but we're not 
there yet.  In any case, the ref counts help, not hurt.

	Jeff

Re: Modules and the Interfaces who Love Them (Take I)

[Roman Zippel]

Hi,

On Thu, 14 Nov 2002, Rusty Russell wrote:

> Look, your implementation was slow, confusing, invasive, inflexible
> *and* buggy.

My patch was meant as demonstration, you reduce it to a single aspect - 
the module - driver interface.
The patch was intended to demonstrate more. First of all, how to fix the 
module mess without breaking everything. It demonstrated a way to 
introduce a new interface without breaking compability. The new driver 
interface on top of it was optional, it could also have been the old 
interface or your monster refs.

> I seriously question your taste in this matter.  You obviously hold a
> personal dislike for my code.   Fine.

Doing to the linking in the kernel is just plain wrong. Most of the module 
code could perfectly live in user space and it could be as simple as your 
kernel loader.

> > maybe I'm missing something, but it doesn't fix anything
> 
> Then you don't understand the problem.

Maybe you could explain, what problems it fixes, that justifies complete 
breakage?

bye, Roman

Re: Modules and the Interfaces who Love Them (Take I)

[Roman Zippel]

Hi,

On Thu, 14 Nov 2002, Rusty Russell wrote:

> I seriously question your taste in this matter.

BTW while talking about taste: I'd rather make certain modules not 
unloadable, than have code like stop_refcounts() in the kernel.
Module loading _and_ unloading could be so simple and fast, but it seems 
people prefer to keep the module stuff black magic. If nobody else cares I 
won't care either, alsa was the last reason I needed modules, now I can 
live without it.

bye, Roman