Discourage Uniform Driver Model?

Discourage Uniform Driver Model?

[James Clark]

If it was possible to create a driver model that allowed module compatibility 
across different releases/revisions without recompilation and with minimal 
performance hit would this be desirable? 

If such a model was possible should development be avoided to discourage 
binary only modules?

James

Re: Discourage Uniform Driver Model?

[Dr. David Alan Gilbert]

* James Clark (jimwclark@ntlworld.com) wrote:
> If it was possible to create a driver model that allowed module compatibility 
> across different releases/revisions without recompilation and with minimal 
> performance hit would this be desirable? 
> 
> If such a model was possible should development be avoided to discourage 
> binary only modules?

I would argue that is exactly the situation seen by XFree with quite a
few cards requiring binary modules for full functionality.

Dave
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    | Running GNU/Linux on Alpha,68K| Happy  \ 
\ gro.gilbert @ treblig.org | MIPS,x86,ARM,SPARC,PPC & HPPA | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/

Discourage Uniform Driver Model?

[James Clark]

1. If it was possible to create a driver model that allowed module
compatibility across different releases/revisions without recompilation and
with minimal performance hit would this be desirable?

2. If such a model was possible should development be avoided to discourage
binary only modules?

James

Re: Discourage Uniform Driver Model?

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 397 bytes --]

On Sun, 14 Sep 2003 21:38:32 BST, James Clark <jimwclark@ntlworld.com>  said:
> 1. If it was possible to create a driver model that allowed module
> compatibility across different releases/revisions without recompilation and
> with minimal performance hit would this be desirable?
> 
> 2. If such a model was possible should development be avoided to discourage
> binary only modules?

-ENOTAGAIN

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Discourage Uniform Driver Model?

[Alan Cox]

On Sul, 2003-09-14 at 21:38, James Clark wrote:
> 1. If it was possible to create a driver model that allowed module
> compatibility across different releases/revisions without recompilation and
> with minimal performance hit would this be desirable?

We've spent years trying to reduce the number of binary differences.
Since it involves the most basic blocks of code, choice of instructions
it isnt going to be avoidable.

Re: Discourage Uniform Driver Model

[James Clark]

> How does your proposal differ/improve on the demand-load module system?

Modules must be (re)compiled for each specific kernel release. This is because 
of the very tight coupling with the kernel. The kernel changes so much that 
it is often necessary to amend a module to keep up. My suggestion is to 
create an abstract interface between the two. The interface (called a binary 
interface) would define the way the two 'talk' to one another. Once the 
interface has been released as v1.0 any module could be made compatible with 
the v1.0 interface. After a while it would be necessary to release improved 
interfaces. This would not break the existing modules as the old interface 
could remain due to its abstract nature. This technique is used in everything 
from Windows drivers, Netscape pluggins and even normal programs - image 
saying you must buy a new version of Word Perfect for every minor release of 
the OS - it has 'binary' compatibility. Unfortunately many see this as active 
encouragement of 'binary only' modules which have no source code. This idea 
would cost in performance. I suggest this loss is worthwhile to resolve 
compatibility and usability.

>   This seemed to work well in Mandrake, or am I missing the point?!?

Unless you build the module yourself you are the mercy of Mandrake to make 
each module - I don't see this as being much different from the situation 
with Windows.  Poor old Mandrake must remake almost every module almost 
everytime. You cannot use a module from Redhat's kernel. In effect everyone 
is duplicating effort. Modules must currently be compiled, this requires a 
degree of expertise and can be daunting to someone who just wants to use the 
OS. My suggestion would not remove need to release modules in source form, 
but once compiled they could be plugged into any kernel and distributed in 
binary+source form. 

> Also, the only reply to your posting that I read (and I've only read a
> fraction of them!) which offered any genuine usefulness was the point
> that if you offer the code then they'll consider the merits.  Are you
> considering an actual solution, or just suggesting a policy change?

I understand the way competitor binary interfaces work. Currently I'm not 
going to roll up my sleeves and write this system. I don't have the technical 
expertise to design such a thing and although I could learn the curve at this 
time is too steep. I do think that my experience with similar competitor 
systems allows me to speak on the subject. However, I feel that pushing for 
this change is a positive thing. It has started a debate on 'misusing' the 
GPL to prevent binary only modules and has resulted in some positive 
comments. If they debate it  rationally and then decide not to bother I will 
have achieved a lot. I do  feel qualified to make this (obvious) suggestion 
and comment on the design of any resulting interface.

James

Re: Discourage Uniform Driver Model

[Tim Hockin]

On Tue, Sep 16, 2003 at 12:13:38AM +0100, James Clark wrote:
> I understand the way competitor binary interfaces work. Currently I'm not 
> going to roll up my sleeves and write this system. I don't have the technical 
> expertise to design such a thing and although I could learn the curve at this 
> time is too steep. I do think that my experience with similar competitor 
> systems allows me to speak on the subject. However, I feel that pushing for 
> this change is a positive thing. It has started a debate on 'misusing' the 
> GPL to prevent binary only modules and has resulted in some positive 
> comments. If they debate it  rationally and then decide not to bother I will 
> have achieved a lot. I do  feel qualified to make this (obvious) suggestion 
> and comment on the design of any resulting interface.

I and others have repeatedly pointed out the big failings in this, which you
have ignored.

To do this means you must freeze the size of any structure passed across the
kernel-module boundary.  You must freeze any functions which are in-lined.

spin_lock() must not have debugging code which is enabled by a CONFIG_
option, or it must become a function call.  You can never optimize away
stuff that only matters on SMP systems.  You can never have a structure
include members conditionally upon CONFIG_ options, or you have to provide
accessor functions for every field of every struct.

This is what you are not getting.  Because module code is a first-class
citizen in Linux, you CAN'T decouple them, without imposing BIG UGLY
restrictions on all modules.  Do you understand the examples I am giving
you?  I can be more explicit, if you like.

Now, if you, or someone can devise a cleanish way to do it, SHOW US CODE.
It doesn't have to work fully, just enough to convince us it could work AT
ALL.

I assert that any solution will have one or more of the following
attributes, any of which are unacceptable:

* will perform terribly
* will be cumbersome to code
* will impose new restrictions on kernel developers
* will be unmaintainable


I BEG YOU to prove me wrong.

-- 
Notice that as computers are becoming easier and easier to use,
suddenly there's a big market for "Dummies" books.  Cause and effect,
or merely an ironic juxtaposition of unrelated facts?

Re: Discourage Uniform Driver Model

[Timothy Miller]

James Clark wrote:
>>How does your proposal differ/improve on the demand-load module system?
> 
> 
> Modules must be (re)compiled for each specific kernel release. This is because 
> of the very tight coupling with the kernel. The kernel changes so much that 
> it is often necessary to amend a module to keep up. My suggestion is to 
> create an abstract interface between the two. The interface (called a binary 
> interface) would define the way the two 'talk' to one another. Once the 
> interface has been released as v1.0 any module could be made compatible with 
> the v1.0 interface. After a while it would be necessary to release improved 
> interfaces. This would not break the existing modules as the old interface 
> could remain due to its abstract nature. This technique is used in everything 
> from Windows drivers, Netscape pluggins and even normal programs - image 
> saying you must buy a new version of Word Perfect for every minor release of 
> the OS - it has 'binary' compatibility. Unfortunately many see this as active 
> encouragement of 'binary only' modules which have no source code. This idea 
> would cost in performance. I suggest this loss is worthwhile to resolve 
> compatibility and usability.

I have been flamed before for "suggesting" things without code to back 
it up.  There have been lots of technical explanations given for why a 
consistent binary-only interface is impractical.  However, the idea has 
been suggested before, and some implementations have been done.  You 
should look into those.

This "suggestion" is sufficiently uninteresting to everyone that your 
suggestion will not be taken.  If you want something like this 
implemented, you're going to have to do it yourself.  That's the beauty 
of Free Software.

> 
>>  This seemed to work well in Mandrake, or am I missing the point?!?
> 
> 
> Unless you build the module yourself you are the mercy of Mandrake to make 
> each module - I don't see this as being much different from the situation 
> with Windows.  Poor old Mandrake must remake almost every module almost 
> everytime. You cannot use a module from Redhat's kernel. In effect everyone 
> is duplicating effort. Modules must currently be compiled, this requires a 
> degree of expertise and can be daunting to someone who just wants to use the 
> OS. My suggestion would not remove need to release modules in source form, 
> but once compiled they could be plugged into any kernel and distributed in 
> binary+source form. 

If everyone used the vger kernel, they wouldn't have some of these 
problems.  However, due to the way Linux strives to remain lean and 
efficient, it is technically unwise to standardize any driver interface. 
  That is the nature of the beast.  If you want to write an interposing 
layer, please feel free to do so.

> 
>>Also, the only reply to your posting that I read (and I've only read a
>>fraction of them!) which offered any genuine usefulness was the point
>>that if you offer the code then they'll consider the merits.  Are you
>>considering an actual solution, or just suggesting a policy change?
> 
> 
> I understand the way competitor binary interfaces work. Currently I'm not 
> going to roll up my sleeves and write this system. I don't have the technical 
> expertise to design such a thing and although I could learn the curve at this 
> time is too steep. I do think that my experience with similar competitor 
> systems allows me to speak on the subject. However, I feel that pushing for 
> this change is a positive thing. It has started a debate on 'misusing' the 
> GPL to prevent binary only modules and has resulted in some positive 
> comments. If they debate it  rationally and then decide not to bother I will 
> have achieved a lot. I do  feel qualified to make this (obvious) suggestion 
> and comment on the design of any resulting interface.

Yor suggestion has been listened to.  However, it has become clear that 
kernel maintainers actively dislike this idea.  Pushing it is only going 
to piss them off.