[kernel-abi] symbol versioning vs. out-of-tree modules.

[kernel-abi] symbol versioning vs. out-of-tree modules.

[Paweł Sikora]

Hi All,

with out-of-tree kernel modules we (users and linux distributions)
have a problem - with every frequent kernel release we need
to rebuild them all due to vermagic and possible abi changes.
this leads to lot of useless work (bumping rpm specs releases,
preparing rpm packages and so on).

naturally we could workaround the modprobe vermagic errors
but we currently can't detect the big-bang kernel abi changes.
e.g., some kernel function changes its parameter list while
the exported function(symbol) name still staying unchanged.
finally we get a modprobe-big-bang.

in fact, we can't use a c++ symbol name mangling to avoid
such problems but kernel build system could use a linker script
to versioning symbols (like glibc/libgcc). during the abi change,
kernel developers could just bump the symbol version and
external modules could be refused during loading with
'unresolved symbol' error which is imho much better solution
that an oops.

is it acceptable and possible to introducing such solution?
thanks for comments.

BR,
pawel.

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Christoph Hellwig]

On Wed, Sep 24, 2008 at 05:07:59PM +0200, Pawe? Sikora wrote:
> Hi All,
> 
> with out-of-tree kernel modules we (users and linux distributions)

"we" aka the idiots that use out of tree modules on production systems.
Just stop doing that, and your problem is solved.

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Tilman Schmidt]

[-- Attachment #1: Type: text/plain, Size: 814 bytes --]

Christoph Hellwig schrieb:
> On Wed, Sep 24, 2008 at 05:07:59PM +0200, Pawe? Sikora wrote:
>> Hi All,
>> 
>> with out-of-tree kernel modules we (users and linux distributions)
> 
> "we" aka the idiots that use out of tree modules on production systems.

Tsk. Tsk. Insults are no substitute for arguments.

> Just stop doing that, and your problem is solved.

s/solved/transformed into an even harder one/

Before throwing out something used on production systems you
have to have a working replacement solution. Yes, I know, people
are working on that, but "being worked on" != "working".

HTH
T.

-- 
Tilman Schmidt                    E-Mail: tilman@imap.cc
Bonn, Germany
Diese Nachricht besteht zu 100% aus wiederverwerteten Bits.
Ungeöffnet mindestens haltbar bis: (siehe Rückseite)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Alexey Dobriyan]

On Wed, Sep 24, 2008 at 05:07:59PM +0200, Paweł Sikora wrote:
> with out-of-tree kernel modules we (users and linux distributions)
> have a problem - with every frequent kernel release we need
> to rebuild them all due to vermagic and possible abi changes.
> this leads to lot of useless work (bumping rpm specs releases,
> preparing rpm packages and so on).

That's your price for using out-of-tree junk.

> naturally we could workaround the modprobe vermagic errors
> but we currently can't detect the big-bang kernel abi changes.

And never will be able to.

> e.g., some kernel function changes its parameter list while
> the exported function(symbol) name still staying unchanged.
> finally we get a modprobe-big-bang.

*bang*

> in fact, we can't use a c++ symbol name mangling to avoid
> such problems but kernel build system could use a linker script
> to versioning symbols (like glibc/libgcc). during the abi change,
> kernel developers could just bump the symbol version and
> external modules could be refused during loading with
> 'unresolved symbol' error which is imho much better solution
> that an oops.

So kernel developers should waste time thinking if some change will
result in some innumerable junk being broken, resulting in symbol
version bump?

> is it acceptable and possible to introducing such solution?

No. Don't even bother.

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Arjan van de Ven]

On Wed, 24 Sep 2008 17:07:59 +0200
"Paweł Sikora" <pluto@agmk.net> wrote:

> Hi All,
> 
> with out-of-tree kernel modules we (users and linux distributions)
> have a problem - with every frequent kernel release we need
> to rebuild them all due to vermagic and possible abi changes.
> this leads to lot of useless work (bumping rpm specs releases,
> preparing rpm packages and so on).
> 
> naturally we could workaround the modprobe vermagic errors
> but we currently can't detect the big-bang kernel abi changes.
> e.g., some kernel function changes its parameter list while
> the exported function(symbol) name still staying unchanged.
> finally we get a modprobe-big-bang.
> 
> in fact, we can't use a c++ symbol name mangling to avoid
> such problems but kernel build system could use a linker script
> to versioning symbols (like glibc/libgcc). during the abi change,
> kernel developers could just bump the symbol version and
> external modules could be refused during loading with
> 'unresolved symbol' error which is imho much better solutio


if it were up to me we would remove modversions from the kernel and
just hard require an exact build of the module to the kernel; anything
else is .. at best an educated wild guess.

-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Greg KH]

On Wed, Sep 24, 2008 at 05:07:59PM +0200, Paweł Sikora wrote:
> Hi All,
> 
> with out-of-tree kernel modules we (users and linux distributions)
> have a problem - with every frequent kernel release we need
> to rebuild them all due to vermagic and possible abi changes.
> this leads to lot of useless work (bumping rpm specs releases,
> preparing rpm packages and so on).

"we" are working hard to remove any and all barriers for modules to be
out-of-tree.  See the recent drivers/staging/ announcement, I can now
take pretty much any code that at least builds and works for someone,
and get it into the kernel tree where it can be properly cleaned up and
cared for over time before it moves to the "correct" location within the
kernel.

What kernel modules do you currently use/rely on that are not in the
main kernel tree?  Do you have links to them?  I'd be glad to add them.

thanks,

greg k-h

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Paweł Sikora]

On Thursday 25 of September 2008 04:48:40 Greg KH wrote:
> On Wed, Sep 24, 2008 at 05:07:59PM +0200, Paweł Sikora wrote:
> > Hi All,
> >
> > with out-of-tree kernel modules we (users and linux distributions)
> > have a problem - with every frequent kernel release we need
> > to rebuild them all due to vermagic and possible abi changes.
> > this leads to lot of useless work (bumping rpm specs releases,
> > preparing rpm packages and so on).
>
> "we" are working hard to remove any and all barriers for modules to be
> out-of-tree.  See the recent drivers/staging/ announcement, I can now
> take pretty much any code that at least builds and works for someone,
> and get it into the kernel tree where it can be properly cleaned up and
> cared for over time before it moves to the "correct" location within the
> kernel.
>
> What kernel modules do you currently use/rely on that are not in the
> main kernel tree?  Do you have links to them?  I'd be glad to add them.

here's a quick list:

- http://www.lirc.org
- http://sourceforge.net/projects/bluetooth-alsa
- http://rt2x00.serialmonkey.com
- http://www.unav-micro.com/Drivers.aspx -> AR81Family-linux-v1.0.1.0.tar.gz
  (e.g. asus-p5q-pro m/b has such ethernet port, pciid=1969:1026)
- http://aufs.sourceforge.net
- http://linux-uvc.berlios.de
- http://www.misdn.org
- toshiba bluetooth module [http://0bits.com/toshbt/toshbt-1.0.tar.gz]
- zaptel telephony driver [http://www.asterisk.org]
  with firmware [http://downloads.digium.com/pub/telephony/firmware/releases]

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Sitsofe Wheeler]

Paweł Sikora wrote:
> On Thursday 25 of September 2008 04:48:40 Greg KH wrote:
>> What kernel modules do you currently use/rely on that are not in the
>> main kernel tree?  Do you have links to them?  I'd be glad to add them.
> 
> here's a quick list:
> 
> - http://www.lirc.org

^^^^ People are taking steps to adapt this to be included in kernel. See 
  http://lwn.net/Articles/297770/ . I could have sworn I saw someone 
posting an input layer patch for lirc the other day too... Aha - 
http://lkml.org/lkml/2008/9/24/328 .

> - http://rt2x00.serialmonkey.com

Isn't this one already in the wireless tree? I believe it was merged in 
2.6.24 - 
http://kernelnewbies.org/Linux_2_6_24#head-62e9ebf067c978bbf70898986c0aa3904d1a3543 
.

> - http://aufs.sourceforge.net

I've read rumblings about one of the stacking FSes being merged but thus 
far nothing seems to have happened - http://lwn.net/Articles/265240/ .

> - http://linux-uvc.berlios.de

USB video has already been merged into mainline in 2.6.25 - 
http://kernelnewbies.org/Linux_2_6_26#head-6d4ee1ddea983f5c8ac491b8685a94ea6fa4132b 
! I'm using it on my eeepc...

Those are the ones I recognise off the top of my head. I'll have to 
search with regard to the other ones you mentioned...

-- 
Sitsofe | http://sucs.org/~sits/

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Sitsofe Wheeler]

Sitsofe Wheeler wrote:
> Paweł Sikora wrote:
>> On Thursday 25 of September 2008 04:48:40 Greg KH wrote:
>>> What kernel modules do you currently use/rely on that are not in the
>>> main kernel tree?  Do you have links to them?  I'd be glad to add them.
>>
>> here's a quick list:
>>
>> - http://www.lirc.org
> 
> ^^^^ People are taking steps to adapt this to be included in kernel. See 
>  http://lwn.net/Articles/297770/ . I could have sworn I saw someone 
> posting an input layer patch for lirc the other day too... Aha - 
> http://lkml.org/lkml/2008/9/24/328 .
> 
>> - http://rt2x00.serialmonkey.com
> 
> Isn't this one already in the wireless tree? I believe it was merged in 
> 2.6.24 - 
> http://kernelnewbies.org/Linux_2_6_24#head-62e9ebf067c978bbf70898986c0aa3904d1a3543 
> .
> 
>> - http://aufs.sourceforge.net
> 
> I've read rumblings about one of the stacking FSes being merged but thus 
> far nothing seems to have happened - http://lwn.net/Articles/265240/ .
> 
>> - http://linux-uvc.berlios.de
> 
> USB video has already been merged into mainline in 2.6.25 - 
> http://kernelnewbies.org/Linux_2_6_26#head-6d4ee1ddea983f5c8ac491b8685a94ea6fa4132b 
> ! I'm using it on my eeepc...
> 
> Those are the ones I recognise off the top of my head. I'll have to 
> search with regard to the other ones you mentioned...


 > - http://sourceforge.net/projects/bluetooth-alsa

Allegedly deprecated in favour of a userspace ALSA plugin: 
http://www.mail-archive.com/debian-devel@lists.debian.org/msg253253.html


 > - http://www.unav-micro.com/Drivers.aspx -> 
AR81Family-linux-v1.0.1.0.tar.gz
 >   (e.g. asus-p5q-pro m/b has such ethernet port, pciid=1969:1026)

Isn't this the atl1e gigabit ethernet driver? Isn't this being merged 
into 2.6.27? - 
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=a6a5325239c20202e18e21e94291bccc659fbf9e

 > - http://www.misdn.org

Isn't this going into 2.6.27? - http://lwn.net/Articles/291630/

 > - zaptel telephony driver [http://www.asterisk.org]
 >   with firmware 
[http://downloads.digium.com/pub/telephony/firmware/releases]

Renamed to DAHDI and developers apparently don't want it merged into 
mainline - 
http://driverdev.linuxdriverproject.org/pipermail/devel/2008-June/000475.html 
.

 > - toshiba bluetooth module [http://0bits.com/toshbt/toshbt-1.0.tar.gz]

I can't reach that URI on 0bits.com at the moment but a cached page 
linking to it says the following:

"I was fedup patching the kernel everytime i needed this bluetooth 
enabled on my Tecra M2, so since we can use modules, i modularised the 
necessary bits to make the bluetooth enabling independent of kernel. 
Here it is in all it's glory."

I'm not sure what the state of that one is or what issue it is working 
around...

-- 
Sitsofe | http://sucs.org/~sits/

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[el es]

Paweł Sikora <pluto <at> agmk.net> writes:

> 
> On Thursday 25 of September 2008 04:48:40 Greg KH wrote:

> > What kernel modules do you currently use/rely on that are not in the
> > main kernel tree?  Do you have links to them?  I'd be glad to add them.
> 
> here's a quick list:
> 

+ http://www.mcentral.de

(there is lots of discussion on linux.drivers.dvb, the maintainer has his own
voes but... the driver actually works very good for me and I believe it should
be mainline regardless)

el es

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Greg KH]

On Thu, Sep 25, 2008 at 08:29:22AM +0200, Paweł Sikora wrote:
> On Thursday 25 of September 2008 04:48:40 Greg KH wrote:
> > On Wed, Sep 24, 2008 at 05:07:59PM +0200, Paweł Sikora wrote:
> > > Hi All,
> > >
> > > with out-of-tree kernel modules we (users and linux distributions)
> > > have a problem - with every frequent kernel release we need
> > > to rebuild them all due to vermagic and possible abi changes.
> > > this leads to lot of useless work (bumping rpm specs releases,
> > > preparing rpm packages and so on).
> >
> > "we" are working hard to remove any and all barriers for modules to be
> > out-of-tree.  See the recent drivers/staging/ announcement, I can now
> > take pretty much any code that at least builds and works for someone,
> > and get it into the kernel tree where it can be properly cleaned up and
> > cared for over time before it moves to the "correct" location within the
> > kernel.
> >
> > What kernel modules do you currently use/rely on that are not in the
> > main kernel tree?  Do you have links to them?  I'd be glad to add them.
> 
> here's a quick list:
> 
> - http://www.lirc.org
> - http://sourceforge.net/projects/bluetooth-alsa
> - http://rt2x00.serialmonkey.com
> - http://www.unav-micro.com/Drivers.aspx -> AR81Family-linux-v1.0.1.0.tar.gz
>   (e.g. asus-p5q-pro m/b has such ethernet port, pciid=1969:1026)
> - http://aufs.sourceforge.net
> - http://linux-uvc.berlios.de
> - http://www.misdn.org
> - toshiba bluetooth module [http://0bits.com/toshbt/toshbt-1.0.tar.gz]
> - zaptel telephony driver [http://www.asterisk.org]
>   with firmware [http://downloads.digium.com/pub/telephony/firmware/releases]

I think Sitsofe has answered all of these links as to if they are
already merged, working on being merged, or refuse to be merged (like
zaptel.)

Are there any other out-of-tree modules that I need to be aware of to
help integrate?

If not, then there really isn't an issue here, right?  :)

thanks,

greg k-h

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Paweł Sikora]

25/9/2008, "Greg KH" <greg@kroah.com> napisał/a:

>Are there any other out-of-tree modules that I need to be aware of to
>help integrate?

could you look at these:
- http://www.tahoe.pl/eng/patch.php
-
ftp://distfiles.pld-linux.org/distfiles/by-md5/8/e/8e4764f6c438427b00fb9aa93abb3cd7/toshbt-1.0.tar.gz

thanks,
pawel.

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Greg KH]

On Thu, Sep 25, 2008 at 03:48:19PM +0200, Paweł Sikora wrote:
> 25/9/2008, "Greg KH" <greg@kroah.com> napisał/a:
> 
> >Are there any other out-of-tree modules that I need to be aware of to
> >help integrate?
> 
> could you look at these:
> - http://www.tahoe.pl/eng/patch.php

This looks almost sane, I'll suck it in and contact the author.  It does
look like they are suggesting that a core network change is also needed,
I'll find out what is going on there.

> - ftp://distfiles.pld-linux.org/distfiles/by-md5/8/e/8e4764f6c438427b00fb9aa93abb3cd7/toshbt-1.0.tar.gz

This is a bit odder, but might just be needed for one brand of laptop.
I'll ask to find out, at the least it needs to be changed to not use
/proc/ anymore, which is easily done.

thanks for the pointers, if anyone has any others, please let me know.

greg k-h

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Tilman Schmidt]

[-- Attachment #1: Type: text/plain, Size: 590 bytes --]

Am 25.09.2008 14:54 schrieb Greg KH:
> Are there any other out-of-tree modules that I need to be aware of to
> help integrate?
> 
> If not, then there really isn't an issue here, right?  :)

These are probably already on your list, but for the sake of completeness:

- http://download.avm.de/cardware/fritzwlanusb.stick/linux/

- http://acx100.sourceforge.net/

Thanks,
Tilman

-- 
Tilman Schmidt                    E-Mail: tilman@imap.cc
Bonn, Germany
Diese Nachricht besteht zu 100% aus wiederverwerteten Bits.
Ungeöffnet mindestens haltbar bis: (siehe Rückseite)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Greg KH]

On Fri, Sep 26, 2008 at 01:25:05PM +0200, Tilman Schmidt wrote:
> Am 25.09.2008 14:54 schrieb Greg KH:
>> Are there any other out-of-tree modules that I need to be aware of to
>> help integrate?
>> If not, then there really isn't an issue here, right?  :)
>
> These are probably already on your list, but for the sake of completeness:
>
> - http://download.avm.de/cardware/fritzwlanusb.stick/linux/

Ah, no, I had missed this one, thanks.

> - http://acx100.sourceforge.net/

I think this one has some dubious legal issues involved, and I can't
take it at this time :(

thanks,

greg k-h

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Arkadiusz Miskiewicz]

On Friday 26 September 2008, Greg KH wrote:
> On Fri, Sep 26, 2008 at 01:25:05PM +0200, Tilman Schmidt wrote:
[...]
> > - http://acx100.sourceforge.net/
>
> I think this one has some dubious legal issues involved, and I can't
> take it at this time :(

Is this only about drivers? What about for example apparmor?

> thanks,
> greg k-h

-- 
Arkadiusz Miśkiewicz        PLD/Linux Team
arekm / maven.pl            http://ftp.pld-linux.org/

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Greg KH]

On Fri, Sep 26, 2008 at 07:19:52PM +0200, Arkadiusz Miskiewicz wrote:
> On Friday 26 September 2008, Greg KH wrote:
> > On Fri, Sep 26, 2008 at 01:25:05PM +0200, Tilman Schmidt wrote:
> [...]
> > > - http://acx100.sourceforge.net/
> >
> > I think this one has some dubious legal issues involved, and I can't
> > take it at this time :(
> 
> Is this only about drivers? What about for example apparmor?

apparmor requires patches to core parts of the kernel in order for it to
work.  This is for stand-alone drivers/filesystems/whatever only.

thanks,

greg k-h

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Alan Young]

On Thu, 25 Sep 2008 at 05:10:07 +0200, Greg KH wrote:

> What kernel modules do you currently use/rely on that are not in the
> main kernel tree?  Do you have links to them?  I'd be glad to add them.

I still use the ES56-PI modem driver @ http://tx.technion.ac.il/~raindel.
It has a 2.6 port up to 2.6.24, but it's not more current than that due to
serial/tty interface changes.  And there's a challenge that it uses a binary
blob. :-(  Best case would be to see if the vendor could open up like
Atheros (Thanks! Yea!) did.  If not, I wonder if the blob could be minimalized
to the bare modem functions and the serial interfaces be brought out into open code.

Alan

Re: [kernel-abi] symbol versioning vs. out-of-tree modules.

[Greg KH]

On Sun, Sep 28, 2008 at 11:51:03PM -0700, Alan Young wrote:
> On Thu, 25 Sep 2008 at 05:10:07 +0200, Greg KH wrote:
> 
> > What kernel modules do you currently use/rely on that are not in the
> > main kernel tree?  Do you have links to them?  I'd be glad to add them.
> 
> I still use the ES56-PI modem driver @ http://tx.technion.ac.il/~raindel.
> It has a 2.6 port up to 2.6.24, but it's not more current than that due to
> serial/tty interface changes.  And there's a challenge that it uses a binary
> blob. :-(  Best case would be to see if the vendor could open up like
> Atheros (Thanks! Yea!) did.  If not, I wonder if the blob could be
> minimalized to the bare modem functions and the serial interfaces be
> brought out into open code.

If the blob is removed, then yes, we can include it into the kernel tree
and I will be glad to help get it there.  But until then, sorry, there's
not much we can do.

thanks,

greg k-h