searching exported symbols from modules

searching exported symbols from modules

[Antti Halonen]

Hi,

Apologies if I posted this question to wrong place.

Here's the thing: when loading my module 'a', I want to search modules
list to check if module 'b' is presents, and if it is, initialize my
function pointers to the functions module b has exported. Or at least
search symbols from module b, whatever. The main question is; how to
locate modules 
by name from my module a?

Is this doable? Can anyone give me pointers? 

Sorry for posting such a stupid question, but I didn't run into
satisfactory when searching the archive & many of the functions which
would have resolved this are apparently not exported anymore.

Any comments are really appreciated!

Thanking in advance,
antti

Re: searching exported symbols from modules

[linux-os (Dick Johnson)]

On Wed, 19 Apr 2006, Antti Halonen wrote:

>
> Hi,
>
> Apologies if I posted this question to wrong place.
>
> Here's the thing: when loading my module 'a', I want to search modules
> list to check if module 'b' is presents, and if it is, initialize my
> function pointers to the functions module b has exported. Or at least
> search symbols from module b, whatever. The main question is; how to
> locate modules by name from my module a?
>

`insmod` (or modprobe) does all this automatically. Anything that's
'extern' will get resolved. You don't do anything special. You can
also use `depmod` to verify that you won't have any problems loading.
`man depmod`.

> Is this doable? Can anyone give me pointers?
>
> Sorry for posting such a stupid question, but I didn't run into
> satisfactory when searching the archive & many of the functions which
> would have resolved this are apparently not exported anymore.
>

If you are accessing functions or other objects that are not exported
anymore, how do you know that they even exist? You can search through
System.map to see if they are still global, but they might not be
the same size or type as before, or functions might take different
parameters.

You need to have built your modules using the current kernel headers
of the target OS version, using the current procedures (yes there are
some cheats that can be used), so that the current tools (like insmod
and modprobe) can properly handle your module.

> Any comments are really appreciated!
>
> Thanking in advance,
> antti
>

Cheers,
Dick Johnson
Penguin : Linux version 2.6.15.4 on an i686 machine (5589.53 BogoMips).
Warning : 98.36% of all statistics are fiction, book release in April.
_
\x1a\x04

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

RE: searching exported symbols from modules

[Antti Halonen]

Hi Dick,

Thanks for your response.

> `insmod` (or modprobe) does all this automatically. Anything that's
> 'extern' will get resolved. You don't do anything special. You can
> also use `depmod` to verify that you won't have any problems loading.
> `man depmod`.

Yes, I know insmod and herein the problem lies. I have a module where
I want to use functions provided by another module, _if_ it is present, 
otherwise use modules internal functions. 

So if I hardcode the function calls statically in my module, the insmod
goes trough of course, if the service module is present. But, it fails
if it's not. 

So, I want to load up, check if the service module is present, and use
it's servides.
 
> If you are accessing functions or other objects that are not exported
> anymore, how do you know that they even exist? 

I meant that in previous kernel version, some module symbol searching
functions were available.

Br,
antti

Re: searching exported symbols from modules

[Arjan van de Ven]

On Wed, 2006-04-19 at 15:44 +0300, Antti Halonen wrote:
> Hi,
> 
> Apologies if I posted this question to wrong place.
> 
> Here's the thing: when loading my module 'a', I want to search modules
> list to check if module 'b' is presents, and if it is, initialize my
> function pointers to the functions module b has exported. Or at least
> search symbols from module b, whatever. The main question is; how to
> locate modules 
> by name from my module a?
> 
> Is this doable? Can anyone give me pointers? 
> 
> Sorry for posting such a stupid question, but I didn't run into
> satisfactory when searching the archive & many of the functions which
> would have resolved this are apparently not exported anymore.

they still are... but.. very often if you want to do this you have a
crooked design, you were very vague about what you are really trying to
achieve (you only described your solution, not the problem)

RE: searching exported symbols from modules

[Antti Halonen]

Hi Arjan,

Thanks for your response.

> they still are... but.. very often if you want to do this you have a
> crooked design, you were very vague about what you are really trying
to
> achieve (you only described your solution, not the problem)

The problem is that I cannot seem to have access to the module linked
list. Ie. I want to traverse the linked list who's nodes contain module
structures.

Or

Access to some global symbol table would be helpful as well. Where's
listed the symbol address and name. Same stuff which is found at
/proc/kallsyms.

Br,
Antti

RE: searching exported symbols from modules

[linux-os (Dick Johnson)]

On Wed, 19 Apr 2006, Antti Halonen wrote:

>
> Hi Dick,
>
> Thanks for your response.
>
>> `insmod` (or modprobe) does all this automatically. Anything that's
>> 'extern' will get resolved. You don't do anything special. You can
>> also use `depmod` to verify that you won't have any problems loading.
>> `man depmod`.
>
> Yes, I know insmod and herein the problem lies. I have a module where
> I want to use functions provided by another module, _if_ it is present,
> otherwise use modules internal functions.
>
> So if I hardcode the function calls statically in my module, the insmod
> goes trough of course, if the service module is present. But, it fails
> if it's not.

/proc/kallsyms contains the offset and names of all the global kernel
symbols. This is not supposed to be accessed from inside the kernel,
but it could tell a program loader, that you write, if the required
symbols exist. If not, you can link in the required procedures
before you load your module. Note that 'module.ko' is just an
object file and you can use `ld` to link in other objects.

You could even do this with a shell-script (not tested):

SYMS="printk foo bar"
BASE="mymodule.ko"
FINAL="final.ko"

for x in $SYMS ; do if ! grep $x /proc/kallsyms>/dev/null ;
  then echo $x ; ld -o tmp.o mymodule.ko $x.o ; mv tmp.o mymodule.ko;
  fi;
  done

>
> So, I want to load up, check if the service module is present, and use
> it's servides.
>

Another thing you may be able to use are 'weak' symbols. I don't
know if the new `insmod` handles these. Weak symbols are used in
linking if globals are not found (which is what you want). You can
try "# pragma weak" or __attribute__((weak)) for all functions you
want to use if globals are not found. You can see if that works.
If it works, you are home free, if not you can contact the module-
tools people and see if they could add handling weak symbols.

 	Rusty Russell <rusty@rustcorp.com.au>
 	Adam J. Richter <adam@yggdrasil.com>


>> If you are accessing functions or other objects that are not exported
>> anymore, how do you know that they even exist?
>
> I meant that in previous kernel version, some module symbol searching
> functions were available.
>


Cheers,
Dick Johnson
Penguin : Linux version 2.6.15.4 on an i686 machine (5589.53 BogoMips).
Warning : 98.36% of all statistics are fiction, book release in April.
_
\x1a\x04

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: searching exported symbols from modules

[Jim Cromie]

linux-os (Dick Johnson) wrote:
>
> `insmod` (or modprobe) does all this automatically


only modprobe does dependency loading.



soekris:~# rmmod scx200_gpio scx200
soekris:~#
soekris:~# insmod scx200_gpio
insmod: can't read 'scx200_gpio': No such file or directory
soekris:~#
soekris:~# modprobe scx200_gpio
[  237.848000] scx200: NatSemi SCx200 Driver
[  238.344000] scx200: GPIO base 0x6100
[  238.356000] scx200: Configuration Block base 0x6000

soekris:~# lsmod
Module                  Size  Used by
scx200_gpio             4292  0
scx200                  4912  1 scx200_gpio

RE: searching exported symbols from modules

[Arjan van de Ven]

On Wed, 2006-04-19 at 17:54 +0300, Antti Halonen wrote:
> Hi Arjan,
> 
> Thanks for your response.
> 
> > they still are... but.. very often if you want to do this you have a
> > crooked design, you were very vague about what you are really trying
> to
> > achieve (you only described your solution, not the problem)
> 
> The problem is that I cannot seem to have access to the module linked
> list. Ie. I want to traverse the linked list who's nodes contain module
> structures.
> 
> Or
> 
> Access to some global symbol table would be helpful as well. Where's
> listed the symbol address and name. Same stuff which is found at
> /proc/kallsyms.

but to what purpose?
what on earth do you need this for?
Again you talk about solutions not about the problem you're trying to
solve.

RE: searching exported symbols from modules

[Antti Halonen]

Thanks for your persistence!

> but to what purpose?
> what on earth do you need this for?
> Again you talk about solutions not about the problem you're trying to
> solve.

I have module which has set of algorithms. When I load my module, I will
initialize my function pointer interface to point to these algorithms.
And then go ahead and use my pointers trough the application...

This way I can easily add new algorithm implementations into my product
and decide on the fly which ones to initialize into use.
 
Now. It happens that I have another module which offers algorithms I
want to be able to use. So when I shoot up my module it should be able
to initialize
either the ones the external module offers or use the internal
algorithms. In order to do this, I need to figure out on the fly if the
other module is present, where it is, where I can find the function
addresses...

Hope you can make any sense out of this!

br,
antti
 

RE: searching exported symbols from modules

[Arjan van de Ven]

On Wed, 2006-04-19 at 18:34 +0300, Antti Halonen wrote:
> Thanks for your persistence!
> 
> > but to what purpose?
> > what on earth do you need this for?
> > Again you talk about solutions not about the problem you're trying to
> > solve.
> 
> I have module which has set of algorithms. When I load my module, I will
> initialize my function pointer interface to point to these algorithms.
> And then go ahead and use my pointers trough the application...
> 
> This way I can easily add new algorithm implementations into my product
> and decide on the fly which ones to initialize into use.
>  
> Now. It happens that I have another module which offers algorithms I
> want to be able to use. So when I shoot up my module it should be able
> to initialize
> either the ones the external module offers or use the internal
> algorithms. In order to do this, I need to figure out on the fly if the
> other module is present, where it is, where I can find the function
> addresses...

a better way would be then to have a "core" module which is basically
collecting these algorithms, and then have the algorithm modules, when
they are loaded, register themselves with this core module. (and
unregister at unload). It's sort of inside-out with they way you're
trying to do it, but it'll work out a lot nicer. Obviously the user of
the algorithms can be another module in addition to this core module.
(and even register algorithms itself)
There's many examples in the kernel that work similar to this, for
example the crypto framework.

Re: searching exported symbols from modules

[Robin Holt]

On Wed, Apr 19, 2006 at 04:08:37PM +0300, Antti Halonen wrote:
> 
> Hi Dick,
> 
> Thanks for your response.
> 
> > `insmod` (or modprobe) does all this automatically. Anything that's
> > 'extern' will get resolved. You don't do anything special. You can
> > also use `depmod` to verify that you won't have any problems loading.
> > `man depmod`.
> 
> Yes, I know insmod and herein the problem lies. I have a module where
> I want to use functions provided by another module, _if_ it is present, 
> otherwise use modules internal functions. 
> 
> So if I hardcode the function calls statically in my module, the insmod
> goes trough of course, if the service module is present. But, it fails
> if it's not. 
> 
> So, I want to load up, check if the service module is present, and use
> it's servides.

xpc does something similar to this.  We have a module (xp.ko) which gets
loaded.  The client modules (xpnet.ko and xpmem.ko) register with xp.ko.
If xpc.ko is not loaded, the modules can not do any communication with
the other partitions of an SGI machine.  If xpc gets loaded, connections
are made to the other partitions and the modules begin to work across
the boundaries.

Hope that helps,
Robin Holt

RE: searching exported symbols from modules

[Antti Halonen]

> a better way would be then to have a "core" module which is basically
> collecting these algorithms, and then have the algorithm modules, when
> they are loaded, register themselves with this core module. (and
> unregister at unload). It's sort of inside-out with they way you're
> trying to do it, but it'll work out a lot nicer. Obviously the user of
> the algorithms can be another module in addition to this core module.
> (and even register algorithms itself)

Exactly, I agree 100%. But here's the catch: it's not an option at this
point in time. What I described earlier is what I currently need to do. 

RE: searching exported symbols from modules

[Arjan van de Ven]

On Wed, 2006-04-19 at 18:59 +0300, Antti Halonen wrote:
> > a better way would be then to have a "core" module which is basically
> > collecting these algorithms, and then have the algorithm modules, when
> > they are loaded, register themselves with this core module. (and
> > unregister at unload). It's sort of inside-out with they way you're
> > trying to do it, but it'll work out a lot nicer. Obviously the user of
> > the algorithms can be another module in addition to this core module.
> > (and even register algorithms itself)
> 
> Exactly, I agree 100%. But here's the catch: it's not an option at this
> point in time. 

eh why not? For your module to be ever merged into mainline this change
will need to be done anyway (and it'll save you time as well even on the
short term)

RE: searching exported symbols from modules

[Antti Halonen]

> > Exactly, I agree 100%. But here's the catch: it's not an option at
this
> > point in time.
> 
> eh why not? For your module to be ever merged into mainline this
change
> will need to be done anyway (and it'll save you time as well even on
the
> short term)

This is commercial module and will not be merged into mainline.

It would be helpful if I could get at least confirmation if the thing is
doable what I described earlier. 

Pardon me as I don't mean to sound rude; I'm not interested if this
should be done in some totally different way. Let's just say that my
curiosity is peaked to the point that I want & need to know how to
search modules by name in kernel space. I don't know how to be more
specific what comes to my problem.

Thanks in advance for any input!

Br,
antti

RE: searching exported symbols from modules

[James Morris]

On Thu, 20 Apr 2006, Antti Halonen wrote:

> > > Exactly, I agree 100%. But here's the catch: it's not an option at
> this
> > > point in time.
> > 
> > eh why not? For your module to be ever merged into mainline this
> change
> > will need to be done anyway (and it'll save you time as well even on
> the
> > short term)
> 
> This is commercial module and will not be merged into mainline.

Is it GPL or compatible?



- James
-- 
James Morris
<jmorris@namei.org>

RE: searching exported symbols from modules

[Antti Halonen]

> > This is commercial module and will not be merged into mainline.
> 
> Is it GPL or compatible?

Nope, MODULE_LICENSE("Proprietary").

BR,
Antti

Re: searching exported symbols from modules

[Kyle Moffett]

On Apr 20, 2006, at 02:12:09, Antti Halonen wrote:
>>> This is commercial module and will not be merged into mainline.
>>
>> Is it GPL or compatible?
>
> Nope, MODULE_LICENSE("Proprietary").

This makes me wonder why the hell you think you'll get help from this  
(open-source) list.  A large number of people on this list (including  
copyright-holders) consider what you are doing blatantly illegal,  
although nobody has yet gone to court over it.  Please reconsider  
your decision and either opensource your code or find a way to do it  
in userspace.  For example, it is quite easy to write a low-latency  
USB device driver completely in userspace with only standard kernel  
drivers.

Cheers,
Kyle Moffett

RE: searching exported symbols from modules

[Antti Halonen]

Hi Kyle, 

> This makes me wonder why the hell you think you'll get help from this
> (open-source) list.  

I see, I figured this is the place to ask technical question related 
to programming in kernel space. 

> A large number of people on this list (including
> copyright-holders) consider what you are doing blatantly illegal,
> although nobody has yet gone to court over it.  

Um, which part is illegal? Are you saying that I cannot have a non-open
source kernel module? If I figured correctly, to violate GPL I should
compile GPL code into my module.

It is a standalone module, not distributed with any custom kernel.

> Please reconsider
> your decision and either opensource your code or find a way to do it
> in userspace.  

Certainly this needs to be clarified.

Br,
Antti

Re: searching exported symbols from modules

[Kyle Moffett]

On Apr 20, 2006, at 03:15:42, Antti Halonen wrote:
>> This makes me wonder why the hell you think you'll get help from  
>> this (open-source) list.
>
> I see, I figured this is the place to ask technical question  
> related to programming in kernel space.

Typically yes; however there are two problems:
   1)  If we can't see your code, we probably can't help you very  
well.  Since you're not providing your code, you're wasting both your  
time and ours with questions that would be much more easily answered  
with access to that code.
   2)  Many people will refuse to provide help for what they perceive  
as violations of their copyright.

>> A large number of people on this list (including copyright- 
>> holders) consider what you are doing blatantly illegal, although  
>> nobody has yet gone to court over it.
>
> Um, which part is illegal?

Many people consider any Linux kernel module to be a derivative work  
of the Linux kernel.  For example, you use non-standardized internal  
header files containing inline functions which end up compiled into  
your code, you use interfaces which are only present in the Linux  
kernel itself, your binary module does not run without a specific  
particular version of the Linux kernel, etc.

> Are you saying that I cannot have a non-open source kernel module?

My personal opinion is not relevant here, it's that of the major  
copyright holders (of which there are thousands) and the courts.  If  
nobody ever sues you over your kernel module, or if you win any  
lawsuit, you're ok.  On the other hand, it's such a sensitive topic  
you should go talk to your lawyer first.  Many significant kernel  
copyright holders believe that essentially any kernel module is a  
derivative work of the Linux kernel, and as such must be GPLed to  
comply with copyright law in most major countries.

> If I figured correctly, to violate GPL I should compile GPL code  
> into my module.

You are doing exactly that when you use atomic_t, spinlocks, or a  
thousand other inline functions from the kernel headers.

> It is a standalone module, not distributed with any custom kernel.

Which kernel does it run against?  Does this run on other operating  
systems?  Do you require certain Linux-only interfaces?  Can your  
work reasonably be considered independent?  I suggest talking with a  
lawyer to answer these questions before attempting to release any non- 
GPL-compatible kernel module; just as you would if you were releasing  
a module for Windows for which there is not a defined API.

Cheers,
Kyle Moffett

Re: searching exported symbols from modules

[Adrian Bunk]

On Thu, Apr 20, 2006 at 10:15:42AM +0300, Antti Halonen wrote:
>...
> > A large number of people on this list (including
> > copyright-holders) consider what you are doing blatantly illegal,
> > although nobody has yet gone to court over it.  
> 
> Um, which part is illegal? Are you saying that I cannot have a non-open
> source kernel module? If I figured correctly, to violate GPL I should
> compile GPL code into my module.
> 
> It is a standalone module, not distributed with any custom kernel.
>...

A short discussion is e.g. at [1]. But AFAIK none of this has yet been 
brought to court.

Things are even more funny considering that if you distribute your 
module in N countries, you can be sued in N different countries based on 
N different copyright laws...

> Br,
> Antti

cu
Adrian

[1] http://en.wikipedia.org/wiki/Linux_kernel#Licensing_terms

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: searching exported symbols from modules

[Keith Owens]

"Antti Halonen" (on Wed, 19 Apr 2006 16:08:37 +0300) wrote:
>
>Hi Dick,
>
>Thanks for your response.
>
>> `insmod` (or modprobe) does all this automatically. Anything that's
>> 'extern' will get resolved. You don't do anything special. You can
>> also use `depmod` to verify that you won't have any problems loading.
>> `man depmod`.
>
>Yes, I know insmod and herein the problem lies. I have a module where
>I want to use functions provided by another module, _if_ it is present, 
>otherwise use modules internal functions. 

symbol_get() and symbol_put().  See include/linux/module.h.  If
symbol_get() returns NULL then the symbol does not exist.

Re: searching exported symbols from modules

[Vernon Mauery]

On Friday 21 April 2006 00:33, Keith Owens wrote:
> "Antti Halonen" (on Wed, 19 Apr 2006 16:08:37 +0300) wrote:
> >Hi Dick,
> >
> >Thanks for your response.
> >
> >> `insmod` (or modprobe) does all this automatically. Anything that's
> >> 'extern' will get resolved. You don't do anything special. You can
> >> also use `depmod` to verify that you won't have any problems loading.
> >> `man depmod`.
> >
> >Yes, I know insmod and herein the problem lies. I have a module where
> >I want to use functions provided by another module, _if_ it is present,
> >otherwise use modules internal functions.
>
> symbol_get() and symbol_put().  See include/linux/module.h.  If
> symbol_get() returns NULL then the symbol does not exist.

The trick with symbol_get is that it is marked as EXPORT_SYMBOL_GPL, so only 
GPL modules can use it.  Not that that is a bad thing though. :)

If you want all of these nice kernel features, GPL your module!

--Vernon