Incremental Linking

Incremental Linking

[somanath sahoo]

Hi,
?
As newbie in linux kernel,?I would like to?understand?the concept of? "incremental linking?" w.r.t to linux kernel module. 
?
I have read that?kernel loadable modules are?being generated and inserted?into a running kernel?due to this?"incremental linking " technology.
?
If anyone can provide me some pointers to understand the "incremental linking" concept, it will be needful.
?
?
Thanks,
Somanath
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120522/31618fbc/attachment.html 

Incremental Linking

[Jonathan Neuschäfer]

On Tue, May 22, 2012 at 02:56:35AM -0700, somanath sahoo wrote:
> Hi,
> ?
> As newbie in linux kernel,?I would like to?understand?the concept of? "incremental linking?" w.r.t to linux kernel module. 
> ?
> I have read that?kernel loadable modules are?being generated and inserted?into a running kernel?due to this?"incremental linking " technology.

Can you give us a pointer to where you read this?

Thanks,
	Jonathan Neusch?fer

> ?
> If anyone can provide me some pointers to understand the "incremental linking" concept, it will be needful.
> ?
> ?
> Thanks,
> Somanath

Incremental Linking

[somanath sahoo]

Hi Jonathan,
?
I came to know about "incremental linking" while reading "Embedded linux Primer".
?
?
Regards,
Somanath?


________________________________
From: Jonathan Neusch?fer <j.neuschaefer@gmx.net>
To: somanath sahoo <bapi_mvit2004@yahoo.com> 
Cc: "kernelnewbies at kernelnewbies.org" <kernelnewbies@kernelnewbies.org> 
Sent: Tuesday, May 22, 2012 8:47 PM
Subject: Re: Incremental Linking

On Tue, May 22, 2012 at 02:56:35AM -0700, somanath sahoo wrote:
> Hi,
> ?
> As newbie in linux kernel,?I would like to?understand?the concept of? "incremental linking?" w.r.t to linux kernel module. 
> ?
> I have read that?kernel loadable modules are?being generated and inserted?into a running kernel?due to this?"incremental linking " technology.

Can you give us a pointer to where you read this?

Thanks,
??? Jonathan Neusch?fer

> ?
> If anyone can provide me some pointers to understand the "incremental linking" concept, it will be needful.
> ?
> ?
> Thanks,
> Somanath
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120522/bbf150ca/attachment.html 

Incremental Linking

[Sarbojit Ganguly]

Can you please elaborate on that? What exactly you are trying to find
out and other details.


On 23 May 2012 12:51, somanath sahoo <bapi_mvit2004@yahoo.com> wrote:
> Hi Sarbojit,
>
> Does?the Incremental link correspond to "incremental build" ?
>
> I think that?incremental link corresponds to lazy binding by which kernel
> modules are being inserted into running kernel.
>
> please CMIIW.
>
> Regards,
> Somanath
>
> From: Sarbojit Ganguly <unixman.linuxboy@gmail.com>
> To: somanath sahoo <bapi_mvit2004@yahoo.com>
> Sent: Wednesday, May 23, 2012 11:20 AM
> Subject: Re: Incremental Linking
>
> Somanath, IMHO linux kernel build system _uses_ incremental build. It
> checks which object files are out dated and then updates.
>
> On 23 May 2012 11:09, somanath sahoo <bapi_mvit2004@yahoo.com> wrote:
>> Hi Jonathan,
>>
>> I came to know about "incremental linking" while reading "Embedded linux
>> Primer".
>>
>>
>> Regards,
>> Somanath
>>
>> From: Jonathan Neusch?fer <j.neuschaefer@gmx.net>
>> To: somanath sahoo <bapi_mvit2004@yahoo.com>
>> Cc: "kernelnewbies at kernelnewbies.org" <kernelnewbies@kernelnewbies.org>
>> Sent: Tuesday, May 22, 2012 8:47 PM
>> Subject: Re: Incremental Linking
>>
>> On Tue, May 22, 2012 at 02:56:35AM -0700, somanath sahoo wrote:
>>> Hi,
>>>
>>> As newbie in linux kernel,?I would like to?understand?the concept of
>>> "incremental linking?" w.r.t to linux kernel module.
>>>
>>> I have read that?kernel loadable modules are?being generated and
>>> inserted?into a running kernel?due to this?"incremental linking "
>>> technology.
>>
>> Can you give us a pointer to where you read this?
>>
>> Thanks,
>> ??? Jonathan Neusch?fer
>>
>>>
>>> If anyone can provide me some pointers to understand the "incremental
>>> linking" concept, it will be needful.
>>>
>>>
>>> Thanks,
>>> Somanath
>>
>>
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>
>

Incremental Linking

[Sarbojit Ganguly]

I think this might help (taking a cue from your last statement)

http://www.ibm.com/developerworks/linux/library/l-lkm/

On 23 May 2012 13:20, Sarbojit Ganguly <unixman.linuxboy@gmail.com> wrote:
> Can you please elaborate on that? What exactly you are trying to find
> out and other details.
>
>
> On 23 May 2012 12:51, somanath sahoo <bapi_mvit2004@yahoo.com> wrote:
>> Hi Sarbojit,
>>
>> Does?the Incremental link correspond to "incremental build" ?
>>
>> I think that?incremental link corresponds to lazy binding by which kernel
>> modules are being inserted into running kernel.
>>
>> please CMIIW.
>>
>> Regards,
>> Somanath
>>
>> From: Sarbojit Ganguly <unixman.linuxboy@gmail.com>
>> To: somanath sahoo <bapi_mvit2004@yahoo.com>
>> Sent: Wednesday, May 23, 2012 11:20 AM
>> Subject: Re: Incremental Linking
>>
>> Somanath, IMHO linux kernel build system _uses_ incremental build. It
>> checks which object files are out dated and then updates.
>>
>> On 23 May 2012 11:09, somanath sahoo <bapi_mvit2004@yahoo.com> wrote:
>>> Hi Jonathan,
>>>
>>> I came to know about "incremental linking" while reading "Embedded linux
>>> Primer".
>>>
>>>
>>> Regards,
>>> Somanath
>>>
>>> From: Jonathan Neusch?fer <j.neuschaefer@gmx.net>
>>> To: somanath sahoo <bapi_mvit2004@yahoo.com>
>>> Cc: "kernelnewbies at kernelnewbies.org" <kernelnewbies@kernelnewbies.org>
>>> Sent: Tuesday, May 22, 2012 8:47 PM
>>> Subject: Re: Incremental Linking
>>>
>>> On Tue, May 22, 2012 at 02:56:35AM -0700, somanath sahoo wrote:
>>>> Hi,
>>>>
>>>> As newbie in linux kernel,?I would like to?understand?the concept of
>>>> "incremental linking?" w.r.t to linux kernel module.
>>>>
>>>> I have read that?kernel loadable modules are?being generated and
>>>> inserted?into a running kernel?due to this?"incremental linking "
>>>> technology.
>>>
>>> Can you give us a pointer to where you read this?
>>>
>>> Thanks,
>>> ??? Jonathan Neusch?fer
>>>
>>>>
>>>> If anyone can provide me some pointers to understand the "incremental
>>>> linking" concept, it will be needful.
>>>>
>>>>
>>>> Thanks,
>>>> Somanath
>>>
>>>
>>>
>>> _______________________________________________
>>> Kernelnewbies mailing list
>>> Kernelnewbies at kernelnewbies.org
>>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>>
>>
>>

Incremental Linking

[Mulyadi Santosa]

On Tue, May 22, 2012 at 4:56 PM, somanath sahoo <bapi_mvit2004@yahoo.com> wrote:
> Hi,
>
> As newbie in linux kernel,?I would like to?understand?the concept of
> "incremental linking?" w.r.t to linux kernel module.
>
> I have read that?kernel loadable modules are?being generated and
> inserted?into a running kernel?due to this?"incremental linking "
> technology.
>
> If anyone can provide me some pointers to understand the "incremental
> linking" concept, it will be needful.

perhaps you meant lazy binding? same thing like what glibc does in
user space I believe.

In short, symbols (functions etc) are not resolved right away, but
looked up and referenced when needed only.

Hopefully I point you the correct meaning. If not, feel free to CMIIW.

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Incremental Linking

[Dave Hylands]

Hi Somanath,

On Tue, May 22, 2012 at 2:56 AM, somanath sahoo <bapi_mvit2004@yahoo.com> wrote:
> Hi,
>
> As newbie in linux kernel,?I would like to?understand?the concept of
> "incremental linking?" w.r.t to linux kernel module.

Incremental linking is basically just taking some objects files,
linking them together to produce a larger object file. The object file
still has undefined references. It will also coalesce sections of the
same name. The kernel likes to use sections for storing pointers to
initcall functions and other things like that.

Some people might also call this partial linking. The kernel uses this
technique for the main portion of the kernel as well. If you look
through your build directory, you will find a whole bunch of
built-in.o files. Each one of these is a partially linked object file
containing all of the object files from the current directory and
built-in.o files from directories below.

With kernel modules, there are some special automatically generated C
files which also get linked in (IIRC that have a name like foo.mod.c)

A kernel module is conceptually identical to a shared library (which
is also partially linked and may contain unresolved references).

You can do incremental linking  by doing:

echo "int foo1(void) { return 1; }" > foo1.c
echo "int foo2(void) { return 2; }" > foo2.c
gcc -c foo1.c
gcc -c foo2.c
ld -r -o foo.o foo1.o foo2.o

You'll now have foo.o which has both foo1.o and foo2.o partially
linked together, which you can see by using:

nm foo.o

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

Incremental Linking

[Sarbojit Ganguly]

Hello Dave,

I tried to explain this feature (no doubt you have explain it
perfectly) but he is looking for _how_ kernel module gets loaded and
somehow (I wonder how!) dubs the entire process as "incremental link"
!

On 23 May 2012 21:47, Dave Hylands <dhylands@gmail.com> wrote:
> Hi Somanath,
>
> On Tue, May 22, 2012 at 2:56 AM, somanath sahoo <bapi_mvit2004@yahoo.com> wrote:
>> Hi,
>>
>> As newbie in linux kernel,?I would like to?understand?the concept of
>> "incremental linking?" w.r.t to linux kernel module.
>
> Incremental linking is basically just taking some objects files,
> linking them together to produce a larger object file. The object file
> still has undefined references. It will also coalesce sections of the
> same name. The kernel likes to use sections for storing pointers to
> initcall functions and other things like that.
>
> Some people might also call this partial linking. The kernel uses this
> technique for the main portion of the kernel as well. If you look
> through your build directory, you will find a whole bunch of
> built-in.o files. Each one of these is a partially linked object file
> containing all of the object files from the current directory and
> built-in.o files from directories below.
>
> With kernel modules, there are some special automatically generated C
> files which also get linked in (IIRC that have a name like foo.mod.c)
>
> A kernel module is conceptually identical to a shared library (which
> is also partially linked and may contain unresolved references).
>
> You can do incremental linking ?by doing:
>
> echo "int foo1(void) { return 1; }" > foo1.c
> echo "int foo2(void) { return 2; }" > foo2.c
> gcc -c foo1.c
> gcc -c foo2.c
> ld -r -o foo.o foo1.o foo2.o
>
> You'll now have foo.o which has both foo1.o and foo2.o partially
> linked together, which you can see by using:
>
> nm foo.o
>
> --
> Dave Hylands
> Shuswap, BC, Canada
> http://www.davehylands.com
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Incremental Linking

[Dave Hylands]

Hi,

On Wed, May 23, 2012 at 9:51 AM, Sarbojit Ganguly
<unixman.linuxboy@gmail.com> wrote:
> Hello Dave,
>
> I tried to explain this feature (no doubt you have explain it
> perfectly) but he is looking for _how_ kernel module gets loaded and
> somehow (I wonder how!) dubs the entire process as "incremental link"
> !

Then you'll probably want to look at the source code for kmod, which
does most of the work.
http://git.profusion.mobi/cgit.cgi/kmod.git/tree/
Older kernels used module-init-tools

Basically, you load the module into memory, enumerate the list of
unresolved symbols, look those symbols up in the kernel to find their
address and plug them in.

There are lots of little details that I've glossed over, but those
little details are pretty standard stuff as far as linkers and loaders
go and isn't really kernel specific.

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

Incremental Linking

[Jim Cromie]

On Wed, May 23, 2012 at 10:51 AM, Sarbojit Ganguly
<unixman.linuxboy@gmail.com> wrote:
> Hello Dave,
>
> I tried to explain this feature (no doubt you have explain it
> perfectly) but he is looking for _how_ kernel module gets loaded and
> somehow (I wonder how!) dubs the entire process as "incremental link"
> !
>

The easiest way to see the whole process happening is to turn on debug,
then modprobe a module.  youll see lots of details.

If you build 3.5-rc0 (ie rc1 is not done yet), and turn on CONFIG_DYNAMIC_DEBUG,
you can enable the pr_debugs selectively, and narrow down to the info
that you want.

heres some of the info youll see:

jimc at jimc-desktop:~/projects/lx/linux-2.6$ grep pr_debug kernel/module.c
	pr_debug("Failed to find symbol %s\n", name);
			pr_debug("%s uses %s!\n", a->name, b->name);
	pr_debug("%s does not use %s!\n", a->name, b->name);
	pr_debug("Allocating new usage for %s.\n", a->name);
		pr_debug("%s unusing %s\n", mod->name, i->name);
		pr_debug("Looking at refcount...\n");
		pr_debug("%s already dying\n", mod->name);
		pr_debug("Found checksum %lX vs module %lX\n",
			pr_debug("Common symbol: %s\n", name);
			pr_debug("Absolute symbol: 0x%08lx\n",
	pr_debug("Core section allocation order:\n");
			pr_debug("\t%s\n", sname);
	pr_debug("Init section allocation order:\n");
			pr_debug("\t%s\n", sname);
	pr_debug("\t%s\n", info->secstrings + symsect->sh_name);
	pr_debug("\t%s\n", info->secstrings + strsect->sh_name);
	pr_debug("final section addresses:\n");
		pr_debug("\t0x%lx %s\n",
	pr_debug("load_module: umod=%p, len=%lu, uargs=%p\n",

I dont recall if it gets into symbol-by-symbol linking, but it might.
Read Doc/dynamic_debug_howto for more usage info.

Daves simple example would be valuable to look at,
esp with readelf and objdump.


> On 23 May 2012 21:47, Dave Hylands <dhylands@gmail.com> wrote:
>> Hi Somanath,
>>
>> On Tue, May 22, 2012 at 2:56 AM, somanath sahoo <bapi_mvit2004@yahoo.com> wrote:
>>> Hi,
>>>
>>> As newbie in linux kernel,?I would like to?understand?the concept of
>>> "incremental linking?" w.r.t to linux kernel module.
>>
>> Incremental linking is basically just taking some objects files,
>> linking them together to produce a larger object file. The object file
>> still has undefined references. It will also coalesce sections of the
>> same name. The kernel likes to use sections for storing pointers to
>> initcall functions and other things like that.
>>
>> Some people might also call this partial linking. The kernel uses this
>> technique for the main portion of the kernel as well. If you look
>> through your build directory, you will find a whole bunch of
>> built-in.o files. Each one of these is a partially linked object file
>> containing all of the object files from the current directory and
>> built-in.o files from directories below.
>>
>> With kernel modules, there are some special automatically generated C
>> files which also get linked in (IIRC that have a name like foo.mod.c)
>>
>> A kernel module is conceptually identical to a shared library (which
>> is also partially linked and may contain unresolved references).
>>
>> You can do incremental linking ?by doing:
>>
>> echo "int foo1(void) { return 1; }" > foo1.c
>> echo "int foo2(void) { return 2; }" > foo2.c
>> gcc -c foo1.c
>> gcc -c foo2.c
>> ld -r -o foo.o foo1.o foo2.o
>>
>> You'll now have foo.o which has both foo1.o and foo2.o partially
>> linked together, which you can see by using:
>>
>> nm foo.o
>>
>> --
>> Dave Hylands
>> Shuswap, BC, Canada
>> http://www.davehylands.com
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies