Searching for insight into lib-y and lib-m

Searching for insight into lib-y and lib-m

[George Spelvin]

I'm working on a patch to factor out some useful helper code
(specifically, the glob_match() function) from drivers/ata/libata-core.c
to a separate file lib/glob.c, so other drivers can use it.  But I'm
having a hard time figuring out how to arrange for every caller to be
able to call it.

In particular, suppose that CONFIG_ATA=m.  I'm trying to figure out
what the various options I have are for handling my new option CONFIG_GLOB.

Here are the four basic possibilities (all combined with "select GLOB"
by ATA):

1) bool GLOB; obj-$(CONFIG_GLOB) += glob.o

I think this will atatically link the glob.o code into the kernel if
CONFIG_ATA=y or CONFIG_ATA=m.  So everything will work, but there's no
way for an out-of-tree driver to depend on the code if CONFIG_ATA=n.
(Not supporting that case for now is definitely an option.)

2) tristate GLOB; obj-$(CONFIG_GLOB) += glob.o

In this case, if CONFIG_ATA=m, I'll create a separate module glob.ko.
I'd need to include <linux/module.h> and add the appropriate module
declarations to glob.c.  Lots of code in lib/ seems to do this, but a
proliferation if microscopic modules is annoying.

3) bool GLOB; lib-$(CONFIG_GLOB) += glob.o

This is where I get confused.  Documentation/kbuild/makefiles.txt says
that glob.o will be included in the library lib.a.

But it then doesn't say anything about what (if anything) is linked
against lib.a.  I assume (although it's not actually stated) that vmlinux
is linked against it, so the linker will import any called functions.

But if CONFIG_ATA=m, presumably the linker won't import glob_match().
What then will happen if ata.ko is loaded?  Does it get linked statically
against lib.a at module build time?  Or will module loading just fail?
Looking at the output of "make V=1 modules", I don't see any .a files
mentioned.

Is there some other magic I don't understand that will make it work?

The static linking would be nice; although that would result in code
duplication if multiple modules used the code, the code is awfully
small (658 bytes of text) and it would take a lot of modules to make it
a problem.

As to what this would do to out-of-tree drivers, I have no idea...

4) tristate GLOB; lib-$(CONFIG_GLOB) += glob.o

Documentation/kbuild/makefiles.txt suggests that CONFIG_GLOB=m would
be synonymous with CONFIG_GLOB=y in this case, so the net effect would
be identical to case 3.  Am I right?


May I humbly beg some guidance as to the best way to proceed?  My current code
takes option 2 and builds its own module if CONFIG_ATA=m, but people are asking if
that's really necessary.

(v1 patch at https://lkml.org/lkml/2014/5/9/663 if anyone wants more details.)

The follow-on question is how to integrate the self-test code.  I had defined
a preprocessor symbol which allowed a standalone "gcc -DUNITTEST glob.c" to
make a user-space test driver, but I've been asked to kernelize it.  If using
the full module route, I can just use __init code for it, like lib/list_sort.c.
But for other options, its less clear.

Re: Searching for insight into lib-y and lib-m

[Randy Dunlap]

On 05/10/2014 07:24 PM, George Spelvin wrote:
> I'm working on a patch to factor out some useful helper code
> (specifically, the glob_match() function) from drivers/ata/libata-core.c
> to a separate file lib/glob.c, so other drivers can use it.  But I'm
> having a hard time figuring out how to arrange for every caller to be
> able to call it.
> 
> In particular, suppose that CONFIG_ATA=m.  I'm trying to figure out
> what the various options I have are for handling my new option CONFIG_GLOB.
> 
> Here are the four basic possibilities (all combined with "select GLOB"
> by ATA):
> 
> 1) bool GLOB; obj-$(CONFIG_GLOB) += glob.o
> 
> I think this will atatically link the glob.o code into the kernel if
> CONFIG_ATA=y or CONFIG_ATA=m.  So everything will work, but there's no
> way for an out-of-tree driver to depend on the code if CONFIG_ATA=n.
> (Not supporting that case for now is definitely an option.)
> 
> 2) tristate GLOB; obj-$(CONFIG_GLOB) += glob.o
> 
> In this case, if CONFIG_ATA=m, I'll create a separate module glob.ko.
> I'd need to include <linux/module.h> and add the appropriate module
> declarations to glob.c.  Lots of code in lib/ seems to do this, but a
> proliferation if microscopic modules is annoying.
> 
> 3) bool GLOB; lib-$(CONFIG_GLOB) += glob.o
> 
> This is where I get confused.  Documentation/kbuild/makefiles.txt says
> that glob.o will be included in the library lib.a.
> 
> But it then doesn't say anything about what (if anything) is linked
> against lib.a.  I assume (although it's not actually stated) that vmlinux
> is linked against it, so the linker will import any called functions.
> 
> But if CONFIG_ATA=m, presumably the linker won't import glob_match().
> What then will happen if ata.ko is loaded?  Does it get linked statically
> against lib.a at module build time?  Or will module loading just fail?
> Looking at the output of "make V=1 modules", I don't see any .a files
> mentioned.
> 
> Is there some other magic I don't understand that will make it work?
> 
> The static linking would be nice; although that would result in code
> duplication if multiple modules used the code, the code is awfully
> small (658 bytes of text) and it would take a lot of modules to make it
> a problem.
> 
> As to what this would do to out-of-tree drivers, I have no idea...
> 
> 4) tristate GLOB; lib-$(CONFIG_GLOB) += glob.o
> 
> Documentation/kbuild/makefiles.txt suggests that CONFIG_GLOB=m would
> be synonymous with CONFIG_GLOB=y in this case, so the net effect would
> be identical to case 3.  Am I right?
> 

'git log lib/Makefile' shows me patches that I have submitted where
kernel modules were built as loadable modules (=m) and library code was
built but then apparently discarded, so the "driver" modules could not
load (were missing some needed symbols).

See fcd40d69afea8700eda8cbf381f6d160cf757ebf
and b4d3ba3346f092b9185da991414775281ceacaac.
I'm sure that there have been others that were similar.
These are just the most recent two patches in that category.
> 
> May I humbly beg some guidance as to the best way to proceed?  My current code
> takes option 2 and builds its own module if CONFIG_ATA=m, but people are asking if
> that's really necessary.

I don't see a problem with that, but Michal should be able to speak about that.

> (v1 patch at https://lkml.org/lkml/2014/5/9/663 if anyone wants more details.)
> 
> The follow-on question is how to integrate the self-test code.  I had defined
> a preprocessor symbol which allowed a standalone "gcc -DUNITTEST glob.c" to
> make a user-space test driver, but I've been asked to kernelize it.  If using
> the full module route, I can just use __init code for it, like lib/list_sort.c.
> But for other options, its less clear.



-- 
~Randy

[PATCH] Documentation/kbuild/makefiles.txt: Improve description of lib-* targets

[George Spelvin]

I had to ask on the mailing list, so save developers the bother of
answering the question again.  (Wanna-be kernel developers might also
be helped, but who cares about them?)

Signed-off-by: George Spelvin <linux@horizon.com>
---
As with most documentation patches, this is also an exercise in eliciting
corrections by proposing a wrong answer which will be jumped on.

 Documentation/kbuild/makefiles.txt | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)

diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
index d567a7cc55..f6cc266163 100644
--- a/Documentation/kbuild/makefiles.txt
+++ b/Documentation/kbuild/makefiles.txt
@@ -237,10 +237,29 @@ more details, with real examples.
 	be included in a library, lib.a.
 	All objects listed with lib-y are combined in a single
 	library for that directory.
+
+	The entire kernel is then linked against these libraries,
+	so the linker will include the code in the final kernel only
+	if it is referenced somewhere.  Thus, lib-* goals may be
+	designed with false positives.
+
 	Objects that are listed in obj-y and additionally listed in
 	lib-y will not be included in the library, since they will
 	be accessible anyway.
-	For consistency, objects listed in lib-m will be included in lib.a.
+
+	For consistency, objects listed in lib-m will be included
+	in lib.a, but this will probably not do what you want.	If a
+	function in lib.a is referenced only from a module, and not from
+	the main kernel image, the linker will not incorporate it into
+	the kernel image and later attempts to load the module will fail.
+
+	One solution to this issue are to declare the object boolean
+	and link it into the kernel with obj-y if it's depended on by
+	a module.  The object's code will be included in the main kernel
+	but unused until the module is loaded.
+
+	The other is to make the object a module itself, and use an
+	obj-m goal.  Many helper modules in lib are used this way.
 
 	Note that the same kbuild makefile may list files to be built-in
 	and to be part of a library. Therefore the same directory
@@ -252,7 +271,7 @@ more details, with real examples.
 
 	This will create a library lib.a based on delay.o. For kbuild to
 	actually recognize that there is a lib.a being built, the directory
-	shall be listed in libs-y.
+	must be listed in libs-y in its parent makefile.
 	See also "6.4 List directories to visit when descending".
 
 	Use of lib-y is normally restricted to lib/ and arch/*/lib.
-- 
1.9.2

Re: [PATCH] Documentation/kbuild/makefiles.txt: Improve description of lib-* targets

[Randy Dunlap]

On 05/10/2014 10:02 PM, George Spelvin wrote:
> I had to ask on the mailing list, so save developers the bother of
> answering the question again.  (Wanna-be kernel developers might also
> be helped, but who cares about them?)
> 
> Signed-off-by: George Spelvin <linux@horizon.com>
> ---
> As with most documentation patches, this is also an exercise in eliciting
> corrections by proposing a wrong answer which will be jumped on.

Hi Michal,

Any comments on this patch?

Thanks.

> 
>  Documentation/kbuild/makefiles.txt | 23 +++++++++++++++++++++--
>  1 file changed, 21 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
> index d567a7cc55..f6cc266163 100644
> --- a/Documentation/kbuild/makefiles.txt
> +++ b/Documentation/kbuild/makefiles.txt
> @@ -237,10 +237,29 @@ more details, with real examples.
>  	be included in a library, lib.a.
>  	All objects listed with lib-y are combined in a single
>  	library for that directory.
> +
> +	The entire kernel is then linked against these libraries,
> +	so the linker will include the code in the final kernel only
> +	if it is referenced somewhere.  Thus, lib-* goals may be
> +	designed with false positives.
> +
>  	Objects that are listed in obj-y and additionally listed in
>  	lib-y will not be included in the library, since they will
>  	be accessible anyway.
> -	For consistency, objects listed in lib-m will be included in lib.a.
> +
> +	For consistency, objects listed in lib-m will be included
> +	in lib.a, but this will probably not do what you want.	If a
> +	function in lib.a is referenced only from a module, and not from
> +	the main kernel image, the linker will not incorporate it into
> +	the kernel image and later attempts to load the module will fail.
> +
> +	One solution to this issue are to declare the object boolean
> +	and link it into the kernel with obj-y if it's depended on by
> +	a module.  The object's code will be included in the main kernel
> +	but unused until the module is loaded.
> +
> +	The other is to make the object a module itself, and use an
> +	obj-m goal.  Many helper modules in lib are used this way.
>  
>  	Note that the same kbuild makefile may list files to be built-in
>  	and to be part of a library. Therefore the same directory
> @@ -252,7 +271,7 @@ more details, with real examples.
>  
>  	This will create a library lib.a based on delay.o. For kbuild to
>  	actually recognize that there is a lib.a being built, the directory
> -	shall be listed in libs-y.
> +	must be listed in libs-y in its parent makefile.
>  	See also "6.4 List directories to visit when descending".
>  
>  	Use of lib-y is normally restricted to lib/ and arch/*/lib.
> 


-- 
~Randy

Re: [PATCH] Documentation/kbuild/makefiles.txt: Improve description of lib-* targets

[Sam Ravnborg]

On Wed, May 14, 2014 at 02:21:07PM -0700, Randy Dunlap wrote:
> On 05/10/2014 10:02 PM, George Spelvin wrote:
> > I had to ask on the mailing list, so save developers the bother of
> > answering the question again.  (Wanna-be kernel developers might also
> > be helped, but who cares about them?)
> > 
> > Signed-off-by: George Spelvin <linux@horizon.com>
> > ---
> > As with most documentation patches, this is also an exercise in eliciting
> > corrections by proposing a wrong answer which will be jumped on.
> 
> Hi Michal,
> 
> Any comments on this patch?
> 
> Thanks.
> 
> > 
> >  Documentation/kbuild/makefiles.txt | 23 +++++++++++++++++++++--
> >  1 file changed, 21 insertions(+), 2 deletions(-)
> > 
> > diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
> > index d567a7cc55..f6cc266163 100644
> > --- a/Documentation/kbuild/makefiles.txt
> > +++ b/Documentation/kbuild/makefiles.txt
> > @@ -237,10 +237,29 @@ more details, with real examples.
> >  	be included in a library, lib.a.
> >  	All objects listed with lib-y are combined in a single
> >  	library for that directory.
> > +
> > +	The entire kernel is then linked against these libraries,
> > +	so the linker will include the code in the final kernel only
> > +	if it is referenced somewhere.  Thus, lib-* goals may be
> > +	designed with false positives.
> > +
> >  	Objects that are listed in obj-y and additionally listed in
> >  	lib-y will not be included in the library, since they will
> >  	be accessible anyway.
> > -	For consistency, objects listed in lib-m will be included in lib.a.
> > +
> > +	For consistency, objects listed in lib-m will be included
> > +	in lib.a, but this will probably not do what you want.

So maybe we should catch this and error out?

	Sam

Re: [PATCH] Documentation/kbuild/makefiles.txt: Improve description of lib-* targets

[George Spelvin]

>> On 05/10/2014 10:02 PM, George Spelvin wrote:
>>> -	For consistency, objects listed in lib-m will be included in lib.a.
>>> +
>>> +	For consistency, objects listed in lib-m will be included
>>> +	in lib.a, but this will probably not do what you want.

Sam Ravneborg wrote:
> So maybe we should catch this and error out?

Given my current poor state of knowledge, that does indeed seem
like a better solution; I can't imagine a situation where the current
behavior is useful.

But can we not stall a straightforward and harmless documentation
upgrade waiting for a more ambitious solution?  A significant
change to kbuild like that is going to trigger a (beneficial, but
slow) wave of bug reports and Kconfig/Makefile fixes.

Re: [PATCH] Documentation/kbuild/makefiles.txt: Improve description of lib-* targets

[Michal Marek]

On 2014-05-16 12:14, Sam Ravnborg wrote:
> On Wed, May 14, 2014 at 02:21:07PM -0700, Randy Dunlap wrote:
>> On 05/10/2014 10:02 PM, George Spelvin wrote:
>>> I had to ask on the mailing list, so save developers the bother of
>>> answering the question again.  (Wanna-be kernel developers might also
>>> be helped, but who cares about them?)
>>>
>>> Signed-off-by: George Spelvin <linux@horizon.com>
>>> ---
>>> As with most documentation patches, this is also an exercise in eliciting
>>> corrections by proposing a wrong answer which will be jumped on.
>>
>> Hi Michal,
>>
>> Any comments on this patch?
>>
>> Thanks.
>>
>>>
>>>  Documentation/kbuild/makefiles.txt | 23 +++++++++++++++++++++--
>>>  1 file changed, 21 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
>>> index d567a7cc55..f6cc266163 100644
>>> --- a/Documentation/kbuild/makefiles.txt
>>> +++ b/Documentation/kbuild/makefiles.txt
>>> @@ -237,10 +237,29 @@ more details, with real examples.
>>>  	be included in a library, lib.a.
>>>  	All objects listed with lib-y are combined in a single
>>>  	library for that directory.
>>> +
>>> +	The entire kernel is then linked against these libraries,
>>> +	so the linker will include the code in the final kernel only
>>> +	if it is referenced somewhere.  Thus, lib-* goals may be
>>> +	designed with false positives.
>>> +
>>>  	Objects that are listed in obj-y and additionally listed in
>>>  	lib-y will not be included in the library, since they will
>>>  	be accessible anyway.
>>> -	For consistency, objects listed in lib-m will be included in lib.a.
>>> +
>>> +	For consistency, objects listed in lib-m will be included
>>> +	in lib.a, but this will probably not do what you want.
> 
> So maybe we should catch this and error out?

(sorry for the late reply)

That makes sense to me. Check if $(lib-m) is non-empty and throw an
error. George, do you feel like submitting a patch that does this, plus
changing the documentation to clearly state that lib-* is only meant for
built-in code?

Thanks,
Michal

Re: [PATCH] Documentation/kbuild/makefiles.txt: Improve description of lib-* targets

[George Spelvin]

Ping.  Michal, any comments?

On Wed, 14 May 2014, Randy Dunlap wrote:
> Hi Michal,
> 
> Any comments on this patch?
> 
> Thanks.


On 05/10/2014 10:02 PM, George Spelvin wrote:
> I had to ask on the mailing list, so save developers the bother of
> answering the question again.  (Wanna-be kernel developers might also
> be helped, but who cares about them?)
> 
> Signed-off-by: George Spelvin <linux@horizon.com>
> ---
> As with most documentation patches, this is also an exercise in eliciting
> corrections by proposing a wrong answer which will be jumped on.
> 
>  Documentation/kbuild/makefiles.txt | 23 +++++++++++++++++++++--
>  1 file changed, 21 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
> index d567a7cc55..f6cc266163 100644
> --- a/Documentation/kbuild/makefiles.txt
> +++ b/Documentation/kbuild/makefiles.txt
> @@ -237,10 +237,29 @@ more details, with real examples.
>  	be included in a library, lib.a.
>  	All objects listed with lib-y are combined in a single
>  	library for that directory.
> +
> +	The entire kernel is then linked against these libraries,
> +	so the linker will include the code in the final kernel only
> +	if it is referenced somewhere.  Thus, lib-* goals may be
> +	designed with false positives.
> +
>  	Objects that are listed in obj-y and additionally listed in
>  	lib-y will not be included in the library, since they will
>  	be accessible anyway.
> -	For consistency, objects listed in lib-m will be included in lib.a.
> +
> +	For consistency, objects listed in lib-m will be included
> +	in lib.a, but this will probably not do what you want.	If a
> +	function in lib.a is referenced only from a module, and not from
> +	the main kernel image, the linker will not incorporate it into
> +	the kernel image and later attempts to load the module will fail.
> +
> +	One solution to this issue are to declare the object boolean
> +	and link it into the kernel with obj-y if it's depended on by
> +	a module.  The object's code will be included in the main kernel
> +	but unused until the module is loaded.
> +
> +	The other is to make the object a module itself, and use an
> +	obj-m goal.  Many helper modules in lib are used this way.
>  
>  	Note that the same kbuild makefile may list files to be built-in
>  	and to be part of a library. Therefore the same directory
> @@ -252,7 +271,7 @@ more details, with real examples.
>  
>  	This will create a library lib.a based on delay.o. For kbuild to
>  	actually recognize that there is a lib.a being built, the directory
> -	shall be listed in libs-y.
> +	must be listed in libs-y in its parent makefile.
>  	See also "6.4 List directories to visit when descending".
>  
>  	Use of lib-y is normally restricted to lib/ and arch/*/lib.
> 

Re: Searching for insight into lib-y and lib-m

[Tejun Heo]

On Sat, May 10, 2014 at 09:28:33PM -0700, Randy Dunlap wrote:
> > Documentation/kbuild/makefiles.txt suggests that CONFIG_GLOB=m would
> > be synonymous with CONFIG_GLOB=y in this case, so the net effect would
> > be identical to case 3.  Am I right?
> > 
> 
> 'git log lib/Makefile' shows me patches that I have submitted where
> kernel modules were built as loadable modules (=m) and library code was
> built but then apparently discarded, so the "driver" modules could not
> load (were missing some needed symbols).

Shouldn't we fix that?  At least for in-kernel modules?

> See fcd40d69afea8700eda8cbf381f6d160cf757ebf
> and b4d3ba3346f092b9185da991414775281ceacaac.
> I'm sure that there have been others that were similar.
> These are just the most recent two patches in that category.
> > 
> > May I humbly beg some guidance as to the best way to proceed?  My current code
> > takes option 2 and builds its own module if CONFIG_ATA=m, but people are asking if
> > that's really necessary.
> 
> I don't see a problem with that, but Michal should be able to speak about that.

It's more cumbersome and involved.  It's kinda sad to have to do this
because our lib-* can't handle in-kernel modules.

Thanks.

-- 
tejun

Re: Searching for insight into lib-y and lib-m

[Sam Ravnborg]

Hi George.

> I'm working on a patch to factor out some useful helper code
> (specifically, the glob_match() function) from drivers/ata/libata-core.c
> to a separate file lib/glob.c, so other drivers can use it.  But I'm
> having a hard time figuring out how to arrange for every caller to be
> able to call it.
> 
> In particular, suppose that CONFIG_ATA=m.  I'm trying to figure out
> what the various options I have are for handling my new option CONFIG_GLOB.
> 
> Here are the four basic possibilities (all combined with "select GLOB"
> by ATA):
> 
> 1) bool GLOB; obj-$(CONFIG_GLOB) += glob.o
> 
> I think this will atatically link the glob.o code into the kernel if
> CONFIG_ATA=y or CONFIG_ATA=m.  So everything will work, but there's no
> way for an out-of-tree driver to depend on the code if CONFIG_ATA=n.
> (Not supporting that case for now is definitely an option.)A
This is the typical way to solve this problem.
If you prepare your kernel for using a module that uses glob.c then
you pull in the required function in lib.
If out-of-tree modles has problems with this solution they
can fix this by becoming in-tree modules (via staging for example).

> 2) tristate GLOB; obj-$(CONFIG_GLOB) += glob.o
> 
> In this case, if CONFIG_ATA=m, I'll create a separate module glob.ko.
> I'd need to include <linux/module.h> and add the appropriate module
> declarations to glob.c.  Lots of code in lib/ seems to do this, but a
> proliferation if microscopic modules is annoying
This is overkill for glob.c

> 3) bool GLOB; lib-$(CONFIG_GLOB) += glob.o
> 
> This is where I get confused.  Documentation/kbuild/makefiles.txt says
> that glob.o will be included in the library lib.a.
> 
> But it then doesn't say anything about what (if anything) is linked
> against lib.a.  I assume (although it's not actually stated) that vmlinux
> is linked against it, so the linker will import any called functions.
This feature is made to support generic implmentations where the
architecture can provide there own implmentation.
This is not for drivers to use - becasue this prevents modles from being
loaded as you also conclude.

> 4) tristate GLOB; lib-$(CONFIG_GLOB) += glob.o
> 
> Documentation/kbuild/makefiles.txt suggests that CONFIG_GLOB=m would
> be synonymous with CONFIG_GLOB=y in this case, so the net effect would
> be identical to case 3.  Am I right?
Right

> The follow-on question is how to integrate the self-test code.  I had defined
> a preprocessor symbol which allowed a standalone "gcc -DUNITTEST glob.c" to
> make a user-space test driver, but I've been asked to kernelize it.  If using
> the full module route, I can just use __init code for it, like lib/list_sort.c.
> But for other options, its less clear.
Take a look at tools/testing/selftest - this should be the best "kernel" way to do this.

	Sam