* no version magic, tainting kernel.
@ 2003-01-23 13:59 Thomas Schlichter
2003-01-23 16:29 ` Randy.Dunlap
` (2 more replies)
0 siblings, 3 replies; 47+ messages in thread
From: Thomas Schlichter @ 2003-01-23 13:59 UTC (permalink / raw)
To: LKML, Rusty Russell
Hi,
I have writen a small kernel module and it works perfectly, but currently (I
think it just begun with 2.5.59) I get the warning above when the module is
inserted. Now I am just interested what I have to change so this message
won't appear anymore...
Thank you much!
Thomas Schlichter
P.S.: If my Makefile or source will help I'll give it to you...
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: no version magic, tainting kernel. 2003-01-23 13:59 no version magic, tainting kernel Thomas Schlichter @ 2003-01-23 16:29 ` Randy.Dunlap 2003-01-23 16:52 ` Sam Ravnborg 2003-01-27 18:52 ` Jerry Cooperstein 2 siblings, 0 replies; 47+ messages in thread From: Randy.Dunlap @ 2003-01-23 16:29 UTC (permalink / raw) To: Thomas Schlichter; +Cc: LKML, Rusty Russell On Thu, 23 Jan 2003, Thomas Schlichter wrote: | I have writen a small kernel module and it works perfectly, but currently (I | think it just begun with 2.5.59) I get the warning above when the module is | inserted. Now I am just interested what I have to change so this message | won't appear anymore... | | P.S.: If my Makefile or source will help I'll give it to you... | - Did you rebuild the module with a 2.5.59 Makefile? Yes, it's a 2.5.59 change according to the Changelog at kernel.org: <QUOTE> <kai@tp1.ruhr-uni-bochum.de> Module Sanity Check This patch, based on Rusty's implementation, adds a special section to vmlinux and all modules, which contain the kernel version string, values of some particularly important config options (SMP,preempt,proc family) and the gcc version. When inserting a module, the version string is checked against the kernel version string and loading is rejected if they don't match. The version string is actually added to the modules during the final .ko generation, so that a changed version string does only cause relinking, not recompilation, which is a major performance improvement over the old 2.4 way of doing things. </QUOTE> -- ~Randy ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-23 13:59 no version magic, tainting kernel Thomas Schlichter 2003-01-23 16:29 ` Randy.Dunlap @ 2003-01-23 16:52 ` Sam Ravnborg 2003-01-23 17:32 ` Thomas Schlichter 2003-01-27 18:52 ` Jerry Cooperstein 2 siblings, 1 reply; 47+ messages in thread From: Sam Ravnborg @ 2003-01-23 16:52 UTC (permalink / raw) To: Thomas Schlichter; +Cc: LKML, Rusty Russell On Thu, Jan 23, 2003 at 02:59:22PM +0100, Thomas Schlichter wrote: > Hi, > > I have writen a small kernel module and it works perfectly, but currently (I > think it just begun with 2.5.59) I get the warning above when the module is > inserted. Now I am just interested what I have to change so this message > won't appear anymore... > > Thank you much! > > Thomas Schlichter > > P.S.: If my Makefile or source will help I'll give it to you... What command did you use to build your module? If you did no use: make -C path/to/kernel/src SUBDIRS=$PWD modules chances are big you did not compile the module correct. This requires the Makefile to look like any other kernel (kbuild) makefile. Sam ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-23 16:52 ` Sam Ravnborg @ 2003-01-23 17:32 ` Thomas Schlichter 2003-01-23 18:22 ` Sam Ravnborg 0 siblings, 1 reply; 47+ messages in thread From: Thomas Schlichter @ 2003-01-23 17:32 UTC (permalink / raw) To: Randy.Dunlap, Sam Ravnborg; +Cc: LKML, Rusty Russell [-- Attachment #1: Type: text/plain, Size: 1759 bytes --] Thanks for your answers! I did not compile my module with a kernel Makefile, I used the very small and simple one attatched to this mail. So it seems I miss something when the module is linked and I have to know what I have to link to the module or which header-file I have to include... For me it seems link I have to link the init/vermagic.c file to my module, but how would this be possible if only the kernel includes were available?? I think only these should be needed to compile a module... Thomas Schlichter Am Donnerstag, 23. Januar 2003 17:29 schrieb Randy.Dunlap: > Did you rebuild the module with a 2.5.59 Makefile? > > > Yes, it's a 2.5.59 change according to the Changelog at kernel.org: > > <QUOTE> > <kai@tp1.ruhr-uni-bochum.de> > Module Sanity Check > > This patch, based on Rusty's implementation, > adds a special section to vmlinux and all modules, which > contain the kernel version string, values of some > particularly important config options (SMP,preempt,proc > family) and the gcc version. > > When inserting a module, the version string is checked against the > kernel version string and loading is rejected if they don't match. > > The version string is actually added to the modules during the final > .ko generation, so that a changed version string does only cause relinking, > not recompilation, which is a major performance improvement over the old > 2.4 way of doing things. > </QUOTE> Am Donnerstag, 23. Januar 2003 17:52 schrieb Sam Ravnborg: > What command did you use to build your module? > If you did no use: > make -C path/to/kernel/src SUBDIRS=$PWD modules > > chances are big you did not compile the module correct. > This requires the Makefile to look like any other kernel (kbuild) makefile. > > Sam [-- Attachment #2: Makefile --] [-- Type: text/x-makefile, Size: 384 bytes --] # get current release for include path RELEASE = $(shell uname -r) # set compile flags, defines and include directory CFLAGS = -Wall -O2 -fno-common INCLUDES = -I/lib/modules/$(RELEASE)/build/include DEFINES = -D__KERNEL__ -DMODULE -DKBUILD_MODNAME="tlbstat" all: tlbstat.o tlbstat.o: tlbstat.c gcc -c $(CFLAGS) $(INCLUDES) $(DEFINES) $< clean: rm -f tlbstat.o *~ core ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-23 17:32 ` Thomas Schlichter @ 2003-01-23 18:22 ` Sam Ravnborg 2003-01-23 19:35 ` Mark Fasheh 0 siblings, 1 reply; 47+ messages in thread From: Sam Ravnborg @ 2003-01-23 18:22 UTC (permalink / raw) To: Thomas Schlichter; +Cc: Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Thu, Jan 23, 2003 at 06:32:59PM +0100, Thomas Schlichter wrote: > Thanks for your answers! > > I did not compile my module with a kernel Makefile, I used the very small and > simple one attatched to this mail. So it seems I miss something when the > module is linked and I have to know what I have to link to the module or > which header-file I have to include... > > For me it seems link I have to link the init/vermagic.c file to my module, but > how would this be possible if only the kernel includes were available?? > I think only these should be needed to compile a module... As it is today the only sane way is to have the full kernel src available. It should be possible to minimize that - but I do not feel tempted to do so. We want to do too much to rely on whatever makefile people write for their module. Sam ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-23 18:22 ` Sam Ravnborg @ 2003-01-23 19:35 ` Mark Fasheh 2003-01-26 13:29 ` Christian Zander ` (2 more replies) 0 siblings, 3 replies; 47+ messages in thread From: Mark Fasheh @ 2003-01-23 19:35 UTC (permalink / raw) To: Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell Can't the stuff in init/vermagic.c be moved into a header file? Maybe vermagic.h? Most of the code can be cut 'n pasted right out of vermagic.c and the bit that defines "const char vermagic[]..." could be placed inside a macro which modules would then stick in the bottom of one of their c files. This is what I'm getting at (warning I haven't checked this code or even tried to clean it up): in vermagic.h: #include <linux/version.h> #include <linux/module.h> /* Simply sanity version stamp for modules. */ #ifdef CONFIG_SMP #define MODULE_VERMAGIC_SMP "SMP " #else #define MODULE_VERMAGIC_SMP "" #endif #ifdef CONFIG_PREEMPT #define MODULE_VERMAGIC_PREEMPT "preempt " #else #define MODULE_VERMAGIC_PREEMPT "" #endif #ifndef MODULE_ARCH_VERMAGIC #define MODULE_ARCH_VERMAGIC "" #endif #define KERNEL_VERSIONMAGIC const char vermagic[] \ __attribute__((section("__vermagic"))) = \ UTS_RELEASE " " \ MODULE_VERMAGIC_SMP MODULE_VERMAGIC_PREEMPT MODULE_ARCH_VERMAGIC \ "gcc-" __stringify(__GNUC__) "." __stringify(__GNUC_MINOR__) in my_external_module.c, and init/vermagic.c I'd just do: #include <linux/vermagic.h> KERNEL_VERSIONMAGIC(); and be done with it. Modules that ship with the kernel wouldn't have to change a thing (and still be linked against vermagic.c, and it'd only add two lines of code to everyones externally shipped modules. I don't really think that this would be "doing too much" for those whose modules are included in Linus's kernel, and it wouldn't require people to keep entire source tree's around to compile a module or two... Am I totally missing something here or wouldn't this solve our problem? Someone please correct me if I'm wrong :) --Mark On Thu, Jan 23, 2003 at 07:22:36PM +0100, Sam Ravnborg wrote: > As it is today the only sane way is to have the full kernel src available. > It should be possible to minimize that - but I do not feel tempted to > do so. > > We want to do too much to rely on whatever makefile people write for > their module. > > Sam > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- Mark Fasheh Software Developer, Oracle Corp mark.fasheh@oracle.com ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-23 19:35 ` Mark Fasheh @ 2003-01-26 13:29 ` Christian Zander 2003-01-26 13:33 ` Keith Owens 2003-01-26 17:51 ` Kai Germaschewski 2003-01-26 17:43 ` Kai Germaschewski 2003-01-28 1:58 ` Rusty Russell 2 siblings, 2 replies; 47+ messages in thread From: Christian Zander @ 2003-01-26 13:29 UTC (permalink / raw) To: Mark Fasheh Cc: Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Thu, Jan 23, 2003 at 11:35:40AM -0800, Mark Fasheh wrote: > > Can't the stuff in init/vermagic.c be moved into a header file? > Maybe vermagic.h? Most of the code can be cut 'n pasted right out of > vermagic.c and the bit that defines "const char vermagic[]..." could > be placed inside a macro which modules would then stick in the > bottom of one of their c files. This is what I'm getting at > (warning I haven't checked this code or even tried to clean it up): > > in vermagic.h: > #include <linux/version.h> > #include <linux/module.h> > > /* Simply sanity version stamp for modules. */ > #ifdef CONFIG_SMP > #define MODULE_VERMAGIC_SMP "SMP " > #else > #define MODULE_VERMAGIC_SMP "" > #endif > #ifdef CONFIG_PREEMPT > #define MODULE_VERMAGIC_PREEMPT "preempt " > #else > #define MODULE_VERMAGIC_PREEMPT "" > #endif > #ifndef MODULE_ARCH_VERMAGIC > #define MODULE_ARCH_VERMAGIC "" > #endif > > #define KERNEL_VERSIONMAGIC const char vermagic[] \ > __attribute__((section("__vermagic"))) = \ > UTS_RELEASE " " \ > MODULE_VERMAGIC_SMP MODULE_VERMAGIC_PREEMPT MODULE_ARCH_VERMAGIC \ > "gcc-" __stringify(__GNUC__) "." __stringify(__GNUC_MINOR__) > > > in my_external_module.c, and init/vermagic.c I'd just do: > #include <linux/vermagic.h> > KERNEL_VERSIONMAGIC(); > > and be done with it. > > Modules that ship with the kernel wouldn't have to change a thing > (and still be linked against vermagic.c, and it'd only add two lines > of code to everyones externally shipped modules. I don't really > think that this would be "doing too much" for those whose modules > are included in Linus's kernel, and it wouldn't require people to > keep entire source tree's around to compile a module or two... Am I > totally missing something here or wouldn't this solve our problem? > Someone please correct me if I'm wrong :) > I wouldn't go as far as saying that the current solution imposes an onerous requirement on any user wishing to build third-party modules by relying on a complete and properly configured kernel source tree, but it certainly seems like a step backwards from Linux 2.4, where properly configured kernel headers are sufficient. Of course, this only holds true for external projects using kbuild to build the modules; other build systems would not only require that a complete, configured kernel source tree be installed, they would also rely on that source tree to be uncleaned since they have no knowledge of how init/vermagic.o is to be built (and shouldn't make assumptions, not considering possible legal/licensing implications). This I really do consider an unnecessary, burdensome prerequisite. Since I'm not intimately familiar with kbuild, I'm also wondering if this scenario would be a problem: Somebody downloaded Linux 2.5.59, configured it and built it using a pre-3.0 version of gcc, e.g. gcc 2.95. The user had plenty of disk space and decided, based on past experiences, that leaving the source tree uncleaned is least likely to cause problems, should he/she ever be intersted in building third-party modules. For some time, the user placed no interest in external modules and used his/her system quite happily; with the release of a new, improved version of a driver, the user decided that he wants to give it a try, however, and built the driver module, which picked up init/vermagic.o; our imaginary user is using a distribution that provides frequent updates and he/she makes regular use of this service - it just so happens that one of these updates installed gcc 3.0 as the new default compiler. The new module is thus built using gcc 3.0, but init/vermagic.o still indicates gcc 2.95; the module loader will erroneously believe everything is fine. I agree with Mark that moving the code in init/vermagic.c into a new header file like linux/vermagic.h is a better solution. -- christian zander zander@minion.de ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 13:29 ` Christian Zander @ 2003-01-26 13:33 ` Keith Owens 2003-01-26 18:02 ` Kai Germaschewski 2003-01-26 17:51 ` Kai Germaschewski 1 sibling, 1 reply; 47+ messages in thread From: Keith Owens @ 2003-01-26 13:33 UTC (permalink / raw) To: LKML On Sun, 26 Jan 2003 14:29:23 +0100, Christian Zander <zander@minion.de> wrote: >The new module >is thus built using gcc 3.0, but init/vermagic.o still indicates gcc >2.95; the module loader will erroneously believe everything is fine. Congratulations, you have put your finger on a major design flaw in modversions that has been there since 2.0 kernel days. The modversion data is generated once and everything else blindly uses it, with _NO_ checks on whether it is still valid or not. Rusty knows damn well that this is broken, but appears to be ignoring that fact (Rusty, see my mail to you and Alan Cox on Wed, 24 Oct 2001 14:14:18 +1000). See http://unc.dl.sourceforge.net/sourceforge/kbuild/kbuild-2.5-history.tar.bz2, in particular makefile-2.5_whereto.html. modversions relies on assumptions that are not valid and therefore modversions are broken, as noted by Christian Zander and others. To do module versioning right needs global information about this definitions and usage of exported symbols. It was one of the reasons that kbuild 2.5 had a global database, in order to track symbol usage and handle ABI versioning correctly. ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 13:33 ` Keith Owens @ 2003-01-26 18:02 ` Kai Germaschewski 0 siblings, 0 replies; 47+ messages in thread From: Kai Germaschewski @ 2003-01-26 18:02 UTC (permalink / raw) To: Keith Owens; +Cc: LKML On Mon, 27 Jan 2003, Keith Owens wrote: > Christian Zander <zander@minion.de> wrote: > >The new module > >is thus built using gcc 3.0, but init/vermagic.o still indicates gcc > >2.95; the module loader will erroneously believe everything is fine. > > Congratulations, you have put your finger on a major design flaw in > modversions that has been there since 2.0 kernel days. The modversion > data is generated once and everything else blindly uses it, with _NO_ > checks on whether it is still valid or not. Rusty knows damn well that > this is broken, but appears to be ignoring that fact (Rusty, see my > mail to you and Alan Cox on Wed, 24 Oct 2001 14:14:18 +1000). First of all, "modversions" has been used to designate the process of module symbols versioning in the past years, which the above is not. The version magic string is supposed to be a simple check against obvious errors like kernel version mismatch between kernel and module and also to protect against incompatibilities which the ABI checksums (which modversions is about) cannot detect, like incompatible compiler versions. As such, the information is known beforehand and now complicated bookkeeping needed here. (Also, as I just pointed out in a another mail, the above scenario is not possible). Now your comments seem to rather apply to the actual module symbol versioning process, which I just posted a patch for. Things are handled very much differently now, if you have technical comments on that patch, they'd be much appreciated. --Kai ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 13:29 ` Christian Zander 2003-01-26 13:33 ` Keith Owens @ 2003-01-26 17:51 ` Kai Germaschewski 2003-01-26 21:57 ` Christian Zander 1 sibling, 1 reply; 47+ messages in thread From: Kai Germaschewski @ 2003-01-26 17:51 UTC (permalink / raw) To: Christian Zander Cc: Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, 26 Jan 2003, Christian Zander wrote: > Of course, this only holds true for external projects using kbuild to > build the modules; other build systems would not only require that a > complete, configured kernel source tree be installed, they would also > rely on that source tree to be uncleaned since they have no knowledge > of how init/vermagic.o is to be built (and shouldn't make assumptions, > not considering possible legal/licensing implications). This I really > do consider an unnecessary, burdensome prerequisite. Well, what I'm trying to say is that external build system will always break one way or other. Since they're external, they're naturally out of reach for me to influence, so there's really nothing I can do it but telling people to using the internal system instead. > Somebody downloaded Linux 2.5.59, configured it and built it using a > pre-3.0 version of gcc, e.g. gcc 2.95. The user had plenty of disk > space and decided, based on past experiences, that leaving the source > tree uncleaned is least likely to cause problems, should he/she ever > be intersted in building third-party modules. For some time, the user > placed no interest in external modules and used his/her system quite > happily; with the release of a new, improved version of a driver, the > user decided that he wants to give it a try, however, and built the > driver module, which picked up init/vermagic.o; our imaginary user is > using a distribution that provides frequent updates and he/she makes > regular use of this service - it just so happens that one of these > updates installed gcc 3.0 as the new default compiler. The new module > is thus built using gcc 3.0, but init/vermagic.o still indicates gcc > 2.95; the module loader will erroneously believe everything is fine. Again, when you're using your own external build system, it's up to you to mess it up in all possible ways, the above being one of them. When you're using the kernel build, the above cannot happen, since the kernel build system knows about this dependency and builds a new init/vermagic.o with the correct information before it gets linked into the external module. --Kai ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 17:51 ` Kai Germaschewski @ 2003-01-26 21:57 ` Christian Zander 2003-01-26 21:46 ` Kai Germaschewski 2003-01-26 22:23 ` Christian Zander 0 siblings, 2 replies; 47+ messages in thread From: Christian Zander @ 2003-01-26 21:57 UTC (permalink / raw) To: Kai Germaschewski Cc: Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, Jan 26, 2003 at 11:51:22AM -0600, Kai Germaschewski wrote: > > Well, what I'm trying to say is that external build system will > always break one way or other. Since they're external, they're > naturally out of reach for me to influence, so there's really > nothing I can do it but telling people to using the internal system > instead. > External build systems have been working just fine even after kbuild was adopted as the build system of choice for the Linux kernel, there should be more convincing reasons for deliberately breaking them now. > Again, when you're using your own external build system, it's up to > you to mess it up in all possible ways, the above being one of them. > > When you're using the kernel build, the above cannot happen, since > the kernel build system knows about this dependency and builds a new > init/vermagic.o with the correct information before it gets linked > into the external module. > Is that so? Does kbuild determine that vermagic.o needs rebuilding if the compiler version changed? -- christian zander zander@minion.de ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 21:57 ` Christian Zander @ 2003-01-26 21:46 ` Kai Germaschewski 2003-01-26 23:12 ` Christian Zander ` (2 more replies) 2003-01-26 22:23 ` Christian Zander 1 sibling, 3 replies; 47+ messages in thread From: Kai Germaschewski @ 2003-01-26 21:46 UTC (permalink / raw) To: Christian Zander Cc: Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, 26 Jan 2003, Christian Zander wrote: > On Sun, Jan 26, 2003 at 11:51:22AM -0600, Kai Germaschewski wrote: > > > > Well, what I'm trying to say is that external build system will > > always break one way or other. Since they're external, they're > > naturally out of reach for me to influence, so there's really > > nothing I can do it but telling people to using the internal system > > instead. > > > > External build systems have been working just fine even after kbuild > was adopted as the build system of choice for the Linux kernel, there > should be more convincing reasons for deliberately breaking them now. That's not true. For example, how would an old external build system magically starting to compile modules as .ko without updating? How would it have added -DKBUILD_BASENAME and -DKBUILD_MODNAME, which are required by the new module code. And, how did they avoid subtle breakage like not giving the same switches on the command line? (This list goes on...) Also, it's not true that they've been broken deliberately. As work progresses, breakage occurs, that's just a fact of live. However, introduction of __vermagic was not introduced in order to make live for maintainers of external modules harder, it was introduced since loading modules compiled with gcc3 into a kernel compiled with gcc2 caused crashes for people. > > When you're using the kernel build, the above cannot happen, since > > the kernel build system knows about this dependency and builds a new > > init/vermagic.o with the correct information before it gets linked > > into the external module. > > Is that so? Does kbuild determine that vermagic.o needs rebuilding if > the compiler version changed? Okay, you have a point here, there's still a bug. vermagic.o will be rebuilt when the version changes or any of the recorded config options change, but it doesn't pick up changes in the compiler version, if the new gcc has the same name. That's a bug for internal use as well, the patch below fixes it. Two more things: o You say kbuild has been adopted as the build system of choice. That's misleading, the new build system for 2.5, called kbuild-2.5 has never been adopted. It's just that the existing build system has been improved quite a bit by me and others. o One thing I do not understand at all: What is the problem with using the internal build system? It makes maintainance of external modules much easier than keeping track of what happens in the kernel and patching a private solution all the time. I don't even see any license issues, first of all you don't even distribute it, the user who's building the module will already have it along with his kernel source. And if you're using it to compile (possibly binary) modules you want to distribute, you can just use it just like gcc without any further obligations, so no problem there either. (IANAL, of course) --Kai ===== init/Makefile 1.18 vs edited ===== --- 1.18/init/Makefile Tue Jan 14 14:29:02 2003 +++ edited/init/Makefile Sun Jan 26 15:24:21 2003 @@ -12,6 +12,13 @@ $(obj)/version.o: include/linux/compile.h +# vermagic doesn't actually include compile.h, but it records +# the compiler version. To make sure we notice when the compiler +# changes, add a dependency on compile.h, which changes when the +# compiler changes. + +$(obj)/vermagic.o: include/linux/compile.h + # compile.h changes depending on hostname, generation number, etc, # so we regenerate it always. # mkcompile_h will make sure to only update the ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 21:46 ` Kai Germaschewski @ 2003-01-26 23:12 ` Christian Zander 2003-01-26 22:55 ` David Woodhouse 2003-01-27 6:17 ` Petr Vandrovec 2003-01-27 17:59 ` Joel Becker 2 siblings, 1 reply; 47+ messages in thread From: Christian Zander @ 2003-01-26 23:12 UTC (permalink / raw) To: Kai Germaschewski Cc: Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, Jan 26, 2003 at 03:46:30PM -0600, Kai Germaschewski wrote: > > That's not true. For example, how would an old external build system > magically starting to compile modules as .ko without updating? How > would it have added -DKBUILD_BASENAME and -DKBUILD_MODNAME, which > are required by the new module code. And, how did they avoid subtle > breakage like not giving the same switches on the command line? > (This list goes on...) > I hear you, but these changes were easy enough to adapt to. > Also, it's not true that they've been broken deliberately. As work > progresses, breakage occurs, that's just a fact of live. However, > introduction of __vermagic was not introduced in order to make live > for maintainers of external modules harder, it was introduced since > loading modules compiled with gcc3 into a kernel compiled with gcc2 > caused crashes for people. > Well, in this specific case an alternate solution was proposed that would have solved any of the potential problems pointed out. > Okay, you have a point here, there's still a bug. vermagic.o will be > rebuilt when the version changes or any of the recorded config > options change, but it doesn't pick up changes in the compiler > version, if the new gcc has the same name. > > That's a bug for internal use as well, the patch below fixes it. > Fair enough. > o One thing I do not understand at all: What is the problem with > using the internal build system? It makes maintainance of external > modules much easier than keeping track of what happens in the kernel > and patching a private solution all the time. > My primary concern is compatibility with those kernels that do not use kbuild or a different version of it. Ideally, one would want to use the same build system for all possible kernel versions rather than use Makefiles that attempt to pick the best choice. I guess I'm convinced that the latter is the "best" solution to dealing with this problem at this point, and I can live with that. What's the most reliable way to tell if kbuild is available, and what differences among kbuild versions will one have to look out for? > I don't even see any license issues, first of all you don't even > distribute it, the user who's building the module will already > have it along with his kernel source. And if you're using it to > compile (possibly binary) modules you want to distribute, you can > just use it just like gcc without any further obligations, so no > problem there either. (IANAL, of course) > I don't see any problems with kbuild, I was referring to vermagic.c. -- christian zander zander@minion.de ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 23:12 ` Christian Zander @ 2003-01-26 22:55 ` David Woodhouse 2003-01-27 0:07 ` Christian Zander 0 siblings, 1 reply; 47+ messages in thread From: David Woodhouse @ 2003-01-26 22:55 UTC (permalink / raw) To: Christian Zander Cc: Kai Germaschewski, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell zander@minion.de said: > The problem isn't necessarily lack of flexibility, but the lack of > unity across kernel versions. I agree that kbuild is the preferable > solution for Linux 2.5, but it isn't for all incarnations of Linux 2.4 > and definetely not for Linux 2.2. /me blinks... what's wrong with 2.2? Looks fine to me... imladris /home/dwmw2/working/mtd/drivers/mtd $ make LINUXDIR=/inst/linux/linux-2.2 make -C /inst/linux/linux-2.2 SUBDIRS=`pwd` modules make[1]: Entering directory `/inst/linux/linux-2.2' make -C /home/dwmw2/working/mtd/drivers/mtd CFLAGS="-Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DMODULE" MAKING_MODULES=1 modules make[2]: Entering directory `/home/dwmw2/working/mtd/drivers/mtd' gcc296 -D__KERNEL__ -I/inst/linux/linux-2.2/include -I/home/dwmw2/working/mtd/drivers/mtd/../../include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DMODULE -DEXPORT_SYMTAB -c afs.c {standard input}: Assembler messages: {standard input}:9: Warning: ignoring changed section attributes for .modinfo gcc296 -D__KERNEL__ -I/inst/linux/linux-2.2/include -I/home/dwmw2/working/mtd/drivers/mtd/../../include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DMODULE -DEXPORT_SYMTAB -c mtdcore.cmake[2]: *** Deleting file `mtdcore.o' make[2]: *** [mtdcore.o] Interrupt make[1]: *** [_mod_/home/dwmw2/working/mtd/drivers/mtd] Interrupt make: *** [modules] Interrupt -- dwmw2 ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 22:55 ` David Woodhouse @ 2003-01-27 0:07 ` Christian Zander 2003-01-26 23:16 ` David Woodhouse 0 siblings, 1 reply; 47+ messages in thread From: Christian Zander @ 2003-01-27 0:07 UTC (permalink / raw) To: David Woodhouse Cc: Christian Zander, Kai Germaschewski, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, Jan 26, 2003 at 10:55:49PM +0000, David Woodhouse wrote: > > /me blinks... what's wrong with 2.2? Looks fine to me... > > imladris /home/dwmw2/working/mtd/drivers/mtd $ make LINUXDIR=/inst/linux/linux-2.2 > make -C /inst/linux/linux-2.2 SUBDIRS=`pwd` modules > make[1]: Entering directory `/inst/linux/linux-2.2' > make -C /home/dwmw2/working/mtd/drivers/mtd CFLAGS="-Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DMODULE" > MAKING_MODULES=1 modules > I apologize if I have argued based on false assumptions; is it true then that a Makefile written for use with Linux 2.5 kbuild will work unchanged with any Linux 2.2/2.5 kernel (w/ custom CFLAGS, ...)? -- christian zander zander@minion.de ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-27 0:07 ` Christian Zander @ 2003-01-26 23:16 ` David Woodhouse 2003-01-27 0:24 ` Christian Zander 2003-01-27 16:25 ` Kai Germaschewski 0 siblings, 2 replies; 47+ messages in thread From: David Woodhouse @ 2003-01-26 23:16 UTC (permalink / raw) To: Christian Zander Cc: Kai Germaschewski, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell zander@minion.de said: > I apologize if I have argued based on false assumptions; is it true > then that a Makefile written for use with Linux 2.5 kbuild will work > unchanged with any Linux 2.2/2.5 kernel (w/ custom CFLAGS, ...)? There are some differences, but nothing insurmountable. For <=2.4 you have to include Rules.make at the end of your own Makefile, and for 2.5 that file doesn't exist any more. For older kernels you must set O_TARGET, for 2.5 I think the mere act of setting it causes the build to break -- that one is gratuitously making it harder to make external modules which compile in both and I've complained about it before. Per-file CFLAGS must have a full path specified now in 2.5, whereas in 2.4 and earlier it was just 'CFLAGS_filename.o'. -- dwmw2 ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 23:16 ` David Woodhouse @ 2003-01-27 0:24 ` Christian Zander 2003-01-27 16:25 ` Kai Germaschewski 1 sibling, 0 replies; 47+ messages in thread From: Christian Zander @ 2003-01-27 0:24 UTC (permalink / raw) To: David Woodhouse Cc: Christian Zander, Kai Germaschewski, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, Jan 26, 2003 at 11:16:11PM +0000, David Woodhouse wrote: > > There are some differences, but nothing insurmountable. > > For <=2.4 you have to include Rules.make at the end of your own > Makefile, and for 2.5 that file doesn't exist any more. > > For older kernels you must set O_TARGET, for 2.5 I think the mere > act of setting it causes the build to break -- that one is > gratuitously making it harder to make external modules which compile > in both and I've complained about it before. > > Per-file CFLAGS must have a full path specified now in 2.5, whereas > in 2.4 and earlier it was just 'CFLAGS_filename.o'. > Thanks for pointing that out, I'll look into this more closely. If it is as simple as you say, I'll be inclined to share your sentiment towards custom build systems interfacing with the Linux kernel. -- christian zander zander@minion.de ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 23:16 ` David Woodhouse 2003-01-27 0:24 ` Christian Zander @ 2003-01-27 16:25 ` Kai Germaschewski 2003-01-27 16:29 ` David Woodhouse 1 sibling, 1 reply; 47+ messages in thread From: Kai Germaschewski @ 2003-01-27 16:25 UTC (permalink / raw) To: David Woodhouse Cc: Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, 26 Jan 2003, David Woodhouse wrote: > For older kernels you must set O_TARGET, for 2.5 I think the mere act of > setting it causes the build to break -- that one is gratuitously making it > harder to make external modules which compile in both and I've complained > about it before. Okay, I might get persuaded to take that check out if it really makes life harder. Wouldn't it work to not have O_TARGET in the Makefile at all, I think 2.4 shouldn't care as long as you just "make modules"? > Per-file CFLAGS must have a full path specified now in 2.5, whereas in 2.4 > and earlier it was just 'CFLAGS_filename.o'. It shouldn't, I kinda deliberately decided to keep the old syntax for this. So it would be a bug. --Kai ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-27 16:25 ` Kai Germaschewski @ 2003-01-27 16:29 ` David Woodhouse 2003-01-27 16:39 ` Kai Germaschewski 0 siblings, 1 reply; 47+ messages in thread From: David Woodhouse @ 2003-01-27 16:29 UTC (permalink / raw) To: Kai Germaschewski Cc: Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell kai@tp1.ruhr-uni-bochum.de said: > Okay, I might get persuaded to take that check out if it really makes > life harder. Wouldn't it work to not have O_TARGET in the Makefile at > all, I think 2.4 shouldn't care as long as you just "make modules"? Er, I think O_TARGET is in fact the target _module_ name when building a file system as a module. Try removing O_TARGET from 2.4 fs/ext2/Makefile and building ext2 as a module. > It shouldn't, I kinda deliberately decided to keep the old syntax for > this. So it would be a bug. Ah, OK -- sorry. -- dwmw2 ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-27 16:29 ` David Woodhouse @ 2003-01-27 16:39 ` Kai Germaschewski 0 siblings, 0 replies; 47+ messages in thread From: Kai Germaschewski @ 2003-01-27 16:39 UTC (permalink / raw) To: David Woodhouse Cc: Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Mon, 27 Jan 2003, David Woodhouse wrote: > Er, I think O_TARGET is in fact the target _module_ name when building > a file system as a module. Try removing O_TARGET from 2.4 fs/ext2/Makefile > and building ext2 as a module. Well, okay, there's two ways you can build a single module in a directory in 2.4, one is to have O_TARGET := module.o obj-m := $(O_TARGET) obj-y := part1.o part2.o The other is O_TARGET := something.o obj-m := module.o module-objs := part1.o part2.o (plus a link rule for module.o) In 2.5, only the second way is legal, so if you're aiming for a compatible Makefile, you'd use that one, and then O_TARGET shouldn't matter for "make modules". --Kai ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 21:46 ` Kai Germaschewski 2003-01-26 23:12 ` Christian Zander @ 2003-01-27 6:17 ` Petr Vandrovec 2003-01-27 9:02 ` David Woodhouse 2003-01-27 17:59 ` Joel Becker 2 siblings, 1 reply; 47+ messages in thread From: Petr Vandrovec @ 2003-01-27 6:17 UTC (permalink / raw) To: Kai Germaschewski Cc: Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, Jan 26, 2003 at 03:46:30PM -0600, Kai Germaschewski wrote: > > o One thing I do not understand at all: What is the problem with using > the internal build system? It makes maintainance of external modules > much easier than keeping track of what happens in the kernel and > patching a private solution all the time. > > I don't even see any license issues, first of all you don't even > distribute it, the user who's building the module will already have it > along with his kernel source. And if you're using it to compile > (possibly binary) modules you want to distribute, you can just use it > just like gcc without any further obligations, so no problem there > either. (IANAL, of course) >From my exprience at VMware newsgroups distros have bad troubles even with delivery of basic configured kernel headers matching to kernel binaries they provide (it is not unusual that for example they go from 1GB to 4GB kernel without make mrproper so kmap/kunmap do not have proper versions attached :-( or they even sell headers and binaries with different configs). Trying to ask them to provide usable configured kernel sources (and not one which automagically replace -3 with -3custom so that modules built by internal build system do not match to kernel at all...) is not going to work - it did not work in the past, and I see no reason why it should start working now. And dialup users do not want to download 30MB package when 2MB headers were always enough (and especially if they'll use 1.5MB from the downloaded package to build 3rd party module). There are few exceptinons of distros which provide kernel sources in usable form (== without any user intervention /lib/modules/`uname -r`/build points to working correctly configured kernel tree), but usually their users build their own kernels anyway, so nobody profits from that :-( Petr Vandrovec vandrove@vc.cvut.cz ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-27 6:17 ` Petr Vandrovec @ 2003-01-27 9:02 ` David Woodhouse 2003-01-27 9:24 ` Petr Vandrovec 0 siblings, 1 reply; 47+ messages in thread From: David Woodhouse @ 2003-01-27 9:02 UTC (permalink / raw) To: Petr Vandrovec Cc: Kai Germaschewski, Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell vandrove@vc.cvut.cz said: > From my exprience at VMware newsgroups distros have bad troubles even > with delivery of basic configured kernel headers matching to kernel > binaries they provide (it is not unusual that for example they go from > 1GB to 4GB kernel without make mrproper so kmap/kunmap do not have > proper versions attached :-( or they even sell headers and binaries > with different configs). But you _need_ the config. Even with your own makefiles, how are you going to get it right for all new kernels that $CRAPDISTRO ships in a broken form, if you don't have the configs? If distros ship broken crap which doesn't let you build modules, there's really not a lot you can do except note their quality control for the record and report it in their bug tracking system. You are _always_ going to have problems with people shipping shite, or possibly even actively going our of their way to prevent out-of-tree modules building. It's fairly much an orthogonal problem though, isn't it? -- dwmw2 ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-27 9:02 ` David Woodhouse @ 2003-01-27 9:24 ` Petr Vandrovec 0 siblings, 0 replies; 47+ messages in thread From: Petr Vandrovec @ 2003-01-27 9:24 UTC (permalink / raw) To: David Woodhouse Cc: Kai Germaschewski, Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Mon, Jan 27, 2003 at 09:02:11AM +0000, David Woodhouse wrote: > > vandrove@vc.cvut.cz said: > > From my exprience at VMware newsgroups distros have bad troubles even > > with delivery of basic configured kernel headers matching to kernel > > binaries they provide (it is not unusual that for example they go from > > 1GB to 4GB kernel without make mrproper so kmap/kunmap do not have > > proper versions attached :-( or they even sell headers and binaries > > with different configs). > > But you _need_ the config. Even with your own makefiles, how are you going > to get it right for all new kernels that $CRAPDISTRO ships in a broken form, > if you don't have the configs? Yes, but currently you need only 3 files: autoconf.h, version.h, and, eventually .config, as other headers are read-only and nobody managed to get them wrong (yet). When using kernel Makefiles there are at least two more files (main and arch Makefile), of which main Makefile is known to be modified by vendors to get their cloneconfig working: they define additional rules and variables, and you may run into conflict. > If distros ship broken crap which doesn't let you build modules, there's > really not a lot you can do except note their quality control for the > record and report it in their bug tracking system. > > You are _always_ going to have problems with people shipping shite, or > possibly even actively going our of their way to prevent out-of-tree modules > building. It's fairly much an orthogonal problem though, isn't it? Increasing number of files needed to build module also increases possibility that something will go wrong :-( Ok, I'll modify modules build script to use kernel's build system if /lib/modules/`uname -r`/build/Makefile exists, and I'll see. BTW, is there any way how code outside of Makefile can detect what file was generated and where it was stored: whether vmmon.o or vmmon.ko? As it is now, Makefile contains obj-y := vmmon.o, but you can end up with vmmon.ko... I need it to make a note for uninstaller which files we added to the system (and which we can overwrite without prompting on upgrade). Petr Vandrovec ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 21:46 ` Kai Germaschewski 2003-01-26 23:12 ` Christian Zander 2003-01-27 6:17 ` Petr Vandrovec @ 2003-01-27 17:59 ` Joel Becker 2003-01-27 18:31 ` Kai Germaschewski 2 siblings, 1 reply; 47+ messages in thread From: Joel Becker @ 2003-01-27 17:59 UTC (permalink / raw) To: Kai Germaschewski Cc: Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, Jan 26, 2003 at 03:46:30PM -0600, Kai Germaschewski wrote: > it have added -DKBUILD_BASENAME and -DKBUILD_MODNAME, which are required > by the new module code. And, how did they avoid subtle breakage like not > giving the same switches on the command line? (This list goes on...) As opposed to the kernel forcing subtle breakage by specifying options that break said module? Don't say that doesn't happen (just like the kernel has to add/remove compiler switches to make some of its code work). > Okay, you have a point here, there's still a bug. vermagic.o will be > rebuilt when the version changes or any of the recorded config options > change, but it doesn't pick up changes in the compiler version, if the > new gcc has the same name. IOW, you not only need a kernel source tree (built, no less, taking up space) but it needs to be writable! > o One thing I do not understand at all: What is the problem with using > the internal build system? It makes maintainance of external modules > much easier than keeping track of what happens in the kernel and > patching a private solution all the time. Well, the Red Hat kernel tree is designed to allow you to build against there UP, SMP, and large memory kernels from one tree. In your example, you require three(!) fully built kernel trees lying around, each with a different configuration. That's a lot of space. Granted, Red Hat (as our example) can do what they have to do, since the generic kernel isn't responsible for whatever Red Hat does. However, even without their setup, in 2.4 all I have to do is keep around version.h and autoconf.h and I can usually do the right thing with one completely clean kernel tree. Joel -- "And yet I fight, And yet I fight this battle all alone. No one to cry to; No place to call home." Joel Becker Senior Member of Technical Staff Oracle Corporation E-mail: joel.becker@oracle.com Phone: (650) 506-8127 ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-27 17:59 ` Joel Becker @ 2003-01-27 18:31 ` Kai Germaschewski 2003-01-27 22:15 ` Joel Becker 0 siblings, 1 reply; 47+ messages in thread From: Kai Germaschewski @ 2003-01-27 18:31 UTC (permalink / raw) To: Joel Becker Cc: Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Mon, 27 Jan 2003, Joel Becker wrote: > On Sun, Jan 26, 2003 at 03:46:30PM -0600, Kai Germaschewski wrote: > > it have added -DKBUILD_BASENAME and -DKBUILD_MODNAME, which are required > > by the new module code. And, how did they avoid subtle breakage like not > > giving the same switches on the command line? (This list goes on...) > > As opposed to the kernel forcing subtle breakage by specifying > options that break said module? Don't say that doesn't happen (just > like the kernel has to add/remove compiler switches to make some of its > code work). Well, if you're doing things in your module which break with the command line options the rest of the kernel is using, I'd claim you're playing tricks in your module which you shouldn't. The only place I'm aware of where someone really needs to change the options is kernel/sched.c, and that's a very internal part of the kernel, an area which surely shouldn't be touched by an external module. (Of course mere adding of -I -Dmacro happens quite a bit, you can do that for your module as you like) > > Okay, you have a point here, there's still a bug. vermagic.o will be > > rebuilt when the version changes or any of the recorded config options > > change, but it doesn't pick up changes in the compiler version, if the > > new gcc has the same name. > > IOW, you not only need a kernel source tree (built, no less, > taking up space) but it needs to be writable! Basically, yes. The build process needs to be able to write, e.g. to compile its helper code in scripts/, so init/vermagic.o is just another file being written. I guess you'll hate me even more with CONFIG_MODVERSIONING coming back soon. Because that means that we need to generate the versions checksum from all the kernel objects, and we need to write the somewhere, too. In fact, these checksums are generated as part of the compiled objects, so recording checksums needs all other compiled objects to be around. If you know a sensible way to get around this limitation, I'd be happy to hear it, but I fear there isn't. So it boils down to the fact that we need a full writeable kernel tree. I agree that this is asking a lot, and I'm sure open for suggestions. The writeable part can and will be lifted using separate object directories, which means that the source can remain read-only and you can specify your own writeable object tree. > > o One thing I do not understand at all: What is the problem with using > > the internal build system? It makes maintainance of external modules > > much easier than keeping track of what happens in the kernel and > > patching a private solution all the time. > > Well, the Red Hat kernel tree is designed to allow you to build > against there UP, SMP, and large memory kernels from one tree. In your > example, you require three(!) fully built kernel trees lying around, each > with a different configuration. That's a lot of space. As I said, I am sure interested in working with people and distros to get something which everybody can live with. I'm wondering how RedHat manages to have one tree for different configurations, since in that case, at least .config/autoconf.h, EXTRAVERSION and the module version files (*.ver) need to differ, so that kinda seems not possible in one (read-only) tree. Generally, you need to have the configured source around for the kernel you want to build the module against. If that's three kernels, you need the source three times. (Or, of course, you reconfigure it between the builds) > Granted, Red Hat (as our example) can do what they have to do, > since the generic kernel isn't responsible for whatever Red Hat does. > However, even without their setup, in 2.4 all I have to do is keep > around version.h and autoconf.h and I can usually do the right thing > with one completely clean kernel tree. And that's with module versioning? --Kai ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-27 18:31 ` Kai Germaschewski @ 2003-01-27 22:15 ` Joel Becker 2003-01-27 23:08 ` Kai Germaschewski 2003-01-28 15:43 ` David Woodhouse 0 siblings, 2 replies; 47+ messages in thread From: Joel Becker @ 2003-01-27 22:15 UTC (permalink / raw) To: Kai Germaschewski Cc: Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Mon, Jan 27, 2003 at 12:31:07PM -0600, Kai Germaschewski wrote: > Well, if you're doing things in your module which break with the command > line options the rest of the kernel is using, I'd claim you're playing > tricks in your module which you shouldn't. The only place I'm aware of I'm not so sure about that. Some gcc things tweak us, and the some code has had to deal with it. This isn't something that happens often, but it still can. In addition, CFLAGS_filename.o does not allow removal of options, merely the addition if I am not mistaken. > Basically, yes. The build process needs to be able to write, e.g. to > compile its helper code in scripts/, so init/vermagic.o is just another > file being written. If my distribution has installed /usr/src/linux-x.y, I can't compile against it. Even though the 200MB of a kernel tree is already taking up space on my system, I have to download *another* 30MB and install it as *another* 200MB and build it to an eventual *another* 260MB of kernel tree. So, for every kernel I want to support, I have to have 260MB of built tree. And that's just for my userid. Anyone else on the box has to have their own n_kernels * 260MB of space waste. > fact, these checksums are generated as part of the compiled objects, so > recording checksums needs all other compiled objects to be around. If you But, once the checksums are recorded, the compiled objects are no longer needed, no? It still remains that a kernel header package with associated correct autoconf.h and checksums is at least an order of magnitude smaller than a built kernel tree. > As I said, I am sure interested in working with people and distros to get > something which everybody can live with. I'm wondering how RedHat manages > to have one tree for different configurations, since in that case, at > least .config/autoconf.h, EXTRAVERSION and the module version files > (*.ver) need to differ, so that kinda seems not possible in one > (read-only) tree. Red Hat plays tricks. They add a <rhconfig.h> to the top of autoconf.h and have some extra defines so that chunks of autoconf.h look like: #ifdef UP_FLAG ... some UP CONFIG_* options #else #ifdef SMP_FLAG ... some SMP CONFIG_* options and so on. This does indeed track modversions as well (I don't recall which files do the switching). This actually works pretty well, but it depends on the fact that their kernel flavours (up, smp, large ram) are known at the time they build this setup. This isn't necessarily the proper solution for the generic kernel. It still remains that in 2.4 you need the headers for the kernel plus the proper bits created by config/modversions. You don't need anything else, and you don't need any writability after the initial generation. This takes significantly less space than an entire built tree, and is usable from /usr/src as a readonly entity. Requiring that *each user* have the kernels they wish to build installed and fully built is a step back, IMHO. Joel -- "I always thought the hardest questions were those I could not answer. Now I know they are the ones I can never ask." - Charlie Watkins Joel Becker Senior Member of Technical Staff Oracle Corporation E-mail: joel.becker@oracle.com Phone: (650) 506-8127 ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-27 22:15 ` Joel Becker @ 2003-01-27 23:08 ` Kai Germaschewski 2003-01-27 23:37 ` Joel Becker 2003-01-28 15:43 ` David Woodhouse 1 sibling, 1 reply; 47+ messages in thread From: Kai Germaschewski @ 2003-01-27 23:08 UTC (permalink / raw) To: Joel Becker Cc: Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Mon, 27 Jan 2003, Joel Becker wrote: > On Mon, Jan 27, 2003 at 12:31:07PM -0600, Kai Germaschewski wrote: > > Well, if you're doing things in your module which break with the command > > line options the rest of the kernel is using, I'd claim you're playing > > tricks in your module which you shouldn't. The only place I'm aware of > > I'm not so sure about that. Some gcc things tweak us, and the > some code has had to deal with it. This isn't something that happens > often, but it still can. In addition, CFLAGS_filename.o does not allow > removal of options, merely the addition if I am not mistaken. Well, I suppose arguing about that without a concrete example is kinda pointless. > > Basically, yes. The build process needs to be able to write, e.g. to > > compile its helper code in scripts/, so init/vermagic.o is just another > > file being written. > > If my distribution has installed /usr/src/linux-x.y, I can't > compile against it. Even though the 200MB of a kernel tree is already > taking up space on my system, I have to download *another* 30MB and > install it as *another* 200MB and build it to an eventual *another* > 260MB of kernel tree. So, for every kernel I want to support, I have to > have 260MB of built tree. And that's just for my userid. Anyone else > on the box has to have their own n_kernels * 260MB of space waste. You ignored the fact that I said you will be able to use separate src/objdir, which means you can have your source read-only. > > fact, these checksums are generated as part of the compiled objects, so > > recording checksums needs all other compiled objects to be around. If you > > But, once the checksums are recorded, the compiled objects are > no longer needed, no? It still remains that a kernel header package > with associated correct autoconf.h and checksums is at least an order of > magnitude smaller than a built kernel tree. Yes, all you really need is the checksums. Then again, you also want a way to verify a way that the checksums match the ABI as determined by the current .config. I mean, just using the previously recorded checksums without verifying is kinda pointless, they'll just always match and not fulfill their function. > > As I said, I am sure interested in working with people and distros to get > > something which everybody can live with. I'm wondering how RedHat manages > > to have one tree for different configurations, since in that case, at > > least .config/autoconf.h, EXTRAVERSION and the module version files > > (*.ver) need to differ, so that kinda seems not possible in one > > (read-only) tree. > > Red Hat plays tricks. They add a <rhconfig.h> to the top of > autoconf.h and have some extra defines so that chunks of autoconf.h look > like: > > #ifdef UP_FLAG > ... some UP CONFIG_* options > #else > #ifdef SMP_FLAG > ... some SMP CONFIG_* options > > and so on. > > This does indeed track modversions as well (I don't recall which > files do the switching). This actually works pretty well, but it depends > on the fact that their kernel flavours (up, smp, large ram) are known > at the time they build this setup. This isn't necessarily the proper > solution for the generic kernel. > It still remains that in 2.4 you need the headers for the kernel > plus the proper bits created by config/modversions. You don't need > anything else, and you don't need any writability after the initial > generation. This takes significantly less space than an entire built > tree, and is usable from /usr/src as a readonly entity. Requiring that > *each user* have the kernels they wish to build installed and fully > built is a step back, IMHO. Yup, I now looked into what Redhat does. It's an obvious sign that there is work to be done, in particular making the build system work in a way that vendors don't need to kludge around it would definitely be nice. I have to admit that I did not think about the needs of distro vendors when implementing the new module version code (the other small issues I think can be worked around easily enough), and it's definitely an area which needs serious thinking and improvement. However, I think I need to finish the current work first, i.e. make sure the module versions actually work and then the separate obj / src dir stuff. Afterwards I think it's a really good idea to come up with a way to sensibly handle external modules, and it'd be much appreciated when all of the affected people (i.e. distro and external module guys) would provide input for that. --Kai ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-27 23:08 ` Kai Germaschewski @ 2003-01-27 23:37 ` Joel Becker 0 siblings, 0 replies; 47+ messages in thread From: Joel Becker @ 2003-01-27 23:37 UTC (permalink / raw) To: Kai Germaschewski Cc: Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Mon, Jan 27, 2003 at 05:08:50PM -0600, Kai Germaschewski wrote: > Well, I suppose arguing about that without a concrete example is kinda > pointless. Fair enough. > You ignored the fact that I said you will be able to use separate > src/objdir, which means you can have your source read-only. Yes. That still doesn't change the fact that in 2.4 I need the headers. In 2.5 I need all 200MB of source + 60MB per different .config of built tree. And that's per user using that source. > Yes, all you really need is the checksums. Then again, you also want a way > to verify a way that the checksums match the ABI as determined by the > current .config. I mean, just using the previously recorded checksums > without verifying is kinda pointless, they'll just always match and not > fulfill their function. You'd obviously have to have a way in the readonly /usr/src bits to make sure that autoconf.h and the checksums match. That would have to be enforced (as Red Hat does now with their method). > Yup, I now looked into what Redhat does. It's an obvious sign that there > is work to be done, in particular making the build system work in a way > that vendors don't need to kludge around it would definitely be nice. Maybe a way to track the differnet builds. I'm not sure. But say you had a way of keeping multiple autoconf.h files in a tree. If the kernel sources could be smart enough to figure out the version.h and modversions locations from that path, then you could have an external module create a kbuild makefile for itself and run something like: make KERN_CONF=/usr/src/linux-2.6.1/include/autoconf/build-smp/autoconf.h modules where kbuild figures out from KERN_CONF that modversions stuff is in autoconf/buid-smp/modversions/ and version.h is in autoconf/build-smp/version.h, etc. Something like that (I can beef with this example, so no one tell me exactly why this specific example is bad :-) > which needs serious thinking and improvement. However, I think I need to > finish the current work first, i.e. make sure the module versions actually > work and then the separate obj / src dir stuff. Yes, I understand that there are priorities. Joel -- "The cynics are right nine times out of ten." - H. L. Mencken Joel Becker Senior Member of Technical Staff Oracle Corporation E-mail: joel.becker@oracle.com Phone: (650) 506-8127 ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-27 22:15 ` Joel Becker 2003-01-27 23:08 ` Kai Germaschewski @ 2003-01-28 15:43 ` David Woodhouse 2003-01-28 17:03 ` Joel Becker 1 sibling, 1 reply; 47+ messages in thread From: David Woodhouse @ 2003-01-28 15:43 UTC (permalink / raw) To: Joel Becker Cc: Kai Germaschewski, Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell Joel.Becker@oracle.com said: > If my distribution has installed /usr/src/linux-x.y, I can't compile > against it. Even though the 200MB of a kernel tree is already taking > up space on my system, I have to download *another* 30MB and install > it as *another* 200MB and build it to an eventual *another* 260MB of > kernel tree. So, for every kernel I want to support, I have to have > 260MB of built tree. And that's just for my userid. Anyone else on > the box has to have their own n_kernels * 260MB of space waste. Er, if vermagic.o needed to change, then your module build was broken already and wouldn't have worked against the precompiled kernel. That's what vermagic.o is there for. You shouldn't need write permissions to the kernel source tree. Neither do you need _all_ of the kernel source; just the headers and appropriate bits of infrastructure. -- dwmw2 ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-28 15:43 ` David Woodhouse @ 2003-01-28 17:03 ` Joel Becker 0 siblings, 0 replies; 47+ messages in thread From: Joel Becker @ 2003-01-28 17:03 UTC (permalink / raw) To: David Woodhouse Cc: Kai Germaschewski, Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Tue, Jan 28, 2003 at 03:43:57PM +0000, David Woodhouse wrote: > Er, if vermagic.o needed to change, then your module build was broken > already and wouldn't have worked against the precompiled kernel. That's If you've precompiled the kernel, you have a full kernel built tree. If you haven't, then vermagic.o isn't there. Plus, modversions requires the objects according to kai. In 2.4, a "make config" and "make dep" pretty much does what is needed. The kernel is ready to be used by external modules. Joel -- "You can get more with a kind word and a gun than you can with a kind word alone." - Al Capone Joel Becker Senior Member of Technical Staff Oracle Corporation E-mail: joel.becker@oracle.com Phone: (650) 506-8127 ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 21:57 ` Christian Zander 2003-01-26 21:46 ` Kai Germaschewski @ 2003-01-26 22:23 ` Christian Zander 1 sibling, 0 replies; 47+ messages in thread From: Christian Zander @ 2003-01-26 22:23 UTC (permalink / raw) To: Kai Germaschewski Cc: Christian Zander, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, Jan 26, 2003 at 10:57:14PM +0100, Christian Zander wrote: > > Is that so? Does kbuild determine that vermagic.o needs rebuilding if > the compiler version changed? > Nevermind, I just checked - it does. -- christian zander zander@minion.de ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-23 19:35 ` Mark Fasheh 2003-01-26 13:29 ` Christian Zander @ 2003-01-26 17:43 ` Kai Germaschewski 2003-01-26 22:08 ` Christian Zander 2003-01-28 1:58 ` Rusty Russell 2 siblings, 1 reply; 47+ messages in thread From: Kai Germaschewski @ 2003-01-26 17:43 UTC (permalink / raw) To: Mark Fasheh Cc: Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Thu, 23 Jan 2003, Mark Fasheh wrote: > Can't the stuff in init/vermagic.c be moved into a header file? Maybe > vermagic.h? Most of the code can be cut 'n pasted right out of vermagic.c > and the bit that defines "const char vermagic[]..." could be placed inside a > macro which modules would then stick in the bottom of one of their c files. > This is what I'm getting at (warning I haven't checked this code or even > tried to clean it up): Your suggestion is sensible, yet it is just an indication that you're using the wrong way to build your external module. The thing is, open source projects like the linux kernel tend to move fast, and they don't care about changing the interfaces are the way to build things much, since you get the source and can recompile yourself. Due to that fact, all solutions which try to build modules externally are bound to fail sooner or later. IMO the only sensible way to overcome this is to accept help from the kernel build system instead of adding one kludge after the other to your home-made Makefile. The kernel build provides this facility today, use the make -C $KERNELSRC SUBDIRS=$PWD approach that Sam Ravnborg pointed out. It's true that it has not been clearly separated how much of the kernel source tree is needed to do that, (it's at least include/*, .config, scripts, init/, ...) so the rule is: You need the entire configured kernel tree. Now, is that so bad? When you're building kernels yourself, you obviously have enough room for a full tree anyway. When you're using a distribution, you have to install the kernel source rpm anyway, to get the headers. For all I know, these days the headers are not distributed separately from the rest of the kernel source anymore.. You might say that this is a regression w.r.t 2.4. But actually, even in 2.4, you need e.g. Makefile and arch/i386/Makefile to figure out the correct flags and things for your compile, and those are not headers, either. > in my_external_module.c, and init/vermagic.c I'd just do: > #include <linux/vermagic.h> > KERNEL_VERSIONMAGIC(); This is a good solution to this specific problem. But it does not solve the rest, e.g. your Makefile doesn't set -fomit-frame-pointer depending on CONFIG_FRAME_POINTER. It doesn't set the proper march=x86 flags. IA-64 even needs a special flag just for modules. And it'll get even worse with the reintroduction of module symbol versioning. So the above would work around this specific problem, leaving the other more subtle ones unsolved. And if you're using modules which have been built in such a fragile way with subtle differences, I think it's justified to have your kernel tainted. --Kai ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 17:43 ` Kai Germaschewski @ 2003-01-26 22:08 ` Christian Zander 2003-01-26 21:29 ` Sam Ravnborg 2003-01-26 21:40 ` David Woodhouse 0 siblings, 2 replies; 47+ messages in thread From: Christian Zander @ 2003-01-26 22:08 UTC (permalink / raw) To: Kai Germaschewski Cc: Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, Jan 26, 2003 at 11:43:44AM -0600, Kai Germaschewski wrote: > > Now, is that so bad? When you're building kernels yourself, you > obviously have enough room for a full tree anyway. When you're using > a distribution, you have to install the kernel source rpm anyway, to > get the headers. For all I know, these days the headers are not > distributed separately from the rest of the kernel source anymore.. > Debian GNU/Linux is a good example of a distribution shipping source and configured kernel header packages seperately. It seems fair enough to require a configured source tree installed, however, and I don't have a strong opinion on this particular question. > You might say that this is a regression w.r.t 2.4. But actually, > even in 2.4, you need e.g. Makefile and arch/i386/Makefile to figure > out the correct flags and things for your compile, and those are not > headers, either. > Not necessarily. > This is a good solution to this specific problem. But it does not > solve the rest, e.g. your Makefile doesn't set -fomit-frame-pointer > depending on CONFIG_FRAME_POINTER. It doesn't set the proper > march=x86 flags. IA-64 even needs a special flag just for modules. > And it'll get even worse with the reintroduction of module symbol > versioning. > > So the above would work around this specific problem, leaving the > other more subtle ones unsolved. And if you're using modules which > have been built in such a fragile way with subtle differences, I > think it's justified to have your kernel tainted. > External projects may not want to use the build flags unchanged, they may have good reasons for using their own. It seems sensible to make the kernel build system available to those who wish to use it, but it should be optional rather than mandatory. In this specific case, there is no technical reason to require the use of kbuild. -- christian zander zander@minion.de ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 22:08 ` Christian Zander @ 2003-01-26 21:29 ` Sam Ravnborg 2003-01-26 23:03 ` Christian Zander 2003-01-26 21:40 ` David Woodhouse 1 sibling, 1 reply; 47+ messages in thread From: Sam Ravnborg @ 2003-01-26 21:29 UTC (permalink / raw) To: Christian Zander Cc: Kai Germaschewski, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, Jan 26, 2003 at 11:08:42PM +0100, Christian Zander wrote: > External projects may not want to use the build flags unchanged, they > may have good reasons for using their own. It seems sensible to make > the kernel build system available to those who wish to use it, but it > should be optional rather than mandatory. In this specific case, there > is no technical reason to require the use of kbuild. Judging on the number of people that got bid by the mach- include changes, I conclude that we better ask people to use kbuild. The whole issues boils down to the following options: 1) The "kernel-headers" package shall be extended to include the vital part of kbuild 2) To develop modules you need the full kernel src 3) Do-it-yourself makefiles Can we agree on that - and take the discussion from there? In 1) and 2) you have total freedom to change options etc. Several architectures filter out generic options they dislike. Adding extra options is supported by kbuild (EXTRA_CFLAGS). In 3) you have all possibilities to screw up things. You would probarly argue you have full flexibility. But then I wonder what kind of flexibility you need for your module, that is not needed for all the modules included in the kernel? To take your argument and turn it around: What technical reasons are there to avoid kbuild? Please realise that you will be hit by changes in include paths, compiler options etc. That is visible in the number of mails seen on lkml the last couple of months. Sam ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 21:29 ` Sam Ravnborg @ 2003-01-26 23:03 ` Christian Zander 0 siblings, 0 replies; 47+ messages in thread From: Christian Zander @ 2003-01-26 23:03 UTC (permalink / raw) To: Christian Zander, Kai Germaschewski, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, Jan 26, 2003 at 10:29:50PM +0100, Sam Ravnborg wrote: > > Judging on the number of people that got bid by the mach- include > changes, I conclude that we better ask people to use kbuild. > Fair enough. > The whole issues boils down to the following options: > 1) The "kernel-headers" package shall be extended to include the > vital part of kbuild > 2) To develop modules you need the full kernel src > 3) Do-it-yourself makefiles > > Can we agree on that - and take the discussion from there? > > In 1) and 2) you have total freedom to change options etc. > Several architectures filter out generic options they dislike. > Adding extra options is supported by kbuild (EXTRA_CFLAGS). > > In 3) you have all possibilities to screw up things. You would > probarly argue you have full flexibility. But then I wonder what > kind of flexibility you need for your module, that is not needed for > all the modules included in the kernel? To take your argument and > turn it around: What technical reasons are there to avoid kbuild? > > Please realise that you will be hit by changes in include paths, > compiler options etc. That is visible in the number of mails seen > on lkml the last couple of months. > The problem isn't necessarily lack of flexibility, but the lack of unity across kernel versions. I agree that kbuild is the preferable solution for Linux 2.5, but it isn't for all incarnations of Linux 2.4 and definetely not for Linux 2.2. I do realize that changes have resulted in problems for external build systems and understand that future changes may result in similar problems. I guess a reasonable solution is adjusting existing Makefiles to be smart enough to detect kbuild and use it when available. -- christian zander zander@minion.de ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 22:08 ` Christian Zander 2003-01-26 21:29 ` Sam Ravnborg @ 2003-01-26 21:40 ` David Woodhouse 2003-01-26 23:28 ` Christian Zander 1 sibling, 1 reply; 47+ messages in thread From: David Woodhouse @ 2003-01-26 21:40 UTC (permalink / raw) To: Christian Zander Cc: Kai Germaschewski, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell zander@minion.de said: > External projects may not want to use the build flags unchanged, they > may have good reasons for using their own. That's what EXTRA_CFLAGS and CFLAGS_someobject.o are for. > It seems sensible to make the kernel build system available to those > who wish to use it, but it should be optional rather than mandatory. > In this specific case, there is no technical reason to require the use > of kbuild. The use of the kernel build process to build kernel modules is not _mandatory_, it's just that it's the only sane option. You are, of course, welcome to hack up your own broken and short-term solutions which happen to work this week for some platforms. But don't come crying to us when (not if) they stop working. The use of vermagic.c doesn't stop you from making your own build system; you can have your own vermagic.c to make your hacks work this week. -- dwmw2 ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 21:40 ` David Woodhouse @ 2003-01-26 23:28 ` Christian Zander 2003-01-26 22:46 ` David Woodhouse 0 siblings, 1 reply; 47+ messages in thread From: Christian Zander @ 2003-01-26 23:28 UTC (permalink / raw) To: David Woodhouse Cc: Christian Zander, Kai Germaschewski, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, Jan 26, 2003 at 09:40:22PM +0000, David Woodhouse wrote: > > That's what EXTRA_CFLAGS and CFLAGS_someobject.o are for. > I am well aware of the documentation in Documentation/kbuild and know that kbuild is flexible enough to support customized CFLAGS. I didn't argue that. > The use of the kernel build process to build kernel modules is not > _mandatory_, it's just that it's the only sane option. > > You are, of course, welcome to hack up your own broken and > short-term solutions which happen to work this week for some > platforms. But don't come crying to us when (not if) they stop > working. > > The use of vermagic.c doesn't stop you from making your own build > system; you can have your own vermagic.c to make your hacks work > this week. > I can't follow your hostility in this matter; I didn't "come crying" to you when things broke in the past and I'm not crying now. Last time I checked, voicing concerns was a legitimate thing to do. The "broken and short-term solutions" are needed with any kernel that doesn't provide the module building support introduced with Linux 2.5, which will likely be the majority of kernels in use for quite a while, still. -- christian zander zander@minion.de ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 23:28 ` Christian Zander @ 2003-01-26 22:46 ` David Woodhouse 2003-01-26 23:56 ` Christian Zander 0 siblings, 1 reply; 47+ messages in thread From: David Woodhouse @ 2003-01-26 22:46 UTC (permalink / raw) To: Christian Zander Cc: Kai Germaschewski, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell zander@minion.de said: > I am well aware of the documentation in Documentation/kbuild and know > that kbuild is flexible enough to support customized CFLAGS. I didn't > argue that. Ok, but why else would you want your own makefiles if that's not that you wanted them for? > I can't follow your hostility in this matter; I didn't "come crying" > to you when things broke in the past and I'm not crying now. Last time > I checked, voicing concerns was a legitimate thing to do. True, but in this case you are voicing concern about the potential breakage of something which was always known to be bad practice, fragile and unreliable. Your expression of concern is noted, but with about as much sympathy as is granted to those who express concern because kernel headers which they were including from userspace have changed. Yes, it breaks if you invent you own makefiles. We knew that. Don't Do That Then -- or if you must, then just deal with it breaking in the kernel-de-jour. > The "broken and short-term solutions" are needed with any kernel that > doesn't provide the module building support introduced with Linux 2.5, > which will likely be the majority of kernels in use for quite a while, > still. 'make -C $LINUXDIR SUBDIRS=$PWD modules' has worked for as long as I can remember; it's not new in 2.5. It's _always_ been the only reliable way to get kernel modules to build with the correct options. -- dwmw2 ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 22:46 ` David Woodhouse @ 2003-01-26 23:56 ` Christian Zander 2003-01-26 23:04 ` David Woodhouse 0 siblings, 1 reply; 47+ messages in thread From: Christian Zander @ 2003-01-26 23:56 UTC (permalink / raw) To: David Woodhouse Cc: Christian Zander, Kai Germaschewski, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell On Sun, Jan 26, 2003 at 10:46:39PM +0000, David Woodhouse wrote: > > Ok, but why else would you want your own makefiles if that's not > that you wanted them for? > Essentially because they already existed and worked (well enough). > True, but in this case you are voicing concern about the potential > breakage of something which was always known to be bad practice, > fragile and unreliable. > > Your expression of concern is noted, but with about as much sympathy > as is granted to those who express concern because kernel headers > which they were including from userspace have changed. > > Yes, it breaks if you invent you own makefiles. We knew that. Don't > Do That Then -- or if you must, then just deal with it breaking in > the kernel-de-jour. > Fair enough. > 'make -C $LINUXDIR SUBDIRS=$PWD modules' has worked for as long as > I can remember; it's not new in 2.5. It's _always_ been the only > reliable way to get kernel modules to build with the correct > options. > Since Linux 2.2 and including any specifics involved in the process of customizing CFLAGS, ...? If that's the case, I admit ignorance and ask that my earlier remarks be ignored. -- christian zander zander@minion.de ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-26 23:56 ` Christian Zander @ 2003-01-26 23:04 ` David Woodhouse 0 siblings, 0 replies; 47+ messages in thread From: David Woodhouse @ 2003-01-26 23:04 UTC (permalink / raw) To: Christian Zander Cc: Kai Germaschewski, Mark Fasheh, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, Rusty Russell zander@minion.de said: > Since Linux 2.2 and including any specifics involved in the process > of customizing CFLAGS, ...? If that's the case, I admit ignorance and > ask that my earlier remarks be ignored. imladris /home/dwmw2/working/mtd/drivers/mtd $ make LINUXDIR=/inst/linux/linux-2.2 CFLAGS_afs.o=-DFISH EXTRA_CFLAGS=-DTURNIP make -C /inst/linux/linux-2.2 SUBDIRS=`pwd` modules make[1]: Entering directory `/inst/linux/linux-2.2' make -C /home/dwmw2/working/mtd/drivers/mtd CFLAGS="-Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DMODULE" MAKING_MODULES=1 modules make[2]: Entering directory `/home/dwmw2/working/mtd/drivers/mtd' gcc296 -D__KERNEL__ -I/inst/linux/linux-2.2/include -I/home/dwmw2/working/mtd/drivers/mtd/../../include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DMODULE -DTURNIP -DFISH -DEXPORT_SYMTAB -c afs.c make[2]: *** Deleting file `afs.o' make[2]: *** [afs.o] Interrupt make[1]: *** [_mod_/home/dwmw2/working/mtd/drivers/mtd] Interrupt make: *** [modules] Interrupt -- dwmw2 ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-23 19:35 ` Mark Fasheh 2003-01-26 13:29 ` Christian Zander 2003-01-26 17:43 ` Kai Germaschewski @ 2003-01-28 1:58 ` Rusty Russell 2003-01-28 19:10 ` Mark Fasheh 2 siblings, 1 reply; 47+ messages in thread From: Rusty Russell @ 2003-01-28 1:58 UTC (permalink / raw) To: Mark Fasheh; +Cc: Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML In message <20030123193540.GD13137@ca-server1.us.oracle.com> you write: > Can't the stuff in init/vermagic.c be moved into a header file? Maybe > vermagic.h? Most of the code can be cut 'n pasted right out of vermagic.c > and the bit that defines "const char vermagic[]..." could be placed inside a > macro which modules would then stick in the bottom of one of their c files. And then you'll die horribly next time Kai or I change the way modules are built. Really, using the Makefiles is always the most future-proof way! Hope that helps! Rusty. -- Anyone who quotes me in their sig is an idiot. -- Rusty Russell. ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-28 1:58 ` Rusty Russell @ 2003-01-28 19:10 ` Mark Fasheh 2003-01-28 19:17 ` Kai Germaschewski 0 siblings, 1 reply; 47+ messages in thread From: Mark Fasheh @ 2003-01-28 19:10 UTC (permalink / raw) To: Rusty Russell; +Cc: Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML, kai On Tue, Jan 28, 2003 at 12:58:40PM +1100, Rusty Russell wrote: > And then you'll die horribly next time Kai or I change the way modules > are built. > > Really, using the Makefiles is always the most future-proof way! I've actually converted my stuff to use the Makefiles, which does simplify things, but the problem I had was mainly with the whole vermagic.c stuff and needing a full source tree + compiled objects (or having the tree writeable by users who want to compile modules) around just to build my module. Anyway, Joel Becker, Christian Zander and others have eloquently voiced my complaints previously in this thread, so I won't go over them again :) As far as using the kernel build system, so far so good - it's nice, and builds things correctly though I'm having some trouble getting it to install my module correctly, which may just indicate that I need to re-read the docs. --Mark -- Mark Fasheh Software Developer, Oracle Corp mark.fasheh@oracle.com ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-28 19:10 ` Mark Fasheh @ 2003-01-28 19:17 ` Kai Germaschewski 0 siblings, 0 replies; 47+ messages in thread From: Kai Germaschewski @ 2003-01-28 19:17 UTC (permalink / raw) To: Mark Fasheh Cc: Rusty Russell, Thomas Schlichter, Randy.Dunlap, Sam Ravnborg, LKML On Tue, 28 Jan 2003, Mark Fasheh wrote: > On Tue, Jan 28, 2003 at 12:58:40PM +1100, Rusty Russell wrote: > > And then you'll die horribly next time Kai or I change the way modules > > are built. > > > > Really, using the Makefiles is always the most future-proof way! > I've actually converted my stuff to use the Makefiles, which does simplify > things, but the problem I had was mainly with the whole vermagic.c stuff and > needing a full source tree + compiled objects (or having the tree writeable > by users who want to compile modules) around just to build my module. > Anyway, Joel Becker, Christian Zander and others have eloquently voiced my > complaints previously in this thread, so I won't go over them again :) > > As far as using the kernel build system, so far so good - it's nice, and > builds things correctly though I'm having some trouble getting it to > install my module correctly, which may just indicate that I need to re-read > the docs. The docs are kinda out-of-date, expect an update (thanks to Sam Ravnborg) soon. However, there aren't provisions for installing third party modules yet, so that's on my todo list together with the points above. --Kai ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-23 13:59 no version magic, tainting kernel Thomas Schlichter 2003-01-23 16:29 ` Randy.Dunlap 2003-01-23 16:52 ` Sam Ravnborg @ 2003-01-27 18:52 ` Jerry Cooperstein 2003-01-27 19:12 ` Sam Ravnborg 2003-01-27 19:54 ` Gerd Knorr 2 siblings, 2 replies; 47+ messages in thread From: Jerry Cooperstein @ 2003-01-27 18:52 UTC (permalink / raw) To: linux-kernel The solution make -C KERNEL_SOURCE SUBDIRS=$PWD modules is fine, but you have to have a Makefile in the current directory, and that Makefile needs a somewhat different form for 2.4 and 2.5 kernels. Here is a quick and dirty script for generating a quick boilerplate for the sources in that directory. Straightforward to customize and extend etc. (You may have to change some whitespace to TABS for make...) ====================================================================== Jerry Cooperstein, Senior Consultant, <coop@axian.com> Axian, Inc., Software Consulting and Training 4800 SW Griffith Dr., Ste. 202, Beaverton, OR 97005 USA http://www.axian.com/ ====================================================================== #!/bin/bash # script for generating external Makefile for kernel modules # Jerry Cooperstein, Axian Inc 2003_01_27 # Too trivial to GPL; use as desired. # do either "makeit"; assumes kernel source at /usr/src/linux-`uname -r` # or "makeit /usr/src/linux-2.5.59 or makeit /usr/src/linux-2.4.20" etc. # Should work on all 2.4, 2.5 kernels. # assumes all .c files in current directory are modules if [ "$1" == "" ] ; then KROOT=/usr/src/linux-`uname -r` else KROOT="$1" fi if [ `echo $KROOT | grep 2.5` ] ; then VERSION=2.5 else VERSION=2.4 fi OBJS="" for names in *.c ; do OBJS=$OBJS" `basename $names .c`.o" done PHONYS="$OBJS" rm -f Makefile cat <<EOF > Makefile obj-m += $OBJS KROOT=$KROOT all: mmodules EOF if [ $VERSION == 2.5 ] ; then CLEANSTUFF="*.o *.ko .*cmd" cat <<EOF >> Makefile PHONYS=$PHONYS .PHONY: \$(PHONYS) clean EOF else CLEANSTUFF="*.o .*flags" cat <<EOF >> Makefile TOPDIR=\$(KROOT) include \$(TOPDIR)/Rules.make .PHONY: clean EOF fi cat <<EOF >> Makefile mmodules: \$(MAKE) -C \$(KROOT) SUBDIRS=$PWD modules clean: rm $CLEANSTUFF EOF ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-27 18:52 ` Jerry Cooperstein @ 2003-01-27 19:12 ` Sam Ravnborg 2003-01-27 19:35 ` Jerry Cooperstein 2003-01-27 19:54 ` Gerd Knorr 1 sibling, 1 reply; 47+ messages in thread From: Sam Ravnborg @ 2003-01-27 19:12 UTC (permalink / raw) To: Jerry Cooperstein; +Cc: linux-kernel On Mon, Jan 27, 2003 at 10:52:28AM -0800, Jerry Cooperstein wrote: > The solution > > make -C KERNEL_SOURCE SUBDIRS=$PWD modules > > is fine, but you have to have a Makefile in the current directory, > and that Makefile needs a somewhat different form for 2.4 and > 2.5 kernels. Notes solely to the Kernel 2.5 part of your script: 1) You do not need any targets like all: or mmodules: 2) The clean: rule will not be used in 2.5, use clean-files instead. 3) No reason to special case on .ko or not. .ko will not harm on 2.4 4) I cannot see any reason to declare all OBJS as .PHONY 5) It looks like you try to enable the use of make in the directory where the module exists. This is an unusual extension, which will clutter up the makefile a bit. I would prefer a mmake script or similar that did the make -c ... trick. Sam ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-27 19:12 ` Sam Ravnborg @ 2003-01-27 19:35 ` Jerry Cooperstein 0 siblings, 0 replies; 47+ messages in thread From: Jerry Cooperstein @ 2003-01-27 19:35 UTC (permalink / raw) To: linux-kernel; +Cc: sam On Mon, Jan 27, 2003 at 08:12:25PM +0100, Sam Ravnborg wrote: > > 1) You do not need any targets like all: or mmodules: > 2) The clean: rule will not be used in 2.5, use clean-files instead. > 3) No reason to special case on .ko or not. .ko will not harm on 2.4 > 4) I cannot see any reason to declare all OBJS as .PHONY > 5) It looks like you try to enable the use of make in the directory > where the module exists. > This is an unusual extension, which will clutter up the makefile > a bit. > I would prefer a mmake script or similar that did the make -c ... trick. > > Sam on 4): yep, no need for PHONY stuff (simplified script attached) The reason for this cropping in is that I had a more complex script that reflected my actual situation; I have a directory that contains some kernel modules, some testing programs and other junk, and I build a complete Makefile that has rules for all these guys, not just the kernel modules. Which is why I violated your sensibilties about the other points I guess. I didn't just want to pass the work off to the kernel building stuff. If you want to do this I agree all you need is the part to generate a objs-m list and the Rules.make stuff for 2.4, and that is simpler. You could indeed remove all the targets etc, but then as you note, you can't just say make, but have to do the make -C KERNEL_SOURCE SUBDIRS=$PWD modules business. I've attached that one too. coop ====================================================================== Jerry Cooperstein, Senior Consultant, <coop@axian.com> Axian, Inc., Software Consulting and Training 4800 SW Griffith Dr., Ste. 202, Beaverton, OR 97005 USA http://www.axian.com/ ====================================================================== #(FULL SCRIPT, NO PHONY): ###################################################################### #!/bin/bash # script for generating external Makefile for kernel modules # Jerry Cooperstein, Axian Inc 2003_01_27 # Too trivial to GPL; use as desired. # do either "makeit"; assumes kernel source at /usr/src/linux-`uname -r` # or "makeit /usr/src/linux-2.5.59 or makeit /usr/src/linux-2.4.20" etc. # Should work on all 2.4, 2.5 kernels. # assumes all .c files in current directory are modules if [ "$1" == "" ] ; then KROOT=/usr/src/linux-`uname -r` else KROOT="$1" fi if [ `echo $KROOT | grep 2.5` ] ; then VERSION=2.5 else VERSION=2.4 fi OBJS="" for names in *.c ; do OBJS=$OBJS" `basename $names .c`.o" done rm -f Makefile cat <<EOF > Makefile obj-m += $OBJS KROOT=$KROOT all: mmodules EOF if [ $VERSION == 2.5 ] ; then CLEANSTUFF="*.o *.ko .*cmd" cat <<EOF >> Makefile EOF else CLEANSTUFF="*.o .*flags" cat <<EOF >> Makefile TOPDIR=\$(KROOT) include \$(TOPDIR)/Rules.make EOF fi cat <<EOF >> Makefile mmodules: \$(MAKE) -C \$(KROOT) SUBDIRS=$PWD modules clean: rm $CLEANSTUFF EOF ###################################################################### #(SHORT SCRIPT: no local make targets) #!/bin/bash # script for generating external Makefile for kernel modules # Jerry Cooperstein, Axian Inc 2003_01_27 # Too trivial to GPL; use as desired. # do either "makeit"; assumes kernel source at /usr/src/linux-`uname -r` # or "makeit /usr/src/linux-2.5.59 or makeit /usr/src/linux-2.4.20" etc. # Should work on all 2.4, 2.5 kernels. # assumes all .c files in current directory are modules if [ "$1" == "" ] ; then KROOT=/usr/src/linux-`uname -r` else KROOT="$1" fi if [ `echo $KROOT | grep 2.5` ] ; then VERSION=2.5 else VERSION=2.4 fi OBJS="" for names in *.c ; do OBJS=$OBJS" `basename $names .c`.o" done rm -f Makefile cat <<EOF > Makefile obj-m += $OBJS KROOT=$KROOT EOF if [ $VERSION == 2.5 ] ; then cat <<EOF >> Makefile EOF else cat <<EOF >> Makefile TOPDIR=\$(KROOT) include \$(TOPDIR)/Rules.make EOF fi ^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: no version magic, tainting kernel. 2003-01-27 18:52 ` Jerry Cooperstein 2003-01-27 19:12 ` Sam Ravnborg @ 2003-01-27 19:54 ` Gerd Knorr 1 sibling, 0 replies; 47+ messages in thread From: Gerd Knorr @ 2003-01-27 19:54 UTC (permalink / raw) To: linux-kernel Jerry Cooperstein <coop@axian.com> writes: > is fine, but you have to have a Makefile in the current directory, > and that Makefile needs a somewhat different form for 2.4 and > 2.5 kernels. It is no problem to use the same Makefile for both 2.4.x and 2.5.x, and even let the Makefile do the RightThing[tm] in case the user just types 'make'. My Makefiles for kernel stuff look like this (stripped down a bit): ==============================[ cut here ]============================== ifneq ($(KERNELRELEASE),) # call from kernel build system obj-m := btaudio.o -include $(TOPDIR)/Rules.make else KDIR := /lib/modules/$(shell uname -r)/build PWD := $(shell pwd) default: $(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules endif ==============================[ cut here ]============================== If you need different stuff for 2.4.x and 2.5.x you can handle it using ifeq ($(VERSION).$(PATCHLEVEL),2.4) Gerd ^ permalink raw reply [flat|nested] 47+ messages in thread
end of thread, other threads:[~2003-01-28 19:08 UTC | newest] Thread overview: 47+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2003-01-23 13:59 no version magic, tainting kernel Thomas Schlichter 2003-01-23 16:29 ` Randy.Dunlap 2003-01-23 16:52 ` Sam Ravnborg 2003-01-23 17:32 ` Thomas Schlichter 2003-01-23 18:22 ` Sam Ravnborg 2003-01-23 19:35 ` Mark Fasheh 2003-01-26 13:29 ` Christian Zander 2003-01-26 13:33 ` Keith Owens 2003-01-26 18:02 ` Kai Germaschewski 2003-01-26 17:51 ` Kai Germaschewski 2003-01-26 21:57 ` Christian Zander 2003-01-26 21:46 ` Kai Germaschewski 2003-01-26 23:12 ` Christian Zander 2003-01-26 22:55 ` David Woodhouse 2003-01-27 0:07 ` Christian Zander 2003-01-26 23:16 ` David Woodhouse 2003-01-27 0:24 ` Christian Zander 2003-01-27 16:25 ` Kai Germaschewski 2003-01-27 16:29 ` David Woodhouse 2003-01-27 16:39 ` Kai Germaschewski 2003-01-27 6:17 ` Petr Vandrovec 2003-01-27 9:02 ` David Woodhouse 2003-01-27 9:24 ` Petr Vandrovec 2003-01-27 17:59 ` Joel Becker 2003-01-27 18:31 ` Kai Germaschewski 2003-01-27 22:15 ` Joel Becker 2003-01-27 23:08 ` Kai Germaschewski 2003-01-27 23:37 ` Joel Becker 2003-01-28 15:43 ` David Woodhouse 2003-01-28 17:03 ` Joel Becker 2003-01-26 22:23 ` Christian Zander 2003-01-26 17:43 ` Kai Germaschewski 2003-01-26 22:08 ` Christian Zander 2003-01-26 21:29 ` Sam Ravnborg 2003-01-26 23:03 ` Christian Zander 2003-01-26 21:40 ` David Woodhouse 2003-01-26 23:28 ` Christian Zander 2003-01-26 22:46 ` David Woodhouse 2003-01-26 23:56 ` Christian Zander 2003-01-26 23:04 ` David Woodhouse 2003-01-28 1:58 ` Rusty Russell 2003-01-28 19:10 ` Mark Fasheh 2003-01-28 19:17 ` Kai Germaschewski 2003-01-27 18:52 ` Jerry Cooperstein 2003-01-27 19:12 ` Sam Ravnborg 2003-01-27 19:35 ` Jerry Cooperstein 2003-01-27 19:54 ` Gerd Knorr
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox