* Where did vm_operations_struct->unmap in 2.4.0 go? @ 2001-01-10 3:27 Allen Unueco 2001-01-10 3:50 ` Keith Owens 2001-01-11 5:38 ` Antony Suter 0 siblings, 2 replies; 41+ messages in thread From: Allen Unueco @ 2001-01-10 3:27 UTC (permalink / raw) To: linux-kernel Sometime around test10 or test11 unmap left vm_operations_struct. The comment implies its there but it's gone. Where did it go? How do I get a call back for a page unmap? I ran into this while hacking the Nvidia kernel driver to work with 2.4.0. I got the driver working but it's not 100% Also where did get_module_symbol() and put_module_symbol() go? -Allen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-10 3:27 Where did vm_operations_struct->unmap in 2.4.0 go? Allen Unueco @ 2001-01-10 3:50 ` Keith Owens 2001-01-11 5:38 ` Antony Suter 1 sibling, 0 replies; 41+ messages in thread From: Keith Owens @ 2001-01-10 3:50 UTC (permalink / raw) To: Allen Unueco; +Cc: linux-kernel On Tue, 09 Jan 2001 19:27:24 -0800, Allen Unueco <allen@premierweb.com> wrote: >Also where did get_module_symbol() and put_module_symbol() go? http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg08791.html http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg11497.html - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-10 3:27 Where did vm_operations_struct->unmap in 2.4.0 go? Allen Unueco 2001-01-10 3:50 ` Keith Owens @ 2001-01-11 5:38 ` Antony Suter 2001-01-11 6:05 ` Keith Owens 1 sibling, 1 reply; 41+ messages in thread From: Antony Suter @ 2001-01-11 5:38 UTC (permalink / raw) To: List Linux-Kernel; +Cc: Allen Unueco Allen Unueco wrote: > > Sometime around test10 or test11 unmap left vm_operations_struct. The > comment implies its there but it's gone. Where did it go? > > How do I get a call back for a page unmap? > > I ran into this while hacking the Nvidia kernel driver to work with > 2.4.0. I got the driver working but it's not 100% > > Also where did get_module_symbol() and put_module_symbol() go? > > -Allen Patches for the NVIDIA binary X drivers following all these kernel changes can be gotten from IRC server irc.openprojects.net, channel #nvidia. Or from http://ex.shafted.com.au/nvidia/ -- - Antony Suter (antony@mira.net) "ExWired" openpgp:71ADFC87 - "...to condense fact from the vapor of nuance." - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-11 5:38 ` Antony Suter @ 2001-01-11 6:05 ` Keith Owens 2001-01-11 11:42 ` David Woodhouse 0 siblings, 1 reply; 41+ messages in thread From: Keith Owens @ 2001-01-11 6:05 UTC (permalink / raw) To: Antony Suter; +Cc: List Linux-Kernel, Allen Unueco On Thu, 11 Jan 2001 16:38:50 +1100, Antony Suter <antony@mira.net> wrote: >Allen Unueco wrote: >> I ran into this while hacking the Nvidia kernel driver to work with >> 2.4.0. I got the driver working but it's not 100% >> >> Also where did get_module_symbol() and put_module_symbol() go? > >Patches for the NVIDIA binary X drivers following all these kernel >changes can be gotten from IRC server irc.openprojects.net, channel >#nvidia. Or from http://ex.shafted.com.au/nvidia/ And what a pile of crud those patches are!! Instead of using the clean replacement interface for get_module_symbol, nvidia/patch-2.4.0-PR hard codes the old get_module_symbol algorithm as inline code. This patch violates the modules interface by accessing modules.c internal data. It still suffers from all the problems that get_module_symbol had. Because it is hard coded as inline code instead of a common function, will be much harder to fix when it breaks. Whoever coded that patch should be taken out and shot, hung, drawn and quartered then forced to write COBOL for the rest of their natural life. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-11 6:05 ` Keith Owens @ 2001-01-11 11:42 ` David Woodhouse 2001-01-11 12:12 ` Keith Owens 0 siblings, 1 reply; 41+ messages in thread From: David Woodhouse @ 2001-01-11 11:42 UTC (permalink / raw) To: Keith Owens; +Cc: Antony Suter, List Linux-Kernel, Allen Unueco kaos@ocs.com.au said: > And what a pile of crud those patches are!! Instead of using the > clean replacement interface for get_module_symbol, nvidia/ > patch-2.4.0-PR hard codes the old get_module_symbol algorithm as > inline code. Taking away get_module_symbol() and providing a replacement which has link order problems wasn't really very sensible. You've changed a lookup in a static table built at compile time to a lookup in a dynamic table which has to be built in the right order at runtime. It's too late to do the sensible thing and deprecate the old version rather than having a 'flag day'. But can we at least fix the link order crap? struct static_inter_module_entry { const char *im_name; const void *userdata; }; #define inter_module_register_static(x,y) \ static struct static_inter_module_entry __ime_##x \ __attribute__((unused,__section__(".intermodule")) \ = { #x, y }; .. and the obvious for looking in that table in inter_module_get(). -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-11 11:42 ` David Woodhouse @ 2001-01-11 12:12 ` Keith Owens 2001-01-11 12:32 ` David Woodhouse 0 siblings, 1 reply; 41+ messages in thread From: Keith Owens @ 2001-01-11 12:12 UTC (permalink / raw) To: David Woodhouse; +Cc: Antony Suter, List Linux-Kernel, Allen Unueco On Thu, 11 Jan 2001 11:42:24 +0000, David Woodhouse <dwmw2@infradead.org> wrote: >Taking away get_module_symbol() and providing a replacement which has link >order problems wasn't really very sensible. > >It's too late to do the sensible thing and deprecate the old version rather >than having a 'flag day'. But can we at least fix the link order crap? > >struct static_inter_module_entry { > const char *im_name; > const void *userdata; >}; > >#define inter_module_register_static(x,y) \ > static struct static_inter_module_entry __ime_##x \ > __attribute__((unused,__section__(".intermodule")) \ > = { #x, y }; > >.. and the obvious for looking in that table in inter_module_get(). If object X registers data for object Y to use then X _must_ initialise before Y. It does not matter whether the registration method is static or dynamic, the initialisation order must be observed. Q. With your suggested static method, what happens when Y initialises before X, calls inter_module_get, retrieves X's static data and starts to use it before X has initialised? A. Oops! The whole point of registration methods is that the owner of the data decides when they are ready to provide the service. Ensuring that code is initialised in the correct order, with providers starting before consumers, is a fact of life. I dislike the method that the kernel uses to control initialisation order, but that is an entirely separate problem from inter_module_xxx. What we really want at startup is a correct initialisation order. What we have is the order that objects are selected in a Makefile which maps to the link order of objects in vmlinux which maps to the listed order of init routines in section .init.text which maps to initialisation order. The mechanism is three layers away from the problem and it is difficult to understand for many people. It would be much nicer to define ordering sets. Code in driver foo needs the code in driver bar to initialise first. cfi_probe cannot initialise until cfi_cmdset_0001 and cfi_cmdset_0002 have initialised. Declare it that way so it is clear what is going on and why, instead of being implied by the Makefile order via three layers of indirection. Then let the kernel build system do whatever it takes to honour the documented initialisation order. The problem is, Linus likes the current method. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-11 12:12 ` Keith Owens @ 2001-01-11 12:32 ` David Woodhouse 2001-01-11 12:46 ` Keith Owens 0 siblings, 1 reply; 41+ messages in thread From: David Woodhouse @ 2001-01-11 12:32 UTC (permalink / raw) To: Keith Owens; +Cc: Antony Suter, List Linux-Kernel, Allen Unueco kaos@ocs.com.au said: > Q. With your suggested static method, what happens when Y initialises > before X, calls inter_module_get, retrieves X's static data and > starts to use it before X has initialised? > A. Oops! No. You'd explicitly only use the static registration when object X doesn't _need_ initialisation, which is the case for my code. As it is, I've had to add completely now init routines for modules which didn't have them before, and all those init routines do is inter_module_register() and pray that they're called in time. kaos@ocs.com.au said: > The whole point of registration methods is that the owner of the data > decides when they are ready to provide the service. Ensuring that > code is initialised in the correct order, with providers starting > before consumers, is a fact of life. I have decided when I'm ready to provide the service. At compile time. Ensuring that the code is initialised in the correct order is not a fact of life. It's an artifact of the limited functionality of the new setup. kaos@ocs.com.au said: > It would be much nicer to define ordering sets. Code in driver foo > needs the code in driver bar to initialise first. cfi_probe cannot > initialise until cfi_cmdset_0001 and cfi_cmdset_0002 have initialised. > Declare it that way so it is clear what is going on and why, instead > of being implied by the Makefile order via three layers of > indirection. For cases where the code really does need initialisation, that is true. It should be done properly rather than just implicitly ordered by the Makefile. But in this case, the command set drivers don't need initialisation. They just provide a function for use by the generic CFI init code. They don't _need_ to initialise themselves beforehand. The dynamic registration has introduced this ordering dependency when previously there was none. > The problem is, Linus likes the current method. Then he either hasn't considered this particular case, or he's wrong. It happens. I'm not suggesting that we change it drastically, only that we add the option of static (compile-time) registration for those entries which require it. Actually, I'd rather do this with weak symbols. Something along the lines of... extern cfi_cmdset_fn_t cfi_cmdset_0001 __attribute__((weak)); extern cfi_cmdset_fn_t cfi_cmdset_0002 __attribute__((weak)); ... if (cfi_cmdset_0001 && chip_is_type_1()) return cfi_cmdset_0001(args...); if (cfi_cmdset_0002 && chip_is_type_2()) return cfi_cmdset_0002(args...); return cfi_cmdset_load_module(args...) Unfortunately, I couldn't get it to work reliably on anything but x86. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-11 12:32 ` David Woodhouse @ 2001-01-11 12:46 ` Keith Owens 2001-01-11 13:09 ` Alan Cox 2001-01-11 13:25 ` David Woodhouse 0 siblings, 2 replies; 41+ messages in thread From: Keith Owens @ 2001-01-11 12:46 UTC (permalink / raw) To: David Woodhouse; +Cc: List Linux-Kernel On Thu, 11 Jan 2001 12:32:10 +0000, David Woodhouse <dwmw2@infradead.org> wrote: >I'm not suggesting that we change it drastically, only that we add >the option of static (compile-time) registration for those entries which >require it. So you want two services, one static for code that does not do any initialisation and one dynamic for code that does do initialisation. Can you imagine the fun when somebody adds startup code to a routine that was using static registration? Oh dear, I added init code so I have to remember to change from static to dynamic registration, and that affects the link order so now I have to tweak the Makefile. Thanks, but no thanks! Stick to one method that works for all routines, dynamic registration. If that imposes the occasional need for a couple of extra calls in some routines and for people to think about initialisation order right from the start then so be it, it is a small price to pay for long term stability and ease of maintenance. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-11 12:46 ` Keith Owens @ 2001-01-11 13:09 ` Alan Cox 2001-01-11 13:14 ` Keith Owens 2001-01-11 13:25 ` David Woodhouse 1 sibling, 1 reply; 41+ messages in thread From: Alan Cox @ 2001-01-11 13:09 UTC (permalink / raw) To: Keith Owens; +Cc: David Woodhouse, List Linux-Kernel > Stick to one method that works for all routines, dynamic registration. > If that imposes the occasional need for a couple of extra calls in some > routines and for people to think about initialisation order right from > the start then so be it, it is a small price to pay for long term > stability and ease of maintenance. What happens when we get a loop in init order because of binding and other init order conflicts? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-11 13:09 ` Alan Cox @ 2001-01-11 13:14 ` Keith Owens 2001-01-12 2:12 ` Ingo Oeser 0 siblings, 1 reply; 41+ messages in thread From: Keith Owens @ 2001-01-11 13:14 UTC (permalink / raw) To: Alan Cox; +Cc: David Woodhouse, List Linux-Kernel On Thu, 11 Jan 2001 13:09:13 +0000 (GMT), Alan Cox <alan@lxorguk.ukuu.org.uk> wrote: >> Stick to one method that works for all routines, dynamic registration. >> If that imposes the occasional need for a couple of extra calls in some >> routines and for people to think about initialisation order right from >> the start then so be it, it is a small price to pay for long term >> stability and ease of maintenance. > >What happens when we get a loop in init order because of binding and other init >order conflicts? The kernel does not support circular dependencies between providers and consumers. It does not matter whether they are built into vmlinux or loaded as modules, there can be no loops in the directed graph of dependencies. It just does not make sense. A while ago there was accidentally a loop between two ppp related modules, each needed a routine in the other module. modprobe would not load them. Even if it could have loaded them, it would have been impossible to unload, both modules would have had a use count on the other. The fix was to remove the incorrect loop, it was a programming error. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-11 13:14 ` Keith Owens @ 2001-01-12 2:12 ` Ingo Oeser 2001-01-12 2:30 ` Keith Owens 0 siblings, 1 reply; 41+ messages in thread From: Ingo Oeser @ 2001-01-12 2:12 UTC (permalink / raw) To: Keith Owens; +Cc: Alan Cox, David Woodhouse, List Linux-Kernel On Fri, Jan 12, 2001 at 12:14:44AM +1100, Keith Owens wrote: > >What happens when we get a loop in init order because of binding and other init > >order conflicts? > > The kernel does not support circular dependencies between providers and > consumers. It does not matter whether they are built into vmlinux or > loaded as modules, there can be no loops in the directed graph of > dependencies. It just does not make sense. So why don't we use sth. like depmod for these issues and get the link order automagically (like we get module load order)? Keith: Perhaps you could explain, why this is impossible. Regards Ingo Oeser -- 10.+11.03.2001 - 3. Chemnitzer LinuxTag <http://www.tu-chemnitz.de/linux/tag> <<<<<<<<<<<< come and join the fun >>>>>>>>>>>> - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-12 2:12 ` Ingo Oeser @ 2001-01-12 2:30 ` Keith Owens 2001-01-12 10:27 ` David Woodhouse 2001-01-12 12:01 ` Daniel Phillips 0 siblings, 2 replies; 41+ messages in thread From: Keith Owens @ 2001-01-12 2:30 UTC (permalink / raw) To: Ingo Oeser; +Cc: Alan Cox, David Woodhouse, List Linux-Kernel On Fri, 12 Jan 2001 03:12:47 +0100, Ingo Oeser <ingo.oeser@informatik.tu-chemnitz.de> wrote: >So why don't we use sth. like depmod for these issues and get the >link order automagically (like we get module load order)? depmod handles dependencies on symbols. Module Y needs a symbol from module X so modprobe must load X before Y. This is a link/load problem which depmod handles fairly well. The initialisation order is a dependency on actions, not on symbols. Code F cannot start until code E has initialised so execute E before F. This should have *NOTHING* to do with link order, but it is implemented as a side effect of link ordering which confuses people. People need to realise that the problem is initialisation order, nothing more, nothing less. You have to determine and document the startup requirements for your code. Only you know what the startup pre-requisites for your code are, there is no way for another program to determine this from the source. Document your startup requirements, implement according to the documentation and your problems go away. Initialisation order is not the problem, the implementation is the problem. The method currently used to control initialisation order sucks. It is better than the original method (lots of conditional calls in main.c) but it still sucks. * Initialisation order is set by the order of structures in section .initcall.init. * The order of the structures in .initcall.init is set by the order that objects are linked into vmlinux. * The link order for vmlinux is derived from a combination of line order within a Makefile plus an overriding directory link order from the top level Makefile and parent Makefiles. * Because order is a side effect of adding a line to a Makefile, the order requirements are rarely documented. It is no wonder that people have problems getting the initialisation order correct. I want to completely remove this multi layered method for setting initialisation order and go back to basics. I want the programmer to say "initialise E and F after G, H and I". The kernel build system works out the directed graph of initialisation order then controls the execution of startup code to satisfy this graph. That still means controlling link order to achieve the required result. But with my approach the complexity would be handled by kbuild based on explicit rules which are documented in the local Makefile, instead of the complexity being handled by programmer via implicit rules scattered over several layers of Makefiles. A typical graph would have scsi disk depends on scsi host adaptor group which depends on pci. Within the scsi host adaptor group you might need to initialise one driver before another, so just declare the few inter-driver dependencies. kbuild would automatically initialise pci then the scsi host adaptors (in the correct order) then scsi disk. Most of the objects have fairly simple execution dependencies, e.g. all file systems depend on core fs code having already executed. There are no dependencies between most file systems so most file systems could initialise in any order[1] which means they could be linked in any order within the file system group. I am ready and willing to code this method, it would make kbuild a lot easier to code, as well as making future maintainence a lot easier. Linus refuses to accept this approach. He insists that kernel coders explicitly specify the link order for everything, via Makefile order[2]. As long as Linus insists on kernel coders explicitly controlling the entire link order, we are stuck with the current method. I have tried to change his mind without success. [1] vfat is one obvious exception, it needs dos first. Also the first few built in file systems must execute in a defined order because that in turn controls the probe order for mount. But this order should be explicitly declared, not as a side effect of the line order in fs/Makefile. [2] http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg10520.html - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-12 2:30 ` Keith Owens @ 2001-01-12 10:27 ` David Woodhouse 2001-01-12 11:55 ` Keith Owens 2001-01-12 12:01 ` Daniel Phillips 1 sibling, 1 reply; 41+ messages in thread From: David Woodhouse @ 2001-01-12 10:27 UTC (permalink / raw) To: Keith Owens; +Cc: Ingo Oeser, Alan Cox, List Linux-Kernel On Fri, 12 Jan 2001, Keith Owens wrote: > People need to realise that the problem is initialisation order, > nothing more, nothing less. You have to determine and document the > startup requirements for your code. This is true. But I'd also agree with the implication which you probably didn't mean to make there - that initialisation order is a problem. :) Where an init ordering is required, it must be documented and the Makefiles set up accordingly. But I believe that we should also try to avoid requiring such ordering where possible, too. > It is no wonder that people have problems getting the initialisation > order correct. True. And while that situation continues, I desire to avoid the issue completely by not having any dependencies on init order. > I want to completely remove this multi layered method for setting > initialisation order and go back to basics. I want the programmer to > say "initialise E and F after G, H and I". The kernel build system > works out the directed graph of initialisation order then controls the > execution of startup code to satisfy this graph. But the fewer such constraints there are, the better. We don't want everyone starting to impose unnecessary link order restrictions instead of thinking about the code a little more and just eliminating them completely. > A typical graph would have scsi disk depends on scsi host adaptor group > which depends on pci. No. sd will happily take over any existing devices when as and when they arrive. It doesn't have to be loaded last. Likewise, in theory at least, host adaptor drivers using the new PCI driver code would respond correctly to the PCI code being initialised (and calling their ->probe routine) later, although that doesn't happen now. Why have these dependencies where they're not necessary? > Within the scsi host adaptor group you might need to initialise one > driver before another, so just declare the few inter-driver > dependencies. And there are few. And there should _remain_ few. We shouldn't start imposing link order restrictions on other code which doesn't really need it. > Most of the objects have fairly simple execution dependencies, e.g. > all file systems depend on core fs code having already executed. Er... Why? They call register_filesystem() which uses a lock which is staticly initialised. Don't order stuff for the sake of it. If there are filesystems which _do_ require the VFS to be initialised first, those filesystems can be marked as such. I'm not aware of any. > [1] vfat is one obvious exception, it needs dos first. Also the first > few built in file systems must execute in a defined order because > that in turn controls the probe order for mount. But this order > should be explicitly declared, not as a side effect of the line > order in fs/Makefile. If the fat_inode_hashtable were staticly initialised, that one wouldn't be necessary either. But unfortunately that would be quite ugly. I don't want to get involved in the link order stuff. I would very much like to avoid having such dependencies. All I want is weak symbols. I want to call a function directly if it's present, and if not I want to attempt to load its module. Without having my probe module have a hard dependency on _all_ the submodules it may decide to ask for. get_module_symbol() did that for me, and was perfectly acceptable. I needed to use EXPORT_SYMBOL_NOVERS to export the functions for the individual command sets. Big deal. inter_module_get() almost does that for me, but it imposes link order dependencies, which I want to avoid because I think in this case they're not necessary. All I want is a way to staticly add entries to the inter_module_xxx tables at compile time, because I _have_ done the analysis, and I'm saying that's when they should be made available. Alternatively, show me how to get weak symbols actually working on all architectures, and I'll forget the whole thing and be happy. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-12 10:27 ` David Woodhouse @ 2001-01-12 11:55 ` Keith Owens 2001-01-12 13:40 ` David Woodhouse 0 siblings, 1 reply; 41+ messages in thread From: Keith Owens @ 2001-01-12 11:55 UTC (permalink / raw) To: David Woodhouse; +Cc: Ingo Oeser, Alan Cox, List Linux-Kernel On Fri, 12 Jan 2001 10:27:34 +0000 (GMT), David Woodhouse <dwmw2@infradead.org> wrote: >On Fri, 12 Jan 2001, Keith Owens wrote: >> A typical graph would have scsi disk depends on scsi host adaptor group >> which depends on pci. > >No. sd will happily take over any existing devices when as and when they >arrive. It doesn't have to be loaded last. Likewise, in theory at least, >host adaptor drivers using the new PCI driver code would respond correctly >to the PCI code being initialised (and calling their ->probe routine) >later, although that doesn't happen now. You just proved my point. It is extremely difficult to deduce the required initialisation order by reading an undocumented Makefile where the init order is implemented as a side effect of selection order. The existing method implies link order when none is required. >> Most of the objects have fairly simple execution dependencies, e.g. >> all file systems depend on core fs code having already executed. > >Er... Why? They call register_filesystem() which uses a lock which is >staticly initialised. Don't order stuff for the sake of it. If there are >filesystems which _do_ require the VFS to be initialised first, those >filesystems can be marked as such. I'm not aware of any. I was using scsi and fs as examples, no need to pick holes in the examples. But ... fs/buffer.c:module_init(bdflush_init) fs/pipe.c:module_init(init_pipe_fs) fs/fcntl.c:module_init(fasync_init) fs/locks.c:module_init(filelock_init) fs/dnotify.c:module_init(dnotify_init) I'm no filesystem expert but it appears that some of those core initialisation routines must be executed before most filesystems can start. In particular, filelock_init() must be executed before any filesystem that supports locks is initialised. >All I want is a way to staticly add entries to the >inter_module_xxx tables at compile time, because I _have_ done the >analysis, and I'm saying that's when they should be made available. Firstly inter_module_xxx is only used by that very small subset of code where there are no constraints on whether the caller and callee can be in kernel, in modules or a mixture _and_ some of the objects are optional. It is a special case because most code handles this problem through CONFIG options. If X needs (Y, Z) and X is built into the kernel then (Y, Z) must also be built into the kernel. If Y or Z are optional then control the calls to those functions with CONFIG options. Almost all of the kernel handles it properly though CONFIG, so inter_module_xxx is already a process to handle a few special cases. Secondly you want static inter_module_xxx tables for a small subset of these special cases, where the source has no other initialisation code. This is piling special cases on special cases. Adding static inter_module_xxx tables requires * changes to linux/modules.h to define the new table format and * changes to vmlinux.lds for _every_ architecture to bring all the static tables together in vmlinux and * new initialisation code in module.c to read and load all the static tables at boot time and * extra code in modutils to find any static tables in modules and * an extension to struct modules to let modutils pass information about the static tables to the kernel and * the kernel code will only work with an upgraded modutils. That is a lot of work for a very few special cases. OTOH, you could just swap the order of 3 lines in drivers/mtd/Makefile. Guess which alternative I am going for? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-12 11:55 ` Keith Owens @ 2001-01-12 13:40 ` David Woodhouse 0 siblings, 0 replies; 41+ messages in thread From: David Woodhouse @ 2001-01-12 13:40 UTC (permalink / raw) To: Keith Owens; +Cc: Ingo Oeser, Alan Cox, List Linux-Kernel kaos@ocs.com.au said: > You just proved my point. It is extremely difficult to deduce the > required initialisation order by reading an undocumented Makefile > where the init order is implemented as a side effect of selection > order. The existing method implies link order when none is required. I agree entirely. But you're confusing the debate on how to satisfy init order dependencies with my desire to avoid them altogether (for certain situations). kaos@ocs.com.au said: > I'm no filesystem expert but it appears that some of those core > initialisation routines must be executed before most filesystems can > start. In particular, filelock_init() must be executed before any > filesystem that supports locks is initialised. No, before any filesystem that supports locks is _mounted_. Big difference. But I'm picking holes in your examples again. I accept that in some cases it is necessary, but I still think it's best to avoid it where possible. kaos@ocs.com.au said: > Firstly inter_module_xxx is only used by that very small subset of > code where there are no constraints on whether the caller and callee > can be in kernel, in modules or a mixture _and_ some of the objects > are optional. It is a special case because most code handles this > problem through CONFIG options. If X needs (Y, Z) and X is built into the > kernel then (Y, Z) must also be built into the kernel. If Y or Z are > optional then control the calls to those functions with CONFIG options. > Almost all of the kernel handles it properly though CONFIG, so > inter_module_xxx is already a process to handle a few special cases. 'properly through CONFIG'? I thought you agreed that doing it through preprocessor options was ugly, and that it was preferable to get rid of such things and deal with the presence or absence of such code cleanly through some mechanism with similar semantics to inter_module_get(). The 'special case' code to which you refer is in fact the first set of such code to attempt to deal with this properly rather than just giving up and hacking it with preprocessor options. Stepping back a moment and considering it, what we actually appear to be doing is trying to reinvent weak symbols, as far as I can tell. kaos@ocs.com.au said: > Secondly you want static inter_module_xxx tables for a small subset > of these special cases, where the source has no other initialisation > code. This is piling special cases on special cases. Adding static > inter_module_xxx tables requires I want the registration method not to impose init order restrictions where previously there were none. Where the previous code already had init order ugliness, it's not so much of a problem. But replacing the safe get_module_symbol() with the unsafe inter_module_get() just because the AGP author forgot to use EXPORT_SYMBOL_NOVERS to export the symbols doesn't really strike me as being useful. The only real difference between the two, other than the module symbol mangling, appears to be that inter_module_get() looks stuff up in a dynamic table which is made at runtime, where get_module_symbol() looks it up in a static table which is made at compile (or module load) time. That was a backward step, and wasn't necessary. If you object to having both the runtime inter_module_register() and the compile-time one, then ditch the dynamic one. Both current users of inter_module_xxx would be able to use the static version. But to be honest, I'd settle for ditching the whole blinkin' lot and getting weak symbols working right, if I could. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-12 2:30 ` Keith Owens 2001-01-12 10:27 ` David Woodhouse @ 2001-01-12 12:01 ` Daniel Phillips 2001-01-12 12:18 ` Keith Owens 1 sibling, 1 reply; 41+ messages in thread From: Daniel Phillips @ 2001-01-12 12:01 UTC (permalink / raw) To: Keith Owens, linux-kernel Keith Owens wrote: > I want to completely remove this multi layered method for setting > initialisation order and go back to basics. I want the programmer to > say "initialise E and F after G, H and I". The kernel build system > works out the directed graph of initialisation order then controls the > execution of startup code to satisfy this graph. I don't doubt you will come up with a workable solution at build time. However, working out a valid graph at execution time is trivial and efficient, given a list of precedence relations of the kind you're suggesting. In fact you don't even have to work out the graph before starting the initialization, it's also trivial to keep a count of unsatisfied initialization conditions at the beginning of each initialization sequence and block until the count goes to zero. (In essence, evaluate a priority sort on the fly.) -- Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-12 12:01 ` Daniel Phillips @ 2001-01-12 12:18 ` Keith Owens 2001-01-14 10:16 ` Kai Henningsen 0 siblings, 1 reply; 41+ messages in thread From: Keith Owens @ 2001-01-12 12:18 UTC (permalink / raw) To: Daniel Phillips; +Cc: linux-kernel On Fri, 12 Jan 2001 13:01:12 +0100, Daniel Phillips <phillips@innominate.de> wrote: >Keith Owens wrote: >> I want to completely remove this multi layered method for setting >> initialisation order and go back to basics. I want the programmer to >> say "initialise E and F after G, H and I". The kernel build system >> works out the directed graph of initialisation order then controls the >> execution of startup code to satisfy this graph. > >I don't doubt you will come up with a workable solution at build time. >However, working out a valid graph at execution time is trivial and >efficient, given a list of precedence relations of the kind you're >suggesting. In fact you don't even have to work out the graph before >starting the initialization, it's also trivial to keep a count of >unsatisfied initialization conditions at the beginning of each >initialization sequence and block until the count goes to zero. (In >essence, evaluate a priority sort on the fly.) It is safer to evaluate the graph at link time in case somebody mistakenly codes a dependency loop, that is an abort case. Finding that you have a loop at load time and aborting the kernel is a little too drastic for my tastes. In any case it is academic. Linus insists on link order being explicitly and completely specified by the programmer and he likes the link order being implied by Makefile order. So there is no point in working on a better system. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-12 12:18 ` Keith Owens @ 2001-01-14 10:16 ` Kai Henningsen 0 siblings, 0 replies; 41+ messages in thread From: Kai Henningsen @ 2001-01-14 10:16 UTC (permalink / raw) To: linux-kernel kaos@ocs.com.au (Keith Owens) wrote on 12.01.01 in <31714.979301892@ocs3.ocs-net>: > On Fri, 12 Jan 2001 13:01:12 +0100, > Daniel Phillips <phillips@innominate.de> wrote: > >Keith Owens wrote: > >> I want to completely remove this multi layered method for setting > >> initialisation order and go back to basics. I want the programmer to > >> say "initialise E and F after G, H and I". The kernel build system > >> works out the directed graph of initialisation order then controls the > >> execution of startup code to satisfy this graph. > > > >I don't doubt you will come up with a workable solution at build time. > >However, working out a valid graph at execution time is trivial and > >efficient, given a list of precedence relations of the kind you're > >suggesting. In fact you don't even have to work out the graph before > >starting the initialization, it's also trivial to keep a count of > >unsatisfied initialization conditions at the beginning of each > >initialization sequence and block until the count goes to zero. (In > >essence, evaluate a priority sort on the fly.) > > It is safer to evaluate the graph at link time in case somebody > mistakenly codes a dependency loop, that is an abort case. Finding > that you have a loop at load time and aborting the kernel is a little > too drastic for my tastes. > > In any case it is academic. Linus insists on link order being > explicitly and completely specified by the programmer and he likes the > link order being implied by Makefile order. So there is no point in > working on a better system. I'm not so sure about that. I think it _should_ be possible to do both, and get better documentation at the same time. How about this: * Invent some method for modules to declare these dependencies. Maybe even in /** */ type comments, so it goes right into the documentation. * Write a program to collect all dependencies, do a tsort on them (alerting the developer if a loop is found), determining a reasonable initialization order, and spitting out the relevant Makefiles or Makefile fragments to *create* that initialization order the Linus way. Or in other words, don't do it at runtime, don't do it at compile time, do it at develop time. (Or patch time, perhaps.) MfG Kai - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-11 12:46 ` Keith Owens 2001-01-11 13:09 ` Alan Cox @ 2001-01-11 13:25 ` David Woodhouse 1 sibling, 0 replies; 41+ messages in thread From: David Woodhouse @ 2001-01-11 13:25 UTC (permalink / raw) To: Keith Owens; +Cc: List Linux-Kernel kaos@ocs.com.au said: > So you want two services, one static for code that does not do any > initialisation and one dynamic for code that does do initialisation. > Can you imagine the fun when somebody adds startup code to a routine > that was using static registration? Oh come on. If you change a module from being 'self-contained' and registered at compile time to requiring initialisation it's hardly unreasonable to expect you switch the registration too. Besides, I'm not going to allow any link order dependencies into code I maintain without them being impossible to avoid - and if anyone's thought about the problem hard enough to convince me to accept such a change, they'll have noticed the need to change the registration. > Oh dear, I added init code so I have to remember to change from static > to dynamic registration, and that affects the link order so now I have > to tweak the Makefile. cf. "Oh dear, I added init code but put it _after_ the registration instead of before, so stuff blows up." Neither of these two programmers will get their code into anything I maintain. cf. "Oh dear, I need registration, but I have to remember that inter_module_xxx can't do static registration so now I have to tweak the Makefile." kaos@ocs.com.au said: > Stick to one method that works for all routines, dynamic registration. It doesn't work for all routines. It introduces unnecessary brokenness - link order dependencies - where previously there were none. > If that imposes the occasional need for a couple of extra calls in > some routines and for people to think about initialisation order right > from the start then so be it, it is a small price to pay for long term > stability and ease of maintenance. I'm thinking about link order. If I _wasn't_ thinking about link order, then I'd just put the lines in the 'right' order in the Makefile and put up with it. But I'm thinking about it, and I object to it. It is absolutely unnecessary in this case. As far as I'm concerned, fixing the registration problems introduced by the dynamic inter_module_register is a small price to pay for long term stability and ease of maintenance :) -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
[parent not found: <3A5EFC56.F1A5BCE0@mira.net>]
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? [not found] <3A5EFC56.F1A5BCE0@mira.net> @ 2001-01-12 19:11 ` Christian Zander 2001-01-13 1:11 ` Keith Owens 0 siblings, 1 reply; 41+ messages in thread From: Christian Zander @ 2001-01-12 19:11 UTC (permalink / raw) To: linux-kernel; +Cc: Keith Owens [-- Attachment #1: Type: text/plain, Size: 2777 bytes --] > >> I ran into this while hacking the Nvidia kernel driver to work with > >> 2.4.0. I got the driver working but it's not 100% > >> > >> Also where did get_module_symbol() and put_module_symbol() go? > > > >Patches for the NVIDIA binary X drivers following all these kernel > >changes can be gotten from IRC server irc.openprojects.net, channel > >#nvidia. Or from http://ex.shafted.com.au/nvidia/ > > And what a pile of crud those patches are!! Instead of using the clean > replacement interface for get_module_symbol, nvidia/patch-2.4.0-PR hard > codes the old get_module_symbol algorithm as inline code. > > This patch violates the modules interface by accessing modules.c > internal data. It still suffers from all the problems that > get_module_symbol had. Because it is hard coded as inline code instead > of a common function, will be much harder to fix when it breaks. > The way I understand the inter_module mechanism, module A registers one or several of its symbols using inter_module_register to make it or them available to other modules. Module B can then request any of the symbols with inter_module_request and get a pointer. The inter_module mechanism guarantees that the symbol will be available until module B decides that the symbol is no longer needed and releases it by calling inter_module_put. Saying that I should have made use of this mechanism for the specific code in the Nvidia driver that we are talking about clearly shows that you didn't look at it. The module used get_module_symbol to search its own symbol table for parameters that may have been passed to it at load time. Arguably, this is bad practise, but it is also the reason why using your mechanism doesn't make any sense. Obviously, the module wouldn't want to register private data to request it later on; the information that would have to be passed to inter_module_register is the same that the code in question intends to retrieve in the first place. Contrary to what you're saying, the patch does not just inline the old get_module_symbol algorithm nor does it access any of module.c's internal data. What is does is to browse the list of the modules's _own_ symbols looking for a match. If it finds one, it returns the desired data. > Whoever coded that patch should be taken out and shot, hung, drawn and > quartered then forced to write COBOL for the rest of their natural > life. Excellent comment - it is just as appropriate as it is helpful. -- ---------------------------------------------------------------------- christian zander we come to bury dos, not to praise it. zander@hdz.uni-dortmund.de -- paul vojta ---------------------------------------------------------------------- [-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --] ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-12 19:11 ` Christian Zander @ 2001-01-13 1:11 ` Keith Owens 2001-01-13 10:46 ` David Woodhouse 2001-01-13 11:46 ` Christian Zander 0 siblings, 2 replies; 41+ messages in thread From: Keith Owens @ 2001-01-13 1:11 UTC (permalink / raw) To: Christian Zander; +Cc: linux-kernel, David Woodhouse On Fri, 12 Jan 2001 20:11:30 +0100, Christian Zander <phoenix@minion.de> wrote: >Saying that I should have made use of this mechanism for the specific >code in the Nvidia driver that we are talking about clearly shows that >you didn't look at it. The module used get_module_symbol to search its >own symbol table for parameters that may have been passed to it at load >time. My apologies. I read the patch, not the full source code and the patch does not have enough programming context to show that the driver is only searching its own symbol space. In my own defense, the references to spinlock_t unload_lock and MOD_CAN_QUERY(mp) in the patch are highly misleading, those statements only make sense when you are looking at a symbol table for another module. When searching your own symbol table the current module must be live with a non-zero use count, not being unloaded and it can always be queried. >Contrary to what you're saying, the patch does not just inline the old >get_module_symbol algorithm nor does it access any of module.c's internal >data. unload_lock and MOD_CAN_QUERY were copied verbatim from the old get_module_symbol, even though they are completely unnecessary. That looks like inlining the old algorithm to me. struct module_symbol, mp->nsyms and mp->syms are module.c internal data. If it is ever necessary to change those structures, nothing outside module.c, the 32/64 handlers for module system calls and modutils should be affected. Now if I change module_symbol, other bits of the kernel will unexpectedly break, this is not good. >> Whoever coded that patch should be taken out and shot, hung, drawn and >> quartered then forced to write COBOL for the rest of their natural >> life. > >Excellent comment - it is just as appropriate as it is helpful. Over emphasis for humorous effect. Must remember to add smiley. What this patch and David Woodhouse's comments show is that I need to look at a generic and safe mechanism for kernel/module symbol lookup. The existing static mechanism works for fixed symbol names but does not work for symbol names that are generated at run time nor for symbols that may or may not be present. get_module_symbol() "worked" but was horribly unsafe. It broke with module versions, it did zero type checking which left the code open to version skew and it assumed that all addresses are equivalent to an unsigned long. That last point is especially important for IA64 where function pointers do not reference the function directly, instead they point to a function descriptor with two fields, one of which is the function address. Casting the unsigned long address of a function into a function pointer fails miserably on IA64, and gcc does not even give any warnings. foo = (int (*)(int))get_module_symbol(NULL, "funcname") is architecture dependent. Using EXPORT_SYMBOL_NOVERS() to "fix" the modversions problem for get_module_symbol() removes all inter module checks on the relevant symbols. Not just for the caller of get_module_symbol for all modules that access those symbols. This leaves too much code open to version skew and is not acceptable. inter_module_xxx is modversions safe. It still does no type checking because it uses void * for the data structure, but the exporter and user have to declare their common data area which reduces the chance of version skew. I am still not happy about this possibility of skew but anything is better than no checks at all. Passing a data structure which contains real declarations for function pointers instead of assuming you can cast a number to a function pointer makes inter_module_xxx architecture independent. I will look at a general kernel and module symbol lookup routine that does the job properly. The hard part is making sure that the provider and consumer have exactly the same types for a symbol. Both get_module_symbol and inter_module_xxx completely bypass the modversions checks and are wide open to undetectable version skew, although inter_module_xxx is a little bit safer. Any replacement for these functions must be able to do type checking at run time, which probably means it is 2.5 code. And yes, David, it should be able to handle static data. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-13 1:11 ` Keith Owens @ 2001-01-13 10:46 ` David Woodhouse 2001-01-13 12:06 ` Keith Owens 2001-01-13 11:46 ` Christian Zander 1 sibling, 1 reply; 41+ messages in thread From: David Woodhouse @ 2001-01-13 10:46 UTC (permalink / raw) To: Keith Owens; +Cc: Christian Zander, linux-kernel, alan On Sat, 13 Jan 2001, Keith Owens wrote: > Over emphasis for humorous effect. Must remember to add smiley. Heh. But it does deserve to get into the fortune file. > What this patch and David Woodhouse's comments show is that I need to > look at a generic and safe mechanism for kernel/module symbol lookup. > The existing static mechanism works for fixed symbol names but does not > work for symbol names that are generated at run time nor for symbols > that may or may not be present. Actually, my testing showed that modutils was quite OK with symbols which may or may not be present. But compiling such code into the kernel, at least on MIPS and m68k, didn't work. cat >weaktest.c <<EOF #include <linux/module.h> extern char *myfun(void) __attribute__((weak)); int init_module(void) { char *txt= "myfun() not present\n"; if (myfun) txt = myfun(); printk(txt); return 0; } EOF cat > myfun.c <<EOF #include <linux/module.h> char *myfun(void) { return "Hello World\n"; } EOF I doubt this would have implemented weak symbols completely, though. Fixing up the reference in weaktest.o if myfun.o was loaded later, etc. And we don't really want to 'fix' that either. So it'd still have needed request_module(); get_module_symbol() if it found that myfun wasn't present and it needed it. So it might as well have used get_module_symbol() from the start instead of the weak declaration. > get_module_symbol() "worked" but was horribly unsafe. It broke with > module versions, it did zero type checking which left the code open to > version skew and it assumed that all addresses are equivalent to an > unsigned long. > > That last point is especially important for IA64 where function > pointers do not reference the function directly, instead they point to > a function descriptor with two fields, one of which is the function > address. Casting the unsigned long address of a function into a > function pointer fails miserably on IA64, and gcc does not even give > any warnings. foo = (int (*)(int))get_module_symbol(NULL, "funcname") > is architecture dependent. But fixable. > Using EXPORT_SYMBOL_NOVERS() to "fix" the modversions problem for > get_module_symbol() removes all inter module checks on the relevant > symbols. Not just for the caller of get_module_symbol for all modules > that access those symbols. This leaves too much code open to version > skew and is not acceptable. I'm not sure there's anything which was expected to be obtained by get_module_symbol() which was also obtained by normal linking. The nature of these routines is that they're optional. Usually, the routine would be optional for all callers or it'd be mandatory for all callers. Rarely a mixture. But what about keeping a separate table, with PUBLISH_SYMBOL() or something slightly more sensibly named? That way, get_published_symbol() can only get at those symbols which were supposed to be listed, and if someone really wants EXPORT_SYMBOL() and PUBLISH_SYMBOL() they can do that. > > inter_module_xxx is modversions safe. It still does no type checking > because it uses void * for the data structure, but the exporter and > user have to declare their common data area which reduces the chance of > version skew. I'm not sure I follow. The exporter and the user have to each declare their common data area, which means they don't have to declare it identically, and there's just as much chance of version skew as before, surely? With get_module_symbol both had to declare it, too. And 'modversions safe' just means that there's no attempt to mangle the names so it's identical to the EXPORT_SYMBOL_NOVERS case above? > I am still not happy about this possibility of skew but > anything is better than no checks at all. Passing a data structure > which contains real declarations for function pointers instead of > assuming you can cast a number to a function pointer makes > inter_module_xxx architecture independent. On the other hand, it's also simple enough to define a macro which does the arch-dependent equivalent of the 'foo=(fnptr_t)get_module_symbol()' above, isn't it? But I'm not particularly attached to the method. The static initialisation is what I miss. > I will look at a general kernel and module symbol lookup routine that > does the job properly. Thanks. > The hard part is making sure that the provider and consumer have > exactly the same types for a symbol. That would be useful, but it's not 100% imperative. Runtime type-checking is a nice sanity check where it's almost free, as it was in the modversions case, but I don't want to program in Java. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-13 10:46 ` David Woodhouse @ 2001-01-13 12:06 ` Keith Owens 2001-01-13 15:09 ` David Woodhouse 0 siblings, 1 reply; 41+ messages in thread From: Keith Owens @ 2001-01-13 12:06 UTC (permalink / raw) To: David Woodhouse; +Cc: Christian Zander, linux-kernel, alan On Sat, 13 Jan 2001 10:46:44 +0000 (GMT), David Woodhouse <dwmw2@infradead.org> wrote: >On Sat, 13 Jan 2001, Keith Owens wrote: >Actually, my testing showed that modutils was quite OK with symbols which >may or may not be present. But compiling such code into the kernel, at >least on MIPS and m68k, didn't work. Weak symbols were added to gcc somewhere between 2.7.2.3 and 2.91.66. At least two architectures are using versions of gcc that predate (by a few days) the addition of weak symbols. Davem hit this problem on sparc with the weak references to kallsyms, he had to define the symbols instead of letting them resolve to zero, gcc on sparc silently ignored weak. >I doubt this would have implemented weak symbols completely, though. >Fixing up the reference in weaktest.o if myfun.o was loaded later, etc. >And we don't really want to 'fix' that either. Weak is not enough. We need dynamic symbol binding if we plan to support a cooperative model for objects instead of a strict hierarchic model. BTW, modutils cannot automatically fill in upward references when a module is loaded. A reference is a use count, an automatic reference would be an automatic use count with no way of removing it. Code that calls upwards to a symbol must perform an overt action to get the reference and cope with the case when the symbol is not there. Think of it as a probe, "do I have facility XXX yet?". It is up to the caller to probe as often as required. Hot plugging for symbols, wheee! >> That last point is especially important for IA64 where function >> pointers do not reference the function directly, instead they point to >> a function descriptor with two fields, one of which is the function >> address. Casting the unsigned long address of a function into a >> function pointer fails miserably on IA64, and gcc does not even give >> any warnings. foo = (int (*)(int))get_module_symbol(NULL, "funcname") >> is architecture dependent. > >But fixable. Probably not. The generated IA64 object code for this case is completely wrong, not surprising since we are lying to gcc. x.o: file format elf64-ia64-little Disassembly of section .text: 0000000000000000 <test1>: unsigned long value = 0xc0002000; // example, not a real ia64 address void test1(void) { 0: [MII] alloc r34=ar.pfs,4,4,0 void (*test2)(void); 6: mov r35=r12 // &test2 c: adds r12=-16,r12 // adjust stack pointer 10: [MII] nop.m 0x0 test2 = (void (*)(void))value; 16: mov r33=b0 // save return address 12: GPREL22 value 1c: addl r14=0,r1;; // &value 20: [MMI] ld8 r14=[r14];; // value 26: st8 [r35]=r14 // test2 = value 0xc0002000 2c: nop.i 0x0 test2(); 30: [MII] ld8 r14=[r35] // read test2, 0xc0002000 36: mov r32=r1;; // save current global pointer 3c: nop.i 0x0 40: [MII] ld8 r15=[r14] // dereference test2, wrong 46: adds r14=8,r14;; // test2 + 8, 0xc0002008 4c: nop.i 0x0 50: [MII] ld8 r1=[r14] // new gp from 0xc0002008, wrong 56: mov b6=r15;; // & target function, wrong 5c: nop.i 0x0 60: [MIB] nop.m 0x0 66: nop.i 0x0 6c: br.call.sptk.many b0=b6;; // call indirect function, oops 70: [MII] mov r1=r32 // restore gp By casting 'test2 = (void (*)(void))value' we claim that value is the address of the the function descriptor which must contain { actual address of function, global data pointer for function } gcc trusts us and tries to use the data at location 0xc0002000 as a function descriptor. Because get_module_symbol() returns the address of the first instruction in a function, that code would load the actual address and global pointer from the first 16 bytes of the function's code area. Needless to say, it does not work. Fixing this would mean tweaking get_module_symbol() on IA64 to recognise that the symbol is a function, build a function descriptor on the fly and return the address of the descriptor. But the information about the types of symbols is not available in the kernel nor in modules after they are loaded, get_module_symbol() cannot differentiate between functions and anything else. There is also the small matter of grubbing around in the arch dependent bit of struct modules to find the global pointer for the target function, more complexity. >But what about keeping a separate table, with PUBLISH_SYMBOL() or >something slightly more sensibly named? That way, get_published_symbol() >can only get at those symbols which were supposed to be listed, and if >someone really wants EXPORT_SYMBOL() and PUBLISH_SYMBOL() they can do >that. I don't see the point. EXPORT_SYMBOL() says that the symbol can be accessed by anybody. The current hierarchical binding model restricts access to modules that load after this module. If we remove the strict hierarchical binding of module symbols, why worry whether the caller is above or below this module? IOW, there is no need for a different definition that says the symbol can also be accessed by the kernel or by earlier module loads. Upward references to a symbol are another story. >> > inter_module_xxx is modversions safe. It still does no type checking >> because it uses void * for the data structure, but the exporter and >> user have to declare their common data area which reduces the chance of >> version skew. > >I'm not sure I follow. The exporter and the user have to each declare >their common data area, which means they don't have to declare it >identically, and there's just as much chance of version skew as before, >surely? With get_module_symbol both had to declare it, too. get_module_symbol() was usually used with a cast from unsigned long to some type that was defined in the calling .c file. That made the caller code responsible for using the correct declaration. It is better to define interfaces as shared data in a shared header. Not perfect, but better. >And 'modversions safe' just means that there's no attempt to mangle the >names so it's identical to the EXPORT_SYMBOL_NOVERS case above? With inter_module_xxx you have no type checking on the registered data. But you do not have to use EXPORT_SYMBOL_NOVERS on the shared symbols which means that any other users of the symbols will still get type checking. Again, not perfect, but better. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-13 12:06 ` Keith Owens @ 2001-01-13 15:09 ` David Woodhouse 2001-01-13 19:03 ` Russell King ` (2 more replies) 0 siblings, 3 replies; 41+ messages in thread From: David Woodhouse @ 2001-01-13 15:09 UTC (permalink / raw) To: Keith Owens; +Cc: Christian Zander, linux-kernel, alan On Sat, 13 Jan 2001, Keith Owens wrote: > BTW, modutils cannot automatically fill in upward references when a > module is loaded. A reference is a use count, an automatic reference > would be an automatic use count with no way of removing it. Code that > calls upwards to a symbol must perform an overt action to get the > reference and cope with the case when the symbol is not there. Think > of it as a probe, "do I have facility XXX yet?". It is up to the > caller to probe as often as required. Hot plugging for symbols, wheee! We don't need to overdesign it. get_module_symbol() basically provided this for us. The only thing really wrong with it was the lack of use count handling, which I fixed a while ago. Lack of runtime typechecking isn't a showstopper. Otherwise we'd have thrown out GCC by now and rewritten the kernel in Modula-3. That leaves the IA64 problem. > Fixing this would mean tweaking get_module_symbol() on IA64 to > recognise that the symbol is a function, build a function descriptor on > the fly and return the address of the descriptor. But the information > about the types of symbols is not available in the kernel nor in > modules after they are loaded, get_module_symbol() cannot differentiate > between functions and anything else. There is also the small matter of > grubbing around in the arch dependent bit of struct modules to find the > global pointer for the target function, more complexity. This is already handled by modutils, presumably. How? By 'grubbing arouund in the arch dependent bit of struct modules'? There's already a handful of gp handling surrounded by #ifdef __alpha__ in module.c which doesn't seem too unreasonable. The caller knows what it's expecting to find. How about separate get_module_function() and get_module_data() routines? Which are identical on most architectures, but on (Alpha and?) IA64 could be defined to return an appropriate structure. > get_module_symbol() was usually used with a cast from unsigned long to > some type that was defined in the calling .c file. That made the > caller code responsible for using the correct declaration. It is > better to define interfaces as shared data in a shared header. Not > perfect, but better. We could quite happily define the function type in a shared header file, and coding the original function and the subsequent cast from get_module_symbol() using that definition. Conversely, nothing stops you from using only local definitions for the inter_module_xxx objects, rather than a single shared definition. Nothing's changed. You just changed the users to use shared definitions while you converted them to inter_module_xxx. But there's no fundamental difference in the interface used, in this respect. > With inter_module_xxx you have no type checking on the registered data. > But you do not have to use EXPORT_SYMBOL_NOVERS on the shared symbols > which means that any other users of the symbols will still get type > checking. Again, not perfect, but better. Slightly. But for the cases where inter_module_xxx or get_module_symbol are used, A. AFAIK there are no such 'direct' users who get the benefit of the runtime typechecking. B. The authors are already having to pay attention to any changes in the type of the exported data, rather than just pretending that they're writing Java code and expecting the runtime system to wipe the dribble from their chins. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-13 15:09 ` David Woodhouse @ 2001-01-13 19:03 ` Russell King 2001-01-14 0:21 ` Keith Owens 2001-01-14 4:04 ` Linus Torvalds 2 siblings, 0 replies; 41+ messages in thread From: Russell King @ 2001-01-13 19:03 UTC (permalink / raw) To: David Woodhouse; +Cc: Keith Owens, Christian Zander, linux-kernel, alan David Woodhouse writes: > We don't need to overdesign it. get_module_symbol() basically provided > this for us. The only thing really wrong with it was the lack of use > count handling, which I fixed a while ago. And the fact that it doesn't work if you turn module support off, which you'd want to do on an embedded kernel. Unfortunately, this is one of the times when you do want the MTD stuff. You either have to put up with no MTD support, write your own, or put up with the extra space of module symbols. Therefore, get_module_symbol() as it stood was the wrong interface to use, and I completely agree with Keiths decision to remove it. However, I'm not sure that the inter_* stuff that replaced it is much better for the reasons David has highlighted previously wrt link ordering. _____ |_____| ------------------------------------------------- ---+---+- | | Russell King rmk@arm.linux.org.uk --- --- | | | | http://www.arm.linux.org.uk/personal/aboutme.html / / | | +-+-+ --- -+- / | THE developer of ARM Linux |+| /|\ / | | | --- | +-+-+ ------------------------------------------------- /\\\ | - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-13 15:09 ` David Woodhouse 2001-01-13 19:03 ` Russell King @ 2001-01-14 0:21 ` Keith Owens 2001-01-14 9:43 ` David Woodhouse 2001-01-14 4:04 ` Linus Torvalds 2 siblings, 1 reply; 41+ messages in thread From: Keith Owens @ 2001-01-14 0:21 UTC (permalink / raw) To: David Woodhouse; +Cc: linux-kernel On Sat, 13 Jan 2001 15:09:40 +0000 (GMT), David Woodhouse <dwmw2@infradead.org> wrote: >Lack of [module symbol] runtime typechecking isn't a showstopper. It is when users try to insert modules from kernel A into kernel B when the ABI changed between A and B. This is not type checking to catch kernel programmers, it is ABI checking to catch user errors. This is becoming more important as the kernel moves towards hot plugging devices, especially for binary only drivers. It is far better for the kernel community if modutils can say "cannot load module foo because its interfaces do not match the kernel, upgrade module foo". That forces the maintenance load back onto the binary supplier and removes the questions from l-k, including many of the oops reports with binary only drivers in the module list. Module symbol versions are the only way to catch ABI changes. I do not want to add a mechanism for accessing symbols dynamically if it cannot detect ABI changes, it leaves the kernel open to difficult to diagnose user errors. I'm doing the hard work now to save everybody time later. Ignore the fact that the existing module symbol version implementation is broken as designed. http://gear.torque.net/kbuild/archive/1280.html lists the major problems with make dep, genksyms has all those problems plus several of its own. As part of the Makefile rewrite for 2.5, I am redesigning module symbol versions from scratch. I agree that inter_module_xxx does not check ABI. That was not for lack of trying, but it cannot be done in 2.4, it needs a major redesign of module symbols and the makefiles. It will be possible in 2.5. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-14 0:21 ` Keith Owens @ 2001-01-14 9:43 ` David Woodhouse 2001-01-14 10:05 ` Keith Owens 0 siblings, 1 reply; 41+ messages in thread From: David Woodhouse @ 2001-01-14 9:43 UTC (permalink / raw) To: Keith Owens; +Cc: linux-kernel On Sun, 14 Jan 2001, Keith Owens wrote: > This is becoming more important as the kernel moves towards hot > plugging devices, especially for binary only drivers. It is far better > for the kernel community if modutils can say "cannot load module foo > because its interfaces do not match the kernel, upgrade module foo". > That forces the maintenance load back onto the binary supplier and > removes the questions from l-k, including many of the oops reports with > binary only drivers in the module list. No. The correct response to that is _already_ "You have a binary-only module. Even in the kernel it was compiled against, you are not supported. Goodbye". To quote our Lord and Master: (http://lwn.net/1999/0211/a/lt-afs.html) >> I will strive for binary compatibility for modules, but I _expect_ >> that it will be broken. It's just too easy to have to make changes >> that break binary-only modules, and I have too little incentive to try >> to avoid it. >> >> If people feel this is a problem, I see a few alternatives: >> - don't use stuff with binary-only modules. Just say no. >> - work hard at making a source-version of the thing available (it >> doesn't have to be under the GPL if it's a module, but it has to be >> available as source so that it can be recompiled). >> - don't upgrade >> - drop Linux (http://lwn.net/1999/0211/a/lt-binary.html) >> Basically, I want people to know that when they use binary-only >> modules, it's THEIR problem. I want people to know that in their >> bones, and I want it shouted out from the rooftops. I want people to >> wake up in a cold sweat every once in a while if they use binary-only >> modules. kaos@ocs.com.au wrote: > Ignore the fact that the existing module symbol version implementation > is broken as designed. http://gear.torque.net/kbuild/archive/1280.html > lists the major problems with make dep, genksyms has all those problems > plus several of its own. As part of the Makefile rewrite for 2.5, I am > redesigning module symbol versions from scratch. > > I agree that inter_module_xxx does not check ABI. That was not for > lack of trying, but it cannot be done in 2.4, it needs a major redesign > of module symbols and the makefiles. It will be possible in 2.5. This is a good thing, as long as it doesn't get in the way of real functionality. We don't _need_ to make life easier for people running binary-only modules. But if we can do it without making life harder for real people, then that's nice. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-14 9:43 ` David Woodhouse @ 2001-01-14 10:05 ` Keith Owens 2001-01-14 10:45 ` David Woodhouse 0 siblings, 1 reply; 41+ messages in thread From: Keith Owens @ 2001-01-14 10:05 UTC (permalink / raw) To: David Woodhouse; +Cc: linux-kernel On Sun, 14 Jan 2001 09:43:21 +0000 (GMT), David Woodhouse <dwmw2@infradead.org> wrote: >On Sun, 14 Jan 2001, Keith Owens wrote: >> That forces the maintenance load back onto the binary supplier and >> removes the questions from l-k, including many of the oops reports with >> binary only drivers in the module list. > >No. The correct response to that is _already_ "You have a binary-only >module. Even in the kernel it was compiled against, you are not supported. >Goodbye". I wish that Linus had never agreed to binary only modules. But as long as they are allowed, I want to detect problems with binary only modules before they hit the rest of the kernel and end up as questions on l-k. Note I said allowed, not supported. I refuse to support any binary only modules, my standard response to problems logged against binary modules is "remove them and reproduce the problem". Checking for ABI violations is not supporting binary modules, it is detecting that they are stuffed and telling the user to go pester their supplier instead of polluting l-k with questions that will be ignored. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-14 10:05 ` Keith Owens @ 2001-01-14 10:45 ` David Woodhouse 0 siblings, 0 replies; 41+ messages in thread From: David Woodhouse @ 2001-01-14 10:45 UTC (permalink / raw) To: Keith Owens; +Cc: linux-kernel On Sun, 14 Jan 2001, Keith Owens wrote: > Note I said allowed, not supported. I refuse to support any binary > only modules, my standard response to problems logged against binary > modules is "remove them and reproduce the problem". Checking for ABI > violations is not supporting binary modules, it is detecting that they > are stuffed and telling the user to go pester their supplier instead of > polluting l-k with questions that will be ignored. Sensible. As long as it doesn't give rise to reports of the type "modutils didn't whinge so it's not the binary-only module's fault." -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-13 15:09 ` David Woodhouse 2001-01-13 19:03 ` Russell King 2001-01-14 0:21 ` Keith Owens @ 2001-01-14 4:04 ` Linus Torvalds 2001-01-14 17:46 ` David Woodhouse 2 siblings, 1 reply; 41+ messages in thread From: Linus Torvalds @ 2001-01-14 4:04 UTC (permalink / raw) To: linux-kernel In article <Pine.LNX.4.30.0101131413190.21182-100000@imladris.demon.co.uk>, David Woodhouse <dwmw2@infradead.org> wrote: > >We don't need to overdesign it. get_module_symbol() basically provided >this for us. The only thing really wrong with it was the lack of use >count handling, which I fixed a while ago. NO NO NO! You miss _entirely_ the reason why "get_module_symbol()" was removed, and why I will not _ever_ accept it coming back. Hint #1: get_MODULE_symbol(). Hint #2: compiled in functionality. The fact is, that get_module_symbol() was seriously and totally mis-designed from the very beginning, and it was removed for THAT reason, and it had nothing at all to do with the count handling. So stop this discussion. It's not coming back. Live with the current interfaces. Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-14 4:04 ` Linus Torvalds @ 2001-01-14 17:46 ` David Woodhouse 2001-01-14 19:12 ` Linus Torvalds 0 siblings, 1 reply; 41+ messages in thread From: David Woodhouse @ 2001-01-14 17:46 UTC (permalink / raw) To: Linus Torvalds; +Cc: linux-kernel On 13 Jan 2001, Linus Torvalds wrote: > You miss _entirely_ the reason why "get_module_symbol()" was removed, > and why I will not _ever_ accept it coming back. > > Hint #1: get_MODULE_symbol(). > Hint #2: compiled in functionality. Er,... forgive me if I'm being overly dense here, but I can't see anything fundamentally wrong in the above that wouldn't be fixed with a judicious application of s/module_// But I have no particular attachment to it. All I'm asking for is a way to avoid having init order dependencies where previously there was no need for them, by having a way to put entries in the inter_module_get() table at compile time. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-14 17:46 ` David Woodhouse @ 2001-01-14 19:12 ` Linus Torvalds 2001-01-14 20:02 ` David Woodhouse 0 siblings, 1 reply; 41+ messages in thread From: Linus Torvalds @ 2001-01-14 19:12 UTC (permalink / raw) To: David Woodhouse; +Cc: linux-kernel On Sun, 14 Jan 2001, David Woodhouse wrote: > > But I have no particular attachment to it. All I'm asking for is a way to > avoid having init order dependencies where previously there was no need > for them, by having a way to put entries in the inter_module_get() table > at compile time. Note that previously there _were_ order dependencies. In fact, I consider it very tasteless to have modules that act differently on whether another module is loaded. I saw some arguments saying that this is th "right thing", and I disagree completely. Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-14 19:12 ` Linus Torvalds @ 2001-01-14 20:02 ` David Woodhouse 2001-01-14 20:15 ` Linus Torvalds 0 siblings, 1 reply; 41+ messages in thread From: David Woodhouse @ 2001-01-14 20:02 UTC (permalink / raw) To: Linus Torvalds; +Cc: linux-kernel On Sun, 14 Jan 2001, Linus Torvalds wrote: > Note that previously there _were_ order dependencies. In fact, I consider > it very tasteless to have modules that act differently on whether another > module is loaded. I saw some arguments saying that this is th "right > thing", and I disagree completely. The only valid behaviour I can think of is... if (!feature_present) try_to_load_it(); if (feature_present) use_it(); else if (we_can_live_without()) deal_with_it(); else whinge_and_die(); In this case, it's not really depending on whether the desired feature has been loaded yet or not. It's depending on whether the desired feature is available or not. In this sense, inter_module_get_request() is an improvement, because it makes that explicit. Obviously, in the case where we'd eventually just whinge_and_die(), usually it's best to just make this code depend on whatever feature it is that's being used, by referencing it directly. But in the case of the CFI probe code and also I believe DRM, we don't actually know precisely which feature we're going to require until we've done the hardware probe at runtime. We don't want the generic code having a hard dependency on each possible submodule, and doing it with ifdefs according to what happens to be compiled in is just ugly. So the above logic was useful, and get_module_symbol(), even though it wasn't wonderful, provided it. The reason you didn't get the current CFI code with the rest of the update I gave you for 2.4.0-test12 is because I'm intending to rewrite it first, and hopefully deal with this issue in a better way while I'm at it. But as it stands, the best option I can see is to have the generic probe code have ifdefs on the chipset-specific options, and reference only the ones which are present. It's not the end of the world, and as rmk suggests, many embedded systems are run without module support in production anyway. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-14 20:02 ` David Woodhouse @ 2001-01-14 20:15 ` Linus Torvalds 2001-01-14 21:15 ` David Woodhouse 0 siblings, 1 reply; 41+ messages in thread From: Linus Torvalds @ 2001-01-14 20:15 UTC (permalink / raw) To: David Woodhouse; +Cc: linux-kernel On Sun, 14 Jan 2001, David Woodhouse wrote: > > But in the case of the CFI probe code and also I believe DRM, we don't > actually know precisely which feature we're going to require until we've > done the hardware probe at runtime. That's ok. This is what "request_module()" and "kmod" is all about. Once we probe the hardware, the drievr itself can ask for more drivers. I completely fail to see the arguments that have been brought up for drm doing ugly things. The code should simply do drm_agp_head_t * head = inter_module_get("drm_agp"); if (!head) { request_module("drm-agp"); head = inter_module_get("drm_agp"); if (!head) return -ENOAGP; } and be done with it. THE ABOVE MAKES SENSE. The code says _exactly_ what the module wants to do: it wants to find the AGP support, and if it cannot find the AGP support it wants to load them. The arguments about how the user should load things in some specific order or whatever are complete crap. All the support is there, and whining about it is not going to change my opinion in the least. Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-14 20:15 ` Linus Torvalds @ 2001-01-14 21:15 ` David Woodhouse 2001-01-14 21:47 ` Linus Torvalds 0 siblings, 1 reply; 41+ messages in thread From: David Woodhouse @ 2001-01-14 21:15 UTC (permalink / raw) To: Linus Torvalds; +Cc: linux-kernel On Sun, 14 Jan 2001, Linus Torvalds wrote: > This is what "request_module()" and "kmod" is all about. Once we probe the > hardware, the drievr itself can ask for more drivers. > > I completely fail to see the arguments that have been brought up for drm > doing ugly things. The code should simply do > > drm_agp_head_t * head = inter_module_get("drm_agp"); > > if (!head) { > request_module("drm-agp"); > head = inter_module_get("drm_agp"); > if (!head) > return -ENOAGP; > } > > and be done with it. THE ABOVE MAKES SENSE. The code says _exactly_ what > the module wants to do: it wants to find the AGP support, and if it cannot > find the AGP support it wants to load them. It's the same with CFI command-set-specific code. Except that the command-set specific code didn't previously have to be initialised at all, and now we've got to initialise it (and have it call inter_module_register) before it's required by the cfi_probe code. The difference here is that while drm_agp actually had to do some hardware initialisation, the CFI command set handlers didn't - the only thing their module_init routine does is call inter_module_register(). So we've introduced the init order dependencies where previously they weren't necessary. That's the one flaw in the inter_module_get() stuff - we could do with a way to put entries in the table at _compile_ time, rather than _only_ at run time. While I accept that we can't eliminate init order dependencies completely, I still think we should avoid them where it's possible. Which it would be in this case, without much difficulty at all. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-14 21:15 ` David Woodhouse @ 2001-01-14 21:47 ` Linus Torvalds 2001-01-14 21:57 ` David Woodhouse 2001-01-14 23:00 ` Keith Owens 0 siblings, 2 replies; 41+ messages in thread From: Linus Torvalds @ 2001-01-14 21:47 UTC (permalink / raw) To: David Woodhouse; +Cc: linux-kernel On Sun, 14 Jan 2001, David Woodhouse wrote: > > That's the one flaw in the inter_module_get() stuff - we could do with a > way to put entries in the table at _compile_ time, rather than _only_ at > run time. Ok, I can buy that. Not having to initialize explicitly would be nice, but if so we should make module loading do it automatically too, so that we don't generate unnecessary differences between module and compiled in (ie I'd rather avoid the situation that "if you're a module, you need to explicitly export your inter_module_stuff(), while if you're compiled-in it will be exported automatically"). Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-14 21:47 ` Linus Torvalds @ 2001-01-14 21:57 ` David Woodhouse 2001-01-14 23:00 ` Keith Owens 1 sibling, 0 replies; 41+ messages in thread From: David Woodhouse @ 2001-01-14 21:57 UTC (permalink / raw) To: Linus Torvalds; +Cc: linux-kernel, kaos On Sun, 14 Jan 2001, Linus Torvalds wrote: > On Sun, 14 Jan 2001, David Woodhouse wrote: > > That's the one flaw in the inter_module_get() stuff - we could do with a > > way to put entries in the table at _compile_ time, rather than _only_ > > at run time. > Ok, I can buy that. Not having to initialize explicitly would be nice, but > if so we should make module loading do it automatically too, so that we > don't generate unnecessary differences between module and compiled in (ie > I'd rather avoid the situation that "if you're a module, you need to > explicitly export your inter_module_stuff(), while if you're compiled-in > it will be exported automatically"). Yep. Modutils can probably handle that case without too much difficulty, if we go with sticking the static inter_module_entries in a special ELF section as I originally suggested. Keith? -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-14 21:47 ` Linus Torvalds 2001-01-14 21:57 ` David Woodhouse @ 2001-01-14 23:00 ` Keith Owens 2001-01-15 9:09 ` David Woodhouse 1 sibling, 1 reply; 41+ messages in thread From: Keith Owens @ 2001-01-14 23:00 UTC (permalink / raw) To: Linus Torvalds; +Cc: David Woodhouse, linux-kernel On Sun, 14 Jan 2001 13:47:29 -0800 (PST), Linus Torvalds <torvalds@transmeta.com> wrote: >On Sun, 14 Jan 2001, David Woodhouse wrote: >> That's the one flaw in the inter_module_get() stuff - we could do with a >> way to put entries in the table at _compile_ time, rather than _only_ at >> run time. > >Ok, I can buy that. Not having to initialize explicitly would be nice, but >if so we should make module loading do it automatically too ... It might be nice but it is also expensive. Adding static inter_module_xxx tables requires * changes to linux/modules.h to define the new table format and * changes to vmlinux.lds for _every_ architecture to bring all the static tables together in vmlinux and * new initialisation code in module.c to read and load all the static tables at boot time and * extra code in modutils to find any static tables in modules and * an extension to struct modules to let modutils pass information about the static tables to the kernel and * the kernel code will only work with an upgraded modutils. That is a lot of work for a very few special cases. OTOH, you could just add a few lines of __initcall code in two source files (which I did when I wrote inter_module_xxx) and swap the order of 3 lines in drivers/mtd/Makefile. Guess which alternative I am going for? IMHO any automatic method that relies on ELF sections and/or modutils support is the wrong approach, it is a complex solution with external dependencies when we already have a simple solution with no external dependencies. What next, static tables for file system registration, for device registration? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-14 23:00 ` Keith Owens @ 2001-01-15 9:09 ` David Woodhouse 0 siblings, 0 replies; 41+ messages in thread From: David Woodhouse @ 2001-01-15 9:09 UTC (permalink / raw) To: Keith Owens; +Cc: Linus Torvalds, linux-kernel kaos@ocs.com.au said: > That is a lot of work for a very few special cases. OTOH, you could > just add a few lines of __initcall code in two source files (which I > did when I wrote inter_module_xxx) and swap the order of 3 lines in > drivers/mtd/Makefile. Guess which alternative I am going for? I've already worked round it for the 2.[024] case by reintroducing the ifdefs. I assume here that we're planning for 2.5. As part of the many changes that are going to be introduced during 2.5, this shouldn't be too intrusive. Once it's actually usable for the common case, it won't just be 'a very few special cases' any more. But that's all 2.5 material. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-13 1:11 ` Keith Owens 2001-01-13 10:46 ` David Woodhouse @ 2001-01-13 11:46 ` Christian Zander 2001-01-13 12:23 ` Keith Owens 1 sibling, 1 reply; 41+ messages in thread From: Christian Zander @ 2001-01-13 11:46 UTC (permalink / raw) To: Keith Owens; +Cc: linux-kernel [-- Attachment #1: Type: text/plain, Size: 2015 bytes --] On Sat, Jan 13, 2001 at 12:11:31PM +1100, kaos@ocs.com.au wrote: > My apologies. I read the patch, not the full source code and the patch > does not have enough programming context to show that the driver is > only searching its own symbol space. In my own defense, the references > to spinlock_t unload_lock and MOD_CAN_QUERY(mp) in the patch are highly > misleading, those statements only make sense when you are looking at a > symbol table for another module. When searching your own symbol table > the current module must be live with a non-zero use count, not being > unloaded and it can always be queried. > > >Contrary to what you're saying, the patch does not just inline the old > >get_module_symbol algorithm nor does it access any of module.c's internal > >data. > > unload_lock and MOD_CAN_QUERY were copied verbatim from the old > get_module_symbol, even though they are completely unnecessary. That > looks like inlining the old algorithm to me. > > struct module_symbol, mp->nsyms and mp->syms are module.c internal > data. If it is ever necessary to change those structures, nothing > outside module.c, the 32/64 handlers for module system calls and > modutils should be affected. Now if I change module_symbol, other bits > of the kernel will unexpectedly break, this is not good. I see what you mean. What do you suggest should be done in the context of the driver? As you can easily tell, I'm not overly familiar with the internal workings of the kernel. That and the mere impossibility to get any kind of help at the mere mention of the Nvidia driver module ("go bitch at nvidia", "who cares", ...) do not exactly make it easier to fix problems that arise from changes to the kernel. -- ---------------------------------------------------------------------- christian zander we come to bury dos, not to praise it. zander@hdz.uni-dortmund.de -- paul vojta ---------------------------------------------------------------------- [-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --] ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go? 2001-01-13 11:46 ` Christian Zander @ 2001-01-13 12:23 ` Keith Owens 0 siblings, 0 replies; 41+ messages in thread From: Keith Owens @ 2001-01-13 12:23 UTC (permalink / raw) To: Christian Zander; +Cc: linux-kernel On Sat, 13 Jan 2001 12:46:00 +0100, Christian Zander <phoenix@minion.de> wrote: >I see what you mean. What do you suggest should be done in the context of >the driver? As you can easily tell, I'm not overly familiar with the >internal workings of the kernel. That and the mere impossibility to get >any kind of help at the mere mention of the Nvidia driver module ("go bitch >at nvidia", "who cares", ...) do not exactly make it easier to fix problems >that arise from changes to the kernel. Hmm, can I charge Nvidia for this fix? The only reason you are looking at symbols is to map Nvidia registry names to module symbols. There are 9 registry names, 4 of which are #ifdeffed out. MODULE_PARM(NVreg_resman_debuglevel, "i"); MODULE_PARM(NVreg_VideoMemoryTypeOverride, "i"); #ifdeffed out MODULE_PARM(NVreg_EnableVia4x, "i"); MODULE_PARM(NVreg_ReqAGPRate, "i"); #ifdeffed out MODULE_PARM(NVreg_SkipBiosPost, "i"); #ifdeffed out MODULE_PARM(NVreg_UseKernelAGP, "i"); MODULE_PARM(NVreg_UpdateKernelAGP, "i"); #ifdeffed out MODULE_PARM(NVreg_ReqAGPSBA, "i"); MODULE_PARM(NVreg_ReqAGPFW, "i"); Simple fix is an array to map names to addresses. struct { const char *name; int *value; } linux_registry[] = { { "resman_debuglevel", &NVreg_resman_debuglevel }, { "EnableVia4x", &NVreg_EnableVia4x }, { "UseKernelAGP", &NVreg_UseKernelAGP }, { "ReqAGPSBA", &NVreg_ReqAGPSBA }, { "ReqAGPFW", &NVreg_ReqAGPFW }, }; Changing osRegistryLookup to scan that array for the registry name and return the address of the corresponding variable is left as an exercise for the reader. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 41+ messages in thread
end of thread, other threads:[~2001-01-15 9:09 UTC | newest]
Thread overview: 41+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2001-01-10 3:27 Where did vm_operations_struct->unmap in 2.4.0 go? Allen Unueco
2001-01-10 3:50 ` Keith Owens
2001-01-11 5:38 ` Antony Suter
2001-01-11 6:05 ` Keith Owens
2001-01-11 11:42 ` David Woodhouse
2001-01-11 12:12 ` Keith Owens
2001-01-11 12:32 ` David Woodhouse
2001-01-11 12:46 ` Keith Owens
2001-01-11 13:09 ` Alan Cox
2001-01-11 13:14 ` Keith Owens
2001-01-12 2:12 ` Ingo Oeser
2001-01-12 2:30 ` Keith Owens
2001-01-12 10:27 ` David Woodhouse
2001-01-12 11:55 ` Keith Owens
2001-01-12 13:40 ` David Woodhouse
2001-01-12 12:01 ` Daniel Phillips
2001-01-12 12:18 ` Keith Owens
2001-01-14 10:16 ` Kai Henningsen
2001-01-11 13:25 ` David Woodhouse
[not found] <3A5EFC56.F1A5BCE0@mira.net>
2001-01-12 19:11 ` Christian Zander
2001-01-13 1:11 ` Keith Owens
2001-01-13 10:46 ` David Woodhouse
2001-01-13 12:06 ` Keith Owens
2001-01-13 15:09 ` David Woodhouse
2001-01-13 19:03 ` Russell King
2001-01-14 0:21 ` Keith Owens
2001-01-14 9:43 ` David Woodhouse
2001-01-14 10:05 ` Keith Owens
2001-01-14 10:45 ` David Woodhouse
2001-01-14 4:04 ` Linus Torvalds
2001-01-14 17:46 ` David Woodhouse
2001-01-14 19:12 ` Linus Torvalds
2001-01-14 20:02 ` David Woodhouse
2001-01-14 20:15 ` Linus Torvalds
2001-01-14 21:15 ` David Woodhouse
2001-01-14 21:47 ` Linus Torvalds
2001-01-14 21:57 ` David Woodhouse
2001-01-14 23:00 ` Keith Owens
2001-01-15 9:09 ` David Woodhouse
2001-01-13 11:46 ` Christian Zander
2001-01-13 12:23 ` Keith Owens
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox