* Where did vm_operations_struct->unmap in 2.4.0 go?
@ 2001-01-10 3:27 Allen Unueco
2001-01-10 3:50 ` Keith Owens
2001-01-11 5:38 ` Antony Suter
0 siblings, 2 replies; 41+ messages in thread
From: Allen Unueco @ 2001-01-10 3:27 UTC (permalink / raw)
To: linux-kernel
Sometime around test10 or test11 unmap left vm_operations_struct. The
comment implies its there but it's gone. Where did it go?
How do I get a call back for a page unmap?
I ran into this while hacking the Nvidia kernel driver to work with
2.4.0. I got the driver working but it's not 100%
Also where did get_module_symbol() and put_module_symbol() go?
-Allen
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-10 3:27 Where did vm_operations_struct->unmap in 2.4.0 go? Allen Unueco
@ 2001-01-10 3:50 ` Keith Owens
2001-01-11 5:38 ` Antony Suter
1 sibling, 0 replies; 41+ messages in thread
From: Keith Owens @ 2001-01-10 3:50 UTC (permalink / raw)
To: Allen Unueco; +Cc: linux-kernel
On Tue, 09 Jan 2001 19:27:24 -0800,
Allen Unueco <allen@premierweb.com> wrote:
>Also where did get_module_symbol() and put_module_symbol() go?
http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg08791.html
http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg11497.html
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-10 3:27 Where did vm_operations_struct->unmap in 2.4.0 go? Allen Unueco
2001-01-10 3:50 ` Keith Owens
@ 2001-01-11 5:38 ` Antony Suter
2001-01-11 6:05 ` Keith Owens
1 sibling, 1 reply; 41+ messages in thread
From: Antony Suter @ 2001-01-11 5:38 UTC (permalink / raw)
To: List Linux-Kernel; +Cc: Allen Unueco
Allen Unueco wrote:
>
> Sometime around test10 or test11 unmap left vm_operations_struct. The
> comment implies its there but it's gone. Where did it go?
>
> How do I get a call back for a page unmap?
>
> I ran into this while hacking the Nvidia kernel driver to work with
> 2.4.0. I got the driver working but it's not 100%
>
> Also where did get_module_symbol() and put_module_symbol() go?
>
> -Allen
Patches for the NVIDIA binary X drivers following all these kernel
changes can be gotten from IRC server irc.openprojects.net, channel
#nvidia. Or from http://ex.shafted.com.au/nvidia/
--
- Antony Suter (antony@mira.net) "ExWired" openpgp:71ADFC87
- "...to condense fact from the vapor of nuance."
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-11 5:38 ` Antony Suter
@ 2001-01-11 6:05 ` Keith Owens
2001-01-11 11:42 ` David Woodhouse
0 siblings, 1 reply; 41+ messages in thread
From: Keith Owens @ 2001-01-11 6:05 UTC (permalink / raw)
To: Antony Suter; +Cc: List Linux-Kernel, Allen Unueco
On Thu, 11 Jan 2001 16:38:50 +1100,
Antony Suter <antony@mira.net> wrote:
>Allen Unueco wrote:
>> I ran into this while hacking the Nvidia kernel driver to work with
>> 2.4.0. I got the driver working but it's not 100%
>>
>> Also where did get_module_symbol() and put_module_symbol() go?
>
>Patches for the NVIDIA binary X drivers following all these kernel
>changes can be gotten from IRC server irc.openprojects.net, channel
>#nvidia. Or from http://ex.shafted.com.au/nvidia/
And what a pile of crud those patches are!! Instead of using the clean
replacement interface for get_module_symbol, nvidia/patch-2.4.0-PR hard
codes the old get_module_symbol algorithm as inline code.
This patch violates the modules interface by accessing modules.c
internal data. It still suffers from all the problems that
get_module_symbol had. Because it is hard coded as inline code instead
of a common function, will be much harder to fix when it breaks.
Whoever coded that patch should be taken out and shot, hung, drawn and
quartered then forced to write COBOL for the rest of their natural
life.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-11 6:05 ` Keith Owens
@ 2001-01-11 11:42 ` David Woodhouse
2001-01-11 12:12 ` Keith Owens
0 siblings, 1 reply; 41+ messages in thread
From: David Woodhouse @ 2001-01-11 11:42 UTC (permalink / raw)
To: Keith Owens; +Cc: Antony Suter, List Linux-Kernel, Allen Unueco
kaos@ocs.com.au said:
> And what a pile of crud those patches are!! Instead of using the
> clean replacement interface for get_module_symbol, nvidia/
> patch-2.4.0-PR hard codes the old get_module_symbol algorithm as
> inline code.
Taking away get_module_symbol() and providing a replacement which has link
order problems wasn't really very sensible.
You've changed a lookup in a static table built at compile time to a lookup
in a dynamic table which has to be built in the right order at runtime.
It's too late to do the sensible thing and deprecate the old version rather
than having a 'flag day'. But can we at least fix the link order crap?
struct static_inter_module_entry {
const char *im_name;
const void *userdata;
};
#define inter_module_register_static(x,y) \
static struct static_inter_module_entry __ime_##x \
__attribute__((unused,__section__(".intermodule")) \
= { #x, y };
.. and the obvious for looking in that table in inter_module_get().
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-11 11:42 ` David Woodhouse
@ 2001-01-11 12:12 ` Keith Owens
2001-01-11 12:32 ` David Woodhouse
0 siblings, 1 reply; 41+ messages in thread
From: Keith Owens @ 2001-01-11 12:12 UTC (permalink / raw)
To: David Woodhouse; +Cc: Antony Suter, List Linux-Kernel, Allen Unueco
On Thu, 11 Jan 2001 11:42:24 +0000,
David Woodhouse <dwmw2@infradead.org> wrote:
>Taking away get_module_symbol() and providing a replacement which has link
>order problems wasn't really very sensible.
>
>It's too late to do the sensible thing and deprecate the old version rather
>than having a 'flag day'. But can we at least fix the link order crap?
>
>struct static_inter_module_entry {
> const char *im_name;
> const void *userdata;
>};
>
>#define inter_module_register_static(x,y) \
> static struct static_inter_module_entry __ime_##x \
> __attribute__((unused,__section__(".intermodule")) \
> = { #x, y };
>
>.. and the obvious for looking in that table in inter_module_get().
If object X registers data for object Y to use then X _must_ initialise
before Y. It does not matter whether the registration method is static
or dynamic, the initialisation order must be observed.
Q. With your suggested static method, what happens when Y initialises
before X, calls inter_module_get, retrieves X's static data and
starts to use it before X has initialised?
A. Oops!
The whole point of registration methods is that the owner of the data
decides when they are ready to provide the service. Ensuring that code
is initialised in the correct order, with providers starting before
consumers, is a fact of life.
I dislike the method that the kernel uses to control initialisation
order, but that is an entirely separate problem from inter_module_xxx.
What we really want at startup is a correct initialisation order. What
we have is the order that objects are selected in a Makefile which maps
to the link order of objects in vmlinux which maps to the listed order
of init routines in section .init.text which maps to initialisation
order. The mechanism is three layers away from the problem and it is
difficult to understand for many people.
It would be much nicer to define ordering sets. Code in driver foo
needs the code in driver bar to initialise first. cfi_probe cannot
initialise until cfi_cmdset_0001 and cfi_cmdset_0002 have initialised.
Declare it that way so it is clear what is going on and why, instead of
being implied by the Makefile order via three layers of indirection.
Then let the kernel build system do whatever it takes to honour the
documented initialisation order.
The problem is, Linus likes the current method.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-11 12:12 ` Keith Owens
@ 2001-01-11 12:32 ` David Woodhouse
2001-01-11 12:46 ` Keith Owens
0 siblings, 1 reply; 41+ messages in thread
From: David Woodhouse @ 2001-01-11 12:32 UTC (permalink / raw)
To: Keith Owens; +Cc: Antony Suter, List Linux-Kernel, Allen Unueco
kaos@ocs.com.au said:
> Q. With your suggested static method, what happens when Y initialises
> before X, calls inter_module_get, retrieves X's static data and
> starts to use it before X has initialised?
> A. Oops!
No. You'd explicitly only use the static registration when object X doesn't
_need_ initialisation, which is the case for my code.
As it is, I've had to add completely now init routines for
modules which didn't have them before, and all those init routines do is
inter_module_register() and pray that they're called in time.
kaos@ocs.com.au said:
> The whole point of registration methods is that the owner of the data
> decides when they are ready to provide the service. Ensuring that
> code is initialised in the correct order, with providers starting
> before consumers, is a fact of life.
I have decided when I'm ready to provide the service. At compile time.
Ensuring that the code is initialised in the correct order is not a fact of
life. It's an artifact of the limited functionality of the new setup.
kaos@ocs.com.au said:
> It would be much nicer to define ordering sets. Code in driver foo
> needs the code in driver bar to initialise first. cfi_probe cannot
> initialise until cfi_cmdset_0001 and cfi_cmdset_0002 have initialised.
> Declare it that way so it is clear what is going on and why, instead
> of being implied by the Makefile order via three layers of
> indirection.
For cases where the code really does need initialisation, that is true. It
should be done properly rather than just implicitly ordered by the
Makefile. But in this case, the command set drivers don't need
initialisation. They just provide a function for use by the generic CFI
init code. They don't _need_ to initialise themselves beforehand.
The dynamic registration has introduced this ordering dependency when
previously there was none.
> The problem is, Linus likes the current method.
Then he either hasn't considered this particular case, or he's wrong. It
happens. I'm not suggesting that we change it drastically, only that we add
the option of static (compile-time) registration for those entries which
require it.
Actually, I'd rather do this with weak symbols. Something along the lines
of...
extern cfi_cmdset_fn_t cfi_cmdset_0001 __attribute__((weak));
extern cfi_cmdset_fn_t cfi_cmdset_0002 __attribute__((weak));
...
if (cfi_cmdset_0001 && chip_is_type_1())
return cfi_cmdset_0001(args...);
if (cfi_cmdset_0002 && chip_is_type_2())
return cfi_cmdset_0002(args...);
return cfi_cmdset_load_module(args...)
Unfortunately, I couldn't get it to work reliably on anything but x86.
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-11 12:32 ` David Woodhouse
@ 2001-01-11 12:46 ` Keith Owens
2001-01-11 13:09 ` Alan Cox
2001-01-11 13:25 ` David Woodhouse
0 siblings, 2 replies; 41+ messages in thread
From: Keith Owens @ 2001-01-11 12:46 UTC (permalink / raw)
To: David Woodhouse; +Cc: List Linux-Kernel
On Thu, 11 Jan 2001 12:32:10 +0000,
David Woodhouse <dwmw2@infradead.org> wrote:
>I'm not suggesting that we change it drastically, only that we add
>the option of static (compile-time) registration for those entries which
>require it.
So you want two services, one static for code that does not do any
initialisation and one dynamic for code that does do initialisation.
Can you imagine the fun when somebody adds startup code to a routine
that was using static registration? Oh dear, I added init code so I
have to remember to change from static to dynamic registration, and
that affects the link order so now I have to tweak the Makefile.
Thanks, but no thanks!
Stick to one method that works for all routines, dynamic registration.
If that imposes the occasional need for a couple of extra calls in some
routines and for people to think about initialisation order right from
the start then so be it, it is a small price to pay for long term
stability and ease of maintenance.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-11 12:46 ` Keith Owens
@ 2001-01-11 13:09 ` Alan Cox
2001-01-11 13:14 ` Keith Owens
2001-01-11 13:25 ` David Woodhouse
1 sibling, 1 reply; 41+ messages in thread
From: Alan Cox @ 2001-01-11 13:09 UTC (permalink / raw)
To: Keith Owens; +Cc: David Woodhouse, List Linux-Kernel
> Stick to one method that works for all routines, dynamic registration.
> If that imposes the occasional need for a couple of extra calls in some
> routines and for people to think about initialisation order right from
> the start then so be it, it is a small price to pay for long term
> stability and ease of maintenance.
What happens when we get a loop in init order because of binding and other init
order conflicts?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-11 13:09 ` Alan Cox
@ 2001-01-11 13:14 ` Keith Owens
2001-01-12 2:12 ` Ingo Oeser
0 siblings, 1 reply; 41+ messages in thread
From: Keith Owens @ 2001-01-11 13:14 UTC (permalink / raw)
To: Alan Cox; +Cc: David Woodhouse, List Linux-Kernel
On Thu, 11 Jan 2001 13:09:13 +0000 (GMT),
Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> Stick to one method that works for all routines, dynamic registration.
>> If that imposes the occasional need for a couple of extra calls in some
>> routines and for people to think about initialisation order right from
>> the start then so be it, it is a small price to pay for long term
>> stability and ease of maintenance.
>
>What happens when we get a loop in init order because of binding and other init
>order conflicts?
The kernel does not support circular dependencies between providers and
consumers. It does not matter whether they are built into vmlinux or
loaded as modules, there can be no loops in the directed graph of
dependencies. It just does not make sense.
A while ago there was accidentally a loop between two ppp related
modules, each needed a routine in the other module. modprobe would not
load them. Even if it could have loaded them, it would have been
impossible to unload, both modules would have had a use count on the
other. The fix was to remove the incorrect loop, it was a programming
error.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-11 12:46 ` Keith Owens
2001-01-11 13:09 ` Alan Cox
@ 2001-01-11 13:25 ` David Woodhouse
1 sibling, 0 replies; 41+ messages in thread
From: David Woodhouse @ 2001-01-11 13:25 UTC (permalink / raw)
To: Keith Owens; +Cc: List Linux-Kernel
kaos@ocs.com.au said:
> So you want two services, one static for code that does not do any
> initialisation and one dynamic for code that does do initialisation.
> Can you imagine the fun when somebody adds startup code to a routine
> that was using static registration?
Oh come on. If you change a module from being 'self-contained' and
registered at compile time to requiring initialisation it's hardly
unreasonable to expect you switch the registration too.
Besides, I'm not going to allow any link order dependencies into code I
maintain without them being impossible to avoid - and if anyone's thought
about the problem hard enough to convince me to accept such a change,
they'll have noticed the need to change the registration.
> Oh dear, I added init code so I have to remember to change from static
> to dynamic registration, and that affects the link order so now I have
> to tweak the Makefile.
cf. "Oh dear, I added init code but put it _after_ the registration instead
of before, so stuff blows up."
Neither of these two programmers will get their code into anything I
maintain.
cf. "Oh dear, I need registration, but I have to remember that
inter_module_xxx can't do static registration so now I have to tweak the
Makefile."
kaos@ocs.com.au said:
> Stick to one method that works for all routines, dynamic registration.
It doesn't work for all routines. It introduces unnecessary brokenness -
link order dependencies - where previously there were none.
> If that imposes the occasional need for a couple of extra calls in
> some routines and for people to think about initialisation order right
> from the start then so be it, it is a small price to pay for long term
> stability and ease of maintenance.
I'm thinking about link order. If I _wasn't_ thinking about link order,
then I'd just put the lines in the 'right' order in the Makefile and put up
with it. But I'm thinking about it, and I object to it. It is absolutely
unnecessary in this case.
As far as I'm concerned, fixing the registration problems introduced by the
dynamic inter_module_register is a small price to pay for long term
stability and ease of maintenance :)
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-11 13:14 ` Keith Owens
@ 2001-01-12 2:12 ` Ingo Oeser
2001-01-12 2:30 ` Keith Owens
0 siblings, 1 reply; 41+ messages in thread
From: Ingo Oeser @ 2001-01-12 2:12 UTC (permalink / raw)
To: Keith Owens; +Cc: Alan Cox, David Woodhouse, List Linux-Kernel
On Fri, Jan 12, 2001 at 12:14:44AM +1100, Keith Owens wrote:
> >What happens when we get a loop in init order because of binding and other init
> >order conflicts?
>
> The kernel does not support circular dependencies between providers and
> consumers. It does not matter whether they are built into vmlinux or
> loaded as modules, there can be no loops in the directed graph of
> dependencies. It just does not make sense.
So why don't we use sth. like depmod for these issues and get the
link order automagically (like we get module load order)?
Keith: Perhaps you could explain, why this is impossible.
Regards
Ingo Oeser
--
10.+11.03.2001 - 3. Chemnitzer LinuxTag <http://www.tu-chemnitz.de/linux/tag>
<<<<<<<<<<<< come and join the fun >>>>>>>>>>>>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-12 2:12 ` Ingo Oeser
@ 2001-01-12 2:30 ` Keith Owens
2001-01-12 10:27 ` David Woodhouse
2001-01-12 12:01 ` Daniel Phillips
0 siblings, 2 replies; 41+ messages in thread
From: Keith Owens @ 2001-01-12 2:30 UTC (permalink / raw)
To: Ingo Oeser; +Cc: Alan Cox, David Woodhouse, List Linux-Kernel
On Fri, 12 Jan 2001 03:12:47 +0100,
Ingo Oeser <ingo.oeser@informatik.tu-chemnitz.de> wrote:
>So why don't we use sth. like depmod for these issues and get the
>link order automagically (like we get module load order)?
depmod handles dependencies on symbols. Module Y needs a symbol from
module X so modprobe must load X before Y. This is a link/load problem
which depmod handles fairly well.
The initialisation order is a dependency on actions, not on symbols.
Code F cannot start until code E has initialised so execute E before F.
This should have *NOTHING* to do with link order, but it is implemented
as a side effect of link ordering which confuses people.
People need to realise that the problem is initialisation order,
nothing more, nothing less. You have to determine and document the
startup requirements for your code. Only you know what the startup
pre-requisites for your code are, there is no way for another program
to determine this from the source. Document your startup requirements,
implement according to the documentation and your problems go away.
Initialisation order is not the problem, the implementation is the
problem. The method currently used to control initialisation order
sucks. It is better than the original method (lots of conditional
calls in main.c) but it still sucks.
* Initialisation order is set by the order of structures in section
.initcall.init.
* The order of the structures in .initcall.init is set by the order
that objects are linked into vmlinux.
* The link order for vmlinux is derived from a combination of line
order within a Makefile plus an overriding directory link order from
the top level Makefile and parent Makefiles.
* Because order is a side effect of adding a line to a Makefile, the
order requirements are rarely documented.
It is no wonder that people have problems getting the initialisation
order correct.
I want to completely remove this multi layered method for setting
initialisation order and go back to basics. I want the programmer to
say "initialise E and F after G, H and I". The kernel build system
works out the directed graph of initialisation order then controls the
execution of startup code to satisfy this graph.
That still means controlling link order to achieve the required result.
But with my approach the complexity would be handled by kbuild based on
explicit rules which are documented in the local Makefile, instead of
the complexity being handled by programmer via implicit rules scattered
over several layers of Makefiles.
A typical graph would have scsi disk depends on scsi host adaptor group
which depends on pci. Within the scsi host adaptor group you might
need to initialise one driver before another, so just declare the few
inter-driver dependencies. kbuild would automatically initialise pci
then the scsi host adaptors (in the correct order) then scsi disk.
Most of the objects have fairly simple execution dependencies, e.g.
all file systems depend on core fs code having already executed. There
are no dependencies between most file systems so most file systems
could initialise in any order[1] which means they could be linked in
any order within the file system group.
I am ready and willing to code this method, it would make kbuild a lot
easier to code, as well as making future maintainence a lot easier.
Linus refuses to accept this approach. He insists that kernel coders
explicitly specify the link order for everything, via Makefile
order[2]. As long as Linus insists on kernel coders explicitly
controlling the entire link order, we are stuck with the current
method. I have tried to change his mind without success.
[1] vfat is one obvious exception, it needs dos first. Also the first
few built in file systems must execute in a defined order because
that in turn controls the probe order for mount. But this order
should be explicitly declared, not as a side effect of the line
order in fs/Makefile.
[2] http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg10520.html
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-12 2:30 ` Keith Owens
@ 2001-01-12 10:27 ` David Woodhouse
2001-01-12 11:55 ` Keith Owens
2001-01-12 12:01 ` Daniel Phillips
1 sibling, 1 reply; 41+ messages in thread
From: David Woodhouse @ 2001-01-12 10:27 UTC (permalink / raw)
To: Keith Owens; +Cc: Ingo Oeser, Alan Cox, List Linux-Kernel
On Fri, 12 Jan 2001, Keith Owens wrote:
> People need to realise that the problem is initialisation order,
> nothing more, nothing less. You have to determine and document the
> startup requirements for your code.
This is true. But I'd also agree with the implication which you probably
didn't mean to make there - that initialisation order is a problem. :)
Where an init ordering is required, it must be documented and the
Makefiles set up accordingly. But I believe that we should also try to
avoid requiring such ordering where possible, too.
> It is no wonder that people have problems getting the initialisation
> order correct.
True. And while that situation continues, I desire to avoid the issue
completely by not having any dependencies on init order.
> I want to completely remove this multi layered method for setting
> initialisation order and go back to basics. I want the programmer to
> say "initialise E and F after G, H and I". The kernel build system
> works out the directed graph of initialisation order then controls the
> execution of startup code to satisfy this graph.
But the fewer such constraints there are, the better. We don't want
everyone starting to impose unnecessary link order restrictions instead of
thinking about the code a little more and just eliminating them
completely.
> A typical graph would have scsi disk depends on scsi host adaptor group
> which depends on pci.
No. sd will happily take over any existing devices when as and when they
arrive. It doesn't have to be loaded last. Likewise, in theory at least,
host adaptor drivers using the new PCI driver code would respond correctly
to the PCI code being initialised (and calling their ->probe routine)
later, although that doesn't happen now.
Why have these dependencies where they're not necessary?
> Within the scsi host adaptor group you might need to initialise one
> driver before another, so just declare the few inter-driver
> dependencies.
And there are few. And there should _remain_ few. We shouldn't start
imposing link order restrictions on other code which doesn't really need
it.
> Most of the objects have fairly simple execution dependencies, e.g.
> all file systems depend on core fs code having already executed.
Er... Why? They call register_filesystem() which uses a lock which is
staticly initialised. Don't order stuff for the sake of it. If there are
filesystems which _do_ require the VFS to be initialised first, those
filesystems can be marked as such. I'm not aware of any.
> [1] vfat is one obvious exception, it needs dos first. Also the first
> few built in file systems must execute in a defined order because
> that in turn controls the probe order for mount. But this order
> should be explicitly declared, not as a side effect of the line
> order in fs/Makefile.
If the fat_inode_hashtable were staticly initialised, that one wouldn't be
necessary either. But unfortunately that would be quite ugly.
I don't want to get involved in the link order stuff. I would very much
like to avoid having such dependencies. All I want is weak symbols. I
want to call a function directly if it's present, and if not I want to
attempt to load its module. Without having my probe module have a hard
dependency on _all_ the submodules it may decide to ask for.
get_module_symbol() did that for me, and was perfectly acceptable. I
needed to use EXPORT_SYMBOL_NOVERS to export the functions for the
individual command sets. Big deal.
inter_module_get() almost does that for me, but it imposes link order
dependencies, which I want to avoid because I think in this case they're
not necessary. All I want is a way to staticly add entries to the
inter_module_xxx tables at compile time, because I _have_ done the
analysis, and I'm saying that's when they should be made available.
Alternatively, show me how to get weak symbols actually working on all
architectures, and I'll forget the whole thing and be happy.
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-12 10:27 ` David Woodhouse
@ 2001-01-12 11:55 ` Keith Owens
2001-01-12 13:40 ` David Woodhouse
0 siblings, 1 reply; 41+ messages in thread
From: Keith Owens @ 2001-01-12 11:55 UTC (permalink / raw)
To: David Woodhouse; +Cc: Ingo Oeser, Alan Cox, List Linux-Kernel
On Fri, 12 Jan 2001 10:27:34 +0000 (GMT),
David Woodhouse <dwmw2@infradead.org> wrote:
>On Fri, 12 Jan 2001, Keith Owens wrote:
>> A typical graph would have scsi disk depends on scsi host adaptor group
>> which depends on pci.
>
>No. sd will happily take over any existing devices when as and when they
>arrive. It doesn't have to be loaded last. Likewise, in theory at least,
>host adaptor drivers using the new PCI driver code would respond correctly
>to the PCI code being initialised (and calling their ->probe routine)
>later, although that doesn't happen now.
You just proved my point. It is extremely difficult to deduce the
required initialisation order by reading an undocumented Makefile where
the init order is implemented as a side effect of selection order. The
existing method implies link order when none is required.
>> Most of the objects have fairly simple execution dependencies, e.g.
>> all file systems depend on core fs code having already executed.
>
>Er... Why? They call register_filesystem() which uses a lock which is
>staticly initialised. Don't order stuff for the sake of it. If there are
>filesystems which _do_ require the VFS to be initialised first, those
>filesystems can be marked as such. I'm not aware of any.
I was using scsi and fs as examples, no need to pick holes in the
examples. But ...
fs/buffer.c:module_init(bdflush_init)
fs/pipe.c:module_init(init_pipe_fs)
fs/fcntl.c:module_init(fasync_init)
fs/locks.c:module_init(filelock_init)
fs/dnotify.c:module_init(dnotify_init)
I'm no filesystem expert but it appears that some of those core
initialisation routines must be executed before most filesystems can
start. In particular, filelock_init() must be executed before any
filesystem that supports locks is initialised.
>All I want is a way to staticly add entries to the
>inter_module_xxx tables at compile time, because I _have_ done the
>analysis, and I'm saying that's when they should be made available.
Firstly inter_module_xxx is only used by that very small subset of code
where there are no constraints on whether the caller and callee can be
in kernel, in modules or a mixture _and_ some of the objects are
optional. It is a special case because most code handles this problem
through CONFIG options. If X needs (Y, Z) and X is built into the
kernel then (Y, Z) must also be built into the kernel. If Y or Z are
optional then control the calls to those functions with CONFIG options.
Almost all of the kernel handles it properly though CONFIG, so
inter_module_xxx is already a process to handle a few special cases.
Secondly you want static inter_module_xxx tables for a small subset of
these special cases, where the source has no other initialisation code.
This is piling special cases on special cases. Adding static
inter_module_xxx tables requires
* changes to linux/modules.h to define the new table format and
* changes to vmlinux.lds for _every_ architecture to bring all the
static tables together in vmlinux and
* new initialisation code in module.c to read and load all the static
tables at boot time and
* extra code in modutils to find any static tables in modules and
* an extension to struct modules to let modutils pass information about
the static tables to the kernel and
* the kernel code will only work with an upgraded modutils.
That is a lot of work for a very few special cases. OTOH, you could
just swap the order of 3 lines in drivers/mtd/Makefile. Guess which
alternative I am going for?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-12 2:30 ` Keith Owens
2001-01-12 10:27 ` David Woodhouse
@ 2001-01-12 12:01 ` Daniel Phillips
2001-01-12 12:18 ` Keith Owens
1 sibling, 1 reply; 41+ messages in thread
From: Daniel Phillips @ 2001-01-12 12:01 UTC (permalink / raw)
To: Keith Owens, linux-kernel
Keith Owens wrote:
> I want to completely remove this multi layered method for setting
> initialisation order and go back to basics. I want the programmer to
> say "initialise E and F after G, H and I". The kernel build system
> works out the directed graph of initialisation order then controls the
> execution of startup code to satisfy this graph.
I don't doubt you will come up with a workable solution at build time.
However, working out a valid graph at execution time is trivial and
efficient, given a list of precedence relations of the kind you're
suggesting. In fact you don't even have to work out the graph before
starting the initialization, it's also trivial to keep a count of
unsatisfied initialization conditions at the beginning of each
initialization sequence and block until the count goes to zero. (In
essence, evaluate a priority sort on the fly.)
--
Daniel
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-12 12:01 ` Daniel Phillips
@ 2001-01-12 12:18 ` Keith Owens
2001-01-14 10:16 ` Kai Henningsen
0 siblings, 1 reply; 41+ messages in thread
From: Keith Owens @ 2001-01-12 12:18 UTC (permalink / raw)
To: Daniel Phillips; +Cc: linux-kernel
On Fri, 12 Jan 2001 13:01:12 +0100,
Daniel Phillips <phillips@innominate.de> wrote:
>Keith Owens wrote:
>> I want to completely remove this multi layered method for setting
>> initialisation order and go back to basics. I want the programmer to
>> say "initialise E and F after G, H and I". The kernel build system
>> works out the directed graph of initialisation order then controls the
>> execution of startup code to satisfy this graph.
>
>I don't doubt you will come up with a workable solution at build time.
>However, working out a valid graph at execution time is trivial and
>efficient, given a list of precedence relations of the kind you're
>suggesting. In fact you don't even have to work out the graph before
>starting the initialization, it's also trivial to keep a count of
>unsatisfied initialization conditions at the beginning of each
>initialization sequence and block until the count goes to zero. (In
>essence, evaluate a priority sort on the fly.)
It is safer to evaluate the graph at link time in case somebody
mistakenly codes a dependency loop, that is an abort case. Finding
that you have a loop at load time and aborting the kernel is a little
too drastic for my tastes.
In any case it is academic. Linus insists on link order being
explicitly and completely specified by the programmer and he likes the
link order being implied by Makefile order. So there is no point in
working on a better system.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-12 11:55 ` Keith Owens
@ 2001-01-12 13:40 ` David Woodhouse
0 siblings, 0 replies; 41+ messages in thread
From: David Woodhouse @ 2001-01-12 13:40 UTC (permalink / raw)
To: Keith Owens; +Cc: Ingo Oeser, Alan Cox, List Linux-Kernel
kaos@ocs.com.au said:
> You just proved my point. It is extremely difficult to deduce the
> required initialisation order by reading an undocumented Makefile
> where the init order is implemented as a side effect of selection
> order. The existing method implies link order when none is required.
I agree entirely. But you're confusing the debate on how to satisfy init
order dependencies with my desire to avoid them altogether (for certain
situations).
kaos@ocs.com.au said:
> I'm no filesystem expert but it appears that some of those core
> initialisation routines must be executed before most filesystems can
> start. In particular, filelock_init() must be executed before any
> filesystem that supports locks is initialised.
No, before any filesystem that supports locks is _mounted_. Big difference.
But I'm picking holes in your examples again. I accept that in some cases
it is necessary, but I still think it's best to avoid it where possible.
kaos@ocs.com.au said:
> Firstly inter_module_xxx is only used by that very small subset of
> code where there are no constraints on whether the caller and callee
> can be in kernel, in modules or a mixture _and_ some of the objects
> are optional. It is a special case because most code handles this
> problem through CONFIG options. If X needs (Y, Z) and X is built into the
> kernel then (Y, Z) must also be built into the kernel. If Y or Z are
> optional then control the calls to those functions with CONFIG options.
> Almost all of the kernel handles it properly though CONFIG, so
> inter_module_xxx is already a process to handle a few special cases.
'properly through CONFIG'? I thought you agreed that doing it through
preprocessor options was ugly, and that it was preferable to get rid of such
things and deal with the presence or absence of such code cleanly through
some mechanism with similar semantics to inter_module_get(). The 'special
case' code to which you refer is in fact the first set of such code to
attempt to deal with this properly rather than just giving up and hacking
it with preprocessor options.
Stepping back a moment and considering it, what we actually appear to be
doing is trying to reinvent weak symbols, as far as I can tell.
kaos@ocs.com.au said:
> Secondly you want static inter_module_xxx tables for a small subset
> of these special cases, where the source has no other initialisation
> code. This is piling special cases on special cases. Adding static
> inter_module_xxx tables requires
I want the registration method not to impose init order restrictions where
previously there were none. Where the previous code already had init order
ugliness, it's not so much of a problem. But replacing the safe
get_module_symbol() with the unsafe inter_module_get() just because the AGP
author forgot to use EXPORT_SYMBOL_NOVERS to export the symbols doesn't
really strike me as being useful.
The only real difference between the two, other than the module symbol
mangling, appears to be that inter_module_get() looks stuff up in a dynamic
table which is made at runtime, where get_module_symbol() looks it up in a
static table which is made at compile (or module load) time. That was a
backward step, and wasn't necessary.
If you object to having both the runtime inter_module_register() and the
compile-time one, then ditch the dynamic one. Both current users of
inter_module_xxx would be able to use the static version.
But to be honest, I'd settle for ditching the whole blinkin' lot and
getting weak symbols working right, if I could.
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
[not found] <3A5EFC56.F1A5BCE0@mira.net>
@ 2001-01-12 19:11 ` Christian Zander
2001-01-13 1:11 ` Keith Owens
0 siblings, 1 reply; 41+ messages in thread
From: Christian Zander @ 2001-01-12 19:11 UTC (permalink / raw)
To: linux-kernel; +Cc: Keith Owens
[-- Attachment #1: Type: text/plain, Size: 2777 bytes --]
> >> I ran into this while hacking the Nvidia kernel driver to work with
> >> 2.4.0. I got the driver working but it's not 100%
> >>
> >> Also where did get_module_symbol() and put_module_symbol() go?
> >
> >Patches for the NVIDIA binary X drivers following all these kernel
> >changes can be gotten from IRC server irc.openprojects.net, channel
> >#nvidia. Or from http://ex.shafted.com.au/nvidia/
>
> And what a pile of crud those patches are!! Instead of using the clean
> replacement interface for get_module_symbol, nvidia/patch-2.4.0-PR hard
> codes the old get_module_symbol algorithm as inline code.
>
> This patch violates the modules interface by accessing modules.c
> internal data. It still suffers from all the problems that
> get_module_symbol had. Because it is hard coded as inline code instead
> of a common function, will be much harder to fix when it breaks.
>
The way I understand the inter_module mechanism, module A registers one
or several of its symbols using inter_module_register to make it or them
available to other modules. Module B can then request any of the symbols
with inter_module_request and get a pointer. The inter_module mechanism
guarantees that the symbol will be available until module B decides that
the symbol is no longer needed and releases it by calling inter_module_put.
Saying that I should have made use of this mechanism for the specific
code in the Nvidia driver that we are talking about clearly shows that
you didn't look at it. The module used get_module_symbol to search its
own symbol table for parameters that may have been passed to it at load
time. Arguably, this is bad practise, but it is also the reason why using
your mechanism doesn't make any sense. Obviously, the module wouldn't
want to register private data to request it later on; the information
that would have to be passed to inter_module_register is the same that
the code in question intends to retrieve in the first place.
Contrary to what you're saying, the patch does not just inline the old
get_module_symbol algorithm nor does it access any of module.c's internal
data. What is does is to browse the list of the modules's _own_ symbols
looking for a match. If it finds one, it returns the desired data.
> Whoever coded that patch should be taken out and shot, hung, drawn and
> quartered then forced to write COBOL for the rest of their natural
> life.
Excellent comment - it is just as appropriate as it is helpful.
--
----------------------------------------------------------------------
christian zander we come to bury dos, not to praise it.
zander@hdz.uni-dortmund.de -- paul vojta
----------------------------------------------------------------------
[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-12 19:11 ` Christian Zander
@ 2001-01-13 1:11 ` Keith Owens
2001-01-13 10:46 ` David Woodhouse
2001-01-13 11:46 ` Christian Zander
0 siblings, 2 replies; 41+ messages in thread
From: Keith Owens @ 2001-01-13 1:11 UTC (permalink / raw)
To: Christian Zander; +Cc: linux-kernel, David Woodhouse
On Fri, 12 Jan 2001 20:11:30 +0100,
Christian Zander <phoenix@minion.de> wrote:
>Saying that I should have made use of this mechanism for the specific
>code in the Nvidia driver that we are talking about clearly shows that
>you didn't look at it. The module used get_module_symbol to search its
>own symbol table for parameters that may have been passed to it at load
>time.
My apologies. I read the patch, not the full source code and the patch
does not have enough programming context to show that the driver is
only searching its own symbol space. In my own defense, the references
to spinlock_t unload_lock and MOD_CAN_QUERY(mp) in the patch are highly
misleading, those statements only make sense when you are looking at a
symbol table for another module. When searching your own symbol table
the current module must be live with a non-zero use count, not being
unloaded and it can always be queried.
>Contrary to what you're saying, the patch does not just inline the old
>get_module_symbol algorithm nor does it access any of module.c's internal
>data.
unload_lock and MOD_CAN_QUERY were copied verbatim from the old
get_module_symbol, even though they are completely unnecessary. That
looks like inlining the old algorithm to me.
struct module_symbol, mp->nsyms and mp->syms are module.c internal
data. If it is ever necessary to change those structures, nothing
outside module.c, the 32/64 handlers for module system calls and
modutils should be affected. Now if I change module_symbol, other bits
of the kernel will unexpectedly break, this is not good.
>> Whoever coded that patch should be taken out and shot, hung, drawn and
>> quartered then forced to write COBOL for the rest of their natural
>> life.
>
>Excellent comment - it is just as appropriate as it is helpful.
Over emphasis for humorous effect. Must remember to add smiley.
What this patch and David Woodhouse's comments show is that I need to
look at a generic and safe mechanism for kernel/module symbol lookup.
The existing static mechanism works for fixed symbol names but does not
work for symbol names that are generated at run time nor for symbols
that may or may not be present.
get_module_symbol() "worked" but was horribly unsafe. It broke with
module versions, it did zero type checking which left the code open to
version skew and it assumed that all addresses are equivalent to an
unsigned long.
That last point is especially important for IA64 where function
pointers do not reference the function directly, instead they point to
a function descriptor with two fields, one of which is the function
address. Casting the unsigned long address of a function into a
function pointer fails miserably on IA64, and gcc does not even give
any warnings. foo = (int (*)(int))get_module_symbol(NULL, "funcname")
is architecture dependent.
Using EXPORT_SYMBOL_NOVERS() to "fix" the modversions problem for
get_module_symbol() removes all inter module checks on the relevant
symbols. Not just for the caller of get_module_symbol for all modules
that access those symbols. This leaves too much code open to version
skew and is not acceptable.
inter_module_xxx is modversions safe. It still does no type checking
because it uses void * for the data structure, but the exporter and
user have to declare their common data area which reduces the chance of
version skew. I am still not happy about this possibility of skew but
anything is better than no checks at all. Passing a data structure
which contains real declarations for function pointers instead of
assuming you can cast a number to a function pointer makes
inter_module_xxx architecture independent.
I will look at a general kernel and module symbol lookup routine that
does the job properly. The hard part is making sure that the provider
and consumer have exactly the same types for a symbol. Both
get_module_symbol and inter_module_xxx completely bypass the
modversions checks and are wide open to undetectable version skew,
although inter_module_xxx is a little bit safer. Any replacement for
these functions must be able to do type checking at run time, which
probably means it is 2.5 code. And yes, David, it should be able to
handle static data.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-13 1:11 ` Keith Owens
@ 2001-01-13 10:46 ` David Woodhouse
2001-01-13 12:06 ` Keith Owens
2001-01-13 11:46 ` Christian Zander
1 sibling, 1 reply; 41+ messages in thread
From: David Woodhouse @ 2001-01-13 10:46 UTC (permalink / raw)
To: Keith Owens; +Cc: Christian Zander, linux-kernel, alan
On Sat, 13 Jan 2001, Keith Owens wrote:
> Over emphasis for humorous effect. Must remember to add smiley.
Heh. But it does deserve to get into the fortune file.
> What this patch and David Woodhouse's comments show is that I need to
> look at a generic and safe mechanism for kernel/module symbol lookup.
> The existing static mechanism works for fixed symbol names but does not
> work for symbol names that are generated at run time nor for symbols
> that may or may not be present.
Actually, my testing showed that modutils was quite OK with symbols which
may or may not be present. But compiling such code into the kernel, at
least on MIPS and m68k, didn't work.
cat >weaktest.c <<EOF
#include <linux/module.h>
extern char *myfun(void) __attribute__((weak));
int init_module(void)
{
char *txt= "myfun() not present\n";
if (myfun)
txt = myfun();
printk(txt);
return 0;
}
EOF
cat > myfun.c <<EOF
#include <linux/module.h>
char *myfun(void)
{
return "Hello World\n";
}
EOF
I doubt this would have implemented weak symbols completely, though.
Fixing up the reference in weaktest.o if myfun.o was loaded later, etc.
And we don't really want to 'fix' that either.
So it'd still have needed request_module(); get_module_symbol() if it
found that myfun wasn't present and it needed it. So it might as well have
used get_module_symbol() from the start instead of the weak declaration.
> get_module_symbol() "worked" but was horribly unsafe. It broke with
> module versions, it did zero type checking which left the code open to
> version skew and it assumed that all addresses are equivalent to an
> unsigned long.
>
> That last point is especially important for IA64 where function
> pointers do not reference the function directly, instead they point to
> a function descriptor with two fields, one of which is the function
> address. Casting the unsigned long address of a function into a
> function pointer fails miserably on IA64, and gcc does not even give
> any warnings. foo = (int (*)(int))get_module_symbol(NULL, "funcname")
> is architecture dependent.
But fixable.
> Using EXPORT_SYMBOL_NOVERS() to "fix" the modversions problem for
> get_module_symbol() removes all inter module checks on the relevant
> symbols. Not just for the caller of get_module_symbol for all modules
> that access those symbols. This leaves too much code open to version
> skew and is not acceptable.
I'm not sure there's anything which was expected to be obtained by
get_module_symbol() which was also obtained by normal linking. The nature
of these routines is that they're optional. Usually, the routine would be
optional for all callers or it'd be mandatory for all callers. Rarely a
mixture.
But what about keeping a separate table, with PUBLISH_SYMBOL() or
something slightly more sensibly named? That way, get_published_symbol()
can only get at those symbols which were supposed to be listed, and if
someone really wants EXPORT_SYMBOL() and PUBLISH_SYMBOL() they can do
that.
> > inter_module_xxx is modversions safe. It still does no type checking
> because it uses void * for the data structure, but the exporter and
> user have to declare their common data area which reduces the chance of
> version skew.
I'm not sure I follow. The exporter and the user have to each declare
their common data area, which means they don't have to declare it
identically, and there's just as much chance of version skew as before,
surely? With get_module_symbol both had to declare it, too.
And 'modversions safe' just means that there's no attempt to mangle the
names so it's identical to the EXPORT_SYMBOL_NOVERS case above?
> I am still not happy about this possibility of skew but
> anything is better than no checks at all. Passing a data structure
> which contains real declarations for function pointers instead of
> assuming you can cast a number to a function pointer makes
> inter_module_xxx architecture independent.
On the other hand, it's also simple enough to define a macro which does
the arch-dependent equivalent of the 'foo=(fnptr_t)get_module_symbol()'
above, isn't it? But I'm not particularly attached to the method. The
static initialisation is what I miss.
> I will look at a general kernel and module symbol lookup routine that
> does the job properly.
Thanks.
> The hard part is making sure that the provider and consumer have
> exactly the same types for a symbol.
That would be useful, but it's not 100% imperative. Runtime type-checking
is a nice sanity check where it's almost free, as it was in the
modversions case, but I don't want to program in Java.
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-13 1:11 ` Keith Owens
2001-01-13 10:46 ` David Woodhouse
@ 2001-01-13 11:46 ` Christian Zander
2001-01-13 12:23 ` Keith Owens
1 sibling, 1 reply; 41+ messages in thread
From: Christian Zander @ 2001-01-13 11:46 UTC (permalink / raw)
To: Keith Owens; +Cc: linux-kernel
[-- Attachment #1: Type: text/plain, Size: 2015 bytes --]
On Sat, Jan 13, 2001 at 12:11:31PM +1100, kaos@ocs.com.au wrote:
> My apologies. I read the patch, not the full source code and the patch
> does not have enough programming context to show that the driver is
> only searching its own symbol space. In my own defense, the references
> to spinlock_t unload_lock and MOD_CAN_QUERY(mp) in the patch are highly
> misleading, those statements only make sense when you are looking at a
> symbol table for another module. When searching your own symbol table
> the current module must be live with a non-zero use count, not being
> unloaded and it can always be queried.
>
> >Contrary to what you're saying, the patch does not just inline the old
> >get_module_symbol algorithm nor does it access any of module.c's internal
> >data.
>
> unload_lock and MOD_CAN_QUERY were copied verbatim from the old
> get_module_symbol, even though they are completely unnecessary. That
> looks like inlining the old algorithm to me.
>
> struct module_symbol, mp->nsyms and mp->syms are module.c internal
> data. If it is ever necessary to change those structures, nothing
> outside module.c, the 32/64 handlers for module system calls and
> modutils should be affected. Now if I change module_symbol, other bits
> of the kernel will unexpectedly break, this is not good.
I see what you mean. What do you suggest should be done in the context of
the driver? As you can easily tell, I'm not overly familiar with the
internal workings of the kernel. That and the mere impossibility to get
any kind of help at the mere mention of the Nvidia driver module ("go bitch
at nvidia", "who cares", ...) do not exactly make it easier to fix problems
that arise from changes to the kernel.
--
----------------------------------------------------------------------
christian zander we come to bury dos, not to praise it.
zander@hdz.uni-dortmund.de -- paul vojta
----------------------------------------------------------------------
[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-13 10:46 ` David Woodhouse
@ 2001-01-13 12:06 ` Keith Owens
2001-01-13 15:09 ` David Woodhouse
0 siblings, 1 reply; 41+ messages in thread
From: Keith Owens @ 2001-01-13 12:06 UTC (permalink / raw)
To: David Woodhouse; +Cc: Christian Zander, linux-kernel, alan
On Sat, 13 Jan 2001 10:46:44 +0000 (GMT),
David Woodhouse <dwmw2@infradead.org> wrote:
>On Sat, 13 Jan 2001, Keith Owens wrote:
>Actually, my testing showed that modutils was quite OK with symbols which
>may or may not be present. But compiling such code into the kernel, at
>least on MIPS and m68k, didn't work.
Weak symbols were added to gcc somewhere between 2.7.2.3 and 2.91.66.
At least two architectures are using versions of gcc that predate (by a
few days) the addition of weak symbols. Davem hit this problem on
sparc with the weak references to kallsyms, he had to define the
symbols instead of letting them resolve to zero, gcc on sparc silently
ignored weak.
>I doubt this would have implemented weak symbols completely, though.
>Fixing up the reference in weaktest.o if myfun.o was loaded later, etc.
>And we don't really want to 'fix' that either.
Weak is not enough. We need dynamic symbol binding if we plan to
support a cooperative model for objects instead of a strict hierarchic
model.
BTW, modutils cannot automatically fill in upward references when a
module is loaded. A reference is a use count, an automatic reference
would be an automatic use count with no way of removing it. Code that
calls upwards to a symbol must perform an overt action to get the
reference and cope with the case when the symbol is not there. Think
of it as a probe, "do I have facility XXX yet?". It is up to the
caller to probe as often as required. Hot plugging for symbols, wheee!
>> That last point is especially important for IA64 where function
>> pointers do not reference the function directly, instead they point to
>> a function descriptor with two fields, one of which is the function
>> address. Casting the unsigned long address of a function into a
>> function pointer fails miserably on IA64, and gcc does not even give
>> any warnings. foo = (int (*)(int))get_module_symbol(NULL, "funcname")
>> is architecture dependent.
>
>But fixable.
Probably not. The generated IA64 object code for this case is
completely wrong, not surprising since we are lying to gcc.
x.o: file format elf64-ia64-little
Disassembly of section .text:
0000000000000000 <test1>:
unsigned long value = 0xc0002000; // example, not a real ia64 address
void test1(void)
{
0: [MII] alloc r34=ar.pfs,4,4,0
void (*test2)(void);
6: mov r35=r12 // &test2
c: adds r12=-16,r12 // adjust stack pointer
10: [MII] nop.m 0x0
test2 = (void (*)(void))value;
16: mov r33=b0 // save return address
12: GPREL22 value
1c: addl r14=0,r1;; // &value
20: [MMI] ld8 r14=[r14];; // value
26: st8 [r35]=r14 // test2 = value 0xc0002000
2c: nop.i 0x0
test2();
30: [MII] ld8 r14=[r35] // read test2, 0xc0002000
36: mov r32=r1;; // save current global pointer
3c: nop.i 0x0
40: [MII] ld8 r15=[r14] // dereference test2, wrong
46: adds r14=8,r14;; // test2 + 8, 0xc0002008
4c: nop.i 0x0
50: [MII] ld8 r1=[r14] // new gp from 0xc0002008, wrong
56: mov b6=r15;; // & target function, wrong
5c: nop.i 0x0
60: [MIB] nop.m 0x0
66: nop.i 0x0
6c: br.call.sptk.many b0=b6;; // call indirect function, oops
70: [MII] mov r1=r32 // restore gp
By casting 'test2 = (void (*)(void))value' we claim that value is the
address of the the function descriptor which must contain
{ actual address of function, global data pointer for function }
gcc trusts us and tries to use the data at location 0xc0002000 as a
function descriptor. Because get_module_symbol() returns the address
of the first instruction in a function, that code would load the actual
address and global pointer from the first 16 bytes of the function's
code area. Needless to say, it does not work.
Fixing this would mean tweaking get_module_symbol() on IA64 to
recognise that the symbol is a function, build a function descriptor on
the fly and return the address of the descriptor. But the information
about the types of symbols is not available in the kernel nor in
modules after they are loaded, get_module_symbol() cannot differentiate
between functions and anything else. There is also the small matter of
grubbing around in the arch dependent bit of struct modules to find the
global pointer for the target function, more complexity.
>But what about keeping a separate table, with PUBLISH_SYMBOL() or
>something slightly more sensibly named? That way, get_published_symbol()
>can only get at those symbols which were supposed to be listed, and if
>someone really wants EXPORT_SYMBOL() and PUBLISH_SYMBOL() they can do
>that.
I don't see the point. EXPORT_SYMBOL() says that the symbol can be
accessed by anybody. The current hierarchical binding model restricts
access to modules that load after this module. If we remove the strict
hierarchical binding of module symbols, why worry whether the caller is
above or below this module? IOW, there is no need for a different
definition that says the symbol can also be accessed by the kernel or
by earlier module loads. Upward references to a symbol are another
story.
>> > inter_module_xxx is modversions safe. It still does no type checking
>> because it uses void * for the data structure, but the exporter and
>> user have to declare their common data area which reduces the chance of
>> version skew.
>
>I'm not sure I follow. The exporter and the user have to each declare
>their common data area, which means they don't have to declare it
>identically, and there's just as much chance of version skew as before,
>surely? With get_module_symbol both had to declare it, too.
get_module_symbol() was usually used with a cast from unsigned long to
some type that was defined in the calling .c file. That made the
caller code responsible for using the correct declaration. It is
better to define interfaces as shared data in a shared header. Not
perfect, but better.
>And 'modversions safe' just means that there's no attempt to mangle the
>names so it's identical to the EXPORT_SYMBOL_NOVERS case above?
With inter_module_xxx you have no type checking on the registered data.
But you do not have to use EXPORT_SYMBOL_NOVERS on the shared symbols
which means that any other users of the symbols will still get type
checking. Again, not perfect, but better.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-13 11:46 ` Christian Zander
@ 2001-01-13 12:23 ` Keith Owens
0 siblings, 0 replies; 41+ messages in thread
From: Keith Owens @ 2001-01-13 12:23 UTC (permalink / raw)
To: Christian Zander; +Cc: linux-kernel
On Sat, 13 Jan 2001 12:46:00 +0100,
Christian Zander <phoenix@minion.de> wrote:
>I see what you mean. What do you suggest should be done in the context of
>the driver? As you can easily tell, I'm not overly familiar with the
>internal workings of the kernel. That and the mere impossibility to get
>any kind of help at the mere mention of the Nvidia driver module ("go bitch
>at nvidia", "who cares", ...) do not exactly make it easier to fix problems
>that arise from changes to the kernel.
Hmm, can I charge Nvidia for this fix?
The only reason you are looking at symbols is to map Nvidia registry
names to module symbols. There are 9 registry names, 4 of which are
#ifdeffed out.
MODULE_PARM(NVreg_resman_debuglevel, "i");
MODULE_PARM(NVreg_VideoMemoryTypeOverride, "i"); #ifdeffed out
MODULE_PARM(NVreg_EnableVia4x, "i");
MODULE_PARM(NVreg_ReqAGPRate, "i"); #ifdeffed out
MODULE_PARM(NVreg_SkipBiosPost, "i"); #ifdeffed out
MODULE_PARM(NVreg_UseKernelAGP, "i");
MODULE_PARM(NVreg_UpdateKernelAGP, "i"); #ifdeffed out
MODULE_PARM(NVreg_ReqAGPSBA, "i");
MODULE_PARM(NVreg_ReqAGPFW, "i");
Simple fix is an array to map names to addresses.
struct {
const char *name;
int *value;
} linux_registry[] = {
{ "resman_debuglevel", &NVreg_resman_debuglevel },
{ "EnableVia4x", &NVreg_EnableVia4x },
{ "UseKernelAGP", &NVreg_UseKernelAGP },
{ "ReqAGPSBA", &NVreg_ReqAGPSBA },
{ "ReqAGPFW", &NVreg_ReqAGPFW },
};
Changing osRegistryLookup to scan that array for the registry name and
return the address of the corresponding variable is left as an exercise
for the reader.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-13 12:06 ` Keith Owens
@ 2001-01-13 15:09 ` David Woodhouse
2001-01-13 19:03 ` Russell King
` (2 more replies)
0 siblings, 3 replies; 41+ messages in thread
From: David Woodhouse @ 2001-01-13 15:09 UTC (permalink / raw)
To: Keith Owens; +Cc: Christian Zander, linux-kernel, alan
On Sat, 13 Jan 2001, Keith Owens wrote:
> BTW, modutils cannot automatically fill in upward references when a
> module is loaded. A reference is a use count, an automatic reference
> would be an automatic use count with no way of removing it. Code that
> calls upwards to a symbol must perform an overt action to get the
> reference and cope with the case when the symbol is not there. Think
> of it as a probe, "do I have facility XXX yet?". It is up to the
> caller to probe as often as required. Hot plugging for symbols, wheee!
We don't need to overdesign it. get_module_symbol() basically provided
this for us. The only thing really wrong with it was the lack of use
count handling, which I fixed a while ago.
Lack of runtime typechecking isn't a showstopper. Otherwise we'd have
thrown out GCC by now and rewritten the kernel in Modula-3.
That leaves the IA64 problem.
> Fixing this would mean tweaking get_module_symbol() on IA64 to
> recognise that the symbol is a function, build a function descriptor on
> the fly and return the address of the descriptor. But the information
> about the types of symbols is not available in the kernel nor in
> modules after they are loaded, get_module_symbol() cannot differentiate
> between functions and anything else. There is also the small matter of
> grubbing around in the arch dependent bit of struct modules to find the
> global pointer for the target function, more complexity.
This is already handled by modutils, presumably. How? By 'grubbing arouund
in the arch dependent bit of struct modules'? There's already a handful of
gp handling surrounded by #ifdef __alpha__ in module.c which doesn't seem
too unreasonable.
The caller knows what it's expecting to find. How about separate
get_module_function() and get_module_data() routines? Which are identical
on most architectures, but on (Alpha and?) IA64 could be defined to return
an appropriate structure.
> get_module_symbol() was usually used with a cast from unsigned long to
> some type that was defined in the calling .c file. That made the
> caller code responsible for using the correct declaration. It is
> better to define interfaces as shared data in a shared header. Not
> perfect, but better.
We could quite happily define the function type in a shared header file,
and coding the original function and the subsequent cast from
get_module_symbol() using that definition.
Conversely, nothing stops you from using only local definitions for
the inter_module_xxx objects, rather than a single shared definition.
Nothing's changed. You just changed the users to use shared definitions
while you converted them to inter_module_xxx. But there's no fundamental
difference in the interface used, in this respect.
> With inter_module_xxx you have no type checking on the registered data.
> But you do not have to use EXPORT_SYMBOL_NOVERS on the shared symbols
> which means that any other users of the symbols will still get type
> checking. Again, not perfect, but better.
Slightly. But for the cases where inter_module_xxx or get_module_symbol
are used,
A. AFAIK there are no such 'direct' users who get the
benefit of the runtime typechecking.
B. The authors are already having to pay attention to any
changes in the type of the exported data, rather than
just pretending that they're writing Java code and
expecting the runtime system to wipe the dribble
from their chins.
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-13 15:09 ` David Woodhouse
@ 2001-01-13 19:03 ` Russell King
2001-01-14 0:21 ` Keith Owens
2001-01-14 4:04 ` Linus Torvalds
2 siblings, 0 replies; 41+ messages in thread
From: Russell King @ 2001-01-13 19:03 UTC (permalink / raw)
To: David Woodhouse; +Cc: Keith Owens, Christian Zander, linux-kernel, alan
David Woodhouse writes:
> We don't need to overdesign it. get_module_symbol() basically provided
> this for us. The only thing really wrong with it was the lack of use
> count handling, which I fixed a while ago.
And the fact that it doesn't work if you turn module support off, which
you'd want to do on an embedded kernel. Unfortunately, this is one of
the times when you do want the MTD stuff. You either have to put up
with no MTD support, write your own, or put up with the extra space of
module symbols.
Therefore, get_module_symbol() as it stood was the wrong interface to
use, and I completely agree with Keiths decision to remove it. However,
I'm not sure that the inter_* stuff that replaced it is much better for
the reasons David has highlighted previously wrt link ordering.
_____
|_____| ------------------------------------------------- ---+---+-
| | Russell King rmk@arm.linux.org.uk --- ---
| | | | http://www.arm.linux.org.uk/personal/aboutme.html / / |
| +-+-+ --- -+-
/ | THE developer of ARM Linux |+| /|\
/ | | | --- |
+-+-+ ------------------------------------------------- /\\\ |
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-13 15:09 ` David Woodhouse
2001-01-13 19:03 ` Russell King
@ 2001-01-14 0:21 ` Keith Owens
2001-01-14 9:43 ` David Woodhouse
2001-01-14 4:04 ` Linus Torvalds
2 siblings, 1 reply; 41+ messages in thread
From: Keith Owens @ 2001-01-14 0:21 UTC (permalink / raw)
To: David Woodhouse; +Cc: linux-kernel
On Sat, 13 Jan 2001 15:09:40 +0000 (GMT),
David Woodhouse <dwmw2@infradead.org> wrote:
>Lack of [module symbol] runtime typechecking isn't a showstopper.
It is when users try to insert modules from kernel A into kernel B when
the ABI changed between A and B. This is not type checking to catch
kernel programmers, it is ABI checking to catch user errors.
This is becoming more important as the kernel moves towards hot
plugging devices, especially for binary only drivers. It is far better
for the kernel community if modutils can say "cannot load module foo
because its interfaces do not match the kernel, upgrade module foo".
That forces the maintenance load back onto the binary supplier and
removes the questions from l-k, including many of the oops reports with
binary only drivers in the module list.
Module symbol versions are the only way to catch ABI changes. I do not
want to add a mechanism for accessing symbols dynamically if it cannot
detect ABI changes, it leaves the kernel open to difficult to diagnose
user errors. I'm doing the hard work now to save everybody time later.
Ignore the fact that the existing module symbol version implementation
is broken as designed. http://gear.torque.net/kbuild/archive/1280.html
lists the major problems with make dep, genksyms has all those problems
plus several of its own. As part of the Makefile rewrite for 2.5, I am
redesigning module symbol versions from scratch.
I agree that inter_module_xxx does not check ABI. That was not for
lack of trying, but it cannot be done in 2.4, it needs a major redesign
of module symbols and the makefiles. It will be possible in 2.5.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-13 15:09 ` David Woodhouse
2001-01-13 19:03 ` Russell King
2001-01-14 0:21 ` Keith Owens
@ 2001-01-14 4:04 ` Linus Torvalds
2001-01-14 17:46 ` David Woodhouse
2 siblings, 1 reply; 41+ messages in thread
From: Linus Torvalds @ 2001-01-14 4:04 UTC (permalink / raw)
To: linux-kernel
In article <Pine.LNX.4.30.0101131413190.21182-100000@imladris.demon.co.uk>,
David Woodhouse <dwmw2@infradead.org> wrote:
>
>We don't need to overdesign it. get_module_symbol() basically provided
>this for us. The only thing really wrong with it was the lack of use
>count handling, which I fixed a while ago.
NO NO NO!
You miss _entirely_ the reason why "get_module_symbol()" was removed,
and why I will not _ever_ accept it coming back.
Hint #1: get_MODULE_symbol().
Hint #2: compiled in functionality.
The fact is, that get_module_symbol() was seriously and totally
mis-designed from the very beginning, and it was removed for THAT
reason, and it had nothing at all to do with the count handling.
So stop this discussion. It's not coming back. Live with the current
interfaces.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-14 0:21 ` Keith Owens
@ 2001-01-14 9:43 ` David Woodhouse
2001-01-14 10:05 ` Keith Owens
0 siblings, 1 reply; 41+ messages in thread
From: David Woodhouse @ 2001-01-14 9:43 UTC (permalink / raw)
To: Keith Owens; +Cc: linux-kernel
On Sun, 14 Jan 2001, Keith Owens wrote:
> This is becoming more important as the kernel moves towards hot
> plugging devices, especially for binary only drivers. It is far better
> for the kernel community if modutils can say "cannot load module foo
> because its interfaces do not match the kernel, upgrade module foo".
> That forces the maintenance load back onto the binary supplier and
> removes the questions from l-k, including many of the oops reports with
> binary only drivers in the module list.
No. The correct response to that is _already_ "You have a binary-only
module. Even in the kernel it was compiled against, you are not supported.
Goodbye".
To quote our Lord and Master:
(http://lwn.net/1999/0211/a/lt-afs.html)
>> I will strive for binary compatibility for modules, but I _expect_
>> that it will be broken. It's just too easy to have to make changes
>> that break binary-only modules, and I have too little incentive to try
>> to avoid it.
>>
>> If people feel this is a problem, I see a few alternatives:
>> - don't use stuff with binary-only modules. Just say no.
>> - work hard at making a source-version of the thing available (it
>> doesn't have to be under the GPL if it's a module, but it has to be
>> available as source so that it can be recompiled).
>> - don't upgrade
>> - drop Linux
(http://lwn.net/1999/0211/a/lt-binary.html)
>> Basically, I want people to know that when they use binary-only
>> modules, it's THEIR problem. I want people to know that in their
>> bones, and I want it shouted out from the rooftops. I want people to
>> wake up in a cold sweat every once in a while if they use binary-only
>> modules.
kaos@ocs.com.au wrote:
> Ignore the fact that the existing module symbol version implementation
> is broken as designed. http://gear.torque.net/kbuild/archive/1280.html
> lists the major problems with make dep, genksyms has all those problems
> plus several of its own. As part of the Makefile rewrite for 2.5, I am
> redesigning module symbol versions from scratch.
>
> I agree that inter_module_xxx does not check ABI. That was not for
> lack of trying, but it cannot be done in 2.4, it needs a major redesign
> of module symbols and the makefiles. It will be possible in 2.5.
This is a good thing, as long as it doesn't get in the way of real
functionality. We don't _need_ to make life easier for people running
binary-only modules. But if we can do it without making life harder for
real people, then that's nice.
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-14 9:43 ` David Woodhouse
@ 2001-01-14 10:05 ` Keith Owens
2001-01-14 10:45 ` David Woodhouse
0 siblings, 1 reply; 41+ messages in thread
From: Keith Owens @ 2001-01-14 10:05 UTC (permalink / raw)
To: David Woodhouse; +Cc: linux-kernel
On Sun, 14 Jan 2001 09:43:21 +0000 (GMT),
David Woodhouse <dwmw2@infradead.org> wrote:
>On Sun, 14 Jan 2001, Keith Owens wrote:
>> That forces the maintenance load back onto the binary supplier and
>> removes the questions from l-k, including many of the oops reports with
>> binary only drivers in the module list.
>
>No. The correct response to that is _already_ "You have a binary-only
>module. Even in the kernel it was compiled against, you are not supported.
>Goodbye".
I wish that Linus had never agreed to binary only modules. But as long
as they are allowed, I want to detect problems with binary only modules
before they hit the rest of the kernel and end up as questions on l-k.
Note I said allowed, not supported. I refuse to support any binary
only modules, my standard response to problems logged against binary
modules is "remove them and reproduce the problem". Checking for ABI
violations is not supporting binary modules, it is detecting that they
are stuffed and telling the user to go pester their supplier instead of
polluting l-k with questions that will be ignored.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-12 12:18 ` Keith Owens
@ 2001-01-14 10:16 ` Kai Henningsen
0 siblings, 0 replies; 41+ messages in thread
From: Kai Henningsen @ 2001-01-14 10:16 UTC (permalink / raw)
To: linux-kernel
kaos@ocs.com.au (Keith Owens) wrote on 12.01.01 in <31714.979301892@ocs3.ocs-net>:
> On Fri, 12 Jan 2001 13:01:12 +0100,
> Daniel Phillips <phillips@innominate.de> wrote:
> >Keith Owens wrote:
> >> I want to completely remove this multi layered method for setting
> >> initialisation order and go back to basics. I want the programmer to
> >> say "initialise E and F after G, H and I". The kernel build system
> >> works out the directed graph of initialisation order then controls the
> >> execution of startup code to satisfy this graph.
> >
> >I don't doubt you will come up with a workable solution at build time.
> >However, working out a valid graph at execution time is trivial and
> >efficient, given a list of precedence relations of the kind you're
> >suggesting. In fact you don't even have to work out the graph before
> >starting the initialization, it's also trivial to keep a count of
> >unsatisfied initialization conditions at the beginning of each
> >initialization sequence and block until the count goes to zero. (In
> >essence, evaluate a priority sort on the fly.)
>
> It is safer to evaluate the graph at link time in case somebody
> mistakenly codes a dependency loop, that is an abort case. Finding
> that you have a loop at load time and aborting the kernel is a little
> too drastic for my tastes.
>
> In any case it is academic. Linus insists on link order being
> explicitly and completely specified by the programmer and he likes the
> link order being implied by Makefile order. So there is no point in
> working on a better system.
I'm not so sure about that. I think it _should_ be possible to do both,
and get better documentation at the same time.
How about this:
* Invent some method for modules to declare these dependencies. Maybe even
in /** */ type comments, so it goes right into the documentation.
* Write a program to collect all dependencies, do a tsort on them
(alerting the developer if a loop is found), determining a reasonable
initialization order, and spitting out the relevant Makefiles or
Makefile fragments to *create* that initialization order the Linus way.
Or in other words, don't do it at runtime, don't do it at compile time, do
it at develop time. (Or patch time, perhaps.)
MfG Kai
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-14 10:05 ` Keith Owens
@ 2001-01-14 10:45 ` David Woodhouse
0 siblings, 0 replies; 41+ messages in thread
From: David Woodhouse @ 2001-01-14 10:45 UTC (permalink / raw)
To: Keith Owens; +Cc: linux-kernel
On Sun, 14 Jan 2001, Keith Owens wrote:
> Note I said allowed, not supported. I refuse to support any binary
> only modules, my standard response to problems logged against binary
> modules is "remove them and reproduce the problem". Checking for ABI
> violations is not supporting binary modules, it is detecting that they
> are stuffed and telling the user to go pester their supplier instead of
> polluting l-k with questions that will be ignored.
Sensible. As long as it doesn't give rise to reports of the type "modutils
didn't whinge so it's not the binary-only module's fault."
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-14 4:04 ` Linus Torvalds
@ 2001-01-14 17:46 ` David Woodhouse
2001-01-14 19:12 ` Linus Torvalds
0 siblings, 1 reply; 41+ messages in thread
From: David Woodhouse @ 2001-01-14 17:46 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel
On 13 Jan 2001, Linus Torvalds wrote:
> You miss _entirely_ the reason why "get_module_symbol()" was removed,
> and why I will not _ever_ accept it coming back.
>
> Hint #1: get_MODULE_symbol().
> Hint #2: compiled in functionality.
Er,... forgive me if I'm being overly dense here, but I can't see anything
fundamentally wrong in the above that wouldn't be fixed with a
judicious application of s/module_//
But I have no particular attachment to it. All I'm asking for is a way to
avoid having init order dependencies where previously there was no need
for them, by having a way to put entries in the inter_module_get() table
at compile time.
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-14 17:46 ` David Woodhouse
@ 2001-01-14 19:12 ` Linus Torvalds
2001-01-14 20:02 ` David Woodhouse
0 siblings, 1 reply; 41+ messages in thread
From: Linus Torvalds @ 2001-01-14 19:12 UTC (permalink / raw)
To: David Woodhouse; +Cc: linux-kernel
On Sun, 14 Jan 2001, David Woodhouse wrote:
>
> But I have no particular attachment to it. All I'm asking for is a way to
> avoid having init order dependencies where previously there was no need
> for them, by having a way to put entries in the inter_module_get() table
> at compile time.
Note that previously there _were_ order dependencies. In fact, I consider
it very tasteless to have modules that act differently on whether another
module is loaded. I saw some arguments saying that this is th "right
thing", and I disagree completely.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-14 19:12 ` Linus Torvalds
@ 2001-01-14 20:02 ` David Woodhouse
2001-01-14 20:15 ` Linus Torvalds
0 siblings, 1 reply; 41+ messages in thread
From: David Woodhouse @ 2001-01-14 20:02 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel
On Sun, 14 Jan 2001, Linus Torvalds wrote:
> Note that previously there _were_ order dependencies. In fact, I consider
> it very tasteless to have modules that act differently on whether another
> module is loaded. I saw some arguments saying that this is th "right
> thing", and I disagree completely.
The only valid behaviour I can think of is...
if (!feature_present)
try_to_load_it();
if (feature_present)
use_it();
else
if (we_can_live_without())
deal_with_it();
else
whinge_and_die();
In this case, it's not really depending on whether the desired feature has
been loaded yet or not. It's depending on whether the desired feature is
available or not. In this sense, inter_module_get_request() is an
improvement, because it makes that explicit.
Obviously, in the case where we'd eventually just whinge_and_die(),
usually it's best to just make this code depend on whatever feature it is
that's being used, by referencing it directly.
But in the case of the CFI probe code and also I believe DRM, we don't
actually know precisely which feature we're going to require until we've
done the hardware probe at runtime. We don't want the generic code having
a hard dependency on each possible submodule, and doing it with ifdefs
according to what happens to be compiled in is just ugly. So the above
logic was useful, and get_module_symbol(), even though it wasn't
wonderful, provided it.
The reason you didn't get the current CFI code with the rest of the update
I gave you for 2.4.0-test12 is because I'm intending to rewrite it first,
and hopefully deal with this issue in a better way while I'm at it.
But as it stands, the best option I can see is to have the generic probe
code have ifdefs on the chipset-specific options, and reference only the
ones which are present. It's not the end of the world, and as rmk
suggests, many embedded systems are run without module support in
production anyway.
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-14 20:02 ` David Woodhouse
@ 2001-01-14 20:15 ` Linus Torvalds
2001-01-14 21:15 ` David Woodhouse
0 siblings, 1 reply; 41+ messages in thread
From: Linus Torvalds @ 2001-01-14 20:15 UTC (permalink / raw)
To: David Woodhouse; +Cc: linux-kernel
On Sun, 14 Jan 2001, David Woodhouse wrote:
>
> But in the case of the CFI probe code and also I believe DRM, we don't
> actually know precisely which feature we're going to require until we've
> done the hardware probe at runtime.
That's ok.
This is what "request_module()" and "kmod" is all about. Once we probe the
hardware, the drievr itself can ask for more drivers.
I completely fail to see the arguments that have been brought up for drm
doing ugly things. The code should simply do
drm_agp_head_t * head = inter_module_get("drm_agp");
if (!head) {
request_module("drm-agp");
head = inter_module_get("drm_agp");
if (!head)
return -ENOAGP;
}
and be done with it. THE ABOVE MAKES SENSE. The code says _exactly_ what
the module wants to do: it wants to find the AGP support, and if it cannot
find the AGP support it wants to load them.
The arguments about how the user should load things in some specific order
or whatever are complete crap. All the support is there, and whining about
it is not going to change my opinion in the least.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-14 20:15 ` Linus Torvalds
@ 2001-01-14 21:15 ` David Woodhouse
2001-01-14 21:47 ` Linus Torvalds
0 siblings, 1 reply; 41+ messages in thread
From: David Woodhouse @ 2001-01-14 21:15 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel
On Sun, 14 Jan 2001, Linus Torvalds wrote:
> This is what "request_module()" and "kmod" is all about. Once we probe the
> hardware, the drievr itself can ask for more drivers.
>
> I completely fail to see the arguments that have been brought up for drm
> doing ugly things. The code should simply do
>
> drm_agp_head_t * head = inter_module_get("drm_agp");
>
> if (!head) {
> request_module("drm-agp");
> head = inter_module_get("drm_agp");
> if (!head)
> return -ENOAGP;
> }
>
> and be done with it. THE ABOVE MAKES SENSE. The code says _exactly_ what
> the module wants to do: it wants to find the AGP support, and if it cannot
> find the AGP support it wants to load them.
It's the same with CFI command-set-specific code. Except that the
command-set specific code didn't previously have to be initialised at all,
and now we've got to initialise it (and have it call
inter_module_register) before it's required by the cfi_probe code.
The difference here is that while drm_agp actually had to do some hardware
initialisation, the CFI command set handlers didn't - the only thing their
module_init routine does is call inter_module_register(). So we've
introduced the init order dependencies where previously they weren't
necessary.
That's the one flaw in the inter_module_get() stuff - we could do with a
way to put entries in the table at _compile_ time, rather than _only_ at
run time.
While I accept that we can't eliminate init order dependencies completely,
I still think we should avoid them where it's possible. Which it would be
in this case, without much difficulty at all.
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-14 21:15 ` David Woodhouse
@ 2001-01-14 21:47 ` Linus Torvalds
2001-01-14 21:57 ` David Woodhouse
2001-01-14 23:00 ` Keith Owens
0 siblings, 2 replies; 41+ messages in thread
From: Linus Torvalds @ 2001-01-14 21:47 UTC (permalink / raw)
To: David Woodhouse; +Cc: linux-kernel
On Sun, 14 Jan 2001, David Woodhouse wrote:
>
> That's the one flaw in the inter_module_get() stuff - we could do with a
> way to put entries in the table at _compile_ time, rather than _only_ at
> run time.
Ok, I can buy that. Not having to initialize explicitly would be nice, but
if so we should make module loading do it automatically too, so that we
don't generate unnecessary differences between module and compiled in (ie
I'd rather avoid the situation that "if you're a module, you need to
explicitly export your inter_module_stuff(), while if you're compiled-in
it will be exported automatically").
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-14 21:47 ` Linus Torvalds
@ 2001-01-14 21:57 ` David Woodhouse
2001-01-14 23:00 ` Keith Owens
1 sibling, 0 replies; 41+ messages in thread
From: David Woodhouse @ 2001-01-14 21:57 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, kaos
On Sun, 14 Jan 2001, Linus Torvalds wrote:
> On Sun, 14 Jan 2001, David Woodhouse wrote:
> > That's the one flaw in the inter_module_get() stuff - we could do with a
> > way to put entries in the table at _compile_ time, rather than _only_
> > at run time.
> Ok, I can buy that. Not having to initialize explicitly would be nice, but
> if so we should make module loading do it automatically too, so that we
> don't generate unnecessary differences between module and compiled in (ie
> I'd rather avoid the situation that "if you're a module, you need to
> explicitly export your inter_module_stuff(), while if you're compiled-in
> it will be exported automatically").
Yep. Modutils can probably handle that case without too much difficulty,
if we go with sticking the static inter_module_entries in a special ELF
section as I originally suggested. Keith?
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-14 21:47 ` Linus Torvalds
2001-01-14 21:57 ` David Woodhouse
@ 2001-01-14 23:00 ` Keith Owens
2001-01-15 9:09 ` David Woodhouse
1 sibling, 1 reply; 41+ messages in thread
From: Keith Owens @ 2001-01-14 23:00 UTC (permalink / raw)
To: Linus Torvalds; +Cc: David Woodhouse, linux-kernel
On Sun, 14 Jan 2001 13:47:29 -0800 (PST),
Linus Torvalds <torvalds@transmeta.com> wrote:
>On Sun, 14 Jan 2001, David Woodhouse wrote:
>> That's the one flaw in the inter_module_get() stuff - we could do with a
>> way to put entries in the table at _compile_ time, rather than _only_ at
>> run time.
>
>Ok, I can buy that. Not having to initialize explicitly would be nice, but
>if so we should make module loading do it automatically too ...
It might be nice but it is also expensive. Adding static
inter_module_xxx tables requires
* changes to linux/modules.h to define the new table format and
* changes to vmlinux.lds for _every_ architecture to bring all the
static tables together in vmlinux and
* new initialisation code in module.c to read and load all the static
tables at boot time and
* extra code in modutils to find any static tables in modules and
* an extension to struct modules to let modutils pass information about
the static tables to the kernel and
* the kernel code will only work with an upgraded modutils.
That is a lot of work for a very few special cases. OTOH, you could
just add a few lines of __initcall code in two source files (which I
did when I wrote inter_module_xxx) and swap the order of 3 lines in
drivers/mtd/Makefile. Guess which alternative I am going for?
IMHO any automatic method that relies on ELF sections and/or modutils
support is the wrong approach, it is a complex solution with external
dependencies when we already have a simple solution with no external
dependencies. What next, static tables for file system registration,
for device registration?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Where did vm_operations_struct->unmap in 2.4.0 go?
2001-01-14 23:00 ` Keith Owens
@ 2001-01-15 9:09 ` David Woodhouse
0 siblings, 0 replies; 41+ messages in thread
From: David Woodhouse @ 2001-01-15 9:09 UTC (permalink / raw)
To: Keith Owens; +Cc: Linus Torvalds, linux-kernel
kaos@ocs.com.au said:
> That is a lot of work for a very few special cases. OTOH, you could
> just add a few lines of __initcall code in two source files (which I
> did when I wrote inter_module_xxx) and swap the order of 3 lines in
> drivers/mtd/Makefile. Guess which alternative I am going for?
I've already worked round it for the 2.[024] case by reintroducing the
ifdefs. I assume here that we're planning for 2.5. As part of the many
changes that are going to be introduced during 2.5, this shouldn't be too
intrusive.
Once it's actually usable for the common case, it won't just be 'a very few
special cases' any more. But that's all 2.5 material.
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 41+ messages in thread
end of thread, other threads:[~2001-01-15 9:09 UTC | newest]
Thread overview: 41+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2001-01-10 3:27 Where did vm_operations_struct->unmap in 2.4.0 go? Allen Unueco
2001-01-10 3:50 ` Keith Owens
2001-01-11 5:38 ` Antony Suter
2001-01-11 6:05 ` Keith Owens
2001-01-11 11:42 ` David Woodhouse
2001-01-11 12:12 ` Keith Owens
2001-01-11 12:32 ` David Woodhouse
2001-01-11 12:46 ` Keith Owens
2001-01-11 13:09 ` Alan Cox
2001-01-11 13:14 ` Keith Owens
2001-01-12 2:12 ` Ingo Oeser
2001-01-12 2:30 ` Keith Owens
2001-01-12 10:27 ` David Woodhouse
2001-01-12 11:55 ` Keith Owens
2001-01-12 13:40 ` David Woodhouse
2001-01-12 12:01 ` Daniel Phillips
2001-01-12 12:18 ` Keith Owens
2001-01-14 10:16 ` Kai Henningsen
2001-01-11 13:25 ` David Woodhouse
[not found] <3A5EFC56.F1A5BCE0@mira.net>
2001-01-12 19:11 ` Christian Zander
2001-01-13 1:11 ` Keith Owens
2001-01-13 10:46 ` David Woodhouse
2001-01-13 12:06 ` Keith Owens
2001-01-13 15:09 ` David Woodhouse
2001-01-13 19:03 ` Russell King
2001-01-14 0:21 ` Keith Owens
2001-01-14 9:43 ` David Woodhouse
2001-01-14 10:05 ` Keith Owens
2001-01-14 10:45 ` David Woodhouse
2001-01-14 4:04 ` Linus Torvalds
2001-01-14 17:46 ` David Woodhouse
2001-01-14 19:12 ` Linus Torvalds
2001-01-14 20:02 ` David Woodhouse
2001-01-14 20:15 ` Linus Torvalds
2001-01-14 21:15 ` David Woodhouse
2001-01-14 21:47 ` Linus Torvalds
2001-01-14 21:57 ` David Woodhouse
2001-01-14 23:00 ` Keith Owens
2001-01-15 9:09 ` David Woodhouse
2001-01-13 11:46 ` Christian Zander
2001-01-13 12:23 ` Keith Owens
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox