Re: [patch] drivers: wait for threaded probes between initcall levels

Re: [patch] drivers: wait for threaded probes between initcall levels

[Adam J. Richter]

On Fri, 27 Oct 2006 13:42:44 -0700 (PDT), Linus Torvals wrote:
>        static struct semaphore outstanding;
[...]
>        static void allow_parallel(int n)
[...]
>        static void wait_for_parallel(int n)
[...]
>        static void execute_in_parallel(int (*fn)(void *), void *arg)

	This interface would have problems with nesting.

	As a realistic example of nesting, imagine if this facility were
used for initcalls, and one of those initcalls also used this facility to
attempt parallel initialization of several hardware devices attached
to some bus.  In this scenario, do_initcalls would call allow_parallelism(10),
and then one of the initcalls that wanted to spawn its own set of
parallel processes would also call allow_parallel(10) (unintentionally
making the number of allowed processes 20), kick off its parallel
processes, and then call wait_for_parallel(), which could return
prematurely, which could easily lead to one of the child threads that
was incorrectly assumed to have finished to then corrupt memory or do any
number of other things that leads to a crash that is not reliably
reproducible.

	Here are some possible ways to address this problem, and
their drawbacks.

	1. Just document that nesting is prohibited, adding some BUG()
test to prevent such use.  This would limit the usefulness of this
facility, and create some unnecessary coordination issues among
kernel developers in arguing policy over who gets to use this facility.

	2. Turn the "outstanding" counting semaphore into a parameter.
Each group of related threads would share this parameter.  For example,
do_initcalls might look something like this:

struct semaphore initcalls_sem = SEMAPHORE_INITIALIZER(...);

allow_parallel(PARALLELISM, &initcalls_sem);

for (call = __initcall_start; call < __initcall_end; call++)
	execute_in_parallel(call, NULL, &initcalls_sem);

wait_for_parallel(PARALLEISM, &initcalls_sem);

	The drawback of this solution is that the limitation on the total
number of parallel processes is not global.  So, the number of
parallel processes in a nesting situation could get quite high.
For example, if 10 top level threads each initiated another 10
secondary threads, that's up to 111 threads with just a nesting
depth of 2.

	3. Add an rw_semaphore passed as a parameter for
wait_for_parallelism, but keep original static semaphore for limiting
the parallelism.  wait_for_parallel would be the only function ever
to block on the rw_semaphore, so that should avoid any problem with
ordering of taking the two semaphores--I think.

	This solution deadlocks if the nesting depth exceeds the maximum
number of threads allowed, which could realistically occur when the maximum
parallelism allowed is 1.

	4. Same as #3, but also increase the global thread count by 1 on
every call to allow_parallel, and decrease it by one in the matching
wait_for_parallel.  The drawback here is that setting the global
number of threads to one at the outset would no longer guarantee
strict serial execution, which is minor compared to the other
problems listed above.  If we want to guarantee serial execution
when PARALLELISM=1, I think that can be arranged with a little more
complexity, but I'd like to know if that is actually important.

	I have attached an example of approach #4 below, also completely
untested, with no attempt made to try to compile it.

Adam Richter




#define PARALLELISM (10)

static struct semaphore outstanding;

struct thread_exec {
        int (*fn)(void *);
        void *arg;
	struct completion args_copied;
	struct rw_semaphore *fn_done;
};

/* In some .h file: */
static inline void start_parallel(void)
{
	up(&outstanding);
}

/* In some .h file: */
static inline void wait_for_parallel(struct rw_semaphore *fn_done)
{
	down_write(fn_done);
	down(&outstanding);
}

void allow_parallel(int n)	/* called once, at boot time */
{
        while (--n >= 0)
                up(&outstanding);
}

static int do_in_parallel(void *arg)
{
        struct thread_exec *p = arg;
        int (*fn)(void *) = p->fn;
        void *arg = p->arg;
	struct rw_semaphore *fn_done = p->fn_done;
        int retval;

        /* Tell the caller we are done with the arguments */
        complete(&p->args_copied);

        /* Do the actual work in parallel */
        retval = p->fn(p->arg);

        /*
         * And then tell the rest of the world that we've
         * got one less parallel thing outstanding..
         */
	up_read(fn_done);
        up(&outstanding);
        return retval;
}

void execute_in_parallel(int (*fn)(void *),
			void *arg,
			struct semaphore *fn_done)
{
        struct thread_exec arg = { .fn = fn, .arg = arg };

        /* Make sure we can have more outstanding parallel work */
        down(&outstanding);

        arg.fn = fn;
        arg.arg = arg;
	arg.fn_done = fn_done;
	down_read(fn_done);
        init_completion(&arg.args_copied);

        kernel_thread(do_in_parallel, &arg);

        wait_for_completion(&arg.args_copied)
}


Example of usage:

	/* Earlier in the boot process... */
        allow_parallel(PARALLELISM);


static void __init do_initcalls(void)
{
	DECLARE_RWSEM(initcalls_done);
	start_parallel();
	for (call = __initcall_start; call < __initcall_end; call++)
		execute_in_parallel(call, NULL, &initcalls_done);

	wait_for_parallel(&initcalls_done);
}

Re: [patch] drivers: wait for threaded probes between initcall levels

[Russell King]

On Sat, Oct 28, 2006 at 04:23:35PM +0800, Adam J. Richter wrote:
> 	This interface would have problems with nesting.

Adam (and the rest of the parallel crowd),

Just a passing thought (and nothing more)...

How does this behave with PCMCIA initialisation with a Cardbus card
inserted?

This is one scenario which needs checking before any of this parallel
probe code goes anywhere near mainline, since it's possible for the
Cardbus (PCI) device to be added and therefore probed while the Yenta
probe (PCI) is still running.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: [patch] drivers: wait for threaded probes between initcall levels

[Russell King]

On Sat, Oct 28, 2006 at 10:22:54AM +0100, Russell King wrote:
> On Sat, Oct 28, 2006 at 04:23:35PM +0800, Adam J. Richter wrote:
> > 	This interface would have problems with nesting.
> 
> Adam (and the rest of the parallel crowd),
> 
> Just a passing thought (and nothing more)...
> 
> How does this behave with PCMCIA initialisation with a Cardbus card
> inserted?
> 
> This is one scenario which needs checking before any of this parallel
> probe code goes anywhere near mainline, since it's possible for the
> Cardbus (PCI) device to be added and therefore probed while the Yenta
> probe (PCI) is still running.

Can someone make sure Adam gets my email please?  His server rejects
messages from my domain:

  adam@yggdrasil.com
    SMTP error from remote mail server after MAIL FROM:<rmk+adam=yggdrasil.com@arm.linux.org.uk> SIZE=3022:
    host yggdrasil.com [61.49.148.168]: 553 5.1.8 <rmk+adam=yggdrasil.com@arm.linux.org.uk>... Domain of sender address rmk+adam=yggdrasil.com@arm.linux.org.uk does not exist

(the error is obviously utter rubbish.)

Thanks.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: [patch] drivers: wait for threaded probes between initcall levels

[Linus Torvalds]

On Sat, 28 Oct 2006, Adam J. Richter wrote:

> On Fri, 27 Oct 2006 13:42:44 -0700 (PDT), Linus Torvals wrote:
> >        static struct semaphore outstanding;
> [...]
> >        static void allow_parallel(int n)
> [...]
> >        static void wait_for_parallel(int n)
> [...]
> >        static void execute_in_parallel(int (*fn)(void *), void *arg)
> 
> 	This interface would have problems with nesting.

You miss the point.

They _wouldn't_ be nested.

The "allow_parallel()" and "wait_for_parallel()" calls would be at some 
top-level situation (ie initcalls looping).

Nobody else than the top level would _ever_ use them. Anything under that 
level would just say "I want to do this in parallel" - which is just a 
statement, and has no nesting issues in itself.

The whole notion of "I want to do this in parallel" is basically 
non-nesting. If something is parallel, it's by definition not ordered, and 
thus nesting cannot make sense. All the "ordered" stuff would be either 
done without using "execute_in_parallel()" at all, or it would be ordered 
_within_ one thread that is executed in parallel.

		Linus

Re: [patch] drivers: wait for threaded probes between initcall levels

[Adam J. Richter]

On 2006-10-28 23:55:42, Linus Torvalds wrote:
>On Sun, 29 Oct 2006, Adam J. Richter wrote:
>> 
>>       If only calls to execute_in_parallel nest, your original
>> implementation would always deadlock when the nesting depth exceeds
>> the allowed number of threads, and [...]
>
>No, I'm saying that nesting simply shouldn't be _done_. There's no real 
>reason. Any user would be already either parallel or doesn't need to be 
>parallel at all. Why would something that already _is_ parallel start 
>another parallel task?

	Suppose the system is initializing PCI cards in parallel.  The
thread that is initializing one particular PCI card discovers that it
is initializing a firewire controller.  After the already "parallel"
PCI firewire probe function initializes the card, it is going to
enumerate the devices attached to the firewire cable and spawn
separate threads to initialize drivers for each of these several
firewire devices.

	One way avoid this depth-first descent would be to change
device_attach() in drivers/base/dd.c to queue its work to helper daemon.

	Either way, we're talking about a few lines of code in
execute_in_parallel that can easily be added later if needed.  If you
really think all calls to execute_parallel will be done by the main
kernel thread, I suggest someone add a BUG_ON() statement to that
effect to execute_parallel to see.

	I would also like to suggest a very different approach, which
would not be quite as fast, but which I think would be more flexible
and would work partly by making the kernel do _less_.

	Perhaps we could offer a boot option to limit device_attach to
consider only drivers named by that option, such as
"limit_drivers=vc,ramdisk".  (If a user screwed his boot process with
the wrong limit_drivers= options, fixing the problem would be just a
matter of just eliminating the option.)  All other driver-device
bindings would be done explicitly by a user level mechanism, which
would implicitly provide the process context for blocking.  That is,
the parallelization would occur by a sysfs watcher like udev spawning
separate threads to call the user level sysfs interface for attaching
devices to drivers.  User level would also handle matching driver and
device ID information, determining parallelization limits, probe
order, choosing between multiple drivers available for devices or
deliberately not binding a driver to a device, and perhaps executing
other custom user level code along the way.

	Because the threads involved would come from clone() executed
by a user level daemon sysfs watcher like udev, execute_in_parallel()
would be less used, perhaps not used at all, depending on whether
parts the boot process besides walking the device tree would benefit
much from parallelization.

Adam Richter

Re: [patch] drivers: wait for threaded probes between initcall levels

[Adam J. Richter]

On 2006-10-28 19:41:50, Linus Torvalds wrote:
>On Sat, 28 Oct 2006, Adam J. Richter wrote:
>
>> On Fri, 27 Oct 2006 13:42:44 -0700 (PDT), Linus Torvals wrote:
>> >        static struct semaphore outstanding;
>> [...]
>> >        static void allow_parallel(int n)
>> [...]
>> >        static void wait_for_parallel(int n)
>> [...]
>> >        static void execute_in_parallel(int (*fn)(void *), void *arg)
>> 
>>       This interface would have problems with nesting.
>
>You miss the point.
>
>They _wouldn't_ be nested.
>
>The "allow_parallel()" and "wait_for_parallel()" calls would be at some 
>top-level situation (ie initcalls looping).
>
>Nobody else than the top level would _ever_ use them. Anything under that 
>level would just say "I want to do this in parallel" - which is just a 
>statement, and has no nesting issues in itself.

	If only calls to execute_in_parallel nest, your original
implementation would always deadlock when the nesting depth exceeds
the allowed number of threads, and also potentially in some shallower
nesting depths given a very unlucky order of execution.  In your
original message, you mentioned allowing the parallelism limit to be
set as low as 1.

	One solution to this problem would be to have
execute_in_parallel execute the function directly when no threads are
available to do it, rather than blocking.  The disadvantage is that,
if no thread is immediately available, the call to
execute_in_parallel would not return until the function that was
passed in finishes, even if more threads become available much sooner.

	Here is what I am referring to, again completely untested:


	static void execute_in_parallel(int (*fn)(void *), void *arg)
	{
		struct thread_exec arg = { .fn = fn, .arg = arg };

		/* If no threads are available, call the function directly. */
		if (down_trylock(&outstanding) != 0) {
			fn(arg);
			return;
		}

		arg.fn = fn;
		arg.arg = arg;
		init_completion(&arg.args_copied);

		kernel_thread(do_in_parallel, &arg);

		/* We need to wait until our "arg" is safe */
		wait_for_completion(&arg.args_copied)
	}


Adam Richter

Re: [patch] drivers: wait for threaded probes between initcall levels

[Linus Torvalds]

On Sun, 29 Oct 2006, Adam J. Richter wrote:
> 
> 	If only calls to execute_in_parallel nest, your original
> implementation would always deadlock when the nesting depth exceeds
> the allowed number of threads, and also potentially in some shallower
> nesting depths given a very unlucky order of execution.  In your
> original message, you mentioned allowing the parallelism limit to be
> set as low as 1.

No, I'm saying that nesting simply shouldn't be _done_. There's no real 
reason. Any user would be already either parallel or doesn't need to be 
parallel at all. Why would something that already _is_ parallel start 
another parallel task?

IOW, what I was trying to say (perhaps badly) is that "nesting" really 
isn't a sensible operation - you'd never do it. You'd do the "startup" and 
"shutdown" things at the very highest level, and then in between those 
calls you can start a parallel activity at any depth of the call stack, 
but at no point does it really make sense to start it from within 
something that is already parallel.

Hmm?

(Btw, you do seem to have some strange email setup that doesn't allow me 
to email you directly, I just get a bounce. I'll try again, but you'll 
probably pick this up on linux-kernel rather than in your private 
mailbox).

		Linus

Re: [patch] drivers: wait for threaded probes between initcall levels

[Kyle Moffett]

On Oct 28, 2006, at 19:55:42, Linus Torvalds wrote:
> On Sun, 29 Oct 2006, Adam J. Richter wrote:
>> If only calls to execute_in_parallel nest, your original  
>> implementation would always deadlock when the nesting depth  
>> exceeds the allowed number of threads, and also potentially in  
>> some shallower nesting depths given a very unlucky order of  
>> execution.  In your original message, you mentioned allowing the  
>> parallelism limit to be set as low as 1.
>
> No, I'm saying that nesting simply shouldn't be _done_. There's no  
> real reason. Any user would be already either parallel or doesn't  
> need to be parallel at all. Why would something that already _is_  
> parallel start another parallel task?

Well, I would argue that there actually _is_ a reason; the same  
reason that GNU make communicates between recursive invocations to  
control the maximum number of in-progress execution threads ("-J4"  
will have 4 make targets running at once, _even_ in the presence of  
recursive make invocations and nested directories).  Likewise in the  
context of recursively nested busses and devices; multiple PCI  
domains, USB, Firewire, etc.

> IOW, what I was trying to say (perhaps badly) is that "nesting"  
> really isn't a sensible operation - you'd never do it. You'd do the  
> "startup" and "shutdown" things at the very highest level, and then  
> in between those calls you can start a parallel activity at any  
> depth of the call stack, but at no point does it really make sense  
> to start it from within something that is already parallel.

Well, perhaps it does.  If I have (hypothetically) a 64-way system  
with several PCI domains, I should be able to not only start scanning  
each PCI domain individually,  but once each domain has been scanned  
it should be able to launch multiple probing threads, one for each  
device on the PCI bus.  That is, assuming that I have properly set up  
my udev to statically name devices.

Perhaps it would make more sense for the allow_parallel() call to  
specify instead a number of *additional* threads to spawn, such that  
allow_parallel(0) on the top level would force the normal serial boot  
order, allow_parallel(1) would allow one probing thread and the init  
thread to both probe hardware at once, etc.

With a little per-thread context on the stack, you could fairly  
easily keep track of the number of allowed sub-threads on a per- 
allow_parallel() basis.  Before you spawn each new thread, create its  
new per-thread state for it and pass its pointer to the child  
thread.  With each new do_in_parallel() call it would down the  
semaphores for each "context" up the tree until it hit the top, and  
then it would allocate a new context and fork off a new thread for  
the _previous_ call to do_in_parallel().  The last call would remain  
unforked, and so finalize_parallel() would first execute that call in  
the current thread and then reap all of the children by waiting on  
their completions then freeing their contexts.

I admit the complexity is a bit high, but since the maximum nesting  
is bounded by the complexity of the hardware and the number of  
busses, and the maximum memory-allocation is strictly limited in the  
single-threaded case this could allow 64-way systems to probe all  
their hardware an order of magnitude faster than today without  
noticeably impacting an embedded system even in the absolute worst case.

I _believe_ that this should also be coupled with a bit of cleanup of  
probe-order dependencies.  If a subsystem depends on another being  
initialized, the depended-on one could very easily export a  
wait_for_foo_init() function:

DECLARE_COMPLETION(foo_init_completion);
static int foo_init_result;

int wait_for_foo_init()
{
	wait_for_completion(&foo_init_completion);
	return foo_init_result;
}

int foo_init(struct parallel_state *state)
{
	struct foo_device *dev;
	
	allow_parallel(state, 3);

#if 1
	/* Assumes: int foo_probe_device(void *dev); */
	for_each_foo_device(dev)
		do_in_parallel(state, foo_probe_device, dev);
#else
	/* Assumes: int foo_probe_device(struct parallel_state *state,
			void *dev); */
	for_each_foo_device(dev)
		do_in_parallel_nested(state, foo_probe_device, dev);
#endif

	foo_init_result = finalize_parallel(state);
	complete(&foo_init_completion);
	return foo_init_result;
}

And of course if you wanted to init both the foo and bar busses in  
parallel you could implement a virtually identical function using the  
do_in_parallel_nested() variant on top of the foo_init() function.

I'm working on a sample implementation of the allow_parallel()  
do_in_parallel() and finalize_parallel() functions, but I'm going to  
take the time to make sure its right.  In the mean-time, I'm  
interested in any comments.

Cheers,
Kyle Moffett

Re: [patch] drivers: wait for threaded probes between initcall levels

[Arjan van de Ven]

> I admit the complexity is a bit high, but since the maximum nesting  
> is bounded by the complexity of the hardware and the number of  
> busses, and the maximum memory-allocation is strictly limited in the  
> single-threaded case this could allow 64-way systems to probe all  
> their hardware an order of magnitude faster than today without  
> noticeably impacting an embedded system even in the absolute worst case.


how much of this complexity goes away if you consider the
scanning/probing as a series of "work elements", and you end up with a
queue of work elements that threads can pull work off one at a time (so
that if one element blocks the others just continue to flow). If you
then find, say, a new PCI bus you just put another work element to
process it at the end of the queue, or you process it synchronously. Etc
etc.

All you need to scale then is the number of worker threads on the
system, which should be relatively easy to size....
(check every X miliseconds if there are more than X outstanding work
elements, if there are, spawn one new worker thread if the total number
of worker threads is less than the system wide max. Worker threads die
if they have nothing to do for more than Y miliseconds)

Oh and... we have a concept for this already, or at least mostly, via
the work queue mechanism :)

Re: [patch] drivers: wait for threaded probes between initcall levels

[Xavier Bestel]

On Mon, 2006-10-30 at 15:38 +0100, Arjan van de Ven wrote:

> how much of this complexity goes away if you consider the
> scanning/probing as a series of "work elements", and you end up with a
> queue of work elements that threads can pull work off one at a time (so
> that if one element blocks the others just continue to flow). If you
> then find, say, a new PCI bus you just put another work element to
> process it at the end of the queue, or you process it synchronously. Etc
> etc.
> 
> All you need to scale then is the number of worker threads on the
> system, which should be relatively easy to size....
> (check every X miliseconds if there are more than X outstanding work
> elements, if there are, spawn one new worker thread if the total number
> of worker threads is less than the system wide max. Worker threads die
> if they have nothing to do for more than Y miliseconds)

Instead of checking every X ms, just check at each job insertion.

	Xav

Re: [patch] drivers: wait for threaded probes between initcall levels

[Arjan van de Ven]

On Mon, 2006-10-30 at 16:00 +0100, Xavier Bestel wrote:
> On Mon, 2006-10-30 at 15:38 +0100, Arjan van de Ven wrote:
> 
> > how much of this complexity goes away if you consider the
> > scanning/probing as a series of "work elements", and you end up with a
> > queue of work elements that threads can pull work off one at a time (so
> > that if one element blocks the others just continue to flow). If you
> > then find, say, a new PCI bus you just put another work element to
> > process it at the end of the queue, or you process it synchronously. Etc
> > etc.
> > 
> > All you need to scale then is the number of worker threads on the
> > system, which should be relatively easy to size....
> > (check every X miliseconds if there are more than X outstanding work
> > elements, if there are, spawn one new worker thread if the total number
> > of worker threads is less than the system wide max. Worker threads die
> > if they have nothing to do for more than Y miliseconds)
> 
> Instead of checking every X ms, just check at each job insertion.

that would lead to a too eager amount of threads if processing the jobs
is really really quick ...


-- 
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org

Re: [patch] drivers: wait for threaded probes between initcall levels

[Xavier Bestel]

On Mon, 2006-10-30 at 16:05 +0100, Arjan van de Ven wrote:
> On Mon, 2006-10-30 at 16:00 +0100, Xavier Bestel wrote:
> > On Mon, 2006-10-30 at 15:38 +0100, Arjan van de Ven wrote:
> > 
> > > how much of this complexity goes away if you consider the
> > > scanning/probing as a series of "work elements", and you end up with a
> > > queue of work elements that threads can pull work off one at a time (so
> > > that if one element blocks the others just continue to flow). If you
> > > then find, say, a new PCI bus you just put another work element to
> > > process it at the end of the queue, or you process it synchronously. Etc
> > > etc.
> > > 
> > > All you need to scale then is the number of worker threads on the
> > > system, which should be relatively easy to size....
> > > (check every X miliseconds if there are more than X outstanding work
> > > elements, if there are, spawn one new worker thread if the total number
> > > of worker threads is less than the system wide max. Worker threads die
> > > if they have nothing to do for more than Y miliseconds)
> > 
> > Instead of checking every X ms, just check at each job insertion.
> 
> that would lead to a too eager amount of threads if processing the jobs
> is really really quick ...

Don't you have a "no more than X threads at once" limit ? You just
*check* at job insertion, not unconditionnaly fork.

	Xav

Re: [patch] drivers: wait for threaded probes between initcall levels

[Kyle Moffett]

On Oct 30, 2006, at 09:38:00, Arjan van de Ven wrote:
>> I admit the complexity is a bit high, but since the maximum  
>> nesting is bounded by the complexity of the hardware and the  
>> number of busses, and the maximum memory-allocation is strictly  
>> limited in the single-threaded case this could allow 64-way  
>> systems to probe all their hardware an order of magnitude faster  
>> than today without noticeably impacting an embedded system even in  
>> the absolute worst case.
>
> how much of this complexity goes away if you consider the scanning/ 
> probing as a series of "work elements", and you end up with a queue  
> of work elements that threads can pull work off one at a time (so  
> that if one element blocks the others just continue to flow). If  
> you then find, say, a new PCI bus you just put another work element  
> to process it at the end of the queue, or you process it  
> synchronously. Etc etc.

Well, I suppose the "trick" would be to ensure that the top-level  
code can probe multiple independent busses in parallel, while  
allowing certain of those to serialize their execution order for  
whatever reason without changing the resulting linear order.  This  
would make it possible to have independent pci.multithread_probe=1  
and scsi.multithread_probe=1 arguments so the sysadmin can force  
serialization for one subsystem if they don't have their device- 
numbering issues with that subsystem entirely sorted out.

Cheers,
Kyle Movvett

Re: [patch] drivers: wait for threaded probes between initcall levels

[Matthew Wilcox]

On Mon, Oct 30, 2006 at 09:23:10AM -0500, Kyle Moffett wrote:
> recursive make invocations and nested directories).  Likewise in the  
> context of recursively nested busses and devices; multiple PCI  
> domains, USB, Firewire, etc.

I don't think you know what a PCI domain is ...

> Well, perhaps it does.  If I have (hypothetically) a 64-way system  
> with several PCI domains, I should be able to not only start scanning  
> each PCI domain individually,  but once each domain has been scanned  
> it should be able to launch multiple probing threads, one for each  
> device on the PCI bus.  That is, assuming that I have properly set up  
> my udev to statically name devices.

There's still one spinlock that protects *all* accesses to PCI config
space.  Maybe we should make it one per PCI root bridge or something,
but even that wouldn't help some architectures.

> I admit the complexity is a bit high, but since the maximum nesting  
> is bounded by the complexity of the hardware and the number of  
> busses, and the maximum memory-allocation is strictly limited in the  
> single-threaded case this could allow 64-way systems to probe all  
> their hardware an order of magnitude faster than today without  
> noticeably impacting an embedded system even in the absolute worst case.

To be honest, I think just scaling PARALLEL to NR_CPUS*4 or something
would be a reasonable way to go.

If people actually want to get serious about this, I know the PPC folks
have some openfirmware call that tells them about power domains and how
many scsi discs they can spin up at one time (for example).  Maybe
that's not necessary; if we can figure out what the system's max power
draw is and how close we are to it, we can decide whether to spawn
another thread or not.

It's quite complicated.  You can spin up a disc over *here*, but not
over *there* ... this really is a gigantic can of worms being opened.

Re: [patch] drivers: wait for threaded probes between initcall levels

[Kyle Moffett]

On Oct 30, 2006, at 09:42:59, Matthew Wilcox wrote:
> On Mon, Oct 30, 2006 at 09:23:10AM -0500, Kyle Moffett wrote:
>> recursive make invocations and nested directories).  Likewise in  
>> the context of recursively nested busses and devices; multiple PCI  
>> domains, USB, Firewire, etc.
>
> I don't think you know what a PCI domain is ...

Fair enough, I guess I don't, really...

>> Well, perhaps it does.  If I have (hypothetically) a 64-way system  
>> with several PCI domains, I should be able to not only start  
>> scanning each PCI domain individually,  but once each domain has  
>> been scanned it should be able to launch multiple probing threads,  
>> one for each device on the PCI bus.  That is, assuming that I have  
>> properly set up my udev to statically name devices.
>
> There's still one spinlock that protects *all* accesses to PCI  
> config space.  Maybe we should make it one per PCI root bridge or  
> something, but even that wouldn't help some architectures.

Well, yes, but it would help some architectures.  It would seem  
rather stupid to build a hardware limitation into a 64+ cpu system  
such that it cannot initialize or reconfigure multiple pieces of  
hardware at once.  It also would help for more "mundane" systems such  
as my "Quad" G5 desktop which takes an appreciable time to probe all  
the various PCI, USB, SATA, and  Firewire devices in the system.

Cheers,
Kyle Moffett

Re: [patch] drivers: wait for threaded probes between initcall levels

[Matthew Wilcox]

On Mon, Oct 30, 2006 at 01:47:53PM -0500, Kyle Moffett wrote:
> Well, yes, but it would help some architectures.  It would seem  
> rather stupid to build a hardware limitation into a 64+ cpu system  
> such that it cannot initialize or reconfigure multiple pieces of  
> hardware at once.  It also would help for more "mundane" systems such  
> as my "Quad" G5 desktop which takes an appreciable time to probe all  
> the various PCI, USB, SATA, and  Firewire devices in the system.

Probing PCI devices really doesn't take that long.  It's the extra stuff
the drivers do at ->probe that takes the time.  And the stand-out
offender here is SCSI (and FC), which I'm working to fix.  Firewire, USB
and SATA are somewhere intermediate.

Re: [patch] drivers: wait for threaded probes between initcall levels

[Grant Grundler]

On Mon, Oct 30, 2006 at 12:13:07PM -0700, Matthew Wilcox wrote:
> Probing PCI devices really doesn't take that long.

Yeah - usually measured in "milliseconds".

> It's the extra stuff
> the drivers do at ->probe that takes the time.  And the stand-out
> offender here is SCSI (and FC), which I'm working to fix.  Firewire, USB
> and SATA are somewhere intermediate.

ISTR that the SATA Port timeout is 5 seconds or something like that.
And some cards have lots of ports...so my impression is SATA would
benefit alot from parallelism as well.

I'm certainly no SATA expert...maybe someone else could speak
more definitely on the topic of worst case SATA timeout.

thanks,
grant

Linux 2.6.19-rc3

[Linus Torvalds]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 36476 bytes --]


Ok,
 a few days late, because I'm a retard and didn't think of doing a release 
when I should have. 

I'm also a bit grumpy, because I think people are sending me more stuff 
than they should at this stage in the game. We've been pretty good about 
keeping the later -rc releases quiet, please keep it that way.

That said, it's mostly harmless cleanups and some architecture updates. 
And some of the added warnings about unused return values have caused a 
number of people to add error handling. And on the driver front, there's 
mainly new driver ID's, and a couple of new drivers.

Shortlog appended,

		Linus
----
Adrian Bunk (12):
      RDMA/amso1100: Fix a NULL dereference in error path
      drivers/char/specialix.c: fix the baud conversion
      USB: ftdi-elan.c: remove dead code
      USB: mos7840.c: fix a check-after-dereference
      [GFS2] fs/gfs2/dir.c:gfs2_dir_write_data(): remove dead code
      [GFS2] fs/gfs2/ops_fstype.c:gfs2_get_sb_meta(): remove unused variable
      [GFS2] fs/gfs2/dir.c:gfs2_dir_write_data(): don't use an uninitialized variable
      [GFS2] fs/gfs2/ops_fstype.c:fill_super_meta(): fix NULL dereference
      [GFS2] gfs2_dir_read_data(): fix uninitialized variable usage
      one more ARM IRQ fix
      ATA must depend on BLOCK
      drivers/ide/pci/generic.c: re-add the __setup("all-generic-ide",...)

Akinobu Mita (8):
      [CRYPTO] api: fix crypto_alloc_base() return value
      md: fix /proc/mdstat refcounting
      rd: memory leak on rd_init() failure
      epca: prevent panic on tty_register_driver() failure
      usb devio: handle class_device_create() error
      cpcihp_generic: prevent loading without "bridge" parameter
      driver core: kmalloc() failure check in driver_probe_device
      ocfs2: delete redundant memcmp()

Al Viro (31):
      gfp_t in netlabel
      new cifs endianness bugs
      hp drivers/input stuff: C99 initializers, NULL noise removal, __user annotations
      sun3_ioremap() prototype
      serial167 __user annotations, NULL noise removal
      [GFS2] gfs2 endianness bug: be16 assigned to be32 field
      bug: nfsd/nfs4xdr.c misuse of ERR_PTR()
      fix svc_procfunc declaration
      lockd endianness annotations
      xdr annotations: NFSv2
      xdr annotations: NFSv3
      xdr annotations: NFSv4
      xdr annotations: NFS readdir entries
      fs/nfs/callback* passes error values big-endian
      xdr annotations: fs/nfs/callback*
      nfs: verifier is network-endian
      xdr annotations: mount_clnt
      nfs_common endianness annotations
      nfsd: nfserrno() endianness annotations
      nfsfh simple endianness annotations
      xdr annotations: nfsd_dispatch()
      xdr annotations: NFSv2 server
      xdr annotations: NFSv3 server
      xdr annotations: NFSv4 server
      nfsd: vfs.c endianness annotations
      nfsd: nfs4 code returns error values in net-endian
      nfsd: NFSv{2,3} trivial endianness annotations for error values
      nfsd: NFSv4 errno endianness annotations
      xdr annotations: nfsd callback*
      nfsd: misc endianness annotations
      nfsd: nfs_replay_me

Alan Cox (10):
      rio: fix array checking
      ide: add sanity checking to ide taskfile ioctl
      [ARM] switch to new pci_get_bus_and_slot API
      pci: Stamp out pci_find_* usage in fakephp
      PCI: quirks: switch quirks code offender to use pci_get API
      pci: Additional search functions
      irq updates: make eata_pio compile
      ahci: readability tweak
      libata-sff: Allow for wacky systems
      [ATM] nicstar: Fix a bogus casting warning

Alan Stern (6):
      USB: unusual_devs entry for Nokia 6131
      UHCI: workaround for Asus motherboard
      usbcore: fix refcount bug in endpoint removal
      usbcore: fix endpoint device creation
      USB: unusual_devs entry for Nokia 6234
      Driver core: Don't ignore error returns from probing

Alexey Dobriyan (6):
      ACPI: asus_acpi: don't printk on writing garbage to proc files
      sx: fix user-visible typo (devic)
      USB: drivers/usb/net/*: use BUILD_BUG_ON
      OOM killer meets userspace headers
      kernel/nsproxy.c: use kmemdup()
      i2o/exec-osm.c: use "unsigned long flags;"

Alexey Y. Starikovskiy (2):
      ACPI: Remove deferred execution from global lock acquire wakeup path
      ACPI: created a dedicated workqueue for notify() execution

Allan Stephens (12):
      [TIPC]: Add missing unlock in port timeout code.
      [TIPC]: Debug print buffer enhancements and fixes
      [TIPC]: Stream socket can now send > 66000 bytes at a time
      [TIPC]: Added duplicate node address detection capability
      [TIPC]: Optimize wakeup logic when socket has no waiting processes
      [TIPC]: Remove code bloat introduced by print buffer rework
      [TIPC]: Add support for Ethernet VLANs
      [TIPC]: Name publication events now delivered in chronological order
      [TIPC]: Fixed slow link reactivation when link tolerance is large
      [TIPC]: Can now list multicast link on an isolated network node
      [TIPC]: Unrecognized configuration command now returns error message
      [TIPC]: Updated TIPC version number to 1.6.2

Amit Choudhary (5):
      V4L/DVB (4738): Bt8xx/dvb-bt8xx.c: check kmalloc() return value.
      [ALSA] sound/isa/gus/interwave.c: check kmalloc() return value
      [ALSA] sound/isa/cmi8330.c: check kmalloc() return value
      [ALSA] sound/isa/ad1816a/ad1816a.c: check kmalloc() return value
      [ALSA] sound/isa/opti9xx/opti92x-ad1848.c: check kmalloc() return value

Amol Lad (5):
      [WATCHDOG] ioremap balanced with iounmap for drivers/char/watchdog/s3c2410_wdt.c
      drivers/isdn/hysdn: save_flags()/cli(), restore_flags() replaced appropriately
      drivers/isdn/isdnloop: save_flags()/cli(), restore_flags() replaced appropriately
      PCI hotplug: ioremap balanced with iounmap
      drivers/isdn: ioremap balanced with iounmap

Andi Kleen (8):
      x86-64: Update defconfig
      i386: Update defconfig
      x86: Use -maccumulate-outgoing-args
      x86-64: Revert interrupt backlink changes
      i386: Disable nmi watchdog on all ThinkPads
      x86: Revert new unwind kernel stack termination
      x86-64: Revert timer routing behaviour back to 2.6.16 state
      x86-64: Fix C3 timer test

Andrew Morton (14):
      r8169: PCI ID for Corega Gigabit network card
      Lockdep: fix compile error in drivers/input/serio/serio.c
      invalidate: remove_mapping() fix
      PROC_NUMBUF is wrong
      remove carta_random32
      vmalloc(): don't pass __GFP_ZERO to slab
      acpi_processor_latency_notifier(): UP warning fix
      swsusp: fix memory leaks
      USB: fix usbatm tiny race
      PCI: pcie-check-and-return-bus_register-errors fix
      separate bdi congestion functions from queue congestion functions
      highest_possible_node_id() linkage fix
      i386: fix .cfi_signal_frame copy-n-paste error
      pci: declare pci_get_device_reverse()

Andrew Victor (1):
      [WATCHDOG] Atmel AT91RM9200 rename.

Andrey Mirkin (1):
      scsi: megaraid_{mm,mbox}: 64-bit DMA capability fix

Andy Whitcroft (1):
      Reintroduce NODES_SPAN_OTHER_NODES for powerpc

Aneesh Kumar K.V (1):
      Add entry.S labels to tag file

Anton Blanchard (4):
      [POWERPC] Never panic when taking altivec exceptions from userspace
      [POWERPC] POWER6 has 6 PMCs
      [POWERPC] Better check in show_instructions
      [POWERPC] Check for offline nodes in pci NUMA code

Arnaud Patard (1):
      r8169: fix infinite loop during hotplug

Arnaud Patard (Rtp) (1):
      [WATCHDOG] add ich8 support to iTCO_wdt.c

Arnd Bergmann (3):
      USB: driver for mcs7830 (aka DeLOCK) USB ethernet adapter
      usbnet: improve generic ethtool support
      usbnet: add a mutex around phy register access

Aron Griffis (1):
      [IA64] move ioremap/ioremap_nocache under __KERNEL__

Arthur Kepner (1):
      IB/mthca: Use mmiowb after doorbell ring

Atsushi Nemoto (1):
      [MIPS] save_context_stack fix

Auke Kok (1):
      e100: fix reboot -f with netconsole enabled

Ben Collins (13):
      [SPARC]: Fix some section mismatch warnings in sparc drivers.
      [alim7101] Add pci dev table for auto module loading.
      [mv643xx] Add pci device table for auto module loading.
      [BusLogic] Add pci dev table for auto module loading.
      [fdomain] Add pci dev table for module auto loading.
      [initio] Add pci dev table for module auto loading.
      [ixj] Add pci dev table for module auto loading.
      [hid-core] TurboX Keyboard needs NOGET quirk.
      [controlfb] Ifdef for when CONFIG_NVRAM isn't enabled.
      [igafb] Add pci dev table for module auto loading.
      [platinumfb] Ifdef for when CONFIG_NVRAM isn't enabled.
      [valkyriefb] Ifdef for when CONFIG_NVRAM isn't enabled.
      [pci_ids] Add Quicknet XJ vendor/device ID's.

Benjamin Herrenschmidt (1):
      [POWERPC] Don't crash on cell with 2 BEs when !CONFIG_NUMA

bibo,mao (1):
      x86-64: x86_64 add NX mask for PTE entry

Bjorn Helgaas (3):
      [IA64] remove unused PAL_CALL_IC_OFF
      [IA64] reformat pal.S to fit in 80 columns, fix typos
      [IA64] remove unused acpi_kbd_controller_present, acpi_legacy_devices

Björn Steinbrink (1):
      [NETFILTER]: Missing check for CAP_NET_ADMIN in iptables compat layer

Borislav Petkov (1):
      readjust comments of task_timeslice for kernel doc

Brent Casavant (2):
      ioc4: Remove SN2 feature and config dependencies
      ioc4: Enable build on non-SN2

Brice Goglin (2):
      PCI: Improve pci_msi_supported() comments
      PCI: Update MSI-HOWTO.txt according to pci_msi_supported()

Cedric Le Goater (1):
      [S390] fix vmlinux link when CONFIG_SYSIPC=n

Chandra Seetharaman (1):
      configfs: handle kzalloc() failure in check_perm()

Chris Malley (1):
      USB: Support for BT On-Air USB modem in cdc-acm.c

Christoph Lameter (1):
      Slab: Do not fallback to nodes that have not been bootstrapped yet

Chuck Lever (5):
      NFS: fix minor bug in new NFS symlink code
      NFS: __nfs_revalidate_inode() can use "inode" before checking it is non-NULL
      NFS: remove unused check in nfs4_open_revalidate
      SUNRPC: fix race in in-kernel RPC portmapper client
      SUNRPC: fix a typo

Corey Minyard (1):
      x86-64: Fix for arch/x86_64/pci/Makefile CFLAGS

Cornelia Huck (8):
      [S390] cio: sch_no -> schid.sch_no conversion.
      [S390] cio: update documentation.
      driver core fixes: sysfs_create_link() retval check in class.c
      driver core fixes: bus_add_attrs() retval check
      driver core fixes: bus_add_device() cleanup on error
      driver core fixes: device_add() cleanup on error
      driver core fixes: device_create_file() retval check in dmapool.c
      driver core fixes: sysfs_create_group() retval in topology.c

Craig Shelley (1):
      USB-SERIAL:cp2101 Add new device ID

Daniel Drake (1):
      PCI: VIA IRQ quirk behaviour change

Daniel Ritz (2):
      usbtouchscreen: fix data reading for ITM touchscreens
      PCI: add ICH7/8 ACPI/GPIO io resource quirks

Daniel Walker (1):
      clocksource: acpi_pm: add another greylist chipset

Darren Jenkins (1):
      ACPI: asus_acpi: fix proc files parsing

Darrick J. Wong (1):
      fix "ACPI: Processor native C-states using MWAIT"

Dave Jones (2):
      ipmi: fix return codes in failure case
      Remove useless comment from sb1250

Dave Kleikamp (3):
      JFS: pageno needs to be long
      airo: check if need to freeze
      null dereference in fs/jbd2/journal.c

David Brownell (2):
      USB: ohci-pnx4008 build fixes
      USB: ftdi_sio whitespace fixes

David Gibson (2):
      ibmveth: Fix index increment calculation
      ibmveth: Fix index increment calculation

David Howells (2):
      FRV: Use the correct preemption primitives in kmap_atomic() and co
      autofs3: Make sure all dentries refs are released before calling kill_anon_super()

David M. Grimes (1):
      knfsd: add nfs-export support to tmpfs

David S. Miller (12):
      [XFRM]: Fix xfrm_state_num going negative.
      [SPARC]: Kill BOOTME_SINGLE.
      [SPARC64]: Fix PCI memory space root resource on Hummingbird.
      [SPARC] {bbc_,}envctrl: Use call_usermodehelper().
      [SPARC64]: Update defconfig.
      [TG3]: Bump driver version and release date.
      [IPV6]: Fix route.c warnings when multiple tables are disabled.
      [SPARC64]: Compute dma_end argument to sabre_pbm_init() correctly.
      [SPARC64]: Fix of_ioremap().
      [DCCP] ipv6: Fix opt_skb leak.
      [PKT_SCHED] netem: Orphan SKB when adding to queue.
      [SPARC64]: 8-byte align return value from compat_alloc_user_space()

David Woodhouse (2):
      fix `make headers_install'
      [SPARC]: Clean up asm-sparc/elf.h pollution in userspace.

Deepak Saxena (1):
      Update smc91x driver with ARM Versatile board info

Denis M. Sadykov (5):
      ACPI: EC: Remove unnecessary delay added by previous transation patch.
      ACPI: EC: Remove unused variables and duplicated code
      ACPI: EC: Unify poll and interrupt mode transaction functions
      ACPI: EC: Unify poll and interrupt gpe handlers
      ACPI: EC: Simplify acpi_hw_low_level*() with inb()/outb().

Diego Calleja (1):
      HOWTO: bug report addition

Dmitriy Monakhov (1):
      mm: D-cache aliasing issue in cow_user_page

Dmitry Torokhov (6):
      Input: add missing exports to fix modular build
      Input: i8042 - supress ACK/NAKs when blinking during panic
      Input: atkbd - supress "too many keys" error message
      Input: serio core - handle errors returned by device_bind_driver()
      Input: gameport core - handle errors returned by device_bind_driver()
      ACPI: fix potential OOPS in power driver with CONFIG_ACPI_DEBUG

Dominic Cerquetti (1):
      USB: xpad: dance pad support

Dominik Brodowski (1):
      Documentation: feature-removal-schedule typo

Doug Warzecha (1):
      firmware/dcdbas: add size check in smi_data_write

Duncan Sands (4):
      usbatm: fix tiny race
      speedtch: "extended reach"
      cxacru: add the ZTE ZXDSL 852
      Driver core: plug device probe memory leak

Ed L Cashin (14):
      aoe: eliminate isbusy message
      aoe: update copyright date
      aoe: remove unused NARGS enum
      aoe: zero copy write 1 of 2
      aoe: jumbo frame support 1 of 2
      aoe: clean up printks via macros
      aoe: jumbo frame support 2 of 2
      aoe: improve retransmission heuristics
      aoe: zero copy write 2 of 2
      aoe: module parameter for device timeout
      aoe: use bio->bi_idx
      aoe: remove sysfs comment
      aoe: update driver version
      aoe: revert printk macros

Eiichiro Oiwa (1):
      ACPICA: Fix incorrect handling of PCI Express Root Bridge _HID

eiichiro.oiwa.nm@hitachi.com (1):
      PCI: Turn pci_fixup_video into generic for embedded VGA

Enrico Scholz (1):
      V4L/DVB (4740): Fixed an if-block to avoid floating with debug-messages

Eric Dumazet (6):
      [NET]: reduce sizeof(struct inet_peer), cleanup, change in peer_check_expire()
      [NET]: reduce per cpu ram used for loopback stats
      [TCP]: One NET_INC_STATS() could be NET_INC_STATS_BH in tcp_v4_err()
      [IPV4] inet_peer: Group together avl_left, avl_right, v4daddr to speedup lookups on some CPUS
      [NET]: Can use __get_cpu_var() instead of per_cpu() in loopback driver.
      [NET]: Reduce sizeof(struct flowi) by 20 bytes.

Eric Sesterhenn (7):
      [POWERPC] Off-by-one in /arch/ppc/platforms/mpc8*
      zd1201: Possible NULL dereference
      USB: BUG_ON conversion for wacom.c
      USB: fix use after free in wacom_sys.c
      USB: fix dereference in drivers/usb/misc/adutux.c
      USB: Memory leak in drivers/usb/serial/airprime.c
      pciehp: Remove unnecessary check in pciehp_ctrl.c

Eric W. Biederman (2):
      x86-64: Use irq_domain in ioapic_retrigger_irq
      x86-64: Put more than one cpu in TARGET_CPUS

Evgeniy Polyakov (1):
      w1 kconfig fix

Felix Kuehling (1):
      [ALSA] hda_intel: add ATI RS690 HDMI audio support

Florin Malita (1):
      airo.c: check returned values

Francisco Larramendi (1):
      rtc-max6902: month conversion fix

Franck Bui-Huu (1):
      [MIPS] Use kallsyms_lookup_size_offset() instead of kallsyms_lookup()

Geert Uytterhoeven (1):
      CONFIG_TELCLOCK depends on X86

Gerrit Renker (1):
      [DCCP]: Fix Oops in DCCPv6

Grant Coady (1):
      adm9240: Update Grant Coady's email address

Grant Grundler (1):
      USB: input: extract() and implement() are bit field manipulation routines

Greg Banks (1):
      kbuild: allow multi-word $M in Makefile.modpost

Greg Kroah-Hartman (7):
      USB: revert EHCI VIA workaround patch
      USB: ftdi-elan: fix sparse warnings
      USB: move trancevibrator.c to the proper usb directory
      USB: add USB serial mos7720 driver
      USB: cleanup sierra wireless driver a bit
      PCI Hotplug: move pci_hotplug.h to include/linux/
      aoe: fix sysfs_create_file warnings

Hans Verkuil (3):
      V4L/DVB (4729): Fix VIDIOC_G_FMT for NTSC in cx25840.
      V4L/DVB (4744): The Samsung TCPN2121P30A does not have a tda9887
      V4L/DVB (4746): HM12 is YUV 4:2:0, not YUV 4:1:1

Hartmut Hackmann (1):
      V4L/DVB (4727): Support status readout for saa713x based FM radio

Heiko Carstens (1):
      [S390] Wire up epoll_pwait syscall.

Henrik Kretzschmar (1):
      RDMA/amso1100: pci_module_init() conversion

Herbert Xu (1):
      [CRYPTO] api: Select cryptomgr where needed

Hidetoshi Seto (2):
      sysfs: remove duplicated dput in sysfs_update_file
      sysfs: update obsolete comment in sysfs_update_file

Ingo Molnar (3):
      lockdep: increase max allowed recursion depth
      genirq: clean up irq-flow-type naming
      genirq: clean up irq-flow-type naming, fix

J. Bruce Fields (4):
      knfsd: nfsd4: fix owner-override on open
      knfsd: nfsd4: fix open permission checking
      knfsd: nfsd4: Fix error handling in nfsd's callback client
      nfs4: initialize cl_ipaddr

Jack Steiner (2):
      [IA64] - Allow IPIs in timer loop
      [IA64] Count resched interrupts

Jan Beulich (2):
      x86-64: Speed up dwarf2 unwinder
      x86-64: Fix ENOSYS in system call tracing

Jan Dittmer (1):
      [IPV6] sit: Add missing MODULE_LICENSE

Jan Kara (1):
      Fix IO error reporting on fsync()

Jan Luebbe (1):
      USB: Add device id for Sierra Wireless MC8755

Jan Mate (1):
      USB Storage: unusual_devs.h entry for Sony Ericsson P990i

Jarek Poplawski (1):
      USB: fix cdc-acm problems with hard irq? (inconsistent lock state)

Jaroslav Kysela (1):
      [ALSA] version 1.0.13

Jean Delvare (5):
      [WATCHDOG] includes for sample watchdog program.
      hwmon: Fix documentation typos
      smsc47m1: List the SMSC LPC47M112 as supported
      hwmon: Let w83781d and lm78 load again
      hwmon: Fix debug messages in w83781d

Jean Tourrilhes (2):
      orinoco: fix WE-21 buffer overflow
      wireless: More WE-21 potential overflows...

Jeff Dike (1):
      uml: MODE_TT is bust

Jeff Garzik (15):
      [WATCHDOG] watchdog/iTCO_wdt: fix bug related to gcc uninit warning
      Input: fm801-gp - handle errors from pci_enable_device()
      V4L/DVB (4741): {ov511,stv680}: handle sysfs errors
      V4L/DVB (4742): Drivers/media/video: handle sysfs errors
      drivers/led: handle sysfs errors
      I2O: handle a few sysfs errors
      fs/partitions/check: add sysfs error handling
      rtc: fix printk of 64-bit res on 32-bit platform
      ISDN: fix drivers, by handling errors thrown by ->readstat()
      ISDN: check for userspace copy faults
      USB/gadget/net2280: handle sysfs errors
      Driver core: bus: remove indentation level
      WAN/pc300: handle, propagate minor errors
      [ATM]: handle sysfs errors
      [ATM] firestream: handle thrown error

Jeff Moyer (1):
      direct-io: sync and invalidate file region when falling back to buffered write

Jens Axboe (2):
      Add lockless helpers for remove_suid()
      Remove SUID when splicing into an inode

Jeremy Fitzhardinge (1):
      i386: Fix fake return address

Jes Sorensen (1):
      [IA64] update sn2_defconfig

Jesper Juhl (1):
      Driver core: Don't leak 'old_class_name' in drivers/base/core.c::device_rename()

Jim Cromie (1):
      w83791d: Fix unchecked return status

Jiri Kosina (3):
      Input: serio - add lockdep annotations
      ACPI: acpi_pci_link_set() can allocate with either GFP_ATOMIC or GFP_KERNEL
      ACPI: check battery status on resume for un/plug events during sleep

Jiri Slaby (1):
      Char: correct pci_get_device changes

John Heffner (1):
      [TCP]: Bound TSO defer time

john stultz (1):
      i386 Time: Avoid PIT SMP lockups

John W. Linville (2):
      zd1211rw: fix build-break caused by association race fix
      wireless: WE-20 compatibility for ESSID and NICKN ioctls

Jonathan Corbet (1):
      V4L/DVB (4743): Fix oops in VIDIOC_G_PARM

keith mannthey (1):
      x86-64: x86_64 hot-add memory srat.c fix

Kenji Kaneshige (5):
      shpchp: fix shpchp_wait_cmd in poll
      pciehp: fix improper info messages
      pciehp - add missing locking
      shpchp: fix command completion check
      shpchp: remove unnecessary cmd_busy member from struct controller

Kevin Lloyd (1):
      USB: Sierra Wireless driver update

Kimball Murray (1):
      ACPI: SCI interrupt source override

Kristen Carlson Accardi (2):
      change pci hotplug subsystem maintainer to Kristen
      libata: use correct map_db values for ICH8

Kristoffer Ericson (2):
      [ARM] 3889/1: [Jornada7xx] Addition of correct SDRAM params into cpu-sa1110.c
      [ARM] 3890/1: [Jornada7xx] Addition of MCU commands into jornada720.h

Krzysztof Helt (1):
      [SPARC]: Sparc compilation fix with floppy enabled

Kumar Gala (1):
      [POWERPC] ppc: Add missing calls to set_irq_regs

Larry Finger (2):
      bcm43xx-softmac: check returned value from pci_enable_device
      bcm43xx-softmac: Fix system hang for x86-64 with >1GB RAM

Laurent Riffard (1):
      sotftmac: fix a slab corruption in WEP restricted key association

Lebedev, Vladimir P (2):
      ACPI: sbs: check for NULL device pointer
      ACPI: sbs: fix module_param() initializers

Len Brown (1):
      ACPI: update comments in motherboard.c

Lennart Poettering (3):
      ACPI: consolidate functions in acpi ec driver
      ACPI: EC: export ec_transaction() for msi-laptop driver
      MSI S270 Laptop support: backlight, wlan, bluetooth states

Li Yang (3):
      [POWERPC] Fix MPC8360EMDS PB board support
      [POWERPC] Add Makefile entry for MPC832x_mds support
      ucc_geth: changes to ucc_geth driver as a result of qe_lib changes and bugfixes

Liam Girdwood (1):
      [ARM] 3888/1: add pxa27x SSP FSRT register bit definition

Lijun Chen (1):
      [TIPC]: Added subscription cancellation capability

Linas Vepstas (1):
      e1000: Reset all functions after a PCI error

Linus Torvalds (5):
      Fix VM_MAYEXEC calculation
      Fix USB gadget net2280.c compile
      Revert "[mv643xx] Add pci device table for auto module loading."
      Revert unintentional and bogus change to drivers/pci/quirks.c
      Linux 2.6.19-rc3

Luiz Fernando N. Capitulino (1):
      airprime: New device ID.

Marcel Holtmann (12):
      [Bluetooth] Fix compat ioctl for BNEP, CMTP and HIDP
      [Bluetooth] Handle return values from driver core functions
      [Bluetooth] Make use of virtual devices tree
      [Bluetooth] Support concurrent connect requests
      [Bluetooth] Disconnect HID interrupt channel first
      [Bluetooth] Fix reference count when connection lookup fails
      [Bluetooth] Check if DLC is still attached to the TTY
      [Bluetooth] Add locking for bt_proto array manipulation
      [Bluetooth] Use work queue to trigger URB submission
      [Bluetooth] Add support for newer ANYCOM USB dongles
      [Bluetooth] Add missing entry for Nokia DTL-4 PCMCIA card
      [Bluetooth] Fix HID disconnect NULL pointer dereference

Marcus Junker (1):
      [WATCHDOG] w83697hf WDT driver

Marek W (1):
      ACPI: asus_acpi: W3000 support

Mark Fasheh (4):
      Take i_mutex in splice_from_pipe()
      Introduce generic_file_splice_write_nolock()
      ocfs2: fix page zeroing during simple extends
      ocfs2: cond_resched() in ocfs2_zero_extend()

Martin Habets (1):
      [SPARC]: Add sparc profiling support

Martin Schwidefsky (2):
      [S390] Fix pte type checking.
      [S390] update default configuration

Matt Domsch (1):
      PCI: optionally sort device lists breadth-first

Matthew Wilcox (3):
      V4L/DVB (4725): Fix vivi compile on parisc
      Fix dev_printk() is now GPL-only
      cciss: Fix warnings (and bug on 1TB discs)

matthieu castet (4):
      UEAGLE : be suspend friendly
      UEAGLE : use interruptible sleep
      UEAGLE : comestic changes
      UEAGLE: fix ueagle-atm Oops

Melissa Howland (1):
      [S390] monwriter find header logic.

Michael Buesch (2):
      bcm43xx: fix race condition in periodic work handler
      softmac: Fix WX and association related races

Michael Chan (1):
      [TG3]: Add lower bound checks for tx ring size.

Michael Krufky (3):
      V4L/DVB (4731a): Kconfig: restore pvrusb2 menu items
      V4L/DVB (4733): Tda10086: fix frontend selection for dvb_attach
      V4L/DVB (4734): Tda826x: fix frontend selection for dvb_attach

Michel Dänzer (1):
      [AGPGART] uninorth: Add module param 'aperture' for aperture size

Miklos Szeredi (6):
      fuse: fix hang on SMP
      document i_size_write locking rules
      fuse: locking fix for nlookup
      fuse: fix spurious BUG
      fuse: fix handling of moved directory
      fuse: fix dereferencing dentry parent

Muli Ben-Yehuda (1):
      x86-64: increase PHB1 split transaction timeout

Neil Brown (1):
      Convert cpu hotplug notifiers to use raw_notifier instead of blocking_notifier

NeilBrown (7):
      knfsd: Fix bug in recent lockd patches that can cause reclaim to fail
      knfsd: Allow lockd to drop replies as appropriate
      knfsd: fix race that can disable NFS server
      md: fix calculation of ->degraded for multipath and raid10
      md: add another COMPAT_IOCTL for md
      md: endian annotation for v1 superblock access
      md: endian annotations for the bitmap superblock

Nick Piggin (1):
      mm: more commenting on lock ordering

Nicolas Pitre (1):
      fix PXA2xx UDC compilation error

Noguchi, Masato (1):
      [POWERPC] spufs: fix support for read/write on cntl

OGAWA Hirofumi (1):
      ext3/4: fix J_ASSERT(transaction->t_updates > 0) in journal_stop()

Olaf Hering (1):
      Fix up rpaphp driver for pci hotplug header move

Oliver Neukum (3):
      USB: remove private debug macros from kaweth
      USB: suspend/resume support for kaweth
      USB: fix suspend support for usblp

Pablo Neira Ayuso (1):
      [NETFILTER]: ctnetlink: Remove debugging messages

Paolo 'Blaisorblade' Giarrusso (9):
      fix typo in memory barrier docs
      uml: remove some leftover PPC code
      uml: split memory allocation prototypes out of user.h
      uml: code convention cleanup of a file
      uml: reenable compilation of enable_timer, disabled by mistake
      uml: use DEFCONFIG_LIST to avoid reading host's config
      uml: cleanup run_helper() API to fix a leak
      uml: kconfig - silence warning
      uml: mmapper - remove just added but wrong "const" attribute

Patrick Boettcher (2):
      V4L/DVB (4748): Fixed oops for Nova-T USB2
      V4L/DVB (4750): AGC command1/2 is board specific

Patrick Caulfield (1):
      [DLM] fix iovec length in recvmsg

Patrick McHardy (6):
      [DECNET]: Use correct config option for routing by fwmark in compare_keys()
      [NETFILTER]: fix cut-and-paste error in exit functions
      [NETFILTER]: arp_tables: missing unregistration on module unload
      [NETFILTER]: ipt_ECN/ipt_TOS: fix incorrect checksum update
      [NETFILTER]: xt_CONNSECMARK: fix Kconfig dependencies
      [NETFILTER]: Update MAINTAINERS entry

Paul Fulghum (1):
      synclink: remove PAGE_SIZE reference

Paul Jackson (1):
      cpuset: mempolicy migration typo fix

Paul Moore (3):
      NetLabel: only deref the CIPSOv4 standard map fields when using standard mapping
      NetLabel: better error handling involving mls_export_cat()
      NetLabel: the CIPSOv4 passthrough mapping does not pass categories correctly

Paul Mundt (7):
      sh: Proper show_stack/show_trace() implementation.
      sh: Remove board-specific ide.h headers.
      sh: Cleanup board header directories.
      sh: Fix exception_handling_table alignment.
      sh: Add some missing board headers.
      sh: Updates for irq-flow-type naming changes.
      sh: Convert INTC2 to IRQ table registration.

Pavel Machek (1):
      ACPI: ibm_acpi: delete obsolete documentation

Pekka Enberg (1):
      ecryptfs: use special_file()

Peter Oberparleiter (1):
      [S390] cio: invalid device operational notification

Peter Zijlstra (3):
      Lockdep: add lockdep_set_class_and_subclass() and lockdep_set_subclass()
      lockdep: annotate i386 apm
      rt-mutex: fixup rt-mutex debug code

Petr Vandrovec (1):
      Fix core files so they make sense to gdb...

Pierre Ossman (2):
      ACPI: fix section for CPU init functions
      New MMC maintainer

Ping Cheng (1):
      USB: Wacom driver updates

Pádraig Brady (1):
      V4L/DVB (4739): SECAM support for saa7113 into saa7115

Ralf Baechle (11):
      [MIPS] Delete unneeded pt_regs forward declaration.
      [MIPS] Malta: Fix uninitialized regs pointer.
      [MIPS] A few more pt_regs fixups.
      [MIPS] Use compat_sys_mount.
      [MIPS] Reserve syscall numbers for kexec_load.
      [MIPS] Fix iounmap argument to const volatile.
      Make <linux/personality.h> userspace proof
      Fix warnings for WARN_ON if CONFIG_BUG is disabled
      Fix timer race
      [MIPS] Cleanup remaining references to mips_counter_frequency.
      [MIPS] Fix aliasing bug in copy_to_user_page / copy_from_user_page

Randy Dunlap (6):
      ACPI: fix printk format warnings
      fix epoll_pwait when EPOLL=n
      Kconfig serial typos
      cad_pid sysctl with PROC_FS=n
      fs/Kconfig: move GENERIC_ACL, fix acl() call errors
      [NET]: kernel-doc fix for sock.h

Randy Vinson (1):
      [POWERPC] Fix IO Window Updates on P2P bridges.

Ranjit Manomohan (1):
      [TG3]: Fix set ring params tx ring size implementation

Robert Walsh (1):
      IB/ipath: Initialize diagpkt file on device init only

Rudolf Marek (2):
      k8temp: Documentation update
      w83627ehf: Fix the detection of fan5

Russell King (4):
      [ARM] Fix fallout from IRQ regs changes
      [ARM] Fix Zaurii keyboard/touchscreen drivers
      [ARM] Update mach-types
      Remove __must_check for device_for_each_child()

Samuel Tardieu (17):
      [WATCHDOG] w83697hf/hg WDT driver - patch 1
      [WATCHDOG] w83697hf/hg WDT driver - patch 2
      [WATCHDOG] w83697hf/hg WDT driver - patch 3
      [WATCHDOG] w83697hf/hg WDT driver - patch 4
      [WATCHDOG] w83697hf/hg WDT driver - patch 5
      [WATCHDOG] w83697hf/hg WDT driver - patch 6
      [WATCHDOG] w83697hf/hg WDT driver - patch 7
      [WATCHDOG] w83697hf/hg WDT driver - patch 8
      [WATCHDOG] w83697hf/hg WDT driver - patch 9
      [WATCHDOG] w83697hf/hg WDT driver - patch 10
      [WATCHDOG] w83697hf/hg WDT driver - patch 11
      [WATCHDOG] w83697hf/hg WDT driver - patch 12
      [WATCHDOG] w83697hf/hg WDT driver - patch 13
      [WATCHDOG] w83697hf/hg WDT driver - patch 14
      [WATCHDOG] w83697hf/hg WDT driver - patch 15
      [WATCHDOG] w83697hf/hg WDT driver - patch 16
      [WATCHDOG] w83697hf/hg WDT driver - Kconfig patch

Satoru Takeuchi (1):
      doc: fixing cpu-hotplug documentation

Stefan Schmidt (3):
      ACPI: ibm_acpi: Remove experimental status for brightness and volume.
      ACPI: ibm_acpi: Update documentation for brightness and volume.
      ACPI: ibm_acpi: Documentation the wan feature.

Stefan Weinhuber (1):
      [S390] dasd: clean up timer.

Stephen Hemminger (16):
      [BRIDGE]: flush forwarding table when device carrier off
      rename net_random to random32
      sky2: MSI test is only a warning
      sky2: turn of workaround timer
      sky2: phy irq on shutdown
      sky2: fiber pause bits
      sky2: advertising register 16 bits
      sky2: use duplex result bits
      sky2: don't reset PHY twice
      sky2: flow control setting fixes
      sky2: no message on rx fifo overflow
      sky2: version 1.9
      sky2: accept multicast pause frames
      sky2: GMAC pause frame
      [NETPOLL]: initialize skb for UDP
      sky2: 88E803X transmit lockup

Steven Toth (1):
      V4L/DVB (4692): Add WinTV-HVR3000 DVB-T support

Steven Whitehouse (2):
      [DECNET]: Fix input routing bug
      [GFS2] Fix bmap to map extents properly

Sunil Mushran (1):
      ocfs2: remove spurious d_count check in ocfs2_rename()

Sven Anders (1):
      [WATCHDOG] Winbond SMsC37B787 watchdog driver

Takashi Iwai (8):
      [ALSA] hda-codec - Fix assignment of PCM devices for Realtek codecs
      [ALSA] Various fixes for suspend/resume of ALSA PCI drivers
      [ALSA] Fix dependency of snd-adlib driver in Kconfig
      [ALSA] hda-codec - Add model entry for ASUS U5F laptop
      [ALSA] Fix re-use of va_list
      [ALSA] Fix AC97 power-saving mode
      [ALSA] Fix addition of user-defined boolean controls
      [ALSA] hda-intel - Add check of MSI availabity

Tejun Heo (1):
      libata: typo fix

Thiemo Seufer (1):
      [MIPS] Fix O32 personality(2) call with 0xffffffff argument.

Thomas Gleixner (1):
      posix-cpu-timers: prevent signal delivery starvation

Thomas Graf (4):
      [IPv6] rules: Use RT6_LOOKUP_F_HAS_SADDR and fix source based selectors
      [IPv4] fib: Remove unused fib_config members
      [IPv6] route: Fix prohibit and blackhole routing decision
      [IPv6] fib: initialize tb6_lock in common place to give lockdep a key

Thomas Maier (1):
      export clear_queue_congested and set_queue_congested

Timur Tabi (1):
      [POWERPC] Add DOS partition table support to mpc834x_itx_defconfig

Tobias Klauser (1):
      [ATM]: No need to return void

Tobias Lorenz (1):
      USB: Mitsumi USB FDD 061M: UNUSUAL_DEV multilun fix

Tony Luck (1):
      [IA64] perfmon fix for global IRQ fix

Trond Myklebust (7):
      NFSv4: Fix thinko in fs/nfs/super.c
      NFS: Fix oops in nfs_cancel_commit_list
      NFS: Fix error handling in nfs_direct_write_result()
      NFS: Fix NFSv4 callback regression
      NFS: Deal with failure of invalidate_inode_pages2()
      VFS: Make d_materialise_unique() enforce directory uniqueness
      NFS: Cache invalidation fixup

Ulrich Drepper (1):
      make UML compile (FC6/x86-64)

Uwe Bugla (1):
      V4L/DVB (4732): Fix spelling error in Kconfig help text for DVB_CORE_ATTACH

Venkatesh Pallipadi (1):
      ACPI: Processor native C-states using MWAIT

Ville Nuorvala (6):
      [IPV6]: Remove struct pol_chain.
      [SCTP]: Fix minor typo
      [IPV6]: Make sure error handling is done when calling ip6_route_output().
      [IPV6]: Clean up BACKTRACK().
      [IPV6]: Make IPV6_SUBTREES depend on IPV6_MULTIPLE_TABLES.
      [IPV6]: Always copy rt->u.dst.error when copying a rt6_info.

Vivek Goyal (2):
      x86-64: fix page align in e820 allocator
      x86-64: Overlapping program headers in physical addr space fix

Vojtech Pavlik (1):
      Fix DMA resource allocation in ACPIPnP

Wim Van Sebroeck (9):
      [WATCHDOG] Winbond SMsC37B787 - remove trailing whitespace
      [WATCHDOG] Winbond SMsC37B787 watchdog fixes
      [WATCHDOG] Kconfig clean-up
      [WATCHDOG] w836?7hf_wdt spinlock fixes.
      [WATCHDOG] Kconfig clean up
      [WATCHDOG] use ENOTTY instead of ENOIOCTLCMD in ioctl()
      [WATCHDOG] w83697hf/hg WDT driver - autodetect patch
      [WATCHDOG] add ich8 support to iTCO_wdt.c (patch 2)
      [WATCHDOG] remove experimental on iTCO_wdt.c

Yasunori Goto (2):
      Change log level of a message of acpi_memhotplug to KERN_DEBUG
      acpi memory hotplug: remove strange add_memory fail message

Yinghai Lu (1):
      x86-64: typo in __assign_irq_vector when updating pos for vector and offset

Yoichi Yuasa (4):
      [MIPS] More vr41xx pt_regs fixups
      [MIPS] Update pnx8500-jbs_defconfig
      [MIPS] Update pnx8550-v2pci_defconfig
      [MIPS] Update tb0287_defconfig

YOSHIFUJI Hideaki (1):
      [IPV6]: Remove bogus WARN_ON in Proxy-NA handling.

Zachary Amsden (1):
      Fix potential interrupts during alternative patching

Zhang, Yanmin (1):
      PCI: fix pcie_portdrv_restore_config undefined without CONFIG_PM error

2.6.19-rc3: known unfixed regressions (v2)

[Adrian Bunk]

This email lists some known regressions in 2.6.19-rc3 compared to 2.6.18
that are not yet fixed Linus' tree.

If you find your name in the Cc header, you are either submitter of one
of the bugs, maintainer of an affectected subsystem or driver, a patch
of you caused a breakage or I'm considering you in any other way possibly
involved with one or more of these issues.

Due to the huge amount of recipients, please trim the Cc when answering.


Subject    : swsusp initialized after SATA (CONFIG_PCI_MULTITHREAD_PROBE)
References : http://lkml.org/lkml/2006/10/14/31
Submitter  : Pavel Machek <pavel@ucw.cz>
Status     : unknown


Subject    : MSI errors during boot (CONFIG_PCI_MULTITHREAD_PROBE)
References : http://lkml.org/lkml/2006/10/16/291
Submitter  : Stephen Hemminger <shemminger@osdl.org>
Handled-By : Greg KH <greg@kroah.com>
Status     : Greg is working on a fix


Subject    : arm: Oops in __wake_up_common during htc magician resume
References : http://lkml.org/lkml/2006/10/25/73
Submitter  : Philipp Zabel <philipp.zabel@gmail.com>
Status     : unknown


Subject    : X60s: BUG()s, lose ACPI events after suspend/resume
References : http://lkml.org/lkml/2006/10/10/39
Submitter  : Martin Lorenz <martin@lorenz.eu.org>
Status     : unknown


Subject    : T60 stops triggering any ACPI events
References : http://lkml.org/lkml/2006/10/4/425
             http://lkml.org/lkml/2006/10/16/262
             http://bugzilla.kernel.org/show_bug.cgi?id=7408
Submitter  : "Michael S. Tsirkin" <mst@mellanox.co.il>
Status     : unknown


Subject    : sata-via doesn't detect anymore disks attached to VIA vt6421
References : http://bugzilla.kernel.org/show_bug.cgi?id=7255
Submitter  : Thierry Vignaud <tvignaud@mandriva.com>
Status     : unknown


Subject    : unable to rip cd
References : http://lkml.org/lkml/2006/10/13/100
Submitter  : Alex Romosan <romosan@sycorax.lbl.gov>
Status     : unknown


Subject    : SMP kernel can not generate ISA irq properly
References : http://lkml.org/lkml/2006/10/22/15
Submitter  : Komuro <komurojun-mbn@nifty.com>
Handled-By : Thomas Gleixner <tglx@linutronix.de>
Status     : Thomas will investigate


Subject    : ohci1394 on PPC_PMAC:
             pci_set_power_state() failure and breaking suspend
References : http://lkml.org/lkml/2006/10/24/13
Submitter  : Benjamin Herrenschmidt <benh@kernel.crashing.org>
Caused-By  : Stefan Richter <stefanr@s5r6.in-berlin.de>
             commit ea6104c22468239083857fa07425c312b1ecb424
Handled-By : Stefan Richter <stefanr@s5r6.in-berlin.de>
Status     : Stefan Richter: looking for an answer when to ignore
             the return code of pci_set_power_state


Subject    : cpufreq not working on AMD K8
References : http://lkml.org/lkml/2006/10/10/114
Submitter  : Christian <christiand59@web.de>
Handled-By : Mark Langsdorf <mark.langsdorf@amd.com>
Status     : Mark is investigating

[RFC: 2.6.19 patch] let PCI_MULTITHREAD_PROBE depend on BROKEN

[Adrian Bunk]

On Fri, Oct 27, 2006 at 12:45:41AM +0200, Adrian Bunk wrote:
>...
> Subject    : swsusp initialized after SATA (CONFIG_PCI_MULTITHREAD_PROBE)
> References : http://lkml.org/lkml/2006/10/14/31
> Submitter  : Pavel Machek <pavel@ucw.cz>
> Status     : unknown
> 
> 
> Subject    : MSI errors during boot (CONFIG_PCI_MULTITHREAD_PROBE)
> References : http://lkml.org/lkml/2006/10/16/291
> Submitter  : Stephen Hemminger <shemminger@osdl.org>
> Handled-By : Greg KH <greg@kroah.com>
> Status     : Greg is working on a fix
>...


PCI_MULTITHREAD_PROBE is an interesting feature, but in it's current 
state it seems to be more of a trap for users who accidentally
enable it.

This patch lets PCI_MULTITHREAD_PROBE depend on BROKEN for 2.6.19.

The intention is to get this patch reversed in -mm as soon as it's in 
Linus' tree, and reverse it for 2.6.20 or 2.6.21 after the fallout of 
in-kernel problems PCI_MULTITHREAD_PROBE causes got fixed.

Signed-off-by: Adrian Bunk <bunk@stusta.de>

--- linux-2.6/drivers/pci/Kconfig.old	2006-10-27 02:40:02.000000000 +0200
+++ linux-2.6/drivers/pci/Kconfig	2006-10-27 02:58:25.000000000 +0200
@@ -19,7 +19,7 @@
 
 config PCI_MULTITHREAD_PROBE
 	bool "PCI Multi-threaded probe (EXPERIMENTAL)"
-	depends on PCI && EXPERIMENTAL
+	depends on PCI && EXPERIMENTAL && BROKEN
 	help
 	  Say Y here if you want the PCI core to spawn a new thread for
 	  every PCI device that is probed.  This can cause a huge

Re: [RFC: 2.6.19 patch] let PCI_MULTITHREAD_PROBE depend on BROKEN

[Matthew Wilcox]

On Fri, Oct 27, 2006 at 03:02:52AM +0200, Adrian Bunk wrote:
> PCI_MULTITHREAD_PROBE is an interesting feature, but in it's current 
> state it seems to be more of a trap for users who accidentally
> enable it.
> 
> This patch lets PCI_MULTITHREAD_PROBE depend on BROKEN for 2.6.19.
> 
> The intention is to get this patch reversed in -mm as soon as it's in 
> Linus' tree, and reverse it for 2.6.20 or 2.6.21 after the fallout of 
> in-kernel problems PCI_MULTITHREAD_PROBE causes got fixed.

People who enable features clearly marked as EXPERIMENTAL deserve what
they get, IMO.

Re: [RFC: 2.6.19 patch] let PCI_MULTITHREAD_PROBE depend on BROKEN

[Andrew Morton]

On Thu, 26 Oct 2006 19:20:58 -0600
Matthew Wilcox <matthew@wil.cx> wrote:

> On Fri, Oct 27, 2006 at 03:02:52AM +0200, Adrian Bunk wrote:
> > PCI_MULTITHREAD_PROBE is an interesting feature, but in it's current 
> > state it seems to be more of a trap for users who accidentally
> > enable it.
> > 
> > This patch lets PCI_MULTITHREAD_PROBE depend on BROKEN for 2.6.19.
> > 
> > The intention is to get this patch reversed in -mm as soon as it's in 
> > Linus' tree, and reverse it for 2.6.20 or 2.6.21 after the fallout of 
> > in-kernel problems PCI_MULTITHREAD_PROBE causes got fixed.
> 
> People who enable features clearly marked as EXPERIMENTAL deserve what
> they get, IMO.

It's not the impact on "people" which is of concern - it's the impact on
kernel developers - specifically those who spend time looking at bug
reports :(

Re: [RFC: 2.6.19 patch] let PCI_MULTITHREAD_PROBE depend on BROKEN

[Stephen Hemminger]

On Thu, 26 Oct 2006 18:28:38 -0700
Andrew Morton <akpm@osdl.org> wrote:

> On Thu, 26 Oct 2006 19:20:58 -0600
> Matthew Wilcox <matthew@wil.cx> wrote:
> 
> > On Fri, Oct 27, 2006 at 03:02:52AM +0200, Adrian Bunk wrote:
> > > PCI_MULTITHREAD_PROBE is an interesting feature, but in it's current 
> > > state it seems to be more of a trap for users who accidentally
> > > enable it.
> > > 
> > > This patch lets PCI_MULTITHREAD_PROBE depend on BROKEN for 2.6.19.
> > > 
> > > The intention is to get this patch reversed in -mm as soon as it's in 
> > > Linus' tree, and reverse it for 2.6.20 or 2.6.21 after the fallout of 
> > > in-kernel problems PCI_MULTITHREAD_PROBE causes got fixed.
> > 
> > People who enable features clearly marked as EXPERIMENTAL deserve what
> > they get, IMO.
> 
> It's not the impact on "people" which is of concern - it's the impact on
> kernel developers - specifically those who spend time looking at bug
> reports :(

Either it is broken and should be removed, or is barely working and
should be fixed. If Greg wants to take bug reports then it can stay.

Re: [RFC: 2.6.19 patch] let PCI_MULTITHREAD_PROBE depend on BROKEN

[Greg KH]

On Thu, Oct 26, 2006 at 07:11:31PM -0700, Stephen Hemminger wrote:
> On Thu, 26 Oct 2006 18:28:38 -0700
> Andrew Morton <akpm@osdl.org> wrote:
> 
> > On Thu, 26 Oct 2006 19:20:58 -0600
> > Matthew Wilcox <matthew@wil.cx> wrote:
> > 
> > > On Fri, Oct 27, 2006 at 03:02:52AM +0200, Adrian Bunk wrote:
> > > > PCI_MULTITHREAD_PROBE is an interesting feature, but in it's current 
> > > > state it seems to be more of a trap for users who accidentally
> > > > enable it.
> > > > 
> > > > This patch lets PCI_MULTITHREAD_PROBE depend on BROKEN for 2.6.19.
> > > > 
> > > > The intention is to get this patch reversed in -mm as soon as it's in 
> > > > Linus' tree, and reverse it for 2.6.20 or 2.6.21 after the fallout of 
> > > > in-kernel problems PCI_MULTITHREAD_PROBE causes got fixed.
> > > 
> > > People who enable features clearly marked as EXPERIMENTAL deserve what
> > > they get, IMO.
> > 
> > It's not the impact on "people" which is of concern - it's the impact on
> > kernel developers - specifically those who spend time looking at bug
> > reports :(
> 
> Either it is broken and should be removed, or is barely working and
> should be fixed. If Greg wants to take bug reports then it can stay.

I want to keep taking bug reports.

I've found only 1 real bug from all of this.  The pci MSI initialization
issue.  It's on my queue of things to fix.  Andrew has also sent me
another "interesting" patch about making sure devices are found by the
time we hit another init level which I'll see about adding too.

But that MSI bug was a real bug, which is good to have found.  It's also
found other real bugs in other subsystems that could easily be hit by
other users.

So no, this should not be marked BROKEN.

It's a very experimental feature, as the help text says.  If you can
think of any harsher language to put in that text, please let me know.

And yes, there also has been a proposed change to the driver core to fix
up how the multi-thread stuff works that looks very good, but it's down
in my queue that I'm trying to catch up on right now.

So consider this a NAK for this change.

thanks,

greg k-h

Re: [RFC: 2.6.19 patch] let PCI_MULTITHREAD_PROBE depend on BROKEN

[Pavel Machek]

Hi!

> > > > > PCI_MULTITHREAD_PROBE is an interesting feature, but in it's current 
> > > > > state it seems to be more of a trap for users who accidentally
> > > > > enable it.
> > > > > 
> > > > > This patch lets PCI_MULTITHREAD_PROBE depend on BROKEN for 2.6.19.
> > > > > 
> > > > > The intention is to get this patch reversed in -mm as soon as it's in 
> > > > > Linus' tree, and reverse it for 2.6.20 or 2.6.21 after the fallout of 
> > > > > in-kernel problems PCI_MULTITHREAD_PROBE causes got fixed.
> > > > 
> > > > People who enable features clearly marked as EXPERIMENTAL deserve what
> > > > they get, IMO.
> > > 
> > > It's not the impact on "people" which is of concern - it's the impact on
> > > kernel developers - specifically those who spend time looking at bug
> > > reports :(
> > 
> > Either it is broken and should be removed, or is barely working and
> > should be fixed. If Greg wants to take bug reports then it can stay.
> 
> I want to keep taking bug reports.
> 
> I've found only 1 real bug from all of this.  The pci MSI initialization
> issue.  It's on my queue of things to fix.  Andrew has also sent me
> another "interesting" patch about making sure devices are found by the
> time we hit another init level which I'll see about adding too.

And the swsusp vs. SATA issue? Currently, SATA can be initialized
after swsusp, leading to swsusp not finding its image and failing...

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [RFC: 2.6.19 patch] let PCI_MULTITHREAD_PROBE depend on BROKEN

[Andrew Morton]

On Fri, 27 Oct 2006 19:22:19 +0200
Pavel Machek <pavel@ucw.cz> wrote:

> > I've found only 1 real bug from all of this.  The pci MSI initialization
> > issue.  It's on my queue of things to fix.  Andrew has also sent me
> > another "interesting" patch about making sure devices are found by the
> > time we hit another init level which I'll see about adding too.
> 
> And the swsusp vs. SATA issue? Currently, SATA can be initialized
> after swsusp, leading to swsusp not finding its image and failing...

How can sata be initialised after swsusp?  Are they each using correctly
prioritised initcall levels?

If so then the problem is presumably that sata initialisation is started
before swsusp initialisation, but sata completes _after_ swsusp
initialisation has run.  In which case the as-yet-untested
drivers-wait-for-threaded-probes-between-initcall-levels.patch should fix
this.

Greg, I think I'll send vmlinuxlds-consolidate-initcall-sections.patch into
Linus for 2.6.19, make things easier for everyone.

I'll send both patches.  Please test ;)

vmlinux.lds: consolidate initcall sections

[Andrew Morton]

From: Andrew Morton <akpm@osdl.org>

Add a vmlinux.lds.h helper macro for defining the eight-level initcall table,
teach all the architectures to use it.

This is a prerequisite for a patch which performs initcall synchronisation for
multithreaded-probing.

Cc: Greg KH <greg@kroah.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
---

 arch/alpha/kernel/vmlinux.lds.S     |    8 +-------
 arch/arm/kernel/vmlinux.lds.S       |    8 +-------
 arch/frv/kernel/vmlinux.lds.S       |    8 +-------
 arch/h8300/kernel/vmlinux.lds.S     |    8 +-------
 arch/i386/kernel/vmlinux.lds.S      |    8 +-------
 arch/ia64/kernel/vmlinux.lds.S      |    8 +-------
 arch/m32r/kernel/vmlinux.lds.S      |    8 +-------
 arch/m68knommu/kernel/vmlinux.lds.S |    8 +-------
 arch/mips/kernel/vmlinux.lds.S      |    8 +-------
 arch/parisc/kernel/vmlinux.lds.S    |    8 +-------
 arch/powerpc/kernel/vmlinux.lds.S   |    8 +-------
 arch/ppc/kernel/vmlinux.lds.S       |    8 +-------
 arch/s390/kernel/vmlinux.lds.S      |    8 +-------
 arch/sh/kernel/vmlinux.lds.S        |    8 +-------
 arch/sh64/kernel/vmlinux.lds.S      |    8 +-------
 arch/sparc/kernel/vmlinux.lds.S     |    8 +-------
 arch/sparc64/kernel/vmlinux.lds.S   |    8 +-------
 arch/v850/kernel/vmlinux.lds.S      |    8 +-------
 arch/x86_64/kernel/vmlinux.lds.S    |    8 +-------
 arch/xtensa/kernel/vmlinux.lds.S    |    8 +-------
 include/asm-generic/vmlinux.lds.h   |   10 ++++++++++
 21 files changed, 30 insertions(+), 140 deletions(-)

diff -puN arch/alpha/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/alpha/kernel/vmlinux.lds.S
--- a/arch/alpha/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/alpha/kernel/vmlinux.lds.S
@@ -48,13 +48,7 @@ SECTIONS
   . = ALIGN(8);
   __initcall_start = .;
   .initcall.init : {
-	*(.initcall1.init) 
-	*(.initcall2.init) 
-	*(.initcall3.init) 
-	*(.initcall4.init) 
-	*(.initcall5.init) 
-	*(.initcall6.init) 
-	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
 
diff -puN arch/arm/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/arm/kernel/vmlinux.lds.S
--- a/arch/arm/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/arm/kernel/vmlinux.lds.S
@@ -45,13 +45,7 @@ SECTIONS
 			*(.early_param.init)
 		__early_end = .;
 		__initcall_start = .;
-			*(.initcall1.init)
-			*(.initcall2.init)
-			*(.initcall3.init)
-			*(.initcall4.init)
-			*(.initcall5.init)
-			*(.initcall6.init)
-			*(.initcall7.init)
+			INITCALLS
 		__initcall_end = .;
 		__con_initcall_start = .;
 			*(.con_initcall.init)
diff -puN arch/arm26/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/arm26/kernel/vmlinux.lds.S
diff -puN arch/frv/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/frv/kernel/vmlinux.lds.S
--- a/arch/frv/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/frv/kernel/vmlinux.lds.S
@@ -44,13 +44,7 @@ SECTIONS
 
   __initcall_start = .;
   .initcall.init : {
-	*(.initcall1.init)
-	*(.initcall2.init)
-	*(.initcall3.init)
-	*(.initcall4.init)
-	*(.initcall5.init)
-	*(.initcall6.init)
-	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
   __con_initcall_start = .;
diff -puN arch/h8300/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/h8300/kernel/vmlinux.lds.S
--- a/arch/h8300/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/h8300/kernel/vmlinux.lds.S
@@ -118,13 +118,7 @@ SECTIONS
 	. = ALIGN(0x4) ;
 	___setup_end = .;
 	___initcall_start = .;
-		*(.initcall1.init)
-		*(.initcall2.init)
-		*(.initcall3.init)
-		*(.initcall4.init)
-		*(.initcall5.init)
-		*(.initcall6.init)
-		*(.initcall7.init)
+		INITCALLS
 	___initcall_end = .;
 	___con_initcall_start = .;
 		*(.con_initcall.init)
diff -puN arch/i386/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/i386/kernel/vmlinux.lds.S
--- a/arch/i386/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/i386/kernel/vmlinux.lds.S
@@ -126,13 +126,7 @@ SECTIONS
   __setup_end = .;
   __initcall_start = .;
   .initcall.init : AT(ADDR(.initcall.init) - LOAD_OFFSET) {
-	*(.initcall1.init) 
-	*(.initcall2.init) 
-	*(.initcall3.init) 
-	*(.initcall4.init) 
-	*(.initcall5.init) 
-	*(.initcall6.init) 
-	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
   __con_initcall_start = .;
diff -puN arch/ia64/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/ia64/kernel/vmlinux.lds.S
--- a/arch/ia64/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/ia64/kernel/vmlinux.lds.S
@@ -128,13 +128,7 @@ SECTIONS
   .initcall.init : AT(ADDR(.initcall.init) - LOAD_OFFSET)
 	{
 	  __initcall_start = .;
-	  *(.initcall1.init)
-	  *(.initcall2.init)
-	  *(.initcall3.init)
-	  *(.initcall4.init)
-	  *(.initcall5.init)
-	  *(.initcall6.init)
-	  *(.initcall7.init)
+	INITCALLS
 	  __initcall_end = .;
 	}
 
diff -puN arch/m32r/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/m32r/kernel/vmlinux.lds.S
--- a/arch/m32r/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/m32r/kernel/vmlinux.lds.S
@@ -83,13 +83,7 @@ SECTIONS
   __setup_end = .;
   __initcall_start = .;
   .initcall.init : {
-	*(.initcall1.init)
-	*(.initcall2.init)
-	*(.initcall3.init)
-	*(.initcall4.init)
-	*(.initcall5.init)
-	*(.initcall6.init)
-	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
   __con_initcall_start = .;
diff -puN arch/m68k/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/m68k/kernel/vmlinux.lds.S
diff -puN arch/m68knommu/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/m68knommu/kernel/vmlinux.lds.S
--- a/arch/m68knommu/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/m68knommu/kernel/vmlinux.lds.S
@@ -140,13 +140,7 @@ SECTIONS {
 		*(.init.setup)
 		__setup_end = .;
 		__initcall_start = .;
-		*(.initcall1.init)
-		*(.initcall2.init)
-		*(.initcall3.init)
-		*(.initcall4.init)
-		*(.initcall5.init)
-		*(.initcall6.init)
-		*(.initcall7.init)
+		INITCALLS
 		__initcall_end = .;
 		__con_initcall_start = .;
 		*(.con_initcall.init)
diff -puN arch/mips/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/mips/kernel/vmlinux.lds.S
--- a/arch/mips/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/mips/kernel/vmlinux.lds.S
@@ -91,13 +91,7 @@ SECTIONS
 
   __initcall_start = .;
   .initcall.init : {
-	*(.initcall1.init)
-	*(.initcall2.init)
-	*(.initcall3.init)
-	*(.initcall4.init)
-	*(.initcall5.init)
-	*(.initcall6.init)
-	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
 
diff -puN arch/parisc/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/parisc/kernel/vmlinux.lds.S
--- a/arch/parisc/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/parisc/kernel/vmlinux.lds.S
@@ -153,13 +153,7 @@ SECTIONS
   __setup_end = .;
   __initcall_start = .;
   .initcall.init : {
-	*(.initcall1.init) 
-	*(.initcall2.init) 
-	*(.initcall3.init) 
-	*(.initcall4.init) 
-	*(.initcall5.init) 
-	*(.initcall6.init) 
-	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
   __con_initcall_start = .;
diff -puN arch/powerpc/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/powerpc/kernel/vmlinux.lds.S
--- a/arch/powerpc/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/powerpc/kernel/vmlinux.lds.S
@@ -108,13 +108,7 @@ SECTIONS
 
 	.initcall.init : {
 		__initcall_start = .;
-		*(.initcall1.init)
-		*(.initcall2.init)
-		*(.initcall3.init)
-		*(.initcall4.init)
-		*(.initcall5.init)
-		*(.initcall6.init)
-		*(.initcall7.init)
+		INITCALLS
 		__initcall_end = .;
 		}
 
diff -puN arch/ppc/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/ppc/kernel/vmlinux.lds.S
--- a/arch/ppc/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/ppc/kernel/vmlinux.lds.S
@@ -115,13 +115,7 @@ SECTIONS
   __setup_end = .;
   __initcall_start = .;
   .initcall.init : {
-	*(.initcall1.init)
-	*(.initcall2.init)
-	*(.initcall3.init)
-	*(.initcall4.init)
-	*(.initcall5.init)
-	*(.initcall6.init)
-	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
 
diff -puN arch/s390/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/s390/kernel/vmlinux.lds.S
--- a/arch/s390/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/s390/kernel/vmlinux.lds.S
@@ -83,13 +83,7 @@ SECTIONS
   __setup_end = .;
   __initcall_start = .;
   .initcall.init : {
-	*(.initcall1.init) 
-	*(.initcall2.init) 
-	*(.initcall3.init) 
-	*(.initcall4.init) 
-	*(.initcall5.init) 
-	*(.initcall6.init) 
-	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
   __con_initcall_start = .;
diff -puN arch/sh/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/sh/kernel/vmlinux.lds.S
--- a/arch/sh/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/sh/kernel/vmlinux.lds.S
@@ -76,13 +76,7 @@ SECTIONS
   __setup_end = .;
   __initcall_start = .;
   .initcall.init : {
-	*(.initcall1.init) 
-	*(.initcall2.init) 
-	*(.initcall3.init) 
-	*(.initcall4.init) 
-	*(.initcall5.init) 
-	*(.initcall6.init) 
-	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
   __con_initcall_start = .;
diff -puN arch/sh64/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/sh64/kernel/vmlinux.lds.S
--- a/arch/sh64/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/sh64/kernel/vmlinux.lds.S
@@ -108,13 +108,7 @@ SECTIONS
   __setup_end = .;
   __initcall_start = .;
   .initcall.init : C_PHYS(.initcall.init) {
-  	*(.initcall1.init)
-  	*(.initcall2.init)
-  	*(.initcall3.init)
-  	*(.initcall4.init)
-  	*(.initcall5.init)
-  	*(.initcall6.init)
-  	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
   __con_initcall_start = .;
diff -puN arch/sparc/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/sparc/kernel/vmlinux.lds.S
--- a/arch/sparc/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/sparc/kernel/vmlinux.lds.S
@@ -49,13 +49,7 @@ SECTIONS
   __setup_end = .;
   __initcall_start = .;
   .initcall.init : {
-	*(.initcall1.init) 
-	*(.initcall2.init) 
-	*(.initcall3.init) 
-	*(.initcall4.init) 
-	*(.initcall5.init) 
-	*(.initcall6.init) 
-	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
   __con_initcall_start = .;
diff -puN arch/sparc64/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/sparc64/kernel/vmlinux.lds.S
--- a/arch/sparc64/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/sparc64/kernel/vmlinux.lds.S
@@ -57,13 +57,7 @@ SECTIONS
   __setup_end = .;
   __initcall_start = .;
   .initcall.init : {
-	*(.initcall1.init) 
-	*(.initcall2.init) 
-	*(.initcall3.init) 
-	*(.initcall4.init) 
-	*(.initcall5.init) 
-	*(.initcall6.init) 
-	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
   __con_initcall_start = .;
diff -puN arch/um/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/um/kernel/vmlinux.lds.S
diff -puN arch/v850/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/v850/kernel/vmlinux.lds.S
--- a/arch/v850/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/v850/kernel/vmlinux.lds.S
@@ -140,13 +140,7 @@
 		___setup_end = . ;					      \
 		___initcall_start = . ;					      \
 			*(.initcall.init)				      \
-			*(.initcall1.init)				      \
-			*(.initcall2.init)				      \
-			*(.initcall3.init)				      \
-			*(.initcall4.init)				      \
-			*(.initcall5.init)				      \
-			*(.initcall6.init)				      \
-			*(.initcall7.init)				      \
+			INITCALLS					      \
 		. = ALIGN (4) ;						      \
 		___initcall_end = . ;					      \
 		___con_initcall_start = .;				      \
diff -puN arch/x86_64/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/x86_64/kernel/vmlinux.lds.S
--- a/arch/x86_64/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/x86_64/kernel/vmlinux.lds.S
@@ -175,13 +175,7 @@ SECTIONS
   __setup_end = .;
   __initcall_start = .;
   .initcall.init : AT(ADDR(.initcall.init) - LOAD_OFFSET) {
-	*(.initcall1.init) 
-	*(.initcall2.init) 
-	*(.initcall3.init) 
-	*(.initcall4.init) 
-	*(.initcall5.init) 
-	*(.initcall6.init) 
-	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
   __con_initcall_start = .;
diff -puN arch/xtensa/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections arch/xtensa/kernel/vmlinux.lds.S
--- a/arch/xtensa/kernel/vmlinux.lds.S~vmlinuxlds-consolidate-initcall-sections
+++ a/arch/xtensa/kernel/vmlinux.lds.S
@@ -184,13 +184,7 @@ SECTIONS
 
   __initcall_start = .;
   .initcall.init : {
-  	*(.initcall1.init)
-  	*(.initcall2.init)
-  	*(.initcall3.init)
-  	*(.initcall4.init)
-  	*(.initcall5.init)
-  	*(.initcall6.init)
-  	*(.initcall7.init)
+	INITCALLS
   }
   __initcall_end = .;
 
diff -puN include/asm-generic/vmlinux.lds.h~vmlinuxlds-consolidate-initcall-sections include/asm-generic/vmlinux.lds.h
--- a/include/asm-generic/vmlinux.lds.h~vmlinuxlds-consolidate-initcall-sections
+++ a/include/asm-generic/vmlinux.lds.h
@@ -213,3 +213,13 @@
 
 #define NOTES								\
 		.notes : { *(.note.*) } :note
+
+#define INITCALLS							\
+  	*(.initcall1.init)						\
+  	*(.initcall2.init)						\
+  	*(.initcall3.init)						\
+  	*(.initcall4.init)						\
+  	*(.initcall5.init)						\
+  	*(.initcall6.init)						\
+  	*(.initcall7.init)
+
_

[patch] drivers: wait for threaded probes between initcall levels

[Andrew Morton]

From: Andrew Morton <akpm@osdl.org>

The multithreaded-probing code has a problem: after one initcall level (eg,
core_initcall) has been processed, we will then start processing the next
level (postcore_initcall) while the kernel threads which are handling
core_initcall are still executing.  This breaks the guarantees which the
layered initcalls previously gave us.

IOW, we want to be multithreaded _within_ an initcall level, but not between
different levels.

Fix that up by causing the probing code to wait for all outstanding probes at
one level to complete before we start processing the next level.

Cc: Greg KH <greg@kroah.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
---

 drivers/base/dd.c                 |   30 ++++++++++++++++++++++++++++
 include/asm-generic/vmlinux.lds.h |    9 +++++++-
 include/linux/init.h              |   28 +++++++++++++++++---------
 3 files changed, 57 insertions(+), 10 deletions(-)

diff -puN drivers/base/dd.c~drivers-wait-for-threaded-probes-between-initcall-levels drivers/base/dd.c
--- a/drivers/base/dd.c~drivers-wait-for-threaded-probes-between-initcall-levels
+++ a/drivers/base/dd.c
@@ -18,6 +18,7 @@
 #include <linux/device.h>
 #include <linux/module.h>
 #include <linux/kthread.h>
+#include <linux/wait.h>
 
 #include "base.h"
 #include "power/power.h"
@@ -70,6 +71,8 @@ struct stupid_thread_structure {
 };
 
 static atomic_t probe_count = ATOMIC_INIT(0);
+static DECLARE_WAIT_QUEUE_HEAD(probe_waitqueue);
+
 static int really_probe(void *void_data)
 {
 	struct stupid_thread_structure *data = void_data;
@@ -121,6 +124,7 @@ probe_failed:
 done:
 	kfree(data);
 	atomic_dec(&probe_count);
+	wake_up(&probe_waitqueue);
 	return ret;
 }
 
@@ -337,6 +341,32 @@ void driver_detach(struct device_driver 
 	}
 }
 
+#ifdef CONFIG_PCI_MULTITHREAD_PROBE
+static int __init wait_for_probes(void)
+{
+	DEFINE_WAIT(wait);
+
+	printk(KERN_INFO "%s: waiting for %d threads\n", __FUNCTION__,
+			atomic_read(&probe_count));
+	if (!atomic_read(&probe_count))
+		return 0;
+	while (atomic_read(&probe_count)) {
+		prepare_to_wait(&probe_waitqueue, &wait, TASK_UNINTERRUPTIBLE);
+		if (atomic_read(&probe_count))
+			schedule();
+	}
+	finish_wait(&probe_waitqueue, &wait);
+	return 0;
+}
+
+core_initcall_sync(wait_for_probes);
+postcore_initcall_sync(wait_for_probes);
+arch_initcall_sync(wait_for_probes);
+subsys_initcall_sync(wait_for_probes);
+fs_initcall_sync(wait_for_probes);
+device_initcall_sync(wait_for_probes);
+late_initcall_sync(wait_for_probes);
+#endif
 
 EXPORT_SYMBOL_GPL(device_bind_driver);
 EXPORT_SYMBOL_GPL(device_release_driver);
diff -puN include/asm-generic/vmlinux.lds.h~drivers-wait-for-threaded-probes-between-initcall-levels include/asm-generic/vmlinux.lds.h
--- a/include/asm-generic/vmlinux.lds.h~drivers-wait-for-threaded-probes-between-initcall-levels
+++ a/include/asm-generic/vmlinux.lds.h
@@ -216,10 +216,17 @@
 
 #define INITCALLS							\
   	*(.initcall1.init)						\
+  	*(.initcall1s.init)						\
   	*(.initcall2.init)						\
+  	*(.initcall2s.init)						\
   	*(.initcall3.init)						\
+  	*(.initcall3s.init)						\
   	*(.initcall4.init)						\
+  	*(.initcall4s.init)						\
   	*(.initcall5.init)						\
+  	*(.initcall5s.init)						\
   	*(.initcall6.init)						\
-  	*(.initcall7.init)
+  	*(.initcall6s.init)						\
+  	*(.initcall7.init)						\
+  	*(.initcall7s.init)
 
diff -puN include/linux/init.h~drivers-wait-for-threaded-probes-between-initcall-levels include/linux/init.h
--- a/include/linux/init.h~drivers-wait-for-threaded-probes-between-initcall-levels
+++ a/include/linux/init.h
@@ -84,19 +84,29 @@ extern void setup_arch(char **);
  * by link order. 
  * For backwards compatibility, initcall() puts the call in 
  * the device init subsection.
+ *
+ * The `id' arg to __define_initcall() is needed so that multiple initcalls
+ * can point at the same handler without causing duplicate-symbol build errors.
  */
 
-#define __define_initcall(level,fn) \
-	static initcall_t __initcall_##fn __attribute_used__ \
+#define __define_initcall(level,fn,id) \
+	static initcall_t __initcall_##fn##id __attribute_used__ \
 	__attribute__((__section__(".initcall" level ".init"))) = fn
 
-#define core_initcall(fn)		__define_initcall("1",fn)
-#define postcore_initcall(fn)		__define_initcall("2",fn)
-#define arch_initcall(fn)		__define_initcall("3",fn)
-#define subsys_initcall(fn)		__define_initcall("4",fn)
-#define fs_initcall(fn)			__define_initcall("5",fn)
-#define device_initcall(fn)		__define_initcall("6",fn)
-#define late_initcall(fn)		__define_initcall("7",fn)
+#define core_initcall(fn)		__define_initcall("1",fn,1)
+#define core_initcall_sync(fn)		__define_initcall("1s",fn,1s)
+#define postcore_initcall(fn)		__define_initcall("2",fn,2)
+#define postcore_initcall_sync(fn)	__define_initcall("2s",fn,2s)
+#define arch_initcall(fn)		__define_initcall("3",fn,3)
+#define arch_initcall_sync(fn)		__define_initcall("3s",fn,3s)
+#define subsys_initcall(fn)		__define_initcall("4",fn,4)
+#define subsys_initcall_sync(fn)	__define_initcall("4s",fn,4s)
+#define fs_initcall(fn)			__define_initcall("5",fn,5)
+#define fs_initcall_sync(fn)		__define_initcall("5s",fn,5s)
+#define device_initcall(fn)		__define_initcall("6",fn,6)
+#define device_initcall_sync(fn)	__define_initcall("6s",fn,6s)
+#define late_initcall(fn)		__define_initcall("7",fn,7)
+#define late_initcall_sync(fn)		__define_initcall("7s",fn,7s)
 
 #define __initcall(fn) device_initcall(fn)
 
_

Re: [patch] drivers: wait for threaded probes between initcall levels

[Stephen Hemminger]

On Fri, 27 Oct 2006 11:42:37 -0700
Andrew Morton <akpm@osdl.org> wrote:

> From: Andrew Morton <akpm@osdl.org>
> 
> The multithreaded-probing code has a problem: after one initcall level (eg,
> core_initcall) has been processed, we will then start processing the next
> level (postcore_initcall) while the kernel threads which are handling
> core_initcall are still executing.  This breaks the guarantees which the
> layered initcalls previously gave us.
> 
> IOW, we want to be multithreaded _within_ an initcall level, but not between
> different levels.
> 
> Fix that up by causing the probing code to wait for all outstanding probes at
> one level to complete before we start processing the next level.
> 
> Cc: Greg KH <greg@kroah.com>
> Signed-off-by: Andrew Morton <akpm@osdl.org>
> ---
> 

This looks like a good place to use a counting semaphore.

-- 
Stephen Hemminger <shemminger@osdl.org>

Re: [patch] drivers: wait for threaded probes between initcall levels

[Andrew Morton]

On Fri, 27 Oct 2006 11:47:29 -0700
Stephen Hemminger <shemminger@osdl.org> wrote:

> On Fri, 27 Oct 2006 11:42:37 -0700
> Andrew Morton <akpm@osdl.org> wrote:
> 
> > From: Andrew Morton <akpm@osdl.org>
> > 
> > The multithreaded-probing code has a problem: after one initcall level (eg,
> > core_initcall) has been processed, we will then start processing the next
> > level (postcore_initcall) while the kernel threads which are handling
> > core_initcall are still executing.  This breaks the guarantees which the
> > layered initcalls previously gave us.
> > 
> > IOW, we want to be multithreaded _within_ an initcall level, but not between
> > different levels.
> > 
> > Fix that up by causing the probing code to wait for all outstanding probes at
> > one level to complete before we start processing the next level.
> > 
> > Cc: Greg KH <greg@kroah.com>
> > Signed-off-by: Andrew Morton <akpm@osdl.org>
> > ---
> > 
> 
> This looks like a good place to use a counting semaphore.
> 

I couldn't work out a way of doing that.  I guess one could a) count the
number of threads which are going to be started, b) start them all, c) do
an up() when each thread ends and d) handle errors somehow.

Re: [patch] drivers: wait for threaded probes between initcall levels

[Linus Torvalds]

On Fri, 27 Oct 2006, Andrew Morton wrote:
> 
> I couldn't work out a way of doing that.  I guess one could a) count the
> number of threads which are going to be started, b) start them all, c) do
> an up() when each thread ends and d) handle errors somehow.

No. First off, you want to _limit_ the maximum number of parallelism 
anyway (memory pressure and sanity), so you want to use the counting 
semaphore for that too.

The easiest way to do it would probably be something like this:

	#define PARALLELISM (10)

	static struct semaphore outstanding;

	struct thread_exec {
		int (*fn)(void *);
		void *arg;
		struct completion completion;
	};

	static void allow_parallel(int n)
	{
		while (--n >= 0)
			up(&outstanding);
	}

	static void wait_for_parallel(int n)
	{
		while (--n >= 0)
			down(&outstanding);
	}

	static int do_in_parallel(void *arg)
	{
		struct thread_exec *p = arg;
		int (*fn)(void *) = p->fn;
		void *arg = p->arg;
		int retval;

		/* Tell the caller we are done with the arguments */
		complete(&p->completion);

		/* Do the actual work in parallel */
		retval = p->fn(p->arg);

		/*
		 * And then tell the rest of the world that we've
		 * got one less parallel thing outstanding..
		 */
		up(&outstanding);
		return retval;
	}

	static void execute_in_parallel(int (*fn)(void *), void *arg)
	{
		struct thread_exec arg = { .fn = fn, .arg = arg };

		/* Make sure we can have more outstanding parallel work */
		down(&outstanding);

		arg.fn = fn;
		arg.arg = arg;
		init_completion(&arg.completion);

		kernel_thread(do_in_parallel, &arg);

		/* We need to wait until our "arg" is safe */
		wait_for_completion(&arg.completion)
	}

The above is ENTIRELY UNTESTED, but the point of it is that it should now 
allow you to do something like this:

	/* Set up how many parallel threads we can run */
	allow_parallel(PARALLELISM);

	...

	/*
	 * Run an arbitrary number of threads with that
	 * parallelism.
	 */
	for (i = 0; i < ... ; i++)
		execute_in_parallel(fnarray[i].function, 
				    fnarray[i].argument);

	...

	/* And wait for all of them to complete */
	wait_for_parallel(PARALLELISM);

and this is totally generic (ie this is useful for initcalls or anything 
else). Note also how you can set up the parallelism (and wait for it) 
totally independently (ie that can be done at some earlier stage, and the 
"execute_in_parallel()" can just be executed in any random situation in 
between - as many times as you like. It will always honor the parallelism.

By setting PARALLELISM to 1, you basically only ever allow one outstanding 
call at any time (ie it becomes serial), so you don't even have to make 
this a config option, you could do it as a runtime setup thing.

Hmm?

(And I repeat: the above code is untested, and was written in the email 
client. It has never seen a compiler, and not gotten a _whole_ lot of 
thinking).

		Linus

Re: [patch] drivers: wait for threaded probes between initcall levels

[Linus Torvalds]

On Fri, 27 Oct 2006, Linus Torvalds wrote:

> 
> 	static int do_in_parallel(void *arg)
> 	{
> 		struct thread_exec *p = arg;
> 		int (*fn)(void *) = p->fn;
> 		void *arg = p->arg;
> 		int retval;
> 
> 		/* Tell the caller we are done with the arguments */
> 		complete(&p->completion);
> 
> 		/* Do the actual work in parallel */
> 		retval = p->fn(p->arg);

Duh. The whole reason I copied them was to _not_ do that. That last line 
should obviously be

		retval = fn(arg);

because "p" may gone after we've done the "complete()".

> (And I repeat: the above code is untested, and was written in the email 
> client. It has never seen a compiler, and not gotten a _whole_ lot of 
> thinking).

.. This hasn't changed, I just looked through the code once and found that 
obvious bug.


		Linus

Re: [patch] drivers: wait for threaded probes between initcall levels

[Greg KH]

On Fri, Oct 27, 2006 at 01:48:54PM -0700, Linus Torvalds wrote:
> 
> 
> On Fri, 27 Oct 2006, Linus Torvalds wrote:
> 
> > 
> > 	static int do_in_parallel(void *arg)
> > 	{
> > 		struct thread_exec *p = arg;
> > 		int (*fn)(void *) = p->fn;
> > 		void *arg = p->arg;
> > 		int retval;
> > 
> > 		/* Tell the caller we are done with the arguments */
> > 		complete(&p->completion);
> > 
> > 		/* Do the actual work in parallel */
> > 		retval = p->fn(p->arg);
> 
> Duh. The whole reason I copied them was to _not_ do that. That last line 
> should obviously be
> 
> 		retval = fn(arg);
> 
> because "p" may gone after we've done the "complete()".
> 
> > (And I repeat: the above code is untested, and was written in the email 
> > client. It has never seen a compiler, and not gotten a _whole_ lot of 
> > thinking).
> 
> .. This hasn't changed, I just looked through the code once and found that 
> obvious bug.

Heh, ok, I'll take this idea, and Andrew's patch, and rework things for
the next round of 2.6.20-rc kernels, and mark the current stuff as
BROKEN for now.

thanks,

greg k-h

Re: [patch] drivers: wait for threaded probes between initcall levels

[Linus Torvalds]

On Fri, 27 Oct 2006, Greg KH wrote:
> 
> Heh, ok, I'll take this idea, and Andrew's patch, and rework things for
> the next round of 2.6.20-rc kernels, and mark the current stuff as
> BROKEN for now.

My interface stuff is kind of designed for:

 - keep the current "init" sequence as-is for now

 - keep the _actual_ PCI probing non-parallel, so that we actually do all 
   the bus _discovery_ in a repeatable and sane order.

 - use the new "execute_in_parallel()" interface for things that actually 
   _sleep_. For example, all the PCI IDE _driver_ attachement should be 
   done synchronously, but perhaps the code that tries to see if there are 
   actual disks (ie the stuff that has timeouts etc) can use the parallel 
   execution.

 - module loading would do a "allow_parallel(1)" and 
   "wait_for_parallel(1)" thing when calling the module init function (so 
   that a module could use "execute_in_parallel()" whether compiled in or 
   not, and each "init level" at boot would also do this (with some bigger 
   number, like 10), so that for drivers etc that want to do async stuff, 
   they can do so in parallel (but they'd still have done the actual hard 
   device find/init serially - keeping the link order relevant for things 
   like IDE controller discovery)

How does this sound?

There's really no reason to parallelise the actual PCI config cycles and 
probing and stuff. It's only when some driver knows that it might have to 
do some longer-running thing that it might want to execute a thread in 
parallel with other things - but it still needs to be done in a controlled 
situation, so that when "driver_init()" stops and "filesystem_init()" 
starts, we must wait for all the driver-init parallel tasks to finish 
(since "filesystem_init()" is allowed to depend on them).

Hmm?

		Linus

Re: [patch] drivers: wait for threaded probes between initcall levels

[Alan Cox]

Ar Gwe, 2006-10-27 am 11:42 -0700, ysgrifennodd Andrew Morton:
> IOW, we want to be multithreaded _within_ an initcall level, but not between
> different levels.

Thats actually insufficient. We have link ordered init sequences in
large numbers of driver subtrees (ATA, watchdog, etc). We'll need
several more initcall layers to fix that.

Alan

Re: [patch] drivers: wait for threaded probes between initcall levels

[Andrew Morton]

On Fri, 27 Oct 2006 23:59:30 +0100
Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

> Ar Gwe, 2006-10-27 am 11:42 -0700, ysgrifennodd Andrew Morton:
> > IOW, we want to be multithreaded _within_ an initcall level, but not between
> > different levels.
> 
> Thats actually insufficient. We have link ordered init sequences in
> large numbers of driver subtrees (ATA, watchdog, etc). We'll need
> several more initcall layers to fix that.
> 

It would be nice to express those dependencies in some clearer and less
fragile manner than link order.  I guess finer-grained initcall levels
would do that, but it doesn't scale very well.

But whatever.  I think multithreaded probing just doesn't pass the
benefit-versus-hassle test, sorry.   Make it dependent on CONFIG_GREGKH ;)

Re: [patch] drivers: wait for threaded probes between initcall levels

[Grant Grundler]

On Fri, Oct 27, 2006 at 04:06:26PM -0700, Andrew Morton wrote:
> On Fri, 27 Oct 2006 23:59:30 +0100
> Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> 
> > Ar Gwe, 2006-10-27 am 11:42 -0700, ysgrifennodd Andrew Morton:
> > > IOW, we want to be multithreaded _within_ an initcall level, but not between
> > > different levels.
> > 
> > Thats actually insufficient. We have link ordered init sequences in
> > large numbers of driver subtrees (ATA, watchdog, etc). We'll need
> > several more initcall layers to fix that.
> > 
> 
> It would be nice to express those dependencies in some clearer and less
> fragile manner than link order.  I guess finer-grained initcall levels
> would do that, but it doesn't scale very well.

Would making use of depmod data be a step in the right direction?
ie nic driver calls extern function (e.g. pci_enable_device())
and therefore must depend on module which provides that function.

My guess is this probably isn't 100% sufficient to replace all initcall
levels.  But likely sufficient within a given initcall level.
My main concern are circular dependencies (which are rare).

> But whatever.  I think multithreaded probing just doesn't pass the
> benefit-versus-hassle test, sorry.   Make it dependent on CONFIG_GREGKH ;)

Isn't already? :)

I thought parallel PCI and SCSI probing on system with multiple NICs and
"SCSI" storage requires udev to create devices with consistent naming.

thanks,
grant

Re: [patch] drivers: wait for threaded probes between initcall levels

[Andrew Morton]

On Fri, 27 Oct 2006 23:09:05 -0600
Grant Grundler <grundler@parisc-linux.org> wrote:

> On Fri, Oct 27, 2006 at 04:06:26PM -0700, Andrew Morton wrote:
> > On Fri, 27 Oct 2006 23:59:30 +0100
> > Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> > 
> > > Ar Gwe, 2006-10-27 am 11:42 -0700, ysgrifennodd Andrew Morton:
> > > > IOW, we want to be multithreaded _within_ an initcall level, but not between
> > > > different levels.
> > > 
> > > Thats actually insufficient. We have link ordered init sequences in
> > > large numbers of driver subtrees (ATA, watchdog, etc). We'll need
> > > several more initcall layers to fix that.
> > > 
> > 
> > It would be nice to express those dependencies in some clearer and less
> > fragile manner than link order.  I guess finer-grained initcall levels
> > would do that, but it doesn't scale very well.
> 
> Would making use of depmod data be a step in the right direction?

Nope.  The linkage-order problem is by definition applicable to
linked-into-vmlinux code, not to modules.

> ie nic driver calls extern function (e.g. pci_enable_device())
> and therefore must depend on module which provides that function.
> 
> My guess is this probably isn't 100% sufficient to replace all initcall
> levels.  But likely sufficient within a given initcall level.
> My main concern are circular dependencies (which are rare).

The simplest implementation of "A needs B to have run" is for A to simply
call B, and B arranges to not allow itself to be run more than once.

But that doesn't work in the case "A needs B to be run, but only if B is
present".  Resolving this one would require something like a fancy
"synchronisation object" against which dependers and dependees can register
interest, and a core engine which takes care of the case where a depender
registers against something which no dependees have registered.

The mind boggles.

> > But whatever.  I think multithreaded probing just doesn't pass the
> > benefit-versus-hassle test, sorry.   Make it dependent on CONFIG_GREGKH ;)
> 
> Isn't already? :)
> 
> I thought parallel PCI and SCSI probing on system with multiple NICs and
> "SCSI" storage requires udev to create devices with consistent naming.

For some reason people get upset when we rename all their devices.  They're
a humourless lot.

Re: [patch] drivers: wait for threaded probes between initcall levels

[Andrew Morton]

On Fri, 27 Oct 2006 22:19:25 -0700
Andrew Morton <akpm@osdl.org> wrote:

> The simplest implementation of "A needs B to have run" is for A to simply
> call B, and B arranges to not allow itself to be run more than once.
> 
> But that doesn't work in the case "A needs B to be run, but only if B is
> present".  Resolving this one would require something like a fancy
> "synchronisation object" against which dependers and dependees can register
> interest, and a core engine which takes care of the case where a depender
> registers against something which no dependees have registered.

otoh, we could stick with the simple "A calls B" solution, and A also
provides an attribute-weak implementation of B to cover the "A needs B but
only if B is present" problems.

Had to say, really - one would need to study some specific problem cases.

Re: [patch] drivers: wait for threaded probes between initcall levels

[Grant Grundler]

On Fri, Oct 27, 2006 at 10:19:25PM -0700, Andrew Morton wrote:
> > > It would be nice to express those dependencies in some clearer and less
> > > fragile manner than link order.  I guess finer-grained initcall levels
> > > would do that, but it doesn't scale very well.
> > 
> > Would making use of depmod data be a step in the right direction?
> 
> Nope.  The linkage-order problem is by definition applicable to
> linked-into-vmlinux code, not to modules.

But wouldn't the same concept apply to non-module symbols that
are tagged with EXPORT_SYMBOL()?
Maybe I'm just showing my ignorance about kernel linking here...

> > ie nic driver calls extern function (e.g. pci_enable_device())
> > and therefore must depend on module which provides that function.
> > 
> > My guess is this probably isn't 100% sufficient to replace all initcall
> > levels.  But likely sufficient within a given initcall level.
> > My main concern are circular dependencies (which are rare).
> 
> The simplest implementation of "A needs B to have run" is for A to simply
> call B, and B arranges to not allow itself to be run more than once.

Yes. we already have code like this in the kernel.
e.g. superio support in drivers/parisc.

> But that doesn't work in the case "A needs B to be run, but only if B is
> present".

I was thinking of "A is present and calls into module B, therefore B needs
to init first". In this case, A won't care if B is really present or not.
A depends on B to figure that out at runtime. If B is not configured into
the kernel, A won't ever call B since the "function" will be a NOP.
(e.g. #ifndef CONFIG_B/#define b_service() /* NOP */ /#endif)
 

>  Resolving this one would require something like a fancy
> "synchronisation object" against which dependers and dependees can register
> interest, and a core engine which takes care of the case where a depender
> registers against something which no dependees have registered.

I guess I was wondering if the kernel link step could use symbol information
in a similar way the kernel module autoloader uses depmod info. But other
parts of the kernel might not be as modular as most of the IO subsystems
are.

I'm not looking for ways to make the process more complicated for
the people maintaining code. Keeping the registrations of dependencies
up-to-date manually would just be another PITA.

...
> > I thought parallel PCI and SCSI probing on system with multiple NICs and
> > "SCSI" storage requires udev to create devices with consistent naming.
> 
> For some reason people get upset when we rename all their devices.  They're
> a humourless lot.

Hey! I resemble that remark! ;)

(yeah, I've been a victim of that problem way too many times.)

thanks,
grant

Re: [patch] drivers: wait for threaded probes between initcall levels

[Stefan Richter]

Grant Grundler wrote:
> On Fri, Oct 27, 2006 at 10:19:25PM -0700, Andrew Morton wrote:
>>> I thought parallel PCI and SCSI probing on system with multiple NICs and
>>> "SCSI" storage requires udev to create devices with consistent naming.
>> For some reason people get upset when we rename all their devices.  They're
>> a humourless lot.
> 
> Hey! I resemble that remark! ;)
> 
> (yeah, I've been a victim of that problem way too many times.)

I hear network interfaces can be selected by their MACs, which are
globally unique and persistent. Most SCSI hardware has globally unique
and persistent unit properties too, and udev indeed uses them these days.
-- 
Stefan Richter
-=====-=-==- =-=- ===--
http://arcgraph.de/sr/

Re: [patch] drivers: wait for threaded probes between initcall levels

[Alan Cox]

Ar Sad, 2006-10-28 am 22:48 +0200, ysgrifennodd Stefan Richter:
> I hear network interfaces can be selected by their MACs, which are
> globally unique and persistent. Most SCSI hardware has globally unique
> and persistent unit properties too, and udev indeed uses them these days.

You hear incorrectly. The MAC address is only required to be *machine
unique*, please see the 802.1/2 specification documents for more detail.
Distinguishing by card MAC is a hack that works on some systems only.

SCSI is also unreliable for serial numbers because of USB, brain-dead
raid controllers and other devices that fake the same ident for many
devices.

There is another ugly too - many driver/library layers "know" that
during boot the code is not re-entered so has no locking. Before you can
go multi-probe you also have to fix all the locking in the drivers that
have boot time specific functionality (eg IDE).

Alan

Re: [patch] drivers: wait for threaded probes between initcall levels

[Randy Dunlap]

On Sun, 29 Oct 2006 00:34:57 +0100 Alan Cox wrote:

> Ar Sad, 2006-10-28 am 22:48 +0200, ysgrifennodd Stefan Richter:
> > I hear network interfaces can be selected by their MACs, which are
> > globally unique and persistent. Most SCSI hardware has globally unique
> > and persistent unit properties too, and udev indeed uses them these days.
> 
> You hear incorrectly. The MAC address is only required to be *machine
> unique*, please see the 802.1/2 specification documents for more detail.
> Distinguishing by card MAC is a hack that works on some systems only.

I would have expected "most" instead of "some", but you have
different experiences than I do.  What spec requires a MAC address
to be machine-unique?

IEEE "makes it possible for organizations to employ unique individual 
LAN MAC addresses, group addresses, and protocol identifiers."

IEEE 802 goes on to say:
"The concept of universal addressing is based on the 
idea that all potential members of a network need to
have a unique identifier (if they are going to coexist 
in the network). The advantage of a universal address is
that a station with such an address can be attached to any 
LAN in the world with an assurance that the address is unique."

and then "recommended" (but not required):

"The recommended approach is for each device associated 
with a distinct point of attachment to a LAN to
have its own unique MAC address. Typically, therefore, 
a LAN adapter card (or, e.g., an equivalent chip or
set of chips on a motherboard) should have one unique 
MAC address for each LAN attachment that it can
support at a given time.

and then this part seems contrary to the machine-unique quality
that you mentioned (I guess--don't know--that this is what
Sun used to do ?):

"NOTE—It is recognized that an alternative approach has 
gained currency in some LAN implementations, in which the
device is interpreted as a complete computer system, 
which can have multiple attachments to different LANs. Under this
interpretation, a single LAN MAC address is used to 
identify all of the system’s points of attachment to the LANs in
question. This approach, unlike the recommended one, does 
not automatically meet the requirements of
IEEE Std 802.1D-1998 MAC bridging."


> SCSI is also unreliable for serial numbers because of USB, brain-dead
> raid controllers and other devices that fake the same ident for many
> devices.
> 
> There is another ugly too - many driver/library layers "know" that
> during boot the code is not re-entered so has no locking. Before you can
> go multi-probe you also have to fix all the locking in the drivers that
> have boot time specific functionality (eg IDE).

---
~Randy

Re: [patch] drivers: wait for threaded probes between initcall levels

[Cornelia Huck]

On Fri, 27 Oct 2006 16:06:26 -0700,
Andrew Morton <akpm@osdl.org> wrote:

> On Fri, 27 Oct 2006 23:59:30 +0100
> Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> 
> > Ar Gwe, 2006-10-27 am 11:42 -0700, ysgrifennodd Andrew Morton:
> > > IOW, we want to be multithreaded _within_ an initcall level, but not between
> > > different levels.
> > 
> > Thats actually insufficient. We have link ordered init sequences in
> > large numbers of driver subtrees (ATA, watchdog, etc). We'll need
> > several more initcall layers to fix that.
> > 
> 
> It would be nice to express those dependencies in some clearer and less
> fragile manner than link order.  I guess finer-grained initcall levels
> would do that, but it doesn't scale very well.

Would it be sufficient just to make the busses wait until all their
devices are through with their setup? This is what the ccw bus on s390
does:
- Increase counter for device and start device recognition for it
- Continue for next device
- When async device recognition (and probable enablement) is finished,
  register device and decrease counter
- ccw bus init function waits till counter has dropped to 0

This has worked fine for us since 2.5. OTOH, s390 doesn't have such a
diverse set as hardware as a PC :)

-- 
Cornelia Huck
Linux for zSeries Developer
Tel.: +49-7031-16-4837, Mail: cornelia.huck@de.ibm.com

Re: [patch] drivers: wait for threaded probes between initcall levels

[Alan Cox]

Ar Llu, 2006-10-30 am 10:44 +0100, ysgrifennodd Cornelia Huck:
> Would it be sufficient just to make the busses wait until all their
> devices are through with their setup? This is what the ccw bus on s390
> does:

For ATA and IDE no, it might work with SCSI but your devices would
randomly re-order which is also obnoxious. IDE relies on both link probe
order and also has code that knows boot time processing is single
threaded. 

Re: [patch] drivers: wait for threaded probes between initcall levels

[Matthew Wilcox]

On Mon, Oct 30, 2006 at 10:48:51AM +0000, Alan Cox wrote:
> Ar Llu, 2006-10-30 am 10:44 +0100, ysgrifennodd Cornelia Huck:
> > Would it be sufficient just to make the busses wait until all their
> > devices are through with their setup? This is what the ccw bus on s390
> > does:
> 
> For ATA and IDE no, it might work with SCSI but your devices would
> randomly re-order which is also obnoxious. IDE relies on both link probe
> order and also has code that knows boot time processing is single
> threaded. 

There's no need to parallelise the scanning of SCSI host adapters.
Indeed, it only causes pain.  With

http://git.kernel.org/git/?p=linux/kernel/git/jejb/scsi-misc-2.6.git;a=commitdiff;h=3e082a910d217b2e7b186077ebf5a1126a68c62f

and

http://git.parisc-linux.org/?p=linux-2.6.git;a=shortlog;h=scsi-async-scan

some bugfixing, and moving the scsi initialisation earlier (so it has
longer to complete while other things initialise), we should never have
to wait for scsi scans again.

And your devices only reorder as much as they ever used to with scsi.

Re: [patch] drivers: wait for threaded probes between initcall levels

[Olaf Hering]

On Fri, Oct 27, Alan Cox wrote:

> Ar Gwe, 2006-10-27 am 11:42 -0700, ysgrifennodd Andrew Morton:
> > IOW, we want to be multithreaded _within_ an initcall level, but not between
> > different levels.
> 
> Thats actually insufficient. We have link ordered init sequences in
> large numbers of driver subtrees (ATA, watchdog, etc). We'll need
> several more initcall layers to fix that.

Is it time for something better?
True dependencies, an addition to or as replacement for module_init()?
random example: hfs/super.c:
depends_on_initialized(init_hfs_fs: init_hfsplus_fs,kmem_cache_thingie,core_filesystem_thingie,foo,bar,worldpeace);
If init_hfsplus_fs() does not exist it should be no error.

Whatever the sytax will be and however its parsed during build, that link order
requirement bites every other month.