* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-08 20:01 ` Linus Torvalds
@ 2007-05-08 20:26 ` david
2007-05-09 7:58 ` Cornelia Huck
2007-05-09 22:21 ` Phillip Susi
2007-05-08 20:51 ` Cornelia Huck
` (2 subsequent siblings)
3 siblings, 2 replies; 55+ messages in thread
From: david @ 2007-05-08 20:26 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Cornelia Huck, Adrian Bunk, Greg K-H, linux-kernel
On Tue, 8 May 2007, Linus Torvalds wrote:
> Instead of changing existign probe functionality to be asynchronous, we
> could *add* a new and asynchronous part to it. For example, we could make
> the rule for PCI - or other bus - devices be:
>
> - the bus will *first* call the "probe()" function synchronously.
>
> - after that one has completed, we will call "probe_async()"
> asynchronously at some point (it ie might be scheduled immediately
> after the "probe()" call, but delayed by some arbitrary issues like
> just already having too many asynchronous probes on-going or similar)
>
> (A variation of the above might be that *everybody*s synchronous probe
> function will be called first, and then the asynchronous probe functions
> will be called only when they are all done. That might help with drivers
> that have dependencies between different PCI functions - Cardbus comes to
> mind, where the different slots look like independent PCI devices, but
> slot zero is literally the master and controls some of the functions on
> slot 1 too - similar issues may well happen in other multi-function
> devices, and it might simplify things if you knew that the serial probe
> had completed fully before the asynchronous parallel part even starts).
is it really enough to do the sync followed by the async piece?
it seems to me that there may be a need for three steps
1. prep (sync, what we do today)
which will all complete before
2. parallel (async)
which will all complete before
3. cleanup (sync)
for some busses you can do the device enumeration in the prep phase, but I
would expect that for others you can't enumerate the devices until you
actually go looking for them. so by allowing the cleanup phase you have a
spot that can order everything as if the second phase was sequential.
I would expect that almost no drivers would use all three, but I could see
some useing #1 and #2 while others use #2 and #3
in any case there needs to be a way to wait until the async portion
finishes before going on to next steps (you don't want to try to probe
drives for md partitions until the drives have finished spinning up for
example)
David Lang
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-08 20:26 ` david
@ 2007-05-09 7:58 ` Cornelia Huck
2007-05-09 8:33 ` david
2007-05-09 22:21 ` Phillip Susi
1 sibling, 1 reply; 55+ messages in thread
From: Cornelia Huck @ 2007-05-09 7:58 UTC (permalink / raw)
To: david; +Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel
On Tue, 8 May 2007 13:26:00 -0700 (PDT),
david@lang.hm wrote:
> is it really enough to do the sync followed by the async piece?
>
> it seems to me that there may be a need for three steps
>
> 1. prep (sync, what we do today)
> which will all complete before
> 2. parallel (async)
> which will all complete before
> 3. cleanup (sync)
>
> for some busses you can do the device enumeration in the prep phase, but I
> would expect that for others you can't enumerate the devices until you
> actually go looking for them. so by allowing the cleanup phase you have a
> spot that can order everything as if the second phase was sequential.
>
> I would expect that almost no drivers would use all three, but I could see
> some useing #1 and #2 while others use #2 and #3
Hm, how about:
1. ->probe() called (wait until it is finished)
2. ->probe_async() called (only kick off, don't wait, go to next device)
3. bus waits until all async probes have returned
This would imply a wait mechanism on the bus, like upping a counter for
each successfully called ->probe_async(), the driver downing the
counter when all actions triggered by ->probe_async() have finshed, and
the bus waiting until the counter has reached 0.
(Individual drivers can have 1., 2., or 1. & 2. A bus may support 1.
(like now), 2. & 3. or 1., 2. and 3.)
> in any case there needs to be a way to wait until the async portion
> finishes before going on to next steps (you don't want to try to probe
> drives for md partitions until the drives have finished spinning up for
> example)
Agreed. Would it be enough if we only have one bus in flight
simuntaneously?
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 7:58 ` Cornelia Huck
@ 2007-05-09 8:33 ` david
2007-05-09 9:15 ` Cornelia Huck
0 siblings, 1 reply; 55+ messages in thread
From: david @ 2007-05-09 8:33 UTC (permalink / raw)
To: Cornelia Huck; +Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel
On Wed, 9 May 2007, Cornelia Huck wrote:
> On Tue, 8 May 2007 13:26:00 -0700 (PDT),
> david@lang.hm wrote:
>
>> is it really enough to do the sync followed by the async piece?
>>
>> it seems to me that there may be a need for three steps
>>
>> 1. prep (sync, what we do today)
>> which will all complete before
>> 2. parallel (async)
>> which will all complete before
>> 3. cleanup (sync)
>>
>> for some busses you can do the device enumeration in the prep phase, but I
>> would expect that for others you can't enumerate the devices until you
>> actually go looking for them. so by allowing the cleanup phase you have a
>> spot that can order everything as if the second phase was sequential.
>>
>> I would expect that almost no drivers would use all three, but I could see
>> some useing #1 and #2 while others use #2 and #3
>
> Hm, how about:
> 1. ->probe() called (wait until it is finished)
> 2. ->probe_async() called (only kick off, don't wait, go to next device)
> 3. bus waits until all async probes have returned
>
> This would imply a wait mechanism on the bus, like upping a counter for
> each successfully called ->probe_async(), the driver downing the
> counter when all actions triggered by ->probe_async() have finshed, and
> the bus waiting until the counter has reached 0.
>
> (Individual drivers can have 1., 2., or 1. & 2. A bus may support 1.
> (like now), 2. & 3. or 1., 2. and 3.)
>
>> in any case there needs to be a way to wait until the async portion
>> finishes before going on to next steps (you don't want to try to probe
>> drives for md partitions until the drives have finished spinning up for
>> example)
>
> Agreed. Would it be enough if we only have one bus in flight
> simuntaneously?
two things I see
1. why should different, unrelated busses need to wait for each other?
picking two, why can't you have SCSI and USB going through their timeouts
at the same time?
2. I'm not sure that you can always do the device enumeration before you
do the slow probing, there's another message in this thread that talks
about a USB device where you need to load firmware to it before you can
find out what is really there. in a case like this devices would either
show up in a random order during step #2, or they should not be added to
the system until step #3, which makes step #3 more then just waiting for
the async portions to finish.
David Lang
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 8:33 ` david
@ 2007-05-09 9:15 ` Cornelia Huck
2007-05-09 9:25 ` david
2007-05-09 9:30 ` Stefan Richter
0 siblings, 2 replies; 55+ messages in thread
From: Cornelia Huck @ 2007-05-09 9:15 UTC (permalink / raw)
To: david; +Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel, Duncan Sands
On Wed, 9 May 2007 01:33:14 -0700 (PDT),
david@lang.hm wrote:
> 1. why should different, unrelated busses need to wait for each other?
> picking two, why can't you have SCSI and USB going through their timeouts
> at the same time?
If they don't have dependencies on each other, yes. Some busses should
be finished before probing for others start (e.g. low-level busses).
> 2. I'm not sure that you can always do the device enumeration before you
> do the slow probing, there's another message in this thread that talks
> about a USB device where you need to load firmware to it before you can
> find out what is really there. in a case like this devices would either
> show up in a random order during step #2, or they should not be added to
> the system until step #3, which makes step #3 more then just waiting for
> the async portions to finish.
I'm not sure whether that is not really a question of "one depends upon
the other". If the low level bus knows where to point its probe at, the
higher level should be able to look at sane devices after the firmware
has been loaded (or am I misunderstanding the situation here?)
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 9:15 ` Cornelia Huck
@ 2007-05-09 9:25 ` david
2007-05-09 13:20 ` Cornelia Huck
2007-05-09 9:30 ` Stefan Richter
1 sibling, 1 reply; 55+ messages in thread
From: david @ 2007-05-09 9:25 UTC (permalink / raw)
To: Cornelia Huck
Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel, Duncan Sands
On Wed, 9 May 2007, Cornelia Huck wrote:
> On Wed, 9 May 2007 01:33:14 -0700 (PDT),
> david@lang.hm wrote:
>
>> 1. why should different, unrelated busses need to wait for each other?
>> picking two, why can't you have SCSI and USB going through their timeouts
>> at the same time?
>
> If they don't have dependencies on each other, yes. Some busses should
> be finished before probing for others start (e.g. low-level busses).
I think we are agreeing on this, I was responding to your question of if
we could get away to limiting things to one bus at a time.
>> 2. I'm not sure that you can always do the device enumeration before you
>> do the slow probing, there's another message in this thread that talks
>> about a USB device where you need to load firmware to it before you can
>> find out what is really there. in a case like this devices would either
>> show up in a random order during step #2, or they should not be added to
>> the system until step #3, which makes step #3 more then just waiting for
>> the async portions to finish.
>
> I'm not sure whether that is not really a question of "one depends upon
> the other". If the low level bus knows where to point its probe at, the
> higher level should be able to look at sane devices after the firmware
> has been loaded (or am I misunderstanding the situation here?)
makeing up an example
today the process would be
call driver X to initialize it's devices.
driver X follows these steps
1. see if a compatible chipset/device appears to be available
2. initialize the chipset/device (loading firmware)
3. query the now partially initialized chipset/device to see what specific
options are enabled in the hardware
4. for each 'thing' that's enabled in hardware, initialize and register
it.
step 2 can take a long time and so you want it to be async so that it's
timeouts can be running in parallel (for that matter step 1 may have some
timeouts as well)
however, you can't tell what devices to register until after step 3
so in at least some cases, it seems as if the registratin of devices needs
to be after the async section.
Linus made a good case fo an example (hard drives) where the registration
can happen first and fast, but the async section (spinning up) takes a
while and could come second.
to acommodate both of these device models it seems to me that you can't
defice either
async followed by sync
or
sync followed by async
but instead need to support the combined
sync followed by async followed by sync.
now, it's very possible that I'm mistaken and one of the two-part models
can be used for everything, if so then it's definantly simpler to do so.
David Lang
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 9:25 ` david
@ 2007-05-09 13:20 ` Cornelia Huck
2007-05-09 16:18 ` david
0 siblings, 1 reply; 55+ messages in thread
From: Cornelia Huck @ 2007-05-09 13:20 UTC (permalink / raw)
To: david; +Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel, Duncan Sands
On Wed, 9 May 2007 02:25:10 -0700 (PDT),
david@lang.hm wrote:
> makeing up an example
>
> today the process would be
> call driver X to initialize it's devices.
>
> driver X follows these steps
>
> 1. see if a compatible chipset/device appears to be available
>
> 2. initialize the chipset/device (loading firmware)
>
> 3. query the now partially initialized chipset/device to see what specific
> options are enabled in the hardware
>
> 4. for each 'thing' that's enabled in hardware, initialize and register
> it.
The sync ->probe() could return after step 1 (when it is clear the
driver wants to bind to the device). The remainder should be done in
the async function, so that the bus could go on with other devices. The
driver should then signal when it is done with its registrations.
> step 2 can take a long time and so you want it to be async so that it's
> timeouts can be running in parallel (for that matter step 1 may have some
> timeouts as well)
>
> however, you can't tell what devices to register until after step 3
>
> so in at least some cases, it seems as if the registratin of devices needs
> to be after the async section.
If you're talking about devices registered by the driver, these should
be in the async section.
> Linus made a good case fo an example (hard drives) where the registration
> can happen first and fast, but the async section (spinning up) takes a
> while and could come second.
I don't really see the difference?
> to acommodate both of these device models it seems to me that you can't
> defice either
> async followed by sync
> or
> sync followed by async
> but instead need to support the combined
> sync followed by async followed by sync.
>
> now, it's very possible that I'm mistaken and one of the two-part models
> can be used for everything, if so then it's definantly simpler to do so.
Hm, maybe I'm dense, but I really don't understand the problem here.
Why should (optional) sync followed by (optional) async followed by bus
synchonization not cover all cases? The async portion can also do some
"sync" stuff and then signal completion, can't it?
If you're worried about notifications to userspace, you could first
suppress the add uevent via uevent_suppress and then generate it when
you signal completion to the bus.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 13:20 ` Cornelia Huck
@ 2007-05-09 16:18 ` david
2007-05-09 17:07 ` Cornelia Huck
2007-05-09 17:07 ` Greg KH
0 siblings, 2 replies; 55+ messages in thread
From: david @ 2007-05-09 16:18 UTC (permalink / raw)
To: Cornelia Huck
Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel, Duncan Sands
On Wed, 9 May 2007, Cornelia Huck wrote:
> On Wed, 9 May 2007 02:25:10 -0700 (PDT),
> david@lang.hm wrote:
>
>> makeing up an example
>>
>> today the process would be
>> call driver X to initialize it's devices.
>>
>> driver X follows these steps
>>
>> 1. see if a compatible chipset/device appears to be available
>>
>> 2. initialize the chipset/device (loading firmware)
>>
>> 3. query the now partially initialized chipset/device to see what specific
>> options are enabled in the hardware
>>
>> 4. for each 'thing' that's enabled in hardware, initialize and register
>> it.
>
> The sync ->probe() could return after step 1 (when it is clear the
> driver wants to bind to the device). The remainder should be done in
> the async function, so that the bus could go on with other devices. The
> driver should then signal when it is done with its registrations.
but then won't the devices get registered in a random order? (i.e.
whenever the async portion finishes the probing and finds the details of
what there is to register)
>> to acommodate both of these device models it seems to me that you can't
>> defice either
>> async followed by sync
>> or
>> sync followed by async
>> but instead need to support the combined
>> sync followed by async followed by sync.
>>
>> now, it's very possible that I'm mistaken and one of the two-part models
>> can be used for everything, if so then it's definantly simpler to do so.
>
> Hm, maybe I'm dense, but I really don't understand the problem here.
> Why should (optional) sync followed by (optional) async followed by bus
> synchonization not cover all cases? The async portion can also do some
> "sync" stuff and then signal completion, can't it?
>
> If you're worried about notifications to userspace, you could first
> suppress the add uevent via uevent_suppress and then generate it when
> you signal completion to the bus.
I'm not worried about notification to userspace, I'm worried about devices
getting registered during the async stage appearing in different orders on
different boots due to different timeings.
if you can identify the device well enough to register it quickly then the
approach that Linus proposed works well, which is
sync probe, identify devices, register devices
async initialize devices
wait for all async to complete
however I'm talking about cases where you can't fully identify the devices
(at least not well enough to register them with the kernel) and am saying
that doing
sync probe
async initialize device, register device
wait for all async to complete
results in unpredictable device ordering, but if instead you did
sync probe
async initialize device
sync wait for all async to complete, register device
you can get back to predictable ordering
David Lang
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 16:18 ` david
@ 2007-05-09 17:07 ` Cornelia Huck
2007-05-09 17:09 ` david
2007-05-09 17:07 ` Greg KH
1 sibling, 1 reply; 55+ messages in thread
From: Cornelia Huck @ 2007-05-09 17:07 UTC (permalink / raw)
To: david; +Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel, Duncan Sands
On Wed, 9 May 2007 09:18:10 -0700 (PDT),
david@lang.hm wrote:
> I'm not worried about notification to userspace, I'm worried about devices
> getting registered during the async stage appearing in different orders on
> different boots due to different timeings.
Ah, now I think I understand what you mean. You're talking about stuff
like block devices or net devices, right?
>
> if you can identify the device well enough to register it quickly then the
> approach that Linus proposed works well, which is
>
> sync probe, identify devices, register devices
> async initialize devices
> wait for all async to complete
>
> however I'm talking about cases where you can't fully identify the devices
> (at least not well enough to register them with the kernel) and am saying
> that doing
>
> sync probe
> async initialize device, register device
> wait for all async to complete
>
> results in unpredictable device ordering, but if instead you did
>
> sync probe
> async initialize device
> sync wait for all async to complete, register device
>
> you can get back to predictable ordering
Hm, so that sound like a case for a distinct setup() routine:
1. bus calls ->probe(), which return synchronously
2. bus calls ->probe_async() for all devices (optional)
3. bus waits for all probes to finish
4. bus calls ->setup() for all devices (which does the registering)
(->setup() can but need not be sync, although it should be for your
case)
Note that ordering is not guaranteed on hotpluggable busses anyway, and
if you use ->setup() as a function that may or may not be called later
on, there's no ordering guarantee either. (Unless the bus/device driver
implements something to that effect, or you have udev rules in place.)
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 17:07 ` Cornelia Huck
@ 2007-05-09 17:09 ` david
2007-05-09 17:48 ` Cornelia Huck
0 siblings, 1 reply; 55+ messages in thread
From: david @ 2007-05-09 17:09 UTC (permalink / raw)
To: Cornelia Huck
Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel, Duncan Sands
On Wed, 9 May 2007, Cornelia Huck wrote:
> On Wed, 9 May 2007 09:18:10 -0700 (PDT),
> david@lang.hm wrote:
>
>> I'm not worried about notification to userspace, I'm worried about devices
>> getting registered during the async stage appearing in different orders on
>> different boots due to different timeings.
>
> Ah, now I think I understand what you mean. You're talking about stuff
> like block devices or net devices, right?
they will work as examples, but I am not sure that that is the limit of
this sort of problem
>>
>> if you can identify the device well enough to register it quickly then the
>> approach that Linus proposed works well, which is
>>
>> sync probe, identify devices, register devices
>> async initialize devices
>> wait for all async to complete
>>
>> however I'm talking about cases where you can't fully identify the devices
>> (at least not well enough to register them with the kernel) and am saying
>> that doing
>>
>> sync probe
>> async initialize device, register device
>> wait for all async to complete
>>
>> results in unpredictable device ordering, but if instead you did
>>
>> sync probe
>> async initialize device
>> sync wait for all async to complete, register device
>>
>> you can get back to predictable ordering
>
> Hm, so that sound like a case for a distinct setup() routine:
>
> 1. bus calls ->probe(), which return synchronously
> 2. bus calls ->probe_async() for all devices (optional)
> 3. bus waits for all probes to finish
> 4. bus calls ->setup() for all devices (which does the registering)
this is exactly what I've been trying to describe.
> (->setup() can but need not be sync, although it should be for your
> case)
if it's not sync then you have race conditions again
> Note that ordering is not guaranteed on hotpluggable busses anyway, and
> if you use ->setup() as a function that may or may not be called later
> on, there's no ordering guarantee either. (Unless the bus/device driver
> implements something to that effect, or you have udev rules in place.)
correct.
David Lang
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 17:09 ` david
@ 2007-05-09 17:48 ` Cornelia Huck
2007-05-09 17:53 ` david
0 siblings, 1 reply; 55+ messages in thread
From: Cornelia Huck @ 2007-05-09 17:48 UTC (permalink / raw)
To: david; +Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel, Duncan Sands
On Wed, 9 May 2007 10:09:33 -0700 (PDT),
david@lang.hm wrote:
> > Hm, so that sound like a case for a distinct setup() routine:
> >
> > 1. bus calls ->probe(), which return synchronously
> > 2. bus calls ->probe_async() for all devices (optional)
> > 3. bus waits for all probes to finish
> > 4. bus calls ->setup() for all devices (which does the registering)
>
> this is exactly what I've been trying to describe.
Nearly, but with a slightly different spin...
>
> > (->setup() can but need not be sync, although it should be for your
> > case)
>
> if it's not sync then you have race conditions again
...but not all busses will care. If your bus wants to enforce ordering,
it must enforce it to be sync. If your bus allows hotplug and calling
setup() later on, it may also allow async. (setup() is a bit
dual-purpose in this idea.)
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 17:48 ` Cornelia Huck
@ 2007-05-09 17:53 ` david
2007-05-09 18:36 ` Cornelia Huck
0 siblings, 1 reply; 55+ messages in thread
From: david @ 2007-05-09 17:53 UTC (permalink / raw)
To: Cornelia Huck
Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel, Duncan Sands
On Wed, 9 May 2007, Cornelia Huck wrote:
> On Wed, 9 May 2007 10:09:33 -0700 (PDT),
> david@lang.hm wrote:
>
>>> Hm, so that sound like a case for a distinct setup() routine:
>>>
>>> 1. bus calls ->probe(), which return synchronously
>>> 2. bus calls ->probe_async() for all devices (optional)
>>> 3. bus waits for all probes to finish
>>> 4. bus calls ->setup() for all devices (which does the registering)
>>
>> this is exactly what I've been trying to describe.
>
> Nearly, but with a slightly different spin...
>
>>
>>> (->setup() can but need not be sync, although it should be for your
>>> case)
>>
>> if it's not sync then you have race conditions again
>
> ...but not all busses will care. If your bus wants to enforce ordering,
> it must enforce it to be sync. If your bus allows hotplug and calling
> setup() later on, it may also allow async. (setup() is a bit
> dual-purpose in this idea.)
if you want the registration to use hotplug and be async then do do the
registration during step 2 and make step 4 be a noop, in fact some drivers
may do all their work in step 2, while others (everything currently) will
do all their work during step 1
David Lang
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 17:53 ` david
@ 2007-05-09 18:36 ` Cornelia Huck
2007-05-09 18:52 ` david
0 siblings, 1 reply; 55+ messages in thread
From: Cornelia Huck @ 2007-05-09 18:36 UTC (permalink / raw)
To: david; +Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel, Duncan Sands
On Wed, 9 May 2007 10:53:52 -0700 (PDT),
david@lang.hm wrote:
> if you want the registration to use hotplug and be async then do do the
> registration during step 2 and make step 4 be a noop, in fact some drivers
> may do all their work in step 2, while others (everything currently) will
> do all their work during step 1
The added benefit of setup() I was thinking of was being able to
enable/disable devices later on (where enable entails the "heavy
lifting").
Say you have a range of devices you only use seldomly for some special
purpose. They may take a long time to get up and use a lot of resources
when they're on. You don't want to slow the boot process down for them,
so they start disabled with just very basic setup (the fast stuff)
done. (It is also useful if you want some manual configuration between
the fast stuff and the heavy lifting.) When you need them, you spin
them up (may take some time) and then use them. Afterwards, you could
spin them down again. (This is what the ccw/ccwgroup/ap bus online
attribute does today.)
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 18:36 ` Cornelia Huck
@ 2007-05-09 18:52 ` david
2007-05-10 7:38 ` Cornelia Huck
0 siblings, 1 reply; 55+ messages in thread
From: david @ 2007-05-09 18:52 UTC (permalink / raw)
To: Cornelia Huck
Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel, Duncan Sands
On Wed, 9 May 2007, Cornelia Huck wrote:
> On Wed, 9 May 2007 10:53:52 -0700 (PDT),
> david@lang.hm wrote:
>
>> if you want the registration to use hotplug and be async then do do the
>> registration during step 2 and make step 4 be a noop, in fact some drivers
>> may do all their work in step 2, while others (everything currently) will
>> do all their work during step 1
>
> The added benefit of setup() I was thinking of was being able to
> enable/disable devices later on (where enable entails the "heavy
> lifting").
>
> Say you have a range of devices you only use seldomly for some special
> purpose. They may take a long time to get up and use a lot of resources
> when they're on. You don't want to slow the boot process down for them,
> so they start disabled with just very basic setup (the fast stuff)
> done. (It is also useful if you want some manual configuration between
> the fast stuff and the heavy lifting.) When you need them, you spin
> them up (may take some time) and then use them. Afterwards, you could
> spin them down again. (This is what the ccw/ccwgroup/ap bus online
> attribute does today.)
this sounds like something that you want to load the driver module for
when you want to use it and unload the driver module when you are done.
if you want to make a seperate enable/disable function I think it would
probably be more appropriate to use the power management functions.
David Lang
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 18:52 ` david
@ 2007-05-10 7:38 ` Cornelia Huck
0 siblings, 0 replies; 55+ messages in thread
From: Cornelia Huck @ 2007-05-10 7:38 UTC (permalink / raw)
To: david; +Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel, Duncan Sands
On Wed, 9 May 2007 11:52:59 -0700 (PDT),
david@lang.hm wrote:
> this sounds like something that you want to load the driver module for
> when you want to use it and unload the driver module when you are done.
A common case is that you only want to use some of all your devices of
a certain model. (My view may be strongly influenced by the s390
perspective, where we only have a quite limited amount of different
models and device drivers. In the highly diverse PC world, module
load/unload may indeed be the better way, since it doesn't require
additional work.)
> if you want to make a seperate enable/disable function I think it would
> probably be more appropriate to use the power management functions.
It might. (s390 doesn't currently have power management.)
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 16:18 ` david
2007-05-09 17:07 ` Cornelia Huck
@ 2007-05-09 17:07 ` Greg KH
2007-05-09 17:25 ` Linus Torvalds
1 sibling, 1 reply; 55+ messages in thread
From: Greg KH @ 2007-05-09 17:07 UTC (permalink / raw)
To: david
Cc: Cornelia Huck, Linus Torvalds, Adrian Bunk, linux-kernel,
Duncan Sands
On Wed, May 09, 2007 at 09:18:10AM -0700, david@lang.hm wrote:
> On Wed, 9 May 2007, Cornelia Huck wrote:
>
> > On Wed, 9 May 2007 02:25:10 -0700 (PDT),
> > david@lang.hm wrote:
> >
> >> makeing up an example
> >>
> >> today the process would be
> >> call driver X to initialize it's devices.
> >>
> >> driver X follows these steps
> >>
> >> 1. see if a compatible chipset/device appears to be available
> >>
> >> 2. initialize the chipset/device (loading firmware)
> >>
> >> 3. query the now partially initialized chipset/device to see what specific
> >> options are enabled in the hardware
> >>
> >> 4. for each 'thing' that's enabled in hardware, initialize and register
> >> it.
> >
> > The sync ->probe() could return after step 1 (when it is clear the
> > driver wants to bind to the device). The remainder should be done in
> > the async function, so that the bus could go on with other devices. The
> > driver should then signal when it is done with its registrations.
>
> but then won't the devices get registered in a random order? (i.e. whenever
> the async portion finishes the probing and finds the details of what there
> is to register)
So? We have busses today that have devices get registered in random
order, our userspace tools can handle it just fine now.
> >> to acommodate both of these device models it seems to me that you can't
> >> defice either
> >> async followed by sync
> >> or
> >> sync followed by async
> >> but instead need to support the combined
> >> sync followed by async followed by sync.
> >>
> >> now, it's very possible that I'm mistaken and one of the two-part models
> >> can be used for everything, if so then it's definantly simpler to do so.
> >
> > Hm, maybe I'm dense, but I really don't understand the problem here.
> > Why should (optional) sync followed by (optional) async followed by bus
> > synchonization not cover all cases? The async portion can also do some
> > "sync" stuff and then signal completion, can't it?
> >
> > If you're worried about notifications to userspace, you could first
> > suppress the add uevent via uevent_suppress and then generate it when
> > you signal completion to the bus.
>
> I'm not worried about notification to userspace, I'm worried about devices
> getting registered during the async stage appearing in different orders on
> different boots due to different timeings.
>
> if you can identify the device well enough to register it quickly then the
> approach that Linus proposed works well, which is
>
> sync probe, identify devices, register devices
> async initialize devices
> wait for all async to complete
>
> however I'm talking about cases where you can't fully identify the devices
> (at least not well enough to register them with the kernel) and am saying
> that doing
>
> sync probe
> async initialize device, register device
> wait for all async to complete
>
> results in unpredictable device ordering, but if instead you did
>
> sync probe
> async initialize device
> sync wait for all async to complete, register device
>
> you can get back to predictable ordering
No, don't worry about that, the ordering can be made different due to
any number of different things (bios upgrade, new device added, phase of
the moon, etc.) So stop worrying about being predictable, we have never
really had that...
thanks,
greg k-h
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 17:07 ` Greg KH
@ 2007-05-09 17:25 ` Linus Torvalds
0 siblings, 0 replies; 55+ messages in thread
From: Linus Torvalds @ 2007-05-09 17:25 UTC (permalink / raw)
To: Greg KH; +Cc: david, Cornelia Huck, Adrian Bunk, linux-kernel, Duncan Sands
On Wed, 9 May 2007, Greg KH wrote:
> >
> > but then won't the devices get registered in a random order? (i.e. whenever
> > the async portion finishes the probing and finds the details of what there
> > is to register)
>
> So? We have busses today that have devices get registered in random
> order, our userspace tools can handle it just fine now.
However, that does *not* translate to: "..so we can do it for all buses".
People *do* depend on simple things like internal harddisks showing up in
a particular order. The fact that you *can* handle it with UUID's etc does
not mean that people do, or that they should be forced to do so.
There is no inherent goodness in making the bootup more random. In fact,
there's a lot of badness to it, ranging from just not being very nice to
"really a bitch to debug". And no amount of "user level _could_ handle it"
changes that.
Linus
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 9:15 ` Cornelia Huck
2007-05-09 9:25 ` david
@ 2007-05-09 9:30 ` Stefan Richter
1 sibling, 0 replies; 55+ messages in thread
From: Stefan Richter @ 2007-05-09 9:30 UTC (permalink / raw)
To: Cornelia Huck
Cc: david, Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel,
Duncan Sands
Cornelia Huck wrote:
> On Wed, 9 May 2007 01:33:14 -0700 (PDT),
> david@lang.hm wrote:
>
>> 1. why should different, unrelated busses need to wait for each other?
>> picking two, why can't you have SCSI and USB going through their timeouts
>> at the same time?
>
> If they don't have dependencies on each other, yes. Some busses should
> be finished before probing for others start (e.g. low-level busses).
If there are dependencies, then a driver on the lower-level bus simply
hot-inserts a bus to the higher-level bus's driver when the lower-level
probe is done.
E.g. PCI or CardBus detects insertion of a IEEE 1394 controller,
ohci1394 adds a new IEEE 1394 bus to ieee1394 core, ieee1394core
discovers e.g. storage devices on this bus and kicks off sbp2's probe,
sbp2 adds them to SCSI core, SCSI core does the inquiry and kicks off a
SCSI command set driver's probe. Meanwhile, userspace just waits for a
certain SCSI device to appear. (And userspace inserts drivers according
to uevents a.k.a. hotplug events if the drivers weren't statically linked.)
--
Stefan Richter
-=====-=-=== -=-= -=--=
http://arcgraph.de/sr/
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-08 20:26 ` david
2007-05-09 7:58 ` Cornelia Huck
@ 2007-05-09 22:21 ` Phillip Susi
2007-05-09 22:37 ` Stefan Richter
1 sibling, 1 reply; 55+ messages in thread
From: Phillip Susi @ 2007-05-09 22:21 UTC (permalink / raw)
To: david; +Cc: Linus Torvalds, Cornelia Huck, Adrian Bunk, Greg K-H,
linux-kernel
david@lang.hm wrote:
> is it really enough to do the sync followed by the async piece?
>
> it seems to me that there may be a need for three steps
>
> 1. prep (sync, what we do today)
> which will all complete before
> 2. parallel (async)
> which will all complete before
> 3. cleanup (sync)
I agree. Considering the particular case of scsi disks, you want to do
the long INQUIRY command to probe each target in phase 2, so that this
time consuming step can be done in parallel between multiple scsi
busses. After the INQUIRY though, you have to register with the scsi
layer and assign it a minor number and so on, and this you want to do
with some predictability, so you need to do this in phase 3.
Take a hypothetical system containing a PCI bus with a video card and
two scsi cards in it, and each scsi bus has a hard disk on it. The
sequence would go:
1) PCI Bus invokes phase 1 on video card, scsi a, scsi b, in that order
2) PCI Bus invokes phase 2 on video card, scsi a, scsi b, each in their
own thread
2.1) video card uploads firmware
2.2) scsi bus A issues INQUERY commands to each target to detect devices
2.3) scsi bus B issues INQUERY commands to each target to detect devices
3) PCI Bus invokes phase 3 on video card, scsi a, scsi b, in that order
3.1) video card registers with the frame buffer layer
3.2) scsi bus A registers its detected disk with the scsi layer
3.3) scsi bus B registers its detected disk with the scsi layer
Because 3.2 and 3.3 always happen in order, /dev/sda will stay /dev/sda
even if 2.3 finishes before 2.2. Likewise if you add a new video card
to PCI slot 4 which does not require a firmware upload, it will not
displace the existing video card from /dev/fb0 because phase 3 will take
place for the first card first, even if phase 2 completed first on the
new card.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 22:21 ` Phillip Susi
@ 2007-05-09 22:37 ` Stefan Richter
2007-05-10 14:23 ` Phillip Susi
0 siblings, 1 reply; 55+ messages in thread
From: Stefan Richter @ 2007-05-09 22:37 UTC (permalink / raw)
To: Phillip Susi
Cc: david, Linus Torvalds, Cornelia Huck, Adrian Bunk, Greg K-H,
linux-kernel
Phillip Susi wrote:
> Considering the particular case of scsi disks, you want to do
> the long INQUIRY command to probe each target in phase 2, so that this
> time consuming step can be done in parallel between multiple scsi
> busses.
The SCSI stack already has infrastructure for multi-threaded discovery
and probing.
--
Stefan Richter
-=====-=-=== -=-= -=-=-
http://arcgraph.de/sr/
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 22:37 ` Stefan Richter
@ 2007-05-10 14:23 ` Phillip Susi
2007-05-10 14:55 ` Stefan Richter
0 siblings, 1 reply; 55+ messages in thread
From: Phillip Susi @ 2007-05-10 14:23 UTC (permalink / raw)
To: Stefan Richter
Cc: david, Linus Torvalds, Cornelia Huck, Adrian Bunk, Greg K-H,
linux-kernel
Stefan Richter wrote:
> The SCSI stack already has infrastructure for multi-threaded discovery
> and probing.
So? It would still benefit from using a generic framework that other
buses can use as well. Not to mention that the current scsi specific
framework tends to cause unstable names doesn't it?
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-10 14:23 ` Phillip Susi
@ 2007-05-10 14:55 ` Stefan Richter
2007-05-11 7:22 ` Cornelia Huck
0 siblings, 1 reply; 55+ messages in thread
From: Stefan Richter @ 2007-05-10 14:55 UTC (permalink / raw)
To: Phillip Susi
Cc: david, Linus Torvalds, Cornelia Huck, Adrian Bunk, Greg K-H,
linux-kernel
Phillip Susi wrote:
> Stefan Richter wrote:
>> The SCSI stack already has infrastructure for multi-threaded discovery
>> and probing.
>
> So? It would still benefit from using a generic framework that other
> buses can use as well.
Perhaps, perhaps not. Many details of the If and How of asynchronous,
parallelized probing rest with the low-level drivers.
[BTW, which ever team attempts to design this generic framework please
brings in detailed knowledge of a variety of bus architectures. I for
one would like to contribute with what I know about IEEE 1394, but
before that I still have to experiment on my own before I have a good
understanding of how to parallelize IEEE 1394 scanning and probing, and
the IEEE 1394 core is being radically reworked at the moment anyway.]
> Not to mention that the current scsi specific
> framework tends to cause unstable names doesn't it?
No. Reality causes "unstable" names. Or more precisely, we ultimately
can't get persistent names from dumb enumerations according to probing
order. We get persistent names by reading persistent device properties
in userspace and by letting userspace rename or give aliases to devices.
Names based on probing order are fine for really simplistic setups like
the famous Aunt Tilly's home computer --- but not for the general case
that you are trying to cover.
(Sorry for repeating what has been said before in this thread.)
--
Stefan Richter
-=====-=-=== -=-= -=-=-
http://arcgraph.de/sr/
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-10 14:55 ` Stefan Richter
@ 2007-05-11 7:22 ` Cornelia Huck
0 siblings, 0 replies; 55+ messages in thread
From: Cornelia Huck @ 2007-05-11 7:22 UTC (permalink / raw)
To: Stefan Richter
Cc: Phillip Susi, david, Linus Torvalds, Adrian Bunk, Greg K-H,
linux-kernel
On Thu, 10 May 2007 16:55:54 +0200,
Stefan Richter <stefanr@s5r6.in-berlin.de> wrote:
> Phillip Susi wrote:
> > Stefan Richter wrote:
> >> The SCSI stack already has infrastructure for multi-threaded discovery
> >> and probing.
> >
> > So? It would still benefit from using a generic framework that other
> > buses can use as well.
>
> Perhaps, perhaps not. Many details of the If and How of asynchronous,
> parallelized probing rest with the low-level drivers.
A mix of bus/driver parallelism would probably be the most flexible
approach.
> [BTW, which ever team attempts to design this generic framework please
> brings in detailed knowledge of a variety of bus architectures. I for
> one would like to contribute with what I know about IEEE 1394, but
> before that I still have to experiment on my own before I have a good
> understanding of how to parallelize IEEE 1394 scanning and probing, and
> the IEEE 1394 core is being radically reworked at the moment anyway.]
I guess I'm too tainted by s390 :/ (which in comparison provides a
quite unified way at probing), but I'd be happy to contibute my
experiences as well.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-08 20:01 ` Linus Torvalds
2007-05-08 20:26 ` david
@ 2007-05-08 20:51 ` Cornelia Huck
2007-05-08 21:41 ` David Miller
2007-05-09 8:14 ` Duncan Sands
3 siblings, 0 replies; 55+ messages in thread
From: Cornelia Huck @ 2007-05-08 20:51 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Adrian Bunk, Greg K-H, linux-kernel
On Tue, 8 May 2007 13:01:21 -0700 (PDT),
Linus Torvalds <torvalds@linux-foundation.org> wrote:
> Instead of changing existign probe functionality to be asynchronous, we
> could *add* a new and asynchronous part to it. For example, we could make
> the rule for PCI - or other bus - devices be:
>
> - the bus will *first* call the "probe()" function synchronously.
>
> - after that one has completed, we will call "probe_async()"
> asynchronously at some point (it ie might be scheduled immediately
> after the "probe()" call, but delayed by some arbitrary issues like
> just already having too many asynchronous probes on-going or similar)
Hm. This would mean that probe() would be the decision "I want to use
this device", while probe_async() would be "setup my device, may take
some time"? Could work.
> So an unmodified driver would basically work exactly like it does now, but
> if a driver is happy with being called asynchronously, it could just
> change it's
>
> .probe = mydriver_probe
>
> thing into a
>
> .probe_async = mydriver_probe
>
> and we can do that ona per-driver basis with that kind of really simple
> one-liner change.
But probe would return int, while probe_async would return void? Two
liner :)
>
> In fact, there is nothing wrong with having *both* a synchronous part, and
> an async part:
>
> .probe = mydriver_setup,
> .probe_async = mydriver_spin_up_and_probe_devices,
>
> and it would do basic setup (including, for example, the fast enumeration
> of whatever devices are connected) synchronously, but then do anything
> more in the async part - and the async part would still be guaranteed that
> the setup has been run by the time it is scheduled (but not really have
> any other guarantees).
Looks like an idea.
I'll sleep on it, just too tired to make sense.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-08 20:01 ` Linus Torvalds
2007-05-08 20:26 ` david
2007-05-08 20:51 ` Cornelia Huck
@ 2007-05-08 21:41 ` David Miller
2007-05-09 9:55 ` Greg KH
2007-05-09 8:14 ` Duncan Sands
3 siblings, 1 reply; 55+ messages in thread
From: David Miller @ 2007-05-08 21:41 UTC (permalink / raw)
To: torvalds; +Cc: cornelia.huck, bunk, greg, linux-kernel
From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Tue, 8 May 2007 13:01:21 -0700 (PDT)
> In fact, there is nothing wrong with having *both* a synchronous part, and
> an async part:
>
> .probe = mydriver_setup,
> .probe_async = mydriver_spin_up_and_probe_devices,
...
> Hmm? Would something like this work? I dunno, but it seems a hell of a lot
> safer and more capable than the aborted PCI multithreaded probing that was
> an "all or nothing" approach.
I definitely agree that we need a transitonary approach to this.
Although I kind of preferred the idea you mentioned where the
device could launch the asynchronous probe and just return from
the normal ->probe() immediately.
This might get tricky if the callers do some kind of reference
counting or other resource management based upon the ->probe()
return value since it wouldn't know what happened to the
launched asynchronous probe when it returns from ->probe().
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-08 21:41 ` David Miller
@ 2007-05-09 9:55 ` Greg KH
0 siblings, 0 replies; 55+ messages in thread
From: Greg KH @ 2007-05-09 9:55 UTC (permalink / raw)
To: David Miller; +Cc: torvalds, cornelia.huck, bunk, linux-kernel
On Tue, May 08, 2007 at 02:41:54PM -0700, David Miller wrote:
> From: Linus Torvalds <torvalds@linux-foundation.org>
> Date: Tue, 8 May 2007 13:01:21 -0700 (PDT)
>
> > In fact, there is nothing wrong with having *both* a synchronous part, and
> > an async part:
> >
> > .probe = mydriver_setup,
> > .probe_async = mydriver_spin_up_and_probe_devices,
> ...
> > Hmm? Would something like this work? I dunno, but it seems a hell of a lot
> > safer and more capable than the aborted PCI multithreaded probing that was
> > an "all or nothing" approach.
>
> I definitely agree that we need a transitonary approach to this.
>
> Although I kind of preferred the idea you mentioned where the
> device could launch the asynchronous probe and just return from
> the normal ->probe() immediately.
Yes, let this be a decision the individual PCI driver does, I don't want
to put this two-stage thing in the driver core, but any individual bus
can implement it if they really want to.
> This might get tricky if the callers do some kind of reference
> counting or other resource management based upon the ->probe()
> return value since it wouldn't know what happened to the
> launched asynchronous probe when it returns from ->probe().
As long as the ->probe() call returns that the driver has clamed the
device, and the ->remove() call can be handled properly while the driver
is off doing whatever it wants to in the initialization, the driver core
should work just fine, no changes needed.
thanks,
greg k-h
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-08 20:01 ` Linus Torvalds
` (2 preceding siblings ...)
2007-05-08 21:41 ` David Miller
@ 2007-05-09 8:14 ` Duncan Sands
2007-05-09 8:45 ` Cornelia Huck
3 siblings, 1 reply; 55+ messages in thread
From: Duncan Sands @ 2007-05-09 8:14 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Cornelia Huck, Adrian Bunk, Greg K-H, linux-kernel
Hi,
> Instead of changing existign probe functionality to be asynchronous, we
> could *add* a new and asynchronous part to it. For example, we could make
> the rule for PCI - or other bus - devices be:
>
> - the bus will *first* call the "probe()" function synchronously.
>
> - after that one has completed, we will call "probe_async()"
> asynchronously at some point (it ie might be scheduled immediately
> after the "probe()" call, but delayed by some arbitrary issues like
> just already having too many asynchronous probes on-going or similar)
the usbatm USB ADSL modem drivers have this functionality. These drivers
need to load firmware before they become useful. The natural place to do
this is in the probe() method, but because firmware loading can take quite
some time (10 seconds, or even an infinite amount of time if the firmware
is not available and the timeout has been turned off) and would block the
USB hub thread if done from probe(), it's done in a separate kernel thread.
Clients of usbatm, like the speedtch driver, register themselves with usbatm,
providing a "bind" and a "heavy_init" method. "bind" is like probe(), while
"heavy_init" is like probe_async(). First bind is called, and if successful
and heavy_init has been defined, then heavy_init is run in its own thread.
If the device is unplugged, usbatm takes care of making sure that the heavy_init
thread has stopped before calling unbind and destroying device related structures.
A bunch of other USB drivers could do with similar functionality for the
same reason (slow probe), and there was some discussion about generalizing
this functionality to the USB layer but I didn't find time to do anything
about it yet. See http://marc.info/?l=linux-usb-devel&m=116551653026075&w=2
Best wishes,
Duncan.
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 8:14 ` Duncan Sands
@ 2007-05-09 8:45 ` Cornelia Huck
2007-05-09 9:16 ` Duncan Sands
0 siblings, 1 reply; 55+ messages in thread
From: Cornelia Huck @ 2007-05-09 8:45 UTC (permalink / raw)
To: Duncan Sands; +Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel
On Wed, 9 May 2007 10:14:16 +0200,
Duncan Sands <duncan.sands@math.u-psud.fr> wrote:
> the usbatm USB ADSL modem drivers have this functionality. These drivers
> need to load firmware before they become useful. The natural place to do
> this is in the probe() method, but because firmware loading can take quite
> some time (10 seconds, or even an infinite amount of time if the firmware
> is not available and the timeout has been turned off) and would block the
> USB hub thread if done from probe(), it's done in a separate kernel thread.
> Clients of usbatm, like the speedtch driver, register themselves with usbatm,
> providing a "bind" and a "heavy_init" method. "bind" is like probe(), while
> "heavy_init" is like probe_async(). First bind is called, and if successful
> and heavy_init has been defined, then heavy_init is run in its own thread.
> If the device is unplugged, usbatm takes care of making sure that the heavy_init
> thread has stopped before calling unbind and destroying device related structures.
>
> A bunch of other USB drivers could do with similar functionality for the
> same reason (slow probe), and there was some discussion about generalizing
> this functionality to the USB layer but I didn't find time to do anything
> about it yet. See http://marc.info/?l=linux-usb-devel&m=116551653026075&w=2
Would a general split between probe() (check if we can handle the
device, do very basic stuff) and setup() (get the device up and
running) make sense? Drivers could stay with today's probe() function
if they want to (and still be working). setup() could be, but need not
be async. As an added benefit for huge systems, setup() might only be
called if explicitly requested (like "only do this heavy lifting
if/when we really want to use the device"). probe() could call bind()
and setup() (doing the firmware load etc.) heavy_init(). (This may be
orthogonal to the probe()/probe_async() idea.)
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 8:45 ` Cornelia Huck
@ 2007-05-09 9:16 ` Duncan Sands
2007-05-09 12:37 ` Cornelia Huck
0 siblings, 1 reply; 55+ messages in thread
From: Duncan Sands @ 2007-05-09 9:16 UTC (permalink / raw)
To: Cornelia Huck; +Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel
Hi Cornelia,
> > the usbatm USB ADSL modem drivers have this functionality. These drivers
> > need to load firmware before they become useful. The natural place to do
> > this is in the probe() method, but because firmware loading can take quite
> > some time (10 seconds, or even an infinite amount of time if the firmware
> > is not available and the timeout has been turned off) and would block the
> > USB hub thread if done from probe(), it's done in a separate kernel thread.
> > Clients of usbatm, like the speedtch driver, register themselves with usbatm,
> > providing a "bind" and a "heavy_init" method. "bind" is like probe(), while
> > "heavy_init" is like probe_async(). First bind is called, and if successful
> > and heavy_init has been defined, then heavy_init is run in its own thread.
> > If the device is unplugged, usbatm takes care of making sure that the heavy_init
> > thread has stopped before calling unbind and destroying device related structures.
> >
> > A bunch of other USB drivers could do with similar functionality for the
> > same reason (slow probe), and there was some discussion about generalizing
> > this functionality to the USB layer but I didn't find time to do anything
> > about it yet. See http://marc.info/?l=linux-usb-devel&m=116551653026075&w=2
>
> Would a general split between probe() (check if we can handle the
> device, do very basic stuff) and setup() (get the device up and
> running) make sense? Drivers could stay with today's probe() function
> if they want to (and still be working). setup() could be, but need not
> be async. As an added benefit for huge systems, setup() might only be
> called if explicitly requested (like "only do this heavy lifting
> if/when we really want to use the device"). probe() could call bind()
> and setup() (doing the firmware load etc.) heavy_init(). (This may be
> orthogonal to the probe()/probe_async() idea.)
yes that would be fine. Like with your setup() suggestion, it is possible
to specify whether heavy_init should be called or not. This is useful if
the firmware has already been uploaded - the probe() can detect this and
signal that heavy_init is not needed.
The main difficulties encountered with usbatm were: (1) avoiding race
conditions on device disconnect; (2) providing a way to cancel heavy_init.
As mentioned in my original email, heavy_init could run forever depending
on how the system is configured. In the current implementation a signal
is sent on disconnect(), to indicate that the game is up, then disconnect()
blocks until heavy_init terminates, if it hasn't already. A better method
would be to allow registration of a cancel() routine, which disconnect
would call before blocking on heavy_init termination. This could send a
signal, or complete a completion, or whatever. Sadly, the generic firmware
loading infrastructure doesn't have a way of asynchronously cancelling a
firmware load, so right now cancellation only does something if the firmware
has already been loaded from disk, in which case it cancels the upload to the
modem (which is the most time consuming part).
Best wishes,
Duncan.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: Please revert 5adc55da4a7758021bcc374904b0f8b076508a11 (PCI_MULTITHREAD_PROBE)
2007-05-09 9:16 ` Duncan Sands
@ 2007-05-09 12:37 ` Cornelia Huck
0 siblings, 0 replies; 55+ messages in thread
From: Cornelia Huck @ 2007-05-09 12:37 UTC (permalink / raw)
To: Duncan Sands; +Cc: Linus Torvalds, Adrian Bunk, Greg K-H, linux-kernel
On Wed, 9 May 2007 11:16:06 +0200,
Duncan Sands <duncan.sands@math.u-psud.fr> wrote:
> The main difficulties encountered with usbatm were: (1) avoiding race
> conditions on device disconnect; (2) providing a way to cancel heavy_init.
> As mentioned in my original email, heavy_init could run forever depending
> on how the system is configured. In the current implementation a signal
> is sent on disconnect(), to indicate that the game is up, then disconnect()
> blocks until heavy_init terminates, if it hasn't already. A better method
> would be to allow registration of a cancel() routine, which disconnect
> would call before blocking on heavy_init termination. This could send a
> signal, or complete a completion, or whatever.
Yes, a cancel routine would make sense also in the generic
probe()/setup() case.
> Sadly, the generic firmware
> loading infrastructure doesn't have a way of asynchronously cancelling a
> firmware load, so right now cancellation only does something if the firmware
> has already been loaded from disk, in which case it cancels the upload to the
> modem (which is the most time consuming part).
Hm, a firmware load cancellation interface looks like a worthwile
addition to the firmware code (especially since fw_load_abort() already
exists and can even be triggered by userspace).
^ permalink raw reply [flat|nested] 55+ messages in thread