Re: Remove scsi_wait_scan module

Re: Remove scsi_wait_scan module

[maximilian attems]

On Sun, May 27, 2012 at 10:13:46AM +0100, James Bottomley wrote:
> scsi_wait_scan was introduced with asynchronous host scanning as a hack
> for distributions that weren't using proper udev based wait for root to
> appear in their initramfs scripts.  In 2.6.30 Commit 
 
> c751085943362143f84346d274e0011419c84202
> Author: Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org>
> Date:   Sun Apr 12 20:06:56 2009 +0200
> 
>     PM/Hibernate: Wait for SCSI devices scan to complete during resume
> 
> Actually broke scsi_wait_scan because it renders
> scsi_complete_async_scans() a nop for modular SCSI if you include
> scsi_scans.h (which this module does).
> 
> The lack of bug reports is sufficient proof that this module is no
> longer used.

We do use it in initramfs-tools.

There is quite a number of bug reports moaning about having to boot with
`scsi_mod.scan=sync'. I didn't pass them on, because I didn't knew that
the module itself got broken, for example:
http://bugs.debian.org/616689

-- 
maks

Re: Remove scsi_wait_scan module

[James Bottomley]

On Mon, 2012-05-28 at 10:00 +0000, maximilian attems wrote:
> On Sun, May 27, 2012 at 10:13:46AM +0100, James Bottomley wrote:
> > scsi_wait_scan was introduced with asynchronous host scanning as a hack
> > for distributions that weren't using proper udev based wait for root to
> > appear in their initramfs scripts.  In 2.6.30 Commit 
>  
> > c751085943362143f84346d274e0011419c84202
> > Author: Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org>
> > Date:   Sun Apr 12 20:06:56 2009 +0200
> > 
> >     PM/Hibernate: Wait for SCSI devices scan to complete during resume
> > 
> > Actually broke scsi_wait_scan because it renders
> > scsi_complete_async_scans() a nop for modular SCSI if you include
> > scsi_scans.h (which this module does).
> > 
> > The lack of bug reports is sufficient proof that this module is no
> > longer used.
> 
> We do use it in initramfs-tools.
> 
> There is quite a number of bug reports moaning about having to boot with
> `scsi_mod.scan=sync'. I didn't pass them on, because I didn't knew that
> the module itself got broken, for example:
> http://bugs.debian.org/616689

OK, so what these bugs show is the breakage ... basically scsi_wait_scan
isn't really waiting for the scans to complete.  I can fix it in stable
so you can close your bug reports, but if I do, can you also transition
away from using it so I can remove it in 3.5?

James

Re: Remove scsi_wait_scan module

[maximilian attems]

On Mon, May 28, 2012 at 04:07:46PM +0400, James Bottomley wrote:
> On Mon, 2012-05-28 at 10:00 +0000, maximilian attems wrote:
> > 
> > There is quite a number of bug reports moaning about having to boot with
> > `scsi_mod.scan=sync'. I didn't pass them on, because I didn't knew that
> > the module itself got broken, for example:
> > http://bugs.debian.org/616689
> 
> OK, so what these bugs show is the breakage ... basically scsi_wait_scan
> isn't really waiting for the scans to complete.  I can fix it in stable
> so you can close your bug reports, but if I do, can you also transition
> away from using it so I can remove it in 3.5?
> 

Thank you very much for the patch, guess Ben will pick it for 3.2 via
stable and we'll add it for 2.6.32 too.

I'll have enough time for afterwards fix to have the udev wait root,
even if right now I'm a bit unsure what it means beyond adding a specific
rule?

-- 
maks

Re: Remove scsi_wait_scan module

[Dan Williams]

On Mon, May 28, 2012 at 5:07 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Mon, 2012-05-28 at 10:00 +0000, maximilian attems wrote:
>> On Sun, May 27, 2012 at 10:13:46AM +0100, James Bottomley wrote:
>> > scsi_wait_scan was introduced with asynchronous host scanning as a hack
>> > for distributions that weren't using proper udev based wait for root to
>> > appear in their initramfs scripts.  In 2.6.30 Commit
>>
>> > c751085943362143f84346d274e0011419c84202
>> > Author: Rafael J. Wysocki <rjw@sisk.pl>
>> > Date:   Sun Apr 12 20:06:56 2009 +0200
>> >
>> >     PM/Hibernate: Wait for SCSI devices scan to complete during resume
>> >
>> > Actually broke scsi_wait_scan because it renders
>> > scsi_complete_async_scans() a nop for modular SCSI if you include
>> > scsi_scans.h (which this module does).
>> >
>> > The lack of bug reports is sufficient proof that this module is no
>> > longer used.
>>
>> We do use it in initramfs-tools.
>>
>> There is quite a number of bug reports moaning about having to boot with
>> `scsi_mod.scan=sync'. I didn't pass them on, because I didn't knew that
>> the module itself got broken, for example:
>> http://bugs.debian.org/616689
>
> OK, so what these bugs show is the breakage ... basically scsi_wait_scan
> isn't really waiting for the scans to complete.  I can fix it in stable
> so you can close your bug reports, but if I do, can you also transition
> away from using it so I can remove it in 3.5?

Is there some other method whereby userspace can sync all driver
probing actions?

We won't need scsi_complete_async_scans() after:

  http://marc.info/?l=linux-scsi&m=133840132007532&w=2

...but won't initramfs environments still need a way to trigger
wait_for_device_probe()?  Something like echo "flush" >
/sys/devices/async_probe. and maybe reading that file indicates if
some async probing is still in-flight?

--
Dan
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Remove scsi_wait_scan module

[James Bottomley]

On Wed, 2012-05-30 at 11:26 -0700, Dan Williams wrote:
> On Mon, May 28, 2012 at 5:07 AM, James Bottomley
> <James.Bottomley-JuX6DAaQMKPCXq6kfMZ53/egYHeGw8Jk@public.gmane.org> wrote:
> > On Mon, 2012-05-28 at 10:00 +0000, maximilian attems wrote:
> >> On Sun, May 27, 2012 at 10:13:46AM +0100, James Bottomley wrote:
> >> > scsi_wait_scan was introduced with asynchronous host scanning as a hack
> >> > for distributions that weren't using proper udev based wait for root to
> >> > appear in their initramfs scripts.  In 2.6.30 Commit
> >>
> >> > c751085943362143f84346d274e0011419c84202
> >> > Author: Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org>
> >> > Date:   Sun Apr 12 20:06:56 2009 +0200
> >> >
> >> >     PM/Hibernate: Wait for SCSI devices scan to complete during resume
> >> >
> >> > Actually broke scsi_wait_scan because it renders
> >> > scsi_complete_async_scans() a nop for modular SCSI if you include
> >> > scsi_scans.h (which this module does).
> >> >
> >> > The lack of bug reports is sufficient proof that this module is no
> >> > longer used.
> >>
> >> We do use it in initramfs-tools.
> >>
> >> There is quite a number of bug reports moaning about having to boot with
> >> `scsi_mod.scan=sync'. I didn't pass them on, because I didn't knew that
> >> the module itself got broken, for example:
> >> http://bugs.debian.org/616689
> >
> > OK, so what these bugs show is the breakage ... basically scsi_wait_scan
> > isn't really waiting for the scans to complete.  I can fix it in stable
> > so you can close your bug reports, but if I do, can you also transition
> > away from using it so I can remove it in 3.5?
> 
> Is there some other method whereby userspace can sync all driver
> probing actions?

No,  but then there never really was.  The theory is you know all the
disks you need (/ /usr and so on) and you just wait for them to appear
before mounting them and proceeding with boot.

> We won't need scsi_complete_async_scans() after:
> 
>   http://marc.info/?l=linux-scsi&m=133840132007532&w=2
> 
> ...but won't initramfs environments still need a way to trigger
> wait_for_device_probe()?  Something like echo "flush" >
> /sys/devices/async_probe. and maybe reading that file indicates if
> some async probing is still in-flight?

Why?  The job of an initramfs is to mount root.  All it has to do is
wait for root to appear via udev and then proceed.  The whole reason for
doing stuff async initially was to speed boot, so probing can still be
ongoing even after the initrd exits.

If you think about it, most modern fabrics are hot plug.  Just because
the initial scan has completed there's no guarantee that all the devices
have appeared yet.

James

Re: Remove scsi_wait_scan module

[Dan Williams]

On Wed, May 30, 2012 at 4:32 PM, James Bottomley
<James.Bottomley-JuX6DAaQMKPCXq6kfMZ53/egYHeGw8Jk@public.gmane.org> wrote:
> On Wed, 2012-05-30 at 11:26 -0700, Dan Williams wrote:
>> On Mon, May 28, 2012 at 5:07 AM, James Bottomley
>> <James.Bottomley-JuX6DAaQMKPCXq6kfMZ53/egYHeGw8Jk@public.gmane.org> wrote:
>> > On Mon, 2012-05-28 at 10:00 +0000, maximilian attems wrote:
>> >> On Sun, May 27, 2012 at 10:13:46AM +0100, James Bottomley wrote:
>> >> > scsi_wait_scan was introduced with asynchronous host scanning as a hack
>> >> > for distributions that weren't using proper udev based wait for root to
>> >> > appear in their initramfs scripts.  In 2.6.30 Commit
>> >>
>> >> > c751085943362143f84346d274e0011419c84202
>> >> > Author: Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org>
>> >> > Date:   Sun Apr 12 20:06:56 2009 +0200
>> >> >
>> >> >     PM/Hibernate: Wait for SCSI devices scan to complete during resume
>> >> >
>> >> > Actually broke scsi_wait_scan because it renders
>> >> > scsi_complete_async_scans() a nop for modular SCSI if you include
>> >> > scsi_scans.h (which this module does).
>> >> >
>> >> > The lack of bug reports is sufficient proof that this module is no
>> >> > longer used.
>> >>
>> >> We do use it in initramfs-tools.
>> >>
>> >> There is quite a number of bug reports moaning about having to boot with
>> >> `scsi_mod.scan=sync'. I didn't pass them on, because I didn't knew that
>> >> the module itself got broken, for example:
>> >> http://bugs.debian.org/616689
>> >
>> > OK, so what these bugs show is the breakage ... basically scsi_wait_scan
>> > isn't really waiting for the scans to complete.  I can fix it in stable
>> > so you can close your bug reports, but if I do, can you also transition
>> > away from using it so I can remove it in 3.5?
>>
>> Is there some other method whereby userspace can sync all driver
>> probing actions?
>
> No,  but then there never really was.  The theory is you know all the
> disks you need (/ /usr and so on) and you just wait for them to appear
> before mounting them and proceeding with boot.
>
>> We won't need scsi_complete_async_scans() after:
>>
>>   http://marc.info/?l=linux-scsi&m=133840132007532&w=2
>>
>> ...but won't initramfs environments still need a way to trigger
>> wait_for_device_probe()?  Something like echo "flush" >
>> /sys/devices/async_probe. and maybe reading that file indicates if
>> some async probing is still in-flight?
>
> Why?  The job of an initramfs is to mount root.  All it has to do is
> wait for root to appear via udev and then proceed.  The whole reason for
> doing stuff async initially was to speed boot, so probing can still be
> ongoing even after the initrd exits.
>
> If you think about it, most modern fabrics are hot plug.  Just because
> the initial scan has completed there's no guarantee that all the devices
> have appeared yet.

Fine for single device root, but what about raid and degraded assembly?

Last time I checked scsi_wait_scan was still being used by dracut in
the case where it decides to stop waiting for all raid members to
appear.  It's a "last call" before proceeding with degraded assembly.

If you immediately assemble and mount root as soon as the root device
could be started it will almost always be a degraded array.  Sure the
initramfs can just timeout arrival, but at a minimum that timeout
should be "load module + flush scanning".  Without a flush mechanism
it's just a shot in the dark what that minimum timeout should be.

If ata error recovery is kicking in and needs 10s of seconds to
recover a drive I'd want my initramfs to wait for that process to
quiesce before timing out and moving on.

--
Dan

Re: Remove scsi_wait_scan module

[Harald Hoyer]

Am 31.05.2012 04:34, schrieb Dan Williams:
> On Wed, May 30, 2012 at 4:32 PM, James Bottomley
> <James.Bottomley-JuX6DAaQMKPCXq6kfMZ53/egYHeGw8Jk@public.gmane.org> wrote:
>> On Wed, 2012-05-30 at 11:26 -0700, Dan Williams wrote:
>>> On Mon, May 28, 2012 at 5:07 AM, James Bottomley
>>> <James.Bottomley-JuX6DAaQMKPCXq6kfMZ53/egYHeGw8Jk@public.gmane.org> wrote:
>>>> On Mon, 2012-05-28 at 10:00 +0000, maximilian attems wrote:
>>>>> On Sun, May 27, 2012 at 10:13:46AM +0100, James Bottomley wrote:
>>>>>> scsi_wait_scan was introduced with asynchronous host scanning as a hack
>>>>>> for distributions that weren't using proper udev based wait for root to
>>>>>> appear in their initramfs scripts.  In 2.6.30 Commit
>>>>>
>>>>>> c751085943362143f84346d274e0011419c84202
>>>>>> Author: Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org>
>>>>>> Date:   Sun Apr 12 20:06:56 2009 +0200
>>>>>>
>>>>>>     PM/Hibernate: Wait for SCSI devices scan to complete during resume
>>>>>>
>>>>>> Actually broke scsi_wait_scan because it renders
>>>>>> scsi_complete_async_scans() a nop for modular SCSI if you include
>>>>>> scsi_scans.h (which this module does).
>>>>>>
>>>>>> The lack of bug reports is sufficient proof that this module is no
>>>>>> longer used.
>>>>>
>>>>> We do use it in initramfs-tools.
>>>>>
>>>>> There is quite a number of bug reports moaning about having to boot with
>>>>> `scsi_mod.scan=sync'. I didn't pass them on, because I didn't knew that
>>>>> the module itself got broken, for example:
>>>>> http://bugs.debian.org/616689
>>>>
>>>> OK, so what these bugs show is the breakage ... basically scsi_wait_scan
>>>> isn't really waiting for the scans to complete.  I can fix it in stable
>>>> so you can close your bug reports, but if I do, can you also transition
>>>> away from using it so I can remove it in 3.5?
>>>
>>> Is there some other method whereby userspace can sync all driver
>>> probing actions?
>>
>> No,  but then there never really was.  The theory is you know all the
>> disks you need (/ /usr and so on) and you just wait for them to appear
>> before mounting them and proceeding with boot.
>>
>>> We won't need scsi_complete_async_scans() after:
>>>
>>>   http://marc.info/?l=linux-scsi&m=133840132007532&w=2
>>>
>>> ...but won't initramfs environments still need a way to trigger
>>> wait_for_device_probe()?  Something like echo "flush" >
>>> /sys/devices/async_probe. and maybe reading that file indicates if
>>> some async probing is still in-flight?
>>
>> Why?  The job of an initramfs is to mount root.  All it has to do is
>> wait for root to appear via udev and then proceed.  The whole reason for
>> doing stuff async initially was to speed boot, so probing can still be
>> ongoing even after the initrd exits.
>>
>> If you think about it, most modern fabrics are hot plug.  Just because
>> the initial scan has completed there's no guarantee that all the devices
>> have appeared yet.
> 
> Fine for single device root, but what about raid and degraded assembly?
> 
> Last time I checked scsi_wait_scan was still being used by dracut in
> the case where it decides to stop waiting for all raid members to
> appear.  It's a "last call" before proceeding with degraded assembly.
> 
> If you immediately assemble and mount root as soon as the root device
> could be started it will almost always be a degraded array.  Sure the
> initramfs can just timeout arrival, but at a minimum that timeout
> should be "load module + flush scanning".  Without a flush mechanism
> it's just a shot in the dark what that minimum timeout should be.
> 
> If ata error recovery is kicking in and needs 10s of seconds to
> recover a drive I'd want my initramfs to wait for that process to
> quiesce before timing out and moving on.
> 
> --
> Dan

For Fedora17 scsi_wait_scan is not used anymore in the normal initramfs. I
removed it and raid is only tried to be started in degraded mode after a timeout
(several udevadm settle waits plus some extra 10 seconds).

Re: Remove scsi_wait_scan module

[Dan Williams]

On Thu, May 31, 2012 at 1:15 AM, Harald Hoyer <harald.hoyer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> For Fedora17 scsi_wait_scan is not used anymore in the normal initramfs. I
> removed it and raid is only tried to be started in degraded mode after a timeout
> (several udevadm settle waits plus some extra 10 seconds).

I'm probably missing something, but that sounds like "sprinkle
timeouts until it magically works"?  If udevadm settle is being used
as a discovery barrier shouldn't it be closing the loop with
wait_for_device_probe() in the kernel?

--
Dan

Re: Remove scsi_wait_scan module

[James Bottomley]

On Thu, 2012-05-31 at 09:45 -0700, Dan Williams wrote:
> On Thu, May 31, 2012 at 1:15 AM, Harald Hoyer <harald.hoyer@gmail.com> wrote:
> > For Fedora17 scsi_wait_scan is not used anymore in the normal initramfs. I
> > removed it and raid is only tried to be started in degraded mode after a timeout
> > (several udevadm settle waits plus some extra 10 seconds).
> 
> I'm probably missing something, but that sounds like "sprinkle
> timeouts until it magically works"?  If udevadm settle is being used
> as a discovery barrier shouldn't it be closing the loop with
> wait_for_device_probe() in the kernel?

No, that was pretty much the whole point of booting async.  It only
really matters in huge numbers of devices systems (like san connected
beasts).  What you *really* want is the boot to proceed immediately
after root appears and let the rest of the device probing continue in
parallel.

James

Re: Remove scsi_wait_scan module

[Dave Jones]

On Thu, May 31, 2012 at 10:15:08AM +0200, Harald Hoyer wrote:

 > For Fedora17 scsi_wait_scan is not used anymore in the normal initramfs.

Did/can you backport the same changes to F16 ?
If this module goes away, we don't want unpleasant surprises if we're still
dependant on it when we rebase the kernel.

	Dave

Re: Remove scsi_wait_scan module

[James Bottomley]

On Wed, 2012-05-30 at 19:34 -0700, Dan Williams wrote:
> On Wed, May 30, 2012 at 4:32 PM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> > On Wed, 2012-05-30 at 11:26 -0700, Dan Williams wrote:
> >> On Mon, May 28, 2012 at 5:07 AM, James Bottomley
> >> <James.Bottomley@hansenpartnership.com> wrote:
> >> > On Mon, 2012-05-28 at 10:00 +0000, maximilian attems wrote:
> >> >> On Sun, May 27, 2012 at 10:13:46AM +0100, James Bottomley wrote:
> >> >> > scsi_wait_scan was introduced with asynchronous host scanning as a hack
> >> >> > for distributions that weren't using proper udev based wait for root to
> >> >> > appear in their initramfs scripts.  In 2.6.30 Commit
> >> >>
> >> >> > c751085943362143f84346d274e0011419c84202
> >> >> > Author: Rafael J. Wysocki <rjw@sisk.pl>
> >> >> > Date:   Sun Apr 12 20:06:56 2009 +0200
> >> >> >
> >> >> >     PM/Hibernate: Wait for SCSI devices scan to complete during resume
> >> >> >
> >> >> > Actually broke scsi_wait_scan because it renders
> >> >> > scsi_complete_async_scans() a nop for modular SCSI if you include
> >> >> > scsi_scans.h (which this module does).
> >> >> >
> >> >> > The lack of bug reports is sufficient proof that this module is no
> >> >> > longer used.
> >> >>
> >> >> We do use it in initramfs-tools.
> >> >>
> >> >> There is quite a number of bug reports moaning about having to boot with
> >> >> `scsi_mod.scan=sync'. I didn't pass them on, because I didn't knew that
> >> >> the module itself got broken, for example:
> >> >> http://bugs.debian.org/616689
> >> >
> >> > OK, so what these bugs show is the breakage ... basically scsi_wait_scan
> >> > isn't really waiting for the scans to complete.  I can fix it in stable
> >> > so you can close your bug reports, but if I do, can you also transition
> >> > away from using it so I can remove it in 3.5?
> >>
> >> Is there some other method whereby userspace can sync all driver
> >> probing actions?
> >
> > No,  but then there never really was.  The theory is you know all the
> > disks you need (/ /usr and so on) and you just wait for them to appear
> > before mounting them and proceeding with boot.
> >
> >> We won't need scsi_complete_async_scans() after:
> >>
> >>   http://marc.info/?l=linux-scsi&m=133840132007532&w=2
> >>
> >> ...but won't initramfs environments still need a way to trigger
> >> wait_for_device_probe()?  Something like echo "flush" >
> >> /sys/devices/async_probe. and maybe reading that file indicates if
> >> some async probing is still in-flight?
> >
> > Why?  The job of an initramfs is to mount root.  All it has to do is
> > wait for root to appear via udev and then proceed.  The whole reason for
> > doing stuff async initially was to speed boot, so probing can still be
> > ongoing even after the initrd exits.
> >
> > If you think about it, most modern fabrics are hot plug.  Just because
> > the initial scan has completed there's no guarantee that all the devices
> > have appeared yet.
> 
> Fine for single device root, but what about raid and degraded assembly?
> 
> Last time I checked scsi_wait_scan was still being used by dracut in
> the case where it decides to stop waiting for all raid members to
> appear.  It's a "last call" before proceeding with degraded assembly.

That's pretty pointless behaviour, isn't it?  What it's basically doing
is allowing a set time for the devices to appear, waiting out that time
(so presumably something is wrong with one or more of the devices), then
inserting scsi_wait_scan as some type of magic incantation to just make
it work.

> If you immediately assemble and mount root as soon as the root device
> could be started it will almost always be a degraded array.  Sure the
> initramfs can just timeout arrival, but at a minimum that timeout
> should be "load module + flush scanning".  Without a flush mechanism
> it's just a shot in the dark what that minimum timeout should be.

No, you wait a specified time for all the devices to appear before
assembling the raid.  If they don't, you try to bring a degraded raid
up.

The behaviour is also dependent on the user: If I'm a savvy user and I
have a raid log, I want my system up as fast as possible, so I only want
to wait until the minimum number of devices appears before assembling
the raid and moving on, knowing that hotplug of the remaining will cause
a log replay.

> If ata error recovery is kicking in and needs 10s of seconds to
> recover a drive I'd want my initramfs to wait for that process to
> quiesce before timing out and moving on.

That's the timeout you specify.

James

Re: Remove scsi_wait_scan module

[Dan Williams]

On Thu, May 31, 2012 at 1:21 AM, James Bottomley
<James.Bottomley-JuX6DAaQMKPCXq6kfMZ53/egYHeGw8Jk@public.gmane.org> wrote:
>> Last time I checked scsi_wait_scan was still being used by dracut in
>> the case where it decides to stop waiting for all raid members to
>> appear.  It's a "last call" before proceeding with degraded assembly.
>
> That's pretty pointless behaviour, isn't it?  What it's basically doing
> is allowing a set time for the devices to appear, waiting out that time
> (so presumably something is wrong with one or more of the devices), then
> inserting scsi_wait_scan as some type of magic incantation to just make
> it work.

Random timeouts are magic, waiting for known probing to complete is not.

>> If you immediately assemble and mount root as soon as the root device
>> could be started it will almost always be a degraded array.  Sure the
>> initramfs can just timeout arrival, but at a minimum that timeout
>> should be "load module + flush scanning".  Without a flush mechanism
>> it's just a shot in the dark what that minimum timeout should be.
>
> No, you wait a specified time for all the devices to appear before
> assembling the raid.  If they don't, you try to bring a degraded raid
> up.
>
> The behaviour is also dependent on the user: If I'm a savvy user and I
> have a raid log, I want my system up as fast as possible, so I only want
> to wait until the minimum number of devices appears before assembling
> the raid and moving on, knowing that hotplug of the remaining will cause
> a log replay.
>
>> If ata error recovery is kicking in and needs 10s of seconds to
>> recover a drive I'd want my initramfs to wait for that process to
>> quiesce before timing out and moving on.
>
> That's the timeout you specify.

I'm a savvy raid user who wants his raid arrays to start as soon as
possible and one that knows that  lldds, libsas, and libata can inject
long unpredictable delays to discover devices that were attached since
boot.

So it's not about hotplug it's about letting the initial scan /
discovery process (which may involve 3 resets and 1 minute of timeouts
per-disk in a pathological ata configuration) complete for devices
that were minimally indicated in the initial scan.  The initramfs
timeout is then an optional cushion to allow a hotplug event to land,
but for the "as fast as possible case" that means letting link
recovery actions quiesce.

--
Dan

Re: Remove scsi_wait_scan module

[James Bottomley]

On Thu, 2012-05-31 at 09:40 -0700, Dan Williams wrote:
> On Thu, May 31, 2012 at 1:21 AM, James Bottomley
> <James.Bottomley-JuX6DAaQMKPCXq6kfMZ53/egYHeGw8Jk@public.gmane.org> wrote:
> >> Last time I checked scsi_wait_scan was still being used by dracut in
> >> the case where it decides to stop waiting for all raid members to
> >> appear.  It's a "last call" before proceeding with degraded assembly.
> >
> > That's pretty pointless behaviour, isn't it?  What it's basically doing
> > is allowing a set time for the devices to appear, waiting out that time
> > (so presumably something is wrong with one or more of the devices), then
> > inserting scsi_wait_scan as some type of magic incantation to just make
> > it work.
> 
> Random timeouts are magic, waiting for known probing to complete is not.

Well, yes it is ... it's just a question of whose magic timeouts.
Probing mostly waits the device appearance timeout on the bus.

The timeout in question is how long the user is prepared to wait for the
root device.  Most distros tend to code for 120s+, so that's way beyond
any bus timeout.  If the user wishes forever, that's fine too.

> >> If you immediately assemble and mount root as soon as the root device
> >> could be started it will almost always be a degraded array.  Sure the
> >> initramfs can just timeout arrival, but at a minimum that timeout
> >> should be "load module + flush scanning".  Without a flush mechanism
> >> it's just a shot in the dark what that minimum timeout should be.
> >
> > No, you wait a specified time for all the devices to appear before
> > assembling the raid.  If they don't, you try to bring a degraded raid
> > up.
> >
> > The behaviour is also dependent on the user: If I'm a savvy user and I
> > have a raid log, I want my system up as fast as possible, so I only want
> > to wait until the minimum number of devices appears before assembling
> > the raid and moving on, knowing that hotplug of the remaining will cause
> > a log replay.
> >
> >> If ata error recovery is kicking in and needs 10s of seconds to
> >> recover a drive I'd want my initramfs to wait for that process to
> >> quiesce before timing out and moving on.
> >
> > That's the timeout you specify.
> 
> I'm a savvy raid user who wants his raid arrays to start as soon as
> possible and one that knows that  lldds, libsas, and libata can inject
> long unpredictable delays to discover devices that were attached since
> boot.
> 
> So it's not about hotplug it's about letting the initial scan /
> discovery process (which may involve 3 resets and 1 minute of timeouts
> per-disk in a pathological ata configuration) complete for devices
> that were minimally indicated in the initial scan.  The initramfs
> timeout is then an optional cushion to allow a hotplug event to land,
> but for the "as fast as possible case" that means letting link
> recovery actions quiesce.

If there are errors on the bus, the chances are the devices are missed
by the initial discover and appear later anyway, so neither approach is
foolproof.  I like the how long is the user prepared to wait approach vs
how long does it take to complete error recovery simply because mostly
if the device doesn't appear within that time it is a bug (or a hardware
configuration problem).

Conversely, if you think of the usual case: the root device appears long
before bus probing finishes, so by only waiting for root, we assist the
fast boot process.

James

Re: Remove scsi_wait_scan module

[Dan Williams]

On Thu, May 31, 2012 at 10:42 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> If there are errors on the bus, the chances are the devices are missed
> by the initial discover and appear later anyway, so neither approach is
> foolproof.  I like the how long is the user prepared to wait approach vs
> how long does it take to complete error recovery simply because mostly
> if the device doesn't appear within that time it is a bug (or a hardware
> configuration problem).

I agree that the "how long is the user prepared to wait" approach is
optimal.  But there is no way for the user to express "I want to be
fast and safe" i.e. wait just the minimum time the subsystem needs to
complete an initial scan.  Outside of building the module into the
kernel to take advantage of the built-in wait_for_device_probe() in
prepare_namespace().  That minimum time varies based on configuration.
 I've watched dracut fail raid assembly even though the driver had not
returned 'done' from ->scan_finished().  A generous timeout would fix
that, but now you've overshot what the kernel was prepared to call the
initial discovery window.

> Conversely, if you think of the usual case: the root device appears long
> before bus probing finishes, so by only waiting for root, we assist the
> fast boot process.

Yes, it sucks to do "wait world" when you just need a subset, but I'm
only saying to do the global wait in the case where the raid metadata
expects all devices to be present and we are about to degrade the
array for the first time.  So in the "all devices appear case" or the
"array already marked degraded case" we wouldn't need to wait and boot
would proceed soon as root is able to be assembled.

...but maybe that is my answer.  A generous timeout incurred only when
that first degraded event happens?  I don't know just makes me
uncomfortable that userspace has no visibility into all the machinery
that makes wait_for_device_probe() work in the built-in case.  But
maybe that is because prepare_namespace() is not free to be as
expressive as an initramfs and can only use the big hammer.

--
Dan
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html