Re: Asynchronous scsi scanning, version 9

Re: Asynchronous scsi scanning, version 9

[Patrick Mansfield]

[adding hotplug-devel ... maybe Marco or Kay can comment]

On Mon, May 29, 2006 at 07:05:15AM -0600, Matthew Wilcox wrote:
> On Mon, May 29, 2006 at 10:38:13AM +0200, Stefan Richter wrote:
> > Matthew Wilcox wrote:
> > > Add the scsi_mod.scan kernel parameter to determine how scsi busses
> > > are scanned.  "sync" is the current behaviour.  "none" punts scanning
> > > scsi busses to userspace.  "async" is the new default.
> > 
> > This parameter is only relevant with LLDDs which use scsi_scan_host, right?
> 
> Not entirely.  If you set it to "none", scsi_scan_target() also returns
> without doing anything.  If you use the scsi_prep_async_scan() and
> scsi_finish_async_scan() API, you can also use this infrastructure to
> make scanning sbp2 synchronised with other scsi hosts.  Then the setting
> of sync vs async also triggers old vs new behaviour.
> 
> > Furthermore, "sync|async" basically means "serialized|parallelized
> > across host adapters". Does it also mean "finishing before|after driver
> > initialization"? (With LLDDs which use scsi_scan_host.)
> 
> That's what scsi_complete_async_scans() is for.  If you have a built-in
> module, it will wait for the async scans to finish before we get as far
> as trying to mount root.  It does change observable behaviour in that
> sys_module_init() will return before scans are complete.  However, I
> believe most distros userspace copes with this these days.  For example,
> Debian has:
> 
>     # wait for the udevd childs to finish
>     log_action_begin_msg "Waiting for /dev to be fully populated"
>     while [ -d /dev/.udev/queue/ ]; do
>         sleep 1
>         udevd_timeout=$(($udevd_timeout - 1))
> [...]

Not sure where that is, but AFAIR that is to process the cold plug case,
where udev starts up, the hotplug/netlink events are replayed, and we
don't want to continue until all those events have been processed.

SLES 10 has similar code, but a sleep of 0.1 (see their /etc/init.d/boot.udev,
and I think /sbin/mkinitrd).

> Since the scsi scan is going to be finding new devices the entire time,
> the queue directory is going to not empty.

It won't always be finding new devices, there could be glitches like a
timeout, or some read (partitition check) that happens to take more than a
second, and the udev queue becomes empty even though the scsi /sd scan is
still in progress.

You really want some udev rule that mounts root or such and then the boot
continues from there ... rather than waiting for an unrelated sets of
events, and then trying to mount root unconditionally (and possibly
failing). I thought Hannes or someone had posted an example udev rule or
such for this. Maybe it is even in SLES 10?

Same for applications - you want them to start after a dev (or set of
devs) shows up, though if we wait for the root dev it is even less likely
that an app's dev will be unavailable.

Of course if you aren't using udev in your init{rd|ramfs}, udev rules
and such can't fix the problem :-(

-- Patrick Mansfield


-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x107521&bid$8729&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Kay Sievers]

On Wed, 2006-05-31 at 16:21 -0700, Patrick Mansfield wrote:
> [adding hotplug-devel ... maybe Marco or Kay can comment]
> 
> On Mon, May 29, 2006 at 07:05:15AM -0600, Matthew Wilcox wrote:
> > On Mon, May 29, 2006 at 10:38:13AM +0200, Stefan Richter wrote:
> > > Matthew Wilcox wrote:
> > > > Add the scsi_mod.scan kernel parameter to determine how scsi busses
> > > > are scanned.  "sync" is the current behaviour.  "none" punts scanning
> > > > scsi busses to userspace.  "async" is the new default.
> > > 
> > > This parameter is only relevant with LLDDs which use scsi_scan_host, right?
> > 
> > Not entirely.  If you set it to "none", scsi_scan_target() also returns
> > without doing anything.  If you use the scsi_prep_async_scan() and
> > scsi_finish_async_scan() API, you can also use this infrastructure to
> > make scanning sbp2 synchronised with other scsi hosts.  Then the setting
> > of sync vs async also triggers old vs new behaviour.
> > 
> > > Furthermore, "sync|async" basically means "serialized|parallelized
> > > across host adapters". Does it also mean "finishing before|after driver
> > > initialization"? (With LLDDs which use scsi_scan_host.)
> > 
> > That's what scsi_complete_async_scans() is for.  If you have a built-in
> > module, it will wait for the async scans to finish before we get as far
> > as trying to mount root.  It does change observable behaviour in that
> > sys_module_init() will return before scans are complete.  However, I
> > believe most distros userspace copes with this these days.  For example,
> > Debian has:
> > 
> >     # wait for the udevd childs to finish
> >     log_action_begin_msg "Waiting for /dev to be fully populated"
> >     while [ -d /dev/.udev/queue/ ]; do
> >         sleep 1
> >         udevd_timeout=$(($udevd_timeout - 1))
> > [...]

That has replaced by a binary called "udevsettle" which waits for events
to finish, by comparing the current kernel event sequence number
exported in sysfs with the latest handled event by udev.

> Not sure where that is, but AFAIR that is to process the cold plug case,
> where udev starts up, the hotplug/netlink events are replayed, and we
> don't want to continue until all those events have been processed.
> 
> SLES 10 has similar code, but a sleep of 0.1 (see their /etc/init.d/boot.udev,
> and I think /sbin/mkinitrd).

It uses only udevsettle now, also the partitioner and similar needs this
to wait for the partiton table rescan to finish, before continuing using
the new devices. 

> > Since the scsi scan is going to be finding new devices the entire time,
> > the queue directory is going to not empty.

Watching only the queue is not enough, cause there are only received
events exported, but not events still in the kernel netlink queue.
Therefore you need to compare the current kernel seqnum like udevsettle
is doing it.

> It won't always be finding new devices, there could be glitches like a
> timeout, or some read (partitition check) that happens to take more than a
> second, and the udev queue becomes empty even though the scsi /sd scan is
> still in progress.

Right. For the settle time of usb-storage we watch for the kernel tread
to go away. :)

> You really want some udev rule that mounts root or such and then the boot
> continues from there ... rather than waiting for an unrelated sets of
> events, and then trying to mount root unconditionally (and possibly
> failing). I thought Hannes or someone had posted an example udev rule or
> such for this. Maybe it is even in SLES 10?

It does not only wait for the queue to become empty. Initramfs creates
dynamic udev rules based on the kernel commandline and waits until the
device appears. The real root (localfs) waits for all needed devices
mentioned in /etc/fstab before continuing.

> Same for applications - you want them to start after a dev (or set of
> devs) shows up, though if we wait for the root dev it is even less likely
> that an app's dev will be unavailable.
> 
> Of course if you aren't using udev in your init{rd|ramfs}, udev rules
> and such can't fix the problem :-(

Kay



-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x107521&bid$8729&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Alexander E. Patrakov]

Patrick Mansfield wrote:
>> That's what scsi_complete_async_scans() is for.  If you have a built-in
>> module, it will wait for the async scans to finish before we get as far
>> as trying to mount root.  It does change observable behaviour in that
>> sys_module_init() will return before scans are complete.

What are the expected interactions of this code with early userspace aka "kinit" 
that comes with (soon to be merged) klibc? Does this scsi_complete_async_scans() 
function get called at all if such "default" initramfs with kinit is used?

The problem is that if it doesn't, kinit will not wait.

-- 
Alexander E. Patrakov


-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x107521&bid$8729&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[maximilian attems]

On Thu, Jun 01, 2006 at 07:14:05PM +0600, Alexander E. Patrakov wrote:
> Patrick Mansfield wrote:
> >>That's what scsi_complete_async_scans() is for.  If you have a built-in
> >>module, it will wait for the async scans to finish before we get as far
> >>as trying to mount root.  It does change observable behaviour in that
> >>sys_module_init() will return before scans are complete.
> 
> What are the expected interactions of this code with early userspace aka 
> "kinit" that comes with (soon to be merged) klibc? Does this 
> scsi_complete_async_scans() function get called at all if such "default" 
> initramfs with kinit is used?
> 
> The problem is that if it doesn't, kinit will not wait.

unless you build an hardcoded initramfs for an specific box,
you want anyway an coldplugging utility on board.

udev can be compiled with klibc, put both inside your initramfs.

-- 
maks


-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x107521&bid$8729&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Matthew Wilcox]

On Thu, Jun 01, 2006 at 07:14:05PM +0600, Alexander E. Patrakov wrote:
> Patrick Mansfield wrote:
> >>That's what scsi_complete_async_scans() is for.  If you have a built-in
> >>module, it will wait for the async scans to finish before we get as far
> >>as trying to mount root.  It does change observable behaviour in that
> >>sys_module_init() will return before scans are complete.
> 
> What are the expected interactions of this code with early userspace aka 
> "kinit" that comes with (soon to be merged) klibc? Does this 
> scsi_complete_async_scans() function get called at all if such "default" 
> initramfs with kinit is used?

It gets called as a late_initcall -- before initramfs gets started.  The
problem comes when you load your scsi driver as a module.

There's a potential solution to this.  If we add a hook to sys_module_init
such that it'll call a specified function before returning, we can make
it wait until all scans are done.  This way, userspace never sees the
asynchronous scanning behaviour.  But if you have devices of different
types, you won't get the overlapping scans.


-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x107521&bid$8729&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Alexander E. Patrakov]

Matthew Wilcox wrote:
> On Thu, Jun 01, 2006 at 07:14:05PM +0600, Alexander E. Patrakov wrote:
>> Patrick Mansfield wrote:
>>>> That's what scsi_complete_async_scans() is for.  If you have a built-in
>>>> module, it will wait for the async scans to finish before we get as far
>>>> as trying to mount root.  It does change observable behaviour in that
>>>> sys_module_init() will return before scans are complete.
>> What are the expected interactions of this code with early userspace aka 
>> "kinit" that comes with (soon to be merged) klibc? Does this 
>> scsi_complete_async_scans() function get called at all if such "default" 
>> initramfs with kinit is used?
> 
> It gets called as a late_initcall -- before initramfs gets started.

Many thanks for clarifying the situation.

-- 
Alexander E. Patrakov


-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x107521&bid$8729&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Arjan van de Ven]

On Thu, 2006-06-01 at 07:23 -0600, Matthew Wilcox wrote:
> On Thu, Jun 01, 2006 at 07:14:05PM +0600, Alexander E. Patrakov wrote:
> > Patrick Mansfield wrote:
> > >>That's what scsi_complete_async_scans() is for.  If you have a built-in
> > >>module, it will wait for the async scans to finish before we get as far
> > >>as trying to mount root.  It does change observable behaviour in that
> > >>sys_module_init() will return before scans are complete.
> > 
> > What are the expected interactions of this code with early userspace aka 
> > "kinit" that comes with (soon to be merged) klibc? Does this 
> > scsi_complete_async_scans() function get called at all if such "default" 
> > initramfs with kinit is used?
> 
> It gets called as a late_initcall -- before initramfs gets started.  The
> problem comes when you load your scsi driver as a module.
> 
> There's a potential solution to this.  If we add a hook to sys_module_init
> such that it'll call a specified function before returning, we can make
> it wait until all scans are done.  This way, userspace never sees the
> asynchronous scanning behaviour.  But if you have devices of different
> types, you won't get the overlapping scans.

alternative is a syscall/ioctl/sysfs thing that userland can trigger to
cause this wait to happen... would be nice I suppose as a general
option.....



-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x107521&bid$8729&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[James Bottomley]

On Thu, 2006-06-01 at 07:23 -0600, Matthew Wilcox wrote:
> > What are the expected interactions of this code with early userspace aka 
> > "kinit" that comes with (soon to be merged) klibc? Does this 
> > scsi_complete_async_scans() function get called at all if such "default" 
> > initramfs with kinit is used?
> 
> It gets called as a late_initcall -- before initramfs gets started.  The
> problem comes when you load your scsi driver as a module.

Actually, the first problem is that SCSI doesn't compile as a module at
all:  late_initcalls are defined to be module_init calls for modules.
Your adding this to scsi_scan.c now gives us two module_init calls,
which won't build.  I fixed this by putting an #ifdef MODULE around the
late_initcall.

> There's a potential solution to this.  If we add a hook to sys_module_init
> such that it'll call a specified function before returning, we can make
> it wait until all scans are done.  This way, userspace never sees the
> asynchronous scanning behaviour.  But if you have devices of different
> types, you won't get the overlapping scans.

How about this solution which works for me with debian on a huge 6 bus
scsi machine:  it introduces a new module scsi_wait_scan.ko whose sole
job is to wait for the scans to complete in its init function.  The
initrd/initramfs sequence now becomes:

insert all other modules
modprobe scsi_wait_scan

James

---

Index: linux-2.6/drivers/scsi/Makefile
=================================--- linux-2.6.orig/drivers/scsi/Makefile	2006-06-24 13:37:50.000000000 -0500
+++ linux-2.6/drivers/scsi/Makefile	2006-06-24 13:38:08.000000000 -0500
@@ -22,6 +22,10 @@
 
 obj-$(CONFIG_SCSI)		+= scsi_mod.o
 
+ifeq ("$(CONFIG_SCSI)", "m")
+	obj-m			+= scsi_wait_scan.o
+endif
+
 obj-$(CONFIG_RAID_ATTRS)	+= raid_class.o
 
 # --- NOTE ORDERING HERE ---
Index: linux-2.6/drivers/scsi/scsi_priv.h
=================================--- linux-2.6.orig/drivers/scsi/scsi_priv.h	2006-06-24 13:37:50.000000000 -0500
+++ linux-2.6/drivers/scsi/scsi_priv.h	2006-06-24 13:38:08.000000000 -0500
@@ -49,6 +49,9 @@
 	{ };
 #endif
 
+/* scsi_scan.c */
+int scsi_complete_async_scans(void);
+
 /* scsi_devinfo.c */
 extern int scsi_get_device_flags(struct scsi_device *sdev,
 				 const unsigned char *vendor,
Index: linux-2.6/drivers/scsi/scsi_scan.c
=================================--- linux-2.6.orig/drivers/scsi/scsi_scan.c	2006-06-24 13:37:50.000000000 -0500
+++ linux-2.6/drivers/scsi/scsi_scan.c	2006-06-24 13:38:08.000000000 -0500
@@ -125,7 +125,7 @@
 	struct completion prev_finished;
 };
 
-static int scsi_complete_async_scans(void)
+int scsi_complete_async_scans(void)
 {
 	struct async_scan_data *data;
 
@@ -157,8 +157,10 @@
 	kfree(data);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(scsi_complete_async_scans);
+#ifndef MODULE
 late_initcall(scsi_complete_async_scans);
-
+#endif
 
 /**
  * scsi_unlock_floptical - unlock device via a special MODE SENSE command
Index: linux-2.6/drivers/scsi/scsi_wait_scan.c
=================================--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6/drivers/scsi/scsi_wait_scan.c	2006-06-24 13:38:45.000000000 -0500
@@ -0,0 +1,31 @@
+/*
+ * scsi_wait_scan.c
+ *
+ * Copyright (C) 2006 James Bottomley <James.Bottomley@SteelEye.com>
+ *
+ * This is a simple module to wait until all the async scans are
+ * complete.  The idea is to use it in initrd/initramfs scripts.  You
+ * modprobe it after all the modprobes of the root SCSI drivers and it
+ * will wait until they have all finished scanning their busses before
+ * allowing the boot to proceed
+ */
+
+#include <linux/module.h>
+#include "scsi_priv.h"
+
+static int __init wait_scan_init(void)
+{
+	scsi_complete_async_scans();
+	return 0;
+}
+
+static void __exit wait_scan_exit(void)
+{
+}
+
+MODULE_DESCRIPTION("SCSI wait for scans");
+MODULE_AUTHOR("James Bottomley");
+MODULE_LICENSE("GPL");
+
+module_init(wait_scan_init);
+module_exit(wait_scan_exit);



Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Matthew Wilcox]

On Sun, Jun 25, 2006 at 04:15:26PM -0500, James Bottomley wrote:
> Actually, the first problem is that SCSI doesn't compile as a module at
> all:  late_initcalls are defined to be module_init calls for modules.
> Your adding this to scsi_scan.c now gives us two module_init calls,
> which won't build.  I fixed this by putting an #ifdef MODULE around the
> late_initcall.

I think that's a good idea (well, I would, I suggested it to you ;-)

> > There's a potential solution to this.  If we add a hook to sys_module_init
> > such that it'll call a specified function before returning, we can make
> > it wait until all scans are done.  This way, userspace never sees the
> > asynchronous scanning behaviour.  But if you have devices of different
> > types, you won't get the overlapping scans.
> 
> How about this solution which works for me with debian on a huge 6 bus
> scsi machine:  it introduces a new module scsi_wait_scan.ko whose sole
> job is to wait for the scans to complete in its init function.  The
> initrd/initramfs sequence now becomes:
> 
> insert all other modules
> modprobe scsi_wait_scan

I think that's a great idea.  I wonder about making it always fail
initialisation, since then it can be loaded multiple times without being
unloaded.  I think Arjan was suggesting some proc or sysfs file that
would cause scsi_complete_async_scans() to be called, but I couldn't
figure out where a good place to put such a file would be. 

> @@ -22,6 +22,10 @@
>  
>  obj-$(CONFIG_SCSI)		+= scsi_mod.o
>  
> +ifeq ("$(CONFIG_SCSI)", "m")
> +	obj-m			+= scsi_wait_scan.o
> +endif

We seem to have a bit of an allergy to conditionals in Makefiles these
days; how about:

tmp-$(CONFIG_SCSI)	:= scsi_wait_scan.o
obj-m 			+= $(tmp-m)

>  
> -static int scsi_complete_async_scans(void)
> +int scsi_complete_async_scans(void)
>  {
>  	struct async_scan_data *data;
>  
> @@ -157,8 +157,10 @@
>  	kfree(data);
>  	return 0;
>  }

Do you think it's worth putting in something like:

/* Only exported for the benefit of scsi_wait_scan */

And maybe ...

#ifdef MODULE
/* Only exported for the benefit of scsi_wait_scan */
EXPORT_SYMBOL_GPL(scsi_complete_async_scans);
#else
late_initcall(scsi_complete_async_scans);
#endif

I really don't want to see driver authors calling it -- if they are,
something's gone pretty horribly wrong.

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Arjan van de Ven]

> > How about this solution which works for me with debian on a huge 6 bus
> > scsi machine:  it introduces a new module scsi_wait_scan.ko whose sole
> > job is to wait for the scans to complete in its init function.  The
> > initrd/initramfs sequence now becomes:
> > 
> > insert all other modules
> > modprobe scsi_wait_scan
> 
> I think that's a great idea.  I wonder about making it always fail
> initialisation, since then it can be loaded multiple times without being
> unloaded.  I think Arjan was suggesting some proc or sysfs file that
> would cause scsi_complete_async_scans() to be called, but I couldn't
> figure out where a good place to put such a file would be. 

Hi,

just to expand on this: The reason I am suggesting this is to allow the
initrd to have a way to wait for device scans before (re)trying to mount
the root filesystem. One of the interesting challenges today for the
initrd with USB is that you don't know when the devices are visible; now
I know this won't fix USB, but it at least introduces a proper method
for ensuring that for scsi; I suppose we should make it a generic thing
with a notifier chain so that all subsystems that want to can get a
callback and finalize their initialization... I can imagine fiber
channel cards wanting to use this to wait for LIP etc...

One question is if this should get a timeout parameter or if that should
be left up to the devices...

(and I think the initrd needs to try to find the rootfs at least once
without waiting, or there should be 2 levels of expensiveness to the
wait, so that it'll first try the asynchronous way, but that it can do
the more expensive wait rather than causing an outright panic() as
happens today when the rootfs cannot be found..)

Greetings,
    Arjan van de Ven


Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Matthew Wilcox]

On Mon, Jun 26, 2006 at 10:24:45AM +0200, Arjan van de Ven wrote:
> just to expand on this: The reason I am suggesting this is to allow the
> initrd to have a way to wait for device scans before (re)trying to mount
> the root filesystem. One of the interesting challenges today for the
> initrd with USB is that you don't know when the devices are visible; now
> I know this won't fix USB, but it at least introduces a proper method
> for ensuring that for scsi; I suppose we should make it a generic thing
> with a notifier chain so that all subsystems that want to can get a
> callback and finalize their initialization... I can imagine fiber
> channel cards wanting to use this to wait for LIP etc...

Fortunately, USB is SCSI, so we can actually use this code to solve that
problem too.  I wasn't thinking of USB when I wrote the code (because I
didn't know there was a problem).  But really, it's very generic stuff;
there's a list_head, a pointer to a Scsi_Host and a completion.

I'm not 100% sure what the problem is with USB.  If it's that we may not
have discovered all the USB devices currently plugged in, then I think
we need to change the way USB works to use one Scsi_Host for all USB
storage devices, and then make each device either its own target or its
own channel (probably the former; the latter is less well-tested code).

If it's simply the chunk of code beginning with:
        /* Wait for the timeout to expire or for a disconnect */
that's causing the problem, then this is easily fixable in the current
scheme:

Index: drivers/usb/storage/usb.c
=================================RCS file: /var/cvs/linux-2.6/drivers/usb/storage/usb.c,v
retrieving revision 1.22
diff -u -p -r1.22 usb.c
--- drivers/usb/storage/usb.c	3 Apr 2006 13:45:11 -0000	1.22
+++ drivers/usb/storage/usb.c	26 Jun 2006 12:34:45 -0000
@@ -849,10 +849,13 @@ static void release_everything(struct us
 static int usb_stor_scan_thread(void * __us)
 {
 	struct us_data *us = (struct us_data *)__us;
+	struct async_scan_data *data;
 
 	printk(KERN_DEBUG
 		"usb-storage: device found at %d\n", us->pusb_dev->devnum);
 
+	data = scsi_prep_async_scan(us_to_host(us));
+
 	/* Wait for the timeout to expire or for a disconnect */
 	if (delay_use > 0) {
 		printk(KERN_DEBUG "usb-storage: waiting for device "
@@ -875,12 +878,14 @@ retry:
 			us->max_lun = usb_stor_Bulk_max_lun(us);
 			mutex_unlock(&us->dev_mutex);
 		}
-		scsi_scan_host(us_to_host(us));
+		scsi_scan_target(&us_to_host(us)->shost_gendev, 0, 0,
+				SCAN_WILD_CARD, 0);
 		printk(KERN_DEBUG "usb-storage: device scan complete\n");
 
 		/* Should we unbind if no devices were detected? */
 	}
 
+	scsi_finish_async_scan(data);
 	scsi_host_put(us_to_host(us));
 	complete_and_exit(&threads_gone, 0);
 }

By the way, if USB intends to stick to its one-host-per-device scheme,
it would do well to set host->max_id to 1, rather than failing targets
in queuecommand.

> One question is if this should get a timeout parameter or if that should
> be left up to the devices...

No timeout.  It's up to the scanners to say they're done.

> (and I think the initrd needs to try to find the rootfs at least once
> without waiting, or there should be 2 levels of expensiveness to the
> wait, so that it'll first try the asynchronous way, but that it can do
> the more expensive wait rather than causing an outright panic() as
> happens today when the rootfs cannot be found..)

It's not just root, it's swap and all the other bits of fstab too.
No point in being able to reliably mount root if the sysadmin has to
manually intervene to mount /home on every boot.  Or worse, occasionally
intervene ...

All the waiting is done in parallel anyway, so you're really trying to
squeeze the last 0.0001% out of it; better to wait and be safe.

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Arjan van de Ven]

> Fortunately, USB is SCSI, so we can actually use this code to solve that
> problem too.  I wasn't thinking of USB when I wrote the code (because I
> didn't know there was a problem).  But really, it's very generic stuff;
> there's a list_head, a pointer to a Scsi_Host and a completion.
> 
> I'm not 100% sure what the problem is with USB. 

I seem to remember that the problem with USB was that you can't even
know how many things are still outstanding; devices are free to appear
on the bus at any time after bus power on, without specified timeout...
If the kernel can't know what's out there, it can't wait until it has a
complete picture either, just by virtue of not being able to know when
that is achieved ;-)

But I could be wrong entirely; USB storage is not something I delve into
daily...

Greetings,
   Arjan van de Ven


Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Matthew Dharm]

[-- Attachment #1.1: Type: text/plain, Size: 1611 bytes --]

On Mon, Jun 26, 2006 at 06:40:01AM -0600, Matthew Wilcox wrote:
> I'm not 100% sure what the problem is with USB.  If it's that we may not
> have discovered all the USB devices currently plugged in, then I think
> we need to change the way USB works to use one Scsi_Host for all USB
> storage devices, and then make each device either its own target or its
> own channel (probably the former; the latter is less well-tested code).

There are a couple of things which make this difficult for USB.

First, some (many?) USB devices need to be left alone for several seconds
after attachment in order to allow them to initialize to the point where
they are usable.

Second, depending on how many hubs are between the host and target, the
time-to-discover the device is highly variable.

Third, once discovered the device may still take a long time to be "ready".
Think of this as a slow spin-up time.  This is different from my first
point in that the device can actually accept commands, but all commands
will fail with some sort of not-ready type error.

As for using one Scsi_Host... there are several usb-storage devices which
attach to an entire SCSI bus (not just a single target), so can't make each
device it's own target.  Also, at the time this was all written it wasn't
possible to dynamically remove (with any stability) individual targets or
channels.  Perhaps that has changed?

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

You suck Stef.
					-- Greg 
User Friendly, 11/29/97

[-- Attachment #1.2: Type: application/pgp-signature, Size: 191 bytes --]

[-- Attachment #2: Type: text/plain, Size: 300 bytes --]

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

[-- Attachment #3: Type: text/plain, Size: 226 bytes --]

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Matthew Wilcox]

On Mon, Jun 26, 2006 at 07:44:42AM -0700, Matthew Dharm wrote:
> On Mon, Jun 26, 2006 at 06:40:01AM -0600, Matthew Wilcox wrote:
> > I'm not 100% sure what the problem is with USB.  If it's that we may not
> > have discovered all the USB devices currently plugged in, then I think
> > we need to change the way USB works to use one Scsi_Host for all USB
> > storage devices, and then make each device either its own target or its
> > own channel (probably the former; the latter is less well-tested code).
> 
> There are a couple of things which make this difficult for USB.
> 
> First, some (many?) USB devices need to be left alone for several seconds
> after attachment in order to allow them to initialize to the point where
> they are usable.

That's OK.  Once we know they're there, we can reserve their place
and delay until they're ready to go.

> Second, depending on how many hubs are between the host and target, the
> time-to-discover the device is highly variable.

Can, or does, USB keep track of hub discovery, and hence know whether
or not it's completed USB discovery?

> Third, once discovered the device may still take a long time to be "ready".
> Think of this as a slow spin-up time.  This is different from my first
> point in that the device can actually accept commands, but all commands
> will fail with some sort of not-ready type error.

That's OK too.

> As for using one Scsi_Host... there are several usb-storage devices which
> attach to an entire SCSI bus (not just a single target), so can't make each
> device it's own target.

Oh.  My fault for reading the comment rather than the code.

                /* reject if target != 0 or if LUN is higher than
                 * the maximum known LUN
                 */
                else if (us->srb->device->id && 
                                !(us->flags & US_FL_SCM_MULT_TARG)) {

Do these devices stick to occupying only target IDs from 0-7?  If not,
you may wish to increase ->max_id for those devices.  I think it'd be worth
exporting scsi_scan_channel() from the midlayer (and rearranging it to have
__scsi_scan_channel() as was done with __scsi_scan_target) for USB's benefit.

> Also, at the time this was all written it wasn't
> possible to dynamically remove (with any stability) individual targets or
> channels.  Perhaps that has changed?

It's definitely possible to remove individual targets dynamically now;
Fibre Channel has sorted that out (and will complain loudly if it breaks).
The scsi core doesn't really have a channel object; channel is just an integer 
that describes a path to a target.  So I think there should be no problem in
converting USB to have one host and many channels.

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[James Bottomley]

On Mon, 2006-06-26 at 09:18 -0600, Matthew Wilcox wrote:
> It's definitely possible to remove individual targets dynamically now;
> Fibre Channel has sorted that out (and will complain loudly if it
> breaks).
> The scsi core doesn't really have a channel object; channel is just an
> integer 
> that describes a path to a target.  So I think there should be no
> problem in
> converting USB to have one host and many channels.

Actually, I think usb should be exactly like FC ... in that should be an
unscanned attachment, so if and when it gets done properly, it won't
have any need for the async scanning updates anyway.

James



Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Greg KH]

On Mon, Jun 26, 2006 at 09:18:28AM -0600, Matthew Wilcox wrote:
> On Mon, Jun 26, 2006 at 07:44:42AM -0700, Matthew Dharm wrote:
> > Second, depending on how many hubs are between the host and target, the
> > time-to-discover the device is highly variable.
> 
> Can, or does, USB keep track of hub discovery, and hence know whether
> or not it's completed USB discovery?

Nope, it never knows this.  So that is why you can't have any type of
"just wait until USB is done scanning all devices before continuing"
behavior, because there is no such thing to trigger off of.

sorry,

greg k-h

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Greg KH]

On Mon, Jun 26, 2006 at 02:59:11PM +0200, Arjan van de Ven wrote:
> 
> > Fortunately, USB is SCSI, so we can actually use this code to solve that
> > problem too.  I wasn't thinking of USB when I wrote the code (because I
> > didn't know there was a problem).  But really, it's very generic stuff;
> > there's a list_head, a pointer to a Scsi_Host and a completion.
> > 
> > I'm not 100% sure what the problem is with USB. 
> 
> I seem to remember that the problem with USB was that you can't even
> know how many things are still outstanding; devices are free to appear
> on the bus at any time after bus power on, without specified timeout...
> If the kernel can't know what's out there, it can't wait until it has a
> complete picture either, just by virtue of not being able to know when
> that is achieved ;-)

This is true.

thanks,

greg k-h

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: [SPAM] Re: Asynchronous scsi scanning, version 9

[Doug Ledford]

On Mon, 2006-06-26 at 07:44 -0700, Matthew Dharm wrote:

> As for using one Scsi_Host... there are several usb-storage devices which
> attach to an entire SCSI bus (not just a single target), so can't make each
> device it's own target.

This makes no sense to me at all.  The USB devices attach to a USB bus,
the whole notion of a SCSI bus for USB devices is as nothing more than
an abstraction, and should be able to be arbitrarily changed in the USB
stack any time you wish.  In short, you aren't doing SCSI selections
over a USB bus, so why should SCSI target IDs vs. host IDs matter?
Aren't they just all mapped in the driver to the USB device anyway?

-- 
Doug Ledford <dledford@xsintricity.com>



Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Linas Vepstas]

On Sun, Jun 25, 2006 at 04:15:26PM -0500, James Bottomley wrote:
> On Thu, 2006-06-01 at 07:23 -0600, Matthew Wilcox wrote:
> 
> > There's a potential solution to this.  If we add a hook to sys_module_init
> > such that it'll call a specified function before returning, we can make
> > it wait until all scans are done.  This way, userspace never sees the
> > asynchronous scanning behaviour.  But if you have devices of different
> > types, you won't get the overlapping scans.
> 
> How about this solution which works for me with debian on a huge 6 bus
> scsi machine:  it introduces a new module scsi_wait_scan.ko whose sole
> job is to wait for the scans to complete in its init function.  The
> initrd/initramfs sequence now becomes:
> 
> insert all other modules
> modprobe scsi_wait_scan

I guess there's no way of figuring out where the root disk is, and
waiting only for that? 

On large servers, there may be hundreds of attached disks. Anything which 
can parallelize and perform an async scan will help boot times; however, 
waiting for all disks to come on-line when only one is actually needed can 
cause trouble.

In particular, some of the attached disks/scsi chains may be bad, 
in which case one has to wait for a tediously long device/bus/host reset
sequence to complete, and off-line the broken disk.

--linas


Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: [SPAM] Re: Asynchronous scsi scanning, version 9

[Matthew Dharm]

[-- Attachment #1.1: Type: text/plain, Size: 1353 bytes --]

On Mon, Jun 26, 2006 at 02:55:50PM -0400, Doug Ledford wrote:
> On Mon, 2006-06-26 at 07:44 -0700, Matthew Dharm wrote:
> 
> > As for using one Scsi_Host... there are several usb-storage devices which
> > attach to an entire SCSI bus (not just a single target), so can't make each
> > device it's own target.
> 
> This makes no sense to me at all.  The USB devices attach to a USB bus,
> the whole notion of a SCSI bus for USB devices is as nothing more than
> an abstraction, and should be able to be arbitrarily changed in the USB
> stack any time you wish.  In short, you aren't doing SCSI selections
> over a USB bus, so why should SCSI target IDs vs. host IDs matter?
> Aren't they just all mapped in the driver to the USB device anyway?

I guess my point is that there are some devices for which the concept of a
'bus' is -not- an abstraction.  These devices have a USB connector at one
end and a SCSI HD-68 connector on the other, and can be attached to 15
devices in a SCSI chain.

Perhaps I misunderstand your objection?

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

E:  You run this ship with Windows?!  YOU IDIOT!
L:  Give me a break, it came bundled with the computer!
					-- ESR and Lan Solaris
User Friendly, 12/8/1998

[-- Attachment #1.2: Type: application/pgp-signature, Size: 191 bytes --]

[-- Attachment #2: Type: text/plain, Size: 300 bytes --]

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

[-- Attachment #3: Type: text/plain, Size: 226 bytes --]

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Matthew Dharm]

[-- Attachment #1.1: Type: text/plain, Size: 1854 bytes --]

On Mon, Jun 26, 2006 at 09:18:28AM -0600, Matthew Wilcox wrote:
> On Mon, Jun 26, 2006 at 07:44:42AM -0700, Matthew Dharm wrote:
> > As for using one Scsi_Host... there are several usb-storage devices which
> > attach to an entire SCSI bus (not just a single target), so can't make each
> > device it's own target.
> 
> Oh.  My fault for reading the comment rather than the code.
> 
>                 /* reject if target != 0 or if LUN is higher than
>                  * the maximum known LUN
>                  */
>                 else if (us->srb->device->id && 
>                                 !(us->flags & US_FL_SCM_MULT_TARG)) {
> 
> Do these devices stick to occupying only target IDs from 0-7?  If not,
> you may wish to increase ->max_id for those devices.  I think it'd be worth
> exporting scsi_scan_channel() from the midlayer (and rearranging it to have
> __scsi_scan_channel() as was done with __scsi_scan_target) for USB's benefit.

These support target IDs to up 15.  But I see the point of limiting
->max_id for devices which do not have the SCM_MULT_TARG flag set.

> It's definitely possible to remove individual targets dynamically now;
> Fibre Channel has sorted that out (and will complain loudly if it breaks).
> The scsi core doesn't really have a channel object; channel is just an integer 
> that describes a path to a target.  So I think there should be no problem in
> converting USB to have one host and many channels.

Interesting.

What's the limit on the number of channels you can have?

How do I set up multiple channels in code?

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

What, are you one of those Microsoft-bashing Linux freaks?
					-- Customer to Greg
User Friendly, 2/10/1999

[-- Attachment #1.2: Type: application/pgp-signature, Size: 191 bytes --]

[-- Attachment #2: Type: text/plain, Size: 300 bytes --]

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

[-- Attachment #3: Type: text/plain, Size: 226 bytes --]

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[James Bottomley]

On Mon, 2006-06-26 at 15:58 -0500, Linas Vepstas wrote:
> I guess there's no way of figuring out where the root disk is, and
> waiting only for that? 

Well, sure, but you'd do that at user level:  you know what the root
disk is, so you just wait on the udev event announcing its arrival.

James




Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: [SPAM] Re: Asynchronous scsi scanning, version 9

[Doug Ledford]

On Mon, 2006-06-26 at 14:04 -0700, Matthew Dharm wrote:
> On Mon, Jun 26, 2006 at 02:55:50PM -0400, Doug Ledford wrote:
> > On Mon, 2006-06-26 at 07:44 -0700, Matthew Dharm wrote:
> > 
> > > As for using one Scsi_Host... there are several usb-storage devices which
> > > attach to an entire SCSI bus (not just a single target), so can't make each
> > > device it's own target.
> > 
> > This makes no sense to me at all.  The USB devices attach to a USB bus,
> > the whole notion of a SCSI bus for USB devices is as nothing more than
> > an abstraction, and should be able to be arbitrarily changed in the USB
> > stack any time you wish.  In short, you aren't doing SCSI selections
> > over a USB bus, so why should SCSI target IDs vs. host IDs matter?
> > Aren't they just all mapped in the driver to the USB device anyway?
> 
> I guess my point is that there are some devices for which the concept of a
> 'bus' is -not- an abstraction.  These devices have a USB connector at one
> end and a SCSI HD-68 connector on the other, and can be attached to 15
> devices in a SCSI chain.

Yuk!  OK, so such a nasty beast exists, but that just means
realistically you should treat such a device as a scsi host instead of a
scsi device.  For all other normal target devices, the issue of scsi bus
vs. scsi target shouldn't matter.

> Perhaps I misunderstand your objection?
> 
> Matt
> 
-- 
Doug Ledford <dledford@xsintricity.com>



Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Linas Vepstas]

On Mon, Jun 26, 2006 at 04:14:37PM -0500, James Bottomley wrote:
> On Mon, 2006-06-26 at 15:58 -0500, Linas Vepstas wrote:
> > I guess there's no way of figuring out where the root disk is, and
> > waiting only for that? 
> 
> Well, sure, but you'd do that at user level:  you know what the root
> disk is, so you just wait on the udev event announcing its arrival.

Its possible that I'm making a confused commont based on my
mis-understanding a conversation I'm not following closely, but ... 

Didn't the last patch result in the boot process hanging until
all scsi devices were discovered? I was concerned that this might 
take too long. But if there's no hang, then no problem.

--linas



Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[James Bottomley]

On Mon, 2006-06-26 at 16:21 -0500, Linas Vepstas wrote:
> Its possible that I'm making a confused commont based on my
> mis-understanding a conversation I'm not following closely, but ... 
> 
> Didn't the last patch result in the boot process hanging until
> all scsi devices were discovered? I was concerned that this might 
> take too long. But if there's no hang, then no problem.

The last patch was a module distros could use to wait for all bus scans
to complete.

If you simply want the behaviour where you launch aync scans and wait
for root to show up before proceeding, then you code that in udev.

James



Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Matthew Wilcox]

On Mon, Jun 26, 2006 at 02:08:25PM -0700, Matthew Dharm wrote:
> On Mon, Jun 26, 2006 at 09:18:28AM -0600, Matthew Wilcox wrote:
> > Do these devices stick to occupying only target IDs from 0-7?  If not,
> > you may wish to increase ->max_id for those devices.  I think it'd be worth
> > exporting scsi_scan_channel() from the midlayer (and rearranging it to have
> > __scsi_scan_channel() as was done with __scsi_scan_target) for USB's benefit.
> 
> These support target IDs to up 15.  But I see the point of limiting
> ->max_id for devices which do not have the SCM_MULT_TARG flag set.

Ever tried that?  ;-)

drivers/scsi/hosts.c sets max_id to 8 in scsi_host_alloc().  I don't see
anything in USB setting max_id to 16.

> > It's definitely possible to remove individual targets dynamically now;
> > Fibre Channel has sorted that out (and will complain loudly if it breaks).
> > The scsi core doesn't really have a channel object; channel is just an integer 
> > that describes a path to a target.  So I think there should be no problem in
> > converting USB to have one host and many channels.
> 
> Interesting.
> 
> What's the limit on the number of channels you can have?

Just an unsigned int, I think.

> How do I set up multiple channels in code?

Increase shost->max_channel each time you want to create a new one.
Then pass it to scsi_scan_target().


Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[Hannes Reinecke]

James Bottomley wrote:
> On Mon, 2006-06-26 at 15:58 -0500, Linas Vepstas wrote:
>> I guess there's no way of figuring out where the root disk is, and
>> waiting only for that? 
> 
> Well, sure, but you'd do that at user level:  you know what the root
> disk is, so you just wait on the udev event announcing its arrival.
> 
Exactly. And it works well.

The only point left addressing is that you never know how long you 
should be waiting. It would be nice if the transport class / SCSI ML 
could give some hint somewhere to the effect 'scanning in progress'.
Otherwise you'll end up using arbitrary timeouts and you'll always find 
machines where this timeout doesn't work :-(

Cheers,

Hannes
-- 
Dr. Hannes Reinecke			hare@suse.de
SuSE Linux Products GmbH		S390 & zSeries
Maxfeldstraße 5				+49 911 74053 688
90409 Nürnberg				http://www.suse.de

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[James Bottomley]

On Wed, 2006-06-28 at 09:52 +0200, Hannes Reinecke wrote:
> The only point left addressing is that you never know how long you 
> should be waiting. It would be nice if the transport class / SCSI ML 
> could give some hint somewhere to the effect 'scanning in progress'.
> Otherwise you'll end up using arbitrary timeouts and you'll always
> find 
> machines where this timeout doesn't work :-(

Well ... about 20s for a parallel bus is usual.  All the other modern
busses (FC, SAS etc.) are unscanned, so there's nothing to wait for at
the mid-layer.  Each does, however, have an internal discovery type
process which it might be possible to expose via a new API.

James



Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Asynchronous scsi scanning, version 9

[maximilian attems]

On Thu, 01 Jun 2006, Kay Sievers wrote:

> On Wed, 2006-05-31 at 16:21 -0700, Patrick Mansfield wrote:
> > [adding hotplug-devel ... maybe Marco or Kay can comment]
> > 
> > On Mon, May 29, 2006 at 07:05:15AM -0600, Matthew Wilcox wrote:
> > > On Mon, May 29, 2006 at 10:38:13AM +0200, Stefan Richter wrote:
<snipp unrelated>
> > > 
> > > That's what scsi_complete_async_scans() is for.  If you have a built-in
> > > module, it will wait for the async scans to finish before we get as far
> > > as trying to mount root.  It does change observable behaviour in that
> > > sys_module_init() will return before scans are complete.  However, I
> > > believe most distros userspace copes with this these days.  For example,
> > > Debian has:
> > > 
> > >     # wait for the udevd childs to finish
> > >     log_action_begin_msg "Waiting for /dev to be fully populated"
> > >     while [ -d /dev/.udev/queue/ ]; do
> > >         sleep 1
> > >         udevd_timeout=$(($udevd_timeout - 1))
> > > [...]
> 
> That has replaced by a binary called "udevsettle" which waits for events
> to finish, by comparing the current kernel event sequence number
> exported in sysfs with the latest handled event by udev.

usb-storage is still giving troubles in that area.
in the case of usb-storage udevsettle exists much too early.
the /sys uevent_seqnum is the same as the udev worked on,
while dmesg is saying:
usb-storage: waiting for device to settle before scanning

it would be really helpful if udevsettle would have an uevent
to wait on. [adding gregkh to cc]

 
> > second, and the udev queue becomes empty even though the scsi /sd scan is
> > still in progress.
> 
> Right. For the settle time of usb-storage we watch for the kernel tread
> to go away. :)

bug reports don't comfirm that statement.
nor do i see any code for it in udevsettle.c.
 
-- 
maks

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel