Re: [PATCH] serialize bus scanning

[Patrick Mansfield <patmans@us.ibm.com>]

[-- Attachment #1: Type: text/plain, Size: 2379 bytes --]

On Wed, Sep 10, 2003 at 05:57:42PM +1000, Anton Blanchard wrote:
> 
> How does this look? This restores the ability to do parallel scsi scan
> via sysfs.
> 

The patch looks fine to me, I hope James will apply it.

I think we might still have to allow only a scan or delete (per host), and
not both in parallel - it looks like we could have some races otherwise,
I'm not sure though (mainly where the driver model points to the parent
device). Maybe a get of the host device during scanning is enough.

I tested it out, and ran some timings pre and post-patch, mainly out of
curiousity.

I have:

[root@elm3b79 root]# ls -1d /sysfs/bus/scsi/devices/[01]* | wc
      6       6     192
[root@elm3b79 root]# ls -1d /sysfs/bus/scsi/devices/[2-5]* | wc
    288     288    9428

Really 72 separate LUNs attached to qlogic 2310 (FCP) host adapters (hosts
2-5), each with four paths. 40 disk drives (10 on a loop per enclosure),
plus 32 LUNs connected to a disk array. All attached to a 1Gb switch.

I ran a simple script (attached) to scan via sysfs in parallel. Of course
without the patch scanning is serialized.

System is a 8 cpu (two node) NUMAQ.

--------------------------------------------
Before the patch:

Rescanning the four qlogic adapters without finding any new devices,
seemed to vary up to about 34 seconds:

[root@elm3b79 root]# time ./scanall.sh > xx

real    0m31.947s
user    0m0.007s
sys     0m0.113s

Rescan in parallel re-discovering all devices:

[root@elm3b79 root]# time ./scanall.sh  > xx

real    0m34.189s
user    0m0.009s
sys     0m0.615s

--------------------------------------------
With the patch:

Generally went from 34 seconds down to 9.

Rescanning the four qlogic adapters without finding any new devices:

[root@elm3b79 root]# time ./scanall.sh > xx

real    0m8.057s
user    0m0.007s
sys     0m0.120s

Rescan in parallel re-discovering all devices:

[root@elm3b79 root]# time ./scanall.sh > xx

real    0m9.608s
user    0m0.008s
sys     0m0.783s

-----------------------------------------------------------

Also: time to sequentially remove all devices (on host 2-5, patch does not
matter but I find this interesting):

time ./delall.sh > xx

real    0m7.276s
user    0m0.823s
sys     0m3.106s

And time to remove in parallel

[root@elm3b79 root]# time ./delall.sh > xx 
real    0m4.586s
user    0m1.336s
sys     0m7.204s

-- Patrick Mansfield

[-- Attachment #2: scanall.sh --]
[-- Type: application/x-sh, Size: 238 bytes --]

[-- Attachment #3: delall.sh --]
[-- Type: application/x-sh, Size: 198 bytes --]