/proc/scsi/map

/proc/scsi/map

[Kurt Garloff]

[-- Attachment #1.1: Type: text/plain, Size: 4733 bytes --]

Hi SCSI users,

from people using SCSI devices, there's one question that turns up again 
and again: How can one assign stable device names to SCSI devices in
case there are devices that may or may not be switched on or connected.

There are a couple of ways to address this problem:
(a) mount by uuid
(b) userspace programs that collect information to create
    alternative and persistent names / device nodes, such
    as Eric Youngdale's scsidev[1], Doug Gilbert's sg_map[2], scsimon[3],
    or Mike Sullivan's scsiname/devnaming[4]
(c) devfs

[1] http://www.garloff.de/kurt/linux/scsidev/
[2] http://www.torque.net/sg/
[3] http://www.torque.net/scsi/scsimon.html
[4] http://oss.software.ibm.com/devreg/

Unfortunately, those approaches all have some deficiencies.
Ad (a): Does only work for ext2 filesystems. For locating
        / one needs additional initrd work.
Ad (b): A considerable amount of work needs to be done in userspace:
        - For all devices types you need to probe possible devices
	- You need to do SCSI_IOCTL_GET_IDLUN to get controller,bus,
	  target and unit number
	The problem is that the collection of this information is 
	not always successful. If a medium is not inserted, the
	open() fails for some device types, despite O_NONBLOCK.
	Assumptions about the order of devices OTOH are not safe,
	as remove-single-device/add-single-device may result in a
	non-straightforward ordering.
Ad (c): devfs is currently not (yet?) an option for distributions
        due to security and stability considerations.

Life would be easier if the scsi subsystem would just report which SCSI
device (uniquely identified by the controller,bus,target,unit tuple) belongs
to which high-level device. The information is available in the kernel.

Attached patch does this:
garloff@pckurt:/raid5/Kernel/src $ cat /proc/scsi/map 
# C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
0,0,00,00       0x05    1       sg0     c:15:00 sr0     b:0b:00
1,0,01,00       0x05    1       sg1     c:15:01 sr1     b:0b:01
1,0,02,00       0x01    1       sg2     c:15:02 osst0   c:ce:00
1,0,03,00       0x05    1       sg3     c:15:03 sr2     b:0b:02
1,0,05,00       0x00    1       sg4     c:15:04 sda     b:08:00
1,0,09,00       0x00    1       sg5     c:15:05 sdb     b:08:10
2,0,01,00       0x05    1       sg6     c:15:06 sr3     b:0b:03
2,0,02,00       0x01    1       sg7     c:15:07 osst1   c:ce:01
2,0,03,00       0x05    1       sg8     c:15:08 sr4     b:0b:04
2,0,05,00       0x00    1       sg9     c:15:09 sdc     b:08:20
2,0,09,00       0x00    1       sg10    c:15:0a sdd     b:08:30
3,0,10,00       0x00    1       sg11    c:15:0b sde     b:08:40
3,0,12,00       0x00    1       sg12    c:15:0c sdf     b:08:50

This allows a simple script to parse the map and create device nodes as
needed.

The patch does work the following way.
- Add a find_kdev() function pointer to the high-level driver template
  structure. The function takes a Scsi_Device pointer (points to a 
  low-level device) and returns a name and a kdev_t if the device
  is attached to this high-level driver.
- Implement the function for sg, sd, sr, st, osst
- Make scsi/scsi_proc iterate over all devices and calls the high-level
  drivers find_kdev() to find out about it.

Obviously, it can only report the assignment of high-level drivers,
if they are loaded, otherwise the last two columns will stay empty.
(sg is handled especially, as we know it supports all devices.)
If we attach a third high-level device driver, two more columns would show 
up. 
(Is this variable column number format a problem?)

The patch also includes a simple shell script that does assign
/dev/scsi/sdc2b0t9u0 type names to those devices and making a device nodes
(or optionally symlinks to the old name devices) with this name.

The design allows for two more things:
* using root=/dev/scsi/sdc1b0t9u0p5 without much additional code
  (patch follows in another mail)
* in case we want to support more than 128 scsi disks, the information
  about additional majors can be reported by /proc/scsi/map without further
  change

Patch is against 2.4.19pre10.

I'd like to get it accepted into the kernel. 
So please give your criticism ... 
I already got some by Doug Gilbert :-)

A patch for 2.5 should be done as well, if the design is OK, of course.

Regards,
-- 
Kurt Garloff                   <kurt@garloff.de>         [Eindhoven, NL]
Physics: Plasma simulations    <K.Garloff@TUE.NL>     [TU Eindhoven, NL]
Linux: SCSI, Security          <garloff@suse.de>    [SuSE Nuernberg, DE]
 (See mail header or public key servers for PGP2 and GPG public keys.)

[-- Attachment #1.2: scsi-map6.diff --]
[-- Type: text/plain, Size: 17108 bytes --]

diff -uNr linux-2.4.18.S18.fixed/Documentation/scsi-devs.sh linux-2.4.18.S18.sdmany/Documentation/scsi-devs.sh
--- linux-2.4.18.S18.fixed/Documentation/scsi-devs.sh	Thu Jan  1 01:00:00 1970
+++ linux-2.4.18.S18.sdmany/Documentation/scsi-devs.sh	Fri Jun 14 22:37:11 2002
@@ -0,0 +1,154 @@
+#!/bin/bash
+# Script to create SCSI device nodes according to their physical
+# address (host controller no,bus/channel,target SCSI ID,unit SCSI LUN)
+# using the /proc/scsi/map introduced in Linux 2.4.20pre/2.5.2x
+# (c) Kurt Garloff <garloff@suse.de>, 2002-06-13
+# License: GNU GPL v2
+
+# Settings
+
+# Target directory for device nodes/links
+TDIR=/dev/scsi
+
+# Make symbolic links or create fresh nodes in $TDIR
+# For link mode, the dev nodes will still be made if needed
+MODE=
+
+# In link mode, where is the real /dev dir ?
+DEVDIR=..
+
+# Ownership and permissions of newly created device nodes
+OWNER="root.disk"
+PERM="0660"
+SDIRPERM="0755"
+
+# Make scsi disk partitions (0 = off)
+SDPART=15
+
+# End of settings
+
+# Options
+# -f: Remove all old devs
+if test "$1" = "-f"; then
+  rm -f $TDIR/*
+  shift
+fi
+# -l: Make links 
+if test "$1" = "-l"; then
+  MODE=LINK
+  shift
+fi
+
+# Sanitize environment
+export LANG=posix
+unset IFS
+#umask `printf %o $[0777-$PERM]`
+PATH=/bin:/usr/bin:/sbin:/usr/sbin
+
+# Functions
+function exit_fail 
+{
+  echo $1
+  exit $2
+}
+
+# Build new scsi name based on oldname (prefix) and 
+# append host,channel,id,lun numbers that are unique
+# and persistent identifiers for the device.
+function builddevnm
+{ 
+  if test "${5:0:2}" = "sd"; then
+    NM="sdc$1b$2t${3#0}u${4#0}"
+  else
+    NM="${5%%[0-9]*}c$1b$2t${3#0}u${4#0}"
+  fi
+}
+
+# Make device node
+# Parameters filename type maj(hex) min(hex) minadd(dec)
+function mk_nod
+{
+  rm -f $1
+  mknod -m $PERM $1 $2 $[0x$3] $[0x$4+$5]
+  chown $OWNER $1
+}
+
+# Make device by creating node or symlinking
+# Parameters oldname device(tp:maj:min) newname minadd
+function mk_dev 
+{
+  IFS=":"
+  if test "$MODE" = "LINK"; then
+    if test ! -e $TDIR/$DEVDIR/$1; then
+      mk_nod $TDIR/$DEVDIR/$1 $2 $4
+    fi
+    ln -sf $DEVDIR/$1 $TDIR/$3
+  else
+    mk_nod $TDIR/$3 $2 $4
+  fi
+  unset IFS
+}
+
+# Make multiple devices in case we do have sd partitions
+# Parameters oldname device(tp:maj:min) newname
+function mk_devs
+{
+  mk_dev $1 $2 $3 0
+  # Handle partitions
+  if test "${1:0:2}" = "sd" -a "$SDPART" != "0"; then
+    #unset IFS
+    for no in `seq 1 $SDPART`; do
+      mk_dev $1$no $2 $3p$no $no
+    done
+  fi
+  # Handle nst/nosst
+  if test "${1:0:2}" = "st" -o "${1:0:4}" = "osst"; then
+    mk_dev n$1 $2 n$3 128
+  fi
+}
+
+# Main() Script
+if ! test -e /proc/scsi/map; then
+  exit_fail "/proc/scsi/map does not exist" 1
+fi
+if ! test -d $TDIR; then
+  mkdir -m $SDIRMODE $TDIR || exit_fail "Failed to create $TDIR" 2
+  chown $OWNER $TDIR
+fi
+
+# We might have been called by some sort of hotplug event
+# and are only support to add a single device
+# Syntax for this is [-l] add host channel id lun
+if test "$1" = "add"; then
+  CMPAGAINST="$2,$3,`printf %02i $4`,`printf %02i $5`"
+else
+  unset CMPAGAINST
+fi
+# The main processing loop
+while read cbtu tp onl sgnm sgdev othnm othdev oothnm oothdev rest; do
+  # Skip comment line(s)
+  if test "${cbtu:0:1}" = "#"; then continue; fi
+  # If we're just dealing with one device, do skip the others
+  if test ! -z "$CMPAGAINST" -a "$CMPAGAINST" != "$cbtu"; then continue; fi
+  # now parse line
+  IFS=","
+  # Test for validity of sg device
+  if test "$sgnm" != "sg?"; then
+    builddevnm $cbtu $sgnm
+    mk_dev $sgnm $sgdev $NM 0
+  fi
+  # There is possibly a second device
+  if test ! -z "$othnm" -a ! "$othnm" = "none"; then
+    IFS=","
+    builddevnm $cbtu $othnm
+    mk_devs $othnm $othdev $NM
+    # Maybe even a third one
+    if test ! -z "$oothnm" -a ! "$oothnm" = "none"; then
+      IFS=","
+      builddevnm $cbtu $oothnm
+      mk_devs $oothnm $oothdev $NM
+    fi
+  fi
+  unset IFS
+done < /proc/scsi/map
+
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/hosts.h linux-2.4.18.S18.sdmany/drivers/scsi/hosts.h
--- linux-2.4.18.S18.fixed/drivers/scsi/hosts.h	Wed Jun 12 11:37:09 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/hosts.h	Wed Jun 12 11:53:54 2002
@@ -530,6 +530,7 @@
     void (*detach)(Scsi_Device *);
     int (*init_command)(Scsi_Cmnd *);     /* Used by new queueing code. 
                                            Selects command for blkdevs */
+    int (*find_kdev)(Scsi_Device *, char*, kdev_t*);  /* find back dev. */
 };
 
 void  scsi_initialize_queue(Scsi_Device * SDpnt, struct Scsi_Host * SHpnt);
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/osst.c linux-2.4.18.S18.sdmany/drivers/scsi/osst.c
--- linux-2.4.18.S18.fixed/drivers/scsi/osst.c	Fri Dec 21 18:41:55 2001
+++ linux-2.4.18.S18.sdmany/drivers/scsi/osst.c	Wed Jun 12 11:39:47 2002
@@ -156,6 +156,7 @@
 static int osst_attach(Scsi_Device *);
 static int osst_detect(Scsi_Device *);
 static void osst_detach(Scsi_Device *);
+static int osst_find_kdev(Scsi_Device *, char*, kdev_t*);
 
 struct Scsi_Device_Template osst_template =
 {
@@ -166,7 +167,8 @@
        detect:		osst_detect,
        init:		osst_init,
        attach:		osst_attach,
-       detach:		osst_detach
+       detach:		osst_detach,
+       find_kdev:	osst_find_kdev,
 };
 
 static int osst_int_ioctl(OS_Scsi_Tape *STp, Scsi_Request ** aSRpnt, unsigned int cmd_in,unsigned long arg);
@@ -5417,6 +5419,24 @@
 	return 0;
 }
 
+static int osst_find_kdev(Scsi_Device *sdp, char* nm, kdev_t *dev)
+{
+	int i;
+	OS_Scsi_Tape *ostp;
+	
+	if (sdp && sdp->type == TYPE_TAPE && osst_supports(sdp)) {
+		for (ostp = os_scsi_tapes[i = 0]; i < osst_template.dev_max;
+		     ostp = os_scsi_tapes[++i]) {
+			if (ostp && ostp->device == sdp) {
+				sprintf (nm, "osst%i", i);
+				*dev = MKDEV(OSST_MAJOR, i);
+				return 0;
+			}
+		}
+	}
+	return 1;
+}
+
 static int osst_attach(Scsi_Device * SDp)
 {
 	OS_Scsi_Tape * tpnt;
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/scsi.c linux-2.4.18.S18.sdmany/drivers/scsi/scsi.c
--- linux-2.4.18.S18.fixed/drivers/scsi/scsi.c	Wed Jun 12 11:37:07 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/scsi.c	Fri Jun 14 12:49:15 2002
@@ -80,6 +80,7 @@
 
 #ifdef CONFIG_PROC_FS
 static int scsi_proc_info(char *buffer, char **start, off_t offset, int length);
+static int scsi_proc_map (char *buffer, char **start, off_t offset, int length);
 static void scsi_dump_status(int level);
 #endif
 
@@ -1554,6 +1555,70 @@
 }
 
 #ifdef CONFIG_PROC_FS
+/* 
+ * Output the mapping of physical devices (contorller,channel.id,lun)
+ * to devices (sg and other highlevel drivers) to /proc/scsi/map.
+ * Caveat: No locking is done, so if your scsi config changes during
+ * reading this file, you may read garbled data.	KG, 2002-06-14
+ */
+static int scsi_proc_map(char *buffer, char **start, off_t offset, int length)
+{
+	Scsi_Device *scd;
+	struct Scsi_Host *HBA_ptr;
+	int size = 0, len = 0, repeat = 0;
+	off_t begin = 0;
+	off_t pos = 0;
+
+	struct Scsi_Device_Template *sg_t;
+	do {
+		sg_t = scsi_devicelist;
+		while (sg_t && !(sg_t->tag && !strcmp(sg_t->tag, "sg")))
+			sg_t = sg_t->next;
+#ifdef CONFIG_KMOD
+		if (!repeat && !sg_t)
+			request_module("sg");
+	} while (!repeat++);
+#else
+	} while (0);
+#endif	
+	/*
+	 * First, see if there are any attached devices or not.
+	 */
+	for (HBA_ptr = scsi_hostlist; HBA_ptr; HBA_ptr = HBA_ptr->next) {
+		if (HBA_ptr->host_queue != NULL) {
+			break;
+		}
+	}
+	if (!HBA_ptr)
+		goto stop_map_output;
+	
+	size = sprintf(buffer + len, "# C,B,T,U\tType\tonl\tsg_nm\tsg_dev\tnm\tdev(hex)\n");
+	len += size;
+	pos = begin + len;
+
+	for (HBA_ptr = scsi_hostlist; HBA_ptr; HBA_ptr = HBA_ptr->next) {
+		for (scd = HBA_ptr->host_queue; scd; scd = scd->next) {
+			proc_print_scsimap(scd, buffer, &size, len, sg_t);
+			len += size;
+			pos = begin + len;
+
+			if (pos < offset) {
+				len = 0;
+				begin = pos;
+			}
+			if (pos > offset + length)
+				goto stop_map_output;
+		}
+	}
+
+ stop_map_output:
+	*start = buffer + (offset - begin);	/* Start of wanted data */
+	len -= (offset - begin);	/* Start slop */
+	if (len > length)
+		len = length;	/* Ending slop */
+	return (len);
+}
+
 static int scsi_proc_info(char *buffer, char **start, off_t offset, int length)
 {
 	Scsi_Device *scd;
@@ -2578,6 +2643,7 @@
 static int __init init_scsi(void)
 {
 	struct proc_dir_entry *generic;
+	struct proc_dir_entry *map;
 
 	printk(KERN_INFO "SCSI subsystem driver " REVISION "\n");
 
@@ -2602,6 +2668,13 @@
 		return -ENOMEM;
 	}
 	generic->write_proc = proc_scsi_gen_write;
+	
+	map = create_proc_info_entry ("scsi/map", 0, 0, scsi_proc_map);
+	if (!map) {
+		printk (KERN_ERR "cannot init /proc/scsi/map\n");
+		remove_proc_entry("scsi", 0);
+		return -ENOMEM;
+	}
 #endif
 
         scsi_devfs_handle = devfs_mk_dir (NULL, "scsi", NULL);
@@ -2636,6 +2709,7 @@
 
 #ifdef CONFIG_PROC_FS
 	/* No, we're not here anymore. Don't show the /proc/scsi files. */
+	remove_proc_entry ("scsi/map", 0);
 	remove_proc_entry ("scsi/scsi", 0);
 	remove_proc_entry ("scsi", 0);
 #endif
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/scsi.h linux-2.4.18.S18.sdmany/drivers/scsi/scsi.h
--- linux-2.4.18.S18.fixed/drivers/scsi/scsi.h	Wed Jun 12 11:37:07 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/scsi.h	Wed Jun 12 11:53:54 2002
@@ -517,6 +517,8 @@
  * Prototypes for functions in scsi_proc.c
  */
 extern void proc_print_scsidevice(Scsi_Device *, char *, int *, int);
+extern void proc_print_scsimap(Scsi_Device *, char *, int *, int, 
+			       struct Scsi_Device_Template *);
 extern struct proc_dir_entry *proc_scsi;
 
 /*
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/scsi_proc.c linux-2.4.18.S18.sdmany/drivers/scsi/scsi_proc.c
--- linux-2.4.18.S18.fixed/drivers/scsi/scsi_proc.c	Thu Jun 28 02:10:55 2001
+++ linux-2.4.18.S18.sdmany/drivers/scsi/scsi_proc.c	Fri Jun 14 00:30:42 2002
@@ -301,11 +301,66 @@
 	return;
 }
 
+
+
+
+void proc_print_scsimap(Scsi_Device *scd, char *buffer, int *size, int len,
+			struct Scsi_Device_Template *sg_t)
+{
+	int y, err = 0;
+	char nm[16];
+	kdev_t kdev; 
+	int att = scd->attached;
+	struct Scsi_Device_Template *sd_t = scsi_devicelist;
+
+	y = sprintf(buffer + len,
+	     "%i,%i,%02i,%02i\t0x%02x\t%i",
+		    scd->host->host_no, scd->channel, scd->id, scd->lun,
+		    scd->type, scd->online);
+	if (sg_t && sg_t->find_kdev) {
+		err = sg_t->find_kdev(scd, nm, &kdev);
+		if (!err) {
+			y += sprintf(buffer + len + y,
+				     "\t%s\tc:%02x:%02x", nm, MAJOR(kdev), MINOR(kdev));
+			--att;
+		} else {
+			printk (KERN_ERR "scsimap: sg device not found?!\n");
+			y += sprintf(buffer + len + y,
+				     "\tsg?\tc:NN:NN");
+		}
+	} else {
+		printk (KERN_INFO "scsimap: need sg support to report sg devices\n");
+		y += sprintf(buffer + len + y,
+			     "\tsg?\tc:NN:NN");
+	}
+	while (att && sd_t) {
+		if (sd_t->scsi_type != 0xff && sd_t->find_kdev) {
+			err = sd_t->find_kdev(scd, nm, &kdev);
+			if (!err) {
+				y += sprintf(buffer + len + y,
+					     "\t%s\t%c:%02x:%02x",
+					     nm, (sd_t->blk? 'b': 'c'),
+					     MAJOR(kdev), MINOR(kdev));
+				--att;
+			}
+		}
+		sd_t = sd_t->next;
+	}
+
+ map_out:
+	y += sprintf(buffer + len + y, "\n");
+	*size = y;
+	return;
+}
+
 #else				/* if !CONFIG_PROC_FS */
 
 void proc_print_scsidevice(Scsi_Device * scd, char *buffer, int *size, int len)
 {
 }
+void proc_print_scsimap(Scsi_Device *scd, char *buffer, int *size, int len)
+{
+}
 
 #endif				/* CONFIG_PROC_FS */
 
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/sd.c linux-2.4.18.S18.sdmany/drivers/scsi/sd.c
--- linux-2.4.18.S18.fixed/drivers/scsi/sd.c	Wed Jun 12 11:37:13 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/sd.c	Wed Jun 12 11:39:47 2002
@@ -109,6 +109,7 @@
 static int sd_detect(Scsi_Device *);
 static void sd_detach(Scsi_Device *);
 static int sd_init_command(Scsi_Cmnd *);
+static int sd_find_kdev(Scsi_Device*, char*, kdev_t*);
 
 static struct Scsi_Device_Template sd_template = {
 	name:"disk",
@@ -127,6 +128,7 @@
 	attach:sd_attach,
 	detach:sd_detach,
 	init_command:sd_init_command,
+	find_kdev:sd_find_kdev,
 };
 
 
@@ -281,6 +283,23 @@
 	}
 }
 
+static int sd_find_kdev(Scsi_Device *sdp, char* nm, kdev_t *dev)
+{
+	Scsi_Disk *dp; 
+	int i;
+	
+	if (sdp && (sdp->type == TYPE_DISK || sdp->type == TYPE_MOD)) {
+		for (dp = rscsi_disks, i = 0; i < sd_template.dev_max; ++i, ++dp) {
+			if (dp->device == sdp) {
+				sd_devname(i, nm);
+				*dev = MKDEV_SD(i);
+				return 0;
+			}
+		}
+	}
+	return 1;
+}
+
 static request_queue_t *sd_find_queue(kdev_t dev)
 {
 	Scsi_Disk *dpnt;
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/sg.c linux-2.4.18.S18.sdmany/drivers/scsi/sg.c
--- linux-2.4.18.S18.fixed/drivers/scsi/sg.c	Wed Jun 12 11:37:04 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/sg.c	Wed Jun 12 15:27:55 2002
@@ -115,6 +115,7 @@
 static void sg_finish(void);
 static int sg_detect(Scsi_Device *);
 static void sg_detach(Scsi_Device *);
+static int sg_find_kdev(Scsi_Device *, char*, kdev_t*);
 
 static Scsi_Request * dummy_cmdp;	/* only used for sizeof */
 
@@ -123,6 +124,7 @@
 
 static struct Scsi_Device_Template sg_template =
 {
+      name:"generic",
       tag:"sg",
       scsi_type:0xff,
       major:SCSI_GENERIC_MAJOR,
@@ -130,7 +132,8 @@
       init:sg_init,
       finish:sg_finish,
       attach:sg_attach,
-      detach:sg_detach
+      detach:sg_detach,
+      find_kdev:sg_find_kdev
 };
 
 
@@ -2696,6 +2699,36 @@
 }
 
 #ifdef CONFIG_PROC_FS
+static int sg_find_kdev(Scsi_Device* sdp, char *nm, kdev_t *dev)
+{
+    unsigned long iflags;
+    int err = 1; 
+
+    if (sdp && sg_dev_arr) {
+	int k;
+	read_lock_irqsave(&sg_dev_arr_lock, iflags);
+	for (k = 0; k < sg_template.dev_max; ++k) {
+	    if (sg_dev_arr[k] && sg_dev_arr[k]->device == sdp) {
+		sprintf (nm, "sg%i", k);
+	        *dev = sg_dev_arr[k]->i_rdev;
+		err = 0;
+		break;
+	    }
+	}
+	read_unlock_irqrestore(&sg_dev_arr_lock, iflags);
+    }
+    return err;
+}
+#else
+/* Not needed without procfs support */
+static int sg_find_kdev(Scsi_Device* sdp, char *nm, kdev_t *dev)
+{
+    *nm = 0; *kdev = MKDEV(255,255);
+    return 1;
+}
+#endif
+
+#ifdef CONFIG_PROC_FS
 
 static struct proc_dir_entry * sg_proc_sgp = NULL;
 
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/sr.c linux-2.4.18.S18.sdmany/drivers/scsi/sr.c
--- linux-2.4.18.S18.fixed/drivers/scsi/sr.c	Wed Jun 12 11:37:15 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/sr.c	Wed Jun 12 11:39:47 2002
@@ -69,6 +69,8 @@
 
 static int sr_init_command(Scsi_Cmnd *);
 
+static int sr_find_kdev(Scsi_Device*, char*, kdev_t*);
+
 static struct Scsi_Device_Template sr_template =
 {
 	name:"cdrom",
@@ -81,7 +83,8 @@
 	finish:sr_finish,
 	attach:sr_attach,
 	detach:sr_detach,
-	init_command:sr_init_command
+	init_command:sr_init_command,
+	find_kdev:sr_find_kdev,
 };
 
 Scsi_CD *scsi_CDs;
@@ -586,6 +589,22 @@
 	return 0;
 }
 
+static int sr_find_kdev(Scsi_Device *sdp, char* nm, kdev_t *dev)
+{
+	Scsi_CD *srp;
+	int i;
+	
+	if (sdp && (sdp->type == TYPE_ROM || sdp->type == TYPE_WORM)) {
+		for (srp = scsi_CDs, i = 0; i < sr_template.dev_max; ++i, ++srp) {
+			if (srp->device == sdp) {
+				sprintf(nm, "sr%i", i);
+				*dev = MKDEV(SCSI_CDROM_MAJOR,i);
+				return 0;
+			}
+		}
+	}
+	return 1;
+}
 
 void get_sectorsize(int i)
 {
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/st.c linux-2.4.18.S18.sdmany/drivers/scsi/st.c
--- linux-2.4.18.S18.fixed/drivers/scsi/st.c	Mon Feb 25 20:38:04 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/st.c	Wed Jun 12 11:39:47 2002
@@ -164,6 +164,7 @@
 static int st_attach(Scsi_Device *);
 static int st_detect(Scsi_Device *);
 static void st_detach(Scsi_Device *);
+static int st_find_kdev(Scsi_Device*, char*, kdev_t*);
 
 static struct Scsi_Device_Template st_template =
 {
@@ -174,7 +175,8 @@
 	detect:st_detect, 
 	init:st_init,
 	attach:st_attach, 
-	detach:st_detach
+	detach:st_detach,
+	find_kdev:st_find_kdev,
 };
 
 static int st_compression(Scsi_Tape *, int);
@@ -3827,6 +3829,23 @@
 	return 1;
 }
 
+static int st_find_kdev(Scsi_Device * sdp, char* nm, kdev_t *dev)
+{
+	int i;
+	Scsi_Tape *stp;
+	
+	if (sdp && sdp->type == TYPE_TAPE && !st_incompatible(sdp)) {
+		for (stp = scsi_tapes[0], i = 0; i < st_template.dev_max; stp=scsi_tapes[++i]) {
+			if (stp && stp->device == sdp) {
+				sprintf(nm, "st%i", i);
+				*dev = MKDEV (SCSI_TAPE_MAJOR, i);
+				return 0;
+			}
+		}
+	}
+	return 1;
+}
+
 static int st_registered = 0;
 
 /* Driver initialization (not __init because may be called later) */

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: /proc/scsi/map

[John Summerfield]

> 
> Life would be easier if the scsi subsystem would just report which SCSI
> device (uniquely identified by the controller,bus,target,unit tuple) belongs
> to which high-level device. The information is available in the kernel.


Does this not fail if I pull a device off, change its ID (perhaps to fit into another system), then plug it in again? Or if I move it from one adaptor to another?





-- 
Cheers
John Summerfield

Microsoft's most solid OS: http://www.geocities.com/rcwoolley/

Note: mail delivered to me is deemed to be intended for me, for my disposition.

==============================
If you don't like being told you're wrong,
	be right!

Re: /proc/scsi/map

[Richard Gooch]

Kurt Garloff writes:
> Hi SCSI users,
> 
> from people using SCSI devices, there's one question that turns up again=20
> and again: How can one assign stable device names to SCSI devices in
> case there are devices that may or may not be switched on or connected.
> 
> There are a couple of ways to address this problem:
[...]
> (c) devfs
[...]
> Unfortunately, those approaches all have some deficiencies.
[...]
> Ad (c): devfs is currently not (yet?) an option for distributions
>         due to security and stability considerations.

Mandrake is using devfs. And the security and stability issues have
been fixed many months ago. The "devfs races" that Al used to complain
about regularly have been fixed. I haven't heard from Al for many
months (I see that as a positive sign:-). The current devfs code is in
maintenance mode. The next release of code will be a new devfs core
which uses the VFS for tree maintenance, making the code much smaller.
I.e. not a bugfixing release.

If there *are* remaining bugs with the devfs core (or devfsd for that
matter), I've not been made aware of them. If you know something I
don't, please let me know. AFAICT, all the bugs are long since solved.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: /proc/scsi/map

[Sancho Dauskardt]

>Life would be easier if the scsi subsystem would just report which SCSI
>device (uniquely identified by the controller,bus,target,unit tuple) belongs
>to which high-level device. The information is available in the kernel.
>
>Attached patch does this:
>garloff@pckurt:/raid5/Kernel/src $ cat /proc/scsi/map
># C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
>0,0,00,00       0x05    1       sg0     c:15:00 sr0     b:0b:00
[...]

Great, this was really missing badly.

But how about adding another column: GUID.
Most usb-storage and (all?) FireWire devices have such a unique identitiy.
In contrast to native SCSI devices, these emulated SCSI devices on 
hot-plugging busses will change their LUNs/IDs. Therefor the GUID is really 
a must to be able to create stable names (laptop suspend, etc.).

Both usb-storage and iee1394-sbp2 know the GUID. It only needs to be 
communicated..

I'd guess that FibreChannel has similar problems ?

- sda

Re: /proc/scsi/map

[Albert D. Cahalan]

Kurt Garloff writes:
> garloff@pckurt:/raid5/Kernel/src $ cat /proc/scsi/map
> # C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
> 0,0,00,00       0x05    1       sg0     c:15:00 sr0     b:0b:00
> 1,0,01,00       0x05    1       sg1     c:15:01 sr1     b:0b:01
> 1,0,02,00       0x01    1       sg2     c:15:02 osst0   c:ce:00
> 1,0,03,00       0x05    1       sg3     c:15:03 sr2     b:0b:02
> 1,0,05,00       0x00    1       sg4     c:15:04 sda     b:08:00
> 1,0,09,00       0x00    1       sg5     c:15:05 sdb     b:08:10
> 2,0,01,00       0x05    1       sg6     c:15:06 sr3     b:0b:03
> 2,0,02,00       0x01    1       sg7     c:15:07 osst1   c:ce:01
> 2,0,03,00       0x05    1       sg8     c:15:08 sr4     b:0b:04
> 2,0,05,00       0x00    1       sg9     c:15:09 sdc     b:08:20
> 2,0,09,00       0x00    1       sg10    c:15:0a sdd     b:08:30
> 3,0,10,00       0x00    1       sg11    c:15:0b sde     b:08:40
> 3,0,12,00       0x00    1       sg12    c:15:0c sdf     b:08:50
>
> This allows a simple script to parse the map and create device
> nodes as needed.
...
> Obviously, it can only report the assignment of high-level drivers,
> if they are loaded, otherwise the last two columns will stay empty.
> (sg is handled especially, as we know it supports all devices.)
> If we attach a third high-level device driver, two more columns
> would show up. (Is this variable column number format a problem?)

The variable column format is of course annoying, but use
it if you must. The also-annoying alternative is to pick
a fill character that would be easy for a beginner to
handle in a script. Maybe one of:  @ - . / ?

The header line is far worse. It's too terse to be very helpful.
It gets in the way of every person writing a parser. Even in
your example script, you had to hack your way around it:

> +while read cbtu tp onl sgnm sgdev othnm othdev oothnm oothdev rest; do
> +  # Skip comment line(s)
> +  if test "${cbtu:0:1}" = "#"; then continue; fi
> +  # If we're just dealing with one device, do skip the others
> +  if test ! -z "$CMPAGAINST" -a "$CMPAGAINST" != "$cbtu"; then continue;
>  fi

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 947 bytes --]

Hi Richard,

I was in no way intending to trigger a discussion about devfs.
Some of the things addressed by the scsi/map patch indeed are no issue if
you use devfs; that's why I mentioned devfs at all.
I don't want to bash devfs and I think it's nice that it's in the kernel, 
so users have the choice to use it and the motivation to improve it.

But the problem that I wanted to address IMHO should also be solved
for those people that for one or another reason decided not to use
devfs. 

And face it: I do not think that all major Linux distributions will
start to use devfs within short future. The example you mentioned
(Mandrake) is certainly not a good one: Look at their update kernel.

Best regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE Linux AG, Nuernberg, DE                            SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 1861 bytes --]

Hi Albert,

On Sun, Jun 16, 2002 at 03:24:33PM -0400, Albert D. Cahalan wrote:
> Kurt Garloff writes:
> > If we attach a third high-level device driver, two more columns
> > would show up. (Is this variable column number format a problem?)
> 
> The variable column format is of course annoying, but use
> it if you must. The also-annoying alternative is to pick
> a fill character that would be easy for a beginner to
> handle in a script. Maybe one of:  @ - . / ?

Yes, as you correctly mention in your other mail, this would make it easier
to add more columns later.
But the problem then would be that we would need to fix (and limit) the
number of high-level devices that may be reported this way, which is not so
nice either. At this moment it's not a problem, of course, AFAIK.

> The header line is far worse. It's too terse to be very helpful.
> It gets in the way of every person writing a parser. Even in
> your example script, you had to hack your way around it:

I would not call it a hack. Ignoring comment lines is one of the basic
things each parser needs to do. Defining a format that does not allow
for comments actually would not be a very clever move.

But for a file exported from kernel, you may have a valid point.

Actually, the exact format of /proc/scsi/map is certainly something 
that can be discussed separately from the basic idea of adding a file
that does expose the mapping of a SCSI address (CBTU) and the attached
high level drivers. 
The way I designed it just should make it easy for a shell script to use
it. And keeping it simple certainly is a good thing.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE Linux AG, Nuernberg, DE                            SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 1496 bytes --]

Hi John,

On Sat, Jun 15, 2002 at 10:08:54PM +0800, John Summerfield wrote:
> > Life would be easier if the scsi subsystem would just report which SCSI
> > device (uniquely identified by the controller,bus,target,unit tuple) belongs
> > to which high-level device. The information is available in the kernel.
> 
> Does this not fail if I pull a device off, change its ID (perhaps to fit
> into another system), then plug it in again? Or if I move it from one
> adaptor to another?  

Sure it does.
The kernel can offer you the knowledge of a hardware path to your device,
which is given by controller,bust,SCSI target and unit numbers. This is
pretty stable in most configurations.

If you want to have more, you will probably use some sort of signatures.

But that's nothing which happens at SCSI layer.
For plain old SCSI devices, you may e.g. inquire the serial number (INQUIRY,
page code 0x80) which gives you a unique identifier if combined with
vendor and model strings. scsidev does this for you. But it occasinally
fails, as the open on scsi device may fail and we don't know the relation
between the sg devices (that can be used reliably to collect such
information) and the other high level devices. /proc/scsi/map solves this.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE Linux AG, Nuernberg, DE                            SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: /proc/scsi/map

[Patrick Mansfield]

On Sat, Jun 15, 2002 at 03:36:06PM +0200, Kurt Garloff wrote:
> Life would be easier if the scsi subsystem would just report which SCSI
> device (uniquely identified by the controller,bus,target,unit tuple) belongs
> to which high-level device. The information is available in the kernel.

I prefer we refer to the tuple as host, channel, id, lun (H, C, I, L), so
as to more closely match /proc/scsi/scsi, /proc/scsi/sg, and attached
messages:

[root@elm3a50 root]# cat /proc/scsi/scsi | head -4
Attached devices: 
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: IBM-PSG  Model: ST318203LC    !# Rev: B222
  Type:   Direct-Access                    ANSI SCSI revision: 02

root # cat /proc/scsi/sg/device_hdr  /proc/scsi/sg/devices | head -2
host    chan    id      lun     type    opens   qdepth  busy    online
0       0       0       0       0       4       253     0       1

Attached messages are of the form:

Attached scsi disk sdn at scsi2, channel 0, id 2, lun 0

> Attached patch does this:
> garloff@pckurt:/raid5/Kernel/src $ cat /proc/scsi/map 
> # C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
> 0,0,00,00       0x05    1       sg0     c:15:00 sr0     b:0b:00
> 1,0,01,00       0x05    1       sg1     c:15:01 sr1     b:0b:01
> 1,0,02,00       0x01    1       sg2     c:15:02 osst0   c:ce:00
> 1,0,03,00       0x05    1       sg3     c:15:03 sr2     b:0b:02
> 1,0,05,00       0x00    1       sg4     c:15:04 sda     b:08:00
> 1,0,09,00       0x00    1       sg5     c:15:05 sdb     b:08:10
> 2,0,01,00       0x05    1       sg6     c:15:06 sr3     b:0b:03
> 2,0,02,00       0x01    1       sg7     c:15:07 osst1   c:ce:01
> 2,0,03,00       0x05    1       sg8     c:15:08 sr4     b:0b:04
> 2,0,05,00       0x00    1       sg9     c:15:09 sdc     b:08:20
> 2,0,09,00       0x00    1       sg10    c:15:0a sdd     b:08:30
> 3,0,10,00       0x00    1       sg11    c:15:0b sde     b:08:40
> 3,0,12,00       0x00    1       sg12    c:15:0c sdf     b:08:50

Why not treat each upper layer driver the same? Type is already
in /proc/scsi/scsi, or implied by the upper level drivers attached.
Online should really be part of /proc/scsi/scsi.

Then, each line is a path followed by a list of upper level devices.
This would also simplify the code, although the ordering of the upper
level devices becomes link or module load order dependent.

And similiar to sg (someone commented on parsing '^#'), have a _hdr
entry; something like:

$ cat /proc/scsi/map_hdr /proc/scsi/map

H:C:I:L      online  type:name:block/char:maj:min
00:00:00:00     1       sg:sg0:c:15:00      sr:sr0:b:0b:00
01:00:01:00     1       sg:sg1:c:15:01      sr:sr1:b:0b:01
01:00:02:00     1       sg:sg2:c:15:02      osst:osst0:c:ce:00
02:00:09:00     1       sg:sg3:c:15:03      sd:sdd:b:08:30

Or:

H:C:I:L      online  type:enumeration:block/char:maj:min
00:00:00:00     1      sg:0:c:15:00      sr:0:b:0b:00
01:00:01:00     1      sg:1:c:15:01      sr:1:b:0b:01
01:00:02:00     1      sg:2:c:15:02      osst:0:c:ce:00
02:00:09:00     1      sg:3:c:15:03      sd:d:b:08:30


> A patch for 2.5 should be done as well, if the design is OK, of course.
> 

IMO, we should use driverfs for this in 2.5. Mike Sullivan's scsi driverfs
patch currently ends up with a driverfs layout (showing one Scsi_Device
with two partitions, sg and sd attached) like this:

[root@elm3a50 devices]# tree ./root/pci1/01\:06.0/scsi2/2\:0\:2\:0/
./root/pci1/01:06.0/scsi2/2:0:2:0/
|-- 2:0:2:0:disc
|   |-- kdev
|   |-- name
|   |-- power
|   `-- type
|-- 2:0:2:0:gen
|   |-- kdev
|   |-- name
|   |-- power
|   `-- type
|-- 2:0:2:0:p1
|   |-- kdev
|   |-- name
|   |-- power
|   `-- type
|-- 2:0:2:0:p2
|   |-- kdev
|   |-- name
|   |-- power
|   `-- type
|-- name
|-- power
`-- type

Right now, the name is storing an ID; the ID is retrieved in the kernel (using
page 0x80 or page 0x83 or the path). For example disc has:

[root@elm3a50 2:0:2:0:disc]# pwd
/devices/root/pci1/01:06.0/scsi2/2:0:2:0/2:0:2:0:disc
[root@elm3a50 2:0:2:0:disc]# ls  
kdev  name  power  type
[root@elm3a50 2:0:2:0:disc]# cat *
8d0
U20000020371719e8disc
0
BLK

-- Patrick Mansfield

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 3862 bytes --]

Hi Patrick,

On Mon, Jun 17, 2002 at 01:35:34PM -0700, Patrick Mansfield wrote:
> On Sat, Jun 15, 2002 at 03:36:06PM +0200, Kurt Garloff wrote:
> > Life would be easier if the scsi subsystem would just report which SCSI
> > device (uniquely identified by the controller,bus,target,unit tuple) belongs
> > to which high-level device. The information is available in the kernel.
> 
> I prefer we refer to the tuple as host, channel, id, lun (H, C, I, L), so
> as to more closely match /proc/scsi/scsi, /proc/scsi/sg, and attached
> messages:

You are refering to the naming of this 4-tuple, right: HCIL vs. CBTU?
I chose for CBTU, because that on's used in devfs. Actually, as you can see
from scsidev, I like HCIL more. But that's a detail the kernel should not
care about. The header line should be removed anyway as Albert remarked.
And helping those people who think that 200 bytes is unacceptable bloat.

[...]
> > 3,0,12,00       0x00    1       sg12    c:15:0c sdf     b:08:50
> 
> Why not treat each upper layer driver the same? Type is already
> in /proc/scsi/scsi, or implied by the upper level drivers attached.
> Online should really be part of /proc/scsi/scsi.

I'm not sure I know what you mean. The fact that I decided to put
the sg device name first independently of the (potentially) random
order in which high-level drivers are assigned?

> Then, each line is a path followed by a list of upper level devices.

It is. 

> This would also simplify the code, although the ordering of the upper
> level devices becomes link or module load order dependent.

Just I decided to report shg first. This has a very pratical reason:
I you want to use userspace tools to collect more advanced (and maybe type
dependant information), you will always want to use the sg device, which 
you can use to send SCSI commands and which you can open, even if there is
no medium or if the device is in use.

> And similiar to sg (someone commented on parsing '^#'), have a _hdr
> entry; something like:
> 
> $ cat /proc/scsi/map_hdr /proc/scsi/map
> 
> H:C:I:L      online  type:name:block/char:maj:min
> 00:00:00:00     1       sg:sg0:c:15:00      sr:sr0:b:0b:00
> 01:00:01:00     1       sg:sg1:c:15:01      sr:sr1:b:0b:01
> 01:00:02:00     1       sg:sg2:c:15:02      osst:osst0:c:ce:00
> 02:00:09:00     1       sg:sg3:c:15:03      sd:sdd:b:08:30

This looks find to me as well, by the way.
The reason why I chose to additionally report the device type reported by
inquiry is that you will only see the attached (and thus only the loaded)
high-level drivers of a device. With the device type, a userspace tool could
easily decide whether to trigger a modprobe and start again ...

> Or:
> 
> H:C:I:L      online  type:enumeration:block/char:maj:min
> 00:00:00:00     1      sg:0:c:15:00      sr:0:b:0b:00
> 01:00:01:00     1      sg:1:c:15:01      sr:1:b:0b:01
> 01:00:02:00     1      sg:2:c:15:02      osst:0:c:ce:00
> 02:00:09:00     1      sg:3:c:15:03      sd:d:b:08:30
> 
> 
> > A patch for 2.5 should be done as well, if the design is OK, of course.
> 
> IMO, we should use driverfs for this in 2.5. Mike Sullivan's scsi driverfs
> patch currently ends up with a driverfs layout (showing one Scsi_Device
> with two partitions, sg and sd attached) like this:

I still think the easy /proc/scsi/map format would be a nice basis to
inquire more information on the SCSI devices from userspace, even if you add
hierarchical attachment information via driverfs. And I think a solution
that works with both 2.4 and 2.5 would help most users, of course.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE Linux AG, Nuernberg, DE                            SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: /proc/scsi/map

[Patrick Mansfield]

Kurt -

On Mon, Jun 17, 2002 at 10:57:50PM +0200, Kurt Garloff wrote:
> Hi Patrick,
> 
> > I prefer we refer to the tuple as host, channel, id, lun (H, C, I, L), so
> > as to more closely match /proc/scsi/scsi, /proc/scsi/sg, and attached
> > messages:
> 
> You are refering to the naming of this 4-tuple, right: HCIL vs. CBTU?

Yes.

> I chose for CBTU, because that on's used in devfs. Actually, as you can see
> from scsidev, I like HCIL more. But that's a detail the kernel should not
> care about. The header line should be removed anyway as Albert remarked.
> And helping those people who think that 200 bytes is unacceptable bloat.
> 
> [...]
> > > 3,0,12,00       0x00    1       sg12    c:15:0c sdf     b:08:50
> > 
> > Why not treat each upper layer driver the same? Type is already
> > in /proc/scsi/scsi, or implied by the upper level drivers attached.
> > Online should really be part of /proc/scsi/scsi.
> 
> I'm not sure I know what you mean. The fact that I decided to put
> the sg device name first independently of the (potentially) random
> order in which high-level drivers are assigned?

Yes, I don't know why I took that to mean that sg was displayed differently.

> Just I decided to report shg first. This has a very pratical reason:
> I you want to use userspace tools to collect more advanced (and maybe type
> dependant information), you will always want to use the sg device, which 
> you can use to send SCSI commands and which you can open, even if there is
> no medium or if the device is in use.

No matter the column position sg can be found if each column includes
the upper level name (sg, sd etc.). Then you do not need to know or
check the scsi_type of the template, or explicitly locate the sg
template in scsi_proc_map().

And then without the header scsi_proc_map() gets really simple.

> Regards,
> -- 
> Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
> GPG key: See mail header, key servers         Linux kernel development
> SuSE Linux AG, Nuernberg, DE                            SCSI, Security

-- Patrick Mansfield

Re: /proc/scsi/map

[Doug Ledford]

On Sat, Jun 15, 2002 at 03:36:06PM +0200, Kurt Garloff wrote:
> Hi SCSI users,
> 
> from people using SCSI devices, there's one question that turns up again 
> and again: How can one assign stable device names to SCSI devices in
> case there are devices that may or may not be switched on or connected.


> Life would be easier if the scsi subsystem would just report which SCSI
> device (uniquely identified by the controller,bus,target,unit tuple) belongs
> to which high-level device. The information is available in the kernel.

Umm, this patently fails to meet the criteria you posted of "stable device 
name".  Adding a controller to a system is just as likely to blow this 
naming scheme to hell as it is to blow the traditional linux /dev/sd? 
scheme.  IOW, even though the /proc/scsi/map file looks nice and usefull, 
it fails to solve the very problem you are trying to solve.


-- 
  Doug Ledford <dledford@redhat.com>     919-754-3700 x44233
         Red Hat, Inc. 
         1801 Varsity Dr.
         Raleigh, NC 27606
  

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 2452 bytes --]

Hi Doug,

On Mon, Jun 17, 2002 at 06:08:18PM -0400, Doug Ledford wrote:
> On Sat, Jun 15, 2002 at 03:36:06PM +0200, Kurt Garloff wrote:
> > Life would be easier if the scsi subsystem would just report which SCSI
> > device (uniquely identified by the controller,bus,target,unit tuple) belongs
> > to which high-level device. The information is available in the kernel.
> 
> Umm, this patently fails to meet the criteria you posted of "stable device 
> name".  Adding a controller to a system is just as likely to blow this 
> naming scheme to hell as it is to blow the traditional linux /dev/sd? 
> scheme.  IOW, even though the /proc/scsi/map file looks nice and usefull, 
> it fails to solve the very problem you are trying to solve.

In case you just add controllers, you just need to make sure you get them the
same numbers again. A solution for this exists already:
* For a kernel where SCSI low-level drivers are loaded as modules,
  you just need to keep the order constant
* For compiled in SCSI drivers, use scsihosts=


But actually, the patch is not meant to be the holy grail of persistent
device naming. But it enables userspace tools to collect information 
* reliably 
  (fails so far due to possible open() failures with unknown
   relation to the corresponding sg device (which could be opened))
* without too much trouble

Both things I consider important and useful.

The patch basically does provide two pieces of information:
* mapping between sg vs. other high level devices
* mapping CBTU to high-level devices
The latter one is enough for many setups, and the former can be used for
more elaborate solutions involving userspace tools more advanced than the
simple script I included in the patch.

If you want to go for the holy grail, you may either come up with a 
unique address at hardware level (which does currently not exist for all
types dealt with by the SCSI subsystem) and make it available to SCSI mid
level or use signatures that allows you to find devices back. LVM, e.g.
does the latter. 
But at this moment, I fear, neither of them are possible in all cases.

Regards,
-- 
Kurt Garloff                   <kurt@garloff.de>         [Eindhoven, NL]
Physics: Plasma simulations    <K.Garloff@TUE.NL>     [TU Eindhoven, NL]
Linux: SCSI, Security          <garloff@suse.de>    [SuSE Nuernberg, DE]
 (See mail header or public key servers for PGP2 and GPG public keys.)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: /proc/scsi/map

[Doug Ledford]

On Tue, Jun 18, 2002 at 01:06:48AM +0200, Kurt Garloff wrote:
> Hi Doug,
> 
> On Mon, Jun 17, 2002 at 06:08:18PM -0400, Doug Ledford wrote:
> > On Sat, Jun 15, 2002 at 03:36:06PM +0200, Kurt Garloff wrote:
> > > Life would be easier if the scsi subsystem would just report which SCSI
> > > device (uniquely identified by the controller,bus,target,unit tuple) belongs
> > > to which high-level device. The information is available in the kernel.
> > 
> > Umm, this patently fails to meet the criteria you posted of "stable device 
> > name".  Adding a controller to a system is just as likely to blow this 
> > naming scheme to hell as it is to blow the traditional linux /dev/sd? 
> > scheme.  IOW, even though the /proc/scsi/map file looks nice and usefull, 
> > it fails to solve the very problem you are trying to solve.
> 
> In case you just add controllers, you just need to make sure you get them the
> same numbers again. A solution for this exists already:
> * For a kernel where SCSI low-level drivers are loaded as modules,
>   you just need to keep the order constant
> * For compiled in SCSI drivers, use scsihosts=

No, this is not true.  If you add a new controller (for some new disks in 
a new external enclosure or whatever), and that controller uses the same 
driver as other controller(s) in your system, then you have no guarantee 
of order.  For example, adding a 4th aic7xxx controller to your system 
might or might not place the new controller at the end of the list 
depending on PCI scan order, etc.  There simply is *no* guarantee here of 
any consistent naming, so don't bother trying to claim there is.

Now don't get me wrong, I'm not saying the patch isn't usefull, but the 
patch doesn't provide *any* guarantee of consistent device naming and so 
using that as a reason to put the patch into the mainstream kernel is 
utter crap.  Go ahead and make your case for why it should be in the 
kernel, but don't use reasons that aren't correct.
 
> But actually, the patch is not meant to be the holy grail of persistent
> device naming. But it enables userspace tools to collect information 
> * reliably 
>   (fails so far due to possible open() failures with unknown
>    relation to the corresponding sg device (which could be opened))

This can be done without your patch (the mapping from /dev/sg to /dev/sd? 
or /dev/st? or /dev/scd? or whatever is not impossible from user space 
without your patch, it just requires a user space tool to open the files 
and start comparing host/bus/id/lun combinations from dev file to dev 
file).

> * without too much trouble

This part is true enough, it is easier to read the map file than to 
program the information retrieval I mentioned above.

> Both things I consider important and useful.
> 
> The patch basically does provide two pieces of information:
> * mapping between sg vs. other high level devices

This I think is usefull.

> * mapping CBTU to high-level devices

This I don't think is usefull at all.  It's no more reliable than our 
current system and people that are depending on this to solve their "I 
can't tell what device is what" delima are going to be sorely upset when 
they realize that hardware changes can change this stuff around just as 
fast as it changes around the /dev/sd? mappings.

> The latter one is enough for many setups,

The latter one is just as broken in design as the original /dev/sd? 
enumeration problem (which stands to reason since this method also is an 
enumeration method, it's just that instead of enumerating the disks 
starting at 0, we are enumerating the SCSI controllers starting at the 
first one we find and going from there).

> and the former can be used for
> more elaborate solutions involving userspace tools more advanced than the
> simple script I included in the patch.

Which is the much better way to go.

> If you want to go for the holy grail, you may either come up with a 
> unique address at hardware level (which does currently not exist for all
> types dealt with by the SCSI subsystem) and make it available to SCSI mid
> level or use signatures that allows you to find devices back. LVM, e.g.
> does the latter. 
> But at this moment, I fear, neither of them are possible in all cases.
> 
> Regards,
> -- 
> Kurt Garloff                   <kurt@garloff.de>         [Eindhoven, NL]
> Physics: Plasma simulations    <K.Garloff@TUE.NL>     [TU Eindhoven, NL]
> Linux: SCSI, Security          <garloff@suse.de>    [SuSE Nuernberg, DE]
>  (See mail header or public key servers for PGP2 and GPG public keys.)



-- 
  Doug Ledford <dledford@redhat.com>     919-754-3700 x44233
         Red Hat, Inc. 
         1801 Varsity Dr.
         Raleigh, NC 27606
  

[Possibly OT] Re: /proc/scsi/map

[Austin Gonyou]

On Mon, 2002-06-17 at 21:40, Doug Ledford wrote:
> On Tue, Jun 18, 2002 at 01:06:48AM +0200, Kurt Garloff wrote:
> > Hi Doug,
> > 
> .... 
> > In case you just add controllers, you just need to make sure you get them the
> > same numbers again. A solution for this exists already:
> > * For a kernel where SCSI low-level drivers are loaded as modules,
> >   you just need to keep the order constant
> > * For compiled in SCSI drivers, use scsihosts=
> 
....
> No, this is not true.  If you add a new controller (for some new disks in 
> a new external enclosure or whatever), and that controller uses the same 
> driver as other controller(s) in your system, then you have no guarantee 
> of order.  For example, adding a 4th aic7xxx controller to your system 
> might or might not place the new controller at the end of the list 
> depending on PCI scan order, etc.  There simply is *no* guarantee here of 
> any consistent naming, so don't bother trying to claim there is.
> 

Taking a bit of an example from Veritas, would it be, at all, feasible
if n+ blocks were used at the end of the disk or partition(beginning
maybe?), to write a specific identifier that is unique to a specific
controller, or to make note of the drive serial number and store that on
the disk somewhere in some agreed upon understood way. 

Much like the private region on a veritas disk or volume. With the extra
accounting, which should only be needed during boot, or during
disk/volume manipulation, one could conceivably always have a sane
device map, all the time. 

As to the rest of the comments lower down on the original mail, I'd say
that this is *a lot* of trouble, versus the opposite, but if implemented
properly would be highly useful. Using LVM and the like, which does
something like this, seems to be fine for most people(even when moving
disks around, etc), but this ability, without the overhead of LVM in the
mix would seem a good idea for some. 

Just my $.02
TIA
-- 
Austin Gonyou <austin@digitalroadkill.net>

Re: /proc/scsi/map

[Douglas Gilbert]

Doug Ledford wrote:
> 
> On Tue, Jun 18, 2002 at 01:06:48AM +0200, Kurt Garloff wrote:
> > Hi Doug,
> >
> > On Mon, Jun 17, 2002 at 06:08:18PM -0400, Doug Ledford wrote:
> > > On Sat, Jun 15, 2002 at 03:36:06PM +0200, Kurt Garloff wrote:
> > > > Life would be easier if the scsi subsystem would just report which SCSI
> > > > device (uniquely identified by the controller,bus,target,unit tuple) belongs
> > > > to which high-level device. The information is available in the kernel.
> > >
> > > Umm, this patently fails to meet the criteria you posted of "stable device
> > > name".  Adding a controller to a system is just as likely to blow this
> > > naming scheme to hell as it is to blow the traditional linux /dev/sd?
> > > scheme.  IOW, even though the /proc/scsi/map file looks nice and usefull,
> > > it fails to solve the very problem you are trying to solve.
> >
> > In case you just add controllers, you just need to make sure you get them the
> > same numbers again. A solution for this exists already:
> > * For a kernel where SCSI low-level drivers are loaded as modules,
> >   you just need to keep the order constant
> > * For compiled in SCSI drivers, use scsihosts=
> 
> No, this is not true.  If you add a new controller (for some new disks in
> a new external enclosure or whatever), and that controller uses the same
> driver as other controller(s) in your system, then you have no guarantee
> of order.  For example, adding a 4th aic7xxx controller to your system
> might or might not place the new controller at the end of the list
> depending on PCI scan order, etc.  There simply is *no* guarantee here of
> any consistent naming, so don't bother trying to claim there is.

It only been two days and it seems appropriate to refer
to Martin Petersen's summation of persistent naming issues again:
http://marc.theaimsgroup.com/?l=linux-scsi&m=101840990116069&w=2


As for "scsihosts" its current syntax is:
 scsihosts=aic7xxx:sym53c8xx::aic7xxx

This could be extended to
  scsihosts=aic7xxx[pci=00:0c.0]:sym53c8xx::aic7xxx
and if "pci=" was assumed to be the default then this
would have the same effect as:
  scsihosts=[00:0c.0]:sym53c8xx::aic7xxx
assuming there was a aic7xxx controlled HBA at that PCI slot.

No consistent persistence here, just a little more precision.

> Now don't get me wrong, I'm not saying the patch isn't usefull, but the
> patch doesn't provide *any* guarantee of consistent device naming and so
> using that as a reason to put the patch into the mainstream kernel is
> utter crap.  Go ahead and make your case for why it should be in the
> kernel, but don't use reasons that aren't correct.
> 
> > But actually, the patch is not meant to be the holy grail of persistent
> > device naming. But it enables userspace tools to collect information
> > * reliably
> >   (fails so far due to possible open() failures with unknown
> >    relation to the corresponding sg device (which could be opened))
> 
> This can be done without your patch (the mapping from /dev/sg to /dev/sd?
> or /dev/st? or /dev/scd? or whatever is not impossible from user space
> without your patch, it just requires a user space tool to open the files
> and start comparing host/bus/id/lun combinations from dev file to dev
> file).
> 
> > * without too much trouble
> 
> This part is true enough, it is easier to read the map file than to
> program the information retrieval I mentioned above.

You can say that again. Joerg Schilling is still some way
from getting it right in cdrecord, SANE still has problems
in this area (and I rewrote the scsi scanning code). Kurt
and I have a fair amount of experience in this particular 
area. There are lots of gotchas ...


Kurt's implementation is worthy of note as well. When I have
pondered this problem I just wanted to poke each sg kdev_t into
the corresponding Scsi_Device. People objected that this was
a layering violation. Kurt has added a new upper level callback
find_kdev() which is cleaner. Based on this new callback we
could add a new mid level ioctl that yielded the principal
(k)dev_t (i.e. sd, st or sr) and the generic one. That would
make life easier for cdrecord and cdparanoia (and sg_utils). 
As for SANE, by looking at /proc/scsi/map it could only analyse 
sg devices that where "printer" or "processor" device types.

Doug Gilbert

Re: /proc/scsi/map

[Doug Ledford]

On Tue, Jun 18, 2002 at 12:32:16AM -0400, Douglas Gilbert wrote:
> As for "scsihosts" its current syntax is:
>  scsihosts=aic7xxx:sym53c8xx::aic7xxx
> 
> This could be extended to
>   scsihosts=aic7xxx[pci=00:0c.0]:sym53c8xx::aic7xxx
> and if "pci=" was assumed to be the default then this
> would have the same effect as:
>   scsihosts=[00:0c.0]:sym53c8xx::aic7xxx
> assuming there was a aic7xxx controlled HBA at that PCI slot.
> 
> No consistent persistence here, just a little more precision.

I think that relies upon the SCSI controller using the newer PCI scanning 
methods (which, for example, my aic7xxx driver does not use).

-- 
  Doug Ledford <dledford@redhat.com>     919-754-3700 x44233
         Red Hat, Inc. 
         1801 Varsity Dr.
         Raleigh, NC 27606
  

Re: [Possibly OT] Re: /proc/scsi/map

[Doug Ledford]

On Mon, Jun 17, 2002 at 10:24:40PM -0500, Austin Gonyou wrote:
> Taking a bit of an example from Veritas, would it be, at all, feasible
> if n+ blocks were used at the end of the disk or partition(beginning
> maybe?), to write a specific identifier that is unique to a specific
> controller, or to make note of the drive serial number and store that on
> the disk somewhere in some agreed upon understood way. 

Both LVM and the md code already do this.  Ext2 and ext3 also have volume 
labels that can be used for this purpose.  As much as I hate to admit it, 
this is the one area where I think MicroSoft did the right thing and 
snagged an unused byte in the partition table to mark the disks ordering 
(although we would need more than one byte).  By putting it in the 
partition table, it would only need to be dealt with by one area of code 
(the partition reading code), would work for all filesystems, would work 
for all LVM and md types of code, and would be universal on linux systems 
and provide consistent, persistent device naming.  Of course, if a disk 
dies and you put a new one in, then you have to rename the new disk to the 
old disks names when you partition it, but you would have to do that or 
something similar to that with all such possible solutions.

The simple fact of the matter is that to provide truly consistent, 
persistent device naming requires that the naming be "end-to-end".  You 
can not rely on *any* ordering issues (such as controllers, PCI busses, 
devices, etc), you have to read the name from the device itself and the 
name has to be totally irrespective of the devices current location on 
whatever bus it uses.

-- 
  Doug Ledford <dledford@redhat.com>     919-754-3700 x44233
         Red Hat, Inc. 
         1801 Varsity Dr.
         Raleigh, NC 27606
  

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 6339 bytes --]

Hi Doug,

On Mon, Jun 17, 2002 at 10:40:47PM -0400, Doug Ledford wrote:
> On Tue, Jun 18, 2002 at 01:06:48AM +0200, Kurt Garloff wrote:
> > * For compiled in SCSI drivers, use scsihosts=
> 
> No, this is not true.  If you add a new controller (for some new disks in 
> a new external enclosure or whatever), and that controller uses the same 
> driver as other controller(s) in your system, then you have no guarantee 
> of order.  For example, adding a 4th aic7xxx controller to your system 
> might or might not place the new controller at the end of the list 
> depending on PCI scan order, etc.  There simply is *no* guarantee here of 
> any consistent naming, so don't bother trying to claim there is.

You're right; there is no guarantee.

In scsidev, the IO port is added as identifier to the host number, but even
that may change in case your Plug'n'Pray BIOS assigns a different IO port
upon PCI configuration ...

> > But actually, the patch is not meant to be the holy grail of persistent
> > device naming. But it enables userspace tools to collect information 
> > * reliably 
> >   (fails so far due to possible open() failures with unknown
> >    relation to the corresponding sg device (which could be opened))
> 
> This can be done without your patch (the mapping from /dev/sg to /dev/sd? 
> or /dev/st? or /dev/scd? or whatever is not impossible from user space 
> without your patch, it just requires a user space tool to open the files 
> and start comparing host/bus/id/lun combinations from dev file to dev 
> file).

No, it unfortunately can't. The reason is you just can't reliably open a
device to do the SCSI_IOCTL_GET_IDLUN to find out about CBTU. 
Devices with removable media are a good example. The open on a tape
that is not inserted just fails. Or it may be in use

Or cause very nasty side effects, like stalling for a minute because the
tape driver rewinds the drive and looks for meta-information (osst).

This problem is really the reason why scsidev is not foolproof.
sg_map does not do magic either.

root@pckurt:~ $ sg_map 
sg_map: close error: Input/output error
/dev/sg0  /dev/scd0
/dev/sg1  /dev/sda
/dev/sg2  /dev/sdb
/dev/sg3  /dev/sdc
/dev/sg4  /dev/sdd
/dev/sg5  /dev/sde
/dev/sg6  /dev/sdf
/dev/sg7  /dev/scd1
/dev/sg8  /dev/osst0
/dev/sg9  /dev/scd2
/dev/sg10  /dev/scd3
/dev/sg11
/dev/sg12  /dev/scd4

The /proc/scsi/map does provide the missing piece of information.

root@pckurt:~ $ cat /proc/scsi/map
# C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
0,0,00,00       0x05    1       sg0     c:15:00 sr0     b:0b:00
1,0,09,00       0x00    1       sg2     c:15:02 sdb     b:08:10
1,0,05,00       0x00    1       sg1     c:15:01 sda     b:08:00
1,0,01,00       0x05    1       sg7     c:15:07 sr1     b:0b:01
1,0,02,00       0x01    1       sg8     c:15:08 osst0   c:ce:00
1,0,03,00       0x05    1       sg9     c:15:09 sr2     b:0b:02
2,0,05,00       0x00    1       sg3     c:15:03 sdc     b:08:20
2,0,09,00       0x00    1       sg4     c:15:04 sdd     b:08:30
2,0,01,00       0x05    1       sg10    c:15:0a sr3     b:0b:03
2,0,02,00       0x01    1       sg11    c:15:0b osst1   c:ce:01
2,0,03,00       0x05    1       sg12    c:15:0c sr4     b:0b:04
3,0,10,00       0x00    1       sg5     c:15:05 sde     b:08:40
3,0,12,00       0x00    1       sg6     c:15:06 sdf     b:08:50

> > * without too much trouble
> 
> This part is true enough, it is easier to read the map file than to 
> program the information retrieval I mentioned above.
> 
> > Both things I consider important and useful.
> > 
> > The patch basically does provide two pieces of information:
> > * mapping between sg vs. other high level devices
> 
> This I think is usefull.
> 
> > * mapping CBTU to high-level devices
> 
> This I don't think is usefull at all.  It's no more reliable than our 
> current system and people that are depending on this to solve their "I 
> can't tell what device is what" delima are going to be sorely upset when 
> they realize that hardware changes can change this stuff around just as 
> fast as it changes around the /dev/sd? mappings.

You less often build in new host adpaters or jumper your drives' IDs
differently than you switch on/off an external ZIP drive/scanner/ e.g.

And if you take a disk from one controller to another one, you arguably even
want the address to change as your path to the device changed. At least you
can expect it.

CBTU still gives you reasonable information about the path to a device.

The "C" identification may need improvement ... and I'm certainly open
for suggestions.

And I personally do think that at SCSI midlayer in kernel, you want to
report about this path to a device.

> > The latter one is enough for many setups,
> 
> The latter one is just as broken in design as the original /dev/sd? 
> enumeration problem (which stands to reason since this method also is an 
> enumeration method, it's just that instead of enumerating the disks 
> starting at 0, we are enumerating the SCSI controllers starting at the 
> first one we find and going from there).

I don't think the original /dev/sd? design is broken. It's a choice that
makes some sense if you think about the fact that you only have a limited
amount of majors reserved for SCSI disks.
But it has some disadvantages ...

> > and the former can be used for
> > more elaborate solutions involving userspace tools more advanced than the
> > simple script I included in the patch.
> 
> Which is the much better way to go.

Expect a scsidev release (2.25) that does use /proc/scsi/map (if present) to
present a somewhat better mapping than the little script included.
It optionally does assign aliases based on the serial number
(INQUIRY,evpd=1,pg.cd. 0x80), which is rather stable in case your do provide
this piece of information. But not unique in case you provide multiple
paths to your devices (like having more than one SCSI host adapter on a bus,
which I BTW do for testing purposes).

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE Linux AG, Nuernberg, DE                            SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [Possibly OT] Re: /proc/scsi/map

[Bryan Henderson]

>The simple fact of the matter is that to provide truly consistent,
>persistent device naming requires that the naming be "end-to-end".  You
>can not rely on *any* ordering issues (such as controllers, PCI busses,
>devices, etc), you have to read the name from the device itself and the
>name has to be totally irrespective of the devices current location on
>whatever bus it uses.

Plus, in a case like this, you don't want the name to change if you move
the data from one disk drive to another.  It's still the same volume, and
the volume is what you care about.

But we do have look out for other cases.  Like non-storage devices.

Re: [Possibly OT] Re: /proc/scsi/map

[Austin Gonyou]

On Tue, 2002-06-18 at 10:49, Bryan Henderson wrote:
....
> But we do have look out for other cases.  Like non-storage devices.
> 

I'm not sure what you mean here?


-- 
Austin Gonyou <austin@digitalroadkill.net>

Re: [Possibly OT] Re: /proc/scsi/map

[Doug Ledford]

On Tue, Jun 18, 2002 at 11:06:35AM -0500, Austin Gonyou wrote:
> 
> 
> On Tue, 2002-06-18 at 10:49, Bryan Henderson wrote:
> ....
> > But we do have look out for other cases.  Like non-storage devices.
> > 
> 
> I'm not sure what you mean here?

He's talking about the fact that you can put a volume label on a disk 
drive, but it's hard to do the same to a tape drive or maybe a SCSI 
DVD-RAM drive or anything else where you can't write to a permanent part 
of the device instead of a removeable part of the device.

-- 
  Doug Ledford <dledford@redhat.com>     919-754-3700 x44233
         Red Hat, Inc. 
         1801 Varsity Dr.
         Raleigh, NC 27606
  

Re: [Possibly OT] Re: /proc/scsi/map

[Austin Gonyou]

On Tue, 2002-06-18 at 11:08, Doug Ledford wrote:
> On Tue, Jun 18, 2002 at 11:06:35AM -0500, Austin Gonyou wrote:
> > 
> > 
> > On Tue, 2002-06-18 at 10:49, Bryan Henderson wrote:
> > ....
> > > But we do have look out for other cases.  Like non-storage devices.
> > > 
> > 
> > I'm not sure what you mean here?
> 
> He's talking about the fact that you can put a volume label on a disk 
> drive, but it's hard to do the same to a tape drive or maybe a SCSI 
> DVD-RAM drive or anything else where you can't write to a permanent part 
> of the device instead of a removeable part of the device.

Ahh...off-line storage devices. Kind of hard to write a label to a
non-storage device. (/dev/null is a non-storage device right? :) )


-- 
Austin Gonyou <austin@digitalroadkill.net>