[PATCH 00/17] mdadm: SCSI enclosure based array

[PATCH 00/17] mdadm: SCSI enclosure based array

[Song Liu]

These patches by Dan Williams enable creating RAID array based on
SCSI enclosures.

mdadm gets information about SCSI enclosure through ses module in
kernel, and creates array for HDDs in an enclosure.

New udev rules are added to make drive swap in the enclosure based
array easier.

Typical array create flow looks like:

  /* examine enclosures in the system */
  mdadm --detail-platform --brief --enclosure > /etc/mdadm.conf

  /* create policy for enclosure0 and enclosure1 */
  echo "POLICY domain=domain0 action=spare slot=enclosure0" >> /etc/mdadm.conf
  echo "POLICY domain=domain1 action=spare slot=enclosure1" >> /etc/mdadm.conf

  /* add md-ses program */
  echo "PROGRAM /usr/share/mdadm/md-ses" >> /etc/mdadm.conf

  /* create md0 and md1 */
  mdadm --create /dev/md0 enclosure0 -n 15 -l 6 -e 1.2 -c 1024 --run
  mdadm --create /dev/md1 enclosure1 -n 15 -l 6 -e 1.2 -c 1024 --run

  /* add array to mdadm.conf */
  mdadm -Eb enclosure0 enclosure1 >> /etc/mdadm.conf

  /* create udev rules that add drives to array */
  mdadm --udev-rules --enclosure > /lib/udev/rules.d/65-md-raid-hotswap.rules

Dan Williams (15):
  udev rules and infrastructure for /dev/disk/by-slot
  mdadm: allow a /dev/disk/by-slot as domain path descriptor
  'act_spare' should be the minimum requirement for dynamic domains
  fix variable offset when ':' is present in the device name
  imsm: quiet detail platform
  ses: workaround sysfs deprecated
  prepare for enclosure platform details
  enclosure detection/enumeration
  Add --brief support to --enclosure
  config: ENCLOSURE keyword and enclosure device list expansion
  policy: enable enclosure names for domain definitions
  invoke hot-add policy on "change" events
  Toggle enclosure leds in response to raid events
  scsi mode sense/select support
  SCSIMODE: configuration file support for mode page settings.

Song Liu (2):
  Clear MBR in when Kill superblock
  In generated udev rules, skip RAID members

 Create.c                            |  10 +-
 Detail.c                            |  63 +++----
 Incremental.c                       |   9 +
 Kill.c                              |  22 +++
 Makefile                            |  31 +++-
 Monitor.c                           |   3 +
 ReadMe.c                            |   2 +
 config.c                            | 281 +++++++++++++++++++++++++++-
 contrib/md-ses                      | 310 +++++++++++++++++++++++++++++++
 enclosure.c                         | 355 ++++++++++++++++++++++++++++++++++++
 forward-scsi-uevents                |  14 ++
 mdadm.c                             |  99 +++++++---
 mdadm.h                             |  31 +++-
 policy.c                            | 116 +++++++++---
 ses_slot_id                         | 144 +++++++++++++++
 sg_io.c                             | 170 ++++++++++++++++-
 sg_io.h                             |  27 +++
 super-intel.c                       |  29 ++-
 sysfs.c                             |  11 +-
 udev-enclosure-slot.rules.in        |  11 ++
 udev-workaround-sysfs-deprecated.in |   2 +
 util.c                              |   7 +
 22 files changed, 1648 insertions(+), 99 deletions(-)
 create mode 100755 contrib/md-ses
 create mode 100644 enclosure.c
 create mode 100755 forward-scsi-uevents
 create mode 100755 ses_slot_id
 create mode 100644 sg_io.h
 create mode 100644 udev-enclosure-slot.rules.in
 create mode 100644 udev-workaround-sysfs-deprecated.in

--
2.4.6

[PATCH 01/17] udev rules and infrastructure for /dev/disk/by-slot

[Song Liu]

From: Dan Williams <djbw@fb.com>

/dev/disk/by-path maps the device by its sysfs path, but uses
data not necessarily guaranteed to be static.  It also takes extra work
to map where the device is physically attached.
static data for creating the path name.

For example:

pci-0000:04:00.0-sas-0x500605b005846061:1:0-0x5000cca01a7c5606:0
phy_id------------------------------------^
end_device-sas_address----------------------^

...on mpt2sas 'phy_id' is a dynamically allocated number based on when
the port comes up.  There is no differentiation based on where the end
device is attached as the end device sas address is the only indication
of the exit point from the topology.

In contrast:
enclosure-0x5f80f41f1d6a60ff-slot9

...is a static name for this physical location in an enclosure.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 Makefile                  |   3 +
 ses_slot_id               | 144 ++++++++++++++++++++++++++++++++++++++++++++++
 udev-enclosure-slot.rules |  13 +++++
 3 files changed, 160 insertions(+)
 create mode 100755 ses_slot_id
 create mode 100644 udev-enclosure-slot.rules

diff --git a/Makefile b/Makefile
index fde2e63..4eb2dad 100644
--- a/Makefile
+++ b/Makefile
@@ -110,6 +110,7 @@ LDLIBS=-ldl
 INSTALL = /usr/bin/install
 DESTDIR =
 BINDIR  = /sbin
+LIBDIR	= /lib/mdadm
 MANDIR  = /usr/share/man
 MAN4DIR = $(MANDIR)/man4
 MAN5DIR = $(MANDIR)/man5
@@ -292,6 +293,8 @@ install-man: mdadm.8 md.4 mdadm.conf.5 mdmon.8
 	$(INSTALL) -D -m 644 mdadm.conf.5 $(DESTDIR)$(MAN5DIR)/mdadm.conf.5
 
 install-udev: udev-md-raid-arrays.rules udev-md-raid-assembly.rules
+	$(INSTALL) -D -m 655 ses_slot_id $(DESTDIR)$(LIBDIR)/ses_slot_id
+	$(INSTALL) -D -m 644 udev-enclosure-slot.rules $(DESTDIR)$(UDEVDIR)/rules.d/62-md-enclosure-slot.rules
 	@for file in 63-md-raid-arrays.rules 64-md-raid-assembly.rules ; \
 	do sed -e 's,BINDIR,$(BINDIR),g' udev-$${file#??-} > .install.tmp.1 && \
 	   $(ECHO) $(INSTALL) -D -m 644 udev-$${file#??-} $(DESTDIR)$(UDEVDIR)/rules.d/$$file ; \
diff --git a/ses_slot_id b/ses_slot_id
new file mode 100755
index 0000000..629f1d1
--- /dev/null
+++ b/ses_slot_id
@@ -0,0 +1,144 @@
+#!/bin/sh
+
+# lookup ses slot information for a given device a /dev/disk/by-slot to
+# supplement the not-always-persistent path in the sas case
+# based on the the original shell script implementation of path_id
+
+SYSFS=/sys
+RESULT=1
+TYPE=
+OPWD="`pwd`"
+full_sysfs_path=
+full_sysfs_device_path=
+
+if [ -z "$DEVPATH" -a -z "$1" ] ; then
+	exit 1
+fi
+
+if [ -z "$DEVPATH" ] ; then
+	case "$1" in
+		$SYSFS/*)
+			DEVPATH="${1#$SYSFS}"
+			;;
+		*)
+			DEVPATH=$1
+			;;
+	esac
+fi
+
+if [ ! -e $SYSFS$DEVPATH/dev ] ; then
+	exit 1
+fi
+
+case "$DEVPATH" in
+	/devices/*)
+		cd "$SYSFS$DEVPATH/subsystem";
+		TYPE="`pwd -P`"
+		cd "$OPWD"
+		TYPE="${TYPE##*/}"
+		;;
+	/class/*)
+		TYPE="${DEVPATH#/class/}"
+		TYPE="${TYPE%%/*}"
+		;;
+	/block/*)
+		TYPE=block
+		;;
+	*)
+		exit 1
+		;;
+esac
+
+handle_ses () {
+	: handle_ses $*
+	local DEV=$1
+
+	ses_dev=$(find $DEV -maxdepth 1 -name 'enclosure_device:*' -print -quit)
+	ses_dev=$(readlink -f "$ses_dev")
+	enc_dev="${ses_dev%/*}"
+
+	if [ -f "$ses_dev/slot" ]; then
+		read slot < "$ses_dev/slot"
+	fi
+
+	if [ -f "$enc_dev/id" ]; then
+		read enclosure_id < "$enc_dev/id"
+	fi
+
+	if [ -z "$slot" -o -z "$enclosure_id" ]; then
+		: no enclosure device
+		D=
+		RESULT=1
+		return
+	fi
+
+	d="enclosure-${enclosure_id}-slot${slot}"
+	D=
+	RESULT=0
+}
+
+handle_device () {
+	full_sysfs_path="$SYSFS$DEVPATH"
+	case "$DEVPATH" in
+		/devices/*)
+			# new sysfs layout
+			if [ -L $full_sysfs_path/subsystem ]; then
+				full_sysfs_path="${full_sysfs_path%/*}"
+				cd "$full_sysfs_path/subsystem";
+				subsys="`pwd -P`"
+				cd "$OPWD"
+				subsys="${subsys##*/}"
+				if [ "$subsys" = "block" ]; then
+					# parent is "block", it's a partition, move one up
+				full_sysfs_path="${full_sysfs_path%/*}"
+				fi
+				cd $full_sysfs_path
+			fi
+			;;
+		*)
+			# old sysfs layout
+			if [ ! -L $full_sysfs_path/device ] ; then
+				if [ -f $full_sysfs_path/range ] ; then return ; fi
+				full_sysfs_path="${full_sysfs_path%/*}"
+				: full_sysfs_path "$full_sysfs_path"
+				if [ ! -L $full_sysfs_path/device -o ! -f $full_sysfs_path/dev ] ; then
+					return
+				fi
+			fi
+			cd $full_sysfs_path/device
+			;;
+	esac
+	full_sysfs_device_path="`pwd -P`"
+	cd "$OPWD"
+	D=$full_sysfs_device_path
+	while [ ! -z "$D" ] ; do
+		case "$D" in
+			*/end_device-[0-9]*:[0-9]*:[0-9]*/*)
+				handle_ses "$D"
+				if [ $RESULT = 0 ]; then
+					found_ses="yes"
+				fi
+				;;
+			*/devices)
+				D=
+				;;
+			*)
+				: not handled
+				RESULT=1
+				return
+				;;
+		esac
+	done
+}
+
+case "$TYPE" in
+	block)
+		handle_device
+		echo "ID_SLOT='$d'"
+		;;
+	*)
+		RESULT=1
+		;;
+esac
+
+exit $RESULT
diff --git a/udev-enclosure-slot.rules b/udev-enclosure-slot.rules
new file mode 100644
index 0000000..75c6376
--- /dev/null
+++ b/udev-enclosure-slot.rules
@@ -0,0 +1,13 @@
+# do not edit this file, it will be overwritten on update
+
+SUBSYSTEM!="block", GOTO="ses_slot_end"
+
+ACTION!="add|change", GOTO="ses_slot_end"
+KERNEL!="sd*", GOTO="ses_slot_end"
+
+# by-slot (parent device slot)
+ENV{DEVTYPE}=="disk", IMPORT{program}="/lib/mdadm/ses_slot_id %p"
+ENV{DEVTYPE}=="disk", ENV{ID_SLOT}=="?*", SYMLINK+="disk/by-slot/$env{ID_SLOT}"
+ENV{DEVTYPE}=="partition", ENV{ID_SLOT}=="?*", SYMLINK+="disk/by-slot/$env{ID_SLOT}-part%n"
+
+LABEL="ses_slot_end"
-- 
2.4.6

[PATCH 02/17] mdadm: allow a /dev/disk/by-slot as domain path descriptor

[Song Liu]

From: Dan Williams <djbw@fb.com>

In a POLICY line let 'slot=' statements define a domain by
/dev/disk/by-slot and take precedence over 'path=' (/dev/disk/by-path)
definitions.

Also support writing udev rules for slot paths.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 mdadm.h  |  2 +-
 policy.c | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++-----------
 2 files changed, 69 insertions(+), 15 deletions(-)

diff --git a/mdadm.h b/mdadm.h
index 5d5e97f..167ca0c 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -1126,7 +1126,7 @@ struct pol_rule {
 };
 
 extern char rule_policy[], rule_part[];
-extern char rule_path[], rule_type[];
+extern char rule_path[], rule_slot[], rule_type[];
 extern char type_part[], type_disk[];
 
 extern void policyline(char *line, char *type);
diff --git a/policy.c b/policy.c
index 064d349..960a37e 100644
--- a/policy.c
+++ b/policy.c
@@ -189,16 +189,17 @@ struct dev_policy *pol_find(struct dev_policy *pol, char *name)
 	return pol;
 }
 
-static char *disk_path(struct mdinfo *disk)
+static char *__disk_path(struct mdinfo *disk, const char *path_prefix, int null_ok)
 {
 	struct stat stb;
 	int prefix_len;
 	DIR *by_path;
-	char symlink[PATH_MAX] = "/dev/disk/by-path/";
+	char symlink[PATH_MAX];
 	char nm[PATH_MAX];
 	struct dirent *ent;
 	int rv;
 
+	strcpy(symlink, path_prefix);
 	by_path = opendir(symlink);
 	if (by_path) {
 		prefix_len = strlen(symlink);
@@ -219,6 +220,9 @@ static char *disk_path(struct mdinfo *disk)
 		}
 		closedir(by_path);
 	}
+	closedir(by_path);
+	if (null_ok)
+		return NULL;
 	/* A NULL path isn't really acceptable - use the devname.. */
 	sprintf(symlink, "/sys/dev/block/%d:%d", disk->disk.major, disk->disk.minor);
 	rv = readlink(symlink, nm, sizeof(nm)-1);
@@ -232,6 +236,15 @@ static char *disk_path(struct mdinfo *disk)
 	return xstrdup("unknown");
 }
 
+static char *disk_path(struct mdinfo *disk)
+{
+	char *path = __disk_path(disk, "/dev/disk/by-slot/", 1);
+
+	if (!path)
+		return __disk_path(disk, "/dev/disk/by-path/", 0);
+	return path;
+}
+
 char type_part[] = "part";
 char type_disk[] = "disk";
 static char *disk_type(struct mdinfo *disk)
@@ -246,6 +259,13 @@ static char *disk_type(struct mdinfo *disk)
 		return type_disk;
 }
 
+static int rule_is_path(struct rule *rule)
+{
+	if (rule->name == rule_path || rule->name == rule_slot)
+		return 1;
+	return 0;
+}
+
 static int pol_match(struct rule *rule, char *path, char *type)
 {
 	/* check if this rule matches on path and type */
@@ -253,7 +273,7 @@ static int pol_match(struct rule *rule, char *path, char *type)
 	int typeok = 0;
 
 	while (rule) {
-		if (rule->name == rule_path) {
+		if (rule_is_path(rule)) {
 			if (pathok == 0)
 				pathok = -1;
 			if (path && fnmatch(rule->value, path, 0) == 0)
@@ -403,6 +423,7 @@ void pol_add(struct dev_policy **pol,
 /*
  * disk_policy() gathers policy information for the
  * disk described in the given mdinfo (disk.{major,minor}).
+ * prefer 'by-slot' over 'by-path'
  */
 struct dev_policy *disk_policy(struct mdinfo *disk)
 {
@@ -432,6 +453,7 @@ struct dev_policy *devid_policy(int dev)
  */
 
 char rule_path[] = "path";
+char rule_slot[] = "slot";
 char rule_type[] = "type";
 
 char rule_policy[] = "policy";
@@ -470,7 +492,8 @@ void policyline(char *line, char *type)
 	pr->type = type;
 	pr->rule = NULL;
 	for (w = dl_next(line); w != line ; w = dl_next(w)) {
-		if (try_rule(w, rule_path, &pr->rule))
+		if (try_rule(w, rule_slot, &pr->rule)
+		    || try_rule(w, rule_path, &pr->rule))
 			config_rules_has_path = 1;
 		else if (! try_rule(w, rule_type, &pr->rule) &&
 			 ! try_rule(w, pol_metadata, &pr->rule) &&
@@ -793,31 +816,59 @@ char *find_rule(struct rule *rule, char *rule_type)
 	return NULL;
 }
 
+/* prefer 'slot' rules over plain 'path' rules */
+static struct rule *find_path_rule(struct rule *rule)
+{
+	struct rule *path_rule = NULL;
+
+	while (rule) {
+		if (rule->name == rule_slot)
+			return rule;
+		if (!path_rule && rule->name == rule_path)
+			path_rule = rule;
+		rule = rule->next;
+	}
+
+	return path_rule;
+}
+
 #define UDEV_RULE_FORMAT \
 "ACTION==\"add\", SUBSYSTEM==\"block\", " \
-"ENV{DEVTYPE}==\"%s\", ENV{ID_PATH}==\"%s\", " \
+"ENV{DEVTYPE}==\"%s\", ENV{%s}==\"%s\", " \
 "RUN+=\"" BINDIR "/mdadm --incremental $env{DEVNAME}\"\n"
 
 #define UDEV_RULE_FORMAT_NOTYPE \
 "ACTION==\"add\", SUBSYSTEM==\"block\", " \
-"ENV{ID_PATH}==\"%s\", " \
+"ENV{%s}==\"%s\", " \
 "RUN+=\"" BINDIR "/mdadm --incremental $env{DEVNAME}\"\n"
 
+
+
 /* Write rule in the rule file. Use format from UDEV_RULE_FORMAT */
 int write_rule(struct rule *rule, int fd, int force_part)
 {
 	char line[1024];
-	char *pth = find_rule(rule, rule_path);
+	struct rule *path_rule = find_path_rule(rule);
 	char *typ = find_rule(rule, rule_type);
-	if (!pth)
+	char *slot_path, *pth;
+
+	if (!path_rule)
 		return -1;
 
+	pth = path_rule->value;
+	if (path_rule->name == rule_slot)
+		slot_path = "ID_SLOT";
+	else
+		slot_path = "ID_PATH";
+
 	if (force_part)
 		typ = type_part;
 	if (typ)
-		snprintf(line, sizeof(line) - 1, UDEV_RULE_FORMAT, typ, pth);
+		snprintf(line, sizeof(line) - 1, UDEV_RULE_FORMAT,
+			 typ, slot_path, pth);
 	else
-		snprintf(line, sizeof(line) - 1, UDEV_RULE_FORMAT_NOTYPE, pth);
+		snprintf(line, sizeof(line) - 1, UDEV_RULE_FORMAT_NOTYPE,
+			 slot_path, pth);
 	return write(fd, line, strlen(line)) == (int)strlen(line);
 }
 
@@ -829,6 +880,7 @@ int generate_entries(int fd)
 {
 	struct pol_rule *loop, *dup;
 	char *loop_value, *dup_value;
+	struct rule *rule;
 	int duplicate;
 
 	for (loop = config_rules; loop; loop = loop->next) {
@@ -841,18 +893,20 @@ int generate_entries(int fd)
 		loop_value = find_rule(loop->rule, pol_act);
 		if (!loop_value || map_act(loop_value) < act_spare_same_slot)
 			continue;
-		loop_value = find_rule(loop->rule, rule_path);
-		if (!loop_value)
+		rule = find_path_rule(loop->rule);
+		if (!rule)
 			continue;
+		loop_value = rule->value;
 		for (dup = config_rules; dup != loop; dup = dup->next) {
 			if (dup->type != rule_policy && loop->type != rule_part)
 				continue;
 			dup_value = find_rule(dup->rule, pol_act);
 			if (!dup_value || map_act(dup_value) < act_spare_same_slot)
 				continue;
-			dup_value = find_rule(dup->rule, rule_path);
-			if (!dup_value)
+			rule = find_path_rule(dup->rule);
+			if (!rule)
 				continue;
+			dup_value = rule->value;
 			if (strcmp(loop_value, dup_value) == 0) {
 				duplicate = 1;
 				break;
-- 
2.4.6

[PATCH 03/17] 'act_spare' should be the minimum requirement for dynamic domains

[Song Liu]

From: Dan Williams <djbw@fb.com>

Let bare devices that fit the domain become raid members.  Rather than
require the tracking of 'failed slots', simply define a group of slots
for the domain and let any bare devices attached in those domains join
a compatible array.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 policy.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/policy.c b/policy.c
index 960a37e..6afafad 100644
--- a/policy.c
+++ b/policy.c
@@ -874,7 +874,7 @@ int write_rule(struct rule *rule, int fd, int force_part)
 
 /* Generate single entry in udev rule basing on POLICY line found in config
  * file. Take only those with paths, only first occurrence if paths are equal
- * and if actions supports handling of spares (>=act_spare_same_slot)
+ * and if actions supports handling of spares (>=act_spare)
  */
 int generate_entries(int fd)
 {
@@ -891,7 +891,7 @@ int generate_entries(int fd)
 		/* only policies with paths and with actions supporting
 		 * bare disks are considered */
 		loop_value = find_rule(loop->rule, pol_act);
-		if (!loop_value || map_act(loop_value) < act_spare_same_slot)
+		if (!loop_value || map_act(loop_value) < act_spare)
 			continue;
 		rule = find_path_rule(loop->rule);
 		if (!rule)
@@ -901,7 +901,7 @@ int generate_entries(int fd)
 			if (dup->type != rule_policy && loop->type != rule_part)
 				continue;
 			dup_value = find_rule(dup->rule, pol_act);
-			if (!dup_value || map_act(dup_value) < act_spare_same_slot)
+			if (!dup_value || map_act(dup_value) < act_spare)
 				continue;
 			rule = find_path_rule(dup->rule);
 			if (!rule)
-- 
2.4.6

[PATCH 04/17] fix variable offset when ':' is present in the device name

[Song Liu]

From: Dan Williams <djbw@fb.com>

Stopping at the first ':' precludes using device names like
/dev/disk/by-path/pci-0000:04:00.0-sas-0x500605b005846060:1:0-0x5000cca01a7c5605:10

so, check that the name is not ambiguous before chopping off the offset at the end

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 Create.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/Create.c b/Create.c
index 21d1374..2642b36 100644
--- a/Create.c
+++ b/Create.c
@@ -306,7 +306,13 @@ int Create(struct supertype *st, char *mddev,
 			continue;
 		}
 		if (data_offset == VARIABLE_OFFSET) {
-			doff = strchr(dname, ':');
+			doff = strrchr(dname, ':');
+			if (stat(dname, &stb) == 0) {
+				pr_err("cannot determine if %s is a device name, or a device with a data-offset argument of '%s'\n",
+				       dname, doff+1);
+				pr_err("check that you have specified a data-offset for all array members\n");
+				exit(2);
+			}
 			if (doff) {
 				*doff++ = 0;
 				dv->data_offset = parse_size(doff);
-- 
2.4.6

[PATCH 05/17] imsm: quiet detail platform

[Song Liu]

From: Dan Williams <djbw@fb.com>

Don't squawk when platform capabilities missing, in preparation for
generic SES platform support.  Undo some indentation as well.

Cc: Lukasz Dorau <lukasz.dorau@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 super-intel.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index e609e0c..330b24f 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -1935,8 +1935,9 @@ static int detail_platform_imsm(int verbose, int enumerate_only, char *controlle
 		if (controller_path && (compare_paths(hba->path, controller_path) != 0))
 			continue;
 		if (!find_imsm_capability(hba)) {
-			pr_err("imsm capabilities not found for controller: %s (type %s)\n",
-				hba->path, get_sys_dev_type(hba->type));
+			if (verbose > 0)
+				pr_err("imsm capabilities not found for controller: %s (type %s)\n",
+				       hba->path, get_sys_dev_type(hba->type));
 			continue;
 		}
 		result = 0;
-- 
2.4.6

[PATCH 06/17] ses: workaround sysfs deprecated

[Song Liu]

From: Dan Williams <djbw@fb.com>

sysfs deprecated makes forwarding udev events from a scsi device to its
block device difficult.  Provide a workalike for the udev rule:

ACTION=="change", SUBSYSTEM=="scsi", ENV{DEVTYPE}=="scsi_device", \
	TEST=="block", ATTR{block/*/uevent}="change"

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 Makefile                                                | 17 +++++++++++++++--
 forward-scsi-uevents                                    | 14 ++++++++++++++
 ...enclosure-slot.rules => udev-enclosure-slot.rules.in |  2 --
 udev-workaround-sysfs-deprecated.in                     |  2 ++
 4 files changed, 31 insertions(+), 4 deletions(-)
 create mode 100755 forward-scsi-uevents
 rename udev-enclosure-slot.rules => udev-enclosure-slot.rules.in (87%)
 create mode 100644 udev-workaround-sysfs-deprecated.in

diff --git a/Makefile b/Makefile
index 4eb2dad..2963884 100644
--- a/Makefile
+++ b/Makefile
@@ -292,7 +292,17 @@ install-man: mdadm.8 md.4 mdadm.conf.5 mdmon.8
 	$(INSTALL) -D -m 644 md.4 $(DESTDIR)$(MAN4DIR)/md.4
 	$(INSTALL) -D -m 644 mdadm.conf.5 $(DESTDIR)$(MAN5DIR)/mdadm.conf.5
 
-install-udev: udev-md-raid-arrays.rules udev-md-raid-assembly.rules
+udev-enclosure-slot.rules: udev-enclosure-slot.rules.in udev-workaround-sysfs-deprecated.in Makefile
+	@echo "$@: generate"
+	@/bin/echo -e "# do not edit this file, it will be overwritten on update\n" > $@
+	@if [ ! -d /sys/class/block ]; then \
+		echo "$@: applying sysfs-deprecated workaround"; \
+		cat udev-workaround-sysfs-deprecated.in >> $@; \
+	fi
+	@cat udev-enclosure-slot.rules.in >> $@
+
+install-udev: udev-md-raid-arrays.rules udev-md-raid-assembly.rules udev-enclosure-slot.rules \
+	      forward-scsi-uevents
 	$(INSTALL) -D -m 655 ses_slot_id $(DESTDIR)$(LIBDIR)/ses_slot_id
 	$(INSTALL) -D -m 644 udev-enclosure-slot.rules $(DESTDIR)$(UDEVDIR)/rules.d/62-md-enclosure-slot.rules
 	@for file in 63-md-raid-arrays.rules 64-md-raid-assembly.rules ; \
@@ -301,6 +311,9 @@ install-udev: udev-md-raid-arrays.rules udev-md-raid-assembly.rules
 	   $(INSTALL) -D -m 644 .install.tmp.1 $(DESTDIR)$(UDEVDIR)/rules.d/$$file ; \
 	   rm -f .install.tmp.1; \
 	done
+	@if [ ! -d /sys/class/block ]; then \
+		$(INSTALL) -D -m 655 forward-scsi-uevents $(DESTDIR)$(UDEVDIR)/forward-scsi-uevents; \
+	fi
 
 install-systemd: systemd/mdmon@.service
 	@for file in mdmon@.service mdmonitor.service mdadm-last-resort@.timer \
@@ -331,7 +344,7 @@ clean :
 	mdassemble mdassemble.static mdassemble.auto mdassemble.uclibc \
 	mdassemble.klibc swap_super \
 	init.cpio.gz mdadm.uclibc.static test_stripe raid6check raid6check.o mdmon \
-	mdadm.8
+	mdadm.8 udev-enclosure-slot.rules
 
 dist : clean
 	./makedist
diff --git a/forward-scsi-uevents b/forward-scsi-uevents
new file mode 100755
index 0000000..d43687f
--- /dev/null
+++ b/forward-scsi-uevents
@@ -0,0 +1,14 @@
+#!/bin/sh
+
+# on sysfs-deprecated hosts the default udev rules are unable to forward
+# udev events to the block device, this is a hack to enable this
+# forwarding by searching for the matching device path
+
+for i in /sys/block/sd*
+do
+	DEV=$(readlink -f $i/device)
+	if [ $DEV = /sys$1 ]; then
+		echo "change" > $i/uevent
+		break
+	fi
+done
diff --git a/udev-enclosure-slot.rules b/udev-enclosure-slot.rules.in
similarity index 87%
rename from udev-enclosure-slot.rules
rename to udev-enclosure-slot.rules.in
index 75c6376..50e3c45 100644
--- a/udev-enclosure-slot.rules
+++ b/udev-enclosure-slot.rules.in
@@ -1,5 +1,3 @@
-# do not edit this file, it will be overwritten on update
-
 SUBSYSTEM!="block", GOTO="ses_slot_end"
 
 ACTION!="add|change", GOTO="ses_slot_end"
diff --git a/udev-workaround-sysfs-deprecated.in b/udev-workaround-sysfs-deprecated.in
new file mode 100644
index 0000000..65a21b3
--- /dev/null
+++ b/udev-workaround-sysfs-deprecated.in
@@ -0,0 +1,2 @@
+ACTION=="change", SUBSYSTEM=="scsi", ENV{DEVTYPE}=="scsi_device", TEST=="scsi_disk", RUN="forward-scsi-uevents %p"
+
-- 
2.4.6

[PATCH 07/17] prepare for enclosure platform details

[Song Liu]

From: Dan Williams <djbw@localhost.localdomain>

Make platform description entry points independent of the superswitch
for platform details like enclosures that have no metadata ties.  This
allows generically looping through a list of platform_detail operations.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 Create.c      |  2 +-
 Detail.c      | 59 ++++++++++++++++++++++++++++-------------------------------
 mdadm.c       |  9 ++++++++-
 mdadm.h       | 19 ++++++++++++++++---
 super-intel.c |  8 ++++++--
 util.c        |  6 ++++++
 6 files changed, 65 insertions(+), 38 deletions(-)

diff --git a/Create.c b/Create.c
index 2642b36..fda7f67 100644
--- a/Create.c
+++ b/Create.c
@@ -535,7 +535,7 @@ int Create(struct supertype *st, char *mddev,
 		warn = 1;
 	}
 
-	if (st->ss->detail_platform && st->ss->detail_platform(0, 1, NULL) != 0) {
+	if (st->ss->platform && st->ss->platform->detail(0, 1, NULL) != 0) {
 		if (c->runstop != 1 || c->verbose >= 0)
 			pr_err("%s unable to enumerate platform support\n"
 				"    array may not be compatible with hardware/firmware\n",
diff --git a/Detail.c b/Detail.c
index 200f65f..f71afb9 100644
--- a/Detail.c
+++ b/Detail.c
@@ -717,47 +717,44 @@ out:
 	return rv;
 }
 
-int Detail_Platform(struct superswitch *ss, int scan, int verbose, int export, char *controller_path)
+int Detail_Platform(const struct platform_ops *platform, int scan, int verbose, int export, char *controller_path)
 {
-	/* display platform capabilities for the given metadata format
-	 * 'scan' in this context means iterate over all metadata types
+	/* display platform capabilities
+	 * 'scan' in this context means iterate over all platform types
 	 */
-	int i;
-	int err = 1;
-
-	if (ss && export && ss->export_detail_platform)
-		err = ss->export_detail_platform(verbose, controller_path);
-	else if (ss && ss->detail_platform)
-		err = ss->detail_platform(verbose, 0, controller_path);
-	else if (ss) {
+	struct platform_ops const **p;
+	int err = 0;
+
+	if (platform && verbose > 0)
+		pr_err("checking for %s platform components\n",
+			platform->name);
+
+	if (platform && export)
+		err = platform->export_detail(verbose, controller_path);
+	else if (platform)
+		err = platform->detail(verbose, 0, controller_path);
+	else if (!scan) {
 		if (verbose > 0)
-			pr_err("%s metadata is platform independent\n",
-				ss->name ? : "[no name]");
-	} else if (!scan) {
-		if (verbose > 0)
-			pr_err("specify a metadata type or --scan\n");
+			pr_err("no platform components found\n");
 	}
 
 	if (!scan)
 		return err;
 
-	err = 0;
-	for (i = 0; superlist[i]; i++) {
-		struct superswitch *meta = superlist[i];
-
-		if (meta == ss)
+	for (p = platform_list; p; p++) {
+		/* enumerated above */
+		if (*p == platform)
 			continue;
+
 		if (verbose > 0)
-			pr_err("checking metadata %s\n",
-				meta->name ? : "[no name]");
-		if (!meta->detail_platform) {
-			if (verbose > 0)
-				pr_err("%s metadata is platform independent\n",
-					meta->name ? : "[no name]");
-		} else if (export && meta->export_detail_platform) {
-			err |= meta->export_detail_platform(verbose, controller_path);
-		} else
-			err |= meta->detail_platform(verbose, 0, controller_path);
+			pr_err("checking for %s platform components\n",
+				(*p)->name);
+
+		if (export)
+			err |= (*p)->export_detail(verbose, controller_path);
+		else
+			err |= (*p)->detail(verbose, 0, controller_path);
+
 	}
 
 	return err;
diff --git a/mdadm.c b/mdadm.c
index f56a8cf..98ef898 100644
--- a/mdadm.c
+++ b/mdadm.c
@@ -1492,7 +1492,14 @@ int main(int argc, char *argv[])
 			}
 			rv = Examine(devlist, &c, ss);
 		} else if (devmode == DetailPlatform) {
-			rv = Detail_Platform(ss ? ss->ss : NULL, ss ? c.scan : 1,
+			const struct platform_ops *platform;
+
+			if (ss)
+				platform = ss->ss->platform;
+			else
+				platform = NULL;
+
+			rv = Detail_Platform(platform, platform ? c.scan : 1,
 					     c.verbose, c.export,
 					     devlist ? devlist->devname : NULL);
 		} else if (devlist == NULL) {
diff --git a/mdadm.h b/mdadm.h
index 167ca0c..f3b24b3 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -687,6 +687,17 @@ struct reshape {
 	unsigned long long new_size; /* New size of array in sectors */
 };
 
+/* Platform details describe hardware / firmware capabilites and
+ * limitations.  They can be tied to a metadata format.  For example,
+ * system firmware with a set of raid capabilities and expected
+ * controller, or describe the topology disk enclosures.
+ */
+extern const struct platform_ops {
+	int (*detail)(int verbose, int enumerate_only, char *hwdevice);
+	int (*export_detail)(int verbose, char *hwdevice);
+	char *name;
+} *platform_list[];
+
 /* A superswitch provides entry point the a metadata handler.
  *
  * The superswitch primarily operates on some "metadata" that
@@ -734,8 +745,7 @@ extern struct superswitch {
 	void (*export_detail_super)(struct supertype *st);
 
 	/* Optional: platform hardware / firmware details */
-	int (*detail_platform)(int verbose, int enumerate_only, char *controller_path);
-	int (*export_detail_platform)(int verbose, char *controller_path);
+	const struct platform_ops *platform;
 
 	/* Used:
 	 *   to get uuid to storing in bitmap metadata
@@ -1008,6 +1018,8 @@ extern struct superswitch super0, super1;
 extern struct superswitch super_imsm, super_ddf;
 extern struct superswitch mbr, gpt;
 
+extern const struct platform_ops imsm_platform;
+
 struct metadata_update {
 	int	len;
 	char	*buf;
@@ -1263,7 +1275,8 @@ extern int Create(struct supertype *st, char *mddev,
 		  unsigned long long data_offset);
 
 extern int Detail(char *dev, struct context *c);
-extern int Detail_Platform(struct superswitch *ss, int scan, int verbose, int export, char *controller_path);
+extern int Detail_Platform(const struct platform_ops *platform, int scan, int verbose,
+			   int export, char *controller_path);
 extern int Query(char *dev);
 extern int ExamineBadblocks(char *devname, int brief, struct supertype *forcest);
 extern int Examine(struct mddev_dev *devlist, struct context *c,
diff --git a/super-intel.c b/super-intel.c
index 330b24f..53a9238 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -10606,6 +10606,11 @@ abort:
 	return ret_val;
 }
 
+const struct platform_ops imsm_platform = {
+	.detail = detail_platform_imsm,
+	.export_detail = export_detail_platform_imsm,
+	.name = "imsm",
+};
 #endif /* MDASSEMBLE */
 
 struct superswitch super_imsm = {
@@ -10620,8 +10625,6 @@ struct superswitch super_imsm = {
 	.validate_geometry = validate_geometry_imsm,
 	.add_to_super	= add_to_super_imsm,
 	.remove_from_super = remove_from_super_imsm,
-	.detail_platform = detail_platform_imsm,
-	.export_detail_platform = export_detail_platform_imsm,
 	.kill_subarray = kill_subarray_imsm,
 	.update_subarray = update_subarray_imsm,
 	.load_container	= load_container_imsm,
@@ -10631,6 +10634,7 @@ struct superswitch super_imsm = {
 	.manage_reshape = imsm_manage_reshape,
 	.recover_backup = recover_backup_imsm,
 	.copy_metadata = copy_metadata_imsm,
+	.platform = &imsm_platform,
 #endif
 	.match_home	= match_home_imsm,
 	.uuid_from_super= uuid_from_super_imsm,
diff --git a/util.c b/util.c
index 8217e11..e81c45d 100644
--- a/util.c
+++ b/util.c
@@ -1147,6 +1147,12 @@ struct superswitch *superlist[] =
 
 #if !defined(MDASSEMBLE) || defined(MDASSEMBLE) && defined(MDASSEMBLE_AUTO)
 
+const struct platform_ops *platform_list[] =
+{
+	&imsm_platform,
+	NULL,
+};
+
 struct supertype *super_by_fd(int fd, char **subarrayp)
 {
 	mdu_array_info_t array;
-- 
2.4.6

[PATCH 08/17] enclosure detection/enumeration

[Song Liu]

From: Dan Williams <djbw@fb.com>

enumerate enclosures via --detail-platform --enclosure

Add the plumbing for the new command line parameter to
--detail-platform, and implement simple enumeration of an enclosure via
the attributes exported by ses and the /dev/disk/by-slot infrastructure.

Ex:
Enclosure0:
             Id : 0x5f80f41f1d6a60ff
         Vendor : facebook
          Model : Knox2U
          Slots : 15

         Slot00 : "OK" /dev/sdk YHJ9RP2D
         Slot01 : "OK" /dev/sdj YHJ6ZWJD
         Slot02 : "OK" /dev/sdi YHJ6BTLD
         Slot03 : "OK" /dev/sdg YHJ9WG2D
         Slot04 : "OK" /dev/sdf YHJ6WYGD
         Slot05 : "OK" /dev/sdl YHJ9S47D
         Slot06 : "OK" /dev/sdh YHJ9PZED
         Slot07 : "OK" /dev/sde YHJ7LW8D
         Slot08 : "OK" /dev/sdd YHJ7SSDD
         Slot09 : "OK" /dev/sdc YHJ7SRYD
         Slot10 : "OK" /dev/sdaa CVCV153103Z5120BGN
         Slot11 : "OK" /dev/sdx CVCV213103AK120BGN
         Slot12 : "OK" /dev/sdy CVCV203506EU120BGN
         Slot13 : "OK" /dev/sdz CVCV203602X2120BGN
         Slot14 : "OK" /dev/sdb YHJ9S3PD

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 Detail.c      |  12 +--
 Makefile      |   4 +-
 ReadMe.c      |   2 +
 enclosure.c   | 314 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 mdadm.c       |  42 ++++++--
 mdadm.h       |   5 +
 super-intel.c |   2 -
 sysfs.c       |  11 +-
 util.c        |   1 +
 9 files changed, 374 insertions(+), 19 deletions(-)
 create mode 100644 enclosure.c

diff --git a/Detail.c b/Detail.c
index f71afb9..8fb61cc 100644
--- a/Detail.c
+++ b/Detail.c
@@ -717,7 +717,7 @@ out:
 	return rv;
 }
 
-int Detail_Platform(const struct platform_ops *platform, int scan, int verbose, int export, char *controller_path)
+int Detail_Platform(const struct platform_ops *platform, int scan, int verbose, int export, char *hwdevice)
 {
 	/* display platform capabilities
 	 * 'scan' in this context means iterate over all platform types
@@ -730,9 +730,9 @@ int Detail_Platform(const struct platform_ops *platform, int scan, int verbose,
 			platform->name);
 
 	if (platform && export)
-		err = platform->export_detail(verbose, controller_path);
+		err = platform->export_detail(verbose, hwdevice);
 	else if (platform)
-		err = platform->detail(verbose, 0, controller_path);
+		err = platform->detail(verbose, 0, hwdevice);
 	else if (!scan) {
 		if (verbose > 0)
 			pr_err("no platform components found\n");
@@ -741,7 +741,7 @@ int Detail_Platform(const struct platform_ops *platform, int scan, int verbose,
 	if (!scan)
 		return err;
 
-	for (p = platform_list; p; p++) {
+	for (p = platform_list; *p; p++) {
 		/* enumerated above */
 		if (*p == platform)
 			continue;
@@ -751,9 +751,9 @@ int Detail_Platform(const struct platform_ops *platform, int scan, int verbose,
 				(*p)->name);
 
 		if (export)
-			err |= (*p)->export_detail(verbose, controller_path);
+			err |= (*p)->export_detail(verbose, hwdevice);
 		else
-			err |= (*p)->detail(verbose, 0, controller_path);
+			err |= (*p)->detail(verbose, 0, hwdevice);
 
 	}
 
diff --git a/Makefile b/Makefile
index 2963884..3f9c7d2 100644
--- a/Makefile
+++ b/Makefile
@@ -134,7 +134,7 @@ OBJS =  mdadm.o config.o policy.o mdstat.o  ReadMe.o util.o maps.o lib.o \
 	mdopen.o super0.o super1.o super-ddf.o super-intel.o bitmap.o \
 	super-mbr.o super-gpt.o \
 	restripe.o sysfs.o sha1.o mapfile.o crc32.o sg_io.o msg.o xmalloc.o \
-	platform-intel.o probe_roms.o crc32c.o
+	platform-intel.o probe_roms.o crc32c.o enclosure.o
 
 CHECK_OBJS = restripe.o sysfs.o maps.o lib.o xmalloc.o dlink.o
 
@@ -147,7 +147,7 @@ MON_OBJS = mdmon.o monitor.o managemon.o util.o maps.o mdstat.o sysfs.o \
 	Kill.o sg_io.o dlink.o ReadMe.o super-intel.o \
 	super-mbr.o super-gpt.o \
 	super-ddf.o sha1.o crc32.o msg.o bitmap.o xmalloc.o \
-	platform-intel.o probe_roms.o
+	platform-intel.o probe_roms.o enclosure.o
 
 MON_SRCS = $(patsubst %.o,%.c,$(MON_OBJS))
 
diff --git a/ReadMe.c b/ReadMe.c
index fb5a671..8989e5c 100644
--- a/ReadMe.c
+++ b/ReadMe.c
@@ -97,6 +97,7 @@ struct option long_options[] = {
     {"examine-bitmap", 0, 0, 'X'},
     {"auto-detect", 0, 0, AutoDetect},
     {"detail-platform", 0, 0, DetailPlatform},
+    {"enclosure", 2, 0, Enclosure},
     {"kill-subarray", 1, 0, KillSubarray},
     {"update-subarray", 1, 0, UpdateSubarray},
     {"udev-rules", 2, 0, UdevRules},
@@ -502,6 +503,7 @@ char Help_misc[] =
 "                       device relates to the md driver\n"
 "  --detail      -D   : Display details of an array\n"
 "  --detail-platform  : Display hardware/firmware details\n"
+"  --enclosure        : Display disks by physical/bay location\n"
 "  --examine     -E   : Examine superblock on an array component\n"
 "  --examine-bitmap -X: Display contents of a bitmap file\n"
 "  --examine-badblocks: Display list of known bad blocks on device\n"
diff --git a/enclosure.c b/enclosure.c
new file mode 100644
index 0000000..76ed30d
--- /dev/null
+++ b/enclosure.c
@@ -0,0 +1,314 @@
+/*
+ * mdadm - Enclosure management support
+ *
+ * Copyright (C) 2013 Facebook
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+ */
+#include "mdadm.h"
+
+#include <ctype.h>
+#include <sys/types.h>
+#include <dirent.h>
+
+
+#define ENCLOSURE_STRING 40
+struct slot {
+	struct slot *next;
+	char *devname;
+	int slot_num;
+};
+
+const char ENC_BASE[] = "/sys/class/enclosure";
+
+struct enclosure {
+	char vendor[ENCLOSURE_STRING];
+	char model[ENCLOSURE_STRING];
+	char id[ENCLOSURE_STRING];
+	struct slot *slot;
+	char *devname;
+	int num_slots;
+	struct enclosure *next;
+};
+
+static void free_slot(struct slot *slot)
+{
+	while (slot) {
+		struct slot *next = slot->next;
+
+		free(slot->devname);
+		free(slot);
+		slot = next;
+	}
+}
+
+static void free_enclosure(struct enclosure *enclosure)
+{
+	while (enclosure) {
+		struct enclosure *next = enclosure->next;
+
+		free_slot(enclosure->slot);
+		free(enclosure->devname);
+		free(enclosure);
+		enclosure = next;
+	}
+}
+
+static char *devpath_to_diskname_deprecated(const char *devpath)
+{
+	struct dirent *de = NULL;
+	char *r_devpath;
+	DIR *b_dir;
+	char *diskname = NULL;
+
+	r_devpath = realpath(devpath, NULL);
+	if (!r_devpath)
+		return NULL;
+
+	b_dir = opendir("/sys/block");
+	if (!b_dir)
+		goto out;
+
+	for (de = readdir(b_dir); de; de = readdir(b_dir)) {
+		char diskpath[100];
+		char *r_diskpath;
+		int done;
+
+		if (de->d_name[0] == '.')
+			continue;
+		sprintf(diskpath, "/sys/block/%s/device", de->d_name);
+		r_diskpath = realpath(diskpath, NULL);
+		if (!r_diskpath)
+			continue;
+
+		done = strcmp(r_devpath, r_diskpath) == 0;
+		free(r_diskpath);
+		if (done) {
+			diskname = xstrdup(de->d_name);
+			break;
+		}
+	}
+	closedir(b_dir);
+ out:
+	free(r_devpath);
+	return de ? diskname : NULL;
+}
+
+static char *device_to_diskname(const char *devpath)
+{
+	struct dirent *de = NULL;
+	struct stat st;
+	char *b_path;
+	DIR *b_dir;
+
+	if (stat(devpath, &st))
+		return NULL; /* no device */
+
+	xasprintf(&b_path, "%s/block", devpath);
+	b_dir = opendir(b_path);
+
+	/* on sysfs-deprecated hosts we need to do this the long way... */
+	if (!b_dir)
+		goto out;
+
+	for (de = readdir(b_dir); de; de = readdir(b_dir)) {
+		if (de->d_name[0] == '.')
+			continue;
+		if (de->d_type != DT_DIR)
+			continue;
+		/* 1:1 mapping of scsi device to block device */
+		break;
+	}
+	closedir(b_dir);
+ out:
+	free(b_path);
+
+	return de ? xstrdup(de->d_name) : devpath_to_diskname_deprecated(devpath);
+}
+
+static struct slot *parse_slots(struct enclosure *enclosure, DIR *slot_dir)
+{
+	struct slot *slot_list = NULL, **pos = &slot_list;
+	char path[PATH_MAX];
+	struct dirent *de;
+
+	if (!slot_dir)
+		return NULL;
+
+	for (de = readdir(slot_dir); de; de = readdir(slot_dir)) {
+		char slot_id[ENCLOSURE_STRING];
+		struct slot *slot;
+		char *e;
+
+		if (de->d_name[0] == '.')
+			continue;
+		if (de->d_type != DT_DIR)
+			continue;
+
+		snprintf(path, sizeof(path), "%s/%s/%s/slot",
+			 ENC_BASE, enclosure->devname, de->d_name);
+		if (load_sys_n(path, slot_id, sizeof(slot_id)) != 0)
+			continue;
+
+		slot = xcalloc(1, sizeof(*slot));
+		*pos = slot;
+		pos = &slot->next;
+		slot->slot_num = strtoul(slot_id, &e, 10);
+
+		snprintf(path, sizeof(path), "%s/%s/%s/device",
+			 ENC_BASE, enclosure->devname, de->d_name);
+		slot->devname = device_to_diskname(path);
+	}
+
+	if (de) {
+		free_slot(slot_list);
+		slot_list = NULL;
+	}
+
+	return slot_list;
+}
+
+static struct enclosure *parse_enclosures(void)
+{
+	struct enclosure *enc_list = NULL, **pos = &enc_list;
+	struct dirent *de;
+	DIR *enc_dir;
+
+	enc_dir = opendir(ENC_BASE);
+	if (!enc_dir)
+		return NULL;
+
+	for (de = readdir(enc_dir); de; de = readdir(enc_dir)) {
+		char num_slots[ENCLOSURE_STRING];
+		struct enclosure *enclosure;
+		char path[PATH_MAX];
+		DIR *slots;
+		char *e;
+
+		if (de->d_name[0] == '.')
+			continue;
+
+		enclosure = xcalloc(1, sizeof(*enclosure));
+		enclosure->devname = xstrdup(de->d_name);
+		*pos = enclosure;
+		pos = &enclosure->next;
+
+		snprintf(path, sizeof(path), "%s/%s/id", ENC_BASE, de->d_name);
+		if (load_sys_n(path, enclosure->id, sizeof(enclosure->id)) != 0)
+			break;
+
+		snprintf(path, sizeof(path), "%s/%s/components", ENC_BASE, de->d_name);
+		if (load_sys_n(path, num_slots, sizeof(num_slots)) != 0)
+			break;
+		enclosure->num_slots = strtoul(num_slots, &e, 10);
+
+		snprintf(path, sizeof(path), "%s/%s/device/model", ENC_BASE, de->d_name);
+		load_sys_n(path, enclosure->model, sizeof(enclosure->model));
+
+		snprintf(path, sizeof(path), "%s/%s/device/vendor", ENC_BASE, de->d_name);
+		load_sys_n(path, enclosure->vendor, sizeof(enclosure->vendor));
+
+		snprintf(path, sizeof(path), "%s/%s", ENC_BASE, de->d_name);
+		slots = opendir(path);
+		if (!slots)
+			break;
+
+		enclosure->slot = parse_slots(enclosure, slots);
+		closedir(slots);
+
+		if (!enclosure->slot)
+			break;
+
+	}
+
+	if (de) {
+		free_enclosure(enc_list);
+		enc_list = NULL;
+	}
+
+	closedir(enc_dir);
+
+	return enc_list;
+}
+
+static int detail_platform_enclosure(int verbose, int enumerate, char *enclosure_name)
+{
+	struct enclosure *enclosure = parse_enclosures();
+	struct enclosure *e;
+	int i;
+
+	if (!enclosure)
+		return 0;
+
+	for (e = enclosure, i = 0; e; e = e->next, i++) {
+		struct slot *s;
+
+		if (i)
+			printf("\n");
+		printf("Enclosure%d:\n", i);
+		printf("             Id : %s\n", e->id);
+		if (e->vendor[0])
+			printf("         Vendor : %s\n", e->vendor);
+		if (e->model[0])
+			printf("          Model : %s\n", e->model);
+		printf("          Slots : %d\n", e->num_slots);
+		if (e->slot)
+			printf("\n");
+
+		for (s = e->slot; s; s = s->next) {
+			int fd;
+			char buf[255];
+
+			printf("         Slot%02d :", s->slot_num);
+			snprintf(buf, sizeof(buf), "/dev/%s", s->devname ? : "");
+			printf("%s%s", buf[0] ? " " : "", buf);
+			if ((fd = dev_open(buf, O_RDONLY)) >= 0) {
+				memset(buf, 0, sizeof(buf));
+				if (scsi_get_serial(fd, buf, sizeof(buf)) == 0
+				    && buf[3]) {
+					char *serial;
+					char *end;
+
+					serial = &buf[4];
+
+					while (isspace(*serial))
+						serial++;
+					end = serial;
+					while (!isspace(*end) && *end != '\0')
+						end++;
+					if (end - &buf[4] > buf[3])
+						end = &buf[4] + buf[3] - 1;
+					*end = '\0';
+					printf(" %s", serial);
+				}
+				close(fd);
+			}
+			printf("\n");
+		}
+	}
+
+	free_enclosure(enclosure);
+	return 0;
+}
+
+static int export_detail_enclosure(int verbose, char *enclosure_name)
+{
+	return 0;
+}
+
+const struct platform_ops enclosure_platform = {
+	.detail = detail_platform_enclosure,
+	.export_detail = export_detail_enclosure,
+	.name = "enclosure",
+};
diff --git a/mdadm.c b/mdadm.c
index 98ef898..0e46177 100644
--- a/mdadm.c
+++ b/mdadm.c
@@ -154,6 +154,14 @@ int main(int argc, char *argv[])
 		case 'Y': c.export++;
 			continue;
 
+		case Enclosure:
+			  if (c.enclosure) {
+				  pr_err("--enclosure already specified\n");
+				  exit(2);
+			  }
+			  c.enclosure = optarg ? optarg : "scan";
+			  continue;
+
 		case HomeHost:
 			if (strcasecmp(optarg, "<ignore>") == 0)
 				c.require_homehost = 0;
@@ -1492,16 +1500,38 @@ int main(int argc, char *argv[])
 			}
 			rv = Examine(devlist, &c, ss);
 		} else if (devmode == DetailPlatform) {
+			char *controller = devlist ? devlist->devname : NULL;
 			const struct platform_ops *platform;
+			int scan, rv = 0;
 
-			if (ss)
-				platform = ss->ss->platform;
+			if (ss || c.enclosure)
+				scan = c.scan;
 			else
-				platform = NULL;
+				scan = 1;
 
-			rv = Detail_Platform(platform, platform ? c.scan : 1,
-					     c.verbose, c.export,
-					     devlist ? devlist->devname : NULL);
+			if (scan) {
+				rv = Detail_Platform(NULL, scan, c.verbose,
+						      c.export, controller);
+			} else {
+				/* metadata format specific details that
+				 * may be tied to a given controller
+				 */
+				if (ss) {
+					platform = ss->ss->platform;
+					rv = Detail_Platform(platform, scan,
+							     c.verbose, c.export,
+							     controller);
+				}
+				/* detail given enclosure or scan all
+				 * enclosures
+				 */
+				if (c.enclosure) {
+					platform = &enclosure_platform;
+					rv |= Detail_Platform(platform, scan,
+							      c.verbose, c.export,
+							      c.enclosure);
+				}
+			}
 		} else if (devlist == NULL) {
 			if (devmode == 'S' && c.scan)
 				rv = stop_scan(c.verbose);
diff --git a/mdadm.h b/mdadm.h
index f3b24b3..881c0fd 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -333,6 +333,7 @@ enum special_options {
 	AutoDetect,
 	Waitclean,
 	DetailPlatform,
+	Enclosure,
 	KillSubarray,
 	UpdateSubarray,
 	IncrementalPath,
@@ -459,6 +460,7 @@ struct context {
 	char	*action;
 	int	nodes;
 	char	*homecluster;
+	char	*enclosure;
 };
 
 struct shape {
@@ -599,10 +601,12 @@ extern int sysfs_set_safemode(struct mdinfo *sra, unsigned long ms);
 extern int sysfs_set_array(struct mdinfo *info, int vers);
 extern int sysfs_add_disk(struct mdinfo *sra, struct mdinfo *sd, int resume);
 extern int sysfs_disk_to_scsi_id(int fd, __u32 *id);
+extern int scsi_get_serial(int fd, void *buf, size_t buf_len);
 extern int sysfs_unique_holder(char *devnm, long rdev);
 extern int sysfs_freeze_array(struct mdinfo *sra);
 extern int sysfs_wait(int fd, int *msec);
 extern int load_sys(char *path, char *buf);
+extern int load_sys_n(char *path, char *buf, int len);
 extern int reshape_prepare_fdlist(char *devname,
 				  struct mdinfo *sra,
 				  int raid_disks,
@@ -1019,6 +1023,7 @@ extern struct superswitch super_imsm, super_ddf;
 extern struct superswitch mbr, gpt;
 
 extern const struct platform_ops imsm_platform;
+extern const struct platform_ops enclosure_platform;
 
 struct metadata_update {
 	int	len;
diff --git a/super-intel.c b/super-intel.c
index 53a9238..4ce46ef 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -3234,8 +3234,6 @@ static void fd2devname(int fd, char *name)
 	}
 }
 
-extern int scsi_get_serial(int fd, void *buf, size_t buf_len);
-
 static int imsm_read_serial(int fd, char *devname,
 			    __u8 serial[MAX_RAID_SERIAL_LEN])
 {
diff --git a/sysfs.c b/sysfs.c
index f1fd610..d20cf34 100644
--- a/sysfs.c
+++ b/sysfs.c
@@ -27,15 +27,15 @@
 #include	<dirent.h>
 #include	<ctype.h>
 
-int load_sys(char *path, char *buf)
+int load_sys_n(char *path, char *buf, int len)
 {
 	int fd = open(path, O_RDONLY);
 	int n;
 	if (fd < 0)
 		return -1;
-	n = read(fd, buf, 1024);
+	n = read(fd, buf, len);
 	close(fd);
-	if (n <0 || n >= 1024)
+	if (n <0 || n >= len)
 		return -1;
 	buf[n] = 0;
 	if (n && buf[n-1] == '\n')
@@ -43,6 +43,11 @@ int load_sys(char *path, char *buf)
 	return 0;
 }
 
+int load_sys(char *path, char *buf)
+{
+	return load_sys_n(path, buf, 1024);
+}
+
 void sysfs_free(struct mdinfo *sra)
 {
 	while (sra) {
diff --git a/util.c b/util.c
index e81c45d..bdc84e3 100644
--- a/util.c
+++ b/util.c
@@ -1150,6 +1150,7 @@ struct superswitch *superlist[] =
 const struct platform_ops *platform_list[] =
 {
 	&imsm_platform,
+	&enclosure_platform,
 	NULL,
 };
 
-- 
2.4.6

[PATCH 09/17] Add --brief support to --enclosure

[Song Liu]

From: Dan Williams <djbw@fb.com>

Generate simple names to locally identify enclosures by id.  This is a
step towards defining enclosure names in the configuration file.

Ex:
ENCLOSURE enclosure0 id=0x5f80f41f1d6a60ff
ENCLOSURE enclosure1 id=0x5f80f41f1d6e80ff

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 Detail.c      |  6 +++++-
 enclosure.c   | 14 ++++++++++++++
 mdadm.c       |  5 ++++-
 mdadm.h       |  3 ++-
 super-intel.c | 14 +++++++++++++-
 5 files changed, 38 insertions(+), 4 deletions(-)

diff --git a/Detail.c b/Detail.c
index 8fb61cc..03764ad 100644
--- a/Detail.c
+++ b/Detail.c
@@ -717,7 +717,7 @@ out:
 	return rv;
 }
 
-int Detail_Platform(const struct platform_ops *platform, int scan, int verbose, int export, char *hwdevice)
+int Detail_Platform(const struct platform_ops *platform, int scan, int verbose, int export, int brief, char *hwdevice)
 {
 	/* display platform capabilities
 	 * 'scan' in this context means iterate over all platform types
@@ -731,6 +731,8 @@ int Detail_Platform(const struct platform_ops *platform, int scan, int verbose,
 
 	if (platform && export)
 		err = platform->export_detail(verbose, hwdevice);
+	else if (platform && brief)
+		err = platform->brief_detail(verbose, hwdevice);
 	else if (platform)
 		err = platform->detail(verbose, 0, hwdevice);
 	else if (!scan) {
@@ -752,6 +754,8 @@ int Detail_Platform(const struct platform_ops *platform, int scan, int verbose,
 
 		if (export)
 			err |= (*p)->export_detail(verbose, hwdevice);
+		else if (brief)
+			err |= (*p)->brief_detail(verbose, hwdevice);
 		else
 			err |= (*p)->detail(verbose, 0, hwdevice);
 
diff --git a/enclosure.c b/enclosure.c
index 76ed30d..bb5e88e 100644
--- a/enclosure.c
+++ b/enclosure.c
@@ -307,8 +307,22 @@ static int export_detail_enclosure(int verbose, char *enclosure_name)
 	return 0;
 }
 
+static int brief_detail_enclosure(int verbose, char *enclosure_name)
+{
+	struct enclosure *e, *enclosure = parse_enclosures();
+	int i = 0;
+
+	for (e = enclosure; e; e = e->next)
+		printf("ENCLOSURE enclosure%d id=%s\n", i++, e->id);
+
+	free_enclosure(enclosure);
+	
+	return 0;
+}
+
 const struct platform_ops enclosure_platform = {
 	.detail = detail_platform_enclosure,
 	.export_detail = export_detail_enclosure,
+	.brief_detail = brief_detail_enclosure,
 	.name = "enclosure",
 };
diff --git a/mdadm.c b/mdadm.c
index 0e46177..7eeeb4f 100644
--- a/mdadm.c
+++ b/mdadm.c
@@ -1511,7 +1511,8 @@ int main(int argc, char *argv[])
 
 			if (scan) {
 				rv = Detail_Platform(NULL, scan, c.verbose,
-						      c.export, controller);
+						     c.export, c.brief,
+						     controller);
 			} else {
 				/* metadata format specific details that
 				 * may be tied to a given controller
@@ -1520,6 +1521,7 @@ int main(int argc, char *argv[])
 					platform = ss->ss->platform;
 					rv = Detail_Platform(platform, scan,
 							     c.verbose, c.export,
+							     c.brief,
 							     controller);
 				}
 				/* detail given enclosure or scan all
@@ -1529,6 +1531,7 @@ int main(int argc, char *argv[])
 					platform = &enclosure_platform;
 					rv |= Detail_Platform(platform, scan,
 							      c.verbose, c.export,
+							      c.brief,
 							      c.enclosure);
 				}
 			}
diff --git a/mdadm.h b/mdadm.h
index 881c0fd..7ac0b95 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -699,6 +699,7 @@ struct reshape {
 extern const struct platform_ops {
 	int (*detail)(int verbose, int enumerate_only, char *hwdevice);
 	int (*export_detail)(int verbose, char *hwdevice);
+	int (*brief_detail)(int verbose, char *hwdevice);
 	char *name;
 } *platform_list[];
 
@@ -1281,7 +1282,7 @@ extern int Create(struct supertype *st, char *mddev,
 
 extern int Detail(char *dev, struct context *c);
 extern int Detail_Platform(const struct platform_ops *platform, int scan, int verbose,
-			   int export, char *controller_path);
+			   int export, int brief, char *controller_path);
 extern int Query(char *dev);
 extern int ExamineBadblocks(char *devname, int brief, struct supertype *forcest);
 extern int Examine(struct mddev_dev *devlist, struct context *c,
diff --git a/super-intel.c b/super-intel.c
index 4ce46ef..c4a8ac7 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -1985,7 +1985,7 @@ static int detail_platform_imsm(int verbose, int enumerate_only, char *controlle
 	return result;
 }
 
-static int export_detail_platform_imsm(int verbose, char *controller_path)
+static int export_detail_print(int verbose, int print, char *controller_path)
 {
 	struct sys_dev *list, *hba;
 	int result=1;
@@ -2015,6 +2015,11 @@ static int export_detail_platform_imsm(int verbose, char *controller_path)
 	return result;
 }
 
+static int export_detail_platform_imsm(int verbose, char *controller_path)
+{
+	return export_detail_print(0, 1, controller_path);
+}
+
 #endif
 
 static int match_home_imsm(struct supertype *st, char *homehost)
@@ -10604,9 +10609,16 @@ abort:
 	return ret_val;
 }
 
+
+static int brief_detail_platform_imsm(int verbose, char *controller)
+{
+	return export_detail_print(verbose, 0, controller);
+}
+
 const struct platform_ops imsm_platform = {
 	.detail = detail_platform_imsm,
 	.export_detail = export_detail_platform_imsm,
+	.brief_detail = brief_detail_platform_imsm,
 	.name = "imsm",
 };
 #endif /* MDASSEMBLE */
-- 
2.4.6

[PATCH 10/17] config: ENCLOSURE keyword and enclosure device list expansion

[Song Liu]

From: Dan Williams <djbw@fb.com>

Parse the output of "--detail-platform --brief --enclosure" to generate
a mapping of of enclosure names to enclosure ids.

When a single sub-device name is specified try to interpret it as an
enclosure id/name and grab the device list from the enclosure to scsi
device mapping.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 config.c    | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 enclosure.c | 27 +++++++++++++++++++++++++
 mdadm.c     | 53 +++++++++++++++++++++++++++++++------------------
 mdadm.h     |  2 ++
 4 files changed, 128 insertions(+), 20 deletions(-)

diff --git a/config.c b/config.c
index c58c8fe..6ea5dd1 100644
--- a/config.c
+++ b/config.c
@@ -77,7 +77,7 @@ char DefaultAltConfFile[] = CONFFILE2;
 char DefaultAltConfDir[] = CONFFILE2 ".d";
 
 enum linetype { Devices, Array, Mailaddr, Mailfrom, Program, CreateDev,
-		Homehost, HomeCluster, AutoMode, Policy, PartPolicy, LTEnd };
+		Homehost, HomeCluster, AutoMode, Policy, PartPolicy, Enclosures, LTEnd };
 char *keywords[] = {
 	[Devices]  = "devices",
 	[Array]    = "array",
@@ -90,6 +90,7 @@ char *keywords[] = {
 	[AutoMode] = "auto",
 	[Policy]   = "policy",
 	[PartPolicy]="part-policy",
+	[Enclosures] = "enclosure",
 	[LTEnd]    = NULL
 };
 
@@ -116,6 +117,12 @@ struct conf_dev {
 	char *name;
 } *cdevlist = NULL;
 
+struct conf_enclosure {
+	struct conf_enclosure *next;
+	char *name;
+	char *id;
+} *edevlist = NULL;
+
 struct mddev_dev *load_partitions(void)
 {
 	FILE *f = fopen("/proc/partitions", "r");
@@ -506,6 +513,28 @@ void arrayline(char *line)
 	}
 }
 
+static void enclosureline(char *line)
+{
+	char *name = NULL, *id = NULL;
+	char *w;
+
+	for (w = dl_next(line); w != line; w = dl_next(w)) {
+		if (strchr(w, '=') == NULL)
+			name = w;
+		if (strncasecmp(w, "id=", 3) == 0)
+			id = w+3;
+	}
+
+	if (name && id) {
+		struct conf_enclosure *e = xmalloc(sizeof(*e));
+
+		e->name = xstrdup(name);
+		e->id = xstrdup(id);
+		e->next = edevlist;
+		edevlist = e;
+	}
+}
+
 static char *alert_email = NULL;
 void mailline(char *line)
 {
@@ -725,6 +754,9 @@ void conf_file(FILE *f)
 		case Array:
 			arrayline(line);
 			break;
+		case Enclosures:
+			enclosureline(line);
+			break;
 		case Mailaddr:
 			mailline(line);
 			break;
@@ -981,6 +1013,38 @@ struct mddev_dev *conf_get_devs()
 	return dlist;
 }
 
+extern struct mddev_dev *enclosure_get_devs(char *id);
+
+struct mddev_dev *conf_get_enclosure_devs(char *enclosure_id)
+{
+	struct conf_enclosure *e;
+	struct mddev_dev *dv;
+
+	load_conffile();
+
+	for (e = edevlist; e; e = e->next)
+		if (strcmp(e->name, enclosure_id) == 0)
+			break;
+
+	dv = enclosure_get_devs(e ? e->id : enclosure_id);
+	if (!e && !dv && strncmp("encl", enclosure_id, 4) == 0)
+		pr_err("No enclosure name '%s' defined in configuration file\n",
+		       enclosure_id);
+	return dv;
+}
+
+void conf_put_enclosure_devs(struct mddev_dev *devs)
+{
+	while (devs) {
+		struct mddev_dev *next;
+
+		next = devs->next;
+		free(devs->devname);
+		free(devs);
+		devs = next;
+	}
+}
+
 int conf_test_dev(char *devname)
 {
 	struct conf_dev *cd;
diff --git a/enclosure.c b/enclosure.c
index bb5e88e..9c569ea 100644
--- a/enclosure.c
+++ b/enclosure.c
@@ -242,6 +242,33 @@ static struct enclosure *parse_enclosures(void)
 	return enc_list;
 }
 
+/* return a mddev_dev for each populated slot in an enclosure */
+struct mddev_dev *enclosure_get_devs(char *id)
+{
+	struct enclosure *enclosure = parse_enclosures(), *e;
+	struct mddev_dev *dlist = NULL, **pos = &dlist;
+	struct slot *s;
+
+	for (e = enclosure; e; e = e->next)
+		if (strcmp(e->id, id) == 0)
+			break;
+
+	for (s = e ? e->slot : NULL; s; s = s->next) {
+		struct mddev_dev *d;
+
+		if (!s->devname)
+			continue;
+		d = xcalloc(1, sizeof(*d));
+		xasprintf(&d->devname, "/dev/%s", s->devname);
+		*pos = d;
+		pos = &d->next;
+	}
+	free_enclosure(enclosure);
+
+	return dlist;
+
+}
+
 static int detail_platform_enclosure(int verbose, int enumerate, char *enclosure_name)
 {
 	struct enclosure *enclosure = parse_enclosures();
diff --git a/mdadm.c b/mdadm.c
index 7eeeb4f..066fb35 100644
--- a/mdadm.c
+++ b/mdadm.c
@@ -298,16 +298,23 @@ int main(int argc, char *argv[])
 			/* If first option is a device, don't force the mode yet */
 			if (opt == 1) {
 				if (devs_found == 0) {
-					dv = xmalloc(sizeof(*dv));
-					dv->devname = optarg;
-					dv->disposition = devmode;
-					dv->writemostly = writemostly;
-					dv->used = 0;
-					dv->next = NULL;
-					*devlistend = dv;
-					devlistend = &dv->next;
-
-					devs_found++;
+					dv = conf_get_enclosure_devs(optarg);
+					if (!dv) {
+						dv = xmalloc(sizeof(*dv));
+						dv->devname = optarg;
+						dv->next = NULL;
+					}
+
+					for (; dv; dv = dv->next) {
+						dv->disposition = devmode;
+						dv->writemostly = writemostly;
+						dv->used = 0;
+						*devlistend = dv;
+						devlistend = &dv->next;
+
+						devs_found++;
+					}
+
 					continue;
 				}
 				/* No mode yet, and this is the second device ... */
@@ -354,16 +361,24 @@ int main(int argc, char *argv[])
 				pr_err("Must give -a/--add for devices to add: %s\n", optarg);
 				exit(2);
 			}
-			dv = xmalloc(sizeof(*dv));
-			dv->devname = optarg;
-			dv->disposition = devmode;
-			dv->writemostly = writemostly;
-			dv->used = 0;
-			dv->next = NULL;
-			*devlistend = dv;
-			devlistend = &dv->next;
 
-			devs_found++;
+			dv = conf_get_enclosure_devs(optarg);
+			if (!dv) {
+				dv = xmalloc(sizeof(*dv));
+				dv->devname = optarg;
+				dv->next = NULL;
+			}
+
+			for (; dv; dv = dv->next) {
+				dv->disposition = devmode;
+				dv->writemostly = writemostly;
+				dv->used = 0;
+				*devlistend = dv;
+				devlistend = &dv->next;
+
+				devs_found++;
+			}
+
 			continue;
 		}
 
diff --git a/mdadm.h b/mdadm.h
index 7ac0b95..f318f1d 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -1351,6 +1351,8 @@ extern void enable_fds(int devices);
 extern int parse_auto(char *str, char *msg, int config);
 extern struct mddev_ident *conf_get_ident(char *dev);
 extern struct mddev_dev *conf_get_devs(void);
+extern struct mddev_dev *conf_get_enclosure_devs(char *id);
+extern void conf_put_enclosure_devs(struct mddev_dev *devs);
 extern int conf_test_dev(char *devname);
 extern int conf_test_metadata(const char *version, struct dev_policy *pol, int is_homehost);
 extern struct createinfo *conf_get_create_info(void);
-- 
2.4.6

[PATCH 11/17] policy: enable enclosure names for domain definitions

[Song Liu]

From: Dan Williams <djbw@fb.com>

The argument to "slot=" in a POLICY line can either be a path glob
relative to /dev/disk/by-slot, or an enclosure name.

In other words, now an enclosure name can be used as a domain boundary:
mdadm --detail-platform --enclosure --brief >> /etc/mdadm.conf
cat /etc/mdadm.conf
ENCLOSURE enclosure0 id=0x5f80f41f1d6a60ff
ENCLOSURE enclosure1 id=0x5f80f41f1d6e80ff
echo "POLICY domain=domain0 slot=enclosure0" >> /etc/mdadm.conf

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 config.c | 26 ++++++++++++++++++++++++++
 mdadm.h  |  1 +
 policy.c | 18 +++++++++++++++---
 3 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/config.c b/config.c
index 6ea5dd1..d06928b 100644
--- a/config.c
+++ b/config.c
@@ -1045,6 +1045,32 @@ void conf_put_enclosure_devs(struct mddev_dev *devs)
 	}
 }
 
+/* this succeeds whether the enclosure is currently attached or not, just
+ * translates enclosure names to ids from the configuration file
+ */
+int conf_get_enclosure_glob(const char *slot_val, char *buf, int len)
+{
+	struct conf_enclosure *e;
+
+	load_conffile();
+	for (e = edevlist; e; e = e->next) {
+		/* slot=<enclosure_name> */
+		if (strcmp(e->name, slot_val) == 0 &&
+		    snprintf(buf, len, "enclosure-%s-slot*", e->id) < len)
+			return 1;
+
+		/* slot=enclosure-<enclosure_name>[-<slot-glob>] */
+		if (snprintf(buf, len, "enclosure-%s", e->name) < len &&
+		    strncmp(slot_val, buf, strlen(buf)) == 0 &&
+		    snprintf(buf, len, "enclosure-%s%s", e->id,
+			     &slot_val[strlen(buf)]) < len)
+			return 1;
+	}
+
+	/* slot= is already a glob */
+	return 0;
+}
+
 int conf_test_dev(char *devname)
 {
 	struct conf_dev *cd;
diff --git a/mdadm.h b/mdadm.h
index f318f1d..4f6c733 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -1353,6 +1353,7 @@ extern struct mddev_ident *conf_get_ident(char *dev);
 extern struct mddev_dev *conf_get_devs(void);
 extern struct mddev_dev *conf_get_enclosure_devs(char *id);
 extern void conf_put_enclosure_devs(struct mddev_dev *devs);
+extern int conf_get_enclosure_glob(const char *slot_val, char *buf, int len);
 extern int conf_test_dev(char *devname);
 extern int conf_test_metadata(const char *version, struct dev_policy *pol, int is_homehost);
 extern struct createinfo *conf_get_create_info(void);
diff --git a/policy.c b/policy.c
index 6afafad..0558351 100644
--- a/policy.c
+++ b/policy.c
@@ -274,9 +274,17 @@ static int pol_match(struct rule *rule, char *path, char *type)
 
 	while (rule) {
 		if (rule_is_path(rule)) {
+			char *value = rule->value;
+			char enc_desc[256];
+
+			if (rule->name == rule_slot &&
+			    conf_get_enclosure_glob(value, enc_desc,
+						    sizeof(enc_desc)))
+			    value = enc_desc;
+
 			if (pathok == 0)
 				pathok = -1;
-			if (path && fnmatch(rule->value, path, 0) == 0)
+			if (path && fnmatch(value, path, 0) == 0)
 				pathok = 1;
 		}
 		if (rule->name == rule_type) {
@@ -848,6 +856,7 @@ static struct rule *find_path_rule(struct rule *rule)
 int write_rule(struct rule *rule, int fd, int force_part)
 {
 	char line[1024];
+	char enc_desc[256];
 	struct rule *path_rule = find_path_rule(rule);
 	char *typ = find_rule(rule, rule_type);
 	char *slot_path, *pth;
@@ -856,9 +865,12 @@ int write_rule(struct rule *rule, int fd, int force_part)
 		return -1;
 
 	pth = path_rule->value;
-	if (path_rule->name == rule_slot)
+	if (path_rule->name == rule_slot) {
+		if (conf_get_enclosure_glob(path_rule->value, enc_desc,
+					    sizeof(enc_desc)))
+			pth = enc_desc;
 		slot_path = "ID_SLOT";
-	else
+	} else
 		slot_path = "ID_PATH";
 
 	if (force_part)
-- 
2.4.6

[PATCH 12/17] invoke hot-add policy on "change" events

[Song Liu]

From: Dan Williams <djbw@fb.com>

For enclosure managed disks the links identifying the enclosure topology
may not be available until after a change event.  Modify the dynamic
udev rules to run on change events emitted on enclosure attach.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 policy.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/policy.c b/policy.c
index 0558351..177544a 100644
--- a/policy.c
+++ b/policy.c
@@ -841,12 +841,12 @@ static struct rule *find_path_rule(struct rule *rule)
 }
 
 #define UDEV_RULE_FORMAT \
-"ACTION==\"add\", SUBSYSTEM==\"block\", " \
+"ACTION==\"add|change\", SUBSYSTEM==\"block\", " \
 "ENV{DEVTYPE}==\"%s\", ENV{%s}==\"%s\", " \
 "RUN+=\"" BINDIR "/mdadm --incremental $env{DEVNAME}\"\n"
 
 #define UDEV_RULE_FORMAT_NOTYPE \
-"ACTION==\"add\", SUBSYSTEM==\"block\", " \
+"ACTION==\"add|change\", SUBSYSTEM==\"block\", " \
 "ENV{%s}==\"%s\", " \
 "RUN+=\"" BINDIR "/mdadm --incremental $env{DEVNAME}\"\n"
 
-- 
2.4.6

[PATCH 13/17] Toggle enclosure leds in response to raid events

[Song Liu]

From: Dan Williams <djbw@fb.com>

The configuration file lists the domains that the system is monitoring
for events.  When an array arrives all the leds for the active member
drives are set to 'normal' status.  Any device that fits a domain
description but is not active (in-sync) is marked faulty.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 Makefile       |   5 +-
 Monitor.c      |   3 +
 contrib/md-ses | 310 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 317 insertions(+), 1 deletion(-)
 create mode 100755 contrib/md-ses

diff --git a/Makefile b/Makefile
index 3f9c7d2..009073c 100644
--- a/Makefile
+++ b/Makefile
@@ -270,7 +270,7 @@ $(MON_OBJS) : $(INCL) mdmon.h
 sha1.o : sha1.c sha1.h md5.h
 	$(CC) $(CFLAGS) -DHAVE_STDINT_H -o sha1.o -c sha1.c
 
-install : mdadm mdmon install-man install-udev
+install : mdadm mdmon install-man install-udev install-contrib
 	$(INSTALL) -D $(STRIP) -m 755 mdadm $(DESTDIR)$(BINDIR)/mdadm
 	$(INSTALL) -D $(STRIP) -m 755 mdmon $(DESTDIR)$(BINDIR)/mdmon
 
@@ -292,6 +292,9 @@ install-man: mdadm.8 md.4 mdadm.conf.5 mdmon.8
 	$(INSTALL) -D -m 644 md.4 $(DESTDIR)$(MAN4DIR)/md.4
 	$(INSTALL) -D -m 644 mdadm.conf.5 $(DESTDIR)$(MAN5DIR)/mdadm.conf.5
 
+install-contrib: contrib/md-ses
+	$(INSTALL) -D -m 755 contrib/md-ses $(DESTDIR)/usr/share/mdadm/md-ses
+
 udev-enclosure-slot.rules: udev-enclosure-slot.rules.in udev-workaround-sysfs-deprecated.in Makefile
 	@echo "$@: generate"
 	@/bin/echo -e "# do not edit this file, it will be overwritten on update\n" > $@
diff --git a/Monitor.c b/Monitor.c
index f19c2e5..3f3d02e 100644
--- a/Monitor.c
+++ b/Monitor.c
@@ -666,6 +666,9 @@ static int check_array(struct state *st, struct mdstat_ent *mdstat,
 				alert("FailSpare", dev, dv, ainfo);
 			else if ((newstate&change)&(1<<MD_DISK_SYNC))
 				alert("SpareActive", dev, dv, ainfo);
+			else if ((st->devstate[i]&change)&(1<<MD_DISK_REMOVED))
+				alert("SpareSync", dev, dv, ainfo);
+
 		}
 		st->devstate[i] = newstate;
 		st->devid[i] = makedev(disc.major, disc.minor);
diff --git a/contrib/md-ses b/contrib/md-ses
new file mode 100755
index 0000000..9255e96
--- /dev/null
+++ b/contrib/md-ses
@@ -0,0 +1,310 @@
+#!/usr/bin/env python
+import os
+import sys
+import stat
+from glob import glob
+from optparse import OptionParser
+
+"""
+md-ses: update ses leds in response to mdadm --monitor events
+
+1/ at startup: set the "fault" indicator for every block device in a
+   ses slot that is also in a defined domain
+2/ at failure: set the "fault" indicator for the specified device
+3/ at spare active: clear the "fault" indicator for the specified
+   device
+
+partitioned devices are ignored
+"""
+
+LOGFILE = None
+
+def log(msg):
+    if LOGFILE != None:
+        LOGFILE.write('%s: %s' % (sys.argv[0], msg))
+
+def devt_to_devpath(devt):
+    # where devt is a string of "<major:minor>"
+    path = "/sys/dev/block/%s/device" % devt
+    if not os.path.exists(path):
+         return None
+    path = os.path.realpath(path)
+    return path
+
+def devname_to_devpath(name):
+    if not name:
+        return None
+
+    st = os.stat(name)
+    if not stat.S_ISBLK(st.st_mode):
+        return None
+    major = os.major(st.st_rdev)
+    minor = os.minor(st.st_rdev)
+    # note this path will be missing for partitions
+    return devt_to_devpath('%s:%s' % (major, minor))
+
+def devpath_to_sespath(path):
+    if not path:
+        return None
+    slot = glob('%s/enclosure_device:*' % path)
+    if len(slot) != 1:
+        return None
+    return os.path.realpath(slot[0])
+
+def devname_to_sespath(name):
+    dp = devname_to_devpath(name)
+    sp = devpath_to_sespath(dp)
+    return sp
+
+def parse_config(config_name, header, field_name):
+    """
+    Take mdadm config lines like:
+    POLICY domain=domain0 slot=X
+    POLICY domain=domain0 slot=Y
+    ENCLOSURE enclosure0 id=X
+
+    ...and create a dictionary of name and field_name values
+    For example parse_config(cfg, 'POLICY', 'slot=') for the above example
+    yields:
+    { 'POLICY': [ X, Y ] }
+
+    ...for parse_config(cfg, 'ENCLOSURE', 'id=') yields:
+    { 'enclosure0': [ X ] }
+    """
+
+    def parse_field(field_name, line):
+        (key, val) = (None, None)
+        for f in line.split():
+            if f.startswith(field_name):
+                (tempkey, val) = f.split('=')
+            elif not '=' in f:
+                key = f
+        return (key, val)
+
+
+    def finish_line(line, field_name, items):
+        if line != None and field_name in line:
+            (key, val) = parse_field(field_name, line)
+            if key == None or val == None:
+                return
+            if key in items:
+                items[key].append(val)
+            else:
+                items[key] = [val]
+
+    items = {}
+    with open(config_name, 'r') as config_file:
+        line = None
+        for l in config_file:
+            l = l.rstrip()
+            if l.startswith(header):
+                finish_line(line, field_name, items)
+                line = l
+                continue
+            if not line:
+                continue
+            # continue policy lines
+            if len(l) and l[0].isspace():
+                line = line + l
+            else:
+                finish_line(line, field_name, items)
+                line = None
+        finish_line(line, field_name, items)
+    return items
+
+"""
+assumes that POLICY lines only include named enclosures or ids, not paths
+"""
+def enclosure_ids_from_domains(config_name):
+    slots = parse_config(config_name, 'POLICY', 'slot=')
+    enclosures = parse_config(config_name, 'ENCLOSURE', 'id=')
+
+    all_slots = []
+    for s in slots.values():
+        all_slots = all_slots + s
+
+    ids = []
+    for s in all_slots:
+        if s in enclosures:
+            s = enclosures[s][0]
+        ids.append(s)
+    return ids
+
+def parse_domains(config_name):
+    """
+    Return a set of all disks (by "major:minor") that the configuration file cares about
+    to be compared against the rdevs read from the active md devices
+    """
+    device_list = []
+    for enc_id in enclosure_ids_from_domains(config_name):
+        by_slot = 'enclosure-%s-slot*' % enc_id
+        for f in glob('/dev/disk/by-slot/%s' % by_slot):
+            st = os.stat(f)
+            dev = "%s:%s" % (os.major(st.st_rdev), os.minor(st.st_rdev))
+            device_list.append(dev)
+
+    return set(device_list)
+
+def parse_rdevs(check_state):
+    device_list = []
+    for md in glob('/sys/block/md*'):
+        for rd in glob('%s/md/rd*' % md):
+            with open('%s/block/dev' % rd, 'r') as _dev:
+                dev = _dev.readline().rstrip()
+            with open('%s/state' % rd , 'r') as _state:
+                state = _state.readline().rstrip()
+            if check_state == None or state == check_state:
+                device_list.append(dev)
+    return set(device_list)
+
+def update_ses(device, fault, sp=None):
+    if not sp:
+        sp = devname_to_sespath(device)
+    if not sp:
+        log('unable to determine slot for %s\n' % device)
+        return 1
+
+    log('write \'%d\' to %s\n' % (fault, sp))
+    with open('%s/fault' % sp, 'w') as f:
+        f.write('%d\n' % fault)
+
+    return 0
+
+def debug_event(event):
+    dp = devname_to_devpath(event.device)
+    sp = devpath_to_sespath(dp)
+    log('name: %s array: %s, dev: %s ses: %s\n' % (
+             event.name, event.array, event.device,
+             sp
+           )
+    )
+    device_set = parse_domains(event.config)
+    active_set = parse_rdevs('in_sync')
+    print device_set - active_set
+
+    return 0
+
+def parse_enclosures(config_name):
+    path = '/sys/class/enclosure'
+    slots = []
+    ids = enclosure_ids_from_domains(config_name)
+    for e in glob('%s/*' % path):
+        enc_id = None
+        with open('%s/id' % e) as _id:
+            enc_id = _id.readline().rstrip()
+        if not enc_id in ids:
+            continue
+        for slot in glob('%s/*' % e):
+            if os.path.exists('%s/slot' % slot):
+                slots.append(os.path.realpath(slot))
+    return set(slots)
+
+def ignore_event(event):
+    return 0
+
+def handle_active(event):
+    return update_ses(event.device, 0)
+
+def handle_fail(event):
+    return update_ses(event.device, 1)
+
+def handle_new(event):
+    device_set = parse_domains(event.config)
+    active_devs = parse_rdevs('in_sync')
+
+    active_slots = []
+    for devt in device_set & active_devs:
+        dp = devt_to_devpath(devt)
+        sp = devpath_to_sespath(dp)
+        active_slots.append(sp)
+        update_ses(None, 0, sp)
+
+    for sp in parse_enclosures(event.config) - set(active_slots):
+        update_ses(None, 1, sp)
+
+    return 0
+
+def handle_missing_hdd(event):
+    for sp in parse_enclosures(event.config):
+        if not os.path.exists(sp + '/device'):
+            update_ses(None, 1, sp)
+    return 0
+
+class MDEvent:
+    event_fn = {
+                 'DeviceDisappeared':   ignore_event,
+                 'HDDDisappeared':      handle_missing_hdd,
+                 'RebuildStarted':      ignore_event,
+                 'RebuildNN':           ignore_event,
+                 'RebuildFinished':     ignore_event,
+                 'Fail':                handle_fail,
+                 'FailSpare':           handle_fail,
+                 'SpareActive':         handle_active,
+                 'SpareSync':           handle_active,
+                 'NewArray':            handle_new,
+                 'DegradedArray':       ignore_event,
+                 'MoveSpare':           ignore_event,
+                 'SparesMissing':       ignore_event,
+                 'TestMessage':         ignore_event,
+               }
+    def __init__(self, event, array, device, conf):
+        self.name = event
+        self.array = array
+        self.device = device
+        self.config = conf
+    def handle(self):
+        try:
+            handler = MDEvent.event_fn[self.name]
+            if handler == ignore_event:
+                log('ignoring %s for array %s (dev: %s)\n' % (self.name, self.array, self.device))
+            else:
+                log('handling %s for array %s (dev: %s)\n' % (self.name, self.array, self.device))
+            return handler(self)
+        except KeyError:
+            log('unsupported event \'%s\'\n' % self.name)
+            return 1
+
+CONF1 = '/etc/mdadm.conf'
+CONF2 = '/etc/mdadm/mdadm.conf'
+
+def main():
+    parser = OptionParser()
+    parser.add_option("-c", "--config", type="string", dest="config",
+                      help="override configuration file location", default='<default>')
+    parser.add_option("-l", "--log", type="string", dest="log",
+                      help="log actions to <file>", default=None)
+    (options, args) = parser.parse_args()
+
+    if len(args) <= 1:
+        sys.stderr.write('usage: %s <md event name> <array> [device]\n' % sys.argv[0])
+        return 1
+
+    if len(args) > 1:
+        event = args[0]
+        array = args[1]
+        device = None
+
+    if len(args) > 2:
+        device = args[2]
+
+    if options.config == '<default>':
+        if os.path.exists(CONF1):
+            options.config = CONF1
+        elif os.path.exists(CONF2):
+            options.config = CONF2
+
+    global LOGFILE
+    if LOGFILE != None:
+        LOGFILE = open(LOGFILE, 'a')
+    elif options.log == 'stderr':
+        LOGFILE = sys.stderr
+    elif options.log == 'stdout':
+        LOGFILE = sys.stdout
+    elif options.log != None:
+        LOGFILE = open(options.log, 'a')
+
+    md = MDEvent(event, array, device, options.config)
+    return md.handle()
+
+if __name__ == "__main__":
+    sys.exit(main())
-- 
2.4.6

[PATCH 14/17] scsi mode sense/select support

[Song Liu]

From: Dan Williams <djbw@fb.com>

Add scsi_mode_{sense|select} support in preparation for allowing drive
settings to be specified in the configuration file.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 sg_io.c | 170 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 sg_io.h |  27 +++++++++++
 2 files changed, 195 insertions(+), 2 deletions(-)
 create mode 100644 sg_io.h

diff --git a/sg_io.c b/sg_io.c
index 50ad180..813cbdd 100644
--- a/sg_io.c
+++ b/sg_io.c
@@ -16,15 +16,20 @@
  * this program; if not, write to the Free Software Foundation, Inc.,
  * 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
  */
+#include "mdadm.h"
+#include <stdio.h>
 #include <string.h>
 #include <scsi/scsi.h>
 #include <scsi/sg.h>
 #include <sys/ioctl.h>
+#include <stdlib.h>
+
+#include "sg_io.h"
 
 int scsi_get_serial(int fd, void *buf, size_t buf_len)
 {
-	unsigned char inq_cmd[] = {INQUIRY, 1, 0x80, 0, buf_len, 0};
-	unsigned char sense[32];
+	__u8 inq_cmd[] = {INQUIRY, 1, 0x80, 0, buf_len, 0};
+	__u8 sense[32];
 	struct sg_io_hdr io_hdr;
 
 	memset(&io_hdr, 0, sizeof(io_hdr));
@@ -40,3 +45,164 @@ int scsi_get_serial(int fd, void *buf, size_t buf_len)
 
 	return ioctl(fd, SG_IO, &io_hdr);
 }
+
+static int process_response(const char *cmd, struct sg_io_hdr *io)
+{
+	if ((io->info & SG_INFO_OK_MASK) != SG_INFO_OK) {
+		if (io->sb_len_wr > 0) {
+			int i;
+
+			dprintf("resp: %x sense: %x asc: %x asq: %x\n",
+				io->sbp[0], io->sbp[2] & 0xf, io->sbp[12],
+				io->sbp[13]);
+			dprintf("%s sense data:\n", cmd);
+			for (i = 0; i < io->sb_len_wr; ++i) {
+				if ((i > 0) && (0 == (i % 10)))
+					printf("\n  ");
+				dprintf("0x%02x ", io->sbp[i]);
+			}
+			dprintf("\n");
+		}
+		if (io->masked_status)
+			dprintf("%s SCSI status=0x%x\n", cmd, io->status);
+		if (io->host_status)
+			dprintf("%s host_status=0x%x\n", cmd, io->host_status);
+		if (io->driver_status)
+			dprintf("%s driver_status=0x%x\n", cmd, io->driver_status);
+		return -1;
+	}
+	return 0;
+}
+
+/**
+ * scsi_mode_sense - fetch mode page(s) and a description
+ *
+ * cdb len based on buffer size
+ */
+static int __scsi_mode_sense(int fd, __u8 pg, __u8 spg,
+			     struct scsi_mode_sense_data *data,
+			     __u8 *buf, size_t buf_len, int force6)
+{
+	struct sg_io_hdr io_hdr;
+	int rc, cdb_len;
+	__u8 sense[32];
+	__u8 cmd[10];
+
+	if (buf_len < 8)
+		return -1;
+
+	memset(data, 0, sizeof(data));
+	memset(cmd, 0, sizeof(cmd));
+	cmd[2] = pg;
+	cmd[3] = spg;
+
+	if (buf_len > 255 || !force6) {
+		cdb_len = 10;
+		cmd[0] = MODE_SENSE_10;
+		cmd[7] = buf_len >> 8;
+		cmd[8] = buf_len;
+	} else {
+		cdb_len = 6;
+		cmd[0] = MODE_SENSE;
+		cmd[4] = buf_len;
+	}
+	memset(&io_hdr, 0, sizeof(io_hdr));
+	memset(sense, 0, sizeof(sense));
+	io_hdr.interface_id = 'S';
+	io_hdr.cmdp = cmd;
+	io_hdr.cmd_len = cdb_len;
+	io_hdr.dxferp = buf;
+	io_hdr.dxfer_len = buf_len;
+	io_hdr.dxfer_direction = SG_DXFER_FROM_DEV;
+	io_hdr.sbp = sense;
+	io_hdr.mx_sb_len = sizeof(sense);
+	io_hdr.timeout = 30000;
+
+	rc = ioctl(fd, SG_IO, &io_hdr);
+	if (rc)
+		return rc;
+
+	if (cdb_len == 10) {
+		data->data_len = buf[0]*256 + buf[1] + 2;
+		data->media_type = buf[2];
+		data->device_specific = buf[3];
+		data->long_lba = buf[4] & 1;
+		data->block_len = buf[6]*256 + buf[7];
+	} else {
+		data->data_len = buf[0] + 1;
+		data->media_type = buf[1];
+		data->device_specific = buf[2];
+		data->block_len = buf[3];
+	}
+	data->hdr_len = cdb_len - 2;
+
+	if ((size_t) (to_mode_page(buf, data) - buf) > buf_len)
+		return -1;
+
+	return process_response("MODE SENSE", &io_hdr);
+}
+
+int scsi_mode_sense(int fd, __u8 pg, __u8 spg,
+		    struct scsi_mode_sense_data *data,
+		    __u8 *buf, size_t buf_len)
+{
+	return __scsi_mode_sense(fd, pg, spg, data, buf, buf_len, 0);
+}
+
+int scsi_mode_select(int fd, enum page_format pf, int save,
+		     const struct scsi_mode_sense_data *data,
+		     __u8 *param_list)
+{
+	struct sg_io_hdr io_hdr;
+	int rc, cdb_len;
+	__u8 sense[32];
+	__u8 cmd[10];
+	__u8 *page;
+
+	memset(cmd, 0, sizeof(cmd));
+	page = to_mode_page(param_list, data);
+	cmd[1] = (!!pf << 4) | ((page[0] >> 7) & !!save);
+
+	/* ps reserved */
+	page[0] &= 0x7f;
+
+	if (data->hdr_len > 4) {
+		cdb_len = 10;
+		cmd[0] = MODE_SELECT_10;
+		cmd[7] = data->data_len >> 8;
+		cmd[8] = data->data_len;
+
+		param_list[0] = 0; /* data len reserved */
+		param_list[1] = 0; /*  "    "     "     */
+		param_list[2] = data->media_type;
+		param_list[3] = 0; /* whole disk parameter */
+		param_list[6] = data->block_len >> 8;
+		param_list[7] = data->block_len;
+	} else {
+		cdb_len = 6;
+		cmd[0] = MODE_SELECT;
+		cmd[4] = data->data_len;
+
+		param_list[0] = 0; /* data len reserved */
+		param_list[1] = data->media_type;
+		param_list[2] = 0; /* whole disk parameter */
+		param_list[3] = data->block_len;
+	}
+	memset(&io_hdr, 0, sizeof(io_hdr));
+	memset(sense, 0, sizeof(sense));
+	io_hdr.interface_id = 'S';
+	io_hdr.cmdp = cmd;
+	io_hdr.cmd_len = cdb_len;
+	io_hdr.dxferp = param_list;
+	io_hdr.dxfer_len = data->data_len;
+	io_hdr.dxfer_direction = SG_DXFER_TO_DEV;
+	io_hdr.sbp = sense;
+	io_hdr.mx_sb_len = sizeof(sense);
+	io_hdr.timeout = 30000;
+
+	rc = ioctl(fd, SG_IO, &io_hdr);
+	if (rc)
+		return rc;
+
+	return process_response("MODE SELECT", &io_hdr);
+}
diff --git a/sg_io.h b/sg_io.h
new file mode 100644
index 0000000..f0988a5
--- /dev/null
+++ b/sg_io.h
@@ -0,0 +1,27 @@
+#include <asm/types.h>
+
+struct scsi_mode_sense_data {
+	int hdr_len;
+	int block_len;
+	int media_type;
+	int data_len;
+	int device_specific;
+	char long_lba;
+};
+
+enum page_format {
+	PF_VENDOR = 0,
+	PF_STANDARD = 1,
+};
+
+static inline __u8 *to_mode_page(__u8 *buf, const struct scsi_mode_sense_data *data)
+{
+	return buf + data->hdr_len + data->block_len;
+}
+
+int scsi_mode_sense(int fd, __u8 pg, __u8 spg,
+		    struct scsi_mode_sense_data *data,
+		    __u8 *buf, size_t buf_len);
+int scsi_mode_select(int fd, enum page_format pf, int save,
+		     const struct scsi_mode_sense_data *data,
+		     __u8 *page);
-- 
2.4.6

[PATCH 15/17] SCSIMODE: configuration file support for mode page settings.

[Song Liu]

From: Dan Williams <djbw@fb.com>

Before a a drive is incorporated into an array we want to be sure that settings
like write-cache-disable and error-recovery-response are set consistently for
each drive in the array.  Add a "SCSIMODE" configuration parameter that:

1/ identifies the disks to apply the policy to by domain name.  When
   domain= is specified multiple times the scsi mode page settings are
   applied to the union of those domains.

2/ Modifies the mode page bytes identified by a tuple of (page, subpage,
   byte).  For now only the '=' operator is supported to assign the
   entire byte.

      _____ page
     /   __ subpage
     | /  _ byte
     | | /
     v v v
     x.y.z=val

Example disable the write cache on all drives in domain0 and domain1:
SCSIMODE domain=domain0 domain=domain1
     # disable write cache
     8.0.2=0
     # enable automatic read/write re-assignment
     1.0.2=0xc0

For multi-byte values specifying something like:
     8.0.0=0
     8.0.1=0
     8.0.2=0
     8.0.3=0
...will result in one write to mode-page-8.  Settings to the same page
are aggregated and the values are set in specified order within a page.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 Incremental.c |   9 +++
 Makefile      |   2 +-
 config.c      | 191 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 mdadm.h       |   1 +
 4 files changed, 201 insertions(+), 2 deletions(-)

diff --git a/Incremental.c b/Incremental.c
index 781d27d..88a945a 100644
--- a/Incremental.c
+++ b/Incremental.c
@@ -194,6 +194,15 @@ int Incremental(struct mddev_dev *devlist, struct context *c,
 	policy = disk_policy(&dinfo);
 	have_target = policy_check_path(&dinfo, &target_array);
 
+	/* before taking any action on the disk check if we need to modify settings */
+	if (conf_apply_scsimode(&dinfo, policy) != 0) {
+		if (c->verbose >= 0)
+			pr_err("%s failed to apply scsi mode settings\n",
+			       devname);
+		rv = 2;
+		goto out;
+	}
+
 	if (st == NULL && (st = guess_super_type(dfd, guess_array)) == NULL) {
 		if (c->verbose >= 0)
 			pr_err("no recognisable superblock on %s.\n",
diff --git a/Makefile b/Makefile
index 009073c..3a5f28d 100644
--- a/Makefile
+++ b/Makefile
@@ -143,7 +143,7 @@ SRCS =  $(patsubst %.o,%.c,$(OBJS))
 INCL = mdadm.h part.h bitmap.h
 
 MON_OBJS = mdmon.o monitor.o managemon.o util.o maps.o mdstat.o sysfs.o \
-	policy.o lib.o \
+	policy.o lib.o config.o mapfile.o \
 	Kill.o sg_io.o dlink.o ReadMe.o super-intel.o \
 	super-mbr.o super-gpt.o \
 	super-ddf.o sha1.o crc32.o msg.o bitmap.o xmalloc.o \
diff --git a/config.c b/config.c
index d06928b..f1ab62b 100644
--- a/config.c
+++ b/config.c
@@ -24,6 +24,7 @@
 
 #include	"mdadm.h"
 #include	"dlink.h"
+#include	"sg_io.h"
 #include	<dirent.h>
 #include	<glob.h>
 #include	<fnmatch.h>
@@ -77,7 +78,8 @@ char DefaultAltConfFile[] = CONFFILE2;
 char DefaultAltConfDir[] = CONFFILE2 ".d";
 
 enum linetype { Devices, Array, Mailaddr, Mailfrom, Program, CreateDev,
-		Homehost, HomeCluster, AutoMode, Policy, PartPolicy, Enclosures, LTEnd };
+		Homehost, HomeCluster, AutoMode, Policy, PartPolicy, Enclosures,
+		ScsiMode, LTEnd };
 char *keywords[] = {
 	[Devices]  = "devices",
 	[Array]    = "array",
@@ -91,6 +93,7 @@ char *keywords[] = {
 	[Policy]   = "policy",
 	[PartPolicy]="part-policy",
 	[Enclosures] = "enclosure",
+	[ScsiMode] = "scsimode",
 	[LTEnd]    = NULL
 };
 
@@ -123,6 +126,22 @@ struct conf_enclosure {
 	char *id;
 } *edevlist = NULL;
 
+struct conf_scsimode {
+	struct conf_scsimode *next;
+	int strict;
+	struct mode_domain {
+		struct mode_domain *next;
+		char *name;
+	} *domains;
+	struct mode_setting {
+		struct mode_setting *next;
+		int page;
+		int subpage;
+		int byte;
+		unsigned char val;
+	} *settings;
+} *scsimodelist;
+
 struct mddev_dev *load_partitions(void)
 {
 	FILE *f = fopen("/proc/partitions", "r");
@@ -535,6 +554,99 @@ static void enclosureline(char *line)
 	}
 }
 
+static void scsimodeline(char *line)
+{
+	char *w;
+	struct conf_scsimode *mode = xmalloc(sizeof(*mode));
+	struct mode_setting **s_pos = &mode->settings;
+
+	mode->domains = NULL;
+	mode->settings = NULL;
+	mode->strict = 0;
+	for (w = dl_next(line); w != line; w = dl_next(w)) {
+		if (strncasecmp(w, "domain=", 7) == 0) {
+			struct mode_domain *d = xmalloc(sizeof(*d));
+
+			d->name = xstrdup(w+7);
+			d->next = mode->domains;
+			mode->domains = d;
+		} else if (strncasecmp(w, "strict", 6) == 0) {
+			mode->strict = 1;
+		} else {
+			char *start, *end;
+			char sep[] = "..=";
+			int fields[4];
+			size_t i;
+
+			start = w;
+			for (i = 0; i < sizeof(sep); i++) {
+				fields[i] = strtoul(start, &end, 0);
+				if (end && end > start && *end == sep[i]) {
+					start = end+1;
+				} else
+					break;
+			}
+
+			if (i < sizeof(sep))
+				pr_err("unrecognized format for SCSIMODE line: %s\n", w);
+			else {
+				struct mode_setting *s = xmalloc(sizeof(*s));
+				struct mode_setting *insert;
+
+				s->page = fields[0];
+				s->subpage = fields[1];
+				s->byte = fields[2];
+				s->val = fields[3];
+
+				/* keep pages adjacent, as we will apply
+				 * entire pages at a time
+				 */
+				for (insert = mode->settings; insert; insert = insert->next) {
+					if (insert->next == NULL)
+						break;
+					if (insert->next->page == s->page &&
+					    insert->next->subpage == s->subpage)
+						continue;
+					if (insert->page == s->page &&
+					    insert->subpage == s->subpage)
+						break;
+				}
+
+				if (insert) {
+					s->next = insert->next;
+					insert->next = s;
+				} else {
+					s->next = NULL;
+					while (*s_pos)
+						s_pos = &(*s_pos)->next;
+					*s_pos = s;
+					s_pos = &s->next;
+				}
+			}
+		}
+	}
+
+	if (mode->domains && mode->settings) {
+		mode->next = scsimodelist;
+		scsimodelist = mode;
+	} else {
+		struct mode_domain *d;
+		struct mode_setting *s;
+
+		while (mode->domains) {
+			d = mode->domains->next;
+			free(mode->domains);
+			mode->domains = d;
+		}
+		while (mode->settings) {
+			s = mode->settings->next;
+			free(mode->settings);
+			mode->settings = s;
+		}
+		free(mode);
+	}
+}
+
 static char *alert_email = NULL;
 void mailline(char *line)
 {
@@ -757,6 +869,9 @@ void conf_file(FILE *f)
 		case Enclosures:
 			enclosureline(line);
 			break;
+		case ScsiMode:
+			scsimodeline(line);
+			break;
 		case Mailaddr:
 			mailline(line);
 			break;
@@ -1259,6 +1374,80 @@ struct mddev_ident *conf_match(struct supertype *st,
 	return match;
 }
 
+static int apply_mode(struct mdinfo *info, struct mode_setting *settings)
+{
+	struct mode_setting *s = settings;
+	struct scsi_mode_sense_data data;
+	__u8 mode_buf[512];
+	int errors = 0;
+
+	for (s = settings; s; s = s->next) {
+		__u8 *mode_page;
+		char devt[22];
+		int fd, rc;
+
+		sprintf(devt, "%d:%d",
+			info->disk.major,
+			info->disk.minor);
+		fd = dev_open(devt, O_RDWR);
+
+		rc = scsi_mode_sense(fd, s->page, s->subpage, &data,
+				     mode_buf, sizeof(mode_buf));
+		mode_page = to_mode_page(mode_buf, &data);
+		for (;;) {
+
+			mode_page[s->byte] = s->val;
+			if (s->next &&
+			    s->next->page == s->page &&
+			    s->next->subpage == s->subpage)
+				s = s->next;
+			else
+				break;
+		}
+		if (rc == 0)
+			rc = scsi_mode_select(fd, 1, 1, &data, mode_buf);
+		if (rc)
+			errors++;
+		close(fd);
+	}
+
+	return errors;
+}
+
+int conf_apply_scsimode(struct mdinfo *info, struct dev_policy *pol)
+{
+	struct conf_scsimode *mode;
+	int err = 0;
+
+	load_conffile();
+
+	pol = pol_find(pol, pol_domain);
+	if (!scsimodelist || !pol)
+		return 0;
+
+	for (mode = scsimodelist; mode; mode = mode->next) {
+		struct mode_domain *domain = mode->domains;
+		struct dev_policy *p;
+		int found = 0;
+
+		for (domain = mode->domains; domain; domain = domain->next) {
+			pol_for_each(p, pol, NULL) {
+				if (strcmp(p->value, domain->name) == 0) {
+					err += apply_mode(info, mode->settings);
+					found = 1;
+					break;
+				}
+			}
+			if (found)
+				break;
+		}
+		if (mode->strict && err)
+			return err;
+	}
+
+	return 0;
+}
+
 int conf_verify_devnames(struct mddev_ident *array_list)
 {
 	struct mddev_ident *a1, *a2;
diff --git a/mdadm.h b/mdadm.h
index 4f6c733..e0b1a07 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -1376,6 +1376,7 @@ extern struct mddev_ident *conf_match(struct supertype *st,
 				      struct mdinfo *info,
 				      char *devname,
 				      int verbose, int *rvp);
+extern int conf_apply_scsimode(struct mdinfo *info, struct dev_policy *pol);
 extern int experimental(void);
 
 extern void free_line(char *line);
-- 
2.4.6

[PATCH 16/17] Clear MBR in when Kill superblock

[Song Liu]

When killing the superblock, also clear MBR.

Signed-off-by: Song Liu <songliubraving@fb.com>
---
 Kill.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/Kill.c b/Kill.c
index f2fdb85..06ced78 100644
--- a/Kill.c
+++ b/Kill.c
@@ -28,6 +28,23 @@
 #include	"mdadm.h"
 #include	"md_u.h"
 #include	"md_p.h"
+#include	"part.h"
+
+int KillMBR(int fd)
+{
+	struct MBR *null_mbr;
+
+	null_mbr = xmalloc(sizeof(*null_mbr));
+	memset(null_mbr, 0, sizeof(*null_mbr));
+	lseek(fd, 0, 0);
+	if (write(fd, null_mbr, sizeof(*null_mbr)) != sizeof(*null_mbr)) {
+		free(null_mbr);
+		return 1;
+	}
+	free(null_mbr);
+	fsync(fd);
+	return 0;
+}
 
 int Kill(char *dev, struct supertype *st, int force, int verbose, int noexcl)
 {
@@ -76,6 +93,11 @@ int Kill(char *dev, struct supertype *st, int force, int verbose, int noexcl)
 			rv = 0;
 		}
 	}
+
+	if (KillMBR(fd))
+		if (verbose >=0)
+			pr_err("Cannot clear MBR on %s\n", dev);
+
 	close(fd);
 	return rv;
 }
-- 
2.4.6

[PATCH 17/17] In generated udev rules, skip RAID members

[Song Liu]

Add the following

ENV{ID_FS_TYPE}=="ddf_raid_member|isw_raid_member|linux_raid_member", GOTO="end"
...
LABEL="end"

to generated udev rules.

Signed-off-by: Song Liu <songliubraving@fb.com>
---
 policy.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/policy.c b/policy.c
index 177544a..7341659 100644
--- a/policy.c
+++ b/policy.c
@@ -810,7 +810,10 @@ int policy_check_path(struct mdinfo *disk, struct map_ent *array)
 /* invocation of udev rule file */
 char udev_template_start[] =
 "# do not edit this file, it is automatically generated by mdadm\n"
-"\n";
+"ENV{ID_FS_TYPE}==\"ddf_raid_member|isw_raid_member|linux_raid_member\", "
+"GOTO=\"end\"\n";
+
+char udev_template_end[] = "LABEL=\"end\"\n";
 
 /* find rule named rule_type and return its value */
 char *find_rule(struct rule *rule, char *rule_type)
@@ -962,6 +965,11 @@ int Write_rules(char *rule_name)
 	if (generate_entries(fd) < 0)
 		goto abort;
 
+	if (write(fd, udev_template_end,
+		  sizeof(udev_template_end) - 1)
+	    != (int)sizeof(udev_template_end)-1)
+		goto abort;
+
 	fsync(fd);
 	if (rule_name) {
 		close(fd);
-- 
2.4.6

Re: [PATCH 00/17] mdadm: SCSI enclosure based array

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 758 bytes --]

On Wed, Dec 02 2015, Song Liu wrote:

> These patches by Dan Williams enable creating RAID array based on
> SCSI enclosures.

Is there any background reading available so I can have some idea what
these patches are supposed to do.
I have a vague idea what an "enclosure" is but I didn't know that Linux
knew about them.  Is there a link to the LWN article about them or some
other useful documentation?  "git grep -i enclosur Documentation/"
didn't show anything likely to be useful.

And what is "/dev/disk/by-slot"?? I've never come across it before
and google doesn't help much.  Just these patches and
  http://comments.gmane.org/gmane.linux.scsi/93809

which doesn't seem very conclusive.

So currently I am not able to review these patches.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: [PATCH 00/17] mdadm: SCSI enclosure based array

[Dan Williams]

On Tue, Dec 15, 2015 at 6:01 PM, NeilBrown <neilb@suse.com> wrote:
> On Wed, Dec 02 2015, Song Liu wrote:
>
>> These patches by Dan Williams enable creating RAID array based on
>> SCSI enclosures.
>
> Is there any background reading available so I can have some idea what
> these patches are supposed to do.

"Blink the light when the disk dies", is the terse summary.

> I have a vague idea what an "enclosure" is but I didn't know that Linux
> knew about them.  Is there a link to the LWN article about them or some
> other useful documentation?  "git grep -i enclosur Documentation/"
> didn't show anything likely to be useful.

The interface these patches use is presented by the "ses" driver
(drivers/scsi/ses.c) which leverages the sysfs interface presented by
the "enclosure" driver (drivers/misc/enclosure.c).  I don't recall
their being much in the way of documentation.

For each disk in an enclosure the driver exposes an enclosure object
with the following attributes:
fault
status
active
locate
power_status
type
slot

Where 'fault', 'active', and 'locate' select different led signal
patterns.  The 'power_status' attribute allows power to be turned off
to the slot.  The 'slot' attribute is meant to match the slot
identifier that is on the board's silk screen or some physical label a
technician can observe.

> And what is "/dev/disk/by-slot"?? I've never come across it before
> and google doesn't help much.  Just these patches and
>   http://comments.gmane.org/gmane.linux.scsi/93809

/dev/disk/by-slot are new proposed symlinks for udev.  The new udev
rules for this are in patch1.

When the scsi core is probing devices and finds an enclosure target
device it associates enclosure slot objects with the scsi_target.
With that association we can create a symlink for the resulting disk
based on its physical slot identifier.

At the time mdadm was chosen to house these rules for convenience, but
I think they should be submitted to upstream udev.

Re: [PATCH 00/17] mdadm: SCSI enclosure based array

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2972 bytes --]

On Thu, Dec 17 2015, Dan Williams wrote:

> On Tue, Dec 15, 2015 at 6:01 PM, NeilBrown <neilb@suse.com> wrote:
>> On Wed, Dec 02 2015, Song Liu wrote:
>>
>>> These patches by Dan Williams enable creating RAID array based on
>>> SCSI enclosures.
>>
>> Is there any background reading available so I can have some idea what
>> these patches are supposed to do.
>
> "Blink the light when the disk dies", is the terse summary.
>
>> I have a vague idea what an "enclosure" is but I didn't know that Linux
>> knew about them.  Is there a link to the LWN article about them or some
>> other useful documentation?  "git grep -i enclosur Documentation/"
>> didn't show anything likely to be useful.
>
> The interface these patches use is presented by the "ses" driver
> (drivers/scsi/ses.c) which leverages the sysfs interface presented by
> the "enclosure" driver (drivers/misc/enclosure.c).  I don't recall
> their being much in the way of documentation.
>
> For each disk in an enclosure the driver exposes an enclosure object
> with the following attributes:
> fault
> status
> active
> locate
> power_status
> type
> slot
>
> Where 'fault', 'active', and 'locate' select different led signal
> patterns.  The 'power_status' attribute allows power to be turned off
> to the slot.  The 'slot' attribute is meant to match the slot
> identifier that is on the board's silk screen or some physical label a
> technician can observe.

 cat > Documentation/scsi/ses.txt ???

Thanks.  That is useful background.

>
>> And what is "/dev/disk/by-slot"?? I've never come across it before
>> and google doesn't help much.  Just these patches and
>>   http://comments.gmane.org/gmane.linux.scsi/93809
>
> /dev/disk/by-slot are new proposed symlinks for udev.  The new udev
> rules for this are in patch1.

If this is proposed, then there must be a proposal - right?
Written?? (I doubt it).

So "by-path" is about the data-path that leads to the device, but
"by-slot" is about the physical location of the device (which may
correlate with the path, but is conceptually different).  I guess I can
accept that.
I don't suppose "by-location" was considered?
Can a "disk" have other locations that "slots"?
I guess 'sd' cards live in slots.
But USB sticks plug into sockets (or are sockets just slots?)
CDROMs live in trays.

Is there some convention for naming slots?
  enclosure-rank-row-column ??
Is there some convention for naming enclosures?  Bob, Jane, Ahmed?
>
> When the scsi core is probing devices and finds an enclosure target
> device it associates enclosure slot objects with the scsi_target.
> With that association we can create a symlink for the resulting disk
> based on its physical slot identifier.
>
> At the time mdadm was chosen to house these rules for convenience, but
> I think they should be submitted to upstream udev.

Yes, I would definitely like to see this "by-location" ;-) proposal
accepted by upstream udev before committing to it in mdadm.

Thanks,
NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

RE: [PATCH 00/17] mdadm: SCSI enclosure based array

[Song Liu]

> So "by-path" is about the data-path that leads to the device, but "by-slot" is
> about the physical location of the device (which may correlate with the path,
> but is conceptually different).  I guess I can accept that.
> I don't suppose "by-location" was considered?
> Can a "disk" have other locations that "slots"?
> I guess 'sd' cards live in slots.
> But USB sticks plug into sockets (or are sockets just slots?) CDROMs live in trays.
> 
> Is there some convention for naming slots?
>   enclosure-rank-row-column ??
> Is there some convention for naming enclosures?  Bob, Jane, Ahmed?

The enclosure:slots name comes from SCSI Enclosure Service (SES) definitions, 
and it is just enclosure-slot (no rank, row, column). 

Here, we use the SAS address of the SCSI Enclosure Processor (SEP) to identify 
the enclosure, while the slot is just numbers. For example:

enclosure-0x570e2840203080ff-slot0 -> /dev/sdl
enclosure-0x570e2840203080ff-slot1 -> /dev/sdj
enclosure-0x570e2840203080ff-slot10 -> /dev/sdk

At this time, this concept only applies to drives in SCSI enclosure. Other drives
do not have this symlink. 

Thanks,
Song