[PATCH 00/34] OLCE for external metadata (devel3.2)

[PATCH 00/34] OLCE for external metadata (devel3.2)

[Adam Kwolek]

This patch series implements Online Capacity Expansion (OLCE) feature in mdadm devel3.2. for Raid5 and Raid0.
This is implemented as reshape of whole container (all arrays that belongs to container are reshaped).
Raid0 reshape is implemented using takeover to raid4 transition.

---

Adam Kwolek (31):
      Raid0: detect reshape on array start
      Raid0: Reload disk list on 'next' raid0 array
      Raid0: Run 'next' reshapes without meta update
      imsm: FIX: do not repair raid4 arrays
      Raid0: execute backward takeover
      Detect level change
      Take in mind takeover during disk add
      imsm: Update raid0 metadata for reshape
      imsm: Move reshape update processing to function
      Add spares to raid0 in mdadm
      imsm: Update metadata for second array
      Set array size after adding new disks
      imsm: update array size information in metadata
      FIX: Initialize subarray variable in reshape_array
      imsm: FIX: Division by 0
      imsm: Finalize reshape in metadata
      Finalize reshape after adding disks to array
      imsm: FIX: Fill sys_name in container_content()
      FIX: Use sysfs to change array parameters
      FIX: added disks are not used by reshape process /md/
      FIX: change adding disks criteria
      FIX: Get array information in reshape_array()
      imsm: FIX: support general migration by getinfo_super_imsm_volume
      FIX: Arrays cannot be opened exclusively
      imsm: FIX: update first array in container only
      FIX: get updated information from metadata
      imsm: FIX: Do not update anchor directly
      imsm: FIX: Perform first metadata update for container operation
      imsm: FIX: display error message
      imsm: FIX: display correct information for '-E' option
      mdadm: second_map enhancement for imsm_get_map()

Krzysztof Wojcik (3):
      FIX: Position calculation in mdstat_by_subdev
      FIX: Change size condition in imsm_reshape_is_allowed_on_container
      Manage reshape process in manage_reshape vector.


 Grow.c        |  174 ++++++++++++------
 Manage.c      |   13 -
 managemon.c   |   12 +
 mdstat.c      |    4 
 monitor.c     |   20 ++
 super-intel.c |  561 ++++++++++++++++++++++++++++++++++++++++++---------------
 6 files changed, 576 insertions(+), 208 deletions(-)

-- 
Signature

[PATCH 01/34] Manage reshape process in manage_reshape vector.

[Adam Kwolek]

From: Krzysztof Wojcik <krzysztof.wojcik@intel.com>

The manage_reshape vector is intended to be be used as
reshape process supervisor for external metadata types.
This patch introduces dummy function and necessary definitions.

Signed-off-by: Krzysztof Wojcik <krzysztof.wojcik@intel.com>
---

 Grow.c        |    7 ++++---
 super-intel.c |   12 ++++++++++++
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/Grow.c b/Grow.c
index 8fdb597..9e0a51d 100644
--- a/Grow.c
+++ b/Grow.c
@@ -1941,9 +1941,10 @@ static int reshape_array(char *container, int fd, char *devname,
 	if (st->ss->external) {
 		/* metadata handler takes it from here */
 		ping_manager(container);
-		st->ss->manage_reshape(st, backup_file);
-		frozen = 0;
-		goto release;
+		if (st->ss->manage_reshape(st, backup_file) != 1) {
+			frozen = 0;
+			goto release;
+		}
 	}
 
 	/* set up the backup-super-block.  This requires the
diff --git a/super-intel.c b/super-intel.c
index c4100ef..17737cf 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -6432,6 +6432,17 @@ static int imsm_reshape_super(struct supertype *st, long long size, int level,
 	return ret_val;
 }
 
+/* function: manage_reshape_imsm
+* Parameters: supertype structure, backup file
+* Returms:	0 on success,
+* 		-1 if failed,
+* 		1 - management of process not supported
+*/
+int manage_reshape_imsm(struct supertype *st, char *backup)
+{
+	return 1;
+}
+
 struct superswitch super_imsm = {
 #ifndef	MDASSEMBLE
 	.examine_super	= examine_super_imsm,
@@ -6469,6 +6480,7 @@ struct superswitch super_imsm = {
 	.default_geometry = default_geometry_imsm,
 	.get_disk_controller_domain = imsm_get_disk_controller_domain,
 	.reshape_super  = imsm_reshape_super,
+	.manage_reshape	= manage_reshape_imsm,
 
 	.external	= 1,
 	.name = "imsm",

[PATCH 02/34] FIX: Change size condition in imsm_reshape_is_allowed_on_container

[Adam Kwolek]

From: Krzysztof Wojcik <krzysztof.wojcik@intel.com>

Size has to be "0" when imsm_reshape_is_allowed_on_container
function is called.

Signed-off-by: Krzysztof Wojcik <krzysztof.wojcik@intel.com>
---

 super-intel.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 17737cf..06b795e 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -6149,7 +6149,7 @@ static int imsm_reshape_is_allowed_on_container(struct supertype *st,
 		"st->devnum = (%i)\n",
 		st->devnum);
 
-	if (geo->size != -1 ||
+	if (geo->size != 0 ||
 	    geo->level != UnSet ||
 	    geo->layout != UnSet ||
 	    geo->chunksize != 0 ||

[PATCH 03/34] FIX: Position calculation in mdstat_by_subdev

[Adam Kwolek]

From: Krzysztof Wojcik <krzysztof.wojcik@intel.com>

Signed-off-by: Krzysztof Wojcik <krzysztof.wojcik@intel.com>
---

 mdstat.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/mdstat.c b/mdstat.c
index bac3742..3d2edad 100644
--- a/mdstat.c
+++ b/mdstat.c
@@ -399,9 +399,9 @@ struct mdstat_ent *mdstat_by_subdev(char *subdev, int container)
 		    strncmp(mdstat->metadata_version, "external:", 9) == 0 &&
 		    strchr("/-", mdstat->metadata_version[9]) != NULL &&
 		    strncmp(mdstat->metadata_version+10, "md", 2) == 0 &&
-		    strtoul(mdstat->metadata_version+11, &pos, 10)
+		    strtoul(mdstat->metadata_version+12, &pos, 10)
 		    == (unsigned)container &&
-		    pos > mdstat->metadata_version+11 &&
+		    pos > mdstat->metadata_version+12 &&
 		    *pos == '/' &&
 		    strcmp(pos+1, subdev) == 0
 			) {

[PATCH 04/34] mdadm: second_map enhancement for imsm_get_map()

[Adam Kwolek]

Allow map related operations for the given map: first of second.
For reshape specific functionality it is required to have an access

Until now, the active map was chosen according to the current volume status.

Signed-off-by: Maciej Trela <maciej.trela@intel.com>
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   70 +++++++++++++++++++++++++++++++++------------------------
 1 files changed, 41 insertions(+), 29 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 06b795e..8fe24b3 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -509,23 +509,35 @@ static struct imsm_dev *get_imsm_dev(struct intel_super *super, __u8 index)
 	return NULL;
 }
 
-static __u32 get_imsm_ord_tbl_ent(struct imsm_dev *dev, int slot)
+/*
+ * for second_map:
+ *  == 0 get first map
+ *  == 1 get second map
+ *  == -1 than get map according to the current migr_state
+ */
+static __u32 get_imsm_ord_tbl_ent(struct imsm_dev *dev,
+				  int slot,
+				  int second_map)
 {
 	struct imsm_map *map;
 
-	if (dev->vol.migr_state)
-		map = get_imsm_map(dev, 1);
-	else
-		map = get_imsm_map(dev, 0);
+	if (second_map == -1) {
+		if (dev->vol.migr_state)
+			map = get_imsm_map(dev, 1);
+		else
+			map = get_imsm_map(dev, 0);
+	} else {
+		map = get_imsm_map(dev, second_map);
+	}
 
 	/* top byte identifies disk under rebuild */
 	return __le32_to_cpu(map->disk_ord_tbl[slot]);
 }
 
 #define ord_to_idx(ord) (((ord) << 8) >> 8)
-static __u32 get_imsm_disk_idx(struct imsm_dev *dev, int slot)
+static __u32 get_imsm_disk_idx(struct imsm_dev *dev, int slot, int second_map)
 {
-	__u32 ord = get_imsm_ord_tbl_ent(dev, slot);
+	__u32 ord = get_imsm_ord_tbl_ent(dev, slot, second_map);
 
 	return ord_to_idx(ord);
 }
@@ -733,13 +745,13 @@ static void print_imsm_dev(struct imsm_dev *dev, char *uuid, int disk_idx)
 	printf("        Members : %d\n", map->num_members);
 	printf("          Slots : [");
 	for (i = 0; i < map->num_members; i++) {
-		ord = get_imsm_ord_tbl_ent(dev, i);
+		ord = get_imsm_ord_tbl_ent(dev, i, -1);
 		printf("%s", ord & IMSM_ORD_REBUILD ? "_" : "U");
 	}
 	printf("]\n");
 	slot = get_imsm_disk_slot(map, disk_idx);
 	if (slot >= 0) {
-		ord = get_imsm_ord_tbl_ent(dev, slot);
+		ord = get_imsm_ord_tbl_ent(dev, slot, -1);
 		printf("      This Slot : %d%s\n", slot,
 		       ord & IMSM_ORD_REBUILD ? " (out-of-sync)" : "");
 	} else
@@ -1397,12 +1409,12 @@ static __u32 num_stripes_per_unit_rebuild(struct imsm_dev *dev)
 		return num_stripes_per_unit_resync(dev);
 }
 
-static __u8 imsm_num_data_members(struct imsm_dev *dev)
+static __u8 imsm_num_data_members(struct imsm_dev *dev, int second_map)
 {
 	/* named 'imsm_' because raid0, raid1 and raid10
 	 * counter-intuitively have the same number of data disks
 	 */
-	struct imsm_map *map = get_imsm_map(dev, 0);
+	struct imsm_map *map = get_imsm_map(dev, second_map);
 
 	switch (get_imsm_raid_level(map)) {
 	case 0:
@@ -1485,7 +1497,7 @@ static __u64 blocks_per_migr_unit(struct imsm_dev *dev)
 		 */
 		stripes_per_unit = num_stripes_per_unit_resync(dev);
 		migr_chunk = migr_strip_blocks_resync(dev);
-		disks = imsm_num_data_members(dev);
+		disks = imsm_num_data_members(dev, 0);
 		blocks_per_unit = stripes_per_unit * migr_chunk * disks;
 		stripe = __le32_to_cpu(map->blocks_per_strip) * disks;
 		segment = blocks_per_unit / stripe;
@@ -1616,7 +1628,7 @@ static void getinfo_super_imsm_volume(struct supertype *st, struct mdinfo *info,
 			dmap[i] = 0;
 			if (i < info->array.raid_disks) {
 				struct imsm_disk *dsk;
-				j = get_imsm_disk_idx(dev, i);
+				j = get_imsm_disk_idx(dev, i, -1);
 				dsk = get_imsm_disk(super, j);
 				if (dsk && (dsk->status & CONFIGURED_DISK))
 					dmap[i] = 1;
@@ -1693,7 +1705,7 @@ static void getinfo_super_imsm(struct supertype *st, struct mdinfo *info, char *
 		 * (catches single-degraded vs double-degraded)
 		 */
 		for (j = 0; j < map->num_members; j++) {
-			__u32 ord = get_imsm_ord_tbl_ent(dev, i);
+			__u32 ord = get_imsm_ord_tbl_ent(dev, i, -1);
 			__u32 idx = ord_to_idx(ord);
 
 			if (!(ord & IMSM_ORD_REBUILD) &&
@@ -3367,7 +3379,7 @@ static int add_to_super_imsm_volume(struct supertype *st, mdu_disk_info_t *dk,
 	/* Check the device has not already been added */
 	slot = get_imsm_disk_slot(map, dl->index);
 	if (slot >= 0 &&
-	    (get_imsm_ord_tbl_ent(dev, slot) & IMSM_ORD_REBUILD) == 0) {
+	    (get_imsm_ord_tbl_ent(dev, slot, -1) & IMSM_ORD_REBUILD) == 0) {
 		fprintf(stderr, Name ": %s has been included in this array twice\n",
 			devname);
 		return 1;
@@ -3651,7 +3663,7 @@ static int create_array(struct supertype *st, int dev_idx)
 	imsm_copy_dev(&u->dev, dev);
 	inf = get_disk_info(u);
 	for (i = 0; i < map->num_members; i++) {
-		int idx = get_imsm_disk_idx(dev, i);
+		int idx = get_imsm_disk_idx(dev, i, -1);
 
 		disk = get_imsm_disk(super, idx);
 		serialcpy(inf[i].serial, disk->serial);
@@ -4537,8 +4549,8 @@ static struct mdinfo *container_content_imsm(struct supertype *st, char *subarra
 			__u32 ord;
 
 			skip = 0;
-			idx = get_imsm_disk_idx(dev, slot);
-			ord = get_imsm_ord_tbl_ent(dev, slot); 
+			idx = get_imsm_disk_idx(dev, slot, 0);
+			ord = get_imsm_ord_tbl_ent(dev, slot, 0);
 			for (d = super->disks; d ; d = d->next)
 				if (d->index == idx)
 					break;
@@ -4637,7 +4649,7 @@ static __u8 imsm_check_degraded(struct intel_super *super, struct imsm_dev *dev,
 		int insync = insync;
 
 		for (i = 0; i < map->num_members; i++) {
-			__u32 ord = get_imsm_ord_tbl_ent(dev, i);
+			__u32 ord = get_imsm_ord_tbl_ent(dev, i, -1);
 			int idx = ord_to_idx(ord);
 			struct imsm_disk *disk;
 
@@ -4855,7 +4867,7 @@ static int imsm_set_array_state(struct active_array *a, int consistent)
 				dev->vol.migr_type = 0;
 				dev->vol.curr_migr_unit = 0;
 
-				used_disks = imsm_num_data_members(dev);
+				used_disks = imsm_num_data_members(dev, -1);
 				array_blocks = map->blocks_per_member * used_disks;
 				/* round array size down to closest MB */
 				array_blocks = (array_blocks >> SECT_PER_MB_SHIFT)
@@ -4965,7 +4977,7 @@ static void imsm_set_disk(struct active_array *a, int n, int state)
 
 	dprintf("imsm: set_disk %d:%x\n", n, state);
 
-	ord = get_imsm_ord_tbl_ent(dev, n);
+	ord = get_imsm_ord_tbl_ent(dev, n, -1);
 	disk = get_imsm_disk(super, ord_to_idx(ord));
 
 	/* check for new failures */
@@ -5073,7 +5085,7 @@ static void imsm_sync_metadata(struct supertype *container)
 static struct dl *imsm_readd(struct intel_super *super, int idx, struct active_array *a)
 {
 	struct imsm_dev *dev = get_imsm_dev(super, a->info.container_member);
-	int i = get_imsm_disk_idx(dev, idx);
+	int i = get_imsm_disk_idx(dev, idx, -1);
 	struct dl *dl;
 
 	for (dl = super->disks; dl; dl = dl->next)
@@ -5094,7 +5106,7 @@ static struct dl *imsm_add_spare(struct intel_super *super, int slot,
 				 struct mdinfo *additional_test_list)
 {
 	struct imsm_dev *dev = get_imsm_dev(super, a->info.container_member);
-	int idx = get_imsm_disk_idx(dev, slot);
+	int idx = get_imsm_disk_idx(dev, slot, -1);
 	struct imsm_super *mpb = super->anchor;
 	struct imsm_map *map;
 	unsigned long long pos;
@@ -5222,7 +5234,7 @@ static int imsm_rebuild_allowed(struct supertype *cont, int dev_idx, int failed)
 				 * Check if failed disks are deleted from intel
 				 * disk list or are marked to be deleted
 				 */
-				idx = get_imsm_disk_idx(dev2, slot);
+				idx = get_imsm_disk_idx(dev2, slot, -1);
 				idisk = get_imsm_dl_disk(cont->sb, idx);
 				/*
 				 * Do not rebuild the array if failed disks
@@ -5426,7 +5438,7 @@ static int disks_overlap(struct intel_super *super, int idx, struct imsm_update_
 	int j;
 
 	for (i = 0; i < map->num_members; i++) {
-		disk = get_imsm_disk(super, get_imsm_disk_idx(dev, i));
+		disk = get_imsm_disk(super, get_imsm_disk_idx(dev, i, -1));
 		for (j = 0; j < new_map->num_members; j++)
 			if (serialcmp(disk->serial, inf[j].serial) == 0)
 				return 1;
@@ -5567,7 +5579,6 @@ static void imsm_process_update(struct supertype *st,
 
 		/* enable spares to use in array */
 		for (i = 0; i < delta_disks; i++) {
-
 			new_disk = get_disk_super(super,
 						  major(u->new_disks[i]),
 						  minor(u->new_disks[i]));
@@ -5636,7 +5647,7 @@ update_reshape_exit:
 		struct dl *dl;
 		unsigned int found;
 		int failed;
-		int victim = get_imsm_disk_idx(dev, u->slot);
+		int victim = get_imsm_disk_idx(dev, u->slot, -1);
 		int i;
 
 		for (dl = super->disks; dl; dl = dl->next)
@@ -5659,7 +5670,8 @@ update_reshape_exit:
 		for (i = 0; i < map->num_members; i++) {
 			if (i == u->slot)
 				continue;
-			disk = get_imsm_disk(super, get_imsm_disk_idx(dev, i));
+			disk = get_imsm_disk(super,
+					     get_imsm_disk_idx(dev, i, -1));
 			if (!disk || is_failed(disk))
 				failed++;
 		}
@@ -6064,7 +6076,7 @@ static void imsm_delete(struct intel_super *super, struct dl **dlp, unsigned ind
 			/* update ord entries being careful not to propagate
 			 * ord-flags to the first map
 			 */
-			ord = get_imsm_ord_tbl_ent(dev, j);
+			ord = get_imsm_ord_tbl_ent(dev, j, -1);
 
 			if (ord_to_idx(ord) <= index)
 				continue;

[PATCH 05/34] imsm: FIX: display correct information for '-E' option

[Adam Kwolek]

Correct information displayed by '-E' option.
1. FIX: Slot information during raid0 migration is displayed incorrectly
        (missing disk position is taken from wrong map)
2. Improvement: information about (level, members, chunk size) migration is displayed.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   30 +++++++++++++++++++++++++-----
 1 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 8fe24b3..35a3769 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -736,19 +736,35 @@ static void print_imsm_dev(struct imsm_dev *dev, char *uuid, int disk_idx)
 	__u64 sz;
 	int slot, i;
 	struct imsm_map *map = get_imsm_map(dev, 0);
+	struct imsm_map *map2 = get_imsm_map(dev, 1);
 	__u32 ord;
 
 	printf("\n");
 	printf("[%.16s]:\n", dev->volume);
 	printf("           UUID : %s\n", uuid);
-	printf("     RAID Level : %d\n", get_imsm_raid_level(map));
-	printf("        Members : %d\n", map->num_members);
+	printf("     RAID Level : %d", get_imsm_raid_level(map));
+	if (map2)
+		printf(" <-- %d", get_imsm_raid_level(map2));
+	printf("\n");
+	printf("        Members : %d", map->num_members);
+	if (map2)
+		printf(" <-- %d", map2->num_members);
+	printf("\n");
 	printf("          Slots : [");
 	for (i = 0; i < map->num_members; i++) {
-		ord = get_imsm_ord_tbl_ent(dev, i, -1);
+		ord = get_imsm_ord_tbl_ent(dev, i, 0);
 		printf("%s", ord & IMSM_ORD_REBUILD ? "_" : "U");
 	}
-	printf("]\n");
+	printf("]");
+	if (map2) {
+		printf(" <-- [");
+		for (i = 0; i < map2->num_members; i++) {
+			ord = get_imsm_ord_tbl_ent(dev, i, 1);
+			printf("%s", ord & IMSM_ORD_REBUILD ? "_" : "U");
+		}
+		printf("]");
+	}
+	printf("\n");
 	slot = get_imsm_disk_slot(map, disk_idx);
 	if (slot >= 0) {
 		ord = get_imsm_ord_tbl_ent(dev, slot, -1);
@@ -768,8 +784,12 @@ static void print_imsm_dev(struct imsm_dev *dev, char *uuid, int disk_idx)
 		__le32_to_cpu(map->pba_of_lba0));
 	printf("    Num Stripes : %u\n",
 		__le32_to_cpu(map->num_data_stripes));
-	printf("     Chunk Size : %u KiB\n",
+	printf("     Chunk Size : %u KiB",
 		__le16_to_cpu(map->blocks_per_strip) / 2);
+	if (map2)
+		printf(" <-- %u KiB",
+			__le16_to_cpu(map2->blocks_per_strip) / 2);
+	printf("\n");
 	printf("       Reserved : %d\n", __le32_to_cpu(dev->reserved_blocks));
 	printf("  Migrate State : ");
 	if (dev->vol.migr_state) {

[PATCH 06/34] imsm: FIX: display error message

[Adam Kwolek]

When container operation is not allowed user has to get proper information on console about it
Currently this information was displayed as debug info only.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 35a3769..ffbf741 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -6455,7 +6455,7 @@ static int imsm_reshape_super(struct supertype *st, long long size, int level,
 				dprintf("imsm: Cannot prepare "\
 					"update\n");
 		} else
-			dprintf("imsm: Operation is not allowed "\
+			fprintf(stderr, Name "imsm: Operation is not allowed "\
 				"on this container\n");
 	} else
 		dprintf("imsm: not a container operation\n");

[PATCH 07/34] imsm: FIX: Perform first metadata update for container operation

[Adam Kwolek]

Meta data was not updated due to the following problems:
1.disk index < 0 was treated as invalid, but this is spare device
2. disk index greater than currently used disks is correct also
3. newmap pointer has to be refreshed for second map copy operation
4. size calculation has to be guarded for shrinking operation

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   18 ++++++++++++++----
 1 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index ffbf741..7c73b67 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -5602,7 +5602,13 @@ static void imsm_process_update(struct supertype *st,
 			new_disk = get_disk_super(super,
 						  major(u->new_disks[i]),
 						  minor(u->new_disks[i]));
-			if (new_disk == NULL || new_disk->index < 0)
+			dprintf("imsm: imsm_process_update(): new disk "\
+				"for reshape is: %i:%i (%p, index = %i)\n",
+				major(u->new_disks[i]), minor(u->new_disks[i]),
+				new_disk, new_disk->index);
+			if ((new_disk == NULL) ||
+			    ((new_disk->index >= 0) &&
+			     (new_disk->index < u->old_raid_disks)))
 				goto update_reshape_exit;
 
 			new_disk->index = mpb->num_disks++;
@@ -5642,7 +5648,7 @@ static void imsm_process_update(struct supertype *st,
 						     u->old_raid_disks + i);
 			}
 			/* New map is correct, now need to save old map */
-			oldmap = get_imsm_map(newdev, 1);
+			newmap = get_imsm_map(newdev, 1);
 			memcpy(newmap, oldmap, sizeof_imsm_map(oldmap));
 
 			sp = (void **)id->dev;
@@ -5987,8 +5993,9 @@ static void imsm_prepare_update(struct supertype *st,
 		for (dl = super->devlist; dl; dl = dl->next) {
 			int size = sizeof_imsm_dev(dl->dev, 1);
 			void *s;
-			size += sizeof(__u32) * 2 * 
-				(u->new_raid_disks - u->old_raid_disks);
+			if (u->new_raid_disks > u->old_raid_disks)
+				size += sizeof(__u32)*2*
+					(u->new_raid_disks - u->old_raid_disks);
 			s = malloc(size);
 			if (!s)
 				break;
@@ -6395,11 +6402,14 @@ abort:
 	 */
 	sysfs_free(spares);
 
+	dprintf("imsm: reshape update preparation :");
 	if (i == delta_disks) {
+		dprintf(" OK\n");
 		*updatep = u;
 		return update_memory_size;
 	}
 	free(u);
+	dprintf(" Error\n");
 
 	return 0;
 }

[PATCH 08/34] imsm: FIX: Do not update anchor directly

[Adam Kwolek]

During preparing metadata update local /mdadm/ anchor shouldn't be updated directly.
information should be get from metadata after update.
It makes us sure that metadata is on disk(s) already.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   31 ++++++++-----------------------
 1 files changed, 8 insertions(+), 23 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 7c73b67..0a915c0 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -6329,13 +6329,13 @@ static int imsm_create_metadata_update_for_reshape(
 	int old_raid_disks,
 	struct imsm_update_reshape **updatep)
 {
-	struct intel_super *super = st->sb;
-	struct imsm_super *mpb = super->anchor;
 	int update_memory_size = 0;
 	struct imsm_update_reshape *u = NULL;
 	struct mdinfo *spares = NULL;
 	int i;
 	int delta_disks = 0;
+	struct mdinfo *dev;
+	int added_disks = 0;
 
 	dprintf("imsm_update_metadata_for_reshape(enter) raid_disks = %i\n",
 		geo->raid_disks);
@@ -6374,27 +6374,12 @@ static int imsm_create_metadata_update_for_reshape(
 	dprintf("imsm: %i spares are available.\n\n",
 		spares->array.spare_disks);
 
-	for (i = 0; i < delta_disks; i++) {
-		struct mdinfo *dev = spares->devs;
-		struct dl *dl;
-
+	dev = spares->devs;
+	for (i = 0; (i < delta_disks) && dev; i++) {
 		u->new_disks[i] = makedev(dev->disk.major,
 					  dev->disk.minor);
-		dl = get_disk_super(super, dev->disk.major, dev->disk.minor);
-		dl->index = mpb->num_disks++;
-	}
-	/* Now update the metadata so that container_content will find
-	 * the new devices
-	 */
-	for (i = 0; i < mpb->num_raid_devs; i++) {
-		int d;
-		struct imsm_dev *dev = get_imsm_dev(super, i);
-		struct imsm_map *map = get_imsm_map(dev, 0);
-		map->num_members = geo->raid_disks;
-		for (d = 0; d < delta_disks; d++) {
-			set_imsm_ord_tbl_ent(map, old_raid_disks + d,
-					     mpb->num_disks - delta_disks + d);
-		}
+		dev = dev->next;
+		added_disks++;
 	}
 
 abort:
@@ -6403,13 +6388,13 @@ abort:
 	sysfs_free(spares);
 
 	dprintf("imsm: reshape update preparation :");
-	if (i == delta_disks) {
+	if (added_disks == delta_disks) {
 		dprintf(" OK\n");
 		*updatep = u;
 		return update_memory_size;
 	}
 	free(u);
-	dprintf(" Error\n");
+	dprintf(" Error: added_disks = %i\n", added_disks);
 
 	return 0;
 }

[PATCH 09/34] FIX: get updated information from metadata

[Adam Kwolek]

Metadata is not modified by metadata preparation handler.
It has to be read again from array.
There is 2 read required:
1. before 'for' entry to get updated information after reshape_super() call
2. inside 'for' loop to get updated information for every processed array
  (it can happen /i.e. imsm case/ that container operation is a set of array operations
   and information in metadata is changed after every loop).

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 Grow.c |   19 +++++++++++++++++--
 1 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/Grow.c b/Grow.c
index 9e0a51d..5d73365 100644
--- a/Grow.c
+++ b/Grow.c
@@ -2083,6 +2083,8 @@ int reshape_container(char *container, int cfd, char *devname,
 		      int quiet)
 {
 	struct mdinfo *cc;
+	struct mdinfo *cc_fresh = NULL;
+
 	if (reshape_super(st, info->component_size, info->new_level,
 			  info->new_layout, info->new_chunk,
 			  info->array.raid_disks + info->delta_disks,
@@ -2091,6 +2093,14 @@ int reshape_container(char *container, int cfd, char *devname,
 
 	sync_metadata(st);
 
+	/* ping monitor to be sure that update is on disk
+	 */
+	ping_monitor(container);
+
+	/* reload metadata to get updated information
+	 */
+	st->ss->load_container(st, cfd, NULL);
+
 	cc = st->ss->container_content(st, NULL);
 
 	if (!cc)
@@ -2106,7 +2116,6 @@ int reshape_container(char *container, int cfd, char *devname,
 	case 0: /* child */
 		break;
 	}
-
 	/* For each member array, we need to perform the reshape */
 	for (; cc; cc = cc->next) {
 		int rv;
@@ -2128,12 +2137,18 @@ int reshape_container(char *container, int cfd, char *devname,
 		if (!adev)
 			adev = cc->text_version;
 
-		rv = reshape_array(container, fd, adev, st, cc, force,
+		sysfs_free(cc_fresh);
+		st->ss->load_container(st, cfd, NULL);
+		cc_fresh = st->ss->container_content(st, subarray);
+		rv = reshape_array(container, fd, adev, st,
+				   cc_fresh, force,
 				   backup_file, quiet, 1);
 		close(fd);
 		if (rv)
 			break;
 	}
+	sysfs_free(cc_fresh);
+	sysfs_free(cc);
 	exit(0);
 }
 

[PATCH 10/34] imsm: FIX: update first array in container only

[Adam Kwolek]

During first metadata update imsm for compatibility reason should update
only one array.
Buffers in prepare_update() are prepared for second update as well.
---

 super-intel.c |   33 +++++++++++++++++++++------------
 1 files changed, 21 insertions(+), 12 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 0a915c0..6d76205 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -5594,6 +5594,7 @@ static void imsm_process_update(struct supertype *st,
 		int i;
 		int delta_disks = u->new_raid_disks - u->old_raid_disks;
 		void **tofree = NULL;
+		int devices_to_reshape = 1;
 
 		dprintf("imsm: imsm_process_update() for update_reshape\n");
 
@@ -5621,7 +5622,7 @@ static void imsm_process_update(struct supertype *st,
 
 		dprintf("imsm: process_update(): update_reshape: volume set"\
 			" mpb->num_raid_devs = %i\n", mpb->num_raid_devs);
-		/* manage changes in volumes
+		/* manage changes in volume
 		 */
 		for (id = super->devlist ; id; id = id->next) {
 			void **sp = update->space_list;
@@ -5638,18 +5639,26 @@ static void imsm_process_update(struct supertype *st,
 			newmap = get_imsm_map(newdev, 0);
 			/* Copy the current map */
 			memcpy(newmap, oldmap, sizeof_imsm_map(oldmap));
-			newdev->vol.migr_state = 1;
-			newdev->vol.curr_migr_unit = 0;
-			newdev->vol.migr_type = MIGR_GEN_MIGR;
-			newmap->num_members = u->new_raid_disks;
-			for (i = 0; i < delta_disks; i++) {
-				set_imsm_ord_tbl_ent(newmap,
-						     u->old_raid_disks + i,
-						     u->old_raid_disks + i);
+			/* update one device only
+			 */
+			if (devices_to_reshape) {
+				dprintf("process_update(): modifying "\
+					"subdev: %i\n", id->index);
+				devices_to_reshape--;
+				newdev->vol.migr_state = 1;
+				newdev->vol.curr_migr_unit = 0;
+				newdev->vol.migr_type = MIGR_GEN_MIGR;
+				newmap->num_members = u->new_raid_disks;
+				for (i = 0; i < delta_disks; i++) {
+					set_imsm_ord_tbl_ent(newmap,
+							u->old_raid_disks + i,
+							u->old_raid_disks + i);
+				}
+				/* New map is correct, now need to save old map
+				 */
+				newmap = get_imsm_map(newdev, 1);
+				memcpy(newmap, oldmap, sizeof_imsm_map(oldmap));
 			}
-			/* New map is correct, now need to save old map */
-			newmap = get_imsm_map(newdev, 1);
-			memcpy(newmap, oldmap, sizeof_imsm_map(oldmap));
 
 			sp = (void **)id->dev;
 			id->dev = newdev;

[PATCH 11/34] FIX: Arrays cannot be opened exclusively

[Adam Kwolek]

During container operation arrays cannot be opened exclusively.
Use open_dev() instead.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 Grow.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/Grow.c b/Grow.c
index 5d73365..c568a55 100644
--- a/Grow.c
+++ b/Grow.c
@@ -2128,7 +2128,7 @@ int reshape_container(char *container, int cfd, char *devname,
 
 		if (!mdstat)
 			continue;
-		fd = open_dev_excl(mdstat->devnum);
+		fd = open_dev(mdstat->devnum);
 		if (fd < 0)
 			break;
 		adev = map_dev(dev2major(mdstat->devnum),

[PATCH 12/34] imsm: FIX: support general migration by getinfo_super_imsm_volume

[Adam Kwolek]

Add support for reading volume information during migration process.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   29 ++++++++++++++++++++++-------
 1 files changed, 22 insertions(+), 7 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 6d76205..de3585e 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -1563,26 +1563,39 @@ static void getinfo_super_imsm_volume(struct supertype *st, struct mdinfo *info,
 	struct imsm_dev *dev = get_imsm_dev(super, super->current_vol);
 	struct imsm_map *map = get_imsm_map(dev, 0);
 	struct imsm_map *prev_map = get_imsm_map(dev, 1);
+	struct imsm_map *map_to_analyse = map;
 	struct dl *dl;
 	char *devname;
 	int map_disks = info->array.raid_disks;
 
+	if (prev_map)
+		map_to_analyse = prev_map;
+
 	for (dl = super->disks; dl; dl = dl->next)
 		if (dl->raiddisk == info->disk.raid_disk)
 			break;
 	info->container_member	  = super->current_vol;
-	info->array.raid_disks    = map->num_members;
-	info->array.level	  = get_imsm_raid_level(map);
+	info->array.raid_disks    = map_to_analyse->num_members;
+	info->array.level	  = get_imsm_raid_level(map_to_analyse);
 	info->array.layout	  = imsm_level_to_layout(info->array.level);
 	info->array.md_minor	  = -1;
 	info->array.ctime	  = 0;
 	info->array.utime	  = 0;
-	info->array.chunk_size	  = __le16_to_cpu(map->blocks_per_strip) << 9;
+	info->array.chunk_size	  =
+		__le16_to_cpu(map_to_analyse->blocks_per_strip) << 9;
 	info->array.state	  = !dev->vol.dirty;
 	info->custom_array_size   = __le32_to_cpu(dev->size_high);
 	info->custom_array_size   <<= 32;
 	info->custom_array_size   |= __le32_to_cpu(dev->size_low);
-
+	if (prev_map) {
+		info->new_level = get_imsm_raid_level(map);
+		info->new_layout = imsm_level_to_layout(info->new_level);
+		info->new_chunk = __le16_to_cpu(map->blocks_per_strip) << 9;
+	} else {
+		info->new_level = UnSet;
+		info->new_layout = UnSet;
+		info->new_chunk = info->array.chunk_size;
+	}
 	info->disk.major = 0;
 	info->disk.minor = 0;
 	if (dl) {
@@ -1590,8 +1603,9 @@ static void getinfo_super_imsm_volume(struct supertype *st, struct mdinfo *info,
 		info->disk.minor = dl->minor;
 	}
 
-	info->data_offset	  = __le32_to_cpu(map->pba_of_lba0);
-	info->component_size	  = __le32_to_cpu(map->blocks_per_member);
+	info->data_offset	  = __le32_to_cpu(map_to_analyse->pba_of_lba0);
+	info->component_size	  =
+		__le32_to_cpu(map_to_analyse->blocks_per_member);
 	memset(info->uuid, 0, sizeof(info->uuid));
 	info->recovery_start = MaxSector;
 	info->reshape_active = (prev_map != NULL);
@@ -1600,7 +1614,8 @@ static void getinfo_super_imsm_volume(struct supertype *st, struct mdinfo *info,
 	else
 		info->delta_disks = 0;
 
-	if (map->map_state == IMSM_T_STATE_UNINITIALIZED || dev->vol.dirty) {
+	if (map_to_analyse->map_state == IMSM_T_STATE_UNINITIALIZED ||
+	    dev->vol.dirty) {
 		info->resync_start = 0;
 	} else if (dev->vol.migr_state) {
 		switch (migr_type(dev)) {

[PATCH 13/34] FIX: Get array information in reshape_array()

[Adam Kwolek]

Uninitialized array structure is used.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 Grow.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/Grow.c b/Grow.c
index c568a55..6dbc3b7 100644
--- a/Grow.c
+++ b/Grow.c
@@ -1637,6 +1637,10 @@ static int reshape_array(char *container, int fd, char *devname,
 		fprintf(stderr, Name ": %s\n", msg);
 		return 1;
 	}
+	if (ioctl(fd, GET_ARRAY_INFO, &array) != 0) {
+		dprintf("Canot get array information.\n");
+		return 1;
+	}
 	spares_needed =  max(reshape.before.raid_disks,
 			     reshape.after.raid_disks) - array.raid_disks;
 

[PATCH 14/34] FIX: change adding disks criteria

[Adam Kwolek]

Some of disks tried to add are array member already.
New disks should be added only.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 Grow.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Grow.c b/Grow.c
index 6dbc3b7..9801a45 100644
--- a/Grow.c
+++ b/Grow.c
@@ -1696,8 +1696,8 @@ static int reshape_array(char *container, int fd, char *devname,
 
 		if (info2)
 			for (d = info2->devs; d; d = d->next) {
-				if (d->disk.state == 0 &&
-				    d->disk.raid_disk >= 0) {
+				if (d->disk.raid_disk >=
+				    reshape.before.raid_disks) {
 					/* This is a spare that wants to
 					 * be part of the array.
 					 */

[PATCH 15/34] FIX: added disks are not used by reshape process /md/

[Adam Kwolek]

md runs reshape but added disk is not touched
/proc/mdstat reports that disk is used.

To fix this let md to set slot for reshape.
It is possible that slot verification mechanism has to be added.
Second possibility is to investigate problem in md

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 Grow.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/Grow.c b/Grow.c
index 9801a45..c72b89f 100644
--- a/Grow.c
+++ b/Grow.c
@@ -1701,6 +1701,7 @@ static int reshape_array(char *container, int fd, char *devname,
 					/* This is a spare that wants to
 					 * be part of the array.
 					 */
+					d->disk.raid_disk = -1;
 					add_disk(fd, st, info2, d);
 				}
 			}

[PATCH 16/34] FIX: Use sysfs to change array parameters

[Adam Kwolek]

For external metadata parameters has to be changed via sysfs.
i.e. change of raid_disks requires handshake mdmon<->md (md_allow_write())

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 Grow.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/Grow.c b/Grow.c
index c72b89f..dae9bbf 100644
--- a/Grow.c
+++ b/Grow.c
@@ -1896,7 +1896,8 @@ static int reshape_array(char *container, int fd, char *devname,
 	sync_metadata(st);
 
 	if (info->array.chunk_size == info->new_chunk &&
-	    reshape.before.layout == reshape.after.layout) {
+	    reshape.before.layout == reshape.after.layout &&
+	    st->ss->external == 0) {
 		array.raid_disks = reshape.after.raid_disks;
 		if (ioctl(fd, SET_ARRAY_INFO, &array) != 0) {
 			int err = errno;

[PATCH 17/34] imsm: FIX: Fill sys_name in container_content()

[Adam Kwolek]

If sys_name field is empty structures produced by container_content()
cannot be used for sysfs access.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index de3585e..f07665b 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -4517,6 +4517,9 @@ static void update_recovery_start(struct imsm_dev *dev, struct mdinfo *array)
 	rebuild->recovery_start = units * blocks_per_migr_unit(dev);
 }
 
+static int imsm_find_array_minor_by_subdev(int subdev,
+					   int container,
+					   int *minor);
 
 static struct mdinfo *container_content_imsm(struct supertype *st, char *subarray)
 {
@@ -4545,6 +4548,7 @@ static struct mdinfo *container_content_imsm(struct supertype *st, char *subarra
 		struct mdinfo *this;
 		int slot;
 		char *ep;
+		int minor;
 
 		if (subarray &&
 		    (i != strtoul(subarray, &ep, 10) || *ep != '\0'))
@@ -4575,6 +4579,14 @@ static struct mdinfo *container_content_imsm(struct supertype *st, char *subarra
 
 		super->current_vol = i;
 		getinfo_super_imsm_volume(st, this, NULL);
+
+		if (imsm_find_array_minor_by_subdev(i,
+						    st->container_dev,
+						    &minor) >= 0) {
+			sprintf(this->sys_name, "md%i", minor);
+		}
+
+
 		for (slot = 0 ; slot <  map->num_members; slot++) {
 			unsigned long long recovery_start;
 			struct mdinfo *info_d;

[PATCH 18/34] Finalize reshape after adding disks to array

[Adam Kwolek]

When reshape is finished monitor, has to finalize reshape in metadata.
To do this set_array_state() should be called.
This finishes migration and stores metadata on disks.

reshape_delta_disks is set to not active value.
This finishes reshape flow in mdmon.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 monitor.c |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/monitor.c b/monitor.c
index 3962d9e..65fcba1 100644
--- a/monitor.c
+++ b/monitor.c
@@ -315,6 +315,17 @@ static int read_and_act(struct active_array *a)
 		 */
 		check_reshape = 1;
 
+	/* finalize reshape detection
+	 */
+	if ((a->curr_action != reshape) &&
+	    (a->prev_action == reshape)) {
+		/* A reshape has finished.
+		 * Some disks may be in sync now.
+		 */
+		a->container->ss->set_array_state(a, a->curr_state <= clean);
+		check_degraded = 1;
+	}
+
 	/* Check for failures and if found:
 	 * 1/ Record the failure in the metadata and unblock the device.
 	 *    FIXME update the kernel to stop notifying on failed drives when

[PATCH 19/34] imsm: Finalize reshape in metadata

[Adam Kwolek]

When reshape is finished monitor calls set_array_state() and finishes migration in metadata.
This change allows for finishing metadata migration on reshape end.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index f07665b..58ed702 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -4894,6 +4894,7 @@ static int imsm_set_array_state(struct active_array *a, int consistent)
 				dev->vol.curr_migr_unit = __cpu_to_le32(unit);
 				super->updates_pending++;
 			}
+			return 0;
 		} else {
 			if (a->last_checkpoint == 0 && a->prev_action == reshape) {
 				/* for some reason we aborted the reshape.
@@ -4928,7 +4929,6 @@ static int imsm_set_array_state(struct active_array *a, int consistent)
 				super->updates_pending++;
 			}				
 		}
-		return 0;
 	}
 
 	/* before we activate this array handle any missing disks */

[PATCH 20/34] imsm: FIX: Division by 0

[Adam Kwolek]

For general migration function blocks_per_migr_unit() has to return valid value.
If there is no valid return, 0 is returned instead and causes division by 0 error.
Additionally guard in function was added for such case.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   15 +++++++++------
 1 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 58ed702..dc4f308 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -1498,6 +1498,7 @@ static __u64 blocks_per_migr_unit(struct imsm_dev *dev)
 		return 0;
 
 	switch (migr_type(dev)) {
+	case MIGR_GEN_MIGR:
 	case MIGR_VERIFY:
 	case MIGR_REPAIR:
 	case MIGR_INIT: {
@@ -1534,8 +1535,6 @@ static __u64 blocks_per_migr_unit(struct imsm_dev *dev)
 		migr_chunk = migr_strip_blocks_rebuild(dev);
 		return migr_chunk * stripes_per_unit;
 	}
-	case MIGR_GEN_MIGR:
-		/* FIXME I need a number here */
 	case MIGR_STATE_CHANGE:
 	default:
 		return 0;
@@ -4889,10 +4888,14 @@ static int imsm_set_array_state(struct active_array *a, int consistent)
 			/* still reshaping, maybe update curr_migr_unit */
 			long long blocks_per_unit = blocks_per_migr_unit(dev);
 			long long unit = a->last_checkpoint;
-			unit /= blocks_per_unit;
-			if (unit > __le32_to_cpu(dev->vol.curr_migr_unit)) {
-				dev->vol.curr_migr_unit = __cpu_to_le32(unit);
-				super->updates_pending++;
+			if (blocks_per_unit) {
+				unit /= blocks_per_unit;
+				if (unit >
+				    __le32_to_cpu(dev->vol.curr_migr_unit)) {
+					dev->vol.curr_migr_unit =
+						__cpu_to_le32(unit);
+					super->updates_pending++;
+				}
 			}
 			return 0;
 		} else {

[PATCH 21/34] FIX: Initialize subarray variable in reshape_array

[Adam Kwolek]

subarray variable was not initialized in reshape_array()

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 Grow.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/Grow.c b/Grow.c
index dae9bbf..1c10889 100644
--- a/Grow.c
+++ b/Grow.c
@@ -1616,7 +1616,7 @@ static int reshape_array(char *container, int fd, char *devname,
 	struct mdu_array_info_s array;
 	char *c;
 	int rv = 0;
-	char *subarray = NULL;
+	char *subarray = strchr(info->text_version+1, '/') + 1;
 
 	int *fdlist;
 	unsigned long long *offsets;

[PATCH 22/34] imsm: update array size information in metadata

[Adam Kwolek]

When disks are added size has to increase in metadata.
This size should be used by common code to set size in md when reshape will be finished.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   23 +++++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index dc4f308..72b6cf5 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -5672,6 +5672,7 @@ static void imsm_process_update(struct supertype *st,
 			/* update one device only
 			 */
 			if (devices_to_reshape) {
+				int used_disks;
 				dprintf("process_update(): modifying "\
 					"subdev: %i\n", id->index);
 				devices_to_reshape--;
@@ -5688,6 +5689,28 @@ static void imsm_process_update(struct supertype *st,
 				 */
 				newmap = get_imsm_map(newdev, 1);
 				memcpy(newmap, oldmap, sizeof_imsm_map(oldmap));
+
+				/* calculate new size
+				 */
+				used_disks = imsm_num_data_members(newdev, 0);
+				if (used_disks) {
+					unsigned long long array_blocks;
+
+					array_blocks =
+						newmap->blocks_per_member
+						* used_disks;
+					/* round array size down to closest MB
+					 */
+					array_blocks = (array_blocks
+							>> SECT_PER_MB_SHIFT)
+							<< SECT_PER_MB_SHIFT;
+					newdev->size_low =
+						__cpu_to_le32(
+							(__u32)array_blocks);
+					newdev->size_high =
+						__cpu_to_le32(
+						(__u32)(array_blocks >> 32));
+				}
 			}
 
 			sp = (void **)id->dev;

[PATCH 23/34] Set array size after adding new disks

[Adam Kwolek]

When new disks are added array size has to be set by mdadm as array grows.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 Grow.c |   67 +++++++++++++++++++++++++++++++++++++++++++++++++---------------
 1 files changed, 51 insertions(+), 16 deletions(-)

diff --git a/Grow.c b/Grow.c
index 1c10889..0b151f1 100644
--- a/Grow.c
+++ b/Grow.c
@@ -2034,26 +2034,61 @@ static int reshape_array(char *container, int fd, char *devname,
 			unlink(backup_file);
 		if (!done)
 			abort_reshape(sra);
-		else if (info->new_level != info->array.level) {
-			/* We need to wait for the reshape to finish
-			 * (which will have happened unless odata < ndata)
-			 * and then set the level
+		else {
+			/* set new array size size - if required
 			 */
+			if (reshape.before.raid_disks <
+			    reshape.after.raid_disks) {
+				struct mdinfo *info2;
 
-			c = map_num(pers, info->new_level);
-			if (c == NULL) {
-				if (forked)
-					return 1;
-				exit(0);/* not possible */
+				wait_reshape(sra);
+				ping_monitor(container);
+
+				info2 = st->ss->container_content(st, subarray);
+				if (info2) {
+					unsigned long long current_size = 0;
+					unsigned long long new_size =
+						info2->custom_array_size/2;
+
+					sysfs_get_ll(sra,
+						     NULL,
+						     "array_size",
+						     &current_size);
+					if (new_size > current_size) {
+						if (sysfs_set_num(sra, NULL,
+							"array_size", new_size)
+						    < 0)
+							dprintf("Error: Cannot"\
+							     " set array size");
+						else
+							dprintf("Array size "\
+								"changed");
+						dprintf(" from %llu to %llu.\n",
+							current_size, new_size);
+					}
+					sysfs_free(info2);
+				}
 			}
 
-			if (reshape.before.raid_disks < 
-			    reshape.after.raid_disks)
-				wait_reshape(sra);
-			err = sysfs_set_str(sra, NULL, "level", c);
-			if (err)
-				fprintf(stderr, Name ": %s: could not set level to %s\n",
-					devname, c);
+			if (info->new_level != info->array.level) {
+				/* We need to wait for the reshape to finish
+				* (which will have happened unless
+				 * odata < ndata) and then set the level
+				*/
+
+				c = map_num(pers, info->new_level);
+				if (c == NULL) {
+					if (forked)
+						return 1;
+					exit(0);/* not possible */
+				}
+
+				err = sysfs_set_str(sra, NULL, "level", c);
+				if (err)
+					fprintf(stderr, Name\
+						": %s: could not set level"\
+						"to %s\n", devname, c);
+			}
 		}
 		if (forked)
 			return 0;

[PATCH 24/34] imsm: Update metadata for second array

[Adam Kwolek]

When second array reshape is about to start metadata should be update
by mdmon in imsm_set_array_state().

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   77 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 77 insertions(+), 0 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 72b6cf5..0f10e32 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -5002,6 +5002,83 @@ static int imsm_set_array_state(struct active_array *a, int consistent)
 
 		for (mdi = a->info.devs; mdi; mdi = mdi->next)
 			imsm_set_disk(a, mdi->disk.raid_disk, mdi->curr_state);
+	} else if (dev->vol.migr_state == 0) {
+		/* check here if this is container action
+		 * and next metadata should be prepared
+		*/
+		struct active_array *aa = a->container->arrays;
+		int arrays_under_reshape = 0;
+
+		/* check if no other arrays is under reshape
+		 */
+		while (aa) {
+			if (aa->container) {
+				int inst = aa->info.container_member;
+				struct intel_super *super = aa->container->sb;
+				struct imsm_dev *dev =
+						get_imsm_dev(super, inst);
+				struct imsm_map *map2 = get_imsm_map(dev, 1);
+
+				if (map2)
+					arrays_under_reshape++;
+			}
+			aa = aa->next;
+		}
+		if (arrays_under_reshape == 0) {
+			struct imsm_map *map = get_imsm_map(dev, 0);
+			struct dl *dl = NULL;
+			int disks_count = 0;
+
+			/* check if this array should be reshaped
+			*/
+			for (dl = super->disks; dl; dl = dl->next)
+				if (dl->index >= 0)
+					disks_count++;
+			if (disks_count > map->num_members) {
+				/* manage changes in volume
+				 */
+				struct imsm_map *map, *map2;
+				int prev_num_members;
+				int used_disks;
+				int i;
+
+				map = get_imsm_map(dev, 0);
+				prev_num_members = map->num_members;
+				map->num_members = disks_count;
+				dev->vol.migr_state = 1;
+				dev->vol.curr_migr_unit = 0;
+				dev->vol.migr_type = MIGR_GEN_MIGR;
+				for (i = prev_num_members;
+					i < map->num_members; i++)
+					set_imsm_ord_tbl_ent(map, i, i);
+				map2 = get_imsm_map(dev, 1);
+				/* Copy the current map */
+				memcpy(map2, map, sizeof_imsm_map(map));
+				map2->num_members = prev_num_members;
+
+				/* calculate new size
+				 */
+				used_disks = imsm_num_data_members(dev, 0);
+				if (used_disks) {
+					unsigned long long array_blocks;
+
+					array_blocks =
+						map->blocks_per_member
+						* used_disks;
+					/* round array size down to closest MB
+					 */
+					array_blocks = (array_blocks
+							>> SECT_PER_MB_SHIFT)
+							<< SECT_PER_MB_SHIFT;
+					dev->size_low =
+					     __cpu_to_le32((__u32)array_blocks);
+					dev->size_high =
+						__cpu_to_le32(
+						(__u32)(array_blocks >> 32));
+				}
+				super->updates_pending++;
+			}
+		}
 	}
 
 	return consistent;

[PATCH 25/34] Add spares to raid0 in mdadm

[Adam Kwolek]

When user wants to add spares to container with raid0 arrays only
it is not possible to update metadata due to lack of running mdmon.
To allow for this direct metadata update by mdadm is used in such case.

Signed-off-by: Krzysztof Wojcik <krzysztof.wojcik@intel.com>
---

 Manage.c      |   13 +++----------
 super-intel.c |    8 ++++++++
 2 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/Manage.c b/Manage.c
index 81fa986..6b5e4be 100644
--- a/Manage.c
+++ b/Manage.c
@@ -844,9 +844,8 @@ int Manage_subdevs(char *devname, int fd,
 			if (dv->writemostly == 1)
 				disc.state |= (1 << MD_DISK_WRITEMOSTLY);
 			if (tst->ss->external) {
-				/* add a disk to an external metadata container
-				 * only if mdmon is around to see it
-				 */
+				/* add a disk
+				 * to an external metadata container */
 				struct mdinfo new_mdi;
 				struct mdinfo *sra;
 				int container_fd;
@@ -860,13 +859,6 @@ int Manage_subdevs(char *devname, int fd,
 					return 1;
 				}
 
-				if (!mdmon_running(devnum)) {
-					fprintf(stderr, Name ": add failed for %s: mdmon not running\n",
-						dv->devname);
-					close(container_fd);
-					return 1;
-				}
-
 				sra = sysfs_read(container_fd, -1, 0);
 				if (!sra) {
 					fprintf(stderr, Name ": add failed for %s: sysfs_read failed\n",
@@ -884,6 +876,7 @@ int Manage_subdevs(char *devname, int fd,
 					fprintf(stderr, Name ": add new device to external metadata"
 						" failed for %s\n", dv->devname);
 					close(container_fd);
+					sysfs_free(sra);
 					return 1;
 				}
 				ping_monitor(devnum2devname(devnum));
diff --git a/super-intel.c b/super-intel.c
index 0f10e32..276e72d 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -3442,6 +3442,8 @@ static int add_to_super_imsm_volume(struct supertype *st, mdu_disk_info_t *dk,
 	return 0;
 }
 
+static int write_super_imsm(struct supertype *st, int doclose);
+
 static int add_to_super_imsm(struct supertype *st, mdu_disk_info_t *dk,
 			      int fd, char *devname)
 {
@@ -3506,6 +3508,12 @@ static int add_to_super_imsm(struct supertype *st, mdu_disk_info_t *dk,
 		super->disks = dd;
 	}
 
+	if (!mdmon_running(st->container_dev)) {
+		dprintf("imsm: mdmon is not active- write metadata by mdadm\n");
+		super->updates_pending++;
+		write_super_imsm(st, 0);
+	}
+
 	return 0;
 }
 

[PATCH 26/34] imsm: Move reshape update processing to function

[Adam Kwolek]

For code reuse in raid0 reshape case when monitor is not loaded.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |  229 +++++++++++++++++++++++++++++++--------------------------
 1 files changed, 126 insertions(+), 103 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 276e72d..565afd4 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -5660,6 +5660,121 @@ static int add_remove_disk_update(struct intel_super *super)
 	return check_degraded;
 }
 
+static int apply_reshape_container_disks_update(struct imsm_update_reshape *u,
+						 struct intel_super *super,
+						 void ***space_list,
+						 int devices_to_reshape)
+{
+	struct dl *new_disk;
+	struct intel_dev *id;
+	int i;
+	int delta_disks = u->new_raid_disks - u->old_raid_disks;
+	void **tofree = NULL;
+	struct imsm_super *mpb;
+	int ret_val = 0;
+
+	dprintf("imsm: apply_reshape_container_disks_update() for reshape\n");
+	mpb = super->anchor;
+	/* enable spares to use in array
+	 */
+	for (i = 0; i < delta_disks; i++) {
+		new_disk = get_disk_super(super,
+					  major(u->new_disks[i]),
+					  minor(u->new_disks[i]));
+		dprintf("imsm: apply_reshape_container_disks_update():"\
+			"new disk for reshape is: %i:%i (%p, index = %i)\n",
+			major(u->new_disks[i]), minor(u->new_disks[i]),
+			new_disk, new_disk->index);
+		if ((new_disk == NULL) ||
+		    ((new_disk->index >= 0) &&
+		     (new_disk->index < u->old_raid_disks)))
+			goto update_reshape_exit;
+		new_disk->index = mpb->num_disks++;
+		/* slot to fill in autolayout
+		 */
+		new_disk->raiddisk = new_disk->index;
+		new_disk->disk.status |=
+			CONFIGURED_DISK;
+		new_disk->disk.status &= ~SPARE_DISK;
+	}
+
+	dprintf("imsm: apply_reshape_container_disks_update():"
+		"update_reshape: volume set mpb->num_raid_devs = %i\n",
+		mpb->num_raid_devs);
+	/* manage changes in volume
+	 */
+	for (id = super->devlist ; id; id = id->next) {
+		void **sp = *space_list;
+		struct imsm_dev *newdev;
+		struct imsm_map *newmap, *oldmap;
+
+		if (!sp)
+			continue;
+		*space_list = *sp;
+		newdev = (void *)sp;
+		/* Copy the dev, but not (all of) the map
+		 */
+		memcpy(newdev, id->dev, sizeof(*newdev));
+		oldmap = get_imsm_map(id->dev, 0);
+		newmap = get_imsm_map(newdev, 0);
+		/* Copy the current map
+		 */
+		memcpy(newmap, oldmap, sizeof_imsm_map(oldmap));
+		/* update one device only
+		 */
+		if (devices_to_reshape) {
+			int used_disks;
+
+			dprintf("apply_reshape_container_disks_update():"\
+				"modifying subdev: %i\n", id->index);
+			devices_to_reshape--;
+			newdev->vol.migr_state = 1;
+			newdev->vol.curr_migr_unit = 0;
+			newdev->vol.migr_type = MIGR_GEN_MIGR;
+			newmap->num_members = u->new_raid_disks;
+			for (i = 0; i < delta_disks; i++) {
+				set_imsm_ord_tbl_ent(newmap,
+						     u->old_raid_disks + i,
+						     u->old_raid_disks + i);
+			}
+			/* New map is correct, now need to save old map
+			 */
+			newmap = get_imsm_map(newdev, 1);
+			memcpy(newmap, oldmap, sizeof_imsm_map(oldmap));
+
+			/* calculate new size
+			 */
+			used_disks = imsm_num_data_members(newdev, 0);
+			if (used_disks) {
+				unsigned long long array_blocks;
+
+				array_blocks =
+					newmap->blocks_per_member * used_disks;
+				/* round array size down to closest MB
+				 */
+				array_blocks = (array_blocks
+					>> SECT_PER_MB_SHIFT)
+					<< SECT_PER_MB_SHIFT;
+				newdev->size_low =
+					__cpu_to_le32((__u32)array_blocks);
+				newdev->size_high =
+					__cpu_to_le32((__u32)
+							(array_blocks >> 32));
+			}
+		}
+
+		sp = (void **)id->dev;
+		id->dev = newdev;
+		*sp = tofree;
+		tofree = sp;
+	}
+	ret_val = 1;
+
+update_reshape_exit:
+
+	return ret_val;
+}
+
 static void imsm_process_update(struct supertype *st,
 			        struct metadata_update *update)
 {
@@ -5704,109 +5819,12 @@ static void imsm_process_update(struct supertype *st,
 	switch (type) {
 	case update_reshape_container_disks: {
 		struct imsm_update_reshape *u = (void *)update->buf;
-		struct dl *new_disk;
-		struct intel_dev *id;
-		int i;
-		int delta_disks = u->new_raid_disks - u->old_raid_disks;
-		void **tofree = NULL;
-		int devices_to_reshape = 1;
-
-		dprintf("imsm: imsm_process_update() for update_reshape\n");
-
-		/* enable spares to use in array */
-		for (i = 0; i < delta_disks; i++) {
-			new_disk = get_disk_super(super,
-						  major(u->new_disks[i]),
-						  minor(u->new_disks[i]));
-			dprintf("imsm: imsm_process_update(): new disk "\
-				"for reshape is: %i:%i (%p, index = %i)\n",
-				major(u->new_disks[i]), minor(u->new_disks[i]),
-				new_disk, new_disk->index);
-			if ((new_disk == NULL) ||
-			    ((new_disk->index >= 0) &&
-			     (new_disk->index < u->old_raid_disks)))
-				goto update_reshape_exit;
-
-			new_disk->index = mpb->num_disks++;
-			/* slot to fill in autolayout */
-			new_disk->raiddisk = new_disk->index;
-			new_disk->disk.status |=
-				CONFIGURED_DISK;
-			new_disk->disk.status &= ~SPARE_DISK;
-		}
-
-		dprintf("imsm: process_update(): update_reshape: volume set"\
-			" mpb->num_raid_devs = %i\n", mpb->num_raid_devs);
-		/* manage changes in volume
-		 */
-		for (id = super->devlist ; id; id = id->next) {
-			void **sp = update->space_list;
-			struct imsm_dev *newdev;
-			struct imsm_map *newmap, *oldmap;
-
-			if (!sp)
-				continue;
-			update->space_list = *sp;
-			newdev = (void*)sp;
-			/* Copy the dev, but not (all of) the map */
-			memcpy(newdev, id->dev, sizeof(*newdev));
-			oldmap = get_imsm_map(id->dev, 0);
-			newmap = get_imsm_map(newdev, 0);
-			/* Copy the current map */
-			memcpy(newmap, oldmap, sizeof_imsm_map(oldmap));
-			/* update one device only
-			 */
-			if (devices_to_reshape) {
-				int used_disks;
-				dprintf("process_update(): modifying "\
-					"subdev: %i\n", id->index);
-				devices_to_reshape--;
-				newdev->vol.migr_state = 1;
-				newdev->vol.curr_migr_unit = 0;
-				newdev->vol.migr_type = MIGR_GEN_MIGR;
-				newmap->num_members = u->new_raid_disks;
-				for (i = 0; i < delta_disks; i++) {
-					set_imsm_ord_tbl_ent(newmap,
-							u->old_raid_disks + i,
-							u->old_raid_disks + i);
-				}
-				/* New map is correct, now need to save old map
-				 */
-				newmap = get_imsm_map(newdev, 1);
-				memcpy(newmap, oldmap, sizeof_imsm_map(oldmap));
+		int ret_val;
 
-				/* calculate new size
-				 */
-				used_disks = imsm_num_data_members(newdev, 0);
-				if (used_disks) {
-					unsigned long long array_blocks;
-
-					array_blocks =
-						newmap->blocks_per_member
-						* used_disks;
-					/* round array size down to closest MB
-					 */
-					array_blocks = (array_blocks
-							>> SECT_PER_MB_SHIFT)
-							<< SECT_PER_MB_SHIFT;
-					newdev->size_low =
-						__cpu_to_le32(
-							(__u32)array_blocks);
-					newdev->size_high =
-						__cpu_to_le32(
-						(__u32)(array_blocks >> 32));
-				}
-			}
-
-			sp = (void **)id->dev;
-			id->dev = newdev;
-			*sp = tofree;
-			tofree = sp;
-		}
-
-		update->space_list = tofree;
-		super->updates_pending++;
-update_reshape_exit:
+		ret_val = apply_reshape_container_disks_update(u, super,
+							&update->space_list, 1);
+		if (ret_val)
+			super->updates_pending++;
 		break;
 	}
 	case update_activate_spare: {
@@ -6592,7 +6610,12 @@ static int imsm_reshape_super(struct supertype *st, long long size, int level,
 
 			if (len) {
 				ret_val = 0;
-				append_metadata_update(st, u, len);
+				if (mdmon_running(st->container_dev))
+					append_metadata_update(st, u, len);
+				else {
+					/* no mdmon - apply update
+					 */
+				}
 			} else
 				dprintf("imsm: Cannot prepare "\
 					"update\n");

[PATCH 27/34] imsm: Update raid0 metadata for reshape

[Adam Kwolek]

When raid0 reshape is performed metadata has to be applied by mdadm.
(without mdmon)

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   82 +++++++++++++++++++++++++++++++++++++++++++++++++--------
 1 files changed, 71 insertions(+), 11 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 565afd4..1be0e72 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -6595,36 +6595,96 @@ static int imsm_reshape_super(struct supertype *st, long long size, int level,
 		return ret_val;
 
 	/* verify reshape conditions
-	 * on container level we can only increase number of devices. */
+	 * on container level we can only increase number of devices.
+	 */
 	if (st->container_dev == st->devnum) {
 		/* check for delta_disks > 0
-		 *and supported raid levels 0 and 5 only in container */
+		 * and supported raid levels 0 and 5 only in container
+		 */
 		int old_raid_disks = 0;
 		if (imsm_reshape_is_allowed_on_container(
 			    st, &geo, &old_raid_disks)) {
 			struct imsm_update_reshape *u = NULL;
 			int len;
+			struct intel_super *super = st->sb;
+			void **space_list;
+			struct intel_dev *dl;
+			void **space_tail = (void **)&space_list;
+
 
 			len = imsm_create_metadata_update_for_reshape(
 				st, &geo, old_raid_disks, &u);
 
-			if (len) {
+			if (len <= 0) {
+				dprintf("imsm: Cannot prepare update\n");
+				goto exit_imsm_reshape_super;
+
+			}
+			if (mdmon_running(st->container_dev)) {
 				ret_val = 0;
-				if (mdmon_running(st->container_dev))
-					append_metadata_update(st, u, len);
-				else {
-					/* no mdmon - apply update
-					 */
+				append_metadata_update(st, u, len);
+				goto exit_imsm_reshape_super;
+			}
+
+			/* no mdmon - apply update
+			 */
+
+			dprintf("imsm:prepare space list for update_reshape\n");
+			for (dl = super->devlist; dl;
+			     dl = dl->next) {
+				int size = sizeof_imsm_dev(dl->dev, 1);
+				void *s;
+				if (u->new_raid_disks > u->old_raid_disks)
+					size += sizeof(__u32)*2*
+					(u->new_raid_disks - u->old_raid_disks);
+				s = malloc(size);
+				if (!s)
+					break;
+				*space_tail = s;
+				space_tail = s;
+				*space_tail = NULL;
+			}
+			ret_val = apply_reshape_container_disks_update(u, super,
+								&space_list, 1);
+			if (ret_val) {
+				/* reallocate anchor
+				 */
+				size_t buf_len = super->len;
+				size_t len =
+					disks_to_mpb_size(u->new_raid_disks);
+				struct imsm_super *mpb = super->anchor;
+				void *new_anchor;
+
+				if (__le32_to_cpu(mpb->mpb_size) + len >
+				    buf_len) {
+					buf_len = ROUND_UP(__le32_to_cpu(
+						mpb->mpb_size) + len, 512);
+					if (posix_memalign(&new_anchor,
+							   512, buf_len) == 0) {
+						memcpy(new_anchor, super->buf,
+						       super->len);
+						free(super->buf);
+						super->buf = new_anchor;
+						super->len = buf_len;
+					}
+					super->updates_pending++;
+					ret_val = 0;
+				}
+			} else {
+				while (space_list) {
+					void *space = space_list;
+					space_list = *space_list;
+					free(space);
 				}
-			} else
-				dprintf("imsm: Cannot prepare "\
-					"update\n");
+				free(u);
+			}
 		} else
 			fprintf(stderr, Name "imsm: Operation is not allowed "\
 				"on this container\n");
 	} else
 		dprintf("imsm: not a container operation\n");
 
+exit_imsm_reshape_super:
 	dprintf("imsm: reshape_super Exit code = %i\n", ret_val);
 	return ret_val;
 }

[PATCH 28/34] Take in mind takeover during disk add

[Adam Kwolek]

Reduce disk number of added disks if disk number was increased by takeover.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 Grow.c |   11 +++++++----
 1 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/Grow.c b/Grow.c
index 0b151f1..3814579 100644
--- a/Grow.c
+++ b/Grow.c
@@ -1694,10 +1694,12 @@ static int reshape_array(char *container, int fd, char *devname,
 			st->ss->container_content(st, subarray);
 		struct mdinfo *d;
 
-		if (info2)
+		if (info2) {
+			int before_raid_disks = reshape.before.raid_disks;
+			if (orig_level == 0)
+				before_raid_disks--;
 			for (d = info2->devs; d; d = d->next) {
-				if (d->disk.raid_disk >=
-				    reshape.before.raid_disks) {
+				if (d->disk.raid_disk >= before_raid_disks) {
 					/* This is a spare that wants to
 					 * be part of the array.
 					 */
@@ -1705,7 +1707,8 @@ static int reshape_array(char *container, int fd, char *devname,
 					add_disk(fd, st, info2, d);
 				}
 			}
-		sysfs_free(info2);
+			sysfs_free(info2);
+		}
 	}
 
 	if (reshape.blocks == 0) {

[PATCH 29/34] Detect level change

[Adam Kwolek]

For level migration support it is necessary to allow mdmon to react for level changes.
It has to have ability to change configuration of active array,
and for array level change to raid0 finish array monitoring.

Signed-off-by: Maciej Trela <maciej.trela@intel.com>
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 managemon.c |   12 ++++++++++++
 monitor.c   |    2 +-
 2 files changed, 13 insertions(+), 1 deletions(-)

diff --git a/managemon.c b/managemon.c
index 9cb8049..269d06e 100644
--- a/managemon.c
+++ b/managemon.c
@@ -448,6 +448,18 @@ static void manage_member(struct mdstat_ent *mdstat,
 	else
 		frozen = 1; /* can't read metadata_version assume the worst */
 
+	if (mdstat->level) {
+		int level = map_name(pers, mdstat->level);
+		if (a->info.array.level != level && level >= 0) {
+			struct active_array *newa = duplicate_aa(a);
+			if (newa) {
+				newa->info.array.level = level;
+				replace_array(a->container, a, newa);
+				a = newa;
+			}
+		}
+	}
+
 	if (a->check_degraded && !frozen) {
 		struct metadata_update *updates = NULL;
 		struct mdinfo *newdev = NULL;
diff --git a/monitor.c b/monitor.c
index 65fcba1..937cc6b 100644
--- a/monitor.c
+++ b/monitor.c
@@ -517,7 +517,7 @@ static int wait_and_act(struct supertype *container, int nowait)
 		/* once an array has been deactivated we want to
 		 * ask the manager to discard it.
 		 */
-		if (!a->container) {
+		if (!a->container || (a->info.array.level == 0)) {
 			if (discard_this) {
 				ap = &(*ap)->next;
 				continue;

[PATCH 30/34] Raid0: execute backward takeover

[Adam Kwolek]

After raid0 reshape is finished backward takeover has to be executed.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 Grow.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Grow.c b/Grow.c
index 3814579..c88df1b 100644
--- a/Grow.c
+++ b/Grow.c
@@ -2072,8 +2072,8 @@ static int reshape_array(char *container, int fd, char *devname,
 					sysfs_free(info2);
 				}
 			}
-
-			if (info->new_level != info->array.level) {
+			if ((info->new_level != info->array.level) ||
+			    ((info->new_level == 0) && (orig_level == 0))) {
 				/* We need to wait for the reshape to finish
 				* (which will have happened unless
 				 * odata < ndata) and then set the level

[PATCH 31/34] imsm: FIX: do not repair raid4 arrays

[Adam Kwolek]

As raid4 is not supported by imsm (this is takeovered raid0)
do not fix degraded raid4 arrays.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 1be0e72..b01d52a 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -5432,6 +5432,12 @@ static struct mdinfo *imsm_activate_spare(struct active_array *a,
 		/* No repair during migration */
 		return NULL;
 
+	if (a->info.array.level == 4)
+		/* No repair for takeovered array
+		 * imsm doesn't support raid4
+		 */
+		return NULL;
+
 	if (imsm_check_degraded(super, dev, failed) != IMSM_T_STATE_DEGRADED)
 		return NULL;
 

[PATCH 32/34] Raid0: Run 'next' reshapes without meta update

[Adam Kwolek]

For metadatas that container operation requires set of arrays operations
for 2nd (and next) arrays in raid0 case reshape_array() is executed before
array is monitored and before metadat update is stored,
because in such case mdmon updates metadata via set_array_state().

for such cases some of parameters (i.e. delta_disks) should be used from input parameters,
instead from metadata.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 Grow.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/Grow.c b/Grow.c
index c88df1b..4d12eda 100644
--- a/Grow.c
+++ b/Grow.c
@@ -2128,6 +2128,7 @@ int reshape_container(char *container, int cfd, char *devname,
 {
 	struct mdinfo *cc;
 	struct mdinfo *cc_fresh = NULL;
+	int delta_disks = 0;
 
 	if (reshape_super(st, info->component_size, info->new_level,
 			  info->new_layout, info->new_chunk,
@@ -2184,6 +2185,10 @@ int reshape_container(char *container, int cfd, char *devname,
 		sysfs_free(cc_fresh);
 		st->ss->load_container(st, cfd, NULL);
 		cc_fresh = st->ss->container_content(st, subarray);
+		if (delta_disks && cc_fresh->array.level == 0) {
+			cc_fresh->delta_disks = delta_disks;
+		} else
+			delta_disks = cc_fresh->delta_disks;
 		rv = reshape_array(container, fd, adev, st,
 				   cc_fresh, force,
 				   backup_file, quiet, 1);

[PATCH 33/34] Raid0: Reload disk list on 'next' raid0 array

[Adam Kwolek]

To execute second raid0 array reshape, metadata has to be reloaded after takeover (and update by monitor).
To do this container handle has to be passed to reshape_array() and takeover operation is moved to begin of reshape_array().
For adding disks metadata is reloaded to get added disk list, updated after takeover).

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 Grow.c |   63 ++++++++++++++++++++++++++++++++++-----------------------------
 1 files changed, 34 insertions(+), 29 deletions(-)

diff --git a/Grow.c b/Grow.c
index 4d12eda..ef75b97 100644
--- a/Grow.c
+++ b/Grow.c
@@ -1277,7 +1277,7 @@ char *analyse_change(struct mdinfo *info, struct reshape *re)
 	return NULL;
 }
 
-static int reshape_array(char *container, int fd, char *devname,
+static int reshape_array(char *container, int cfd, int fd, char *devname,
 			 struct supertype *st, struct mdinfo *info,
 			 int force, char *backup_file, int quiet, int forked);
 static int reshape_container(char *container, int cfd, char *devname,
@@ -1594,15 +1594,15 @@ int Grow_reshape(char *devname, int fd, int quiet, char *backup_file,
 			goto release;
 		}
 		sync_metadata(st);
-		rv = reshape_array(container, fd, devname, st, &info, force,
-				   backup_file, quiet, 0);
+		rv = reshape_array(container, cfd, fd, devname, st, &info,
+				   force, backup_file, quiet, 0);
 	}
 release:
 	unfreeze(st, frozen);
 	return rv;
 }
 
-static int reshape_array(char *container, int fd, char *devname,
+static int reshape_array(char *container, int cfd, int fd, char *devname,
 			 struct supertype *st, struct mdinfo *info,
 			 int force,
 			 char *backup_file, int quiet, int forked)
@@ -1637,24 +1637,6 @@ static int reshape_array(char *container, int fd, char *devname,
 		fprintf(stderr, Name ": %s\n", msg);
 		return 1;
 	}
-	if (ioctl(fd, GET_ARRAY_INFO, &array) != 0) {
-		dprintf("Canot get array information.\n");
-		return 1;
-	}
-	spares_needed =  max(reshape.before.raid_disks,
-			     reshape.after.raid_disks) - array.raid_disks;
-
-	if (!force && spares_needed < info->array.spare_disks) {
-		fprintf(stderr,
-			Name ": Need %d spare%s to avoid degraded array,"
-			" and only have %d.\n"
-			"       Use --force to over-ride this check.\n",
-			spares_needed,
-			spares_needed == 1 ? "" : "s", 
-			info->array.spare_disks);
-		return 1;
-	}
-
 	if (reshape.level != info->array.level) {
 		char *c = map_num(pers, reshape.level);
 		int err;
@@ -1676,12 +1658,33 @@ static int reshape_array(char *container, int fd, char *devname,
 			fprintf(stderr, Name " level of %s changed to %s\n",
 				devname, c);	
 		orig_level = info->array.level;
-	}
+		if (mdmon_running(st->container_dev) &&
+		    (orig_level == 0))
+			ping_manager(container);
 
-	if (reshape.level > 0 && st->ss->external &&
-	    !mdmon_running(st->container_dev)) {
-		start_mdmon(st->container_dev);
+		if (reshape.level > 0 && st->ss->external &&
+		    !mdmon_running(st->container_dev))
+			start_mdmon(st->container_dev);
 		ping_monitor(container);
+
+	}
+
+	if (ioctl(fd, GET_ARRAY_INFO, &array) != 0) {
+		dprintf("Canot get array information.\n");
+		return 1;
+	}
+	spares_needed =  max(reshape.before.raid_disks,
+			     reshape.after.raid_disks) - array.raid_disks;
+
+	if (!force && spares_needed < info->array.spare_disks) {
+		fprintf(stderr,
+			Name ": Need %d spare%s to avoid degraded array,"
+			" and only have %d.\n"
+			"       Use --force to over-ride this check.\n",
+			spares_needed,
+			spares_needed == 1 ? "" : "s",
+			info->array.spare_disks);
+		return 1;
 	}
 
 	/* ->reshape_super might have chosen some spares from the
@@ -1690,10 +1693,11 @@ static int reshape_array(char *container, int fd, char *devname,
 	 * them to the kernel.
 	 */
 	if (st->ss->reshape_super && st->ss->container_content) {
-		struct mdinfo *info2 =
-			st->ss->container_content(st, subarray);
+		struct mdinfo *info2;
 		struct mdinfo *d;
 
+		st->ss->load_container(st, cfd, NULL);
+		info2 = st->ss->container_content(st, subarray);
 		if (info2) {
 			int before_raid_disks = reshape.before.raid_disks;
 			if (orig_level == 0)
@@ -2189,7 +2193,8 @@ int reshape_container(char *container, int cfd, char *devname,
 			cc_fresh->delta_disks = delta_disks;
 		} else
 			delta_disks = cc_fresh->delta_disks;
-		rv = reshape_array(container, fd, adev, st,
+
+		rv = reshape_array(container, cfd, fd, adev, st,
 				   cc_fresh, force,
 				   backup_file, quiet, 1);
 		close(fd);

[PATCH 34/34] Raid0: detect reshape on array start

[Adam Kwolek]

When raid0 array is takeovered to raid4 for reshape it should be possible to detect
that array for reshape is monitored now for metadata update.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 monitor.c |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/monitor.c b/monitor.c
index 937cc6b..a0068d3 100644
--- a/monitor.c
+++ b/monitor.c
@@ -235,6 +235,13 @@ static int read_and_act(struct active_array *a)
 		}
 	}
 
+	if (a->curr_state > inactive &&
+	    a->prev_state == inactive) {
+		/* array has been started
+		 * possible that container operation has to be completed
+		 */
+		a->container->ss->set_array_state(a, 0);
+	}
 	if (a->curr_state <= inactive &&
 	    a->prev_state > inactive) {
 		/* array has been stopped */

Re: [PATCH 00/34] OLCE for external metadata (devel3.2)

[NeilBrown]

On Tue, 04 Jan 2011 15:34:38 +0100 Adam Kwolek <adam.kwolek@intel.com> wrote:

> This patch series implements Online Capacity Expansion (OLCE) feature in mdadm devel3.2. for Raid5 and Raid0.
> This is implemented as reshape of whole container (all arrays that belongs to container are reshaped).
> Raid0 reshape is implemented using takeover to raid4 transition.
>
> ---

Thanks.
I have reviewed all of these and applied several, changed other, rejected the
rest as detailed below.

Before doing that I finished the refactoring that I was doing so devel-3.2
compiles again.

My next step with this is to do some testing.

As I say below, feel free to resend anything that you find that you still
need.  Maybe I'll understand it better the second time around, or will be
able to suggest a better way of doing it.

In particular, there is no ->manage_reshape method (I didn't take that patch)
so any attempt to reshape an imsm array will crash.  We need to write a
manage_reshape based on the new child_monitor and using the new
progress_reshape.

BTW, when you split a string e.g.
   "this is a long string"
to
   "this is a"
   " long string"

you do *not* need a '\' at the end of the first line.


Thanks,
NeilBrown


> 
> Adam Kwolek (31):
>       Raid0: detect reshape on array start
Applied, but I'm not sure that it is needed or that set_array_state
does what is expected any more...
>       Raid0: Reload disk list on 'next' raid0 array
Not applied - probably because I am too tired to think clearly
about it.  If you still need this, feel free to resubmit it
and I'll think about it then.
>       Raid0: Run 'next' reshapes without meta update
Not applied.  I don't think the fix is right, but I'm not
100% sure, and it might be fixed by other changes that I have
made.  So feel free to resubmit something like this if it is
still a problem.
>       imsm: FIX: do not repair raid4 arrays
Applied
>       Raid0: execute backward takeover
Applied, but I used a different test which is more generic.
>       Detect level change
Applied
>       Take in mind takeover during disk add
Not applied.  This patch is irrelevant because of an earlier patch
which I didn't apply.
>       imsm: Update raid0 metadata for reshape
Applied, though reshape_super should *both* queue and update and
change the metadata in-place, so I changed it to do that.
>       imsm: Move reshape update processing to function
Applied, though I had to do it by hand - hope I didn't break anything.
However the little extra bit in imsm_reshape_super didn't make sense,
and wasn't documented, so I left it out.
>       Add spares to raid0 in mdadm
Applied with changes.  The Assemble.c part of this was OK.
The super-intel.c was not.  That logic needed to be in Assemble.c
I have put it there (untested).
>       imsm: Update metadata for second array
Applied with a few changes.  I broke the new code out into
a separate function, and then called that function from a
different place - I think a more correct place but you might
like to check.
>       Set array size after adding new disks
Applied, with a few changes.  e.g. 'array_size' can read as 'default'.
And we might want to change array_size when raid_disks decreases as well.
>       imsm: update array size information in metadata
Applied.
>       FIX: Initialize subarray variable in reshape_array
No applied - I fixed this a different way already.
>       imsm: FIX: Division by 0
Applied
>       imsm: Finalize reshape in metadata
Applied
>       Finalize reshape after adding disks to array
Not applied.  This should not be needed as the code near the comment:
		/* Reshape has progressed or completed so we need to
		 * update the array state - and possibly the array size
		 */
should handle this case.  If it doesn't it should be fixed.
>       imsm: FIX: Fill sys_name in container_content()
Not applied.  container_content should not set sys_name as the volume could
easily not have a sys_name yet.  Rather some other code should call
sysfs_init() at an appropriate time.  I have done that.
>       FIX: Use sysfs to change array parameters
Applied
>       FIX: added disks are not used by reshape process /md/
Not applied.  Definitely the wrong fix.  Need more details of the
problem.
>       FIX: change adding disks criteria
Not applied.  I'm not convinced this is the correct fix.
For devices already in the array, disk.state should not be zero.
So if it is, then something else is wrong.   I would need to know more
about that problem you are seeing.
>       FIX: Get array information in reshape_array()
Applied
>       imsm: FIX: support general migration by getinfo_super_imsm_volume
Applied.
>       FIX: Arrays cannot be opened exclusively
Why do you say that arrays cannot be opened exclusively?  You might be right,
but I need mor explanation before I can apply this patch - it "seems" wrong
to me.
Not applied.
>       imsm: FIX: update first array in container only
Applied.  Note that when starting array that is part-way through a reshape
of the first array we will need to alloc space just like we allocate
spare here.
>       FIX: get updated information from metadata
Applied, but with substantial changes.
reshape_super reloads the metadata, not reshape_container, so that
it can know if the array size needs to be changed.
>       imsm: FIX: Do not update anchor directly
Not applied.  I don't think this is correct.  The metadata *should* be
updated directly.
When converting a RAID0 to a RAID5 there is no mdadm running so we should:
  ->reshape_super
  ->sync_super
  start mdmon
  proceed with reshape.

>       imsm: FIX: Perform first metadata update for container operation
Applied.
>       imsm: FIX: display error message
Applied ... and I change the other dprintfs to fprintfs as I think
reshape_super should give an message any time it doesn't succeed.
>       imsm: FIX: display correct information for '-E' option
Applied
>       mdadm: second_map enhancement for imsm_get_map()
Applied.
> 
> Krzysztof Wojcik (3):
>       FIX: Position calculation in mdstat_by_subdev
Applied.
>       FIX: Change size condition in imsm_reshape_is_allowed_on_container
Not applied.  Size of '-1' means 'not change was requested'.  Size of '0'
means 'set to maximum size'.  So the test requiring '-1' seems correct.
>       Manage reshape process in manage_reshape vector.
Not applied.  Seems pointless, and ->manage_reshape is not allowed to say
  "management of process not supported".  If ->reshape_super allows the
  reshape, then ->manage_reshape *must* support it.

> 
> 
>  Grow.c        |  174 ++++++++++++------
>  Manage.c      |   13 -
>  managemon.c   |   12 +
>  mdstat.c      |    4 
>  monitor.c     |   20 ++
>  super-intel.c |  561 ++++++++++++++++++++++++++++++++++++++++++---------------
>  6 files changed, 576 insertions(+), 208 deletions(-)
>