[PATCH 00/11] Degradation during reshape

[PATCH 00/11] Degradation during reshape

[Adam Kwolek]

The following series implements support for array degradation during reshape.

Series mostly fixes problems in handling degradation during reshape 
in imsm metadata .

Main common problem that last patch resolves is is lack of BBM support.
md on disk failure reports BBM event to user space and waits for an answer.
The side effect of this action is stopping reshape process. The last patch
/together with md patch sent separately/ allows for disabling BBM mechanism.
This is similar as native metadata v0.9 works.

BR
Adam

---

Adam Kwolek (11):
      Disable BBM when it is not supported
      imsm: FIX: Check maximum allowed degradation level in recover_backup_imsm()
      imsm: FIX: Check maximum allowed degradation level in open_backup_targets()
      imsm: FIX: Function rework - imsm_count_failed()
      imsm: FIX: Manage second map state on array degradation
      imsm: FIX: Restore critical section on degraded array
      imsm: FIX: Remove single map state limitation in getinfo
      imsm: FIX: Finalize degraded migration
      imsm: FIX: Do not end migration when missing drive is handled
      imsm: FIX: Mark both maps on degradation while migrating
      imsm: FIX: Return longer map for failure setting


 Assemble.c    |    2 
 Manage.c      |    3 -
 managemon.c   |    3 -
 mdadm.h       |    4 +
 super-intel.c |  323 +++++++++++++++++++++++++++++++++++++++++++++------------
 sysfs.c       |    7 +
 util.c        |    2 
 7 files changed, 268 insertions(+), 76 deletions(-)

-- 
Signature

[PATCH 01/11] imsm: FIX: Return longer map for failure setting

[Adam Kwolek]

When 2 maps are present, IMSM can use shorter map for setting disk
in to degraded state. It can happen that degraded disk can be not present
in shorter map.
We should use longer map for setting disk in to degraded state.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   35 ++++++++++++++++++++++++++---------
 1 files changed, 26 insertions(+), 9 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index a0672bf..5f2c4a2 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -665,18 +665,35 @@ struct imsm_map *get_imsm_map(struct imsm_dev *dev, int second_map)
 	 *    0   - we return the first map
 	 *    1   - we return the second map if it exists, else NULL
 	 *   -1   - we return the second map if it exists, else the first
+	 *   -2   - we return longer map /excluding uninitialized state/
 	 */
 	struct imsm_map *map = &dev->vol.map[0];
+	void *map2 = NULL;
 
-	if (second_map == 1 && !dev->vol.migr_state)
-		return NULL;
-	else if (second_map == 1 ||
-		 (second_map < 0 && dev->vol.migr_state)) {
-		void *ptr = map;
+	if (dev->vol.migr_state)
+		map2 = (void *)map + sizeof_imsm_map(map);
 
-		return ptr + sizeof_imsm_map(map);
-	} else
-		return map;
+	switch (second_map) {
+	case 0:
+		break;
+	case 1:
+		map = map2;
+		break;
+	case -1:
+		if (map2)
+			map = map2;
+		break;
+	case -2:
+		if (map2 &&
+		    (((struct imsm_map *)map2)->num_members > map->num_members)
+		    && (((struct imsm_map *)map2)->map_state !=
+						   IMSM_T_STATE_UNINITIALIZED))
+			map = map2;
+		break;
+	default:
+		map = NULL;
+	}
+	return map;
 
 }
 
@@ -6343,7 +6360,7 @@ static void imsm_set_disk(struct active_array *a, int n, int state)
 
 	dprintf("imsm: set_disk %d:%x\n", n, state);
 
-	ord = get_imsm_ord_tbl_ent(dev, n, -1);
+	ord = get_imsm_ord_tbl_ent(dev, n, -2);
 	disk = get_imsm_disk(super, ord_to_idx(ord));
 
 	/* check for new failures */

Re: [PATCH 01/11] imsm: FIX: Return longer map for failure setting

[Dan Williams]

On Thu, Nov 24, 2011 at 4:17 AM, Adam Kwolek <adam.kwolek@intel.com> wrote:
> When 2 maps are present, IMSM can use shorter map for setting disk
> in to degraded state. It can happen that degraded disk can be not present
> in shorter map.
> We should use longer map for setting disk in to degraded state.

I don't understand this patch.  If we are setting a new degraded disk
migration is idle and the degraded state is set in map[0].

--
Dan

RE: [PATCH 01/11] imsm: FIX: Return longer map for failure setting

[Kwolek, Adam]

> -----Original Message-----
> From: dan.j.williams@gmail.com [mailto:dan.j.williams@gmail.com] On
> Behalf Of Dan Williams
> Sent: Wednesday, November 30, 2011 3:31 AM
> To: Kwolek, Adam
> Cc: neilb@suse.de; linux-raid@vger.kernel.org; Ciechanowski, Ed; Labun,
> Marcin
> Subject: Re: [PATCH 01/11] imsm: FIX: Return longer map for failure setting
> 
> On Thu, Nov 24, 2011 at 4:17 AM, Adam Kwolek <adam.kwolek@intel.com>
> wrote:
> > When 2 maps are present, IMSM can use shorter map for setting disk in
> > to degraded state. It can happen that degraded disk can be not present
> > in shorter map.
> > We should use longer map for setting disk in to degraded state.
> 
> I don't understand this patch.  If we are setting a new degraded disk
> migration is idle and the degraded state is set in map[0].
> 
> --
> Dan

This is made for future in case when  support for shrinking will be added.
When second map is longer than first one and degradation occurs on position that is not present in first map. 
It situation should be stored in map also.

BR
Adam

Re: [PATCH 01/11] imsm: FIX: Return longer map for failure setting

[Williams, Dan J]

On Tue, Nov 29, 2011 at 11:56 PM, Kwolek, Adam <adam.kwolek@intel.com> wrote:
>> I don't understand this patch.  If we are setting a new degraded disk
>> migration is idle and the degraded state is set in map[0].
>>
>> --
>> Dan
>
> This is made for future in case when  support for shrinking will be added.

If it's for a future unimplemented case then this is not a "FIX".

> When second map is longer than first one and degradation occurs on position that is not present in first map.
> It situation should be stored in map also.

What's wrong with using get_imsm_map(dev, 1) for this case?
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 01/11] imsm: FIX: Return longer map for failure setting

[Kwolek, Adam]

> -----Original Message-----
> From: Williams, Dan J [mailto:dan.j.williams@intel.com]
> Sent: Wednesday, November 30, 2011 4:38 PM
> To: Kwolek, Adam
> Cc: neilb@suse.de; linux-raid@vger.kernel.org; Ciechanowski, Ed; Labun,
> Marcin
> Subject: Re: [PATCH 01/11] imsm: FIX: Return longer map for failure setting
> 
> On Tue, Nov 29, 2011 at 11:56 PM, Kwolek, Adam <adam.kwolek@intel.com>
> wrote:
> >> I don't understand this patch.  If we are setting a new degraded disk
> >> migration is idle and the degraded state is set in map[0].
> >>
> >> --
> >> Dan
> >
> > This is made for future in case when  support for shrinking will be added.
> 
> If it's for a future unimplemented case then this is not a "FIX".


Agree

> 
> > When second map is longer than first one and degradation occurs on
> position that is not present in first map.
> > It situation should be stored in map also.
> 
> What's wrong with using get_imsm_map(dev, 1) for this case?

If second map exists and it is shorter than first one, we will put update in not used place.
If decision about selecting longer map will be put outside, the answer is short: nothing.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 02/11] imsm: FIX: Mark both maps on degradation while migrating

[Adam Kwolek]

During migration degradation is set in first map only. This means that
according to second map disk is present. This is not true and not compatible
with OROM behavior.

Set disks in both maps to degraded state during migration.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   20 ++++++++++++++++++++
 1 files changed, 20 insertions(+), 0 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 5f2c4a2..5828922 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -4485,6 +4485,20 @@ static int add_to_super_imsm_volume(struct supertype *st, mdu_disk_info_t *dk,
 			if ((ord & IMSM_ORD_REBUILD) == 0)
 				continue;
 			set_imsm_ord_tbl_ent(map, slot, df->index | IMSM_ORD_REBUILD);
+			if (is_gen_migration(dev)) {
+				struct imsm_map *map2 = get_imsm_map(dev, 1);
+				if (slot < map2->num_members) {
+					__u32 ord2 = get_imsm_ord_tbl_ent(dev,
+									  slot,
+									  1);
+					if ((unsigned)df->index ==
+							       ord_to_idx(ord2))
+						set_imsm_ord_tbl_ent(map2,
+							slot,
+							df->index |
+							IMSM_ORD_REBUILD);
+				}
+			}
 			dprintf("set slot:%d to missing disk:%d\n", slot, df->index);
 			break;
 		}
@@ -6075,6 +6089,12 @@ static int mark_failure(struct imsm_dev *dev, struct imsm_disk *disk, int idx)
 
 	disk->status |= FAILED_DISK;
 	set_imsm_ord_tbl_ent(map, slot, idx | IMSM_ORD_REBUILD);
+	if (is_gen_migration(dev)) {
+		struct imsm_map *map2 = get_imsm_map(dev, 1);
+		if (slot < map2->num_members)
+			set_imsm_ord_tbl_ent(map2, slot,
+					     idx | IMSM_ORD_REBUILD);
+	}
 	if (map->failed_disk_num == 0xff)
 		map->failed_disk_num = slot;
 	return 1;

[PATCH 03/11] imsm: FIX: Do not end migration when missing drive is handled

[Adam Kwolek]

Currently when degradation occurs migration is finalized. This is wrong.
Finalizing migration when it is not finished can lead to data corruption
after next array assembly.

Do not finish migration when degradation occurs.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |    5 -----
 1 files changed, 0 insertions(+), 5 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 5828922..7587c8c 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -6113,17 +6113,12 @@ static void mark_missing(struct imsm_dev *dev, struct imsm_disk *disk, int idx)
 
 static void handle_missing(struct intel_super *super, struct imsm_dev *dev)
 {
-	__u8 map_state;
 	struct dl *dl;
-	int failed;
 
 	if (!super->missing)
 		return;
-	failed = imsm_count_failed(super, dev);
-	map_state = imsm_check_degraded(super, dev, failed);
 
 	dprintf("imsm: mark missing\n");
-	end_migration(dev, map_state);
 	for (dl = super->missing; dl; dl = dl->next)
 		mark_missing(dev, &dl->disk, dl->index);
 	super->updates_pending++;

[PATCH 04/11] imsm: FIX: Finalize degraded migration

[Adam Kwolek]

Currently mdadm for IMSM can finalize not-degraded migration only.
Add support for IMSM  for migration finalization when array
are is degraded state.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   24 ++++++++++++++----------
 1 files changed, 14 insertions(+), 10 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 7587c8c..a3ca78a 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -6417,20 +6417,24 @@ static void imsm_set_disk(struct active_array *a, int n, int state)
 		a->last_checkpoint = 0;
 	} else if (is_gen_migration(dev)) {
 		dprintf("imsm: Detected General Migration in state: ");
-		if (map_state == IMSM_T_STATE_NORMAL) {
-			end_migration(dev, map_state);
+
+		switch (map_state) {
+		case IMSM_T_STATE_NORMAL:
+			dprintf("normal\n");
+			if (a->last_checkpoint >= a->info.component_size)
+				end_migration(dev, map_state);
 			map = get_imsm_map(dev, 0);
 			map->failed_disk_num = ~0;
-			dprintf("normal\n");
-		} else {
-			if (map_state == IMSM_T_STATE_DEGRADED) {
-				printf("degraded\n");
+		break;
+		case IMSM_T_STATE_DEGRADED:
+			dprintf("degraded\n");
+			if (a->last_checkpoint >= a->info.component_size)
 				end_migration(dev, map_state);
-			} else {
-				dprintf("failed\n");
-			}
-			map->map_state = map_state;
+		break;
+		default:
+			dprintf("failed\n");
 		}
+		map->map_state = map_state;
 		super->updates_pending++;
 	}
 }

[PATCH 05/11] imsm: FIX: Remove single map state limitation in getinfo

[Adam Kwolek]

It can occur that degradation during migration occurs on disks that are not
present in both maps /e.g. degradation on just added disk during OLCE/.
This can cause that maps will be in different states (one will be in degraded
and second in normal state). In such situation getinfo_super_imsm_volume()
will not return migration information.

Remove single state limitation in both maps to allow migration information
retrieving.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index a3ca78a..0fc907d 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -2314,8 +2314,7 @@ static void getinfo_super_imsm_volume(struct supertype *st, struct mdinfo *info,
 	info->custom_array_size   |= __le32_to_cpu(dev->size_low);
 	info->recovery_blocked = imsm_reshape_blocks_arrays_changes(st->sb);
 
-	if (prev_map && map->map_state == prev_map->map_state &&
-	    (migr_type(dev) == MIGR_GEN_MIGR)) {
+	if (is_gen_migration(dev)) {
 		info->reshape_active = 1;
 		info->new_level = get_imsm_raid_level(map);
 		info->new_layout = imsm_level_to_layout(info->new_level);

[PATCH 06/11] imsm: FIX: Restore critical section on degraded array

[Adam Kwolek]

When during assembly degradation occurs restoring metadata critical section
fails whole assembly.
Allow for degradation during assembly and not restore data on degraded disk.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   10 +++++++++-
 1 files changed, 9 insertions(+), 1 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 0fc907d..ccd9883 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -7984,6 +7984,10 @@ static void imsm_delete(struct intel_super *super, struct dl **dlp, unsigned ind
 int open_backup_targets(struct mdinfo *info, int raid_disks, int *raid_fds)
 {
 	struct mdinfo *sd;
+	int i;
+
+	for (i = 0; i < raid_disks; i++)
+		raid_fds[i] = -1;
 
 	for (sd = info->devs ; sd ; sd = sd->next) {
 		char *dn;
@@ -8293,7 +8297,11 @@ int recover_backup_imsm(struct supertype *st, struct mdinfo *info)
 	if (!targets)
 		goto abort;
 
-	open_backup_targets(info, new_disks, targets);
+	if (open_backup_targets(info, new_disks, targets)) {
+		fprintf(stderr,
+			Name ": Cannot open devices belongs to array.\n");
+		goto abort;
+	}
 
 	for (i = 0; i < new_disks; i++) {
 		if (targets[i] < 0) {

[PATCH 07/11] imsm: FIX: Manage second map state on array degradation

[Adam Kwolek]

When degradation during migration occurs second map state is not set
to degraded value (map are updated correctly).

Correct second map state according to it's degradation level.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   76 ++++++++++++++++++++++++++++++++++++++++++++-------------
 1 files changed, 58 insertions(+), 18 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index ccd9883..147de36 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -106,6 +106,11 @@ struct imsm_disk {
 	__u32 filler[IMSM_DISK_FILLERS]; /* 0xF4 - 0x107 MPB_DISK_FILLERS for future expansion */
 };
 
+/* map selector for map managment
+ */
+#define MAP_0		2
+#define MAP_1		4
+
 /* RAID map configuration infos. */
 struct imsm_map {
 	__u32 pba_of_lba0;	/* start address of partition */
@@ -662,7 +667,9 @@ struct imsm_map *get_imsm_map(struct imsm_dev *dev, int second_map)
 {
 	/* A device can have 2 maps if it is in the middle of a migration.
 	 * If second_map is:
+	 *    MAP_0 and
 	 *    0   - we return the first map
+	 *    MAP_1 and
 	 *    1   - we return the second map if it exists, else NULL
 	 *   -1   - we return the second map if it exists, else the first
 	 *   -2   - we return longer map /excluding uninitialized state/
@@ -674,8 +681,10 @@ struct imsm_map *get_imsm_map(struct imsm_dev *dev, int second_map)
 		map2 = (void *)map + sizeof_imsm_map(map);
 
 	switch (second_map) {
+	case MAP_0:
 	case 0:
 		break;
+	case MAP_1:
 	case 1:
 		map = map2;
 		break;
@@ -2485,8 +2494,28 @@ static void getinfo_super_imsm_volume(struct supertype *st, struct mdinfo *info,
 	}
 }
 
-static __u8 imsm_check_degraded(struct intel_super *super, struct imsm_dev *dev, int failed);
-static int imsm_count_failed(struct intel_super *super, struct imsm_dev *dev);
+static __u8 imsm_check_degraded(struct intel_super *super, struct imsm_dev *dev,
+				int failed, int look_in_map);
+
+static int imsm_count_failed(struct intel_super *super, struct imsm_dev *dev,
+			     int look_in_map);
+
+static void manage_second_map(struct intel_super *super, struct imsm_dev *dev)
+{
+	if (is_gen_migration(dev)) {
+		int failed;
+		__u8 map_state;
+		struct imsm_map *map2 = get_imsm_map(dev, MAP_1);
+
+		failed = imsm_count_failed(super, dev, MAP_1);
+		map_state = imsm_check_degraded(super, dev, failed,
+						MAP_1);
+		if (map2->map_state != map_state) {
+			map2->map_state = map_state;
+			super->updates_pending++;
+		}
+	}
+}
 
 static struct imsm_disk *get_imsm_missing(struct intel_super *super, __u8 index)
 {
@@ -2547,8 +2576,8 @@ static void getinfo_super_imsm(struct supertype *st, struct mdinfo *info, char *
 		struct imsm_map *map;
 		__u8 state;
 
-		failed = imsm_count_failed(super, dev);
-		state = imsm_check_degraded(super, dev, failed);
+		failed = imsm_count_failed(super, dev, MAP_0);
+		state = imsm_check_degraded(super, dev, failed, MAP_0);
 		map = get_imsm_map(dev, 0);
 
 		/* any newly missing disks?
@@ -5914,9 +5943,12 @@ static struct mdinfo *container_content_imsm(struct supertype *st, char *subarra
 }
 
 
-static __u8 imsm_check_degraded(struct intel_super *super, struct imsm_dev *dev, int failed)
+static __u8 imsm_check_degraded(struct intel_super *super, struct imsm_dev *dev,
+				int failed, int look_in_map)
 {
-	struct imsm_map *map = get_imsm_map(dev, 0);
+	struct imsm_map *map;
+
+	map = get_imsm_map(dev, look_in_map);
 
 	if (!failed)
 		return map->map_state == IMSM_T_STATE_UNINITIALIZED ? 
@@ -5980,7 +6012,8 @@ static __u8 imsm_check_degraded(struct intel_super *super, struct imsm_dev *dev,
 	return map->map_state;
 }
 
-static int imsm_count_failed(struct intel_super *super, struct imsm_dev *dev)
+static int imsm_count_failed(struct intel_super *super, struct imsm_dev *dev,
+			     int look_in_map)
 {
 	int i;
 	int failed = 0;
@@ -5999,14 +6032,15 @@ static int imsm_count_failed(struct intel_super *super, struct imsm_dev *dev)
 	 */
 	map_for_loop = prev;
 	if (is_gen_migration(dev))
-		if (prev && (map->num_members > prev->num_members))
+		if (prev &&
+		    (map->num_members > prev->num_members))
 			map_for_loop = map;
 
 	for (i = 0; i < map_for_loop->num_members; i++) {
 		ord = 0;
-		if (i < prev->num_members)
+		if ((look_in_map & MAP_1) && (i < prev->num_members))
 			ord |= __le32_to_cpu(prev->disk_ord_tbl[i]);
-		if (i < map->num_members)
+		if ((look_in_map & MAP_0) && (i < map->num_members))
 			ord |= __le32_to_cpu(map->disk_ord_tbl[i]);
 		idx = ord_to_idx(ord);
 
@@ -6216,8 +6250,8 @@ static int imsm_set_array_state(struct active_array *a, int consistent)
 	struct intel_super *super = a->container->sb;
 	struct imsm_dev *dev = get_imsm_dev(super, inst);
 	struct imsm_map *map = get_imsm_map(dev, 0);
-	int failed = imsm_count_failed(super, dev);
-	__u8 map_state = imsm_check_degraded(super, dev, failed);
+	int failed = imsm_count_failed(super, dev, MAP_0);
+	__u8 map_state = imsm_check_degraded(super, dev, failed, MAP_0);
 	__u32 blocks_per_unit;
 
 	if (dev->vol.migr_state &&
@@ -6391,8 +6425,8 @@ static void imsm_set_disk(struct active_array *a, int n, int state)
 		super->updates_pending++;
 	}
 
-	failed = imsm_count_failed(super, dev);
-	map_state = imsm_check_degraded(super, dev, failed);
+	failed = imsm_count_failed(super, dev, MAP_0);
+	map_state = imsm_check_degraded(super, dev, failed, MAP_0);
 
 	/* check if recovery complete, newly degraded, or failed */
 	if (map_state == IMSM_T_STATE_NORMAL && is_rebuilding(dev)) {
@@ -6429,6 +6463,8 @@ static void imsm_set_disk(struct active_array *a, int n, int state)
 			dprintf("degraded\n");
 			if (a->last_checkpoint >= a->info.component_size)
 				end_migration(dev, map_state);
+			else
+				manage_second_map(super, dev);
 		break;
 		default:
 			dprintf("failed\n");
@@ -6625,7 +6661,8 @@ static int imsm_rebuild_allowed(struct supertype *cont, int dev_idx, int failed)
 
 	dev2 = get_imsm_dev(cont->sb, dev_idx);
 	if (dev2) {
-		state = imsm_check_degraded(cont->sb, dev2, failed);
+		state = imsm_check_degraded(cont->sb, dev2, failed,
+					    MAP_0);
 		if (state == IMSM_T_STATE_FAILED) {
 			map = get_imsm_map(dev2, 0);
 			if (!map)
@@ -6702,7 +6739,8 @@ static struct mdinfo *imsm_activate_spare(struct active_array *a,
 		 */
 		return NULL;
 
-	if (imsm_check_degraded(super, dev, failed) != IMSM_T_STATE_DEGRADED)
+	if (imsm_check_degraded(super, dev, failed, MAP_0) !=
+			IMSM_T_STATE_DEGRADED)
 		return NULL;
 
 	/*
@@ -7109,7 +7147,8 @@ static int apply_update_activate_spare(struct imsm_update_activate_spare *u,
 		disk->status &= ~SPARE_DISK;
 
 		/* mark rebuild */
-		to_state = imsm_check_degraded(super, dev, failed);
+		to_state = imsm_check_degraded(super, dev, failed,
+					       MAP_0);
 		if (!second_map_created) {
 			second_map_created = 1;
 			map->map_state = IMSM_T_STATE_DEGRADED;
@@ -7294,7 +7333,8 @@ static int apply_takeover_update(struct imsm_update_takeover *u,
 
 	if (u->direction == R10_TO_R0) {
 		/* Number of failed disks must be half of initial disk number */
-		if (imsm_count_failed(super, dev) != (map->num_members / 2))
+		if (imsm_count_failed(super, dev, MAP_0) !=
+				(map->num_members / 2))
 			return 0;
 
 		/* iterate through devices to mark removed disks as spare */

[PATCH 08/11] imsm: FIX: Function rework - imsm_count_failed()

[Adam Kwolek]

imsm_count_failed() assumes that on the same positions in both maps
the same disk indexes are kept. This is not always true /e.g. rebuild/
It can occur that disk taken for rebuild fails at once.
Degradation on the same positions in both maps refers to different disks.
Sum of both ords can point on not failed disk. This can cause wrong
failed disk counting.

Check both maps independently. This allows for getting real degradation
information.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   43 +++++++++++++++++++++++++++----------------
 1 files changed, 27 insertions(+), 16 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 147de36..3e7cfb4 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -6018,11 +6018,12 @@ static int imsm_count_failed(struct intel_super *super, struct imsm_dev *dev,
 	int i;
 	int failed = 0;
 	struct imsm_disk *disk;
-	struct imsm_map *map = get_imsm_map(dev, 0);
-	struct imsm_map *prev = get_imsm_map(dev, dev->vol.migr_state);
+	struct imsm_map *map = get_imsm_map(dev, MAP_0);
+	struct imsm_map *prev = get_imsm_map(dev, MAP_1);
 	struct imsm_map *map_for_loop;
 	__u32 ord;
 	int idx;
+	int idx_1;
 
 	/* at the beginning of migration we set IMSM_ORD_REBUILD on
 	 * disks that are being rebuilt.  New failures are recorded to
@@ -6030,23 +6031,33 @@ static int imsm_count_failed(struct intel_super *super, struct imsm_dev *dev,
 	 * see if any failures are still present, or if any new ones
 	 * have arrived
 	 */
-	map_for_loop = prev;
-	if (is_gen_migration(dev))
-		if (prev &&
-		    (map->num_members > prev->num_members))
-			map_for_loop = map;
+	map_for_loop = map;
+	if (prev &&
+	    (map->num_members < prev->num_members))
+		map_for_loop = prev;
 
 	for (i = 0; i < map_for_loop->num_members; i++) {
-		ord = 0;
-		if ((look_in_map & MAP_1) && (i < prev->num_members))
-			ord |= __le32_to_cpu(prev->disk_ord_tbl[i]);
-		if ((look_in_map & MAP_0) && (i < map->num_members))
-			ord |= __le32_to_cpu(map->disk_ord_tbl[i]);
-		idx = ord_to_idx(ord);
+		idx_1 = -255;
+		if (prev &&
+		    (look_in_map & MAP_1) && (i < prev->num_members)) {
+			ord = __le32_to_cpu(prev->disk_ord_tbl[i]);
+			idx_1 = ord_to_idx(ord);
 
-		disk = get_imsm_disk(super, idx);
-		if (!disk || is_failed(disk) || ord & IMSM_ORD_REBUILD)
-			failed++;
+			disk = get_imsm_disk(super, idx_1);
+			if (!disk || is_failed(disk) || ord & IMSM_ORD_REBUILD)
+				failed++;
+		}
+		if ((look_in_map & MAP_0) && (i < map->num_members)) {
+			ord = __le32_to_cpu(map->disk_ord_tbl[i]);
+			idx = ord_to_idx(ord);
+
+			if (idx != idx_1) {
+				disk = get_imsm_disk(super, idx);
+				if (!disk || is_failed(disk) ||
+				    ord & IMSM_ORD_REBUILD)
+					failed++;
+			}
+		}
 	}
 
 	return failed;

[PATCH 09/11] imsm: FIX: Check maximum allowed degradation level in open_backup_targets()

[Adam Kwolek]

Any degradation during opening any backup device can causes error
and array assembly failure.

Allow for degradation during opening backup devices.
This allows for degraded array assembly.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   97 +++++++++++++++++++++++++++++++++++++++++++++++++++++----
 1 files changed, 90 insertions(+), 7 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 3e7cfb4..e2b3c05 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -8020,6 +8020,75 @@ static void imsm_delete(struct intel_super *super, struct dl **dlp, unsigned ind
 	}
 }
 #endif /* MDASSEMBLE */
+
+static void close_targets(int *targets, int new_disks)
+{
+	int i;
+
+	if (!targets)
+		return;
+
+	for (i = 0; i < new_disks; i++) {
+		if (targets[i] >= 0) {
+			close(targets[i]);
+			targets[i] = -1;
+		}
+	}
+}
+
+static int imsm_get_allowed_degradation(int level, int raid_disks,
+					struct intel_super *super,
+					struct imsm_dev *dev)
+{
+	switch (level) {
+	case 10:{
+		int ret_val = 0;
+		struct imsm_map *map;
+		int i;
+
+		ret_val = raid_disks/2;
+		/* check map if all disks pairs not failed
+		 * in both maps
+		 */
+		map = get_imsm_map(dev, 0);
+		for (i = 0; i < ret_val; i++) {
+			int degradation = 0;
+			if (get_imsm_disk(super, i) == NULL)
+				degradation++;
+			if (get_imsm_disk(super, i + 1) == NULL)
+				degradation++;
+			if (degradation == 2)
+				return 0;
+		}
+		map = get_imsm_map(dev, 1);
+		/* if there is no second map
+		 * result can be returned
+		 */
+		if (map == NULL)
+			return ret_val;
+		/* check degradation in second map
+		 */
+		for (i = 0; i < ret_val; i++) {
+			int degradation = 0;
+		if (get_imsm_disk(super, i) == NULL)
+				degradation++;
+			if (get_imsm_disk(super, i + 1) == NULL)
+				degradation++;
+			if (degradation == 2)
+				return 0;
+		}
+		return ret_val;
+	}
+	case 5:
+		return 1;
+	case 6:
+		return 2;
+	default:
+		return 0;
+	}
+}
+
+
 /*******************************************************************************
  * Function:	open_backup_targets
  * Description:	Function opens file descriptors for all devices given in
@@ -8028,14 +8097,18 @@ static void imsm_delete(struct intel_super *super, struct dl **dlp, unsigned ind
  *	info		: general array info
  *	raid_disks	: number of disks
  *	raid_fds	: table of device's file descriptors
+ *	super		: intel super for raid10 degradation check
+ *	dev		: intel device for raid10 degradation check
  * Returns:
  *	 0 : success
  *	-1 : fail
  ******************************************************************************/
-int open_backup_targets(struct mdinfo *info, int raid_disks, int *raid_fds)
+int open_backup_targets(struct mdinfo *info, int raid_disks, int *raid_fds,
+			struct intel_super *super, struct imsm_dev *dev)
 {
 	struct mdinfo *sd;
 	int i;
+	int opened = 0;
 
 	for (i = 0; i < raid_disks; i++)
 		raid_fds[i] = -1;
@@ -8057,8 +8130,19 @@ int open_backup_targets(struct mdinfo *info, int raid_disks, int *raid_fds)
 		raid_fds[sd->disk.raid_disk] = dev_open(dn, O_RDWR);
 		if (raid_fds[sd->disk.raid_disk] < 0) {
 			fprintf(stderr, "cannot open component\n");
-			return -1;
+			continue;
 		}
+		opened++;
+	}
+	/* check if maximum array degradation level is not exceeded
+	*/
+	if ((raid_disks - opened) >
+			imsm_get_allowed_degradation(info->new_level,
+						     raid_disks,
+						     super, dev)) {
+		fprintf(stderr, "Not enough disks can be opened.\n");
+		close_targets(raid_fds, raid_disks);
+		return -2;
 	}
 	return 0;
 }
@@ -8191,7 +8275,8 @@ int save_backup_imsm(struct supertype *st,
 		target_offsets[i] -= start/data_disks;
 	}
 
-	if (open_backup_targets(info, new_disks, targets))
+	if (open_backup_targets(info, new_disks, targets,
+				super, dev))
 		goto abort;
 
 	dest_layout = imsm_level_to_layout(map_dest->raid_level);
@@ -8217,9 +8302,7 @@ int save_backup_imsm(struct supertype *st,
 
 abort:
 	if (targets) {
-		for (i = 0; i < new_disks; i++)
-			if (targets[i] >= 0)
-				close(targets[i]);
+		close_targets(targets, new_disks);
 		free(targets);
 	}
 	free(target_offsets);
@@ -8348,7 +8431,7 @@ int recover_backup_imsm(struct supertype *st, struct mdinfo *info)
 	if (!targets)
 		goto abort;
 
-	if (open_backup_targets(info, new_disks, targets)) {
+	if (open_backup_targets(info, new_disks, targets, super, id->dev)) {
 		fprintf(stderr,
 			Name ": Cannot open devices belongs to array.\n");
 		goto abort;

[PATCH 10/11] imsm: FIX: Check maximum allowed degradation level in recover_backup_imsm()

[Adam Kwolek]

Any degradation during backup recovery causes error and array assembly
failure.

Allow for degradation during backup recovery.
This allows for degraded array assembly.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 super-intel.c |   19 ++++++++++++-------
 1 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index e2b3c05..208af80 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -8391,7 +8391,6 @@ int recover_backup_imsm(struct supertype *st, struct mdinfo *info)
 	unsigned long num_migr_units = __le32_to_cpu(migr_rec->num_migr_units);
 	char buffer[20];
 	int skipped_disks = 0;
-	int max_degradation;
 
 	err = sysfs_get_str(info, NULL, "array_state", (char *)buffer, 20);
 	if (err < 1)
@@ -8415,7 +8414,6 @@ int recover_backup_imsm(struct supertype *st, struct mdinfo *info)
 
 	map_dest = get_imsm_map(id->dev, 0);
 	new_disks = map_dest->num_members;
-	max_degradation = new_disks - imsm_num_data_members(id->dev, 0);
 
 	read_offset = (unsigned long long)
 			__le32_to_cpu(migr_rec->ckpt_area_pba) * 512;
@@ -8446,29 +8444,36 @@ int recover_backup_imsm(struct supertype *st, struct mdinfo *info)
 			fprintf(stderr,
 				Name ": Cannot seek to block: %s\n",
 				strerror(errno));
-			goto abort;
+			skipped_disks++;
+			continue;
 		}
 		if ((unsigned)read(targets[i], buf, unit_len) != unit_len) {
 			fprintf(stderr,
 				Name ": Cannot read copy area block: %s\n",
 				strerror(errno));
-			goto abort;
+			skipped_disks++;
+			continue;
 		}
 		if (lseek64(targets[i], write_offset, SEEK_SET) < 0) {
 			fprintf(stderr,
 				Name ": Cannot seek to block: %s\n",
 				strerror(errno));
-			goto abort;
+			skipped_disks++;
+			continue;
 		}
 		if ((unsigned)write(targets[i], buf, unit_len) != unit_len) {
 			fprintf(stderr,
 				Name ": Cannot restore block: %s\n",
 				strerror(errno));
-			goto abort;
+			skipped_disks++;
+			continue;
 		}
 	}
 
-	if (skipped_disks > max_degradation) {
+	if (skipped_disks > imsm_get_allowed_degradation(info->new_level,
+							 new_disks,
+							 super,
+							 id->dev)) {
 		fprintf(stderr,
 			Name ": Cannot restore data from backup."
 			" Too many failed disks\n");

[PATCH 11/11] Disable BBM when it is not supported

[Adam Kwolek]

During reshape for external metadata any disk failure /e.g. disk removing/
is reported to user space via sysfs and requires to be confirmed by mdadm
to allow reshape to proceed. When mdadm for given external metadata doesn't
support BBM reshape continuation is not possible.

To resolve this situation external metadata can work as native metadata
v0.90 and direct md to ignore bad blocks management.

To achieve this, patch in md /provided separately/ for setting badblocks.shift
field to '-1' value per rdev device by storing in sysfs 'disable' word
is used.

To use this md ability bbm_enabled enabled field is added to super switch.
According to this field during disk add /in sysfs_add_disk()/ bad blocks
management is disabled in md.

Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
---

 Assemble.c    |    2 +-
 Manage.c      |    3 ++-
 managemon.c   |    3 ++-
 mdadm.h       |    4 +++-
 super-intel.c |    1 +
 sysfs.c       |    7 ++++++-
 util.c        |    2 +-
 7 files changed, 16 insertions(+), 6 deletions(-)

diff --git a/Assemble.c b/Assemble.c
index 74708d1..88af585 100644
--- a/Assemble.c
+++ b/Assemble.c
@@ -1542,7 +1542,7 @@ int assemble_container_content(struct supertype *st, int mdfd,
 		sysfs_free(sra);
 	old_raid_disks = content->array.raid_disks - content->delta_disks;
 	for (dev = content->devs; dev; dev = dev->next)
-		if (sysfs_add_disk(content, dev, 1) == 0) {
+		if (sysfs_add_disk(content, dev, 1, st->ss->bbm_enabled) == 0) {
 			if (dev->disk.raid_disk >= old_raid_disks &&
 			    content->reshape_active)
 				expansion++;
diff --git a/Manage.c b/Manage.c
index d9775de..d3ea140 100644
--- a/Manage.c
+++ b/Manage.c
@@ -998,7 +998,8 @@ int Manage_subdevs(char *devname, int fd,
 				/* Make sure fds are closed as they are O_EXCL which
 				 * would block add_disk */
 				tst->ss->free_super(tst);
-				if (sysfs_add_disk(sra, &new_mdi, 0) != 0) {
+				if (sysfs_add_disk(sra, &new_mdi, 0,
+						   tst->ss->bbm_enabled) != 0) {
 					fprintf(stderr, Name ": add new device to external metadata"
 						" failed for %s\n", dv->devname);
 					close(container_fd);
diff --git a/managemon.c b/managemon.c
index cde0d8b..6d24bbc 100644
--- a/managemon.c
+++ b/managemon.c
@@ -522,7 +522,8 @@ static void manage_member(struct mdstat_ent *mdstat,
 			newd = malloc(sizeof(*newd));
 			if (!newd)
 				continue;
-			if (sysfs_add_disk(&newa->info, d, 0) < 0) {
+			if (sysfs_add_disk(&newa->info, d, 0,
+					   container->ss->bbm_enabled) < 0) {
 				free(newd);
 				continue;
 			}
diff --git a/mdadm.h b/mdadm.h
index 1351d42..5e6b9a7 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -486,7 +486,8 @@ extern int sysfs_get_str(struct mdinfo *sra, struct mdinfo *dev,
 			 char *name, char *val, int size);
 extern int sysfs_set_safemode(struct mdinfo *sra, unsigned long ms);
 extern int sysfs_set_array(struct mdinfo *info, int vers);
-extern int sysfs_add_disk(struct mdinfo *sra, struct mdinfo *sd, int resume);
+extern int sysfs_add_disk(struct mdinfo *sra, struct mdinfo *sd,
+			  int resume, int bbm_enabled);
 extern int sysfs_disk_to_scsi_id(int fd, __u32 *id);
 extern int sysfs_unique_holder(int devnum, long rdev);
 extern int sysfs_freeze_array(struct mdinfo *sra);
@@ -812,6 +813,7 @@ extern struct superswitch {
 
 	int swapuuid; /* true if uuid is bigending rather than hostendian */
 	int external;
+	int bbm_enabled;
 	const char *name; /* canonical metadata name */
 } *superlist[];
 
diff --git a/super-intel.c b/super-intel.c
index 208af80..f17347f 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -9511,6 +9511,7 @@ struct superswitch super_imsm = {
 
 	.external	= 1,
 	.name = "imsm",
+	.bbm_enabled = 0,
 
 #ifndef MDASSEMBLE
 /* for mdmon */
diff --git a/sysfs.c b/sysfs.c
index d923f7f..06f1c16 100644
--- a/sysfs.c
+++ b/sysfs.c
@@ -605,7 +605,8 @@ int sysfs_set_array(struct mdinfo *info, int vers)
 	return rv;
 }
 
-int sysfs_add_disk(struct mdinfo *sra, struct mdinfo *sd, int resume)
+int sysfs_add_disk(struct mdinfo *sra, struct mdinfo *sd, int resume,
+		  int bbm_enabled)
 {
 	char dv[PATH_MAX];
 	char nm[PATH_MAX];
@@ -648,6 +649,10 @@ int sysfs_add_disk(struct mdinfo *sra, struct mdinfo *sd, int resume)
 		if (resume)
 			sysfs_set_num(sra, sd, "recovery_start", sd->recovery_start);
 	}
+	/* disable bad blocks managment in md
+	 */
+	if (bbm_enabled == 0)
+		rv = sysfs_set_str(sra, sd, "bad_blocks", "disable");
 	return rv;
 }
 
diff --git a/util.c b/util.c
index 6985a70..8645daf 100644
--- a/util.c
+++ b/util.c
@@ -1475,7 +1475,7 @@ int add_disk(int mdfd, struct supertype *st,
 			info->recovery_start = MaxSector;
 		else
 			info->recovery_start = 0;
-		rv = sysfs_add_disk(sra, info, 0);
+		rv = sysfs_add_disk(sra, info, 0, st->ss->bbm_enabled);
 		if (! rv) {
 			struct mdinfo *sd2;
 			for (sd2 = sra->devs; sd2; sd2=sd2->next)

Re: [PATCH 00/11] Degradation during reshape

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2512 bytes --]

On Thu, 24 Nov 2011 13:17:10 +0100 Adam Kwolek <adam.kwolek@intel.com> wrote:

> The following series implements support for array degradation during reshape.
> 
> Series mostly fixes problems in handling degradation during reshape 
> in imsm metadata .
> 
> Main common problem that last patch resolves is is lack of BBM support.
> md on disk failure reports BBM event to user space and waits for an answer.
> The side effect of this action is stopping reshape process. The last patch
> /together with md patch sent separately/ allows for disabling BBM mechanism.
> This is similar as native metadata v0.9 works.
> 
> BR
> Adam
> 
>

Sorry for the long delay in getting to these - I've been busy :-(

> 
> Adam Kwolek (11):
>       Disable BBM when it is not supported

Not applied as discussed separately.  I'll follow up on this issue separately.

>       imsm: FIX: Check maximum allowed degradation level in recover_backup_imsm()
>       imsm: FIX: Check maximum allowed degradation level in open_backup_targets()
>       imsm: FIX: Function rework - imsm_count_failed()

These 3 applied.

>       imsm: FIX: Manage second map state on array degradation

I've applied this, but I don't like the fact that you have used '2' and '4'
for MAP_0 and MAP_1.
I see that you use '&' to test a bit and you wanted separate bits, but I
don't see any place where "look_in_map" could have multiple bits set.
So why not MAP_0==0 and MAP_1==1 and use e.g. "look_in_map == MAP_0".

I'm quite happy with defining the symbolic named (MAP_0 and MAP_1),
just confused by the values chosen.

Could you please explain the logic, or fix it up with a new patch?  Thanks.

>       imsm: FIX: Restore critical section on degraded array
>       imsm: FIX: Remove single map state limitation in getinfo
>       imsm: FIX: Finalize degraded migration
>       imsm: FIX: Do not end migration when missing drive is handled

These 4 applied.

>       imsm: FIX: Mark both maps on degradation while migrating

applied, but I think mark_failure might still be wrong.
What if the device that fails is in MAP_1 but not in MAP_0?  I don't think it
gets marked as failed in that case.

>       imsm: FIX: Return longer map for failure setting

I changed the type of 'map2' from 'void *' to 'struct imsm_map *', and applied the result.

Thanks.

BTW if you want these in SLES11-SP2 I'll need a request through bugzilla.
Just one is enough, not one for every patch.

NeilBrown


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

RE: [PATCH 00/11] Degradation during reshape

[Kwolek, Adam]

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
> owner@vger.kernel.org] On Behalf Of NeilBrown
> Sent: Tuesday, December 06, 2011 1:54 AM
> To: Kwolek, Adam
> Cc: linux-raid@vger.kernel.org; Ciechanowski, Ed; Labun, Marcin; Williams,
> Dan J
> Subject: Re: [PATCH 00/11] Degradation during reshape
> 
> On Thu, 24 Nov 2011 13:17:10 +0100 Adam Kwolek
> <adam.kwolek@intel.com> wrote:
> 
> > The following series implements support for array degradation during
> reshape.
> >
> > Series mostly fixes problems in handling degradation during reshape in
> > imsm metadata .
> >
> > Main common problem that last patch resolves is is lack of BBM support.
> > md on disk failure reports BBM event to user space and waits for an
> answer.
> > The side effect of this action is stopping reshape process. The last
> > patch /together with md patch sent separately/ allows for disabling BBM
> mechanism.
> > This is similar as native metadata v0.9 works.
> >
> > BR
> > Adam
> >
> >
> 
> Sorry for the long delay in getting to these - I've been busy :-(
> 
> >
> > Adam Kwolek (11):
> >       Disable BBM when it is not supported
> 
> Not applied as discussed separately.  I'll follow up on this issue separately.
> 
> >       imsm: FIX: Check maximum allowed degradation level in
> recover_backup_imsm()
> >       imsm: FIX: Check maximum allowed degradation level in
> open_backup_targets()
> >       imsm: FIX: Function rework - imsm_count_failed()
> 
> These 3 applied.
> 
> >       imsm: FIX: Manage second map state on array degradation
> 
> I've applied this, but I don't like the fact that you have used '2' and '4'
> for MAP_0 and MAP_1.
> I see that you use '&' to test a bit and you wanted separate bits, but I don't
> see any place where "look_in_map" could have multiple bits set.
> So why not MAP_0==0 and MAP_1==1 and use e.g. "look_in_map ==
> MAP_0".
> 
> I'm quite happy with defining the symbolic named (MAP_0 and MAP_1), just
> confused by the values chosen.
> 
> Could you please explain the logic, or fix it up with a new patch?  Thanks.

Using 'bits' gives ability to test in both maps at a time. 
Now I've got this series a 'little' reworked so I try to remove it.

> 
> >       imsm: FIX: Restore critical section on degraded array
> >       imsm: FIX: Remove single map state limitation in getinfo
> >       imsm: FIX: Finalize degraded migration
> >       imsm: FIX: Do not end migration when missing drive is handled
> 
> These 4 applied.
> 
> >       imsm: FIX: Mark both maps on degradation while migrating
> 
> applied, but I think mark_failure might still be wrong.
> What if the device that fails is in MAP_1 but not in MAP_0?  I don't think it
> gets marked as failed in that case.

Currently there is not disk replace functionality, so this case is not possible.
If something is failed in MAP_1 this means that it is marked as failed in MAP_0 already
Or it is rebuild of this failed disk, but I'll think about preparing it for the future.



> 
> >       imsm: FIX: Return longer map for failure setting
> 
> I changed the type of 'map2' from 'void *' to 'struct imsm_map *', and
> applied the result.
> 
> Thanks.
> 
> BTW if you want these in SLES11-SP2 I'll need a request through bugzilla.
> Just one is enough, not one for every patch.

It will be nice:) I'll do this when whole solution will be in place.

BR
Adam

> 
> NeilBrown