Re: Can't reshape raid0 to raid10

[NeilBrown <neilb@suse.de>]

[-- Attachment #1: Type: text/plain, Size: 4916 bytes --]

On Mon, 29 Dec 2014 22:13:41 -0500 (EST) Xiao Ni <xni@redhat.com> wrote:

> Hi Neil
> 
>    When I try to reshape a raid0 to raid10, it'll fail like this:
> 
> [root@dhcp-12-133 mdadm-3.3.2]# lsblk 
> NAME                    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
> sda                       8:0    0 111.8G  0 disk 
> ├─sda1                    8:1    0  1000M  0 part /boot
> ├─sda2                    8:2    0  29.3G  0 part /
> ├─sda3                    8:3    0   512M  0 part [SWAP]
> ├─sda4                    8:4    0     1K  0 part 
> ├─sda5                    8:5    0   102M  0 part 
> └─sda6                    8:6    0  10.1G  0 part 
>   └─VolGroup00-LogVol00 254:0    0   9.9G  0 lvm  
> sdb                       8:16   0 111.8G  0 disk 
> ├─sdb1                    8:17   0     2G  0 part 
> └─sdb2                    8:18   0    10G  0 part 
> sdc                       8:32   0 186.3G  0 disk 
> ├─sdc1                    8:33   0     2G  0 part 
> └─sdc2                    8:34   0    10G  0 part 
> sdd                       8:48   0 111.8G  0 disk 
> ├─sdd1                    8:49   0     2G  0 part 
> └─sdd2                    8:50   0    10G  0 part 
> [root@dhcp-12-133 mdadm-3.3.2]# mdadm -CR /dev/md0 -l0 -n3 /dev/sdb1 /dev/sdc1 /dev/sdd1 
> mdadm: Defaulting to version 1.2 metadata
> mdadm: array /dev/md0 started.
> [root@dhcp-12-133 mdadm-3.3.2]# mdadm --grow /dev/md0 -l10 -a /dev/sdb2 /dev/sdc2 /dev/sdd2 
> mdadm: level of /dev/md0 changed to raid10
> mdadm: add new device failed for /dev/sdb2 as 6: No space left on device
> 
>    
>    But if I reshape the raid0 to raid5, reshape raid5 to raid0, then reshape raid0 to raid10 use
> the same command it'll succeed. 
> 
> [root@dhcp-12-133 mdadm-3.3.2]# mdadm -CR /dev/md0 -l0 -n3 /dev/sdb1 /dev/sdc1 /dev/sdd1
> [root@dhcp-12-133 mdadm-3.3.2]# mdadm --grow /dev/md0 -l5
> [root@dhcp-12-133 mdadm-3.3.2]# cat /proc/mdstat 
> Personalities : [raid6] [raid5] [raid4] [raid0] [raid10] 
> md0 : active raid5 sdd1[2] sdc1[1] sdb1[0]
>       6285312 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/3] [UUU_]
>       
> unused devices: <none>
> 
> [root@dhcp-12-133 mdadm-3.3.2]# mdadm --grow /dev/md0 -l0
> [root@dhcp-12-133 mdadm-3.3.2]# cat /proc/mdstat 
> Personalities : [raid6] [raid5] [raid4] [raid0] [raid10] 
> md0 : active raid0 sdd1[2] sdc1[1] sdb1[0]
>       6285312 blocks super 1.2 512k chunks
>       
> unused devices: <none>
> [root@dhcp-12-133 mdadm-3.3.2]# mdadm --grow /dev/md0 -l10 -a /dev/sdb2 /dev/sdc2 /dev/sdd2 
> mdadm: level of /dev/md0 changed to raid10
> mdadm: added /dev/sdb2
> mdadm: added /dev/sdc2
> mdadm: added /dev/sdd2
> 
>     So I guess it's the problem add the disk to raid10 after the reshaping. In the function
> super_1_validate, it'll set the  mddev->dev_sectors using the superblock read from disks. 
> If it's raid0, the le64_to_cpu(sb-size) is 0. So when add disk to raid10 bind_rdev_to_array
> return -ENOSPC.
> 
>     When create raid0, it doesn't write give the value to s->size. So the sb-size is 0.
> I modify the code about Create.c. I'm not sure whether it's right to do so. But it can resolve 
> the problem.

Thanks for the report.

I think this need to be fixing in the md driver, when the reshape happens.

Something like the following.

NeilBrown

diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index d1203cddb024..b8d76b1fba64 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -3872,7 +3872,7 @@ static int raid10_resize(struct mddev *mddev, sector_t sectors)
 	return 0;
 }
 
-static void *raid10_takeover_raid0(struct mddev *mddev)
+static void *raid10_takeover_raid0(struct mddev *mddev, sector_t size, int devs)
 {
 	struct md_rdev *rdev;
 	struct r10conf *conf;
@@ -3882,6 +3882,7 @@ static void *raid10_takeover_raid0(struct mddev *mddev)
 		       mdname(mddev));
 		return ERR_PTR(-EINVAL);
 	}
+	sector_div(size, devs);
 
 	/* Set new parameters */
 	mddev->new_level = 10;
@@ -3892,12 +3893,15 @@ static void *raid10_takeover_raid0(struct mddev *mddev)
 	mddev->raid_disks *= 2;
 	/* make sure it will be not marked as dirty */
 	mddev->recovery_cp = MaxSector;
+	mddev->dev_sectors = size;
 
 	conf = setup_conf(mddev);
 	if (!IS_ERR(conf)) {
 		rdev_for_each(rdev, mddev)
-			if (rdev->raid_disk >= 0)
+			if (rdev->raid_disk >= 0) {
 				rdev->new_raid_disk = rdev->raid_disk * 2;
+				rdev->sectors = size;
+			}
 		conf->barrier = 1;
 	}
 
@@ -3920,7 +3924,9 @@ static void *raid10_takeover(struct mddev *mddev)
 			       mdname(mddev));
 			return ERR_PTR(-EINVAL);
 		}
-		return raid10_takeover_raid0(mddev);
+		return raid10_takeover_raid0(mddev,
+			raid0_conf->strip_zone->zone_end,
+			raid0_conf->strip_zone->nb_dev);
 	}
 	return ERR_PTR(-EINVAL);
 }

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]