From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Evans Subject: Re: Removing drives Date: Sun, 7 Mar 2010 12:49:52 -0800 Message-ID: <4877c76c1003071249y69368e2bt2f2c70dd0274df8@mail.gmail.com> References: <4B8F26AF.7080308@vorgon.com> <4877c76c1003041445g2ac9aaf6ua5fac64db975da1@mail.gmail.com> <4B9154C0.7060200@vorgon.com> <4877c76c1003051222i584bdb19r65002981060a6360@mail.gmail.com> <4B91DF69.5030804@vorgon.com> <4877c76c1003052243r790e0528v71efe38038bfafeb@mail.gmail.com> <4B934FB2.3080104@vorgon.com> <4877c76c1003062333r13d83d6atd85d004a56218c7a@mail.gmail.com> <4B93F209.9030901@vorgon.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Return-path: In-Reply-To: <4B93F209.9030901@vorgon.com> Sender: linux-raid-owner@vger.kernel.org To: "Timothy D. Lenz" Cc: linux-raid@vger.kernel.org List-Id: linux-raid.ids On Sun, Mar 7, 2010 at 10:35 AM, Timothy D. Lenz wro= te: > > > On 3/7/2010 12:33 AM, Michael Evans wrote: >> >> On Sat, Mar 6, 2010 at 11:03 PM, Timothy D. Lenz =A0= wrote: >>> >>> >>> On 3/5/2010 11:43 PM, Michael Evans wrote: >>>> >>>> On Fri, Mar 5, 2010 at 8:51 PM, Timothy D. Lenz >>>> =A0wrote: >>>>> >>>>> >>>>> On 3/5/2010 1:22 PM, Michael Evans wrote: >>>>>> >>>>>> On Fri, Mar 5, 2010 at 11:00 AM, Timothy D. Lenz >>>>>> =A0wrote: >>>>>>> >>>>>>> Current setup, 3 500gb sata drives, each with 3 partitions. >>>>>>> The first partition of each drive make up raid1 md0 boot and mo= st >>>>>>> software >>>>>>> The next partition of each drive make up raid1 md1 swap >>>>>>> The 3rd partition of each drive make up raid5 md2 main data sto= rage >>>>>>> >>>>>>> There is also a 40gb ide drive with 2 partitions, boot/software= and >>>>>>> swap. >>>>>>> It >>>>>>> was used for install and setup. But I never got boot changed ov= er to >>>>>>> md0. >>>>>>> So >>>>>>> currently md0 is not in use. md0 and md2 are mounted to folders= on >>>>>>> the >>>>>>> 40gb >>>>>>> so a precopy to md0 could be made before booting with a cd and >>>>>>> copying >>>>>>> what >>>>>>> ever is left that needs coping. and to use md2. >>>>>>> >>>>>>> Current >>>>>>> # /etc/fstab: static file system information. >>>>>>> # >>>>>>> # =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 >>>>>>> >>>>>>> =A0 >>>>>>> proc =A0 =A0 =A0 =A0 =A0 =A0/proc =A0 =A0 =A0 =A0 =A0 proc =A0 = =A0defaults =A0 =A0 =A0 =A00 =A0 =A0 =A0 0 >>>>>>> /dev/hda1 =A0 =A0 =A0 / =A0 =A0 =A0 =A0 =A0 =A0 =A0 ext3 =A0 =A0= defaults,errors=3Dremount-ro 0 >>>>>>> =A0 1 >>>>>>> /dev/hda5 =A0 =A0 =A0 none =A0 =A0 =A0 =A0 =A0 =A0swap =A0 =A0s= w =A0 =A0 =A0 =A0 =A0 =A0 =A00 =A0 =A0 =A0 0 >>>>>>> /dev/md0 =A0 =A0 =A0 =A0/mnt/md0 =A0 =A0 =A0 =A0ext3 =A0 =A0def= aults =A0 =A0 =A0 =A00 =A0 =A0 =A0 0 >>>>>>> /dev/md2 =A0 =A0 =A0 =A0/mnt/md2 =A0 =A0 =A0 =A0ext3 =A0 =A0def= aults =A0 =A0 =A0 =A00 =A0 =A0 =A0 0 >>>>>>> /dev/hdb =A0 =A0 =A0 =A0/media/cdrom0 =A0 udf,iso9660 user,noau= to =A0 =A0 0 =A0 =A0 =A0 0 >>>>>>> /dev/fd0 =A0 =A0 =A0 =A0/media/floppy0 =A0auto =A0 =A0rw,user,n= oauto =A00 =A0 =A0 =A0 0 >>>>>>> >>>>>>> I want to change md0 and md1 from 3 drive mirriors to 2 drive >>>>>>> mirrors. >>>>>>> >>>>>>> Finish changing over to booting from md0, move swap to md1 and = move >>>>>>> the >>>>>>> mount point for md2 to md0 >>>>>>> >>>>>>> Remove the the ide drive to free up space for the 4th 500gb dri= ve. >>>>>>> >>>>>>> Copy md2 over to the new 500gb temparally. >>>>>>> >>>>>>> Get rid of the current md2 freeing up the 3rd drive since it wa= s >>>>>>> already >>>>>>> taken out of the mirrors above. >>>>>>> >>>>>>> Make a new md2 raid1 with the remaining space of the first 2 sa= ta >>>>>>> drives. >>>>>>> >>>>>>> Move the data from the 4th drive back to the new md2 >>>>>>> >>>>>>> Repartition the 3rd drive to 1 partition same as the 4th drive. >>>>>>> >>>>>>> Make raid1 md3 from the 3rd and 4th drives. >>>>>>> >>>>>>> It's the steps/commands to change md0 and md1 from 3 drive mirr= ors to >>>>>>> 2 >>>>>>> drive mirrors that I'm not sure about. Though now looking at fs= tab I >>>>>>> see >>>>>>> I >>>>>>> never even switched over the swap. So I guess, those to arrays = would >>>>>>> be >>>>>>> rebuilt. So it's more how to do that without messing with md2. >>>>>>> >>>>>>> The computer is still on grub1. I haven't updated it it. >>>>>>> >>>>>> >>>>>> So you have: >>>>>> >>>>>> [3-devices] >>>>>> Raid 1: Unused >>>>>> Raid 1: Unused >>>>>> Raid 5: Used - 3 disks >>>>>> >>>>>> [1-device] >>>>>> Boot + swap >>>>>> >>>>>> Just swapoff the raid-swap you want to re-create, then: >>>>>> mdadm -S /dev/md(swap) >>>>>> mdadm -S /dev/md(boot) >>>>>> mdadm --zero-superblock /dev/devices in those arrays >>>>>> >>>>>> Repartition those two areas of the disks as necessary. >>>>>> >>>>>> Create new boot and swap partitions. >>>>>> For boot make SURE you use either -e 0.90 OR -e 1.0 . =A0 Given = the >>>>>> nature of /boot I'd say use -e 0.90 on it. >>>>>> For everything else, including swap use -e 1.1 and optionally >>>>>> write-intent bitmaps. >>>>>> >>>>>> At this point you should be able to move /boot and your swap off= of >>>>>> the 40gb drive; just remember to re-install grub and that your B= IOS >>>>>> likely sets the boot drive as bios-drive 0 regardless of which S= DA/HDA >>>>>> linux sees it as. =A0This is what the device.map file is used to= tell >>>>>> grub. >>>>>> >>>>>> >>>>>> I lost exactly what you wanted the result to look like amid a lo= ng >>>>>> list of steps you /thought/ you needed to make to get there and >>>>>> references to md numbers that only have meaning to you. =A0Howev= er it >>>>>> seems that you were mostly stuck getting to this point, so you m= ight >>>>>> be able to determine a plan using the data you've yet to share w= ith >>>>>> the rest of us with that 40gb drive out of the equation. >>>>>> >>>>>> Remember that you can't reshape raid10 yet, but you can start ra= id10 >>>>>> with 'missing' devices (and add in the spares later). >>>>>> >>>>> >>>>> can't make it much clearer, and I don't know where you got raid10= from. >>>>> >>>>> ARRAY /dev/md0 level=3Draid1 num-devices=3D3 >>>>> UUID=3De4926be6:8d6f08e5:0ab6b006:621c4ec0 >>>>> ARRAY /dev/md1 level=3Draid1 num-devices=3D3 >>>>> UUID=3Deac96451:66efa3ab:0ab6b006:621c4ec0 >>>>> ARRAY /dev/md2 level=3Draid5 num-devices=3D3 >>>>> UUID=3Da7ed721e:04b10ab6:0ab6b006:621c4ec0 >>>>> >>>>> md 0 and 1 I want to change to 2 deivces. >>>>> Then get it booting from md0 which I think I know how to do as I = got >>>>> another >>>>> computer working that way. Then I can dump the ide drive making r= oom >>>>> for >>>>> another 500gb sata. It will be used to store the data from md2 wh= ile >>>>> md2 >>>>> is >>>>> remade from a 3 device raid5 to a 2 device raid1. This frees up a= drive >>>>> giving me 2 500gb drives to make another 2 device raid1. Its the >>>>> seperating >>>>> out 1 device from each of the current raid1's, md0 and md1 that I= was >>>>> asking >>>>> about. >>>>> >>>> >>>> That was concise enough to be worth reading through to get your de= sired >>>> result. >>>> >>>> I already told you how to do the safe portion of your last operati= on. >>>> >>>> Actually, I can see one benefit from two raid 1s over raid10; you >>>> could stripe them using LVM and for very little extra cost have a = lot >>>> more flexibility. >>>> >>>> After you've followed my last email to get the system booting off = your >>>> raid devices, you can replace the 40 with a 500, and then use the >>>> /new/ 500 to start a raid 1 with one device set as 'missing'. >>>> StartSlower but totally safe way: >>>> >>>> However what I'd do if I were you is get everything I could off of= the >>>> 40gb drive; it would only take 10 DVDs at worse case; presuming it >>>> won't fit within ~500gb. >>>> >>>> 1) Duplicate all your data from the raid5 on to the single disk in= a >>>> raid 1 + missing configuration. >>>> * you now have 1 copy with parity, and 1 copy waiting for mirrorin= g. >>>> (two copies and some parity) >>>> >>>> 2) Fail one of the partitions from the raid5, --zero superblock it= =2E >>>> * you now have 1 copy without parity, and 1 copy waiting for >>>> mirroring, and one free drive. (two copies) >>>> >>>> 3) --add the previous parity partition as part of the mirror set; = IT >>>> MUST BE>=3D the size of the other partition in that mirror. >>>> * You have 1 copy without parity, 1 full copy being mirrored in to= two >>>> copies. >>>> >>>> 4) WAIT for the mirroring operation to finish. >>>> >>>> 5) Optionally 'check' the mirror copy. >>>> * You now have 1 copy without parity, 1 fully mirrored copy (3 cop= ies >>>> of your data) >>>> >>>> 6) With a fully safe copy of your data, and two partitions you can >>>> start a wider range of procedures. >>>> >>>> You could create two more raid 1 + missing arrays >>>> setup LVM with striping across them >>>> copy your data over in to one logical volume >>>> ++THEN AGAIN in to a second logical volume >>>> (Manually creating two copies of your data; this won't protect aga= inst >>>> drive failure but it will protect against individual failed sector= s, >>>> which may be good enough.) >>>> Finally one at a time fail out members of the intact raid1 set and= add >>>> them to the new raid 1s. >>>> >>>> >>>> Or probably proceed through other ideas; though none seems as >>>> appealing to me as what I just wrote; 4 drives isn't enough to >>>> seriously consider raid 5 or 6, and with the size of the drives yo= u'd >>>> really be better off with going for raid 6, which is much slower a= nd >>>> only slightly less risky than raid 1 + striping via LVM. >>>> >>> >>> I don't need to combine the two large arrays. They are mostly for s= toring >>> recordings for vdr. With vdr, the storage folders are video0, video= 1, >>> video2,... and you provide the path to video0 with the others being= in >>> the >>> same parent folder. When it records, it sends the file to the folde= r with >>> the most free space and if that is not video0, then video0 gets a l= ink to >>> where it really is. >>> >>> >>> I don't understand where "For boot make SURE you use either -e 0.90= OR -e >>> 1.0 ." comes in. When I partitioned the drives, I set them all to t= ype >>> fd. >>> and made the first partition of each drive bootable. To create the = arrays >>> I >>> used: >>> sudo mdadm --create /dev/md0 --level=3D1 --raid-devices=3D2 /dev/sd= a1 >>> /dev/sdb1 >>> --spare-devices=3D1 /dev/sdc1 >>> sudo mdadm --create /dev/md1 --level=3D1 --raid-devices=3D2 /dev/sd= a2 >>> /dev/sdb2 >>> --spare-devices=3D1 /dev/sdc2 >>> >>> And to format them: >>> sudo mkfs.ext3 /dev/md0 >>> sudo mkswap /dev/md1 >>> >>> Latter I used grow to change from a spare to a 3 way mirror. >>> >> >> Please, reply to ALL or manually add the list back to the message. >> >> You'll want the two types I specified for any boot device since it >> then appears to be a normal partition that happens to have an exact >> copy on another partition, except for a little bit at the end which = is >> where the raid metadata is. >> >> For that same reason, using mdadm 1.1 or 1.2 would be preferable for >> any devices which are not directly used for boot. =A0Those place the >> data at the beginning thus ensuring that any set of layering >> (raid/lvm/filesystems) gets unpacked in the correct order since ther= e >> is no question how they are stacked. >> >> Also, I don't really see how that changes the directions for getting >> your smaller drive out of the system so that you can proceed. =A0It >> sounds like you'll probably be able to adapt the steps to suit your >> needs and you've not actually told any of us what you're confused >> about or still have a problem with. >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-raid= " in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html >> > > First, with the other list I'm on, simply doing a reply replies back = to the > list. This one for some reason forwards the responders email for the = reply > to. I was in a hurry and forgot to put the right address back > > Next, you have now confused me with the part: >> You'll want the two types I specified for any boot device since it >> then appears to be a normal partition that happens to have an exact > Two types of what? To make both drives of the boot array bootable I h= ave > been doing: > =A0sudo grub > =A0grub>device (hd0) /dev/sda > =A0grub>root (hd0,0) > =A0grub>setup (hd0) > > =A0grub>device (hd0) /dev/sdb > =A0grub>root (hd0,0) > =A0grub>setup (hd0) > > That was done after the array was made. That worked for the other com= puter I > setup which has 2 sata drives and raid1 for all 3 partitions. It has = been > booting ok for nearly a year. I did just update it from lenny to test= ing and > started the change to grub2. It is now booting using chain to grub1. > > And what I was asking about in the last message was when you said: > "For boot make SURE you use either -e 0.90 OR -e 1.0 ." > Are those mdadm switches to use older version methods? > -- > To unsubscribe from this list: send the line "unsubscribe linux-raid"= in > the body of a message to majordomo@vger.kernel.org > More majordomo info at =A0http://vger.kernel.org/majordomo-info.html > Yes, those are switches to the mdadm command; PLEASE RTFM: man mdadm . After reading the full manual (yeah, there's a 'fun' expansion too) most of the easy questions will already be answered. Grub2 / grub 2.0 / 'grub not legacy' , as I've stated, wants to see version 0.90 metadata for /boot . Installing grub is off topic for this mailing list, but you should be able to find documentation in the places you should expect to look. I have not yet experienced booting grub2 from mirrored devices, but the above information is what I have gleaned from various previous posts to the two related mailing lists. Remember to read the f* manuals. -- To unsubscribe from this list: send the line "unsubscribe linux-raid" i= n the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html