From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx1.redhat.com (ext-mx09.extmail.prod.ext.phx2.redhat.com [10.5.110.13]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id oAENqbX9019915 for ; Sun, 14 Nov 2010 18:52:37 -0500 Received: from mail-ey0-f174.google.com (mail-ey0-f174.google.com [209.85.215.174]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id oAENqOEv026663 for ; Sun, 14 Nov 2010 18:52:24 -0500 Received: by eye27 with SMTP id 27so2734819eye.33 for ; Sun, 14 Nov 2010 15:52:23 -0800 (PST) MIME-Version: 1.0 In-Reply-To: <20101114215850.GB10579@barkeeper1-xen.linbit> References: <20101113230310.GH14546@barkeeper1-xen.linbit> <20101114215850.GB10579@barkeeper1-xen.linbit> Date: Sun, 14 Nov 2010 18:52:23 -0500 Message-ID: From: Stirling Westrup Content-Transfer-Encoding: 8bit Subject: Re: [linux-lvm] Need help with a particular use-case for pvmove. Reply-To: swestrup@gmail.com, LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: Content-Type: text/plain; charset="utf-8" To: LVM general discussion and development On Sun, Nov 14, 2010 at 4:58 PM, Lars Ellenberg wrote: > On Sat, Nov 13, 2010 at 10:56:05PM -0500, Stirling Westrup wrote: >> On Sat, Nov 13, 2010 at 6:03 PM, Lars Ellenberg >> wrote: >> > On Sat, Nov 13, 2010 at 04:45:36PM -0500, Stirling Westrup wrote: >> >> ... >> >> All I want to do is move physical extents from one physical volume to >> >> another. Both of those volumes are present and accessible. Why should >> >> uninvolved missing volumes be an issue, and is there any way around >> >> it? �pmmove suggests running "vgreduce --removemissing" but the >> >> documentation for vgreduce seems to say that I'd need to 1) use >> >> --force and 2) it would likely result in data loss. >> >> >> >> Is there anything I can do, short of borrowing another storage array >> >> somewhere, just so I can have an extra slot to do this move? My other >> >> option is to put the new drive into a USB case, but the server only >> >> supports USB1, so moving a terrabyte will take over a week. >> >> >> >> > If you do it offline anyways: >> > >> > Shut down. >> > Unplug one of the good old drives, plug in the new drive. >> > If you want to be extra sure against typos, >> > unplug all but the bad-old drive ;-) >> > >> > Boot into maintenance mode, use a live-cd if you have to. >> > >> > Don't activate the VG. �It won't activate with one pv missing anyways, >> > unless you really want it to. >> > >> > Then dd_rescue copy the disk image from bad-old to new, including >> > everything (partition table, if any, LVM signature, the full image). >> > >> > Remove bad-old drive, have all other old and the new plugged, >> > reboot normally. >> > >> > Done. >> > >> > Estimated downtime, assuming a sustained linear write speed of 80 MiB/s: >> > 1 TiB / (80 MiB/s), well under 4 hours. >> > >> >> Thanks. I did consider this, but the new drive is twice the size of >> the old one, so I would need to make sure I had created a partition on >> the new drive the exact size of the old one, and had dd-ed everything >> correctly. Even then, I wasn't sure if it would work, because I don't >> know what the metadata records in terms of the drive configurations. > > No. Don't ask for advice, if you don't take it. �I don't just post > nonsense on mailing lists, just to read my own words in the net ;-) > > Demo run, simulating a two drive VG, replacing one drive with a bigger one, > using LVs of some other VG as "drives". > > root@racke:~/demo# export LVM_SYSTEM_DIR=$PWD > root@racke:~/demo# pvs > root@racke:~/demo# pvcreate /dev/demo/dummy-a > �Physical volume "/dev/demo/dummy-a" successfully created > root@racke:~/demo# pvcreate /dev/demo/dummy-b > �Physical volume "/dev/demo/dummy-b" successfully created > root@racke:~/demo# vgcreate Data /dev/demo/dummy-{a,b} > �Volume group "Data" successfully created > root@racke:~/demo# vgs > �VG � #PV #LV #SN Attr � VSize VFree > �Data � 2 � 0 � 0 wz--n- 1.99g 1.99g > root@racke:~/demo# lvcreate -n data -L 1.8g Data > �Rounding up size to full physical extent 1.80 GiB > �Logical volume "data" created > root@racke:~/demo# mkefs.ext4 /dev/Data/data > mke2fs 1.41.11 (14-Mar-2010) > ... > Creating journal (8192 blocks): done > Writing superblocks and filesystem accounting information: done > ... > root@racke:~/demo# vgs > �VG � #PV #LV #SN Attr � VSize VFree > �Data � 2 � 1 � 0 wz--n- 1.99g 196.00m > root@racke:~/demo# blockdev --getsize64 /dev/demo/dummy-* > 1073741824 > 1073741824 > 2147483648 > root@racke:~/demo# blockdev --getsize64 /dev/demo/dummy-c > 2147483648 > > Ok, so you now have a LV data in a VG Data consisting of two "drives", 1gig each, > and a third drive of two gig. > > Now, simulating drive replacement with the method I told you. > > root@racke:~/demo# vgchange -an Data > �0 logical volume(s) in volume group "Data" now active > root@racke:~/demo# lvs > �LV � VG � Attr � LSize Origin Snap% �Move Log Copy% �Convert > �data Data -wi--- 1.80g > > > (you may want to play with some options of dd_rescue > to get best performance) > root@racke:~/demo# dd_rescue /dev/demo/dummy-a /dev/demo/dummy-c > ... > dd_rescue: (info): /dev/demo/dummy-a (1048576.0k): EOF > Summary for /dev/demo/dummy-a -> /dev/demo/dummy-c: > dd_rescue: (info): ipos: � 1048576.0k, opos: � 1048576.0k, xferd: � 1048576.0k > ... > > root@racke:~/demo# pvs -v > � �Scanning for physical volume names > �Found duplicate PV 8Miyc0iXMErbqbTBMfCxrMSLMIo0F2IP: using /dev/demo/dummy-c not /dev/demo/dummy-a > �PV � � � � � � � � � � � � �VG � Fmt �Attr PSize � �PFree � DevSize PV UUID > �/dev/demo/dummy-b Data lvm2 a- � 1020.00m 196.00m � 1.00g N8tSRP-qwxK-M8wo-wy5A-4f8V-u07s-8xDwgi > �/dev/demo/dummy-c Data lvm2 a- � 1020.00m � � �0 � �2.00g 8Miyc0-iXME-rbqb-TBMf-CxrM-SLMI-o0F2IP > > Uh oh... duplicate PV signature... �Well, yes, of course, we physically > copied the image, including any signatures.. > > � � � �=>> if you have your PVs on partitions, not the full disks, > � � � � � �just create a partition fully covering the new drive, > � � � � � �and dd the old PV partition into the new partition. > � � � � � �No matter if the new partition is bigger. > > Now, simulate unplugging the old drive by adjusting my demo filter: > root@racke:~/demo# vi lvm.conf +/filter\ = > > ... > - � �filter = [ "a|^/dev/demo/dummy-[abc]$|", "r/.*/" ] > + � �filter = [ "a|^/dev/demo/dummy-[bc]$|", "r/.*/" ] > > root@racke:~/demo# vgscan > �Reading all physical volumes. �This may take a while... > �Found volume group "Data" using metadata type lvm2 > root@racke:~/demo# pvscan > �PV /dev/demo/dummy-c � VG Data � lvm2 [1020.00 MiB / 0 � �free] > �PV /dev/demo/dummy-b � VG Data � lvm2 [1020.00 MiB / 196.00 MiB free] > �Total: 2 [1.99 GiB] / in use: 2 [1.99 GiB] / in no VG: 0 [0 � ] > > There. No more duplicate PV signatures. > > root@racke:~/demo# lvs > �LV � VG � Attr � LSize Origin Snap% �Move Log Copy% �Convert > �data Data -wi-a- 1.80g > root@racke:~/demo# vgs > �VG � #PV #LV #SN Attr � VSize VFree > �Data � 2 � 1 � 0 wz--n- 1.99g 196.00m > > have lvm recognize the new PV size: > root@racke:~/demo# pvresize /dev/demo/dummy-c > �Physical volume "/dev/demo/dummy-c" changed > �1 physical volume(s) resized / 0 physical volume(s) not resized > root@racke:~/demo# vgs > �VG � #PV #LV #SN Attr � VSize VFree > �Data � 2 � 1 � 0 wz--n- 2.99g 1.19g > � � � � � � � � � � � � �^^^^^ ^^^^^ > root@racke:~/demo# lvs > �LV � VG � Attr � LSize Origin Snap% �Move Log Copy% �Convert > �data Data -wi-a- 1.80g > > reactivate the VG: > root@racke:~/demo# vgchange -ay Data > > and now mount your LV, and be happy. > > > You now can add more space to your LV, > or pvmove an other drive onto the new space of the new bigger drive, > reducing it from the VGs, replace it with an other bigger one, > pvcreate and add it into the VG to grow your VG further. > > Still you should consider some RAID, as in _redundancy_ of your data. > Thanks for going through all those steps. It does make the procedure a lot clearer in my mind, and it does look like dd_rescue is the way to go then. I'm going to head off to try it now.