[linux-lvm] setting up LVM, lost everything with a typo

[linux-lvm] setting up LVM, lost everything with a typo

[Craig Hagerman]

Hi,

I recently got a new HD and thought it was about time I started using
LVM. Unfortunatly, I think I have ended up loosing all of my data due
to a typo. I would like to know if my data is lost forever. If not,
what can I do now to recover. Here is the whole story...

Craig

Originally it was set up with 2 HD like this:
   hda - containing  /  /home  /usr
   sda - containing one big partition (/dev/sda2) for data storage (SATA drive)

I added a new SATA drive and used cfdisk to format it and make one big
partition (ext3). My plan was to create a LV on the new HD, copy
everything over from the old HD, then add that old HD to the new LV.
I followed the LVM HOWTO to create a vg etc:

%  pvcreate /dev/sdb1

%  vgcreate media_vg /dev/sdb1

%  lvcreate -L 233G -n media_lv media_vg

%  mke2fs -j /dev/media_vg/media_lv

%  mount /dev/media_vg/media_lv /mnt/media

... and everything worked beautifully. I copied everything (188GB)
from sda2 onto the new LV successfully. So next was time to prep sda2
to be added to the LV.... this is where I screwed up.

% pvcreate /dev/sda2
  Physical volume "/dev/sda2" successfully created

% vgextend media_vg /dev/sda2
  Volume group "media_vg" successfully extended


... I was following some example on a web site, and typed in the wrong
device next. The web site had said to use fdisk to change the
partition type to 8e. This SHOULD have been sda, but I wrote sdb by
mistake:

$ fdisk /dev/sdb

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 8e
Changed system type of partition 1 to 8e (Linux LVM)

Command (m for help): p

Disk /dev/sdb: 251.0 GB, 251000193024 bytes
255 heads, 63 sectors/track, 30515 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1       30515   245111706   8e  Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device
or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.


.... BUT.... I didn't notice the screw up. I wanted to see if the LV
would be automaticaly mounted on a reboot without having to add to
fstab or an init script (I have been told that this would happen on
Debian), so I rebooted.
Now neither disc can be seen or mounted. dmesg | tail shows this:

 % dmesg | tail
EXT3-fs error (device sda2): ext3_check_descriptors: Block bitmap for
group 0 not in group (block 3258811534)!
EXT3-fs: group descriptors corrupted !
EXT3-fs error (device sda2): ext3_check_descriptors: Block bitmap for
group 0 not in group (block 3258811534)!
EXT3-fs: group descriptors corrupted !
VFS: Can't find ext3 filesystem on dev sdb.

... vgchange can't help out....

% vgchange -a y media_vg
  Couldn't find device with uuid 'ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6'.
  Couldn't find all physical volumes for volume group media_vg.
  Couldn't find device with uuid 'ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6'.
  Couldn't find all physical volumes for volume group media_vg.
  Unable to find volume group "media_vg"

....vgdisplay can't see either disc (exact same story with lvdisplay)...

vgdisplay
  Couldn't find device with uuid 'ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6'.
  Couldn't find all physical volumes for volume group media_vg.
  Couldn't find device with uuid 'ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6'.
  Couldn't find all physical volumes for volume group media_vg.
  Volume group "media_vg" doesn't exist

... and pvdisplay gives me...

% pvdisplay
  Couldn't find device with uuid 'ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6'.
  --- Physical volume ---
  PV Name               /dev/sdb1
  VG Name               media_vg
  PV Size               233.75 GB / not usable 0
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              59841
  Free PE               193
  Allocated PE          59648
  PV UUID               rlnV4y-B4eJ-Pyc0-RLJm-2Zbv-t04C-dxzU1A

  --- Physical volume ---
  PV Name               unknown device
  VG Name               media_vg
  PV Size               186.30 GB / not usable 0
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              47694
  Free PE               47694
  Allocated PE          0
  PV UUID               ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6

Re: [linux-lvm] setting up LVM, lost everything with a typo

[Klaus Strebel]

Craig Hagerman schrieb:
> ... vgchange can't help out....
> 
> % vgchange -a y media_vg
>   Couldn't find device with uuid 'ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6'.
>   Couldn't find all physical volumes for volume group media_vg.
>   Couldn't find device with uuid 'ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6'.
>   Couldn't find all physical volumes for volume group media_vg.
>   Unable to find volume group "media_vg"
> 
> ....vgdisplay can't see either disc (exact same story with lvdisplay)...
> 
> vgdisplay
>   Couldn't find device with uuid 'ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6'.
>   Couldn't find all physical volumes for volume group media_vg.
>   Couldn't find device with uuid 'ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6'.
>   Couldn't find all physical volumes for volume group media_vg.
>   Volume group "media_vg" doesn't exist
> 
> ... and pvdisplay gives me...
> 
> % pvdisplay
>   Couldn't find device with uuid 'ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6'.
>   --- Physical volume ---
>   PV Name               /dev/sdb1
>   VG Name               media_vg
>   PV Size               233.75 GB / not usable 0
>   Allocatable           yes
>   PE Size (KByte)       4096
>   Total PE              59841
>   Free PE               193
>   Allocated PE          59648
>   PV UUID               rlnV4y-B4eJ-Pyc0-RLJm-2Zbv-t04C-dxzU1A
 > ...

What says vgscan in debug mode (you need vgscan before doing a vgchange).

Ciao
Klaus
--
Klaus Strebel, Dipl.-Inform. (FH), mailto:klaus.strebel@gmx.net

/"\
\ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
/ \

Re: [linux-lvm] setting up LVM, lost everything with a typo

[Craig Hagerman]

[-- Attachment #1: Type: text/plain, Size: 533 bytes --]

On 10/20/05, Klaus Strebel <klaus.strebel@gmx.net> wrote:
>
> What says vgscan in debug mode (you need vgscan before doing a vgchange).

  vgscan gives the same results as vgdisplay and lvdisplay...

vgscan
Couldn't find device with uuid 'ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6'.
Couldn't find all physical volumes for volume group media_vg.
Couldn't find device with uuid 'ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6'.
Couldn't find all physical volumes for volume group media_vg.
Volume group "media_vg" doesn't exist

:(

[-- Attachment #2: Type: text/html, Size: 886 bytes --]

Re: [linux-lvm] setting up LVM, lost everything with a typo

[Craig Hagerman]

[-- Attachment #1: Type: text/plain, Size: 623 bytes --]

On 10/20/05, Craig Hagerman <craighagerman@gmail.com> wrote:
>
> On 10/20/05, Klaus Strebel <klaus.strebel@gmx.net> wrote:
> >
> > What says vgscan in debug mode (you need vgscan before doing a
> > vgchange).
>
>
Just in case this is helpful, here is what pvscan reports. Note, where it
says "unknown device" is has got to be /dev/sda2:

$ pvscan
Couldn't find device with uuid 'ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6'.
PV /dev/sdb1 VG media_vg lvm2 [233.75 GB / 772.00 MB free]
PV unknown device VG media_vg lvm2 [186.30 GB / 186.30 GB free]
Total: 2 [420.06 GB] / in use: 2 [420.06 GB] / in no VG: 0 [0 ]

[-- Attachment #2: Type: text/html, Size: 1432 bytes --]

Re: [linux-lvm] setting up LVM, lost everything with a typo

[Mike Snitzer]

[-- Attachment #1: Type: text/plain, Size: 1073 bytes --]

On 10/19/05, Craig Hagerman <craighagerman@gmail.com> wrote:
>
> Hi,
>
> I recently got a new HD and thought it was about time I started using
> LVM. Unfortunatly, I think I have ended up loosing all of my data due
> to a typo. I would like to know if my data is lost forever. If not,
> what can I do now to recover. Here is the whole story...


You've changed the partition table of /dev/sdb but the data itself _should_
still be intact (changing the type shouldn't adversely affect the offsets of
your sdb1 partition). You might try the type change on /dev/sda2 and then
restore the backup of the media_vg's VGDA from /etc/lvm/backup/media_vg
using pvcreate and vgcfgrestore, something like:

pvcreate --restorefile /etc/lvm/backup/media_vg --uuid
rlnV4y-B4eJ-Pyc0-RLJm-2Zbv-t04C-dxzU1A /dev/sdb1
pvcreate --restorefile /etc/lvm/backup/media_vg --uuid
ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6
/dev/sda2
vgcfgrestore -f /etc/lvm/backup/media_vg test_vg

That said, its unclear to me why /dev/sda2 is unknown... so take my advise
at your own risk ;)

[-- Attachment #2: Type: text/html, Size: 1469 bytes --]

Re: [linux-lvm] setting up LVM, lost everything with a typo

[Alasdair G Kergon]

On Thu, Oct 20, 2005 at 11:07:42AM +0900, Craig Hagerman wrote:
> %  pvcreate /dev/sdb1
 
> /dev/sdb1               1       30515   245111706   8e  Linux LVM
 
> VFS: Can't find ext3 filesystem on dev sdb.
 
Confusion between sdb and sdb1?

Similar with sda perhaps?

If your device is partitioned, then you *only* use the partitions - never 
sdb at the same time as sdb1.  Which is it meant to be?

Alasdair
-- 
agk@redhat.com

Re: [linux-lvm] setting up LVM, lost everything with a typo

[Craig Hagerman]

On 10/21/05, Mike Snitzer <snitzer@gmail.com> wrote:
> Last line should've been:
>  vgcfgrestore -f /etc/lvm/backup/media_vg media_vg
>
>
>
> ---------- Forwarded message ----------
> From: Mike Snitzer <snitzer@gmail.com>
> Date: Oct 20, 2005 3:54 PM
> Subject: Re: [linux-lvm] setting up LVM, lost everything with a typo
> To: LVM general discussion and development <linux-lvm@redhat.com>
>
>
>  On 10/19/05, Craig Hagerman <craighagerman@gmail.com > wrote:
> > Hi,
> >
> > I recently got a new HD and thought it was about time I started using
> > LVM. Unfortunatly, I think I have ended up loosing all of my data due
> > to a typo. I would like to know if my data is lost forever. If not,
> > what can I do now to recover. Here is the whole story...
>
>  You've changed the partition table of /dev/sdb but the data itself _should_
> still be intact (changing the type shouldn't adversely affect the offsets of
> your sdb1 partition).  You might try the type change on /dev/sda2 and then
> restore the backup of the media_vg's VGDA from /etc/lvm/backup/media_vg
> using pvcreate and vgcfgrestore, something like:
>
>  pvcreate --restorefile /etc/lvm/backup/media_vg --uuid
> rlnV4y-B4eJ-Pyc0-RLJm-2Zbv-t04C-dxzU1A /dev/sdb1
>  pvcreate --restorefile /etc/lvm/backup/media_vg --uuid
> ZHl4Ak-mGS4-wFlF-4neF-G1Ed-GxWB-C4Smj6 /dev/sda2
>  vgcfgrestore -f /etc/lvm/backup/media_vg test_vg
>
>  That said, its unclear to me why /dev/sda2 is unknown...  so take my advise
> at your own risk ;)
>
>


Thank you Mike - that did it. I am SO happy! I can mount the vg and
everything is there. I think I will backup everything onto DVD (about
180GB) before I do another reboot.

Craig