[linux-lvm] pv recovery after trashing partition table.

[linux-lvm] pv recovery after trashing partition table.

[Jim N Cromie]

[-- Attachment #1: Type: text/plain, Size: 2637 bytes --]


Q:  how is 'dd' like motorcycles ?
A:  there are 2 types of users, those who have crashed, and those who
will.

Well, Ive now joined the ranks of folks who have.  I blew away the hda
partition table.

/dev/hda4 used to have a good PV.

Miraculously, Ive successfully recovered hda[1-3], by a combination of
luck,
a simple partition table (1=vfat, 2=swap, 3=ext2, 4=lvm, on cylinder
boundaries), good notes (I had the cylinder numbers mostly written down),
and perseverance with fdisk.  I wrote experimental partition-tables
untill I was able to mount them.

Just imagine my glee when hda3 fsck'd good !


So now, Im trying to recover hda4 now, with LVM volumes intact;

vgcfgrestore -l -l -v -d -n _vgb, reports 2 PVs,  hdb2, hda4,  both are
available, with some unusable, ( I cant cut-paste here, different
computer, cant boot clean without /var

pvscan however reports 2 different PVs,  hdb2, and hdc9

Im guessing that I did a pvcreate /dev/hdc9 after some snapshot/backup,
since hdc9 isnt there.  This suggests that my chances of recovery are
poorer than otherwize.  I never actually did a vgcfgsave - Ive only been
a couple of weeks with LVM, and it was only /tmp...

anyway, pvdata /dev/hda4 shows bogus (ie unfavorable, not wrong) data in
the PV, it also segfaults, $? is 139

PV size                8 / Not usable 262 GB [LVM 3.71 GB]
PV#                     0
PE Size(KB)        250
uuid                    none

btw - 3.71 GB is reasonable, the hda4 has ~3.9 GB

$> pvdisplay /dev/hda4
-- no physical volume identifier


vgcfgrestore gives error when I try above:

ERROR: pv_read(): PV identifer invalid" reading physical volume
/dev/hda4

I should mention; I dont understand how sectors in hda4 got corrupted,
at least by my dd ( I did have bs=512 count=1, and hda[1-3] are ok).
Something else may be missing..


SO, WHAT IF ANY ARE MY OPTIONS ?

1) Would a carefully considered set of 'dd' commands possibly
restore it?   I have nothing to lose in hda4 at this point.

2) IS IT POSSIBLE THAT DESPITE MY SUCCESS MOUNTING hda 1-3, particularly
hda3, since thats ext2, (ie a real filesystem, with a real fsck).  Ive
somehow got the partitions wrong, and thus pvdata is looking in the wrong
disk sector for its meta-data  ?

3) is it possible to use fdisk to change partition type to Linux, then do
an fsck ?
Im theorizing that fsck will find 3 different ext2 super-blocks somewhere
within the previously mountable LVM partition ?

I think I can claim enough space elsewhere, and make a file-clone of the
partition, if so, I can let fsck do all its repair attempts w/o trashing
the possibility
of using approaches 1,2


tia
jimc

[-- Attachment #2: Type: text/html, Size: 4191 bytes --]

Re: [linux-lvm] pv recovery after trashing partition table.

[Andreas Dilger]

On Oct 12, 2001  18:18 -0600, Jim N Cromie wrote:
> Miraculously, Ive successfully recovered hda[1-3], by a combination of
> luck,
> a simple partition table (1=vfat, 2=swap, 3=ext2, 4=lvm, on cylinder
> boundaries), good notes (I had the cylinder numbers mostly written down),
> and perseverance with fdisk.  I wrote experimental partition-tables
> untill I was able to mount them.
> 
> SO, WHAT IF ANY ARE MY OPTIONS ?
> 
> 1) Would a carefully considered set of 'dd' commands possibly
> restore it?   I have nothing to lose in hda4 at this point.
> 
> 2) IS IT POSSIBLE THAT DESPITE MY SUCCESS MOUNTING hda 1-3, particularly
> hda3, since thats ext2, (ie a real filesystem, with a real fsck).  Ive
> somehow got the partitions wrong, and thus pvdata is looking in the wrong
> disk sector for its meta-data  ?

Correct.  It is possible that hda3 is larger than it should be, and is
stealing the beginning of hda4.  What you really want to have is "gpart"
which _should_ find the partition tables for you automatically.

Failing that, you could look at "dumpe2fs -h /dev/hda3" to find the
previous size of the filesystem there so you know how big to make it.

Cheers, Andreas
--
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] pv recovery after trashing partition table. OK - back to normal

[Jim Cromie]

Jim N Cromie wrote:

>  /dev/hda4 used to have a good PV.

Somehow, it does again.

this morning, the box booted clean, and LVM found all logical partitions
and mounted them properly.
I did vgcfgbackup, rebooted to be sure, it rebooted w/o errors.

I dont know why - I did nothing that I thought 'THIS WILL FIX IT'   My
only WAGs are that all those fdisks,
(now including sfdisk - which apparently is more correct) fixed
something that was broken.  Or maybe
one of the many LVM tools fixed it on a *scan, *display, etc.


One of these days Ill actually use a new drive to back stuff up, rather
than filling it with more toys & music.

Re: [linux-lvm] pv recovery after trashing partition table.

[Jim N Cromie]

thanks Andreas,

to my supposition:

> 2) IS IT POSSIBLE THAT DESPITE MY SUCCESS MOUNTING hda 1-3,
particularly
> hda3, since thats ext2, (ie a real filesystem, with a real fsck).  Ive
> somehow got the partitions wrong, and thus pvdata is looking in the
wrong
> disk sector for its meta-data  ?

you wrote:
 
Correct.  It is possible that hda3 is larger than it should be, and is
stealing the beginning of hda4.  What you really want to have is "gpart"
which _should_ find the partition tables for you automatically.
 
Failing that, you could look at "dumpe2fs -h /dev/hda3" to find the
previous size of the filesystem there so you know how big to make it.
 

so following your suggestions,,

gpart.linux is telling me unexpected results:

hda1 data matches, allowing for the -1 offset of cylinder numbers, wrt
fdisk.
hda2 is reported as 94Mb DOS, not  Linux swap.
hda3-4 are un-recognized. (reported as zeros)


dumpe2fs is closer to expected.
dumpe2fs /dev/hda2 shows nothing - which is good, since this is a swap
partition.

dumpe2fs /dev/hda3 shows
Block-ct = 8032, Block-size = 1024

fdisk shows same partition having 16065 512 byte blocks, ie different by
1 block.
So I tried this new size - one less block. gpart.linux now finds 0
partitions.

fsck -V /dev/hda3 works, but doesnt really seem to check much.  No errors
reported, and
if the earlier partition table was correct, then this is silently wrong,
but...

Any suggestions on what this all means ?



Separately, is there an LVM PV super-block pattern that I could look for
?

I believe I caould take a representative od -x output fragment, replace
varying fields
with periods, and use it as a perl regular-expression to test against my
own od-x output.
If this works, then the block offset should be evident from the matching
addresses.

Am I half baked here ?




what does a 

Re: [linux-lvm] pv recovery after trashing partition table.

[Andreas Dilger]

On Oct 12, 2001  20:45 -0600, Jim N Cromie wrote:
> so following your suggestions,,
> 
> gpart.linux is telling me unexpected results:
> 
> hda1 data matches, allowing for the -1 offset of cylinder numbers, wrt fdisk.
> hda2 is reported as 94Mb DOS, not  Linux swap.
> hda3-4 are un-recognized. (reported as zeros)

Yeah, I tried gpart once as well, and it was less-than-stellar at finding
LVM stuff (possibly because of the fact that the ext2 superblock wasn't
aligned properly for a normal partition table).

> dumpe2fs /dev/hda3 shows
> Block-ct = 8032, Block-size = 1024
> 
> fdisk shows same partition having 16065 512 byte blocks, ie different by
> 1 block.

Well, since the block size is 1kB, the last 512-byte chunk is ignored by
ext2, so 16065 is probably correct.  Also, it is hard to specify "1 block"
with a partition table (usually it must be aligned on a cylinder start
or something like that).

> fsck -V /dev/hda3 works, but doesnt really seem to check much.  No errors
> reported, and if the earlier partition table was correct, then this is
> silently wrong, but...

Well, unless you use "-f" it is only looking at the superblock "status"
flag and not much else.  It does still verify that the device size is
big enough for the filesystem, however.

> Separately, is there an LVM PV super-block pattern that I could look for?

Yes, it will be "HM\001\000" (in hex 48 4d 01 00), unless you did the
"LVM upgrade dance" with version 1.0, in which case it may be "HM\002\000".
This would be at byte 0 of your missing partition.

> I believe I caould take a representative od -x output fragment, replace
> varying fields with periods, and use it as a perl regular-expression to
> test against my own od-x output.  If this works, then the block offset
> should be evident from the matching addresses.

If you download the e2fsprogs source (http://sf.net/projects/e2fsprogs)
there is a program "findsuper" which will look for ext2 superblock
signatures.  They will be some arbitrary distance from the start of the
partition because of LVM stuff, but at least you will have an idea if
the filesystem is still there.

Cheers, Andreas
--
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] pv recovery after trashing partition table.

[Jim N Cromie]

hi all,  if anyone responds to this thread, please also cc me.
my usual mail account is tied up inside my broken computer,
and I dont know how long it will take to show up in the mail archives.

tia, jimc

Re: [linux-lvm] pv recovery after trashing partition table.

[Jim Cromie]

[-- Attachment #1: Type: text/plain, Size: 732 bytes --]

again, thanks Andreas,

Having read to the end of `man gpart`, I encountered the useful -

gpart -vdg /boot/boot.bak

which I had lying around (good luck again).  The output(s) confirm 
that my manually fdisk'd partition matches the original.

So somehow I managed to corrupt hda4 at about the same time that
I blanked the MBR.  I dont know how, but Ill accept that theyre
causally (not casually) related.

I looked a bit at gpart's partition-type-guessing code, it shouldnt
be too hard to work out which bytes are failing gpart's guesser.
Getting LVM to accept my hacks may be much harder.

So how big is an LVM super-block ?

I always have the (probably 1-way) fallback - a partion type change, 
then an fsck to see what it finds.

[-- Attachment #2: Type: text/html, Size: 1361 bytes --]

Re: [linux-lvm] pv recovery after trashing partition table.

[Andreas Dilger]

On Oct 13, 2001  02:20 -0600, Jim Cromie wrote:
> So somehow I managed to corrupt hda4 at about the same time that
> I blanked the MBR.  I dont know how, but Ill accept that theyre
> causally (not casually) related.
> 
> I looked a bit at gpart's partition-type-guessing code, it shouldnt
> be too hard to work out which bytes are failing gpart's guesser.
> Getting LVM to accept my hacks may be much harder.

Well, I just ran gpart on a system I have with LVM partitions, and it
does detect them properly.

> So how big is an LVM super-block ?

There are several parts to the LVM metadata.  The PV data (first on the
partition), the VG data (next), LV data, and then the PE map.  Because
LVM metadata has been known to have problems in the past, one thing
you can try is to restore the metadata from backups (see vgcfgrestore).

The real problem is that the start of the filesystem depends on how big
the actual disk is, and the alignment.  If any of the LVM data is there,
doing something like "od -Ax -a /dev/hda4 | more", or even better
"strings -tx /dev/hda4" should show you the VG name, LV names, etc
(along with other stuff, of course).  If you find such data, it may be
possible to reconstruct your LVM setup, or at least recover the data.

> I always have the (probably 1-way) fallback - a partion type change, 
> then an fsck to see what it finds.

It will find nothing, because the filesystem doesn't start until (maybe)
a few MB into the partition.  Any fsck program will expect the filesystem
to start right at the beginning.

Cheers, Andreas
--
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert