[linux-lvm] deleted LV recovery: understanding metadata format and determining LV extents

[linux-lvm] deleted LV recovery: understanding metadata format and determining LV extents

[Thanos Makatos]

Hi, I'm working on a case where a bunch of logical volumes were deleted but now we need them back. We have disabled archives and back ups in lvm.conf. They way I am trying to approach the problem is to examine LVM's metadata (the first MB of the block device) and determine whether the extents used by these logical volumes have been assigned to other logical volumes.

I am developing a script (which I hope to upstream) that does that however I need some clarification regarding the meaning of some of the metadata. I have focused on the list of segments in each logical volume definitation but I don't know how to interpret the "stripes" member. These are all linear LVs.

What is this "stripes" member and how is it related to "start_extent" and "extent_count"?
E.g. I have the following two LV definitions (the rest of the metadata is ommitted):

		foo {
			id = "..."
			status = ["READ", "WRITE", "VISIBLE"]
			flags = []
			segment_count = 1
			
			segment1 {
				start_extent = 0
				extent_count = 2
				
				type = "striped"
				stripe_count = 1 # linear
				
				stripes = [
				"pv0", 306705
				]
			}
		}

		bar {
			id = "..."
			status = ["READ", "WRITE", "VISIBLE"]
			flags = []
			segment_count = 9

                        # segments 1-8 ommitted

			segment9 {
				start_extent = 59162
				extent_count = 3662
				
				type = "striped"
				stripe_count = 1 # linear
				
				stripes = [
				"pv0", 246616
				]
			}
		}

I thought that the "stripes" is some extent from which the "start_extent" is calculated but I think I'm wrong because the segments of these two LVs overlap. How can I determine which extents are used by each LV?

Any help greatly appreciated.

--
Thanos Makatos

Re: [linux-lvm] deleted LV recovery: understanding metadata format and determining LV extents

[Thanos Makatos]

> > know how to interpret the "stripes" member. These are all linear LVs.
> >
> >     stripes = [� "pv0", 306705     ]
>
> do the values make sense from a 512byte block point of view? Either as extent-size * #extents = size in 512kb blocks,
> or as an absolute offset from the beginning of PV0 device? or it might be the offset in extent-size from the beginning
> of the PV0 data area. Yeah, it's probably that.

I don't think these units are expressed in anything else but extent size units (http://tldp.org/HOWTO/LVM-HOWTO/pe.html).

I did the following test: I created two LV and compared the LVM metadata against the output of lvdisplay -m, here's what I got:

First volume, LVM metadata:
segment1 {
	start_extent = 0
	extent_count = 256
	type = "striped"
	stripe_count = 1
	stripes = ["pv0", 2568]
}
First volume, lvdisplay -m:
--- Segments ---
Logical extent 0 to 255:
  Type                linear
  Physical volume     /dev/sda3
  Physical extents    2568 to 2823

Second volume, LVM metadata:
segment1 {
	start_extent = 0
	extent_count = 2567
	type = "striped"
	stripe_count = 1
	stripes = ["pv0", 1]
}
Second volume, lvdisplay -m:
--- Segments ---
Logical extent 0 to 2566:
  Type                linear
  Physical volume     /dev/sda3
  Physical extents    1 to 2567

Or have I got it completely wrong?

Re: [linux-lvm] deleted LV recovery: understanding metadata format and determining LV extents

[matthew patton]

you're right. the PV,XXXX is to enumerate where the extents start and would be more interesting if you had multiple PV in the VG and interleaved the LV across them or otherwise had multiple disjoint elements for a LV that had been grown over time.

Interesting that LVM leaves a an extent at the front unused. I wonder why. metadata and such I thought was stored outside the extent area.

Re: [linux-lvm] deleted LV recovery: understanding metadata format and determining LV extents

[Alasdair G Kergon]

On Sat, Jan 10, 2015 at 06:12:03PM +0000, Thanos Makatos wrote:
> Hi, I'm working on a case where a bunch of logical volumes were deleted but now we need them back. We have disabled archives and back ups in lvm.conf. They way I am trying to approach the problem is to examine LVM's metadata (the first MB of the block device) and determine whether the extents used by these logical volumes have been assigned to other logical volumes.
 
So start by looking at the contents of the metadata areas and see if they still
contain backups you can extract.
They are circular buffers so look for the seqno field which is incremented
each time the metadata is changed.

lvmdump -m can grab the metadata.
'pvck -v $device' contains the beginnings of code to extract the backups

Alasdair