[linux-lvm] next step of recovering from tha vgchange problem

[linux-lvm] next step of recovering from tha vgchange problem

[Magosanyi Arpad]

Hi!

I am very unhappy with the fact that I had zero feedback from
the developers of lvm, in spite that I have lost all my data.
Yes, this is an open source software, and I was using a feature
which is (I believe) marked as experimental. On the other side
the product is one of the flagship ones of the company for the
developers work, and it was (with the snapshotting feature) in
the stable kernel.

I could backup my data, and my volume group seems to be nearly
consistent now. I still have questions and concerns. For the record,
here is how I have recovered, and my questions are at the end
of the mail.

Keywords: OOPS, vgchange, snapshot, rdev = 65535, "Bad address" activating volume group

It started with not being able to do vgchange -ay, see my previous posts
for details. It turned out that I had a snapshot of one of my lv's, and
vgchange could not activate this one. Because of that, I could not do
anything with my volume group.
After some 4 days and zero feedback from the developers,
I succeded to activate my volume group. The trick was to get rid of 
the code path which had the comment:
        /* Second path to correct snapshot logical volumes which are not
          in place during first path above */
You can find the report of that stage and a patch in the archives.
In that point I still has not got a consistent volume group in spite
of the fact that vgck found it to be OK. I was asking for pointers to
clean up the thing, but again, zero feedback.
After that I was trying to cut the wrong snapshot volume of the vg.
First of course I did a backup, because at this point I could at least.
It took cuts in the following points:
	-zero out the lv entry from the pv
	-set the mode of the snapshotted lv to 3 (don't ask me what does
	that 3 means)
	-decrement the number of lv's in the pv and the vg structure
I did it with dd and by reading lvm.h, along the same lines as the
uuid restorer trick does it.
first dd-ed out the data needeed from the disk to a file, made a backup of it,
edited it with vi, and dd-ed back to it original place. The areas to dd out
could be obtained from the output of pvdata -PP.
Sorry about not giving more details, but I feel that those who are smart
enough not to fuzz their data up by following a more detailed howto can
do it by this description, and I simply have no nerve to be more exact after
10 ours of binary editing.

After the surgery was done, I created a logical volume with the same size
my original snapshotted lv had, and backed it up by dd.
Now I could lvremove my snapshotted lv.
I created and removed logical volumes in the hope
that the extents alocated to the snapshot and the snapshotted lv are freed.

My questions and concerns:
It seems that I have some "lost clusters" of extents. They are listed as
belonging to the two logical volumes I got rid off.
At the first sight, the sum of lv sizes is less than the
size of all allocated extents. (only calculated roughly, so not sure)
It is normal, or some inconsistency remaining?
If it isn't ok, how to make my vg consistent?
Is there any other source of inconsistency I am not aware of?

-- 
GNU GPL: csak tiszta forrásból

[OT] List concerns and dieas (was Re: [linux-lvm] next step of recovering from tha vgchange problem)

[Chad C. Walstrom]

[-- Attachment #1: Type: text/plain, Size: 957 bytes --]

List concern: please shut off reply-to munging.  I wanted to reply to
the author of this email directly, but could not w/o manually cutting
and pasting because of the reply-to munging done by this list.  PLEASE
SHUT IT OFF!

References:
[1] http://www.unicom.com/pw/reply-to-harmful.html
[2] http://www.ietf.org/rfc/rfc822.txt
[3] http://www.ietf.org/rfc/rfc2076.txt
[3] http://www.ietf.org/rfc/rfc2822.txt

On Fri, Nov 02, 2001 at 07:55:38PM +0100, Magosanyi Arpad wrote:
> Keywords: OOPS, vgchange, snapshot, rdev = 65535, "Bad address"
> activating volume group

Nice use of this field, one that was added as per RFC822 (see above).
You can actually place this in the header of your email, if you wish
(Recommended).

-- 
Chad Walstrom <chewie@wookimus.net>                 | a.k.a. ^chewie
http://www.wookimus.net/                            | s.k.a. gunnarr
Key fingerprint = B4AB D627 9CBD 687E 7A31  1950 0CC7 0B18 206C 5AFD


[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [linux-lvm] next step of recovering from tha vgchange problem

[Andreas Dilger]

On Nov 02, 2001  19:55 +0100, Magosanyi Arpad wrote:
> It seems that I have some "lost clusters" of extents. They are listed as
> belonging to the two logical volumes I got rid off.

Yes, I recently ran into this problem as well.  vgck is not very good
at doing anything to check these values.  I was accidentally running
an old version of LVM (partly patched Linus kernel) and when I tried
to lvreduce it would oops.  The LV data was correct (smaller size)
but the PEs were still allocated.  The point isn't that I had a bad
old kernel, but that vgck does not detect/fix these sorts of problems,
which (IMHO) is exactly what it should be doing.

It also turns out that vgcfgrestore is fundamentally broken when restoring
PV UUIDs, I had to always use uuid_fixer to get it to work (even if I tried
backing up the fixed VGDA and then restoring it).

> At the first sight, the sum of lv sizes is less than the
> size of all allocated extents. (only calculated roughly, so not sure)
> It is normal, or some inconsistency remaining?
> If it isn't ok, how to make my vg consistent?
> Is there any other source of inconsistency I am not aware of?

One problem in the design of LVM metadata is that values such as PE/LE
counts are stored in several places on disk, and depending on what piece
of code is used, you may get one or the other.  When everything is OK
it does not matter, in other times it does matter.

If you have time/effort/knowledge, I would suggest that rather than
trying to find/fix the original problem, you spend time trying to make
vgck better.  Put the function of uuid_fixer into vgck.  Make vgck only
read data directly from disk (if it does not already do so).  This will
not only help fix your current problem, but also help fix all other
problems in the future.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: [OT] List concerns and dieas (was Re: [linux-lvm] next step of recovering from tha vgchange problem)

[Friedrich Lobenstock]

On Fri, 2 Nov 2001, Chad C. Walstrom wrote:

> List concern: please shut off reply-to munging.

Leave this on. Most mails are to be mailed to the list anyway. It's
annoying if you'd alway needed to use the reply-to-all feature of your
mail client and then delete the original sender address to keep the
mail on the list and not sending the original sender two mails. I'm
subscribed to some lists and the missing list-reply-to-header on those
is more than annoying.

[X] KEEP the reply-to-header

-- 
MfG / Regards
Friedrich Lobenstock

Re: [OT] List concerns and dieas (was Re: [linux-lvm] next step of recovering from tha vgchange problem)

[Mark van Walraven]

Hello,

On Fri, Nov 02, 2001 at 10:18:11PM +0100, Friedrich Lobenstock wrote:
> Leave this on. Most mails are to be mailed to the list anyway. It's
> annoying if you'd alway needed to use the reply-to-all feature of your
> mail client and then delete the original sender address to keep the
> mail on the list and not sending the original sender two mails. I'm

In my opinion, an annoying mail client is not reason to break the 'reply'
function for people with sensible clients.

> [X] KEEP the reply-to-header

I strongly oppose reply-to munging.

Greetings,

Mark.

Re: [OT] List concerns and dieas (was Re: [linux-lvm] next step of recovering from tha vgchange problem)

[Friedrich Lobenstock]

On Sat, 3 Nov 2001, Mark van Walraven wrote:

> Hello,
>
> On Fri, Nov 02, 2001 at 10:18:11PM +0100, Friedrich Lobenstock wrote:
> > Leave this on. Most mails are to be mailed to the list anyway. It's
> > annoying if you'd alway needed to use the reply-to-all feature of your
> > mail client and then delete the original sender address to keep the
> > mail on the list and not sending the original sender two mails. I'm
>
> In my opinion, an annoying mail client is not reason to break the 'reply'
> function for people with sensible clients.

In one or the other aspect each mail client is broken - this world is
not perfect. I couldnt' find the ultimate one. One might be good at
this filtering duplicate message-ids but it might be totaly crap when
it comes to user-interface, IMAP4, and and and....YMMV


> I strongly oppose reply-to munging.

I strongly advocate reply-to munging on this list. Period

-- 
MfG / Regards
Friedrich Lobenstock

Re: [OT] List concerns and dieas (was Re: [linux-lvm] next step ofrecovering from tha vgchange problem)

[Jason Edgecombe]

> I strongly advocate reply-to munging on this list. Period

as do I,

  Replying to the list should be the default so that everyone can benfit
from an answer t a question posted on the list.

Jason Edgecombe

Re: [OT] List concerns and dieas (was Re: [linux-lvm] next step ofrecovering from tha vgchange problem)

[Alasdair G Kergon]

From the messages in this thread, there is clearly no consensus 
for changing this setting on this particular list.  

End of discussion, please.

However, if someone who feels strongly about this has the time
to get a patch into mailman that makes this configurable at user 
level instead of list level, please go ahead!  

I think a lot of people would be grateful!

Alasdair
-- 
agk@uk.sistina.com