[linux-lvm] How to fix inconsistent LV structs?

[linux-lvm] How to fix inconsistent LV structs?

[Raffael Herzog]

Hi,

After a normal reboot I lost my volumes. I don't know, what
exatly happended, but it looks like there was some problem
with AVFS that caused umount not to work properly. I turned
AVFS off now, but this doesn't get me my LVs back.

pvdata shows me, that the logical volume structs 125 through
139 are inconsistent (there are no LVs there), at some later
point, it segfaults. Debug output showed me, that there's
some garbage in these logical volume structs.

My basic idea is to just clear these structs and then use
vgcfgrestore and/or the other recovery tools (which current-
ly all fail with "pv_read(): read") to restore my LVs.

But how do I clear these structs?


TIA,

   Raffi

Re: [linux-lvm] How to fix inconsistent LV structs?

[Heinz J . Mauelshagen]

On Sun, Oct 06, 2002 at 12:49:06AM +0200, Raffael Herzog wrote:
> Hi,
> 
> After a normal reboot I lost my volumes. I don't know, what
> exatly happended, but it looks like there was some problem
> with AVFS that caused umount not to work properly. I turned
> AVFS off now, but this doesn't get me my LVs back.
> 
> pvdata shows me, that the logical volume structs 125 through
> 139 are inconsistent (there are no LVs there), at some later
> point, it segfaults. Debug output showed me, that there's
> some garbage in these logical volume structs.

Hmmm...
Sounds like a nasty overwrite but it is hard to tell because you
can't remmeber the exact details :(

> 
> My basic idea is to just clear these structs and then use
> vgcfgrestore and/or the other recovery tools (which current-
> ly all fail with "pv_read(): read") to restore my LVs.
> 
> But how do I clear these structs?

Presuming that the metadata backups are intact, you need to "pvcreate -ff"
the physical volumes and run vgcfgrestore on each of them.
"vgscan ; vgchange -ay" should get you back to business afterwards.

> 
> 
> TIA,
> 
>    Raffi
> 
> 
> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

-- 

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [linux-lvm] How to fix inconsistent LV structs?

[Raffael Herzog]

Hi Heinz,

Heinz J . Mauelshagen wrote:

> Hmmm...
> Sounds like a nasty overwrite but it is hard to tell because you
> can't remmeber the exact details :(

Well, I can, the syslog is one of the only things that still
exist, besides the backup... :-) These are the last few
messages of the catastrophic reboot:

,----[ /var/log/syslog ]
| Oct  5 21:08:33 rumba kernel: Coda: Bye bye.
| Oct  5 21:08:33 rumba kernel: redir cleanup
| Oct  5 21:08:33 rumba kernel: replacing syscall nr.  12 [e0a01674] with [c012e408]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr. 106 [e0a017a0] with [c0134ad0]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr. 107 [e0a0184c] with [c0134bb0]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr.  33 [e0a018fc] with [c012e2dc]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr.   5 [e0a019c0] with [c012ecb4]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr.  85 [e0a01a9c] with [c0134ce0]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr. 183 [e0a01bd0] with [c013ef44]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr. 195 [e0a01d7c] with [c0134ebc]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr. 196 [e0a01e30] with [c0134f30]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr.  11 [e0a01f4c] with [c0105a30]
| Oct  5 21:08:33 rumba kernel: Unable to handle kernel paging request at virtual address e0a019fb
| Oct  5 21:08:33 rumba kernel:  printing eip:
| Oct  5 21:08:33 rumba kernel: e0a019fb
| Oct  5 21:08:33 rumba kernel: *pde = 01870067
| Oct  5 21:08:33 rumba kernel: *pte = 00000000
| Oct  5 21:08:33 rumba kernel: Oops: 0000
| Oct  5 21:08:33 rumba kernel: CPU:    0
| Oct  5 21:08:33 rumba kernel: EIP:    0010:[<e0a019fb>]    Tainted: P 
| Oct  5 21:08:33 rumba kernel: EFLAGS: 00010286
| Oct  5 21:08:33 rumba kernel: eax: 00000005   ebx: 08094482   ecx: d27ea3e0   edx: c1807ea0
| Oct  5 21:08:33 rumba kernel: esi: 00000241   edi: 08094482   ebp: dcda5fbc   esp: dcda5f94
| Oct  5 21:08:33 rumba kernel: ds: 0018   es: 0018   ss: 0018
| Oct  5 21:08:33 rumba kernel: Process avfscoda (pid: 354, stackpage=dcda5000)
| Oct  5 21:08:33 rumba kernel: Stack: 08094482 00000241 000001b6 dd27e360 dcda4000 00000241 08094482 00000001 
| Oct  5 21:08:33 rumba kernel:        c0141df8 c0106e0c bffff6f8 c0106d1b 08094482 00000241 000001b6 00000241 
| Oct  5 21:08:33 rumba kernel:        08094482 bffff6f8 00000005 0000002b 0000002b 00000005 4017b2e4 00000023 
| Oct  5 21:08:33 rumba kernel: Call Trace: [sys_oldumount+12/16] [error_code+52/60] [system_call+51/56] 
| Oct  5 21:08:33 rumba kernel: 
| Oct  5 21:08:33 rumba kernel: Code:  Bad EIP value.
| Oct  5 21:08:33 rumba kernel:  <6>i8k: module unloaded
| Oct  5 21:08:35 rumba nmbd[7091]: [2002/10/05 21:08:35, 0] nmbd/nmbd.c:sig_term(63) 
| Oct  5 21:08:35 rumba nmbd[7091]:   Got SIGTERM: going down... 
| Oct  5 21:08:35 rumba xfs[593]: terminating 
| Oct  5 21:08:35 rumba xfs[594]: terminating 
| Oct  5 21:08:35 rumba ntpd[604]: ntpd exiting on signal 15
| Oct  5 21:08:36 rumba usbmgr[12064]: umount /proc/bus/usb
| Oct  5 21:08:36 rumba rpc.statd[265]: Caught signal 15, un-registering and exiting.
| Oct  5 21:08:36 rumba kernel: Kernel logging (proc) stopped.
| Oct  5 21:08:36 rumba kernel: Kernel log daemon terminating.
| Oct  5 21:08:36 rumba exiting on signal 15
`----

For a very short time (that laptop is *fast* :-) I've seen a
message about a failed umount, then it went down and never
came up again.


> > But how do I clear these structs?
> 
> Presuming that the metadata backups are intact, you need to "pvcreate -ff"
> the physical volumes and run vgcfgrestore on each of them.
> "vgscan ; vgchange -ay" should get you back to business afterwards.

Yes, I thought this would help, too. But it didn't. :-(

Commands always failed with "pv_read(): read" or "pv_read():
<something about creating names from kdev>". Because I
needed my laptop back up again today I restored my backup
yesterday evening, so unfortunately I can't help anymore to
find out what actually happened... :-(


cu,

   Raffi


-- 
=> Neu im Usenet? Fragen?    http://www.use-net.ch/usenet_intro_de.html <=
  The difference between theory and practice is that in theory, there is
                 no difference, but in practice, there is.
Raffael Herzog - herzog@raffael.ch - http://www.raffael.ch - ICQ #67961355

RE: [linux-lvm] How to fix inconsistent LV structs?

[Glenn Shannon]

Was this computer connected to a network when it went down?

Looks like a stack overflow to me, but where was the originator? Coda?
Unlikely but possible. Replaced syscalls usually (IIRC) indicate that
not only did a stack overflow occur but also that registers were
modified by whatever overflowed the stack. The kernel is noting itself
as tainted, which means that some form on non-GPLed module is running
(or has entered the stack to converse with the kernel directly, acting
as a module..) After the initial oops it appears to cascade to the rest
of the network-aware daemons, finally uprooting xfs and (presumably)
b0rking the drive(s).

Just a thought I had while reading the syslog.....forgive me if anything
I say is oncorrect I am by no means the kernel hacker that Heinz is :)

Glenn
--Dawn is Nature's way of telling you that it is time for bed.

-----Original Message-----
From: linux-lvm-admin@sistina.com [mailto:linux-lvm-admin@sistina.com]
On Behalf Of Raffael Herzog
Sent: Monday, October 07, 2002 2:19 AM
To: linux-lvm@sistina.com
Subject: Re: [linux-lvm] How to fix inconsistent LV structs?

Hi Heinz,

Heinz J . Mauelshagen wrote:

> Hmmm...
> Sounds like a nasty overwrite but it is hard to tell because you
> can't remmeber the exact details :(

Well, I can, the syslog is one of the only things that still
exist, besides the backup... :-) These are the last few
messages of the catastrophic reboot:

,----[ /var/log/syslog ]
| Oct  5 21:08:33 rumba kernel: Coda: Bye bye.
| Oct  5 21:08:33 rumba kernel: redir cleanup
| Oct  5 21:08:33 rumba kernel: replacing syscall nr.  12 [e0a01674]
with [c012e408]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr. 106 [e0a017a0]
with [c0134ad0]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr. 107 [e0a0184c]
with [c0134bb0]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr.  33 [e0a018fc]
with [c012e2dc]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr.   5 [e0a019c0]
with [c012ecb4]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr.  85 [e0a01a9c]
with [c0134ce0]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr. 183 [e0a01bd0]
with [c013ef44]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr. 195 [e0a01d7c]
with [c0134ebc]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr. 196 [e0a01e30]
with [c0134f30]
| Oct  5 21:08:33 rumba kernel: replacing syscall nr.  11 [e0a01f4c]
with [c0105a30]
| Oct  5 21:08:33 rumba kernel: Unable to handle kernel paging request
at virtual address e0a019fb
| Oct  5 21:08:33 rumba kernel:  printing eip:
| Oct  5 21:08:33 rumba kernel: e0a019fb
| Oct  5 21:08:33 rumba kernel: *pde = 01870067
| Oct  5 21:08:33 rumba kernel: *pte = 00000000
| Oct  5 21:08:33 rumba kernel: Oops: 0000
| Oct  5 21:08:33 rumba kernel: CPU:    0
| Oct  5 21:08:33 rumba kernel: EIP:    0010:[<e0a019fb>]    Tainted: P 
| Oct  5 21:08:33 rumba kernel: EFLAGS: 00010286
| Oct  5 21:08:33 rumba kernel: eax: 00000005   ebx: 08094482   ecx:
d27ea3e0   edx: c1807ea0
| Oct  5 21:08:33 rumba kernel: esi: 00000241   edi: 08094482   ebp:
dcda5fbc   esp: dcda5f94
| Oct  5 21:08:33 rumba kernel: ds: 0018   es: 0018   ss: 0018
| Oct  5 21:08:33 rumba kernel: Process avfscoda (pid: 354,
stackpage=dcda5000)
| Oct  5 21:08:33 rumba kernel: Stack: 08094482 00000241 000001b6
dd27e360 dcda4000 00000241 08094482 00000001 
| Oct  5 21:08:33 rumba kernel:        c0141df8 c0106e0c bffff6f8
c0106d1b 08094482 00000241 000001b6 00000241 
| Oct  5 21:08:33 rumba kernel:        08094482 bffff6f8 00000005
0000002b 0000002b 00000005 4017b2e4 00000023 
| Oct  5 21:08:33 rumba kernel: Call Trace: [sys_oldumount+12/16]
[error_code+52/60] [system_call+51/56] 
| Oct  5 21:08:33 rumba kernel: 
| Oct  5 21:08:33 rumba kernel: Code:  Bad EIP value.
| Oct  5 21:08:33 rumba kernel:  <6>i8k: module unloaded
| Oct  5 21:08:35 rumba nmbd[7091]: [2002/10/05 21:08:35, 0]
nmbd/nmbd.c:sig_term(63) 
| Oct  5 21:08:35 rumba nmbd[7091]:   Got SIGTERM: going down... 
| Oct  5 21:08:35 rumba xfs[593]: terminating 
| Oct  5 21:08:35 rumba xfs[594]: terminating 
| Oct  5 21:08:35 rumba ntpd[604]: ntpd exiting on signal 15
| Oct  5 21:08:36 rumba usbmgr[12064]: umount /proc/bus/usb
| Oct  5 21:08:36 rumba rpc.statd[265]: Caught signal 15, un-registering
and exiting.
| Oct  5 21:08:36 rumba kernel: Kernel logging (proc) stopped.
| Oct  5 21:08:36 rumba kernel: Kernel log daemon terminating.
| Oct  5 21:08:36 rumba exiting on signal 15
`----

For a very short time (that laptop is *fast* :-) I've seen a
message about a failed umount, then it went down and never
came up again.


> > But how do I clear these structs?
> 
> Presuming that the metadata backups are intact, you need to "pvcreate
-ff"
> the physical volumes and run vgcfgrestore on each of them.
> "vgscan ; vgchange -ay" should get you back to business afterwards.

Yes, I thought this would help, too. But it didn't. :-(

Commands always failed with "pv_read(): read" or "pv_read():
<something about creating names from kdev>". Because I
needed my laptop back up again today I restored my backup
yesterday evening, so unfortunately I can't help anymore to
find out what actually happened... :-(


cu,

   Raffi


-- 
=> Neu im Usenet? Fragen?    http://www.use-net.ch/usenet_intro_de.html
<=
  The difference between theory and practice is that in theory, there is
                 no difference, but in practice, there is.
Raffael Herzog - herzog@raffael.ch - http://www.raffael.ch - ICQ
#67961355

_______________________________________________
linux-lvm mailing list
linux-lvm@sistina.com
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO@http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] How to fix inconsistent LV structs?

[Raffael Herzog]

Hi Glenn,

Glenn Shannon wrote:

> Was this computer connected to a network when it went down?

Yes. I'm using the 3c95x driver.


> Looks like a stack overflow to me, but where was the originator? Coda?
> Unlikely but possible. Replaced syscalls usually (IIRC) indicate that
> not only did a stack overflow occur but also that registers were
> modified by whatever overflowed the stack.

Coda always replaces Syscalls, AFAIK. At least, it does so
on initialization.


> The kernel is noting itself
> as tainted, which means that some form on non-GPLed module is running
> (or has entered the stack to converse with the kernel directly, acting
> as a module..)

Correct, the nVidia driver taints the kernel.


cu,

   Raffi


-- 
=> Neu im Usenet? Fragen?    http://www.use-net.ch/usenet_intro_de.html <=
  The difference between theory and practice is that in theory, there is
                 no difference, but in practice, there is.
Raffael Herzog - herzog@raffael.ch - http://www.raffael.ch - ICQ #67961355

Re: [linux-lvm] How to fix inconsistent LV structs?

[Heinz J . Mauelshagen]

Raffael,

nothing in the log directly related to LVM :(
But your hint WRT naming devices could help us.
Do you have devfs mounted on /dev and don't use the full devfs
names in all cases?
If so, please retry giving the full names.

Regards,
Heinz    -- The LVM Guy --

On Mon, Oct 07, 2002 at 11:18:39AM +0200, Raffael Herzog wrote:
> Hi Heinz,
> 
> Heinz J . Mauelshagen wrote:
> 
> > Hmmm...
> > Sounds like a nasty overwrite but it is hard to tell because you
> > can't remmeber the exact details :(
> 
> Well, I can, the syslog is one of the only things that still
> exist, besides the backup... :-) These are the last few
> messages of the catastrophic reboot:
> 
> ,----[ /var/log/syslog ]
> | Oct  5 21:08:33 rumba kernel: Coda: Bye bye.
> | Oct  5 21:08:33 rumba kernel: redir cleanup
> | Oct  5 21:08:33 rumba kernel: replacing syscall nr.  12 [e0a01674] with [c012e408]
> | Oct  5 21:08:33 rumba kernel: replacing syscall nr. 106 [e0a017a0] with [c0134ad0]
> | Oct  5 21:08:33 rumba kernel: replacing syscall nr. 107 [e0a0184c] with [c0134bb0]
> | Oct  5 21:08:33 rumba kernel: replacing syscall nr.  33 [e0a018fc] with [c012e2dc]
> | Oct  5 21:08:33 rumba kernel: replacing syscall nr.   5 [e0a019c0] with [c012ecb4]
> | Oct  5 21:08:33 rumba kernel: replacing syscall nr.  85 [e0a01a9c] with [c0134ce0]
> | Oct  5 21:08:33 rumba kernel: replacing syscall nr. 183 [e0a01bd0] with [c013ef44]
> | Oct  5 21:08:33 rumba kernel: replacing syscall nr. 195 [e0a01d7c] with [c0134ebc]
> | Oct  5 21:08:33 rumba kernel: replacing syscall nr. 196 [e0a01e30] with [c0134f30]
> | Oct  5 21:08:33 rumba kernel: replacing syscall nr.  11 [e0a01f4c] with [c0105a30]
> | Oct  5 21:08:33 rumba kernel: Unable to handle kernel paging request at virtual address e0a019fb
> | Oct  5 21:08:33 rumba kernel:  printing eip:
> | Oct  5 21:08:33 rumba kernel: e0a019fb
> | Oct  5 21:08:33 rumba kernel: *pde = 01870067
> | Oct  5 21:08:33 rumba kernel: *pte = 00000000
> | Oct  5 21:08:33 rumba kernel: Oops: 0000
> | Oct  5 21:08:33 rumba kernel: CPU:    0
> | Oct  5 21:08:33 rumba kernel: EIP:    0010:[<e0a019fb>]    Tainted: P 
> | Oct  5 21:08:33 rumba kernel: EFLAGS: 00010286
> | Oct  5 21:08:33 rumba kernel: eax: 00000005   ebx: 08094482   ecx: d27ea3e0   edx: c1807ea0
> | Oct  5 21:08:33 rumba kernel: esi: 00000241   edi: 08094482   ebp: dcda5fbc   esp: dcda5f94
> | Oct  5 21:08:33 rumba kernel: ds: 0018   es: 0018   ss: 0018
> | Oct  5 21:08:33 rumba kernel: Process avfscoda (pid: 354, stackpage=dcda5000)
> | Oct  5 21:08:33 rumba kernel: Stack: 08094482 00000241 000001b6 dd27e360 dcda4000 00000241 08094482 00000001 
> | Oct  5 21:08:33 rumba kernel:        c0141df8 c0106e0c bffff6f8 c0106d1b 08094482 00000241 000001b6 00000241 
> | Oct  5 21:08:33 rumba kernel:        08094482 bffff6f8 00000005 0000002b 0000002b 00000005 4017b2e4 00000023 
> | Oct  5 21:08:33 rumba kernel: Call Trace: [sys_oldumount+12/16] [error_code+52/60] [system_call+51/56] 
> | Oct  5 21:08:33 rumba kernel: 
> | Oct  5 21:08:33 rumba kernel: Code:  Bad EIP value.
> | Oct  5 21:08:33 rumba kernel:  <6>i8k: module unloaded
> | Oct  5 21:08:35 rumba nmbd[7091]: [2002/10/05 21:08:35, 0] nmbd/nmbd.c:sig_term(63) 
> | Oct  5 21:08:35 rumba nmbd[7091]:   Got SIGTERM: going down... 
> | Oct  5 21:08:35 rumba xfs[593]: terminating 
> | Oct  5 21:08:35 rumba xfs[594]: terminating 
> | Oct  5 21:08:35 rumba ntpd[604]: ntpd exiting on signal 15
> | Oct  5 21:08:36 rumba usbmgr[12064]: umount /proc/bus/usb
> | Oct  5 21:08:36 rumba rpc.statd[265]: Caught signal 15, un-registering and exiting.
> | Oct  5 21:08:36 rumba kernel: Kernel logging (proc) stopped.
> | Oct  5 21:08:36 rumba kernel: Kernel log daemon terminating.
> | Oct  5 21:08:36 rumba exiting on signal 15
> `----
> 
> For a very short time (that laptop is *fast* :-) I've seen a
> message about a failed umount, then it went down and never
> came up again.
> 
> 
> > > But how do I clear these structs?
> > 
> > Presuming that the metadata backups are intact, you need to "pvcreate -ff"
> > the physical volumes and run vgcfgrestore on each of them.
> > "vgscan ; vgchange -ay" should get you back to business afterwards.
> 
> Yes, I thought this would help, too. But it didn't. :-(
> 
> Commands always failed with "pv_read(): read" or "pv_read():
> <something about creating names from kdev>". Because I
> needed my laptop back up again today I restored my backup
> yesterday evening, so unfortunately I can't help anymore to
> find out what actually happened... :-(
> 
> 
> cu,
> 
>    Raffi
> 
> 
> -- 
> => Neu im Usenet? Fragen?    http://www.use-net.ch/usenet_intro_de.html <=
>   The difference between theory and practice is that in theory, there is
>                  no difference, but in practice, there is.
> Raffael Herzog - herzog@raffael.ch - http://www.raffael.ch - ICQ #67961355
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [linux-lvm] How to fix inconsistent LV structs?

[Raffael Herzog]

Hi Heinz,

Heinz J . Mauelshagen wrote:

> nothing in the log directly related to LVM :(
> But your hint WRT naming devices could help us.
> Do you have devfs mounted on /dev and don't use the full devfs
> names in all cases?

I don't use devfs at all (yet), but the device nodes were
present all the time. What do you mean with "full devfs
names"?

Maybe these excerpts of the output of pvdata help (this is
from a run after I did pvcreate -ff):

,----[ pvdata -d /dev/hda7 ]
| [...]
| <1> lv_copy_from_disk -- CALLED
| <1> lv_copy_from_disk -- LEAVING
| <1> lv_check_consistency -- CALLED
| <22> lv_check_name -- CALLED with lv_name: "HM^A"
| <333> lvm_check_chars -- CALLED with name: "HM^A"
| <333> lvm_check_chars -- LEAVING with ret: -100
| <22> lv_check_name -- LEAVING with ret: -132
| <1> lv_check_consistency -- LEAVING with ret: -149
| pvdata -- logical volume struct at offset   0 is inconsistent
| [...]
| 
| <1> lv_copy_from_disk -- CALLED
| <1> lv_copy_from_disk -- LEAVING
| <1> lv_check_consistency -- CALLED
| <22> lv_check_name -- CALLED with lv_name: ""
| <22> lv_check_name -- LEAVING with ret: -132
| <1> lv_check_consistency -- LEAVING with ret: -149
| pvdata -- logical volume struct at offset  13 is inconsistent
| <1> lv_copy_from_disk -- CALLED
| <1> lv_copy_from_disk -- LEAVING
| <1> lv_check_consistency -- CALLED
| <22> lv_check_name -- CALLED with lv_name: "ªªªªªªªªªªªªªªªªªªªªªªªªªªªªªªªªªªªª
| <22> lv_check_name -- LEAVING with ret: -132
| <1> lv_check_consistency -- LEAVING with ret: -149
| pvdata -- logical volume struct at offset  14 is inconsistent
| [...]
| 
| <1> lv_copy_from_disk -- CALLED
| <1> lv_copy_from_disk -- LEAVING
| --- NEW Physical volume ---
| PV Name               /dev/hda7
| VG Name               
| PV Size               32.44 GB [68035212 secs]
| PV#                   0
| PV Status             NOT available
| Allocatable           NO
| Cur LV                0
| PE Size (KByte)       0
| Total PE              0
| Free PE               0
| Allocated PE          0
| PV UUID               0KMQsG-r21J-GwKA-qpZD-1FC9-plP0-t0aryJ
| 
| --- Volume group ---
| VG Name               
| VG Access             read/write
| VG Status             NOT available/resizable
| VG #                  0
| MAX LV                256
| Cur LV                3
| Open LV               0
| MAX LV Size           255.99 GB
| Max PV                256
| Cur PV                1
| Act PV                1
| VG Size               32.44 GB
| PE Size               4 MB
| Total PE              8304
| Alloc PE / Size       4375 / 17.09 GB
| Free  PE / Size       3929 / 15.35 GB
| VG UUID               BC9DHB-T9v3-6LNt-7qtv-7uHE-7pg2-K4flga
| 
| --- List of logical volumes ---
| 
| pvdata -- logical volume struct at offset   1 is empty
| [...]
| <1> lv_copy_from_disk -- CALLED
| <1> lv_copy_from_disk -- LEAVING
| <1> lv_check_consistency -- CALLED
| <22> lv_check_name -- CALLED with lv_name: "ªªªªªªªªªªªªªªªªªªªªªªªªªªªªªªªªªªªª
| <22> lv_check_name -- LEAVING with ret: -132
| <1> lv_check_consistency -- LEAVING with ret: -149
| pvdata -- logical volume struct at offset 125 is inconsistent
| [like this through to LV 139]
| [...]
| 
| <1> lv_copy_from_disk -- CALLED
| <1> lv_copy_from_disk -- LEAVING
| <1> pv_read_uuidlist -- CALLED with /dev/hda7
| <1> pv_read_uuidlist -- LEAVING with ret: 0
| <1> lvm_check_uuid -- CALLED with uuidstr: "(null)"
| <1> lvm_check_uuid -- LEAVING with ret: -1
| 
| Segmentation fault.
`----

So you see, there was really a lot of garbage... :-)

I also saved the outputs of pvscan -d and vgscan -d, in case
you think they help.


> If so, please retry giving the full names.

I can't try anything anymore because I decided not to use
LVM anymore at least until I know what happened and I know
that it won't happen again. Sure, it's probably not the
fault of LVM, but I think, if I would have been using "plain
ext2/3", I probably wouldn't have lost just *all* of my data
(lucky that I had a very young backup, so I actually didn't
loose anything).


cu,

   Raffi


-- 
=> Neu im Usenet? Fragen?    http://www.use-net.ch/usenet_intro_de.html <=
  The difference between theory and practice is that in theory, there is
                 no difference, but in practice, there is.
Raffael Herzog - herzog@raffael.ch - http://www.raffael.ch - ICQ #67961355

Re: [linux-lvm] How to fix inconsistent LV structs?

[Heinz J . Mauelshagen]

On Mon, Oct 07, 2002 at 03:14:06PM +0200, Raffael Herzog wrote:
> Hi Heinz,
> 
> Heinz J . Mauelshagen wrote:
> 
> > nothing in the log directly related to LVM :(
> > But your hint WRT naming devices could help us.
> > Do you have devfs mounted on /dev and don't use the full devfs
> > names in all cases?
> 
> I don't use devfs at all (yet), but the device nodes were
> present all the time. What do you mean with "full devfs
> names"?

I meant nasty devfs names like /dev/scsi/host0/bus0/target0/lun0/part1 ;)

But as you say, you don't use it, which brings us back to the overwrite
assumption and the necessary vgcfgrestore's.

I recommend to save the metadata of all PVs belonging to the gone VG
for potential later analysis with "dd if=/dev/PV of=PV.vgda bs=1k count=4k".
Insert all your PVs (say AllPVs) in sequence for "PV"
(i.e. for PV in AllPVs; do dd if=/dev/$PV of=${PV}.vgda bs=1k count=4k;done).

The procedure to restore your metadata to all those PVs goes:

- run "pvcreate -yff AllPVs"
- run "for PV in AllPVs;do vgcfgrestore -n NameOfTheVG /dev/$PV;done"
- run "vgscan ; vgchange -ay"

Should be it...

> 
> Maybe these excerpts of the output of pvdata help (this is
> from a run after I did pvcreate -ff):
> 
> ,----[ pvdata -d /dev/hda7 ]
> | [...]
> | <1> lv_copy_from_disk -- CALLED
> | <1> lv_copy_from_disk -- LEAVING
> | <1> lv_check_consistency -- CALLED
<SNIP>
> `----
> 
> So you see, there was really a lot of garbage... :-)

Was likely if overwriten.

> 
> I also saved the outputs of pvscan -d and vgscan -d, in case
> you think they help.

Well, they should just harden the garbage ;)

> 
> 
> > If so, please retry giving the full names.
> 
> I can't try anything anymore because I decided not to use
> LVM anymore at least until I know what happened and I know
> that it won't happen again. Sure, it's probably not the
> fault of LVM, but I think, if I would have been using "plain
> ext2/3", I probably wouldn't have lost just *all* of my data
> (lucky that I had a very young backup, so I actually didn't
> loose anything).

That depends on the nature of the overwrite. Could at least kill a filesystem
if enough is written to the device.

Regards,
Heinz    -- The LVM Guy --

> 
> cu,
> 
>    Raffi
> 
> 
> -- 
> => Neu im Usenet? Fragen?    http://www.use-net.ch/usenet_intro_de.html <=
>   The difference between theory and practice is that in theory, there is
>                  no difference, but in practice, there is.
> Raffael Herzog - herzog@raffael.ch - http://www.raffael.ch - ICQ #67961355
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-