From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx1.redhat.com (ext-mx11.extmail.prod.ext.phx2.redhat.com [10.5.110.16]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id r4VHKcWf028176 for ; Fri, 31 May 2013 13:20:39 -0400 Received: from mail-pd0-f171.google.com (mail-pd0-f171.google.com [209.85.192.171]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id r4VHKbug013555 for ; Fri, 31 May 2013 13:20:37 -0400 Received: by mail-pd0-f171.google.com with SMTP id z11so2542648pdj.30 for ; Fri, 31 May 2013 10:20:37 -0700 (PDT) Received: from [10.10.102.70] (207.47.42.180.static.nextweb.net. [207.47.42.180]) by mx.google.com with ESMTPSA id ov2sm47456659pbc.34.2013.05.31.10.20.36 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 31 May 2013 10:20:36 -0700 (PDT) Message-ID: <51A8DBE4.607@gmail.com> Date: Fri, 31 May 2013 10:20:36 -0700 From: Christian Lahti MIME-Version: 1.0 References: <51A79EEE.6020308@gmail.com> In-Reply-To: <51A79EEE.6020308@gmail.com> Content-Type: multipart/alternative; boundary="------------020605080408050502090306" Subject: Re: [linux-lvm] file-based locking initialization failed after pvresize Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: To: linux-lvm@redhat.com This is a multi-part message in MIME format. --------------020605080408050502090306 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit I was assisted on the #lvm IRC channel by kabi_ (thank you), the solution for others who might find themselves in the same boat: * look in /etc/lvm/archive and find the file that has the information just prior to the destructive act * in this text file is a UUID for the volume that has been corrupted * boot into rescue disk (or single user mode if the corrupted disk is not your boot volume) and run the following pvcreate --config 'global{locking_type=0}' --uuid --restorefile /etc/lvm/backup/ /dev/ vgcfgrestore -f /etc/lvm/archive/ Thanks again! /Christian On 5/30/2013 11:48 AM, Christian Lahti wrote: > Hello all: > > I have a VMWare virtual CentOS 6 server with the following disk layout: > > /dev/sda 15GB > /dev/sdb 100GB > > Standard CentOS LVM setup > > on sda1 > /dev/vg_centos6/lv_root mounted on / > /dev/vg_centos6/lv_swap > > on sdb1 > /dev/vg_centos6/lv_var mounted on /var > > I was starting to run low on disk space for /var, so I shut down the > machine gracefully, extended the disk in VMWare to 250G and rebooted > in single user mode. I then ran the following two commands: > > pvresize --setphysicalvolumesize 250G /dev/sdb1 > lvresize -L250G /dev/vg_centos6/lv_var > > I then rebooted expecting the /var to now be 250G instead of 100G. > What I got was: > > No such file or directory trying to open /dev/vg_centos6/lv_var > > After dropping to a maintenance shell I could not run any pv* or lv* > commands, always resulting in the error: > > File-based locking initialization failed > > After Googling forever I found some lv* and pv* commands take > --ignorelockingfailure as a parameter, so now I can "see" the pv and > lv information but I do not know how to recover this partition: > > pvdisplay --ignorelockingfailure /dev/sdb1 > File-based locking initialization failed > --- Physical volume --- > PV Name /dev/sdb1 > VG Name vg_centos6 > PV Size 250 GiB / not useable 3.00 MiB > Allocatable yes > PE Size 4.00 MiB > Total PE 63999 > Free PE 25599 > Allocated PE 38400 > > lvdisplay --ignorelockingfailure /dev/vg_centos6/lv_var > File-based locking initialization failed > --- Logical volume --- > LV Path /dev/vg_centos6/lv_var > LV Name lv_var > VG Name vg_centos6 > LV Write Access read/write > LV Status suspended > # open 0 > LV Size 150.00 GiB > Current LE 38400 > Segments 1 > Allocation inherit > Read ahead sectors auto > - currently set to 256 > Block device 253:2 > > I cannot mount this, I cannot fsck, /dev/mapper/vg_centos6/lv_var does > not even get created. This is a critical server, can anyone please > tell me how to reverse this or at least recover the data? > > Thanks! > > /Christian --------------020605080408050502090306 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
I was assisted on the #lvm IRC channel by kabi_ (thank you), the solution for others who might find themselves in the same boat:
  • look in /etc/lvm/archive and find the file that has the information just prior to the destructive act
  • in this text file is a UUID for the volume that has been corrupted
  • boot into rescue disk (or single user mode if the corrupted disk is not your boot volume) and run the following
    pvcreate --config 'global{locking_type=0}' --uuid <UUID> --restorefile /etc/lvm/backup/<file containing previous info> /dev/<device>
    vgcfgrestore -f /etc/lvm/archive/<file containing previous info> <volume group name>

Thanks again!

/Christian

On 5/30/2013 11:48 AM, Christian Lahti wrote:
Hello all:

I have a VMWare virtual CentOS 6 server with the following disk layout:

/dev/sda 15GB
/dev/sdb 100GB

Standard CentOS LVM setup

on sda1
/dev/vg_centos6/lv_root mounted on /
/dev/vg_centos6/lv_swap

on sdb1
/dev/vg_centos6/lv_var mounted on /var

I was starting to run low on disk space for /var, so I shut down the machine gracefully, extended the disk in VMWare to 250G and rebooted in single user mode. I then ran the following two commands:

pvresize --setphysicalvolumesize 250G /dev/sdb1
lvresize -L250G /dev/vg_centos6/lv_var

I then rebooted expecting the /var to now be 250G instead of 100G. What I got was:

No such file or directory trying to open /dev/vg_centos6/lv_var

After dropping to a maintenance shell I could not run any pv* or lv* commands, always resulting in the error:

File-based locking initialization failed

After Googling forever I found some lv* and pv* commands take --ignorelockingfailure as a parameter, so now I can "see" the pv and lv information but I do not know how to recover this partition:

pvdisplay --ignorelockingfailure /dev/sdb1
File-based locking initialization failed
--- Physical volume ---
PV Name /dev/sdb1
VG Name vg_centos6
PV Size 250 GiB / not useable 3.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 63999
Free PE 25599
Allocated PE 38400

lvdisplay --ignorelockingfailure /dev/vg_centos6/lv_var
File-based locking initialization failed
--- Logical volume ---
LV Path /dev/vg_centos6/lv_var
LV Name lv_var
VG Name vg_centos6
LV Write Access read/write
LV Status suspended
# open 0
LV Size 150.00 GiB
Current LE 38400
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:2

I cannot mount this, I cannot fsck, /dev/mapper/vg_centos6/lv_var does not even get created. This is a critical server, can anyone please tell me how to reverse this or at least recover the data?

Thanks!

/Christian

--------------020605080408050502090306--