[linux-lvm] LVM disk died

[linux-lvm] LVM disk died

[Melinda Taylor]

I was searching through the LVM mailing list archives and someone said:

 > I read the FAQs and one of them shows how to recover from a dead drive. It
 > is says you lose the data in the Logical Volume but I'm hoping just the
 > data on the physical disk that died and not the entire volume. Yep, as 
explained above

can someone direct me to the faq that shows me how to recover from a dead 
drive.

I had LVM spanning 4 HDDs and one will no longer spin up. That was also the 
disk housing the / filesystem. The / filesystem was not LVM but ext3 but 
there was an LVM partition on there. I have backups, thank god but they a 
couple of days old.

I have to reinstall the Operating system (redhat 7.3) from scratch, if I 
install as I did before, will I be able to recover the data on the 
remaining 3 disks? A friend said it should be as easy as a couple of 
commands but I am still such a newbie to LVM and this is the first time, 
since installing it I have had to think about it again.

Any tips anyone can offer would be appreciated or directions to the faq 
mentioned above. I searched the sistina site but to no avail.

Many Thanks,

Melinda

Re: [linux-lvm] LVM disk died

[Gert van der Knokke]

Melinda Taylor wrote:

> can someone direct me to the faq that shows me how to recover from a 
> dead drive.
>
> I had LVM spanning 4 HDDs and one will no longer spin up. 

Hi Melinda,

Sometimes a 'dead' drive can be brought back to (temporary) live again 
by giving it a hard knock or better: a sharp twist, as the heads tend to 
stick to the surface of the platters. Mind though that such a drive is 
not trustworthy anymore but if it does spin up you might be able to get 
the data off it.

Best way to this is to take the drive out of the PC and  hold it flat in 
your hand and give it a sharp twist in the horizontal direction. This  
helped me  a few times with such 'sticky' drives. After that, mount it 
back in the PC and try it.

Good luck,
Gert

Re: [linux-lvm] LVM disk died

[Melinda Taylor]

On Sun, 2003-07-13 at 18:03, Gert van der Knokke wrote:
> Melinda Taylor wrote:
> 
> > can someone direct me to the faq that shows me how to recover from a 
> > dead drive.
> >
> > I had LVM spanning 4 HDDs and one will no longer spin up. 
> 
> Hi Melinda,
> 
> Sometimes a 'dead' drive can be brought back to (temporary) live again 
> by giving it a hard knock or better: a sharp twist, as the heads tend to 
> stick to the surface of the platters. Mind though that such a drive is 
> not trustworthy anymore but if it does spin up you might be able to get 
> the data off it.
> 
> Best way to this is to take the drive out of the PC and  hold it flat in 
> your hand and give it a sharp twist in the horizontal direction. This  
> helped me  a few times with such 'sticky' drives. After that, mount it 
> back in the PC and try it.

Hi Gert,

I'll try what you suggest tomorrow. It is all very strange since my last
email another of the 4 disks now has the same error. I am that unlucky
that 2 disks fail? 

When the machine boots the SCSI bios says:

ASYN start unit request failed

It does seem to be spinning.I can hear it and if I rotate it I can feel
the centrifugal force from the disks spinning inside.

If i boot with my linux boot disk it sees the drives sda sdb and the two
partitions on each but it says 

"sda spinning up disk....................." 

and ultimately fails It can also identify the vendor and model number on
the linux boot.

I'll take this to another list but its just interesting so I thought I
would post (I'd still like some advice for my original question though
regarding rebuilding from what is now 2 disks of 4 in my LVM)

Thanks,

Melinda

[linux-lvm] Enough rope to hang self (was: LVM disk died)

[William Blunn]

> I had LVM spanning 4 HDDs and one will no longer spin up.

Made me think.

If you have a VG with one PV, you're no worse off for resilience than if
you didn't have LVM.

Trouble with LVM is that it tempts people into having a VG made up from
several disks. People then end up gambling that none of their disks will
stop working and trash the VG.

About every couple of weeks, someone pipes up with "One of the disks in
my VG has died. Please help!".

You give people enough rope and they will hang themselves.

If there is to be more than one PV in a VG, it makes sense to make all
the PVs be resilient devices (e.g. RAID). Then when a hard disk goes
down (as it inevitably will), it won't trash the VG.

Makes me wonder if there is a way of making LVM encourage people to do
it properly.

For a VG with a single PV, LVM could let you use just any old device.

For a VG with more than one PV, LVM could offer warnings to the effect
that it would be a good idea to make the PVs be resilient devices.

Perhaps at the point where the number of PVs change from one or less to
more than one, LVM could foist the advice upon the user.

"If you wish to have more than on PV in a VG, please see
 /usr/share/doc/lvm/multiple_pv.txt".

This file would explain the issues and at the end, tell you the
following:

    It is necessary to have a file:

      /etc/lvm/dont_blame_lvm

    in order to be able to have more than one PV in a VG.

Once the file exists, LVM will let you do what you want.

Bill
-- 
William H. Blunn - <bill+s.9sfuw6uo@tao-group.com> - Developer Support
Tao
62/63 Suttons Business Park, Earley, READING, RG6 1AZ, United Kingdom
Tel: +44 118 901 2999 - Fax: +44 118 901 2963 - http://tao-group.com/

The contents of this e-mail and any attachments are confidential and may
be legally privileged. If you have received this e-mail and you are not
a named addressee, please inform us as soon as possible on
+44 118 901 2999 and then delete the e-mail from your system. If you are
not a named addressee you must not copy, use, disclose, distribute,
print or rely on this e-mail. Any views expressed in this e-mail or any
attachments may not necessarily reflect those of Tao's management.
Although we routinely screen for viruses, addressees should scan this
e-mail and any attachments for viruses. Tao makes no representation or
warranty as to the absence of viruses in this e-mail or any attachments.
Please note that for the protection of our business, we may monitor and
read e-mails sent to and from our server(s).

[linux-lvm] LVM recovery

[Melinda Taylor]

Hi All,

I haven't yet recieved any advice on how to recover any of the data on
my remaining 2 of 4 disks but i think I know what to do.....think.....

I originally setup the system with the following steps:

- vgscan
vgscan -- reading all physical volumes (this may take a while...)
vgscan -- "/etc/lvmtab" and "/etc/lvmtab.d" successfully created
vgscan -- WARNING: This program does not do a VGDA backup of your volume
group
- pvcreate /dev/sda2 ; pvcreate /dev/sdb1; pvcreate /dev/sdc2; pvcreate
/dev/sdd2
- vgcreate vg01 /dev/sda2 /dev/sdb1 /dev/sdc2 /dev/sdd2
- check no of LE pvdata -E /dev/sdc2 (5710 LE)
 lvcreate -l 5711 -n lv03 vg01 /dev/sdc2
- Make ext3 fs on each LV
 mkfs.ext3 /dev/vg01/lv01

/dev/sda2 and /dev/sdb1 are kaput (both drives died within days of
eachother).

I believe I had 4 logical volumes.

/dev/vg01/lv01
/dev/vg01/lv02
/dev/vg01/lv03
/dev/vg01/lv04

in my volume group.

So I presume this means each LV resided on a separate drive and probably
lv01 and lv02 were the 2 on the 2 dead disk drives.

    * PVs on four SCSI devices
    * A VG called "vg01" comprised of said PVs

If I reinstall lvm on my newly installed system can I recover lv03 and
lv04?

I have had a search and see something called importvg which I think may
be what I need.

Had anyone ever had any experience recovering a volume group with
missing disks?

I have backups of my lvm config but it seems like lvmtab etc are all
binary files, are they of any use to me?

Many Thanks,

melinda

Re: [linux-lvm] LVM recovery

[Melinda Taylor]

Ok I am now getting somewhere with this:


I reinstalled LVM (but I am not sure what version I was running I think
it was 1.0.5 - will this make a difference?)

"pvscan" told me remaining LV and the VG they belong to:

pvscan -- reading all physical volumes (this may take a while...)
pvscan -- inactive PV "/dev/sdc1" of VG "vg02" [9.85 GB / 0 free]
pvscan -- inactive PV "/dev/sdc2"  is associated to unknown VG "vg01"
(run vgscan)
pvscan -- inactive PV "/dev/sdd1" of VG "vg02" [19.54 GB / 0 free]
pvscan -- inactive PV "/dev/sdd2"  is associated to unknown VG "vg01"
(run vgscan)
pvscan -- total: 4 [64.35 GB] / in use: 4 [64.35 GB] / in no VG: 0 [0]

vg02 was comprised of /dev/sdc1 and /dev/sdd1 which was /sw and /tmp so
it can be rebuilt as sdc and sdd both still exist

I tried to rebuild it: vgimport vg02 /dev/sdc1 /dev/sdd1

and recieved the error:

vgimport -- error: physical volume "dev/sdc1" doesn't belong to exported
volume group

So I tried: 

vgimport -f vg02 /dev/sdc1 /dev/sdd1
vgimport -- physical volumes "/dev/sdd1" and "/dev/sdc1" are in
different volume groups

Run vgscan:

[root@astro melinda]# /sbin/vgscan
vgscan -- reading all physical volumes (this may take a while...)
vgscan -- found inactive volume group "vg02"
vgscan -- ERROR "vg_read_with_pv_and_lv(): current PV" can't get data of
volume group "vg01" from physical volume(s)
vgscan -- "/etc/lvmtab" and "/etc/lvmtab.d" successfully created
vgscan -- WARNING: This program does not do a VGDA backup of your volume
groups

Obviously vg01 is going to be the biggest problem but as both lv's from
vg02 still exist - how do I make it active?

I then tried to mount it but it creates an error: /dev/vg02/lv02 is not
a valid block device but it is there. So I am getting somewhere now at
least but I presume I need to make everything active.

Thanks,

melinda



On Thu, 2003-07-17 at 10:35, Melinda Taylor wrote:
> Hi All,
> 
> I haven't yet recieved any advice on how to recover any of the data on
> my remaining 2 of 4 disks but i think I know what to do.....think.....
> 
> I originally setup the system with the following steps:
> 
> - vgscan
> vgscan -- reading all physical volumes (this may take a while...)
> vgscan -- "/etc/lvmtab" and "/etc/lvmtab.d" successfully created
> vgscan -- WARNING: This program does not do a VGDA backup of your volume
> group
> - pvcreate /dev/sda2 ; pvcreate /dev/sdb1; pvcreate /dev/sdc2; pvcreate
> /dev/sdd2
> - vgcreate vg01 /dev/sda2 /dev/sdb1 /dev/sdc2 /dev/sdd2
> - check no of LE pvdata -E /dev/sdc2 (5710 LE)
>  lvcreate -l 5711 -n lv03 vg01 /dev/sdc2
> - Make ext3 fs on each LV
>  mkfs.ext3 /dev/vg01/lv01
> 
> /dev/sda2 and /dev/sdb1 are kaput (both drives died within days of
> eachother).
> 
> I believe I had 4 logical volumes.
> 
> /dev/vg01/lv01
> /dev/vg01/lv02
> /dev/vg01/lv03
> /dev/vg01/lv04
> 
> in my volume group.
> 
> So I presume this means each LV resided on a separate drive and probably
> lv01 and lv02 were the 2 on the 2 dead disk drives.
> 
>     * PVs on four SCSI devices
>     * A VG called "vg01" comprised of said PVs
> 
> If I reinstall lvm on my newly installed system can I recover lv03 and
> lv04?
> 
> I have had a search and see something called importvg which I think may
> be what I need.
> 
> Had anyone ever had any experience recovering a volume group with
> missing disks?
> 
> I have backups of my lvm config but it seems like lvmtab etc are all
> binary files, are they of any use to me?
> 
> Many Thanks,
> 
> melinda
> 
> 
> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] LVM recovery

[Melinda Taylor]

Oh ok stupid question 

vgchange -a y 

activates vg02 now back to vg01 the one that is missing 2 partitions, is
it impossible to create this vg01 without the other 2 partitions?

Thanks,

melinda


On Thu, 2003-07-17 at 14:20, Melinda Taylor wrote:
> 
> Ok I am now getting somewhere with this:
> 
> 
> I reinstalled LVM (but I am not sure what version I was running I think
> it was 1.0.5 - will this make a difference?)
> 
> "pvscan" told me remaining LV and the VG they belong to:
> 
> pvscan -- reading all physical volumes (this may take a while...)
> pvscan -- inactive PV "/dev/sdc1" of VG "vg02" [9.85 GB / 0 free]
> pvscan -- inactive PV "/dev/sdc2"  is associated to unknown VG "vg01"
> (run vgscan)
> pvscan -- inactive PV "/dev/sdd1" of VG "vg02" [19.54 GB / 0 free]
> pvscan -- inactive PV "/dev/sdd2"  is associated to unknown VG "vg01"
> (run vgscan)
> pvscan -- total: 4 [64.35 GB] / in use: 4 [64.35 GB] / in no VG: 0 [0]
> 
> vg02 was comprised of /dev/sdc1 and /dev/sdd1 which was /sw and /tmp so
> it can be rebuilt as sdc and sdd both still exist
> 
> I tried to rebuild it: vgimport vg02 /dev/sdc1 /dev/sdd1
> 
> and recieved the error:
> 
> vgimport -- error: physical volume "dev/sdc1" doesn't belong to exported
> volume group
> 
> So I tried: 
> 
> vgimport -f vg02 /dev/sdc1 /dev/sdd1
> vgimport -- physical volumes "/dev/sdd1" and "/dev/sdc1" are in
> different volume groups
> 
> Run vgscan:
> 
> [root@astro melinda]# /sbin/vgscan
> vgscan -- reading all physical volumes (this may take a while...)
> vgscan -- found inactive volume group "vg02"
> vgscan -- ERROR "vg_read_with_pv_and_lv(): current PV" can't get data of
> volume group "vg01" from physical volume(s)
> vgscan -- "/etc/lvmtab" and "/etc/lvmtab.d" successfully created
> vgscan -- WARNING: This program does not do a VGDA backup of your volume
> groups
> 
> Obviously vg01 is going to be the biggest problem but as both lv's from
> vg02 still exist - how do I make it active?
> 
> I then tried to mount it but it creates an error: /dev/vg02/lv02 is not
> a valid block device but it is there. So I am getting somewhere now at
> least but I presume I need to make everything active.
> 
> Thanks,
> 
> melinda
> 
> 
> 
> On Thu, 2003-07-17 at 10:35, Melinda Taylor wrote:
> > Hi All,
> > 
> > I haven't yet recieved any advice on how to recover any of the data on
> > my remaining 2 of 4 disks but i think I know what to do.....think.....
> > 
> > I originally setup the system with the following steps:
> > 
> > - vgscan
> > vgscan -- reading all physical volumes (this may take a while...)
> > vgscan -- "/etc/lvmtab" and "/etc/lvmtab.d" successfully created
> > vgscan -- WARNING: This program does not do a VGDA backup of your volume
> > group
> > - pvcreate /dev/sda2 ; pvcreate /dev/sdb1; pvcreate /dev/sdc2; pvcreate
> > /dev/sdd2
> > - vgcreate vg01 /dev/sda2 /dev/sdb1 /dev/sdc2 /dev/sdd2
> > - check no of LE pvdata -E /dev/sdc2 (5710 LE)
> >  lvcreate -l 5711 -n lv03 vg01 /dev/sdc2
> > - Make ext3 fs on each LV
> >  mkfs.ext3 /dev/vg01/lv01
> > 
> > /dev/sda2 and /dev/sdb1 are kaput (both drives died within days of
> > eachother).
> > 
> > I believe I had 4 logical volumes.
> > 
> > /dev/vg01/lv01
> > /dev/vg01/lv02
> > /dev/vg01/lv03
> > /dev/vg01/lv04
> > 
> > in my volume group.
> > 
> > So I presume this means each LV resided on a separate drive and probably
> > lv01 and lv02 were the 2 on the 2 dead disk drives.
> > 
> >     * PVs on four SCSI devices
> >     * A VG called "vg01" comprised of said PVs
> > 
> > If I reinstall lvm on my newly installed system can I recover lv03 and
> > lv04?
> > 
> > I have had a search and see something called importvg which I think may
> > be what I need.
> > 
> > Had anyone ever had any experience recovering a volume group with
> > missing disks?
> > 
> > I have backups of my lvm config but it seems like lvmtab etc are all
> > binary files, are they of any use to me?
> > 
> > Many Thanks,
> > 
> > melinda
> > 
> > 
> > 
> > 
> > _______________________________________________
> > linux-lvm mailing list
> > linux-lvm@sistina.com
> > http://lists.sistina.com/mailman/listinfo/linux-lvm
> > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 
> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] LVM recovery

[Emmanuel Varagnat]

Melinda Taylor wrote:

> - vgscan
> vgscan -- reading all physical volumes (this may take a while...)
> vgscan -- "/etc/lvmtab" and "/etc/lvmtab.d" successfully created
> vgscan -- WARNING: This program does not do a VGDA backup of your volume
> group
> - pvcreate /dev/sda2 ; pvcreate /dev/sdb1; pvcreate /dev/sdc2; pvcreate
> /dev/sdd2
> - vgcreate vg01 /dev/sda2 /dev/sdb1 /dev/sdc2 /dev/sdd2
> - check no of LE pvdata -E /dev/sdc2 (5710 LE)
>  lvcreate -l 5711 -n lv03 vg01 /dev/sdc2
> - Make ext3 fs on each LV
>  mkfs.ext3 /dev/vg01/lv01
> 
> /dev/sda2 and /dev/sdb1 are kaput (both drives died within days of
> eachother).
> 
> I believe I had 4 logical volumes.

I would say that it can depend on what is the block allocation policy of 
the LVM system...

> So I presume this means each LV resided on a separate drive and probably
> lv01 and lv02 were the 2 on the 2 dead disk drives.
> 
>     * PVs on four SCSI devices
>     * A VG called "vg01" comprised of said PVs
> 
> If I reinstall lvm on my newly installed system can I recover lv03 and
> lv04?

You should try LVM2 that can deal with partial/truncated volumes.

-=( manu )=-

Re: [linux-lvm] LVM recovery

[Melinda Taylor]

Ok I have installed LVM2 with readline support and I tried:

lvm> vgchange --partial -a y
  Partial mode. Incomplete volume groups will be activated read-only.
  2 PV(s) found for VG vg01: expected 4
  Logical volume (lv02) contains an incomplete mapping table.
  2 PV(s) found for VG vg01: expected 4
  Logical volume (lv02) contains an incomplete mapping table.
  2 PV(s) found for VG vg01: expected 4
  Logical volume (lv02) contains an incomplete mapping table.
  2 PV(s) found for VG vg01: expected 4
  Logical volume (lv02) contains an incomplete mapping table.
  2 PV(s) found for VG vg01: expected 4
  Logical volume (lv02) contains an incomplete mapping table.
  2 PV(s) found for VG vg01: expected 4
  Logical volume (lv02) contains an incomplete mapping table.
  2 PV(s) found for VG vg01: expected 4
  Logical volume (lv02) contains an incomplete mapping table.
  2 PV(s) found for VG vg01: expected 4
  Logical volume (lv02) contains an incomplete mapping table.
  2 PV(s) found for VG vg01: expected 4
  Logical volume (lv02) contains an incomplete mapping table.
  4 logical volume(s) in volume group "vg01" now active
  2 logical volume(s) in volume group "vg02" now active

I then tried to mount my VG with the missing PV's (VG01) with the line 

[root@astro root]# mount -o ro /dev/vg01/lv03 /home2
mount: wrong fs type, bad option, bad superblock on /dev/vg01/lv03,
       or too many mounted file systems


and I see the following in /var/log/messages:

Jul 21 11:53:48 astro kernel: EXT3-fs: INFO: recovery required on readonly
filesystem.
Jul 21 11:53:48 astro kernel: EXT3-fs: write access unavailable, cannot proceed.
Jul 21 11:53:48 astro kernel: EXT2-fs: device-mapper(254,4): couldn't mount
because of unsupported optional features (4).

Anyone have any ideas on this one? I am getting closer I guess. I just want to
grab the data off the remaining 2 LV if possible.

Many Thanks,

melinda







Quoting Emmanuel Varagnat <emmanuel.varagnat@free.fr>:

> Melinda Taylor wrote:
> 
> > - vgscan
> > vgscan -- reading all physical volumes (this may take a while...)
> > vgscan -- "/etc/lvmtab" and "/etc/lvmtab.d" successfully created
> > vgscan -- WARNING: This program does not do a VGDA backup of your volume
> > group
> > - pvcreate /dev/sda2 ; pvcreate /dev/sdb1; pvcreate /dev/sdc2; pvcreate
> > /dev/sdd2
> > - vgcreate vg01 /dev/sda2 /dev/sdb1 /dev/sdc2 /dev/sdd2
> > - check no of LE pvdata -E /dev/sdc2 (5710 LE)
> >  lvcreate -l 5711 -n lv03 vg01 /dev/sdc2
> > - Make ext3 fs on each LV
> >  mkfs.ext3 /dev/vg01/lv01
> > 
> > /dev/sda2 and /dev/sdb1 are kaput (both drives died within days of
> > eachother).
> > 
> > I believe I had 4 logical volumes.
> 
> I would say that it can depend on what is the block allocation policy of 
> the LVM system...
> 
> > So I presume this means each LV resided on a separate drive and probably
> > lv01 and lv02 were the 2 on the 2 dead disk drives.
> > 
> >     * PVs on four SCSI devices
> >     * A VG called "vg01" comprised of said PVs
> > 
> > If I reinstall lvm on my newly installed system can I recover lv03 and
> > lv04?
> 
> You should try LVM2 that can deal with partial/truncated volumes.
> 
> -=( manu )=-
> 
> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 

Re: [linux-lvm] LVM recovery

[Alasdair G Kergon]

On Mon, Jul 21, 2003 at 11:59:38AM +1000, Melinda Taylor wrote:
> lvm> vgchange --partial -a y

First off, you should take complete disk-level backups of your remaining disks
in the VG.

If the messages are accurate, there's likely to be only a problem with lv02, 
not lv03 which you're trying to mount, so try a normal (rw) mount of
lv03 and see if the ext3 recovery copes, or else try fsck.  If you took
the disk-level backups first you can restore the disks from the backups
if fsck makes things worse and you need to try other restoration
methods.

*AFTER* you've completed extracting your data, you can use LVM2's vgreduce
to tidy up the LVM configuration, removing the lost PVs and
partially-missing LV completely. But read the vgreduce man page first
and use --test before running it for real to check it's going to do
something sensible in your circumstances.

Alasdair

Re: [linux-lvm] LVM disk died

[Emmanuel Varagnat]

Melinda Taylor wrote:
> 
> I was searching through the LVM mailing list archives and someone said:
> 
>  > I read the FAQs and one of them shows how to recover from a dead 
> drive. It
>  > is says you lose the data in the Logical Volume but I'm hoping just the
>  > data on the physical disk that died and not the entire volume. Yep, 
> as explained above
> 
> can someone direct me to the faq that shows me how to recover from a 
> dead drive.

I don't know if you already fixed your problem, but you can try 
e2retrieve (http://coredump.free.fr/linux) that I just released.

-=( manu )=-