[linux-lvm] hd read error / some PEs won't pvmove - how to resolve?

[linux-lvm] hd read error / some PEs won't pvmove - how to resolve?

[rick]

Hi people,

I've searched the net, the mailing lists, and the documentation for the
solution to this, but I don't see anything obvious.

I had partitions on 45GB and 20GB drives in a volume group. Problems
developed with the 20GB drive, where certain blocks became undreadable.
I added a 13GB drive, and juggled some files to come up with another 8GB on
the 45GB, and pvmoved all the data off the 20 GB drive.

All except for 8 PEs, that is. pvmove fails with an error when I try to
move those particular PEs where the read errors are occuring. 

I don't need to worry about recovering that data. None of the affected
files are important enough to worry about. But I do need to remove the 
PV on the 20GB drive from the VG, which I can't do while these 8 PEs 
are still assigned to my LV. 

So! The only hint at a solution I've found for this in the mailing lists
is to replace the 20GB drive with an equal or greater size drive, and 
use vgcfgrestore to restore the PV information on that drive. I would assume
the PEs will still be allocated, but they'll just be empty of data. From 
that point on, recovery is a filesystem layer issue.

The problem is, I don't have a 20GB or greater drive lying around to replace
this one with until I get this one replaced on warranty, which I can't send
in for warranty until I can remove it from the volume group. Chicken and egg.

The other option I can consider is a painful piece-by-piece move of the data
from this VG into a new VG, gradually reducing one VG and expanding the other
until there's no data left on the first VG. If there's no better option, I'll
do this, but I'm not excited about it. Can I do this within the existing volume
group, or will the currently allocated PEs cause problems later on when I try
to remove that failed drive? (vgsplit?)

Any help appreciated, much thanks.


Rick

Re: [linux-lvm] hd read error / some PEs won't pvmove - how to resolve?

[Andreas Dilger]

Rick, you write:
> All except for 8 PEs, that is. pvmove fails with an error when I try to
> move those particular PEs where the read errors are occuring. 

Yes, AIX always had this same problem.  However, because you could mirror
inside the LVM on AIX, you could mirror the bad LEs, and then remove the
old PV because there were 2 copies of that LE.  Generally you were better
off throwing the whole filesystem out and restoring from a backup, if you
didn't have a mirrored LV in the first place.  AIX also warned you at the
time of temporary (rather than permanent) read errors, so sometimes you
had a chance to replace the disk before it totally crapped out.

> I don't need to worry about recovering that data. None of the affected
> files are important enough to worry about. But I do need to remove the 
> PV on the 20GB drive from the VG, which I can't do while these 8 PEs 
> are still assigned to my LV. 

If the bad PEs are in the middle of the filesystem, you are pretty much
hosed in terms of recovery for this filesystem, regardless of whether
you can force LVM to move the PEs for you (if you are lucky fsck may be
able to recover, but it really depends on where/how big the bad PEs are
and how you initially configured the filesystem).  Hopefully the bad
PEs are only in one LV.

I would simply back up what files you can (via tar) and make a new LV
and filesystem on the other disk.  You _should_ be able to delete the
LV. Once the LV using those PEs is gone is gone you should be able to
remove the PV.

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert