[linux-lvm] pvmoving from a pv with bad sectors

[linux-lvm] pvmoving from a pv with bad sectors

[Erkki Seppala]

It turns out that a 60GB disc I bought some months ago has regions
that cannot be read. I'm going to get the disc to warranty, but before
that I'd like to remove the disc from the VG. But, this happens:

[12:10] lectric(pts/1):/var/log% sudo pvmove /dev/ide/host0/bus0/target1/lun0/part1
pvmove -- moving physical extents in active volume group "archive"
pvmove -- WARNING: if you lose power during the move you may need
        to restore your LVM metadata from backup!
pvmove -- do you want to continue? [y/n] y
pvmove -- ERROR reading input physical volume "/dev/ide/host0/bus0/target1/lun0/part1" (still 1310720 bytes to read)
pvmove -- ERROR "pv_move_pe(): read input PV" pv_move_pe
pvmove -- ERROR "pv_move_pe(): read input PV" moving physical extents

/proc/lvm/global says:
VG:  archive  [3 PV, 6 LV/5 open]  PE Size: 16384 KB
  Usage [KB/PE]: 238665728 /14567 total  172032000 /10500 used  66633728 /4067 free
  PVs: [AN] ide/host0/bus0/target1/lun0/part1 60030976 /3664    21331968 /1302    38699008 /2362  
       [AA] ide/host0/bus1/target1/lun0/part1 58605568 /3577    58605568 /3577           0 /0     
       [AA] ide/host0/bus1/target0/lun0/part1 120029184 /7326    92094464 /5621    27934720 /1705  

The first PV is the one I'm trying to empty, and the last PV has the
space to hold it. Consequent attempt:

[16:06] lectric(pts/1):/var/log% sudo pvmove -v /dev/ide/host0/bus0/target1/lun0/part1
pvmove -- checking name of source physical volume "/dev/ide/host0/bus0/target1/lun0/part1"
pvmove -- locking logical volume manager
pvmove -- reading data of source physical volume from "/dev/ide/host0/bus0/target1/lun0/part1"
pvmove -- ERROR "pv_read(): read" reading source physical volume "/dev/ide/host0/bus0/target1/lun0/part1"

Now, I know there's some data lost, but they don't seem to affect the
actual filesystem structure integrity, only data, which isn't really
that important in that VG. But if I don't have the device around, I
imagine I lose the whole volume group, not just the data's that is on
the one disc. I don't mind if the unreadable blocks are replaced with
zeroed blocks, as long as I can get rid of the device :-).

Any suggestions?

-- 
  _____________________________________________________________________
     / __// /__ ____  __                              Erkki Sepp�l�\   \
    / /_ / // // /\ \/ //ircnet                           Modeemi Ry\  /
   /_/  /_/ \___/ /_/\_\@modeemi.fi        http://www.modeemi.fi/~flux/

Re: [linux-lvm] pvmoving from a pv with bad sectors

[Erkki Seppala]

It appears I didn't try rescuing the data a moment too soon, as the
drive stopped giving me data shortly after writing that message..

And as I predicted, /dev/archive is gone. Is there a way to retrieve
it, I imagine the lost device physically had only one logical volume
in it?  The device had 20 gigabytes of data. I imagine reiserfs cannot
handle so big holes in filesystem, so I'll just hope the hole hits one
of the smaller, backed up logical volumes instead of the one, huge,
non-backed up volume :).

The output of vgscan -d is available at
http://xulfad.inside.org/~flux/vgscan . Also the latest backup of LVM
is at http://xulfad.inside.org/~flux/archive.conf .

I imagine I need to somehow remove the invalid device from the volume
group, but as I cannot even activate it, it cannot be done with
default utilities? Also the now-gone-and-disconnected physical volume
contained some data, and cannot be removed from the volume group
before the VG knows the device is empty. I can propably get a new disc
next week, would that make things easier?

-- 
  _____________________________________________________________________
     / __// /__ ____  __                              Erkki Sepp�l�\   \
    / /_ / // // /\ \/ //ircnet                           Modeemi Ry\  /
   /_/  /_/ \___/ /_/\_\@modeemi.fi        http://www.modeemi.fi/~flux/

Re: [linux-lvm] rescuing a VG with one missing PV

[Erkki Seppala]

The drive I got was two gigabytes smaller than the original, and thus:

[21:42] lectric(pts/8):~# vgcfgrestore -n archive /dev/ide/host0/bus0/target1/lun0/part1
vgcfgrestore -- size of physical volume /dev/ide/host0/bus0/target1/lun0/part1 differs from backup

The original disc was:

  PVs: [AN] ide/host0/bus0/target1/lun0/part1 60030976 /3664    
		21331968 /1302    38699008/2362  

and it now is 58633312 blocks according to fdisk.

Is there a way to still restore the remaining VG?-o Sounds like the
backup would need to be modified. The physical extents that were in
the dead device would need to be either removed from the VG or
allocated from another device?
-- 
  _____________________________________________________________________
     / __// /__ ____  __                              Erkki Sepp�l�\   \
    / /_ / // // /\ \/ //ircnet                           Modeemi Ry\  /
   /_/  /_/ \___/ /_/\_\@modeemi.fi        http://www.modeemi.fi/~flux/

Re: [linux-lvm] rescuing a VG with one missing PV

[Heinz J . Mauelshagen]

On Mon, Feb 11, 2002 at 10:53:25PM +0200, Erkki Seppala wrote:
> The drive I got was two gigabytes smaller than the original, and thus:
> 
> [21:42] lectric(pts/8):~# vgcfgrestore -n archive /dev/ide/host0/bus0/target1/lun0/part1
> vgcfgrestore -- size of physical volume /dev/ide/host0/bus0/target1/lun0/part1 differs from backup
> 
> The original disc was:
> 
>   PVs: [AN] ide/host0/bus0/target1/lun0/part1 60030976 /3664    
> 		21331968 /1302    38699008/2362  
> 
> and it now is 58633312 blocks according to fdisk.

Change your partition to have the very same size as before and vgcfgrestore.

> 
> Is there a way to still restore the remaining VG?-o Sounds like the
> backup would need to be modified. The physical extents that were in
> the dead device would need to be either removed from the VG or
> allocated from another device?
> -- 
>   _____________________________________________________________________
>      / __// /__ ____  __                              Erkki Seppälä\   \
>     / /_ / // // /\ \/ //ircnet                           Modeemi Ry\  /
>    /_/  /_/ \___/ /_/\_\@modeemi.fi        http://www.modeemi.fi/~flux/
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

-- 

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [linux-lvm] rescuing a VG with one missing PV

[Erkki Seppala]

On Tue, Feb 12, 2002 at 10:22:28AM +0100, Heinz J . Mauelshagen wrote:
> On Mon, Feb 11, 2002 at 10:53:25PM +0200, Erkki Seppala wrote:
> > The drive I got was two gigabytes smaller than the original, and thus:
> > 
> > The original disc was:
> > 
> >   PVs: [AN] ide/host0/bus0/target1/lun0/part1 60030976 /3664    
> > 		21331968 /1302    38699008/2362  
> > 
> > and it now is 58633312 blocks according to fdisk.
> 
> Change your partition to have the very same size as before and vgcfgrestore.

Thanks for the suggestion, and I would very much like to do that, but
as I said, the replacement drive is a bit smaller than the old one,
thus I cannot make as big partition :-).

I was considering hacking vgcfgrestore to ignore the fact that the
size mismatches, are there problems to be expected? Like severe VG
corruption.. I might just as well try anything, it can't be any worse
than the current situation.

-- 
  _____________________________________________________________________
     / __// /__ ____  __                              Erkki Sepp�l�\   \
    / /_ / // // /\ \/ //ircnet                           Modeemi Ry\  /
   /_/  /_/ \___/ /_/\_\@modeemi.fi        http://www.modeemi.fi/~flux/

Re: [linux-lvm] rescuing a VG with one missing PV

[Heinz J . Mauelshagen]

On Tue, Feb 12, 2002 at 02:58:17PM +0200, Erkki Seppala wrote:
> On Tue, Feb 12, 2002 at 10:22:28AM +0100, Heinz J . Mauelshagen wrote:
> > On Mon, Feb 11, 2002 at 10:53:25PM +0200, Erkki Seppala wrote:
> > > The drive I got was two gigabytes smaller than the original, and thus:
> > > 
> > > The original disc was:
> > > 
> > >   PVs: [AN] ide/host0/bus0/target1/lun0/part1 60030976 /3664    
> > > 		21331968 /1302    38699008/2362  
> > > 
> > > and it now is 58633312 blocks according to fdisk.
> > 
> > Change your partition to have the very same size as before and vgcfgrestore.
> 
> Thanks for the suggestion, and I would very much like to do that, but
> as I said, the replacement drive is a bit smaller than the old one,
> thus I cannot make as big partition :-).

Sorry, missed that :-(

> 
> I was considering hacking vgcfgrestore to ignore the fact that the
> size mismatches, are there problems to be expected? Like severe VG
> corruption.. I might just as well try anything, it can't be any worse
> than the current situation.

Get LVM from CVS at www.sistina.com and either use the -s option of pvcreate
I implemented recently to fake the size you need or run "vgcfgrestore -i ..."
which ignores the size of the device and restores metadata anyway
at your own risk.

Both will only help if you don't have data in the last 2 GB, because you
can't pvmove it away. Even though you lost your data anyway with the old drive,
you need to either pvmove it away to get rid of allocated extents in that PV
again or lvreduce LVs to make them free which you potentially don't like,
because the extents might be further to the beginning of the LV.

> 
> -- 
>   _____________________________________________________________________
>      / __// /__ ____  __                              Erkki Seppälä\   \
>     / /_ / // // /\ \/ //ircnet                           Modeemi Ry\  /
>    /_/  /_/ \___/ /_/\_\@modeemi.fi        http://www.modeemi.fi/~flux/
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [linux-lvm] rescuing a VG with one missing PV

[Erkki Seppala]

On Tue, Feb 12, 2002 at 03:05:51PM +0100, Heinz J . Mauelshagen wrote:
> Get LVM from CVS at www.sistina.com and either use the -s option of pvcreate
> I implemented recently to fake the size you need or run "vgcfgrestore -i ..."
> which ignores the size of the device and restores metadata anyway
> at your own risk.

Thanks! This seems to be doing the trick. There seemed to be one typo
in the CVS at the moment I checked out my version, but I imagine it'll
be fixed as soon as someone tries to compile it :-) :

---8<---
Index: vgscan.c
===================================================================
RCS file: /data/cvs/LVM/tools/vgscan.c,v
retrieving revision 1.12
diff -u -r1.12 vgscan.c
--- vgscan.c    2002/02/08 14:59:37     1.12
+++ vgscan.c    2002/02/12 21:29:00
@@ -294,7 +294,7 @@
                for ( blk_dev = l = 0; l < vg->lv_max; l++) {
                   lv = vg->lv[l];
                   if ( lv == NULL) continue;
-                  if ( lvm_tab_check_free_lv_number ( lv) == FALSE) {
+                  if ( lvm_tab_check_free_lv_numbers ( lv) == FALSE) {
                      printf ( "%s -- changing minor number on \"%s\"\n",
                               cmd, lv->lv_name);
                      if ( lv->lv_access & LV_SNAPSHOT_ORG) {
---8<---

> Even though you lost your data anyway with the old drive, you need
> to either pvmove it away to get rid of allocated extents in that PV
> again or lvreduce LVs to make them free which you potentially don't
> like, because the extents might be further to the beginning of the
> LV.

I don't completely understand this, but I guess I will when the pvmove
has completed - it's a surprisingly slow operation. I was glad to
notice though that only one logical volume was in the failed disc.

Once again, thanks for the help :).
-- 
  _____________________________________________________________________
     / __// /__ ____  __                              Erkki Sepp�l�\   \
    / /_ / // // /\ \/ //ircnet                           Modeemi Ry\  /
   /_/  /_/ \___/ /_/\_\@modeemi.fi        http://www.modeemi.fi/~flux/

Re: [linux-lvm] rescuing a VG with one missing PV

[Heinz J . Mauelshagen]

On Tue, Feb 12, 2002 at 11:37:14PM +0200, Erkki Seppala wrote:
> On Tue, Feb 12, 2002 at 03:05:51PM +0100, Heinz J . Mauelshagen wrote:
> > Get LVM from CVS at www.sistina.com and either use the -s option of pvcreate
> > I implemented recently to fake the size you need or run "vgcfgrestore -i ..."
> > which ignores the size of the device and restores metadata anyway
> > at your own risk.
> 
> Thanks! This seems to be doing the trick. There seemed to be one typo
> in the CVS at the moment I checked out my version, but I imagine it'll
> be fixed as soon as someone tries to compile it :-) :

No, that's actually a new function which checks for a particular lv device
number in use. Strange that you couldn't compile it.
Could you please try a checkout from scratch and send me the error you see.
Thanks.

> 
> ---8<---
> Index: vgscan.c
> ===================================================================
> RCS file: /data/cvs/LVM/tools/vgscan.c,v
> retrieving revision 1.12
> diff -u -r1.12 vgscan.c
> --- vgscan.c    2002/02/08 14:59:37     1.12
> +++ vgscan.c    2002/02/12 21:29:00
> @@ -294,7 +294,7 @@
>                 for ( blk_dev = l = 0; l < vg->lv_max; l++) {
>                    lv = vg->lv[l];
>                    if ( lv == NULL) continue;
> -                  if ( lvm_tab_check_free_lv_number ( lv) == FALSE) {
> +                  if ( lvm_tab_check_free_lv_numbers ( lv) == FALSE) {
>                       printf ( "%s -- changing minor number on \"%s\"\n",
>                                cmd, lv->lv_name);
>                       if ( lv->lv_access & LV_SNAPSHOT_ORG) {
> ---8<---
> 
> > Even though you lost your data anyway with the old drive, you need
> > to either pvmove it away to get rid of allocated extents in that PV
> > again or lvreduce LVs to make them free which you potentially don't
> > like, because the extents might be further to the beginning of the
> > LV.
> 
> I don't completely understand this, but I guess I will when the pvmove
> has completed - it's a surprisingly slow operation.

Well, it needs to copy a lot of data ;-)
Get back to me if there's any further problems.

> I was glad to
> notice though that only one logical volume was in the failed disc.
> 
> Once again, thanks for the help :).
> -- 
>   _____________________________________________________________________
>      / __// /__ ____  __                              Erkki Seppälä\   \
>     / /_ / // // /\ \/ //ircnet                           Modeemi Ry\  /
>    /_/  /_/ \___/ /_/\_\@modeemi.fi        http://www.modeemi.fi/~flux/
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

-- 

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-