[linux-lvm] How to fix missmatch between VG-size and LV-size?

[linux-lvm] How to fix missmatch between VG-size and LV-size?

[Fredrik Skog]

[-- Attachment #1: Type: text/plain, Size: 1890 bytes --]

Hello,

While adding a new disk to my LV i ended up with a crash during the resize2fs. Now I have LVM reporting different sizes for my VG and LV.
How do I fix this?

I tried to do a "vgreduce vgftp /dev/sde1" to remove my new drive again. This only gives me an error "Physical volume "/dev/sde1" still in use"
If I can revert the adding of the new drive, I planned on trying to vgcfgrestore to an archived config from before I added the drive and did the lvextend and resize2fs.
Is this a good way of solving the problem? I guess it's not possible to lvreduce the LV by the same amount i did lvextend it before, because the system still thinks the LV is the old size. I just want to get rid of the PV i added to the VG

I'm a bit confused on how to do this.

# vgdisplay
  --- Volume group ---
  VG Name               vgftp
  System ID             
  Format                lvm2
  Metadata Areas        13
  Metadata Sequence No  67
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               0
  Max PV                0
  Cur PV                12
  Act PV                12
  VG Size               5.59 TiB
  PE Size               4.00 MiB
  Total PE              1464180
  Alloc PE / Size       1089649 / 4.16 TiB
  Free  PE / Size       374531 / 1.43 TiB
  VG UUID               kOApX5-oTeV-cPWr-h41J-NxML-sbbj-WcKObC

 # lvdisplay
  --- Logical volume ---
  LV Name                /dev/vgftp/lvftp
  VG Name                vgftp
  LV UUID                l4EWSh-GOfD-n0Sx-7ZSG-RTVi-p7fC-ApyHZS
  LV Write Access        read/write
  LV Status              NOT available
  LV Size                4.16 TiB
  Current LE             1089649
  Segments               14
  Allocation             inherit
  Read ahead sectors     auto
   

thank you
/Fredrik Skog

[-- Attachment #2: Type: text/html, Size: 5413 bytes --]

Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

[Ray Morris]

> While adding a new disk to my LV i ended up with a crash during the
> resize2fs.

resize2fs would be run AFTER lvextend, so if resize2fs was running, 
the LV would need to be the new, larger size.  Yet, you said 
"the system still thinks the LV is the old size".  Did you run
vgcfgrestore already, restoring to the point between where you 
ran vgextend and lvextend?  Perhaps you forgot to run lvextend, 
but then one would expect resize2fs to tell you it was already 
full sized.  Double check what you did, because this information
is not consistent.

> If I can revert the adding of the new drive, I planned on
> trying to vgcfgrestore to an archived config from before I added the
> drive and did the lvextend and resize2fs. 

Indeed you could "undo" the vgextend and lvextend via vgcfgrestore, 
or via lvreduce + vgreduce, but first you probably need to undo 
resize2fs.  Use fsck to get the filesystem consistent, if possible.
Then resize it back to the old smaller size.  At this point you 
can move forward by using lvextend, then resize2fs. You could also 
move backward from that point with lvreduce and vgreduce.
-- 
Ray Morris
support@bettercgi.com

Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/

Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/

Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php




On Tue, 29 Mar 2011 17:21:55 +0200
"Fredrik Skog" <fredrik.skog@rodang.se> wrote:

> Hello,
> 
> While adding a new disk to my LV i ended up with a crash during the
> resize2fs. Now I have LVM reporting different sizes for my VG and LV.
> How do I fix this?
> 
> I tried to do a "vgreduce vgftp /dev/sde1" to remove my new drive
> again. This only gives me an error "Physical volume "/dev/sde1" still
> in use" If I can revert the adding of the new drive, I planned on
> trying to vgcfgrestore to an archived config from before I added the
> drive and did the lvextend and resize2fs. Is this a good way of
> solving the problem? I guess it's not possible to lvreduce the LV by
> the same amount i did lvextend it before, because the system still
> thinks the LV is the old size. I just want to get rid of the PV i
> added to the VG
> 
> I'm a bit confused on how to do this.
> 
> # vgdisplay
>   --- Volume group ---
>   VG Name               vgftp
>   System ID             
>   Format                lvm2
>   Metadata Areas        13
>   Metadata Sequence No  67
>   VG Access             read/write
>   VG Status             resizable
>   MAX LV                0
>   Cur LV                1
>   Open LV               0
>   Max PV                0
>   Cur PV                12
>   Act PV                12
>   VG Size               5.59 TiB
>   PE Size               4.00 MiB
>   Total PE              1464180
>   Alloc PE / Size       1089649 / 4.16 TiB
>   Free  PE / Size       374531 / 1.43 TiB
>   VG UUID               kOApX5-oTeV-cPWr-h41J-NxML-sbbj-WcKObC
> 
>  # lvdisplay
>   --- Logical volume ---
>   LV Name                /dev/vgftp/lvftp
>   VG Name                vgftp
>   LV UUID                l4EWSh-GOfD-n0Sx-7ZSG-RTVi-p7fC-ApyHZS
>   LV Write Access        read/write
>   LV Status              NOT available
>   LV Size                4.16 TiB
>   Current LE             1089649
>   Segments               14
>   Allocation             inherit
>   Read ahead sectors     auto
>    
> 
> thank you
> /Fredrik Skog

Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

[Fredrik Skog]

>> While adding a new disk to my LV i ended up with a crash during the
>> resize2fs.
>
> resize2fs would be run AFTER lvextend, so if resize2fs was running,
> the LV would need to be the new, larger size.  Yet, you said
> "the system still thinks the LV is the old size".  Did you run
> vgcfgrestore already, restoring to the point between where you
> ran vgextend and lvextend?  Perhaps you forgot to run lvextend,
> but then one would expect resize2fs to tell you it was already
> full sized.  Double check what you did, because this information
> is not consistent.

Thank you for your reply
Sorry, maybe my english is a bit confusing. I'll try again.

1> I made a partition /dev/sde1 with Linux LVM
2> Run a "pvcreate /dev/sde1"
3> Run "vgextend vgftp /dev/sd1"
4> Run "lvextend -L+400G /dev/vgftp/lvftp"
5> Run "umount /dev/vgftp/lvftp"
6> Run "e2fsck -f /dev/vgftp/lvftp"
7> Run "resize2fs /dev/vgftp/lvftp"
    resize2fs 1.41.10 (10-Feb-2009)
    Resizing the filesystem on /dev/vgftp/lvftp to 1115800576 (4k) blocks
8> Here it all hangs. I cant do anything with the filesystem or LVM. every 
command i do hangs.
     I edited fstab so my LVM is not remounted on reboot, and rebooted.
     Here I am right now.

I can run LVM commands fine now after the reboot and the output from 
pvdisplay and lvdisplay is like i posted earlier.
How can i revert this in a safe way? I was thinking of just removing the PV 
and do a vgcfgrestore to "before" the lvextend.
But since lvdisplay says the volume is 4.16TiB and pvdisplay says 5.59TiB 
something is wrong? Or am I missing something?

I tried a " vgreduce vgftp /dev/sde1" to remove my new drive again, but this 
only gives me an error "Physical volume "/dev/sde1" still in use"
I think this is strange because the pvdisplay seems to think i have not yet 
added the PV but pvdisplay does.
If i do a lvreduce i fear something will break.

Is it better to do a e2fsck now?

Thank you for your help

>
>> If I can revert the adding of the new drive, I planned on
>> trying to vgcfgrestore to an archived config from before I added the
>> drive and did the lvextend and resize2fs.
>
> Indeed you could "undo" the vgextend and lvextend via vgcfgrestore,
> or via lvreduce + vgreduce, but first you probably need to undo
> resize2fs.  Use fsck to get the filesystem consistent, if possible.
> Then resize it back to the old smaller size.  At this point you
> can move forward by using lvextend, then resize2fs. You could also
> move backward from that point with lvreduce and vgreduce.
> -- 
> Ray Morris
> support@bettercgi.com
>
> Strongbox - The next generation in site security:
> http://www.bettercgi.com/strongbox/
>
> Throttlebox - Intelligent Bandwidth Control
> http://www.bettercgi.com/throttlebox/
>
> Strongbox / Throttlebox affiliate program:
> http://www.bettercgi.com/affiliates/user/register.php
>
>
>
>
> On Tue, 29 Mar 2011 17:21:55 +0200
> "Fredrik Skog" <fredrik.skog@rodang.se> wrote:
>
>> Hello,
>>
>> While adding a new disk to my LV i ended up with a crash during the
>> resize2fs. Now I have LVM reporting different sizes for my VG and LV.
>> How do I fix this?
>>
>> I tried to do a "vgreduce vgftp /dev/sde1" to remove my new drive
>> again. This only gives me an error "Physical volume "/dev/sde1" still
>> in use" If I can revert the adding of the new drive, I planned on
>> trying to vgcfgrestore to an archived config from before I added the
>> drive and did the lvextend and resize2fs. Is this a good way of
>> solving the problem? I guess it's not possible to lvreduce the LV by
>> the same amount i did lvextend it before, because the system still
>> thinks the LV is the old size. I just want to get rid of the PV i
>> added to the VG
>>
>> I'm a bit confused on how to do this.
>>
>> # vgdisplay
>>   --- Volume group ---
>>   VG Name               vgftp
>>   System ID
>>   Format                lvm2
>>   Metadata Areas        13
>>   Metadata Sequence No  67
>>   VG Access             read/write
>>   VG Status             resizable
>>   MAX LV                0
>>   Cur LV                1
>>   Open LV               0
>>   Max PV                0
>>   Cur PV                12
>>   Act PV                12
>>   VG Size               5.59 TiB
>>   PE Size               4.00 MiB
>>   Total PE              1464180
>>   Alloc PE / Size       1089649 / 4.16 TiB
>>   Free  PE / Size       374531 / 1.43 TiB
>>   VG UUID               kOApX5-oTeV-cPWr-h41J-NxML-sbbj-WcKObC
>>
>>  # lvdisplay
>>   --- Logical volume ---
>>   LV Name                /dev/vgftp/lvftp
>>   VG Name                vgftp
>>   LV UUID                l4EWSh-GOfD-n0Sx-7ZSG-RTVi-p7fC-ApyHZS
>>   LV Write Access        read/write
>>   LV Status              NOT available
>>   LV Size                4.16 TiB
>>   Current LE             1089649
>>   Segments               14
>>   Allocation             inherit
>>   Read ahead sectors     auto
>>

Thanks
 /Fredrik Skog 

Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

[Stuart D. Gathman]

On Tue, 29 Mar 2011, Fredrik Skog wrote:

> 1> I made a partition /dev/sde1 with Linux LVM
> 2> Run a "pvcreate /dev/sde1"
> 3> Run "vgextend vgftp /dev/sd1"
> 4> Run "lvextend -L+400G /dev/vgftp/lvftp"
> 5> Run "umount /dev/vgftp/lvftp"
> 6> Run "e2fsck -f /dev/vgftp/lvftp"
> 7> Run "resize2fs /dev/vgftp/lvftp"
>   resize2fs 1.41.10 (10-Feb-2009)
>   Resizing the filesystem on /dev/vgftp/lvftp to 1115800576 (4k) blocks
> 8> Here it all hangs. I cant do anything with the filesystem or LVM. every 
> command i do hangs.

The resize will take a very long time.  Did you try these commands on another
terminal?  Did you check the disk activity light?

>    I edited fstab so my LVM is not remounted on reboot, and rebooted.
>    Here I am right now.

That was probably a bad idea if the disk was still busy extending your
filesystem.

> I can run LVM commands fine now after the reboot and the output from 
> pvdisplay and lvdisplay is like i posted earlier.
> How can i revert this in a safe way? I was thinking of just removing the PV 
> and do a vgcfgrestore to "before" the lvextend.
> But since lvdisplay says the volume is 4.16TiB and pvdisplay says 5.59TiB 
> something is wrong? Or am I missing something?

lvdisplay will almost never show the same as pvdisplay.  One displays
*logical* volumes and the other *physical* volumes.

> I tried a " vgreduce vgftp /dev/sde1" to remove my new drive again, but this 
> only gives me an error "Physical volume "/dev/sde1" still in use"

Of course, because your extended LV is using it.

> I think this is strange because the pvdisplay seems to think i have not yet 
> added the PV but pvdisplay does.

Presumably you meant "lvdisplay" for one of the above.  What does
lvdisplay show for the size of your LV?  Also, try out "lvs" and "pvs"
for shorter output.  Include output in your response.

> If i do a lvreduce i fear something will break.

Even more than it is now, yes.

> Is it better to do a e2fsck now?

That is probably the only way to salvage what is left of your filesystem,
but don't do it until we find out whether the LV is the old or the new size.
I suspect the LV will be the new size, but the filesystem may still show
the old size.  Since the resize2fs should be just adding free space, 
there may not be any data loss.  e2fsck would fix the (now corrupt)
free space, and you can extend again.

--
 	      Stuart D. Gathman <stuart@bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

[Ray Morris]

7> Run "resize2fs /dev/vgftp/lvftp"  
>    resize2fs 1.41.10 (10-Feb-2009)
>    Resizing the filesystem on /dev/vgftp/lvftp to 1115800576 (4k)
> blocks
> Resizing the filesystem on /dev/vgftp/lvftp to 1115800576 (4k)
> blocks
8> Here it all hangs. I cant do anything with the filesystem or LVM.
8> every 

You can expect resize2fs to take a couple of hours on a 4 TB 
filesystem, so the "crash" may well be nothing but impatience.

> If i do a lvreduce i fear something will break.
> Is it better to do a e2fsck now?

The problem that we know about is that the resize2fs was
interrupted.  Reports defer in the level in danger in that, 
but definitely we want to get the filesystem consistent and
small enough before we pull the block device out from underneath
it.  So yes, you need a passing e2fsck before an lvreduce at 
this points.

The safest thing to do would be make a copy of at least the LV, 
if not both PVs, so you can get back to where you are if needed.
That's almost always the first best step for any kind of data
recovery type of effort.

Once you have a copy or have checked that your backups are Ok, 
you can proceed with e2fsck. After e2fsck passes, you can try 
to pvmove the extents off of the new drive, if you wish.  If 
the LV won't fit on the old drive, you'll need to resize2fs 
and lvreduce to make it fit.  It can be hard to know just how
big to make the LV for a given FS size, due to filesystem overhead
and such.  A safe way to do it would be to resize2fs as small 
as possible first. That'll probably give you at least 10%-20% 
of margin between the size of the FS and the size of the PV.
Then resize the LV to match the PV.  Once the LV is at it's final 
size, resize2fs to max.

Check that your LV is in fact the same size as it was 
before your lvextend and the output of "history". It sounds
like either a) you didn't run exactly the commands that you
are reporting or b) the LV is in fact larger than it used 
to be.  If you ran lvextend to increase the size of the LV
and did not reduce it after, a crash during resize2fs isn't 
going to shrink it automagically.  Also, if it were at the 
old size, it probably wouldn't be on both PVs.  Therefore, 
the LV probably actually is 400GB larger than it used to be.
-- 
Ray Morris
support@bettercgi.com

Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/

Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/

Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php




On Tue, 29 Mar 2011 23:56:03 +0200
"Fredrik Skog" <fredrik.skog@rodang.se> wrote:

> But since lvdisplay says the volume is 4.16TiB and pvdisplay says
> 5.59TiB something is wrong? Or am I missing something?

Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

[Fredrik Skog]

Hi,
----- Original Message ----- 
From: "Stuart D. Gathman" <stuart@bmsi.com>
To: "LVM general discussion and development" <linux-lvm@redhat.com>
Sent: Wednesday, March 30, 2011 12:14 AM
Subject: Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?


> On Tue, 29 Mar 2011, Fredrik Skog wrote:
>
>> 1> I made a partition /dev/sde1 with Linux LVM
>> 2> Run a "pvcreate /dev/sde1"
>> 3> Run "vgextend vgftp /dev/sd1"
>> 4> Run "lvextend -L+400G /dev/vgftp/lvftp"
>> 5> Run "umount /dev/vgftp/lvftp"
>> 6> Run "e2fsck -f /dev/vgftp/lvftp"
>> 7> Run "resize2fs /dev/vgftp/lvftp"
>>   resize2fs 1.41.10 (10-Feb-2009)
>>   Resizing the filesystem on /dev/vgftp/lvftp to 1115800576 (4k) blocks
>> 8> Here it all hangs. I cant do anything with the filesystem or LVM. 
>> every command i do hangs.
>
> The resize will take a very long time.  Did you try these commands on 
> another
> terminal?  Did you check the disk activity light?
>

I waited for a few days before i tried to interupt anything. I rebooted 
after a few weeks of not touching the computer. I checked for disk activity 
and found nothing.

>>    I edited fstab so my LVM is not remounted on reboot, and rebooted.
>>    Here I am right now.
>
> That was probably a bad idea if the disk was still busy extending your
> filesystem.
>
>> I can run LVM commands fine now after the reboot and the output from 
>> pvdisplay and lvdisplay is like i posted earlier.
>> How can i revert this in a safe way? I was thinking of just removing the 
>> PV and do a vgcfgrestore to "before" the lvextend.
>> But since lvdisplay says the volume is 4.16TiB and pvdisplay says 5.59TiB 
>> something is wrong? Or am I missing something?
>
> lvdisplay will almost never show the same as pvdisplay.  One displays
> *logical* volumes and the other *physical* volumes.
>
>> I tried a " vgreduce vgftp /dev/sde1" to remove my new drive again, but 
>> this only gives me an error "Physical volume "/dev/sde1" still in use"
>
> Of course, because your extended LV is using it.
>
>> I think this is strange because the pvdisplay seems to think i have not 
>> yet added the PV but pvdisplay does.
>
> Presumably you meant "lvdisplay" for one of the above.  What does
> lvdisplay show for the size of your LV?  Also, try out "lvs" and "pvs"
> for shorter output.  Include output in your response.
>

Yes, the later should be  lvdisplay

# vgs
  VG    #PV #LV #SN Attr   VSize VFree
  vgftp  12   1   0 wz--n- 5.59t 1.43t

# lvs
  LV    VG    Attr   LSize Origin Snap%  Move Log Copy%  Convert
  lvftp vgftp -wi--- 4.16t


>> If i do a lvreduce i fear something will break.
>
> Even more than it is now, yes.
>
>> Is it better to do a e2fsck now?
>
> That is probably the only way to salvage what is left of your filesystem,
> but don't do it until we find out whether the LV is the old or the new 
> size.
> I suspect the LV will be the new size, but the filesystem may still show
> the old size.  Since the resize2fs should be just adding free space, there 
> may not be any data loss.  e2fsck would fix the (now corrupt)
> free space, and you can extend again.
>
> --
>        Stuart D. Gathman <stuart@bmsi.com>
>     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 
> 591-6154
> "Confutatis maledictis, flammis acribus addictis" - background song for
> a Microsoft sponsored "Where do you want to go from here?" commercial.
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Thanks
/Fredrik Skog 

Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

[Fredrik Skog]

Hello,
thanks for your time

----- Original Message ----- 
From: "Ray Morris" <support@bettercgi.com>
To: <linux-lvm@redhat.com>
Sent: Wednesday, March 30, 2011 12:26 AM
Subject: Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?


>7> Run "resize2fs /dev/vgftp/lvftp"
>>    resize2fs 1.41.10 (10-Feb-2009)
>>    Resizing the filesystem on /dev/vgftp/lvftp to 1115800576 (4k)
>> blocks
>> Resizing the filesystem on /dev/vgftp/lvftp to 1115800576 (4k)
>> blocks
> 8> Here it all hangs. I cant do anything with the filesystem or LVM.
> 8> every
>
> You can expect resize2fs to take a couple of hours on a 4 TB
> filesystem, so the "crash" may well be nothing but impatience.

I doubt that is the case after waiting for at least 3 days. It has only 
taken 5-10 hours before.

>
>> If i do a lvreduce i fear something will break.
>> Is it better to do a e2fsck now?
>
> The problem that we know about is that the resize2fs was
> interrupted.  Reports defer in the level in danger in that,
> but definitely we want to get the filesystem consistent and
> small enough before we pull the block device out from underneath
> it.  So yes, you need a passing e2fsck before an lvreduce at
> this points.
>
> The safest thing to do would be make a copy of at least the LV,
> if not both PVs, so you can get back to where you are if needed.
> That's almost always the first best step for any kind of data
> recovery type of effort.
>
> Once you have a copy or have checked that your backups are Ok,
> you can proceed with e2fsck. After e2fsck passes, you can try
> to pvmove the extents off of the new drive, if you wish.  If
> the LV won't fit on the old drive, you'll need to resize2fs
> and lvreduce to make it fit.  It can be hard to know just how
> big to make the LV for a given FS size, due to filesystem overhead
> and such.  A safe way to do it would be to resize2fs as small
> as possible first. That'll probably give you at least 10%-20%
> of margin between the size of the FS and the size of the PV.
> Then resize the LV to match the PV.  Once the LV is at it's final
> size, resize2fs to max.
>
> Check that your LV is in fact the same size as it was
> before your lvextend and the output of "history". It sounds
> like either a) you didn't run exactly the commands that you
> are reporting or b) the LV is in fact larger than it used
> to be.  If you ran lvextend to increase the size of the LV
> and did not reduce it after, a crash during resize2fs isn't
> going to shrink it automagically.  Also, if it were at the
> old size, it probably wouldn't be on both PVs.  Therefore,
> the LV probably actually is 400GB larger than it used to be.

Sadly I have no backups of this system due to the size of it and lack of 
disks to make a full backup of it. :( I have alot of PV's making up the VG 
and LV and it totals 4TB before i added the new disk.
I have not used the system at all, and the commands i printed before is the 
exact ones i did on the machine. I have a file with the console buffer still 
here.
I did not execute any other commands in between or wrote anything to the 
volume. The old volume was almost full before i did the lvextend. But since 
i have not used it at all after the resize2fs crash.....i could do an e2fsck 
and then an lvreduce /dev/vgftp -L-400G /dev/vgftp/lvftp and I should be 
back to where i started? Given that i can run e2fsck.

I really dont like this situation and I feel like a newbie trying to 
understand whats gone wrong. really appreciate your help.

thanks
/Fredrik Skog


> -- 
> Ray Morris
> support@bettercgi.com
>
> Strongbox - The next generation in site security:
> http://www.bettercgi.com/strongbox/
>
> Throttlebox - Intelligent Bandwidth Control
> http://www.bettercgi.com/throttlebox/
>
> Strongbox / Throttlebox affiliate program:
> http://www.bettercgi.com/affiliates/user/register.php
>
>
>
>
> On Tue, 29 Mar 2011 23:56:03 +0200
> "Fredrik Skog" <fredrik.skog@rodang.se> wrote:
>
>> But since lvdisplay says the volume is 4.16TiB and pvdisplay says
>> 5.59TiB something is wrong? Or am I missing something?
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ 

Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

[Fredrik Skog]

----- Original Message ----- 
From: "Stuart D. Gathman" <stuart@bmsi.com>
To: "LVM general discussion and development" <linux-lvm@redhat.com>
Sent: Wednesday, March 30, 2011 12:14 AM
Subject: Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

>SNIP>

>> I can run LVM commands fine now after the reboot and the output from 
>> pvdisplay and lvdisplay is like i posted earlier.
>> How can i revert this in a safe way? I was thinking of just removing the 
>> PV and do a vgcfgrestore to "before" the lvextend.
>> But since lvdisplay says the volume is 4.16TiB and pvdisplay says 5.59TiB 
>> something is wrong? Or am I missing something?

>SNIP>

>> Is it better to do a e2fsck now?

> That is probably the only way to salvage what is left of your filesystem,
> but don't do it until we find out whether the LV is the old or the new 
> size.
> I suspect the LV will be the new size, but the filesystem may still show
> the old size.  Since the resize2fs should be just adding free space, there 
> may not be any data loss.  e2fsck would fix the (now corrupt)
> free space, and you can extend again.

I just added all the segments of the LV from an old backed up config file. 
It adds up to 3,856TB, so I guess  you were right and i assumed wrong. The 
old LV was about 3,8TB and the new is about 4,16TB or about 400GB bigger 
than before as expected by lvextend -L+400G.

Also it tells me I have 1,43TiB free wich is 5,59TiB - 4,16TiB. It also 
looks right.

maybe I should try an e2fsck now?

again, thanks for your time

/Fredrik Skog

>
> --
>        Stuart D. Gathman <stuart@bmsi.com>
>     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 
> 591-6154
> "Confutatis maledictis, flammis acribus addictis" - background song for
> a Microsoft sponsored "Where do you want to go from here?" commercial.
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ 

Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

[Ray Morris]

  Yes i sounds like it's time to fsck, then if needed 
resize2fs to the proper size.  It would be best to 
mount it read only only and copy whatever data copies
to be safe.  4TB is a lot of data, but then again it's
only $130 worth of consumer drives to back it up.
-- 
Ray Morris
support@bettercgi.com

Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/

Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/

Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php




On Wed, 30 Mar 2011 02:27:49 +0200
"Fredrik Skog" <fredrik.skog@rodang.se> wrote:

> 
> ----- Original Message ----- 
> From: "Stuart D. Gathman" <stuart@bmsi.com>
> To: "LVM general discussion and development" <linux-lvm@redhat.com>
> Sent: Wednesday, March 30, 2011 12:14 AM
> Subject: Re: [linux-lvm] How to fix missmatch between VG-size and
> LV-size?
> 
> >SNIP>
> 
> >> I can run LVM commands fine now after the reboot and the output
> >> from pvdisplay and lvdisplay is like i posted earlier.
> >> How can i revert this in a safe way? I was thinking of just
> >> removing the PV and do a vgcfgrestore to "before" the lvextend.
> >> But since lvdisplay says the volume is 4.16TiB and pvdisplay says
> >> 5.59TiB something is wrong? Or am I missing something?
> 
> >SNIP>
> 
> >> Is it better to do a e2fsck now?
> 
> > That is probably the only way to salvage what is left of your
> > filesystem, but don't do it until we find out whether the LV is the
> > old or the new size.
> > I suspect the LV will be the new size, but the filesystem may still
> > show the old size.  Since the resize2fs should be just adding free
> > space, there may not be any data loss.  e2fsck would fix the (now
> > corrupt) free space, and you can extend again.
> 
> I just added all the segments of the LV from an old backed up config
> file. It adds up to 3,856TB, so I guess  you were right and i assumed
> wrong. The old LV was about 3,8TB and the new is about 4,16TB or
> about 400GB bigger than before as expected by lvextend -L+400G.
> 
> Also it tells me I have 1,43TiB free wich is 5,59TiB - 4,16TiB. It
> also looks right.
> 
> maybe I should try an e2fsck now?
> 
> again, thanks for your time
> 
> /Fredrik Skog
> 
> >
> > --
> >        Stuart D. Gathman <stuart@bmsi.com>
> >     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 
> > 591-6154
> > "Confutatis maledictis, flammis acribus addictis" - background song
> > for a Microsoft sponsored "Where do you want to go from here?"
> > commercial.
> >
> > _______________________________________________
> > linux-lvm mailing list
> > linux-lvm@redhat.com
> > https://www.redhat.com/mailman/listinfo/linux-lvm
> > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 

Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

[Fredrik Skog]

Hello,

If I ever get all this data back I will definitely try to either RAID it or 
make a backup of the most importatnt stuff on the volume.

I have now run e2fsck -n -t -v /dev/vgftp/lvftp successfully. It gave me the 
following output. I'm not really sure what it means, but I think it looks 
pretty ok. Do you think it's ok to do this for real now?

################
# e2fsck -n -t -v /dev/vgftp/lvftp
e2fsck 1.41.10 (10-Feb-2009)
/dev/vgftp/lvftp contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes

Running additional passes to resolve blocks claimed by more than one 
inode...
Pass 1B: Rescanning for multiply-claimed blocks
Multiply-claimed block(s) in inode 51036510: 102452460
Multiply-claimed block(s) in inode 96436268: 203786736
Multiply-claimed block(s) in inode 122208259: 244427502
Multiply-claimed block(s) in inode 195693509: 396636221
Multiply-claimed block(s) in inode 331595778: 665071629
Pass 1C: Scanning directories for inodes with multiply-claimed blocks
Pass 1D: Reconciling multiply-claimed blocks
(There are 5 inodes containing multiply-claimed blocks.)

File /XX/XXXXXXXX.XXX (inode #51036510, mod time Sun Apr 30 03:39:57 2006)
  has 1 multiply-claimed block(s), shared with 0 file(s):
Multiply-claimed blocks already reassigned or cloned.

File /XXXX/XXXXXXX.XXX (inode #96436268, mod time Sun Feb 25 19:15:11 2007)
  has 1 multiply-claimed block(s), shared with 0 file(s):
Multiply-claimed blocks already reassigned or cloned.

File /XXXX/XXXX/XX.XXX (inode #122208259, mod time Sun Jul 15 18:37:56 2007)
  has 1 multiply-claimed block(s), shared with 0 file(s):
Multiply-claimed blocks already reassigned or cloned.

File /XXX/XXXX.XXX (inode #195693509, mod time Sat Oct 18 01:00:53 2008)
  has 1 multiply-claimed block(s), shared with 0 file(s):
Multiply-claimed blocks already reassigned or cloned.

File /XX/XXX.XXX (inode #331595778, mod time Thu Mar 19 00:51:54 2009)
  has 1 multiply-claimed block(s), shared with 0 file(s):
Multiply-claimed blocks already reassigned or cloned.

Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Block bitmap 
differences:  -(244--268) -(98548--98572) -(164084--164108) -(229620--229644) 
 -(295156--295180) -(819444--819468) -(884980--885004) -(1605876--1605900) -(2654452--2654476) 
 -(4096244--4096268) -(7962868--7962892) -(11239668--11239692) 
+(14601899--14601923) 
+(14601949--14602273) -(20480244--20480268) -(23888116--23888140) -472222192 
 -512862958 -639323372 -933507085 -933507133
Fix? no


/dev/vgftp/lvftp: ********** WARNING: Filesystem still has errors **********


  308343 inodes used (0.06%)
   56220 non-contiguous files (18.2%)
     206 non-contiguous directories (0.1%)
         # of inodes with ind/dind/tind blocks: 247855/206685/61
1006593162 blocks used (99.57%)
       0 bad blocks
      82 large files

  290797 regular files
   17537 directories
       0 character device files
       0 block device files
       0 fifos
       0 links
       0 symbolic links (0 fast symbolic links)
       0 sockets
--------
  308334 files
Memory used: 736k/309504k (407k/330k), time: 20076.40/1223.57/795.83
I/O read: 133788MB, write: 0MB, rate: 6.66MB/s
######################

I will try to copy the most important stuff off of the volume before i try 
this though.

Before running resize2fs later, how do I figure the right size of the 
filesystem out? Without risking shrinking it too much.

Thanks
/Fredrik Skog


----- Original Message ----- 
From: "Ray Morris" <support@bettercgi.com>
To: <linux-lvm@redhat.com>
Sent: Wednesday, March 30, 2011 5:04 PM
Subject: Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?


>  Yes i sounds like it's time to fsck, then if needed
> resize2fs to the proper size.  It would be best to
> mount it read only only and copy whatever data copies
> to be safe.  4TB is a lot of data, but then again it's
> only $130 worth of consumer drives to back it up.
> -- 
> Ray Morris
> support@bettercgi.com
>
> Strongbox - The next generation in site security:
> http://www.bettercgi.com/strongbox/
>
> Throttlebox - Intelligent Bandwidth Control
> http://www.bettercgi.com/throttlebox/
>
> Strongbox / Throttlebox affiliate program:
> http://www.bettercgi.com/affiliates/user/register.php
>
>
>
>
> On Wed, 30 Mar 2011 02:27:49 +0200
> "Fredrik Skog" <fredrik.skog@rodang.se> wrote:
>
>>
>> ----- Original Message ----- 
>> From: "Stuart D. Gathman" <stuart@bmsi.com>
>> To: "LVM general discussion and development" <linux-lvm@redhat.com>
>> Sent: Wednesday, March 30, 2011 12:14 AM
>> Subject: Re: [linux-lvm] How to fix missmatch between VG-size and
>> LV-size?
>>
>> >SNIP>
>>
>> >> I can run LVM commands fine now after the reboot and the output
>> >> from pvdisplay and lvdisplay is like i posted earlier.
>> >> How can i revert this in a safe way? I was thinking of just
>> >> removing the PV and do a vgcfgrestore to "before" the lvextend.
>> >> But since lvdisplay says the volume is 4.16TiB and pvdisplay says
>> >> 5.59TiB something is wrong? Or am I missing something?
>>
>> >SNIP>
>>
>> >> Is it better to do a e2fsck now?
>>
>> > That is probably the only way to salvage what is left of your
>> > filesystem, but don't do it until we find out whether the LV is the
>> > old or the new size.
>> > I suspect the LV will be the new size, but the filesystem may still
>> > show the old size.  Since the resize2fs should be just adding free
>> > space, there may not be any data loss.  e2fsck would fix the (now
>> > corrupt) free space, and you can extend again.
>>
>> I just added all the segments of the LV from an old backed up config
>> file. It adds up to 3,856TB, so I guess  you were right and i assumed
>> wrong. The old LV was about 3,8TB and the new is about 4,16TB or
>> about 400GB bigger than before as expected by lvextend -L+400G.
>>
>> Also it tells me I have 1,43TiB free wich is 5,59TiB - 4,16TiB. It
>> also looks right.
>>
>> maybe I should try an e2fsck now?
>>
>> again, thanks for your time
>>
>> /Fredrik Skog
>>
>> >
>> > --
>> >        Stuart D. Gathman <stuart@bmsi.com>
>> >     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703
>> > 591-6154
>> > "Confutatis maledictis, flammis acribus addictis" - background song
>> > for a Microsoft sponsored "Where do you want to go from here?"
>> > commercial.
>> >
>> > _______________________________________________
>> > linux-lvm mailing list
>> > linux-lvm@redhat.com
>> > https://www.redhat.com/mailman/listinfo/linux-lvm
>> > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>
>> _______________________________________________
>> linux-lvm mailing list
>> linux-lvm@redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ 

Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

[Ron Johnson]

On 03/31/2011 10:06 AM, Fredrik Skog wrote:
> Hello,
>
> If I ever get all this data back I will definitely try to either RAID it
> or make a backup of the most importatnt stuff on the volume.
>

Tsk tsk tsk.  No one ever *really* learns until thy lose something 
important.

High-capacity "Green" hard drives are dirt cheap, as are external USB 
and eSATA enclosures.  (Though I haven't figured out how to pull the 
eSATA plug on a running system without it scribbling all over something 
important...)  So, if today I were building a backup system, I'd buy a 
USB 3.0 card, a 2-drive USB 3.0 enclosure and a couple of 3TB Green drives.

-- 
"Neither the wisest constitution nor the wisest laws will secure
the liberty and happiness of a people whose manners are universally
corrupt."
Samuel Adams, essay in The Public Advertiser, 1749

Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

[Stuart D. Gathman]

On Thu, 31 Mar 2011, Fredrik Skog wrote:

> If I ever get all this data back I will definitely try to either RAID it or 
> make a backup of the most importatnt stuff on the volume.

Of course, RAID will not protect you from what just happened.

--
 	      Stuart D. Gathman <stuart@bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

[Georges Giralt]

Le 31/03/2011 17:06, Fredrik Skog a �crit :
> Hello,
>
> If I ever get all this data back I will definitely try to either RAID
> it or make a backup of the most importatnt stuff on the volume.
>
> I have now run e2fsck -n -t -v /dev/vgftp/lvftp successfully. It gave
> me the following output. I'm not really sure what it means, but I
> think it looks pretty ok. Do you think it's ok to do this for real now?
>
> ################
> # e2fsck -n -t -v /dev/vgftp/lvftp
Snip......
/dev/vgftp/lvftp: ********** WARNING: Filesystem still has errors **********
Snip
> I will try to copy the most important stuff off of the volume before i
> try this though.
Hello Fredrik
You used "-n" option  in the e2fsck command. So  the "WARNING:
Filesystem still has errors" tell all.
You SHOULD run e2fsck without the -n option BEFORE trying to mount it
and back up data or trying to resize it !
Otherwise..... you could well loose all the data !
Just my 2� .....
BTW : it is strange that all multiply allocated blocks are found to
belong only to one file. What was the system doing during the resize ?
What was the file system activity during that time ? And las t but not
least, you once told us that before running the resize, you ran the
fsck. Was it with -n option also ?  This may explain all the subsequent
events.

-- 
If the only tool you have is a hammer, you tend to see every problem as a nail.
                Abraham Maslow
A British variant :
Any tool can serve as a hammer but a screwdriver makes the best chisel. 

Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?

[Fredrik Skog]

----- Original Message ----- 
From: "Georges Giralt" <georges.giralt@free.fr>
To: "LVM general discussion and development" <linux-lvm@redhat.com>
Sent: Thursday, March 31, 2011 9:19 PM
Subject: Re: [linux-lvm] How to fix missmatch between VG-size and LV-size?


Le 31/03/2011 17:06, Fredrik Skog a �crit :
> Hello,
>
> If I ever get all this data back I will definitely try to either RAID
> it or make a backup of the most importatnt stuff on the volume.
>
> I have now run e2fsck -n -t -v /dev/vgftp/lvftp successfully. It gave
> me the following output. I'm not really sure what it means, but I
> think it looks pretty ok. Do you think it's ok to do this for real now?
>
> ################
> # e2fsck -n -t -v /dev/vgftp/lvftp
Snip......
/dev/vgftp/lvftp: ********** WARNING: Filesystem still has errors **********
Snip
> I will try to copy the most important stuff off of the volume before i
> try this though.
Hello Fredrik
You used "-n" option  in the e2fsck command. So  the "WARNING:
Filesystem still has errors" tell all.
You SHOULD run e2fsck without the -n option BEFORE trying to mount it
and back up data or trying to resize it !
Otherwise..... you could well loose all the data !
Just my 2� .....
BTW : it is strange that all multiply allocated blocks are found to
belong only to one file. What was the system doing during the resize ?
What was the file system activity during that time ? And las t but not
least, you once told us that before running the resize, you ran the
fsck. Was it with -n option also ?  This may explain all the subsequent
events.

-- 
If the only tool you have is a hammer, you tend to see every problem as a 
nail.
                Abraham Maslow
A British variant :
Any tool can serve as a hammer but a screwdriver makes the best chisel.

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/


Hello,

Sorry I did not tell you that i used the -n option now just to avoid any 
changes to the filesystem before i have copied my most important data from 
the volume. I only wanted to see what kind errors i got.
I'm in the process of copying some stuff right now and will try to e2fsck it 
later when that is done.

I did not use the -n option before the resize2fs earlier.
I have X'd out the filenames in the mail. The errors all where on different 
files.

Sorry for not being clear on this.

/Fredrik Skog