[linux-lvm] lvremove or vgextend: metadata too large for circular buffer

[linux-lvm] lvremove or vgextend: metadata too large for circular buffer

[Andrew Gideon]

We're hitting the "metadata too large for circular buffer" problem.  Last 
time this occurred, I was able to clear out a little space by removing 
some unused volumes, and then make more space by selective use of pvmove 
to defragment.  This worked well.

However, that time I didn't get the error from lvremove.  This time, that 
is occurring.  So I don't seem to be able to make any space.

Once I execute lvremove on a volume, the "LV Status" becomes "NOT 
available".  The command 'lvdisplay -m' still shows the segments, however.

How can I resolve this?  Trashing the entire VG is not something I can 
afford.

I created new PVs with larger allocations of space for metadata, but I 
cannot even add these to the VG.  The vgextend command also fails with 
"metadata too large for circular buffer".

	- Andrew

Re: [linux-lvm] lvremove or vgextend: metadata too large for circular buffer

[Andrew Gideon]

On Fri, 11 Mar 2011 20:05:12 +0000, Andrew Gideon wrote:

> However, that time I didn't get the error from lvremove.  This time,
> that is occurring.  So I don't seem to be able to make any space.

I've tried vgcfgrestore, but this too yields:

	metadata too large for circular buffer

I ran some commands with -vvv.  The output which seems to always 
immediately precede the error is:

	Doubling metadata output buffer to 131072

I also noticed that it appears to be looking at /dev/sdi and /dev/sdj.  
These are two PVs I'd tried to add, but where vgextend also issued the 
"too large..." error.  These two PVs do not appear in 'vgdisplay -v'.

[root@backup2 backup]# /usr/sbin/lvremove -d -vvv -f /dev/jetarray004/i2-backup0 2>&1  | egrep 'sd[ij]' 
        Opened /dev/sdi RW O_DIRECT
        /dev/sdi: block size is 4096 bytes
      /dev/sdi: lvm2 label detected
        lvmcache: /dev/sdi: now in VG #orphans_lvm2 (#orphans_lvm2)
        Closed /dev/sdi
        Opened /dev/sdj RW O_DIRECT
        /dev/sdj: block size is 4096 bytes
      /dev/sdj: lvm2 label detected
        lvmcache: /dev/sdj: now in VG #orphans_lvm2 (#orphans_lvm2)
        Closed /dev/sdj
[root@backup2 backup]# 

Is there perhaps some "internal" VG (ie. #orphans_lvm2) that I'd be
able to successfully remove?

Thanks...

	Andrew

Re: [linux-lvm] lvremove or vgextend: metadata too large for circular buffer

[Ray Morris]

> However, that time I didn't get the error from lvremove.
> This time, that is occurring.  So I don't seem to be able to make any
> space.

> Once I execute lvremove on a volume, the "LV Status" becomes "NOT 
> available".  The command 'lvdisplay -m' still shows the segments,
> however.

> How can I resolve this?  Trashing the entire VG is not something I
> can afford.

You may be able to manually do your removes on the output of 
vgcfgbackup, then use vgcfgrestore.  Be careful.

> I created new PVs with larger allocations of space for metadata, but
> I cannot even add these to the VG.  The vgextend command also fails
> with "metadata too large for circular buffer"

   What I did was to allocate MUCH more space on the new PVs, 
make a new VG from them, then copy LVs from one of the old PVs.
Then recreate that PV with more metadata space, add it to the 
new VG, etc. until all of the PVs are in the new VG, with far 
more metadata space.
-- 
Ray Morris
support@bettercgi.com

Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/

Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/

Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php




On Fri, 11 Mar 2011 20:05:12 +0000 (UTC)
Andrew Gideon <ag2827189@tagmall.com> wrote:

> We're hitting the "metadata too large for circular buffer" problem.
> Last time this occurred, I was able to clear out a little space by
> removing some unused volumes, and then make more space by selective
> use of pvmove to defragment.  This worked well.
> 
> However, that time I didn't get the error from lvremove.  This time,
> that is occurring.  So I don't seem to be able to make any space.
> 
> Once I execute lvremove on a volume, the "LV Status" becomes "NOT 
> available".  The command 'lvdisplay -m' still shows the segments,
> however.
> 
> How can I resolve this?  Trashing the entire VG is not something I
> can afford.
> 
> I created new PVs with larger allocations of space for metadata, but
> I cannot even add these to the VG.  The vgextend command also fails
> with "metadata too large for circular buffer".
> 
> 	- Andrew
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 

Re: [linux-lvm] lvremove or vgextend: metadata too large for circular buffer

[Andrew Gideon]

On Fri, 11 Mar 2011 17:01:54 -0600, Ray Morris wrote:


>    What I did was to allocate MUCH more space on the new PVs,
> make a new VG from them, then copy LVs from one of the old PVs. Then
> recreate that PV with more metadata space, add it to the new VG, etc.
> until all of the PVs are in the new VG, with far more metadata space.

Creating a new VG is my worst-case scenario, but that will be expensive 
in terms of time - time during which I cannot do expansion of volumes.  
It's a little frustrating.

How risky is the idea of editing a backup file and doing the restore?  If 
I screw it up, I can always do a re-restore of the proper data, no?

Thanks...

	Andrew