[linux-lvm] which files, which physical volumes?

[linux-lvm] which files, which physical volumes?

[Steve Wray]

Hi there,

Is there any way to find which files in a given logical volume occupy 
extents on a given physical volume?

I am trying to decommission a drive and pvmove still isn't working 
properly. Either that or;

1. pvmove -tv claims to fail when in fact without the -t it would 
actually work. Thie would seem like a joke in very bad taste (on the 
part of the developers.

or

2. pvmove -v which is supposed to give verbose progress output, in fact 
will sit there for a *very* long time (half an hour or more) with no 
progress output and push load up extremely high.

Or maybe pvmove is *still* broken, in which case I need to find the 
files which occupy that physical volume, move them to a different 
logical volume altogether and then forcibly remove that physical volume 
without regard to the files that are no longer on it... and hopefuly 
this won't cause corruption on the filesystem on that logical volume.

help?

Thanks!

Re: [linux-lvm] which files, which physical volumes?

[Luca Berra]

On Fri, Sep 24, 2004 at 02:29:37PM +1200, Steve Wray wrote:
>Or maybe pvmove is *still* broken, in which case I need to find the 
>files which occupy that physical volume, move them to a different 
>logical volume altogether and then forcibly remove that physical volume 
>without regard to the files that are no longer on it... and hopefuly 
>this won't cause corruption on the filesystem on that logical volume.
this will certainly cause corruption.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: [linux-lvm] which files, which physical volumes?

[Steve Wray]

Luca Berra wrote:

> On Fri, Sep 24, 2004 at 02:29:37PM +1200, Steve Wray wrote:
> 
>> Or maybe pvmove is *still* broken, in which case I need to find the 
>> files which occupy that physical volume, move them to a different 
>> logical volume altogether and then forcibly remove that physical 
>> volume without regard to the files that are no longer on it... and 
>> hopefuly this won't cause corruption on the filesystem on that logical 
>> volume.
> 
> this will certainly cause corruption.

So is there any other way to decomission that physical volume that 
doesn't involve pvmove?

Actually, this just got more interesting because after the last
pvmove --abort I now get this error out of pvmove -tv;

Insufficient contiguous allocatable extents (780) for logical volume 
pvmove0: 1292 required
Allocation for temporary pvmove LV failed

which is new.

I am guessing here, but if I moved some of the larger files out of that 
logical volume, might pvmove be able to work around this?

Re: [linux-lvm] which files, which physical volumes?

[Clint Byrum]

On Thursday, September 23, 2004, at 11:31 PM, Steve Wray wrote:

> Luca Berra wrote:
>
>> On Fri, Sep 24, 2004 at 02:29:37PM +1200, Steve Wray wrote:
>>> Or maybe pvmove is *still* broken, in which case I need to find the 
>>> files which occupy that physical volume, move them to a different 
>>> logical volume altogether and then forcibly remove that physical 
>>> volume without regard to the files that are no longer on it... and 
>>> hopefuly this won't cause corruption on the filesystem on that 
>>> logical volume.
>> this will certainly cause corruption.
>
> So is there any other way to decomission that physical volume that 
> doesn't involve pvmove?
>
> Actually, this just got more interesting because after the last
> pvmove --abort I now get this error out of pvmove -tv;
>
> Insufficient contiguous allocatable extents (780) for logical volume 
> pvmove0: 1292 required
> Allocation for temporary pvmove LV failed
>
> which is new.
>
> I am guessing here, but if I moved some of the larger files out of 
> that logical volume, might pvmove be able to work around this?
>

LVM knows nothing of your files. It knows blocks, and it allocates them 
to Volume Groups and Logical Volumess without regard to whether or not 
the filesystem will use them.

Yours seems to be telling you that you can't move 1292 physical extents 
into 780. You'll have to add physical volumes (NFS mounted loopback? 
External USB harddrive?), or remove other logical volumes.

Re: [linux-lvm] which files, which physical volumes?

[Alasdair G Kergon]

On Sat, Sep 25, 2004 at 09:20:20AM -0700, Clint Byrum wrote:
> Yours seems to be telling you that you can't move 1292 physical extents 
> into 780. You'll have to add physical volumes (NFS mounted loopback? 
> External USB harddrive?), or remove other logical volumes.
 
Or break the 1292 up into smaller parts if there *is* already room 
elsewhere: the allocation routines don't do this for you yet.

e.g. pvmove /dev/hda:1000-1779 
to move a specific 780 extents off hda.

[I suggest using version 2.00.24 is best for pvmove]

Alasdair
-- 
agk@redhat.com

Re: [linux-lvm] which files, which physical volumes?

[Steve Wray]

Clint Byrum wrote:
> 
> On Thursday, September 23, 2004, at 11:31 PM, Steve Wray wrote:
> 
>> Luca Berra wrote:
>>
>>> On Fri, Sep 24, 2004 at 02:29:37PM +1200, Steve Wray wrote:
>>>
[snip]
>> Actually, this just got more interesting because after the last
>> pvmove --abort I now get this error out of pvmove -tv;
>>
>> Insufficient contiguous allocatable extents (780) for logical volume 
>> pvmove0: 1292 required
>> Allocation for temporary pvmove LV failed
>>
>> which is new.
>>
>> I am guessing here, but if I moved some of the larger files out of 
>> that logical volume, might pvmove be able to work around this?
>>
[snip]
> Yours seems to be telling you that you can't move 1292 physical extents 
> into 780. You'll have to add physical volumes (NFS mounted loopback? 
> External USB harddrive?), or remove other logical volumes.

Well then its bogus; the physical volume I am trying to pvmove has a 
capacity of around 9G whereas the volume group of which it is a part has 
a nice 42G free.

So how does that work out to not enough extents?

Re: [linux-lvm] which files, which physical volumes?

[Michael T. Babcock]

Steve Wray wrote:

> Well then its bogus; the physical volume I am trying to pvmove has a 
> capacity of around 9G whereas the volume group of which it is a part 
> has a nice 42G free.
>
> So how does that work out to not enough extents?


Does the Volume Group in question have 42GB of unallocated extents, or 
42GB free on the allocated Logical Volumes?

If you had VG main with 60GB total containing LVs lv1 and lv2 each 20GB 
then you'd have 2GB unallocated on the VG.

If you want more unallocated space on e.g. /dev/hdc1 (a member of VG 
main) to move part of lv1 onto (from /dev/hda1), you'd have to do 
something like:

resize_filesystem /dev/main/lv2 (down to smaller size)
lvresize -L (new size) /dev/main/lv2 # to resize the logical volume 
using the space
pvmove -n /dev/main/lv2 /dev/hdc1 # to move as much as possible of lv2 
off this PV
pvmove -n /dev/main/lv1 /dev/hda1 /dev/hdc1

Does that make sense to you?
-- 
Michael T. Babcock

Re: [linux-lvm] which files, which physical volumes?

[Eric Hopper]

[-- Attachment #1: Type: text/plain, Size: 1383 bytes --]

On Fri, Sep 24, 2004 at 06:31:15PM +1200, Steve Wray wrote:
> So is there any other way to decomission that physical volume that
> doesn't involve pvmove?

I end up using lvcreate (with some options) to create a new logical
volume that has no extents on the physical volume you want to move off
of, and has the exact same number of extents as the logical volume you
want to move.  Then use dd to copy the blocks from the filesystem you
want to move (which has to be unmounted, or mounted read-only) onto the
new one.  Then unmount the old one, lvremove it, and change fstab and
things so the new one is mounted in its place.

It's ugly and stupid.  But, since pvmove doesn't seem to work yet, I
think that's the best that can be done.  I'm sorely disappointed that
LVM2 was even put into the production kernel without pvmove working.
:-(

I know LVM2 and the device mapper concept are much better than LVM1.
But, pvmove is a core piece of functionality.  It's like releasing a
filesystem for which the lstat call doesn't work or something similar.

*sigh*,
-- 
"It does me no injury for my neighbor to say there are twenty gods or no God.
It neither picks my pocket nor breaks my leg."  --- Thomas Jefferson
"Go to Heaven for the climate, Hell for the company."  -- Mark Twain
-- Eric Hopper (hopper@omnifarious.org  http://www.omnifarious.org/~hopper) --

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [linux-lvm] which files, which physical volumes?

[Steve Wray]

Eric Hopper wrote:
> On Fri, Sep 24, 2004 at 06:31:15PM +1200, Steve Wray wrote:
> 
>>So is there any other way to decomission that physical volume that
>>doesn't involve pvmove?
> 
> 
> I end up using lvcreate (with some options) to create a new logical
> volume that has no extents on the physical volume you want to move off
> of, and has the exact same number of extents as the logical volume you
> want to move.

Unfortunately I don't have enough disk for this... nice idea though!

> It's ugly and stupid.  But, since pvmove doesn't seem to work yet, I
> think that's the best that can be done.  I'm sorely disappointed that
> LVM2 was even put into the production kernel without pvmove working.
> :-(

Phew! At least I am not the only one who feels like that!
This is for my home system, but at work, since discovering the 
non-workingness of pvmove, I have tried to make sure that we don't have 
to use it...

> I know LVM2 and the device mapper concept are much better than LVM1.
> But, pvmove is a core piece of functionality.  It's like releasing a
> filesystem for which the lstat call doesn't work or something similar.
> 
> *sigh*,

Absolutely. I am actually amazed that its still like this after so long.
I mean, whats the problem? Is Redhat blocking it somehow or what?
(I sort of sniff a scenario where Redhat have some proprietary volume 
management system in the pipeline and want to discourage people from 
using LVM2 so that they will be more likely to but into Redhat 
Enterprise Server or whatever, but maybe thats just paranoia speaking 
and I should just take the tinfoil hat off and let the rays in)

Re: [linux-lvm] which files, which physical volumes?

[Alasdair G Kergon]

On Sun, Sep 26, 2004 at 01:27:57PM -0700, Eric Hopper wrote:
> But, since pvmove doesn't seem to work yet, I

pvmove does work, but I am aware of the following restrictions 
if you want to avoid the risk of freezing:

  You must not disable o_direct when configuring;
    [I've heard some distributions/architectures may do this]
  You must not set log activation to 1 in the config file;
    [That settings's only needed for a particular sort of debugging]
  You must not pvmove a filesystem that lvm itself uses directly;
    [For 2.6.8 kernels, version 2.00.24 (if compiled against up-to-date 
    headers that define O_NOATIME correctly) is reported to improve 
    things substantially in this case; for 2.4 a kernel patch
    doing something similar gets applied by default.]
  Don't try to mix snapshots with pvmove [untested].
  I have also not tested with udev, nor with md underneath PVs.

Also:
  The allocation algorithm is as yet unable to break up contiguous 
  space: you still have to break it up yourself if it doesn't find 
  enough contiguous space. [e.g. pvmove /dev/hda:100-199]

  Test mode works a different way in LVM2 - it's designed for testing
  and simply disables all writes at the lowest level of the code 
  (to guarantee safety) - but reports success to the calling function.  
  Many code paths don't yet say 'stop here if this is test mode' so
  they stop instead with an error when they find an inconsistency
  because they thought they changed something, but it got read back
  unchanged.

Alasdair
-- 
agk@redhat.com

Re: [linux-lvm] which files, which physical volumes?

[Steve Wray]

Alasdair G Kergon wrote:
> On Sun, Sep 26, 2004 at 01:27:57PM -0700, Eric Hopper wrote:
> 
[snip]
>   Test mode works a different way in LVM2 - it's designed for testing
>   and simply disables all writes at the lowest level of the code 
>   (to guarantee safety) - but reports success to the calling function.  
>   Many code paths don't yet say 'stop here if this is test mode' so
>   they stop instead with an error when they find an inconsistency
>   because they thought they changed something, but it got read back
>   unchanged.

wow it makes a lot more sense now.

This could really do with going into the pvmove man page, which is a bit 
sparse as it is...

Re: [linux-lvm] which files, which physical volumes?

[John Stoffel]

>>>>> "Steve" == Steve Wray <steve@myself.gen.nz> writes:

Steve> Or maybe pvmove is *still* broken, in which case I need to find
Steve> the files which occupy that physical volume, move them to a
Steve> different logical volume altogether and then forcibly remove
Steve> that physical volume without regard to the files that are no
Steve> longer on it... and hopefuly this won't cause corruption on the
Steve> filesystem on that logical volume.

You will destroy that filesystem, no ifs, ands or buts about it, if
you just remove the phyiscal volume.  Don't even think of doing it.
Think about it for a seond, maybe in terms of layers.

You've got the physical disks on the bottom layer, with an LV on top
of that, and a filesystem on top of that.  If you remove any lower
layer, you will immediately impact and probably corrupt the upper
layers, unless you have the proper redundancy in the lower layer.

Sorry I can't help with the pmove stuff though, I haven't tried it
yet.

John