[linux-lvm] Running vgscan while swap on LVM is already active

[linux-lvm] Running vgscan while swap on LVM is already active

[Urs Thuermann]

This problem has been described before here on the mailing list, but
no reason/solution was posted.

I have two volume groups
* vg0 consisting of one PV  /dev/sda2
* vg1 consisting of two PVs /dev/hda and /dev/hdc

Most of the system is in vg0, containing
/dev/vg0/{root,var,usr,local,home,swap,...}, and vg0 is activated in
the boot process.  At this time no IDE drivers are loaded, thus only
vg0 is detected by vgscan in the boot process.  swap is activated on
/dev/vg0/swap successfully.

When I later decide that I want to load the IDE modules, start the IDE
hard drives, and mount the file systems from vg1 (which I need
seldomly, which is why the drives are spun down most of the time), I
need to run vgscan after activating drives and IDE modules.

However, running vgscan while /dev/vg0/swap is active, breaks things:

isnogud:root# swapon -s
Filename                        Type            Size    Used    Priority
/dev/vg0/swap                   partition       262136  0       -1
isnogud:root# swapoff /dev/vg0/swap
isnogud:root# swapon /dev/vg0/swap
isnogud:root# swapon -s   
Filename                        Type            Size    Used    Priority
/dev/vg0/swap                   partition       262136  0       -2
isnogud:root# vgscan
vgscan -- reading all physical volumes (this may take a while...)
vgscan -- found active volume group "vg0"
vgscan -- "/etc/lvmtab" and "/etc/lvmtab.d" successfully created
vgscan -- WARNING: This program does not do a VGDA backup of your volume group

isnogud:root# swapon -s
Filename                        Type            Size    Used    Priority
/dev/vg0/swap (deleted)         partition       262136  0       -2
isnogud:root# swapoff /dev/vg0/swap
swapoff: /dev/vg0/swap: Invalid argument

What does the attribute deleted mean here?  I can see the the device
node /dev/vg0/swap is recreated, but with the same major/minor numbers.

isnogud:root# date; ls -l /dev/vg0/swap
Fri Jun 11 15:22:12 CEST 2004
brw-rw----    1 root     disk      58,   8 Jun 11 15:19 /dev/vg0/swap

This reminds of open files which are unlinked while still being
opened, which are displayed as deleted by eg. lsof:

isnogud:root# cat >/tmp/foo & rm /tmp/foo; lsof |grep /tmp/foo
[1] 3490

[1]+  Stopped                 cat >/tmp/foo
cat       3490  root    1w   REG        0,7      0   20112 /tmp/foo (deleted)


But is the active swap space somehow tied to the device node
/dev/vg0/swap in the file system.  I thought this is not the case.
And even if it is, this behavior still suprises me because it the
device node is recreated with the same device number and even inode
number.

On the other hand, unmounting file systems on LVs in vg0 is possible
without any problem after running vgscan, using both, the block device
name or the mount point.

Can someone explain what's going on here?

urs

Re: [linux-lvm] Running vgscan while swap on LVM is already active

[Alasdair G Kergon]

On Fri, Jun 11, 2004 at 03:33:59PM +0200, Urs Thuermann wrote:
> /dev/vg0/swap (deleted)         partition       262136  0       -2

> But is the active swap space somehow tied to the device node
> /dev/vg0/swap in the file system.  

That certainly used to be the case - if you deleted an active swap file
it then became impossible to 'swapoff': don't do it and avoid running
tools that do it.

Don't know if this kernel issue's been addressed or not.

Alasdair
-- 
agk@redhat.com

Re: [linux-lvm] Running vgscan while swap on LVM is already active

[Urs Thuermann]

OK, following up to myself.  I've done some experiments to understand
what's going on in the kernel.  For those interested, here is what
I've just learned:

Urs Thuermann <urs@isnogud.escape.de> writes:

> But is the active swap space somehow tied to the device node
> /dev/vg0/swap in the file system?  I thought this is not the case.
> And even if it is, this behavior still suprises me because it the
> device node is recreated with the same device number and even inode
> number.

This wasn't quite correct.  When running vgscan while /dev/vg0/swap is
active, the device node has of course the same major/minor device
number after the vgscan but a new inode number.  So it seems that the
kernel keeps the inode of the device node open as long as the device
is used as swap.  vgscan unlinks /dev/vg0/swap, but the inode isn't
removed since it's still open.  Then vgscan recreates /dev/vg0/swap
but on a different inode.  This is why I get

isnogud:root# swapon -s
Filename                        Type            Size    Used    Priority
/dev/vg0/swap (deleted)         partition       262136  0       -2

like lsof on deleted files.  Another experiment also shows this:  I
mounted a newly creates file system to /mnt, created a device node for
my swap space on it and turn swap on on it:

   $ mkfs /dev/vg0/foo
   $ mount /dev/vg0/foo /mnt
   $ cp -a /dev/vg0/swap /mnt/s
   $ swapon /mnt/s

Then I can't unmount /mnt since it's busy because the kernel holds an
open inode on /mnt.  I first have to swapoff

   $ swapoff
   $ umount /mnt

This is different from mounting a file system, where the kernel
doesn't seem to keep the inode of the block device open.  You can
unmount the file system holding the device node while the device is
still being mounted and also vgscan can unlink and recreate the device
node with the same inode number because the inode isn't busy.
Therefore, you can still unlink LVs after vgscan but not run swapoff
on an LV after vgscan.

BTW, is there a way to see all open inodes in the kernel?  I haven't
found anything in the /proc FS.  lsof shows open inodes held by
processes but obviously the kernel itself can also have open inodes.


urs