[linux-lvm] Cannot delete lv

[linux-lvm] Cannot delete lv

[James B. Byrne]

CentOS-6.2

lvm2-2.02.87-6.el6

qemu-kvm-0.12.1.2-2.209.el6_2.1

I have encountered a problem deleting logical volumes
associated with kvm guests that have been removed.  These
guest virtual machines were deleted because somehow each
had become able to see some of the VirtIO disks assigned
as physical volumes to the others.

It was suggested to me that this may have been caused by
attempting to add a new VirtIO disk created through
virt-manager as a physical device before it was initially
partitioned.  Surprisingly, if this is indeed the case,
virt-manager does not initially partition VirtIO disks, as
it does IDE virtual disks.

However, whatever the proximate or root causes are, when
the damaged guests were removed via virt-manager, together
with their storage, the guest vms were removed but the
lvremove command failed to remove the assigned storage:

 /sbin/lvremove -f /dev/vg_vhost01/lv_vm_base
    Can't remove open logical volume "lv_vm_base"

This pattern of failure repeated because the device showed
that was opened by two processes.

[root@vhost01 ~]#  dmsetup info -c vg_vhost01-lv_vm_base
Name                  Maj Min Stat Open Targ Event  UUID
vg_vhost01-lv_vm_base 253   5 L--w    2    1      0
LVM-gXMt00E1RDjpSX3INLZ35Prtg66aX36BeAOlKIkmfSNQRNol3Hni920R4YVaZr52

However, these processes seem to be invisible to me:

[root@vhost01 ~]#
[root@vhost01 ~]# fuser /dev/vg_vhost01/lv_vm_base
[root@vhost01 ~]#
[root@vhost01 ~]# lsof /dev/vg_vhost01/lv_vm_base
[root@vhost01 ~]#
[root@vhost01 ~]# umount /dev/vg_vhost01/lv_vm_base
umount: /dev/vg_vhost01/lv_vm_base: not mounted
[root@vhost01 ~]#
[root@vhost01 ~]# /sbin/lvremove -f
/dev/vg_vhost01/lv_vm_base
  Can't remove open logical volume "lv_vm_base"
[root@vhost01 ~]#

It was suggested on the centos-virt mailing list that I
try using dmsetup to suspend this device:

[root@vhost01 ~]# dmsetup suspend vg_vhost01-lv_vm_base
[root@vhost01 ~]#
[root@vhost01 ~]# dmsetup info -c vg_vhost01-lv_vm_base
Name                  Maj Min Stat Open Targ Event  UUID
vg_vhost01-lv_vm_base 253   5 L-sw    2    1      0
LVM-gXMt00E1RDjpSX3INLZ35Prtg66aX36BeAOlKIkmfSNQRNol3Hni920R4YVaZr52
[root@vhost01 ~]#

Which apparently had no effect:

[root@vhost01 ~]#  dmsetup info -c vg_vhost01-lv_vm_base
Name                  Maj Min Stat Open Targ Event  UUID
vg_vhost01-lv_vm_base 253   5 L-sw    2    1      0
LVM-gXMt00E1RDjpSX3INLZ35Prtg66aX36BeAOlKIkmfSNQRNol3Hni920R4YVaZr52

However, now when I run lvremove, the command simply
becomes unresponsive and does not return, even when a ^C
interrupt is attempted.

[root@vhost01 ~]#  /sbin/lvremove -f
/dev/vg_vhost01/lv_vm_base

^C
^C


On another terminal, after some 15 minutes of waiting, I
see this:

[root@vhost01 ~]# ps -ef | grep remove
root      1460   307  0 10:07 pts/0    00:00:00
/sbin/lvremove -f /dev/vg_vhost01/lv_vm_base


How do I delete these logical volumes, for there are three
of them?

-- 
***          E-Mail is NOT a SECURE channel          ***
James B. Byrne                mailto:ByrneJB@Harte-Lyne.ca
Harte & Lyne Limited          http://www.harte-lyne.ca
9 Brockley Drive              vox: +1 905 561 1241
Hamilton, Ontario             fax: +1 905 561 0757
Canada  L8E 3C3

Re: [linux-lvm] Cannot delete lv

[James B. Byrne]

As a follow up the lvremove hang produces this in
/var/log/messages:

Jan 30 10:21:58 vhost01 kernel: INFO: task lvremove:1460
blocked for more than 120 seconds.
Jan 30 10:21:58 vhost01 kernel: "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this
message.
Jan 30 10:21:58 vhost01 kernel: lvremove      D
0000000000000002     0  1460    307 0x00000080
Jan 30 10:21:58 vhost01 kernel: ffff8801a170db18
0000000000000082 0000000000000000 ffffffffa00041fc
Jan 30 10:21:58 vhost01 kernel: ffff8801a170dae8
00000000d9f78a4a ffff8801a170db08 ffff8802750f7cc0
Jan 30 10:21:58 vhost01 kernel: ffff88042d589b38
ffff8801a170dfd8 000000000000f4e8 ffff88042d589b38
Jan 30 10:21:58 vhost01 kernel: Call Trace:
Jan 30 10:21:58 vhost01 kernel: [<ffffffffa00041fc>] ?
dm_table_unplug_all+0x5c/0x100 [dm_mod]
Jan 30 10:21:58 vhost01 kernel: [<ffffffff814ed1c3>]
io_schedule+0x73/0xc0
Jan 30 10:21:58 vhost01 kernel: [<ffffffff811b1a2e>]
__blockdev_direct_IO_newtrunc+0x6fe/0xb90
Jan 30 10:21:58 vhost01 kernel: [<ffffffff812581fd>] ?
get_disk+0x7d/0xf0
Jan 30 10:21:58 vhost01 kernel: [<ffffffff811b1f1e>]
__blockdev_direct_IO+0x5e/0xd0
Jan 30 10:21:58 vhost01 kernel: [<ffffffff811ae820>] ?
blkdev_get_blocks+0x0/0xc0
Jan 30 10:21:58 vhost01 kernel: [<ffffffff811af687>]
blkdev_direct_IO+0x57/0x60
Jan 30 10:21:58 vhost01 kernel: [<ffffffff811ae820>] ?
blkdev_get_blocks+0x0/0xc0
Jan 30 10:21:58 vhost01 kernel: [<ffffffff811128db>]
generic_file_aio_read+0x6bb/0x700
Jan 30 10:21:58 vhost01 kernel: [<ffffffff81213a21>] ?
avc_has_perm+0x71/0x90
Jan 30 10:21:58 vhost01 kernel: [<ffffffff8120d51f>] ?
security_inode_permission+0x1f/0x30
Jan 30 10:21:58 vhost01 kernel: [<ffffffff8117641a>]
do_sync_read+0xfa/0x140
Jan 30 10:21:58 vhost01 kernel: [<ffffffff81090bf0>] ?
autoremove_wake_function+0x0/0x40
Jan 30 10:21:58 vhost01 kernel: [<ffffffff811aec5c>] ?
block_ioctl+0x3c/0x40
Jan 30 10:21:58 vhost01 kernel: [<ffffffff81189342>] ?
vfs_ioctl+0x22/0xa0
Jan 30 10:21:58 vhost01 kernel: [<ffffffff8121900b>] ?
selinux_file_permission+0xfb/0x150
Jan 30 10:21:58 vhost01 kernel: [<ffffffff8120c3b6>] ?
security_file_permission+0x16/0x20
Jan 30 10:21:58 vhost01 kernel: [<ffffffff81176e15>]
vfs_read+0xb5/0x1a0
Jan 30 10:21:58 vhost01 kernel: [<ffffffff810d46e2>] ?
audit_syscall_entry+0x272/0x2a0
Jan 30 10:21:58 vhost01 kernel: [<ffffffff81176f51>]
sys_read+0x51/0x90
Jan 30 10:21:58 vhost01 kernel: [<ffffffff8100b0f2>]
system_call_fastpath+0x16/0x1b


Which repeats at two minute intevals.

-- 
***          E-Mail is NOT a SECURE channel          ***
James B. Byrne                mailto:ByrneJB@Harte-Lyne.ca
Harte & Lyne Limited          http://www.harte-lyne.ca
9 Brockley Drive              vox: +1 905 561 1241
Hamilton, Ontario             fax: +1 905 561 0757
Canada  L8E 3C3

Re: [linux-lvm] Cannot delete lv

[Bryn M. Reeves]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/30/2012 03:44 PM, James B. Byrne wrote:
> It was suggested on the centos-virt mailing list that I try using
> dmsetup to suspend this device:
> 
> [root@vhost01 ~]# dmsetup suspend vg_vhost01-lv_vm_base 
> [root@vhost01 ~]# [root@vhost01 ~]# dmsetup info -c
> vg_vhost01-lv_vm_base Name                  Maj Min Stat Open Targ
> Event  UUID vg_vhost01-lv_vm_base 253   5 L-sw    2    1      0 
> LVM-gXMt00E1RDjpSX3INLZ35Prtg66aX36BeAOlKIkmfSNQRNol3Hni920R4YVaZr52
>
> 
[root@vhost01 ~]#

Just suspending isn't going to help. Device-mapper allows you to
suspend a device, replace its definition (table) with a new one and
then resume it - on occasion this can be useful to allow you to
replace a device with a fake layer that always returns I/O errors (it
will cause outstanding I/O to fail and "unstick" any apps that were
blocked on the device).

This doesn't seem to be one of those cases however since your problem
is that something has the device open rather than that access to the
device itself blocks.

> However, now when I run lvremove, the command simply becomes
> unresponsive and does not return, even when a ^C interrupt is
> attempted.

That's because the device is suspended (all I/O will block). Resume it
and the lvremove will complete (with the same error as before).

# dmsetup resume <dev>

> How do I delete these logical volumes, for there are three of
> them?

You need to find out what has them open and get it to close them. If
the VMs have really been shut down then they should have closed the
devices already - run lsof to check that no qemu-kvm processes are
using them (but resume your LV first to avoid lsof blocking..).

Since these are VM images though I guess you may have some partition
mappings for them created by kpartx.

Examine the device dependencies with dmsetup ls --tree or lsblk:

# lsblk
NAME                            MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                               8:0    0 232.9G  0 disk
??sda1                            8:1    0   200M  0 part
??sda2                            8:2    0 195.3G  0 part
  ??vg_mother-lv_t0 (dm-12)     253:12   0   100M  0 lvm
    ??vg_mother-lv_t0p1 (dm-13) 253:13   0  94.1M  0 part <<<<<<<<

In this case I have an LV named vg_mother/lv_t0 that has a single
kpartx partition mapping.

If this is the case on your LV you can remove these mappings with
"kpartx -d <dev>" where <dev> is the whole LV device.

Regards,
Bryn.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8mvoEACgkQ6YSQoMYUY94hjgCgkeemE4oBzIcDVpHZrirGl9wv
AU4AoIF1VtGcB8nMatiJ+d3juXGNXjD0
=RxPI
-----END PGP SIGNATURE-----

Re: [linux-lvm] Cannot delete lv

[Bryn M. Reeves]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/30/2012 03:57 PM, James B. Byrne wrote:
> 
> As a follow up the lvremove hang produces this in 
> /var/log/messages:
> 
> Jan 30 10:21:58 vhost01 kernel: INFO: task lvremove:1460 blocked
> for more than 120 seconds. Jan 30 10:21:58 vhost01 kernel: "echo 0
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

You suspended a device and then issued I/O to it. It will carry on
doing this until you resume the suspended device.

Regards,
Bryn.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8mvqwACgkQ6YSQoMYUY94Z4gCg03oyG2ISLKW5BW8boVJdxPdv
WiEAoNXNWbrUtO1+qJoRUV5XGBcKGodP
=72FE
-----END PGP SIGNATURE-----

Re: [linux-lvm] Cannot delete lv

[James B. Byrne]

On Mon, January 30, 2012 11:00, Bryn M. Reeves wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 01/30/2012 03:44 PM, James B. Byrne wrote:
>> It was suggested on the centos-virt mailing list that I
>> try using
>> dmsetup to suspend this device:
>>
>> [root@vhost01 ~]# dmsetup suspend vg_vhost01-lv_vm_base
>> [root@vhost01 ~]# [root@vhost01 ~]# dmsetup info -c
>> vg_vhost01-lv_vm_base Name                  Maj Min Stat
>> Open Targ
>> Event  UUID vg_vhost01-lv_vm_base 253   5 L-sw    2    1
>>      0
>> LVM-gXMt00E1RDjpSX3INLZ35Prtg66aX36BeAOlKIkmfSNQRNol3Hni920R4YVaZr52
>>
>>
> [root@vhost01 ~]#
>
> Just suspending isn't going to help. Device-mapper allows
> you to
> suspend a device, replace its definition (table) with a
> new one and
> then resume it - on occasion this can be useful to allow
> you to
> replace a device with a fake layer that always returns I/O
> errors (it
> will cause outstanding I/O to fail and "unstick" any apps
> that were
> blocked on the device).
>
> This doesn't seem to be one of those cases however since
> your problem
> is that something has the device open rather than that
> access to the
> device itself blocks.
>
>> However, now when I run lvremove, the command simply
>> becomes
>> unresponsive and does not return, even when a ^C
>> interrupt is
>> attempted.
>
> That's because the device is suspended (all I/O will
> block). Resume it
> and the lvremove will complete (with the same error as
> before).
>
> # dmsetup resume <dev>
>
Done, blocking cleared and session returned.

>
> You need to find out what has them open and get it to
> close them. If
> the VMs have really been shut down then they should have
> closed the
> devices already - run lsof to check that no qemu-kvm
> processes are
> using them (but resume your LV first to avoid lsof
> blocking..).
>
> Since these are VM images though I guess you may have some
> partition mappings for them created by kpartx.
>
> Examine the device dependencies with dmsetup ls --tree or
> lsblk:

[root@vhost01 ~]# dmsetup resume vg_vhost01-lv_vm_base
[root@vhost01 ~]# dmsetup ls --tree
vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00p1 (253:10)
 &#9492;&#9472;vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00
(253:6)
    &#9492;&#9472; (8:2)
vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_01p1 (253:9)
 &#9492;&#9472;vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_01
(253:7)
    &#9492;&#9472; (8:2)
vg_vhost01-lv_swap (253:1)
 &#9492;&#9472; (8:2)
vg_vhost01-lv_root (253:0)
 &#9492;&#9472; (8:2)

---> this is a problem instance
vg_vhost01-lv_vm_basep2 (253:18)
 &#9492;&#9472;vg_vhost01-lv_vm_base (253:5)
    &#9492;&#9472; (8:2)
vg_vhost01-lv_vm_basep1 (253:17)
 &#9492;&#9472;vg_vhost01-lv_vm_base (253:5)
    &#9492;&#9472; (8:2)
<---

vg_vhost01-lv_centos_repos (253:8)
 &#9492;&#9472; (8:2)
vg_vhost01-lv_tmp (253:2)
 &#9492;&#9472; (8:2)
vg_vhost01-lv_vm_inet02.harte--lyne.ca_00 (253:21)
 &#9492;&#9472; (8:2)
vg_vhost01-lv_vm_inet03.harte--lyne.ca_00 (253:23)
 &#9492;&#9472; (8:2)

---> This is a problem instance
vg_vhost01-lv_vm_pas.harte--lyne.cap2 (253:16)
 &#9492;&#9472;vg_vhost01-lv_vm_pas.harte--lyne.ca (253:12)
    &#9492;&#9472; (8:2)
<---

vg_vhost01-lv_log (253:4)
 &#9492;&#9472; (8:2)
vg_vhost01-lv_vm_inet04.harte--lyne.ca_00 (253:25)
 &#9492;&#9472; (8:2)

---> This is a problem instance
vg_vhost01-lv_vm_pas.harte--lyne.cap1 (253:15)
 &#9492;&#9472;vg_vhost01-lv_vm_pas.harte--lyne.ca (253:12)
    &#9492;&#9472; (8:2)
<---

vg_vhost01-lv_spool (253:3)
 &#9492;&#9472; (8:2)

---> This is a prolem instance
vg_vhost01-lv_vm_pas.harte--lyne.ca_01p1 (253:14)
 &#9492;&#9472;vg_vhost01-lv_vm_pas.harte--lyne.ca_01
(253:13)
    &#9492;&#9472; (8:2)
<---

vg_vhost01-lv_vm_inet08.harte--lyne.ca_00 (253:24)
 &#9492;&#9472; (8:2)
vg_vhost01-lv_vm_inet09.harte--lyne.ca_00 (253:22)
 &#9492;&#9472; (8:2)

---> This is a problem instance
vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00p2 (253:11)
 &#9492;&#9472;vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00
(253:6)
    &#9492;&#9472; (8:2)
<---

[root@vhost01 ~]#

[root@vhost01 ~]# lsblk
NAME                                                      
    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sr0                                                       
     11:0    1     4G  0 rom
sda                                                       
      8:0    0 931.5G  0 disk
&#9500;&#9472;sda1                                        
                  8:1    0   500M  0 part /boot
&#9492;&#9472;sda2                                        
                  8:2    0   931G  0 part
  &#9500;&#9472;vg_vhost01-lv_root (dm-0)                 
                253:0    0  15.6G  0 lvm  /
  &#9500;&#9472;vg_vhost01-lv_swap (dm-1)                 
                253:1    0   7.8G  0 lvm  [SWAP]
  &#9500;&#9472;vg_vhost01-lv_tmp (dm-2)                  
                253:2    0   3.9G  0 lvm  /tmp
  &#9500;&#9472;vg_vhost01-lv_spool (dm-3)                
                253:3    0   7.8G  0 lvm  /var/spool
  &#9500;&#9472;vg_vhost01-lv_log (dm-4)                  
                253:4    0   7.8G  0 lvm  /var/log
  &#9500;&#9472;vg_vhost01-lv_vm_base (dm-5)              
                253:5    0   7.8G  0 lvm
  &#9474; &#9500;&#9472;vg_vhost01-lv_vm_basep1 (dm-17)   
                      253:17   0   500M  0 dm
  &#9474; &#9492;&#9472;vg_vhost01-lv_vm_basep2 (dm-18)   
                      253:18   0   7.3G  0 dm
  &#9500;&#9472;vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00
(dm-6)      253:6    0   7.8G  0 lvm
  &#9474;
&#9500;&#9472;vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00p1
(dm-10) 253:10   0   500M  0 dm
  &#9474;
&#9492;&#9472;vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00p2
(dm-11) 253:11   0   7.3G  0 dm
  &#9500;&#9472;vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_01
(dm-7)      253:7    0  31.3G  0 lvm
  &#9474;
&#9492;&#9472;vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_01p1
(dm-9)  253:9    0  31.3G  0 dm
  &#9500;&#9472;vg_vhost01-lv_vm_inet02.harte--lyne.ca_00
(dm-21)          253:21   0  31.3G  0 lvm
  &#9500;&#9472;vg_vhost01-lv_vm_inet08.harte--lyne.ca_00
(dm-24)          253:24   0  62.5G  0 lvm
  &#9500;&#9472;vg_vhost01-lv_centos_repos (dm-8)         
                253:8    0    64G  0 lvm  /var/data/
  &#9500;&#9472;vg_vhost01-lv_vm_pas.harte--lyne.ca
(dm-12)                253:12   0   7.8G  0 lvm
  &#9474;
&#9500;&#9472;vg_vhost01-lv_vm_pas.harte--lyne.cap1
(dm-15)            253:15   0   500M  0 dm
  &#9474;
&#9492;&#9472;vg_vhost01-lv_vm_pas.harte--lyne.cap2
(dm-16)            253:16   0   7.3G  0 dm
  &#9500;&#9472;vg_vhost01-lv_vm_pas.harte--lyne.ca_01
(dm-13)             253:13   0  62.5G  0 lvm
  &#9474;
&#9492;&#9472;vg_vhost01-lv_vm_pas.harte--lyne.ca_01p1
(dm-14)         253:14   0  62.5G  0 dm
  &#9500;&#9472;vg_vhost01-lv_vm_inet09.harte--lyne.ca_00
(dm-22)          253:22   0  31.3G  0 lvm
  &#9500;&#9472;vg_vhost01-lv_vm_inet03.harte--lyne.ca_00
(dm-23)          253:23   0  31.3G  0 lvm
  &#9492;&#9472;vg_vhost01-lv_vm_inet04.harte--lyne.ca_00
(dm-25)          253:25   0  31.3G  0 lvm


> In this case I have an LV named vg_mother/lv_t0 that
> has a single kpartx partition mapping.
>
> If this is the case on your LV you can remove these
> mappings with
> "kpartx -d <dev>" where <dev> is the whole LV device.

At this point I could use a little more guidance.  I wish
to remove all the lvs associated with the following
defunct vm guests:

base  <--- guest name
  vg_vhost01-lv_vm_base (dm-5)
  vg_vhost01-lv_vm_basep1 (dm-17)
  vg_vhost01-lv_vm_basep2 (dm-18)

pas.harte--lyne.ca  <--- guest name
  vg_vhost01-lv_vm_pas.harte--lyne.ca (dm-12)
  vg_vhost01-lv_vm_pas.harte--lyne.cap1 (dm-15)
  vg_vhost01-lv_vm_pas.harte--lyne.cap2 (dm-16)
  vg_vhost01-lv_vm_pas.harte--lyne.ca_01 (dm-13)
  vg_vhost01-lv_vm_pas.harte--lyne.ca_01p1 (dm-14)

pgsql--dbms.harte--lyne.ca  <--- guest name
  vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00 (dm-6)
  vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00p1 (dm-10)
  vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00p2 (dm-11)
  vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_01 (dm-7)
  vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_01p1 (dm-9)


-- 
***          E-Mail is NOT a SECURE channel          ***
James B. Byrne                mailto:ByrneJB@Harte-Lyne.ca
Harte & Lyne Limited          http://www.harte-lyne.ca
9 Brockley Drive              vox: +1 905 561 1241
Hamilton, Ontario             fax: +1 905 561 0757
Canada  L8E 3C3

Re: [linux-lvm] Cannot delete lv

[Bryn M. Reeves]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/30/2012 04:25 PM, James B. Byrne wrote:
> At this point I could use a little more guidance.  I wish to remove
> all the lvs associated with the following defunct vm guests:
> 
> base  <--- guest name vg_vhost01-lv_vm_base (dm-5) 
> vg_vhost01-lv_vm_basep1 (dm-17) vg_vhost01-lv_vm_basep2 (dm-18)
> 
> pas.harte--lyne.ca  <--- guest name 
> vg_vhost01-lv_vm_pas.harte--lyne.ca (dm-12) 
> vg_vhost01-lv_vm_pas.harte--lyne.cap1 (dm-15) 
> vg_vhost01-lv_vm_pas.harte--lyne.cap2 (dm-16) 
> vg_vhost01-lv_vm_pas.harte--lyne.ca_01 (dm-13) 
> vg_vhost01-lv_vm_pas.harte--lyne.ca_01p1 (dm-14)
> 
> pgsql--dbms.harte--lyne.ca  <--- guest name 
> vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00 (dm-6) 
> vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00p1 (dm-10) 
> vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00p2 (dm-11) 
> vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_01 (dm-7) 
> vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_01p1 (dm-9)

These LVs all have active kpartx mappings (the names with pN
suffixes). Remove them with kpartx -d:

# kpartx -d /dev/mapper/vg_vhost01-lv_vm_base
# kpartx -d /dev/mapper/vg_vhost01-lv_vm_pas.harte--lyne.ca
# kpartx -d /dev/mapper/vg_vhost01-lv_vm_pas.harte--lyne.ca_01
# kpartx -d /dev/mapper/vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00
# kpartx -d /dev/mapper/vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_01

Regards,
Bryn.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8mxaQACgkQ6YSQoMYUY957KACg3LuLaueEx2NDBUDyLe5vVILR
d3cAn0+qhBcPlYaEz73d2gywKHPzu5kZ
=Oi4S
-----END PGP SIGNATURE-----

Re: [linux-lvm] Cannot delete lv

[James B. Byrne]

On Mon, January 30, 2012 11:30, Bryn M. Reeves wrote:
)
>
> These LVs all have active kpartx mappings (the names with
> pN
> suffixes). Remove them with kpartx -d:
>
> # kpartx -d /dev/mapper/vg_vhost01-lv_vm_base
> # kpartx -d
> /dev/mapper/vg_vhost01-lv_vm_pas.harte--lyne.ca
> # kpartx -d
> /dev/mapper/vg_vhost01-lv_vm_pas.harte--lyne.ca_01
> # kpartx -d
> /dev/mapper/vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_00
> # kpartx -d
> /dev/mapper/vg_vhost01-lv_vm_pgsql--dbms.harte--lyne.ca_01
>
> Regards,
> Bryn.


I tried this with no change in the result.

[root@vhost01 ~]# kpartx -d /dev/mapper/vg_vhost01-lv_vm_base
[root@vhost01 ~]# kpartx -d
/dev/mapper/vg_vhost01-lv_vm_basep1
[root@vhost01 ~]# kpartx -d
/dev/mapper/vg_vhost01-lv_vm_basep2
[root@vhost01 ~]# /sbin/lvremove -f
/dev/vg_vhost01/lv_vm_base
  Can't remove open logical volume "lv_vm_base"

[root@vhost01 ~]# dmsetup info -c vg_vhost01-lv_vm_base
Name                  Maj Min Stat Open Targ Event  UUID
vg_vhost01-lv_vm_base 253   5 L--w    2    1      0
LVM-gXMt00E1RDjpSX3INLZ35Prtg66aX36BeAOlKIkmfSNQRNol3Hni920R4YVaZr52

[root@vhost01 ~]# lsblk
NAME                                 MAJ:MIN RM   SIZE RO
TYPE MOUNTPOINT
.  .  .
  ├─vg_vhost01-lv_vm_base (dm-5)              
                253:5    0   7.8G  0 lvm
  │ ├─vg_vhost01-lv_vm_basep1 (dm-17)   
                      253:17   0   500M  0 dm
  │ └─vg_vhost01-lv_vm_basep2 (dm-18)   
                      253:18   0   7.3G  0 dm

.  .  .

Am I misunderstanding your instructions?


-- 
***          E-Mail is NOT a SECURE channel          ***
James B. Byrne                mailto:ByrneJB@Harte-Lyne.ca
Harte & Lyne Limited          http://www.harte-lyne.ca
9 Brockley Drive              vox: +1 905 561 1241
Hamilton, Ontario             fax: +1 905 561 0757
Canada  L8E 3C3

Re: [linux-lvm] Cannot delete lv

[Bryn M. Reeves]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/30/2012 04:50 PM, James B. Byrne wrote:
> [root@vhost01 ~]# lsblk NAME
> MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT .  .  . 
> ├─vg_vhost01-lv_vm_base (dm-5) 253:5    0   7.8G  0
> lvm │ ├─vg_vhost01-lv_vm_basep1 (dm-17) 253:17
> 0   500M  0 dm │ └─vg_vhost01-lv_vm_basep2
> (dm-18) 253:18   0   7.3G  0 dm
> 
> .  .  .
> 
> Am I misunderstanding your instructions?

The partition maps were apparently created with kpartx's "-p p" option
to force a partition separator of 'p' (the default is to use a 'p' if
the device name ends with a numeral but no separator otherwise).

If the maps were created with -pp kpartx will refuse to remove them if
the option is not given (but helpfully doesn't report any error.. :-/):

# ls /dev/mapper/vg_mother-lv_base*
/dev/mapper/vg_mother-lv_base  /dev/mapper/vg_mother-lv_basep1
# kpartx -d /dev/mapper/vg_mother-lv_base
# ls /dev/mapper/vg_mother-lv_base*
/dev/mapper/vg_mother-lv_base  /dev/mapper/vg_mother-lv_basep1
# kpartx -d -pp /dev/mapper/vg_mother-lv_base
# ls /dev/mapper/vg_mother-lv_base*
/dev/mapper/vg_mother-lv_base

Just add the -pp to your kpartx command lines and it should remove the
devices.

Regards,
Bryn.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8mzX0ACgkQ6YSQoMYUY96ySwCZATKsF6+UiQ1FKtwQK/4XX2uP
G20AniKYezGGkZreHCvp0onymhIpXp8M
=tTC2
-----END PGP SIGNATURE-----

Re: [linux-lvm] Cannot delete lv

[James B. Byrne]

On Mon, January 30, 2012 12:03, Bryn M. Reeves wrote:

>
> The partition maps were apparently created with kpartx's
> "-p p" option
> to force a partition separator of 'p' (the default is to
> use a 'p' if
> the device name ends with a numeral but no separator
> otherwise).
>
> If the maps were created with -pp kpartx will refuse to
> remove them if
> the option is not given (but helpfully doesn't report any
> error.. :-/):
>
> # ls /dev/mapper/vg_mother-lv_base*
> /dev/mapper/vg_mother-lv_base
> /dev/mapper/vg_mother-lv_basep1
> # kpartx -d /dev/mapper/vg_mother-lv_base
> # ls /dev/mapper/vg_mother-lv_base*
> /dev/mapper/vg_mother-lv_base
> /dev/mapper/vg_mother-lv_basep1
> # kpartx -d -pp /dev/mapper/vg_mother-lv_base
> # ls /dev/mapper/vg_mother-lv_base*
> /dev/mapper/vg_mother-lv_base
>
> Just add the -pp to your kpartx command lines and it
> should remove the
> devices.
>

[root@vhost01 ~]# kpartx -d -pp
/dev/mapper/vg_vhost01-lv_vm_base
[root@vhost01 ~]# kpartx -d -pp
/dev/mapper/vg_vhost01-lv_vm_basep1
failed to stat() /dev/mapper/vg_vhost01-lv_vm_basep1
[root@vhost01 ~]# /sbin/lvremove -f
/dev/vg_vhost01/lv_vm_base
  Logical volume "lv_vm_base" successfully removed

And repeated as required.

Success!  Thank you so very much. I would never have
discovered this on my own. Never.

[root@vhost01 ~]# dmsetup ls --tree
vg_vhost01-lv_swap (253:1)
 └─ (8:2)
vg_vhost01-lv_root (253:0)
 └─ (8:2)
vg_vhost01-lv_centos_repos (253:8)
 └─ (8:2)
vg_vhost01-lv_tmp (253:2)
 └─ (8:2)
vg_vhost01-lv_vm_inet02.harte--lyne.ca_00 (253:21)
 └─ (8:2)
vg_vhost01-lv_vm_inet03.harte--lyne.ca_00 (253:23)
 └─ (8:2)
vg_vhost01-lv_log (253:4)
 └─ (8:2)
vg_vhost01-lv_vm_inet04.harte--lyne.ca_00 (253:25)
 └─ (8:2)
vg_vhost01-lv_spool (253:3)
 └─ (8:2)
vg_vhost01-lv_vm_inet08.harte--lyne.ca_00 (253:24)
 └─ (8:2)
vg_vhost01-lv_vm_inet09.harte--lyne.ca_00 (253:22)
 └─ (8:2)

Everything appears clean now and I have confirmed that the
disk space has been returned to the virt-manager storage
pool.

Again, thank you.

-- 
***          E-Mail is NOT a SECURE channel          ***
James B. Byrne                mailto:ByrneJB@Harte-Lyne.ca
Harte & Lyne Limited          http://www.harte-lyne.ca
9 Brockley Drive              vox: +1 905 561 1241
Hamilton, Ontario             fax: +1 905 561 0757
Canada  L8E 3C3