[linux-lvm] Converting existing thick LVs into thin LVs possible?

[linux-lvm] Converting existing thick LVs into thin LVs possible?

[Jeremy Smith]

I have a system currently using LVM2 that was originally set up on
RHEL5 (where thin provisioning was not supported).
This system would greatly benefit from the newer thin provisioning
features, and so I am investigating upgrading to RHEL6 so that I may
take advantage. My question is this: Can I convert existing LVM2
"thick" logical volumes into thinly-provisioned volumes on the fly?

This page:
 https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Logical_Volume_Manager_Administration/thinly_provisioned_volume_creation.html

lead me to believe that it is possible:
 "You can use the --thinpool parameter of the lvconvert command to
convert an existing logical volume to a thin volume."

with the following:

 lvconvert --thinpool vg001/lv1 --poolmetadata vg001/lv2

However, I seem to not understand how that works. From the example, I
would assume that vg001/lv1 is the existing "thick" LV that I want to
convert and that vg001/lv2 is an existing LV with enough space to
become the metadata. So, I create a metadata LV with:

test@testbox:~$ sudo lvcreate  -L16G MyVG -n Metadata
  Logical volume "Metadata" created

But when I attempt the conversion, I do not get what I would expect:

test@testbox:~$ sudo lvconvert --thinpool  MyVG/OldThickLV1
--poolmetadata MyVG/Metadata
  Converted MyVG/OldThickLV1 to thin pool.

Then the OldThickLV1 disappears, leaving Metadata around:

test@testbox:~$ sudo lvdisplay
  --- Logical volume ---
  LV Name                Metadata
  VG Name                MyVG
  LV UUID                IZb6zP-D7iD-NswX-XcDZ-K5fN-sGBj-lCMiqO
  LV Write Access        read/write
  LV Creation host, time testbox, 2014-02-14 13:50:04 -0500
  LV Pool transaction ID 0
  LV Pool metadata       Metadata_tmeta
  LV Pool data           Metadata_tdata
  LV Pool chunk size     64.00 KiB
  LV Zero new blocks     no
  LV Status              available
  # open                 0
  LV Size                16.00 GiB
  Allocated pool data    0.00%
  Allocated metadata     0.01%
  Current LE             4096
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           252:5


Am I doing something wrong here?
Or am I going to have to create a new thin pool, new thin logical
volumes, and copy the data over?


Thanks,

-- 
Jeremy

Re: [linux-lvm] Converting existing thick LVs into thin LVs possible?

[Zdenek Kabelac]

Dne 14.2.2014 20:21, Jeremy Smith napsal(a):
> I have a system currently using LVM2 that was originally set up on
> RHEL5 (where thin provisioning was not supported).
> This system would greatly benefit from the newer thin provisioning
> features, and so I am investigating upgrading to RHEL6 so that I may
> take advantage. My question is this: Can I convert existing LVM2
> "thick" logical volumes into thinly-provisioned volumes on the fly?
>
> This page:
>   https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Logical_Volume_Manager_Administration/thinly_provisioned_volume_creation.html
>
> lead me to believe that it is possible:
>   "You can use the --thinpool parameter of the lvconvert command to
> convert an existing logical volume to a thin volume."
>
> with the following:
>
>   lvconvert --thinpool vg001/lv1 --poolmetadata vg001/lv2


Sounds like misunderstanding which needs fix/enhancement of this doc page  - 
we will need to enhance lvconvert command to write big warning about this 
operation.

This conversion is converting  volume to 'data' volume - but it's not about
preserving content of LV1 as  LV1 - it will serve as en empty pool.

The primary purpose is - you can build 'any' type of LV for data volume
and 'any' type of LV for metadata volume (i.e. special layout for raid/mirror)
and then you 'create' for these 2 volumes your thin pool - since normally
you can't create mirrored data & metadata volumes and thin pool in 1 command,
and you can't easily select where the metadata should be placed
(i.e. data on spindle, metadata on SSD).

>
> Am I doing something wrong here?
> Or am I going to have to create a new thin pool, new thin logical
> volumes, and copy the data over?

There is not yet support for direct conversion of LV to thin-LV.
The only currently supported method is to use your  thick-LV as an external 
origin LV for your  thin-LV volume.

i.e. you create your  thin-pool  LV  + and then thin-LV volume with external 
origin that will use original thick-LV  (external origin need to be read-only 
volume)

Here are the conversion steps:

lvcreate -Lsize_of_data_pool -T vg/pool
lvconvert -T  vg/thick_LV --originname new_thick_LV_name --thinpool  vg/pool

Disadvantage here is you cannot merge changes from thin volume back to 
original thick volume. You can however remove  thin-LV and use again your 
thick-LV as normal volume - i.e. thin-LV is basically snapshot.

Now hint for recovery - if you have not yet written anything to thin-LV from 
newly create thin-pool.  You could restore back to your previous configuration
via   vgcfgrestore --force.

Zdenek

Re: [linux-lvm] Converting existing thick LVs into thin LVs possible?

[Jeremy Smith]

On Mon, Feb 17, 2014 at 4:00 AM, Zdenek Kabelac <zkabelac@redhat.com> wrote:
> Dne 14.2.2014 20:21, Jeremy Smith napsal(a):
>>
>> I have a system currently using LVM2 that was originally set up on
>> RHEL5 (where thin provisioning was not supported).
>> This system would greatly benefit from the newer thin provisioning
>> features, and so I am investigating upgrading to RHEL6 so that I may
>> take advantage. My question is this: Can I convert existing LVM2
>> "thick" logical volumes into thinly-provisioned volumes on the fly?
>>
>> This page:
>>
>> https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Logical_Volume_Manager_Administration/thinly_provisioned_volume_creation.html
>>
>> lead me to believe that it is possible:
>>   "You can use the --thinpool parameter of the lvconvert command to
>> convert an existing logical volume to a thin volume."
>>
>> with the following:
>>
>>   lvconvert --thinpool vg001/lv1 --poolmetadata vg001/lv2
>
>
>
> Sounds like misunderstanding which needs fix/enhancement of this doc page  -
> we will need to enhance lvconvert command to write big warning about this
> operation.
>
> This conversion is converting  volume to 'data' volume - but it's not about
> preserving content of LV1 as  LV1 - it will serve as en empty pool.
>
> The primary purpose is - you can build 'any' type of LV for data volume
> and 'any' type of LV for metadata volume (i.e. special layout for
> raid/mirror)
> and then you 'create' for these 2 volumes your thin pool - since normally
> you can't create mirrored data & metadata volumes and thin pool in 1
> command,
> and you can't easily select where the metadata should be placed
> (i.e. data on spindle, metadata on SSD).
>
>>
>> Am I doing something wrong here?
>> Or am I going to have to create a new thin pool, new thin logical
>> volumes, and copy the data over?
>
>
> There is not yet support for direct conversion of LV to thin-LV.
> The only currently supported method is to use your  thick-LV as an external
> origin LV for your  thin-LV volume.
>
> i.e. you create your  thin-pool  LV  + and then thin-LV volume with external
> origin that will use original thick-LV  (external origin need to be
> read-only volume)
>
> Here are the conversion steps:
>
> lvcreate -Lsize_of_data_pool -T vg/pool
> lvconvert -T  vg/thick_LV --originname new_thick_LV_name --thinpool  vg/pool
>
> Disadvantage here is you cannot merge changes from thin volume back to
> original thick volume. You can however remove  thin-LV and use again your
> thick-LV as normal volume - i.e. thin-LV is basically snapshot.
>
> Now hint for recovery - if you have not yet written anything to thin-LV from
> newly create thin-pool.  You could restore back to your previous
> configuration
> via   vgcfgrestore --force.
>
> Zdenek
>

Zdenek,

Thank you. I was able to perform migration of some of my LVs, but now
I have ran into a problem: the thin pool metadata has ran out of space
:-(
I am running LVM 2.02.98(2) (2012-10-15) on kernel 3.11.0-18-generic
(Ubuntu server 13.10), which -- as I understand from your talk at
https://www.youtube.com/watch?v=dGax09aw7WI -- does not support live
resizing of the pool metadata.
You mention in that talk that there is a manual offline pool metadata
resize method that can be done on kernels prior to 3.14. I was not
able to find details anywhere on how to accomplish this. Could you
point me in the right direction on how to fix a thin pool (and its
thin logical volumes) that has reached the metadata size limit? It
would be greatly appreciated.

Regards,

-- 
Jeremy Smith
Systems Administrator
eSilo, LLC.
1530 Cypress Drive, Suite H
Jupiter, FL. 33469
561-747-6880 x 412 (Phone)
800-747-9255 x 412 (Toll Free)
866-598-1358 (After Hours)
561-747-6488 (Fax)
jeremy@esilo.com

Re: [linux-lvm] Converting existing thick LVs into thin LVs possible?

[Zdenek Kabelac]

Dne 4.4.2014 18:50, Jeremy Smith napsal(a):
> On Mon, Feb 17, 2014 at 4:00 AM, Zdenek Kabelac <zkabelac@redhat.com> wrote:
>> Dne 14.2.2014 20:21, Jeremy Smith napsal(a):
>>>
>>> I have a system currently using LVM2 that was originally set up on
>>> RHEL5 (where thin provisioning was not supported).
>>> This system would greatly benefit from the newer thin provisioning
>>> features, and so I am investigating upgrading to RHEL6 so that I may
>>> take advantage. My question is this: Can I convert existing LVM2
>>> "thick" logical volumes into thinly-provisioned volumes on the fly?
>>>
>>> This page:
>>>
>>> https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Logical_Volume_Manager_Administration/thinly_provisioned_volume_creation.html
>>>
>>> lead me to believe that it is possible:
>>>    "You can use the --thinpool parameter of the lvconvert command to
>>> convert an existing logical volume to a thin volume."
>>>
>>> with the following:
>>>
>>>    lvconvert --thinpool vg001/lv1 --poolmetadata vg001/lv2
>>
>>
>>
>> Sounds like misunderstanding which needs fix/enhancement of this doc page  -
>> we will need to enhance lvconvert command to write big warning about this
>> operation.
>>
>> This conversion is converting  volume to 'data' volume - but it's not about
>> preserving content of LV1 as  LV1 - it will serve as en empty pool.
>>
>> The primary purpose is - you can build 'any' type of LV for data volume
>> and 'any' type of LV for metadata volume (i.e. special layout for
>> raid/mirror)
>> and then you 'create' for these 2 volumes your thin pool - since normally
>> you can't create mirrored data & metadata volumes and thin pool in 1
>> command,
>> and you can't easily select where the metadata should be placed
>> (i.e. data on spindle, metadata on SSD).
>>
>>>
>>> Am I doing something wrong here?
>>> Or am I going to have to create a new thin pool, new thin logical
>>> volumes, and copy the data over?
>>
>>
>> There is not yet support for direct conversion of LV to thin-LV.
>> The only currently supported method is to use your  thick-LV as an external
>> origin LV for your  thin-LV volume.
>>
>> i.e. you create your  thin-pool  LV  + and then thin-LV volume with external
>> origin that will use original thick-LV  (external origin need to be
>> read-only volume)
>>
>> Here are the conversion steps:
>>
>> lvcreate -Lsize_of_data_pool -T vg/pool
>> lvconvert -T  vg/thick_LV --originname new_thick_LV_name --thinpool  vg/pool
>>
>> Disadvantage here is you cannot merge changes from thin volume back to
>> original thick volume. You can however remove  thin-LV and use again your
>> thick-LV as normal volume - i.e. thin-LV is basically snapshot.
>>
>> Now hint for recovery - if you have not yet written anything to thin-LV from
>> newly create thin-pool.  You could restore back to your previous
>> configuration
>> via   vgcfgrestore --force.
>>
>> Zdenek
>>
>
> Zdenek,
>
> Thank you. I was able to perform migration of some of my LVs, but now
> I have ran into a problem: the thin pool metadata has ran out of space
> :-(
> I am running LVM 2.02.98(2) (2012-10-15) on kernel 3.11.0-18-generic
> (Ubuntu server 13.10), which -- as I understand from your talk at
> https://www.youtube.com/watch?v=dGax09aw7WI -- does not support live
> resizing of the pool metadata.
> You mention in that talk that there is a manual offline pool metadata
> resize method that can be done on kernels prior to 3.14. I was not
> able to find details anywhere on how to accomplish this. Could you
> point me in the right direction on how to fix a thin pool (and its
> thin logical volumes) that has reached the metadata size limit? It
> would be greatly appreciated.

Offline resize is basically  metadata recovery into bigger LV.
So step for resize apply in the same way as for recovery.

1. create temp LV
2. replace temp LV with  metadata pool LV
3. now you can activate your metadata LV as regular LV
4. create bigger LV for restore
5. thin_restore  -i old_metadata_lv_path  -o  new_metadata_lv_path
6. swap new_metadata_lv  into your pool
7. activate pool

yep - about the time to integrate this offline resize into lvm2 I guess... :)

Zdenek

Re: [linux-lvm] Converting existing thick LVs into thin LVs possible?

[Zdenek Kabelac]

Dne 14.4.2014 10:01, Zdenek Kabelac napsal(a):
> Dne 4.4.2014 18:50, Jeremy Smith napsal(a):
>> On Mon, Feb 17, 2014 at 4:00 AM, Zdenek Kabelac <zkabelac@redhat.com> wrote:
>>> Dne 14.2.2014 20:21, Jeremy Smith napsal(a):
>>>>
>>>> I have a system currently using LVM2 that was originally set up on
>>>> RHEL5 (where thin provisioning was not supported).
>>>> This system would greatly benefit from the newer thin provisioning
>>>> features, and so I am investigating upgrading to RHEL6 so that I may
>>>> take advantage. My question is this: Can I convert existing LVM2
>>>> "thick" logical volumes into thinly-provisioned volumes on the fly?
>>>>
>>>> This page:
>>>>
>>>> https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Logical_Volume_Manager_Administration/thinly_provisioned_volume_creation.html
>>>>
>>>>
>>>> lead me to believe that it is possible:
>>>>    "You can use the --thinpool parameter of the lvconvert command to
>>>> convert an existing logical volume to a thin volume."
>>>>
>>>> with the following:
>>>>
>>>>    lvconvert --thinpool vg001/lv1 --poolmetadata vg001/lv2
>>>
>>>
>>>
>>> Sounds like misunderstanding which needs fix/enhancement of this doc page  -
>>> we will need to enhance lvconvert command to write big warning about this
>>> operation.
>>>
>>> This conversion is converting  volume to 'data' volume - but it's not about
>>> preserving content of LV1 as  LV1 - it will serve as en empty pool.
>>>
>>> The primary purpose is - you can build 'any' type of LV for data volume
>>> and 'any' type of LV for metadata volume (i.e. special layout for
>>> raid/mirror)
>>> and then you 'create' for these 2 volumes your thin pool - since normally
>>> you can't create mirrored data & metadata volumes and thin pool in 1
>>> command,
>>> and you can't easily select where the metadata should be placed
>>> (i.e. data on spindle, metadata on SSD).
>>>
>>>>
>>>> Am I doing something wrong here?
>>>> Or am I going to have to create a new thin pool, new thin logical
>>>> volumes, and copy the data over?
>>>
>>>
>>> There is not yet support for direct conversion of LV to thin-LV.
>>> The only currently supported method is to use your  thick-LV as an external
>>> origin LV for your  thin-LV volume.
>>>
>>> i.e. you create your  thin-pool  LV  + and then thin-LV volume with external
>>> origin that will use original thick-LV  (external origin need to be
>>> read-only volume)
>>>
>>> Here are the conversion steps:
>>>
>>> lvcreate -Lsize_of_data_pool -T vg/pool
>>> lvconvert -T  vg/thick_LV --originname new_thick_LV_name --thinpool  vg/pool
>>>
>>> Disadvantage here is you cannot merge changes from thin volume back to
>>> original thick volume. You can however remove  thin-LV and use again your
>>> thick-LV as normal volume - i.e. thin-LV is basically snapshot.
>>>
>>> Now hint for recovery - if you have not yet written anything to thin-LV from
>>> newly create thin-pool.  You could restore back to your previous
>>> configuration
>>> via   vgcfgrestore --force.
>>>
>>> Zdenek
>>>
>>
>> Zdenek,
>>
>> Thank you. I was able to perform migration of some of my LVs, but now
>> I have ran into a problem: the thin pool metadata has ran out of space
>> :-(
>> I am running LVM 2.02.98(2) (2012-10-15) on kernel 3.11.0-18-generic
>> (Ubuntu server 13.10), which -- as I understand from your talk at
>> https://www.youtube.com/watch?v=dGax09aw7WI -- does not support live
>> resizing of the pool metadata.
>> You mention in that talk that there is a manual offline pool metadata
>> resize method that can be done on kernels prior to 3.14. I was not
>> able to find details anywhere on how to accomplish this. Could you
>> point me in the right direction on how to fix a thin pool (and its
>> thin logical volumes) that has reached the metadata size limit? It
>> would be greatly appreciated.
>
> Offline resize is basically  metadata recovery into bigger LV.
> So step for resize apply in the same way as for recovery.
>
> 1. create temp LV
> 2. replace temp LV with  metadata pool LV
> 3. now you can activate your metadata LV as regular LV
> 4. create bigger LV for restore
> 5. thin_restore  -i old_metadata_lv_path  -o  new_metadata_lv_path

oops hit send button to early...
of course thin_repair


> 6. swap new_metadata_lv  into your pool
> 7. activate pool
>
> yep - about the time to integrate this offline resize into lvm2 I guess... :)
>
> Zdenek
>