[linux-lvm] Virtualization and LVM data security

[linux-lvm] Virtualization and LVM data security

[IB Development Team]

Hello,

Is there any way to make LVM2 tools wipe added/freed LV space or plans to add such functionality?

When LVM based storage is used for guest virtual disks, it is possible that after 
resizing/snapshoting LV, disk data fragments from one guest will be visible to other guest, which 
may cause serious security problems if not wiped somehow; some pages with more info in this topic:

http://blog.brightbox.co.uk/posts/secure-virtual-disk-deletion-is-your-data-safe
http://brightbox.com/blog/2012/04/27/dirty-disks/
http://docs.openstack.org/security-guide/content/ch046_data-residency.html

Don't know LVM2 internals well but if there is no such functionality in LVM2 now, maybe adding 
options like --wipe and --wipe-bandwidth (to allow one to control I/O load while wiping) for 
create/resize/remove/snapshot commands (and other maybe if such risk exist there) will be possible 
in future LVM versions to better meet security requirements in virtualized environments?

Regards,
Pawel

IB Development Team
http://dev.ib.pl/

Re: [linux-lvm] Virtualization and LVM data security

[Zdenek Kabelac]

Dne 24.10.2014 v 19:30 IB Development Team napsal(a):
> Hello,
>
> Is there any way to make LVM2 tools wipe added/freed LV space or plans to add
> such functionality?
>

lvm.conf    devices { issue_discard = 1 }

See it that fits your need ?
Note: when using this option - vg/lvremove becomes 'irreversible'operation.

> When LVM based storage is used for guest virtual disks, it is possible that
> after resizing/snapshoting LV, disk data fragments from one guest will be
> visible to other guest, which may cause serious security problems if not wiped
> somehow; some pages with more info in this topic:
>
> http://blog.brightbox.co.uk/posts/secure-virtual-disk-deletion-is-your-data-safe
> http://brightbox.com/blog/2012/04/27/dirty-disks/
> http://docs.openstack.org/security-guide/content/ch046_data-residency.html
>
> Don't know LVM2 internals well but if there is no such functionality in LVM2
> now, maybe adding options like --wipe and --wipe-bandwidth (to allow one to
> control I/O load while wiping) for create/resize/remove/snapshot commands (and
> other maybe if such risk exist there) will be possible in future LVM versions
> to better meet security requirements in virtualized environments?

thin provisioning with zeroing enabled for thin-pool -Zy is likely better option.

There is no extra utility like  'fstrim' from empty space in volume group.

Note: you could obviously implement 'workaround' something like:

lvcreate -l100%FREE -n trim_me vg
blkdiscard /dev/vg/trim_me
(or if disk doesn't support TRIM -   dd if=/dev/zero of=/dev/vg/trim_me....)
lvremove vg/trim_me

Zdenek

Re: [linux-lvm] Virtualization and LVM data security

[IB Development Team]

W dniu 2014-10-25 o 14:50, Zdenek Kabelac pisze:

>> Is there any way to make LVM2 tools wipe added/freed LV space or plans to add
>> such functionality?

> lvm.conf    devices { issue_discard = 1 }
>
> See it that fits your need ?
> Note: when using this option - vg/lvremove becomes 'irreversible'operation.

issue_discard seems to require "underlying storage support" which is probably not available in 
common RAID/SATA/SAS/DRBD scenarios. Universal, open (source) solution would be better here probably 
(with hardware alternatives where possible).

>> When LVM based storage is used for guest virtual disks, it is possible that
>> after resizing/snapshoting LV, disk data fragments from one guest will be
>> visible to other guest, which may cause serious security problems if not wiped
>> somehow[...]

> thin provisioning with zeroing enabled for thin-pool -Zy is likely better option.

Sounds interesting. Is it stable solution for production systems? Does it perform not worse than 
"regular" preallocated LV?

> Note: you could obviously implement 'workaround' something like:
>
> lvcreate -l100%FREE -n trim_me vg
> blkdiscard /dev/vg/trim_me
> (or if disk doesn't support TRIM -   dd if=/dev/zero of=/dev/vg/trim_me....)
> lvremove vg/trim_me

If I understand correctly, in this scenario, guest data may still be present outside "cleaned" LV 
(i.e. data that was saved outside LV in snapshot LV during backups). If so - cleaning should be 
probably done transparently by LVM "software" layer, even without "underlying storage support".

Regards,
Pawel

IB Development Team
https://dev.ib.pl/

Re: [linux-lvm] Virtualization and LVM data security

[Zdenek Kabelac]

Dne 25.10.2014 v 19:38 IB Development Team napsal(a):
> W dniu 2014-10-25 o 14:50, Zdenek Kabelac pisze:
>
>>> Is there any way to make LVM2 tools wipe added/freed LV space or plans to add
>>> such functionality?
>
>> lvm.conf    devices { issue_discard = 1 }
>>
>> See it that fits your need ?
>> Note: when using this option - vg/lvremove becomes 'irreversible'operation.
>
> issue_discard seems to require "underlying storage support" which is probably
> not available in common RAID/SATA/SAS/DRBD scenarios. Universal, open (source)
> solution would be better here probably (with hardware alternatives where
> possible).

Yes - this discard needs to be implemented by underlaying storage.

>
>>> When LVM based storage is used for guest virtual disks, it is possible that
>>> after resizing/snapshoting LV, disk data fragments from one guest will be
>>> visible to other guest, which may cause serious security problems if not wiped
>>> somehow[...]
>
>> thin provisioning with zeroing enabled for thin-pool -Zy is likely better
>> option.
>
> Sounds interesting. Is it stable solution for production systems? Does it
> perform not worse than "regular" preallocated LV?

Provisioning does not come for 'free' - so you pay some price when you zero 
provisioned blocks obviously - but blocks are zeroed on demand when they are 
going to be used.

For production system - do not over provision space - if you promise too much 
space and you don't have it - it currently require certain manual admin skills 
to proceed with overfilled pool volumes - it's not yet fully automated.

>> Note: you could obviously implement 'workaround' something like:
>>
>> lvcreate -l100%FREE -n trim_me vg
>> blkdiscard /dev/vg/trim_me
>> (or if disk doesn't support TRIM -   dd if=/dev/zero of=/dev/vg/trim_me....)
>> lvremove vg/trim_me
>
> If I understand correctly, in this scenario, guest data may still be present
> outside "cleaned" LV (i.e. data that was saved outside LV in snapshot LV
> during backups). If so - cleaning should be probably done transparently by LVM
> "software" layer, even without "underlying storage support".

There is no such support on lvm2 side - it's much easier to be implemented
on the user side.

Before you call 'lvremove' - just dd zero volume - if lvm would be zeroing
devices i.e. 1TB volume on lvremove - that would be timely insane operation.

Regards

Zdenek