deduplication and SE virtual machines

deduplication and SE virtual machines

[Russell Coker]

Are there any good software-only (IE nothing expensive like NetApp) options 
for deduplication of SE Linux virtual machine images?

A basic LVM writable snapshot allows creating a SE Linux test image with less 
disk space used.  When LVM is used for Xen does the Dom0 do any caching?  
Ideally we would have the Dom0 do read-caching of the read-only device.

http://micolous.id.au/archives/2010/01/23/linux-iscsi-cow-images-and-windows-
integration/

Apparently dmsetup supports COW images, the man page on my system doesn't 
document it so maybe I would need a newer version.  I'm not sure what benefit 
this would give over LVM snapshots given that LVM does it's stuff on top of DM 
anyway.

http://user-mode-linux.sourceforge.net/old/UserModeLinux-HOWTO-7.html

A COW block device for User-Mode Linux does the same thing but possibly works 
better with caching, the above URL documents UML COW as giving performance 
improvements but I'm unsure of LVM.

Are there any others?


I'm thinking of setting up a Linux virtual machine server for SE Linux 
training, and with most COW setups the initial relabel of the filesystem will 
put all the meta-data into a separate copy.  So ideally there would be a way 
of deduplicating this.  Also there's no guarantee that the disk blocks used 
would be the same so a simple un-COW operation on LVM or something probably 
wouldn't be a viable option.

I believe that NetApp does this well, but there's no chance of getting that 
sort of hardware.

-- 
russell@coker.com.au
http://etbe.coker.com.au/          My Main Blog
http://doc.coker.com.au/           My Documents Blog

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: deduplication and SE virtual machines

[Serge E. Hallyn]

Quoting Russell Coker (russell@coker.com.au):
> I'm thinking of setting up a Linux virtual machine server for SE Linux 
> training, and with most COW setups the initial relabel of the filesystem will 
> put all the meta-data into a separate copy.  So ideally there would be a way 
> of deduplicating this.  Also there's no guarantee that the disk blocks used 
> would be the same so a simple un-COW operation on LVM or something probably 
> wouldn't be a viable option.

Here's an idea - you could create the base fs as a qcow2 block device.
Create copy-on-write images based on that
	for i in `seq 1 20`; do
		qemu-img create -f qcow2 -b selinux-base.img selinux-vm$i.img
	done
Then use qemu-nbd to export those as /dev/nbdX devices
	for i in `seq 1 20`; do
		qemu-nbd -c /dev/nbd$i selinux-vm$i
	done

I'm guessing at the commands as I haven't quite done it.  But then your
containers or VMS or chroots or whatever can mount /dev/nbd$i like a
normal block device, COW based on the same base image.

I'm not sure that would suffice though, if there are a lot of small
files, since presumably the xattrs will be spread out along with the
data.  So if that does not suffice (I'd love to hear a report if anyone
tests this), then I think we have another motivator for pushing a
'xattr_file=' mount option, where the specified file has
(inode_num,name,value) triplets for the inodes on the fs, i.e.
	25,security.selinux,root_u:root_r:root_t
	25,security.capabilities,<whatever>
	30,security.selinux,user_u:user_r:serge_t
That way the base fs wouldn't need to change much at all for each
of your VMs.  The other motivator of course is common filesystems
which don't support xattrs like squashfs and CIFS.  I wonder what
sort of reception such a patch would receive...  "welcome back to
year 2000"?

-serge

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: deduplication and SE virtual machines

[Russell Coker]

On Thu, 8 Jul 2010, "Serge E. Hallyn" <serge@hallyn.com> wrote:
> Here's an idea - you could create the base fs as a qcow2 block device.
> Create copy-on-write images based on that
> 	for i in `seq 1 20`; do
> 		qemu-img create -f qcow2 -b selinux-base.img selinux-vm$i.img
> 	done
> Then use qemu-nbd to export those as /dev/nbdX devices
> 	for i in `seq 1 20`; do
> 		qemu-nbd -c /dev/nbd$i selinux-vm$i
> 	done
> 
> I'm guessing at the commands as I haven't quite done it.  But then your
> containers or VMS or chroots or whatever can mount /dev/nbd$i like a
> normal block device, COW based on the same base image.

That's an interesting concept.  I guess I can use NBD over ::1 for Xen too.

Also I guess if I wanted to have multiple Xen servers then I could have one 
machine supplying all the main storage disk and the others just having disks 
for swap spaces.

Is there any Xen management software to setup dozens of virtual machines with 
user-names and passwords associated with them to permit all management tasks 
including create, destroy, and view the console of Xen servers?  Please reply 
off-list if you know of any such software.
 
> I'm not sure that would suffice though, if there are a lot of small
> files, since presumably the xattrs will be spread out along with the
> data.  So if that does not suffice (I'd love to hear a report if anyone
> tests this), then I think we have another motivator for pushing a
> 'xattr_file=' mount option, where the specified file has
> (inode_num,name,value) triplets for the inodes on the fs, i.e.
> 	25,security.selinux,root_u:root_r:root_t
> 	25,security.capabilities,<whatever>
> 	30,security.selinux,user_u:user_r:serge_t
> That way the base fs wouldn't need to change much at all for each
> of your VMs.  The other motivator of course is common filesystems
> which don't support xattrs like squashfs and CIFS.  I wonder what
> sort of reception such a patch would receive...  "welcome back to
> year 2000"?

No, it's more like back to 2003 or whenever it was such support was removed.  
;)

But seriously I think there is a good reason for having this, probably not for 
deduplication of unusual cases of virtual machines but for filesystems that 
don't have native support for labeling.

-- 
russell@coker.com.au
http://etbe.coker.com.au/          My Main Blog
http://doc.coker.com.au/           My Documents Blog

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.