cannot create VM with image on zfs filesystem (zfsonlinux)

cannot create VM with image on zfs filesystem (zfsonlinux)

[Peter Lees]

hello all

when trying to create a VM using a RAW image on a zfs filesystem, i get an 
"invalid argument" error.

has anyone made this work? is it a known issue and/or is there a workaround?




more detail:

- i can create the VM on a ext4 filesystem
- i can create the VM on a ZFS zvol ("raw" device - too slow, however)
- linux is centos 6 (2.6.32-71.el6.x86_64 )
- zfsonlinux is v0.6.0 (http://zfsonlinux.org)
- qemu-kvm is v0.12.1.2 ; release  2.113.el6_0.8
- copying the VM disk from ext4fs to zfs & then running it results in a similar error


disk creation line was:

# qemu-img create -f raw elveros_001.raw 15G


full error was:
Unable to complete install '<class 'libvirt.libvirtError'> internal error Process exited while reading console log output: char device redirected to /dev/pts/3
qemu: could not open disk image /space/vm_store/elveros_001.raw: Invalid argument

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/create.py", line 1561, in do_install
    dom = guest.start_install(False, meter = meter)
  File "/usr/lib/python2.6/site-packages/virtinst/Guest.py", line 973, in start_install
    return self._do_install(consolecb, meter, removeOld, wait)
  File "/usr/lib/python2.6/site-packages/virtinst/Guest.py", line 1038, in _do_install
    "install")
  File "/usr/lib/python2.6/site-packages/virtinst/Guest.py", line 1009, in _create_guest
    dom = self.conn.createLinux(start_xml, 0)
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 1270, in createLinux
    if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self)



any suggestions in getting ZFS as the vm_store, working with decent performance ?

tia

p

--
peter lees

 

Re: cannot create VM with image on zfs filesystem (zfsonlinux)

[Michael Tokarev]

On 07.11.2011 17:57, Peter Lees wrote:
> when trying to create a VM using a RAW image on a zfs filesystem, i get an 
> "invalid argument" error.
[]
> qemu: could not open disk image /space/vm_store/elveros_001.raw: Invalid argument

Please run qemu under strace to show what exactly it is doing.

/mjt

Re: cannot create VM with image on zfs filesystem (zfsonlinux)

[Avi Kivity]

On 11/07/2011 03:57 PM, Peter Lees wrote:
> hello all
>
> when trying to create a VM using a RAW image on a zfs filesystem, i get an 
> "invalid argument" error.
>
> has anyone made this work? is it a known issue and/or is there a workaround?
>
>
>
>
> more detail:
>
> - i can create the VM on a ext4 filesystem
> - i can create the VM on a ZFS zvol ("raw" device - too slow, however)
> - linux is centos 6 (2.6.32-71.el6.x86_64 )
> - zfsonlinux is v0.6.0 (http://zfsonlinux.org)
> - qemu-kvm is v0.12.1.2 ; release  2.113.el6_0.8
> - copying the VM disk from ext4fs to zfs & then running it results in a similar error
>
>
> disk creation line was:
>
> # qemu-img create -f raw elveros_001.raw 15G
>
>
> full error was:
> Unable to complete install '<class 'libvirt.libvirtError'> internal error Process exited while reading console log output: char device redirected to /dev/pts/3
> qemu: could not open disk image /space/vm_store/elveros_001.raw: Invalid argument

I'm guessing that qemu tried to open the file with O_DIRECT, and
zfsonlinux doesn't support O_DIRECT.  Please try changing the cache mode
to writeback.

> any suggestions in getting ZFS as the vm_store, working with decent performance ?
>

zfsonlinux uses fuse, yes?  I'd be surprised if you can get decent
performance with it.

-- 
error compiling committee.c: too many arguments to function

Re: cannot create VM with image on zfs filesystem (zfsonlinux)

[Peter Lees]

On 08/11/2011, at 2:22 AM, Avi Kivity <avi@redhat.com> wrote:

> On 11/07/2011 03:57 PM, Peter Lees wrote:
>> hello all
>> 
>> when trying to create a VM using a RAW image on a zfs filesystem, i get an 
>> "invalid argument" error.
>> 
>> has anyone made this work? is it a known issue and/or is there a workaround?
>> 
>> 
>> 
>> 
>> more detail:
>> 
>> - i can create the VM on a ext4 filesystem
>> - i can create the VM on a ZFS zvol ("raw" device - too slow, however)
>> - linux is centos 6 (2.6.32-71.el6.x86_64 )
>> - zfsonlinux is v0.6.0 (http://zfsonlinux.org)
>> - qemu-kvm is v0.12.1.2 ; release  2.113.el6_0.8
>> - copying the VM disk from ext4fs to zfs & then running it results in a similar error
>> 
>> 
>> disk creation line was:
>> 
>> # qemu-img create -f raw elveros_001.raw 15G
>> 
>> 
>> full error was:
>> Unable to complete install '<class 'libvirt.libvirtError'> internal error Process exited while reading console log output: char device redirected to /dev/pts/3
>> qemu: could not open disk image /space/vm_store/elveros_001.raw: Invalid argument
> 
> I'm guessing that qemu tried to open the file with O_DIRECT, and
> zfsonlinux doesn't support O_DIRECT.  Please try changing the cache mode
> to writeback.
> 
>> any suggestions in getting ZFS as the vm_store, working with decent performance ?
>> 
> 
> zfsonlinux uses fuse, yes?  I'd be surprised if you can get decent
> performance with it.

http://zfsonlinux.org

this is a source-release kernel module; it doesn't use fuse, and performance is ok for filesystem operations. no good at all for zvol operations, however.

thanks for the tip, i will give that a go

p

--
Peter Lees

Re: cannot create VM with image on zfs filesystem (zfsonlinux)

[Peter Lees]

On 08/11/2011, at 8:51 AM, Peter Lees wrote:

> 
> 
> On 08/11/2011, at 2:22 AM, Avi Kivity <avi@redhat.com> wrote:
> 
>> On 11/07/2011 03:57 PM, Peter Lees wrote:
>>> hello all
>>> 
>>> when trying to create a VM using a RAW image on a zfs filesystem, i get an 
>>> "invalid argument" error.
>>> 
>>> has anyone made this work? is it a known issue and/or is there a workaround?
>>> [...]
>>> full error was:
>>> Unable to complete install '<class 'libvirt.libvirtError'> internal error Process exited while reading console log output: char device redirected to /dev/pts/3
>>> qemu: could not open disk image /space/vm_store/elveros_001.raw: Invalid argument
>> 
>> I'm guessing that qemu tried to open the file with O_DIRECT, and
>> zfsonlinux doesn't support O_DIRECT.  Please try changing the cache mode
>> to writeback.
>> 
>>> any suggestions in getting ZFS as the vm_store, working with decent performance ?
>>> 
>> 
>> zfsonlinux uses fuse, yes?  I'd be surprised if you can get decent
>> performance with it.
> 
> http://zfsonlinux.org
> 
> this is a source-release kernel module; it doesn't use fuse, and performance is ok for filesystem operations. no good at all for zvol operations, however.
> 
> thanks for the tip, i will give that a go



as a follow-up, setting the cache mode to writeback did fix the problem.

performance is many times what i was getting with zvols, even if it's not as good as with ext4fs. i now have a more portable vm datastore, however.

thanks

p