Re: EXT4 vs LVM performance for VMs

[Theodore Ts'o <tytso@mit.edu>]

On Fri, Feb 12, 2016 at 10:09:39AM +0100, Premysl Kouril wrote:
> > Except for the last the backtraces you're showing are for futex locks,
> > which are not used by the kernel, but some user process. So the locking
> > problem is somewhere in the user space setup (perhaps in qemu).
> > This would indicate your ext4 set up is not the same as LVM.
> 
> Setup on the qemu/kvm is exactly the same (same test box, same command
> line arguments execept for the argument for the virtual machine disk
> which reference LVM volume or EXT4 based file)

You mentioned using qcow; if you're using qcow, than the userspace
qemu/kvm process will need to do its own locking to manage its own
space management file.

In general, if you need to do allocation management, either because
you are writing to a sparse file, and ext4 has to do block allocation,
or if you are writing to a fallocated file, and ext4 has to keep track
of whether a block has been written to that location before (and if
not, do a journalled transaction to clear the unwritten bit), there
will be extra work that has to be done at the qcow or ext4 layer that
doesn't have to be done at the LVM layer.  Of course, this work is
providing extra services (such as space management and/or not
revealing previously written block contents from another customer to
your current customer's VM, which might make your local data
protection authorities cranky).

So the devil is very much in the details of how you set up the
hypervisor, and "except for the argument for the virtual machine disk"
is leaving an awful lot unspecified.  Whether you preallocated and
pre-zeroed the file can make a difference, whether you are using
buffered or direct I/O based on the cache parameter makes a
difference, etc., etc.

Cheers,

						- Ted