Better to use iSCSI on host or guest?

Better to use iSCSI on host or guest?

[Emmanuel Noobadmin]

I'm planning to use iSCSI targets (over gigabit VLANs) for KVM guest
disks. The question I'm wondering about is whether it's better to md
(multi-path + mirror) the iSCSI targets on the host, then create LVM
partitions for the guests. Or to directly md the iSCSI targets within
the guest.

On one hand I think it would be slower to process the additional
layers in the guest, but on the other hand, readings seems to indicate
that the kernel is able to perform better disk i/o if it's aware of
multiple disks rather than just seeing a single disk.

I've not been able to find any definitive article/data on how these
might balance out. Would anybody in the list have a good idea which
way is better in terms of i/o performance?

Re: Better to use iSCSI on host or guest?

[Avi Kivity]

On 06/13/2011 07:05 PM, Emmanuel Noobadmin wrote:
> I'm planning to use iSCSI targets (over gigabit VLANs) for KVM guest
> disks. The question I'm wondering about is whether it's better to md
> (multi-path + mirror) the iSCSI targets on the host, then create LVM
> partitions for the guests. Or to directly md the iSCSI targets within
> the guest.
>
> On one hand I think it would be slower to process the additional
> layers in the guest, but on the other hand, readings seems to indicate
> that the kernel is able to perform better disk i/o if it's aware of
> multiple disks rather than just seeing a single disk.
>
> I've not been able to find any definitive article/data on how these
> might balance out. Would anybody in the list have a good idea which
> way is better in terms of i/o performance?

My gut feeling is to do iscsi in the host.  I guess it's best to measure 
though.  Please post your findings if you do that.

-- 
error compiling committee.c: too many arguments to function

Re: Better to use iSCSI on host or guest?

[Emmanuel Noobadmin]

On 6/14/11, Avi Kivity <avi@redhat.com> wrote:
> My gut feeling is to do iscsi in the host.  I guess it's best to measure
> though.  Please post your findings if you do that.

Any suggestions or recommendations as to how/what should I be measuring with?

So far in trying to determine how bad is the qcow2 disk bottleneck on
my VMs, I've been using dd with dsync and direct options as well as
hdparm which errors out (not sure if this is considered a KVM/virtio
bug?) on the virtio disk after getting a buffered result.

But these don't seem to be very good tools since it's all sequential
only. I've tried iozone on my home machine but it takes too long per
run; unfortunately this host and its VMs are live and I don't have the
luxury of another full set of hardware to test on.

Re: Better to use iSCSI on host or guest?

[Avi Kivity]

On 06/13/2011 10:00 PM, Emmanuel Noobadmin wrote:
> On 6/14/11, Avi Kivity<avi@redhat.com>  wrote:
> >  My gut feeling is to do iscsi in the host.  I guess it's best to measure
> >  though.  Please post your findings if you do that.
>
> Any suggestions or recommendations as to how/what should I be measuring with?

Whatever workload you'll be running on those guests.

> So far in trying to determine how bad is the qcow2 disk bottleneck on
> my VMs, I've been using dd with dsync and direct options as well as
> hdparm which errors out (not sure if this is considered a KVM/virtio
> bug?) on the virtio disk after getting a buffered result.
>
> But these don't seem to be very good tools since it's all sequential
> only. I've tried iozone on my home machine but it takes too long per
> run; unfortunately this host and its VMs are live and I don't have the
> luxury of another full set of hardware to test on.

fio is a good tool.

-- 
error compiling committee.c: too many arguments to function