From mboxrd@z Thu Jan  1 00:00:00 1970
From: infernix <infernix@infernix.net>
Subject: KVM incompatible with multipath?
Date: Sat, 27 Jun 2009 17:57:02 +0200
Message-ID: <4A46414E.6050505@infernix.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
To: kvm@vger.kernel.org
Return-path: <kvm-owner@vger.kernel.org>
Received: from dx.infernix.net ([83.96.151.193]:60446 "EHLO dx.infernix.net"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1754532AbZF0QJy (ORCPT <rfc822;kvm@vger.kernel.org>);
	Sat, 27 Jun 2009 12:09:54 -0400
Received: from [192.168.0.163] (believe.infernix.lan [192.168.0.163])
	by dx.infernix.net (8.14.3/8.14.3/Debian-5) with ESMTP id n5RFv3rq030254
	for <kvm@vger.kernel.org>; Sat, 27 Jun 2009 17:57:04 +0200
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

Hi,

I've got an Equallogic iSCSI box with 3x1GBit nics on my storage
network. I also have a server with 4 nics of which 3 are connected to
the storage network with 3x1GBit.

On this host, open-iscsi is running and connecting to the SAN over 3 
interfaces. I'm seeing /dev/sdb, sdc and sdd, which are all the same 
harddisk (same WWID).

With multipath I combine them to one disk to get a nice boost in
performance. This works perfectly fine (and very speedy) on the host.
Relevant multipath.conf data:

device {
  vendor "EQLOGIC"
  product "100E-00"
  path_grouping_policy multibus
  getuid_callout "/lib/udev/scsi_id -g -u -s /block/%n"
  features "1 queue_if_no_path"
  path_checker readsector0
  failback immediate
  path_selector "round-robin 0"
  rr_weight uniform
  rr_min_io 128
}

The problem with KVM (version 87) is that KVM cannot make use of the
multipath device in /dev/mapper.

If I boot a debootstrapped Debian Lenny KVM guest and configure it with
-hda /dev/mapper/my-multipathed-iscsi-disk, I cannot mount the root
partition. It'll complain about "Invalid target".

If I use -hda /dev/sdb or sdc or sdd (e.g. any of the multipathed disks,
which again are all the same but just through 3 separate connections),
everything works perfectly fine. Note that the behavior is the same when
using virtio.


I have tried to put LVM on top of the multipath device in the host, and
then run the KVM guest with -hda /dev/volumegroup/logicalvolume-blah but
that leads to serious data corruption. If I mount one of the lvm
partitions on the host, copy data to it, then unmount it and boot the
KVM guest, the data isn't there. If I shutdown the guest and do an
e2fsck on the host, I get lots of missing/unlinked inodes and a lot of
it is moved to /lost+found.


Does anyone have experience with KVM on multipathed disks? I'm really
puzzled here because everything works perfectly fine from the hosts
perspective.

Thanks for any help!

Regards,

infernix