Xen PVSCSI: status, issues and some tests

[Thomas Weyergraf <T.Weyergraf@virtfinity.de>]

Hi, I am working as a system administrator at an internet platform 
service provider, and I am currently seeking to re-new our Xen 
virtualization infrastructure for which I am mostly responsible for.
Currently, we run Xen 3.4.2/3.4.3 on RHEL/CentOS 5.x (5.7) as Dom0 with 
CentOS 5.x pv-guests.

Based on my experiments, I am currently looking into Xen 4.1.2 on 
RHEL/CentOS 6.x (6.2), with a vanilla 3.2 (3.2.0/3.2.6) kernel as Dom0 
and stock RHEL/CentOS 6.x guests as HVM/pv-ops DomUs.

Our DomU's have their disk-devices on iscsi-storage backends (EMC, 
netapp, Linux IET), we attach the disks to Dom0 via open-iscsi initiator 
and pass them as 'phy'-type blockdevices, using blkfront/back.

As this is the time for a major overhaul anyway, I looked at 'pvscsi', 
which is very attractive to me for a bunch of (administrative) reasons. 
I have created a working pvscsi-setup with the following steps:
1. Use CentOS 6.2 as base system
2. Add Xen hypervisor from Michael Young's repository [1]. This is stock 
4.1.2 with a patch from 4.1-testing pulled-in and various patches fixing 
compile-time issues, toolchain problems and system service management on 
RHEL-alike systems. (Nice work, btw!)
3. Use a vanilla 3.2.0 kernel from kernel.org and a .config based on a 
Fedora 16 stock-kernel config, tweaked to use pvscsi. As you know, 
Fedora 16 supports Dom0 operation and the only changes to the config are 
non-Xen related or add the Xen pvscsi config. I refrained from attaching 
the config to avoid mailinglist-clutter, but can do so on demand.
4. I pulled and applied the pvscsi-patch found in this post [2] from 
Konrad Rzeszutek Wilk.

Putting it all together yielded a "working setup" in which I am able to 
pass an Dom0-attached iscsi target to a DomU as a vscsi-device. In DomU 
context, I could partition the device, create a filesystem on it and 
copy/read/verify some 10gig of data to it. Very basic testing until now, 
essentially. However, the whole issue raised some questions:

- I pvscsi actively maintained? Is someone working on this or even 
prepare it for upstream inclusion in the pv-ops framework of the 3.x series?
- Has anybody started on adding the vscsi-semantics to the xl toolstack?
- Using 'xm', I was only able to add a device via its SCSI tupel (HCTL, 
e.g: vscsi = [ '7:0:0:0, 0:0:0:1' ]) but not by any other device-node, 
such as '/dev/sde' or the '/dev/disk/by-path/<iscsi-iqn>' link. For 
these invocations, an "Error: Cannot find device <device>" message is 
given. I have not yet looked at the issue in detail (in 
/usr/lib64/python2.6/site-packages/xen/util/vscsi_util.py). Is this a 
known issue?

Is there any developer(s) working on getting pvscsi ready for kernel.org 
3.x upstream inclusion? Should i better retool my efforts using pv-ops 
xen/stable 2.6.32 series?

If applicable, I'd be more than happy to provide assistance to move 
forward pvscsi. Maybe, as a start, provide a short, practical howto 
about what I did above to get pvscsi working and reference it by the Xen 
PVSCSI wiki page [3]?

Regards,
Thomas Weyergraf

[1] http://xenbits.xen.org/people/mayoung/EL6.xen/
[2] http://lists.xen.org/archives/html/xen-devel/2011-11/msg02268.html
[3] http://wiki.xen.org/xenwiki/XenPVSCSI