Re: Xen PVSCSI: status, issues and some tests

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Sun, Feb 19, 2012 at 03:22:08AM +0100, Thomas Weyergraf wrote:
> Hi, I am working as a system administrator at an internet platform
> service provider, and I am currently seeking to re-new our Xen
> virtualization infrastructure for which I am mostly responsible for.
> Currently, we run Xen 3.4.2/3.4.3 on RHEL/CentOS 5.x (5.7) as Dom0
> with CentOS 5.x pv-guests.
> 
> Based on my experiments, I am currently looking into Xen 4.1.2 on
> RHEL/CentOS 6.x (6.2), with a vanilla 3.2 (3.2.0/3.2.6) kernel as
> Dom0 and stock RHEL/CentOS 6.x guests as HVM/pv-ops DomUs.

Sweet.
> 
> Our DomU's have their disk-devices on iscsi-storage backends (EMC,
> netapp, Linux IET), we attach the disks to Dom0 via open-iscsi
> initiator and pass them as 'phy'-type blockdevices, using
> blkfront/back.

OK.
> 
> As this is the time for a major overhaul anyway, I looked at
> 'pvscsi', which is very attractive to me for a bunch of
> (administrative) reasons. I have created a working pvscsi-setup with
> the following steps:
> 1. Use CentOS 6.2 as base system
> 2. Add Xen hypervisor from Michael Young's repository [1]. This is
> stock 4.1.2 with a patch from 4.1-testing pulled-in and various
> patches fixing compile-time issues, toolchain problems and system
> service management on RHEL-alike systems. (Nice work, btw!)
> 3. Use a vanilla 3.2.0 kernel from kernel.org and a .config based on
> a Fedora 16 stock-kernel config, tweaked to use pvscsi. As you know,
> Fedora 16 supports Dom0 operation and the only changes to the config
> are non-Xen related or add the Xen pvscsi config. I refrained from
> attaching the config to avoid mailinglist-clutter, but can do so on
> demand.

Awesome.
> 4. I pulled and applied the pvscsi-patch found in this post [2] from
> Konrad Rzeszutek Wilk.
> 
> Putting it all together yielded a "working setup" in which I am able
> to pass an Dom0-attached iscsi target to a DomU as a vscsi-device.
> In DomU context, I could partition the device, create a filesystem
> on it and copy/read/verify some 10gig of data to it. Very basic
> testing until now, essentially. However, the whole issue raised some
> questions:
> 
> - I pvscsi actively maintained? Is someone working on this or even
> prepare it for upstream inclusion in the pv-ops framework of the 3.x
> series?

So nobody has stepped up to do the upstream task and it is on my todo list
but mostly at the end.

> - Has anybody started on adding the vscsi-semantics to the xl toolstack?

Not that I know of.

> - Using 'xm', I was only able to add a device via its SCSI tupel
> (HCTL, e.g: vscsi = [ '7:0:0:0, 0:0:0:1' ]) but not by any other
> device-node, such as '/dev/sde' or the
> '/dev/disk/by-path/<iscsi-iqn>' link. For these invocations, an
> "Error: Cannot find device <device>" message is given. I have not
> yet looked at the issue in detail (in
> /usr/lib64/python2.6/site-packages/xen/util/vscsi_util.py). Is this
> a known issue?

First time I hear of it. But then I didn't even think to use /dev/sde
to pass in a device since SCSI devices aren't neccessarily block ones.
They can be scanners, medical devices, tape drivers, etc - which
might not have a /dev/sdX (thought they will have an /dev/sgX).

> 
> Is there any developer(s) working on getting pvscsi ready for
> kernel.org 3.x upstream inclusion? Should i better retool my efforts
> using pv-ops xen/stable 2.6.32 series?

For 3.x upstream inclusion - not that I know off.
Why would you want to retool your effort in using the 2.6.32 series
as opposed to 3.x? What other items is the 3.x missing compared to 2.6.32?

Now the good news is that I am happy to keep this going (pv-scsi)
and fixing it, and having incremental updates to make it better.

> 
> If applicable, I'd be more than happy to provide assistance to move
> forward pvscsi. Maybe, as a start, provide a short, practical howto
> about what I did above to get pvscsi working and reference it by the
> Xen PVSCSI wiki page [3]?

Sure. Also, did you use the git tree to pull the patches or just
the big patch file ? The git tree has fixes to make it work under 3.2 as well.

> 
> Regards,
> Thomas Weyergraf
> 
> [1] http://xenbits.xen.org/people/mayoung/EL6.xen/
> [2] http://lists.xen.org/archives/html/xen-devel/2011-11/msg02268.html
> [3] http://wiki.xen.org/xenwiki/XenPVSCSI
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel