tcmu-runner (target userspace passthrough daemon) development

tcmu-runner (target userspace passthrough daemon) development

[Andy Grover]

Hi all,

tcmu-runner is a userspace daemon that simplifies the configuration and 
processing of SCSI commands from LIO to userspace handlers, via the new 
TCMU userspace passthrough backstore.

https://github.com/agrover/tcmu-runner

As a proof-of-concept, I've implemented a Gluster backend handler. I'm 
looking for code review and collaborators, as well as suggestions on 
what other userspace handlers we might want to work on. Tape or optical 
jukebox emulation? VMDK files?? Video RAM???

Thanks -- Regards -- Andy

(BTW, tcmu-runner is not kernel code but it's very tightly linked to 
LIO, so recommend we use this list (target-devel) for discussion.)

Re: tcmu-runner (target userspace passthrough daemon) development

[Nicholas A. Bellinger]

On Tue, 2015-01-20 at 15:27 -0800, Andy Grover wrote:
> Hi all,
> 
> tcmu-runner is a userspace daemon that simplifies the configuration and 
> processing of SCSI commands from LIO to userspace handlers, via the new 
> TCMU userspace passthrough backstore.
> 
> https://github.com/agrover/tcmu-runner
> 

Thanks for posting.  ;)

> As a proof-of-concept, I've implemented a Gluster backend handler. I'm 
> looking for code review and collaborators, as well as suggestions on 
> what other userspace handlers we might want to work on. Tape or optical 
> jukebox emulation? VMDK files?? Video RAM???

How about a user-space handler for qcow2, so we could have a mechanism
for exposing qcow2 images into KVM guest using vhost-scsi export..?

Ideally there would be a library that QEMU provides so that from TCMU's
user-space handler perspective, it's simply issuing fd reads/writes into
qcow2 library code that is doing the fancy feature stuff outside of
user-space handler logic..

--nab

Re: tcmu-runner (target userspace passthrough daemon) development

[Andy Grover]

On 01/20/2015 04:35 PM, Nicholas A. Bellinger wrote:
>> As a proof-of-concept, I've implemented a Gluster backend handler. I'm
>> looking for code review and collaborators, as well as suggestions on
>> what other userspace handlers we might want to work on. Tape or optical
>> jukebox emulation? VMDK files?? Video RAM???
>
> How about a user-space handler for qcow2, so we could have a mechanism
> for exposing qcow2 images into KVM guest using vhost-scsi export..?
>
> Ideally there would be a library that QEMU provides so that from TCMU's
> user-space handler perspective, it's simply issuing fd reads/writes into
> qcow2 library code that is doing the fancy feature stuff outside of
> user-space handler logic..

Reusing qemu's BlockDriver support would enable LIO luns to be backed 
not just by qcow2 files, but by pretty much EVERY image format, plus 
Gluster, Ceph, sheepdog, all in one go.

The obstacle is that there is not a library. The programs in qemu like 
qemu-img and qemu-nbd that share blockdriver code are in the qemu tree 
and just link in the block .o files. The most straightforward solution 
is to add a qemu-lio-tcmu.so that similarly statically links in all the 
BlockDrivers, and is also a tcmu-runner handler module.

The issues with this are:

1) Getting the qemu build system to do what we want. It has an elaborate 
build system and I lack build-fu. I got this working about 5 months ago 
but then my changes experienced bitrot.

2) Code-wise, decide on whether to be a user of the qemu block 
subsystem, or to pretend to *be* the qemu block subsystem. If we're a 
user then we get nice APIs like bdrv_pwrite() that hide things like 
async or coroutines from us, but require us to call qemu_init_main_loop, 
which does a lot of stuff that assumes it's its own process, mucks with 
signals, etc. If we pretend to be the block subsystem then we need to do 
ourselves much of what the block subsystem does, to use all of the 
registered BlockDrivers in the way they expect.

3) Get it accepted into qemu. Qemu developers' interest in LIO is that a 
baremetal instance booted off an iscsi lun, and the lun is backed by a 
qcow2 file, enables snapshots & migration between the baremetal and virt 
really easily. This involves us supporting QMP, the qemu remote 
configuration protocol. Which sounds weird to me. Check out this msg and 
others on the thread:

http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00920.html


I'm still confused, honestly. I think we just want the blockdrivers, and 
to not have to maintain them ourselves, but need a solution the qemu 
developers will accept, too.

-- Andy