[LSF/MM TOPIC] Target fabric module for SCSI para-virtualization

[LSF/MM TOPIC] Target fabric module for SCSI para-virtualization

[Nicholas A. Bellinger]

Greetings all,

[Topic]

A hybrid target fabric module for QEMU/KVM SCSI para-virtualization

[Abstract]

Currently we use the TCM_Loop v4 fabric module with mainline .38 target
code to present SCSI LUNs with high level multi-fabric (iSCSI, FC, SAS)
SPC-3 port WWN emulation into KVM guests using two different forms of
QEMU HBA hardware emulation.  However both of these currently require
interaction with the QEMU block layer and extra overhead of OS
independent code running in user-space connected to our existing
SG_IO/BSG interfaces.  This also has the limitiation that we currently
cannot perform explict SCSI Initiator Port access management for LUNs
also present via TCM_Loop SCSI LLD driver on the KVM host.

Recently there has been an off-list discussion between myself and
members of the QEMU/KVM community, and they have expressed an interest
in seeing a native paravirtualized SCSI passthrough available to QEMU
guests with direct interaction into TCM fabric module code running on
KVM host.  This would be using existing CDB level port emulation code in
target_core_fabric_lib.c, and (mostly) generic fabric control plane in
target_core_fabric_configfs.c..  Beyond the TCM fabric module specifics,
this will require:

*) An asynchronous I/O capable virtio interface for KVM that does not
require direct QEMU block-layer interaction

*) A multi-fabric WWN naming capable TCM fabric module using native
virtio-scsi connections to individual KVM guests for kernel-level
passthrough into TCM backend storage.  (eg: similar to TCM_Loop, but w/o
the Linux/SCSI LLD frontend, and full explict Initiator port NodeACL +
MappedLUN control abstraction)

*) Guest OS specific para-virtualized SCSI driver packages (similar to
virtio-blk) performing SCSI I/O handoff into passthrough interface via
SCSI I_T nexues, with proper T10 WWN Port and LUN naming provided from
underlying TCM v4.x infrastructure  (eg: full compat w/ existing SPC-3
capable cluster clients)

--nab

Re: [LSF/MM TOPIC] Target fabric module for SCSI para-virtualization

[Hannes Reinecke]

On 02/07/2011 08:57 AM, Nicholas A. Bellinger wrote:
> Greetings all,
> 
> [Topic]
> 
> A hybrid target fabric module for QEMU/KVM SCSI para-virtualization
> 
> [Abstract]
> 
> Currently we use the TCM_Loop v4 fabric module with mainline .38 target
> code to present SCSI LUNs with high level multi-fabric (iSCSI, FC, SAS)
> SPC-3 port WWN emulation into KVM guests using two different forms of
> QEMU HBA hardware emulation.  However both of these currently require
> interaction with the QEMU block layer and extra overhead of OS
> independent code running in user-space connected to our existing
> SG_IO/BSG interfaces.  This also has the limitiation that we currently
> cannot perform explict SCSI Initiator Port access management for LUNs
> also present via TCM_Loop SCSI LLD driver on the KVM host.
> 
> Recently there has been an off-list discussion between myself and
> members of the QEMU/KVM community, and they have expressed an interest
> in seeing a native paravirtualized SCSI passthrough available to QEMU
> guests with direct interaction into TCM fabric module code running on
> KVM host.  This would be using existing CDB level port emulation code in
> target_core_fabric_lib.c, and (mostly) generic fabric control plane in
> target_core_fabric_configfs.c..  Beyond the TCM fabric module specifics,
> this will require:
> 
> *) An asynchronous I/O capable virtio interface for KVM that does not
> require direct QEMU block-layer interaction
> 
> *) A multi-fabric WWN naming capable TCM fabric module using native
> virtio-scsi connections to individual KVM guests for kernel-level
> passthrough into TCM backend storage.  (eg: similar to TCM_Loop, but w/o
> the Linux/SCSI LLD frontend, and full explict Initiator port NodeACL +
> MappedLUN control abstraction)
> 
> *) Guest OS specific para-virtualized SCSI driver packages (similar to
> virtio-blk) performing SCSI I/O handoff into passthrough interface via
> SCSI I_T nexues, with proper T10 WWN Port and LUN naming provided from
> underlying TCM v4.x infrastructure  (eg: full compat w/ existing SPC-3
> capable cluster clients)
> 
YES!
One item less on my long-term to-do list.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [LSF/MM TOPIC] Target fabric module for SCSI para-virtualization

[Boaz Harrosh]

On 02/07/2011 09:57 AM, Nicholas A. Bellinger wrote:
> Greetings all,
> 
> [Topic]
> 
> A hybrid target fabric module for QEMU/KVM SCSI para-virtualization
> 
> [Abstract]
> 
> Currently we use the TCM_Loop v4 fabric module with mainline .38 target
> code to present SCSI LUNs with high level multi-fabric (iSCSI, FC, SAS)
> SPC-3 port WWN emulation into KVM guests using two different forms of
> QEMU HBA hardware emulation.  However both of these currently require
> interaction with the QEMU block layer and extra overhead of OS
> independent code running in user-space connected to our existing
> SG_IO/BSG interfaces.  This also has the limitiation that we currently
> cannot perform explict SCSI Initiator Port access management for LUNs
> also present via TCM_Loop SCSI LLD driver on the KVM host.
> 
> Recently there has been an off-list discussion between myself and
> members of the QEMU/KVM community, and they have expressed an interest
> in seeing a native paravirtualized SCSI passthrough available to QEMU
> guests with direct interaction into TCM fabric module code running on
> KVM host.  This would be using existing CDB level port emulation code in
> target_core_fabric_lib.c, and (mostly) generic fabric control plane in
> target_core_fabric_configfs.c..  Beyond the TCM fabric module specifics,
> this will require:
> 
> *) An asynchronous I/O capable virtio interface for KVM that does not
> require direct QEMU block-layer interaction
> 
> *) A multi-fabric WWN naming capable TCM fabric module using native
> virtio-scsi connections to individual KVM guests for kernel-level
> passthrough into TCM backend storage.  (eg: similar to TCM_Loop, but w/o
> the Linux/SCSI LLD frontend, and full explict Initiator port NodeACL +
> MappedLUN control abstraction)
> 
> *) Guest OS specific para-virtualized SCSI driver packages (similar to
> virtio-blk) performing SCSI I/O handoff into passthrough interface via
> SCSI I_T nexues, with proper T10 WWN Port and LUN naming provided from
> underlying TCM v4.x infrastructure  (eg: full compat w/ existing SPC-3
> capable cluster clients)
> 

I agree with the 2 above points on a better scsi-based Kernel-only
passthrough code in the KVM host. But the 3rd point, could we not used any
one of the few LLDs that already exist? I think there are already 3 of these,
no? including the one Microsoft submitted.

> --nab
> 

Thanks
Boaz

Re: [LSF/MM TOPIC] Target fabric module for SCSI para-virtualization

[Stefan Hajnoczi]

On Mon, Feb 7, 2011 at 10:27 AM, Boaz Harrosh <openosd@gmail.com> wrote:
> On 02/07/2011 09:57 AM, Nicholas A. Bellinger wrote:
>> *) Guest OS specific para-virtualized SCSI driver packages (similar to
>> virtio-blk) performing SCSI I/O handoff into passthrough interface via
>> SCSI I_T nexues, with proper T10 WWN Port and LUN naming provided from
>> underlying TCM v4.x infrastructure  (eg: full compat w/ existing SPC-3
>> capable cluster clients)
>>
>
> I agree with the 2 above points on a better scsi-based Kernel-only
> passthrough code in the KVM host. But the 3rd point, could we not used any
> one of the few LLDs that already exist? I think there are already 3 of these,
> no? including the one Microsoft submitted.

It could be done either way but there are some unique properties of a
virtio-scsi HBA.

A virtio-scsi HBA can be implemented on the host as a vhost-scsi
in-kernel device (like vhost-net today).  That allows it to hand off
requests to the SCSI target without switching to userspace.  It can
interact directly with the TCM fabric module because it is in the
kernel, like TCM_Loop today.

The alternative is to emulate an existing HBA and that would be done
in QEMU userspace, at which point you need an interface to talk to the
SCSI target stack inside the kernel - this already exists today with
QEMU SCSI emulation and scsi-generic/BSG devices but is not the same
as fabric-level support.

Virtio is extensible and this has proven essential for evolving and
optimizing the I/O path.  If we choose an interface whose
specification we cannot modify, then we're stuck with its feature set
and performance limitations.

Guest support is important though, so it's worth looking more at
well-supported SCSI HBAs and see if there is a middle path to take.

Stefan
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html