From mboxrd@z Thu Jan  1 00:00:00 1970
Content-Type: multipart/mixed; boundary="===============4632651120352942070=="
MIME-Version: 1.0
From: Vladislav Bolkhovitin <vst at vlnb.net>
Subject: Re: [SPDK] SCST Usermode iSCSI Storage Server now handles Intel SPDK
 backing storage
Date: Tue, 05 Sep 2017 20:25:50 -0700
Message-ID: <59AF6ABE.7070806@vlnb.net>
In-Reply-To: 4021f317-59b8-48f6-6317-1307ee381b78@gmail.com
List-ID: <spdk@lists.01.org>
To: spdk@lists.01.org

--===============4632651120352942070==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable

Wonderful!

The only note would be that, as I have already mentioned before, tcmu does =
data copy
between user mode module and kernel, so usage with SCST zero-copy scst_user=
 instead
would be more performance efficient.

Thanks,
Vlad

David Butterfield wrote on 09/05/2017 12:25 AM:
> The SCST Usermode iSCSI Storage Server can now utilize backing storage th=
rough
> the Intel Storage Performance Development Kit (SPDK) API.
> =

> The SCST Usermode Server is a port of about 80 KLOC of the SCST Linux ker=
nel
> software to run entirely in usermode on an unmodified kernel, with virtua=
lly
> no change to the existing SCST source code.
> =

> The diagram on the left side of this PDF page compares the usual kernel-b=
ased
> SCST configuration [blue box] with the configuration adapted for usermode
> [purple box]
> https://github.com/DavidButterfield/SCST-Usermode-Adaptation/blob/usermod=
e/usermode/scstu_tcmur.pdf
> =

> The diagram on the right side of that page illustrates the datapath from
> Initiator to backing storage API -- showing paths through LIO (in-kernel)=
, and
> through Usermode SCST [purple box].  The Usermode SCST server can access
> backing storage through any of these interfaces: preadv(2) and pwritev(2),
> aio(7), or the tcmu-runner backstorage API [red arrow].
> =

> The tcmu-runner backstorage API is a usermode interface point between the
> kernel-based LIO facility and usermode backstore-specific handlers.  The
> tcmu-runner project implements backstore handlers for Ceph/rbd, Gluster/g=
lfs,
> and QEMU/qcow [green box].  I have re-used that same API for Usermode SCS=
T so
> that it can make use of those same backstore handlers [red arrow].
> =

> I have also implemented two additional backstore handlers: a "ram" driver=
 that
> uses mmap(2) either anonymously or with a persistent backing file; and mo=
st
> recently, an interface module to the Intel Storage Performance Developmen=
t Kit
> (SPDK) [red circle -- note that the new SPDK module is a prototype, prese=
ntly
> functional with Usermode SCST, but not yet through the LIO datapath].
> =

> Project is at https://github.com/DavidButterfield/SCST-Usermode-Adaptatio=
n --
> the README there has a few diagrams and a link to a technical paper.  The=
 new
> SPDK backstore handler is in usermode/spdk.c
> =

> The paper starts by describing the port of SCST from the Linux kernel to
> usermode, including diagrams showing how this was done without changing t=
he
> SCST source code.  Next I specify the configuration used for performance
> measurements, followed by plots and analysis interpreting the results.  I
> introduce an experimental "Adaptive Nagle" algorithm to improve performan=
ce of
> small Read operations.  An appendix develops a performance model that att=
empts
> to maintain some intuition in a fairly complicated analysis.
> =

> David Butterfield


--===============4632651120352942070==--