[SPDK] Re: SPDK socket abstraction layer

[Sasha Kotchubievsky <sashakot at dev.mellanox.co.il>]

[-- Attachment #1: Type: text/plain, Size: 3259 bytes --]

Hi Tomek,

Are you looking for community feedback regarding VPP implementation of TCP
stack, or about having socket abstraction layer in SPDK?
I think, socket abstraction layer is critical for future integration between
SPDK and user-space stacks. In Mellanox, we're evaluating integration
between VMA (https://github.com/Mellanox/libvma) and SPDK. Although, VMA can
be used as replacement for Kernel implementation of Posix socket interface,
we see great potential in "deep" integration, which definitely needs keep
existing abstraction layer. For example, one of potential improvements can
be zero-copy in RX (receive) flow. I don't see how that can be implemented
on top of Linux Kernel stack. 

Best regards
Sasha

-----Original Message-----
From: Zawadzki, Tomasz <tomasz.zawadzki(a)intel.com> 
Sent: Monday, October 21, 2019 3:01 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] SPDK socket abstraction layer

Hello everyone,

Summary:

With this message I wanted to update SPDK community on state of VPP socket
abstraction as of SPDK 19.07 release.
At this time there does not seem to be a clear efficiency improvements with
VPP. There is no further work planned on SPDK and VPP integration.

Details:

As some of you may remember, SPDK 18.04 release introduced support for
alternative socket types. Along with that release, Vector Packet Processing
(VPP)<https://wiki.fd.io/view/VPP> 18.01 was integrated with SPDK, by
expanding socket abstraction to use VPP Communications Library (VCL). TCP/IP
stack in VPP<https://wiki.fd.io/view/VPP/HostStack> was in early stages back
then and has seen improvements throughout the last year.

To better use VPP capabilities, following fruitful collaboration with VPP
team, in SPDK 19.07, this implementation was changed from VCL to VPP Session
API from VPP 19.04.2.

VPP socket abstraction has met some challenges due to inherent design of
both projects, in particular related to running separate processes and
memory copies.
Seeing improvements from original implementation was encouraging, yet
measuring against posix socket abstraction (taking into consideration entire
system, i.e. both processes), results are comparable. In other words,  at
this time there does not seem to be a clear benefit of either socket
abstraction from standpoint of CPU efficiency or IOPS.

With this message I just wanted to update SPDK community on state of socket
abstraction layers as of SPDK 19.07 release. Each SPDK release always brings
improvements to the abstraction and its implementations, with exciting work
on more efficient use of kernel TCP stack - changes in SPDK 19.10 and SPDK
20.01.

However there is no active involvement at this point around VPP
implementation of socket abstraction in SPDK. Contributions in this area are
always welcome. In case you're interested in implementing further
enhancements of VPP and SPDK integration feel free to reply, or to use one
of the many SPDK community communications
channels<https://spdk.io/community/>.

Thanks,
Tomek

_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org