From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============1007221845707597189==" MIME-Version: 1.0 From: Walker, Benjamin Subject: [SPDK] Re: SPDK socket abstraction layer Date: Wed, 30 Oct 2019 18:46:34 +0000 Message-ID: In-Reply-To: 8E5DF026-B365-4350-A296-E6B8C405C289@intel.com List-ID: To: spdk@lists.01.org --===============1007221845707597189== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Wed, 2019-10-30 at 17:54 +0000, Harris, James R wrote: > Hi Sasha, > = > Tomek is only talking about the VPP implementation. There are no plans to > remove the socket abstraction layer. If anything, the project needs to l= ook > at extending it in ways as you suggested. To expand on this, there's a lot of activity right now in the SPDK sock abstraction layer to begin to implement asynchronous operations, zero copy operations, etc. For example, see: Asynchronous writev https://review.gerrithub.io/c/spdk/spdk/+/470523 MSG_ZEROCOPY use in the posix implementation https://review.gerrithub.io/c/spdk/spdk/+/471752 A new sock implementation based on io_uring/libaio: https://review.gerrithub.io/c/spdk/spdk/+/471314 And a new sock implementation based on Seastar: https://review.gerrithub.io/c/spdk/spdk/+/466629 So not only is the sock abstraction layer sticking around, but it's getting= a lot of focus going forward. There is a lot of innovation happening in the L= inux kernel around networking at all layers that we need to keep up with. One thing I would like community feedback on is what to do about the curren= t VPP implementation. As we make improvements and additions to the sock abstracti= on, it will necessarily require updates to the VPP implementation. We can of co= urse continue to make those, but does the community see value in maintaining sup= port here? I'd really love to see someone take up the mantle on VPP if they beli= eve there is value that we just haven't been able to unlock yet, but absent that it's just a maintenance burden. Personally speaking, it would be easier for me, as someone trying to evolve= the sock abstraction layer, to drop VPP. That's one less implementation that I = then have to go update and test each time. But I'm very open to opinions and fee= dback here if anyone has something to say. SPDK obviously can't just drop support without strong consensus and a considerable amount of forewarning. Thanks, Ben > = > -Jim > = > = > =EF=BB=BFOn 10/30/19, 10:50 AM, "Sasha Kotchubievsky" > wrote: > = > Hi Tomek, > = > Are you looking for community feedback regarding VPP implementation o= f TCP > stack, or about having socket abstraction layer in SPDK? > I think, socket abstraction layer is critical for future integration > between > SPDK and user-space stacks. In Mellanox, we're evaluating integration > between VMA (https://github.com/Mellanox/libvma) and SPDK. Although, = VMA > can > be used as replacement for Kernel implementation of Posix socket > interface, > we see great potential in "deep" integration, which definitely needs = keep > existing abstraction layer. For example, one of potential improvement= s can > be zero-copy in RX (receive) flow. I don't see how that can be implem= ented > on top of Linux Kernel stack. = > = > Best regards > Sasha > = > -----Original Message----- > From: Zawadzki, Tomasz = > Sent: Monday, October 21, 2019 3:01 PM > To: Storage Performance Development Kit > Subject: [SPDK] SPDK socket abstraction layer > = > Hello everyone, > = > Summary: > = > With this message I wanted to update SPDK community on state of VPP s= ocket > abstraction as of SPDK 19.07 release. > At this time there does not seem to be a clear efficiency improvements > with > VPP. There is no further work planned on SPDK and VPP integration. > = > Details: > = > As some of you may remember, SPDK 18.04 release introduced support for > alternative socket types. Along with that release, Vector Packet > Processing > (VPP) 18.01 was integrated with SPDK, by > expanding socket abstraction to use VPP Communications Library (VCL). > TCP/IP > stack in VPP was in early stag= es > back > then and has seen improvements throughout the last year. > = > To better use VPP capabilities, following fruitful collaboration with= VPP > team, in SPDK 19.07, this implementation was changed from VCL to VPP > Session > API from VPP 19.04.2. > = > VPP socket abstraction has met some challenges due to inherent design= of > both projects, in particular related to running separate processes and > memory copies. > Seeing improvements from original implementation was encouraging, yet > measuring against posix socket abstraction (taking into consideration > entire > system, i.e. both processes), results are comparable. In other words,= at > this time there does not seem to be a clear benefit of either socket > abstraction from standpoint of CPU efficiency or IOPS. > = > With this message I just wanted to update SPDK community on state of > socket > abstraction layers as of SPDK 19.07 release. Each SPDK release always > brings > improvements to the abstraction and its implementations, with exciting > work > on more efficient use of kernel TCP stack - changes in SPDK 19.10 and= SPDK > 20.01. > = > However there is no active involvement at this point around VPP > implementation of socket abstraction in SPDK. Contributions in this a= rea > are > always welcome. In case you're interested in implementing further > enhancements of VPP and SPDK integration feel free to reply, or to us= e one > of the many SPDK community communications > channels;. > = > Thanks, > Tomek > = > _______________________________________________ > SPDK mailing list -- spdk(a)lists.01.org > To unsubscribe send an email to spdk-leave(a)lists.01.org > _______________________________________________ > SPDK mailing list -- spdk(a)lists.01.org > To unsubscribe send an email to spdk-leave(a)lists.01.org > = > = > _______________________________________________ > SPDK mailing list -- spdk(a)lists.01.org > To unsubscribe send an email to spdk-leave(a)lists.01.org --===============1007221845707597189==--