From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============6050045571479265355==" MIME-Version: 1.0 From: Walker, Benjamin Subject: Re: [SPDK] OS memory allocation Date: Thu, 12 Jul 2018 17:46:35 +0000 Message-ID: In-Reply-To: 82C9F782B054C94B9FC04A331649C77AAD4C3FA6@fmsmsx104.amr.corp.intel.com List-ID: To: spdk@lists.01.org --===============6050045571479265355== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Hi Paul On Thu, 2018-07-12 at 17:43 +0000, Luse, Paul E wrote: > Ben, isn't it also the case that the the spdk_malloc routines (I think al= l of > them ) return mem from pre-allocated huge pages. In cases where that's n= ot > required, there's no reason to use that scarce resource (depending on how= you > configured it) so it's better to use malloc/calloc. Right? Absolutely - spdk_malloc and its variants return DMA-safe and also potentia= lly cross-process shared memory. They are defined in include/spdk/env.h because those types of memory allocations are not operations defined in the POSIX A= PI. DMA-safe memory is certainly scarce and often much less flexible in terms of dynamic allocation, so we only allocate DMA-safe memory when we actually ne= ed to perform DMA operations. For "regular" memory allocations, we just use POSIX everywhere. > = > Thx > Paul > = > -----Original Message----- > From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Walker, Benj= amin > Sent: Thursday, July 12, 2018 10:41 AM > To: spdk(a)lists.01.org > Subject: Re: [SPDK] OS memory allocation > = > Hi Leonid, > = > On Thu, 2018-07-12 at 07:10 +0000, Ravich, Leonid wrote: > > Looks like SPDK uses OS dynamic memory (malloc, zalloc ...) I am not = > > understand why , It can use spdk_malloc/ spdk_zmalloc instead and be = > > more "system friendly" . > > = > > If there is some good reason for such behavior I would like to suggest = > > wrapping all the "raw" linux system calls used by SPDK . > > = > > This way SPDK will be more flexible in different systems integrations . > = > = > SPDK very intentionally depends on the POSIX API for a large number of > operations. We've chosen to go this direction because SPDK is intended to= run > as regular a user space application and we want developers to be able to = use > all of the regular tools available in that environment. We're concerned t= hat > not allowing the use of POSIX APIs makes it much more difficult for peopl= e to > jump in and contribute to the code, and also that it would be quite diffi= cult > to enforce in practice. Note that this is also the choice that DPDK has m= ade. > = > We have taken some steps to support more exotic environments though. Firs= t, > all POSIX include files are only included via include/spdk/stdinc.h, and = we > actually have tests to verify that this is the case. If you have a non-PO= SIX > environment, you can swap out that header to include other implementation= s and > map them to the POSIX calls we make. > = > SPDK also requires a number of operations that are outside of the purview= of > POSIX. These are all encapsulated by include/spdk/env.h. The implementati= on of > that header is in lib/env_dpdk, and by default uses DPDK to implement all= of > those operations. However, this portion is designed to be swapped out. Yo= u can > point SPDK at a different implementation of include/spdk/env.h through the > configuration script: > = > ./configure --with-env=3D/path/to/your/lib.a > = > We further have a number of abstractions coming for threading that aren't > quite done yet. SPDK is designed around cooperative multi-tasking and many > locations in the code need to be able to send messages to other threads. > However, SPDK doesn't want to force users into a particular green > thread/cooperative multitasking/futures+promises framework. We're current= ly > designing abstractions for "light weight" threads (without an associated > implementation) that users should be able to map trivially to their parti= cular > framework. > = > I hope that helps clear things up. It's all about making SPDK as flexible= as > possible for as many people as we can, but also making the "simple" case = as > easy to program and use as possible. > = > Thanks, > Ben > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk --===============6050045571479265355==--