Re: [SPDK] OS memory allocation

[Walker, Benjamin <benjamin.walker at intel.com>]

[-- Attachment #1: Type: text/plain, Size: 2573 bytes --]

Hi Leonid,

On Thu, 2018-07-12 at 07:10 +0000, Ravich, Leonid wrote:
> Looks like SPDK uses OS dynamic memory  (malloc, zalloc ...) I am not
> understand why , 
> It can use spdk_malloc/ spdk_zmalloc instead and be more "system friendly" . 
> 
> If there is some good reason for such behavior I would like to suggest
> wrapping all the "raw"  linux system calls used by SPDK .
> 
> This way SPDK will be more flexible in different systems integrations .


SPDK very intentionally depends on the POSIX API for a large number of
operations. We've chosen to go this direction because SPDK is intended to run as
regular a user space application and we want developers to be able to use all of
the regular tools available in that environment. We're concerned that not
allowing the use of POSIX APIs makes it much more difficult for people to jump
in and contribute to the code, and also that it would be quite difficult to
enforce in practice. Note that this is also the choice that DPDK has made.

We have taken some steps to support more exotic environments though. First, all
POSIX include files are only included via include/spdk/stdinc.h, and we actually
have tests to verify that this is the case. If you have a non-POSIX environment,
you can swap out that header to include other implementations and map them to
the POSIX calls we make.

SPDK also requires a number of operations that are outside of the purview of
POSIX. These are all encapsulated by include/spdk/env.h. The implementation of
that header is in lib/env_dpdk, and by default uses DPDK to implement all of
those operations. However, this portion is designed to be swapped out. You can
point SPDK at a different implementation of include/spdk/env.h through the
configuration script:

./configure --with-env=/path/to/your/lib.a

We further have a number of abstractions coming for threading that aren't quite
done yet. SPDK is designed around cooperative multi-tasking and many locations
in the code need to be able to send messages to other threads. However, SPDK
doesn't want to force users into a particular green thread/cooperative
multitasking/futures+promises framework. We're currently designing abstractions
for "light weight" threads (without an associated implementation) that users
should be able to map trivially to their particular framework.

I hope that helps clear things up. It's all about making SPDK as flexible as
possible for as many people as we can, but also making the "simple" case as easy
to program and use as possible.

Thanks,
Ben