Re: [SPDK] Howto build with an external env, i.e. alternate CONFIG_ENV

[Walker, Benjamin <benjamin.walker at intel.com>]

[-- Attachment #1: Type: text/plain, Size: 5014 bytes --]

On Mon, 2018-07-23 at 13:46 -0500, Lance Hartmann ORACLE wrote:
> > On Jul 20, 2018, at 5:17 PM, Walker, Benjamin <benjamin.walker(a)intel.com> wrote:
> > 
> > On Fri, 2018-07-20 at 16:34 -0500, Lance Hartmann ORACLE wrote:
> > > Another especially notable take-away for me was the realization that the
> > > environment library -- I'll refer to the default env for example here,
> > > libspdk_env_dpdk.a -- consists only of the objects compiled from
> > > SPDK_ROOT_DIR/lib/env_dpdk.   With the ability to specify an alternate
> > > environment, semantically, I had mistakenly assumed that the env library would
> > > contain not only the SPDK implementation of the environment API, but also
> > > everything on which it depended; i.e. for the default environment, the DPDK
> > > objects.   But, that doesn't appear to be so.   Instead, when performing final
> > > linking of each SPDK executable, not only is the SPDK environment library
> > > specified, but so are the necessary DPDK libs on which it depends.   Variables
> > > pulled in from the SPDK's environment makefile, env.mk, are used to specify
> > > those DPDK libs along with additional, special linking flags.
> > > 
> > > We need to update the doc's Porting Guide to reflect these details, and I'd be
> > > happy to volunteer to work on that effort. 
> > 
> > Please do!
> > 
> > > However, before embarking on that task, I'd like to pose the question:   are
> > > there reasons why we couldn't or shouldn't produce the environment library to
> > > consist of both the SPDK env implementation layer and the objects on which it
> > > depends?  It would seem that would make final linking of executables a little
> > > easier and reduce the complexity/effort of someone wanting to develop and use
> > > an alternate environment.
> > 
> > How do you envision this playing with environment libraries that need to link
> > against shared libraries, especially ones provided with the system itself? I'm
> > not an expert in all of the available linker options, so maybe there is some way
> > to embed enough information to properly link/load a shared library directly into
> > the static library.
> > 
> 
> This is an area in which I would greatly appreciate additional discussion, weighing the engineering cost/benefits of such flexibility. I, myself, could for example embrace the idea that for the sake of simplifying final linking that we propose the SPDK env implementation is built as a shared library which could be generated with a dependency on the DPDK (or its replaced equivalent) which also be shared lib(s).   Alternatively, I could also potentially be persuaded that we take an all static approach, though I hasten to add I can appreciate there may be stakeholders who strongly advocate/need shared libs.  I am aware of precedents where a package provided both static and shared versions of their libraries, and sometimes even in separate packages; i.e. a "static" package.   So, for greater flexibility, I could also envision a new SPDK configure option enabling the developer to specify static vs. dynamic, and perhaps with the options for the needed linking flags.  On the latter, I'm aware per configure that it appears the build of the SPDK can/will honor shell environment variables for CFLAGS, CXXFLAGS, LDFLAGS and DESTDIR, but I don't know how challenging the use of those could be to override (or concatenate) those already figured out in the *mk/Makefile files.
> 
> Out of curiosity, I retrieved the available DPDK packages from Ubuntu, and there are quite a few of them, including those with a static and dynamic dpdk library.   I haven't dug through all of the SPDK build options, but my initial pass would suggest it's currently built with only static env (and static DPDK) libs, at least for the default configuration.
> 
> Taking all of this a step further is the expectation that in the future we will provide two very different ways to build with the SPDK.   Today, we're checking out the SPDK repo and relying on the collection of makefiles therein to build the SPDK.   But, moving forward, we hope to produce an spdk-devel package(s) so that SPDK app developers could build against it/them.   Such a build environment there is outside our purview save for our need to provide the proper contents of those devel package(s).
> 
> In summary, I'm seeking, if possible, to simplify the build with respect to the env while keeping in mind the future with -devel package(s).   I'm very much hoping to get more input from others in the SPDK community on this topic, and am happy to volunteer on the implementation and documentation thereof once we can form a broader consensus.
> 
> thanks much,
> 

I've thought about this some and I think we should make alternate environment
library implementations shared libraries. It simplifies the linking and
configuration considerably. If you'd like to take on the changes required to
make this happen, please do so!

Thanks,
Ben