Re: [RFCv2 00/15] RFCv2: Consolidated userspace RDMA library repo

[Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 3161 bytes --]

On Sun, Aug 28, 2016 at 12:36:27PM -0600, Jason Gunthorpe wrote:
> On Sun, Aug 28, 2016 at 04:28:04PM +0300, Leon Romanovsky wrote:
> > On Fri, Aug 26, 2016 at 11:17:52PM -0400, Doug Ledford wrote:
> > > On 8/25/2016 4:13 PM, Jason Gunthorpe wrote:
> > > > We've never been able to run the rdma stack from the build trees,
> > >
> > > It's possible...you just have to do every step in the right order ;-)  I
> > > had a spreadsheet that I tracked packages, versions, build states,
> > > install states, and dependencies.  Made things a lot easier.
> > >
> > > But I get your point.
> > >
> >
> > Sorry, but I missed your point. On my development machine, I'm
> > running all relevant RDMA stack directly from their git trees
> > without any excel file. It worked flawlessly without any
> > obstacles, did I do anything wrong?
>
> Maybe, because it is fairly hard to do correctly.
>
> How do you make sure that eg librdmacm or a provider is built against
> the git checkout of verbs, headers, shlib and all?
>
> How do you rebuild the right trees if a change is made to verbs?
>
> How do you have verbs load the provider driver from the git directory build?
>
> When I said 'from the build trees' I mean without a 'make install',
> just run 'build/bin/ibvdev_info' for instance and it guarenteed uses
> the build products exclusively, including all the libraries and
> providers.
>
> Or set LD_LIBRARY_PATH=`pwd`/build/lib and have everything
> use the new build.
>
> This is the goal, and the consolidated repo can do everything except
> the providers right now.

I'm doing that by running number of one liners [1], it builds me
development setup in shared folder, so my base image is continuing
to be clean.

It is not final version, I didn't upload it yet.

Libraries:
	@echo "Build libibverbs"
	@cd $(LIBIBVERBS_SRC)/; ./autogen.sh; ./configure --prefix=$(KVM_SHARED) CFLAGS=-I$(KVM_SHARED)/include LDFLAGS=-L$(KVM_SHARED)/lib CPPFLAGS=-I$(KVM_SHARED)/include; $(MAKE); $(MAKE) install
	@echo "Build libmlx5"
	@cd $(LIBMLX5_SRC)/; ./autogen.sh; ./configure --prefix=$(KVM_SHARED) CFLAGS=-I$(KVM_SHARED)/include LDFLAGS=-L$(KVM_SHARED)/lib CPPFLAGS=-I$(KVM_SHARED)/include; $(MAKE); $(MAKE) install

Kernel headers:
	@echo "Install kernel headers"
	@make -C $(KERNEL_SRC) headers_install INSTALL_HDR_PATH=$(KVM_SHARED)

>
> Doug is also adressing a larger issue with packaging and getting
> everything to build correctly as the distro - he talked about it here:
>
> https://www.spinics.net/lists/linux-rdma/msg37855.html

I read that thread and think that it is related to unoptimized build
process of one distro who for any reason decided do not use repo tool
[2] for managing complex build dependencies between different git trees,
but used excel file and manual download of source tarballs.

We used such tool for our 100+ git trees project and it worked like a charm.
There is worth to invest time and revise internal process of releasing
RDMA stack in that specific distro and no need to invent new beast
(rdma-plumbers).

[1] https://github.com/rleon/dev-scripts/blob/master/Makefile#L89
[2] https://source.android.com/source/developing.html

>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]