Re: OFED-4.8, rdma-core, and library paths

[Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 2497 bytes --]

On Tue, Feb 07, 2017 at 11:18:14AM -0700, Jason Gunthorpe wrote:
> On Tue, Feb 07, 2017 at 08:06:42PM +0200, Leon Romanovsky wrote:
> > On Tue, Feb 07, 2017 at 10:27:52AM -0700, Jason Gunthorpe wrote:
> > > On Tue, Feb 07, 2017 at 11:19:59AM -0600, Steve Wise wrote:
> > > > >
> > > > > Hm, possibly yes. ibverbs first checks the new location, if the
> > > > > provider is not there then it will fall back to a naked dlopen which
> > > > > could find providers in the system library path if there was a .driver
> > > > > file for it.
> > > >
> > > > Hmm, so it will load the provider libraries directly specifying the
> > > > full path?  IE 'ldconfig -p' doesn't matter?
> > >
> > > As the first try, yes. That is the usual way to locate
> > > plugins. Typically you don't want the system linker searching plugin
> > > directories.
> >
> > Jason,
> > I have a slightly different question, but it is still in context of
> > plugins and linkers.
> >
> > In v0 of DV, you asked from us to create normal shared library for mlx5,
> > e.g. libmlx5.so. In order to be visible to "gcc -lmlx5"i command, it should be
> > placed in the same directory as libibverbs.so, while plugins should be placed
> > in libibverbs folder.
> >
> > I did it by using symlinks
> > $l /usr/lib64/libibverbs
> > lrwxrwxrwx    1 root root   10 Feb  7 18:09 libmlx5-rdmav2.so -> ../libmlx5.so
> > $l /usr/lib64 | grep mlx5
> > lrwxrwxrwx    1 root root   12 Feb  7 18:09 libmlx5.so -> libmlx5.so.1
> > lrwxrwxrwx    1 root root   17 Feb  7 18:09 libmlx5.so.1 -> libmlx5.so.1.0.13
> >
> > It works, but I don't know if it is right approach from distro/packaging
> > perspective. Is it ok?
>
> I think this is basically OK. The symlink should point to the real
> file (libmlx5.so.1.0.13) though.

Right, to be sure that provider and libibverbs are coming from same version.

>
> Also, make sure that cmake computes the .. properly, something like
>
> execute_process("realpath --relative-to ${VERBS_PROVIDER_DIR} ${CMAKE_INSTALL_LIBDIR}/libmlx5.so.1.0.13" OUTPUT_VARIBALE LINK_PATH)
>
> You'll want to create a new rdma_dv_provider function to handle all of
> this.

I made it (rdma_shared_provider function), but have a very hard time
to properly create ".." symlink, because during the build (in place too)
the output is placed in build/lib in flat structure and symlinks need to
be without "..". But during installation phase, these symlinks should
be changed to ".." and it doesn't work for me in automatic way :(

>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]