Re: [Qemu-devel] [PATCH v3 2/2] rbd: link and load librbd dynamically

[Stefan Hajnoczi <stefanha@gmail.com>]

On Wed, Apr 10, 2013 at 2:05 AM, Josh Durgin <josh.durgin@inktank.com> wrote:
> This allows the rbd block driver to detect symbols in the installed
> version of librbd, and enable or disable features appropriately.  This
> obviates the #ifdefs regarding librbd versions.
>
> Loading librbd dynamically also makes the rbd block driver easier to
> install and package, since it removes the dependency on librbd at
> build time.
>
> Add structures containing the necessary function pointer signatures
> and types from librbd, and fill them in the first time the rbd module
> is used. Use glib's g_module interface so we don't preclude future
> portability, and don't have to deal with odd dlopen behavior directly.
>
> Internally, librbd and some libraries it depends on use C++ templates,
> which mean that they each contain a defined weak symbol for their
> definition.  Due to the way the linker resolves duplicate symbols, the
> libraries loaded by librbd end up using the template definitions from
> librbd, creating a circular dependency. This means that once librbd is
> loaded, it won't be unloaded. Changing this behavior might work with a
> Sun ld, but there doesn't seem to be a portable (or even working with
> GNU ld) way to hide these C++ symbols correctly. Instead, never unload
> librbd, and explicitly make it resident.
>
> Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
> ---
>  block/Makefile.objs |    3 +-
>  block/rbd.c         |  292 ++++++++++++++++++++++++++++++++++++---------------
>  block/rbd_types.h   |   95 +++++++++++++++++
>  configure           |   41 +-------
>  4 files changed, 309 insertions(+), 122 deletions(-)
>  create mode 100644 block/rbd_types.h

NACK

I think we're solving the problem at the wrong level.  Writing our own
dynamic linker and adding boilerplate to juggle function pointers
every time we use a library dependency is ugly.

There are two related problems here:

1. Packagers do not want to enable niche dependencies since users will
complain that the package is bloated and pulls in too much stuff.

2. QEMU linked against a newer library version fails to run on hosts
that have an older library.

Problem #1 has several solutions:

1. Let packagers take care of it.  For example, vim is often shipped
in several packages that have various amounts of dependencies
(vim-tiny, vim-gtk, etc).  Packagers create the specialized packages
for specific groups of users to meet their demands without dragging in
too many dependencies.

2. Make QEMU modular - host devices should be shared libraries that
are loaded at runtime.  There should be no stable API so that
development stays flexible and we discourage binary-only modules.
This lets packagers easily ship a qemu-rbd package, for example, that
drops in a .so file that QEMU can load at runtime.

Problem #2 is already solved:

The dynamic linker will refuse to load the program if there are
missing symbols.  It's not possible to mix and match binaries across
environments while downgrading their library dependencies.  With
effort, this could be doable but it's not an interesting use case that
many users care about - they get their binaries from a distro or build
them from source with correct dependencies.

Maybe it's time to move block drivers and other components into modules?

Stefan