[Qemu-devel] [PATCH v7 0/8] Shared Library Module Support

[Qemu-devel] [PATCH v7 0/8] Shared Library Module Support

[Fam Zheng]

This series implements feature of shared object building as described in:

http://wiki.qemu.org/Features/Modules

The main idea behind modules is to isolate dependencies on third party
libraries from qemu executables, such as libglusterfs or librbd, so that the
end users can install core qemu package with fewer dependencies.  And only for
those who want to use particular modules, need they install qemu-foo
sub-package, which in turn requires libbar and libbiz packages.

It's implemented in three steps:

1. The first patches fix current build system to correctly handle nested
   variables and object specific options:

    [01/08] ui/Makefile.objs: delete unnecessary cocoa.o dependency
    [02/08] make.rule: fix $(obj) to a real relative path
    [03/08] rule.mak: allow per object cflags and libs

2. The Makefile changes adds necessary options and rules to build DSO objects:

    [04/08] build-sys: introduce common-obj-m and block-obj-m for DSO

3. The next patch adds framework to load modules from installed directory:

    [05/08] module: implement module loading function

A few more changes are following to complete it:

    [06/08] Makefile: install modules with "make install"
    [07/08] .gitignore: ignore module related files (dll, so, mo)

In the end of series, the block drivers are converted:

    [08/08] block: convert block drivers linked with libs to modules

v7: While the discussion on module path and loading restriction goes on, more
input and revisions are apparently necessary to achieve a proper design.
However I'd like to send this revision for building system fixes spotted from
reviewers, to make it easier for early testing:

    [01] Add Peter's patch for ui/Makefile.objs. (Peter)
    [02] Move nested vars prefixing to immediately after unnest-vars-1 (Paolo)
         Fix order of libcacard/Makefile and unnest-vars in Makefile. (Daniel)
    [03] Fix link command if no libtool. (Peter)
    [04] Squash in "--enable-modules". (Paolo)
         Fix a missing comma in no dso building.
    [05] Don't load modules in *-user. (Peter)
         #ifdef out module loading code if no --enable-modules. (Daniel)

v6: Dropping RFC.

    [01] Move addprefix to unnest-vars.
    [01] Drop unnest-vars in tests/Makefile.
    [03] Move "all: modules" to Makefile.
    [03] Add empty modules: in rules.mak, for clarity.
    [03] .mo is no longer storing object list, just an empty file now. In
    expand-objs, objects are extracted with $foo.mo-objs). another reason for
    this is for makefile.target, objects in .mo file would be prefixed with
    ../, so generated two times in one "make all", which always outdates the
    link target.
    [04] Use CONFIG_MODDIR instead of CONFIG_PREFIX for module searching.
    [06] Added make install.
    [07] New.
    [08] Drop qed changes, added iscsi, ssh, rbd and gluster as modules.

v5: Keep foo.mo-objs idea for module objects.
    Unnest block-obj-m and common-obj-m in Makefile.target.
    Move add-modules to unnest-vars to be reused in Makefile.target.
    Use /dev/null to replace realpath for expand-objs.

v4: Added --enable-modules in the end of series.
    Make nested-vars and obj-base as arguemnts to unnest-vars.
    Take Paolo's idea in comments for v2 and switch back module objects syntax
    to:
        $(obj)/foo.mo : $(addprefix $(obj)/, bar.o biz.o qux.o)

    because this needs less duplication among Makefiles.

Fam Zheng (7):
  make.rule: fix $(obj) to a real relative path
  rule.mak: allow per object cflags and libs
  build-sys: introduce common-obj-m and block-obj-m for DSO
  module: implement module loading function
  Makefile: install modules with "make install"
  .gitignore: ignore module related files (dll, so, mo)
  block: convert block drivers linked with libs to modules

Peter Maydell (1):
  ui/Makefile.objs: delete unnecessary cocoa.o dependency

 .gitignore            |  3 ++
 Makefile              | 28 ++++++++++++++++++-
 Makefile.objs         | 18 ++----------
 Makefile.target       | 21 +++++++++++---
 block.c               |  1 +
 block/Makefile.objs   | 11 +++++++-
 configure             | 74 ++++++++++++++++++++++++++++++++-----------------
 include/qemu/module.h |  9 ++++++
 rules.mak             | 77 ++++++++++++++++++++++++++++++++++++++++++---------
 scripts/create_config | 10 +++++++
 ui/Makefile.objs      |  2 --
 util/module.c         | 55 ++++++++++++++++++++++++++++++++++++
 vl.c                  |  2 ++
 13 files changed, 249 insertions(+), 62 deletions(-)

-- 
1.8.3.1

[Qemu-devel] [PATCH v7 1/8] ui/Makefile.objs: delete unnecessary cocoa.o dependency

[Fam Zheng]

From: Peter Maydell <peter.maydell@linaro.org>

Delete an unnecessary dependency for cocoa.o; we already have
a general rule that tells Make that we can build a .o file
from a .m source using an ObjC compiler, so this specific
rule is unnecessary. Further, it is using the dubious construct
"$(SRC_PATH)/$(obj)" to get at the source directory, which will
break when $(obj) is redefined as part of the preparation for
per-object library support.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 ui/Makefile.objs | 2 --
 1 file changed, 2 deletions(-)

diff --git a/ui/Makefile.objs b/ui/Makefile.objs
index 6ddc0de..f33be47 100644
--- a/ui/Makefile.objs
+++ b/ui/Makefile.objs
@@ -17,6 +17,4 @@ common-obj-$(CONFIG_GTK) += gtk.o x_keymap.o
 
 $(obj)/sdl.o $(obj)/sdl_zoom.o: QEMU_CFLAGS += $(SDL_CFLAGS) 
 
-$(obj)/cocoa.o: $(SRC_PATH)/$(obj)/cocoa.m
-
 $(obj)/gtk.o: QEMU_CFLAGS += $(GTK_CFLAGS) $(VTE_CFLAGS)
-- 
1.8.3.1

[Qemu-devel] [PATCH v7 2/8] make.rule: fix $(obj) to a real relative path

[Fam Zheng]

Makefile.target includes rule.mak and unnested common-obj-y, then prefix
them with '../', this will ignore object specific QEMU_CFLAGS in subdir
Makefile.objs:

    $(obj)/curl.o: QEMU_CFLAGS += $(CURL_CFLAGS)

Because $(obj) here is './block', instead of '../block'. This doesn't
hurt compiling because we basically build all .o from top Makefile,
before entering Makefile.target, but it will affact arriving per-object
libs support.

The starting point of $(obj) is passed in as argument of unnest-vars, as
well as nested variables, so that different Makefiles can pass in a
right value.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 Makefile        | 14 ++++++++++++++
 Makefile.objs   | 16 +---------------
 Makefile.target | 17 +++++++++++++----
 configure       |  1 +
 rules.mak       | 14 +++++++++-----
 5 files changed, 38 insertions(+), 24 deletions(-)

diff --git a/Makefile b/Makefile
index 806946e..a23b95c 100644
--- a/Makefile
+++ b/Makefile
@@ -115,6 +115,16 @@ defconfig:
 
 ifneq ($(wildcard config-host.mak),)
 include $(SRC_PATH)/Makefile.objs
+endif
+
+dummy := $(call unnest-vars,, \
+                stub-obj-y \
+                util-obj-y \
+                qga-obj-y \
+                block-obj-y \
+                common-obj-y)
+
+ifneq ($(wildcard config-host.mak),)
 include $(SRC_PATH)/tests/Makefile
 endif
 ifeq ($(CONFIG_SMARTCARD_NSS),y)
@@ -123,6 +133,10 @@ endif
 
 all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all
 
+vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
+
+vl.o: QEMU_CFLAGS+=$(SDL_CFLAGS)
+
 config-host.h: config-host.h-timestamp
 config-host.h-timestamp: config-host.mak
 qemu-options.def: $(SRC_PATH)/qemu-options.hx
diff --git a/Makefile.objs b/Makefile.objs
index f46a4cd..4f7a364 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -41,7 +41,7 @@ libcacard-y += libcacard/vcardt.o
 # single QEMU executable should support all CPUs and machines.
 
 ifeq ($(CONFIG_SOFTMMU),y)
-common-obj-y = $(block-obj-y) blockdev.o blockdev-nbd.o block/
+common-obj-y = blockdev.o blockdev-nbd.o block/
 common-obj-y += net/
 common-obj-y += readline.o
 common-obj-y += qdev-monitor.o device-hotplug.o
@@ -109,17 +109,3 @@ version-lobj-$(CONFIG_WIN32) += $(BUILD_DIR)/version.lo
 # FIXME: a few definitions from qapi-types.o/qapi-visit.o are needed
 # by libqemuutil.a.  These should be moved to a separate .json schema.
 qga-obj-y = qga/ qapi-types.o qapi-visit.o
-
-vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
-
-vl.o: QEMU_CFLAGS+=$(SDL_CFLAGS)
-
-QEMU_CFLAGS+=$(GLIB_CFLAGS)
-
-nested-vars += \
-	stub-obj-y \
-	util-obj-y \
-	qga-obj-y \
-	block-obj-y \
-	common-obj-y
-dummy := $(call unnest-vars)
diff --git a/Makefile.target b/Makefile.target
index 9a49852..87906ea 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -143,13 +143,22 @@ endif # CONFIG_SOFTMMU
 # Workaround for http://gcc.gnu.org/PR55489, see configure.
 %/translate.o: QEMU_CFLAGS += $(TRANSLATE_OPT_CFLAGS)
 
-nested-vars += obj-y
+dummy := $(call unnest-vars,,obj-y)
 
-# This resolves all nested paths, so it must come last
+# we are making another call to unnest-vars with different vars, protect obj-y,
+# it can be overriden in subdir Makefile.objs
+obj-y-save := $(obj-y)
+
+block-obj-y :=
+common-obj-y :=
 include $(SRC_PATH)/Makefile.objs
+dummy := $(call unnest-vars,..,block-obj-y common-obj-y)
+
+# Now restore obj-y
+obj-y := $(obj-y-save)
+
+all-obj-y = $(obj-y) $(common-obj-y) $(block-obj-y)
 
-all-obj-y = $(obj-y)
-all-obj-y += $(addprefix ../, $(common-obj-y))
 
 ifndef CONFIG_HAIKU
 LIBS+=-lm
diff --git a/configure b/configure
index e989609..cc3cd4d 100755
--- a/configure
+++ b/configure
@@ -2251,6 +2251,7 @@ fi
 if $pkg_config --atleast-version=$glib_req_ver gthread-2.0; then
     glib_cflags=`$pkg_config --cflags gthread-2.0`
     glib_libs=`$pkg_config --libs gthread-2.0`
+    CFLAGS="$glib_cflags $CFLAGS"
     LIBS="$glib_libs $LIBS"
     libs_qga="$glib_libs $libs_qga"
 else
diff --git a/rules.mak b/rules.mak
index 4499745..0c5125d 100644
--- a/rules.mak
+++ b/rules.mak
@@ -103,9 +103,6 @@ clean: clean-timestamp
 
 # magic to descend into other directories
 
-obj := .
-old-nested-dirs :=
-
 define push-var
 $(eval save-$2-$1 = $(value $1))
 $(eval $1 :=)
@@ -119,9 +116,11 @@ endef
 
 define unnest-dir
 $(foreach var,$(nested-vars),$(call push-var,$(var),$1/))
-$(eval obj := $(obj)/$1)
+$(eval obj-parent-$1 := $(obj))
+$(eval obj := $(if $(obj),$(obj)/$1,$1))
 $(eval include $(SRC_PATH)/$1/Makefile.objs)
-$(eval obj := $(patsubst %/$1,%,$(obj)))
+$(eval obj := $(obj-parent-$1))
+$(eval obj-parent-$1 := )
 $(foreach var,$(nested-vars),$(call pop-var,$(var),$1/))
 endef
 
@@ -136,7 +135,12 @@ $(if $(nested-dirs),
 endef
 
 define unnest-vars
+$(eval obj := $1)
+$(eval nested-vars := $2)
+$(eval old-nested-dirs := )
 $(call unnest-vars-1)
+$(if $1,$(foreach v,$(nested-vars),$(eval \
+	$v := $(addprefix $1/,$($v)))))
 $(foreach var,$(nested-vars),$(eval $(var) := $(filter-out %/, $($(var)))))
 $(shell mkdir -p $(sort $(foreach var,$(nested-vars),$(dir $($(var))))))
 $(foreach var,$(nested-vars), $(eval \
-- 
1.8.3.1

[Qemu-devel] [PATCH v7 3/8] rule.mak: allow per object cflags and libs

[Fam Zheng]

Adds extract-libs in LINK to expand any "per object libs", the syntax to define
such a libs options is like:

        foo.o-libs := $(CURL_LIBS)

in block/Makefile.objs.

Similarly,

        foo.o-cflags := $(FOO_CFLAGS)

is also supported.

"foo.o" must be listed a nested var (e.g. common-obj-y) to make the
option variables effective.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 rules.mak | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/rules.mak b/rules.mak
index 0c5125d..355c275 100644
--- a/rules.mak
+++ b/rules.mak
@@ -17,15 +17,17 @@ QEMU_DGFLAGS += -MMD -MP -MT $@ -MF $(*D)/$(*F).d
 # Same as -I$(SRC_PATH) -I., but for the nested source/object directories
 QEMU_INCLUDES += -I$(<D) -I$(@D)
 
+extract-libs = $(strip $(foreach o,$1,$($o-libs)))
+
 %.o: %.c
-	$(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
+	$(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
 %.o: %.rc
 	$(call quiet-command,$(WINDRES) -I. -o $@ $<,"  RC    $(TARGET_DIR)$@")
 
 ifeq ($(LIBTOOL),)
 LINK = $(call quiet-command,$(CC) $(QEMU_CFLAGS) $(CFLAGS) $(LDFLAGS) -o $@ \
        $(sort $(filter %.o, $1)) $(filter-out %.o, $1) $(version-obj-y) \
-       $(LIBS),"  LINK  $(TARGET_DIR)$@")
+       $(call extract-libs,$^) $(LIBS),"  LINK  $(TARGET_DIR)$@")
 else
 LIBTOOL += $(if $(V),,--quiet)
 %.lo: %.c
@@ -41,7 +43,7 @@ LINK = $(call quiet-command,\
        $(sort $(filter %.o, $1)) $(filter-out %.o, $1) \
        $(if $(filter %.lo %.la,$^),$(version-lobj-y),$(version-obj-y)) \
        $(if $(filter %.lo %.la,$^),$(LIBTOOLFLAGS)) \
-       $(LIBS),$(if $(filter %.lo %.la,$^),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
+       $(call extract-libs,$^) $(LIBS),$(if $(filter %.lo %.la,$^),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
 endif
 
 %.asm: %.S
@@ -114,11 +116,22 @@ $(eval $1 = $(value save-$2-$1) $$(subdir-$2-$1))
 $(eval save-$2-$1 :=)
 endef
 
+define fix-obj-vars
+$(foreach v,$($1), \
+	$(if $($v-cflags), \
+		$(eval $2$v-cflags := $($v-cflags)) \
+		$(eval $v-cflags := )) \
+	$(if $($v-libs), \
+		$(eval $2$v-libs := $($v-libs)) \
+		$(eval $v-libs := )))
+endef
+
 define unnest-dir
 $(foreach var,$(nested-vars),$(call push-var,$(var),$1/))
 $(eval obj-parent-$1 := $(obj))
 $(eval obj := $(if $(obj),$(obj)/$1,$1))
 $(eval include $(SRC_PATH)/$1/Makefile.objs)
+$(foreach v,$(nested-vars),$(call fix-obj-vars,$v,$(if $(obj),$(obj)/)))
 $(eval obj := $(obj-parent-$1))
 $(eval obj-parent-$1 := )
 $(foreach var,$(nested-vars),$(call pop-var,$(var),$1/))
-- 
1.8.3.1

[Qemu-devel] [PATCH v7 4/8] build-sys: introduce common-obj-m and block-obj-m for DSO

[Fam Zheng]

Add necessary rules and flags for shared object generation.
$(common-obj-m) will include $(block-obj-m), like $(common-obj-y) does
for $(block-obj-y). The new rules introduced here are:

0) For all %.so compiling:

    QEMU_CFLAGS += -fPIC

1) %.o in $(common-obj-m) is compiled to %.o, then linked to %.so.

2) %.mo in $(common-obj-m) is the placeholder for %.so for pattern
matching in Makefile. It's linked to "-shared" with all its dependencies
(multiple *.o) as input. Which means the list of depended objects must
be specified in each sub-Makefile.objs:

    foo.mo-objs := bar.o baz.o qux.o

in the same style with foo.o-cflags and foo.o-libs. The objects here
will be prefixed with "$(obj)/" if it's a subdirectory Makefile.objs.

Also introduce --enable-modules in configure, the option will enable
support of shared object build. Otherwise objects are static linked to
executables.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 Makefile        | 10 ++++++++--
 Makefile.objs   |  2 ++
 Makefile.target |  6 +++++-
 configure       | 14 ++++++++++++++
 rules.mak       | 50 ++++++++++++++++++++++++++++++++++++++++++--------
 5 files changed, 71 insertions(+), 11 deletions(-)

diff --git a/Makefile b/Makefile
index a23b95c..ef76967 100644
--- a/Makefile
+++ b/Makefile
@@ -122,7 +122,9 @@ dummy := $(call unnest-vars,, \
                 util-obj-y \
                 qga-obj-y \
                 block-obj-y \
-                common-obj-y)
+                block-obj-m \
+                common-obj-y \
+                common-obj-m)
 
 ifneq ($(wildcard config-host.mak),)
 include $(SRC_PATH)/tests/Makefile
@@ -131,7 +133,7 @@ ifeq ($(CONFIG_SMARTCARD_NSS),y)
 include $(SRC_PATH)/libcacard/Makefile
 endif
 
-all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all
+all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all modules
 
 vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
 
@@ -249,6 +251,10 @@ clean:
 	rm -f qemu-options.def
 	find . -name '*.[oda]' -type f -exec rm -f {} +
 	find . -name '*.l[oa]' -type f -exec rm -f {} +
+	find . -name '*.so' -type f -exec rm -f {} +
+	find . -name '*.mo' -type f -exec rm -f {} +
+	find . -name '*.dll' -type f -exec rm -f {} +
+
 	rm -f $(TOOLS) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~
 	rm -Rf .libs
 	rm -f qemu-img-cmds.h
diff --git a/Makefile.objs b/Makefile.objs
index 4f7a364..023166b 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -19,6 +19,8 @@ block-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o
 block-obj-y += qemu-coroutine-sleep.o
 block-obj-y += coroutine-$(CONFIG_COROUTINE_BACKEND).o
 
+block-obj-m = block/
+
 ifeq ($(CONFIG_VIRTIO)$(CONFIG_VIRTFS)$(CONFIG_PCI),yyy)
 # Lots of the fsdev/9pcode is pulled in by vl.c via qemu_fsdev_add.
 # only pull in the actual virtio-9p device if we also enabled virtio.
diff --git a/Makefile.target b/Makefile.target
index 87906ea..7fb9e4d 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -152,7 +152,11 @@ obj-y-save := $(obj-y)
 block-obj-y :=
 common-obj-y :=
 include $(SRC_PATH)/Makefile.objs
-dummy := $(call unnest-vars,..,block-obj-y common-obj-y)
+dummy := $(call unnest-vars,.., \
+               block-obj-y \
+               block-obj-m \
+               common-obj-y \
+               common-obj-m)
 
 # Now restore obj-y
 obj-y := $(obj-y-save)
diff --git a/configure b/configure
index cc3cd4d..5bc7256 100755
--- a/configure
+++ b/configure
@@ -190,6 +190,9 @@ mingw32="no"
 gcov="no"
 gcov_tool="gcov"
 EXESUF=""
+DSOSUF=".so"
+LDFLAGS_SHARED="-shared"
+modules="no"
 prefix="/usr/local"
 mandir="\${prefix}/share/man"
 datadir="\${prefix}/share"
@@ -485,6 +488,7 @@ OpenBSD)
 Darwin)
   bsd="yes"
   darwin="yes"
+  LDFLAGS_SHARED="-bundle"
   if [ "$cpu" = "x86_64" ] ; then
     QEMU_CFLAGS="-arch x86_64 $QEMU_CFLAGS"
     LDFLAGS="-arch x86_64 $LDFLAGS"
@@ -584,6 +588,7 @@ fi
 
 if test "$mingw32" = "yes" ; then
   EXESUF=".exe"
+  DSOSUF=".dll"
   QEMU_CFLAGS="-DWIN32_LEAN_AND_MEAN -DWINVER=0x501 $QEMU_CFLAGS"
   # enable C99/POSIX format strings (needs mingw32-runtime 3.15 or later)
   QEMU_CFLAGS="-D__USE_MINGW_ANSI_STDIO=1 $QEMU_CFLAGS"
@@ -646,6 +651,8 @@ for opt do
   ;;
   --disable-debug-info)
   ;;
+  --enable-modules) modules="yes"
+  ;;
   --cpu=*)
   ;;
   --target-list=*) target_list="$optarg"
@@ -1048,6 +1055,7 @@ echo "  --libdir=PATH            install libraries in PATH"
 echo "  --sysconfdir=PATH        install config in PATH$confsuffix"
 echo "  --localstatedir=PATH     install local state in PATH (set at runtime on win32)"
 echo "  --with-confsuffix=SUFFIX suffix for QEMU data inside datadir and sysconfdir [$confsuffix]"
+echo "  --enable-modules         enable modules support"
 echo "  --enable-debug-tcg       enable TCG debugging"
 echo "  --disable-debug-tcg      disable TCG debugging (default)"
 echo "  --enable-debug-info       enable debugging information (default)"
@@ -3572,6 +3580,7 @@ echo "python            $python"
 if test "$slirp" = "yes" ; then
     echo "smbd              $smbd"
 fi
+echo "module support    $modules"
 echo "host CPU          $cpu"
 echo "host big endian   $bigendian"
 echo "target list       $target_list"
@@ -3689,6 +3698,9 @@ echo "libs_softmmu=$libs_softmmu" >> $config_host_mak
 
 echo "ARCH=$ARCH" >> $config_host_mak
 
+if test "$modules" = "yes"; then
+  echo "CONFIG_MODULES=y" >> $config_host_mak
+fi
 case "$cpu" in
   arm|i386|x86_64|x32|ppc|aarch64)
     # The TCG interpreter currently does not support ld/st optimization.
@@ -4175,6 +4187,8 @@ echo "LIBTOOLFLAGS=$LIBTOOLFLAGS" >> $config_host_mak
 echo "LIBS+=$LIBS" >> $config_host_mak
 echo "LIBS_TOOLS+=$libs_tools" >> $config_host_mak
 echo "EXESUF=$EXESUF" >> $config_host_mak
+echo "DSOSUF=$DSOSUF" >> $config_host_mak
+echo "LDFLAGS_SHARED=$LDFLAGS_SHARED" >> $config_host_mak
 echo "LIBS_QGA+=$libs_qga" >> $config_host_mak
 echo "POD2MAN=$POD2MAN" >> $config_host_mak
 echo "TRANSLATE_OPT_CFLAGS=$TRANSLATE_OPT_CFLAGS" >> $config_host_mak
diff --git a/rules.mak b/rules.mak
index 355c275..48907d3 100644
--- a/rules.mak
+++ b/rules.mak
@@ -18,6 +18,9 @@ QEMU_DGFLAGS += -MMD -MP -MT $@ -MF $(*D)/$(*F).d
 QEMU_INCLUDES += -I$(<D) -I$(@D)
 
 extract-libs = $(strip $(foreach o,$1,$($o-libs)))
+expand-objs = $(strip $(sort $(filter %.o,$1)) \
+                  $(foreach o,$(filter %.mo,$1),$($o-objs)) \
+                  $(filter-out %.o %.mo,$1))
 
 %.o: %.c
 	$(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
@@ -26,8 +29,8 @@ extract-libs = $(strip $(foreach o,$1,$($o-libs)))
 
 ifeq ($(LIBTOOL),)
 LINK = $(call quiet-command,$(CC) $(QEMU_CFLAGS) $(CFLAGS) $(LDFLAGS) -o $@ \
-       $(sort $(filter %.o, $1)) $(filter-out %.o, $1) $(version-obj-y) \
-       $(call extract-libs,$^) $(LIBS),"  LINK  $(TARGET_DIR)$@")
+       $(call expand-objs,$1) $(version-obj-y) \
+       $(call extract-libs,$1) $(LIBS),"  LINK  $(TARGET_DIR)$@")
 else
 LIBTOOL += $(if $(V),,--quiet)
 %.lo: %.c
@@ -38,12 +41,12 @@ LIBTOOL += $(if $(V),,--quiet)
 	$(call quiet-command,$(LIBTOOL) --mode=compile --tag=CC dtrace -o $@ -G -s $<, " lt GEN $(TARGET_DIR)$@")
 
 LINK = $(call quiet-command,\
-       $(if $(filter %.lo %.la,$^),$(LIBTOOL) --mode=link --tag=CC \
+       $(if $(filter %.lo %.la,$1),$(LIBTOOL) --mode=link --tag=CC \
        )$(CC) $(QEMU_CFLAGS) $(CFLAGS) $(LDFLAGS) -o $@ \
-       $(sort $(filter %.o, $1)) $(filter-out %.o, $1) \
-       $(if $(filter %.lo %.la,$^),$(version-lobj-y),$(version-obj-y)) \
-       $(if $(filter %.lo %.la,$^),$(LIBTOOLFLAGS)) \
-       $(call extract-libs,$^) $(LIBS),$(if $(filter %.lo %.la,$^),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
+       $(call expand-objs,$1) \
+       $(if $(filter %.lo %.la,$1),$(version-lobj-y),$(version-obj-y)) \
+       $(if $(filter %.lo %.la,$1),$(LIBTOOLFLAGS)) \
+       $(call extract-libs,$1) $(LIBS),$(if $(filter %.lo %.la,$1),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
 endif
 
 %.asm: %.S
@@ -58,6 +61,17 @@ endif
 %.o: %.dtrace
 	$(call quiet-command,dtrace -o $@ -G -s $<, "  GEN   $(TARGET_DIR)$@")
 
+%$(DSOSUF): QEMU_CFLAGS += -fPIC
+%$(DSOSUF): LDFLAGS += $(LDFLAGS_SHARED)
+%$(DSOSUF): %.mo
+	$(call LINK,$^)
+
+.PHONY: modules
+modules:
+
+%.mo:
+	$(call quiet-command,touch $@,"  GEN   $(TARGET_DIR)$@")
+
 %$(EXESUF): %.o
 	$(call LINK,$^)
 
@@ -123,7 +137,10 @@ $(foreach v,$($1), \
 		$(eval $v-cflags := )) \
 	$(if $($v-libs), \
 		$(eval $2$v-libs := $($v-libs)) \
-		$(eval $v-libs := )))
+		$(eval $v-libs := )) \
+	$(if $($v-objs), \
+		$(eval $2$v-objs := $(addprefix $2,$($v-objs))) \
+		$(eval $v-objs := )))
 endef
 
 define unnest-dir
@@ -147,6 +164,14 @@ $(if $(nested-dirs),
   $(call unnest-vars-1))
 endef
 
+define add-modules
+$(foreach o,$(filter %.o,$($1)),$(eval \
+    $(patsubst %.o,%.mo,$o): $o))
+$(foreach o,$(filter %.mo,$($1)),$(eval \
+    $o: $($o-objs)))
+$(eval modules-m += $(patsubst %.o,%.mo,$($1)))
+endef
+
 define unnest-vars
 $(eval obj := $1)
 $(eval nested-vars := $2)
@@ -158,4 +183,13 @@ $(foreach var,$(nested-vars),$(eval $(var) := $(filter-out %/, $($(var)))))
 $(shell mkdir -p $(sort $(foreach var,$(nested-vars),$(dir $($(var))))))
 $(foreach var,$(nested-vars), $(eval \
   -include $(addsuffix *.d, $(sort $(dir $($(var)))))))
+$(foreach v,$(filter %-m,$(nested-vars)), \
+    $(call add-modules,$v))
+
+$(if $(CONFIG_MODULES), \
+    $(eval modules: $(patsubst %.mo,%$(DSOSUF),$(modules-m))), \
+    $(foreach v,$(filter %-m,$(nested-vars)), \
+        $(eval $(patsubst %-m,%-y,$v) += $($v)) \
+        $(eval $v := )))
+
 endef
-- 
1.8.3.1

[Qemu-devel] [PATCH v7 5/8] module: implement module loading function

[Fam Zheng]

Added three types of modules:

    typedef enum {
        MODULE_LOAD_BLOCK = 0,
        MODULE_LOAD_UI,
        MODULE_LOAD_NET,
        MODULE_LOAD_MAX,
    } module_load_type;

and their loading function:

    void module_load(module_load_type).

which loads all ".so" files in a subdir under "${PREFIX}/qemu/", e.g.
"/usr/lib/qemu/block". Modules of each type should be loaded before
respective subsystem initialization code.

Requires gmodule-2.0 from glib.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block.c               |  1 +
 configure             | 28 +++++++++++++++++---------
 include/qemu/module.h |  9 +++++++++
 scripts/create_config | 10 ++++++++++
 util/module.c         | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++
 vl.c                  |  2 ++
 6 files changed, 96 insertions(+), 9 deletions(-)

diff --git a/block.c b/block.c
index 26639e8..16ceaaf 100644
--- a/block.c
+++ b/block.c
@@ -4008,6 +4008,7 @@ BlockDriverAIOCB *bdrv_aio_discard(BlockDriverState *bs,
 
 void bdrv_init(void)
 {
+    module_load(MODULE_LOAD_BLOCK);
     module_call_init(MODULE_INIT_BLOCK);
 }
 
diff --git a/configure b/configure
index 5bc7256..275b1a0 100755
--- a/configure
+++ b/configure
@@ -199,6 +199,7 @@ datadir="\${prefix}/share"
 qemu_docdir="\${prefix}/share/doc/qemu"
 bindir="\${prefix}/bin"
 libdir="\${prefix}/lib"
+moddir="\${prefix}/lib/qemu"
 libexecdir="\${prefix}/libexec"
 includedir="\${prefix}/include"
 sysconfdir="\${prefix}/etc"
@@ -676,6 +677,8 @@ for opt do
   ;;
   --libdir=*) libdir="$optarg"
   ;;
+  --moddir=*) moddir="$optarg"
+  ;;
   --libexecdir=*) libexecdir="$optarg"
   ;;
   --includedir=*) includedir="$optarg"
@@ -1052,6 +1055,7 @@ echo "  --datadir=PATH           install firmware in PATH$confsuffix"
 echo "  --docdir=PATH            install documentation in PATH$confsuffix"
 echo "  --bindir=PATH            install binaries in PATH"
 echo "  --libdir=PATH            install libraries in PATH"
+echo "  --moddir=PATH            install modules in PATH"
 echo "  --sysconfdir=PATH        install config in PATH$confsuffix"
 echo "  --localstatedir=PATH     install local state in PATH (set at runtime on win32)"
 echo "  --with-confsuffix=SUFFIX suffix for QEMU data inside datadir and sysconfdir [$confsuffix]"
@@ -2256,15 +2260,19 @@ if test "$mingw32" = yes; then
 else
     glib_req_ver=2.12
 fi
-if $pkg_config --atleast-version=$glib_req_ver gthread-2.0; then
-    glib_cflags=`$pkg_config --cflags gthread-2.0`
-    glib_libs=`$pkg_config --libs gthread-2.0`
-    CFLAGS="$glib_cflags $CFLAGS"
-    LIBS="$glib_libs $LIBS"
-    libs_qga="$glib_libs $libs_qga"
-else
-    error_exit "glib-$glib_req_ver required to compile QEMU"
-fi
+
+for i in gthread-2.0 gmodule-2.0; do
+    if $pkg_config --atleast-version=$glib_req_ver $i; then
+        glib_cflags=`$pkg_config --cflags $i`
+        glib_libs=`$pkg_config --libs $i`
+        CFLAGS="$glib_cflags $CFLAGS"
+        LIBS="$glib_libs $LIBS"
+        libs_qga="$glib_libs $libs_qga"
+    else
+        error_exit "glib-$glib_req_ver required to compile QEMU"
+    fi
+done
+
 
 ##########################################
 # pixman support probe
@@ -3557,6 +3565,7 @@ echo "Install prefix    $prefix"
 echo "BIOS directory    `eval echo $qemu_datadir`"
 echo "binary directory  `eval echo $bindir`"
 echo "library directory `eval echo $libdir`"
+echo "module directory  `eval echo $moddir`"
 echo "libexec directory `eval echo $libexecdir`"
 echo "include directory `eval echo $includedir`"
 echo "config directory  `eval echo $sysconfdir`"
@@ -3680,6 +3689,7 @@ echo all: >> $config_host_mak
 echo "prefix=$prefix" >> $config_host_mak
 echo "bindir=$bindir" >> $config_host_mak
 echo "libdir=$libdir" >> $config_host_mak
+echo "moddir=$moddir" >> $config_host_mak
 echo "libexecdir=$libexecdir" >> $config_host_mak
 echo "includedir=$includedir" >> $config_host_mak
 echo "mandir=$mandir" >> $config_host_mak
diff --git a/include/qemu/module.h b/include/qemu/module.h
index c4ccd57..f00bc25 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -37,4 +37,13 @@ void register_module_init(void (*fn)(void), module_init_type type);
 
 void module_call_init(module_init_type type);
 
+typedef enum {
+    MODULE_LOAD_BLOCK = 0,
+    MODULE_LOAD_UI,
+    MODULE_LOAD_NET,
+    MODULE_LOAD_MAX,
+} module_load_type;
+
+void module_load(module_load_type type);
+
 #endif
diff --git a/scripts/create_config b/scripts/create_config
index b1adbf5..50391c2 100755
--- a/scripts/create_config
+++ b/scripts/create_config
@@ -26,6 +26,13 @@ case $line in
     # save for the next definitions
     prefix=${line#*=}
     ;;
+ moddir=*)
+    eval "moddir=\"${line#*=}\""
+    echo "#define CONFIG_MODDIR \"$moddir\""
+    ;;
+ CONFIG_MODULES=*)
+    echo "#define CONFIG_MODULES \"${line#*=}\""
+    ;;
  CONFIG_AUDIO_DRIVERS=*)
     drivers=${line#*=}
     echo "#define CONFIG_AUDIO_DRIVERS \\"
@@ -104,6 +111,9 @@ case $line in
     value=${line#*=}
     echo "#define $name $value"
     ;;
+ DSOSUF=*)
+    echo "#define HOST_DSOSUF \"${line#*=}\""
+    ;;
 esac
 
 done # read
diff --git a/util/module.c b/util/module.c
index 7acc33d..8dd3544 100644
--- a/util/module.c
+++ b/util/module.c
@@ -13,6 +13,8 @@
  * GNU GPL, version 2 or (at your option) any later version.
  */
 
+#include <gmodule.h>
+#include <dirent.h>
 #include "qemu-common.h"
 #include "qemu/queue.h"
 #include "qemu/module.h"
@@ -79,3 +81,56 @@ void module_call_init(module_init_type type)
         e->init();
     }
 }
+
+void module_load(module_load_type type)
+{
+#ifdef CONFIG_MODULES
+    const char *path;
+    const char *dsosuf = HOST_DSOSUF;
+    char *fname;
+    int suf_len = strlen(dsosuf);
+    DIR *dp;
+    struct dirent *ep = NULL;
+    GModule *g_module;
+
+    if (!g_module_supported()) {
+        return;
+    }
+
+    switch (type) {
+    case MODULE_LOAD_BLOCK:
+        path = CONFIG_MODDIR "/block/";
+        break;
+    case MODULE_LOAD_UI:
+        path = CONFIG_MODDIR "/ui/";
+        break;
+    case MODULE_LOAD_NET:
+        path = CONFIG_MODDIR "/net/";
+        break;
+    default:
+        return;
+    }
+
+    dp = opendir(path);
+    if (!dp) {
+        fprintf(stderr, "Failed to open dir %s\n", path);
+        return;
+    }
+    for (ep = readdir(dp); ep; ep = readdir(dp)) {
+        int len = strlen(ep->d_name);
+        if (len > suf_len &&
+                !strcmp(&ep->d_name[len - suf_len], dsosuf)) {
+            fname = g_strdup_printf("%s%s", path, ep->d_name);
+            g_module = g_module_open(fname,
+                                     G_MODULE_BIND_LAZY | G_MODULE_BIND_LOCAL);
+            if (!g_module) {
+                fprintf(stderr, "Failed to open module file %s\n",
+                        g_module_error());
+                g_free(fname);
+                continue;
+            }
+            g_free(fname);
+        }
+    }
+#endif
+}
diff --git a/vl.c b/vl.c
index b4b119a..659e53a 100644
--- a/vl.c
+++ b/vl.c
@@ -2940,6 +2940,8 @@ int main(int argc, char **argv, char **envp)
 #endif
     }
 
+    module_load(MODULE_LOAD_UI);
+    module_load(MODULE_LOAD_NET);
     module_call_init(MODULE_INIT_QOM);
 
     qemu_add_opts(&qemu_drive_opts);
-- 
1.8.3.1

[Qemu-devel] [PATCH v7 6/8] Makefile: install modules with "make install"

[Fam Zheng]

Install all the subdirs for modules under configure option "moddir".

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 Makefile | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/Makefile b/Makefile
index ef76967..00c2a52 100644
--- a/Makefile
+++ b/Makefile
@@ -360,6 +360,12 @@ install-datadir install-localstatedir
 ifneq ($(TOOLS),)
 	$(INSTALL_PROG) $(STRIP_OPT) $(TOOLS) "$(DESTDIR)$(bindir)"
 endif
+ifneq ($(CONFIG_MODULES),)
+	for s in $(patsubst %.mo,%(DSOSUF),$(modules-m)); do \
+		$(INSTALL_DIR) "$(DESTDIR)$(moddir)/$$(dirname $$s)"; \
+		$(INSTALL_PROG) $(STRIP_OPT) $$s "$(DESTDIR)$(moddir)/$$(dirname $$s)"; \
+	done
+endif
 ifneq ($(HELPERS-y),)
 	$(INSTALL_DIR) "$(DESTDIR)$(libexecdir)"
 	$(INSTALL_PROG) $(STRIP_OPT) $(HELPERS-y) "$(DESTDIR)$(libexecdir)"
-- 
1.8.3.1

[Qemu-devel] [PATCH v7 7/8] .gitignore: ignore module related files (dll, so, mo)

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 .gitignore | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/.gitignore b/.gitignore
index d2c5c2f..4d0ac09 100644
--- a/.gitignore
+++ b/.gitignore
@@ -63,6 +63,9 @@ fsdev/virtfs-proxy-helper.pod
 *.cp
 *.dvi
 *.exe
+*.dll
+*.so
+*.mo
 *.fn
 *.ky
 *.log
-- 
1.8.3.1

[Qemu-devel] [PATCH v7 8/8] block: convert block drivers linked with libs to modules

[Fam Zheng]

The converted block drivers are:

    curl
    iscsi
    rbd
    ssh
    glusterfs

no longer adds flags and libs for them to global variables, instead
create config-host.mak variables like FOO_CFLAGS and FOO_LIBS, which is
used as per object cflags and libs.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/Makefile.objs | 11 ++++++++++-
 configure           | 33 +++++++++++++++------------------
 2 files changed, 25 insertions(+), 19 deletions(-)

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 3bb85b5..f98d379 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -23,4 +23,13 @@ common-obj-y += commit.o
 common-obj-y += mirror.o
 common-obj-y += backup.o
 
-$(obj)/curl.o: QEMU_CFLAGS+=$(CURL_CFLAGS)
+iscsi.o-cflags     := $(LIBISCSI_CFLAGS)
+iscsi.o-libs       := $(LIBISCSI_LIBS)
+curl.o-cflags      := $(CURL_CFLAGS)
+curl.o-libs        := $(CURL_LIBS)
+rbd.o-cflags       := $(RBD_CFLAGS)
+rbd.o-libs         := $(RBD_LIBS)
+gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
+gluster.o-libs     := $(GLUSTERFS_LIBS)
+ssh.o-cflags       := $(LIBSSH2_CFLAGS)
+ssh.o-libs         := $(LIBSSH2_LIBS)
diff --git a/configure b/configure
index 275b1a0..2f02b3d 100755
--- a/configure
+++ b/configure
@@ -2221,8 +2221,6 @@ EOF
   curl_libs=`$curlconfig --libs 2>/dev/null`
   if compile_prog "$curl_cflags" "$curl_libs" ; then
     curl=yes
-    libs_tools="$curl_libs $libs_tools"
-    libs_softmmu="$curl_libs $libs_softmmu"
   else
     if test "$curl" = "yes" ; then
       feature_not_found "curl"
@@ -2382,8 +2380,6 @@ EOF
   rbd_libs="-lrbd -lrados"
   if compile_prog "" "$rbd_libs" ; then
     rbd=yes
-    libs_tools="$rbd_libs $libs_tools"
-    libs_softmmu="$rbd_libs $libs_softmmu"
   else
     if test "$rbd" = "yes" ; then
       feature_not_found "rados block device"
@@ -2400,9 +2396,6 @@ if test "$libssh2" != "no" ; then
     libssh2_cflags=`$pkg_config libssh2 --cflags`
     libssh2_libs=`$pkg_config libssh2 --libs`
     libssh2=yes
-    libs_tools="$libssh2_libs $libs_tools"
-    libs_softmmu="$libssh2_libs $libs_softmmu"
-    QEMU_CFLAGS="$QEMU_CFLAGS $libssh2_cflags"
   else
     if test "$libssh2" = "yes" ; then
       error_exit "libssh2 >= $min_libssh2_version required for --enable-libssh2"
@@ -2618,9 +2611,6 @@ if test "$glusterfs" != "no" ; then
     glusterfs="yes"
     glusterfs_cflags=`$pkg_config --cflags glusterfs-api`
     glusterfs_libs=`$pkg_config --libs glusterfs-api`
-    CFLAGS="$CFLAGS $glusterfs_cflags"
-    libs_tools="$glusterfs_libs $libs_tools"
-    libs_softmmu="$glusterfs_libs $libs_softmmu"
     if $pkg_config --atleast-version=5 glusterfs-api; then
       glusterfs_discard="yes"
     fi
@@ -2988,11 +2978,9 @@ EOF
     libiscsi="yes"
     libiscsi_cflags=$($pkg_config --cflags libiscsi)
     libiscsi_libs=$($pkg_config --libs libiscsi)
-    CFLAGS="$CFLAGS $libiscsi_cflags"
-    LIBS="$LIBS $libiscsi_libs"
   elif compile_prog "" "-liscsi" ; then
     libiscsi="yes"
-    LIBS="$LIBS -liscsi"
+    libiscsi_libs="-liscsi"
   else
     if test "$libiscsi" = "yes" ; then
       feature_not_found "libiscsi"
@@ -3907,8 +3895,9 @@ if test "$bswap_h" = "yes" ; then
   echo "CONFIG_MACHINE_BSWAP_H=y" >> $config_host_mak
 fi
 if test "$curl" = "yes" ; then
-  echo "CONFIG_CURL=y" >> $config_host_mak
+  echo "CONFIG_CURL=m" >> $config_host_mak
   echo "CURL_CFLAGS=$curl_cflags" >> $config_host_mak
+  echo "CURL_LIBS=$curl_libs" >> $config_host_mak
 fi
 if test "$brlapi" = "yes" ; then
   echo "CONFIG_BRLAPI=y" >> $config_host_mak
@@ -3997,7 +3986,9 @@ if test "$glx" = "yes" ; then
 fi
 
 if test "$libiscsi" = "yes" ; then
-  echo "CONFIG_LIBISCSI=y" >> $config_host_mak
+  echo "CONFIG_LIBISCSI=m" >> $config_host_mak
+  echo "LIBISCSI_CFLAGS=$libiscsi_cflags" >> $config_host_mak
+  echo "LIBISCSI_LIBS=$libiscsi_libs" >> $config_host_mak
 fi
 
 if test "$seccomp" = "yes"; then
@@ -4018,7 +4009,9 @@ if test "$qom_cast_debug" = "yes" ; then
   echo "CONFIG_QOM_CAST_DEBUG=y" >> $config_host_mak
 fi
 if test "$rbd" = "yes" ; then
-  echo "CONFIG_RBD=y" >> $config_host_mak
+  echo "CONFIG_RBD=m" >> $config_host_mak
+  echo "RBD_CFLAGS=$rbd_cflags" >> $config_host_mak
+  echo "RBD_LIBS=$rbd_libs" >> $config_host_mak
 fi
 
 echo "CONFIG_COROUTINE_BACKEND=$coroutine" >> $config_host_mak
@@ -4056,7 +4049,9 @@ if test "$getauxval" = "yes" ; then
 fi
 
 if test "$glusterfs" = "yes" ; then
-  echo "CONFIG_GLUSTERFS=y" >> $config_host_mak
+  echo "CONFIG_GLUSTERFS=m" >> $config_host_mak
+  echo "GLUSTERFS_CFLAGS=$glusterfs_cflags" >> $config_host_mak
+  echo "GLUSTERFS_LIBS=$glusterfs_libs" >> $config_host_mak
 fi
 
 if test "$glusterfs_discard" = "yes" ; then
@@ -4064,7 +4059,9 @@ if test "$glusterfs_discard" = "yes" ; then
 fi
 
 if test "$libssh2" = "yes" ; then
-  echo "CONFIG_LIBSSH2=y" >> $config_host_mak
+  echo "CONFIG_LIBSSH2=m" >> $config_host_mak
+  echo "LIBSSH2_CFLAGS=$libssh2_cflags" >> $config_host_mak
+  echo "LIBSSH2_LIBS=$libssh2_libs" >> $config_host_mak
 fi
 
 if test "$virtio_blk_data_plane" = "yes" ; then
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH v7 6/8] Makefile: install modules with "make install"

[Daniel P. Berrange]

On Thu, Sep 12, 2013 at 03:04:57PM +0800, Fam Zheng wrote:
> Install all the subdirs for modules under configure option "moddir".
> 
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  Makefile | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/Makefile b/Makefile
> index ef76967..00c2a52 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -360,6 +360,12 @@ install-datadir install-localstatedir
>  ifneq ($(TOOLS),)
>  	$(INSTALL_PROG) $(STRIP_OPT) $(TOOLS) "$(DESTDIR)$(bindir)"
>  endif
> +ifneq ($(CONFIG_MODULES),)
> +	for s in $(patsubst %.mo,%(DSOSUF),$(modules-m)); do \

Typo, causing install failure:

for s in block/iscsi(DSOSUF) block/curl(DSOSUF) block/rbd(DSOSUF) block/gluster(DSOSUF) block/ssh(DSOSUF); do \
	install -d -m 0755 "/home/berrange/usr/qemu-git-mod/lib/qemu/$(dirname $s)"; \
	libtool --quiet --mode=install install -c -m 0755  $s "/home/berrange/usr/qemu-git-mod/lib/qemu/$(dirname $s)"; \
done
/bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `for s in block/iscsi(DSOSUF) block/curl(DSOSUF) block/rbd(DSOSUF) block/gluster(DSOSUF) block/ssh(DSOSUF); do \'
make: *** [install] Error 1


Just  s/%(DSOSUF)/%$(DSOSUF)/ I believe



Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [PATCH v7 5/8] module: implement module loading function

[Daniel P. Berrange]

On Thu, Sep 12, 2013 at 03:04:56PM +0800, Fam Zheng wrote:
> Added three types of modules:
> 
>     typedef enum {
>         MODULE_LOAD_BLOCK = 0,
>         MODULE_LOAD_UI,
>         MODULE_LOAD_NET,
>         MODULE_LOAD_MAX,
>     } module_load_type;
> 
> and their loading function:
> 
>     void module_load(module_load_type).
> 
> which loads all ".so" files in a subdir under "${PREFIX}/qemu/", e.g.
> "/usr/lib/qemu/block". Modules of each type should be loaded before
> respective subsystem initialization code.
> 
> Requires gmodule-2.0 from glib.
> 
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  block.c               |  1 +
>  configure             | 28 +++++++++++++++++---------
>  include/qemu/module.h |  9 +++++++++
>  scripts/create_config | 10 ++++++++++
>  util/module.c         | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  vl.c                  |  2 ++
>  6 files changed, 96 insertions(+), 9 deletions(-)
> 
> diff --git a/block.c b/block.c
> index 26639e8..16ceaaf 100644
> --- a/block.c
> +++ b/block.c
> @@ -4008,6 +4008,7 @@ BlockDriverAIOCB *bdrv_aio_discard(BlockDriverState *bs,
>  
>  void bdrv_init(void)
>  {
> +    module_load(MODULE_LOAD_BLOCK);
>      module_call_init(MODULE_INIT_BLOCK);
>  }
>  
> diff --git a/configure b/configure
> index 5bc7256..275b1a0 100755
> --- a/configure
> +++ b/configure
> @@ -199,6 +199,7 @@ datadir="\${prefix}/share"
>  qemu_docdir="\${prefix}/share/doc/qemu"
>  bindir="\${prefix}/bin"
>  libdir="\${prefix}/lib"
> +moddir="\${prefix}/lib/qemu"
>  libexecdir="\${prefix}/libexec"
>  includedir="\${prefix}/include"
>  sysconfdir="\${prefix}/etc"
> @@ -676,6 +677,8 @@ for opt do
>    ;;
>    --libdir=*) libdir="$optarg"
>    ;;
> +  --moddir=*) moddir="$optarg"
> +  ;;
>    --libexecdir=*) libexecdir="$optarg"
>    ;;
>    --includedir=*) includedir="$optarg"
> @@ -1052,6 +1055,7 @@ echo "  --datadir=PATH           install firmware in PATH$confsuffix"
>  echo "  --docdir=PATH            install documentation in PATH$confsuffix"
>  echo "  --bindir=PATH            install binaries in PATH"
>  echo "  --libdir=PATH            install libraries in PATH"
> +echo "  --moddir=PATH            install modules in PATH"
>  echo "  --sysconfdir=PATH        install config in PATH$confsuffix"
>  echo "  --localstatedir=PATH     install local state in PATH (set at runtime on win32)"
>  echo "  --with-confsuffix=SUFFIX suffix for QEMU data inside datadir and sysconfdir [$confsuffix]"
> @@ -2256,15 +2260,19 @@ if test "$mingw32" = yes; then
>  else
>      glib_req_ver=2.12
>  fi
> -if $pkg_config --atleast-version=$glib_req_ver gthread-2.0; then
> -    glib_cflags=`$pkg_config --cflags gthread-2.0`
> -    glib_libs=`$pkg_config --libs gthread-2.0`
> -    CFLAGS="$glib_cflags $CFLAGS"
> -    LIBS="$glib_libs $LIBS"
> -    libs_qga="$glib_libs $libs_qga"
> -else
> -    error_exit "glib-$glib_req_ver required to compile QEMU"
> -fi
> +
> +for i in gthread-2.0 gmodule-2.0; do
> +    if $pkg_config --atleast-version=$glib_req_ver $i; then
> +        glib_cflags=`$pkg_config --cflags $i`
> +        glib_libs=`$pkg_config --libs $i`
> +        CFLAGS="$glib_cflags $CFLAGS"
> +        LIBS="$glib_libs $LIBS"
> +        libs_qga="$glib_libs $libs_qga"
> +    else
> +        error_exit "glib-$glib_req_ver required to compile QEMU"
> +    fi
> +done
> +
>  
>  ##########################################
>  # pixman support probe
> @@ -3557,6 +3565,7 @@ echo "Install prefix    $prefix"
>  echo "BIOS directory    `eval echo $qemu_datadir`"
>  echo "binary directory  `eval echo $bindir`"
>  echo "library directory `eval echo $libdir`"
> +echo "module directory  `eval echo $moddir`"
>  echo "libexec directory `eval echo $libexecdir`"
>  echo "include directory `eval echo $includedir`"
>  echo "config directory  `eval echo $sysconfdir`"
> @@ -3680,6 +3689,7 @@ echo all: >> $config_host_mak
>  echo "prefix=$prefix" >> $config_host_mak
>  echo "bindir=$bindir" >> $config_host_mak
>  echo "libdir=$libdir" >> $config_host_mak
> +echo "moddir=$moddir" >> $config_host_mak
>  echo "libexecdir=$libexecdir" >> $config_host_mak
>  echo "includedir=$includedir" >> $config_host_mak
>  echo "mandir=$mandir" >> $config_host_mak
> diff --git a/include/qemu/module.h b/include/qemu/module.h
> index c4ccd57..f00bc25 100644
> --- a/include/qemu/module.h
> +++ b/include/qemu/module.h
> @@ -37,4 +37,13 @@ void register_module_init(void (*fn)(void), module_init_type type);
>  
>  void module_call_init(module_init_type type);
>  
> +typedef enum {
> +    MODULE_LOAD_BLOCK = 0,
> +    MODULE_LOAD_UI,
> +    MODULE_LOAD_NET,
> +    MODULE_LOAD_MAX,
> +} module_load_type;
> +
> +void module_load(module_load_type type);
> +
>  #endif
> diff --git a/scripts/create_config b/scripts/create_config
> index b1adbf5..50391c2 100755
> --- a/scripts/create_config
> +++ b/scripts/create_config
> @@ -26,6 +26,13 @@ case $line in
>      # save for the next definitions
>      prefix=${line#*=}
>      ;;
> + moddir=*)
> +    eval "moddir=\"${line#*=}\""
> +    echo "#define CONFIG_MODDIR \"$moddir\""
> +    ;;
> + CONFIG_MODULES=*)
> +    echo "#define CONFIG_MODULES \"${line#*=}\""
> +    ;;
>   CONFIG_AUDIO_DRIVERS=*)
>      drivers=${line#*=}
>      echo "#define CONFIG_AUDIO_DRIVERS \\"
> @@ -104,6 +111,9 @@ case $line in
>      value=${line#*=}
>      echo "#define $name $value"
>      ;;
> + DSOSUF=*)
> +    echo "#define HOST_DSOSUF \"${line#*=}\""
> +    ;;
>  esac
>  
>  done # read
> diff --git a/util/module.c b/util/module.c
> index 7acc33d..8dd3544 100644
> --- a/util/module.c
> +++ b/util/module.c
> @@ -13,6 +13,8 @@
>   * GNU GPL, version 2 or (at your option) any later version.
>   */
>  
> +#include <gmodule.h>
> +#include <dirent.h>
>  #include "qemu-common.h"
>  #include "qemu/queue.h"
>  #include "qemu/module.h"
> @@ -79,3 +81,56 @@ void module_call_init(module_init_type type)
>          e->init();
>      }
>  }
> +
> +void module_load(module_load_type type)
> +{
> +#ifdef CONFIG_MODULES
> +    const char *path;
> +    const char *dsosuf = HOST_DSOSUF;
> +    char *fname;
> +    int suf_len = strlen(dsosuf);
> +    DIR *dp;
> +    struct dirent *ep = NULL;
> +    GModule *g_module;
> +
> +    if (!g_module_supported()) {
> +        return;
> +    }
> +
> +    switch (type) {
> +    case MODULE_LOAD_BLOCK:
> +        path = CONFIG_MODDIR "/block/";
> +        break;
> +    case MODULE_LOAD_UI:
> +        path = CONFIG_MODDIR "/ui/";
> +        break;
> +    case MODULE_LOAD_NET:
> +        path = CONFIG_MODDIR "/net/";
> +        break;
> +    default:
> +        return;
> +    }
> +
> +    dp = opendir(path);
> +    if (!dp) {
> +        fprintf(stderr, "Failed to open dir %s\n", path);
> +        return;
> +    }
> +    for (ep = readdir(dp); ep; ep = readdir(dp)) {

Per previous posting, I believe you should not be blindly loading
all modules in the directory using readdir(). Should whitelist
load only individual module files that are part of QEMU, to add
a block against 3rd party out of tree modules.

In addition this still seems to lack any kind of symbol hash
addition, which should be a blocker for merging this series.


Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [PATCH v7 6/8] Makefile: install modules with "make install"

[Fam Zheng]

On Thu, 09/12 15:06, Daniel P. Berrange wrote:
> On Thu, Sep 12, 2013 at 03:04:57PM +0800, Fam Zheng wrote:
> > Install all the subdirs for modules under configure option "moddir".
> > 
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> > ---
> >  Makefile | 6 ++++++
> >  1 file changed, 6 insertions(+)
> > 
> > diff --git a/Makefile b/Makefile
> > index ef76967..00c2a52 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -360,6 +360,12 @@ install-datadir install-localstatedir
> >  ifneq ($(TOOLS),)
> >  	$(INSTALL_PROG) $(STRIP_OPT) $(TOOLS) "$(DESTDIR)$(bindir)"
> >  endif
> > +ifneq ($(CONFIG_MODULES),)
> > +	for s in $(patsubst %.mo,%(DSOSUF),$(modules-m)); do \
> 
> Typo, causing install failure:
> 
> for s in block/iscsi(DSOSUF) block/curl(DSOSUF) block/rbd(DSOSUF) block/gluster(DSOSUF) block/ssh(DSOSUF); do \
> 	install -d -m 0755 "/home/berrange/usr/qemu-git-mod/lib/qemu/$(dirname $s)"; \
> 	libtool --quiet --mode=install install -c -m 0755  $s "/home/berrange/usr/qemu-git-mod/lib/qemu/$(dirname $s)"; \
> done
> /bin/sh: -c: line 0: syntax error near unexpected token `('
> /bin/sh: -c: line 0: `for s in block/iscsi(DSOSUF) block/curl(DSOSUF) block/rbd(DSOSUF) block/gluster(DSOSUF) block/ssh(DSOSUF); do \'
> make: *** [install] Error 1
> 
> 
> Just  s/%(DSOSUF)/%$(DSOSUF)/ I believe
> 
Yes, thanks for catching.

Fam

Re: [Qemu-devel] [PATCH v7 5/8] module: implement module loading function

[Fam Zheng]

On Thu, 09/12 15:08, Daniel P. Berrange wrote:
> On Thu, Sep 12, 2013 at 03:04:56PM +0800, Fam Zheng wrote:
> > Added three types of modules:
> > 
> >     typedef enum {
> >         MODULE_LOAD_BLOCK = 0,
> >         MODULE_LOAD_UI,
> >         MODULE_LOAD_NET,
> >         MODULE_LOAD_MAX,
> >     } module_load_type;
> > 
> > and their loading function:
> > 
> >     void module_load(module_load_type).
> > 
> > which loads all ".so" files in a subdir under "${PREFIX}/qemu/", e.g.
> > "/usr/lib/qemu/block". Modules of each type should be loaded before
> > respective subsystem initialization code.
> > 
> > Requires gmodule-2.0 from glib.
> > 
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> > ---
> >  block.c               |  1 +
> >  configure             | 28 +++++++++++++++++---------
> >  include/qemu/module.h |  9 +++++++++
> >  scripts/create_config | 10 ++++++++++
> >  util/module.c         | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++
> >  vl.c                  |  2 ++
> >  6 files changed, 96 insertions(+), 9 deletions(-)
> > 
> > diff --git a/block.c b/block.c
> > index 26639e8..16ceaaf 100644
> > --- a/block.c
> > +++ b/block.c
> > @@ -4008,6 +4008,7 @@ BlockDriverAIOCB *bdrv_aio_discard(BlockDriverState *bs,
> >  
> >  void bdrv_init(void)
> >  {
> > +    module_load(MODULE_LOAD_BLOCK);
> >      module_call_init(MODULE_INIT_BLOCK);
> >  }
> >  
> > diff --git a/configure b/configure
> > index 5bc7256..275b1a0 100755
> > --- a/configure
> > +++ b/configure
> > @@ -199,6 +199,7 @@ datadir="\${prefix}/share"
> >  qemu_docdir="\${prefix}/share/doc/qemu"
> >  bindir="\${prefix}/bin"
> >  libdir="\${prefix}/lib"
> > +moddir="\${prefix}/lib/qemu"
> >  libexecdir="\${prefix}/libexec"
> >  includedir="\${prefix}/include"
> >  sysconfdir="\${prefix}/etc"
> > @@ -676,6 +677,8 @@ for opt do
> >    ;;
> >    --libdir=*) libdir="$optarg"
> >    ;;
> > +  --moddir=*) moddir="$optarg"
> > +  ;;
> >    --libexecdir=*) libexecdir="$optarg"
> >    ;;
> >    --includedir=*) includedir="$optarg"
> > @@ -1052,6 +1055,7 @@ echo "  --datadir=PATH           install firmware in PATH$confsuffix"
> >  echo "  --docdir=PATH            install documentation in PATH$confsuffix"
> >  echo "  --bindir=PATH            install binaries in PATH"
> >  echo "  --libdir=PATH            install libraries in PATH"
> > +echo "  --moddir=PATH            install modules in PATH"
> >  echo "  --sysconfdir=PATH        install config in PATH$confsuffix"
> >  echo "  --localstatedir=PATH     install local state in PATH (set at runtime on win32)"
> >  echo "  --with-confsuffix=SUFFIX suffix for QEMU data inside datadir and sysconfdir [$confsuffix]"
> > @@ -2256,15 +2260,19 @@ if test "$mingw32" = yes; then
> >  else
> >      glib_req_ver=2.12
> >  fi
> > -if $pkg_config --atleast-version=$glib_req_ver gthread-2.0; then
> > -    glib_cflags=`$pkg_config --cflags gthread-2.0`
> > -    glib_libs=`$pkg_config --libs gthread-2.0`
> > -    CFLAGS="$glib_cflags $CFLAGS"
> > -    LIBS="$glib_libs $LIBS"
> > -    libs_qga="$glib_libs $libs_qga"
> > -else
> > -    error_exit "glib-$glib_req_ver required to compile QEMU"
> > -fi
> > +
> > +for i in gthread-2.0 gmodule-2.0; do
> > +    if $pkg_config --atleast-version=$glib_req_ver $i; then
> > +        glib_cflags=`$pkg_config --cflags $i`
> > +        glib_libs=`$pkg_config --libs $i`
> > +        CFLAGS="$glib_cflags $CFLAGS"
> > +        LIBS="$glib_libs $LIBS"
> > +        libs_qga="$glib_libs $libs_qga"
> > +    else
> > +        error_exit "glib-$glib_req_ver required to compile QEMU"
> > +    fi
> > +done
> > +
> >  
> >  ##########################################
> >  # pixman support probe
> > @@ -3557,6 +3565,7 @@ echo "Install prefix    $prefix"
> >  echo "BIOS directory    `eval echo $qemu_datadir`"
> >  echo "binary directory  `eval echo $bindir`"
> >  echo "library directory `eval echo $libdir`"
> > +echo "module directory  `eval echo $moddir`"
> >  echo "libexec directory `eval echo $libexecdir`"
> >  echo "include directory `eval echo $includedir`"
> >  echo "config directory  `eval echo $sysconfdir`"
> > @@ -3680,6 +3689,7 @@ echo all: >> $config_host_mak
> >  echo "prefix=$prefix" >> $config_host_mak
> >  echo "bindir=$bindir" >> $config_host_mak
> >  echo "libdir=$libdir" >> $config_host_mak
> > +echo "moddir=$moddir" >> $config_host_mak
> >  echo "libexecdir=$libexecdir" >> $config_host_mak
> >  echo "includedir=$includedir" >> $config_host_mak
> >  echo "mandir=$mandir" >> $config_host_mak
> > diff --git a/include/qemu/module.h b/include/qemu/module.h
> > index c4ccd57..f00bc25 100644
> > --- a/include/qemu/module.h
> > +++ b/include/qemu/module.h
> > @@ -37,4 +37,13 @@ void register_module_init(void (*fn)(void), module_init_type type);
> >  
> >  void module_call_init(module_init_type type);
> >  
> > +typedef enum {
> > +    MODULE_LOAD_BLOCK = 0,
> > +    MODULE_LOAD_UI,
> > +    MODULE_LOAD_NET,
> > +    MODULE_LOAD_MAX,
> > +} module_load_type;
> > +
> > +void module_load(module_load_type type);
> > +
> >  #endif
> > diff --git a/scripts/create_config b/scripts/create_config
> > index b1adbf5..50391c2 100755
> > --- a/scripts/create_config
> > +++ b/scripts/create_config
> > @@ -26,6 +26,13 @@ case $line in
> >      # save for the next definitions
> >      prefix=${line#*=}
> >      ;;
> > + moddir=*)
> > +    eval "moddir=\"${line#*=}\""
> > +    echo "#define CONFIG_MODDIR \"$moddir\""
> > +    ;;
> > + CONFIG_MODULES=*)
> > +    echo "#define CONFIG_MODULES \"${line#*=}\""
> > +    ;;
> >   CONFIG_AUDIO_DRIVERS=*)
> >      drivers=${line#*=}
> >      echo "#define CONFIG_AUDIO_DRIVERS \\"
> > @@ -104,6 +111,9 @@ case $line in
> >      value=${line#*=}
> >      echo "#define $name $value"
> >      ;;
> > + DSOSUF=*)
> > +    echo "#define HOST_DSOSUF \"${line#*=}\""
> > +    ;;
> >  esac
> >  
> >  done # read
> > diff --git a/util/module.c b/util/module.c
> > index 7acc33d..8dd3544 100644
> > --- a/util/module.c
> > +++ b/util/module.c
> > @@ -13,6 +13,8 @@
> >   * GNU GPL, version 2 or (at your option) any later version.
> >   */
> >  
> > +#include <gmodule.h>
> > +#include <dirent.h>
> >  #include "qemu-common.h"
> >  #include "qemu/queue.h"
> >  #include "qemu/module.h"
> > @@ -79,3 +81,56 @@ void module_call_init(module_init_type type)
> >          e->init();
> >      }
> >  }
> > +
> > +void module_load(module_load_type type)
> > +{
> > +#ifdef CONFIG_MODULES
> > +    const char *path;
> > +    const char *dsosuf = HOST_DSOSUF;
> > +    char *fname;
> > +    int suf_len = strlen(dsosuf);
> > +    DIR *dp;
> > +    struct dirent *ep = NULL;
> > +    GModule *g_module;
> > +
> > +    if (!g_module_supported()) {
> > +        return;
> > +    }
> > +
> > +    switch (type) {
> > +    case MODULE_LOAD_BLOCK:
> > +        path = CONFIG_MODDIR "/block/";
> > +        break;
> > +    case MODULE_LOAD_UI:
> > +        path = CONFIG_MODDIR "/ui/";
> > +        break;
> > +    case MODULE_LOAD_NET:
> > +        path = CONFIG_MODDIR "/net/";
> > +        break;
> > +    default:
> > +        return;
> > +    }
> > +
> > +    dp = opendir(path);
> > +    if (!dp) {
> > +        fprintf(stderr, "Failed to open dir %s\n", path);
> > +        return;
> > +    }
> > +    for (ep = readdir(dp); ep; ep = readdir(dp)) {
> 
> Per previous posting, I believe you should not be blindly loading
> all modules in the directory using readdir(). Should whitelist
> load only individual module files that are part of QEMU, to add
> a block against 3rd party out of tree modules.
> 
> In addition this still seems to lack any kind of symbol hash
> addition, which should be a blocker for merging this series.
> 
Yes, I'll add module list and symbol hash in next revision.

Thanks.

Fam