[Qemu-devel] [PATCH v16 0/9] Shared library module support

[Qemu-devel] [PATCH v16 0/9] Shared library module support

[Fam Zheng]

A few changes on Paolo's v15, to fix MacOSX build (in fact fix Linux as well)
and get rid of $RANDOM:

    [05/09] build-sys: introduce common-obj-m and block-obj-m for DSO

            Add " -undefined dynamic_lookup" to Darwin LDFLAGS. Otherwise the
            linker complains about undefined symbols.

    [06/09] module: implement module loading

            Don't use $RANDOM for stamp symbol generation. Hash version,
            pkgversion and configure content instead.

            Add back module-common.c from previous revisions.

            Reformat module_init macro definition to keep scripts/checkpatch.pl
            happy.

            Improve error message when gmodule not present.


Fam Zheng (8):
  rules.mak: fix $(obj) to a real relative path
  rules.mak: allow per object cflags and libs
  block: use per-object cflags and libs
  build-sys: introduce common-obj-m and block-obj-m for DSO
  module: implement module loading
  Makefile: install modules with "make install"
  .gitignore: ignore module related files (dll, so, mo)
  block: convert block drivers linked with libs to modules

Paolo Bonzini (1):
  darwin: do not use -mdynamic-no-pic

 .gitignore            |   3 ++
 Makefile              |  30 +++++++++++++-
 Makefile.objs         |  19 ++-------
 Makefile.target       |  21 ++++++++--
 block/Makefile.objs   |  13 +++++-
 configure             |  79 ++++++++++++++++++++++---------------
 include/qemu/module.h |  18 ++++++++-
 module-common.c       |  10 +++++
 rules.mak             |  80 +++++++++++++++++++++++++++++++------
 scripts/create_config |  14 +++++++
 util/module.c         | 107 +++++++++++++++++++++++++++++++++++++++++++++++++-
 11 files changed, 325 insertions(+), 69 deletions(-)
 create mode 100644 module-common.c

-- 
1.8.5.2

[Qemu-devel] [PATCH v16 1/9] rules.mak: fix $(obj) to a real relative path

[Fam Zheng]

Makefile.target includes rule.mak and unnested common-obj-y, then prefix
them with '../', this will ignore object specific QEMU_CFLAGS in subdir
Makefile.objs:

    $(obj)/curl.o: QEMU_CFLAGS += $(CURL_CFLAGS)

Because $(obj) here is './block', instead of '../block'. This doesn't
hurt compiling because we basically build all .o from top Makefile,
before entering Makefile.target, but it will affact arriving per-object
libs support.

The starting point of $(obj) is passed in as argument of unnest-vars, as
well as nested variables, so that different Makefiles can pass in a
right value.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Makefile        | 14 ++++++++++++++
 Makefile.objs   | 17 +----------------
 Makefile.target | 17 +++++++++++++----
 configure       |  1 +
 rules.mak       | 14 +++++++++-----
 5 files changed, 38 insertions(+), 25 deletions(-)

diff --git a/Makefile b/Makefile
index bdff4e4..423ace5 100644
--- a/Makefile
+++ b/Makefile
@@ -122,6 +122,16 @@ defconfig:
 
 ifneq ($(wildcard config-host.mak),)
 include $(SRC_PATH)/Makefile.objs
+endif
+
+dummy := $(call unnest-vars,, \
+                stub-obj-y \
+                util-obj-y \
+                qga-obj-y \
+                block-obj-y \
+                common-obj-y)
+
+ifneq ($(wildcard config-host.mak),)
 include $(SRC_PATH)/tests/Makefile
 endif
 ifeq ($(CONFIG_SMARTCARD_NSS),y)
@@ -130,6 +140,10 @@ endif
 
 all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all
 
+vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
+
+vl.o: QEMU_CFLAGS+=$(SDL_CFLAGS)
+
 config-host.h: config-host.h-timestamp
 config-host.h-timestamp: config-host.mak
 qemu-options.def: $(SRC_PATH)/qemu-options.hx
diff --git a/Makefile.objs b/Makefile.objs
index 857bb53..489c4b1 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -41,7 +41,7 @@ libcacard-y += libcacard/vcardt.o
 # single QEMU executable should support all CPUs and machines.
 
 ifeq ($(CONFIG_SOFTMMU),y)
-common-obj-y = $(block-obj-y) blockdev.o blockdev-nbd.o block/
+common-obj-y = blockdev.o blockdev-nbd.o block/
 common-obj-y += net/
 common-obj-y += readline.o
 common-obj-y += qdev-monitor.o device-hotplug.o
@@ -112,18 +112,3 @@ version-lobj-$(CONFIG_WIN32) += $(BUILD_DIR)/version.lo
 # by libqemuutil.a.  These should be moved to a separate .json schema.
 qga-obj-y = qga/ qapi-types.o qapi-visit.o
 qga-vss-dll-obj-y = qga/
-
-vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
-
-vl.o: QEMU_CFLAGS+=$(SDL_CFLAGS)
-
-QEMU_CFLAGS+=$(GLIB_CFLAGS)
-
-nested-vars += \
-	stub-obj-y \
-	util-obj-y \
-	qga-obj-y \
-	qga-vss-dll-obj-y \
-	block-obj-y \
-	common-obj-y
-dummy := $(call unnest-vars)
diff --git a/Makefile.target b/Makefile.target
index af6ac7e..9a6e7dd 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -139,13 +139,22 @@ endif # CONFIG_SOFTMMU
 # Workaround for http://gcc.gnu.org/PR55489, see configure.
 %/translate.o: QEMU_CFLAGS += $(TRANSLATE_OPT_CFLAGS)
 
-nested-vars += obj-y
+dummy := $(call unnest-vars,,obj-y)
 
-# This resolves all nested paths, so it must come last
+# we are making another call to unnest-vars with different vars, protect obj-y,
+# it can be overriden in subdir Makefile.objs
+obj-y-save := $(obj-y)
+
+block-obj-y :=
+common-obj-y :=
 include $(SRC_PATH)/Makefile.objs
+dummy := $(call unnest-vars,..,block-obj-y common-obj-y)
+
+# Now restore obj-y
+obj-y := $(obj-y-save)
 
-all-obj-y = $(obj-y)
-all-obj-y += $(addprefix ../, $(common-obj-y))
+all-obj-y = $(obj-y) $(common-obj-y)
+all-obj-$(CONFIG_SOFTMMU) += $(block-obj-y)
 
 ifndef CONFIG_HAIKU
 LIBS+=-lm
diff --git a/configure b/configure
index 3782a6a..4990648 100755
--- a/configure
+++ b/configure
@@ -2345,6 +2345,7 @@ fi
 if $pkg_config --atleast-version=$glib_req_ver gthread-2.0; then
     glib_cflags=`$pkg_config --cflags gthread-2.0`
     glib_libs=`$pkg_config --libs gthread-2.0`
+    CFLAGS="$glib_cflags $CFLAGS"
     LIBS="$glib_libs $LIBS"
     libs_qga="$glib_libs $libs_qga"
 else
diff --git a/rules.mak b/rules.mak
index 49edb9b..7d27602 100644
--- a/rules.mak
+++ b/rules.mak
@@ -138,9 +138,6 @@ clean: clean-timestamp
 
 # magic to descend into other directories
 
-obj := .
-old-nested-dirs :=
-
 define push-var
 $(eval save-$2-$1 = $(value $1))
 $(eval $1 :=)
@@ -154,9 +151,11 @@ endef
 
 define unnest-dir
 $(foreach var,$(nested-vars),$(call push-var,$(var),$1/))
-$(eval obj := $(obj)/$1)
+$(eval obj-parent-$1 := $(obj))
+$(eval obj := $(if $(obj),$(obj)/$1,$1))
 $(eval include $(SRC_PATH)/$1/Makefile.objs)
-$(eval obj := $(patsubst %/$1,%,$(obj)))
+$(eval obj := $(obj-parent-$1))
+$(eval obj-parent-$1 := )
 $(foreach var,$(nested-vars),$(call pop-var,$(var),$1/))
 endef
 
@@ -171,7 +170,12 @@ $(if $(nested-dirs),
 endef
 
 define unnest-vars
+$(eval obj := $1)
+$(eval nested-vars := $2)
+$(eval old-nested-dirs := )
 $(call unnest-vars-1)
+$(if $1,$(foreach v,$(nested-vars),$(eval \
+	$v := $(addprefix $1/,$($v)))))
 $(foreach var,$(nested-vars),$(eval $(var) := $(filter-out %/, $($(var)))))
 $(shell mkdir -p $(sort $(foreach var,$(nested-vars),$(dir $($(var))))))
 $(foreach var,$(nested-vars), $(eval \
-- 
1.8.5.2

[Qemu-devel] [PATCH v16 2/9] rules.mak: allow per object cflags and libs

[Fam Zheng]

Adds extract-libs in LINK to expand any "per object libs", the syntax to define
such a libs options is like:

        foo.o-libs := $(CURL_LIBS)

in block/Makefile.objs.

Similarly,

        foo.o-cflags := $(FOO_CFLAGS)

is also supported.

"foo.o" must be listed in a nested var (e.g. common-obj-y) to make the
option variables effective.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 rules.mak | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/rules.mak b/rules.mak
index 7d27602..9398268 100644
--- a/rules.mak
+++ b/rules.mak
@@ -21,15 +21,17 @@ QEMU_DGFLAGS += -MMD -MP -MT $@ -MF $(*D)/$(*F).d
 # Same as -I$(SRC_PATH) -I., but for the nested source/object directories
 QEMU_INCLUDES += -I$(<D) -I$(@D)
 
+extract-libs = $(strip $(foreach o,$1,$($o-libs)))
+
 %.o: %.c
-	$(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
+	$(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
 %.o: %.rc
 	$(call quiet-command,$(WINDRES) -I. -o $@ $<,"  RC    $(TARGET_DIR)$@")
 
 ifeq ($(LIBTOOL),)
 LINK = $(call quiet-command,$(CC) $(QEMU_CFLAGS) $(CFLAGS) $(LDFLAGS) -o $@ \
        $(sort $(filter %.o, $1)) $(filter-out %.o, $1) $(version-obj-y) \
-       $(LIBS),"  LINK  $(TARGET_DIR)$@")
+       $(call extract-libs,$^) $(LIBS),"  LINK  $(TARGET_DIR)$@")
 else
 LIBTOOL += $(if $(V),,--quiet)
 %.lo: %.c
@@ -45,7 +47,7 @@ LINK = $(call quiet-command,\
        $(sort $(filter %.o, $1)) $(filter-out %.o, $1) \
        $(if $(filter %.lo %.la,$^),$(version-lobj-y),$(version-obj-y)) \
        $(if $(filter %.lo %.la,$^),$(LIBTOOLFLAGS)) \
-       $(LIBS),$(if $(filter %.lo %.la,$^),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
+       $(call extract-libs,$^) $(LIBS),$(if $(filter %.lo %.la,$^),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
 endif
 
 %.asm: %.S
@@ -149,11 +151,22 @@ $(eval $1 = $(value save-$2-$1) $$(subdir-$2-$1))
 $(eval save-$2-$1 :=)
 endef
 
+define fix-obj-vars
+$(foreach v,$($1), \
+	$(if $($v-cflags), \
+		$(eval $2$v-cflags := $($v-cflags)) \
+		$(eval $v-cflags := )) \
+	$(if $($v-libs), \
+		$(eval $2$v-libs := $($v-libs)) \
+		$(eval $v-libs := )))
+endef
+
 define unnest-dir
 $(foreach var,$(nested-vars),$(call push-var,$(var),$1/))
 $(eval obj-parent-$1 := $(obj))
 $(eval obj := $(if $(obj),$(obj)/$1,$1))
 $(eval include $(SRC_PATH)/$1/Makefile.objs)
+$(foreach v,$(nested-vars),$(call fix-obj-vars,$v,$(if $(obj),$(obj)/)))
 $(eval obj := $(obj-parent-$1))
 $(eval obj-parent-$1 := )
 $(foreach var,$(nested-vars),$(call pop-var,$(var),$1/))
-- 
1.8.5.2

Re: [Qemu-devel] [PATCH v16 2/9] rules.mak: allow per object cflags and libs

[Peter Maydell]

On 15 January 2014 08:48, Fam Zheng <famz@redhat.com> wrote:
> Adds extract-libs in LINK to expand any "per object libs", the syntax to define
> such a libs options is like:
>
>         foo.o-libs := $(CURL_LIBS)
>
> in block/Makefile.objs.
>
> Similarly,
>
>         foo.o-cflags := $(FOO_CFLAGS)
>
> is also supported.
>
> "foo.o" must be listed in a nested var (e.g. common-obj-y) to make the
> option variables effective.
>
> Signed-off-by: Fam Zheng <famz@redhat.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  rules.mak | 19 ++++++++++++++++---
>  1 file changed, 16 insertions(+), 3 deletions(-)
>
> diff --git a/rules.mak b/rules.mak
> index 7d27602..9398268 100644
> --- a/rules.mak
> +++ b/rules.mak
> @@ -21,15 +21,17 @@ QEMU_DGFLAGS += -MMD -MP -MT $@ -MF $(*D)/$(*F).d
>  # Same as -I$(SRC_PATH) -I., but for the nested source/object directories
>  QEMU_INCLUDES += -I$(<D) -I$(@D)
>
> +extract-libs = $(strip $(foreach o,$1,$($o-libs)))
> +
>  %.o: %.c
> -       $(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
> +       $(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")

Shouldn't this change also be made in the rules for compiling
.cpp and .m files (which also use CFLAGS and could benefit
from per-object flags) ?

(For instance it would be nice if I could compile the libvixl
C++ sources with a different set of flags.)

thanks
-- PMM

Re: [Qemu-devel] [PATCH v16 2/9] rules.mak: allow per object cflags and libs

[Fam Zheng]

On Wed, 01/15 19:35, Peter Maydell wrote:
> On 15 January 2014 08:48, Fam Zheng <famz@redhat.com> wrote:
> > Adds extract-libs in LINK to expand any "per object libs", the syntax to define
> > such a libs options is like:
> >
> >         foo.o-libs := $(CURL_LIBS)
> >
> > in block/Makefile.objs.
> >
> > Similarly,
> >
> >         foo.o-cflags := $(FOO_CFLAGS)
> >
> > is also supported.
> >
> > "foo.o" must be listed in a nested var (e.g. common-obj-y) to make the
> > option variables effective.
> >
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> > ---
> >  rules.mak | 19 ++++++++++++++++---
> >  1 file changed, 16 insertions(+), 3 deletions(-)
> >
> > diff --git a/rules.mak b/rules.mak
> > index 7d27602..9398268 100644
> > --- a/rules.mak
> > +++ b/rules.mak
> > @@ -21,15 +21,17 @@ QEMU_DGFLAGS += -MMD -MP -MT $@ -MF $(*D)/$(*F).d
> >  # Same as -I$(SRC_PATH) -I., but for the nested source/object directories
> >  QEMU_INCLUDES += -I$(<D) -I$(@D)
> >
> > +extract-libs = $(strip $(foreach o,$1,$($o-libs)))
> > +
> >  %.o: %.c
> > -       $(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
> > +       $(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
> 
> Shouldn't this change also be made in the rules for compiling
> .cpp and .m files (which also use CFLAGS and could benefit
> from per-object flags) ?
> 
> (For instance it would be nice if I could compile the libvixl
> C++ sources with a different set of flags.)
> 

Sounds reasonable. As I've already sent v17 without this change, I'll remember
to add this in later revisions or follow-up patches.

Fam

Re: [Qemu-devel] [PATCH v16 2/9] rules.mak: allow per object cflags and libs

[Peter Maydell]

On 15 January 2014 08:48, Fam Zheng <famz@redhat.com> wrote:.objs.
> Similarly,
>
>         foo.o-cflags := $(FOO_CFLAGS)
>
> is also supported.

I noticed that we already support per-object cflags via:

$(obj)/adlib.o $(obj)/fmopl.o: QEMU_CFLAGS += -DBUILD_Y8950=0

(this example from audio/Makefile.objs). Is your new
method better in some way? Are we going to convert
users of the old-style method?

thanks
-- PMM

Re: [Qemu-devel] [PATCH v16 2/9] rules.mak: allow per object cflags and libs

[Fam Zheng]

On Thu, 01/16 11:04, Peter Maydell wrote:
> On 15 January 2014 08:48, Fam Zheng <famz@redhat.com> wrote:.objs.
> > Similarly,
> >
> >         foo.o-cflags := $(FOO_CFLAGS)
> >
> > is also supported.
> 
> I noticed that we already support per-object cflags via:
> 
> $(obj)/adlib.o $(obj)/fmopl.o: QEMU_CFLAGS += -DBUILD_Y8950=0
> 
> (this example from audio/Makefile.objs). Is your new
> method better in some way? Are we going to convert
> users of the old-style method?
> 

I added this new method as it's cleaner to use, especially when defining
multiple-object module (implemented in this series):

    foo.mo-objs := bar.o biz.o qux.o

, compared to

    $(obj)/foo.mo-objs := $(obj)/bar.o $(obj)/biz.o $(obj)/qux.o

or slightly better

    $(obj)/foo.mo-objs := $(addprefix $(obj), bar.o, biz.o, qux.o)

I've already converted block/curl.o's cflags to this, later in this series. So
yes I think this is a better syntax.

Thanks,
Fam

Re: [Qemu-devel] [PATCH v16 2/9] rules.mak: allow per object cflags and libs

[Paolo Bonzini]

Il 16/01/2014 13:40, Fam Zheng ha scritto:
> , compared to
> 
>     $(obj)/foo.mo-objs := $(obj)/bar.o $(obj)/biz.o $(obj)/qux.o
> 
> or slightly better
> 
>     $(obj)/foo.mo-objs := $(addprefix $(obj), bar.o, biz.o, qux.o)
> 
> I've already converted block/curl.o's cflags to this, later in this series. So
> yes I think this is a better syntax.

Also the previous mechanism is not usable for libraries, so that we have
to add them to libs_softmmu and/or libs_tools manually in the configure
script.  So the new scheme is easier as well as more flexible.

Paolo

[Qemu-devel] [PATCH v16 3/9] block: use per-object cflags and libs

[Fam Zheng]

No longer adds flags and libs for them to global variables, instead
create config-host.mak variables like FOO_CFLAGS and FOO_LIBS, which is
used as per object cflags and libs.

This removes unwanted dependencies from libcacard.

Signed-off-by: Fam Zheng <famz@redhat.com>
[Split from Fam's patch to enable modules. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/Makefile.objs | 13 ++++++++++++-
 configure           | 25 ++++++++++---------------
 2 files changed, 22 insertions(+), 16 deletions(-)

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 4e8c91e..a1db63f 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -23,4 +23,15 @@ common-obj-y += commit.o
 common-obj-y += mirror.o
 common-obj-y += backup.o
 
-$(obj)/curl.o: QEMU_CFLAGS+=$(CURL_CFLAGS)
+iscsi.o-cflags     := $(LIBISCSI_CFLAGS)
+iscsi.o-libs       := $(LIBISCSI_LIBS)
+curl.o-cflags      := $(CURL_CFLAGS)
+curl.o-libs        := $(CURL_LIBS)
+rbd.o-cflags       := $(RBD_CFLAGS)
+rbd.o-libs         := $(RBD_LIBS)
+gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
+gluster.o-libs     := $(GLUSTERFS_LIBS)
+ssh.o-cflags       := $(LIBSSH2_CFLAGS)
+ssh.o-libs         := $(LIBSSH2_LIBS)
+qcow.o-libs        := -lz
+linux-aio.o-libs   := -laio
diff --git a/configure b/configure
index 4990648..9d71867 100755
--- a/configure
+++ b/configure
@@ -2303,8 +2303,6 @@ EOF
   curl_libs=`$curlconfig --libs 2>/dev/null`
   if compile_prog "$curl_cflags" "$curl_libs" ; then
     curl=yes
-    libs_tools="$curl_libs $libs_tools"
-    libs_softmmu="$curl_libs $libs_softmmu"
   else
     if test "$curl" = "yes" ; then
       feature_not_found "curl"
@@ -2460,8 +2458,6 @@ EOF
   rbd_libs="-lrbd -lrados"
   if compile_prog "" "$rbd_libs" ; then
     rbd=yes
-    libs_tools="$rbd_libs $libs_tools"
-    libs_softmmu="$rbd_libs $libs_softmmu"
   else
     if test "$rbd" = "yes" ; then
       feature_not_found "rados block device"
@@ -2478,9 +2474,6 @@ if test "$libssh2" != "no" ; then
     libssh2_cflags=`$pkg_config libssh2 --cflags`
     libssh2_libs=`$pkg_config libssh2 --libs`
     libssh2=yes
-    libs_tools="$libssh2_libs $libs_tools"
-    libs_softmmu="$libssh2_libs $libs_softmmu"
-    QEMU_CFLAGS="$QEMU_CFLAGS $libssh2_cflags"
   else
     if test "$libssh2" = "yes" ; then
       error_exit "libssh2 >= $min_libssh2_version required for --enable-libssh2"
@@ -2526,8 +2519,6 @@ int main(void) { io_setup(0, NULL); io_set_eventfd(NULL, 0); eventfd(0, 0); retu
 EOF
   if compile_prog "" "-laio" ; then
     linux_aio=yes
-    libs_softmmu="$libs_softmmu -laio"
-    libs_tools="$libs_tools -laio"
   else
     if test "$linux_aio" = "yes" ; then
       feature_not_found "linux AIO"
@@ -2696,9 +2687,6 @@ if test "$glusterfs" != "no" ; then
     glusterfs="yes"
     glusterfs_cflags=`$pkg_config --cflags glusterfs-api`
     glusterfs_libs=`$pkg_config --libs glusterfs-api`
-    CFLAGS="$CFLAGS $glusterfs_cflags"
-    libs_tools="$glusterfs_libs $libs_tools"
-    libs_softmmu="$glusterfs_libs $libs_softmmu"
     if $pkg_config --atleast-version=5 glusterfs-api; then
       glusterfs_discard="yes"
     fi
@@ -3066,11 +3054,9 @@ EOF
     libiscsi="yes"
     libiscsi_cflags=$($pkg_config --cflags libiscsi)
     libiscsi_libs=$($pkg_config --libs libiscsi)
-    CFLAGS="$CFLAGS $libiscsi_cflags"
-    LIBS="$LIBS $libiscsi_libs"
   elif compile_prog "" "-liscsi" ; then
     libiscsi="yes"
-    LIBS="$LIBS -liscsi"
+    libiscsi_libs="-liscsi"
   else
     if test "$libiscsi" = "yes" ; then
       feature_not_found "libiscsi"
@@ -4068,6 +4054,7 @@ fi
 if test "$curl" = "yes" ; then
   echo "CONFIG_CURL=y" >> $config_host_mak
   echo "CURL_CFLAGS=$curl_cflags" >> $config_host_mak
+  echo "CURL_LIBS=$curl_libs" >> $config_host_mak
 fi
 if test "$brlapi" = "yes" ; then
   echo "CONFIG_BRLAPI=y" >> $config_host_mak
@@ -4160,6 +4147,8 @@ if test "$libiscsi" = "yes" ; then
   if test "$libiscsi_version" = "1.4.0"; then
     echo "CONFIG_LIBISCSI_1_4=y" >> $config_host_mak
   fi
+  echo "LIBISCSI_CFLAGS=$libiscsi_cflags" >> $config_host_mak
+  echo "LIBISCSI_LIBS=$libiscsi_libs" >> $config_host_mak
 fi
 
 if test "$seccomp" = "yes"; then
@@ -4181,6 +4170,8 @@ if test "$qom_cast_debug" = "yes" ; then
 fi
 if test "$rbd" = "yes" ; then
   echo "CONFIG_RBD=y" >> $config_host_mak
+  echo "RBD_CFLAGS=$rbd_cflags" >> $config_host_mak
+  echo "RBD_LIBS=$rbd_libs" >> $config_host_mak
 fi
 
 echo "CONFIG_COROUTINE_BACKEND=$coroutine" >> $config_host_mak
@@ -4224,6 +4215,8 @@ fi
 
 if test "$glusterfs" = "yes" ; then
   echo "CONFIG_GLUSTERFS=y" >> $config_host_mak
+  echo "GLUSTERFS_CFLAGS=$glusterfs_cflags" >> $config_host_mak
+  echo "GLUSTERFS_LIBS=$glusterfs_libs" >> $config_host_mak
 fi
 
 if test "$glusterfs_discard" = "yes" ; then
@@ -4232,6 +4225,8 @@ fi
 
 if test "$libssh2" = "yes" ; then
   echo "CONFIG_LIBSSH2=y" >> $config_host_mak
+  echo "LIBSSH2_CFLAGS=$libssh2_cflags" >> $config_host_mak
+  echo "LIBSSH2_LIBS=$libssh2_libs" >> $config_host_mak
 fi
 
 if test "$virtio_blk_data_plane" = "yes" ; then
-- 
1.8.5.2

[Qemu-devel] [PATCH v16 4/9] darwin: do not use -mdynamic-no-pic

[Fam Zheng]

From: Paolo Bonzini <pbonzini@redhat.com>

While -mdynamic-no-pic can speed up the code somewhat, it is only used
on the legacy PowerPC Mac OS X, and I am not sure if anyone is still
testing that.  Disabling PIC can cause problems when enabling modules,
so do not do that.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 configure | 2 --
 1 file changed, 2 deletions(-)

diff --git a/configure b/configure
index 9d71867..99434e6 100755
--- a/configure
+++ b/configure
@@ -516,8 +516,6 @@ Darwin)
   if [ "$cpu" = "x86_64" ] ; then
     QEMU_CFLAGS="-arch x86_64 $QEMU_CFLAGS"
     LDFLAGS="-arch x86_64 $LDFLAGS"
-  else
-    QEMU_CFLAGS="-mdynamic-no-pic $QEMU_CFLAGS"
   fi
   cocoa="yes"
   audio_drv_list="coreaudio"
-- 
1.8.5.2

[Qemu-devel] [PATCH v16 5/9] build-sys: introduce common-obj-m and block-obj-m for DSO

[Fam Zheng]

Add necessary rules and flags for shared object generation.
$(common-obj-m) will include $(block-obj-m), like $(common-obj-y) does
for $(block-obj-y). The new rules introduced here are:

0) For all %.so compiling:

    QEMU_CFLAGS += -fPIC

1) %.o in $(common-obj-m) is compiled to %.o, then linked to %.so.

2) %.mo in $(common-obj-m) is the placeholder for %.so for pattern
matching in Makefile. It's linked to "-shared" with all its dependencies
(multiple *.o) as input. Which means the list of depended objects must
be specified in each sub-Makefile.objs:

    foo.mo-objs := bar.o baz.o qux.o

in the same style with foo.o-cflags and foo.o-libs. The objects here
will be prefixed with "$(obj)/" if it's a subdirectory Makefile.objs.

Also introduce --enable-modules in configure, the option will enable
support of shared object build. Otherwise objects are static linked to
executables.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Makefile        |  9 +++++++--
 Makefile.objs   |  2 ++
 Makefile.target |  6 +++++-
 configure       | 14 ++++++++++++++
 rules.mak       | 54 +++++++++++++++++++++++++++++++++++++++++++++---------
 5 files changed, 73 insertions(+), 12 deletions(-)

diff --git a/Makefile b/Makefile
index 423ace5..9de66cb 100644
--- a/Makefile
+++ b/Makefile
@@ -129,7 +129,9 @@ dummy := $(call unnest-vars,, \
                 util-obj-y \
                 qga-obj-y \
                 block-obj-y \
-                common-obj-y)
+                block-obj-m \
+                common-obj-y \
+                common-obj-m)
 
 ifneq ($(wildcard config-host.mak),)
 include $(SRC_PATH)/tests/Makefile
@@ -138,7 +140,7 @@ ifeq ($(CONFIG_SMARTCARD_NSS),y)
 include $(SRC_PATH)/libcacard/Makefile
 endif
 
-all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all
+all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all modules
 
 vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
 
@@ -256,6 +258,9 @@ clean:
 	rm -f qemu-options.def
 	find . -name '*.[oda]' -type f -exec rm -f {} +
 	find . -name '*.l[oa]' -type f -exec rm -f {} +
+	find . -name '*.so' -type f -exec rm -f {} +
+	find . -name '*.mo' -type f -exec rm -f {} +
+	find . -name '*.dll' -type f -exec rm -f {} +
 	rm -f $(filter-out %.tlb,$(TOOLS)) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~
 	rm -f fsdev/*.pod
 	rm -rf .libs */.libs
diff --git a/Makefile.objs b/Makefile.objs
index 489c4b1..56f256f 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -19,6 +19,8 @@ block-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o
 block-obj-y += qemu-coroutine-sleep.o
 block-obj-y += coroutine-$(CONFIG_COROUTINE_BACKEND).o
 
+block-obj-m = block/
+
 ifeq ($(CONFIG_VIRTIO)$(CONFIG_VIRTFS)$(CONFIG_PCI),yyy)
 # Lots of the fsdev/9pcode is pulled in by vl.c via qemu_fsdev_add.
 # only pull in the actual virtio-9p device if we also enabled virtio.
diff --git a/Makefile.target b/Makefile.target
index 9a6e7dd..3945260 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -148,7 +148,11 @@ obj-y-save := $(obj-y)
 block-obj-y :=
 common-obj-y :=
 include $(SRC_PATH)/Makefile.objs
-dummy := $(call unnest-vars,..,block-obj-y common-obj-y)
+dummy := $(call unnest-vars,.., \
+               block-obj-y \
+               block-obj-m \
+               common-obj-y \
+               common-obj-m)
 
 # Now restore obj-y
 obj-y := $(obj-y-save)
diff --git a/configure b/configure
index 99434e6..a6a626f 100755
--- a/configure
+++ b/configure
@@ -205,6 +205,9 @@ mingw32="no"
 gcov="no"
 gcov_tool="gcov"
 EXESUF=""
+DSOSUF=".so"
+LDFLAGS_SHARED="-shared"
+modules="no"
 prefix="/usr/local"
 mandir="\${prefix}/share/man"
 datadir="\${prefix}/share"
@@ -513,6 +516,7 @@ OpenBSD)
 Darwin)
   bsd="yes"
   darwin="yes"
+  LDFLAGS_SHARED="-bundle -undefined dynamic_lookup"
   if [ "$cpu" = "x86_64" ] ; then
     QEMU_CFLAGS="-arch x86_64 $QEMU_CFLAGS"
     LDFLAGS="-arch x86_64 $LDFLAGS"
@@ -606,6 +610,7 @@ fi
 
 if test "$mingw32" = "yes" ; then
   EXESUF=".exe"
+  DSOSUF=".dll"
   QEMU_CFLAGS="-DWIN32_LEAN_AND_MEAN -DWINVER=0x501 $QEMU_CFLAGS"
   # enable C99/POSIX format strings (needs mingw32-runtime 3.15 or later)
   QEMU_CFLAGS="-D__USE_MINGW_ANSI_STDIO=1 $QEMU_CFLAGS"
@@ -672,6 +677,8 @@ for opt do
   ;;
   --disable-debug-info)
   ;;
+  --enable-modules) modules="yes"
+  ;;
   --cpu=*)
   ;;
   --target-list=*) target_list="$optarg"
@@ -1124,6 +1131,7 @@ Advanced options (experts only):
   --sysconfdir=PATH        install config in PATH$confsuffix
   --localstatedir=PATH     install local state in PATH (set at runtime on win32)
   --with-confsuffix=SUFFIX suffix for QEMU data inside datadir and sysconfdir [$confsuffix]
+  --enable-modules         enable modules support
   --enable-debug-tcg       enable TCG debugging
   --disable-debug-tcg      disable TCG debugging (default)
   --enable-debug-info       enable debugging information (default)
@@ -3736,6 +3744,7 @@ echo "python            $python"
 if test "$slirp" = "yes" ; then
     echo "smbd              $smbd"
 fi
+echo "module support    $modules"
 echo "host CPU          $cpu"
 echo "host big endian   $bigendian"
 echo "target list       $target_list"
@@ -3858,6 +3867,9 @@ echo "libs_softmmu=$libs_softmmu" >> $config_host_mak
 
 echo "ARCH=$ARCH" >> $config_host_mak
 
+if test "$modules" = "yes"; then
+  echo "CONFIG_MODULES=y" >> $config_host_mak
+fi
 if test "$debug_tcg" = "yes" ; then
   echo "CONFIG_DEBUG_TCG=y" >> $config_host_mak
 fi
@@ -4353,6 +4365,8 @@ echo "LIBTOOLFLAGS=$LIBTOOLFLAGS" >> $config_host_mak
 echo "LIBS+=$LIBS" >> $config_host_mak
 echo "LIBS_TOOLS+=$libs_tools" >> $config_host_mak
 echo "EXESUF=$EXESUF" >> $config_host_mak
+echo "DSOSUF=$DSOSUF" >> $config_host_mak
+echo "LDFLAGS_SHARED=$LDFLAGS_SHARED" >> $config_host_mak
 echo "LIBS_QGA+=$libs_qga" >> $config_host_mak
 echo "POD2MAN=$POD2MAN" >> $config_host_mak
 echo "TRANSLATE_OPT_CFLAGS=$TRANSLATE_OPT_CFLAGS" >> $config_host_mak
diff --git a/rules.mak b/rules.mak
index 9398268..fb3482b 100644
--- a/rules.mak
+++ b/rules.mak
@@ -21,7 +21,11 @@ QEMU_DGFLAGS += -MMD -MP -MT $@ -MF $(*D)/$(*F).d
 # Same as -I$(SRC_PATH) -I., but for the nested source/object directories
 QEMU_INCLUDES += -I$(<D) -I$(@D)
 
-extract-libs = $(strip $(foreach o,$1,$($o-libs)))
+extract-libs = $(strip $(sort $(foreach o,$1,$($o-libs)) \
+                  $(foreach o,$(call expand-objs,$1),$($o-libs))))
+expand-objs = $(strip $(sort $(filter %.o,$1)) \
+                  $(foreach o,$(filter %.mo,$1),$($o-objs)) \
+                  $(filter-out %.o %.mo,$1))
 
 %.o: %.c
 	$(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
@@ -30,8 +34,8 @@ extract-libs = $(strip $(foreach o,$1,$($o-libs)))
 
 ifeq ($(LIBTOOL),)
 LINK = $(call quiet-command,$(CC) $(QEMU_CFLAGS) $(CFLAGS) $(LDFLAGS) -o $@ \
-       $(sort $(filter %.o, $1)) $(filter-out %.o, $1) $(version-obj-y) \
-       $(call extract-libs,$^) $(LIBS),"  LINK  $(TARGET_DIR)$@")
+       $(call expand-objs,$1) $(version-obj-y) \
+       $(call extract-libs,$1) $(LIBS),"  LINK  $(TARGET_DIR)$@")
 else
 LIBTOOL += $(if $(V),,--quiet)
 %.lo: %.c
@@ -42,12 +46,12 @@ LIBTOOL += $(if $(V),,--quiet)
 	$(call quiet-command,$(LIBTOOL) --mode=compile --tag=CC dtrace -o $@ -G -s $<, " lt GEN $(TARGET_DIR)$@")
 
 LINK = $(call quiet-command,\
-       $(if $(filter %.lo %.la,$^),$(LIBTOOL) --mode=link --tag=CC \
+       $(if $(filter %.lo %.la,$1),$(LIBTOOL) --mode=link --tag=CC \
        )$(CC) $(QEMU_CFLAGS) $(CFLAGS) $(LDFLAGS) -o $@ \
-       $(sort $(filter %.o, $1)) $(filter-out %.o, $1) \
-       $(if $(filter %.lo %.la,$^),$(version-lobj-y),$(version-obj-y)) \
-       $(if $(filter %.lo %.la,$^),$(LIBTOOLFLAGS)) \
-       $(call extract-libs,$^) $(LIBS),$(if $(filter %.lo %.la,$^),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
+       $(call expand-objs,$1) \
+       $(if $(filter %.lo %.la,$1),$(version-lobj-y),$(version-obj-y)) \
+       $(if $(filter %.lo %.la,$1),$(LIBTOOLFLAGS)) \
+       $(call extract-libs,$1) $(LIBS),$(if $(filter %.lo %.la,$1),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
 endif
 
 %.asm: %.S
@@ -65,6 +69,17 @@ endif
 %.o: %.dtrace
 	$(call quiet-command,dtrace -o $@ -G -s $<, "  GEN   $(TARGET_DIR)$@")
 
+%$(DSOSUF): QEMU_CFLAGS += -fPIC
+%$(DSOSUF): LDFLAGS += $(LDFLAGS_SHARED)
+%$(DSOSUF): %.mo libqemustub.a
+	$(call LINK,$^)
+
+.PHONY: modules
+modules:
+
+%.mo:
+	$(call quiet-command,touch $@,"  GEN   $(TARGET_DIR)$@")
+
 %$(EXESUF): %.o
 	$(call LINK,$^)
 
@@ -158,7 +173,10 @@ $(foreach v,$($1), \
 		$(eval $v-cflags := )) \
 	$(if $($v-libs), \
 		$(eval $2$v-libs := $($v-libs)) \
-		$(eval $v-libs := )))
+		$(eval $v-libs := )) \
+	$(if $($v-objs), \
+		$(eval $2$v-objs := $(addprefix $2,$($v-objs))) \
+		$(eval $v-objs := )))
 endef
 
 define unnest-dir
@@ -182,6 +200,15 @@ $(if $(nested-dirs),
   $(call unnest-vars-1))
 endef
 
+define add-modules
+$(foreach o,$(filter %.o,$($1)),
+	$(eval $(patsubst %.o,%.mo,$o): $o) \
+	$(eval $(patsubst %.o,%.mo,$o)-objs := $o))
+$(foreach o,$(filter %.mo,$($1)),$(eval \
+    $o: $($o-objs)))
+$(eval modules-m += $(patsubst %.o,%.mo,$($1)))
+endef
+
 define unnest-vars
 $(eval obj := $1)
 $(eval nested-vars := $2)
@@ -193,4 +220,13 @@ $(foreach var,$(nested-vars),$(eval $(var) := $(filter-out %/, $($(var)))))
 $(shell mkdir -p $(sort $(foreach var,$(nested-vars),$(dir $($(var))))))
 $(foreach var,$(nested-vars), $(eval \
   -include $(addsuffix *.d, $(sort $(dir $($(var)))))))
+$(foreach v,$(filter %-m,$(nested-vars)), \
+    $(call add-modules,$v))
+
+$(if $(CONFIG_MODULES), \
+    $(eval modules: $(patsubst %.mo,%$(DSOSUF),$(modules-m))), \
+    $(foreach v,$(filter %-m,$(nested-vars)), \
+        $(eval $(patsubst %-m,%-y,$v) += $($v)) \
+        $(eval $v := )))
+
 endef
-- 
1.8.5.2

[Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Fam Zheng]

This patch adds loading, stamp checking and initialization of modules.

The init function of dynamic module is no longer directly called as
__attribute__((constructor)) in static linked version, it is called
only after passed the checking of presense of stamp symbol:

    qemu_stamp_$RELEASEHASH

where $RELEASEHASH is generated by hashing version strings and content
of configure script.

With this, modules built from a different tree/version/configure will
not be loaded.

The module loading code requires gmodule-2.0.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Makefile              |   3 ++
 configure             |  37 ++++++++++-------
 include/qemu/module.h |  18 ++++++++-
 module-common.c       |  10 +++++
 rules.mak             |   7 ++--
 scripts/create_config |  14 +++++++
 util/module.c         | 107 +++++++++++++++++++++++++++++++++++++++++++++++++-
 7 files changed, 176 insertions(+), 20 deletions(-)
 create mode 100644 module-common.c

diff --git a/Makefile b/Makefile
index 9de66cb..670ce44 100644
--- a/Makefile
+++ b/Makefile
@@ -203,6 +203,9 @@ Makefile: $(version-obj-y) $(version-lobj-y)
 libqemustub.a: $(stub-obj-y)
 libqemuutil.a: $(util-obj-y) qapi-types.o qapi-visit.o
 
+block-modules = $(foreach o,$(block-obj-m),"$(basename $(subst /,-,$o))",) NULL
+util/module.o-cflags = -D'CONFIG_BLOCK_MODULES=$(block-modules)'
+
 ######################################################################
 
 qemu-img.o: qemu-img-cmds.h
diff --git a/configure b/configure
index a6a626f..77f09ba 100755
--- a/configure
+++ b/configure
@@ -677,7 +677,8 @@ for opt do
   ;;
   --disable-debug-info)
   ;;
-  --enable-modules) modules="yes"
+  --enable-modules)
+      modules="yes"
   ;;
   --cpu=*)
   ;;
@@ -1130,7 +1131,7 @@ Advanced options (experts only):
   --libdir=PATH            install libraries in PATH
   --sysconfdir=PATH        install config in PATH$confsuffix
   --localstatedir=PATH     install local state in PATH (set at runtime on win32)
-  --with-confsuffix=SUFFIX suffix for QEMU data inside datadir and sysconfdir [$confsuffix]
+  --with-confsuffix=SUFFIX suffix for QEMU data inside datadir/libdir/sysconfdir [$confsuffix]
   --enable-modules         enable modules support
   --enable-debug-tcg       enable TCG debugging
   --disable-debug-tcg      disable TCG debugging (default)
@@ -2346,15 +2347,19 @@ if test "$mingw32" = yes; then
 else
     glib_req_ver=2.12
 fi
-if $pkg_config --atleast-version=$glib_req_ver gthread-2.0; then
-    glib_cflags=`$pkg_config --cflags gthread-2.0`
-    glib_libs=`$pkg_config --libs gthread-2.0`
-    CFLAGS="$glib_cflags $CFLAGS"
-    LIBS="$glib_libs $LIBS"
-    libs_qga="$glib_libs $libs_qga"
-else
-    error_exit "glib-$glib_req_ver required to compile QEMU"
-fi
+
+for i in gthread-2.0 gmodule-2.0; do
+    if $pkg_config --atleast-version=$glib_req_ver $i; then
+        glib_cflags=`$pkg_config --cflags $i`
+        glib_libs=`$pkg_config --libs $i`
+        CFLAGS="$glib_cflags $CFLAGS"
+        LIBS="$glib_libs $LIBS"
+        libs_qga="$glib_libs $libs_qga"
+    else
+        error_exit "glib-$glib_req_ver $i is required to compile QEMU"
+    fi
+done
+
 
 ##########################################
 # pixman support probe
@@ -3628,6 +3633,7 @@ if test "$mingw32" = "yes" ; then
 fi
 
 qemu_confdir=$sysconfdir$confsuffix
+moddir=$libdir$confsuffix
 qemu_datadir=$datadir$confsuffix
 qemu_localedir="$datadir/locale"
 
@@ -3718,6 +3724,7 @@ echo "Install prefix    $prefix"
 echo "BIOS directory    `eval echo $qemu_datadir`"
 echo "binary directory  `eval echo $bindir`"
 echo "library directory `eval echo $libdir`"
+echo "module directory  `eval echo $moddir`"
 echo "libexec directory `eval echo $libexecdir`"
 echo "include directory `eval echo $includedir`"
 echo "config directory  `eval echo $sysconfdir`"
@@ -3849,6 +3856,7 @@ echo all: >> $config_host_mak
 echo "prefix=$prefix" >> $config_host_mak
 echo "bindir=$bindir" >> $config_host_mak
 echo "libdir=$libdir" >> $config_host_mak
+echo "moddir=$moddir" >> $config_host_mak
 echo "libexecdir=$libexecdir" >> $config_host_mak
 echo "includedir=$includedir" >> $config_host_mak
 echo "mandir=$mandir" >> $config_host_mak
@@ -3867,9 +3875,6 @@ echo "libs_softmmu=$libs_softmmu" >> $config_host_mak
 
 echo "ARCH=$ARCH" >> $config_host_mak
 
-if test "$modules" = "yes"; then
-  echo "CONFIG_MODULES=y" >> $config_host_mak
-fi
 if test "$debug_tcg" = "yes" ; then
   echo "CONFIG_DEBUG_TCG=y" >> $config_host_mak
 fi
@@ -3991,6 +3996,10 @@ echo "TARGET_DIRS=$target_list" >> $config_host_mak
 if [ "$docs" = "yes" ] ; then
   echo "BUILD_DOCS=yes" >> $config_host_mak
 fi
+echo "CONFIG_STAMP=`(echo $qemu_version; echo $pkgversion; cat $0) | sha256sum - | cut -f1 -d\ `" >> $config_host_mak
+if test "$modules" = "yes"; then
+  echo "CONFIG_MODULES=y" >> $config_host_mak
+fi
 if test "$sdl" = "yes" ; then
   echo "CONFIG_SDL=y" >> $config_host_mak
   echo "SDL_CFLAGS=$sdl_cflags" >> $config_host_mak
diff --git a/include/qemu/module.h b/include/qemu/module.h
index c4ccd57..d6da64f 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -14,11 +14,26 @@
 #ifndef QEMU_MODULE_H
 #define QEMU_MODULE_H
 
+#ifdef BUILD_DSO
+void DSO_STAMP_FUN(void);
+/* This is a dummy symbol to identify a loaded DSO as a QEMU module, so we can
+ * distinguish "version mismatch" from "not a QEMU module", when the stamp
+ * check fails during module loading */
+void qemu_module_dummy(void);
+
+#define module_init(function, type)                                         \
+static void __attribute__((constructor)) do_qemu_init_ ## function(void)    \
+{                                                                           \
+    register_dso_module_init(function, type);                               \
+}
+#else
 /* This should not be used directly.  Use block_init etc. instead.  */
 #define module_init(function, type)                                         \
-static void __attribute__((constructor)) do_qemu_init_ ## function(void) {  \
+static void __attribute__((constructor)) do_qemu_init_ ## function(void)    \
+{                                                                           \
     register_module_init(function, type);                                   \
 }
+#endif
 
 typedef enum {
     MODULE_INIT_BLOCK,
@@ -34,6 +49,7 @@ typedef enum {
 #define type_init(function) module_init(function, MODULE_INIT_QOM)
 
 void register_module_init(void (*fn)(void), module_init_type type);
+void register_dso_module_init(void (*fn)(void), module_init_type type);
 
 void module_call_init(module_init_type type);
 
diff --git a/module-common.c b/module-common.c
new file mode 100644
index 0000000..50c6750
--- /dev/null
+++ b/module-common.c
@@ -0,0 +1,10 @@
+#include "config-host.h"
+#include "qemu/module.h"
+
+void qemu_module_dummy(void)
+{
+}
+
+void DSO_STAMP_FUN(void)
+{
+}
diff --git a/rules.mak b/rules.mak
index fb3482b..b3f7a85 100644
--- a/rules.mak
+++ b/rules.mak
@@ -69,9 +69,9 @@ endif
 %.o: %.dtrace
 	$(call quiet-command,dtrace -o $@ -G -s $<, "  GEN   $(TARGET_DIR)$@")
 
-%$(DSOSUF): QEMU_CFLAGS += -fPIC
+%$(DSOSUF): QEMU_CFLAGS += -fPIC -DBUILD_DSO
 %$(DSOSUF): LDFLAGS += $(LDFLAGS_SHARED)
-%$(DSOSUF): %.mo libqemustub.a
+%$(DSOSUF): %.mo libqemustub.a module-common.o
 	$(call LINK,$^)
 
 .PHONY: modules
@@ -206,7 +206,8 @@ $(foreach o,$(filter %.o,$($1)),
 	$(eval $(patsubst %.o,%.mo,$o)-objs := $o))
 $(foreach o,$(filter %.mo,$($1)),$(eval \
     $o: $($o-objs)))
-$(eval modules-m += $(patsubst %.o,%.mo,$($1)))
+$(eval t := $(patsubst %.o,%.mo,$($1)))
+$(foreach o,$t,$(eval modules-m += $o)))
 endef
 
 define unnest-vars
diff --git a/scripts/create_config b/scripts/create_config
index b1adbf5..d7ba61d 100755
--- a/scripts/create_config
+++ b/scripts/create_config
@@ -26,6 +26,17 @@ case $line in
     # save for the next definitions
     prefix=${line#*=}
     ;;
+ moddir=*)
+    eval "moddir=\"${line#*=}\""
+    echo "#define CONFIG_MODDIR \"$moddir\""
+    ;;
+ CONFIG_STAMP=*)
+    echo "#define DSO_STAMP_FUN qemu_stamp_${line#*=}"
+    echo "#define DSO_STAMP_FUN_STR \"qemu_stamp_${line#*=}\""
+    ;;
+ CONFIG_MODULES=*)
+    echo "#define CONFIG_MODULES \"${line#*=}\""
+    ;;
  CONFIG_AUDIO_DRIVERS=*)
     drivers=${line#*=}
     echo "#define CONFIG_AUDIO_DRIVERS \\"
@@ -104,6 +115,9 @@ case $line in
     value=${line#*=}
     echo "#define $name $value"
     ;;
+ DSOSUF=*)
+    echo "#define HOST_DSOSUF \"${line#*=}\""
+    ;;
 esac
 
 done # read
diff --git a/util/module.c b/util/module.c
index 7acc33d..c4115be 100644
--- a/util/module.c
+++ b/util/module.c
@@ -13,6 +13,7 @@
  * GNU GPL, version 2 or (at your option) any later version.
  */
 
+#include <gmodule.h>
 #include "qemu-common.h"
 #include "qemu/queue.h"
 #include "qemu/module.h"
@@ -21,13 +22,16 @@ typedef struct ModuleEntry
 {
     void (*init)(void);
     QTAILQ_ENTRY(ModuleEntry) node;
+    module_init_type type;
 } ModuleEntry;
 
 typedef QTAILQ_HEAD(, ModuleEntry) ModuleTypeList;
 
 static ModuleTypeList init_type_list[MODULE_INIT_MAX];
 
-static void init_types(void)
+static ModuleTypeList dso_init_list;
+
+static void init_lists(void)
 {
     static int inited;
     int i;
@@ -40,6 +44,8 @@ static void init_types(void)
         QTAILQ_INIT(&init_type_list[i]);
     }
 
+    QTAILQ_INIT(&dso_init_list);
+
     inited = 1;
 }
 
@@ -48,7 +54,7 @@ static ModuleTypeList *find_type(module_init_type type)
 {
     ModuleTypeList *l;
 
-    init_types();
+    init_lists();
 
     l = &init_type_list[type];
 
@@ -62,20 +68,117 @@ void register_module_init(void (*fn)(void), module_init_type type)
 
     e = g_malloc0(sizeof(*e));
     e->init = fn;
+    e->type = type;
 
     l = find_type(type);
 
     QTAILQ_INSERT_TAIL(l, e, node);
 }
 
+void register_dso_module_init(void (*fn)(void), module_init_type type)
+{
+    ModuleEntry *e;
+
+    init_lists();
+
+    e = g_malloc0(sizeof(*e));
+    e->init = fn;
+    e->type = type;
+
+    QTAILQ_INSERT_TAIL(&dso_init_list, e, node);
+}
+
+static void module_load(module_init_type type);
+
 void module_call_init(module_init_type type)
 {
     ModuleTypeList *l;
     ModuleEntry *e;
 
+    module_load(type);
     l = find_type(type);
 
     QTAILQ_FOREACH(e, l, node) {
         e->init();
     }
 }
+
+#ifdef CONFIG_MODULES
+static void module_load_file(const char *fname)
+{
+    GModule *g_module;
+    void (*sym)(void);
+    const char *dsosuf = HOST_DSOSUF;
+    int len = strlen(fname);
+    int suf_len = strlen(dsosuf);
+    ModuleEntry *e, *next;
+
+    if (len <= suf_len || strcmp(&fname[len - suf_len], dsosuf)) {
+        /* wrong suffix */
+        return;
+    }
+    if (access(fname, F_OK)) {
+        return;
+    }
+
+    assert(QTAILQ_EMPTY(&dso_init_list));
+
+    g_module = g_module_open(fname, G_MODULE_BIND_LAZY | G_MODULE_BIND_LOCAL);
+    if (!g_module) {
+        fprintf(stderr, "Failed to load module: %s\n",
+                g_module_error());
+        return;
+    }
+    if (!g_module_symbol(g_module, DSO_STAMP_FUN_STR, (gpointer *)&sym)) {
+        fprintf(stderr, "Failed to initialize module: %s\n",
+                fname);
+        /* Print some info if this is a QEMU module (but from different build),
+         * this will make debugging user problems easier. */
+        if (g_module_symbol(g_module, "qemu_module_dummy", (gpointer *)&sym)) {
+            fprintf(stderr,
+                    "Note: only modules from the same build can be loaded.\n");
+        }
+        g_module_close(g_module);
+    } else {
+        QTAILQ_FOREACH(e, &dso_init_list, node) {
+            register_module_init(e->init, e->type);
+        }
+    }
+
+    QTAILQ_FOREACH_SAFE(e, &dso_init_list, node, next) {
+        QTAILQ_REMOVE(&dso_init_list, e, node);
+        g_free(e);
+    }
+}
+#endif
+
+void module_load(module_init_type type)
+{
+#ifdef CONFIG_MODULES
+    char *fname = NULL;
+    const char **mp;
+    static const char *block_modules[] = {
+        CONFIG_BLOCK_MODULES
+    };
+
+    if (!g_module_supported()) {
+        return;
+    }
+
+    switch (type) {
+    case MODULE_INIT_BLOCK:
+        mp = block_modules;
+        break;
+    /* no other types have dynamic modules for now*/
+    default:
+        return;
+    }
+
+    for ( ; *mp; mp++) {
+        fname = g_strdup_printf("%s/%s%s", CONFIG_MODDIR, *mp, HOST_DSOSUF);
+        module_load_file(fname);
+        g_free(fname);
+    }
+
+#endif
+}
-- 
1.8.5.2

Re: [Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Peter Maydell]

On 15 January 2014 08:48, Fam Zheng <famz@redhat.com> wrote:
> This patch adds loading, stamp checking and initialization of modules.

> +echo "CONFIG_STAMP=`(echo $qemu_version; echo $pkgversion; cat $0) | sha256sum - | cut -f1 -d\ `" >> $config_host_mak

This fails to configure under MacOSX, I'm afraid -- there is
no sha256sum there.

thanks
-- PMM

Re: [Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Paolo Bonzini]

Il 15/01/2014 12:53, Peter Maydell ha scritto:
>> > This patch adds loading, stamp checking and initialization of modules.
>> > +echo "CONFIG_STAMP=`(echo $qemu_version; echo $pkgversion; cat $0) | sha256sum - | cut -f1 -d\ `" >> $config_host_mak
> This fails to configure under MacOSX, I'm afraid -- there is
> no sha256sum there.

Is sha1sum there?  If yes, let's use it.

If not, I guess you'll have to install GNU coreutils from
Homebrew/Fink/whatever is fancy these days, but Fam will have to add a test.

Paolo

Re: [Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Peter Maydell]

On 15 January 2014 11:56, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 15/01/2014 12:53, Peter Maydell ha scritto:
>>> > This patch adds loading, stamp checking and initialization of modules.
>>> > +echo "CONFIG_STAMP=`(echo $qemu_version; echo $pkgversion; cat $0) | sha256sum - | cut -f1 -d\ `" >> $config_host_mak
>> This fails to configure under MacOSX, I'm afraid -- there is
>> no sha256sum there.
>
> Is sha1sum there?  If yes, let's use it.

Nope. There is shasum (because perl provides it) and md5
(not md5sum).

thanks
-- PMM

Re: [Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Paolo Bonzini]

Il 15/01/2014 13:03, Peter Maydell ha scritto:
>>>>> >>> > This patch adds loading, stamp checking and initialization of modules.
>>>>> >>> > +echo "CONFIG_STAMP=`(echo $qemu_version; echo $pkgversion; cat $0) | sha256sum - | cut -f1 -d\ `" >> $config_host_mak
>>> >> This fails to configure under MacOSX, I'm afraid -- there is
>>> >> no sha256sum there.
>> >
>> > Is sha1sum there?  If yes, let's use it.
> Nope. There is shasum (because perl provides it) and md5
> (not md5sum).

Then we need a test that tries both sha1sum and shasum.

Paolo

Re: [Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Peter Maydell]

On 15 January 2014 12:05, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 15/01/2014 13:03, Peter Maydell ha scritto:
>>>>>> >>> > This patch adds loading, stamp checking and initialization of modules.
>>>>>> >>> > +echo "CONFIG_STAMP=`(echo $qemu_version; echo $pkgversion; cat $0) | sha256sum - | cut -f1 -d\ `" >> $config_host_mak
>>>> >> This fails to configure under MacOSX, I'm afraid -- there is
>>>> >> no sha256sum there.
>>> >
>>> > Is sha1sum there?  If yes, let's use it.
>> Nope. There is shasum (because perl provides it) and md5
>> (not md5sum).
>
> Then we need a test that tries both sha1sum and shasum

Which systems do you expect to have sha1sum but not
shasum ?

Googling suggests the BSDs call the native program 'sha256'
but probably have perl's shasum anyway.

thanks
-- PMM

Re: [Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Paolo Bonzini]

Il 15/01/2014 13:11, Peter Maydell ha scritto:
>> >
>> > Then we need a test that tries both sha1sum and shasum
> Which systems do you expect to have sha1sum but not
> shasum ?

Windows (mingw/Cygwin) mainly, but on my Fedora machine only three
programs require perl-Digest-SHA (which provides shasum): cpan, shutter
and git-send-email.  So it's plausible that a system does not have it.

> Googling suggests the BSDs call the native program 'sha256'
> but probably have perl's shasum anyway.

We can add BSD's sha1 (or sha256, but it's simpler to use shasum without
arguments) to the test, too.

Paolo

Re: [Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Alex Bligh]

On 15 Jan 2014, at 12:03, Peter Maydell wrote:

> Nope. There is shasum (because perl provides it) and md5
> (not md5sum).

shasum -a 256 should produce the same output as sha256sum

From the manpage:

       Since shasum uses the same interface employed by the familiar sha1sum
       program (and its somewhat outmoded anscestor md5sum), you can install
       this script as a convenient drop-in replacement.
 
-- 
Alex Bligh

Re: [Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Paolo Bonzini]

Il 15/01/2014 13:09, Alex Bligh ha scritto:
>> > Nope. There is shasum (because perl provides it) and md5
>> > (not md5sum).
> shasum -a 256 should produce the same output as sha256sum
> 
> From the manpage:
> 
>        Since shasum uses the same interface employed by the familiar sha1sum
>        program (and its somewhat outmoded anscestor md5sum), you can install
>        this script as a convenient drop-in replacement.

The exact hash we use is not a problem, we're not looking for crypto safety.

Paolo

Re: [Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Alex Bligh]

On 15 Jan 2014, at 12:10, Paolo Bonzini wrote:

> Il 15/01/2014 13:09, Alex Bligh ha scritto:
>>>> Nope. There is shasum (because perl provides it) and md5
>>>> (not md5sum).
>> shasum -a 256 should produce the same output as sha256sum
>> 
>> From the manpage:
>> 
>>       Since shasum uses the same interface employed by the familiar sha1sum
>>       program (and its somewhat outmoded anscestor md5sum), you can install
>>       this script as a convenient drop-in replacement.
> 
> The exact hash we use is not a problem, we're not looking for crypto safety.

I phrased that badly. I meant same output for the same calling convention, so
surely just replacing 'sha256sum' with 'shasum -a 256' is going to work fine on
all systems?

-- 
Alex Bligh

Re: [Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Paolo Bonzini]

Il 15/01/2014 13:18, Alex Bligh ha scritto:
> I phrased that badly. I meant same output for the same calling convention, so
> surely just replacing 'sha256sum' with 'shasum -a 256' is going to work fine on
> all systems?

I don't think all systems are guaranteed to have shasum.  If we use
SHA-1, and just look for the three binaries sha1sum/sha1/shasum, we
don't have to bother about arguments and only add 10 lines or so to
configure.

Paolo

Re: [Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Fam Zheng]

On Wed, 01/15 13:30, Paolo Bonzini wrote:
> Il 15/01/2014 13:18, Alex Bligh ha scritto:
> > I phrased that badly. I meant same output for the same calling convention, so
> > surely just replacing 'sha256sum' with 'shasum -a 256' is going to work fine on
> > all systems?
> 
> I don't think all systems are guaranteed to have shasum.  If we use
> SHA-1, and just look for the three binaries sha1sum/sha1/shasum, we
> don't have to bother about arguments and only add 10 lines or so to
> configure.
> 

Agreed. I'll add a test and probe for any one of shasum, sha256 and sha256sum.
Easy enough.

Fam

Re: [Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Fam Zheng]

On Wed, 01/15 11:53, Peter Maydell wrote:
> On 15 January 2014 08:48, Fam Zheng <famz@redhat.com> wrote:
> > This patch adds loading, stamp checking and initialization of modules.
> 
> > +echo "CONFIG_STAMP=`(echo $qemu_version; echo $pkgversion; cat $0) | sha256sum - | cut -f1 -d\ `" >> $config_host_mak
> 
> This fails to configure under MacOSX, I'm afraid -- there is
> no sha256sum there.

Can we assume shasum [1] always present on MaxOSX? (Seems it comes with perl.)

[1]: https://developer.apple.com/library/mac/documentation/Darwin/Reference/Manpages/man1/shasum.1.html

Fam

Re: [Qemu-devel] [PATCH v16 6/9] module: implement module loading

[Paolo Bonzini]

Il 15/01/2014 13:34, Fam Zheng ha scritto:
>>> > > +echo "CONFIG_STAMP=`(echo $qemu_version; echo $pkgversion; cat $0) | sha256sum - | cut -f1 -d\ `" >> $config_host_mak
>> > 
>> > This fails to configure under MacOSX, I'm afraid -- there is
>> > no sha256sum there.
> Can we assume shasum [1] always present on MaxOSX? (Seems it comes with perl.)
> 
> [1]: https://developer.apple.com/library/mac/documentation/Darwin/Reference/Manpages/man1/shasum.1.html

We will document the requirement of any of shasum/sha1sum/sha1 in the
release notes anyway.

Paolo

[Qemu-devel] [PATCH v16 7/9] Makefile: install modules with "make install"

[Fam Zheng]

Install all the modules to ${MODDIR}.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Makefile | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/Makefile b/Makefile
index 670ce44..a91f119 100644
--- a/Makefile
+++ b/Makefile
@@ -371,6 +371,12 @@ install-datadir install-localstatedir
 ifneq ($(TOOLS),)
 	$(INSTALL_PROG) $(STRIP_OPT) $(TOOLS) "$(DESTDIR)$(bindir)"
 endif
+ifneq ($(CONFIG_MODULES),)
+	$(INSTALL_DIR) "$(DESTDIR)$(moddir)"
+	for s in $(patsubst %.mo,%$(DSOSUF),$(modules-m)); do \
+		$(INSTALL_PROG) $(STRIP_OPT) $$s "$(DESTDIR)$(moddir)/$${s//\//-}"; \
+	done
+endif
 ifneq ($(HELPERS-y),)
 	$(INSTALL_DIR) "$(DESTDIR)$(libexecdir)"
 	$(INSTALL_PROG) $(STRIP_OPT) $(HELPERS-y) "$(DESTDIR)$(libexecdir)"
-- 
1.8.5.2

[Qemu-devel] [PATCH v16 8/9] .gitignore: ignore module related files (dll, so, mo)

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitignore | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/.gitignore b/.gitignore
index 1c9d63d..7702b0c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -64,6 +64,9 @@ fsdev/virtfs-proxy-helper.pod
 *.cp
 *.dvi
 *.exe
+*.dll
+*.so
+*.mo
 *.fn
 *.ky
 *.log
-- 
1.8.5.2

[Qemu-devel] [PATCH v16 9/9] block: convert block drivers linked with libs to modules

[Fam Zheng]

The converted block drivers are:

    curl
    iscsi
    rbd
    ssh
    glusterfs

Signed-off-by: Fam Zheng <famz@redhat.com>

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 configure | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/configure b/configure
index 77f09ba..80093ea 100755
--- a/configure
+++ b/configure
@@ -4071,7 +4071,7 @@ if test "$bswap_h" = "yes" ; then
   echo "CONFIG_MACHINE_BSWAP_H=y" >> $config_host_mak
 fi
 if test "$curl" = "yes" ; then
-  echo "CONFIG_CURL=y" >> $config_host_mak
+  echo "CONFIG_CURL=m" >> $config_host_mak
   echo "CURL_CFLAGS=$curl_cflags" >> $config_host_mak
   echo "CURL_LIBS=$curl_libs" >> $config_host_mak
 fi
@@ -4162,7 +4162,7 @@ if test "$glx" = "yes" ; then
 fi
 
 if test "$libiscsi" = "yes" ; then
-  echo "CONFIG_LIBISCSI=y" >> $config_host_mak
+  echo "CONFIG_LIBISCSI=m" >> $config_host_mak
   if test "$libiscsi_version" = "1.4.0"; then
     echo "CONFIG_LIBISCSI_1_4=y" >> $config_host_mak
   fi
@@ -4188,7 +4188,7 @@ if test "$qom_cast_debug" = "yes" ; then
   echo "CONFIG_QOM_CAST_DEBUG=y" >> $config_host_mak
 fi
 if test "$rbd" = "yes" ; then
-  echo "CONFIG_RBD=y" >> $config_host_mak
+  echo "CONFIG_RBD=m" >> $config_host_mak
   echo "RBD_CFLAGS=$rbd_cflags" >> $config_host_mak
   echo "RBD_LIBS=$rbd_libs" >> $config_host_mak
 fi
@@ -4233,7 +4233,7 @@ if test "$getauxval" = "yes" ; then
 fi
 
 if test "$glusterfs" = "yes" ; then
-  echo "CONFIG_GLUSTERFS=y" >> $config_host_mak
+  echo "CONFIG_GLUSTERFS=m" >> $config_host_mak
   echo "GLUSTERFS_CFLAGS=$glusterfs_cflags" >> $config_host_mak
   echo "GLUSTERFS_LIBS=$glusterfs_libs" >> $config_host_mak
 fi
@@ -4243,7 +4243,7 @@ if test "$glusterfs_discard" = "yes" ; then
 fi
 
 if test "$libssh2" = "yes" ; then
-  echo "CONFIG_LIBSSH2=y" >> $config_host_mak
+  echo "CONFIG_LIBSSH2=m" >> $config_host_mak
   echo "LIBSSH2_CFLAGS=$libssh2_cflags" >> $config_host_mak
   echo "LIBSSH2_LIBS=$libssh2_libs" >> $config_host_mak
 fi
-- 
1.8.5.2

Re: [Qemu-devel] [PATCH v16 0/9] Shared library module support

[Peter Maydell]

On 15 January 2014 08:48, Fam Zheng <famz@redhat.com> wrote:
> A few changes on Paolo's v15, to fix MacOSX build (in fact fix Linux as well)
> and get rid of $RANDOM:

Thanks. I applied a local patch to use shasum, and it seems
to build OK with modules enabled. Is there a simple test I can
do that will exercise a code path that actually requires a module
to be loaded?

-- PMM

Re: [Qemu-devel] [PATCH v16 0/9] Shared library module support

[Paolo Bonzini]

Il 15/01/2014 14:11, Peter Maydell ha scritto:
> On 15 January 2014 08:48, Fam Zheng <famz@redhat.com> wrote:
>> A few changes on Paolo's v15, to fix MacOSX build (in fact fix Linux as well)
>> and get rid of $RANDOM:
> 
> Thanks. I applied a local patch to use shasum, and it seems
> to build OK with modules enabled. Is there a simple test I can
> do that will exercise a code path that actually requires a module
> to be loaded?

On RHEL6 I tried "qemu-system-x86_64 -cdrom
http://boot.ipxe.org/ipxe.iso" and it worked, but it failed on Fedora.
I don't know if it's a QEMU or curl bug.

Paolo

Re: [Qemu-devel] [PATCH v16 0/9] Shared library module support

[Peter Maydell]

On 15 January 2014 14:11, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 15/01/2014 14:11, Peter Maydell ha scritto:
>> On 15 January 2014 08:48, Fam Zheng <famz@redhat.com> wrote:
>>> A few changes on Paolo's v15, to fix MacOSX build (in fact fix Linux as well)
>>> and get rid of $RANDOM:
>>
>> Thanks. I applied a local patch to use shasum, and it seems
>> to build OK with modules enabled. Is there a simple test I can
>> do that will exercise a code path that actually requires a module
>> to be loaded?
>
> On RHEL6 I tried "qemu-system-x86_64 -cdrom
> http://boot.ipxe.org/ipxe.iso" and it worked, but it failed on Fedora.
> I don't know if it's a QEMU or curl bug.

Doesn't work on MacOSX either: it says
qemu-system-x86_64: -cdrom http://boot.ipxe.org/ipxe.iso: could not
open disk image http://boot.ipxe.org/ipxe.iso: Unknown protocol

(a non module build works, or at least sits there without giving an
error message; I didn't feel like sitting there without a progress bar
while it downloaded a whole ISO image, so I ^C'd it...)

thanks
-- PMM

Re: [Qemu-devel] [PATCH v16 0/9] Shared library module support

[Paolo Bonzini]

Il 15/01/2014 15:20, Peter Maydell ha scritto:
>> >
>> > On RHEL6 I tried "qemu-system-x86_64 -cdrom
>> > http://boot.ipxe.org/ipxe.iso" and it worked, but it failed on Fedora.
>> > I don't know if it's a QEMU or curl bug.
> Doesn't work on MacOSX either: it says
> qemu-system-x86_64: -cdrom http://boot.ipxe.org/ipxe.iso: could not
> open disk image http://boot.ipxe.org/ipxe.iso: Unknown protocol

Had you installed QEMU before starting it?  Fam, I think you need a way
to override CONFIG_MODDIR (even an environment variable could do).

> (a non module build works, or at least sits there without giving an
> error message; I didn't feel like sitting there without a progress bar
> while it downloaded a whole ISO image, so I ^C'd it...)

It's really hung.  The ISO is just 1M.  With RHEL6 it boots fine.

Paolo

Re: [Qemu-devel] [PATCH v16 0/9] Shared library module support

[Peter Maydell]

On 15 January 2014 14:36, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 15/01/2014 15:20, Peter Maydell ha scritto:
>>> >
>>> > On RHEL6 I tried "qemu-system-x86_64 -cdrom
>>> > http://boot.ipxe.org/ipxe.iso" and it worked, but it failed on Fedora.
>>> > I don't know if it's a QEMU or curl bug.
>> Doesn't work on MacOSX either: it says
>> qemu-system-x86_64: -cdrom http://boot.ipxe.org/ipxe.iso: could not
>> open disk image http://boot.ipxe.org/ipxe.iso: Unknown protocol
>
> Had you installed QEMU before starting it?  Fam, I think you need a way
> to override CONFIG_MODDIR (even an environment variable could do).

No, I pretty much never install QEMU. I always run it out of a build
directory. Module loading should IMHO work with similar semantics
to the data dir we use for BIOS file loading: one of the places checked
is relative to the executable path.

Maybe we should also have better error messages for "I tried
to load a module and this error is because I couldn't find one",
so the user might be prompted to install the modules package his
distro provides or investigate config if it's self-built, or whatever...

>> (a non module build works, or at least sits there without giving an
>> error message; I didn't feel like sitting there without a progress bar
>> while it downloaded a whole ISO image, so I ^C'd it...)
>
> It's really hung.  The ISO is just 1M.  With RHEL6 it boots fine.

Hrrm. I guess I should investigate that...

thanks
-- PMM

Re: [Qemu-devel] [PATCH v16 0/9] Shared library module support

[Paolo Bonzini]

Il 15/01/2014 15:40, Peter Maydell ha scritto:
> Module loading should IMHO work with similar semantics
> to the data dir we use for BIOS file loading: one of the places checked
> is relative to the executable path.

Makes sense.

>>> (a non module build works, or at least sits there without giving an
>>> error message; I didn't feel like sitting there without a progress bar
>>> while it downloaded a whole ISO image, so I ^C'd it...)
>>
>> It's really hung.  The ISO is just 1M.  With RHEL6 it boots fine.
>
> Hrrm. I guess I should investigate that...

Probably not specific to Mac OS X.  As I said, Fedora fails too.

Paolo

Re: [Qemu-devel] [PATCH v16 0/9] Shared library module support

[Peter Maydell]

On 15 January 2014 14:51, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 15/01/2014 15:40, Peter Maydell ha scritto:

>>>> (a non module build works, or at least sits there without giving an
>>>> error message; I didn't feel like sitting there without a progress bar
>>>> while it downloaded a whole ISO image, so I ^C'd it...)
>>>
>>> It's really hung.  The ISO is just 1M.  With RHEL6 it boots fine.
>>
>> Hrrm. I guess I should investigate that...
>
> Probably not specific to Mac OS X.  As I said, Fedora fails too.

I threw some debug printouts in, and it looks like libcurl
is just never calling the curl_sock_cb, so we never register
the fd with QEMU's event loop, and so nothing happens once
the first read AIO has commenced.

thanks
-- PMM

Re: [Qemu-devel] [PATCH v16 0/9] Shared library module support

[Peter Maydell]

On 15 January 2014 15:29, Peter Maydell <peter.maydell@linaro.org> wrote:
> I threw some debug printouts in, and it looks like libcurl
> is just never calling the curl_sock_cb, so we never register
> the fd with QEMU's event loop, and so nothing happens once
> the first read AIO has commenced.

Further hacking suggests this is probably because we're not
implementing the curl timeout callback, which curl uses to say
"call me back in X milliseconds, even if there's no filedescriptor
action". It looks like the first time or two around curl says "call
back in 1ms" before it gets things rolling enough to be waiting
on a filedescriptor. (Then after that it sets a 2second timeout
presumably so it can deal with unexpected situations like the
far end never responding.)

What's the best way to get QEMU's event loop to call back
after a specified timeout?

thanks
-- PMM

Re: [Qemu-devel] [PATCH v16 0/9] Shared library module support

[Paolo Bonzini]

Il 15/01/2014 16:40, Peter Maydell ha scritto:
> Further hacking suggests this is probably because we're not
> implementing the curl timeout callback, which curl uses to say
> "call me back in X milliseconds, even if there's no filedescriptor
> action". It looks like the first time or two around curl says "call
> back in 1ms" before it gets things rolling enough to be waiting
> on a filedescriptor. (Then after that it sets a 2second timeout
> presumably so it can deal with unexpected situations like the
> far end never responding.)
> 
> What's the best way to get QEMU's event loop to call back
> after a specified timeout?

You can use aio_timer_new or aio_timer_init.

Paolo

Re: [Qemu-devel] [PATCH v16 0/9] Shared library module support

[Fam Zheng]

On Wed, 01/15 14:40, Peter Maydell wrote:
> On 15 January 2014 14:36, Paolo Bonzini <pbonzini@redhat.com> wrote:
> > Il 15/01/2014 15:20, Peter Maydell ha scritto:
> >>> >
> >>> > On RHEL6 I tried "qemu-system-x86_64 -cdrom
> >>> > http://boot.ipxe.org/ipxe.iso" and it worked, but it failed on Fedora.
> >>> > I don't know if it's a QEMU or curl bug.
> >> Doesn't work on MacOSX either: it says
> >> qemu-system-x86_64: -cdrom http://boot.ipxe.org/ipxe.iso: could not
> >> open disk image http://boot.ipxe.org/ipxe.iso: Unknown protocol
> >
> > Had you installed QEMU before starting it?  Fam, I think you need a way
> > to override CONFIG_MODDIR (even an environment variable could do).
> 
> No, I pretty much never install QEMU. I always run it out of a build
> directory. Module loading should IMHO work with similar semantics
> to the data dir we use for BIOS file loading: one of the places checked
> is relative to the executable path.
> 
> Maybe we should also have better error messages for "I tried
> to load a module and this error is because I couldn't find one",
> so the user might be prompted to install the modules package his
> distro provides or investigate config if it's self-built, or whatever...
> 

Sounds good to me. Will add a message.

Fam