[Qemu-devel] [PATCH 0/2] Resolve link errors on Mac OS X

[Qemu-devel] [PATCH 0/2] Resolve link errors on Mac OS X

[Christopher Friedt]

[-- Attachment #1: Type: text/plain, Size: 1612 bytes --]

Hi list,

I recently tried to build Qemu on Mac and ran into a couple of trivial issues
that I've provided patches for. I suppose that normally people just use
'brew install qemu', but there is really no reason that it can't be built from
source, particularly for those modifying Qemu regularly.

In any case, the first change moves to using 'libtool -static' to create
libraries on Mac OS X. If one attempts to use ar and ranlib, then the final
link will fail with error messages resembling the following:

  ld: warning: ignoring file libqemuutil.a, file was built for archive which
  is not the architecture being linked (x86_64): libqemuutil.a
  Undefined symbols for architecture x86_64:
  ...

Notice ld (Apple's ld64) presumes the static library is a relocatable with
architecture 'archive' rather than x86_64, in this case.

A similar fix is required for dtc - again quite trivial, but I can provide that if necessary.

The second patch removes the preprocessor conditional around the function
event_notifier_init_fd() in util/event_notifier-posix.c so that the link does
not fail on systems where CONFIG_POSIX is defined but CONFIG_EVENTFD is not
(such as under Mac OS X).

There is more information in each of the commits that follows.

Please feel free to comment.

Cheers,

C

Christopher Friedt (2):
  Use libtool instead of ar to create static libraries on Darwin.
  Remove unnecessary CONFIG_EVENTFD preprocessor conditional to satisfy
    link

 rules.mak                   | 4 ++++
 util/event_notifier-posix.c | 2 --
 2 files changed, 4 insertions(+), 2 deletions(-)

-- 
2.6.4 (Apple Git-63)

[Qemu-devel] [PATCH 1/2] Use libtool instead of ar to create static libraries on Darwin.

[Christopher Friedt]

[-- Attachment #1: Type: text/plain, Size: 686 bytes --]


Currently, at least on Mac OS X 10.11.4 (El Capitan), Qemu fails to build for a few reasons.

One of those reasons is that Apple's ld (at least ld64) does not properly process archive files created with ar (even Apple's ar).

After some RTFM'ing, I came upon this tidbit, which is unfortunate. Luckily, autotools packages are not broken.

"Libtool with -static is intended to replace ar(5) and ranlib."
http://www.manpages.info/macosx/libtool.1.html

In any case, this change takes Apple's recommendations into account and allows Qemu to build on Mac OS X El Capitan.

Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
---
 rules.mak | 4 ++++
 1 file changed, 4 insertions(+)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Use-libtool-instead-of-ar-to-create-static-libraries.patch --]
[-- Type: text/x-patch; name="0001-Use-libtool-instead-of-ar-to-create-static-libraries.patch", Size: 456 bytes --]

diff --git a/rules.mak b/rules.mak
index d1ff311..44421af 100644
--- a/rules.mak
+++ b/rules.mak
@@ -105,7 +105,11 @@ modules:
 	$(call LINK,$(filter %.o %.a %.mo, $^))
 
 %.a:
+ifeq ($(shell uname),Darwin)
+	$(call quiet-command,rm -f $@ && libtool -static -o $@ $^,"  libtool    $(TARGET_DIR)$@")
+else
 	$(call quiet-command,rm -f $@ && $(AR) rcs $@ $^,"  AR    $(TARGET_DIR)$@")
+endif
 
 quiet-command = $(if $(V),$1,$(if $(2),@echo $2 && $1, @$1))
 

[Qemu-devel] [PATCH 2/2] Remove unnecessary CONFIG_EVENTFD preprocessor conditional to satisfy link

[Christopher Friedt]

[-- Attachment #1: Type: text/plain, Size: 637 bytes --]


The file ivshmem.c unconditionally references event_notifier_init_fd() in util/event_notifier-posix.c, even if CONFIG_EVENTFD is not defined. On platforms where CONFIG_POSIX is defined, but CONFIG_EVENTFD is not defined, that results in an undefined symbol referenced from ivshmem.c and the link fails. That applies to Mac OS X, but possibly other BSD-based distros.

Note: there is nothing specific to eventfd inside and event_notifier_init() also fails unconditionally if CONFIG_EVENTFD is not defined.
Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
---
 util/event_notifier-posix.c | 2 --
 1 file changed, 2 deletions(-)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0002-Remove-unnecessary-CONFIG_EVENTFD-preprocessor-condi.patch --]
[-- Type: text/x-patch; name="0002-Remove-unnecessary-CONFIG_EVENTFD-preprocessor-condi.patch", Size: 554 bytes --]

diff --git a/util/event_notifier-posix.c b/util/event_notifier-posix.c
index c1f0d79..c9bb34d 100644
--- a/util/event_notifier-posix.c
+++ b/util/event_notifier-posix.c
@@ -21,7 +21,6 @@
 #include <sys/eventfd.h>
 #endif
 
-#ifdef CONFIG_EVENTFD
 /*
  * Initialize @e with existing file descriptor @fd.
  * @fd must be a genuine eventfd object, emulation with pipe won't do.
@@ -31,7 +30,6 @@ void event_notifier_init_fd(EventNotifier *e, int fd)
     e->rfd = fd;
     e->wfd = fd;
 }
-#endif
 
 int event_notifier_init(EventNotifier *e, int active)
 {

Re: [Qemu-devel] [PATCH 1/2] Use libtool instead of ar to create static libraries on Darwin.

[Peter Maydell]

On 3 May 2016 at 01:47, Christopher Friedt <chrisfriedt@gmail.com> wrote:
>
> Currently, at least on Mac OS X 10.11.4 (El Capitan), Qemu fails
> to build for a few reasons.

I guess this is a new-in-ElCapitan thing? I run Yosemite, which is
fine.

thanks
-- PMM

Re: [Qemu-devel] [PATCH 2/2] Remove unnecessary CONFIG_EVENTFD preprocessor conditional to satisfy link

[Peter Maydell]

[ccing somebody else who ran into this, since I've figured out why.]

On 3 May 2016 at 01:47, Christopher Friedt <chrisfriedt@gmail.com> wrote:
> The file ivshmem.c unconditionally references event_notifier_init_fd()
> in util/event_notifier-posix.c, even if CONFIG_EVENTFD is not defined.

Yes, but ivshmem.c is only built if CONFIG_IVSHMEM, and
CONFIG_IVSHMEM is set (in default-configs/pci.mak) to CONFIG_EVENTFD.
So if CONFIG_EVENTFD is not defined then we should never build ivshmem.o.

The problem here is that you've run into a bug in QEMU's makefiles,
where a change in an included .mak file in default-configs fails
to cause the config-devices.mak file to be rebuilt. In commit
330b583 we changed pci.mak so that CONFIG_IVSHMEM is set to
CONFIG_EVENTFD rather than CONFIG_POSIX, so if your config-devices.mak
predates that commit then it will have incorrectly not been recreated
and so your build will still try to build ivshmem.c.

You can check whether this is true by looking at
(for instance) x86_64-softmmu/config-devices.mak in your build
tree: if it has a line "CONFIG_IVSHMEM=$(CONFIG_POSIX)" in it
then it's the out of date version.

You should be able to fix this by deleting */config-devices.mak
from your build tree (or by blowing away the build tree entirely
and recreating it.) Then try rebuilding -- ivshmem.c should not
be compiled.

We really must track down this dependency bug, it is very confusing
when it bites people.

thanks
-- PMM

Re: [Qemu-devel] [PATCH 1/2] Use libtool instead of ar to create static libraries on Darwin.

[Fam Zheng]

Cc'ing Peter Maydell, who must have better ideas than me on building on Mac.

On Mon, 05/02 20:47, Christopher Friedt wrote:
> 
> Currently, at least on Mac OS X 10.11.4 (El Capitan), Qemu fails to build for a few reasons.
> 
> One of those reasons is that Apple's ld (at least ld64) does not properly process archive files created with ar (even Apple's ar).
> 
> After some RTFM'ing, I came upon this tidbit, which is unfortunate. Luckily, autotools packages are not broken.
> 
> "Libtool with -static is intended to replace ar(5) and ranlib."
> http://www.manpages.info/macosx/libtool.1.html
> 
> In any case, this change takes Apple's recommendations into account and allows Qemu to build on Mac OS X El Capitan.
> 
> Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
> ---
>  rules.mak | 4 ++++
>  1 file changed, 4 insertions(+)
> 

> diff --git a/rules.mak b/rules.mak
> index d1ff311..44421af 100644
> --- a/rules.mak
> +++ b/rules.mak
> @@ -105,7 +105,11 @@ modules:
>  	$(call LINK,$(filter %.o %.a %.mo, $^))
>  
>  %.a:
> +ifeq ($(shell uname),Darwin)

I think you can use "ifdef CONFIG_DARWIN" here.

> +	$(call quiet-command,rm -f $@ && libtool -static -o $@ $^,"  libtool    $(TARGET_DIR)$@")

Is libtool always available on Mac OS X? If not, we may need to add the
detection to ./configure.

Fam

> +else
>  	$(call quiet-command,rm -f $@ && $(AR) rcs $@ $^,"  AR    $(TARGET_DIR)$@")
> +endif
>  
>  quiet-command = $(if $(V),$1,$(if $(2),@echo $2 && $1, @$1))
>  

Re: [Qemu-devel] [PATCH 1/2] Use libtool instead of ar to create static libraries on Darwin.

[Christopher Friedt]

On Mon, May 2, 2016 at 8:53 PM, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 3 May 2016 at 01:47, Christopher Friedt <chrisfriedt@gmail.com> wrote:
>>
>> Currently, at least on Mac OS X 10.11.4 (El Capitan), Qemu fails
>> to build for a few reasons.
>
> I guess this is a new-in-ElCapitan thing? I run Yosemite, which is
> fine.

I think you're partially correct, although the information on the man page dates
back to at least Snow Leopard (10.6.2) [1], so Apple has preferred the
libtool method for a very long time.

It's possible that it only finally broke in El Capitan.

[1] http://www.unix.com/man-page/osx/1/libtool/

Re: [Qemu-devel] [PATCH 1/2] Use libtool instead of ar to create static libraries on Darwin.

[Christopher Friedt]

On Mon, May 2, 2016 at 9:04 PM, Fam Zheng <famz@redhat.com> wrote:
> Cc'ing Peter Maydell, who must have better ideas than me on building on Mac.
>
> On Mon, 05/02 20:47, Christopher Friedt wrote:
>>
>> Currently, at least on Mac OS X 10.11.4 (El Capitan), Qemu fails to build for a few reasons.
>>
>> One of those reasons is that Apple's ld (at least ld64) does not properly process archive files created with ar (even Apple's ar).
>>
>> After some RTFM'ing, I came upon this tidbit, which is unfortunate. Luckily, autotools packages are not broken.
>>
>> "Libtool with -static is intended to replace ar(5) and ranlib."
>> http://www.manpages.info/macosx/libtool.1.html
>>
>> In any case, this change takes Apple's recommendations into account and allows Qemu to build on Mac OS X El Capitan.
>>
>> Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
>> ---
>>  rules.mak | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>
>> diff --git a/rules.mak b/rules.mak
>> index d1ff311..44421af 100644
>> --- a/rules.mak
>> +++ b/rules.mak
>> @@ -105,7 +105,11 @@ modules:
>>       $(call LINK,$(filter %.o %.a %.mo, $^))
>>
>>  %.a:
>> +ifeq ($(shell uname),Darwin)
>
> I think you can use "ifdef CONFIG_DARWIN" here.

Good suggestion. I missed that entirely.

>> +     $(call quiet-command,rm -f $@ && libtool -static -o $@ $^,"  libtool    $(TARGET_DIR)$@")
>
> Is libtool always available on Mac OS X? If not, we may need to add the
> detection to ./configure.

Apple's libtool should be installed with their command line tools, at
least as far back as Snow Leopard.

Re: [Qemu-devel] [PATCH 2/2] Remove unnecessary CONFIG_EVENTFD preprocessor conditional to satisfy link

[Christopher Friedt]

On Mon, May 2, 2016 at 9:01 PM, Peter Maydell <peter.maydell@linaro.org> wrote:
> [ccing somebody else who ran into this, since I've figured out why.]
>
> On 3 May 2016 at 01:47, Christopher Friedt <chrisfriedt@gmail.com> wrote:
>> The file ivshmem.c unconditionally references event_notifier_init_fd()
>> in util/event_notifier-posix.c, even if CONFIG_EVENTFD is not defined.
>
> Yes, but ivshmem.c is only built if CONFIG_IVSHMEM, and
> CONFIG_IVSHMEM is set (in default-configs/pci.mak) to CONFIG_EVENTFD.
> So if CONFIG_EVENTFD is not defined then we should never build ivshmem.o.
>
> The problem here is that you've run into a bug in QEMU's makefiles,

That would explain things.

> where a change in an included .mak file in default-configs fails
> to cause the config-devices.mak file to be rebuilt. In commit
> 330b583 we changed pci.mak so that CONFIG_IVSHMEM is set to
> CONFIG_EVENTFD rather than CONFIG_POSIX, so if your config-devices.mak
> predates that commit then it will have incorrectly not been recreated
> and so your build will still try to build ivshmem.c.
>
> You can check whether this is true by looking at
> (for instance) x86_64-softmmu/config-devices.mak in your build
> tree: if it has a line "CONFIG_IVSHMEM=$(CONFIG_POSIX)" in it
> then it's the out of date version.

$ grep "CONFIG_IVSHMEM" x86_64-softmmu/config-devices.mak
CONFIG_IVSHMEM=$(CONFIG_EVENTFD)

I have been re-configuring on every build, just to eliminate that
potential error source.

> You should be able to fix this by deleting */config-devices.mak
> from your build tree (or by blowing away the build tree entirely
> and recreating it.) Then try rebuilding -- ivshmem.c should not
> be compiled.

I'll give that a go anyway.

Re: [Qemu-devel] [PATCH 2/2] Remove unnecessary CONFIG_EVENTFD preprocessor conditional to satisfy link

[Peter Maydell]

On 3 May 2016 at 02:14, Christopher Friedt <chrisfriedt@gmail.com> wrote:
> On Mon, May 2, 2016 at 9:01 PM, Peter Maydell <peter.maydell@linaro.org> wrote:
>> [ccing somebody else who ran into this, since I've figured out why.]
>>
>> On 3 May 2016 at 01:47, Christopher Friedt <chrisfriedt@gmail.com> wrote:
>>> The file ivshmem.c unconditionally references event_notifier_init_fd()
>>> in util/event_notifier-posix.c, even if CONFIG_EVENTFD is not defined.
>>
>> Yes, but ivshmem.c is only built if CONFIG_IVSHMEM, and
>> CONFIG_IVSHMEM is set (in default-configs/pci.mak) to CONFIG_EVENTFD.
>> So if CONFIG_EVENTFD is not defined then we should never build ivshmem.o.
>>
>> The problem here is that you've run into a bug in QEMU's makefiles,
>
> That would explain things.
>
>> where a change in an included .mak file in default-configs fails
>> to cause the config-devices.mak file to be rebuilt. In commit
>> 330b583 we changed pci.mak so that CONFIG_IVSHMEM is set to
>> CONFIG_EVENTFD rather than CONFIG_POSIX, so if your config-devices.mak
>> predates that commit then it will have incorrectly not been recreated
>> and so your build will still try to build ivshmem.c.
>>
>> You can check whether this is true by looking at
>> (for instance) x86_64-softmmu/config-devices.mak in your build
>> tree: if it has a line "CONFIG_IVSHMEM=$(CONFIG_POSIX)" in it
>> then it's the out of date version.
>
> $ grep "CONFIG_IVSHMEM" x86_64-softmmu/config-devices.mak
> CONFIG_IVSHMEM=$(CONFIG_EVENTFD)

Hmm. (Is it set the same for every config-devices.mak for
every target you're trying to build?)

Next things to check: is CONFIG_EVENTFD definitely not set
in config-host.mak (it shouldn't be)? hw/misc/Makefile.objs
ought to say "obj-$(CONFIG_IVSHMEM) += ivshmem.o", and that
should mean that we don't try to build it...

thanks
-- PMM

Re: [Qemu-devel] [PATCH 2/2] Remove unnecessary CONFIG_EVENTFD preprocessor conditional to satisfy link

[Christopher Friedt]

On Mon, May 2, 2016 at 9:18 PM, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 3 May 2016 at 02:14, Christopher Friedt <chrisfriedt@gmail.com> wrote:
>> On Mon, May 2, 2016 at 9:01 PM, Peter Maydell <peter.maydell@linaro.org> wrote:
>>> [ccing somebody else who ran into this, since I've figured out why.]
>>>
>>> On 3 May 2016 at 01:47, Christopher Friedt <chrisfriedt@gmail.com> wrote:
>>>> The file ivshmem.c unconditionally references event_notifier_init_fd()
>>>> in util/event_notifier-posix.c, even if CONFIG_EVENTFD is not defined.
>>>
>>> Yes, but ivshmem.c is only built if CONFIG_IVSHMEM, and
>>> CONFIG_IVSHMEM is set (in default-configs/pci.mak) to CONFIG_EVENTFD.
>>> So if CONFIG_EVENTFD is not defined then we should never build ivshmem.o.
>>>
>>> The problem here is that you've run into a bug in QEMU's makefiles,
>>
>> That would explain things.



> Hmm. (Is it set the same for every config-devices.mak for
> every target you're trying to build?)

Ack - too late! I manually removed all of the *-softmmu/ directories
after doing a make clean, because they were still hanging around.
Rebuilding worked fine without PATCH 2/2.

Thanks!

Re: [Qemu-devel] [PATCH 2/2] Remove unnecessary CONFIG_EVENTFD preprocessor conditional to satisfy link

[Peter Maydell]

On 3 May 2016 at 02:25, Christopher Friedt <chrisfriedt@gmail.com> wrote:
> Ack - too late! I manually removed all of the *-softmmu/ directories
> after doing a make clean, because they were still hanging around.
> Rebuilding worked fine without PATCH 2/2.

Oh well. If blowing away the build dirs fixed it then it seems
pretty likely that the makefile bug is what you were running into.

thanks
-- PMM

Re: [Qemu-devel] [PATCH 1/2] Use libtool instead of ar to create static libraries on Darwin.

[Michael Tokarev]

Hmm.  We removed libtool support just two moths ago... :)

/mjt

Re: [Qemu-devel] [PATCH 1/2] Use libtool instead of ar to create static libraries on Darwin.

[Peter Maydell]

On 6 May 2016 at 20:07, Michael Tokarev <mjt@tls.msk.ru> wrote:
> Hmm.  We removed libtool support just two moths ago... :)

That was GNU libtool, which is a completely different thing
from OSX libtool. They just have an unhelpful name clash.

thanks
-- PMM