[Buildroot] [PATCH 0/2] qemu: allow selection of host qemu-user from menuconfig

[Buildroot] [PATCH 0/2] qemu: allow selection of host qemu-user from menuconfig

[Thomas Petazzoni]

Hello,

This patch series is a respin of those old patches posted by Frank
Hunleth.

The first patch is almost unchanged (just refreshed on top of the
current master) and creates a menuconfig entry in 'Host utilities' to
enable Qemu.

The second patch is inspired by
http://patchwork.ozlabs.org/patch/319108/ (from Frank as well), but
has a completely different implementation, which I find nicer (based
on make code, plus the check is done much earlier, plus we don't need
to depend on the toolchain package). Ideas on how to make the 'make'
checks nicer are welcome.

This set of patches superseds:

  http://patchwork.ozlabs.org/patch/319108/ (from Frank)
  http://patchwork.ozlabs.org/patch/299014/ (from Frank)
  http://patchwork.ozlabs.org/patch/419140/ (from Thierry Bultel)

Comments welcome.

Thomas

Frank Hunleth (1):
  qemu: add to host utilities menu

Thomas Petazzoni (1):
  qemu: add host/target Linux version check

 package/Config.in.host      |  1 +
 package/qemu/Config.in.host |  8 ++++++++
 package/qemu/qemu.mk        | 28 ++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+)
 create mode 100644 package/qemu/Config.in.host

-- 
2.1.0

[Buildroot] [PATCH 1/2] qemu: add to host utilities menu

[Thomas Petazzoni]

From: Frank Hunleth <fhunleth@troodon-software.com>

This allows qemu-user to be selected by the user. One use case
for this is to call qemu-user from post build scripts to
run regression tests against the build.

Signed-off-by: Frank Hunleth <fhunleth@troodon-software.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/Config.in.host      | 1 +
 package/qemu/Config.in.host | 8 ++++++++
 2 files changed, 9 insertions(+)
 create mode 100644 package/qemu/Config.in.host

diff --git a/package/Config.in.host b/package/Config.in.host
index 0b7bc36..9b127c8 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -18,6 +18,7 @@ menu "Host utilities"
 	source "package/parted/Config.in.host"
 	source "package/patchelf/Config.in.host"
 	source "package/pwgen/Config.in.host"
+	source "package/qemu/Config.in.host"
 	source "package/sam-ba/Config.in.host"
 	source "package/squashfs/Config.in.host"
 	source "package/sunxi-tools/Config.in.host"
diff --git a/package/qemu/Config.in.host b/package/qemu/Config.in.host
new file mode 100644
index 0000000..a1f0531
--- /dev/null
+++ b/package/qemu/Config.in.host
@@ -0,0 +1,8 @@
+config BR2_PACKAGE_HOST_QEMU
+	bool "host qemu"
+	help
+	  QEMU is a generic and open source machine emulator and virtualizer.
+
+	  This option builds a user emulator for your selected architecture.
+
+	  http://www.qemu.org
-- 
2.1.0

[Buildroot] [PATCH 2/2] qemu: add host/target Linux version check

[Thomas Petazzoni]

Raise an error if the host is using an older kernel than the target.
Since qemu-user passes emulated system calls to the host kernel,
this prevents usage of qemu-user in situations where those system
calls will fail.

This is based on an original patch from Frank Hunleth
<fhunleth@troodon-software.com>, but completely rewritten in a
different way:

 * Instead of using shell based testing, we use pure make tests, which
   allows to detect the problem not when host-qemu starts to build,
   but at the very beginning of the entire Buildroot build.

 * Instead of looking at $(STAGING_DIR)/usr/include/linux/version.h
   (which requires having a dependency on the 'toolchain' package,
   which is a bit unusual for a host package), we use the
   BR2_TOOLCHAIN_HEADERS_AT_LEAST Config.in option which tells us the
   version of the kernel headers used in the toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/qemu/qemu.mk | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/package/qemu/qemu.mk b/package/qemu/qemu.mk
index 9cff5ed..6c5ddf3 100644
--- a/package/qemu/qemu.mk
+++ b/package/qemu/qemu.mk
@@ -59,6 +59,34 @@ HOST_QEMU_ARCH = ppc
 endif
 HOST_QEMU_TARGETS = $(HOST_QEMU_ARCH)-linux-user
 
+ifeq ($(BR2_PACKAGE_HOST_QEMU),y)
+HOST_QEMU_HOST_SYSTEM_TYPE = $(shell uname -s)
+ifneq ($(HOST_QEMU_HOST_SYSTEM_TYPE),Linux)
+$(error "qemu-user can only be used on Linux hosts")
+endif
+
+HOST_QEMU_HOST_SYSTEM_VERSION_MAJOR = $(shell uname -r | cut -f1 -d'.')
+HOST_QEMU_HOST_SYSTEM_VERSION_MINOR = $(shell uname -r | cut -f2 -d'.')
+HOST_QEMU_TARGET_SYSTEM_VERSION_MAJOR = $(shell echo $(BR2_TOOLCHAIN_HEADERS_AT_LEAST) | cut -f1 -d'.')
+HOST_QEMU_TARGET_SYSTEM_VERSION_MINOR = $(shell echo $(BR2_TOOLCHAIN_HEADERS_AT_LEAST) | cut -f2 -d'.')
+HOST_QEMU_COMPARE_VERSION_MAJOR = $(shell test $(HOST_QEMU_HOST_SYSTEM_VERSION_MAJOR) -ge $(HOST_QEMU_TARGET_SYSTEM_VERSION_MAJOR) && echo OK)
+HOST_QEMU_COMPARE_VERSION_MINOR = $(shell test $(HOST_QEMU_HOST_SYSTEM_VERSION_MINOR) -ge $(HOST_QEMU_TARGET_SYSTEM_VERSION_MINOR) && echo OK)
+
+#
+# The principle of qemu-user is that it emulates the instructions of
+# the target architecture when running the binary, and then when this
+# binary does a system call, it converts this system call into a
+# system call on the host machine. This mechanism makes an assumption:
+# that the target binary will not do system calls that do not exist on
+# the host. This basically requires that the target binary should be
+# built with kernel headers that are older or the same as the kernel
+# version running on the host machine.
+#
+ifneq ($(HOST_QEMU_COMPARE_VERSION_MAJOR)$(HOST_QEMU_COMPARE_VERSION_MINOR),OKOK)
+$(error "Refusing to build qemu-user: target Linux version newer than host's.")
+endif
+endif
+
 define HOST_QEMU_CONFIGURE_CMDS
 	cd $(@D); $(HOST_CONFIGURE_OPTS) ./configure    \
 		--target-list="$(HOST_QEMU_TARGETS)"    \
-- 
2.1.0

[Buildroot] [PATCH 2/2] qemu: add host/target Linux version check

[Frank Hunleth]

Hi Thomas,

On Wed, Dec 10, 2014 at 5:11 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
>
> Raise an error if the host is using an older kernel than the target.
> Since qemu-user passes emulated system calls to the host kernel,
> this prevents usage of qemu-user in situations where those system
> calls will fail.
>
> This is based on an original patch from Frank Hunleth
> <fhunleth@troodon-software.com>, but completely rewritten in a
> different way:
>
>  * Instead of using shell based testing, we use pure make tests, which
>    allows to detect the problem not when host-qemu starts to build,
>    but at the very beginning of the entire Buildroot build.
>
>  * Instead of looking at $(STAGING_DIR)/usr/include/linux/version.h
>    (which requires having a dependency on the 'toolchain' package,
>    which is a bit unusual for a host package), we use the
>    BR2_TOOLCHAIN_HEADERS_AT_LEAST Config.in option which tells us the
>    version of the kernel headers used in the toolchain.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Thanks for revisiting and improving the old host-qemu patch. I'm using
your new changes to qemu.mk right now and everything seems to work as
expected. So,

Tested-by: Frank Hunleth <fhunleth@troodon-software.com>

Thanks,
Frank

> ---
>  package/qemu/qemu.mk | 28 ++++++++++++++++++++++++++++
>  1 file changed, 28 insertions(+)
>
> diff --git a/package/qemu/qemu.mk b/package/qemu/qemu.mk
> index 9cff5ed..6c5ddf3 100644
> --- a/package/qemu/qemu.mk
> +++ b/package/qemu/qemu.mk
> @@ -59,6 +59,34 @@ HOST_QEMU_ARCH = ppc
>  endif
>  HOST_QEMU_TARGETS = $(HOST_QEMU_ARCH)-linux-user
>
> +ifeq ($(BR2_PACKAGE_HOST_QEMU),y)
> +HOST_QEMU_HOST_SYSTEM_TYPE = $(shell uname -s)
> +ifneq ($(HOST_QEMU_HOST_SYSTEM_TYPE),Linux)
> +$(error "qemu-user can only be used on Linux hosts")
> +endif
> +
> +HOST_QEMU_HOST_SYSTEM_VERSION_MAJOR = $(shell uname -r | cut -f1 -d'.')
> +HOST_QEMU_HOST_SYSTEM_VERSION_MINOR = $(shell uname -r | cut -f2 -d'.')
> +HOST_QEMU_TARGET_SYSTEM_VERSION_MAJOR = $(shell echo $(BR2_TOOLCHAIN_HEADERS_AT_LEAST) | cut -f1 -d'.')
> +HOST_QEMU_TARGET_SYSTEM_VERSION_MINOR = $(shell echo $(BR2_TOOLCHAIN_HEADERS_AT_LEAST) | cut -f2 -d'.')
> +HOST_QEMU_COMPARE_VERSION_MAJOR = $(shell test $(HOST_QEMU_HOST_SYSTEM_VERSION_MAJOR) -ge $(HOST_QEMU_TARGET_SYSTEM_VERSION_MAJOR) && echo OK)
> +HOST_QEMU_COMPARE_VERSION_MINOR = $(shell test $(HOST_QEMU_HOST_SYSTEM_VERSION_MINOR) -ge $(HOST_QEMU_TARGET_SYSTEM_VERSION_MINOR) && echo OK)
> +
> +#
> +# The principle of qemu-user is that it emulates the instructions of
> +# the target architecture when running the binary, and then when this
> +# binary does a system call, it converts this system call into a
> +# system call on the host machine. This mechanism makes an assumption:
> +# that the target binary will not do system calls that do not exist on
> +# the host. This basically requires that the target binary should be
> +# built with kernel headers that are older or the same as the kernel
> +# version running on the host machine.
> +#
> +ifneq ($(HOST_QEMU_COMPARE_VERSION_MAJOR)$(HOST_QEMU_COMPARE_VERSION_MINOR),OKOK)
> +$(error "Refusing to build qemu-user: target Linux version newer than host's.")
> +endif
> +endif
> +
>  define HOST_QEMU_CONFIGURE_CMDS
>         cd $(@D); $(HOST_CONFIGURE_OPTS) ./configure    \
>                 --target-list="$(HOST_QEMU_TARGETS)"    \
> --
> 2.1.0
>


-- 
Frank Hunleth
Troodon Software LLC
Embedded Software Development
http://troodon-software.com/

[Buildroot] [PATCH 0/2] qemu: allow selection of host qemu-user from menuconfig

[Thomas Petazzoni]

Hello,

On Wed, 10 Dec 2014 23:11:57 +0100, Thomas Petazzoni wrote:

> Frank Hunleth (1):
>   qemu: add to host utilities menu
> 
> Thomas Petazzoni (1):
>   qemu: add host/target Linux version check

I've applied both patches.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 2/2] qemu: add host/target Linux version check

[Bernd Kuhls]

Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote
in
news:1418249519-15794-3-git-send-email-thomas.petazzoni at free-electrons.com:

> +ifneq
> ($(HOST_QEMU_COMPARE_VERSION_MAJOR)$(HOST_QEMU_COMPARE_VERSION_MINOR),OKO
> K) +$(error "Refusing to build qemu-user: target Linux version newer
> than host's.") +endif

Hi,

while testing an allyesconfig with "make source-check" this error was
raised. I had to comment this line in order to run the source-check. Is it
possible to run this check only during a build phase? 

Regards, Bernd

[Buildroot] [PATCH 2/2] qemu: add host/target Linux version check

[Thomas Petazzoni]

Dear Bernd Kuhls,

On Thu, 21 May 2015 09:51:43 +0200, Bernd Kuhls wrote:

> > +ifneq
> > ($(HOST_QEMU_COMPARE_VERSION_MAJOR)$(HOST_QEMU_COMPARE_VERSION_MINOR),OKO
> > K) +$(error "Refusing to build qemu-user: target Linux version newer
> > than host's.") +endif
> 
> while testing an allyesconfig with "make source-check" this error was
> raised. I had to comment this line in order to run the source-check. Is it
> possible to run this check only during a build phase? 

Yes, but that's not the proper solution. The proper solution is to
enclose the test with:

ifeq ($(BR_BUILDING),y)
...
endif

See how it's used in linux.mk or uboot.mk.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com