[Qemu-devel] [PULL 28/36] tests: Add ubuntu.i386 image

[Qemu-devel] [PULL 28/36] tests: Add ubuntu.i386 image

[Programmingkid]

Could a Darwin test be added? Both x86 and PowerPC versions would be great.

Re: [Qemu-devel] [PULL 28/36] tests: Add ubuntu.i386 image

[Fam Zheng]

On Fri, 09/22 08:10, Programmingkid wrote:
> Could a Darwin test be added? Both x86 and PowerPC versions would be great.

It's nice to cover macOS in our test, but to be honest I don't know how to do
it. If there isn't any copyright problem, and if there are instructions
available in wiki.qemu.org descirbing how to create such a guest step by step
like [1], we can add one.

[1]: https://wiki.qemu.org/Hosts/BSD

Fam

Re: [Qemu-devel] [PULL 28/36] tests: Add ubuntu.i386 image

[Daniel P. Berrange]

On Fri, Sep 22, 2017 at 08:44:05PM +0800, Fam Zheng wrote:
> On Fri, 09/22 08:10, Programmingkid wrote:
> > Could a Darwin test be added? Both x86 and PowerPC versions would be great.
> 
> It's nice to cover macOS in our test, but to be honest I don't know how to do
> it. If there isn't any copyright problem, and if there are instructions
> available in wiki.qemu.org descirbing how to create such a guest step by step
> like [1], we can add one.

AFAICT we can not just provide or host OS-X images and let individual
devs run the without it being a license violation, so best avoided.

IMHO it easier to just rely on Travis' OS-X builders for testing OS-X
platform builds, so that licensing compliance is their responsibility.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PULL 28/36] tests: Add ubuntu.i386 image

[Programmingkid]

> On Sep 22, 2017, at 9:18 AM, Daniel P. Berrange <berrange@redhat.com> wrote:
> 
> On Fri, Sep 22, 2017 at 08:44:05PM +0800, Fam Zheng wrote:
>> On Fri, 09/22 08:10, Programmingkid wrote:
>>> Could a Darwin test be added? Both x86 and PowerPC versions would be great.
>> 
>> It's nice to cover macOS in our test, but to be honest I don't know how to do
>> it. If there isn't any copyright problem, and if there are instructions
>> available in wiki.qemu.org descirbing how to create such a guest step by step
>> like [1], we can add one.
> 
> AFAICT we can not just provide or host OS-X images and let individual
> devs run the without it being a license violation, so best avoided.

Actually Darwin and Mac OS X are not the same. Darwin is this minimalistic environment and Mac OS X is built on top of that environment. There would be no license problems when using Darwin only.

> IMHO it easier to just rely on Travis' OS-X builders for testing OS-X
> platform builds, so that licensing compliance is their responsibility.

That might work for the x86 version of Mac OS X, but there doesn't appear to be any support for the PowerPC version of Mac OS X.

Re: [Qemu-devel] [PULL 28/36] tests: Add ubuntu.i386 image

[Daniel P. Berrange]

On Fri, Sep 22, 2017 at 04:26:40PM -0400, Programmingkid wrote:
> 
> > On Sep 22, 2017, at 9:18 AM, Daniel P. Berrange <berrange@redhat.com> wrote:
> > 
> > On Fri, Sep 22, 2017 at 08:44:05PM +0800, Fam Zheng wrote:
> >> On Fri, 09/22 08:10, Programmingkid wrote:
> >>> Could a Darwin test be added? Both x86 and PowerPC versions would be great.
> >> 
> >> It's nice to cover macOS in our test, but to be honest I don't know how to do
> >> it. If there isn't any copyright problem, and if there are instructions
> >> available in wiki.qemu.org descirbing how to create such a guest step by step
> >> like [1], we can add one.
> > 
> > AFAICT we can not just provide or host OS-X images and let individual
> > devs run the without it being a license violation, so best avoided.
> 
> Actually Darwin and Mac OS X are not the same. Darwin is this minimalistic
> environment and Mac OS X is built on top of that environment. There would
> be no license problems when using Darwin only.

I looked for images of Darwin but haven't been able to find any modern
ones. Wikipedia claims that Apple stopped releasing installable images
of Darwin so you have to build it from source :-(

> > IMHO it easier to just rely on Travis' OS-X builders for testing OS-X
> > platform builds, so that licensing compliance is their responsibility.
> 
> That might work for the x86 version of Mac OS X, but there doesn't appear
> to be any support for the PowerPC version of Mac OS X.

Yeah, no PPC support, but then that's essentially a dead platform, so it
is questionable whether we need to care about OS-X PPC support for QEMU.
The last PPC OS-X hardware shipped 2006.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PULL 28/36] tests: Add ubuntu.i386 image

[Programmingkid]

> On Sep 25, 2017, at 4:57 AM, Daniel P. Berrange <berrange@redhat.com> wrote:
> 
> On Fri, Sep 22, 2017 at 04:26:40PM -0400, Programmingkid wrote:
>> 
>>> On Sep 22, 2017, at 9:18 AM, Daniel P. Berrange <berrange@redhat.com> wrote:
>>> 
>>> On Fri, Sep 22, 2017 at 08:44:05PM +0800, Fam Zheng wrote:
>>>> On Fri, 09/22 08:10, Programmingkid wrote:
>>>>> Could a Darwin test be added? Both x86 and PowerPC versions would be great.
>>>> 
>>>> It's nice to cover macOS in our test, but to be honest I don't know how to do
>>>> it. If there isn't any copyright problem, and if there are instructions
>>>> available in wiki.qemu.org descirbing how to create such a guest step by step
>>>> like [1], we can add one.
>>> 
>>> AFAICT we can not just provide or host OS-X images and let individual
>>> devs run the without it being a license violation, so best avoided.
>> 
>> Actually Darwin and Mac OS X are not the same. Darwin is this minimalistic
>> environment and Mac OS X is built on top of that environment. There would
>> be no license problems when using Darwin only.
> 
> I looked for images of Darwin but haven't been able to find any modern
> ones. Wikipedia claims that Apple stopped releasing installable images
> of Darwin so you have to build it from source :-(

I did find this project: http://www.puredarwin.org, but they haven't released an image file yet.

> 
>>> IMHO it easier to just rely on Travis' OS-X builders for testing OS-X
>>> platform builds, so that licensing compliance is their responsibility.
>> 
>> That might work for the x86 version of Mac OS X, but there doesn't appear
>> to be any support for the PowerPC version of Mac OS X.
> 
> Yeah, no PPC support, but then that's essentially a dead platform, so it
> is questionable whether we need to care about OS-X PPC support for QEMU.
> The last PPC OS-X hardware shipped 2006.

Good point.

Re: [Qemu-devel] [PULL 28/36] tests: Add ubuntu.i386 image

[Programmingkid]

> On Sep 22, 2017, at 8:44 AM, Fam Zheng <famz@redhat.com> wrote:
> 
> On Fri, 09/22 08:10, Programmingkid wrote:
>> Could a Darwin test be added? Both x86 and PowerPC versions would be great.
> 
> It's nice to cover macOS in our test, but to be honest I don't know how to do
> it. If there isn't any copyright problem, and if there are instructions
> available in wiki.qemu.org descirbing how to create such a guest step by step
> like [1], we can add one.
> 
> [1]: https://wiki.qemu.org/Hosts/BSD
> 
> Fam

Mac OS X is built on top of Darwin. Darwin is completely open source and free. There are no copyright issues because it is released under the Apple Public Source License. 

I think I can make the directions available in the wiki soon. 

Re: [Qemu-devel] [PULL 28/36] tests: Add ubuntu.i386 image

[Gerd Hoffmann]

  Hi,

> Mac OS X is built on top of Darwin. Darwin is completely open source
> and free. There are no copyright issues because it is released under
> the Apple Public Source License. 

As far I know cocoa is not open source though.

So, can you actually build qemu on darwin?
And if so, it will probably be GUI-less?

cheers,
  Gerd

Re: [Qemu-devel] [PULL 28/36] tests: Add ubuntu.i386 image

[Daniel P. Berrange]

On Mon, Sep 25, 2017 at 09:31:15AM +0200, Gerd Hoffmann wrote:
>   Hi,
> 
> > Mac OS X is built on top of Darwin. Darwin is completely open source
> > and free. There are no copyright issues because it is released under
> > the Apple Public Source License. 
> 
> As far I know cocoa is not open source though.
> 
> So, can you actually build qemu on darwin?
> And if so, it will probably be GUI-less?

I guess even a GUI-less build would be beneficial to catch any portability
concerns in the of the code. The majority of people submitting patches to
cocoa GUI will have tested OS-X builds directly themselves. It is all the
patches to non-cocoa code that are greatest risk of regressions and least
likely to have been tested on OS-X by the patch author.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Qemu-devel] [PULL v5 00/36] Build and test automation patches

[Fam Zheng]

The following changes since commit 0a8066f0c068f1e318a1aacd7864fc00e455a37b:

  Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20170921' into staging (2017-09-21 17:42:27 +0100)

are available in the git repository at:

  git://github.com/famz/qemu.git tags/build-and-test-automation-pull-request

for you to fetch changes up to 4f6afe41f2ee3d8170eef7a9122d256588b707b2:

  docker: Drop 'set -e' from run script (2017-09-22 14:51:43 +0800)

----------------------------------------------------------------

Since v4:

- Drop three iffy -cflags patches from me (for gtk/vnc/curses).
- Revised VM test scripts after addressing comments.
- Add Eduardo's small tweak on MAINTAINERS.

----------------------------------------------------------------

Alex Bennée (4):
  docker: ensure NOUSER for travis images
  docker: docker.py make --no-cache skip checksum test
  docker: don't install device-tree-compiler build-deps in travis.docker
  docker: reduce noise when building travis.docker

Eduardo Habkost (1):
  MAINTAINERS: Fix subsystem name for "Build and test automation"

Fam Zheng (31):
  docker: Update ubuntu image
  docker: Enable features explicitly in test-full
  tests/docker: Clean up paths
  docker: Fix return code of build_qemu()
  docker: Add test_fail and prep_fail
  docker: Use unconfined security profile
  docker: Add nettle-devel to fedora image
  docker: Add test-block
  vl: Don't include vde header
  buildsys: Move vde libs to per object
  buildsys: Move sdl cflags/libs to per object
  buildsys: Move audio libs to per object
  buildsys: Move libcacard cflags/libs to per object
  buildsys: Move libusb cflags/libs to per object
  buildsys: Move usb redir cflags/libs to per object
  buildsys: Move brlapi libs to per object
  buildsys: Move rdma libs to per object
  gitignore: Ignore vm test images
  qemu.py: Add "wait()" method
  scripts: Add archive-source.sh
  tests: Add a test key pair
  tests: Add vm test lib
  tests: Add ubuntu.i386 image
  tests: Add FreeBSD image
  tests: Add NetBSD image
  tests: Add OpenBSD image
  Makefile: Add rules to run vm tests
  MAINTAINERS: Add tests/vm entry
  tests: Add README for vm tests
  docker: Use archive-source.py
  docker: Drop 'set -e' from run script

 .gitignore                             |   1 +
 MAINTAINERS                            |   2 +
 Makefile                               |   2 +
 audio/Makefile.objs                    |   6 +
 chardev/Makefile.objs                  |   1 +
 configure                              |  38 ++---
 hw/usb/Makefile.objs                   |  13 +-
 migration/Makefile.objs                |   1 +
 net/Makefile.objs                      |   2 +
 scripts/archive-source.sh              |  51 +++++++
 scripts/qemu.py                        |   7 +
 tests/.gitignore                       |   1 +
 tests/docker/Makefile.include          |  16 +-
 tests/docker/common.rc                 |  20 ++-
 tests/docker/docker.py                 |   3 +-
 tests/docker/dockerfiles/fedora.docker |   1 +
 tests/docker/dockerfiles/travis.docker |   6 +-
 tests/docker/dockerfiles/ubuntu.docker |  11 +-
 tests/docker/run                       |  18 +--
 tests/docker/test-block                |  21 +++
 tests/docker/test-full                 |  82 ++++++++++-
 tests/keys/README                      |   6 +
 tests/keys/id_rsa                      |  27 ++++
 tests/keys/id_rsa.pub                  |   1 +
 tests/vm/Makefile.include              |  42 ++++++
 tests/vm/README                        |  89 +++++++++++
 tests/vm/basevm.py                     | 262 +++++++++++++++++++++++++++++++++
 tests/vm/freebsd                       |  42 ++++++
 tests/vm/netbsd                        |  42 ++++++
 tests/vm/openbsd                       |  43 ++++++
 tests/vm/ubuntu.i386                   |  89 +++++++++++
 ui/Makefile.objs                       |   1 +
 vl.c                                   |   4 -
 33 files changed, 886 insertions(+), 65 deletions(-)
 create mode 100755 scripts/archive-source.sh
 create mode 100755 tests/docker/test-block
 create mode 100644 tests/keys/README
 create mode 100644 tests/keys/id_rsa
 create mode 100644 tests/keys/id_rsa.pub
 create mode 100644 tests/vm/Makefile.include
 create mode 100644 tests/vm/README
 create mode 100755 tests/vm/basevm.py
 create mode 100755 tests/vm/freebsd
 create mode 100755 tests/vm/netbsd
 create mode 100755 tests/vm/openbsd
 create mode 100755 tests/vm/ubuntu.i386

-- 
2.13.5

[Qemu-devel] [PULL 28/36] tests: Add ubuntu.i386 image

[Fam Zheng]

This adds a 32bit guest.

The official LTS cloud image is downloaded and initialized with
cloud-init.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/vm/ubuntu.i386 | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 89 insertions(+)
 create mode 100755 tests/vm/ubuntu.i386

diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
new file mode 100755
index 0000000000..fc319e0e6e
--- /dev/null
+++ b/tests/vm/ubuntu.i386
@@ -0,0 +1,89 @@
+#!/usr/bin/env python
+#
+# Ubuntu i386 image
+#
+# Copyright 2017 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later.  See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import sys
+import subprocess
+import basevm
+import time
+
+class UbuntuX86VM(basevm.BaseVM):
+    name = "ubuntu.i386"
+    BUILD_SCRIPT = """
+        set -e;
+        cd $(mktemp -d);
+        sudo chmod a+r /dev/vdb;
+        tar -xf /dev/vdb;
+        ./configure {configure_opts};
+        make -j{jobs};
+        make check;
+    """
+
+    def _gen_cloud_init_iso(self):
+        cidir = self._tmpdir
+        mdata = open(os.path.join(cidir, "meta-data"), "w")
+        mdata.writelines(["instance-id: ubuntu-vm-0\n",
+                          "local-hostname: ubuntu-guest\n"])
+        mdata.close()
+        udata = open(os.path.join(cidir, "user-data"), "w")
+        udata.writelines(["#cloud-config\n",
+                          "chpasswd:\n",
+                          "  list: |\n",
+                          "    root:%s\n" % self.ROOT_PASS,
+                          "    %s:%s\n" % (self.GUEST_USER, self.GUEST_PASS),
+                          "  expire: False\n",
+                          "users:\n",
+                          "  - name: %s\n" % self.GUEST_USER,
+                          "    sudo: ALL=(ALL) NOPASSWD:ALL\n",
+                          "    ssh-authorized-keys:\n",
+                          "    - %s\n" % basevm.SSH_PUB_KEY,
+                          "  - name: root\n",
+                          "    ssh-authorized-keys:\n",
+                          "    - %s\n" % basevm.SSH_PUB_KEY,
+                          "locale: en_US.UTF-8\n"])
+        udata.close()
+        subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
+                               "-volid", "cidata", "-joliet", "-rock",
+                               "user-data", "meta-data"],
+                               cwd=cidir,
+                               stdin=self._devnull, stdout=self._stdout,
+                               stderr=self._stdout)
+        return os.path.join(cidir, "cloud-init.iso")
+
+    def build_image(self, img):
+        cimg = self._download_with_cache("https://cloud-images.ubuntu.com/releases/16.04/release/ubuntu-16.04-server-cloudimg-i386-disk1.img")
+        img_tmp = img + ".tmp"
+        subprocess.check_call(["cp", "-f", cimg, img_tmp])
+        subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
+        self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
+        self.wait_ssh()
+        self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
+        self.ssh_root_check("apt-get update")
+        self.ssh_root_check("apt-get install -y cloud-initramfs-growroot")
+        # Don't check the status in case the guest hang up too quickly
+        self.ssh_root("sync && reboot")
+        time.sleep(5)
+        self.wait_ssh()
+        # The previous update sometimes doesn't survive a reboot, so do it again
+        self.ssh_root_check("apt-get update")
+        self.ssh_root_check("apt-get build-dep -y qemu")
+        self.ssh_root_check("apt-get install -y libfdt-dev")
+        self.ssh_root("poweroff")
+        self.wait()
+        if os.path.exists(img):
+            os.remove(img)
+        os.rename(img_tmp, img)
+        return 0
+
+if __name__ == "__main__":
+    sys.exit(basevm.main(UbuntuX86VM))
-- 
2.13.5