[Qemu-devel] Hosted CI for FreeBSD - Cirrus CI

[Qemu-devel] Hosted CI for FreeBSD - Cirrus CI

[Ed Maste]

I'd very much like to have CI coverage for QEMU on FreeBSD, and as of
yesterday there's a hosted CI service that supports FreeBSD: Cirrus
CI. They also offer free service for OSS projects. I created a
.cirrus.yml (shown below) in a QEMU GitHub fork and successfully
executed a build test.

I can submit a patch to add the config file, but wanted to ask for
advice or comments from those who set up the existing Linux CI first.
Is there anything I should be aware of or should address? Although
this is a very basic build / test I believe it will prove quite
valuable in reducing the latency between a commit and finding an issue
on FreeBSD.

.cirrus.yml:
freebsd_instance:
  image: freebsd-12-0-rc3-amd64

env:
  CIRRUS_CLONE_DEPTH: 1

task:
  install_script: pkg install -y git glib gmake perl5 pixman pkgconf
  script:
    - mkdir build
    - cd build
    - ../configure || { cat config.log; exit 1; }
    - gmake
    - gmake check

Re: [Qemu-devel] Hosted CI for FreeBSD - Cirrus CI

[Kamil Rytarowski]

[-- Attachment #1: Type: text/plain, Size: 1208 bytes --]

On 05.12.2018 21:43, Ed Maste wrote:
> I'd very much like to have CI coverage for QEMU on FreeBSD, and as of
> yesterday there's a hosted CI service that supports FreeBSD: Cirrus
> CI. They also offer free service for OSS projects. I created a
> .cirrus.yml (shown below) in a QEMU GitHub fork and successfully
> executed a build test.
> 
> I can submit a patch to add the config file, but wanted to ask for
> advice or comments from those who set up the existing Linux CI first.
> Is there anything I should be aware of or should address? Although
> this is a very basic build / test I believe it will prove quite
> valuable in reducing the latency between a commit and finding an issue
> on FreeBSD.
> 
> .cirrus.yml:
> freebsd_instance:
>   image: freebsd-12-0-rc3-amd64
> 
> env:
>   CIRRUS_CLONE_DEPTH: 1
> 
> task:
>   install_script: pkg install -y git glib gmake perl5 pixman pkgconf
>   script:
>     - mkdir build
>     - cd build
>     - ../configure || { cat config.log; exit 1; }
>     - gmake
>     - gmake check
> 

There are already FreeBSD, OpenBSD and NetBSD test scripts in the qemu
project.

I'm not sure whether there is any integration with GitHub.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 850 bytes --]

Re: [Qemu-devel] Hosted CI for FreeBSD - Cirrus CI

[Ed Maste]

On Wed, 5 Dec 2018 at 15:59, Kamil Rytarowski <n54@gmx.com> wrote:
>
> There are already FreeBSD, OpenBSD and NetBSD test scripts in the qemu
> project.

I see scripts under tests/vm/ for FreeBSD, NetBSD, and OpenBSD, but
they're for testing BSD guests on QEMU, while I'm interested in
building and testing QEMU on a FreeBSD host. Is there something else
that I've missed?

Re: [Qemu-devel] Hosted CI for FreeBSD - Cirrus CI

[Kamil Rytarowski]

[-- Attachment #1: Type: text/plain, Size: 538 bytes --]

On 05.12.2018 22:58, Ed Maste wrote:
> On Wed, 5 Dec 2018 at 15:59, Kamil Rytarowski <n54@gmx.com> wrote:
>>
>> There are already FreeBSD, OpenBSD and NetBSD test scripts in the qemu
>> project.
> 
> I see scripts under tests/vm/ for FreeBSD, NetBSD, and OpenBSD, but
> they're for testing BSD guests on QEMU, while I'm interested in
> building and testing QEMU on a FreeBSD host. Is there something else
> that I've missed?
> 

These tests build qemu in guest and execute tests, so BSDs are tested as
an [emulated] host.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 850 bytes --]

Re: [Qemu-devel] Hosted CI for FreeBSD - Cirrus CI

[Daniel P. Berrangé]

On Wed, Dec 05, 2018 at 10:01:00PM +0100, Kamil Rytarowski wrote:
> On 05.12.2018 21:43, Ed Maste wrote:
> > I'd very much like to have CI coverage for QEMU on FreeBSD, and as of
> > yesterday there's a hosted CI service that supports FreeBSD: Cirrus
> > CI. They also offer free service for OSS projects. I created a
> > .cirrus.yml (shown below) in a QEMU GitHub fork and successfully
> > executed a build test.
> > 
> > I can submit a patch to add the config file, but wanted to ask for
> > advice or comments from those who set up the existing Linux CI first.
> > Is there anything I should be aware of or should address? Although
> > this is a very basic build / test I believe it will prove quite
> > valuable in reducing the latency between a commit and finding an issue
> > on FreeBSD.
> > 
> > .cirrus.yml:
> > freebsd_instance:
> >   image: freebsd-12-0-rc3-amd64
> > 
> > env:
> >   CIRRUS_CLONE_DEPTH: 1
> > 
> > task:
> >   install_script: pkg install -y git glib gmake perl5 pixman pkgconf

I'd encourage you to loook at installing as many deps as possible to
maximise the coverage. eg gnutls, nettle, spice, and anything else that
is relevant

> >   script:
> >     - mkdir build
> >     - cd build
> >     - ../configure || { cat config.log; exit 1; }
> >     - gmake
> >     - gmake check

Are the Cirrus CI environments SMP ? If so adding a -j flag would
give much improved build times.

> > 
> 
> There are already FreeBSD, OpenBSD and NetBSD test scripts in the qemu
> project.
> 
> I'm not sure whether there is any integration with GitHub.

The trouble is that these are all VMs, and (AFAIK) none of the free CI
services let you spin up arbitrary VM images with KVM, only allow for
custom docker images.

So unless there's some way for us actually run the VM images on a public
CI that we already use, I think it is reasonable to use Cirrus CI for
FeeBSD testing.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] Hosted CI for FreeBSD - Cirrus CI

[Ed Maste]

On Thu, 6 Dec 2018 at 05:23, Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> I'd encourage you to loook at installing as many deps as possible to
> maximise the coverage. eg gnutls, nettle, spice, and anything else that
> is relevant

I've now added: bison curl cyrus-sasl fontconfig freetype2 gnutls
nettle png usbredir
I think this should give a decent starting point and can post a patch
to add .cirrus.yml, then someone in the GitHub qemu organization would
have to add Cirrus CI for it to start running when commits happen.

> Are the Cirrus CI environments SMP ? If so adding a -j flag would
> give much improved build times.

For FreeBSD Cirrus CI spins up a VM (it doesn't run the test in a
jail/container) and supports a configurable CPU count. I changed it to
specify 8 vCPUs and ran a -j8 build/test in about 15 minutes - here's
a link to the run: https://cirrus-ci.com/task/6214818623127552

Re: [Qemu-devel] Hosted CI for FreeBSD - Cirrus CI

[Daniel P. Berrangé]

On Wed, Dec 05, 2018 at 03:43:07PM -0500, Ed Maste wrote:
> I'd very much like to have CI coverage for QEMU on FreeBSD, and as of
> yesterday there's a hosted CI service that supports FreeBSD: Cirrus
> CI. They also offer free service for OSS projects. I created a
> .cirrus.yml (shown below) in a QEMU GitHub fork and successfully
> executed a build test.

Looking at this more generally I see they support Linux containers,
native Windows containers, macOS and FreeBSD. IOW, they offer more
platforms than our current Travis setup does and aren't stuck on an
amcient Ubuntu version.

Thus I wonder if we could in fact switch all our travis setup over to
use Cirrus CI, and stop using Travis entirely.

One key thing I can't find out is what, if any, limitations they put
on resources used by the free service for OSS projects.  Does anyone
know if they limit the number of concurrent build jobs like Travis
does ? Do they put a fixed time limit on execution of a single job ?


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] Hosted CI for FreeBSD - Cirrus CI

[Ed Maste]

On Fri, 7 Dec 2018 at 04:54, Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> Looking at this more generally I see they support Linux containers,
> native Windows containers, macOS and FreeBSD. IOW, they offer more
> platforms than our current Travis setup does and aren't stuck on an
> amcient Ubuntu version.

Indeed - having a hosted CI service to build on FreeBSD is something
I've wanted for a long time; so much better if one service can support
most or all platforms of interest.

> One key thing I can't find out is what, if any, limitations they put
> on resources used by the free service for OSS projects.  Does anyone
> know if they limit the number of concurrent build jobs like Travis
> does ? Do they put a fixed time limit on execution of a single job ?

I'm not sure what concurrency limits they have. I have empirically
discovered a 1hr limit for a job to complete, but I believe it can be
increased via the config file.

Re: [Qemu-devel] Hosted CI for FreeBSD - Cirrus CI

[Ed Maste]

On Fri, 14 Dec 2018 at 09:57, Ed Maste <emaste@freebsd.org> wrote:
>
> Indeed - having a hosted CI service to build on FreeBSD is something
> I've wanted for a long time; so much better if one service can support
> most or all platforms of interest.

My most recent test build with cirrus-ci
(https://cirrus-ci.com/github/emaste/qemu) failed with:
/usr/bin/ld: error: undefined symbol: fdt_check_full
>>> referenced by spapr_hcall.c:1797 (/tmp/cirrus-ci-build/hw/ppc/spapr_hcall.c:1797)
>>>               hw/ppc/spapr_hcall.o:(h_update_dt)
c++: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[1]: *** [Makefile:196: qemu-system-ppc64] Error 1
gmake: *** [Makefile:428: subdir-ppc64-softmmu] Error 2
gmake: *** Waiting for unfinished jobs....

It looks like this was introduced between
9b2e891ec5ccdb4a7d583b77988848282606fdea (works) and
44ba6010635641a538c9b9b1f377dfa288751906 (fails). The .cirrus.yml file
in my branch is at least good enough to catch this sort of thing, so
I'll send the patch over (while trying to look into this).

Re: [Qemu-devel] Hosted CI for FreeBSD - Cirrus CI

[Peter Maydell]

On Tue, 15 Jan 2019 at 18:34, Ed Maste <emaste@freebsd.org> wrote:
> My most recent test build with cirrus-ci
> (https://cirrus-ci.com/github/emaste/qemu) failed with:
> /usr/bin/ld: error: undefined symbol: fdt_check_full
> >>> referenced by spapr_hcall.c:1797 (/tmp/cirrus-ci-build/hw/ppc/spapr_hcall.c:1797)
> >>>               hw/ppc/spapr_hcall.o:(h_update_dt)
> c++: error: linker command failed with exit code 1 (use -v to see invocation)
> gmake[1]: *** [Makefile:196: qemu-system-ppc64] Error 1
> gmake: *** [Makefile:428: subdir-ppc64-softmmu] Error 2
> gmake: *** Waiting for unfinished jobs....
>
> It looks like this was introduced between
> 9b2e891ec5ccdb4a7d583b77988848282606fdea (works) and
> 44ba6010635641a538c9b9b1f377dfa288751906 (fails). The .cirrus.yml file
> in my branch is at least good enough to catch this sort of thing, so
> I'll send the patch over (while trying to look into this).

Commit fea35ca4b8e68b4fa updates our configure test to require
a libfdt version that has fdt_check_full() in it, so in theory
what is supposed to happen is that we will use the local
version in dtc/ if the system libfdt is not new enough.
Has that process gone wrong? Is the build using the system
libfdt or the one we provide in the submodule ?

thanks
-- PMM

Re: [Qemu-devel] Hosted CI for FreeBSD - Cirrus CI

[Ed Maste]

On Tue, 15 Jan 2019 at 13:47, Peter Maydell <peter.maydell@linaro.org> wrote:
> >
> > It looks like this was introduced between
> > 9b2e891ec5ccdb4a7d583b77988848282606fdea (works) and
> > 44ba6010635641a538c9b9b1f377dfa288751906 (fails). The .cirrus.yml file
> > in my branch is at least good enough to catch this sort of thing, so
> > I'll send the patch over (while trying to look into this).
>
> Commit fea35ca4b8e68b4fa updates our configure test to require
> a libfdt version that has fdt_check_full() in it, so in theory
> what is supposed to happen is that we will use the local
> version in dtc/ if the system libfdt is not new enough.
> Has that process gone wrong? Is the build using the system
> libfdt or the one we provide in the submodule ?

>From the log[1] it looks like it's building the libfdt from dtc/.
Unfortunately I'm unable to dig in deeper into the hosted build (I
have only the logs), and my local build attempts are currently failing
due to unrelated issues. I'll investigate further as soon as I can.

[1] https://cirrus-ci.com/task/6254685977575424

Re: [Qemu-devel] Hosted CI for FreeBSD - Cirrus CI

[Ed Maste]

On Tue, 15 Jan 2019 at 13:58, Ed Maste <emaste@freebsd.org> wrote:
>
> From the log[1] it looks like it's building the libfdt from dtc/.
> Unfortunately I'm unable to dig in deeper into the hosted build (I
> have only the logs), and my local build attempts are currently failing
> due to unrelated issues. I'll investigate further as soon as I can.
>
> [1] https://cirrus-ci.com/task/6254685977575424

Somehow I had an unintended change in my branch where I added
.cirrus.yml that reverted the dtc submodule to the previous version.
With that fixed the build and test is successful on FreeBSD 12:
https://cirrus-ci.com/task/5528406336733184