Re: [PATCH v2 2/2] GitLab Gating CI: initial set of jobs, documentation and scripts

["Daniel P. Berrangé" <berrange@redhat.com>]

On Thu, Sep 03, 2020 at 08:18:17PM -0400, Cleber Rosa wrote:
> On Thu, Jul 09, 2020 at 01:28:27PM +0200, Andrea Bolognani wrote:
> > On Thu, 2020-07-09 at 11:30 +0100, Daniel P. Berrangé wrote:
> > > On Wed, Jul 08, 2020 at 10:46:57PM -0400, Cleber Rosa wrote:
> > > > +- name: Installation of basic packages to build QEMU
> > > > +  hosts: all
> > > > +  vars_files:
> > > > +    - vars.yml
> > > > +  tasks:
> > > > +    - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
> > > > +      apt:
> > > > +        update_cache: yes
> > > > +        # This matches the packages on tests/docker/Dockerfiles/ubuntu1804.docker
> > > 
> > > I'd be inclined to actually use docker on the custom runners.
> > > 
> > > eg. instead of having separate physical machines or VMs for each
> > > (distro, arch) pair, have a single host distro for the arch. Then
> > > use docker to provide the build environment against each distro.
> > > 
> > > IOW, a RHEL-8 aarch64 host, running docker for ubuntu18.04, fedora30
> > > etc.
> > > 
> > > That way we don't end up duplicating all these packages, and instead
> > > can use  tests/docker/Dockerfiles/ubuntu1804.docker.  This ensures
> > > that if a user needs to reproduce a build failure on their own local
> > > aarch64 machine, they can run docker and get the exact same build
> > > architecture.
> > > 
> > > It also has the benefit that we don't need to worry about how to
> > > setup gitlab runners for every distro we care about. We only need to
> > > do gitlab runner for the standard host distro, which spawns a pristine
> > > throwaway docker env.
> > > 
> > > I appreciate this is a big change from what you've done in this patch
> > > though, so don't consider this comment a blocker for initial merge.
> > > I think we should do this as the long term strategy though. Essentially
> > > for Linux builds, everything should always be container based.
> > 
> > Agreed. You should be able to set up a fairly minimal environment,
> > which consists of Docker, gitlab-runner and not much else, using a
> > long-term supported distro such as CentOS and then just schedule
> > whatever container build on it. No need to provision a new machine
> > every time a new Fedora release comes out, just create a container
> > image for it and add it to the mix.
> >
> 
> Hi Andrea,
> 
> There's nothing preventing this from happening, but limiting the
> runners to this configuration, prevents a lot more from happening.
> 
> > Additionally, the gitlab-runner Docker executor provides more
> > isolation than the shell executor, so running untrusted builds
> > becomes a more reasonable proposition - this is how the shared
> > runners on gitlab.com work - and you don't have to worry about your
> > jobs cleaning up properly after themselves nearly as much.
> >
> 
> I understand and agree to the the benefits of using the gitlab-runner
> Docker executor... until you want to run tests on non-Docker
> environments :).
> 
> Hopefully the explanation on my previous reply to Daniel will also
> serve for the points you raised here.  I would have loved to have
> worked on a more abstract, container only environments, but that
> proved to be unrealistic.

For Linux targets, it should be possible to have exclusively container
based testing environments. At worst you can run a privileged container
and expose arbitrary host resources to it, so you can do anything in
the container that you would otherwise do in bare metal. For non-Linux,
we should be able to satisfy our needs with VMs, and indeed VMs can
be used for Linux too if we want to emulate some specific hardware for
testing that we don't have accessible to containers on bare metal.
IOW, the testing environment can be entirely defined by the recipes
we have in tests/docker and tests/vm. Bare metal hosts are simply a
way to host the containers or vms.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|