[RFC PATCH] of: Add a GitLab CI config file for unittests

[RFC PATCH] of: Add a GitLab CI config file for unittests

[Rob Herring]

This adds a GitLab CI config file running the DT unittest in a usermode
Linux build. The corresponding CI job can be found here:

https://gitlab.com/robherring/linux-dt-unittest/pipelines

This CI job can be duplicated by others by creating a kernel repo on a
GitLab instance and configuring GitLab CI to use
drivers/of/.gitlab-ci.yml config file.

Cc: Frank Rowand <frowand.list@gmail.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Rob Herring <robh@kernel.org>
---
 drivers/of/.gitlab-ci.yml | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
 create mode 100644 drivers/of/.gitlab-ci.yml

diff --git a/drivers/of/.gitlab-ci.yml b/drivers/of/.gitlab-ci.yml
new file mode 100644
index 000000000000..44a4824f5c33
--- /dev/null
+++ b/drivers/of/.gitlab-ci.yml
@@ -0,0 +1,18 @@
+# SPDX-License-Identifier: GPL-2.0+
+
+image: registry.gitlab.com/robherring/docker-images/ubuntu-kernel-build
+
+variables:
+  ARCH: um
+
+job-unittest:
+  artifacts:
+    paths:
+      - "*.log"
+
+  script:
+    - echo -e "CONFIG_OF=y\nCONFIG_OF_UNITTEST=y\nCONFIG_OF_OVERLAY=y" > kernel/configs/extra.config
+    - make defconfig extra.config
+    - make -s -j $(nproc) vmlinux | tee build.log
+    - TMP=/tmp ./vmlinux > boot.log || true
+    - grep -E '\#\#\# dt-test \#\#\# end of unittest - [0-9]* passed, 0 failed' boot.log
-- 
2.19.1

Re: [RFC PATCH] of: Add a GitLab CI config file for unittests

[Daniel Vetter]

On Fri, Nov 30, 2018 at 9:32 PM Rob Herring <robh@kernel.org> wrote:
>
> This adds a GitLab CI config file running the DT unittest in a usermode
> Linux build. The corresponding CI job can be found here:
>
> https://gitlab.com/robherring/linux-dt-unittest/pipelines
>
> This CI job can be duplicated by others by creating a kernel repo on a
> GitLab instance and configuring GitLab CI to use
> drivers/of/.gitlab-ci.yml config file.
>
> Cc: Frank Rowand <frowand.list@gmail.com>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Signed-off-by: Rob Herring <robh@kernel.org>

Adding dri-devel.

> ---
>  drivers/of/.gitlab-ci.yml | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
>  create mode 100644 drivers/of/.gitlab-ci.yml
>
> diff --git a/drivers/of/.gitlab-ci.yml b/drivers/of/.gitlab-ci.yml
> new file mode 100644
> index 000000000000..44a4824f5c33
> --- /dev/null
> +++ b/drivers/of/.gitlab-ci.yml
> @@ -0,0 +1,18 @@
> +# SPDX-License-Identifier: GPL-2.0+
> +
> +image: registry.gitlab.com/robherring/docker-images/ubuntu-kernel-build

I think it's better to include the docker recipe too. One because
shipping pre-built dockers is a license nightmare, second because
better hackability. For a full on example see:

https://gitlab.freedesktop.org/drm/igt-gpu-tools/blob/master/.gitlab-ci.yml

For the kernel I guess the question is where we should put all the
docker files. There's going to be some need for subsystem specific
tooling (e.g. we want something that has igt installed eventually for
anything drm), so maybe just keeping them in subsystem directories is
best. Was at least my plan with drivers/gpu.

Needs latest gitlab to make the build-on-demand stuff work, but
gitlab.com has that.
-Daniel
> +
> +variables:
> +  ARCH: um
> +
> +job-unittest:
> +  artifacts:
> +    paths:
> +      - "*.log"
> +
> +  script:
> +    - echo -e "CONFIG_OF=y\nCONFIG_OF_UNITTEST=y\nCONFIG_OF_OVERLAY=y" > kernel/configs/extra.config
> +    - make defconfig extra.config
> +    - make -s -j $(nproc) vmlinux | tee build.log
> +    - TMP=/tmp ./vmlinux > boot.log || true
> +    - grep -E '\#\#\# dt-test \#\#\# end of unittest - [0-9]* passed, 0 failed' boot.log
> --
> 2.19.1
>


-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [RFC PATCH] of: Add a GitLab CI config file for unittests

[Rob Herring]

On Fri, Nov 30, 2018 at 2:53 PM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> On Fri, Nov 30, 2018 at 9:32 PM Rob Herring <robh@kernel.org> wrote:
> >
> > This adds a GitLab CI config file running the DT unittest in a usermode
> > Linux build. The corresponding CI job can be found here:
> >
> > https://gitlab.com/robherring/linux-dt-unittest/pipelines
> >
> > This CI job can be duplicated by others by creating a kernel repo on a
> > GitLab instance and configuring GitLab CI to use
> > drivers/of/.gitlab-ci.yml config file.
> >
> > Cc: Frank Rowand <frowand.list@gmail.com>
> > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > Signed-off-by: Rob Herring <robh@kernel.org>
>
> Adding dri-devel.
>
> > ---
> >  drivers/of/.gitlab-ci.yml | 18 ++++++++++++++++++
> >  1 file changed, 18 insertions(+)
> >  create mode 100644 drivers/of/.gitlab-ci.yml
> >
> > diff --git a/drivers/of/.gitlab-ci.yml b/drivers/of/.gitlab-ci.yml
> > new file mode 100644
> > index 000000000000..44a4824f5c33
> > --- /dev/null
> > +++ b/drivers/of/.gitlab-ci.yml
> > @@ -0,0 +1,18 @@
> > +# SPDX-License-Identifier: GPL-2.0+
> > +
> > +image: registry.gitlab.com/robherring/docker-images/ubuntu-kernel-build
>
> I think it's better to include the docker recipe too. One because
> shipping pre-built dockers is a license nightmare, second because
> better hackability. For a full on example see:
>
> https://gitlab.freedesktop.org/drm/igt-gpu-tools/blob/master/.gitlab-ci.yml

Humm, but in the end you are still publishing the docker images either
way. It's just a difference of publishing across projects and whether
it's 2 jobs or 1 job with 2 stages.

I do tend to prefer the Travis-CI simplicity of just listing packages
to install. It seems like the same could be done with docker and some
magic on the server side. At least for the simple case of 'install
this list of packages'.

> For the kernel I guess the question is where we should put all the
> docker files. There's going to be some need for subsystem specific
> tooling (e.g. we want something that has igt installed eventually for
> anything drm), so maybe just keeping them in subsystem directories is
> best. Was at least my plan with drivers/gpu.

My plan (to the extent I have one) was to put the common parts into
the docker image and leave the small job specific things in the job
setup. Then I can use the same docker image across jobs. Maybe that's
a pointless goal in the docker world.

My dockerfile is simple enough, I could probably just create it within
the .gitlab-ci.yml.

Rob

Re: [RFC PATCH] of: Add a GitLab CI config file for unittests

[Daniel Vetter]

On Fri, Nov 30, 2018 at 11:22 PM Rob Herring <robh@kernel.org> wrote:
>
> On Fri, Nov 30, 2018 at 2:53 PM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >
> > On Fri, Nov 30, 2018 at 9:32 PM Rob Herring <robh@kernel.org> wrote:
> > >
> > > This adds a GitLab CI config file running the DT unittest in a usermode
> > > Linux build. The corresponding CI job can be found here:
> > >
> > > https://gitlab.com/robherring/linux-dt-unittest/pipelines
> > >
> > > This CI job can be duplicated by others by creating a kernel repo on a
> > > GitLab instance and configuring GitLab CI to use
> > > drivers/of/.gitlab-ci.yml config file.
> > >
> > > Cc: Frank Rowand <frowand.list@gmail.com>
> > > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > > Signed-off-by: Rob Herring <robh@kernel.org>
> >
> > Adding dri-devel.
> >
> > > ---
> > >  drivers/of/.gitlab-ci.yml | 18 ++++++++++++++++++
> > >  1 file changed, 18 insertions(+)
> > >  create mode 100644 drivers/of/.gitlab-ci.yml
> > >
> > > diff --git a/drivers/of/.gitlab-ci.yml b/drivers/of/.gitlab-ci.yml
> > > new file mode 100644
> > > index 000000000000..44a4824f5c33
> > > --- /dev/null
> > > +++ b/drivers/of/.gitlab-ci.yml
> > > @@ -0,0 +1,18 @@
> > > +# SPDX-License-Identifier: GPL-2.0+
> > > +
> > > +image: registry.gitlab.com/robherring/docker-images/ubuntu-kernel-build
> >
> > I think it's better to include the docker recipe too. One because
> > shipping pre-built dockers is a license nightmare, second because
> > better hackability. For a full on example see:
> >
> > https://gitlab.freedesktop.org/drm/igt-gpu-tools/blob/master/.gitlab-ci.yml
>
> Humm, but in the end you are still publishing the docker images either
> way. It's just a difference of publishing across projects and whether
> it's 2 jobs or 1 job with 2 stages.

Yeah, but it comes with sources for rebuilding the image :-)

> I do tend to prefer the Travis-CI simplicity of just listing packages
> to install. It seems like the same could be done with docker and some
> magic on the server side. At least for the simple case of 'install
> this list of packages'.

Yeah, gitlab is a bunch more explicit, and you need to do your caching
of the setup you want yourself.

> > For the kernel I guess the question is where we should put all the
> > docker files. There's going to be some need for subsystem specific
> > tooling (e.g. we want something that has igt installed eventually for
> > anything drm), so maybe just keeping them in subsystem directories is
> > best. Was at least my plan with drivers/gpu.
>
> My plan (to the extent I have one) was to put the common parts into
> the docker image and leave the small job specific things in the job
> setup. Then I can use the same docker image across jobs. Maybe that's
> a pointless goal in the docker world.

We do that too. The different docker images we have is for different
testing (new/old distros, and the cross build chain in userspace
conflicts, so need separate docker images for that). Aside from those
cases where we need different images because they're incompatible, we
do reuse them.

For the kernel there's much less of these userspace problems, so one
image should be good enough for all jobs.
-Daniel

> My dockerfile is simple enough, I could probably just create it within
> the .gitlab-ci.yml.
>
> Rob



-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel