[LTP] [PATCH] Add simple Containerfile

[LTP] [PATCH] Add simple Containerfile

[Richard Palethorpe via ltp]

This adds a regular container (or Docker) file which builds LTP from
source. This is initially intended for testing the LTP itself.

The resulting container has just the LTP installation and runtime
dependencies. However it is still quite large, probably due to debug
symbols.

The container can be built with a command like:

podman build -t tumbleweed/ltp \
       --build-arg PREFIX=registry.opensuse.org/opensuse/ \
       --build-arg DISTRO_NAME=tumbleweed \
       --build-arg DISTRO_RELEASE=20230925 .

Or just

podman build -t alpine/ltp .

It contains Kirk in /opt/kirk. So

cd /opt/kirk && ./kirk -f ltp -r syscalls

will run some tests.

Note a much smaller container can be found at:
https://registry.opensuse.org/cgi-bin/cooverview?srch_term=project%3D%5Ebenchmark+container%3D.*
This is created with SUSE's build system which does not use container files

Signed-off-by: Richard Palethorpe <rpalethorpe@suse.com>
Cc: Petr Vorel <pvorel@suse.cz>
Cc: Marius Kittler <mkittler@suse.de>
---

RFC comments:
    * Add git clean -fdX which should remove any build artifacts
      this is different from the suggestion of just doing a check. I just
      found it easier to remove the build files.
    * Added seperate alpine and tumbleweed runtime scripts. Again it's
      different from the suggestion just because it's easier to add
      seperate scripts than adding a switch
    * Obviously a number of distros are missing runtime scripts. They can
      be added when someone is motivated to do so.

 .dockerignore            |  1 +
 Containerfile            | 32 ++++++++++++++++++++++++++++++++
 ci/alpine-runtime.sh     | 16 ++++++++++++++++
 ci/tumbleweed-runtime.sh | 13 +++++++++++++
 4 files changed, 62 insertions(+)
 create mode 100644 .dockerignore
 create mode 100644 Containerfile
 create mode 100755 ci/alpine-runtime.sh
 create mode 100755 ci/tumbleweed-runtime.sh

diff --git a/.dockerignore b/.dockerignore
new file mode 100644
index 000000000..bbcd7072f
--- /dev/null
+++ b/.dockerignore
@@ -0,0 +1 @@
+Containerfile
diff --git a/Containerfile b/Containerfile
new file mode 100644
index 000000000..2f8192c3b
--- /dev/null
+++ b/Containerfile
@@ -0,0 +1,32 @@
+ARG PREFIX=docker.io/
+ARG DISTRO_NAME=alpine
+ARG DISTRO_RELEASE=3.18
+
+FROM $PREFIX$DISTRO_NAME:$DISTRO_RELEASE AS build
+ARG LTPROOT=/opt/ltp
+ARG DISTRO_NAME=alpine
+ARG DISTRO_RELEASE=3.18
+
+RUN mkdir /build
+WORKDIR /build
+COPY . /build
+RUN ./ci/${DISTRO_NAME}.sh
+RUN git clean -fdX
+RUN ./build.sh -p $LTPROOT -i
+
+FROM $PREFIX$DISTRO_NAME:$DISTRO_RELEASE
+ARG LTPROOT=/opt/ltp
+ARG KIRKROOT=/opt/kirk
+ARG DISTRO_NAME=alpine
+
+COPY --from=build /build/ci/${DISTRO_NAME}-runtime.sh $LTPROOT/runtime-deps.sh
+RUN $LTPROOT/runtime-deps.sh
+
+COPY --from=build $LTPROOT $LTPROOT
+ENV LTPROOT=$LTPROOT
+ENV PATH=$LTPROOT/testcases/bin:$LTPROOT/bin:$PATH
+
+RUN mkdir -p $KIRKROOT
+COPY --from=build /build/tools/kirk $KIRKROOT
+
+USER ltp
diff --git a/ci/alpine-runtime.sh b/ci/alpine-runtime.sh
new file mode 100755
index 000000000..e4941f329
--- /dev/null
+++ b/ci/alpine-runtime.sh
@@ -0,0 +1,16 @@
+#!/bin/sh -eux
+
+apk add \
+        acl \
+        keyutils \
+        libaio \
+        libacl \
+        libcap \
+        libselinux \
+        libsepol \
+        libtirpc \
+        numactl \
+        openssl \
+        py3-msgpack
+
+adduser -D -g "Unprivileged LTP user" ltp
diff --git a/ci/tumbleweed-runtime.sh b/ci/tumbleweed-runtime.sh
new file mode 100755
index 000000000..5207988e7
--- /dev/null
+++ b/ci/tumbleweed-runtime.sh
@@ -0,0 +1,13 @@
+#!/bin/sh -eux
+
+zyp="zypper --non-interactive install --force-resolution --no-recommends"
+
+$zyp \
+	iproute2 \
+	keyutils \
+	libaio1 \
+	libmnl0 \
+	libnuma1 \
+	libtirpc3
+
+useradd ltp
-- 
2.40.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] Add simple Containerfile

[Petr Vorel]

Hi Richie,

I suppose this is v2 of "[RFC] Add simple Alpine container" [1], thus I set [1]
in patchwork as superseded.

This LGTM, thanks! Minor comments below.

Reviewed-by: Petr Vorel <pvorel@suse.cz>

> This adds a regular container (or Docker) file which builds LTP from
> source. This is initially intended for testing the LTP itself.

> The resulting container has just the LTP installation and runtime
> dependencies. However it is still quite large, probably due to debug
> symbols.

> The container can be built with a command like:

> podman build -t tumbleweed/ltp \
>        --build-arg PREFIX=registry.opensuse.org/opensuse/ \
>        --build-arg DISTRO_NAME=tumbleweed \
>        --build-arg DISTRO_RELEASE=20230925 .

> Or just

> podman build -t alpine/ltp .

> It contains Kirk in /opt/kirk. So

> cd /opt/kirk && ./kirk -f ltp -r syscalls

> will run some tests.

> Note a much smaller container can be found at:
> https://registry.opensuse.org/cgi-bin/cooverview?srch_term=project%3D%5Ebenchmark+container%3D.*
> This is created with SUSE's build system which does not use container files

> Signed-off-by: Richard Palethorpe <rpalethorpe@suse.com>
> Cc: Petr Vorel <pvorel@suse.cz>
> Cc: Marius Kittler <mkittler@suse.de>
> ---

> RFC comments:
>     * Add git clean -fdX which should remove any build artifacts
>       this is different from the suggestion of just doing a check. I just
>       found it easier to remove the build files.
FYI what we do in release scripts, is to do a local clone to a different
directory [2]:
git clone ltp ltp-full-YYYYMMDD

Not sure what is faster.

>     * Added seperate alpine and tumbleweed runtime scripts. Again it's
>       different from the suggestion just because it's easier to add
>       seperate scripts than adding a switch
+1

But maybe put it into container directory, because it's not used in GitHub CI?

>     * Obviously a number of distros are missing runtime scripts. They can
>       be added when someone is motivated to do so.

+1

> diff --git a/.dockerignore b/.dockerignore
> new file mode 100644
> index 000000000..bbcd7072f
> --- /dev/null
> +++ b/.dockerignore
> @@ -0,0 +1 @@
> +Containerfile
> diff --git a/Containerfile b/Containerfile
> new file mode 100644
> index 000000000..2f8192c3b
> --- /dev/null
> +++ b/Containerfile
> @@ -0,0 +1,32 @@
> +ARG PREFIX=docker.io/
> +ARG DISTRO_NAME=alpine
> +ARG DISTRO_RELEASE=3.18
> +
> +FROM $PREFIX$DISTRO_NAME:$DISTRO_RELEASE AS build
> +ARG LTPROOT=/opt/ltp
> +ARG DISTRO_NAME=alpine
> +ARG DISTRO_RELEASE=3.18
> +
> +RUN mkdir /build
> +WORKDIR /build
> +COPY . /build
> +RUN ./ci/${DISTRO_NAME}.sh
> +RUN git clean -fdX
> +RUN ./build.sh -p $LTPROOT -i
> +
> +FROM $PREFIX$DISTRO_NAME:$DISTRO_RELEASE
> +ARG LTPROOT=/opt/ltp
> +ARG KIRKROOT=/opt/kirk
> +ARG DISTRO_NAME=alpine
> +
> +COPY --from=build /build/ci/${DISTRO_NAME}-runtime.sh $LTPROOT/runtime-deps.sh
> +RUN $LTPROOT/runtime-deps.sh
> +
> +COPY --from=build $LTPROOT $LTPROOT
> +ENV LTPROOT=$LTPROOT
> +ENV PATH=$LTPROOT/testcases/bin:$LTPROOT/bin:$PATH
> +
> +RUN mkdir -p $KIRKROOT
> +COPY --from=build /build/tools/kirk $KIRKROOT
> +
> +USER ltp
> diff --git a/ci/alpine-runtime.sh b/ci/alpine-runtime.sh
> new file mode 100755
> index 000000000..e4941f329
> --- /dev/null
> +++ b/ci/alpine-runtime.sh
> @@ -0,0 +1,16 @@
> +#!/bin/sh -eux

nit: out of curiosity, why -u (fail unset variables and parameters)?
> +
> +apk add \
> +        acl \
> +        keyutils \
> +        libaio \
> +        libacl \
> +        libcap \
> +        libselinux \
> +        libsepol \
> +        libtirpc \
> +        numactl \
> +        openssl \
> +        py3-msgpack

I'd add more runtime packages (at least for syscalls).  If I manage next week to
test this I might ask for more packages, but let's start with this

Kind regards,
Petr

[1] https://patchwork.ozlabs.org/project/ltp/patch/20230926090101.7565-1-rpalethorpe@suse.com/
[2] https://patchwork.ozlabs.org/project/ltp/patch/20230927202121.300325-6-pvorel@suse.cz/

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] Add simple Containerfile

[Richard Palethorpe]

Petr Vorel <pvorel@suse.cz> writes:

> Hi Richie,
>
> I suppose this is v2 of "[RFC] Add simple Alpine container" [1], thus I set [1]
> in patchwork as superseded.
>
> This LGTM, thanks! Minor comments below.
>
> Reviewed-by: Petr Vorel <pvorel@suse.cz>
>
>> This adds a regular container (or Docker) file which builds LTP from
>> source. This is initially intended for testing the LTP itself.
>
>> The resulting container has just the LTP installation and runtime
>> dependencies. However it is still quite large, probably due to debug
>> symbols.
>
>> The container can be built with a command like:
>
>> podman build -t tumbleweed/ltp \
>>        --build-arg PREFIX=registry.opensuse.org/opensuse/ \
>>        --build-arg DISTRO_NAME=tumbleweed \
>>        --build-arg DISTRO_RELEASE=20230925 .
>
>> Or just
>
>> podman build -t alpine/ltp .
>
>> It contains Kirk in /opt/kirk. So
>
>> cd /opt/kirk && ./kirk -f ltp -r syscalls
>
>> will run some tests.
>
>> Note a much smaller container can be found at:
>> https://registry.opensuse.org/cgi-bin/cooverview?srch_term=project%3D%5Ebenchmark+container%3D.*
>> This is created with SUSE's build system which does not use container files
>
>> Signed-off-by: Richard Palethorpe <rpalethorpe@suse.com>
>> Cc: Petr Vorel <pvorel@suse.cz>
>> Cc: Marius Kittler <mkittler@suse.de>
>> ---
>
>> RFC comments:
>>     * Add git clean -fdX which should remove any build artifacts
>>       this is different from the suggestion of just doing a check. I just
>>       found it easier to remove the build files.
> FYI what we do in release scripts, is to do a local clone to a different
> directory [2]:
> git clone ltp ltp-full-YYYYMMDD
>
> Not sure what is faster.

I guess that is something a script could also do then 'git clean'
becomes a null op. git clean -X will only remove untracked files so
pending changes should get picked up. Which is probably what people want
during development. Doing a fresh checkout is probably more like a hard
reset and clean.

AFAICT git clean is very quick, far faster than 'make distclean'.

>
>>     * Added seperate alpine and tumbleweed runtime scripts. Again it's
>>       different from the suggestion just because it's easier to add
>>       seperate scripts than adding a switch
> +1
>
> But maybe put it into container directory, because it's not used in
> GitHub CI?

I was thinking it could be used in CI. All we need is a CI that runs VMs
and we can do some testing. (e.g. srchut).

>
>>     * Obviously a number of distros are missing runtime scripts. They can
>>       be added when someone is motivated to do so.
>
> +1
>
>> diff --git a/.dockerignore b/.dockerignore
>> new file mode 100644
>> index 000000000..bbcd7072f
>> --- /dev/null
>> +++ b/.dockerignore
>> @@ -0,0 +1 @@
>> +Containerfile
>> diff --git a/Containerfile b/Containerfile
>> new file mode 100644
>> index 000000000..2f8192c3b
>> --- /dev/null
>> +++ b/Containerfile
>> @@ -0,0 +1,32 @@
>> +ARG PREFIX=docker.io/
>> +ARG DISTRO_NAME=alpine
>> +ARG DISTRO_RELEASE=3.18
>> +
>> +FROM $PREFIX$DISTRO_NAME:$DISTRO_RELEASE AS build
>> +ARG LTPROOT=/opt/ltp
>> +ARG DISTRO_NAME=alpine
>> +ARG DISTRO_RELEASE=3.18
>> +
>> +RUN mkdir /build
>> +WORKDIR /build
>> +COPY . /build
>> +RUN ./ci/${DISTRO_NAME}.sh
>> +RUN git clean -fdX
>> +RUN ./build.sh -p $LTPROOT -i
>> +
>> +FROM $PREFIX$DISTRO_NAME:$DISTRO_RELEASE
>> +ARG LTPROOT=/opt/ltp
>> +ARG KIRKROOT=/opt/kirk
>> +ARG DISTRO_NAME=alpine
>> +
>> +COPY --from=build /build/ci/${DISTRO_NAME}-runtime.sh $LTPROOT/runtime-deps.sh
>> +RUN $LTPROOT/runtime-deps.sh
>> +
>> +COPY --from=build $LTPROOT $LTPROOT
>> +ENV LTPROOT=$LTPROOT
>> +ENV PATH=$LTPROOT/testcases/bin:$LTPROOT/bin:$PATH
>> +
>> +RUN mkdir -p $KIRKROOT
>> +COPY --from=build /build/tools/kirk $KIRKROOT
>> +
>> +USER ltp
>> diff --git a/ci/alpine-runtime.sh b/ci/alpine-runtime.sh
>> new file mode 100755
>> index 000000000..e4941f329
>> --- /dev/null
>> +++ b/ci/alpine-runtime.sh
>> @@ -0,0 +1,16 @@
>> +#!/bin/sh -eux
>
> nit: out of curiosity, why -u (fail unset variables and parameters)?

I find it finds errors in shell scripts or when using them. E.g. typo's
in env variable names. I just include it wherever possible.

>> +
>> +apk add \
>> +        acl \
>> +        keyutils \
>> +        libaio \
>> +        libacl \
>> +        libcap \
>> +        libselinux \
>> +        libsepol \
>> +        libtirpc \
>> +        numactl \
>> +        openssl \
>> +        py3-msgpack
>
> I'd add more runtime packages (at least for syscalls).  If I manage next week to
> test this I might ask for more packages, but let's start with this

+1

>
> Kind regards,
> Petr
>
> [1] https://patchwork.ozlabs.org/project/ltp/patch/20230926090101.7565-1-rpalethorpe@suse.com/
> [2] https://patchwork.ozlabs.org/project/ltp/patch/20230927202121.300325-6-pvorel@suse.cz/


-- 
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] Add simple Containerfile

[Petr Vorel]

Hi Richie,

> >> RFC comments:
> >>     * Add git clean -fdX which should remove any build artifacts
> >>       this is different from the suggestion of just doing a check. I just
> >>       found it easier to remove the build files.
> > FYI what we do in release scripts, is to do a local clone to a different
> > directory [2]:
> > git clone ltp ltp-full-YYYYMMDD

> > Not sure what is faster.

> I guess that is something a script could also do then 'git clean'
> becomes a null op. git clean -X will only remove untracked files so
> pending changes should get picked up. Which is probably what people want
"remove untracked files" - if you develop a new test, forget to add it with 'git
add' and run the container, you will get disappointed :).

> during development. Doing a fresh checkout is probably more like a hard
> reset and clean.

The benefit is that you have not only a clean git repo for the container,
but also not touching your working copy directory. But unless nobody else
raise any concern, I'm ok with your current proposal.

> AFAICT git clean is very quick, far faster than 'make distclean'.


> >>     * Added seperate alpine and tumbleweed runtime scripts. Again it's
> >>       different from the suggestion just because it's easier to add
> >>       seperate scripts than adding a switch
> > +1

> > But maybe put it into container directory, because it's not used in
> > GitHub CI?

> I was thinking it could be used in CI. All we need is a CI that runs VMs
> and we can do some testing. (e.g. srchut).

Makes sense. Also, having scripts on two directories can lead to confusion,
let's keep it in ci directory.

...
> >> +#!/bin/sh -eux
> > nit: out of curiosity, why -u (fail unset variables and parameters)?

> I find it finds errors in shell scripts or when using them. E.g. typo's
> in env variable names. I just include it wherever possible.

+1, maybe we should add it to the current ci scripts as well (+ use params
instead of setting it via set command, it should work in dash and busybox shell
as well).

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] Add simple Containerfile

[Richard Palethorpe]

Hello,

Petr Vorel <pvorel@suse.cz> writes:

> Hi Richie,
>
>> >> RFC comments:
>> >>     * Add git clean -fdX which should remove any build artifacts
>> >>       this is different from the suggestion of just doing a check. I just
>> >>       found it easier to remove the build files.
>> > FYI what we do in release scripts, is to do a local clone to a different
>> > directory [2]:
>> > git clone ltp ltp-full-YYYYMMDD
>
>> > Not sure what is faster.
>
>> I guess that is something a script could also do then 'git clean'
>> becomes a null op. git clean -X will only remove untracked files so
>> pending changes should get picked up. Which is probably what people want
> "remove untracked files" - if you develop a new test, forget to add it with 'git
> add' and run the container, you will get disappointed :).

Possibly it is only files that are not tracked due to .gitignore. At
least that is how I interpret the Git help. Either way it's not such a
concern due to the below.

>
>> during development. Doing a fresh checkout is probably more like a hard
>> reset and clean.
>
> The benefit is that you have not only a clean git repo for the container,
> but also not touching your working copy directory. But unless nobody else
> raise any concern, I'm ok with your current proposal.

Ah I think I see your concern now. It does not touch the directory
outside the container. It copies everything in first (IDK if it actually
copies the data if you are doing a local build).

Note that I tried using a .dockerignore to stop build artifacts being
copied, but it's not as rich as the .gitignore(s) we have sprinkled
throughout the LTP.

>
>> AFAICT git clean is very quick, far faster than 'make distclean'.
>
>
>> >>     * Added seperate alpine and tumbleweed runtime scripts. Again it's
>> >>       different from the suggestion just because it's easier to add
>> >>       seperate scripts than adding a switch
>> > +1
>
>> > But maybe put it into container directory, because it's not used in
>> > GitHub CI?
>
>> I was thinking it could be used in CI. All we need is a CI that runs VMs
>> and we can do some testing. (e.g. srchut).
>
> Makes sense. Also, having scripts on two directories can lead to confusion,
> let's keep it in ci directory.
>
> ...
>> >> +#!/bin/sh -eux
>> > nit: out of curiosity, why -u (fail unset variables and parameters)?
>
>> I find it finds errors in shell scripts or when using them. E.g. typo's
>> in env variable names. I just include it wherever possible.
>
> +1, maybe we should add it to the current ci scripts as well (+ use params
> instead of setting it via set command, it should work in dash and busybox shell
> as well).

Yeah, I think it should be the default.

>
> Kind regards,
> Petr


-- 
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] Add simple Containerfile

[Petr Vorel]

Hi Richie,

...
> > The benefit is that you have not only a clean git repo for the container,
> > but also not touching your working copy directory. But unless nobody else
> > raise any concern, I'm ok with your current proposal.

> Ah I think I see your concern now. It does not touch the directory
> outside the container. It copies everything in first (IDK if it actually
> copies the data if you are doing a local build).

Ah, sorry, of course.

> Note that I tried using a .dockerignore to stop build artifacts being
> copied, but it's not as rich as the .gitignore(s) we have sprinkled
> throughout the LTP.

...
> >> >> +#!/bin/sh -eux
> >> > nit: out of curiosity, why -u (fail unset variables and parameters)?

> >> I find it finds errors in shell scripts or when using them. E.g. typo's
> >> in env variable names. I just include it wherever possible.

> > +1, maybe we should add it to the current ci scripts as well (+ use params
> > instead of setting it via set command, it should work in dash and busybox shell
> > as well).

> Yeah, I think it should be the default.

I add it at least to the original scripts in ci directory, it should be safe
here. I'll test tst_test.sh and tst_net.sh and send a patch for this.

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] Add simple Containerfile

[Marius Kittler]

> The container can be built with a command like:
> 
> podman build -t tumbleweed/ltp \
>        --build-arg PREFIX=registry.opensuse.org/opensuse/ \
>        --build-arg DISTRO_NAME=tumbleweed \
>        --build-arg DISTRO_RELEASE=20230925 .
> 
> Or just
> 
> podman build -t alpine/ltp .
> 
> It contains Kirk in /opt/kirk. So
> 
> cd /opt/kirk && ./kirk -f ltp -r syscalls
> 
> will run some tests.

It would likely make sense to add a section with these information in the 
README as well.

I'm also wondering about the workflow of making changes (e.g. to a single test) 
and then re-compiling and re-running the specific test. Maybe that's also worth 
considering/documenting.

> 
> Note a much smaller container can be found at:
> https://registry.opensuse.org/cgi-bin/cooverview?srch_term=project%3D%5Ebenc
> hmark+container%3D.* This is created with SUSE's build system which does not
> use container files
> 
> Signed-off-by: Richard Palethorpe <rpalethorpe@suse.com>
> Cc: Petr Vorel <pvorel@suse.cz>
> Cc: Marius Kittler <mkittler@suse.de>
> ---
> 
> RFC comments:
>     * Add git clean -fdX which should remove any build artifacts
>       this is different from the suggestion of just doing a check. I just
>       found it easier to remove the build files.

Where we document the usage of the container we should also document that 
it'll do this kind of cleanup so there are no surprises.

>     * Added seperate alpine and tumbleweed runtime scripts. Again it's
>       different from the suggestion just because it's easier to add
>       seperate scripts than adding a switch
>     * Obviously a number of distros are missing runtime scripts. They can
>       be added when someone is motivated to do so.
> 
…
> +RUN mkdir /build
> +WORKDIR /build
> +COPY . /build
> +RUN ./ci/${DISTRO_NAME}.sh
> +RUN git clean -fdX
> +RUN ./build.sh -p $LTPROOT -i
> +

It might make sense to combine consecutive run commands (see
https://github.com/hadolint/hadolint/wiki/DL3059).




-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] Add simple Containerfile

[Richard Palethorpe]

Hello,

Marius Kittler <mkittler@suse.de> writes:

>> The container can be built with a command like:
>> 
>> podman build -t tumbleweed/ltp \
>>        --build-arg PREFIX=registry.opensuse.org/opensuse/ \
>>        --build-arg DISTRO_NAME=tumbleweed \
>>        --build-arg DISTRO_RELEASE=20230925 .
>> 
>> Or just
>> 
>> podman build -t alpine/ltp .
>> 
>> It contains Kirk in /opt/kirk. So
>> 
>> cd /opt/kirk && ./kirk -f ltp -r syscalls
>> 
>> will run some tests.
>
> It would likely make sense to add a section with these information in the 
> README as well.

I agree, but I think I will be doing some more work with containers with
Kirk and LTX. So I would prefer to document it after I know more about
the best workflow.

>
> I'm also wondering about the workflow of making changes (e.g. to a single test) 
> and then re-compiling and re-running the specific test. Maybe that's also worth 
> considering/documenting.

TBH I don't have a good workflow for that. I know that we can mount the
source in a volume. However we probably need to move the build out of
the tree. I'd be interested in suggestions.

>
>> 
>> Note a much smaller container can be found at:
>> https://registry.opensuse.org/cgi-bin/cooverview?srch_term=project%3D%5Ebenc
>> hmark+container%3D.* This is created with SUSE's build system which does not
>> use container files
>> 
>> Signed-off-by: Richard Palethorpe <rpalethorpe@suse.com>
>> Cc: Petr Vorel <pvorel@suse.cz>
>> Cc: Marius Kittler <mkittler@suse.de>
>> ---
>> 
>> RFC comments:
>>     * Add git clean -fdX which should remove any build artifacts
>>       this is different from the suggestion of just doing a check. I just
>>       found it easier to remove the build files.
>
> Where we document the usage of the container we should also document that 
> it'll do this kind of cleanup so there are no surprises.

To be clear this only happens inside the container and *should* only
remove things covered by .gitignore (i.e. build artifacts). So does it
need documenting in that case?

>
>>     * Added seperate alpine and tumbleweed runtime scripts. Again it's
>>       different from the suggestion just because it's easier to add
>>       seperate scripts than adding a switch
>>     * Obviously a number of distros are missing runtime scripts. They can
>>       be added when someone is motivated to do so.
>> 
> …
>> +RUN mkdir /build
>> +WORKDIR /build
>> +COPY . /build
>> +RUN ./ci/${DISTRO_NAME}.sh
>> +RUN git clean -fdX
>> +RUN ./build.sh -p $LTPROOT -i
>> +
>
> It might make sense to combine consecutive run commands (see
> https://github.com/hadolint/hadolint/wiki/DL3059).

I think this depends on exactly how caching works and where the best
place to put git clean is. Ideally we only want to run the ci script
when the dependencies change.

Maybe we should also split the COPY, to make sure the container
runtime can detect which files have changed.

More importantly I think this relates to your comment on workflow for
changing a single test. Ideally we should be able to change and compile
a single test without doing a bunch of unecessary work. Right now I am
not sure how to do that?

-- 
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] Add simple Containerfile

[Andrea Cervesato via ltp]

Hi,

thanks for this patch. It's really useful.

Regards,
Andrea Cervesato

On 9/28/23 12:44, Richard Palethorpe via ltp wrote:
> This adds a regular container (or Docker) file which builds LTP from
> source. This is initially intended for testing the LTP itself.
>
> The resulting container has just the LTP installation and runtime
> dependencies. However it is still quite large, probably due to debug
> symbols.
>
> The container can be built with a command like:
>
> podman build -t tumbleweed/ltp \
>         --build-arg PREFIX=registry.opensuse.org/opensuse/ \
>         --build-arg DISTRO_NAME=tumbleweed \
>         --build-arg DISTRO_RELEASE=20230925 .
>
> Or just
>
> podman build -t alpine/ltp .
>
> It contains Kirk in /opt/kirk. So
>
> cd /opt/kirk && ./kirk -f ltp -r syscalls
>
> will run some tests.
>
> Note a much smaller container can be found at:
> https://registry.opensuse.org/cgi-bin/cooverview?srch_term=project%3D%5Ebenchmark+container%3D.*
> This is created with SUSE's build system which does not use container files
>
> Signed-off-by: Richard Palethorpe <rpalethorpe@suse.com>
> Cc: Petr Vorel <pvorel@suse.cz>
> Cc: Marius Kittler <mkittler@suse.de>
> ---
>
> RFC comments:
>      * Add git clean -fdX which should remove any build artifacts
>        this is different from the suggestion of just doing a check. I just
>        found it easier to remove the build files.
>      * Added seperate alpine and tumbleweed runtime scripts. Again it's
>        different from the suggestion just because it's easier to add
>        seperate scripts than adding a switch
>      * Obviously a number of distros are missing runtime scripts. They can
>        be added when someone is motivated to do so.
>
>   .dockerignore            |  1 +
>   Containerfile            | 32 ++++++++++++++++++++++++++++++++
>   ci/alpine-runtime.sh     | 16 ++++++++++++++++
>   ci/tumbleweed-runtime.sh | 13 +++++++++++++
>   4 files changed, 62 insertions(+)
>   create mode 100644 .dockerignore
>   create mode 100644 Containerfile
>   create mode 100755 ci/alpine-runtime.sh
>   create mode 100755 ci/tumbleweed-runtime.sh
>
> diff --git a/.dockerignore b/.dockerignore
> new file mode 100644
> index 000000000..bbcd7072f
> --- /dev/null
> +++ b/.dockerignore
> @@ -0,0 +1 @@
> +Containerfile
> diff --git a/Containerfile b/Containerfile
> new file mode 100644
> index 000000000..2f8192c3b
> --- /dev/null
> +++ b/Containerfile
> @@ -0,0 +1,32 @@
> +ARG PREFIX=docker.io/
> +ARG DISTRO_NAME=alpine
> +ARG DISTRO_RELEASE=3.18
> +
> +FROM $PREFIX$DISTRO_NAME:$DISTRO_RELEASE AS build
> +ARG LTPROOT=/opt/ltp
> +ARG DISTRO_NAME=alpine
> +ARG DISTRO_RELEASE=3.18
> +
> +RUN mkdir /build
> +WORKDIR /build
> +COPY . /build
> +RUN ./ci/${DISTRO_NAME}.sh
> +RUN git clean -fdX
> +RUN ./build.sh -p $LTPROOT -i
> +
> +FROM $PREFIX$DISTRO_NAME:$DISTRO_RELEASE
> +ARG LTPROOT=/opt/ltp
> +ARG KIRKROOT=/opt/kirk
> +ARG DISTRO_NAME=alpine
> +
> +COPY --from=build /build/ci/${DISTRO_NAME}-runtime.sh $LTPROOT/runtime-deps.sh
> +RUN $LTPROOT/runtime-deps.sh
> +
> +COPY --from=build $LTPROOT $LTPROOT
> +ENV LTPROOT=$LTPROOT
> +ENV PATH=$LTPROOT/testcases/bin:$LTPROOT/bin:$PATH
> +
> +RUN mkdir -p $KIRKROOT
> +COPY --from=build /build/tools/kirk $KIRKROOT
> +
> +USER ltp
> diff --git a/ci/alpine-runtime.sh b/ci/alpine-runtime.sh
> new file mode 100755
> index 000000000..e4941f329
> --- /dev/null
> +++ b/ci/alpine-runtime.sh
> @@ -0,0 +1,16 @@
> +#!/bin/sh -eux
> +
> +apk add \
> +        acl \
> +        keyutils \
> +        libaio \
> +        libacl \
> +        libcap \
> +        libselinux \
> +        libsepol \
> +        libtirpc \
> +        numactl \
> +        openssl \
> +        py3-msgpack
> +
> +adduser -D -g "Unprivileged LTP user" ltp
> diff --git a/ci/tumbleweed-runtime.sh b/ci/tumbleweed-runtime.sh
> new file mode 100755
> index 000000000..5207988e7
> --- /dev/null
> +++ b/ci/tumbleweed-runtime.sh
> @@ -0,0 +1,13 @@
> +#!/bin/sh -eux
> +
> +zyp="zypper --non-interactive install --force-resolution --no-recommends"
> +
> +$zyp \
> +	iproute2 \
> +	keyutils \
> +	libaio1 \
> +	libmnl0 \
> +	libnuma1 \
> +	libtirpc3
> +
> +useradd ltp



-- 
Mailing list info: https://lists.linux.it/listinfo/ltp