* Re: TuxBuild Now Supports Clang 11 and 5 new architectures
2020-11-19 9:13 ` TuxBuild Now Supports Clang 11 and 5 new architectures Guillaume Tucker
@ 2020-11-23 17:05 ` Dan Rue
0 siblings, 0 replies; 2+ messages in thread
From: Dan Rue @ 2020-11-23 17:05 UTC (permalink / raw)
To: Guillaume Tucker; +Cc: tuxbuild, clang-built-linux, kernelci
On Thu, Nov 19, 2020 at 09:13:41AM +0000, Guillaume Tucker wrote:
> Hi Dan,
Hi Guillaume!
>
> On 17/11/2020 21:41, Dan Rue wrote:
> > Cheers!
> >
> > Today we've deployed support for clang-11, and we dropped support for clang 8 and 9 based on user feedback.
> >
> > In addition, we have added the following architectures with gcc 8, 9, and 10:
> > - s390
> > - powerpc
> > - sparc
> > - hppa
> > - sh
> >
> > TuxBuild's full support matrix is as follows:
> >
> > arc arm arm64 hppa i386 mips powerpc riscv s390 sh sparc x86_64
> > clang-10 no yes yes no yes yes no yes no no no yes
> > clang-11 no yes yes no yes yes no yes no no no yes
>
> Out of interest, are there builds produced with tuxbuild for all
> the supported configurations that can be downloaded with a build
> log?
I don't think there's anything published, but Antonio might have
something related to the container tests that are done in tuxmake before
publishing new containers.
>
> I've just been working on building riscv with Clang for KernelCI
> and there are quite a few issues, we've dropped clang-10 as it's
> actually broken. So I wonder if you define "supported" solely by
> having a Docker image and the ability to start a build, or if
> it's required to actually complete a kernel build?
>
> See related discussion about riscv here:
>
> https://github.com/kernelci/kernelci-core/pull/534
If there is a legitimate problem in the kernel sources or in a toolchain
implementation, the tuxmake would be expected to fail, and tuxmake
shouldn't work around things that are broken.
If tuxmake is doing the wrong thing, then it's a bug that we should fix.
If a combination might work but currently doesn't, tuxmake can support
it. If it will never work, we won't bother.
`tuxmake -r podman -t riscv -t clang-10` works for me on v5.10-rc5.
Here's the tuxbuild of the same, with logs and such:
https://builds.tuxbuild.com/1khN9GYmp4AKa1g6m4MwM7yhgnE/
>
>
> > gcc-10 no yes yes yes yes yes yes yes yes yes yes yes
> > gcc-8 yes yes yes yes yes yes yes yes yes yes yes yes
> > gcc-9 yes yes yes yes yes yes yes yes yes yes yes yes
> >
> > We're still updating the tuxbuild documentation, but the new toolchains and architectures are available immediately without a tuxbuild update (the argument validation is all done server-side).
> >
> > Now that we have TuxMake integration which is enabling these new features, we would like to add tuxmake build reproducers to tuxbuild. A build reproducer is a simple one-line shell script provided alongside the other tuxbuild artifacts, which contain the tuxmake command-line invocation to perform the same build locally. For example:
> >
> > $ cat tuxmake_reproducer.sh
> > tuxmake --runtime docker --image docker.io/tuxmake/arm64_clang-11 <http://docker.io/tuxmake/arm64_clang-11> --target-arch arm64 --kconfig defconfig --kconfig-add "CONFIG_KASAN=y" --toolchain clang-11
>
> That's nice but I believe it would be easier to adopt if the
> syntax was simpler. For example:
>
> tuxmake --runtime docker --target-arch arm64 --kconfig defconfig --kconfig-add "CONFIG_KASAN=y" --toolchain clang-11
>
> or if you really want to keep the docker image mentioned to
> ensure reproducibility:
>
> tuxmake --runtime docker --image docker.io/tuxmake/arm64_clang-11 --kconfig defconfig --kconfig-add "CONFIG_KASAN=y"
>
> I can see why you would want to define all the parameters
> explicitly, but it seems like there's some repetition there. If
> the docker image is already arch-specific, then do you also need
> to specify the arch as a separate parameter?
You're right, --image in the above example was superfluous. We were
struggling to decide how granular to be with reproducers because you can
be as simple as:
tuxmake --runtime docker --target-arch x86_64 --kconfig defconfig --toolchain gcc
And as "correct" as e.g.:
tuxmake --image docker.io/tuxmake/x86_64_gcc@sha256:f8218cbfad8ecf6628fc44db864a402070feb87ff43a880e1409649172d4bc8c -r docker --target-arch x86_64 --kconfig tinyconfig --toolchain gcc -e "KBUILD_BUILD_TIMESTAMP='Tue May 26 16:16:14 2020 -0500'" -e "KBUILD_BUILD_USER=tuxmake" -e "KBUILD_BUILD_HOST=tuxmake"
The latter provides (at least potentially) an actual reproducible build,
but the former is simpler and usually sufficient.
We decided to favor simplicity and all tuxbuilds now publish a
tuxbuild_reproducer.sh and tuxmake_reproducer.sh which consists of the
simple version of the command.
For example in the risc5/clang-10 build from above, we see the following
two reproducers now:
tuxbuild_reproducer.sh:
https://builds.tuxbuild.com/1khN9GYmp4AKa1g6m4MwM7yhgnE/tuxbuild_reproducer.sh
#!/bin/sh
tuxbuild build --git-repo https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git --git-sha 418baf2c28f3473039f2f7377760bd8f6897ae18 --target-arch riscv --toolchain clang-10 --kconfig defconfig
tuxmake_reproducer.sh:
https://builds.tuxbuild.com/1khN9GYmp4AKa1g6m4MwM7yhgnE/tuxmake_reproducer.sh
#!/bin/sh
tuxmake --runtime docker --target-arch riscv --toolchain clang-10 --kconfig defconfig
Hope that helps!
Dan
>
> Best wishes,
> Guillaume
>
> > As a kernel developer, would you appreciate receiving such a command as a part of a build discussion on a mailing list?
> >
> > Would you send something like that to your colleagues?
> >
> > Do you find the concept of a local reproducer helpful?
> >
> > Thanks again to everyone using and supporting TuxBuild and TuxMake - we see you and we appreciate you,
> >
> > Dan
> >
> > p.s. Did you know that it's possible to perform actual reproducible builds with tuxmake? It's true. Try it for yourself, and see if your bzImage sha256sum matches:
> >
> > $ git describe
> > v5.10-rc4
> > $ tuxmake --image docker.io/tuxmake/x86_64_gcc@sha256:f8218cbfad8ecf6628fc44db864a402070feb87ff43a880e1409649172d4bc8c <http://docker.io/tuxmake/x86_64_gcc@sha256:f8218cbfad8ecf6628fc44db864a402070feb87ff43a880e1409649172d4bc8c> -r docker -k tinyconfig -e "KBUILD_BUILD_TIMESTAMP='Tue May 26 16:16:14 2020 -0500'" -e "KBUILD_BUILD_USER=tuxmake" -e "KBUILD_BUILD_HOST=tuxmake"
> > ...
> > $ sha256sum /home/drue/.cache/tuxmake/builds/634/bzImage
> > 5581ed4583fb4b9d76760ee329e248fe7281d7ea3b6ae0737efc7a6a4462affa /home/drue/.cache/tuxmake/builds/634/bzImage
> >
> > p.p.s. Did you know that tuxmake also supports podman? Just use -r podman instead of -r docker.
>
--
Linaro LKFT
https://lkft.linaro.org
^ permalink raw reply [flat|nested] 2+ messages in thread