New container build error: mountpoint does not exit

New container build error: mountpoint does not exit

[Richard Henderson]

Hi Eldon,

New this morning are some odd failures in the container build stage, e.g

https://gitlab.com/qemu-project/qemu/-/jobs/4345796216#L235

cgroups: cgroup mountpoint does not exist: unknown

There are several such failures in that pipeline. I've not seen this before, nor was it 
happening yesterday.

Any ideas?


r~

Re: New container build error: mountpoint does not exit

[Eldon Stegall]

On Wed, May 24, 2023 at 01:09:30PM -0700, Richard Henderson wrote:
> Hi Eldon,
> 
> New this morning are some odd failures in the container build stage, e.g
> 
> https://gitlab.com/qemu-project/qemu/-/jobs/4345796216#L235
> 
> cgroups: cgroup mountpoint does not exist: unknown
> 
> There are several such failures in that pipeline. I've not seen this before, nor was it 
> happening yesterday.
> 
> Any ideas?

Hi Richard,
If I had to guess, it's probably related to some changes in
Camille's patchset. My first guess is that it is related to 
https://gitlab.com/qemu-project/qemu/-/commit/5f63a67adb58478974b91f5e5c2b1222b5c7f2cc

which downgrades docker-in-docker from latest to stable-dind.

I'm not sure how the stable tags are treated with docker, but it
does seem that those tags haven't been updated in a couple of years.

This person on SO seems to have fixed a similar issue in gitlab ci
by upgrading: https://stackoverflow.com/q/70707572

The cgroups error leads me to believe that the older dind service is 
expecting cgroups v1, while the hardware runner is running FCOS with
cgroups v2.
 
I ran a test with just that patch omitted on my repo, with better
results:

https://gitlab.com/eldondev/qemu/-/pipelines/878387857

I will see if the opensbi build which also uses stable-dind and failed
in that pipeline is successful when attempting to use latest dind
instead.

If using stable is compelling, I can work on making the hardware runner
support it.

Thanks,
Eldon