Re: s390 private runner CI job timing out

[Christian Borntraeger <borntraeger@linux.ibm.com>]

Am 06.04.23 um 14:30 schrieb Thomas Huth:
> On 06/04/2023 14.13, Christian Borntraeger wrote:
>>
>>
>> Am 06.04.23 um 14:05 schrieb Peter Maydell:
>>> On Thu, 6 Apr 2023 at 12:17, Christian Borntraeger
>>> <borntraeger@linux.ibm.com> wrote:
>>>>
>>>> Am 06.04.23 um 12:44 schrieb Peter Maydell:
>>>>> On Thu, 6 Apr 2023 at 11:40, Christian Borntraeger
>>>>> <borntraeger@linux.ibm.com> wrote:
>>>>>> Am 06.04.23 um 11:21 schrieb Peter Maydell:
>>>>>>> Christian, does our S390X machine get a guaranteed amount of CPU,
>>>>>>> or does it depend on what else is running on the hardware?
>>>>>>
>>>>>> I think its a shared system with shared CPUs. Can you check the steal
>>>>>> time in top or proc? If this is far too high we could ask to give you
>>>>>> more weight for that VM.
>>>>>
>>>>> It's idle at the moment and steal time seems to be low (0.0 .. 0.3);
>>>>> I'll try to remember to check next time it's running a job.
>>>>>
>>>>
>>>> Do you have /proc/stat ?
>>>
>>> Yes; hopefully it means more to you than it does to me :-)
>>>
>>> linux1@qemu01:~$ cat /proc/stat
>>> cpu  60904459 604975 15052194 1435958176 17128179 351949 758578 22218760 0 0
>>> cpu0 15022535 146734 3786909 358774818 4283172 98313 237156 5894809 0 0
>>> cpu1 15306890 151164 3746024 358968957 4378864 85629 172492 5434255 0 0
>>> cpu2 15307709 157180 3762691 359141276 4138714 85736 176367 5474594 0 0
>>> cpu3 15267324 149895 3756569 359073124 4327428 82269 172562 5415101 0 0
>>
>> This is
>> user,nice,system,idle,iowait,irq,softirq,steal,guest,guest_nice
>> So overall there is some (20-30% since the last reboot) steal going on.
>> Not sure if this is the real problem since it is only Ubuntu 20.04.
>> Does a higher timeout make the problem go away?
> 
> The thing is: it shouldn't take that long to build QEMU and run the tests here, theoretically. Some days ago, the job was finishing in 39 minutes:
> 
>   https://gitlab.com/qemu-project/qemu/-/jobs/3973481571
> 
> The recent run took 74 minutes:
> 
>   https://gitlab.com/qemu-project/qemu/-/jobs/4066136770
> 
> That's almost a factor of two! So there is definitely something strange going on.

But this has 786 instead of 659 tests, no?