* [PATCH] cirrus.yml: Exclude some targets in the FreeBSD job to speed up the build
@ 2020-08-24 9:44 Thomas Huth
2020-08-27 15:00 ` Daniel P. Berrangé
2020-08-28 8:59 ` Daniel P. Berrangé
0 siblings, 2 replies; 6+ messages in thread
From: Thomas Huth @ 2020-08-24 9:44 UTC (permalink / raw)
To: Ed Maste, qemu-devel; +Cc: Paolo Bonzini, Alex Bennée, Li-Wen Hsu
The FreeBSD jobs currently hit the 1h time limit in the Cirrus-CI.
We've got to exclude some build targets here to make sure that the job
finishes in time again. The targets that are excluded should not hurt
much, since e.g. all the code from i386-softmmu is covered again by
x86_64-softmmu.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
Not sure why the FreeBSD builds suddenly got slower and hit the 1h
time limit now. Looking at https://cirrus-ci.com/github/qemu/qemu
it seems like the FreeBSD jobs were already taking ca. 48 minutes
since quite a while, but since the Meson build system has been merged,
they now always hit the 1h limit. Could it be that Meson is slow on
FreeBSD?
I hope disabling some of the "redundant" targets is ok for everybody
to get this working again. Alternatively, we could also introduce a
second FreeBSD job and run half of the targets in one job, and half
of the targets in the other job if that's preferred?
.cirrus.yml | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index 0742aaf8a3..bd3329b663 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -12,7 +12,9 @@ freebsd_12_task:
script:
- mkdir build
- cd build
- - ../configure --enable-werror || { cat config.log; exit 1; }
+ - ../configure --enable-werror --target-list-exclude='mips64el-softmmu
+ mips-softmmu arm-softmmu i386-softmmu ppc-softmmu riscv32-softmmu
+ sh4eb-softmmu xtensaeb-softmmu' || { cat config.log; exit 1; }
- gmake -j8
- gmake V=1 check
--
2.18.2
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH] cirrus.yml: Exclude some targets in the FreeBSD job to speed up the build
2020-08-24 9:44 [PATCH] cirrus.yml: Exclude some targets in the FreeBSD job to speed up the build Thomas Huth
@ 2020-08-27 15:00 ` Daniel P. Berrangé
2020-08-28 4:28 ` Thomas Huth
2020-08-28 8:59 ` Daniel P. Berrangé
1 sibling, 1 reply; 6+ messages in thread
From: Daniel P. Berrangé @ 2020-08-27 15:00 UTC (permalink / raw)
To: Thomas Huth
Cc: Alex Bennée, Paolo Bonzini, Ed Maste, qemu-devel, Li-Wen Hsu
On Mon, Aug 24, 2020 at 11:44:10AM +0200, Thomas Huth wrote:
> The FreeBSD jobs currently hit the 1h time limit in the Cirrus-CI.
> We've got to exclude some build targets here to make sure that the job
> finishes in time again. The targets that are excluded should not hurt
> much, since e.g. all the code from i386-softmmu is covered again by
> x86_64-softmmu.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> Not sure why the FreeBSD builds suddenly got slower and hit the 1h
> time limit now. Looking at https://cirrus-ci.com/github/qemu/qemu
> it seems like the FreeBSD jobs were already taking ca. 48 minutes
> since quite a while, but since the Meson build system has been merged,
> they now always hit the 1h limit. Could it be that Meson is slow on
> FreeBSD?
No, it appears to just be compiling a lot more than in the past.
$ grep Compiling main.log-meson | wc -l
5468
$ grep CC main.log-config | wc -l
4855
This seems to be caused my compiling lots of stuff under tests/
that we did not previously build
$ grep Compiling main.log-meson | grep tests/ | wc -l
616
not sure if this is specific to FreeBSD, or whether we're doing
this on all platforms and merely noticed first on FreeBSD due
to the timeout.
Mostly it seems to the libtestfloat ad libsoftfloat, but also
a bit of libqtest stuff.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] cirrus.yml: Exclude some targets in the FreeBSD job to speed up the build
2020-08-27 15:00 ` Daniel P. Berrangé
@ 2020-08-28 4:28 ` Thomas Huth
0 siblings, 0 replies; 6+ messages in thread
From: Thomas Huth @ 2020-08-28 4:28 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Paolo Bonzini, Ed Maste, Alex Bennée, qemu-devel, Li-Wen Hsu
On 27/08/2020 17.00, Daniel P. Berrangé wrote:
> On Mon, Aug 24, 2020 at 11:44:10AM +0200, Thomas Huth wrote:
>> The FreeBSD jobs currently hit the 1h time limit in the Cirrus-CI.
>> We've got to exclude some build targets here to make sure that the job
>> finishes in time again. The targets that are excluded should not hurt
>> much, since e.g. all the code from i386-softmmu is covered again by
>> x86_64-softmmu.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> Not sure why the FreeBSD builds suddenly got slower and hit the 1h
>> time limit now. Looking at https://cirrus-ci.com/github/qemu/qemu
>> it seems like the FreeBSD jobs were already taking ca. 48 minutes
>> since quite a while, but since the Meson build system has been merged,
>> they now always hit the 1h limit. Could it be that Meson is slow on
>> FreeBSD?
>
> No, it appears to just be compiling a lot more than in the past.
>
> $ grep Compiling main.log-meson | wc -l
> 5468
>
> $ grep CC main.log-config | wc -l
> 4855
Oh, that's interesting, thanks!
> This seems to be caused my compiling lots of stuff under tests/
> that we did not previously build
>
> $ grep Compiling main.log-meson | grep tests/ | wc -l
> 616
>
> not sure if this is specific to FreeBSD, or whether we're doing
> this on all platforms and merely noticed first on FreeBSD due
> to the timeout.
I also did some build time tests on my x86 Linux laptop, but I did not
notice any significant differences there.
> Mostly it seems to the libtestfloat ad libsoftfloat, but also
> a bit of libqtest stuff.
Now that you've mentioned softfloat ... I think that matches what I've
seen in the log output on cirrus-ci.com : Suddenly there is a lot of
verbose softloat test output there that I never noticed before.
I assume that's ok that we now compile/test more on FreeBSD, so I'll go
ahead with this patch and put it into my next "CI" pull
request to fix the cirrus-ci builds again.
Thomas
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] cirrus.yml: Exclude some targets in the FreeBSD job to speed up the build
2020-08-24 9:44 [PATCH] cirrus.yml: Exclude some targets in the FreeBSD job to speed up the build Thomas Huth
2020-08-27 15:00 ` Daniel P. Berrangé
@ 2020-08-28 8:59 ` Daniel P. Berrangé
2020-08-28 9:54 ` Thomas Huth
1 sibling, 1 reply; 6+ messages in thread
From: Daniel P. Berrangé @ 2020-08-28 8:59 UTC (permalink / raw)
To: Thomas Huth
Cc: Alex Bennée, Paolo Bonzini, Ed Maste, qemu-devel, Li-Wen Hsu
On Mon, Aug 24, 2020 at 11:44:10AM +0200, Thomas Huth wrote:
> The FreeBSD jobs currently hit the 1h time limit in the Cirrus-CI.
> We've got to exclude some build targets here to make sure that the job
> finishes in time again. The targets that are excluded should not hurt
> much, since e.g. all the code from i386-softmmu is covered again by
> x86_64-softmmu.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> Not sure why the FreeBSD builds suddenly got slower and hit the 1h
> time limit now. Looking at https://cirrus-ci.com/github/qemu/qemu
> it seems like the FreeBSD jobs were already taking ca. 48 minutes
> since quite a while, but since the Meson build system has been merged,
> they now always hit the 1h limit. Could it be that Meson is slow on
> FreeBSD?
>
> I hope disabling some of the "redundant" targets is ok for everybody
> to get this working again. Alternatively, we could also introduce a
> second FreeBSD job and run half of the targets in one job, and half
> of the targets in the other job if that's preferred?
I'd probably suggest we go for two jobs, as there doesn't look like
any constraint on running multiple jobs that we'll hit any time
soon.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] cirrus.yml: Exclude some targets in the FreeBSD job to speed up the build
2020-08-28 8:59 ` Daniel P. Berrangé
@ 2020-08-28 9:54 ` Thomas Huth
2020-08-31 6:13 ` Thomas Huth
0 siblings, 1 reply; 6+ messages in thread
From: Thomas Huth @ 2020-08-28 9:54 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Alex Bennée, Paolo Bonzini, Ed Maste, qemu-devel, Li-Wen Hsu
On 28/08/2020 10.59, Daniel P. Berrangé wrote:
> On Mon, Aug 24, 2020 at 11:44:10AM +0200, Thomas Huth wrote:
>> The FreeBSD jobs currently hit the 1h time limit in the Cirrus-CI.
>> We've got to exclude some build targets here to make sure that the job
>> finishes in time again. The targets that are excluded should not hurt
>> much, since e.g. all the code from i386-softmmu is covered again by
>> x86_64-softmmu.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> Not sure why the FreeBSD builds suddenly got slower and hit the 1h
>> time limit now. Looking at https://cirrus-ci.com/github/qemu/qemu
>> it seems like the FreeBSD jobs were already taking ca. 48 minutes
>> since quite a while, but since the Meson build system has been merged,
>> they now always hit the 1h limit. Could it be that Meson is slow on
>> FreeBSD?
>>
>> I hope disabling some of the "redundant" targets is ok for everybody
>> to get this working again. Alternatively, we could also introduce a
>> second FreeBSD job and run half of the targets in one job, and half
>> of the targets in the other job if that's preferred?
>
> I'd probably suggest we go for two jobs, as there doesn't look like
> any constraint on running multiple jobs that we'll hit any time
> soon.
I gave it a try, but either I'm doing something wrong, or it is not
possible to run multiple FreeBSD tasks on cirrus-ci for one commit -
either the second task does not show up, or both FreeBSD tasks do not
show up here. I assume there are some limitations in place (see
https://cirrus-ci.org/faq/#are-there-any-limits), but it even does not
work for me when I limit the number of cpus in the tasks to a lower
value like 4 or 2.
Could somebody else please have a try? Maybe I'm just doing something
stupid here...
Thomas
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] cirrus.yml: Exclude some targets in the FreeBSD job to speed up the build
2020-08-28 9:54 ` Thomas Huth
@ 2020-08-31 6:13 ` Thomas Huth
0 siblings, 0 replies; 6+ messages in thread
From: Thomas Huth @ 2020-08-31 6:13 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Paolo Bonzini, Ed Maste, Alex Bennée, qemu-devel, Li-Wen Hsu
On 28/08/2020 11.54, Thomas Huth wrote:
> On 28/08/2020 10.59, Daniel P. Berrangé wrote:
>> On Mon, Aug 24, 2020 at 11:44:10AM +0200, Thomas Huth wrote:
>>> The FreeBSD jobs currently hit the 1h time limit in the Cirrus-CI.
>>> We've got to exclude some build targets here to make sure that the job
>>> finishes in time again. The targets that are excluded should not hurt
>>> much, since e.g. all the code from i386-softmmu is covered again by
>>> x86_64-softmmu.
>>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>> Not sure why the FreeBSD builds suddenly got slower and hit the 1h
>>> time limit now. Looking at https://cirrus-ci.com/github/qemu/qemu
>>> it seems like the FreeBSD jobs were already taking ca. 48 minutes
>>> since quite a while, but since the Meson build system has been merged,
>>> they now always hit the 1h limit. Could it be that Meson is slow on
>>> FreeBSD?
>>>
>>> I hope disabling some of the "redundant" targets is ok for everybody
>>> to get this working again. Alternatively, we could also introduce a
>>> second FreeBSD job and run half of the targets in one job, and half
>>> of the targets in the other job if that's preferred?
>>
>> I'd probably suggest we go for two jobs, as there doesn't look like
>> any constraint on running multiple jobs that we'll hit any time
>> soon.
>
> I gave it a try, but either I'm doing something wrong, or it is not
> possible to run multiple FreeBSD tasks on cirrus-ci for one commit
I apparently needed a weekend in between to get a clean head again. I
was doing indeed something wrong... apparently you have to name your
tasks on cirrus-ci with "_task" at the end. If you use "_task1" and
"_task2", it does now work anymore. Sigh.
So I'll try to cook a proper patch to split the job instead.
Thomas
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-08-31 6:13 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-08-24 9:44 [PATCH] cirrus.yml: Exclude some targets in the FreeBSD job to speed up the build Thomas Huth
2020-08-27 15:00 ` Daniel P. Berrangé
2020-08-28 4:28 ` Thomas Huth
2020-08-28 8:59 ` Daniel P. Berrangé
2020-08-28 9:54 ` Thomas Huth
2020-08-31 6:13 ` Thomas Huth
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).