Re: [PATCH 02/24] scripts/archive-source: speed up downloading subprojects

["Marc-André Lureau" <marcandre.lureau@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 2611 bytes --]

Hi

On Fri, Sep 19, 2025 at 5:50 PM Daniel P. Berrangé <berrange@redhat.com>
wrote:

> On Fri, Sep 19, 2025 at 05:32:56PM +0400, marcandre.lureau@redhat.com
> wrote:
> > From: Marc-André Lureau <marcandre.lureau@redhat.com>
> >
> > Running meson on each subproject is quite slow.
>
> Why is this faster ?  Does it make meson do parallel downloads
> if you list then all at once ?
>

It doesn't look like it does it in paralllel, yet it is x times faster...

time sh -c 'for s in keycodemapdb libvfio-user berkeley-softfloat-3
berkeley-testfloat-3 anyhow-1-rs arbitrary-int-1-rs bilge-0.2-rs
bilge-impl-0.2-rs either-1-rs foreign-0.3-rs itertools-0.11-rs libc-0.2-rs
proc-macro2-1-rs proc-macro-error-1-rs proc-macro-error-attr-1-rs
quote-1-rs syn-2-rs unicode-ident-1-rs; do
  meson subprojects download $s >/dev/null
done'
sh -c   5.27s user 0.33s system 99% cpu 5.613 total

time sh -c 'meson subprojects download keycodemapdb libvfio-user
berkeley-softfloat-3 berkeley-testfloat-3 anyhow-1-rs arbitrary-int-1-rs
bilge-0.2-rs bilge-impl-0.2-rs either-1-rs foreign-0.3-rs itertools-0.11-rs
libc-0.2-rs proc-macro2-1-rs proc-macro-error-1-rs
proc-macro-error-attr-1-rs quote-1-rs syn-2-rs unicode-ident-1-rs'
sh -c   0.32s user 0.01s system 99% cpu 0.339 total


>
> >
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  scripts/archive-source.sh | 5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> >
> > diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
> > index 035828c532..85cd0a63a0 100755
> > --- a/scripts/archive-source.sh
> > +++ b/scripts/archive-source.sh
> > @@ -77,9 +77,10 @@ function subproject_dir() {
> >  git archive --format tar "$(tree_ish)" > "$tar_file"
> >  test $? -ne 0 && error "failed to archive qemu"
> >
> > +meson subprojects download $subprojects
> > +test $? -ne 0 && error "failed to download subprojects $subprojects"
> > +
> >  for sp in $subprojects; do
> > -    meson subprojects download $sp
> > -    test $? -ne 0 && error "failed to download subproject $sp"
> >      tar --append --file "$tar_file" --exclude=.git
> subprojects/"$(subproject_dir $sp)"
> >      test $? -ne 0 && error "failed to append subproject $sp to
> $tar_file"
> >  done
> > --
> > 2.51.0
> >
>
> With regards,
> Daniel
> --
> |: https://berrange.com      -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-
> https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-
> https://www.instagram.com/dberrange :|
>
>

[-- Attachment #2: Type: text/html, Size: 4157 bytes --]