Re: Moving QEMU downloads to GitLab Releases?

[Stefan Hajnoczi <stefanha@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 5830 bytes --]

On Mon, Oct 04, 2021 at 02:34:49PM -0500, Michael Roth wrote:
> Quoting Stefan Hajnoczi (2021-10-04 04:01:22)
> > On Fri, Oct 01, 2021 at 09:39:13AM +0200, Philippe Mathieu-Daudé wrote:
> > > On 9/30/21 15:40, Stefan Hajnoczi wrote:
> > > > Hi Mike,
> > > > QEMU downloads are currently hosted on qemu.org's Apache web server.
> > > > Paolo and I were discussing ways to reduce qemu.org network traffic to
> > > > save money and eventually turn off the qemu.org server since there is no
> > > > full-time sysadmin for it. I'd like to discuss moving QEMU downloads to
> > > > GitLab Releases.
> > > > 
> > > > Since you create and sign QEMU releases I wanted to see what you think
> > > > about the idea. GitLab Releases has two ways of creating release assets:
> > > > archiving a git tree and attaching arbitrary binaries. The
> > > > scripts/make-release script fetches submodules and generates version
> > > > files, so it may be necessary to treat QEMU tarballs as arbitrary
> > > > binaries instead of simply letting GitLab create git tree archives:
> > > > https://docs.gitlab.com/ee/user/project/releases/#use-a-generic-package-for-attaching-binaries
> > > > 
> > > > Releases can be uploaded via the GitLab API from your local machine or
> > > > deployed as a GitLab CI job. Uploading from your local machine would be
> > > > the closest to the current workflow.
> > > > 
> > > > In the long term we could have a CI job that automatically publishes
> > > > QEMU releases when a new qemu.git tag is pushed. The release process
> > > > could be fully automated so that manual steps are no longer necessary,
> > > > although we'd have to trust GitLab with QEMU GPG signing keys.
> > > 
> > > Before having to trust a SaaS for GPG signing, could this work?
> > > 
> > > - make-release script should produce a reproducible tarball in a
> > >   gitlab job, along with a file containing the tarball hash.
> > > 
> > > - Mike (or whoever is responsible of releases) keeps doing a local
> > >   manual build
> > > 
> > > - Mike checks the local hash matches the Gitlab artifact hash
> > > 
> > > - Mike signs its local build/hash and uses the GitLab API to upload
> > >   that .sig to job artifacts
> > > 
> > > - we can have an extra manual job that checks the tarball.sig
> > >   (hash and pubkey) and on success deploys updating the download
> > >   page, adding the new release
> > 
> > I wonder what Mike sees as the way forward.
> 
> Hi Stefan, Philippe,
> 
> In general I like the idea, since we could also have the CI do the full
> gamut of testing against the binaries built from said tarball, so the
> Release Person can just regenerate the tarball and provide a sig to
> attest that it came from the proper sources. Currently I do make check
> and make check-acceptance and a few other sanity checks, which I guess
> would no longer be needed then.
> 
> But I think the more immediate issue is where/how to host those
> tarballs. Even moving all the ROMs/capstone out of the source tree still
> results in an xz-compressed tarball size ~25MB, which is well above the
> 10MB limit mentioned earlier. We could break it out into target-specific
> tarballs, maybe further into softmmu/user variants, but that sounds
> painful for both users and maintainers who need to deal with the
> resulting source tree differences.
> 
> What I'm wondering is whether we could just use the archive files
> generated by gitlab when we tag our releases? E.g.:
> 
>   https://gitlab.com/qemu-project/qemu/-/archive/v6.1.0/qemu-v6.1.0.tar.bz2
> 
> If we paired that with an in-tree script similar to make-release for
> users to download individual ROM sources/subprojects used for a particular
> tagged release, would that be sufficient for GPL compliance and verifying
> what sources the binaries were built from? Are there any other
> considerations WRT ROMs/etc.?
> 
> With something like that in place, Release Person could just do a git
> checkout, verify the Maintainer's sig/tag (in case we don't necessarily
> trust the git host), generate the tarball, verify the hash matches what
> gitlab published (or verify/diff individual files if the bz/gz hashes
> require a specific environment), then sign the gitlab tarball and add
> the sig to qemu.org download page along with a link the gitlab-generated
> tarball.
> 
> We could also publish the Maintainer and Release Person public keys on
> qemu.org download page so users can verify this as well using the same
> process.
> 
> Users that want the additional sources can then do it locally via
> above-mentioned script, which would be part of the now-signed tarball
> and so could be 'trusted' assuming the individual project hosts weren't
> compromised (which is still an assumption that's needed with the current
> process anyway).
> 
> I guess the main question is who is using the ROM/BIOS sources in the
> tarballs, and would this 2-step process work for those users? If there
> are distros relying on them then maybe there are some more logistics to
> consider since the make-release downloads are both time-consuming and
> prone to network errors/stalls since it relies on the availability of a
> good number of different hosts.

Great, maybe Gerd can comment on splitting out firmware.

QEMU mirrors firmware sources on GitLab too. We're able to provide the
same level of download availability on our mirror firmware repos as for
the main qemu.git repo.

I think qemu.git should include information about where to get the
firmware sources in order to comply with the GPL. I'm not sure if the
git submodules metadata counts from a GPL perspective, but that seems
enough since it has both a mirror repo URL and the exact hash used to
build the binary blob.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]