Re: [PULL 00/17] Net patches

["Daniel P. Berrangé" <berrange@redhat.com>]

On Mon, Sep 18, 2023 at 09:36:10PM +0200, Ilya Maximets wrote:
> On 9/14/23 10:13, Daniel P. Berrangé wrote:
> > On Wed, Sep 13, 2023 at 08:46:42PM +0200, Ilya Maximets wrote:
> >> On 9/8/23 16:15, Daniel P. Berrangé wrote:
> >>> On Fri, Sep 08, 2023 at 04:06:35PM +0200, Ilya Maximets wrote:
> >>>> On 9/8/23 14:15, Daniel P. Berrangé wrote:
> >>>>> On Fri, Sep 08, 2023 at 02:00:47PM +0200, Ilya Maximets wrote:
> >>>>>> On 9/8/23 13:49, Daniel P. Berrangé wrote:
> >>>>>>> On Fri, Sep 08, 2023 at 01:34:54PM +0200, Ilya Maximets wrote:
> >>>>>>>> On 9/8/23 13:19, Stefan Hajnoczi wrote:
> >>>>>>>>> Hi Ilya and Jason,
> >>>>>>>>> There is a CI failure related to a missing Debian libxdp-dev package:
> >>>>>>>>> https://gitlab.com/qemu-project/qemu/-/jobs/5046139967
> >>>>>>>>>
> >>>>>>>>> I think the issue is that the debian-amd64 container image that QEMU
> >>>>>>>>> uses for testing is based on Debian 11 ("bullseye" aka "oldstable")
> >>>>>>>>> and libxdp is not available on that release:
> >>>>>>>>> https://packages.debian.org/search?keywords=libxdp&searchon=names&suite=oldstable&section=all
> >>>>>>>>
> >>>>>>>> Hmm.  Sorry about that.
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> If we need to support Debian 11 CI then either XDP could be disabled
> >>>>>>>>> for that distro or libxdp could be compiled from source.
> >>>>>>>>
> >>>>>>>> I'd suggest we just remove the attempt to install the package for now,
> >>>>>>>> building libxdp from sources may be a little painful to maintain.
> >>>>>>>>
> >>>>>>>> Can be re-added later once distributions with libxdp 1.4+ will be more
> >>>>>>>> widely available, i.e. when fedora dockerfile will be updated to 39,
> >>>>>>>> for example.  That should be soon-ish, right?
> >>>>>>>
> >>>>>>> If you follow the process in docs/devel/testing.rst for adding
> >>>>>>> libxdp in libvirt-ci, then lcitool will "do the right thing"
> >>>>>>> when we move the auto-generated dockerfiles to new distro versions.
> >>>>>>
> >>>>>> Thanks!  I'll prepare changes for libvirt-ci.
> >>>>>>
> >>>>>> In the meantime, none of the currently tested images will have a required
> >>>>>> version of libxdp anyway, so I'm suggesting to just drop this one dockerfile
> >>>>>> modification from the patch.  What do you think?
> >>>>>
> >>>>> Sure, if none of the distros have it, then lcitool won't emit the
> >>>>> dockerfile changes until we update the inherited distro version.
> >>>>> So it is sufficient to just update libvirt-ci.git with the mappings.yml
> >>>>> info for libxdp, and add 'libxdp' to the tests/lcitool/projects/qemu.yml
> >>>>> file in qemu.git. It will then 'just work' when someone updates the
> >>>>> distro versions later.
> >>>>
> >>>> I posted an MR for libvirt-ci adding libxdp:
> >>>>   https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/429
> >>>>
> >>>> Please, take a look.
> >>>>
> >>>> The docs say that CI will try to build containers with the MR changes,
> >>>> but I don't think anything except sanity checks is actually tested on MR.
> >>>> Sorry if I missed something, never used GitLab pipelines before.
> >>>
> >>> No, that's our fault - we've broken the CI and your change alerted
> >>> me to that fact :-)
> >>>
> >>>> Note that with this update we will be installing older version of libxdp
> >>>> in many containers, even though they will not be used by QEMU, unless
> >>>> they are newer than 1.4.0.
> >>>
> >>> No problem, as it means QEMU CI will demonstrate the the meson.build
> >>> change is ignoring the outdatd libxdp.
> >>>
> >>>> tests/lcitool/projects/qemu.yml in qemu.git cannot be updated without
> >>>> updating a submodule after the MR merge.
> >>>
> >>> Yep.
> >>
> >> Since all the required changes went into libvirt-ci project, I posted an
> >> updated patch set named:
> >>
> >>   '[PATCH v4 0/2] net: add initial support for AF_XDP network backend'
> >>
> >> Please, take a look.
> >>
> >> This should fix the CI issues, though I'm not sure how to run QEMU gitlab
> >> pipelines myself, so I didn't actually test all the images.
> > 
> >   git push gitlab  -o ci.variable=QEMU_CI=2
> > 
> > will create pipeline and immediately run all jobs.
> 
> Thanks!  That worked.  Though I wasn't able to test much anyway as
> this thing burned through all my free compute credits less than
> half way through the pipeline. :D
> 
> So, AFAIU, it's not something an occasional contributor like me can
> use, unless they are spending their own money.

That is not the expected behaviour.

If your repo is a fork of https://gitlab.com/qemu-project/qemu it
should benefit from a *massive* x125 reduction on CI costs.

The critical thing is that it *MUST* have been created with the
'Fork' button on qemu-project/qemu. If that's not the case then
you will burn CI credits at a cost of 1 minute == 1 credit,
instead of 1 minute == 0.008 credits. Check this by going to
the top page of your repo, and looking for a box a little above
the file list, that says

    "Forked from QEMU / QEMU"

If that is not the case, then you'll have to rename your existing
repo to get it out of the way, and then use the 'Fork' button to
create a new copy that is tracked as a fork.

With most accounts getting 400 CI minutes per month, an averge
QEMU CI run should consume about 7 CI minutes.

NB, CI credits reset on the 1st of each month

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|