Re: [PATCH 07/12] testing: update ubuntu2004 to ubuntu2204

["Daniel P. Berrangé" <berrange@redhat.com>]

On Fri, Feb 17, 2023 at 11:35:44AM -0500, John Snow wrote:
> On Thu, Feb 16, 2023, 2:44 PM Daniel P. Berrangé <berrange@redhat.com>
> wrote:
> 
> > On Thu, Feb 16, 2023 at 01:15:30PM -0500, John Snow wrote:
> > > On Wed, Feb 15, 2023 at 2:25 PM Alex Bennée <alex.bennee@linaro.org>
> > wrote:
> > > >
> > > > The 22.04 LTS release has been out for almost a year now so its time
> > > > to update all the remaining images to the current LTS. We can also
> > > > drop some hacks we need for older clang TSAN support.
> > >
> > > We still support Ubuntu 20.04 until 2024 though, don't we? Is it safe
> > > to not test this platform?
> > >
> > > I've long been uncertain about what our policy actually is for docker
> > > tests, if we want to test every platform we support or only some of
> > > them; and if it's only some of them, when do we choose the older and
> > > when do we choose the newer?
> >
> > Ideally we would test both the oldest & newest versions of each
> > distro we support. Practically though, we're compromised by the
> > limited CI resources available.
> >
> 
> Yes, understood.
> 
> 
> > Dropping older Ubuntu images is a reasonable tradeoff, since we
> > still have Debian images covered in CI. Debian can be thought
> > of as an older version of Ubuntu to some extent, giving coverage
> > that will mitigate the risks of dropping 20.04.
> >
> 
> Okay, I'll take your word for that. I am not personally familiar with how
> much those distros diverge; I know Ubuntu is debian-based but that's the
> extent of my knowledge as I don't daily-drive either.
> 
> So, firstly:
> 
> Reviewed-by: John Snow <jsnow@redhat.com>
> 
> because I suspect we all have our reasons and I also agree testing newer is
> generally of higher value than testing older.
> 
> However, would it be possible to keep the older Ubuntu test as a manual
> execution that we could invoke at will, only during RC testing phase? If
> it's not a lot of work, I could even check that in myself as a follow-up if
> it isn't unwanted.
> 
> I find that "oldest version of x" is quite useful to me for testing Python
> stuff in particular, as that ecosystem moves pretty fast. It'd be mighty
> convenient to me in particular to keep an old Ubuntu test around to run
> manually as needed.
> 
> (Heck, even if it wasn't on CI at all but was just a container I could run
> locally, that would still be quite useful.)
> 
> Whaddaya think?

It would be pretty trivial to have tests/docker/dockerfiles contain
Dockerfiles for *every* supported distro version we have, and then
only build & test a subset in CI. It would merely suggest that we
change our naming convention so the dockerfiles in that dir include
the version. Basically adopting the standard libvirt-ci naming
convention for targets of $OSNAME-$OSVERSION:

$ lcitool targets
almalinux-8
almalinux-9
alpine-315
alpine-316
alpine-edge
centos-stream-8
centos-stream-9
debian-10
debian-11
debian-sid
fedora-36
fedora-37
fedora-rawhide
freebsd-12
freebsd-13
freebsd-current
macos-12
macos-13
opensuse-leap-153
opensuse-leap-154
opensuse-tumbleweed
ubuntu-1804
ubuntu-2004
ubuntu-2204

Contributors can then use 'make docker-XXXX' to run build tests
locally on specific distros when they need to test something
that isn't covered by default in out gating CI


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|