Re: [PATCH 2/4] tests: rearrange suites for I/O tests

["Daniel P. Berrangé" <berrange@redhat.com>]

On Wed, Oct 15, 2025 at 03:47:26PM +0200, Thomas Huth wrote:
> On 15/10/2025 15.44, Daniel P. Berrangé wrote:
> > On Tue, Oct 14, 2025 at 10:26:49AM +0200, Kevin Wolf wrote:
> > > Am 08.10.2025 um 13:35 hat Daniel P. Berrangé geschrieben:
> > > > Currently there are 5 block formats, each classified as either quick,
> > > > slow or thorough. This classification then determines what tests we
> > > > add to meson suites. The suites are as follows
> > > > 
> > > >    * block
> > > > 
> > > >      => tests listed by 'check -g auto -n' for 'quick' formats
> > > > 
> > > >    * slow, block-slow
> > > > 
> > > >      => tests listed by 'check -n' for 'slow' formats or 'check -g auto -n'
> > > >         for 'quick' formats
> > > > 
> > > >    * thorough, block-thorough
> > > > 
> > > >      => tests listed by 'check -n' for 'thorough' formats
> > > > 
> > > > The pairs of suites 'slow' / 'block-slow' and 'thorough' / 'block-thorough'
> > > > match in terms of what tests are enabled. The only difference is whether
> > > > non-block related tests are also in the suite.
> > > > 
> > > > There are two problems with this
> > > > 
> > > >   * If a format is classified as 'quick', we don't expose any
> > > >     meson suite for running *all* tests, only the 'auto' tests.
> > > > 
> > > >     eg there is no suite to run all qcow2 tests, only 'quick'
> > > >     tests can be run via meson, even if using 'SPEED=slow' we
> > > >     still filter to only 'auto' tests.
> > > > 
> > > >   * There is no suite that allows running all tests for a given
> > > >     format.
> > > > 
> > > >     eg there is no suite to run only 'raw' tests - you can only
> > > >     use 'block-slow' which runs both raw and "auto" qcow2 tests.
> > > > 
> > > >     eg there is no suite to run only 'vpc' tests - you can only
> > > >     use 'block-thorough' which runs qed, vmdk & vpc tests.
> > > > 
> > > > This patch suggests that 'block-slow' and 'block-thorough' are
> > > > not actually compelling use cases, and should be dropped. ie it
> > > > is not expected that people need to run all VPC, VMDK and QED
> > > > tests at the same time. Instead a more useful feature is the
> > > > ability to run all tests for a given format. Further the 'auto'
> > > > filtering should only apply in the default 'block' target/suite.
> > > > 
> > > > IOW, with this patch we get the follows meson suites:
> > > > 
> > > >   * 'block' - 'auto' tests for any format listed as 'quick'
> > > >               Currently just qcow2 'auto' tests
> > > >   * 'block-$FORMAT' - ALL tests for the given $FORMAT, for each
> > > >                       of qcow2, raw, qed, vmdk & vpc
> > > >   * 'slow' - ALL tests for formats tagged with 'quick' or 'slow'
> > > >   * 'thorough' - ALL tests formats tagged with 'thorough'
> > > > 
> > > > This corresponds to the following make targets.
> > > > 
> > > >   * 'make check-block'
> > > > 
> > > >      => runs only 'auto' qcow2 tests  (unchanged)
> > > > 
> > > >   * 'make check-block SPEED=thorough'
> > > > 
> > > >      => runs all 'qed', 'vmdk', 'vpc' tests (unchanged)
> > > 
> > > Also all qcow2 and raw tests, right?
> > 
> > Sigh, I made a mistake here, this should read:
> >    * 'make check' / 'make check-block'
> >       => runs only 'auto' qcow2 tests  (unchanged)
> > 
> > The 'make check-block SPEED=thorough/slow' feature was removed
> > on the basis that running all targets at once is not that
> > useful, instead the 'check-block-$FORMAT' targets are
> > replacing it. IOW the meson 'block-slow' and 'block-through'
> > suites are gone.
> 
> Can we still run all tests (i.e. iotests, qtests, unit tests and functional
> tests in parallel) at once with a simple "make check SPEED=thorough" ? If
> not, that would be a regression for me.

Yes, the tests are still added to the 'thorough' and 'slow' meson
suites, only the 'block-thorough' and 'block-slow' suites were
removed.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|