IDE failures under netbsd unit tests

IDE failures under netbsd unit tests

[John Snow]

Heya Denis;

In the process of upgrading Python to 3.12, I found that our netbsd
tests were a little broken - I will post patches soon - but if you add
"bison" to the list of deps in the netbsd vm test file, you can get
far enough to see this:

▶  163/1096 ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion
failed: (ret == 0)     ERROR
 163/1096 qtest+qtest-i386 - qemu:qtest-i386/ide-test
                     ERROR            0.01s   killed by signal 6
SIGABRT
>>> G_TEST_DBUS_DAEMON=/home/qemu/qemu-test.kgtky7/src/tests/dbus-daemon.sh ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 MALLOC_PERTURB_=224 UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 QTEST_QEMU_BINARY=./qemu-system-i386 PYTHON=/home/qemu/qemu-test.kgtky7/build/pyvenv/bin/python3.13 QTEST_QEMU_IMG=./qemu-img MESON_TEST_ITERATION=1 MSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 RUST_BACKTRACE=1 /home/qemu/qemu-test.kgtky7/build/tests/qtest/ide-test --tap -k

Maybe something that cropped up due to recent changes? I am not sure,
but figured I'd give you a heads up.

--js

[PATCH 0/7] tests/qtest: fix the disk tests on a host with a small /tmp

[Denis V. Lunev]

John reported ide-test dying at startup in the NetBSD VM:

  ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion failed:
  (ret == 0)

The hypothesys is that the ftruncate() of a 64 MiB scratch image. NetBSD
mounts /tmp as a tmpfs sized at 25% of RAM and charges a file its full
length themoment it is extended, so a sparse image is not free there and
the call returns ENOSPC. The assert dates to 2013, nothing regressed.

The series is designed to do two things:
* reduce the amount of space needed switching to QCOW2
* tolerate the failure skipping the test if there is not enough space

Patches 1-4 skip instead of aborting, the way hd-geo-test has since
a0d6d7454a. Patches 5-7 build the images with qemu-img where only the
virtual size matters: peak scratch space goes from 129 MiB to 1.6 MiB
for ide-test and from 4126 MiB to 1.5 MiB for hd-geo-test.

Hope this should help. John, can you give a try to this in the test?
I do not have NetBSD handy and will start with this tomorrow if you
will share guest config.

Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>

Denis V. Lunev (7):
  tests/qtest/libqos: let mkqcow2() report failure
  tests/qtest/ide-test: skip when the scratch files cannot be created
  tests/qtest/ahci-test: skip when the scratch files cannot be created
  tests/qtest/hd-geo-test: skip when the scratch file cannot be created
  tests/qtest/libqtest: create images with a byte-precise size
  tests/qtest/ide-test: build the shared disks with qemu-img
  tests/qtest/hd-geo-test: build the test images with qemu-img

 tests/qtest/ahci-test.c        |  48 ++++++++---
 tests/qtest/hd-geo-test.c      | 145 ++++++++++++++++++++++-----------
 tests/qtest/ide-test.c         |  96 +++++++++++++++++-----
 tests/qtest/libqos/libqos.c    |   4 +-
 tests/qtest/libqos/libqos.h    |   2 +-
 tests/qtest/libqtest.c         |   9 +-
 tests/qtest/libqtest.h         |  12 +++
 tests/qtest/virtio-scsi-test.c |   5 +-
 8 files changed, 237 insertions(+), 84 deletions(-)

-- 
2.53.0

[PATCH 1/7] tests/qtest/libqos: let mkqcow2() report failure

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

mkqcow2() asserts that qemu-img succeeded, so a caller that has
qemu-img but cannot use it aborts the binary rather than skipping.
Return the result instead. virtio-scsi-test takes the skip path it
already has for a missing qemu-img; ahci-test grows one two patches
on.

Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 tests/qtest/libqos/libqos.c    | 4 ++--
 tests/qtest/libqos/libqos.h    | 2 +-
 tests/qtest/virtio-scsi-test.c | 5 ++++-
 3 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/tests/qtest/libqos/libqos.c b/tests/qtest/libqos/libqos.c
index 9b49d0d4dd..ff0d595346 100644
--- a/tests/qtest/libqos/libqos.c
+++ b/tests/qtest/libqos/libqos.c
@@ -138,9 +138,9 @@ void migrate(QOSState *from, QOSState *to, const char *uri)
     migrate_allocator(&from->alloc, &to->alloc);
 }
 
-void mkqcow2(const char *file, unsigned size_mb)
+bool mkqcow2(const char *file, unsigned size_mb)
 {
-    g_assert_true(mkimg(file, "qcow2", size_mb));
+    return mkimg(file, "qcow2", size_mb);
 }
 
 void prepare_blkdebug_script(const char *debug_fn, const char *event)
diff --git a/tests/qtest/libqos/libqos.h b/tests/qtest/libqos/libqos.h
index c04950e2b1..8390d4fa98 100644
--- a/tests/qtest/libqos/libqos.h
+++ b/tests/qtest/libqos/libqos.h
@@ -27,7 +27,7 @@ QOSState *qtest_boot(QOSOps *ops, const char *cmdline_fmt, ...)
     G_GNUC_PRINTF(2, 3);
 void qtest_common_shutdown(QOSState *qs);
 void qtest_shutdown(QOSState *qs);
-void mkqcow2(const char *file, unsigned size_mb);
+bool mkqcow2(const char *file, unsigned size_mb);
 void migrate(QOSState *from, QOSState *to, const char *uri);
 void prepare_blkdebug_script(const char *debug_fn, const char *event);
 void generate_pattern(void *buffer, size_t len, size_t cycle_len);
diff --git a/tests/qtest/virtio-scsi-test.c b/tests/qtest/virtio-scsi-test.c
index e2350c52f6..bce6c11813 100644
--- a/tests/qtest/virtio-scsi-test.c
+++ b/tests/qtest/virtio-scsi-test.c
@@ -292,7 +292,10 @@ static void test_iothread_attach_node(void *obj, void *data,
         goto fail;
     }
 
-    mkqcow2(tmp_path, 64);
+    if (!mkqcow2(tmp_path, 64)) {
+        g_test_message("Could not create the qcow2 overlay");
+        goto fail;
+    }
 
     /* Attach the overlay to the null0 node */
     qtest_qmp_assert_success(scsi_pci->pci_vdev.pdev->bus->qts,
-- 
2.53.0

[PATCH 2/7] tests/qtest/ide-test: skip when the scratch files cannot be created

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

main() creates a blkdebug script and two 64 MiB images and asserts
that every step worked. The images are sparse, which is free on Linux
but not everywhere: NetBSD mounts /tmp as a tmpfs sized at 25% of RAM
and charges a file its full length the moment it is extended, so the
ftruncate() returns ENOSPC and the binary dies before a test has run:

  ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion failed:
  (ret == 0)

A file system with no room left fails the g_mkstemp() calls too, the
blkdebug script first of all.

Skip instead, as hd-geo-test does since commit a0d6d7454a
("tests/hd-geo-test: Skip test when images can not be created"): move
g_test_init() ahead of the setup, report the errno and register no
tests. The cleanup loop has to cope with a path never filled in.

Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Reported-by: John Snow <jsnow@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 tests/qtest/ide-test.c | 29 +++++++++++++++++++++++------
 1 file changed, 23 insertions(+), 6 deletions(-)

diff --git a/tests/qtest/ide-test.c b/tests/qtest/ide-test.c
index b36b0f8875..805afb9461 100644
--- a/tests/qtest/ide-test.c
+++ b/tests/qtest/ide-test.c
@@ -1231,6 +1231,7 @@ int main(int argc, char **argv)
     int i;
     int fd;
     int ret;
+    int err;
 
     /*
      * "base" stores the starting point where we create temporary files.
@@ -1245,25 +1246,37 @@ int main(int argc, char **argv)
     base = ".";
 #endif
 
+    g_test_init(&argc, &argv, NULL);
+
     /* Create temporary blkdebug instructions */
     debug_path = g_strdup_printf("%s/qtest-blkdebug.XXXXXX", base);
     fd = g_mkstemp(debug_path);
-    g_assert(fd >= 0);
+    if (fd < 0) {
+        g_test_message("Could not create %s: %s", debug_path,
+                       strerror(errno));
+        goto test_add_done;
+    }
     close(fd);
 
     /* Create a temporary raw image */
     for (i = 0; i < 2; ++i) {
         tmp_path[i] = g_strdup_printf("%s/qtest.XXXXXX", base);
         fd = g_mkstemp(tmp_path[i]);
-        g_assert(fd >= 0);
+        if (fd < 0) {
+            g_test_message("Could not create %s: %s", tmp_path[i],
+                           strerror(errno));
+            goto test_add_done;
+        }
         ret = ftruncate(fd, TEST_IMAGE_SIZE);
-        g_assert(ret == 0);
+        err = errno;
         close(fd);
+        if (ret < 0) {
+            g_test_message("Could not size %s: %s", tmp_path[i],
+                           strerror(err));
+            goto test_add_done;
+        }
     }
 
-    /* Run the tests */
-    g_test_init(&argc, &argv, NULL);
-
     qtest_add_func("/ide/read_native", test_specify);
 
     qtest_add_func("/ide/identify", test_identify);
@@ -1288,10 +1301,14 @@ int main(int argc, char **argv)
     qtest_add_func("/ide/cdrom/pio_raw", test_cdrom_pio_raw);
     qtest_add_func("/ide/cdrom/dma_raw", test_cdrom_dma_raw);
 
+test_add_done:
     ret = g_test_run();
 
     /* Cleanup */
     for (i = 0; i < 2; ++i) {
+        if (!tmp_path[i]) {
+            continue;
+        }
         unlink(tmp_path[i]);
         g_free(tmp_path[i]);
     }
-- 
2.53.0

[PATCH 3/7] tests/qtest/ahci-test: skip when the scratch files cannot be created

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

The fallback ahci-test takes when QTEST_QEMU_IMG is unset extends a
raw image to 64 MiB with ftruncate() and asserts it worked, which is
the assert ide-test just lost. The temporary files are asserted on
too, and a file system with no room left fails g_mkstemp() and
g_file_open_tmp() as surely as the ftruncate().

Report the reason and register no tests instead. The cleanup path has
to cope with the files now created after the point of no return.

Cc: Fabiano Rosas <farosas@suse.de>
Cc: John Snow <jsnow@redhat.com>
Cc: Laurent Vivier <lvivier@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 tests/qtest/ahci-test.c | 48 +++++++++++++++++++++++++++++++----------
 1 file changed, 37 insertions(+), 11 deletions(-)

diff --git a/tests/qtest/ahci-test.c b/tests/qtest/ahci-test.c
index 71d23fe56a..d6af5b30a0 100644
--- a/tests/qtest/ahci-test.c
+++ b/tests/qtest/ahci-test.c
@@ -2110,8 +2110,10 @@ static void create_ahci_io_test(enum IOMode type, enum AddrMode addr,
 
 int main(int argc, char **argv)
 {
+    g_autoptr(GError) error = NULL;
     const char *arch, *base;
     int ret;
+    int err;
     int fd;
     int c;
     int i, j, k, m;
@@ -2164,30 +2166,49 @@ int main(int argc, char **argv)
     /* Create a temporary image */
     tmp_path = g_strdup_printf("%s/qtest.XXXXXX", base);
     fd = g_mkstemp(tmp_path);
-    g_assert(fd >= 0);
+    if (fd < 0) {
+        g_test_message("Could not create %s: %s", tmp_path, strerror(errno));
+        goto test_add_done;
+    }
     if (have_qemu_img()) {
         imgfmt = "qcow2";
         test_image_size_mb = TEST_IMAGE_SIZE_MB_LARGE;
-        mkqcow2(tmp_path, TEST_IMAGE_SIZE_MB_LARGE);
+        close(fd);
+        if (!mkqcow2(tmp_path, TEST_IMAGE_SIZE_MB_LARGE)) {
+            g_test_message("Could not create %s with qemu-img", tmp_path);
+            goto test_add_done;
+        }
     } else {
         g_test_message("QTEST_QEMU_IMG not set or qemu-img missing; "
                        "skipping LBA48 high-sector tests");
         imgfmt = "raw";
         test_image_size_mb = TEST_IMAGE_SIZE_MB_SMALL;
         ret = ftruncate(fd, test_image_size_mb * 1024 * 1024);
-        g_assert(ret == 0);
+        err = errno;
+        close(fd);
+        if (ret < 0) {
+            g_test_message("Could not size %s: %s", tmp_path, strerror(err));
+            goto test_add_done;
+        }
     }
-    close(fd);
 
     /* Create temporary blkdebug instructions */
     debug_path = g_strdup_printf("%s/qtest-blkdebug.XXXXXX", base);
     fd = g_mkstemp(debug_path);
-    g_assert(fd >= 0);
+    if (fd < 0) {
+        g_test_message("Could not create %s: %s", debug_path,
+                       strerror(errno));
+        goto test_add_done;
+    }
     close(fd);
 
     /* Reserve a hollow file to use as a socket for migration tests */
-    fd = g_file_open_tmp("qtest-migration.XXXXXX", &mig_socket, NULL);
-    g_assert(fd >= 0);
+    fd = g_file_open_tmp("qtest-migration.XXXXXX", &mig_socket, &error);
+    if (fd < 0) {
+        g_test_message("Could not create a temporary file: %s",
+                       error->message);
+        goto test_add_done;
+    }
     close(fd);
 
     /* Run the tests */
@@ -2244,15 +2265,20 @@ int main(int argc, char **argv)
     qtest_add_func("/ahci/cdrom/drain/pio", test_atapi_drain_pio);
     qtest_add_func("/ahci/cdrom/drain/dma", test_atapi_drain_dma);
 
+test_add_done:
     ret = g_test_run();
 
     /* Cleanup */
     unlink(tmp_path);
     g_free(tmp_path);
-    unlink(debug_path);
-    g_free(debug_path);
-    unlink(mig_socket);
-    g_free(mig_socket);
+    if (debug_path) {
+        unlink(debug_path);
+        g_free(debug_path);
+    }
+    if (mig_socket) {
+        unlink(mig_socket);
+        g_free(mig_socket);
+    }
 
     return ret;
 }
-- 
2.53.0

[PATCH 4/7] tests/qtest/hd-geo-test: skip when the scratch file cannot be created

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

Commit a0d6d7454a ("tests/hd-geo-test: Skip test when images can not
be created") skips when the 4 GiB image cannot be sized, but the
temporary file itself is still asserted on, and a file system with no
room left fails g_file_open_tmp() as surely as the ftruncate().

Skip there too and say why in both cases, from inside
create_test_img() where the GError is. Unlink the image that could not
be sized while here.

Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 tests/qtest/hd-geo-test.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/tests/qtest/hd-geo-test.c b/tests/qtest/hd-geo-test.c
index 41481a5e09..e3b4a4db31 100644
--- a/tests/qtest/hd-geo-test.c
+++ b/tests/qtest/hd-geo-test.c
@@ -27,15 +27,24 @@
 
 static char *create_test_img(int secs)
 {
+    g_autoptr(GError) error = NULL;
     char *template;
-    int fd, ret;
+    int fd, ret, err;
+
+    fd = g_file_open_tmp("qtest.XXXXXX", &template, &error);
+    if (fd < 0) {
+        g_test_message("Could not create a temporary file: %s",
+                       error->message);
+        return NULL;
+    }
 
-    fd = g_file_open_tmp("qtest.XXXXXX", &template, NULL);
-    g_assert(fd >= 0);
     ret = ftruncate(fd, (off_t)secs * 512);
+    err = errno;
     close(fd);
 
     if (ret) {
+        g_test_message("Could not size %s: %s", template, strerror(err));
+        unlink(template);
         g_free(template);
         template = NULL;
     }
@@ -1059,7 +1068,6 @@ int main(int argc, char **argv)
         if (img_secs[i] >= 0) {
             img_file_name[i] = create_test_img(img_secs[i]);
             if (!img_file_name[i]) {
-                g_test_message("Could not create test images.");
                 goto test_add_done;
             }
         } else {
-- 
2.53.0

[PATCH 5/7] tests/qtest/libqtest: create images with a byte-precise size

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

mkimg() takes megabytes, which is all its callers have needed so far.
ide-test wants an image whose length is an exact multiple of its CHS
geometry, 130 * 16 * 63 sectors, and that is not a whole number of
megabytes. Add mkimg_bytes() and let mkimg() go through it.

Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 tests/qtest/libqtest.c |  9 +++++++--
 tests/qtest/libqtest.h | 12 ++++++++++++
 2 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c
index bec37c71b8..1ae004e6f3 100644
--- a/tests/qtest/libqtest.c
+++ b/tests/qtest/libqtest.c
@@ -2115,6 +2115,11 @@ bool have_qemu_img(void)
 }
 
 bool mkimg(const char *file, const char *fmt, unsigned size_mb)
+{
+    return mkimg_bytes(file, fmt, (uint64_t)size_mb * 1024 * 1024);
+}
+
+bool mkimg_bytes(const char *file, const char *fmt, uint64_t size)
 {
     gchar *cli;
     bool ret;
@@ -2133,8 +2138,8 @@ bool mkimg(const char *file, const char *fmt, unsigned size_mb)
         return false;
     }
 
-    cli = g_strdup_printf("%s create -f %s %s %uM", qemu_img_abs_path,
-                          fmt, file, size_mb);
+    cli = g_strdup_printf("%s create -f %s %s %" PRIu64, qemu_img_abs_path,
+                          fmt, file, size);
     ret = g_spawn_command_line_sync(cli, &out, &out2, &rc, &err);
     if (err || !g_spawn_check_exit_status(rc, &err)) {
         fprintf(stderr, "%s\n", err->message);
diff --git a/tests/qtest/libqtest.h b/tests/qtest/libqtest.h
index 45217fb8dc..6720ca2de2 100644
--- a/tests/qtest/libqtest.h
+++ b/tests/qtest/libqtest.h
@@ -1186,6 +1186,18 @@ bool have_qemu_img(void);
  */
 bool mkimg(const char *file, const char *fmt, unsigned size_mb);
 
+/**
+ * mkimg_bytes:
+ * @file: File name of the image that should be created
+ * @fmt: Format, e.g. "qcow2" or "raw"
+ * @size: Size of the image in bytes
+ *
+ * As mkimg(), for images whose size is not a whole number of megabytes.
+ *
+ * Returns: true if the image has been created successfully.
+ */
+bool mkimg_bytes(const char *file, const char *fmt, uint64_t size);
+
 /**
  * qtest_verbose:
  * @domain: The logging domain
-- 
2.53.0

[PATCH 6/7] tests/qtest/ide-test: build the shared disks with qemu-img

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

The two 64 MiB disks are raw and are grown with ftruncate(), which is
free only where a sparse file is free. Build them as qcow2 when
qemu-img is available, the way ahci-test and hd-geo-test do, keeping
the raw ftruncate() as the fallback. Only the virtual size matters to
their users: the CHS geometry READ NATIVE MAX ADDRESS reports and the
end-of-disk sector the TRIM range check refuses.

Three tests do care what sits below the guest and get a raw 1 MiB
image of their own:

  - the CD-ROM tests write their ATAPI pattern in with fopen()
  - bmdma/trim_reset needs the guest TRIM to reach blkdebug, where a
    1 ms delay lets the reset catch the request in flight. qcow2 stops
    the discard at the format layer, the clusters never having been
    allocated, and the test would pass while racing nothing
  - the two flush tests trip a blkdebug event. Both pass over qcow2,
    but on raw the event reported is the flush the guest issued

Peak scratch space drops from 129 MiB to 1.6 MiB.

Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 tests/qtest/ide-test.c | 69 +++++++++++++++++++++++++++++++++---------
 1 file changed, 54 insertions(+), 15 deletions(-)

diff --git a/tests/qtest/ide-test.c b/tests/qtest/ide-test.c
index 805afb9461..89590e66cc 100644
--- a/tests/qtest/ide-test.c
+++ b/tests/qtest/ide-test.c
@@ -37,6 +37,12 @@
 /* Specified by ATA (physical) CHS geometry for ~64 MiB device.  */
 #define TEST_IMAGE_SIZE ((130 * 16 * 63) * 512)
 
+/*
+ * The tests that reach the image behind QEMU's back, or that need an
+ * event to reach blkdebug, get a raw one of their own.
+ */
+#define RAW_IMAGE_SIZE (1 * 1024 * 1024)
+
 #define IDE_PCI_DEV     1
 #define IDE_PCI_FUNC    1
 
@@ -131,7 +137,9 @@ static QPCIBus *pcibus = NULL;
 static QGuestAllocator guest_malloc;
 
 static char *tmp_path[2];
+static char *raw_path;
 static char *debug_path;
+static const char *imgfmt = "raw";
 
 G_GNUC_PRINTF(1, 2)
 static QTestState *ide_test_start(const char *cmdline_fmt, ...)
@@ -348,9 +356,9 @@ static QTestState *test_bmdma_setup(void)
     QTestState *qts;
 
     qts = ide_test_start(
-        "-drive file=%s,if=ide,cache=writeback,format=raw "
+        "-drive file=%s,if=ide,cache=writeback,format=%s "
         "-global ide-hd.serial=%s -global ide-hd.ver=%s",
-        tmp_path[0], "testdisk", "version");
+        tmp_path[0], imgfmt, "testdisk", "version");
     qtest_irq_intercept_in(qts, "ioapic");
 
     return qts;
@@ -506,7 +514,7 @@ static void test_bmdma_trim_reset(void)
         "inject-error.0.event=none,inject-error.0.iotype=discard,"
         "inject-error.0.errno=0,inject-error.0.delay-ns=1000000 "
         "-device ide-hd,drive=dbg,bus=ide.0",
-        tmp_path[0]);
+        raw_path);
     qtest_irq_intercept_in(qts, "ioapic");
 
     guest_buf = guest_alloc(&guest_malloc, len);
@@ -664,9 +672,9 @@ static void test_specify(void)
     uint8_t heads, spt;
 
     qts = ide_test_start(
-        "-blockdev driver=file,node-name=hda,filename=%s "
+        "-drive if=none,id=hda,format=%s,file=%s "
         "-device ide-hd,drive=hda,bus=ide.0,unit=0 ",
-        tmp_path[0]);
+        imgfmt, tmp_path[0]);
 
     dev = get_pci_device(qts, &bmdma_bar, &ide_bar);
 
@@ -706,9 +714,9 @@ static void test_identify(void)
     int ret;
 
     qts = ide_test_start(
-        "-drive file=%s,if=ide,cache=writeback,format=raw "
+        "-drive file=%s,if=ide,cache=writeback,format=%s "
         "-global ide-hd.serial=%s -global ide-hd.ver=%s",
-        tmp_path[0], "testdisk", "version");
+        tmp_path[0], imgfmt, "testdisk", "version");
 
     dev = get_pci_device(qts, &bmdma_bar, &ide_bar);
 
@@ -756,11 +764,11 @@ static void test_diagnostic(void)
     uint8_t data;
 
     qts = ide_test_start(
-        "-blockdev driver=file,node-name=hda,filename=%s "
-        "-blockdev driver=file,node-name=hdb,filename=%s "
+        "-drive if=none,id=hda,format=%s,file=%s "
+        "-drive if=none,id=hdb,format=%s,file=%s "
         "-device ide-hd,drive=hda,bus=ide.0,unit=0 "
         "-device ide-hd,drive=hdb,bus=ide.0,unit=1 ",
-        tmp_path[0], tmp_path[1]);
+        imgfmt, tmp_path[0], imgfmt, tmp_path[1]);
 
     dev = get_pci_device(qts, &bmdma_bar, &ide_bar);
 
@@ -826,7 +834,7 @@ static void test_flush(void)
 
     qts = ide_test_start(
         "-drive file=blkdebug::%s,if=ide,cache=writeback,format=raw",
-        tmp_path[0]);
+        raw_path);
 
     dev = get_pci_device(qts, &bmdma_bar, &ide_bar);
 
@@ -877,7 +885,7 @@ static void test_pci_retry_flush(void)
     qts = ide_test_start(
         "-drive file=blkdebug:%s:%s,if=ide,cache=writeback,format=raw,"
         "rerror=stop,werror=stop",
-        debug_path, tmp_path[0]);
+        debug_path, raw_path);
 
     dev = get_pci_device(qts, &bmdma_bar, &ide_bar);
 
@@ -1100,14 +1108,14 @@ static void cdrom_read_impl(int nblocks, unsigned flags)
 
     /* Prepopulate the CDROM with an interesting pattern */
     generate_pattern(pattern, patt_len, ATAPI_BLOCK_SIZE);
-    fh = fopen(tmp_path[0], "wb+");
+    fh = fopen(raw_path, "wb+");
     ret = fwrite(pattern, ATAPI_BLOCK_SIZE, patt_blocks, fh);
     g_assert_cmpint(ret, ==, patt_blocks);
     fclose(fh);
 
     qts = ide_test_start(
             "-drive if=none,file=%s,media=cdrom,format=raw,id=sr0,index=0 "
-            "-device ide-cd,drive=sr0,bus=ide.0", tmp_path[0]);
+            "-device ide-cd,drive=sr0,bus=ide.0", raw_path);
     dev = get_pci_device(qts, &bmdma_bar, &ide_bar);
     qtest_irq_intercept_in(qts, "ioapic");
 
@@ -1258,7 +1266,11 @@ int main(int argc, char **argv)
     }
     close(fd);
 
-    /* Create a temporary raw image */
+    if (have_qemu_img()) {
+        imgfmt = "qcow2";
+    }
+
+    /* Create the disks the tests share */
     for (i = 0; i < 2; ++i) {
         tmp_path[i] = g_strdup_printf("%s/qtest.XXXXXX", base);
         fd = g_mkstemp(tmp_path[i]);
@@ -1267,6 +1279,15 @@ int main(int argc, char **argv)
                            strerror(errno));
             goto test_add_done;
         }
+        if (!strcmp(imgfmt, "qcow2")) {
+            close(fd);
+            if (!mkimg_bytes(tmp_path[i], imgfmt, TEST_IMAGE_SIZE)) {
+                g_test_message("Could not create %s with qemu-img",
+                               tmp_path[i]);
+                goto test_add_done;
+            }
+            continue;
+        }
         ret = ftruncate(fd, TEST_IMAGE_SIZE);
         err = errno;
         close(fd);
@@ -1277,6 +1298,20 @@ int main(int argc, char **argv)
         }
     }
 
+    raw_path = g_strdup_printf("%s/qtest.XXXXXX", base);
+    fd = g_mkstemp(raw_path);
+    if (fd < 0) {
+        g_test_message("Could not create %s: %s", raw_path, strerror(errno));
+        goto test_add_done;
+    }
+    ret = ftruncate(fd, RAW_IMAGE_SIZE);
+    err = errno;
+    close(fd);
+    if (ret < 0) {
+        g_test_message("Could not size %s: %s", raw_path, strerror(err));
+        goto test_add_done;
+    }
+
     qtest_add_func("/ide/read_native", test_specify);
 
     qtest_add_func("/ide/identify", test_identify);
@@ -1312,6 +1347,10 @@ test_add_done:
         unlink(tmp_path[i]);
         g_free(tmp_path[i]);
     }
+    if (raw_path) {
+        unlink(raw_path);
+        g_free(raw_path);
+    }
     unlink(debug_path);
     g_free(debug_path);
 
-- 
2.53.0

[PATCH 7/7] tests/qtest/hd-geo-test: build the test images with qemu-img

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

The two backing images are raw and are grown with ftruncate(), one to
30 MiB and one to 4 GiB, so on a host where sparse files are not free
the test skips and the BIOS geometry goes untested.

The override tests already build their images with qemu-img, so use
the same route here. The convert and resize pair moves into
mkqcow2_with_mbr(), which both users call and which reports a failed
qemu-img rather than asserting. setup_mbr() stamps the MBR by
rebuilding the image, that being the only way to reach sector 0 of a
qcow2 from outside QEMU. Only the setup running before any test can
skip, so the rest keep asserting. Without qemu-img the images stay
raw.

Peak scratch space drops from 4126 MiB to 1.5 MiB.

Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 tests/qtest/hd-geo-test.c | 129 +++++++++++++++++++++++++-------------
 1 file changed, 86 insertions(+), 43 deletions(-)

diff --git a/tests/qtest/hd-geo-test.c b/tests/qtest/hd-geo-test.c
index e3b4a4db31..160bdc7216 100644
--- a/tests/qtest/hd-geo-test.c
+++ b/tests/qtest/hd-geo-test.c
@@ -25,9 +25,69 @@
 
 #define ARGV_SIZE 256
 
+#define MBR_SIZE 512
+
+static const char *imgfmt = "raw";
+
+static bool mkqcow2_with_mbr(const char *path, const uint8_t *mbr,
+                             uint64_t size)
+{
+    g_autoptr(GError) error = NULL;
+    g_autofree char *raw_path = NULL;
+    char cmd[100 + 2 * PATH_MAX];
+    char *qemu_img_path;
+    char *qemu_img_abs_path = NULL;
+    bool ok = false;
+    int fd, ret;
+
+    fd = g_file_open_tmp("qtest.XXXXXX", &raw_path, &error);
+    if (fd < 0) {
+        g_test_message("Could not create a temporary file: %s",
+                       error->message);
+        return false;
+    }
+    ret = write(fd, mbr, MBR_SIZE);
+    g_assert(ret == MBR_SIZE);
+    close(fd);
+
+    qemu_img_path = getenv("QTEST_QEMU_IMG");
+    g_assert(qemu_img_path);
+    qemu_img_abs_path = realpath(qemu_img_path, NULL);
+    g_assert(qemu_img_abs_path);
+
+    ret = snprintf(cmd, sizeof(cmd),
+                   "%s convert -f raw -O qcow2 %s %s > /dev/null",
+                   qemu_img_abs_path,
+                   raw_path, path);
+    g_assert((0 < ret) && (ret <= sizeof(cmd)));
+    if (system(cmd) != 0) {
+        g_test_message("Could not create %s with qemu-img", path);
+        goto out;
+    }
+
+    ret = snprintf(cmd, sizeof(cmd),
+                   "%s resize %s %" PRIu64 " > /dev/null",
+                   qemu_img_abs_path,
+                   path, size);
+    g_assert((0 < ret) && (ret <= sizeof(cmd)));
+    if (system(cmd) != 0) {
+        g_test_message("Could not size %s with qemu-img", path);
+        goto out;
+    }
+
+    ok = true;
+
+out:
+    free(qemu_img_abs_path);
+    unlink(raw_path);
+
+    return ok;
+}
+
 static char *create_test_img(int secs)
 {
     g_autoptr(GError) error = NULL;
+    uint8_t blank_mbr[MBR_SIZE] = {};
     char *template;
     int fd, ret, err;
 
@@ -38,6 +98,16 @@ static char *create_test_img(int secs)
         return NULL;
     }
 
+    if (!strcmp(imgfmt, "qcow2")) {
+        close(fd);
+        if (!mkqcow2_with_mbr(template, blank_mbr, (uint64_t)secs * 512)) {
+            unlink(template);
+            g_free(template);
+            return NULL;
+        }
+        return template;
+    }
+
     ret = ftruncate(fd, (off_t)secs * 512);
     err = errno;
     close(fd);
@@ -207,7 +277,7 @@ static void setup_mbr(int img_idx, MBRcontents mbr)
         /* chs 0,1,1 (lba 63) to chs 7,15,63 (8001 sectors) */
         0x80, 1, 1, 0, 6,  15, 63, 7, 63, 0, 0, 0, 0x41, 0x1F, 0, 0,
     };
-    uint8_t buf[512];
+    uint8_t buf[MBR_SIZE];
     int fd, ret;
 
     memset(buf, 0, sizeof(buf));
@@ -218,6 +288,12 @@ static void setup_mbr(int img_idx, MBRcontents mbr)
         memcpy(buf + 0x1BE, mbr == mbr_lba ? part_lba : part_chs, 16);
     }
 
+    if (!strcmp(imgfmt, "qcow2")) {
+        g_assert_true(mkqcow2_with_mbr(img_file_name[img_idx], buf,
+                                       (uint64_t)img_secs[img_idx] * 512));
+        return;
+    }
+
     fd = open(img_file_name[img_idx], O_WRONLY);
     g_assert(fd >= 0);
     ret = write(fd, buf, sizeof(buf));
@@ -237,7 +313,8 @@ static int setup_ide(int argc, char *argv[], int argv_sz,
 
     if (img_secs[img_idx] >= 0) {
         setup_mbr(img_idx, mbr);
-        s3 = g_strdup_printf(",format=raw,file=%s", img_file_name[img_idx]);
+        s3 = g_strdup_printf(",format=%s,file=%s", imgfmt,
+                             img_file_name[img_idx]);
     } else {
         s3 = g_strdup(",media=cdrom");
     }
@@ -431,15 +508,10 @@ static MBRpartitions empty_mbr = { {false, 0, 0, 0, 0, 0, 0, 0, 0},
 
 static char *create_qcow2_with_mbr(MBRpartitions mbr, uint64_t sectors)
 {
-    g_autofree char *raw_path = NULL;
     char *qcow2_path;
-    char cmd[100 + 2 * PATH_MAX];
-    uint8_t buf[512] = {};
-    int i, ret, fd, offset;
-    uint64_t qcow2_size = sectors * 512;
+    uint8_t buf[MBR_SIZE] = {};
+    int i, fd, offset;
     uint8_t status, parttype, head, sector, cyl;
-    char *qemu_img_path;
-    char *qemu_img_abs_path;
 
     offset = 0xbe;
 
@@ -476,44 +548,11 @@ static char *create_qcow2_with_mbr(MBRpartitions mbr, uint64_t sectors)
         offset += 0x10;
     }
 
-    fd = g_file_open_tmp("qtest.XXXXXX", &raw_path, NULL);
-    g_assert(fd >= 0);
-    close(fd);
-
-    fd = open(raw_path, O_WRONLY);
-    g_assert(fd >= 0);
-    ret = write(fd, buf, sizeof(buf));
-    g_assert(ret == sizeof(buf));
-    close(fd);
-
     fd = g_file_open_tmp("qtest.XXXXXX", &qcow2_path, NULL);
     g_assert(fd >= 0);
     close(fd);
 
-    qemu_img_path = getenv("QTEST_QEMU_IMG");
-    g_assert(qemu_img_path);
-    qemu_img_abs_path = realpath(qemu_img_path, NULL);
-    g_assert(qemu_img_abs_path);
-
-    ret = snprintf(cmd, sizeof(cmd),
-                   "%s convert -f raw -O qcow2 %s %s > /dev/null",
-                   qemu_img_abs_path,
-                   raw_path, qcow2_path);
-    g_assert((0 < ret) && (ret <= sizeof(cmd)));
-    ret = system(cmd);
-    g_assert(ret == 0);
-
-    ret = snprintf(cmd, sizeof(cmd),
-                   "%s resize %s %" PRIu64 " > /dev/null",
-                   qemu_img_abs_path,
-                   qcow2_path, qcow2_size);
-    g_assert((0 < ret) && (ret <= sizeof(cmd)));
-    ret = system(cmd);
-    g_assert(ret == 0);
-
-    free(qemu_img_abs_path);
-
-    unlink(raw_path);
+    g_assert_true(mkqcow2_with_mbr(qcow2_path, buf, sectors * 512));
 
     return qcow2_path;
 }
@@ -1064,6 +1103,10 @@ int main(int argc, char **argv)
 
     g_test_init(&argc, &argv, NULL);
 
+    if (have_qemu_img()) {
+        imgfmt = "qcow2";
+    }
+
     for (i = 0; i < backend_last; i++) {
         if (img_secs[i] >= 0) {
             img_file_name[i] = create_test_img(img_secs[i]);
-- 
2.53.0

Re: [PATCH 0/7] tests/qtest: fix the disk tests on a host with a small /tmp

[Thomas Huth]

  Hi Denis!

Please make sure to CC: the qtest maintainer (Fabiano Rosas) on series like 
this.

On 24/08/2026 22.06, Denis V. Lunev wrote:
> John reported ide-test dying at startup in the NetBSD VM:
> 
>    ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion failed:
>    (ret == 0)
> 
> The hypothesys is that the ftruncate() of a 64 MiB scratch image. NetBSD
> mounts /tmp as a tmpfs sized at 25% of RAM and charges a file its full
> length themoment it is extended, so a sparse image is not free there and
> the call returns ENOSPC. The assert dates to 2013, nothing regressed.

Does NetBSD have another file like /var/tmp that might be friendlier to 
sparse files? If so, maybe that should be used instead?

OTOH, our NetBSD VM in tests/vm/ uses 4G of RAM, so a temporary file with 
just 64 MiB should really not be a problem...?

...> Hope this should help. John, can you give a try to this in the test?
> I do not have NetBSD handy and will start with this tomorrow if you
> will share guest config.
Simply type "make vm-build-netbsd" from your build folder, this should take 
care of everything automatically. No need to install NetBSD on your own.

  Thomas

Re: [PATCH 0/7] tests/qtest: fix the disk tests on a host with a small /tmp

[Daniel P. Berrangé]

On Thu, Aug 27, 2026 at 12:16:56PM +0200, Thomas Huth wrote:
> 
>  Hi Denis!
> 
> Please make sure to CC: the qtest maintainer (Fabiano Rosas) on series like
> this.
> 
> On 24/08/2026 22.06, Denis V. Lunev wrote:
> > John reported ide-test dying at startup in the NetBSD VM:
> > 
> >    ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion failed:
> >    (ret == 0)
> > 
> > The hypothesys is that the ftruncate() of a 64 MiB scratch image. NetBSD
> > mounts /tmp as a tmpfs sized at 25% of RAM and charges a file its full
> > length themoment it is extended, so a sparse image is not free there and
> > the call returns ENOSPC. The assert dates to 2013, nothing regressed.
> 
> Does NetBSD have another file like /var/tmp that might be friendlier to
> sparse files? If so, maybe that should be used instead?
> 
> OTOH, our NetBSD VM in tests/vm/ uses 4G of RAM, so a temporary file with
> just 64 MiB should really not be a problem...?

Or this is a concurrency scaling problem, or racing with another test
that also uses stuff ?

We hard code memory to 4 GB, but -smp we scale to "$NUM-CPUs / 2"
for the "make vm-build-DIST" commands.

IOW, regardless of whether QEMU is launched with -smp 1 or -smp 20,
we only give it 4 GB to play with. We could be using a lot of RAM
for concurrent build jobs leaving almost nothing for the tmpfs for
the test.

If we were that close to exhaustion I'd expected to see out of
memory errors, but I'm unclear what NetBSD's behaviour is in
this respect ?  Maybe normal RAM usage can be pushed to swap
(of which I see another 4 GB) while tmpfs can't be pushed
to swap ?




With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: [PATCH 0/7] tests/qtest: fix the disk tests on a host with a small /tmp

[Denis V. Lunev]

On 8/27/26 13:03, Daniel P. Berrangé wrote:
> On Thu, Aug 27, 2026 at 12:16:56PM +0200, Thomas Huth wrote:
>>  Hi Denis!
>>
>> Please make sure to CC: the qtest maintainer (Fabiano Rosas) on series like
>> this.
>>
>> On 24/08/2026 22.06, Denis V. Lunev wrote:
>>> John reported ide-test dying at startup in the NetBSD VM:
>>>
>>>    ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion failed:
>>>    (ret == 0)
>>>
>>> The hypothesys is that the ftruncate() of a 64 MiB scratch image. NetBSD
>>> mounts /tmp as a tmpfs sized at 25% of RAM and charges a file its full
>>> length themoment it is extended, so a sparse image is not free there and
>>> the call returns ENOSPC. The assert dates to 2013, nothing regressed.
>> Does NetBSD have another file like /var/tmp that might be friendlier to
>> sparse files? If so, maybe that should be used instead?
>>
>> OTOH, our NetBSD VM in tests/vm/ uses 4G of RAM, so a temporary file with
>> just 64 MiB should really not be a problem...?
> Or this is a concurrency scaling problem, or racing with another test
> that also uses stuff ?
>
> We hard code memory to 4 GB, but -smp we scale to "$NUM-CPUs / 2"
> for the "make vm-build-DIST" commands.
>
> IOW, regardless of whether QEMU is launched with -smp 1 or -smp 20,
> we only give it 4 GB to play with. We could be using a lot of RAM
> for concurrent build jobs leaving almost nothing for the tmpfs for
> the test.
>
> If we were that close to exhaustion I'd expected to see out of
> memory errors, but I'm unclear what NetBSD's behaviour is in
> this respect ?  Maybe normal RAM usage can be pushed to swap
> (of which I see another 4 GB) while tmpfs can't be pushed
> to swap ?
>
>
>
>
> With regards,
> Daniel
Guys,

will somebody take a look into what was done?
This makes sense anyway - unified same class error tracking
plus requirements reduction.

Or if nobody care I could just push along with another
test fix?

Thank you in advance,
    Den

Re: [PATCH 0/7] tests/qtest: fix the disk tests on a host with a small /tmp

[Fabiano Rosas]

"Denis V. Lunev" <den@virtuozzo.com> writes:

> On 8/27/26 13:03, Daniel P. Berrangé wrote:
>> On Thu, Aug 27, 2026 at 12:16:56PM +0200, Thomas Huth wrote:
>>>  Hi Denis!
>>>
>>> Please make sure to CC: the qtest maintainer (Fabiano Rosas) on series like
>>> this.
>>>
>>> On 24/08/2026 22.06, Denis V. Lunev wrote:
>>>> John reported ide-test dying at startup in the NetBSD VM:
>>>>
>>>>    ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion failed:
>>>>    (ret == 0)
>>>>
>>>> The hypothesys is that the ftruncate() of a 64 MiB scratch image. NetBSD
>>>> mounts /tmp as a tmpfs sized at 25% of RAM and charges a file its full
>>>> length themoment it is extended, so a sparse image is not free there and
>>>> the call returns ENOSPC. The assert dates to 2013, nothing regressed.
>>> Does NetBSD have another file like /var/tmp that might be friendlier to
>>> sparse files? If so, maybe that should be used instead?
>>>
>>> OTOH, our NetBSD VM in tests/vm/ uses 4G of RAM, so a temporary file with
>>> just 64 MiB should really not be a problem...?
>> Or this is a concurrency scaling problem, or racing with another test
>> that also uses stuff ?
>>
>> We hard code memory to 4 GB, but -smp we scale to "$NUM-CPUs / 2"
>> for the "make vm-build-DIST" commands.
>>
>> IOW, regardless of whether QEMU is launched with -smp 1 or -smp 20,
>> we only give it 4 GB to play with. We could be using a lot of RAM
>> for concurrent build jobs leaving almost nothing for the tmpfs for
>> the test.
>>
>> If we were that close to exhaustion I'd expected to see out of
>> memory errors, but I'm unclear what NetBSD's behaviour is in
>> this respect ?  Maybe normal RAM usage can be pushed to swap
>> (of which I see another 4 GB) while tmpfs can't be pushed
>> to swap ?
>>
>>
>>
>>
>> With regards,
>> Daniel
> Guys,
>
> will somebody take a look into what was done?
> This makes sense anyway - unified same class error tracking
> plus requirements reduction.
>

I cannot reproduce this. I also don't think ENOSPC on 64MB is something
worth the churn at all.

> Or if nobody care I could just push along with another
> test fix?
>
> Thank you in advance,
>     Den

Re: [PATCH 0/7] tests/qtest: fix the disk tests on a host with a small /tmp

[Denis V. Lunev]

On 9/3/26 18:57, Fabiano Rosas wrote:
> "Denis V. Lunev" <den@virtuozzo.com> writes:
>
>> On 8/27/26 13:03, Daniel P. Berrangé wrote:
>>> On Thu, Aug 27, 2026 at 12:16:56PM +0200, Thomas Huth wrote:
>>>>  Hi Denis!
>>>>
>>>> Please make sure to CC: the qtest maintainer (Fabiano Rosas) on series like
>>>> this.
>>>>
>>>> On 24/08/2026 22.06, Denis V. Lunev wrote:
>>>>> John reported ide-test dying at startup in the NetBSD VM:
>>>>>
>>>>>    ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion failed:
>>>>>    (ret == 0)
>>>>>
>>>>> The hypothesys is that the ftruncate() of a 64 MiB scratch image. NetBSD
>>>>> mounts /tmp as a tmpfs sized at 25% of RAM and charges a file its full
>>>>> length themoment it is extended, so a sparse image is not free there and
>>>>> the call returns ENOSPC. The assert dates to 2013, nothing regressed.
>>>> Does NetBSD have another file like /var/tmp that might be friendlier to
>>>> sparse files? If so, maybe that should be used instead?
>>>>
>>>> OTOH, our NetBSD VM in tests/vm/ uses 4G of RAM, so a temporary file with
>>>> just 64 MiB should really not be a problem...?
>>> Or this is a concurrency scaling problem, or racing with another test
>>> that also uses stuff ?
>>>
>>> We hard code memory to 4 GB, but -smp we scale to "$NUM-CPUs / 2"
>>> for the "make vm-build-DIST" commands.
>>>
>>> IOW, regardless of whether QEMU is launched with -smp 1 or -smp 20,
>>> we only give it 4 GB to play with. We could be using a lot of RAM
>>> for concurrent build jobs leaving almost nothing for the tmpfs for
>>> the test.
>>>
>>> If we were that close to exhaustion I'd expected to see out of
>>> memory errors, but I'm unclear what NetBSD's behaviour is in
>>> this respect ?  Maybe normal RAM usage can be pushed to swap
>>> (of which I see another 4 GB) while tmpfs can't be pushed
>>> to swap ?
>>>
>>>
>>>
>>>
>>> With regards,
>>> Daniel
>> Guys,
>>
>> will somebody take a look into what was done?
>> This makes sense anyway - unified same class error tracking
>> plus requirements reduction.
>>
> I cannot reproduce this. I also don't think ENOSPC on 64MB is something
> worth the churn at all.
Great.

Clearly postponed, dropping this from watch list.

Thanks a lot for your time.

Den