From: Thomas Huth <thuth@redhat.com>
To: qemu-devel@nongnu.org
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Subject: [PULL 21/38] tests/functional: add a generalized archive_extract
Date: Wed, 18 Dec 2024 12:09:41 +0100 [thread overview]
Message-ID: <20241218110958.226932-22-thuth@redhat.com> (raw)
In-Reply-To: <20241218110958.226932-1-thuth@redhat.com>
From: Daniel P. Berrangé <berrange@redhat.com>
There are many types of archives that the tests deal with. Provide
a generalized 'archive_extract' that can detect the format and
delegate to the appropriate helper for extraction. This ensures
that all archive extraction code follows the same design pattern.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-22-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/functional/qemu_test/__init__.py | 1 +
tests/functional/qemu_test/archive.py | 58 ++++++++++++++++++++++++++
2 files changed, 59 insertions(+)
diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
index fe6cbe3a8a..665c482d13 100644
--- a/tests/functional/qemu_test/__init__.py
+++ b/tests/functional/qemu_test/__init__.py
@@ -16,3 +16,4 @@
from .decorators import skipIfMissingCommands, skipIfNotMachine, \
skipFlakyTest, skipUntrustedTest, skipBigDataTest, \
skipIfMissingImports
+from .archive import archive_extract
diff --git a/tests/functional/qemu_test/archive.py b/tests/functional/qemu_test/archive.py
index bc448dee4a..c439d9413a 100644
--- a/tests/functional/qemu_test/archive.py
+++ b/tests/functional/qemu_test/archive.py
@@ -10,8 +10,10 @@
import os
from subprocess import check_call, run, DEVNULL
import tarfile
+from urllib.parse import urlparse
import zipfile
+from .asset import Asset
from .cmd import run_cmd
@@ -56,3 +58,59 @@ def deb_extract(archive, dest_dir, member=None):
tar_extract(file_path, dest_dir, member)
finally:
os.chdir(cwd)
+
+'''
+@params archive: filename, Asset, or file-like object to extract
+@params dest_dir: target directory to extract into
+@params member: optional member file to limit extraction to
+
+Extracts @archive into @dest_dir. All files are extracted
+unless @member specifies a limit.
+
+If @format is None, heuristics will be applied to guess the format
+from the filename or Asset URL. @format must be non-None if @archive
+is a file-like object.
+'''
+def archive_extract(archive, dest_dir, format=None, member=None):
+ if format is None:
+ format = guess_archive_format(archive)
+ if type(archive) == Asset:
+ archive = str(archive)
+
+ if format == "tar":
+ tar_extract(archive, dest_dir, member)
+ elif format == "zip":
+ zip_extract(archive, dest_dir, member)
+ elif format == "cpio":
+ if member is not None:
+ raise Exception("Unable to filter cpio extraction")
+ cpio_extract(archive, dest_dir)
+ elif format == "deb":
+ if type(archive) != str:
+ raise Exception("Unable to use file-like object with deb archives")
+ deb_extract(archive, dest_dir, "./" + member)
+ else:
+ raise Exception(f"Unknown archive format {format}")
+
+'''
+@params archive: filename, or Asset to guess
+
+Guess the format of @compressed, raising an exception if
+no format can be determined
+'''
+def guess_archive_format(archive):
+ if type(archive) == Asset:
+ archive = urlparse(archive.url).path
+ elif type(archive) != str:
+ raise Exception(f"Unable to guess archive format for {archive}")
+
+ if ".tar." in archive or archive.endswith("tgz"):
+ return "tar"
+ elif archive.endswith(".zip"):
+ return "zip"
+ elif archive.endswith(".cpio"):
+ return "cpio"
+ elif archive.endswith(".deb") or archive.endswith(".udeb"):
+ return "deb"
+ else:
+ raise Exception(f"Unknown archive format for {archive}")
--
2.47.1
next prev parent reply other threads:[~2024-12-18 11:13 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-12-18 11:09 [PULL 00/38] Functional test improvements, and disallow libnfs v6 Thomas Huth
2024-12-18 11:09 ` [PULL 01/38] tests/functional: add execute permission to aspeed tests Thomas Huth
2024-12-18 11:09 ` [PULL 02/38] tests/functional: remove many unused imports Thomas Huth
2024-12-18 11:09 ` [PULL 03/38] tests/functional: resolve str(Asset) to cache file path Thomas Huth
2024-12-18 11:09 ` [PULL 04/38] tests/functional: remove duplicated 'which' function impl Thomas Huth
2024-12-18 11:09 ` [PULL 05/38] tests/functional: simplify 'which' implementation Thomas Huth
2024-12-18 11:09 ` [PULL 06/38] tests/functional: drop 'tesseract_available' helper Thomas Huth
2024-12-18 11:09 ` [PULL 07/38] tests/functional: introduce some helpful decorators Thomas Huth
2024-12-18 11:09 ` [PULL 08/38] tests/functional: switch to new test skip decorators Thomas Huth
2024-12-18 11:09 ` [PULL 09/38] tests/functional: drop 'has_cmd' and 'has_cmds' helpers Thomas Huth
2024-12-18 11:09 ` [PULL 10/38] tests/functional: add helpers for building file paths Thomas Huth
2024-12-18 11:09 ` [PULL 11/38] tests/functional: switch over to using self.log_file(...) Thomas Huth
2024-12-18 11:09 ` [PULL 12/38] tests/functional: switch over to using self.build_file(...) Thomas Huth
2024-12-18 11:09 ` [PULL 13/38] tests/functional: switch over to using self.data_file(...) Thomas Huth
2024-12-18 11:09 ` [PULL 14/38] tests/functional: switch over to using self.scratch_file() Thomas Huth
2024-12-18 11:09 ` [PULL 15/38] tests/functional: remove redundant 'rmtree' call Thomas Huth
2024-12-18 11:09 ` [PULL 16/38] tests/functional: move archive handling into new archive.py file Thomas Huth
2024-12-18 11:09 ` [PULL 17/38] tests/functional: move uncompress handling into new uncompress.py file Thomas Huth
2024-12-18 11:09 ` [PULL 18/38] tests/functional: add common zip_extract helper Thomas Huth
2024-12-18 11:09 ` [PULL 19/38] tests/functional: add common deb_extract helper Thomas Huth
2024-12-18 11:09 ` [PULL 20/38] tests/functional: let cpio_extract accept filenames Thomas Huth
2024-12-18 11:09 ` Thomas Huth [this message]
2024-12-18 11:09 ` [PULL 22/38] tests/functional: add 'archive_extract' to QemuBaseTest Thomas Huth
2024-12-18 11:09 ` [PULL 23/38] tests/functional: convert tests to new archive_extract helper Thomas Huth
2024-12-18 11:09 ` [PULL 24/38] tests/functional: add a generalized uncompress helper Thomas Huth
2024-12-18 11:09 ` [PULL 25/38] tests/functional: add 'uncompress' to QemuBaseTest Thomas Huth
2024-12-18 11:09 ` [PULL 26/38] tests/functional: convert tests to new uncompress helper Thomas Huth
2024-12-18 11:09 ` [PULL 27/38] tests/functional: drop back compat imports from utils.py Thomas Huth
2024-12-18 11:09 ` [PULL 28/38] tests/functional: replace 'run_cmd' with subprocess helpers Thomas Huth
2024-12-18 11:09 ` [PULL 29/38] tests/functional: remove now unused 'run_cmd' helper Thomas Huth
2024-12-18 11:09 ` [PULL 30/38] tests/functional: skip tests if assets are not available Thomas Huth
2024-12-18 11:09 ` [PULL 31/38] tests/functional: ignore errors when caching assets, except for 404 Thomas Huth
2024-12-18 11:09 ` [PULL 32/38] MAINTAINERS: add myself as reviewer for functional test suite Thomas Huth
2024-12-18 11:09 ` [PULL 33/38] tests/functional: Convert the quanta-gsj avocado test Thomas Huth
2024-12-18 11:09 ` [PULL 34/38] tests/functional: Convert the arm virt " Thomas Huth
2024-12-18 11:09 ` [PULL 35/38] tests/functional: Add a helper function for retrieving the hostfwd port Thomas Huth
2024-12-18 11:09 ` [PULL 36/38] tests/functional: Convert the intel_iommu avocado test Thomas Huth
2024-12-18 11:09 ` [PULL 37/38] tests/functional: Convert the hotplug_cpu " Thomas Huth
2024-12-18 11:09 ` [PULL 38/38] meson.build: Disallow libnfs v6 to fix the broken macOS build Thomas Huth
2024-12-19 12:29 ` [PULL 00/38] Functional test improvements, and disallow libnfs v6 Stefan Hajnoczi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20241218110958.226932-22-thuth@redhat.com \
--to=thuth@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.