qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v4 00/10] Introduce new acpi/smbios avocado tests using biosbits
@ 2022-10-14 17:34 Ani Sinha
  2022-10-14 17:34 ` [PATCH v4 01/10] acpi/tests/avocado/bits: initial commit of test scripts that are run by biosbits Ani Sinha
                   ` (10 more replies)
  0 siblings, 11 replies; 22+ messages in thread
From: Ani Sinha @ 2022-10-14 17:34 UTC (permalink / raw)
  To: Cleber Rosa, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Beraldo Leal
  Cc: Ani Sinha, Qemu Devel, Daniel P . Berrangé, Paolo Bonzini,
	Maydell Peter, John Snow, Thomas Huth, Alex Bennée,
	Igor Mammedov, Michael Tsirkin

Please see the README file in patch 9 for more details.
Sample runs are as follows:

Passed tests:

$ ./tests/venv/bin/avocado run -t acpi tests/avocado
Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
JOB ID     : e95c7413e996bfb59389839e5ca5105464ef098f
JOB LOG    : /home/anisinha/avocado/job-results/job-2022-10-14T19.15-e95c741/job.log
 (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS (33.08 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 39.18 s


$ ./tests/venv/bin/avocado run ./tests/avocado/acpi-bits.py 
Fetching asset from ./tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
JOB ID     : c6e588594bceb2c5fdbdf45095559c58e4030f9c
JOB LOG    : /home/anisinha/avocado/job-results/job-2022-10-14T19.13-c6e5885/job.log
 (1/1) ./tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS (32.29 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 39.27 s

When PSS tests are enabled:

$ git diff
diff --git a/tests/avocado/acpi-bits/bits-tests/testacpi.py2 b/tests/avocado/acpi-bits/bits-tests/testacpi.py2
index f818a9cce6..18dc818d62 100644
--- a/tests/avocado/acpi-bits/bits-tests/testacpi.py2
+++ b/tests/avocado/acpi-bits/bits-tests/testacpi.py2
@@ -40,8 +40,8 @@ import time
 
 def register_tests():
     testsuite.add_test("ACPI _MAT (Multiple APIC Table Entry) under Processor objects", test_mat, submenu="ACPI Tests")
-#    testsuite.add_test("ACPI _PSS (Pstate) table conformance tests", test_pss, submenu="ACPI Tests")
-#    testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates, submenu="ACPI Tests")
+    testsuite.add_test("ACPI _PSS (Pstate) table conformance tests", test_pss, submenu="ACPI Tests")
+    testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates, submenu="ACPI Tests")
     testsuite.add_test("ACPI DSDT (Differentiated System Description Table)", test_dsdt, submenu="ACPI Tests")
     testsuite.add_test("ACPI FACP (Fixed ACPI Description Table)", test_facp, submenu="ACPI Tests")
     testsuite.add_test("ACPI HPET (High Precision Event Timer Table)", test_hpet, submenu="ACPI Tests")

$ ./tests/venv/bin/avocado run -t acpi tests/avocado
Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
JOB ID     : 16a8b8c7f8dc87c4b84f8a2f95135bb63464a71e
JOB LOG    : /home/anisinha/avocado/job-results/job-2022-10-14T22.43-16a8b8c/job.log
 (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: FAIL: '1' != '0'\n- 1\n+ 0\n : Some bits tests seems to have failed. Please check the test logs for more info. (33.01 s)
RESULTS    : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 39.17 s


Changelog:
v4:
  - renamed the bits test files as .py2.
  - fixed a bug with missing log file parsing. dump bits logs automatically
    for test failures (no need to re-run with V=1).
  - fixed issues with README file and enhanced it.
  - cosmetic comment updates.
v3:
  - converted the existing test to avocado tests as per the popular
    recommendation. Added appropriate tags.
  - artifact download URL modified based on gitlab-CI URL.

  For biosbits repo:
  - moved to a gitlab repo with me being the maintainer.
  - added .gitlab-ci.yml file to generate the artifacts.
v2:
 - a new class of python based tests introduced that is separate from avocado
   tests or qtests. Can be run by using "make check-pytest".
 - acpi biosbits tests are the first tests to use pytest environment.
 - bios bits tests now download the bits binary archives from a remote
   repository if they are not found locally. The test skips if download
   fails.
 - A new environment variable is introduced that can be passed by the tester
   to specify the location of the bits archives locally. test skips if the
   bits binaries are not found in that location.
 - if pip install of python module fails for whatever reaoson, the test skips.
 - misc code fixes including spell check of the README doc. README has been
   updated as well.
 - addition of SPDX license headers to bits test files.
 - update MAINTAINERS to reflect the new pytest test class.

For biosbits repo:
 - added Dockerfile and build script. Made bios bits build on gcc 11.
   https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile
   https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh
   The build script generates the zip archive and tarball used by the test.

v1: initial patchset. uses qtest to implement the bios bits tests.

Cc: Qemu Devel <qemu-devel@nongnu.org>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Maydell Peter <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Michael Tsirkin <mst@redhat.com>


Ani Sinha (10):
  acpi/tests/avocado/bits: initial commit of test scripts that are run
    by biosbits
  acpi/tests/avocado/bits: add SPDX license identifiers for bios bits
    tests
  acpi/tests/avocado/bits: disable acpi PSS tests that are failing in
    biosbits
  acpi/tests/avocado/bits: add smilatency test suite from bits in order
    to disable it
  acpi/tests/avocado/bits: add SPDX license identifiers for bios bits
    smilatency tests
  acpi/tests/avocado/bits: disable smilatency test since it does not
    pass everytime
  acpi/tests/avocado/bits: add biosbits config file for running bios
    tests
  acpi/tests/avocado/bits: add acpi and smbios avocado tests that uses
    biosbits
  acpi/tests/avocado/bits: add a README file to describe the test
  MAINTAINERS: add myself as the maintainer for acpi biosbits avocado
    tests

 MAINTAINERS                                   |    6 +
 tests/avocado/acpi-bits.py                    |  353 +++
 tests/avocado/acpi-bits/README                |  128 +
 .../acpi-bits/bits-config/bits-cfg.txt        |   18 +
 tests/avocado/acpi-bits/bits-tests/smbios.py2 | 2434 +++++++++++++++++
 .../acpi-bits/bits-tests/smilatency.py2       |  107 +
 .../avocado/acpi-bits/bits-tests/testacpi.py2 |  287 ++
 .../acpi-bits/bits-tests/testcpuid.py2        |   87 +
 8 files changed, 3420 insertions(+)
 create mode 100644 tests/avocado/acpi-bits.py
 create mode 100644 tests/avocado/acpi-bits/README
 create mode 100644 tests/avocado/acpi-bits/bits-config/bits-cfg.txt
 create mode 100644 tests/avocado/acpi-bits/bits-tests/smbios.py2
 create mode 100644 tests/avocado/acpi-bits/bits-tests/smilatency.py2
 create mode 100644 tests/avocado/acpi-bits/bits-tests/testacpi.py2
 create mode 100644 tests/avocado/acpi-bits/bits-tests/testcpuid.py2

-- 
2.34.1



^ permalink raw reply related	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2022-10-19  6:36 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-10-14 17:34 [PATCH v4 00/10] Introduce new acpi/smbios avocado tests using biosbits Ani Sinha
2022-10-14 17:34 ` [PATCH v4 01/10] acpi/tests/avocado/bits: initial commit of test scripts that are run by biosbits Ani Sinha
2022-10-17 21:13   ` Philippe Mathieu-Daudé
2022-10-17 22:06     ` Michael S. Tsirkin
2022-10-18  3:18     ` Ani Sinha
2022-10-14 17:35 ` [PATCH v4 02/10] acpi/tests/avocado/bits: add SPDX license identifiers for bios bits tests Ani Sinha
2022-10-14 17:35 ` [PATCH v4 03/10] acpi/tests/avocado/bits: disable acpi PSS tests that are failing in biosbits Ani Sinha
2022-10-14 17:35 ` [PATCH v4 04/10] acpi/tests/avocado/bits: add smilatency test suite from bits in order to disable it Ani Sinha
2022-10-14 17:35 ` [PATCH v4 05/10] acpi/tests/avocado/bits: add SPDX license identifiers for bios bits smilatency tests Ani Sinha
2022-10-14 17:35 ` [PATCH v4 06/10] acpi/tests/avocado/bits: disable smilatency test since it does not pass everytime Ani Sinha
2022-10-14 17:35 ` [PATCH v4 07/10] acpi/tests/avocado/bits: add biosbits config file for running bios tests Ani Sinha
2022-10-14 17:35 ` [PATCH v4 08/10] acpi/tests/avocado/bits: add acpi and smbios avocado tests that uses biosbits Ani Sinha
2022-10-18 15:37   ` Alex Bennée
2022-10-18 15:43     ` Ani Sinha
2022-10-14 17:35 ` [PATCH v4 09/10] acpi/tests/avocado/bits: add a README file to describe the test Ani Sinha
2022-10-17  9:39   ` Ani Sinha
2022-10-14 17:35 ` [PATCH v4 10/10] MAINTAINERS: add myself as the maintainer for acpi biosbits avocado tests Ani Sinha
2022-10-18 15:36   ` Alex Bennée
2022-10-18 15:30 ` [PATCH v4 00/10] Introduce new acpi/smbios avocado tests using biosbits Alex Bennée
2022-10-18 15:34   ` Ani Sinha
2022-10-19  6:22     ` Thomas Huth
2022-10-19  6:26       ` Ani Sinha

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).