* [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits
@ 2022-07-10 17:00 Ani Sinha
2022-07-10 17:00 ` [PATCH v2 01/11] acpi/tests/bits: initial commit of test scripts that are run by biosbits Ani Sinha
` (11 more replies)
0 siblings, 12 replies; 90+ messages in thread
From: Ani Sinha @ 2022-07-10 17:00 UTC (permalink / raw)
To: qemu-devel
Cc: thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo, mst,
Ani Sinha
Changelog:
v2:
- a new class of python based tests introduced that is separate from avocado
tests or qtests. Can be run by using "make check-pytest".
- acpi biosbits tests are the first tests to use pytest environment.
- bios bits tests now download the bits binary archives from a remote
repository if they are not found locally. The test skips if download
fails.
- A new environment variable is introduced that can be passed by the tester
to specify the location of the bits archives locally. test skips if the
bits binaries are not found in that location.
- if pip install of python module fails for whatever reaoson, the test skips.
- misc code fixes including spell check of the README doc. README has been
updated as well.
- addition of SPDX license headers to bits test files.
- update MAINTAINERS to reflect the new pytest test class.
For biosbits repo:
- added Dockerfile and build script. Made bios bits build on gcc 11.
https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile
https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh
The build script generates the zip archive and tarball used by the test.
v1: initial patchset. uses qtest to implement the bios bits tests.
Ani Sinha (11):
acpi/tests/bits: initial commit of test scripts that are run by
biosbits
acpi/tests/bits: add SPDX license identifiers for bios bits tests
acpi/tests/bits: disable acpi PSS tests that are failing in biosbits
acpi/tests/bits: add smilatency test suite from bits in order to
disable it
acpi/tests/bits: add SPDX license identifiers for bios bits smilatency
tests
acpi/tests/bits: disable smilatency test since it does not pass
everytime
acpi/tests/bits: add python test that exercizes QEMU bios tables using
biosbits
acpi/tests/bits: add biosbits config file for running bios tests
acpi/tests/bits: add a README file describing bits pytests
pytest: add pytest to the meson build system
MAINTAINERS: add myself as the maintainer for acpi biosbits pytests
MAINTAINERS | 5 +
tests/Makefile.include | 4 +-
tests/meson.build | 1 +
tests/pytest/acpi-bits/README | 129 +
tests/pytest/acpi-bits/acpi-bits-test-venv.sh | 59 +
tests/pytest/acpi-bits/acpi-bits-test.py | 382 +++
.../pytest/acpi-bits/bits-config/bits-cfg.txt | 18 +
.../pytest/acpi-bits/bits-config/meson.build | 11 +
tests/pytest/acpi-bits/bits-tests/meson.build | 11 +
tests/pytest/acpi-bits/bits-tests/smbios.py | 2432 +++++++++++++++++
.../pytest/acpi-bits/bits-tests/smilatency.py | 105 +
tests/pytest/acpi-bits/bits-tests/testacpi.py | 285 ++
.../pytest/acpi-bits/bits-tests/testcpuid.py | 85 +
tests/pytest/acpi-bits/meson.build | 33 +
tests/pytest/acpi-bits/requirements.txt | 1 +
tests/pytest/meson.build | 49 +
16 files changed, 3609 insertions(+), 1 deletion(-)
create mode 100644 tests/pytest/acpi-bits/README
create mode 100644 tests/pytest/acpi-bits/acpi-bits-test-venv.sh
create mode 100644 tests/pytest/acpi-bits/acpi-bits-test.py
create mode 100644 tests/pytest/acpi-bits/bits-config/bits-cfg.txt
create mode 100644 tests/pytest/acpi-bits/bits-config/meson.build
create mode 100644 tests/pytest/acpi-bits/bits-tests/meson.build
create mode 100644 tests/pytest/acpi-bits/bits-tests/smbios.py
create mode 100644 tests/pytest/acpi-bits/bits-tests/smilatency.py
create mode 100644 tests/pytest/acpi-bits/bits-tests/testacpi.py
create mode 100644 tests/pytest/acpi-bits/bits-tests/testcpuid.py
create mode 100644 tests/pytest/acpi-bits/meson.build
create mode 100644 tests/pytest/acpi-bits/requirements.txt
create mode 100644 tests/pytest/meson.build
--
2.25.1
^ permalink raw reply [flat|nested] 90+ messages in thread* [PATCH v2 01/11] acpi/tests/bits: initial commit of test scripts that are run by biosbits 2022-07-10 17:00 [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Ani Sinha @ 2022-07-10 17:00 ` Ani Sinha 2022-07-11 20:38 ` John Snow 2022-07-10 17:00 ` [PATCH v2 02/11] acpi/tests/bits: add SPDX license identifiers for bios bits tests Ani Sinha ` (10 subsequent siblings) 11 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-07-10 17:00 UTC (permalink / raw) To: qemu-devel Cc: thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo, mst, Ani Sinha This is initial commit of cpuid, acpi and smbios python test scripts for biosbits to execute. No change has been made to them from the original code written by the biosbits author Josh Triplett. They are required to be installed into the bits iso file and then run from within the virtual machine booted off with biosbits iso. The original location of these tests are here: https://github.com/biosbits/bits/blob/master/python/testacpi.py https://github.com/biosbits/bits/blob/master/python/smbios.py https://github.com/biosbits/bits/blob/master/python/testcpuid.py Signed-off-by: Ani Sinha <ani@anisinha.ca> --- tests/pytest/acpi-bits/bits-tests/meson.build | 11 + tests/pytest/acpi-bits/bits-tests/smbios.py | 2430 +++++++++++++++++ tests/pytest/acpi-bits/bits-tests/testacpi.py | 283 ++ .../pytest/acpi-bits/bits-tests/testcpuid.py | 83 + 4 files changed, 2807 insertions(+) create mode 100644 tests/pytest/acpi-bits/bits-tests/meson.build create mode 100644 tests/pytest/acpi-bits/bits-tests/smbios.py create mode 100644 tests/pytest/acpi-bits/bits-tests/testacpi.py create mode 100644 tests/pytest/acpi-bits/bits-tests/testcpuid.py diff --git a/tests/pytest/acpi-bits/bits-tests/meson.build b/tests/pytest/acpi-bits/bits-tests/meson.build new file mode 100644 index 0000000000..3056731a53 --- /dev/null +++ b/tests/pytest/acpi-bits/bits-tests/meson.build @@ -0,0 +1,11 @@ +test_files = ['smbios.py', 'testacpi.py', 'testcpuid.py'] + +copytestfiles = custom_target('copy test files', + input : test_files, + output : test_files, + command : ['cp', '@INPUT@', '@OUTDIR@'], + install : true, + install_dir : 'bits-tests', + build_by_default : true) + +other_deps += copytestfiles diff --git a/tests/pytest/acpi-bits/bits-tests/smbios.py b/tests/pytest/acpi-bits/bits-tests/smbios.py new file mode 100644 index 0000000000..9667d0542c --- /dev/null +++ b/tests/pytest/acpi-bits/bits-tests/smbios.py @@ -0,0 +1,2430 @@ +# Copyright (c) 2015, Intel Corporation +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# * Redistributions of source code must retain the above copyright notice, +# this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# * Neither the name of Intel Corporation nor the names of its contributors +# may be used to endorse or promote products derived from this software +# without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""SMBIOS/DMI module.""" + +import bits +import bitfields +import ctypes +import redirect +import struct +import uuid +import unpack +import ttypager +import sys + +class SMBIOS(unpack.Struct): + def __new__(cls): + if sys.platform == "BITS-EFI": + import efi + sm_ptr = efi.system_table.ConfigurationTableDict.get(efi.SMBIOS_TABLE_GUID) + else: + address = 0xF0000 + mem = bits.memory(0xF0000, 0x10000) + for offset in range(0, len(mem), 16): + signature = (ctypes.c_char * 4).from_address(address + offset).value + if signature == "_SM_": + entry_point_length = ctypes.c_ubyte.from_address(address + offset + 5).value + csum = sum(map(ord, mem[offset:offset + entry_point_length])) & 0xff + if csum == 0: + sm_ptr = address + offset + break + else: + return None + + if not sm_ptr: + return None + + sm = super(SMBIOS, cls).__new__(cls) + sm._header_memory = bits.memory(sm_ptr, 0x1f) + return sm + + def __init__(self): + super(SMBIOS, self).__init__() + u = unpack.Unpackable(self._header_memory) + self.add_field('header', Header(u)) + self._structure_memory = bits.memory(self.header.structure_table_address, self.header.structure_table_length) + u = unpack.Unpackable(self._structure_memory) + self.add_field('structures', unpack.unpack_all(u, _smbios_structures, self), unpack.format_each("\n\n{!r}")) + + def structure_type(self, num): + '''Dumps structure of given Type if present''' + try: + types_present = [self.structures[x].smbios_structure_type for x in range(len(self.structures))] + matrix = dict() + for index in range(len(types_present)): + if types_present.count(types_present[index]) == 1: + matrix[types_present[index]] = self.structures[index] + else: # if multiple structures of the same type, return a list of structures for the type number + if matrix.has_key(types_present[index]): + matrix[types_present[index]].append(self.structures[index]) + else: + matrix[types_present[index]] = [self.structures[index]] + return matrix[num] + except: + print "Failure: Type {} - not found".format(num) + +class Header(unpack.Struct): + def __new__(cls, u): + return super(Header, cls).__new__(cls) + + def __init__(self, u): + super(Header, self).__init__() + self.raw_data = u.unpack_rest() + u = unpack.Unpackable(self.raw_data) + self.add_field('anchor_string', u.unpack_one("4s")) + self.add_field('checksum', u.unpack_one("B")) + self.add_field('length', u.unpack_one("B")) + self.add_field('major_version', u.unpack_one("B")) + self.add_field('minor_version', u.unpack_one("B")) + self.add_field('max_structure_size', u.unpack_one("<H")) + self.add_field('entry_point_revision', u.unpack_one("B")) + self.add_field('formatted_area', u.unpack_one("5s")) + self.add_field('intermediate_anchor_string', u.unpack_one("5s")) + self.add_field('intermediate_checksum', u.unpack_one("B")) + self.add_field('structure_table_length', u.unpack_one("<H")) + self.add_field('structure_table_address', u.unpack_one("<I")) + self.add_field('number_structures', u.unpack_one("<H")) + self.add_field('bcd_revision', u.unpack_one("B")) + if not u.at_end(): + self.add_field('data', u.unpack_rest()) + +class SmbiosBaseStructure(unpack.Struct): + def __new__(cls, u, sm): + t = u.unpack_peek_one("B") + if cls.smbios_structure_type is not None and t != cls.smbios_structure_type: + return None + return super(SmbiosBaseStructure, cls).__new__(cls) + + def __init__(self, u, sm): + super(SmbiosBaseStructure, self).__init__() + self.start_offset = u.offset + length = u.unpack_peek_one("<xB") + self.raw_data = u.unpack_raw(length) + self.u = unpack.Unpackable(self.raw_data) + + self.strings_offset = u.offset + def unpack_string(): + return "".join(iter(lambda: u.unpack_one("c"), "\x00")) + strings = list(iter(unpack_string, "")) + if not strings: + u.skip(1) + + self.strings_length = u.offset - self.strings_offset + self.raw_strings = str(bits.memory(sm.header.structure_table_address + self.strings_offset, self.strings_length)) + + if len(strings): + self.strings = strings + + self.add_field('type', self.u.unpack_one("B")) + self.add_field('length', self.u.unpack_one("B")) + self.add_field('handle', self.u.unpack_one("<H")) + + def fini(self): + if not self.u.at_end(): + self.add_field('data', self.u.unpack_rest()) + del self.u + + def fmtstr(self, i): + """Format the specified index and the associated string""" + return "{} '{}'".format(i, self.getstr(i)) + + def getstr(self, i): + """Get the string associated with the given index""" + if i == 0: + return "(none)" + if not hasattr(self, "strings"): + return "(error: structure has no strings)" + if i > len(self.strings): + return "(error: string index out of range)" + return self.strings[i - 1] + +class BIOSInformation(SmbiosBaseStructure): + smbios_structure_type = 0 + + def __init__(self, u, sm): + super(BIOSInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('vendor', u.unpack_one("B"), self.fmtstr) + self.add_field('version', u.unpack_one("B"), self.fmtstr) + self.add_field('starting_address_segment', u.unpack_one("<H")) + self.add_field('release_date', u.unpack_one("B"), self.fmtstr) + self.add_field('rom_size', u.unpack_one("B")) + self.add_field('characteristics', u.unpack_one("<Q")) + minor_version_str = str(sm.header.minor_version) # 34 is .34, 4 is .4, 41 is .41; compare ASCIIbetically to compare initial digits rather than numeric value + if (sm.header.major_version, minor_version_str) >= (2,"4"): + characteristic_bytes = 2 + else: + characteristic_bytes = self.length - 0x12 + self.add_field('characteristics_extensions', [u.unpack_one("B") for b in range(characteristic_bytes)]) + if (sm.header.major_version, minor_version_str) >= (2,"4"): + self.add_field('major_release', u.unpack_one("B")) + self.add_field('minor_release', u.unpack_one("B")) + self.add_field('ec_major_release', u.unpack_one("B")) + self.add_field('ec_minor_release', u.unpack_one("B")) + except: + self.decode_failure = True + print "Error parsing BIOSInformation" + import traceback + traceback.print_exc() + self.fini() + +class SystemInformation(SmbiosBaseStructure): + smbios_structure_type = 1 + + def __init__(self, u, sm): + super(SystemInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr) + self.add_field('product_name', u.unpack_one("B"), self.fmtstr) + self.add_field('version', u.unpack_one("B"), self.fmtstr) + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0x8: + self.add_field('uuid', uuid.UUID(bytes_le=u.unpack_one("16s"))) + wakeup_types = { + 0: 'Reserved', + 1: 'Other', + 2: 'Unknown', + 3: 'APM Timer', + 4: 'Modem Ring', + 5: 'LAN Remote', + 6: 'Power Switch', + 7: 'PCI PME#', + 8: 'AC Power Restored' + } + self.add_field('wakeup_type', u.unpack_one("B"), unpack.format_table("{}", wakeup_types)) + if self.length > 0x19: + self.add_field('sku_number', u.unpack_one("B"), self.fmtstr) + self.add_field('family', u.unpack_one("B"), self.fmtstr) + except: + self.decode_failure = True + print "Error parsing SystemInformation" + import traceback + traceback.print_exc() + self.fini() + +_board_types = { + 1: 'Unknown', + 2: 'Other', + 3: 'Server Blade', + 4: 'Connectivity Switch', + 5: 'System Management Module', + 6: 'Processor Module', + 7: 'I/O Module', + 8: 'Memory Module', + 9: 'Daughter Board', + 0xA: 'Motherboard', + 0xB: 'Processor/Memory Module', + 0xC: 'Processor/IO Module', + 0xD: 'Interconnect Board' +} + +class BaseboardInformation(SmbiosBaseStructure): + smbios_structure_type = 2 + + def __init__(self, u, sm): + super(BaseboardInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr) + self.add_field('product', u.unpack_one("B"), self.fmtstr) + self.add_field('version', u.unpack_one("B"), self.fmtstr) + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + + if self.length > 0x8: + self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr) + + if self.length > 0x9: + self.add_field('feature_flags', u.unpack_one("B")) + self.add_field('hosting_board', bool(bitfields.getbits(self.feature_flags, 0)), "feature_flags[0]={}") + self.add_field('requires_daughter_card', bool(bitfields.getbits(self.feature_flags, 1)), "feature_flags[1]={}") + self.add_field('removable', bool(bitfields.getbits(self.feature_flags, 2)), "feature_flags[2]={}") + self.add_field('replaceable', bool(bitfields.getbits(self.feature_flags, 3)), "feature_flags[3]={}") + self.add_field('hot_swappable', bool(bitfields.getbits(self.feature_flags, 4)), "feature_flags[4]={}") + + if self.length > 0xA: + self.add_field('location', u.unpack_one("B"), self.fmtstr) + + if self.length > 0xB: + self.add_field('chassis_handle', u.unpack_one("<H")) + + if self.length > 0xD: + self.add_field('board_type', u.unpack_one("B"), unpack.format_table("{}", _board_types)) + + if self.length > 0xE: + self.add_field('handle_count', u.unpack_one("B")) + if self.handle_count > 0: + self.add_field('contained_object_handles', tuple(u.unpack_one("<H") for i in range(self.handle_count))) + except: + self.decode_failure = True + print "Error parsing BaseboardInformation" + import traceback + traceback.print_exc() + self.fini() + +class SystemEnclosure(SmbiosBaseStructure): + smbios_structure_type = 3 + + def __init__(self, u, sm): + super(SystemEnclosure, self).__init__(u, sm) + u = self.u + try: + self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr) + self.add_field('enumerated_type', u.unpack_one("B")) + self.add_field('chassis_lock_present', bool(bitfields.getbits(self.enumerated_type, 7)), "enumerated_type[7]={}") + board_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Desktop', + 0x04: 'Low Profile Desktop', + 0x05: 'Pizza Box', + 0x06: 'Mini Tower', + 0x07: 'Tower', + 0x08: 'Portable', + 0x09: 'Laptop', + 0x0A: 'Notebook', + 0x0B: 'Hand Held', + 0x0C: 'Docking Station', + 0x0D: 'All in One', + 0x0E: 'Sub Notebook', + 0x0F: 'Space-saving', + 0x10: 'Lunch Box', + 0x11: 'Main Server Chassis', + 0x12: 'Expansion Chassis', + 0x13: 'SubChassis', + 0x14: 'Bus Expansion Chassis', + 0x15: 'Peripheral Chassis', + 0x16: 'RAID Chassis', + 0x17: 'Rack Mount Chassis', + 0x18: 'Sealed-case PC', + 0x19: 'Multi-system chassis W', + 0x1A: 'Compact PCI', + 0x1B: 'Advanced TCA', + 0x1C: 'Blade', + 0x1D: 'Blade Enclosure', + } + self.add_field('system_enclosure_type', bitfields.getbits(self.enumerated_type, 6, 0), unpack.format_table("enumerated_type[6:0]={}", board_types)) + self.add_field('version', u.unpack_one("B"), self.fmtstr) + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr) + minor_version_str = str(sm.header.minor_version) # 34 is .34, 4 is .4, 41 is .41; compare ASCIIbetically to compare initial digits rather than numeric value + if self.length > 9: + chassis_states = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Safe', + 0x04: 'Warning', + 0x05: 'Critical', + 0x06: 'Non-recoverable', + } + self.add_field('bootup_state', u.unpack_one("B"), unpack.format_table("{}", chassis_states)) + self.add_field('power_supply_state', u.unpack_one("B"), unpack.format_table("{}", chassis_states)) + self.add_field('thermal_state', u.unpack_one("B"), unpack.format_table("{}", chassis_states)) + security_states = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'None', + 0x04: 'External interface locked out', + 0x05: 'External interface enabled', + } + self.add_field('security_status', u.unpack_one("B"), unpack.format_table("{}", security_states)) + if self.length > 0xd: + self.add_field('oem_defined', u.unpack_one("<I")) + if self.length > 0x11: + self.add_field('height', u.unpack_one("B")) + self.add_field('num_power_cords', u.unpack_one("B")) + self.add_field('contained_element_count', u.unpack_one("B")) + self.add_field('contained_element_length', u.unpack_one("B")) + if getattr(self, 'contained_element_count', 0): + self.add_field('contained_elements', tuple(SystemEnclosureContainedElement(u, self.contained_element_length) for i in range(self.contained_element_count))) + if self.length > (0x15 + (getattr(self, 'contained_element_count', 0) * getattr(self, 'contained_element_length', 0))): + self.add_field('sku_number', u.unpack_one("B"), self.fmtstr) + except: + self.decode_failure = True + print "Error parsing SystemEnclosure" + import traceback + traceback.print_exc() + self.fini() + +class SystemEnclosureContainedElement(unpack.Struct): + def __init__(self, u, length): + super(SystemEnclosureContainedElement, self).__init__() + self.start_offset = u.offset + self.raw_data = u.unpack_raw(length) + self.u = unpack.Unpackable(self.raw_data) + u = self.u + self.add_field('contained_element_type', u.unpack_one("B")) + type_selections = { + 0: 'SMBIOS baseboard type enumeration', + 1: 'SMBIOS structure type enumeration', + } + self.add_field('type_select', bitfields.getbits(self.contained_element_type, 7), unpack.format_table("contained_element_type[7]={}", type_selections)) + self.add_field('type', bitfields.getbits(self.contained_element_type, 6, 0)) + if self.type_select == 0: + self.add_field('smbios_board_type', self.type, unpack.format_table("{}", _board_types)) + else: + self.add_field('smbios_structure_type', self.type) + self.add_field('minimum', u.unpack_one("B")) + self.add_field('maximum', u.unpack_one("B")) + if not u.at_end(): + self.add_field('data', u.unpack_rest()) + del self.u + +class ProcessorInformation(SmbiosBaseStructure): + smbios_structure_type = 4 + + def __init__(self, u, sm): + super(ProcessorInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('socket_designation', u.unpack_one("B"), self.fmtstr) + processor_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Central Processor', + 0x04: 'Math Processor', + 0x05: 'DSP Processor', + 0x06: 'Video Processor', + } + self.add_field('processor_type', u.unpack_one("B"), unpack.format_table("{}", processor_types)) + self.add_field('processor_family', u.unpack_one("B")) + self.add_field('processor_manufacturer', u.unpack_one("B"), self.fmtstr) + self.add_field('processor_id', u.unpack_one("<Q")) + self.add_field('processor_version', u.unpack_one("B"), self.fmtstr) + self.add_field('voltage', u.unpack_one("B")) + self.add_field('external_clock', u.unpack_one("<H")) + self.add_field('max_speed', u.unpack_one("<H")) + self.add_field('current_speed', u.unpack_one("<H")) + self.add_field('status', u.unpack_one("B")) + processor_upgrades = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Daughter Board', + 0x04: 'ZIF Socket', + 0x05: 'Replaceable Piggy Back', + 0x06: 'None', + 0x07: 'LIF Socket', + 0x08: 'Slot 1', + 0x09: 'Slot 2', + 0x0A: '370-pin socket', + 0x0B: 'Slot A', + 0x0C: 'Slot M', + 0x0D: 'Socket 423', + 0x0E: 'Socket A (Socket 462)', + 0x0F: 'Socket 478', + 0x10: 'Socket 754', + 0x11: 'Socket 940', + 0x12: 'Socket 939', + 0x13: 'Socket mPGA604', + 0x14: 'Socket LGA771', + 0x15: 'Socket LGA775', + 0x16: 'Socket S1', + 0x17: 'Socket AM2', + 0x18: 'Socket F (1207)', + 0x19: 'Socket LGA1366', + 0x1A: 'Socket G34', + 0x1B: 'Socket AM3', + 0x1C: 'Socket C32', + 0x1D: 'Socket LGA1156', + 0x1E: 'Socket LGA1567', + 0x1F: 'Socket PGA988A', + 0x20: 'Socket BGA1288', + 0x21: 'Socket rPGA988B', + 0x22: 'Socket BGA1023', + 0x23: 'Socket BGA1224', + 0x24: 'Socket BGA1155', + 0x25: 'Socket LGA1356', + 0x26: 'Socket LGA2011', + 0x27: 'Socket FS1', + 0x28: 'Socket FS2', + 0x29: 'Socket FM1', + 0x2A: 'Socket FM2', + } + self.add_field('processor_upgrade', u.unpack_one("B"), unpack.format_table("{}", processor_upgrades)) + if self.length > 0x1A: + self.add_field('l1_cache_handle', u.unpack_one("<H")) + self.add_field('l2_cache_handle', u.unpack_one("<H")) + self.add_field('l3_cache_handle', u.unpack_one("<H")) + if self.length > 0x20: + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr) + self.add_field('part_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0x24: + self.add_field('core_count', u.unpack_one("B")) + self.add_field('core_enabled', u.unpack_one("B")) + self.add_field('thread_count', u.unpack_one("B")) + self.add_field('processor_characteristics', u.unpack_one("<H")) + if self.length > 0x28: + self.add_field('processor_family_2', u.unpack_one("<H")) + if self.length > 0x2A: + self.add_field('core_count2', u.unpack_one("<H")) + self.add_field('core_enabled2', u.unpack_one("<H")) + self.add_field('thread_count2', u.unpack_one("<H")) + except: + self.decode_failure = True + print "Error parsing Processor Information" + import traceback + traceback.print_exc() + self.fini() + +class MemoryControllerInformation(SmbiosBaseStructure): #obsolete starting with v2.1 + smbios_structure_type = 5 + + def __init__(self, u, sm): + super(MemoryControllerInformation, self).__init__(u, sm) + u = self.u + try: + _error_detecting_method = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'None', + 0x04: '8-bit Parity', + 0x05: '32-bit ECC', + 0x06: '64-bit ECC', + 0x07: '128-bit ECC', + 0x08: 'CRC' + } + self.add_field('error_detecting_method', u.unpack_one("B"), unpack.format_table("{}", _error_detecting_method)) + self.add_field('error_correcting_capability', u.unpack_one("B")) + _interleaves = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'One-Way Interleave', + 0x04: 'Two-Way Interleave', + 0x05: 'Four-Way Interleave', + 0x06: 'Eight-Way Interleave', + 0x07: 'Sixteen-Way Interleave' + } + self.add_field('supported_interleave', u.unpack_one("B"), unpack.format_table("{}", _interleaves)) + self.add_field('current_interleave', u.unpack_one("B"), unpack.format_table("{}", _interleaves)) + self.add_field('max_memory_module_size', u.unpack_one("B"), self.fmtstr) + self.add_field('supported_speeds', u.unpack_one("<H")) + self.add_field('supported_memory_types', u.unpack_one("<H")) + self.add_field('memory_module_voltage', u.unpack_one("B")) + self.add_field('req_voltage_b2', bitfields.getbits(self.memory_module_voltage, 2), "memory_module_voltage[2]={}") + self.add_field('req_voltage_b1', bitfields.getbits(self.memory_module_voltage, 1), "memory_module_voltage[1]={}") + self.add_field('req_voltage_b0', bitfields.getbits(self.memory_module_voltage, 0), "memory_module_voltage[0]={}") + self.add_field('num_associated_memory_slots', u.unpack_one("B")) + self.add_field('memory_module_configuration_handles', u.unpack_one("<(self.num_associated_memory_slots)H")) + self.add_field('enabled_error_correcting_capabilities', u.unpack_one("B")) + except: + self.decode_failure = True + print "Error parsing MemoryControllerInformation" + import traceback + traceback.print_exc() + self.fini() + +class MemoryModuleInformation(SmbiosBaseStructure): #obsolete starting with v2.1 + smbios_structure_type = 6 + + def __init__(self, u, sm): + super(MemoryModuleInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('socket_designation', u.unpack_one("B"), self.fmtstr) + self.add_field('bank_connections', u.unpack_one("B")) + self.add_field('current_speed', u.unpack_one("B")) + self.add_field('current_memory_type', u.unpack_one("<H")) + _mem_connection = { + 0: 'single', + 1: 'double-bank' + } + self.add_field('installed_mem', u.unpack_one("B")) + self.add_field('installed_size', bitfields.getbits(self.installed_mem, 6, 0), "installed_mem[6:0]={}") + self.add_field('installed_memory_module_connection', bitfields.getbits(self.installed_mem, 7), unpack.format_table("installed_mem[7]={}", _mem_connection)) + self.add_field('enabled_mem', u.unpack_one("B")) + self.add_field('enabled_size', bitfields.getbits(self.installed_mem, 6, 0), "enabled_mem[6:0]={}") + self.add_field('enabled_memory_module_connection', bitfields.getbits(self.installed_mem, 7), unpack.format_table("enabled_mem[7]={}", _mem_connection)) + self.add_field('error_status', u.unpack_one("B")) + self.add_field('error_status_info_obstained_from_event_log', bool(bitfields.getbits(self.error_status, 2)), unpack.format_table("error_status[2]={}", _mem_connection)) + self.add_field('correctable_errors_received', bool(bitfields.getbits(self.error_status, 1)), unpack.format_table("error_status[1]={}", _mem_connection)) + self.add_field('uncorrectable_errors_received', bool(bitfields.getbits(self.error_status, 0)), unpack.format_table("error_status[0]={}", _mem_connection)) + except: + self.decode_failure = True + print "Error parsing MemoryModuleInformation" + import traceback + traceback.print_exc() + self.fini() + +class CacheInformation(SmbiosBaseStructure): + smbios_structure_type = 7 + + def __init__(self, u, sm): + super(CacheInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('socket_designation', u.unpack_one("B"), self.fmtstr) + processor_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Central Processor', + 0x04: 'Math Processor', + 0x05: 'DSP Processor', + 0x06: 'Video Processor', + } + self.add_field('cache_configuration', u.unpack_one("<H")) + _operational_mode = { + 0b00: 'Write Through', + 0b01: 'Write Back', + 0b10: 'Varies with Memory Address', + 0b11: 'Unknown' + } + self.add_field('operational_mode', bitfields.getbits(self.cache_configuration, 9, 8), unpack.format_table("cache_configuration[9:8]={}", _operational_mode)) + self.add_field('enabled_at_boot_time', bool(bitfields.getbits(self.cache_configuration, 7)), "cache_configuration[7]={}") + _location = { + 0b00: 'Internal', + 0b01: 'External', + 0b10: 'Reserved', + 0b11: 'Unknown' + } + self.add_field('location_relative_to_cpu_module', bitfields.getbits(self.cache_configuration, 6, 5), unpack.format_table("cache_configuration[6:5]={}", _location)) + self.add_field('cache_socketed', bool(bitfields.getbits(self.cache_configuration, 3)), "cache_configuration[3]={}") + self.add_field('cache_level', bitfields.getbits(self.cache_configuration, 2, 0), "cache_configuration[2:0]={}") + self.add_field('max_cache_size', u.unpack_one("<H")) + _granularity = { + 0: '1K granularity', + 1: '64K granularity' + } + self.add_field('max_granularity', bitfields.getbits(self.cache_configuration, 15), unpack.format_table("max_cache_size[15]={}", _granularity)) + self.add_field('max_size_in_granularity', bitfields.getbits(self.cache_configuration, 14, 0), "max_cache_size[14, 0]={}") + self.add_field('installed_size', u.unpack_one("<H")) + if self.installed_size != 0: + self.add_field('installed_granularity', bitfields.getbits(self.cache_configuration, 15), unpack.format_table("installed_size[15]={}", _granularity)) + self.add_field('installed_size_in_granularity', bitfields.getbits(self.cache_configuration, 14, 0), "installed_size[14, 0]={}") + self.add_field('supported_sram_type', u.unpack_one("<H")) + self.add_field('current_sram_type', u.unpack_one("<H")) + if self.length > 0x0F: + self.add_field('cache_speed', u.unpack_one("B")) + if self.length > 0x10: + _error_correction = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'None', + 0x04: 'Parity', + 0x05: 'Single-bit ECC', + 0x06: 'Multi-bit ECC' + } + self.add_field('error_correction', u.unpack_one("B"), unpack.format_table("{}", _error_correction)) + if self.length > 0x10: + _system_cache_type = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Instruction', + 0x04: 'Data', + 0x05: 'Unified' + } + self.add_field('system_cache_type', u.unpack_one("B"), unpack.format_table("{}", _system_cache_type)) + if self.length > 0x12: + _associativity = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Direct Mapped', + 0x04: '2-way Set-Associative', + 0x05: '4-way Set-Associative', + 0x06: 'Fully Associative', + 0x07: '8-way Set-Associative', + 0x08: '16-way Set-Associative', + 0x09: '12-way Set-Associative', + 0x0A: '24-way Set-Associative', + 0x0B: '32-way Set-Associative', + 0x0C: '48-way Set-Associative', + 0x0D: '64-way Set-Associative', + 0x0E: '20-way Set-Associative' + } + self.add_field('associativity', u.unpack_one("B"), unpack.format_table("{}", _associativity)) + + except: + self.decode_failure = True + print "Error parsing CacheInformation" + import traceback + traceback.print_exc() + self.fini() + +class PortConnectorInfo(SmbiosBaseStructure): + smbios_structure_type = 8 + + def __init__(self, u, sm): + super(PortConnectorInfo, self).__init__(u, sm) + u = self.u + try: + self.add_field('internal_reference_designator', u.unpack_one("B"), self.fmtstr) + connector_types = { + 0x00: 'None', + 0x01: 'Centronics', + 0x02: 'Mini Centronics', + 0x03: 'Proprietary', + 0x04: 'DB-25 pin male', + 0x05: 'DB-25 pin female', + 0x06: 'DB-15 pin male', + 0x07: 'DB-15 pin female', + 0x08: 'DB-9 pin male', + 0x09: 'DB-9 pin female', + 0x0A: 'RJ-11', + 0x0B: 'RJ-45', + 0x0C: '50-pin MiniSCSI', + 0x0D: 'Mini-DIN', + 0x0E: 'Micro-DIN', + 0x0F: 'PS/2', + 0x10: 'Infrared', + 0x11: 'HP-HIL', + 0x12: 'Access Bus (USB)', + 0x13: 'SSA SCSI', + 0x14: 'Circular DIN-8 male', + 0x15: 'Circular DIN-8 female', + 0x16: 'On Board IDE', + 0x17: 'On Board Floppy', + 0x18: '9-pin Dual Inline (pin 10 cut)', + 0x19: '25-pin Dual Inline (pin 26 cut)', + 0x1A: '50-pin Dual Inline', + 0x1B: '68-pin Dual Inline', + 0x1C: 'On Board Sound Input from CD-ROM', + 0x1D: 'Mini-Centronics Type-14', + 0x1E: 'Mini-Centronics Type-26', + 0x1F: 'Mini-jack (headphones)', + 0x20: 'BNC', + 0x21: '1394', + 0x22: 'SAS/SATA Plug Receptacle', + 0xA0: 'PC-98', + 0xA1: 'PC-98Hireso', + 0xA2: 'PC-H98', + 0xA3: 'PC-98Note', + 0xA4: 'PC-98Full', + 0xFF: 'Other', + } + self.add_field('internal_connector_type', u.unpack_one("B"), unpack.format_table("{}", connector_types)) + self.add_field('external_reference_designator', u.unpack_one("B"), self.fmtstr) + self.add_field('external_connector_type', u.unpack_one("B"), unpack.format_table("{}", connector_types)) + port_types = { + 0x00: 'None', + 0x01: 'Parallel Port XT/AT Compatible', + 0x02: 'Parallel Port PS/2', + 0x03: 'Parallel Port ECP', + 0x04: 'Parallel Port EPP', + 0x05: 'Parallel Port ECP/EPP', + 0x06: 'Serial Port XT/AT Compatible', + 0x07: 'Serial Port 16450 Compatible', + 0x08: 'Serial Port 16550 Compatible', + 0x09: 'Serial Port 16550A Compatible', + 0x0A: 'SCSI Port', + 0x0B: 'MIDI Port', + 0x0C: 'Joy Stick Port', + 0x0D: 'Keyboard Port', + 0x0E: 'Mouse Port', + 0x0F: 'SSA SCSI', + 0x10: 'USB', + 0x11: 'FireWire (IEEE P1394)', + 0x12: 'PCMCIA Type I2', + 0x13: 'PCMCIA Type II', + 0x14: 'PCMCIA Type III', + 0x15: 'Cardbus', + 0x16: 'Access Bus Port', + 0x17: 'SCSI II', + 0x18: 'SCSI Wide', + 0x19: 'PC-98', + 0x1A: 'PC-98-Hireso', + 0x1B: 'PC-H98', + 0x1C: 'Video Port', + 0x1D: 'Audio Port', + 0x1E: 'Modem Port', + 0x1F: 'Network Port', + 0x20: 'SATA', + 0x21: 'SAS', + 0xA0: '8251 Compatible', + 0xA1: '8251 FIFO Compatible', + 0xFF: 'Other', + } + self.add_field('port_type', u.unpack_one("B"), unpack.format_table("{}", port_types)) + except: + self.decodeFailure = True + print "Error parsing PortConnectorInfo" + import traceback + traceback.print_exc() + self.fini() + +class SystemSlots(SmbiosBaseStructure): + smbios_structure_type = 9 + + def __init__(self, u, sm): + super(SystemSlots, self).__init__(u, sm) + u = self.u + try: + self.add_field('designation', u.unpack_one("B"), self.fmtstr) + _slot_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'ISA', + 0x04: 'MCA', + 0x05: 'EISA', + 0x06: 'PCI', + 0x07: 'PC Card (PCMCIA)', + 0x08: 'VL-VESA', + 0x09: 'Proprietary', + 0x0A: 'Processor Card Slot', + 0x0B: 'Proprietary Memory Card Slot', + 0x0C: 'I/O Riser Card Slot', + 0x0D: 'NuBus', + 0x0E: 'PCI 66MHz Capable', + 0x0F: 'AGP', + 0x10: 'AGP 2X', + 0x11: 'AGP 4X', + 0x12: 'PCI-X', + 0x13: 'AGP 8X', + 0xA0: 'PC-98/C20', + 0xA1: 'PC-98/C24', + 0xA2: 'PC-98/E', + 0xA3: 'PC-98/Local Bus', + 0xA4: 'PC-98/Card', + 0xA5: 'PCI Express', + 0xA6: 'PCI Express x1', + 0xA7: 'PCI Express x2', + 0xA8: 'PCI Express x4', + 0xA9: 'PCI Express x8', + 0xAA: 'PCI Express x16', + 0xAB: 'PCI Express Gen 2', + 0xAC: 'PCI Express Gen 2 x1', + 0xAD: 'PCI Express Gen 2 x2', + 0xAE: 'PCI Express Gen 2 x4', + 0xAF: 'PCI Express Gen 2 x8', + 0xB0: 'PCI Express Gen 2 x16', + 0xB1: 'PCI Express Gen 3', + 0xB2: 'PCI Express Gen 3 x1', + 0xB3: 'PCI Express Gen 3 x2', + 0xB4: 'PCI Express Gen 3 x4', + 0xB5: 'PCI Express Gen 3 x8', + 0xB6: 'PCI Express Gen 3 x16', + } + self.add_field('slot_type', u.unpack_one("B"), unpack.format_table("{}", _slot_types)) + _slot_data_bus_widths = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: '8 bit', + 0x04: '16 bit', + 0x05: '32 bit', + 0x06: '64 bit', + 0x07: '128 bit', + 0x08: '1x or x1', + 0x09: '2x or x2', + 0x0A: '4x or x4', + 0x0B: '8x or x8', + 0x0C: '12x or x12', + 0x0D: '16x or x16', + 0x0E: '32x or x32', + } + self.add_field('slot_data_bus_width', u.unpack_one('B'), unpack.format_table("{}", _slot_data_bus_widths)) + _current_usages = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Available', + 0x04: 'In use', + } + self.add_field('current_usage', u.unpack_one('B'), unpack.format_table("{}", _current_usages)) + _slot_lengths = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Short Length', + 0x04: 'Long Length', + } + self.add_field('slot_length', u.unpack_one('B'), unpack.format_table("{}", _slot_lengths)) + self.add_field('slot_id', u.unpack_one('<H')) + self.add_field('characteristics1', u.unpack_one('B')) + self.add_field('characteristics_unknown', bool(bitfields.getbits(self.characteristics1, 0)), "characteristics1[0]={}") + self.add_field('provides_5_0_volts', bool(bitfields.getbits(self.characteristics1, 1)), "characteristics1[1]={}") + self.add_field('provides_3_3_volts', bool(bitfields.getbits(self.characteristics1, 2)), "characteristics1[2]={}") + self.add_field('shared_slot', bool(bitfields.getbits(self.characteristics1, 3)), "characteristics1[3]={}") + self.add_field('supports_pc_card_16', bool(bitfields.getbits(self.characteristics1, 4)), "characteristics1[4]={}") + self.add_field('supports_cardbus', bool(bitfields.getbits(self.characteristics1, 5)), "characteristics1[5]={}") + self.add_field('supports_zoom_video', bool(bitfields.getbits(self.characteristics1, 6)), "characteristics1[6]={}") + self.add_field('supports_modem_ring_resume', bool(bitfields.getbits(self.characteristics1, 7)), "characteristics1[7]={}") + if self.length > 0x0C: + self.add_field('characteristics2', u.unpack_one('B')) + self.add_field('supports_PME', bool(bitfields.getbits(self.characteristics2, 0)), "characteristics2[0]={}") + self.add_field('supports_hot_plug', bool(bitfields.getbits(self.characteristics2, 1)), "characteristics2[1]={}") + self.add_field('supports_smbus', bool(bitfields.getbits(self.characteristics2, 2)), "characteristics2[2]={}") + if self.length > 0x0D: + self.add_field('segment_group_number', u.unpack_one('<H')) + self.add_field('bus_number', u.unpack_one('B')) + self.add_field('device_function_number', u.unpack_one('B')) + self.add_field('device_number', bitfields.getbits(self.device_function_number, 7, 3), "device_function_number[7:3]={}") + self.add_field('function_number', bitfields.getbits(self.device_function_number, 2, 0), "device_function_number[2:0]={}") + except: + self.decodeFailure = True + print "Error parsing SystemSlots" + import traceback + traceback.print_exc() + self.fini() + +class OnBoardDevicesInformation(SmbiosBaseStructure): + smbios_structure_type = 10 + + def __init__(self, u, sm): + super(OnBoardDevicesInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('device_type', u.unpack_one("B")) + self.add_field('device_enabled', bool(bitfields.getbits(self.device_type, 7)), "device_type[7]={}") + _device_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Video', + 0x04: 'SCSI Controller', + 0x05: 'Ethernet', + 0x06: 'Token Ring', + 0x07: 'Sound', + 0x08: 'PATA Controller', + 0x09: 'SATA Controller', + 0x0A: 'SAS Controller' + } + self.add_field('type_of_device', bitfields.getbits(self.device_type, 6, 0), unpack.format_table("device_type[6:0]={}", _device_types)) + self.add_field('description_string', u.unpack_one("B"), self.fmtstr) + except: + self.decodeFailure = True + print "Error parsing OnBoardDevicesInformation" + import traceback + traceback.print_exc() + self.fini() + +class OEMStrings(SmbiosBaseStructure): + smbios_structure_type = 11 + + def __init__(self, u, sm): + super(OEMStrings, self).__init__(u, sm) + u = self.u + try: + self.add_field('count', u.unpack_one("B")) + except: + self.decodeFailure = True + print "Error parsing OEMStrings" + import traceback + traceback.print_exc() + self.fini() + +class SystemConfigOptions(SmbiosBaseStructure): + smbios_structure_type = 12 + + def __init__(self, u, sm): + super(SystemConfigOptions, self).__init__(u, sm) + u = self.u + try: + self.add_field('count', u.unpack_one("B")) + except: + self.decodeFailure = True + print "Error parsing SystemConfigOptions" + import traceback + traceback.print_exc() + self.fini() + +class BIOSLanguageInformation(SmbiosBaseStructure): + smbios_structure_type = 13 + + def __init__(self, u, sm): + super(BIOSLanguageInformation, self).__init__(u, sm) + u = self.u + try: + self.add_field('installable_languages', u.unpack_one("B")) + if self.length > 0x05: + self.add_field('flags', u.unpack_one('B')) + self.add_field('abbreviated_format', bool(bitfields.getbits(self.flags, 0)), "flags[0]={}") + if self.length > 0x6: + u.skip(15) + self.add_field('current_language', u.unpack_one('B'), self.fmtstr) + except: + self.decodeFailure = True + print "Error parsing BIOSLanguageInformation" + import traceback + traceback.print_exc() + self.fini() + +class GroupAssociations(SmbiosBaseStructure): + smbios_structure_type = 14 + + def __init__(self, u, sm): + super(GroupAssociations, self).__init__(u, sm) + u = self.u + try: + self.add_field('group_name', u.unpack_one("B"), self.fmtstr) + self.add_field('item_type', u.unpack_one('B')) + self.add_field('item_handle', u.unpack_one('<H')) + except: + self.decodeFailure = True + print "Error parsing GroupAssociations" + import traceback + traceback.print_exc() + self.fini() + +class SystemEventLog(SmbiosBaseStructure): + smbios_structure_type = 15 + + def __init__(self, u, sm): + super(SystemEventLog, self).__init__(u, sm) + u = self.u + try: + self.add_field('log_area_length', u.unpack_one("<H")) + self.add_field('log_header_start_offset', u.unpack_one('<H')) + self.add_field('log_data_start_offset', u.unpack_one('<H')) + _access_method = { + 0x00: 'Indexed I/O: 1 8-bit index port, 1 8-bit data port', + 0x01: 'Indexed I/O: 2 8-bit index ports, 1 8-bit data port', + 0x02: 'Indexed I/O: 1 16-bit index port, 1 8-bit data port', + 0x03: 'Memory-mapped physical 32-bit address', + 0x04: 'Available through General-Purpose NonVolatile Data functions', + xrange(0x05, 0x07F): 'Available for future assignment', + xrange(0x80, 0xFF): 'BIOS Vendor/OEM-specific' + } + self.add_field('access_method', u.unpack_one('B'), unpack.format_table("{}", _access_method)) + self.add_field('log_status', u.unpack_one('B')) + self.add_field('log_area_full', bool(bitfields.getbits(self.log_status, 1)), "log_status[1]={}") + self.add_field('log_area_valid', bool(bitfields.getbits(self.log_status, 0)), "log_status[0]={}") + self.add_field('log_change_token', u.unpack_one('<I')) + self.add_field('access_method_address', u.unpack_one('<I')) + if self.length > 0x14: + _log_header_formats = { + 0: 'No header', + 1: 'Type 1 log header', + xrange(2, 0x7f): 'Available for future assignment', + xrange(0x80, 0xff): 'BIOS vendor or OEM-specific format' + } + self.add_field('log_header_format', u.unpack_one("B"), unpack.format_table("{}", _log_header_formats)) + if self.length > 0x15: + self.add_field('num_supported_log_type_descriptors', u.unpack_one('B')) + if self.length > 0x16: + self.add_field('length_log_type_descriptor', u.unpack_one('B')) + if self.length != (0x17 + (self.num_supported_log_type_descriptors * self.length_log_type_descriptor)): + print "Error: structure length ({}) != 0x17 + (num_supported_log_type_descriptors ({}) * length_log_type_descriptor({}))".format(self.length, self.num_supported_log_type_descriptors, self.length_log_type_descriptor) + print "structure length = {}".format(self.length) + print "num_supported_log_type_descriptors = {}".format(self.num_supported_log_type_descriptors) + print "length_log_type_descriptor = {}".format(self.length_log_type_descriptor) + self.decodeFailure = True + self.add_field('descriptors', tuple(EventLogDescriptor.unpack(u) for i in range(self.num_supported_log_type_descriptors)), unpack.format_each("\n{!r}")) + except: + self.decodeFailure = True + print "Error parsing SystemEventLog" + import traceback + traceback.print_exc() + self.fini() + +class EventLogDescriptor(unpack.Struct): + @staticmethod + def _unpack(u): + _event_log_type_descriptors = { + 0x00: 'Reserved', + 0x01: 'Single-bit ECC memory error', + 0x02: 'Multi-bit ECC memory error', + 0x03: 'Parity memory error', + 0x04: 'Bus time-out', + 0x05: 'I/O Channel Check', + 0x06: 'Software NMI', + 0x07: 'POST Memory Resize', + 0x08: 'POST Error', + 0x09: 'PCI Parity Error', + 0x0A: 'PCI System Error', + 0x0B: 'CPU Failure', + 0x0C: 'EISA FailSafe Timer time-out', + 0x0D: 'Correctable memory log disabled', + 0x0E: 'Logging disabled for a specific Event Type - too many errors of the same type received in a short amount of time', + 0x0F: 'Reserved', + 0x10: 'System Limit Exceeded', + 0x11: 'Asynchronous hardware timer expired and issued a system reset', + 0x12: 'System configuration information', + 0x13: 'Hard-disk information', + 0x14: 'System reconfigured', + 0x15: 'Uncorrectable CPU-complex error', + 0x16: 'Log Area Reset/Cleared', + 0x17: 'System boot', + xrange(0x18, 0x7F): 'Unused, available for assignment', + xrange(0x80, 0xFE): 'Availalbe for system- and OEM-specific assignments', + 0xFF: 'End of log' + } + yield 'log_type', u.unpack_one('B'), unpack.format_table("{}", _event_log_type_descriptors) + _event_log_format = { + 0x00: 'None', + 0x01: 'Handle', + 0x02: 'Multiple-Event', + 0x03: 'Multiple-Event Handle', + 0x04: 'POST Results Bitmap', + 0x05: 'System Management Type', + 0x06: 'Multiple-Event System Management Type', + xrange(0x80, 0xFF): 'OEM assigned' + } + yield 'variable_data_format_type', u.unpack_one('B'), unpack.format_table("{}", _event_log_format) + +class PhysicalMemoryArray(SmbiosBaseStructure): + smbios_structure_type = 16 + + def __init__(self, u, sm): + super(PhysicalMemoryArray, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + _location_field = { + 0x01: "Other", + 0x02: "Unknown", + 0x03: "System board or motherboard", + 0x04: "ISA add-on card", + 0x05: "EISA add-on card", + 0x06: "PCI add-on card", + 0x07: "MCA add-on card", + 0x08: "PCMCIA add-on card", + 0x09: "Proprietary add-on card", + 0x0A: "NuBus", + 0xA0: "PC-98/C20 add-on card", + 0xA1: "PC-98/C24 add-on card", + 0xA2: "PC-98/E add-on card", + 0xA3: "PC-98/Local bus add-on card" + } + self.add_field('location', u.unpack_one("B"), unpack.format_table("{}", _location_field)) + if self.length > 0x05: + _use = { + 0x01: "Other", + 0x02: "Unknown", + 0x03: "System memory", + 0x04: "Video memory", + 0x05: "Flash memory", + 0x06: "Non-volatile RAM", + 0x07: "Cache memory" + } + self.add_field('use', u.unpack_one('B'), unpack.format_table("{}", _use)) + if self.length > 0x06: + _error_correction = { + 0x01: "Other", + 0x02: "Unknown", + 0x03: "None", + 0x04: "Parity", + 0x05: "Single-bit ECC", + 0x06: "Multi-bit ECC", + 0x07: "CRC" + } + self.add_field('memory_error_correction', u.unpack_one('B'), unpack.format_table("{}", _error_correction)) + if self.length > 0x07: + self.add_field('maximum_capacity', u.unpack_one('<I')) + if self.length > 0x0B: + self.add_field('memory_error_information_handle', u.unpack_one('<H')) + if self.length > 0x0D: + self.add_field('num_memory_devices', u.unpack_one('<H')) + if self.length > 0x0F: + self.add_field('extended_maximum_capacity', u.unpack_one('<Q')) + except: + self.decodeFailure = True + print "Error parsing PhysicalMemoryArray" + import traceback + traceback.print_exc() + self.fini() + +class MemoryDevice(SmbiosBaseStructure): + smbios_structure_type = 17 + + def __init__(self, u, sm): + super(MemoryDevice, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('physical_memory_array_handle', u.unpack_one("<H")) + if self.length > 0x6: + self.add_field('memory_error_information_handle', u.unpack_one("<H")) + if self.length > 0x8: + self.add_field('total_width', u.unpack_one("<H")) + if self.length > 0xA: + self.add_field('data_width', u.unpack_one("<H")) + if self.length > 0xC: + self.add_field('size', u.unpack_one("<H")) + if self.length > 0xE: + _form_factors = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'SIMM', + 0x04: 'SIP', + 0x05: 'Chip', + 0x06: 'DIP', + 0x07: 'ZIP', + 0x08: 'Proprietary Card', + 0x09: 'DIMM', + 0x0A: 'TSOP', + 0x0B: 'Row of chips', + 0x0C: 'RIMM', + 0x0D: 'SODIMM', + 0x0E: 'SRIMM', + 0x0F: 'FB-DIMM' + } + self.add_field('form_factor', u.unpack_one("B"), unpack.format_table("{}", _form_factors)) + if self.length > 0xF: + self.add_field('device_set', u.unpack_one("B")) + if self.length > 0x10: + self.add_field('device_locator', u.unpack_one("B"), self.fmtstr) + if self.length > 0x11: + self.add_field('bank_locator', u.unpack_one("B"), self.fmtstr) + if self.length > 0x12: + _memory_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'DRAM', + 0x04: 'EDRAM', + 0x05: 'VRAM', + 0x06: 'SRAM', + 0x07: 'RAM', + 0x08: 'ROM', + 0x09: 'FLASH', + 0x0A: 'EEPROM', + 0x0B: 'FEPROM', + 0x0C: 'EPROM', + 0x0D: 'CDRAM', + 0x0E: '3DRAM', + 0x0F: 'SDRAM', + 0x10: 'SGRAM', + 0x11: 'RDRAM', + 0x12: 'DDR', + 0x13: 'DDR2', + 0x14: 'DDR2 FB-DIMM', + xrange(0x15, 0x17): 'Reserved', + 0x18: 'DDR3', + 0x19: 'FBD2' + } + self.add_field('memory_type', u.unpack_one("B"), unpack.format_table("{}", _memory_types)) + if self.length > 0x13: + self.add_field('type_detail', u.unpack_one('<H')) + if self.length > 0x15: + self.add_field('speed', u.unpack_one("<H")) + if self.length > 0x17: + self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr) + if self.length > 0x18: + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0x19: + self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr) + if self.length > 0x1A: + self.add_field('part_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0x1B: + self.add_field('attributes', u.unpack_one("B")) + self.add_field('rank', bitfields.getbits(self.attributes, 3, 0), "attributes[3:0]={}") + if self.length > 0x1C: + if self.size == 0x7FFF: + self.add_field('extended_size', u.unpack_one('<I')) + self.add_field('mem_size', bitfields.getbits(self.type_detail, 30, 0), "type_detail[30:0]={}") + else: + u.skip(4) + if self.length > 0x20: + self.add_field('configured_memory_clock_speed', u.unpack_one("<H")) + if self.length > 0x22: + self.add_field('minimum_voltage', u.unpack_one("<H")) + if self.length > 0x24: + self.add_field('maximum_voltage', u.unpack_one("<H")) + if self.length > 0x26: + self.add_field('configured_voltage', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing MemoryDevice" + import traceback + traceback.print_exc() + self.fini() + +class MemoryErrorInfo32Bit(SmbiosBaseStructure): + smbios_structure_type = 18 + + def __init__(self, u, sm): + super(MemoryErrorInfo32Bit, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + _error_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'OK', + 0x04: 'Bad read', + 0x05: 'Parity error', + 0x06: 'Single-bit error', + 0x07: 'Double-bit error', + 0x08: 'Multi-bit error', + 0x09: 'Nibble error', + 0x0A: 'Checksum error', + 0x0B: 'CRC error', + 0x0C: 'Corrected single-bit error', + 0x0D: 'Corrected error', + 0x0E: 'Uncorrectable error' + } + self.add_field('error_type', u.unpack_one("B"), unpack.format_table("{}", _error_types)) + if self.length > 0x5: + _error_granularity_field = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Device level', + 0x04: 'Memory partition level' + } + self.add_field('error_granularity', u.unpack_one("B"), unpack.format_table("{}", _error_granularity_field)) + if self.length > 0x6: + _error_operation_field = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Read', + 0x04: 'Write', + 0x05: 'Partial write' + } + self.add_field('error_operation', u.unpack_one("B"), unpack.format_table("{}", _error_operation_field)) + if self.length > 0x7: + self.add_field('vendor_syndrome', u.unpack_one("<I")) + if self.length > 0xB: + self.add_field('memory_array_error_address', u.unpack_one("<I")) + if self.length > 0xF: + self.add_field('device_error_address', u.unpack_one("<I")) + if self.length > 0x13: + self.add_field('error_resolution', u.unpack_one("<I")) + except: + self.decodeFailure = True + print "Error parsing MemoryErrorInfo32Bit" + import traceback + traceback.print_exc() + self.fini() + +class MemoryArrayMappedAddress(SmbiosBaseStructure): + smbios_structure_type = 19 + + def __init__(self, u, sm): + super(MemoryArrayMappedAddress, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('starting_address', u.unpack_one("<I")) + # if FFFF FFFF: address stored in Extended Starting Address + if self.length > 0x8: + self.add_field('ending_address', u.unpack_one("<I")) + if self.length > 0xC: + self.add_field('memory_array_handle', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('partition_width', u.unpack_one("B")) + if self.length > 0xF: + # valid if starting_address = FFFF FFFF + if self.starting_address == 0xFFFFFFFF: + self.add_field('extended_starting_address', u.unpack_one("<Q")) + if self.length > 0x17: + self.add_field('extended_ending_address', u.unpack_one("<Q")) + else: + u.skip(16) + + except: + self.decodeFailure = True + print "Error parsing MemoryArrayMappedAddress" + import traceback + traceback.print_exc() + self.fini() + +class MemoryDeviceMappedAddress(SmbiosBaseStructure): + smbios_structure_type = 20 + + def __init__(self, u, sm): + super(MemoryDeviceMappedAddress, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('starting_address', u.unpack_one("<I")) + # if FFFF FFFF: address stored in Extended Starting Address + if self.length > 0x8: + self.add_field('ending_address', u.unpack_one("<I")) + if self.length > 0xC: + self.add_field('memory_device_handle', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('memory_array_mapped_address_handle', u.unpack_one("<H")) + if self.length > 0x10: + self.add_field('partition_row_position', u.unpack_one("B")) + if self.length > 0x11: + self.add_field('interleave_position', u.unpack_one("B")) + if self.length > 0x12: + self.add_field('interleave_data_depth', u.unpack_one("B")) + if self.length > 0x13: + # valid if starting_address = FFFF FFFF + if self.starting_address == 0xFFFFFFFF: + self.add_field('extended_starting_address', u.unpack_one("<Q")) + if self.length > 0x1B: + self.add_field('extended_ending_address', u.unpack_one("<Q")) + else: + u.skip(16) + except: + self.decodeFailure = True + print "Error parsing MemoryDeviceMappedAddress" + import traceback + traceback.print_exc() + self.fini() + +class BuiltInPointingDevice(SmbiosBaseStructure): + smbios_structure_type = 21 + + def __init__(self, u, sm): + super(BuiltInPointingDevice, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + _pointing_device_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Mouse', + 0x04: 'Track Ball', + 0x05: 'Track Point', + 0x06: 'Glide Point', + 0x07: 'Touch Pad', + 0x08: 'Touch Screen', + 0x09: 'Optical Sensor' + } + self.add_field('pointing_device_type', u.unpack_one("B"), unpack.format_table("{}", _pointing_device_types)) + if self.length > 0x5: + _interfaces = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Serial', + 0x04: 'PS/2', + 0x05: 'Infared', + 0x06: 'HP-HIL', + 0x07: 'Bus mouse', + 0x08: 'ADB (Apple Desktop Bus)', + 0x09: 'Bus mouse DB-9', + 0x0A: 'Bus mouse micro-DIN', + 0x0B: 'USB' + } + self.add_field('interface', u.unpack_one("B"), unpack.format_table("{}", _interfaces)) + if self.length > 0x6: + self.add_field('num_buttons', u.unpack_one("B")) + except: + self.decodeFailure = True + print "Error parsing BuiltInPointingDevice" + import traceback + traceback.print_exc() + self.fini() + +class PortableBattery(SmbiosBaseStructure): + smbios_structure_type = 22 + + def __init__(self, u, sm): + super(PortableBattery, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('location', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr) + if self.length > 0x6: + self.add_field('manufacturer_date', u.unpack_one("B"), self.fmtstr) + if self.length > 0x7: + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0x8: + self.add_field('device_name', u.unpack_one("B"), self.fmtstr) + if self.length > 0x9: + _device_chemistry = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Lead Acid', + 0x04: 'Nickel Cadmium', + 0x05: 'Nickel metal hydride', + 0x06: 'Lithium-ion', + 0x07: 'Zinc air', + 0x08: 'Lithium Polymer' + } + self.add_field('device_chemistry', u.unpack_one("B"), unpack.format_table("{}", _device_chemistry)) + if self.length > 0xA: + self.add_field('design_capacity', u.unpack_one("<H")) + if self.length > 0xC: + self.add_field('design_voltage', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('sbds_version_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0xF: + self.add_field('max_error_battery_data', u.unpack_one("B"), self.fmtstr) + if self.length > 0x10: + if self.serial_number == 0: + self.add_field('sbds_serial_number', u.unpack_one("<H")) + else: + u.skip(2) + if self.length > 0x12: + if self.manufacturer_date == 0: + self.add_field('sbds_manufacture_date', u.unpack_one("<H")) + self.add_field('year_biased_by_1980', bitfields.getbits(self.sbds_manufacture_date, 15, 9), "sbds_manufacture_date[15:9]={}") + self.add_field('month', bitfields.getbits(self.sbds_manufacture_date, 8, 5), "sbds_manufacture_date[8:5]={}") + self.add_field('date', bitfields.getbits(self.sbds_manufacture_date, 4, 0), "sbds_manufacture_date[4:0]={}") + else: + u.skip(2) + if self.length > 0x14: + if self.device_chemistry == 0x02: + self.add_field('sbds_device_chemistry', u.unpack_one("B"), self.fmtstr) + else: + u.skip(1) + if self.length > 0x15: + self.add_field('design_capacity_multiplier', u.unpack_one("B")) + if self.length > 0x16: + self.add_field('oem_specific', u.unpack_one("<I")) + except: + self.decodeFailure = True + print "Error parsing PortableBattery" + import traceback + traceback.print_exc() + self.fini() + +class SystemReset(SmbiosBaseStructure): + smbios_structure_type = 23 + + def __init__(self, u, sm): + super(SystemReset, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('capabilities', u.unpack_one("B")) + self.add_field('contains_watchdog_timer', bool(bitfields.getbits(self.capabilities, 5)), "capabilities[5]={}") + _boot_option = { + 0b00: 'Reserved, do not use', + 0b01: 'Operating System', + 0b10: 'System utilities', + 0b11: 'Do not reboot' + } + self.add_field('boot_option_on_limit', bitfields.getbits(self.capabilities, 4, 3), unpack.format_table("capabilities[4:3]={}", _boot_option)) + self.add_field('boot_option_after_watchdog_reset', bitfields.getbits(self.capabilities, 2, 1), unpack.format_table("capabilities[2:1]={}", _boot_option)) + self.add_field('system_reset_enabled_by_user', bool(bitfields.getbits(self.capabilities, 0)), "capabilities[0]={}") + if self.length > 0x5: + self.add_field('reset_count', u.unpack_one("<H")) + if self.length > 0x5: + self.add_field('reset_limit', u.unpack_one("<H")) + if self.length > 0x9: + self.add_field('timer_interval', u.unpack_one("<H")) + if self.length > 0xB: + self.add_field('timeout', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing SystemReset" + import traceback + traceback.print_exc() + self.fini() + +class HardwareSecurity(SmbiosBaseStructure): + smbios_structure_type = 24 + + def __init__(self, u, sm): + super(HardwareSecurity, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('hardware_security_settings', u.unpack_one("B")) + _status = { + 0x00: 'Disabled', + 0x01: 'Enabled', + 0x02: 'Not Implemented', + 0x03: 'Unknown' + } + self.add_field('power_on_password_status', bitfields.getbits(self.hardware_security_settings, 7, 6), unpack.format_table("hardware_security_settings[7:6]={}", _status)) + self.add_field('keyboard_password_status', bitfields.getbits(self.hardware_security_settings, 5, 4), unpack.format_table("hardware_security_settings[5:4]={}", _status)) + self.add_field('admin_password_status', bitfields.getbits(self.hardware_security_settings, 3, 2), unpack.format_table("hardware_security_settings0[3:2]={}", _status)) + self.add_field('front_panel_reset_status', bitfields.getbits(self.hardware_security_settings, 1, 0), unpack.format_table("hardware_security_settings[1:0]={}", _status)) + except: + self.decodeFailure = True + print "Error parsing HardwareSecurity" + import traceback + traceback.print_exc() + self.fini() + +class SystemPowerControls(SmbiosBaseStructure): + smbios_structure_type = 25 + + def __init__(self, u, sm): + super(SystemPowerControls, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('next_scheduled_poweron_month', u.unpack_one("B")) + self.add_field('next_scheduled_poweron_day_of_month', u.unpack_one("B")) + self.add_field('next_scheduled_poweron_hour', u.unpack_one("B")) + self.add_field('next_scheduled_poweron_minute', u.unpack_one("B")) + self.add_field('next_scheduled_poweron_second', u.unpack_one("B")) + except: + self.decodeFailure = True + print "Error parsing SystemPowerControls" + import traceback + traceback.print_exc() + self.fini() + +class VoltageProbe(SmbiosBaseStructure): + smbios_structure_type = 26 + + def __init__(self, u, sm): + super(VoltageProbe, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('description', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('location_and_status', u.unpack_one("B")) + _status = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'OK', + 0b100: 'Non-critical', + 0b101: 'Critical', + 0b110: 'Non-recoverable' + } + _location = { + 0b00001: 'Other', + 0b00010: 'Unknown', + 0b00011: 'Processor', + 0b00100: 'Disk', + 0b00101: 'Peripheral Bay', + 0b00110: 'System Management Module', + 0b00111: 'Motherboard', + 0b01000: 'Memory Module', + 0b01001: 'Processor Module', + 0b01010: 'Power Unit', + 0b01011: 'Add-in Card' + } + self.add_field('status', bitfields.getbits(self.location_and_status, 7, 5), unpack.format_table("location_and_status[7:5]={}", _status)) + self.add_field('location', bitfields.getbits(self.location_and_status, 4, 0), unpack.format_table("location_and_status[4:0]={}", _location)) + if self.length > 0x6: + self.add_field('max_value', u.unpack_one("<H")) + if self.length > 0x8: + self.add_field('min_value', u.unpack_one("<H")) + if self.length > 0xA: + self.add_field('resolution', u.unpack_one("<H")) + if self.length > 0xC: + self.add_field('tolerance', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('accuracy', u.unpack_one("<H")) + if self.length > 0x10: + self.add_field('oem_defined', u.unpack_one("<I")) + if self.length > 0x14: + self.add_field('nominal_value', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing VoltageProbe" + import traceback + traceback.print_exc() + self.fini() + +class CoolingDevice(SmbiosBaseStructure): + smbios_structure_type = 27 + + def __init__(self, u, sm): + super(CoolingDevice, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('temperature_probe_handle', u.unpack_one("<H")) + if self.length > 0x6: + self.add_field('device_type_and_status', u.unpack_one("B")) + _status = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'OK', + 0b100: 'Non-critical', + 0b101: 'Critical', + 0b110: 'Non-recoverable' + } + _type = { + 0b00001: 'Other', + 0b00010: 'Unknown', + 0b00011: 'Fan', + 0b00100: 'Centrifugal Blower', + 0b00101: 'Chip Fan', + 0b00110: 'Cabinet Fan', + 0b00111: 'Power Supply Fan', + 0b01000: 'Heat Pipe', + 0b01001: 'Integrated Refrigeration', + 0b10000: 'Active Cooling', + 0b10001: 'Passive Cooling' + } + self.add_field('status', bitfields.getbits(self.device_type_and_status, 7, 5), unpack.format_table("device_type_and_status[7:5]={}", _status)) + self.add_field('device_type', bitfields.getbits(self.device_type_and_status, 4, 0), unpack.format_table("device_type_and_status[4:0]={}", _type)) + if self.length > 0x7: + self.add_field('cooling_unit_group', u.unpack_one("B")) + if self.length > 0x8: + self.add_field('OEM_defined', u.unpack_one("<I")) + if self.length > 0xC: + self.add_field('nominal_speed', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('description', u.unpack_one("B"), self.fmtstr) + except: + self.decodeFailure = True + print "Error parsing CoolingDevice" + import traceback + traceback.print_exc() + self.fini() + +class TemperatureProbe(SmbiosBaseStructure): + smbios_structure_type = 28 + + def __init__(self, u, sm): + super(TemperatureProbe, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('description', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('location_and_status', u.unpack_one("B")) + _status = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'OK', + 0b100: 'Non-critical', + 0b101: 'Critical', + 0b110: 'Non-recoverable' + } + _location = { + 0b00001: 'Other', + 0b00010: 'Unknown', + 0b00011: 'Processor', + 0b00100: 'Disk', + 0b00101: 'Peripheral Bay', + 0b00110: 'System Management Module', + 0b00111: 'Motherboard', + 0b01000: 'Memory Module', + 0b01001: 'Processor Module', + 0b01010: 'Power Unit', + 0b01011: 'Add-in Card', + 0b01100: 'Front Panel Board', + 0b01101: 'Back Panel Board', + 0b01110: 'Power System Board', + 0b01111: 'Drive Back Plane' + } + self.add_field('status', bitfields.getbits(self.location_and_status, 7, 5), unpack.format_table("location_and_status[7:5]={}", _status)) + self.add_field('location', bitfields.getbits(self.location_and_status, 4, 0), unpack.format_table("location_and_status[4:0]={}", _location)) + if self.length > 0x6: + self.add_field('maximum_value', u.unpack_one("<H")) + if self.length > 0x8: + self.add_field('minimum_value', u.unpack_one("<H")) + if self.length > 0xA: + self.add_field('resolution', u.unpack_one("<H")) + if self.length > 0xC: + self.add_field('tolerance', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('accuracy', u.unpack_one("<H")) + if self.length > 0x10: + self.add_field('OEM_defined', u.unpack_one("<I")) + if self.length > 0x14: + self.add_field('nominal_value', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing TemperatureProbe" + import traceback + traceback.print_exc() + self.fini() + +class ElectricalCurrentProbe(SmbiosBaseStructure): + smbios_structure_type = 29 + + def __init__(self, u, sm): + super(ElectricalCurrentProbe, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('description', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('location_and_status', u.unpack_one("B")) + _status = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'OK', + 0b100: 'Non-critical', + 0b101: 'Critical', + 0b110: 'Non-recoverable' + } + _location = { + 0b00001: 'Other', + 0b00010: 'Unknown', + 0b00011: 'Processor', + 0b00100: 'Disk', + 0b00101: 'Peripheral Bay', + 0b00110: 'System Management Module', + 0b00111: 'Motherboard', + 0b01000: 'Memory Module', + 0b01001: 'Processor Module', + 0b01010: 'Power Unit', + 0b01011: 'Add-in Card', + 0b01100: 'Front Panel Board', + 0b01101: 'Back Panel Board', + 0b01110: 'Power System Board', + 0b01111: 'Drive Back Plane' + } + self.add_field('status', bitfields.getbits(self.location_and_status, 7, 5), unpack.format_table("location_and_status[7:5]={}", _status)) + self.add_field('location', bitfields.getbits(self.location_and_status, 4, 0), unpack.format_table("location_and_status[4:0]={}", _location)) + if self.length > 0x6: + self.add_field('maximum_value', u.unpack_one("<H")) + if self.length > 0x8: + self.add_field('minimum_value', u.unpack_one("<H")) + if self.length > 0xA: + self.add_field('resolution', u.unpack_one("<H")) + if self.length > 0xC: + self.add_field('tolerance', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('accuracy', u.unpack_one("<H")) + if self.length > 0x10: + self.add_field('OEM_defined', u.unpack_one("<I")) + if self.length > 0x14: + self.add_field('nominal_value', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing ElectricalCurrentProbe" + import traceback + traceback.print_exc() + self.fini() + +class OutOfBandRemoteAccess(SmbiosBaseStructure): + smbios_structure_type = 30 + + def __init__(self, u, sm): + super(OutOfBandRemoteAccess, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('manufacturer_name', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('connections', u.unpack_one("B")) + self.add_field('outbound_connection_enabled', bool(bitfields.getbits(self.connections, 1)), "connections[1]={}") + self.add_field('inbound_connection_enabled', bool(bitfields.getbits(self.connections, 0)), "connections[0]={}") + except: + self.decodeFailure = True + print "Error parsing OutOfBandRemoteAccess" + import traceback + traceback.print_exc() + self.fini() + +class BootIntegrityServicesEntryPoint(SmbiosBaseStructure): + smbios_structure_type = 31 + +class SystemBootInformation(SmbiosBaseStructure): + smbios_structure_type = 32 + + def __init__(self, u, sm): + super(SystemBootInformation, self).__init__(u, sm) + u = self.u + try: + if self.length > 0xA: + u.skip(6) + _boot_status = { + 0: 'No errors detected', + 1: 'No bootable media', + 2: '"normal" operating system failed to load', + 3: 'Firmware-detected hardware failure, including "unknown" failure types', + 4: 'Operating system-detected hardware failure', + 5: 'User-requested boot, usually through a keystroke', + 6: 'System security violation', + 7: 'Previously-requested image', + 8: 'System watchdog timer expired, causing the system to reboot', + xrange(9,127): 'Reserved for future assignment', + xrange(128, 191): 'Vendor/OEM-specific implementations', + xrange(192, 255): 'Product-specific implementations' + } + self.add_field('boot_status', u.unpack_one("B"), unpack.format_table("{}", _boot_status)) + except: + self.decodeFailure = True + print "Error parsing SystemBootInformation" + import traceback + traceback.print_exc() + self.fini() + +class MemoryErrorInfo64Bit(SmbiosBaseStructure): + smbios_structure_type = 33 + + def __init__(self, u, sm): + super(MemoryErrorInfo64Bit, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + _error_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'OK', + 0x04: 'Bad read', + 0x05: 'Parity error', + 0x06: 'Single-bit error', + 0x07: 'Double-bit error', + 0x08: 'Multi-bit error', + 0x09: 'Nibble error', + 0x0A: 'Checksum error', + 0x0B: 'CRC error', + 0x0C: 'Corrected single-bit error', + 0x0D: 'Corrected error', + 0x0E: 'Uncorrectable error' + } + self.add_field('error_type', u.unpack_one("B"), unpack.format_table("{}", _error_types)) + if self.length > 0x5: + _error_granularity_field = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Device level', + 0x04: 'Memory partition level' + } + self.add_field('error_granularity', u.unpack_one("B"), unpack.format_table("{}", _error_granularity_field)) + if self.length > 0x6: + _error_operation_field = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Read', + 0x04: 'Write', + 0x05: 'Partial write' + } + self.add_field('error_operation', u.unpack_one("B"), unpack.format_table("{}", _error_operation_field)) + if self.length > 0x7: + self.add_field('vendor_syndrome', u.unpack_one("<I")) + if self.length > 0xB: + self.add_field('memory_array_error_address', u.unpack_one("<Q")) + if self.length > 0xF: + self.add_field('device_error_address', u.unpack_one("<Q")) + if self.length > 0x13: + self.add_field('error_resolution', u.unpack_one("<Q")) + except: + self.decodeFailure = True + print "Error parsing MemoryErrorInfo64Bit" + import traceback + traceback.print_exc() + self.fini() + +class ManagementDevice(SmbiosBaseStructure): + smbios_structure_type = 34 + + def __init__(self, u, sm): + super(ManagementDevice, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('description', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + _type = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'National Semiconductor LM75', + 0x04: 'National Semiconductor LM78', + 0x05: 'National Semiconductor LM79', + 0x06: 'National Semiconductor LM80', + 0x07: 'National Semiconductor LM81', + 0x08: 'Analog Devices ADM9240', + 0x09: 'Dallas Semiconductor DS1780', + 0x0A: 'Maxim 1617', + 0x0B: 'Genesys GL518SM', + 0x0C: 'Winbond W83781D', + 0x0D: 'Holtek HT82H791' + } + self.add_field('device_type', u.unpack_one("B"), unpack.format_table("{}", _type)) + if self.length > 0x6: + self.add_field('address', u.unpack_one("<I")) + if self.length > 0xA: + _address_type = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'I/O Port', + 0x04: 'Memory', + 0x05: 'SM Bus' + } + self.add_field('address_type', u.unpack_one("B"), unpack.format_table("{}", _address_type)) + except: + self.decodeFailure = True + print "Error parsing ManagementDevice" + import traceback + traceback.print_exc() + self.fini() + +class ManagementDeviceComponent(SmbiosBaseStructure): + smbios_structure_type = 35 + + def __init__(self, u, sm): + super(ManagementDeviceComponent, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('description', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('management_device_handle', u.unpack_one("<H")) + if self.length > 0x7: + self.add_field('component_handle', u.unpack_one("<H")) + if self.length > 0x9: + self.add_field('threshold_handle', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing ManagementDeviceComponent" + import traceback + traceback.print_exc() + self.fini() + +class ManagementDeviceThresholdData(SmbiosBaseStructure): + smbios_structure_type = 36 + + def __init__(self, u, sm): + super(ManagementDeviceThresholdData, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('lower_threshold_noncritical', u.unpack_one("<H")) + if self.length > 0x6: + self.add_field('upper_threshold_noncritical', u.unpack_one("<H")) + if self.length > 0x8: + self.add_field('lower_threshold_critical', u.unpack_one("<H")) + if self.length > 0xA: + self.add_field('upper_threshold_critical', u.unpack_one("<H")) + if self.length > 0xC: + self.add_field('lower_threshold_nonrecoverable', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('upper_threshold_nonrecoverable', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing ManagementDeviceThresholdData" + import traceback + traceback.print_exc() + self.fini() + +class MemoryChannel(SmbiosBaseStructure): + smbios_structure_type = 37 + + def __init__(self, u, sm): + super(MemoryChannel, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + _channel_type = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'RamBus', + 0x04: 'SyncLink' + } + self.add_field('channel_type', u.unpack_one("B"), unpack.format_table("{}", _channel_type)) + if self.length > 0x6: + self.add_field('max_channel_load', u.unpack_one("B")) + if self.length > 0x8: + self.add_field('memory_device_count', u.unpack_one("B")) + if self.length > 0xA: + self.add_field('memory_device_load', u.unpack_one("B")) + if self.length > 0xC: + self.add_field('memory_device_handle', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing MemoryChannel" + import traceback + traceback.print_exc() + self.fini() + +class IPMIDeviceInformation(SmbiosBaseStructure): + smbios_structure_type = 38 + + def __init__(self, u, sm): + super(IPMIDeviceInformation, self).__init__(u, sm) + u = self.u + try: + _interface_type = { + 0x00: 'Unknown', + 0x01: 'KCS: Keyboard Controller Style', + 0x02: 'SMIC: Server Management Interface Chip', + 0x03: 'BT: Block Transfer', + xrange(0x04, 0xFF): 'Reserved' + } + self.add_field('interface_type', u.unpack_one("B"), unpack.format_table("{}", _interface_type)) + self.add_field('ipmi_specification_revision', u.unpack_one("B")) + self.add_field('msd_revision', bitfields.getbits(self.ipmi_specification_revision, 7, 4), "ipmi_specification_revision[7:4]={}") + self.add_field('lsd_revision', bitfields.getbits(self.ipmi_specification_revision, 3, 0), "ipmi_specification_revision[3:0]={}") + + self.add_field('i2c_slave_address', u.unpack_one("B")) + self.add_field('nv_storage_device_address', u.unpack_one("B")) + self.add_field('base_address', u.unpack_one("<Q")) + # if lsb is 1, address is in IO space. otherwise, memory-mapped + self.add_field('base_address_modifier_interrupt_info', u.unpack_one("B")) + _reg_spacing = { + 0b00: 'Interface registers are on successive byte boundaries', + 0b01: 'Interface registers are on 32-bit boundaries', + 0b10: 'Interface registers are on 16-byte boundaries', + 0b11: 'Reserved' + } + self.add_field('register_spacing', bitfields.getbits(self.base_address_modifier_interrupt_info, 7, 6), unpack.format_table("base_address_modifier_interrupt_info[7:6]={}", _reg_spacing)) + self.add_field('ls_bit_for_addresses', bitfields.getbits(self.base_address_modifier_interrupt_info, 4), "base_address_modifier_interrupt_info[4]={}") + self.add_field('interrupt_info_specified', bool(bitfields.getbits(self.base_address_modifier_interrupt_info, 3)), "base_address_modifier_interrupt_info[3]={}") + _polarity = { + 0: 'active low', + 1: 'active high' + } + self.add_field('interrupt_polarity', bitfields.getbits(self.base_address_modifier_interrupt_info, 1), unpack.format_table("base_address_modifier_interrupt_info[1]={}", _polarity)) + _interrupt_trigger = { + 0: 'edge', + 1: 'level' + } + self.add_field('interrupt_trigger_mode', bitfields.getbits(self.base_address_modifier_interrupt_info, 0), unpack.format_table("base_address_modifier_interrupt_info[0]={}", _interrupt_trigger)) + self.add_field('interrupt_number', u.unpack_one("B")) + except: + self.decodeFailure = True + print "Error parsing IPMIDeviceInformation" + import traceback + traceback.print_exc() + self.fini() + +class SystemPowerSupply(SmbiosBaseStructure): + smbios_structure_type = 39 + + def __init__(self, u, sm): + super(SystemPowerSupply, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('power_unit_group', u.unpack_one("B")) + if self.length > 0x5: + self.add_field('location', u.unpack_one("B"), self.fmtstr) + if self.length > 0x6: + self.add_field('device_name', u.unpack_one("B"), self.fmtstr) + if self.length > 0x7: + self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr) + if self.length > 0x8: + self.add_field('serial_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0x9: + self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr) + if self.length > 0xA: + self.add_field('model_part_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0xB: + self.add_field('revision_level', u.unpack_one("B"), self.fmtstr) + if self.length > 0xC: + self.add_field('max_power_capacity', u.unpack_one("<H")) + if self.length > 0xE: + self.add_field('power_supply_characteristics', u.unpack_one("<H")) + _dmtf_power_supply_type = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'Linear', + 0b100: 'Switching', + 0b101: 'Battery', + 0b110: 'UPS', + 0b111: 'Converter', + 0b1000: 'Regulator', + xrange(0b1001, 0b1111): 'Reserved' + } + self.add_field('dmtf_power_supply_type', bitfields.getbits(self.power_supply_characteristics, 13, 10), unpack.format_table("power_supply_characteristics[13:10]={}", _dmtf_power_supply_type)) + _status = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'OK', + 0b100: 'Non-critical', + 0b101: 'Critical; power supply has failed and has been taken off-line' + } + self.add_field('status', bitfields.getbits(self.power_supply_characteristics, 9, 7), unpack.format_table("power_supply_characteristics[9:7]={}", _status)) + _dmtf_input_voltage_range_switching = { + 0b001: 'Other', + 0b010: 'Unknown', + 0b011: 'Manual', + 0b100: 'Auto-switch', + 0b101: 'Wide range', + 0b110: 'Not applicable', + xrange(0b0111, 0b1111): 'Reserved' + } + self.add_field('dmtf_input_voltage_range_switching', bitfields.getbits(self.power_supply_characteristics, 6, 3), unpack.format_table("power_supply_characteristics[6:3]={}", _dmtf_input_voltage_range_switching)) + self.add_field('power_supply_unplugged', bool(bitfields.getbits(self.power_supply_characteristics, 2)), "power_supply_characteristics[2]={}") + self.add_field('power_supply_present', bool(bitfields.getbits(self.power_supply_characteristics, 1)), "power_supply_characteristics[1]={}") + self.add_field('power_supply_hot_replaceable', bool(bitfields.getbits(self.power_supply_characteristics, 0)), "power_supply_characteristics[0]={}") + if self.length > 0x10: + self.add_field('input_voltage_probe_handle', u.unpack_one("<H")) + if self.length > 0x12: + self.add_field('cooling_device_handle', u.unpack_one("<H")) + if self.length > 0x14: + self.add_field('input_current_probe_handle', u.unpack_one("<H")) + except: + self.decodeFailure = True + print "Error parsing SystemPowerSupply" + import traceback + traceback.print_exc() + self.fini() + +class AdditionalInformation(SmbiosBaseStructure): + smbios_structure_type = 40 + + def __init__(self, u, sm): + super(AdditionalInformation, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('num_additional_information_entries', u.unpack_one("B")) + if self.length > 0x5: + self.add_field('additional_information_entry_length', u.unpack_one("B")) + self.add_field('referenced_handle', u.unpack_one("<H")) + self.add_field('referenced_offset', u.unpack_one("B")) + self.add_field('string', u.unpack_one("B"), self.fmtstr) + self.add_field('value', u.unpack_rest()) + except: + self.decodeFailure = True + print "Error parsing AdditionalInformation" + import traceback + traceback.print_exc() + self.fini() + +class OnboardDevicesExtendedInformation(SmbiosBaseStructure): + smbios_structure_type = 41 + + def __init__(self, u, sm): + super(OnboardDevicesExtendedInformation, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + self.add_field('reference_designation', u.unpack_one("B"), self.fmtstr) + if self.length > 0x5: + self.add_field('device_type', u.unpack_one("B")) + self.add_field('device_enabled', bool(bitfields.getbits(self.device_type, 7)), "device_type[7]={}") + _device_types = { + 0x01: 'Other', + 0x02: 'Unknown', + 0x03: 'Video', + 0x04: 'SCSI Controller', + 0x05: 'Ethernet', + 0x06: 'Token Ring', + 0x07: 'Sound', + 0x08: 'PATA Controller', + 0x09: 'SATA Controller', + 0x0A: 'SAS Controller' + } + self.add_field('type_of_device', bitfields.getbits(self.device_type, 6, 0), unpack.format_table("device_type[6:0]={}", _device_types)) + if self.length > 0x6: + self.add_field('device_type_instance', u.unpack_one("B")) + if self.length > 0x7: + self.add_field('segment_group_number', u.unpack_one("<H")) + if self.length > 0x9: + self.add_field('bus_number', u.unpack_one("B"), self.fmtstr) + if self.length > 0xA: + self.add_field('device_and_function_number', u.unpack_one("B")) + self.add_field('device_number', bitfields.getbits(self.device_type, 7, 3), "device_and_function_number[7:3]={}") + self.add_field('function_number', bitfields.getbits(self.device_type, 2, 0), "device_and_function_number[2:0]={}") + except: + self.decodeFailure = True + print "Error parsing OnboardDevicesExtendedInformation" + import traceback + traceback.print_exc() + self.fini() + +class ManagementControllerHostInterface(SmbiosBaseStructure): + smbios_structure_type = 42 + + def __init__(self, u, sm): + super(ManagementControllerHostInterface, self).__init__(u, sm) + u = self.u + try: + if self.length > 0x4: + _interface_types = { + 0x00: 'Reserved', + 0x01: 'Reserved', + 0x02: 'KCS: Keyboard Controller Style', + 0x03: '8250 UART Register Compatible', + 0x04: '16450 UART Register Compatible', + 0x05: '16550/16550A UART Register Compatible', + 0x06: '16650/16650A UART Register Compatible', + 0x07: '16750/16750A UART Register Compatible', + 0x08: '16850/16850A UART Register Compatible', + 0xF0: 'OEM' + } + self.add_field('interface_type', u.unpack_one("B"), unpack.format_table("{}", _interface_types)) + if self.length > 0x5: + self.add_field('mc_host_interface_data', u.unpack_rest(), self.fmtstr) + except: + self.decodeFailure = True + print "Error parsing ManagementControllerHostInterface" + import traceback + traceback.print_exc() + self.fini() + +class Inactive(SmbiosBaseStructure): + smbios_structure_type = 126 + + def __init__(self, u, sm): + super(Inactive, self).__init__(u, sm) + self.fini() + +class EndOfTable(SmbiosBaseStructure): + smbios_structure_type = 127 + + def __init__(self, u, sm): + super(EndOfTable, self).__init__(u, sm) + self.fini() + +class SmbiosStructureUnknown(SmbiosBaseStructure): + smbios_structure_type = None + + def __init__(self, u, sm): + super(SmbiosStructureUnknown, self).__init__(u, sm) + self.fini() + +_smbios_structures = [ + BIOSInformation, + SystemInformation, + BaseboardInformation, + SystemEnclosure, + ProcessorInformation, + MemoryControllerInformation, + MemoryModuleInformation, + CacheInformation, + PortConnectorInfo, + SystemSlots, + OnBoardDevicesInformation, + OEMStrings, + SystemConfigOptions, + BIOSLanguageInformation, + GroupAssociations, + SystemEventLog, + PhysicalMemoryArray, + MemoryDevice, + MemoryErrorInfo32Bit, + MemoryArrayMappedAddress, + MemoryDeviceMappedAddress, + BuiltInPointingDevice, + PortableBattery, + SystemReset, + HardwareSecurity, + SystemPowerControls, + VoltageProbe, + CoolingDevice, + TemperatureProbe, + ElectricalCurrentProbe, + OutOfBandRemoteAccess, + BootIntegrityServicesEntryPoint, + SystemBootInformation, + MemoryErrorInfo64Bit, + ManagementDevice, + ManagementDeviceComponent, + ManagementDeviceThresholdData, + MemoryChannel, + IPMIDeviceInformation, + SystemPowerSupply, + AdditionalInformation, + OnboardDevicesExtendedInformation, + ManagementControllerHostInterface, + Inactive, + EndOfTable, + SmbiosStructureUnknown, # Must always come last +] + +def log_smbios_info(): + with redirect.logonly(): + try: + sm = SMBIOS() + print + if sm is None: + print "No SMBIOS structures found" + return + output = {} + known_types = (0, 1) + for sm_struct in sm.structures: + if sm_struct.type in known_types: + output.setdefault(sm_struct.type, []).append(sm_struct) + if len(output) == len(known_types): + break + + print "SMBIOS information:" + for key in sorted(known_types): + for s in output.get(key, ["No structure of type {} found".format(key)]): + print ttypager._wrap("{}: {}".format(key, s)) + except: + print "Error parsing SMBIOS information:" + import traceback + traceback.print_exc() + +def dump_raw(): + try: + sm = SMBIOS() + if sm: + s = "SMBIOS -- Raw bytes and structure decode.\n\n" + + s += str(sm.header) + '\n' + s += bits.dumpmem(sm._header_memory) + '\n' + + s += "Raw bytes for the SMBIOS structures\n" + s += bits.dumpmem(sm._structure_memory) + '\n' + + for sm_struct in sm.structures: + s += str(sm_struct) + '\n' + s += bits.dumpmem(sm_struct.raw_data) + + s += "Strings:\n" + for n in range(1, len(getattr(sm_struct, "strings", [])) + 1): + s += str(sm_struct.fmtstr(n)) + '\n' + s += bits.dumpmem(sm_struct.raw_strings) + '\n' + else: + s = "No SMBIOS structures found" + ttypager.ttypager_wrap(s, indent=False) + except: + print "Error parsing SMBIOS information:" + import traceback + traceback.print_exc() + +def dump(): + try: + sm = SMBIOS() + if sm: + s = str(sm) + else: + s = "No SMBIOS structures found" + ttypager.ttypager_wrap(s, indent=False) + except: + print "Error parsing SMBIOS information:" + import traceback + traceback.print_exc() + +def annex_a_conformance(): + try: + sm = SMBIOS() + + # check: 1. The table anchor string "_SM_" is present in the address range 0xF0000 to 0xFFFFF on a 16-byte bound + + def table_entry_point_verification(): + ''' Verify table entry-point''' + if (sm.header.length < 0x1F): + print "Failure: Table entry-point - The entry-point Length must be at least 0x1F" + if sm.header.checksum != 0: + print "Failure: Table entry-point - The entry-point checksum must evaluate to 0" + if ((sm.header.major_version < 2) and (sm.header.minor_version < 4)): + print "Failure: Table entry-point - SMBIOS version must be at least 2.4" + if (sm.header.intermediate_anchor_string == '_DMI_'): + print "Failure: Table entry-point - The Intermediate Anchor String must be '_DMI_'" + if (sm.header.intermediate_checksum != 0): + print "Failure: Table entry-point - The Intermediate checksum must evaluate to 0" + + #check: 3. The structure-table is traversable and conforms to the entry-point specifications: + + def req_structures(): + '''Checks for required structures and corresponding data''' + types_present = [sm.structures[x].smbios_structure_type for x in range(len(sm.structures))] + required = [0, 1, 4, 7, 9, 16, 17, 19, 31, 32] + for s in required: + if s not in set(types_present): + print "Failure: Type {} required but not found".format(s) + + else: + if s == 0: + if types_present.count(s) > 1: + print "Failure: Type {} - One and only one structure of this type must be present.".format(s) + if sm.structure_type(s).length < 0x18: + print "Failure: Type {} - The structure Length field must be at least 0x18".format(s) + if sm.structure_type(s).version is None: + print "Failure: Type {} - BIOS Version string must be present and non-null.".format(s) + if sm.structure_type(s).release_date is None: + print "Failure: Type {} - BIOS Release Date string must be present, non-null, and include a 4-digit year".format(s) + if bitfields.getbits(sm.structure_type(s).characteristics, 3, 0) != 0 or bitfields.getbits(sm.structure_type(s).characteristics, 31, 4) == 0: + print "Failure: Type {} - BIOS Characteristics: bits 3:0 must all be 0, and at least one of bits 31:4 must be set to 1.".format(s) + elif s == 1: + if types_present.count(s) > 1: + print "Failure: Type {} - One and only one structure of this type must be present.".format(s) + if sm.structure_type(s).length < 0x1B: + print "Failure: Type {} - The structure Length field must be at least 0x1B".format(s) + if sm.structure_type(s).manufacturer == None: + print "Failure: Type {} - Manufacturer string must be present and non-null.".format(s) + if sm.structure_type(s).product_name == None: + print "Failure: Type {} - Product Name string must be present and non-null".format(s) + if sm.structure_type(s).uuid == '00000000 00000000' and sm.structure_type(s).uuid == 'FFFFFFFF FFFFFFFF': + print "Failure: Type {} - UUID field must be neither 00000000 00000000 nor FFFFFFFF FFFFFFFF.".format(s) + if sm.structure_type(s).wakeup_type == 00 and sm.structure_type(s).wakeup_type == 0x02: + print "Failure: Type {} - Wake-up Type field must be neither 00h (Reserved) nor 02h (Unknown).".format(s) + # continue for remaining required types + + # check remaining conformance guidelines + + table_entry_point_verification() + req_structures() + except: + print "Error checking ANNEX A conformance guidelines" + import traceback + traceback.print_exc() diff --git a/tests/pytest/acpi-bits/bits-tests/testacpi.py b/tests/pytest/acpi-bits/bits-tests/testacpi.py new file mode 100644 index 0000000000..9ec452f330 --- /dev/null +++ b/tests/pytest/acpi-bits/bits-tests/testacpi.py @@ -0,0 +1,283 @@ +# Copyright (c) 2015, Intel Corporation +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# * Redistributions of source code must retain the above copyright notice, +# this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# * Neither the name of Intel Corporation nor the names of its contributors +# may be used to endorse or promote products derived from this software +# without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Tests for ACPI""" + +import acpi +import bits +import bits.mwait +import struct +import testutil +import testsuite +import time + +def register_tests(): + testsuite.add_test("ACPI _MAT (Multiple APIC Table Entry) under Processor objects", test_mat, submenu="ACPI Tests") + testsuite.add_test("ACPI _PSS (Pstate) table conformance tests", test_pss, submenu="ACPI Tests") + testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates, submenu="ACPI Tests") + testsuite.add_test("ACPI DSDT (Differentiated System Description Table)", test_dsdt, submenu="ACPI Tests") + testsuite.add_test("ACPI FACP (Fixed ACPI Description Table)", test_facp, submenu="ACPI Tests") + testsuite.add_test("ACPI HPET (High Precision Event Timer Table)", test_hpet, submenu="ACPI Tests") + testsuite.add_test("ACPI MADT (Multiple APIC Description Table)", test_apic, submenu="ACPI Tests") + testsuite.add_test("ACPI MPST (Memory Power State Table)", test_mpst, submenu="ACPI Tests") + testsuite.add_test("ACPI RSDP (Root System Description Pointer Structure)", test_rsdp, submenu="ACPI Tests") + testsuite.add_test("ACPI XSDT (Extended System Description Table)", test_xsdt, submenu="ACPI Tests") + +def test_mat(): + cpupaths = acpi.get_cpupaths() + apic = acpi.parse_apic() + procid_apicid = apic.procid_apicid + uid_x2apicid = apic.uid_x2apicid + for cpupath in cpupaths: + # Find the ProcId defined by the processor object + processor = acpi.evaluate(cpupath) + # Find the UID defined by the processor object's _UID method + uid = acpi.evaluate(cpupath + "._UID") + mat_buffer = acpi.evaluate(cpupath + "._MAT") + if mat_buffer is None: + continue + # Process each _MAT subtable + mat = acpi._MAT(mat_buffer) + for index, subtable in enumerate(mat): + if subtable.subtype == acpi.MADT_TYPE_LOCAL_APIC: + if subtable.flags.bits.enabled: + testsuite.test("{} Processor declaration ProcId = _MAT ProcId".format(cpupath), processor.ProcId == subtable.proc_id) + testsuite.print_detail("{} ProcId ({:#02x}) != _MAT ProcId ({:#02x})".format(cpupath, processor.ProcId, subtable.proc_id)) + testsuite.print_detail("Processor Declaration: {}".format(processor)) + testsuite.print_detail("_MAT entry[{}]: {}".format(index, subtable)) + if testsuite.test("{} with local APIC in _MAT has local APIC in MADT".format(cpupath), processor.ProcId in procid_apicid): + testsuite.test("{} ApicId derived using Processor declaration ProcId = _MAT ApicId".format(cpupath), procid_apicid[processor.ProcId] == subtable.apic_id) + testsuite.print_detail("{} ApicId derived from MADT ({:#02x}) != _MAT ApicId ({:#02x})".format(cpupath, procid_apicid[processor.ProcId], subtable.apic_id)) + testsuite.print_detail("Processor Declaration: {}".format(processor)) + testsuite.print_detail("_MAT entry[{}]: {}".format(index, subtable)) + if subtable.subtype == acpi.MADT_TYPE_LOCAL_X2APIC: + if subtable.flags.bits.enabled: + if testsuite.test("{} with x2Apic in _MAT has _UID".format(cpupath), uid is not None): + testsuite.test("{}._UID = _MAT UID".format(cpupath), uid == subtable.uid) + testsuite.print_detail("{}._UID ({:#x}) != _MAT UID ({:#x})".format(cpupath, uid, subtable.uid)) + testsuite.print_detail("_MAT entry[{}]: {}".format(index, subtable)) + if testsuite.test("{} with _MAT x2Apic has x2Apic in MADT".format(cpupath), subtable.uid in uid_x2apicid): + testsuite.test("{} x2ApicId derived from MADT using UID = _MAT x2ApicId".format(cpupath), uid_x2apicid[subtable.uid] == subtable.x2apicid) + testsuite.print_detail("{} x2ApicId derived from MADT ({:#02x}) != _MAT x2ApicId ({:#02x})".format(cpupath, uid_x2apicid[subtable.uid], subtable.x2apicid)) + testsuite.print_detail("_MAT entry[{}]: {}".format(index, subtable)) + +def test_pss(): + uniques = acpi.parse_cpu_method("_PSS") + # We special-case None here to avoid a double-failure for CPUs without a _PSS + testsuite.test("_PSS must be identical for all CPUs", len(uniques) <= 1 or (len(uniques) == 2 and None in uniques)) + for pss, cpupaths in uniques.iteritems(): + if not testsuite.test("_PSS must exist", pss is not None): + testsuite.print_detail(acpi.factor_commonprefix(cpupaths)) + testsuite.print_detail('No _PSS exists') + continue + + if not testsuite.test("_PSS must not be empty", pss.pstates): + testsuite.print_detail(acpi.factor_commonprefix(cpupaths)) + testsuite.print_detail('_PSS is empty') + continue + + testsuite.print_detail(acpi.factor_commonprefix(cpupaths)) + for index, pstate in enumerate(pss.pstates): + testsuite.print_detail("P[{}]: {}".format(index, pstate)) + + testsuite.test("_PSS must contain at most 16 Pstates", len(pss.pstates) <= 16) + testsuite.test("_PSS must have no duplicate Pstates", len(pss.pstates) == len(set(pss.pstates))) + + frequencies = [p.core_frequency for p in pss.pstates] + testsuite.test("_PSS must list Pstates in descending order of frequency", frequencies == sorted(frequencies, reverse=True)) + + testsuite.test("_PSS must have Pstates with no duplicate frequencies", len(frequencies) == len(set(frequencies))) + + dissipations = [p.power for p in pss.pstates] + testsuite.test("_PSS must list Pstates in descending order of power dissipation", dissipations == sorted(dissipations, reverse=True)) + +def test_pstates(): + """Execute and verify frequency for each Pstate in the _PSS""" + IA32_PERF_CTL = 0x199 + with bits.mwait.use_hint(), bits.preserve_msr(IA32_PERF_CTL): + cpupath_procid = acpi.find_procid() + cpupath_uid = acpi.find_uid() + apic = acpi.parse_apic() + procid_apicid = apic.procid_apicid + uid_x2apicid = apic.uid_x2apicid + def cpupath_apicid(cpupath): + if procid_apicid is not None: + procid = cpupath_procid.get(cpupath, None) + if procid is not None: + apicid = procid_apicid.get(procid, None) + if apicid is not None: + return apicid + if uid_x2apicid is not None: + uid = cpupath_uid.get(cpupath, None) + if uid is not None: + apicid = uid_x2apicid.get(uid, None) + if apicid is not None: + return apicid + return bits.cpus()[0] + + bclk = testutil.adjust_to_nearest(bits.bclk(), 100.0/12) * 1000000 + + uniques = acpi.parse_cpu_method("_PSS") + for pss, cpupaths in uniques.iteritems(): + if not testsuite.test("_PSS must exist", pss is not None): + testsuite.print_detail(acpi.factor_commonprefix(cpupaths)) + testsuite.print_detail('No _PSS exists') + continue + + for n, pstate in enumerate(pss.pstates): + for cpupath in cpupaths: + apicid = cpupath_apicid(cpupath) + if apicid is None: + print 'Failed to find apicid for cpupath {}'.format(cpupath) + continue + bits.wrmsr(apicid, IA32_PERF_CTL, pstate.control) + + # Detecting Turbo frequency requires at least 2 pstates + # since turbo frequency = max non-turbo frequency + 1 + turbo = False + if len(pss.pstates) >= 2: + turbo = (n == 0 and pstate.core_frequency == (pss.pstates[1].core_frequency + 1)) + if turbo: + # Needs to busywait, not sleep + start = time.time() + while (time.time() - start < 2): + pass + + for duration in (0.1, 1.0): + frequency_data = bits.cpu_frequency(duration) + # Abort the test if no cpu frequency is not available + if frequency_data is None: + continue + aperf = frequency_data[1] + aperf = testutil.adjust_to_nearest(aperf, bclk/2) + aperf = int(aperf / 1000000) + if turbo: + if aperf >= pstate.core_frequency: + break + else: + if aperf == pstate.core_frequency: + break + + if turbo: + testsuite.test("P{}: Turbo measured frequency {} >= expected {} MHz".format(n, aperf, pstate.core_frequency), aperf >= pstate.core_frequency) + else: + testsuite.test("P{}: measured frequency {} MHz == expected {} MHz".format(n, aperf, pstate.core_frequency), aperf == pstate.core_frequency) + +def test_psd_thread_scope(): + uniques = acpi.parse_cpu_method("_PSD") + if not testsuite.test("_PSD (P-State Dependency) must exist for each processor", None not in uniques): + testsuite.print_detail(acpi.factor_commonprefix(uniques[None])) + testsuite.print_detail('No _PSD exists') + return + unique_num_dependencies = {} + unique_num_entries = {} + unique_revision = {} + unique_domain = {} + unique_coordination_type = {} + unique_num_processors = {} + for value, cpupaths in uniques.iteritems(): + unique_num_dependencies.setdefault(len(value.dependencies), []).extend(cpupaths) + unique_num_entries.setdefault(value.dependencies[0].num_entries, []).extend(cpupaths) + unique_revision.setdefault(value.dependencies[0].revision, []).extend(cpupaths) + unique_domain.setdefault(value.dependencies[0].domain, []).extend(cpupaths) + unique_coordination_type.setdefault(value.dependencies[0].coordination_type, []).extend(cpupaths) + unique_num_processors.setdefault(value.dependencies[0].num_processors, []).extend(cpupaths) + def detail(d, fmt): + for value, cpupaths in sorted(d.iteritems(), key=(lambda (k,v): v)): + testsuite.print_detail(acpi.factor_commonprefix(cpupaths)) + testsuite.print_detail(fmt.format(value)) + + testsuite.test('Dependency count for each processor must be 1', unique_num_dependencies.keys() == [1]) + detail(unique_num_dependencies, 'Dependency count for each processor = {} (Expected 1)') + testsuite.test('_PSD.num_entries must be 5', unique_num_entries.keys() == [5]) + detail(unique_num_entries, 'num_entries = {} (Expected 5)') + testsuite.test('_PSD.revision must be 0', unique_revision.keys() == [0]) + detail(unique_revision, 'revision = {}') + testsuite.test('_PSD.coordination_type must be 0xFE (HW_ALL)', unique_coordination_type.keys() == [0xfe]) + detail(unique_coordination_type, 'coordination_type = {:#x} (Expected 0xFE HW_ALL)') + testsuite.test('_PSD.domain must be unique (thread-scoped) for each processor', len(unique_domain) == len(acpi.get_cpupaths())) + detail(unique_domain, 'domain = {:#x} (Expected a unique value for each processor)') + testsuite.test('_PSD.num_processors must be 1', unique_num_processors.keys() == [1]) + detail(unique_num_processors, 'num_processors = {} (Expected 1)') + +def test_table_checksum(data): + csum = sum(ord(c) for c in data) % 0x100 + testsuite.test('ACPI table cumulative checksum must equal 0', csum == 0) + testsuite.print_detail("Cumulative checksum = {} (Expected 0)".format(csum)) + +def test_apic(): + data = acpi.get_table("APIC") + if data is None: + return + test_table_checksum(data) + apic = acpi.parse_apic() + +def test_dsdt(): + data = acpi.get_table("DSDT") + if data is None: + return + test_table_checksum(data) + +def test_facp(): + data = acpi.get_table("FACP") + if data is None: + return + test_table_checksum(data) + facp = acpi.parse_facp() + +def test_hpet(): + data = acpi.get_table("HPET") + if data is None: + return + test_table_checksum(data) + hpet = acpi.parse_hpet() + +def test_mpst(): + data = acpi.get_table("MPST") + if data is None: + return + test_table_checksum(data) + mpst = acpi.MPST(data) + +def test_rsdp(): + data = acpi.get_table("RSD PTR ") + if data is None: + return + + # Checksum the first 20 bytes per ACPI 1.0 + csum = sum(ord(c) for c in data[:20]) % 0x100 + testsuite.test('ACPI 1.0 table first 20 bytes cummulative checksum must equal 0', csum == 0) + testsuite.print_detail("Cummulative checksum = {} (Expected 0)".format(csum)) + + test_table_checksum(data) + rsdp = acpi.parse_rsdp() + +def test_xsdt(): + data = acpi.get_table("XSDT") + if data is None: + return + test_table_checksum(data) + xsdt = acpi.parse_xsdt() diff --git a/tests/pytest/acpi-bits/bits-tests/testcpuid.py b/tests/pytest/acpi-bits/bits-tests/testcpuid.py new file mode 100644 index 0000000000..ac55d912e1 --- /dev/null +++ b/tests/pytest/acpi-bits/bits-tests/testcpuid.py @@ -0,0 +1,83 @@ +# Copyright (c) 2012, Intel Corporation +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# * Redistributions of source code must retain the above copyright notice, +# this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# * Neither the name of Intel Corporation nor the names of its contributors +# may be used to endorse or promote products derived from this software +# without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Tests and helpers for CPUID.""" + +import bits +import testsuite +import testutil + +def cpuid_helper(function, index=None, shift=0, mask=~0, eax_mask=~0, ebx_mask=~0, ecx_mask=~0, edx_mask=~0): + if index is None: + index = 0 + indexdesc = "" + else: + indexdesc = " index {0:#x}".format(index) + + def find_mask(m): + if m == ~0: + return mask + return m + masks = map(find_mask, [eax_mask, ebx_mask, ecx_mask, edx_mask]) + + uniques = {} + for cpu in bits.cpus(): + regs = bits.cpuid_result(*[(r >> shift) & m for r, m in zip(bits.cpuid(cpu, function, index), masks)]) + uniques.setdefault(regs, []).append(cpu) + + desc = ["CPUID function {:#x}{}".format(function, indexdesc)] + + if shift != 0: + desc.append("Register values have been shifted by {}".format(shift)) + if mask != ~0 or eax_mask != ~0 or ebx_mask != ~0 or ecx_mask != ~0 or edx_mask != ~0: + desc.append("Register values have been masked:") + shifted_masks = bits.cpuid_result(*[m << shift for m in masks]) + desc.append("Masks: eax={eax:#010x} ebx={ebx:#010x} ecx={ecx:#010x} edx={edx:#010x}".format(**shifted_masks._asdict())) + + if len(uniques) > 1: + regvalues = zip(*uniques.iterkeys()) + common_masks = bits.cpuid_result(*map(testutil.find_common_mask, regvalues)) + common_values = bits.cpuid_result(*[v[0] & m for v, m in zip(regvalues, common_masks)]) + desc.append('Register values are not unique across all logical processors') + desc.append("Common bits: eax={eax:#010x} ebx={ebx:#010x} ecx={ecx:#010x} edx={edx:#010x}".format(**common_values._asdict())) + desc.append("Mask of common bits: {eax:#010x} {ebx:#010x} {ecx:#010x} {edx:#010x}".format(**common_masks._asdict())) + + for regs in sorted(uniques.iterkeys()): + cpus = uniques[regs] + desc.append("Register value: eax={eax:#010x} ebx={ebx:#010x} ecx={ecx:#010x} edx={edx:#010x}".format(**regs._asdict())) + desc.append("On {0} CPUs: {1}".format(len(cpus), testutil.apicid_list(cpus))) + + return uniques, desc + +def test_cpuid_consistency(text, function, index=None, shift=0, mask=~0, eax_mask=~0, ebx_mask=~0, ecx_mask=~0, edx_mask=~0): + uniques, desc = cpuid_helper(function, index, shift, mask, eax_mask, ebx_mask, ecx_mask, edx_mask) + desc[0] += " Consistency Check" + if text: + desc.insert(0, text) + status = testsuite.test(desc[0], len(uniques) == 1) + for line in desc[1:]: + testsuite.print_detail(line) + return status -- 2.25.1 ^ permalink raw reply related [flat|nested] 90+ messages in thread
* Re: [PATCH v2 01/11] acpi/tests/bits: initial commit of test scripts that are run by biosbits 2022-07-10 17:00 ` [PATCH v2 01/11] acpi/tests/bits: initial commit of test scripts that are run by biosbits Ani Sinha @ 2022-07-11 20:38 ` John Snow 2022-07-12 7:17 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: John Snow @ 2022-07-11 20:38 UTC (permalink / raw) To: Ani Sinha Cc: qemu-devel, Thomas Huth, Peter Maydell, Daniel Berrange, Paolo Bonzini, Igor Mammedov, Michael Tsirkin On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > This is initial commit of cpuid, acpi and smbios python test scripts for > biosbits to execute. No change has been made to them from the original code > written by the biosbits author Josh Triplett. They are required to be installed > into the bits iso file and then run from within the virtual machine booted off > with biosbits iso. > > The original location of these tests are here: > https://github.com/biosbits/bits/blob/master/python/testacpi.py > https://github.com/biosbits/bits/blob/master/python/smbios.py > https://github.com/biosbits/bits/blob/master/python/testcpuid.py > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > --- > tests/pytest/acpi-bits/bits-tests/meson.build | 11 + > tests/pytest/acpi-bits/bits-tests/smbios.py | 2430 +++++++++++++++++ > tests/pytest/acpi-bits/bits-tests/testacpi.py | 283 ++ > .../pytest/acpi-bits/bits-tests/testcpuid.py | 83 + > 4 files changed, 2807 insertions(+) > create mode 100644 tests/pytest/acpi-bits/bits-tests/meson.build > create mode 100644 tests/pytest/acpi-bits/bits-tests/smbios.py > create mode 100644 tests/pytest/acpi-bits/bits-tests/testacpi.py > create mode 100644 tests/pytest/acpi-bits/bits-tests/testcpuid.py > > diff --git a/tests/pytest/acpi-bits/bits-tests/meson.build b/tests/pytest/acpi-bits/bits-tests/meson.build > new file mode 100644 > index 0000000000..3056731a53 > --- /dev/null > +++ b/tests/pytest/acpi-bits/bits-tests/meson.build > @@ -0,0 +1,11 @@ > +test_files = ['smbios.py', 'testacpi.py', 'testcpuid.py'] > + > +copytestfiles = custom_target('copy test files', > + input : test_files, > + output : test_files, > + command : ['cp', '@INPUT@', '@OUTDIR@'], > + install : true, > + install_dir : 'bits-tests', > + build_by_default : true) > + > +other_deps += copytestfiles > diff --git a/tests/pytest/acpi-bits/bits-tests/smbios.py b/tests/pytest/acpi-bits/bits-tests/smbios.py > new file mode 100644 > index 0000000000..9667d0542c > --- /dev/null > +++ b/tests/pytest/acpi-bits/bits-tests/smbios.py > @@ -0,0 +1,2430 @@ > +# Copyright (c) 2015, Intel Corporation > +# All rights reserved. > +# > +# Redistribution and use in source and binary forms, with or without > +# modification, are permitted provided that the following conditions are met: > +# > +# * Redistributions of source code must retain the above copyright notice, > +# this list of conditions and the following disclaimer. > +# * Redistributions in binary form must reproduce the above copyright notice, > +# this list of conditions and the following disclaimer in the documentation > +# and/or other materials provided with the distribution. > +# * Neither the name of Intel Corporation nor the names of its contributors > +# may be used to endorse or promote products derived from this software > +# without specific prior written permission. > +# > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND > +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED > +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE > +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR > +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES > +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; > +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON > +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS > +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + > +"""SMBIOS/DMI module.""" > + > +import bits > +import bitfields New deps? > +import ctypes > +import redirect Also a new dep? > +import struct > +import uuid > +import unpack And another? > +import ttypager > +import sys What's the proposed strategy for dependency management for these tests? I know there's some mail I'm backlogged on ... ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 01/11] acpi/tests/bits: initial commit of test scripts that are run by biosbits 2022-07-11 20:38 ` John Snow @ 2022-07-12 7:17 ` Ani Sinha 2022-07-12 15:23 ` John Snow 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-07-12 7:17 UTC (permalink / raw) To: John Snow Cc: Ani Sinha, qemu-devel, Thomas Huth, Peter Maydell, Daniel Berrange, Paolo Bonzini, Igor Mammedov, Michael Tsirkin On Mon, 11 Jul 2022, John Snow wrote: > On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > > > This is initial commit of cpuid, acpi and smbios python test scripts for > > biosbits to execute. No change has been made to them from the original code > > written by the biosbits author Josh Triplett. They are required to be installed > > into the bits iso file and then run from within the virtual machine booted off > > with biosbits iso. > > > > The original location of these tests are here: > > https://github.com/biosbits/bits/blob/master/python/testacpi.py > > https://github.com/biosbits/bits/blob/master/python/smbios.py > > https://github.com/biosbits/bits/blob/master/python/testcpuid.py > > > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > > --- > > tests/pytest/acpi-bits/bits-tests/meson.build | 11 + > > tests/pytest/acpi-bits/bits-tests/smbios.py | 2430 +++++++++++++++++ > > tests/pytest/acpi-bits/bits-tests/testacpi.py | 283 ++ > > .../pytest/acpi-bits/bits-tests/testcpuid.py | 83 + > > 4 files changed, 2807 insertions(+) > > create mode 100644 tests/pytest/acpi-bits/bits-tests/meson.build > > create mode 100644 tests/pytest/acpi-bits/bits-tests/smbios.py > > create mode 100644 tests/pytest/acpi-bits/bits-tests/testacpi.py > > create mode 100644 tests/pytest/acpi-bits/bits-tests/testcpuid.py > > > > diff --git a/tests/pytest/acpi-bits/bits-tests/meson.build b/tests/pytest/acpi-bits/bits-tests/meson.build > > new file mode 100644 > > index 0000000000..3056731a53 > > --- /dev/null > > +++ b/tests/pytest/acpi-bits/bits-tests/meson.build > > @@ -0,0 +1,11 @@ > > +test_files = ['smbios.py', 'testacpi.py', 'testcpuid.py'] > > + > > +copytestfiles = custom_target('copy test files', > > + input : test_files, > > + output : test_files, > > + command : ['cp', '@INPUT@', '@OUTDIR@'], > > + install : true, > > + install_dir : 'bits-tests', > > + build_by_default : true) > > + > > +other_deps += copytestfiles > > diff --git a/tests/pytest/acpi-bits/bits-tests/smbios.py b/tests/pytest/acpi-bits/bits-tests/smbios.py > > new file mode 100644 > > index 0000000000..9667d0542c > > --- /dev/null > > +++ b/tests/pytest/acpi-bits/bits-tests/smbios.py > > @@ -0,0 +1,2430 @@ > > +# Copyright (c) 2015, Intel Corporation > > +# All rights reserved. > > +# > > +# Redistribution and use in source and binary forms, with or without > > +# modification, are permitted provided that the following conditions are met: > > +# > > +# * Redistributions of source code must retain the above copyright notice, > > +# this list of conditions and the following disclaimer. > > +# * Redistributions in binary form must reproduce the above copyright notice, > > +# this list of conditions and the following disclaimer in the documentation > > +# and/or other materials provided with the distribution. > > +# * Neither the name of Intel Corporation nor the names of its contributors > > +# may be used to endorse or promote products derived from this software > > +# without specific prior written permission. > > +# > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND > > +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED > > +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE > > +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR > > +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES > > +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; > > +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON > > +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS > > +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > > + > > +"""SMBIOS/DMI module.""" > > + > > +import bits > > +import bitfields > > New deps? > > > +import ctypes > > +import redirect > > Also a new dep? > > > +import struct > > +import uuid > > +import unpack > > And another? > > > +import ttypager > > +import sys > > What's the proposed strategy for dependency management for these > tests? I know there's some mail I'm backlogged on ... > For some reason I did not need to add those libraries as dependencies. If needed we can add them in requirements.txt for pyenv. ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 01/11] acpi/tests/bits: initial commit of test scripts that are run by biosbits 2022-07-12 7:17 ` Ani Sinha @ 2022-07-12 15:23 ` John Snow 2022-07-12 15:38 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: John Snow @ 2022-07-12 15:23 UTC (permalink / raw) To: Ani Sinha Cc: qemu-devel, Thomas Huth, Peter Maydell, Daniel Berrange, Paolo Bonzini, Igor Mammedov, Michael Tsirkin [-- Attachment #1: Type: text/plain, Size: 5309 bytes --] On Tue, Jul 12, 2022, 3:17 AM Ani Sinha <ani@anisinha.ca> wrote: > > > On Mon, 11 Jul 2022, John Snow wrote: > > > On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > This is initial commit of cpuid, acpi and smbios python test scripts > for > > > biosbits to execute. No change has been made to them from the original > code > > > written by the biosbits author Josh Triplett. They are required to be > installed > > > into the bits iso file and then run from within the virtual machine > booted off > > > with biosbits iso. > > > > > > The original location of these tests are here: > > > https://github.com/biosbits/bits/blob/master/python/testacpi.py > > > https://github.com/biosbits/bits/blob/master/python/smbios.py > > > https://github.com/biosbits/bits/blob/master/python/testcpuid.py > > > > > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > > > --- > > > tests/pytest/acpi-bits/bits-tests/meson.build | 11 + > > > tests/pytest/acpi-bits/bits-tests/smbios.py | 2430 +++++++++++++++++ > > > tests/pytest/acpi-bits/bits-tests/testacpi.py | 283 ++ > > > .../pytest/acpi-bits/bits-tests/testcpuid.py | 83 + > > > 4 files changed, 2807 insertions(+) > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/meson.build > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/smbios.py > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/testacpi.py > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/testcpuid.py > > > > > > diff --git a/tests/pytest/acpi-bits/bits-tests/meson.build > b/tests/pytest/acpi-bits/bits-tests/meson.build > > > new file mode 100644 > > > index 0000000000..3056731a53 > > > --- /dev/null > > > +++ b/tests/pytest/acpi-bits/bits-tests/meson.build > > > @@ -0,0 +1,11 @@ > > > +test_files = ['smbios.py', 'testacpi.py', 'testcpuid.py'] > > > + > > > +copytestfiles = custom_target('copy test files', > > > + input : test_files, > > > + output : test_files, > > > + command : ['cp', '@INPUT@', '@OUTDIR@'], > > > + install : true, > > > + install_dir : 'bits-tests', > > > + build_by_default : true) > > > + > > > +other_deps += copytestfiles > > > diff --git a/tests/pytest/acpi-bits/bits-tests/smbios.py > b/tests/pytest/acpi-bits/bits-tests/smbios.py > > > new file mode 100644 > > > index 0000000000..9667d0542c > > > --- /dev/null > > > +++ b/tests/pytest/acpi-bits/bits-tests/smbios.py > > > @@ -0,0 +1,2430 @@ > > > +# Copyright (c) 2015, Intel Corporation > > > +# All rights reserved. > > > +# > > > +# Redistribution and use in source and binary forms, with or without > > > +# modification, are permitted provided that the following conditions > are met: > > > +# > > > +# * Redistributions of source code must retain the above > copyright notice, > > > +# this list of conditions and the following disclaimer. > > > +# * Redistributions in binary form must reproduce the above > copyright notice, > > > +# this list of conditions and the following disclaimer in the > documentation > > > +# and/or other materials provided with the distribution. > > > +# * Neither the name of Intel Corporation nor the names of its > contributors > > > +# may be used to endorse or promote products derived from this > software > > > +# without specific prior written permission. > > > +# > > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > "AS IS" AND > > > +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, > THE IMPLIED > > > +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > ARE > > > +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS > BE LIABLE FOR > > > +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR > CONSEQUENTIAL DAMAGES > > > +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR > SERVICES; > > > +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER > CAUSED AND ON > > > +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR > TORT > > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE > USE OF THIS > > > +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > > > + > > > +"""SMBIOS/DMI module.""" > > > + > > > +import bits > > > +import bitfields > > > > New deps? > > > > > +import ctypes > > > +import redirect > > > > Also a new dep? > > > > > +import struct > > > +import uuid > > > +import unpack > > > > And another? > > > > > +import ttypager > > > +import sys > > > > What's the proposed strategy for dependency management for these > > tests? I know there's some mail I'm backlogged on ... > > > > For some reason I did not need to add those libraries as dependencies. If > needed we can add them in requirements.txt for pyenv. > Probably pulled in as deps to what you do install; pinning them may be helpful for repeatability reasons and to avoid playing whackamole with new pypi releases during the rc testing phase - speaking from experience with the other python test packages in the tree. That said, explicitly enumerating all of the deps is also a costly maintenance activity, so ... YMMV. It depends on how fastidiously the main requirements you state manage their own sub deps. Some projects do it well, some don't. Just something to consider. --js > [-- Attachment #2: Type: text/html, Size: 7491 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 01/11] acpi/tests/bits: initial commit of test scripts that are run by biosbits 2022-07-12 15:23 ` John Snow @ 2022-07-12 15:38 ` Ani Sinha 2022-07-12 15:42 ` John Snow 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-07-12 15:38 UTC (permalink / raw) To: John Snow Cc: Ani Sinha, qemu-devel, Thomas Huth, Peter Maydell, Daniel Berrange, Paolo Bonzini, Igor Mammedov, Michael Tsirkin On Tue, 12 Jul 2022, John Snow wrote: > On Tue, Jul 12, 2022, 3:17 AM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > On Mon, 11 Jul 2022, John Snow wrote: > > > > > On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > This is initial commit of cpuid, acpi and smbios python test scripts > > for > > > > biosbits to execute. No change has been made to them from the original > > code > > > > written by the biosbits author Josh Triplett. They are required to be > > installed > > > > into the bits iso file and then run from within the virtual machine > > booted off > > > > with biosbits iso. > > > > > > > > The original location of these tests are here: > > > > https://github.com/biosbits/bits/blob/master/python/testacpi.py > > > > https://github.com/biosbits/bits/blob/master/python/smbios.py > > > > https://github.com/biosbits/bits/blob/master/python/testcpuid.py > > > > > > > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > > > > --- > > > > tests/pytest/acpi-bits/bits-tests/meson.build | 11 + > > > > tests/pytest/acpi-bits/bits-tests/smbios.py | 2430 +++++++++++++++++ > > > > tests/pytest/acpi-bits/bits-tests/testacpi.py | 283 ++ > > > > .../pytest/acpi-bits/bits-tests/testcpuid.py | 83 + > > > > 4 files changed, 2807 insertions(+) > > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/meson.build > > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/smbios.py > > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/testacpi.py > > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/testcpuid.py > > > > > > > > diff --git a/tests/pytest/acpi-bits/bits-tests/meson.build > > b/tests/pytest/acpi-bits/bits-tests/meson.build > > > > new file mode 100644 > > > > index 0000000000..3056731a53 > > > > --- /dev/null > > > > +++ b/tests/pytest/acpi-bits/bits-tests/meson.build > > > > @@ -0,0 +1,11 @@ > > > > +test_files = ['smbios.py', 'testacpi.py', 'testcpuid.py'] > > > > + > > > > +copytestfiles = custom_target('copy test files', > > > > + input : test_files, > > > > + output : test_files, > > > > + command : ['cp', '@INPUT@', '@OUTDIR@'], > > > > + install : true, > > > > + install_dir : 'bits-tests', > > > > + build_by_default : true) > > > > + > > > > +other_deps += copytestfiles > > > > diff --git a/tests/pytest/acpi-bits/bits-tests/smbios.py > > b/tests/pytest/acpi-bits/bits-tests/smbios.py > > > > new file mode 100644 > > > > index 0000000000..9667d0542c > > > > --- /dev/null > > > > +++ b/tests/pytest/acpi-bits/bits-tests/smbios.py > > > > @@ -0,0 +1,2430 @@ > > > > +# Copyright (c) 2015, Intel Corporation > > > > +# All rights reserved. > > > > +# > > > > +# Redistribution and use in source and binary forms, with or without > > > > +# modification, are permitted provided that the following conditions > > are met: > > > > +# > > > > +# * Redistributions of source code must retain the above > > copyright notice, > > > > +# this list of conditions and the following disclaimer. > > > > +# * Redistributions in binary form must reproduce the above > > copyright notice, > > > > +# this list of conditions and the following disclaimer in the > > documentation > > > > +# and/or other materials provided with the distribution. > > > > +# * Neither the name of Intel Corporation nor the names of its > > contributors > > > > +# may be used to endorse or promote products derived from this > > software > > > > +# without specific prior written permission. > > > > +# > > > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > > "AS IS" AND > > > > +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, > > THE IMPLIED > > > > +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > > ARE > > > > +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS > > BE LIABLE FOR > > > > +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR > > CONSEQUENTIAL DAMAGES > > > > +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR > > SERVICES; > > > > +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER > > CAUSED AND ON > > > > +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR > > TORT > > > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE > > USE OF THIS > > > > +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > > > > + > > > > +"""SMBIOS/DMI module.""" > > > > + > > > > +import bits > > > > +import bitfields > > > > > > New deps? > > > > > > > +import ctypes > > > > +import redirect > > > > > > Also a new dep? > > > > > > > +import struct > > > > +import uuid > > > > +import unpack > > > > > > And another? > > > > > > > +import ttypager > > > > +import sys > > > > > > What's the proposed strategy for dependency management for these > > > tests? I know there's some mail I'm backlogged on ... > > > > > > > For some reason I did not need to add those libraries as dependencies. If > > needed we can add them in requirements.txt for pyenv. > > > > Probably pulled in as deps to what you do install; pinning them may be > helpful for repeatability reasons and to avoid playing whackamole with new > pypi releases during the rc testing phase - speaking from experience with > the other python test packages in the tree. Actually wait, I completely forgot something. These test scrips are run *inside* bios-bits VM, That is, we need not worry about ensuring the availability of these libraries and the deps they introduce ourselves. Bios bits takes care of it. It is a software written and maintained outside of QEMU. The only python script whose deps we need to worry about is the one I wrote myself - acpi-bits-test.py . > > That said, explicitly enumerating all of the deps is also a costly > maintenance activity, so ... YMMV. > > It depends on how fastidiously the main requirements you state manage their > own sub deps. Some projects do it well, some don't. > > Just something to consider. > > --js > > > > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 01/11] acpi/tests/bits: initial commit of test scripts that are run by biosbits 2022-07-12 15:38 ` Ani Sinha @ 2022-07-12 15:42 ` John Snow 2022-07-12 15:46 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: John Snow @ 2022-07-12 15:42 UTC (permalink / raw) To: Ani Sinha Cc: qemu-devel, Thomas Huth, Peter Maydell, Daniel Berrange, Paolo Bonzini, Igor Mammedov, Michael Tsirkin [-- Attachment #1: Type: text/plain, Size: 6533 bytes --] On Tue, Jul 12, 2022, 11:38 AM Ani Sinha <ani@anisinha.ca> wrote: > > > On Tue, 12 Jul 2022, John Snow wrote: > > > On Tue, Jul 12, 2022, 3:17 AM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > > > > On Mon, 11 Jul 2022, John Snow wrote: > > > > > > > On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > > > This is initial commit of cpuid, acpi and smbios python test > scripts > > > for > > > > > biosbits to execute. No change has been made to them from the > original > > > code > > > > > written by the biosbits author Josh Triplett. They are required to > be > > > installed > > > > > into the bits iso file and then run from within the virtual machine > > > booted off > > > > > with biosbits iso. > > > > > > > > > > The original location of these tests are here: > > > > > https://github.com/biosbits/bits/blob/master/python/testacpi.py > > > > > https://github.com/biosbits/bits/blob/master/python/smbios.py > > > > > https://github.com/biosbits/bits/blob/master/python/testcpuid.py > > > > > > > > > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > > > > > --- > > > > > tests/pytest/acpi-bits/bits-tests/meson.build | 11 + > > > > > tests/pytest/acpi-bits/bits-tests/smbios.py | 2430 > +++++++++++++++++ > > > > > tests/pytest/acpi-bits/bits-tests/testacpi.py | 283 ++ > > > > > .../pytest/acpi-bits/bits-tests/testcpuid.py | 83 + > > > > > 4 files changed, 2807 insertions(+) > > > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/meson.build > > > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/smbios.py > > > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/testacpi.py > > > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/testcpuid.py > > > > > > > > > > diff --git a/tests/pytest/acpi-bits/bits-tests/meson.build > > > b/tests/pytest/acpi-bits/bits-tests/meson.build > > > > > new file mode 100644 > > > > > index 0000000000..3056731a53 > > > > > --- /dev/null > > > > > +++ b/tests/pytest/acpi-bits/bits-tests/meson.build > > > > > @@ -0,0 +1,11 @@ > > > > > +test_files = ['smbios.py', 'testacpi.py', 'testcpuid.py'] > > > > > + > > > > > +copytestfiles = custom_target('copy test files', > > > > > + input : test_files, > > > > > + output : test_files, > > > > > + command : ['cp', '@INPUT@', '@OUTDIR@'], > > > > > + install : true, > > > > > + install_dir : 'bits-tests', > > > > > + build_by_default : true) > > > > > + > > > > > +other_deps += copytestfiles > > > > > diff --git a/tests/pytest/acpi-bits/bits-tests/smbios.py > > > b/tests/pytest/acpi-bits/bits-tests/smbios.py > > > > > new file mode 100644 > > > > > index 0000000000..9667d0542c > > > > > --- /dev/null > > > > > +++ b/tests/pytest/acpi-bits/bits-tests/smbios.py > > > > > @@ -0,0 +1,2430 @@ > > > > > +# Copyright (c) 2015, Intel Corporation > > > > > +# All rights reserved. > > > > > +# > > > > > +# Redistribution and use in source and binary forms, with or > without > > > > > +# modification, are permitted provided that the following > conditions > > > are met: > > > > > +# > > > > > +# * Redistributions of source code must retain the above > > > copyright notice, > > > > > +# this list of conditions and the following disclaimer. > > > > > +# * Redistributions in binary form must reproduce the above > > > copyright notice, > > > > > +# this list of conditions and the following disclaimer in > the > > > documentation > > > > > +# and/or other materials provided with the distribution. > > > > > +# * Neither the name of Intel Corporation nor the names of its > > > contributors > > > > > +# may be used to endorse or promote products derived from > this > > > software > > > > > +# without specific prior written permission. > > > > > +# > > > > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > CONTRIBUTORS > > > "AS IS" AND > > > > > +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED > TO, > > > THE IMPLIED > > > > > +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR > PURPOSE > > > ARE > > > > > +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR > CONTRIBUTORS > > > BE LIABLE FOR > > > > > +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR > > > CONSEQUENTIAL DAMAGES > > > > > +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > OR > > > SERVICES; > > > > > +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER > > > CAUSED AND ON > > > > > +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, > OR > > > TORT > > > > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF > THE > > > USE OF THIS > > > > > +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > > > > > + > > > > > +"""SMBIOS/DMI module.""" > > > > > + > > > > > +import bits > > > > > +import bitfields > > > > > > > > New deps? > > > > > > > > > +import ctypes > > > > > +import redirect > > > > > > > > Also a new dep? > > > > > > > > > +import struct > > > > > +import uuid > > > > > +import unpack > > > > > > > > And another? > > > > > > > > > +import ttypager > > > > > +import sys > > > > > > > > What's the proposed strategy for dependency management for these > > > > tests? I know there's some mail I'm backlogged on ... > > > > > > > > > > For some reason I did not need to add those libraries as dependencies. > If > > > needed we can add them in requirements.txt for pyenv. > > > > > > > Probably pulled in as deps to what you do install; pinning them may be > > helpful for repeatability reasons and to avoid playing whackamole with > new > > pypi releases during the rc testing phase - speaking from experience with > > the other python test packages in the tree. > > > Actually wait, I completely forgot something. These test scrips are run > *inside* bios-bits VM, That is, we need not worry about ensuring the > availability of these libraries and the deps they introduce ourselves. > Bios bits takes care of it. It is a software written and maintained > outside of QEMU. > > The only python script whose deps we need to worry about is the one I > wrote myself - acpi-bits-test.py . > Aha. Could I please ask for a comment in each .py file that isn't designed to run directly in the host v/environment? Just a small nod to the fact that it runs in a special VM so I can exclude it from other regular maintenance operations / considerations for dependency management in the host testing env. # Designed to run only in the biosbits VM. would be more than sufficient. --js [-- Attachment #2: Type: text/html, Size: 9498 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 01/11] acpi/tests/bits: initial commit of test scripts that are run by biosbits 2022-07-12 15:42 ` John Snow @ 2022-07-12 15:46 ` Ani Sinha 0 siblings, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-07-12 15:46 UTC (permalink / raw) To: John Snow Cc: Ani Sinha, qemu-devel, Thomas Huth, Peter Maydell, Daniel Berrange, Paolo Bonzini, Igor Mammedov, Michael Tsirkin On Tue, 12 Jul 2022, John Snow wrote: > On Tue, Jul 12, 2022, 11:38 AM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > On Tue, 12 Jul 2022, John Snow wrote: > > > > > On Tue, Jul 12, 2022, 3:17 AM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > > > > > > > > On Mon, 11 Jul 2022, John Snow wrote: > > > > > > > > > On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > > > > > This is initial commit of cpuid, acpi and smbios python test > > scripts > > > > for > > > > > > biosbits to execute. No change has been made to them from the > > original > > > > code > > > > > > written by the biosbits author Josh Triplett. They are required to > > be > > > > installed > > > > > > into the bits iso file and then run from within the virtual machine > > > > booted off > > > > > > with biosbits iso. > > > > > > > > > > > > The original location of these tests are here: > > > > > > https://github.com/biosbits/bits/blob/master/python/testacpi.py > > > > > > https://github.com/biosbits/bits/blob/master/python/smbios.py > > > > > > https://github.com/biosbits/bits/blob/master/python/testcpuid.py > > > > > > > > > > > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > > > > > > --- > > > > > > tests/pytest/acpi-bits/bits-tests/meson.build | 11 + > > > > > > tests/pytest/acpi-bits/bits-tests/smbios.py | 2430 > > +++++++++++++++++ > > > > > > tests/pytest/acpi-bits/bits-tests/testacpi.py | 283 ++ > > > > > > .../pytest/acpi-bits/bits-tests/testcpuid.py | 83 + > > > > > > 4 files changed, 2807 insertions(+) > > > > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/meson.build > > > > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/smbios.py > > > > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/testacpi.py > > > > > > create mode 100644 tests/pytest/acpi-bits/bits-tests/testcpuid.py > > > > > > > > > > > > diff --git a/tests/pytest/acpi-bits/bits-tests/meson.build > > > > b/tests/pytest/acpi-bits/bits-tests/meson.build > > > > > > new file mode 100644 > > > > > > index 0000000000..3056731a53 > > > > > > --- /dev/null > > > > > > +++ b/tests/pytest/acpi-bits/bits-tests/meson.build > > > > > > @@ -0,0 +1,11 @@ > > > > > > +test_files = ['smbios.py', 'testacpi.py', 'testcpuid.py'] > > > > > > + > > > > > > +copytestfiles = custom_target('copy test files', > > > > > > + input : test_files, > > > > > > + output : test_files, > > > > > > + command : ['cp', '@INPUT@', '@OUTDIR@'], > > > > > > + install : true, > > > > > > + install_dir : 'bits-tests', > > > > > > + build_by_default : true) > > > > > > + > > > > > > +other_deps += copytestfiles > > > > > > diff --git a/tests/pytest/acpi-bits/bits-tests/smbios.py > > > > b/tests/pytest/acpi-bits/bits-tests/smbios.py > > > > > > new file mode 100644 > > > > > > index 0000000000..9667d0542c > > > > > > --- /dev/null > > > > > > +++ b/tests/pytest/acpi-bits/bits-tests/smbios.py > > > > > > @@ -0,0 +1,2430 @@ > > > > > > +# Copyright (c) 2015, Intel Corporation > > > > > > +# All rights reserved. > > > > > > +# > > > > > > +# Redistribution and use in source and binary forms, with or > > without > > > > > > +# modification, are permitted provided that the following > > conditions > > > > are met: > > > > > > +# > > > > > > +# * Redistributions of source code must retain the above > > > > copyright notice, > > > > > > +# this list of conditions and the following disclaimer. > > > > > > +# * Redistributions in binary form must reproduce the above > > > > copyright notice, > > > > > > +# this list of conditions and the following disclaimer in > > the > > > > documentation > > > > > > +# and/or other materials provided with the distribution. > > > > > > +# * Neither the name of Intel Corporation nor the names of its > > > > contributors > > > > > > +# may be used to endorse or promote products derived from > > this > > > > software > > > > > > +# without specific prior written permission. > > > > > > +# > > > > > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > > CONTRIBUTORS > > > > "AS IS" AND > > > > > > +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED > > TO, > > > > THE IMPLIED > > > > > > +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR > > PURPOSE > > > > ARE > > > > > > +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR > > CONTRIBUTORS > > > > BE LIABLE FOR > > > > > > +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR > > > > CONSEQUENTIAL DAMAGES > > > > > > +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > > OR > > > > SERVICES; > > > > > > +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER > > > > CAUSED AND ON > > > > > > +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, > > OR > > > > TORT > > > > > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF > > THE > > > > USE OF THIS > > > > > > +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > > > > > > + > > > > > > +"""SMBIOS/DMI module.""" > > > > > > + > > > > > > +import bits > > > > > > +import bitfields > > > > > > > > > > New deps? > > > > > > > > > > > +import ctypes > > > > > > +import redirect > > > > > > > > > > Also a new dep? > > > > > > > > > > > +import struct > > > > > > +import uuid > > > > > > +import unpack > > > > > > > > > > And another? > > > > > > > > > > > +import ttypager > > > > > > +import sys > > > > > > > > > > What's the proposed strategy for dependency management for these > > > > > tests? I know there's some mail I'm backlogged on ... > > > > > > > > > > > > > For some reason I did not need to add those libraries as dependencies. > > If > > > > needed we can add them in requirements.txt for pyenv. > > > > > > > > > > Probably pulled in as deps to what you do install; pinning them may be > > > helpful for repeatability reasons and to avoid playing whackamole with > > new > > > pypi releases during the rc testing phase - speaking from experience with > > > the other python test packages in the tree. > > > > > > Actually wait, I completely forgot something. These test scrips are run > > *inside* bios-bits VM, That is, we need not worry about ensuring the > > availability of these libraries and the deps they introduce ourselves. > > Bios bits takes care of it. It is a software written and maintained > > outside of QEMU. > > > > The only python script whose deps we need to worry about is the one I > > wrote myself - acpi-bits-test.py . > > > > Aha. Could I please ask for a comment in each .py file that isn't designed > to run directly in the host v/environment? Just a small nod to the fact > that it runs in a special VM so I can exclude it from other regular > maintenance operations / considerations for dependency management in the > host testing env. > > # Designed to run only in the biosbits VM. > > would be more than sufficient. Perfect. Will do in the next iteration. ^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v2 02/11] acpi/tests/bits: add SPDX license identifiers for bios bits tests 2022-07-10 17:00 [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Ani Sinha 2022-07-10 17:00 ` [PATCH v2 01/11] acpi/tests/bits: initial commit of test scripts that are run by biosbits Ani Sinha @ 2022-07-10 17:00 ` Ani Sinha 2022-07-14 13:56 ` Daniel P. Berrangé 2022-07-10 17:00 ` [PATCH v2 03/11] acpi/tests/bits: disable acpi PSS tests that are failing in biosbits Ani Sinha ` (9 subsequent siblings) 11 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-07-10 17:00 UTC (permalink / raw) To: qemu-devel Cc: thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo, mst, Ani Sinha Added the SPDX license identifiers based on the following output from the licensee tool for bios bits: $ licensee detect bits/ License: NOASSERTION Matched files: COPYING COPYING: Content hash: 7a1fdfa894728ea69588977442c92073aad69e50 License: NOASSERTION Closest non-matching licenses: BSD-3-Clause-Clear similarity: 85.82% BSD-4-Clause similarity: 83.69% BSD-3-Clause similarity: 77.27% Signed-off-by: Ani Sinha <ani@anisinha.ca> --- tests/pytest/acpi-bits/bits-tests/smbios.py | 2 ++ tests/pytest/acpi-bits/bits-tests/testacpi.py | 2 ++ tests/pytest/acpi-bits/bits-tests/testcpuid.py | 2 ++ 3 files changed, 6 insertions(+) diff --git a/tests/pytest/acpi-bits/bits-tests/smbios.py b/tests/pytest/acpi-bits/bits-tests/smbios.py index 9667d0542c..05ee8661c2 100644 --- a/tests/pytest/acpi-bits/bits-tests/smbios.py +++ b/tests/pytest/acpi-bits/bits-tests/smbios.py @@ -1,6 +1,8 @@ # Copyright (c) 2015, Intel Corporation # All rights reserved. # +# SPDX-License-Identifier: BSD-3-Clause-Clear +# # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are met: # diff --git a/tests/pytest/acpi-bits/bits-tests/testacpi.py b/tests/pytest/acpi-bits/bits-tests/testacpi.py index 9ec452f330..8e6bb3bbc6 100644 --- a/tests/pytest/acpi-bits/bits-tests/testacpi.py +++ b/tests/pytest/acpi-bits/bits-tests/testacpi.py @@ -1,6 +1,8 @@ # Copyright (c) 2015, Intel Corporation # All rights reserved. # +# SPDX-License-Identifier: BSD-3-Clause-Clear +# # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are met: # diff --git a/tests/pytest/acpi-bits/bits-tests/testcpuid.py b/tests/pytest/acpi-bits/bits-tests/testcpuid.py index ac55d912e1..94cc149556 100644 --- a/tests/pytest/acpi-bits/bits-tests/testcpuid.py +++ b/tests/pytest/acpi-bits/bits-tests/testcpuid.py @@ -1,6 +1,8 @@ # Copyright (c) 2012, Intel Corporation # All rights reserved. # +# SPDX-License-Identifier: BSD-3-Clause-Clear +# # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are met: # -- 2.25.1 ^ permalink raw reply related [flat|nested] 90+ messages in thread
* Re: [PATCH v2 02/11] acpi/tests/bits: add SPDX license identifiers for bios bits tests 2022-07-10 17:00 ` [PATCH v2 02/11] acpi/tests/bits: add SPDX license identifiers for bios bits tests Ani Sinha @ 2022-07-14 13:56 ` Daniel P. Berrangé 2022-07-14 14:12 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: Daniel P. Berrangé @ 2022-07-14 13:56 UTC (permalink / raw) To: Ani Sinha Cc: qemu-devel, thuth, peter.maydell, jsnow, pbonzini, imammedo, mst On Sun, Jul 10, 2022 at 10:30:05PM +0530, Ani Sinha wrote: > Added the SPDX license identifiers based on the following output from the > licensee tool for bios bits: > > $ licensee detect bits/ > License: NOASSERTION > Matched files: COPYING > COPYING: > Content hash: 7a1fdfa894728ea69588977442c92073aad69e50 > License: NOASSERTION > Closest non-matching licenses: > BSD-3-Clause-Clear similarity: 85.82% > BSD-4-Clause similarity: 83.69% > BSD-3-Clause similarity: 77.27% This report looks pretty bogus to me. smbios.py license header pretty clearly matches BSD-3-Clause with 99% similarity. https://spdx.org/licenses/BSD-3-Clause.html and is pretty different from https://spdx.org/licenses/BSD-3-Clause-Clear.html which adds a statement about patent rights which doens't exist. If we're going to add SPDX tags to existing files it needs to be done by a human, not an automated tool. If not possible, we should leave off SPDX tags, because adding incorrect SPDX tags creates more harm With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 02/11] acpi/tests/bits: add SPDX license identifiers for bios bits tests 2022-07-14 13:56 ` Daniel P. Berrangé @ 2022-07-14 14:12 ` Ani Sinha 0 siblings, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-07-14 14:12 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Ani Sinha, qemu-devel, thuth, peter.maydell, jsnow, pbonzini, imammedo, mst [-- Attachment #1: Type: text/plain, Size: 1576 bytes --] On Thu, 14 Jul 2022, Daniel P. Berrangé wrote: > On Sun, Jul 10, 2022 at 10:30:05PM +0530, Ani Sinha wrote: > > Added the SPDX license identifiers based on the following output from the > > licensee tool for bios bits: > > > > $ licensee detect bits/ > > License: NOASSERTION > > Matched files: COPYING > > COPYING: > > Content hash: 7a1fdfa894728ea69588977442c92073aad69e50 > > License: NOASSERTION > > Closest non-matching licenses: > > BSD-3-Clause-Clear similarity: 85.82% > > BSD-4-Clause similarity: 83.69% > > BSD-3-Clause similarity: 77.27% > > This report looks pretty bogus to me. > > smbios.py license header pretty clearly matches BSD-3-Clause with > 99% similarity. OK in the next iteration I will fix this. This tool likely just looked at the COPYING file in bits and not the individual python files. > > https://spdx.org/licenses/BSD-3-Clause.html > > and is pretty different from > > https://spdx.org/licenses/BSD-3-Clause-Clear.html > > which adds a statement about patent rights which doens't exist. > > > If we're going to add SPDX tags to existing files it needs to be > done by a human, not an automated tool. If not possible, we should > leave off SPDX tags, because adding incorrect SPDX tags creates > more harm > > > With regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > > ^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v2 03/11] acpi/tests/bits: disable acpi PSS tests that are failing in biosbits 2022-07-10 17:00 [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Ani Sinha 2022-07-10 17:00 ` [PATCH v2 01/11] acpi/tests/bits: initial commit of test scripts that are run by biosbits Ani Sinha 2022-07-10 17:00 ` [PATCH v2 02/11] acpi/tests/bits: add SPDX license identifiers for bios bits tests Ani Sinha @ 2022-07-10 17:00 ` Ani Sinha 2022-07-10 17:00 ` [PATCH v2 04/11] acpi/tests/bits: add smilatency test suite from bits in order to disable it Ani Sinha ` (8 subsequent siblings) 11 siblings, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-07-10 17:00 UTC (permalink / raw) To: qemu-devel Cc: thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo, mst, Ani Sinha PSS tests in acpi test suite seems to be failing in biosbits. This is because the test is unable to find PSS support in QEMU bios. Let us disable them for now so that make check does not fail. We can fix the tests and re-enable them later. Example failure: ---- ACPI _PSS (Pstate) table conformance tests ---- [assert] _PSS must exist FAIL \_SB_.CPUS.C000 No _PSS exists Summary: 1 passed, 1 failed ---- ACPI _PSS (Pstate) runtime tests ---- [assert] _PSS must exist FAIL \_SB_.CPUS.C000 No _PSS exists Summary: 0 passed, 1 failed Signed-off-by: Ani Sinha <ani@anisinha.ca> --- tests/pytest/acpi-bits/bits-tests/testacpi.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/pytest/acpi-bits/bits-tests/testacpi.py b/tests/pytest/acpi-bits/bits-tests/testacpi.py index 8e6bb3bbc6..cd2e0ef856 100644 --- a/tests/pytest/acpi-bits/bits-tests/testacpi.py +++ b/tests/pytest/acpi-bits/bits-tests/testacpi.py @@ -38,8 +38,8 @@ def register_tests(): testsuite.add_test("ACPI _MAT (Multiple APIC Table Entry) under Processor objects", test_mat, submenu="ACPI Tests") - testsuite.add_test("ACPI _PSS (Pstate) table conformance tests", test_pss, submenu="ACPI Tests") - testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates, submenu="ACPI Tests") +# testsuite.add_test("ACPI _PSS (Pstate) table conformance tests", test_pss, submenu="ACPI Tests") +# testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates, submenu="ACPI Tests") testsuite.add_test("ACPI DSDT (Differentiated System Description Table)", test_dsdt, submenu="ACPI Tests") testsuite.add_test("ACPI FACP (Fixed ACPI Description Table)", test_facp, submenu="ACPI Tests") testsuite.add_test("ACPI HPET (High Precision Event Timer Table)", test_hpet, submenu="ACPI Tests") -- 2.25.1 ^ permalink raw reply related [flat|nested] 90+ messages in thread
* [PATCH v2 04/11] acpi/tests/bits: add smilatency test suite from bits in order to disable it 2022-07-10 17:00 [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Ani Sinha ` (2 preceding siblings ...) 2022-07-10 17:00 ` [PATCH v2 03/11] acpi/tests/bits: disable acpi PSS tests that are failing in biosbits Ani Sinha @ 2022-07-10 17:00 ` Ani Sinha 2022-07-10 17:00 ` [PATCH v2 05/11] acpi/tests/bits: add SPDX license identifiers for bios bits smilatency tests Ani Sinha ` (7 subsequent siblings) 11 siblings, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-07-10 17:00 UTC (permalink / raw) To: qemu-devel Cc: thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo, mst, Ani Sinha smilatency tests does not reliably pass every time it is run from QEMU. This change adds the test file unchanged from bits so that the next change can disable the test. Signed-off-by: Ani Sinha <ani@anisinha.ca> --- tests/pytest/acpi-bits/bits-tests/meson.build | 2 +- .../pytest/acpi-bits/bits-tests/smilatency.py | 102 ++++++++++++++++++ 2 files changed, 103 insertions(+), 1 deletion(-) create mode 100644 tests/pytest/acpi-bits/bits-tests/smilatency.py diff --git a/tests/pytest/acpi-bits/bits-tests/meson.build b/tests/pytest/acpi-bits/bits-tests/meson.build index 3056731a53..06bca75d99 100644 --- a/tests/pytest/acpi-bits/bits-tests/meson.build +++ b/tests/pytest/acpi-bits/bits-tests/meson.build @@ -1,4 +1,4 @@ -test_files = ['smbios.py', 'testacpi.py', 'testcpuid.py'] +test_files = ['smbios.py', 'smilatency.py', 'testacpi.py', 'testcpuid.py'] copytestfiles = custom_target('copy test files', input : test_files, diff --git a/tests/pytest/acpi-bits/bits-tests/smilatency.py b/tests/pytest/acpi-bits/bits-tests/smilatency.py new file mode 100644 index 0000000000..fb1b7228e3 --- /dev/null +++ b/tests/pytest/acpi-bits/bits-tests/smilatency.py @@ -0,0 +1,102 @@ +# Copyright (c) 2015, Intel Corporation +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# * Redistributions of source code must retain the above copyright notice, +# this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# * Neither the name of Intel Corporation nor the names of its contributors +# may be used to endorse or promote products derived from this software +# without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR +# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""SMI latency test.""" + +import bits +from collections import namedtuple +import testsuite +import time +import usb + +def register_tests(): + testsuite.add_test("SMI latency test", smi_latency); + testsuite.add_test("SMI latency test with USB disabled via BIOS handoff", test_with_usb_disabled, runall=False); + +def smi_latency(): + MSR_SMI_COUNT = 0x34 + + print "Warning: touching the keyboard can affect the results of this test." + + tsc_per_sec = bits.tsc_per_sec() + tsc_per_usec = tsc_per_sec / (1000 * 1000) + bins = [long(tsc_per_usec * 10**i) for i in range(9)] + bin_descs = [ + "0 < t <= 1us", + "1us < t <= 10us", + "10us < t <= 100us", + "100us < t <= 1ms", + "1ms < t <= 10ms", + "10ms < t <= 100ms", + "100ms < t <= 1s ", + "1s < t <= 10s ", + "10s < t <= 100s ", + "100s < t ", + ] + + print "Starting test. Wait here, I will be back in 15 seconds." + (max_latency, smi_count_delta, bins) = bits.smi_latency(long(15 * tsc_per_sec), bins) + BinType = namedtuple('BinType', ("max", "total", "count", "times")) + bins = [BinType(*b) for b in bins] + + testsuite.test("SMI latency < 150us to minimize risk of OS timeouts", max_latency / tsc_per_usec <= 150) + if not testsuite.show_detail(): + return + + for bin, desc in zip(bins, bin_descs): + if bin.count == 0: + continue + testsuite.print_detail("{}; average = {}; count = {}".format(desc, bits.format_tsc(bin.total/bin.count), bin.count)) + deltas = (bits.format_tsc(t2 - t1) for t1,t2 in zip(bin.times, bin.times[1:])) + testsuite.print_detail(" Times between first few observations: {}".format(" ".join("{:>6}".format(delta) for delta in deltas))) + + if smi_count_delta is not None: + testsuite.print_detail("{} SMI detected using MSR_SMI_COUNT (MSR {:#x})".format(smi_count_delta, MSR_SMI_COUNT)) + + testsuite.print_detail("Summary of impact: observed maximum latency = {}".format(bits.format_tsc(max_latency))) + +def test_with_usb_disabled(): + if usb.handoff_to_os(): + smi_latency() + +def average_io_smi(port, value, count): + def f(): + tsc_start = bits.rdtsc() + bits.outb(port, value) + return bits.rdtsc() - tsc_start + counts = [f() for i in range(count)] + return sum(counts)/len(counts) + +def time_io_smi(port=0xb2, value=0, count=1000): + count_for_estimate = 10 + start = time.time() + average_io_smi(port, value, count_for_estimate) + avg10 = time.time() - start + estimate = avg10 * count/count_for_estimate + if estimate > 1: + print "Running test, estimated time: {}s".format(int(estimate)) + average = average_io_smi(port, value, count) + print "Average of {} SMIs (via outb, port={:#x}, value={:#x}): {}".format(count, port, value, bits.format_tsc(average)) -- 2.25.1 ^ permalink raw reply related [flat|nested] 90+ messages in thread
* [PATCH v2 05/11] acpi/tests/bits: add SPDX license identifiers for bios bits smilatency tests 2022-07-10 17:00 [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Ani Sinha ` (3 preceding siblings ...) 2022-07-10 17:00 ` [PATCH v2 04/11] acpi/tests/bits: add smilatency test suite from bits in order to disable it Ani Sinha @ 2022-07-10 17:00 ` Ani Sinha 2022-07-10 17:00 ` [PATCH v2 06/11] acpi/tests/bits: disable smilatency test since it does not pass everytime Ani Sinha ` (6 subsequent siblings) 11 siblings, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-07-10 17:00 UTC (permalink / raw) To: qemu-devel Cc: thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo, mst, Ani Sinha Added the SPDX license identifier based on the following output from the licensee tool for bios bits: $ licensee detect bits/ License: NOASSERTION Matched files: COPYING COPYING: Content hash: 7a1fdfa894728ea69588977442c92073aad69e50 License: NOASSERTION Closest non-matching licenses: BSD-3-Clause-Clear similarity: 85.82% BSD-4-Clause similarity: 83.69% BSD-3-Clause similarity: 77.27% Signed-off-by: Ani Sinha <ani@anisinha.ca> --- tests/pytest/acpi-bits/bits-tests/smilatency.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/pytest/acpi-bits/bits-tests/smilatency.py b/tests/pytest/acpi-bits/bits-tests/smilatency.py index fb1b7228e3..9b11623b2d 100644 --- a/tests/pytest/acpi-bits/bits-tests/smilatency.py +++ b/tests/pytest/acpi-bits/bits-tests/smilatency.py @@ -1,6 +1,8 @@ # Copyright (c) 2015, Intel Corporation # All rights reserved. # +# SPDX-License-Identifier: BSD-3-Clause-Clear +# # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are met: # -- 2.25.1 ^ permalink raw reply related [flat|nested] 90+ messages in thread
* [PATCH v2 06/11] acpi/tests/bits: disable smilatency test since it does not pass everytime 2022-07-10 17:00 [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Ani Sinha ` (4 preceding siblings ...) 2022-07-10 17:00 ` [PATCH v2 05/11] acpi/tests/bits: add SPDX license identifiers for bios bits smilatency tests Ani Sinha @ 2022-07-10 17:00 ` Ani Sinha 2022-07-10 17:00 ` [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits Ani Sinha ` (5 subsequent siblings) 11 siblings, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-07-10 17:00 UTC (permalink / raw) To: qemu-devel Cc: thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo, mst, Ani Sinha smilatency test is latency sensitive and does not pass deterministically when run in QEMU environment under biosbits. Disable the test suite for now. Example failure: ==== SMI latency test ==== Warning: touching the keyboard can affect the results of this test. Starting test. Wait here, I will be back in 15 seconds. [assert] SMI latency < 150us to minimize risk of OS timeouts FAIL 1us < t <= 10us; average = 1372ns; count = 10912449 Times between first few observations: 176us 1646ns 1441ns 1450ns 1462ns 10us < t <= 100us; average = 16us; count = 1187 Times between first few observations: 15ms 3148us 5856us 49ms 33ms 100us < t <= 1ms; average = 259us; count = 8 Times between first few observations: 111ms 2227ms 1779ms 999ms 219ms 0 SMI detected using MSR_SMI_COUNT (MSR 0x34) Summary of impact: observed maximum latency = 298us Summary: 0 passed, 1 failed Signed-off-by: Ani Sinha <ani@anisinha.ca> --- tests/pytest/acpi-bits/bits-tests/smilatency.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/pytest/acpi-bits/bits-tests/smilatency.py b/tests/pytest/acpi-bits/bits-tests/smilatency.py index 9b11623b2d..31d588fb18 100644 --- a/tests/pytest/acpi-bits/bits-tests/smilatency.py +++ b/tests/pytest/acpi-bits/bits-tests/smilatency.py @@ -35,8 +35,9 @@ import usb def register_tests(): - testsuite.add_test("SMI latency test", smi_latency); - testsuite.add_test("SMI latency test with USB disabled via BIOS handoff", test_with_usb_disabled, runall=False); + pass + # testsuite.add_test("SMI latency test", smi_latency); + # testsuite.add_test("SMI latency test with USB disabled via BIOS handoff", test_with_usb_disabled, runall=False); def smi_latency(): MSR_SMI_COUNT = 0x34 -- 2.25.1 ^ permalink raw reply related [flat|nested] 90+ messages in thread
* [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-10 17:00 [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Ani Sinha ` (5 preceding siblings ...) 2022-07-10 17:00 ` [PATCH v2 06/11] acpi/tests/bits: disable smilatency test since it does not pass everytime Ani Sinha @ 2022-07-10 17:00 ` Ani Sinha 2022-07-11 20:42 ` John Snow ` (3 more replies) 2022-07-10 17:00 ` [PATCH v2 08/11] acpi/tests/bits: add biosbits config file for running bios tests Ani Sinha ` (4 subsequent siblings) 11 siblings, 4 replies; 90+ messages in thread From: Ani Sinha @ 2022-07-10 17:00 UTC (permalink / raw) To: qemu-devel, Ani Sinha Cc: thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo, mst This change adds python based test environment that can be used to run pytest from within a virtual environment. A bash script sets up a virtual environment and then runs the python based tests from within that environment. All dependent python packages are installed in the virtual environment using pip python module. QEMU python test modules are also available in the environment for spawning the QEMU based VMs. It also introduces QEMU acpi/smbios biosbits python test script which is run from within the python virtual environment. When the bios bits tests are run, bios bits binaries are downloaded from an external repo/location. Currently, the test points to an external private github repo where the bits archives are checked in. Signed-off-by: Ani Sinha <ani@anisinha.ca> --- tests/pytest/acpi-bits/acpi-bits-test-venv.sh | 59 +++ tests/pytest/acpi-bits/acpi-bits-test.py | 382 ++++++++++++++++++ tests/pytest/acpi-bits/meson.build | 33 ++ tests/pytest/acpi-bits/requirements.txt | 1 + 4 files changed, 475 insertions(+) create mode 100644 tests/pytest/acpi-bits/acpi-bits-test-venv.sh create mode 100644 tests/pytest/acpi-bits/acpi-bits-test.py create mode 100644 tests/pytest/acpi-bits/meson.build create mode 100644 tests/pytest/acpi-bits/requirements.txt diff --git a/tests/pytest/acpi-bits/acpi-bits-test-venv.sh b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh new file mode 100644 index 0000000000..186395473b --- /dev/null +++ b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh @@ -0,0 +1,59 @@ +#!/usr/bin/env bash +# Generates a python virtual environment for the test to run. +# Then runs python test scripts from within that virtual environment. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# +# Author: Ani Sinha <ani@anisinha.ca> + +set -e + +MYPATH=$(realpath ${BASH_SOURCE:-$0}) +MYDIR=$(dirname $MYPATH) + +if [ -z "$PYTEST_SOURCE_ROOT" ]; then + echo -n "Please set QTEST_SOURCE_ROOT env pointing" + echo " to the root of the qemu source tree." + echo -n "This is required so that the test can find the " + echo "python modules that it needs for execution." + exit 1 +fi +SRCDIR=$PYTEST_SOURCE_ROOT +TESTSCRIPTS=("acpi-bits-test.py") +PIPCMD="-m pip -q --disable-pip-version-check" +# we need to save the old value of PWD before we do a change-dir later +PYTEST_PWD=$PWD + +TESTS_PYTHON=/usr/bin/python3 +TESTS_VENV_REQ=requirements.txt + +# sadly for pip -e and -t options do not work together. +# please see https://github.com/pypa/pip/issues/562 +cd $MYDIR + +$TESTS_PYTHON -m venv . +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ +[ -f $TESTS_VENV_REQ ] && \ + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ || exit 0 + +# venv is activated at this point. + +# run the test +for testscript in ${TESTSCRIPTS[@]} ; do + export PYTEST_PWD; python3 $testscript +done + +cd $PYTEST_PWD + +exit 0 diff --git a/tests/pytest/acpi-bits/acpi-bits-test.py b/tests/pytest/acpi-bits/acpi-bits-test.py new file mode 100644 index 0000000000..97e61eb709 --- /dev/null +++ b/tests/pytest/acpi-bits/acpi-bits-test.py @@ -0,0 +1,382 @@ +#!/usr/bin/env python3 +# group: rw quick +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits, +# https://biosbits.org/ +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# +# Some parts are slightly taken from qtest.py and iotests.py +# +# Authors: +# Ani Sinha <ani@anisinha.ca> + +# pylint: disable=invalid-name + +""" +QEMU bios tests using biosbits available at +https://biosbits.org/. +""" + +import logging +import os +import re +import shutil +import subprocess +import sys +import tarfile +import tempfile +import time +import unittest +from urllib import request +import zipfile +from typing import ( + List, + Optional, + Sequence, +) +from tap import TAPTestRunner +from qemu.machine import QEMUMachine + +PYTESTQEMUBIN = os.getenv('PYTEST_QEMU_BINARY') +PYTEST_PWD = os.getenv('PYTEST_PWD') + +def get_arch(): + """finds the arch from the qemu binary name""" + match = re.search('.*qemu-system-(.*)', PYTESTQEMUBIN) + if match: + return match.group(1) + return 'x86_64' + +ARCH = get_arch() + +class QEMUBitsMachine(QEMUMachine): + """ + A QEMU VM, with isa-debugcon enabled and bits iso passed + using -cdrom to QEMU commandline. + """ + def __init__(self, + binary: str, + args: Sequence[str] = (), + wrapper: Sequence[str] = (), + name: Optional[str] = None, + base_temp_dir: str = "/var/tmp", + debugcon_log: str = "debugcon-log.txt", + debugcon_addr: str = "0x403", + sock_dir: Optional[str] = None, + qmp_timer: Optional[float] = None): + # pylint: disable=too-many-arguments + + if name is None: + name = "qemu-bits-%d" % os.getpid() + if sock_dir is None: + sock_dir = base_temp_dir + super().__init__(binary, args, wrapper=wrapper, name=name, + base_temp_dir=base_temp_dir, + sock_dir=sock_dir, qmp_timer=qmp_timer) + self.debugcon_log = debugcon_log + self.debugcon_addr = debugcon_addr + self.base_temp_dir = base_temp_dir + + @property + def _base_args(self) -> List[str]: + args = super()._base_args + args.extend([ + '-chardev', + 'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir, + self.debugcon_log), + '-device', + 'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr, + ]) + return args + + def base_args(self): + """return the base argument to QEMU binary""" + return self._base_args + +class AcpiBitsTest(unittest.TestCase): + """ACPI and SMBIOS tests using biosbits.""" + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self._vm = None + self._workDir = None + self._bitsVer = 2100 + self._bitsLoc = "https://github.com/ani-sinha/bits/raw/bits-builds/" + self._debugcon_addr = '0x403' + self._debugcon_log = 'debugcon-log.txt' + logging.basicConfig(level=logging.INFO) + + def copy_bits_config(self): + """ copies the bios bits config file into bits. + """ + config_file = 'bits-cfg.txt' + qemu_bits_config_dir = os.path.join(os.getcwd(), 'bits-config') + target_config_dir = os.path.join(self._workDir, + 'bits-%d' %self._bitsVer, 'boot') + self.assertTrue(os.path.exists(qemu_bits_config_dir)) + self.assertTrue(os.path.exists(target_config_dir)) + self.assertTrue(os.access(os.path.join(qemu_bits_config_dir, + config_file), os.R_OK)) + shutil.copy2(os.path.join(qemu_bits_config_dir, config_file), + target_config_dir) + logging.info('copied config file %s to %s', + config_file, target_config_dir) + + def copy_test_scripts(self): + """copies the python test scripts into bits. """ + qemu_test_dir = os.path.join(os.getcwd(), 'bits-tests') + target_test_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer, + 'boot', 'python') + + self.assertTrue(os.path.exists(qemu_test_dir)) + self.assertTrue(os.path.exists(target_test_dir)) + + for filename in os.listdir(qemu_test_dir): + if os.path.isfile(os.path.join(qemu_test_dir, filename)) and \ + filename.endswith('.py'): + shutil.copy2(os.path.join(qemu_test_dir, filename), + target_test_dir) + logging.info('copied test file %s to %s', + filename, target_test_dir) + + # now remove the pyc test file if it exists, otherwise the + # changes in the python test script won't be executed. + testfile_pyc = os.path.splitext(filename)[0] + '.pyc' + if os.access(os.path.join(target_test_dir, testfile_pyc), + os.F_OK): + os.remove(os.path.join(target_test_dir, testfile_pyc)) + logging.info('removed compiled file %s', + os.path.join(target_test_dir, testfile_pyc)) + + def fix_mkrescue(self, mkrescue): + """ grub-mkrescue is a bash script with two variables, 'prefix' and + 'libdir'. They must be pointed to the right location so that the + iso can be generated appropriately. We point the two variables to + the directory where we have extracted our pre-built bits grub + tarball. + """ + grub_x86_64_mods = os.path.join(self._workDir, 'grub-inst-x86_64-efi') + grub_i386_mods = os.path.join(self._workDir, 'grub-inst') + + self.assertTrue(os.path.exists(grub_x86_64_mods)) + self.assertTrue(os.path.exists(grub_i386_mods)) + + new_script = "" + with open(mkrescue, 'r') as filehandle: + orig_script = filehandle.read() + new_script = re.sub('(^prefix=)(.*)', + r'\1"%s"' %grub_x86_64_mods, + orig_script, flags=re.M) + new_script = re.sub('(^libdir=)(.*)', r'\1"%s/lib"' %grub_i386_mods, + new_script, flags=re.M) + + with open(mkrescue, 'w') as filehandle: + filehandle.write(new_script) + + def generate_bits_iso(self): + """ Uses grub-mkrescue to generate a fresh bits iso with the python + test scripts + """ + bits_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer) + iso_file = os.path.join(self._workDir, 'bits-%d.iso' %self._bitsVer) + mkrescue_script = os.path.join(self._workDir, + 'grub-inst-x86_64-efi', 'bin', + 'grub-mkrescue') + + self.assertTrue(os.access(mkrescue_script, + os.R_OK | os.W_OK | os.X_OK)) + + self.fix_mkrescue(mkrescue_script) + + logging.info('calling grub-mkrescue to generate the biosbits iso ...') + + try: + if os.getenv('V'): + subprocess.check_call([mkrescue_script, '-o', + iso_file, bits_dir], + stdout=subprocess.DEVNULL) + else: + subprocess.check_call([mkrescue_script, '-o', + iso_file, bits_dir], + stderr=subprocess.DEVNULL, + stdout=subprocess.DEVNULL) + except Exception as e: # pylint: disable=broad-except + self.skipTest("Error while generating the bits iso. " + "Pass V=1 in the environment to get more details. " + + str(e)) + + self.assertTrue(os.access(iso_file, os.R_OK)) + + logging.info('iso file %s successfully generated.', iso_file) + + def setUp(self): + BITS_LOC = os.getenv("PYTEST_BITSLOC") + if BITS_LOC: + prefix = BITS_LOC + else: + prefix = os.path.join(os.getcwd(), 'prebuilt') + if not os.path.isdir(prefix): + os.mkdir(prefix, mode=0o775) + + bits_zip_file = os.path.join(prefix, 'bits-%d.zip' + %self._bitsVer) + grub_tar_file = os.path.join(prefix, + 'bits-%d-grub.tar.gz' %self._bitsVer) + # if the location of the bits binaries has been specified by the user + # and they are not found in that location, skip the test. + if BITS_LOC and not os.access(bits_zip_file, os.F_OK): + self.skipTest("test skipped since biosbits binaries " + + "could not be found in the specified location %s." \ + %BITS_LOC) + if BITS_LOC and not os.access(grub_tar_file, os.F_OK): + self.skipTest("test skipped since biosbits binaries " + + "could not be found in the specified location %s." \ + %BITS_LOC) + + self._workDir = tempfile.mkdtemp(prefix='acpi-bits-', + suffix='.tmp') + logging.info('working dir: %s', self._workDir) + + localArchive = "bits-%d.zip" % self._bitsVer + if not os.access(bits_zip_file, os.F_OK): + logging.info("archive %s not found in %s, downloading ...", + localArchive, bits_zip_file) + try: + req = request.urlopen(self._bitsLoc + localArchive) + with open(os.path.join(prefix, localArchive), + 'wb') as archivef: + archivef.write(req.read()) + except Exception as e: # pylint: disable=broad-except + self.skipTest("test skipped since biosbits binaries " + + "could not be obtained." + str(e)) + else: + logging.info('using locally found %s', localArchive) + + localArchive = "bits-%d-grub.tar.gz" % self._bitsVer + if not os.access(grub_tar_file, os.F_OK): + logging.info("archive %s not found in %s, downloading ...", + localArchive, bits_zip_file) + try: + req = request.urlopen(self._bitsLoc + localArchive) + with open(os.path.join(prefix, localArchive), + 'wb') as archivef: + archivef.write(req.read()) + except Exception as e: # pylint: disable=broad-except + self.skipTest("test skipped since biosbits binaries " + + "could not be obtained." + str(e)) + else: + logging.info('using locally found %s', localArchive) + + # extract the bits software in the temp working directory + with zipfile.ZipFile(bits_zip_file, 'r') as zref: + zref.extractall(self._workDir) + + with tarfile.open(grub_tar_file, 'r') as tarball: + tarball.extractall(self._workDir) + + self.copy_test_scripts() + self.copy_bits_config() + self.generate_bits_iso() + + def parse_log(self): + """parse the log generated by running bits tests and + check for failures. + """ + debugconf = os.path.join(self._workDir, self._debugcon_log) + log = "" + with open(debugconf, 'r') as filehandle: + log = filehandle.read() + + if os.getenv('V'): + print('\nlogs from biosbits follows:') + print('==========================================\n') + print(log) + print('==========================================\n') + + matchiter = re.finditer(r'(.*Summary: )(\d+ passed), (\d+ failed).*', + log) + for match in matchiter: + # verify that no test cases failed. + self.assertEqual(match.group(3).split()[0], '0', + 'Some bits tests seems to have failed. ' \ + 'Set V=1 in the environment to get the entire ' \ + 'log from bits.') + + def tearDown(self): + if self._vm: + self.assertFalse(not self._vm.is_running) + logging.info('removing the work directory %s', self._workDir) + shutil.rmtree(self._workDir) + + def test_acpi_smbios_bits(self): + """The main test case implementaion.""" + + qemu_bin = PYTESTQEMUBIN + iso_file = os.path.join(self._workDir, 'bits-%d.iso' %self._bitsVer) + + # PYTESTQEMUBIN could be relative to the current directory + if not os.access(PYTESTQEMUBIN, os.X_OK) and PYTEST_PWD: + qemu_bin = os.path.join(PYTEST_PWD, PYTESTQEMUBIN) + + logging.info('QEMU binary used: %s', qemu_bin) + + self.assertTrue(os.access(qemu_bin, os.X_OK)) + self.assertTrue(os.access(iso_file, os.R_OK)) + + self._vm = QEMUBitsMachine(binary=qemu_bin, + base_temp_dir=self._workDir, + debugcon_log=self._debugcon_log, + debugcon_addr=self._debugcon_addr) + + self._vm.add_args('-cdrom', '%s' %iso_file) + + args = " ".join(str(arg) for arg in self._vm.base_args()) + \ + " " + " ".join(str(arg) for arg in self._vm.args) + + logging.info("launching QEMU vm with the following arguments: %s", + args) + + self._vm.launch() + # biosbits has been configured to run all the specified test suites + # in batch mode and then automatically initiate a vm shutdown. + # sleep for maximum of one minute + max_sleep_time = time.monotonic() + 60 + while self._vm.is_running() and time.monotonic() < max_sleep_time: + time.sleep(1) + + self.assertFalse(time.monotonic() > max_sleep_time, + 'The VM seems to have failed to shutdown in time') + + self.parse_log() + +def execute_unittest(argv: List[str], debug: bool = False, + runner: TAPTestRunner = None) -> None: + """Executes unittests within the calling module.""" + + unittest.main(argv=argv, + testRunner=runner, + verbosity=2 if debug else 1, + warnings=None if sys.warnoptions else 'ignore') + +def main(): + """ The main function where execution begins. """ + + assert PYTESTQEMUBIN is not None, \ + "Environment variable PYTEST_QEMU_BINARY required." + + runner = TAPTestRunner() + runner.set_stream(True) + runner.set_format("%s/acpi-bits-test" %ARCH) + execute_unittest(sys.argv, False, runner) + +main() diff --git a/tests/pytest/acpi-bits/meson.build b/tests/pytest/acpi-bits/meson.build new file mode 100644 index 0000000000..099c191d57 --- /dev/null +++ b/tests/pytest/acpi-bits/meson.build @@ -0,0 +1,33 @@ +xorriso = find_program('xorriso', required: true) +if not xorriso.found() + message('xorriso not found ... disabled bits acpi tests.') + subdir_done() +endif + +subdir('bits-tests') +subdir('bits-config') + +test_files = ['acpi-bits-test.py'] +requirements = 'requirements.txt' + +copytestfiles = custom_target('copy test files', + input : test_files, + output : test_files, + command : ['cp', '@INPUT@', '@OUTPUT@'], + install : false, + build_by_default : true) + +requirementsfiles = custom_target('copy py req files', + input : requirements, + output : requirements, + command : ['cp', '@INPUT@', '@OUTPUT@'], + install : false, + build_by_default : true) + +other_deps += [copytestfiles,requirementsfiles] + +pytest_executables += { + 'acpi-bits-test': configure_file(copy:true, + input:'acpi-bits-test-venv.sh', + output:'acpi-bits-test') +} diff --git a/tests/pytest/acpi-bits/requirements.txt b/tests/pytest/acpi-bits/requirements.txt new file mode 100644 index 0000000000..00cdad09ef --- /dev/null +++ b/tests/pytest/acpi-bits/requirements.txt @@ -0,0 +1 @@ +tap.py -- 2.25.1 ^ permalink raw reply related [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-10 17:00 ` [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits Ani Sinha @ 2022-07-11 20:42 ` John Snow 2022-07-12 7:15 ` Ani Sinha 2022-07-14 14:05 ` Daniel P. Berrangé ` (2 subsequent siblings) 3 siblings, 1 reply; 90+ messages in thread From: John Snow @ 2022-07-11 20:42 UTC (permalink / raw) To: Ani Sinha Cc: qemu-devel, Thomas Huth, Peter Maydell, Daniel Berrange, Paolo Bonzini, Igor Mammedov, Michael Tsirkin On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > This change adds python based test environment that can be used to run pytest > from within a virtual environment. A bash script sets up a virtual environment > and then runs the python based tests from within that environment. > All dependent python packages are installed in the virtual environment using > pip python module. QEMU python test modules are also available in the environment > for spawning the QEMU based VMs. > > It also introduces QEMU acpi/smbios biosbits python test script which is run > from within the python virtual environment. When the bios bits tests are run, > bios bits binaries are downloaded from an external repo/location. > Currently, the test points to an external private github repo where the bits > archives are checked in. > Oh, I see -- requirements are handled here in this patch. Is this test designed to run the host/build system? I'm a little confused about the environment here. Is this test going to be run "by default" or will users have to opt into running it using a special invocation? --js > Signed-off-by: Ani Sinha <ani@anisinha.ca> > --- > tests/pytest/acpi-bits/acpi-bits-test-venv.sh | 59 +++ > tests/pytest/acpi-bits/acpi-bits-test.py | 382 ++++++++++++++++++ > tests/pytest/acpi-bits/meson.build | 33 ++ > tests/pytest/acpi-bits/requirements.txt | 1 + > 4 files changed, 475 insertions(+) > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test-venv.sh > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test.py > create mode 100644 tests/pytest/acpi-bits/meson.build > create mode 100644 tests/pytest/acpi-bits/requirements.txt > > diff --git a/tests/pytest/acpi-bits/acpi-bits-test-venv.sh b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > new file mode 100644 > index 0000000000..186395473b > --- /dev/null > +++ b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > @@ -0,0 +1,59 @@ > +#!/usr/bin/env bash > +# Generates a python virtual environment for the test to run. > +# Then runs python test scripts from within that virtual environment. > +# > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; either version 2 of the License, or > +# (at your option) any later version. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > +# > +# Author: Ani Sinha <ani@anisinha.ca> > + > +set -e > + > +MYPATH=$(realpath ${BASH_SOURCE:-$0}) > +MYDIR=$(dirname $MYPATH) > + > +if [ -z "$PYTEST_SOURCE_ROOT" ]; then > + echo -n "Please set QTEST_SOURCE_ROOT env pointing" > + echo " to the root of the qemu source tree." > + echo -n "This is required so that the test can find the " > + echo "python modules that it needs for execution." > + exit 1 > +fi > +SRCDIR=$PYTEST_SOURCE_ROOT > +TESTSCRIPTS=("acpi-bits-test.py") > +PIPCMD="-m pip -q --disable-pip-version-check" > +# we need to save the old value of PWD before we do a change-dir later > +PYTEST_PWD=$PWD > + > +TESTS_PYTHON=/usr/bin/python3 > +TESTS_VENV_REQ=requirements.txt > + > +# sadly for pip -e and -t options do not work together. > +# please see https://github.com/pypa/pip/issues/562 > +cd $MYDIR > + > +$TESTS_PYTHON -m venv . > +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ > +[ -f $TESTS_VENV_REQ ] && \ > + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ || exit 0 > + > +# venv is activated at this point. > + > +# run the test > +for testscript in ${TESTSCRIPTS[@]} ; do > + export PYTEST_PWD; python3 $testscript > +done > + > +cd $PYTEST_PWD > + > +exit 0 > diff --git a/tests/pytest/acpi-bits/acpi-bits-test.py b/tests/pytest/acpi-bits/acpi-bits-test.py > new file mode 100644 > index 0000000000..97e61eb709 > --- /dev/null > +++ b/tests/pytest/acpi-bits/acpi-bits-test.py > @@ -0,0 +1,382 @@ > +#!/usr/bin/env python3 > +# group: rw quick > +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits, > +# https://biosbits.org/ > +# > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; either version 2 of the License, or > +# (at your option) any later version. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > +# > +# Some parts are slightly taken from qtest.py and iotests.py > +# > +# Authors: > +# Ani Sinha <ani@anisinha.ca> > + > +# pylint: disable=invalid-name > + > +""" > +QEMU bios tests using biosbits available at > +https://biosbits.org/. > +""" > + > +import logging > +import os > +import re > +import shutil > +import subprocess > +import sys > +import tarfile > +import tempfile > +import time > +import unittest > +from urllib import request > +import zipfile > +from typing import ( > + List, > + Optional, > + Sequence, > +) > +from tap import TAPTestRunner > +from qemu.machine import QEMUMachine > + > +PYTESTQEMUBIN = os.getenv('PYTEST_QEMU_BINARY') > +PYTEST_PWD = os.getenv('PYTEST_PWD') > + > +def get_arch(): > + """finds the arch from the qemu binary name""" > + match = re.search('.*qemu-system-(.*)', PYTESTQEMUBIN) > + if match: > + return match.group(1) > + return 'x86_64' > + > +ARCH = get_arch() > + > +class QEMUBitsMachine(QEMUMachine): > + """ > + A QEMU VM, with isa-debugcon enabled and bits iso passed > + using -cdrom to QEMU commandline. > + """ > + def __init__(self, > + binary: str, > + args: Sequence[str] = (), > + wrapper: Sequence[str] = (), > + name: Optional[str] = None, > + base_temp_dir: str = "/var/tmp", > + debugcon_log: str = "debugcon-log.txt", > + debugcon_addr: str = "0x403", > + sock_dir: Optional[str] = None, > + qmp_timer: Optional[float] = None): > + # pylint: disable=too-many-arguments > + > + if name is None: > + name = "qemu-bits-%d" % os.getpid() > + if sock_dir is None: > + sock_dir = base_temp_dir > + super().__init__(binary, args, wrapper=wrapper, name=name, > + base_temp_dir=base_temp_dir, > + sock_dir=sock_dir, qmp_timer=qmp_timer) > + self.debugcon_log = debugcon_log > + self.debugcon_addr = debugcon_addr > + self.base_temp_dir = base_temp_dir > + > + @property > + def _base_args(self) -> List[str]: > + args = super()._base_args > + args.extend([ > + '-chardev', > + 'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir, > + self.debugcon_log), > + '-device', > + 'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr, > + ]) > + return args > + > + def base_args(self): > + """return the base argument to QEMU binary""" > + return self._base_args > + > +class AcpiBitsTest(unittest.TestCase): > + """ACPI and SMBIOS tests using biosbits.""" > + def __init__(self, *args, **kwargs): > + super().__init__(*args, **kwargs) > + self._vm = None > + self._workDir = None > + self._bitsVer = 2100 > + self._bitsLoc = "https://github.com/ani-sinha/bits/raw/bits-builds/" > + self._debugcon_addr = '0x403' > + self._debugcon_log = 'debugcon-log.txt' > + logging.basicConfig(level=logging.INFO) > + > + def copy_bits_config(self): > + """ copies the bios bits config file into bits. > + """ > + config_file = 'bits-cfg.txt' > + qemu_bits_config_dir = os.path.join(os.getcwd(), 'bits-config') > + target_config_dir = os.path.join(self._workDir, > + 'bits-%d' %self._bitsVer, 'boot') > + self.assertTrue(os.path.exists(qemu_bits_config_dir)) > + self.assertTrue(os.path.exists(target_config_dir)) > + self.assertTrue(os.access(os.path.join(qemu_bits_config_dir, > + config_file), os.R_OK)) > + shutil.copy2(os.path.join(qemu_bits_config_dir, config_file), > + target_config_dir) > + logging.info('copied config file %s to %s', > + config_file, target_config_dir) > + > + def copy_test_scripts(self): > + """copies the python test scripts into bits. """ > + qemu_test_dir = os.path.join(os.getcwd(), 'bits-tests') > + target_test_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer, > + 'boot', 'python') > + > + self.assertTrue(os.path.exists(qemu_test_dir)) > + self.assertTrue(os.path.exists(target_test_dir)) > + > + for filename in os.listdir(qemu_test_dir): > + if os.path.isfile(os.path.join(qemu_test_dir, filename)) and \ > + filename.endswith('.py'): > + shutil.copy2(os.path.join(qemu_test_dir, filename), > + target_test_dir) > + logging.info('copied test file %s to %s', > + filename, target_test_dir) > + > + # now remove the pyc test file if it exists, otherwise the > + # changes in the python test script won't be executed. > + testfile_pyc = os.path.splitext(filename)[0] + '.pyc' > + if os.access(os.path.join(target_test_dir, testfile_pyc), > + os.F_OK): > + os.remove(os.path.join(target_test_dir, testfile_pyc)) > + logging.info('removed compiled file %s', > + os.path.join(target_test_dir, testfile_pyc)) > + > + def fix_mkrescue(self, mkrescue): > + """ grub-mkrescue is a bash script with two variables, 'prefix' and > + 'libdir'. They must be pointed to the right location so that the > + iso can be generated appropriately. We point the two variables to > + the directory where we have extracted our pre-built bits grub > + tarball. > + """ > + grub_x86_64_mods = os.path.join(self._workDir, 'grub-inst-x86_64-efi') > + grub_i386_mods = os.path.join(self._workDir, 'grub-inst') > + > + self.assertTrue(os.path.exists(grub_x86_64_mods)) > + self.assertTrue(os.path.exists(grub_i386_mods)) > + > + new_script = "" > + with open(mkrescue, 'r') as filehandle: > + orig_script = filehandle.read() > + new_script = re.sub('(^prefix=)(.*)', > + r'\1"%s"' %grub_x86_64_mods, > + orig_script, flags=re.M) > + new_script = re.sub('(^libdir=)(.*)', r'\1"%s/lib"' %grub_i386_mods, > + new_script, flags=re.M) > + > + with open(mkrescue, 'w') as filehandle: > + filehandle.write(new_script) > + > + def generate_bits_iso(self): > + """ Uses grub-mkrescue to generate a fresh bits iso with the python > + test scripts > + """ > + bits_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer) > + iso_file = os.path.join(self._workDir, 'bits-%d.iso' %self._bitsVer) > + mkrescue_script = os.path.join(self._workDir, > + 'grub-inst-x86_64-efi', 'bin', > + 'grub-mkrescue') > + > + self.assertTrue(os.access(mkrescue_script, > + os.R_OK | os.W_OK | os.X_OK)) > + > + self.fix_mkrescue(mkrescue_script) > + > + logging.info('calling grub-mkrescue to generate the biosbits iso ...') > + > + try: > + if os.getenv('V'): > + subprocess.check_call([mkrescue_script, '-o', > + iso_file, bits_dir], > + stdout=subprocess.DEVNULL) > + else: > + subprocess.check_call([mkrescue_script, '-o', > + iso_file, bits_dir], > + stderr=subprocess.DEVNULL, > + stdout=subprocess.DEVNULL) > + except Exception as e: # pylint: disable=broad-except > + self.skipTest("Error while generating the bits iso. " > + "Pass V=1 in the environment to get more details. " > + + str(e)) > + > + self.assertTrue(os.access(iso_file, os.R_OK)) > + > + logging.info('iso file %s successfully generated.', iso_file) > + > + def setUp(self): > + BITS_LOC = os.getenv("PYTEST_BITSLOC") > + if BITS_LOC: > + prefix = BITS_LOC > + else: > + prefix = os.path.join(os.getcwd(), 'prebuilt') > + if not os.path.isdir(prefix): > + os.mkdir(prefix, mode=0o775) > + > + bits_zip_file = os.path.join(prefix, 'bits-%d.zip' > + %self._bitsVer) > + grub_tar_file = os.path.join(prefix, > + 'bits-%d-grub.tar.gz' %self._bitsVer) > + # if the location of the bits binaries has been specified by the user > + # and they are not found in that location, skip the test. > + if BITS_LOC and not os.access(bits_zip_file, os.F_OK): > + self.skipTest("test skipped since biosbits binaries " + > + "could not be found in the specified location %s." \ > + %BITS_LOC) > + if BITS_LOC and not os.access(grub_tar_file, os.F_OK): > + self.skipTest("test skipped since biosbits binaries " + > + "could not be found in the specified location %s." \ > + %BITS_LOC) > + > + self._workDir = tempfile.mkdtemp(prefix='acpi-bits-', > + suffix='.tmp') > + logging.info('working dir: %s', self._workDir) > + > + localArchive = "bits-%d.zip" % self._bitsVer > + if not os.access(bits_zip_file, os.F_OK): > + logging.info("archive %s not found in %s, downloading ...", > + localArchive, bits_zip_file) > + try: > + req = request.urlopen(self._bitsLoc + localArchive) > + with open(os.path.join(prefix, localArchive), > + 'wb') as archivef: > + archivef.write(req.read()) > + except Exception as e: # pylint: disable=broad-except > + self.skipTest("test skipped since biosbits binaries " + > + "could not be obtained." + str(e)) > + else: > + logging.info('using locally found %s', localArchive) > + > + localArchive = "bits-%d-grub.tar.gz" % self._bitsVer > + if not os.access(grub_tar_file, os.F_OK): > + logging.info("archive %s not found in %s, downloading ...", > + localArchive, bits_zip_file) > + try: > + req = request.urlopen(self._bitsLoc + localArchive) > + with open(os.path.join(prefix, localArchive), > + 'wb') as archivef: > + archivef.write(req.read()) > + except Exception as e: # pylint: disable=broad-except > + self.skipTest("test skipped since biosbits binaries " + > + "could not be obtained." + str(e)) > + else: > + logging.info('using locally found %s', localArchive) > + > + # extract the bits software in the temp working directory > + with zipfile.ZipFile(bits_zip_file, 'r') as zref: > + zref.extractall(self._workDir) > + > + with tarfile.open(grub_tar_file, 'r') as tarball: > + tarball.extractall(self._workDir) > + > + self.copy_test_scripts() > + self.copy_bits_config() > + self.generate_bits_iso() > + > + def parse_log(self): > + """parse the log generated by running bits tests and > + check for failures. > + """ > + debugconf = os.path.join(self._workDir, self._debugcon_log) > + log = "" > + with open(debugconf, 'r') as filehandle: > + log = filehandle.read() > + > + if os.getenv('V'): > + print('\nlogs from biosbits follows:') > + print('==========================================\n') > + print(log) > + print('==========================================\n') > + > + matchiter = re.finditer(r'(.*Summary: )(\d+ passed), (\d+ failed).*', > + log) > + for match in matchiter: > + # verify that no test cases failed. > + self.assertEqual(match.group(3).split()[0], '0', > + 'Some bits tests seems to have failed. ' \ > + 'Set V=1 in the environment to get the entire ' \ > + 'log from bits.') > + > + def tearDown(self): > + if self._vm: > + self.assertFalse(not self._vm.is_running) > + logging.info('removing the work directory %s', self._workDir) > + shutil.rmtree(self._workDir) > + > + def test_acpi_smbios_bits(self): > + """The main test case implementaion.""" > + > + qemu_bin = PYTESTQEMUBIN > + iso_file = os.path.join(self._workDir, 'bits-%d.iso' %self._bitsVer) > + > + # PYTESTQEMUBIN could be relative to the current directory > + if not os.access(PYTESTQEMUBIN, os.X_OK) and PYTEST_PWD: > + qemu_bin = os.path.join(PYTEST_PWD, PYTESTQEMUBIN) > + > + logging.info('QEMU binary used: %s', qemu_bin) > + > + self.assertTrue(os.access(qemu_bin, os.X_OK)) > + self.assertTrue(os.access(iso_file, os.R_OK)) > + > + self._vm = QEMUBitsMachine(binary=qemu_bin, > + base_temp_dir=self._workDir, > + debugcon_log=self._debugcon_log, > + debugcon_addr=self._debugcon_addr) > + > + self._vm.add_args('-cdrom', '%s' %iso_file) > + > + args = " ".join(str(arg) for arg in self._vm.base_args()) + \ > + " " + " ".join(str(arg) for arg in self._vm.args) > + > + logging.info("launching QEMU vm with the following arguments: %s", > + args) > + > + self._vm.launch() > + # biosbits has been configured to run all the specified test suites > + # in batch mode and then automatically initiate a vm shutdown. > + # sleep for maximum of one minute > + max_sleep_time = time.monotonic() + 60 > + while self._vm.is_running() and time.monotonic() < max_sleep_time: > + time.sleep(1) > + > + self.assertFalse(time.monotonic() > max_sleep_time, > + 'The VM seems to have failed to shutdown in time') > + > + self.parse_log() > + > +def execute_unittest(argv: List[str], debug: bool = False, > + runner: TAPTestRunner = None) -> None: > + """Executes unittests within the calling module.""" > + > + unittest.main(argv=argv, > + testRunner=runner, > + verbosity=2 if debug else 1, > + warnings=None if sys.warnoptions else 'ignore') > + > +def main(): > + """ The main function where execution begins. """ > + > + assert PYTESTQEMUBIN is not None, \ > + "Environment variable PYTEST_QEMU_BINARY required." > + > + runner = TAPTestRunner() > + runner.set_stream(True) > + runner.set_format("%s/acpi-bits-test" %ARCH) > + execute_unittest(sys.argv, False, runner) > + > +main() > diff --git a/tests/pytest/acpi-bits/meson.build b/tests/pytest/acpi-bits/meson.build > new file mode 100644 > index 0000000000..099c191d57 > --- /dev/null > +++ b/tests/pytest/acpi-bits/meson.build > @@ -0,0 +1,33 @@ > +xorriso = find_program('xorriso', required: true) > +if not xorriso.found() > + message('xorriso not found ... disabled bits acpi tests.') > + subdir_done() > +endif > + > +subdir('bits-tests') > +subdir('bits-config') > + > +test_files = ['acpi-bits-test.py'] > +requirements = 'requirements.txt' > + > +copytestfiles = custom_target('copy test files', > + input : test_files, > + output : test_files, > + command : ['cp', '@INPUT@', '@OUTPUT@'], > + install : false, > + build_by_default : true) > + > +requirementsfiles = custom_target('copy py req files', > + input : requirements, > + output : requirements, > + command : ['cp', '@INPUT@', '@OUTPUT@'], > + install : false, > + build_by_default : true) > + > +other_deps += [copytestfiles,requirementsfiles] > + > +pytest_executables += { > + 'acpi-bits-test': configure_file(copy:true, > + input:'acpi-bits-test-venv.sh', > + output:'acpi-bits-test') > +} > diff --git a/tests/pytest/acpi-bits/requirements.txt b/tests/pytest/acpi-bits/requirements.txt > new file mode 100644 > index 0000000000..00cdad09ef > --- /dev/null > +++ b/tests/pytest/acpi-bits/requirements.txt > @@ -0,0 +1 @@ > +tap.py > -- > 2.25.1 > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-11 20:42 ` John Snow @ 2022-07-12 7:15 ` Ani Sinha 2022-09-06 13:05 ` Daniel P. Berrangé 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-07-12 7:15 UTC (permalink / raw) To: John Snow Cc: Ani Sinha, qemu-devel, Thomas Huth, Peter Maydell, Daniel Berrange, Paolo Bonzini, Igor Mammedov, Michael Tsirkin On Mon, 11 Jul 2022, John Snow wrote: > On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > > > This change adds python based test environment that can be used to run pytest > > from within a virtual environment. A bash script sets up a virtual environment > > and then runs the python based tests from within that environment. > > All dependent python packages are installed in the virtual environment using > > pip python module. QEMU python test modules are also available in the environment > > for spawning the QEMU based VMs. > > > > It also introduces QEMU acpi/smbios biosbits python test script which is run > > from within the python virtual environment. When the bios bits tests are run, > > bios bits binaries are downloaded from an external repo/location. > > Currently, the test points to an external private github repo where the bits > > archives are checked in. > > > > Oh, I see -- requirements are handled here in this patch. > > Is this test designed to run the host/build system? I'm a little > confused about the environment here. yes it is supposed to run on the build system where "make check" is executed. > > Is this test going to be run "by default" or will users have to opt > into running it using a special invocation? run by default through "make check" or "make check-pytest" or "make check-pytest-x86_64". Users can also run this test manually by invoking the command directly just like qtests. If the bits artifacts are not found locally and cannot be downloaded, the test is skipped with a (hopefully) helpful message. $ make check-pytest GIT ui/keycodemapdb meson tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc slirp [1/9] Generating qemu-version.h with a custom command (wrapped by meson to capture output) /usr/bin/python3 -B /home/anisinha/workspace/qemu/meson/meson.py test --no-rebuild -t 0 --num-processes 1 --print-errorlogs --suite pytest 1/1 qemu:pytest+pytest-x86_64 / pytest-x86_64/acpi-bits-test SKIP 3.96s 0 subtests passed Ok: 0 Expected Fail: 0 Fail: 0 Unexpected Pass: 0 Skipped: 1 Timeout: 0 Full log written to /home/anisinha/workspace/qemu/build/meson-logs/testlog.txt ----------------------------------- output ----------------------------------- stdout: # TAP results for AcpiBitsTest ok 1 x86_64/acpi-bits-test # SKIP test skipped since biosbits binaries could not be obtained.<urlopen error [Errno -2] Name or service not known> 1..1 stderr: INFO:root:working dir: /tmp/acpi-bits-nbjjjvkt.tmp INFO:root:archive bits-2100.zip not found in /home/anisinha/workspace/qemu/build/tests/pytest/acpi-bits/prebuilt/bits-2100.zip, downloading ... ------------------------------------------------------------------------------ > > --js > > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > > --- > > tests/pytest/acpi-bits/acpi-bits-test-venv.sh | 59 +++ > > tests/pytest/acpi-bits/acpi-bits-test.py | 382 ++++++++++++++++++ > > tests/pytest/acpi-bits/meson.build | 33 ++ > > tests/pytest/acpi-bits/requirements.txt | 1 + > > 4 files changed, 475 insertions(+) > > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test-venv.sh > > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test.py > > create mode 100644 tests/pytest/acpi-bits/meson.build > > create mode 100644 tests/pytest/acpi-bits/requirements.txt > > > > diff --git a/tests/pytest/acpi-bits/acpi-bits-test-venv.sh b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > > new file mode 100644 > > index 0000000000..186395473b > > --- /dev/null > > +++ b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > > @@ -0,0 +1,59 @@ > > +#!/usr/bin/env bash > > +# Generates a python virtual environment for the test to run. > > +# Then runs python test scripts from within that virtual environment. > > +# > > +# This program is free software; you can redistribute it and/or modify > > +# it under the terms of the GNU General Public License as published by > > +# the Free Software Foundation; either version 2 of the License, or > > +# (at your option) any later version. > > +# > > +# This program is distributed in the hope that it will be useful, > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +# GNU General Public License for more details. > > +# > > +# You should have received a copy of the GNU General Public License > > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > > +# > > +# Author: Ani Sinha <ani@anisinha.ca> > > + > > +set -e > > + > > +MYPATH=$(realpath ${BASH_SOURCE:-$0}) > > +MYDIR=$(dirname $MYPATH) > > + > > +if [ -z "$PYTEST_SOURCE_ROOT" ]; then > > + echo -n "Please set QTEST_SOURCE_ROOT env pointing" > > + echo " to the root of the qemu source tree." > > + echo -n "This is required so that the test can find the " > > + echo "python modules that it needs for execution." > > + exit 1 > > +fi > > +SRCDIR=$PYTEST_SOURCE_ROOT > > +TESTSCRIPTS=("acpi-bits-test.py") > > +PIPCMD="-m pip -q --disable-pip-version-check" > > +# we need to save the old value of PWD before we do a change-dir later > > +PYTEST_PWD=$PWD > > + > > +TESTS_PYTHON=/usr/bin/python3 > > +TESTS_VENV_REQ=requirements.txt > > + > > +# sadly for pip -e and -t options do not work together. > > +# please see https://github.com/pypa/pip/issues/562 > > +cd $MYDIR > > + > > +$TESTS_PYTHON -m venv . > > +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ > > +[ -f $TESTS_VENV_REQ ] && \ > > + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ || exit 0 > > + > > +# venv is activated at this point. > > + > > +# run the test > > +for testscript in ${TESTSCRIPTS[@]} ; do > > + export PYTEST_PWD; python3 $testscript > > +done > > + > > +cd $PYTEST_PWD > > + > > +exit 0 > > diff --git a/tests/pytest/acpi-bits/acpi-bits-test.py b/tests/pytest/acpi-bits/acpi-bits-test.py > > new file mode 100644 > > index 0000000000..97e61eb709 > > --- /dev/null > > +++ b/tests/pytest/acpi-bits/acpi-bits-test.py > > @@ -0,0 +1,382 @@ > > +#!/usr/bin/env python3 > > +# group: rw quick > > +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits, > > +# https://biosbits.org/ > > +# > > +# This program is free software; you can redistribute it and/or modify > > +# it under the terms of the GNU General Public License as published by > > +# the Free Software Foundation; either version 2 of the License, or > > +# (at your option) any later version. > > +# > > +# This program is distributed in the hope that it will be useful, > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +# GNU General Public License for more details. > > +# > > +# You should have received a copy of the GNU General Public License > > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > > +# > > +# Some parts are slightly taken from qtest.py and iotests.py > > +# > > +# Authors: > > +# Ani Sinha <ani@anisinha.ca> > > + > > +# pylint: disable=invalid-name > > + > > +""" > > +QEMU bios tests using biosbits available at > > +https://biosbits.org/. > > +""" > > + > > +import logging > > +import os > > +import re > > +import shutil > > +import subprocess > > +import sys > > +import tarfile > > +import tempfile > > +import time > > +import unittest > > +from urllib import request > > +import zipfile > > +from typing import ( > > + List, > > + Optional, > > + Sequence, > > +) > > +from tap import TAPTestRunner > > +from qemu.machine import QEMUMachine > > + > > +PYTESTQEMUBIN = os.getenv('PYTEST_QEMU_BINARY') > > +PYTEST_PWD = os.getenv('PYTEST_PWD') > > + > > +def get_arch(): > > + """finds the arch from the qemu binary name""" > > + match = re.search('.*qemu-system-(.*)', PYTESTQEMUBIN) > > + if match: > > + return match.group(1) > > + return 'x86_64' > > + > > +ARCH = get_arch() > > + > > +class QEMUBitsMachine(QEMUMachine): > > + """ > > + A QEMU VM, with isa-debugcon enabled and bits iso passed > > + using -cdrom to QEMU commandline. > > + """ > > + def __init__(self, > > + binary: str, > > + args: Sequence[str] = (), > > + wrapper: Sequence[str] = (), > > + name: Optional[str] = None, > > + base_temp_dir: str = "/var/tmp", > > + debugcon_log: str = "debugcon-log.txt", > > + debugcon_addr: str = "0x403", > > + sock_dir: Optional[str] = None, > > + qmp_timer: Optional[float] = None): > > + # pylint: disable=too-many-arguments > > + > > + if name is None: > > + name = "qemu-bits-%d" % os.getpid() > > + if sock_dir is None: > > + sock_dir = base_temp_dir > > + super().__init__(binary, args, wrapper=wrapper, name=name, > > + base_temp_dir=base_temp_dir, > > + sock_dir=sock_dir, qmp_timer=qmp_timer) > > + self.debugcon_log = debugcon_log > > + self.debugcon_addr = debugcon_addr > > + self.base_temp_dir = base_temp_dir > > + > > + @property > > + def _base_args(self) -> List[str]: > > + args = super()._base_args > > + args.extend([ > > + '-chardev', > > + 'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir, > > + self.debugcon_log), > > + '-device', > > + 'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr, > > + ]) > > + return args > > + > > + def base_args(self): > > + """return the base argument to QEMU binary""" > > + return self._base_args > > + > > +class AcpiBitsTest(unittest.TestCase): > > + """ACPI and SMBIOS tests using biosbits.""" > > + def __init__(self, *args, **kwargs): > > + super().__init__(*args, **kwargs) > > + self._vm = None > > + self._workDir = None > > + self._bitsVer = 2100 > > + self._bitsLoc = "https://github.com/ani-sinha/bits/raw/bits-builds/" > > + self._debugcon_addr = '0x403' > > + self._debugcon_log = 'debugcon-log.txt' > > + logging.basicConfig(level=logging.INFO) > > + > > + def copy_bits_config(self): > > + """ copies the bios bits config file into bits. > > + """ > > + config_file = 'bits-cfg.txt' > > + qemu_bits_config_dir = os.path.join(os.getcwd(), 'bits-config') > > + target_config_dir = os.path.join(self._workDir, > > + 'bits-%d' %self._bitsVer, 'boot') > > + self.assertTrue(os.path.exists(qemu_bits_config_dir)) > > + self.assertTrue(os.path.exists(target_config_dir)) > > + self.assertTrue(os.access(os.path.join(qemu_bits_config_dir, > > + config_file), os.R_OK)) > > + shutil.copy2(os.path.join(qemu_bits_config_dir, config_file), > > + target_config_dir) > > + logging.info('copied config file %s to %s', > > + config_file, target_config_dir) > > + > > + def copy_test_scripts(self): > > + """copies the python test scripts into bits. """ > > + qemu_test_dir = os.path.join(os.getcwd(), 'bits-tests') > > + target_test_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer, > > + 'boot', 'python') > > + > > + self.assertTrue(os.path.exists(qemu_test_dir)) > > + self.assertTrue(os.path.exists(target_test_dir)) > > + > > + for filename in os.listdir(qemu_test_dir): > > + if os.path.isfile(os.path.join(qemu_test_dir, filename)) and \ > > + filename.endswith('.py'): > > + shutil.copy2(os.path.join(qemu_test_dir, filename), > > + target_test_dir) > > + logging.info('copied test file %s to %s', > > + filename, target_test_dir) > > + > > + # now remove the pyc test file if it exists, otherwise the > > + # changes in the python test script won't be executed. > > + testfile_pyc = os.path.splitext(filename)[0] + '.pyc' > > + if os.access(os.path.join(target_test_dir, testfile_pyc), > > + os.F_OK): > > + os.remove(os.path.join(target_test_dir, testfile_pyc)) > > + logging.info('removed compiled file %s', > > + os.path.join(target_test_dir, testfile_pyc)) > > + > > + def fix_mkrescue(self, mkrescue): > > + """ grub-mkrescue is a bash script with two variables, 'prefix' and > > + 'libdir'. They must be pointed to the right location so that the > > + iso can be generated appropriately. We point the two variables to > > + the directory where we have extracted our pre-built bits grub > > + tarball. > > + """ > > + grub_x86_64_mods = os.path.join(self._workDir, 'grub-inst-x86_64-efi') > > + grub_i386_mods = os.path.join(self._workDir, 'grub-inst') > > + > > + self.assertTrue(os.path.exists(grub_x86_64_mods)) > > + self.assertTrue(os.path.exists(grub_i386_mods)) > > + > > + new_script = "" > > + with open(mkrescue, 'r') as filehandle: > > + orig_script = filehandle.read() > > + new_script = re.sub('(^prefix=)(.*)', > > + r'\1"%s"' %grub_x86_64_mods, > > + orig_script, flags=re.M) > > + new_script = re.sub('(^libdir=)(.*)', r'\1"%s/lib"' %grub_i386_mods, > > + new_script, flags=re.M) > > + > > + with open(mkrescue, 'w') as filehandle: > > + filehandle.write(new_script) > > + > > + def generate_bits_iso(self): > > + """ Uses grub-mkrescue to generate a fresh bits iso with the python > > + test scripts > > + """ > > + bits_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer) > > + iso_file = os.path.join(self._workDir, 'bits-%d.iso' %self._bitsVer) > > + mkrescue_script = os.path.join(self._workDir, > > + 'grub-inst-x86_64-efi', 'bin', > > + 'grub-mkrescue') > > + > > + self.assertTrue(os.access(mkrescue_script, > > + os.R_OK | os.W_OK | os.X_OK)) > > + > > + self.fix_mkrescue(mkrescue_script) > > + > > + logging.info('calling grub-mkrescue to generate the biosbits iso ...') > > + > > + try: > > + if os.getenv('V'): > > + subprocess.check_call([mkrescue_script, '-o', > > + iso_file, bits_dir], > > + stdout=subprocess.DEVNULL) > > + else: > > + subprocess.check_call([mkrescue_script, '-o', > > + iso_file, bits_dir], > > + stderr=subprocess.DEVNULL, > > + stdout=subprocess.DEVNULL) > > + except Exception as e: # pylint: disable=broad-except > > + self.skipTest("Error while generating the bits iso. " > > + "Pass V=1 in the environment to get more details. " > > + + str(e)) > > + > > + self.assertTrue(os.access(iso_file, os.R_OK)) > > + > > + logging.info('iso file %s successfully generated.', iso_file) > > + > > + def setUp(self): > > + BITS_LOC = os.getenv("PYTEST_BITSLOC") > > + if BITS_LOC: > > + prefix = BITS_LOC > > + else: > > + prefix = os.path.join(os.getcwd(), 'prebuilt') > > + if not os.path.isdir(prefix): > > + os.mkdir(prefix, mode=0o775) > > + > > + bits_zip_file = os.path.join(prefix, 'bits-%d.zip' > > + %self._bitsVer) > > + grub_tar_file = os.path.join(prefix, > > + 'bits-%d-grub.tar.gz' %self._bitsVer) > > + # if the location of the bits binaries has been specified by the user > > + # and they are not found in that location, skip the test. > > + if BITS_LOC and not os.access(bits_zip_file, os.F_OK): > > + self.skipTest("test skipped since biosbits binaries " + > > + "could not be found in the specified location %s." \ > > + %BITS_LOC) > > + if BITS_LOC and not os.access(grub_tar_file, os.F_OK): > > + self.skipTest("test skipped since biosbits binaries " + > > + "could not be found in the specified location %s." \ > > + %BITS_LOC) > > + > > + self._workDir = tempfile.mkdtemp(prefix='acpi-bits-', > > + suffix='.tmp') > > + logging.info('working dir: %s', self._workDir) > > + > > + localArchive = "bits-%d.zip" % self._bitsVer > > + if not os.access(bits_zip_file, os.F_OK): > > + logging.info("archive %s not found in %s, downloading ...", > > + localArchive, bits_zip_file) > > + try: > > + req = request.urlopen(self._bitsLoc + localArchive) > > + with open(os.path.join(prefix, localArchive), > > + 'wb') as archivef: > > + archivef.write(req.read()) > > + except Exception as e: # pylint: disable=broad-except > > + self.skipTest("test skipped since biosbits binaries " + > > + "could not be obtained." + str(e)) > > + else: > > + logging.info('using locally found %s', localArchive) > > + > > + localArchive = "bits-%d-grub.tar.gz" % self._bitsVer > > + if not os.access(grub_tar_file, os.F_OK): > > + logging.info("archive %s not found in %s, downloading ...", > > + localArchive, bits_zip_file) > > + try: > > + req = request.urlopen(self._bitsLoc + localArchive) > > + with open(os.path.join(prefix, localArchive), > > + 'wb') as archivef: > > + archivef.write(req.read()) > > + except Exception as e: # pylint: disable=broad-except > > + self.skipTest("test skipped since biosbits binaries " + > > + "could not be obtained." + str(e)) > > + else: > > + logging.info('using locally found %s', localArchive) > > + > > + # extract the bits software in the temp working directory > > + with zipfile.ZipFile(bits_zip_file, 'r') as zref: > > + zref.extractall(self._workDir) > > + > > + with tarfile.open(grub_tar_file, 'r') as tarball: > > + tarball.extractall(self._workDir) > > + > > + self.copy_test_scripts() > > + self.copy_bits_config() > > + self.generate_bits_iso() > > + > > + def parse_log(self): > > + """parse the log generated by running bits tests and > > + check for failures. > > + """ > > + debugconf = os.path.join(self._workDir, self._debugcon_log) > > + log = "" > > + with open(debugconf, 'r') as filehandle: > > + log = filehandle.read() > > + > > + if os.getenv('V'): > > + print('\nlogs from biosbits follows:') > > + print('==========================================\n') > > + print(log) > > + print('==========================================\n') > > + > > + matchiter = re.finditer(r'(.*Summary: )(\d+ passed), (\d+ failed).*', > > + log) > > + for match in matchiter: > > + # verify that no test cases failed. > > + self.assertEqual(match.group(3).split()[0], '0', > > + 'Some bits tests seems to have failed. ' \ > > + 'Set V=1 in the environment to get the entire ' \ > > + 'log from bits.') > > + > > + def tearDown(self): > > + if self._vm: > > + self.assertFalse(not self._vm.is_running) > > + logging.info('removing the work directory %s', self._workDir) > > + shutil.rmtree(self._workDir) > > + > > + def test_acpi_smbios_bits(self): > > + """The main test case implementaion.""" > > + > > + qemu_bin = PYTESTQEMUBIN > > + iso_file = os.path.join(self._workDir, 'bits-%d.iso' %self._bitsVer) > > + > > + # PYTESTQEMUBIN could be relative to the current directory > > + if not os.access(PYTESTQEMUBIN, os.X_OK) and PYTEST_PWD: > > + qemu_bin = os.path.join(PYTEST_PWD, PYTESTQEMUBIN) > > + > > + logging.info('QEMU binary used: %s', qemu_bin) > > + > > + self.assertTrue(os.access(qemu_bin, os.X_OK)) > > + self.assertTrue(os.access(iso_file, os.R_OK)) > > + > > + self._vm = QEMUBitsMachine(binary=qemu_bin, > > + base_temp_dir=self._workDir, > > + debugcon_log=self._debugcon_log, > > + debugcon_addr=self._debugcon_addr) > > + > > + self._vm.add_args('-cdrom', '%s' %iso_file) > > + > > + args = " ".join(str(arg) for arg in self._vm.base_args()) + \ > > + " " + " ".join(str(arg) for arg in self._vm.args) > > + > > + logging.info("launching QEMU vm with the following arguments: %s", > > + args) > > + > > + self._vm.launch() > > + # biosbits has been configured to run all the specified test suites > > + # in batch mode and then automatically initiate a vm shutdown. > > + # sleep for maximum of one minute > > + max_sleep_time = time.monotonic() + 60 > > + while self._vm.is_running() and time.monotonic() < max_sleep_time: > > + time.sleep(1) > > + > > + self.assertFalse(time.monotonic() > max_sleep_time, > > + 'The VM seems to have failed to shutdown in time') > > + > > + self.parse_log() > > + > > +def execute_unittest(argv: List[str], debug: bool = False, > > + runner: TAPTestRunner = None) -> None: > > + """Executes unittests within the calling module.""" > > + > > + unittest.main(argv=argv, > > + testRunner=runner, > > + verbosity=2 if debug else 1, > > + warnings=None if sys.warnoptions else 'ignore') > > + > > +def main(): > > + """ The main function where execution begins. """ > > + > > + assert PYTESTQEMUBIN is not None, \ > > + "Environment variable PYTEST_QEMU_BINARY required." > > + > > + runner = TAPTestRunner() > > + runner.set_stream(True) > > + runner.set_format("%s/acpi-bits-test" %ARCH) > > + execute_unittest(sys.argv, False, runner) > > + > > +main() > > diff --git a/tests/pytest/acpi-bits/meson.build b/tests/pytest/acpi-bits/meson.build > > new file mode 100644 > > index 0000000000..099c191d57 > > --- /dev/null > > +++ b/tests/pytest/acpi-bits/meson.build > > @@ -0,0 +1,33 @@ > > +xorriso = find_program('xorriso', required: true) > > +if not xorriso.found() > > + message('xorriso not found ... disabled bits acpi tests.') > > + subdir_done() > > +endif > > + > > +subdir('bits-tests') > > +subdir('bits-config') > > + > > +test_files = ['acpi-bits-test.py'] > > +requirements = 'requirements.txt' > > + > > +copytestfiles = custom_target('copy test files', > > + input : test_files, > > + output : test_files, > > + command : ['cp', '@INPUT@', '@OUTPUT@'], > > + install : false, > > + build_by_default : true) > > + > > +requirementsfiles = custom_target('copy py req files', > > + input : requirements, > > + output : requirements, > > + command : ['cp', '@INPUT@', '@OUTPUT@'], > > + install : false, > > + build_by_default : true) > > + > > +other_deps += [copytestfiles,requirementsfiles] > > + > > +pytest_executables += { > > + 'acpi-bits-test': configure_file(copy:true, > > + input:'acpi-bits-test-venv.sh', > > + output:'acpi-bits-test') > > +} > > diff --git a/tests/pytest/acpi-bits/requirements.txt b/tests/pytest/acpi-bits/requirements.txt > > new file mode 100644 > > index 0000000000..00cdad09ef > > --- /dev/null > > +++ b/tests/pytest/acpi-bits/requirements.txt > > @@ -0,0 +1 @@ > > +tap.py > > -- > > 2.25.1 > > > > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-12 7:15 ` Ani Sinha @ 2022-09-06 13:05 ` Daniel P. Berrangé 0 siblings, 0 replies; 90+ messages in thread From: Daniel P. Berrangé @ 2022-09-06 13:05 UTC (permalink / raw) To: Ani Sinha Cc: John Snow, qemu-devel, Thomas Huth, Peter Maydell, Paolo Bonzini, Igor Mammedov, Michael Tsirkin On Tue, Jul 12, 2022 at 12:45:59PM +0530, Ani Sinha wrote: > > > On Mon, 11 Jul 2022, John Snow wrote: > > > On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > This change adds python based test environment that can be used to run pytest > > > from within a virtual environment. A bash script sets up a virtual environment > > > and then runs the python based tests from within that environment. > > > All dependent python packages are installed in the virtual environment using > > > pip python module. QEMU python test modules are also available in the environment > > > for spawning the QEMU based VMs. > > > > > > It also introduces QEMU acpi/smbios biosbits python test script which is run > > > from within the python virtual environment. When the bios bits tests are run, > > > bios bits binaries are downloaded from an external repo/location. > > > Currently, the test points to an external private github repo where the bits > > > archives are checked in. > > > > > > > Oh, I see -- requirements are handled here in this patch. > > > > Is this test designed to run the host/build system? I'm a little > > confused about the environment here. > > yes it is supposed to run on the build system where "make check" is > executed. > > > > > Is this test going to be run "by default" or will users have to opt > > into running it using a special invocation? > > run by default through "make check" or "make check-pytest" or "make > check-pytest-x86_64". I don't think that a default 'make check' should be downloading disk images, especially when they're stored in the local build dir by default and so lost when the build dir is cleaned. The downloading of disk images is one of the reasons why avocado is not set to run by default with 'make check', and I think this should be the same. This again takes me to back to believing that we should simply be using Avocado here instead of creating something that has to solve the same problems. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-10 17:00 ` [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits Ani Sinha 2022-07-11 20:42 ` John Snow @ 2022-07-14 14:05 ` Daniel P. Berrangé 2022-07-14 14:19 ` Ani Sinha 2022-07-14 20:41 ` Michael S. Tsirkin 2022-09-27 21:44 ` Paolo Bonzini 3 siblings, 1 reply; 90+ messages in thread From: Daniel P. Berrangé @ 2022-07-14 14:05 UTC (permalink / raw) To: Ani Sinha Cc: qemu-devel, thuth, peter.maydell, jsnow, pbonzini, imammedo, mst On Sun, Jul 10, 2022 at 10:30:10PM +0530, Ani Sinha wrote: > This change adds python based test environment that can be used to run pytest > from within a virtual environment. A bash script sets up a virtual environment > and then runs the python based tests from within that environment. > All dependent python packages are installed in the virtual environment using > pip python module. QEMU python test modules are also available in the environment > for spawning the QEMU based VMs. > > It also introduces QEMU acpi/smbios biosbits python test script which is run > from within the python virtual environment. When the bios bits tests are run, > bios bits binaries are downloaded from an external repo/location. > Currently, the test points to an external private github repo where the bits > archives are checked in. > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > --- > tests/pytest/acpi-bits/acpi-bits-test-venv.sh | 59 +++ > tests/pytest/acpi-bits/acpi-bits-test.py | 382 ++++++++++++++++++ > tests/pytest/acpi-bits/meson.build | 33 ++ > tests/pytest/acpi-bits/requirements.txt | 1 + > 4 files changed, 475 insertions(+) > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test-venv.sh > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test.py > create mode 100644 tests/pytest/acpi-bits/meson.build > create mode 100644 tests/pytest/acpi-bits/requirements.txt > > diff --git a/tests/pytest/acpi-bits/acpi-bits-test-venv.sh b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > new file mode 100644 > index 0000000000..186395473b > --- /dev/null > +++ b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > @@ -0,0 +1,59 @@ > +#!/usr/bin/env bash > +# Generates a python virtual environment for the test to run. > +# Then runs python test scripts from within that virtual environment. > +# > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; either version 2 of the License, or > +# (at your option) any later version. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > +# > +# Author: Ani Sinha <ani@anisinha.ca> > + > +set -e > + > +MYPATH=$(realpath ${BASH_SOURCE:-$0}) > +MYDIR=$(dirname $MYPATH) > + > +if [ -z "$PYTEST_SOURCE_ROOT" ]; then > + echo -n "Please set QTEST_SOURCE_ROOT env pointing" > + echo " to the root of the qemu source tree." > + echo -n "This is required so that the test can find the " > + echo "python modules that it needs for execution." > + exit 1 > +fi > +SRCDIR=$PYTEST_SOURCE_ROOT > +TESTSCRIPTS=("acpi-bits-test.py") > +PIPCMD="-m pip -q --disable-pip-version-check" > +# we need to save the old value of PWD before we do a change-dir later > +PYTEST_PWD=$PWD > + > +TESTS_PYTHON=/usr/bin/python3 > +TESTS_VENV_REQ=requirements.txt > + > +# sadly for pip -e and -t options do not work together. > +# please see https://github.com/pypa/pip/issues/562 > +cd $MYDIR > + > +$TESTS_PYTHON -m venv . > +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ > +[ -f $TESTS_VENV_REQ ] && \ > + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ || exit 0 > + > +# venv is activated at this point. > + > +# run the test > +for testscript in ${TESTSCRIPTS[@]} ; do > + export PYTEST_PWD; python3 $testscript > +done > + > +cd $PYTEST_PWD > + > +exit 0 > diff --git a/tests/pytest/acpi-bits/acpi-bits-test.py b/tests/pytest/acpi-bits/acpi-bits-test.py > new file mode 100644 > index 0000000000..97e61eb709 > --- /dev/null > +++ b/tests/pytest/acpi-bits/acpi-bits-test.py > @@ -0,0 +1,382 @@ > +#!/usr/bin/env python3 > +# group: rw quick > +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits, > +# https://biosbits.org/ > +# > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; either version 2 of the License, or > +# (at your option) any later version. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > +# > +# Some parts are slightly taken from qtest.py and iotests.py > +# > +# Authors: > +# Ani Sinha <ani@anisinha.ca> > + > +# pylint: disable=invalid-name > + > +""" > +QEMU bios tests using biosbits available at > +https://biosbits.org/. > +""" > + > +import logging > +import os > +import re > +import shutil > +import subprocess > +import sys > +import tarfile > +import tempfile > +import time > +import unittest > +from urllib import request > +import zipfile > +from typing import ( > + List, > + Optional, > + Sequence, > +) > +from tap import TAPTestRunner > +from qemu.machine import QEMUMachine > + > +PYTESTQEMUBIN = os.getenv('PYTEST_QEMU_BINARY') > +PYTEST_PWD = os.getenv('PYTEST_PWD') > + > +def get_arch(): > + """finds the arch from the qemu binary name""" > + match = re.search('.*qemu-system-(.*)', PYTESTQEMUBIN) > + if match: > + return match.group(1) > + return 'x86_64' > + > +ARCH = get_arch() > + > +class QEMUBitsMachine(QEMUMachine): > + """ > + A QEMU VM, with isa-debugcon enabled and bits iso passed > + using -cdrom to QEMU commandline. > + """ > + def __init__(self, > + binary: str, > + args: Sequence[str] = (), > + wrapper: Sequence[str] = (), > + name: Optional[str] = None, > + base_temp_dir: str = "/var/tmp", > + debugcon_log: str = "debugcon-log.txt", > + debugcon_addr: str = "0x403", > + sock_dir: Optional[str] = None, > + qmp_timer: Optional[float] = None): > + # pylint: disable=too-many-arguments > + > + if name is None: > + name = "qemu-bits-%d" % os.getpid() > + if sock_dir is None: > + sock_dir = base_temp_dir > + super().__init__(binary, args, wrapper=wrapper, name=name, > + base_temp_dir=base_temp_dir, > + sock_dir=sock_dir, qmp_timer=qmp_timer) > + self.debugcon_log = debugcon_log > + self.debugcon_addr = debugcon_addr > + self.base_temp_dir = base_temp_dir > + > + @property > + def _base_args(self) -> List[str]: > + args = super()._base_args > + args.extend([ > + '-chardev', > + 'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir, > + self.debugcon_log), > + '-device', > + 'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr, > + ]) > + return args > + > + def base_args(self): > + """return the base argument to QEMU binary""" > + return self._base_args > + > +class AcpiBitsTest(unittest.TestCase): > + """ACPI and SMBIOS tests using biosbits.""" > + def __init__(self, *args, **kwargs): > + super().__init__(*args, **kwargs) > + self._vm = None > + self._workDir = None > + self._bitsVer = 2100 > + self._bitsLoc = "https://github.com/ani-sinha/bits/raw/bits-builds/" This URL location gives a 404 - was it supposed to be poiinting to the bits-builds branch ? eg to form a URL like: https://github.com/ani-sinha/bits/blob/bits-builds/bits-2100.zip?raw=true > + self._debugcon_addr = '0x403' > + self._debugcon_log = 'debugcon-log.txt' > + logging.basicConfig(level=logging.INFO) > + > + def copy_bits_config(self): > + """ copies the bios bits config file into bits. > + """ > + config_file = 'bits-cfg.txt' > + qemu_bits_config_dir = os.path.join(os.getcwd(), 'bits-config') > + target_config_dir = os.path.join(self._workDir, > + 'bits-%d' %self._bitsVer, 'boot') > + self.assertTrue(os.path.exists(qemu_bits_config_dir)) > + self.assertTrue(os.path.exists(target_config_dir)) > + self.assertTrue(os.access(os.path.join(qemu_bits_config_dir, > + config_file), os.R_OK)) > + shutil.copy2(os.path.join(qemu_bits_config_dir, config_file), > + target_config_dir) > + logging.info('copied config file %s to %s', > + config_file, target_config_dir) > + > + def copy_test_scripts(self): > + """copies the python test scripts into bits. """ > + qemu_test_dir = os.path.join(os.getcwd(), 'bits-tests') > + target_test_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer, > + 'boot', 'python') > + > + self.assertTrue(os.path.exists(qemu_test_dir)) > + self.assertTrue(os.path.exists(target_test_dir)) > + > + for filename in os.listdir(qemu_test_dir): > + if os.path.isfile(os.path.join(qemu_test_dir, filename)) and \ > + filename.endswith('.py'): > + shutil.copy2(os.path.join(qemu_test_dir, filename), > + target_test_dir) > + logging.info('copied test file %s to %s', > + filename, target_test_dir) > + > + # now remove the pyc test file if it exists, otherwise the > + # changes in the python test script won't be executed. > + testfile_pyc = os.path.splitext(filename)[0] + '.pyc' > + if os.access(os.path.join(target_test_dir, testfile_pyc), > + os.F_OK): > + os.remove(os.path.join(target_test_dir, testfile_pyc)) > + logging.info('removed compiled file %s', > + os.path.join(target_test_dir, testfile_pyc)) > + > + def fix_mkrescue(self, mkrescue): > + """ grub-mkrescue is a bash script with two variables, 'prefix' and > + 'libdir'. They must be pointed to the right location so that the > + iso can be generated appropriately. We point the two variables to > + the directory where we have extracted our pre-built bits grub > + tarball. > + """ > + grub_x86_64_mods = os.path.join(self._workDir, 'grub-inst-x86_64-efi') > + grub_i386_mods = os.path.join(self._workDir, 'grub-inst') > + > + self.assertTrue(os.path.exists(grub_x86_64_mods)) > + self.assertTrue(os.path.exists(grub_i386_mods)) > + > + new_script = "" > + with open(mkrescue, 'r') as filehandle: > + orig_script = filehandle.read() > + new_script = re.sub('(^prefix=)(.*)', > + r'\1"%s"' %grub_x86_64_mods, > + orig_script, flags=re.M) > + new_script = re.sub('(^libdir=)(.*)', r'\1"%s/lib"' %grub_i386_mods, > + new_script, flags=re.M) > + > + with open(mkrescue, 'w') as filehandle: > + filehandle.write(new_script) > + > + def generate_bits_iso(self): > + """ Uses grub-mkrescue to generate a fresh bits iso with the python > + test scripts > + """ > + bits_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer) > + iso_file = os.path.join(self._workDir, 'bits-%d.iso' %self._bitsVer) > + mkrescue_script = os.path.join(self._workDir, > + 'grub-inst-x86_64-efi', 'bin', > + 'grub-mkrescue') > + > + self.assertTrue(os.access(mkrescue_script, > + os.R_OK | os.W_OK | os.X_OK)) > + > + self.fix_mkrescue(mkrescue_script) > + > + logging.info('calling grub-mkrescue to generate the biosbits iso ...') > + > + try: > + if os.getenv('V'): > + subprocess.check_call([mkrescue_script, '-o', > + iso_file, bits_dir], > + stdout=subprocess.DEVNULL) > + else: > + subprocess.check_call([mkrescue_script, '-o', > + iso_file, bits_dir], > + stderr=subprocess.DEVNULL, > + stdout=subprocess.DEVNULL) > + except Exception as e: # pylint: disable=broad-except > + self.skipTest("Error while generating the bits iso. " > + "Pass V=1 in the environment to get more details. " > + + str(e)) > + > + self.assertTrue(os.access(iso_file, os.R_OK)) > + > + logging.info('iso file %s successfully generated.', iso_file) > + > + def setUp(self): > + BITS_LOC = os.getenv("PYTEST_BITSLOC") > + if BITS_LOC: > + prefix = BITS_LOC > + else: > + prefix = os.path.join(os.getcwd(), 'prebuilt') > + if not os.path.isdir(prefix): > + os.mkdir(prefix, mode=0o775) > + > + bits_zip_file = os.path.join(prefix, 'bits-%d.zip' > + %self._bitsVer) > + grub_tar_file = os.path.join(prefix, > + 'bits-%d-grub.tar.gz' %self._bitsVer) > + # if the location of the bits binaries has been specified by the user > + # and they are not found in that location, skip the test. > + if BITS_LOC and not os.access(bits_zip_file, os.F_OK): > + self.skipTest("test skipped since biosbits binaries " + > + "could not be found in the specified location %s." \ > + %BITS_LOC) > + if BITS_LOC and not os.access(grub_tar_file, os.F_OK): > + self.skipTest("test skipped since biosbits binaries " + > + "could not be found in the specified location %s." \ > + %BITS_LOC) > + > + self._workDir = tempfile.mkdtemp(prefix='acpi-bits-', > + suffix='.tmp') > + logging.info('working dir: %s', self._workDir) > + > + localArchive = "bits-%d.zip" % self._bitsVer > + if not os.access(bits_zip_file, os.F_OK): > + logging.info("archive %s not found in %s, downloading ...", > + localArchive, bits_zip_file) > + try: > + req = request.urlopen(self._bitsLoc + localArchive) > + with open(os.path.join(prefix, localArchive), > + 'wb') as archivef: > + archivef.write(req.read()) > + except Exception as e: # pylint: disable=broad-except > + self.skipTest("test skipped since biosbits binaries " + > + "could not be obtained." + str(e)) > + else: > + logging.info('using locally found %s', localArchive) so you skip downlaod if it already exists locally. IIUC it is looking in the CWD, which is presumably the directory the QEMU build is performed in ? So if dev cleans their build tree, the cache is lost ? Avocado has a more persistent cache outside the build tree IIUC. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-14 14:05 ` Daniel P. Berrangé @ 2022-07-14 14:19 ` Ani Sinha 2022-07-14 17:49 ` Ani Sinha 2022-07-14 20:43 ` Michael S. Tsirkin 0 siblings, 2 replies; 90+ messages in thread From: Ani Sinha @ 2022-07-14 14:19 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Ani Sinha, qemu-devel, thuth, peter.maydell, jsnow, pbonzini, imammedo, mst [-- Attachment #1: Type: text/plain, Size: 17774 bytes --] On Thu, 14 Jul 2022, Daniel P. Berrangé wrote: > On Sun, Jul 10, 2022 at 10:30:10PM +0530, Ani Sinha wrote: > > This change adds python based test environment that can be used to run pytest > > from within a virtual environment. A bash script sets up a virtual environment > > and then runs the python based tests from within that environment. > > All dependent python packages are installed in the virtual environment using > > pip python module. QEMU python test modules are also available in the environment > > for spawning the QEMU based VMs. > > > > It also introduces QEMU acpi/smbios biosbits python test script which is run > > from within the python virtual environment. When the bios bits tests are run, > > bios bits binaries are downloaded from an external repo/location. > > Currently, the test points to an external private github repo where the bits > > archives are checked in. > > > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > > --- > > tests/pytest/acpi-bits/acpi-bits-test-venv.sh | 59 +++ > > tests/pytest/acpi-bits/acpi-bits-test.py | 382 ++++++++++++++++++ > > tests/pytest/acpi-bits/meson.build | 33 ++ > > tests/pytest/acpi-bits/requirements.txt | 1 + > > 4 files changed, 475 insertions(+) > > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test-venv.sh > > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test.py > > create mode 100644 tests/pytest/acpi-bits/meson.build > > create mode 100644 tests/pytest/acpi-bits/requirements.txt > > > > diff --git a/tests/pytest/acpi-bits/acpi-bits-test-venv.sh b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > > new file mode 100644 > > index 0000000000..186395473b > > --- /dev/null > > +++ b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > > @@ -0,0 +1,59 @@ > > +#!/usr/bin/env bash > > +# Generates a python virtual environment for the test to run. > > +# Then runs python test scripts from within that virtual environment. > > +# > > +# This program is free software; you can redistribute it and/or modify > > +# it under the terms of the GNU General Public License as published by > > +# the Free Software Foundation; either version 2 of the License, or > > +# (at your option) any later version. > > +# > > +# This program is distributed in the hope that it will be useful, > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +# GNU General Public License for more details. > > +# > > +# You should have received a copy of the GNU General Public License > > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > > +# > > +# Author: Ani Sinha <ani@anisinha.ca> > > + > > +set -e > > + > > +MYPATH=$(realpath ${BASH_SOURCE:-$0}) > > +MYDIR=$(dirname $MYPATH) > > + > > +if [ -z "$PYTEST_SOURCE_ROOT" ]; then > > + echo -n "Please set QTEST_SOURCE_ROOT env pointing" > > + echo " to the root of the qemu source tree." > > + echo -n "This is required so that the test can find the " > > + echo "python modules that it needs for execution." > > + exit 1 > > +fi > > +SRCDIR=$PYTEST_SOURCE_ROOT > > +TESTSCRIPTS=("acpi-bits-test.py") > > +PIPCMD="-m pip -q --disable-pip-version-check" > > +# we need to save the old value of PWD before we do a change-dir later > > +PYTEST_PWD=$PWD > > + > > +TESTS_PYTHON=/usr/bin/python3 > > +TESTS_VENV_REQ=requirements.txt > > + > > +# sadly for pip -e and -t options do not work together. > > +# please see https://github.com/pypa/pip/issues/562 > > +cd $MYDIR > > + > > +$TESTS_PYTHON -m venv . > > +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ > > +[ -f $TESTS_VENV_REQ ] && \ > > + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ || exit 0 > > + > > +# venv is activated at this point. > > + > > +# run the test > > +for testscript in ${TESTSCRIPTS[@]} ; do > > + export PYTEST_PWD; python3 $testscript > > +done > > + > > +cd $PYTEST_PWD > > + > > +exit 0 > > diff --git a/tests/pytest/acpi-bits/acpi-bits-test.py b/tests/pytest/acpi-bits/acpi-bits-test.py > > new file mode 100644 > > index 0000000000..97e61eb709 > > --- /dev/null > > +++ b/tests/pytest/acpi-bits/acpi-bits-test.py > > @@ -0,0 +1,382 @@ > > +#!/usr/bin/env python3 > > +# group: rw quick > > +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits, > > +# https://biosbits.org/ > > +# > > +# This program is free software; you can redistribute it and/or modify > > +# it under the terms of the GNU General Public License as published by > > +# the Free Software Foundation; either version 2 of the License, or > > +# (at your option) any later version. > > +# > > +# This program is distributed in the hope that it will be useful, > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +# GNU General Public License for more details. > > +# > > +# You should have received a copy of the GNU General Public License > > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > > +# > > +# Some parts are slightly taken from qtest.py and iotests.py > > +# > > +# Authors: > > +# Ani Sinha <ani@anisinha.ca> > > + > > +# pylint: disable=invalid-name > > + > > +""" > > +QEMU bios tests using biosbits available at > > +https://biosbits.org/. > > +""" > > + > > +import logging > > +import os > > +import re > > +import shutil > > +import subprocess > > +import sys > > +import tarfile > > +import tempfile > > +import time > > +import unittest > > +from urllib import request > > +import zipfile > > +from typing import ( > > + List, > > + Optional, > > + Sequence, > > +) > > +from tap import TAPTestRunner > > +from qemu.machine import QEMUMachine > > + > > +PYTESTQEMUBIN = os.getenv('PYTEST_QEMU_BINARY') > > +PYTEST_PWD = os.getenv('PYTEST_PWD') > > + > > +def get_arch(): > > + """finds the arch from the qemu binary name""" > > + match = re.search('.*qemu-system-(.*)', PYTESTQEMUBIN) > > + if match: > > + return match.group(1) > > + return 'x86_64' > > + > > +ARCH = get_arch() > > + > > +class QEMUBitsMachine(QEMUMachine): > > + """ > > + A QEMU VM, with isa-debugcon enabled and bits iso passed > > + using -cdrom to QEMU commandline. > > + """ > > + def __init__(self, > > + binary: str, > > + args: Sequence[str] = (), > > + wrapper: Sequence[str] = (), > > + name: Optional[str] = None, > > + base_temp_dir: str = "/var/tmp", > > + debugcon_log: str = "debugcon-log.txt", > > + debugcon_addr: str = "0x403", > > + sock_dir: Optional[str] = None, > > + qmp_timer: Optional[float] = None): > > + # pylint: disable=too-many-arguments > > + > > + if name is None: > > + name = "qemu-bits-%d" % os.getpid() > > + if sock_dir is None: > > + sock_dir = base_temp_dir > > + super().__init__(binary, args, wrapper=wrapper, name=name, > > + base_temp_dir=base_temp_dir, > > + sock_dir=sock_dir, qmp_timer=qmp_timer) > > + self.debugcon_log = debugcon_log > > + self.debugcon_addr = debugcon_addr > > + self.base_temp_dir = base_temp_dir > > + > > + @property > > + def _base_args(self) -> List[str]: > > + args = super()._base_args > > + args.extend([ > > + '-chardev', > > + 'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir, > > + self.debugcon_log), > > + '-device', > > + 'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr, > > + ]) > > + return args > > + > > + def base_args(self): > > + """return the base argument to QEMU binary""" > > + return self._base_args > > + > > +class AcpiBitsTest(unittest.TestCase): > > + """ACPI and SMBIOS tests using biosbits.""" > > + def __init__(self, *args, **kwargs): > > + super().__init__(*args, **kwargs) > > + self._vm = None > > + self._workDir = None > > + self._bitsVer = 2100 > > + self._bitsLoc = "https://github.com/ani-sinha/bits/raw/bits-builds/" > > This URL location gives a 404 - was it supposed to be poiinting to the > bits-builds branch ? eg to form a URL like: > You are looking at a partial URL. The code completes the URL based on the version we are using. Append to this URL the archive name, like bits-2100.zip or something. eg, https://github.com/ani-sinha/bits/raw/bits-builds/bits-2100.zip > https://github.com/ani-sinha/bits/blob/bits-builds/bits-2100.zip?raw=true > > > + self._debugcon_addr = '0x403' > > + self._debugcon_log = 'debugcon-log.txt' > > + logging.basicConfig(level=logging.INFO) > > + > > + def copy_bits_config(self): > > + """ copies the bios bits config file into bits. > > + """ > > + config_file = 'bits-cfg.txt' > > + qemu_bits_config_dir = os.path.join(os.getcwd(), 'bits-config') > > + target_config_dir = os.path.join(self._workDir, > > + 'bits-%d' %self._bitsVer, 'boot') > > + self.assertTrue(os.path.exists(qemu_bits_config_dir)) > > + self.assertTrue(os.path.exists(target_config_dir)) > > + self.assertTrue(os.access(os.path.join(qemu_bits_config_dir, > > + config_file), os.R_OK)) > > + shutil.copy2(os.path.join(qemu_bits_config_dir, config_file), > > + target_config_dir) > > + logging.info('copied config file %s to %s', > > + config_file, target_config_dir) > > + > > + def copy_test_scripts(self): > > + """copies the python test scripts into bits. """ > > + qemu_test_dir = os.path.join(os.getcwd(), 'bits-tests') > > + target_test_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer, > > + 'boot', 'python') > > + > > + self.assertTrue(os.path.exists(qemu_test_dir)) > > + self.assertTrue(os.path.exists(target_test_dir)) > > + > > + for filename in os.listdir(qemu_test_dir): > > + if os.path.isfile(os.path.join(qemu_test_dir, filename)) and \ > > + filename.endswith('.py'): > > + shutil.copy2(os.path.join(qemu_test_dir, filename), > > + target_test_dir) > > + logging.info('copied test file %s to %s', > > + filename, target_test_dir) > > + > > + # now remove the pyc test file if it exists, otherwise the > > + # changes in the python test script won't be executed. > > + testfile_pyc = os.path.splitext(filename)[0] + '.pyc' > > + if os.access(os.path.join(target_test_dir, testfile_pyc), > > + os.F_OK): > > + os.remove(os.path.join(target_test_dir, testfile_pyc)) > > + logging.info('removed compiled file %s', > > + os.path.join(target_test_dir, testfile_pyc)) > > + > > + def fix_mkrescue(self, mkrescue): > > + """ grub-mkrescue is a bash script with two variables, 'prefix' and > > + 'libdir'. They must be pointed to the right location so that the > > + iso can be generated appropriately. We point the two variables to > > + the directory where we have extracted our pre-built bits grub > > + tarball. > > + """ > > + grub_x86_64_mods = os.path.join(self._workDir, 'grub-inst-x86_64-efi') > > + grub_i386_mods = os.path.join(self._workDir, 'grub-inst') > > + > > + self.assertTrue(os.path.exists(grub_x86_64_mods)) > > + self.assertTrue(os.path.exists(grub_i386_mods)) > > + > > + new_script = "" > > + with open(mkrescue, 'r') as filehandle: > > + orig_script = filehandle.read() > > + new_script = re.sub('(^prefix=)(.*)', > > + r'\1"%s"' %grub_x86_64_mods, > > + orig_script, flags=re.M) > > + new_script = re.sub('(^libdir=)(.*)', r'\1"%s/lib"' %grub_i386_mods, > > + new_script, flags=re.M) > > + > > + with open(mkrescue, 'w') as filehandle: > > + filehandle.write(new_script) > > + > > + def generate_bits_iso(self): > > + """ Uses grub-mkrescue to generate a fresh bits iso with the python > > + test scripts > > + """ > > + bits_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer) > > + iso_file = os.path.join(self._workDir, 'bits-%d.iso' %self._bitsVer) > > + mkrescue_script = os.path.join(self._workDir, > > + 'grub-inst-x86_64-efi', 'bin', > > + 'grub-mkrescue') > > + > > + self.assertTrue(os.access(mkrescue_script, > > + os.R_OK | os.W_OK | os.X_OK)) > > + > > + self.fix_mkrescue(mkrescue_script) > > + > > + logging.info('calling grub-mkrescue to generate the biosbits iso ...') > > + > > + try: > > + if os.getenv('V'): > > + subprocess.check_call([mkrescue_script, '-o', > > + iso_file, bits_dir], > > + stdout=subprocess.DEVNULL) > > + else: > > + subprocess.check_call([mkrescue_script, '-o', > > + iso_file, bits_dir], > > + stderr=subprocess.DEVNULL, > > + stdout=subprocess.DEVNULL) > > + except Exception as e: # pylint: disable=broad-except > > + self.skipTest("Error while generating the bits iso. " > > + "Pass V=1 in the environment to get more details. " > > + + str(e)) > > + > > + self.assertTrue(os.access(iso_file, os.R_OK)) > > + > > + logging.info('iso file %s successfully generated.', iso_file) > > + > > + def setUp(self): > > + BITS_LOC = os.getenv("PYTEST_BITSLOC") > > + if BITS_LOC: > > + prefix = BITS_LOC > > + else: > > + prefix = os.path.join(os.getcwd(), 'prebuilt') > > + if not os.path.isdir(prefix): > > + os.mkdir(prefix, mode=0o775) > > + > > + bits_zip_file = os.path.join(prefix, 'bits-%d.zip' > > + %self._bitsVer) > > + grub_tar_file = os.path.join(prefix, > > + 'bits-%d-grub.tar.gz' %self._bitsVer) > > + # if the location of the bits binaries has been specified by the user > > + # and they are not found in that location, skip the test. > > + if BITS_LOC and not os.access(bits_zip_file, os.F_OK): > > + self.skipTest("test skipped since biosbits binaries " + > > + "could not be found in the specified location %s." \ > > + %BITS_LOC) > > + if BITS_LOC and not os.access(grub_tar_file, os.F_OK): > > + self.skipTest("test skipped since biosbits binaries " + > > + "could not be found in the specified location %s." \ > > + %BITS_LOC) > > + > > + self._workDir = tempfile.mkdtemp(prefix='acpi-bits-', > > + suffix='.tmp') > > + logging.info('working dir: %s', self._workDir) > > + > > + localArchive = "bits-%d.zip" % self._bitsVer > > + if not os.access(bits_zip_file, os.F_OK): > > + logging.info("archive %s not found in %s, downloading ...", > > + localArchive, bits_zip_file) > > + try: > > + req = request.urlopen(self._bitsLoc + localArchive) > > + with open(os.path.join(prefix, localArchive), > > + 'wb') as archivef: > > + archivef.write(req.read()) > > + except Exception as e: # pylint: disable=broad-except > > + self.skipTest("test skipped since biosbits binaries " + > > + "could not be obtained." + str(e)) > > + else: > > + logging.info('using locally found %s', localArchive) > > so you skip downlaod if it already exists locally. IIUC it is looking > in the CWD, which is presumably the directory the QEMU build is > performed in ? Yes, build/test/pytest/bits-test > So if dev cleans their build tree, the cache is lost ? Yes. > > Avocado has a more persistent cache outside the build tree IIUC. > So that is why I have the environment variable mechanism for passing to the test that will point to the location of the archives. The developer can download the files there and point to the test. Or I can change the scriprt accordingly if we know where we were downloading. We were discussing submodules and it was fiercely hated. So mst suggested another repo where to keep the binaries. My script that check out that repo somewhere outside the build directory and point the test to that location. I have kept several options open. We just need to make some decisions. > With regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-14 14:19 ` Ani Sinha @ 2022-07-14 17:49 ` Ani Sinha 2022-07-14 20:43 ` Michael S. Tsirkin 1 sibling, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-07-14 17:49 UTC (permalink / raw) To: Daniel P. Berrangé Cc: imammedo, jsnow, mst, pbonzini, peter.maydell, qemu-devel, thuth [-- Attachment #1: Type: text/plain, Size: 19318 bytes --] On Thu, Jul 14, 2022 at 19:49 Ani Sinha <ani@anisinha.ca> wrote: > > > On Thu, 14 Jul 2022, Daniel P. Berrangé wrote: > > > On Sun, Jul 10, 2022 at 10:30:10PM +0530, Ani Sinha wrote: > > > This change adds python based test environment that can be used to run > pytest > > > from within a virtual environment. A bash script sets up a virtual > environment > > > and then runs the python based tests from within that environment. > > > All dependent python packages are installed in the virtual environment > using > > > pip python module. QEMU python test modules are also available in the > environment > > > for spawning the QEMU based VMs. > > > > > > It also introduces QEMU acpi/smbios biosbits python test script which > is run > > > from within the python virtual environment. When the bios bits tests > are run, > > > bios bits binaries are downloaded from an external repo/location. > > > Currently, the test points to an external private github repo where > the bits > > > archives are checked in. > > > > > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > > > --- > > > tests/pytest/acpi-bits/acpi-bits-test-venv.sh | 59 +++ > > > tests/pytest/acpi-bits/acpi-bits-test.py | 382 ++++++++++++++++++ > > > tests/pytest/acpi-bits/meson.build | 33 ++ > > > tests/pytest/acpi-bits/requirements.txt | 1 + > > > 4 files changed, 475 insertions(+) > > > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test-venv.sh > > > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test.py > > > create mode 100644 tests/pytest/acpi-bits/meson.build > > > create mode 100644 tests/pytest/acpi-bits/requirements.txt > > > > > > diff --git a/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > > > new file mode 100644 > > > index 0000000000..186395473b > > > --- /dev/null > > > +++ b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > > > @@ -0,0 +1,59 @@ > > > +#!/usr/bin/env bash > > > +# Generates a python virtual environment for the test to run. > > > +# Then runs python test scripts from within that virtual environment. > > > +# > > > +# This program is free software; you can redistribute it and/or modify > > > +# it under the terms of the GNU General Public License as published by > > > +# the Free Software Foundation; either version 2 of the License, or > > > +# (at your option) any later version. > > > +# > > > +# This program is distributed in the hope that it will be useful, > > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > > +# GNU General Public License for more details. > > > +# > > > +# You should have received a copy of the GNU General Public License > > > +# along with this program. If not, see <http://www.gnu.org/licenses/ > >. > > > +# > > > +# Author: Ani Sinha <ani@anisinha.ca> > > > + > > > +set -e > > > + > > > +MYPATH=$(realpath ${BASH_SOURCE:-$0}) > > > +MYDIR=$(dirname $MYPATH) > > > + > > > +if [ -z "$PYTEST_SOURCE_ROOT" ]; then > > > + echo -n "Please set QTEST_SOURCE_ROOT env pointing" > > > + echo " to the root of the qemu source tree." > > > + echo -n "This is required so that the test can find the " > > > + echo "python modules that it needs for execution." > > > + exit 1 > > > +fi > > > +SRCDIR=$PYTEST_SOURCE_ROOT > > > +TESTSCRIPTS=("acpi-bits-test.py") > > > +PIPCMD="-m pip -q --disable-pip-version-check" > > > +# we need to save the old value of PWD before we do a change-dir later > > > +PYTEST_PWD=$PWD > > > + > > > +TESTS_PYTHON=/usr/bin/python3 > > > +TESTS_VENV_REQ=requirements.txt > > > + > > > +# sadly for pip -e and -t options do not work together. > > > +# please see https://github.com/pypa/pip/issues/562 > > > +cd $MYDIR > > > + > > > +$TESTS_PYTHON -m venv . > > > +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ > > > +[ -f $TESTS_VENV_REQ ] && \ > > > + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ || exit 0 > > > + > > > +# venv is activated at this point. > > > + > > > +# run the test > > > +for testscript in ${TESTSCRIPTS[@]} ; do > > > + export PYTEST_PWD; python3 $testscript > > > +done > > > + > > > +cd $PYTEST_PWD > > > + > > > +exit 0 > > > diff --git a/tests/pytest/acpi-bits/acpi-bits-test.py > b/tests/pytest/acpi-bits/acpi-bits-test.py > > > new file mode 100644 > > > index 0000000000..97e61eb709 > > > --- /dev/null > > > +++ b/tests/pytest/acpi-bits/acpi-bits-test.py > > > @@ -0,0 +1,382 @@ > > > +#!/usr/bin/env python3 > > > +# group: rw quick > > > +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits, > > > +# https://biosbits.org/ > > > +# > > > +# This program is free software; you can redistribute it and/or modify > > > +# it under the terms of the GNU General Public License as published by > > > +# the Free Software Foundation; either version 2 of the License, or > > > +# (at your option) any later version. > > > +# > > > +# This program is distributed in the hope that it will be useful, > > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > > +# GNU General Public License for more details. > > > +# > > > +# You should have received a copy of the GNU General Public License > > > +# along with this program. If not, see <http://www.gnu.org/licenses/ > >. > > > +# > > > +# Some parts are slightly taken from qtest.py and iotests.py > > > +# > > > +# Authors: > > > +# Ani Sinha <ani@anisinha.ca> > > > + > > > +# pylint: disable=invalid-name > > > + > > > +""" > > > +QEMU bios tests using biosbits available at > > > +https://biosbits.org/. > > > +""" > > > + > > > +import logging > > > +import os > > > +import re > > > +import shutil > > > +import subprocess > > > +import sys > > > +import tarfile > > > +import tempfile > > > +import time > > > +import unittest > > > +from urllib import request > > > +import zipfile > > > +from typing import ( > > > + List, > > > + Optional, > > > + Sequence, > > > +) > > > +from tap import TAPTestRunner > > > +from qemu.machine import QEMUMachine > > > + > > > +PYTESTQEMUBIN = os.getenv('PYTEST_QEMU_BINARY') > > > +PYTEST_PWD = os.getenv('PYTEST_PWD') > > > + > > > +def get_arch(): > > > + """finds the arch from the qemu binary name""" > > > + match = re.search('.*qemu-system-(.*)', PYTESTQEMUBIN) > > > + if match: > > > + return match.group(1) > > > + return 'x86_64' > > > + > > > +ARCH = get_arch() > > > + > > > +class QEMUBitsMachine(QEMUMachine): > > > + """ > > > + A QEMU VM, with isa-debugcon enabled and bits iso passed > > > + using -cdrom to QEMU commandline. > > > + """ > > > + def __init__(self, > > > + binary: str, > > > + args: Sequence[str] = (), > > > + wrapper: Sequence[str] = (), > > > + name: Optional[str] = None, > > > + base_temp_dir: str = "/var/tmp", > > > + debugcon_log: str = "debugcon-log.txt", > > > + debugcon_addr: str = "0x403", > > > + sock_dir: Optional[str] = None, > > > + qmp_timer: Optional[float] = None): > > > + # pylint: disable=too-many-arguments > > > + > > > + if name is None: > > > + name = "qemu-bits-%d" % os.getpid() > > > + if sock_dir is None: > > > + sock_dir = base_temp_dir > > > + super().__init__(binary, args, wrapper=wrapper, name=name, > > > + base_temp_dir=base_temp_dir, > > > + sock_dir=sock_dir, qmp_timer=qmp_timer) > > > + self.debugcon_log = debugcon_log > > > + self.debugcon_addr = debugcon_addr > > > + self.base_temp_dir = base_temp_dir > > > + > > > + @property > > > + def _base_args(self) -> List[str]: > > > + args = super()._base_args > > > + args.extend([ > > > + '-chardev', > > > + 'file,path=%s,id=debugcon' > %os.path.join(self.base_temp_dir, > > > + > self.debugcon_log), > > > + '-device', > > > + 'isa-debugcon,iobase=%s,chardev=debugcon' > %self.debugcon_addr, > > > + ]) > > > + return args > > > + > > > + def base_args(self): > > > + """return the base argument to QEMU binary""" > > > + return self._base_args > > > + > > > +class AcpiBitsTest(unittest.TestCase): > > > + """ACPI and SMBIOS tests using biosbits.""" > > > + def __init__(self, *args, **kwargs): > > > + super().__init__(*args, **kwargs) > > > + self._vm = None > > > + self._workDir = None > > > + self._bitsVer = 2100 > > > + self._bitsLoc = " > https://github.com/ani-sinha/bits/raw/bits-builds/" > > > > This URL location gives a 404 - was it supposed to be poiinting to the > > bits-builds branch ? eg to form a URL like: > > > > You are looking at a partial URL. The code completes the URL based on the > version we are using. Append to this URL the archive name, like > bits-2100.zip or something. > > eg, https://github.com/ani-sinha/bits/raw/bits-builds/bits-2100.zip > > > > https://github.com/ani-sinha/bits/blob/bits-builds/bits-2100.zip?raw=true > > > > > + self._debugcon_addr = '0x403' > > > + self._debugcon_log = 'debugcon-log.txt' > > > + logging.basicConfig(level=logging.INFO) > > > + > > > + def copy_bits_config(self): > > > + """ copies the bios bits config file into bits. > > > + """ > > > + config_file = 'bits-cfg.txt' > > > + qemu_bits_config_dir = os.path.join(os.getcwd(), > 'bits-config') > > > + target_config_dir = os.path.join(self._workDir, > > > + 'bits-%d' %self._bitsVer, > 'boot') > > > + self.assertTrue(os.path.exists(qemu_bits_config_dir)) > > > + self.assertTrue(os.path.exists(target_config_dir)) > > > + self.assertTrue(os.access(os.path.join(qemu_bits_config_dir, > > > + config_file), os.R_OK)) > > > + shutil.copy2(os.path.join(qemu_bits_config_dir, config_file), > > > + target_config_dir) > > > + logging.info('copied config file %s to %s', > > > + config_file, target_config_dir) > > > + > > > + def copy_test_scripts(self): > > > + """copies the python test scripts into bits. """ > > > + qemu_test_dir = os.path.join(os.getcwd(), 'bits-tests') > > > + target_test_dir = os.path.join(self._workDir, 'bits-%d' > %self._bitsVer, > > > + 'boot', 'python') > > > + > > > + self.assertTrue(os.path.exists(qemu_test_dir)) > > > + self.assertTrue(os.path.exists(target_test_dir)) > > > + > > > + for filename in os.listdir(qemu_test_dir): > > > + if os.path.isfile(os.path.join(qemu_test_dir, filename)) > and \ > > > + filename.endswith('.py'): > > > + shutil.copy2(os.path.join(qemu_test_dir, filename), > > > + target_test_dir) > > > + logging.info('copied test file %s to %s', > > > + filename, target_test_dir) > > > + > > > + # now remove the pyc test file if it exists, > otherwise the > > > + # changes in the python test script won't be executed. > > > + testfile_pyc = os.path.splitext(filename)[0] + '.pyc' > > > + if os.access(os.path.join(target_test_dir, > testfile_pyc), > > > + os.F_OK): > > > + os.remove(os.path.join(target_test_dir, > testfile_pyc)) > > > + logging.info('removed compiled file %s', > > > + os.path.join(target_test_dir, > testfile_pyc)) > > > + > > > + def fix_mkrescue(self, mkrescue): > > > + """ grub-mkrescue is a bash script with two variables, > 'prefix' and > > > + 'libdir'. They must be pointed to the right location so > that the > > > + iso can be generated appropriately. We point the two > variables to > > > + the directory where we have extracted our pre-built bits > grub > > > + tarball. > > > + """ > > > + grub_x86_64_mods = os.path.join(self._workDir, > 'grub-inst-x86_64-efi') > > > + grub_i386_mods = os.path.join(self._workDir, 'grub-inst') > > > + > > > + self.assertTrue(os.path.exists(grub_x86_64_mods)) > > > + self.assertTrue(os.path.exists(grub_i386_mods)) > > > + > > > + new_script = "" > > > + with open(mkrescue, 'r') as filehandle: > > > + orig_script = filehandle.read() > > > + new_script = re.sub('(^prefix=)(.*)', > > > + r'\1"%s"' %grub_x86_64_mods, > > > + orig_script, flags=re.M) > > > + new_script = re.sub('(^libdir=)(.*)', r'\1"%s/lib"' > %grub_i386_mods, > > > + new_script, flags=re.M) > > > + > > > + with open(mkrescue, 'w') as filehandle: > > > + filehandle.write(new_script) > > > + > > > + def generate_bits_iso(self): > > > + """ Uses grub-mkrescue to generate a fresh bits iso with the > python > > > + test scripts > > > + """ > > > + bits_dir = os.path.join(self._workDir, 'bits-%d' > %self._bitsVer) > > > + iso_file = os.path.join(self._workDir, 'bits-%d.iso' > %self._bitsVer) > > > + mkrescue_script = os.path.join(self._workDir, > > > + 'grub-inst-x86_64-efi', 'bin', > > > + 'grub-mkrescue') > > > + > > > + self.assertTrue(os.access(mkrescue_script, > > > + os.R_OK | os.W_OK | os.X_OK)) > > > + > > > + self.fix_mkrescue(mkrescue_script) > > > + > > > + logging.info('calling grub-mkrescue to generate the biosbits > iso ...') > > > + > > > + try: > > > + if os.getenv('V'): > > > + subprocess.check_call([mkrescue_script, '-o', > > > + iso_file, bits_dir], > > > + stdout=subprocess.DEVNULL) > > > + else: > > > + subprocess.check_call([mkrescue_script, '-o', > > > + iso_file, bits_dir], > > > + stderr=subprocess.DEVNULL, > > > + stdout=subprocess.DEVNULL) > > > + except Exception as e: # pylint: disable=broad-except > > > + self.skipTest("Error while generating the bits iso. " > > > + "Pass V=1 in the environment to get more > details. " > > > + + str(e)) > > > + > > > + self.assertTrue(os.access(iso_file, os.R_OK)) > > > + > > > + logging.info('iso file %s successfully generated.', iso_file) > > > + > > > + def setUp(self): > > > + BITS_LOC = os.getenv("PYTEST_BITSLOC") > > > + if BITS_LOC: > > > + prefix = BITS_LOC > > > + else: > > > + prefix = os.path.join(os.getcwd(), 'prebuilt') > > > + if not os.path.isdir(prefix): > > > + os.mkdir(prefix, mode=0o775) > > > + > > > + bits_zip_file = os.path.join(prefix, 'bits-%d.zip' > > > + %self._bitsVer) > > > + grub_tar_file = os.path.join(prefix, > > > + 'bits-%d-grub.tar.gz' > %self._bitsVer) > > > + # if the location of the bits binaries has been specified by > the user > > > + # and they are not found in that location, skip the test. > > > + if BITS_LOC and not os.access(bits_zip_file, os.F_OK): > > > + self.skipTest("test skipped since biosbits binaries " + > > > + "could not be found in the specified > location %s." \ > > > + %BITS_LOC) > > > + if BITS_LOC and not os.access(grub_tar_file, os.F_OK): > > > + self.skipTest("test skipped since biosbits binaries " + > > > + "could not be found in the specified > location %s." \ > > > + %BITS_LOC) > > > + > > > + self._workDir = tempfile.mkdtemp(prefix='acpi-bits-', > > > + suffix='.tmp') > > > + logging.info('working dir: %s', self._workDir) > > > + > > > + localArchive = "bits-%d.zip" % self._bitsVer > > > + if not os.access(bits_zip_file, os.F_OK): > > > + logging.info("archive %s not found in %s, downloading > ...", > > > + localArchive, bits_zip_file) > > > + try: > > > + req = request.urlopen(self._bitsLoc + localArchive) > > > + with open(os.path.join(prefix, localArchive), > > > + 'wb') as archivef: > > > + archivef.write(req.read()) > > > + except Exception as e: # pylint: disable=broad-except > > > + self.skipTest("test skipped since biosbits binaries " > + > > > + "could not be obtained." + str(e)) > > > + else: > > > + logging.info('using locally found %s', localArchive) > > > > so you skip downlaod if it already exists locally. IIUC it is looking > > in the CWD, which is presumably the directory the QEMU build is > > performed in ? > > Yes, build/test/pytest/bits-test > > > So if dev cleans their build tree, the cache is lost ? > > Yes. > > > > > Avocado has a more persistent cache outside the build tree IIUC. To be honest I’m not sure if I like that. If I clear out my build directory I would want all build related artifacts to go away, including downloaded build stuff from the cache. > > > > So that is why I have the environment variable mechanism for passing to > the test that will point to the location of the archives. The developer > can download the files there and point to the test. Or I can change the > scriprt accordingly if we know where we were downloading. We were > discussing submodules and it was fiercely hated. So mst suggested another > repo where to keep the binaries. My script that check out that repo > somewhere outside the build directory and point the test to that location. > > I have kept several options open. We just need to make some decisions. > > > > With regards, > > Daniel > > -- > > |: https://berrange.com -o- > https://www.flickr.com/photos/dberrange :| > > |: https://libvirt.org -o- > https://fstop138.berrange.com :| > > |: https://entangle-photo.org -o- > https://www.instagram.com/dberrange :| > > > > [-- Attachment #2: Type: text/html, Size: 27090 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-14 14:19 ` Ani Sinha 2022-07-14 17:49 ` Ani Sinha @ 2022-07-14 20:43 ` Michael S. Tsirkin 1 sibling, 0 replies; 90+ messages in thread From: Michael S. Tsirkin @ 2022-07-14 20:43 UTC (permalink / raw) To: Ani Sinha Cc: Daniel P. Berrangé, qemu-devel, thuth, peter.maydell, jsnow, pbonzini, imammedo On Thu, Jul 14, 2022 at 07:49:36PM +0530, Ani Sinha wrote: > > so you skip downlaod if it already exists locally. IIUC it is looking > > in the CWD, which is presumably the directory the QEMU build is > > performed in ? > > Yes, build/test/pytest/bits-test > > > So if dev cleans their build tree, the cache is lost ? > > Yes. > > > > > Avocado has a more persistent cache outside the build tree IIUC. > > > > So that is why I have the environment variable mechanism for passing to > the test that will point to the location of the archives. The developer > can download the files there and point to the test. Or I can change the > scriprt accordingly if we know where we were downloading. We were > discussing submodules and it was fiercely hated. So mst suggested another > repo where to keep the binaries. My script that check out that repo > somewhere outside the build directory and point the test to that location. > > I have kept several options open. We just need to make some decisions. > For now I would probably just have a script to fetch into source tree. -- MST ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-10 17:00 ` [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits Ani Sinha 2022-07-11 20:42 ` John Snow 2022-07-14 14:05 ` Daniel P. Berrangé @ 2022-07-14 20:41 ` Michael S. Tsirkin 2022-07-15 4:17 ` Ani Sinha 2022-09-27 21:44 ` Paolo Bonzini 3 siblings, 1 reply; 90+ messages in thread From: Michael S. Tsirkin @ 2022-07-14 20:41 UTC (permalink / raw) To: Ani Sinha Cc: qemu-devel, thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo On Sun, Jul 10, 2022 at 10:30:10PM +0530, Ani Sinha wrote: > This change adds python based test environment that can be used to run pytest > from within a virtual environment. A bash script sets up a virtual environment > and then runs the python based tests from within that environment. > All dependent python packages are installed in the virtual environment using > pip python module. QEMU python test modules are also available in the environment > for spawning the QEMU based VMs. > > It also introduces QEMU acpi/smbios biosbits python test script which is run > from within the python virtual environment. When the bios bits tests are run, > bios bits binaries are downloaded from an external repo/location. > Currently, the test points to an external private github repo where the bits > archives are checked in. > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > --- > tests/pytest/acpi-bits/acpi-bits-test-venv.sh | 59 +++ > tests/pytest/acpi-bits/acpi-bits-test.py | 382 ++++++++++++++++++ > tests/pytest/acpi-bits/meson.build | 33 ++ > tests/pytest/acpi-bits/requirements.txt | 1 + > 4 files changed, 475 insertions(+) > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test-venv.sh > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test.py > create mode 100644 tests/pytest/acpi-bits/meson.build > create mode 100644 tests/pytest/acpi-bits/requirements.txt > > diff --git a/tests/pytest/acpi-bits/acpi-bits-test-venv.sh b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > new file mode 100644 > index 0000000000..186395473b > --- /dev/null > +++ b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > @@ -0,0 +1,59 @@ > +#!/usr/bin/env bash > +# Generates a python virtual environment for the test to run. > +# Then runs python test scripts from within that virtual environment. > +# > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; either version 2 of the License, or > +# (at your option) any later version. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > +# > +# Author: Ani Sinha <ani@anisinha.ca> > + > +set -e > + > +MYPATH=$(realpath ${BASH_SOURCE:-$0}) > +MYDIR=$(dirname $MYPATH) > + > +if [ -z "$PYTEST_SOURCE_ROOT" ]; then > + echo -n "Please set QTEST_SOURCE_ROOT env pointing" > + echo " to the root of the qemu source tree." > + echo -n "This is required so that the test can find the " > + echo "python modules that it needs for execution." > + exit 1 > +fi > +SRCDIR=$PYTEST_SOURCE_ROOT > +TESTSCRIPTS=("acpi-bits-test.py") > +PIPCMD="-m pip -q --disable-pip-version-check" > +# we need to save the old value of PWD before we do a change-dir later > +PYTEST_PWD=$PWD > + > +TESTS_PYTHON=/usr/bin/python3 > +TESTS_VENV_REQ=requirements.txt > + > +# sadly for pip -e and -t options do not work together. > +# please see https://github.com/pypa/pip/issues/562 > +cd $MYDIR > + > +$TESTS_PYTHON -m venv . > +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ > +[ -f $TESTS_VENV_REQ ] && \ > + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ || exit 0 > + > +# venv is activated at this point. > + > +# run the test > +for testscript in ${TESTSCRIPTS[@]} ; do > + export PYTEST_PWD; python3 $testscript > +done > + > +cd $PYTEST_PWD > + > +exit 0 > diff --git a/tests/pytest/acpi-bits/acpi-bits-test.py b/tests/pytest/acpi-bits/acpi-bits-test.py > new file mode 100644 > index 0000000000..97e61eb709 > --- /dev/null > +++ b/tests/pytest/acpi-bits/acpi-bits-test.py > @@ -0,0 +1,382 @@ > +#!/usr/bin/env python3 > +# group: rw quick > +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits, > +# https://biosbits.org/ > +# > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; either version 2 of the License, or > +# (at your option) any later version. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > +# > +# Some parts are slightly taken from qtest.py and iotests.py > +# > +# Authors: > +# Ani Sinha <ani@anisinha.ca> > + > +# pylint: disable=invalid-name > + > +""" > +QEMU bios tests using biosbits available at > +https://biosbits.org/. > +""" > + > +import logging > +import os > +import re > +import shutil > +import subprocess > +import sys > +import tarfile > +import tempfile > +import time > +import unittest > +from urllib import request > +import zipfile > +from typing import ( > + List, > + Optional, > + Sequence, > +) > +from tap import TAPTestRunner > +from qemu.machine import QEMUMachine > + > +PYTESTQEMUBIN = os.getenv('PYTEST_QEMU_BINARY') > +PYTEST_PWD = os.getenv('PYTEST_PWD') > + > +def get_arch(): > + """finds the arch from the qemu binary name""" > + match = re.search('.*qemu-system-(.*)', PYTESTQEMUBIN) > + if match: > + return match.group(1) > + return 'x86_64' > + > +ARCH = get_arch() > + > +class QEMUBitsMachine(QEMUMachine): > + """ > + A QEMU VM, with isa-debugcon enabled and bits iso passed > + using -cdrom to QEMU commandline. > + """ > + def __init__(self, > + binary: str, > + args: Sequence[str] = (), > + wrapper: Sequence[str] = (), > + name: Optional[str] = None, > + base_temp_dir: str = "/var/tmp", > + debugcon_log: str = "debugcon-log.txt", > + debugcon_addr: str = "0x403", > + sock_dir: Optional[str] = None, > + qmp_timer: Optional[float] = None): > + # pylint: disable=too-many-arguments > + > + if name is None: > + name = "qemu-bits-%d" % os.getpid() > + if sock_dir is None: > + sock_dir = base_temp_dir > + super().__init__(binary, args, wrapper=wrapper, name=name, > + base_temp_dir=base_temp_dir, > + sock_dir=sock_dir, qmp_timer=qmp_timer) > + self.debugcon_log = debugcon_log > + self.debugcon_addr = debugcon_addr > + self.base_temp_dir = base_temp_dir > + > + @property > + def _base_args(self) -> List[str]: > + args = super()._base_args > + args.extend([ > + '-chardev', > + 'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir, > + self.debugcon_log), > + '-device', > + 'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr, > + ]) > + return args > + > + def base_args(self): > + """return the base argument to QEMU binary""" > + return self._base_args > + > +class AcpiBitsTest(unittest.TestCase): > + """ACPI and SMBIOS tests using biosbits.""" > + def __init__(self, *args, **kwargs): > + super().__init__(*args, **kwargs) > + self._vm = None > + self._workDir = None > + self._bitsVer = 2100 > + self._bitsLoc = "https://github.com/ani-sinha/bits/raw/bits-builds/" > + self._debugcon_addr = '0x403' > + self._debugcon_log = 'debugcon-log.txt' > + logging.basicConfig(level=logging.INFO) > + > + def copy_bits_config(self): > + """ copies the bios bits config file into bits. > + """ > + config_file = 'bits-cfg.txt' > + qemu_bits_config_dir = os.path.join(os.getcwd(), 'bits-config') > + target_config_dir = os.path.join(self._workDir, > + 'bits-%d' %self._bitsVer, 'boot') > + self.assertTrue(os.path.exists(qemu_bits_config_dir)) > + self.assertTrue(os.path.exists(target_config_dir)) > + self.assertTrue(os.access(os.path.join(qemu_bits_config_dir, > + config_file), os.R_OK)) > + shutil.copy2(os.path.join(qemu_bits_config_dir, config_file), > + target_config_dir) > + logging.info('copied config file %s to %s', > + config_file, target_config_dir) > + > + def copy_test_scripts(self): > + """copies the python test scripts into bits. """ > + qemu_test_dir = os.path.join(os.getcwd(), 'bits-tests') > + target_test_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer, > + 'boot', 'python') > + > + self.assertTrue(os.path.exists(qemu_test_dir)) > + self.assertTrue(os.path.exists(target_test_dir)) > + > + for filename in os.listdir(qemu_test_dir): > + if os.path.isfile(os.path.join(qemu_test_dir, filename)) and \ > + filename.endswith('.py'): > + shutil.copy2(os.path.join(qemu_test_dir, filename), > + target_test_dir) > + logging.info('copied test file %s to %s', > + filename, target_test_dir) > + > + # now remove the pyc test file if it exists, otherwise the > + # changes in the python test script won't be executed. > + testfile_pyc = os.path.splitext(filename)[0] + '.pyc' > + if os.access(os.path.join(target_test_dir, testfile_pyc), > + os.F_OK): > + os.remove(os.path.join(target_test_dir, testfile_pyc)) > + logging.info('removed compiled file %s', > + os.path.join(target_test_dir, testfile_pyc)) > + > + def fix_mkrescue(self, mkrescue): > + """ grub-mkrescue is a bash script with two variables, 'prefix' and > + 'libdir'. They must be pointed to the right location so that the > + iso can be generated appropriately. We point the two variables to > + the directory where we have extracted our pre-built bits grub > + tarball. > + """ > + grub_x86_64_mods = os.path.join(self._workDir, 'grub-inst-x86_64-efi') > + grub_i386_mods = os.path.join(self._workDir, 'grub-inst') > + > + self.assertTrue(os.path.exists(grub_x86_64_mods)) > + self.assertTrue(os.path.exists(grub_i386_mods)) > + > + new_script = "" > + with open(mkrescue, 'r') as filehandle: > + orig_script = filehandle.read() > + new_script = re.sub('(^prefix=)(.*)', > + r'\1"%s"' %grub_x86_64_mods, > + orig_script, flags=re.M) > + new_script = re.sub('(^libdir=)(.*)', r'\1"%s/lib"' %grub_i386_mods, > + new_script, flags=re.M) > + > + with open(mkrescue, 'w') as filehandle: > + filehandle.write(new_script) > + > + def generate_bits_iso(self): > + """ Uses grub-mkrescue to generate a fresh bits iso with the python > + test scripts > + """ > + bits_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer) > + iso_file = os.path.join(self._workDir, 'bits-%d.iso' %self._bitsVer) > + mkrescue_script = os.path.join(self._workDir, > + 'grub-inst-x86_64-efi', 'bin', > + 'grub-mkrescue') > + > + self.assertTrue(os.access(mkrescue_script, > + os.R_OK | os.W_OK | os.X_OK)) > + > + self.fix_mkrescue(mkrescue_script) > + > + logging.info('calling grub-mkrescue to generate the biosbits iso ...') > + > + try: > + if os.getenv('V'): > + subprocess.check_call([mkrescue_script, '-o', > + iso_file, bits_dir], > + stdout=subprocess.DEVNULL) > + else: > + subprocess.check_call([mkrescue_script, '-o', > + iso_file, bits_dir], > + stderr=subprocess.DEVNULL, > + stdout=subprocess.DEVNULL) > + except Exception as e: # pylint: disable=broad-except > + self.skipTest("Error while generating the bits iso. " > + "Pass V=1 in the environment to get more details. " > + + str(e)) > + > + self.assertTrue(os.access(iso_file, os.R_OK)) > + > + logging.info('iso file %s successfully generated.', iso_file) > + > + def setUp(self): > + BITS_LOC = os.getenv("PYTEST_BITSLOC") > + if BITS_LOC: > + prefix = BITS_LOC > + else: > + prefix = os.path.join(os.getcwd(), 'prebuilt') > + if not os.path.isdir(prefix): > + os.mkdir(prefix, mode=0o775) > + > + bits_zip_file = os.path.join(prefix, 'bits-%d.zip' > + %self._bitsVer) > + grub_tar_file = os.path.join(prefix, > + 'bits-%d-grub.tar.gz' %self._bitsVer) > + # if the location of the bits binaries has been specified by the user > + # and they are not found in that location, skip the test. > + if BITS_LOC and not os.access(bits_zip_file, os.F_OK): > + self.skipTest("test skipped since biosbits binaries " + > + "could not be found in the specified location %s." \ > + %BITS_LOC) > + if BITS_LOC and not os.access(grub_tar_file, os.F_OK): > + self.skipTest("test skipped since biosbits binaries " + > + "could not be found in the specified location %s." \ > + %BITS_LOC) > + > + self._workDir = tempfile.mkdtemp(prefix='acpi-bits-', > + suffix='.tmp') > + logging.info('working dir: %s', self._workDir) > + > + localArchive = "bits-%d.zip" % self._bitsVer > + if not os.access(bits_zip_file, os.F_OK): > + logging.info("archive %s not found in %s, downloading ...", > + localArchive, bits_zip_file) > + try: > + req = request.urlopen(self._bitsLoc + localArchive) > + with open(os.path.join(prefix, localArchive), > + 'wb') as archivef: > + archivef.write(req.read()) > + except Exception as e: # pylint: disable=broad-except > + self.skipTest("test skipped since biosbits binaries " + > + "could not be obtained." + str(e)) > + else: > + logging.info('using locally found %s', localArchive) Poking at URLs from python is really gross IMHO. Instead of all this mess, can't we just spawn e.g. "git clone --depth 1"? And if the directory exists I would fetch and checkout. Fundamentally people hate submodules so we are recreating them ok, but at least we can use git and not open-code http and file management ... > + > + localArchive = "bits-%d-grub.tar.gz" % self._bitsVer > + if not os.access(grub_tar_file, os.F_OK): > + logging.info("archive %s not found in %s, downloading ...", > + localArchive, bits_zip_file) > + try: > + req = request.urlopen(self._bitsLoc + localArchive) > + with open(os.path.join(prefix, localArchive), > + 'wb') as archivef: > + archivef.write(req.read()) > + except Exception as e: # pylint: disable=broad-except > + self.skipTest("test skipped since biosbits binaries " + > + "could not be obtained." + str(e)) > + else: > + logging.info('using locally found %s', localArchive) > + > + # extract the bits software in the temp working directory > + with zipfile.ZipFile(bits_zip_file, 'r') as zref: > + zref.extractall(self._workDir) > + > + with tarfile.open(grub_tar_file, 'r') as tarball: > + tarball.extractall(self._workDir) > + > + self.copy_test_scripts() > + self.copy_bits_config() > + self.generate_bits_iso() > + > + def parse_log(self): > + """parse the log generated by running bits tests and > + check for failures. > + """ > + debugconf = os.path.join(self._workDir, self._debugcon_log) > + log = "" > + with open(debugconf, 'r') as filehandle: > + log = filehandle.read() > + > + if os.getenv('V'): > + print('\nlogs from biosbits follows:') > + print('==========================================\n') > + print(log) > + print('==========================================\n') > + > + matchiter = re.finditer(r'(.*Summary: )(\d+ passed), (\d+ failed).*', > + log) > + for match in matchiter: > + # verify that no test cases failed. > + self.assertEqual(match.group(3).split()[0], '0', > + 'Some bits tests seems to have failed. ' \ > + 'Set V=1 in the environment to get the entire ' \ > + 'log from bits.') > + > + def tearDown(self): > + if self._vm: > + self.assertFalse(not self._vm.is_running) > + logging.info('removing the work directory %s', self._workDir) > + shutil.rmtree(self._workDir) > + > + def test_acpi_smbios_bits(self): > + """The main test case implementaion.""" > + > + qemu_bin = PYTESTQEMUBIN > + iso_file = os.path.join(self._workDir, 'bits-%d.iso' %self._bitsVer) > + > + # PYTESTQEMUBIN could be relative to the current directory > + if not os.access(PYTESTQEMUBIN, os.X_OK) and PYTEST_PWD: > + qemu_bin = os.path.join(PYTEST_PWD, PYTESTQEMUBIN) > + > + logging.info('QEMU binary used: %s', qemu_bin) > + > + self.assertTrue(os.access(qemu_bin, os.X_OK)) > + self.assertTrue(os.access(iso_file, os.R_OK)) > + > + self._vm = QEMUBitsMachine(binary=qemu_bin, > + base_temp_dir=self._workDir, > + debugcon_log=self._debugcon_log, > + debugcon_addr=self._debugcon_addr) > + > + self._vm.add_args('-cdrom', '%s' %iso_file) > + > + args = " ".join(str(arg) for arg in self._vm.base_args()) + \ > + " " + " ".join(str(arg) for arg in self._vm.args) > + > + logging.info("launching QEMU vm with the following arguments: %s", > + args) > + > + self._vm.launch() > + # biosbits has been configured to run all the specified test suites > + # in batch mode and then automatically initiate a vm shutdown. > + # sleep for maximum of one minute > + max_sleep_time = time.monotonic() + 60 > + while self._vm.is_running() and time.monotonic() < max_sleep_time: > + time.sleep(1) > + > + self.assertFalse(time.monotonic() > max_sleep_time, > + 'The VM seems to have failed to shutdown in time') > + > + self.parse_log() > + > +def execute_unittest(argv: List[str], debug: bool = False, > + runner: TAPTestRunner = None) -> None: > + """Executes unittests within the calling module.""" > + > + unittest.main(argv=argv, > + testRunner=runner, > + verbosity=2 if debug else 1, > + warnings=None if sys.warnoptions else 'ignore') > + > +def main(): > + """ The main function where execution begins. """ > + > + assert PYTESTQEMUBIN is not None, \ > + "Environment variable PYTEST_QEMU_BINARY required." > + > + runner = TAPTestRunner() > + runner.set_stream(True) > + runner.set_format("%s/acpi-bits-test" %ARCH) > + execute_unittest(sys.argv, False, runner) > + > +main() > diff --git a/tests/pytest/acpi-bits/meson.build b/tests/pytest/acpi-bits/meson.build > new file mode 100644 > index 0000000000..099c191d57 > --- /dev/null > +++ b/tests/pytest/acpi-bits/meson.build > @@ -0,0 +1,33 @@ > +xorriso = find_program('xorriso', required: true) > +if not xorriso.found() > + message('xorriso not found ... disabled bits acpi tests.') > + subdir_done() > +endif > + > +subdir('bits-tests') > +subdir('bits-config') > + > +test_files = ['acpi-bits-test.py'] > +requirements = 'requirements.txt' > + > +copytestfiles = custom_target('copy test files', > + input : test_files, > + output : test_files, > + command : ['cp', '@INPUT@', '@OUTPUT@'], > + install : false, > + build_by_default : true) > + > +requirementsfiles = custom_target('copy py req files', > + input : requirements, > + output : requirements, > + command : ['cp', '@INPUT@', '@OUTPUT@'], > + install : false, > + build_by_default : true) > + > +other_deps += [copytestfiles,requirementsfiles] > + > +pytest_executables += { > + 'acpi-bits-test': configure_file(copy:true, > + input:'acpi-bits-test-venv.sh', > + output:'acpi-bits-test') > +} > diff --git a/tests/pytest/acpi-bits/requirements.txt b/tests/pytest/acpi-bits/requirements.txt > new file mode 100644 > index 0000000000..00cdad09ef > --- /dev/null > +++ b/tests/pytest/acpi-bits/requirements.txt > @@ -0,0 +1 @@ > +tap.py > -- > 2.25.1 ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-14 20:41 ` Michael S. Tsirkin @ 2022-07-15 4:17 ` Ani Sinha 2022-07-15 6:50 ` Michael S. Tsirkin 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-07-15 4:17 UTC (permalink / raw) To: Michael S. Tsirkin Cc: Ani Sinha, qemu-devel, thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo On Thu, 14 Jul 2022, Michael S. Tsirkin wrote: > On Sun, Jul 10, 2022 at 10:30:10PM +0530, Ani Sinha wrote: > > This change adds python based test environment that can be used to run pytest > > from within a virtual environment. A bash script sets up a virtual environment > > and then runs the python based tests from within that environment. > > All dependent python packages are installed in the virtual environment using > > pip python module. QEMU python test modules are also available in the environment > > for spawning the QEMU based VMs. > > > > It also introduces QEMU acpi/smbios biosbits python test script which is run > > from within the python virtual environment. When the bios bits tests are run, > > bios bits binaries are downloaded from an external repo/location. > > Currently, the test points to an external private github repo where the bits > > archives are checked in. > > > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > > --- > > tests/pytest/acpi-bits/acpi-bits-test-venv.sh | 59 +++ > > tests/pytest/acpi-bits/acpi-bits-test.py | 382 ++++++++++++++++++ > > tests/pytest/acpi-bits/meson.build | 33 ++ > > tests/pytest/acpi-bits/requirements.txt | 1 + > > 4 files changed, 475 insertions(+) > > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test-venv.sh > > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test.py > > create mode 100644 tests/pytest/acpi-bits/meson.build > > create mode 100644 tests/pytest/acpi-bits/requirements.txt > > > > diff --git a/tests/pytest/acpi-bits/acpi-bits-test-venv.sh b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > > new file mode 100644 > > index 0000000000..186395473b > > --- /dev/null > > +++ b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > > @@ -0,0 +1,59 @@ > > +#!/usr/bin/env bash > > +# Generates a python virtual environment for the test to run. > > +# Then runs python test scripts from within that virtual environment. > > +# > > +# This program is free software; you can redistribute it and/or modify > > +# it under the terms of the GNU General Public License as published by > > +# the Free Software Foundation; either version 2 of the License, or > > +# (at your option) any later version. > > +# > > +# This program is distributed in the hope that it will be useful, > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +# GNU General Public License for more details. > > +# > > +# You should have received a copy of the GNU General Public License > > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > > +# > > +# Author: Ani Sinha <ani@anisinha.ca> > > + > > +set -e > > + > > +MYPATH=$(realpath ${BASH_SOURCE:-$0}) > > +MYDIR=$(dirname $MYPATH) > > + > > +if [ -z "$PYTEST_SOURCE_ROOT" ]; then > > + echo -n "Please set QTEST_SOURCE_ROOT env pointing" > > + echo " to the root of the qemu source tree." > > + echo -n "This is required so that the test can find the " > > + echo "python modules that it needs for execution." > > + exit 1 > > +fi > > +SRCDIR=$PYTEST_SOURCE_ROOT > > +TESTSCRIPTS=("acpi-bits-test.py") > > +PIPCMD="-m pip -q --disable-pip-version-check" > > +# we need to save the old value of PWD before we do a change-dir later > > +PYTEST_PWD=$PWD > > + > > +TESTS_PYTHON=/usr/bin/python3 > > +TESTS_VENV_REQ=requirements.txt > > + > > +# sadly for pip -e and -t options do not work together. > > +# please see https://github.com/pypa/pip/issues/562 > > +cd $MYDIR > > + > > +$TESTS_PYTHON -m venv . > > +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ > > +[ -f $TESTS_VENV_REQ ] && \ > > + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ || exit 0 > > + > > +# venv is activated at this point. > > + > > +# run the test > > +for testscript in ${TESTSCRIPTS[@]} ; do > > + export PYTEST_PWD; python3 $testscript > > +done > > + > > +cd $PYTEST_PWD > > + > > +exit 0 > > diff --git a/tests/pytest/acpi-bits/acpi-bits-test.py b/tests/pytest/acpi-bits/acpi-bits-test.py > > new file mode 100644 > > index 0000000000..97e61eb709 > > --- /dev/null > > +++ b/tests/pytest/acpi-bits/acpi-bits-test.py > > @@ -0,0 +1,382 @@ > > +#!/usr/bin/env python3 > > +# group: rw quick > > +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits, > > +# https://biosbits.org/ > > +# > > +# This program is free software; you can redistribute it and/or modify > > +# it under the terms of the GNU General Public License as published by > > +# the Free Software Foundation; either version 2 of the License, or > > +# (at your option) any later version. > > +# > > +# This program is distributed in the hope that it will be useful, > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +# GNU General Public License for more details. > > +# > > +# You should have received a copy of the GNU General Public License > > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > > +# > > +# Some parts are slightly taken from qtest.py and iotests.py > > +# > > +# Authors: > > +# Ani Sinha <ani@anisinha.ca> > > + > > +# pylint: disable=invalid-name > > + > > +""" > > +QEMU bios tests using biosbits available at > > +https://biosbits.org/. > > +""" > > + > > +import logging > > +import os > > +import re > > +import shutil > > +import subprocess > > +import sys > > +import tarfile > > +import tempfile > > +import time > > +import unittest > > +from urllib import request > > +import zipfile > > +from typing import ( > > + List, > > + Optional, > > + Sequence, > > +) > > +from tap import TAPTestRunner > > +from qemu.machine import QEMUMachine > > + > > +PYTESTQEMUBIN = os.getenv('PYTEST_QEMU_BINARY') > > +PYTEST_PWD = os.getenv('PYTEST_PWD') > > + > > +def get_arch(): > > + """finds the arch from the qemu binary name""" > > + match = re.search('.*qemu-system-(.*)', PYTESTQEMUBIN) > > + if match: > > + return match.group(1) > > + return 'x86_64' > > + > > +ARCH = get_arch() > > + > > +class QEMUBitsMachine(QEMUMachine): > > + """ > > + A QEMU VM, with isa-debugcon enabled and bits iso passed > > + using -cdrom to QEMU commandline. > > + """ > > + def __init__(self, > > + binary: str, > > + args: Sequence[str] = (), > > + wrapper: Sequence[str] = (), > > + name: Optional[str] = None, > > + base_temp_dir: str = "/var/tmp", > > + debugcon_log: str = "debugcon-log.txt", > > + debugcon_addr: str = "0x403", > > + sock_dir: Optional[str] = None, > > + qmp_timer: Optional[float] = None): > > + # pylint: disable=too-many-arguments > > + > > + if name is None: > > + name = "qemu-bits-%d" % os.getpid() > > + if sock_dir is None: > > + sock_dir = base_temp_dir > > + super().__init__(binary, args, wrapper=wrapper, name=name, > > + base_temp_dir=base_temp_dir, > > + sock_dir=sock_dir, qmp_timer=qmp_timer) > > + self.debugcon_log = debugcon_log > > + self.debugcon_addr = debugcon_addr > > + self.base_temp_dir = base_temp_dir > > + > > + @property > > + def _base_args(self) -> List[str]: > > + args = super()._base_args > > + args.extend([ > > + '-chardev', > > + 'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir, > > + self.debugcon_log), > > + '-device', > > + 'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr, > > + ]) > > + return args > > + > > + def base_args(self): > > + """return the base argument to QEMU binary""" > > + return self._base_args > > + > > +class AcpiBitsTest(unittest.TestCase): > > + """ACPI and SMBIOS tests using biosbits.""" > > + def __init__(self, *args, **kwargs): > > + super().__init__(*args, **kwargs) > > + self._vm = None > > + self._workDir = None > > + self._bitsVer = 2100 > > + self._bitsLoc = "https://github.com/ani-sinha/bits/raw/bits-builds/" > > + self._debugcon_addr = '0x403' > > + self._debugcon_log = 'debugcon-log.txt' > > + logging.basicConfig(level=logging.INFO) > > + > > + def copy_bits_config(self): > > + """ copies the bios bits config file into bits. > > + """ > > + config_file = 'bits-cfg.txt' > > + qemu_bits_config_dir = os.path.join(os.getcwd(), 'bits-config') > > + target_config_dir = os.path.join(self._workDir, > > + 'bits-%d' %self._bitsVer, 'boot') > > + self.assertTrue(os.path.exists(qemu_bits_config_dir)) > > + self.assertTrue(os.path.exists(target_config_dir)) > > + self.assertTrue(os.access(os.path.join(qemu_bits_config_dir, > > + config_file), os.R_OK)) > > + shutil.copy2(os.path.join(qemu_bits_config_dir, config_file), > > + target_config_dir) > > + logging.info('copied config file %s to %s', > > + config_file, target_config_dir) > > + > > + def copy_test_scripts(self): > > + """copies the python test scripts into bits. """ > > + qemu_test_dir = os.path.join(os.getcwd(), 'bits-tests') > > + target_test_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer, > > + 'boot', 'python') > > + > > + self.assertTrue(os.path.exists(qemu_test_dir)) > > + self.assertTrue(os.path.exists(target_test_dir)) > > + > > + for filename in os.listdir(qemu_test_dir): > > + if os.path.isfile(os.path.join(qemu_test_dir, filename)) and \ > > + filename.endswith('.py'): > > + shutil.copy2(os.path.join(qemu_test_dir, filename), > > + target_test_dir) > > + logging.info('copied test file %s to %s', > > + filename, target_test_dir) > > + > > + # now remove the pyc test file if it exists, otherwise the > > + # changes in the python test script won't be executed. > > + testfile_pyc = os.path.splitext(filename)[0] + '.pyc' > > + if os.access(os.path.join(target_test_dir, testfile_pyc), > > + os.F_OK): > > + os.remove(os.path.join(target_test_dir, testfile_pyc)) > > + logging.info('removed compiled file %s', > > + os.path.join(target_test_dir, testfile_pyc)) > > + > > + def fix_mkrescue(self, mkrescue): > > + """ grub-mkrescue is a bash script with two variables, 'prefix' and > > + 'libdir'. They must be pointed to the right location so that the > > + iso can be generated appropriately. We point the two variables to > > + the directory where we have extracted our pre-built bits grub > > + tarball. > > + """ > > + grub_x86_64_mods = os.path.join(self._workDir, 'grub-inst-x86_64-efi') > > + grub_i386_mods = os.path.join(self._workDir, 'grub-inst') > > + > > + self.assertTrue(os.path.exists(grub_x86_64_mods)) > > + self.assertTrue(os.path.exists(grub_i386_mods)) > > + > > + new_script = "" > > + with open(mkrescue, 'r') as filehandle: > > + orig_script = filehandle.read() > > + new_script = re.sub('(^prefix=)(.*)', > > + r'\1"%s"' %grub_x86_64_mods, > > + orig_script, flags=re.M) > > + new_script = re.sub('(^libdir=)(.*)', r'\1"%s/lib"' %grub_i386_mods, > > + new_script, flags=re.M) > > + > > + with open(mkrescue, 'w') as filehandle: > > + filehandle.write(new_script) > > + > > + def generate_bits_iso(self): > > + """ Uses grub-mkrescue to generate a fresh bits iso with the python > > + test scripts > > + """ > > + bits_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer) > > + iso_file = os.path.join(self._workDir, 'bits-%d.iso' %self._bitsVer) > > + mkrescue_script = os.path.join(self._workDir, > > + 'grub-inst-x86_64-efi', 'bin', > > + 'grub-mkrescue') > > + > > + self.assertTrue(os.access(mkrescue_script, > > + os.R_OK | os.W_OK | os.X_OK)) > > + > > + self.fix_mkrescue(mkrescue_script) > > + > > + logging.info('calling grub-mkrescue to generate the biosbits iso ...') > > + > > + try: > > + if os.getenv('V'): > > + subprocess.check_call([mkrescue_script, '-o', > > + iso_file, bits_dir], > > + stdout=subprocess.DEVNULL) > > + else: > > + subprocess.check_call([mkrescue_script, '-o', > > + iso_file, bits_dir], > > + stderr=subprocess.DEVNULL, > > + stdout=subprocess.DEVNULL) > > + except Exception as e: # pylint: disable=broad-except > > + self.skipTest("Error while generating the bits iso. " > > + "Pass V=1 in the environment to get more details. " > > + + str(e)) > > + > > + self.assertTrue(os.access(iso_file, os.R_OK)) > > + > > + logging.info('iso file %s successfully generated.', iso_file) > > + > > + def setUp(self): > > + BITS_LOC = os.getenv("PYTEST_BITSLOC") > > + if BITS_LOC: > > + prefix = BITS_LOC > > + else: > > + prefix = os.path.join(os.getcwd(), 'prebuilt') > > + if not os.path.isdir(prefix): > > + os.mkdir(prefix, mode=0o775) > > + > > + bits_zip_file = os.path.join(prefix, 'bits-%d.zip' > > + %self._bitsVer) > > + grub_tar_file = os.path.join(prefix, > > + 'bits-%d-grub.tar.gz' %self._bitsVer) > > + # if the location of the bits binaries has been specified by the user > > + # and they are not found in that location, skip the test. > > + if BITS_LOC and not os.access(bits_zip_file, os.F_OK): > > + self.skipTest("test skipped since biosbits binaries " + > > + "could not be found in the specified location %s." \ > > + %BITS_LOC) > > + if BITS_LOC and not os.access(grub_tar_file, os.F_OK): > > + self.skipTest("test skipped since biosbits binaries " + > > + "could not be found in the specified location %s." \ > > + %BITS_LOC) > > + > > + self._workDir = tempfile.mkdtemp(prefix='acpi-bits-', > > + suffix='.tmp') > > + logging.info('working dir: %s', self._workDir) > > + > > + localArchive = "bits-%d.zip" % self._bitsVer > > + if not os.access(bits_zip_file, os.F_OK): > > + logging.info("archive %s not found in %s, downloading ...", > > + localArchive, bits_zip_file) > > + try: > > + req = request.urlopen(self._bitsLoc + localArchive) > > + with open(os.path.join(prefix, localArchive), > > + 'wb') as archivef: > > + archivef.write(req.read()) > > + except Exception as e: # pylint: disable=broad-except > > + self.skipTest("test skipped since biosbits binaries " + > > + "could not be obtained." + str(e)) > > + else: > > + logging.info('using locally found %s', localArchive) > > Poking at URLs from python is really gross IMHO. No its not gross :) Avocado framework does the same but in a more complicarted way. > Instead of all this mess, can't we just spawn e.g. "git clone --depth 1"? > And if the directory exists I would fetch and checkout. There are two reasons I can think of why I do not like this idea: (a) a git clone of a whole directory would download all versions of the binary whereas we want only a specific version. Downloading a single file by shallow cloning or creating a git archive is overkill IMHO when a wget style retrieval works just fine. (b) we may later move the binary archives to a ftp server or a google drive. git/version control mechanisms are not the best place to store binary blobs IMHO. In this case also, wget also works. > > > Fundamentally people hate submodules so we are recreating > them ok, but at least we can use git and not open-code http > and file management ... > > > > > + > > + localArchive = "bits-%d-grub.tar.gz" % self._bitsVer > > + if not os.access(grub_tar_file, os.F_OK): > > + logging.info("archive %s not found in %s, downloading ...", > > + localArchive, bits_zip_file) > > + try: > > + req = request.urlopen(self._bitsLoc + localArchive) > > + with open(os.path.join(prefix, localArchive), > > + 'wb') as archivef: > > + archivef.write(req.read()) > > + except Exception as e: # pylint: disable=broad-except > > + self.skipTest("test skipped since biosbits binaries " + > > + "could not be obtained." + str(e)) > > + else: > > + logging.info('using locally found %s', localArchive) > > + > > + # extract the bits software in the temp working directory > > + with zipfile.ZipFile(bits_zip_file, 'r') as zref: > > + zref.extractall(self._workDir) > > + > > + with tarfile.open(grub_tar_file, 'r') as tarball: > > + tarball.extractall(self._workDir) > > + > > + self.copy_test_scripts() > > + self.copy_bits_config() > > + self.generate_bits_iso() > > + > > + def parse_log(self): > > + """parse the log generated by running bits tests and > > + check for failures. > > + """ > > + debugconf = os.path.join(self._workDir, self._debugcon_log) > > + log = "" > > + with open(debugconf, 'r') as filehandle: > > + log = filehandle.read() > > + > > + if os.getenv('V'): > > + print('\nlogs from biosbits follows:') > > + print('==========================================\n') > > + print(log) > > + print('==========================================\n') > > + > > + matchiter = re.finditer(r'(.*Summary: )(\d+ passed), (\d+ failed).*', > > + log) > > + for match in matchiter: > > + # verify that no test cases failed. > > + self.assertEqual(match.group(3).split()[0], '0', > > + 'Some bits tests seems to have failed. ' \ > > + 'Set V=1 in the environment to get the entire ' \ > > + 'log from bits.') > > + > > + def tearDown(self): > > + if self._vm: > > + self.assertFalse(not self._vm.is_running) > > + logging.info('removing the work directory %s', self._workDir) > > + shutil.rmtree(self._workDir) > > + > > + def test_acpi_smbios_bits(self): > > + """The main test case implementaion.""" > > + > > + qemu_bin = PYTESTQEMUBIN > > + iso_file = os.path.join(self._workDir, 'bits-%d.iso' %self._bitsVer) > > + > > + # PYTESTQEMUBIN could be relative to the current directory > > + if not os.access(PYTESTQEMUBIN, os.X_OK) and PYTEST_PWD: > > + qemu_bin = os.path.join(PYTEST_PWD, PYTESTQEMUBIN) > > + > > + logging.info('QEMU binary used: %s', qemu_bin) > > + > > + self.assertTrue(os.access(qemu_bin, os.X_OK)) > > + self.assertTrue(os.access(iso_file, os.R_OK)) > > + > > + self._vm = QEMUBitsMachine(binary=qemu_bin, > > + base_temp_dir=self._workDir, > > + debugcon_log=self._debugcon_log, > > + debugcon_addr=self._debugcon_addr) > > + > > + self._vm.add_args('-cdrom', '%s' %iso_file) > > + > > + args = " ".join(str(arg) for arg in self._vm.base_args()) + \ > > + " " + " ".join(str(arg) for arg in self._vm.args) > > + > > + logging.info("launching QEMU vm with the following arguments: %s", > > + args) > > + > > + self._vm.launch() > > + # biosbits has been configured to run all the specified test suites > > + # in batch mode and then automatically initiate a vm shutdown. > > + # sleep for maximum of one minute > > + max_sleep_time = time.monotonic() + 60 > > + while self._vm.is_running() and time.monotonic() < max_sleep_time: > > + time.sleep(1) > > + > > + self.assertFalse(time.monotonic() > max_sleep_time, > > + 'The VM seems to have failed to shutdown in time') > > + > > + self.parse_log() > > + > > +def execute_unittest(argv: List[str], debug: bool = False, > > + runner: TAPTestRunner = None) -> None: > > + """Executes unittests within the calling module.""" > > + > > + unittest.main(argv=argv, > > + testRunner=runner, > > + verbosity=2 if debug else 1, > > + warnings=None if sys.warnoptions else 'ignore') > > + > > +def main(): > > + """ The main function where execution begins. """ > > + > > + assert PYTESTQEMUBIN is not None, \ > > + "Environment variable PYTEST_QEMU_BINARY required." > > + > > + runner = TAPTestRunner() > > + runner.set_stream(True) > > + runner.set_format("%s/acpi-bits-test" %ARCH) > > + execute_unittest(sys.argv, False, runner) > > + > > +main() > > diff --git a/tests/pytest/acpi-bits/meson.build b/tests/pytest/acpi-bits/meson.build > > new file mode 100644 > > index 0000000000..099c191d57 > > --- /dev/null > > +++ b/tests/pytest/acpi-bits/meson.build > > @@ -0,0 +1,33 @@ > > +xorriso = find_program('xorriso', required: true) > > +if not xorriso.found() > > + message('xorriso not found ... disabled bits acpi tests.') > > + subdir_done() > > +endif > > + > > +subdir('bits-tests') > > +subdir('bits-config') > > + > > +test_files = ['acpi-bits-test.py'] > > +requirements = 'requirements.txt' > > + > > +copytestfiles = custom_target('copy test files', > > + input : test_files, > > + output : test_files, > > + command : ['cp', '@INPUT@', '@OUTPUT@'], > > + install : false, > > + build_by_default : true) > > + > > +requirementsfiles = custom_target('copy py req files', > > + input : requirements, > > + output : requirements, > > + command : ['cp', '@INPUT@', '@OUTPUT@'], > > + install : false, > > + build_by_default : true) > > + > > +other_deps += [copytestfiles,requirementsfiles] > > + > > +pytest_executables += { > > + 'acpi-bits-test': configure_file(copy:true, > > + input:'acpi-bits-test-venv.sh', > > + output:'acpi-bits-test') > > +} > > diff --git a/tests/pytest/acpi-bits/requirements.txt b/tests/pytest/acpi-bits/requirements.txt > > new file mode 100644 > > index 0000000000..00cdad09ef > > --- /dev/null > > +++ b/tests/pytest/acpi-bits/requirements.txt > > @@ -0,0 +1 @@ > > +tap.py > > -- > > 2.25.1 > > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-15 4:17 ` Ani Sinha @ 2022-07-15 6:50 ` Michael S. Tsirkin 2022-07-16 6:36 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: Michael S. Tsirkin @ 2022-07-15 6:50 UTC (permalink / raw) To: Ani Sinha Cc: qemu-devel, thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > Instead of all this mess, can't we just spawn e.g. "git clone --depth 1"? > > And if the directory exists I would fetch and checkout. > > There are two reasons I can think of why I do not like this idea: > > (a) a git clone of a whole directory would download all versions of the > binary whereas we want only a specific version. You mention shallow clone yourself, and I used --depth 1 above. > Downloading a single file > by shallow cloning or creating a git archive is overkill IMHO when a wget > style retrieval works just fine. However, it does not provide for versioning, tagging etc so you have to implement your own schema. > (b) we may later move the binary archives to a ftp server or a google > drive. git/version control mechanisms are not the best place to store > binary blobs IMHO. In this case also, wget also works. surely neither ftp nor google drive are reasonable dependencies for a free software project. But qemu does maintain an http server already so that't a plus. I am not insisting on git, but I do not like it that security, mirroring, caching, versioning all have to be hand rolled and then figured out by users and maintainers. Who frankly have other things to do besides learning yet another boutique configuration language. And I worry that after a while we come up with a new organization schema for the files, old ones are moved around and nothing relying on the URL works. git is kind of good in that it enforces the idea that history is immutable. If not vanilla git can we find another utility we can reuse? git lfs? It seems to be supported by both github and gitlab though bizarrely github has bandwidth limits on git lfs but apparently not on vanilla git. Hosting on qemu.org will require maintaining a server there though. All that said maybe we should just run with it as it is, just so we get *something* in the door, and then worry about getting the storage side straight before making this test a requirement for all acpi developers. -- MST ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-15 6:50 ` Michael S. Tsirkin @ 2022-07-16 6:36 ` Ani Sinha 2022-07-16 15:30 ` Michael S. Tsirkin 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-07-16 6:36 UTC (permalink / raw) To: Michael S. Tsirkin Cc: berrange, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth [-- Attachment #1: Type: text/plain, Size: 2656 bytes --] On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin <mst@redhat.com> wrote: > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > > Instead of all this mess, can't we just spawn e.g. "git clone --depth > 1"? > > > And if the directory exists I would fetch and checkout. > > > > There are two reasons I can think of why I do not like this idea: > > > > (a) a git clone of a whole directory would download all versions of the > > binary whereas we want only a specific version. > > You mention shallow clone yourself, and I used --depth 1 above. > > > Downloading a single file > > by shallow cloning or creating a git archive is overkill IMHO when a wget > > style retrieval works just fine. > > However, it does not provide for versioning, tagging etc so you have > to implement your own schema. Hmm I’m not sure if we need all that. Bits has its own versioning mechanism and I think all we need to do is maintain the same versioning logic and maintain binaries of different versions. Do we really need the power of git/version control here? Dunno. > > > > (b) we may later move the binary archives to a ftp server or a google > > drive. git/version control mechanisms are not the best place to store > > binary blobs IMHO. In this case also, wget also works. > > surely neither ftp nor google drive are reasonable dependencies > for a free software project. But qemu does maintain an http server > already so that't a plus. > > > > I am not insisting on git, but I do not like it that security, > mirroring, caching, versioning all have to be hand rolled and then > figured out by users and maintainers. Who frankly have other things to > do besides learning yet another boutique configuration language. Yeah we do not want to reinvent the wheel all over again. > > And I worry that after a while we come up with a new organization schema > for the files, old ones are moved around and nothing relying on the URL > works. git is kind of good in that it enforces the idea that history is > immutable. Ah I see your point here. > > If not vanilla git can we find another utility we can reuse? > > git lfs? It seems to be supported by both github and gitlab though > bizarrely github has bandwidth limits on git lfs but apparently not on > vanilla git. Hosting on qemu.org will require maintaining a server > there though. > > > > All that said maybe we should just run with it as it is, just so we get > *something* in the door, and then worry about getting the storage side > straight before making this test a requirement for all acpi developers. > [-- Attachment #2: Type: text/html, Size: 3898 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-16 6:36 ` Ani Sinha @ 2022-07-16 15:30 ` Michael S. Tsirkin 2022-07-25 18:02 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: Michael S. Tsirkin @ 2022-07-16 15:30 UTC (permalink / raw) To: Ani Sinha Cc: berrange, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote: > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin <mst@redhat.com> wrote: > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > > Instead of all this mess, can't we just spawn e.g. "git clone --depth > 1"? > > > And if the directory exists I would fetch and checkout. > > > > There are two reasons I can think of why I do not like this idea: > > > > (a) a git clone of a whole directory would download all versions of the > > binary whereas we want only a specific version. > > You mention shallow clone yourself, and I used --depth 1 above. > > > Downloading a single file > > by shallow cloning or creating a git archive is overkill IMHO when a wget > > style retrieval works just fine. > > However, it does not provide for versioning, tagging etc so you have > to implement your own schema. > > > Hmm I’m not sure if we need all that. Bits has its own versioning mechanism and > I think all we need to do is maintain the same versioning logic and maintain > binaries of different versions. Do we really need the power of git/version > control here? Dunno. Well we need some schema. Given we are not using official bits releases I don't think we can reuse theirs. > > > > > > (b) we may later move the binary archives to a ftp server or a google > > drive. git/version control mechanisms are not the best place to store > > binary blobs IMHO. In this case also, wget also works. > > surely neither ftp nor google drive are reasonable dependencies > for a free software project. But qemu does maintain an http server > already so that't a plus. > > > > I am not insisting on git, but I do not like it that security, > mirroring, caching, versioning all have to be hand rolled and then > figured out by users and maintainers. Who frankly have other things to > do besides learning yet another boutique configuration language. > > > Yeah we do not want to reinvent the wheel all over again. > > > > > And I worry that after a while we come up with a new organization schema > for the files, old ones are moved around and nothing relying on the URL > works. git is kind of good in that it enforces the idea that history is > immutable. > > > Ah I see your point here. > > > > > If not vanilla git can we find another utility we can reuse? > > git lfs? It seems to be supported by both github and gitlab though > bizarrely github has bandwidth limits on git lfs but apparently not on > vanilla git. Hosting on qemu.org will require maintaining a server > there though. > > > > All that said maybe we should just run with it as it is, just so we get > *something* in the door, and then worry about getting the storage side > straight before making this test a requirement for all acpi developers. > > > > > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-16 15:30 ` Michael S. Tsirkin @ 2022-07-25 18:02 ` Ani Sinha 2022-07-27 18:38 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-07-25 18:02 UTC (permalink / raw) To: Michael S. Tsirkin Cc: Ani Sinha, berrange, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth [-- Attachment #1: Type: text/plain, Size: 3360 bytes --] On Sat, 16 Jul 2022, Michael S. Tsirkin wrote: > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote: > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > > > Instead of all this mess, can't we just spawn e.g. "git clone --depth > > 1"? > > > > And if the directory exists I would fetch and checkout. > > > > > > There are two reasons I can think of why I do not like this idea: > > > > > > (a) a git clone of a whole directory would download all versions of the > > > binary whereas we want only a specific version. > > > > You mention shallow clone yourself, and I used --depth 1 above. > > > > > Downloading a single file > > > by shallow cloning or creating a git archive is overkill IMHO when a wget > > > style retrieval works just fine. > > > > However, it does not provide for versioning, tagging etc so you have > > to implement your own schema. > > > > > > Hmm I’m not sure if we need all that. Bits has its own versioning mechanism and > > I think all we need to do is maintain the same versioning logic and maintain > > binaries of different versions. Do we really need the power of git/version > > control here? Dunno. > > Well we need some schema. Given we are not using official bits releases > I don't think we can reuse theirs. OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and the binaries in some other repo first. Everything else hinges on that. We can fix the rest of the bits later incrementally. > > > > > > > > > > > > (b) we may later move the binary archives to a ftp server or a google > > > drive. git/version control mechanisms are not the best place to store > > > binary blobs IMHO. In this case also, wget also works. > > > > surely neither ftp nor google drive are reasonable dependencies > > for a free software project. But qemu does maintain an http server > > already so that't a plus. > > > > > > > > I am not insisting on git, but I do not like it that security, > > mirroring, caching, versioning all have to be hand rolled and then > > figured out by users and maintainers. Who frankly have other things to > > do besides learning yet another boutique configuration language. > > > > > > Yeah we do not want to reinvent the wheel all over again. > > > > > > > > > > And I worry that after a while we come up with a new organization schema > > for the files, old ones are moved around and nothing relying on the URL > > works. git is kind of good in that it enforces the idea that history is > > immutable. > > > > > > Ah I see your point here. > > > > > > > > > > If not vanilla git can we find another utility we can reuse? > > > > git lfs? It seems to be supported by both github and gitlab though > > bizarrely github has bandwidth limits on git lfs but apparently not on > > vanilla git. Hosting on qemu.org will require maintaining a server > > there though. > > > > > > > > All that said maybe we should just run with it as it is, just so we get > > *something* in the door, and then worry about getting the storage side > > straight before making this test a requirement for all acpi developers. > > > > > > > > > > > > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-25 18:02 ` Ani Sinha @ 2022-07-27 18:38 ` Ani Sinha 2022-09-16 16:00 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-07-27 18:38 UTC (permalink / raw) To: Ani Sinha Cc: Michael S. Tsirkin, berrange, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth [-- Attachment #1: Type: text/plain, Size: 1827 bytes --] On Mon, 25 Jul 2022, Ani Sinha wrote: > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote: > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote: > > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > > > > Instead of all this mess, can't we just spawn e.g. "git clone --depth > > > 1"? > > > > > And if the directory exists I would fetch and checkout. > > > > > > > > There are two reasons I can think of why I do not like this idea: > > > > > > > > (a) a git clone of a whole directory would download all versions of the > > > > binary whereas we want only a specific version. > > > > > > You mention shallow clone yourself, and I used --depth 1 above. > > > > > > > Downloading a single file > > > > by shallow cloning or creating a git archive is overkill IMHO when a wget > > > > style retrieval works just fine. > > > > > > However, it does not provide for versioning, tagging etc so you have > > > to implement your own schema. > > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own versioning mechanism and > > > I think all we need to do is maintain the same versioning logic and maintain > > > binaries of different versions. Do we really need the power of git/version > > > control here? Dunno. > > > > Well we need some schema. Given we are not using official bits releases > > I don't think we can reuse theirs. > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > the binaries in some other repo first. Everything else hinges on that. We > can fix the rest of the bits later incrementally. DanPB, any thoughts on putting bits on git.qemu.org or where and how to keep the binaries? ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-27 18:38 ` Ani Sinha @ 2022-09-16 16:00 ` Ani Sinha 2022-09-17 20:28 ` Michael S. Tsirkin 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-09-16 16:00 UTC (permalink / raw) To: Ani Sinha Cc: Michael S. Tsirkin, berrange, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha <ani@anisinha.ca> wrote: > > > > On Mon, 25 Jul 2022, Ani Sinha wrote: > > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote: > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote: > > > > > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > > > > > Instead of all this mess, can't we just spawn e.g. "git clone --depth > > > > 1"? > > > > > > And if the directory exists I would fetch and checkout. > > > > > > > > > > There are two reasons I can think of why I do not like this idea: > > > > > > > > > > (a) a git clone of a whole directory would download all versions of the > > > > > binary whereas we want only a specific version. > > > > > > > > You mention shallow clone yourself, and I used --depth 1 above. > > > > > > > > > Downloading a single file > > > > > by shallow cloning or creating a git archive is overkill IMHO when a wget > > > > > style retrieval works just fine. > > > > > > > > However, it does not provide for versioning, tagging etc so you have > > > > to implement your own schema. > > > > > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own versioning mechanism and > > > > I think all we need to do is maintain the same versioning logic and maintain > > > > binaries of different versions. Do we really need the power of git/version > > > > control here? Dunno. > > > > > > Well we need some schema. Given we are not using official bits releases > > > I don't think we can reuse theirs. > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > the binaries in some other repo first. Everything else hinges on that. We > > can fix the rest of the bits later incrementally. > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > keep the binaries? Can we please conclude on this? Peter, can you please fork the repo? I have tried many times to reach you on IRC but failed. ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-16 16:00 ` Ani Sinha @ 2022-09-17 20:28 ` Michael S. Tsirkin 2022-09-27 8:13 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-17 20:28 UTC (permalink / raw) To: Ani Sinha Cc: berrange, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote: > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > On Mon, 25 Jul 2022, Ani Sinha wrote: > > > > > > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote: > > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > > > > > > Instead of all this mess, can't we just spawn e.g. "git clone --depth > > > > > 1"? > > > > > > > And if the directory exists I would fetch and checkout. > > > > > > > > > > > > There are two reasons I can think of why I do not like this idea: > > > > > > > > > > > > (a) a git clone of a whole directory would download all versions of the > > > > > > binary whereas we want only a specific version. > > > > > > > > > > You mention shallow clone yourself, and I used --depth 1 above. > > > > > > > > > > > Downloading a single file > > > > > > by shallow cloning or creating a git archive is overkill IMHO when a wget > > > > > > style retrieval works just fine. > > > > > > > > > > However, it does not provide for versioning, tagging etc so you have > > > > > to implement your own schema. > > > > > > > > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own versioning mechanism and > > > > > I think all we need to do is maintain the same versioning logic and maintain > > > > > binaries of different versions. Do we really need the power of git/version > > > > > control here? Dunno. > > > > > > > > Well we need some schema. Given we are not using official bits releases > > > > I don't think we can reuse theirs. > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > the binaries in some other repo first. Everything else hinges on that. We > > > can fix the rest of the bits later incrementally. > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > keep the binaries? > > Can we please conclude on this? > Peter, can you please fork the repo? I have tried many times to reach > you on IRC but failed. Probably because of travel around KVM forum. I think given our CI is under pressure again due to gitlab free tier limits, tying binaries to CI isn't a great idea at this stage. Can Ani just upload binaies to qemu.org for now? -- MST ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-17 20:28 ` Michael S. Tsirkin @ 2022-09-27 8:13 ` Ani Sinha 2022-09-27 8:33 ` Daniel P. Berrangé 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-09-27 8:13 UTC (permalink / raw) To: Michael S. Tsirkin Cc: berrange, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote: > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > > > > > On Mon, 25 Jul 2022, Ani Sinha wrote: > > > > > > > > > > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote: > > > > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > > > > > > > Instead of all this mess, can't we just spawn e.g. "git clone --depth > > > > > > 1"? > > > > > > > > And if the directory exists I would fetch and checkout. > > > > > > > > > > > > > > There are two reasons I can think of why I do not like this idea: > > > > > > > > > > > > > > (a) a git clone of a whole directory would download all versions of the > > > > > > > binary whereas we want only a specific version. > > > > > > > > > > > > You mention shallow clone yourself, and I used --depth 1 above. > > > > > > > > > > > > > Downloading a single file > > > > > > > by shallow cloning or creating a git archive is overkill IMHO when a wget > > > > > > > style retrieval works just fine. > > > > > > > > > > > > However, it does not provide for versioning, tagging etc so you have > > > > > > to implement your own schema. > > > > > > > > > > > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own versioning mechanism and > > > > > > I think all we need to do is maintain the same versioning logic and maintain > > > > > > binaries of different versions. Do we really need the power of git/version > > > > > > control here? Dunno. > > > > > > > > > > Well we need some schema. Given we are not using official bits releases > > > > > I don't think we can reuse theirs. > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > can fix the rest of the bits later incrementally. > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > keep the binaries? > > > > Can we please conclude on this? > > Peter, can you please fork the repo? I have tried many times to reach > > you on IRC but failed. > > Probably because of travel around KVM forum. > > I think given our CI is under pressure again due to gitlab free tier > limits, tying binaries to CI isn't a great idea at this stage. > Can Ani just upload binaies to qemu.org for now? I agree with Michael here. Having a full ci/cd job for this is overkill IMHO. We should create a repo just for the binaries, have a README there to explain how we generate them and check in new versions as and when needed (it won't be frequent). How about biosbits-bin repo? > > -- > MST > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 8:13 ` Ani Sinha @ 2022-09-27 8:33 ` Daniel P. Berrangé 2022-09-27 10:07 ` Ani Sinha 2022-09-27 21:18 ` Michael S. Tsirkin 0 siblings, 2 replies; 90+ messages in thread From: Daniel P. Berrangé @ 2022-09-27 8:33 UTC (permalink / raw) To: Ani Sinha Cc: Michael S. Tsirkin, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Tue, Sep 27, 2022 at 01:43:15PM +0530, Ani Sinha wrote: > On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote: > > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > > > > > > > > > On Mon, 25 Jul 2022, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote: > > > > > > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > > > > > > > > Instead of all this mess, can't we just spawn e.g. "git clone --depth > > > > > > > 1"? > > > > > > > > > And if the directory exists I would fetch and checkout. > > > > > > > > > > > > > > > > There are two reasons I can think of why I do not like this idea: > > > > > > > > > > > > > > > > (a) a git clone of a whole directory would download all versions of the > > > > > > > > binary whereas we want only a specific version. > > > > > > > > > > > > > > You mention shallow clone yourself, and I used --depth 1 above. > > > > > > > > > > > > > > > Downloading a single file > > > > > > > > by shallow cloning or creating a git archive is overkill IMHO when a wget > > > > > > > > style retrieval works just fine. > > > > > > > > > > > > > > However, it does not provide for versioning, tagging etc so you have > > > > > > > to implement your own schema. > > > > > > > > > > > > > > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own versioning mechanism and > > > > > > > I think all we need to do is maintain the same versioning logic and maintain > > > > > > > binaries of different versions. Do we really need the power of git/version > > > > > > > control here? Dunno. > > > > > > > > > > > > Well we need some schema. Given we are not using official bits releases > > > > > > I don't think we can reuse theirs. > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > > can fix the rest of the bits later incrementally. > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > > keep the binaries? > > > > > > Can we please conclude on this? > > > Peter, can you please fork the repo? I have tried many times to reach > > > you on IRC but failed. > > > > Probably because of travel around KVM forum. > > > > I think given our CI is under pressure again due to gitlab free tier > > limits, tying binaries to CI isn't a great idea at this stage. > > Can Ani just upload binaies to qemu.org for now? > > I agree with Michael here. Having a full ci/cd job for this is > overkill IMHO. We should create a repo just for the binaries, have a > README there to explain how we generate them and check in new versions > as and when needed (it won't be frequent). > How about biosbits-bin repo? If QEMU is hosting binaries, where any part contains GPL code, then we need to be providing the full and corresponding source and the build scripts needed to re-create the binary. Once we have such scripts it should be trivial to trigger that from a CI job. If it isn't then we're doing something wrong. The CI quota is not an issue, because this is not a job that we need to run continuously. It can be triggered manually as & when we decide we need to refresh the binary, so would be a small one-off CI quota hit. Also note that gitlab is intending to start enforcing storage quota on projects in the not too distant future. This makes it unappealing to store binaries in git repos, unless we genuinely need the ability to access historical versions of the binary. I don't believe we need that for biosbits. The binary can be published as a CI artifact and accessed directly from the latest artifact download link. This ensures we only consume quota for the most recently published binary artifact. So I don't see a compelling reason to upload binaries into git. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 8:33 ` Daniel P. Berrangé @ 2022-09-27 10:07 ` Ani Sinha 2022-09-27 10:18 ` Daniel P. Berrangé 2022-09-27 21:19 ` Michael S. Tsirkin 2022-09-27 21:18 ` Michael S. Tsirkin 1 sibling, 2 replies; 90+ messages in thread From: Ani Sinha @ 2022-09-27 10:07 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Michael S. Tsirkin, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth > > > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > > > can fix the rest of the bits later incrementally. > > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > > > keep the binaries? > > > > > > > > Can we please conclude on this? > > > > Peter, can you please fork the repo? I have tried many times to reach > > > > you on IRC but failed. > > > > > > Probably because of travel around KVM forum. > > > > > > I think given our CI is under pressure again due to gitlab free tier > > > limits, tying binaries to CI isn't a great idea at this stage. > > > Can Ani just upload binaies to qemu.org for now? > > > > I agree with Michael here. Having a full ci/cd job for this is > > overkill IMHO. We should create a repo just for the binaries, have a > > README there to explain how we generate them and check in new versions > > as and when needed (it won't be frequent). > > How about biosbits-bin repo? > > If QEMU is hosting binaries, where any part contains GPL code, then we > need to be providing the full and corresponding source and the build > scripts needed to re-create the binary. Once we have such scripts it > should be trivial to trigger that from a CI job. If it isn't then > we're doing something wrong. I was thinking of commiting the build scripts in a branch of https://gitlab.com/qemu-project/biosbits-bits. This would separate them from the main source. The scripts would build a version of qemu-bits based on the version information passed to it from the environment. Before I committed the scripts, I wanted to check whether we would want to do that or have a separate repo containing the binaries and the build scripts. Seems we want the former. As for the gitlab-ci, I looked at the yaml file and the qemu ones looks quite complicated. Can someone help me generate one based on the build script here? https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > The CI quota is not an issue, because > this is not a job that we need to run continuously. It can be triggered > manually as & when we decide we need to refresh the binary, so would > be a small one-off CI quota hit. > > Also note that gitlab is intending to start enforcing storage quota > on projects in the not too distant future. This makes it unappealing > to store binaries in git repos, unless we genuinely need the ability > to access historical versions of the binary. I don't believe we need > that for biosbits. > > The binary can be published as a CI artifact and accessed directly > from the latest artifact download link. This ensures we only consume > quota for the most recently published binary artifact. So I don't see > a compelling reason to upload binaries into git. > > With regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 10:07 ` Ani Sinha @ 2022-09-27 10:18 ` Daniel P. Berrangé 2022-09-27 10:24 ` Ani Sinha 2022-09-27 11:42 ` Ani Sinha 2022-09-27 21:19 ` Michael S. Tsirkin 1 sibling, 2 replies; 90+ messages in thread From: Daniel P. Berrangé @ 2022-09-27 10:18 UTC (permalink / raw) To: Ani Sinha Cc: Michael S. Tsirkin, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > > > > can fix the rest of the bits later incrementally. > > > > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > > > > keep the binaries? > > > > > > > > > > Can we please conclude on this? > > > > > Peter, can you please fork the repo? I have tried many times to reach > > > > > you on IRC but failed. > > > > > > > > Probably because of travel around KVM forum. > > > > > > > > I think given our CI is under pressure again due to gitlab free tier > > > > limits, tying binaries to CI isn't a great idea at this stage. > > > > Can Ani just upload binaies to qemu.org for now? > > > > > > I agree with Michael here. Having a full ci/cd job for this is > > > overkill IMHO. We should create a repo just for the binaries, have a > > > README there to explain how we generate them and check in new versions > > > as and when needed (it won't be frequent). > > > How about biosbits-bin repo? > > > > If QEMU is hosting binaries, where any part contains GPL code, then we > > need to be providing the full and corresponding source and the build > > scripts needed to re-create the binary. Once we have such scripts it > > should be trivial to trigger that from a CI job. If it isn't then > > we're doing something wrong. > > I was thinking of commiting the build scripts in a branch of > https://gitlab.com/qemu-project/biosbits-bits. > This would separate them from the main source. The scripts would build > a version of qemu-bits based on the version information passed to it > from the environment. > Before I committed the scripts, I wanted to check whether we would > want to do that or have a separate repo containing the binaries and > the build scripts. > Seems we want the former. > > As for the gitlab-ci, I looked at the yaml file and the qemu ones > looks quite complicated. Can someone help me generate one based on the > build script here? > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh Yes, QEMU's rules aren't a good place to start if you're trying to learn gitlab CI, as they're very advanced. The simple case though is quite simple. * You need a container image to act as the build env * In 'before_script' install any packages you need on top of the base container image for build deps * In 'script' run whatever shell commands you need in order to build the project * Add an 'artifacts' section to publish one (or more) files/dirs as output The simplest example would be something like mybuild: image: fedora:36 before_script: - dnf install -y gcc script: - gcc -o myapp myapp.c artifacts paths: - myapp So essentially use 'docker run yourimage:tag' locally to bring up the build env, and interactively work out what packags yuo need to install, and what scripts to invoke to build it. When you have it figured out, just copy those commands into the before_script/script sections in .gitlab-ci.yml Note, the .gitlab-ci.yml file would be in the git repo of the project you're building typically, which would be the main 'biosbits' project i presume. There's a fairly decent reference of syntax here: https://docs.gitlab.com/ee/ci/yaml/ And broader set of docs https://docs.gitlab.com/ee/ci/ With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 10:18 ` Daniel P. Berrangé @ 2022-09-27 10:24 ` Ani Sinha 2022-09-27 11:42 ` Ani Sinha 1 sibling, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-09-27 10:24 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Michael S. Tsirkin, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth [-- Attachment #1: Type: text/plain, Size: 4336 bytes --] On Tue, Sep 27, 2022 at 15:48 Daniel P. Berrangé <berrange@redhat.com> wrote: > On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in > git.qemu.org and > > > > > > > > the binaries in some other repo first. Everything else > hinges on that. We > > > > > > > > can fix the rest of the bits later incrementally. > > > > > > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where > and how to > > > > > > > keep the binaries? > > > > > > > > > > > > Can we please conclude on this? > > > > > > Peter, can you please fork the repo? I have tried many times to > reach > > > > > > you on IRC but failed. > > > > > > > > > > Probably because of travel around KVM forum. > > > > > > > > > > I think given our CI is under pressure again due to gitlab free > tier > > > > > limits, tying binaries to CI isn't a great idea at this stage. > > > > > Can Ani just upload binaies to qemu.org for now? > > > > > > > > I agree with Michael here. Having a full ci/cd job for this is > > > > overkill IMHO. We should create a repo just for the binaries, have a > > > > README there to explain how we generate them and check in new > versions > > > > as and when needed (it won't be frequent). > > > > How about biosbits-bin repo? > > > > > > If QEMU is hosting binaries, where any part contains GPL code, then we > > > need to be providing the full and corresponding source and the build > > > scripts needed to re-create the binary. Once we have such scripts it > > > should be trivial to trigger that from a CI job. If it isn't then > > > we're doing something wrong. > > > > I was thinking of commiting the build scripts in a branch of > > https://gitlab.com/qemu-project/biosbits-bits. > > This would separate them from the main source. The scripts would build > > a version of qemu-bits based on the version information passed to it > > from the environment. > > Before I committed the scripts, I wanted to check whether we would > > want to do that or have a separate repo containing the binaries and > > the build scripts. > > Seems we want the former. > > > > As for the gitlab-ci, I looked at the yaml file and the qemu ones > > looks quite complicated. Can someone help me generate one based on the > > build script here? > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > Yes, QEMU's rules aren't a good place to start if you're trying > to learn gitlab CI, as they're very advanced. > > The simple case though is quite simple. > > * You need a container image to act as the build env > * In 'before_script' install any packages you need on top of the > base container image for build deps > * In 'script' run whatever shell commands you need in order > to build the project > * Add an 'artifacts' section to publish one (or more) files/dirs > as output > > The simplest example would be something like > > mybuild: > image: fedora:36 > before_script: > - dnf install -y gcc > script: > - gcc -o myapp myapp.c > artifacts > paths: > - myapp > > > So essentially use 'docker run yourimage:tag' locally to bring > up the build env, and interactively work out what packags yuo > need to install, and what scripts to invoke to build it. When > you have it figured out, just copy those commands into the > before_script/script sections in .gitlab-ci.yml I already have a Dockerfile and a build script rahe uses it to generate the binaries https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile > > > Note, the .gitlab-ci.yml file would be in the git repo of the > project you're building typically, which would be the main > 'biosbits' project i presume. > > There's a fairly decent reference of syntax here: > > https://docs.gitlab.com/ee/ci/yaml/ > > And broader set of docs > > https://docs.gitlab.com/ee/ci/ > > With regards, > Daniel > -- > |: https://berrange.com -o- > https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- > https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- > https://www.instagram.com/dberrange :| > > [-- Attachment #2: Type: text/html, Size: 6685 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 10:18 ` Daniel P. Berrangé 2022-09-27 10:24 ` Ani Sinha @ 2022-09-27 11:42 ` Ani Sinha 2022-09-27 14:05 ` Ani Sinha 1 sibling, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-09-27 11:42 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Michael S. Tsirkin, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Tue, Sep 27, 2022 at 3:48 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > > > > > can fix the rest of the bits later incrementally. > > > > > > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > > > > > keep the binaries? > > > > > > > > > > > > Can we please conclude on this? > > > > > > Peter, can you please fork the repo? I have tried many times to reach > > > > > > you on IRC but failed. > > > > > > > > > > Probably because of travel around KVM forum. > > > > > > > > > > I think given our CI is under pressure again due to gitlab free tier > > > > > limits, tying binaries to CI isn't a great idea at this stage. > > > > > Can Ani just upload binaies to qemu.org for now? > > > > > > > > I agree with Michael here. Having a full ci/cd job for this is > > > > overkill IMHO. We should create a repo just for the binaries, have a > > > > README there to explain how we generate them and check in new versions > > > > as and when needed (it won't be frequent). > > > > How about biosbits-bin repo? > > > > > > If QEMU is hosting binaries, where any part contains GPL code, then we > > > need to be providing the full and corresponding source and the build > > > scripts needed to re-create the binary. Once we have such scripts it > > > should be trivial to trigger that from a CI job. If it isn't then > > > we're doing something wrong. > > > > I was thinking of commiting the build scripts in a branch of > > https://gitlab.com/qemu-project/biosbits-bits. > > This would separate them from the main source. The scripts would build > > a version of qemu-bits based on the version information passed to it > > from the environment. > > Before I committed the scripts, I wanted to check whether we would > > want to do that or have a separate repo containing the binaries and > > the build scripts. > > Seems we want the former. > > > > As for the gitlab-ci, I looked at the yaml file and the qemu ones > > looks quite complicated. Can someone help me generate one based on the > > build script here? > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > Yes, QEMU's rules aren't a good place to start if you're trying > to learn gitlab CI, as they're very advanced. > > The simple case though is quite simple. > > * You need a container image to act as the build env > * In 'before_script' install any packages you need on top of the > base container image for build deps > * In 'script' run whatever shell commands you need in order > to build the project > * Add an 'artifacts' section to publish one (or more) files/dirs > as output > > The simplest example would be something like > > mybuild: > image: fedora:36 > before_script: > - dnf install -y gcc > script: > - gcc -o myapp myapp.c > artifacts > paths: > - myapp > How does this look? https://pastebin.com/0YyLFmi3 > > So essentially use 'docker run yourimage:tag' locally to bring > up the build env, and interactively work out what packags yuo > need to install, and what scripts to invoke to build it. When > you have it figured out, just copy those commands into the > before_script/script sections in .gitlab-ci.yml > > > Note, the .gitlab-ci.yml file would be in the git repo of the > project you're building typically, which would be the main > 'biosbits' project i presume. > > There's a fairly decent reference of syntax here: > > https://docs.gitlab.com/ee/ci/yaml/ > > And broader set of docs > > https://docs.gitlab.com/ee/ci/ > > With regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 11:42 ` Ani Sinha @ 2022-09-27 14:05 ` Ani Sinha 2022-09-27 15:45 ` Daniel P. Berrangé 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-09-27 14:05 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Michael S. Tsirkin, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Tue, Sep 27, 2022 at 5:12 PM Ani Sinha <ani@anisinha.ca> wrote: > > On Tue, Sep 27, 2022 at 3:48 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > > > > On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > > > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > > > > > > can fix the rest of the bits later incrementally. > > > > > > > > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > > > > > > keep the binaries? > > > > > > > > > > > > > > Can we please conclude on this? > > > > > > > Peter, can you please fork the repo? I have tried many times to reach > > > > > > > you on IRC but failed. > > > > > > > > > > > > Probably because of travel around KVM forum. > > > > > > > > > > > > I think given our CI is under pressure again due to gitlab free tier > > > > > > limits, tying binaries to CI isn't a great idea at this stage. > > > > > > Can Ani just upload binaies to qemu.org for now? > > > > > > > > > > I agree with Michael here. Having a full ci/cd job for this is > > > > > overkill IMHO. We should create a repo just for the binaries, have a > > > > > README there to explain how we generate them and check in new versions > > > > > as and when needed (it won't be frequent). > > > > > How about biosbits-bin repo? > > > > > > > > If QEMU is hosting binaries, where any part contains GPL code, then we > > > > need to be providing the full and corresponding source and the build > > > > scripts needed to re-create the binary. Once we have such scripts it > > > > should be trivial to trigger that from a CI job. If it isn't then > > > > we're doing something wrong. > > > > > > I was thinking of commiting the build scripts in a branch of > > > https://gitlab.com/qemu-project/biosbits-bits. > > > This would separate them from the main source. The scripts would build > > > a version of qemu-bits based on the version information passed to it > > > from the environment. > > > Before I committed the scripts, I wanted to check whether we would > > > want to do that or have a separate repo containing the binaries and > > > the build scripts. > > > Seems we want the former. > > > > > > As for the gitlab-ci, I looked at the yaml file and the qemu ones > > > looks quite complicated. Can someone help me generate one based on the > > > build script here? > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > > > Yes, QEMU's rules aren't a good place to start if you're trying > > to learn gitlab CI, as they're very advanced. > > > > The simple case though is quite simple. > > > > * You need a container image to act as the build env > > * In 'before_script' install any packages you need on top of the > > base container image for build deps > > * In 'script' run whatever shell commands you need in order > > to build the project > > * Add an 'artifacts' section to publish one (or more) files/dirs > > as output > > > > The simplest example would be something like > > > > mybuild: > > image: fedora:36 > > before_script: > > - dnf install -y gcc > > script: > > - gcc -o myapp myapp.c > > artifacts > > paths: > > - myapp > > > > How does this look? > https://pastebin.com/0YyLFmi3 Alright, .gitlab-ci.yml is produced and the pipeline succeeds. However, the question still remains, where do we keep the generated artifacts? ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 14:05 ` Ani Sinha @ 2022-09-27 15:45 ` Daniel P. Berrangé 2022-09-27 21:21 ` Michael S. Tsirkin 0 siblings, 1 reply; 90+ messages in thread From: Daniel P. Berrangé @ 2022-09-27 15:45 UTC (permalink / raw) To: Ani Sinha Cc: Michael S. Tsirkin, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote: > On Tue, Sep 27, 2022 at 5:12 PM Ani Sinha <ani@anisinha.ca> wrote: > > > > On Tue, Sep 27, 2022 at 3:48 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > > > > > > On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > > > > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > > > > > > > can fix the rest of the bits later incrementally. > > > > > > > > > > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > > > > > > > keep the binaries? > > > > > > > > > > > > > > > > Can we please conclude on this? > > > > > > > > Peter, can you please fork the repo? I have tried many times to reach > > > > > > > > you on IRC but failed. > > > > > > > > > > > > > > Probably because of travel around KVM forum. > > > > > > > > > > > > > > I think given our CI is under pressure again due to gitlab free tier > > > > > > > limits, tying binaries to CI isn't a great idea at this stage. > > > > > > > Can Ani just upload binaies to qemu.org for now? > > > > > > > > > > > > I agree with Michael here. Having a full ci/cd job for this is > > > > > > overkill IMHO. We should create a repo just for the binaries, have a > > > > > > README there to explain how we generate them and check in new versions > > > > > > as and when needed (it won't be frequent). > > > > > > How about biosbits-bin repo? > > > > > > > > > > If QEMU is hosting binaries, where any part contains GPL code, then we > > > > > need to be providing the full and corresponding source and the build > > > > > scripts needed to re-create the binary. Once we have such scripts it > > > > > should be trivial to trigger that from a CI job. If it isn't then > > > > > we're doing something wrong. > > > > > > > > I was thinking of commiting the build scripts in a branch of > > > > https://gitlab.com/qemu-project/biosbits-bits. > > > > This would separate them from the main source. The scripts would build > > > > a version of qemu-bits based on the version information passed to it > > > > from the environment. > > > > Before I committed the scripts, I wanted to check whether we would > > > > want to do that or have a separate repo containing the binaries and > > > > the build scripts. > > > > Seems we want the former. > > > > > > > > As for the gitlab-ci, I looked at the yaml file and the qemu ones > > > > looks quite complicated. Can someone help me generate one based on the > > > > build script here? > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > > > > > Yes, QEMU's rules aren't a good place to start if you're trying > > > to learn gitlab CI, as they're very advanced. > > > > > > The simple case though is quite simple. > > > > > > * You need a container image to act as the build env > > > * In 'before_script' install any packages you need on top of the > > > base container image for build deps > > > * In 'script' run whatever shell commands you need in order > > > to build the project > > > * Add an 'artifacts' section to publish one (or more) files/dirs > > > as output > > > > > > The simplest example would be something like > > > > > > mybuild: > > > image: fedora:36 > > > before_script: > > > - dnf install -y gcc > > > script: > > > - gcc -o myapp myapp.c > > > artifacts > > > paths: > > > - myapp > > > > > > > How does this look? > > https://pastebin.com/0YyLFmi3 > > Alright, .gitlab-ci.yml is produced and the pipeline succeeds. > However, the question still remains, where do we keep the generated > artifacts? The following link will always reflect the published artifacts from the most recently fully successful CI pipeline, on the 'qemu-bits' branch, and 'qemu-bits-build' CI job: https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build Tweak as needed if you push the CI to master branch instead. This link can be considered the permanent home of the artifact. I'd just suggest that the QEMU job automatically skip if it fails to download the artifact, as occassionally transient infra errors can impact it. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 15:45 ` Daniel P. Berrangé @ 2022-09-27 21:21 ` Michael S. Tsirkin 2022-09-28 8:31 ` Thomas Huth 0 siblings, 1 reply; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-27 21:21 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Ani Sinha, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote: > On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote: > > On Tue, Sep 27, 2022 at 5:12 PM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > On Tue, Sep 27, 2022 at 3:48 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > > > > > > > > On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > > > > > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > > > > > > > > can fix the rest of the bits later incrementally. > > > > > > > > > > > > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > > > > > > > > keep the binaries? > > > > > > > > > > > > > > > > > > Can we please conclude on this? > > > > > > > > > Peter, can you please fork the repo? I have tried many times to reach > > > > > > > > > you on IRC but failed. > > > > > > > > > > > > > > > > Probably because of travel around KVM forum. > > > > > > > > > > > > > > > > I think given our CI is under pressure again due to gitlab free tier > > > > > > > > limits, tying binaries to CI isn't a great idea at this stage. > > > > > > > > Can Ani just upload binaies to qemu.org for now? > > > > > > > > > > > > > > I agree with Michael here. Having a full ci/cd job for this is > > > > > > > overkill IMHO. We should create a repo just for the binaries, have a > > > > > > > README there to explain how we generate them and check in new versions > > > > > > > as and when needed (it won't be frequent). > > > > > > > How about biosbits-bin repo? > > > > > > > > > > > > If QEMU is hosting binaries, where any part contains GPL code, then we > > > > > > need to be providing the full and corresponding source and the build > > > > > > scripts needed to re-create the binary. Once we have such scripts it > > > > > > should be trivial to trigger that from a CI job. If it isn't then > > > > > > we're doing something wrong. > > > > > > > > > > I was thinking of commiting the build scripts in a branch of > > > > > https://gitlab.com/qemu-project/biosbits-bits. > > > > > This would separate them from the main source. The scripts would build > > > > > a version of qemu-bits based on the version information passed to it > > > > > from the environment. > > > > > Before I committed the scripts, I wanted to check whether we would > > > > > want to do that or have a separate repo containing the binaries and > > > > > the build scripts. > > > > > Seems we want the former. > > > > > > > > > > As for the gitlab-ci, I looked at the yaml file and the qemu ones > > > > > looks quite complicated. Can someone help me generate one based on the > > > > > build script here? > > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > > > > > > > Yes, QEMU's rules aren't a good place to start if you're trying > > > > to learn gitlab CI, as they're very advanced. > > > > > > > > The simple case though is quite simple. > > > > > > > > * You need a container image to act as the build env > > > > * In 'before_script' install any packages you need on top of the > > > > base container image for build deps > > > > * In 'script' run whatever shell commands you need in order > > > > to build the project > > > > * Add an 'artifacts' section to publish one (or more) files/dirs > > > > as output > > > > > > > > The simplest example would be something like > > > > > > > > mybuild: > > > > image: fedora:36 > > > > before_script: > > > > - dnf install -y gcc > > > > script: > > > > - gcc -o myapp myapp.c > > > > artifacts > > > > paths: > > > > - myapp > > > > > > > > > > How does this look? > > > https://pastebin.com/0YyLFmi3 > > > > Alright, .gitlab-ci.yml is produced and the pipeline succeeds. > > However, the question still remains, where do we keep the generated > > artifacts? > > The following link will always reflect the published artifacts from > the most recently fully successful CI pipeline, on the 'qemu-bits' > branch, and 'qemu-bits-build' CI job: > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build > > Tweak as needed if you push the CI to master branch instead. This > link can be considered the permanent home of the artifact. I'd just > suggest that the QEMU job automatically skip if it fails to download > the artifact, as occassionally transient infra errors can impact > it. > > With regards, > Daniel This just means once we change the test old qemu source can no longer use it. Why is this a good idea? Are we so short on disk space? I thought CPU is the limiting factor? > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 21:21 ` Michael S. Tsirkin @ 2022-09-28 8:31 ` Thomas Huth 2022-09-28 8:35 ` Daniel P. Berrangé 2022-09-28 9:35 ` Michael S. Tsirkin 0 siblings, 2 replies; 90+ messages in thread From: Thomas Huth @ 2022-09-28 8:31 UTC (permalink / raw) To: Michael S. Tsirkin, Daniel P. Berrangé Cc: Ani Sinha, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel On 27/09/2022 23.21, Michael S. Tsirkin wrote: > On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote: >> On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote: ... >>> Alright, .gitlab-ci.yml is produced and the pipeline succeeds. >>> However, the question still remains, where do we keep the generated >>> artifacts? >> >> The following link will always reflect the published artifacts from >> the most recently fully successful CI pipeline, on the 'qemu-bits' >> branch, and 'qemu-bits-build' CI job: >> >> https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build >> >> Tweak as needed if you push the CI to master branch instead. This >> link can be considered the permanent home of the artifact. I'd just >> suggest that the QEMU job automatically skip if it fails to download >> the artifact, as occassionally transient infra errors can impact >> it. > > This just means once we change the test old qemu source can no longer use it. > Why is this a good idea? Are we so short on disk space? I thought CPU > is the limiting factor? FYI, we'll soon be short on disk space, gitlab plans to introduce storage limits: https://about.gitlab.com/pricing/faq-paid-storage-transfer/ Thomas ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 8:31 ` Thomas Huth @ 2022-09-28 8:35 ` Daniel P. Berrangé 2022-09-28 9:47 ` Michael S. Tsirkin 2022-09-28 9:35 ` Michael S. Tsirkin 1 sibling, 1 reply; 90+ messages in thread From: Daniel P. Berrangé @ 2022-09-28 8:35 UTC (permalink / raw) To: Thomas Huth Cc: Michael S. Tsirkin, Ani Sinha, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel On Wed, Sep 28, 2022 at 10:31:39AM +0200, Thomas Huth wrote: > On 27/09/2022 23.21, Michael S. Tsirkin wrote: > > On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote: > > > On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote: > ... > > > > Alright, .gitlab-ci.yml is produced and the pipeline succeeds. > > > > However, the question still remains, where do we keep the generated > > > > artifacts? > > > > > > The following link will always reflect the published artifacts from > > > the most recently fully successful CI pipeline, on the 'qemu-bits' > > > branch, and 'qemu-bits-build' CI job: > > > > > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build > > > > > > Tweak as needed if you push the CI to master branch instead. This > > > link can be considered the permanent home of the artifact. I'd just > > > suggest that the QEMU job automatically skip if it fails to download > > > the artifact, as occassionally transient infra errors can impact > > > it. > > > > This just means once we change the test old qemu source can no longer use it. > > Why is this a good idea? Are we so short on disk space? I thought CPU > > is the limiting factor? > > FYI, we'll soon be short on disk space, gitlab plans to introduce storage > limits: > > https://about.gitlab.com/pricing/faq-paid-storage-transfer/ That's the key reason I prefer the binary as CI artifact rather than in Git. Once checked into git, you can never reclaim that storage usage, as the git repo is append only, only option is to delete the repo and recreate. With CI artifacts we can control exactly which binaries consume storage quota at any time. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 8:35 ` Daniel P. Berrangé @ 2022-09-28 9:47 ` Michael S. Tsirkin 0 siblings, 0 replies; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-28 9:47 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Thomas Huth, Ani Sinha, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel On Wed, Sep 28, 2022 at 09:35:59AM +0100, Daniel P. Berrangé wrote: > On Wed, Sep 28, 2022 at 10:31:39AM +0200, Thomas Huth wrote: > > On 27/09/2022 23.21, Michael S. Tsirkin wrote: > > > On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote: > > > > On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote: > > ... > > > > > Alright, .gitlab-ci.yml is produced and the pipeline succeeds. > > > > > However, the question still remains, where do we keep the generated > > > > > artifacts? > > > > > > > > The following link will always reflect the published artifacts from > > > > the most recently fully successful CI pipeline, on the 'qemu-bits' > > > > branch, and 'qemu-bits-build' CI job: > > > > > > > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build > > > > > > > > Tweak as needed if you push the CI to master branch instead. This > > > > link can be considered the permanent home of the artifact. I'd just > > > > suggest that the QEMU job automatically skip if it fails to download > > > > the artifact, as occassionally transient infra errors can impact > > > > it. > > > > > > This just means once we change the test old qemu source can no longer use it. > > > Why is this a good idea? Are we so short on disk space? I thought CPU > > > is the limiting factor? > > > > FYI, we'll soon be short on disk space, gitlab plans to introduce storage > > limits: > > > > https://about.gitlab.com/pricing/faq-paid-storage-transfer/ > > That's the key reason I prefer the binary as CI artifact rather than > in Git. Once checked into git, you can never reclaim that storage > usage, as the git repo is append only, only option is to delete the > repo and recreate. With CI artifacts we can control exactly which > binaries consume storage quota at any time. > > With regards, > Daniel I agree binaries in git are a bit of a hack. But I also feel managing files as part of a test tool is a hack too, it's an SCM issue. How about e.g. git-lfs? Seems to be reasonably well supported on gitlab. There's also gitlab but that seems to be older. > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 8:31 ` Thomas Huth 2022-09-28 8:35 ` Daniel P. Berrangé @ 2022-09-28 9:35 ` Michael S. Tsirkin 2022-09-28 9:39 ` Thomas Huth 1 sibling, 1 reply; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-28 9:35 UTC (permalink / raw) To: Thomas Huth Cc: Daniel P. Berrangé, Ani Sinha, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel On Wed, Sep 28, 2022 at 10:31:39AM +0200, Thomas Huth wrote: > On 27/09/2022 23.21, Michael S. Tsirkin wrote: > > On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote: > > > On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote: > ... > > > > Alright, .gitlab-ci.yml is produced and the pipeline succeeds. > > > > However, the question still remains, where do we keep the generated > > > > artifacts? > > > > > > The following link will always reflect the published artifacts from > > > the most recently fully successful CI pipeline, on the 'qemu-bits' > > > branch, and 'qemu-bits-build' CI job: > > > > > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build > > > > > > Tweak as needed if you push the CI to master branch instead. This > > > link can be considered the permanent home of the artifact. I'd just > > > suggest that the QEMU job automatically skip if it fails to download > > > the artifact, as occassionally transient infra errors can impact > > > it. > > > > This just means once we change the test old qemu source can no longer use it. > > Why is this a good idea? Are we so short on disk space? I thought CPU > > is the limiting factor? > > FYI, we'll soon be short on disk space, gitlab plans to introduce storage > limits: > > https://about.gitlab.com/pricing/faq-paid-storage-transfer/ > > Thomas A good reason not to use CI artifacts to store images maybe? I was proposing storing binaries on qemu.org not on gitlab. -- MST ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 9:35 ` Michael S. Tsirkin @ 2022-09-28 9:39 ` Thomas Huth 2022-09-28 9:56 ` Michael S. Tsirkin 0 siblings, 1 reply; 90+ messages in thread From: Thomas Huth @ 2022-09-28 9:39 UTC (permalink / raw) To: Michael S. Tsirkin, Stefan Hajnoczi, pbonzini Cc: Daniel P. Berrangé, Ani Sinha, imammedo, jsnow, peter.maydell, qemu-devel On 28/09/2022 11.35, Michael S. Tsirkin wrote: > On Wed, Sep 28, 2022 at 10:31:39AM +0200, Thomas Huth wrote: >> On 27/09/2022 23.21, Michael S. Tsirkin wrote: >>> On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote: >>>> On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote: >> ... >>>>> Alright, .gitlab-ci.yml is produced and the pipeline succeeds. >>>>> However, the question still remains, where do we keep the generated >>>>> artifacts? >>>> >>>> The following link will always reflect the published artifacts from >>>> the most recently fully successful CI pipeline, on the 'qemu-bits' >>>> branch, and 'qemu-bits-build' CI job: >>>> >>>> https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build >>>> >>>> Tweak as needed if you push the CI to master branch instead. This >>>> link can be considered the permanent home of the artifact. I'd just >>>> suggest that the QEMU job automatically skip if it fails to download >>>> the artifact, as occassionally transient infra errors can impact >>>> it. >>> >>> This just means once we change the test old qemu source can no longer use it. >>> Why is this a good idea? Are we so short on disk space? I thought CPU >>> is the limiting factor? >> >> FYI, we'll soon be short on disk space, gitlab plans to introduce storage >> limits: >> >> https://about.gitlab.com/pricing/faq-paid-storage-transfer/ >> >> Thomas > > A good reason not to use CI artifacts to store images maybe? > I was proposing storing binaries on qemu.org not on gitlab. For qemu.org, you should maybe talk to Paolo and Stefan first, I'm not sure whether we could allow additional network traffic beside the normal release tarballs there... Thomas ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 9:39 ` Thomas Huth @ 2022-09-28 9:56 ` Michael S. Tsirkin 2022-10-06 7:58 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-28 9:56 UTC (permalink / raw) To: Thomas Huth Cc: Stefan Hajnoczi, pbonzini, Daniel P. Berrangé, Ani Sinha, imammedo, jsnow, peter.maydell, qemu-devel On Wed, Sep 28, 2022 at 11:39:36AM +0200, Thomas Huth wrote: > On 28/09/2022 11.35, Michael S. Tsirkin wrote: > > On Wed, Sep 28, 2022 at 10:31:39AM +0200, Thomas Huth wrote: > > > On 27/09/2022 23.21, Michael S. Tsirkin wrote: > > > > On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote: > > > > > On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote: > > > ... > > > > > > Alright, .gitlab-ci.yml is produced and the pipeline succeeds. > > > > > > However, the question still remains, where do we keep the generated > > > > > > artifacts? > > > > > > > > > > The following link will always reflect the published artifacts from > > > > > the most recently fully successful CI pipeline, on the 'qemu-bits' > > > > > branch, and 'qemu-bits-build' CI job: > > > > > > > > > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build > > > > > > > > > > Tweak as needed if you push the CI to master branch instead. This > > > > > link can be considered the permanent home of the artifact. I'd just > > > > > suggest that the QEMU job automatically skip if it fails to download > > > > > the artifact, as occassionally transient infra errors can impact > > > > > it. > > > > > > > > This just means once we change the test old qemu source can no longer use it. > > > > Why is this a good idea? Are we so short on disk space? I thought CPU > > > > is the limiting factor? > > > > > > FYI, we'll soon be short on disk space, gitlab plans to introduce storage > > > limits: > > > > > > https://about.gitlab.com/pricing/faq-paid-storage-transfer/ > > > > > > Thomas > > > > A good reason not to use CI artifacts to store images maybe? > > I was proposing storing binaries on qemu.org not on gitlab. > > For qemu.org, you should maybe talk to Paolo and Stefan first, I'm not sure > whether we could allow additional network traffic > beside the normal release tarballs there... > > Thomas I guess we need to design this sensibly to checksum local files and only fetch if there's change, and that only for people who work on ACPI. -- MST ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 9:56 ` Michael S. Tsirkin @ 2022-10-06 7:58 ` Ani Sinha 2022-10-06 8:11 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-10-06 7:58 UTC (permalink / raw) To: Michael S. Tsirkin Cc: Thomas Huth, Stefan Hajnoczi, pbonzini, Daniel P. Berrangé, Ani Sinha, imammedo, jsnow, peter.maydell, qemu-devel [-- Attachment #1: Type: text/plain, Size: 3178 bytes --] On Wed, 28 Sep 2022, Michael S. Tsirkin wrote: > On Wed, Sep 28, 2022 at 11:39:36AM +0200, Thomas Huth wrote: > > On 28/09/2022 11.35, Michael S. Tsirkin wrote: > > > On Wed, Sep 28, 2022 at 10:31:39AM +0200, Thomas Huth wrote: > > > > On 27/09/2022 23.21, Michael S. Tsirkin wrote: > > > > > On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote: > > > > > > On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote: > > > > ... > > > > > > > Alright, .gitlab-ci.yml is produced and the pipeline succeeds. > > > > > > > However, the question still remains, where do we keep the generated > > > > > > > artifacts? > > > > > > > > > > > > The following link will always reflect the published artifacts from > > > > > > the most recently fully successful CI pipeline, on the 'qemu-bits' > > > > > > branch, and 'qemu-bits-build' CI job: > > > > > > > > > > > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build > > > > > > > > > > > > Tweak as needed if you push the CI to master branch instead. This > > > > > > link can be considered the permanent home of the artifact. I'd just > > > > > > suggest that the QEMU job automatically skip if it fails to download > > > > > > the artifact, as occassionally transient infra errors can impact > > > > > > it. > > > > > > > > > > This just means once we change the test old qemu source can no longer use it. > > > > > Why is this a good idea? Are we so short on disk space? I thought CPU > > > > > is the limiting factor? I did some expriments and it seems we can keep latest artifacts for every tagged release of bits. So I have adjusted the yaml file so that everytime I push a new tag, a build is triggered and the artifacts are preserved without expiry. Ofcourse for non-tagged changes, one can trigger the build manually from the web UI as well. For exmaple, this link https://gitlab.com/qemu-project/biosbits-bits/-/jobs/3134519120/artifacts/download?file_type=archive should download the current artifacts for the tag qemu-bits-latest. What I am not sure is how to get a downloadable link for the latest build for a particular tag without the numeric job ID (which can change across builds)? So for example, we can have a consistent URLs to download archives for every tagged releases and then the test can choose which tagged release to use. If we can have this then its as good as keeping binaries in a version control system like git. > > > > > > > > FYI, we'll soon be short on disk space, gitlab plans to introduce storage > > > > limits: > > > > > > > > https://about.gitlab.com/pricing/faq-paid-storage-transfer/ > > > > > > > > Thomas > > > > > > A good reason not to use CI artifacts to store images maybe? > > > I was proposing storing binaries on qemu.org not on gitlab. > > > > For qemu.org, you should maybe talk to Paolo and Stefan first, I'm not sure > > whether we could allow additional network traffic > > beside the normal release tarballs there... > > > > Thomas > > I guess we need to design this sensibly to checksum local files > and only fetch if there's change, and that only for > people who work on ACPI. > > -- > MST > > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-10-06 7:58 ` Ani Sinha @ 2022-10-06 8:11 ` Ani Sinha 2022-10-06 10:18 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-10-06 8:11 UTC (permalink / raw) To: Ani Sinha Cc: Michael S. Tsirkin, Thomas Huth, Stefan Hajnoczi, pbonzini, Daniel P. Berrangé, imammedo, jsnow, peter.maydell, qemu-devel [-- Attachment #1: Type: text/plain, Size: 3923 bytes --] On Thu, 6 Oct 2022, Ani Sinha wrote: > > > On Wed, 28 Sep 2022, Michael S. Tsirkin wrote: > > > On Wed, Sep 28, 2022 at 11:39:36AM +0200, Thomas Huth wrote: > > > On 28/09/2022 11.35, Michael S. Tsirkin wrote: > > > > On Wed, Sep 28, 2022 at 10:31:39AM +0200, Thomas Huth wrote: > > > > > On 27/09/2022 23.21, Michael S. Tsirkin wrote: > > > > > > On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote: > > > > > > > On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote: > > > > > ... > > > > > > > > Alright, .gitlab-ci.yml is produced and the pipeline succeeds. > > > > > > > > However, the question still remains, where do we keep the generated > > > > > > > > artifacts? > > > > > > > > > > > > > > The following link will always reflect the published artifacts from > > > > > > > the most recently fully successful CI pipeline, on the 'qemu-bits' > > > > > > > branch, and 'qemu-bits-build' CI job: > > > > > > > > > > > > > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build > > > > > > > > > > > > > > Tweak as needed if you push the CI to master branch instead. This > > > > > > > link can be considered the permanent home of the artifact. I'd just > > > > > > > suggest that the QEMU job automatically skip if it fails to download > > > > > > > the artifact, as occassionally transient infra errors can impact > > > > > > > it. > > > > > > > > > > > > This just means once we change the test old qemu source can no longer use it. > > > > > > Why is this a good idea? Are we so short on disk space? I thought CPU > > > > > > is the limiting factor? > > > I did some expriments and it seems we can keep latest artifacts for every > tagged release of bits. So I have adjusted the yaml file so that everytime > I push a new tag, a build is > triggered and the artifacts are preserved without expiry. Ofcourse for > non-tagged changes, one can trigger the build manually from the web UI as > well. > > For exmaple, this link > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/3134519120/artifacts/download?file_type=archive > should download the current artifacts for the tag qemu-bits-latest. > > What I am not sure is how to get a downloadable link for the latest build > for a particular tag without the numeric job ID (which can change across > builds)? So for example, we can have a consistent URLs to download > archives > for every tagged releases and then the test can choose which tagged > release to > use. If we can have this then its as good as keeping binaries in a version > control system like git. To answer my own question, this is the URL for the qemu-bits-latest tag: https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits-latest/download?job=qemu-bits-build which is the same as https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits-09272022/download?job=qemu-bits-build currently. If the latest version of bits changes, we can make "qemu-bits-latest" tag always point to the latest version while artifacts for the older tagged releases will continue to be available. danPB, please correct if I am mistaken. > > > > > > > > > > > FYI, we'll soon be short on disk space, gitlab plans to introduce storage > > > > > limits: > > > > > > > > > > https://about.gitlab.com/pricing/faq-paid-storage-transfer/ > > > > > > > > > > Thomas > > > > > > > > A good reason not to use CI artifacts to store images maybe? > > > > I was proposing storing binaries on qemu.org not on gitlab. > > > > > > For qemu.org, you should maybe talk to Paolo and Stefan first, I'm not sure > > > whether we could allow additional network traffic > > > beside the normal release tarballs there... > > > > > > Thomas > > > > I guess we need to design this sensibly to checksum local files > > and only fetch if there's change, and that only for > > people who work on ACPI. > > > > -- > > MST > > > > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-10-06 8:11 ` Ani Sinha @ 2022-10-06 10:18 ` Ani Sinha 0 siblings, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-10-06 10:18 UTC (permalink / raw) To: Ani Sinha Cc: Michael S. Tsirkin, Thomas Huth, Stefan Hajnoczi, pbonzini, Daniel P. Berrangé, imammedo, jsnow, peter.maydell, qemu-devel [-- Attachment #1: Type: text/plain, Size: 4531 bytes --] On Thu, 6 Oct 2022, Ani Sinha wrote: > > > On Thu, 6 Oct 2022, Ani Sinha wrote: > > > > > > > On Wed, 28 Sep 2022, Michael S. Tsirkin wrote: > > > > > On Wed, Sep 28, 2022 at 11:39:36AM +0200, Thomas Huth wrote: > > > > On 28/09/2022 11.35, Michael S. Tsirkin wrote: > > > > > On Wed, Sep 28, 2022 at 10:31:39AM +0200, Thomas Huth wrote: > > > > > > On 27/09/2022 23.21, Michael S. Tsirkin wrote: > > > > > > > On Tue, Sep 27, 2022 at 04:45:09PM +0100, Daniel P. Berrangé wrote: > > > > > > > > On Tue, Sep 27, 2022 at 07:35:13PM +0530, Ani Sinha wrote: > > > > > > ... > > > > > > > > > Alright, .gitlab-ci.yml is produced and the pipeline succeeds. > > > > > > > > > However, the question still remains, where do we keep the generated > > > > > > > > > artifacts? > > > > > > > > > > > > > > > > The following link will always reflect the published artifacts from > > > > > > > > the most recently fully successful CI pipeline, on the 'qemu-bits' > > > > > > > > branch, and 'qemu-bits-build' CI job: > > > > > > > > > > > > > > > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits/download?job=qemu-bits-build > > > > > > > > > > > > > > > > Tweak as needed if you push the CI to master branch instead. This > > > > > > > > link can be considered the permanent home of the artifact. I'd just > > > > > > > > suggest that the QEMU job automatically skip if it fails to download > > > > > > > > the artifact, as occassionally transient infra errors can impact > > > > > > > > it. > > > > > > > > > > > > > > This just means once we change the test old qemu source can no longer use it. > > > > > > > Why is this a good idea? Are we so short on disk space? I thought CPU > > > > > > > is the limiting factor? > > > > > > I did some expriments and it seems we can keep latest artifacts for every > > tagged release of bits. So I have adjusted the yaml file so that everytime > > I push a new tag, a build is > > triggered and the artifacts are preserved without expiry. Ofcourse for > > non-tagged changes, one can trigger the build manually from the web UI as > > well. > > > > For exmaple, this link > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/3134519120/artifacts/download?file_type=archive > > should download the current artifacts for the tag qemu-bits-latest. > > > > What I am not sure is how to get a downloadable link for the latest build > > for a particular tag without the numeric job ID (which can change across > > builds)? So for example, we can have a consistent URLs to download > > archives > > for every tagged releases and then the test can choose which tagged > > release to > > use. If we can have this then its as good as keeping binaries in a version > > control system like git. > > > To answer my own question, this is the URL for the qemu-bits-latest tag: > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits-latest/download?job=qemu-bits-build > > which is the same as > > https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits-09272022/download?job=qemu-bits-build > > currently. > > If the latest version of bits changes, we can make "qemu-bits-latest" tag > always point to the latest version while artifacts for the older tagged > releases will continue to be available. > > danPB, please correct if I am mistaken. Seems this answers it: https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#access-the-latest-job-artifacts-by-url So even we can download individual files from the artifact: https://gitlab.com/qemu-project/biosbits-bits/-/jobs/artifacts/qemu-bits-latest/raw/bits-2020-e40af4a7-grub.tar.gz?job=qemu-bits-build where the ref can be a tag or a branch. This makes me happy. > > > > > > > > > > > > > > > FYI, we'll soon be short on disk space, gitlab plans to introduce storage > > > > > > limits: > > > > > > > > > > > > https://about.gitlab.com/pricing/faq-paid-storage-transfer/ > > > > > > > > > > > > Thomas > > > > > > > > > > A good reason not to use CI artifacts to store images maybe? > > > > > I was proposing storing binaries on qemu.org not on gitlab. > > > > > > > > For qemu.org, you should maybe talk to Paolo and Stefan first, I'm not sure > > > > whether we could allow additional network traffic > > > > beside the normal release tarballs there... > > > > > > > > Thomas > > > > > > I guess we need to design this sensibly to checksum local files > > > and only fetch if there's change, and that only for > > > people who work on ACPI. > > > > > > -- > > > MST > > > > > > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 10:07 ` Ani Sinha 2022-09-27 10:18 ` Daniel P. Berrangé @ 2022-09-27 21:19 ` Michael S. Tsirkin 1 sibling, 0 replies; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-27 21:19 UTC (permalink / raw) To: Ani Sinha Cc: Daniel P. Berrangé, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Tue, Sep 27, 2022 at 03:37:39PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > > > > can fix the rest of the bits later incrementally. > > > > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > > > > keep the binaries? > > > > > > > > > > Can we please conclude on this? > > > > > Peter, can you please fork the repo? I have tried many times to reach > > > > > you on IRC but failed. > > > > > > > > Probably because of travel around KVM forum. > > > > > > > > I think given our CI is under pressure again due to gitlab free tier > > > > limits, tying binaries to CI isn't a great idea at this stage. > > > > Can Ani just upload binaies to qemu.org for now? > > > > > > I agree with Michael here. Having a full ci/cd job for this is > > > overkill IMHO. We should create a repo just for the binaries, have a > > > README there to explain how we generate them and check in new versions > > > as and when needed (it won't be frequent). > > > How about biosbits-bin repo? > > > > If QEMU is hosting binaries, where any part contains GPL code, then we > > need to be providing the full and corresponding source and the build > > scripts needed to re-create the binary. Once we have such scripts it > > should be trivial to trigger that from a CI job. If it isn't then > > we're doing something wrong. > > I was thinking of commiting the build scripts in a branch of > https://gitlab.com/qemu-project/biosbits-bits. > This would separate them from the main source. The scripts would build > a version of qemu-bits based on the version information passed to it > from the environment. > Before I committed the scripts, I wanted to check whether we would > want to do that or have a separate repo containing the binaries and > the build scripts. > Seems we want the former. A separate repo is standard imho. Don't see any advantages to abusing git branches like that. > As for the gitlab-ci, I looked at the yaml file and the qemu ones > looks quite complicated. Can someone help me generate one based on the > build script here? > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > > The CI quota is not an issue, because > > this is not a job that we need to run continuously. It can be triggered > > manually as & when we decide we need to refresh the binary, so would > > be a small one-off CI quota hit. > > > > Also note that gitlab is intending to start enforcing storage quota > > on projects in the not too distant future. This makes it unappealing > > to store binaries in git repos, unless we genuinely need the ability > > to access historical versions of the binary. I don't believe we need > > that for biosbits. > > > > The binary can be published as a CI artifact and accessed directly > > from the latest artifact download link. This ensures we only consume > > quota for the most recently published binary artifact. So I don't see > > a compelling reason to upload binaries into git. > > > > With regards, > > Daniel > > -- > > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > > |: https://libvirt.org -o- https://fstop138.berrange.com :| > > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 8:33 ` Daniel P. Berrangé 2022-09-27 10:07 ` Ani Sinha @ 2022-09-27 21:18 ` Michael S. Tsirkin 2022-09-28 3:08 ` Ani Sinha 2022-09-28 6:58 ` Daniel P. Berrangé 1 sibling, 2 replies; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-27 21:18 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Ani Sinha, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Tue, Sep 27, 2022 at 09:33:27AM +0100, Daniel P. Berrangé wrote: > On Tue, Sep 27, 2022 at 01:43:15PM +0530, Ani Sinha wrote: > > On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote: > > > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > > > > > > > > > > > > > On Mon, 25 Jul 2022, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote: > > > > > > > > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > > > > > > > > > Instead of all this mess, can't we just spawn e.g. "git clone --depth > > > > > > > > 1"? > > > > > > > > > > And if the directory exists I would fetch and checkout. > > > > > > > > > > > > > > > > > > There are two reasons I can think of why I do not like this idea: > > > > > > > > > > > > > > > > > > (a) a git clone of a whole directory would download all versions of the > > > > > > > > > binary whereas we want only a specific version. > > > > > > > > > > > > > > > > You mention shallow clone yourself, and I used --depth 1 above. > > > > > > > > > > > > > > > > > Downloading a single file > > > > > > > > > by shallow cloning or creating a git archive is overkill IMHO when a wget > > > > > > > > > style retrieval works just fine. > > > > > > > > > > > > > > > > However, it does not provide for versioning, tagging etc so you have > > > > > > > > to implement your own schema. > > > > > > > > > > > > > > > > > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own versioning mechanism and > > > > > > > > I think all we need to do is maintain the same versioning logic and maintain > > > > > > > > binaries of different versions. Do we really need the power of git/version > > > > > > > > control here? Dunno. > > > > > > > > > > > > > > Well we need some schema. Given we are not using official bits releases > > > > > > > I don't think we can reuse theirs. > > > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > > > can fix the rest of the bits later incrementally. > > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > > > keep the binaries? > > > > > > > > Can we please conclude on this? > > > > Peter, can you please fork the repo? I have tried many times to reach > > > > you on IRC but failed. > > > > > > Probably because of travel around KVM forum. > > > > > > I think given our CI is under pressure again due to gitlab free tier > > > limits, tying binaries to CI isn't a great idea at this stage. > > > Can Ani just upload binaies to qemu.org for now? > > > > I agree with Michael here. Having a full ci/cd job for this is > > overkill IMHO. We should create a repo just for the binaries, have a > > README there to explain how we generate them and check in new versions > > as and when needed (it won't be frequent). > > How about biosbits-bin repo? > > If QEMU is hosting binaries, where any part contains GPL code, then we > need to be providing the full and corresponding source and the build > scripts needed to re-create the binary. Once we have such scripts it > should be trivial to trigger that from a CI job. If it isn't then > we're doing something wrong. The CI quota is not an issue, because > this is not a job that we need to run continuously. It can be triggered > manually as & when we decide we need to refresh the binary, so would > be a small one-off CI quota hit. > > Also note that gitlab is intending to start enforcing storage quota > on projects in the not too distant future. This makes it unappealing > to store binaries in git repos, unless we genuinely need the ability > to access historical versions of the binary. I don't believe we need > that for biosbits. > > The binary can be published as a CI artifact and accessed directly > from the latest artifact download link. This ensures we only consume > quota for the most recently published binary artifact. So I don't see > a compelling reason to upload binaries into git. > > With regards, > Daniel I don't really care where we upload them but only having the latest version is just going to break anything expecting the old binary. > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 21:18 ` Michael S. Tsirkin @ 2022-09-28 3:08 ` Ani Sinha 2022-09-28 3:58 ` Michael S. Tsirkin 2022-09-28 6:58 ` Daniel P. Berrangé 1 sibling, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-09-28 3:08 UTC (permalink / raw) To: Michael S. Tsirkin Cc: Daniel P. Berrangé, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Wed, Sep 28, 2022 at 2:48 AM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Tue, Sep 27, 2022 at 09:33:27AM +0100, Daniel P. Berrangé wrote: > > On Tue, Sep 27, 2022 at 01:43:15PM +0530, Ani Sinha wrote: > > > On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote: > > > > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > > > > > > > > > > > > > > > > > On Mon, 25 Jul 2022, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote: > > > > > > > > > > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > > > > > > > > > > Instead of all this mess, can't we just spawn e.g. "git clone --depth > > > > > > > > > 1"? > > > > > > > > > > > And if the directory exists I would fetch and checkout. > > > > > > > > > > > > > > > > > > > > There are two reasons I can think of why I do not like this idea: > > > > > > > > > > > > > > > > > > > > (a) a git clone of a whole directory would download all versions of the > > > > > > > > > > binary whereas we want only a specific version. > > > > > > > > > > > > > > > > > > You mention shallow clone yourself, and I used --depth 1 above. > > > > > > > > > > > > > > > > > > > Downloading a single file > > > > > > > > > > by shallow cloning or creating a git archive is overkill IMHO when a wget > > > > > > > > > > style retrieval works just fine. > > > > > > > > > > > > > > > > > > However, it does not provide for versioning, tagging etc so you have > > > > > > > > > to implement your own schema. > > > > > > > > > > > > > > > > > > > > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own versioning mechanism and > > > > > > > > > I think all we need to do is maintain the same versioning logic and maintain > > > > > > > > > binaries of different versions. Do we really need the power of git/version > > > > > > > > > control here? Dunno. > > > > > > > > > > > > > > > > Well we need some schema. Given we are not using official bits releases > > > > > > > > I don't think we can reuse theirs. > > > > > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > > > > can fix the rest of the bits later incrementally. > > > > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > > > > keep the binaries? > > > > > > > > > > Can we please conclude on this? > > > > > Peter, can you please fork the repo? I have tried many times to reach > > > > > you on IRC but failed. > > > > > > > > Probably because of travel around KVM forum. > > > > > > > > I think given our CI is under pressure again due to gitlab free tier > > > > limits, tying binaries to CI isn't a great idea at this stage. > > > > Can Ani just upload binaies to qemu.org for now? > > > > > > I agree with Michael here. Having a full ci/cd job for this is > > > overkill IMHO. We should create a repo just for the binaries, have a > > > README there to explain how we generate them and check in new versions > > > as and when needed (it won't be frequent). > > > How about biosbits-bin repo? > > > > If QEMU is hosting binaries, where any part contains GPL code, then we > > need to be providing the full and corresponding source and the build > > scripts needed to re-create the binary. Once we have such scripts it > > should be trivial to trigger that from a CI job. If it isn't then > > we're doing something wrong. The CI quota is not an issue, because > > this is not a job that we need to run continuously. It can be triggered > > manually as & when we decide we need to refresh the binary, so would > > be a small one-off CI quota hit. > > > > Also note that gitlab is intending to start enforcing storage quota > > on projects in the not too distant future. This makes it unappealing > > to store binaries in git repos, unless we genuinely need the ability > > to access historical versions of the binary. I don't believe we need > > that for biosbits. > > > > The binary can be published as a CI artifact and accessed directly > > from the latest artifact download link. This ensures we only consume > > quota for the most recently published binary artifact. So I don't see > > a compelling reason to upload binaries into git. > > > > With regards, > > Daniel > > I don't really care where we upload them but only having the > latest version is just going to break anything expecting > the old binary. In fairness, I am not entirely certain if there is a tight coupling between the qemu tests and the bits binaries. I have written the test framework in a way such that test modifications and new tests can be pushed into the bits binaries and the iso gets regenerated with the new tests from QEMU itself before running the tests. Only when we need bits bugfixes or say upgrade to new acpica that we would need to regenerate the bits binaries. > > > > > -- > > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > > |: https://libvirt.org -o- https://fstop138.berrange.com :| > > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 3:08 ` Ani Sinha @ 2022-09-28 3:58 ` Michael S. Tsirkin 2022-09-28 6:06 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-28 3:58 UTC (permalink / raw) To: Ani Sinha Cc: Daniel P. Berrangé, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Wed, Sep 28, 2022 at 08:38:54AM +0530, Ani Sinha wrote: > > I don't really care where we upload them but only having the > > latest version is just going to break anything expecting > > the old binary. > > In fairness, I am not entirely certain if there is a tight coupling > between the qemu tests and the bits binaries. I have written the test > framework in a way such that test modifications and new tests can be > pushed into the bits binaries and the iso gets regenerated with the > new tests from QEMU itself before running the tests. Only when we need > bits bugfixes or say upgrade to new acpica that we would need to > regenerate the bits binaries. Theoretically, that's correct. But if we did not have bugs we would not need tests. -- MST ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 3:58 ` Michael S. Tsirkin @ 2022-09-28 6:06 ` Ani Sinha 2022-10-06 12:12 ` Michael S. Tsirkin 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-09-28 6:06 UTC (permalink / raw) To: Michael S. Tsirkin Cc: Daniel P. Berrangé, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Wed, Sep 28, 2022 at 9:28 AM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Wed, Sep 28, 2022 at 08:38:54AM +0530, Ani Sinha wrote: > > > I don't really care where we upload them but only having the > > > latest version is just going to break anything expecting > > > the old binary. > > > > In fairness, I am not entirely certain if there is a tight coupling > > between the qemu tests and the bits binaries. I have written the test > > framework in a way such that test modifications and new tests can be > > pushed into the bits binaries and the iso gets regenerated with the > > new tests from QEMU itself before running the tests. Only when we need > > bits bugfixes or say upgrade to new acpica that we would need to > > regenerate the bits binaries. > > Theoretically, that's correct. But if we did not have bugs we would > not need tests. Hmm, you might have a point. Curious, do we keep versioned binaries of edk for example? If so, why we can't do the same for bits? ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 6:06 ` Ani Sinha @ 2022-10-06 12:12 ` Michael S. Tsirkin 0 siblings, 0 replies; 90+ messages in thread From: Michael S. Tsirkin @ 2022-10-06 12:12 UTC (permalink / raw) To: Ani Sinha Cc: Daniel P. Berrangé, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Wed, Sep 28, 2022 at 11:36:51AM +0530, Ani Sinha wrote: > On Wed, Sep 28, 2022 at 9:28 AM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Wed, Sep 28, 2022 at 08:38:54AM +0530, Ani Sinha wrote: > > > > I don't really care where we upload them but only having the > > > > latest version is just going to break anything expecting > > > > the old binary. > > > > > > In fairness, I am not entirely certain if there is a tight coupling > > > between the qemu tests and the bits binaries. I have written the test > > > framework in a way such that test modifications and new tests can be > > > pushed into the bits binaries and the iso gets regenerated with the > > > new tests from QEMU itself before running the tests. Only when we need > > > bits bugfixes or say upgrade to new acpica that we would need to > > > regenerate the bits binaries. > > > > Theoretically, that's correct. But if we did not have bugs we would > > not need tests. > > Hmm, you might have a point. Curious, do we keep versioned binaries of > edk for example? We keep them in git, don't we? Under pc-bios. > If so, why we can't do the same for bits? What can I say - let's get this upstream in some form and then worry about improving the infrastructure. -- MST ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 21:18 ` Michael S. Tsirkin 2022-09-28 3:08 ` Ani Sinha @ 2022-09-28 6:58 ` Daniel P. Berrangé 2022-09-28 7:15 ` Ani Sinha 2022-09-28 9:19 ` Michael S. Tsirkin 1 sibling, 2 replies; 90+ messages in thread From: Daniel P. Berrangé @ 2022-09-28 6:58 UTC (permalink / raw) To: Michael S. Tsirkin Cc: Ani Sinha, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Tue, Sep 27, 2022 at 05:18:10PM -0400, Michael S. Tsirkin wrote: > On Tue, Sep 27, 2022 at 09:33:27AM +0100, Daniel P. Berrangé wrote: > > On Tue, Sep 27, 2022 at 01:43:15PM +0530, Ani Sinha wrote: > > > On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote: > > > > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > > > > > > > > > > > > > > > > > On Mon, 25 Jul 2022, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote: > > > > > > > > > > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > > > > > > > > > > Instead of all this mess, can't we just spawn e.g. "git clone --depth > > > > > > > > > 1"? > > > > > > > > > > > And if the directory exists I would fetch and checkout. > > > > > > > > > > > > > > > > > > > > There are two reasons I can think of why I do not like this idea: > > > > > > > > > > > > > > > > > > > > (a) a git clone of a whole directory would download all versions of the > > > > > > > > > > binary whereas we want only a specific version. > > > > > > > > > > > > > > > > > > You mention shallow clone yourself, and I used --depth 1 above. > > > > > > > > > > > > > > > > > > > Downloading a single file > > > > > > > > > > by shallow cloning or creating a git archive is overkill IMHO when a wget > > > > > > > > > > style retrieval works just fine. > > > > > > > > > > > > > > > > > > However, it does not provide for versioning, tagging etc so you have > > > > > > > > > to implement your own schema. > > > > > > > > > > > > > > > > > > > > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own versioning mechanism and > > > > > > > > > I think all we need to do is maintain the same versioning logic and maintain > > > > > > > > > binaries of different versions. Do we really need the power of git/version > > > > > > > > > control here? Dunno. > > > > > > > > > > > > > > > > Well we need some schema. Given we are not using official bits releases > > > > > > > > I don't think we can reuse theirs. > > > > > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > > > > can fix the rest of the bits later incrementally. > > > > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > > > > keep the binaries? > > > > > > > > > > Can we please conclude on this? > > > > > Peter, can you please fork the repo? I have tried many times to reach > > > > > you on IRC but failed. > > > > > > > > Probably because of travel around KVM forum. > > > > > > > > I think given our CI is under pressure again due to gitlab free tier > > > > limits, tying binaries to CI isn't a great idea at this stage. > > > > Can Ani just upload binaies to qemu.org for now? > > > > > > I agree with Michael here. Having a full ci/cd job for this is > > > overkill IMHO. We should create a repo just for the binaries, have a > > > README there to explain how we generate them and check in new versions > > > as and when needed (it won't be frequent). > > > How about biosbits-bin repo? > > > > If QEMU is hosting binaries, where any part contains GPL code, then we > > need to be providing the full and corresponding source and the build > > scripts needed to re-create the binary. Once we have such scripts it > > should be trivial to trigger that from a CI job. If it isn't then > > we're doing something wrong. The CI quota is not an issue, because > > this is not a job that we need to run continuously. It can be triggered > > manually as & when we decide we need to refresh the binary, so would > > be a small one-off CI quota hit. > > > > Also note that gitlab is intending to start enforcing storage quota > > on projects in the not too distant future. This makes it unappealing > > to store binaries in git repos, unless we genuinely need the ability > > to access historical versions of the binary. I don't believe we need > > that for biosbits. > > > > The binary can be published as a CI artifact and accessed directly > > from the latest artifact download link. This ensures we only consume > > quota for the most recently published binary artifact. So I don't see > > a compelling reason to upload binaries into git. > > I don't really care where we upload them but only having the > latest version is just going to break anything expecting > the old binary. biosbits isn't tied to QEMU versions, it is an entirely separate 3rd party project. This binary is just providing the test env, and IIUC, control over what executes in this env is still done by the QEMU side test scripts. I'm not seeing a coupling here that requires precise matching. In any case biosbit is a dead project so does not look likely to have any changes. If we did want to have different versions though, we can stil publish artifacts from different branches of biosbits code. Gitlab will preserve & publish the latest artifacts from each branch in parallel. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 6:58 ` Daniel P. Berrangé @ 2022-09-28 7:15 ` Ani Sinha 2022-09-28 7:26 ` Daniel P. Berrangé 2022-09-28 9:19 ` Michael S. Tsirkin 1 sibling, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-09-28 7:15 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Michael S. Tsirkin, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Wed, Sep 28, 2022 at 12:28 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Tue, Sep 27, 2022 at 05:18:10PM -0400, Michael S. Tsirkin wrote: > > On Tue, Sep 27, 2022 at 09:33:27AM +0100, Daniel P. Berrangé wrote: > > > On Tue, Sep 27, 2022 at 01:43:15PM +0530, Ani Sinha wrote: > > > > On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote: > > > > > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, 25 Jul 2022, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote: > > > > > > > > > > > > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > > > > > > > > > > > Instead of all this mess, can't we just spawn e.g. "git clone --depth > > > > > > > > > > 1"? > > > > > > > > > > > > And if the directory exists I would fetch and checkout. > > > > > > > > > > > > > > > > > > > > > > There are two reasons I can think of why I do not like this idea: > > > > > > > > > > > > > > > > > > > > > > (a) a git clone of a whole directory would download all versions of the > > > > > > > > > > > binary whereas we want only a specific version. > > > > > > > > > > > > > > > > > > > > You mention shallow clone yourself, and I used --depth 1 above. > > > > > > > > > > > > > > > > > > > > > Downloading a single file > > > > > > > > > > > by shallow cloning or creating a git archive is overkill IMHO when a wget > > > > > > > > > > > style retrieval works just fine. > > > > > > > > > > > > > > > > > > > > However, it does not provide for versioning, tagging etc so you have > > > > > > > > > > to implement your own schema. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own versioning mechanism and > > > > > > > > > > I think all we need to do is maintain the same versioning logic and maintain > > > > > > > > > > binaries of different versions. Do we really need the power of git/version > > > > > > > > > > control here? Dunno. > > > > > > > > > > > > > > > > > > Well we need some schema. Given we are not using official bits releases > > > > > > > > > I don't think we can reuse theirs. > > > > > > > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > > > > > can fix the rest of the bits later incrementally. > > > > > > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > > > > > keep the binaries? > > > > > > > > > > > > Can we please conclude on this? > > > > > > Peter, can you please fork the repo? I have tried many times to reach > > > > > > you on IRC but failed. > > > > > > > > > > Probably because of travel around KVM forum. > > > > > > > > > > I think given our CI is under pressure again due to gitlab free tier > > > > > limits, tying binaries to CI isn't a great idea at this stage. > > > > > Can Ani just upload binaies to qemu.org for now? > > > > > > > > I agree with Michael here. Having a full ci/cd job for this is > > > > overkill IMHO. We should create a repo just for the binaries, have a > > > > README there to explain how we generate them and check in new versions > > > > as and when needed (it won't be frequent). > > > > How about biosbits-bin repo? > > > > > > If QEMU is hosting binaries, where any part contains GPL code, then we > > > need to be providing the full and corresponding source and the build > > > scripts needed to re-create the binary. Once we have such scripts it > > > should be trivial to trigger that from a CI job. If it isn't then > > > we're doing something wrong. The CI quota is not an issue, because > > > this is not a job that we need to run continuously. It can be triggered > > > manually as & when we decide we need to refresh the binary, so would > > > be a small one-off CI quota hit. > > > > > > Also note that gitlab is intending to start enforcing storage quota > > > on projects in the not too distant future. This makes it unappealing > > > to store binaries in git repos, unless we genuinely need the ability > > > to access historical versions of the binary. I don't believe we need > > > that for biosbits. > > > > > > The binary can be published as a CI artifact and accessed directly > > > from the latest artifact download link. This ensures we only consume > > > quota for the most recently published binary artifact. So I don't see > > > a compelling reason to upload binaries into git. > > > > I don't really care where we upload them but only having the > > latest version is just going to break anything expecting > > the old binary. > > biosbits isn't tied to QEMU versions, it is an entirely separate 3rd > party project. This binary is just providing the test env, and IIUC, > control over what executes in this env is still done by the QEMU side > test scripts. I'm not seeing a coupling here that requires precise > matching. In any case biosbit is a dead project so does not look > likely to have any changes. > > If we did want to have different versions though, we can stil > publish artifacts from different branches of biosbits code. No, that is just ridiculous. Say we have a bug in bits that we fixed and released a new version. Do we now create a new branch for that? Multiple branches makes things needlessly complicated. We have one branch, qemu-bits and all fixes go into that branch. We can have different tags if we need. Nothing beyond that. Gitlab > will preserve & publish the latest artifacts from each branch in > parallel. > > With regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 7:15 ` Ani Sinha @ 2022-09-28 7:26 ` Daniel P. Berrangé 0 siblings, 0 replies; 90+ messages in thread From: Daniel P. Berrangé @ 2022-09-28 7:26 UTC (permalink / raw) To: Ani Sinha Cc: Michael S. Tsirkin, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Wed, Sep 28, 2022 at 12:45:46PM +0530, Ani Sinha wrote: > On Wed, Sep 28, 2022 at 12:28 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > > > > On Tue, Sep 27, 2022 at 05:18:10PM -0400, Michael S. Tsirkin wrote: > > > On Tue, Sep 27, 2022 at 09:33:27AM +0100, Daniel P. Berrangé wrote: > > > > On Tue, Sep 27, 2022 at 01:43:15PM +0530, Ani Sinha wrote: > > > > > On Sun, Sep 18, 2022 at 1:58 AM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > > > On Fri, Sep 16, 2022 at 09:30:42PM +0530, Ani Sinha wrote: > > > > > > > On Thu, Jul 28, 2022 at 12:08 AM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, 25 Jul 2022, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, 16 Jul 2022, Michael S. Tsirkin wrote: > > > > > > > > > > > > > > > > > > > On Sat, Jul 16, 2022 at 12:06:00PM +0530, Ani Sinha wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 11:20 Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > > > > > > > > > > > > > On Fri, Jul 15, 2022 at 09:47:27AM +0530, Ani Sinha wrote: > > > > > > > > > > > > > Instead of all this mess, can't we just spawn e.g. "git clone --depth > > > > > > > > > > > 1"? > > > > > > > > > > > > > And if the directory exists I would fetch and checkout. > > > > > > > > > > > > > > > > > > > > > > > > There are two reasons I can think of why I do not like this idea: > > > > > > > > > > > > > > > > > > > > > > > > (a) a git clone of a whole directory would download all versions of the > > > > > > > > > > > > binary whereas we want only a specific version. > > > > > > > > > > > > > > > > > > > > > > You mention shallow clone yourself, and I used --depth 1 above. > > > > > > > > > > > > > > > > > > > > > > > Downloading a single file > > > > > > > > > > > > by shallow cloning or creating a git archive is overkill IMHO when a wget > > > > > > > > > > > > style retrieval works just fine. > > > > > > > > > > > > > > > > > > > > > > However, it does not provide for versioning, tagging etc so you have > > > > > > > > > > > to implement your own schema. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hmm I’m not sure if we need all that. Bits has its own versioning mechanism and > > > > > > > > > > > I think all we need to do is maintain the same versioning logic and maintain > > > > > > > > > > > binaries of different versions. Do we really need the power of git/version > > > > > > > > > > > control here? Dunno. > > > > > > > > > > > > > > > > > > > > Well we need some schema. Given we are not using official bits releases > > > > > > > > > > I don't think we can reuse theirs. > > > > > > > > > > > > > > > > > > OK fine. Lets figuire out how to push bits somewhere in git.qemu.org and > > > > > > > > > the binaries in some other repo first. Everything else hinges on that. We > > > > > > > > > can fix the rest of the bits later incrementally. > > > > > > > > > > > > > > > > DanPB, any thoughts on putting bits on git.qemu.org or where and how to > > > > > > > > keep the binaries? > > > > > > > > > > > > > > Can we please conclude on this? > > > > > > > Peter, can you please fork the repo? I have tried many times to reach > > > > > > > you on IRC but failed. > > > > > > > > > > > > Probably because of travel around KVM forum. > > > > > > > > > > > > I think given our CI is under pressure again due to gitlab free tier > > > > > > limits, tying binaries to CI isn't a great idea at this stage. > > > > > > Can Ani just upload binaies to qemu.org for now? > > > > > > > > > > I agree with Michael here. Having a full ci/cd job for this is > > > > > overkill IMHO. We should create a repo just for the binaries, have a > > > > > README there to explain how we generate them and check in new versions > > > > > as and when needed (it won't be frequent). > > > > > How about biosbits-bin repo? > > > > > > > > If QEMU is hosting binaries, where any part contains GPL code, then we > > > > need to be providing the full and corresponding source and the build > > > > scripts needed to re-create the binary. Once we have such scripts it > > > > should be trivial to trigger that from a CI job. If it isn't then > > > > we're doing something wrong. The CI quota is not an issue, because > > > > this is not a job that we need to run continuously. It can be triggered > > > > manually as & when we decide we need to refresh the binary, so would > > > > be a small one-off CI quota hit. > > > > > > > > Also note that gitlab is intending to start enforcing storage quota > > > > on projects in the not too distant future. This makes it unappealing > > > > to store binaries in git repos, unless we genuinely need the ability > > > > to access historical versions of the binary. I don't believe we need > > > > that for biosbits. > > > > > > > > The binary can be published as a CI artifact and accessed directly > > > > from the latest artifact download link. This ensures we only consume > > > > quota for the most recently published binary artifact. So I don't see > > > > a compelling reason to upload binaries into git. > > > > > > I don't really care where we upload them but only having the > > > latest version is just going to break anything expecting > > > the old binary. > > > > biosbits isn't tied to QEMU versions, it is an entirely separate 3rd > > party project. This binary is just providing the test env, and IIUC, > > control over what executes in this env is still done by the QEMU side > > test scripts. I'm not seeing a coupling here that requires precise > > matching. In any case biosbit is a dead project so does not look > > likely to have any changes. > > > > If we did want to have different versions though, we can stil > > publish artifacts from different branches of biosbits code. > > No, that is just ridiculous. Say we have a bug in bits that we fixed > and released a new version. Do we now create a new branch for that? > Multiple branches makes things needlessly complicated. We have one > branch, qemu-bits and all fixes go into that branch. We can have > different tags if we need. Nothing beyond that. I didn't mean we needed different branches for bug fixes. We would only need different branches if biosbits framework changed in some way that was incompatible with how QEMU used it previously. I find that unlikely to happen given its a dead project. We'll almost certainly be fine with one branch taking bug fixes. We do have the option for multiple branches in the unlikely even we find we need it. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 6:58 ` Daniel P. Berrangé 2022-09-28 7:15 ` Ani Sinha @ 2022-09-28 9:19 ` Michael S. Tsirkin 1 sibling, 0 replies; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-28 9:19 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Ani Sinha, imammedo, jsnow, pbonzini, peter.maydell, qemu-devel, thuth On Wed, Sep 28, 2022 at 07:58:46AM +0100, Daniel P. Berrangé wrote: > biosbits isn't tied to QEMU versions, it is an entirely separate 3rd > party project. This binary is just providing the test env, and IIUC, > control over what executes in this env is still done by the QEMU side > test scripts. I'm not seeing a coupling here that requires precise > matching. In any case biosbit is a dead project so does not look > likely to have any changes. > > If we did want to have different versions though, we can stil > publish artifacts from different branches of biosbits code. Gitlab > will preserve & publish the latest artifacts from each branch in > parallel. > > With regards, > Daniel The issue is bugs, testing, support. If biosbits needs to support old qemu versions that's a ton of work for no real benefit. CI is CI, it's not a binary distribution mechanism, abusing it will just lead to problems down the road... my $.02 -- MST ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-07-10 17:00 ` [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits Ani Sinha ` (2 preceding siblings ...) 2022-07-14 20:41 ` Michael S. Tsirkin @ 2022-09-27 21:44 ` Paolo Bonzini 2022-09-27 22:09 ` Michael S. Tsirkin 3 siblings, 1 reply; 90+ messages in thread From: Paolo Bonzini @ 2022-09-27 21:44 UTC (permalink / raw) To: Ani Sinha Cc: qemu-devel, Thomas Huth, Maydell, Peter, P. Berrange, Daniel, John Snow, Mammedov, Igor, S. Tsirkin, Michael [-- Attachment #1: Type: text/plain, Size: 22213 bytes --] Il dom 10 lug 2022, 19:01 Ani Sinha <ani@anisinha.ca> ha scritto: > This change adds python based test environment that can be used to run > pytest > from within a virtual environment. A bash script sets up a virtual > environment > and then runs the python based tests from within that environment. > All dependent python packages are installed in the virtual environment > using > pip python module. QEMU python test modules are also available in the > environment > for spawning the QEMU based VMs. > > It also introduces QEMU acpi/smbios biosbits python test script which is > run > from within the python virtual environment. When the bios bits tests are > run, > bios bits binaries are downloaded from an external repo/location. > Currently, the test points to an external private github repo where the > bits > archives are checked in. The virtual environment should be set up from configure, similar to git submodules. John was working on it and probably can point you at some earlier discussions in the archives about how to do it. I also second the idea of using avocado instead of pytest, by the way. Paolo > Signed-off-by: Ani Sinha <ani@anisinha.ca> > --- > tests/pytest/acpi-bits/acpi-bits-test-venv.sh | 59 +++ > tests/pytest/acpi-bits/acpi-bits-test.py | 382 ++++++++++++++++++ > tests/pytest/acpi-bits/meson.build | 33 ++ > tests/pytest/acpi-bits/requirements.txt | 1 + > 4 files changed, 475 insertions(+) > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test-venv.sh > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test.py > create mode 100644 tests/pytest/acpi-bits/meson.build > create mode 100644 tests/pytest/acpi-bits/requirements.txt > > diff --git a/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > new file mode 100644 > index 0000000000..186395473b > --- /dev/null > +++ b/tests/pytest/acpi-bits/acpi-bits-test-venv.sh > @@ -0,0 +1,59 @@ > +#!/usr/bin/env bash > +# Generates a python virtual environment for the test to run. > +# Then runs python test scripts from within that virtual environment. > +# > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; either version 2 of the License, or > +# (at your option) any later version. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > +# > +# Author: Ani Sinha <ani@anisinha.ca> > + > +set -e > + > +MYPATH=$(realpath ${BASH_SOURCE:-$0}) > +MYDIR=$(dirname $MYPATH) > + > +if [ -z "$PYTEST_SOURCE_ROOT" ]; then > + echo -n "Please set QTEST_SOURCE_ROOT env pointing" > + echo " to the root of the qemu source tree." > + echo -n "This is required so that the test can find the " > + echo "python modules that it needs for execution." > + exit 1 > +fi > +SRCDIR=$PYTEST_SOURCE_ROOT > +TESTSCRIPTS=("acpi-bits-test.py") > +PIPCMD="-m pip -q --disable-pip-version-check" > +# we need to save the old value of PWD before we do a change-dir later > +PYTEST_PWD=$PWD > + > +TESTS_PYTHON=/usr/bin/python3 > +TESTS_VENV_REQ=requirements.txt > + > +# sadly for pip -e and -t options do not work together. > +# please see https://github.com/pypa/pip/issues/562 > +cd $MYDIR > + > +$TESTS_PYTHON -m venv . > +$TESTS_PYTHON $PIPCMD install -e $SRCDIR/python/ > +[ -f $TESTS_VENV_REQ ] && \ > + $TESTS_PYTHON $PIPCMD install -r $TESTS_VENV_REQ || exit 0 > + > +# venv is activated at this point. > + > +# run the test > +for testscript in ${TESTSCRIPTS[@]} ; do > + export PYTEST_PWD; python3 $testscript > +done > + > +cd $PYTEST_PWD > + > +exit 0 > diff --git a/tests/pytest/acpi-bits/acpi-bits-test.py > b/tests/pytest/acpi-bits/acpi-bits-test.py > new file mode 100644 > index 0000000000..97e61eb709 > --- /dev/null > +++ b/tests/pytest/acpi-bits/acpi-bits-test.py > @@ -0,0 +1,382 @@ > +#!/usr/bin/env python3 > +# group: rw quick > +# Exercize QEMU generated ACPI/SMBIOS tables using biosbits, > +# https://biosbits.org/ > +# > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; either version 2 of the License, or > +# (at your option) any later version. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > +# > +# Some parts are slightly taken from qtest.py and iotests.py > +# > +# Authors: > +# Ani Sinha <ani@anisinha.ca> > + > +# pylint: disable=invalid-name > + > +""" > +QEMU bios tests using biosbits available at > +https://biosbits.org/. > +""" > + > +import logging > +import os > +import re > +import shutil > +import subprocess > +import sys > +import tarfile > +import tempfile > +import time > +import unittest > +from urllib import request > +import zipfile > +from typing import ( > + List, > + Optional, > + Sequence, > +) > +from tap import TAPTestRunner > +from qemu.machine import QEMUMachine > + > +PYTESTQEMUBIN = os.getenv('PYTEST_QEMU_BINARY') > +PYTEST_PWD = os.getenv('PYTEST_PWD') > + > +def get_arch(): > + """finds the arch from the qemu binary name""" > + match = re.search('.*qemu-system-(.*)', PYTESTQEMUBIN) > + if match: > + return match.group(1) > + return 'x86_64' > + > +ARCH = get_arch() > + > +class QEMUBitsMachine(QEMUMachine): > + """ > + A QEMU VM, with isa-debugcon enabled and bits iso passed > + using -cdrom to QEMU commandline. > + """ > + def __init__(self, > + binary: str, > + args: Sequence[str] = (), > + wrapper: Sequence[str] = (), > + name: Optional[str] = None, > + base_temp_dir: str = "/var/tmp", > + debugcon_log: str = "debugcon-log.txt", > + debugcon_addr: str = "0x403", > + sock_dir: Optional[str] = None, > + qmp_timer: Optional[float] = None): > + # pylint: disable=too-many-arguments > + > + if name is None: > + name = "qemu-bits-%d" % os.getpid() > + if sock_dir is None: > + sock_dir = base_temp_dir > + super().__init__(binary, args, wrapper=wrapper, name=name, > + base_temp_dir=base_temp_dir, > + sock_dir=sock_dir, qmp_timer=qmp_timer) > + self.debugcon_log = debugcon_log > + self.debugcon_addr = debugcon_addr > + self.base_temp_dir = base_temp_dir > + > + @property > + def _base_args(self) -> List[str]: > + args = super()._base_args > + args.extend([ > + '-chardev', > + 'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir, > + self.debugcon_log), > + '-device', > + 'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr, > + ]) > + return args > + > + def base_args(self): > + """return the base argument to QEMU binary""" > + return self._base_args > + > +class AcpiBitsTest(unittest.TestCase): > + """ACPI and SMBIOS tests using biosbits.""" > + def __init__(self, *args, **kwargs): > + super().__init__(*args, **kwargs) > + self._vm = None > + self._workDir = None > + self._bitsVer = 2100 > + self._bitsLoc = " > https://github.com/ani-sinha/bits/raw/bits-builds/" > + self._debugcon_addr = '0x403' > + self._debugcon_log = 'debugcon-log.txt' > + logging.basicConfig(level=logging.INFO) > + > + def copy_bits_config(self): > + """ copies the bios bits config file into bits. > + """ > + config_file = 'bits-cfg.txt' > + qemu_bits_config_dir = os.path.join(os.getcwd(), 'bits-config') > + target_config_dir = os.path.join(self._workDir, > + 'bits-%d' %self._bitsVer, 'boot') > + self.assertTrue(os.path.exists(qemu_bits_config_dir)) > + self.assertTrue(os.path.exists(target_config_dir)) > + self.assertTrue(os.access(os.path.join(qemu_bits_config_dir, > + config_file), os.R_OK)) > + shutil.copy2(os.path.join(qemu_bits_config_dir, config_file), > + target_config_dir) > + logging.info('copied config file %s to %s', > + config_file, target_config_dir) > + > + def copy_test_scripts(self): > + """copies the python test scripts into bits. """ > + qemu_test_dir = os.path.join(os.getcwd(), 'bits-tests') > + target_test_dir = os.path.join(self._workDir, 'bits-%d' > %self._bitsVer, > + 'boot', 'python') > + > + self.assertTrue(os.path.exists(qemu_test_dir)) > + self.assertTrue(os.path.exists(target_test_dir)) > + > + for filename in os.listdir(qemu_test_dir): > + if os.path.isfile(os.path.join(qemu_test_dir, filename)) and \ > + filename.endswith('.py'): > + shutil.copy2(os.path.join(qemu_test_dir, filename), > + target_test_dir) > + logging.info('copied test file %s to %s', > + filename, target_test_dir) > + > + # now remove the pyc test file if it exists, otherwise the > + # changes in the python test script won't be executed. > + testfile_pyc = os.path.splitext(filename)[0] + '.pyc' > + if os.access(os.path.join(target_test_dir, testfile_pyc), > + os.F_OK): > + os.remove(os.path.join(target_test_dir, testfile_pyc)) > + logging.info('removed compiled file %s', > + os.path.join(target_test_dir, > testfile_pyc)) > + > + def fix_mkrescue(self, mkrescue): > + """ grub-mkrescue is a bash script with two variables, 'prefix' > and > + 'libdir'. They must be pointed to the right location so that > the > + iso can be generated appropriately. We point the two > variables to > + the directory where we have extracted our pre-built bits grub > + tarball. > + """ > + grub_x86_64_mods = os.path.join(self._workDir, > 'grub-inst-x86_64-efi') > + grub_i386_mods = os.path.join(self._workDir, 'grub-inst') > + > + self.assertTrue(os.path.exists(grub_x86_64_mods)) > + self.assertTrue(os.path.exists(grub_i386_mods)) > + > + new_script = "" > + with open(mkrescue, 'r') as filehandle: > + orig_script = filehandle.read() > + new_script = re.sub('(^prefix=)(.*)', > + r'\1"%s"' %grub_x86_64_mods, > + orig_script, flags=re.M) > + new_script = re.sub('(^libdir=)(.*)', r'\1"%s/lib"' > %grub_i386_mods, > + new_script, flags=re.M) > + > + with open(mkrescue, 'w') as filehandle: > + filehandle.write(new_script) > + > + def generate_bits_iso(self): > + """ Uses grub-mkrescue to generate a fresh bits iso with the > python > + test scripts > + """ > + bits_dir = os.path.join(self._workDir, 'bits-%d' %self._bitsVer) > + iso_file = os.path.join(self._workDir, 'bits-%d.iso' > %self._bitsVer) > + mkrescue_script = os.path.join(self._workDir, > + 'grub-inst-x86_64-efi', 'bin', > + 'grub-mkrescue') > + > + self.assertTrue(os.access(mkrescue_script, > + os.R_OK | os.W_OK | os.X_OK)) > + > + self.fix_mkrescue(mkrescue_script) > + > + logging.info('calling grub-mkrescue to generate the biosbits iso > ...') > + > + try: > + if os.getenv('V'): > + subprocess.check_call([mkrescue_script, '-o', > + iso_file, bits_dir], > + stdout=subprocess.DEVNULL) > + else: > + subprocess.check_call([mkrescue_script, '-o', > + iso_file, bits_dir], > + stderr=subprocess.DEVNULL, > + stdout=subprocess.DEVNULL) > + except Exception as e: # pylint: disable=broad-except > + self.skipTest("Error while generating the bits iso. " > + "Pass V=1 in the environment to get more > details. " > + + str(e)) > + > + self.assertTrue(os.access(iso_file, os.R_OK)) > + > + logging.info('iso file %s successfully generated.', iso_file) > + > + def setUp(self): > + BITS_LOC = os.getenv("PYTEST_BITSLOC") > + if BITS_LOC: > + prefix = BITS_LOC > + else: > + prefix = os.path.join(os.getcwd(), 'prebuilt') > + if not os.path.isdir(prefix): > + os.mkdir(prefix, mode=0o775) > + > + bits_zip_file = os.path.join(prefix, 'bits-%d.zip' > + %self._bitsVer) > + grub_tar_file = os.path.join(prefix, > + 'bits-%d-grub.tar.gz' %self._bitsVer) > + # if the location of the bits binaries has been specified by the > user > + # and they are not found in that location, skip the test. > + if BITS_LOC and not os.access(bits_zip_file, os.F_OK): > + self.skipTest("test skipped since biosbits binaries " + > + "could not be found in the specified location > %s." \ > + %BITS_LOC) > + if BITS_LOC and not os.access(grub_tar_file, os.F_OK): > + self.skipTest("test skipped since biosbits binaries " + > + "could not be found in the specified location > %s." \ > + %BITS_LOC) > + > + self._workDir = tempfile.mkdtemp(prefix='acpi-bits-', > + suffix='.tmp') > + logging.info('working dir: %s', self._workDir) > + > + localArchive = "bits-%d.zip" % self._bitsVer > + if not os.access(bits_zip_file, os.F_OK): > + logging.info("archive %s not found in %s, downloading ...", > + localArchive, bits_zip_file) > + try: > + req = request.urlopen(self._bitsLoc + localArchive) > + with open(os.path.join(prefix, localArchive), > + 'wb') as archivef: > + archivef.write(req.read()) > + except Exception as e: # pylint: disable=broad-except > + self.skipTest("test skipped since biosbits binaries " + > + "could not be obtained." + str(e)) > + else: > + logging.info('using locally found %s', localArchive) > + > + localArchive = "bits-%d-grub.tar.gz" % self._bitsVer > + if not os.access(grub_tar_file, os.F_OK): > + logging.info("archive %s not found in %s, downloading ...", > + localArchive, bits_zip_file) > + try: > + req = request.urlopen(self._bitsLoc + localArchive) > + with open(os.path.join(prefix, localArchive), > + 'wb') as archivef: > + archivef.write(req.read()) > + except Exception as e: # pylint: disable=broad-except > + self.skipTest("test skipped since biosbits binaries " + > + "could not be obtained." + str(e)) > + else: > + logging.info('using locally found %s', localArchive) > + > + # extract the bits software in the temp working directory > + with zipfile.ZipFile(bits_zip_file, 'r') as zref: > + zref.extractall(self._workDir) > + > + with tarfile.open(grub_tar_file, 'r') as tarball: > + tarball.extractall(self._workDir) > + > + self.copy_test_scripts() > + self.copy_bits_config() > + self.generate_bits_iso() > + > + def parse_log(self): > + """parse the log generated by running bits tests and > + check for failures. > + """ > + debugconf = os.path.join(self._workDir, self._debugcon_log) > + log = "" > + with open(debugconf, 'r') as filehandle: > + log = filehandle.read() > + > + if os.getenv('V'): > + print('\nlogs from biosbits follows:') > + print('==========================================\n') > + print(log) > + print('==========================================\n') > + > + matchiter = re.finditer(r'(.*Summary: )(\d+ passed), (\d+ > failed).*', > + log) > + for match in matchiter: > + # verify that no test cases failed. > + self.assertEqual(match.group(3).split()[0], '0', > + 'Some bits tests seems to have failed. ' \ > + 'Set V=1 in the environment to get the > entire ' \ > + 'log from bits.') > + > + def tearDown(self): > + if self._vm: > + self.assertFalse(not self._vm.is_running) > + logging.info('removing the work directory %s', self._workDir) > + shutil.rmtree(self._workDir) > + > + def test_acpi_smbios_bits(self): > + """The main test case implementaion.""" > + > + qemu_bin = PYTESTQEMUBIN > + iso_file = os.path.join(self._workDir, 'bits-%d.iso' > %self._bitsVer) > + > + # PYTESTQEMUBIN could be relative to the current directory > + if not os.access(PYTESTQEMUBIN, os.X_OK) and PYTEST_PWD: > + qemu_bin = os.path.join(PYTEST_PWD, PYTESTQEMUBIN) > + > + logging.info('QEMU binary used: %s', qemu_bin) > + > + self.assertTrue(os.access(qemu_bin, os.X_OK)) > + self.assertTrue(os.access(iso_file, os.R_OK)) > + > + self._vm = QEMUBitsMachine(binary=qemu_bin, > + base_temp_dir=self._workDir, > + debugcon_log=self._debugcon_log, > + debugcon_addr=self._debugcon_addr) > + > + self._vm.add_args('-cdrom', '%s' %iso_file) > + > + args = " ".join(str(arg) for arg in self._vm.base_args()) + \ > + " " + " ".join(str(arg) for arg in self._vm.args) > + > + logging.info("launching QEMU vm with the following arguments: > %s", > + args) > + > + self._vm.launch() > + # biosbits has been configured to run all the specified test > suites > + # in batch mode and then automatically initiate a vm shutdown. > + # sleep for maximum of one minute > + max_sleep_time = time.monotonic() + 60 > + while self._vm.is_running() and time.monotonic() < max_sleep_time: > + time.sleep(1) > + > + self.assertFalse(time.monotonic() > max_sleep_time, > + 'The VM seems to have failed to shutdown in > time') > + > + self.parse_log() > + > +def execute_unittest(argv: List[str], debug: bool = False, > + runner: TAPTestRunner = None) -> None: > + """Executes unittests within the calling module.""" > + > + unittest.main(argv=argv, > + testRunner=runner, > + verbosity=2 if debug else 1, > + warnings=None if sys.warnoptions else 'ignore') > + > +def main(): > + """ The main function where execution begins. """ > + > + assert PYTESTQEMUBIN is not None, \ > + "Environment variable PYTEST_QEMU_BINARY required." > + > + runner = TAPTestRunner() > + runner.set_stream(True) > + runner.set_format("%s/acpi-bits-test" %ARCH) > + execute_unittest(sys.argv, False, runner) > + > +main() > diff --git a/tests/pytest/acpi-bits/meson.build > b/tests/pytest/acpi-bits/meson.build > new file mode 100644 > index 0000000000..099c191d57 > --- /dev/null > +++ b/tests/pytest/acpi-bits/meson.build > @@ -0,0 +1,33 @@ > +xorriso = find_program('xorriso', required: true) > +if not xorriso.found() > + message('xorriso not found ... disabled bits acpi tests.') > + subdir_done() > +endif > + > +subdir('bits-tests') > +subdir('bits-config') > + > +test_files = ['acpi-bits-test.py'] > +requirements = 'requirements.txt' > + > +copytestfiles = custom_target('copy test files', > + input : test_files, > + output : test_files, > + command : ['cp', '@INPUT@', '@OUTPUT@'], > + install : false, > + build_by_default : true) > + > +requirementsfiles = custom_target('copy py req files', > + input : requirements, > + output : requirements, > + command : ['cp', '@INPUT@', '@OUTPUT@'], > + install : false, > + build_by_default : true) > + > +other_deps += [copytestfiles,requirementsfiles] > + > +pytest_executables += { > + 'acpi-bits-test': configure_file(copy:true, > + input:'acpi-bits-test-venv.sh', > + output:'acpi-bits-test') > +} > diff --git a/tests/pytest/acpi-bits/requirements.txt > b/tests/pytest/acpi-bits/requirements.txt > new file mode 100644 > index 0000000000..00cdad09ef > --- /dev/null > +++ b/tests/pytest/acpi-bits/requirements.txt > @@ -0,0 +1 @@ > +tap.py > -- > 2.25.1 > > [-- Attachment #2: Type: text/html, Size: 29864 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 21:44 ` Paolo Bonzini @ 2022-09-27 22:09 ` Michael S. Tsirkin 2022-09-27 23:10 ` Paolo Bonzini 2022-09-28 7:06 ` Daniel P. Berrangé 0 siblings, 2 replies; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-27 22:09 UTC (permalink / raw) To: Paolo Bonzini Cc: Ani Sinha, qemu-devel, Thomas Huth, Maydell, Peter, P. Berrange, Daniel, John Snow, Mammedov, Igor On Tue, Sep 27, 2022 at 11:44:56PM +0200, Paolo Bonzini wrote: > I also second the idea of using avocado instead of pytest, by the way. > > Paolo I do not think this is a good fit for bios tests. bios tests are intended for a wide audience of ACPI developers across a variety of host systems. They basically do not need anything from the host and they need to be super easy to configure since we have lots of drive through contributors. Problem is I don't think avocado is yet at the level where I can ask random developers to use it to check their ACPI patches. I just went ahead and rechecked and the situation isn't much better yet. I think the focus of avocado is system testing of full guests with KVM, not unit testing of ACPI. Let's start with installation on a clean box: following https://avocado-framework.readthedocs.io/en/98.0/guides/user/chapters/installing.html Ugh pip, will install a bunch of stuff in ~/.local and ask me to tweak PATH ... and what about security? No thanks! So ... do I want LTS or latest? Well I donnu .... let's try LTS? $ dnf module enable avocado:82lts [sudo] password for mst: Last metadata expiration check: 6 days, 15:20:21 ago on Wed 21 Sep 2022 02:33:31 AM EDT. Dependencies resolved. ========================================================================================================================================== Package Architecture Version Repository Size ========================================================================================================================================== Enabling module streams: avocado 82lts Transaction Summary ========================================================================================================================================== Is this ok [y/N]: y Complete! [mst@tuck linux]$ dnf module install avocado Last metadata expiration check: 6 days, 15:20:41 ago on Wed 21 Sep 2022 02:33:31 AM EDT. No default profiles for module avocado:82lts. Available profiles: default, minimal Error: Problems in request: broken groups or modules: avocado Ugh I guess latest then? [mst@tuck linux]$ dnf module enable avocado:latest Last metadata expiration check: 6 days, 15:25:21 ago on Wed 21 Sep 2022 02:33:31 AM EDT. Dependencies resolved. The operation would result in switching of module 'avocado' stream '82lts' to stream 'latest' Error: It is not possible to switch enabled streams of a module unless explicitly enabled via configuration option module_stream_switch. It is recommended to rather remove all installed content from the module, and reset the module using 'dnf module reset <module_name>' command. After you reset the module, you can install the other stream. Scary ... I don't really know what are streams and I am guessing module is avocado here? and what will this affect. Oh well, I'll risk this: [mst@tuck linux]$ dnf module reset avocado Last metadata expiration check: 6 days, 15:25:46 ago on Wed 21 Sep 2022 02:33:31 AM EDT. Dependencies resolved. ========================================================================================================================================== Package Architecture Version Repository Size ========================================================================================================================================== Resetting modules: avocado Transaction Summary ========================================================================================================================================== Is this ok [y/N]: y Complete! [mst@tuck linux]$ dnf module enable avocado:latest Last metadata expiration check: 6 days, 15:25:55 ago on Wed 21 Sep 2022 02:33:31 AM EDT. Dependencies resolved. ========================================================================================================================================== Package Architecture Version Repository Size ========================================================================================================================================== Enabling module streams: avocado latest Transaction Summary ========================================================================================================================================== Is this ok [y/N]: y Complete! [mst@tuck linux]$ dnf module install avocado Last metadata expiration check: 6 days, 15:26:03 ago on Wed 21 Sep 2022 02:33:31 AM EDT. Dependencies resolved. ========================================================================================================================================== Package Architecture Version Repository Size ========================================================================================================================================== Installing group/module packages: python3-avocado noarch 97.0-1.module_f35+14550+3bc43fee updates-modular 603 k python3-avocado-plugins-output-html noarch 97.0-1.module_f35+14550+3bc43fee updates-modular 99 k python3-avocado-plugins-varianter-yaml-to-mux noarch 97.0-1.module_f35+14550+3bc43fee updates-modular 29 k Installing dependencies: gdb-gdbserver x86_64 12.1-1.fc35 updates 300 k python-avocado-common noarch 97.0-1.module_f35+14550+3bc43fee updates-modular 29 k Installing module profiles: avocado/default Transaction Summary ========================================================================================================================================== Install 5 Packages Total download size: 1.0 M Installed size: 3.5 M Is this ok [y/N]: y Downloading Packages: (1/5): python-avocado-common-97.0-1.module_f35+14550+3bc43fee.noarch.rpm 40 kB/s | 29 kB 00:00 (2/5): python3-avocado-plugins-output-html-97.0-1.module_f35+14550+3bc43fee.noarch.rpm 109 kB/s | 99 kB 00:00 (3/5): python3-avocado-plugins-varianter-yaml-to-mux-97.0-1.module_f35+14550+3bc43fee.noarch.rpm 28 kB/s | 29 kB 00:01 (4/5): python3-avocado-97.0-1.module_f35+14550+3bc43fee.noarch.rpm 479 kB/s | 603 kB 00:01 (5/5): gdb-gdbserver-12.1-1.fc35.x86_64.rpm 232 kB/s | 300 kB 00:01 ------------------------------------------------------------------------------------------------------------------------------------------ Total 204 kB/s | 1.0 MB 00:05 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Regex version mismatch, expected: 10.40 2022-04-14 actual: 10.35 2020-05-09 Preparing : 1/1 Installing : python-avocado-common-97.0-1.module_f35+14550+3bc43fee.noarch 1/5 Installing : gdb-gdbserver-12.1-1.fc35.x86_64 2/5 Installing : python3-avocado-97.0-1.module_f35+14550+3bc43fee.noarch 3/5 Installing : python3-avocado-plugins-output-html-97.0-1.module_f35+14550+3bc43fee.noarch 4/5 Installing : python3-avocado-plugins-varianter-yaml-to-mux-97.0-1.module_f35+14550+3bc43fee.noarch 5/5 Running scriptlet: python3-avocado-plugins-varianter-yaml-to-mux-97.0-1.module_f35+14550+3bc43fee.noarch 5/5 Verifying : gdb-gdbserver-12.1-1.fc35.x86_64 1/5 Verifying : python-avocado-common-97.0-1.module_f35+14550+3bc43fee.noarch 2/5 Verifying : python3-avocado-97.0-1.module_f35+14550+3bc43fee.noarch 3/5 Verifying : python3-avocado-plugins-output-html-97.0-1.module_f35+14550+3bc43fee.noarch 4/5 Verifying : python3-avocado-plugins-varianter-yaml-to-mux-97.0-1.module_f35+14550+3bc43fee.noarch 5/5 Installed: gdb-gdbserver-12.1-1.fc35.x86_64 python-avocado-common-97.0-1.module_f35+14550+3bc43fee.noarch python3-avocado-97.0-1.module_f35+14550+3bc43fee.noarch python3-avocado-plugins-output-html-97.0-1.module_f35+14550+3bc43fee.noarch python3-avocado-plugins-varianter-yaml-to-mux-97.0-1.module_f35+14550+3bc43fee.noarch Complete! Great! Except I want avocado-vt of course. I will pretend someone told me that I need it. Installing Avocado-VT on Fedora or Enterprise Linux is a matter of installing the avocado-plugins-vt package. Install it with: $ yum install avocado-plugins-vt I guess I will replace yum with dnf then? [mst@tuck linux]$ dnf install avocado-plugins-vt Last metadata expiration check: 6 days, 15:31:16 ago on Wed 21 Sep 2022 02:33:31 AM EDT. No match for argument: avocado-plugins-vt Error: Unable to find a match: avocado-plugins-vt And this is Fedora. What do people do on debian? non Linux? -- MST ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 22:09 ` Michael S. Tsirkin @ 2022-09-27 23:10 ` Paolo Bonzini 2022-09-28 5:53 ` Michael S. Tsirkin 2022-09-28 7:06 ` Daniel P. Berrangé 1 sibling, 1 reply; 90+ messages in thread From: Paolo Bonzini @ 2022-09-27 23:10 UTC (permalink / raw) To: Michael S. Tsirkin Cc: Ani Sinha, qemu-devel, Thomas Huth, Maydell, Peter, P. Berrange, Daniel, John Snow, Mammedov, Igor [-- Attachment #1: Type: text/plain, Size: 772 bytes --] Il mer 28 set 2022, 00:09 Michael S. Tsirkin <mst@redhat.com> ha scritto: > On Tue, Sep 27, 2022 at 11:44:56PM +0200, Paolo Bonzini wrote: > > I also second the idea of using avocado instead of pytest, by the way. > > > > Paolo > > I do not think this is a good fit for bios tests. > bios tests are intended for a wide audience of ACPI developers > across a variety of host systems. They basically do not need anything > from the host and they need to be super easy to configure > since we have lots of drive through contributors. > The setup would be the same, with avocado installed in a virtual environment via pip. It doesn't need to be set up outside, neither with distro packages nor in ~/.local, and especially it is not necessary to deal with avocado-vt. Paolo [-- Attachment #2: Type: text/html, Size: 1178 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 23:10 ` Paolo Bonzini @ 2022-09-28 5:53 ` Michael S. Tsirkin 0 siblings, 0 replies; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-28 5:53 UTC (permalink / raw) To: Paolo Bonzini Cc: Ani Sinha, qemu-devel, Thomas Huth, Maydell, Peter, P. Berrange, Daniel, John Snow, Mammedov, Igor On Wed, Sep 28, 2022 at 01:10:15AM +0200, Paolo Bonzini wrote: > > > Il mer 28 set 2022, 00:09 Michael S. Tsirkin <mst@redhat.com> ha scritto: > > On Tue, Sep 27, 2022 at 11:44:56PM +0200, Paolo Bonzini wrote: > > I also second the idea of using avocado instead of pytest, by the way. > > > > Paolo > > I do not think this is a good fit for bios tests. > bios tests are intended for a wide audience of ACPI developers > across a variety of host systems. They basically do not need anything > from the host and they need to be super easy to configure > since we have lots of drive through contributors. > > > The setup would be the same, with avocado installed in a virtual environment > via pip. It doesn't need to be set up outside, neither with distro packages nor > in ~/.local, and especially it is not necessary to deal with avocado-vt. > > Paolo Hmm, good point. -- MST ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-27 22:09 ` Michael S. Tsirkin 2022-09-27 23:10 ` Paolo Bonzini @ 2022-09-28 7:06 ` Daniel P. Berrangé 2022-09-28 7:43 ` Thomas Huth 1 sibling, 1 reply; 90+ messages in thread From: Daniel P. Berrangé @ 2022-09-28 7:06 UTC (permalink / raw) To: Michael S. Tsirkin Cc: Paolo Bonzini, Ani Sinha, qemu-devel, Thomas Huth, Maydell, Peter, John Snow, Mammedov, Igor On Tue, Sep 27, 2022 at 06:09:22PM -0400, Michael S. Tsirkin wrote: > On Tue, Sep 27, 2022 at 11:44:56PM +0200, Paolo Bonzini wrote: > > I also second the idea of using avocado instead of pytest, by the way. snip > Problem is I don't think avocado is yet at the level where I can > ask random developers to use it to check their ACPI patches. > > I just went ahead and rechecked and the situation isn't much better > yet. I think the focus of avocado is system testing of full guests with > KVM, not unit testing of ACPI. > > Let's start with installation on a clean box: ...snip... Do not do any of this stuff, it is irrelevant to QEMU's needs. A developer using Avocado with QEMU does nothing more than: make check-avocado The avocado framework itself is setup in a virtual env on first run by the makefile. Thus the developers don't need to install nor interact with avocado at all, merely write a test script and put it in tests/avocado/ in QEMU git. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 7:06 ` Daniel P. Berrangé @ 2022-09-28 7:43 ` Thomas Huth 2022-10-09 5:21 ` Ani Sinha 2022-10-09 16:06 ` Ani Sinha 0 siblings, 2 replies; 90+ messages in thread From: Thomas Huth @ 2022-09-28 7:43 UTC (permalink / raw) To: Daniel P. Berrangé, Michael S. Tsirkin Cc: Paolo Bonzini, Ani Sinha, qemu-devel, Maydell, Peter, John Snow, Mammedov, Igor On 28/09/2022 09.06, Daniel P. Berrangé wrote: > On Tue, Sep 27, 2022 at 06:09:22PM -0400, Michael S. Tsirkin wrote: >> On Tue, Sep 27, 2022 at 11:44:56PM +0200, Paolo Bonzini wrote: >>> I also second the idea of using avocado instead of pytest, by the way. > > snip > >> Problem is I don't think avocado is yet at the level where I can >> ask random developers to use it to check their ACPI patches. >> >> I just went ahead and rechecked and the situation isn't much better >> yet. I think the focus of avocado is system testing of full guests with >> KVM, not unit testing of ACPI. >> >> Let's start with installation on a clean box: > > ...snip... > > Do not do any of this stuff, it is irrelevant to QEMU's needs. > A developer using Avocado with QEMU does nothing more than: > > make check-avocado Right. And if you want to run individual tests, you can also do it like this: make check-venv # Only for the first time ./tests/venv/bin/avocado run tests/avocado/boot_linux.py Or run tests via tags (very convenient for maintainers): ./tests/venv/bin/avocado run -t arch:s390x tests/avocado/ HTH, Thomas ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 7:43 ` Thomas Huth @ 2022-10-09 5:21 ` Ani Sinha 2022-10-09 6:30 ` Ani Sinha 2022-10-09 16:06 ` Ani Sinha 1 sibling, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-10-09 5:21 UTC (permalink / raw) To: Thomas Huth Cc: Daniel P. Berrangé, Michael S. Tsirkin, Paolo Bonzini, qemu-devel, Maydell, Peter, John Snow, Mammedov, Igor On Wed, Sep 28, 2022 at 1:14 PM Thomas Huth <thuth@redhat.com> wrote: > > On 28/09/2022 09.06, Daniel P. Berrangé wrote: > > On Tue, Sep 27, 2022 at 06:09:22PM -0400, Michael S. Tsirkin wrote: > >> On Tue, Sep 27, 2022 at 11:44:56PM +0200, Paolo Bonzini wrote: > >>> I also second the idea of using avocado instead of pytest, by the way. > > > > snip > > > >> Problem is I don't think avocado is yet at the level where I can > >> ask random developers to use it to check their ACPI patches. > >> > >> I just went ahead and rechecked and the situation isn't much better > >> yet. I think the focus of avocado is system testing of full guests with > >> KVM, not unit testing of ACPI. > >> > >> Let's start with installation on a clean box: > > > > ...snip... > > > > Do not do any of this stuff, it is irrelevant to QEMU's needs. > > A developer using Avocado with QEMU does nothing more than: > > > > make check-avocado > > Right. And if you want to run individual tests, you can also do it like this: > > make check-venv # Only for the first time > ./tests/venv/bin/avocado run tests/avocado/boot_linux.py Ok this seems to work after I did a pip3 install of avocado in the host. ./tests/venv/bin/avocado run tests/avocado/version.py JOB ID : 8dd90b1cb5baf3780cc764ca4a1ae838374a0a5f JOB LOG : /home/anisinha/avocado/job-results/job-2022-10-09T10.48-8dd90b1/job.log (1/1) tests/avocado/version.py:Version.test_qmp_human_info_version: PASS (0.04 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 3.51 s I see that the output is not tap compliant like the qtests tests are. how do I choose tap? ./tests/venv/bin/avocado-runner-tap --help usage: avocado-runner-tap [-h] {capabilities,runnable-run,runnable-run-recipe,task-run,task-run-recipe} ... nrunner application for executable tests that produce TAP positional arguments: {capabilities,runnable-run,runnable-run-recipe,task-run,task-run-recipe} capabilities Outputs capabilities, including runnables and commands runnable-run Runs a runnable definition from arguments runnable-run-recipe Runs a runnable definition from a recipe task-run Runs a task from arguments task-run-recipe Runs a task from a recipe options: -h, --help show this help message and exit > > Or run tests via tags (very convenient for maintainers): > > ./tests/venv/bin/avocado run -t arch:s390x tests/avocado/ > > HTH, > Thomas > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-10-09 5:21 ` Ani Sinha @ 2022-10-09 6:30 ` Ani Sinha 0 siblings, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-10-09 6:30 UTC (permalink / raw) To: Thomas Huth Cc: Daniel P. Berrangé, Michael S. Tsirkin, Paolo Bonzini, qemu-devel, Maydell, Peter, John Snow, Mammedov, Igor On Sun, Oct 9, 2022 at 10:51 AM Ani Sinha <ani@anisinha.ca> wrote: > > On Wed, Sep 28, 2022 at 1:14 PM Thomas Huth <thuth@redhat.com> wrote: > > > > > > > Do not do any of this stuff, it is irrelevant to QEMU's needs. > > > A developer using Avocado with QEMU does nothing more than: > > > > > > make check-avocado > > > > Right. And if you want to run individual tests, you can also do it like this: > > > > make check-venv # Only for the first time > > ./tests/venv/bin/avocado run tests/avocado/boot_linux.py > > Ok this seems to work after I did a pip3 install of avocado in the host. > > ./tests/venv/bin/avocado run tests/avocado/version.py > JOB ID : 8dd90b1cb5baf3780cc764ca4a1ae838374a0a5f > JOB LOG : /home/anisinha/avocado/job-results/job-2022-10-09T10.48-8dd90b1/job.log > (1/1) tests/avocado/version.py:Version.test_qmp_human_info_version: > PASS (0.04 s) > RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 > | CANCEL 0 > JOB TIME : 3.51 s > > I see that the output is not tap compliant like the qtests tests are. > how do I choose tap? > > ./tests/venv/bin/avocado-runner-tap --help > usage: avocado-runner-tap [-h] > {capabilities,runnable-run,runnable-run-recipe,task-run,task-run-recipe} > ... > > nrunner application for executable tests that produce TAP > > positional arguments: > {capabilities,runnable-run,runnable-run-recipe,task-run,task-run-recipe} > capabilities Outputs capabilities, including runnables and commands > runnable-run Runs a runnable definition from arguments > runnable-run-recipe > Runs a runnable definition from a recipe > task-run Runs a task from arguments > task-run-recipe Runs a task from a recipe > > options: > -h, --help show this help message and exit > Never mind $ ./tests/venv/bin/avocado run tests/avocado/version.py --tap - 1..1 ok 1 tests/avocado/version.py:Version.test_qmp_human_info_version from https://avocado-framework.readthedocs.io/en/52.0/ResultFormats.html . ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits 2022-09-28 7:43 ` Thomas Huth 2022-10-09 5:21 ` Ani Sinha @ 2022-10-09 16:06 ` Ani Sinha 1 sibling, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-10-09 16:06 UTC (permalink / raw) To: Thomas Huth Cc: Daniel P. Berrangé, Michael S. Tsirkin, Paolo Bonzini, qemu-devel, Maydell, Peter, John Snow, Mammedov, Igor On Wed, Sep 28, 2022 at 1:14 PM Thomas Huth <thuth@redhat.com> wrote: > <snip> > > > > Do not do any of this stuff, it is irrelevant to QEMU's needs. > > A developer using Avocado with QEMU does nothing more than: > > > > make check-avocado > > Right. And if you want to run individual tests, you can also do it like this: > > make check-venv # Only for the first time > ./tests/venv/bin/avocado run tests/avocado/boot_linux.py > > Or run tests via tags (very convenient for maintainers): > > ./tests/venv/bin/avocado run -t arch:s390x tests/avocado/ yeah this seems to work! $ ./tests/venv/bin/avocado run -t acpi tests/avocado ERROR: Missing parentheses in call to 'print'. Did you mean print(...)? (smbios.py, line 92) ERROR: Missing parentheses in call to 'print'. Did you mean print(...)? (smilatency.py, line 47) ERROR: Missing parentheses in call to 'print'. Did you mean print(...)? (testacpi.py, line 158) Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits JOB ID : 328a83d7d0ea628ea8054f16fe2065826d4481e9 JOB LOG : /home/anisinha/avocado/job-results/job-2022-10-09T21.31-328a83d/job.log (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS (37.15 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 43.39 s $ ./tests/venv/bin/avocado run -t acpi tests/avocado --tap - ERROR: Missing parentheses in call to 'print'. Did you mean print(...)? (smbios.py, line 92) ERROR: Missing parentheses in call to 'print'. Did you mean print(...)? (smilatency.py, line 47) ERROR: Missing parentheses in call to 'print'. Did you mean print(...)? (testacpi.py, line 158) 1..1 ok 1 tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits ^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v2 08/11] acpi/tests/bits: add biosbits config file for running bios tests 2022-07-10 17:00 [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Ani Sinha ` (6 preceding siblings ...) 2022-07-10 17:00 ` [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits Ani Sinha @ 2022-07-10 17:00 ` Ani Sinha 2022-07-10 17:00 ` [PATCH v2 09/11] acpi/tests/bits: add a README file describing bits pytests Ani Sinha ` (3 subsequent siblings) 11 siblings, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-07-10 17:00 UTC (permalink / raw) To: qemu-devel Cc: thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo, mst, Ani Sinha This change adds initial biosbits config file that instructs biosbits to run bios test suits in batch mode. Additionally acpi and smbios structures are also dumped. Signed-off-by: Ani Sinha <ani@anisinha.ca> --- .../pytest/acpi-bits/bits-config/bits-cfg.txt | 18 ++++++++++++++++++ tests/pytest/acpi-bits/bits-config/meson.build | 11 +++++++++++ 2 files changed, 29 insertions(+) create mode 100644 tests/pytest/acpi-bits/bits-config/bits-cfg.txt create mode 100644 tests/pytest/acpi-bits/bits-config/meson.build diff --git a/tests/pytest/acpi-bits/bits-config/bits-cfg.txt b/tests/pytest/acpi-bits/bits-config/bits-cfg.txt new file mode 100644 index 0000000000..8010804453 --- /dev/null +++ b/tests/pytest/acpi-bits/bits-config/bits-cfg.txt @@ -0,0 +1,18 @@ +# BITS configuration file +[bits] + +# To run BITS in batch mode, set batch to a list of one or more of the +# following keywords; BITS will then run all of the requested operations, then +# save the log file to disk. +# +# test: Run the full BITS testsuite. +# acpi: Dump all ACPI structures. +# smbios: Dump all SMBIOS structures. +# +# Leave batch set to an empty string to disable batch mode. +# batch = + +# Uncomment the following to run all available batch operations +# please take a look at boot/python/init.py in bits zip file +# to see how these options are parsed and used. +batch = test acpi smbios diff --git a/tests/pytest/acpi-bits/bits-config/meson.build b/tests/pytest/acpi-bits/bits-config/meson.build new file mode 100644 index 0000000000..bbd7a940dc --- /dev/null +++ b/tests/pytest/acpi-bits/bits-config/meson.build @@ -0,0 +1,11 @@ +config_files = ['bits-cfg.txt'] + +copycfgfiles = custom_target('copy cfg files', + input : config_files, + output : config_files, + command : ['cp', '@INPUT@', '@OUTPUT@'], + install : true, + install_dir : 'bits-config', + build_by_default : true) + +other_deps += copycfgfiles -- 2.25.1 ^ permalink raw reply related [flat|nested] 90+ messages in thread
* [PATCH v2 09/11] acpi/tests/bits: add a README file describing bits pytests 2022-07-10 17:00 [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Ani Sinha ` (7 preceding siblings ...) 2022-07-10 17:00 ` [PATCH v2 08/11] acpi/tests/bits: add biosbits config file for running bios tests Ani Sinha @ 2022-07-10 17:00 ` Ani Sinha 2022-07-10 17:00 ` [PATCH v2 10/11] pytest: add pytest to the meson build system Ani Sinha ` (2 subsequent siblings) 11 siblings, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-07-10 17:00 UTC (permalink / raw) To: qemu-devel, Ani Sinha Cc: thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo, mst A README file is added describing the directory structure and the purpose of every file it contains. It also describes the test added, how to add new tests, make changes to existing tests or bits config files or regenerate the bios bits software binaries. Signed-off-by: Ani Sinha <ani@anisinha.ca> --- tests/pytest/acpi-bits/README | 129 ++++++++++++++++++++++++++++++++++ 1 file changed, 129 insertions(+) create mode 100644 tests/pytest/acpi-bits/README diff --git a/tests/pytest/acpi-bits/README b/tests/pytest/acpi-bits/README new file mode 100644 index 0000000000..be2ddba8ec --- /dev/null +++ b/tests/pytest/acpi-bits/README @@ -0,0 +1,129 @@ +============================================================================= +ACPI/SMBIOS PYTESTS USING BIOSBITS +============================================================================= + +Biosbits is a software written by Josh Triplett that can be downloaded by +visiting https://biosbits.org/. The github codebase can be found here: +https://github.com/biosbits/bits/tree/master. It is a software that executes +the bios components such as acpi and smbios tables directly through acpica +bios interpreter (a freely available C based library written by Intel, +downloadable from https://acpica.org/ and is included with biosbits) without an +operating system getting involved in between. +There are several advantages to directly testing the bios in a real physical +machine or VM as opposed to indirectly discovering bios issues through the +operating system. For one thing, the OSes tend to hide bios problems from the +end user. The other is that we have more control of what we wanted to test +and how by directly using acpica interpreter on top of the bios on a running +system. More details on the inspiration for developing biosbits and its real +life uses can be found in (a) and (b). +This directory contains QEMU pytests written in python that tests the QEMU +bios components using biosbits and reports test failures. + +These tests use python virtual environment. In debian/ubuntu system, the tests +would require python3.8-venv and python3-pip packages to be installed. + +A brief description of the contents of this directory follows: + +├── acpi-bits-test.py +├── acpi-bits-test-venv.sh +├── bits-config +│ ├── bits-cfg.txt +│ └── meson.build +├── bits-tests +│ ├── meson.build +│ ├── smbios.py +│ ├── smilatency.py +│ ├── testacpi.py +│ └── testcpuid.py +├── meson.build +├── README +└── requirements.txt + +acpi-bits: + - acpi-bits-test-venv.sh: This is a shell script that sets up the virtual + environment necessary for the actual python test script to run. Amongst + other things, it makes sure that QEMU python library is available within + that venv so that QEMU machines can be forked. QEMU python library can be + found within python/ directory in QEMU source. + After setting up the virtual environment, it runs the python test script + from within that environment. + If you want to enable verbose mode only for bits test and run make check, + one trick is to add V=1 before the call to execute the python script in + this file. + - acpi-bits-test.py: This is the main python test script that generates a + biosbits iso. It then spawns a QEMU VM with it, collects the logs and reports + test failures. This is the script one would be interested in if they wanted + to add or change some component of the log parsing, add a new command line to + how QEMU is spawned etc. Test writers typically would not need to modify + this script unless they wanted to enhance or change the log parsing for + their tests. Following environment variables are used in this test: + - PYTEST_QEMU_BINARY : This should point to the QEMU binary that is being + tested. Passing this environment variable is mandatory. + - PYTEST_BITSLOC : This is the location that should point to the grub tar + archive and the bios bits zip archive needed for this test. If they are + absent in that location, the test is skipped. If this environment + variable is not passed, the test would first check within the prebuilt/ + subdirectory to see if those binaries already exists. If they do, then + the test would use them. If those binaries are absent, the test would + try to download them from a remote repository. If the download fails, + the test is skipped. If the download succeeds, the test uses the + downloaded binaries. + When the user wants to run the test in offline mode, this environment + variable is very useful. The user can check out the bios bits archive + earlier when connectivity existed. Then in the offline mode, the user + can point this environment variable to the location where the archives + were downloaded and run the test. + - V=1 : This enables verbose mode for the test. It dumps the entire log + from bios bits and also more details in case failure happens. It is + useful for debugging the test failures or tests themselves. + - PYTEST_PWD : Present working directory where the test would look for + the QEMU binary. Optional. + +- requirements.txt: This text file contains the dependent python libraries + needed for the tests to run. If a new dependent library is needed, it would + be added here as a new entry and then acpi-bits-test-venv.sh would install + it when setting up the python virtual environment using pip. + - README: This text file. + +acpi-bits/bits-config: + This location contains biosbits config files that determine how the software + runs the tests. + - bits-config.txt: this is the biosbits config file that determines what tests + or actions are performed by bits. The description of the config options are + provided in the file itself. + +acpi-bits/bits-tests: + This directory contains biosbits python based tests that are run from within + the biosbits environment in the spawned VM. New additions of test cases can + be made in the appropriate test file. For example, new acpi tests can go + into testacpi.py and one would call testsuite.add_test() to register the new + test so that it gets executed as a part of the ACPI tests. + It might be occasionally necessary to disable some subtests or add a new + test that belongs to a test suite not already present in this directory. To + do this, please extract the bits source from the zip file mentioned above. + Copy the test suite/script that needs modification (addition of new tests + or disabling them) from boot/python location of the extracted directory + into this directory. + + step (a): copy unmodified test script to this directory. + step (b): update meson.build and add this file to the list. + Commit (a) and (b) together in the same commit. + + step (c): perform modifications to the test. + Commit (c) in a separate commit. + + The test framework will then use your modified test script to run the test. + No further changes would be needed. Please check the logs to make sure that + appropriate changes have taken effect. + +meson.build files makes sure that the bits pytests are appropriately integrated +into the QEMU pytest framework and are run with "make check-pytest". +Currently, the bits test is configured to run only for x86_64 architectures. Bits +has been built only for this architecture. + + +Author: Ani Sinha <ani@anisinha.ca> + +References: +(a) https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf +(b) https://www.youtube.com/watch?v=36QIepyUuhg -- 2.25.1 ^ permalink raw reply related [flat|nested] 90+ messages in thread
* [PATCH v2 10/11] pytest: add pytest to the meson build system 2022-07-10 17:00 [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Ani Sinha ` (8 preceding siblings ...) 2022-07-10 17:00 ` [PATCH v2 09/11] acpi/tests/bits: add a README file describing bits pytests Ani Sinha @ 2022-07-10 17:00 ` Ani Sinha 2022-07-11 20:46 ` John Snow 2022-07-10 17:00 ` [PATCH v2 11/11] MAINTAINERS: add myself as the maintainer for acpi biosbits pytests Ani Sinha 2022-07-11 9:33 ` [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Michael S. Tsirkin 11 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-07-10 17:00 UTC (permalink / raw) To: qemu-devel Cc: thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo, mst, Ani Sinha Integrate the pytest framework with the meson build system. This will make meson run all the pytests under the pytest directory. Signed-off-by: Ani Sinha <ani@anisinha.ca> --- tests/Makefile.include | 4 +++- tests/meson.build | 1 + tests/pytest/meson.build | 49 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 53 insertions(+), 1 deletion(-) create mode 100644 tests/pytest/meson.build diff --git a/tests/Makefile.include b/tests/Makefile.include index 3accb83b13..40755a6bd1 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -3,12 +3,14 @@ .PHONY: check-help check-help: @echo "Regression testing targets:" - @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests" + @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest, pytest and decodetree tests" @echo " $(MAKE) bench Run speed tests" @echo @echo "Individual test suites:" @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target" @echo " $(MAKE) check-qtest Run qtest tests" + @echo " $(MAKE) check-pytest Run pytest tests" + @echo " $(MAKE) check-pytest-TARGET Run pytest for a given target" @echo " $(MAKE) check-unit Run qobject tests" @echo " $(MAKE) check-qapi-schema Run QAPI schema tests" @echo " $(MAKE) check-block Run block tests" diff --git a/tests/meson.build b/tests/meson.build index 8e318ec513..f344cbdc6c 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -93,3 +93,4 @@ subdir('unit') subdir('qapi-schema') subdir('qtest') subdir('migration') +subdir('pytest') diff --git a/tests/pytest/meson.build b/tests/pytest/meson.build new file mode 100644 index 0000000000..e60d481ae4 --- /dev/null +++ b/tests/pytest/meson.build @@ -0,0 +1,49 @@ +slow_pytests = { + 'acpi-bits' : 120, +} + +pytests_generic = [] + +# biosbits tests are currenly only supported on x86_64 platforms. +pytests_x86_64 = ['acpi-bits-test'] + +pytest_executables = {} +other_deps = [] + +subdir('acpi-bits') + +foreach dir : target_dirs + if not dir.endswith('-softmmu') + continue + endif + + target_base = dir.split('-')[0] + pytest_emulator = emulators['qemu-system-' + target_base] + target_pytests = get_variable('pytests_' + target_base, []) + pytests_generic + + test_deps = roms + pytest_env = environment() + if have_tools + pytest_env.set('QTEST_QEMU_IMG', './qemu-img') + test_deps += [qemu_img] + endif + pytest_env.set('G_TEST_DBUS_DAEMON', meson.project_source_root() / 'tests/dbus-vmstate-daemon.sh') + pytest_env.set('PYTEST_QEMU_BINARY', './qemu-system-' + target_base) + pytest_env.set('PYTEST_SOURCE_ROOT', meson.project_source_root()) + if have_tools and have_vhost_user_blk_server + pytest_env.set('PYTEST_QEMU_STORAGE_DAEMON_BINARY', './storage-daemon/qemu-storage-daemon') + test_deps += [qsd] + endif + + foreach test : target_pytests + test('pytest-@0@/@1@'.format(target_base, test), + pytest_executables[test], + depends: [test_deps, pytest_emulator, emulator_modules, other_deps], + env: pytest_env, + args: ['--tap', '-k'], + protocol: 'tap', + timeout: slow_pytests.get(test, 30), + priority: slow_pytests.get(test, 30), + suite: ['pytest', 'pytest-' + target_base]) + endforeach +endforeach -- 2.25.1 ^ permalink raw reply related [flat|nested] 90+ messages in thread
* Re: [PATCH v2 10/11] pytest: add pytest to the meson build system 2022-07-10 17:00 ` [PATCH v2 10/11] pytest: add pytest to the meson build system Ani Sinha @ 2022-07-11 20:46 ` John Snow 2022-07-12 6:52 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: John Snow @ 2022-07-11 20:46 UTC (permalink / raw) To: Ani Sinha Cc: qemu-devel, Thomas Huth, Peter Maydell, Daniel Berrange, Paolo Bonzini, Igor Mammedov, Michael Tsirkin On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > Integrate the pytest framework with the meson build system. This will make meson > run all the pytests under the pytest directory. > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > --- > tests/Makefile.include | 4 +++- > tests/meson.build | 1 + > tests/pytest/meson.build | 49 ++++++++++++++++++++++++++++++++++++++++ > 3 files changed, 53 insertions(+), 1 deletion(-) > create mode 100644 tests/pytest/meson.build > > diff --git a/tests/Makefile.include b/tests/Makefile.include > index 3accb83b13..40755a6bd1 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -3,12 +3,14 @@ > .PHONY: check-help > check-help: > @echo "Regression testing targets:" > - @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests" > + @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest, pytest and decodetree tests" Does this mean that "make check" *requires* an internet connection? If so, I'm afraid that introduces some complications for downstreams which require that "make check" can run without an internet connection. It's something I've been trying to wrestle with as I split the qemu.qmp library out of the QEMU tree, and I have been working (slowly) on remedying it with some additional Makefile logic. I have been looking into making a testing "dummy package" for python with optional dependency groups that use a venv building script to either pull from PyPI in online mode, or use system packages in offline mode. In the case of offline mode, I am working on augmenting the configure script to test that dependencies are met, and disabling test groups when the correct dependencies cannot be located. I hope to have another version of that series soon; it should be trivial to add a new optional dependency group to it. I'll CC you on the next version of the series. --js > @echo " $(MAKE) bench Run speed tests" > @echo > @echo "Individual test suites:" > @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target" > @echo " $(MAKE) check-qtest Run qtest tests" > + @echo " $(MAKE) check-pytest Run pytest tests" > + @echo " $(MAKE) check-pytest-TARGET Run pytest for a given target" > @echo " $(MAKE) check-unit Run qobject tests" > @echo " $(MAKE) check-qapi-schema Run QAPI schema tests" > @echo " $(MAKE) check-block Run block tests" > diff --git a/tests/meson.build b/tests/meson.build > index 8e318ec513..f344cbdc6c 100644 > --- a/tests/meson.build > +++ b/tests/meson.build > @@ -93,3 +93,4 @@ subdir('unit') > subdir('qapi-schema') > subdir('qtest') > subdir('migration') > +subdir('pytest') > diff --git a/tests/pytest/meson.build b/tests/pytest/meson.build > new file mode 100644 > index 0000000000..e60d481ae4 > --- /dev/null > +++ b/tests/pytest/meson.build > @@ -0,0 +1,49 @@ > +slow_pytests = { > + 'acpi-bits' : 120, > +} > + > +pytests_generic = [] > + > +# biosbits tests are currenly only supported on x86_64 platforms. > +pytests_x86_64 = ['acpi-bits-test'] > + > +pytest_executables = {} > +other_deps = [] > + > +subdir('acpi-bits') > + > +foreach dir : target_dirs > + if not dir.endswith('-softmmu') > + continue > + endif > + > + target_base = dir.split('-')[0] > + pytest_emulator = emulators['qemu-system-' + target_base] > + target_pytests = get_variable('pytests_' + target_base, []) + pytests_generic > + > + test_deps = roms > + pytest_env = environment() > + if have_tools > + pytest_env.set('QTEST_QEMU_IMG', './qemu-img') > + test_deps += [qemu_img] > + endif > + pytest_env.set('G_TEST_DBUS_DAEMON', meson.project_source_root() / 'tests/dbus-vmstate-daemon.sh') > + pytest_env.set('PYTEST_QEMU_BINARY', './qemu-system-' + target_base) > + pytest_env.set('PYTEST_SOURCE_ROOT', meson.project_source_root()) > + if have_tools and have_vhost_user_blk_server > + pytest_env.set('PYTEST_QEMU_STORAGE_DAEMON_BINARY', './storage-daemon/qemu-storage-daemon') > + test_deps += [qsd] > + endif > + > + foreach test : target_pytests > + test('pytest-@0@/@1@'.format(target_base, test), > + pytest_executables[test], > + depends: [test_deps, pytest_emulator, emulator_modules, other_deps], > + env: pytest_env, > + args: ['--tap', '-k'], > + protocol: 'tap', > + timeout: slow_pytests.get(test, 30), > + priority: slow_pytests.get(test, 30), > + suite: ['pytest', 'pytest-' + target_base]) > + endforeach > +endforeach > -- > 2.25.1 > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 10/11] pytest: add pytest to the meson build system 2022-07-11 20:46 ` John Snow @ 2022-07-12 6:52 ` Ani Sinha 2022-09-06 13:10 ` Daniel P. Berrangé 0 siblings, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-07-12 6:52 UTC (permalink / raw) To: John Snow Cc: Ani Sinha, qemu-devel, Thomas Huth, Peter Maydell, Daniel Berrange, Paolo Bonzini, Igor Mammedov, Michael Tsirkin On Mon, 11 Jul 2022, John Snow wrote: > On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > > > Integrate the pytest framework with the meson build system. This will make meson > > run all the pytests under the pytest directory. > > > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > > --- > > tests/Makefile.include | 4 +++- > > tests/meson.build | 1 + > > tests/pytest/meson.build | 49 ++++++++++++++++++++++++++++++++++++++++ > > 3 files changed, 53 insertions(+), 1 deletion(-) > > create mode 100644 tests/pytest/meson.build > > > > diff --git a/tests/Makefile.include b/tests/Makefile.include > > index 3accb83b13..40755a6bd1 100644 > > --- a/tests/Makefile.include > > +++ b/tests/Makefile.include > > @@ -3,12 +3,14 @@ > > .PHONY: check-help > > check-help: > > @echo "Regression testing targets:" > > - @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests" > > + @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest, pytest and decodetree tests" > > Does this mean that "make check" *requires* an internet connection? No. My test will be skipped if it is unable to download the artifacts it requires due to lack of Internet connectivity. If > so, I'm afraid that introduces some complications for downstreams > which require that "make check" can run without an internet > connection. It's something I've been trying to wrestle with as I split > the qemu.qmp library out of the QEMU tree, and I have been working > (slowly) on remedying it with some additional Makefile logic. > > I have been looking into making a testing "dummy package" for python > with optional dependency groups that use a venv building script to > either pull from PyPI in online mode, or use system packages in > offline mode. In the case of offline mode, I am working on augmenting > the configure script to test that dependencies are met, and disabling > test groups when the correct dependencies cannot be located. > > I hope to have another version of that series soon; it should be > trivial to add a new optional dependency group to it. I'll CC you on > the next version of the series. > > --js > > > @echo " $(MAKE) bench Run speed tests" > > @echo > > @echo "Individual test suites:" > > @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target" > > @echo " $(MAKE) check-qtest Run qtest tests" > > + @echo " $(MAKE) check-pytest Run pytest tests" > > + @echo " $(MAKE) check-pytest-TARGET Run pytest for a given target" > > @echo " $(MAKE) check-unit Run qobject tests" > > @echo " $(MAKE) check-qapi-schema Run QAPI schema tests" > > @echo " $(MAKE) check-block Run block tests" > > diff --git a/tests/meson.build b/tests/meson.build > > index 8e318ec513..f344cbdc6c 100644 > > --- a/tests/meson.build > > +++ b/tests/meson.build > > @@ -93,3 +93,4 @@ subdir('unit') > > subdir('qapi-schema') > > subdir('qtest') > > subdir('migration') > > +subdir('pytest') > > diff --git a/tests/pytest/meson.build b/tests/pytest/meson.build > > new file mode 100644 > > index 0000000000..e60d481ae4 > > --- /dev/null > > +++ b/tests/pytest/meson.build > > @@ -0,0 +1,49 @@ > > +slow_pytests = { > > + 'acpi-bits' : 120, > > +} > > + > > +pytests_generic = [] > > + > > +# biosbits tests are currenly only supported on x86_64 platforms. > > +pytests_x86_64 = ['acpi-bits-test'] > > + > > +pytest_executables = {} > > +other_deps = [] > > + > > +subdir('acpi-bits') > > + > > +foreach dir : target_dirs > > + if not dir.endswith('-softmmu') > > + continue > > + endif > > + > > + target_base = dir.split('-')[0] > > + pytest_emulator = emulators['qemu-system-' + target_base] > > + target_pytests = get_variable('pytests_' + target_base, []) + pytests_generic > > + > > + test_deps = roms > > + pytest_env = environment() > > + if have_tools > > + pytest_env.set('QTEST_QEMU_IMG', './qemu-img') > > + test_deps += [qemu_img] > > + endif > > + pytest_env.set('G_TEST_DBUS_DAEMON', meson.project_source_root() / 'tests/dbus-vmstate-daemon.sh') > > + pytest_env.set('PYTEST_QEMU_BINARY', './qemu-system-' + target_base) > > + pytest_env.set('PYTEST_SOURCE_ROOT', meson.project_source_root()) > > + if have_tools and have_vhost_user_blk_server > > + pytest_env.set('PYTEST_QEMU_STORAGE_DAEMON_BINARY', './storage-daemon/qemu-storage-daemon') > > + test_deps += [qsd] > > + endif > > + > > + foreach test : target_pytests > > + test('pytest-@0@/@1@'.format(target_base, test), > > + pytest_executables[test], > > + depends: [test_deps, pytest_emulator, emulator_modules, other_deps], > > + env: pytest_env, > > + args: ['--tap', '-k'], > > + protocol: 'tap', > > + timeout: slow_pytests.get(test, 30), > > + priority: slow_pytests.get(test, 30), > > + suite: ['pytest', 'pytest-' + target_base]) > > + endforeach > > +endforeach > > -- > > 2.25.1 > > > > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 10/11] pytest: add pytest to the meson build system 2022-07-12 6:52 ` Ani Sinha @ 2022-09-06 13:10 ` Daniel P. Berrangé 2022-09-27 21:26 ` Michael S. Tsirkin 0 siblings, 1 reply; 90+ messages in thread From: Daniel P. Berrangé @ 2022-09-06 13:10 UTC (permalink / raw) To: Ani Sinha Cc: John Snow, qemu-devel, Thomas Huth, Peter Maydell, Paolo Bonzini, Igor Mammedov, Michael Tsirkin On Tue, Jul 12, 2022 at 12:22:10PM +0530, Ani Sinha wrote: > > > On Mon, 11 Jul 2022, John Snow wrote: > > > On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > Integrate the pytest framework with the meson build system. This will make meson > > > run all the pytests under the pytest directory. > > > > > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > > > --- > > > tests/Makefile.include | 4 +++- > > > tests/meson.build | 1 + > > > tests/pytest/meson.build | 49 ++++++++++++++++++++++++++++++++++++++++ > > > 3 files changed, 53 insertions(+), 1 deletion(-) > > > create mode 100644 tests/pytest/meson.build > > > > > > diff --git a/tests/Makefile.include b/tests/Makefile.include > > > index 3accb83b13..40755a6bd1 100644 > > > --- a/tests/Makefile.include > > > +++ b/tests/Makefile.include > > > @@ -3,12 +3,14 @@ > > > .PHONY: check-help > > > check-help: > > > @echo "Regression testing targets:" > > > - @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests" > > > + @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest, pytest and decodetree tests" > > > > Does this mean that "make check" *requires* an internet connection? > > No. My test will be skipped if it is unable to download the artifacts it > requires due to lack of Internet connectivity. That's not the only concern, there are also people who have metered internet connections, or whose connections are slow and thus have long download times. Any test that downloads should be opt-in only. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 10/11] pytest: add pytest to the meson build system 2022-09-06 13:10 ` Daniel P. Berrangé @ 2022-09-27 21:26 ` Michael S. Tsirkin 2022-09-28 7:32 ` Thomas Huth 0 siblings, 1 reply; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-27 21:26 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Ani Sinha, John Snow, qemu-devel, Thomas Huth, Peter Maydell, Paolo Bonzini, Igor Mammedov On Tue, Sep 06, 2022 at 02:10:56PM +0100, Daniel P. Berrangé wrote: > On Tue, Jul 12, 2022 at 12:22:10PM +0530, Ani Sinha wrote: > > > > > > On Mon, 11 Jul 2022, John Snow wrote: > > > > > On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > Integrate the pytest framework with the meson build system. This will make meson > > > > run all the pytests under the pytest directory. > > > > > > > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > > > > --- > > > > tests/Makefile.include | 4 +++- > > > > tests/meson.build | 1 + > > > > tests/pytest/meson.build | 49 ++++++++++++++++++++++++++++++++++++++++ > > > > 3 files changed, 53 insertions(+), 1 deletion(-) > > > > create mode 100644 tests/pytest/meson.build > > > > > > > > diff --git a/tests/Makefile.include b/tests/Makefile.include > > > > index 3accb83b13..40755a6bd1 100644 > > > > --- a/tests/Makefile.include > > > > +++ b/tests/Makefile.include > > > > @@ -3,12 +3,14 @@ > > > > .PHONY: check-help > > > > check-help: > > > > @echo "Regression testing targets:" > > > > - @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests" > > > > + @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest, pytest and decodetree tests" > > > > > > Does this mean that "make check" *requires* an internet connection? > > > > No. My test will be skipped if it is unable to download the artifacts it > > requires due to lack of Internet connectivity. > > That's not the only concern, there are also people who have metered > internet connections, or whose connections are slow and thus have > long download times. Any test that downloads should be opt-in only. > > > With regards, > Daniel <rant> This is why I wanted git submodules. A well understood decentralized model. Now we are reinventing them badly. I asked on the maintainers summit what issues people have with submodules, no one volunteered any information. It might make sense to figure out if there's a way to use submodules sanely. </rant> Anyway, download should just be done separately, make check should just verify it has the correct binary and if not fail. And I'd like to have a target that fails if it can not run the tests a opposed to skipping. > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 10/11] pytest: add pytest to the meson build system 2022-09-27 21:26 ` Michael S. Tsirkin @ 2022-09-28 7:32 ` Thomas Huth 2022-09-28 9:23 ` Michael S. Tsirkin 0 siblings, 1 reply; 90+ messages in thread From: Thomas Huth @ 2022-09-28 7:32 UTC (permalink / raw) To: Michael S. Tsirkin, Daniel P. Berrangé Cc: Ani Sinha, John Snow, qemu-devel, Peter Maydell, Paolo Bonzini, Igor Mammedov On 27/09/2022 23.26, Michael S. Tsirkin wrote: > On Tue, Sep 06, 2022 at 02:10:56PM +0100, Daniel P. Berrangé wrote: >> On Tue, Jul 12, 2022 at 12:22:10PM +0530, Ani Sinha wrote: >>> >>> >>> On Mon, 11 Jul 2022, John Snow wrote: >>> >>>> On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: >>>>> >>>>> Integrate the pytest framework with the meson build system. This will make meson >>>>> run all the pytests under the pytest directory. >>>>> >>>>> Signed-off-by: Ani Sinha <ani@anisinha.ca> >>>>> --- >>>>> tests/Makefile.include | 4 +++- >>>>> tests/meson.build | 1 + >>>>> tests/pytest/meson.build | 49 ++++++++++++++++++++++++++++++++++++++++ >>>>> 3 files changed, 53 insertions(+), 1 deletion(-) >>>>> create mode 100644 tests/pytest/meson.build >>>>> >>>>> diff --git a/tests/Makefile.include b/tests/Makefile.include >>>>> index 3accb83b13..40755a6bd1 100644 >>>>> --- a/tests/Makefile.include >>>>> +++ b/tests/Makefile.include >>>>> @@ -3,12 +3,14 @@ >>>>> .PHONY: check-help >>>>> check-help: >>>>> @echo "Regression testing targets:" >>>>> - @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests" >>>>> + @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest, pytest and decodetree tests" >>>> >>>> Does this mean that "make check" *requires* an internet connection? >>> >>> No. My test will be skipped if it is unable to download the artifacts it >>> requires due to lack of Internet connectivity. >> >> That's not the only concern, there are also people who have metered >> internet connections, or whose connections are slow and thus have >> long download times. Any test that downloads should be opt-in only. >> >> >> With regards, >> Daniel > > <rant> > This is why I wanted git submodules. A well understood decentralized > model. Now we are reinventing them badly. I don't see much of a difference here with submodules with regards to Daniel's concerns: If you clone the QEMU repo with a good internet connection, you also don't get the submodules by default. If you then move to a bad internet connection or offline and the build system tries to pull in the submodule, you've also lost. OTOH, with downloads, you can also make sure to have all downloads cached while you're still on the good internet connection. The cache should continue working if you're then moving offline. > I asked on the maintainers summit what issues people have with > submodules, no one volunteered any information. You know my position already (https://lore.kernel.org/qemu-devel/d7a7b28f-a665-2567-0fb6-e31e7ecbb5c8@redhat.com/), so at least I did not feel the need to repeat that again (and we were pretty short in time anyway). Thomas ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 10/11] pytest: add pytest to the meson build system 2022-09-28 7:32 ` Thomas Huth @ 2022-09-28 9:23 ` Michael S. Tsirkin 0 siblings, 0 replies; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-28 9:23 UTC (permalink / raw) To: Thomas Huth Cc: Daniel P. Berrangé, Ani Sinha, John Snow, qemu-devel, Peter Maydell, Paolo Bonzini, Igor Mammedov On Wed, Sep 28, 2022 at 09:32:57AM +0200, Thomas Huth wrote: > On 27/09/2022 23.26, Michael S. Tsirkin wrote: > > On Tue, Sep 06, 2022 at 02:10:56PM +0100, Daniel P. Berrangé wrote: > > > On Tue, Jul 12, 2022 at 12:22:10PM +0530, Ani Sinha wrote: > > > > > > > > > > > > On Mon, 11 Jul 2022, John Snow wrote: > > > > > > > > > On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote: > > > > > > > > > > > > Integrate the pytest framework with the meson build system. This will make meson > > > > > > run all the pytests under the pytest directory. > > > > > > > > > > > > Signed-off-by: Ani Sinha <ani@anisinha.ca> > > > > > > --- > > > > > > tests/Makefile.include | 4 +++- > > > > > > tests/meson.build | 1 + > > > > > > tests/pytest/meson.build | 49 ++++++++++++++++++++++++++++++++++++++++ > > > > > > 3 files changed, 53 insertions(+), 1 deletion(-) > > > > > > create mode 100644 tests/pytest/meson.build > > > > > > > > > > > > diff --git a/tests/Makefile.include b/tests/Makefile.include > > > > > > index 3accb83b13..40755a6bd1 100644 > > > > > > --- a/tests/Makefile.include > > > > > > +++ b/tests/Makefile.include > > > > > > @@ -3,12 +3,14 @@ > > > > > > .PHONY: check-help > > > > > > check-help: > > > > > > @echo "Regression testing targets:" > > > > > > - @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests" > > > > > > + @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest, pytest and decodetree tests" > > > > > > > > > > Does this mean that "make check" *requires* an internet connection? > > > > > > > > No. My test will be skipped if it is unable to download the artifacts it > > > > requires due to lack of Internet connectivity. > > > > > > That's not the only concern, there are also people who have metered > > > internet connections, or whose connections are slow and thus have > > > long download times. Any test that downloads should be opt-in only. > > > > > > > > > With regards, > > > Daniel > > > > <rant> > > This is why I wanted git submodules. A well understood decentralized > > model. Now we are reinventing them badly. > > I don't see much of a difference here with submodules with regards to > Daniel's concerns: If you clone the QEMU repo with a good internet > connection, you also don't get the submodules by default. If you then move > to a bad internet connection or offline and the build system tries to pull > in the submodule, you've also lost. > OTOH, with downloads, you can also make sure to have all downloads cached > while you're still on the good internet connection. The cache should > continue working if you're then moving offline. submodules as a cache include checksumming, a way to make local changes without too much pain, documentation .... > > I asked on the maintainers summit what issues people have with > > submodules, no one volunteered any information. > > You know my position already (https://lore.kernel.org/qemu-devel/d7a7b28f-a665-2567-0fb6-e31e7ecbb5c8@redhat.com/), > so at least I did not feel the need to repeat that again (and we were pretty > short in time anyway). Pity, but ok, I will revive that thread. > > Thomas ^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v2 11/11] MAINTAINERS: add myself as the maintainer for acpi biosbits pytests 2022-07-10 17:00 [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Ani Sinha ` (9 preceding siblings ...) 2022-07-10 17:00 ` [PATCH v2 10/11] pytest: add pytest to the meson build system Ani Sinha @ 2022-07-10 17:00 ` Ani Sinha 2022-07-11 9:33 ` [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Michael S. Tsirkin 11 siblings, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-07-10 17:00 UTC (permalink / raw) To: qemu-devel Cc: thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo, mst, Ani Sinha I wrote the biosbits test framework, the pytest for testing QEMU with biosbits and all the related changes. Making myself as the maintainer for biosbits related files and test scripts. Signed-off-by: Ani Sinha <ani@anisinha.ca> --- MAINTAINERS | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index aaa649a50d..32fbd27b66 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1860,6 +1860,11 @@ S: Supported F: hw/acpi/viot.c F: hw/acpi/viot.h +ACPI/PYTEST/BIOSBITS +M: Ani Sinha <ani@anisinha.ca> +S: Supported +F: tests/pytest/acpi-bits/* + ACPI/HEST/GHES R: Dongjiu Geng <gengdongjiu1@gmail.com> L: qemu-arm@nongnu.org -- 2.25.1 ^ permalink raw reply related [flat|nested] 90+ messages in thread
* Re: [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits 2022-07-10 17:00 [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Ani Sinha ` (10 preceding siblings ...) 2022-07-10 17:00 ` [PATCH v2 11/11] MAINTAINERS: add myself as the maintainer for acpi biosbits pytests Ani Sinha @ 2022-07-11 9:33 ` Michael S. Tsirkin 2022-07-14 13:24 ` Peter Maydell 11 siblings, 1 reply; 90+ messages in thread From: Michael S. Tsirkin @ 2022-07-11 9:33 UTC (permalink / raw) To: Ani Sinha Cc: qemu-devel, thuth, peter.maydell, berrange, jsnow, pbonzini, imammedo On Sun, Jul 10, 2022 at 10:30:03PM +0530, Ani Sinha wrote: > Changelog: > v2: > - a new class of python based tests introduced that is separate from avocado > tests or qtests. Can be run by using "make check-pytest". > - acpi biosbits tests are the first tests to use pytest environment. > - bios bits tests now download the bits binary archives from a remote > repository if they are not found locally. The test skips if download > fails. > - A new environment variable is introduced that can be passed by the tester > to specify the location of the bits archives locally. test skips if the > bits binaries are not found in that location. > - if pip install of python module fails for whatever reaoson, the test skips. > - misc code fixes including spell check of the README doc. README has been > updated as well. > - addition of SPDX license headers to bits test files. > - update MAINTAINERS to reflect the new pytest test class. > > For biosbits repo: > - added Dockerfile and build script. Made bios bits build on gcc 11. > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > The build script generates the zip archive and tarball used by the test. So far so good, I think it's ok for a start. It's probably a good idea to host the source on qemu.org. Peter - any objection to this? > v1: initial patchset. uses qtest to implement the bios bits tests. > > Ani Sinha (11): > acpi/tests/bits: initial commit of test scripts that are run by > biosbits > acpi/tests/bits: add SPDX license identifiers for bios bits tests > acpi/tests/bits: disable acpi PSS tests that are failing in biosbits > acpi/tests/bits: add smilatency test suite from bits in order to > disable it > acpi/tests/bits: add SPDX license identifiers for bios bits smilatency > tests > acpi/tests/bits: disable smilatency test since it does not pass > everytime > acpi/tests/bits: add python test that exercizes QEMU bios tables using > biosbits > acpi/tests/bits: add biosbits config file for running bios tests > acpi/tests/bits: add a README file describing bits pytests > pytest: add pytest to the meson build system > MAINTAINERS: add myself as the maintainer for acpi biosbits pytests > > MAINTAINERS | 5 + > tests/Makefile.include | 4 +- > tests/meson.build | 1 + > tests/pytest/acpi-bits/README | 129 + > tests/pytest/acpi-bits/acpi-bits-test-venv.sh | 59 + > tests/pytest/acpi-bits/acpi-bits-test.py | 382 +++ > .../pytest/acpi-bits/bits-config/bits-cfg.txt | 18 + > .../pytest/acpi-bits/bits-config/meson.build | 11 + > tests/pytest/acpi-bits/bits-tests/meson.build | 11 + > tests/pytest/acpi-bits/bits-tests/smbios.py | 2432 +++++++++++++++++ > .../pytest/acpi-bits/bits-tests/smilatency.py | 105 + > tests/pytest/acpi-bits/bits-tests/testacpi.py | 285 ++ > .../pytest/acpi-bits/bits-tests/testcpuid.py | 85 + > tests/pytest/acpi-bits/meson.build | 33 + > tests/pytest/acpi-bits/requirements.txt | 1 + > tests/pytest/meson.build | 49 + > 16 files changed, 3609 insertions(+), 1 deletion(-) > create mode 100644 tests/pytest/acpi-bits/README > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test-venv.sh > create mode 100644 tests/pytest/acpi-bits/acpi-bits-test.py > create mode 100644 tests/pytest/acpi-bits/bits-config/bits-cfg.txt > create mode 100644 tests/pytest/acpi-bits/bits-config/meson.build > create mode 100644 tests/pytest/acpi-bits/bits-tests/meson.build > create mode 100644 tests/pytest/acpi-bits/bits-tests/smbios.py > create mode 100644 tests/pytest/acpi-bits/bits-tests/smilatency.py > create mode 100644 tests/pytest/acpi-bits/bits-tests/testacpi.py > create mode 100644 tests/pytest/acpi-bits/bits-tests/testcpuid.py > create mode 100644 tests/pytest/acpi-bits/meson.build > create mode 100644 tests/pytest/acpi-bits/requirements.txt > create mode 100644 tests/pytest/meson.build > > -- > 2.25.1 ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits 2022-07-11 9:33 ` [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Michael S. Tsirkin @ 2022-07-14 13:24 ` Peter Maydell 2022-09-06 6:26 ` Ani Sinha 2022-09-06 13:15 ` Daniel P. Berrangé 0 siblings, 2 replies; 90+ messages in thread From: Peter Maydell @ 2022-07-14 13:24 UTC (permalink / raw) To: Michael S. Tsirkin Cc: Ani Sinha, qemu-devel, thuth, berrange, jsnow, pbonzini, imammedo On Mon, 11 Jul 2022 at 10:34, Michael S. Tsirkin <mst@redhat.com> wrote: > > On Sun, Jul 10, 2022 at 10:30:03PM +0530, Ani Sinha wrote: > > Changelog: > > v2: > > - a new class of python based tests introduced that is separate from avocado > > tests or qtests. Can be run by using "make check-pytest". > > - acpi biosbits tests are the first tests to use pytest environment. > > - bios bits tests now download the bits binary archives from a remote > > repository if they are not found locally. The test skips if download > > fails. > > - A new environment variable is introduced that can be passed by the tester > > to specify the location of the bits archives locally. test skips if the > > bits binaries are not found in that location. > > - if pip install of python module fails for whatever reaoson, the test skips. > > - misc code fixes including spell check of the README doc. README has been > > updated as well. > > - addition of SPDX license headers to bits test files. > > - update MAINTAINERS to reflect the new pytest test class. > > > > For biosbits repo: > > - added Dockerfile and build script. Made bios bits build on gcc 11. > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > The build script generates the zip archive and tarball used by the test. > > So far so good, I think it's ok for a start. It's probably a good idea > to host the source on qemu.org. Peter - any objection to this? Dan was looking at v1 from the point of view of how we handle the guest binary blobs for these tests -- I'd rather defer to him rather than taking the time to get up to speed on the issue myself. thanks -- PMM ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits 2022-07-14 13:24 ` Peter Maydell @ 2022-09-06 6:26 ` Ani Sinha 2022-09-06 8:19 ` Michael S. Tsirkin 2022-09-06 13:15 ` Daniel P. Berrangé 1 sibling, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-09-06 6:26 UTC (permalink / raw) To: Peter Maydell Cc: Michael S. Tsirkin, qemu-devel, thuth, berrange, jsnow, pbonzini, imammedo On Thu, Jul 14, 2022 at 6:54 PM Peter Maydell <peter.maydell@linaro.org> wrote: > > On Mon, 11 Jul 2022 at 10:34, Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Sun, Jul 10, 2022 at 10:30:03PM +0530, Ani Sinha wrote: > > > Changelog: > > > v2: > > > - a new class of python based tests introduced that is separate from avocado > > > tests or qtests. Can be run by using "make check-pytest". > > > - acpi biosbits tests are the first tests to use pytest environment. > > > - bios bits tests now download the bits binary archives from a remote > > > repository if they are not found locally. The test skips if download > > > fails. > > > - A new environment variable is introduced that can be passed by the tester > > > to specify the location of the bits archives locally. test skips if the > > > bits binaries are not found in that location. > > > - if pip install of python module fails for whatever reaoson, the test skips. > > > - misc code fixes including spell check of the README doc. README has been > > > updated as well. > > > - addition of SPDX license headers to bits test files. > > > - update MAINTAINERS to reflect the new pytest test class. > > > > > > For biosbits repo: > > > - added Dockerfile and build script. Made bios bits build on gcc 11. > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > > The build script generates the zip archive and tarball used by the test. > > > > So far so good, I think it's ok for a start. It's probably a good idea > > to host the source on qemu.org. Peter - any objection to this? > > Dan was looking at v1 from the point of view of how we handle the > guest binary blobs for these tests -- I'd rather defer to him rather > than taking the time to get up to speed on the issue myself. Ok let's resurrect this discussion again. What are we going to do with bios bits? Put it in git.qemu.org then? I have put a lot of time and effort into this work and I believe this will add another valuable tool to test acpi stuff, so I am not going away :-) ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits 2022-09-06 6:26 ` Ani Sinha @ 2022-09-06 8:19 ` Michael S. Tsirkin 0 siblings, 0 replies; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-06 8:19 UTC (permalink / raw) To: Ani Sinha Cc: Peter Maydell, qemu-devel, thuth, berrange, jsnow, pbonzini, imammedo On Tue, Sep 06, 2022 at 11:56:33AM +0530, Ani Sinha wrote: > On Thu, Jul 14, 2022 at 6:54 PM Peter Maydell <peter.maydell@linaro.org> wrote: > > > > On Mon, 11 Jul 2022 at 10:34, Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > On Sun, Jul 10, 2022 at 10:30:03PM +0530, Ani Sinha wrote: > > > > Changelog: > > > > v2: > > > > - a new class of python based tests introduced that is separate from avocado > > > > tests or qtests. Can be run by using "make check-pytest". > > > > - acpi biosbits tests are the first tests to use pytest environment. > > > > - bios bits tests now download the bits binary archives from a remote > > > > repository if they are not found locally. The test skips if download > > > > fails. > > > > - A new environment variable is introduced that can be passed by the tester > > > > to specify the location of the bits archives locally. test skips if the > > > > bits binaries are not found in that location. > > > > - if pip install of python module fails for whatever reaoson, the test skips. > > > > - misc code fixes including spell check of the README doc. README has been > > > > updated as well. > > > > - addition of SPDX license headers to bits test files. > > > > - update MAINTAINERS to reflect the new pytest test class. > > > > > > > > For biosbits repo: > > > > - added Dockerfile and build script. Made bios bits build on gcc 11. > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > > > The build script generates the zip archive and tarball used by the test. > > > > > > So far so good, I think it's ok for a start. It's probably a good idea > > > to host the source on qemu.org. Peter - any objection to this? > > > > Dan was looking at v1 from the point of view of how we handle the > > guest binary blobs for these tests -- I'd rather defer to him rather > > than taking the time to get up to speed on the issue myself. > > Ok let's resurrect this discussion again. What are we going to do with > bios bits? Put it in git.qemu.org then? > I have put a lot of time and effort into this work and I believe this > will add another valuable tool to test acpi stuff, so I am not going > away :-) This makes sense to me. Peter, ok? Could you create a repo for Ani if yes? -- MST ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits 2022-07-14 13:24 ` Peter Maydell 2022-09-06 6:26 ` Ani Sinha @ 2022-09-06 13:15 ` Daniel P. Berrangé 2022-09-06 13:28 ` Ani Sinha 2022-09-06 14:03 ` Michael S. Tsirkin 1 sibling, 2 replies; 90+ messages in thread From: Daniel P. Berrangé @ 2022-09-06 13:15 UTC (permalink / raw) To: Peter Maydell Cc: Michael S. Tsirkin, Ani Sinha, qemu-devel, thuth, jsnow, pbonzini, imammedo On Thu, Jul 14, 2022 at 02:24:18PM +0100, Peter Maydell wrote: > On Mon, 11 Jul 2022 at 10:34, Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Sun, Jul 10, 2022 at 10:30:03PM +0530, Ani Sinha wrote: > > > Changelog: > > > v2: > > > - a new class of python based tests introduced that is separate from avocado > > > tests or qtests. Can be run by using "make check-pytest". > > > - acpi biosbits tests are the first tests to use pytest environment. > > > - bios bits tests now download the bits binary archives from a remote > > > repository if they are not found locally. The test skips if download > > > fails. > > > - A new environment variable is introduced that can be passed by the tester > > > to specify the location of the bits archives locally. test skips if the > > > bits binaries are not found in that location. > > > - if pip install of python module fails for whatever reaoson, the test skips. > > > - misc code fixes including spell check of the README doc. README has been > > > updated as well. > > > - addition of SPDX license headers to bits test files. > > > - update MAINTAINERS to reflect the new pytest test class. > > > > > > For biosbits repo: > > > - added Dockerfile and build script. Made bios bits build on gcc 11. > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > > The build script generates the zip archive and tarball used by the test. > > > > So far so good, I think it's ok for a start. It's probably a good idea > > to host the source on qemu.org. Peter - any objection to this? > > Dan was looking at v1 from the point of view of how we handle the > guest binary blobs for these tests -- I'd rather defer to him rather > than taking the time to get up to speed on the issue myself. Storing the *source* git repo for biosbits on gitlab.com/qemu-project is sensible, as that's what we've done for other 3rd party bits that we bundle/depend on git repo access for. The above git repo, however, has extra branches that also store the binary builds, and I'm not convinced that is a good idea. I feel like the source git repo should have a .gitlab-ci.yml file that builds the binary and publishes it as an CI artifact. This lets us keep the binary outside of GIT, have a CI job that periodically refreshes the binary (eg so when the Ubuntu version that Dockerfile uses goes EOL we can use something else). The test can access the CI artifact directly. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits 2022-09-06 13:15 ` Daniel P. Berrangé @ 2022-09-06 13:28 ` Ani Sinha 2022-09-06 13:30 ` Daniel P. Berrangé 2022-09-06 14:03 ` Michael S. Tsirkin 1 sibling, 1 reply; 90+ messages in thread From: Ani Sinha @ 2022-09-06 13:28 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Michael S. Tsirkin, Peter Maydell, imammedo, jsnow, pbonzini, qemu-devel, thuth [-- Attachment #1: Type: text/plain, Size: 3348 bytes --] On Tue, Sep 6, 2022 at 18:45 Daniel P. Berrangé <berrange@redhat.com> wrote: > On Thu, Jul 14, 2022 at 02:24:18PM +0100, Peter Maydell wrote: > > On Mon, 11 Jul 2022 at 10:34, Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > On Sun, Jul 10, 2022 at 10:30:03PM +0530, Ani Sinha wrote: > > > > Changelog: > > > > v2: > > > > - a new class of python based tests introduced that is separate > from avocado > > > > tests or qtests. Can be run by using "make check-pytest". > > > > - acpi biosbits tests are the first tests to use pytest environment. > > > > - bios bits tests now download the bits binary archives from a > remote > > > > repository if they are not found locally. The test skips if > download > > > > fails. > > > > - A new environment variable is introduced that can be passed by > the tester > > > > to specify the location of the bits archives locally. test skips > if the > > > > bits binaries are not found in that location. > > > > - if pip install of python module fails for whatever reaoson, the > test skips. > > > > - misc code fixes including spell check of the README doc. README > has been > > > > updated as well. > > > > - addition of SPDX license headers to bits test files. > > > > - update MAINTAINERS to reflect the new pytest test class. > > > > > > > > For biosbits repo: > > > > - added Dockerfile and build script. Made bios bits build on gcc 11. > > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile > > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > > > The build script generates the zip archive and tarball used by > the test. > > > > > > So far so good, I think it's ok for a start. It's probably a good idea > > > to host the source on qemu.org. Peter - any objection to this? > > > > Dan was looking at v1 from the point of view of how we handle the > > guest binary blobs for these tests -- I'd rather defer to him rather > > than taking the time to get up to speed on the issue myself. > > Storing the *source* git repo for biosbits on gitlab.com/qemu-project > is sensible, as that's what we've done for other 3rd party bits that > we bundle/depend on git repo access for. Great. Can you or Peter please create a git repo cloned from the official bios bits repo please? You don’t have to clone mine. Please provide me with push access so that I can push fixes that I have made in order to build it. > > > The above git repo, however, has extra branches that also store the > binary builds, and I'm not convinced that is a good idea. > > I feel like the source git repo should have a .gitlab-ci.yml file > that builds the binary and publishes it as an CI artifact. This > lets us keep the binary outside of GIT, have a CI job that periodically > refreshes the binary (eg so when the Ubuntu version that Dockerfile > uses goes EOL we can use something else). The test can access the CI > artifact directly. I can look into this incrementally. There is already a dockerfile for the build. I can look into pushing the yaml file as well. As for the binaries yes we need to decide where to keep thrm. But one step at a time . As for your other complaints like making the test opt-in, I’ll look into addressing them in v3. [-- Attachment #2: Type: text/html, Size: 4738 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits 2022-09-06 13:28 ` Ani Sinha @ 2022-09-06 13:30 ` Daniel P. Berrangé 2022-09-09 12:53 ` Ani Sinha 0 siblings, 1 reply; 90+ messages in thread From: Daniel P. Berrangé @ 2022-09-06 13:30 UTC (permalink / raw) To: Ani Sinha Cc: Michael S. Tsirkin, Peter Maydell, imammedo, jsnow, pbonzini, qemu-devel, thuth On Tue, Sep 06, 2022 at 06:58:02PM +0530, Ani Sinha wrote: > On Tue, Sep 6, 2022 at 18:45 Daniel P. Berrangé <berrange@redhat.com> wrote: > > > On Thu, Jul 14, 2022 at 02:24:18PM +0100, Peter Maydell wrote: > > > On Mon, 11 Jul 2022 at 10:34, Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > On Sun, Jul 10, 2022 at 10:30:03PM +0530, Ani Sinha wrote: > > > > > Changelog: > > > > > v2: > > > > > - a new class of python based tests introduced that is separate > > from avocado > > > > > tests or qtests. Can be run by using "make check-pytest". > > > > > - acpi biosbits tests are the first tests to use pytest environment. > > > > > - bios bits tests now download the bits binary archives from a > > remote > > > > > repository if they are not found locally. The test skips if > > download > > > > > fails. > > > > > - A new environment variable is introduced that can be passed by > > the tester > > > > > to specify the location of the bits archives locally. test skips > > if the > > > > > bits binaries are not found in that location. > > > > > - if pip install of python module fails for whatever reaoson, the > > test skips. > > > > > - misc code fixes including spell check of the README doc. README > > has been > > > > > updated as well. > > > > > - addition of SPDX license headers to bits test files. > > > > > - update MAINTAINERS to reflect the new pytest test class. > > > > > > > > > > For biosbits repo: > > > > > - added Dockerfile and build script. Made bios bits build on gcc 11. > > > > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile > > > > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > > > > The build script generates the zip archive and tarball used by > > the test. > > > > > > > > So far so good, I think it's ok for a start. It's probably a good idea > > > > to host the source on qemu.org. Peter - any objection to this? > > > > > > Dan was looking at v1 from the point of view of how we handle the > > > guest binary blobs for these tests -- I'd rather defer to him rather > > > than taking the time to get up to speed on the issue myself. > > > > Storing the *source* git repo for biosbits on gitlab.com/qemu-project > > is sensible, as that's what we've done for other 3rd party bits that > > we bundle/depend on git repo access for. > > > Great. Can you or Peter please create a git repo cloned from the official > bios bits repo please? You don’t have to clone mine. Please provide me with > push access so that I can push fixes that I have made in order to build it. I don't have any admin privileges for qemu infra, so can't do this myself. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits 2022-09-06 13:30 ` Daniel P. Berrangé @ 2022-09-09 12:53 ` Ani Sinha 0 siblings, 0 replies; 90+ messages in thread From: Ani Sinha @ 2022-09-09 12:53 UTC (permalink / raw) To: Daniel P. Berrangé, Alex Bennée Cc: Michael S. Tsirkin, Peter Maydell, imammedo, jsnow, pbonzini, qemu-devel, thuth +alexb On Tue, Sep 6, 2022 at 7:00 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Tue, Sep 06, 2022 at 06:58:02PM +0530, Ani Sinha wrote: > > On Tue, Sep 6, 2022 at 18:45 Daniel P. Berrangé <berrange@redhat.com> wrote: > > > > > On Thu, Jul 14, 2022 at 02:24:18PM +0100, Peter Maydell wrote: > > > > On Mon, 11 Jul 2022 at 10:34, Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > On Sun, Jul 10, 2022 at 10:30:03PM +0530, Ani Sinha wrote: > > > > > > Changelog: > > > > > > v2: > > > > > > - a new class of python based tests introduced that is separate > > > from avocado > > > > > > tests or qtests. Can be run by using "make check-pytest". > > > > > > - acpi biosbits tests are the first tests to use pytest environment. > > > > > > - bios bits tests now download the bits binary archives from a > > > remote > > > > > > repository if they are not found locally. The test skips if > > > download > > > > > > fails. > > > > > > - A new environment variable is introduced that can be passed by > > > the tester > > > > > > to specify the location of the bits archives locally. test skips > > > if the > > > > > > bits binaries are not found in that location. > > > > > > - if pip install of python module fails for whatever reaoson, the > > > test skips. > > > > > > - misc code fixes including spell check of the README doc. README > > > has been > > > > > > updated as well. > > > > > > - addition of SPDX license headers to bits test files. > > > > > > - update MAINTAINERS to reflect the new pytest test class. > > > > > > > > > > > > For biosbits repo: > > > > > > - added Dockerfile and build script. Made bios bits build on gcc 11. > > > > > > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile > > > > > > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > > > > > The build script generates the zip archive and tarball used by > > > the test. > > > > > > > > > > So far so good, I think it's ok for a start. It's probably a good idea > > > > > to host the source on qemu.org. Peter - any objection to this? > > > > > > > > Dan was looking at v1 from the point of view of how we handle the > > > > guest binary blobs for these tests -- I'd rather defer to him rather > > > > than taking the time to get up to speed on the issue myself. > > > > > > Storing the *source* git repo for biosbits on gitlab.com/qemu-project > > > is sensible, as that's what we've done for other 3rd party bits that > > > we bundle/depend on git repo access for. > > > > > > Great. Can you or Peter please create a git repo cloned from the official > > bios bits repo please? You don’t have to clone mine. Please provide me with > > push access so that I can push fixes that I have made in order to build it. > > I don't have any admin privileges for qemu infra, so can't do this > myself. > > With regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits 2022-09-06 13:15 ` Daniel P. Berrangé 2022-09-06 13:28 ` Ani Sinha @ 2022-09-06 14:03 ` Michael S. Tsirkin 2022-09-06 14:10 ` Daniel P. Berrangé 1 sibling, 1 reply; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-06 14:03 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Peter Maydell, Ani Sinha, qemu-devel, thuth, jsnow, pbonzini, imammedo On Tue, Sep 06, 2022 at 02:15:41PM +0100, Daniel P. Berrangé wrote: > On Thu, Jul 14, 2022 at 02:24:18PM +0100, Peter Maydell wrote: > > On Mon, 11 Jul 2022 at 10:34, Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > On Sun, Jul 10, 2022 at 10:30:03PM +0530, Ani Sinha wrote: > > > > Changelog: > > > > v2: > > > > - a new class of python based tests introduced that is separate from avocado > > > > tests or qtests. Can be run by using "make check-pytest". > > > > - acpi biosbits tests are the first tests to use pytest environment. > > > > - bios bits tests now download the bits binary archives from a remote > > > > repository if they are not found locally. The test skips if download > > > > fails. > > > > - A new environment variable is introduced that can be passed by the tester > > > > to specify the location of the bits archives locally. test skips if the > > > > bits binaries are not found in that location. > > > > - if pip install of python module fails for whatever reaoson, the test skips. > > > > - misc code fixes including spell check of the README doc. README has been > > > > updated as well. > > > > - addition of SPDX license headers to bits test files. > > > > - update MAINTAINERS to reflect the new pytest test class. > > > > > > > > For biosbits repo: > > > > - added Dockerfile and build script. Made bios bits build on gcc 11. > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > > > The build script generates the zip archive and tarball used by the test. > > > > > > So far so good, I think it's ok for a start. It's probably a good idea > > > to host the source on qemu.org. Peter - any objection to this? > > > > Dan was looking at v1 from the point of view of how we handle the > > guest binary blobs for these tests -- I'd rather defer to him rather > > than taking the time to get up to speed on the issue myself. > > Storing the *source* git repo for biosbits on gitlab.com/qemu-project > is sensible, as that's what we've done for other 3rd party bits that > we bundle/depend on git repo access for. > > The above git repo, however, has extra branches that also store the > binary builds, and I'm not convinced that is a good idea. > > I feel like the source git repo should have a .gitlab-ci.yml file > that builds the binary and publishes it as an CI artifact. This > lets us keep the binary outside of GIT, have a CI job that periodically > refreshes the binary (eg so when the Ubuntu version that Dockerfile > uses goes EOL we can use something else). The test can access the CI > artifact directly. > > > With regards, > Daniel CI is for biosbits developers, bits that everyone uses is production. I don't think our CI is yet robust enough - I think when it comes to pushing bits that all other developers are going to use, it is wise to have them pushed manually. Random CI failures are not unheard of, and if these cause a corrupted binary to be downloaded to all developers, it's going to be bad. > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits 2022-09-06 14:03 ` Michael S. Tsirkin @ 2022-09-06 14:10 ` Daniel P. Berrangé 2022-09-06 14:19 ` Michael S. Tsirkin 0 siblings, 1 reply; 90+ messages in thread From: Daniel P. Berrangé @ 2022-09-06 14:10 UTC (permalink / raw) To: Michael S. Tsirkin Cc: Peter Maydell, Ani Sinha, qemu-devel, thuth, jsnow, pbonzini, imammedo On Tue, Sep 06, 2022 at 10:03:55AM -0400, Michael S. Tsirkin wrote: > On Tue, Sep 06, 2022 at 02:15:41PM +0100, Daniel P. Berrangé wrote: > > On Thu, Jul 14, 2022 at 02:24:18PM +0100, Peter Maydell wrote: > > > On Mon, 11 Jul 2022 at 10:34, Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > On Sun, Jul 10, 2022 at 10:30:03PM +0530, Ani Sinha wrote: > > > > > Changelog: > > > > > v2: > > > > > - a new class of python based tests introduced that is separate from avocado > > > > > tests or qtests. Can be run by using "make check-pytest". > > > > > - acpi biosbits tests are the first tests to use pytest environment. > > > > > - bios bits tests now download the bits binary archives from a remote > > > > > repository if they are not found locally. The test skips if download > > > > > fails. > > > > > - A new environment variable is introduced that can be passed by the tester > > > > > to specify the location of the bits archives locally. test skips if the > > > > > bits binaries are not found in that location. > > > > > - if pip install of python module fails for whatever reaoson, the test skips. > > > > > - misc code fixes including spell check of the README doc. README has been > > > > > updated as well. > > > > > - addition of SPDX license headers to bits test files. > > > > > - update MAINTAINERS to reflect the new pytest test class. > > > > > > > > > > For biosbits repo: > > > > > - added Dockerfile and build script. Made bios bits build on gcc 11. > > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile > > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > > > > The build script generates the zip archive and tarball used by the test. > > > > > > > > So far so good, I think it's ok for a start. It's probably a good idea > > > > to host the source on qemu.org. Peter - any objection to this? > > > > > > Dan was looking at v1 from the point of view of how we handle the > > > guest binary blobs for these tests -- I'd rather defer to him rather > > > than taking the time to get up to speed on the issue myself. > > > > Storing the *source* git repo for biosbits on gitlab.com/qemu-project > > is sensible, as that's what we've done for other 3rd party bits that > > we bundle/depend on git repo access for. > > > > The above git repo, however, has extra branches that also store the > > binary builds, and I'm not convinced that is a good idea. > > > > I feel like the source git repo should have a .gitlab-ci.yml file > > that builds the binary and publishes it as an CI artifact. This > > lets us keep the binary outside of GIT, have a CI job that periodically > > refreshes the binary (eg so when the Ubuntu version that Dockerfile > > uses goes EOL we can use something else). The test can access the CI > > artifact directly. > > > > > > With regards, > > Daniel > > CI is for biosbits developers, bits that everyone uses > is production. I don't think our CI is yet robust enough - > I think when it comes to pushing bits that all > other developers are going to use, it is wise to have them > pushed manually. Random CI failures are not unheard of, > and if these cause a corrupted binary to be downloaded > to all developers, it's going to be bad. The artifacts would only be published if the CI job succeeds. If it fails the artifacts from the previous successful job remain available. Second, we don't need to run the CI continuously. It could be done once a month, or could be triggered manually on demand. Or it could run frequently, but we only publish artifacts from a manual invokation. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits 2022-09-06 14:10 ` Daniel P. Berrangé @ 2022-09-06 14:19 ` Michael S. Tsirkin 0 siblings, 0 replies; 90+ messages in thread From: Michael S. Tsirkin @ 2022-09-06 14:19 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Peter Maydell, Ani Sinha, qemu-devel, thuth, jsnow, pbonzini, imammedo On Tue, Sep 06, 2022 at 03:10:05PM +0100, Daniel P. Berrangé wrote: > On Tue, Sep 06, 2022 at 10:03:55AM -0400, Michael S. Tsirkin wrote: > > On Tue, Sep 06, 2022 at 02:15:41PM +0100, Daniel P. Berrangé wrote: > > > On Thu, Jul 14, 2022 at 02:24:18PM +0100, Peter Maydell wrote: > > > > On Mon, 11 Jul 2022 at 10:34, Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > On Sun, Jul 10, 2022 at 10:30:03PM +0530, Ani Sinha wrote: > > > > > > Changelog: > > > > > > v2: > > > > > > - a new class of python based tests introduced that is separate from avocado > > > > > > tests or qtests. Can be run by using "make check-pytest". > > > > > > - acpi biosbits tests are the first tests to use pytest environment. > > > > > > - bios bits tests now download the bits binary archives from a remote > > > > > > repository if they are not found locally. The test skips if download > > > > > > fails. > > > > > > - A new environment variable is introduced that can be passed by the tester > > > > > > to specify the location of the bits archives locally. test skips if the > > > > > > bits binaries are not found in that location. > > > > > > - if pip install of python module fails for whatever reaoson, the test skips. > > > > > > - misc code fixes including spell check of the README doc. README has been > > > > > > updated as well. > > > > > > - addition of SPDX license headers to bits test files. > > > > > > - update MAINTAINERS to reflect the new pytest test class. > > > > > > > > > > > > For biosbits repo: > > > > > > - added Dockerfile and build script. Made bios bits build on gcc 11. > > > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile > > > > > > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > > > > > > The build script generates the zip archive and tarball used by the test. > > > > > > > > > > So far so good, I think it's ok for a start. It's probably a good idea > > > > > to host the source on qemu.org. Peter - any objection to this? > > > > > > > > Dan was looking at v1 from the point of view of how we handle the > > > > guest binary blobs for these tests -- I'd rather defer to him rather > > > > than taking the time to get up to speed on the issue myself. > > > > > > Storing the *source* git repo for biosbits on gitlab.com/qemu-project > > > is sensible, as that's what we've done for other 3rd party bits that > > > we bundle/depend on git repo access for. > > > > > > The above git repo, however, has extra branches that also store the > > > binary builds, and I'm not convinced that is a good idea. > > > > > > I feel like the source git repo should have a .gitlab-ci.yml file > > > that builds the binary and publishes it as an CI artifact. This > > > lets us keep the binary outside of GIT, have a CI job that periodically > > > refreshes the binary (eg so when the Ubuntu version that Dockerfile > > > uses goes EOL we can use something else). The test can access the CI > > > artifact directly. > > > > > > > > > With regards, > > > Daniel > > > > CI is for biosbits developers, bits that everyone uses > > is production. I don't think our CI is yet robust enough - > > I think when it comes to pushing bits that all > > other developers are going to use, it is wise to have them > > pushed manually. Random CI failures are not unheard of, > > and if these cause a corrupted binary to be downloaded > > to all developers, it's going to be bad. > > The artifacts would only be published if the CI job succeeds. If it fails > the artifacts from the previous successful job remain available. > > Second, we don't need to run the CI continuously. It could be done once > a month, or could be triggered manually on demand. Or it could run > frequently, but we only publish artifacts from a manual invokation. > > > With regards, > Daniel I think publishing should happen manually after maintainer decides it's worth it. In particular, I think we want to keep old qemu source working wrt tests and that means we don't ever erase old binaries, instead adding new ones on demand. IIUC at the moment no other subproject within qemu uses CI for production, we just use it for testing, and it seems somewhat weird to start with a new subproject as opposed to an established one. The new kid and a new maintainer will have problems enough as it is. > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 90+ messages in thread
end of thread, other threads:[~2022-10-09 16:08 UTC | newest] Thread overview: 90+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-07-10 17:00 [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Ani Sinha 2022-07-10 17:00 ` [PATCH v2 01/11] acpi/tests/bits: initial commit of test scripts that are run by biosbits Ani Sinha 2022-07-11 20:38 ` John Snow 2022-07-12 7:17 ` Ani Sinha 2022-07-12 15:23 ` John Snow 2022-07-12 15:38 ` Ani Sinha 2022-07-12 15:42 ` John Snow 2022-07-12 15:46 ` Ani Sinha 2022-07-10 17:00 ` [PATCH v2 02/11] acpi/tests/bits: add SPDX license identifiers for bios bits tests Ani Sinha 2022-07-14 13:56 ` Daniel P. Berrangé 2022-07-14 14:12 ` Ani Sinha 2022-07-10 17:00 ` [PATCH v2 03/11] acpi/tests/bits: disable acpi PSS tests that are failing in biosbits Ani Sinha 2022-07-10 17:00 ` [PATCH v2 04/11] acpi/tests/bits: add smilatency test suite from bits in order to disable it Ani Sinha 2022-07-10 17:00 ` [PATCH v2 05/11] acpi/tests/bits: add SPDX license identifiers for bios bits smilatency tests Ani Sinha 2022-07-10 17:00 ` [PATCH v2 06/11] acpi/tests/bits: disable smilatency test since it does not pass everytime Ani Sinha 2022-07-10 17:00 ` [PATCH v2 07/11] acpi/tests/bits: add python test that exercizes QEMU bios tables using biosbits Ani Sinha 2022-07-11 20:42 ` John Snow 2022-07-12 7:15 ` Ani Sinha 2022-09-06 13:05 ` Daniel P. Berrangé 2022-07-14 14:05 ` Daniel P. Berrangé 2022-07-14 14:19 ` Ani Sinha 2022-07-14 17:49 ` Ani Sinha 2022-07-14 20:43 ` Michael S. Tsirkin 2022-07-14 20:41 ` Michael S. Tsirkin 2022-07-15 4:17 ` Ani Sinha 2022-07-15 6:50 ` Michael S. Tsirkin 2022-07-16 6:36 ` Ani Sinha 2022-07-16 15:30 ` Michael S. Tsirkin 2022-07-25 18:02 ` Ani Sinha 2022-07-27 18:38 ` Ani Sinha 2022-09-16 16:00 ` Ani Sinha 2022-09-17 20:28 ` Michael S. Tsirkin 2022-09-27 8:13 ` Ani Sinha 2022-09-27 8:33 ` Daniel P. Berrangé 2022-09-27 10:07 ` Ani Sinha 2022-09-27 10:18 ` Daniel P. Berrangé 2022-09-27 10:24 ` Ani Sinha 2022-09-27 11:42 ` Ani Sinha 2022-09-27 14:05 ` Ani Sinha 2022-09-27 15:45 ` Daniel P. Berrangé 2022-09-27 21:21 ` Michael S. Tsirkin 2022-09-28 8:31 ` Thomas Huth 2022-09-28 8:35 ` Daniel P. Berrangé 2022-09-28 9:47 ` Michael S. Tsirkin 2022-09-28 9:35 ` Michael S. Tsirkin 2022-09-28 9:39 ` Thomas Huth 2022-09-28 9:56 ` Michael S. Tsirkin 2022-10-06 7:58 ` Ani Sinha 2022-10-06 8:11 ` Ani Sinha 2022-10-06 10:18 ` Ani Sinha 2022-09-27 21:19 ` Michael S. Tsirkin 2022-09-27 21:18 ` Michael S. Tsirkin 2022-09-28 3:08 ` Ani Sinha 2022-09-28 3:58 ` Michael S. Tsirkin 2022-09-28 6:06 ` Ani Sinha 2022-10-06 12:12 ` Michael S. Tsirkin 2022-09-28 6:58 ` Daniel P. Berrangé 2022-09-28 7:15 ` Ani Sinha 2022-09-28 7:26 ` Daniel P. Berrangé 2022-09-28 9:19 ` Michael S. Tsirkin 2022-09-27 21:44 ` Paolo Bonzini 2022-09-27 22:09 ` Michael S. Tsirkin 2022-09-27 23:10 ` Paolo Bonzini 2022-09-28 5:53 ` Michael S. Tsirkin 2022-09-28 7:06 ` Daniel P. Berrangé 2022-09-28 7:43 ` Thomas Huth 2022-10-09 5:21 ` Ani Sinha 2022-10-09 6:30 ` Ani Sinha 2022-10-09 16:06 ` Ani Sinha 2022-07-10 17:00 ` [PATCH v2 08/11] acpi/tests/bits: add biosbits config file for running bios tests Ani Sinha 2022-07-10 17:00 ` [PATCH v2 09/11] acpi/tests/bits: add a README file describing bits pytests Ani Sinha 2022-07-10 17:00 ` [PATCH v2 10/11] pytest: add pytest to the meson build system Ani Sinha 2022-07-11 20:46 ` John Snow 2022-07-12 6:52 ` Ani Sinha 2022-09-06 13:10 ` Daniel P. Berrangé 2022-09-27 21:26 ` Michael S. Tsirkin 2022-09-28 7:32 ` Thomas Huth 2022-09-28 9:23 ` Michael S. Tsirkin 2022-07-10 17:00 ` [PATCH v2 11/11] MAINTAINERS: add myself as the maintainer for acpi biosbits pytests Ani Sinha 2022-07-11 9:33 ` [PATCH v2 00/11] Introduce new acpi/smbios python tests using biosbits Michael S. Tsirkin 2022-07-14 13:24 ` Peter Maydell 2022-09-06 6:26 ` Ani Sinha 2022-09-06 8:19 ` Michael S. Tsirkin 2022-09-06 13:15 ` Daniel P. Berrangé 2022-09-06 13:28 ` Ani Sinha 2022-09-06 13:30 ` Daniel P. Berrangé 2022-09-09 12:53 ` Ani Sinha 2022-09-06 14:03 ` Michael S. Tsirkin 2022-09-06 14:10 ` Daniel P. Berrangé 2022-09-06 14:19 ` Michael S. Tsirkin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).