Re: [Qemu-devel] [PATCH 0/6] QTests: use Python to run complex tests

[Stefan Hajnoczi <stefanha@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 2336 bytes --]

On Wed, Dec 13, 2017 at 06:35:51PM -0300, Philippe Mathieu-Daudé wrote:
> Hi,
> 
> With this series we can now write tests using Python rather than C.
> For complex tests this can reduce the test development time, we can focus on
> the test purposes instead of his implementation details.
> 
> - 1,2: we already have Python classes to run Block tests, move all the
>   non Block-related methods to qtest.py,
> - 3: default TEST_DIR to TMPDIR,
> - 4: add a method to restrict tests to a list of supported machines,
> - 5: since the Block tests are output sensitive, do not mess with their
>   current tuned iotests::main(unittest), add a more generic one in qtest.py,
> - 6: finally modify the tests Makefile to run C/Python tests with the same
>   rule.

Python tests are great for functional tests of qemu-system-* that
interact via the command-line and QMP monitor.  From this perspective I
think the series is good.

> to have a better idea, here is a snippet from the next series:
> 
>     class TestSdCardSpecV2(qtest.QMPTestCase):
>         [...]
>         def test_power_on_spec_v2(self):
>             self.bus.do_cmd(GO_IDLE_STATE)
>             [...]
>             # verify Card ID
>             data = self.bus.do_cmd(ALL_SEND_CID)
>             oid, pnm, psn = struct.unpack(">x2s5sxLxxx", data)
>             self.assertEqual(oid, "XY") # QEMU default
>             self.assertEqual(pnm, "QEMU!") # QEMU default
>             self.assertEqual(psn, 0xdeadbeef) # QEMU default

Device qtests are better done in C than Python.  Python is not good at
binary I/O and porting this to Python 3 will be extra work later (Python
2 is set for End-of-Life in 2020, see https://pythonclock.org/).

More importantly, we already have libqos in C with a guest memory
allocator, PCI, and virtio support.  Fragmenting the small amount effort
that goes into device testing will delay libqos reaching critical mass.
Critical mass is where libqos provides all the infrastructure you need
to set up a device and focus on your actual test instead of machine,
bus, or device initialization.  Starting a Python device testing effort
will just lead to duplication and 2 underdeveloped device testing
frameworks.

Is there a specific reason why adding SD Card support to libqos is not
possible in C?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]