Re: [PATCH] tests/acceptance: Count multiple Tux logos displayed on framebuffer

["Philippe Mathieu-Daudé" <f4bug@amsat.org>]

On 2/1/20 9:58 PM, Aleksandar Markovic wrote:
> 21:48 Sub, 01.02.2020. Philippe Mathieu-Daudé <f4bug@amsat.org
> <mailto:f4bug@amsat.org>> је написао/ла:
>>
>> Add a test that verifies that each core properly displays the Tux
>> logo on the framebuffer device.
>>
> 
> Excellent test!
> 
> Glad that Leon blog post was the inspiration. I'll email him about this,
> if I find his address, one of my colegues might have it.
> 
> I don't have dev setup at hand, is 8 cores limit? I think it is. If not,
> perhaps you can push the limit with 16 tuxes.

I tried, QEMU instantiates the 16 cores but the kernel only initialize
8. I don't see any restriction on the kernel side. Maybe some QEMU
device is slashing a value.

16 is the upper limit of the Malta machine, however this doesn't seem to
make much sense, the limit should be on the CPU used (or better, the
daughter card used).

1438 static void mips_malta_machine_init(MachineClass *mc)
1439 {
1440     mc->desc = "MIPS Malta Core LV";
1441     mc->init = mips_malta_init;
1442     mc->block_default_type = IF_IDE;
1443     mc->max_cpus = 16;

> 
> THANK YOU, PHILIPPE!

You are welcome :)

> 
> Aleksandar
> 
>> We simply follow the OpenCV "Template Matching with Multiple Objects"
>> tutorial, replacing Lionel Messi by Tux:
>> https://docs.opencv.org/4.2.0/d4/dc6/tutorial_py_template_matching.html
>>
>> When OpenCV and NumPy are installed, this test can be run using:
>>
>>   $ avocado --show=app,framebuffer \
>>     run -t cpu:i6400 \
>>     tests/acceptance/machine_mips_malta.py
>>   JOB ID     : 54f3d8efd8674f289b8aa01a87f5d70c5814544c
>>   JOB LOG    : avocado/job-results/job-2020-02-01T20.52-54f3d8e/job.log
>>    (1/3)
> tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_1core:
>>   framebuffer: found Tux at position (x, y) = (0, 0)
>>   PASS (3.37 s)
>>    (2/3)
> tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_7cores:
>>   framebuffer: found Tux at position (x, y) = (0, 0)
>>   framebuffer: found Tux at position (x, y) = (88, 0)
>>   framebuffer: found Tux at position (x, y) = (176, 0)
>>   framebuffer: found Tux at position (x, y) = (264, 0)
>>   framebuffer: found Tux at position (x, y) = (352, 0)
>>   framebuffer: found Tux at position (x, y) = (440, 0)
>>   framebuffer: found Tux at position (x, y) = (528, 0)
>>   PASS (5.80 s)
>>    (3/3)
> tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_8cores:
>>   framebuffer: found Tux at position (x, y) = (0, 0)
>>   framebuffer: found Tux at position (x, y) = (88, 0)
>>   framebuffer: found Tux at position (x, y) = (176, 0)
>>   framebuffer: found Tux at position (x, y) = (264, 0)
>>   framebuffer: found Tux at position (x, y) = (352, 0)
>>   framebuffer: found Tux at position (x, y) = (440, 0)
>>   framebuffer: found Tux at position (x, y) = (528, 0)
>>   framebuffer: found Tux at position (x, y) = (616, 0)
>>   PASS (6.67 s)
>>   RESULTS    : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT
> 0 | CANCEL 0
>>   JOB TIME   : 16.79 s
>>
>> If the AVOCADO_CV2_SCREENDUMP_PNG_PATH environment variable is set, the
>> test will save the screenshot with matched squares to it.
>>
>> Test inspired by the following post:
>>
> https://www.mips.com/blog/how-to-run-smp-linux-in-qemu-on-a-mips64-release-6-cpu/
>> Kernel built with the following Docker file:
>>
> https://github.com/philmd/qemu-testing-blob/blob/malta_i6400/mips/malta/mips64el/Dockerfile
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org
> <mailto:f4bug@amsat.org>>
>> ---
>> The resulting match can be visualised at https://pasteboard.co/ISIj6ZA.png
>> Generated using:
>>
>>   $ AVOCADO_CV2_SCREENDUMP_PNG_PATH=/tmp/screendump_matched.png \
>>     avocado --show=app run -t cpu:i6400 \
>>     tests/acceptance/machine_mips_malta.py
>>   JOB ID     : 54f3d8efd8674f289b8aa01a87f5d70c5814544c
>>   JOB LOG    : avocado/job-results/job-2020-02-01T20.52-54f3d8e/job.log
>>    (1/3)
> tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_1core:
> PASS (3.37 s)
>>    (2/3)
> tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_7cores:
> PASS (5.80 s)
>>    (3/3)
> tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_8cores:
> PASS (6.67 s)
>>   RESULTS    : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT
> 0 | CANCEL 0
>>   JOB TIME   : 16.79 s
>> ---
>>  MAINTAINERS                            |   1 +
>>  tests/acceptance/machine_mips_malta.py | 116 +++++++++++++++++++++++++
>>  2 files changed, 117 insertions(+)
>>  create mode 100644 tests/acceptance/machine_mips_malta.py
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 4ceb1ad882..65a3dfb564 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -983,6 +983,7 @@ F: hw/mips/mips_malta.c
>>  F: hw/mips/gt64xxx_pci.c
>>  F: include/hw/southbridge/piix.h
>>  F: tests/acceptance/linux_ssh_mips_malta.py
>> +F: tests/acceptance/machine_mips_malta.py
>>
>>  Mipssim
>>  M: Aleksandar Markovic <amarkovic@wavecomp.com
> <mailto:amarkovic@wavecomp.com>>
>> diff --git a/tests/acceptance/machine_mips_malta.py
> b/tests/acceptance/machine_mips_malta.py
>> new file mode 100644
>> index 0000000000..2764143d15
>> --- /dev/null
>> +++ b/tests/acceptance/machine_mips_malta.py
>> @@ -0,0 +1,116 @@
>> +# Functional tests for the MIPS Malta board
>> +#
>> +# Copyright (c) Philippe Mathieu-Daudé <f4bug@amsat.org
> <mailto:f4bug@amsat.org>>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or
> later.
>> +# See the COPYING file in the top-level directory.
>> +#
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +
>> +import os
>> +import gzip
>> +import logging
>> +
>> +from avocado import skipUnless
>> +from avocado_qemu import Test
>> +from avocado_qemu import wait_for_console_pattern
>> +from avocado.utils import archive
>> +
>> +
>> +NUMPY_AVAILABLE = True
>> +try:
>> +    import numpy as np
>> +except ImportError:
>> +    NUMPY_AVAILABLE = False
>> +
>> +CV2_AVAILABLE = True
>> +try:
>> +    import cv2
>> +except ImportError:
>> +    CV2_AVAILABLE = False
>> +
>> +
>> +@skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
>> +@skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
>> +class MaltaMachineFramebuffer(Test):
>> +
>> +    timeout = 30
>> +
>> +    KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
>> +
>> +    def do_test_i6400_framebuffer_logo(self, cpu_cores_count):
>> +        """
>> +        Boot Linux kernel and check Tux logo is displayed on the
> framebuffer.
>> +        """
>> +        screendump_path = os.path.join(self.workdir, 'screendump.pbm')
>> +
>> +        kernel_url = ('https://github.com/philmd/qemu-testing-blob/raw/'
>> +                      'a5966ca4b5/mips/malta/mips64el/'
>> +                      'vmlinux-4.7.0-rc1.I6400.gz')
>> +        kernel_hash = '096f50c377ec5072e6a366943324622c312045f6'
>> +        kernel_path_gz = self.fetch_asset(kernel_url,
> asset_hash=kernel_hash)
>> +        kernel_path = self.workdir + "vmlinux"
>> +        archive.gzip_uncompress(kernel_path_gz, kernel_path)
>> +
>> +        tuxlogo_url = ('https://github.com/torvalds/linux/raw/v2.6.12/'
>> +                       'drivers/video/logo/logo_linux_vga16.ppm')
>> +        tuxlogo_hash = '3991c2ddbd1ddaecda7601f8aafbcf5b02dc86af'
>> +        tuxlogo_path = self.fetch_asset(tuxlogo_url,
> asset_hash=tuxlogo_hash)
>> +
>> +        self.vm.set_console()
>> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
>> +                               'clocksource=GIC console=tty0
> console=ttyS0')
>> +        self.vm.add_args('-kernel', kernel_path,
>> +                         '-cpu', 'I6400',
>> +                         '-smp', '%u' % cpu_cores_count,
>> +                         '-vga', 'std',
>> +                         '-append', kernel_command_line)
>> +        self.vm.launch()
>> +        framebuffer_ready = 'Console: switching to colour frame
> buffer device'
>> +        wait_for_console_pattern(self, framebuffer_ready,
>> +                                 failure_message='Kernel panic - not
> syncing')
>> +        self.vm.command('human-monitor-command', command_line='stop')
>> +        self.vm.command('human-monitor-command',
>> +                        command_line='screendump %s' % screendump_path)
>> +        logger = logging.getLogger('framebuffer')
>> +
>> +        match_threshold = 0.95
>> +        screendump_bgr = cv2.imread(screendump_path, cv2.IMREAD_COLOR)
>> +        tuxlogo_bgr = cv2.imread(tuxlogo_path, cv2.IMREAD_COLOR)
>> +        result = cv2.matchTemplate(screendump_bgr, tuxlogo_bgr,
>> +                                   cv2.TM_CCOEFF_NORMED)
>> +        loc = np.where(result >= match_threshold)
>> +        tuxlogo_count = 0
>> +        h, w = tuxlogo_bgr.shape[:2]
>> +        debug_png = os.getenv('AVOCADO_CV2_SCREENDUMP_PNG_PATH')
>> +        for tuxlogo_count, pt in enumerate(zip(*loc[::-1]), start=1):
>> +            logger.debug('found Tux at position (x, y) = %s', pt)
>> +            cv2.rectangle(screendump_bgr, pt,
>> +                          (pt[0] + w, pt[1] + h), (0, 0, 255), 2)
>> +        if debug_png:
>> +            cv2.imwrite(debug_png, screendump_bgr)
>> +        self.assertGreaterEqual(tuxlogo_count, cpu_cores_count)
>> +
>> +    def test_mips_malta_i6400_framebuffer_logo_1core(self):
>> +        """
>> +        :avocado: tags=arch:mips64el
>> +        :avocado: tags=machine:malta
>> +        :avocado: tags=cpu:i6400
>> +        """
>> +        self.do_test_i6400_framebuffer_logo(1)
>> +
>> +    def test_mips_malta_i6400_framebuffer_logo_7cores(self):
>> +        """
>> +        :avocado: tags=arch:mips64el
>> +        :avocado: tags=machine:malta
>> +        :avocado: tags=cpu:i6400
>> +        """
>> +        self.do_test_i6400_framebuffer_logo(7)
>> +
>> +    def test_mips_malta_i6400_framebuffer_logo_8cores(self):
>> +        """
>> +        :avocado: tags=arch:mips64el
>> +        :avocado: tags=machine:malta
>> +        :avocado: tags=cpu:i6400
>> +        """
>> +        self.do_test_i6400_framebuffer_logo(8)
>> --
>> 2.21.1
>>
>>
>