Re: [PATCH for-10.0] tests/functional: Convert the intel_iommu avocado test

[Thomas Huth <thuth@redhat.com>]

On 09/12/2024 17.37, Daniel P. Berrangé wrote:
> On Mon, Dec 09, 2024 at 05:31:37PM +0100, Philippe Mathieu-Daudé wrote:
>> On 9/12/24 10:09, Daniel P. Berrangé wrote:
>>> On Mon, Dec 09, 2024 at 09:26:21AM +0100, Thomas Huth wrote:
>>>> On 09/12/2024 09.12, Eric Auger wrote:
>>>>> Hi Thomas,
>>>>>
>>>>> On 12/6/24 19:17, Thomas Huth wrote:
>>>>>> Convert the intel_iommu test to the new functional framework.
>>>>>> This test needs some changes since we neither support the old 'LinuxTest'
>>>>>> class in the functional framework yet, nor a way to use SSH for running
>>>>>> commands in the guest. So we now directly download a Fedora kernel and
>>>>>> initrd and set up the serial console for executing the commands and for
>>>>>> looking for the results.
>>>>>>
>>>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>>>> ---
>>>>>>     MAINTAINERS                                   |   1 +
>>>>>>     tests/functional/meson.build                  |   1 +
>>>>>>     .../test_intel_iommu.py}                      | 119 ++++++++----------
>>>>>>     3 files changed, 51 insertions(+), 70 deletions(-)
>>>>>>     rename tests/{avocado/intel_iommu.py => functional/test_intel_iommu.py} (41%)
>>>>>>     mode change 100644 => 100755
>>
>>
>>>>>> -        self.launch_and_wait()
>>>>>> -        self.ssh_command('cat /proc/cmdline')
>>>>>> -        self.ssh_command('dmesg | grep -e DMAR -e IOMMU')
>>>>>> -        self.ssh_command('find /sys/kernel/iommu_groups/ -type l')
>>>>>> -        self.ssh_command('dnf -y install numactl-devel')
>>>>> I understand you cannot use ssh yet but the bulk of the test was the
>>>>> execution of the dnf install meaning we lose the main substance of it
>>>>> through the conversion.
>>>>
>>>> Ah, I see, so this was exercising the virtio-net device with the IOMMU ...
>>>> and I already wondered why there was this "dnf install" at the end without
>>>> doing anything with  the numactl-devel package ... (a comment would have
>>>> been helpful here)
>>>
>>> FYI, I find 'dnf instal' to be a *highly* undesirable thing todo in
>>> our test functional. Its performance is highly non-deterministic
>>> depending on what mirror you happen to get sent to, such that it could
>>> easily push us over the timeouts. It is also susceptible to periodic
>>> broken mirrors, and instability around time of Fefdora EOL. I can't
>>> remember if it was this test case, or a different one, but I've seen
>>> problems before in avocado with 'dnf install'.
>>>
>>> If we want to test working networking, then can we arrange for something
>>> more simple & targetted to run, with better worst case performance.
>>
>> Could we use 2 virtio-net interfaces inter-connected and stress with
>> https://linux.die.net/man/1/ab ?

The problem is that this tool is not available in the guest image by 
default, so we'd need to dial with "dnf install" during runtime again, 
relying on the availability of a third party server... so that would not be 
any better than the "dnf install" that was in this test here before, I think.

> Do we actually need to stress this ? IMHO for a functional tests we just
> need to prove that the device is working at a fairly basic level.
> "wget example.com"

My current plan is to start a HTTP server in the guest ("python3 -m 
http.server 8080" is your friend), and then to download some megabytes from 
the guest to the host to see whether we can transfer them successfully.

  Thomas