command line, guest console output missing from avocado log

command line, guest console output missing from avocado log

[Peter Maydell]

I just noticed that the guest console output seems to no longer
be in the avocado log file. Can it be reinstated, please?
The console logs are typically the most useful clue to "why did this
test fail" and without it you're just guessing in the dark...
The details of what QEMU command line avocado is running
also seem to have vanished : that also is among the most
useful items of information to have in the log.

thanks
-- PMM

Re: command line, guest console output missing from avocado log

[Peter Maydell]

On Tue, 9 May 2023 at 19:58, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> I just noticed that the guest console output seems to no longer
> be in the avocado log file. Can it be reinstated, please?
> The console logs are typically the most useful clue to "why did this
> test fail" and without it you're just guessing in the dark...
> The details of what QEMU command line avocado is running
> also seem to have vanished : that also is among the most
> useful items of information to have in the log.

Ping? Where can I find these bits of the log for avocado
test runs now? I tried looking in the individual per-test
directories but they're not there either...

thanks
-- PMM

Re: command line, guest console output missing from avocado log

[Alex Bennée]

Peter Maydell <peter.maydell@linaro.org> writes:

> On Tue, 9 May 2023 at 19:58, Peter Maydell <peter.maydell@linaro.org> wrote:
>>
>> I just noticed that the guest console output seems to no longer
>> be in the avocado log file. Can it be reinstated, please?
>> The console logs are typically the most useful clue to "why did this
>> test fail" and without it you're just guessing in the dark...
>> The details of what QEMU command line avocado is running
>> also seem to have vanished : that also is among the most
>> useful items of information to have in the log.
>
> Ping? Where can I find these bits of the log for avocado
> test runs now? I tried looking in the individual per-test
> directories but they're not there either...

Hmm they have indeed disappeared. According to the docs doing:

  ./tests/venv/bin/avocado --show console:DEBUG run tests/avocado/tuxrun_baselines.py:TuxRunBaselineTest.test_arm64

should be enough, and there is another option --store-logging-stream
which takes the format. However I wasn't able to get this to work.
However moving the logging into the avocado name space with the
following:

--8<---------------cut here---------------start------------->8---
modified   tests/avocado/avocado_qemu/__init__.py
@@ -138,7 +138,7 @@ def _console_interaction(test, success_message, failure_message,
     if vm is None:
         vm = test.vm
     console = vm.console_socket.makefile(mode='rb', encoding='utf-8')
-    console_logger = logging.getLogger('console')
+    console_logger = logging.getLogger('avocado.guest.console')
     while True:
         if send_string:
             vm.console_socket.sendall(send_string.encode())
@@ -407,7 +407,7 @@ class LinuxSSHMixIn:
     """Contains utility methods for interacting with a guest via SSH."""
 
     def ssh_connect(self, username, credential, credential_is_key=True):
-        self.ssh_logger = logging.getLogger('ssh')
+        self.ssh_logger = logging.getLogger('avocado.guest.ssh')
         res = self.vm.command('human-monitor-command',
                               command_line='info usernet')
         port = get_info_usernet_hostfwd_port(res)
--8<---------------cut here---------------end--------------->8---

then makes things work with --show avocado.guest.console:DEBUG and the
job log automatically get the output in the logs.

I note that:

  https://avocado-framework.readthedocs.io/en/101.0/guides/writer/chapters/logging.html

has a big TODO note in it.

Cleber,

Is this the change we should make or should we be able to pick arbitary
name-spaces for logging?

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: command line, guest console output missing from avocado log

[Peter Maydell]

On Fri, 12 May 2023 at 18:10, Alex Bennée <alex.bennee@linaro.org> wrote:
>
>
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> > On Tue, 9 May 2023 at 19:58, Peter Maydell <peter.maydell@linaro.org> wrote:
> >>
> >> I just noticed that the guest console output seems to no longer
> >> be in the avocado log file. Can it be reinstated, please?
> >> The console logs are typically the most useful clue to "why did this
> >> test fail" and without it you're just guessing in the dark...
> >> The details of what QEMU command line avocado is running
> >> also seem to have vanished : that also is among the most
> >> useful items of information to have in the log.
> >
> > Ping? Where can I find these bits of the log for avocado
> > test runs now? I tried looking in the individual per-test
> > directories but they're not there either...
>
> Hmm they have indeed disappeared. According to the docs doing:
>
>   ./tests/venv/bin/avocado --show console:DEBUG run tests/avocado/tuxrun_baselines.py:TuxRunBaselineTest.test_arm64
>
> should be enough, and there is another option --store-logging-stream
> which takes the format. However I wasn't able to get this to work.
> However moving the logging into the avocado name space with the
> following:
>
> --8<---------------cut here---------------start------------->8---
> modified   tests/avocado/avocado_qemu/__init__.py
> @@ -138,7 +138,7 @@ def _console_interaction(test, success_message, failure_message,
>      if vm is None:
>          vm = test.vm
>      console = vm.console_socket.makefile(mode='rb', encoding='utf-8')
> -    console_logger = logging.getLogger('console')
> +    console_logger = logging.getLogger('avocado.guest.console')
>      while True:
>          if send_string:
>              vm.console_socket.sendall(send_string.encode())
> @@ -407,7 +407,7 @@ class LinuxSSHMixIn:
>      """Contains utility methods for interacting with a guest via SSH."""
>
>      def ssh_connect(self, username, credential, credential_is_key=True):
> -        self.ssh_logger = logging.getLogger('ssh')
> +        self.ssh_logger = logging.getLogger('avocado.guest.ssh')
>          res = self.vm.command('human-monitor-command',
>                                command_line='info usernet')
>          port = get_info_usernet_hostfwd_port(res)
> --8<---------------cut here---------------end--------------->8---
>
> then makes things work with --show avocado.guest.console:DEBUG and the
> job log automatically get the output in the logs.
>
> I note that:
>
>   https://avocado-framework.readthedocs.io/en/101.0/guides/writer/chapters/logging.html
>
> has a big TODO note in it.
>
> Cleber,
>
> Is this the change we should make or should we be able to pick arbitary
> name-spaces for logging?

Ping! Can we either fix this or roll back to the old Avocado
version, please ? I've just run into "want to find out why
the test failed, log has 0 information" problem again.

thanks
-- PMM

Re: command line, guest console output missing from avocado log

[Richard Henderson]

On 5/18/23 05:08, Peter Maydell wrote:
> On Fri, 12 May 2023 at 18:10, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>>
>> Peter Maydell <peter.maydell@linaro.org> writes:
>>
>>> On Tue, 9 May 2023 at 19:58, Peter Maydell <peter.maydell@linaro.org> wrote:
>>>>
>>>> I just noticed that the guest console output seems to no longer
>>>> be in the avocado log file. Can it be reinstated, please?
>>>> The console logs are typically the most useful clue to "why did this
>>>> test fail" and without it you're just guessing in the dark...
>>>> The details of what QEMU command line avocado is running
>>>> also seem to have vanished : that also is among the most
>>>> useful items of information to have in the log.
>>>
>>> Ping? Where can I find these bits of the log for avocado
>>> test runs now? I tried looking in the individual per-test
>>> directories but they're not there either...
>>
>> Hmm they have indeed disappeared. According to the docs doing:
>>
>>    ./tests/venv/bin/avocado --show console:DEBUG run tests/avocado/tuxrun_baselines.py:TuxRunBaselineTest.test_arm64
>>
>> should be enough, and there is another option --store-logging-stream
>> which takes the format. However I wasn't able to get this to work.
>> However moving the logging into the avocado name space with the
>> following:
>>
>> --8<---------------cut here---------------start------------->8---
>> modified   tests/avocado/avocado_qemu/__init__.py
>> @@ -138,7 +138,7 @@ def _console_interaction(test, success_message, failure_message,
>>       if vm is None:
>>           vm = test.vm
>>       console = vm.console_socket.makefile(mode='rb', encoding='utf-8')
>> -    console_logger = logging.getLogger('console')
>> +    console_logger = logging.getLogger('avocado.guest.console')
>>       while True:
>>           if send_string:
>>               vm.console_socket.sendall(send_string.encode())
>> @@ -407,7 +407,7 @@ class LinuxSSHMixIn:
>>       """Contains utility methods for interacting with a guest via SSH."""
>>
>>       def ssh_connect(self, username, credential, credential_is_key=True):
>> -        self.ssh_logger = logging.getLogger('ssh')
>> +        self.ssh_logger = logging.getLogger('avocado.guest.ssh')
>>           res = self.vm.command('human-monitor-command',
>>                                 command_line='info usernet')
>>           port = get_info_usernet_hostfwd_port(res)
>> --8<---------------cut here---------------end--------------->8---
>>
>> then makes things work with --show avocado.guest.console:DEBUG and the
>> job log automatically get the output in the logs.
>>
>> I note that:
>>
>>    https://avocado-framework.readthedocs.io/en/101.0/guides/writer/chapters/logging.html
>>
>> has a big TODO note in it.
>>
>> Cleber,
>>
>> Is this the change we should make or should we be able to pick arbitary
>> name-spaces for logging?
> 
> Ping! Can we either fix this or roll back to the old Avocado
> version, please ? I've just run into "want to find out why
> the test failed, log has 0 information" problem again.

Seconded.


r~