[PATCH] rust: Oe-selftest error log on console when it fails.

[PATCH] rust: Oe-selftest error log on console when it fails.

[Yash.Shinde]

From: Yash Shinde <Yash.Shinde@windriver.com>

The rust oe-selftest output error log doesn't show any information on console when it fails.
The following changes emit stderr logs in terminal along with re-directing stdout and stderr to "summary.txt" file.

Changes made::
- cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s > summary.txt 2>&1;" % (builddir, testargs, targetsys)
+ cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 2> >(tee summary.txt >&2);" % (builddir, testargs, targetsys)

> summary.txt:                  Redirects the standard output (stdout) of the command to a file 'summary.txt'
2> >(tee summary.txt >&2):      Redirects stderr & stdout to summary.txt & writes stderr on terminal

The overall effect is that both stdout and stderr are captured in the summary.txt file, while stderr still being displayed in the terminal.

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
---
 meta/lib/oeqa/selftest/cases/rust.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/selftest/cases/rust.py b/meta/lib/oeqa/selftest/cases/rust.py
index 164ad11ecd..07f1b5706c 100644
--- a/meta/lib/oeqa/selftest/cases/rust.py
+++ b/meta/lib/oeqa/selftest/cases/rust.py
@@ -213,7 +213,7 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase):
             cmd = cmd + " export RUST_TARGET_PATH=%s/rust-targets;" % rustlibpath
             # Trigger testing.
             cmd = cmd + " export TEST_DEVICE_ADDR=\"%s:12345\";" % qemu.ip
-            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s > summary.txt 2>&1;" % (builddir, testargs, targetsys)
+            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 2> >(tee summary.txt >&2);" % (builddir, testargs, targetsys)
             runCmd(cmd)
             end_time = time.time()
 
-- 
2.43.0

Re: [PATCH] rust: Oe-selftest error log on console when it fails.

[Randy MacLeod]

[-- Attachment #1: Type: text/plain, Size: 4171 bytes --]

On 2024-02-13 8:04 a.m., Yash.Shinde@windriver.com wrote:
> From: Yash Shinde<Yash.Shinde@windriver.com>
>
> The rust oe-selftest output error log doesn't show any information on console when it fails.
> The following changes emit stderr logs in terminal along with re-directing stdout and stderr to "summary.txt" file.
>
> Changes made::
> - cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s > summary.txt 2>&1;" % (builddir, testargs, targetsys)
> + cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 2> >(tee summary.txt >&2);" % (builddir, testargs, targetsys)
>
>> summary.txt:                  Redirects the standard output (stdout) of the command to a file 'summary.txt'
> 2> >(tee summary.txt >&2):      Redirects stderr & stdout to summary.txt & writes stderr on terminal
>
> The overall effect is that both stdout and stderr are captured in the summary.txt file, while stderr still being displayed in the terminal.
>
> Signed-off-by: Yash Shinde<Yash.Shinde@windriver.com>
> ---
>   meta/lib/oeqa/selftest/cases/rust.py | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/lib/oeqa/selftest/cases/rust.py b/meta/lib/oeqa/selftest/cases/rust.py
> index 164ad11ecd..07f1b5706c 100644
> --- a/meta/lib/oeqa/selftest/cases/rust.py
> +++ b/meta/lib/oeqa/selftest/cases/rust.py
> @@ -213,7 +213,7 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase):
>               cmd = cmd + " export RUST_TARGET_PATH=%s/rust-targets;" % rustlibpath
>               # Trigger testing.
>               cmd = cmd + " export TEST_DEVICE_ADDR=\"%s:12345\";" % qemu.ip
> -            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s > summary.txt 2>&1;" % (builddir, testargs, targetsys)
> +            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 2> >(tee summary.txt >&2);" % (builddir, testargs, targetsys)
>               runCmd(cmd)
>               end_time = time.time()
>   

Hi Yash,

We talked about this change and decided that it's just too odd a shell 
pipeline to merge!

Also, it depends on the bash shell as you can tell by pasting:

    #!/bin/sh
    python3 src/bootstrap/bootstrap.py test %s --target %s  > 
summary.txt 2> >(tee summary.txt >&2);

into https://www.shellcheck.net/ . You'll see the following log and error:

$ shellcheck myscript

Line 2:
python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 2> 
 >(tee summary.txt >&2);
                                                           ^-- SC2094 
<https://www.shellcheck.net/wiki/SC2094> (info): Make sure not to read 
and write the same file in the same pipeline.
>>                                                                       ^--  SC3001 <https://www.shellcheck.net/wiki/SC3001> (warning): In POSIX 
sh, process substitution is undefined.
>>                                                                             ^--  SC2094 <https://www.shellcheck.net/wiki/SC2094> (info): Make sure not 
to read and write the same file in the same pipeline.


The YP goal / requirement is to avoid bash dependencies since user may 
be using dash or some other shell for /bin/sh.


I think that the command is being run locally (not on target), and we 
are already in a python context so
it seems that we would have a simpler solution if we handle the IO 
redirection from python.

Perhaps Richard or Joshua can recommend some code that you can reference 
when making this change.

Did you have time to look at the log file handling in runCmd()

https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/utils/commands.py#n168

Please do some analysis and reply here every day or so as  you learn 
things or have questions.


Thanks,

../Randy



-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2: Type: text/html, Size: 6907 bytes --]

Re: [OE-core] [PATCH] rust: Oe-selftest error log on console when it fails.

[Alexander Kanavin]

It's not difficult to see that runCmd() captures and returns both
stdout and stderr in an object, so rather than redirecting things to a
file, the code should actually assign the return value to a variable
and do useful things with it.

Alex

On Tue, 13 Feb 2024 at 19:12, Randy MacLeod via lists.openembedded.org
<randy.macleod=windriver.com@lists.openembedded.org> wrote:
>
> On 2024-02-13 8:04 a.m., Yash.Shinde@windriver.com wrote:
>
> From: Yash Shinde <Yash.Shinde@windriver.com>
>
> The rust oe-selftest output error log doesn't show any information on console when it fails.
> The following changes emit stderr logs in terminal along with re-directing stdout and stderr to "summary.txt" file.
>
> Changes made::
> - cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s > summary.txt 2>&1;" % (builddir, testargs, targetsys)
> + cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 2> >(tee summary.txt >&2);" % (builddir, testargs, targetsys)
>
> summary.txt:                  Redirects the standard output (stdout) of the command to a file 'summary.txt'
>
> 2> >(tee summary.txt >&2):      Redirects stderr & stdout to summary.txt & writes stderr on terminal
>
> The overall effect is that both stdout and stderr are captured in the summary.txt file, while stderr still being displayed in the terminal.
>
> Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
> ---
>  meta/lib/oeqa/selftest/cases/rust.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/lib/oeqa/selftest/cases/rust.py b/meta/lib/oeqa/selftest/cases/rust.py
> index 164ad11ecd..07f1b5706c 100644
> --- a/meta/lib/oeqa/selftest/cases/rust.py
> +++ b/meta/lib/oeqa/selftest/cases/rust.py
> @@ -213,7 +213,7 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase):
>              cmd = cmd + " export RUST_TARGET_PATH=%s/rust-targets;" % rustlibpath
>              # Trigger testing.
>              cmd = cmd + " export TEST_DEVICE_ADDR=\"%s:12345\";" % qemu.ip
> -            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s > summary.txt 2>&1;" % (builddir, testargs, targetsys)
> +            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 2> >(tee summary.txt >&2);" % (builddir, testargs, targetsys)
>              runCmd(cmd)
>              end_time = time.time()
>
>
> Hi Yash,
>
> We talked about this change and decided that it's just too odd a shell pipeline to merge!
>
> Also, it depends on the bash shell as you can tell by pasting:
>
>    #!/bin/sh
>    python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 2> >(tee summary.txt >&2);
>
> into https://www.shellcheck.net/ . You'll see the following log and error:
>
> $ shellcheck myscript
>
> Line 2:
> python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 2> >(tee summary.txt >&2);
>                                                           ^-- SC2094 (info): Make sure not to read and write the same file in the same pipeline.
> >>                                                                       ^-- SC3001 (warning): In POSIX sh, process substitution is undefined.
> >>                                                                             ^-- SC2094 (info): Make sure not to read and write the same file in the same pipeline.
>
>
> The YP goal / requirement is to avoid bash dependencies since user may be using dash or some other shell for /bin/sh.
>
>
> I think that the command is being run locally (not on target), and we are already in a python context so
> it seems that we would have a simpler solution if we handle the IO redirection from python.
>
> Perhaps Richard or Joshua can recommend some code that you can reference when making this change.
>
> Did you have time to look at the log file handling in runCmd()
>
> https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/utils/commands.py#n168
>
> Please do some analysis and reply here every day or so as  you learn things or have questions.
>
>
> Thanks,
>
> ../Randy
>
>
>
> --
> # Randy MacLeod
> # Wind River Linux
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#195424): https://lists.openembedded.org/g/openembedded-core/message/195424
> Mute This Topic: https://lists.openembedded.org/mt/104331501/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [PATCH] rust: Oe-selftest error log on console when it fails.

[Yash Shinde]

[-- Attachment #1: Type: text/plain, Size: 5879 bytes --]


On 13-02-2024 23:42, Randy MacLeod wrote:
> On 2024-02-13 8:04 a.m., Yash.Shinde@windriver.com wrote:
>> From: Yash Shinde<Yash.Shinde@windriver.com>
>>
>> The rust oe-selftest output error log doesn't show any information on console when it fails.
>> The following changes emit stderr logs in terminal along with re-directing stdout and stderr to "summary.txt" file.
>>
>> Changes made::
>> - cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s > summary.txt 2>&1;" % (builddir, testargs, targetsys)
>> + cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 2> >(tee summary.txt >&2);" % (builddir, testargs, targetsys)
>>
>>> summary.txt:                  Redirects the standard output (stdout) of the command to a file 'summary.txt'
>> 2> >(tee summary.txt >&2):      Redirects stderr & stdout to summary.txt & writes stderr on terminal
>>
>> The overall effect is that both stdout and stderr are captured in the summary.txt file, while stderr still being displayed in the terminal.
>>
>> Signed-off-by: Yash Shinde<Yash.Shinde@windriver.com>
>> ---
>>   meta/lib/oeqa/selftest/cases/rust.py | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/lib/oeqa/selftest/cases/rust.py b/meta/lib/oeqa/selftest/cases/rust.py
>> index 164ad11ecd..07f1b5706c 100644
>> --- a/meta/lib/oeqa/selftest/cases/rust.py
>> +++ b/meta/lib/oeqa/selftest/cases/rust.py
>> @@ -213,7 +213,7 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase):
>>               cmd = cmd + " export RUST_TARGET_PATH=%s/rust-targets;" % rustlibpath
>>               # Trigger testing.
>>               cmd = cmd + " export TEST_DEVICE_ADDR=\"%s:12345\";" % qemu.ip
>> -            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s > summary.txt 2>&1;" % (builddir, testargs, targetsys)
>> +            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 2> >(tee summary.txt >&2);" % (builddir, testargs, targetsys)
>>               runCmd(cmd)
>>               end_time = time.time()
>>   
>
> Hi Yash,
>
> We talked about this change and decided that it's just too odd a shell 
> pipeline to merge!
>
> Also, it depends on the bash shell as you can tell by pasting:
>
>    #!/bin/sh
>    python3 src/bootstrap/bootstrap.py test %s --target %s  > 
> summary.txt 2> >(tee summary.txt >&2);
>
> into https://www.shellcheck.net/ . You'll see the following log and error:
>
> $ shellcheck myscript
>
> Line 2:
> python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 
> 2> >(tee summary.txt >&2);
>                                                           ^-- SC2094 
> <https://www.shellcheck.net/wiki/SC2094> (info): Make sure not to read 
> and write the same file in the same pipeline.
> >>                                                                       ^-- SC3001 <https://www.shellcheck.net/wiki/SC3001> (warning): In POSIX 
> sh, process substitution is undefined.
> >>                                                                             ^-- SC2094 <https://www.shellcheck.net/wiki/SC2094> (info): Make sure not 
> to read and write the same file in the same pipeline.
>
>
> The YP goal / requirement is to avoid bash dependencies since user may 
> be using dash or some other shell for /bin/sh.
>
>
> I think that the command is being run locally (not on target), and we 
> are already in a python context so
> it seems that we would have a simpler solution if we handle the IO 
> redirection from python.
>
> Perhaps Richard or Joshua can recommend some code that you can 
> reference when making this change.
>
> Did you have time to look at the log file handling in runCmd()
>
> https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/utils/commands.py#n168
>
> Please do some analysis and reply here every day or so as  you learn 
> things or have questions.
>
>
The runCmd() returns the 'Result' object containing information about 
the command execution. It has the following attributes:
    result.command = command
    result.status = cmd.status
    result.output = cmd.output
    result.error = cmd.error
    result.pid = cmd.process.pid
https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/utils/commands.py#n198 
<https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/utils/commands.py#n198>

I tried to capture the return object value (stderr i.e result.error) and 
print it to the terminal, but that didn't work as expected.
Even I tried to print some debug statements in rust.py file and it also 
didn't show up in the terminal or in the summary.txt file.
I assume there's something in oe-selftest framework that doesn't print 
statements directly.

Also, I see there's a "output_log 
<https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/utils/commands.py#n198:~:text=%3D0%2C-,output_log%3DNone,-%2C%20**options>" 
parameter in the runCmd function parameters, which I understand is used 
to redirect stdout of the
command being executed. Currently, I am checking with different values 
by referring to other oe-selftests and their corresponding behavior with it.

I am checking with some functions and procedures from unittest and 
subprocess.Popen frameworks to get the error logs:

https://docs.python.org/3/library/unittest.html
https://docs.python.org/3/library/subprocess.html#subprocess.Popen

Regards,
Yash

> Thanks,
>
> ../Randy
>
>
>
> -- 
> # Randy MacLeod
> # Wind River Linux

[-- Attachment #2: Type: text/html, Size: 9702 bytes --]

Re: [PATCH] rust: Oe-selftest error log on console when it fails.

[Richard Purdie]

On Wed, 2024-02-21 at 13:30 +0530, Yash Shinde wrote:
> 
>  The runCmd() returns the 'Result' object containing information about the command execution. It has the following attributes:
>     result.command = command
>     result.status = cmd.status
>     result.output = cmd.output
>     result.error = cmd.error
>     result.pid = cmd.process.pid
>  https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/utils/commands.py#n198
>   
>  I tried to capture the return object value (stderr i.e result.error) and print it to the terminal, but that didn't work as expected.
>  Even I tried to print some debug statements in rust.py file and it also didn't show up in the terminal or in the summary.txt file.
>  I assume there's something in oe-selftest framework that doesn't print statements directly.
>   
>  Also, I see there's a "output_log" parameter in the runCmd function parameters, which I understand is used to redirect stdout of the
>  command being executed. Currently, I am checking with different values by referring to other oe-selftests and their corresponding behavior with it.
>  
> 
> 
> 
>  I am checking with some functions and procedures from unittest and subprocess.Popen frameworks to get the error logs:
>  
> 
> 
> 
>  https://docs.python.org/3/library/unittest.html
>  https://docs.python.org/3/library/subprocess.html#subprocess.Popen
>  


I had a look at this and tried an experiment locally. This seemed to
work:

diff --git a/meta/lib/oeqa/selftest/cases/rust.py b/meta/lib/oeqa/selftest/cases/rust.py
index 120be6454fa..ad14189c6df 100644
--- a/meta/lib/oeqa/selftest/cases/rust.py
+++ b/meta/lib/oeqa/selftest/cases/rust.py
@@ -216,13 +216,16 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase):
             cmd = cmd + " export RUST_TARGET_PATH=%s/rust-targets;" % rustlibpath
             # Trigger testing.
             cmd = cmd + " export TEST_DEVICE_ADDR=\"%s:12345\";" % qemu.ip
-            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s > summary.txt 2>&1;" % (builddir, testargs, targetsys)
-            runCmd(cmd)
+            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s" % (builddir, testargs, targetsys)
+            retval = runCmd(cmd)
             end_time = time.time()
 
+            resultlog = rustlibpath + "/results-log.txt"
+            with open(resultlog, "w") as f:
+                f.write(retval.output)
+
             ptestsuite = "rust"
-            self.ptest_section(ptestsuite, duration = int(end_time - start_time), logfile = builddir + "/summary.txt")
-            filename = builddir + "/summary.txt"
-            test_results = parse_results(filename)
+            self.ptest_section(ptestsuite, duration = int(end_time - start_time), logfile=resultlog)
+            test_results = parse_results(resultlog)
             for test in test_results:
                 self.ptest_result(ptestsuite, test, test_results[test])


I'm fairly sure we could improve it further but this does start that
process.

Cheers,

Richard