[PATCH v2] selftests: ksft: Fix finished() helper exit code on skipped tests

[PATCH v2] selftests: ksft: Fix finished() helper exit code on skipped tests

[Laura Nao]

The Python finished() helper currently exits with KSFT_FAIL when there
are only passed and skipped tests. Fix the logic to exit with KSFT_PASS
instead, making it consistent with its C and bash counterparts
(ksft_finished() and ktap_finished() respectively).

Reviewed-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
Fixes: dacf1d7a78bf ("kselftest: Add test to verify probe of devices from discoverable buses")
Signed-off-by: Laura Nao <laura.nao@collabora.com>
---
This is a revised version of the patch initially submitted as "[PATCH]
selftests: ksft: Track skipped tests when finishing the test suite":
https://lore.kernel.org/all/20240722154319.619944-1-laura.nao@collabora.com/

Depends on "[PATCH v2 2/3] kselftest: Move ksft helper module to common
directory":
https://lore.kernel.org/all/20240705-dev-err-log-selftest-v2-2-163b9cd7b3c1@collabora.com/
(picked through the usb tree, queued for 6.11-rc1)

Changes in v2:
- Reworded the commit title and message to more accurately describe the
incorrect behavior of the finished() helper addressed by the patch.
---
 tools/testing/selftests/kselftest/ksft.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/kselftest/ksft.py b/tools/testing/selftests/kselftest/ksft.py
index cd89fb2bc10e..bf215790a89d 100644
--- a/tools/testing/selftests/kselftest/ksft.py
+++ b/tools/testing/selftests/kselftest/ksft.py
@@ -70,7 +70,7 @@ def test_result(condition, description=""):
 
 
 def finished():
-    if ksft_cnt["pass"] == ksft_num_tests:
+    if ksft_cnt["pass"] + ksft_cnt["skip"] == ksft_num_tests:
         exit_code = KSFT_PASS
     else:
         exit_code = KSFT_FAIL
-- 
2.30.2

Re: [PATCH v2] selftests: ksft: Fix finished() helper exit code on skipped tests

[Muhammad Usama Anjum]

On 7/30/24 3:29 PM, Laura Nao wrote:
> The Python finished() helper currently exits with KSFT_FAIL when there
> are only passed and skipped tests. Fix the logic to exit with KSFT_PASS
> instead, making it consistent with its C and bash counterparts
> (ksft_finished() and ktap_finished() respectively).
> 
> Reviewed-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
> Fixes: dacf1d7a78bf ("kselftest: Add test to verify probe of devices from discoverable buses")
> Signed-off-by: Laura Nao <laura.nao@collabora.com>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>

> ---
> This is a revised version of the patch initially submitted as "[PATCH]
> selftests: ksft: Track skipped tests when finishing the test suite":
> https://lore.kernel.org/all/20240722154319.619944-1-laura.nao@collabora.com/
> 
> Depends on "[PATCH v2 2/3] kselftest: Move ksft helper module to common
> directory":
> https://lore.kernel.org/all/20240705-dev-err-log-selftest-v2-2-163b9cd7b3c1@collabora.com/
> (picked through the usb tree, queued for 6.11-rc1)
> 
> Changes in v2:
> - Reworded the commit title and message to more accurately describe the
> incorrect behavior of the finished() helper addressed by the patch.
> ---
>  tools/testing/selftests/kselftest/ksft.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/kselftest/ksft.py b/tools/testing/selftests/kselftest/ksft.py
> index cd89fb2bc10e..bf215790a89d 100644
> --- a/tools/testing/selftests/kselftest/ksft.py
> +++ b/tools/testing/selftests/kselftest/ksft.py
> @@ -70,7 +70,7 @@ def test_result(condition, description=""):
>  
>  
>  def finished():
> -    if ksft_cnt["pass"] == ksft_num_tests:
> +    if ksft_cnt["pass"] + ksft_cnt["skip"] == ksft_num_tests:
LGTM

>          exit_code = KSFT_PASS
>      else:
>          exit_code = KSFT_FAIL

-- 
BR,
Muhammad Usama Anjum

Re: [PATCH v2] selftests: ksft: Fix finished() helper exit code on skipped tests

[Shuah Khan]

On 7/31/24 07:05, Muhammad Usama Anjum wrote:
> On 7/30/24 3:29 PM, Laura Nao wrote:
>> The Python finished() helper currently exits with KSFT_FAIL when there
>> are only passed and skipped tests. Fix the logic to exit with KSFT_PASS
>> instead, making it consistent with its C and bash counterparts
>> (ksft_finished() and ktap_finished() respectively).
>>
>> Reviewed-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
>> Fixes: dacf1d7a78bf ("kselftest: Add test to verify probe of devices from discoverable buses")
>> Signed-off-by: Laura Nao <laura.nao@collabora.com>
> Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> 
>> ---
>> This is a revised version of the patch initially submitted as "[PATCH]
>> selftests: ksft: Track skipped tests when finishing the test suite":
>> https://lore.kernel.org/all/20240722154319.619944-1-laura.nao@collabora.com/
>>
>> Depends on "[PATCH v2 2/3] kselftest: Move ksft helper module to common
>> directory":
>> https://lore.kernel.org/all/20240705-dev-err-log-selftest-v2-2-163b9cd7b3c1@collabora.com/
>> (picked through the usb tree, queued for 6.11-rc1)
>>
>> Changes in v2:
>> - Reworded the commit title and message to more accurately describe the
>> incorrect behavior of the finished() helper addressed by the patch.

Thank you - Applied to linux-kselftest fixes for Linux 6.11-rc2

thanks,
-- Shuah