* [Bug Report] Multiple S390x KUNIT clk failures
@ 2024-05-14 1:08 Nico Pache
2024-05-14 3:38 ` Stephen Boyd
0 siblings, 1 reply; 9+ messages in thread
From: Nico Pache @ 2024-05-14 1:08 UTC (permalink / raw)
To: KUnit Development, linux-clk, sboyd
Cc: Shuah Khan, Audra Mitchell, Donald Zickus
Hi,
We are seeing a number of the CLK kunit tests failing on S390x. This
has been occurring for some time now (as early as v6.1). We run these
tests as modules (without the UML wrappers), and have not seen these
failures on any other architectures.
The failing tests are:
clk-gate-hiword-test
------------------------------
# clk_gate_test_hiword_enable: EXPECTATION FAILED at
drivers/clk/clk-gate_test.c:322
Expected enable_val == (__u32)__builtin_bswap32((__u32)((
__u32)(__le32)(ctx->fake_reg))), but
enable_val == 33554944 (0x2000200)
(__u32)__builtin_bswap32((__u32)((
__u32)(__le32)(ctx->fake_reg))) == 0 (0x0)
not ok 1 clk_gate_test_hiword_enable
# clk_gate_test_hiword_disable: ASSERTION FAILED at
drivers/clk/clk-gate_test.c:339
Expected enable_val == (__u32)__builtin_bswap32((__u32)((
__u32)(__le32)(ctx->fake_reg))), but
enable_val == 33554944 (0x2000200)
(__u32)__builtin_bswap32((__u32)((
__u32)(__le32)(ctx->fake_reg))) == 0 (0x0)
not ok 2 clk_gate_test_hiword_disable
clk-gate-invert-test
------------------------------
# clk_gate_test_invert_enable: EXPECTATION FAILED at
drivers/clk/clk-gate_test.c:249
Expected enable_val == (__u32)__builtin_bswap32((__u32)((
__u32)(__le32)(ctx->fake_reg))), but
enable_val == 0 (0x0)
(__u32)__builtin_bswap32((__u32)((
__u32)(__le32)(ctx->fake_reg))) == 32768 (0x8000)
# clk_gate_test_invert_enable: EXPECTATION FAILED at
drivers/clk/clk-gate_test.c:250
Expected clk_hw_is_enabled(hw) to be true, but is false
not ok 1 clk_gate_test_invert_enable
# clk_gate_test_invert_disable: ASSERTION FAILED at
drivers/clk/clk-gate_test.c:266
Expected enable_val == (__u32)__builtin_bswap32((__u32)((
__u32)(__le32)(ctx->fake_reg))), but
enable_val == 0 (0x0)
(__u32)__builtin_bswap32((__u32)((
__u32)(__le32)(ctx->fake_reg))) == 32768 (0x8000)
not ok 2 clk_gate_test_invert_disable
clk-gate-is_enabled-test
------------------------------
ok 1 clk_gate_test_is_enabled
# clk_gate_test_is_disabled: ASSERTION FAILED at
drivers/clk/clk-gate_test.c:409
Expected clk_hw_is_enabled(hw) to be false, but is true
not ok 2 clk_gate_test_is_disabled
# clk_gate_test_is_enabled_inverted: ASSERTION FAILED at
drivers/clk/clk-gate_test.c:423
Expected hw is not error, but is: -17
not ok 3 clk_gate_test_is_enabled_inverted
# clk_gate_test_is_disabled_inverted: ASSERTION FAILED at
drivers/clk/clk-gate_test.c:438
Expected hw is not error, but is: -17
not ok 4 clk_gate_test_is_disabled_inverted
clk-gate-test
------------------------------
ok 1 clk_gate_test_parent_rate
# clk_gate_test_enable: EXPECTATION FAILED at
drivers/clk/clk-gate_test.c:169
Expected enable_val == (__u32)__builtin_bswap32((__u32)((
__u32)(__le32)(ctx->fake_reg))), but
enable_val == 32 (0x20)
(__u32)__builtin_bswap32((__u32)((
__u32)(__le32)(ctx->fake_reg))) == 0 (0x0)
not ok 2 clk_gate_test_enable
# clk_gate_test_disable: ASSERTION FAILED at drivers/clk/clk-gate_test.c:186
Expected enable_val == (__u32)__builtin_bswap32((__u32)((
__u32)(__le32)(ctx->fake_reg))), but
enable_val == 32 (0x20)
(__u32)__builtin_bswap32((__u32)((
__u32)(__le32)(ctx->fake_reg))) == 0 (0x0)
not ok 3 clk_gate_test_disable
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Bug Report] Multiple S390x KUNIT clk failures
2024-05-14 1:08 [Bug Report] Multiple S390x KUNIT clk failures Nico Pache
@ 2024-05-14 3:38 ` Stephen Boyd
2024-05-14 7:14 ` Nico Pache
0 siblings, 1 reply; 9+ messages in thread
From: Stephen Boyd @ 2024-05-14 3:38 UTC (permalink / raw)
To: KUnit Development, Nico Pache, linux-clk
Cc: Shuah Khan, Audra Mitchell, Donald Zickus
Quoting Nico Pache (2024-05-13 18:08:40)
> Hi,
>
> We are seeing a number of the CLK kunit tests failing on S390x. This
> has been occurring for some time now (as early as v6.1). We run these
> tests as modules (without the UML wrappers), and have not seen these
> failures on any other architectures.
Do you have commit 75357829cc8e ("clk: Fix clk gate kunit test on
big-endian CPUs"). What is the kernel version?
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Bug Report] Multiple S390x KUNIT clk failures
2024-05-14 3:38 ` Stephen Boyd
@ 2024-05-14 7:14 ` Nico Pache
2024-05-14 22:04 ` Stephen Boyd
0 siblings, 1 reply; 9+ messages in thread
From: Nico Pache @ 2024-05-14 7:14 UTC (permalink / raw)
To: Stephen Boyd
Cc: KUnit Development, linux-clk, Shuah Khan, Audra Mitchell,
Donald Zickus
Hi Stephen,
Yes, we have that commit. This is failing on fedora-ark, so it's
constantly updated to match upstream, and the earliest recorded
instance of it failing in our environment is v6.1.
Cheers,
-- Nico
On Mon, May 13, 2024 at 9:44 PM Stephen Boyd <sboyd@kernel.org> wrote:
>
> Quoting Nico Pache (2024-05-13 18:08:40)
> > Hi,
> >
> > We are seeing a number of the CLK kunit tests failing on S390x. This
> > has been occurring for some time now (as early as v6.1). We run these
> > tests as modules (without the UML wrappers), and have not seen these
> > failures on any other architectures.
>
> Do you have commit 75357829cc8e ("clk: Fix clk gate kunit test on
> big-endian CPUs"). What is the kernel version?
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Bug Report] Multiple S390x KUNIT clk failures
2024-05-14 7:14 ` Nico Pache
@ 2024-05-14 22:04 ` Stephen Boyd
[not found] ` <CAK18DXZyEHZ=1TC52kQQ89gscFLph0e_4zB_bt=DTwR-A=0UPA@mail.gmail.com>
0 siblings, 1 reply; 9+ messages in thread
From: Stephen Boyd @ 2024-05-14 22:04 UTC (permalink / raw)
To: Nico Pache
Cc: KUnit Development, linux-clk, Shuah Khan, Audra Mitchell,
Donald Zickus
Quoting Nico Pache (2024-05-14 00:14:24)
> Hi Stephen,
>
> Yes, we have that commit. This is failing on fedora-ark, so it's
> constantly updated to match upstream, and the earliest recorded
> instance of it failing in our environment is v6.1.
>
Please don't top post. I'm unable to run s390 kunit tests in qemu. They
seem to crash before finishing.
./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit --arch=s390 --cross_compile=/path/to/s390-linux-
[14:55:10] Starting KUnit Kernel (1/1)...
[14:55:10] ============================================================
Running tests with:
$ qemu-system-s390x -nodefaults -m 1024 -kernel .kunit/arch/s390/boot/bzImage -append 'kunit.enable=1 console=ttyS0 kunit_shutdown=reboot' -no-reboot -nographic -serial stdio -machine s390-ccw-virtio -cpu qemu
[14:55:11] ============================= =============================
[14:55:11] ================= example_init (1 subtest) =================
[14:55:11] [PASSED] example_init_test
[14:55:11] ================== [PASSED] example_init ===================
[14:55:11] ============================= =============================
[14:55:11] ============= kunit_executor_test (8 subtests) =============
[14:55:11] [PASSED] parse_filter_test
[14:55:11] [PASSED] filter_suites_test
[14:55:11] [PASSED] filter_suites_test_glob_test
[14:55:11] [PASSED] filter_suites_to_empty_test
[14:55:11] [PASSED] parse_filter_attr_test
[14:55:11] [PASSED] filter_attr_test
[14:55:11] [PASSED] filter_attr_empty_test
[14:55:11] [PASSED] filter_attr_skip_test
[14:55:11] [ERROR] Test: kunit_executor_test: Expected test number 1 but found 2
[14:55:11] =============== [PASSED] kunit_executor_test ===============
[14:55:11] ============================= =============================
[14:55:11] ============ kunit-try-catch-test (2 subtests) =============
[14:55:11] [PASSED] kunit_test_try_catch_successful_try_no_catch
[14:55:11] [PASSED] kunit_test_try_catch_unsuccessful_try_does_catch
[14:55:11] [ERROR] Test: kunit-try-catch-test: Expected test number 1 but found 3
[14:55:11] ============== [PASSED] kunit-try-catch-test ===============
[14:55:11] ============================= =============================
[14:55:11] ============ kunit-resource-test (12 subtests) =============
[14:55:11] [PASSED] kunit_resource_test_init_resources
[14:55:11] [PASSED] kunit_resource_test_alloc_resource
[14:55:11] [PASSED] kunit_resource_test_destroy_resource
[14:55:11] [PASSED] kunit_resource_test_remove_resource
[14:55:11] [PASSED] kunit_resource_test_cleanup_resources
[14:55:11] [PASSED] kunit_resource_test_proper_free_ordering
[14:55:11] [PASSED] kunit_resource_test_static
[14:55:11] [PASSED] kunit_resource_test_named
[14:55:11] [PASSED] kunit_resource_test_action
[14:55:11] [PASSED] kunit_resource_test_remove_action
[14:55:11] [PASSED] kunit_resource_test_release_action
[14:55:11] [PASSED] kunit_resource_test_action_ordering
[14:55:11] [ERROR] Test: kunit-resource-test: Expected test number 1 but found 4
[14:55:11] =============== [PASSED] kunit-resource-test ===============
[14:55:11] ============================= =============================
[14:55:11] =============== kunit-log-test (2 subtests) ================
[14:55:11] [PASSED] kunit_log_test
[14:55:11] [SKIPPED] kunit_log_newline_test
[14:55:11] [ERROR] Test: kunit-log-test: Expected test number 1 but found 5
[14:55:11] ================= [PASSED] kunit-log-test ==================
[14:55:11] ============================= =============================
[14:55:11] ================ kunit_status (2 subtests) =================
[14:55:11] [PASSED] kunit_status_set_failure_test
[14:55:11] [PASSED] kunit_status_mark_skipped_test
[14:55:11] [ERROR] Test: kunit_status: Expected test number 1 but found 6
[14:55:11] ================== [PASSED] kunit_status ===================
[14:55:11] ============================= =============================
[14:55:11] ================ kunit_current (2 subtests) ================
[14:55:11] [PASSED] kunit_current_test
[14:55:11] [PASSED] kunit_current_fail_test
[14:55:11] [ERROR] Test: kunit_current: Expected test number 1 but found 7
[14:55:11] ================== [PASSED] kunit_current ==================
[14:55:11] ============================= =============================
[14:55:11] ================ kunit_device (3 subtests) =================
[14:55:11] [PASSED] kunit_device_test
[14:55:11] [PASSED] kunit_device_cleanup_test
[14:55:11] [PASSED] kunit_device_driver_test
[14:55:11] [ERROR] Test: kunit_device: Expected test number 1 but found 8
[14:55:11] ================== [PASSED] kunit_device ===================
[14:55:11] ============================= =============================
[14:55:11] ============= string-stream-test (12 subtests) =============
[14:55:11] [PASSED] string_stream_managed_init_test
[14:55:11] [PASSED] string_stream_unmanaged_init_test
[14:55:11] [PASSED] string_stream_managed_free_test
[14:55:11] [PASSED] string_stream_resource_free_test
[14:55:11] [PASSED] string_stream_line_add_test
[14:55:11] [PASSED] string_stream_variable_length_line_test
[14:55:11] [PASSED] string_stream_append_test
[14:55:11] [PASSED] string_stream_append_auto_newline_test
[14:55:11] [PASSED] string_stream_append_empty_string_test
[14:55:11] [PASSED] string_stream_no_auto_newline_test
[14:55:11] [PASSED] string_stream_auto_newline_test
[14:55:11] [PASSED] string_stream_performance_test
[14:55:11] [ERROR] Test: string-stream-test: Expected test number 1 but found 9
[14:55:11] =============== [PASSED] string-stream-test ================
[14:55:11] ============================= =============================
[14:55:11] =================== example (9 subtests) ===================
[14:55:11] [PASSED] example_simple_test
[14:55:11] [SKIPPED] example_skip_test
[14:55:11] [ERROR] Test: example: missing expected subtest!
[14:55:11]
[14:55:11] # example_mark_skipped_test: initializing
[14:55:11]
[14:55:11] # example_mark_skipped_test: You should see a line below.
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: example: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: example: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: example: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: example: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: example: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: example: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: example: missing subtest result line!
[14:55:11]
[14:55:11] # module: kunit_example_test
[14:55:11] ==================== [CRASHED] example =====================
[14:55:11]
[14:55:11] # example: initializing suite
[14:55:11]
[14:55:11] ======================== [CRASHED] ========================
[14:55:11]
[14:55:11] ======================== [CRASHED] ========================
[14:55:11]
[14:55:11] ======================== [CRASHED] ========================
[14:55:11]
[14:55:11] ======================== [CRASHED] ========================
[14:55:11]
[14:55:11] ======================== [CRASHED] ========================
[14:55:11]
[14:55:11] ======================== [CRASHED] ========================
[14:55:11]
[14:55:11] ======================== [CRASHED] ========================
[14:55:11]
[14:55:11] ======================== [CRASHED] ========================
[14:55:11]
[14:55:11] ======================== [CRASHED] ========================
[14:55:11]
[14:55:11] ======================== [CRASHED] ========================
[14:55:11] [ERROR] Test: main: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: main: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: main: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: main: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: main: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: main: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: main: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: main: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] [ERROR] Test: main: missing expected subtest!
[14:55:11] [CRASHED]
[14:55:11] ============================================================
[14:55:11] Testing complete. Ran 62 tests: passed: 44, crashed: 16, skipped: 2, errors: 25
The kernel seems to have crashed; you can decode the stack traces with:
$ scripts/decode_stacktrace.sh .kunit/vmlinux .kunit < .kunit/test.log | tee .kunit/decoded.log | ./tools/testing/kunit/kunit.py parse
[14:55:11] Elapsed time: 16.240s total, 1.476s configuring, 14.294s building, 0.469s running
I wonder if something with my local environment is causing troubles.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Bug Report] Multiple S390x KUNIT clk failures
[not found] ` <CAK18DXZyEHZ=1TC52kQQ89gscFLph0e_4zB_bt=DTwR-A=0UPA@mail.gmail.com>
@ 2024-05-28 18:49 ` Donald Zickus
2024-05-28 20:49 ` Audra Mitchell
0 siblings, 1 reply; 9+ messages in thread
From: Donald Zickus @ 2024-05-28 18:49 UTC (permalink / raw)
To: Stephen Boyd
Cc: Nico Pache, KUnit Development, linux-clk, Shuah Khan,
Audra Mitchell
(trying again without the html part that gmail likes to add. Apologies)
On Tue, May 28, 2024 at 2:45 PM Donald Zickus <dzickus@redhat.com> wrote:
>
> Hi Stephen,
>
> On Tue, May 14, 2024 at 6:04 PM Stephen Boyd <sboyd@kernel.org> wrote:
>>
>> Quoting Nico Pache (2024-05-14 00:14:24)
>> > Hi Stephen,
>> >
>> > Yes, we have that commit. This is failing on fedora-ark, so it's
>> > constantly updated to match upstream, and the earliest recorded
>> > instance of it failing in our environment is v6.1.
>> >
>>
>> Please don't top post. I'm unable to run s390 kunit tests in qemu. They
>> seem to crash before finishing.
>
>
> I am working with Nico on moving this forward. What can we do to help resolve this?
>
> Cheers,
> Don
>
>>
>>
>> ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit --arch=s390 --cross_compile=/path/to/s390-linux-
>>
>> [14:55:10] Starting KUnit Kernel (1/1)...
>> [14:55:10] ============================================================
>> Running tests with:
>> $ qemu-system-s390x -nodefaults -m 1024 -kernel .kunit/arch/s390/boot/bzImage -append 'kunit.enable=1 console=ttyS0 kunit_shutdown=reboot' -no-reboot -nographic -serial stdio -machine s390-ccw-virtio -cpu qemu
>> [14:55:11] ============================= =============================
>> [14:55:11] ================= example_init (1 subtest) =================
>> [14:55:11] [PASSED] example_init_test
>> [14:55:11] ================== [PASSED] example_init ===================
>> [14:55:11] ============================= =============================
>> [14:55:11] ============= kunit_executor_test (8 subtests) =============
>> [14:55:11] [PASSED] parse_filter_test
>> [14:55:11] [PASSED] filter_suites_test
>> [14:55:11] [PASSED] filter_suites_test_glob_test
>> [14:55:11] [PASSED] filter_suites_to_empty_test
>> [14:55:11] [PASSED] parse_filter_attr_test
>> [14:55:11] [PASSED] filter_attr_test
>> [14:55:11] [PASSED] filter_attr_empty_test
>> [14:55:11] [PASSED] filter_attr_skip_test
>> [14:55:11] [ERROR] Test: kunit_executor_test: Expected test number 1 but found 2
>> [14:55:11] =============== [PASSED] kunit_executor_test ===============
>> [14:55:11] ============================= =============================
>> [14:55:11] ============ kunit-try-catch-test (2 subtests) =============
>> [14:55:11] [PASSED] kunit_test_try_catch_successful_try_no_catch
>> [14:55:11] [PASSED] kunit_test_try_catch_unsuccessful_try_does_catch
>> [14:55:11] [ERROR] Test: kunit-try-catch-test: Expected test number 1 but found 3
>> [14:55:11] ============== [PASSED] kunit-try-catch-test ===============
>> [14:55:11] ============================= =============================
>> [14:55:11] ============ kunit-resource-test (12 subtests) =============
>> [14:55:11] [PASSED] kunit_resource_test_init_resources
>> [14:55:11] [PASSED] kunit_resource_test_alloc_resource
>> [14:55:11] [PASSED] kunit_resource_test_destroy_resource
>> [14:55:11] [PASSED] kunit_resource_test_remove_resource
>> [14:55:11] [PASSED] kunit_resource_test_cleanup_resources
>> [14:55:11] [PASSED] kunit_resource_test_proper_free_ordering
>> [14:55:11] [PASSED] kunit_resource_test_static
>> [14:55:11] [PASSED] kunit_resource_test_named
>> [14:55:11] [PASSED] kunit_resource_test_action
>> [14:55:11] [PASSED] kunit_resource_test_remove_action
>> [14:55:11] [PASSED] kunit_resource_test_release_action
>> [14:55:11] [PASSED] kunit_resource_test_action_ordering
>> [14:55:11] [ERROR] Test: kunit-resource-test: Expected test number 1 but found 4
>> [14:55:11] =============== [PASSED] kunit-resource-test ===============
>> [14:55:11] ============================= =============================
>> [14:55:11] =============== kunit-log-test (2 subtests) ================
>> [14:55:11] [PASSED] kunit_log_test
>> [14:55:11] [SKIPPED] kunit_log_newline_test
>> [14:55:11] [ERROR] Test: kunit-log-test: Expected test number 1 but found 5
>> [14:55:11] ================= [PASSED] kunit-log-test ==================
>> [14:55:11] ============================= =============================
>> [14:55:11] ================ kunit_status (2 subtests) =================
>> [14:55:11] [PASSED] kunit_status_set_failure_test
>> [14:55:11] [PASSED] kunit_status_mark_skipped_test
>> [14:55:11] [ERROR] Test: kunit_status: Expected test number 1 but found 6
>> [14:55:11] ================== [PASSED] kunit_status ===================
>> [14:55:11] ============================= =============================
>> [14:55:11] ================ kunit_current (2 subtests) ================
>> [14:55:11] [PASSED] kunit_current_test
>> [14:55:11] [PASSED] kunit_current_fail_test
>> [14:55:11] [ERROR] Test: kunit_current: Expected test number 1 but found 7
>> [14:55:11] ================== [PASSED] kunit_current ==================
>> [14:55:11] ============================= =============================
>> [14:55:11] ================ kunit_device (3 subtests) =================
>> [14:55:11] [PASSED] kunit_device_test
>> [14:55:11] [PASSED] kunit_device_cleanup_test
>> [14:55:11] [PASSED] kunit_device_driver_test
>> [14:55:11] [ERROR] Test: kunit_device: Expected test number 1 but found 8
>> [14:55:11] ================== [PASSED] kunit_device ===================
>> [14:55:11] ============================= =============================
>> [14:55:11] ============= string-stream-test (12 subtests) =============
>> [14:55:11] [PASSED] string_stream_managed_init_test
>> [14:55:11] [PASSED] string_stream_unmanaged_init_test
>> [14:55:11] [PASSED] string_stream_managed_free_test
>> [14:55:11] [PASSED] string_stream_resource_free_test
>> [14:55:11] [PASSED] string_stream_line_add_test
>> [14:55:11] [PASSED] string_stream_variable_length_line_test
>> [14:55:11] [PASSED] string_stream_append_test
>> [14:55:11] [PASSED] string_stream_append_auto_newline_test
>> [14:55:11] [PASSED] string_stream_append_empty_string_test
>> [14:55:11] [PASSED] string_stream_no_auto_newline_test
>> [14:55:11] [PASSED] string_stream_auto_newline_test
>> [14:55:11] [PASSED] string_stream_performance_test
>> [14:55:11] [ERROR] Test: string-stream-test: Expected test number 1 but found 9
>> [14:55:11] =============== [PASSED] string-stream-test ================
>> [14:55:11] ============================= =============================
>> [14:55:11] =================== example (9 subtests) ===================
>> [14:55:11] [PASSED] example_simple_test
>> [14:55:11] [SKIPPED] example_skip_test
>> [14:55:11] [ERROR] Test: example: missing expected subtest!
>> [14:55:11]
>> [14:55:11] # example_mark_skipped_test: initializing
>> [14:55:11]
>> [14:55:11] # example_mark_skipped_test: You should see a line below.
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: example: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: example: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: example: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: example: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: example: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: example: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: example: missing subtest result line!
>> [14:55:11]
>> [14:55:11] # module: kunit_example_test
>> [14:55:11] ==================== [CRASHED] example =====================
>> [14:55:11]
>> [14:55:11] # example: initializing suite
>> [14:55:11]
>> [14:55:11] ======================== [CRASHED] ========================
>> [14:55:11]
>> [14:55:11] ======================== [CRASHED] ========================
>> [14:55:11]
>> [14:55:11] ======================== [CRASHED] ========================
>> [14:55:11]
>> [14:55:11] ======================== [CRASHED] ========================
>> [14:55:11]
>> [14:55:11] ======================== [CRASHED] ========================
>> [14:55:11]
>> [14:55:11] ======================== [CRASHED] ========================
>> [14:55:11]
>> [14:55:11] ======================== [CRASHED] ========================
>> [14:55:11]
>> [14:55:11] ======================== [CRASHED] ========================
>> [14:55:11]
>> [14:55:11] ======================== [CRASHED] ========================
>> [14:55:11]
>> [14:55:11] ======================== [CRASHED] ========================
>> [14:55:11] [ERROR] Test: main: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: main: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: main: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: main: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: main: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: main: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: main: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: main: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] [ERROR] Test: main: missing expected subtest!
>> [14:55:11] [CRASHED]
>> [14:55:11] ============================================================
>> [14:55:11] Testing complete. Ran 62 tests: passed: 44, crashed: 16, skipped: 2, errors: 25
>> The kernel seems to have crashed; you can decode the stack traces with:
>> $ scripts/decode_stacktrace.sh .kunit/vmlinux .kunit < .kunit/test.log | tee .kunit/decoded.log | ./tools/testing/kunit/kunit.py parse
>> [14:55:11] Elapsed time: 16.240s total, 1.476s configuring, 14.294s building, 0.469s running
>>
>> I wonder if something with my local environment is causing troubles.
>>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Bug Report] Multiple S390x KUNIT clk failures
2024-05-28 18:49 ` Donald Zickus
@ 2024-05-28 20:49 ` Audra Mitchell
2024-05-28 22:53 ` msalter
0 siblings, 1 reply; 9+ messages in thread
From: Audra Mitchell @ 2024-05-28 20:49 UTC (permalink / raw)
To: Donald Zickus
Cc: Stephen Boyd, Nico Pache, KUnit Development, linux-clk,
Shuah Khan, Audra Mitchell, Mark Salter
On Tue, May 28, 2024 at 02:49:56PM -0400, Donald Zickus wrote:
> (trying again without the html part that gmail likes to add. Apologies)
>
> On Tue, May 28, 2024 at 2:45 PM Donald Zickus <dzickus@redhat.com> wrote:
> >
> > Hi Stephen,
> >
> > On Tue, May 14, 2024 at 6:04 PM Stephen Boyd <sboyd@kernel.org> wrote:
> >>
> >> Quoting Nico Pache (2024-05-14 00:14:24)
> >> > Hi Stephen,
> >> >
> >> > Yes, we have that commit. This is failing on fedora-ark, so it's
> >> > constantly updated to match upstream, and the earliest recorded
> >> > instance of it failing in our environment is v6.1.
> >> >
> >>
> >> Please don't top post. I'm unable to run s390 kunit tests in qemu. They
> >> seem to crash before finishing.
> >
> >
> > I am working with Nico on moving this forward. What can we do to help resolve this?
Hello all,
I spent some time last week or so working on debugging these failures and I
believe I have found the problem. I reached out to Malk Salter for advice on
the best way to move forward with a fix on Friday the 17th, but he was on
PTO for the last week. I was waiting for his reply before I replied to this
thread.
Also as a side note, I also ran into the same issue as Stephen with running
the kunit tests on s390 QEMU. I did not pursue resolving that issue and
instead just compiled the test as a module.
For clarity, this is what I sent to Mark and were I believe the failure is
occurring:
The tests create a pretend clk-gate and use a "fake_reg" to emulate
the expected behavior of the clk_gate->reg. I added some debug
statements to the driver and noticed that the reg changes after
initialization to -1. I also noticed that we call this to read the
data in the clk-gate->reg:
static inline u32 clk_gate_readl(struct clk_gate *gate)
{
if (gate->flags & CLK_GATE_BIG_ENDIAN)
return ioread32be(gate->reg);
return readl(gate->reg);
}
However, it does not look like ioread32be is defined for s390, so
instead the compiler uses read1 (which becomes zpci_load). I checked
this by dumping the assembler of the compiled kunit clk-gate_test:
/root/linux/drivers/clk/clk-gate.c: 29
0x13214c9c6 <clk_gate_is_enabled+38>: tm 33(%r2),4
0x13214c9ca <clk_gate_is_enabled+42>: jne 0x13214ca82
<clk_gate_is_enabled+226>
/root/linux/./arch/s390/include/asm/pci_io.h: 64
0x13214c9ce <clk_gate_is_enabled+46>: lghi %r4,4
0x13214c9d2 <clk_gate_is_enabled+50>: la %r2,160(%r15)
0x13214c9d6 <clk_gate_is_enabled+54>: brasl %r14,0x131bbc3f0 <zpci_load>
0x13214c9dc <clk_gate_is_enabled+60>: cije %r2,0,0x13214ca78
<clk_gate_is_enabled+216>
Following up on this, I noticed that if the zpci_load is not
successful, we will return a -1:
static inline RETTYPE zpci_read_##RETTYPE(const volatile void __iomem
*addr) \
{
\
u64 data;
\
int rc;
\
\
rc = zpci_load(&data, addr, LENGTH);
\
if (rc)
\
data = -1ULL;
\
return (RETTYPE) data;
\
}
All this to say, I'm not sure the best way to resolve this problem. We
need to patch clk_gate_readl() but I am not sure the best way to go
about doing this for big_endian systems.
Any recommedations on the best way to resolve this problem is welcomed!
Thanks in advance!
-- Audra Mitchell
> >
> > Cheers,
> > Don
> >
> >>
> >>
> >> ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit --arch=s390 --cross_compile=/path/to/s390-linux-
> >>
> >> [14:55:10] Starting KUnit Kernel (1/1)...
> >> [14:55:10] ============================================================
> >> Running tests with:
> >> $ qemu-system-s390x -nodefaults -m 1024 -kernel .kunit/arch/s390/boot/bzImage -append 'kunit.enable=1 console=ttyS0 kunit_shutdown=reboot' -no-reboot -nographic -serial stdio -machine s390-ccw-virtio -cpu qemu
> >> [14:55:11] ============================= =============================
> >> [14:55:11] ================= example_init (1 subtest) =================
> >> [14:55:11] [PASSED] example_init_test
> >> [14:55:11] ================== [PASSED] example_init ===================
> >> [14:55:11] ============================= =============================
> >> [14:55:11] ============= kunit_executor_test (8 subtests) =============
> >> [14:55:11] [PASSED] parse_filter_test
> >> [14:55:11] [PASSED] filter_suites_test
> >> [14:55:11] [PASSED] filter_suites_test_glob_test
> >> [14:55:11] [PASSED] filter_suites_to_empty_test
> >> [14:55:11] [PASSED] parse_filter_attr_test
> >> [14:55:11] [PASSED] filter_attr_test
> >> [14:55:11] [PASSED] filter_attr_empty_test
> >> [14:55:11] [PASSED] filter_attr_skip_test
> >> [14:55:11] [ERROR] Test: kunit_executor_test: Expected test number 1 but found 2
> >> [14:55:11] =============== [PASSED] kunit_executor_test ===============
> >> [14:55:11] ============================= =============================
> >> [14:55:11] ============ kunit-try-catch-test (2 subtests) =============
> >> [14:55:11] [PASSED] kunit_test_try_catch_successful_try_no_catch
> >> [14:55:11] [PASSED] kunit_test_try_catch_unsuccessful_try_does_catch
> >> [14:55:11] [ERROR] Test: kunit-try-catch-test: Expected test number 1 but found 3
> >> [14:55:11] ============== [PASSED] kunit-try-catch-test ===============
> >> [14:55:11] ============================= =============================
> >> [14:55:11] ============ kunit-resource-test (12 subtests) =============
> >> [14:55:11] [PASSED] kunit_resource_test_init_resources
> >> [14:55:11] [PASSED] kunit_resource_test_alloc_resource
> >> [14:55:11] [PASSED] kunit_resource_test_destroy_resource
> >> [14:55:11] [PASSED] kunit_resource_test_remove_resource
> >> [14:55:11] [PASSED] kunit_resource_test_cleanup_resources
> >> [14:55:11] [PASSED] kunit_resource_test_proper_free_ordering
> >> [14:55:11] [PASSED] kunit_resource_test_static
> >> [14:55:11] [PASSED] kunit_resource_test_named
> >> [14:55:11] [PASSED] kunit_resource_test_action
> >> [14:55:11] [PASSED] kunit_resource_test_remove_action
> >> [14:55:11] [PASSED] kunit_resource_test_release_action
> >> [14:55:11] [PASSED] kunit_resource_test_action_ordering
> >> [14:55:11] [ERROR] Test: kunit-resource-test: Expected test number 1 but found 4
> >> [14:55:11] =============== [PASSED] kunit-resource-test ===============
> >> [14:55:11] ============================= =============================
> >> [14:55:11] =============== kunit-log-test (2 subtests) ================
> >> [14:55:11] [PASSED] kunit_log_test
> >> [14:55:11] [SKIPPED] kunit_log_newline_test
> >> [14:55:11] [ERROR] Test: kunit-log-test: Expected test number 1 but found 5
> >> [14:55:11] ================= [PASSED] kunit-log-test ==================
> >> [14:55:11] ============================= =============================
> >> [14:55:11] ================ kunit_status (2 subtests) =================
> >> [14:55:11] [PASSED] kunit_status_set_failure_test
> >> [14:55:11] [PASSED] kunit_status_mark_skipped_test
> >> [14:55:11] [ERROR] Test: kunit_status: Expected test number 1 but found 6
> >> [14:55:11] ================== [PASSED] kunit_status ===================
> >> [14:55:11] ============================= =============================
> >> [14:55:11] ================ kunit_current (2 subtests) ================
> >> [14:55:11] [PASSED] kunit_current_test
> >> [14:55:11] [PASSED] kunit_current_fail_test
> >> [14:55:11] [ERROR] Test: kunit_current: Expected test number 1 but found 7
> >> [14:55:11] ================== [PASSED] kunit_current ==================
> >> [14:55:11] ============================= =============================
> >> [14:55:11] ================ kunit_device (3 subtests) =================
> >> [14:55:11] [PASSED] kunit_device_test
> >> [14:55:11] [PASSED] kunit_device_cleanup_test
> >> [14:55:11] [PASSED] kunit_device_driver_test
> >> [14:55:11] [ERROR] Test: kunit_device: Expected test number 1 but found 8
> >> [14:55:11] ================== [PASSED] kunit_device ===================
> >> [14:55:11] ============================= =============================
> >> [14:55:11] ============= string-stream-test (12 subtests) =============
> >> [14:55:11] [PASSED] string_stream_managed_init_test
> >> [14:55:11] [PASSED] string_stream_unmanaged_init_test
> >> [14:55:11] [PASSED] string_stream_managed_free_test
> >> [14:55:11] [PASSED] string_stream_resource_free_test
> >> [14:55:11] [PASSED] string_stream_line_add_test
> >> [14:55:11] [PASSED] string_stream_variable_length_line_test
> >> [14:55:11] [PASSED] string_stream_append_test
> >> [14:55:11] [PASSED] string_stream_append_auto_newline_test
> >> [14:55:11] [PASSED] string_stream_append_empty_string_test
> >> [14:55:11] [PASSED] string_stream_no_auto_newline_test
> >> [14:55:11] [PASSED] string_stream_auto_newline_test
> >> [14:55:11] [PASSED] string_stream_performance_test
> >> [14:55:11] [ERROR] Test: string-stream-test: Expected test number 1 but found 9
> >> [14:55:11] =============== [PASSED] string-stream-test ================
> >> [14:55:11] ============================= =============================
> >> [14:55:11] =================== example (9 subtests) ===================
> >> [14:55:11] [PASSED] example_simple_test
> >> [14:55:11] [SKIPPED] example_skip_test
> >> [14:55:11] [ERROR] Test: example: missing expected subtest!
> >> [14:55:11]
> >> [14:55:11] # example_mark_skipped_test: initializing
> >> [14:55:11]
> >> [14:55:11] # example_mark_skipped_test: You should see a line below.
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: example: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: example: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: example: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: example: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: example: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: example: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: example: missing subtest result line!
> >> [14:55:11]
> >> [14:55:11] # module: kunit_example_test
> >> [14:55:11] ==================== [CRASHED] example =====================
> >> [14:55:11]
> >> [14:55:11] # example: initializing suite
> >> [14:55:11]
> >> [14:55:11] ======================== [CRASHED] ========================
> >> [14:55:11]
> >> [14:55:11] ======================== [CRASHED] ========================
> >> [14:55:11]
> >> [14:55:11] ======================== [CRASHED] ========================
> >> [14:55:11]
> >> [14:55:11] ======================== [CRASHED] ========================
> >> [14:55:11]
> >> [14:55:11] ======================== [CRASHED] ========================
> >> [14:55:11]
> >> [14:55:11] ======================== [CRASHED] ========================
> >> [14:55:11]
> >> [14:55:11] ======================== [CRASHED] ========================
> >> [14:55:11]
> >> [14:55:11] ======================== [CRASHED] ========================
> >> [14:55:11]
> >> [14:55:11] ======================== [CRASHED] ========================
> >> [14:55:11]
> >> [14:55:11] ======================== [CRASHED] ========================
> >> [14:55:11] [ERROR] Test: main: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: main: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: main: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: main: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: main: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: main: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: main: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: main: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] [ERROR] Test: main: missing expected subtest!
> >> [14:55:11] [CRASHED]
> >> [14:55:11] ============================================================
> >> [14:55:11] Testing complete. Ran 62 tests: passed: 44, crashed: 16, skipped: 2, errors: 25
> >> The kernel seems to have crashed; you can decode the stack traces with:
> >> $ scripts/decode_stacktrace.sh .kunit/vmlinux .kunit < .kunit/test.log | tee .kunit/decoded.log | ./tools/testing/kunit/kunit.py parse
> >> [14:55:11] Elapsed time: 16.240s total, 1.476s configuring, 14.294s building, 0.469s running
> >>
> >> I wonder if something with my local environment is causing troubles.
> >>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Bug Report] Multiple S390x KUNIT clk failures
2024-05-28 20:49 ` Audra Mitchell
@ 2024-05-28 22:53 ` msalter
2024-05-29 19:39 ` Stephen Boyd
0 siblings, 1 reply; 9+ messages in thread
From: msalter @ 2024-05-28 22:53 UTC (permalink / raw)
To: Audra Mitchell, Donald Zickus
Cc: Stephen Boyd, Nico Pache, KUnit Development, linux-clk,
Shuah Khan, Audra Mitchell
On Tue, 2024-05-28 at 16:49 -0400, Audra Mitchell wrote:
> On Tue, May 28, 2024 at 02:49:56PM -0400, Donald Zickus wrote:
> > (trying again without the html part that gmail likes to add. Apologies)
> >
> > On Tue, May 28, 2024 at 2:45 PM Donald Zickus <dzickus@redhat.com> wrote:
> > >
> > > Hi Stephen,
> > >
> > > On Tue, May 14, 2024 at 6:04 PM Stephen Boyd <sboyd@kernel.org> wrote:
> > > >
> > > > Quoting Nico Pache (2024-05-14 00:14:24)
> > > > > Hi Stephen,
> > > > >
> > > > > Yes, we have that commit. This is failing on fedora-ark, so it's
> > > > > constantly updated to match upstream, and the earliest recorded
> > > > > instance of it failing in our environment is v6.1.
> > > > >
> > > >
> > > > Please don't top post. I'm unable to run s390 kunit tests in qemu. They
> > > > seem to crash before finishing.
> > >
> > >
> > > I am working with Nico on moving this forward. What can we do to help resolve this?
>
> Hello all,
>
> I spent some time last week or so working on debugging these failures and I
> believe I have found the problem. I reached out to Malk Salter for advice on
> the best way to move forward with a fix on Friday the 17th, but he was on
> PTO for the last week. I was waiting for his reply before I replied to this
> thread.
>
> Also as a side note, I also ran into the same issue as Stephen with running
> the kunit tests on s390 QEMU. I did not pursue resolving that issue and
> instead just compiled the test as a module.
>
> For clarity, this is what I sent to Mark and were I believe the failure is
> occurring:
>
> The tests create a pretend clk-gate and use a "fake_reg" to emulate
> the expected behavior of the clk_gate->reg. I added some debug
> statements to the driver and noticed that the reg changes after
> initialization to -1. I also noticed that we call this to read the
> data in the clk-gate->reg:
>
> static inline u32 clk_gate_readl(struct clk_gate *gate)
> {
> if (gate->flags & CLK_GATE_BIG_ENDIAN)
> return ioread32be(gate->reg);
>
> return readl(gate->reg);
> }
>
> However, it does not look like ioread32be is defined for s390, so
It is defined. arch/s390/include/asm/io.h defines:
#define __raw_readl zpci_read_u32
and then includes include/asm-generic/io.h which has:
static inline u32 readl(const volatile void __iomem *addr)
{
u32 val;
log_read_mmio(32, addr, _THIS_IP_, _RET_IP_);
__io_br();
val = __le32_to_cpu((__le32 __force)__raw_readl(addr));
__io_ar(val);
log_post_read_mmio(val, 32, addr, _THIS_IP_, _RET_IP_);
return val;
}
...
static inline u32 ioread32be(const volatile void __iomem *addr)
{
return swab32(readl(addr));
}
which should do the right thing (s390 being BE and readl() is for 32-bit LE reads).
But I don't know the s390 compiler or ISA, so I'm not sure where the zpci_load
is coming from.
> instead the compiler uses read1 (which becomes zpci_load). I checked
> this by dumping the assembler of the compiled kunit clk-gate_test:
>
> /root/linux/drivers/clk/clk-gate.c: 29
> 0x13214c9c6 <clk_gate_is_enabled+38>: tm 33(%r2),4
> 0x13214c9ca <clk_gate_is_enabled+42>: jne 0x13214ca82
> <clk_gate_is_enabled+226>
> /root/linux/./arch/s390/include/asm/pci_io.h: 64
> 0x13214c9ce <clk_gate_is_enabled+46>: lghi %r4,4
> 0x13214c9d2 <clk_gate_is_enabled+50>: la %r2,160(%r15)
> 0x13214c9d6 <clk_gate_is_enabled+54>: brasl %r14,0x131bbc3f0 <zpci_load>
> 0x13214c9dc <clk_gate_is_enabled+60>: cije %r2,0,0x13214ca78
> <clk_gate_is_enabled+216>
>
> Following up on this, I noticed that if the zpci_load is not
> successful, we will return a -1:
>
> static inline RETTYPE zpci_read_##RETTYPE(const volatile void __iomem
> *addr) \
> {
> \
> u64 data;
> \
> int rc;
> \
>
> \
> rc = zpci_load(&data, addr, LENGTH);
> \
> if (rc)
> \
> data = -1ULL;
> \
> return (RETTYPE) data;
> \
> }
>
>
> All this to say, I'm not sure the best way to resolve this problem. We
> need to patch clk_gate_readl() but I am not sure the best way to go
> about doing this for big_endian systems.
>
> Any recommedations on the best way to resolve this problem is welcomed!
>
> Thanks in advance!
>
> -- Audra Mitchell
>
>
> > >
> > > Cheers,
> > > Don
> > >
> > > >
> > > >
> > > > ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit --arch=s390 --cross_compile=/path/to/s390-linux-
> > > >
> > > > [14:55:10] Starting KUnit Kernel (1/1)...
> > > > [14:55:10] ============================================================
> > > > Running tests with:
> > > > $ qemu-system-s390x -nodefaults -m 1024 -kernel .kunit/arch/s390/boot/bzImage -append 'kunit.enable=1 console=ttyS0 kunit_shutdown=reboot' -no-reboot
> > > > -nographic -serial stdio -machine s390-ccw-virtio -cpu qemu
> > > > [14:55:11] ============================= =============================
> > > > [14:55:11] ================= example_init (1 subtest) =================
> > > > [14:55:11] [PASSED] example_init_test
> > > > [14:55:11] ================== [PASSED] example_init ===================
> > > > [14:55:11] ============================= =============================
> > > > [14:55:11] ============= kunit_executor_test (8 subtests) =============
> > > > [14:55:11] [PASSED] parse_filter_test
> > > > [14:55:11] [PASSED] filter_suites_test
> > > > [14:55:11] [PASSED] filter_suites_test_glob_test
> > > > [14:55:11] [PASSED] filter_suites_to_empty_test
> > > > [14:55:11] [PASSED] parse_filter_attr_test
> > > > [14:55:11] [PASSED] filter_attr_test
> > > > [14:55:11] [PASSED] filter_attr_empty_test
> > > > [14:55:11] [PASSED] filter_attr_skip_test
> > > > [14:55:11] [ERROR] Test: kunit_executor_test: Expected test number 1 but found 2
> > > > [14:55:11] =============== [PASSED] kunit_executor_test ===============
> > > > [14:55:11] ============================= =============================
> > > > [14:55:11] ============ kunit-try-catch-test (2 subtests) =============
> > > > [14:55:11] [PASSED] kunit_test_try_catch_successful_try_no_catch
> > > > [14:55:11] [PASSED] kunit_test_try_catch_unsuccessful_try_does_catch
> > > > [14:55:11] [ERROR] Test: kunit-try-catch-test: Expected test number 1 but found 3
> > > > [14:55:11] ============== [PASSED] kunit-try-catch-test ===============
> > > > [14:55:11] ============================= =============================
> > > > [14:55:11] ============ kunit-resource-test (12 subtests) =============
> > > > [14:55:11] [PASSED] kunit_resource_test_init_resources
> > > > [14:55:11] [PASSED] kunit_resource_test_alloc_resource
> > > > [14:55:11] [PASSED] kunit_resource_test_destroy_resource
> > > > [14:55:11] [PASSED] kunit_resource_test_remove_resource
> > > > [14:55:11] [PASSED] kunit_resource_test_cleanup_resources
> > > > [14:55:11] [PASSED] kunit_resource_test_proper_free_ordering
> > > > [14:55:11] [PASSED] kunit_resource_test_static
> > > > [14:55:11] [PASSED] kunit_resource_test_named
> > > > [14:55:11] [PASSED] kunit_resource_test_action
> > > > [14:55:11] [PASSED] kunit_resource_test_remove_action
> > > > [14:55:11] [PASSED] kunit_resource_test_release_action
> > > > [14:55:11] [PASSED] kunit_resource_test_action_ordering
> > > > [14:55:11] [ERROR] Test: kunit-resource-test: Expected test number 1 but found 4
> > > > [14:55:11] =============== [PASSED] kunit-resource-test ===============
> > > > [14:55:11] ============================= =============================
> > > > [14:55:11] =============== kunit-log-test (2 subtests) ================
> > > > [14:55:11] [PASSED] kunit_log_test
> > > > [14:55:11] [SKIPPED] kunit_log_newline_test
> > > > [14:55:11] [ERROR] Test: kunit-log-test: Expected test number 1 but found 5
> > > > [14:55:11] ================= [PASSED] kunit-log-test ==================
> > > > [14:55:11] ============================= =============================
> > > > [14:55:11] ================ kunit_status (2 subtests) =================
> > > > [14:55:11] [PASSED] kunit_status_set_failure_test
> > > > [14:55:11] [PASSED] kunit_status_mark_skipped_test
> > > > [14:55:11] [ERROR] Test: kunit_status: Expected test number 1 but found 6
> > > > [14:55:11] ================== [PASSED] kunit_status ===================
> > > > [14:55:11] ============================= =============================
> > > > [14:55:11] ================ kunit_current (2 subtests) ================
> > > > [14:55:11] [PASSED] kunit_current_test
> > > > [14:55:11] [PASSED] kunit_current_fail_test
> > > > [14:55:11] [ERROR] Test: kunit_current: Expected test number 1 but found 7
> > > > [14:55:11] ================== [PASSED] kunit_current ==================
> > > > [14:55:11] ============================= =============================
> > > > [14:55:11] ================ kunit_device (3 subtests) =================
> > > > [14:55:11] [PASSED] kunit_device_test
> > > > [14:55:11] [PASSED] kunit_device_cleanup_test
> > > > [14:55:11] [PASSED] kunit_device_driver_test
> > > > [14:55:11] [ERROR] Test: kunit_device: Expected test number 1 but found 8
> > > > [14:55:11] ================== [PASSED] kunit_device ===================
> > > > [14:55:11] ============================= =============================
> > > > [14:55:11] ============= string-stream-test (12 subtests) =============
> > > > [14:55:11] [PASSED] string_stream_managed_init_test
> > > > [14:55:11] [PASSED] string_stream_unmanaged_init_test
> > > > [14:55:11] [PASSED] string_stream_managed_free_test
> > > > [14:55:11] [PASSED] string_stream_resource_free_test
> > > > [14:55:11] [PASSED] string_stream_line_add_test
> > > > [14:55:11] [PASSED] string_stream_variable_length_line_test
> > > > [14:55:11] [PASSED] string_stream_append_test
> > > > [14:55:11] [PASSED] string_stream_append_auto_newline_test
> > > > [14:55:11] [PASSED] string_stream_append_empty_string_test
> > > > [14:55:11] [PASSED] string_stream_no_auto_newline_test
> > > > [14:55:11] [PASSED] string_stream_auto_newline_test
> > > > [14:55:11] [PASSED] string_stream_performance_test
> > > > [14:55:11] [ERROR] Test: string-stream-test: Expected test number 1 but found 9
> > > > [14:55:11] =============== [PASSED] string-stream-test ================
> > > > [14:55:11] ============================= =============================
> > > > [14:55:11] =================== example (9 subtests) ===================
> > > > [14:55:11] [PASSED] example_simple_test
> > > > [14:55:11] [SKIPPED] example_skip_test
> > > > [14:55:11] [ERROR] Test: example: missing expected subtest!
> > > > [14:55:11]
> > > > [14:55:11] # example_mark_skipped_test: initializing
> > > > [14:55:11]
> > > > [14:55:11] # example_mark_skipped_test: You should see a line below.
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: example: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: example: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: example: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: example: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: example: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: example: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: example: missing subtest result line!
> > > > [14:55:11]
> > > > [14:55:11] # module: kunit_example_test
> > > > [14:55:11] ==================== [CRASHED] example =====================
> > > > [14:55:11]
> > > > [14:55:11] # example: initializing suite
> > > > [14:55:11]
> > > > [14:55:11] ======================== [CRASHED] ========================
> > > > [14:55:11]
> > > > [14:55:11] ======================== [CRASHED] ========================
> > > > [14:55:11]
> > > > [14:55:11] ======================== [CRASHED] ========================
> > > > [14:55:11]
> > > > [14:55:11] ======================== [CRASHED] ========================
> > > > [14:55:11]
> > > > [14:55:11] ======================== [CRASHED] ========================
> > > > [14:55:11]
> > > > [14:55:11] ======================== [CRASHED] ========================
> > > > [14:55:11]
> > > > [14:55:11] ======================== [CRASHED] ========================
> > > > [14:55:11]
> > > > [14:55:11] ======================== [CRASHED] ========================
> > > > [14:55:11]
> > > > [14:55:11] ======================== [CRASHED] ========================
> > > > [14:55:11]
> > > > [14:55:11] ======================== [CRASHED] ========================
> > > > [14:55:11] [ERROR] Test: main: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: main: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: main: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: main: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: main: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: main: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: main: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: main: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] [ERROR] Test: main: missing expected subtest!
> > > > [14:55:11] [CRASHED]
> > > > [14:55:11] ============================================================
> > > > [14:55:11] Testing complete. Ran 62 tests: passed: 44, crashed: 16, skipped: 2, errors: 25
> > > > The kernel seems to have crashed; you can decode the stack traces with:
> > > > $ scripts/decode_stacktrace.sh .kunit/vmlinux .kunit < .kunit/test.log | tee .kunit/decoded.log | ./tools/testing/kunit/kunit.py parse
> > > > [14:55:11] Elapsed time: 16.240s total, 1.476s configuring, 14.294s building, 0.469s running
> > > >
> > > > I wonder if something with my local environment is causing troubles.
> > > >
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Bug Report] Multiple S390x KUNIT clk failures
2024-05-28 22:53 ` msalter
@ 2024-05-29 19:39 ` Stephen Boyd
2024-05-30 20:12 ` Audra Mitchell
0 siblings, 1 reply; 9+ messages in thread
From: Stephen Boyd @ 2024-05-29 19:39 UTC (permalink / raw)
To: Audra Mitchell, Donald Zickus, msalter
Cc: Nico Pache, KUnit Development, linux-clk, Shuah Khan,
Audra Mitchell
Quoting msalter@redhat.com (2024-05-28 15:53:48)
> On Tue, 2024-05-28 at 16:49 -0400, Audra Mitchell wrote:
> >
> > I spent some time last week or so working on debugging these failures and I
> > believe I have found the problem. I reached out to Malk Salter for advice on
> > the best way to move forward with a fix on Friday the 17th, but he was on
> > PTO for the last week. I was waiting for his reply before I replied to this
> > thread.
> >
> > Also as a side note, I also ran into the same issue as Stephen with running
> > the kunit tests on s390 QEMU. I did not pursue resolving that issue and
> > instead just compiled the test as a module.
> >
> > For clarity, this is what I sent to Mark and were I believe the failure is
> > occurring:
> >
> > The tests create a pretend clk-gate and use a "fake_reg" to emulate
> > the expected behavior of the clk_gate->reg. I added some debug
> > statements to the driver and noticed that the reg changes after
> > initialization to -1. I also noticed that we call this to read the
> > data in the clk-gate->reg:
> >
> > static inline u32 clk_gate_readl(struct clk_gate *gate)
> > {
> > if (gate->flags & CLK_GATE_BIG_ENDIAN)
> > return ioread32be(gate->reg);
> >
> > return readl(gate->reg);
> > }
> >
> > However, it does not look like ioread32be is defined for s390, so
>
> It is defined. arch/s390/include/asm/io.h defines:
>
> #define __raw_readl zpci_read_u32
>
> and then includes include/asm-generic/io.h which has:
>
> static inline u32 readl(const volatile void __iomem *addr)
> {
> u32 val;
>
> log_read_mmio(32, addr, _THIS_IP_, _RET_IP_);
> __io_br();
> val = __le32_to_cpu((__le32 __force)__raw_readl(addr));
> __io_ar(val);
> log_post_read_mmio(val, 32, addr, _THIS_IP_, _RET_IP_);
> return val;
> }
> ...
> static inline u32 ioread32be(const volatile void __iomem *addr)
> {
> return swab32(readl(addr));
> }
>
> which should do the right thing (s390 being BE and readl() is for 32-bit LE reads).
>
> But I don't know the s390 compiler or ISA, so I'm not sure where the zpci_load
> is coming from.
>
So the problem is that the zpci_read_u32() fails and returns -1?
This test isn't the best because it uses fakes iomem and architectures
may not like that. We really need to implement something in KUnit core
to allocate a fake iomem region and then plumb that through all the
architectures so that the iomem functions like readl, writel, etc. go a
different direction when the pointer is for the fake region.
Probably the best thing to do in the short term here is to prevent this
test from running on S390 via Kconfig.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Bug Report] Multiple S390x KUNIT clk failures
2024-05-29 19:39 ` Stephen Boyd
@ 2024-05-30 20:12 ` Audra Mitchell
0 siblings, 0 replies; 9+ messages in thread
From: Audra Mitchell @ 2024-05-30 20:12 UTC (permalink / raw)
To: Stephen Boyd
Cc: Donald Zickus, msalter, Nico Pache, KUnit Development, linux-clk,
Shuah Khan, Audra Mitchell
On Wed, May 29, 2024 at 12:39:34PM -0700, Stephen Boyd wrote:
> Quoting msalter@redhat.com (2024-05-28 15:53:48)
> > On Tue, 2024-05-28 at 16:49 -0400, Audra Mitchell wrote:
> > >
> > > I spent some time last week or so working on debugging these failures and I
> > > believe I have found the problem. I reached out to Malk Salter for advice on
> > > the best way to move forward with a fix on Friday the 17th, but he was on
> > > PTO for the last week. I was waiting for his reply before I replied to this
> > > thread.
> > >
> > > Also as a side note, I also ran into the same issue as Stephen with running
> > > the kunit tests on s390 QEMU. I did not pursue resolving that issue and
> > > instead just compiled the test as a module.
> > >
> > > For clarity, this is what I sent to Mark and were I believe the failure is
> > > occurring:
> > >
> > > The tests create a pretend clk-gate and use a "fake_reg" to emulate
> > > the expected behavior of the clk_gate->reg. I added some debug
> > > statements to the driver and noticed that the reg changes after
> > > initialization to -1. I also noticed that we call this to read the
> > > data in the clk-gate->reg:
> > >
> > > static inline u32 clk_gate_readl(struct clk_gate *gate)
> > > {
> > > if (gate->flags & CLK_GATE_BIG_ENDIAN)
> > > return ioread32be(gate->reg);
> > >
> > > return readl(gate->reg);
> > > }
> > >
> > > However, it does not look like ioread32be is defined for s390, so
> >
> > It is defined. arch/s390/include/asm/io.h defines:
> >
> > #define __raw_readl zpci_read_u32
> >
> > and then includes include/asm-generic/io.h which has:
> >
> > static inline u32 readl(const volatile void __iomem *addr)
> > {
> > u32 val;
> >
> > log_read_mmio(32, addr, _THIS_IP_, _RET_IP_);
> > __io_br();
> > val = __le32_to_cpu((__le32 __force)__raw_readl(addr));
> > __io_ar(val);
> > log_post_read_mmio(val, 32, addr, _THIS_IP_, _RET_IP_);
> > return val;
> > }
> > ...
> > static inline u32 ioread32be(const volatile void __iomem *addr)
> > {
> > return swab32(readl(addr));
> > }
> >
> > which should do the right thing (s390 being BE and readl() is for 32-bit LE reads).
> >
> > But I don't know the s390 compiler or ISA, so I'm not sure where the zpci_load
> > is coming from.
> >
>
> So the problem is that the zpci_read_u32() fails and returns -1?
>
> This test isn't the best because it uses fakes iomem and architectures
> may not like that. We really need to implement something in KUnit core
> to allocate a fake iomem region and then plumb that through all the
> architectures so that the iomem functions like readl, writel, etc. go a
> different direction when the pointer is for the fake region.
>
> Probably the best thing to do in the short term here is to prevent this
> test from running on S390 via Kconfig.
Hey Stephen, thanks a bunch for looking at this. I do not have a lot of
experience with s390 to vote one way or another for how best to approach
resolving this problem.
By the way zpci_load is coming from here:
readl calls __raw_readl
s390 defines raw_readl as zpci_read_u32:
#define __raw_readl zpci_read_u32
And zpci_read is defined here, which then calls zpci_load:
#define zpci_read(LENGTH, RETTYPE) \
static inline RETTYPE zpci_read_##RETTYPE(const volatile void __iomem *addr) \
{ \
u64 data; \
int rc; \
\
rc = zpci_load(&data, addr, LENGTH); \
if (rc) \
data = -1ULL; \
return (RETTYPE) data; \
}
Thanks again!
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2024-05-30 20:12 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-05-14 1:08 [Bug Report] Multiple S390x KUNIT clk failures Nico Pache
2024-05-14 3:38 ` Stephen Boyd
2024-05-14 7:14 ` Nico Pache
2024-05-14 22:04 ` Stephen Boyd
[not found] ` <CAK18DXZyEHZ=1TC52kQQ89gscFLph0e_4zB_bt=DTwR-A=0UPA@mail.gmail.com>
2024-05-28 18:49 ` Donald Zickus
2024-05-28 20:49 ` Audra Mitchell
2024-05-28 22:53 ` msalter
2024-05-29 19:39 ` Stephen Boyd
2024-05-30 20:12 ` Audra Mitchell
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox