From: Guenter Roeck <linux@roeck-us.net>
To: Rae Moar <rmoar@google.com>
Cc: shuah@kernel.org, davidgow@google.com, dlatypov@google.com,
brendan.higgins@linux.dev, linux-kselftest@vger.kernel.org,
kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org,
keescook@chromium.org, linux-hardening@vger.kernel.org,
jstultz@google.com, tglx@linutronix.de, sboyd@kernel.org
Subject: Re: [PATCH v3 4/9] kunit: Add ability to filter attributes
Date: Tue, 29 Aug 2023 15:46:18 -0700 [thread overview]
Message-ID: <55950256-c00a-4d21-a2c0-cf9f0e5b8a9a@roeck-us.net> (raw)
In-Reply-To: <20230725212522.1622716-5-rmoar@google.com>
Hi,
On Tue, Jul 25, 2023 at 09:25:15PM +0000, Rae Moar wrote:
> Add filtering of test attributes. Users can filter tests using the
> module_param called "filter".
>
> Filters are imputed in the format: <attribute_name><operation><value>
>
> Example: kunit.filter="speed>slow"
>
> Operations include: >, <, >=, <=, !=, and =. These operations will act the
> same for attributes of the same type but may not between types.
>
> Note multiple filters can be inputted by separating them with a comma.
> Example: kunit.filter="speed=slow, module!=example"
>
> Since both suites and test cases can have attributes, there may be
> conflicts. The process of filtering follows these rules:
> - Filtering always operates at a per-test level.
> - If a test has an attribute set, then the test's value is filtered on.
> - Otherwise, the value falls back to the suite's value.
> - If neither are set, the attribute has a global "default" value, which
> is used.
>
> Filtered tests will not be run or show in output. The tests can instead be
> skipped using the configurable option "kunit.filter_action=skip".
>
> Note the default settings for running tests remains unfiltered.
>
> Finally, add "filter" methods for the speed and module attributes to parse
> and compare attribute values.
>
> Note this filtering functionality will be added to kunit.py in the next
> patch.
>
> Reviewed-by: David Gow <davidgow@google.com>
> Signed-off-by: Rae Moar <rmoar@google.com>
This patch causes all my s390 boot tests to crash.
[ 17.009188] # module: kunit
[ 17.009239] 1..8
[ 17.014967] ok 1 parse_filter_test
[ 17.020755] ok 2 filter_suites_test
[ 17.026350] ok 3 filter_suites_test_glob_test
[ 17.033102] ok 4 filter_suites_to_empty_test
[ 17.035422] Unable to handle kernel pointer dereference in virtual kernel address space
[ 17.035546] Failing address: 0000000001501000 TEID: 0000000001501407
[ 17.035617] Fault in home space mode while using kernel ASCE.
[ 17.035833] AS:0000000002164007 R3:000000001f71c007 S:000000001f711000 P:000000000150131f
[ 17.036313] Oops: 0004 ilc:2 [#1] SMP
[ 17.036851] Modules linked in:
[ 17.037026] CPU: 0 PID: 193 Comm: kunit_try_catch Tainted: G N 6.5.0-01207-g1c59d383390f #1
[ 17.037106] Hardware name: QEMU 8561 QEMU (KVM/Linux)
[ 17.037199] Krnl PSW : 0704e00180000000 0000000000950e18 (kunit_next_attr_filter+0xc8/0x1c8)
[ 17.037451] R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3
[ 17.037531] Krnl GPRS: 0000037f00000000 000000000150120a 000000000000003e 000000000000003e
[ 17.037583] 0000038000197e04 0000038000197c04 0000038000197df0 000003800000000a
[ 17.037629] 0000038000000005 0000000001501203 00000000015011fe 0000037f0000000c
[ 17.037675] 0000000005404100 0000038000197e08 0000000000950dda 0000038000197cd8
[ 17.038176] Krnl Code: 0000000000950e0a: b9180098 agfr %r9,%r8
[ 17.038176] 0000000000950e0e: a7080000 lhi %r0,0
[ 17.038176] #0000000000950e12: e32090000090 llgc %r2,0(%r9)
[ 17.038176] >0000000000950e18: 92009000 mvi 0(%r9),0
[ 17.038176] 0000000000950e1c: b904003a lgr %r3,%r10
[ 17.038176] 0000000000950e20: c418007bd73c lgrl %r1,00000000018cbc98
[ 17.038176] 0000000000950e26: b25d0013 clst %r1,%r3
[ 17.038176] 0000000000950e2a: a714fffe brc 1,0000000000950e26
[ 17.038625] Call Trace:
[ 17.038679] [<0000000000950e18>] kunit_next_attr_filter+0xc8/0x1c8
[ 17.038749] [<000000000094e5f2>] parse_filter_attr_test+0x92/0x360
[ 17.038796] [<000000000094e39a>] kunit_generic_run_threadfn_adapter+0x32/0x48
[ 17.038840] [<0000000000186702>] kthread+0x142/0x148
[ 17.038884] [<0000000000103138>] __ret_from_fork+0x40/0x58
[ 17.038927] [<000000000102df9a>] ret_from_fork+0xa/0x30
[ 17.038984] INFO: lockdep is turned off.
[ 17.039027] Last Breaking-Event-Address:
[ 17.039062] [<0000000000950da4>] kunit_next_attr_filter+0x54/0x1c8
[ 17.039303] Kernel panic - not syncing: Fatal exception: panic_on_oops
Bisect log is attached.
Guenter
---
# bad: [1c59d383390f970b891b503b7f79b63a02db2ec5] Merge tag 'linux-kselftest-nolibc-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
# good: [2dde18cd1d8fac735875f2e4987f11817cc0bc2c] Linux 6.5
git bisect start '1c59d383390f' 'v6.5'
# good: [42a7f6e3ffe06308c1ec43a7dac39a27de101574] Merge tag 'x86_microcode_for_v6.6_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 42a7f6e3ffe06308c1ec43a7dac39a27de101574
# good: [330235e87410349042468b52baff02af7cb7d331] Merge tag 'acpi-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
git bisect good 330235e87410349042468b52baff02af7cb7d331
# good: [5a31cc7297072a7266a910ca5266b640d27803b4] Merge tag 'linux-kselftest-next-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
git bisect good 5a31cc7297072a7266a910ca5266b640d27803b4
# good: [eea70cdac61087fdb7a1c1e6f306c0bf38ec39d4] tools/nolibc: riscv: shrink _start with _start_c
git bisect good eea70cdac61087fdb7a1c1e6f306c0bf38ec39d4
# good: [faeb4e09fe77262f9a6b2f9f874eec0b6850721e] selftests/nolibc: add test support for ppc64
git bisect good faeb4e09fe77262f9a6b2f9f874eec0b6850721e
# bad: [76066f93f1df27657eb937b7c9c091e3a6abf4db] kunit: add tests for filtering attributes
git bisect bad 76066f93f1df27657eb937b7c9c091e3a6abf4db
# good: [64bd4641310c41a1ecf07c13c67bc0ed61045dfd] MAINTAINERS: add Rust KUnit files to the KUnit entry
git bisect good 64bd4641310c41a1ecf07c13c67bc0ed61045dfd
# bad: [529534e8cba3e60f843a682e2a3149612b30d608] kunit: Add ability to filter attributes
git bisect bad 529534e8cba3e60f843a682e2a3149612b30d608
# good: [02c2d0c2a84172c3c7ec0229c61db55d23dd4730] kunit: Add speed attribute
git bisect good 02c2d0c2a84172c3c7ec0229c61db55d23dd4730
# good: [a00a72709175a4d53096301a8792b8171d1223e5] kunit: Add module attribute
git bisect good a00a72709175a4d53096301a8792b8171d1223e5
# first bad commit: [529534e8cba3e60f843a682e2a3149612b30d608] kunit: Add ability to filter attributes
next prev parent reply other threads:[~2023-08-29 22:47 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-07-25 21:25 [PATCH v3 0/9] kunit: Add test attributes API Rae Moar
2023-07-25 21:25 ` [PATCH v3 1/9] kunit: Add test attributes API structure Rae Moar
2023-07-25 21:25 ` [PATCH v3 2/9] kunit: Add speed attribute Rae Moar
2023-07-25 21:25 ` [PATCH v3 3/9] kunit: Add module attribute Rae Moar
2023-07-25 21:25 ` [PATCH v3 4/9] kunit: Add ability to filter attributes Rae Moar
2023-08-29 22:46 ` Guenter Roeck [this message]
2023-08-30 0:24 ` David Gow
2023-08-30 1:48 ` Guenter Roeck
2023-07-25 21:25 ` [PATCH v3 5/9] kunit: tool: Add command line interface to filter and report attributes Rae Moar
2023-07-25 21:25 ` [PATCH v3 6/9] kunit: memcpy: Mark tests as slow using test attributes Rae Moar
2023-07-25 21:25 ` [PATCH v3 7/9] kunit: time: Mark test " Rae Moar
2023-07-25 21:25 ` [PATCH v3 8/9] kunit: add tests for filtering attributes Rae Moar
2023-07-25 21:25 ` [PATCH v3 9/9] kunit: Add documentation of KUnit test attributes Rae Moar
2023-07-26 14:56 ` [PATCH v3 0/9] kunit: Add test attributes API Arthur Grillo Queiroz Cabral
2023-07-26 14:57 ` Arthur Grillo Queiroz Cabral
2023-07-28 10:38 ` Guenter Roeck
2023-07-29 7:53 ` David Gow
2023-07-29 13:55 ` Guenter Roeck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=55950256-c00a-4d21-a2c0-cf9f0e5b8a9a@roeck-us.net \
--to=linux@roeck-us.net \
--cc=brendan.higgins@linux.dev \
--cc=davidgow@google.com \
--cc=dlatypov@google.com \
--cc=jstultz@google.com \
--cc=keescook@chromium.org \
--cc=kunit-dev@googlegroups.com \
--cc=linux-hardening@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=rmoar@google.com \
--cc=sboyd@kernel.org \
--cc=shuah@kernel.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox