From: Rob Herring <robh+dt@kernel.org>
To: Frank Rowand <frowand.list@gmail.com>
Cc: Naresh Kamboju <naresh.kamboju@linaro.org>,
Brendan Higgins <brendanhiggins@google.com>,
"open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS"
<devicetree@vger.kernel.org>,
Anders Roxell <anders.roxell@linaro.org>
Subject: Re: Device Tree runtime unit tests: Harmonisation
Date: Wed, 2 Feb 2022 14:29:29 -0600 [thread overview]
Message-ID: <CAL_JsqLnv_=mcs+pAXRArSg7uccPFpDhcbBOpHGaWwjOZdt9DQ@mail.gmail.com> (raw)
In-Reply-To: <0b6ab190-4c35-94fc-6565-6382d461ae84@gmail.com>
On Wed, Feb 2, 2022 at 12:38 PM Frank Rowand <frowand.list@gmail.com> wrote:
>
> On 2/2/22 5:31 AM, Naresh Kamboju wrote:
> > Linaro started doing Linux kernel Functional Validation (LKFT).
> > As part of LKFT recently we have enabled CONFIG_OF_UNITTEST=y in our
> > daily test CI.
> >
> > The output of the test looks as below. The current problem is that we
> > have a hard time to see (grep) pass/fail for each individual test. We
> > only see a summary at the end with x pass and y fails.
>
> The FAIL messages are printed at loglevel KERN_ERR. The pass messages
> are printed at loglevel KERN_DEBUG. To see the pass messages, set the
> loglevel to allow debug output.
That alone is not enough. Unless there's a DEBUG define, the
pr_debug() is going to print nothing.
> Unfortunately this can add lots of debug output, unless you use dynamic
> debug to only enable debug for drivers/of/unittest.o. There are only
> a few other pr_debug() messages in unittest.
Dynamic debug is one option. Another would be a module param to enable
running the tests. Then it can be built, but has to be explicitly
enabled at boot time. A 3rd option is making it work as a module, then
it's run when loaded. (That was the original plan.)
> I think a better solution would be to add a config option, something
> like CONFIG_OF_UNITTEST_VERBOSE, that would print the pass messages
> at loglevel KERN_ERR. I'll submit a patch for that and see what the
> review responses are.
Nak for another config option.
> > We would like to get your opinion of how hard it would be to include
> > that in the output per test. Maybe like TAP version 14?
> > Another question would be how hard do you think it would be to rewrite
> > this to a kunit test, if even applicable? I have provided the kunit
> > output links at the end of this email.
>
> Devicetree unittests were suggested as a good candidate as a first
> test to convert to kunit when kunit was implemented. Brendan tried
> to convert it, and we quickly saw that it was not a good candidate.
> Devicetree unittests do not fit the unit test mold; they are a very
> different creature. Brendan has a good term for this type of test
> (Brendan, was it "acceptance" test?).
I thought you ended up agreeing with using kunit? Whatever you want to
call the DT tests, there's not really any good reason to do our own
pass/fail messages.
Rob
next prev parent reply other threads:[~2022-02-02 20:29 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-02 11:31 Device Tree runtime unit tests: Harmonisation Naresh Kamboju
2022-02-02 18:38 ` Frank Rowand
2022-02-02 20:29 ` Rob Herring [this message]
2022-02-02 21:14 ` Brendan Higgins
2022-02-02 22:01 ` Frank Rowand
2022-02-03 0:15 ` Rob Herring
2022-02-03 4:52 ` Frank Rowand
2022-02-02 20:54 ` Brendan Higgins
2022-02-02 22:04 ` Frank Rowand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAL_JsqLnv_=mcs+pAXRArSg7uccPFpDhcbBOpHGaWwjOZdt9DQ@mail.gmail.com' \
--to=robh+dt@kernel.org \
--cc=anders.roxell@linaro.org \
--cc=brendanhiggins@google.com \
--cc=devicetree@vger.kernel.org \
--cc=frowand.list@gmail.com \
--cc=naresh.kamboju@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).