From: Greg KH <gregkh@linuxfoundation.org>
To: Brendan Higgins <brendanhiggins@google.com>
Cc: Frank Rowand <frowand.list@gmail.com>,
Kees Cook <keescook@google.com>,
Kieran Bingham <kieran.bingham@ideasonboard.com>,
Luis Chamberlain <mcgrof@kernel.org>,
Rob Herring <robh@kernel.org>, Stephen Boyd <sboyd@kernel.org>,
shuah <shuah@kernel.org>, devicetree <devicetree@vger.kernel.org>,
dri-devel <dri-devel@lists.freedesktop.org>,
kunit-dev@googlegroups.com, linux-doc@vger.kernel.org,
linux-fsdevel@vger.kernel.org, linux-kbuild@vger.kernel.org,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
linux-kselftest@vger.kernel.org,
linux-nvdimm <linux-nvdimm@lists.01.org>,
linux-um@lists.infradead.org,
Sasha Levin <Alexander.Levin@microsoft.com>,
"Bird, Timothy" <Tim.Bird@sony.com>,
Amir Goldstein <amir73il@gmail.com>,
Dan Carpenter <dan.carpenter@oracle.com>,
Dan Williams <dan.j.williams@intel.com>,
Daniel Vetter <daniel@ffwll.ch>, Jeff Dike <jdike@addtoit.com>,
Joel Stanley <joel@jms.id.au>,
Julia Lawall <julia.lawall@lip6.fr>,
Kevin Hilman <khilman@baylibre.com>,
Knut Omang <knut.omang@oracle.com>,
Logan Gunthorpe <logang@deltatee.com>,
Michael Ellerman <mpe@ellerman.id.au>,
Petr Mladek <pmladek@suse.com>,
Richard Weinberger <richard@nod.at>,
David Rientjes <rientjes@google.com>,
Steven Rostedt <rostedt@goodmis.org>,
wfg@linux.intel.com, Felix Guo <felixguoxiuping@gmail.com>
Subject: Re: [PATCH v2 12/17] kunit: tool: add Python wrappers for running KUnit tests
Date: Sat, 4 May 2019 12:42:51 +0200 [thread overview]
Message-ID: <20190504104251.GB1478@kroah.com> (raw)
In-Reply-To: <CAFd5g47aY-CL+d7DfiyTidY4aAVY+eg1TM1UJ4nYqKSfHOi-0w@mail.gmail.com>
On Fri, May 03, 2019 at 04:14:49PM -0700, Brendan Higgins wrote:
> In any case, it sounds like you and Greg are in agreement on the core
> libraries generating the output in TAP13, so I won't argue that point
> further.
Great!
> ## Analysis of using TAP13
>
> One of my earlier concerns was that TAP13 is a bit over constrained
> for what I would like to output from the KUnit core. It only allows
> data to be output as either:
> - test number
> - ok/not ok with single line description
> - directive
> - diagnostics
> - YAML block
>
> The test number must become before a set of ok/not ok lines, and does
> not contain any additional information. One annoying thing about this
> is it doesn't provide any kind of nesting or grouping.
It should handle nesting just fine, I think we do that already today.
> There is one ok/not ok line per test and it may have a short
> description of the test immediately after 'ok' or 'not ok'; this is
> problematic because it wants the first thing you say about a test to
> be after you know whether it passes or not.
Take a look at the output of our current tests, I think you might find
it to be a bit more flexible than you think.
Also, this isn't our standard, we picked it because we needed a standard
that the tools of today already understand. It might have issues and
other problems, but we are not in the business of writing test output
parsing tools, and we don't want to force everyone out there to write
custom parsers. We want them to be able to use the tools they already
have so they can test the kernel, and to do so as easily as possible.
thanks,
greg k-h
next prev parent reply other threads:[~2019-05-04 10:42 UTC|newest]
Thread overview: 131+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-01 23:01 [PATCH v2 00/17] kunit: introduce KUnit, the Linux kernel unit testing framework Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 01/17] kunit: test: add KUnit test runner core Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 02/17] kunit: test: add test resource management API Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 03/17] kunit: test: add string_stream a std::stream like string builder Brendan Higgins
2019-05-03 1:26 ` shuah
2019-05-03 4:37 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 04/17] kunit: test: add kunit_stream a std::stream like logger Brendan Higgins
2019-05-02 11:00 ` Greg KH
2019-05-02 20:25 ` Brendan Higgins
2019-05-02 21:18 ` Frank Rowand
2019-05-03 1:50 ` shuah
2019-05-03 5:48 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 05/17] kunit: test: add the concept of expectations Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 06/17] kbuild: enable building KUnit Brendan Higgins
2019-05-10 3:03 ` Masahiro Yamada
2019-05-10 10:27 ` Brendan Higgins
2019-05-10 10:30 ` Masahiro Yamada
2019-05-10 10:33 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 07/17] kunit: test: add initial tests Brendan Higgins
2019-05-02 10:58 ` Greg KH
2019-05-02 20:30 ` Brendan Higgins
2019-05-03 1:27 ` shuah
2019-05-03 5:18 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 08/17] kunit: test: add support for test abort Brendan Higgins
2019-05-03 3:14 ` Logan Gunthorpe
2019-05-03 6:48 ` Brendan Higgins
2019-05-03 12:33 ` Logan Gunthorpe
2019-05-06 8:48 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 09/17] kunit: test: add tests for kunit " Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 10/17] kunit: test: add the concept of assertions Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 11/17] kunit: test: add test managed resource tests Brendan Higgins
2019-05-03 14:34 ` shuah
2019-05-06 9:03 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 12/17] kunit: tool: add Python wrappers for running KUnit tests Brendan Higgins
2019-05-02 11:02 ` Greg KH
2019-05-02 18:07 ` Brendan Higgins
2019-05-02 21:16 ` Frank Rowand
2019-05-02 23:45 ` Brendan Higgins
2019-05-03 1:45 ` Frank Rowand
2019-05-03 5:36 ` Brendan Higgins
2019-05-03 18:59 ` Frank Rowand
2019-05-03 23:14 ` Brendan Higgins
2019-05-04 10:42 ` Greg KH [this message]
2019-05-06 0:19 ` Frank Rowand
2019-05-06 17:43 ` Kees Cook
2019-05-06 21:42 ` Brendan Higgins
2019-05-06 21:39 ` Brendan Higgins
2019-05-07 19:13 ` Tim.Bird
2019-05-03 6:41 ` Greg KH
2019-05-01 23:01 ` [PATCH v2 13/17] kunit: defconfig: add defconfigs for building " Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 14/17] Documentation: kunit: add documentation for KUnit Brendan Higgins
2019-05-09 5:08 ` Randy Dunlap
2019-05-09 17:38 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 15/17] MAINTAINERS: add entry for KUnit the unit testing framework Brendan Higgins
2019-05-03 14:38 ` shuah
2019-05-06 9:18 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 16/17] kernel/sysctl-test: Add null pointer test for sysctl.c:proc_dointvec() Brendan Higgins
2019-05-02 11:03 ` Greg KH
2019-05-02 18:14 ` Tim.Bird
2019-05-02 18:45 ` Brendan Higgins
2019-05-03 6:42 ` Greg KH
2019-05-03 23:41 ` Brendan Higgins
2019-05-04 10:40 ` Greg KH
2019-05-01 23:01 ` [PATCH v2 17/17] MAINTAINERS: add proc sysctl KUnit test to PROC SYSCTL section Brendan Higgins
2019-05-02 10:50 ` [PATCH v2 00/17] kunit: introduce KUnit, the Linux kernel unit testing framework Greg KH
2019-05-02 11:05 ` Greg KH
2019-05-03 0:41 ` Brendan Higgins
2019-05-02 14:04 ` shuah
2019-05-03 0:44 ` Brendan Higgins
2019-05-03 3:18 ` Logan Gunthorpe
2019-05-07 3:14 ` Frank Rowand
2019-05-07 8:01 ` Greg KH
2019-05-07 15:23 ` shuah
2019-05-09 1:01 ` Frank Rowand
2019-05-07 17:22 ` Theodore Ts'o
2019-05-08 19:17 ` Brendan Higgins
2019-05-09 0:58 ` Frank Rowand
2019-05-09 1:44 ` Theodore Ts'o
2019-05-09 2:18 ` Frank Rowand
2019-05-14 8:22 ` Brendan Higgins
2019-05-09 0:43 ` Frank Rowand
2019-05-09 1:58 ` Theodore Ts'o
2019-05-09 2:13 ` Frank Rowand
2019-05-09 3:20 ` Theodore Ts'o
2019-05-09 11:52 ` Knut Omang
2019-05-09 13:35 ` Theodore Ts'o
2019-05-09 14:48 ` Knut Omang
2019-05-09 17:00 ` Tim.Bird
2019-05-09 17:42 ` Daniel Vetter
2019-05-09 18:12 ` Frank Rowand
2019-05-09 21:42 ` Theodore Ts'o
2019-05-09 22:20 ` Logan Gunthorpe
2019-05-09 23:30 ` Theodore Ts'o
2019-05-09 23:40 ` Logan Gunthorpe
2019-05-10 4:47 ` Theodore Ts'o
2019-05-10 5:18 ` Frank Rowand
2019-05-10 5:48 ` Knut Omang
2019-05-10 8:12 ` Daniel Vetter
2019-05-10 10:23 ` Brendan Higgins
2019-05-10 12:12 ` Knut Omang
2019-05-10 20:54 ` Brendan Higgins
2019-05-10 22:18 ` Frank Rowand
2019-05-11 6:17 ` Knut Omang
2019-05-14 6:39 ` Brendan Higgins
2019-05-10 21:59 ` Frank Rowand
2019-05-11 6:43 ` Knut Omang
2019-05-14 8:00 ` Brendan Higgins
2019-05-10 11:36 ` Knut Omang
2019-05-10 16:17 ` Logan Gunthorpe
2019-05-10 22:13 ` Frank Rowand
2019-05-14 8:38 ` Brendan Higgins
2019-05-15 0:14 ` Frank Rowand
2019-05-15 0:26 ` Logan Gunthorpe
2019-05-10 21:52 ` Frank Rowand
2019-05-14 20:54 ` Brendan Higgins
2019-05-10 21:12 ` Frank Rowand
2019-05-11 17:33 ` Theodore Ts'o
2019-05-13 14:44 ` Daniel Vetter
2019-05-14 6:04 ` Brendan Higgins
2019-05-14 12:05 ` Daniel Vetter
2019-05-14 18:36 ` Brendan Higgins
2019-05-15 7:41 ` Daniel Vetter
2019-05-22 21:38 ` Brendan Higgins
2019-05-23 8:40 ` Daniel Vetter
2019-05-15 0:26 ` Frank Rowand
2019-05-15 4:28 ` Theodore Ts'o
2019-05-10 5:11 ` Frank Rowand
2019-05-10 10:43 ` Theodore Ts'o
2019-05-10 21:05 ` Frank Rowand
2019-05-09 15:19 ` Masahiro Yamada
2019-05-10 10:25 ` Brendan Higgins
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190504104251.GB1478@kroah.com \
--to=gregkh@linuxfoundation.org \
--cc=Alexander.Levin@microsoft.com \
--cc=Tim.Bird@sony.com \
--cc=amir73il@gmail.com \
--cc=brendanhiggins@google.com \
--cc=dan.carpenter@oracle.com \
--cc=dan.j.williams@intel.com \
--cc=daniel@ffwll.ch \
--cc=devicetree@vger.kernel.org \
--cc=dri-devel@lists.freedesktop.org \
--cc=felixguoxiuping@gmail.com \
--cc=frowand.list@gmail.com \
--cc=jdike@addtoit.com \
--cc=joel@jms.id.au \
--cc=julia.lawall@lip6.fr \
--cc=keescook@google.com \
--cc=khilman@baylibre.com \
--cc=kieran.bingham@ideasonboard.com \
--cc=knut.omang@oracle.com \
--cc=kunit-dev@googlegroups.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kbuild@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-nvdimm@lists.01.org \
--cc=linux-um@lists.infradead.org \
--cc=logang@deltatee.com \
--cc=mcgrof@kernel.org \
--cc=mpe@ellerman.id.au \
--cc=pmladek@suse.com \
--cc=richard@nod.at \
--cc=rientjes@google.com \
--cc=robh@kernel.org \
--cc=rostedt@goodmis.org \
--cc=sboyd@kernel.org \
--cc=shuah@kernel.org \
--cc=wfg@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox