From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stephen Warren Date: Thu, 28 Jan 2016 22:08:22 -0700 Subject: [U-Boot] [PATCH] test/py: make each unit test a pytest In-Reply-To: References: <1454024708-867-1-git-send-email-swarren@wwwdotorg.org> Message-ID: <56AAF3C6.1030605@wwwdotorg.org> List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: u-boot@lists.denx.de On 01/28/2016 08:52 PM, Simon Glass wrote: > Hi Stephen, > > On 28 January 2016 at 16:45, Stephen Warren wrote: >> From: Stephen Warren >> >> A custom fixture named ut_subtest is implemented which is parametrized >> with the names of all unit tests that the U-Boot binary supports. This >> causes each U-Boot unit test to be exposes as a separate pytest. In turn, >> this allows more fine-grained pass/fail counts and test selection, e.g.: >> >> test.py --bd sandbox -k ut_dm_usb >> >> ... will run about 8 tests at present. >> >> Signed-off-by: Stephen Warren >> --- >> This depends on at least my recently sent "test/py: run C-based unit tests". >> >> test/py/conftest.py | 105 ++++++++++++++++++++++++++++++++++++----------- >> test/py/tests/test_ut.py | 14 +++---- >> 2 files changed, 86 insertions(+), 33 deletions(-) > > This seems a bit extreme. It might be better to move the remaining > three commands under the 'ut' subcommand. Then all unit tests would be > visible from the 'ut' help... I'm not sure what you mean by "extreme"? Do you mean you don't want each unit test exposed as a separate pytest? I thought based on our previous conversation that was exactly what you wanted. If not, I'm not sure what the deficiency in the current code is; either all the dm subtests are executed at once by a single pytest with a single overall status, or they're each a separate pytest with individual status. Any grouping that's in between those seems like it would be entirely arbitrary?