Re: [PATCH 5/8] oeqa/sdkext/devtool.py: don't reset when the test is failed

[Robert Yang <liezhi.yang@windriver.com>]

On 12/13/2016 02:33 PM, Paul Eggleton wrote:
> On Tue, 13 Dec 2016 14:01:08 Robert Yang wrote:
>> On 12/13/2016 12:48 PM, Paul Eggleton wrote:
>>> On Wed, 16 Nov 2016 22:19:34 Robert Yang wrote:
>>>> The contents are helpful to debug when the error happens.
>>>
>>> In this case removing this is OK because we're operating on an eSDK that
>>> the tests install. However, this wouldn't be appropriate for the devtool
>>> oe- selftest tests since they are operating on the user's build system
>>> itself - not that you probably have any intention of changing those, I
>>> just wanted to note that.
>>
>> Sorry, I don't quite understand, does selftest uses oeqa/sdkext/devtool.py,
>> please ?
>
> No, this is for bitbake -c testsdkext.
>
>> Even if it uses devtool.py, I still think that we need keep the
>> error contents for debugging when the error happens. It's very hard to
>> debug when there is no error log or no workspace.
>
> I mean the equivalent oe-selftest tests in oeqa/selftest/devtool.py.
>
> Actually, I've realised something that applies here as well - leaving the
> files around would be OK if we stopped on the first failure, but we don't -
> the next test proceeds after it. How will you be able to rely on what's in the
> workspace if several other tests have run afterwards - not to mention ensure
> those tests aren't disrupted by the leftover files?

That's a problem, but we really need consider debugging, otherwise it's
painful when test are failed but nothing left. How about run test cases
in different workspaces ?

// Robert

>
> Cheers,
> Paul
>