Re: [PATCH v6 12/12] sandbox: add test of wdt-uclass' watchdog_reset()

[Tom Rini <trini@konsulko.com>]

[-- Attachment #1: Type: text/plain, Size: 2081 bytes --]

On Tue, Aug 31, 2021 at 11:29:51AM +0200, Rasmus Villemoes wrote:
> On 31/08/2021 10.17, Stefan Roese wrote:
> > Hi Rasmus,
> > 
> > I've pulled this patchset now into next [1] and have run it through
> > CI via Azure. Here an error occurs:
> > 
> > https://dev.azure.com/sr0718/u-boot/_build/results?buildId=109&view=logs&j=50449d1b-398e-53ae-48fa-6bf338edeb51&t=97605dd2-f5a5-5dd7-2118-315ffdc8bcd6&l=533
> > 
> > 
> > Could you please take a look at this?
> 
> I'm pretty confident that has nothing to do with my patches, but is a
> broken (or, to put it more gently, fragile) test.
> 
> It does
> 
> // fetch the emulated device's current base_time value, setting it to 0
>         old_base_time = sandbox_i2c_rtc_get_set_base_time(emul, 0);
> 
> // fetch the emuluated device's current base_time value, don't (-1) set
> // a new value, check that we got 0
>         ut_asserteq(0, sandbox_i2c_rtc_get_set_base_time(emul, -1));
> 
> // call the device's ->reset method
>         /* Resetting the RTC should put he base time back to normal */
>         ut_assertok(dm_rtc_reset(dev));
> // fetch the new base_time value, without altering it
>         base_time = sandbox_i2c_rtc_get_set_base_time(emul, -1);
> // and check that the base time was put back to "normal"
>         ut_asserteq(old_base_time, base_time);
> 
> The thing is, the ->reset method does
> 
> static void reset_time(struct udevice *dev)
> {
>         struct sandbox_i2c_rtc_plat_data *plat = dev_get_plat(dev);
>         struct rtc_time now;
> 
>         os_localtime(&now);
>         plat->base_time = rtc_mktime(&now);
> 
> It's inevitable that this will cause occasional spurious failures. I can
> trivially reproduce it with
> 
> => while ut dm rtc_reset ; do echo . ; done
> 
> it fails after a few screenfuls of successes.

Yes, this test fails sometimes and just needs to be re-run.  Currently
we even have code in the test framework to allow for a little bit of
wiggle room in the value, but perhaps it needs to be bumped up slightly.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]