From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 817F8C433F5 for ; Thu, 28 Oct 2021 11:59:00 +0000 (UTC) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A5F8A60EBD for ; Thu, 28 Oct 2021 11:58:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A5F8A60EBD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=konsulko.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.denx.de Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 59D61832AF; Thu, 28 Oct 2021 13:58:57 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=konsulko.com header.i=@konsulko.com header.b="QcD6IoeK"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id D4DEF83216; Thu, 28 Oct 2021 13:58:54 +0200 (CEST) Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [IPv6:2607:f8b0:4864:20::72c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id DB23E82F33 for ; Thu, 28 Oct 2021 13:58:50 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=trini@konsulko.com Received: by mail-qk1-x72c.google.com with SMTP id bj31so5470547qkb.2 for ; Thu, 28 Oct 2021 04:58:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=nh9jbXnFD6BJ5zXeIKuO6Ah+cI6+qjjSmaKHUCULTWo=; b=QcD6IoeKPo/IUBOR8PU4QPd1gejMfqgANkXJxiUR8EjVM8bY/edUUtLdedg1/8+xZm KZMRg7oJ0D4ypPBWtWSlWT/vfNwgNSHIG9G4LSQsYgMy3XqtcAV7auknyldhW5KtpXzD qntiIkkPMwxFgvVNf/sV4vPwQ/SsIqfVD6UoA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=nh9jbXnFD6BJ5zXeIKuO6Ah+cI6+qjjSmaKHUCULTWo=; b=acleiFhEZgzM98lTqLjGKnOMmBNnvdITSxJNIS1/giT5sNyorhHWC7Zh2sTPIFDlSg inWxBa/rjMu5BnTrnbusTL5T5lqpQvsv3l7ryxyBwuULkYnKJCKVjEomvviLhIEzhvLQ dUpLa8Z9TEzE2xryzOEcO92Y20JxoPsdxs8Jkg8sSjQ8+CuVHQhbzqvkklfghQBL4c6o c3zEEwxH4KNXsApxclUFlXbNyB3/QdqKSEzMQ5PIjP0oUSFC8kO6XZfP/jWzF1LCfomQ p42JL3Qx8upiEtg8PHsCaPreavBRMJRm+WF0nGXFI22U+h4L8C/P0QxbLB6UR8V6/f3O V+cg== X-Gm-Message-State: AOAM5326NdVvJPl46njGw5YzULfI7zfoVzAs0OC+86nt439MBBoF8ed8 c5rR4xTUnM+ZEhlynx1V7iPoAw== X-Google-Smtp-Source: ABdhPJwFpnLn9RErwysGknRV8Pse9O1x3AntyjMVNdl65tOU07pLF6pyDNL+JToKZM7wYmnvc7aduQ== X-Received: by 2002:a37:b9c3:: with SMTP id j186mr3171528qkf.329.1635422329768; Thu, 28 Oct 2021 04:58:49 -0700 (PDT) Received: from bill-the-cat (2603-6081-7b01-cbda-015b-1704-43dd-8832.res6.spectrum.com. [2603:6081:7b01:cbda:15b:1704:43dd:8832]) by smtp.gmail.com with ESMTPSA id f2sm1750262qtv.45.2021.10.28.04.58.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Oct 2021 04:58:48 -0700 (PDT) Date: Thu, 28 Oct 2021 07:58:47 -0400 From: Tom Rini To: Rasmus Villemoes Cc: Bin Meng , Simon Glass , U-Boot Mailing List Subject: Re: an off-by-one error in dm_test_rtc_set_get()? Message-ID: <20211028115847.GR8284@bill-the-cat> References: <20211027132247.GU8284@bill-the-cat> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="8so8ZvRz607RvwFp" Content-Disposition: inline In-Reply-To: X-Clacks-Overhead: GNU Terry Pratchett X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.2 at phobos.denx.de X-Virus-Status: Clean --8so8ZvRz607RvwFp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 28, 2021 at 09:22:30AM +0200, Rasmus Villemoes wrote: > On 27/10/2021 15.22, Tom Rini wrote: > > On Wed, Oct 27, 2021 at 12:43:38PM +0800, Bin Meng wrote: > >> Hi Simon, > >> > >> gitlab reported the following test error below: > >> > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D FAILURES =3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D > >> __________________________ test_ut[ut_dm_rtc_set_get] ________________= __________ > >> test/py/tests/test_ut.py:43: in test_ut > >> assert output.endswith('Failures: 0') > >> E AssertionError: assert False > >> E + where False =3D >> 0x7f3bb792dcb0>('Failures: 0') > >> E + where = =3D > >> 'Test: dm_test_rtc_set_get: rtc.c\r\r\nexpected: 27/10/2021 > >> 03:38:15\r\r\nactual: 27/10/2021 03:38:14\r\r\ntest/dm/rtc...w, &cmp, > >> 1): Expected 0x0 (0), got 0xffffffea (-22)\r\r\nTest: > >> dm_test_rtc_set_get: rtc.c (flat tree)\r\r\nFailures: 1'.endswith > >> ----------------------------- Captured stdout call -------------------= ---------- > >> =3D> > >> > >> See https://source.denx.de/u-boot/custodians/u-boot-x86/-/jobs/341905 > >> > >> But the same branch same commit, azure test results passed: > >> https://dev.azure.com/bmeng/GitHub/_build/results?buildId=3D460&view= =3Dresults > >> > >> It looks like the error is an off-by-one where actual time is 1 second > >> behind the expected time? > >> > >> expected: 27/10/2021 03:38:15 > >> actual: 27/10/2021 03:38:14 > >> > >> Is this a known issue? > >=20 > > Yes, which is why the test checks for a certain amount of "fuzz" around > > the return value.=20 >=20 > You said the same thing about dm_test_rtc_reset() in > https://lore.kernel.org/u-boot/20210831124441.GC858@bill-the-cat/ , but > I can't find anything about any fuzz in the code. Could you point out > where you think that's implemented? In both cases, the expected and > actual values were just 1 from each other, and I fail to see how any > fuzz value could be smaller than that. >=20 > I've wondered about if we need to increase that value > > slightly sometimes, or just live with hitting the re-run failed jobs > > button on whatever CI system was a bit too slow sometimes. >=20 > It has nothing to do with a CI being slow, it's plain and simple buggy > test code AFAICT. It's essentially "assert(time(NULL) =3D=3D time(NULL))". > If a call to time() takes 1us, do this a million times and it will on > average fail once. Obviously, a loaded system increases the chance of > being preempted between the two calls and hence effectively increases > the delta and proportionally the probability of hitting this. You're right, I confused this with the sleep test, which does have a bit of fuzz to it. --=20 Tom --8so8ZvRz607RvwFp Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEGjx/cOCPqxcHgJu/FHw5/5Y0tywFAmF6kHcACgkQFHw5/5Y0 tyxRjwv9EaBjw9l7f42sAobd0OjY0+kEVlf8IDcNLswaNGrLARt/tfGs8nxA0lPv VPFveBSZYKTFoNNjveTQXJFHmFLeRVRlBMHtAn9tuk94o+b6/avsAIW8xpKT66ag mUDc1rce3vB06IhscdYH0UIwO40XIch4QPEVunJT9g8oqDyjecZL83W2Ywi5cYwf nF5yiRHg/UCAVpiOZgsYpBms2Nzv2j4WYfrEftMP4R5gnn2fLnhaKUJt9YyDPV+0 KHuNovuK2kLBG2AmXkeC14mQ0AtEYPb4F1Q9rAeKJld2+OLlrdSDYUk3wlk8vWjy eewm5+6uHSbla3pRg5qmrYeLJz+Opx2Tj/AlQHyz2H+d6YcRNjKqHnuCogrEHdno nDWF/oTvqkGsa1wO1M0WJhT9gGLGi/AsG6QGB0E73fkA5xhqHTDJ3u04Tw26l4t3 PV8diz0sc1XQxMDsaFpqZV9GHKW+8T9PJM7vhZP++Y+HmHdiDk9ER6s6RLLcnjaC rmbneaE9 =QIpo -----END PGP SIGNATURE----- --8so8ZvRz607RvwFp--