From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFA55C433EF for ; Thu, 28 Oct 2021 07:22:43 +0000 (UTC) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BF8EC60F39 for ; Thu, 28 Oct 2021 07:22:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BF8EC60F39 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=prevas.dk Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.denx.de Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 6E28080615; Thu, 28 Oct 2021 09:22:40 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=reject dis=none) header.from=prevas.dk Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=prevas.dk header.i=@prevas.dk header.b="WXUAD0kp"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 2B81580516; Thu, 28 Oct 2021 09:22:39 +0200 (CEST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2070a.outbound.protection.outlook.com [IPv6:2a01:111:f400:7e1b::70a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 5439380615 for ; Thu, 28 Oct 2021 09:22:34 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=reject dis=none) header.from=prevas.dk Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=rasmus.villemoes@prevas.dk ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ffXiF34RL7baZeb0AvxIhB3xCMlKBW2q+0ydnRZqw+N1OHZn3kAV+TDk5+py12g16+NmtRn9l5BPuV2QOXSAsmqbZ8etpyRTNBBMhOWC5qRVCSDZYCYhIvI19f8B8wbwybFBVtDik4in8jr5YKFN+krPvIiNtWTjJu4MRLhqHkqIQsyBBQD9vLpjR32vY44d4i6Jd163jM0AaN090f2deo8mXSmjI3la/FP66+M2eTWv3zQ7Gcw6UqC52xFblEXnc3qrOhX1QFx14UHpxMMexkkPNXS2cx0KMBlca7YNdIPsBPdChU7aU7Erp78C3wKh+OhIDlekyH2lQB9Cz71yFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IC6Hhvt3a3Er04PoY/4b/ICQt7RUv+3OGlaHyH/e2PQ=; b=JvPfBlbmVSZdAoSB5PWTcia0FRddc8cqhsJPoUeRFqgqT+jozYteHW+Sr5QwnmnWmJvGR7mVrugq2UYIiY3+ZLeuQmCdqqIAjVFX/g+AlzVypAV8R8F9XU11jTxiUm73fjoCnDPBy5ZnJ1E12y/xySlkG27D8+r+AuepgoxTLd2TiHmqYZLoNF0KAdWj44kSLLK8I6dJIq84lrcXqkzit+tqKtkuemspTsXMNI2FrmGlEBVPYYTvzn3zHFms5+DdNipffZdvzdph38F/41gFoAB/K44sQh6uwYLrJhx07oe84Lu7yBiPXNlbcpCBviiHW9hVk96BP14So/oOCUJn6w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=prevas.dk; dmarc=pass action=none header.from=prevas.dk; dkim=pass header.d=prevas.dk; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=prevas.dk; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IC6Hhvt3a3Er04PoY/4b/ICQt7RUv+3OGlaHyH/e2PQ=; b=WXUAD0kpOQ/X9ops31Z76ftnrS3sUpZldJn+uizLcnsuC/LdoghHvo/Z7ZVkk+PM/ssWA/N2L7O5qX+SlQqOCCj/jeSfOxz8r+o09p0+SWcmTZJjJNiu3aNUQqYPKFRYRYb+oOJQ44D3ZQuhnKYm+CqwXiyQzlqQM1VvD4ucuTM= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=prevas.dk; Received: from PR3PR10MB4239.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:96::6) by PR3PR10MB3834.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:2a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4628.20; Thu, 28 Oct 2021 07:22:33 +0000 Received: from PR3PR10MB4239.EURPRD10.PROD.OUTLOOK.COM ([fe80::ac23:766f:c440:5d3a]) by PR3PR10MB4239.EURPRD10.PROD.OUTLOOK.COM ([fe80::ac23:766f:c440:5d3a%6]) with mapi id 15.20.4628.020; Thu, 28 Oct 2021 07:22:32 +0000 Subject: Re: an off-by-one error in dm_test_rtc_set_get()? To: Tom Rini , Bin Meng Cc: Simon Glass , U-Boot Mailing List References: <20211027132247.GU8284@bill-the-cat> From: Rasmus Villemoes Message-ID: Date: Thu, 28 Oct 2021 09:22:30 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 In-Reply-To: <20211027132247.GU8284@bill-the-cat> Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: HE1PR07CA0014.eurprd07.prod.outlook.com (2603:10a6:7:67::24) To PR3PR10MB4239.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:96::6) MIME-Version: 1.0 Received: from [172.16.11.1] (81.216.59.226) by HE1PR07CA0014.eurprd07.prod.outlook.com (2603:10a6:7:67::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4649.10 via Frontend Transport; Thu, 28 Oct 2021 07:22:32 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3d9dc46f-0c0b-410d-8e0d-08d999e3b52e X-MS-TrafficTypeDiagnostic: PR3PR10MB3834: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3044; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KT3VWngsQHRBleikga65jRob721VJkZryJc9T9NF5J1VjJS8yO5evWr0MLALEwkJFR7neEtrgNWmw3mnBTli7R4zhxB4WiC/m5Q+LvnrMir/IITQvK2pz+04sPJwCaFtbItMXaDKAZw6eaFVb1PCp0hsPncEjrzCajDtmq748Kj9qdSYoc9mBQu7Lx6TiG3ncvO0meQcb9DH+Ym4a+fUVOwX2t3WQZt49cm303CreL93lVrDviBCLP0xN5iOjA5Il1HzOmK8+fSyIXdBl1JQ8bhwWnfp5B453NzZMZnw2IuMAZ4GRlsoy2jIXN3Rt51wqfTu9Xj6AgeFZFgBEj37BD5F2Uss4PLuC5qVkcDUdtnpP/zU5QxOlvMtYgmj5bDH7zbBoDy/N7hv2vt6ds5NTO9NDBj2qvANobbnXG77BsI7BRNw6/HaKmM1IL9udbDSxEE59O5EdxknOg//AtJuAx0phi5LK6s1C7S2rccJ2L/Wzno+9QlC8GrZOOHpPKrsB4/9TO6SXyOTSIhJ+VVGTrnLBNjQ/A2yGnDiUUw6V6h7hnj2Evl06acuB0vUhZL1fSdoibsvThbiqwP87h10YqByGOQpxYf572KJySdIEMM91niJiyw8gYAgYHDRUKqJcT/paiuYS17yFfilKIbs1fOzkGc+/y3RvkfQxOHbIi3YWm2xYm+9/SNDz0DoWa0sqHS1h2MDxN26HQlTGSB6sh1UO6HZ2I/nMZQ4K990zEnVrwNOp61Tb26g2Wx6dqnnJYssdL4123ubKb44VhI3b1yRD6HxYmWqwTvb13nAQ3BCgDlex+GmvsjZerC4V6ufFEEgL8Ylmik5ByxMF6ieaA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PR3PR10MB4239.EURPRD10.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(366004)(8976002)(2906002)(8936002)(31696002)(6486002)(956004)(38350700002)(16576012)(31686004)(316002)(52116002)(54906003)(4326008)(8676002)(508600001)(2616005)(5660300002)(66476007)(38100700002)(186003)(66946007)(83380400001)(26005)(966005)(44832011)(86362001)(36756003)(110136005)(66556008)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?scEz8XthEWGdsXtLRKeBgIPwXWStDUFTfVTsHeIKvR9cxLjNRZcEXkjm?= =?Windows-1252?Q?d+0GSGNYJlatx29Wldpoy9aYvhQ8hiWody5WiQfq9bcmCwnI0ETAhwon?= =?Windows-1252?Q?W1BAmI3vG4nx56byK+YBZDfDT5mMPVOubH3aR+YoqPBqH7QOKUoDuls/?= =?Windows-1252?Q?TIo1g9gzZMmbmQCm49Rlr4sLI97zSBZH0X/tqSpHX1eQtj4DinvsIAED?= =?Windows-1252?Q?U8wM/tbI5tgaPKS6mhc9Ips1/0qBChhUs+XBSLjRi/7nelgbgQVVKt8z?= =?Windows-1252?Q?DVCO44czVPk7+QbRQww1goWEPnsFBYpDIEcq2PvdtcymwHOBGXVVPrhz?= =?Windows-1252?Q?PdcVBiXeKfGUF2KvEex6bRll1lVpzKJ0nNvP097mkfTfhKf1yUsdE0yy?= =?Windows-1252?Q?G5cJvoWwz9i2KKc1SIY8ZPTn4onrOO+U6SQAaaxWvTzPVjCzz5evLKNA?= =?Windows-1252?Q?a36h9q6WWkFGxZSqd46oyD81qEwzvdf3ImHV+8tSl2KNuoeWMzVlJCFd?= =?Windows-1252?Q?DkQR8DfPWNkiuS48uW6P5s1i++mwVj6CODSjmymG9vHNVOrBuM0pVh9j?= =?Windows-1252?Q?mrV5LJeifSLLtE71Cg+T4yXjgH0TgCHE1xmLqafowfBcZESFZ+JGR+XO?= =?Windows-1252?Q?d1NmShHf3nMhWJZQtHxlxXUMSsCHLlsjRiohcbZnxiOGKd9elEJnwEfO?= =?Windows-1252?Q?ykGnFcdO1TZ5zDBC+Brduoh3NFQwcdupqh9C+FJaBMSRdiNht0naM9V0?= =?Windows-1252?Q?oQKuVDP/hw6px41EmPBt9Avrxz1nJHDsQqM7jX6b8zZVFWWoKj3IRz44?= =?Windows-1252?Q?zuOyPQQuzAtVfFTL9IqSjX90JQAftTS/vZifNdwSa/1Z2UJgrIkXEYLE?= =?Windows-1252?Q?KH8AvOELNbB4H3Oc/BY5WAlrfX5Z3ahIZyvLhqNEHbOij6H9Yo2D+JBc?= =?Windows-1252?Q?NQUe4Y64u+kNYgzBBgeMBckyy/8vGgNAcHgDL8IeWFi4746c9wdObCMQ?= =?Windows-1252?Q?XRV4h+jsae/SiS/aESy+1Rt6U6FhU1zw9eL/h1I1EyvhDt9iRCjGdPOd?= =?Windows-1252?Q?YxCPUzAn8ByugeU06U0GWu0Gr2eOfbKDACgKDNslaWl2iFi3G49NOkuw?= =?Windows-1252?Q?dDLAtwFOoIXzH/9WCHJ99/qDz7XZOD9jZojKcsUzt6tmi1uogWt6am7n?= =?Windows-1252?Q?fAZbFxEWwF6Y7/biLRIyzv9V01erDBrW/lZfH6Zr5A/dkdub2f3xma7x?= =?Windows-1252?Q?oqggPb0E4qXpVbiiEYcnfOoarv3G0uhE+0nUQXbnrSlJ3hJLZS3Q063B?= =?Windows-1252?Q?1xa3FmdW/9ES4ggU8GgiIboSN1hNjpIm00ELJeqwI5kr7dGjLj/Zyge9?= =?Windows-1252?Q?IAn71K8joFFw4xoahm90BvUaEiBvEQV8pIDo76LzPsFoiS2w7VOyVWp5?= =?Windows-1252?Q?7hoaomUewF36y9Y6/0Q+cLd50bloOJ+29DqhnVdHcmCrAdb2lltHCMsx?= =?Windows-1252?Q?mJpbCGrwsib8/dI6xSxyqwCLEVQLQTT0r1sNHtox5x3XOnoxbbf5jh3v?= =?Windows-1252?Q?C6jtgFc6Q0SmIcD/Hi+6g4BWko/qm2rj/EGcCgpSaMg708ugnmCNRmad?= =?Windows-1252?Q?iZBHeXDeT4seV8VyfhV4rnt/4NnARS+KE9aN2HgUmqVSUHsn50ogR3vv?= =?Windows-1252?Q?GQ/4HxifgHn8MKXFn8eX94ygNpNYLBX9P5miShKMxip12dhs9lzzSJYa?= =?Windows-1252?Q?bEnwTemeiF3TyhPkLCChgRjBRTRycK0LGdhetRjp+5emGwoAVsvIhy3S?= =?Windows-1252?Q?cl//XQ=3D=3D?= X-OriginatorOrg: prevas.dk X-MS-Exchange-CrossTenant-Network-Message-Id: 3d9dc46f-0c0b-410d-8e0d-08d999e3b52e X-MS-Exchange-CrossTenant-AuthSource: PR3PR10MB4239.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Oct 2021 07:22:32.9005 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: d350cf71-778d-4780-88f5-071a4cb1ed61 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2RSHSd2yKri4auGhN+Eb3/3zQssLY66EI8J7ZlkTDXKnVlNr5rrScnBVSRVa5RPbunMEPdkGMisJ8y5lSFr+stL8cuTGJHs3pla0vx6DWcQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR10MB3834 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.2 at phobos.denx.de X-Virus-Status: Clean On 27/10/2021 15.22, Tom Rini wrote: > On Wed, Oct 27, 2021 at 12:43:38PM +0800, Bin Meng wrote: >> Hi Simon, >> >> gitlab reported the following test error below: >> >> =================================== FAILURES =================================== >> __________________________ test_ut[ut_dm_rtc_set_get] __________________________ >> test/py/tests/test_ut.py:43: in test_ut >> assert output.endswith('Failures: 0') >> E AssertionError: assert False >> E + where False = > 0x7f3bb792dcb0>('Failures: 0') >> E + where = >> 'Test: dm_test_rtc_set_get: rtc.c\r\r\nexpected: 27/10/2021 >> 03:38:15\r\r\nactual: 27/10/2021 03:38:14\r\r\ntest/dm/rtc...w, &cmp, >> 1): Expected 0x0 (0), got 0xffffffea (-22)\r\r\nTest: >> dm_test_rtc_set_get: rtc.c (flat tree)\r\r\nFailures: 1'.endswith >> ----------------------------- Captured stdout call ----------------------------- >> => >> >> See https://source.denx.de/u-boot/custodians/u-boot-x86/-/jobs/341905 >> >> But the same branch same commit, azure test results passed: >> https://dev.azure.com/bmeng/GitHub/_build/results?buildId=460&view=results >> >> It looks like the error is an off-by-one where actual time is 1 second >> behind the expected time? >> >> expected: 27/10/2021 03:38:15 >> actual: 27/10/2021 03:38:14 >> >> Is this a known issue? > > Yes, which is why the test checks for a certain amount of "fuzz" around > the return value. You said the same thing about dm_test_rtc_reset() in https://lore.kernel.org/u-boot/20210831124441.GC858@bill-the-cat/ , but I can't find anything about any fuzz in the code. Could you point out where you think that's implemented? In both cases, the expected and actual values were just 1 from each other, and I fail to see how any fuzz value could be smaller than that. I've wondered about if we need to increase that value > slightly sometimes, or just live with hitting the re-run failed jobs > button on whatever CI system was a bit too slow sometimes. It has nothing to do with a CI being slow, it's plain and simple buggy test code AFAICT. It's essentially "assert(time(NULL) == time(NULL))". If a call to time() takes 1us, do this a million times and it will on average fail once. Obviously, a loaded system increases the chance of being preempted between the two calls and hence effectively increases the delta and proportionally the probability of hitting this. Rasmus