From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79A10C433E6 for ; Thu, 11 Feb 2021 13:02:16 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0085764E79 for ; Thu, 11 Feb 2021 13:02:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0085764E79 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36728 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lABbq-0001os-SS for qemu-devel@archiver.kernel.org; Thu, 11 Feb 2021 08:02:14 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59222) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lABSP-0001q3-I3 for qemu-devel@nongnu.org; Thu, 11 Feb 2021 07:52:29 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:60363) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lABSM-0005M8-0s for qemu-devel@nongnu.org; Thu, 11 Feb 2021 07:52:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1613047944; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SdS0WWVK+aHokvxWveyNRSjm4yVgU3xCbjKn59o7+eQ=; b=hznmrjskrzqsE0jUsVZRnz5GVex4/5FmiWIPyTNCvZ5J9Wf+3iHVyHwd798DlDvHIsbGdn xmc6dOAWaiR4YKn9ETKG/1c7zhBvUeyNRcG9kVSD6/uFm+3lczvF+Bect0Nnc9eE447ZoC O2JHgxsxj1sK7461FxUTFb4P1s3+7+Q= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-186-zVQiq4TsNmS0f8aa39x3iA-1; Thu, 11 Feb 2021 07:52:18 -0500 X-MC-Unique: zVQiq4TsNmS0f8aa39x3iA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7EC751005501 for ; Thu, 11 Feb 2021 12:52:17 +0000 (UTC) Received: from redhat.com (ovpn-114-239.ams2.redhat.com [10.36.114.239]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A5F5760CEA; Thu, 11 Feb 2021 12:52:16 +0000 (UTC) Date: Thu, 11 Feb 2021 12:52:13 +0000 From: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= To: Paolo Bonzini Subject: Re: [PATCH] meson: adjust timeouts for some slower tests Message-ID: <20210211125213.GD1302824@redhat.com> References: <20210209174541.150011-1-pbonzini@redhat.com> <20210209175834.GW1166421@redhat.com> <725dd339-1eee-4791-fda8-2922a5d19a44@redhat.com> MIME-Version: 1.0 In-Reply-To: <725dd339-1eee-4791-fda8-2922a5d19a44@redhat.com> User-Agent: Mutt/1.14.6 (2020-07-11) X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=berrange@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=63.128.21.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -33 X-Spam_score: -3.4 X-Spam_bar: --- X-Spam_report: (-3.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.569, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Thu, Feb 11, 2021 at 12:30:35PM +0100, Paolo Bonzini wrote: > On 09/02/21 18:58, Daniel P. Berrangé wrote: > > On Tue, Feb 09, 2021 at 06:45:41PM +0100, Paolo Bonzini wrote: > > > Adjust the timeouts for the longest running tests. These are the > > > times that I measured and the corresponding timeouts. For generic > > > qtests, the target that reported the longest runtime is included. > > > > > > unit tests: > > > test-crypto-tlscredsx509 13.15s 60s > > > test-crypto-tlssession 14.12s 60s > > > > The default meson timeout is 30 seconds which is enough for these > > tests. Of course larger timeouts give more headroom. > > This was a relatively fast run, I've had them take as little as 7s and as > much as 25s on the same machine. I suspect it's because the machine has a > very slow NFS home directory (yes those things still exist :)). In general a > 2x-ish headroom makes sense in case someone is doing a build at the same > time as a test run. > > By the way, with Meson 0.57 there's the possibility of specifying "infinite > timeout", and this could be used for the benchmarks. Giving slower tests a > higher priority is also a good idea, and even though this is not guaranteed > in theory, Make ends up taking into account the priority as well. With > these tweaks "meson test" and "make check" (minus check-block of course) > both clock at 2:20s, which is exactly the time it takes to run the > longest-running test. > > I will also give "meson test" a shot on the GitLab runners before posting > v2, and see if it needs a timeout multiplier. I'd say we should set the default timeouts so that they are suitable for our automated GitLab CI, and what we expect typical developer platforms. Timeout multiplier should only be needed if users are running on outdated hardware, or under slow emulation. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|