From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D1F92C43334 for ; Tue, 28 Jun 2022 10:25:32 +0000 (UTC) Received: from localhost ([::1]:36118 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o68PO-0007Qi-Ft for qemu-devel@archiver.kernel.org; Tue, 28 Jun 2022 06:25:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51252) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o68Ia-0002CM-DS for qemu-devel@nongnu.org; Tue, 28 Jun 2022 06:18:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:24087) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o68IY-00083g-IU for qemu-devel@nongnu.org; Tue, 28 Jun 2022 06:18:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1656411502; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rtHwnxOU8AmO3twewB07+fmbEiHV0yviD3yCEj+OaVI=; b=MiHnetn8IkOpINbZmTnMqxfW51QyJNfe0RTM9vAcv6pGENOzhan/XZowTJeByRnu+EBK6i d8DykeEss/m9CRmx5Bpc9wNIDbNrVUiod6EpgGVwKwfoxnwO7Qglg4G24p1UFnZdPAGd8b mPaSpapgsK2LjJqui2Nh0jN8eTnVFLk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-175-zRHJjfNZO42WeY2VcwINjg-1; Tue, 28 Jun 2022 06:18:18 -0400 X-MC-Unique: zRHJjfNZO42WeY2VcwINjg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 952382999B4F; Tue, 28 Jun 2022 10:18:18 +0000 (UTC) Received: from redhat.com (unknown [10.33.36.4]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0F5F79D54; Tue, 28 Jun 2022 10:18:16 +0000 (UTC) Date: Tue, 28 Jun 2022 11:18:14 +0100 From: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= To: "Michael S. Tsirkin" Cc: Ani Sinha , Thomas Huth , John Snow , qemu-devel@nongnu.org, Laurent Vivier , Paolo Bonzini , imammedo@redhat.com Subject: Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests) Message-ID: References: <4e1c2a45-eb53-e210-1ce1-05837bf1e7c3@redhat.com> <20220628030749-mutt-send-email-mst@kernel.org> <7bf5976e-8277-7c78-f412-44f7be8754f4@redhat.com> <59150265-44ed-0b14-df1c-42e3f2e97b7e@redhat.com> <20220628060510-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20220628060510-mutt-send-email-mst@kernel.org> User-Agent: Mutt/2.2.6 (2022-06-05) X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Tue, Jun 28, 2022 at 06:07:13AM -0400, Michael S. Tsirkin wrote: > On Tue, Jun 28, 2022 at 11:04:30AM +0100, Daniel P. Berrangé wrote: > > If it is actually booting a real guest image (from biosbits) and interacting > > with it, then it does feel like the scope of this testing is more appropriate > > to QEMU's avocado framework than qtest, especially given the desire to use > > python for it all. > > > > With regards, > > Daniel > > I feel avocado is directed towards booting full fledged guest OS. That's essentially what this is doing - its a custom guest OS rather than a common distro IIUC > It makes it much easier to figure out guest issues but it also > prone to false positives and is harder to debug as a result. > Booting a minimal image like this shouldn't require that. Well avocado is as reliable as the tests you write for it. The problems are largely around the images being used in avocado. If the biosbits testing system is reliable, then avocado will be too, and if they not reliable, then it will affect qtest too. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|