From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69FFBC433F5 for ; Thu, 19 May 2022 08:12:39 +0000 (UTC) Received: from localhost ([::1]:34114 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrbGw-0001D8-FV for qemu-devel@archiver.kernel.org; Thu, 19 May 2022 04:12:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53762) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrazw-0001IO-Rq for qemu-devel@nongnu.org; Thu, 19 May 2022 03:55:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:37036) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrazv-0002Ia-D0 for qemu-devel@nongnu.org; Thu, 19 May 2022 03:55:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652946902; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=9mr7ZOLartBOG934gpd+VKWj/Ps+ZhokQq2B9+VVkw0=; b=Ci1gbH0/94eMvEa9YICclHxLxkNJWdGTzuQAf8aq9X9hYpHCWO4A74AyAcFAjZUcaF+5+g cACTBdHw6Vq1YO5hemtjlQIqmU5IsZ0qEQukxLZatTYaXDY1WX5sQDrmxeFD2R6fqWI72c 73ScI8cbg5xWrF4jqcIhpSGbbLbzAQI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-178-0URUsD8pMhyDbKnTPH6RxA-1; Thu, 19 May 2022 03:54:59 -0400 X-MC-Unique: 0URUsD8pMhyDbKnTPH6RxA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E758685A5BE; Thu, 19 May 2022 07:54:58 +0000 (UTC) Received: from redhat.com (unknown [10.39.193.191]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1826C2026D6A; Thu, 19 May 2022 07:54:57 +0000 (UTC) Date: Thu, 19 May 2022 09:54:56 +0200 From: Kevin Wolf To: John Snow Cc: Hanna Reitz , qemu-devel , Qemu-block , Paolo Bonzini Subject: Re: The fate of iotest 297 Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Am 18.05.2022 um 20:21 hat John Snow geschrieben: > To wire it up to "make check" by *default*, I believe I need to expand the > configure script to poll for certain requisites and then create some > wrapper script of some kind that only engages the python tests if the > requisites were met ... and I lose some control over the mypy/pylint > versioning windows. I have to tolerate a wider versioning, or it'll never > get run in practice. > > I have some reluctance to doing this, because pylint and mypy change so > frequently that I don't want "make check" to fail spuriously in the future. > > (In practice, these failures occur 100% of the time when I am on vacation.) So we seem to agree that it's something that we do expect to fail from time to time. Maybe this is how I could express my point better: If it's a hard failure, it should fail as early as possible - i.e. ideally before the developer sends a patch, but certainly before failing a pull request. This allows another option that would work for me (but probably not for you): Don't make it a hard failure. In practice, it would probably mean that you end up fixing up things after people like we sometimes have to do for the non-auto iotests. > That said ... maybe I can add a controlled venv version of "check-python" > and just have a --disable-check-python or something that spec files can opt > into. Maybe that will work well enough? > > i.e. maybe configure can check for the presence of pip, the python venv > module (debian doesnt ship it standard...), and PyPI connectivity and if > so, enables the test. Otherwise, we skip it. I think this should work. If detecting the right environment is hard, I don't think there is even a requirement to do so. You can make --enable-check-python the default and if people don't want it, they can explicitly disable it. (I understand that until you run 'make check', it doesn't make a difference anyway, so pure users would never have to change the option, right?) > Got it. I'll see what I can come up with that checks the boxes for > everyone, thanks for clarifying yours. > > I want to make everything "just work" but I'm also afraid of writing too > much magic crap that could break and frustrate people who have no desire to > understand python packaging junk, so I'm trying to balance that. Yes, sounds like we need to find some balance there. Test infrastructure breaking locally for no obvious reason can be quite frustrating. But sending a patch and getting it queued, only to be notified that it's dropped again because of a mypy problem two weeks later when the maintainer sends the pull request, can be equally (if not even more) frustrating. Kevin