From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55200) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Wkzf9-0004CL-0i for qemu-devel@nongnu.org; Thu, 15 May 2014 13:41:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Wkzf4-0000Y7-GW for qemu-devel@nongnu.org; Thu, 15 May 2014 13:41:46 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38084) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Wkzf4-0000Xz-9M for qemu-devel@nongnu.org; Thu, 15 May 2014 13:41:42 -0400 Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s4FHfeCZ011072 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Thu, 15 May 2014 13:41:41 -0400 Message-ID: <5374FC51.3010109@redhat.com> Date: Thu, 15 May 2014 19:41:37 +0200 From: Max Reitz MIME-Version: 1.0 References: <1399128428-25546-1-git-send-email-mreitz@redhat.com> <87y4y4pmja.fsf@blackfin.pond.sub.org> <5373FF47.7020809@redhat.com> <878uq3jzy7.fsf@blackfin.pond.sub.org> <5374F1CA.2010405@redhat.com> <87r43v0wsk.fsf@blackfin.pond.sub.org> In-Reply-To: <87r43v0wsk.fsf@blackfin.pond.sub.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH] iotests: Use configured python List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Markus Armbruster Cc: Kevin Wolf , qemu-devel@nongnu.org, Stefan Hajnoczi On 15.05.2014 19:35, Markus Armbruster wrote: > Max Reitz writes: > >> On 15.05.2014 08:52, Markus Armbruster wrote: >>> Max Reitz writes: >>> >>>> On 14.05.2014 14:33, Markus Armbruster wrote: >>>>> Max Reitz writes: >>>>> >>>>>> Currently, QEMU's iotests rely on /usr/bin/env to start the correc= t >>>>>> Python (that is, at least Python 2.4, but not 3). On systems where >>>>>> Python 3 is the default, the user has no clean way of making the i= otests >>>>>> use the correct binary. >>>>>> >>>>>> This commit makes the iotests use the Python selected by configure. >>>>>> >>>>>> Signed-off-by: Max Reitz >>>>> I'm afraid this broke qemu-iotests in a separate build tree: >>>>> >>>>> ./check: line 38: ./common.env: No such file or directory >>>> As I already replied to Peter, I see two (or maybe three) ways to fi= x this: >>>> >>>> The first is, we use the correct path to common.env. This would >>>> however result in modification of the source tree although this is >>>> probably not what the user intends with an out-of-tree build. On the >>>> other hand, this would just work. >>> Writing to the source tree works only when the write is exactly the s= ame >>> for every build tree. Example: autoconf writing configure. >>> >>> Modifying files under git-control is right out. >>> >>>> The second is, we do not create common.env for out-of-tree builds an= d >>>> add a default common.env to the repository ("PYTHON =3D python" shou= ld >>>> probably suffice). This would not introduce any regressions, however= , >>>> the iotests would remain problematic on systems with Python 3 being >>>> the default when using out-of-tree builds. >>> A difference between an in-tree and an out-of-tree build is a bug. >>> >>> If plain "python" is good enough for out-of-tree builds, it should be >>> good enough for in-tree builds. >>> >>>> As I guess that out-of-= tree >>>> builds are actually recommended, >>> Correct. >>> >>>> this would mean that the better >>>> solution might be to revert my patch and instead change every "pytho= n" >>>> occurrence in the iotests' Shebangs to "python2", as kind of a third >>>> way to go. However, for this I'm not sure whether all systems which >>>> are supposed to be supported by qemu actually have a "python2" >>>> executable/symlink. I guess so, but then again... >>> I don't know. Try and find out :) >> Okay, so there is a Python Enhancement Proposal, which suggests having >> the symlink python2: >> >> http://legacy.python.org/dev/peps/pep-0394/ >> >> However, at least Debian seems to ignore this (or at least some Debian >> versions do). > Fools :) > >> I think I'll go with Fam's proposal, which is making common.config >> look for python itself, which then can be overwritten by an >> environment variable. > Namely "tell ./check the path to common.env with an env var, like how w= e > tell ./check the path to qemu-img with QEMU_IMG_PROG". I don't like ho= w > we tell check where to find the stuff we build. But you can declare > that a separate problem, and fix your "where is common.env" problem wit= h > the current method, as Fam suggests. > >>>> So, either we fix it but try to write to the source tree without the >>>> user intending to modify it; or we fix it for in-tree builds only; o= r >>>> we drop the magic and just use "python2" in the iotests' Shebangs. >>> The problem is including generated bits, namely results of configure, >>> into source files. >>> >>> The Autoconf way is to substitute placeholders in FOO.in producing FO= O. >>> >>> When you want to limit .in contents as much as possible, you factor o= ut >>> the stuff that needs substitutions into some SUB.in, which you then >>> include into FOO. Requires a suitable include mechanism. In shell, >>> builtin source. >>> >>> But then you need to find SUB from FOO. I've seen two ways used: >>> >>> 1. Assume SUB is in the current directory. Link FOO into the build t= ree >>> to make it so. >>> >>> 2. Require FOO to be run in a way that lets it find its source >>> directory. If whatever runs FOO puts the full path into argv[0], you >>> can use that. Else, require a suitable argument or environment >>> variable. >> Hm, doing this would probably be even more magic than my previous >> patch =E2=80=93 which already had too much magic for myself to handle = and >> therefore broke. I hope finding the correct python to use in >> tests/qemu-iotests/common.config will work out without too much >> hassle. > It's low-level magic at most :) > > The root stupid idea is to run stuff in the source tree. Since a sourc= e > tree can have many build trees, finding the correct build tree can't be > automated. > > If you run stuff in the build tree, there is exactly one source tree, > and putting a pointer to it in the build tree is trivial. In fact, we > already have one: try "make -pn | grep ^SRC_PATH". > > I'd generate a suitable script into the build tree that sets up > necessary variables, then sources $SRC_PATH/qemu-iotests/check. And here I was, just not wanting to do "rm /usr/bin/python; ln -s=20 python2 /usr/bin/python" anymore... Okay, as this coincides mostly with what Peter says, I'll try to allow=20 the iotests to be executed from the build tree (while hopefully keeping=20 compatibility with the current situation). Max