Re: [Qemu-devel] [PULL 19/19] configure: automatically pick python3 is available

["Daniel P. Berrangé" <berrange@redhat.com>]

On Fri, May 03, 2019 at 05:54:35PM +0100, Daniel P. Berrangé wrote:
> On Fri, May 03, 2019 at 06:41:43PM +0200, Thomas Huth wrote:
> > On 03/05/2019 02.41, Eduardo Habkost wrote:
> > > From: Daniel P. Berrangé <berrange@redhat.com>
> > > 
> > > Unless overridden via an env var or configure arg, QEMU will only look
> > > for the 'python' binary in $PATH. This is unhelpful on distros which
> > > are only shipping Python 3.x (eg Fedora) in their default install as,
> > > if they comply with PEP 394, the bare 'python' binary won't exist.
> > > 
> > > This changes configure so that by default it will search for all three
> > > common python binaries, preferring to find Python 3.x versions.
> > > 
> > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > > Message-Id: <20190327170701.23798-1-berrange@redhat.com>
> > > Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> > > ---
> > >  configure | 18 +++++++++++++++---
> > >  1 file changed, 15 insertions(+), 3 deletions(-)
> > 
> > I haven't bisected it, but I think this patch here broke the gitlab-ci tests:
> > 
> >  https://gitlab.com/huth/qemu/-/jobs/206806257
> > 
> > Seems like the test is now failing when you don't have an UTF-8 locale:
> > 
> >  LANG=C make check-qapi-schema
> >  [...]
> >  TEST    tests/qapi-schema/union-base-empty.out
> >  --- /builds/huth/qemu/tests/qapi-schema/unicode-str.err	2019-05-03 15:21:39.000000000 +0000
> >  +++ -	2019-05-03 15:42:01.561762978 +0000
> >  @@ -1 +1 @@
> >  -tests/qapi-schema/unicode-str.json:2: 'command' uses invalid name 'é'
> >  +tests/qapi-schema/unicode-str.json:2: 'command' uses invalid name '\xe9'
> >  /builds/huth/qemu/tests/Makefile.include:1105: recipe for target 'check-tests/qapi-schema/unicode-str.json' failed
> >  make: *** [check-tests/qapi-schema/unicode-str.json] Error 1
> > 
> > Any ideas how to fix this?
> 
> python3 is basically doomed if you use the C locale for LC_CTYPE, as
> it is not 8-bit clean.
> 
> If a python3 program is liable to see UTF-8 input data, the following
> env should generally be set when running python:
> 
>    LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8

Oh, actually I forgot we did that and then changed approach in QEMU,
see these:

commit 0d6b93deeeb3cc190692d629f5927befdc8b1fb8
Author: Matthias Maier <tamiko@43-1.org>
Date:   Mon Jun 18 19:59:58 2018 +0200

    Revert commit d4e5ec877ca
    
    This commit removes the PYTHON_UTF8 workaround. The problem with setting
    
      LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8
    
    is that the en_US.UTF-8 locale might not be available. In this case
    setting above locales results in build errors even though another UTF-8
    locale was originally set [1]. The only stable way of fixing the
    encoding problem is by specifying the encoding in Python, like the
    previous commit does.
    
    [1] https://bugs.gentoo.org/657766
    
    Signed-off-by: Arfrever Frehtes Taifersar Arahesis <arfrever.fta@gmail.com>
    Signed-off-by: Matthias Maier <tamiko@43-1.org>
    Message-Id: <20180618175958.29073-3-armbru@redhat.com>
    Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
    Reviewed-by: Eric Blake <eblake@redhat.com>
    [Commit message tweaked]
    Signed-off-by: Markus Armbruster <armbru@redhat.com>

commit de685ae5e9a4b523513033bd6cadc8187a227170
Author: Markus Armbruster <armbru@redhat.com>
Date:   Mon Jun 18 19:59:57 2018 +0200

    qapi: Open files with encoding='utf-8'
    
    Python 2 happily reads UTF-8 files in text mode, but Python 3 requires
    either UTF-8 locale or an explicit encoding passed to open().  Commit
    d4e5ec877ca fixed this by setting the en_US.UTF-8 locale.  Falls apart
    when the locale isn't be available.
    
    Matthias Maier and Arfrever Frehtes Taifersar Arahesis proposed to use
    binary mode instead, with manual conversion from bytes to str.  Works,
    but opening with an explicit encoding is simpler, so do that.
    
    Since Python 2's open() doesn't support the encoding parameter, we
    need to suppress it with a version check.
    
    Reported-by: Arfrever Frehtes Taifersar Arahesis <arfrever.fta@gmail.com>
    Reported-by: Matthias Maier <tamiko@43-1.org>
    Signed-off-by: Markus Armbruster <armbru@redhat.com>
    Message-Id: <20180618175958.29073-2-armbru@redhat.com>
    Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
    Reviewed-by: Eric Blake <eblake@redhat.com>


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|