[XEN PATCH v1] tools/python: Add python3 compatibility

[XEN PATCH v1] tools/python: Add python3 compatibility

[Javi Merino]

Most of the work for python3 compatibility was done in
1430c5a8cad4 (tools/python: Python 3 compatibility, 2019-12-18).  This
just adds a few builtins needed for python3.

Resolves: xen-project/xen#156

Signed-off-by: Javi Merino <javi.merino@cloud.com>
---

I haven't tested it.

 README                                     | 1 +
 tools/python/scripts/convert-legacy-stream | 3 +++
 2 files changed, 4 insertions(+)

diff --git a/README b/README
index 855db01d36..44ed88c392 100644
--- a/README
+++ b/README
@@ -51,6 +51,7 @@ provided by your OS distributor:
     * POSIX compatible awk
     * Development install of zlib (e.g., zlib-dev)
     * Development install of Python 2.7 or later (e.g., python-dev)
+      - If using Python 2.7, you also need the future module.  This is not needed for Python 3.
     * Development install of curses (e.g., libncurses-dev)
     * Development install of openssl (e.g., openssl-dev)
     * Development install of x11 (e.g. xorg-x11-dev)
diff --git a/tools/python/scripts/convert-legacy-stream b/tools/python/scripts/convert-legacy-stream
index 7fe375a668..26a66c50fc 100755
--- a/tools/python/scripts/convert-legacy-stream
+++ b/tools/python/scripts/convert-legacy-stream
@@ -8,6 +8,9 @@ Convert a legacy migration stream to a v2 stream.
 from __future__ import print_function
 from __future__ import division
 
+from builtins import zip
+from builtins import range
+from builtins import object
 import sys
 import os, os.path
 import syslog
-- 
2.42.0

Re: [XEN PATCH v1] tools/python: Add python3 compatibility

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 1877 bytes --]

On Tue, Oct 10, 2023 at 03:18:45PM +0100, Javi Merino wrote:
> Most of the work for python3 compatibility was done in
> 1430c5a8cad4 (tools/python: Python 3 compatibility, 2019-12-18).  This
> just adds a few builtins needed for python3.
> 
> Resolves: xen-project/xen#156
> 
> Signed-off-by: Javi Merino <javi.merino@cloud.com>
> ---
> 
> I haven't tested it.
> 
>  README                                     | 1 +
>  tools/python/scripts/convert-legacy-stream | 3 +++
>  2 files changed, 4 insertions(+)
> 
> diff --git a/README b/README
> index 855db01d36..44ed88c392 100644
> --- a/README
> +++ b/README
> @@ -51,6 +51,7 @@ provided by your OS distributor:
>      * POSIX compatible awk
>      * Development install of zlib (e.g., zlib-dev)
>      * Development install of Python 2.7 or later (e.g., python-dev)
> +      - If using Python 2.7, you also need the future module.  This is not needed for Python 3.
>      * Development install of curses (e.g., libncurses-dev)
>      * Development install of openssl (e.g., openssl-dev)
>      * Development install of x11 (e.g. xorg-x11-dev)
> diff --git a/tools/python/scripts/convert-legacy-stream b/tools/python/scripts/convert-legacy-stream
> index 7fe375a668..26a66c50fc 100755
> --- a/tools/python/scripts/convert-legacy-stream
> +++ b/tools/python/scripts/convert-legacy-stream
> @@ -8,6 +8,9 @@ Convert a legacy migration stream to a v2 stream.
>  from __future__ import print_function
>  from __future__ import division
>  
> +from builtins import zip
> +from builtins import range
> +from builtins import object

In which python version it's needed? The thing about builtins is they
are available without explicit import.

>  import sys
>  import os, os.path
>  import syslog
> -- 
> 2.42.0
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [XEN PATCH v1] tools/python: Add python3 compatibility

[Javi Merino]

On Tue, Oct 10, 2023 at 05:27:03PM +0200, Marek Marczykowski-Górecki wrote:
> On Tue, Oct 10, 2023 at 03:18:45PM +0100, Javi Merino wrote:
> > Most of the work for python3 compatibility was done in
> > 1430c5a8cad4 (tools/python: Python 3 compatibility, 2019-12-18).  This
> > just adds a few builtins needed for python3.
> > 
> > Resolves: xen-project/xen#156
> > 
> > Signed-off-by: Javi Merino <javi.merino@cloud.com>
> > ---
> > 
> > I haven't tested it.
> > 
> >  README                                     | 1 +
> >  tools/python/scripts/convert-legacy-stream | 3 +++
> >  2 files changed, 4 insertions(+)
> > 
> > diff --git a/README b/README
> > index 855db01d36..44ed88c392 100644
> > --- a/README
> > +++ b/README
> > @@ -51,6 +51,7 @@ provided by your OS distributor:
> >      * POSIX compatible awk
> >      * Development install of zlib (e.g., zlib-dev)
> >      * Development install of Python 2.7 or later (e.g., python-dev)
> > +      - If using Python 2.7, you also need the future module.  This is not needed for Python 3.
> >      * Development install of curses (e.g., libncurses-dev)
> >      * Development install of openssl (e.g., openssl-dev)
> >      * Development install of x11 (e.g. xorg-x11-dev)
> > diff --git a/tools/python/scripts/convert-legacy-stream b/tools/python/scripts/convert-legacy-stream
> > index 7fe375a668..26a66c50fc 100755
> > --- a/tools/python/scripts/convert-legacy-stream
> > +++ b/tools/python/scripts/convert-legacy-stream
> > @@ -8,6 +8,9 @@ Convert a legacy migration stream to a v2 stream.
> >  from __future__ import print_function
> >  from __future__ import division
> >  
> > +from builtins import zip
> > +from builtins import range
> > +from builtins import object
> 
> In which python version it's needed? The thing about builtins is they
> are available without explicit import.

In python3, this change is a noop.  In python2, this change makes zip,
range and object be consistent with its python3 counterparts:

- zip: with the change, zip returns an iterator in python2,
  like it does in python3.  It's equivalent to izip from itertools or
  `from future_builtins import zip`[0]

- range: with the change, in python2 range behaves like xrange() used
  to. It returns an object instead of returning the list.[1]

- object: with the change, in python2 the string representation of
  VM() is in unicode, as it is for python3.

[0] https://stackless.readthedocs.io/en/2.7-slp/library/future_builtins.html#future_builtins.zip
[1] https://docs.python.org/3/library/stdtypes.html#ranges

Cheers,
Javi

Re: [XEN PATCH v1] tools/python: Add python3 compatibility

[Andrew Cooper]

On 10/10/2023 10:18 pm, Javi Merino wrote:
> Most of the work for python3 compatibility was done in
> 1430c5a8cad4 (tools/python: Python 3 compatibility, 2019-12-18).  This
> just adds a few builtins needed for python3.
>
> Resolves: xen-project/xen#156
>
> Signed-off-by: Javi Merino <javi.merino@cloud.com>
> ---
>
> I haven't tested it.
>
>  README                                     | 1 +
>  tools/python/scripts/convert-legacy-stream | 3 +++
>  2 files changed, 4 insertions(+)
>
> diff --git a/README b/README
> index 855db01d36..44ed88c392 100644
> --- a/README
> +++ b/README
> @@ -51,6 +51,7 @@ provided by your OS distributor:
>      * POSIX compatible awk
>      * Development install of zlib (e.g., zlib-dev)
>      * Development install of Python 2.7 or later (e.g., python-dev)
> +      - If using Python 2.7, you also need the future module.  This is not needed for Python 3.
>      * Development install of curses (e.g., libncurses-dev)
>      * Development install of openssl (e.g., openssl-dev)
>      * Development install of x11 (e.g. xorg-x11-dev)
> diff --git a/tools/python/scripts/convert-legacy-stream b/tools/python/scripts/convert-legacy-stream
> index 7fe375a668..26a66c50fc 100755
> --- a/tools/python/scripts/convert-legacy-stream
> +++ b/tools/python/scripts/convert-legacy-stream
> @@ -8,6 +8,9 @@ Convert a legacy migration stream to a v2 stream.
>  from __future__ import print_function
>  from __future__ import division
>  
> +from builtins import zip
> +from builtins import range
> +from builtins import object

It can't be object because most other scripts use it just fine in py2
and py3.

There's only one single use of zip and range in this script, and range
is clearly fine (although it wants to be xrange() on py2 and we do
opencode that places), so I'm guessing the problem is with zip(), but
it's not exactly clear why?

~Andrew

Re: [XEN PATCH v1] tools/python: Add python3 compatibility

[Javi Merino]

On Wed, Oct 11, 2023 at 12:34:27AM +0800, Andrew Cooper wrote:
> On 10/10/2023 10:18 pm, Javi Merino wrote:
> > Most of the work for python3 compatibility was done in
> > 1430c5a8cad4 (tools/python: Python 3 compatibility, 2019-12-18).  This
> > just adds a few builtins needed for python3.
> >
> > Resolves: xen-project/xen#156
> >
> > Signed-off-by: Javi Merino <javi.merino@cloud.com>
> > ---
> >
> > I haven't tested it.
> >
> >  README                                     | 1 +
> >  tools/python/scripts/convert-legacy-stream | 3 +++
> >  2 files changed, 4 insertions(+)
> >
> > diff --git a/README b/README
> > index 855db01d36..44ed88c392 100644
> > --- a/README
> > +++ b/README
> > @@ -51,6 +51,7 @@ provided by your OS distributor:
> >      * POSIX compatible awk
> >      * Development install of zlib (e.g., zlib-dev)
> >      * Development install of Python 2.7 or later (e.g., python-dev)
> > +      - If using Python 2.7, you also need the future module.  This is not needed for Python 3.
> >      * Development install of curses (e.g., libncurses-dev)
> >      * Development install of openssl (e.g., openssl-dev)
> >      * Development install of x11 (e.g. xorg-x11-dev)
> > diff --git a/tools/python/scripts/convert-legacy-stream b/tools/python/scripts/convert-legacy-stream
> > index 7fe375a668..26a66c50fc 100755
> > --- a/tools/python/scripts/convert-legacy-stream
> > +++ b/tools/python/scripts/convert-legacy-stream
> > @@ -8,6 +8,9 @@ Convert a legacy migration stream to a v2 stream.
> >  from __future__ import print_function
> >  from __future__ import division
> >  
> > +from builtins import zip
> > +from builtins import range
> > +from builtins import object
> 
> It can't be object because most other scripts use it just fine in py2
> and py3.

This just makes the VM() class behave like python3: str(vm) will
return a unicode string.  After a quick glance at the code I don't
think the code currently does this, but at least the behaviour will
now be consistent between the two versions.

> There's only one single use of zip and range in this script, and range
> is clearly fine (although it wants to be xrange() on py2 and we do
> opencode that places), so I'm guessing the problem is with zip(), but
> it's not exactly clear why?

These changes just make the code be more consistent between python2
and python3.  As I said under the commit message, I have not tested
the changes.

Cheers,
Javi