[PATCH 0/4] More python fixes

[PATCH 0/4] More python fixes

[Wei Liu]

Wei Liu (4):
  pygrub: fix message in grub parser
  pygrub/grub: always use integer for default entry
  pygrub: decode string in Python 3
  tools/ocaml: make python scripts 2 and 3 compatible

 tools/ocaml/libs/xentoollog/genlevels.py |  5 ++++-
 tools/ocaml/libs/xl/genwrap.py           | 17 ++++++++++-------
 tools/pygrub/src/GrubConf.py             |  8 ++++++--
 tools/pygrub/src/pygrub                  |  5 ++++-
 4 files changed, 24 insertions(+), 11 deletions(-)

-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH 1/4] pygrub: fix message in grub parser

[Wei Liu]

The code suggests 0 is allowed. Zero is not a positive number.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
Backport candadiate.
---
 tools/pygrub/src/GrubConf.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/pygrub/src/GrubConf.py b/tools/pygrub/src/GrubConf.py
index f8d3799dc0..0204d410ac 100644
--- a/tools/pygrub/src/GrubConf.py
+++ b/tools/pygrub/src/GrubConf.py
@@ -236,7 +236,7 @@ class _GrubConfigFile(object):
             self._default = val
 
         if self._default < 0:
-            raise ValueError("default must be positive number")
+            raise ValueError("default must be non-negative number")
     default = property(_get_default, _set_default)
 
     def set_splash(self, val):
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 1/4] pygrub: fix message in grub parser

[Andrew Cooper]

On 01/04/2019 11:32, Wei Liu wrote:
> The code suggests 0 is allowed. Zero is not a positive number.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

I suspect you are opening a can of number theory worms with that commit
message, but the overall change is an improvement.

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH 2/4] pygrub/grub: always use integer for default entry

[Wei Liu]

The original code set the default to either a string or an integer
(0) and relies on a Python 2 specific behaviour to work (integer is
allowed to be compared to string in Python 2 but not 3).

Always use integer. The caller (pygrub) already has code to handle
that.

Reported-by: M A Young <m.a.young@durham.ac.uk>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 tools/pygrub/src/GrubConf.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/tools/pygrub/src/GrubConf.py b/tools/pygrub/src/GrubConf.py
index 0204d410ac..594139bac7 100644
--- a/tools/pygrub/src/GrubConf.py
+++ b/tools/pygrub/src/GrubConf.py
@@ -233,7 +233,11 @@ class _GrubConfigFile(object):
         if val == "saved":
             self._default = 0
         else:
-            self._default = val
+            try:
+                self._default = int(val)
+            except ValueError:
+                logging.warning("Invalid value %s, setting default to 0" %(val,))
+                self._default = 0
 
         if self._default < 0:
             raise ValueError("default must be non-negative number")
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 2/4] pygrub/grub: always use integer for default entry

[Andrew Cooper]

On 01/04/2019 11:32, Wei Liu wrote:
> The original code set the default to either a string or an integer
> (0) and relies on a Python 2 specific behaviour to work (integer is
> allowed to be compared to string in Python 2 but not 3).
>
> Always use integer. The caller (pygrub) already has code to handle
> that.
>
> Reported-by: M A Young <m.a.young@durham.ac.uk>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH 3/4] pygrub: decode string in Python 3

[Wei Liu]

String is unicode in 3 but bytes in 2. We need to call decode function
when using Python 3.

Reported-by: M A Young <m.a.young@durham.ac.uk>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 tools/pygrub/src/pygrub | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tools/pygrub/src/pygrub b/tools/pygrub/src/pygrub
index dbdce315c6..23312eae76 100755
--- a/tools/pygrub/src/pygrub
+++ b/tools/pygrub/src/pygrub
@@ -457,7 +457,10 @@ class Grub:
         # limit read size to avoid pathological cases
         buf = f.read(FS_READ_MAX)
         del f
-        self.cf.parse(buf)
+        if sys.version_info[0] < 3:
+            self.cf.parse(buf)
+        else:
+            self.cf.parse(buf.decode())
 
     def image_index(self):
         if isinstance(self.cf.default, int):
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 3/4] pygrub: decode string in Python 3

[Andrew Cooper]

On 01/04/2019 11:32, Wei Liu wrote:
> String is unicode in 3 but bytes in 2. We need to call decode function
> when using Python 3.
>
> Reported-by: M A Young <m.a.young@durham.ac.uk>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH 4/4] tools/ocaml: make python scripts 2 and 3 compatible

[Wei Liu]

1. Explicitly import reduce because that's required in 3.
2. Change print to function.
3. Eliminate invocations of has_key.

Signed-off-by: M A Young <m.a.young@durham.ac.uk>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 tools/ocaml/libs/xentoollog/genlevels.py |  5 ++++-
 tools/ocaml/libs/xl/genwrap.py           | 17 ++++++++++-------
 2 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/tools/ocaml/libs/xentoollog/genlevels.py b/tools/ocaml/libs/xentoollog/genlevels.py
index 8c233c59b1..f9cf853e26 100755
--- a/tools/ocaml/libs/xentoollog/genlevels.py
+++ b/tools/ocaml/libs/xentoollog/genlevels.py
@@ -1,6 +1,9 @@
 #!/usr/bin/python
 
+from __future__ import print_function
+
 import sys
+from functools import reduce
 
 def read_levels():
 	f = open('../../../libs/toollog/include/xentoollog.h', 'r')
@@ -93,7 +96,7 @@ def autogen_header(open_comment, close_comment):
 
 if __name__ == '__main__':
 	if len(sys.argv) < 3:
-		print >>sys.stderr, "Usage: genlevels.py <mli> <ml> <c-inc>"
+		print("Usage: genlevels.py <mli> <ml> <c-inc>", file=sys.stderr)
 		sys.exit(1)
 
 	levels, olevels = read_levels()
diff --git a/tools/ocaml/libs/xl/genwrap.py b/tools/ocaml/libs/xl/genwrap.py
index 815c1cb0e3..7bf26bdcd8 100644
--- a/tools/ocaml/libs/xl/genwrap.py
+++ b/tools/ocaml/libs/xl/genwrap.py
@@ -1,6 +1,9 @@
 #!/usr/bin/python
 
+from __future__ import print_function
+
 import sys,os
+from functools import reduce
 
 import idl
 
@@ -78,7 +81,7 @@ def ocaml_type_of(ty):
     elif isinstance(ty,idl.Array):
         return "%s array" % ocaml_type_of(ty.elem_type)
     elif isinstance(ty,idl.Builtin):
-        if not builtins.has_key(ty.typename):
+        if ty.typename not in builtins:
             raise NotImplementedError("Unknown Builtin %s (%s)" % (ty.typename, type(ty)))
         typename,_,_ = builtins[ty.typename]
         if not typename:
@@ -251,7 +254,7 @@ def gen_ocaml_ml(ty, interface, indent=""):
             else:
                 s += "\texternal default : ctx -> %sunit -> t = \"stub_libxl_%s_init\"\n" % (union_args, ty.rawname)
 
-        if functions.has_key(ty.rawname):
+        if ty.rawname in functions:
             for name,args in functions[ty.rawname]:
                 s += "\texternal %s : " % name
                 s += " -> ".join(args)
@@ -278,7 +281,7 @@ def c_val(ty, c, o, indent="", parent = None):
         else:
             s += "%s = Int_val(%s);" % (c, o)
     elif isinstance(ty,idl.Builtin):
-        if not builtins.has_key(ty.typename):
+        if ty.typename not in builtins:
             raise NotImplementedError("Unknown Builtin %s (%s)" % (ty.typename, type(ty)))
         _,fn,_ = builtins[ty.typename]
         if not fn:
@@ -375,7 +378,7 @@ def ocaml_Val(ty, o, c, indent="", parent = None):
         else:
             s += "%s = Val_int(%s);" % (o, c)
     elif isinstance(ty,idl.Builtin):
-        if not builtins.has_key(ty.typename):
+        if ty.typename not in builtins:
             raise NotImplementedError("Unknown Builtin %s (%s)" % (ty.typename, type(ty)))
         _,_,fn = builtins[ty.typename]
         if not fn:
@@ -520,7 +523,7 @@ def autogen_header(open_comment, close_comment):
 
 if __name__ == '__main__':
     if len(sys.argv) < 4:
-        print >>sys.stderr, "Usage: genwrap.py <idl> <mli> <ml> <c-inc>"
+        print("Usage: genwrap.py <idl> <mli> <ml> <c-inc>", file=sys.stderr)
         sys.exit(1)
 
     (_,types) = idl.parse(sys.argv[1])
@@ -533,7 +536,7 @@ if __name__ == '__main__':
 
     for t in blacklist:
         if t not in [ty.rawname for ty in types]:
-            print "unknown type %s in blacklist" % t
+            print("unknown type %s in blacklist" % t)
 
     types = [ty for ty in types if not ty.rawname in blacklist]
 
@@ -564,7 +567,7 @@ if __name__ == '__main__':
             cinc.write("\n")
         cinc.write(gen_Val_ocaml(ty))
         cinc.write("\n")
-        if functions.has_key(ty.rawname):
+        if ty.rawname in functions:
             cinc.write(gen_c_stub_prototype(ty, functions[ty.rawname]))
             cinc.write("\n")
         if ty.init_fn is not None:
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 4/4] tools/ocaml: make python scripts 2 and 3 compatible

[Andrew Cooper]

On 01/04/2019 11:32, Wei Liu wrote:
> 1. Explicitly import reduce because that's required in 3.
> 2. Change print to function.
> 3. Eliminate invocations of has_key.
>
> Signed-off-by: M A Young <m.a.young@durham.ac.uk>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 4/4] tools/ocaml: make python scripts 2 and 3 compatible

[Christian Lindig]

> On 1 Apr 2019, at 11:32, Wei Liu <wei.liu2@citrix.com> wrote:
> 
> 1. Explicitly import reduce because that's required in 3.
> 2. Change print to function.
> 3. Eliminate invocations of has_key.
> 
> Signed-off-by: M A Young <m.a.young@durham.ac.uk>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

Acked-by: Christian Lindig <christian.lindig@citrix.com>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 0/4] More python fixes

[YOUNG, MICHAEL A.]

[-- Attachment #1: Type: text/plain, Size: 999 bytes --]

On Mon, 1 Apr 2019, Wei Liu wrote:

> Wei Liu (4):
>  pygrub: fix message in grub parser
>  pygrub/grub: always use integer for default entry
>  pygrub: decode string in Python 3
>  tools/ocaml: make python scripts 2 and 3 compatible
>
> tools/ocaml/libs/xentoollog/genlevels.py |  5 ++++-
> tools/ocaml/libs/xl/genwrap.py           | 17 ++++++++++-------
> tools/pygrub/src/GrubConf.py             |  8 ++++++--
> tools/pygrub/src/pygrub                  |  5 ++++-
> 4 files changed, 24 insertions(+), 11 deletions(-)

There is a bit missing compared to my original patch which is to convert 
the string back to bytes for the final write out and avoid the error

Traceback (most recent call last):
   File "/tmp/xencode/usr/libexec/xen/bin/pygrub", line 967, in <module>
     os.write(fd, ostring)
TypeError: a bytes-like object is required, not 'str'

The attached patch works for me in a quick test on python3 though I 
haven't tested it on python2 yet.

 	Michael Young

[-- Attachment #2: python3.patch --]
[-- Type: text/plain, Size: 445 bytes --]

--- xen-4.12.0/tools/pygrub/src/pygrub.orig	2019-04-01 21:25:33.206405995 +0100
+++ xen-4.12.0/tools/pygrub/src/pygrub	2019-04-01 21:27:36.179929105 +0100
@@ -963,5 +963,8 @@
         ostring = format_simple(bootcfg["kernel"], bootcfg["ramdisk"], args, "\0")
 
     sys.stdout.flush()
-    os.write(fd, ostring)
+    if sys.version_info[0] < 3:
+        os.write(fd, ostring)
+    else:
+        os.write(fd, ostring.encode())
     

[-- Attachment #3: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 0/4] More python fixes

[Wei Liu]

On Mon, Apr 01, 2019 at 08:36:44PM +0000, YOUNG, MICHAEL A. wrote:
> On Mon, 1 Apr 2019, Wei Liu wrote:
> 
> > Wei Liu (4):
> >  pygrub: fix message in grub parser
> >  pygrub/grub: always use integer for default entry
> >  pygrub: decode string in Python 3
> >  tools/ocaml: make python scripts 2 and 3 compatible
> >
> > tools/ocaml/libs/xentoollog/genlevels.py |  5 ++++-
> > tools/ocaml/libs/xl/genwrap.py           | 17 ++++++++++-------
> > tools/pygrub/src/GrubConf.py             |  8 ++++++--
> > tools/pygrub/src/pygrub                  |  5 ++++-
> > 4 files changed, 24 insertions(+), 11 deletions(-)
> 
> There is a bit missing compared to my original patch which is to convert 
> the string back to bytes for the final write out and avoid the error
> 
> Traceback (most recent call last):
>    File "/tmp/xencode/usr/libexec/xen/bin/pygrub", line 967, in <module>
>      os.write(fd, ostring)
> TypeError: a bytes-like object is required, not 'str'
> 
> The attached patch works for me in a quick test on python3 though I 
> haven't tested it on python2 yet.
> 
>  	Michael Young

> --- xen-4.12.0/tools/pygrub/src/pygrub.orig	2019-04-01 21:25:33.206405995 +0100
> +++ xen-4.12.0/tools/pygrub/src/pygrub	2019-04-01 21:27:36.179929105 +0100
> @@ -963,5 +963,8 @@
>          ostring = format_simple(bootcfg["kernel"], bootcfg["ramdisk"], args, "\0")
>  
>      sys.stdout.flush()
> -    os.write(fd, ostring)
> +    if sys.version_info[0] < 3:
> +        os.write(fd, ostring)
> +    else:
> +        os.write(fd, ostring.encode())
>      


This should work. I will fold this in to the pygrub patch.

Thanks for spotting my mistake.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel