[Qemu-devel] [PATCH for-3.1] qemu-iotests: Adapt to moved location of StringIO module in py3

[Qemu-devel] [PATCH for-3.1] qemu-iotests: Adapt to moved location of StringIO module in py3

[Philippe Mathieu-Daudé]

Since 356dc290f the Fedora image default to Python3.

This fixes:

  $ make docker-test-block@fedora
  [...]
  045         [failed, exit status 1] - output mismatch (see 045.out.bad)
  --- /tmp/qemu-test/src/tests/qemu-iotests/045.out       2018-07-17 16:56:18.000000000 +0000
  +++ /tmp/qemu-test/build/tests/qemu-iotests/045.out.bad 2018-07-17 17:19:22.448409007 +0000
  @@ -1,5 +1,6 @@
  -...........
  -----------------------------------------------------------------------
  -Ran 11 tests
  -
  -OK
  +Traceback (most recent call last):
  +  File "045", line 178, in <module>
  +    iotests.main(supported_fmts=['raw'])
  +  File "/tmp/qemu-test/src/tests/qemu-iotests/iotests.py", line 682, in main
  +    import StringIO
  +ModuleNotFoundError: No module named 'StringIO'
  132         [failed, exit status 1] - output mismatch (see 132.out.bad)
  148         [failed, exit status 1] - output mismatch (see 148.out.bad)
  152         [failed, exit status 1] - output mismatch (see 152.out.bad)

  Failures: 045 132 148 152

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 tests/qemu-iotests/iotests.py | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 4e67fbbe96..2fc7165fb9 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -679,13 +679,17 @@ def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[],
 
     # We need to filter out the time taken from the output so that qemu-iotest
     # can reliably diff the results against master output.
-    import StringIO
+    try:
+        from StringIO import StringIO
+    except ImportError:
+        from io import StringIO
+
     if debug:
         output = sys.stdout
         verbosity = 2
         sys.argv.remove('-d')
     else:
-        output = StringIO.StringIO()
+        output = StringIO()
 
     logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN))
 
-- 
2.18.0

Re: [Qemu-devel] [PATCH for-3.1] qemu-iotests: Adapt to moved location of StringIO module in py3

[Daniel P. Berrangé]

On Tue, Jul 17, 2018 at 08:40:15PM -0300, Philippe Mathieu-Daudé wrote:
> Since 356dc290f the Fedora image default to Python3.
> 
> This fixes:
> 
>   $ make docker-test-block@fedora
>   [...]
>   045         [failed, exit status 1] - output mismatch (see 045.out.bad)
>   --- /tmp/qemu-test/src/tests/qemu-iotests/045.out       2018-07-17 16:56:18.000000000 +0000
>   +++ /tmp/qemu-test/build/tests/qemu-iotests/045.out.bad 2018-07-17 17:19:22.448409007 +0000
>   @@ -1,5 +1,6 @@
>   -...........
>   -----------------------------------------------------------------------
>   -Ran 11 tests
>   -
>   -OK
>   +Traceback (most recent call last):
>   +  File "045", line 178, in <module>
>   +    iotests.main(supported_fmts=['raw'])
>   +  File "/tmp/qemu-test/src/tests/qemu-iotests/iotests.py", line 682, in main
>   +    import StringIO
>   +ModuleNotFoundError: No module named 'StringIO'
>   132         [failed, exit status 1] - output mismatch (see 132.out.bad)
>   148         [failed, exit status 1] - output mismatch (see 148.out.bad)
>   152         [failed, exit status 1] - output mismatch (see 152.out.bad)
> 
>   Failures: 045 132 148 152
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  tests/qemu-iotests/iotests.py | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
> index 4e67fbbe96..2fc7165fb9 100644
> --- a/tests/qemu-iotests/iotests.py
> +++ b/tests/qemu-iotests/iotests.py
> @@ -679,13 +679,17 @@ def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[],
>  
>      # We need to filter out the time taken from the output so that qemu-iotest
>      # can reliably diff the results against master output.
> -    import StringIO
> +    try:
> +        from StringIO import StringIO
> +    except ImportError:
> +        from io import StringIO
> +
>      if debug:
>          output = sys.stdout
>          verbosity = 2
>          sys.argv.remove('-d')
>      else:
> -        output = StringIO.StringIO()
> +        output = StringIO()
>  
>      logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN))

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH for-3.1] qemu-iotests: Adapt to moved location of StringIO module in py3

[Eduardo Habkost]

On Tue, Jul 17, 2018 at 08:40:15PM -0300, Philippe Mathieu-Daudé wrote:
[...]
> -    import StringIO
> +    try:
> +        from StringIO import StringIO
> +    except ImportError:
> +        from io import StringIO

Why do we need this?  Python 2.7 has io.StringIO.

-- 
Eduardo

Re: [Qemu-devel] [PATCH for-3.1] qemu-iotests: Adapt to moved location of StringIO module in py3

[Philippe Mathieu-Daudé]

Hi Eduardo,

On 07/18/2018 11:53 AM, Eduardo Habkost wrote:
> On Tue, Jul 17, 2018 at 08:40:15PM -0300, Philippe Mathieu-Daudé wrote:
> [...]
>> -    import StringIO
>> +    try:
>> +        from StringIO import StringIO
>> +    except ImportError:
>> +        from io import StringIO
> 
> Why do we need this?  Python 2.7 has io.StringIO.

Python 2 works fine, the problem is the Fedora Docker image uses Python
3 and the block tests started to fail...

Is this commit message clearer?

"Since 356dc290f the Fedora [Docker] image uses Python3 by default."

Re: [Qemu-devel] [PATCH for-3.1] qemu-iotests: Adapt to moved location of StringIO module in py3

[Eduardo Habkost]

On Wed, Jul 18, 2018 at 12:02:39PM -0300, Philippe Mathieu-Daudé wrote:
> Hi Eduardo,
> 
> On 07/18/2018 11:53 AM, Eduardo Habkost wrote:
> > On Tue, Jul 17, 2018 at 08:40:15PM -0300, Philippe Mathieu-Daudé wrote:
> > [...]
> >> -    import StringIO
> >> +    try:
> >> +        from StringIO import StringIO
> >> +    except ImportError:
> >> +        from io import StringIO
> > 
> > Why do we need this?  Python 2.7 has io.StringIO.
> 
> Python 2 works fine, the problem is the Fedora Docker image uses Python
> 3 and the block tests started to fail...

My question is: why use StringIO.StringIO on Python 2 and
io.StringIO on Python 3, if io.StringIO works on both Python
versions?

-- 
Eduardo

Re: [Qemu-devel] [PATCH for-3.1] qemu-iotests: Adapt to moved location of StringIO module in py3

[Philippe Mathieu-Daudé]

On 07/18/2018 12:05 PM, Eduardo Habkost wrote:
> On Wed, Jul 18, 2018 at 12:02:39PM -0300, Philippe Mathieu-Daudé wrote:
>> Hi Eduardo,
>>
>> On 07/18/2018 11:53 AM, Eduardo Habkost wrote:
>>> On Tue, Jul 17, 2018 at 08:40:15PM -0300, Philippe Mathieu-Daudé wrote:
>>> [...]
>>>> -    import StringIO
>>>> +    try:
>>>> +        from StringIO import StringIO
>>>> +    except ImportError:
>>>> +        from io import StringIO
>>>
>>> Why do we need this?  Python 2.7 has io.StringIO.
>>
>> Python 2 works fine, the problem is the Fedora Docker image uses Python
>> 3 and the block tests started to fail...
> 
> My question is: why use StringIO.StringIO on Python 2 and
> io.StringIO on Python 3, if io.StringIO works on both Python
> versions?

Oh I missed your question because I was not aware of this, and looked
how this was handled in the tree (7a5d936b6fc and 5f90af8e6b).

TIL we can use "from io import StringIO" regardless the version, the
2->3 conversion looks a bit less absurd, thanks!

Phil.

Re: [Qemu-devel] [Qemu-block] [PATCH for-3.1] qemu-iotests: Adapt to moved location of StringIO module in py3

[John Snow]

On 07/18/2018 11:05 AM, Eduardo Habkost wrote:
> On Wed, Jul 18, 2018 at 12:02:39PM -0300, Philippe Mathieu-Daudé wrote:
>> Hi Eduardo,
>>
>> On 07/18/2018 11:53 AM, Eduardo Habkost wrote:
>>> On Tue, Jul 17, 2018 at 08:40:15PM -0300, Philippe Mathieu-Daudé wrote:
>>> [...]
>>>> -    import StringIO
>>>> +    try:
>>>> +        from StringIO import StringIO
>>>> +    except ImportError:
>>>> +        from io import StringIO
>>>
>>> Why do we need this?  Python 2.7 has io.StringIO.
>>
>> Python 2 works fine, the problem is the Fedora Docker image uses Python
>> 3 and the block tests started to fail...
> 
> My question is: why use StringIO.StringIO on Python 2 and
> io.StringIO on Python 3, if io.StringIO works on both Python
> versions?
> 

Holdover from when 2.6 was our requisite version, surely.

--js

Re: [Qemu-devel] [PATCH for-3.1] qemu-iotests: Adapt to moved location of StringIO module in py3

[Eduardo Habkost]

On Wed, Jul 18, 2018 at 12:22:19PM -0300, Philippe Mathieu-Daudé wrote:
> On 07/18/2018 12:05 PM, Eduardo Habkost wrote:
> > On Wed, Jul 18, 2018 at 12:02:39PM -0300, Philippe Mathieu-Daudé wrote:
> >> Hi Eduardo,
> >>
> >> On 07/18/2018 11:53 AM, Eduardo Habkost wrote:
> >>> On Tue, Jul 17, 2018 at 08:40:15PM -0300, Philippe Mathieu-Daudé wrote:
> >>> [...]
> >>>> -    import StringIO
> >>>> +    try:
> >>>> +        from StringIO import StringIO
> >>>> +    except ImportError:
> >>>> +        from io import StringIO
> >>>
> >>> Why do we need this?  Python 2.7 has io.StringIO.
> >>
> >> Python 2 works fine, the problem is the Fedora Docker image uses Python
> >> 3 and the block tests started to fail...
> > 
> > My question is: why use StringIO.StringIO on Python 2 and
> > io.StringIO on Python 3, if io.StringIO works on both Python
> > versions?
> 
> Oh I missed your question because I was not aware of this, and looked
> how this was handled in the tree (7a5d936b6fc and 5f90af8e6b).
> 
> TIL we can use "from io import StringIO" regardless the version, the
> 2->3 conversion looks a bit less absurd, thanks!

Note that it's not always an obvious conversion: on Python 3 you
need to decide if you want a text file or binary file.
io.StringIO is a text file, io.BinaryIO is a binary file.  See
https://www.mail-archive.com/qemu-devel@nongnu.org/msg545627.html
for an example where io.StringIO wasn't appropriate.

In the case of iotests.py it looks like the code expects a text
file, though.

-- 
Eduardo