[PATCH v2 0/3] devpyshell fixes

[PATCH v2 0/3] devpyshell fixes

[Ed Bartosh]

Hi,

This patchset fixes several devpyshell bugs caused by moving to Python 3.

Please, note that it contains fix for double unbuffering, which was sent in
a separate patchset targeting jethro, krogoth and master. It's included here
just to keep it together with the rest of devpyshell fixes for master.

Changes in v2: removed bitbake patch from the series

The following changes since commit 5c11e365e19357f721c49d076971567e7b64b61b:

  lib/oeqa: add Galculator to SDK and runtime tests (2016-07-01 16:22:48 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib ed/oe-core/devpyshell-python3
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/oe-core/devpyshell-python3

Ed Bartosh (3):
  devshell.bbclass: fix double unbuffering
  devpyshell: python3: flush stdout explicitly
  oepydevshell-internal: python3: encode/decode pty content

 meta/classes/devshell.bbclass    |  5 ++---
 scripts/oepydevshell-internal.py | 12 +++++-------
 2 files changed, 7 insertions(+), 10 deletions(-)

--
Regards,
Ed

[PATCH v2 1/3] devshell.bbclass: fix double unbuffering

[Ed Bartosh]

stdout is already unbuffered in bitbake code. Attempt to
do it again in devshell.bbclass causes this crash when
running devpyshell:
  File "scripts/oepydevshell-internal.py", line 29, in <module>
      pty = open(sys.argv[1], "w+b", 0)
  IOError: [Errno 13] Permission denied: '/dev/pts/6'

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 meta/classes/devshell.bbclass | 1 -
 1 file changed, 1 deletion(-)

diff --git a/meta/classes/devshell.bbclass b/meta/classes/devshell.bbclass
index 341d9c0..041ed15 100644
--- a/meta/classes/devshell.bbclass
+++ b/meta/classes/devshell.bbclass
@@ -65,7 +65,6 @@ def devpyshell(d):
         os.dup2(m, sys.stdout.fileno())
         os.dup2(m, sys.stderr.fileno())
 
-        sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
         sys.stdin = os.fdopen(sys.stdin.fileno(), 'r', 0)
 
         bb.utils.nonblockingfd(sys.stdout)
-- 
2.1.4

[PATCH v2 2/3] devpyshell: python3: flush stdout explicitly

[Ed Bartosh]

Opening text stream in unbuffered mode raises the following
exception In Python 3:
    ValueError: can't have unbuffered text I/O

Fixed by leaving std* streams in text mode and flushing
stdout explicitly.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 meta/classes/devshell.bbclass    | 4 ++--
 scripts/oepydevshell-internal.py | 4 +---
 2 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/meta/classes/devshell.bbclass b/meta/classes/devshell.bbclass
index 041ed15..be71aff 100644
--- a/meta/classes/devshell.bbclass
+++ b/meta/classes/devshell.bbclass
@@ -65,8 +65,6 @@ def devpyshell(d):
         os.dup2(m, sys.stdout.fileno())
         os.dup2(m, sys.stderr.fileno())
 
-        sys.stdin = os.fdopen(sys.stdin.fileno(), 'r', 0)
-
         bb.utils.nonblockingfd(sys.stdout)
         bb.utils.nonblockingfd(sys.stderr)
         bb.utils.nonblockingfd(sys.stdin)
@@ -92,6 +90,7 @@ def devpyshell(d):
             else:
                 prompt = ps1
             sys.stdout.write(prompt)
+            sys.stdout.flush()
 
         # Restore Ctrl+C since bitbake masks this
         def signal_handler(signal, frame):
@@ -113,6 +112,7 @@ def devpyshell(d):
                         continue
                 except EOFError as e:
                     sys.stdout.write("\n")
+                    sys.stdout.flush()
                 except (OSError, IOError) as e:
                     if e.errno == 11:
                         continue
diff --git a/scripts/oepydevshell-internal.py b/scripts/oepydevshell-internal.py
index 7761f66..31a75ac 100755
--- a/scripts/oepydevshell-internal.py
+++ b/scripts/oepydevshell-internal.py
@@ -29,9 +29,6 @@ if len(sys.argv) != 3:
 pty = open(sys.argv[1], "w+b", 0)
 parent = int(sys.argv[2])
 
-# Don't buffer output by line endings
-sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
-sys.stdin = os.fdopen(sys.stdin.fileno(), 'r', 0)
 nonblockingfd(pty)
 nonblockingfd(sys.stdin)
 
@@ -64,6 +61,7 @@ try:
                     # Write a page at a time to avoid overflowing output 
                     # d.keys() is a good way to do that
                     sys.stdout.write(i[:4096])
+                    sys.stdout.flush()
                     i = i[4096:]
                 if sys.stdin in ready:
                     echonocbreak(sys.stdin.fileno())
-- 
2.1.4

[PATCH v2 3/3] oepydevshell-internal: python3: encode/decode pty content

[Ed Bartosh]

As /dev/pty opened in binary mode its content has to
be decoded when reading from it and encoded when writing to it.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 scripts/oepydevshell-internal.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/oepydevshell-internal.py b/scripts/oepydevshell-internal.py
index 31a75ac..a22bec3 100755
--- a/scripts/oepydevshell-internal.py
+++ b/scripts/oepydevshell-internal.py
@@ -47,7 +47,7 @@ try:
     # Need cbreak/noecho whilst in select so we trigger on any keypress
     cbreaknoecho(sys.stdin.fileno())
     # Send our PID to the other end so they can kill us.
-    pty.write(str(os.getpid()) + "\n")
+    pty.write(str(os.getpid()).encode('utf-8') + b"\n")
     while True:
         try:
             writers = []
@@ -56,7 +56,7 @@ try:
             (ready, _, _) = select.select([pty, sys.stdin], writers , [], 0)
             try:
                 if pty in ready:
-                    i = i + pty.read()
+                    i = i + pty.read().decode('utf-8')
                 if i:
                     # Write a page at a time to avoid overflowing output 
                     # d.keys() is a good way to do that
@@ -65,9 +65,9 @@ try:
                     i = i[4096:]
                 if sys.stdin in ready:
                     echonocbreak(sys.stdin.fileno())
-                    o = input()
+                    o = input().encode('utf-8')
                     cbreaknoecho(sys.stdin.fileno())
-                    pty.write(o + "\n")
+                    pty.write(o + b"\n")
             except (IOError, OSError) as e:
                 if e.errno == 11:
                     continue
-- 
2.1.4