[PATCH 0/2] [PULL REQUEST,v2] Rework the devshell

[PATCH 0/2] [PULL REQUEST,v2] Rework the devshell

[Christopher Larson]

Changes since the previous sends: resurrects the export of some variables that
were inadvertently removed.

The following changes since commit de4dc5f0741533529789a6094721f708728af386:

  eglibc: fix runtime assertion failure (2011-07-13 12:34:23 +0100)

are available in the git repository at:
  http://github.com/kergoth/oe-core devshell

Chris Larson (2):
  Rework how the devshell functions
  oe.terminal: improve how we spawn screen

 meta/classes/devshell.bbclass |   29 +++++-------
 meta/classes/terminal.bbclass |   30 +++++++++++
 meta/lib/oe/terminal.py       |  107 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 149 insertions(+), 17 deletions(-)
 create mode 100644 meta/classes/terminal.bbclass
 create mode 100644 meta/lib/oe/terminal.py

-- 
1.7.5.4

[PATCH 1/2] Rework how the devshell functions

[Christopher Larson]

From: Chris Larson <chris_larson@mentor.com>

In the new implementation, each known terminal is defined as a class in
oe.terminal, as a subclass of bb.process.Popen.  terminal.bbclass wraps this
functionality, providing the metadata pieces.  It obeys the OE_TERMINAL
variable, which is a 'choice' typed variable.  This variable may be 'auto',
'none', or any of the names of the defined terminals.

When using 'auto', or requesting an unsupported terminal, we attempt to spawn
them in priority order until we get one that's available on this system (and
in the case of the X terminals, has DISPLAY defined).  The 'none' value is
used when we're doing things like automated builds, and want to ensure that no
terminal is *ever* spawned, under any circumstances.

Current available terminals:

    gnome
    konsole
    xterm
    rxvt
    screen

Signed-off-by: Chris Larson <chris_larson@mentor.com>
---
 meta/classes/devshell.bbclass |   29 +++++-------
 meta/classes/terminal.bbclass |   30 ++++++++++++
 meta/lib/oe/terminal.py       |  102 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 144 insertions(+), 17 deletions(-)
 create mode 100644 meta/classes/terminal.bbclass
 create mode 100644 meta/lib/oe/terminal.py

diff --git a/meta/classes/devshell.bbclass b/meta/classes/devshell.bbclass
index 5f262f4..95c29f3 100644
--- a/meta/classes/devshell.bbclass
+++ b/meta/classes/devshell.bbclass
@@ -1,22 +1,17 @@
-do_devshell[dirs] = "${S}"
-do_devshell[nostamp] = "1"
+inherit terminal
 
-XAUTHORITY ?= "${HOME}/.Xauthority"
 
-devshell_do_devshell() {
-	export DISPLAY='${DISPLAY}'
-	export DBUS_SESSION_BUS_ADDRESS='${DBUS_SESSION_BUS_ADDRESS}'
-	export XAUTHORITY='${XAUTHORITY}'
-	export TERMWINDOWTITLE="Bitbake Developer Shell"
-	export EXTRA_OEMAKE='${EXTRA_OEMAKE}'
-	export SHELLCMDS="bash"
-	${TERMCMDRUN}
-	if [ $? -ne 0 ]; then
-	    echo "Fatal: '${TERMCMD}' not found. Check TERMCMD variable."
-	    exit 1
-	fi
+export XAUTHORITY ?= "${HOME}/.Xauthority"
+export SHELL ?= "bash"
+export DBUS_SESSION_BUS_ADDRESS
+export DISPLAY
+export EXTRA_OEMAKE
+
+python do_devshell () {
+    oe_terminal(d.getVar('SHELL', True), 'OpenEmbedded Developer Shell', d)
 }
-addtask devshell after do_patch
 
-EXPORT_FUNCTIONS do_devshell
+addtask devshell after do_patch
 
+do_devshell[dirs] = "${S}"
+do_devshell[nostamp] = "1"
diff --git a/meta/classes/terminal.bbclass b/meta/classes/terminal.bbclass
new file mode 100644
index 0000000..93646f7
--- /dev/null
+++ b/meta/classes/terminal.bbclass
@@ -0,0 +1,30 @@
+OE_TERMINAL ?= 'auto'
+OE_TERMINAL[type] = 'choice'
+OE_TERMINAL[choices] = 'auto none \
+                        ${@" ".join(o.name \
+                                    for o in oe.terminal.prioritized())}'
+
+
+def oe_terminal(command, title, d):
+    import oe.data
+    import oe.terminal
+
+    terminal = oe.data.typed_value('OE_TERMINAL', d).lower()
+    if terminal == 'none':
+        bb.fatal('Devshell usage disabled with OE_TERMINAL')
+    elif terminal != 'auto':
+        try:
+            oe.terminal.spawn(terminal, command, title)
+            return
+        except oe.terminal.UnsupportedTerminal:
+            bb.warn('Unsupported terminal "%s", defaulting to "auto"' %
+                    terminal)
+        except oe.terminal.ExecutionError as exc:
+            bb.fatal('Unable to spawn terminal %s: %s' % (terminal, exc))
+
+    try:
+        oe.terminal.spawn_preferred(command, title)
+    except oe.terminal.NoSupportedTerminals:
+        bb.fatal('No valid terminal found, unable to open devshell')
+    except oe.terminal.ExecutionError as exc:
+        bb.fatal('Unable to spawn terminal %s: %s' % (terminal, exc))
diff --git a/meta/lib/oe/terminal.py b/meta/lib/oe/terminal.py
new file mode 100644
index 0000000..5336167
--- /dev/null
+++ b/meta/lib/oe/terminal.py
@@ -0,0 +1,102 @@
+import logging
+import os
+import oe.classutils
+import shlex
+from bb.process import Popen, ExecutionError
+
+logger = logging.getLogger('BitBake.OE.Terminal')
+
+
+class UnsupportedTerminal(StandardError):
+    pass
+
+class NoSupportedTerminals(StandardError):
+    pass
+
+
+class Registry(oe.classutils.ClassRegistry):
+    command = None
+
+    def __init__(cls, name, bases, attrs):
+        super(Registry, cls).__init__(name.lower(), bases, attrs)
+
+    @property
+    def implemented(cls):
+        return bool(cls.command)
+
+
+class Terminal(Popen):
+    __metaclass__ = Registry
+
+    def __init__(self, command, title=None):
+        self.format_command(command, title)
+        logger.debug(1, "%s: running %s", self.name, self.command)
+
+        try:
+            Popen.__init__(self, self.command, shell=False)
+        except OSError as exc:
+            import errno
+            if exc.errno == errno.ENOENT:
+                raise UnsupportedTerminal(self.name)
+            else:
+                raise
+
+    def format_command(self, command, title):
+        fmt = {'title': title or 'Terminal', 'command': command}
+        if isinstance(self.command, basestring):
+            self.command = shlex.split(self.command.format(**fmt))
+        else:
+            self.command = [element.format(**fmt) for element in self.command]
+
+class XTerminal(Terminal):
+    def __init__(self, command, title=None):
+        Terminal.__init__(self, command, title)
+        if not os.environ.get('DISPLAY'):
+            raise UnsupportedTerminal(self.name)
+
+class Gnome(XTerminal):
+    command = 'gnome-terminal --disable-factory -t "{title}" -x {command}'
+    priority = 2
+
+class Konsole(XTerminal):
+    command = 'konsole -T "{title}" -e {command}'
+    priority = 2
+
+class XTerm(XTerminal):
+    command = 'xterm -T "{title}" -e {command}'
+    priority = 1
+
+class Rxvt(XTerminal):
+    command = 'rxvt -T "{title}" -e {command}'
+    priority = 1
+
+class Screen(Terminal):
+    command = 'screen -D -m -t "{title}" {command}'
+
+
+def prioritized():
+    return Registry.prioritized()
+
+def spawn_preferred(command, title=None):
+    """Spawn the first supported terminal, by priority"""
+    for terminal in prioritized():
+        try:
+            spawn(terminal.name, command, title)
+            break
+        except UnsupportedTerminal:
+            continue
+    else:
+        raise NoSupportedTerminals()
+
+def spawn(name, command, title=None):
+    """Spawn the specified terminal, by name"""
+    logger.debug(1, 'Attempting to spawn terminal "%s"', name)
+    try:
+        terminal = Registry.registry[name]
+    except KeyError:
+        raise UnsupportedTerminal(name)
+
+    pipe = terminal(command, title)
+    output = pipe.communicate()[0]
+    if pipe.returncode != 0:
+        raise ExecutionError(pipe.command, pipe.returncode, output)
-- 
1.7.5.4

[PATCH 2/2] oe.terminal: improve how we spawn screen

[Christopher Larson]

From: Chris Larson <chris_larson@mentor.com>

- Name the screen session 'devshell', to avoid confusion if running bitbake
  itself under a screen session.
- Display a warning message when spawning screen, so it's clear to the user
  that screen has been run (otherwise do_devshell just appears to hang).

Signed-off-by: Chris Larson <chris_larson@mentor.com>
---
 meta/lib/oe/terminal.py |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

diff --git a/meta/lib/oe/terminal.py b/meta/lib/oe/terminal.py
index 5336167..bbff8d0 100644
--- a/meta/lib/oe/terminal.py
+++ b/meta/lib/oe/terminal.py
@@ -71,7 +71,12 @@ class Rxvt(XTerminal):
     priority = 1
 
 class Screen(Terminal):
-    command = 'screen -D -m -t "{title}" {command}'
+    command = 'screen -D -m -t "{title}" -S devshell {command}'
+
+    def __init__(self, command, title=None):
+        Terminal.__init__(self, command, title)
+        logger.warn('Screen started. Please connect in another terminal with '
+                    '"screen -r devshell"')
 
 
 def prioritized():
-- 
1.7.5.4

Re: [PATCH 1/2] Rework how the devshell functions

[Richard Purdie]

On Wed, 2011-07-13 at 11:35 -0700, Christopher Larson wrote:
> From: Chris Larson <chris_larson@mentor.com>
> 
> In the new implementation, each known terminal is defined as a class in
> oe.terminal, as a subclass of bb.process.Popen.  terminal.bbclass wraps this
> functionality, providing the metadata pieces.  It obeys the OE_TERMINAL
> variable, which is a 'choice' typed variable.  This variable may be 'auto',
> 'none', or any of the names of the defined terminals.
> 
> When using 'auto', or requesting an unsupported terminal, we attempt to spawn
> them in priority order until we get one that's available on this system (and
> in the case of the X terminals, has DISPLAY defined).  The 'none' value is
> used when we're doing things like automated builds, and want to ensure that no
> terminal is *ever* spawned, under any circumstances.
> 
> Current available terminals:
> 
>     gnome
>     konsole
>     xterm
>     rxvt
>     screen
> 
> Signed-off-by: Chris Larson <chris_larson@mentor.com>
> ---
>  meta/classes/devshell.bbclass |   29 +++++-------
>  meta/classes/terminal.bbclass |   30 ++++++++++++
>  meta/lib/oe/terminal.py       |  102 +++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 144 insertions(+), 17 deletions(-)
>  create mode 100644 meta/classes/terminal.bbclass
>  create mode 100644 meta/lib/oe/terminal.py
> 
> diff --git a/meta/classes/devshell.bbclass b/meta/classes/devshell.bbclass
> index 5f262f4..95c29f3 100644
> --- a/meta/classes/devshell.bbclass
> +++ b/meta/classes/devshell.bbclass
> @@ -1,22 +1,17 @@
> -do_devshell[dirs] = "${S}"
> -do_devshell[nostamp] = "1"
> +inherit terminal
>  
> -XAUTHORITY ?= "${HOME}/.Xauthority"
>  
> -devshell_do_devshell() {
> -	export DISPLAY='${DISPLAY}'
> -	export DBUS_SESSION_BUS_ADDRESS='${DBUS_SESSION_BUS_ADDRESS}'
> -	export XAUTHORITY='${XAUTHORITY}'
> -	export TERMWINDOWTITLE="Bitbake Developer Shell"
> -	export EXTRA_OEMAKE='${EXTRA_OEMAKE}'
> -	export SHELLCMDS="bash"
> -	${TERMCMDRUN}
> -	if [ $? -ne 0 ]; then
> -	    echo "Fatal: '${TERMCMD}' not found. Check TERMCMD variable."
> -	    exit 1
> -	fi
> +export XAUTHORITY ?= "${HOME}/.Xauthority"
> +export SHELL ?= "bash"
> +export DBUS_SESSION_BUS_ADDRESS
> +export DISPLAY
> +export EXTRA_OEMAKE

Variables that are placed into the global environment affect the
checksums of any shell tasks when they change so I don't think we can do
this (and its why they are in a function as above).

Perhaps we could list the variables we want to export in a variable and
then pass that list to popen's environment?

Cheers,

Richard

Re: [PATCH 1/2] Rework how the devshell functions

[Chris Larson]

On Wed, Jul 13, 2011 at 12:36 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Wed, 2011-07-13 at 11:35 -0700, Christopher Larson wrote:
>> diff --git a/meta/classes/devshell.bbclass b/meta/classes/devshell.bbclass
>> index 5f262f4..95c29f3 100644
>> --- a/meta/classes/devshell.bbclass
>> +++ b/meta/classes/devshell.bbclass
>> @@ -1,22 +1,17 @@
>> -do_devshell[dirs] = "${S}"
>> -do_devshell[nostamp] = "1"
>> +inherit terminal
>>
>> -XAUTHORITY ?= "${HOME}/.Xauthority"
>>
>> -devshell_do_devshell() {
>> -     export DISPLAY='${DISPLAY}'
>> -     export DBUS_SESSION_BUS_ADDRESS='${DBUS_SESSION_BUS_ADDRESS}'
>> -     export XAUTHORITY='${XAUTHORITY}'
>> -     export TERMWINDOWTITLE="Bitbake Developer Shell"
>> -     export EXTRA_OEMAKE='${EXTRA_OEMAKE}'
>> -     export SHELLCMDS="bash"
>> -     ${TERMCMDRUN}
>> -     if [ $? -ne 0 ]; then
>> -         echo "Fatal: '${TERMCMD}' not found. Check TERMCMD variable."
>> -         exit 1
>> -     fi
>> +export XAUTHORITY ?= "${HOME}/.Xauthority"
>> +export SHELL ?= "bash"
>> +export DBUS_SESSION_BUS_ADDRESS
>> +export DISPLAY
>> +export EXTRA_OEMAKE
>
> Variables that are placed into the global environment affect the
> checksums of any shell tasks when they change so I don't think we can do
> this (and its why they are in a function as above).
>
> Perhaps we could list the variables we want to export in a variable and
> then pass that list to popen's environment?

Hmm, fair enough, I'll rework it that way (aside: we really need a
general flag-based mechanism to do per-task exports).
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics