[Buildroot] [PATCH 0/3] graph-depends: split off some functions to pkgutil.py

[Buildroot] [PATCH 0/3] graph-depends: split off some functions to pkgutil.py

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>

The main goal of this patch set is to extract get_version and get_depends from
graph-depends into a separate file pkgutil.py, for reuse by other scripts.

The script I need it for is not really upstreamable (opkg generation for some
packages), but I think the changes I'm submitting here could be useful in other
cases too.

Thomas De Schampheleire (3):
  graph-depends: avoid use of global var 'rule' in get_depends
  graph-depends: split off get_version/get_depends into pkgutil.py
  graph-depends: split off get_rdepends from get_depends

 support/scripts/graph-depends | 59 ++++-----------------------------------
 support/scripts/pkgutil.py    | 64 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+), 53 deletions(-)
 create mode 100644 support/scripts/pkgutil.py

-- 
2.10.2

[Buildroot] [PATCH 1/3] graph-depends: avoid use of global var 'rule' in get_depends

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>

Function get_depends was recently changed to support both normal
dependencies as reverse dependencies, via a global variable 'rule' that
equals 'show-depends' or 'show-rdepends'.

As a subsequent function will extract this function get_depends to a
separate file, the use of globals is problematic.

Instead, pass the global as an argument.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
---
 support/scripts/graph-depends | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
index c3c97cb..095619a 100755
--- a/support/scripts/graph-depends
+++ b/support/scripts/graph-depends
@@ -161,7 +161,7 @@ def get_targets():
 # Execute the "make <pkg>-show-depends" command to get the list of
 # dependencies of a given list of packages, and return the list of
 # dependencies formatted as a Python dictionary.
-def get_depends(pkgs):
+def get_depends(pkgs, rule):
     sys.stderr.write("Getting dependencies for %s\n" % pkgs)
     cmd = ["make", "-s", "--no-print-directory" ]
     for pkg in pkgs:
@@ -204,7 +204,7 @@ def get_all_depends(pkgs):
     if len(filtered_pkgs) == 0:
         return []
 
-    depends = get_depends(filtered_pkgs)
+    depends = get_depends(filtered_pkgs, rule)
 
     deps = set()
     for pkg in filtered_pkgs:
-- 
2.10.2

[Buildroot] [PATCH 1/1] support/scripts: split off get_version/get_depends into pkgutil.py

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>

Functions to obtain the version and dependencies of a package from Python
can be useful for several scripts. Extract this logic out of graph-depends
into pkgutil.py.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
---
 support/scripts/graph-depends | 55 ++++---------------------------------------
 support/scripts/pkgutil.py    | 55 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+), 51 deletions(-)
 create mode 100644 support/scripts/pkgutil.py

diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
index c3c97cb..c1f5c8f 100755
--- a/support/scripts/graph-depends
+++ b/support/scripts/graph-depends
@@ -26,6 +26,8 @@ import subprocess
 import argparse
 from fnmatch import fnmatch
 
+import pkgutil
+
 # Modes of operation:
 MODE_FULL = 1   # draw full dependency graph for all selected packages
 MODE_PKG  = 2   # draw dependency graph for a given package
@@ -122,28 +124,6 @@ host_colour = colours[2]
 
 allpkgs = []
 
-# Execute the "make <pkg>-show-version" command to get the version of a given
-# list of packages, and return the version formatted as a Python dictionary.
-def get_version(pkgs):
-    sys.stderr.write("Getting version for %s\n" % pkgs)
-    cmd = ["make", "-s", "--no-print-directory" ]
-    for pkg in pkgs:
-        cmd.append("%s-show-version" % pkg)
-    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
-    output = p.communicate()[0]
-    if p.returncode != 0:
-        sys.stderr.write("Error getting version %s\n" % pkgs)
-        sys.exit(1)
-    output = output.split("\n")
-    if len(output) != len(pkgs) + 1:
-        sys.stderr.write("Error getting version\n")
-        sys.exit(1)
-    version = {}
-    for i in range(0, len(pkgs)):
-        pkg = pkgs[i]
-        version[pkg] = output[i]
-    return version
-
 # Execute the "make show-targets" command to get the list of the main
 # Buildroot PACKAGES and return it formatted as a Python list. This
 # list is used as the starting point for full dependency graphs
@@ -158,33 +138,6 @@ def get_targets():
         return []
     return output.split(' ')
 
-# Execute the "make <pkg>-show-depends" command to get the list of
-# dependencies of a given list of packages, and return the list of
-# dependencies formatted as a Python dictionary.
-def get_depends(pkgs):
-    sys.stderr.write("Getting dependencies for %s\n" % pkgs)
-    cmd = ["make", "-s", "--no-print-directory" ]
-    for pkg in pkgs:
-        cmd.append("%s-%s" % (pkg, rule))
-    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
-    output = p.communicate()[0]
-    if p.returncode != 0:
-        sys.stderr.write("Error getting dependencies %s\n" % pkgs)
-        sys.exit(1)
-    output = output.split("\n")
-    if len(output) != len(pkgs) + 1:
-        sys.stderr.write("Error getting dependencies\n")
-        sys.exit(1)
-    deps = {}
-    for i in range(0, len(pkgs)):
-        pkg = pkgs[i]
-        pkg_deps = output[i].split(" ")
-        if pkg_deps == ['']:
-            deps[pkg] = []
-        else:
-            deps[pkg] = pkg_deps
-    return deps
-
 # Recursive function that builds the tree of dependencies for a given
 # list of packages. The dependencies are built in a list called
 # 'dependencies', which contains tuples of the form (pkg1 ->
@@ -204,7 +157,7 @@ def get_all_depends(pkgs):
     if len(filtered_pkgs) == 0:
         return []
 
-    depends = get_depends(filtered_pkgs)
+    depends = pkgutil.get_depends(filtered_pkgs)
 
     deps = set()
     for pkg in filtered_pkgs:
@@ -377,7 +330,7 @@ if check_only:
     sys.exit(0)
 
 dict_deps = remove_extra_deps(dict_deps)
-dict_version = get_version([pkg for pkg in allpkgs
+dict_version = pkgutil.get_version([pkg for pkg in allpkgs
                                 if pkg != "all" and not pkg.startswith("root")])
 
 # Print the attributes of a node: label and fill-color
diff --git a/support/scripts/pkgutil.py b/support/scripts/pkgutil.py
new file mode 100644
index 0000000..191134a
--- /dev/null
+++ b/support/scripts/pkgutil.py
@@ -0,0 +1,55 @@
+#!/usr/bin/env python
+
+# Copyright (C) 2010-2013 Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+
+import sys
+import subprocess
+
+# Execute the "make <pkg>-show-version" command to get the version of a given
+# list of packages, and return the version formatted as a Python dictionary.
+def get_version(pkgs):
+    sys.stderr.write("Getting version for %s\n" % pkgs)
+    cmd = ["make", "-s", "--no-print-directory" ]
+    for pkg in pkgs:
+        cmd.append("%s-show-version" % pkg)
+    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
+    output = p.communicate()[0]
+    if p.returncode != 0:
+        sys.stderr.write("Error getting version %s\n" % pkgs)
+        sys.exit(1)
+    output = output.split("\n")
+    if len(output) != len(pkgs) + 1:
+        sys.stderr.write("Error getting version\n")
+        sys.exit(1)
+    version = {}
+    for i in range(0, len(pkgs)):
+        pkg = pkgs[i]
+        version[pkg] = output[i]
+    return version
+
+# Execute the "make <pkg>-show-depends" command to get the list of
+# dependencies of a given list of packages, and return the list of
+# dependencies formatted as a Python dictionary.
+def get_depends(pkgs):
+    sys.stderr.write("Getting dependencies for %s\n" % pkgs)
+    cmd = ["make", "-s", "--no-print-directory" ]
+    for pkg in pkgs:
+        cmd.append("%s-%s" % (pkg, rule))
+    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
+    output = p.communicate()[0]
+    if p.returncode != 0:
+        sys.stderr.write("Error getting dependencies %s\n" % pkgs)
+        sys.exit(1)
+    output = output.split("\n")
+    if len(output) != len(pkgs) + 1:
+        sys.stderr.write("Error getting dependencies\n")
+        sys.exit(1)
+    deps = {}
+    for i in range(0, len(pkgs)):
+        pkg = pkgs[i]
+        pkg_deps = output[i].split(" ")
+        if pkg_deps == ['']:
+            deps[pkg] = []
+        else:
+            deps[pkg] = pkg_deps
+    return deps
-- 
2.10.2

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>

Functions to obtain the version and dependencies of a package from Python
can be useful for several scripts. Extract this logic out of graph-depends
into pkgutil.py.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
---
 support/scripts/graph-depends | 55 ++++---------------------------------------
 support/scripts/pkgutil.py    | 55 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+), 51 deletions(-)
 create mode 100644 support/scripts/pkgutil.py

diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
index 095619a..7aedcb5 100755
--- a/support/scripts/graph-depends
+++ b/support/scripts/graph-depends
@@ -26,6 +26,8 @@ import subprocess
 import argparse
 from fnmatch import fnmatch
 
+import pkgutil
+
 # Modes of operation:
 MODE_FULL = 1   # draw full dependency graph for all selected packages
 MODE_PKG  = 2   # draw dependency graph for a given package
@@ -122,28 +124,6 @@ host_colour = colours[2]
 
 allpkgs = []
 
-# Execute the "make <pkg>-show-version" command to get the version of a given
-# list of packages, and return the version formatted as a Python dictionary.
-def get_version(pkgs):
-    sys.stderr.write("Getting version for %s\n" % pkgs)
-    cmd = ["make", "-s", "--no-print-directory" ]
-    for pkg in pkgs:
-        cmd.append("%s-show-version" % pkg)
-    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
-    output = p.communicate()[0]
-    if p.returncode != 0:
-        sys.stderr.write("Error getting version %s\n" % pkgs)
-        sys.exit(1)
-    output = output.split("\n")
-    if len(output) != len(pkgs) + 1:
-        sys.stderr.write("Error getting version\n")
-        sys.exit(1)
-    version = {}
-    for i in range(0, len(pkgs)):
-        pkg = pkgs[i]
-        version[pkg] = output[i]
-    return version
-
 # Execute the "make show-targets" command to get the list of the main
 # Buildroot PACKAGES and return it formatted as a Python list. This
 # list is used as the starting point for full dependency graphs
@@ -158,33 +138,6 @@ def get_targets():
         return []
     return output.split(' ')
 
-# Execute the "make <pkg>-show-depends" command to get the list of
-# dependencies of a given list of packages, and return the list of
-# dependencies formatted as a Python dictionary.
-def get_depends(pkgs, rule):
-    sys.stderr.write("Getting dependencies for %s\n" % pkgs)
-    cmd = ["make", "-s", "--no-print-directory" ]
-    for pkg in pkgs:
-        cmd.append("%s-%s" % (pkg, rule))
-    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
-    output = p.communicate()[0]
-    if p.returncode != 0:
-        sys.stderr.write("Error getting dependencies %s\n" % pkgs)
-        sys.exit(1)
-    output = output.split("\n")
-    if len(output) != len(pkgs) + 1:
-        sys.stderr.write("Error getting dependencies\n")
-        sys.exit(1)
-    deps = {}
-    for i in range(0, len(pkgs)):
-        pkg = pkgs[i]
-        pkg_deps = output[i].split(" ")
-        if pkg_deps == ['']:
-            deps[pkg] = []
-        else:
-            deps[pkg] = pkg_deps
-    return deps
-
 # Recursive function that builds the tree of dependencies for a given
 # list of packages. The dependencies are built in a list called
 # 'dependencies', which contains tuples of the form (pkg1 ->
@@ -204,7 +157,7 @@ def get_all_depends(pkgs):
     if len(filtered_pkgs) == 0:
         return []
 
-    depends = get_depends(filtered_pkgs, rule)
+    depends = pkgutil.get_depends(filtered_pkgs, rule)
 
     deps = set()
     for pkg in filtered_pkgs:
@@ -377,7 +330,7 @@ if check_only:
     sys.exit(0)
 
 dict_deps = remove_extra_deps(dict_deps)
-dict_version = get_version([pkg for pkg in allpkgs
+dict_version = pkgutil.get_version([pkg for pkg in allpkgs
                                 if pkg != "all" and not pkg.startswith("root")])
 
 # Print the attributes of a node: label and fill-color
diff --git a/support/scripts/pkgutil.py b/support/scripts/pkgutil.py
new file mode 100644
index 0000000..a911123
--- /dev/null
+++ b/support/scripts/pkgutil.py
@@ -0,0 +1,55 @@
+#!/usr/bin/env python
+
+# Copyright (C) 2010-2013 Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+
+import sys
+import subprocess
+
+# Execute the "make <pkg>-show-version" command to get the version of a given
+# list of packages, and return the version formatted as a Python dictionary.
+def get_version(pkgs):
+    sys.stderr.write("Getting version for %s\n" % pkgs)
+    cmd = ["make", "-s", "--no-print-directory" ]
+    for pkg in pkgs:
+        cmd.append("%s-show-version" % pkg)
+    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
+    output = p.communicate()[0]
+    if p.returncode != 0:
+        sys.stderr.write("Error getting version %s\n" % pkgs)
+        sys.exit(1)
+    output = output.split("\n")
+    if len(output) != len(pkgs) + 1:
+        sys.stderr.write("Error getting version\n")
+        sys.exit(1)
+    version = {}
+    for i in range(0, len(pkgs)):
+        pkg = pkgs[i]
+        version[pkg] = output[i]
+    return version
+
+# Execute the "make <pkg>-show-depends" command to get the list of
+# dependencies of a given list of packages, and return the list of
+# dependencies formatted as a Python dictionary.
+def get_depends(pkgs, rule):
+    sys.stderr.write("Getting dependencies for %s\n" % pkgs)
+    cmd = ["make", "-s", "--no-print-directory" ]
+    for pkg in pkgs:
+        cmd.append("%s-%s" % (pkg, rule))
+    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
+    output = p.communicate()[0]
+    if p.returncode != 0:
+        sys.stderr.write("Error getting dependencies %s\n" % pkgs)
+        sys.exit(1)
+    output = output.split("\n")
+    if len(output) != len(pkgs) + 1:
+        sys.stderr.write("Error getting dependencies\n")
+        sys.exit(1)
+    deps = {}
+    for i in range(0, len(pkgs)):
+        pkg = pkgs[i]
+        pkg_deps = output[i].split(" ")
+        if pkg_deps == ['']:
+            deps[pkg] = []
+        else:
+            deps[pkg] = pkg_deps
+    return deps
-- 
2.10.2

[Buildroot] [PATCH 3/3] graph-depends: split off get_rdepends from get_depends

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>

The use of a 'rule' variable that can contain 'show-depends' or
'show-rdepends' is not logical if get_depends is considered as a reusable
function from various scripts. The name of these rules are too much an
implementation detail.

Therefore, split the existing get_depends into two separate functions
get_depends and get_rdepends, while keeping code duplication to a minimum.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
---
 support/scripts/graph-depends |  6 +++---
 support/scripts/pkgutil.py    | 17 +++++++++++++----
 2 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
index 7aedcb5..fbd5917 100755
--- a/support/scripts/graph-depends
+++ b/support/scripts/graph-depends
@@ -102,13 +102,13 @@ else:
 transitive = args.transitive
 
 if args.direct:
-    rule = "show-depends"
+    get_depends_func = pkgutil.get_depends
     arrow_dir = "forward"
 else:
     if mode == MODE_FULL:
         sys.stderr.write("--reverse needs a package\n")
         sys.exit(1)
-    rule = "show-rdepends"
+    get_depends_func = pkgutil.get_rdepends
     arrow_dir = "back"
 
 # Get the colours: we need exactly three colours,
@@ -157,7 +157,7 @@ def get_all_depends(pkgs):
     if len(filtered_pkgs) == 0:
         return []
 
-    depends = pkgutil.get_depends(filtered_pkgs, rule)
+    depends = get_depends_func(filtered_pkgs)
 
     deps = set()
     for pkg in filtered_pkgs:
diff --git a/support/scripts/pkgutil.py b/support/scripts/pkgutil.py
index a911123..cd1bbd5 100644
--- a/support/scripts/pkgutil.py
+++ b/support/scripts/pkgutil.py
@@ -27,10 +27,7 @@ def get_version(pkgs):
         version[pkg] = output[i]
     return version
 
-# Execute the "make <pkg>-show-depends" command to get the list of
-# dependencies of a given list of packages, and return the list of
-# dependencies formatted as a Python dictionary.
-def get_depends(pkgs, rule):
+def _get_depends(pkgs, rule):
     sys.stderr.write("Getting dependencies for %s\n" % pkgs)
     cmd = ["make", "-s", "--no-print-directory" ]
     for pkg in pkgs:
@@ -53,3 +50,15 @@ def get_depends(pkgs, rule):
         else:
             deps[pkg] = pkg_deps
     return deps
+
+# Execute the "make <pkg>-show-depends" command to get the list of
+# dependencies of a given list of packages, and return the list of
+# dependencies formatted as a Python dictionary.
+def get_depends(pkgs):
+    return _get_depends(pkgs, 'show-depends')
+
+# Execute the "make <pkg>-show-rdepends" command to get the list of
+# reverse dependencies of a given list of packages, and return the
+# list of dependencies formatted as a Python dictionary.
+def get_rdepends(pkgs):
+    return _get_depends(pkgs, 'show-rdepends')
-- 
2.10.2

[Buildroot] [PATCH 1/1] support/scripts: split off get_version/get_depends into pkgutil.py

[Thomas De Schampheleire]

On Fri, Feb 3, 2017 at 9:57 PM, Thomas De Schampheleire
<patrickdepinguin@gmail.com> wrote:
> From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
>
> Functions to obtain the version and dependencies of a package from Python
> can be useful for several scripts. Extract this logic out of graph-depends
> into pkgutil.py.
>
> Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> ---
>  support/scripts/graph-depends | 55 ++++---------------------------------------
>  support/scripts/pkgutil.py    | 55 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 59 insertions(+), 51 deletions(-)
>  create mode 100644 support/scripts/pkgutil.py
>

Please disregard this specific patch, it does not work on current
Buildroot. It was accidentally sent along with the real 3-patch series
(which does work).

[Buildroot] [PATCH 1/3] graph-depends: avoid use of global var 'rule' in get_depends

[Thomas Petazzoni]

Hello,

On Fri,  3 Feb 2017 21:57:42 +0100, Thomas De Schampheleire wrote:
> From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> 
> Function get_depends was recently changed to support both normal
> dependencies as reverse dependencies, via a global variable 'rule' that
> equals 'show-depends' or 'show-rdepends'.
> 
> As a subsequent function will extract this function get_depends to a
> separate file, the use of globals is problematic.
> 
> Instead, pass the global as an argument.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> ---
>  support/scripts/graph-depends | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

Applied to master, thanks. We really need to clean up this script to
move the "global" code into a main() function. Right now there is some
global code intermixed with sub-functions, it's horrible.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 1/3] graph-depends: avoid use of global var 'rule' in get_depends

[Thomas De Schampheleire]

On Sun, Feb 5, 2017 at 3:20 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Hello,
>
> On Fri,  3 Feb 2017 21:57:42 +0100, Thomas De Schampheleire wrote:
>> From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
>>
>> Function get_depends was recently changed to support both normal
>> dependencies as reverse dependencies, via a global variable 'rule' that
>> equals 'show-depends' or 'show-rdepends'.
>>
>> As a subsequent function will extract this function get_depends to a
>> separate file, the use of globals is problematic.
>>
>> Instead, pass the global as an argument.
>>
>> Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
>> ---
>>  support/scripts/graph-depends | 4 ++--
>>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> Applied to master, thanks. We really need to clean up this script to
> move the "global" code into a main() function. Right now there is some
> global code intermixed with sub-functions, it's horrible.

Thanks.
Yes, I agree to move the global code into main. I didn't do it in this
series to not wander off too much of the real goal, but since you have
the same idea I can do it as a follow-up later.

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Yann E. MORIN]

Thomas DS, All,

On 2017-02-03 21:57 +0100, Thomas De Schampheleire spake thusly:
[--SNIP--]
> +import pkgutil

So, I always claimed I was not a Python expoert, but should we not do:

    from okgutil import get_versions get_depends

of something similar?

[--SNIP--]
> @@ -204,7 +157,7 @@ def get_all_depends(pkgs):
>      if len(filtered_pkgs) == 0:
>          return []
>  
> -    depends = get_depends(filtered_pkgs, rule)
> +    depends = pkgutil.get_depends(filtered_pkgs, rule)

And then just use get_depends here?

>      deps = set()
>      for pkg in filtered_pkgs:
> @@ -377,7 +330,7 @@ if check_only:
>      sys.exit(0)
>  
>  dict_deps = remove_extra_deps(dict_deps)
> -dict_version = get_version([pkg for pkg in allpkgs
> +dict_version = pkgutil.get_version([pkg for pkg in allpkgs

Ditto?

>                                  if pkg != "all" and not pkg.startswith("root")])
>  
>  # Print the attributes of a node: label and fill-color
> diff --git a/support/scripts/pkgutil.py b/support/scripts/pkgutil.py
> new file mode 100644
> index 0000000..a911123
> --- /dev/null
> +++ b/support/scripts/pkgutil.py
> @@ -0,0 +1,55 @@
> +#!/usr/bin/env python

As I understand it, this file is not supposed to be standalone, but just
a "library" of functions?

So it should not need to have the sha-bang line, yes?

Otherwise, I'm not opposed to this.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 3/3] graph-depends: split off get_rdepends from get_depends

[Yann E. MORIN]

Thomas DS, All,

On 2017-02-03 21:57 +0100, Thomas De Schampheleire spake thusly:
> From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> 
> The use of a 'rule' variable that can contain 'show-depends' or
> 'show-rdepends' is not logical if get_depends is considered as a reusable
> function from various scripts. The name of these rules are too much an
> implementation detail.
> 
> Therefore, split the existing get_depends into two separate functions
> get_depends and get_rdepends, while keeping code duplication to a minimum.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>

Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Regards,
Yann E. MORIN.

> ---
>  support/scripts/graph-depends |  6 +++---
>  support/scripts/pkgutil.py    | 17 +++++++++++++----
>  2 files changed, 16 insertions(+), 7 deletions(-)
> 
> diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
> index 7aedcb5..fbd5917 100755
> --- a/support/scripts/graph-depends
> +++ b/support/scripts/graph-depends
> @@ -102,13 +102,13 @@ else:
>  transitive = args.transitive
>  
>  if args.direct:
> -    rule = "show-depends"
> +    get_depends_func = pkgutil.get_depends
>      arrow_dir = "forward"
>  else:
>      if mode == MODE_FULL:
>          sys.stderr.write("--reverse needs a package\n")
>          sys.exit(1)
> -    rule = "show-rdepends"
> +    get_depends_func = pkgutil.get_rdepends
>      arrow_dir = "back"
>  
>  # Get the colours: we need exactly three colours,
> @@ -157,7 +157,7 @@ def get_all_depends(pkgs):
>      if len(filtered_pkgs) == 0:
>          return []
>  
> -    depends = pkgutil.get_depends(filtered_pkgs, rule)
> +    depends = get_depends_func(filtered_pkgs)
>  
>      deps = set()
>      for pkg in filtered_pkgs:
> diff --git a/support/scripts/pkgutil.py b/support/scripts/pkgutil.py
> index a911123..cd1bbd5 100644
> --- a/support/scripts/pkgutil.py
> +++ b/support/scripts/pkgutil.py
> @@ -27,10 +27,7 @@ def get_version(pkgs):
>          version[pkg] = output[i]
>      return version
>  
> -# Execute the "make <pkg>-show-depends" command to get the list of
> -# dependencies of a given list of packages, and return the list of
> -# dependencies formatted as a Python dictionary.
> -def get_depends(pkgs, rule):
> +def _get_depends(pkgs, rule):
>      sys.stderr.write("Getting dependencies for %s\n" % pkgs)
>      cmd = ["make", "-s", "--no-print-directory" ]
>      for pkg in pkgs:
> @@ -53,3 +50,15 @@ def get_depends(pkgs, rule):
>          else:
>              deps[pkg] = pkg_deps
>      return deps
> +
> +# Execute the "make <pkg>-show-depends" command to get the list of
> +# dependencies of a given list of packages, and return the list of
> +# dependencies formatted as a Python dictionary.
> +def get_depends(pkgs):
> +    return _get_depends(pkgs, 'show-depends')
> +
> +# Execute the "make <pkg>-show-rdepends" command to get the list of
> +# reverse dependencies of a given list of packages, and return the
> +# list of dependencies formatted as a Python dictionary.
> +def get_rdepends(pkgs):
> +    return _get_depends(pkgs, 'show-rdepends')
> -- 
> 2.10.2
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Thomas De Schampheleire]

On Sun, Feb 5, 2017 at 10:13 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> Thomas DS, All,
>
> On 2017-02-03 21:57 +0100, Thomas De Schampheleire spake thusly:
> [--SNIP--]
>> +import pkgutil
>
> So, I always claimed I was not a Python expoert, but should we not do:
>
>     from okgutil import get_versions get_depends
>
> of something similar?

Both methods are technically possible, and I think this is partially a
matter of personal/project preference, and partially some general
guidelines.
When using 'import foo', any symbol used from foo needs to be
qualified explicitly, i.e. foo.one, foo.two. You could consider it as
making a namespace accessible but keeping everything inside that
namespace.

With 'from foo import one, two' you allow future usages to be done
without referring to foo anymore. I consider it as 'using namespace
foo' in C++ terminology.

One could argue about performance of one vs the other, but I think it
hardly matters (although I did never measure that).

Personally, I'm more in favor of 'import foo' and keeping explicit
references for following reasons:
1. it makes it clear inside the code where a symbol comes from -- in
general I consider 'using namespace foo' a bad thing.
2. maintainability: if more functions are added to foo and used from a
script, the list in the import statement becomes larger and larger, to
possibly ridiculous proportions.

There is one case where I do use 'from Foo import Foo' and that is
when Foo is a class that is saved in its own equally-named file
Foo.py. In this case, Foo.Foo seems a bit redundant and it makes sense
to make the class Foo directly accessible.

I found the following 100% reliable reference (ahum) about the topic:
http://stackoverflow.com/questions/710551/import-module-or-from-module-import


>
> [--SNIP--]
>> @@ -204,7 +157,7 @@ def get_all_depends(pkgs):
>>      if len(filtered_pkgs) == 0:
>>          return []
>>
>> -    depends = get_depends(filtered_pkgs, rule)
>> +    depends = pkgutil.get_depends(filtered_pkgs, rule)
>
> And then just use get_depends here?
>
>>      deps = set()
>>      for pkg in filtered_pkgs:
>> @@ -377,7 +330,7 @@ if check_only:
>>      sys.exit(0)
>>
>>  dict_deps = remove_extra_deps(dict_deps)
>> -dict_version = get_version([pkg for pkg in allpkgs
>> +dict_version = pkgutil.get_version([pkg for pkg in allpkgs
>
> Ditto?
>
>>                                  if pkg != "all" and not pkg.startswith("root")])
>>
>>  # Print the attributes of a node: label and fill-color
>> diff --git a/support/scripts/pkgutil.py b/support/scripts/pkgutil.py
>> new file mode 100644
>> index 0000000..a911123
>> --- /dev/null
>> +++ b/support/scripts/pkgutil.py
>> @@ -0,0 +1,55 @@
>> +#!/usr/bin/env python
>
> As I understand it, this file is not supposed to be standalone, but just
> a "library" of functions?
>
> So it should not need to have the sha-bang line, yes?

Yes, you are right, that line is not necessary.

>
> Otherwise, I'm not opposed to this.

Thanks!

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Yann E. MORIN]

Thomas DS, All,

On 2017-02-05 22:27 +0100, Thomas De Schampheleire spake thusly:
> On Sun, Feb 5, 2017 at 10:13 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > Thomas DS, All,
> >
> > On 2017-02-03 21:57 +0100, Thomas De Schampheleire spake thusly:
> > [--SNIP--]
> >> +import pkgutil
> >
> > So, I always claimed I was not a Python expoert, but should we not do:
> >
> >     from okgutil import get_versions get_depends
> >
> > of something similar?
> 
> Both methods are technically possible, and I think this is partially a
> matter of personal/project preference, and partially some general
> guidelines.

Ok for me, then.

Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Regards,
Yann E. MORIN.

> When using 'import foo', any symbol used from foo needs to be
> qualified explicitly, i.e. foo.one, foo.two. You could consider it as
> making a namespace accessible but keeping everything inside that
> namespace.
> 
> With 'from foo import one, two' you allow future usages to be done
> without referring to foo anymore. I consider it as 'using namespace
> foo' in C++ terminology.
> 
> One could argue about performance of one vs the other, but I think it
> hardly matters (although I did never measure that).
> 
> Personally, I'm more in favor of 'import foo' and keeping explicit
> references for following reasons:
> 1. it makes it clear inside the code where a symbol comes from -- in
> general I consider 'using namespace foo' a bad thing.
> 2. maintainability: if more functions are added to foo and used from a
> script, the list in the import statement becomes larger and larger, to
> possibly ridiculous proportions.
> 
> There is one case where I do use 'from Foo import Foo' and that is
> when Foo is a class that is saved in its own equally-named file
> Foo.py. In this case, Foo.Foo seems a bit redundant and it makes sense
> to make the class Foo directly accessible.
> 
> I found the following 100% reliable reference (ahum) about the topic:
> http://stackoverflow.com/questions/710551/import-module-or-from-module-import
> 
> 
> >
> > [--SNIP--]
> >> @@ -204,7 +157,7 @@ def get_all_depends(pkgs):
> >>      if len(filtered_pkgs) == 0:
> >>          return []
> >>
> >> -    depends = get_depends(filtered_pkgs, rule)
> >> +    depends = pkgutil.get_depends(filtered_pkgs, rule)
> >
> > And then just use get_depends here?
> >
> >>      deps = set()
> >>      for pkg in filtered_pkgs:
> >> @@ -377,7 +330,7 @@ if check_only:
> >>      sys.exit(0)
> >>
> >>  dict_deps = remove_extra_deps(dict_deps)
> >> -dict_version = get_version([pkg for pkg in allpkgs
> >> +dict_version = pkgutil.get_version([pkg for pkg in allpkgs
> >
> > Ditto?
> >
> >>                                  if pkg != "all" and not pkg.startswith("root")])
> >>
> >>  # Print the attributes of a node: label and fill-color
> >> diff --git a/support/scripts/pkgutil.py b/support/scripts/pkgutil.py
> >> new file mode 100644
> >> index 0000000..a911123
> >> --- /dev/null
> >> +++ b/support/scripts/pkgutil.py
> >> @@ -0,0 +1,55 @@
> >> +#!/usr/bin/env python
> >
> > As I understand it, this file is not supposed to be standalone, but just
> > a "library" of functions?
> >
> > So it should not need to have the sha-bang line, yes?
> 
> Yes, you are right, that line is not necessary.
> 
> >
> > Otherwise, I'm not opposed to this.
> 
> Thanks!

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Thomas Petazzoni]

Hello,

On Fri,  3 Feb 2017 21:57:44 +0100, Thomas De Schampheleire wrote:
> From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> 
> Functions to obtain the version and dependencies of a package from Python
> can be useful for several scripts. Extract this logic out of graph-depends
> into pkgutil.py.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> ---
>  support/scripts/graph-depends | 55 ++++---------------------------------------
>  support/scripts/pkgutil.py    | 55 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 59 insertions(+), 51 deletions(-)
>  create mode 100644 support/scripts/pkgutil.py

Applied to master, after removing the shebang from pkgutil.py as
suggested by Yann.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 3/3] graph-depends: split off get_rdepends from get_depends

[Thomas Petazzoni]

Hello,

On Fri,  3 Feb 2017 21:57:45 +0100, Thomas De Schampheleire wrote:
> From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> 
> The use of a 'rule' variable that can contain 'show-depends' or
> 'show-rdepends' is not logical if get_depends is considered as a reusable
> function from various scripts. The name of these rules are too much an
> implementation detail.
> 
> Therefore, split the existing get_depends into two separate functions
> get_depends and get_rdepends, while keeping code duplication to a minimum.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> ---
>  support/scripts/graph-depends |  6 +++---
>  support/scripts/pkgutil.py    | 17 +++++++++++++----
>  2 files changed, 16 insertions(+), 7 deletions(-)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Yann E. MORIN]

Thomas DS, All,

On 2017-02-03 21:57 +0100, Thomas De Schampheleire spake thusly:
> From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> 
> Functions to obtain the version and dependencies of a package from Python
> can be useful for several scripts. Extract this logic out of graph-depends
> into pkgutil.py.

Coming back to this script, because I'm rewriting the way graph-depends
gets the dependency tree. When you said "useful for several scripts,"
did you expect it to be useful to scripts that are not in Buildroot
(e.g. user-local scripts)?

Regards,
Yann E. MORIN.

> Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> ---
>  support/scripts/graph-depends | 55 ++++---------------------------------------
>  support/scripts/pkgutil.py    | 55 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 59 insertions(+), 51 deletions(-)
>  create mode 100644 support/scripts/pkgutil.py
> 
> diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends
> index 095619a..7aedcb5 100755
> --- a/support/scripts/graph-depends
> +++ b/support/scripts/graph-depends
> @@ -26,6 +26,8 @@ import subprocess
>  import argparse
>  from fnmatch import fnmatch
>  
> +import pkgutil
> +
>  # Modes of operation:
>  MODE_FULL = 1   # draw full dependency graph for all selected packages
>  MODE_PKG  = 2   # draw dependency graph for a given package
> @@ -122,28 +124,6 @@ host_colour = colours[2]
>  
>  allpkgs = []
>  
> -# Execute the "make <pkg>-show-version" command to get the version of a given
> -# list of packages, and return the version formatted as a Python dictionary.
> -def get_version(pkgs):
> -    sys.stderr.write("Getting version for %s\n" % pkgs)
> -    cmd = ["make", "-s", "--no-print-directory" ]
> -    for pkg in pkgs:
> -        cmd.append("%s-show-version" % pkg)
> -    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
> -    output = p.communicate()[0]
> -    if p.returncode != 0:
> -        sys.stderr.write("Error getting version %s\n" % pkgs)
> -        sys.exit(1)
> -    output = output.split("\n")
> -    if len(output) != len(pkgs) + 1:
> -        sys.stderr.write("Error getting version\n")
> -        sys.exit(1)
> -    version = {}
> -    for i in range(0, len(pkgs)):
> -        pkg = pkgs[i]
> -        version[pkg] = output[i]
> -    return version
> -
>  # Execute the "make show-targets" command to get the list of the main
>  # Buildroot PACKAGES and return it formatted as a Python list. This
>  # list is used as the starting point for full dependency graphs
> @@ -158,33 +138,6 @@ def get_targets():
>          return []
>      return output.split(' ')
>  
> -# Execute the "make <pkg>-show-depends" command to get the list of
> -# dependencies of a given list of packages, and return the list of
> -# dependencies formatted as a Python dictionary.
> -def get_depends(pkgs, rule):
> -    sys.stderr.write("Getting dependencies for %s\n" % pkgs)
> -    cmd = ["make", "-s", "--no-print-directory" ]
> -    for pkg in pkgs:
> -        cmd.append("%s-%s" % (pkg, rule))
> -    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
> -    output = p.communicate()[0]
> -    if p.returncode != 0:
> -        sys.stderr.write("Error getting dependencies %s\n" % pkgs)
> -        sys.exit(1)
> -    output = output.split("\n")
> -    if len(output) != len(pkgs) + 1:
> -        sys.stderr.write("Error getting dependencies\n")
> -        sys.exit(1)
> -    deps = {}
> -    for i in range(0, len(pkgs)):
> -        pkg = pkgs[i]
> -        pkg_deps = output[i].split(" ")
> -        if pkg_deps == ['']:
> -            deps[pkg] = []
> -        else:
> -            deps[pkg] = pkg_deps
> -    return deps
> -
>  # Recursive function that builds the tree of dependencies for a given
>  # list of packages. The dependencies are built in a list called
>  # 'dependencies', which contains tuples of the form (pkg1 ->
> @@ -204,7 +157,7 @@ def get_all_depends(pkgs):
>      if len(filtered_pkgs) == 0:
>          return []
>  
> -    depends = get_depends(filtered_pkgs, rule)
> +    depends = pkgutil.get_depends(filtered_pkgs, rule)
>  
>      deps = set()
>      for pkg in filtered_pkgs:
> @@ -377,7 +330,7 @@ if check_only:
>      sys.exit(0)
>  
>  dict_deps = remove_extra_deps(dict_deps)
> -dict_version = get_version([pkg for pkg in allpkgs
> +dict_version = pkgutil.get_version([pkg for pkg in allpkgs
>                                  if pkg != "all" and not pkg.startswith("root")])
>  
>  # Print the attributes of a node: label and fill-color
> diff --git a/support/scripts/pkgutil.py b/support/scripts/pkgutil.py
> new file mode 100644
> index 0000000..a911123
> --- /dev/null
> +++ b/support/scripts/pkgutil.py
> @@ -0,0 +1,55 @@
> +#!/usr/bin/env python
> +
> +# Copyright (C) 2010-2013 Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> +
> +import sys
> +import subprocess
> +
> +# Execute the "make <pkg>-show-version" command to get the version of a given
> +# list of packages, and return the version formatted as a Python dictionary.
> +def get_version(pkgs):
> +    sys.stderr.write("Getting version for %s\n" % pkgs)
> +    cmd = ["make", "-s", "--no-print-directory" ]
> +    for pkg in pkgs:
> +        cmd.append("%s-show-version" % pkg)
> +    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
> +    output = p.communicate()[0]
> +    if p.returncode != 0:
> +        sys.stderr.write("Error getting version %s\n" % pkgs)
> +        sys.exit(1)
> +    output = output.split("\n")
> +    if len(output) != len(pkgs) + 1:
> +        sys.stderr.write("Error getting version\n")
> +        sys.exit(1)
> +    version = {}
> +    for i in range(0, len(pkgs)):
> +        pkg = pkgs[i]
> +        version[pkg] = output[i]
> +    return version
> +
> +# Execute the "make <pkg>-show-depends" command to get the list of
> +# dependencies of a given list of packages, and return the list of
> +# dependencies formatted as a Python dictionary.
> +def get_depends(pkgs, rule):
> +    sys.stderr.write("Getting dependencies for %s\n" % pkgs)
> +    cmd = ["make", "-s", "--no-print-directory" ]
> +    for pkg in pkgs:
> +        cmd.append("%s-%s" % (pkg, rule))
> +    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
> +    output = p.communicate()[0]
> +    if p.returncode != 0:
> +        sys.stderr.write("Error getting dependencies %s\n" % pkgs)
> +        sys.exit(1)
> +    output = output.split("\n")
> +    if len(output) != len(pkgs) + 1:
> +        sys.stderr.write("Error getting dependencies\n")
> +        sys.exit(1)
> +    deps = {}
> +    for i in range(0, len(pkgs)):
> +        pkg = pkgs[i]
> +        pkg_deps = output[i].split(" ")
> +        if pkg_deps == ['']:
> +            deps[pkg] = []
> +        else:
> +            deps[pkg] = pkg_deps
> +    return deps
> -- 
> 2.10.2
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Thomas De Schampheleire]

Hi Yann,
El mi?., 7 nov. 2018 a las 19:07, Yann E. MORIN
(<yann.morin.1998@free.fr>) escribi?:
>
> Thomas DS, All,
>
> On 2017-02-03 21:57 +0100, Thomas De Schampheleire spake thusly:
> > From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> >
> > Functions to obtain the version and dependencies of a package from Python
> > can be useful for several scripts. Extract this logic out of graph-depends
> > into pkgutil.py.
>
> Coming back to this script, because I'm rewriting the way graph-depends
> gets the dependency tree. When you said "useful for several scripts,"
> did you expect it to be useful to scripts that are not in Buildroot
> (e.g. user-local scripts)?

Yes, exactly. We are using the logic from pkgutil from another python
script and want to avoid code duplication.
This particular script is not something that upstream accepts, i.e.
generating opkg files for specific packages (and for that, we need to
know the dependencies and versions of each package).

Best regards,
Thomas

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Yann E. MORIN]

Thomas, All,

On 2018-11-07 20:06 +0100, Thomas De Schampheleire spake thusly:
> El mi?., 7 nov. 2018 a las 19:07, Yann E. MORIN
> (<yann.morin.1998@free.fr>) escribi?:
> > On 2017-02-03 21:57 +0100, Thomas De Schampheleire spake thusly:
> > > From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> > >
> > > Functions to obtain the version and dependencies of a package from Python
> > > can be useful for several scripts. Extract this logic out of graph-depends
> > > into pkgutil.py.
> >
> > Coming back to this script, because I'm rewriting the way graph-depends
> > gets the dependency tree. When you said "useful for several scripts,"
> > did you expect it to be useful to scripts that are not in Buildroot
> > (e.g. user-local scripts)?
> 
> Yes, exactly. We are using the logic from pkgutil from another python
> script and want to avoid code duplication.
> This particular script is not something that upstream accepts, i.e.
> generating opkg files for specific packages (and for that, we need to
> know the dependencies and versions of each package).

So, if you were to get a new function that would return basically the
same, but in another format, but much quicker (~4s instead of ~45s),
would that be something you could adapt to?

I'm changing the way the dependency tree is extracted from the Makefile
data, and the function now has this API (function name yet to be
bike-shedded about):

    dict_deps, dict_types = get_dependency_tree(direction)

where:

  - direction is either string 'forward' or 'back',

  - dict_deps is a dictionnary, which keys are the package names, and
    which values are lists of packages that are direct dependencies of
    the key package (basically, what get_all_depends() currently
    returns, but the whole dependency tree)

  - dict_types is a dictionnary, which keys are the package names, and
    the values are string representing whtehr the packages are target or
    host, and virtual or not, e.g.: 'target', 'target-virtual', 'host',
    'host-virtual'.

If needed, I could very easily make it return a three-tuple, with the
third one being dict_versions, keyed by package names, with values the
package version.

Would that be something that would be usable in your use case?

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Thomas De Schampheleire]

Hi Yann,

El mi?., 7 nov. 2018 a las 20:39, Yann E. MORIN
(<yann.morin.1998@free.fr>) escribi?:
>
> Thomas, All,
>
> On 2018-11-07 20:06 +0100, Thomas De Schampheleire spake thusly:
> > El mi?., 7 nov. 2018 a las 19:07, Yann E. MORIN
> > (<yann.morin.1998@free.fr>) escribi?:
> > > On 2017-02-03 21:57 +0100, Thomas De Schampheleire spake thusly:
> > > > From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> > > >
> > > > Functions to obtain the version and dependencies of a package from Python
> > > > can be useful for several scripts. Extract this logic out of graph-depends
> > > > into pkgutil.py.
> > >
> > > Coming back to this script, because I'm rewriting the way graph-depends
> > > gets the dependency tree. When you said "useful for several scripts,"
> > > did you expect it to be useful to scripts that are not in Buildroot
> > > (e.g. user-local scripts)?
> >
> > Yes, exactly. We are using the logic from pkgutil from another python
> > script and want to avoid code duplication.
> > This particular script is not something that upstream accepts, i.e.
> > generating opkg files for specific packages (and for that, we need to
> > know the dependencies and versions of each package).
>
> So, if you were to get a new function that would return basically the
> same, but in another format, but much quicker (~4s instead of ~45s),
> would that be something you could adapt to?
>
> I'm changing the way the dependency tree is extracted from the Makefile
> data, and the function now has this API (function name yet to be
> bike-shedded about):
>
>     dict_deps, dict_types = get_dependency_tree(direction)
>
> where:
>
>   - direction is either string 'forward' or 'back',
>
>   - dict_deps is a dictionnary, which keys are the package names, and
>     which values are lists of packages that are direct dependencies of
>     the key package (basically, what get_all_depends() currently
>     returns, but the whole dependency tree)

How is dict_deps different from what get_depends in brpkgutil.py
currently returns? Your statement "but the whole dependency tree" is
not clear to me.

>
>   - dict_types is a dictionnary, which keys are the package names, and
>     the values are string representing whtehr the packages are target or
>     host, and virtual or not, e.g.: 'target', 'target-virtual', 'host',
>     'host-virtual'.
>
> If needed, I could very easily make it return a three-tuple, with the
> third one being dict_versions, keyed by package names, with values the
> package version.
>
> Would that be something that would be usable in your use case?

To help the discussion, here are some snippets from our code:

        Package.versions = brpkgutil.get_version(list(Package.files.keys()))
        Package.depends = brpkgutil.get_depends(list(Package.files.keys()))
        ...
        self.version = Package.versions[self.pkg]        # this
variable will be used in several places, we just need the string
representing the version
    ...
    def get_opkg_depends(self):
        """Return the dependencies that need to be declared in the opkg"""
        deps = set(Package.depends[self.pkg]).intersection(set(Package.requested_pkgs))
        if deps is not None:
            return ','.join(deps)
        else:
            return ''

The code above is basically all we do with the version and dependencies.
As long as we can get this info in some way, I'm fine with changes.
Returning the versions together with the dependencies in one call,
removing the call to brpkgutil.get_version is fine by me, if it is
better for the in-tree usage (I'm fine with either way).

Thanks,
Thomas

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Yann E. MORIN]

On 2018-11-07 21:41 +0100, Thomas De Schampheleire spake thusly:
> El mi?., 7 nov. 2018 a las 20:39, Yann E. MORIN
> (<yann.morin.1998@free.fr>) escribi?:
> > On 2018-11-07 20:06 +0100, Thomas De Schampheleire spake thusly:
> > > El mi?., 7 nov. 2018 a las 19:07, Yann E. MORIN
> > > (<yann.morin.1998@free.fr>) escribi?:
> > > > On 2017-02-03 21:57 +0100, Thomas De Schampheleire spake thusly:
> > > > > From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> > > > >
> > > > > Functions to obtain the version and dependencies of a package from Python
> > > > > can be useful for several scripts. Extract this logic out of graph-depends
> > > > > into pkgutil.py.
> > > >
> > > > Coming back to this script, because I'm rewriting the way graph-depends
> > > > gets the dependency tree. When you said "useful for several scripts,"
> > > > did you expect it to be useful to scripts that are not in Buildroot
> > > > (e.g. user-local scripts)?
> > >
> > > Yes, exactly. We are using the logic from pkgutil from another python
> > > script and want to avoid code duplication.
> > > This particular script is not something that upstream accepts, i.e.
> > > generating opkg files for specific packages (and for that, we need to
> > > know the dependencies and versions of each package).
> >
> > So, if you were to get a new function that would return basically the
> > same, but in another format, but much quicker (~4s instead of ~45s),
> > would that be something you could adapt to?
> >
> > I'm changing the way the dependency tree is extracted from the Makefile
> > data, and the function now has this API (function name yet to be
> > bike-shedded about):
> >
> >     dict_deps, dict_types = get_dependency_tree(direction)
> >
> > where:
> >
> >   - direction is either string 'forward' or 'back',
> >
> >   - dict_deps is a dictionnary, which keys are the package names, and
> >     which values are lists of packages that are direct dependencies of
> >     the key package (basically, what get_all_depends() currently
> >     returns, but the whole dependency tree)
> 
> How is dict_deps different from what get_depends in brpkgutil.py
> currently returns? Your statement "but the whole dependency tree" is
> not clear to me.

Currently, brpkgutil.get_depends() takes a list of packages for which
to get the dependencies. The new get_dependency_tree() does not take
such a list; it always returns the whole dependency tree (i.e. from the
'ALL' meta package).

> >   - dict_types is a dictionnary, which keys are the package names, and
> >     the values are string representing whtehr the packages are target or
> >     host, and virtual or not, e.g.: 'target', 'target-virtual', 'host',
> >     'host-virtual'.
> >
> > If needed, I could very easily make it return a three-tuple, with the
> > third one being dict_versions, keyed by package names, with values the
> > package version.
> >
> > Would that be something that would be usable in your use case?
> 
> To help the discussion, here are some snippets from our code:
> 
>         Package.versions = brpkgutil.get_version(list(Package.files.keys()))
>         Package.depends = brpkgutil.get_depends(list(Package.files.keys()))

So you could write:

    Package.depends, Package.versions = brpkgutil.get_dependency_tree('forward')

>         self.version = Package.versions[self.pkg]        # this
> variable will be used in several places, we just need the string
> representing the version
>     ...
>     def get_opkg_depends(self):
>         """Return the dependencies that need to be declared in the opkg"""
>         deps = set(Package.depends[self.pkg]).intersection(set(Package.requested_pkgs))
>         if deps is not None:
>             return ','.join(deps)
>         else:
>             return ''
> 
> The code above is basically all we do with the version and dependencies.
> As long as we can get this info in some way, I'm fine with changes.
> Returning the versions together with the dependencies in one call,
> removing the call to brpkgutil.get_version is fine by me, if it is
> better for the in-tree usage (I'm fine with either way).

OK, good.

Currently, in Buildroot, we use the version string just because we need
to know whether a package is virutal or not. Do you really need the
version string for what it is, or just to test if it is virtual or not?

Regards,
Yann E. MORIN.

> Thanks,
> Thomas

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Thomas De Schampheleire]

El mi?., 7 nov. 2018 a las 22:07, Yann E. MORIN
(<yann.morin.1998@free.fr>) escribi?:
>
> On 2018-11-07 21:41 +0100, Thomas De Schampheleire spake thusly:
> > El mi?., 7 nov. 2018 a las 20:39, Yann E. MORIN
> > (<yann.morin.1998@free.fr>) escribi?:
> > > On 2018-11-07 20:06 +0100, Thomas De Schampheleire spake thusly:
> > > > El mi?., 7 nov. 2018 a las 19:07, Yann E. MORIN
> > > > (<yann.morin.1998@free.fr>) escribi?:
> > > > > On 2017-02-03 21:57 +0100, Thomas De Schampheleire spake thusly:
> > > > > > From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> > > > > >
> > > > > > Functions to obtain the version and dependencies of a package from Python
> > > > > > can be useful for several scripts. Extract this logic out of graph-depends
> > > > > > into pkgutil.py.
> > > > >
> > > > > Coming back to this script, because I'm rewriting the way graph-depends
> > > > > gets the dependency tree. When you said "useful for several scripts,"
> > > > > did you expect it to be useful to scripts that are not in Buildroot
> > > > > (e.g. user-local scripts)?
> > > >
> > > > Yes, exactly. We are using the logic from pkgutil from another python
> > > > script and want to avoid code duplication.
> > > > This particular script is not something that upstream accepts, i.e.
> > > > generating opkg files for specific packages (and for that, we need to
> > > > know the dependencies and versions of each package).
> > >
> > > So, if you were to get a new function that would return basically the
> > > same, but in another format, but much quicker (~4s instead of ~45s),
> > > would that be something you could adapt to?
> > >
> > > I'm changing the way the dependency tree is extracted from the Makefile
> > > data, and the function now has this API (function name yet to be
> > > bike-shedded about):
> > >
> > >     dict_deps, dict_types = get_dependency_tree(direction)
> > >
> > > where:
> > >
> > >   - direction is either string 'forward' or 'back',
> > >
> > >   - dict_deps is a dictionnary, which keys are the package names, and
> > >     which values are lists of packages that are direct dependencies of
> > >     the key package (basically, what get_all_depends() currently
> > >     returns, but the whole dependency tree)
> >
> > How is dict_deps different from what get_depends in brpkgutil.py
> > currently returns? Your statement "but the whole dependency tree" is
> > not clear to me.
>
> Currently, brpkgutil.get_depends() takes a list of packages for which
> to get the dependencies. The new get_dependency_tree() does not take
> such a list; it always returns the whole dependency tree (i.e. from the
> 'ALL' meta package).

Ok, thanks.

>
> > >   - dict_types is a dictionnary, which keys are the package names, and
> > >     the values are string representing whtehr the packages are target or
> > >     host, and virtual or not, e.g.: 'target', 'target-virtual', 'host',
> > >     'host-virtual'.
> > >
> > > If needed, I could very easily make it return a three-tuple, with the
> > > third one being dict_versions, keyed by package names, with values the
> > > package version.
> > >
> > > Would that be something that would be usable in your use case?
> >
> > To help the discussion, here are some snippets from our code:
> >
> >         Package.versions = brpkgutil.get_version(list(Package.files.keys()))
> >         Package.depends = brpkgutil.get_depends(list(Package.files.keys()))
>
> So you could write:
>
>     Package.depends, Package.versions = brpkgutil.get_dependency_tree('forward')
>
> >         self.version = Package.versions[self.pkg]        # this
> > variable will be used in several places, we just need the string
> > representing the version
> >     ...
> >     def get_opkg_depends(self):
> >         """Return the dependencies that need to be declared in the opkg"""
> >         deps = set(Package.depends[self.pkg]).intersection(set(Package.requested_pkgs))
> >         if deps is not None:
> >             return ','.join(deps)
> >         else:
> >             return ''
> >
> > The code above is basically all we do with the version and dependencies.
> > As long as we can get this info in some way, I'm fine with changes.
> > Returning the versions together with the dependencies in one call,
> > removing the call to brpkgutil.get_version is fine by me, if it is
> > better for the in-tree usage (I'm fine with either way).
>
> OK, good.
>
> Currently, in Buildroot, we use the version string just because we need
> to know whether a package is virutal or not. Do you really need the
> version string for what it is, or just to test if it is virtual or not?

We do need the version, which is used in the .ipk filename as well as
in its metadata.
Whether the package is virtual or not is not a concern for us.

Thanks,
Thomas

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Arnout Vandecappelle]

On 07/11/18 20:39, Yann E. MORIN wrote:
> Thomas, All,
> 
> On 2018-11-07 20:06 +0100, Thomas De Schampheleire spake thusly:
>> El mi?., 7 nov. 2018 a las 19:07, Yann E. MORIN
>> (<yann.morin.1998@free.fr>) escribi?:
>>> On 2017-02-03 21:57 +0100, Thomas De Schampheleire spake thusly:
>>>> From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
>>>>
>>>> Functions to obtain the version and dependencies of a package from Python
>>>> can be useful for several scripts. Extract this logic out of graph-depends
>>>> into pkgutil.py.
>>>
>>> Coming back to this script, because I'm rewriting the way graph-depends
>>> gets the dependency tree. When you said "useful for several scripts,"
>>> did you expect it to be useful to scripts that are not in Buildroot
>>> (e.g. user-local scripts)?
>>
>> Yes, exactly. We are using the logic from pkgutil from another python
>> script and want to avoid code duplication.
>> This particular script is not something that upstream accepts, i.e.
>> generating opkg files for specific packages (and for that, we need to
>> know the dependencies and versions of each package).
> 
> So, if you were to get a new function that would return basically the
> same, but in another format, but much quicker (~4s instead of ~45s),
> would that be something you could adapt to?
> 
> I'm changing the way the dependency tree is extracted from the Makefile
> data, and the function now has this API (function name yet to be
> bike-shedded about):

 Let me take this opportunity to bikeshed on this right away :-)

> 
>     dict_deps, dict_types = get_dependency_tree(direction)

 I like the function name! However...

> 
> where:
> 
>   - direction is either string 'forward' or 'back',

 I would make this a boolean argument:

def get_dependency_tree(backward: bool = True) -> Dict[str, Package]:

 Note however that it is trivial to populate the backward dependencies once you
have the forward ones, so I'm not sure its worth it to make the distinction.

> 
>   - dict_deps is a dictionnary, which keys are the package names, and
>     which values are lists of packages that are direct dependencies of
>     the key package (basically, what get_all_depends() currently
>     returns, but the whole dependency tree)
> 
>   - dict_types is a dictionnary, which keys are the package names, and
>     the values are string representing whtehr the packages are target or
>     host, and virtual or not, e.g.: 'target', 'target-virtual', 'host',
>     'host-virtual'.

 I would take this opportunity to create a class Package with members:

name: str                  # Package name including hsot- prefix
dependencies: Set[Package] # Forward dependencies
dependencies_backwards: Set[Package] # Backward dependencies
virtual: bool              # If true, it's a virtual package
host: bool                 # If true, it's a host package
version: str               # Package version

 The __init__ would set name, virtual, host, version. The dependencies are added
dynamically, and whenever you add one you do it in both directions.


 Regards,
 Arnout


> If needed, I could very easily make it return a three-tuple, with the
> third one being dict_versions, keyed by package names, with values the
> package version.
> 
> Would that be something that would be usable in your use case?
> 
> Regards,
> Yann E. MORIN.
> 

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Yann E. MORIN]

Arnout, All,

On 2018-11-07 23:26 +0100, Arnout Vandecappelle spake thusly:
> On 07/11/18 20:39, Yann E. MORIN wrote:
> > I'm changing the way the dependency tree is extracted from the Makefile
> > data, and the function now has this API (function name yet to be
> > bike-shedded about):
>  Let me take this opportunity to bikeshed on this right away :-)

If that were not you, then who else? ;-)

> >     dict_deps, dict_types = get_dependency_tree(direction)
>  I like the function name! However...

Oh come on, that *is* the most important thing! ;-)

> > where:
> > 
> >   - direction is either string 'forward' or 'back',
>  I would make this a boolean argument:
> def get_dependency_tree(backward: bool = True) -> Dict[str, Package]:

Just for my own understanding: what syntax is that? I was trying to write
the protoype of the function, but in Python there is no prototype where
you can explain the return type. Yours seems like it is what I was
looking for. Is it described somewhere?

(See how I'm trying to sidetrack you into not bikeshedding? ;-))

>  Note however that it is trivial to populate the backward dependencies once you
> have the forward ones, so I'm not sure its worth it to make the distinction.

I was also thinking about that too.

> >   - dict_deps is a dictionnary, which keys are the package names, and
> >     which values are lists of packages that are direct dependencies of
> >     the key package (basically, what get_all_depends() currently
> >     returns, but the whole dependency tree)
> > 
> >   - dict_types is a dictionnary, which keys are the package names, and
> >     the values are string representing whtehr the packages are target or
> >     host, and virtual or not, e.g.: 'target', 'target-virtual', 'host',
> >     'host-virtual'.
> 
>  I would take this opportunity to create a class Package with members:
> 
> name: str                  # Package name including hsot- prefix
> dependencies: Set[Package] # Forward dependencies
> dependencies_backwards: Set[Package] # Backward dependencies
> virtual: bool              # If true, it's a virtual package
> host: bool                 # If true, it's a host package
> version: str               # Package version

Meh, I was thinking about exactly the same thing, but it is quite an
invasive change. But let me bikeshed in turn: I would make your proposed
'host' field an enum instead, because we don't have only packages, but
also rootfs. So I would have 'kind' (or 'type' if that's not reserved?)
which could be one of 'target','host', or 'rootfs'.

>  The __init__ would set name, virtual, host, version. The dependencies are added
> dynamically, and whenever you add one you do it in both directions.

Yep.

Dang, I'm always getting side-tracked into side topics. All I wanted was
a faster graph-depends, and it's already there now! Sob... ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Arnout Vandecappelle]

On 07/11/18 23:45, Yann E. MORIN wrote:
> Arnout, All,
> 
> On 2018-11-07 23:26 +0100, Arnout Vandecappelle spake thusly:
>> On 07/11/18 20:39, Yann E. MORIN wrote:
>>> I'm changing the way the dependency tree is extracted from the Makefile
>>> data, and the function now has this API (function name yet to be
>>> bike-shedded about):
>>  Let me take this opportunity to bikeshed on this right away :-)
> 
> If that were not you, then who else? ;-)
> 
>>>     dict_deps, dict_types = get_dependency_tree(direction)
>>  I like the function name! However...
> 
> Oh come on, that *is* the most important thing! ;-)
> 
>>> where:
>>>
>>>   - direction is either string 'forward' or 'back',
>>  I would make this a boolean argument:
>> def get_dependency_tree(backward: bool = True) -> Dict[str, Package]:
> 
> Just for my own understanding: what syntax is that? I was trying to write
> the protoype of the function, but in Python there is no prototype where
> you can explain the return type. Yours seems like it is what I was
> looking for. Is it described somewhere?

 Those are type annotations. They don't do anything by themselves, but they can
be used by other tools/libraries to do static type checking.

 Of course, they can't be used in Python 2.ancient so you can't actually use
them in graph-depends, but it's a good way of documenting the intention.

 For documentation, I'll refer to PEP 484 [1].

> (See how I'm trying to sidetrack you into not bikeshedding? ;-))
> 
>>  Note however that it is trivial to populate the backward dependencies once you
>> have the forward ones, so I'm not sure its worth it to make the distinction.
> 
> I was also thinking about that too.
> 
>>>   - dict_deps is a dictionnary, which keys are the package names, and
>>>     which values are lists of packages that are direct dependencies of
>>>     the key package (basically, what get_all_depends() currently
>>>     returns, but the whole dependency tree)
>>>
>>>   - dict_types is a dictionnary, which keys are the package names, and
>>>     the values are string representing whtehr the packages are target or
>>>     host, and virtual or not, e.g.: 'target', 'target-virtual', 'host',
>>>     'host-virtual'.
>>
>>  I would take this opportunity to create a class Package with members:
>>
>> name: str                  # Package name including hsot- prefix
>> dependencies: Set[Package] # Forward dependencies
>> dependencies_backwards: Set[Package] # Backward dependencies
>> virtual: bool              # If true, it's a virtual package
>> host: bool                 # If true, it's a host package
>> version: str               # Package version
> 
> Meh, I was thinking about exactly the same thing, but it is quite an
> invasive change.

 I would think that the invasive change you're about to do is the ideal occasion
to do this one :-)

> But let me bikeshed in turn: I would make your proposed
> 'host' field an enum instead,

 Python enums require an external package that we can't assume is installed. But
I suppose you didn't mean an actual enum :-P

> because we don't have only packages, but
> also rootfs. So I would have 'kind' (or 'type' if that's not reserved?)
> which could be one of 'target','host', or 'rootfs'.

 Sounds good to me. 'kind' is better than 'type' IMHO.


>>  The __init__ would set name, virtual, host, version. The dependencies are added
>> dynamically, and whenever you add one you do it in both directions.
> 
> Yep.
> 
> Dang, I'm always getting side-tracked into side topics. All I wanted was
> a faster graph-depends, and it's already there now! Sob... ;-)

 Feel free to ignore my feedback! I'll take progress over perfection any time.

 Regards,
 Arnout

[1] https://www.python.org/dev/peps/pep-0484

[Buildroot] [PATCH 2/3] graph-depends: split off get_version/get_depends into pkgutil.py

[Yann E. MORIN]

Arnout, All,

On 2018-11-08 09:01 +0100, Arnout Vandecappelle spake thusly:
> On 07/11/18 23:45, Yann E. MORIN wrote:
> >> def get_dependency_tree(backward: bool = True) -> Dict[str, Package]:
> > Just for my own understanding: what syntax is that? I was trying to write
> > the protoype of the function, but in Python there is no prototype where
> > you can explain the return type. Yours seems like it is what I was
> > looking for. Is it described somewhere?
>  For documentation, I'll refer to PEP 484 [1].

ACK, thanks. :-)

[--SNIP--]
> > But let me bikeshed in turn: I would make your proposed
> > 'host' field an enum instead,
> 
>  Python enums require an external package that we can't assume is installed. But
> I suppose you didn't mean an actual enum :-P

Yeah, I knew, I was thinking 'emulated' enum, as class-level constants, like:

    class Package():
        T_UNKNOWN_PKG = 0
        T_HOST_PKG = 1
        T_TARGET_PK = 2
        [...]

and then refer to them as Package.T_HOST_PKG etc...

(but of course, they are not constant! they are just class-variables
with default values.)

> > Dang, I'm always getting side-tracked into side topics. All I wanted was
> > a faster graph-depends, and it's already there now! Sob... ;-)
>  Feel free to ignore my feedback! I'll take progress over perfection any time.

Eh! ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'