All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH v2 2/6] manual: add package-list.txt generation support
@ 2012-11-29  7:47 Samuel Martin
  2012-11-29  7:47 ` [Buildroot] [PATCH v2 3/6] manual: add deprecated-list.txt " Samuel Martin
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Samuel Martin @ 2012-11-29  7:47 UTC (permalink / raw)
  To: buildroot


Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---

Change since v1:

* move the generation code in a script that is called in manual.mk (as suggested by Arnout)
* remove unneeded comment in the generated output (as suggested by Arnout)

---
 docs/manual/manual.mk                  |  3 +++
 support/scripts/gen-manual-pkg-list.sh | 19 +++++++++++++++++++
 2 files changed, 22 insertions(+)
 create mode 100755 support/scripts/gen-manual-pkg-list.sh

diff --git a/docs/manual/manual.mk b/docs/manual/manual.mk
index aa20534..d8437ba 100644
--- a/docs/manual/manual.mk
+++ b/docs/manual/manual.mk
@@ -24,6 +24,9 @@ $$(O)/docs/$(1)/$(1).$(4): docs/$(1)/$(1).txt $$($(call UPPERCASE,$(1))_SOURCES)
 	  -D $$(@D) $$<
 endef
 
+$(TOPDIR)/docs/manual/package-list.txt:
+	$(TOPDIR)/support/scripts/gen-manual-pkg-list.sh > $@
+
 ################################################################################
 # GENDOC -- generates the make targets needed to build asciidoc documentation.
 #
diff --git a/support/scripts/gen-manual-pkg-list.sh b/support/scripts/gen-manual-pkg-list.sh
new file mode 100755
index 0000000..f11b4de
--- /dev/null
+++ b/support/scripts/gen-manual-pkg-list.sh
@@ -0,0 +1,19 @@
+#!/bin/sh
+##
+## gen-manual-pkg-list.sh
+##
+## Author(s):
+##  - Samuel MARTIN <s.martin49@gmail.com>
+##
+## Copyright (C) 2012 Samuel MARTIN
+##
+
+# Generate the manual package-list.txt content (in asciidoc format) and print it
+# to the standard output.
+
+printf "\n//\n// Autogenerated file\n//\n\n"
+printf "[[package-list]]\n"
+printf "Available packages\n"
+printf -- "------------------\n\n"
+grep -rlE --color=never '\((autotools|cmake|generic)-package\)' package | \
+  sed -ne '/.*\/\(.*\).mk$/ s//* \1/p' | sort
-- 
1.8.0.1

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v2 3/6] manual: add deprecated-list.txt generation support
  2012-11-29  7:47 [Buildroot] [PATCH v2 2/6] manual: add package-list.txt generation support Samuel Martin
@ 2012-11-29  7:47 ` Samuel Martin
  2012-11-29 22:41   ` Arnout Vandecappelle
  2012-11-29  7:47 ` [Buildroot] [PATCH v2 6/6] manual: add generated *-list.txt Samuel Martin
  2012-11-29 20:48 ` [Buildroot] [PATCH v2 2/6] manual: add package-list.txt generation support Arnout Vandecappelle
  2 siblings, 1 reply; 6+ messages in thread
From: Samuel Martin @ 2012-11-29  7:47 UTC (permalink / raw)
  To: buildroot

The whole deprecated-list.txt file is generated by the deprecated.py
script by searching for symbols depending on BR2_DEPRECATED in the
Buildroot code base.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---

Change since v1:

* rename the script deprecated.py -> gen-deprecated-list.py (to keep consistency)

---
 docs/manual/manual.mk                  |   3 +
 support/scripts/gen-deprecated-list.py | 144 +++++++++++++++++++++++++++++++++
 2 files changed, 147 insertions(+)
 create mode 100755 support/scripts/gen-deprecated-list.py

diff --git a/docs/manual/manual.mk b/docs/manual/manual.mk
index d8437ba..4cd5839 100644
--- a/docs/manual/manual.mk
+++ b/docs/manual/manual.mk
@@ -27,6 +27,9 @@ endef
 $(TOPDIR)/docs/manual/package-list.txt:
 	$(TOPDIR)/support/scripts/gen-manual-pkg-list.sh > $@
 
+$(TOPDIR)/docs/manual/deprecated-list.txt:
+	python2 $(TOPDIR)/support/scripts/gen-deprecated-list.py > $@
+
 ################################################################################
 # GENDOC -- generates the make targets needed to build asciidoc documentation.
 #
diff --git a/support/scripts/gen-deprecated-list.py b/support/scripts/gen-deprecated-list.py
new file mode 100755
index 0000000..cd8c288
--- /dev/null
+++ b/support/scripts/gen-deprecated-list.py
@@ -0,0 +1,144 @@
+#!/usr/bin/env python
+##
+## deprecated-packages.py
+##
+## Author(s):
+##  - Samuel MARTIN <s.martin49@gmail.com>
+##
+## Copyright (C) 2012 Samuel MARTIN
+##
+
+# Python 2.7 script searching for kconfig symbols depending on 'BR2_DEPRECATED'
+# and generating (printing to the standard output) the manual file in asciidoc.
+
+import os
+import re
+import sys
+
+
+ASCIIDOC_HEADER = """\
+//
+// Autogenerated file
+//
+
+[[deprecated]]
+Deprecated list
+---------------
+
+The following stuff are marked as _deprecated_ in Buildroot due to
+their status either too old or unmaintained.
+
+// Please check and sort by grepping the source running:
+//
+// $ git grep -EB4 'depends on BR2_DEPRECATED'
+//
+// and:
+//
+// $ git grep -EB4 'depends on BR2_DEPRECATED' | \\
+//     grep -Eo '(:|-).*?(config|comment) BR2_.*'
+"""
+
+NOT_SEARCHED = (".git", "board", "configs", "docs", "output", "support")
+
+# Relative path to category data map
+#   key:   relative path from buildroot topdir to the search location
+#   value: (rank_in_menuconfig, category_name, recursive_search)
+#       rank_in_menuconfig is used for ordering the diplaying
+#       category_name is used in the diplaying
+#       recursive_search when searching for deprecated stuff
+CAT_DIR2DATA = {
+    'arch'      : (0, "target architecture",  True),
+    '.'         : (1, "build options",        False),
+    'toolchain' : (2, "toolchain",            True),
+    'system'    : (3, "system configuration", True),
+    'package'   : (4, "package selection",    True),
+    'fs'        : (5, "filesystem images",    True),
+    'boot'      : (6, "bootloaders",          True),
+    'linux'     : (7, "kernel",               True),
+    }
+
+DEPR_SYMBOL = "BR2_DEPRECATED"
+
+_REGEX = r"config BR2_(.*?)\n" + \
+    "((.*?(?!config)(prompt|bool|string|int) \"(.*?)\".*?|[^\n]+)\n)*" + \
+    "(.*?(?!config )" + DEPR_SYMBOL + ".*?)\n" + \
+    "((.*?(?!config)(prompt|bool|string|int) \"(.*?)\".*?|[^\n]+)\n)*"
+
+REGEX = re.compile(_REGEX, flags=re.MULTILINE)
+
+
+def _get_buildroot_topdir():
+    topdir = os.path.join(os.path.dirname(__file__), "..", "..")
+    topdir = os.path.abspath(topdir)
+    return topdir
+
+def get_dir_list():
+    root = _get_buildroot_topdir()
+    dirs = ['.']
+    for dir_ in os.listdir(root):
+        if dir_ in NOT_SEARCHED:
+            continue
+        dir__ = os.path.join(root, dir_)
+        if not os.path.isdir(dir__):
+            continue
+        dirs += [dir_]
+    return dirs
+
+def find_deprecated(root, recursive):
+    deprecated = list()
+    for root_, _, files_ in os.walk(root):
+        if not recursive and root_ != root:
+            break
+        for file_ in files_:
+            if not file_.startswith("Config.in"):
+                continue
+            with open(os.path.join(root_, file_), "r") as f:
+                content = f.read()
+            if not DEPR_SYMBOL in content:
+                continue
+            found = REGEX.findall(content)
+            if found:
+                deprecated += found
+    return deprecated
+
+
+class Category():
+
+    def __init__(self, directory):
+        self.path = os.path.join(_get_buildroot_topdir(), directory)
+        rank, name, rec = CAT_DIR2DATA.get(directory, (99, directory, True))
+        self.name = name
+        self.rank = rank
+        self.depr_items = find_deprecated(self.path, rec)
+
+    def __str__(self):
+        items_ = list()
+        for item in self.depr_items:
+            name_ = item[0].lower().replace("_", " ")
+            name_ = re.sub("^package ", "", name_)
+            vers = re.sub(".*?(version )?([0-9].*)", r'\2', name_)
+            if vers:
+                vers = re.sub(" ", ".", vers)
+                name_ = re.sub("(version )?([0-9].*)", vers, name_)
+            symbol = item[4]
+            if not symbol:
+                symbol = item[9]
+            items_ += ["** %-25s +[%s]+" % (name_, symbol)]
+        items_.sort()
+        output = "\n* %s:\n\n" % self.name.capitalize()
+        output += "\n".join(items_)
+        output += "\n"
+        return output
+
+def main():
+    categories = [Category(directory) for directory in get_dir_list()]
+    categories = [category for category in categories if category.depr_items]
+    categories.sort(cmp=lambda x, y: x.rank - y.rank if x.rank != y.rank \
+                        else cmp(x.name, y.name))
+    output = ASCIIDOC_HEADER
+    for category in categories:
+        output += str(category)
+    print output
+
+if __name__ == "__main__":
+    main()
-- 
1.8.0.1

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v2 6/6] manual: add generated *-list.txt
  2012-11-29  7:47 [Buildroot] [PATCH v2 2/6] manual: add package-list.txt generation support Samuel Martin
  2012-11-29  7:47 ` [Buildroot] [PATCH v2 3/6] manual: add deprecated-list.txt " Samuel Martin
@ 2012-11-29  7:47 ` Samuel Martin
  2012-11-29 22:42   ` Arnout Vandecappelle
  2012-11-29 20:48 ` [Buildroot] [PATCH v2 2/6] manual: add package-list.txt generation support Arnout Vandecappelle
  2 siblings, 1 reply; 6+ messages in thread
From: Samuel Martin @ 2012-11-29  7:47 UTC (permalink / raw)
  To: buildroot

* update package-list.txt (formerly named pkg-list.txt)
* update deprecated-list.txt
* update appendix.txt (which include these 2 generated files)

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---

Change since v1:

* regenerate *-list.txt files

---
 docs/manual/appendix.txt                       | 13 ++---
 docs/manual/deprecated-list.txt                | 68 ++++++++++++++++----------
 docs/manual/{pkg-list.txt => package-list.txt} | 10 ++++
 3 files changed, 55 insertions(+), 36 deletions(-)
 rename docs/manual/{pkg-list.txt => package-list.txt} (99%)

diff --git a/docs/manual/appendix.txt b/docs/manual/appendix.txt
index 6f1e9f3..4524073 100644
--- a/docs/manual/appendix.txt
+++ b/docs/manual/appendix.txt
@@ -5,15 +5,8 @@ Appendix
 
 include::makedev-syntax.txt[]
 
-[[package-list]]
-Available packages
-------------------
-// docs/manaual/pkg-list.txt is generated using the following command:
-// $ git grep -E '\((autotools|cmake|generic)-package\)' package/ |  \
-//     cut -d':' -f1 | grep '\.mk$' | \
-//     sed -e 's;.*\?/\(.*\?\).mk;* \1;' | \
-//     sort > docs/manual/pkg-list.txt
-
-include::pkg-list.txt[]
+// autogenerated
+include::package-list.txt[]
 
+// autogenerated
 include::deprecated-list.txt[]
diff --git a/docs/manual/deprecated-list.txt b/docs/manual/deprecated-list.txt
index 6dc87a4..1efab7f 100644
--- a/docs/manual/deprecated-list.txt
+++ b/docs/manual/deprecated-list.txt
@@ -1,4 +1,6 @@
-// -*- mode:doc -*- ;
+//
+// Autogenerated file
+//
 
 [[deprecated]]
 Deprecated list
@@ -7,40 +9,54 @@ Deprecated list
 The following stuff are marked as _deprecated_ in Buildroot due to
 their status either too old or unmaintained.
 
-// list generated using the followings command:
+// Please check and sort by grepping the source running:
+//
 // $ git grep -EB4 'depends on BR2_DEPRECATED'
-// and
+//
+// and:
+//
 // $ git grep -EB4 'depends on BR2_DEPRECATED' | \
 //     grep -Eo '(:|-).*?(config|comment) BR2_.*'
-//
-// Need manual checks and sorting.
 
-* Packages:
+* Build options:
 
-** +busybox+ 1.18.x
-** +customize+
-** +lzma+
-** +microperl+
-** +netkitbase+
-** +netkittelnet+
-** +pkg-config+
-** +squashfs3+
-** +ttcp+
+** have devfiles             +[development files in target filesystem]+
+** have documentation        +[documentation on the target]+
 
 * Toolchain:
 
-** +gdb+ 6.8
-** +gdb+ 7.0.1
-** +gdb+ 7.1
-** +kernel headers+ 2.6.37
-** +kernel headers+ 2.6.38
-** +kernel headers+ 2.6.39
+** gcc target                +[gcc]+
+** gdb                       +[Build gdb debugger for the Target]+
+** gdb 6.8                   +[gdb 6.8]+
+** gdb 7.0.1                 +[gdb 7.0.1]+
+** gdb 7.1                   +[gdb 7.1]+
+** kernel headers 2.6.37     +[Linux 2.6.37.x kernel headers]+
+** kernel headers 2.6.38     +[Linux 2.6.38.x kernel headers]+
+** kernel headers 2.6.39     +[Linux 2.6.39.x kernel headers]+
+
+* Package selection:
+
+** autoconf                  +[autoconf]+
+** automake                  +[automake]+
+** binutils 2.20             +[binutils 2.20]+
+** busybox 1.18.x            +[BusyBox 1.18.x]+
+** customize                 +[customize]+
+** libtool                   +[libtool]+
+** lzma                      +[lzma]+
+** make                      +[make]+
+** microperl                 +[microperl]+
+** netkitbase                +[netkitbase]+
+** netkittelnet              +[netkittelnet]+
+** pkg config                +[pkg-config]+
+** squashfs3                 +[squashfs3]+
+** ttcp                      +[ttcp]+
+
+* Filesystem images:
+
+** target rootfs squashfs3   +[3.x]+
 
 * Bootloaders:
 
-** +u-boot+ 2011-06
-** +u-boot+ 2011-09
-
-* Output images:
+** target uboot 2011.06      +[2011.06]+
+** target uboot 2011.09      +[2011.09]+
 
-** squashfs3 image
diff --git a/docs/manual/pkg-list.txt b/docs/manual/package-list.txt
similarity index 99%
rename from docs/manual/pkg-list.txt
rename to docs/manual/package-list.txt
index 5d9b54f..d06316f 100644
--- a/docs/manual/pkg-list.txt
+++ b/docs/manual/package-list.txt
@@ -1,3 +1,12 @@
+
+//
+// Autogenerated file
+//
+
+[[package-list]]
+Available packages
+------------------
+
 * acl
 * acpid
 * alsa-lib
@@ -346,6 +355,7 @@
 * luaexpat
 * luafilesystem
 * luajit
+* lua-msgpack-native
 * luasocket
 * lvm2
 * lzma
-- 
1.8.0.1

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v2 2/6] manual: add package-list.txt generation support
  2012-11-29  7:47 [Buildroot] [PATCH v2 2/6] manual: add package-list.txt generation support Samuel Martin
  2012-11-29  7:47 ` [Buildroot] [PATCH v2 3/6] manual: add deprecated-list.txt " Samuel Martin
  2012-11-29  7:47 ` [Buildroot] [PATCH v2 6/6] manual: add generated *-list.txt Samuel Martin
@ 2012-11-29 20:48 ` Arnout Vandecappelle
  2 siblings, 0 replies; 6+ messages in thread
From: Arnout Vandecappelle @ 2012-11-29 20:48 UTC (permalink / raw)
  To: buildroot

On 29/11/12 08:47, Samuel Martin wrote:
>
> Signed-off-by: Samuel Martin<s.martin49@gmail.com>

Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
  (untested)

  But since I'm pedantic, I still have some comments :-)

[snip]
> diff --git a/support/scripts/gen-manual-pkg-list.sh b/support/scripts/gen-manual-pkg-list.sh
> new file mode 100755
> index 0000000..f11b4de
> --- /dev/null
> +++ b/support/scripts/gen-manual-pkg-list.sh
> @@ -0,0 +1,19 @@
> +#!/bin/sh
> +##
> +## gen-manual-pkg-list.sh
> +##
> +## Author(s):
> +##  - Samuel MARTIN<s.martin49@gmail.com>
> +##
> +## Copyright (C) 2012 Samuel MARTIN

  If you add a copyright statement, ideally you should also refer to the GPL:

     This program is free software; you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
     the Free Software Foundation; either version 2 of the License, or
     (at your option) any later version.

(The fact that you put the Copyright statement there makes it look as if
it's proprietary.  Adding the GPL reference disambiguates it.  Of course,
it is implied by the global buildroot license even if you leave out the
GPL reference.)

> +##
> +
> +# Generate the manual package-list.txt content (in asciidoc format) and print it
> +# to the standard output.
> +
> +printf "\n//\n// Autogenerated file\n//\n\n"
> +printf "[[package-list]]\n"
> +printf "Available packages\n"
> +printf -- "------------------\n\n"

  Since this is now part of a script, you can just cat a here document.
Much more readable.

> +grep -rlE --color=never '\((autotools|cmake|generic)-package\)' package | \

  We globally unexport GREP_OPTIONS, so the --color=never is redundant.
In addition, it would barf on non-gnu-grep.


  Regards,
  Arnout

> +  sed -ne '/.*\/\(.*\).mk$/ s//* \1/p' | sort

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v2 3/6] manual: add deprecated-list.txt generation support
  2012-11-29  7:47 ` [Buildroot] [PATCH v2 3/6] manual: add deprecated-list.txt " Samuel Martin
@ 2012-11-29 22:41   ` Arnout Vandecappelle
  0 siblings, 0 replies; 6+ messages in thread
From: Arnout Vandecappelle @ 2012-11-29 22:41 UTC (permalink / raw)
  To: buildroot

On 29/11/12 08:47, Samuel Martin wrote:
> The whole deprecated-list.txt file is generated by the deprecated.py
> script by searching for symbols depending on BR2_DEPRECATED in the
> Buildroot code base.
>
> Signed-off-by: Samuel Martin<s.martin49@gmail.com>
> ---
>
> Change since v1:
>
> * rename the script deprecated.py ->  gen-deprecated-list.py (to keep consistency)
>
> ---
>   docs/manual/manual.mk                  |   3 +
>   support/scripts/gen-deprecated-list.py | 144 +++++++++++++++++++++++++++++++++
>   2 files changed, 147 insertions(+)
>   create mode 100755 support/scripts/gen-deprecated-list.py
>
> diff --git a/docs/manual/manual.mk b/docs/manual/manual.mk
> index d8437ba..4cd5839 100644
> --- a/docs/manual/manual.mk
> +++ b/docs/manual/manual.mk
> @@ -27,6 +27,9 @@ endef
>   $(TOPDIR)/docs/manual/package-list.txt:
>   	$(TOPDIR)/support/scripts/gen-manual-pkg-list.sh>  $@
>
> +$(TOPDIR)/docs/manual/deprecated-list.txt:
> +	python2 $(TOPDIR)/support/scripts/gen-deprecated-list.py>  $@

  We don't have python2 in dependencies.sh, and anyway some things
don't currently work if python is python3.  So I'd just call the
script without the python2.  Anyway, does it not work with python3?


> +
>   ################################################################################
>   # GENDOC -- generates the make targets needed to build asciidoc documentation.
>   #
> diff --git a/support/scripts/gen-deprecated-list.py b/support/scripts/gen-deprecated-list.py
> new file mode 100755
> index 0000000..cd8c288
> --- /dev/null
> +++ b/support/scripts/gen-deprecated-list.py
> @@ -0,0 +1,144 @@
> +#!/usr/bin/env python
> +##
> +## deprecated-packages.py
> +##
> +## Author(s):
> +##  - Samuel MARTIN<s.martin49@gmail.com>
> +##
> +## Copyright (C) 2012 Samuel MARTIN
> +##
> +
> +# Python 2.7 script searching for kconfig symbols depending on 'BR2_DEPRECATED'
> +# and generating (printing to the standard output) the manual file in asciidoc.
> +
> +import os
> +import re
> +import sys
> +
> +
> +ASCIIDOC_HEADER = """\
> +//
> +// Autogenerated file
> +//
> +
> +[[deprecated]]
> +Deprecated list
> +---------------
> +
> +The following stuff are marked as _deprecated_ in Buildroot due to
> +their status either too old or unmaintained.

  Better:

The following packages are marked as _deprecated_ because they are
either too old or unmaintained in buildroot.

> +
> +// Please check and sort by grepping the source running:
> +//
> +// $ git grep -EB4 'depends on BR2_DEPRECATED'
> +//
> +// and:
> +//
> +// $ git grep -EB4 'depends on BR2_DEPRECATED' | \\
> +//     grep -Eo '(:|-).*?(config|comment) BR2_.*'

  This is redundant now.  Instead, mention the make target
after 'Autogenerated'.

> +"""
> +
> +NOT_SEARCHED = (".git", "board", "configs", "docs", "output", "support")
> +
> +# Relative path to category data map
> +#   key:   relative path from buildroot topdir to the search location
> +#   value: (rank_in_menuconfig, category_name, recursive_search)
> +#       rank_in_menuconfig is used for ordering the diplaying
> +#       category_name is used in the diplaying
> +#       recursive_search when searching for deprecated stuff
> +CAT_DIR2DATA = {
> +    'arch'      : (0, "target architecture",  True),
> +    '.'         : (1, "build options",        False),
> +    'toolchain' : (2, "toolchain",            True),
> +    'system'    : (3, "system configuration", True),
> +    'package'   : (4, "package selection",    True),
> +    'fs'        : (5, "filesystem images",    True),
> +    'boot'      : (6, "bootloaders",          True),
> +    'linux'     : (7, "kernel",               True),
> +    }
> +
> +DEPR_SYMBOL = "BR2_DEPRECATED"
> +
> +_REGEX = r"config BR2_(.*?)\n" + \
> +    "((.*?(?!config)(prompt|bool|string|int) \"(.*?)\".*?|[^\n]+)\n)*" + \
> +    "(.*?(?!config )" + DEPR_SYMBOL + ".*?)\n" + \
> +    "((.*?(?!config)(prompt|bool|string|int) \"(.*?)\".*?|[^\n]+)\n)*"

  This regex warrants some documentation :-) perhaps with re.VERBOSE

  Why is the !config needed in the prompt lines?

  Is there a reason to use the non-greedy .*? (except between the quotes)?

  I would also name the "interesting" parts with (?P<...>) and use finditer()
and m.group() to retrieve them.  But that's a matter of taste, I guess.

> +
> +REGEX = re.compile(_REGEX, flags=re.MULTILINE)
> +
> +
> +def _get_buildroot_topdir():
> +    topdir = os.path.join(os.path.dirname(__file__), "..", "..")
> +    topdir = os.path.abspath(topdir)
> +    return topdir

  I would just use the current directory as topdir.  All the rest of
buildroot anyway assumes it's being run from the topdir.

> +
> +def get_dir_list():
> +    root = _get_buildroot_topdir()
> +    dirs = ['.']
> +    for dir_ in os.listdir(root):
> +        if dir_ in NOT_SEARCHED:
> +            continue
> +        dir__ = os.path.join(root, dir_)
> +        if not os.path.isdir(dir__):
> +            continue
> +        dirs += [dir_]
> +    return dirs
> +
> +def find_deprecated(root, recursive):
> +    deprecated = list()
> +    for root_, _, files_ in os.walk(root):
> +        if not recursive and root_ != root:
> +            break
> +        for file_ in files_:
> +            if not file_.startswith("Config.in"):
> +                continue
> +            with open(os.path.join(root_, file_), "r") as f:
> +                content = f.read()
> +            if not DEPR_SYMBOL in content:
> +                continue
> +            found = REGEX.findall(content)
> +            if found:
> +                deprecated += found
> +    return deprecated

  It's quite a big change, but: why not use the source statements
in Config.in to do the recursion?  The regex it easy enough:
r'^\s*source "(?<path>.*?)"$'  This would be more future-safe:
when new directories are added, they will be handled correctly.

  It would also be good if you could add a warning when
len(found) != len(re.findall(DEPR_SYMBOL, content))

> +
> +
> +class Category():
> +
> +    def __init__(self, directory):
> +        self.path = os.path.join(_get_buildroot_topdir(), directory)
> +        rank, name, rec = CAT_DIR2DATA.get(directory, (99, directory, True))
> +        self.name = name
> +        self.rank = rank

  You can directly do
self.rank, self.name, rec = CAT_DIR2DATA.get(directory, (99, directory, True))

> +        self.depr_items = find_deprecated(self.path, rec)
> +
> +    def __str__(self):
> +        items_ = list()
> +        for item in self.depr_items:
> +            name_ = item[0].lower().replace("_", " ")
> +            name_ = re.sub("^package ", "", name_)
> +            vers = re.sub(".*?(version )?([0-9].*)", r'\2', name_)
> +            if vers:
> +                vers = re.sub(" ", ".", vers)
> +                name_ = re.sub("(version )?([0-9].*)", vers, name_)
> +            symbol = item[4]
> +            if not symbol:
> +                symbol = item[9]
> +            items_ += ["** %-25s +[%s]+" % (name_, symbol)]

  I don't think the name_ part adds much value.  In most cases, symbol
is either the same or more descriptive.  There are a few exceptions,
but these could be fixed by changing the Config.in.  These need to be
modified: gcc target, squashfs3 rootfs, uboot 11.06 and 11.09.

  BTW, symbol is not really symbol but rather 'description' or something
similar.

> +        items_.sort()
> +        output = "\n* %s:\n\n" % self.name.capitalize()
> +        output += "\n".join(items_)
> +        output += "\n"
> +        return output
> +
> +def main():
> +    categories = [Category(directory) for directory in get_dir_list()]
> +    categories = [category for category in categories if category.depr_items]
> +    categories.sort(cmp=lambda x, y: x.rank - y.rank if x.rank != y.rank \
> +                        else cmp(x.name, y.name))

     categories.sort(key=lambda x: (x.rank, x.name))

> +    output = ASCIIDOC_HEADER
> +    for category in categories:
> +        output += str(category)
> +    print output

  Don't concat the output, just print directly.

     print ASCIIDOC_HEADER
     for category in categories:
         print category


  Regards,
  Arnout


> +
> +if __name__ == "__main__":
> +    main()

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v2 6/6] manual: add generated *-list.txt
  2012-11-29  7:47 ` [Buildroot] [PATCH v2 6/6] manual: add generated *-list.txt Samuel Martin
@ 2012-11-29 22:42   ` Arnout Vandecappelle
  0 siblings, 0 replies; 6+ messages in thread
From: Arnout Vandecappelle @ 2012-11-29 22:42 UTC (permalink / raw)
  To: buildroot

On 29/11/12 08:47, Samuel Martin wrote:

> diff --git a/docs/manual/pkg-list.txt b/docs/manual/package-list.txt
> similarity index 99%
> rename from docs/manual/pkg-list.txt
> rename to docs/manual/package-list.txt
> index 5d9b54f..d06316f 100644
> --- a/docs/manual/pkg-list.txt
> +++ b/docs/manual/package-list.txt
> @@ -1,3 +1,12 @@
> +
> +//
> +// Autogenerated file
> +//
> +
> +[[package-list]]
> +Available packages
> +------------------
> +
>   * acl
>   * acpid
>   * alsa-lib
> @@ -346,6 +355,7 @@
>   * luaexpat
>   * luafilesystem
>   * luajit
> +* lua-msgpack-native
>   * luasocket
>   * lvm2
>   * lzma

  I love this!  Now you can easily see what has changed between two versions.

  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2012-11-29 22:42 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-11-29  7:47 [Buildroot] [PATCH v2 2/6] manual: add package-list.txt generation support Samuel Martin
2012-11-29  7:47 ` [Buildroot] [PATCH v2 3/6] manual: add deprecated-list.txt " Samuel Martin
2012-11-29 22:41   ` Arnout Vandecappelle
2012-11-29  7:47 ` [Buildroot] [PATCH v2 6/6] manual: add generated *-list.txt Samuel Martin
2012-11-29 22:42   ` Arnout Vandecappelle
2012-11-29 20:48 ` [Buildroot] [PATCH v2 2/6] manual: add package-list.txt generation support Arnout Vandecappelle

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.