[PATCH v2 0/1]

[PATCH v2 0/1]

[Amanda Brindle]

For v2, I removed some dead code (recipe_bbvars(), collect_bbvars(), and 
code that handled the -m option). 

I also added the parameter variants=False to all_recipe_files() so that
the script would not scan through variant recipe files. 

I also updated the commit message to describe why the changes were made. 

The following changes since commit 65d23bd7986615fdfb0f1717b615534a2a14ab80:

  README.qemu: qemuppc64 is not supported (2017-10-16 23:54:31 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib abrindle/bbvars_tinfoil
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=abrindle/bbvars_tinfoil

Amanda Brindle (1):
  scripts/contrib/bbvars.py: Rewrite to use tinfoil

 scripts/contrib/bbvars.py | 140 ++++++++++++++++++++++------------------------
 1 file changed, 67 insertions(+), 73 deletions(-)

-- 
2.7.4

[PATCH v2 1/1] scripts/contrib/bbvars.py: Rewrite to use tinfoil

[Amanda Brindle]

Use tinfoil to collect all variable names globally and in each recipe.
This will capture more variable references than parsing manually.

With the use of tinfoil, this script can no longer distinguish unique
references to each variable, so it's no longer showing the count of
undocumented variables.

Removed the -m option since this script now searches through all recipes
in the configuration.

Fixes [YOCTO #2086]

Signed-off-by: Amanda Brindle <amanda.r.brindle@intel.com>
---
 scripts/contrib/bbvars.py | 140 ++++++++++++++++++++++------------------------
 1 file changed, 67 insertions(+), 73 deletions(-)

diff --git a/scripts/contrib/bbvars.py b/scripts/contrib/bbvars.py
index d8d0594..286b5a9 100755
--- a/scripts/contrib/bbvars.py
+++ b/scripts/contrib/bbvars.py
@@ -23,62 +23,38 @@ import os
 import os.path
 import re
 
+# Set up sys.path to let us import tinfoil
+scripts_path = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
+lib_path = scripts_path + '/lib'
+sys.path.insert(0, lib_path)
+import scriptpath
+scriptpath.add_bitbake_lib_path()
+import bb.tinfoil
+
 def usage():
-    print('Usage: %s -d FILENAME [-d FILENAME]* -m METADIR [-m MATADIR]*' % os.path.basename(sys.argv[0]))
+    print('Usage: %s -d FILENAME [-d FILENAME]*' % os.path.basename(sys.argv[0]))
     print('  -d FILENAME         documentation file to search')
     print('  -h, --help          display this help and exit')
-    print('  -m METADIR          meta directory to search for recipes')
     print('  -t FILENAME         documentation config file (for doc tags)')
     print('  -T                  Only display variables with doc tags (requires -t)')
 
-def recipe_bbvars(recipe):
-    ''' Return a unique set of every bbvar encountered in the recipe '''
-    prog = re.compile("[A-Z_]+")
-    vset = set()
-    try:
-        r = open(recipe)
-    except IOError as err:
-        print('WARNING: Failed to open recipe ', recipe)
-        print(err.args[1])
-
-    for line in r:
-        # Strip any comments from the line
-        line = line.rsplit('#')[0]
-        vset = vset.union(set(prog.findall(line)))
-    r.close()
-
-    bbvars = {}
-    for v in vset:
-        bbvars[v] = 1
-
-    return bbvars
-
-def collect_bbvars(metadir):
-    ''' Walk the metadir and collect the bbvars from each recipe found '''
-    bbvars = {}
-    for root,dirs,files in os.walk(metadir):
-        for name in files:
-            if name.find(".bb") >= 0:
-                for key in recipe_bbvars(os.path.join(root,name)).keys():
-                    if key in bbvars:
-                        bbvars[key] = bbvars[key] + 1
-                    else:
-                        bbvars[key] = 1
-    return bbvars
-
-def bbvar_is_documented(var, docfiles):
-    prog = re.compile(".*($|[^A-Z_])%s([^A-Z_]|$)" % (var))
-    for doc in docfiles:
-        try:
-            f = open(doc)
-        except IOError as err:
-            print('WARNING: Failed to open doc ', doc)
-            print(err.args[1])
-        for line in f:
-            if prog.match(line):
-                return True
-        f.close()
-    return False
+def bbvar_is_documented(var, documented_vars):
+    ''' Check if variable (var) is in the list of documented variables(documented_vars) '''
+    if var in documented_vars:
+        return True
+    else:
+        return False
+
+def collect_documented_vars(docfiles):
+    ''' Walk the docfiles and collect the documented variables '''
+    documented_vars = []
+    prog = re.compile(".*($|[^A-Z_])<glossentry id=\'var-")
+    var_prog = re.compile('<glossentry id=\'var-(.*)\'>')
+    for d in docfiles:
+        with open(d) as f:
+            documented_vars += var_prog.findall(f.read())
+
+    return documented_vars
 
 def bbvar_doctag(var, docconf):
     prog = re.compile('^%s\[doc\] *= *"(.*)"' % (var))
@@ -100,8 +76,7 @@ def bbvar_doctag(var, docconf):
 
 def main():
     docfiles = []
-    metadirs = []
-    bbvars = {}
+    bbvars = set()
     undocumented = []
     docconf = ""
     onlydoctags = False
@@ -124,12 +99,6 @@ def main():
             else:
                 print('ERROR: documentation file %s is not a regular file' % a)
                 sys.exit(3)
-        elif o == '-m':
-            if os.path.isdir(a):
-                metadirs.append(a)
-            else:
-                print('ERROR: meta directory %s is not a directory' % a)
-                sys.exit(4)
         elif o == "-t":
             if os.path.isfile(a):
                 docconf = a
@@ -143,43 +112,68 @@ def main():
         usage()
         sys.exit(5)
 
-    if len(metadirs) == 0:
-        print('ERROR: no metadir specified')
-        usage()
-        sys.exit(6)
-
     if onlydoctags and docconf == "":
         print('ERROR: no docconf specified')
         usage()
         sys.exit(7)
 
-    # Collect all the variable names from the recipes in the metadirs
-    for m in metadirs:
-        for key,cnt in collect_bbvars(m).items():
-            if key in bbvars:
-                bbvars[key] = bbvars[key] + cnt
+    prog = re.compile("^[^a-z]*$")
+    with bb.tinfoil.Tinfoil() as tinfoil:
+        tinfoil.prepare(config_only=False)
+        parser = bb.codeparser.PythonParser('parser', None)
+        datastore = tinfoil.config_data
+
+        def bbvars_update(data):
+            if prog.match(data):
+                bbvars.add(data)
+            if tinfoil.config_data.getVarFlag(data, 'python'):
+                try:
+                    parser.parse_python(tinfoil.config_data.getVar(data))
+                except bb.data_smart.ExpansionError:
+                    pass
+                for var in parser.references:
+                    if prog.match(var):
+                        bbvars.add(var)
             else:
-                bbvars[key] = cnt
+                try:
+                    expandedVar = datastore.expandWithRefs(datastore.getVar(data, False), data)
+                    for var in expandedVar.references:
+                        if prog.match(var):
+                            bbvars.add(var)
+                except bb.data_smart.ExpansionError:
+                    pass
+
+        # Use tinfoil to collect all the variable names globally
+        for data in datastore:
+            bbvars_update(data)
+
+        # Collect variables from all recipes
+        for recipe in tinfoil.all_recipe_files(variants=False):
+            print("Checking %s" % recipe)
+            for data in tinfoil.parse_recipe_file(recipe):
+                bbvars_update(data)
+
+    documented_vars = collect_documented_vars(docfiles)
 
     # Check each var for documentation
     varlen = 0
-    for v in bbvars.keys():
+    for v in bbvars:
         if len(v) > varlen:
             varlen = len(v)
-        if not bbvar_is_documented(v, docfiles):
+        if not bbvar_is_documented(v, documented_vars):
             undocumented.append(v)
     undocumented.sort()
     varlen = varlen + 1
 
     # Report all undocumented variables
     print('Found %d undocumented bb variables (out of %d):' % (len(undocumented), len(bbvars)))
-    header = '%s%s%s' % (str("VARIABLE").ljust(varlen), str("COUNT").ljust(6), str("DOCTAG").ljust(7))
+    header = '%s%s' % (str("VARIABLE").ljust(varlen), str("DOCTAG").ljust(7))
     print(header)
     print(str("").ljust(len(header), '='))
     for v in undocumented:
         doctag = bbvar_doctag(v, docconf)
         if not onlydoctags or not doctag == "":
-            print('%s%s%s' % (v.ljust(varlen), str(bbvars[v]).ljust(6), doctag))
+            print('%s%s' % (v.ljust(varlen), doctag))
 
 
 if __name__ == "__main__":
-- 
2.7.4

Re: [PATCH v2 1/1] scripts/contrib/bbvars.py: Rewrite to use tinfoil

[Paul Eggleton]

On Thursday, 16 November 2017 1:21:04 PM NZDT Amanda Brindle wrote:
> Use tinfoil to collect all variable names globally and in each recipe.
> This will capture more variable references than parsing manually.
> 
> With the use of tinfoil, this script can no longer distinguish unique
> references to each variable, so it's no longer showing the count of
> undocumented variables.
> 
> Removed the -m option since this script now searches through all recipes
> in the configuration.
> 
> Fixes [YOCTO #2086]
> 
> Signed-off-by: Amanda Brindle <amanda.r.brindle@intel.com>

Acked-by: Paul Eggleton <paul.eggleton@linux.intel.com>

Thanks,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

✗ patchtest: failure for [v2] scripts/contrib/bbvars.py: Rewrite to use tinfoil

[Patchwork]

== Series Details ==

Series: [v2] scripts/contrib/bbvars.py: Rewrite to use tinfoil
Revision: 1
URL   : https://patchwork.openembedded.org/series/9816/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Series does not apply on top of target branch [test_series_merge_on_head] 
  Suggested fix    Rebase your series on top of targeted branch
  Targeted branch  master (currently at a17f3ec910)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

✗ patchtest: failure for [v2] scripts/contrib/bbvars.py: Rewrite to use tinfoil

[Patchwork]

== Series Details ==

Series: [v2] scripts/contrib/bbvars.py: Rewrite to use tinfoil
Revision: 1
URL   : https://patchwork.openembedded.org/series/9816/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Series does not apply on top of target branch [test_series_merge_on_head] 
  Suggested fix    Rebase your series on top of targeted branch
  Targeted branch  master (currently at a17f3ec910)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

[PATCH v2 0/1]

[Michael Cheng]

To align with the discussion in [1][2], this patch series drops all usage of    
wbvind_on_all_cpus within i915 by either replacing the call with certain        
drm clflush helpers, or reverting to a previous logic.                          
                                                                                 
[1]. https://lists.freedesktop.org/archives/dri-devel/2021-November/330928.html 
[2]. https://patchwork.freedesktop.org/patch/475752/?series=99991&rev=5 

v2 (Michael Cheng) : Per feedback from Thomas and Tvrtko, this series should be
split into two parts; First part is to unblock i915 from build errors and 
second part will focus on implementing range flushes on the propose reverts.


Michael Cheng (1):
  i915/gem: drop wbinvd_on_all_cpus usage

 drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

-- 
2.25.1

[PATCH v2 0/1]

[Philippe Gerum]

From: Philippe Gerum <rpm@xenomai.org>

Use SPDX identifiers to refer to licenses and reformat.

Philippe Gerum (1):
  license: summarize the licensing terms

 COPYING | 45 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)
 create mode 100644 COPYING

-- 
2.31.1

[PATCH v2 0/1]

[Amanda Brindle]

Added a signed-off-by for v2.

The following changes since commit 4ed19ac8c19afd56d445d84e02b622cb056b8359:

  poky: Switch to post release name/version (2017-11-14 17:26:58 +0000)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib abrindle/bbvars_tinfoil2
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=abrindle/bbvars_tinfoil2

Amanda Brindle (1):
  scripts/contrib/bbvars.py: Remove dead code

 scripts/contrib/bbvars.py | 52 ++---------------------------------------------
 1 file changed, 2 insertions(+), 50 deletions(-)

-- 
2.7.4

[PATCH v2 0/1]

[Claudio Imbrenda]

This patch adds a few lines to the KVM common code to fire a
KOBJ_CHANGE uevent whenever a KVM VM is created or destroyed. The event
carries four environment variables:

KVM_VM_CREATED indicates how many times a new VM has been created. It
               is useful for example to trigger specific actions when
               the first VM is started
KVM_VM_COUNT indicates how many VMs are currently active. This can be
             used for logging or monitoring purposes
KVM_VM_PID has the pid of the KVM process that has been started or
           stopped. This can be used to perform process-specific
           tuning.
KVM_VM_STATS_PATH contains the path in /sys to the directory with all
                  the runtime statistics for this VM. This is useful
                  for performance monitoring and profiling.

Specific udev rules can be then set up in userspace to deal with the
creation or destruction of VMs as needed.


v1 -> v2:
* added KVM_VM_PID and KVM_VM_STATS_PATH
* some cleanup, the patch should look nicer now
* rebased on 4.12

Claudio Imbrenda (1):
  KVM: trigger uevents when starting or stopping a VM

 virt/kvm/kvm_main.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)

-- 
2.7.4