[RFC PATCH 0/2] data_smart.py (mostly): Variable and include tracking!

[Peter Seebach <peter.seebach@windriver.com>]

This is a first pass at allowing us to answer that vexing question:

Where the heck did that value come from?

The complexity of bitbake's variable assignment means that even if you
had a complete list of the lines being processed in order, you might need
a few tries to figure out why a variable is set to what it is.  When
debugging, you might want to know what it would have been otherwise,
too; knowing that a given ?= is processed after a +=, for instance,
could save you a great deal of trouble.  To solve this, we add two pieces
of functionality to "bitbake -e".

The first is a list of the .conf/.inc files that were parsed, in order;
the list shows included files indented under the thing that included
them, making it easier to see not just what got processed, but why it got
processed.

The second is a list of all the operations performed on each value, in
order, citing to the file and line number where they happened (whenever
possible).  So, for instance, say I want to know how SSTATETASKS got its
value.  The output of "bitbake -e" now says:

# SSTATETASKS [4]
#   append /home/seebs/poky/meta/classes/staging.bbclass:105:
#     < do_populate_sysroot>
#   append /home/seebs/poky/meta/classes/package.bbclass:1750:
#     < do_populate_sysroot do_package>
#   append /home/seebs/poky/meta/classes/package_rpm.bbclass:1112:
#     < do_populate_sysroot do_package do_package_write_rpm>
#   append /home/seebs/poky/meta/classes/license.bbclass:380:
#     < do_populate_sysroot do_package do_package_write_rpm do_populate_lic>
#
# SSTATETASKS= do_populate_sysroot do_package do_package_write_rpm do_populate_lic

Wait, staging.bbclass?  I don't remember asking for that.  Let's go look
at the include history:

# /home/seebs/poky/meta/classes/base.bbclass includes:
#   /home/seebs/poky/meta/classes/patch.bbclass includes:
#     /home/seebs/poky/meta/classes/terminal.bbclass
#   /home/seebs/poky/meta/classes/staging.bbclass

So this tells me that staging.bbclass was included from base.bbclass.  Yay!
Mystery is solved.  Everything is working because it's doing what it is
supposed to do.  :)

The following changes since commit f8bf4499549f978ce7c93bc088a66f74cd751e2c:
  Martin Jansa (1):
        setup.py: fix path for bitbake-selftest

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib seebs/tracking
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=seebs/tracking

Peter Seebach (2):
  data_smart.py: Provide (optional) logging of variable modifications
  data_smart.py: Track configuration file inclusions

 lib/bb/cooker.py                   |    4 +-
 lib/bb/data.py                     |   46 +++++++++++--
 lib/bb/data_smart.py               |  128 +++++++++++++++++++++++++++--------
 lib/bb/parse/__init__.py           |    8 ++-
 lib/bb/parse/ast.py                |   89 +++++++++++++------------
 lib/bb/parse/parse_py/BBHandler.py |    8 +-
 lib/bb/siggen.py                   |    2 +-
 7 files changed, 198 insertions(+), 87 deletions(-)