[RFC PATCH 0/1] Save the initial environment v2

[RFC PATCH 0/1] Save the initial environment v2

[Joshua Lock]

Here's a second attempt at a patch to store the initial environment before
BitBake cleans it out. We take a copy in the bitbake entry point and pass it
to the cooker for saving.

I've give the parameter to the init method a default, None, value mostly so
that the bitbake-layers script doesn't have to be updated to pass None.

Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit df55199209ef042e5b1ca04c4df75c7d639b51eb:

  Remove unused target tree data for Hob (2011-07-21 22:36:28 +0100)

are available in the git repository at:
  git://github.com/incandescant/bitbake initialenv
  https://github.com/incandescant/bitbake/tree/initialenv

Joshua Lock (1):
  bitbake|cooker: save a copy of the environment when BitBake is
    started

 bin/bitbake      |    5 ++++-
 lib/bb/cooker.py |    7 ++++++-
 2 files changed, 10 insertions(+), 2 deletions(-)

-- 
1.7.6

[RFC PATCH 1/1] bitbake|cooker: save a copy of the environment when BitBake is started

[Joshua Lock]

Keep a saved copy of the environment from when BitBake is launched in the
cooker such that child processes can replicate (and/or use values from) the
host environment, rather than the cleaned up environment that the main
BitBake process uses.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 bin/bitbake      |    5 ++++-
 lib/bb/cooker.py |    7 ++++++-
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/bin/bitbake b/bin/bitbake
index 206d97b..a9dd67e 100755
--- a/bin/bitbake
+++ b/bin/bitbake
@@ -197,6 +197,9 @@ Default BBFILES are the .bb files in the current directory.""")
     handler = bb.event.LogHandler()
     logger.addHandler(handler)
 
+    # Before we start modifying the environment we should take a pristine
+    # copy for possible later use
+    initialenv = os.environ.copy()
     # Clear away any spurious environment variables. But don't wipe the
     # environment totally. This is necessary to ensure the correct operation
     # of the UIs (e.g. for DISPLAY, etc.)
@@ -207,7 +210,7 @@ Default BBFILES are the .bb files in the current directory.""")
     server.initServer()
     idle = server.getServerIdleCB()
 
-    cooker = bb.cooker.BBCooker(configuration, idle)
+    cooker = bb.cooker.BBCooker(configuration, idle, initialenv)
     cooker.parseCommandLine()
 
     server.addcooker(cooker)
diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index 0326b8c..30a1039 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -78,7 +78,7 @@ class BBCooker:
     Manages one bitbake build run
     """
 
-    def __init__(self, configuration, server_registration_cb):
+    def __init__(self, configuration, server_registration_cb, savedenv=None):
         self.status = None
         self.appendlist = {}
         self.skiplist = {}
@@ -87,6 +87,11 @@ class BBCooker:
 
         self.configuration = configuration
 
+        # A copy of the mapping representing the environment when BitBake was
+        # launched is saved to enable child processes to use environment
+        # variables which have been cleaned from the BitBake processes env
+        self.savedenv = savedenv
+
         self.caches_array = []
         # Currently, only Image Creator hob ui needs extra cache.
         # So, we save Extra Cache class name and container file
-- 
1.7.6

Re: [RFC PATCH 1/1] bitbake|cooker: save a copy of the environment when BitBake is started

[Richard Purdie]

On Thu, 2011-07-21 at 15:42 -0700, Joshua Lock wrote:
> Keep a saved copy of the environment from when BitBake is launched in the
> cooker such that child processes can replicate (and/or use values from) the
> host environment, rather than the cleaned up environment that the main
> BitBake process uses.
> 
> Signed-off-by: Joshua Lock <josh@linux.intel.com>
> ---
>  bin/bitbake      |    5 ++++-
>  lib/bb/cooker.py |    7 ++++++-
>  2 files changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/bin/bitbake b/bin/bitbake
> index 206d97b..a9dd67e 100755
> --- a/bin/bitbake
> +++ b/bin/bitbake
> @@ -197,6 +197,9 @@ Default BBFILES are the .bb files in the current directory.""")
>      handler = bb.event.LogHandler()
>      logger.addHandler(handler)
>  
> +    # Before we start modifying the environment we should take a pristine
> +    # copy for possible later use
> +    initialenv = os.environ.copy()
>      # Clear away any spurious environment variables. But don't wipe the
>      # environment totally. This is necessary to ensure the correct operation
>      # of the UIs (e.g. for DISPLAY, etc.)
> @@ -207,7 +210,7 @@ Default BBFILES are the .bb files in the current directory.""")
>      server.initServer()
>      idle = server.getServerIdleCB()
>  
> -    cooker = bb.cooker.BBCooker(configuration, idle)
> +    cooker = bb.cooker.BBCooker(configuration, idle, initialenv)
>      cooker.parseCommandLine()
>  
>      server.addcooker(cooker)
> diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
> index 0326b8c..30a1039 100644
> --- a/lib/bb/cooker.py
> +++ b/lib/bb/cooker.py
> @@ -78,7 +78,7 @@ class BBCooker:
>      Manages one bitbake build run
>      """
>  
> -    def __init__(self, configuration, server_registration_cb):
> +    def __init__(self, configuration, server_registration_cb, savedenv=None):
>          self.status = None
>          self.appendlist = {}
>          self.skiplist = {}
> @@ -87,6 +87,11 @@ class BBCooker:
>  
>          self.configuration = configuration
>  
> +        # A copy of the mapping representing the environment when BitBake was
> +        # launched is saved to enable child processes to use environment
> +        # variables which have been cleaned from the BitBake processes env
> +        self.savedenv = savedenv
> +
>          self.caches_array = []
>          # Currently, only Image Creator hob ui needs extra cache.
>          # So, we save Extra Cache class name and container file

This looks good as a first step of this and I'm happy to merge this
patch. I'd probably go a little further and turn the above into:

self.savedenv = bb.data.init()
for k in savedenv:
    self.savedenv.setVar(k, savedenv[k])

so it becomes a data store in its own right. It then just becomes a case
of obtaining that data store in the UI or task code.

Cheers,

Richard