Re: [PATCH 2/6] data_smart: Improve the calculation of config hash

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Mon, 2012-04-09 at 16:41 +0800, Dongxiao Xu wrote:
> The order of keys are not sensitive for config hash, so we need to
> identify its order while calculating the md5 value.
> 
> While for certain values, order is also not sensitive (for example,
> BBINCLUDED), we also need to identify its order while calculating md5
> value.
> 
> This could fix the problem that Martin Jansa reported in the mailing
> list:
> 
> http://lists.linuxtogo.org/pipermail/bitbake-devel/2012-March/002122.html
> 
> Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
> ---
>  lib/bb/data_smart.py |   10 +++++++---
>  1 files changed, 7 insertions(+), 3 deletions(-)
> 
> diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
> index 2c200db..cc61a03 100644
> --- a/lib/bb/data_smart.py
> +++ b/lib/bb/data_smart.py
> @@ -462,13 +462,17 @@ class DataSmart(MutableMapping):
>          self.delVar(var)
>  
>      def get_hash(self):
> -        data = ""
> +        data = {}
>          config_whitelist = set((self.getVar("BB_HASHCONFIG_WHITELIST", True) or "").split())
> +        config_sort = set((self.getVar("BB_HASHCONFIG_SORT", True) or "").split())
>          keys = set(key for key in iter(self) if not key.startswith("__"))
>          for key in keys:
>              if key in config_whitelist:
>                  continue
>              value = self.getVar(key, False) or ""
> -            data = data + key + ': ' + str(value) + '\n'
> +            if key in config_sort:
> +                value = " ".join(sorted(value.split()))
> +            data.update({key:value})
>  
> -        return hashlib.md5(data).hexdigest()
> +        data_str = str([(k, data[k]) for k in sorted(data.keys())])
> +        return hashlib.md5(data_str).hexdigest()


This and the corresponding change in bitbake.conf look rather worrying
to me. The order in BBINCLUDED is significant and if it changes we
should be reparsing.

Looking at the code, get_file_depends(d) is probably buggy due to the
use of set() which could reorder the data. We need to keep the data in
order there and this issue should then be resolved.

Lets fix the real bug.

Cheers,

Richard