[RFC][PATCH] bitbake: fetch2: fix warning while switching from master to krogoth

[RFC][PATCH] bitbake: fetch2: fix warning while switching from master to krogoth

[Maxin B. John]

While switching from master to krogoth build with a common download directory,
got a large number of warnings like the one listed below:

WARNING: freetype-2.6.3-r0 do_fetch: Couldn't load checksums from
donestamp /home/maxin/downloads/freetype-2.6.3.tar.bz2.done: ValueError
(msg: unsupported pickle protocol: 4)

These warnings are caused by the difference in pickle module
implementation in python3(master) and python2(krogoth). Python2 supports
3 different protocols (0, 1, 2) and pickle.HIGHEST_PROTOCOL is 2 where as
Python3 supports 5 different protocols (0, 1, 2, 3, 4) and
pickle.HIGHEST_PROTOCOL is obviously 4.

My suggestion is to use 2 since it is backward compatible with python2
(all the supported distros for krogoth provides python2 which supports
pickle protocol version 2)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
---
 bitbake/lib/bb/fetch2/__init__.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index 3eabba1..9054b2e 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -664,7 +664,7 @@ def verify_donestamp(ud, d, origud=None):
         # as an upgrade path from the previous done stamp file format.
         if checksums != precomputed_checksums:
             with open(ud.donestamp, "wb") as cachefile:
-                p = pickle.Pickler(cachefile, pickle.HIGHEST_PROTOCOL)
+                p = pickle.Pickler(cachefile, 2)
                 p.dump(checksums)
         return True
     except ChecksumError as e:
@@ -698,7 +698,7 @@ def update_stamp(ud, d):
             checksums = verify_checksum(ud, d)
             # Store the checksums for later re-verification against the recipe
             with open(ud.donestamp, "wb") as cachefile:
-                p = pickle.Pickler(cachefile, pickle.HIGHEST_PROTOCOL)
+                p = pickle.Pickler(cachefile, 2)
                 p.dump(checksums)
         except ChecksumError as e:
             # Checksums failed to verify, trigger re-download and remove the
-- 
2.4.0

Re: [RFC][PATCH] bitbake: fetch2: fix warning while switching from master to krogoth

[Richard Purdie]

On Fri, 2016-07-29 at 11:23 +0300, Maxin B. John wrote:
> While switching from master to krogoth build with a common download
> directory,
> got a large number of warnings like the one listed below:
> 
> WARNING: freetype-2.6.3-r0 do_fetch: Couldn't load checksums from
> donestamp /home/maxin/downloads/freetype-2.6.3.tar.bz2.done:
> ValueError
> (msg: unsupported pickle protocol: 4)
> 
> These warnings are caused by the difference in pickle module
> implementation in python3(master) and python2(krogoth). Python2
> supports
> 3 different protocols (0, 1, 2) and pickle.HIGHEST_PROTOCOL is 2
> where as
> Python3 supports 5 different protocols (0, 1, 2, 3, 4) and
> pickle.HIGHEST_PROTOCOL is obviously 4.
> 
> My suggestion is to use 2 since it is backward compatible with
> python2
> (all the supported distros for krogoth provides python2 which
> supports
> pickle protocol version 2)

This looks good to me, well spotted. I've merged it since the
autobuilder is likely spending a lot of time churning sources around at
the moment due to this.

Ideally we'd use a better data format in these files so I'd welcome
other improvements but this fixes the immediate problem.

Cheers,

Richard