Shortened git hashes causing pain

Shortened git hashes causing pain

[Gary Thomas]

[-- Attachment #1: Type: text/plain, Size: 1521 bytes --]

The shortening of the git hashes in this commit
   commit 77fc40a0f843e2488b356d90b64ef436c11c9973
   Author: Richard Purdie <richard.purdie@linuxfoundation.org>
   Date:   Sun May 19 13:21:55 2013 +0300
     bitbake: fetch2: Shorten long srcrevs
is causing some problems.

I have a simple .bbappend for gtk-sato-engine (attached) which
just applies a one-line patch.  When I build this recipe, I
get two different work trees (this is from scratch):
$ ls -l tmp/work/cobra4430p82-amltd-linux-gnueabi/gtk-sato-engine
total 8
drwxrwxr-x  3 gthomas gthomas 4096 May 23 09:12 0.3.3+gitAUTOINC+4740ad8d53aba4368ce3e03b06cfdc69eb86dcdc-r0
drwxrwxr-x 11 gthomas gthomas 4096 May 23 09:13 0.3.3+gitAUTOINC+4740ad8d53-r0

Note: it doesn't seem to happen for the virgin gtk-sato-engine
recipe, only when my patch is applied.

I discovered this when I tried building in an existing tree which
had been built before the mentioned change to the fetch code.  There
were sstate files that used the long hash that the build seemed to
want to use but got terribly confused and ended up creating empty
packages, breaking my build.  Trying to run 'cleansstate' on the
failing packages did nothing as the code only cleans up based on
the new hash and left the old files still there...

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

[-- Attachment #2: gtk-sato-engine_git.bbappend --]
[-- Type: text/plain, Size: 212 bytes --]

#
# Just tell BitBake where to find my component files
#
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

SRC_URI_append = " file://adjust-sato-theme.patch \
"

PACKAGE_ARCH = "${MACHINE_ARCH}"






[-- Attachment #3: adjust-sato-theme.patch --]
[-- Type: text/x-patch, Size: 432 bytes --]

Index: git/data/gtkrc-vga
===================================================================
--- git.orig/data/gtkrc-vga
+++ git/data/gtkrc-vga
@@ -5,7 +5,7 @@
 #   dark gray: cbccc9
 #   light gray: eeeeec
 
-gtk_button_images = 0
+gtk_button_images = 1
 gtk_menu_images = 0
 
 gtk_color_scheme = "bg_color:#eeeeec;fg_color:#555753;text_color:#555753;base_color:#fff;selected_bg_color:#6c3;selected_fg_color:#555753"

Re: Shortened git hashes causing pain

[Richard Purdie]

On Thu, 2013-05-23 at 09:28 -0600, Gary Thomas wrote:
> The shortening of the git hashes in this commit
>    commit 77fc40a0f843e2488b356d90b64ef436c11c9973
>    Author: Richard Purdie <richard.purdie@linuxfoundation.org>
>    Date:   Sun May 19 13:21:55 2013 +0300
>      bitbake: fetch2: Shorten long srcrevs
> is causing some problems.
> 
> I have a simple .bbappend for gtk-sato-engine (attached) which
> just applies a one-line patch.  When I build this recipe, I
> get two different work trees (this is from scratch):
> $ ls -l tmp/work/cobra4430p82-amltd-linux-gnueabi/gtk-sato-engine
> total 8
> drwxrwxr-x  3 gthomas gthomas 4096 May 23 09:12 0.3.3+gitAUTOINC+4740ad8d53aba4368ce3e03b06cfdc69eb86dcdc-r0
> drwxrwxr-x 11 gthomas gthomas 4096 May 23 09:13 0.3.3+gitAUTOINC+4740ad8d53-r0
> 
> Note: it doesn't seem to happen for the virgin gtk-sato-engine
> recipe, only when my patch is applied.

Try moving the cache directory in tmp/ out the way. I suspect some kind
of caching issue since there should not be two work directories. What is
the contents of each?

Cheers,

Richard

Re: Shortened git hashes causing pain

[Gary Thomas]

On 2013-05-23 16:52, Richard Purdie wrote:
> On Thu, 2013-05-23 at 09:28 -0600, Gary Thomas wrote:
>> The shortening of the git hashes in this commit
>>     commit 77fc40a0f843e2488b356d90b64ef436c11c9973
>>     Author: Richard Purdie <richard.purdie@linuxfoundation.org>
>>     Date:   Sun May 19 13:21:55 2013 +0300
>>       bitbake: fetch2: Shorten long srcrevs
>> is causing some problems.
>>
>> I have a simple .bbappend for gtk-sato-engine (attached) which
>> just applies a one-line patch.  When I build this recipe, I
>> get two different work trees (this is from scratch):
>> $ ls -l tmp/work/cobra4430p82-amltd-linux-gnueabi/gtk-sato-engine
>> total 8
>> drwxrwxr-x  3 gthomas gthomas 4096 May 23 09:12 0.3.3+gitAUTOINC+4740ad8d53aba4368ce3e03b06cfdc69eb86dcdc-r0
>> drwxrwxr-x 11 gthomas gthomas 4096 May 23 09:13 0.3.3+gitAUTOINC+4740ad8d53-r0
>>
>> Note: it doesn't seem to happen for the virgin gtk-sato-engine
>> recipe, only when my patch is applied.
>
> Try moving the cache directory in tmp/ out the way. I suspect some kind
> of caching issue since there should not be two work directories. What is
> the contents of each?

Wow, that caused a boatload of other packages to be rebuilt, including
qemu-native, which took forever :-(

This did remove the problem - there is now only the build tree with
the short hash name.  Sadly, it's also now forcing me to basically
rebuild everything to get back to being able to build my image, just
what I didn't want to happen.

Oh well, once this finished I'm sure all will be fine again.  At least
my pain may help others that might experience this, although I don't
know how many folks do very long term development/builds in a single
tree.  This one has been one of my main projects and was started in
early March with more than 500 builds (entries in tmp/log/cooker)!

Thanks

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Shortened git hashes causing pain

[Gary Thomas]

On 2013-05-23 09:28, Gary Thomas wrote:
> The shortening of the git hashes in this commit
>    commit 77fc40a0f843e2488b356d90b64ef436c11c9973
>    Author: Richard Purdie <richard.purdie@linuxfoundation.org>
>    Date:   Sun May 19 13:21:55 2013 +0300
>      bitbake: fetch2: Shorten long srcrevs
> is causing some problems.
>
> I have a simple .bbappend for gtk-sato-engine (attached) which
> just applies a one-line patch.  When I build this recipe, I
> get two different work trees (this is from scratch):
> $ ls -l tmp/work/cobra4430p82-amltd-linux-gnueabi/gtk-sato-engine
> total 8
> drwxrwxr-x  3 gthomas gthomas 4096 May 23 09:12 0.3.3+gitAUTOINC+4740ad8d53aba4368ce3e03b06cfdc69eb86dcdc-r0
> drwxrwxr-x 11 gthomas gthomas 4096 May 23 09:13 0.3.3+gitAUTOINC+4740ad8d53-r0
>
> Note: it doesn't seem to happen for the virgin gtk-sato-engine
> recipe, only when my patch is applied.
>
> I discovered this when I tried building in an existing tree which
> had been built before the mentioned change to the fetch code.  There
> were sstate files that used the long hash that the build seemed to
> want to use but got terribly confused and ended up creating empty
> packages, breaking my build.  Trying to run 'cleansstate' on the
> failing packages did nothing as the code only cleans up based on
> the new hash and left the old files still there...
>

BTW, this caused such a mess in my build tree that I nearly had to
start completely from scratch.  I was able to recover only by very
drastic measures - the following steps were used on more than a dozen
packages that got confused:
   find sstate-cache -name "*${PKG}*" -exec rm -fr \{} \;
   rm -fr tmp/stamps/*/${PKG}
   bitbake ${PKG} -c cleansstate
   rm -fr tmp/work/*/${PKG}
Anything less that this giant sledge-hammer and the confusion only
continued.  At least now I can build my images again...

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------