[PATCH 0/1] devtool.py: remove fetch results from download directory to avoid failure

[PATCH 0/1] devtool.py: remove fetch results from download directory to avoid failure

[Chen Qi]

*** BLURB HERE ***
The following changes since commit f9fcd07e478af4a091db0ed71f6442a7e108e016:

  scripts/contrib/build-perf-test.sh: Remove it (2019-01-20 22:24:18 +0000)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib ChenQi/devtool-fetch
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=ChenQi/devtool-fetch

Chen Qi (1):
  devtool.py: remove fetch results from download directory to avoid
    failure

 meta/lib/oeqa/selftest/cases/devtool.py | 3 +++
 1 file changed, 3 insertions(+)

-- 
1.9.1

[PATCH 1/1] devtool.py: remove fetch results from download directory to avoid failure

[Chen Qi]

The fetch results need to be removed from the download directory to avoid
failures like below.

  Submodule path 'doxyport': checked out 'db3e1a6eb8677d3166d90d82c3068689803ce547'

  fatal: reference is not a tree: 67cad692720982ac3cbd99bf1c3421edc69b08f9
  Unable to checkout '67cad692720982ac3cbd99bf1c3421edc69b08f9' in submodule path 'doxygen2jsdoc'

  ERROR: Function failed: base_do_unpack

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/lib/oeqa/selftest/cases/devtool.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index 9eb9bad..4fcbdc3 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -446,6 +446,7 @@ class DevtoolAddTests(DevtoolBase):
         # Test devtool add
         self.track_for_cleanup(self.workspacedir)
         self.add_command_to_tearDown('bitbake -c cleansstate %s' % testrecipe)
+        self.add_command_to_tearDown('rm -rf %s/git2/*%s*' % (get_bb_var('DL_DIR', None), testrecipe))
         self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
         result = runCmd('devtool add %s %s -a -f %s' % (testrecipe, srcdir, url))
         self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created: %s' % result.output)
@@ -464,6 +465,8 @@ class DevtoolAddTests(DevtoolBase):
         checkvars['SRCREV'] = '${AUTOREV}'
         self._test_recipe_contents(recipefile, checkvars, [])
         # Try with revision and version specified
+        result = runCmd('bitbake -c cleansstate %s' % testrecipe)
+        result = runCmd('rm -rf %s/git2/*%s*' % (get_bb_var('DL_DIR', None), testrecipe))
         result = runCmd('devtool reset -n %s' % testrecipe)
         shutil.rmtree(srcdir)
         url_rev = '%s;rev=%s' % (url, checkrev)
-- 
1.9.1

Re: [PATCH 1/1] devtool.py: remove fetch results from download directory to avoid failure

[Richard Purdie]

On Mon, 2019-01-21 at 17:29 +0800, Chen Qi wrote:
> The fetch results need to be removed from the download directory to
> avoid
> failures like below.
> 
>   Submodule path 'doxyport': checked out
> 'db3e1a6eb8677d3166d90d82c3068689803ce547'
> 
>   fatal: reference is not a tree:
> 67cad692720982ac3cbd99bf1c3421edc69b08f9
>   Unable to checkout '67cad692720982ac3cbd99bf1c3421edc69b08f9' in
> submodule path 'doxygen2jsdoc'
> 
>   ERROR: Function failed: base_do_unpack

I'd like to understand the problem here a bit more. My worry is that
deleting files manually from DL_DIR is bad practise, particularly if
that directory is shared between builds like on our autobuilder.

We've had similar problems with cleansstate since that can also remove
artefacts from the sstate cache which another build may be using. We
can do this only when the sstate directory is not shared.

Cheers,

Richard

Re: [PATCH 1/1] devtool.py: remove fetch results from download directory to avoid failure

[ChenQi]

On 01/21/2019 05:53 PM, Richard Purdie wrote:
> On Mon, 2019-01-21 at 17:29 +0800, Chen Qi wrote:
>> The fetch results need to be removed from the download directory to
>> avoid
>> failures like below.
>>
>>    Submodule path 'doxyport': checked out
>> 'db3e1a6eb8677d3166d90d82c3068689803ce547'
>>
>>    fatal: reference is not a tree:
>> 67cad692720982ac3cbd99bf1c3421edc69b08f9
>>    Unable to checkout '67cad692720982ac3cbd99bf1c3421edc69b08f9' in
>> submodule path 'doxygen2jsdoc'
>>
>>    ERROR: Function failed: base_do_unpack
> I'd like to understand the problem here a bit more. My worry is that
> deleting files manually from DL_DIR is bad practise, particularly if
> that directory is shared between builds like on our autobuilder.
>
> We've had similar problems with cleansstate since that can also remove
> artefacts from the sstate cache which another build may be using. We
> can do this only when the sstate directory is not shared.
>
> Cheers,
>
> Richard
>
>
I just pulled to latest master and did the tests again. Things worked 
well. This patch is not needed.
It seems that Mark's commits regarding gitsm in bitbake has fixed the 
problem I met.

Best Regards,
Chen Qi