[PATCH V2 0/1] testsdk.bbclass: set PREMIRRORS for kernel to speed up test

[PATCH V2 0/1] testsdk.bbclass: set PREMIRRORS for kernel to speed up test

[Chen Qi]

Changes in V2:
* Remove the 'protocol=file'.

The following changes since commit b6bc5b2840246ac0f0e5d8cebcdffc1a0ca2518f:

  bitbake: toaster: add 'Sumo' to release selection (2018-04-26 11:43:26 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib ChenQi/testsdkext-speed
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=ChenQi/testsdkext-speed

Chen Qi (1):
  testsdk.bbclass: set PREMIRRORS for kernel to speed up test

 meta/classes/testsdk.bbclass | 1 +
 1 file changed, 1 insertion(+)

-- 
1.9.1

[PATCH V2 1/1] testsdk.bbclass: set PREMIRRORS for kernel to speed up test

[Chen Qi]

Currently if we do a testsdkext task for the image, it would take
very long time to finish. The time is mostly spent fetching kernel
source via network.

We have done some configuration in auto.conf, trying to make use
of own-mirrors.bbclass to avoid fetching kernel via network.

However, the solution normally does not work. Below is some log
from log.do_fetch.

  DEBUG: Fetcher failure: Unable to find file \
  file:///path/to/downloads/git2_git.yoctoproject.org.linux-yocto.git.tar.gz \
  anywhere.

The tar.gz file is not available. It is generated only if
BB_GENERATE_MIRROR_TARBALLS is set to "1". The default value of
BB_GENERATE_MIRROR_TARBALLS is "0", and according to the manual, users
need choose to set it to "1" only if they are trying to make a source
mirror. So generally, this var's value is "0".

Anyway, we do need to avoid fetching kernel source from network when
doing testsdkext. So set PREMIRRORS in auto.conf to achieve this.

After this change, the time reduces from 4209.131s to 1399.436s on
my local machine.

[YOCTO #12729]

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/classes/testsdk.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
index 2e43343..57f8f21 100644
--- a/meta/classes/testsdk.bbclass
+++ b/meta/classes/testsdk.bbclass
@@ -157,6 +157,7 @@ def testsdkext_main(d):
             f.write('SSTATE_MIRRORS += " \\n file://.* file://%s/PATH"\n' % test_data.get('SSTATE_DIR'))
             f.write('SOURCE_MIRROR_URL = "file://%s"\n' % test_data.get('DL_DIR'))
             f.write('INHERIT += "own-mirrors"\n')
+            f.write('PREMIRRORS_prepend = " git://git.yoctoproject.org/.* git://%s/git2/git.yoctoproject.org.BASENAME \\n "\n' % test_data.get('DL_DIR'))
 
         # We need to do this in case we have a minimal SDK
         subprocess.check_output(". %s > /dev/null; devtool sdk-install meta-extsdk-toolchain" % \
-- 
1.9.1

Re: [PATCH V2 1/1] testsdk.bbclass: set PREMIRRORS for kernel to speed up test

[ChenQi]

ping

On 05/04/2018 10:47 AM, Chen Qi wrote:
> Currently if we do a testsdkext task for the image, it would take
> very long time to finish. The time is mostly spent fetching kernel
> source via network.
>
> We have done some configuration in auto.conf, trying to make use
> of own-mirrors.bbclass to avoid fetching kernel via network.
>
> However, the solution normally does not work. Below is some log
> from log.do_fetch.
>
>    DEBUG: Fetcher failure: Unable to find file \
>    file:///path/to/downloads/git2_git.yoctoproject.org.linux-yocto.git.tar.gz \
>    anywhere.
>
> The tar.gz file is not available. It is generated only if
> BB_GENERATE_MIRROR_TARBALLS is set to "1". The default value of
> BB_GENERATE_MIRROR_TARBALLS is "0", and according to the manual, users
> need choose to set it to "1" only if they are trying to make a source
> mirror. So generally, this var's value is "0".
>
> Anyway, we do need to avoid fetching kernel source from network when
> doing testsdkext. So set PREMIRRORS in auto.conf to achieve this.
>
> After this change, the time reduces from 4209.131s to 1399.436s on
> my local machine.
>
> [YOCTO #12729]
>
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> ---
>   meta/classes/testsdk.bbclass | 1 +
>   1 file changed, 1 insertion(+)
>
> diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
> index 2e43343..57f8f21 100644
> --- a/meta/classes/testsdk.bbclass
> +++ b/meta/classes/testsdk.bbclass
> @@ -157,6 +157,7 @@ def testsdkext_main(d):
>               f.write('SSTATE_MIRRORS += " \\n file://.* file://%s/PATH"\n' % test_data.get('SSTATE_DIR'))
>               f.write('SOURCE_MIRROR_URL = "file://%s"\n' % test_data.get('DL_DIR'))
>               f.write('INHERIT += "own-mirrors"\n')
> +            f.write('PREMIRRORS_prepend = " git://git.yoctoproject.org/.* git://%s/git2/git.yoctoproject.org.BASENAME \\n "\n' % test_data.get('DL_DIR'))
>   
>           # We need to do this in case we have a minimal SDK
>           subprocess.check_output(". %s > /dev/null; devtool sdk-install meta-extsdk-toolchain" % \