A simpler way of creating an using a local kernel repository - BeagleBone example

A simpler way of creating an using a local kernel repository - BeagleBone example

[Bob Feretich]

I have had problems getting good download performance when accessing the 
kernels at kernel.org. Since I expect to build the kernel several times, 
I decided to create a copy of the kernel repository locally and use that 
for my builds.

There are instructions on how to create a local repository in the Yocto 
manuals, but those are more complex than I needed. (I don't plan on 
checking anything into the repository.)

The below is a simpler way of creating and using the repository.
I'm publishing this because my search though the Yocto/OE/Angstrom 
yielded only the more complicated or incomplete methods.

// First set up local kernel repository
mkdir ~/ksrc3-8
cd ~/ksrc3-8
git clone --bare 
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git 
stable-work-bare.git

// Then modify the kernel recipe to use the local repository instead of 
the the one at kernel.org.
// The beaglebone recipe for the 3.8 kernel is at...
setup-scripts/sources/meta-beagleboard/common-bsp/recipes-kernel/linux/linux-mainline_3.8.bb

// Replace the file's SRC_URI with one that points to your local repository.
#SRC_URI = 
"git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git;branch=linux-3.8.y"
SRC_URI = "git:///home/Bob/ksrc3-8/stable-work-bare.git;branch=linux-3.8.y"

Regards,
Bob

Re: A simpler way of creating an using a local kernel repository - BeagleBone example

[Bruce Ashfield]

On 2014-05-06, 6:31 PM, Bob Feretich wrote:
> I have had problems getting good download performance when accessing the
> kernels at kernel.org. Since I expect to build the kernel several times,
> I decided to create a copy of the kernel repository locally and use that
> for my builds.
>
> There are instructions on how to create a local repository in the Yocto
> manuals, but those are more complex than I needed. (I don't plan on
> checking anything into the repository.)
>
> The below is a simpler way of creating and using the repository.
> I'm publishing this because my search though the Yocto/OE/Angstrom
> yielded only the more complicated or incomplete methods.
>
> // First set up local kernel repository
> mkdir ~/ksrc3-8
> cd ~/ksrc3-8
> git clone --bare
> git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
> stable-work-bare.git
>
> // Then modify the kernel recipe to use the local repository instead of
> the the one at kernel.org.
> // The beaglebone recipe for the 3.8 kernel is at...
> setup-scripts/sources/meta-beagleboard/common-bsp/recipes-kernel/linux/linux-mainline_3.8.bb
>
>
> // Replace the file's SRC_URI with one that points to your local
> repository.
> #SRC_URI =
> "git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git;branch=linux-3.8.y"
>
> SRC_URI = "git:///home/Bob/ksrc3-8/stable-work-bare.git;branch=linux-3.8.y"

That's exactly how we've been doing it all along in the "meta-kernel-dev"
layer found in poky-extras (now meta-yocto-kernel-extras):

http://git.yoctoproject.org/cgit/cgit.cgi/meta-yocto-kernel-extras/tree/meta-kernel-dev/recipes-kernel/linux

Cheers,

Bruce

>
> Regards,
> Bob

Re: A simpler way of creating an using a local kernel repository - BeagleBone example

[Gary Thomas]

On 2014-05-06 16:31, Bob Feretich wrote:
> I have had problems getting good download performance when accessing the kernels at kernel.org. Since I expect to build the kernel several times, I decided to create a copy of the
> kernel repository locally and use that for my builds.
>
> There are instructions on how to create a local repository in the Yocto manuals, but those are more complex than I needed. (I don't plan on checking anything into the repository.)
>
> The below is a simpler way of creating and using the repository.
> I'm publishing this because my search though the Yocto/OE/Angstrom yielded only the more complicated or incomplete methods.
>
> // First set up local kernel repository
> mkdir ~/ksrc3-8
> cd ~/ksrc3-8
> git clone --bare git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git stable-work-bare.git
>
> // Then modify the kernel recipe to use the local repository instead of the the one at kernel.org.
> // The beaglebone recipe for the 3.8 kernel is at...
> setup-scripts/sources/meta-beagleboard/common-bsp/recipes-kernel/linux/linux-mainline_3.8.bb
>
> // Replace the file's SRC_URI with one that points to your local repository.
> #SRC_URI = "git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git;branch=linux-3.8.y"
> SRC_URI = "git:///home/Bob/ksrc3-8/stable-work-bare.git;branch=linux-3.8.y"

IMO a much simpler (and certainly less invasive) way is to just
use your own local source mirrors.  Add these lines to your local.conf:
   SOURCE_MIRROR_URL = "file:///some-mirror-directory/"
   INHERIT += "own-mirrors "
   BB_GENERATE_MIRROR_TARBALLS ?= "1"

Once the git repository has been downloaded, it will be packed up in a
tarball, e.g.
   git2_git.kernel.org.pub.scm.linux.kernel.git.stable.linux-stable.git.tar.gz
If you then put this file into your SOURCE_MIRROR, the next time bitbake will
fetch from there [first].  It's even smart enough that if you need a revision
which is not in the tarball, it will update the git repository and rebuild the
tarball, which you can then move to your mirror.

Fast access, no messing with source recipes!

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: A simpler way of creating an using a local kernel repository - BeagleBone example

[Bob Feretich]

On 5/6/2014 3:45 PM, Gary Thomas wrote:
> On 2014-05-06 16:31, Bob Feretich wrote:
>> I have had problems getting good download performance when accessing 
>> the kernels at kernel.org. Since I expect to build the kernel several 
>> times, I decided to create a copy of the
>> kernel repository locally and use that for my builds.
>>
>> There are instructions on how to create a local repository in the 
>> Yocto manuals, but those are more complex than I needed. (I don't 
>> plan on checking anything into the repository.)
>>
>> The below is a simpler way of creating and using the repository.
>> I'm publishing this because my search though the Yocto/OE/Angstrom 
>> yielded only the more complicated or incomplete methods.
>>
>> // First set up local kernel repository
>> mkdir ~/ksrc3-8
>> cd ~/ksrc3-8
>> git clone --bare 
>> git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git 
>> stable-work-bare.git
>>
>> // Then modify the kernel recipe to use the local repository instead 
>> of the the one at kernel.org.
>> // The beaglebone recipe for the 3.8 kernel is at...
>> setup-scripts/sources/meta-beagleboard/common-bsp/recipes-kernel/linux/linux-mainline_3.8.bb 
>>
>>
>> // Replace the file's SRC_URI with one that points to your local 
>> repository.
>> #SRC_URI = 
>> "git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git;branch=linux-3.8.y"
>> SRC_URI = 
>> "git:///home/Bob/ksrc3-8/stable-work-bare.git;branch=linux-3.8.y"
>
> IMO a much simpler (and certainly less invasive) way is to just
> use your own local source mirrors.  Add these lines to your local.conf:
>   SOURCE_MIRROR_URL = "file:///some-mirror-directory/"
>   INHERIT += "own-mirrors "
>   BB_GENERATE_MIRROR_TARBALLS ?= "1"
>
> Once the git repository has been downloaded, it will be packed up in a
> tarball, e.g.
> git2_git.kernel.org.pub.scm.linux.kernel.git.stable.linux-stable.git.tar.gz
> If you then put this file into your SOURCE_MIRROR, the next time 
> bitbake will
> fetch from there [first].  It's even smart enough that if you need a 
> revision
> which is not in the tarball, it will update the git repository and 
> rebuild the
> tarball, which you can then move to your mirror.
Better!
Questions:
Is     INHERIT += "own-mirrors "
coded as you state it, or am I supposed to substitute something for 
"own-mirrors " ?

Where does      BB_GENERATE_MIRROR_TARBALLS ?= "1"
put the tarball?

Does do_rm_work remove them?

Regards,
Bob

>
> Fast access, no messing with source recipes!
>

Re: A simpler way of creating an using a local kernel repository - BeagleBone example

[Gary Thomas]

On 2014-05-06 20:48, Bob Feretich wrote:
>
> On 5/6/2014 3:45 PM, Gary Thomas wrote:
>> On 2014-05-06 16:31, Bob Feretich wrote:
>>> I have had problems getting good download performance when accessing the kernels at kernel.org. Since I expect to build the kernel several times, I decided to create a copy of the
>>> kernel repository locally and use that for my builds.
>>>
>>> There are instructions on how to create a local repository in the Yocto manuals, but those are more complex than I needed. (I don't plan on checking anything into the repository.)
>>>
>>> The below is a simpler way of creating and using the repository.
>>> I'm publishing this because my search though the Yocto/OE/Angstrom yielded only the more complicated or incomplete methods.
>>>
>>> // First set up local kernel repository
>>> mkdir ~/ksrc3-8
>>> cd ~/ksrc3-8
>>> git clone --bare git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git stable-work-bare.git
>>>
>>> // Then modify the kernel recipe to use the local repository instead of the the one at kernel.org.
>>> // The beaglebone recipe for the 3.8 kernel is at...
>>> setup-scripts/sources/meta-beagleboard/common-bsp/recipes-kernel/linux/linux-mainline_3.8.bb
>>>
>>> // Replace the file's SRC_URI with one that points to your local repository.
>>> #SRC_URI = "git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git;branch=linux-3.8.y"
>>> SRC_URI = "git:///home/Bob/ksrc3-8/stable-work-bare.git;branch=linux-3.8.y"
>>
>> IMO a much simpler (and certainly less invasive) way is to just
>> use your own local source mirrors.  Add these lines to your local.conf:
>>   SOURCE_MIRROR_URL = "file:///some-mirror-directory/"
>>   INHERIT += "own-mirrors "
>>   BB_GENERATE_MIRROR_TARBALLS ?= "1"
>>
>> Once the git repository has been downloaded, it will be packed up in a
>> tarball, e.g.
>> git2_git.kernel.org.pub.scm.linux.kernel.git.stable.linux-stable.git.tar.gz
>> If you then put this file into your SOURCE_MIRROR, the next time bitbake will
>> fetch from there [first].  It's even smart enough that if you need a revision
>> which is not in the tarball, it will update the git repository and rebuild the
>> tarball, which you can then move to your mirror.
> Better!
> Questions:
> Is     INHERIT += "own-mirrors "
> coded as you state it, or am I supposed to substitute something for "own-mirrors " ?

This line is used as written.

>
> Where does      BB_GENERATE_MIRROR_TARBALLS ?= "1"
> put the tarball?

In your downloads directory.

>
> Does do_rm_work remove them?

No (since that only touches files in tmp/work/...)

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------