[PATCH 0/1] Import own-mirrors class from OE

[PATCH 0/1] Import own-mirrors class from OE

[Khem Raj]

Hi Richard



Pull URL: git://git.openembedded.org/openembedded-core-contrib.git
  Branch: kraj/own-mirror
  Browse: http://git.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=kraj/own-mirror

Thanks,
    Khem Raj <raj.khem@gmail.com>
---


Khem Raj (1):
  own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit
    0ef914b250df46a41348479446214575668943fb

 meta/classes/own-mirrors.bbclass |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)
 create mode 100644 meta/classes/own-mirrors.bbclass

-- 
1.7.4.1

[PATCH 1/1] own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit 0ef914b250df46a41348479446214575668943fb

[Khem Raj]

below is the history on this class. Documentation for newly introduced SOURCE_MIRROR_URL
is already present in oe-core

commit 0ef914b250df46a41348479446214575668943fb
Author: Eric BENARD <eric@eukrea.com>
Date:   Wed Nov 3 13:28:54 2010 +0000

    own-mirrors.bbclass: allow mirroring of scm fetched packages

    this way, it's possible to setup a local webserver (for example
    using busybox httpd -p "8081" -h backuped_download_dir) serving
    a presiously fetched download directory and to build wihout the
    need for an internet access

    this can also be used when connected to know to know which packages
    are missing from the local mirror's directory (and thus are fetched
    from internet as a fallback), it's possible to run the server this way :
    busybox httpd -p "8081" -h backuped_download_dir -vv -f | grep -B 1  response:404
    to get the name of the missing packages.

    Signed-off-by: Eric Bénard <eric@eukrea.com>
    Acked-by: Denys Dmytriyenko <denis@denix.org>
    Acked-by: Frans Meulenbroeks <fransmeulenbroeks@gmail.com>
    Signed-off-by: Tom Rini <tom_rini@mentor.com>

commit 1b661974e3f8f844f6ec4cdb7bb42cef9595b626
Author: Marcin Juszkiewicz <hrw@openembedded.org>
Date:   Sat Mar 10 14:10:06 2007 +0000

    own-mirrors.bbclass: added class which can be used to set PREMIRRORS from config
    - SOURCE_MIRROR_URL is new variable which point to source mirror which will be
      used before fetching from original SRC_URI location.

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/classes/own-mirrors.bbclass |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)
 create mode 100644 meta/classes/own-mirrors.bbclass

diff --git a/meta/classes/own-mirrors.bbclass b/meta/classes/own-mirrors.bbclass
new file mode 100644
index 0000000..8a6feaf
--- /dev/null
+++ b/meta/classes/own-mirrors.bbclass
@@ -0,0 +1,12 @@
+PREMIRRORS() {
+cvs://.*/.*     ${SOURCE_MIRROR_URL}
+svn://.*/.*     ${SOURCE_MIRROR_URL}
+git://.*/.*     ${SOURCE_MIRROR_URL}
+hg://.*/.*      ${SOURCE_MIRROR_URL}
+bzr://.*/.*     ${SOURCE_MIRROR_URL}
+svk://.*/.*     ${SOURCE_MIRROR_URL}
+p4://.*/.*      ${SOURCE_MIRROR_URL}
+osc://.*/.*     ${SOURCE_MIRROR_URL}
+https?$://.*/.* ${SOURCE_MIRROR_URL}
+ftp://.*/.*     ${SOURCE_MIRROR_URL}
+}
-- 
1.7.4.1

Re: [PATCH 0/1] Import own-mirrors class from OE

[Gary Thomas]

On 03/12/2011 08:04 AM, Khem Raj wrote:
> Hi Richard
>
>
>
> Pull URL: git://git.openembedded.org/openembedded-core-contrib.git
>    Branch: kraj/own-mirror
>    Browse: http://git.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=kraj/own-mirror
>
> Thanks,
>      Khem Raj<raj.khem@gmail.com>
> ---
>
>
> Khem Raj (1):
>    own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit
>      0ef914b250df46a41348479446214575668943fb
>
>   meta/classes/own-mirrors.bbclass |   12 ++++++++++++
>   1 files changed, 12 insertions(+), 0 deletions(-)
>   create mode 100644 meta/classes/own-mirrors.bbclass
>

Poky/Yocto already has a mechanism for this by overriding PREMIRRORS
I would think sticking with one extension method would be better.

Here's what I have in my [Poky] DISTRO.conf:

SOURCE_DIR ?= "/opt/poky/sources"
PREMIRRORS = "\
http://.*/.*   file://${SOURCE_DIR}/ \n \
https://.*/.*  file://${SOURCE_DIR}/ \n \
ftp://.*/.*    file://${SOURCE_DIR}/ \n \
bzr://.*/.*    file://${SOURCE_DIR}/ \n \
cvs://.*/.*    file://${SOURCE_DIR}/ \n \
git://.*/.*    file://${SOURCE_DIR}/ \n \
hg://.*/.*     file://${SOURCE_DIR}/ \n \
osc://.*/.*    file://${SOURCE_DIR}/ \n \
p4://.*/.*     file://${SOURCE_DIR}/ \n \
svk://.*/.*    file://${SOURCE_DIR}/ \n \
svn://.*/.*    file://${SOURCE_DIR}/ \n"

Users can have their own mirror by defining SOURCE_DIR in local.conf

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: [PATCH 0/1] Import own-mirrors class from OE

[Koen Kooi]

Op 13 mrt 2011, om 16:56 heeft Gary Thomas het volgende geschreven:

> On 03/12/2011 08:04 AM, Khem Raj wrote:
>> Hi Richard
>> 
>> 
>> 
>> Pull URL: git://git.openembedded.org/openembedded-core-contrib.git
>>   Branch: kraj/own-mirror
>>   Browse: http://git.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=kraj/own-mirror
>> 
>> Thanks,
>>     Khem Raj<raj.khem@gmail.com>
>> ---
>> 
>> 
>> Khem Raj (1):
>>   own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit
>>     0ef914b250df46a41348479446214575668943fb
>> 
>>  meta/classes/own-mirrors.bbclass |   12 ++++++++++++
>>  1 files changed, 12 insertions(+), 0 deletions(-)
>>  create mode 100644 meta/classes/own-mirrors.bbclass
>> 
> 
> Poky/Yocto already has a mechanism for this by overriding PREMIRRORS
> I would think sticking with one extension method would be better.
> 
> Here's what I have in my [Poky] DISTRO.conf:
> 
> SOURCE_DIR ?= "/opt/poky/sources"
> PREMIRRORS = "\
> http://.*/.*   file://${SOURCE_DIR}/ \n \
> https://.*/.*  file://${SOURCE_DIR}/ \n \
> ftp://.*/.*    file://${SOURCE_DIR}/ \n \
> bzr://.*/.*    file://${SOURCE_DIR}/ \n \
> cvs://.*/.*    file://${SOURCE_DIR}/ \n \
> git://.*/.*    file://${SOURCE_DIR}/ \n \
> hg://.*/.*     file://${SOURCE_DIR}/ \n \
> osc://.*/.*    file://${SOURCE_DIR}/ \n \
> p4://.*/.*     file://${SOURCE_DIR}/ \n \
> svk://.*/.*    file://${SOURCE_DIR}/ \n \
> svn://.*/.*    file://${SOURCE_DIR}/ \n"
> 
> Users can have their own mirror by defining SOURCE_DIR in local.conf

How is that easier and more maintainable than:

INHERIT += "own-mirrors"
SOURCE_MIRROR_URL = "/some/dir"

?

Especially when more uri methods get added to the fetcher and oe-core.

Re: [PATCH 0/1] Import own-mirrors class from OE

[Gary Thomas]

On 03/13/2011 10:13 AM, Koen Kooi wrote:
>
> Op 13 mrt 2011, om 16:56 heeft Gary Thomas het volgende geschreven:
>
>> On 03/12/2011 08:04 AM, Khem Raj wrote:
>>> Hi Richard
>>>
>>>
>>>
>>> Pull URL: git://git.openembedded.org/openembedded-core-contrib.git
>>>    Branch: kraj/own-mirror
>>>    Browse: http://git.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=kraj/own-mirror
>>>
>>> Thanks,
>>>      Khem Raj<raj.khem@gmail.com>
>>> ---
>>>
>>>
>>> Khem Raj (1):
>>>    own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit
>>>      0ef914b250df46a41348479446214575668943fb
>>>
>>>   meta/classes/own-mirrors.bbclass |   12 ++++++++++++
>>>   1 files changed, 12 insertions(+), 0 deletions(-)
>>>   create mode 100644 meta/classes/own-mirrors.bbclass
>>>
>>
>> Poky/Yocto already has a mechanism for this by overriding PREMIRRORS
>> I would think sticking with one extension method would be better.
>>
>> Here's what I have in my [Poky] DISTRO.conf:
>>
>> SOURCE_DIR ?= "/opt/poky/sources"
>> PREMIRRORS = "\
>> http://.*/.*   file://${SOURCE_DIR}/ \n \
>> https://.*/.*  file://${SOURCE_DIR}/ \n \
>> ftp://.*/.*    file://${SOURCE_DIR}/ \n \
>> bzr://.*/.*    file://${SOURCE_DIR}/ \n \
>> cvs://.*/.*    file://${SOURCE_DIR}/ \n \
>> git://.*/.*    file://${SOURCE_DIR}/ \n \
>> hg://.*/.*     file://${SOURCE_DIR}/ \n \
>> osc://.*/.*    file://${SOURCE_DIR}/ \n \
>> p4://.*/.*     file://${SOURCE_DIR}/ \n \
>> svk://.*/.*    file://${SOURCE_DIR}/ \n \
>> svn://.*/.*    file://${SOURCE_DIR}/ \n"
>>
>> Users can have their own mirror by defining SOURCE_DIR in local.conf
>
> How is that easier and more maintainable than:
>
> INHERIT += "own-mirrors"
> SOURCE_MIRROR_URL = "/some/dir"
>
> ?
>
> Especially when more uri methods get added to the fetcher and oe-core.

Ask Richard?  I was just pointing out that Poky/Yocto already has a mechanism;
perhaps the OE one is better, who knows.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: [PATCH 0/1] Import own-mirrors class from OE

[Chris Larson]

On Sun, Mar 13, 2011 at 10:00 AM, Gary Thomas <gary@mlbassoc.com> wrote:
> Ask Richard?  I was just pointing out that Poky/Yocto already has a
> mechanism;
> perhaps the OE one is better, who knows.

There seems to be some confusion here.  PREMIRRORS isn't a
"poky/yocto" mechanism.  It started in OE, and stills works just fine
there (it's implemented in bitbake).  own-mirrors is simply a
convenience to make premirrors manipulation easier for the user.  If
convenience and usability isn't a priority for poky/yocto, I'd be
curious to hear the reasoning.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: [PATCH 0/1] Import own-mirrors class from OE

[Gary Thomas]

On 03/13/2011 12:50 PM, Chris Larson wrote:
> On Sun, Mar 13, 2011 at 10:00 AM, Gary Thomas<gary@mlbassoc.com>  wrote:
>> Ask Richard?  I was just pointing out that Poky/Yocto already has a
>> mechanism;
>> perhaps the OE one is better, who knows.
>
> There seems to be some confusion here.  PREMIRRORS isn't a
> "poky/yocto" mechanism.  It started in OE, and stills works just fine
> there (it's implemented in bitbake).  own-mirrors is simply a
> convenience to make premirrors manipulation easier for the user.  If
> convenience and usability isn't a priority for poky/yocto, I'd be
> curious to hear the reasoning.

Fair enough; I was merely commenting on how I had been told to handle this
in Poky/Yocto.  If OE has a better way, perhaps it makes sense for it to
be available in both OE-CORE and Poky/Yocto.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: [PATCH 0/1] Import own-mirrors class from OE

[Koen Kooi]

Op 13 mrt 2011, om 23:18 heeft Gary Thomas het volgende geschreven:

> On 03/13/2011 12:50 PM, Chris Larson wrote:
>> On Sun, Mar 13, 2011 at 10:00 AM, Gary Thomas<gary@mlbassoc.com>  wrote:
>>> Ask Richard?  I was just pointing out that Poky/Yocto already has a
>>> mechanism;
>>> perhaps the OE one is better, who knows.
>> 
>> There seems to be some confusion here.  PREMIRRORS isn't a
>> "poky/yocto" mechanism.  It started in OE, and stills works just fine
>> there (it's implemented in bitbake).  own-mirrors is simply a
>> convenience to make premirrors manipulation easier for the user.  If
>> convenience and usability isn't a priority for poky/yocto, I'd be
>> curious to hear the reasoning.
> 
> Fair enough; I was merely commenting on how I had been told to handle this
> in Poky/Yocto.  If OE has a better way, perhaps it makes sense for it to
> be available in both OE-CORE and Poky/Yocto.

And next time please have a look at the actual patch posted before commenting on it. The patch does *exactly* what you are saying, namely overriding PREMIRRORS.

Re: [PATCH 0/1] Import own-mirrors class from OE

[Tom Rini]

On 03/13/2011 08:56 AM, Gary Thomas wrote:
> On 03/12/2011 08:04 AM, Khem Raj wrote:
>> Hi Richard
>>
>>
>>
>> Pull URL: git://git.openembedded.org/openembedded-core-contrib.git
>> Branch: kraj/own-mirror
>> Browse:
>> http://git.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=kraj/own-mirror
>>
>>
>> Thanks,
>> Khem Raj<raj.khem@gmail.com>
>> ---
>>
>>
>> Khem Raj (1):
>> own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit
>> 0ef914b250df46a41348479446214575668943fb
>>
>> meta/classes/own-mirrors.bbclass | 12 ++++++++++++
>> 1 files changed, 12 insertions(+), 0 deletions(-)
>> create mode 100644 meta/classes/own-mirrors.bbclass
>>
>
> Poky/Yocto already has a mechanism for this by overriding PREMIRRORS
> I would think sticking with one extension method would be better.
>
> Here's what I have in my [Poky] DISTRO.conf:
>
> SOURCE_DIR ?= "/opt/poky/sources"
> PREMIRRORS = "\
> http://.*/.* file://${SOURCE_DIR}/ \n \
> https://.*/.* file://${SOURCE_DIR}/ \n \
> ftp://.*/.* file://${SOURCE_DIR}/ \n \
> bzr://.*/.* file://${SOURCE_DIR}/ \n \
> cvs://.*/.* file://${SOURCE_DIR}/ \n \
> git://.*/.* file://${SOURCE_DIR}/ \n \
> hg://.*/.* file://${SOURCE_DIR}/ \n \
> osc://.*/.* file://${SOURCE_DIR}/ \n \
> p4://.*/.* file://${SOURCE_DIR}/ \n \
> svk://.*/.* file://${SOURCE_DIR}/ \n \
> svn://.*/.* file://${SOURCE_DIR}/ \n"
>
> Users can have their own mirror by defining SOURCE_DIR in local.conf

I know this has been beaten to death, but I want to try answering 
differently.  What you mention above isn't so much a solution in poky as 
simply using the normal PREMIRRORS mechanism.  The own-mirrors patch is 
a wrapper around this logic to make it a 2 line local.conf change.  In 
your example, this would become:
INHERIT += "own-mirrors"
SOURCE_MIRROR_URL = "file:///opt/poky/sources"

(Or http://localhost:8080 and do a python -m SimpleHTTPServer 8080 & in 
/opt/poky/sources).

-- 
Tom Rini
Mentor Graphics Corporation

Re: [PATCH 0/1] Import own-mirrors class from OE

[Richard Purdie]

On Sun, 2011-03-13 at 11:50 -0700, Chris Larson wrote:
> On Sun, Mar 13, 2011 at 10:00 AM, Gary Thomas <gary@mlbassoc.com> wrote:
> > Ask Richard?  I was just pointing out that Poky/Yocto already has a
> > mechanism;
> > perhaps the OE one is better, who knows.
> 
> There seems to be some confusion here.  PREMIRRORS isn't a
> "poky/yocto" mechanism.  It started in OE, and stills works just fine
> there (it's implemented in bitbake).  own-mirrors is simply a
> convenience to make premirrors manipulation easier for the user.  If
> convenience and usability isn't a priority for poky/yocto, I'd be
> curious to hear the reasoning.

There were various fixes/improvements to PREMIRRORS as part of the
fetch2 work which is perhaps where the confusion has come from. 

This mirror enhancement is tangential to that and the patch is fine, I
just merged it.

Cheers,

Richard

Re: [PATCH 0/1] Import own-mirrors class from OE

[Richard Purdie]

On Sat, 2011-03-12 at 07:04 -0800, Khem Raj wrote:
> Hi Richard
> 
> 
> 
> Pull URL: git://git.openembedded.org/openembedded-core-contrib.git
>   Branch: kraj/own-mirror
>   Browse: http://git.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=kraj/own-mirror
> 
> Thanks,
>     Khem Raj <raj.khem@gmail.com>
> ---
> 
> 
> Khem Raj (1):
>   own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit
>     0ef914b250df46a41348479446214575668943fb
> 
>  meta/classes/own-mirrors.bbclass |   12 ++++++++++++
>  1 files changed, 12 insertions(+), 0 deletions(-)
>  create mode 100644 meta/classes/own-mirrors.bbclass

Merged into OECore master, thanks.

(and the git tree commits really do help me, thanks)

Cheers,

Richard